|
Interrater Reliability
In statistics, inter-rater reliability (also called by various similar names, such as inter-rater agreement, inter-rater concordance, inter-observer reliability, inter-coder reliability, and so on) is the degree of agreement among independent observers who rate, code, or assess the same phenomenon. Assessment tools that rely on ratings must exhibit good inter-rater reliability, otherwise they are not valid tests. There are a number of statistics that can be used to determine inter-rater reliability. Different statistics are appropriate for different types of measurement. Some options are joint-probability of agreement, such as Cohen's kappa, Scott's pi and Fleiss' kappa; or inter-rater correlation, concordance correlation coefficient, intra-class correlation, and Krippendorff's alpha. Concept There are several operational definitions of "inter-rater reliability," reflecting different viewpoints about what is a reliable agreement between raters. There are three operational defin ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Test Validity
Test validity is the extent to which a test (such as a chemical, physical, or scholastic test) accurately measures what it is supposed to measure. In the fields of psychological testing and educational testing, "validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests".American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999) ''Standards for educational and psychological testing''. Washington, DC: American Educational Research Association. Although classical models divided the concept into various "validities" (such as content validity, criterion validity, and construct validity),Guion, R. M. (1980). On trinitarian doctrines of validity. ''Professional Psychology, 11'', 385-398. the currently dominant view is that validity is a single unitary construct.Messick, S. (1995). Validity of psychological assessment: Validation of inf ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
