The Kappa Statistic in Reliability Studies: Use, Interpretation, and Sample Size Requirements

PURPOSE: This article examines and illustrates the use and interpretation of the kappa statistic in musculoskeletal research. SUMMARY OF KEY POINTS: The reliability of clinicians' ratings is an important consideration in areas such as diagnosis and the interpretation of examination findings. Often, these ratings lie on a nominal or an ordinal scale. For such data, the kappa coefficient is an appropriate measure of reliability. Kappa is defined, in both weighted and unweighted forms, and its use is illustrated with examples from musculoskeletal research. Factors that can influence the magnitude of kappa (prevalence, bias, and non-independent ratings) are discussed, and ways of evaluating the magnitude of an…