Reliability: on the reproducibility of assessment data

CONTEXT: All assessment data, like other scientific experimental data, must be reproducible in order to be meaningfully interpreted. PURPOSE: The purpose of this paper is to discuss applications of reliability to the most common assessment methods in medical education. Typical methods of estimating reliability are discussed intuitively and non-mathematically. SUMMARY: Reliability refers to the consistency of assessment outcomes. The exact type of consistency of greatest interest depends on the type of assessment, its purpose and the consequential use of the data. Written tests of cognitive achievement look to internal test consistency, using estimation methods derived from the test-retest design. Rater-based…

• UO
  University of Illinois at Chicago