HPE 560 Measurement, Assessment and Evaluation

This course provides you with an overview of the theoretical principles of assessment in Health Professions Education (HPE) and best practices in measuring knowledge and skill acquisition in HPE trainees. We will discuss issues around validity and reliability, test construction, and standard-setting. We will explore an array of methods of assessing educational achievement commonly used in HPE, such as written multiple-choice exams, work-based observation, assessments of procedural skills, and simulation-based assessments. This course is aligned with HPE 555 (Instructional Methods) and with HPE 565 (Curriculum Development) and overlaps with HPE 503 (Program Directors Course). Advanced topics in assessment are covered in HPE 805.

 

Course Goals: In order to foster student learning and to make defensible promotion decisions, HPE educators must be able to construct or choose appropriate methods for determining students’ acquisition of knowledge, skills, and attitudes. To this end, HPE educators apply test theory and contemporary approaches to validity, and use proper test construction methods, scale development, scoring, and standard setting. This course is designed to provide learners with: 1. Overview of common assessment methods in HPE; 2. Knowledge of key principles of measurement and assessment, including reliability and validity; 3. Strategies to create or choose defensible assessments, both for formative (providing feedback) and summative (establishing competence) purposes; 4. Approaches to maximizing validity and reliability.

 

Credits

2

Learning Objectives

1. Describe commonly used assessment methods HPE.
2. Apply established quality criteria to choose assessment methods.
3. Establish the validity of an assessment process with Messick’s validity framework.
4. Describe Classical Test Theory estimates of reliability.
5. Identify threats to validity and strategies to mitigate them.
6. Blueprint and construct an NBME-style multiple-choice exam, and use psychometric information for improvement.
7. Set defensible pass/fail standards using the Angoff, Borderline Group, and Contrasting Groups methods.
8. Develop, improve, implement, and evaluate clinical performance and work-based assessments.
9. Implement a program of assessment.