Written Assessment in Medical Education

Preface
Contents
Notes on the Editors
Chapter 1: Basic Concepts
Introduction
Types of Assessments
Formative Assessment
Summative Assessment
Assessments Methods
Written Assessment
Performance-Based Assessment
The Quality of the Assessment
Validity
From “Types of Validity” to Sources of Validity Evidence
What Is a “Construct?
Sources of Validity Evidence
Content
Response Process
Internal Structure
Relationship to Other Variables
Consequences
What Affects Validity (Threats to Validity)?
Reliability
Why Obtaining Reliability Is Important?
What Affects Reliability?
How to Measure Reliability?
Educational Impact, Acceptability, and Cost
Educational Impact
Acceptability
Cost
References
Chapter 2: Assessment of Learning Outcomes
Introduction
Outcome-Based Assessment
Classifications of Intended Learning Outcomes
Levels of the Cognitive Domain (Bloom’s Taxonomy)
Psychomotor and Affective Domains (Tables 2.4 and 2.5)
Miller’s Pyramid
Types of Written Assessment
Restricted-Response Items
Commonly Used Multiple-Choice Questions
Newer Formats of Multiple-Choice Questions
Constructed-Response Items
References
Chapter 3: Blueprint in Assessment
What Is Blueprint? Blueprint in the Context of Health Professional Education
Theoretical Underpinning of Blueprint
Benefits of Blueprinting
Designing the Blueprint
Developing the Blueprint
Phase One: Topics’ Weight Estimation
Phase Two: Selection of the Appropriate Assessment Tool
Examples for Blueprint Templates
Research Methodology Course (Table 3.7)
Conclusion
References
Chapter 4: Constructed Response Items
Overview of Constructed Response Items and Rationale of Their Uses
Types of Constructed Response Items
Long Essay Question
Modified Essay Question (MEQ)
Short Answer Question (SAQ)
Constructing SAQs and MEQs
Using Rubric in CRI
Key Feature Approach Questions
References
Further Reading
Chapter 5: Key Feature Items
The Concept of Key Feature Items
The Structure of Key Feature Items
Psychometric Properties of Key Feature Items
Evidence Based on Content
Evidence-Based Response Process
Evidence Based on Internal Structure
Evidence Based on Relations to Other Variables
Evidence Based on Consequences of Testing
Conclusion
References
Chapter 6: A-Type MCQs
Background
Contextual Items
Noncontextual Items
Advantages and Limitations
How to Construct A-Type MCQs?
Stem
Lead-in
Options
Distractors
What Is the Optimal Number of Options in an Item: Five, Four, or Three?
What Is the Justification for Reducing the Number of Options?
Is Reducing the Number of Options Is the Only Way to Minimize NFDs?
Decreasing the Options from Five to Four Has Undesirable Effects
Evidence Evaluation: Are Studies that Prefer Four Options Credible?
Can we Revisit the Definition of NFDs?
Item Flaws (Table 6.1)
Examples of Item Flaws: General Flaws
Examples of Item Flaws: Flaws Contribute to Irrelevant Difficulty (Table 6.3)
Constructing High Cognitive Level A-Type MCQs
References
Chapter 7: R-Type MCQs (Extended Matching Questions)
Introduction
Definition and Structure
Educational Impact
Psychometric Properties
References
Chapter 8: Script Concordance Test
An Assessment of Clinical Reasoning Process
Construction Principles
Panel of Experts
Scoring
Assessment Utility
References
Chapter 9: Introduction to the Psychometric Analysis
Introduction
Theories Behind Testing
Classical Test Theory (CTT—1968)
Generalizability Theory (G Theory—1972)
Item Response Theory (IRT)
Qualitative Evaluation of Exams
Quantitative Analysis of Exams
Analysis of Examinees’ Scores
Graphical Presentation of Examinees’ Scores (Histogram)
Mean and Standard Deviation (SD)
Median and Interquartile Range (IQR)
Post-Exam Psychometric Analysis
Exam Reliability
Test-Retest Reliability
Internal Consistency Reliability
Inter-Rater Reliability
Assessing and Increasing Exam Reliability
Item Analysis Statistics
Item Difficulty (ID)
Item Discrimination Index
Point-Biserial Correlation
Distractor Analysis
Standard Error of Measurement (SEM)
Conclusion
References
Chapter 10: Standard Setting in Written Assessment
Overview of Standard Setting
Approaches for Assigning a Pass/Fail Status in an Evaluative Setting
Criterion-Referenced Approach
Norm-Referenced Approach
Common Concepts in Standard Setting
General Classification of Standard Setting Methods
Common Methods of Standard Setting in Written Assessment
The Angoff (1971) Method
Nedelsky (1954) Method
The Ebel (1972) Method
References
Further Reading
Chapter 11: Progress Testing in Written Assessment
Overview of Progress Testing
The Concept of PT
The Rationale for PT
Development of PT
Formulating a Central Committee for PT
Designing PT Blueprint
Test Item Format
Scoring of PT
PT Feedback for Students and Participating Schools
References
Further Reading
Chapter 12: How Written Assessment Fits into the Canvas of Programmatic Assessment
Traditional Assessment and Its Contemporary Challenges
What Is Programmatic Assessment?
Why Programmatic Assessment?
Theoretical Underpinning of Programmatic Assessment
Practical Tips to Design and Implement Programmatic Assessment
Determine the Desired Competencies
Design an Assessment Plan
Determine Time Points and Frequencies of Assessment
Develop a Culture of Constructive Feedback
Promote Mentorship Program
Evaluate and Adjust the Master Plan
Plan and Conduct Faculty Development
References
Reading Material
We recommend the readers to consult the following resources to advance their knowledge on programmatic assessment.
Chapter 13: Assessment: Social Accountability and the Society
References
Index