Development of Professional Expertise: Toward Measurement of Expert Performance and Design of Optimal Learning Environments

Cover......Page 1
Half-title......Page 3
Title......Page 5
Copyright......Page 6
Contents......Page 7
Figures......Page 10
Tables......Page 13
Contributors......Page 15
1 The Measurement and Development of Professional Performance: An Introduction to the Topic and a Background to the Design and Origin of This Book......Page 19
A Brief History of Research on Objective Measurement of Professional Performance......Page 22
The Expert Performance Approach to the Study of Superior Performance......Page 25
The Process of Generating This Book......Page 27
The Outline of This Book......Page 28
References......Page 38
Section 1 CHALLENGES IN PAST AND CONTEMPORARY EFFORTS TO MEASURE AND TRAIN THE OBJECTIVE PERFORMANCE OF PROFESSIONALS......Page 43
2 The 20th-Century Revolution in Military Training......Page 45
Training Counts......Page 46
A Revolution in Training......Page 47
A Story to Set Context......Page 48
Connecting Training to Military Proficiency......Page 50
Counting Systems......Page 51
The Learning Curve and Skills Training......Page 53
Skill Atrophy: The Unlearning Curve......Page 57
Warfare and Mission-Level Profi ciency......Page 59
You Fight Like You Train: A Personal Example......Page 60
A Hierarchy of Learning Curves......Page 61
The Measurement Problem......Page 64
Measurable War: Air-to-Air Combat and the Existence of Aces......Page 65
Engagement Simulation and Training......Page 67
Summary of Engagement Simulation......Page 70
Training Surprise: Other Countries Could Use It, Too......Page 71
The Interdeployment Readiness Bathtub Curve......Page 73
Training Counts......Page 74
Resolving the Contradiction: A Hierarchy of Training......Page 75
Admonition to the Defense Department......Page 76
References......Page 77
3 Developing Professional Expertise with a Cognitive Apprenticeship Model: Examples from Avionics and Medicine......Page 79
Assessing Professional Competence......Page 80
Cognitive Apprenticeship as an Instructional Model......Page 82
Avionics Troubleshooting......Page 83
Background......Page 84
The Solution......Page 85
Evidence of Effectiveness......Page 89
Reasons for Success......Page 90
Extending Sherlock’s Success to New Domains: From Avionics Troubleshooting to Medical Problem Solving......Page 92
Using Cognitive Models in the Design of Computer-Based Learning Environments......Page 93
Lessons Learned and Future Directions......Page 96
References......Page 98
4 Leadership Development and Assessment: Describing and Rethinking the State of the Art......Page 102
Leader Behavior......Page 103
Measurement......Page 104
Knowledge Structures......Page 107
Processes......Page 108
Support for the Model......Page 111
Leader Assessment......Page 112
Processes......Page 113
Leader Development......Page 114
Knowledge......Page 115
Processes......Page 117
Conclusions......Page 118
References......Page 120
5 Revolutions, Leaders, and Diagnosticians : Reflections on the Themes in Chapters 2–4......Page 126
Chatham’s “First Revolution” in Military Training: (Hopefully) Some Constructive Comments......Page 127
Sherlock and Its Heirs......Page 136
Is Leadership a Natural Kind?......Page 138
Summary......Page 144
References......Page 145
Section 2 PAST AND CONTEMPORARY EFFORTS TO DESIGN INSTRUCTION, TRAIN, AND MAINTAIN PROFESSIONAL PERFORMANCE......Page 147
The Changing Goals of Professional Training......Page 149
Instructional Systems Design......Page 151
Social-Constructivist Design......Page 152
Whole-Task Design Models......Page 154
Learning Technologies......Page 155
The Internet......Page 156
Learning Objects and Standards......Page 157
State of the Art and Outlook......Page 158
Computer-Based Training......Page 159
Intelligent Tutoring Systems......Page 160
Dynamic Visual Representations and Animations......Page 162
Hypertext and Hypermedia......Page 163
Computer Simulations and Virtual Reality......Page 164
State of the Art and Outlook......Page 165
The Emerging Need for a Paradigm Shift......Page 167
References......Page 170
7 Designing Training for Professionals Based on Subject Matter Experts and Cognitive Task Analysis......Page 175
The Advantages of Task Analysis......Page 176
The Disadvantages of Task Analysis......Page 177
Task Analysis for Instructional Systems Development: A Consensual View......Page 179
Cognitive Task Analysis and the Use of Subject-Matter Experts......Page 180
Training Structured Troubleshooting: A Case Study......Page 183
Structured Troubleshooting as a One-Week Add-On......Page 186
Modifying Existing Courses Based on Structured-Troubleshooting Principles......Page 191
Conclusions......Page 192
References......Page 194
8 How to Help Professionals Maintain and Improve Their Knowledge and Skills: Triangulating Best Practices in Medicine......Page 198
Causes of the Gap Between Current and Best Practices......Page 200
Self-Assessment......Page 202
Competency Assessment......Page 203
Synthesis: Toward a Broader Model of Best Practice......Page 204
The Use of CME Credits by State Medical Boards, Specialty Boards, and Others......Page 205
Premise of CME Credit......Page 206
The Effect of CME Credits: Indirect Effects, Indirect Benefits......Page 207
Pressures for Change......Page 209
Toward a New Best Practice Model for the Professions......Page 210
Some Thoughts About the Future......Page 211
New Technologies......Page 212
A Picture of the Future......Page 213
References......Page 215
Applying the Science of Instruction......Page 221
Phase 1: Specifying Compartmentalized Behaviors......Page 223
Phase 3: Specifying Integrated Knowledge......Page 224
Where Do We Go From Here?......Page 226
References......Page 228
Section 3 THE ASSESSMENT AND TRAINING OF SKILLED AND EXPERT PERFORMERS IN THE MILITARY......Page 231
The First Training Revolution Does Not Scale......Page 233
Recent Changes at the National Training Center......Page 234
Observations on Individual Training......Page 235
The Proximity Effect......Page 236
Simulation to the Rescue?......Page 237
Common Assumptions About Games for Training......Page 238
Tactical Language and Culture Trainer......Page 239
Game from the Start: DARWARS Ambush!......Page 243
Distribution by Diffusion......Page 247
Why DARWARS Ambush! Was a Success......Page 248
DARWARS Ambush! Is an Environment, Not a Trainer......Page 250
Th ere Is No Golden Disk......Page 251
Information Technology Troubleshooting Digital Tutor: Immersive Experiential Training Without a Game......Page 252
Gorman’ s Gambit......Page 254
Does Training Transfer From Lightweight PC Simulations Into Real-World Competence?......Page 255
What Is Missing?......Page 259
Where We Might Go from Here: People Engines in a Simulation of a Mindscape......Page 260
Final Words and the Promised Whimper......Page 262
References......Page 263
A Brief Historical Overview of Flight Simulation......Page 265
Distributed Simulation Development: Challenges for Assessment......Page 268
The DMO Community’s Assumptions......Page 271
Prior Human Performance DMO Data......Page 272
Limitations on Relying on Subjective Assessment in Complex Environments......Page 273
Metric Generation and Validation......Page 275
Conceptual Application/Transition of Measures for Adaptive Training......Page 281
Performance Measurement for Pilot Selection......Page 282
Performance Measurement for Cockpit Design and Testing......Page 283
Conclusion......Page 284
References......Page 285
12 Contrasting Submarine Specialty Training: Sonar and Fire Control......Page 289
The Submarine World......Page 290
Sonar Technicians......Page 291
Fire Control Technicians......Page 292
The Problem with Military Training......Page 296
Training and Performance Measurement: Old Model......Page 297
Training and Performance Measurement: Training Continuum Model......Page 299
Conclusions......Page 302
References......Page 303
13 Training Complex Cognitive Skills: A Theme-Based Approach to the Development of Battlefield Skills......Page 304
Adaptive Performance and Adaptive Thinking......Page 305
Training Adaptive Performance and Mental Models......Page 306
Training Thinking Behaviors......Page 308
The Expert Themes of Battlefield Thinking......Page 309
Deliberate Practice......Page 312
TLAC Training......Page 314
Evaluation of the TLAC Training......Page 318
Applying the Theme-Based Training Method to Crisis Response Training for Military and Civilian Organizations......Page 322
Conclusion......Page 324
References......Page 325
Basic Considerations in Structuring Training to Achieve Elite Performance......Page 330
Learning Without Performance......Page 331
The Potential for Instructors and Learners to Be Misled......Page 332
The Potential to Misunderstand the Meaning and Role of Errors......Page 333
After-Action Reviews......Page 334
The Prevalence of Forgetting and Its Role in Relearning......Page 335
Forgetting as a Necessary Condition for Optimal Relearning......Page 336
Quantifying Warfighter Performance in Distributed Simulation Environments......Page 337
Introducing Desirable Difficulties......Page 338
Contrasting Submarine Specialty Training: Sonar and Fire Control......Page 339
Learning Versus Performance......Page 340
Training Complex Cognitive Skills: A Th eme-Based Approach......Page 341
The Interaction of Experience and Training......Page 342
Deliberate Practice and Learning Versus Performance Revisited......Page 343
Games as Tools for Training......Page 344
Conclusion......Page 345
References......Page 346
Section 4 THE DEVELOPMENT OF EXPERTISE AND EXPERT PERFORMANCE......Page 349
15 The Influence of Learning Research on the Design and Use of Assessment......Page 351
Review of Assessment Parameters......Page 352
The Conflict Between Measuring Stable Constructs and Learning......Page 356
Learning from Procedures......Page 357
Assessment Purposes (or Clarifying Functions of Results)......Page 358
A Status Check on Assessment Design......Page 360
Cognitive Demands and Domain Description......Page 361
Content Representation......Page 363
Examples of Assessment Design Methodology......Page 365
Developed Ontologies in Other Domains......Page 366
Criteria Needed for Excellent Performance: More Validity Built In......Page 368
Validation and Measures of Technical Quality......Page 369
Conclusion......Page 370
References......Page 371
16 Acquiring Conceptual Expertise from Modeling: The Case of Elementary Physics......Page 374
What Knowledge Comprises Conceptual Expertise in Physics?......Page 378
What Happened to the Novices’ Misconceptions?......Page 383
How Does Expertise Affect Quantitative Problem Solving?......Page 384
Learning Description Phase Knowledge......Page 385
Learning Applicability Conditions......Page 386
Learning Confluences......Page 387
Conclusion......Page 391
References......Page 393
17 Teaching for Expertise: Problem-Based Methods in Medicine and Other Professional Domains......Page 397
Traditional Professional Academic Education......Page 399
Curriculum Overload......Page 400
Task and Role of the Professional......Page 401
Implications......Page 402
Problems......Page 403
Courses and Curricula......Page 407
Different Goals and Different Implementations of Problem-Based Learning......Page 408
Medical Expertise Development Research......Page 410
Do Students Learn Better?......Page 413
Have Students Learned Better?......Page 416
Do Problem-Based Learning Graduates Do a Better Job?......Page 417
Conclusion......Page 418
References......Page 419
18 Enhancing the Development of Professional Performance: Implications from the Study of Deliberate Practice......Page 423
The Traditional View of Skill Acquisition and Professional Development: History and Some Recent Criticisms......Page 424
Capturing Reproducibly Superior Performance Under Standardized Conditions......Page 426
The Necessity of Domain-Specific Experience for Attaining Reproducibly Superior Performance......Page 429
From Experience to Designed Practice......Page 430
Applying the Framework to Professional Training and Practice......Page 439
Conclusion......Page 443
References......Page 445
19 It Takes Expertise to Make Expertise: Some Thoughts About Why and How and Reflections on the Themes in Chapters 15–18......Page 450
Learning Expertise......Page 451
Teaching Expertise......Page 452
Feedback for Bi-directional Learning......Page 454
Problems with Low-Quality Feedback Cycles......Page 455
Research with Language Therapists......Page 456
Work in Education......Page 457
Knowledge of Performance Conditions......Page 459
Stable and Variable Performance Conditions......Page 460
Conclusion......Page 463
References......Page 464
Self-Assessment and Self-Directed Learning......Page 467
Assessment of Professional Growth Toward Expertise......Page 468
Design of Learning Environments to Promote Expertise......Page 469
Military Value......Page 470
Human Ability and Military Task Performance......Page 471
Expertise and Military Task Performance......Page 473
Success in Military Operations......Page 477
Top Gun......Page 478
73 Easting......Page 480
Deliberate Practice......Page 481
Assessment and Self-Assessment......Page 482
Design of Learning Environments......Page 483
Conclusion......Page 484
References......Page 485
21 Expertise in the Management of People: A New Frontier for Research on Expert Performance......Page 488
The Tradition of Management Training......Page 489
Case-Based Instruction for Management and Leadership......Page 490
How Effective Is Case-Based Instruction?......Page 491
Expertise in Case-Based Discussion Teaching......Page 493
Expertise in College and Graduate Level Teaching, Generally......Page 495
Teachers’ Mental Models of Students......Page 496
Characterizing Skill in Discussion Teaching......Page 498
The Role of Mental Models in the Management of People......Page 500
The Myers-Briggs Cult: A System for Mental Modeling of Employees......Page 502
How Adequate Are Myers-Briggs Models of People?......Page 504
Studying Management and Leadership Behavior......Page 505
Reprise......Page 507
References......Page 509
Name Index......Page 513
Subject Index......Page 530