This book provides a significant contribution to conversations about teacher quality and graduate readiness for teaching. It presents empirical insights into how a multidisciplinary team of researchers, teacher educators, and policy personnel mobilized for collective change in a standards-driven reform initiative. The insights are research-informed and critically relevant for anyone interested in teacher preparation and credentialing. It gives an account of a bold move to install a collaborative culture of evidence-informed inquiry to professionalize teacher education.
The centerpiece of the book is the use of standards and evidence to show the quality of graduates entering the teaching workforce. The book presents, for the first time, a model of online cross-institutional moderation as benchmarking to generate large-scale evidence of the quality of teacher education. The book also introduces a new conceptualization of a feedback loop using summative data for accountability and formative data to inform curriculum review and program renewal.
This book offers the insider story of the conceptualization, design, and implementation of the Graduate Teacher Performance Assessment (GTPA). It involves going to scale with a large group of Australian universities, government agencies, and schools, and using participatory approaches to advance new thinking about evidence-informed inquiry, cross-institutional moderation, and innovative digital infrastructure.
The discussion of competence assessment, standards, and change processes presented in the book has relevance beyond teacher education to other professions.
Author(s): Claire Wyatt-Smith, Lenore Adie, Michele Haynes, Chantelle Day
Publisher: Routledge
Year: 2022
Language: English
Pages: 272
City: London
Cover
Endorsement Page
Half Title
Title Page
Copyright Page
Table of Contents
List of figures
List of tables
Foreword
Acknowledgments
List of contributors
List of abbreviations
Part I: Conceptualization, design, and implementation
Chapter 1: Standards, large-scale evidence, professional judgment, and the affordances of digital technologies in teacher education
Teacher education and issues of evidence and quality
A turntable of reviews into teacher education
The nature and function of standards in teacher education
The move to teaching performance assessments
The turn to standards, evidence, and quality through competence assessment
Overview of chapters
Conclusion
Notes
References
Chapter 2: The move to assessing competence in professions
Introduction
Measuring complex assessments of professional competence
Performance assessments
Examples of performance assessments for determining professional competence and licensure
Performance assessments in initial teacher education: Examples from the United States
Performance assessments in Australian teacher education: The relationship between competence and readiness
Conclusion
Notes
References
Chapter 3: Three essential features in a new conceptualization of complex performance assessments: Authenticity, system and site validity, and intelligent accountability
Introduction
Authenticity as a design feature of complex performance assessment
System and site validity as a design feature of complex performance assessment
Intelligent accountability as a design feature of complex performance assessment
Designing the Graduate Teacher Performance Assessment (GTPA) for authenticity, system and site validity, and intelligent accountability
Design principles of the GTPA
Conclusion
Notes
References
Chapter 4: How do we know that teacher graduates are ready for professional practice?: Designing an assessment for evidence of readiness
The move to teaching performance assessments
The Australian context for assessing professional competence and readiness for teaching
A strengthened role for standards, evidence, and benchmarking
Designing the Graduate Teacher Performance Assessment (GTPA)
The first cycle of the literature review: Conceptualization and content validity
The second cycle of the literature review: Constructs of teaching
Standards and the design of the GTPA scoring rubric
Expert review
Conclusion
Note
References
Chapter 5: Teaching performance assessments and considerations of fairness
Introduction
Fairness principles
Fairness by ensuring the technical quality of the assessment
Validity standards
Reliability standards
Standard setting and scoring
Treatment of special populations
Opportunity to learn
Other fairness considerations
Assessing fairness of the GTPA
Applying the fairness principles to the GTPA
Effectiveness principles
Assessing effectiveness of the GTPA
Conclusion
Notes
References
Part 2: Data analytics, systems thinking, and digital architecture
Chapter 6: Validation, reliability, and standard setting in the GTPA: A focus on methodologies and judgment
Introduction
Introducing Study 1 and Study 2: A focus on methodologies
Processes and methods
Study 1: Consistency in judgment and criteria discrimination
Study 1.1: Establishing the judgment process
Study 1.2: Moderation to support judgment consistency
Analytic strategy
Results from Study 1
Study 2: Setting the standard with pairwise comparison methodology
Study 2.1: Pairwise comparison of GTPA samples
Study 2.2: Setting the standard with exemplars of the minimum acceptable level
Analytic strategy
Study 2: Results
The cumulative value of two methodologies
Summary
Notes
References
Chapter 7: The design features of cross-institutional moderation for demonstrating comparability: Building a nationally sustainable model
Introduction
Step 1: Learning from research, practice, and literature
Step 2: Making decisions about fitness-for-purpose and the approach to moderation
Benchmarking for external verification of the application of the standard across sites
Scoring
Analysis of data from scoring performances
Step 3: Designing to build dependability: Fidelity of implementation, material artifacts, scorer training, and calibration
Fidelity of implementation
Decision aids
Calibration
Conclusion
Notes
References
Chapter 8: Why teacher education needs a feedback loop: Connecting standards and evidence to inform program planning and renewal
Introduction
Standards as restricting and enabling
Evaluative expertise and data literacy: Scoring teaching performance assessments
The customized GTPA Report
GTPA Report: Part One – Application of the standard
GTPA Report: Part Two – Analysis of program cohort data submitted by the university at the criterion level
The value of feedback loops that are informed by data from multiple institutions
Conclusion
Notes
References
Chapter 9: Culture change in initial teacher education: How shall we know quality and impact?
Introduction
Collaborative professionalism and professional identity
Analyzing teacher educator talk and interactions
The first response: Conserving practice
The second response: Shifting mindsets to discern coherence and make connections
Aligning the practical and theoretical components of teacher education
Developing a professional identity
Aligning the scope and sequence of teacher education
The third response: Embedding the new assessment to enhance practice
The fourth response: Bringing judgment and moderation into sharp focus
The fifth response: Realizing the potential of TPAs
Building confidence through professional learning
Informing accreditation processes
Informing curriculum review and program renewal
Conclusion
Notes
References
Chapter 10: Teaching performance assessments and considerations of potential legal challenges
Introduction
Potential legal issues: The United States perspective
Fairness and potential legal issues in the United States
Fair treatment in academic decisions
Discrimination law in the United States
Human rights
Fairness of assessment
Who may be liable?
Validity
Reliability
Opportunity to learn
Consumer/commercial law or negligence law violations
Contract law
Negligence law claims
Privacy
Consideration of potential TPA legal challenges: An Australian perspective
Fair treatment of student appeals against academic decisions
Discrimination law in Australia
Fairness of the TPA
Who may be liable?
Validity
Reliability
Opportunity to learn
Consumer/contract law and negligence
Consumer/contract law
Negligence law claims
Privacy
An Australian caveat
Conclusion and recommendations
Notes
References
Chapter 11: Our journey of discovery: Looking back, looking sideways, and looking forward
Introduction
A new conceptualization of teaching performance assessments to realize their potential
Lesson 1: Conceptualization and design of the instrument (Layer 1)
Lesson 2: Validation and standard setting (Layer 2)
Lesson 3: Cross-institutional moderation online (Layer 3)
Going to scale calls for investment in digital infrastructure
Lesson 4: Benchmarking and analysis of data (Layer 4)
Lesson 5: Longitudinal workforce studies (Layer 5)
Closing commentary: Looking back, looking sideways, looking forward
Looking back
Looking sideways
Looking forward
Notes
References
Glossary
References
Index
