This edited book is a collection of papers, written by language assessment professionals to reflect the guidance of Professor Lyle F. Bachman, one of the leading second language assessment experts in the field for decades. It has three sub-themes: assessment of evolving language ability constructs, validity and validation of language assessments, and understanding internal structures of language assessments. It provides theoretical guidelines for practical language assessment challenges. Chapters are written by language assessment researchers who graduated from the University of California, Los Angeles, where Professor Bachman trained them including the book editors.
Author(s): Gary J. Ockey, Brent A. Green
Edition: 1
Publisher: Springer Singapore
Year: 2020
Language: English
Pages: 256
Forward: “It’s Been a Great Ride!”
Foreword: Some Reflections on Mentoring Ph.D. Students
Acknowledgements
Contents
1 Introduction
References
Part IAssessment of Evolving Language Ability Constructs
2 A Step Toward the Assessment of English as a Lingua Franca
Introduction
Constructs and Historical Perspective
English as a Lingua Franca
What Is English as a Lingua Franca?
Criticisms of Tests and of ELF
The English Placement Test of Oral Communication (EPT OC)
Critical Issues
Conclusions, Implications, and Future Directions
Appendix: Iowa State University Oral English Placement Test Rubric
References
3 Revisiting the Role of Content in Language Assessment Constructs
Introduction
Historical Perspective: The Role of Content in Bachman’s Model of Communicative Language Use
Critical Issues: Grappling with the Role of Content in Language Assessment Constructs
Language for Specific Purposes Assessment
Assessment of English Learners in U.S. K-12 Education
Content and Language Integrated Learning
Conclusions, Implications, and Future Directions
References
4 What Does Language Testing Have to Offer to Multimodal Listening?
Introduction
Constructs
Historical Perspectives
Critical Issues
Recognizing the Role of Semiotic Resources in Assessment
The Construction of Classroom Assessments
Conclusions, Implications, and Future Directions
References
5 Learner Perceptions of Construct-Centered Feedback on Oral Proficiency
Introduction
Literature Review
Oral Proficiency Assessment and Construct-Centered Feedback
Construct
Construct-Centered Feedback
Method
Participants
Procedure
Analysis
Results
Discussion
Conclusions, Implications, and Future Directions
Appendix A
A Sample Feedback Form
References
Part IIValidity and Validation of Language Assessments
6 A Case for an Ethics-Based Approach to Evaluate Language Assessments
Introduction
Constructs and Historical Perspective
The Standards-Based Approach
The Argument-Based Approach
Alternative Perspective
An Ethics-Based Approach
Conclusions, Implications and Future Directions
References
7 Alignment as a Fundamental Validity Issue in Standards-Based K-12 English Language Proficiency Assessments
Introduction
Constructs
Historical Perspectives
Critical Issues
Expanding an Alignment Framework for ELP Assessments
Identifying Language Demands for Alignment Evaluation
Examining the Relationship Between ELP and Content-Area Assessments
Facilitating Professional Support to Enhance Alignment
Conclusions, Implications, and Future Directions
References
8 Validating a Holistic Rubric for Scoring Short Answer Reading Questions
Introduction
Context of the Study
Literature Review
Constructs
Reading Assessment Tasks
Methods
Participants
Ratings
Data Analysis
Results
Descriptive Statistics
Correlations: Holistic and Analytic Scores
Regression Analysis: Predicting Holistic Scores
Discussion
Conclusions, Implications, and Future Directions
References
9 The Curse of Explanation: Model Selection in Language Testing Research
Introduction
Literature Review
Challenges in Model Selection
Bachman’s Work on Statistical Models
Issue 1: Post-selection Inference
Issue 2: True Model and Well-Fitting Model
Issue 3: Substantive Model and Statistical Model
Modeling for Explanation and Modeling for Prediction
Conclusions, Implications, and Future Directions
References
Part IIIUnderstanding Internal Structures of Language Assessments
10 Developing Summary Content Scoring Criteria for University L2 Writing Instruction in Japan
Introduction
Literature Review
Constructs
The Process of Text Comprehension and Summary Production
Developing Content Assessment Criteria for L2 Summary Writing
Methods
Participants
Materials
Procedures
Scoring Criteria Development
Rater Training and Scoring
Analyses
Results
Discussion
Conclusions, Implications, and Future Directions
Appendix A: Scoring Criteria for Each Specific Main Point
Appendix B: Scoring Criteria for Each Framing Point
Appendix C: The Integration Rating Scale
Appendix D: The Language Rating Scale
References
11 Consistency of Computer-Automated Scoring Keys Across Authors and Authoring Teams
Introduction
Literature Review
Grounding of the Present Study in Bachman’s Work
Constructs Assessed in the Present Study
Web-Based Testing
Research on Computer-Automated Scoring
Generalizability Theory
The Present Study
Methods
Instrument
Participants
Automated Scoring of Responses
Key Authoring
Analyses
Results
Descriptive Statistics
Generalizability Theory Results
Issues in the Scoring Keys
Discussion
Research Question 1
Research Question 2
Research Question 3
Limitations of the Study
Conclusions, Implications, and Future Directions
References
12 Distinguishing Language Ability from the Context in an EFL Speaking Test
Introduction
Literature Review
The Nature of the “Fundamental Dilemma”
Research Question
Methods
Materials
Design
Data and Analysis
Results
Model Comparison
Cross-Validation
Discussion
Distinction Between Language Ability and Context
Language Ability
Performance on Tasks
Conclusions, Implications, and Future Directions
Appendix A: The Brief Rubrics for TEM4-Oral (Translated from Chinese)
References
13 The Effects of Proficiency Differences in Pairs on Korean Learners’ Speaking Performance
Introduction
Literature Review
Heritage Versus Non-heritage Language Learners
Paired Speaking Tests
Research Questions
Methods
Participants
Instruments
Procedures
Data Analysis
Results
Descriptive Statistics
Variance Components
Multivariate G-theory
Discussion
Limitations
Conclusions, Implications, and Future Directions
Appendix A: Analytic Oral Rating Rubrics
References
14 Conclusion
Summaries and Directions for Future Research
A Few Final Thoughts
References
Index
