This volume explores peer review in the scientific community and academia. While peer review is as old as modern science itself, recent changes in the evaluation culture of higher education systems have increased the use of peer review, and its purposes, forms and functions have become more diversified. This book put together a comprehensive set of conceptual and empirical contributions on various peer review practices with relevance for the scientific community and higher education institutions worldwide. Consisting of three parts, the editors and contributors examine the history, problems and developments of peer review, as well as the specificities of various peer review practices. In doing so, this book gives an overview on and examine peer review , and asks how it can move forward.
Author(s): Eva Forsberg, Lars Geschwind, Sara Levander, Wieland Wermke
Edition: 1
Publisher: Palgrave Macmillan
Year: 2022
Language: English
Commentary: TruePDF
Pages: 412
Tags: Higher Education Assessment, Testing And Evaluation; Research Skills
Contents
Notes on Contributors
List of Figures
List of Tables
Part I: Peer Review: Introduction and Its Context
1: Peer Review in Academia
Introduction
The Many Face(t)s of Peer-Review Practices
The Interplay of Primary and Secondary Peer Review
Sites and Reviewer Tasks and Roles
Power-making in the Selection of Expertise
The Academic Genres of Peer Review
Criteria and the Professional Judgement of Quality
The Autonomy of a Profession and a Challenged Contract
Why Study Peer Review?
State of the Art of Research on Peer Review
The Structure and Content of the Volume
References
2: Hierarchies and Universal Inclusion in Scientific Communities
Tensions and Contradictions in Contemporary Society
Universal Inclusion
Publication as the Elementary ‘Unit-Act’ of the System of Science
Authorship of Publications as the Form of Inclusion in Science
Citations as the Internal Structure of Publications
Reading and Writing in Scientific Communities: The Hierarchy of Authorship
The Emergence of Peer Review: The Hierarchy of Readership
The Two Hierarchies: Authorship and Readership
The Third and Fourth Hierarchy: Hierarchy of Publication Places and Hierarchy of Recruitment for Co-authorship
Two Modalities of Quality Control in Science
Anticipatory, Centralized Control by Scientific Elites
Post-hoc, Decentralized Market Control Based on Universal Inclusion
Concluding Remarks
References
3: “Disciplining” Educational Research in the Twentieth Century
Introduction
Reviewers and Authors
Initial Expectations
Further Expansion
Authors and Reviewers
Community of Peers
Blind Peer Review
Papers and References
Suggestions for Contributors
Citation Consciousness
Conclusion
References
Part II: Peer Review and the Higher Education Evaluation Machinery
4: Gatekeepers on Campus: Peer Review in Quality Assurance of Higher Education Institutions
Which Gates Do Peers Keep?
Quality of Higher Education Institutions
Quality of Performance and Performance Indicators
Strengths and Weaknesses of Performance Indicators
Quality of the Organisation and Peer Review
Strengths and Weaknesses of Peer Review in Quality Assurance
Persistent Dilemmas: Instead of a Conclusion
References
5: The Many Faces of Peer Review
Introduction
Classical Peer Review as One of Several Evaluation Models
Informed and Standards-Based Peer Review
Modified Peer Review
Extended Peer Review
Peer Review in the Educational Field
Conclusion and Discussion
References
6: Your Brother’s Gatekeeper: How Effects of Evaluation Machineries in Research Are Sometimes Enhanced
Introduction
Metrics and Ambiguity
A Case Study of the Danish Bibliometric Indicator
Vignettes
Discussion and Conclusion
References
7: Peer Review in Public Administration: The Case of the Swedish Higher Education Authority
The National System for Quality Assurance of Higher Education in Sweden
The History
Critique
2011: A New System Is Launched
Peer Review
Sources
The Role of the Peers and the Method of Peer Review
Early Assessments 1997–1998
Later Assessments 2016–2017
Grasping the Concept of Result
The Early Assessments 1997–1998
Later Assessments 2016–2017
Collegiality as a Form of Governance
Peer Review in the National Quality Assurance System
Sources
Directions on Assessments
Reports Describing Whole Quality Assurance Systems
Guidelines for HEIs and Peers
Reports on Reviews Conducted Within the Framework of National Systems for Quality Assurance of Higher Education
Accounts of Government Assignments
Meta-analyses of the Authority’s/the Authority’s Own Reviews
Other Reports
Other
References
8: Performance-Based Evaluation Metrics: Influence at the Macro, Meso, and Micro Level
Introduction
Algorithmic Historiography and the Birth of Scientometrics
Bibliometrics for the Evaluation of Research
The Citation as Mediator: The Performativity of “Being Cited”
Consequences for Research Policy
Co-production of Science and Society
Performance-Based Research Allocation Models at Three Levels
PRFS at the Macro Level: A Comparison Between the Norwegian and Swedish Systems
At the Meso Level, Within Universities
At the Micro Level, Individual Researchers
Conclusions: The Performativity of Performance-Based Research Funding on Different Levels
References
9: Peer Advocacy: Expressions of Loyalty in Peer Review
Introduction
Aims and Uses of Peer Review
Advocacy in Academia
Cases of Advocacy
Case 1: National Reviews of Subjects and Programmes
Case 2: Institutional Review of an Academic Development Unit
Case 3: External Evaluators Appointed to Promote an Application for University Status
Case 4: HR Competencies in Academic Appointment Processes
Discussion
Conclusions and Recommendations
References
10: Is Peer Review Fit for Purpose?
Introduction
Methodology
Refereed Journal Articles
The Assessment of Doctoral Degrees
The UK Research Excellence Framework (REF)
Conclusion
References
Part III: Specificities of Different Peer-Review Practices
11: Peer Review in Academic Promotion of Excellent Teachers
Introduction
Career and Reward Structure in Swedish Higher Education
Gatekeeping in the Promotion of ‘Distinguished University Teachers’
The Process of Promoting Teaching Excellence
The Mandatory Content of the Application Dossier
Faculty Guidelines and Candidate Applications
Judgement and Legitimation in the Making of ‘Distinguished University Teachers’
The (E)valuation of the ‘Distinguished University Teacher’: The Judgement
The Scholarly Judgement of Criteria and Content
The Scholarly Judgement of Evidence
Justification of the (E)valuation: The Legitimation
Justification In-between Standardization and Professional Judgement
Justification Through Mandated Intertextuality
Justification Through Scholarly Judgement
The Intersection of Promotion, Peer Review, and Teaching Excellence
The Promotion of ‘Distinguished University Teachers’: The Same, But Different
A Latecomer and an Emergent Game Changer?
References
12: Peers in Systematic Review: Gate Keeping Understandings of Research in the Field
Introduction
Peers in Academic Publishing and Systematic Review
Analytical Framework
Method
Examples of Peers in three Review Processes
Swedish Institute for Educational Research
The Norwegian Knowledge Centre for Education
Danish Clearinghouse for Educational Research (DCU)
Peers in the Systematic Review Processes
Concluding Remarks
References
Selected Systematic Review Technical Reports Analysed in the Study
The Example Reports Used for the In-depth Reading Are Underlined
13: The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome
Introduction
Data Sources and Methods
Analysis of the Grant-Review Practices of RCN
The Various Models
Overview of Proposals and Grants
The Quality Criteria in the Assessments
Considerations Other than Research Quality in the Assessments
The Effects of the Various Ranking Processes
Important Organizational Factors Affecting Ranking
Conclusions
Appendix: The Review Criteria in the Different RCN Divisions
The RCN Divisions’ guidelines listed the following review criteria:
References
14: Typecasting in the Recruitment of Full Professors
Introduction
The Field of Educational Science(s)
Recruitment Procedures—A National Context
Typecasting in the Recruitment of Full Professors
The Intellectual and Social Organization of Educational Science(s)
The Emergence of a Heterogeneous Field
The Selection of Peers
The Mobilization of Job Categories
The Intersection Between Institutional Demands and Scientific Values
Eliminatory Criteria, Signs and Retrospective Judgments
Conclusive Notes
Typecasting as a Device of Peer Review in the Formation of Education Science(s)
The Professor of Education Science(s) in Late Modernity
References
15: Assessing Academic Careers: The Peer Review of Professorial Candidates
Introduction
Picking the Best: Peer Review in Assessment Procedures
Analysing Referee Reports
Judgement Devices and Intellectual Structure
Merits and Their Assessment: Five Main Themes
Authorship, or the Reading of By-Lines
Publication Prestige and the Importance of Articles in ‘Top-Journals’
Temporality. The Importance of a ‘Positive Trajectory’.
Racing for the Prize: Reputation Through Awards and Citations
Boundary Keeping and the Shielding of Academic Markets
Discussion
References
16: Bureaucratic, Professional and Managerial Power in University Tenure Track Recruitment
Introduction
Selection and Recruitment of Candidates: A Regulative Perspective
Recruitment and Tenure Track Systems in Finland
Collection and Analysis of Data
Influence and Actors in Recruitment: Bureaucratic, Managerial and Professional Powers
The Recruitment Process in Tenure Track
Defining the Position
Advertisement and Application
Evaluation
Decision and Justification
TUT Findings
Tenure Call
Setting up the Working Group
Evaluation
Decision and Justification
UTA Findings
Tenure Call
Evaluation
Decision and Justification
Conclusions
References
