Questionnaires in Second Language Research

Cover
Half Title
Series Information
Title Page
Copyright Page
Table of Contents
Preface
Introduction
1 Questionnaires in Second Language Research
1.1 What Are “Questionnaires” and What Do They Measure?
1.1.1 What a Questionnaire Is Not
1.1.2 What Do Questionnaires Measure?
1.2 Using Questionnaires: Pros and Cons
1.2.1 Advantages
1.2.2 Disadvantages
Simplicity of Answers
Unreliable and Unmotivated Respondents
Respondent Literacy Problems
Little Or No Opportunity to Correct Respondents’ Mistakes
Social Desirability (Or Prestige) Bias
Self-Deception
Self-Selection Bias
Acquiescence Bias
Halo Effect
Fatigue Effects
1.3 Questionnaires in Quantitative and Qualitative Research
2 Constructing the Questionnaire
2.1 General Features
2.1.1 Length
2.1.2 Layout
2.1.3 Sensitive Topics and Anonymity
Sensitive Topics
Anonymity
2.2 The Main Parts of a Questionnaire
2.2.1 Title
2.2.2 Instructions
General Instruction
Specific Instructions
2.2.3 Questionnaire Items
2.2.4 Sociobiographical Information
2.2.5 Additional Information
2.2.6 Final “Thank You”
2.3 Appropriate Sampling of the Questionnaire Content and the Significance of “Multi-Item Scales”
2.3.1 Appropriate Sampling of the Content
2.3.2 Using Multi-Item Scales
2.4 “Closed-Ended” Questionnaire Items
2.4.1 Rating Scales
Likert Scales
Variations On Likert Scales
Semantic Differential Scales
Numerical Rating Scales
True–False Items
Mixing Scale Types
2.4.2 Multiple-Choice Items
2.4.3 Rank Order Items
2.4.4 Numeric Items
2.4.5 Checklists
2.5 Open-Ended Questions
2.5.1 Specific Open Questions
2.5.2 Clarification Questions
2.5.3 Sentence Completion Items
2.5.4 Short-Answer Questions
2.6 How to Write Good Items
2.6.1 Drawing Up an “Item Pool”
2.6.2 Rules About Writing Items
Aim for Short and Simple Items
Use Simple and Natural Language
Avoid Ambiguous Or Loaded Words and Sentences
Avoid Negative Constructions
Absolutely Avoid Double-Barreled Questions
Avoid Hidden Assumptions
Avoid Items That Are Likely to Be Answered the Same Way By Everybody
Include Both Positively and Negatively Worded Items
Check That Questions and Answer Options Match
Write Translatable Items
2.6.3 Writing Sensitive Items
Confidential Personal Information
Undesirable Social Behavior
Potential Threat
2.7 Grouping and Sequencing Items
2.7.1 Clear and Orderly Structure
2.7.2 Opening Questions
2.7.3 Factual (Or “Personal” Or “Classification”) Questions at the Beginning Or at the End?
2.7.4 Open-Ended Questions at the End
2.8 Translating the Questionnaire
2.8.1 Translation as a Team-Based Approach
2.8.2 Translation With Limited Resources
2.9 Computer Programs for Constructing Questionnaires
2.10 Piloting the Questionnaire and Conducting Item Analysis
2.10.1 Initial Piloting of the Item Pool
2.10.2 Final Piloting (“Dress Rehearsal”)
2.10.3 Item Analysis
3 Administering the Questionnaire
3.1 Selecting the Sample
3.1.1 Sampling Procedures
Random Sampling
Convenience Or Opportunity Sampling
Snowball Sampling
Quota Sampling
Non-probability Sampling and Non-Representativeness
3.1.2 How Large Should the Sample Be?
3.1.3 The Problem of Respondent Self-Selection
3.2 Main Types of Questionnaire Administration
3.2.1 Administration By Mail
The Cover Letter
Follow-up Letters
Guidelines for Increasing Mail Survey Return Rates
3.2.2 One-To-One Administration
3.2.3 Group Administration
3.3 Strategies to Increase the Quality and Quantity of Participant Response
3.3.1 Advance Notice
3.3.2 Attitudes Conveyed By Teachers, Parents, and Other Authority Figures
3.3.3 Respectable Sponsorship
3.3.4 The Presence of a Survey Administrator
3.3.5 The Behavior of the Survey Administrator
3.3.6 Communicating the Purpose and Significance of the Survey
3.3.7 Emphasizing Confidentiality
3.3.8 Reading Out the Questionnaire Instructions
3.3.9 The Style and Layout of the Questionnaire
3.3.10 Promising Feedback On the Results
3.4 Questionnaire Administration, Confidentiality, and Other Ethical Issues
3.4.1 Basic Ethical Principles of Data Collection
3.4.2 Obtaining Consent for Children
3.4.3 Strategies for Getting Around Anonymity
Identification Through the Seating Plan
A Self-Generated Identification Coding Procedure
4 Processing Questionnaire Data
4.1 Coding Questionnaire Data
4.1.1 First Things First: Assigning Identification Codes
4.1.2 Coding Quantitative Data
4.2 Inputting the Data
4.2.1 Creating and Naming the Data File
4.2.2 Keying in the Data
4.3 Processing Closed Questions
4.3.1 Data Cleaning
4.3.2 Data Manipulation
Handling Missing Data
Recoding Values
Standardizing the Data
4.3.3 Reducing the Number of Variables in the Questionnaire
4.3.4 Main Types of Questionnaire Data
4.3.5 Examining the Reliability and Validity of the Data
Internal Consistency Reliability
Measuring and Ensuring Internal Consistency Reliability
4.3.6 Statistical Procedures to Analyze Data
Descriptive Statistics
Inferential Statistics
4.4 Content Analysis of Open-Ended Questions
4.5 Computer Programs for Processing Questionnaire Data
4.6 Summarizing and Reporting Questionnaire Data
4.6.1 General Guidelines
How Much to Generalize
Detachment From Real Life
4.6.2 Technical Information to Accompany Survey Results
4.6.3 Reader-Friendly Data Presentation Methods
Figures
Tables
4.7 Complementing Questionnaire Data With Other Information
4.7.1 Questionnaire Survey With Follow-Up Interview Or Retrospection
4.7.2 Questionnaire Survey Facilitated By Preceding Interview
5 Online Questionnaires
5.1 The Effect of Setting and Social Obligation in Paper and Online Questionnaires
5.1.1 Social Context in Online Questionnaires
5.1.2 The Ephemeral Relationship Between Participant and Researcher
What Is the Minimum Number?
5.2 Fear and Distrust of All Types of Questionnaires
5.3 When Online Questionnaires Are Impractical Or Impossible
5.4 Non-Probability Sampling in Online Questionnaires
5.5 Developing a Good Online Questionnaire
5.5.1 Knowing Something About Statistics Before Creating a Questionnaire
5.5.2 Choosing a Platform
Three Platforms in the Spotlight
5.5.3 Pilot Testing the Online Questionnaire
Pilot Testing the Video
5.6 Getting the Snowball to Roll
5.6.1 Producing an Attractive Call for Participation
5.6.2 The Appearance of the Questionnaire
5.6.3 The Use of a Progression Bar
5.6.4 Friendly Academic Nudging
5.6.5 Rewarding Participants
5.7 When the Harvest Is In
5.7.1 Cleaning the Data
5.7.2 Calculating Internal Consistency
5.7.3 Being Careful With Automatic Coding
5.8 Running the Statistics to Answer the Research Questions: The Hallelujah Moment
5.8.1 Double-Checking Suspicious Results
5.8.2 Playing By the Rules
5.8.3 General Data Protection Regulation (GDPR)
Further Reading
6 Developing Psychometrically Sound Instruments
6.1 Developing and Validating the Short-Form Foreign Language Enjoyment Scale (S-FLES) (Botes Et Al., 2021)
6.1.1 Background
6.1.2 Developing a Short Form Through Sequential Steps
Step 1: Establishing a Factor Structure
Step 2: Item Selection
Step 3: Confirming the Factor Structure
Step 4: Reliability and Validity Checks
6.1.3 Concluding Remarks On the S-FLES
6.1.4 Final Short-Form Foreign Language Enjoyment Scale (S-FLES)
6.2 Developing and Validating a Questionnaire to Measure Intensity and Quality of L2 Engagement (Teravainen-Goff, in Preparation)
6.2.1 Background
6.2.2 Key Challenges in Developing the Intensity and Quality of an L2 Engagement Questionnaire
Defining and Operationalizing the Concepts
Developing Questionnaire Items
Creating the Final Scales
6.2.3 Final Version of Intensity and Quality of L2 Engagement Questionnaire (Teravainen-Goff, in Preparation)
6.3 Developing and Validating the Foreign Language Learning Boredom Scale (FLLBS) (Li Et Al., 2020)
6.3.1 Initial Conceptualization of the Target Construct
6.3.2 Development of the Initial Questionnaire
6.3.3 Validation of the Questionnaire
Face Validity and Item Modifications
Exploration of the Factor Structure
Validity Tests
Reliability Tests
6.3.4 Final Foreign Language Learning Boredom Scale and Its English Translation
7 Conclusion and Checklists
7.1 Constructing the Questionnaire
7.2 Administering the Questionnaire
7.3 Processing Questionnaire Data
References
Appendix
IRIS: A Digital Repository of Instruments and Materials for Research Into Second Languages
Tools for Second Language Speech Research and Teaching
Further Reading
Index