Second Language Research: Methodology and Design

Cover......Page 1
Half Title......Page 2
Title Page......Page 4
Copyright Page......Page 5
Table of Contents......Page 6
Preface......Page 16
1. Introduction to Research......Page 20
1.1 Research Methods......Page 21
1.2 Different Types of Research......Page 22
1.3 What Is a Research Report?......Page 25
1.3.2 Abstract......Page 27
1.3.3 Introduction......Page 28
1.3.4.2 Materials......Page 30
1.3.4.3 Procedures......Page 32
1.3.4.4 Analysis......Page 34
1.3.5 Results......Page 35
1.3.7 Notes......Page 37
1.4 Identifying Research Questions......Page 38
1.4.1 Feasibility......Page 41
1.4.2 Research Questions and Hypotheses......Page 42
1.4.3 Replication......Page 45
Points to Remember......Page 46
More to Do and More to Think About . . .......Page 47
2.1 Ethical Issues in Research Involving Human Subjects......Page 49
2.1.1 Obtaining Informed Consent from Second Language Learners......Page 50
2.1.1.1 Sufficient Information......Page 51
2.1.1.2 Is Withholding Information Ever Necessary?......Page 54
2.1.1.3 Participant Comprehension in Informed Consent......Page 56
2.1.1.4 Voluntary Participation and Informed Consent......Page 58
2.1.1.5 The Informed Consent Document......Page 62
2.1.2.1 Purpose of Reviews and IRB Responsibilities......Page 63
2.1.2.2 Why Guidelines to Protect Human Subjects Were Developed......Page 64
2.1.2.3 Development of Research Codes of Ethics......Page 65
2.1.2.4 Preparing a Protocol for the IRB......Page 66
Points to Remember......Page 67
More to Do and More to Think About . . .......Page 69
3.1 Pilot Testing......Page 71
3.2 The Significance of Data Collection Measures......Page 72
3.3.1 Acceptability Judgments......Page 76
3.3.1.1 Materials......Page 78
3.3.1.2 Procedures......Page 80
3.3.2 Elicited Imitation......Page 84
3.3.3 Magnitude Estimation......Page 85
3.3.4 Truth-Value Judgments and Other Interpretation Tasks......Page 87
3.3.5 Sentence Matching......Page 89
3.4.1 Sentence Interpretation......Page 90
3.4.2 Reaction Time......Page 91
3.4.3 Moving Window......Page 92
3.4.4 Eye-tracking......Page 93
3.5 Interaction-Based Research......Page 94
3.5.1 Picture Description Tasks......Page 95
3.5.2 Spot the Difference......Page 96
3.5.4 Consensus Tasks......Page 100
3.5.6 Computer-Mediated Research......Page 103
3.6.1 Observations......Page 104
3.6.2 Introspective Measures......Page 105
3.6.2.1 Stimulated Recall......Page 106
3.6.2.2 Think-Alouds or On-Line Tasks......Page 108
3.6.2.3 Immediate Recalls......Page 113
3.7.1 Naturalistic Settings......Page 114
3.7.2 Elicited Narratives......Page 115
3.7.2.1 Silent Film......Page 116
3.7.2.3 Picture Tasks......Page 117
3.7.3 Discourse Completion Test (DCT)......Page 118
3.7.5 Video Playback for Interpretation......Page 120
3.8 Questionnaires and Surveys......Page 121
3.9.2 Other Corpora......Page 125
3.9.3 IRIS......Page 126
Points to Remember......Page 127
More to Do and More to Think About . . .......Page 128
4.1 Preparing Data for Coding......Page 131
4.1.1.1 Transcription Conventions......Page 132
4.1.1.3 Technology and Transcription......Page 135
4.2.1 Scales of Measurement......Page 136
4.2.2 Coding Nominal Data......Page 137
4.2.3 Coding Ordinal Data......Page 139
4.2.4 Coding Interval Data......Page 140
4.3 Coding Systems......Page 141
4.3.1.1 T-Units......Page 142
4.3.1.2 Suppliance in Obligatory Contexts (SOC)......Page 144
4.3.1.3 CHAT......Page 145
4.3.2.1 Question Formation......Page 146
4.3.2.3 Classroom Interaction......Page 150
4.3.2.4 Second Language Writing Instruction......Page 153
4.3.3 Coding Qualitative Data......Page 156
4.4 Inter-Rater Reliability......Page 158
4.4.1.3 Additional Measures of Reliability......Page 159
4.4.1.5 How Data Are Selected for Inter-Rater Reliability Tests......Page 160
4.4.1.6 When to Carry out Coding Reliability Checks......Page 161
4.5.1 How Much to Code?......Page 162
4.5.2 When to Make Coding Decisions?......Page 163
4.6 Software for Coding......Page 164
4.7 Conclusion......Page 166
More to Do and More to Think About . . .......Page 167
5.2 Hypotheses......Page 169
5.3 Variable Types......Page 171
5.3.1 Independent and Dependent Variables......Page 173
5.3.3 Intervening Variables......Page 174
5.3.4 Control Variables......Page 175
5.5 Validity......Page 177
5.5.3 Construct Validity......Page 178
5.5.6 Internal Validity......Page 179
5.5.6.1 Participant Characteristics......Page 180
5.5.6.2 Participant Mortality......Page 182
5.5.6.3 Participant Inattention and Attitude......Page 184
5.5.6.4 Participant Maturation......Page 185
5.5.6.6 Instrumentation and Test Effect......Page 186
5.5.7 External Validity......Page 190
5.5.7.1 Sampling......Page 191
5.5.7.2 Representativeness and Generalizability......Page 194
5.5.7.3 Collecting Biodata Information......Page 196
5.6 Reliability......Page 199
5.6.1.1 Inter-Rater and Intra-Rater Reliability......Page 200
5.6.2.3 Internal Consistency......Page 201
Points to Remember......Page 202
More to Do and More to Think About . . .......Page 203
6.1 Introduction......Page 207
6.2 Research Materials......Page 210
6.3 Intact Classes......Page 213
6.4 Counterbalancing......Page 215
6.5.1 Correlational (Associational) Research......Page 217
6.5.2.1 Comparison Group Design......Page 218
6.5.3 Measuring the Effect of Treatment......Page 220
6.5.3.1 Pre-Test/Post-Test Design......Page 221
6.5.3.2 Post-Test Only Design......Page 222
6.5.4 Repeated Measures Design......Page 223
6.5.5 Factorial Design......Page 224
6.5.6 Time-Series Design......Page 226
6.5.7 One-Shot Designs......Page 229
6.5.8 Meta-Analyses......Page 230
Points to Remember......Page 231
More to Do and More to Think About . . .......Page 232
7.1 Defining Qualitative Research......Page 234
7.2 Gathering Qualitative Data......Page 236
7.2.1 Ethnographies......Page 238
7.2.1.1 Advantages......Page 240
7.2.2 Case Studies......Page 241
7.2.2.2 Caveats......Page 243
7.2.3.1 Advantages......Page 244
7.2.3.2 Caveats......Page 245
7.2.4.1 Advantages......Page 246
7.2.5 Diaries/Journals......Page 247
7.2.5.1 Advantages......Page 248
7.3 Analyzing Qualitative Data......Page 249
7.3.1 Credibility, Transferability, Confirmability, and Dependability......Page 250
7.3.2 Triangulation......Page 252
7.4 Conclusion......Page 253
Points to Remember......Page 254
More to Do and More to Think About . . .......Page 255
8.1 Classroom Research Contexts......Page 257
8.2.1 Observations......Page 258
8.2.1.1 Conducting Classroom Observations......Page 259
8.2.1.2 Observation Procedures and Coding Schemes......Page 262
8.3 Introspective Methods in Classroom Research......Page 272
8.3.2 Stimulated Recall......Page 273
8.3.3 Diary Research in Classroom Contexts......Page 275
8.4.1 Logistical Issues to Consider When Carrying out Classroom Research......Page 277
8.4.2.1 Informed Consent......Page 280
8.4.2.3 Ensuring Confidentiality and Minimizing Disruption......Page 281
8.4.2.5 Considering the Instructional Setting......Page 282
8.4.2.6 Summary of Problematics......Page 283
8.5.1 The Relationship between Instruction and Learning in Second Language Classrooms......Page 284
8.5.2.1 Definitions......Page 287
8.5.2.3 Action Research in Practice......Page 288
8.6 Conclusion......Page 291
Points to Remember......Page 292
More to Do and More to Think About . . .......Page 293
9.1 Introduction......Page 294
9.2 What Are Mixed Methods?......Page 296
9.3 Why Use Mixed Methods?......Page 297
9.4 Types of Mixed-Methods Studies......Page 299
9.5 Conducting a Mixed-Methods Study......Page 302
9.5.2 Planning Phase......Page 303
9.6 Analyzing Data from a Mixed-Methods Study......Page 306
Points to Remember......Page 307
More to Do and More to Think About . . .......Page 308
10.2 Descriptive Statistics......Page 311
10.2.1 Measures of Frequency......Page 312
10.2.2.3 Mean......Page 316
10.2.2.4 Outliers......Page 318
10.2.3 Measures of Dispersion......Page 320
10.3 Normal Distribution......Page 323
10.4 Standard Scores......Page 325
10.5 Probability......Page 326
10.6 Inferential Statistics......Page 330
10.6.1.3 Degrees of Freedom......Page 331
10.6.1.5 One-Tailed versus Two-Tailed Hypotheses......Page 332
10.6.2 Parametric versus Nonparametric Statistics......Page 333
10.6.3.1 T-Tests......Page 334
10.6.3.2 Analysis of Variance (ANOVA)......Page 336
10.6.3.4 Analysis of Covariance (ANCOVA)......Page 337
10.6.3.6 Repeated Measures ANOVA......Page 339
10.6.4.1 Chi Square (x2)......Page 340
10.6.4.2 Mann-Whitney U/Wilcoxon Rank Sums......Page 341
10.7 Statistical Tables......Page 342
10.10 Correlation......Page 344
10.10.1 Pearson Product-Moment Correlation......Page 346
10.10.1.1 Linear Regression......Page 349
10.10.1.2 Multiple Regression......Page 350
10.11 New Approaches......Page 351
10.11.2 Effect Size......Page 353
10.11.3 Confidence Intervals......Page 354
10.12 Meta-Analyses......Page 355
10.13.1 SPSS......Page 356
Points to Remember......Page 357
More to Do and More to Think About . . .......Page 358
11.1 The Importance of Reporting Research......Page 363
11.2 The Final Stages in Reporting Quantitative Research......Page 364
11.2.1 The Discussion......Page 366
11.2.2 Limitations, Future Research, and Conclusion Sections......Page 369
11.3 The Final Stages in Reporting Qualitative Research......Page 372
11.4 Reporting Mixed-Methods Research......Page 375
11.5.1 The Research Problem and Questions......Page 376
11.5.2 The Research Hypotheses......Page 377
11.5.4 The Abstract......Page 378
11.5.6 The Design of the Study......Page 379
11.5.8 Participants......Page 380
11.5.10.2 Coding Systems......Page 381
11.5.10.5 Statistics......Page 382
11.5.12 References......Page 383
11.5.13.2 Figures......Page 385
11.5.15 Post-Research Concerns......Page 386
11.5.16 Final Touches and Formatting......Page 387
11.6 Conclusion......Page 388
More to Do and More to Think About . . .......Page 389
Appendix A: Sample Short Form Written Consent Document for Subjects Who Do Not Speak English......Page 391
Appendix B: Consent to Participate in Research......Page 393
Appendix C: Sample Consent Form for a Study in a Foreign Language Context......Page 395
Appendix D: Sample Consent Form for a Classroom Study......Page 397
Appendix E: Sample Transcription Conventions: “Jeffersonian” Transcription Conventions......Page 400
Appendix F: Sample Transcription Conventions for the L2 Classroom......Page 403
Appendix G: Commonly Used Formulae......Page 406
Glossary......Page 408
References......Page 424
Subject Index......Page 444