The Significance Test Controversy: A Reader

Tests of significance have been a key tool in the research kit of behavioral scientists for nearly fifty years, but their widespread and uncritical use has recently led to a rising volume of controversy about their usefulness. This book gathers the central papers in this continuing debate, brings the issues into clear focus, points out practical problems and philosophical pitfalls involved in using the tests, and provides a benchmark from which further analysis can proceed.The papers deal with some of the basic philosophy of science, mathematical and statistical assumptions connected with significance tests and the problems of the interpretation of test results, but the work is essentially non-technical in its emphasis. The collection succeeds in raising a variety of questions about the value of the tests; taken together, the questions present a strong case for vital reform in test use, if not for their total abandonment in research.The book is designed for practicing researchers-those not extensively trained in mathematics and statistics that must nevertheless regularly decide if and how tests of significance are to be used-and for those training for research. While controversy has been centered in sociology and psychology, and the book will be especially useful to researchers and students in those fields, its importance is great across the spectrum of the scientific disciplines in which statistical procedures are essential-notably political science, economics, and the other social sciences, education, and many biological fields as well.Denton E. Morrison is professor, Department of Sociology, Michigan State University.Ramon E. Henkel is associate professor emeritus, Department of Sociology University of Maryland. He teaches as part of the graduate faculty. "Our interest in compiling this volume stems from our concern with the considerable amount of indiscriminate use of significance tests in behavioral research. Even their strongest proponents agree that there is much misuse, misinterpretation, and meaningless use of the tests. More important than the question of how the tests are correctly used, however, is the question of whether the tests are useful, and why or why not. We are concerned that many users lack an understanding of the latter questions. We do not know how much of this lack is because an extensive literature critical of past and current practice in the use of significance tests is unknown to researchers and how much results from a failure to heed the criticism. But we hope that collecting a substantial portion of this literature in one volume will help make researchers more mindful of both the practical problems and philosophical pitfalls involved in using the tests. While the essential tone of this volume is critical of the tests, our broader purpose is to document the controversy over use of the tests in behavioral research. We have used the term "controversy" to characterize the literature on significance tests, but the sense in which a "controversy" exists must be understood in a special way. "Controversy" implies dialogue over points of disagreement, and in view of the fact that such dialogue has not always occurred, the term may be an overstatement. Both in sociology and psychology critics of the tests have reacted to what they view as erroneous research practice based on misguided statistical training. Essays that respond specifically to this criticism by explicitly defending the tests have appeared only in the sociological literature, however, so that the controversy has only in part taken the form of an extended debate or dialogue. In the behavioral sciences in general the overwhelming practice by both researchers and those responsible for statistical training has been to ignore the issues raised by the critics and to continue doing things as before. Thus the preponderance of the negative side of the "debate" in this volume does not represent so much bias as redress, since the amount of behavioral science writing that implicitly supports the tests is far greater than that which is critical."