L I E) R.AR.Y 
 
 OF THL 
 
 UN I VER.5ITY 
 
 or ILLINOIS 
 
 3TO 
 
\\\ Wo r« "^^ 
 
 iOO^* 3tantandvangJ,3in 
 
 
 Return this book on or before the 
 Latest Date stamped below. 
 
 University of Illinois Library 
 
 
 
 t., 
 
 9 
 
 •^ O <".* " 
 
 0"" 2 
 
 H^K 1113 
 
 '■■"7 1 '■ iOuf 
 
 y 
 
 W 
 
 K0\ 
 
 ■X W«W 
 
 'JUN 3 ]90 
 JULl 
 
 21S83 
 
 C 5 200S 
 
 L161 — 1141 
 
BULLETIN NO. 38 
 
 BUREAU OF EDUCATIONAL RESEARCH 
 COLLEGE OF EDUCATION 
 
 THE TECHNIQUES OF EDUCATIONAL 
 RESEARCH 
 
 By 
 
 Walter S. Monroe 
 
 Director, Bureau of Educational Research 
 
 and 
 
 Max D. Engelhart 
 -^ sistant, Bureau of Educational Research 
 
 PRICE 50 CENTS 
 
 PUBLISHED BY THE UNIVERSITY OF ILLINOIS, URBANA 
 '' ^ 1928 
 

PREFACE 
 
 The interest exhibited in the bulletin, "Reporting Educational 
 Research," published by the Bureau of Educational Research, March, 
 1925, indicated that a more extensive treatment of the techniques of 
 educational research would be welcome. With the exception of Chapter 
 \T, Reporting Educational Research, the present bulletin consists 
 largely of references to illustrations of research procedures. This type 
 of treatment has been employed because it is believed that graduate 
 students, or other persons desiring to learn how to carry on educational 
 research, would profit more from noting the procedures employed by 
 previous investigators than by reading an exposition of research 
 techniques which would necessarily be somewhat abstract. There is a 
 bibliography for each chapter except the last. However, the bibliog- 
 raphies for those chapters that deal with research techniques contain 
 relatively few references. This is due to the fact that as yet there have 
 been relatively few attempts by writers to deal with non-statistical 
 procedures. 
 
 Mr. J. A. Clark, a former Assistant in the Bureau of Educational 
 Research, contributed to the preparation of this bulletin by preparing 
 a preliminary list of illustrative references which were mimeographed 
 and placed in the hands of graduate students taking a seminar in 
 educational research. These students contributed a number of valu- 
 able suggestions and several additional references. The authors are 
 glad to acknowledge their indebtedness to all who have contributed to 
 the project. 
 
 Walter S. Monroe, Director. 
 
 Xovember 2, 1927. 
 
Digitized by the Internet Archive 
 
 in 2011 with funding from 
 
 University of Illinois Urbana-Champaign 
 
 http://www.archive.org/details/techniquesofeduc38monr 
 
TABLE OF CONTENTS 
 
 Page 
 
 Preface 3 
 
 Chapter I. Introduction 7 
 
 Chapter II. The Problem and Its Definition 14 
 
 Chapter III. Collecting Data 27 
 
 Chapter IV. Analyzing, Organizing, and Summarizing Data . 42 
 
 Chapter V. Determining Conclusions 49 
 
 Chapter VI. Reporting Educational Research 57 
 
^m CHAPTER I 
 
 ^M INTRODUCTION 
 
 ^^B What is educational research? A brief formal definition of edu- 
 cational research cannot describe the many and varied activities of 
 investigators in the field of education ; it can only give an epitomized 
 description of the general character of their activities. The following 
 definition is given to suggest the point of view of the authors in pre- 
 paring this bulletin. It is not expected that the reading of the state- 
 ment will result in an adequate understanding of educational research. 
 One who desires to understand this procedure must acquaint himself 
 with the techniques employed in representative studies, or better engage 
 in educational research. 
 
 Educational research is the name for a type of procedure employed 
 in answering thought questions^ about education ; that is, questions for 
 which answers must be manufactured by reflective thinking. This 
 procedure is one in which the best data obtainable are used and the 
 thinking is critical. At times a research worker may be concerned 
 with fact questions answerable by routine clerical activities. If the 
 answers to such questions are needed as a means of dealing with 
 thought questions, such activities are a phase of educational research. 
 
 The ultimate purpose of all educational research is the discovery 
 of procedures, rules, and principles relating to the various aspects of 
 education. Or to express it in a somewhat different way, the function 
 of educational research is to determine what we should do or attempt 
 to do in educating children and adults. However, many of the activ- 
 ities of research workers have as their immediate purpose the determi- 
 nation of what is, or what has been, rather than what should be. This 
 is not inappropriate provided the securing of this information is looked 
 upon as a means for determining what should be. 
 
 The general plan of educational research. Educational research 
 begins with the formulation of the problem, usually in the form of a 
 
 'The phrase "thought question" is used in contrast to "fact question." As the terms 
 are used here, a fact question is one for which there is a ready-made answer, either in the 
 possession of the investigator or easily obtainable by him. In some instances the answer is 
 obtainable in the required form; in others some organization or calculations are necessary. 
 Hence, fact questions merge into thought questions and it is not possible to specify a definite 
 line of demarcation between. In general, if the calculations are simple and routine in char- 
 acter, the investigator may be said to be dealing with a fact question. If the work of hand- 
 'ing the collected information involves decisions and relatively complex calculation, the use 
 of the adjective "thought" is justified. 
 
 [7] 
 
8 Bulletin No. 38 
 
 concisely stated question or series of related questions. The culmina- 
 tion of this first step is the '"definition of the problem." The second 
 step is to collect the data that the problem calls for. This is followed 
 by an analysis, organization, and summary of the data so as to make 
 their meaning apparent. The final step is deriving answers to the 
 questions of the problem from the data and expressing these answers 
 in concise terms. 
 
 Although the procedure of educational research has been described 
 in terms of four steps, it is not a mechanical process. No two studies 
 are exactly alike. Variation rather than uniformity characterizes edu- 
 cational research. The four steps described vary in prominence. In 
 some studies the "analysis, organization, and summary of data" is 
 relatively simple ; in others it involves elaborate and intricate statistical 
 methods. In some cases "collecting data" is largely a clerical matter; 
 in others it requires a highly trained investigator. However, the four 
 steps furnish a helpful analysis of educational research. 
 
 The purpose of this bulletin. This bulletin is designed to assist 
 the reader in understanding the techniques of educational research. A 
 chapter is devoted to each of the four steps or phases mentioned in 
 the preceding paragraphs. Each of these chapters begins with a brief 
 exposition of the phase of educational research. This is followed by a 
 list of references to reports that furnish illustrations of the techniques 
 employed by investigators. Each reference is accompanied by an 
 annotation that describes the technique briefly but in most cases the 
 reader will need to consult the report in order to understand the 
 technique fully.- In addition, there is a carefully selected bibliography 
 in which techniques are considered. The final chapter is devoted to 
 "reporting educational research" which is here considered as a separate 
 phase. 
 
 Descriptions of technique in reports of research. In reporting 
 educational research, the writer usually describes how he accomplished 
 some phases of his work, but frequently important techniques are not 
 mentioned. For example, in reporting a historical study it is unusual 
 to describe the methods employed in locating the sources and the; 
 methods of handling the facts obtained. On the other hand, the report! 
 of an experimental investigation to determine the relative merits of' 
 two methods of teaching a subject usually contains a detailed descrip-j' 
 tion of the procedure followed in collecting the data. 
 
 t| 
 
 -The reader should bear in mind that these reports of educational research may not be|| 
 perfect. In some cases they are not in agreement with the statements by the authors of thi.'j 
 bulletin. 
 
 fl 
 
Techniques of Educational Research 9 
 
 BIBLIOGRAPHY 
 
 General Discussion of Educational Research 
 
 Alexander, Carter. "Opportunities for Research in Educational 
 
 Finance," Educational Administration and Supervision, 9 : 209-22, 
 
 April, 1923. Also in The Phi Delta Kappan, 5 :5-8, April, 1923. 
 
 The man starting a research problem should ask himself, "Is there 
 demonstrable need for research in this field ?" — "Just what problems in it 
 most need to be investigated, and why?'"- — "What methods should be used?" 
 — "For what problems in it am I so well equipped personally that I can 
 reasonably hope for success?" The article discusses several important 
 problems worthy of investigation in the field of educational finance. 
 
 Bay, James C. "The Training of a Research Worker in Education," 
 
 School and Society, 25 :274-78, March, 1927. 
 
 A discussion of the qualifications of the research worker is given. 
 Some of these qualifications are "a mastery of detailed processes of 
 analysis," "possession of definite standards of judgment," and "familiarity 
 with materials and methods." Some of the training should comprise 
 philosophy, mathematical analysis, statistics, and scientific attitude. 
 
 Brooks, Fowler D. "Criteria of Educational Research," ScJiool and 
 
 Society, 18:724-29, December, 1923. 
 
 Seven criteria are given for the evaluation of educational research, 
 the application of which would reveal that much so-called educational 
 research is not research at all. The author gives a detailed analysis of 
 typical examples of faulty technique. 
 
 Charters, W. W. "Pure Research and Practical Research," Journal 
 
 of Educational Research, 12:95-101, September, 1925. 
 
 The pure research worker may pick his problems without regard to 
 social use. The practical investigator finds his problems "localized within 
 practice, and in the end they must again be applied to the improvement of 
 practice." The article closes with the listing of five steps of practical 
 research. 
 
 Columbia Associates in Philosophy. An Introduction to Reflective 
 Thinking. Boston : Houghton Mifflin Company, 1923. 342 p. 
 
 A clear knowledge of reflective thinking is essential to the research 
 worker in any field. The investigator in education will find this book an 
 invaluable aid in clarifying his own ideas as to the nature of reflective 
 thinking and its use in the solution of a research problem. Chapter 13 
 contains a summary in which certain "tests of thought and beliefs" are 
 given. 
 
 Courtis, S. A., and Packer, P. C. "Educational Research," Journal 
 of Educational Research, 1 :5-19, January, 1920. 
 
 An analogy is drawn between education and warfare, with the research 
 department serving as scouts. The watchful, continuous collection of facts 
 is called the "survey" fvniction of educational research. The reporting of 
 progress is designated the "appraisal" function of research. The nature of 
 
10 Bulletin No. 38 
 
 these two functions is discussed, and the article closes with emphasis on 
 the importance of research to education. 
 
 CuBBERLY, E. P., Dearborn, W. F., Monroe, Paul, and Thorndike, 
 
 E. L. "Research Within the Field of Education, its Organization 
 
 and Encouragement," The School Review Monograph, Vol. 1, 
 
 1911, p. 1-54. 
 
 The rapid expansion of educational research in the last few years 
 makes this series of articles of especial significance. At that time Cubberly 
 indicated three major fields in need of investigation, those of finance, 
 county reorganization, and units of cost. Dearborn and Thorndike pre- 
 sented pleas for experimentation and quantitative measurement, while Paul 
 Monroe asked for cooperation in research. 
 
 Dewey, John. How We Think. Boston : D. C. Heath and Company, 
 
 1910. 224 p. 
 
 The steps in the solution of a research problem are so closely those 
 of reflective thinking, that the investigator will do well to clarify in his 
 own mind his ideas as to the nature of the thinking process. This book 
 may be designated as one of the classics in its field. 
 
 Hosic, J. F. "Assuming the Major Premise," Journal of Educational 
 
 Method, 2 :229-30, February, 1923. An editorial. 
 
 This is a protest against the practice of making hasty generalizations 
 from insufficient data. The author emphasizes the value of suspended 
 judgment. 
 
 JuDD, Charles H. "Educational Research and the American School 
 
 Program," The Educational Record, 4:165-77, October, 1923. 
 
 Educational research is the typical American method of solving the 
 problem of school control. Comparisons are made between American and 
 European methods of solving school problems. A short history of educa- 
 tional research is given, and the article closes with a discussion of needed 
 investigations. 
 
 JuDD, Charles H. "Research in Elementary Education," Journal of 
 
 Educational Psychology, 17:217-25, April, 1926. 
 
 The author emphasizes the need of research in fundamentals, rather 
 than in the details of present practice. Much of present educational 
 research is superficial, and is characterized by routine. Some of the fields 
 in need of fundamental productive investigations are suggested. 
 
 JuDD, Charles H., Briggs, Thomas H., Kelly, F. J., and Courtis, 
 
 S. A. "Research in Education," Fifteenth Yearbook of the 
 
 National Society of College Teachers of Education. Chicago: 
 
 University of Chicago Press, 1926, p. 56-93. 
 
 This is a series of four short articles which indicate the need of 
 research in the fields of elementarj^ secondary, and college education. 
 The article by Judd is the same as that given in the preceding reference. 
 The last article, that by Courtis, is a discussion of the development of 
 ability in research. 
 
Techniques of Educational Research 11 
 
 Kelly, F. J. "Scientific Method in College Administration and Col- 
 lege Teaching," School and Society, 20 : 390-96, September, 1924. 
 
 The author discusses the need for educational research on the college 
 level, particularly in the fields of evaluation of the results of college teach- 
 ing, rating of college teachers, and budget making. 
 
 Koos, Leonard V. "Research Problems in Collegiate Education," 
 School and Society, 17:169-174, February, 1923. 
 
 This is an interesting discussion of the need for educational research 
 on the college level. "Recent opportunities for observing a large amount 
 of classroom procedure in colleges, . . . have renewed the writer's 
 conviction that the whole realm of method in college instruction can be 
 made a vast field of valuable experimentation." 
 
 Light, U. L. "Muddling through from Guess to Science," Educa- 
 tional Review, 68 :5-9, June, 1924. 
 
 This article gives a discussion of the progress in educational practice 
 through the development of educational research. The author concludes 
 his article with a presentation of problems in need of investigation. 
 
 McCall, William A. Hozv to Experiment in Education. New York: 
 Macmillan Company, 1923. Chapter L 
 
 Hypothesis and experimentation are given as the third stage in the 
 search for truth. The author states that scientific educational research 
 may be grouped conveniently into three major divisions : descriptive, experi- 
 mental, and causal investigations. The purpose of each of these types is 
 discussed, and the nature of experimental investigations is gone into in 
 some detail. 
 
 Mead, A. R. "List of Possible Studies and Researches in Supervised 
 Student-Teaching," Educational Administration and Supervision, 
 11:355-358, August, 1925. 
 
 Those interested in research in the field of teacher training will find 
 this article of interest. Fortj^-eight problems are listed. 
 
 Monroe, Walter S. "Service of Educational Research to School 
 Administrators," The American School Board Journal, 70:37-39, 
 122, 125, April, 1925. 
 
 The reader is first given a definition of educational research. This is 
 followed by a brief discussion of the relative merits of certain procedures. 
 Increase in effiqiency in education, and not decrease in expense, should be 
 the goal of educational research. The article goes on with a discussion of 
 questions in need of solution, and concludes with emphasis on the value of 
 a critical attitude on the part of the research worker. 
 
 National Committee on Research in Secondary Education. "An Out- 
 line of Methods of Research with Suggestions for High School 
 Principals and Teachers," U. S. Bureau of Education Bidletin, 
 
12 Bulletin- No. 38 
 
 1926, No. 24. Washington : Government Printing Office, 1927. 
 63 p. 
 
 The research worker in education will find this bulletin helpful in 
 several ways. The need of educational research is pointed out. This is 
 followed by a brief discussion of the nature of educational research. 
 Qualifications of the research worker in education are outlined. The gen- 
 eral procedures of research are outlined, and a discussion of the question- 
 naire is given. A brief outline is given with reference to the reporting of 
 educational research. The next topic is a summary of suggestions and 
 directions. The bulletin closes with a list of research agencies available 
 to the principal and teacher. 
 
 Newlox, Jesse H. "What Research can do for the Superintendent," 
 Journal of Educational Research, 8:106-12, September, 1923. 
 
 The school superintendent should present policies to his school board 
 which are based on the findings of educational research. The author 
 emphasizes the importance of suspended judgment, understandable reports, 
 and of sound philosophy in educational research. 
 
 Reeder, W. G. Hozv to Write a Thesis. Bloomington, Ilhnois : PubUc 
 School PubHshing Company, 1925. 132 p. 
 
 This little book contains much information that would be of service 
 in writing a thesis. Illustrations of certain details of form will prove 
 useful, but the graduate student should be sure that these details are in 
 harmony with the regulations of his institution. 
 
 Thiesex, W. W, "Recent Progress in Educational Research," Journal 
 of Educational Research, 8: 301-14, November, 1923. 
 
 x\ discussion is given of some recent researches in such fields as 
 testing, statistical methods, curricula, remedial measures, learning, reten- 
 tion, and administration. Attention is called to opportunities for further 
 investigations. 
 
 Trow, Willi.\m C. Scientific Method in Education. Boston : Hough- 
 ton ]\lifflin Company, 1925. 159 p. 
 
 The reader will find this a rather general discussion of the scientific 
 method. The latter part of the book is devoted to the scientific method as 
 applied to educational problems. 
 
 "What is Research?" The School Review, 34:488-89, September, 1926. 
 An editorial. 
 
 The reader is told that much that is labeled educational research is 
 merely descriptive accounts of what is going on in the high school. Some 
 idea of the nature of real educational research may be gained from the 
 following quotations, "True and genuine research is one of the finest per- 
 formances of human intelligence and ingenuity." The title "research" 
 should be reserved "for publications which contribute to the understanding 
 of fundamental principles." 
 
Techkiques of Educational Research 13 
 
 Woody, Clifford. "The Values of Educational Research to the Class- 
 room Teacher," Journal of Educational Research, 26:172-178, 
 October, 1927. 
 
 Research on the part of the classroom teacher is of fundamental 
 significance to education. "Extended experiments dealing with teaching 
 practices in their natural situations are needed." Participation in research 
 is said to result in superior teaching, in professional growth, in under- 
 standing of current literature, in the establishment of teaching as a pro- 
 fession, and in certain monetary and spiritual rewards. 
 
CHAPTER II 
 THE PROBLEM AND ITS DEFINITION 
 
 How should a problem be expressed? A problem should be ex- 
 pressed either in the form of a question or so that the question to be 
 answered is clearly apparent. A statement of the topic is not satisfac- 
 tory. It merely names the particular held in which a problem is to be 
 formulated. The title of a thesis is usually a topic and not a statement 
 of the problem. 
 
 The definition of a problem. To define a problem means to specify 
 it in detail and with precision. Each question and subordinate ques- 
 tion to be answered is to be specified. The limits of the investigation 
 must be determined. Frequently, it is necessary to review previous 
 studies in order to determine just what is to be done. Sometimes it is 
 necessary to formulate the point of view or educational theory on 
 which the investigation is to be based. If certain assumptions are 
 made, they must be explicit!}' noted. 
 
 Relation of definition of problem to other phases of educational 
 research. The definition of the problem affords a basis for the sub- 
 sequent phases of educational research. It is the guide for the collect- 
 ing of data. The data are to be analyzed, organized, and summarized 
 so as to be most useful for answering the questions specified in the 
 definition of the problem and the conclusion is merely a statement of 
 the answers resulting from the investigation. 
 
 The basic importance of the problem and its definition indicates 
 that they should appear early in a report of educational research. 
 However, it is not unusual to find reports in which a large amount of 
 introductory material precedes the statement of the problem. Occasion- 
 ally this may be desirable but frequently a critical reader wonders if 
 the formulation of this introductory material did not precede the 
 formulation of the problem. 
 
 ILLUSTRATIVE REFERENCES 
 
 A. Statement of the problem. The general statement of the problem 
 appears in several forms : 
 
 1. A question or questions: 
 
 a. A single question. 
 
 Germane, Charles E. "Outlining and Summarizing Com- 
 pared with Re-Reading as Methods of Studying," The 
 
 [14] 
 
Techniques of Educational Research 15 
 
 Twentieth Yearbook of the National Society for the 
 
 Study of Education, Part II. Bloomington, Illinois : 
 
 Public School Publishing Company, 1921, p. 103-14. 
 
 After a very brief paragraph stating that there are no 
 scientific data on the subject, the problem is stated as follows: 
 "What is the value 'of making a 'corrected summary-outline' of 
 an article as compared with re-reading the same article for the 
 same length of time?" This is followed by a definition of 
 "corrected summary outline" and method of experimentation. 
 
 BuTTERWECK, JosEPH S. "The problem of teaching High 
 School Pupils How^ to Study," Teachers College, Colum- 
 bia University Contributions to Education, No. 237. New 
 York : Bureau of Publications, Columbia University, 
 1926. 116 p. 
 
 At the end of Chapter I, after a discussion of several 
 related topics, the problem is stated under the caption, "The 
 Problem of This Investigation." 
 
 Rosen, Esther Katz. "A Comparison of the Intellectual 
 and Educational Status of Neurotic and Normal Children 
 in Public Schools," Teachers College, Columbia Univer- 
 sity Contributions to Education, No. 188. New York : 
 Bureau of Publications, Columbia University, 1925. 51 p. 
 
 The question to be answered is on page 3, under the caption, 
 "Purpose of the Study." 
 
 b. Several questions. 
 
 ScATES, Douglas E. "A Study of High-School and First- 
 Year University Grades," School Review, 32:182-92, 
 March, 1924. 
 
 The problem is stated in the first paragraph on page 182 by 
 means of four questions as follows : 1. "To what extent does 
 good work in the high school predict good work in the first year 
 of college? ... 2. What would be the effect of raising the 
 high school scholarship average required for admission by the 
 University of Chicago? ... 3. How can first year college 
 grades be made of particular interest and value to the high school 
 principal? ... 4. Which first year students should be given 
 special study by the University?" 
 
 Powers, Samuel Ralph. "A Diagnostic Study of the 
 Subject Matter of High School Chemistry," Teachers 
 College, Columbia University Contributions to Educa- 
 tion, No. 149. New York : Bureau of Publications, Co- 
 lumbia University, 1924. 84 p. 
 
 The problem is stated on page 3 in terms of nine specific 
 questions the author proposes to answer. 
 
16 Bulletin No. 38 
 
 Graves, Katharine B. "The Influence of Specialized 
 Training on Tests of General Intelligence," Teachers 
 College, Columbia University Contributions to Education, 
 No. 143. Xew York: Bureau of Publications, Columbia 
 University, 1924. 78 p. 
 
 The problem is stated on page 3 in the form of four ques- 
 tions. 
 
 c. A single question followed by several sub-questions. 
 
 WooDRiNG, Maxie Nave. "A Study of the Quality of Eng- 
 lish in Latin Translations," Teachers College, Columbia 
 University Contributions to Education, No. 187. New 
 York : Bureau of Publications, Columbia University, 
 1925. 84 p. 
 
 After a discussion of the status, time required, and objec- 
 tives of Latin in the secondar}' schools, the problem is stated as 
 follows : 
 
 "Do Latin students use correct and effective English in 
 Latin translations? 
 
 "The question immediately resolves into several related 
 questions : 
 
 "1. What is the quality of the English used in Latin trans- 
 lations? 
 
 "2. How does the Enghsh in Latin translations compare 
 with the English in English compositions? 
 
 "3. How effective are the words used in Latin translations? 
 
 "4. What errors in grammar, rhetoric, and spelling occur in 
 the English of Latin translations?" ^hI 
 
 Stoxe, Cliff Winfield. "Arithmetical Abilities and Some 
 Factors Determining Them," Teachers College, Colut 
 bia University Contributions to Education, No. 19. Ne 
 York : Bureau of Publications, Columbia Universitj 
 1908. 101 p. 
 
 A general question is first asked in the opening paragraph 
 followed by two questions of a more particular nature. 
 
 Perrix, H. Ambrose. "The Local Status and Activities of 
 General Supervision in City Schools," Elementary School 
 Journal, 26:345-55, January, 1926. 
 
 At the beginning of the first paragraph, the reader is given 
 a statement of the question followed by two sub-questions. 
 
Techniques of Educational Research . 17 
 
 2. Declarative statement. 
 
 a. A single statement. 
 
 Anibel, Fred G. ''Comparative Efifectiveness of the Lec- 
 ture-Demonstration and Individual Laborator}- Alethod," 
 Journal of Educational Research, 13 :355-65, ^lay, 1926. 
 
 The reader is given a direct statement of the problem in 
 the opening sentence : "The purpose of this study was to de- 
 termine scientifically through the use of objective data just how 
 the results of the lecture-demonstration method of presenting 
 natural science subject matter compared with the individual labo- 
 ratorj' method of presenting exactly the same material." This 
 sentence is followed b}- further definition of the problem. 
 
 Hertzberg, Oscar Edward. "A Comparative Study of 
 Different Methods Used in Teaching Beginners to 
 Write," Teachers College, Columbia University Contri- 
 butions to Education, No. 214. New York : Bureau of 
 Publications, Columbia University, 1926. 60 p. 
 
 After a brief discussion of the work of other investigators, 
 the following statement of the problem is given at the bottom 
 of page 2 : "The purpose of the present study was to obtain 
 results which would adequately measure the effectiveness of the 
 direct learning method of teaching beginners to write as con- 
 trasted with the mechanical device method." 
 
 b. A single statement containing several phases. 
 
 Brooks, Fowler Dell. "Changes in Mental Traits with 
 Age Determined by Annual Retests," Teachers College, 
 Columbia University Contributions to Education, No. 116. 
 New York: Bureau of Publications, Columbia Univer- 
 sity, 1921. 86 p. 
 
 After a short discussion of the work of Thorndike on the 
 same topic, the reader is given a rather general statement of the 
 problem : 
 
 "This investigation seeks to find out what changes in mental 
 traits take place with age, and it seeks to find them out in the 
 only way the)- can be found out accuratel)- — by discovering what 
 changes actually do take place in the same individuals from one 
 year to another." This is followed by two correlated problems : 
 "(1) to investigate the correlation between mental functions at 
 different ages of the same individuals, and (2) to study the 
 relation of intellectual ability to rate of improvement over a 
 longer period of time than has heretofore been reported upon." 
 
18 Bulletin No. 38 
 
 Allen, I, M. "Improving the Professional Status of Teach- 
 ers," Elementary School Journal, 26:430-40, February, 
 1926. 
 
 In the second paragraph, the reader is given a statement of 
 the problem consisting of three separate purposes. 
 
 Anderson, Earl Williams. "The Teacher's Contract and 
 Other Legal Phases of Teacher Status," Teachers Col- 
 lege, Columbia University Contributions to Education, 
 No. 246. New York : Bureau of Publications, Columbia 
 University, 1927. 180 p. 
 
 After a brief discussion as to its importance, the problem 
 is stated on page 1 as undertaking four things. 
 
 c. A series of complete statements. 
 
 Andersen, William Niclaus. "Determination of a Spell- 
 ing Vocabulary Based on Written Correspondence," Uni- 
 versity of lozva Studies in Education, Vol. 2, No. 1. Iowa 
 City: University of Iowa, 1917. 66 p. 
 
 After a survey of existing word lists from writing vocabu- 
 laries, the reader is given a statement of the problem in Chapter 
 II under the caption "Aim." Four divisions are presented in 
 statement form as follows : 
 
 "The aim of the present study is (1) to supplement and 
 expand the work that has been done in the various studies, (2) 
 to show comparatively, by extensive tabulation, the number of 
 words that are common in the writing vocabularies of persons 
 in various callings, (3) to show the extent and range of vocabu- 
 laries used by persons in certain different callings, (4) to point 
 out the educational significance of these findings." 
 
 JuDD, Charles H. "Psychological Analysis of the Funda- 
 mentals of Arithmetic," Supplementary Educational 
 Monograph, No. 32. University of Chicago. Chicago: 
 University of Chicago, 1927. 121 p. 
 
 The reader is given a statement of the problem on page 15 
 under the caption, "purpose of the present study," in a series of 
 complete statements. 
 
 d. A general statement followed by subordinate statements. 
 
 Humphreys, Harry C. "The Factors Operating in the 
 Location of State Normal Schools," Teachers College, 
 Columbia University Contributions to Education, No. 
 142. New York: Bureau of Publications, Columbia Uni- 
 versity, 1923. 152 p. 1| 
 
 In the first chapter, which is entitled "The Problem Stated," 
 the reader is given a statement of three central problems : "The 
 
 i 
 
Techniques of Educational Research 19 
 
 central problems are: first, to ascertain the leading factors that 
 have operated in the locating of state normal schools and teach- 
 ers' colleges; secondly, to find and formulate guiding principles 
 for the location of such institutions ; thirdly, to propose certain 
 critical standards for the locating of normal schools. 
 
 "The minor problems involved in working out the central 
 problems are : first, a comparison of the normal schools of a 
 state as to enrollment, number of graduates, and other factors 
 as an indication of suitability of locations; secondly, a determi- 
 nation of the radii of enrollment of state normal schools in 
 certain states as evidence of poorly located and well located 
 normal schools." 
 
 Jones, Walter Benton. "Job Analysis and Curriculum 
 Construction in the Metal Trades Industry," Teachers 
 College, Columbia University Contributions to Educa- 
 tion, No. 227. New York : Bureau of Publications, Co- 
 lumbia University, 1926. 104 p. 
 
 In the opening sentence of the introductory chapter, the 
 reader is given a direct statement of the problem, followed by 
 subordinate statements. 
 
 Meek, Lois Hayden. "A Study of Learning and Retention 
 in Young Children," Teachers College, Columbia Uni- 
 versity Contributions to Education, No. 164. New^ York : 
 Bureaii of Publications, Columbia University, 1925. 96 p. 
 
 The reader is given a direct statement of the problem in 
 the opening sentence of the first paragraph. 
 
 3. Statement followed by restatement in the form of a question. 
 
 KoRNHAUSER, A. W. "The Economic Standing of Parents 
 and the Intelligence of Their Children," Journal of Edu- 
 cational Psychology, 9: 159-64, March, 1918. 
 
 In the second paragraph of part I, the reader is given a 
 statement of the problem as follows : "The purpose of the fol- 
 lowing inquiry was simply to attempt, by a brief survey, to find 
 whether any clear indication would appear of a correlation 
 between intelligence of children, measured crudely by school 
 advancement, and the economic standing of the parents, measured 
 still more crudely." After further discussion the problem is 
 stated again in the form of a question : "Do poorer parents on 
 the whole have less advanced children; do wealthier parents 
 have more advanced children?" 
 
 Daily, Benjamin W. "The Ability of High School Pupils 
 
 to Select Essential Data in Solving Problems," Teachers 
 
 College, Columbia University Contributions to Education, 
 
 No. 190. New York : Bureau of Publications, Columbia 
 
 University, 1925. 103 p. 
 
 The reader is given a statement of the problem in the first 
 sentence of Chapter I under the heading, "The Subject and 
 
20 Bulletin No. 38 
 
 Method of the Study." It is later stated again in the form of a 
 question. 
 
 YoAKAM, Gerald Alan. "The Effects of a Single Read- 
 ing," University of lozva Studies in Education, Vol. 2, 
 No. 7. Iowa City : University of Iowa, 1924. 100 p. 
 
 The reader is given a statement of the problem in the 
 opening sentence of Chapter I. At the conclusion of Chapter 
 III, the problem is restated in the form of a question. 
 
 4. A Statement followed by a series of theses. 
 
 Crow, Charles Sumner. "Evaluation of English Liter- 
 ature in the High School," Teachers College, Columbia 
 University Contributions to Education, No. 141. New 
 York : Bureau of Publications, Columbia University, 
 1924. 172 p. 
 
 The reader is given a statement of the problem in the first 
 paragraph of this study, under the heading, "Theses and Prob- 
 lems." It is: "To test and arrange the subject-matter in a given 
 field, English literature in the high school, in accordance with the 
 pupils' judgment of its values given in terms of ends that are 
 generally recognized as socially valuable." Following this state- 
 ment of the problem is a brief paragraph introducing the five 
 theses which are to be demonstrated. 
 
 B. Definition of the problem. 
 
 1. Analysis of the major problem or problems in terms of subordi- 
 nate problems. 
 
 Horn, Ernest. "Distribution of Opportunity for Partici- 
 pation among the Various Pupils in Classroom Recita- 
 tions," Teachers College, Columbia University Contribu- 
 tions to Education, No. 67. New York: Bureau of Pub- 
 lications, Columbia University, 1914. 40 p. 
 
 The problem is stated in the following sentence on page 1 : 
 "The purpose of this investigation is to discover the distribution 
 of opportunity for participation among the various pupils in 
 class-room recitations." It is more clearly defined by a resolu- 
 tion into eight subordinate problems, two of which are: "1. How 
 equally is the opportunitj' for participation distributed? 2. What 
 is the relation between the amount of reciting done and the 
 general all-round ability of the pupil?" 
 
 Ashbaugh, E. J. "Senior High-School English as Revealed 
 by a Standard Test," Journal of Educational Research, 
 13 :249-58, April, 1926. 
 
 The reader is given an analysis of the problem in terms 
 of subordinate problems stated in the form of questions on 
 page 257 in the last paragraph. 
 
Techniques of Educational Research 21 
 
 Smith, H. P. "The Business Administration of a City 
 
 School System," Teachers College, Columbia University 
 
 Contributions to Education, No. 197. New York : Bureau 
 
 of Publications, Columbia University, 1925. 129 p. 
 
 The problem is more clearly defined by an analysis into 
 three subordinate problems. This analysis is given on page 9 
 of the monograph. 
 
 2. Statement of the limits or scope of study. 
 
 Ayer, Adelaide M. "Some Difficulties in Elementary 
 
 School History," Teachers College, Columbia University 
 
 Contributions to Education, No. 212. New Yofk : Bureau 
 
 of Publications, Columbia University, 1926. 70 p. 
 
 The reader is given the limits of this study on page 4 under 
 the heading, "Grades Selected for This Study." The last sen- 
 tence of the first paragraph under this caption is the following: 
 "Hov^'ever, this thesis is confined specifically to a study of fifth 
 and seventh grade pupils' reactions to history textbook material." 
 The study is further limited to subject matter which was intended 
 for use in the fifth grade. The seventh grade was tested on this 
 same material. 
 
 Daily, Benjamin W. "The Ability of High School Pupils 
 
 to Select Essential Data in Solving Problems," Teachers 
 
 College, Columbia University Contributions to Education, 
 
 No. 190. New York : Bureau of Publications, Columbia 
 
 University, 1925. 103 p. 
 
 The purpose of this study is given in the first sentence of 
 the opening paragraph. Immediately following this the reader is 
 given four questions which state the scope of the study. 
 
 HuNSiCKER, Lilian May. "A Study of the Relationship 
 
 Between Rate and Ability," Teachers College, Columbia 
 
 University Contributions to Education, No. 185. New 
 
 York : Bureau of Publications, Columbia University, 
 
 1925. 52 p. 
 
 On page 15, after a statement of the assumptions upon 
 which the experiment is built, the reader is given a statement of 
 the limits of the study. 
 
 3. Orientation of the problem by means of : 
 
 a. A historical account, remote or recent. 
 
 HuNsiCKER, Lilian May. "A Study of the Relationship 
 Between Rate and Ability," Teachers College, Columbia 
 University Contributions -to Education, No. 185. New 
 
22 Bulletin No. 38 
 
 York : Bureau of Publications, Columbia University, 
 1925. 52 p. 
 
 In the first chapter of this study, the reader is given a 
 historical account of the problem which serves the purpose of 
 orientation. This historical account is limited to a cursory 
 review of representative types of investigations, and furnishes 
 the reader with a clear idea of the present status of the question. 
 The author is thus able to use this historical account as a point 
 of departure for his own study of the problem. On page 36 
 under the caption, "Background," a summary of the historical 
 survey appears. 
 
 Coy, Genevieve L. "The Interests, Abilities and Achieve- 
 ments of a Special Class for Gifted Children," Teachers 
 College, Columbia University Contributions to Education, 
 No. 131. New York: Bureau of Publications, Columbia 
 University, 1923. 191 p. 
 
 Following the statement of the problem on page 1, andj 
 under the caption, "Historical Summary of Educational Pro- 
 visions for the Gifted," the reader is given a length}^ discussionl 
 of the historical movement of special classes for gifted children." 
 
 b. A survey of previous studies or related studies. 
 
 Reeves, Charles Everand. "An Analysis of Janitor Serv- 
 ice in Elementary Schools," Teachers College, Columbia 
 University Contributions to Education, No. 167. New 
 York : Bureau of Publications, Columbia University, 
 1925. 194 p. 
 
 On pages 7 to 10 imder the heading, "Previous Studies of 
 School Janitor Service," the reader is given a survey of several 
 studies, and their results are compared and evaluated. The author 
 uses this survey of previous studies to show that research in 
 the field of his problem has been very meager. It also serves 
 the purpose of introducing the definition of the problem. 
 
 Reeder, Edwix Hewett. "A Method of Directing Chil- 
 dren's Study of Geography," Teachers College, Colum- 
 bia University Contributions to Education, No. 193. New 
 York : Bureau of Publications, Columbia University, 
 1925. 98 p. 
 
 Beginning with the second paragraph on page 1 and con- 
 tinuing through to the bottom of page 2, the reader is given a 
 survey of several studies in the field of elementary geography. 
 They are brieflj' described and evaluated. 
 
Techniques of Educational Research 23 
 
 c. An analysis of previous studies or related subjects. 
 
 BuTTERWECK, JosEPH S. "The Problem of Teaching High 
 School Pupils How to Study," Teachers College, Colum- 
 bia University Contributions to Education, No. 237. 
 New York : Bureau of Publications, Columbia Univer- 
 sity, 1926. 116 p. 
 
 Beginning with the last paragraph on page 6 and extending 
 to the middle of page 10, the reader is given a critical evaluation 
 of the studies of Hinsdale, Whipple, McMurray, Charters, and 
 others. This analysis leads up to a discussion of the proper 
 method to be used in approaching the problem. 
 
 d. Preliminary surve}*. 
 
 Powers, Samuel Ralph. "A Diagnostic Stud}^ of the Sub- 
 ject Matter of High School Chemistry," Teachers Col- 
 lege, Columbia University Contributions to Education, 
 No. 149. New York : Bureau of Publications, Columbia 
 University, 1924. 84 p. 
 
 On page 2, the reader is given a report of a brief survey 
 of a preliminary investigation made by the author. He uses this 
 means of introducing the hypothesis upon which he bases the 
 statement of his problem. 
 
 4. Description of the general nature of the problem. 
 
 a. Type. 
 
 Camp, Harold Laverne. "Scales for Measuring Results 
 
 of Physics Teaching," University of lozva Studies in 
 
 Education, Vol. 2, No. 2. Iowa City : University of Iowa, 
 
 1922. 51 p. 
 
 Under the caption, "Definition of the Problem," the reader 
 is given a description of its general nature. For instance, "to 
 formulate and select exercises, the correct handling of which 
 involves the above phases of ability." Some further idea of its 
 general nature may be found in the discussion following the 
 caption, "Methods of Evaluating Test Material," on page 6. 
 
 b. Source. 
 
 Hamilton, Otto Templar. "The Courts and the Curric- 
 ulum," Teachers College, Columbia University Contri- 
 butions to Education, No. 250. New York: Bureau of 
 Publications, Columbia University, 1927. 168 p. 
 
 The general nature of the problem is indicated by a dis- 
 cussion of the sources of data given on page 2. 
 
24 Bulletin No. 38 
 
 c. Procedure. 
 
 BuTTERWECK, JosEPH S. "The Problem of Teaching High 
 School Pupils How to Study," Teachers College, Colum- 
 bia University Contributions to Education, Xo. 237. 
 Xew York : Bureau of Publications, Columbia Univer- 
 sity, 1926. 116 p. 
 
 On page 18, near the end of a description of the experi- 
 mental set-up, the general nature of the problem is made clearer 
 to the reader by a concise statement of the procedure. The 
 procedure is described briefly as follows : "The experiment con- 
 sisted of the following: creating three groups of pupils as nearly 
 alike as possible in those factors which form the basis of school 
 success as conceived in the average school to-day ; permitting the 
 control group to continue its usual study procedure ; giving the 
 non-practice group a course in best methods of study; and sub- 
 jecting the practice group to a series of practice material in study 
 situations in which it was engaged throughout the year." 
 
 5. Statement of limitations of technique employed. 
 
 Anibel, Fred G. "Comparative Efifectiveness of the Lec- 
 ture-Demonstration and Individual Laboratory Method," 
 Journal of Educational Research, 13 :355-6S, ^lay, 1926. 
 
 The reader is given a limitation in the defining of the prob- 
 lem in the section entitled "Time Element" on pages 363 and 364 
 where the author says: "The primary object of the study being 
 to determine the comparative effectiveness of the laboratory in- 
 struction, there was no recording of the time element." 
 
 Crow, Charles Sumner. "Evaluation of English Liter- 
 ature in the High School," Teachers College, Columbia 
 University Contributions to Education, No. 141. New 
 York : Bureau of Publications, Columbia L'^niversity, 
 1924. 172 p. 
 
 On page 2, imder the caption, "Limitations of Study," the 
 reader is given several direct references to the limits of the 
 study. After stating that the purpose of the work is the defend- 
 ing of certain theses and the answering of certain questions, the 
 reader is further told : "There is no attempt to take account of 
 the method of teaching." 
 
 Teagarden, Florence M. "A Study of the Upper Limits 
 
 of the Development of Intelligence," Teachers College, 
 
 Columbia University Contributions to Education, No. 
 
 156. New York: Bureau of Publications, Columbia 
 
 University, 1924. 112 p. 
 
 At the beginning of the second chapter, the reader is given 
 a discussion of the limitations of the data, the nature of which 
 
 ll 
 
Techniques of Educational Research 25 
 
 is indicated by the following sentence : "As a preface to the dis- 
 cussion of the selection and use of tests as measures of intelli- 
 gence it should be said that perfection is not here claimed for 
 any of our present tools of measurement." 
 
 6. Recognition of assumptions and implications. 
 
 HuNSiCKER, Lilian May. "A Study of the Relationship 
 
 between Rate and Ability," Teachers College, Columbia 
 
 University Contributions to Education, No. 185. New 
 
 York : Bureau of Publications, Columbia University, 
 
 1925. 52 p. 
 
 In the first sentence of Chapter II, the reader is given a 
 direct statement of the problem. Following this the "two funda- 
 mental assumptions" upon which the experiment is built are 
 given. These are, "1. 'If a thing exists,, it must exist in some 
 amount; and if it exists in some amount, it can be measured.' 
 2. The relationship between rate and ability is an open question 
 of such importance that research should not stop short of a 
 solution that is substantiated by trustworthy, experimental evi- 
 dence." 
 
 7. Importance, value, or significance of study to education. 
 
 Willing, Matthew H. ''Valid Diagnosis in High School 
 
 Composition," Teachers College, Columbia University 
 
 Contributions to Education, No. 230. New York : Bureau 
 
 of Publications, Columbia University, 1926. 64 p. 
 
 The importance of the problem is indicated on page 2, in 
 the discussion as to its significance. 
 
 8. Definition of terms. 
 
 Hamilton, Otto Templar. "The Courts and the Curric- 
 ulum," Teachers College, Columbia University Contri= 
 butions to Education, No. 250. New York : Bureau of 
 Publications, Columbia University, 1927. 168 p. 
 
 Under the heading, "Definitions and Terms Used," the 
 reader is given definitions of some of the words and terms used 
 in the study. For example, the word "curriculum" is defined as 
 follows : "The word curriculum is used in this study as a general 
 term by which to designate all the subject matter of instruction 
 to which the child is exposed as a result of his school experi- 
 ence." 
 
 BIBLIOGRAPHY 
 
 Alexander, Carter. Educational Research. New York : Teachers 
 
 College, Columbia University, 1927. 41 p. A pamphlet. 
 
 After discussing the fields in which investigators will institute in- 
 quiries, the procedures to be used in defining a problem are suggested. 
 
26 Bulletin No. 38 
 
 These relate to such items as originality, value, and sufficient limitation to 
 permit exhaustive treatment. 
 
 Bay, James C. "The Training of a Research Worker in Education," 
 
 School and Society, 25 :274-78, March, 1927, 
 
 The process of educational research gets under way with the formula- 
 tion of a definite problem. The delimiting of the subject establishes "a 
 criterion for judging the pertinency of data." 
 
 Kelly, F. J. "Scientific Method in College Administration and Col- 
 lege Teaching," School and Society, 20 :390-96, September, 1924. 
 
 This is an exceptionally sane aiticle. It reflects a sound conception 
 of the nature of educational research and the service which it can render. 
 In the first two divisions of the article, the necessity of a clear analysis of 
 the problem is stressed. This analysis should be made before the investi- 
 gator begins to gather data. 
 
 McCall, William A. Hoiv to Experiment in Education. New York : 
 The Macmillan Company, 1923, p. 9-11. 
 
 The reader is given a brief discussion of the technique to be used 
 in the formulation of a problem. The scope of the problem should be 
 neither too broad nor too nairow. It is desirable that specific problems be 
 formulated which contribute to larger investigations. 
 
 Trow, William C. Scientific Method in Education. Boston : Hough- 
 ton ^lifflin Company, 1925, p. 38-42. 
 
 Seeing a problem is the starting point of research. The author com- 
 plains that graduate students do not see problems themselves. "So they 
 confer with some member of the graduate faculty and ask him for a 
 problem to work on; they can't seem to think of any. Can't think of any, 
 and the world fairly bursting with them." 
 
CHAPTER III 
 
 COLLECTING DATA 
 
 What are data? Data include all concepts, facts, and principles 
 used in thinking out the answers to thought questions. In other words, 
 data are the things we use in thinking. The meaning of these state- 
 ments will be more apparent if we note some of the types of data : 
 
 1. Ages of school children 
 
 2. Scores made on standardized tests 
 
 3. School marks, and other facts from school records 
 
 4. Answers to questionnaires 
 
 5. Expenditures for educational purposes 
 
 6. Assessed values and tax rates 
 
 7. Measures of school buildings 
 
 8. Statements of beliefs or opinions 
 
 9. School laws 
 
 10. Counts of things such as school children, number of pages, 
 
 etc. 
 
 11. Principles 
 
 12. Descriptions of schools, events, etc. 
 
 13. Observations 
 
 14. Historical information 
 
 15. Errors in children's compositions 
 
 16. Stenographic reports of lessons 
 
 The data to be collected specified by the problem. When a prob- 
 lem has been adequately defined, the data needed for its solution are 
 clearly indicated. Hence "collecting data" does not mean bringing 
 together whatever data may be easily accessible. Instead, it means 
 collecting the data specified by the problem.^ 
 
 Objective data versus subjective data. When data are such that 
 there has been very little or no opportunity for them to be affected by 
 the person collecting them, they are described as "objective." The 
 term "subjective data" has the opposite meaning; that is, they are data 
 of such a character that they might be affected by the prejudices, 
 opinions, and judgment of the person collecting them. 
 
 Methods of collecting data. It is apparent from the above list of 
 types of data that many different procedures are employed in collect- 
 ing data. In general, each type requires unique techniques, and within 
 each type the required techniques may vary according to the problem. 
 
 ^See the reference by James C. Bay, page 26. 
 
 [27] 
 
28 Bulletin No. 38 
 
 Compiling a bibliography, a phase of collecting data. The com- 
 piling of a bibliography, which is a phase of collecting data, is seldom 
 described in a report of educational research but the general method 
 to be employed has been described in several places. - 
 
 ILLUSTRATIVE REFERENCES 
 
 A. Subjective data. 
 
 1. Formulation of criteria to be used as a basis for estimates. 
 
 LooMis, Arthur Kirkwood. "The Technique of Esti- 
 mating School Equipment Costs," Teachers College, 
 Columbia University Contributions to Education, No. 
 208. New York : Bureau of Publications, Columbia Uni- 
 versity, 1926. 112 p. 
 
 Immediately after the statement of the purpose of the 
 problem on page 1, the caption, "Criteria," appears. Under this 
 caption, the reader finds that four criteria are listed and briefly 
 explained. These are : "Reliability, Adequacy, Flexibility, and 
 
 Simplicity." 
 
 Nanxixga, S. p. "A Critical Study of Rating Traits," 
 Educational Administration and Supervision, 12:114-19, 
 February, 1926. 
 
 On page 114, the reader is given a list of fifteen criteria 
 which are to be used by teachers when rating pupils. 
 
 2. Use of criteria. 
 
 Kexxox, Laura Hall Vere. '"Tests for Literar}- Vocabu- 
 lary for Teachers of English," Teachers College, Colum- 
 bia University Contributions to Education, No. 223. New 
 York : Bureau of Publications, Columbia University, 
 1926, 78 p. 
 
 The two literary vocabular}' tests presented to the reader in 
 this study were made up of two hundred words drawn largely 
 from the special fields of English literature. Under the sub- 
 topic, "The Basis of Choice," the reader is told that "Certain 
 considerations governed the choice of words which were con- 
 sidered suitable for literary word knowledge tests." The first 
 two of these considerations may be given as follows: "1. Occur- 
 rence in a supposedly familiar or famous passage of English 
 prose or poetr}'. 2. Occurrence in prose or poetry of a certain 
 historical period included in the special field of English liter- 
 ature." The application of these criteria is illustrated on page 15. 
 
 ^Monroe, Walter S., and Asher, Ollie. "A Bibliography of Bibliographies," University 
 of Illinois Bulletin, Vol. 24, Xo. 44, Bureau of Educational Research Bulletin Xo. 36. Urbana: 
 TJniversity of Illinois, 1927. 60 p. 
 
Techniques of Educational Research 29 
 
 B. Objective data. 
 
 1. Techniques employed in collecting. 
 a. Analysis. 
 
 (1) Analysis of textbooks. 
 
 Strang, Ruth. "Subject IMatter in Health Education," 
 
 Teachers College, Columbia University Contributions to 
 
 Education, No. 222. New York: Bureau of Publications, 
 
 Columbia University, 1926. 108 p. 
 
 The reader is given a report of the resuhs of a detailed 
 analysis of the contents of a number of courses of study and 
 textbooks dealing with health. Under the caption, "Methods of 
 Work," the general plan of analysis is described. 
 
 Lively, Bertha A. and Pressey, S. L. "A Method for 
 Measuring the 'Vocabulary Burden' of Textbooks," Edu- 
 cational Administration and Supervision, 9 :389-98, Oc- 
 tober, 1923. 
 
 The avUhor describes, in the first paragraph on page 390, 
 the method used in making an analysis of the vocabulary of 
 textbooks. Information is given of the way of distributing the 
 sampling in order to cover a large number of books. 
 
 Monroe, Walter S. and Clark, John A. "The Teacher's 
 Responsibility for Devising Learning Exercises in Arith- 
 metic," University of Illinois Bidletin, Vol. 23, No. 41, 
 Bureau of Educational Research Bulletin No. 31. L^r- 
 bana : University of Illinois, 1926. 92 p. 
 
 The technique used in the analysis of arithmetic textbooks 
 is described in Chapter IV. The problems, of certain texts were 
 classified according to tj^pe. The symbols used to facilitate the 
 classification are given in the appendix. 
 
 (2) Analysis of pupil performances. 
 
 TouTON, Frank Charles. "Solving Geometric Originals," 
 
 Teachers College, Columbia University Contributions to 
 
 Education, No. 146. New York : Bureau of Publications, 
 
 Columbia University, 1924. 114 p. 
 
 This is a report of an analysis of 2800 examination papers. 
 On page 4 under the heading, "Analysis of Pupils' Work," the 
 reader is told how the analysis was made. 
 
 (3) Analysis of records. 
 
 Carrother.s, George E. "The Physical Efficiency of Teach- 
 ers," Teachers College, Columbia University Contribu- 
 
30 Bulletin No. 38 
 
 tions to Education, No. 155. New York: Bureau of 
 
 Publications, Columbia University, 1924. 78 p. 
 
 School records and reports constitute the source of data in 
 this study. The tables given are based on an analysis of these 
 records, and all conclusions are drawn from them. 
 
 OjEMANN, Ralph H. "The Constant and Variable Occu- 
 pations in the United States in 1920," University of 
 Illinois Bulletin, Vol. 24, No. 39, Bureau of Educational 
 Research Bulletin No. 35. Urbana : University of Illi- 
 nois, 1927. 47 p. 
 
 Census reports were used as one of the sources of data in 
 this investigation. Information is given on page 10 concerning 
 the procedure used in the collection of pertinent facts. 
 
 b. Experimental procedures. 
 
 ( 1 ) One group method. 
 
 Helseth, Inga O. "Children's Thinking," Teachers Col- 
 lege, Columbia University Contributions to Education, 
 No. 209. New York : Bureau of Publications, Columbia 
 University, 1926. 163 p. 
 
 The experiment was a project in U. S. history conducted 
 with one group of sixteen pupils, two boys and fourteen girls, 
 classified as seventh and eighth grade. The technique employed 
 and the conditions involved in the investigation are stated under 
 the caption, "The Conditions and Records of the Investigation," 
 pages 5-9. 
 
 Click, H. N. "The Effect of Practice on Intelligence 
 Tests," University of Illinois Bulletin, Vol. 23, No. 3, 
 Bureau of Educational Research Bulletin No. 27. Ur- 
 bana : University of Illinois, 1925. 23 p. 
 
 On page 9, the reader is given information pertaining to the 
 general plan of the experiment. It is stated that care was taken 
 to keep testing conditions constant for the several groups. The 
 use of the same procedure on the various groups makes this a 
 "one group method." 
 
 (2) Equivalent group method. 
 
 Obtaining equivalent groups is one of the most important 
 techniques of experimentation. Two general methods are 
 employed. Frequently, an investigator selects the groups by 
 means of random sampling or in some other way that justi- 
 fies the expectation that approximate equivalents will be se- 
 cured. After the selection has been made, information in 
 
Techniques of Educational Research 31 
 
 regard to the groups is sought and if it is found that approxi- 
 mate equivalents have not been secured, the investigator 
 makes a re-determination of the groups or allows for the 
 non-equivalents in interpreting his data. According to the 
 second method, the investigator collects data in regard to 
 the characteristics of the pupils and then builds up his group 
 by pairing ofif equivalent pupils so that equivalent groups 
 will be obtained. The reference by Odell illustrates the 
 first method ; that by Barton, the second. 
 
 Odell, Charles W. "The Use of Intelligence Tests as a 
 Basis of School Organization and Instruction," Univer- 
 sity of Illinois Bulletin, Vol. 20, No. 17, Bureau of Edu- 
 cational Research Bulletin No. 12. Urbana : University 
 of Illinois, 1922. 78 p. 
 
 The method of obtaining equivalents used in this study is 
 indicated by the following sentences from page 8 : "These schools 
 were divided into an experimental and a control group of four 
 each by Assistant Superintendent A. B. Wright. In making this 
 selection, Mr. Wright endeavored to choose two groups of 
 schools in which the investment factors should be approximately 
 equal at the beginning of the experiment." Dr. Odell later de- 
 termined accurately the differences existing between the experi- 
 mental and control schools, and he states that the differences 
 discovered "were not large enough to invalidate the use of the 
 two groups of schools in the experiment." 
 
 Barton, W. A., Jr. "The Effect of Group Activity and 
 Individual Effort in Developing Ability to Solve Prob- 
 lems in First Year Algebra," Educational Administration 
 and Supervision, 12: 512-18, November, 1926. 
 
 Under the caption, "Experimental Procedure," the reader is 
 given a description of the method of obtaining two equivalent 
 groups. The basis of equating used in this experiment was that 
 of chronological age and I. Q. 
 
 Anibel, Fred G. "Comparative Effectiveness of the Lec- 
 ture-Demonstration and Individual Laboratory Method," 
 Journal of Educational Research, 13 :355-65, May, 1926. 
 
 The reader is given considerable information pertaining to 
 the methods of obtaining equivalent groups in this investigation. 
 On page 355 it is stated : "Two classes were selected each year as 
 groups for the experiment. ... In order to eliminate other 
 factors which might influence results, the two classes were bal- 
 anced as nearly as possible on the basis of ability as determined 
 by standardized intelligence tests." Further information is given 
 on page 366 : "Some students were shifted from one group to 
 the other in order that the groups might be about equal in intel- 
 ligence. After adjustment there were thirty pairs available for 
 the experiment." 
 
32 Bulletin" Xo. 3S 
 
 BuTTERWECK, JosEPH S. "The Froblem of Teaching High 
 School Pupils How to Study," Teachers College, Colum- 
 bia University Contributions to Education, Xo. 237. New 
 York: Bureau of Publications, Columbia University. 
 1926. 116 p. 
 
 In this experiment, the reader is told that three groups of 
 high school pupils were used. Group A constituted the practice 
 group, Group B the non-practice group, and Group C the control 
 group. The method of selecting and equating these groups is 
 fulh' described to the reader in the third chapter under the 
 caption, "Experimental Set-up." 
 
 (3) Rotation method. The rotation method is a device for 
 neutralizing the etTect of unequalized factors when two 
 
 or more groups are involved in an experiment. 
 
 ^loxROE, Walter S. "Relation of Sectioning a Class to the 
 Effectiveness of Instruction," University of Illinois Bul- 
 letin, \'o\. 20. Xo. 11. Bureau of Educational Research 
 Bulletin Xo. 11. Urbana : Universitv of Illinois. 1922. 
 IS p. 
 
 On page 6, the reader is informed that two difficulties are 
 avoided through the use of a rotation method. The first of these 
 ditliculties is that of securing equivalence between groups, and 
 the second that of securing constancy of instruction. The rota- 
 tion technique employed in this investigation is described on 
 pages 5-7. 
 
 c. Historical. 
 
 Xoble, Stuart Graysox. "Forty Years of the Public 
 Schools in Mississippi," Teachers College, Columbia Uni- 
 versity Contributions to Education, X'o. 94. Xew York : 
 
 Bureau of Publications. Columbia University. 1918. 142 p. 
 
 In the preface, the reader is told, "I have been forced to 
 draw my conclusions largeh" from data contained in the state 
 records, in the government reports, and in a limited number of 
 local newspapers." To support the conclusions the reader is often 
 given quotations of brief extracts from original sources. See 
 page 7 for several examples. 
 
 Reigart, John Fraxklix. "The Lancasterian System of 
 Instruction in the Schools of Xew York Cit}'," Teachers 
 College, Columbia University Co>itributions to Education, 
 Xo. 81. Xew York: Bureau of Publications, Columbia 
 University, 1916. 105 p. 
 
 Although the reader is not explicitlj- told the technique" 
 which is employed, a perusal of the opening paragraphs of the 
 
 I 
 
Techniques of Euucatioxal Research 33 
 
 study clearly reveals the fact that the technique of the research 
 may be classed under the caption, historical. The sources of the 
 material used in the discussion are letters, histories, reports, 
 journals, addresses, and some current periodicals. Numerous 
 citations from these sources are included. 
 
 (1. Interview. 
 
 Charters, Jes.sie Allex. "Methods of Study Used b}- Col- 
 lege Women,"' Journal of Educational Research, 10 :344- 
 55, December, 1924. 
 
 On page 345, the reader is told that before attempting to 
 collect the data used in this investigation, "A preliminary survey 
 was made of a group of kindly persons, . . . and by this 
 means we developed a technique for interviewing." In the next 
 two paragraphs, the manner of using this technique is explained 
 and a list of the types of subjects interviewed is given. The first 
 sentence in this explanation is the following: "Then fortj^-four 
 college women were interviewed for fort>^-five minutes each on 
 the average." On page 346 under the heading, "Methods of 
 investigation," the author describes the use made of the data 
 secured by means of the interviews. 
 
 RuFi, John. '"The Small High School," Teachers College, 
 
 Columbia University Contributions to Education, Xo. 
 
 236. Xew York : Bureau of Publications, Columbia 
 
 University, 1926. 145 p. 
 
 On page 10, under the caption, "Methods of Carrying Out 
 the Stud}'," the reader is given a list of several methods of col- 
 lecting data, the most prominent of which are b\- personal inter- 
 view with teachers and pupils. 
 
 e. Legal. 
 
 (1) Statutes. 
 
 Powers, J. Orix. '"Legal Provisions and Regulations of 
 
 State Departments of Education Affecting Junior High 
 
 Schools," School Review, 3:280-291, April, 1925. 
 
 The use of statutes as data is indicated by the author on 
 page 280: "The following publications were examined: (1) the 
 school codes of various states, including all references to high 
 schools, particularly those provisions which applj- to junior high 
 schools — ." 
 
 (2) Decisions. 
 
 Hamiltox, Otto Templar. '"The Courts and the Curric- 
 ulum," Teachers College, Columbia University Contri- 
 
34 Bulletin Xo. 38 
 
 hutions to Education, Xo. 250. Xew York: Bureau of 
 
 Publications, Columbia University, 1927. 168 p. 
 
 Man}' citations from the laws and court proceedings are 
 included in the stud}'. For example, on page 11a quotation from 
 "The Illinois Supreme Court in 1881" is found defining the 
 term "common school." 
 
 f. Questionnaire. 
 
 ( 1 ) Opinion. 
 
 Stuart, Hugh. "The Training of Modern Foreign Lang- 
 uage Teachers for the Secondary Schools in the United 
 States," Teachers College, Cohcmbia University Contri- 
 butions to Education, No. 256. New York : Bureavi of 
 Publications, Columbia University, 1927. Ill p. 
 
 The questionnaire method was used in collecting the data 
 for this study. In Chapter I, under the heading "Collection of 
 Data," the reader is informed that the first questionnaire con- 
 sisted of one hundred and eighteen questions covering all phases 
 of the subject. The following examples taken from the question- 
 naire illustrate requests for opinions: "Which subjects, in your 
 opinion, are pedagogically most suitable as minors with a teach- 
 ing major in F. G. I. S?" and "What textbooks have you found 
 best adapted to your courses on phonetics?" The questionnaires 
 are given in the appendix. 
 
 MoxROE, Walter S,., and Herriott, M. E. "Objectives of 
 United States History in Grades Seven and Eight," 
 University of Illinois Bidletin, Vol. 23, No. 3, Bureau of 
 Educational Research Bulletin No. 33. Urbana : Univer- 
 sity of Illinois, 1926. 68 p. 
 
 The questionnaire used in this investigation is given on 
 page 9. The necessity of using judgment or opinion in giving 
 the information desired is illustrated by the following sentence 
 taken from the questionnaire: "V. Give below 15 thought ques- 
 tions in United States history which you think children complet- 
 ing the seventh and eighth grades should be able to answer." 
 
 (2) Facts. 
 
 Larson, E. L. "One-Room and Consolidated Schools of 
 Connecticut," Teachers College, Columbia University 
 Contributions to Education, No. 182. New York : Bu- 
 reau of Publications, Columbia University, 1925. 67 p. 
 
 One portion of the data used in this study was obtained by 
 means of a questionnaire sent to teachers of one-room and con- 
 solidated schools of Connecticut. The nature of the facts ob- 
 tained is indicated by the following questions : "What is vour 
 
Techniques of Educational Research 35 
 
 salary for the 3'ear 1923-24?" and "What opportunities for social 
 life do you have?" 
 
 Reavis, George H. "Factors Controlling Attendance in 
 Rural Schools," Teachers College, Columbia University 
 Contributions to Education, No. 108. New York : Bureau 
 of Publications, Columbia Universit}', 1920. 69 p. 
 
 The reader is told that the facts concerning attendance in 
 this study were gathered from the questionnaire sent to the 
 teachers of one-teacher country schools concerning 6450 pupils 
 of five counties of Maryland. The report sought information 
 about the child, the school, and the community. The question- 
 naire and facts considered are given on pages 26-35. 
 
 Odell, Charles \V. "The Progress and Elimination of 
 School Children in Illinois," University of Illinois Bul- 
 letin, Vol. 21, No. 38, Bureau of Educational Research 
 Bulletin No. 19. Urbana : University of Illinois, 1924. 
 76 p. 
 
 One of the procedures employed in the collection of data 
 for this investigation was the use of a questionnaire. The 
 recipient of the questionnaire was asked to place certain facts in 
 the blanks provided. The questionnaire is reproduced" on page 
 70 of the bulletin. 
 
 g. Survey. 
 
 BuRXHAM, Ernest. 'Two Types of Rural Schools," Teach- 
 ers College, Columbia University Contributions to Edu- 
 cation, No. 51. New York: Bureau of Publications, Co- 
 lumbia University, 1912. 129 p. 
 
 One of the methods employed in collecting the data for this 
 study is implied in the paragraph entitled, "The Survey Idea." 
 The reader is told that "The survey idea, the taking of stock of 
 existing conditions in itemized details, is growing rapidly into 
 wide application in various fields of human activity. This idea 
 seems well adapted for use in a scientific study of education." 
 
 Emmons, F. E. "City School Attendance Service," Teach- 
 ers College, Columbia University Contributions to Edu- 
 cation, No. 200. New York: Bureau of Publications, 
 Columbia University, 1926. 173 p. 
 
 On page 9 is given the source of data, which consists of the 
 1923-24 attendance reports of fifty cities. 
 
 Smith, H. P. "The Business Administration of a City 
 School System," Teachers College, Columbia University 
 
36 Bulletin No. 38 
 
 Contributions to Education, No. 197. New York: Bureau 
 
 of Publications, Columbia University, 1925. 129 p. 
 
 Under the heading, "Source of Data," on page 8 is given a 
 description of a survey made of business activities and school 
 costs in each of twentj^-five cities. 
 
 h. Test construction. 
 
 (1) Scale. 
 
 HoTZ, Henry Gustave. "First Year Algebra Scales," Teach- 
 ers College, Columbia University Contributions to Edu- 
 cation, No. 90. New York : Bureau of Publications, 
 Columbia University, 1918. 87 p. 
 
 In Chapter II, the reader is given information pertaining to 
 the collection of data upon which the scales were based. The 
 author states that a set of preliminary tests were given to de- 
 termine which exercises were of most value as test material and 
 also to determine roughly the order of difficulty. With this 
 information at hand it was possible to construct the scales. 
 
 (2) Tests. 
 
 BuswELL, G. T. "Diagnostic Studies in Arithmetic," Uni- 
 versity of Chicago Supplementary EducatioJial Mono- 
 graph, No. 30. Chicago : University of Chicago Press, 
 1926. 212 p. 
 
 The method of formulating the tests used in this study is 
 explained in detail from pages 88 to 100. The examples used 
 "were constructed after a logical analysis had been made of the 
 various factors which enter into the successive steps in the four 
 fundamental processes." 
 
 i. Observation. 
 
 Helseth, Inga O. "Children's Thinking," Teachers College, 
 Columbia University Contributions to Education, No. 
 209. New York : Bureau of Publications, Columbia Uni- 
 versity, 1926. 163 p. 
 
 A brief discussion of the method of observation is to be 
 found on page 6 : "One or more observers and the critic teacher 
 were present regularly to take notes on what was said and done 
 during the class hour." A description of the manner of keeping 
 the records then follows. A verbatim report of all of the daily 
 records is included in the appendix of this study. 
 
 Symonds, Percival AI. "Study Habits of High School 
 
 Pupils," Teachers College Record, 27:713-24, April, 
 
 1926. 
 
 On page 714, the author gives an account of his general 
 plan of observation. The following are two of the steps in the 
 
Techniques of Educational Research 37 
 
 procedure : "Careful and undivided attention was given to the 
 behavior of a single boy during a period . . . Every act of 
 the boy was entered in a notebook." 
 
 Stevens, Romiett. "The Question as a Measure of Effi- 
 ciency in Instruction," Teachers College, Columbia Uni- 
 versity Contributions to Education, No. 48. New York : 
 Bureau of Publications, Columbia University, 1912. 95 p. 
 
 The procedure used in this study is that of observation, 
 analj^sis, and criticism of actual questions in the class room. For 
 this purpose a stenographic record of twenty lessons was secured. 
 
 2. Forms in which used. 
 
 a. Raw. 
 
 Monroe, Walter S. "The Duties of Men Engaged as 
 
 Physical Directors or Athletic Coaches in High Schools," 
 
 University of Illinois Bulletin, Vol. 23, No. 38, Bureau 
 
 of Educational Research Bulletin No. 30. Urbana : Uni- 
 
 versit}- of Illinois, 1926. 22 p. 
 
 This investigation, although of the fact finding type, is a 
 good example of the use of raw data. Various tables are given 
 in which the information presented is summarized. The reader 
 will note that no statistical procedures have been employed other 
 than that of tabulation, the determination of per cents, and the 
 computing of totals. 
 
 Whipple, Guy M. "Sex Differences in Army Alpha Scores 
 in the Secondary School," Journal of Educational Re- 
 search, 15:269-275, April, 1927. 
 
 The reader is given information relating to the type of 
 data used in this investigation on page 269. "The method of 
 analyzing this material was simply to make numerical and 
 graphic distributions of the scores of each of these four groups 
 for each of the eight tests of Army Alpha, and, of course, for 
 the total score of the examination." 
 
 b. Transmuted. 
 
 Odell, Charles W. "The Use of Intelligence Tests as a 
 Basis for vSchool Organization and Instruction," Univer- 
 sity of Illinois Bidletin, Vol. 20, No. 17, Bureau of Edu- 
 cational Research Bulletin No. 12. Urbana : University 
 of Illinois, 1922. 78 p. 
 
 The conversion of raw data into derived is indicated by the 
 following sentence from page 10 : "The scores made upon these 
 tests were translated into mental or achievement ages, as the 
 case might be, and then further into intelligence and achieve- 
 ment quotients." 
 
38 Bulletin No. 38 
 
 Ross, Clay Campbell. "The Relation Between Grade 
 School Record and High School Achievement," Teachers 
 College, Columbia University Contributions to Education, 
 No. 166. New York : Bureau of Publications, Columbia 
 University, 1925. 70 p. 
 
 The use of transmuted data is indicated by the following 
 sentence from page 11 : "Grade-progress and age at completing 
 the eighth grade, calculated in half-years to nearest birthda}', 
 were originally entered in actual gross scores, but were later 
 converted into transmuted scores according to the tables given 
 below." On page 14 the reader is given the method of trans- 
 muting letter grades into numerical ratings. 
 
 Fleming, Cecile White. "A Detailed Analysis of Achieve- 
 ment in the High School," Teachers College, Columbia 
 University Contributions to Education, No. 196. New 
 York: Bureau of Publications, Columbia University, 
 1925. 209 p. 
 
 Extensive use is made of derived data in this study. For 
 example, it was found necessary to transmute letter marks (A, B, 
 C, D, and E) into numerical values. The reader is given a good 
 discussion of the techniques employed in the third chapter en- 
 titled, "Accumulation and Treatment of Data." 
 
 BIBLIOGRAPHY 
 
 Alexander, Carter. "Opportunities for Research in Educational 
 
 Finance," Educational Administration and Supervision, 9 : 209-222, 
 
 April, 1923. Also in The Phi Delta Kappan, 5 : 5-8, April, 1923. 
 
 The author stresses the value of existing data by saying, "Many of 
 the biggest contributions in education have been made by men willing to 
 work with already recorded data." The point made by Alexander is a 
 good one, although the use of existing data may be over emphasized in 
 many cases. Frequently such data is so inaccurate and so incomplete that 
 one does not get adequate returns for his investment. 
 
 Alexander, Carter. School Statistics and Publicity. Boston : Silver, 
 
 Burdett and Company, 1919, p. 58-71. 
 
 The reader is given a brief discussion of the methods to be used in 
 the actual collecting of data.. Certain information concerning sampling will 
 be found to be very helpful. 
 
 Alexander, Carter. "How to Equip the School Administrator with 
 
 Methods of Locating Data which will Carry Over when He is on 
 
 the Job," Teachers College Record, 28 : 890-99, May, 1927. 
 
 In this article, the reader is given an analysis and a description of a 
 general technique for gathering data of an administrative character. It 
 includes a critical discussion of the "school survev" method on pages 
 896-899. 
 
Techniques of Educational Research 39 
 
 Alexander, Carter. Educational Research. New York : Teachers 
 College, Columbia University, 1927. 41 p. A pamphlet. 
 
 Six criteria are given on page 3 for the evaluation of data. Two of 
 them are as follows : "Must be reliable enough to warrant basing conclu- 
 sions upon them," — "Must be as objective as the field of study permits." 
 
 Buckingham, B. R. "Accuracy," Journal of Educational Research, 
 8 : 63-64, June, 1923. An editorial. 
 
 This consists of a discussion of the need for accuracy in the conduc- 
 tion of an investigation. It also tells what accuracy includes : defining 
 terms, simplifying procedure, and giving statements of reliability. 
 
 BuRRis, W. P. "The Case Method for the Study of Teaching," School 
 and Society, 15 : 121-130, February 4, 1922. 
 
 This is an argument for the utilization of the case method in educa- 
 tional research. "The method is scientific because it is an objective and 
 inductive study of the thing itself, and loses none of its scientific character 
 because based upon records any more than the study of the data of a 
 laboratory notebook ceases to be scientific when it contains a record of 
 observed phenomena." 
 
 BuTTERFiELD, Ernest W. "Educatioual Surveys," Educational Re- 
 view, 68: 1-5, June, 1924. 
 
 This article is critical of school surveys. The point is made that a 
 general survey interferes with the regular work of the school. The writer 
 appears to believe that such surveys are usually conducted by people who 
 like to meddle in school affairs. 
 
 Butterfield, Ernest W. "The Plenary Inspiration of the Dotted 
 Line," Educational Reviezv, 69 : 1-4, January, 1925. 
 
 The research worker about to employ the questionnaire method in 
 the collection of data should be sure to read this article. It will engender 
 in him a very worthwhile critical attitude of questionnaires and their use. 
 
 Fling, Fred Morrow. The Writing of History. New Haven: Yale 
 University Press, 1920. 184 p. 
 
 The research investigator who makes use of historical data will find 
 this book of service. The statement is made that the historian does not 
 deal with fact but with the "residue" of fact. Such sources require careful 
 criticism in order to establish their validity. Rules are given for the testing 
 of evidence, and the establishment of certainty. The book ends with sug- 
 gestions as to writing of history. 
 
 Good, Carter V. "Careless Practices in Educational Writings," School 
 and Society, 23 : 684-686, May, 1926. 
 
 An argument for careful organization and reporting of bibliograph- 
 ical data. The author stresses the importance of good form, and gives ref- 
 erences to certain manuals of writing. 
 
40 Bulletin Xo. 38 
 
 JMcCall, William A. Hoiv to Experiment in Education. New York: 
 
 The ^lacmillan Company, 1923, p. 214-217. 
 
 Several questions are stated to guide the investigator in the formula- 
 tion of a questionnaire. For example: "Are the questions unambiguous?" 
 The different types of questionnaires which may be used to measure degree 
 of preference are explained. 
 
 MoxROE, Walter S. and Asher, Ollie. "A Bibliography of Bibli- 
 ographies." University of Illinois Bulletin, Vol. 24, No. 44, Bu- 
 reau of Educational Research Bulletin No. 36. Urbana : Univer- 
 sity of Illinois, 1925. 60 p. 
 
 The research worker in education will find this bulletin invaluable 
 when compiling a bibliography for his investigation. 
 
 Park, Robert E. and Burgess, Earnest W. Introduction to the 
 Science of Sociology. Chicago : University of Chicago Press, 
 1921, p. 43-60. 
 
 The reader is given a brief discussion of the nature of social problems 
 and social research. A bibliography on page 58 gives several references to 
 methods of sociological investigation. 
 
 RuGG, Harold O. Statistical Methods Applied to Education. New 
 York: Houghton Mifflin and Compan}-, 1917, Chapter II. 
 
 The chapter begins with an outline of the important sources of origi- 
 nal educational data. This is followed by a discussion of the reliability 
 of these sotxrces. The chapter ends with an explanation of the method to 
 be used in formulating a questionnaire. 
 
 Sears, Jesse B. The School Survey. Boston: Houghton Mifflin Com- 
 pany, 1925. 427 p. 
 
 This is a rather exhaustive treatment of the technique of school 
 survej'S. It is stated that many of its procedures are those of educational 
 research. For this reason the book should be of service to those engaged 
 in other types of research, as well as to those about to employ the survey 
 method. Survey is defined as "a scientific inquiry which obtains facts 
 about the present efficiency of the school S3"stem, and on the basis of these 
 facts offers constructive proposals for desirable improvements." 
 
 Sturtevant, Sarah ]\I, and Hayes, Harriet. "The Use of the Inter- 
 view in Advisory Work," Teachers College Record, 28:551-62. 
 February, 1927. 
 
 This article presents to the reader a description of the factors involved 
 in the interview, the limitations of the interview, method of successful 
 interviews, and includes a selected bibliography of the uses of the technique 
 of interviewing. The reader will find this description of the technique 
 of interviewing very helpful. 
 
 Thorxdike, Edward L. Educational Psychology, Vol. I. New York : 
 
 Teachers College, Columbia University, 1913, p. 28-42. 
 
 Information is given pertaining to several means of collecting infor- 
 mation relative to original tendencies. Sj'Stematic observation, concensus 
 
Techniques of Educational Research 41 
 
 of opinion, and the use of the personal judgment of the author are dis- 
 cussed and evahiated. 
 
 Toops, Herbert A. "Validating the Questionnaire Method," Journal 
 
 of Personnel Research, 2: 153-169, August, 1923. 
 
 The author gives reasons for the disrepute of the questionnaire 
 method. The first objection is that only a fraction of the questionnaires 
 are returned, and these may not represent the group. The article describes 
 a method for overcoming this objection. 
 
 Wylie, Andrew T. "To What Extent May We Rely Upon the Answers 
 to a School Questionnaire ?" The Journal of Educational Method, 
 6:252-257, February, 1927. 
 
 The validity or truthfulness of the answers of a questionnaire was 
 determined by means of other sources of information. The conclusions are 
 drawn that the answers of any one individual are not to be given too great 
 credence, the net total result has considerable validity, and the percentage 
 of correct answers on the whole is high. Rules are given for the formu- 
 lation and use of questionnaires. 
 
CHAPTER IV 
 ANALYZING, ORGANIZING, AND SUMMARIZING DATA 
 
 Relation of analyzing and organizing data to collecting them. 
 The analysis and organization of data depend upon the form in which 
 they were collected. For example, when data are taken from records 
 or printed sources, their organization may be facihtated or made more 
 difficult by the form in which they were copied. Hence this phase of 
 educational research overlaps that described in the preceding chapter. 
 
 Decisions to be made. The phase of educational research de- 
 scribed as "analyzing, organizing, and summarizing data" may make 
 great demands upon the investigator's ingenuity. There are many 
 decisions to be made, especially in summarizing data. Typical ques- 
 tions are: What form of table is to be made? How large intervals 
 should be used? How may the data be summarized without obscuring 
 significant details? Shall one large complex table be constructed, or a 
 group of smaller tables? 
 
 Statistical methods. Much of the handling of quantitative data 
 is accomplished by standard statistical methods. Since these methods 
 are described and illustrated in texts on statistics, no attempt has been 
 made to illustrate them in the following references. However, before 
 an average or coefficient of correlation is calculated, the investigator 
 must make decisions. In some instances these are very simple, but 
 frequently an experienced investigator may need to experiment before 
 he finally decides just what calculations to make. 
 
 ILLUSTRATIVE REFERENCES 
 A. Transference from original sources. 
 
 DoziER, jMiriam. "Confidential Recommendations as a Basis 
 for Selecting Teachers," Journal of Educational Re- 
 searcJi, 14:325-355, December, 1926. 
 
 On page 326 under the caption, "Method of procedure in 
 selecting quaHties," the author gives information concerning the 
 transferring of data from their original source, letters of 
 recommendation, to their classified recording on eight different 
 charts. 
 
 Ross, Clay Campbell. "The Relation Between Grade School 
 Record and High School Achievement," Teachers Col- 
 
 [42] 
 
I 
 
 Techniques of Educational Research 43 
 
 lege, Columbia University Contributions to Education, 
 
 No. 166. New York: Bureau of Publications, Columbia 
 
 University, 1925. 70 p. 
 
 On page 9 is shown the "cumulative record card" used in 
 this study. Reasons are given for certain details in the form of 
 this card. For example, entries are grouped in threes to aid in 
 recording and tabulating. A buff card was used for boys, and a 
 white card for girls. Other details are given which made for 
 efficient handling of the data. 
 
 Meek, Lois Hayden. "A Study of Learning and Retention 
 in Young Children," Teachers College, Columbia Uni- 
 versity Contributions to Education, No. 164. New York : 
 Bureau of Publications, Columbia University, 1925. 96 p. 
 
 On page 3 is shown the record sheet used by the experi- 
 menters in this study. It is in tabular form, and the numbers in 
 the "stub" of the table aid the experimenter in controlHng the 
 experiment. Space is given for remarks. 
 
 Jones, Walter Benton. "Job Analysis and Curriculum 
 Construction in the Metal Trades Industry," Teachers 
 College, Columbia University Contributions to Educa- 
 tion, No. 227. New York : Bureau of Publications, Co- 
 lumbia University, 1926. 104 p. 
 
 A trade element chart is given on page 35 which serves the 
 function of summarizing a portion of the data. Such a chart 
 exhibits data in a convenient form for further classification and 
 use. 
 
 Heilman, J. D. "Methods of Reporting the College Teach- 
 er's Load and Administrative Efficiency," Educational 
 Administration and Supervision, 11 : 167-87, March, 1925. 
 
 Information is given in regard to methods used to record 
 data. On page 168 will be found a reproduction of one of the 
 record blanks used. 
 
 B. Interpretation by means of graphs. 
 
 1. Frequency polygon. 
 
 Strachan, Lexie. "Distribution of Intelligence Quotients of 
 Twenty-two Thousand Primary School Children," Jour- 
 nal of Educational Research, 14: 169-177, October, 1926. 
 
 Frequency polygons are used to ilkistrate the distribution 
 of I. Q. among certain groups of white and colored children. 
 The polygons are superimposed on each other, showing the ex- 
 tent of variation between the groups. 
 
44 BuLLETix Xo. 38 
 
 West, Paul V. "A Critical Study of the Right Minus 
 Wrong ^Method," Journal of Educational Research, 8: 1- 
 9, June, 1923. 
 
 On page 7 will be found a graph in which two frequency 
 polj-gons are shown together. The method used clearly indicates 
 the differences in distribution of the two groups. 
 
 2. Frequency curve. 
 
 Kirk, Johx G. "Handwriting Survey to Determine Grade 
 Standards," Journal of Educational Research, 13:185, 
 March, 1926. 
 
 The distribution in quality of one thousand specimens of 
 social correspondence is represented on page 185. Such a smooth 
 curve has the advantage that it eliminates some of the irregu- 
 larities of the frequency polygon, and therefore better represents 
 the true conditions of the whole body of data of which a part is 
 represented. 
 
 3. Histogram. 
 
 Breed, F. S. and Breslich, E. R. "Intelligence Tests and 
 the Classification of Pupils," School Revieii.', 30:56-57, 
 January, 1922. 
 
 Four histograms are given in which the individual cases are 
 represented by means of squares. The individual student may be 
 identified, if necessary, by the numbers which appear within the 
 square representing his scores on two tests. Certain squares are 
 shaded to indicate displacement of individual cases. Such a figure 
 presents graphically, and in a simimarized form much more than 
 could be gained from the context. 
 
 Aberxathy, Ethel ]\I. "Correlations in Physical and ^len- 
 tal Growth," Journal of Educational Psychology, 16: 539- 
 546, November, 1925. 
 
 The reader will find a reproduction of a histogram on cross 
 section paper given on page 539. This might pro\e useful for an 
 unpublished thesis, but is not to be recommended for a published 
 report. 
 
 Geiger, Ruth. "A Study in Reading Diagnosis," Journal of 
 Educational Research, 8 : 283-300, November, 1923. 
 
 The reader will find several examples of the frequency 
 histogram in this article. Individual pupils are represented by 
 numbered squares. 
 
Techniques of Educational Research 45 
 
 4. Normal curve with experimental curve. 
 
 West, Paul V. "A Critical Study of the Right Minus 
 
 Wrong Method," Journal of Educational Research, 8: 1- 
 
 9, June, 1923. 
 
 A frequency polygon is given on page 3, upon which a 
 normal frequency curve has been superimposed. This serves to 
 illustrate that the distribution of scores, except for minor devia- 
 tions, falls according to the laws of chance. 
 
 Thorndike, E. L. and Bregman, E. O. "On the Form of 
 
 Distribution of Intellect in the Ninth Grade," Journal of 
 
 Educational Research, 10:271-278, November, 1924. 
 
 Several histograms are given in this study which are based 
 upon experimental frequency distributions. The normal curve 
 is superimposed on each of these, and indicates very clearly the 
 extent of departure from normal frequency. 
 
 5. Bar graph. 
 
 Fort, L. M. "College Admission Requirements in the North 
 
 Central Association," School Review, 31 : 683, November, 
 
 1923. 
 
 The author uses a bar graph to compare the number and 
 percentage of institutions of the East, of the West, and of the 
 North Central Association in granting credit for certain of the 
 newer high school subjects. The adjacent bars representing the 
 three differenU classes of schools show very clearly the extent of 
 the differences. 
 
 Bishop, Omen. "What is Measured by Intelligence Tests?" 
 
 Journal of Educational Research, 9 : 29-38, January, 1924. 
 
 On page 37, the reader will find an example of the bar 
 graph. In this case it is used to indicate the difference in 
 individual scores before and after study. 
 
 MoEHLMAN, Arthur B. "Child Accounting," Journal of 
 
 Educational Research, 9 : 293-304, April, 1924. 
 
 On pages 295 and 297 are given bar graphs representing 
 the attendance and non-attendance of school children in the 
 United States, and in the city of Detroit. 
 
 6. Scatter diagram, or correlation chart. 
 
 Ros.s, Clay Campbell. "The Relation Between Grade School 
 Records and High School Achievement," Teachers Col- 
 lege, Columbia University Contributions to Education, 
 
46 Bulletin No. 38 
 
 No. 166. New York: Bureau of Publications, Columbia 
 
 University, 1925. 70 p. 
 
 On page 25 is given a scatter diagram or correlation chart 
 which indicates graphically the degree of relationship between 
 two sets of paired facts, average standing in first year high 
 school, and composite score from five grade school factors. 
 
 Herring, John P. "Reliability of the Stanford and the Her- 
 ring Revision of the Binet-Simon Tests," Journal of 
 Educational Psychology, 15 : 217-223, April, 1924. 
 
 The reader is given an example of a correlation scatter 
 diagram on page 219. It is used to show the relationship be- 
 tween I. Q.'s obtained by the use of two tests on the same group. 
 
 C. Summarization by means of tables. 
 
 1. Simple table. 
 
 Feingold, Gustave a. "Intelligence of the First Generation 
 of Immigrant Groups," Journal of Educational Psycho- 
 logy, 15:65-82, February, 1924. 
 
 Two simple tables are given on page 70. It will be observed 
 that a heavy line is given at the tops of tables in this journal, 
 in place of the double line usually seen. 
 
 Lester, John A. "Spelling Ability and Meaning Vocabulary 
 
 as Indications of Other Abilities," Journal of Educational 
 
 Psychology, 16: 175-181, Marcb, 1925. 
 
 Several examples of simple tables will be found in this 
 article. For example Table 1 shows the correlation of spelling 
 ability with the following school subjects: French, Latin, 
 Geometry and Algebra. The word "correlation" appears in the 
 "stub." 
 
 2. Open table. 
 
 SiEHL, B. H. "A Survey of a Night High School," School 
 
 Reviezv, 31 : 535, September, 1923. 
 
 On page 535 will be found a good example of a simple open 
 table. This type is useful when one has a list of items for 
 which there is but one or two columns of figures. 
 
 Rainey, Homer P. "A Study of the Curricula of State 
 Teachers Colleges," Educational Administration and Su- 
 pervision, 11:465-472, October, 1925. 
 
 The reader will find a good example of a simple open table 
 on page 466. Details of form should be noted. 
 
 Douglas, O. B. "The Present Status of the Introductory; 
 Course in Educational Psychology in American Institu- 
 
 i 
 
Techniques of Educational Research 47 
 
 tions of Learning," Journal of Educational Psychology, 
 
 16:396-408, September, 1925. 
 
 On page 401 will be found an example of a simple open 
 table. On page 407 is an example of a simple open table doubled 
 upon itself. 
 
 3. Table doubled upon itself. 
 
 Wager, Ralph E. "Some Outcomes of the Teaching of 
 
 History," School Review, 31 : 215, March, 1923. 
 
 On page 215 will be found a good example of a table 
 doubled upon itself. An economy of space is attained with this 
 type of table, when dealing with a long list of items for which 
 there are but one or two columns of figures. 
 
 4. Table with subdivisions. 
 
 Proctor, William M. "The Junior College in California," 
 
 School Review, 31 : 366, May, 1923. 
 
 On page 366 will be found a table containing subdivisions, 
 and second subdivisions. For example, vmder the heading 
 "Academic Degrees," are to be found such sub-headings as 
 "Ph. D.," "A. M.," and "A. B.," with second sub-headings "Num- 
 ber" and "Percentage." Columns of figures appear below these 
 second sub-headings. 
 
 Price, E. D. "A Plan of Classifying Pupils," Journal of 
 
 Educational Research, 12 : 341-348, December, 1925. 
 
 On page 344 will be found a table containing subdivisions. 
 The individual cases for the years 1918-19 and 1922-23 are sub- 
 divided into three classes, per cent promoted, per cent retained, 
 and per cent dropped. 
 
 5. Table with varied divisions. 
 
 Proctor, William M. "The Junior College in California," 
 
 School Review, 31 : 365, May, 1923. 
 
 The table given on page 365 is a good example of one 
 containing varied divisions. The first two columns have single 
 headings, the next two fall under sub-headings common to one 
 heading, and the last five columns have five sub-headings under 
 one principal heading. Such a table enables a writer to sum- 
 marize a large body of data. 
 
 Chassell, Clara F. "The Promotion Plan in the Horace 
 Mann Elementary School and Kindergarten," Educa- 
 tional Administration and Supervision, 9:445-447, Oc- 
 tober, 1923. 
 
 A table of varied divisions is given on page 446. The dif- 
 ferent grades are represented in the "stub." "Psychological 
 tests" is subdivided into two divisions, "Educational tests" is 
 
48 Bulletin No. 38 
 
 subdivided into three divisions, while "Citizenship scales," and 
 "Teachers' judgments" are not subdivided at all. 
 
 BIBLIOGRAPHY 
 
 Alexander, Carter. School Statistics and Publicity. Boston: Silver, 
 
 Burdett and Company, 1919, p. 71-89. 
 
 The efficient handling of data frequently calls for considerable judg- 
 ment in deciding what procedures to use. The investigator will find this 
 discussion very helpful in deciding the kind of cards, blanks, paper, and 
 tables to be used in the recording of data. Information is given as to 
 methods of summarizing and checking of items. 
 
 McCall, Willi.\m a. Hozv to Measure in Education. New York : 
 
 The ]\Iacmillan Company, 1922, Chapters XII and XIII, 
 
 Various methods of tabulation are given, along wath information 
 concerning their respective values. The form of tables is discussed, and 
 reasons are given for the adherence to certain conventions. Chapter XIII 
 contains a worth while discussion of graphic methods. 
 
 Odell, C. W. Educational Statistics. New York : Century Company, 
 
 1925. 318 p. 
 
 Chapters I and II contain much that will be helpful to the research 
 w^orker in coming to a decision as to what procedures should be used in the 
 handling of data. Discussions are given in regard to such things as 
 significant figures, the grouping of measures into classes, width of class 
 interval, the frequency polygon, histogram, ogives, and smoothed curves. 
 
 RuGG, Harold O. A Primer of Graphics and Statistics. Boston : 
 
 Houghton Mifflin Company, 1925. 142 p. 
 
 Information is given relative to the graphing of frequency distribu- 
 tions. This is followed by the application of graphic methods to such 
 things as correlation, and the expression of a variety of statistics. 
 
 RuGG, Harold O. Statistical Methods Applied to Education. New 
 
 York: Houghton Mifflin Company, 1917, Chapters III and X, 
 
 In Chapter III, the reader is given explanations of certain important 
 phases of tabulation. The use of different types of cards, sheets, and note- 
 books is discussed. The chapter also contains a discussion of tabulating 
 machines. Chapter X gives methods for the preparation of tables and 
 graphs. 
 
 Williams, Harold J. GrapJiic Methods in Education. Boston: Hough- 
 ton Mifflin Company, 1924. 319 p. 
 
 The second chapter contains information relative to the instruments 
 necessary for the construction of charts and graphs. The following chap- 
 ters contain many illustrative examples which should be very helpful to 
 the novice in summarizing his data by graphic methods. 
 
CHAPTER V 
 DETERMINING CONCLUSIONS 
 
 Formulating conclusions. The conclusions are essentially the 
 answers to the questions specified in the definition of the problem. 
 Hence, in formulating the conclusions, an investigator is merely at- 
 tempting to answer the questions that have guided him throughout the 
 research. These answers should be formulated very carefully so that 
 they will express precisely what the writer intended to say. It may be 
 that the data justify only tentative or partial answers. If so, the 
 statements should clearly indicate this fact. 
 
 Hypotheses and vertification. Determining conclusions corre- 
 sponds very closely to two steps of reflective thinking, formulation of 
 hypotheses and verification. In determining the answers to the ques- 
 tions of the problem the investigator formulates tentative statements 
 and then compares them with the data as a means of verification. 
 Sometimes the meaning of the data is obscure and considerable ingenu- 
 ity is required to formulate a hypothesis.^ 
 
 Being scientific.^ To be scientific is to know one's data and to 
 use them with full recognition of the defects that they may possess. 
 Hence in determining conclusions, especially in verifying tentative 
 formulations, the investigator should have clearly in mind all the limit- 
 ations of his data. It is obvious that he cannot do this unless he knows 
 his data. One who has not made an extended and critical study of 
 data is usually ignorant of their limitations. In educational research 
 we seldom work with perfect data. Sometimes their imperfections are 
 not significant but sometimes they are.^ The investigator is responsible 
 for knowing his data. Failure to know them and use them intelligently 
 means that he is not scientific. 
 
 Determining conclusions versus interpreting data. There appear 
 to be two concepts of educational research. One of these was described 
 in Chapter I. According to the other, the problem and its definition 
 
 iFor a discussion of the interpretation of two derived measures, see Odell, Charles W'. 
 "The Interpretation of the Probable Error and the Coefficient of Correlation," University of 
 Illinois Bulletin, Vol. 23, No. 52, Bureau of Educational Research Bulletin No. 32. Urbana: 
 University of Illinois, 1926. 49 p. 
 
 -An illustration of being scientific is furnished by Odell, Charles W. "Are College 
 Students a Select Group?" University of Illinois Bulletin, Vol. 24-, No. 36. Bureau of 
 Educational Research Bulletin No. 34. Urbana: University of Illinois, 1927. 45 p. In this 
 study the author calls attention to certain faults of his data and then shows that the nature 
 of these faults is such that they do not seriously weaken his conclusions. 
 
 ^For a discussion of errors in educational measurements, see Monroe, Walter S. "The 
 Constant and Variable Errors of Educational Measurements," University of Illinois Bulletin, 
 Vol. 21, No. 10, Bureau of Educational Research Bulletin No. 15. Urbana: University of 
 Illinois, 1923. 30 p. 
 
 [49] 
 
50 Bulletin" Xo. 38 
 
 are not made basic. The data collected are not necessarily restricted 
 to those called for by the problem. In fact, investigators who have 
 this concept of education frequently collect data without any clearly 
 defined problem in mind, or they may merely take data that have 
 accumulated as a product of operating a school or engaging in some 
 other activity. In such cases, the data are looked upon as material to 
 be interpreted rather than used as a means of answering certain ques- 
 tions. As a result, we have reports of educational research that pre- 
 sent a long list of "conclusions" which are demonstrated or suggested 
 by the data, but which are not answers to questions raised at the be- 
 ginning of the work. Although there are occasions when it is profitable 
 to analyze, organize, and summarize accumulations, such activity is not 
 educational research as conceived by the authors of this bulletin. Conse- 
 quently, the references illustrate the determination of answers to ques- 
 tions rather than the interpretation of data. 
 
 ILLUSTRATIVE REFERENCES 
 
 A. Statement of conclusions. 
 
 1. Concise statements. 
 
 a. Answers specifically connected with questions asked in 
 statement of problem. 
 
 Maddocks, Carl \\". 'The Factor of Intelligence in School 
 
 Failures," School Review, 35 : 602-11, October, 1927. 
 
 This study gives the reader an excellent example of con- 
 clusions specifically related to the questions expressed in the 
 statement of the problem. For example, in answer to the follow- 
 ing question: "Is intelligence a factor in school failures?" the 
 following answer is given : "Failure in school work indicates a 
 low t\-pe of mentality." Another question was : "Do teachers 
 think that inferior mental ability" causes manj- pupils to fail?" 
 Its answer is stated : "Teachers think that school failures are 
 due to low mentalit}'; this is contrarj' to the general opinion that 
 teachers do not appreciate the degree of mentality of pupils, 
 especially in the elementar\- schools." 
 
 Powers, Samuel Ralph. "A Diagnostic Study of the Sub- 
 ject ^Matter of High School Chemistry," Teachers College, 
 Columbia University Contributions to Education, Xo. 
 149. Xew York : Bureau of Publications, Columbia Uni- 
 versity, 1924. 84 p. 
 
 The problem is stated in the form of several related ques- 
 tions on page 3. The purpose of each of the next five chapters 
 is to suggest the answer to one or more of these questions. At 
 
 I 
 
Techniques of Educatioxal Research 51 
 
 the ends of each of these chapters will be found conclusions. 
 The seventh chapter makes use of the findings of the investiga- 
 tion in recommending certain changes in the subject matter of 
 high school chemistrj-, and in suggesting certain topics for further 
 studies. 
 
 b. Conclusions specifically connected to declarative statements 
 of problems. 
 
 Anibel, Fred G. "Comparative Effectiveness of the Lec- 
 ture-Demonstration and Individual Laboratory Method," 
 Journal of Educational Research, 13 : 355-65, May, 1926. 
 
 The conclusions are given to the reader in several concise 
 statements. The problem demanded a comparison of two meth- 
 ods. The conclusions deal with different points of comparison 
 such as the factors of immediate and delayed retention, the effect 
 on brighter students, the time of presentation, and the compara- 
 tive cost. 
 
 AvERiLL, Laurence A. and Mueller, Alfred D. "Normal 
 
 School Students in Two-Year and Four- Year Courses," 
 
 Educational Administration and Supervision, 11 : 52-55, 
 
 January, 1925. 
 
 The problem is given in a single declarative statement, but 
 it suggests several phases. Each of these phases is given a place 
 as a topic in the body of the report. On page 54 will be found 
 conclusions, each of which is related to one of the phases. 
 
 2. General statements. 
 
 Helseth, Inga O. ''Children's Thinking," Teachers Col- 
 lege, Columbia University Contrihutions to Education, 
 No. 209. New York : Bureau of Publications, Colum- 
 bia University, 1926, p. 74-76. 
 
 A general interpretation of the data is given under the 
 caption, "Conclusions." Diff'erent phases of the investigation are 
 discussed in such a way as to bring out certain results of the 
 study. For example, the author states that at the beginning of 
 the 3'ear the pupils sat passively back and waited for the teacher. 
 At the end of the year the pupils were proposing questions, 
 looking up references for themselves, and going about the co- 
 operative solving of problems. This discussion is summarized 
 by four general conclusions, the first of which may be given : 
 "Children are capable of asking and answering for themselves 
 good questions about history." 
 
 Brueckxer, Leo J. "The Status of Certain Basic Latin 
 Skills," Journal of Educational Research, 9:390-402, 
 May, 1924. 
 
 After a brief statement of the limitations of data, nine 
 general conclusions are given. Two of these may be given : 
 
52 Bulletin No. 38 
 
 "There is a great variation in the scores of pupils on all tests 
 for each semester's training. This is indicated by the wide 
 range between the scores of individuals." — "The ability measured 
 by the Godsey sentence test shows a steady increase from the 
 second to the eighth semester." 
 
 Ross, Clay Campbell. "The Relation Between Grade 
 School Record and High School Achievement," Teach- 
 ers College, Columbia University Contributions to Edu- 
 cation, No. 166. New York : Bureau of Publications, 
 Columbia University, 1925, p. 67-70. 
 
 The reader is given several general conclusions in which 
 use is made of certain derived measures such as the coefficient 
 of correlation. Two tables appear among the conclusions. 
 
 B. Statement of limitations in conclusions. 
 
 1. Recognition of limitation due to faulty or incomplete data. 
 
 Crawford, C. C. "The Correlation Between College Lecture 
 Notes and Quiz Papers," Journal of Educational Re- 
 search, 12:282-91, November, 1925. 
 
 The reader is informed of the possibility of a fault in the 
 data : "The reader should bear in mind that the results of these 
 studies do not prove with certainty that the taking and reviewing 
 on the notes caused the quiz results. It is possible that some 
 third factor may be responsible for the correlations which have 
 been found to exist." 
 
 Garrison, S. C. and Robinson, M. S. "A Study of Re- 
 Tests," Journal of Educational Research, 11 : 190-96, 
 March, 1925. 
 
 A fault in the data is indicated by the following statement 
 from the conclusions : "The results presented in this study are 
 not typical, since the subjects had had an unusual amount of 
 practice in taking tests before these data were secured." 
 
 HuLTEN, C. E. "A Study of the Speed of Upper Grade 
 
 Reading," Journal of Educational Research, 10:141-48, 
 
 September, 1924. 
 
 The author recognizes a defect in the data by stating: 
 "Because of the small number of subjects included, these experi- 
 ments should not be considered conclusive." 
 
 2. Recognition of danger of generalization. 
 
 Run, John. "The Small High School," Teachers College, 
 Columbia University Contributions to Education, No. 
 
 if 
 
Techniques of Educational Research 53 
 
 236. New York : Bureau of Publications, Columbia Uni- 
 versity, 1926, p. 136. 
 
 The author begins his conclusions with the following state- 
 ment : "At the outset it should be clearly understood that such 
 conclusions as are reached as a result of this investigation apply- 
 only to the five schools included in the study and not to small 
 high schools in general." In this way the author prevents gen- 
 eralization on the basis of insufficient data. 
 
 CoY^ Genevieve L. "The Interests, Abilities, and Achieve- 
 ments of a Special Class for Gifted Children," Teachers 
 College, Columbia University Contributions to Educa- 
 tion, No. 131. New York: Bureau of Publications, Co- 
 lumbia University, 1923, p. 187. 
 
 Conclusions are given at the end of each chapter. An 
 explicit recognition of defects in the data is given in the follow- 
 ing statement : "But the number of children enrolled in this class 
 was too few to make it possible to generalize from these con- 
 clusions, and to sa}', 'These statements will hold for all children 
 of similar intelligence, under similar conditions.' Our results 
 show a tendency only." 
 
 Alderman, Grover H. "Improving Comprehension Ability 
 in Silent Reading," Journal of Educational Research, 
 13:11-21, January, 1926. 
 
 The author gives the reader the following caution in regard 
 to generalizing on the basis of his study. "The conclusions 
 apply in strictness only to teaching done under similar circum- 
 stances." 
 
 C. Application of conclusions. 
 
 1. Suggestions for application to practice. 
 
 Irion, T. W. H. "Comprehension Difficulties of Ninth 
 Grade Students in the Study of Literature," Teacliers 
 College, Columbia University Contributions to Educa- 
 tion, No. 189. New York : Bureau of Publications, Co- 
 lumbia University, 1925, p. 73-74. 
 
 Under the caption, "Pedagogic Implications," the reader is 
 given advice concerning the application of conclusions. The fol- 
 lowing statements are made : "The study which we have reported 
 finds its chief justification in the pedagogic applications which 
 follow easily from the obtained results. Therefore, in conclud- 
 ing, it is essential that we summarize the more obvious pedagogic 
 implications of our findings, keeping in mind the restrictions and 
 limitations imposed by the conditions circumscribing our investi- 
 gation." 
 
54 Bulletin No. 38 
 
 Steele, Robert M. "A Study of Teacher Training in Ver- 
 mont," Teachers College, Columbia University Contri- 
 butions to Education, No. 243. New York : Bureau of 
 Publications, Columbia University, 1926. 109 p. 
 
 The last chapter of the monograph is devoted to summary, 
 conckisions, and recommendations. The summary consists of 
 four paragraphs in which are traced the contents of the preced- 
 ing four chapters. Thirteen separate conckisions are then given, 
 and are followed by several pages of recommendations. The 
 reason for giving these recommendations is stated : "From the 
 facts set forth in this study, it is clear that to regain its earlier 
 place of leadership among the states in education and to provide 
 the children of the commonwealth with teachers comparable in 
 training to those in other New England states, Vermont must 
 take immediate steps to place its teacher-training institutions 
 upon a higher level of efficiency. To this end the following 
 recommendations are made :". 
 
 2. Suggestion of problems for further investigation. 
 
 Irion, T. W. H. "Comprehension Difficulties of Ninth 
 Grade Students in the Study of Literature," Teachers 
 College, Columbia University Contributions to Educa- 
 tion, No. 189. New York : Bureau of Publications, Co- 
 lumbia University, 1925, p. 72-73. 
 
 The investigation led to the discovery of certain problems 
 in need of solution. Six of the most important of these prob- 
 lems are given. The first of them grows directly out of one of 
 the conclusions. 
 
 Ullrich, Oscar A. "The Efifect of Required Themes on 
 
 Learning," Journal of Educational Research, 14 : 294-303, 
 
 November, 1926. 
 
 The reader is given the following statement in the conclu- 
 sions which suggests further investigation : "The results of this 
 study point to a question that seems to lie deeper than the one 
 here investigated, that is the question of motivation of theme 
 work — a problem which seems to demand investigation." 
 
 RuFi, John. "The Small High School," Teachers College, 
 Columbia University Contributions to Education, No. 
 236. New York : Bureau of Publications, Columbia Uni- 
 versity, 1926, p. 141. 
 
 The author does not feel justified in making any general- 
 izations on the basis of his data other than : "It is not too much 
 to say that this study gives additional reasons for seriously ques- 
 tioning the value and efficiency of the small high school in gen- 
 eral. It clearly shows the need for further study and educational 
 investigation of this very important part of our educational 
 system." 
 
Techniques of Educational Research 55 
 
 BIBLIOGRAPHY 
 
 Courtis, S. A. "The Validation of Statistical Procedure," Journal of 
 Educational Research, 12:31-40, June, 1925. 
 
 The appropriateness of the statistical procedure to the situation bears 
 a close relationship to the validity of conclusions drawn from the data. 
 "When a non-valid statistical method is used, the conclusion reached de- 
 parts from the truth to the degree to which the method is not valid." 
 The author further states, "It goes without saying that the careful worker 
 who reports selected data will report also the items excluded, the method 
 of selection, and a discussion of the validity of the method." 
 
 Click, H. N. "Effect of Practice on Intelligence Tests," University 
 of Illinois Bulletin, Vol. 23, No. 3, Bureau of Educational Re- 
 search Bulletin No. 27. Urbana : University of Illinois, 1925. 
 23 p. 
 
 In investigations where use is made of intelligence tests it is well to 
 consider the possibility of practice effect. Conclusions derived from intel- 
 ligence scores may be erroneous unless such factors are taken into account. 
 
 Madsen, I. N. "Intelligence as a Factor in School Progress," School 
 and Society, 15:283-288, March, 1922. 
 
 Madsen, I. N. "Anent Criteria of Educational Research," School and 
 
 Society, 19:224-225, February, 1924. 
 
 The first reference given above is to a study which received some- 
 what severe criticism from Brooks.^ The critic states that Madsen has 
 based his conclusions upon data from which relevant items have been 
 excluded. Madsen's reply to his critic is given in the second reference. 
 
 Monroe, Walter S. "The Constant and Variable Errors of Educa- 
 tional Measurements," University of Illinois Bulletin, Vol. 21, No. 
 10, Bureau of Educational Research Bulletin No. 15. Urbana: 
 University of Illinois, 1923. 30 p. 
 
 In the interpretation of certain statistical data an understanding of 
 the magnitude of the errors is essential. The research worker will find 
 much in this bulletin to aid him in arriving at conclusions which are 
 trustworthy. 
 
 National Committee on Research in Secondary Education. "An Out- 
 line of Methods of Research with Suggestions for High School 
 Principals and Teachers," U. S. Bureau of Education Bidletin, 
 No. 24, 1926. Washington: Government Printing Office, 1927, 
 p. 7-8. 
 
 The procedure to be used in arriving at a conclusion is illustrated by 
 the following quotations : "After this careful analysis comes the formation 
 of an hypothesis or tentative conclusion. This involves freeing the mind 
 
 ■"Brooks, Fowler D. "Criteria of Educational Research," School and Society, 18:724- 
 729, December, 1923. 
 
56 Bulletin No. 38 
 
 from prejudice, so that, uninfluenced by tradition or personal bias, the 
 tentative conclusion may be the result of the data obtained. No precon- 
 ceived ideas or prejudices are allowed to have any influence. The habit 
 of suspending judgment until all the evidence is in must be formed. 
 Jumping at conclusions from inadequate grounds is not to be tolerated. — ■ 
 Nor is the research W'orker content with the mere statement of the tenta- 
 tive conclusion. Wherever possible, he attempts to find whether his hjpoth- 
 esis is correct by the process of validation, of trj'ing it out in as many 
 situations as possible to see if it works." 
 
 Odell, Charles W. "The Interpretation of the Probable Error and 
 
 the Coefficient of Correlation," University of Illinois Bulletin, Vol. 
 
 23, No. 52, Bureau of Educational Research Bulletin No. 32. 
 
 Urbana: University of Illinois, 1925. 49 p. 
 
 In order to arrive at valid conclusions, a clear understanding of the 
 manner in which probable errors and coefficients of correlation are to be 
 interpreted is necessary. The investigator will find this bulletin of great 
 help in that regard. 
 
 Whipple, Guy M. "The Improvement of Educational Research," 
 
 School and Society, 26: 249-259, August, 1927. 
 
 After a long discussion of the futility and errors of much educational 
 research, the author makes a plea for conclusions which are not wholly 
 apologies. He feels that there is a limit to the number "inconclusive and 
 merely suggestive" reseaiches that should be published. 
 
CHAPTER VI 
 REPORTING EDUCATIONAL RESEARCH^ 
 
 Reporting an important phase of educational research. The re- 
 port of an investigation not only serves to record and communicate 
 the procedure and the results but it also fulfills an important function 
 in the process of research. In the act of writing, if it is well done, the 
 research worker refines his thinking and the detailed record facilitates 
 the critical testing of the work done. Thus an investigator should not 
 consider that he has completed his task until a complete report has 
 been prepared. If he is interested in communicating his work to others, 
 the report must be well written in order to fulfill that purpose eft"ec- 
 tively. 
 
 The communicatory function of a report of educational research. 
 Usually, in preparing a report of a study, a research worker should 
 not confine his purposes to "telling" the reader the answer that he has 
 obtained to the problem studied.^ Instead, he should try to guide the 
 reader to think about the problem in such a way that at the conclusion 
 of his reading a dependable answer will have been attained. Thus a 
 complete report of an investigation should guide the reader (1) to 
 define the problem clearly so that he will understand just what ques- 
 tions are to be answered and in some cases their relation to other ques- 
 tions, (2) to understand the data introduced, especially to be aware of 
 their limitations, and (3) to test critically the hypothesis which is to 
 be accepted as the answer and as a result to qualify or limit it if the 
 data indicate that this should be done. In case other hypotheses are 
 likely to occur to the reader, the discussion should guide him to an 
 understanding of the reasons for their rejection. 
 
 I. CRITERIA FOR JUDGING OR PLANNING A REPORT OF 
 EDUCATIONAL RESEARCH 
 
 The criteria or rules that should be recognized in planning a report 
 of educational research or in evaluating a report are stated in question 
 form in the following summary. Preliminary to an attempt to apply 
 these criteria a writer should seek a clear understanding of them by 
 
 ^This chapter is reproduced with minor changes from: Monroe, Walter S. and John- 
 ston, Nell B. "Reporting Educational Research," University of Illinois Bulletin, Vol. 22, 
 No.^ 38, Bureau of Educational Research Bulletin No. 25. "Urbana: University of Illinois, 
 1925. 63 p. This bulletin is now out of print. 
 
 -There may be occasions when an abbreviated report should be made. For example, 
 in an oral presentation it is undesirable to give a large number of details and one may 
 appropriately ask his audience to assume that the technical procedure was satisfactory or at 
 least to take his word for it. On such occasions, it is also undesirable to present the details 
 of one's data. 
 
 [57] 
 
58 Bulletin No. 38 
 
 reading the explanation which follows. The topics listed under "de- 
 velopment, evaluation, and organization of ideas" are somewhat in- 
 tangible and are probably the most difficult to apply effectively, but 
 they are important and therefore should be given especial attention. 
 
 Summary of Criteria 
 
 A. General Structure of a report. 
 
 1. Alajor divisions. 
 
 a. Are the following divisions reasonably explicit : defi- 
 nition of problem, collection of data, treatment of data, dis- 
 cussion of each question to be answered, and the conclusion? 
 
 2. Introduction. 
 
 a. Is the problem introduced in such a way that a com- 
 petent reader will understand and appreciate the purpose of 
 the report? 
 
 b. Has superfluous material been eliminated from the 
 introduction ? 
 
 3. Definition of problem. 
 
 a. Is the reader given a precise statement of the questions 
 to be answered? 
 
 b. In case the problem is related to other problems, are 
 the relations made clear to the reader? 
 
 4. Conclusion. 
 
 a. Is the reader given in convenient form an explicit 
 answer to each question included in the "definition of the 
 problem" ? 
 
 B. Development, evaluation, and organization of ideas. 
 
 5. Trend of thought. 
 
 a. As the reader "traces" the writer's thinking, will he be 
 led from a clearly defined problem to a critical and scholarly 
 answer by a route that is satisfying to him? 
 
 b. Is an encyclopedic enumeration of ideas or facts 
 avoided? 
 
Techniques of Educational Research 59 
 
 6. Development of ideas. 
 
 a. Has the writer avoided leaving "gaps" in his "trend of 
 thought" for the reader to fill in? 
 
 b. Have the important ideas been "developed" so that the 
 average reader will fully comprehend them? 
 
 c. Has the writer developed his ideas so completely that 
 no points have been overlooked which might leave the reader 
 with unanswered questions? 
 
 7. Evaluation of ideas. 
 
 a. Have all irrelevant ideas been eliminated? 
 
 b. Have the ideas been grouped properly with reference to 
 their relative importance? 
 
 8. Accuracy of interpretation. 
 
 a. Have the data been accurately interpreted ? 
 
 b. Do the statements agree with generally accepted opinion 
 and "common sense"? If not, is attention called to such dis- 
 agreements ? 
 
 9. Precision of statement. 
 
 a. Are the statements made so that they will convey to the 
 reader exactly the meaning intended ? 
 
 b. Are all statements worded so that ambiguity or indefi- 
 niteness is avoided? 
 
 Details of structure and form. 
 
 10. Diction. 
 
 a. Have appropriate words and phrases been used at all 
 times? 
 
 b. Have particular words and phrases been used with a 
 consistent meaning? 
 
 c. Have words and phrases to which common practice has 
 assigned technical meanings been used correctly ? 
 
 d. Has attention been called explicitly to each word or 
 phrase used with an unusual or restricted meaning ? 
 
 e. Has the "over-working" of certain words been avoided? 
 
60 Bulletin No. 38 
 
 11. Clearness. 
 
 a. Is the vocabulary suitable for the intended audience? 
 
 b. Are the ideas expressed in simple yet definite language? 
 
 12. Rhetoric, grammar, spelling, and punctuation. 
 
 a. Have the rhetorical rules relative to unity, coherence, 
 and emphasis in sentence and paragraph construction been 
 properly observed? 
 
 b. Have rules of grammar been observed? 
 
 c. Are all words correctly spelled? 
 
 d. Has consistency in the plan of punctuation been ob- 
 served ? 
 
 13. Form of tables and graphs. 
 
 a. Are the captions of tables at the top and those of graphs 
 at the bottom? 
 
 b. Are the captions, box headings, and other labels suffi- 
 ciently complete so that a competent reader will be able to un- 
 derstand the table or graph without referring to the accom- 
 panying text? 
 
 14. Explanation and interpretation of tables and graphs. 
 
 a. Has the enumeration of the facts summarized in a table 
 or diagram been minimized in the accompanying text ? 
 
 b. Is the accompanying text sufficiently complete so that 
 it is unnecessary for the reader to refer to the table or diagram 
 in order to follow the trend of thought? 
 
 c. Are references to tables and graphs sufficiently explicit 
 so that the reader will have no difficulty in locating the correct 
 table or graph? 
 
 d. In interpreting a table or graph, is the introduction of 
 irrelevant facts or comments avoided, so that the trend of 
 thought is not broken? 
 
 15. References to sources of information. 
 
 a. Are bibliographical references given for statements or 
 facts taken from the works of other persons? 
 
 16. Bibliographical form. 
 
 a. Are all references both in footnotes and in bibliog- 
 raphies given in an approved bibliographical form? 
 
Techniques of Educational Research 61 
 
 17. Chapter titles, table of contents, preface, title page, order of 
 paging, spacing, kind of paper, and so forth. 
 
 a. Have conventional rules with reference to chapter titles, 
 table of contents, and so forth been observed? 
 
 18. Footnotes. 
 
 a. Have footnotes been used to give needed explanations 
 or other comments which will make more certain a correct and 
 complete understanding by the reader? 
 
 b. Has material which would tend to break the trend of 
 thought but which is desirable to include been placed in a foot- 
 note or in an appendix ? 
 
 19. Miscellaneous. 
 
 a. Have conventional rules with reference to abbreviations, 
 division of words, spelling out numbers, and so forth been 
 complied with? 
 
 II. DISCUSSION OF CRITERIA 
 
 1. Major divisions and their sequence. No general rule can be 
 stated with reference to dividing a report into chapters or other sec- 
 tions. Usually there is an introductory section or chapter in which the 
 problem and its discussion are introduced to the reader. Either as a 
 part of the "introduction" or as divisons coordinate with it, there 
 should be a "definition of a problem" and a statement of the "sources 
 of data" and "plan of treatment." Sometimes a general statement is 
 made in regard to the "limitations of the data." 
 
 The plan of the "discussion of the problem" will be determined 
 largely by the way in which the problem is defined. Usually there 
 should be a distinct division for each major subordinate problem or 
 question. For each problem or question there should be an explicit 
 conclusion. Sometimes these may be placed at the end of the respective 
 chapters or sections, but in lengthy reports it is desirable to set apart a 
 final chapter for this purpose. 
 
 In short manuscripts (twenty-five pages or less) it is not cus- 
 tomary to label the sections except by a paragraph or center heading, 
 but a critical reader should always be able to identify them without 
 difficulty. When a manuscript includes as many as fifty typewritten 
 pages double-spaced, it is customary to form chapters or sections and 
 to label each with an appropriate title. 
 
62 Bulletin No. 38 
 
 2. Introduction. The purpose of the "introduction" is to acquaint 
 the reader with the problem under consideration so that he will both 
 understand and appreciate the purpose of the following discussion. In 
 introducing a problem, the writer may tell how it arose and discuss 
 briefly its importance, but if it is already familiar and of immediate 
 interest to the audience addressed this phase of the introduction should 
 be omitted or at least made very brief. In general, the introduction 
 should contain such explanatory statements as are judged necessary to 
 put the reader in the proper frame of mind for the understanding of 
 the material that follows. It is, however, not possible to specify the 
 portion of a report which may properly be devoted to an "introduc- 
 tion." In case the section so labeled includes, in addition to a state- 
 ment of the writer's purpose, a description of the sources of data and 
 of the plan of treatment and some comment upon the limitations of 
 the data, it may extend over several pages. On the other hand it may 
 be relatively short. A good rule to bear in mind is that as soon as 
 the reader understands the problem, he probably will be interested in 
 what the writer has to say about it and will be inclined to resent an 
 unnecessary delay. 
 
 3. Definition of problem. An effective way to define a problem 
 is to state in precise terms the specific questions to be answered. If it 
 is thought desirable, explanation and comment may be added. If there 
 are closely related questions that are not to be considered, it is some- 
 times helpful to point these out. 
 
 4. Conclusion and summary. A reader should always be able to 
 find at the end of the manuscript or at the end of the major divisions 
 a brief statement of the answers to the questions stated in the defi- 
 nition of the problem. If answers are to be given for a large number 
 of questions, they should be organized by grouping and subordination 
 so that the reader will not be bewildered by an encyclopedic enumera- 
 tion of statements. Furthermore, the conclusion should be confined to 
 a statement of the answers to the questions being considered.^ 
 
 5. Trend of thought. The phrase "trend of thought" is used as a 
 name for a somewhat subtle and indefinite concept. It is, however, 
 implied in a writer's purpose with reference to his readers. The sen- 
 tences and paragraphs which the writer sets down are for the purpose 
 of rendering maximum guidance and assistance to a reader in thinking 
 from the problem defined to its solution. The writer has thought 
 through the problem and he is writing for the purpose of guiding the 
 reader also in thinking out the same answer. This does not mean that 
 the writer should describe his actual mental processes in thinking out 
 
 'See page 7. 
 
Techniques of Educational Research 63 
 
 his solution. Frequently his thinking involves much "scrapped 
 thought," and he should endeavor to guide the reader so that such 
 waste will be eliminated. Therefore, the writer should rearrange his 
 thinking about the question so that the reader's thinking will be as 
 efficient as possible. 
 
 The organization of one's writing should be such that the reader 
 will progress through the several phases or steps of reflective thinking: 
 defining the problem, gathering data or information, forming hypoth- 
 eses, and verifying them. The writer should keep the problem or a 
 subdivision of it before the reader. Ideas should be presented to him 
 at the time he needs them. When he has been guided to the solution 
 which the writer wishes him to reach, he should be led to examine it 
 critically. 
 
 Many qualities enter into a trend of thought. There must first be 
 a good general organization ; then the rules of unity and coherence 
 must be observed in each paragraph and sentence. All irrelevant 
 material must be excluded. Good writing is not a series of isolated or 
 unconnected statements even though each one itself may express an 
 important idea or principle. 
 
 6. Development of ideas. Many writers tend to abbreviate. Im- 
 portant ideas are mentioned incidentally or merely alluded to, and in 
 some cases, omitted altogether. It thus becomes necessary for the 
 critical reader to fill in the "gaps" in the thinking if he is to avoid a 
 feeling of incompleteness. In such a case the writer has failed to fulfill 
 his function in guiding his reader to thinking fully and clearly about 
 the problem under consideration. 
 
 Words and phrases are used as symbols for ideas. Such phrases 
 as "supervised study," "divided period," "individual instruction," 
 "motivation," "curriculum," and "intelligence" occur frequently in 
 educational writings and are intended to stimulate in the mind of the 
 reader certain ideas. One person's concept of "supervised study," for 
 example, is likely to differ from another's in wealth of associations and 
 in richness of meaning, even when there is no actual disagreement. 
 Because a writer has spent some time in formulating, enriching and 
 organizing his concepts, he is very likely to attach to the symbols used 
 a richness of association and meaning much greater than that stimu- 
 lated in the mind of his reader, unless he "develops" his concepts by 
 explicit references to associated ideas and by appropriate illustrations. 
 
 7. Evaluation of ideas. In thinking about a problem* many ideas 
 may come into the mind of the writer. Some are pertinent to the prob- 
 lem; others are not. If extraneous ideas are introduced into the dis- 
 
 "■This includes any reading on the problem which the writer may do. 
 
64 Bulletin No. 38 
 
 cussion, the reader's attention is likely to be distracted from the main 
 issue. It is not sufficient to reject only those ideas which are erroneous 
 or unimportant in general. An idea may be very useful in connection 
 with one problem but have little or no value in the treatment of 
 another. Good writing is not merely recording good ideas. It is rather 
 the recording of ideas which are good for a particular purpose. One 
 of the difficult tasks of a writer is to stick to his text. His prejudices 
 in favor of certain ideas constantly tempt him to introduce topics 
 which have little or no connection with the problem. 
 
 The proper subordination of ideas is also important. As has been 
 pointed out, the writer should seek to build up a "trend of thought." 
 The effectiveness of the ideas included in this "trend of thought" 
 depends upon the organization in somewhat the same way that the 
 convenience of a house depends upon the arrangement of the several 
 rooms. There should be a few principal ideas and all others should 
 be subordinated to these. The reader will then be given a few major 
 points which will serve as organizing centers for the related ideas. 
 
 8. Accuracy of interpretation. The writer is responsible for 
 knowing his data. If they are not representative or if the}^ are subject 
 to errors or other limitations he must qualify his statements accord- 
 ingly. He should make certain that his data justify a statement before 
 he makes it. He should also check it against generally accepted beliefs 
 and "common sense." Even when he is satisfied in regard to its 
 accuracy, he, in many cases, should call the reader's attention to any 
 disagreement with beliefs or statements by other writers and shoukl 
 indicate the basis of his judgment. In the absence of such explanation 
 the critical reader may judge the statement to be inaccurate and con- 
 clude that the writer has been careless in his thinking. 
 
 9. Precision of statement. A precise statement conveys exactly 
 the meaning intended by the writer. When read carefully by a com- 
 petent person there is no uncertainty in his mind. He is not left in 
 doubt on any point. He is not confused. He has no questions to ask 
 about what the writer intended to say. It is obvious to him that the 
 writer has very clear, definite and precise ideas to express. A precise 
 statement usually includes limitations and restrictions which should be 
 kept in mind. The words used have been selected with a view to the 
 precise shades of meaning which they convey. 
 
 Lack of precision may be due to (I) poor choice of words, (2) 
 faulty usage of technical terms, and (3) omission of restrictive words 
 and phrases. 
 
 The following brief quotations from "educational writings" have 
 been carefully selected to illustrate the lack of precision of statement. 
 
Techniques of Educational Research 65 
 
 However, the reader should reahze that these excerpts may lose much 
 of their effectiveness as illustrations by being abstracted from the 
 setting given them by their authors. The first quotation illustrates the 
 effect of the choice of words upon the precision of statement : 
 
 In selecting drill material in reading, methods can be determined by a wise 
 manipulation of reading tests.^ 
 
 The words, "method," "wise," and "manipulation" are poor words 
 to use in this sentence. The following sentence expresses more clearly 
 and precisely what the author appears to have had in mind : 
 
 The use of reading tests designed to reveal the specific deficiencies of 
 children is helpful in selecting drill material in reading. 
 
 A second illustration is : 
 
 To overcome in a single grade the heterogeneous grouping of children 
 from the standpoint of mentality, various administrative devices have been em- 
 ployed writh varied degrees of success. Two of these devices, teacher's estimates 
 of pupil's work, and mentality tests stand out prominently.' 
 
 In this statement one is confused because "teacher's estimates of 
 pupil's work," and "mentality tests" are called administrative devices 
 which overcome the heterogeneous grouping of children. They are 
 not devices of this sort. They are rather instruments for securing 
 information, which in turn may be used as a basis for grouping the 
 children belonging to a single grade. 
 
 A considerable technical vocabulary has gradually been built up in 
 the field of education. Many words which were already in use have 
 been assigned very definite meanings. Failure to use such words in 
 accordance with the technical meanings assigned to them greatly 
 reduces the effectiveness of one's writing. The word "standard" is 
 very frequently used incorrectly. For example : 
 
 In measuring certain educational products there too must be a standard 
 corresponding to the many commercial standards. It is not for us here to 
 question how educational standards are obtained.' 
 
 It appears from what follows that in the second sentence this 
 writer is using "standard" in the sense of "norm." In the first sentence 
 it obviously has the meaning of "standard unit" such as a standard 
 yard, or a standard pound. Hence, the meaning of the quotation is 
 not clear, and the reader is likely to be confused or misled. It is prob- 
 able that the writer failed to think clearly. 
 
 The quotation below illustrates the securing of precision through 
 the use of restrictive terms : 
 
 In this chapter we shall not rehearse the facts concerning the boys and 
 girls of fourteen or fifteen which are available in books on the psychology of 
 
 "Paulu, E. M. Diagnostic Testing and Remedial Teaching. New York: D. C. Heath 
 and Company, 1924, p. 22. 
 ^Ibid., p. 58. 
 Ubid., p. 28-29. 
 
66 Bulletin No. 38 
 
 childhood and adolescence, but shall report three studies which add new infor- 
 mation concerning the select group of boys and girls who enter American high 
 schools.* 
 
 The words printed in italics add greatly to the precision of this 
 statement. Their value becomes apparent when the statement is re- 
 written as follows : 
 
 In this chapter we shall not rehearse the facts concerning boys and girls 
 which are available in books on psychology, but shall report three studies which 
 add new information. 
 
 The following is an illustration of a paragraph which adds to the 
 precision of the chapter in which it appears: 
 
 Care must be taken, however, not to leave the impression that the increased 
 expenditure per child of school age means merely an increased cost of an 
 identical service. During this period the public school system of the state has 
 not only attracted and held in school a larger part of the children of school age, 
 but it has expanded its "program" by establishing a more varied and elaborate 
 curriculum, and additional service such as transportation of children, health 
 service, kindergartens, vocational and continuation schools.' 
 
 This paragraph serves to direct the attention of the reader to the 
 avoidance of a conclusion which otherwise might be inferred from the 
 figures just cited. It is also an illustration of precise writing. The 
 words have been chosen carefully in order to convey exactly the mean- 
 ing intended. Evidently the writer thought that the word "program" 
 might not convey a sufficiently definite meaning and for this reason he 
 has defined it. 
 
 It is obvious that precision in thinking is a prerequisite for pre- 
 cision in expression. It would be only accidental if one wrote precise 
 sentences and paragraphs when he did not have precise ideas to ex- 
 press. On the other hand it is not generally realized that faulty usage 
 of technical terms, omission of restrictive expressions, and even a poor 
 choice of words and phrases constitute evidence that the writer has 
 failed to think clearly and precisely. There may be exceptions but in 
 general this may be accepted as an empirical rule. 
 
 10. Diction. Although "general structure," "trend of thought," 
 "development of ideas," and so forth are potent in determining the 
 quality of a report, a writer cannot communicate his ideas effectively 
 unless he uses appropriate words and phrases. Some words that are 
 used with only a general meaning in conversation and in non-technical 
 fields have been assigned precise meanings in the field of education. 
 Examples of such words are: standards (in educational measure- 
 
 'Thorndike, E. L. The Psychology of Algebra. New York: The Macmillan Company, 
 1923, p. 1. The italics have been inserted by the authors of this bulletin. 
 
 'Strayer, George D. and Haig, Robert M. Financing of Education in the State of 
 New York. New York: The Macmillan Company, 1922, p. 138. 
 
 
Techniques of Educational Research 67 
 
 ments), intelligence, objectives, content, correlation, project, diagnosis, 
 supervision and motivation. Other words and phrases have been 
 coined to represent ideas : junior high school, project method, achieve- 
 ment test, power test, supervised study, divided period, Dalton plan, 
 and so forth. A writer's skill in choosing verbs, adjectives and adverbs 
 also is very potent in determining this quality of his writing. Fine 
 shades of meaning cannot be expressed unless appropriate words are 
 used. 
 
 As a rule a writer should use a word or a phrase only as a symbol 
 for the meaning usually given to it. If he finds it necessary or desir- 
 able to modify this meaning, he should call the reader's attention to the 
 restricted or changed meaning. Usually such explanatory comments 
 should be given in a footnote and the reader may be reminded of the 
 modified meaning by enclosing the word or phrase in quotation marks 
 whenever it appears. 
 
 It is highly important that a writer be consistent in his use of 
 terms. Any inconsistency will be annoying to the reader and may 
 make it impossible for him to understand what has been written. 
 Furthermore, lack of consistency in the use of terms and phrases is 
 usually evidence of careless or superficial thinking about the problem 
 being discussed. 
 
 Frequent recurrence of a word or phrase should be avoided. No 
 term should be over-worked. No definite rule can be stated, but in 
 general a writer should endeavor to avoid the repetition of the same 
 word or phrase in a short sentence or in successive short sentences. ^° 
 This rule is especially important in the case of an emphatic noun or 
 verb. 
 
 11. Clearness. The function of a report is to stimulate in the 
 mind of a reader certain definite ideas. If these ideas are not im- 
 pressed upon him, if he misinterprets what is said or is left in doubt, 
 the writer has failed either in his own thinking or in his medium of 
 expression. This vital principle of clearness is the starting point for 
 any sound treatment of a subject. Every good style is essentially a 
 clear style. Good writers vary in their forms of expression. They 
 may have, and usually do have, certain peculiarities, but they have in 
 common the one aim of presenting their material in such a way that a 
 reader is not troubled by the slightest obscurity or irrelevance. 
 
 Often it is necessary, in addition to making the exposition clear, 
 to make it interesting. Initial interest is secured by the way in which 
 the problem is introduced. Holding the reader's attention is dependent 
 
 '"This general rule does not apply to the more common prepositions, conjunctions, etc. 
 
68 Bulletin No. 38 
 
 upon the manner in which the discussion of the problem is handled. 
 Irrelevant statements tend to hinder the reader in getting the mean- 
 ing. The presence of too many ideas is likely to be distracting and 
 confusing. \"erbosity on the part of the writer makes the reading 
 laborious. On the other hand, if the statements are too bare, if the 
 writer assumes too great knowledge on the part of the reader, the ideas 
 will not be fully grasped nor appreciated. Unfamiliar material or 
 abstract statements should be clarified by the use of illustrations or 
 examples. Statistical data often are confusing unless skillfully pre- 
 sented in tables or diagrams. Xo rule, of course, can be laid down. 
 Each writer must decide whether, considered from the standpoint of 
 the average reader, the particular statement which he makes needs 
 elucidation or will be made more effective by illustration.^^ 
 
 Anything which is of use in increasing the interest of the reader 
 is justified, but anything which is mere ornamentation is to be con- 
 demned. 
 
 12. Rhetoric, grammar, spelling, and punctuation. The authors 
 of this bulletin have not attempted a systematic or a complete treat- 
 ment of rhetoric, grammar, spelling, and punctuation. The follow^ing 
 discussion is limited to a few rules which, judging from our experi- 
 ence, are frequently violated in the kinds of educational writing referred 
 to in this bulletin. For complete and authoritative treatments of 
 rhetoric and grammar the reader is referred to the following manuals : 
 
 Greever, Garland, and Jones, Easley S. The Century Handbook of 
 
 Writing. New York : The Century Company, 1919. 228 p. 
 Leonard, Arthur W., and Fuess, Cl.a.ude M. Good Writing. New York: 
 
 Harcourt, Brace and Company, 1922. 314 p. 
 Manly, John M., and Powell, John A. A Manual for Writers. Chicago: 
 
 University of Chicago Press, 1913. 225 p. 
 AIanly, John M., and Richert, Edith. The Writer's Inde.v. New York: 
 
 Henry Holt and Companj^ 1923. 261 p. 
 Woods, George B., and Str-\tton, Clarence. A Manual of English. Garden 
 
 City, New York : Doubleday, Page and Company, 1926. 265 p. 
 
 Paragraph construction. A paragraph should be restricted to the 
 presentation of a single central idea and very closely related subordi- 
 nate ideas. It should represent a unit of a writer's discussion, and 
 should embody unity, coherence, clearness, and emphasis. Its central 
 idea in all cases should be apparent to the reader, and sometimes should 
 be expressed in the first sentence. A paragraph should never close 
 with a weak ending. 
 
 "Many successful writers have testified that they secure a great deal of help from 
 reading aloud from some good writer. In this way they are able to get the "feel" of the 
 style of the author. One successful young writer made it a practice before beginning the 
 writing of an article to read aloud from the works of William James. He did this not for 
 the purpose of understanding James but rather for stimulating in his own thinking a flow of 
 words which would have the quality of force and beauty as well as of clearness. 
 
Techniques of Educational Research 69 
 
 No rule can be stated in regard to the length of a paragraph, but 
 in general long paragraphs should be avoided. It is seldom that para- 
 graphs exceeding 200 to 250 words in length are really good. The 
 poor structure is not due to the length of the paragraph but rather to 
 the fact that many writers, particularly amateurs, begin a new unit of 
 thought by the time they have written in excess of 200 words. 
 
 Sentence construction. The essentials of a sentence are good con- 
 nection, good organization, and correct grammar. The untrained 
 writer frequently finds it difficult to attain good sentence structure. 
 Faults of grammar usually can be corrected with comparatively little 
 effort, but practice is necessary in order to secure good organization. 
 Frequently it is necessary to divide sentences, to twist them about, to 
 join or to condense them, and in some stubborn cases to make a com- 
 plete reconstruction. 
 
 Punctuation. The main rules of punctuation regarding the more 
 important uses of the period, comma, and so forth are well-fixed and 
 accepted by all authorities and should be carefully observed in sentence 
 structure. However, a variety of usage exists concerning minor points. 
 Certain persons and publishers tend to punctuate freely, others spar- 
 ingly. Each writer should decide regarding the method of punctuation 
 he wishes to adopt and should be consistent throughout his manuscript 
 in the use of that method. He should keep in mind that punctuation 
 depends upon meaning ; some marks are essential to clearness ; others, 
 though not essential, are helpful ; and all which do not aid in making 
 the meaning clear should be avoided. 
 
 13. Form of tables and. graphs, (a) Tables. Statistical data 
 usually should be presented in tabular form with a specific heading. 
 Occasionally when the data are simple and do not occupy more than 
 three or four lines, they are presented informally without a heading; 
 but this practice is to be discouraged. 
 
 The construction of good tables is difficult and few general rules 
 
 can be given that will apply in all cases. Frequently, it will be necessary 
 
 for the writer to exercise his ingenuity in order to set up good tables. 
 
 \ As a guiding principle he should bear in mind that a table should be 
 
 i easily read and should present the data effectively. Also he should 
 
 I consider the available space on the page, and unless the presentation 
 
 of the data will be weakened, he should confine the table to a single 
 
 page of the manuscript. Extremely large tables often can be avoided 
 
 by breaking them up into two or more separate ones. Each table 
 
 should have its title or caption and both the general heading and the 
 
 mbordinate headings should be sufficiently explicit so that the table 
 
 an be understood without reference to the accompanying text. 
 
70 
 
 Bulletin Xo. 38 
 
 The caption of a table is usually written in all capitals with no 
 punctuation mark at its close. (See Table I.) If there is a sub-title, 
 it is placed on the line below the main title and written in capitals and 
 small letters. (See Table IV.) When a second sub-title is considered 
 necessary, it should appear, usually in parentheses, below the sub-title. 
 
 Tables should be numbered with Roman figures consecutively 
 throughout the manuscript. The number followed by a period pre- 
 cedes the caption and appears on the same line. 
 
 Careful attention should be given to the various box heads (head- 
 ings and sub-headings) of the different divisions of the table. Hori- 
 zontal double lines are placed at the top of the table, a horizontal single 
 line at the bottom, and no vertical lines at the sides. Within the table, 
 vertical lines are placed between the columns of data; horizontal lines 
 are seldom used except to mark ofif important divisions as in the box 
 headings or to set ofif totals. Perpendicular double lines should be 
 used only when the table is doubled upon itself, that is, when exactly 
 the same kind of data appears on both sides of the double rules. This 
 is done in order to break long narrow columns of data. (See Table II.) 
 An open table, that is, a table with no perpendicular lines should not 
 be used for more than three columns of data. (See Table III.) If such 
 a table is long, it is often better to double it upon itself as in Table II. 
 
 The following specimen tables are given as illustrations of the 
 forms most commonly used in tabular construction. The first column 
 division is known as "the stub" and consists usually of reading ma- 
 terial. The other columns as a rule contain figures. 
 
 Table I. Example of a Simple Table 
 
 Test 
 
 Form 
 
 Grade 
 
 Scores 
 
 Scot 
 
 Differ 
 
 Table II. Example of a Table Doubled Upon Itself 
 
 Series C 
 
 City 
 
 Number of 
 cases 
 
 Percent of 
 cases 
 
 Series C 
 
 City 
 
 Number of 
 cases 
 
 Percent of 
 cases 
 
Name of City 
 
 Boston 
 
 Chicago 
 
 Techniques of Educational Research 
 Table III. Example of an Open Table 
 
 71 
 
 Time of Testing 
 
 . . . .October 
 
 .... February. . . . 
 
 Number of Tests 
 Given 
 
 3 
 
 6 
 
 Table IV. Example of Table With Subdivisions 
 This is the Most Common Form for Subdivisions and Second Subdivisions. 
 
 
 Achievement Quotients 
 
 IntelHgence 
 Quotients 
 
 Experimental 
 
 Control 
 
 
 Beginning 
 
 End 
 
 Beginning End 
 
 
 
 
 
 
 Table V. Example of Table With Varied Divisions 
 
 States 
 
 Number 
 
 of 
 Factories 
 
 Numbers of Employees 
 
 Men 
 
 Women 
 
 Boys 
 
 Girls 
 
 Total 
 
 
 
 
 
 
 
 
 (b) Graphs. Frequently the effectiveness of one's writing may 
 be increased by presenting certain types of facts in graphical form. 
 Simple diagrams and figures are easily understood by most readers and 
 are especially helpful to those who have difficulty in getting the mean- 
 ing from the printed page. 
 
 When employing graphical methods of expression, accuracy and 
 precision should be observed just as when writing a verbal report. 
 Since excellent treatises on graphical methods are easily accessible, we 
 shall not discuss them in this bulletin but we recommend that a writer 
 become familiar with the rules governing graphic representation before 
 he attempts to apply such methods. Violation of any of the standard' 
 rules may result in conveying misleading if not actually erroneous ideas 
 to the reader. 
 
 In addition to the rules relating to the form of graphs, there are 
 others which define conventional practices relative to arrangement and 
 
12 Bulletin No. 38 
 
 to the labeling of a figure. The caption should always be placed below 
 the graphs and the number should be in Arabic numerals, not Roman, 
 If possible all the labels of the diagram should be arranged so that 
 they can be read when the page is held in one position. A "confused 
 mass" of lines and labels should be avoided, but sufficient identifying 
 and explanatory marks should be included that a competent person will 
 be able to understand the graph without referring to the accompanying 
 text. 
 
 14. Explanation and interpretation of tables and graphs. Unless 
 it is certain that a table or graph will be understood easily, the reading 
 of it should be explained in the text of the manuscript, but no extended 
 enumeration of the facts that have been summarized in the table or 
 graph should be given. The interpretation should be confined to those 
 facts that are essential to the discussion of the problem. The reader's 
 "trend of thought" will be broken if his attention is called to items or 
 inferences not related to the particular problem under consideration. 
 Tables and graphs should be considered supplementary to the text and 
 not intimate parts of it. They supply details in a convenient summary 
 form, they also add emphasis, but they do not express a "trend of 
 thought." 
 
 15. References to sources of information. When specific facts 
 are used or sentences or paragraphs from another's writings are quoted, 
 it is customary to give in a footnote the source from which such ma- 
 terial is taken. This is done for two reasons : custom demands that a 
 writer acknowledge his indebtedness in such cases ; and the reference 
 enables the reader to consult the original material if he so desires. 
 Furthermore, the information regarding the source of a statement 
 frequently furnishes the reader with an index of its probable validity 
 and accuracy. Naturally more confidence is placed in data taken from 
 the writings of a person of reputation or from a source carefully edited 
 than from unknown authors or questionable publishers. 
 
 16. Bibliographical forms. Whenever a reference is made to the 
 work of another author, sufficient information should be given to 
 enable one not only to locate the material but also to purchase it. 
 Sometimes full information is not given on the publication and in such 
 case the reference cannot, of course, be complete. 
 
 All references in either a footnote or a formal bibliography should 
 be given in an approved form. Although slight variations in the order 
 of the items of a reference and in punctuation are found in different 
 authoritative publications, it is highly important that a writer adopt a 
 definite form and follow it consistently. The forms adopted by the 
 
Techniques of Educational Research 73 
 
 Bureau of Educational Research in its own publications are given on 
 the following pages. 
 
 It should be emphasized that the author's name should be copied 
 exactly as it appears, no change of any kind being made in it. If two 
 or more references by the same author occur in the bibliography, his 
 name should appear in the same form in each even though in the actual 
 references initials may have been used in one case and the given 
 names spelled out in another. In a bibliography the references usually 
 should be arranged in alphabetical order according to the names of the 
 authors regardless of the nature of the publications. The illustrations 
 on the following pages give the form to be used for references occurring 
 in footnotes. In a formal bibliography it is customary to have the 
 author's name appear in capitals and small capitals and to indent all 
 lines except the first as has been done in the bibliographies of this 
 bulletin. 
 
 The forms listed here as illustrations have been labeled "book 
 reference," "periodical reference," and so forth, so that they may be 
 easily referred to, but the reader should understand that it is not 
 necessary to label a reference as a book, a periodical, or a report, for 
 when a correct bibliographical form is used, this fact becomes evident. 
 
 Book references. In a book reference there are usually six items 
 to be included. They should be arranged in the following order : 
 author or authors, title, place of publication, publisher, date and pages. 
 If one or more of these items is not given in the publication, the 
 arrangement of those remaining should follow the order stated above. 
 The book references given below show the capitalization and punctu- 
 ation which should be used. Practices in regard to capitalization and 
 the use of italics vary. The name of the author or authors may be 
 capitalized, or "caps and small caps" may be used as in the following 
 references. In a typewritten manuscript not intended for publication, 
 the names may appear in "all caps." The title of the book should be 
 printed in italics, when it appears in the text or in a footnote, but the 
 underlining which indicates this may be omitted in a manuscript not 
 intended for publication. The title of a book is sometimes not itali- 
 cized in a lengthy bibliography. 
 
 Potter, Milton C, Jeschke, H., and Billet, Harry O. Oral and Written 
 English, Intermediate Book. Boston: Ginn and Company, 1921. 270 p. 
 
 Terman, Louis M. The Hygiene of the 'School Child. Boston: 
 Houghton Mifflin Company, 1914, p. 345-56. 
 
 The first reference gives the correct form for indicating the entire 
 ,(| number of pages contained in a book while the second reference gives 
 
74 Bulletin Xo. 38 
 
 the form when a few specific pages are referred to. A reference to a 
 specific section of a book or bulletin has a comma after the date, and 
 for this reason is indicated by p. — rather than — p. 
 
 Periodical references. In a periodical reference, the title of the 
 article is inclosed in quotation marks. This at once sets it off from a 
 book reference. The items necessary for complete information should 
 be given in the following order : author, title, name in full of period- 
 ical,^"^ number of volume in Arabic figures, number of pages inclusive 
 on which article appcars,^^ month and year of publication. 
 
 The publisher of the periodical is not included in the reference; 
 for if desired for purchasing purposes it ma}^ easily be found. The 
 references below show the capitalization and punctuation which should 
 be used, and also give the forms for continued articles, and for edi- 
 torials. 
 
 A single article : 
 
 Fordyce, Charles. "Intelligence Tests in Classif3'ing Children in the 
 Elementary Schools," Journal of Educational Research, 4:40-43, June, 1921. 
 
 An article continued in separate volumes of a periodical : 
 
 Mead, Cyrus D. "Silent Versus Oral Reading with One Hundred 
 Sixth-Grade Children," Journal of Educational PsycJiologv, 6 : 345-48, Tune, 
 1915; 8:367-68, June, 1917. 
 
 An article continued in the same volume of a periodical : 
 
 Henderson, Archibald. "Number, and the Fundamental Laws of Alge- 
 bra," Tlie High School Journal, 4:51-53, 77-80, 171-73; March, April, 
 December, 1921. 
 
 An editorial from a periodical : 
 
 Courtis, S. A. "Scientific Supervision," Journal of Educational Re- 
 search, 1:67-68, January, 1920. (An editorial.) 
 
 Monograph and bulletin references. In addition to the material 
 appearing in books and periodicals, there is a large quantity usually 
 published by a university, a board of education, or some other organ- 
 ized body. This is called "bulletin material." Usually there are seven 
 items to be given in a reference for a bulletin. These items are the 
 same as those included in a reference to a book with the addition of 
 certain information called description, which includes name, volutne 
 and number of the series of which the bulletin is a part, and imme- 
 diately follows the title. The items should be arranged in the follow- 
 ing order: author or authors, title, description, place, publisher, date, 
 and pages. In case one or more of the items is not given in the publi- 
 
 "The name of a periodical is italicized the same as the title of a book. 
 ''If the writer wishes to refer to certain pages or a page instead of the entire article, 
 these should be given rather than the total inclusive pages. 
 
Techniques of Educational Research 75 
 
 cation, the others should be arranged in the order stated above. The 
 references of bulletins given here show the proper capitalization and 
 punctuation. All necessary information concerning the publication 
 should be included. It will be necessary to watch closely that no series, 
 names and numbers, volume numbers, and so forth are omitted. 
 
 United States Bureau of Education Bulletin : 
 
 Capen, Samuel Paul. "Facilities for Foreign Students in American 
 Colleges and Universities," U. S. Bureau of Educatian Bulletin, 1920, No. 
 3P. Washington : Government Printing Office, 1920. 267 p. 
 
 Teachers College Contributions to Education : ^* 
 
 Berkson, I. B. "Theories of Americanization," Teachers College, Co- 
 luinhia University Contributions to Education, No. 109. New York: Bureau 
 of Publications, Columbia University, 1920. 226 p. 
 
 A school survey : 
 
 "An Educational Survey of Janesville, Wisconsin." Madison, Wiscon- 
 sin : State Department of Public Instruction, 1918. 226 p. 
 
 A school report : 
 
 "Annual report, 1918-19, Alton Public Schools." Alton, Illinois: Board 
 of Education, 1919. 156 p. 
 
 A course of study : 
 
 "Synopsis of Courses of Study, 1920-21, in Public Schools." Daven- 
 port, Iowa : Board of Education, 1920. 133 p. 
 
 Bulletins published by universities and other organized bodies : 
 
 Aleyer, Harold D. "The Rural Playground," University of North 
 Carolina Extension Bulletin, Vol. 1, No. 6. Chapel Hill : University of 
 North Carolina, 1921. 78 p. 
 
 Monroe, Walter S. (Edited by)" "Studies in Arithmetic 1916-17," 
 Indiana University Studies, Vol. 5, Bureau of Cooperative Research Study 
 No. 38. Bloomington : Indiana University, 1918. 40 p. 
 
 Holly, Chas. E. "Mental Tests for School Use," University of Illinois 
 Bulletin, Vol. 17, No. 28, Bureau of Educational Research Bulletin No. 4. 
 Urbana : University of Illinois, 1920. 91 p. 
 
 Shultis, Frank W., and Hellman, J. D. "A Study in Addition," Colo- 
 rado State Teachers College Bulletin, Series 16, No. 3, Research Bulletin 
 No. 1. Greeley: Colorado State Teachers College, 1916. 16 p. 
 
 An article within a yearbook or any bound volume : 
 
 Baldwin, B. T. "A Measuring Scale for Physical Growth and Physio- 
 logical Age," Fifteenth Yearbook of the National Society for the Study of 
 Education, Part I. Bloomington, Illinois : Public School Publishing Com- 
 pany, 1916, p. 20-21. 
 
 ^^The name of this series appears in various forms. In some cases a comma is inserted 
 after the word University, while in others Columbia University is omitted. 
 
 ^^If a reference is edited, compiled or prepared by someone in such a way that he can- 
 not strictly be said to be the author, this fact should be indicated by placing a notation in 
 parentheses immediately following his name. 
 
76 Bulletin No. 38 
 
 Miscellaneous material published by boards of education : 
 
 Deamer, Arthur. "An Experiment in Acceleration." Fargo, North 
 Dakota : Board of Education, 1918. 32 p. 
 
 "The Arithmetical Abilities of School Children as Shown by Courtis 
 Tests," Board of Education, Division of Reference and Research Bulletin 
 No. 1. Cleveland, Ohio : Board of Education, 1917. 
 
 Sexton, Elmer K. "Arithmetic Survey in the Public Schools of Newark, 
 N. J.," Board of Education Monograph. Newark, N. J. : Board of Educa- 
 tion, 1919. 30 p. 
 
 Spain, Charles L., Moehlman, Arthur B., and Harrington, H. L. "The 
 Intermediate School in Detroit," The Detroit Educational Bulletin, Research 
 Bulletin No. 6. Detroit : Board of Education, 1921. 39 p. 
 
 17. Chapter titles, table of contents, preface, title page, spacing, 
 order of paging, paper, and so forth. The chapter title should be 
 concisely expressed and should indicate the purpose of the chapter. 
 The table of contents may be very simple, consisting of only the titles 
 of the chapters, or may be elaborate with subordinate headings and 
 second sub-headings. No definite rules regarding the comprehensive- 
 ness of the table of contents can be given. In such material as a thesis 
 or a bulletin in which there is no index, it is probably desirable to list 
 in some detail the divisions and subdivisions of the different parts or 
 chapters. 
 
 The preface usually contains acknowledgments of assistance to 
 persons who have given actual aid in the preparation of the manu- 
 script, or whose books and other writings are used as references. In 
 the case of a thesis the term "acknowledgment" is often used instead 
 of "preface." A preface should be simple and dignified, should give 
 some indication of the audience to whom it is addressed, and should 
 contain some statements of the reasons for the writing of the manu- 
 script. 
 
 18. Footnotes. Footnotes fulfill three principal fitnctions : (1) to 
 give a bibliographical reference for quoted material or for facts and 
 statements; (2) to qualify or elaborate statements of the text when 
 it is undesirable to include such comments in the body of the' manu- 
 script ; (3) to suggest further treatment of the subject. Footnotes are 
 apt to be treated more carelessly and inconsistently than any other part 
 of the manuscript. They should, however, receive a writer's careful 
 attention in regard to sentence structure, punctuation, and so forth, 
 and especially in regard to the form of bibliographical references. The 
 illustrative forms given on p. 73fi". should be followed in footnotes. In 
 a formal bibliography it is customary to have the author's name appear 
 in capitals and small capitals and to indent the lines following the first 
 as has been done in the bibliographies at the end of the preceding 
 
Techniques of Educational Research 11 
 
 chapters of this bulletin. This form of indentation should never be 
 used in footnotes. 
 
 19. Miscellaneous, (a) Abbreviations. Abbreviations of words 
 or phrases are not generally considered good form in the body of the 
 report. Ar^ exception is often made in regard to titles of persons as 
 "Mr.," "Dr.," or "Prof." No one rule can be given in this case but 
 consistency is urged both in the use and in the selection of the title. 
 For example, "Dr." should not be used in referring to one person, and 
 "Mr." or "Prof." to another of the same rank or position; or the title 
 should not be given in one case, and omitted in another. Furthermore 
 when titles are used, extreme care regarding their correctness should 
 be taken. It may be said that "Professor" is usually written in full, 
 "Doctor" frequently, and "Mister" seldom, if at all. A safe and 
 widely used method is to omit titles altogether, especially if the person 
 has attained prominence. In scholastic circles, however, it is customary 
 to refer to the president of an institution by his title written in full. 
 
 (b) Division of words. The unnecessary division of a word at 
 the end of a line should be avoided if possible. Unless objectionable 
 spacing results, it is better to carry the whole word over to the next 
 line. When divisions are necessary, the following suggestions may 
 prove helpful. 
 
 Words in general are divided according to the natural divisions in 
 correct pronunciation as knowl-edge, not know-ledge. Acceptable 
 divisions may be made before such terminations as ing, er, est; imme- 
 diately following a vowel as intri-cate, mascu-line ; or between the 
 consonants when two consonants occur between two vowels as inef- 
 fective, inflam-mable. In language other than English the most im- 
 portant rule is to divide on the vowel as far as possible. If at all avoid- 
 able, two consecutive lines should not terminate in a hyphen. 
 
 In the following cases, divisions should never be made : mono- 
 syllables, as zveighf, stopped; two vowels sounded as one, as mea-dow, 
 peo-ple ; proper names ; initials of a person's name ; abbreviations as, 
 Ph.D., A.B.; or numbers expressed either in Arabic or Roman num- 
 erals. 
 
 (c) Spelling out numbers. In general, all numbers under one 
 hundred, and all round numbers, as two thousand, five hundred, and so 
 forth should be spelled out. Decimals as money and per cents, numbers 
 over one hundred and those used statistically in enumerations are 
 usually written in figures. Any number, however, used to begin a 
 sentence should be spelled out. 
 
78 Bulletin No. 38 
 
 III. DIRECTIONS REGARDING PREPARATION OF 
 
 MANUSCRIPT 
 
 General plan of report formulated in an outline and brief. As a 
 
 rule the first step in the preparation of a report of educational research 
 should be the formulation of an outline. A writer needs an outline for 
 much the same reason that a contractor needs blue-prints and specifi- 
 cations for a building. The outline serves as a guide as he works upon 
 the various details and enables him to stick to his problem. Although 
 it is usually desirable for a writer to prepare a somewhat detailed out- 
 line in written form, this should be followed by a brief. An outline 
 usually consists of separate words or short phrases which announce 
 the topics or points that will be taken up in the report. A brief does 
 more than this ; it expresses in abbreviated form the principal state- 
 ments to be made on each sub-topic or point. Illustrations showing 
 the difference between an outline and the corresponding brief on the 
 topic, "Written Expression as a Type of Learning," are given below. 
 
 THE OUTLINE 
 
 I. Relation of Organization of Ideas to Textbook Reading. 
 
 II. Relation of Expression of Ideas to Learning. 
 III. Clear Thinking and Expression of Ideas. 
 lY. The Brief Compared with an Outline. 
 
 V. Making a Brief. 
 
 (a) Principal points. 
 
 (b) Subordinate points. 
 
 (c) Revision. 
 
 THE BRIEF 
 
 I. Organization of ideas supplements the learning resuUing from the reading 
 
 of a textbook. 
 II. Expression of ideas is an important learning exercise. 
 
 III. Clear thinking is a prerequisite for effective expression. When one has 
 clear ideas it is usuall}' easy to express them. 
 
 IV. A brief differs from an outline in that it consists of phrases or sentences 
 which express ideas instead of merely words or phrases which announce 
 topics about which something will be said. 
 
 V. The steps in making a brief are: 
 
 (a) Clearly define your purpose. 
 
 (b) Formulate the principal and subordinate points of the discussion in 
 complete sentences. 
 
 (c) Organize these so that the}- will present a satisfactory "trend of 
 thought." 
 
 (d) Test each statement for precision and for its relation to the problem 
 being considered. 
 
 A carefully prepared brief an aid to writing. A brief is even 
 more helpful to a writer than an outline. It epitomizes his "trend of 
 thought." In the making of a brief, little writing is required, and he 
 has an opportunity to conceiitrate upon thinking about the problem. 
 
Techniques of Educational Research 79 
 
 Details may be neglected for the time being. A distinct service which 
 the brief renders is due to the fact that it represents a more advanced 
 stage of thinking about the problem than the outline. 
 
 Criteria for good writing should be observed. In preparing the 
 first or working draft of a report, all of the criteria for good writing 
 should be observed so far as possible, but the writer's attention should 
 be focused upon "trend of thought" and "evaluation of ideas." If a 
 brief has been prepared, the general pattern of this "trend of thought" 
 will have been formulated but the details are filled in during the writing 
 of the first draft. As this is done, the writer may find it necessary to 
 make changes in the organization of the report in order to secure an 
 effective "trend of thought." The evaluation of ideas is one of the 
 most difficult tasks which a writer encounters. It is essential to elimi- 
 nate all irrelevant statements from the report as well as to include 
 everything that should be said. 
 
 Paragraph headings. Carefully formulated paragraph headings^'' 
 are now generally used in textbooks, monographs, and bulletins to 
 assist the reader in getting the general "trend of thought." They may 
 assist the writer by providing a series of specific ideas for him to think 
 and write about. If they are to fulfill this function, the writer must 
 formulate them before the following paragraphs are written. If a 
 somewhat detailed brief has been prepared, many of the paragraph 
 headings can be taken directly from it. If a brief has not been pre- 
 pared, the formulation of the paragraph headings should follow the 
 preparation of the outline. They should not be thought of as some- 
 thing to be inserted after the paragraphs have been written. It is un- 
 necessary and frequently undesirable to have a separate heading for 
 each paragraph. However, whenever a new unit of the discussion is 
 taken up, there should be a paragraph heading which indicates to the 
 reader the nature of the "trend of thought" immediately following and 
 its relation to the problem being considered. 
 
 Formation of new paragraphs. In writing, a new paragraph 
 should be formed whenever a new point is taken up or the "trend of 
 thought" changed. If in doubt when writing the first draft it is well 
 to form a new paragraph, as combinations can easily be made in the 
 revision. 
 
 Order of writing chapters. In the case of a report consisting of 
 several chapters, it is sometimes desirable to revise the preliminary 
 
 '^In printed material paragraph headings usually appear in bold-face type as in this 
 
 bulletin. To indicate this type in a manuscript a wavy line ( ) is drawn under 
 
 the heading, but in writing a report it is more convenient merely to underline the headings. 
 However, an author should have in mind that simple underlining indicates italics and if bold- 
 face type is desired in a printed report the straight line must be changed to a wavy line 
 when the manuscript is sent to the printer. 
 
80 Bulletin No. 38 
 
 drafts of some of the earlier chapters before writing the later ones. 
 Some chapters even may be left incomplete while other sections of 
 the report are taken up. Some writers have found it undesirable to 
 spend much time over tedious details in the first draft, because when 
 working on details one tends to lose sight of the general "trend of 
 thought." However, there usually comes a time when it is advisable 
 to prepare a complete preliminary draft of the report in order that 
 each part may be judged in its relation to all others. 
 
 Physical form of working drafts. In preliminary drafts margins 
 of at least an inch and a half should be left at the top and on the left- 
 hand side. The margins on the right and at the bottom may be some- 
 what narrower. Some writers find it desirable to have even wider 
 margins. If the draft is typewritten, it should be double-spaced. Only 
 one side of the paper should be written on. 
 
 All tables except very short ones should be placed on separate 
 sheets which contain no text so that they can be inserted easily in a 
 revised manuscript. Failure to observe this rule will result in con- 
 siderable waste of time in copying tables. In order to facilitate inser- 
 tions and other changes, it is wise to begin each major division and 
 many of the subordinate divisions on a new page. Even in the case, 
 of paragraphs it is usually desirable to begin a new page when the 
 preceding paragraph ends in the last third of a page. An exception to 
 this suggestion may be made in the case of a short paragraph which 
 can be completed in the remaining space on the page. Sometimes it is 
 helpful to leave spaces between paragraphs for insertions. 
 
 Correct bibliographical forms should be observed in the footnotes 
 of preliminary drafts of a manuscript. Care also should be exercised 
 to spell all words correctly and attention should be given to capitaliza- 
 tion, punctuation, and other technical details of form. If this is not 
 done, some of the errors appearing in the preliminary drafts are almost 
 certain to persist in the final form of the manuscript. 
 
 The position of footnotes. In the working drafts of a manu- 
 script, the footnotes should be indicated by an asterisk (*) and should 
 appear on the line immediatel}" following that in which the reference 
 is made. A footnote in this position should be separated from the text 
 by triple-spacing or by horizontal lines drawn immediately above and 
 below the footnote.* An exception to this rule is usually made in the 
 
 *For example, a footnote given at this point should be placed in this posi- 
 tion and set ofif as indicated. 
 
 final form of theses and other manuscripts not intended for publica- 
 
Techniques of Edlxational Research 81 
 
 tion. In such cases, the footnotes should be shifted to the bottom of 
 the page in the final typing. In order to avoid confusion when two or 
 more footnotes appear on a page, it is customary to number them 
 consecutively throughout each chapter as has been done in this bulletin. 
 However, some publishers prefer to number the footnotes consecutively 
 on each page. 
 
 Final form of a report the product of revision. A well-written 
 report is seldom attained in the first draft, even though a detailed brief 
 has been previously prepared. Most successful writers, even after 
 years of experience, are accustomed to refine their reports through a 
 series of revisions. Occasionally there is a writer who possesses un- 
 usual ability in organizing and expressing his ideas and who for that 
 reason is able to produce a creditable report at the first writing. How- 
 ever, such persons are rare and they probably could produce better 
 reports if they carefully revised their manuscripts. 
 
 Application of criteria for judging a report, the basis for revision. 
 
 The process of revising a manuscript involves applying the criteria 
 given on pages 58 to 61 and then making the changes necessary to 
 remove the deficiencies and errors revealed. Thus as a preliminary to 
 the actual work of revision, a writer should attempt to fix these 
 criteria in his mind. However, he should not seek to apply all of them 
 at once. Those given under "general structure," and "development, 
 evaluation and organization of ideas" should receive attention first, as 
 the time devoted to correcting details of structure and form will be 
 wasted if later it is necessary to make major changes. 
 
 Re-thinking required. A revision of a manuscript is not accom- 
 plished by merely correcting rhetorical and grammatical errors and 
 misspelled words. The definition of the problem and its solution 
 should be carefully retraced for the purpose of improving the "trend 
 of thought," the development and evaluation of ideas, and the pre- 
 cision of statement. A writer should try to approach the task of 
 revising his manuscript with the attitude of a critical reader. Criti- 
 cizing one's own writing is difficult because one is handicapped in 
 adopting the impersonal and detached attitude which is necessary, but 
 such criticism is essential in the production of good writing. One 
 should therefore cultivate the art of criticism, particularly with ref- 
 erence to the development, evaluation and organization of ideas. 
 
 Major reorganizations. Frequently a writer will find it necessary 
 to make certain major changes. The order of certain chapters may 
 need to be interchanged. The position of a paragraph or of a series of 
 paragraphs may need to be shifted. Some paragraphs which were 
 
82 Bulletin No. 38 
 
 written only after much effort may prove of so little value that they 
 should be eliminated. 
 
 Improvement of sentence structure. Frequently sentences and 
 even paragraphs need to be rewritten in order to secure more effective 
 expression. Sometimes the desired result can be accomplished by 
 varying one's diction. Qualifying phrases and clauses may be shifted, 
 short sentences may be combined and in other cases long ones broken 
 up. Unnecessary shifting of voice, tense or person should be elimi- 
 nated. A writer should never hesitate to rewrite a paragraph if he 
 feels that he can improve it. Detailed suggestions for improving sen- 
 tence structure are given in rhetoric texts and manuals for writers. 
 
 Correction of errors. The final step in revising a manuscript is 
 to correct all remaining errors of grammar, spelling and punctuation. 
 Much of this work will have been done in the preceding stages of the 
 revision, but there should be a final going over of the entire manu- 
 script for the purpose of eliminating errors of any kind. 
 
 Economy of time in revision. The several phases of the work 
 of revising the manuscript should be taken up in the order in which 
 they have been presented in the preceding discussion. As we have 
 already indicated, time is likely to be wasted if some of the later 
 phases are undertaken before the major changes have been made. 
 
 A writer should not consume time in unnecessary copying. Fre- 
 quently scissors and paste can be used to advantage in making changes 
 in the order of a report. Corrections can be written between the lines 
 and on the margins. Insertions can be placed on separate pages with 
 appropriate indications of their positions. It is a waste of time to 
 copy a manuscript before a fresh draft is needed. However, when a 
 critical reading of the manuscript is made difficult or impossible be- 
 cause the changes in it are distracting or confusing, the entire manu- 
 script, except whole pages which are unmutilated, should be copied in 
 t}'pewritten form if possible. A fresh manuscript often will stimulate 
 a writer to higher levels of effectiveness in his writing. 
 
 IV. GENERAL DIRECTIONS REGARDING FINAL FORM 
 OF MANUSCRIPT 
 
 Title page.^^ The title of the manuscript should be written in all 
 
 capitals and should be placed two inches or more below the top of the 
 
 page. The author's name should appear in full, placed below the title. 
 
 ''In the final typing of a thesis or of a report any rules prescribed by the college or by 
 the department should be carefully followed. The directions listed here are representative of 
 good form and should be observed except in those cases where they conflict with institutional 
 requirements. 
 
Techniques of Educational Research 83 
 
 and usually preceded by the word "By" written just above. On the 
 line below the author's name, his institutional connection and rank are 
 often given. In a thesis, previous degrees, institutions, and dates 
 should be listed. In a term paper, the title and the author's name 
 should be given as in the description above ; the names of the course 
 and instructor in charge may be added, preferably on lines near the 
 bottom of the page. 
 
 Table of contents. When a report is divided into chapters with 
 titles, there should be a table of contents which gives the chapter titles 
 preceded by the Roman numerals I, II, III, and so forth. When 
 subordinate headings are given, they should be indented and may be 
 numbered by Arabic figures or by capital letters. In case there are 
 second sub-headings, further indentation should be used and the num- 
 bering may be done by means of letters a, b, c, and so forth, or by 
 Arabic figures enclosed in parentheses. The plan of numbering in the 
 table of contents should be the same as that observed in the manuscript. 
 When a report is not formally divided into chapters, it is frequently 
 desirable to give a brief description of the contents. In all cases, the 
 table of contents should appear alone upon the page or even upon 
 successive pages. If brief, it should be so placed as to give approxi- 
 mately equal spacing above and below the written material. 
 
 List of illustrations, figures or tables. The list of illustrations 
 or figures should give the titles of the illustrations preceded by the 
 numbers in Arabic as 1, 2, 3 ; the list of tables should give the captions 
 of the tables preceded by the Roman numerals as I, II. In case a 
 manuscript includes illustrations, figures and tables, separate lists 
 should be given for each. 
 
 Spacing. The paper should be of ordinary weight, of uniform 
 size, 8^ by 11 inches, and only one side should be used for writing. 
 The text proper should be double-spaced. Quoted material appearing 
 in a separate paragraph may be single-spaced. The left-hand margin 
 should be from an inch to an inch and one-half. The right-hand mar- 
 gin may be somewhat narrower.^® The first line of a new paragraph 
 should be indented at least one inch. 
 
 First page of text. The first page of the text may or may not 
 contain the title of the manuscript. If the title is given it should be in 
 capitals, and should be dropped about two inches from the top of the 
 page. The word "Chapter," all capitals, followed by the Roman 
 
 '*In preliminary drafts of a manuscript it is well to have both margins somewhat wider. 
 Some writers have found it desirable to triple-space the text in order to provide space for 
 insertions and corrections. 
 

 UNIVERSmr OF ILLINOIS-URBANA 
 
 3 0112 070071508