y 1 UNIVERSITY OF ILLINOIS BULLETIN Issued Weekly Vol. XIX July 3, 1922 . No. 45 [Entered as second-class matter December ii, 1912, at the post office at Urbana, Illinois, under the Act of August 24, 1912. Acceptance for mailing at the special rate of postage provided for in section 1103, Act of October 3, 1917, authorized July 31, 1918.] BULLETIN NO. 10 BUREAU OF EDUCATIONAL RESEARCH COLLEGE OF EDUCATION RELATION OF SIZE OF GLASS TO SCHOOL EFFICIENCY Bureau of Educational Research Prepared, in part, from a report by P. R. Stevenson, former Assistant, Bureau of Educational Research. PRICE 50 CENTS PUBLISHED BY THE UNIVERSITY OF ILLINOIS, URBANA 1922 m BULLETIN NO. 10 BUREAU OF EDUCATIONAL RESEARCH COLLEGE OF EDUCATION RELATION OF SIZE OF GLASS TO SCHOOL EFFICIENCY by Bureau of Educational Research Prepared, in part, from a report by P. R. Stevenson, former Assistant, Bureau of Educational Research. PRICE SO CENTS PUBLISHED BY THE UNIVERSITY OF ILLINOIS, URBANA 1922 V. TABLE OF CONTENTS PAGE Preface 3 I. Introduction: Relation of Size of Class to School Efficiency 5 11. Existing Conditions in Regard to Class Size 10 III. Relation of Size of Class in Elementary School to School Efficiency 16 IV. Relation of Size of Class in High School to School Efficiency 24 V. Suggestions for Educational Experimentation 37 IlBKARY OF CONGH^^SS PREFACE This investigation was initiated by Mr. B. R. Buckingham, formerly Director of the Bureau of Educational Research. It was planned and executed by him with the assistance of Mr. P. R. Steven- son, a full time assistant in the employ of the Bureau of Educational Research during the school year of 1920-21. The present Director of the Bureau of Educational Research had no connection with the study until late in the summer of 1920-21. The portion of this report which deals with "existing conditions in regard to class size" and "the relation of size of class in high schools to school efficiency" is based upon tabulations made by employees of the Bureau of Educational Research under Mr, Stevenson's direction and included by him in a report submitted to the present Director of the Bureau of Educational Research. The chapter devoted to the relation of the size of class In elementary schools to school efficiency Is based upon tabulations made from the original data under the Immediate direction of the present Director. The concept of the efficiency ratio and the use of this concept in the Interpretation of the data are entirely the work of the present Director of the Bureau of Edu- cational Research. The conclusions are also his own. Certain limitations of the investigation, which the report dis- cusses in detail, cause the results to have a limited practical signifi- cance, but it is thought that the publication Is juslfied for two rea- sons. In the first place, the concept of school efficiency and the analysis of the conditions which must be considered in any investi- gation relating to school efficiency should be helpful to future Inves- tigators not only of the question of class size but also of questions of other phases of school procedure. In the second place, the report emphasizes the need for careful planning which will result In the control of all factors involved In the teaching situation. There Is also emphasis upon the need for securing normal conditions If the results are to be interpreted with reference to the modification of practise. Such analysis and careful thinking are not only important phases of educational research but they are the foundation upon which both the data collected and the statistical manipulation of them are based. This investigation was made possible through the cooperation of Superintendent Peter A. Mortenson, of the Chicago Public Schools, and of the school officials in certain other Illinois cities. Not only did they cooperate by permitting the collection of the data, but they actually made substantial contributions to the project by furnishing the test materials. The teachers in the schools concerned made a substantial contribution by scoring the tests and reporting them in a convenient form to the Bureau of Educational Research. The writer is glad to acknowledge the indebtedness of the Bureau of Educational Research to all those who have contributed to the project. Walter S. Monroe, Director. May 26, 1922. RELATION OF SIZE OF CLASS TO SCHOOL EFFICIENCY CHAPTER I INTRODUCTION: ANALYSIS OF PROBLEM The problem. The problem of this investigation is to study the relation of the size of class to school efficiency, or what is the effect upon the efficiency of the school when the size of class is in- creased or decreased within certain limits. Definition of terms: Class. In the high school, a class is de- fined as the number of pupils who are assigned to a single teacher for instruction during a single class period. In the elementary school, unless the instruction has been departmentalized, a class is the number of pupils assigned to a room over which a teacher has charge. For instructional purposes, a teacher in the elementary school may divide a class into two or three groups, but the total number of pupils receiving instruction from her is considered a class, as the term is used in this study. School efficiency. Educators have borrowed the term "effi- ciency" from industry and business. In these fields, efficiency is expressed by a fraction whose maximum is 1.00. The numerator of this fraction, or "efficiency ratio," is the output, and the denom- inator is the input, or educational investment. In education, the output of a school system consists of the changes produced in the pupils, i. e., the controls of conduct that the school engenders. The educational output for a semester or a year is the total of all the changes that have been produced in the pupils during the period due to the influence of the school. The educational investment includes many factors, such as buildings, equipment, textbooks, teachers, supervision, and general administration. Although it is not imper- ative to do so, it is probably best to think of both the output and the investment as being expressed in terms of the average for one pupil. We are accustomed to refer to the educational output as the achievements of the pupils. By means of educational tests and other instruments, we measure these achievements in terms of arbi- trary units. Units of one type are implied in school marks. Other units are defined by educational tests. In order to calculate the numerical value of the efficiency ratio, it would be necessary to de- termine the social value of the output in terms of dollars and cents or in terms of some unit which might be made common to both numerator and denominator. Obviously, we are not prepared to do this. However, we may use the ratio as a definition of school effi- ciency and inquire into the probable nature of the changes produced in it by the variations of certain factors upon which the educational output and educational investment depend. In making inferences concerning the fluctuations in the value of the efficiency ratio, it is necessary to remember that the production of certain achievements may be of little value when viewed in relation to our educational objectives. The attainment of certain levels of achievement may represent an educational output of considerable value, but advance- ment to higher levels may produce only slight increases in the value of the total output in this field. For example, the attainment of certain levels of ability in spelling has a distinct and relatively large social value, but advancement beyond these levels is accompanied by rapidly diminishing increments of value. Therefore, one must avoid the assumption that fluctuations in achievements are to be interpreted as having proportional values in terms of social worth. Factors which affect school efficiency. For a given educa- tional investment per pupil, the value of the efficiency ratio is af- fected when changes are made in the educational output. Methods of instruction, the plan of school organization, or the procedure of supervision may be modified even when there is no change in the investment. When modifications in the methods of using the in- vestment result in changes in the educational output, there are re- sulting changes in the value of the efficiency ratio. On the other hand, it is possible that material changes may be made in the educa- tional investment which are not accompanied by corresponding changes in the educational output. When this happens, the value of the efficiency ratio is changed, even though the actual educational output has remained constant. In many cases, modifications in the method of using the educa- tional investment are accompanied by changes in the magnitude of the educational investment as well as by changes in the educational output. Hence, we may have fluctuations occurring in both the numerator and the denominator of the efficiency ratio. It is possible that these fluctuations may be connected in such a way that the value of the efficiency ratio remains constant, or it may be that its magnitude will vary. Because we are not able to calculate a numerical value of the efficiency ratio, a careful analysis is required to determine the probable changes in it when variations occur in the numerator and the denominator simultaneously. The achievements of pupils are materially affected by their general intelligence or capacity to learn. Since individual pupils and also groups of pupils have been shown to exhibit marked individual differences when measured with respect to this trait, it is necessary to make due allowance for differences in general intelligence when comparing different school units with respect to efficiency. In case this is not done an error of interpretation will be made by attrib- uting a higher degree of efficiency to those units which consist of pupils of superior general intelligence. The effect of varying tlie size of class. The size of class is one item of the plan of the organization. From this point of view, it may be considered one of the methods of using the educational investment. Consequently, we may expect to find that changes in the size of class produce variations in the achievements of pupils. The size of the class is, also, one of the factors which determines the educational investment. In the elementary school, where a class means the number of pupils assigned to a teacher, the cost of in- struction per pupil varies inversely with the size of class.^ In a high school the size of class does not completely determine the number of student hours of instruction which a teacher gives, but it is a potent factor in this determination. In general, an increase in the size of class in the high school will tend to result in a marked de- crease in the educational investment per pupil. Hence, in studying the effect of varying the size of class upon the efficiency of the school, it is necessary for us to inquire into the resulting changes in both the educational output and the educational investment. It is only when we have done this that we are in a position to make inferences concerning the effect of variations in the size of class upon the effi- ciency of the school. *In making this statement, no account is taken of investments made in super- vision, in instruction by special teachers, and in equipment. The practical importance of a study of the relation of class size to school efficiency. During recent years, school adminis- trators have faced the problem of providing instruction for a rap- idly increasing enrollment and, at the same time, of meeting the demands from teachers for increased salaries. In meeting these two demands, there has been a tendency to increase the number of pupils instructed by a teacher in order to keep the total educational expenditure within the income of the school system. In the elemen- tary school, and to a considerable extent in the high school, the number of pupils instructed by a teacher has been increased by increasing the size of classes. It is obvious that pupils in large classes have less opportunity for recitation and, in general, receive less Individual attention from the teacher, both within and outside of the class period. Thus, the question has naturally been raised concerning the effect upon the efficiency of a school when the size of the class is increased. In the secondary school added emphasis is given to the question because certain accrediting agencies require that the size of the class not exceed a certain fixed maximum. Connection between size of class and methods of instruc- tion. It is necessary to bear in mind that, when the number of pupils instructed by a teacher is increased, there is a corresponding increase in the amount of work required of the teacher, unless there are compensating changes in the methods of instruction. For ex- ample, according to our present methods, it is customary to require a great deal of written work of pupils studying English composition. A teacher is expected to read with considerable care the composi- tions submitted by pupils and to provide a systematic procedure for correcting the errors made. Thus, an increase in the number of pupils to be taught increases the work required of the teacher un- less the number of compositions required from each pupil is reduced or a different system of handling them is used. Much the same conditions prevail in a number of other subjects in which notebooks or other written work of some sort are customarily required. Sim- ilar statements can be made with reference to individual work with pupils outside of the regular class period. It is obvious that there is a limit to the amount of work which may legitimately be required of a teacher. When the optimal teaching load has been reached, any increase in the number of pupils assigned to a teacher should be expected to be accompanied by compensating changes in the pro- cedure of instruction. When such changes are made in the teaching procedure, they, as well as the size of the class, must be considered in respect to the effect upon the efficiency of the school. Limitations of the present investigation. In this investiga- tion of the effect of varying the size of class upon school efficiency, it was intended that all other factors which affect either the achieve- ments of pupils or the educational investment should be kept con- stant, i. e., they should be the same for classes of different sizes. No information is available to show the extent to which this inten- tion was realized except in the case of the quality of the pupil ma- terial. Group intelligence tests were used to secure equivalence of capacity to learn in the two groups of pupils. The possibility of a lack of equivalence of such factors as home environment, nation- ality, attitude toward school work, previous school experience with respect to size of class, time of day (applies only to high school), etc., makes it necessary to exercise due caution in interpreting the results of the investigation. So far as the present writer is aware, the instruction that pre- vailed in the groups concerned in this investigation involved no un- usual features, and the same methods of instruction were followed in the two types of classes. It is possible that the highest degree of efficiency for classes of a given size would be attained if the methods of instruction were selected with particular reference to this size of class. It would not be surprising to find that methods of instruction which were most effective in small classes would be considerably less effective with large classes, and that methods well suited to the handling of large classes would not give the best results when used with small classes. This possibility was not considered in this investigation. Hence, the findings should not be accepted as final. There is still need for an investigation in which methods of instruction are adapted to size of class. Finally, it must be remembered that the problem of the size of class is not entirely a problem of the efficiency of the school. It is also a problem of the teacher. It is not humane and it is not socially profitable to assign teaching loads so heavy that teachers become overworked. The problem of the teacher was not considered In this investigation. CHAPTER II EXISTING CONDITIONS IN REGARD TO CLASS SIZE In order that the significance of the two investigations to be reported in the following chapters may be more fully appreciated, certain facts are presented, concerning the size of class under present school conditions in Illinois. Size of class in elementary schools in Illinois outside of Chicago. Data with reference to the size of class in the elementary schools of the state, outside of Chicago, were secured by sending a questionnaire to the superintendent of public schools in all cities and towns listed in the Illinois School Directory for 1920-21 as having six or more elementary teachers. The questionnaire asked for the number of elementary teachers having classes of the following sizes: less than 20, 20 to 29, 30 to 39, 40 to 49, and 50 and over. This information was requested for each of the school years of 1918-19, 1919-20, and 1920-21. Complete reports were received from 180 cities and towns. These are summarized in Table I. The total number of classes for which a report was secured varied from 9,422, 1918-19, to 10,403, in 1920-21. The median size of class varied from 41.4 pupils, 1918-19, to 43.3 pupils, in 1919-20. In 1920-21, the size of class was slightly less than that for the preceding year. This table also shows that, in the cities reporting, slightly more than 3 percent of all classes contained SO or more pupils. On the other hand, between 7 and 8 percent of the classes had less than 30 pupils. The greatest change in regard to the size of class during this period was in the marked decrease in the number of classes having between 30 and 39 pupils. During the school year, 1918-19, the size of slight- ly more than one-third of the classes fell within these limits. During the other two school" years, covered by this study, less than one- fifth of the classes came within these limits of size. This change is offset by a corresponding increase in the number of classes having between 40 and 49 pupils. Although the span of years covered by this table is insufficient to justify conclusions with reference to the trend of the size of class, Table I suggests that there is a tendency, in 10 TABLE I. SIZE OF CLASSES IN THE ELEMENTARY SCHOOLS OF 180 ILLINOIS CITIES FOR 1918-21. Number in Classes 1918-19 1919-20 1920-21 Number of Classes Percent of Classes Number of Classes Percent of Classes Number of Classes Percent of Classes 50 313 5134 3264 598 113 3.3 54.5 34.6 6.3 1.3 342 6865 1845 646 110 3.5 70.0 18.8 6.6 1.1 350 7161 2050 719 123 3 4 40-49 30-39 20-29 Less than 20 68.8 19.7 6.9 1.2 Total 9422 100.0 9808 100.0 10403 100.00 Median 41.4 43.4 43.2 the elementary schools of Illinois outside of Chicago, to assign from 40 to 50 pupils to a teacher. Size of class in Chicago public schools, October, 1920. Because the two investigations to be reported later were carried on, for the most part, in certain schools in Chicago, it was thought de- sirable to compile separately the facts relating to class size in Chi- cago. The information was taken from the records of the superin- tendent of schools. One-half of the elementary schools were selected at random, but all of the high schools were included. Information, with reference to the size of class in each grade, is summarized in Table II. Below the ninth grade, the classes are noticeably larger than in the high school. The median size of class is approximately 46 pupils. In the high school, a greater degree of variability is shown in the size of class. Approximately 14 per cent of the classes have fewer than 20 pupils. The median size of class for the high school is slightly over 30 pupils. It will be noted that the classes for the first year are larger than those for the following years. Of the 183 classes, 178 reported as having from 50 to 54 pupils, are classes in physical training. A number of these have more than 54 pupils. The size of high school classes is given by subjects in Table III. The median size of class for the different subjects ranges from 20.0, 11 TABLE II. SIZE OF CLASSES IN CHICAGO SCHOOLS, OCTOBER, 1920. Grades Less than Num ber in class 55 and To- tal Med- ian 10 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45^9 50-54 over 1 3 1 6 20 40 120 248 87 36 561 46.8 2 1 2 1 6 11 102 247 38 1 409 46.7 3 1 1 1 3 18 96 211 48 3 382 46.7 4 1 2 3 15 90 207 39 257 46.6 5 1 2 4 79 215 48 1 350 47.1 6 3 3 1 2 12 103 183 27 4 338 46.2 7 2 2 15 92 180 32 3 326 46.4 8 1 1 4 54 79 117 15 48 318 44.8 Total Element ary 9 9 13 42 169 761 1618 334 96 3041 46.6 Schools 9 36 108 224 556 613 850 683 254 98 92 3474 32.3 10 24 84 202 342 378 401 283 136 17 69 1936 29.3 11 14 60 90 178 148 189 119 31 10 19 858 27.8 12 15 27 64 67 81 93 77 29 2 3 458 28.4 Total H. S. 89 279 580 1143 1220 1533 1162 450 87 183 6726 30.2 for German, up to 34.5, for arithmetic. If physical education is in- cluded, the maximum median class is 43.8. However, the most sig- nificant aspect of the table is the wide variation in the size of class for a given subject. With few exceptions, there are, in each subject, classes having 10 or less pupils and also classes having more than 45 pupils. The exceptions are shop work, in which the number of pupils is probably limited by the equipment, office practise, of which there are only 37 classes in the entire school system, chemistry, home economics, botany, zoology and agriculture, in which equipment again probably limits the size of class. In German there are no classes having more than 34 pupils, but there are only 8 classes in the entire system. Opinions of city superintendents in regard to the best size of class. A questionnaire was sent to the city superintendents of public schools in all cities in the United States having a population 12 •o c or^ON'*ooTt— lOoovotN oo■^>^'*'<^Qr~-^-'J^o^n'— it^r^t~~ i CS rt rt cs ^ csO'— i-Hcsc^i— ii-H-^i— I r^ocs i o^lOTt^osO■*^^no^■*^csvo^^cor-lOCOCNVO i i i J 2 vo >o r-~ t-H ,-( r^ C-lC^OCS'J^rOCS'— !.-( CS»— (t-i00tJhcO»-i < N . 60 >- s bo JJ ^ CI, Oh j: "rt 6 - ™ Tl e '- iJ E bp g T CJ TO t^ J3 pQfcii:oHl CNUT.*" 00-ll 0->J"C^ ^^Hir> a\CN\o "^ "^ ti J -"' r CN "»-< -^ N r^ o\ -J< 00 ^^ f^ «*! •* f*-» r^ CO Q r r r l' r 1" — r.)\O00 f^Osoo -OOfi ■^Ovt^ _ m l> 6 CO PMOnO t^^CTst^ CNO-H t<-iO-H CM in be 00 =■1 « '-H- « •« ------ ^ ■'-<■ cm' c^ |o u OC^^^H a\ooas (■^cNOs 000 00 be CXJONO f^O\00 ^0^ ur,0(^ in NO >4 ►4 •^ - -^ f*^ t^ tn ^ ■* f^ **1 c^ t~~ r^ CTv tn Q r r OOQO Ot^oo t^ooo ■♦rovo UlrtOO — UOOO c^ eir- OS cm' "S S<^ 60 r^mi/-i sgs OLD 00 cMunTt" Ovr-i^ vOu^i— ' ON ON — 1000 CNLot^ tnr^u-t •OTfio t-~-*\o f^ '". U " J '" ^H ^H '" CM ui (^ 10 CJs 00 ^ ^ V»-i r-l »— 1 »— " Q r r r r .0 — ON 003; OT) < c a "oi 5 ^s ^2 10 ; 5 : ^■ 5 : : : ? t . V. . 5i • ; w w w . . . rt a S St«t« gcOM > gwco gWcO > gcoco gcoco • u"^ »t; , CO *-< ♦-• W^»ti, CO •^ »j w CO ^ « CO ^ *j : « S M g > 1 5""— t vi « 6 > < a: S S e > 1 K-"— 1 00 :s -^ S > e2 1 O^-Ja < O^Jco < ^JcS < O'^Jco < 1 cational opportunities offered in the two semesters and also any practise effect due to acquaintance with the tests, it seemed unwise to weight the averages on the basis of the number of pupils in the two groups. To have used the weighted averages in this case would have resulted in giving greater weight to the gains made by one group of pupils simply because this group happened to be larger. Achievements of the two groups approximately equal. Table V* summarizes the data with reference to the gains made by the two groups. The column headed "Number of pupils" gives the number of pupils whose records were used in the tabulations. (A pupil's record was discarded if he was not a member of both the small class and the large class and if he did not take all tests.) The average size of class is computed from the total enrollment. The computation of the gains has just been explained. In interpreting the table, attention should be focused upon the differences. A posi- tive difference means that the large class is superior in achievement, and a negative difference that the small class is superior. At the bottom of each difference column the differences, calculated from the averages and also from the medians, are summarized. This summary is a total and not an average. To find the average it is necessary to divide by 3. All of the totals of the differences fall between 4-1.00 and —1.00. Six of the 10 differences are negative, and only in the case of reading rate are the difference between the averages and the difference between the medians both positive. In the last three columns of the table. We have the totals and not aver- ages. To find the average, it would be necessary to divide by 4 in the second grade and by 5 in each of the other two grades. Here, again, the negative differences predominate, although none of them are very large. The last two entries in the last column are essentially grand totals and may be considered to summarize the entire table. To find the average difference, each of these numbers should be di- vided by 14. The quotients obtained would be — .04 and — .06. Thus, in general, this table indicates that there is little if any superi- ority in the achievements of pupils in the small classes over those of pupils in the larger classes. *The entries in the column headed "Reading Rate" in the second grade are based upon the Pressey Word Recognition Test. 21 An examination of Table V reveals the fact that, in general, the gains between the first and second testings are much larger than those between the second and third testings. This condition em- phasizes the necessity for equalizing the effect of acquaintance with the test and practise effect. If all of the experimental groups had been taught as large classes the first semester and small classes the second, the gains for the large classes would greatly exceed the gains for the small classes; but this would be due primarily to the effect of acquaintance with the test and the practise effect. When Table V is examined with reference to the conditions in the different grades we find that the gains are relatively greater for the small classes in the fifth grade than in either the second or the seventh grade. However, the number of pupils is so small for the two groups in the seventh grade and the groups differ so little in size that only slight significance can be attached to the results. Even in the second and fifth grades it is unfortunate that the experimental groups are not more nearly equal in size. It is possible that if the experiment had included a larger number of classes which were small the first semester and large the second different results might have been obtained. When the gains for the different subjects are examined we find that in language the gains for the small classes are consistently greater than the gains for the large classes. This is not true for any other subject. Although in spelling the total of the gains is distinctly negative, in both arithmetic and reading comprehension the total when computed by one method is negative and in the other case is approximately 0. Reading rate in the seventh grade is the only case in which the large class is distinctly superior in achievement. Conclusion: the relation of the size of class to school efl5> ciency. Since Table V indicates that, on the whole, there is little difference between the achievements of the pupils when taught in large classes and their achievements when taught in small classes, one might infer that the efficiency of a school would be materially increased by the formation of large classes, because the educational output would be approximately the same and the educational invest- ment would be materially decreased. However, it is doubtful that the present investigation justifies such a conclusion. In the first place, it is obvious that only certain achievements of pupils have 22 been measured. Even in the fields of the four subjects in which tests were given we are not justified in claiming that all achievements of the pupils were measured. The arithmetic tests used were confined to the operations and to only certain types of examples within this division of arithmetic. In silent reading, the test used Is very limited in scope. Similarly, the tests in language and spelling possess very definite limitations with respect to scope. There is some justification for assuming that the measurements made may be considered in- dices of the total achievements of the pupils not only in the fields of the four subjects in which the tests were given but also in the field of instruction in the grades concerned. However, the thesis that the measures of achievement secured in this investigation are indices of the total achievement is largely an assumption, and in in- terpreting the results it is necessary to recognize this fact. It is possible that, if other tests had been used or if the achievements of •the pupils had been more completely measured by including tests in other subjects, the results might have been different. In the second place, it must be remembered that the size of the "small classes" was not less than 33 (with one exception), and in a few cases the enrollment was as much as 44 or 45. The large classes ranged in size from 42 to 54. The average difference between the pairs of experimental groups ranged from 7 in the seventh grade to 11 in the second grade. These conditions with reference to the size of the experimental groups constitute a very significant limitation of the investigation. One is not warranted in making inferences from the facts of Table V with reference to the relative efficiency of classes of 20 to 25 pupils as compared with classes of 35 to 45 pupils. No application should be made except within the limits of size defined by the experimental groups. 23 CHAPTER IV RELATION OF SIZE OF CLASS IN HIGH SCHOOL TO SCHOOL EFFICIENCY General plan of the study in the high school. In the inves- tigation in the high school it was arranged to have both a large class and a small class in a given subject instructed by the same teacher during the same semester. This made it necessary to restrict the experiment to teachers who were instructing two or more sections of the same subject. When a teacher was instructing two sections, pupils were shifted on the basis of their intelligence scores so that the average quality of pupil material was approximately the same in the two sections. Thus, both classes would include some bright, some medium, and some dull pupils. When a teacher had four sections of the same subject, the pupils were shifted so that a large class and a small class would be obtained, consisting of relatively bright pupils. The less capable pupils were also divided into two classes, one large and one small. In establishing the two types of class, there was considerable variation in the size of both the large classes and the small classes. The small classes varied in size from 12 to 26 pupils. The large classes varied in size from 23 to 45 pupils. The average size of the large classes was 36.5 and that of the small classes, 20.8. The dif- ferences in the size of the paired classes ranged from 6 to 26. Source of data. This study was carried on in four large high schools in Chicago and in the high schools of three other Illinois cities.^ During the first semester of 1920-21, the experiment was carried on in beginning tenth grade classes. During the second semester of that year the study was confined to classes completing the first year of high school work. In the following tables no dis- tinction is made between classes for the two grades. Records were secured for 67 pairs of classes, enrolling 3,821 pupils. The project *The high schools in Chicago were Lane Technical, Tilden Technical, Harrison Technical, and Hyde Park. The three other Illinois cities were Macomb, Shelby- ville, and West Aurora. 24 was begun by giving the Terman Group Test of Mental Ability to all pupils concerned, about October 15, 1920. Some pupils in the high schools outside of Chicago were given the Illinois General In- telligence Scale or the Chicago Group Intelligence Test. As soon as the results of the testing could be assembled, the large classes and small classes were arbitrarily formed, following the method in- dicated above. The educational output, or the achievements of the pupils, was measured by requiring each teacher to give the same final examina- tion to both types of classes. A check upon this measurement of achievement was secured by using the "term grades" of the pupils. It is generally recognized that "term grades," as well as examina- tions set by teachers, are highly subjective. However, in this case the same teacher administered the same examination to both the small class and the large class. The same teacher also gave the "term grades." Thus, there is in no place a comparison between either "term grades" or "examination grades" given by different teachers. This tends to eliminate the subjective factor of these measures. In addition. It may be noted that we are concerned with the average "grades" of relatively large groups of pupils and not with the "grades" of individual pupils. A limitation. The plan of carrying on the experiment implies the assumption that the achievements of the, pupils in the large classes were equal to the achievements of the pupils in the small classes at the beginning of the experimental period. There was no attempt to measure the achievements of pupils In the subjects con- cerned at the beginning of the experiment. In the first semester, the two types of classes were not organized until after the high schools had been In session several weeks. In the case of the classes used during the second semester, the pupils had received an entire semes- ter of Instruction in regular classes. It is true that the sections were formed so that the average general intelligence of the paired classes was approximately equal, but this probably does not justify the assumption of equivalent achievements. Details of administration. The Intelligence tests were admin- istered and scored by the teachers. As preparation for this work, the teachers were called together and given definite Instructions con- cerning the nature of the tests. The tests were also administered 25 to them. They were then required to score their own papers. All the tests used were highly objective with reference to scoring, and a sampling of the test papers of the pupils failed to reveal any large errors in this work. Since, in the use of the intelligence test scores, comparisons are always made between the pupils or groups of pupils under the same teacher, variations in the administration of the tests, due to dijfferences between teachers, would not be significant. At the close of each semester the teachers were asked to report both the "final grade" and the "examination grade" for each stu- dent.^ In most cases, the "examination grades" were reported in terms of percents. The "final grades" were generally reported in terms of letters, as follow: S==Superior E=Excellent G=Good F=Fair D=Failure For the purpose of combining "grades," these letters were as- sumed to represent the following numerical ranges on a scale of 100 percent: S is equivalent to 95 and over E is equivalent to 85 to 94 G is equivalent to 75 to 84 F is equivalent to 65 to 74 D is equivalent to 55 to 64^ Plan of summarizing data. The data collected were sum- marized to show the differences, if any, which existed between the final achievements of pupils in the large classes and of pupils in the small classes. Two methods of summarizing were employed. In the first, the achievements of all pupils were considered. According to the second method, the records considered were limited to those of pupils in the large classes who could be paired with pupils having identical scores on the intelligence test in the corresponding small The following tables, in which the data for high school classes are summarized, indicate that the "examination grades" were not received from certain classes. °The midpoints of these intervals were presumably used as the numerical equiv- alents of the grades expressed in terms of letter. However, Mr. Stevenson's report yields no information on this point. 26 classes. For both of these methods two tabulations have been made. The first includes all classes, and the second only those pairs of classes in which the large class was at least twice the size of the small class. Differences in achievements when all pupUs are consid' ered. Table VI illustrates the first method of summarizing the data for 22 pairs of English classes. In the second and third columns of this table the enrollment in the large classes and in the small classes is given. The quantities recorded in the three columns headed "Difference" are found by subtracting the quantities in the two col- umns immediately preceding the difference column. The number for the small class is, in every case, taken from that for the large. Therefore, a positive difference means that the large class is superior in the trait concerned, and a negative difference, that the small class is superior. The line at the bottom of the table gives the average for each column. These averages may be taken as summarizing the data collected from these 22 pairs of Enghsh classes, although the items combined are not entirely comparable. For example, different general intelligence tests were used in different classes. The present writer has not been able to ascertain the particular intelligence test given to any pair of these 22 pairs of classes. It is, however, difficult to explain the extreme differences between the average intelligence scores of classes 10 and 11 on any basis other than the use of different tests in these two pairs of classes. Furthermore, it is not unlikely that different passing marks are in use in the different schools in which the pairs of classes were taught. If this is the case, in the case of differ- ent pairs of classes, both the average term grades and the average examination grades are on different scales. It should be noted that, although an effort was made to organ- ize a large class and a small class so that the average intelligence scores would be approximately the same for the two classes, this was not always accomplished. Because of conflicts or other conditions that could not be disregarded, it was not always possible to shift pupils from one section to the other so as to set up the desired class organ- ization. Table VII summarizes the averages for the classes in the dif- ferent subjects. In interpreting this table, it is necessary to bear in mind that, with the exception of English and algebra, the number of 27 fi < O CO u CO CO a: < o H w (d CO O *^ 2 ^ u ■< I-) ^ ►J v> < (d O CO U to eg [d U Q o u a o W u C oo CO ^J~\ rf rt< Os w% O so oo Os wr> so OO WO wo CN wo OO wo CO Ji r^ r- 6 oo CO ^ d CS ""J* CN CN ^ wo CO CO d CN CO so CO CN .u 1 1 1 1 1 1 1 1 1 1 ■>*< 1 It: 1 1 1 1 1 1 1 1 1 1 1 1 1 ■q ~ wi so o ts —1 CO O oo ■<*< 00 O so CN o C?s oo o ■>*< oo (« , J, w o r^ >j-i CO O CO ■^ lO oo wo f-H CN so CO ■<*< d 1-H C^ CO On r~ oo ni E CA r- CO r^ oo oo r^ r- r- (^ OO r^ OO oo f^ oo oo CO oo so C7N r^ X u bo c u C/5 oo OO ^ so oo cs oo O CS CO CO CO ■<*< ^ r^ wo r-- CO CN Os v\ 4J > < C bO lo r^ ■^ r^ OS 0\ lo d '^ OO WT) OO r~ oo so ,_( d d CN CO so "(J oo r^ r^ r~ r- so r- oo r-~ t^ r- oo r- r~ I^ oo CO oo wo wo r^ u u C CO OO 'O OS so ■ < ^ >J^ OS so CN (N «*< r-^ J-> w^ CO r^ Os ^ so ^ r^ Tf ^ ON -H 00 O — 1 CO ■ 1 1 1 1 1 1 1 J CN tt 1 1 1 1 1 1 1 1 1 Q ~ CO ■* wo CN O .—1 CO r^ Tf OO so > < bO oo w-i ■>*< -^ CN CO oo rj< (^ o >^ SO SO d r^ wo ^ ^ r~- d wo d t^ CN rt •o ON U-> rt Os 0\ CO Os r^ (?S W-) CO *— ( CN 0\ Os CO o CN CO O CO wo O J u ?^ cs r^ r^ OO OO OS CTv ON o O -H CS CN ^ rt< so so Os •^ ^^ en 1—1 ^ CN CN CN CN CN CN CN CN CN CN CN CN CN CN Tf rt m ■*■ d E J2 CN C/} "o 0. bo ^o t^ ■^ o ■* oo W-, CN -H oo >-n CO 1— ( OO so ON «-H o CN r^ 00 oo CO ON en C5 CO CO CO •* ^ CO CO CO' ■* CO CO Tt< Tt< CO CO CO ^ ^ CO CO CO CO CO oo r- n! CO U "o Lri u u ,_ cs CO --f >J-i so r^ OO Os O — CN CO Tf wo so (^ OO Os o _< CN to ft 1-^ «— t y-^ »— » CN CN CN ; u rt ii H ■w O ^ h < pairs of classes is so small that we probably should not consider the result representative. Certainly, little if any significance can be at- tached to the results for Latin, history, and French. The average for the 67 pairs of classes is the weighted average, so that one pair of classes does not influence this average any more than any other pair. Table VIII presents a summary for those pairs of classes in which the large class is at least twice the size of the small class. By doing this, we are able to examine the achievements of pupils in pairs of classes where the difference in size is marked. The number of classes in any one school subject is so small, with the exception of English, that we are probably not justified in drawing any conclu- sions for the separate subjects. Diflferences in the achievements of paired pupils. Table IX is similar to Table VII, the only difference being that it is based upon the records of only those pupils in the large classes who could be paired with pupils in the corresponding small classes, having the same scores on the general intelligence test. Since the pupils were paired on the basis of their intelligence scores, the average general intelligence of those taken from the large classes would be identical with the average general intelligence of those taken from the corre- sponding small classes. Hence, the average general intelligence scores are omitted. Table X is similar to Table VIII. Interpretation of the tables. When all 67 pairs of classes are considered, the average of the general intelligence scores for the large classes is almost identical with that for the small classes, the difference being only one-tenth of a unit. This unit corresponds approximately to one month of mental age. We may, therefore, consider the pupils in the large classes equal in general intelligence to the pupils in the small classes. Both the average "term grade" and the average "examination grade" are slightly larger for the small classes. The question of the significance of the diff'erence of two aver- ages is involved here. Both examination grades and final grades are known to be highly subjective and to involve a large error which is a combination of a constant error and a variable error. Variable errors tend to offset each other in an average because some of them are negative and some positive. On the other hand, constant errors 29 u Q < tC o z o h < 2 S < >< u u o < Oi £/) u u > to -f to < Q >-) Z u < J to ►J <; ^ ro Oi O r^ \n (d H < Id u O o < B<: Oi < u _) > •< r^ v^: tt O o fa u (O C) U ;z; m o D to hJ >-3 H W '4, H fa ?; o: u fa w fa n < Q (V, u w > W <: H fa fa o o 2 o r/) e: < CU § o u s s O On ro o „ O O c> r-i ro WT) 1 ^ c CSO"'-'CNTt<«CSOO'^ ^ Q " 1 1 1 1 1 1 1 1 1 1 c "J . — i « oocs-HCT\voooc^O'*t^aN r^t^r^i^oovo^oor^t^r^ t^ O bO « 2 i-i ^ SC M votoooavTfocn—icssO'^ oo i.i c5 '^ vOvO'^-^OOCSrOO^'^'^ ■-^»oc»CS'C-^CS•-^^>^vOa^^OO^ r- oovor^-*oo^-*'voO' S J2 vO'*'^ror^'^'^nt^'^r~- VAi < J " i^r^r--r^r~r--sr^r^r^i^r~- r^ OVOCT^^^OO'J'•^CNr^CN „ % <— icsO't'c^r^O'^fO-H-H o Q u 1 1 11-" — to ^or^^or^a^-Hooo^cs■^"^ o 13 u .£ 8 CO ^ — icsio\cs>j-i-HO\r--cot^fS •^ OOOCNOOCNCX500t-HrOCT\ o ^H r-H .— 1 >— 1 .— 1 T-( .— 1 '"' J3 JH vO^<^f^vO — CS'<*>0^^t~- ^H 1) 1) > bO (NChO'ontxsof^OCTNCo ^ < J " 00^'— lCNoocso^oo1-Hr^O o T— t »— 1 ^H ^^ ,— 1 ,-^ ^H ^^ — „ Ti^o wo < « JH r--vovnoovOOooC7\i^c^ON ^ hJ " c^nocsn-^ronc^c^co CO <-_ « O "> OJ .i: <^ (SOO>>Ofr'5c^cOOCS(N — r-~ o t« < ■^ 0! CN -H >o Z ^ "u -Q C a lU >. bO C/3 j= u C "bi c rt cn S r "■ bD w E fe -^ £i aj E X t > 1- C _c b > • ) : O tL < C ec CA C < E- U. rr < H CO U Q < 2 "5 < < w 5 05 ^ w fa ^ ° 9 N < S u ^ u w o C3 b: <* i-l hJ ►J U W H W <5 H W 05 > i? < CO u u< rn o rri z < 1-3 o C) en a a: a < < CO n Ci Q 2 ^^* <: u t) »o r- •* o cs !* c —1 O O CN <^ ^ Q *^ II 1 1 6-S oo c^ O >^ >o 00 -o o >^ ^ CM d X ^ c« y r^ oo r^ t^ t^ oo Si c rt .2 «N VO ^ -* CO r^ <^ o 00 -H o^ < .5 hJ ~ r^ oo 1^ i^ r^ r- li VO CTs '*< to ■<*" O to o > < rt J2 t^ Tt< O oo CO w-i hJ " r^ r~- 1^ r-~ t-~ t^ cs cs to «s to «s O ■*" U-, ^ CO ^ Q '^ 1 I 1 -" 1 1 , — -5 « vo •* t^ >vO r-- f 4J « E JS o o 00 r^ w-> ts c o 1/5 " cy, O r-- -H u-i CT\ .« O bO u 2 ^ (3 "• ■^ rJ -^ oo TjH o ^ to oo ts «N < hJ "" (^v On r-- cs >o OS u S to C13 CO 00 CS o o o t^ _N E -2 r~- 00 r^ w-i (JO r-- 'm ^ w >— 1 .— 1 i-H .— 1 I— 1 •—1 U bQ w < c3 "• ^ v^ o o o 0\ CT\ (N to NO oo iJ "" to CO ■*• CO to CO <*- m O « 1) • •-'*- ss 00 l/^ ^^ T-H r-l VO O rt O g 1^ ^ -5 4-> o u 1) •^^ CI -Q c 3 .H >. W CD a, -S 2 "rt >^ M :S -^ fc S o > o 1 w < O m c/^ 1 < ^1 o H O a Q OS, O u u < Q bJ Oi <: Pu CO ij d, D pl< tf) u u. r/) O Cfl O o < z S o CO H r^ < vn Z Q S Z <; < > VO < ?^ Q >?: < c/3 r/) oi U o 5 u CO o a! w 2 o h _) ij U u O H ^ Z u > J < <: O :3 c (d ;z; o u 1 1 u ' c cN r^ r- < r^ 'f ro — ' r^ - < CO Tj - 1 -, cs d ^ 1 1 H -^ CN MD tr> to .- t [ 1 1 1 i d - 1 1 < cs 1 Q 1 1 1 1 1 1 1 c J2 .2 a 1 ^ Z3 « 1 so r^r^r-~r^vosooocor^r--r- r^ 4j a 60 O 1^ tn vo'fioconow-iovCTs-^^c 5 ■* 2^ c3 "• >0 VO >^ OO CS — H vo r^ oo wo c i Tt^ ►J ~ r^r-~r--^vovot^r^vcir--r~ r^ < feb u o c r^o»^^"^ON>^i^«sr^r~ VO ^-»CSCNOOOOCS-HCSw 1 Q 1 1 1 1 1 1 o Z3 CO — cc^r^O>cOOoo«NiOTt u 1> h s, s Tfcocsw-ioo-Hror~-cs-^>x cs 2 a J3 .2 vo-<*<\Orocovovon — 00 c w^ > < -1 " r-~r-^t^r~r^r^r^r~t^r^r- r~- Tt CNl I) bo « k> > S) s VO J " n CO c cNco-^cooc-onc CO « _Q O) VO 00 CN vooO'^'rJ^-^'* » ■* coC7NOro'^«o-0'*ncoc^rocJ (S — t^ ° M CS -H VO 3 ex ^ "o 4-1 a o u 1» c IF 3 'Z c. >v bo CAI _(/ "si c s X " s 1- ° s > c 0. ^ E > dj E c u g S c 4J S>^ L- > c 0. > ) : s to < c P9 < U en h tt ■jf u < H o (d H 1 H « 00 t^ O O r^ ■ CO d r~ oo t^ i^ r- oo < S. S r-~ >^ On ^O oo o S D 2 "^ 13 JS VO CO — < O -H t^ ^ ^ hJ " r-~ 00 r^ t-~- r- t^ Pk' < < bO fa o CO (d CO CO < O c ^ oo O O oo — i (T) d d < w o s CO Id V T3 i ^o r^ o^ r^ vo o ^ CO £ o. rt JH vo hJ " r- r^ r-- r^ r~ t^ < z < u CO > (d 1-3 < CO iJ ^ 52 OO CS O O O r^ Q CO « r~ CO r^ oo >j^ r-- ^ _N ,— 1 -H »-l .— 1 -H '"' <; U (» e/3 u bO CO u pj e« < < CO Sl> S ^ SO o o o CTn es < l3 -2 ON 0\ (N VO CO oo '^ b hJ ^ CO CO •* CO CO CO S < o< ■ (d O (Si < « O < S o •« 1 ^ ^1 \0 ■* vD '-I -^ -H CO -H T— 1 .-H CO CO (N > ■< CO i-i fa C O rt CO OO W-1 ,— 1 ^ r— 1 \o o o "o J ^—' o 3 D. ?; CO "Z o o Id o en 2 z < Ed a< O § ^ o o Cd o c H 1) >-^ ><■ g TO M iJ M i'n C C £ &. : pa ^ ctf _< S 2 <; c C ^ U i! C > o h (d Ci < c c/- PC < HI are not eliminated in averages. The number of pupils included in the 67 pairs of classes is sufficiently large so that the average grades include a variable error which is probably so small as to be negli- gible. For example, if we assume that the probable variable error of a grade is as much as 10 points, which is probably in excess of the actual probable variable error, the probable variable error of the average for the small classes would be less than three-tenths of one point. In the case of the large class, it would be materially less than in the small class. The constant errors, which are expressed in the tendency of some teachers to give higher grades than others, are probably included in both groups in approximately the same pro- portion, since the same teacher assigned grades to bpth a small class and a large class. If this is true, a difference of 1.3 between the average term grades is probably significant, although there is a reasonable doubt. This doubt is materially increased and probably the difference loses its significance when we recall that the achieve- ments of the two groups of students were not measured at the be- ginning of the experimental period. When we consider the summaries for the different subjects, we find the differences between the average achievements of the pupils in the two types of classes materially larger in a number of cases than the differences between the averages for the 67 pairs of classes. How- ever, in interpreting these differences it is necessary to remember that the number of pupils is materially less and, hence, a difference must be materially greater in order to be significant. It is perhaps significant that negative differences predominate. This suggests that the average achievements in the small classes are somewhat superior to those in the large classes, but in drawing conclusions from this condition it is necessary to bear in mind the fact that the achieve- ments of the students were not measured at the beginning of the experimental period. When we turn to Table VIII, which is restricted to those classes in which the large class is at least twice the size of the small class, we find that the pupils in the small classes were slightly superior in general intelligence. If the size of class is a potent factor in de- termining the achievements of pupils, we should naturally expect to find a greater difference in the achievements of the two types of classes than we found in Table VIII. The fact that the difference 34 is less, both absolutely and relatively, suggests that the size of class is not a potent factor in determining the achievements of pupils. However, the number of pairs on which Table VIII is based is so small that no great importance should be attached to this observa- tion. When we examine Table IX, we find that the differences be- tween the averages for the 67 pairs of classes are only slightly larger than those given in Table VII. In general, there are few significant differences to be noted in a comparison of these two tables. One of the most significant is the reversal of the relative achievements of the large class and the small class in Latin. In Table VII, when all pupils were considered, those taught in the large class were shown to be distinctly superior in achievement. When only the paired pupils were considered, those taught in the small class were dis- tinctly superior in achievement. Considering the table as a whole, we are justified in asserting that it tends to corroborate the interpre- tations suggested for Table VII.* Conclusion: relation of size of class in high school to school efficiency. The tables of this chapter show that at the end of the experimental period the achievements of the students in the two types of classes were approximately equal, and there is a slight indication that those taught in small classes were superior. Since the educational investment can be materially decreased by increas- ing the size of class in the high school, one might infer that the effi- ciency of the school would be increased by organizing classes enroll- ing from 35 to 40 students instead of classes enrolling from 20 to 25. In addition to the fact that there are several uncontrolled factors whose influence is unknown, it is necessary to bear in mind the exact conditions of the experiment. Since the same teachers taught both ^In his report, Mr. Stevenson attempted the further analysis of the data by ascertaining the percent of pairs of pupils in which the pupil in the small class re- ceived a higher grade than the corresponding pupil in the large class. When all pairs of pupils are considered, he shows that only in 40 percent of the pairs does the pupil in the small class surpass his mate in the large class. However, it is impossible to draw any conclusion from this fact, because we do not know the per- cent of pairs in which the two pupils received the same mark. Since, in a majority ■of the cases, the grades were reported in terms of letters and only five marks were recognized, it is reasonable to expect that in a relatively large percent of the cases both of the paired pupils received the same final grade. 35 a small class and a large class, there was no difference between the total amount of work done by the teachers who handled the large classes and the teachers who handled the small classes. In fact, they were the same teachers. Thus, this experiment failed to set up the conditions of large classes as a general plan of organization of a high school. It did, however, realize the conditions which not infrequently exist in the smaller high schools where it is desirable to have a few large classes assigned to teachers who are given compensating small classes or who have the number of classes reduced accordingly. The results of the experiment, therefore, can be applied only to those situations in which the teaching load is kept constant. In such cases the evidence collected indicates that approximately the same aver- age achievement can be expected from the pupils taught in large classes as from those taught in small classes. In other words, the results of this experiment indicate that there is no loss of efficiency caused by organizing a few large classes if the other work assigned to the teacher is such that the teaching load is not increased. One should recognize that the results of this experiment should not be applied to the question of the size of class where increasing the size of class results in a distinct increase in the teaching load. The instruction which students receive is given partly in the class room and partly through written work and individual conferences. In such subjects as English composition, algebra, and science re- quiring laboratory work, it is customary with most teachers to re- quire a large amount of written work. A teacher who gives instruc- tion to five classes of 40 students each has a much heavier teaching load than the teacher who instructs five classes of 20 students each, unless he introduces compensating changes in the amount of written work, in the method of handling it, and in the number of individual conferences. In such cases the question of class size is so intimately connected with the method of instruction that we are not justified in drawing any inferences from an investigation in which the method of instruction was assumed to be the same for both types of classes. 36 CHAPTER V SUGGESTIONS FOR EDUCATIONAL EXPERIMENTATION^ The two studies described in Chapters III and IV make very sHght contributions to a scientific determination of the relation be- tween the size of class and the efficiency of a school system. They do, however, seem to the writer of this chapter suggestive with ref- erence to the procedure of educational experimentation. The causes of the failure of these studies to produce reliable and significant re- sults have been mentioned in the two preceding chapters but they may be summarized under two general heads: first, failure to set up and maintain appropriate experimental conditions and, second, the lack of adequate instruments for measuring the achievements of pupils. A prerequisite for setting up and maintaining appropriate ex- perimental conditions is a complete analysis of the problem being studied. The various factors involved must be recognized by the experimenter and the possibility of any relations which may exist between these factors must also be considered. For example, in the high school many factors contribute to the achievements of students, or the educational output of the school. In addition to the size of class, which is the factor whose relationship to school efficiency was studied, it is necessary to recognize methods of instruction, the per- sonality and enthusiasm of the teacher, the discipline of the class and of the school, the general spirit of the school, the general atti- tude of the community toward the school, the time of day when the class recites, the textbooks used, the equipment, including the build- ing, the "spiritual composition" of the class, the general intelligence of the students, their nationality, their past experience, both in school and out of school, the stage of advancement reached in their educa- tion, and possibly even other factors. It appears likely that certain of these factors are interrelated. The size of class is likely to affect the enthusiasm of the teacher, particularly if it determines the in- ^This chapter has no counterpart in the report submitted by Mr. Stevenson. It is entirely the contribution of the present director of the Bureau of Educational Research. 37 structlonal load carried by the teacher. It also appears that some relationship exists between the size of class and methods of instruc- tion, and between the size of class and discipline. The ex- istence of a functional relationship between two or more factors makes it impossible under normal conditions to produce variations in one factor without, at the same time, causing changes in the others. Failure to analyze the problem sufficiently will frequently cause the results of an educational experimentation to have little significance, and consequently the time and money invested in the study will be largely wasted. When one considers the total product of the educational pro- cess one cannot fail to become impressed with the inadequacy of our present educational tests as instruments for the measurement of the various elements of this product. In the study relating to the size of class in the elementary school a more elaborate group of tests might have been used, but even if this had been done, it does not appear likely that one would be justified in asserting that the total product of education had been measured. In the high school no standardized educational tests were used. It was attempted to have the achievements measured by means of a final examination and by the term "grade" given to the students. The writer of this chapter is not aware of the considerations which resulted in the decision not to use any of the standardized educational tests that have been de- vised in the field of high school subjects, but it is likely that this de- cision was due to the belief that none of the available educational tests were sufficiently satisfactory measuring instruments to justify their use in this investigation. The present writer is inclined to share this belief. Thus, we cannot escape the conclusion that at the pres- ent time we do not have available instruments for measuring the outcomes of teaching which permit reliable educational experimenta- tion when it is necessary to measure the total product of instruction. Incidentally, attention may be called to the fact that more con- sideration should be given to the errors involved in the data and to the effect of these errors upon the results of statistical calculations. For example, it is highly important to know what significance should be attached to a difference between two averages. In view of the difficulties encountered in setting up and main- taining appropriate experimental conditions and in view of the im- 38 perfections and limitations of our present educational tests, it is not inappropriate to question the wisdom of undertaking such com- plex educational experimentation as has been described in this mon- ograph. It is true that there are many educational problems which are highly important. For example, an increase in the size of class would result in a material reduction in the educational expenditures for instruction. A reliable scientific determination of the relation- ship existing between the size of class and school efficiency would be a valuable contribution, but it is doubtful whether such a deter- mination of this relation is at the present time possible. To the present writer, it appears highly important that those engaged in educational research should give very careful considera- tion to the sort of problems to which they devote their energies. It is, of course, necessary that pioneer work be done, and in studies of this type, it is not always possible to anticipate the limitations of one's procedure. As a result it may become necessary to "scrap" a project because of the defects that appear in the course of one's work. Such losses are unavoidable in extending the frontiers of ed- ucational research. When an investigation is not pioneer work an experimenter should determine the limitations of his procedure in advance, and unless it appears likely that information of considerable value will be secured in spite of the limitations the investigation should not be undertaken. The fact that a problem is important does not justify its study. Educational experimentation which in- volves the use of faulty method and faulty instruments not only fails to make adequate contributions to our educational progress, but, more important, it tends to reflect unfavorably upon the ap- plication of the methods of research to the field of education. 39 THE UNIVERSITY OF ILLINOIS THE STATE UNIVERSITY URBANA DAVID KINLEY, Ph.D., LL.D., President The University Includes the Following Departments The Graduate School The College of Liberal Arts and Sciences (Ancient and Modern Languages and Literatures; History, Economics, Political Science, Sociology, Philosophy, Psychology, Education; Mathematics; Astronomy; Geology; Physics; Chemistry; Botany, Zoology, Entomology; Physiology, Art and Design) The College of Commerce and Business Administration (General Business, Banking, Insurance, Accountancy, Railway Administration, Foreign Commerce; Courses for Commercial Teachers and Commercial and Civic Secretaries) The College of Engineering (Architecture; Architectural, Ceramic, Civil, Elec- trical, Mechanical, Mining, Municipal and Sanitary, Railway Engineering, and General Engineering Physics) The College of Agriculture (Agronomy; Animal Husbandry; Dairy Husbandry; Horticulture and Landscape Gardening; Agricultural Extension; Teachers' Course; Home Economics) The College of Law (Three-year and four-year curriculums based on two years and one year of college work respectively) The College of Education The Curriculum in Journalism The Curriculums in Chemistry and Chemical Engineering The School of Railway Engineering and Administration The School of Music (four-year curriculum) The Library School (two-year curriculum for college graduates) The College of Medicine (in Chicago) The College of Dentistry (in Chicago) The School of Pharmacy (in Chicago; Ph.G. and Ph.C. curriculums) The Summer Session (eight weeks) Experiment Stations and Scientific Bureaus: U. S. Agricultural Experiment Station; Engineering and Experiment Station; State Laboratory of Natural History; State Entomologist's Office; Biological Experiment Station on Illinois River; State Water Survey; State Geological Survey; U. S. Bureau of Mines Experiment Station. The library collections contain May 1, 1922, 523,230 volumes and 120,151 pam- phlets. For catalogs and information address THE REGISTRAR Urbana, Illinois LIBRARY OF CONGRESS 029 456 297 7 ^ BULLETINS OF THE BUREAU OF EDUCATIONAL RE- SEARCH, COLLEGE OF EDUCATION, UNIVERSITY OF ILLINOIS, URBANA, ILLINOIS. Prite. No. I. Buckingham, B. R. Bureau of Educational Research, Announcement, 1918-19 15 No. 2. First Annual Report 25 No. 3. Bamesberger, Velda C. Standard Requirements for Memorizing Literary Material 50 No. 4. Holley, Charles E. Mental Tests for School Use. (Out of print) 50 No. 5. Monroe, Walter S. Report of Division of Educational- Tests for 1919-20 .25 No. 6, Monroe, Walter S. The Illinois Examination 50 No. 7. Monroe, Walter S. Types of Learning Required of Pupils in the Seventh and Eighth Grades and in the High School 15 No. 8. Monroe, Walter S, A Critical Study of Certain Silent Reading Tests 50 No. 9. Monroe, Walter S. Written Examinations and Their Improvement. (In -preparation) 50 No. 10. Bureau of - Educational Research. Relation of Size of Class to School Efficiency 5°