LI BR A FLY OF THE UNIVERSITY or ILLINOIS 3TO Return this book on or before the Latest Date stamped below. University of Illinois Library '± ...J-i n-- c;,(: in^L H^K maid y^ h^ ^ > ^oW "' f'J HI - • If) [- ^' O 1 if.' :O0 'JUN 3 1983 JULl HA 21983 ^ 8 2006 LI61 — H41 Digitized by the Internet Archive in 2011 with funding from University of Illinois Urbana-Champaign http://www.archive.org/details/predictingschola37odel BULLETIN XO. 37 BUREAU OF EDUCATIONAL RESEARCH COLLEGE OF EDUCATION PREDICTING THE SCHOLASTIC SUCCESS OF COLLEGE FRESHMEN By Charles W. Odell Assistant Director, Bureau of Educational Research PRICE 25 CENTS PUBLISHED BY THE UNIVERSITY OF ILLINOIS, URBANA 1927 TABLE OF CONTENTS PAGE Preface 5 Chapter I. Introduction and Statement of the problem . 7 ' Chapter II. A brief Review of What Has Already Been Done 10 1 Chapter III. The General Plan of This Study .... 20 Chapter IV. The Simple Correlations Between Freshman Marks and the Other Data Collected 28 W Chapter V. The Multiple Correlations Between Freshman Marks and the Other Data Collected .... 34 I Chapter VI. The Accuracy of Predictions Based upon the Obtained Coefficients of Correlations .... 40 I Chapter VII. Is the Change from High School to College Greater than that from Elementary to High School? 48 Chapter VIII. Summary and Conclusions 52 PREFACE The prediction of the scholastic success of college freshmen is com- manding the attention of many persons, especially those who are respon- sible for the administration of colleges and universities. It has been proposed that by making use of a student's high-school record and by administering an intelligence test it would be possible for an institution to predict the probable success of an entering student in the various subjects of instruction. It is, of course, generally recognized that such predictions would not be accurate in all cases. Some authorities contend that, if sufficient information is secured and the prediction is made in- telligently, it will be sufficiently accurate to be very helpful in guiding the student when he enters college. Other authorities maintain that in general the prediction will be so inaccurate that it will not be very useful. In this bulletin Dr. Odell presents the results of a very careful in- quiry into the accuracy of predictions that may be made using certain information. Although knowing the probable accuracy of the predic- tions that may be made does not determine the value of such predic- tions, it should be helpful to college administrators to know the probable accuracy of the predictions they may make. Hence, it is believed that Dr. Odell's study constitutes a significant contribution In the field of college administration. Walter S. ]\ Ion roe, Director May 4, 1927 [5] PREDICTING THE SCHOLASTIC SUCCESS OF COLLEGE FRESHMEN CHAPTER I INTRODUCTION AND STATEMENT OF THE PROBLEM The recent increase in college enrollment and two resulting problems. In a previous publication^ the present writer has called at- tention to the fact that one of the most notable and significant recent educational tendencies in this country has been the marked increase in school enrollment, especially on the secondary and higher levels. Such a tendency has, it is true, existed from the beginning of our educational system, but, since sometime near the end of the Nineteenth Century and even more since the close of the World War, it has been greatly accentu- ated. This may be seen from the last Statistical Summary of Education- issued by the United States Bureau of Education, which shows that at present between three-fourths and one per cent of our whole population is enrolled in college, whereas in 1890 only about one-fourth of one per cent was so enrolled. The figures given also show that the last five-year period has exhibited a much greater increase than any other of similar length. Concurrent with the tendency just stated have been a considerable decline in the purchasing power of the dollar and a general demand that the scope of education be enlarged. The united effect of these three factors has been such that it is practically impossible to secure the amounts of money necessary to provide what are considered adequate educational facilities for all those who wish to enjoy them. The diffi- culty of doing so appears to be greater in the field of higher education than in any other. One outstanding question, which has arisen in connection with the crucial situation just described, is that of whether or not institutions of higher education shall open their doors to practically all those who have completed a secondary course and wish to enter. The general tendency has been for state supported institutions to approximate doing 'Odell, C. W. "Are college students a select group. '"■" University of Illinois Bulle- tin, Vol. 24, Xo. 36, Bureau of Educational Research Bulletin Xo. 34. Urbana: Uni- versity of Illinois, 1927. 45 p. ^Phillips, Frank M. '"Statistical summary of education 1923-24." U. S. Bureau of Education Bulletin, 1926, Xo. 19. Washington, 1926. 7 p. [7] so, whereas those deriving their support from other sources exercise varying degrees of selection among the appHcants for admission. These policies are rarely based upon any thoroughgoing study of the problem and never upon conclusive evidence as to the best practice, so that ■what should be done may be considered still an open question. The data available^ appear to warrant the statement that at present the group of those who actually enter college represents a marked selection of all high-school graduates, but that it still contains many individuals who apparently cannot carry the usual type of college work successfully. On the Avhole. therefore, it seems desirable, perhaps even necessary, that if colleges* are to continue to maintain their present scholastic standards, some degree of selection among applicants for admission should be exer- cised. Such an assumption naturally raises the question as to what is the most desirable basis of making this selection. In other words, do any of the data which are fairly readily obtainable concerning high- school graduates provide a satisfactory, or even a helpful, basis of fore- telling scholastic success in colleger If so, which of these data are most valuable for this purpose and how much confidence should be placed in their use.' A second question arising, in part at least, from the same cause and one which has attracted much attention recently is that of providing for college students of different aptitudes and abilities. This question really divides itself into two parts. In the first place, if the amount of selection at college entrance is not very great it will undoubtedly result that many of those who are allowed to enter can, or at least will, not do satisfactory work in certain subjects, whereas in others they will do passing or even superior work. The college is therefore confronted with the need for providing educational guidance for such students. This requires, if possible, the determination of the subjects or courses in which the students will succeed and those in which they will fail. Even if entrance requirements are decidedly severe and many of those seek- ing admission are barred, educational guidance of the sort just men- tioned is still desirable though the necessity for it is not so acute. In the second place, there has recently been considerable interest in the matter of offering different types or levels of instruction within single subjects and otherwise varying the educational opportunities given stu- dents of different abilities. It is true that this problem has received much more attention in elementary and high schools than in colleges, ^Odell, op. cit., p. 26-29. *The term "college" will be used frequently as a general term including all types of institutions of higher learning. [8] but an increasing number of the latter are giving it serious consideration. In this case also, the less selection there is among applicants for admis- sion to college the greater is the need for attention after admission be- cause the group admitted is more heterogeneous. Even if a relatively high degree of selection is exercised among those who seek to enter col- lege, however, those who gain admission and who enroll for any partic- ular subject practically never constitute a truly homogeneous group. Therefore, there is need to determine the validity of various bases which may be employed for classifying students in advance according to the different amounts and kinds of subject matter, types of instruction, and so forth, which seem best suited to them. The purpose of this bulletin. It is the purpose of this bulletin to present a study and evaluation of some of the more readily available items of information which may be, and in many cases are, used to pre- dict the probable scholastic success of college students. After reviewing briefly a number of studies illustrative of what has already been done in the field, the writer will give an account of one^ along this same line which he has been carrying on. This investigation differs from most of the others in the same field in that the attempt has been made not merely to determine the accuracy of prediction of college success in general, but also for each subject carried by any considerable number of the individuals included. It is limited by the fact that the college data upon which it is based include only records for the freshman year. Its purpose may, therefore, be stated as being to show how accurately the marks of college freshmen in their various subjects can be pre- dicted when their ages, scores upon an intelligence test, and complete high-school records are available. The problem will be attacked pri- marily by the methods of simple and multiple correlation and the ac- curacy of predictions based on the best multiple regression equations obtainable will be shown. ^The first part of this study, which inchided only the data obtained while the in- dividuals embraced were still in high school, has been presented in the following bul- letin: Odell, C. \V. "'Conservation of intelligence in Illinois high schools." University of Illinois Bulletin, \'ol. 22, No. 25, Bureau of Educational Research Bulletin No. 22. Urbana: University of Illinois, 1925. 55 p. A second portion which deals with the question of how great a selection occurs among college entrants as compared with high-school graduates has been dealt with in the following publication: Odell, C. W. "Are college students a select group?" University of Illinois Bulle- tin, Vol. 24, No. 36, Bureau of Educational Research Bulletin No. 34. Urbana: Uni- versity of Illinois, 1927. 45 pp. The present bulletin is the third in the series. [9] CHAPTER II A BRIEF REVIEW OF WHAT HAS ALREADY. BEEN DONE The extent to which intelligence tests have been used in institu- tions of higher learning. Since most of the recent studies dealing with the prediction of scholastic success in college have employed intelligence test scores as the chief criterion, it seems in place to mention several studies which show something of the extent to which intelligence tests have been employed in college, both for this and other purposes. Here and later no attempt will be made to refer to all of the investigations which have been reported, but only a few of the most significant or typ- ical ones will be mentioned in each case. Bridges/ early in 1922, re- ceived answers from 42 of 70 institutions to which he had sent inquir- ies and found that although 3 1 of the 42 had made some use of group Intelligence tests only a few had done so in connection with determining admission. Apparently, in many cases, the tests were administered with no very definite purpose in mind. A year and a half later Laird and Andrews'- reported that 26 out of 64 institutions made some use of tests as part of the routine process of determining the admission of ap- plicants and that others used tests for such purposes as sectioning classes, determining the amount of work to be carried, giving vocational and educational guidance, deciding upon the elimination of students, and dealing with disciplinary cases. Probably the most detailed report of the use of intelligence tests in colleges is that by MacPhail,^ which ap- peared some three years ago. In this he summarized briefly almost every article dealing with this topic and showed that in many institu- tions intelligence tests played a definite part in the admission of appli- cants as well as in other questions of policy. A more recent study by Toops* reported that 66 out of 110 institutions answering a question- naire employed intelligence tests during the year 1923-24. Xone of these 'Bridges, J. W. "The value of intelligence tests in universities," School and So- ciety, 15:295-303, March 18, 1922. 'Laird, D. A., and Andrews, A. "The status of mental testing in colleges and universities in the United States," School and Society, 18:594-600, November 17, 1923. 'M.a.cPh.4il, a. H. The Intelligence of College Students. Baltimore: \\'ar\vick and York," 1924. 176 p. *Toops, H. A. "The status of university intelligence tests in 1923-24," Journal of Educational Psychology, 17:23-36, 110-24, January, February, 1926. [10] based admission entirely upon test results, but 19 used them as a partial basis. Forty-nine took them into account in determining dismissal for low scholarship, 34 in determining probation, 36 used the results in de- termining the amount of work students should carry. 25 in selecting and encouraging bright students to take graduate work, 42 in motivating the work of bright students, and various numbers in selecting assistants, appointing scholars and fellows, and so on. From the studies referred to above it will be seen that intelligence testing is apparently well established in many institutions of higher learning and that the results receive large use in a number of matters having to do with guidance, instruction, and other direction of students, as well as to a somewhat lesser degree with their admission. So far as the WM'iter knows, no institution has yet based admission upon intelli- gence test scores alone, though for certain classes of applicants a few colleges make them the chief criterion."' Summary studies of the relationship of intelligence test scores and other criteria to college marks. Several of the studies mentioned and a number of others present data showing the degree of relationship found between college marks and intelligence test scores, high-school marks, and other items of information. Before considering a few re- ported investigations in greater detail it seems well to give a brief pic- ture of general tendencies. Terman,'' reporting on 25 colleges, found co- efficients of correlation" running from .29 to .83 between test scores and college marks, whereas those between the latter and high-school marks ranged from .38 to .74, and those between them and college entrance examination results from .25 to .62. Incidentally, he states that the Thorndike Intelligence Examination is probably the best of those avail- able for the purpose of predicting scholastic success in college. Roberts^ reports similar ones of .31 to .60. also coefficients between college and I high-school marks of .53 to .69 and between the former and college en- trance examinations of .25 to .62. He makes this statement, "'Combining intelligence scores with all other good measures, the exceedingly high correlations of .75 to .80 are obtained between these measures and "This refers chiefly to the admission or rejection of applicants who have not com- pleted the required secondary school work and who are also above the usual age. *Terman, L. M. "Intelligence tests in colleges and universities," School and So- ciety. 13:481-94. April 23, 1921. 'The meaning and interpretation of coefficients of correlation of \'arious sizes is discussed in Chapter M. ^Roberts, A. C. "Objective measures of intelligence in relation to high-school and college administration," Educational Administration and Supervision. 8:530-40, Decem- ber, 1922. [11] scliool marks.'' He also writes, ''The intelligence scores have shown themselves our surest guide in detecting the very highest and the very- lowest of intellectual ability." MacPhail^ lists about 60 correlations be- tween test scores and college marks, ranging from .13 to .71. The use of intelligence tests at Brown University. Due to the work of Colvin, assisted b}' MacPhail and others. Brown University has for about ten years been among those institutions making the most ex- tensive and careful use of intelligence tests in connection with the ad- mission of students and also, though perhaps to a lesser degree, in con- nection with their guidance and direction after entrance. Not only have intelligence tests played a prominent part in determining the admission of freshmen at Brown University, but also a number of articles have ap- peared describing their use for this purpose. Therefore it seems fitting to select this institution as the example which will be described in more detail than any other as an illustration of what Is being done. The work along this line began during the time of the World War and by 1919 Colvin^'^ reported on the first two or three years' use of tests. At this time he stated that different intelligence tests correlated from about .40 to .60 with freshman marks, and that of the students who did unsatisfactory or unusually good work about two-thirds were indicated by the test scores. On the whole the results were considered sufficiently satisfactory to warrant continuing the use of tests. A year later another article^^ by the same writer gives about the same correla- tions as before, those for the Brown University Psychological Examina- tion and the Thorndike Intelligence Examination being a number of points higher than those for Army Alpha and also being on the whole higher than the corresponding correlations for high-school marks or teachers' estimates. The test results appeared to pick out the superior and inferior students with more accuracy than the average ones. When the Brown and Thorndike scores were averaged 90 per cent of the low- est tenth were found to have failed in one or more subjects. In 1922 Colvin and MacPhaiF- replied to some unfavorable criti- cisms of the use of intelligence tests in college and gave further data concerning their use at Brown University. Most of these merely sub- stantiate previous statements, though in some cases they are presented 'MacPhail, op. cit., p. 29. '"Colvin, S. S. "Psvchological tests at Brown University," School and Society, 10:27-30, July 5, 1919. "Colvin, S. S. "Validity of psychological tests for college entrance," Educational Review, 60:7-17, June, 1920. ' "Colvin, S. S.. and MacPhail, A. H. '"The value of psychological tests at Brown University," School and Society, 16:113-22, July 29, 1922. [12] in a different form. The writers state, for example, that low test scores furnish a more reliable prediction that college work in general will be poor than do low marks made during the first semester but that a com- bination of the two is better than either one alone. Of college honors 80 per cent went to those earning high test scores, 19 per cent to those with medium scores, and only 1 per cent to those with low scores. A'lore recently Burwell and MacPhaiP' have written upon the same topic. They report that the procedure has been changed somewhat by giving the Brown test to all freshmen and the Thorndike test only to the lowest fifth, in place of giving both to all freshmen as had been done for several years. Among the statements made are that "new students who will probably fail in two or more subjects in either semester during their first year in college are far more likely, roughly speaking ten to twenty times more likely, to be found among those who make low psychological scores than among those with high ratings;" that "a freshman whose psychological score places him in the lowest decile has only two chances out of five of remaining more than one year in college and only one chance out of five of graduating;" and, finally, that "the majority of honor men are to be sought among those scoring in the best psychologi- cal third; most of the remainder may be expected to come from the middle third; and a very few (about one out of twenty) from the lowest third." Forty-six has been set as a critical score on the Brown Univer- sity test above which a student must rate to indicate that he will prob- ably receive no grades below "C" during the first semester. It appears that those who have been using the tests at Brown are very firmly convinced of their value. However, they recognize and point out certain limitations and indicate that it is highly desirable to have other data to supplement the test results, but apparently regard them as the one most important criterion for predicting scholastic suc- cess in college. The use of tests at Columbia University. Columbia University, also, has made rather extensive use of intelligence tests in connection with admitting students. Accounts of the work have been given by ^^ ood,^'* Thorndike. '■' and others. The experiments there appear to have begun in 1919. At that time faculty action was taken providing two pos- sible methods of entrance, one of which was the old method based upon "Burwell, W. R.. and MacPhail, A. H. '"Some practical results of psychological testing at Brown University," School and Society, 22:48-56. July 11, 1925. "Wood, B. D. Measurement in Higher Education. New York: World Book Com- pany, 1923, Chapters U-V. "Thorxdike, E. L. '"On the new plan of admitting students at Columbia Univer- sity," Journal of Educational Research, 4:95-101, September. 1921. [13] entrance examinations in high-school subjects, previous school records, health records, and estimates of character and personality. The second method substituted intelligence tests for the subject-matter examinations included in the first. For purposes of record all those desiring to enter by the first plan as well as those entering by the second are given the Thorndike Intelligence Examination. Many different sets of figures are given to indicate the validity of this test as used at Columbia for fore- casting success in college work. The correlations between test scores and college marks average around .65 and are distinctly higher than those of the latter with college entrance examinations, New York Re- gents' examinations, and still more so than those with secondary-school marks. The correlations obtained for the test results are probably in- creased somewhat because no applicant for admission is allowed to take the test unless the data concerning him on the other three points men- tioned are satisfactory. The same is, however, true of those admitted wath examinations covering high-school subjects as one of the criteria and doubtless raises the correlations there also. In conclusion it may be said that the use of intelligence tests, as one of the bases for de- termining admission to Columbia University, has become an integral part of the procedure and is no longer considered an experiment. The use of intelligence tests at the University of Minnesota. The reports^''' from this institution are not as favorable to intelligence tests as those from Brown and Columbia Universities. It appears that high- school marks, the kind of work carried in high school, and marks on three themes at the beginning of the freshman year at the University were all more reliable in indicating students whose university work was poor than were the scores made on a mental test. \\ hen the latter were combined with the former, a correlation of about .70 was obtained. It is pointed out that in most cases of marked discrepancy between the work actually done and the predictions made from the combined cri- teria explanations can be found when the individual cases are studied. What has been accomplished at the University of Minnesota may be summarized as follows: a threshold has been fixed such that only 1 per cent of those falling below it will prove successful in college work; the procedure can be explained to students and all others interested; students of unusual ability can be located; a beginning of vocational se- "JoHNSTON, J. B. ''Predicting success in college at the time of entrance," School and Society, 23:82-88, January 16, 1926. JoHN-STOx, J. B. ""Predicting success or failure in college at the time of en- trance." School and Society, 19:772-76, June 28, 1924; 20:27-32, July 5, 1924. Johnston, J. B. '"Tests for ability before college entrance," School and Society, 15:345-53, April 1, 1922. [14] lection has been made; promising students not in college can be selected and encouraged to attend; college failures can be treated much more adequately; students who need special advice can be selected and given this advice; and finally each student guided "so far as possible into that line of eff^ort in which his native ability will find Its most complete ex- pression." The use of intelligence tests at other institutions. In view of the fact that there is great similarity between the results reported from most of the institutions which have employed intelligence test scores as one of the criteria for determining admission, it seems not worth while to refer to reports from more than a few difi"erent institutions. Those which are mentioned in this section were chosen partly more or less at random and partly because the results obtained were in some way different from the general trend. The results reported from the University of Pittsburgh^" are dis- tinctly lower than those given previously. In this case college marks correlated only .41 with Army Alpha Scores, as compared with .32 with first semester marks. These correlations were undoubtedly lowered somewhat by the fact that the individuals included in the study were more highly selected than an ordinary freshman class, and also by the fact that the Army Alpha Test seems, on the whole, not to predict scholarship as well as do the Thorndike, Brown, and several others. Differing much from this is an unusually high correlation reported from the State Normal School at Indiana, Pennsylvania.^* The National and Illinois Intelligence Tests were used and the scores correlated above .70 with educational psychology mark. May, at Syracuse University,'-' secured information as to the num- ber of hours spent in study and found that combining this with intelli- gence test score gave a multiple correlation of .83, whereas test score and high-school mark gave only .64 with honor points in college. He also found that when the amount of study was held constant the correlation between test score and honor points was .81. A study at the University of Washington'-'^ corroborates this, although it does not present its re- sults in just the same way. Wilson, who reports it, concludes that the "Ernst, J. L. "Psychological tests vs. the first semester's grades as a means of academic prediction," School and Society, 18:419-20, October 6, 1923. "Rich, S. G.. and Skinner, C. E. ''Intelligence among normal school students," Educational Administration and Supervision, 11:639-44, December, 1925. "May, M. a. '"Predicting academic success," Journal of Educational Psychology. 14:429-40, October. 1923. ^"Wilson, W. R. '"Mental tests and college teaching." School and Society, 15:629-35, June 10, 1922. [15] TABLE I. COEFFICIENTS OF CORRELATION BETWEEN THORNDIKE TEST SCORE AND FIRST SEMESTER COLLEGE MARKS Biology 51 i History 46 Chemistry 43 Human Progress 69 English 36 i Mathematics 52 French 42 Physics 50 German 50 Public Speaking 46 Graphics 35 Spanish 57 failure of intelligence tests and college marks to correspond more closely is largely accounted for by the differences in the amounts of time spent in study, especially by the fact that, on the whole, bright students study less than do dull ones. Studies showing correlation in particular subjects with intelli- gence test scores and other data. Most of the many studies made have correlated the various criterion measures with college averages, only a few dealing with marks in particular college subjects. Of the few, two which correlated test scores with college marks and one which used high-school marks instead of test scores will be mentioned. One-^ of the first two was made at the University of Pittsburgh and yielded the average coefficients of correlation between score on the Thorndike test and freshman college marks for the first semester shown in Table I. The correlation of the test result with the general freshman average for the first semester was .51. Root, who reports the study, concludes that test results are decidedly valuable for predicting academic success, but that they are only one of the needed items of information. He points out that if the criterion for admission to the university were taken as being the lower limit of the middle group upon the tests, all applicants scoring above that point being admitted and all below rejected, about one-third of the students would be excluded or admitted improperly, that is, ex- cluded when they could do satisfactory work or admitted when they could not. The other-- of the two studies does not give tables of the exact coefficients of correlation, but summarizes the results found from corre- lating Otis test score with college marks as follows: 'Tn all cases the correlations are positive. In all cases on the average the pupils who stand high in the test stand high in scholarship; those who stand low on the test stand low in scholarship, and those who stand in the middle "Root, \\'. T. "The freshman: Thorndike college entrance tests, first semester grades, Binet tests," Journal of Applied Psychology, 7:77-92, March, 1923. '"JoRDAX, A. M. "Student mortality.'" School and Society, 22:821-24, December 26. 1925. [16] TABLE II. COEFFICIENTS OF CORRELATION BETWEEN HIGH- SCHOOL AND COLLEGE MARKS IN CERTAIN SUBJECTS High -School Subjects College Subjects Eng. Chem. Alg. Geom. Lat. Elem. French. Adv. French English Chemistry .28 .21 .22 .28 .40 .39 .19 .20 .19 .34 .31 .26 .18 .25 .18 .21 .41 .38 .43 .26 .31 .19 .23 .41 .34 .14 .03 .28 .23 .23 .26 .30 .14 .32 .39 .25 .28 .41 .38 ^41 .37 .31 .26 Algebra Analytic Geom Elem. French Adv. French German .33 .40 ^35 .43 on the test are in the middle in scholarship. But in some cases the re- lationship is quite low, while in other cases it is moderately high. In no case is there a high coefficient of correlation between the test and the marks in any subject. With German during the first year the relation- ship is quite respectable, but, not even here, high enough for prognosis. The coefficients are quite substantial (from .45 to .61), then, between the Otis test and the marks in German, English, history, geology, and French for the first year; present but low (.32 to .39) in the case of mathematics, chemistry, Spanish, economics, engineering, and Latin. During the second year the coefficients are substantial in English and Spanish; present but low in French, history, economics, engineering, and German; and negligible in mathematics, chemistry, geology, Latin, and zoology. However, the coefficients of correlation during the second year are necessarily lower because of the contraction of the range of scores (the lowest have largely disappeared). The correlations with average and total grades are marked." Jordan also states the correlations ob- tained between high-school and college marks. These varied from .37 to .59 and on the average were quite similar to the coefficients between college marks and test score. The closest relationships appeared to be in economics, Spanish, and French. Using multiple correlation with combined test score and high-school average mark he obtained a coeffi- cient of .58 with the university average for two years. The conclusion from his study is, therefore, that there is little difference in prognostic power betwen the score on the Otis Group Intelligence Scale, Advanced Examination, and the high-school mark, although the correlations of the former with college marks are lower than those Root found with the Thorndike score. This latter fact is undoubtedly due, at least in part, to the fact that the Thorndike test is considerably longer than that [17] of Otis and so yields a more satisfactory measure for the purpose here discussed. The third study referred to'-^ was conducted at the University of Alaine and dealt with the correlation of high-school and college marks in particular subjects. The correlations found are given in Table II. These coefficients seem to warrant the conclusion that correlations be- tween high-school marks and college freshman marks in single subjects above .40 are rare and that the central tendency of such correlations is not far from .30. This, however, is not supported by the results ob- tained by Jordan, whose corresponding coefficients averaged about .20 higher. The other studies of this sort available tend to yield correlations of about .40 to .50 or .55 between test scores and marks in single college subjects and about the same between high-school and college marks. Most of them are based on smaller numbers of cases covering only a few subjects and are hardly worth mentioning separately. Summary. The work which has been done up to date in attempt- ing to predict the scholastic success of college students by means of intelligence test scores and other criteria may be briefly summarized in the following statements. In a considerable number of institutions of higher learning, including several of the largest ones in this country, the use of intelligence test scores as one of the criteria for admission has passed the experimental stage and is now a settled policy. In many other institutions much use has been made of intelligence tests, but it has had little or no connection with the admission of students. Although the correlations reported vary from near zero up to .70 or above, a range of .40 to .50, or perhaps somewhat higher, may usually be ex- pected between score on an intelligence test and freshmen mark. These correlations are about the same as those of high-school with freshman marks and both slightly higher than those given by entrance examina- tions covering high-school subjects. The true relationship in the latter case is, however, somewhat closer than indicated by the obtained co- efficients of correlation, because applicants for admission making low marks are generally rejected and thus the range decreased and the ap- parent correlation lowered. If one of the best tests is employed the correlations with freshman mark will probably be higher than will those of the high-school average. A combination of test score and high- school mark may be expected to yield correlations of about .60 or higher. ^^GovvEX, J. W'., and Gooch, M. "The mental attainments of college students in relation to pre\ious training," Journal of Educational Psvchologv, 16:547-68. Xovember. 1925. [18] If it is possible to include an adequate measure of study habits also the coefficients will probably rise to near .80. Comparatively little has been done in attempting to predict scholastic success in single college subjects, but apparently the correlations found for single subjects are on the whole not much, if any, below those found for the general freshman average and apparently here, also, a test score gives about the same accuracy of prediction as does a high-school mark. Perhaps the most important conclusion is that the whole problem needs much more care- ful and extensive investigation, especially along the line of finding the best type or types of entrance examinations covering the high-school subjects and of combining the results therefrom with other data to give the best multiple predictions possible. [19] CHAPTER III THE GENERAL PLAN OF THIS STUDY The initial collection of high-school data. The data used in this investigation concern a group of individuals graduated from several hun- dred high schools in the state of Illinois in 1924 and admitted to various institutions of higher learning in the summer or autumn of the same year. In the fall of 1923 all the four-year public high schools in the state were invited to cooperate with the Bureau of Educational Research in this study. The number that did so was 368, a few more than one-' half of all those within the state, and the number of seniors included was about 12,300. The data secured concerning them consisted of their scores upon the Otis Self- Administering Test of Mental Ability, Higher j Examination, Form A and the answers to the questions on an "Informa- tion Blank for High-School Seniors," which called for the following in-j formation : Name Sex Date of Birth Age on September 1, 1923 Name of school Town or city Intentions concerning further education Intention of continuing Institution Course Major subject Vocational choice Father's occupation Information as to previous intelligence tests taken Units of high-school credit High-school subjects liked most High-school subjects liked least Number of failures in high school Average high-school mark^ The tests were given by principals or by teachers designated by them] and the information blanks filled out by the seniors themselves. All scoring of test papers and tabulation of results was done in the offices^ of the Bureau of Educational Research. ^This was the average mark up to date or for the first three years. It was secured] from only a minority of the schools and for about 2700 seniors. [20] The second step in collecting high-school data. A year later, In the fall of 1924. the 368 high schools were asked to furnish the complete high-school scholastic records of all pupils for whom the other Informa- tion had been secured, and also If possible, to state what. If any, Insti- tution of higher learning each Individual was attending. A few of the seniors of the year before had not been graduated, and In a few cases the desired records were not forthcoming, but the loss from these sources was comparatively slight, so that the complete scholastic high- school records of about 11,500 graduates were secured. Since these marks came from several hundred schools which employed a total of over one hundred different marking systems, if all minor variations be counted, it was necessary to transmute them to a uniform basis. For this purpose a percentile system with passing at 70 and no conditions was chosen. The marks given according to all other plans were changed to this system by approved and careful statistical procedure. The collection of college freshman data. Some three hundred institutions of higher learning had been named by the seniors In answer to the question as to where they expected to continue their education. Early in the academic year of 1925-26 letters were addressed to all these institutions asking for the complete 1924-25 scholastic records of all freshmen coming from any of the high schools Included In this study. About 7,700 of the seniors had stated that they intended to continue their education. In addition to many who were undecided, and the ma- jority of them had named the Institutions they expected to attend. De- spite this fact the freshman records of not quite two thousand students were all that were secured. This loss is due to at least four causes. In the first place, a number of the collegiate institutions addressed either were unwilling to cooperate in the study, or, after expressing their will- ingness to do so, failed to send the desired records. A second reason was that a number of the Institutions which did cooperate failed to fur- nish the data for all of their students for whom they were desired. Third, undoubtedly many of the high-school graduates who planned to attend college found It necessary, for financial or other reasons, to post- pone entrance for a year or more after high-school graduation. The last, and probably the most Important, reason was the fact that In filling out the Information blanks the high-school seniors expressed their high- est hopes and ambitions or gave answers which they thought would sound best and that, therefore, many of them who had very slight ex- pectations of ever actually attending college, signified that they intended to do so. [21] Of the approximately two thousand students whose records were secured from various colleges, almost one hundred did not remain in college long enough to have any marks recorded. The number for whom marks for at least one quarter, term, or semester were secured was 1892, and for 1677 of these a full year's marks were obtained. As these marks were given by more than one hundred institutions it was necessary to transmute them to a common basis in the same manner as had been done for the high-school marks, and so all were adjusted to the same basis of a percentile marking system with 70 as passing and no conditions. The reliability- of the data secured in this investigation. There is no doubt that in both intelligence test scores and high-school and col- lege marks large variable errors are present. No group intelligence test so far devised yields highly accurate individual scores and the Otis Self-Administering Test, which requires only half an hour to give, is probably less reliable than one, such as the Thorndike Intelligence Ex- amination, which consumes two or three hours. Moreover, the tests were not administered by a corps of trained and selected examiners, but by several hundred different principals and teachers, many of whom had probably never given a standardized test. This fact undoubtedly served to increase the errors in the scores. It should not be overlooked, how- ever, that the test used reduces the directions to be given by examiners to a minimum and that, therefore, the errors due to lack of training of the persons giving the tests are less than would otherwise be the case. The writer does not believe, however, that this factor of added reliability is sufficient to balance the two of brevity and administration by poorly qualified examiners which make for the opposite effect. The method of computing intelligence quotients, which Otis pro- vides, introduces a constant error into many of those so determined.^ However, as little use will be made of the I. Q. in the discussion, it does not seem worth while to discuss this point further than to call attention to the fact' that the coefficients of correlation between the I. Q. and other data are probably slightly lower than they should be and. "As used in this bulletin, the term '"reliability" is practically equivalent to "ac- curacy." It is not limited to its sometime narrow technical meaning referring to the agreement between two sets of scores on the same measuring instrument, though it includes this. 'For a more complete discu,ssion of this point, see: Odell, C. W. "Are college students a select group?"' University of Illinois Bulle- tin, Vol. 24, Xo. 36, Bureau of Educational Research Bulletin No. 34. Urbana: Uni- versity of Illinois, 1927, p. 16-17. [22] therefore, the estimated accuracy of predictions made on the basis of the I. Q. is also sHghtly too low. An additional fact which probably affected the significance of the test scores was that about half of the seniors tested had never taken an intelligence test before and it is likely that many of their scores, when compared with most of those of the seniors who had taken such tests previously, do not fairly represent their mental ability. Further- more, because of the conditions under which the tests were given, there was generally no particular incentive, apart from the desire to excel, for the pupils to do their best. Hence, it is likely that a considerable number of them did not put forth maximum effort while taking the test. These and all other causes which produce variable or accidental errors in the test scores result in lowering the correlations and other predictive indices based thereon and justify the conclusion that the. real relationships are somewhat closer than those actually computed. Too much evidence and discussion concerning the subjectivity and unreliability of school marks has appeared within the last few years for the subject to need extended comment In this connection. Undoubt- edly the errors present in the marks were Increased somewhat by the fact that marks from several hundred high schools and more than a hundred colleges with different systems and standards were transmuted to a common basis and thrown into a single group. In spite of the fact that the transmutation was made with great care and followed sound statistical procedure, it was not possible. In all cases, to be sure that the transmuted marks were really equivalent to the original ones. The effect of increasing such variable errors was to lower the coefficients of correlation and other predictive measures secured. The computation of zero-order coefficients of correlation. As has been suggested, the chief method employed In determining the relation- ships existing between college freshman marks and the other data avail- able was that of correlation. It was found that there were forty-nine subjects or closely related groups of subjects* each of which had been carried by ten or more freshmen. Correlation tables were made for the mark in each of these subjects or subject groups with age, mental test score, intelligence quotient, general high-school average, and average ^In a number of cases it is doubtful just what really constitutes a "subject" as the term is commonly used. This, for example, is true of agriculture. In cases in which there were only a few freshmen who carried each of the several possible divisions the procedure followed was to group them together as a single subject. Agriculture, therefore, includes various courses in agronomy, animal husbandry, and so forth; art includes freehand drawing, painting and sculpture, and so on with others. [23] mark in each high-school subject or group of subjects with which it seemed Hkely that there was close relationship. Thus, for example, college freshman biology mark was correlated with marks in high-school biology, botany, general science, and zoology and also with the average mark in all high-school science. Likewise, that in French was corre- lated with high-school English, French, Latin, and Spanish marks, and also with the general high-school foreign language average. In addition to these correlations quite a number were made be- tween the freshman marks and the amounts of time devoted to partic- ular subjects or groups of subjects in high school and also between freshman marks and those in the work of particular years in high school. For example, the freshman biology mark was correlated with the number of semesters of biology carried in high school, also with •the total number of semesters of science carried. The freshman French mark was correlated with the number of semesters of high-school French, of high-school Latin, and of all high-school foreign language, also with the marks in first, second, third, and fourth year Latin and French, in so far as each had been carried. After such correlation tables had been made for a dozen or more of the freshman subjects it appeared that the results therefrom would contribute nothing of value to the study, so no more were constructed. The correlations of fresh- man marks with the amounts of particular subjects and groups of sub- jects carried in high school were so near zero as to offer no help in predicting freshman marks. The correlations with a particular year's work in high school were higher, but they appeared to add nothing not already contributed by those of freshman marks with marks in all the high-school work in the various subjects. In some cases they were practically as high as the latter, but the use of the multiple correlation procedure showed that they added almost nothing In accuracy of pre- diction. After constructing the tables described, the next step was naturally to compute the simple or ordinary coefficients of correlation for them. It should be remembered that, since many of these correlations involve as one variable an average mark for a group of similar high-school sub- jects or for all high-school subjects, the coefficients obtained from them are in a sense multiple coefficients although not obtained by the multiple correlation method. In other words, they show the relationship existing between college freshman marks and combinations of several different high-school marks. The computation of coefficients of multiple correlation and re- gression. The calculation of zero order or simple coefficients of corre- [24] latlon was followed by that of multiple coefficients and regression equa- tions. In view of the considerable amount of labor involved in comput- ing the latter they were not found for all freshman subjects, but for only about one-third of them. These were in general the subjects carried by the largest numbers of freshmen and two or three others included because of especial interest in them. In connection with this the admis- sion should be made that since many possible correlations were not com- puted it is probable that some were omitted which should have been found. Since the amount of money available for clerical help, though fairly generous, was not unlimited, it was necessary that the line be drawn somewhere, and it is very likely that the writer's judgment in se- lecting the most promising possibilities was not infallible. In the case of several of the freshman subjects two or three groupings were made ac- cording to the high-school subjects carried and a different set of multi- ple correlations computed for each grouping. For example, in addition to calculating the correlations and regressions for all freshmen who car- ried Latin as a college freshman subject, they were also found separateh' for the portion of this group that had carried high-school French. The general procedure in computing the multiple coefficients was to start with the highest one of zero order and combine the others of the same order with it until the addition of another criterion no longer increased the obtained coefficient by as much as .01. Because of the fact referred to above, that many of the simple coefficients of correlation were really multiple in nature though not in derivation, it could not be expected that on the whole there would be as great an increase in the multiple coefficients over those of zero order as would otherwise have been the case. The question may be raised as to why certain combinations, which will appear later in the chapter containing the multiple correlation re- sults, were made, in view of the fact that one of the simple correlations already used was that of the freshman mark with the general high-school average or the average in a group of similar subjects, and another the correlation with one of the subjects which entered into this group. For example, the highest obtained multiple coefficient for freshman rhetoric was that obtained from a combination of high-school average, high- school English mark, and point score on the test, and of course the high- school average included the high-school English mark. The reason for so doing is, however, clear to any one familiar with multiple correlation. In computing the high-school general average or the average in any group of similar subjects the marks entering into the given average were all allowed the same weight in determining it. By means of mul- [ -'5 ] tiple regression equations, however, one is able to determine the opti- mum weight which should be given to each factor, that is, the weight to give it so that the highest correlation or predictive power will be ob- tained. Therefore, the fact that a combination of high-school English mark and the high-school general average resulted in a higher correla- tion with freshman rhetoric than did the former alone, merely means that the weighting of English equally with other subjects in computing the general average is not high enough to yield the best prediction and, therefore if it is only given equal weighting with the other subjects in this average, it should be introduced again with the relative weight in- dicated by the multiple regression coefficient to accomplish this purpose. The direct method of securing the same result would be to use no averages of marks in different high-school subjects, but to consider each as a separate variable or criterion in the multiple correlation work. The reason this was not done was that it would have increased very greatly the amount of calculation necessary without yielding more helpful re- sults than the method used. It would, of course, have shown exactly just which of the subjects entering into the high-school average were useful for making the best prediction in each case and which were not. but there seems little advantage in knowing this, provided one knows how to make as good an estimate without this knowledge and with even less labor. Xot only was much work saved in computation, but also in the use of results, since the multiple coefficients and regression equa- tions secured involve, on the whole, fewer variables or criteria than would be the case if averages of high-school subjects had not been taken and therefore require less computation in employing them for predic- tive purposes. The objection can be raised that there are included In the general high-school average marks made in subjects which show much lower correlations with the freshman subject being considered than do those of certain other high-school subjects and that the inclusion of these marks may have lowered the correlation between the freshman subject mark and the high-school average. This contention Is true, but the writer believes that for all practical purposes any such results have been taken care of by including in the multiple correlations and regres- sions the subjects which appeared at all likely to make any contribution to them. Thus, for example, if freshman French mark was best pre- dicted by a combination of high-school marks In English, French and Latin and point score, rather than by Including the general high-school average, the method of computation used eliminated the latter. In any event, In view of the practical limitations of time and money, It seemed wise. If not absolutely necessary, to follow the method described above. [26] The measures of accuracy of prediction obtained in this study. Finally, as a measure of the accuracy or reliability of predictions based upon coefficients of correlation and regression equations, the coeffi- cients of alienation and the probable errors of estimate corresponding to each of the former expressions were determined. The first of these, ^ the coefficient of alienation, is an expression which shows the relationship between the prediction based upon a given coefficient of correlation and a pure guess. For example, the coefficient of alienation which corre- sponds to a correlation coefficient of .65 is approximately .76. This means that if two variables or series of scores correlate .65 with each other, the estimates of particular scores in one series based upon corre- sponding known scores in the other will on the average be in error by about .76 as much as if the errors resulted from pure guesses, or, sub- tracting .76 from 1.00, that the errors will be .24 smaller than those in pure guesses. The probable error of estimate describes the same situation by stating the limits within which half of the errors will fall. For example, If the probable error of estimate Is found to be 4 points on a percentile scale. It means that half of the estimated scores will not vary from the true scores by more than 4 per cent, and, of course, that the other half will differ by more than this amount. These two indices, the coefficient of alienation and the probable error of measurement, give a more con- crete and meaningful description of the accuracy of prediction than does the coefficient of correlation. °For a more complete discussion of the coefficient of alienation and the probable error of estimate, see Chapter VI. Also: Odell, C. W. "The interpretation of the probable error and the coefficient of cor- relation." University of Illinois Bulletin, Vol. 23, No. 52, Bureau of Educational Re- search Bulletin No. 32. Urbana: University of Illinois, 1926, p. 28-32 and 41-45, and Odell, C. W. Educational Statistics. New York: The Century Company, 1925, p. 173-74, 230-41, or some other text on the same subject. [27] CHAPTER IV THE SIMPLE CORRELATIONS BETWEEN FRESHMAN MARKS AND THE OTHER DATA COLLECTED The simple correlations computed in this study. At the risk of repeating a portion of the outHne of the study given in the last chap- ter, it seems worth while to state again what correlations were and were not found. The simple or zero-order coefficients obtained are shown in Table III, the first column of which gives the correlations of the fresh- man marks with age, the second those with point score, the third with I. Q., and the fourth with the general high-school average. Following this are the coefficients found between the marks in various freshman subjects and those in high-school subjects or groups of subjects selected as being most similar to the freshman ones, or as most likely to ex- hibit significant correlations with them. Thus, for example, the first row of the table shows that freshman accountancy mark had a correlation of — .18 with age, .28 with point score, .29 with I. Q., .47 with high- school average, .38 with high-school commercial average and .47 with high-school mathematics average. As was mentioned in Chapter III. correlation coefficients between certain possible criteria and college marks are not included in this table because, after computing quite a number of them, it appeared that they were of so little value for the purpose of this investigation as not to be worth further consideration. These were the coefficients of the freshman subject marks with the amounts of particular subjects carried in high school and with particular years' marks in high-school subjects, rather than with the average for all of each subject. It will be noted that a number of the coefficients given in Table III are enclosed in parentheses. These are the ones which, because of the joint effect^ of their small size and the few cases concerned, are less than twice their standard errors or three times their probable errors and so can hardly be considered reliable. The chances are greater than twenty-one or twenty-two to one that all of the co- 1— r" ^The formula for the standard error of a coefficient of correlation is — 7^ and VN that for the probable error .6745 ,—., in which r is the coefficient of correlation and N the number of cases. Thus the greater the Coefficient and also the greater the number of cases the smaller is the error and the greater the reliability of the coeffi- cient. [28] b INT SC( 3L SUBJI French A\ Al Al\l Ar Arl Atl Bil .46 ,50 .60 ree times their ■r. LEm coEr P,aE> TSOF COHRK.Ar.0 VOFF RESHJ RKSU T„« h5, 1' )1NT SCORES. NTELLIGEN'CE QUOTIENTS .,ND ■IIGH-SCHOOL 1 ,,J, s I.Q, SCHO )L SUBJECTS AND GROUPS OE SUBJECTS Av„. A,n- AlE- fln.h^ Sr, B,ol- Bo,- Ck.m- Civ- Com- Eco„- En- ri": FrrnrI, Hi>. Homt "It M.ih- Mak- Hu-i. Phy.. Vab. Sci- Sp.n- Eod- ' 17 30 -^ tory omics Train. ks ing '" Sp^t. ence ogy s .4. r ' . . .. ( 171 1 .« .» « 3, .SO (.2?) .. .2.1 I 0.1) .» ( 3S) ,- ».., .„ ,,» ' .52 .60 (14) " ( .11) z .2, (- 24) (.40) :,; .40 (.4-1) "" '" ""' " '" '"'■' "' ""' ' :n."i """' ''" ■''"'■ 1 efficients not in parentheses are significant or reliable and for most of them the chances are very much greater than this. The correlations between freshman marks and age. A glance at the first column of the table shows, that, as one would expect, most of the correlations with age are negative. In fact none of the few small positive ones are reliable and the smallest reliable negative one is — .11 for economics. Others close to this are those for chemistry and history. From this point they range up to — .44 for philosophy, the only others greater than — .30 being for botany and physiology. The correlation of the general freshman average with age is —.23. Although about half the coefficients in this column are reliable and therefore indicate that there is a definite inverse relationship between age and freshman marks, they are so small as to offer practically no assistance In predicting the quality of freshman work when age alone Is known. Later, In Chapter VI, the question of just how much relationship is indicated by coeffi- cients of correlation of given sizes will be discussed and thus the mean- ing of these and the others obtained In this study made more concrete. For the present it Is sufficient to say that the only prediction justified upon the basis of age is that there Is a very slight tendency for fresh- men who are below the average age of their group to do better work than Is done by those above the average age. The correlations of freshman marks with point scores and intelli- gence quotients. An inspection of columns two and three of the table reveals what anyone familiar with the situation would anticipate, that in most cases the entries in the two columns are very nearly the same, the only exceptions being In cases where the coefficients are too small to be reliable. In other words, because of the fact that the point score Is one of the two factors upon which the I. Q. directly depends, the corre- lations of any other variable except age, which Is the second factor, with these two are very likely to be the same or almost the same. The writer seriously considered the advisability of not computing any correlations with the I. Q., but did so to try to determine whether, on the whole, it makes any difference at all which one of the two is used. By looking at the columns it will be seen that sometimes one and sometimes the other Is the larger and that the coefficients of both with the general freshman average are .38. Thus It appfears that from the standpoint of prediction it makes no difference which one is used. From the practical standpoint, however, It seems clear that the point score should be used since the calculation of the I. Q. Involves an additional step. It will be seen that all of the correlations between freshman marks and test results are positive except in the case of athletic coaching and [29] that here they are not reliable. The smallest ones possessing reliability are those for mechanical drawing and physical education, which are about .16. Other rather low ones at or below .25 are those for art. biology, hygiene, and industrial arts. In addition to these a number of those in parentheses, in fact almost all of them, are small. The closest relation- ship between freshman marks and test results appears to be in the case of pharmacy for which the coefficients are slightly above .50. Other subjects with coefficients above .40 are arithmetic, botany, commerce, engineering, music theory, physiology, and psychology. Table III and the discussion just above indicate that there is a closer relationship between score on the test used and freshman marks than between age and such marks. It may be said that the degree of relationship is about twice as close in the sense that it is twice as far away from zero correlation or no relationship at all. A test score, there- fore, offers a better basis of predicting success in freshman college work than does an age, but, as will be shown in more detail in Chapter \T, it cannot be said to be very satisfactory for that purpose. Even in the case of pharmacy, which exhibited the closest relationship, estimates of fresh- man marks based upon test scores would be only 15 per cent better than pure guesses, whereas for the freshman average they would be only about 7^4 per cent better. The corresponding figures for age are about 10 per cent better for philosophy, which has the largest negative coeffi- cient, and less than 3 per cent better for the freshman average. Thus the most favorable statement which can be made about predicting freshman marks from the mental test scores secured in this study is only slight!}' stronger than that made above concerning the use of age for the same purpose. There is a positive, but not very strong, tendency for those who made high scores on the test also to make high freshman marks. The correlations of freshman marks with the high-school average. The entries in the fourth column of the table, which are the coefficients between the freshman marks and the general high-school average, are, on the whole, higher than those in the preceding columns. Two of those in parentheses are negative and the smallest reliable one, .15 for physical education, is just about the same as the smallest for point score or I. Q. Others below .20 are those for mechanical drawing and military. On the other hand, however, there are a number of these coefficients greater than .50 and two above .60 as compared with only one for point score and I. Q. above .50. The two referred to as being above .60 are .69 for dentistry and .62 for horticulture. Others between .50 and .60 were found in the cases of agriculture, botany, French, Latin, philosophy, physiology, Spanish, and zoology. The correlation between the high- [30] ^^ school and the freshman averages is .55, ahnost half again as large as that of the latter with test results. Despite the decided increase in the size of the coefficients, the ac- curacy of prediction based upon high-school averages is still not at all high. For a coefficient of .69, such as is possessed by dentistry, a pre- diction is only about 28 per cent better than a pure guess and for one of .55, such as that for the freshman average, it is about 16 per cent bet- ter. In other words, just as the improvement over no predictive power at all is roughly twice as great for the test scores as it is for ages, so also it Is about twice as great for high-school averages as for test scores. There is no question but that with a few exceptions much better predic- tions of probable freshman marks could be based upon the high-school averages than upon the test scores obtained in this study. The most marked exceptions to this were in the case of art, commerce, and phar- macy, although in arithmetic, engineering, music theory, and physical education the correlations of freshman marks with the test score were also slightly higher than those with the high-school average. The correlations between freshman marks and those in single high-school subjects or groups of subjects. The part of Table III to the right of the column headed "Average" shows the coefficients be- tween the freshman subject marks and those in particular high-school subjects or groups of subjects. The number for the different freshman subjects varies from one to eight, but in no case are there more than five of the coefficients for a single subject which are significant. Glancing over them one sees that with the exception of two or three of those in parentheses none are negative and that the reliable ones range from .23 up to .87. The one of .87, between freshman zoology and high-school botany mark, is based on a small number of cases and although it is much more than three times its probable error, probably should not be considered as having high reliability. The next In size is one of .65 be- tween freshman philosophy and high-school English mark which, al- though not based on a very large number of cases, can still probably be considered fairly reliable. The only others as high as .60 are those of I freshman horticulture mark with that in all high-school science, fresh- man Spanish with high-school French and freshman stenography with high-school commercial work, of which only the second is based on a large number of cases, although the other two are more than three times their probable errors. It will be seen that the central tendency of these coefficients is around .40, about two-thirds of them being between .30 and .50. [31] In view of the irregularity in the size of the coefficients, it is rather difficult to say that many high-school subjects or groups of subjects are on the whole of more value in predicting freshman marks because they correlate more closely with them, than are others. French, however, may be pointed out as probably the most useful in this connection, since it correlates with freshman French almost as highly as does the general foreign language average, with freshman Latin almost as highly as does English, and with freshman Spanish eight points more highly than any other subject or group of subjects. Thus on the whole it furnishes a better prediction of success in Latin and the two languages derived from it than does Latin itself or the whole high-school foreign language average. High-school English, also, on the whole correlates fairly closely with a number of freshman subjects, though in some cases the coeffi- cients fall below .40. Looking through the table carefully one will see that in the case of almost every freshman subject the correlation with some one high- school subject or group of subjects is higher than that with the high- school average and also with age, point score or 1. Q. In most cases, however, it is not much greater than that with the high-school average. This difference, however, warrants the conclusion that if simple correla- tion alone is to be used in predicting success in freshman subjects, it will in most cases be unnecessary to secure age records, scores on the mental test used in this study, or even the general high-school average, but that the high-school mark in the subject or group of subjects most similar to the freshman subject is usually the best criterion. As was sug- gested in Chapter III, however, accuracy of prediction can usually, if not always, be increased by using multiple rather than simple correla- tion, or, in other words, by basing prediction upon two or more of the items of information rather than upon a single one. How much doing so increases accuracy of prediction will be shown in the next chapter. Summary and comparison of the results secured in this study with those in the studies of others. On the whole the degrees of rela- tionship of freshman marks with age, test score and high-school marks found in this study are not very different from those obtained by other investigators. Those with age are so small as to be negligible for pur- poses of prediction. The correlation between freshman average and test score, .38, is slightly, but not a great deal, lower than the central ten- dency of a large number of studies. Undoubtedly this is accounted for by two facts referred to in discussing the reliability of the results in Chapter III. These are that the test used is considerably shorter than [^'-^ most tests employed for the same purpose and therefore does not yield as reliable measures and that marks from a great many institutions were grouped together, thus introducing more variations in standards than would be found in the marks of a single institution. The correla- tions between freshman and high-school subjects found by the writer tend to run about the same as those obtained in the two studies of this sort to which reference is made. Combining the evidence from all the studies along this line with which the writer is familiar the statement seems warranted that in gen- eral a score on any one of the best intelligence tests and the proper high- school mark have about equal value in predicting probable freshman marks. In each case the general expectation concerning the size of the coefficient of correlation is that it will be somewhere between .40 and .50, though under the best conditions one can reasonably expect to se- cure at least some simple correlations of .60 or higher. [33] CHAPTER V THE MULTIPLE CORRELATIONS BETWEEN FRESHMAN MARKS AND THE OTHER DATA COLLECTED The multiple coefficients of correlation computed. Although a brief statement as to what multiple correlations were found was made in Chapter III it seems best to repeat it here in somewhat more com- plete form. Seventeen subjects from among the 49 carried by ten or more freshmen each were selected for this procedure. All but one or two of these were the subjects carried by the largest numbers of fresh- men, these one or two being added because of some especial interest in them. In three of the subjects, chemistry, French, and Latin, two sets of multiple correlations were computed, one for all, or almost all, fresh- men carrying the subject and another only for those who had also car- ried certain high-school subjects. In Spanish, three sets of coefficients were found, two special groupings being made according to the high- school subjects carried. Thus, including the general freshman average. 23 sets of multiple coefficients were computed. The procedure in computing the multiple coefficients was first to select all of the simple coefficients of correlation which seemed worth using, the number so selected varying from two to six, and then to combine these to secure the multiple ones. The two always used were the general high-school average and the point score. In addition to these the high-school average in the group of subjects and the mark in the one or more single subjects most similar to the college freshman sub- ject were also included, except in two or three cases in which no high- school subjects that could be said to be similar had been carried by enough pupils to be worth including. The one marked example of this was ph}-sical education, almost none of the freshmen who carried this subject having marks recorded for any similar work in high school. In a few cases age was used. In a number of instances the computations were begun with more criteria than were carried through to the finish, since, as the work progressed it could be seen that some of those used made no contributions. In each case the work with those included was carried to the point that no further increase as great as .01 was ob- tained by computing multiple coefficients of higher orders. In the ma- jority of cases the number of criteria required to accomplish this end was three, in three cases four were required and in none more; in one case the highest zero order coefficient could not be increased and in the [34] i remaining ones two were all that was necessary. In this connection the reader should again be reminded that many of the criteria used, such as the general high-school average, the high-school science average, the high-school mathematics average, and the high-school foreign language average, were themselves combinations of marks in several different subjects and therefore the simple correlations with them were in a sense multiple, although not computed by multiple methods. Because of this fact the increases above the simple coefficienets were not nearly as great as if no such averages had been used. The multiple coefRcients of correlation obtained in this study. The highest multiple coefficients obtained in this study along with related data are presented in Table IV. The first set of four columns therein is for the highest simple coefficient of correlation obtained for each of the subjects mentioned, the second group of four for the highest multiple coefficient and the last group of three for the increase of the multiple over the simple coefficient. Within each of the two groups of four the first column, headed "r" and "R," contains the actual coeffi- cients of correlation, the second, headed "k," the corresponding coeffi- cients of alienation, the third, headed "P.E. ," the corresponding prob- able errors of estimate and the fourth the one or more criteria^ used in the correlations. The last three columns contain, in order, the in- creases in the highest multiple over the highest simple coefficients of cor- relation and the accompanying decreases in the coefficients of aliena- tion and the probable errors of estimate.- For example, taking the first line of the table, the highest simple coefficient of correlation of freshman algebra mark with any single criterion was .52, the corresponding co- efficient of alienation was .85, the probable error of estimate 6.4 and the criterion, high-school mathematics average. The highest multiple corre- lation obtained for algebra was .53, with a coefficient of alienation of .85 and a probable error of estimate of 6.4. It was based on two criteria, high-school mathematics average and high-school general average. The increase in the coefficient of correlation was .01, whereas there was no change in the coefficient of alienation or the probable error of estimate. ^It will be noted that in a few cases in the table the abbreviations for two of the criteria are connected by the word '"or." This means that in such cases the correlation based on the two was the same or so nearly the same that it makes no appreciable dif- ference which one is used. For example, in the case of geometry, the simple correla- tions with high-school geometry mark and high-school mathematics mark differed by only .0004, so that it makes no material difference which one is used. ^It should be remembered that an increase in the coefficient of correlation and de- creases in the coefficient of alienation and the probable error of estimate indicate closer relationship or greater accuracy of prediction. [ ro ] z w w ^ X ZH o o •— 1 z 'f 1— t H-1 H w u a: o u Q a; O z Z Q UJ D u < b; Ul w O H u DS w u J r/^ CL, D O D < > Q Q z z < < W en t^ a, cc S < c/2 s H Z O) 1— ' w hJ n < H ^ £ 1 « (L) c U c "" ^~ r — O Q^ o r-^ ~ ri •— ^ c c c o c c O B3 t) OJ S cx-a t:5 E Lh _ t 't <: i>n fN fn — Tf r- «N C 3-^ c ^ E cii " "rt "5 or u cr ■y: 6 ,'u: or. . re . , c/: 'C C < ^a. Cl. < -■ CCL U J= ^ E~ ^ " uT - J" " J r E :: r- J ^ • -S-^ rt 3> u - rcA — > > c > c > . u > c o > ■- > bJC X ^ S3 U-y: -n r^ ^H c (N iJ-. -1- r- O ^ ^ 00 oc oc oc OC oc O. OC OC o f^ r- _^ r r-- ro "^ O ^ ■* Pi •^ u-i -*■ -* -i- vC 1^1 U-. r-2 lo ■* CO c o L- ij c ■w mple with terion £ 0. 8. > 1. > > 0. > 51 > E-^ c - ^ t a. > > u J- < < < < <;j^ UJ < < ■".25 1 't vC r- C o w- 't C r-i o- -t Ov ghes relat one a-* ^ ■vC ^C t "t ^ SC I-~ "-. LT ^ K 8 « '-' rt u- r~ C^ f-N r- Tl •^ r~ Tf c> C w-1 ^ oc oc oc o- O oc oc oc o. oc o^ C> rJ a- m: o- m: u- yj- C Tf ■VO f fN tH t Tf f c ly- w •y^ to '*' "t CO C *-• * 1) 5 ^ _c X 0. t J c : c ' c t. t. c 0. 2 ^ « ^ = 11^ a; oj 4, .- "• C .a to < : L. ) L ) u fcL i tt U C C C I El 136} trt (U c i;j= o en l> oj &■£ S o o _. OJ 1) _C aj o U -H ro O r-4 — . .> .>>cc>co>oo^>> oi (1) cfl 4-* tuO s (U ^ JO 3 O CO w > c > to > o ^ CO LO to ^ CO « r: w •n •;: x c c II a- *"■" -a 3 M j=.t: -IS o 2 ^ S = ^ - . — „ • - c -T3 u•- = -^ 3 C o'c 3 .- uHHjsj; — ° " c^ S « % « i t; _ 3 i. - 1- u rt ;, 3 c >< " — x: ^^ ° 11 u) S Mi-< 2 j=-5 "_3 lij S-c ,3 S ■-J= U-- ^__c j^ « "^ „ o-fi I- S " = S 3 .j'^^-s^.s. « 3=i£'ti'o « «"«.-. ^^-a *-- ^ o -a = n^ °l^-= f M ^. t; 2 ^J=.li-C_^ .:; 6 =1 ^ g 3-c^ 1 J= 'i~ "s-'o P, c = £ - - c b: r' n ' ^ e- *^ n »* _ U O- u CO c 3 S _»; C M Q J* «•-— 3 U _ii o u u JJ >.•■" d_ [37] The increases of multiple over simple indices of relationship. A comparison of the multiple with the simple coefficients shows that on the whole there was little increase in the latter. In one case out of the 23 the highest simple coefficient could not be raised by including other cri- teria. The median increase produced was only .03 and the greatest .07, with corresponding decreases in the coefficient of alienation ranging from zero to .05, over three-fourths of them being .02 or smaller. As re- gards the probable error of estimate over half of the cases were de- creased by .1 and in only one case was the difference more than .2. In other words, the increased reliability of prediction obtained in this study by using the best multiple correlations and regressions is so small that it is very doubtful if it can be said to be worth the additional labor and expense required. The criteria of highest predictive value. In addition to the fact just mentioned probably the one of chief interest in the table is the question of what criteria are most valuable as bases for predicting fresh- man marks. It was shown in the preceding chapter that in many cases the high-school average yielded the highest correlation with the fresh- man mark, whereas in many others some similar subject or group of subjects did so. Of the 25^ criteria employed in the simple correlations given in this table, the high-school average appears in 13 cases, the aver- age mark in a similar group of high-school subjects in six, that in a sim- ilar high-school subject in five, and the point score only a single time, for physical education. The criteria used in obtaining the multiple co- efficients run very similarly except that the point score appears a much larger number of times. In approximately three-fourths of the cases the high-school average Is one of the criteria and the same is true of the point score. Single subject marks occur somewhat less frequently and those in groups of subjects in less than half of the cases. It appears that although the simple correlations between the point score and freshman marks are in general decidedly lower than those of the latter with the high-school average and with marks in various subjects and groups of subjects, yet the point score makes a contribution in prediction some- what distinct from that made by the other criteria mentioned. Summary of this chapter. Multiple coefficients of correlation were computed for about one-third of the freshman subjects, in a few cases more than one set being computed for each subject. These coefficients run from .20 up to .63, most of them being between .40 and .60, al- ibis number includes the two criteria giving practically equal coefficients in ge- ometry and rhetoric. [38] though five are at or above the latter figure. The accompanying co- efficients of alienation range from .98 down to .78, most of them being In the eighties and the probable errors of estimate from 7.3 down to about half that amount. For the general freshman average the multiple coefficient is .58, the coefficient of alienation .81 and the probable error of estimate 3.7. In no case were more than four criteria needed to se- cure the highest coefficient and In most cases only two or three. The increases In the multiple above the simple coefficients and the decreases In the corresponding coefficients of alienation and probable errors of estimate are so small that the very slightly Increased reliability of pre- diction appears not to be worth the additional labor of computing. With two or three bare exceptions the estimates possible are still four-fifths or more pure guesses and In the case of about one-fourth of the subjects they are nine-tenths or more pure guesses. [39] CHAPTER VI THE ACCURACY OF PREDICTIONS BASED UPON THE OBTAINED COEFFICIENTS OF CORRELATIONS Purpose of this chapter. To anyone who has not had considerable experience in deaHng with predictions when the degree of relationship is expressed by coefficients of correlation the mere statement that a co- efficient of correlation is so much, .35 or .60 for example, generally has little definite meaning, especially as indicating how accurate the predic- tions are. In the two chapters dealing with the correlations found in this study a few brief references have been made to their interpretation in other terms, but it seemed best to devote a chapter to a more elabor- ate treatment of the matter. The attempt will be made to show in two or three ways just how many and how great are the errors present in predictions associated with coefficients of correlation of sizes typical of those obtained in this and other similar investigations. The writer has discussed the matter at somewhat greater length elsewhere^ and will not attempt to reproduce in full what has been said there, but will repeat a portion of it with some additional suggestions concerning Interpretation. Interpretation of the coefficient of correlation, in terms of the co- efficient of alienation or of a "pure guess." In connection with the coefficients of correlation presented in Chapters IV and V some figures were given and statements made as to what they meant in terms of pure guesses. For example, in one place it was stated that a coefficient of .52 indicated that predictions based thereon were only 15 per cent better than pure guesses. To understand this and similar statements one needs to know what is meant by a pure guess. It assumes that the person making the prediction or guess knows what the distribution of the meas- ures which are being predicted is, but does not have any information at all which helps him to determine which measure belongs to any partic- ular case. For example, if one were making a pure guess concerning the marks to be assigned the members of a high-school freshman alge- bra class from their previous records it would be assumed that he knew the distribution of marks which would be given, but that he had no in- formation at all as to which mark would be given to each individual pupil. 'Odell, C. W. '"The Interpretation of the probable error and the coefficient of correlation." University- of Illinois Bulletin, Vol. 23, No. 52, Bureau of Educational Re- search Bulletin No. 32. Urbana: University of Illinois, 1926, p. 28-32 and 39-45. [40] The conception of a pure guess can probably be made more mean- ingful by employing a concrete example. For this purpose, let us sup- pose that in a high-school freshman algebra class the teacher has decided that the marks she will issue will consist of two A's, seven B's, twelve C's, five D's and four E's. Let us suppose further that someone is told that this distribution of marks is to be given and that, without knowing anything about the Individual pupils which in any way concerns their scholastic ability or achievement, he attempts to predict the marks each one will receive. He will, of course, select two individuals as those who will receive A's, seven as those who will receive B's, twelve C's and so on. Since his selections or predictions are not based upon any knowledge whatsoever which helps him in determining the marks assigned individ- ual students, they will be subject to the same errors as if any purely chance method was used, such as placing the names of the pupils in a hat and predicting that the first two drawn would receive A's, the next seven B's, and so on. In either case the predictions made are pure guesses and the relationship between them and the actual marks is represented by a coefiicient of correlation of zero.- For predictions to be a certain per cent better than pure guesses means that on the average the errors involved in such predictions are smaller than those involved in pure guesses by the given per cent. To illustrate this, let us suppose that if pure guesses are made in a certain situation they involve one error of 15 points, one of 14, two of 13, two of 12, and so on. With predictions 40 per cent better than pure guesses the errors would be 40 per cent smaller, so that it might be expected that instead of the error of 15 points there would be one of 9 points, in- stead of the one of 14 points there would be one of 8.4 points, likewise two of 7.8 points, two of 7.2 points, and so on. Sometimes the size of the errors is expressed in just the opposite way to that so far used in this paragraph, that is to say, instead of saying that predictions are 40 per cent better than pure guesses one may say that they are 60 per cent pure guesses or that the errors involved are 60 per cent as large as those in pure guesses. Table V shows how the predictions based on coefficients of correla- tion of given sizes compare with pure guesses. In this table, the first column contains values of the coefficient of correlation at intervals of .01 from 1.00 down to .95 and at intervals of .10 from .90 down to .00, with the coefiicient of alienation or fraction of a pure guess correspond- ^In actual practice a coefficient of exactly zero will rarely be obtained because with a small number of cases the element of chance agreement or disagreement between prediction and actual facts is fairly large. [41] TABLE V. COEFFICIENTS OF ALIENATION* CORRESPONDING TO CERTAIN VALUES OF THE COEFFICIENT OF CORRELATION I Coefficient Coefficient 1 Coefficient 1 Coefficient of of of of Correlation Alienation Correlation Alienation 1.00 .0000 .70 .7141 0.99 .1411 .60 .8000 0.98 .1990 .50 .8660 0.97 .2431 .40 .9165 0.96 .2800 .30 .9539 0.95 - .3122 .20 .9798 0.90 .4359 .10 .9950 0.80 .6000 1 .00 1.0000 *The coefficient of alienation is obtained by solvingv 1 — r^, in which r is the symbol for the co- efficient of correlation. ing to each. Beginning at the top of the table It will be seen that when the correlation is perfect and the coefficient 1.00, the coefficient of aliena- tion is zero, or, in other words, prediction can be made with absolute accuracy. When the correlation is .99, the prediction is about .14 of a pure guess, when it is .98 the prediction is .20 of a pure guess, and so on. One can see that the inaccuracy of prediction or the fraction of a pure guess involved therein incfeases very rapidly at first for compara- tively small decreases in the coefficient of correlation. By the time the latter reaches .80, the errors in predictions are .'60 as large as those in pure guesses, and when the correlation is .50 the errors are almost .87 as large as those in pure guesses. If one now recalls the sizes of the coefficients of correlation between freshman marks and other criteria, and then the corresponding coeffi- cients of alienation, he will see at once how unreliable are the best pre- dictions of college marks which can be made upon the basis of these criteria. IMost of the obtained simple coefficients of correlation were be- low .50, very few rising above this. Thus for most of them the best predictions possible are 87 per cent or more pure guesses. A very few of the simple coefficients, and not a great many of the multiple ones, rose above .60. For one of .60 the errors in prediction are .80 as large as those in pure guesses and for one of .65, not given In the table, they are .76 as large. The general statement may therefore be made that, with one or two possible exceptions, the best predictions possible from the criteria used In this study are still subject to errors which are on the whole three-fourths as large as those in pure guesses and that in most cases they are at least four-fifths or five-sixths as large. In other words, they are so large that probably the most that can be said for predictions based upon these or similar criteria Is that if it Is necessary [42] or highly advisable to make selection or classification of some sort, these criteria furnish a somewhat better basis for doing so than would mere guesses. Interpretation of the coefficient of correlation in terms of the probable error of estimate. Another means of describing the accuracy of prediction based on a coefficient of correlation of a given size is to state the probable error of estimate.^ The probable error of estimate is easily obtained after the coefficient of alienation has been found as all that is necessary is to multiply the latter by the median deviation* of the distribution in question. The meaning of the probable error of esti- mate is, in a general way, the same as that of any other probable error or median deviation, that is, half of the errors involved in making esti- mates or predictions are less than the probable error and half are greater, about 82 per cent are less than twice the probable error and 18 per cent greater, almost 96 per cent less than three times the prob- able error and slightly over 4 per cent greater, and so on. For example, a probable error of four points in connection with estimates of fresh- man marks in algebra would mean that half of the estimates or predic- tions of the marks made by individual students would be in error by less than four points and half by more, that about 82 per cent of them would be an error by less than eight points and about 18 per cent by more, and so on. The form of statement may be changed to read that the chances are even that the error in the case of any particular indi- vidual is not greater than four points, that they are about 4.6 to 1 that it is not greater than twice this amount or eight points, 22 to 1 that it is not greater than three times the probable error or twelve points, and so on.^ It will be recalled that the probable errors of estimate given in Chapter V as corresponding to the highest multiple coefficients of corre- lation obtained were mostly between four and six points, though one or two were slightly smaller than four points and several larger than six, one even being above seven. The central tendency was somewhat above five. In other words, on the average, predictions of college freshman 'The standard error of estimate might also be used, but the discussion will be confined to the probable error because it is probably more generally understood and used. The standard error is 1.4826 times the probable error. "Since the median deviation equals .6745 times the standard de\iation, the usual formula for the probable error of estimate is .6745