Jf . / Whole No. 138 1921 Vol. XXX No. 5 PSYCHOLOGICAL REVIEW PUBLICATIONS Psychological Monographs EDITED BY JAMES ROWLAND ANGELL, New Haven, Conn. HOWARD C. WARREN, Princeton University ( Review) JOHN B. WATSON, New York (/. of Exp. Psych.) SHEPHERD I. FRANZ, Govt. Hosp. for Insane ( Bulletin ) and MADISON BENTLEY, University of Illinois {Index) Personnel Selection of Graduate Engineers The Differentiation of Apprentice Engineers for Training as Salesmen, Designers, and Executives of Production g. BRUCE V. MOORE, Ph.D. Bureau of Personnel Research, Division of Applied Psychology, Carnegie Institute of Technology, Pittsburgh, Pennsylvania PSYCHOLOGICAL REVIEW COMPANY PRINCETON, N. J. and LANCASTER, PA. Agents: G. E. STECHERT & CO., London (2 Star Yard, Carey St., W. C.) Paris (16 rue de Cond6) Digitized by the Internet Archive in 2019 with funding from Princeton Theological Seminary Library https://archive.org/details/personnelselectiOOmoor FOREWORD This volume is a contribution to psycho-technology. Mr. Moore has made, in the following experimental and statistical study, the first approach toward a scientific solution of the problem of placement of engineering graduates in a great American indus¬ trial organization. It is a bold and novel undertaking, characteristic of a young century, to essay the application of the principles and technique of scientific method to the solution of human problems in industry. Yet it needs no rare wit to see that these very problems of human adjustment must hold during the twentieth century the focus of study and invention which during the nineteenth century were chiefly concentrated on the problem of perfecting the mechanics of manufacture. Human engineering is destined to a development comparable to that experienced by mechanical, chemical, and elec¬ trical engineering, when they first began to draw largely on the lustily growing sciences of chemistry and physics. Thanks to the foundations laid by these sciences, modern industry has a highly developed technology of materials and processes. It asks now for a technology of human nature. This study in principles and practices of personnel selection is not a popular treatise. But its appeal will be felt by at least three types of readers: the thoughtful leader of industry who ponders the trend of scientific experimentation on human problems; the young engineering graduate who is debating in his own mind whether to become a designer, a manager, or a salesman; and the psychologist who is watching, with some apprehension, the move¬ ment to put into practical use the tools he has forged. W. V. Bingham, Division of Applied Psychology, Carnegie Institute of Technology. m PREFACE This report is a description of a specific study on a practical problem. Certain principles and practices are proposed as conclu¬ sions from this study. The practical problem was to determine methods and means for selecting young engineers just being graduated from college and placing them in the type of work which they could do best in a large electric manufacturing com¬ pany. The outstanding feature of the problem was to select the men best qualified to develop into sales engineers; that is, en¬ gineers who meet the public and sell electrical machinery. The study was made primarily to solve the practical problem; and then the principles underlying the solution were formulated in order that they might be applied to further problems. In other words, the research was made for service, in the belief that it was possible to give service and at the same time make a contribution to science. As a practical investigation the work is already giving service. The results already obtained on one hundred seven engineers employed by the firm during the first year that the new methods recommended here have been used, are very similar to the results obtained on the group studied in this research. Assuming that the executives classified the men correctly at the end of a year, we can say that the results of the test alone would have classified seventy per cent of the men correctly on the day that they were employed. Beginning with June, 1921, the company will use the test and other methods recommended here as definite aids in clas¬ sifying and placing graduate engineers whom they employ. Progress in psychology, as in any other science, has been made by specialization and intensive study on a specific problem. Quite often in science both the motive and the opportunity has been a practical problem in industry. The growing realization that the human factor is a most important one in industry has given both motive and opportunities for research on specific problems in psy- VI PREFACE chology, the science of human behavior. The business world is ap¬ pealing to psychology as a science to solve specific problems. Some psychologists have already brought forth and exploited the facts, the technique, and the point of view of their science, in the hope that the science might be advanced as a science as well as be made to serve man in utilitarian ways. For accomplishing both pur¬ poses, the first and most important principle is that one definitely narrow and specific problem after another must be attacked and solved to contribute its portion to the cumulative content of the science. This report of a study on such a specific problem has been written in a comparatively brief and concise form. It is intended for practical use; but also it is intended to be read in its entirety only by those who understand the language of psychology and statistical methods. The original tables of data are not presented; and much of the explanation is omitted which might be included in an exhaustive treatise. The tables and all other details of the study are on file with the Division of Applied Psychology, Car¬ negie Institute of Technology, and are available for all who may wish to examine them. However, it is believed that the report is sufficiently complete to enable those who have an in¬ telligent interest in such a problem to grasp what the study has to give. ACKNOWLEDGMENTS Many people have helped make this research possible. First, it was the cooperation of the Westinghouse Electric and Manu¬ facturing Company that made it possible to carry on the research in a large industry with a definite practical problem before me that would yield fundamental principles. This large industrial firm made it possible to have the study include a sufficiently large number of selected subjects to make the results more reliable than they would have been otherwise. In this report I wish to make special acknowledgment of the help given me by Mr. C. S. Coler, Manager of the Educational Department, and Mr. E. B. Roberts, Mr. W. E. Freeman, Mr. F. G. Kottman, and others of the Educational Department; also Mr. B. G. Lamme, Chief En¬ gineer, and Mr. H. E. Jordan and Mr. H. B. Bassett of the En¬ gineering Department; and Mr. S. L. Nicholson and others of the Sales Department. I am indebted to Professor E. L. Thorndike of Columbia University for the “True-False” statements which I took from his test and incorporated in Part II of the test which I devised. Also I received many suggestions from Professor L. L. Thurs- tone’s tests which he prepared for the Society for the Promotion of Engineering Education. I wish to acknowledge the valuable technical aid given to me by Professor W. R. Work and Mr. G. M. Porter of the Department of Electrical Engineering, Car¬ negie Institute of Technology. Finally, but not least, I wish to thank Dean W. V. Bingham and Professor L. L. Thurstone of the Division of Applied Psy¬ chology, and Director C. S. Yoakum, Associate Director W. D. Scott, and the Research Fellows of the Bureau of Personnel Research, Carnegie Institute of Technology, for their sugges¬ tions and the many ways in which they have helped make this work have whatever value there is in it. vii TABLE OF CONTENTS Part I. The Problem PAGE I. The Practical Problem. i II. The Psychological Problem Involved. 2 Part II. The Lines of Investigation and the Results III. Occupational Descriptions and Analyses. 1. Functional Classification of Engineers. IV. Criteria of Success of Engineers. . .. V. Evaluation of Technical School Grades. 1. Transmutation and Tabulation of Technical School Grades. 2. Comparison of Technical School Grades with Other Criteria. VI. Evaluation of Grades and Ratings Given During Training in the Educational Department of the Industrial Firm. VII. A Study of Rating Scales. 1. Principles of a Rating Scale. 2. Evaluation of the Rating Scale Used, Entitled, Rating by Interviewer . 3. The Revised Form of Interviewer’s Rating- Scale: The Interviewer's Master Scale . 4. Instructions to Interviewers for Rating Senior Engineers . 5. College Instructor’s Rating Scale. 6. Shop and Class Rating Scale. VIII. Survey and Evaluation of Interests as Criteria for Vocational Placement. 1. The Significance of Interests. 2. Method of Using the Record of Interests . 4 6 7 8 8 10 13 21 21 23 26 2 7 30 34 34 35 IX X CONTENTS 3. Evaluation of Specific Questions in the Record of Interests . 38 IX. Application of Psychological Tests to the Problem 49 1. The Nature of Mental Ability and Its Problems for Tests . 49 2. The Results of a General Intelligence Test. 51 3. Construction and Evaluation of a Special Test for Dififerentiating Graduate Student Engineers 54 (a) Hypotheses of the Test. 54 (b) Directions for the Test.. 55 (c) Construction of the Test. 59 (d) Standardization of the Test. 63 (e) Results of the Test for Graduate Student Engineers . 65 (f) Statistical Interpretation of the Results for Occupational Placement. 70 Part III. Summary of Results and Conclusions, and Recommendations. X. Summary of Results. 76 XI. Conclusions and General Principles Derived from the Study . 80 XII. Specific Methods and Practices Recommended.... 82 Bibliography 85 PERSONNEL SELECTION OF GRADUATE ENGINEERS The Differentiation of Apprentice Engineers for Training as Salesmen, Designers and Executives of Production PART I. THE PROBLEM I. The Practical Problem The Westinghouse Electric and Manufacturing Company each year employs about three hundred engineers who have just been graduated from technical schools and colleges. All these men have had practically the same training; but they must be differentiated and placed in different lines of engineering accord¬ ing to their special interests and aptitudes. The aim is not to discover which engineers are best for any particular kind of work; for a few very capable men might be best in all lines; but the aim is to discover in what line of work each engineer will function most efficiently and satisfactorily. That is, the desired end is to have each member of the organization so placed that he is able to render his greatest service to the Company by having the greatest opportunity to develop his capacities and promote himself in the line of work which he most enjoys. The practical problem is to determine methods and means for select¬ ing and placing young engineers in the type of work which they can do best. For selecting, differentiating, and properly assigning grad¬ uate engineers to the various phases of engineering and sales¬ manship, five methods or means and combinations of them are possible, namely: 1. Grades made in the technical schools; 2. The grades, try-outs, and observations secured during the year of preliminary training in the firm’s educational de¬ partment ; 2 BRUCE V. MOORE 3. Rating scales; 4. Interests and desires of graduate engineers; 5. Psychological tests. The second and fourth, interests, and observation during train¬ ing are the ones emphasized at present; and in fact they are the best methods, at least until new methods are evaluated. How¬ ever, these methods are not entirely satisfactory nor ideal; for the interests or choices of the men are uncertain guides and are not always identical with the needs of the company; and often the graduate engineers cannot decide themselves what work they would prefer. The process of trying out under observation dur¬ ing training can be done only after a certain amount of selec¬ tion and differentiation has been made, and the Company has more or less committed itself in regard to its future engineers and salesmen. The training could be made more definite and intensive if it were known from the beginning, to just what line of work the graduate engineers should be assigned. Moreover, a more or less definite quota is needed for each line of work, and those men should be selected from the technical schools who would most exactly be the number required for each line of work. Finally, but not least important, the method of trying out under observation fails to reveal for what the graduate engineer is best fitted; and at best, there is often an uncertainty or question as to what work should be assigned. To meet the conditions just described, the first and second methods are analyzed, but atten¬ tion is concentrated upon the development and application of the third, fourth, and fifth methods named above, that is, rating scales, survey of actual interests, and psychological tests. II. The Psychological Problem Involved Broadly speaking the psychological problem is one of indi¬ vidual differences relative to capacity for doing specific kinds of work. The previous policy was to classify and assign or place the engineers according to their interests, the practice being limited somewhat by the openings or needs in the Company and by the general impressions which the executives had of the young engineers. The big question raised was whether the men were PERSONNEL SELECTION OF GRADUATE ENGINEERS 3 being placed where they could function most efficiently. Were their interests a reliable criterion of their ability? This leads to the fundamental psychological questions: (i) Is marked ability in a particular line of work only general ability or intelligence directed by interest? (2) Are there different kinds of intelli¬ gence? (3) Is there one thing called general intelligence, and additional abilities which are special and which vary independ¬ ently? (4) Is interest an indication of a special ability in a particular line or work? (5) Are there only special abilities, not necessarily correlating with a general average of these, which is called general intelligence? (6) Does success in a particular occupation depend upon purely intellectual ability, or are the more or less vaguely defined personal traits the more im¬ portant for success? (7) When it is learned which of these factors are important, how can they be measured so as to make a prediction or a statement of the probability of success in a particular job? Summarizing these questions, we state the psy¬ chological problem as follows: To determine what mental abili¬ ties and traits are of practicable use as criteria for differentiating graduate engineers and predicting their success in six different but closely related kinds of work. This problem includes the evaluation of old methods and means and the devising of new methods and means for determining and measuring these mental abilities and traits. It is not to be expected that we should answer all the questions stated above, which have already taken years of work by the best psychologists, and will require much more work; but these questions should be recognized as pertinent in the assumptions and conclusions of this study. This being a specific study with a definite problem in applied psychology, it does not include in its purpose and scope any his¬ torical or theoretical treatment of many general psychological problems which may be touched upon. Very few studies that have been published deal directly with this kind of problem. However, at the end of this report is appended a bibliography or list of studies which bear more or less specifically upon a prob¬ lem of this nature. 4 BRUCE V. MOORE PART II. THE LINES OF INVESTIGATION AND THE RESULTS III. Occupational Descriptions and Analyses The first important step in all personnel selection and place¬ ment is to secure adequate descriptions and analyses of the jobs to be filled. Then from these there must be prepared specifica¬ tions for the persons who will most nearly meet these require¬ ments. The specification is an interpretation of the work done on the job, in terms of capacities, skill, knowledge, and traits re¬ quired to do the work. In practice the degree of specificity to which these descriptions are carried varies considerably from firm to firm and with different lines of work. In general dis¬ cussions it is quite common to refer to the various vocations , which differentiate only the general lines of work typified by the various professions and trades. Often in employment and place¬ ment work, differentiations are made only into occupations, which are understood to be the specialties in the trades, such as punch press hand, lathe hand, die sinker, armature winder, etc. Strictly speaking, the job description is reserved for a description of a particular operation on a particular piece of material or appara¬ tus by means of a particular tool or machine. In the Westing- house Electric and Manufacturing Company these distinctions are made in the use of these terms by the Occupations and Rate Committee and by those in the Employment Department. In the Westinghouse Electric and Manufacturing Company the employment and placement of the engineers is done by the Educational Department. Each spring this department selects between two and three hundred engineers from the senior classes of the technical colleges; and after giving them a year of grad¬ uate training in the Educational Department at the central plant in East Pittsburgh, it places these men in the various lines of work in the organization. Before it was possible to choose any certain engineers as better fitted for certain lines of work, it was necessary to know what was expected of these engineers in each line of work. As there were not on record any descriptions or specifications of the work done by the engineers in the various PERSONNEL SELECTION OF GRADUATE ENGINEERS 5 departments, this information had to be gotten first. The descrip¬ tions of the lines of work were obtained through the Educational Department. After conferences with some of the executives who could tell most about the various lines of work, a description and outline of the nature of the occupational description desired was given to the Educational Department. Under the supervision of the Educa¬ tional Department the occupational descriptions were prepared by the heads of the various sections in each of the departments. These were elaborated further by means of conferences with various executives in which questions were asked to clear up obscure details. Finally, in this manner, a conception of each line of work was obtained, which was detailed enough for work¬ ing intelligently on the problem of selection and placement. It would be out of place here to give an exhaustive description of each job or line of work in each section of each department. Moreover, such detailed descriptions are not necessary for un¬ derstanding the problem that is before us. The complete occu¬ pational descriptions are on file with the Division of Applied Psy¬ chology, Carnegie Institute of Technology, and are available for those who wish to examine them. It is sufficient here to have a description of the main classes of engineering work. In actual work upon the problem, only four classes of engineers are con¬ sidered, namely : (i) research workers and design engineers; (2) general or all other engineers in the engineering department; (3) operating, service, and works management engineers; (4) sales engineers. This was found practicable because nearly all the engineers were graduates of technical colleges and had re¬ ceived essentially the same technical training. Moreover, this process of being admitted to the technical colleges and of being graduated from them, had been highly selective so that any dif¬ ferentiation among the men was possible only by comparatively fine discrimination. In fact, the one real differentiation that was consistently kept in mind in all this study was that between the engineers going into the sales department and all other engineers. This might be qualified or rather more definitely stated by ex¬ plaining that the design engineers were also differentiated from 6 BRUCE V. MOORE the other engineers, and being considered as representing the purest type of engineer, they were also considered as at the opposite extreme from the sales engineer when the differences were looked at as quantitative or linear in nature. The descrip¬ tions of these lines of work are given with particular attention to those lines considered to be most clearly differentiated. Com¬ bined with these descriptions are given the specifications for the kind of men considered best fitted for each particular line of work. i. Functional Classification of Engineers Within the organization, the engineers are classified according to the department and according to the class of product worked upon. Since the specifications for the kind of engineer needed depends upon the kind of work done, we must classify the en- gineeers according to the qualifications required to do the par¬ ticular kinds of work: (a) In all the specialized engineering departments which are responsible for the engineering work on different particular lines of electrical apparatus and machinery, thus excepting the General Engineering Department and the Testing Department, there must be design engineering. This calls for engineering of the purest and highest type. With this type of engineering is also classed the research worker in the Research Department, (b) All the other engineers of the en¬ gineering departments are to be considered as doing a second type of work. This work is engineering; but it is not to be con¬ sidered as calling for as great mathematical and technical ability and as much originality in that line as is needed by the design engineer, (c) The third class includes the engineers in Factory Management, the Service Department, and Operating. The work of these engineers brings them, to a larger extent, in con¬ tact with other people outside the organization and also with those inside the organization. Their work requires more exec¬ utive ability than the work done by the other groups of engin¬ eers. (d) The fourth class is that composed of Sales Engineers. These men are primarily engineers as well as salesmen; for to get and keep the good-will and confidence of the customers, they PERSONNEL SELECTION OF GRADUATE ENGINEERS 7 must act as consulting engineers and give reliable technical advice to the customer or prospective customer. However, the success¬ ful engineer doing the work in the sales department must have a commercial sense and an ability to meet and successfully nego¬ tiate with other people, that differentiates him from the other engineers. IV. Criteria of Success of Engineers Closely allied and of equal importance with a definite know¬ ledge of the jobs to be filled, are criteria of success of men in those jobs. Methods of selection and placement cannot be de¬ veloped or improved upon unless it is possible to know when bet¬ ter results are obtained. The best criteria are objective measures, such as records of production; but it is seldom possible to obtain facts on such objective measures that are comparable. The work of sales engineers and of design engineers is such that it cannot be measured by the product finished. Although records of sales may be kept, many other factors enter into the value of a sales engineer to the company. The professional consulting service which the engineer gives to the customer, and the good-will secured in return, cannot be measured in that way. Also, a design engineer may spend all his time for a year or more on one machine or piece of apparatus; but if he gives the piece of apparatus an original design or makes it in the nature of an invention, his work is scarcely measurable. However, it is necessary to make use of the best criteria available. The year spent in the Educational Department can be consid¬ ered as one phase of the engineer's work. Therefore, grades and ratings on their success there are some criteria. Furthermore, the executives and instructors of that department come to know intimately all the student engineers during the year of training. To give some definite data against which tests and other ratings might be checked, the Educational Department was asked to rank the student engineers in the order of their general intel¬ ligence. General intelligence was defined for them as “the ability to analyze a problem, grasp the point, and deal with a new situa¬ tion.” Two men who both supervise the work of the engineers 8 BRUCE V. MOORE and instruct them, worked together as a committee to rate the student engineers. They first classified the men into five groups according to their general intelligence, as defined. Then they ranked the men in each group in the order of their estimated intelligence. By combining the groups, with the highest rated group first, a ranking of all the engineers was secured. Further checks on the reliability of ratings, test scores, and these grades were possible by intercorrelations. Although none of these were absolute measures of success, it was reasonable to assume that high intercorrelations between different kinds of measures, such as between grades and tests, or between estimates and grades would show that there was a certain consistency in the measures that correlated highly with other measures, and that there was no cause to believe that they were unreliable. V. Evaluation of Technical School Grades The representatives of the Company, in selecting engineers graduating from the technical schools, did not consider primarily the grades received by the engineers in their college courses. They interviewed the seniors for twenty or thirty minutes, and then later inquired about the man’s grades in general. However, the grades were not considered as an absolute index of a man. Lead¬ ership and participation in extra-academic activities were con¬ sidered as important. It is not always easy to obtain the grades in such a way that extensive and consistent use can be made of them. I thought that probably they would show something of the technical training and ability of the men, but that many qualities making for success in engineering and salesmanship were prob¬ ably not to be inferred from such grades. However, a scientific investigation of this sort would not ignore them. Moreover, a study of these grades might give some light on the significance of grades given in the Company’s training courses. Accordingly, I made a study of the grades which graduate engineers received in the technical schools. Many difficulties were encountered in attempting to deal with technical school grades statistically and to derive reliable conclusions. Transmutation and Tabidation of Technical School Grades .— A letter explaining the nature of this study and the purpose in PERSONNEL SELECTION OF GRADUATE ENGINEERS 9 requesting the grades was sent to over sixty different technical schools to secure the grades of 116 students. Transcripts of grades for 94 students were received. No two schools used the same form on which to record the grades; and there were at least seven distinctly different systems of grading with several varia¬ tions of each system. The grades of men from different schools had to be made comparable. There were not enough men from any one school to make it possible to transmute their grades into standard deviations, so another method was adopted. The range of possible grades between the just passing grade and the highest possible grade, was learned either from the college catalogue or from the key on the transcript of grades. The median point on that scale was estimated and called the average grade. All such average grades were given a marking or value of 2. Grades above average were given a value of 3; and grades below average were given a value of 1. Grades below passing were given a value of o; and exceptionally high grades in a subject, shown to be reliable by more than one highest marking on that subject, were given a value of 4. The method of transmutation of the grades can be shown by the following table which gives some comparable values. Table I. Transmutation of Technical School Grades into Common Values. Common Values into which Grades are Transmuted. 0 1 2 3 4 System using per cents.. -69 f 70-79 80-89 90-97 r 1 98-100 System using letters. E D c B A System using passing marks . F P— p p+ All P+ System using descriptive terms . - -- Fail Poor Good Superior Excellent System using merits .... Fail Pass with low mark Pass Merit Honor System using points.... 0-.9 1.0-1.9 2.0-2.9 3-0-37 3.8-4.0 System using class rank.. 10% of class20% of class 40% of class 20% of class io% of class 10 BRUCE V. MOORE In transmuting grades from institutions with high standards and strict grading systems, the evaluations were made higher than with institutions known tO' be below the average standing in the requirements it made of its students. Altogether, the trans¬ mutation was coarse as a statistical method; but it was the best that could be done with the data available; and it is considered sufficiently reliable to give significant indications. All the subjects studied by the engineers were classified under five headings or general subjects, namely,—Languages, including English and foreign languages; Mathematics and Science, in¬ cluding mechanics, physics, chemistry, and all special phases of these exact and theoretical sciences; Shop; Engineering, includ¬ ing all kinds of engineering, but chiefly electrical and some mechanical; and all Academic subjects, which is, in the case of engineers, only history and economics. Comparison of Technical School Grades with Other Criteria .— The first question to raise about the grades is in regard to their relation to success in industry or practical work. These en¬ gineers had not yet been in full-time work, but their year of work in the Educational Department and in the shop under the direction of that department, could be considered as one form of practical work in which success is to be determined. The esti¬ mates made by the committee of the Educational Department, explained under the title, “Criteria of Success,” might be con¬ sidered as one rating of success in the Educational Department. Grades given in the Educational Department classes, and grades given by the foremen on the shop work, can be considered as another check. Ratings on various qualities, explained in a sec¬ tion later, could be considered as the success of the men in im¬ pressing the foremen with their ability, and might be related to school grades. Also, the correlation between technical school grades and the ratings of the men made by the representatives of the Educational Department at the time of the employment inter¬ view, might be some measure of the extent to which the two measure the same qualities. Finally, a positive correlation be¬ tween mental alertness or general intelligence test and college PERSONNEL SELECTION OF GRADUATE ENGINEERS ii grades would be at least some evidence that school grades were some indication of what is measured by tests as general intel¬ ligence or mental ability. The correlations between technical school grades and the various other measures or criteria are as shown in Table II. Table II. Correlations of Technical School Grades with Other Measures. No. Correlation of cases (r) Technical school grades with estimated intelligence, committee’s rating . 88 .0 Technical school grades with educational department class grades . 60 .o Technical school grades with foremen’s shop grades... 62 .0 Technical school grades with foremen’s ratings on traits 85 .0 Technical school grades with ratings at interview for employment . 21 +.22 Technical school grades with Bureau Test VI. 56 +-37 Grades in languages with foremen’s ratings on traits... 59 .0 Grades in mathematics with foremen’s ratings on traits.. 72 .0 Grades in shop with foremen’s ratings on traits. 31 .0 Grades in engineering with foremen’s ratings on traits.. 66 .0 Grades in academic subjects with foremen’s ratings on traits . 44 .0 Grades in languages with Bureau Test VI. 43 .0 Grades in mathematics with Bureau Test VI. 54 +.34 Grades in shop with Bureau Test VI. 24 -j—67 Grades in engineering with Bureau Test VI. 53 +.26 Grades in academic subjects with Bureau Test VI. 35 .0 The correlation coefficients between technical school grades \ and other criteria, particularly the Company’s Educational De¬ partment class grades, shop grades, and shop ratings, might be taken to indicate that the technical school grades are unreliable. However, I do not believe that this conclusion is justified. Rather, I believe that the shop and class grades, and the ratings are unreliable, or are largely the cause for the lack of correlation. The cause for this belief will be shown later in this report. The composite of technical school grades shows a significant correlation with intelligence test scores. Doubtless the correla¬ tions would be higher if the grades were not from so many dif¬ ferent schools with different standards. I believe that in any particular school, an average or composite of the grades received by each student would rank the students in general ability as well as any measure; but the standards of work and the methods of 12 BRUCE V. MOORE grading are so incomparable in different schools that they can not be used by the employer to compare men from different schools. A standardized psychological test would show the same thing in much more reliable and usable form. That is, a general intelligence test given to a group of engineers selected from sixty different technical schools would give a more reliable rank¬ ing of the men according to their ability than a comparison of grades from these different schools; and the ranking could be obtained much easier by a test than by an attempt to transmute various systems of grades into a common scale. The intercorrelations of the grades in the various subjects is shown in Table III. As might be expected, the correlation be¬ tween closely related subjects, such as Mathematics and Engin¬ eering, (r = +.57) is greater than between what are usually considered less closely related subjects, such as Academic Sub¬ jects and Shop (r = +.30). There might be some a priori question about the correlations between Shop and Mathematics or between Shop and Engineering, but the correlations are shown to be comparatively high, being (r = -\-.61 ) (r = —{-.42) respect¬ ively. Shop grades correlate higher with Bureau Test VI scores than any other technical school subjects do; but the number of cases is so small that the large probable error makes the correlation co¬ efficient highly unreliable. However, it will be seen later that the design engineers, who are picked men and are supposed to be the most capable men in pure mathematics and engineering, have the highest grades in Shop, and are differentiated more widely from the other engineers by the average of their grades in Shop. It seems that the grades that a man receives in his shop work in the technical schools is significant of his special engineering apti¬ tude, and even of his general mental capacity. For differentiating the engineers for the different lines of work which they are to do, I thought that grades in different subjects might be significant. The average grade in each main subject for each class of engineers was computed, and curves plotted to show the relation of one group of engineers to the others. (See Diagram 1.) The design engineers are above the PERSONNEL SELECTION OF GRADUATE ENGINEERS 13 Table III. Intercorrelations of Technical School Subjects. Subjects of cases 00 Number Correlation Languages with mathematics. 77 +.38 Languages with shop . 35 +-37 Languages with engineering . 69 +-55 Languages with academic subjects . 53 -(-.30 Mathematics with shop . 43 -(-in Mathematics with engineering . 85 4~-57 Mathematics with academic subjects . 57 -(-.42 Shop with engineering . 63 -j-42 Shop with academic subjects . 21 +-3 0 Engineering with academic subjects . 54 -{-.35 other engineers in every subject, indicating that they were su¬ perior students in the technical schools. The curves of the gen¬ eral engineers and of the operating, service, and works manage¬ ment engineers run comparatively close together, the average of one group in a subject rising or falling with the average of the other group of engineers. The average of the sales engineers varies inversely with the averages of the general and the oper¬ ating, works, and service engineers. The significant feature of the averages, however, is that the design engineers are very much higher in all subjects, except the academic subjects, com¬ posed chiefly of economics, while in this subject the sales en¬ gineers have almost as high an average, although the general or all-around ability of the design engineers is evidently higher. The man who has been selected for other reasons as promising of becoming a good sales engineer, is the one who had special ability in the social sciences. VI. Evaluation of Grades and Ratings Given During Training in the Educational Department of the Industrial Firm As soon as the engineers, usually just graduated from the tech¬ nical school, enter the employ of the Company, they begin the year of training in the Educational Department. Practically all the engineers spend the first two months taking the same courses and doing the same work. In addition to class work, they work about a month in one department of the shop and then pass on to another kind of work so as to become familiar al u o Diagram i. Averages of Technical School Grades for Each Group of Engineers. PERSONNEL SELECTION OF GRADUATE ENGINEERS 15 with as many phases of the manufacturing process as possible. To make the absorption of the men into the shop possible and efficient, the men have to be divided into many groups and sent into many different sections, but the work for all is very similar at first. At the end of two months, however, there is begun the work of segregating the men into groups according to the line of work that they will do permanently. That is, the sales en¬ gineers and the design engineers, works management engineers, general engineers, etc., are picked out. This process of trans¬ ferring the men from one department to another and of deter¬ mining to what line of work each will be eventually assigned, con¬ tinues until the end of the sixth month. Beginning with the sixth month a Sales School and a Design School are started; and the engineers begin specialized training for their future work. From the time that the engineers enter the Educational De¬ partment, extensive records of their work are kept. Most of the men are first assigned to industrial motor winding and are graded on that work, both in the classes and in the actual work in the shop. In this work, the correlation between the shop grades and the class grades was r = +.19, for 76 cases. The grades for any one group of engineers plotted separately showed no different correlations. This low correlation shows the disagree¬ ment of the impressions received by different people, and the un¬ reliability of opinions concerning traits in people, particularly when the judgment must be made only after a short period of observation of the subject’s work. The curves in Diagram 2, show that the design engineers receive the highest grades in the shops. This agrees with the fact that they received the high¬ est grades in Shop in the technical school grades. In addition to the grades in the different subjects or kinds of work, there come from the foremen over each department, ratings on various traits of the engineers working temporarily under these foremen. These ratings are on ten different traits or characteris¬ tics; and they are made in terms of A, B, and C. The names of the ten traits are reliability, industry, initiative, tact, attitude, analytical ability, aptitude, enthusiasm, personality, decision. 16 BRUCE V. MOORE Since the men are transferred each month and rated by different foremen, several ratings in each trait are recorded. By combin¬ ing these ratings a more discriminating rating on each man of the group being studied was obtained. That is, the values A, B, and C were transmuted into the values 2, 1, and o, respectively; and by combining and averaging these ratings, fractional values be¬ tween these integers were obtained, such as 0.8 or 1.3 or 1.7 as a man’s average rating. Correlations of these shop ratings with other data on the men, are given in Table IV. Table IV. Correlations of Shop Ratings with Other Criteria. Number Correlation Subjects of cases (r) Shop ratings with estimated intelligence, committee’s rating . 95 .0 Shop ratings on analytical ability with estimated in¬ telligence, committee’s rating. 94 .0 Shop ratings with Bureau Test VI scores. 58 .0 Shop ratings on analytical ability with Bureau Test VI scores . 56 .0 Shop ratings with ratings at interview for employ¬ ment . 27 .0 Shop ratings on analytical ability with ratings on in¬ telligence at interview for employment. 26 .0 Shop ratings on personality with ratings on personal qualities at employment interview. 27 -{-.25 Shop grades with shop ratings . 74 .0 From these correlations we conclude that the ratings made by the foremen on the men working under them only one month, are very unreliable, and really indicate nothing. Doubtless, this unreliability cannot all be charged to the inability of the fore¬ men to judge the men on some of these traits; but the method of rating, of recording these ratings, and of combining ratings by different foremen, makes them incomparable and unreliable. In the shop and the Educational Department, the interpretations of analytical ability and of aptitude are practically the same as that which is called general intelligence by the psychologists. The ratings on analytical ability were correlated with estimated intel¬ ligence and with test scores in the hope that a significant positive correlation would be found, but there was none. One exception to the lack of positive correlation is the small correlation be¬ tween personality as judged by foremen and personal qualities PERSONNEL SELECTION OF GRADUATE ENGINEERS 1 7 as judged by the representatives of the company who rated the men at the interview for employment. It seems reasonable to expect such a positive correlation; for it is doubtless the general personality which influences both the interviewer’s and the fore¬ man’s rating on a man in most of the traits, in much the same way. Table V. Some Intercorrelations of Ratings by Foremen on Traits of Engineers. Number Correlation of cases (r) Reliability with industry . 117 -f-.66 Initiative with tact . 113 -{-.66 Initiative with enthusiasm . 107 +.42 Analytical ability with aptitude . 102 -j-.72 Analytical ability with personality . 104 -{-.26 Enthusiasm with personality . 106 +-57 Correlations between the ratings on some of the traits were computed. The highest correlation found is between analytical ability and aptitude, + .72, which seems a priori reasonable. The next highest correlation is (r = +.66) between reliability and in¬ dustry, which might be expected; but there is the same correla¬ tion between tact and initiative, two traits which a priori we should judge would tend to be found in somewhat opposite types of individuals. The lowest correlation is between analytical ability and personality, which does not seem unreasonable. To discover whether the ratings on the ten different traits could be of any value in differentiating the engineers, the average of the ratings on each trait for each group was computed. These averages are shown in Diagram 2. The curves tend to follow each other very closely. Also, it was found in most of the rat¬ ings that if a man was rated high in one trait, he was rated com¬ paratively high in another. Personality and enthusiasm differ¬ entiate the groups most widely. Frequency column diagrams, Diagrams 3, 4, 5, and 6, show that the ratings in these two traits and in tact and initiative, tended to separate the sales en¬ gineers, and the work management engineers from the other engineers. By combining the ratings in these four traits we get a very definite differentiation of the four groups of engineers. See Diagram 2. It seems evident that people on coming in con- tn u ^ a rP pq< c • f-H C C /5 53 £-3 o 2 C/2 £ jO 13 PQ 1 1 MM 1 I M Mil IMl MM 1 II II 1 I 1 Diagram 13. Comparison of Scores in Part II with Chief En¬ gineer’s Rating of Design Engi¬ neers. ru = .59. Ratings by Chief Engineer Lower Higher s _. rt _s c/j CJ V, > b P e/T • . O £ O. o eu O CQ 1 1 MM IMl II II Mil III! 1 1 1 11 1 1 1 Diagram 14. Comparison of Scores in Composite of Part I and Part II with Chief Engineer’s Ratings of Design Engineers, r^j — . 6 * 5 . Ratings by Sales Managers 03 ■o ■“-I ti q 03 -P Pp< C /5 2 qj ™ O '"5 C/2^ £ o 13 pq Low r er Higher IMl 1 1 1 1 m ! 1 II Mill Mill! Mill M 1 1 1 1 II II M M M 1 1 M M ll 111 11 1 Diagram 16. Comparison of Scores in Part II with Sales Managers’ Ratings of Sales En¬ gineers. u — -f- .81. Ratings by Sales Managers Below Median Above Median e o3 *3 a .2 C/5 5 (U 03 a 03 Q in O > cu£ C qj offl L> 1,,,. II II 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 II 1 1 II II 1 1 1 1 II /III 1 1 II 1 1 1 1 1 1 Diagram 17. Comparison of Scores in Composite of Part I and Part II with Sales Man¬ agers’ Ratings of Sales Engi¬ neers. r n = -f- .66. PERSONNEL SELECTION OF GRADUATE ENGINEERS 67 tions. It is evident that Part II, the technical part prepared for design engineers, correlates more nearly perfectly with the rat¬ ings than Part I does. The composite score, which takes into account both the special engineering capacity and the general in¬ telligence or mental alertness of the man, correlates a little more nearly perfectly. Diagrams 15, 16, and 17 show the same relations for the sales engineers; that is, the correlations between the test and the ratings made by the sales managers. Considering again only the design engineers, the mean of the scores in Part I was 109.3; and the mean of the scores in Part II was 37.0, which is approximately one-third of the mean for Part I, 37.0 multiplied by three being m.o. For the sales en¬ gineers, the mean of the scores in Part I was 87.6; and the mean of the scores in Part II was 22.7, which is approximately one- fourth of the mean for Part I, 22.7 multiplied by four being 90.8. Thus the sales engineers do relatively better in Part I as compared to Part II than the design engineers do. Comparing in another way, the design engineers do relatively better in Part II as compared to Part I than the sales engineers do; for the mean of their scores in Part II is more than one-third of the mean of the scores in Part I. Diagram 18 is a scatter diagram showing the relation between Part I. and Part II. for the scores of the design engineers and the sales engineers. Diagram 19 shows the same relation between the percentiles of each man in Part I and Part II. The circles (O) represent the sales engineers; and the crosses (X) represent the design engineers. One purpose of the test is to distinguish sales engineers from design engineers. According to the hypo¬ thesis for the test, the men who did well in both parts of the test and appear in the upper right-hand quadrant of the diagram, are mentally capable of being either sales engineers or design engineers because of their superior general all-around intelligence, or because of their superior intelligence in at least these two fields. The men who did poorly in Part II. but very good in Part I, should be those whose kind of intelligence fits them better for the work of a sales engineer. The larger percentage of circles in this quadrant substantiates this part of the hypothesis. Those 63 BRUCE V. MOORE Scores in Scores of Part II (Technical) Parti. 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31 32-35 36-39 40-43 44-47 48-51 52-55 56-60 (General) 155-160 150-154 145-149 140-144 135-139 130-134 125-129 120-124 HS- 1 ^ 110-114 105-109 100-104 95-99 90-94 85-89 80-84 75-79 70-74 65-69 60-74 55-59 50-54 45-49 40-44 35-39 30-34 25-29 20-24 15-19 10-14 5-9 0-4 J -1 a X 1 X q X X 1 1 O X 0 X • 1 O X X —r 1 O 0 X 0 8 X 0 0 O 5 O X X X X 0 | 0° * X O O ■R 1 1 X X X Me Lfl . o .O d — k. X 0J. O O X O 0 X i O 0 0 O X 0 O 1 0 O O 1 0 0 X • 0 0 0 O O ! O X 0 0 o° O 0 0 i X 0 0 0 1 i 1 • e' 1 Diagram 18. Scatter Diagram Showing Relation of Scores in Part I to Scores in Part II for Design Engineers (+) and for Sales Engi¬ neers (o). who did poorly in Part I. but very good in Part II., should be those whose kind of intelligence fits them better for the work of a design engineer. The larger percentage of crosses appear¬ ing* in this quadrant substantiates this part of the hypothesis. Those doing poorly in both Part I. and Part II., might be equally successful in either line of work, as far as mental ability is con¬ cerned; but they are not as successful as any of the others might be if properly placed in the right line of work. As a whole the hypothesis is substantiated by the results. There are some PERSONNEL SELECTION OF GRADUATE ENGINEERS 6g Percentiles Percentiles in Part II (Technical) in Part I. 0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-100 (General) __ 95-100 O X X X X 90-94 0 X 0 X 85-89 o° X X X 80-84 0 X 0 75-79 O o° 0 X X 70-74 O X X X 65-69 O o° X O 60-64 O X 55-59 o° X X 50-54 0 O Ox O X X 45-49 0 O 0 X 40-44 O 0 0 35-39 O X 0 X 30-34 O 0° X O O 0 0 25-29 O O 0 20-24 O O X 15-19 0 O 10-14 O O O O O o°o X 5-9 O O X 0-4 O OO O Diagram 19. Scatter Diagram Showing Relation of Scores in Part I to Scores in Part II in Terms of Percentiles for Design Engineers (+) and for Sales Engineers (0). exceptions as seen from the positions of the circles and crosses on the scatter diagram; but investigation shows reasonable causes for these, which do not disprove the hypothesis. The test was made for graduate student engineers in the Educational Depart¬ ment of the Westinghouse Company, and all these engineers are always just graduated from the technical schools. Therefore, they would not have the difficulty of having specialized in in¬ dustry for several years and having forgotten some of their theor¬ etical or technical training. Some of the design engineers who took this test had been doing very highly specialized work for five years, and had forgotten some of the information in other ;o BRUCE V. MOORE lines of electrical engineering*. For this reason, they may not have done so well in Part II, although they were regarded by the Chief Engineer as among his best engineers, particularly because of their experience, conscientious work, and constant applica¬ tion. However, I do not think that the specialization or ex¬ perience of the design engineers nor of the sales engineers ma¬ terially affected their scores; for Part II is a test of engineering aptitude and not of engineering information, and Part I is a general intelligence or mental alertness test. The correlation between Part I and Part II was r = +.42 for the design en¬ gineers, and r = +.84 for the sales engineers. If the design engineers have special capacity for engineering, as we believe they have, it is to be expected that their score in Part II, which tests engineering capacity, would correlate less with their scores in Part I, which is a general intelligence test, than the sales en¬ gineers’ scores in Part II would correlate with their scores in Part I. The scatter diagram of the scores of the design en¬ gineers in Part II shows that the correlation between the parts is lower because some of the design engineers had special aptitude in engineering, but could not do well in the general intelligence or mental alertness test. (f) Statistical Interpretation of the Results for Occupational Placement .'—The test is to be used in two different ways for two different purposes. The first use is to separate the men who are best fitted to be sales engineers from those who are best fitted to be design engineers. What is desired in this case is to know for what the men are best fitted by their special abilities, not con¬ sidering their general intelligence or mental alertness. A man might be more successful as a design engineer and might make a better score in Part II than another man; but even so he would not be so well placed if he is placed as a design engineer when he could be still more successful as a sales engineer. Each man should be placed not where he would be merely successful, but where he could function most efficiently and be most successful. Therefore a single score in a sales test or in an engineering test could not be relied upon to place the men; for the good all-around man would do best in both kinds of work, and best in both kinds PERSONNEL SELECTION OF GRADUATE ENGINEERS 71 of tests. However, some measure of the kinds of abilities or the qualitative differences in intelligence could be obtained by subtracting the score of each man in one test from his score in the other test. Suppose, for each individual, we subtracted the score of Part II in terms of the standard deviation from the score of Part I in terms of the standard deviation. The man who had a score in Part I which was considerably higher than his score in Part II would have a large algebraically positive dif¬ ference between his scores, and he would be a sales engineer type of man. This great difference between his scores in each part of the test might be because he did well in Part I, or did poorly in Part II, or for both reasons. In any case, he would be better fitted for sales engineering than he would be adapted for design engineering. The man who had a score in Part I considerably lower than his score in Part II, would have a large algebraically negative difference between his scores; and he would be a design engineer type of man. This great difference between his scores in each part of the test might be because he did poorly in Part I, or did well in Part II, or for both causes. By use of this method, that is, transmuting the scores into terms of sigmas and sub¬ tracting the score in Part II from the score in Part I, those who get the greater algebraically positive differences are to be con¬ sidered the more specially fitted for sales engineering; and those who get the greater negative differences are to be considered the more specially fitted for design engineering. The practical use of this method is to rank the men according to the algebraical differences between their respective scores in Part I and Part II, and then count down the list until the desired number of sales engineers is obtained. That is, the man who did well in Part I, but poorly in part II, would rank high in the list according to the algebraic difference between his score in Part I and his score in Part II, and he would be chosen for sales engineering. This method, theoretically, selects the men who can function most efficiently as sales engineers, and leaves those who can function most efficiently as design engineers. To check this method of classifying and ranking the thirty design engineers and the fifty-nine sales engineers, the score of 72 BRUCE V. MOORE each individual for Part II was subtracted from his score in Part I, both scores being in terms of their respective standard devia¬ tions. This gave positive although not the most desirable results in separating the sales engineers from the design engineers. These results are shown graphically in Diagram 20. Consider- ^ Diagram 20. Differentiation Obtained by Subtracting Scores in Part I from Scores in Part I in Terms of Standard Deviations. Considering the diagram as a four-fold table, the engineers indicated by the test as sales engineers are 79 per cent correctly so classified; and the engineers indi¬ cated by the test as design engineers are 49 per cent correctly so classified. r u — + 45 . ing the diagram as a four-fold table, this method of interpreting the results differentiates the men so that of those classed as sales engineers, 79 per cent are correctly so classed; and out of those classed as design engineers, 49 per cent are correctly so classed. The correlation between the test results treated in this way and the actual classification was ru = + .45. Also, the correlation between the classification of the men by the test treated in this way and the classification of the men by the “Section II. Choice of Other Occupations” of the Record of Interests , was r = +.50. A similar and simpler method of treating the results is to trans¬ mute the raw scores into percentile ranks, and subtract the per¬ centile rank in Part II from the percentile rank in Part I. This was done, and the classification of the engineers was practically the same; for the correlation between differences in percentile ranks (Part I minus Part II and the differences in scores in terms PERSONNEL SELECTION OF GRADUATE ENGINEERS 73 of sigma (Part I minus Part II in terms of sigmas) was r = +.97. The best method of dealing with the results is to divide the score in Part I by the score in Part II for each individual. This is the method which is the simplest, and it is the one which gives the most efficient use of the test as a means of differentiating the men. This method magnifies the differences among the in¬ dividuals, which differences are shown by showing that the ability of each individual to deal with Part I differs from his ability to deal with Part II. That is, the ratio or quotient of a man’s score in Part I divided by his score in Part II changes geo¬ metrically, instead of algebraically, as this man’s special ability is greater or less. This probably makes the results agree more nearly with the practical considerations; for the extreme cases of special ability should be markedly differentiated. The genius is valued probably far greater than an actual measurement of capacity would indicate. The results of dealing with the scores by this method are presented graphically in Diagram 21. Consid- Diagram 21 . Differentiation Obtained by Dividing Scores in Part I by Scores in Part II. Considering the diagram as a four-fold table, the engineers indicated by the. test as sales engineers are 81 per cent correctly so classified; and the engineers indi¬ cated by the test as design engineers are 63 per cent correctly so classified. r U = +70. 74 BRUCE V. MOORE ering the diagram as a four-fold table, we have the engineers differentiated as follows: Knowing first the actual classification of the engineers, we can say that 63 per cent of the design engineers were shown by the test to be better fitted for design engineering than 81 per cent of the sales engineers were shown to be. 81 per cent of the sales engineers were shown by the test to be better fitted for sales engineering than 63 per cent of the design engineers were shown to be. Not knowing first the classification of a group of en¬ gineers, this test can be used to differentiate them as follows: Assuming that if an engineer’s score in Part I of the test is more than 3.2 times his score in Part II, he is a sales engineer, and that if his score in Part I is less than 3.2 times his score in Part II, he is a design engineer or a pure type of engineer, this test divides the engineers so that 81 per cent of those above this crit¬ ical ratio (Part I at least as much as 3.2 times Part II) are sales engineers; and 63 per cent of those below it are design engineers or of a purely engineering type. (This critical ratio may be changed slightly by the change from the mimeographed form of the test to the printed form.) The correlation between the test results treated in this way and the actual classification of the men were ru = + .70. Also, the correlation between the classifi¬ cation of the men by the test results treated in this way and the classification of the men by the Record of Interests, Section II, was r == +.46. The correlation between the marks for the en¬ gineers when the scores were treated in this way (score in Part I divided by score in Part II) and the marks for the engineers when the score in part II was subtracted from the score in Part I in terms of sigmas, was r = +.50. Such was the correlation coefficient obtained when the quotients (Part I divided by Part II) were correlated with differences between scores (Part I minus Part II in terms of sigmas) by the Pearson product-mo¬ ment formula. However, this does not show the true relation¬ ship; for the relationship is a non-linear one; and by using the formula for eta, it is shown to be y — .88. PERSONNEL SELECTION OF GRADUATE ENGINEERS 75 The second use of the test is to rank the sales engineers and the design engineers after they have been separated, so as to show who are the best design engineers, and who are the best sales engineers, considering general intelligence or all-around mental ability as well as special ability. In other words, this use of the test makes it a general intelligence or mental alertness test. The relationships of the results of the test with the estimates of the success of the engineers have already been given in the pre¬ vious section. There the composite of Part I and Part II was obtained by simply adding the scores of the two parts in terms of their standard deviations. The composite does not necessarily show a higher correlation with the estimates of suc¬ cess than each part separately shows; for if the two parts test two widely different abilities, then combination may not give a higher correlation with the criterion. The simplest method of combining the scores is to add the percentile rank of each man in Part I to his percentile rank in Part II, and then rank the men according to these sums of percentile ranks. This gives results which are practically as reliable as those obtained by the method of combining scores in terms of sigmas. On the basis of the results obtained so far, the test, both Part I and Part II, is being given to the graduate engineers who are entering the employ of the particular industrial firm. Only by following up these men in the lines of work which they enter later, will we finally know the value of the rating scales, of the Record of Interests, and of this special aptitude test. The results already obtained on one hundred seven engineers employed by the firm during the first year that the test has been used are very similar to the results obtained on the group studied in this research. These hundred and seven men have been with the company approximately one year and they have been classi¬ fied according to previous methods into sales engineers and de¬ sign engineers by the executives supervising their work. The BRUCE V. MOORE 76 test results were not a factor in the classification. However, assuming that the executives classified the men correctly at the end of a year, we can say that the test results alone would have classified seventy per cent of the men correctly on the day that they were employed. Beginning with June, 1921, the company will use the test and other methods recommended here as definite aids in classifying and placing the graduate engineers whom they employ. PART III. SUMMARY OF RESULTS AND CONCLU¬ SIONS AND RECOMMENDATIONS X. Summary of Results The practical problem upon which we began work was to formulate methods and means based upon sound psychological principles for differentiating graduate engineers. The results of studying various existing methods and certain proposed or newly devised ones showed that some were of value and that others were not. The results might be summarized briefly as follows: 1. Technical school grades when coming from various schools of different standards, do not differentiate the men in a large group according to their ability in industry, nor according to the kind of work which they can do best. This is with the one exception that men who are selected to be design engineers are the kind of men who made higher grades in technical schools. This was true in all subjects except in general academic subjects, particularly economics, in which the design engineers fell below their own average, and the grades of the sales engineers were above their own average, so that the grades of the two groups were practically the same in these subjects. 2. Neither the grades given in the classes of the Educational Department nor the grades given in the shop on the courses of the Educational Department show any consistency or reliability. The correlation between the two sets of grades is only r = —19, and the correlation is zero with all other measures such as tech¬ nical school grades, ratings, and test scores. The unreliability of PERSONNEL SELECTION OF GRADUATE ENGINEERS 77 these grades and ratings is doubtless due to the method of making the ratings, of recording them, and of combining them, rather than being altogether due to the inability of the foremen to make the ratings. 3. The ratings made by the foremen on the ten different traits or qualifications show no agreement with any other measures such as shop and class grades, ratings, technical school grades, or test scores. There is high intercorrelation among the traits. This is due to two causes: (a) Some of the traits are very sim¬ ilar, or at least they are not well enough defined for the foremen to distinguish them; and as the foremen are not trained to make such ratings, they decide that a man is either a good man or a bad man, and then rate the man accordingly either good or bad in all traits, (b) The method of making the ratings, in terms of A, B, or C, does not give the foremen any scale or basis of measure or comparison. 4. The only traits rated by the foremen which differentiate the different groups of graduate student engineers, are tact, initiative, enthusiasm, and personality. The trait called per¬ sonality is the most vague and undefined, but it shows the greatest differentiation of the men; for it is a trait concerning which the foremen really get an impression and form an opinion in regard to each man. 5. The Interviewer’s Rating Scale as already in use by the Company, has given results that have significant correlation with test scores, with later estimate of intelligence, and with technical school grades. The trait called Personal Qualities, as previously defined, had a high correlation with other traits, and was too broad, inclusive, and ambiguous to give reliable results. It was then more narrowly defined so as not to be ambiguous and not overlap or include the other traits to be rated. 6. The College Instructor s Rating Scale was prepared to en¬ able the college instructors, professors, or deans to record a definite estimate of a senior on specific traits. This is expected to give more reliable and significant ratings of a man than a verbal expression of opinion of the man in general. 7. The Shop and Class Rating Scale was prepared to take 78 BRUCE V. MOORE the place of the method of rating on ten traits by A, B, and C. It requires ratings on a few very significant traits or qualities; but it is expected to enable the foremen and the instructor to have a common understanding of just what is meant by these traits and to give a reliable and significant rating that will differentiate the engineers according to essential qualifications. 8. The use of the Record of Interests shows that men in dif¬ ferent lines of work have different interests. That is, the insur¬ ance Salesmen as a group, have always had interests different from either sales engineers or design engineers. The sales en¬ gineers are men who have been more interested in economic, public, and social affairs. The design engineers are men who have been more interested in science, machinery, and actual con¬ struction or fabrication of things. 9. The engineers and the salesmen showed a definite tendency to like or be interested in occupations which in nature of work were similar to those they were already following. The kind of occupations which they thought would give them the greatest satisfaction to follow, was a significant criterion of the kind of work in which they could be and were already successful. By computing for each engineer the percentage which his choices of occupations of a sales nature were of his total number of choices, a definite measure of his sales engineering interest as opposed to design engineering interest was obtained. By this measure, 78 per cent of the sales engineers were of a sales type; and 82 per cent of the design engineers were of an engineering type. Or assuming that we did not know the actual occupations of the engineers, those engineers which this test of interests alone would select for sales engineering, would be 89 per cent correctly placed or classified; and the men which the test of in¬ terests selected for design engineering would be 68 per cent cor¬ rectly placed or classified. 10. Personnel Bureau Test VI, a general intelligence test similar to the Army Alpha intelligence test, gave results that did not differentiate the engineers according to the kinds of work for which they were thought to be fitted. The only men who had scores significantly different from the other engineers, were the PERSONNEL SELECTION OF GRADUATE ENGINEERS 79 design engineers. They were a highly selected group of men who had shown ability in the engineering classes of the Company’s Educational Department. 11. The general intelligence test gave scores that correlated, r = +.48, with intelligence as estimated by two executives of the Educational Department who were intimately associated with the graduate student engineers one year. 12. The general intelligence scores correlated, r = +.38, with technical school grades, and with interview ratings at the time of employment, r= +.58; but the correlations were zero with shop and class ratings, which did not correlate with any other criteria, and were evidently unreliable. 13. Test io } A Test for Graduate Student Engineers, gave positive although not perfect results in separating the sales en¬ gineers from the design engineers. Knowing first the actual classification of the engineers, we can say that 63 per cent of the design engineers were shown by the test to be better fitted for design engineering than 81 per cent of the sales engineers were shown to be. 81 per cent of the sales engineers were shown by the test to be better fitted for sales engineering than 63 per cent of the design engineers were shown to be. Not knowing first the classification of a group of engineers, this test can be used to dif¬ ferentiate then as follows: Assuming that if an engineer’s score in Part I of the test is more than 3.2 times his score in Part II, he is a sales engineer, and that if his score in Part I is less than 3.2 times his score in Part II, he is a design engineer or of a pure engineering type, this test divides the engineers so that 81 per cent of those above this critical ratio (Part I 3.2 times Part II) are sales engineers, and 63 per cent of those below it are de¬ sign engineers or of a purely engineering type. (This critical score may be changed slightly by a change from the mimeo¬ graphed form of the test to the printed form.) 14. The differentiation of the engineers resulting from the use of Test 10 showed a comparatively high correlation with the results of the Record of Interests. The correlation was r = +.50 when the scores in Part II were subtracted from those in Part I in terms of sigmas. The correlation was r = +.46 8o BRUCE V. MOORE when the differentiation was obtained by dividing the scores in Part I by the scores in Part II. This shows not only that there is high correlation between interest and special ability, but also that the Record of Interests and Test io give reliable results. 15. Test 10 used as a general intelligence or mental alertness test for sales engineers gave scores that correlated, r = +-86 and -{-.81 with estimated success. The same test used as a gen¬ eral intelligence test for design engineers gave scores that cor¬ related, r=+.22 and +.59, with estimated success. XI. Conclusions and General Principles Derived from the Study 1. The study of the technical school grades show that such grades cannot be made of practical use to differentiate engineers for various kinds of work, except in a very general way. Sales engineers do comparatively better in economics than in engineer¬ ing subjects. The design engineers make much lower grades in economics than they do in mathematics and science, engineering, and shop. 2. When foremen are asked to rate men under them on a large number of traits which are not carefully defined, the ratings are unreliable and of little use as a measure of the value of the men rated; and such ratings are particularly of no use for classi¬ fying the men and placing them in positions for which they are best fitted. Generalizing, we can say that ratings made by dif¬ ferent people on traits which are understood by some to be one thing and by others to be another, are of no value. This difficulty arises if the traits are not carefully defined in terms that the raters can understand; and there is the same difficulty if there are many traits to be rated, some of which overlap or include others so that they can not be differentiated. The ratings are un¬ reliable if there is no scale or standard with which the men can be compared. This basis of comparison may be the other men of the same group; but there must be definite comparison instead of arbitrary grades or marks. Finally, the ratings or marks must be recorded in such a form that they can be readily tabulated and actually used in placing the men. PERSONNEL SELECTION OF GRADUATE ENGINEERS 81 4. Carefully prepared acquaintance-comparison rating scales, such as the Interviewer’s Rating Scale and the Shop and Class Rating Scale, give much more reliable results than forms on which arbitrary grades or marks are recorded opposite certain names of traits. The acquaintance-comparison rating scale en¬ ables the rater to compare the one to be rated with other people as concrete examples of different grades of the same trait. 5. The use of the Record of Interests shows that men in dif¬ ferent lines of work have different interests. The interests of a person are not in just one specific occupation, but they are general to the extent that they pertain to very similar or closely allied oc¬ cupations or activities. Interest in a certain class of activities is a criterion that the person will be interested in any other very sim¬ ilar activity. By similar activity is meant one that requires much the same information, training, experience, kind of materials and tools worked with, mental activity, personality, ideals of accuracy and perfection, and social attitude. 6. In the engineering profession, men cannot be differentiated for different lines of work by tests which measure what is com¬ monly known as general intelligence or mental alertness. Men in the same kind of engineering work differ in this intelligence as greatly as men in different kinds of engineering work. The differences which fit these men for different kinds of work, are something other than differences in intelligence. 7. General intelligence as measured by a general intelligence test does show a significant positive correlation with the success of engineers in the same kind of work. 8. One man differs from another in having special abilities, which functioning in an integrated form as a kind of intelligence, can be measured and used as criteria for placing the man in the kind of work for which he is best fitted. 9. The occupational interests of a man show a definite cor¬ relation with the kind of intelligence or special abilities which he has, and with the kind of occupation in which he is successful. 82 BRUCE V. MOORE XII. Specific Methods and Practices Recommended 1. Before senior engineers who are being graduated from the technical schools, are employed, they should be rated by the use of rating scales. The representatives of the Company who interview the seniors, should use the Interviewer's Rating Scale. Before the interviewers start on their visits to the colleges, they should carefully make a Master Rating Scale from the list of engineers whom they know intimately and who have passed through the Educational Department within the previous five years. If possible, two or three interviewers representing the Company should interview the senior and make ratings independ¬ ently, and then combine the ratings later. 2. After the interviewers have returned to the home office and have prepared a large tentative list of seniors who are to be considered for employment, the name of each senior should be put on a blank College Instructor's Rating Scale, and this blank sent to the dean, professor, or instructor who can be depended upon to give a reliable rating of the man under consideration. A rating by more than one instructor is desirable if it can be ob¬ tained without imposing on the instructors. Most of the pro¬ fessors are glad to help place their students. If the representative believes that the senior will be desired by the Company and that this senior will make application, he should get the rating from the professor while visiting the college or as soon as possible. 3. As soon as the graduate student engineers enter the Edu¬ cational Department of the Company, they should be under ob¬ servation in the shop and in the class to be rated at the end of each month by means of the Shop and Class Rating Scale. 4. Within the first two months after the graduate student engineer enters the Educational Department of the Company, he should be given the Test 10, A Test for Graduate Student En¬ gineers. This should be scored, first to determine for what kind of work the engineer is probably best fitted; second, to determine what the rank in general intelligence is. 5. At the end of two months or just before it is necessary to segregate the graduate student engineers for intensive training in PERSONNEL SELECTION OF GRADUATE ENGINEERS 83 some particular line of work, the student engineers should be requested to fill out the blank entitled Rceord of Interests. 6. At the end of two months or when it is necessary to know into what line of work the student engineers are going, the re¬ sults of the rating scales, the test, and the Record of Interests, should be combined. The impressions and opinions concerning the engineers which the executives, interviewers, and instructors of the Educational Department have recorded on the rating scales, should have a weight or vote of one-third in the final composite measure of the men; the test should have a vote or weight of one- third; and the Record of Interests should have a weight or vote of one-third to determine for what line of work the engineer should be recommended. In case of doubt, any two of these measures or criteria should determine for what work the partic¬ ular man should be recommended. Of course the judgments of the executives concerned will be the final criteria, but their im¬ pressions and opinions should be based upon dependable data and be corrected by this data. 7. The student engineer should be interviewed with his Record of Interests before him and the interviewer; and in the interview the student should be advised to enter the line of work for which he is best fitted as determined by this Record of In¬ terests, the test, and the rating scales. This advice should be given as an intelligent recommendation; but the student should not be urged against his will to enter a particular line of work. If the student objects to the recommendation, it should be dis¬ cussed in the light of the results of the Record of Interests and of the test. 8. After the student engineers have been segregated and as¬ signed tentatively to their respective lines of work, a copy of the Record of Interests and the results of the test should be trans¬ mitted to the heads of the respective departments in which the men will work. 9. The results of all rating scales, tests, and Record of In¬ terests, should be carefully recorded and filed; and the later suc¬ cess or record of the engineers should be systematically followed 8 4 BRUCE V. MOORE up in order to check the reliability and value of the methods and means of selection, classification, and placement being used. 10. Some person qualified for personnel research work should be assigned the responsibility for keeping rating scales and tests in proper use, working up the results, carrying on the research and follow-up work, and planning and directing new lines of research which should be undertaken to increase the efficiency of the educational, placement, and other personnel work. 11. The rating scales, tests, and Record of Interests should be studied to make further improvements as a result of their use. A duplicate or alternate form of the test should be prepared, which could be given in case there is reason to believe there has been coaching for the first form. 12. In the light of results obtained in the study of this prob¬ lem with graduate student engineers, research work in personnel problems should be extended to the apprentices, the clerical work¬ ers and the shop employees of the Company. BIBLIOGRAPHY Benge, E. J. Standard Personnel Practice. H. W. Wilson Co., New York. 1920. Bingham, W. V. Army Personnel Work. Journal of Applied Psychology, March, 1919, vol. 2, pp. 1-12. Frankel, L. K., and Fleischer, A. The Human Factor in Indus¬ try. Macmillan Co. 1920. Committee on Classification of Personnel. The Personnel Sys¬ tem of the United States Army. Vol. I, History of the Personnel System. Vol. II, The Personnel Manual. Su¬ perintendent of Public Documents, Washington, D. C. 1920. Hollingworth, H. L. Vocational Psychology. D. Appleton & Co. 1916. Kitson, H. D. Interest as a Criterion in Vocational Guidance. Educational Review, vol. 24, pp. 207-214, March, 1916. Link, Henry C. Employment Psychology. Macmillan Co. New York. 1919. Mann, Charles R. A Study of Engineering Education. Bulletin No. 11, Carnegie Foundation, 576 Fifth Avenue, New York. 1918. Moore, Bruce V. General Intelligence Determined by Its Weak¬ est Essential Element. Journal of Applied Psychology, vol. 4, pp. 151-161. June-Sept., 1920. Scott, W. D. Changes in Some of Our Conceptions and Prac¬ tices in Personnel. Psychological Review, 1920, No. 27, pp. 81-94. Scott, W. D. A Fourth Method of Checking Results in Voca¬ tional Selection. Journal of Applied Psychology, March, 1917, vol. 1, pp. 61-66. Tead, O., and Metcalfe, H. C. Personnel Administration. McGraw-Hill. New York. 1920. Thorndike, E. L. The Permanence of Interests and Their Rela¬ tion to Abilities. Popular Science Monthly, vol. 81, pp. 449- 456. December, 1912. Thurstone, L. L. Mental Tests for Engineering Students. Proceedings of the Society for the Promotion of Engineer¬ ing Education, vol. 27, 1919, pp. 113-119. Yoakum, C. S., and Yerkes, R. M. Army Mental Tests. Henry Holt Co., New York. 1920.