■137 I 1 SCHOOL DOCUMENT NO. 6-1916 BOSTON PUBLIC SCHOOLS PENMANSHIP DETERMINING THE ACHIEVEMENT OF ELEMENTARY SCHOOL GRADUATES IN HANDWRITING Bulletin No. IX. of the Department of Educational Investigation and Measurement BOSTON printing department I 9 I 6 In School Committee, Boston, February 7, 1916. Ordered, That five thousand (5,000) copies of a bulletin relative to a study of the quality of penman- ship, to be prepared by the Department of Educational Investigation and Measurement, be printed as a school document. Attest : Thornton D. Apollonio, Secretary. 0. of D- C INTEODUCTION. It is probable that there are more differences of opinion among teachers concerning the teaching of penmanship in the elementary schools and the quality of the results than there are concerning any other sub- ject in the course of study. In view of this and as a means of contributing to the improvement of results in handwriting, the department has considered it desirable to find out, in a systematic way, the character of the present achievement of elementary school graduates in penmanship ; this was the purpose of the study reported in this bulletin. The department believes that one of the best methods of securing improved results is by bringing to the attention of teachers an analysis of the merits and defects of the present handwriting of children. In consultation with the late Assistant Superin- tendent White, who had charge of penmanship, plans for this study were made. On his advice and that of Miss Ellen S. Bloomfield the following committee was selected to assist the department: HoNORA T. O'DowD, Master's Assistant, Hancock Dis- trict, Chairman. Thomas J. Baery, Submaster, Thomas N. Hart Dis- trict. Matilda F. Bibbey, Assistant, Hancock District. Emma J. Irving, First Assistant in Charge, Emerson District. Edward J. Muldoon, Submaster, Francis Parkman District. Edgar L. Raub, Submaster, John A. Andrew. District. In its work the committee has profited by the counsel of Miss Bloomfield, and also of Assistant Superin- tendent A. L. Rafter, who now has charge of penman- ship in the public schools. 4 SCHOOL DOCUMENT NO. 6. This committee was appointed March 24, 1915, and in the course of its work has held eight meetings. Sev- eral of these meetings were wholly devoted to the irk- some task of rating and classifying papers, and defining their merits and defects. Whatever value this report has is due largely to the work of this committee of enthusiastic, capable and effective teachers of penman- ship. This bulletin is divided into three rather distinct parts: Part I. is a description of the organization of this study and of the methods of carrying it on, prepared by Mr. Ballon; Part II. consists of a critical analysis of the merits and defects of the handwriting studied, written by Miss O'Dowd, chairman of the committee, assisted by the committee and particularly by Mr. Raub, who prepared Tables 7, 8, 9 and 10; and Part III. is a statement of the present status of penmanship in the city from the administrative point of view, pre- pared by Assistant Superintendent Rafter. Frank W. Ballou, Director. RESULTS OF STUDY IN PENMANSHIP. DETERMINING THE ACHIEVEMENT OF ELEMENTARY SCHOOL GRADUATES IN HANDWRITING. The material constituting this bulletin has been organized under the following heads: Part I. — The organization and methods of the study. Part II. — A critical analysis of the merits and defects of the handwriting studied. Part III. — The administrative status of penmanship in the city. PART I.— THE ORGANIZATION AND METHODS OF THE STUDY. Sources of Handwriting Papers. On November 23, 1914, a test in accurate copying was given to 4,494 pupils in the first-year classes of fourteen of the fifteen high schools in Boston. In the test noth- ing was said to the pupils about theh' penmanship; hence, they did not know that the quality of their hand- writing was to be considered. The handwriting, there- fore, may be thought of as typical of that which will characterize the handwriting of these young people during their high school course and, to some extent at least, similar to that which they may be expected to write during life. Although the test from which these specimens were secured was given to first-year high school pupils, their penmanship ability is the product of the instruction and training given them in the elementary schools and in this study has been considered as such. Method of Rating the Specimens. Obviously, not all of the 4,494 papers written by pupils in the accurate copying test could be conveniently 6 SCHOOL DOCUMENT NO. 6. rated by a small committee. Consequently, from the total number six hundred papers were selected at random. The papers from each high school had already been arranged in alphabetical order according to the names of the pupils, and the bundles from each school were arranged in like order. Approximately every eighth paper was taken from the bundles until the desired six hundred papers had been secured. The papers were then numbered from 1 to 600, inclusive, and throughout the study each paper was known by the number which it bore. For the purpose of rating the quality of the hand- writing the committee was divided into two subcom- mittees of three members each. The first three hundred papers were turned over to one subcommittee and the second three hundred papers to the other. Each one of the three members of the subcommittee rated inde- pendently each one of the three hundred papers, on a scale of 10-30-50-70-90 per cent, using the Ayres' scale for adult handwriting.* The even tens on the Ayres' scale were disregarded. Any paper which could not be rated as good as specimen No. 30 in the Ayres' scale was rated 10. While some papers were rated 10, there was no paper rated 10 by two of the three members of either committee. After each member of each subcommittee had inde- pendently rated each one of the three hundred papers, the whole committee met the director of the department in conference. Each member submitted his or her written report and the results were tabulated. The numbers of the specimens were arranged in ascending order, beginning with specimen No. 1. Opposite the number of each specimen were placed the preliminary ratings of each of the three members of the committee. Each specimen was finally given that rating which at least two of the three members agreed it should have. Where no two members agreed on a rating the paper was * Ayres. A Scale for Measuring the Quality of Handwriting of Adults. Russell Sage Foundation. RESULTS OF STUDY IN PENMANSHIP. given the middle rating. For example: If a specimen had been given preliminary ratings of 30-50-70, that specimen was given a final rating of 50 per cent. Comparison of Ratings of Committee Members. The tabulation showed some interesting variations in judgment of committee members concerning the quality of individual papers. Of the first three hundred papers no two members of the committee agreed on the rating of ninety papers, and in the second three hundred papers no two of the members of the other committee agreed on the rating of twenty-four papers. In the rating of the first three hundred papers there was entire agreement among the three members of the committee in thirty- one cases, and on the second three hundred papers there was entire agreement among the three members of the committee in seventy-six cases. That there was less agreement among the three members of the committee who rated the first three hundred papers is due to the fact that one member of the committee rated all the papers relatively lower than the other tw^o members, giving very few specimens a rating of 90 per cent, and rating ten papers as low as 10 per cent. As has already been pointed out, each specimen was rated independently by three different examiners (com- mittee members). The following table shows the pro- portion of each group of one hundred specimens given the different ratings by each examiner. TABLE 1. Ratings of the First Three Hundred Specimens. Nos. 1 to 100. Scale of Rating. ■ 90%. 70%. 50%. 30%. 10%. A 20 48 30 2 B 11 41 35 13 C 4 17 39 30 10 8 SCHOOL DOCUMENT NO. 6. Nos. 101 to 200. Examinee. Scale op Rating. 90%. 70%. 50%. 30%. 10%. A 40 14 5 48 29 20 12 43 38 14 24 B C 13 Nos. 201 to 300. A ;... 22 52 24 2 B 15 37 34 14 C 9 28 30 25 8 The above table is to be interpreted in the following manner: In rating the specimens numbered from 1 to 100, examiner A rated 20 of them 90 per cent, 48 of them 70 per cent, 30 of them 50 per cent, 2 of them 30 per cent and none of them 10 per cent. The number of speci- mens given each rating by examiners B and C is indi- cated in the same manner. The specimens were rated by each examiner in groups of 100. The tabulation is presented above in that form, because it makes possible the comparison of the judgments of examiners rating the same specimens. To illustrate: Among the first 100 papers, examiner A found 20 specimens of handwriting considered to be of the quality represented by 90 per cent, while examiner C found only 4 specimens which could be thus classified. On the other hand, examiner C rated 10 specimens 10 per cent, while neither of the other two examiners found any papers to be rated as low as this. Further, examiner A found only two specimens among the first 100 which could be rated as low as 30 per cent; whereas examiner C found nearly one third of the hundred papers which were considered to be of RESULTS OF STUDY IN PENMANSHIP. 9 the value indicated by 30 per cent in the Ayres' scale. The remaining portions of Table 1 are to be interpreted in the same manner. What has been pointed out as characteristic of the variations in judgment of examiners A, B and C in rating the quality of the first 100 speci- mens is characteristic also of their judgment on the other 200 specimens. TABLE 2. Ratings of the Second Three Hundred Specimens. Nos. 301 to 400. Examiner. Scale of Rating. 90%. 70%. 50%. 30%. 10%. D 6 1 27 32 53 59 51 39 11 11 7 tl E F Nos. 401 to 500. D 10 3 3 38 23 48 49 63 36 3 11 13 E F Nos. 501 to 600. D 9 2 32 34 47 41 12 *22 E F 4 55 27 tl2 * 1 omitted. \ 2 omitted. The above table shows, in similar manner, how the second 300 specimens were rated by examiners D, E and F. The most marked contrast is shown in the case of specimens numbered 301-400. While examiner E found only 1 specimen which could be rated 90 per cent, examiner F found 27 such specimens. This table 10 SCHOOL DOCUMENT NO. 6. shows that examiners D, E and F varied less in judg- ment concerning the quahty of the handwriting of specimens 301-600 than did examiners A, B and C in rating the quality of the first 300 specimens. It is to be noted that examiners D, E and F did not find any papers which they rated lower than 30 per cent. In order to show how consistent, or inconsistent, in judgment each examiner was in rating the three groups of papers which he or she rated, the following tables have been prepared. TABLE 3. Consistency of the Judgment of Each Examiner Rating Specimens Numbered 1-300. Examiner A. Scale of Rating. 90%. 70%. 50%. 30%. 10%. Nos. 1 to 100 20 40 22 48 48 52 30 12 24 2 2 Nos. 101 to 200 Nos. 201 to 300 Totals 82 148 66 4 Per cent of all 27% 50% 22% 1% 0% Examiner B. Specimens. Scale of Rating. 90%. 70%. 50%. 30%. 10%. Nos. 1 to 100 11 14 15 41 29 37 35 43 34 13 14 14 Nos. 101 to 200 Nos. 201 to 300 Totals 40 107 112 41 Per cent of all 13% 36% 37% 14% 0% RESULTS OF STUDY IN PENMANSHIP. Examiner C. 11 Scale of Rating. Specimens. 90%. 70%. 50%. 30%. 10%. Nos. 1 to 100 4 5 9 17 20 28 39 38 30 30 24 25 10 Nos. 101 to 200 13 Nos. 201 to 300 8 Totals 18 65 107 79 31 Per cent of all 6% 22% 36% 26% 10% Table 3 shows that examiner A gave a rating of 90 per cent to 20 of the first hundred papers, to 40 of the second hundred papers, and to 22 of the third hundred. The same examiner gave 70 per cent to approximately the same number of papers in each hundred. Examiner A found only two papers among the first hundred to be rated as low as 30 per cent, none among the second hundred, and two among the third hundred. Assuming that the papers in each hundred were of the same quality, this table shows that examiner A rated the second hundred relatively higher than the first or third hundred. Table 3 shows that examiner B gave no papers a rating of 10, and was very consistent in the number of papers given 30 per cent or 90 per cent. Examiner B, however, found a larger proportion of the second hundred papers to be given 50 per cent than of the other two groups, the reverse of what examiner A found. Examiner C rated relatively fewer papers 90 per cent and more papers 10 per cent or 30 per cent than did either of the other two examiners. At the foot of each table is given the total number of papers and the per cent of all papers given each rating. This per cent brings out the fact that examiner A rated all the papers relatively higher than either examiner 12 SCHOOL DOCUMENT NO. 6. B or C, and that examiner C rated all the papers rela- tively lower than either of the other two examiners. In view of the wide variations in judgment usually found in such work this table shows that, on the whole, the examiners were fairly consistent in their judgment of the quality of handwriting. TABLE 4. Consistency of the Judgment of Each Examiner Rating Specimens Numbered 301-600. Examiner D. Specimens. Scale of Rating. 90%. 70%. 50%. 30%. 10%. Nos. 301 to 400 .. . Nos. 401 to500... Nos. 501 to 600 .. . Totals Per cent of all 6 10 9 32 38 32 51 49 47 11 3 12 25 102 147 26 8% 34% 49% 9% 0% Examiner E. Scale op Rating. Specimens. 90%. 70%. 50%. 30%. 10%. Nos. 301 to 400 1 3 2 53 23 34 39 63 41 7 11 *22 Nos. 401 to 500 Nos. 501 to 600 Totals 6 110 143 40 Per cent of all 2% 37% 48% 13% 0% * One omitted. RESrLTS OF STUDY IN PENMANSHIP. Examiner F. 13 Specimens. Scale of Rating. 90%. 70%. 50%. 30%. 10%. Nos. 301 to 400 27 3 4 59 48 55 11 36 27 *1 13 *12 Nos. 401 to 500 Nos. 501 to 600 Totals 34 162 74 26 Per cent of all 12% 54% 25% y /o 0% * Two omitted The above table shows the same facts for examiners D, E and F that table 3 showed for examiners A, B and C. It is to be noted that no examiner rated papers as low as 10 per cent. Examiners D, E and F show less variation in judgment than examiners A, B and C, largely because of the lower ratings of examiner C. It should be pointed out, however, that the distribution of ratings of examiner C much more nearly approximates to the theoretical normal distribution than that of any other examiner. TABLE 5. Summary Table Showing the Proportion of Papers Rated by Each Examiner Which Were Given the Ratings Indicated. Scale of Rating. Examiner. 90%. 70%. 50%. 30%. 10%. A B C D 27% 13% 6% 8% 2% 12% 50% 36% 22% 34% 37% 54% 2% 37% 36% 49% 48% 25% 1% 14% 26% 8% 13% 9% 0% 0% 10% 0% E F 0% 0% 14 SCHOOL DOCUMENT NO. 6. This table is made up of data already appearing in Tables 3 and 4, and is introduced merely to afford those interested an opportunity to make a direct comparison of the proportion of papers given each rating by the several examiners. The table shows that examiner A rated 27 per cent of the first three hundred specimens 90 per cent, examiner B 13 per cent of the same papers 90 per cent, examiner C only 6 per cent of the same papers 90 per cent, and so on. Selection of Typical Specimens or Samples. On the basis of the preliminary ratings of examiners, the quality of handwriting of each specimen was given a final rating or evaluation, according to the consensus of the judgments of the examiners. From the final ratings of the specimens the following distribution resulted : TABLE 6. Final Rating of Six Hundred Specimens. Scale of Rating. 90%. 70%. 50%. 30%. 10%. Number of papers Per cent of all 51 8.5% 240 40.0% 253 42.2% 56 9.3% 0% The above table shows that of the six hundred papers 51, or 8.5 per cent, were rated 90 per cent, 240, or 40 per cent, were rated 70 per cent, 253, or 42.2 per cent, were rated 50 per cent, and 56, or 9.3 per cent, were rated 30 per cent. After each paper had been thus finally evalu- ated, and in order to illustrate the character of the hand- writing which the committee considered typical of the various grades in the scale, it then became necessary to select some specimens from each of these four groups of papers which should be typical of the group. This was done in much the same manner as the specimens were originally rated. For example: Each member took the RESULTS OF STUDY IN PENMANSHIP. 15 51 specimens rated 90 per cent and selected therefrom several specimens which, in his or her judgment, most nearly typified the quality of handwriting of all the papers of that group. Each group of papers was handled in the same manner, after which each committee member submitted a memorandum indicating his or her selection of the typical papers from each group. Such papers were then reviewed in a general committee conference and agreement reached concerning those samples which best typified the group from which they were taken. Two papers were finally selected from each group, except the 30 per cent group, from which three papers were selected. These three papers w^ere selected not because that group was proportionately larger than the others, but in order to illustrate three quite distinct kinds of poor handwriting to be found in that group. The Specimens or Samples Selected. The following pages contain facsimile reproductions of the specimens of handwriting which the committee selected as typical of the four groups of papers rated 90, 70, 50, and 30 per cent, respectively. At the head of each page is indicated the rating of the paper and also the proportion of the six hundred papers which were given this rating. At the head of each specimen is given the original number of the specimen. These specimens form the basis of Miss O'Dowd's discussion of the quality of the handwriting which they typify. In Miss O'Dowd's report these specimens are referred to by the number. 16 SCHOOL DOCUMENT NO. 6. SAMPLE OF HANDWRITING RATED 90 PER CENT. 8.5 Per Cent of the 600 Specimens Given This Rating. Original Specimen No. 47. -^^^<^-^c<.^^^ZeIi' C^ -^C^^ ^:2^^^^?^-t^ c:^^ \>/ -5^C^.^--i^y-22^^ dy^-C^.,^ <^^y^ RESULTS OF STUDY IN PENMANSHIP. 17 SAMPLE OF HANDWRITING RATED 90 PER CENT. 8.5 Per Cent of the 600 Specimens Given This Rating. Original Specimen No. 105. ay ^-z--^^ '— r^^V^ 18 SCHOOL DOCUMENT NO. 6. SAMPLE OF HANDWRITING RATED 70 PER CENT. 40 Per Cent of the 600 Specimens Given This Rating. Original Specimen No. 254. & 6 yui^.^^^cZi^-^^oc^^'-'^^ a^