■X I B R.AHY OF THE U N IVLRSITY OF ILLINOIS 510.84 U6r no. 257-264 cop. 2 CENTRAL CIRCULATION AND BOOKSTACKS The person borrowing this material is re- sponsible for its renewal or return before the Latest Date stamped below. You may be charged a minimum fee of $75.00 for each non-returned or lost item. Theft, mutilation, or defacement of library materials can be causei for student disciplinary action. All materials owned by the University of Illinois Library are the property of the State of Illinois and are protected by Article 16B of Illinois Criminal law and Procedure. TO RENEW, CALL (217) 333-8400. University of Illinois Library at Urbana-Champaign JUN 2 8 1999 When renewing by phone, write new due date W .-.1 .-, i«t r.i-ai'iAii r< /Ilia /Infa II rSV below previous due date L162 Digitized by the Internet Archive in 2013 http://archive.org/details/teachingmachines260nord r/o. *r u Report No 260 ~yvitiJt£ te>f 2. TEACHING MACHINES AND PROGRAMMED INSTRUCTION: An Introduction and Review* by B, J. Nordmann, Jr. April 22, 1968 Report No, 260 TEACHING MACHINES AND PROGRAMMED INSTRUCTION: An Introduction and Review* by Bo J . Nordmann , Jr . April' 2?, 1968 Department of Computer Science University of Illinois Urbana, Illinois 618OI "Supported in part by Contract AT(.ll-l)-10l8 with the U,S, Atomic Ener^ Commission and the Advanced Research Projects Agency. I , HISTORICAL INTRODUCTION In the last two decades with the school-age population expansi sorely testing the American egalitarian ideal of universal education, increasing attention has "been devoted to the techniques of instruction. One technology resurrected as a result of this al ' < n Lon has been the idea of the teaching machine. This device, it was hoped, would relieve th< classroom teacher of the teaching drudgery and allow the teacher to spend more time helping pupils on an Individual basis. The first serious attempt to use a teaching machine was made by S.L. Pressey, During the 1920' s Pressey published numerous papers about a machine he had developed. This machine was about the size of a typewriter, was capable of asking multiple choice questions and could grade the answers. [60, 6l, 62] . Prior to this time many other teaching devices had been patented (over 600 between 1809 and 1936) but no investigator had tried to interest the public in these devices. Although Pressey spent much of the next 30 years trying to gain public acceptance of his ideas, it remained for B, F. Skinner to actually get educators stirred up over the teaching machine concept . B. F. Skinner, who developed the "Skinner Box" and used it in his investigation of animal learning, felt that the same techniques which he employed in the laboratory on animals could also be used in the schools on people [79] • Using these techniques he developed a teaching machine which would display the material to be learned in small segments called "frames". These frames were ordered in a logical sequence and each contained some statement with blanks or a question which required the student to construct a written response. As soon as the student finished writing the answer to the frame he was given the correct answer for comparison purposes, The frames were written so that using knowledge from previous frames and cues from the present one, the student was almost certain to get the right answer. -1- This correct response was intended to reinforce the learning of that particular frame An alternative to Skinner's method came from N, A. Crowder who felt tha+ Skinner's method was too inflexible in its handling of the student .[19] . He de^elcped some programs which branched depending on the answer the student made to the last question, A wrong answer to a question caused a branch to a special sequence of frames which dealt with the specific mistake made by the student. A correct answer, on the other hand, led to the next instructional item. This method of programming is known as branching or intrinsic programming as opposed to Skinner's linear programming in which there is only one path through the program- Crowder hoped that by using branching, the advanced students who learned more quickly would be able to skip the more rudimentary material incorporated for the less well-prepared or slower-learning students. The branching program represenx-s a completely different concept of the teaching (learning) process from Skinner's linear program: Skinner viewed the teaching machine as a means of conditioning a student to emit the appropriate response (answer) when faced with a given stimulus (question). Crowder, on the other hand, felt that the educational process was basically a feedback situation in which a teacher presents new material to the student and then determines if he has learned it, If not the teacher reviews the material or presents a corrective sequence before going on. A third programming method, called adaptive programming , usually involves the use of a computer or other complex electronic device, In this method the machine takes into account various individual characteristics such as pers r y (aggressiveness, shyness, etc. )> IQ and also student behavior on previous questions to decide the type of" sequences to present to the student. Gordan Pask of Great Britain helped design one such machine, SAKI. Another, SOCRATES, has been programmed on a computer at the Illinois Training Research Laboratory [36]. -2- Adaptive programming is sometimes also i aJ Led extrinsic since uses information external to the program, (IQ, personality scores, apti- tudes, etc.) in making its sequencing decisions; this is in opposition ' branching or intrinsic programming, which only uses the response to the last question [20, 25]. When Skinner began advocating his machine the economic situation and the teacher shortage caused immediate interest to develop. Cons ible >arch was started to determine the best types of teaching programs and to compare the amount of learning produced by this technique with that found in the ordinary classroom situation. At the same time the educational press became interested and many articles were written both pro and con [ 1 k , 32, 33, h6, 55, 56, 92, 93], Partly as a result of this publicity there was a spurt in the actual production of machines despite the fact that the theory of these had not been completely worked out , W. L. Ross et-al. 1 69] list some UO teaching machines in their buyer's guide published in 1962- Finn and Perrin [31] , also in 1962, list almost 100. These machines ranged in price from $1 to $1600 with one machine priced at $5000. The machines consisted of everything from manual crank devices which contained the program on a strip of paper, to complex devices which showed slides and played recorded instruction via tape recorders. It scon became apparent, however, that without good programs these machines were not appropriate for large-scale school use. Although many programs were produced, the overall quality was not always good and the measurement of program quality became an important, topic [U, 15, ^0, kQ , 58, 183]. This, coupled with the high cost of the machines in many instances and the fact that changes in programming theory or technique quickly made machines obsolete, took the bloom off the rosy expectations that the manufactures had had for the middle and late 1960's, Although school systems have indeed become interested in programmed learning, their plans for large scale integration of teaching machines still appear to be a few years in the future, [28], -3- During the late 1950 's the computer industry "became interested in programmed instruction. A variety of "computer aided instruction" (CAI) projects were begun to utilize the high-speed computer as an instruction control device As early as 1958 the IBM Research Laboratories began a project tc teach binary arithmetic using an IBM 650 computer and typewriter console [65, 17*0 • The computer was programmed to give problems to the student and then grade his answer, On the basis of the correctness of the answer the somputer would then generate another problem. Later the IBM group built an arithmetic tutor for 6th grade students. This project in turn led to further experiments in teaching a broader range of courses such as steno- typmg, statistics and German reading [9, 17^, 196]- Eventually in 19&5 IBM announced the development of a compiler-like language, called COURSEWRITER [36] , which allowed a teacher to program a computer to teach his own subject. In early i960 the Systems Development Corporation began work in automated teaching using a Bendex G-15 computer [il6, 117, 119» 123] > They experimented with various parameters and this eventually led to their develop- ment of CLASS (Computer-Based Laboratory for Automated School Systems), an educational facility designed to explore the potential of a fully integrated classroonic This scheme combined automated instruction with other instruc- tional methods such as films, TV, lectures and textbooks [9, 117 » 123]. CLASS is designed to accommodate as many as 20 students in a variety of modes, Its main purpose is to test the new techniques in an integrated manner similar to the way they might eventually be used in a regular classroom- Also in i960 the PLATO project (Programmed Logic for Automatic Teaching Organization) at the University of Illinois was begun,, [98 s 99, 100, 10 1 , 103] ■ Under the direction of D. L. Bitzer and P„ G. Braunfeid, its original purpose was to investigate student behavior when in contact with a teaching machine. It began with two terminals attached to the -k- Illiac I computer, Today after extensive modifications the system uses a CDC l60U and controls 20 consoles Each console has a TV device which can display slides or computer generated output - all under control of the computer The student is equipped with a television monitor, a typewriter key set and certain other control "buttons with which he can "talk" to the computer. A wide variety of courses have been programmed and taught to students . A good example of a computer oriented adaptive system is the SOCRATES II system (System for Organizing Content to Review and Teach Educational Subjects) [23, 36, 193]. Built by the Illinois Training Research Laboratory, the system controlled several teaching machine stations. The system is initially provided data on each student's IQ, aptitude scores, personality traits and other relevant information- Using this it determines the strategy of instruction to give to each student There are various other computer systems including COBIS (Computer Based Instructional System) at the Decision Sciences Laboratory at Hanscom Field, the TRW MENTOR which uses films and audio magnetic tape recorders [9> 13], and the Stanford I, built at Stanford University to control audio and video stimuli by computer [200]. One other example of an automated teaching system, which shows another direction of research, is an earlier project at the Decision Sciences Laboratory [159] . This project developed at about the same time as the beginning of the IBM project but it was oriented on a more pragmatic basis . In this case the basic, idea was to use an existing large-scale computer system to help teach its potential users how to employ it. A combination of programmed text and computer teaching was used without human intervention. In the next section we will go back to the early cont oversies between linear and branching programming advocates. Although this contro- versy is currently dying out because of the willingness of present day programmers to use whatever is best for a given application [5^ 5 56] , it is still important historically since it led to a great deal of research effort on the part of the various advocates. Although much of this research gave conflicting results it represents the inaugeration of a large-scale effort to discover the effective variables in programmed learning, -5- II. PAST CONTROVERSIES Previous to the development of programmed instruction research, the field of psychology had done a significant amount of work on learning, memory, and other related areas. However no unified theory was generally available. The result of this lack was that a considerable argument arose over the desirable characteristics which a teaching machine program should have . As a result of their research, Skinner and his supporters advocated a very specific instructional format in which small frames requiring an overt response by the student were used. Cues to the correct answer were gradually withdrawn in successive frames until the stu- dent was on his own, A key point in writing the program was to keep the error rate low, If boo many errors were made the reinformcement effect due to the constructive response and the display of the correct answer would weaken and the wrong answers might be remembered . For this reason, Skinner advocated program testing and revision until the average error rate was around 2-5%. Crowder, on the other hand, used questions to determine how much a student had learned, as well as for giving him practice and reinforcement, Errors in questions would result in special remedial sequences to help the student understand why the error he had made was wrong, Correct answers would sometimes allow students to skip instruc- tion which they already knew. Because of these features it was necessary to have a higher error rate, As a result, Crowder typically considered a 15% average error rate to be desirable. Skinner's adherents, however, felt that if the student made too many errors, the wrong answers might be reinforced There has been an interesting rebuttal to this objection. It has been argued that by getting a question wrong and then having to go over the basic principles as a result, the student will end up with a -6- clearer understanding bhe whol« idea [IT, ;,; |. The supporte] this idea claim that LJ a s1 udi I t responds coi si I Lmi , h will not be "set" to the question at all and will completely dismi from his mind. This seems similar to the idea of Zeigarnik [202], who noted a general tendency toward better recall of an unfinished task than a finished one, Another dispute over intrinsic programming devel' p d o*rcr the fact that in order to be able to branch easily, Crowdez used multiple choice questions [198] • This resulted in a problem of transference of learning. It was argued that in the real world, the student will n normally be presented with a limited number of solutions to a given problem. Therefore multiple choice questions may not really prepare him for reality. Another problem is that the student may simply guess at the answer , What Skinner wanted to reinforce is the process of constructing the answer. By guessing, Skinner claimed that the student eliminated part of the learning process, He also felt that merely seeing the wrong answers with the correct one would tend to weaken the learning of the correct response . It should be noted that these objections to branching methods can be partially relieved in some computer oriented machines by the use of complex "judging" routines. In this case a multiple choice question is not necessary, The judging routine determines the correctness of a constructed answer and determines what corrective sequence, if any, should be entered. The PLATO system, as an example, can be programmed to classify errors and transfer to one of several sequences depending on error type. It can also detect misspellings of the correct answer and instead of a "wrong" reply it will return a "misspelled" reply Another area of disagreement was the size of the frame [155, 185]. Skinner maintained that the frame should be small since a response must be elicited from the student after each item to be learned. Since Crowder was trying to present more complete items, his frames tend to be longer. Other observers felt, that short frames were too restrictive for -7- certain types of subject matter regardless of whether or not branching was used. Smaller frames also sometimes tended to make the student take more time to cover a given set of principles '[115] . Another problem was whether or not the student really needed to make an overt response in each frame '[190]. It was felt by certain groups that as long as the student made some sort of covert response, learning would be able to take place. It would also be much faster since there would be no need to write the answer down. Of course occasional overt responses of some sort would probably be necessary if only to make sure that the student was not asleep. In concluding this section, it should be noted that basically the linear programming advocates used an operant conditioning model of learning while the Crowder supporters used a more empirical one. They tried to give the student the information to be learned in the most efficient method possi- ble and left it up to him to learn it. There are several philosophical impli- cations in these different techniques (see [993 fo r a discussion of linear programming and determinism) . The main practical and theoretical differences can be found in [l8, 20, 21, 25, 80, 88]. -8- III. EXPERIMENTAL WORK The result of these various theoretical controversies was a consider- able increase in the amount of research on teaching machines and their effec- tiveness. Various means "were used to display teaching machine programs. Due to cost considerations, one of the most popular was a programmed text or looseleaf notebook . These could be printed up or mineographed in quantities suitable for experimentation at quite a reasonable price [kk] „ They considted basically of sheets of paper with the frames printed on them,, In a linear- program the next frame and the answer to the present one would be on the next page. In a branching program, each multiple choice answer would direct the student to the page number of the frame which he should go to next if he felt that particular answer was right. In the branching program the frames were scrambled in a random order which caused this type of programmed text to be called a "scrambled took". Several publishers have begun printing texts of these types [^1, k-2, 52] and they are presently one of the most popular forms. One problem which may develop with scrambled texts particularly and programmed texts in general is that they are bulky and can become expensive when purchased in large amounts due tc the fact that many books would be needed to cover, say, a year's work in high school. Since usually the student must write his answers in the book, a. new text must be purchased for each stu- dent. Another type of machine which is often used in experiments is the "crank" type in which the program is on a continuous roll of paper which the student cranks or pulls out of the machine as he goes along. These are best suited for linear programs . A third machine which can be used is a computer generated display. Due to its flexibility, an experimenter can make quick and easy changes to the teaching program as soon and as often as he wants. The uses of computers in programmed instruction will be discussed more fully in Section U. .9- In general there have been four basic experiment types in the field of programmed instruction (Pi): 1) Comparison studies between programmed instruction and conven- tional teaching methods . 2) Parametric studies comparing different programming methods and techniques, 3) Exploratory studies investigating new applications of programmed instruction. 4) Implementation studies in which PI is incorporated in an actual classroom on a. continued basis. The results of these different types of experiments seem to vary ac- cording to type. Experiments of the first type were generally successful in that al- most all showed an increase in learning and/or a decrease in time needed to study when programmed instruction was compared to conventional methods [25, 107, 113, 128, 129, 130, 13*1-, 141, 142, 158, 167, 1.71, 173, 176] . There were several exceptions to this [66, 10, 126, 135, 1**3, 1*1-7, 191] tut such results were often due to poor experimental conditions . In one study there was very Little control over the subject's behavior away from class and all subjects went to the same lectures { 1*1-3] ° Tne only difference was that certain groups were given extra time with programmed materials. Under the circumstances, it is not surprising that no differences were measured in the various groups. In another series of experiments conducted in Great Britain, an American program. (Temac-First Year Algebra) was used by Scottish children who did not understand some of the terms used. In still another experiment two groups were used which had significantly different mean IQ scores [66] , An- other reason for poor results was that often the PI sequences were much too short for reliable results [13*1] . -10- In general, though, programmed instruction proved itself to be at least equal and often superior to more conventional learning methods. There has been one argument against these comparison experiments, however, which has so far been difficult to refute: the suggestion that the superior performance of the students using programmed materials may be due to novelty or to the ,: Hawthorne Effect" . These critics argue that once the novelty wears off and students using programmed learning techniques are no longer considered "spe- cial", the advantage of programmed instruction may lessen or disappear com- pletely. Although some experimenters have tried to lessen these effects [i^l, 1^2] or to actually measure them [ 14-6, 170], only time and further experimen- tation with large numbers of subjects over long periods of time will be able to prove or disprove this objection. Due to the generally satisfactory results of the first type of ex- periment, research in this particular area has declined more recently. The second type of experiment is a different story, however. The purpose of these experiments was to determine those programming factors which led to the most efficient learning for the student. Unfortunately there are so many possible variables, and their interactions are so complex, that it has been very diffi- cult to obtain coherent results. As a result these experiments seem, to have generated more confusion than ever. Often when comparing various techniques the experiments were not able to show any significant differences [102, 111, 124, 125, 137, 143, 153, 16'2, 179, I87, 192], or when they did, the results were often in contradiction with some earlier experiment [ll8, I87] . Eventually however it was realised that there were several de- fects in the research, which had been reported. Since nobody was quite sure which of the many variables in a teaching machine program were significant and since it was difficult to obtain a, sufficient population of suitable subjects in order to test all variables, mistakes were often made in the choice of per- tinent variables. ■11 • Some investigators, in an effort to control as much of the experiment as possible and to test only a small number of variables at a time, tried to write one program and then modify it slightly in order to compare different programming variables [e.g., 12 k, lUO] . This is a reasonably sound procedure, providing that the variables being measured and the variables held fixed have independent effects on learning. Unfortunately this is generally not the case in programmed instruction and the net effect of this type of experiment is to show little or no effect on learning. What is more likely important is the variation of combinations of variables so that if two programming techniques 3,re to be compared, the amount of learning which is produced by each should first be optimized with respect to all of the other variables. Up to now, however, there has not been enough time, money, or subjects to do anything like this . Another problem with programmed instruction research was the quality of the programs used. It often turned out that the programs which were used in the experimental research were not written in such a manner as to enable a distinction to be made between two techniques. For example, in one test be- tween linear and branching programs the lack of a difference in the amount of learning was explained (by the experimenter) as being due to the unappropri- ateness of the remedial branching sequences [187] ° Since the remedial se- quences vere not very good, the students with branching learned no more than those without it. In support of this observation, J. L. Evans, et al° [12k] observed that the relavence of any particular program variable in a frame was inversely related to the probability of a correct response by the subject on that frame J. G- Holland has developed a measure for the criticality of the stimulus ma- terial in a frame [3, ^3] ■ This measure is called the black-out ratio and represents the maximum ratio of the amount of material in a frame which can be removed without affecting the student error rate, to the total amount of -12 material in the frame . He discovered that programs which had high black-out ratios, i.e., those which contained much non-critical material, when used in riments to compare overt and covert responding, constructed response vs. multi-choice response, etc., showed no significant differences between the variables being testedo Conversely programs with low black-out ratios were able to show significant differences. Sometimes the results of a programmed learning experiment were af- fected by a lack of familiarity on the part of the subjects. In one case Bivens and Cambell [102, 110] noticed that in a comparison between self -di- rected and non-self -directed learning, students who have become accustomed to the lecture method in school do not really know how to direct their own studying. However after being given a coaching session in self -study methods they were superior to a group using linear programmed instruction. J. E. Coulson, et al. [119] have noticed this same factor in that when they gave students the option of controlling their own branching in a teaching program, they rarely took advantage of it. In addition to this there were many other more standard types of problems. It was often difficult to decide how to, measure the increase in learning [68]. Poor pre- and post-testing of the subjects would often lead to fairly useless results [l6U] . In some cases, where the experimenter,; were using students in college classes there was often very poor control over the kinds of learning the students acquired when away from the teaching programs [96, l'-K3] • Then too, the students motivation often varied from group to group L95J • To conclude this discussion of the type two experiments, there is now at least ' a body of knowledge which can be used to devise new experiments, although there is still much which is unknown about the factors which cause a program to teach efficiently. In the future more experiments will be needed which try to impose some order upon this body of knowledge. One example of -13- this type of experiment is Leigh's results [15,1] wnich were able to experimental- ly show "why experiments by other researchers [104, 124, 133, 1^2, 148, 149, 198, 201] had been so confusing. 3y discovering that the effect of overt as opposed to covert responses varied with the type of material being taught, Leigh was able to explain the often-conflicting results of previous experiments „ There are various examples of the third type of experiment [108, 112, 138, 139. 157, 160, 180, 184, 185, 189] • This type consists of various explor- atory experiments which are aimed at finding qualitative results from various extensions of programmed instruction. Most of the computer experiments which will be described in Section IV are of type three. Other examples of this type of experiment include one which tested the effect of problem setting questions in a "conventional" linear program [137] > investigations into self -directed programmed instruction [102, 110], and various investigations using adaptive programming [154, l6l, 188, 193] ■ The type four experiments, consist of implementations of what is al- ready known. Their orientation is more practical than theoretical, for by the time a school system makes up its mind to use programmed learning techniques, it is no longer interested in detailed scores, learning curves, etc., but in- stead wants to know the effects on teachers' loads, how big the classrooms should be, the grading system to be used, and other mundane affairs which need to be known to make a practical system. In general there have not been too many large scale implementations of programmed instruction. A primary reason for this is the lack of a coordi- nated series of programs spanning several grade levels. Those implementations that do exist, however, can be found in industry [lOf, l45, 163, 168, l69]j> the government [121, 166], and the educational system [9h, ll4, 120, 122, 136, 182, 194] . In general these implementations have shown the following" •14- 1) Programmed instruction techniques are affective teaching methods. 2) Teachers must be instructed in the use of PI before a PI program is begun at school. 3) Although often adaptable to individual student difference s } teaching machine programs can be very inflexible when responding to curriculum changes, h) At the present time there is a chronic lack of good programs, especially series of programs covering one subject through several grade levels . In the following section the application of computers to programmed instruction will be described. -15- IV. THE APPLICATION OF COMPUTERS TO PROGRAMMED INSTRUCTION When Skinner produced his first teaching machine, one of its impor- tant characteristics was its simplicity and low cost. However, this simplicity was also one of its major defects since the type of interaction between the student and the machine was very limited. When Crowder introduced branching programs, he was able to deal with students in a more flexible way with a slight increase in complexity. With the advent of computer usage for automated teaching systems, an even greater amount of flexibility became possible [1, 67] but at a very high cost. As a result very few people could afford to investi- gate the computer applications of teaching machines unless they already had a computer available for other applications. This has meant that most of the work in computer-assisted instruction ( CAI) has been done by computer-oriented people and, as a result, the research in CAI has maintained a completely dif- ferent flavor from other PI research. The CAI work has generally been carried out on a very pragmatic basis. This may be due to the fact that most of the people in this area were originally educated as engineers and not psychologists At any rate these practitioners tend to think (and publish) in terms of appli- cations ard not in theory. The first CAI experiments were, done on free-standing machines where only one student could use the machine at a time. This was fantastically ex- pensive, however. Licklider, of Bolt, Beranek and Newman, estimates a cost of $3?/^ r - or one student using a PDP-1 computer [156] ° When time sharing began to develop, multiple-console systems were quickly developed so that many stu- dents could use the same computer at the same time. Although time-sharing allows the costs to be spread over many stu- dents, it aggravates another problem. In this sort of system each person must be equipped with their own input-output console. An electric typewriter is usually a. good input device. Maximumly effective learning however, requires that there must not be a long delay when material is being presented to the student. This means that the output must be displayed by a fairly high speed -16- device such as a slide projector, TV screen, or similar instrument, Unfortun- ately a slide projector limits versatility since it must be pre-loaded with slides and cannot be used to display computer -generated information- A TV dis- play costs more money than can really be afforded, especially after the neces- sary memory circuitry is added. For the present time it appears that the average cost of such a TV- typewriter console is on the order of $10,000. D« L. Bitzer [5] has estimated that an economical automated instruction system (on the order of $.25/hr/ stu- dent) could be built if the cost of the consoles can be cut to $1200. This in- volves timesharing 4000 students on a CDC 6600 computer, the largest and fastest computer available today. A recent breakthrough in panel display devices [6] seems to indicate, according to Bitzer, that a reliable console with the needed resolution and with an inherent memory capability, can indeed be built for this price within 3 years. With the advent of more sophisticated computer programming the amount and type of control which the student may have over the instruction sequence has engendered extensive discussion and experimentation [29, 102] . In a basic Skinner or Crowder program the student has little conscious control over the process, as he is only answering questions which are put before him by the teacher -programmer . A more recent technique using computers involves a "so- cratic" form of teaching. Swets and Feurzeig [85; 127 ] describe a program which teaches medical diagnosis by setting up a basic case history and then requires the student to request the proper information, to make the diagnosis. The student may ask for laboratory data or initiate a more thorough "physical examination" . If he asks for irrelevant information or makes a stupid diag- nosis he is properly corrected. The complete interaction takes place in a conversational mode. Another example of this type of teaching is the PLATO inquiry logic [99> 100] . In this mode of operation the computer shows the student an exper- iment on the TV screen. The student must then answer a series of questions. -17- In answering them, the student can ask various questions of the computer and request the results of an experiment if some initial condition had been dif- ferent- In this case the experiment is rerun using the new conditions. In some experiments the student has been given sufficient control over the learning process that the computer becomes merely a tool and is no longer a teacher. This was done in one Boston school in which students were given access to a time -shared computer with a very simple programming language and were allowed to use it to solve any type of math problems they wanted [175] In a student- controlled environment other than this extreme case,, however, there are some severe problems concerned with understanding the stu- dent input information. The computer must be able to interact in a conversa- tional mode with the student and to some extent at least be able to "under- stand" what he is saying. This area has been increasingly developed lately, not only from the standpoint of teaching machine programs but also with man- computer interactions in general [_'( , 8, 38, 6 ! 4, 79> 84] . As of yet no com- pletely general solutions to this problem have been found. There are other teaching situations however, which can be considered to be student controlled which are not terribly difficult to achieve right One is the educational game, Leonard and Wing describe a computer pro- gram which is used to teach economics to children [152, 200] . The student is put in a hypothetical situation such as being king of a mesopatanim kingdom. As such, he must make economic decisions such as how much grain to plant and ho- much to feed to his subjects. The results of his decisions are calculated and given back to him (i.e., half your people have starved to death). At first the decisions and their results are fairly obvious in order to teach the ident the economic principle involved. As time goes by the problems become more and more complicated until the student is making a variety of complex de- Lons and in the process learning principles of economics. -18- V. FUTURE PROSPECTS At the present time the future of programmed instruction appears fairly hopeful . There are many advantages in using PI, among them: 1) Providing immediate feedback to the student. 2) Allowing the teacher to determine exactly what part of the in- struction sequence needs improving when the students do not learn a specific item. 3) Providing more and better education for both advanced and re- tarded students by allowing students to go at their own pace with program sequences adjusted to their own level = h) Reduction of the drudgery work for teachers. In the next few years research will continue to investigate the basic programming principles and search for less expensive hardware. In addi- tion to this, however, a great deal of work will need to be done in the school system in order to prepare for the large-scale changes of the next few years. Not all of these changes, of course, will be due to programmed instruction but PI will probably be involved in some major changes to come. For one thing, although programmed lee.rr.ing will definitely not eliminate the teacher from the classroom, it will significantly upgrade the basic teaching task. L. F. Carter [11] mentions seven basic goals of the American school system, including in them teaching socialization, developing creativity, and encouraging physical development in addition to teaching basic "fact" subjects. If teachers are able to use machine programs to teach the more routine fact learning in the school curriculum, the teacher will become more free to develop the students' thinking processes, discuss the relation- ships between groups of facts, and stimulate creativity. .19_ Another effect of extensive use of programmed instruction techniques beaching will be to put an even greater strain on our school systems' at- tempts to reta.in "che chronological grade format presently being used. Since students will tend to move at their own pace, it is obvious that grouping stu- dents into classes on the basis of chronological age will become very diffi- cult. There have indeed been experiments in trying to accomplish group pacing of students using programmed instruction [I31> 165], but these seem to work best when the groups were selected on the basis of IQ, or achievement test scores which means that the chronological age was not important . It appears therefore that the socialogical implications of grouping students according to mental age and interests, instead of chronological age, will have to be faced. Systems have been suggested [39] i n which students spend part of their time on "directed" work and part of their time exploring subjects of their own choosing at whatever level of difficulty they can master. This in- dividualized instruction could be done with programmed instruction, special lectures or discussion groups or a combination of these. Note that with indi- vidual programmed instruction the level of knowledge of a particular group of students can be brought to a specified level before a specific lecture or dis- cussion so that the students will all be able to get maximum benefit from the activity. Another administrative problem which will probably come under review is the problem of grading. With the increase in individualization which a large-scale integrated use of PI will bring, the problem of measuring relative achievement will increase. Some advocates of PI have gone so far as to say that the extensive use of PI will eliminate the need for grades since a good teaching machine program will teach the required subject matter to everyone who can complete the program. Tne fact remains however that different stu- bs can and do learn different amounts from the same program, so some sort of grading scheme may still be desirable. -20- For the immediate future, Programmed Instruction presents a differ- ent set of problems. In the research area more work must be done in ascer- taining the exact characteristics which optimize learning in a teaching pro- gram. This will take a period of extensive testing. There is also a need for a low-cost flexible machine which can be easily updated as new research devel- ops new techniques. Finally a large number of programs must be written and pre-tested before being used in the school system. This is especially true since the majority of programs and machines which were produced in the early 60's were not very useful for practical, large-scale teaching. The main immediate goal for the teachers and administrators in the school system will probably be twofold. First many more teachers and parents will be taught the basic ideas and principles of programmed instruction. Secondly, teachers themselves will need to enter the research activities in programmed instruction and determine more precisely which areas of the curric- ulum will benefit most from this new technique. As this is done implementing programmed instruction into the school system will become practical and bene- ficial. -21- BIBLIOGRAPHY General Works 1. Adams, E.N., "Computer-Assisted Instruction", Computers and Automation , Vol. 15, No. 3, March 1966. Describes the use of computers to provide "individualization of instruction" as it cannot be given in today's classrooms; Discusses IBM research. 2. Austwick, K. (ed. ), Teach ing Machines and Programming , The MacMillan Co., New York, 196U - A Collection of articles by British workers in the PI field. Contains a very good introduction to teaching machines detailing the early history and present points of contention. 3. Balabanian, N. , "The Educational Engineering Called Programmed Instruction IEEE Transactions on Education , Vol. E-9 No. 2, June 1966, pps . U9-5T. General summary of PI. Comments on effectiveness of some of the early research. Illustrates characteristics of PI through various examples . h. Basecu, B. , "On Frames and on Editing and Revising Programmed Instruction", from Filip ( 30 ) , 1963, pps., 116-123. 5. Bitzer, D.L., talk given to U. of 111. Student Chapter of the ACM, October 5, 1967, Urbana, 111. 6. Bitzer, D.L. and Slcttow, H.G. , "The Plasma Display Panel - A Digitally Addressable Display with Inherent Memory", Conf. Proc. - Fall Joint Comp. Ccnf . , 1966 , Spartan Books, Washington, D.C., 1966, pps, 5^1-5^7 • J. Bobrow, D.G., "A Question Answering System for High School Algebra Word Problems", Conf, Proc. Fall Joint Comp,, Conf,-196k , Vol. 26, pps t 591-6l^. 8. Bobrow, D.G., "Problems in Natural Language Communication with Computers", IEEE Trans, on H.F. in Elec , Vol . HFE-8 , No . 1 , March 196? , pps. 52-55. 9. Bushnell, D.D. , "Computer-Based Teaching Machines", J. of Ed, Res . ? Vol. 55, No. 9, June-July 1962, pps., 528-531, Review of several papers presented at a 1961 conference sponsored by the Off- of Nav. Res., and Sys. Dev. Corp. Papers discuss PLATO, MENTOR, CLASS, and systems by IBM, Carnegie Tech, and Bolt, Beranec and Newman. 10. Carr, W.J. , "A Review of the Literature on Certain Aspects of Automated Instruction" from Smith (8l), 1962, pps. 57-80. Discusses in detail the background history, learning theory and characteristics of automated instruction. Reviews some of the pre-1960 work and presents a list of hypotheses to he further tested. 11. Carter, L.F., "The Challenge of Automation in Education", from Coulson (16), 1962, pps. 3-12. Describes how automation can help achieve some of the goals of our educational system. 12. Center for Programmed Instruction Inc., Programs '62; A Guide to Programmed Instruction Materials Available to Educators by Sept . 1962 , United States Printing Office, Washington D.C. , 1962 . An annotated listing of 122 programs. Includes program samples, cost, etc. Also gives statistics for group as a whole . 13. Chapman, R.L. and Carpenter, J.T., "Computer Techniques in Instruction" from Coulson (l6), 1962, pps „ 240-253. Describes the TRW MENTOR, an automatic tutoring device utilizing film and magnetic tape. Ik. Congreve , W.J., "Lets Not Say No to the Teaching Machine", Chicago Schools Journal , Vol, 42, No. 2, Nov. i960, pps. 70-76, 15. Cook, D. "Studying the Performance of a Program" from de Grazia and Sohn (24), 1964, pps, 126-137. Demonstrates the types of information which teachers should have in evaluating a program by describing the testing procedures and results for a chemistry program. 16. Coulson, J.E. , Programmed Learning and Computer Based Instruction John Wiley & Sons, New York, 1962, A Collection of articles oriented toward computer PI. 17. Cram, David, Explaining Teaching Machines and Programs , Fearon Publishers Inc. g San Francisco, Calif., 196l. 18. Crowder, N.A. "Automatic Tutoring by Means of Intrinsic Programming", from Gal ant er (35), 1959 , pps. 109-116. 19- Crowder, N.A. , "Intrinsic Programming: Facts, Fallicies, and Future", from Filep (30), 1963, pps. 84-115. Describes underlying rationale of intrinsic programming. Emphasizes flexibility and lack of commitment to any specific learning theory. 20. Crowder, N.A. "Intrinsic and Extrinsic Programming", from Coulson (l6), 1962, pps, 58-66. 21. Crowder, N.A. , "On the Differences Between Linear & Intrinsic Programming", from de Grazia and Sohn (2U), 196U, pps . 77-85 . Describes the differences in history, development and technique between linear and intrinsic programming. 22. Davey, D.M. & McDonnell, P., Programmed Instruction , Institute of Personnel Management, London, 196U. Brief review of PI. Describes linear & branching programs , advantages of PI, and how to introduce PI into a company training program. Mentions several projects also discussed in Hughes (H5 ) . 23. Davis, D.J., and Stolurow, L.M., "Computer Based Systems - The New Research Aid", Training Research Lab,, U of 111., Tech, Rept . No. 6, November I96U. Describes the effect of PI on attention to individual differences in education. Discusses SOCRATES Project. 2U. de Grazia, A., and Sohn, D.A. , Programs, Teachers, and Machines , Bantum Bocks, New York, 196I+. Collection of reprinted articles on PI including articles by almost every major researcher in the field. Fairly comprehensive . 25. Deterline, W,A. , An Introduction to Programmed Instruction , Prentice Hall, Edgewood Cliffs, N.J. , 1962. Contains an appendix on testing half written in "scrambled book" form and half in linear form. 26. Epstein , S. , The First Book of Teaching Machines , Franklin Watts Inc., New York, 1961. 27. Evans, J. L. , Glaser, R. , and Homme, L.E. , "The Ruleg System for the Construction of Programmed Verbal Learning Sequences", from Glaser (132), Oct. 1961, pps. 17-30. Divides verbal subject matter into two classes, ru's (rules) and eg's (examples) and establishes a procedure for writing linear programs using this class division, 28. Everett, J.B., The Use of Programmed Instruction in U.S, Schools , The Center for Programmed Instruction, Inc., New York, 1962. Reports on the results of a survey made of 3000 school system administrators about the use of PI in 1961-62. Gives statistics on program types, no, of students, etc. 29. Feurzeig, W. , "New Instructional Potentials of Information Technology", IEEE Trans . on H.F. in Elec. Vol. HFE-8, No. 2, 1967 , pps . 8U-88. Discusses computer-dominated vs. student-dominated instruction, 30. Filep, R.T. , Prosnectives in Programming . The MacMillan Co., New York, 1963. 31. Finn, J.D. and Perrin, D.C., Teaching Machines and Programmed Learning, Office of Education, Dept. of HEW, Washington, B.C. 1962. 32. Fine, B. , Teaching Machines , Sterling Publishing Co., New York, 1962. Very enthusiastic about the future of teaching machines. Many examples of student opinions of TM's. Written at a rather low level. 33. Fitzgerald, H.T., "Teaching Machines: A Demurrer", The School Review , Vol. TO, No. 3, Autumn 1962, pps. 2U7-256 , 3^» Fry, E.B., Teaching Machines and Programmed Instruction , McGraw Hill, New York, 1963. Discusses history, how to write programs, individual psychological differences in students and techniques for evaluating programs. 35. Galanter, E. , Automatic Teaching: The State of the Art , John Wiley and Sons, New York, 1959. 36. Garner, W.L., Programmed Instruction , The Center for Applied Research in Education, Inc. New York, 1966 , A good summary of the PI field providing the reader already has some familiarity with it. Discusses programmer training, Stolurow's synthesis of programming elements, computer-aided instruction and future trends. 37. Giaser, R,, Teach in g Machines and Programmed Learning II: Data and Directions . National Education Association- Washington, D,C, 19^5 • 38. Green, B.F. Jr., Wolf, A.K. , Chomsky, C. and Laughery, K. , "BASEBALL: An Automatic Question Answerer", Computers and Thought , E.A. Feigenbaum and J, Feldman (eds,), McGraw-Hill Book Co., Inc., New York, 1963, pps, 207-216. Also in Proc . Wesx, Joint Comp . Conf . . Vol. 19, 1961, pps, 219-22U, 39. Green, E.J., The Learning Process and Programmed Instruction , Holt, Rinehart and Winston, Inc., New York, 1962 A philosophical introduction to PI emphasizing the problems and principles of psychological research in general and conditioning procedures in particular, UO. Hanson, L.F. , Programs '63, A Guide to Programmed Instructional Material Available to Educators by September 1963 , Center for Programmed Instruction, Inc., United States Printing Office, Washington, D.C., 1963, 1963 version of (12). Contains 352 program descriptions. Ul. Hoernes , G.E. and Heilweil, M.F., Introduction to Boolean Algebra and Logic Design , McGraw Hill Book Co., New York, 196U. An example of a programmed text. Contains appendices on advanced topics in non-programmed form. Fairly good program but hard to use as a reference. 1+2. Holland, J,G„ and Skinner, B.F., The Analysis of Behavior , McGraw- Hill Co., New York, 1961. A programmed text version of their original program. U 3 « Holland, J.G. and Kemp, F.D. , "A Measure of Programming in Teaching Machine Programs", Journal of Educational Psychology , Vol. 56, No. 5 1965, pps. 26U-269 • Defines the black-out ratio, a measure for determining the amount of critical material in a frame. kk. Homme, L.E. and Glaser, R. , "Relationships Between the Programmed Textbook and Teaching Machines", from Galanter (35), 1959, pps 103-107. U5. Hughes, J„L„, Programmed Learning: A Critical Evaluation , Aldine Pub Co., Chicago, 111., 1963. Contains 5 papers read at a 1962 conference prepared by the Foundation for Research on Human Behavior. Each paper is followed by a discussion. Very interesting and readable . hG. Jordan, J>A. Jr., "Socratic Teaching?", Harvard Educational Review , Vol., 33, No. 1, 1963, pps. 96-10U. Attacks the claim that teaching machines use the "Socratic Method". U7, Lumsdaine, A. A, and Glaser (eds . ) , Teaching Machines and Programmed Learning , Dept . of Audio-Visual Instruction, N.E.A., Washington D.C. , I960. 48. Lumsdaine, A, A, , "Some Problems in Assessing Instructional Programs", from Filep (30), 1963, pps . 228-262. U9 . Lys aught, J. (ed), Programmed Learning: Evolving Principles and Industrial Applications , Foundation for Research on Human Behavior, Ann Arbor, Michigan, 196l. 50. Lys aught, J.P, & Williams, CM,, A Guide to Programmed Instruction , John Wiley and Sons, Inc. New York, 1963.. 51. Margulies, S. , and Eigen, L.D, , Applied Programmed Instruction , John Wiley and Sons, Inc. New York, 1962. 52. Markle, S.M. , Eigen, L.D. , and Komoski , P.K.. A Programmed Primer on Programming , Center for Programmed INstruction, INc,, iMew York, 1961. 53. Markle, S.M. , Good Frames and Bad , John Wiley and Sons, Inc., New York, 196U. 5^o Markle, S., "Individualized Programmed Instruction: The Programmer's Part", from de Grazia and Sohn (2k), 1964, pps . IH5-I58. Advocates a combination of Skinner's and Crowder's methods. Uses branch-ahead features with constructed responses. 55. McNeil, J,D. , "Teaching Machines Can Help Improve Status of Teachers", Chicago Schools Journal , Vol. k3, No. h, Jan, 1962, pps 178-180. 56. Nordberg, R.B. , "Teaching Machines-Six Dangers and One Advantage", from Roucek (71), 1965, pps. 1-8. The disadvantages seem to consist of generalized anti- mechanistic fears, i.e., "Whatever lacks abstract intelligence is not able to initiate, transmit, or receive ideas". Little in the way of evidence although not all dangers can be easily dismissed. 57. Phi Delta Kappan , Vol. hh , No. 6, March 1963. A special issue devoted to PI. 58. Pipe, P., Practical Programming , Holt, Rienhart and Winston, Inc., New York, 19 66. Very readable book on writing programs-preparation, writing techniques, testing and revision, and editing. Tends to emphasize recent blurring of doctrine as a desirable effect. 59. Porter, D. , "A Critical Review of a Portion of the Literature on Teaching Devices", Harvard Educational Review , Vol. 27 9 No. 2, Spring 1957, pps. 126-147. Reviews a series of immediate reinforcement teaching devices. Contains a fairly long bibliography but most of the articles mentioned are not concerned with what would today be called teaching machines., A good source for the early articles on TM's. 60. Pressey, S.L. , "Apparatus Which Gives Tests and Scores", School and Society. Vol. 23, 1926, pps. 373-376. Presents a machine which presents multiple choice questions and scores the answers. 61. Pressey, S.L., "A Machine for Automatic Teaching of Drill Material", School and Society , Vol. 25, 1927, pps. 5^9-552. A modification of the machine in (60) which allows the repeated presentation of a group of questions until each is answered twice in a row at which time the question is dropped. 62. Pressey, S.L., "A Third and Fourth Contribution Toward the Coming 'Industrial Revolution' in Education", School and Society , Vol. 36, 1932, pps, 668-672. 63. Provus , M. and Stone, D..C. Programmed Instruction in the Classroom , Curriculum Advisory Service Inc., Chicago, 111., 1963. Short book presenting PI to teachers. Presents two composite case histories of PI use and discusses program selection and use. Reviews ^9 programs available in 1963. 6k. Raphael, B. , "A Computer Program Which 'Understands'":. Conf. Proc. Fall Joint Comp. Conf., 196U, Vol. 26, pp. 577-589- Describes SIR (Semantic Information Retriever), a program which can make relationships between various internal and external data and answer questions related to this data. 65. Rath, G.J. , "The Development of Computer-Assisted Instruction" IEEE Transactions on Human Factors in Electronics , Vol HFE-8, No. 2, June 1967 , pps. 6C-63. Gives 5 short histories of "che beginnings of CAI in the period 1958-61. 66. Richmond, W,K,, Teachers and Machines , Collins Pub. Co., London, England, 1965. Describes the history and effect of PI on education in Great Britain, Contains a program on the sonnet. 67. Rigney, J.W. , "Potential Uses of Computers as Teaching Machines", from Coulson (l6), 1962, pps. 155-170. Discusses the use of computers to compensate for individual differences in students, Notes the need for a systematic Lassification of individual differences, subject matter presentation, and their interactions., 68. Roe, A., "Research in Programmed Learning", from Coulson (l6) s 1962, pps., 1 J 3-119. Discusses some difficulties of PI research, large number of variables, performance criteria, etc. 69. Ross, W.L., et.al., Teaching; Machines: Industry Survey and Buyers Guide , The Center for Programmed Instruction, New Ycrk,1962, TO. Roth, R.H. , "Student Reactions to Programmed Learning", from de Grazia and Sohn (24), 1964 , pps. 281-286. Describes some generally unfavorable reactions to Holland & Skinner's, "The Analysis of Behavior". 71. Roucek, J.S. (ed. ), Programmed Teaching: A Symposium of Automation in Education , Philosophical Liorary, New York, 1965' A Series of articles from a teacher's point of view. Describes PI use in certain areas of education. 72. Ryan, W.F. , A Handbook of Programmed Learning Information , University of the State of New York, Albany, N.Y. , April 196U. Produced for N.Y. State teachers, Contains large glossary of terms, selected bibliography and a list of materials on PI available from the State Education Dept, Presents guidelines for PI introduction and a list of PI material manufacturers. 73. Schaefer, H. . "A Vocabulary Program Using Language Redundancy", from Glaser ( 132) , 1961, pps, 151-162. Describes a PI sequence used to teach reading German (see also -- j) "by gradually substituting German vocabulary and word order into English text, 74. Schaefer, H.H., "E. A. Poe as a Reinforcer", Psychological Repts .. Vol- I No. 3, June 196l, p, 398, Short note on a German language program (73) which used partially translated stories by E„ A. Poe, 75. Schramm, W. A., Programmed Instruction - Today and Tomorrow , Fund for the Advancement of Education, November 1962. Reviews history and all experimental work in PI up to 1962 but no reference list. 76. Schramm, W.A,, The Research on Programmed Instruction; An Annotated Bibliography ., Office of Education, Bulletin No, 35, U.S. Dept. H.E.W. , Washington, D.C. 1964. 77, Silberman, H.F., "Self-Teaching Devices and Programmed Materials", Review of Educational Research, Vol. 32, No, 2, 1962, pps. 179-193, Good review of literature on PI and good bibliography. 78. Simmons, R,F. , "Answering English Questions by Computer: A Survey", Comm. of the ACM . Vol. 8, No. 1, January 1965, pps . 53-70, 79, Skinner, B. F,, "The Science of Learning and the Art of Teaching", Harvard Educational Review , Vol. 24, 1954, pps. 86-97- Also in Smith (8l). One of the first important articles on teaching machines, Discusses lab work on learning and conditioning and compares this with current educational systems. 80. Skinner, B.F., , "Why We Need Teaching Machines", Harvard Educational Review . Vol 31, No, 4', Fall 196l, pps . 377-398. Also in de Grazia and Sohn Very complete argument for linearly programmed teaching machines. Presents the arguments against branching programming and the "feedback" principles, 81. Smith, W.I. and Moore, J.W., Programmed Learning: Theory and Research , D. Van Nostrand Co , Inc., Princeton, N,J\ , 1962. 82. Stolurow, L.M , Teaching by Machine , Cooperative Research Monography No. 6, U.S. Dept HEW, CLE.-3U010 , 196l. Excellent oA^erview of field, Describes machines in existence, learning theory, programming techniques, and research up to i960, A comprehensive set of references is included, 83= Stolurow, L.M. , "Principles for Programming Learning Materials in Self- Instructional Devices for Mentally Retarded Children" Final Report Cooperative Research Project No, 66l , SAE 8370, U of 111., 1963. Contains 5 reports and an extensive bibliography. Concerned with the effect of cueing and stimulus control, 8U. Stone, P.J. , Bales, R.F. , Namenwirth, J.Z., and Ogilvie, D.M. , "The General Inquirer: A Computer System for Content Analysis and Retrieval Based on the Sen. as a Unit of Information", Behavioral Science, Vol, '( , October 1962, pps,, i+8'4-498. 85. Swets , J, A, and Feurzeig, W,, "Computer-Aided Instruction", Science, Vol. 150, October 1965, pps 572-576,, Describes conversational mode teaching using a Socratic-type method, Discusses how to reduce rich materials to a step- by-ster. exposi* ion 86. Taber, J.I, . Glaser, R., and Schaefer, H..H , Learning and Programmed Inst r- ;c ~ .yen , Addi s on-We s 1 ey , Re adi ng , Mas s „ , 1 96 5 . Discusses es and pn Lures in PI« Reviews definitions I stim lus , response, reinf< rcement, extinction, etc, as they P e 87. Teal, G.E. (ed. ) , Pr- 1 ^-"^mmed Ins t ro-t ion in Industry and Education , Public Service Research. Inc., Stan , Conn, 1963. Ti Lp1 of a '.962 institute on PI. Covers the field from learning theory + ; programming methods tc evaluation. Hard efei ' bi luse of dialog format, 88 ,, Thomas, C A, end Davies , I.K. , Programmed Learning m Perspective , The Adelpbi Press, Barking, Essex, G.B. , 1963: Approaches PI from systems viewpoint. Discusses program Lng + he ruleg system - rule identification, matrix , diagrams and frame writing. Two complete examples 89. Thornhill, P., "What Future has Programmed Learning in British Schools", from Unwin (91), 1967, pps . 75-81 . Overview of British PI field from a publisher's viewpoint. Presents preliminary results of a 1964-65 survey. Suggests that a vicious circle may develop in which a lack of good integrated program series will stifle demand which, in turn will restrict production. 90. Tompkins, H.E., "Programmed Instruction", Advances in Computers . Vol- 4, F.L. Alt and M. Rubmoff (eds), Academic Press, New York, pps, 158-l6l Quick review of the use of computers in PI. Contains a short but good bibliography with comments. 91. Unwin, D. and Leedham, J., Aspects of Educational Technology , Methuen and Co, , Ltd. , London, 'SCj '. Contains 43 papers presented at a 1966 conference in Loughborough, England. Concerned with production and use of PI in Britain and Europe, 92. Winn, S. , "Programmed Instruction: Panacea or Plague?", Chicago Schools Journal , Vol. 43, No. 3, December 196l, pps. 131-133, 93. Wohlwill, J.F., "The Teaching Machine: Psychology's New Hobbyhorse", Teacher's College Record , Vol. 64 , No. 2, Nov, 1962, pps. 139-146. Experimental Reports: 94. Barkus , D. , Hayman, J.L., and Johnson, J.T., "Programming Instruction in Elementary Spanish", from de Grazia and Schn ( 22) ,1964 ,pps „ 226-232 Discusses Denver School System experiment with programmed learning in Spanish (see 1.82) and presents some of the data, 95. Bartz, W.H„ and Darby, C.L., "A Study of Supervised and Ncnsupervised Programmed Instruction in a University Setting", Journal of Ed . Research . Vol. 58, No. 5. January 1965 96. Bartz, W.H. and Darby, C.L. , "The Effect of a Programmed Textbook on Achievement Under Three Techniques of Instruction", Journal of Experimental Education , Vol. 34, No= 3, pps. 46-52. 97. Beane, D,G,, "A Comparison of Linear and Branching Techniques of Programmed Instruction in Plane Geometry", Journal of Educational Research . Vol,, 58, No. 7, March 1965 , pps. 318-326, 98. Bitzer, D.L. et . al . "The PLATO System: Current Research and Developments", IEEE Transactions on Human Factors in Electronics , Vol. HFE 8, No- 2 5 June 1967 , pps. 64-70. 99- Bitzer, D.L, Lyman, and Suchman, "REPLAB - A Study in Scientific Inquiry Using the PLATO System", U, of HI., Coordinated Science Lab. Report R-260 , December 1965. 100. Bitzer, D.L. , et.al., "The Uses of PLATO: A Computer Controlled Teaching System", U. of 111, Coordinated Science Lab. Report R-268, October 1965. 101. Bitzer, D.L. et. al. , "PLATO II: A Multiple-Student , Computer- Controlled, Automatic Teaching Device", from Coulson (l6) s 1962, pps. 205-206. 102. Bivens, L. W. Cambell, V.N- , and Terry, D.F., "Self Direction in Programmed Instruction: Effects on Learning in Low-Ability Students" American Institute for Research Report AIR-D10 -T163-TR , July 1963. 103. Braunfeld, P.G. , "Problems and Prospects of Teaching with a Computer", Journal of Educational Psychology , Vol. 55 > No. h t 1964, pps ; 201- 211. Describes one of several informal experiments using the PLATO II system. 10U, Briggs, L.J. et.al., "Experimental Results Regarding Form of Response, Size of Step, and Individual Differences in Automated Programs", from Coulson (l6), 1962, pps . 86-98, 105^ Briggs, L.J. et.al., "Investigations of Thinking via Self-Instructional Programs", American Institute for Research, AIR-DU1-6/6U-FR , June 1964. Describes in detail two experiments (106 , 111) with varying formats and presentation techniques. 106, Brooks, L.O., "Laboratory Study of Individual Performances on Programmed Learning Tasks", from Briggs (105), 196U, pps. 32-69 Investigates latency time as a measure of frame difficulty, 107 Bruce, J.S. "Applied Research on Programmed Instruction a"c the Eastman Kodak Company", from Hughes (^5), 1963, pps, 59-110 Describes integrated use of PI in Kodak training programs. 108 Camb v.N , "Studies of Bypassing as a Way of Adapting Self- Ir.str actional Programs to Individual Differences", American Institute lor Research, AIR-CU.l-5.'62-FR, May 1962. Also reported in J, of Ed. Psy. , Vol. 5^, No, 6, 1963, pps, 33 7 -3^5- 109- Cambell, V.N. et . al. , "Effects of Mathematical Ability, Pretraimng, and Interest on Self-Direction in Programmed Instruction", American Institute fox Research, AIR-D10-10/63-TR, Oct,. 1963. 110, Cambell, V.N , "Self-D Lon and Programmed Instruction for Five Different Types of Learning Objectives", American Institute for Research, AIR-D10-12/63-TR(b) , December 1963. 111. Cambell, V.N. , "Learning to Solve Problems: A Classroom Experiment", from Briggs (105), 196U , pps . 12-31. Investigated the effects of various types of reviews after studying a PI geography sequence, on the ability to solve problems in geography. 112. Cambell, V.N., and Chapman, M. , "Degree of Student Control over Programmed Instruction: Long Term Cummulative Effects on Problem Solving and Transfer", American Institute for Research, AIR-E20- 12/65-TR, Dec. 1965 113. Cassel, R.N. and Ullom, W.L., "Preliminary Evaluation of Programmed Instruction with Students of High Ability", Psychological Reports , Vol, 10, No. 1, February 1962, pps. 223-228, 11^. Clarke, J., "The Development and Use of Linear Programmed Instruction in a Rural Primary School", from Unwin (91), 1967, pps . UI-U9. 115. Coulson, J.E. and Silberman, H.F. , "Effects of Three Variables in a Teaching Machine", Journal of Educational Psychology , Vol,. 51, No, 3, I960, pps. 135-1^3. Investigates the effects of multiple choice vs, constructed response, size of the frame, and fixed sequences vs. branching sequences . 116. Coulson, J.E. and Silberman, H.F. "Automated Teaching and Individual Differences", from Smith (8l), 1962, pps . 207-217. Also in Audio-Visual Communications Review , Vol. 9, 196l, pps. 5-15 » Describes the System Development Corp, teaching system using a Bendix G-I5 computer. 117. Coulson, J.E. , "A Computer-Based Laboratory for Research and Develop- ment in Education" from Coulson (l6) 9 1962, pps. 191-20U. Describes CLASS, an integrated approach to computer-aided education, 118. Coulson, J.E, et , al. , "Non-Program Variables in the Appliance of Programmed Instruction", System Development Corp,, TM-2176/201/00 , July 1965. Reports on two experiments in which the teacher augmented ordinary PI or the students controlled their own branching. 119. Coulson, J.E., "Effects of Branching in a Computer Controlled Auto- Instructional Device", Journal of Applied Psychology , Vol, U6, No. 6, 1962, pps. 389-392, 120. Cowan, P„J,, "Autoinstructional Materials in Teaching Physics in Small Schools", J, of Exp, Ed . , Vol. 36, No, 1, Fall 1967, pps . k6- 50. 121= Davies, I.K., "Programmed Learning in the United States Air Force", Programmed Learning, Vol. 1, No. 3, November 196U, pps . 113-123. Describes a two-phase development program to implement PI in the USAF (1961-1963). 122. Edling, J.V., "Programmed Instruction in a 'Continuous Progress' School - Provo, Utah", from Schramm (l8l), 1964, pps. 66-94. Reports on two years of use of PI at the Brigham Young University Laboratory School, a "non-paced" school. 123. Englund, D.E. and Estavan, D.P. , "CLASS - The Automated Classroom", Computer Applications 196l-Proeeedings of the 196l Computer Applications Symposium , R.S. Hollitch and B, Mittman (eds.), The MacMillan Co., New York, 1961, pps. 177-188. 124. Evans, J.L., Glaser, R. , and Homme, L.E, , "An Investigation of 'Teaching Machine' Variables Using Learning Programs in Symbolic Logic", from Glaser (132), 1961, pps. 31-81. 125. Feldhusen, J.F. and Birt, A., "A Study of Nine Methods of Presentation of Programmed Learning Material", J. of Educational Res ., Vol. 55, 1962, pps, h6l-k66. 126. Feldman, M.E. , "Learning by Programmed and Text Format at Three Levels of Difficulty", Journal of Educational Psychology , Vol. 56, No. 3, pps. 133-139. 127. Feurzeig, W. , et.al., "Computer-Aided Teaching in Medical Diagnosis", J. of Medical Ed . , Vol. 39, August 196U, pps. 746-754. 128. Finelli, CM., "The Application of Programmed Learning Techniques to Selected Material in a Professional Curriculum- A Study of Retention Effects", from Glaser (132), 1961, pps. 115-133. 129. Flynn, J.T., "The Influence of Programmed Instruction Upon Learning in Educational Psychology" Journal of Educational Research , Vol. 59, Nco 9, 1966, pps. 387-391 . 130. Friedman, M.I., "The Effectiveness of Machine Instruction in the Teaching of Second and Third Grade Spelling" , J. of Educ . Res . , Vol. 60 , No, 8, April 1967, pps. 366-369. Results tend to show PI better ax 3rd grade level but not at 2nd grade level, 131. Frye, C.H., "Group vs. Individual Pacing in Programmed Instruction", U.S. Dept. of H.E.W. , Report for Nat. Def. Ed. Act of 1958, Grant No. 7-47-0000-158. 132. Glaser, R. , Investigations of the Characteristics of Programme d Learning Sequences . Programmed Learning Laboratory, U. of Pitt., October 1961. Reports on a series of experiments using linear programs (73, 124, 128, 138, 179). Developes the ruleg system of programming (27). Tested effectiveness, response mode, prompts, reinforcement schedules, etc. 133. Goldbeck, R.A. and Cambell, V.N. , "The Effects of Response Mode and Response Difficulty on Programmed Learning" Journal of Educational Psychology , Vol. 53, No, 3, 1962, pps. 110-118. 134. Goldbeck, R.A. , et.al., "Integrating Programmed Instruction with Con- ventional Classroom Teaching", American Institute for Research, AIR-CU9-12/62-FR, December 1962. 135. Gulo, E.V. and Nigro, M.R, , "Classroom Learning as a Function of Method of Presenting Instructional Materials", Psychological Repts . . Vol. 19, No. 3, Part I, December 1966, pps, 971-977. Compared the teaching efficiencies of PI, television, and conventional texts. -Conventional text group learned the most . 136. Herbert, J, and Foshay, A.W. , "Programmed Instruction in the Manhasset Junior High School", from Schramm (l8l), 1964 9 pps, 18-27 > Discusses the administrative problems and results involved in the introduction of the programmed sequence English 2600 during 1960-63. 137- Herrick, M.C., "The Effect of Problem- Setting Questions en Rate and Amount of Learning in Programming Teaching Machines", Office of Education, U.S. Dept , of H.E.W., Rept „ on Grant 712130, May 1962„ 138. Hershberger, tf, , "Self-evaluational Responding and Typographical Cueing: Techniques for Programming Self-Instructional Reading Materials", Journal of Educational Psychology . Vol. 55, No, 5, 1964, pps, 288-296 139. Hershberger, W.A„ and Terry, D,F,, "Delay of Self-Testing in Three Types of Programmed Text", Journal of Educational Psychology , Vol. 56, No. 1, 1965, pps. 22-30* 140. Hessert, R.B. , "A Comparison of Three Methods cf Programming Subject Material for Auto-Instruction", from Glaser fl32)„ 196l, pps, 97- 114. Compared a linear program prepared by the ruleg method, a prompting form of this program, and a third form in which multiple choice questions were used which had to be answered correctly before continuing, lUl. Holt, H.O., "An Exploratory Study of the Use of a Self-instruction Program in Basic Electricity" from Hughes (U5), 1963, pps . 15-58. 1^2. Hughes, J.L. , "Some Aspects of IBM Research on Programmed Instruction" from Hughes (U5), 1963, pps. 161-197. Describes some of the PI techniques used at IBM for employee training. lU3. Hunt, W.A. and Mathis ,, C, "The Use of Programmed Instruction in Introductory Psychology for Teachers", Office of Ed., U.S. Dept. of H.E.W., Report for Nat. Def. Ed. Act of 1958, Grant No. 7-23- 0790-207. ikk. Huskey, H.D. , "Automatic Computers and Teaching Machines", from Coulson (16), 1962, pps. 257-272. Develops a compiler for computers for use in CAI. IU5. Jeffels, A., "Programmed Techniques and Their Use in Apprentice Training" from Unwin (91), 1967, pps. 135-1 1 49. lh6. Kazmier, L.J., "Repeated Exposure to Programmed Instruction and Student Attitude", Psychological Reports , Vol. 19, No, 3, Part I, December 1966, pps. 985-986. Repeated use increased probability of a favorable attitude. 1^7. Keislar, E.R. , "The Development of Understanding in Arithmetic by a Teaching Machine", Journal of Educational Psychology , Vol. 50, No. 6, 1959, pps. 2li7-253. lU8 . Keislar, E.R, and McNeil, J.D. , "A Comparison of Two Response Modes in an Autoinstructional Program with Children in the Primary Grades", Journal of Educational Psychology . Vol. 53, No. 3, 1962, pps. 127-131. 1^9. Krumboltz, J.D. and Weisman, R.G. , "The Effect of Overt vs. Covert Responding to Programmed Instruction on Immediate and Delayed Retention", Journal of Educational Psychology , Vol. 53, No. 2, 1962, pps. 89-92. 150. Krumboltz, J.D. and Weisman, R.G., "The Effect of Intermittent Confirmation in Programmed Instruction", Journal of Educational Psychology , Vol. 53, No. 6, 1962, pps. 250-253. 151. Leith, G.O.M. , "The Role of Overt Responding in Programmed Learning", from Unwin (91), 1967, pps. 503-513. 152. Leonard, J,M. and Wing, R.L. , "Advantages of Using a Computer in Some Kinds of Educational Games", IEEE Transactions on Human Factors in Electronics , Vol, HFE-8, No, 2, June 1967, pps. 75-81. Believes computer is better than teacher in economics oriented educational games. Describes two implementations. 153. Levin, G.R. and Baker s B,L,, "Item Scrambling in a Self-Instructional Program", Brown University Report, Providence, R.I,, 1961, Also in Journal of Educational Psychology , Vol. 5^» No. 3, 1963, pps. 138-1^3. 15^. Levis, B.N, and Pask, G. , "The Development of Communication Skills Under Adaptively Controlled Conditions", Programmed Learning , Vol. 1, No. 2, July 1961*, pps. 69-88, 155. Lewis, D.G. and Gregson, A,, "The Effect of Frame Size and Intelligence on Learning from a Linear Program", Programmed Learning , Vol > 2, No. 3, October 1965, pps. 170-175- 156. Licklider, J.C.R., "Preliminary Experiments in Computer- Aided Teaching", From Coulson (l6), 1962, pps, 217-239 . 157. Mager, R.F. and Clark, D.C, "Normative Feedback in Automated Instruc- tion", Psychological Reports . Vol. 13, No. 2, 1963, pps, 599-6l6. Used PI to try to teach nonobjective subjects (judgement of smoothness 1 ) by normative feedback, i.e. degree of agreement of the learner's response with an averaged expert opinion, 158. Malpass, L,F. et . al., "Programmed Instruction for Retarded Children", from Roucek (71), 1965, pps, 169-179. 159. Mayer, S.R. & Moegan, R, , "Computer- Aided Programmed Techniques in Support of Military Information Systems", Proc, ACM 20th Nat , Conf . , August 1965, pps, 336-3^3 . Reports on the use of a combination of Programmed text and computer to instruct military staff personnel in the use of a computer information system, l60- Merrill, M.D, , "Transfer Effects Within a Hierarchial Learning Task as a Function of Review and Correction on Successive Parts 9 Training Res, Lab., U. of 111., Tech, Rept , #5, September 196U , 161, Mooney, P, , "Individualized Use of AUTHOR I: A Clinical Application of a Computer-Prepared Program", from Stolurow (83), 1963 , pps, 195-207, Describes the computer preparation cf a program for a young boy with a special learning problem. 162, Moore, J.W. & Smith, W,I, 9 "A Comparison of Several Types of 'Immediate Reinforcement'", from Smith (8l), 1962, pps, 192-201, Compared immediate knowledge of answer, immediate knowledge of correctness, and immediate knowledge plus reward for multiple choice and constructed response programs given by machine and texts. l63. Morse, R.J. , "Programmed Training: An Industrial Tool" from Hughes (^5), 1963, pps. 198-238o l6U. Newton, J.M. and Hickey, A.E. , "Sequence Effects in Programmed Learning of a Verbal Concept", Journal of Educational Psychology , Vol. 56, No. 3, 1965, pps, 1UO-IU7. 165. Noble, G. , "An Experimental Attempt to Integrate Programmed Instruction ■with Classroom Instruction", from Unwin (91), 1967, pps . 107-118. 166. Normington, K. , Babb , Jowett , & Holroyd, "Programmed Learning in Local Government - The Large-Scale Use of Programmed Learning in the West Riding Fire Service" from Unwin ( 91) , 1967, pps. 159-165- 167. Oakes, W.F., "Use of Teaching Machines as a Study Aid in an Introductory Psychology Course", Psychological Reports , Vol. 7, No. 2, October I960, pps. 297-303. 168. O'Donnell, L.H. "Training of Plant Operators and Maintenance Personnel", from Hughes (1+5), 1963, pps. 111-160. Describes PI use at the DuPont Company. 169. Pollock, J.M. , "The Preparation of a Programme to Train Boiler Operators", from Unwin (91), 1967, pps. 151-157- 170. Popham, W.J., "The Influence of Novelty Effect Upon Teaching Machine Learning", San Francisco State College, report for Dept. H.E.W., Grant No. 7-1^-1230-155 -0 , August 1962. 171. Porter, D. , "Some Effects of Year Long Teaching Machine Instruction", From Galanter (35), 1959, pps. 85-90. 172. Pressey, S.L. and Kinzer, J.R. "The Effectiveness of Adjunct Auto- Instruction", U. of Arizona, Comparative Res. Proj . #2306, 196U. 173. Price, J.E., "A Comparison of Automated Programs with Conventional Teaching Methods as Applied to Teaching Mentally Retarded Students", The Partlow State School and Hospital, Tuscaloosa, Alabama, 1962. 17l+. Rath, G. Anderson, N.S. and Brainerd, R.C., "The IBM Research Center Teaching Machine Project", from Galanter (35), 1959, pps. 117-130. 175- Richardson, J,0., "Teaching Mathematics Using a Time-Shared Computer System", Computers and Automation , Vol. 15, No. 3, March 1966, pp. lU-17. I'6. Ripple, R.E., "Comparison of the Effectiveness of a Programmed Text with Three Other Methods of Presentation", Psychological Rents.. Vol. 12, No. 1, 1963, pps. 227-237. I'7. Roe, A,, "A Comparison of Branching Methods for Programmed Learning", J. of Ed. Res . Vol, 55, No. 9, June-July 1962, pps, 407-416. fB. Roe, K.V. , Case, H.W. , and Roe, A., "Scrambled vs. Ordered Sequence in Autoinstructional Sequences", Journal of Educational Psychology , Vol. 53, No. 2, 1962, pps, 101-104. BP. Scharf, D.S., "A Study of the Effects of Partial Reinforcement on Behavior in a Programmed Learning Situation", from Glaser, (132), 196l, pps, 83-96. BO. Schlesinger, L.E. , Fischer, E.H. and Cohen, S.L. , "The Effect of Relevant Emotional Content on Performance and Learning in Programmed Instruc- tion", George Washington Univ., Final Rept . Grant No. 7-18-OIOO-197, February 1965. Investigate fear arousing content to motivate students to learn PI sequences on health and safety. 31. Schramm, W. , Four Case Studies of Programmed Instruction, Fund for the Advancement of Education, New York, June 19 6 4, Describes four case studies on the introduction of PI into a school system, 32. Schramm, W. , "Programmed Instruction in Denver", from Schramm (l8l), 196U, pps. 29-40. 33. Schutz, R.E., Baker, R.L. and Gerlach , V.S., "Measurement Procedures in Programmed Instruction", Arizona State University, Tempe , Arizona, Final Rept, Grant No. 7-12-0030-160 , August 196U . 34. Senter, R.J, and Neiberg, A., "An Evaluation of Branching and Motivational Phrases in a Scrambled Book", Programmed Learning ;, Vol, 1, No, 3, November 1964, pps. 124-133. 85. Shay, C.B. , "Relationship of Intelligence to Step Size on a Teaching Machine Program", Journal of Educational Psychology , Vol, 52, No. 2, 1961, 98-103.. 86. Short, J, & Haughey, B.E. , "An Experimental Study of Sequencing Strategies", American Institute for Research, Final Rept. for Grant No, 0E-7-^8- 7670-259, December 1966,, 87. Silberman, H.F., et, al. 9 "Fixed Sequence vs. Branching Auto-Instructional Methods", Journal of Educational Psychology , Vol, 52, No, 3, 196l, pps, 166-172. 188. Silberman, H.F. and Coulson, "Automated Teaching", Computer Applications in the Behavioral Sciences , H. Borko , (ed. ), Prentice Hall, Engle- wood Cliffs, N.Y., 1962, pps . 308-335- Describes the use of computers in PI. Briefly mentions various projects and then goes into detail on the SDC project in which slides are presented on the basis of student errors and pre- inserted information. 189- Silberman, H. , Coulson, J., et. al. , "Use of Exploratory Research and Individual Tutoring Techniques for the Development of Programming Methods and Theory", System Development Corp., TM - 895/200/00, June 1964. 190. Silverman, R.E. and Alter, M. , "Note on the Response in Teaching Machine Programs", Psychological Reports , Vol. 7, No. 3, December i960 , p. U96. Note on preliminary findings which seem to indicate that covert responses produce more learning than overt. 191. Slaichert, W.M. and Stephens, M.L., "The Effectiveness of a Programmed Text in Geometry", J. of Ed. Res . . Vol. 57, No. 10, 1964, pps. 5U2-5UU 192. Stolurow, L.M. , "The Effects of Sequence in Learning Complex Associative Paradigms that are Taught in Both a Forward and a Backward Direction", from Stolurow (83), 1963, pps. 61-108. 193. Stolurow, L.M. , "A Computer Assisted Instructional System in Theory and Research", from Unwin (91), 1967, pps. 257-272. 194. Strum, R.D. and Ward, J.R. , "Some Comments on Computer-Assisted Instruc- tion in Engineering Education", IEEE Trans, on Education . Vol. E-10 , No. 1, March 1967, pps. 1-3. Reports on a 1965 evaluation of the use of COURSEWRITER . Authors felt system was not as good as programmed texts because of high cost, large programming effort needed, inability to interprete student answers and poor man/ machine interaction. 195- Thelen, H. , and Ginther, J.R., "Experiences with Programmed Materials in the Chicago Area", from Schramm (l8l), 1964, pps. hl-62 . 196. Uttal, W.R., "On Conversational Interaction", from Coulson (l6), 1962, pps. 171-190. Discusses 3 experiments using an IBM 650 RAMAC to teach steno-typing, statistics, and German reading. 197 • Wallis , D. and Wicks, R.,P., "The Royal Navy Study", Programmed Learning 9 Vol. 1, No, 1, May 196U, pps, 31-^7 . Mark II autotudors were found to teach better than Tudor texts and conventional instruction. They were also better liked by the students. 198. Williams, J, P., "Comparison of Several Response Modes in a Review Program", Journal of Educational Psychology ^ Vol. 5^ 5 No. 5 9 1963, pps. 253-260. 199= Williams, J.P. , "Effectiveness of Constructed Response and Multiple- Choice Programming Modes as a Function of Test Mode", Journal of Educational Psychology , Vol. 56, No. 3, 1965, pps. 111-117. 200. Wing, R.L. , "Use of Technical Media For Simulating Environments to Provide Individualized Instruction", The Board of Cooperative Educational Services, Westchester County, New York, Report on Cooperative Research Proj . No. 19^8, 1965. Reports on computer simulation of a teacher. Summarizes work on PLATO and at IBM, SDC , Stanford, Bolt, Beranek and Newman, and Michigan U. Mentions Moore's "talking typewriter". 201. Wittrock, M.C., "Response Mode in the Programming of Kinetic. Molecular Theory Concepts", Journal of Educational Psychology s Volo 5^ s No. 2, 1963, pps. 89-93. 202. Zeigarnic, B. , "On Finished and Unfinished Tasks", A Source Book of Gestalt Psychology . W.D. Ellis (ed. ), Harcourt, N.Y., 1938, pps. 300-31U. JUN 2 < 1968 J UN 2 01969