TROUBLE SHOOTING ELECTRONIC EOUIPMENT An Empirical Approach to the Identification of Certain Requirements of a Maintenance Occupation by Joseph L. Saupe A Report of an Aspect of a Research Project Under Air Force Contract No. 33(038)-13236 Project No. 507-007-0001 Francis G. Cornell, Principal Investigator Bureau of Educational Research College of Education University of Illinois, Urbana May 1954 TROUBLE SHOOTING ELECTRONIC EOUIPMENT An Empirical Approach to the Identification of Certain Requirements of a Maintenance Occupation by Joseph L. Saupe A Report of an Aspect of a Research Project Under Air Force Contract No. 33(038)- 13236 Project No. 507-007-0001 Francis G. Cornell, Principal Investigator Bureau of Educational Research College of Education University of Illinois, Urbana May 1954 I ' iuikois Q\ - COLLECT! Qcrp-S v PREFACE This report culminates two years of experimentation directed toward the isolation of strategic aspects of performance in trouble shooting in radio. The orientation of the contract under which this work was accom- plished was proficiency measurement of electronics maintenance mechanics. On rational grounds the area which should be most important in distinguishing between the most effective and the least effective mechanics is that area of problem solving commonly termed ''trouble shooting ," that is, finding the trouble. Early phases of the project of which this is a part involved proficiency measurement of mechanics on complex radar-computer equipment. Such complex equipment did not lend itself readily to economical experimental study. It seemed desirable for purposes of a pilot investigation of problem- solving (or trouble- shooting) behavior in. this maintenance field to use the much simpler circuitry of a simple radio receiver. Hence a specially designed superheterodyne radio receiver was constructed by the proje'ct staff so that troubles could be readily inserted and so that step-by-step trouble- shooting operations of subjects could be observed and recorded. Records were made of the trouble- shooting "process" of 40 radio mechanics each on 8 "troubles.'" Mr. Saupe became a member of the project staff near the comple- tion of the field work and was assigned responsibility for completing the o/ study. His contribution in maximally utilizing information available from a small sample of subjects has resulted in several highly suggestive leads for further research in the area. Once basic components of a maintenance task are isolated and defined in empirical language, the researcher is well on the road to measurement useful not only in determining over-all profi- ciency, but also in the selection and in the aiding, through diagnosis, of various phases of training,, The present publication makes only passing reference to some of the other steps undertaken or contemplated in the larger project of which 4 this is a part One aspect concerns determination of differential effects of training on components of trouble- shooting ability of various types of trainees at various levels. Another aspect deals with the extent to which the testing device used in this project is characteristic of other types of no n- laboratory airborne equipment fe This report by Mr Saupe was submitted as a thesis in partial ful- fillment of the requirements for the degree of doctor of education in the .Graduate College of the University of Illinois. V Francis G. Cornell, Principal Investigator Digitized by the Internet Archive in 2013 http://archive.org/details/troubleshootingeOOsaup TABLE OF CONTENTS Chapter Page I. INTRODUCTION 1 The Need for Research on Maintenance Occupations . 1 Purposes of This Study 4 Review of Research on the Trouble-Shooting Process 5 Prospectus. 9 II. THE THEORETICAL ORIENTATION AND STATEMENT OF HYPOTHESES 10 The Trouble-Shooting Situation. . 10 The Psychology of Problem Solving 13 The Trouble-Shooting Prototype ■ 23 Statement of Hypotheses < 24 Summary 28 III. THE INSTRUMENTS USED AND THE EXPERIMENTAL PROCEDURE e 29 The Ul/BER-1 Performance Test 29 The Measure of Basic Electronics Knowledge 40 The Experimental Procedure. . « . 42 IV. RESULTS 47 Gross Results. . . . 47 Statistical Tests Employed 50 Hypothesis Concerning Knowledge of Facts and Principles . . 53 Hypotheses Concerning Components of the Problem- Solving Process 54 Hypothesis Concerning Methods of Attack. 77 Summary and Conclusions 83 V. IMPLICATIONS. . , 86 Limitations 86 Implications for Further Research 87 Implications for Training Programs. . 89 Summary . 93 VI. SUMMARY AND CONCLUSIONS. . 94 The Specific Hypotheses Investigated 94 The Method of the Study 95 Testing the Hypotheses and Results 96 The Study as an Approach to Determining Training Requirements. „..„...... 98 General Conclusions 98 BIBLIOGRAPHY. 100 iii Page APPENDIX A. HANDBOOK OF MAINTENANCE INSTRUCTIONS FOR RADIO RECEIVER Ul/BER-1 103 B. FAMILIARIZATION PROCEDURE USED IN ADMIN- ISTRATION OF Ul/BER-1 PERFORMANCE TEST. . 117 "TROUBLE AREAS" FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS 119 "CLUES" DEFINED FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS 120 C. CONTINGENCY TABLES USED FOR TESTING HYPOTHESES CONCERNING THE CONTRIBUTION OF COMPONENTS OF THE PROBLEM-SOLVING PROCESS TO THE SOLUTION OF TROUBLE- SHOOTING PROBLEMS 123 IV LIST OF TABLES Table Page 1. Classification of Items Comprising the Basic Electronics Knowledge Test 43 2. Proportion of Agreement among Three Independent Raters for Interpretations Required in Completing Columns (6) and (8) of the Analysis Record Sheets 45 3. Frequency Distribution of Performance Test Pass-Fail Scores 47 4. Frequency Distributions of Basic Electronics Knowledge Test Scores 7T7T" . . . 7TTT7 48 5. Means, Variances, Standard Deviations, Reliabilities, and Intercorrelations of Performance Test Pass-Fail Scores i and Basic Electronics Knowledge Test Scores. ► 49 6. Critical Values for Statistics Employed in This Study. 53 7. Distribution of Number of Correct Statements of Symptom. ... 56 8. Exact Probabilities for Hypothesis 2. ........ , ....... . 57 9. Distribution of Number of Problems with Initial General Checks 58 10. Exact Probabilities for Hypothesis 3. 59 11. Distribution of Number of Problems for which First Hypothesis was Correct. .....«, » 61 12. Exact Probabilities for Hypothesis 4o » ■ • • . ■ 62 13. Distribution of Number of Problems for which Incorrect Hypotheses Occurred 64 14. Exact Probabilities for Hypothesis 5-1. ...... 64 15. Distribution of Number of Problems for which Hypothesis Was Perseverated 66 16. Exact Probabilities for Hypothesis 5-2. ».».»..»»»•.,.. 66 17. Distribution of Total Wrong Hypothesis Behavior Scores 68 18. Distribution of Total Plus Clue Scores. 69 19. Exact Probabilities for Hypothesis 6-1. .,..«.. « 70 Table Page 20. Distribution of Total Minus Clue Scores. . . - 71 21. Exact Probabilities for Hypothesis 6-2 72 22. Distribution of Total Error Scores 73 23. Exact Probabilities for Hypothesis 7 74 24. Distribution of Total Duplication Scores 75 25. Exact Probabilities for Hypothesis 8. . . 7 6 26. Distributions of Numbers of Problems Classified into Five Methods of Attack Categories. „ . . 7 9 27. Analysis of Variance of Pass-Fail Scores for Methods of Attack Groups. 81 28. Analysis of Covariance for Differences between Pass-Fail Means Adjusted for Differences in Basic Knowledge Means „ . . . . . . 82 29. Summary of Tests of Hypotheses Concerning Components of the Problem-Solving Process. .=..... . . . . 84 LIST OF FIGURES Figure Page 1. The Obser v ation Record Used for Recording Overt Trouble-Shooting Behaviors. ■ 35 2. The Analysis Record Sheet Used for Recording Interpretation of Trouble-Shooting Behavior 37 vi ACKNOWLEDGMENTS The writer wishes to acknowledge his gratitude to Professor Francis G. Cornell, his faculty advisor, who suggested this inves- tigation and gave constant encouragement and guidance throughout its execution,, He wishes to thank provessors J. Thomas Hastings, Mo Ray Karnes, Glenn M. Blair, and B. Othanel Smith for critically reading the manuscript and making suggestions for its improvement. He is grateful to Dr. Dora E. Damrin, Dr. Floyd M. Gardner, and Ronald F. Tucker, who developed the experimental, performance- testing device used in the investigation; and to Charles B. Porter and Joseph V. Wenta, who gave needed technical assistance concerning electronics. vii CHAPTER I INTRODUCTION 1 The inventions and discoveries which have characterized the technol- ogical developments of recent decades have created concomitant problems in the maintenance of the many types of complex contrivances today taking on importance in many areas of human life. The specialized occupations required to maintain these products of technology are in a stage of growth and development. Problems which have arisen as a result of the need for competent and efficient maintenance workers are becoming the subject of scientific study. The research which is reported here was an investigation of certain aspects of the duties associated with a particular maintenance occupation, that of the electronics technician charged with maintaining a particular type of electronic equipment. The Need for Research on Maintenance Occupations A primary incentive for research on the requirements of main- tenance occupations is the expanding roles which various types of technical equipment are playing on the modern scene. Associated with this technol- ogical development is the need for specialized training directed toward the development of skills and abilities required in the efficient performance of maintenance duties. It is readily apparent that electronics and many types of electronic principles are becoming increasingly useful in the performance of tasks associated with both civilian and military requirements. For example, the rise of the television industry creates a need for a large body of well- trained television repairmen. It is forseeable that in the near future there will be new commercial applications of the electronics developments which have been evolved and utilized primarily by the military in association with its needs. The present study was executed in connection with one phase of a research project conducted under contract with the United States Air Force, Hence, the more immediate stimulation for it resided in the problem of maintaining the electronic equipment used by this establishment. The *This research was supported in part by the United States Air Force under contract AF 33(038)-13236 , monitored by the Armament Systems Training Research Laboratory of the 6564th Research and Development Group, Human Resources Research Center. Permission is granted for reproduction, translation, publication, use, and disposal in whole and in part by or for the United States Government. ^Under the direction of Francis G. Cornell, University of Illinois. -1- primary objective of the research accomplished on the larger project was concerned with the measurement ot proficiency of mechanics in maintaining a particular radar equipment. The present study involved, however, an experimental analysis of certain aspects of the proficiency required in the maintenance of radio equipment. Radar and radio are both strategic compo- ents of the vast network of electrical and electronic equipment which are assuming tremendously important responsibilities in the military. Further- more, maintenance job requirements and problems encountered by the need for highly efficient maintenance personnel for the two types of equipment are comparable in many respects. The importance of maintenance is well recognized by the military. In discussing implications for the Air Force of research results reported in a military research bulletin, a technical director of Air Force research and a commanding officer recently stated thats "Maintenance of highly complex electronic equipment represents an important element of effective combat operations. 3 a particular iy lucid statement of the maintenance problem has been made by Saltz and Moore s The combat effectiveness of a modern Air Force depends on the speed of its planes and the accuracy of its weapons. However .. as increased speeds are attained, it often becomes necessary to develop increasingly complex devices to control the planes and the weapons. Unfor- tunately, as the complexity of equipment increases, its stability decreases. Equipment, with more components and with more complex functions has a greater probability of developing malfunctions than does simple equipment. When such malfunctions occur, because of the multitude of parts and the complexity of their interrelations, the identification of the particular component or components causing the trouble can be extremely difficult. Once the components are located, replacement of them is rel- atively easy; but the problem of training mechanics how to isolate rapidly the source of trouble (trouble shooting) in complex equipment has not been solved, That the problem is a real one can be readily demonstrated. For example, personnel concerned with the training of radar mechanics estimate that only about five per cent of trouble shooters are able to locate radar ^ Arthur W. Melton and Herbert N Cowles. "Implications for the Air Force, in Francis G. Cornell and Dora E, Damrin. A Preliminar y Report on the Development of a Functional Knowledge Test Battery for the Meas - urement of Proficiency of Radar Mechanics , Research Bulletin 52- 30. San Antonio, Tex.: Human Resources Research Center Lackland Air Force Base, 1952. p. iii. malfunctions rapidly and accurately. Similar difficulties are reported in other equipment areas. One source of difficulty is associated with the introduction into general use of new and more complex types of equipment. The problems created by the lag between the initial production of this equipment and the training of personnel to maintain them is being attacked. The complexity of electronic equipment creates problems of the maintenance task as related to design and physical fabrication. For example, one requirement in airborne equipment is minimization of weight. The compactness of design, thus required, tends to decrease the ease, and hence the efficiency, with which it can be maintained. Suggestions have been made relative to the design of such equipment with regard to maintenance efficiency. On the human resources side, the strategic element is the selection and training of potential maintenance mechanics. i The maintenance problem is complicated by the fact that maintenance personnel are in general;, not electronic engineers. Rather , they are airman specialists, for the most part with no more than high school education, who have received training in Air Force technical schools 7 and on the job. Furthermore, the length of the enlistment period of maintenance personnel poses a problem. By the time a mechanic has received the necessary training to qualify him for the most advanced and strategic types of main- tenance work, his enlistment period has expired and he is frequently lost to the service. A rapid and efficient training program is s therefore, required. It would appear, therefore , that research directed toward improving and expediting the selection and training of maintenance personnel constitutes a current need. A complete program of research in this area would involve the determination of the skills and abilities in the untrained population which best predict success in training programs and on the job. It would involve 4 - Eli Saltz and John V» Moore« A Preliminary Investigation of Trouble Shooting , Technical Report 53-2. San Antonio, Tex.: Human Resources Research Center , Lackland Air Force Base, 1953. p. 1. ^Robert Bo Miller. Anticipating Tomorrow' s Maintenance Job , Research Review 53-1. San Antonio, Tex. ; Human Resources Research Center, Lack- land Air Force Base, 1953„ El p. 6 Ibid. ■7 Milton and Cowles, loc cit. _3„ the development of selection test batteries as well as diagnostic proficiency- measures for the determination of areas of strengths and weaknesses in the performance of maintenance tasks. As a logical prerequisite to the latter, it would require the specification of the strategic components of proficiency in the performance of such maintenance tasks. Purposes of This Study Possibly the most crucial task of the maintenance specialist is the isolation of the cause of trouble in a malfunctioning equipment. When an electronic equipment fails to function properly, it is a maintenance mechan- ic's job to locate the specific component which is causing the trouble. He does this by means of a series of electrical checks and measurements. This process is called ''trouble shooting" the equipment. Miller and Folley concluded". . .that the most difficult part of the job is trouble shooting for malfunctioning components."" The general problem attacked in this study was the identification of components of proficiency in the trouble- shooting situation. More specif- ically, the purpose was to attempt experimentally to isolate strategic aspects of the trouble- shooting procesSo An aspect was considered to be "strategic" to the extent that it differentiated maintenance mechanics at different levels of proficiency. It almost goes without saying that isolating strategic aspects of proficiency should be of utmost significance in planning training or in devising pre-training measures for selection or post-training measures of proficiency. The specification of strategic aspects of the maintenance task may, hence, provide hypotheses to guide further research not only on problems similar in nature to the present one, but also on the more general problems of selection and training. It may, furthermore, provide leads for the construction of proficiency tests which are important aspects of the efficient selection, training, and distribution of maintenance personnel. In order to investigate aspects of the trouble- shooting process, a framework by which to view trouble- shooting behaviors was deemed to be necessary. A major requirement of the study was, therefore, to develop a logical orientation to processes of trouble shooting employed by maintenance mechanics The orientation adopted here was based on the conception of trouble shooting as a type oi diagnostic problem- solving task. In order to attempt to specify strategic aspects of the trouble-shooting process, so conceived, a review of selected research on the psychology of problem solving was made. This review formed o Robert B. Miller and John D. Folley, Jr. The Validity of Maintenance Job Requirements from the Prototype of an Electronic Equipment , Part I_: AN/AP Q -24 Radar Set . Pittsburg; American Institute for Research, 1952. p. 50. .4- the basis for the development of a prototype trouble-shooting procedure from which the particular aspects of trouble- shooting behaviors studied were taken. A further purpose of this study was to illustrate a methodology for the determination of educational objectives of training programs. The approach represented by this study is an empirical one and differs from usual methods which are based on logical analyses of the requirements of the tasks for which training is given. If this study is able to provide hypotheses concerning the importance of certain aspects of the trouble- shooting process, it will have laid an empirical foundation for the determination of some aspects of the trouble- shooting task that may- merit special emphasis in the training program. In summary, then, the purposes of this study were (a) to develop a psychological model by which to view trouble- shooting behavior, (b) to test certain hypotheses taken from this model relative to aspects of the trouble- shooting process, and (c) to illustrate an empirical approach to the identification of important objectives of training programs. Review of Research on the Trouble-Shooting Process To the author's knowledge, there have been no published, defin- itive, experimental investigations of the processes involved in trouble shooting electronic equipment. Several studies concerned with problems created by the need for efficient maintenance procedures have, however, yielded information, concerning trouble shooting, which assisted in the development of the present study. It is pertinent that the nature of these contributions be noted. o Miller summarized the development and preliminary tryout of a methodology for the identification of maintenance job requirements for electronic equipment prior to their introduction into the field. The primary contribution of that study relates to the feasibility of training mechanics to maintain an equipment prior to the time it is put into general use. In that investigation two different methods or techniques of trouble shooting were identified^ These were called "trouble shooting from probability data" and "trouble shooting by logical elim- ination." These two techniques were more fully described in another report 10 which resulted from that investigation. A further discussion of the logical elimination technique was made by Miller, Folley, and Smitho - The two trouble- shooting techniques in these reports are "Miller s op. cit. Miller and Folley „ op. cit. 11. Robert Bo Miller, John D. Folley, jr., and Philip R. Smith. Systemat - ic Trouble Shooting and the Half - Split Technique , Technical Report 53-21, San Antonio j Tex.: Human Resources Research. Center, Lackland Air Force Base, 1*953. 16 p. -5- described in the following chapter where their bearing on the present study- is also indicated. Evans and Smith reported on the development and administration of a performance test and other types of tests for radar maintenance technicians. Although that study was not directly concerned with the trouble- shooting process, in an early phase of the study a performance test was administered to fourteen radio mechanics. A complete analysis of the observed trouble-shooting behaviors was not carried out. It was noted, however , that: During this limited period of observation it appeared that a small percentage of the subjects exhibited marked symptoms of perseveration. They continued to make tests in stages of the equipment which could logically have been eliminated from consideration on the basis of information previously obtained. 13 This observation appeared to be indicative of the desirability of adopting a problem-solving orientation to the trouble- shooting process. One of the aspects of the trouble- shooting process observed in the present study was called "perseveration. " The larger project of which the present investigation is a part has resulted in the development of a battery of tests for the measurement of proficiency of mechanics in maintaining a particular piece of radar equipment. One of the chief contributions of the larger project has been the development of a methodology for the measurement of trouble- shooting * ^Rupert N. Evans and Lyman J. Smith. A Study of Performance Measures of Trouble-Shooting Ability on Electronic Equipment . Urbana College of Education, University of Illinois, 1953. 137 p. 13 Ibid. , p. 7-8. Francis G. Cornell and Dora E. Damrin. A Preliminary Report on the Development of a. Functional Knowledge Test Battery for the Meas- urement of Proficiency of Radar Mechanics , Research Bulletin 52-30. San Antonio, Tex.: Human Resources Research Center, Lackland Air Force Base, 1952. 29 p. -6- ability. This technique is called the Tab Test. ^ It is a group testing device which simulates the performance required in trouble shooting an actual, operating, electronic equipment. Each item of this test consists of: 1. A statement of the symptom of the malfunction. 2. A list of checks which might be made in attempting to localize the malfunction. 3. A list of components among which is the defective one, Accompanying each check is an indication of the result the mechanic would receive if he were to perform that check on the actual equipment with the given trouble. This result is covered by a cardboard tab. Accompanying each of the potentially faulty components is the word YES or NO, also covered by a tab. There is only one YES component. It is the one causing the malfunction for the particular item. In taking the Tab Test the subject "performs checks" and "decides upon defective components," by pulling appropriate tabs. He may perform any of the checks available in the item and continues to pull tabs until he reaches the YES component. The results of Tab Test performance consist of the record of the steps taken, the total process involved, in solving the problem presented by each Tab Test item. Although the major purpose of the Tab Test required that these results be used to derive scores indicative of the subjects levels of trouble- shooting proficiency, they were also used in an analytical investigation of aspects of the trouble-shooting process. Some of the results of this investigation are cited in the following chapter. ' il3 For a more complete description of the Tab Test, see: Robert Glaser, Dora E. Damrin, and Floyd M. Gardner. The Tab Item : A Technique for the Me a sure me nt of Proficiency in Diagnostic Problem - Solving Tasks. Urbana: Bureau of Research and Service, University of Illinois, 1952, 13 p. Cornell and Damrin, ^p_. cit. Francis G. Cornell, Dora E. Damrin, and Joe L. Saupe. The AN/APO-24 Radar Mechanics ' Proficiency Testing Study , the Tab Test ; A Group Test Simulation Performance Behavior for the Measurement of Proficiency in Diagnostic Problem-Solving Tasks . Urbana: Bureau of Educational Research, University of Illinois, 1954. 44 p. (Typewritten) 1 / 1 The trouble- shooting situation is described in the following chapter. 1 7 A more complete discussion is contained in: Cornell, Damrin, and Saupe, op cit. It was this type of analysis of Tab Test results which suggested the present study. A study, more directly concerned with aspects of the trouble- shooting process than those discussed above, was reported by Saltz and Moore. This investigation concerned trouble shooting on three types of equipment: radar, reciprocating engines, and remote control turrets. The first part of the study utilized analysis of variance techniques to test hypotheses concerning differences in good and poor trouble shooters with respect to four factors: (a) knowledge of equipment, (b) previous experience, (c) intelligence, and (d) formation of abstract concepts. From each equipment area five good and five poor trouble shooters were selected by supervisors. It was concluded that: 1. Good trouble shooters know more about the functional relations between components than poor trouble shooters. 2. Good trouble shooters have a history of having partic- pated in more different activities of all sorts than poor trouble shooters. 3. There was no demonstrable difference in intelligence between good and poor trouble shooters. 4. There was no demonstrable difference between good and poor trouble shooters in ability to form abstract concepts. ' Another phase of that investigation consisted of interviews with trouble shooters in an effort to discover what they thought were important procedural aspects of trouble shooting. The categories of differences between good and poor trouble shooters which resulted referred to (a) ''logical analysis" or "thinking out'' the problem;, (b) knowledge of the equipment, (c) past experience with the particular malfunction, and (d) ability to use test equipment. The final phase of that study consisted of observations of mechanics trouble shooting the three types of equipment. "In the rather informal observation of trouble- shooting performance, ° seventeen mechanics were observed shooting for three problems in their respective equipment areas. The following eight sources of error were noted.. No statistical analysis of the importance of these sources of error was made. 1. Checking part of the system which is not in the flow of information from which the symptom arises, or ignoring part of the system because it is not noticed that the component is part of the flow of information relevant to the symptom. 1 8 Saltz and Moore, op. cit. 1 9 Ibid. ; p. 9o Z0 Ibid. , p. 2. 2. Avoidance of a difficult-to-make check. 3. A difficult check is made when a simpler one would have sufficed. 4. Needless repetition of a check. 5. After isolating the trouble between two points, further checks are made beyond rather than between the two points; or a check is made between two points despite the fact that no trouble was found between them. 6. A check is omitted in tracing the flow of information. . . 7. The man thinks he remembers information which he does not. 8. Some piece of rote information. . . is not readily- available. * Several of these sources of error were considered in the hypotheses concern- ing efficient trouble- shooting procedures which guided this study. These hypotheses are stated in the following chapter. The design of the present study allowed that these hypotheses concerning differences in trouble- shoot- ing processes employed by good and poor trouble shooters be subjected'to statistical tests. Prospectus The following chapter contains a brief description of the trouble- shooting situation and a discussion of the problem- solving orientation which was adopted as a guide to the observation of trouble- shooting behavior. The nine hypotheses concerning aspects of the trouble- shooting process and taken from this orientation are stated there. Chapter III contains a descrip- tion of the written and performance tests used to gather the data for testing these hypotheses. The sample of radio mechanics and the methods used in administering and scoring these tests are also reported there. The statis- tical techniques used in testing the hypotheses are reported in Chapter IV which also contains the results of these tests. Chapter V contains a discus- sion of the limitations of the study and an analysis of its results in terms of the illustration of an empirical approach to the identification of objectives of maintenance training programs. The final chapter is a summary of the findings and conclusions of the total investigation. 2l Ibid. s p. 5-6. ■9- CHAPTER II THE THEORETICAL ORIENTATION AND STATEMENT OF HYPOTHESES The purpose of this chapter is threefold. It is, first, an attempt to develop the concept of trouble shooting as a type of diagnostic problem- solving process. To this end is discussed the trouble- shooting situation in the framework of a problem- solving model. This problem- solving model has been derived from a selected literature on the psychology of problem solving. The second objective is to develop this problem- solving model, as it applies to the trouble- shooting situation with which this study is concerned. This development of the "prototype" problem- solving process provides the basis for the statement of the specific hypotheses which were tested in this study. The statement of these hypotheses constitutes the final function of the chapter. The Trouble -Shooting Situation In general, there are two duties associated with the occupation of the mechanic charged with the maintenance of electronic equipment. The first of these is the performance of standard and routine checking proce- dures for the purpose of insuring that the equipment is functioning properly. His second function is to locate the source of trouble in any equipment found to be malfunctioning. This latter function is the trouble- shooting one. It has been found to be convenient to conceive of the proficiency required for this maintenance occupation, or the specific duties involved therewith, to exist at different levels. From an analysis of equipment which is either currently in use in the Air Force or which is in the design stage, the following three job types have been developed. Robert B. Miller and John D. Folley s Jr. The Validity of Maintenance Job Analysis from the Prototype of an Electronic- Equipment , Part I_: AN/ APO-24 Radar Set. Pittsburgh; American Institute for Research, 1952. p. 52-7, University of Illinois, Bureau of Educational Research. Research on the Development of a Radar Mechanic's Functional Knowledge Test Battery . Quarterly Report No. 9, Air Force Contract No. 33 (038) - 13236. Urbana University of Illinois, Bureau of Educational Research, 1953. (Mimeographed) p. 5-6. -10- 1. Line checker : Performs all pre-f light, performance, and other standard maintenance checks. If a malfunction is discovered in the course of such checks, the mechanic usually checks plugs, cables, fuses, etc., but goes no further in determining the cause. 2. Line Trouble - Shooter g Has the task of locating a defec- tive sub-unit in a malfunctioning equipment. When this is accomplished he may check the tubes in that sub- unit or look for obvious faults such as broken wires or burned resistors. If the trouble is not of this type, he goes no further. 3. Bench Trouble-Shooter ; Receives defective sub-units (or complete set depending on equipment) and shoots to the particular component that is defective. The primary concern of the present study was with the bench trouble- shooting situation. This emphasis was taken because the radio equipment presently in use in the Air Force is in general not composed of sub-units. Hence, there is presently little need for trouble shooting of the line type in radio maintenance. It should become evident, however, that the isolation of defective sub-units which is performed by the line trouble shooter has basic logical similarities to the isolation of specific defective components within sub-units or complete sets with which the study is concerned. When an equipment is reported to a mechanic as malfunctioning, he knows that one or possible more of the many components which comprise it are defective. When he receives the equipment, he will usually activate it and attempt to reproduce the audible resultant of the trouble, it's "symptom." This symptom may give him information indicative of the nature of the trouble. Then, with the use of equipment designed for this purpose 5 he performs a series of electrical checks and measurements which yield information concerning the operation of specific components, component groups, or entire sections of the set. If this trouble -shooting procedure is successful, he will ultimately succeed in isolating that component which is defective,, The replacement of that component consti- tutes the repair of the equipment. In the report of a study of the validity of a technique of maintenance job analysis, Miller and Folley- 5 discussed two kinds or methods of trouble- shooting procedure. Although that study was concerned with the main- tenance of a particular radar equipment, these two procedures of trouble shooting seem to apply equally well to other types of electronic equipment, including radio. One of these two procedures is called trouble shooting from probability data. This procedure is based on the history of experience with malfunctions of the particular equipment being serviced. It requires 2 u University of Illinois, op. cit. , p. 5. 3 Miller and Folley, op. cit., p. 57-61 -11- that a catalogue of an empirically determined functional relationship between symptoms and corrective actions be developed. For a given symptom are listed the various corrective actions and the probability levels associated with them. The trouble-shooting procedure consists of determining the symptom and then applying the empirically determined most probable corrective action, the second most probable, and so on, until the malfunction is successfully isolated. If the list of alternative corrective actions is exhausted before the trouble is located, the mechanic may turn to the procedure discussed below, or to a relatively trial-and- error procedure. It is logically possible for this procedure to be used by a mechanic experienced with the particular equipment being maintained, even though the alternative corrective actions are not formally catalogued. In this case, the "probability data " would be said to exist in his memory. The second type of procedure is called trouble shooting by logical elimination . The primary requisite for this procedure consists of diagrams or knowledge of the information-flow in the equipment, and of the nature of the contributions of components and groups of components to its functioning. This information is available in the form of block or schematic diagrams of the circuitry of the equipment. Also required are values of the tolerance limits of the various components of the equipment. The procedure consists of systematically eliminating areas or component groups from consideration as containing the defective component. The ultimate conclusion to this process of logical elimination is the isolation of the specific component which is defective. "Systematic" in this case refers to that series of checking procedures which at each step eliminates a maximum proportion of the set from consideration or, in other words, maximizes the probability that the following check will reveal the faulty component. It should be evident that these two procedures need not be employed independently. When a mechanic who is trouble shooting by logical elim- ination draws upon his past experience with malfunctioning radio sets to assist him in making decisions concerning what check procedure to use, it might be said that to some extent he is using probability data from his t 4 » » memory. The orientation of this study focused primary attention on trouble shooting by logical elimination. The following are reasons for this delineation. 1. It was felt that the greater part of the trouble shooting actually done makes but minor use of formal probability data« Reasons for this are that (a) experience with a given equipment has generally not been complete or systematic enough for the required probability data to be fully and satisfactorily catalogued and (b) the training received by maintenance mechanics is generally directed toward the development of the knowledges and mental abilities which prepare for the use of the logical elimination procedure. 2. It was further felt that the general purpose of the present study required that trouble shooting be viewed as a logical process. It was deemed that the likelihood of isolating strategic aspects of the trouble- shooting process and of discovering individual differences in these aspects would be increased were this -12- orientation adopted. The mechanical aspects of the probability data procedure were judged to lessen the likelihood of the occurrence of those behaviors likely to differentiate good and poor trouble shooters. 3. The decision to employ the particular type of performance test in the study structured the experimental situation in such a manner that formal probability data was excluded as a primary guiding factor in the trouble shooting therein involved. This decision will be dis- cussed in the following chapter. It should suffice here to say that the performance test made use of a unique radio set for which explicit probability data could not be made available. 4. Finally, in at least one respect, the trouble- shooting procedure chosen as a basis for this investigation may be superior as a guide for the training of electronics maintenance mechanics. Miller and Folley stated: "It seems reasonable to hypothesize that this train- ing would tend to transfer successfully from one variety of electronic equipment to another. . .""* With the transfer of mechanics from one equipment to another and with the development and utilization of new equipment which periodically replace outdated equipment, the verifi- cation of this hypothesis would be further justification for the present interest in the logical elimination procedure. The Psychology of Problem Solving The orientation of this study conceived of trouble shooting as a type of diagnostic problem-solving situation. The maintenance mechanic is faced with a malfunctioning equipment; his "problem" is to locate the source of the trouble. This corresponds to a representative definition of a problem situation as one in which ". . .the individual is confronted by external condi- tions in which an obstacle or difficulty must be overcome to reach a goal."-' The particular problem- solving orientation adopted as a guide for this study was developed from a review of selected research and writings in the area of problem solving and thinking. The following section is a summary of this review. It presents the background for the development of the prototype trouble- shooting process which follows. This review was not meant to be comprehensive. The references were selected on the basis of how well the type of problem solving discussed could be transferred to the trouble- shooting situation and how likely it was that the aspects of problem solving considered would prove strategic in differentiating among the problem- solving processes of mechanics at different levels of proficiency. This review is organized with respect to five orientations which have been 4 Ibid. , p. 60. 5 William E. Vinacke. The Psychology of Thinking,, New York: McGraw- Hill, 1952 , p. 160. ~ -13- used in the study of problem-solving, (a) methods of attack, (b) components of the process, (c) hypothesis formation, (d) perseveration, and (e) knowledge of facts and principles. Methods of Attack Problem solvers have been differentiated on the basis of general methods of attack or types of response to problem situations.. Alpert" found that the solving processes employed by preschool children in respond- ing to a particular type of problem situation could be classified into four categories^ (a) the primitive response, which was characterized by reach- ing, (b) the random response which was of a trial-and-error nature, (c) exploration and elimination, which was '*. . .a deliberate trying out. . . of one possibility after another as an investigation of the constituent parts of a situation, '' and (d) immediate solution which occurred when no problem appeared to exist for the subject,, Exploration and elimination was found not only to occur more frequently but also to lead to more successful solutions. Heidbreder" classified methods of responding to a problem into two categories which she called participant and spectator behaviors. With participant behavior the subject becomes personally involved in the situation, reports specific hypotheses which guide his overt action, and tends to meet each situation with definite activity; whereas, with spectator behavior he is disassociated from the situation and does not appear to be attempting to try out hypotheses. Durkin' observed three;, not mutually exclusive, "types of solutions' 5 or "forms of thinking" in the solution of simple puzzle problems. These were called trial-and-error, sudden reorganization; and gradual analysis. Relative to the nature of the trial-and-error method, she stated : In fact, T. E. must always imply something more than random activity, for there must be a goal affecting O's behavior. Mere exploratory activity with no particular concept in view could scarcely belong under the concept, since without a goal there could be no error. u "Augusta Alpert. The S olving of Problem S ituations by Preschool Children. Contributions to Education, No. 323. New York; Bureau of Publications. Teachers College, Columbia University, 1928. 69 p. 7 Ibid. , p. 31. o Edna Heidbreder. "An Experimental Study of Thinking." Archives of Psychology 11:73:1-175; 1924. Q Helen E. Durkin. "Trial and Error, Gradual Analysis, and Sudden Reorganization. An Experimental Study of Problem Solving." Archives of Psychology 30:210:1-83; May 1937. 1 Ibid. , p. 9. -14- It was found further that solution by trial-and-error was usually less efficient than solution by analysis. This was evidenced by longer time and larger error scores, and by less transfer to succeeding problems. Bloom and Broder^ in a study which utilized the verbal reports of college students as they solved written test problems found that successful and unsuccessful problem solvers were differentiated on the basis of their general approach to the solution of problems. Aspects of the general approach which were noted were ". . .extent of thought about the problem, care and system in thinking about the problem, and ability to follow through on a process of reasoning. "l^ Relative to extent of thought about the problem, the successful problem solvers could be characterized as active while the unsuccessful appeared passive in their attack. With respect to care and system in thinking about the problem, the successful solvers dealt with problems systematically by abstracting key terms or ideas, breaking it into sub-problems, eliminating alternative solutions as clearly incorrect, and so on; whereas, the unsuccessful solvers plunged into the problem with no apparent plan of attack and without being able to decide what to do next. Furthermore, unsuccessful solvers might begin their ' process of reasoning in a manner similar to that of successful solvers, but be unable to continue the process, and eventually give up. Part of the research which constituted the background for the present study was concerned with the isolation of characteristics of the methods of attack used by radar maintenance mechanics in the solution of trouble- shooting problems simulated by means of a paper-and-pencil testing device. *^ The following ten categories were used for the purpose of classifying the methods of attack used by different subjects. I. Efficiency: The problem is solved in the most logical and . efficient manner. II. Precipitancy: The problem is solved in an efficient manner, except that a logically necessary step is omitted. III. Caution: The problem is solved in a logical manner, except that additional steps are taken in an apparent attempt to insure that the hypothesis is correct. 11 x Benjamin S. Bloom and Lois J. Broder, Problem - Solving Processes of College Students. Chicago: University of Chicago Press, 1950. 109 p. 1 2 Ibid. , p. 28, i0 This device is briefly described in Chapter I. See, also: Francis G. Cornell, Dora E. Damrin, and Joe L.„ Saupe. The AN/APO-24 Radar Mech- anics 1 Proficiency Testing Study , the Tab Test ° _A Group Test Simulating Performance Behavior for the Measurement of Diagnostic Problem - Solving Proficiency . Urbana : Bureau of Educational Research, University of Illinois, 1954. 44 p. (Typewritten) -15- IV. Perseveration: The problem is solved after what appears to be an extended search for a tenable hypothesis. V. Persistence-Reorientation: The first hypothesis, arrived at after an extended search, is incorrect; but the second hypothesis is quickly chosen and correct. VI. Persistence-Tenacity: The first hypothesis, arrived at after an extended search, is incorrect and several alternative hypotheses are tried out in an active fashion before the correct one is hit upon. VII. Persistence-Rejection: The first hypothesis, arrived at after an extended search, is incorrect and the subject apparently "gives up," the only further behavior being of such a nature as to merely complete the mechanical requirements of the task. VIII. Impulsiveness-Reorientation: The first hypothesis, chosen quickly, is incorrect, and the second hypothesis is quickly chosen and correct. IX. Impulsiveness-Tenacity: The first hypothesis, chosen quickly, is incorrect and several alternative hypotheses are tired out in an active fashion before the correct one is hit upon. X. Impulsiveness-Rejection: The first hypothesis, chosen quickly, is incorrect and the subject apparently "gives up," the only further behavior being of such a nature as to merely complete the mechan- ical requirements of the task. It appeared possible to differentiate among subjects with respect to the fre- quency with which their response patterns to the different problems presented in this simulated trouble-shooting situation were classified into these various categories. Components of the Process A second orientation to the study of the processes involved in problem solving is an analysis of the steps or stages which constitute the total process. The five phases of Dewey's complete act of thought* 4 are fre- quently considered to be indicative of a prototype thought or problem- solving process. The following quotation is indicative of the pertinence of his analysis to the purposes of the present study: Take the case of. . .a mechanic inspecting a piece of com- plicated machinery that does not behave properly <> There is something wrong, so much is sure. But how to remedy l4 John Dewey. How We Think . (Revised Edition.) Boston: D. C. Heath and Co., 1933. 301 p. -16- it cannot be told until it is known what is wrong. An untrained person is likely to make a wild guess. . .Or the person fusses, monkeys with the machine, poking here, hammering there on the chance of making the right move. The trained person proceeds in a very different fashion. He observes with unusual care. . . ■* Although Dewey did not regard these five phases as being necessarily discreet in time or function, they may be roughly ordered as follows. First, there is the origin of the process, which lies in some perplexed or confused situation. This is followed by the initial observations which are made at the outset of the thinking process for the purpose of clearly defining the situation or problem with which the individual must deal. These initial observations also lead to ideas or suggestions concerning possible courses of action. Out of these observations there develops a hypothesis which is an idea or suggestion about a possible solution to the problem. The hypothesis is elaborated and tested mentally to see if it is in accord with the facts of the situation. Finally, the concluding phase is an experimental testing by overt action of the proposed hypothesis, If the hypothesis is empirically supported, the problem is solved? oth- erwise, additional hypotheses are developed and tested. The five phases, terminals, or functions of thought that we have noted do not follow one another in a set order. On the contrary, each step in genuine thinking does something to perfect the formation of a suggestion and promote its change into a leading idea or directive hypothesis. It does something to promote the location and definition of the problem/* " Dunker*' contended that the process of solving practical problems is typically composed of several "solution phases" or proposed solutions. These phases are essentially steps in the solution process. Each phase possesses a certain "functional value" or logical reason with respect to how it connects with the desired solution* Certain of these solution phases mediate between the original problem and the final solution. Each phase represents a restructuration of the problem,, The finding of a general property of a solution means each time a reformulation of the original problem. *° 1 5 Ibid, , p. 110. - Ibid. , p. 115. 17 Karl Dunker. "On Problem Solving." (Translated by Lynne S. Lees.) Psychological Monographs 58 £70 ;1-1 13; 1945. Ibid. , p. 8. -17- The final form of a. solution is typically attained by way of mediating phases of the process , of which each one , in retrospect , possesses the character of _a solution , and , in prospect , that of a problem . 19 There may also be a transition in phases back to an earlier and more general level. Every such transition involves a return to an earlier phase of the problem; an earlier task is set anew; a new branching off from an old point in the family tree occurs.^ Such transitions is what is meant by learning by mistakes in the solution process. Heidbreder^- 1 characterized this problem- solving process in terms of, first, a general orientation, second, a set in a general direction, and finally, an organized search. She found that Dewey's "complete act of thought" fitted her findings fairly well, but felt that it should be considered as an ideal and that deviations from it were to be expected in actual thought processes. Burack studied several methods of attack used in reasoning problems. He was concerned with individual differences in the extent of use and the efficacy of these different methods. He isolated the following eight "methods" and found that more than one method could be used in any given problem situation. It will be noted that "methods" one, two, four, and five, as they are labeled, closely parallel steps or phases in the problem- solving process as these have been discussed by other writers. 1. Clear formulation of the problem. 2. Preliminary survey of all aspects of the problem situation. 3. Application of past experience. 4. Analysis into major variables. 5. Localizing crucial aspects of the problem. 19 Fbid.., p. 9. 20 Ibid. , p. 13. 2 1 Heidbreder, op. cit. , p. 107-14. Benjamin Burack. "The Nature and Efficacy of Methods of Attack on Reasoning Problems." Psychological Monographs 64:313:1-26; 1950. Benjamin Burack. "Methodological Aspects of Problem-Solving." Progressive Education 30:134-38; March 1953. -18- 6. Varied trials. 7. Elimination of sources of error. A source of error might be: a. An aspect of the problem situation irrelevant or detrimental to the solution, or b. An error-producing manner of approach. 8. Insight. Hypothesis Formation That aspect of the problem- solving process which Dewey, J and Dunker and Krechevsky^ referred to as hypothesis formation or hypothesis behavior seems to parallel processes which have been given different names by other writers. In summarizing studies on "set" or "direction." Vinacke stated: i The general conclusions to be reached from experiments of these kinds is that the subject's set is an important aspect of problem- solving. Without an appropriate set or direction, solutions cannot be attained except by accident and the ease with which the individual solves the direction for himself varies widely. D This concept of "direction" has been developed primarily by Maier 26 in a series of experiments designed to differentiate between the processes 23~ Dewey, _ojj cit. 24 Karl Dunker and Isadore Krechevsky. "On Solution-Achievement." Psychological Review 46 :176-85; March 1939. 25 Vinacke, op. cit. „ p. 188. Norman R. F, Maier. "Reasoning in Humans : I. On Direction." Journal of Comparative Psychology 10:115-43; April 1930. Norman R. F. Maier. "Reasoning and Learning." Psychological Review 38:332-46; July 1931. Norman R. F. Maier. "An Aspect of Human Reasoning." British Journal of Psychology 24:144-55; October 1933o Norman R. F. Maier. "Reasoning in Rats and Human Beings." Psychological Review 44:365-78; September 1937. Norman R. F. Maier. "The Behavior Mechanisms Concerned with Problem Solving." Psychological Review 47 :43-58; January 1940. Norman R. F. Maier. "Reasoning in Humans: III. The Mechanisms of Equivalent Stimuli and of Reasoning." Journal of Experimental Psychology 35:349-60; October 1945. 19- in learning and those involved in problem solving. The strategic distin- guishing factor for the two processes is "direction." Of it he said: "The particular attempt at solution which is adopted may be called the direction of solution, and most problems may be attacked in several directions." 2 ' Dunker felt that Maier's concept of direction was superfluous, because it represented only one, an early, "phase" or reformulation of the problem in the progress towards its solution. Another parallel to what has been called hypothesis behavior may be the "search model" which has been discussed by both Dunker 2 9 and Johnson. ° The latter refers to the search model as a ". . .pattern of the requirements of the solution as these requirements are understood by the thinker. "31 It narrows the range of the thinker's activity by showing him what to look for and where to look for it. Ruger also mentioned that a strategic aspect of problem solving was found to be ". . .the picking out of the portion of the puzzle to be attacked. "^ Perseveration Within the field of the psychology of problem solving, there has been much interest in the perseveration of false hypotheses. Johnson, 33 in a review of problem solving literature, noted that this behavior has been variously called stereotypy, rigidity, perseveration, inflexibility, and inability to shift. One type of this perseveration may be called experimen- tally induced rigidity. The classical experiment on induced rigidity utilizes the water battle and other similar problems which were introduced by Luchins. 4 Of more direct interest to the present study is the type of persever- ation which is not intentionally induced in the experimental situation. 27 Maier , op. cit. , 1933. p. 144, °Dunker, o_p. cit. Ibid. Donald M. Johnson. "A Modern Account of Problem Solving." Psycho- logical Bulletin 41 :201-29; April 1944. 3 1 Ibid. , p. 210. 32 H. A. Ruger. "The Psychology of Efficiency." Archives of Psychology 2:15:11; June 1910. 33Donald M. Johnson. "Problem Solving and Symbolic Processes." Annual Review of Psychology , Vol. 1. (Edited by Calvin P. Stone.) Stanford Annual Reviews, Inc., 1950 p. 297-310. 34 A. S. Luchins. "Mechanization in Problem Solving — The Effect of Einstellung." Psychological Monographs 54,6:1-95; 1942. -20- Maier-^ discussed persistence, or perseverated "false direction" and said that: Good reasoners do not indefinitely pursue unsuccessful approaches to a problem. They jump from one direction to another in their attempt to overcome one difficulty and then another. Poor reasoners, on the other hand, continue in one direction and may attempt for hours to overcome the impossible. 3 6 In a similar context Alpert^? referred to "fixations" as causes for failure and Ruger38 noted that both (a) variations in the methods used and (b) consciousness of these variations were conditions of efficiency in problem solving. In these discussions of variations in method, flexibility or ability to "break out" of false hypotheses, it is to be noted that by variation or flexibility is not meant trial-and-error behavior or random action, but a conscious redirection of activity to a possibly more fruitful line of attack. Knowledge A final aspect of the psychology of problems solving concerns the relation between knowledge of the facts and principles which are involved in the material of the problem situation and the ability to solve the problem. It is typically contended that the two aspects of problem- solving ability are different but not unrelated. It is true that the first step is always that of securing factual information, since facts represent the basics of problem solving. But although facts are necessary, they are not sufficient conditions for problem solution. The second step in the process involves the relating of ideas; and where relational thinking is absent, there can be little fruitful attack on any problem. 39 35 Maier, op. cit. , 1933. Maier , op. cit. , 1937. 36 Maier , _ojj. cit. , 1933, p. 145. 37 Alpert , o£. cit. JO Ruger, o£. cit. 39Eugene L. Gaier "What the Teacher Needs to Know about the Role of Knowledge in Problem Solving." Progressive Education 30:138; March 1953. ~ -21 Johnson distinguished between the "materials" and the "processes" of thought. The materials or objects of thought are such things as percep- tions , affects, concepts, and the like. The operations on these materials are the processes. Maier41 defined reasoning as the reorganizing or restructuration of past experience. Hence, a body of past experience or a repertoire of responses relevant to the current problem is a necessary- condition to problem solution. The sufficient condition for solution, however, includes not only past experience but also the proper "direction." Other studies pertinent to the question of the relation between knowl- edge and problem-solving success are those by Bedell, 42 Billings, ^ and Bloom and Broder. 44 Bedell gave tests of the ability to recall and the ability to infer in science subject matter areas. He concluded that his tests measured two different abilities. Billings gave tests of information and tests of problem- solving ability in eight subject matter areas. The intercorrelations of his several tests were as would be expected under the hypothesis that problem- solving ability is a general factor. Bloom and Broder found that the chief distinction between successful and nonsuccessful problem solvers with respect to possession of relevant knowledge was their ability to bring this relevant knowledge to bear on the problem. In the area of proficiency measurement of electronics maintenance mechanics, distinction has also been made between what is called basic knowledge, functional knowledge, and trouble- shooting ability .4 5 For measurement purposes, trouble- shooting ability was here sharply distin- guished from basic and functional knowledge, the former being defined by performance tests on operating electronic equipment. Basic knowledge pertained to the performance of standard and routine checking procedures; whereas, functional knowledge was operationally defined as the ability to ". . .utilize previously learned facts and principles to solve novel prob- lems.'^ 40 Johnson, op, cit. : 1945, 41 Maier , op, cit. , 1931. 42 'Ralph C. Bedell. The Relationsh ip betw een the Ability to Recall and the Ability to Infer in Specific Learning Situation s. Kirksville , Mo. : Journal Printing Co.. 1934. 55 p. 4 3 Marion L. Billings. "Problem Solving in Different Fields of Endeavor. American Journal of Psycho logy 46-259-72; April 1934,, 44 Bloom and Broder , op. cit. 4 ^Francis G. Cornell and Dora E. Damrirn A Pr eliminary Report on the Development of a. Functional Knowledge Test Batter y for the Measurement of Proficiency of Radar Mechanics . Research Bulletin 52-30. San Antonio, Tex.: Human Resources Research Center, Eackland Air Force Base, 1952. 29 p. Ibid. , p. 2. -22- The Trouble-Shooting Prototype With the above review of the psychology of problem solving in general as a background, it is now possible to discuss the trouble-shooting situation in terms of a psychological model. The discussion which follows will present the prototype trouble- shooting process, as this process was conceived in the present study. It should not be regarded as the only possible trouble- shooting method or procedure. This model was developed as a prototype, under the hypothesis that it represents the method of attack used by proficient mechanics. The hypotheses concerning differences in the problem- solving processes used by good and poor mechanics and stated in the following section generally concern behaviors which represent either adherence to or deviation from this prototype model. The problem of the mechanic trouble shooting a malfunctioning equipment is to "locate the faulty component." 9 His first few steps consist of a preliminary observation of or orientation to the problem situation. / During this orientation period he turns the equipment on and varies the various controls on the set, the tuning knob, the volume control, etc. It can be said that this phase of preliminary observations is completed when he has reproduced to his satisfaction the symptom of the malfunction. The nature of the symptom may provide the mechanic with information which directs his attention to a particular stage or area of the equipment as potentially containing the faulty component. In the terminology of Dewey, it would be said that he has developed a hypothesis concerning what steps to take next. In Dunker's terms, the problem situation has become reor- ganized or restructured; whereas, at the outset of the process the entire set had been perceived as malfunctioning, now a particular area or stage of the set has become the "figure 5 ' and the remaining portions of the set "ground. 55 Although an "impulsive" or "precipitant" mechanic might at this point begin to check specific components within -what he has hypothesized to be the "trouble area," the more "cautious" and "careful" mechanic utilizes certain checking procedures which are able to more conclusively demonstrate what particular area of the set is malfunctioning. This procedure of making "general checks "47 is also employed by the (proficient) mechanic who has not received enough information from the nature of the symptom to enable him to form even a tentative hypothesis concerning the "trouble area." An example of the use of "general checks" is signal injection. By means of special electronic equipment a signal is injected into a radio receiver at various points along the electronic system which controls the flow of information from the antenna to the speaker. In general, the stage nearest the speaker at which a signal is injected but cannot be heard on the speaker is indicated as being malfunctioning . 23- The nature of the information derived from the several general checks enables the mechanic either to crystallize his original hypothesis or to form a new one concerning the area or stage of the set within which the defective component is located. He has now developed a "search model" which directs him to make "specific checks"48 within that area. The profi- cient mechanic proceeds in a logical manner, by systematically eliminating component combinations from his region of search, until his final hypothesis is that component X is faulty. This component is checked by the appropriate electrical procedure and the hypothesis is confirmed. In Dunker's terminol- ogy, the problem situation has undergone a series of restructurations until finally the specific defective component stands out as "figure" in its functional relationship to the remainder of the set. It is probable that in the process of trouble shooting for some mal- function, a mechanic will form a faulty hypothesis concerning what area of the set is malfunctioning. He will then proceed to make specific checks on components within an area which is actually functioning properly. If he interprets the information which he derives from these procedures correctly, he will reject that hypothesis and attempt to form another. The "careful" mechanic would return to an earlier phase in the process and perform general checks; whereas, the "impulsive" mechanic would move to another area without deriving sufficient information through the use of general checking procedures to logically isolate the faulty component to that area. The division of the electronic equipment into several areas or stages for trouble- shooting purposes is not arbitrary. Electrically, an equipment is composed of several component groups which function as a unit in doing a particular job. However , the division of the trouble- shooting process into the phases, orientation checks, general checks, and specific checks, may be somewhat artificial. Actually, each check procedure made, or each discreet step in the process, can be considered to be a phase, in the sense of Dunker, for each step should logically follow the last previous step and provide information upon which to base the following step. Each bit of information provided, hence, serves to restructure the problem situation. It is seen then, that the proper interpretation and use of the information derived from each check procedure plays a strategic role in correctly isolating the faulty component. Statement of Hypotheses The following hypotheses concerning differences in the trouble- shooting processes employed by electronics maintenance mechanics at differing levels of proficiency were developed from the preceding analysis of the trouble-shooting situation as a problem-solving one. These hypoth- eses are related to three aspects of problem solving: (a) the relation of °Voltage, resistance, or capacities checks on specific components or component combinations within a particular stage are called "specific checks." -24- knowledge of facts and principles to success in problem solving, (b) the relation of specific components of the problem-solving process to success in problem solving, and (c) the relation of over-all methods of attack in trouble shooting to success in problem-solving. For these hypotheses trouble- shooting success or problem solution is defined solely in terms of the location of defective components. Knowledge of Facts and Principles Indeed a problem could not be solved without the background of knowledge pertinent to the content of the problem. However, a primary consideration for the undertaking of this study was that knowledge alone is insufficient to insure that an individual will be proficient in the trouble- shooting situation. 1. Knowledge of basic electronics fundamentals is a necessary though not sufficient condition for success in the solution of ( trouble- shooting problems. In terms of the experiment reported here, this hypothesis was taken to mean that knowledge of basic electronics fundamentals, as measured by a paper-and-pencil test, predicts some, though not all, of the variance in trouble- shooting ability, as measured by the number of problems solved in a performance test of this ability. Components of the Problem-Solving Process The following hypotheses concern deviations from or adherences to the trouble- shooting prototype discussed in the preceding section. It was expected that the behaviors indicated are strategic components of and contributors to the successful trouble- shooting process. 2. Successful mechanics tend to perceive the symptom of a malfunctioning equipment completely and correctly; whereas, unsuccessful mechanics tend to perceive the symptom incorrectly or incompletely* This hypothesis was taken to mean that in the performance test situation successful trouble shooters would tend to be able to report verbally the symptom of a given trouble completely and accurately., while unsuccessful trouble shooters would tend to be unable to do so. 3. Successful mechanics tend to attempt to secure sufficient information before accepting a hypothesis concerning the specific area of the equipment in which the trouble resides; whereas, unsuccessful mechanics frequently accept a hypothesis without -25- attempting to secure such information. This hypothesis relates to what might be labeled "cautiousness" or "careful- ness" in a mechanic's approach to trouble-shooting problems. It was taken to mean, operationally, that the tendency to employ general check procedures immediately following the perception or verbal statement of the symptom differentiates between successful and unsuccessful trouble shooters. 4. The first hypothesis accepted by successful mechanics will tend to be correct; whereas, for unsuccessful mechanics it will tend to be incorrect. In terms of the trouble-shooting model, the proficient mechanic should correctly hypothesize in what area of the equipment is the trouble before he makes specific checks. This hypothesis, then, meant that the first few specific checks made by successful mechanics would be in the "trouble area" more frequently than would those employed by unsuccessful mechanics. 5. Unsuccessful mechanics tend to (a) entertain more incorrect hypotheses, and (b) perseverate incorrect hypotheses for a longer period of time than successful mechanics. In the performance test situation this hypothesis meant that unsuccessful mechanics tend to (a) make checks in properly functioning areas of the equipment more frequently than successful mechanics, and (b) perform more successive checks in a given such area than successful mechanics. 6. Successful mechanics upon obtaining critical information in their checking procedures tend to recognize and use it; whereas, unsuccessful mechanics do not. In the performance test situation the tendency to use obtained information was indicated in the following fashion. Certain critical or "clue" checks were defined as providing pertinent information. The indication of the use of the information which should have been obtained from these "clue" checks was the nature of checks performed following each "clue" check performed. Hence, the hypothesis, in these terms, is that the checks performed after these "clue" checks indicate that successful mechanics utilized the information and that unsuccessful mechanics did not. 7. Successful mechanics tend to make fewer errors in the use of test equipment than unsuccessful mechanics. -26- Operationally, this meant that in using the various equipment provided for making the various checks and measurements which constitute the behavioral process in the performance test situation, successful mechanics make fewer incorrect dial settings and faulty connections than unsuccessful mechanics. Hence, they receive less faulty information because of such errors. 8. Successful mechanics duplicate the same checks less frequently than do unsuccessful mechanics. Duplication of check procedures was taken as an indication of confu- sion, uncertainty, or inability to follow a logical plan of attack. The confused or uncertain mechanic could logically be expected to repeat check procedures during his somewhat "random"' series of steps. Similarly, the mechanics who are unable to operate in the medium of a systematic plan of attack could be expected to derive little meaning from check results obtained in isolation of such an organized plan. They,, hence, could be expected to unknowingly repeat checking procedures. It is not inconceivable, however, that in some situations duplications are indicative of cautious tendencies of trouble shooters. Cautious mechanics may desire to confirm previous check results. Nevertheless , this hypothesis was included on the basis that duplication reflects confusion or lack of a plan of attack to a greater extent than it reflects caution. Methods of Attack The final orientation to problem solving used as a basis for viewing trouble- shooting behavior concerned methods of attack to problems. It was expected that the prototype trouble- shooting procedure, outlined above, would evidence itself in records of actual trouble shooting by maintenance mechanics and that, furthermore, the pattern of response to trouble- shooting problems representing this procedure would be most efficient in their solution. 9. It is possible to differentiate among mechanics on the basis of over-all methods of attack employed. Furthermore, different methods of attack characterize mechanics at different levels of proficiency with the prototype trouble-shooting process being characteristic of the most successful mechanics. It did not appear to be possible to attempt to define on an a priori basis the specific methods of attack expected to occur as alternatives to the prototype trouble- shooting process. The examination of this hypothesis, hence, involved an analysis of records of trouble- shooting processes in an attempt to identify specific alternative methods of attack. This analysis was constantly guided by the concepts used in previous research on this aspect of problem solving, particularly the analysis of patterns of attack -27- used on problems in a setting similar to the present one. ' Summary This chapter has included a discussion of the trouble- shooting situation. A review of selected research and writings in the field of problem solving was presented as background to the development of a prototype trouble- shooting procedure. From this psychological model of the trouble- shooting process were taken nine hypotheses concerning trouble shooting. The following chapter describes the testing instruments and the experimental procedures used for the purpose of providing empirical tests of these hypotheses. 49 Cornell , Damrin, and Saupe , op. cit, -28- CHAPTER III THE INSTRUMENTS USED AND THE EXPERIMENTAL PROCEDURE The method by which the hypotheses concerning trouble- shooting behavior were investigated made use of the records of the trouble- shooting procedures employed by a sample of radio mechanics on a specially designed radio receiver. This testing device is called the Ul/BER-1 Performance Test. It utilizes a specially constructed radio receiver, the Ul/BER-1 (University of Illinois, Bureau of Educational Research, number one). The present study was not directly concerned with the development of this performance testing device or with the construction of the specific radio receiver which it uses.l The present objective included only the utilization of this experimental technique for the purposes set forth in previous chapters. However, since the present study has involved the use of this technique, it is necessary that it be described. Hence, one purpose of this chapter is to describe the Ul/BER-1 Performance Test and to attempt to demonstrate the necessity for its utilization in the present study. Further purposes of this chapter are to describe the written test of basic electronics knowledge and the experimental procedure by which the data for this study was gathered,, The Ul/BER-1 Performance Test The purposes of this study required that maintenance mechanics be observed while trouble shooting actual, operating, electronic equipment. Although the previously discussed Tab Test, which presents subjects with simulated trouble- shooting situations, has proven useful as a stimulation for the present investigation, it was felt that the structured nature of the situations presented by it lessen the likelihood of occurrence of some of the problem- solving behaviors likely to differentiate mechanics at differing levels of trouble- shooting proficiency. What was needed was an even closer approximation to the actual trouble- shooting situation. First considered was the utilization of actual, airborne, radio equipment currently in use by the Air Force for performance testing purposes. This possibility was rejected on the basis of a survey of such equipment and for the following reasons ; The Ul/BER-1 radio receiver and the Ul/BER~1 Performance Test were developed in conjunction with the larger project by Dr. Dora E. Damrin, Dr. Floyd M. Garner, and Mr. Ronald F. Tucker. The devel- opment of the Ul/BER-1 Performance Test included a preliminary administration and subsequent refinement. Only the final form of the technique is reported here. -29- 1. Complexity . The fabrication of such equipment is of such a detailed and compact nature that (a) it would be extremely- difficult for an observer to become familiar enough with any one equipment to enable him to accurately observe and record trouble-shooting behavior, and (b) the time required to isolate any one trouble would be on the average so long that the time required to administer a sample of problems to a sample of subjects would be prohibitive. 2. Operation. Such equipment requires the use of uniquely designed power supplies and test equipment, both of -which are expensive and difficult to obtain. 3. Practicability. Such equipment is cumbersome and requires special installations for its operation. For purposes of performance testing, what was needed was an equipment which could be transported to the mechanics who were to serve as subjects and one which could be used without any special installation requirements. The UI/BER-1 Radio Receiver In order to overcome the above difficulties, the specially designed and constructed UI/BER-1 Radio Receiver was employed for performance testing purposes. The Ul/BER-1 can be described as a conventional superheterodyne receiver. It utilizes one radio frequency (RF) amplifica- tion stage, a frequency converter stage, two intermediate frequency (IF) amplification stages, a detector and automatic volume control (AVC) stage, and two stages of audio frequency (AF) amplification. The special features of this receiver which make it particularly suited for the purposes of this study are the following: 1. It has a special open construction such that components are easily accessible. This type of construction facilitates the observation of subjects' trouble-shooting procedures without interfering with them while they work. 2. Spare parts and test equipment used with it are readily available. 3. No special source of power is required in that it operates from a 110 volt AC line. 4. Its basic circuitry is fairly standard. It consists of basic stages common to most radio equipment currently in use by the Air Force. In other words, the components of the Ul/BER-1 will be found in readily recongizable form in most receivers now in use or now in the process of development by the Air Force. 5. Its circuitry is complex enough and its layout simple enough to allow a variety of troubles to be quickly and easily inserted* -30- For a more complete description of the UI/BER-1 Radio Receiver, the reader is referred to Appendix A which contains the Handbook of Maintenance Instructions for Radio Recei ver Ul/BER-1 . In order to be able to identify the components of the UI/BER-1 as they are mentioned throughout this chapter, it will be necessary to refer to the layout and schematic diagrams which are contained in this handbook. Performance Test Problems In developing a performance test on the Ul/BER-1 receiver, it was necessary to select a sample of problems which would comprise the test. Each problem is established by the insertion of a defective component in the receiver. Each problem is considered solved when the subject specif- ically identifies this defective part. In selecting problems to be included in the performance test the following five criteria served as guides: 1. The problem must be one which can and does happen to normally operating radio receivers. The problem must not be of the / "trick" type which can happen only because the receiver has been deliberately rigged. 2. The problem must have a symptom which is sufficiently obvious and discreet that the subject may become definitely aware that something is wrong with the operating condition of the receiver. Problems with symptoms which are difficult to recognize do occur in practice, but their solution is many times due largely to chanceo Hence, problems of this type were deemed not suitable in a study of problem- solving behavior. 3. The total sample of problems should be such as to cover all major areas of the set. That is, there must be at least one problem in each area so that a comprehensive picture could be obtained of the trouble- shooting behavior of each mechanic. 4. The total sample of problems should cover a range of difficulty from quite easy to relatively hard,, with the majority being at the fifty percent level of difficulty. 5. Each problem should be one which could be easily and quickly inserted into the receiver,, The insertion time on any problem should not exceed approximately two minutes. The eight problems which were ultimately selected to fulfill these five criteria and to comprise the Ul/BER-1 Performance Test, are described below. Included in these descriptions are (a) the component which is faulty and the nature of its defect , (b) the manner in which the test insertion is made, (c) the symptom of the trouble, and (d) a brief rationale for the inclusion of the problem. Problem 1. (a) Troubles Resistor R31 in the power supply is open. (b) Insertion: A nick is filed on the undersides of a 1000 ohm, 20 watt, resistor, thereby breaking the wire and opening the resistor. This component is substituted for R31. 31 (c Symptom: The receiver is completely dead. (d) Problem 2. (a) (b] (dj Problem 3. (a) (b) (c] (dj Problem 4. (a] (b) (c] (d) Problem 5. (a] Rationale: Power supply troubles are fairly common and readily detected. In this type of problem the entire receiver is dead and no DC voltages appear outside of the power supply. Hence, almost any type of measurement should eventually lead to the trouble. Trouble : The heater in tube V4 in the second IF amplifier stage is burned out. Insertion: V4 is replaced with a 6SK7 tube with a burned out heater. Symptom: No stations can be heard except the very strongest, and these are practically inaudible. Further- more , only a small amount of noise can be heard. Rationale: Since tube troubles are the most common ones to occur in practice, one was included in the performance test. The specific one selected gives the most discreet set of clues and symptoms. In this problem the receiver is almost completely dead and the trouble area should be readily located by signal injection. Trouble out. Condenser C20 in the audio section is burned Insertion: A .05 microfarad condenser is opened by pulling out one pigtail, inserting some insulation, and replacing the pigtail. This component is substituted for C20. Symptom: The gain of the receiver is quite low. Rationale : This is the single problem in the audio section. It is a reasonably common type of trouble, but should be more difficult than the previous two. Trouble: The lead to ground of resistor R17 in the cathode circuit is open. Insertion: The lead from the RF gain control to ground is unsoldered. Symptom: The receiver is completely dead. Rationale: This trouble occurs rather commonly. Its symp- toms are distinct and it should be relatively easy to find. Trouble: Condenser C15 in the second IF amplifier stage is shorted. -32- (b (d Problem 6. (a (b (c (d Problem 7. (a (b (c (d Problem 8 C (a (b (c (d Insertion: A .02 microfarad condenser is shorted by- sticking a straight pin into it and cutting the pin off flush with the case. This component is substituted for C15. Symptom; The gain of the receiver is quite low and some distortion is heard. Rationale : This problem should be at a somewhat higher level of difficulty. The actual behavior of the receiver is not what might be predicted from a knowledge of the circuitry. Hence, the problem should require a maximum of searching behavior. Trouble: Condenser C9 in the RF and IF sections is open. Insertion: A .01 microfarad condenser is opened by pulling out one pigtail, inserting some insulation, and replacing the pigtail. This component is then substituted for C9. Symptom: A whistle is heard on all stations except when the RF gain control is set very low. Rationale : This type of trouble is a relatively common one. The clues and symptoms which it presents may not be recognized by the less proficient mechanics. Hence , it should have a relatively high level of difficulty. Trouble open. Resistor R9 in the RF and Mixer section is Insertion: R9, an 1800 ohm resistor, is replaced by an 18 megohm resistor of the same physical appearance. Symptom: Noise is heard but no stations can be obtained. Rationale: In this problem the power supply is completely removed from one area of the receiver. Hence, the trouble should be readily localized and the problem should be a relatively easy one. Trouble: Condenser C12 in the IF section is open. Insertion: A .05 microfarad condenser is opened as in problem three and is substituted for CI 2. Symptom: Motor boating and howling occur unless the RF gain is set very low. Rationale: This problem should be the most difficult of the entire group. Its symptoms are peculiar and mechanics may fail to connect them with any particular section of the receiver. -33- Recording Trouble-Shooting Behavior When testing a mechanic with the Ul/BER-1 Performance Test, the examiner inserts one of the above "troubles" into the receiver by unsol- dering a functioning component and soldering in a defective one. This process, due to the layout of the receiver, takes two or three minutes. The mechanic then begins making checks and measurements on the receiver in his attempt to locate this trouble. While the mechanic is working the examiner stands well to one side of him. He is thus able to see and record all of the overt steps in the mechanic s behavior while not interfering in any manner with what the mechanic is doing. The examiner records these steps taken by the mechanic in sequential order on the Observation Record which is included here as Figure 1. This record sheet was designed such that observable and relevant overt behaviors which the mechanic performs on the receiver during the trouble- shooting process can be quickly and accurately recorded. When completely filled out, the Observation Record contains information concerning (a) the subject's statement of the symptom of the trouble, (b) the type of checks and measurements performed by the subject, (c) the order in which they occurred, (d) whether or not they were performed correctly, and (e) the approximate amount of time spent by the subject in the process of performing such checks. The general method of recording behavior on the Observation Record is to enter a number, and possibly a short descriptive phrase, in an appropriate space whenever the subject performs a relevant action. The number represents the order in which the action is performed. In a great many instances a number alone is sufficient. Specifically, the record is filled out as follows. The examiner first records the following identifying informations (a) the subject's name, (b) the examiner's name, (c) the place where the testing is being done, (d) the date, and (e) the problem number. He then records the time at which the subject makes his first move toward trouble shooting. After the subject has been working for the first five minutes, the number of the step he has just completed is recorded in the appropriate space at the top of the sheet. This procedure is followed again after ten and fifteen minutes, unless the subject locates the trouble before these time periods elapse. There is a twenty- minute time limit for each problem. If the problem is solved within this time limit, the time at which he "passes" is entered. Otherwise, the time at the end of the time limit is entered in the space marked "Fail." The entries made during the trouble- shooting process which always require some description are those labeled ''Symptom," "Resistance," 'Tube," "Condenser," and those labeled "Misc." These are always entered in the form of a number representing the order of the action and a brief description. Under "Symptom ' is entered the number representing the step at which the subject stated the nature of the symptom, and his exact statement. Under "Resistance" is entered the number of the step at which the resistance measurement is made and an indication of the resistance measured. In the 'Tube'' space is indicated whether the tube is touched or substituted. The space labeled "Condenser" is used to record checks made by shunting a suspect capaciter with a good one; the step number and the designation of the capaciter shunted are entered. -34- o ft •»■< u -t-> OJ o a> 'a ■!T> i — i — • Q > o id 01 02 3 O (U 00 id 2 ! a) H id a o U el id +-> in 3, o !| U3 CO 2 CO 1 •t-« 2 CO a; oo id i—i > 0) id i— i ft 8 rt k •i-i 2 i— i > ft -t-> O .2 rt •— t fM Q O H U -35- The remaining spaces of the Observation Record are sometimes entered with just a number and sometimes require some explanation. If the subject performs two operations simultaneously, for example, makes a voltage measurement and varies a control, the same number is entered in the two appropriate spaces and is circled in each. If a signal is injected by touching a component and not with the signal generator provided, this is indicated. If the subject makes an error in performing a check, this is also indicated on the Observation R ecord . An error may be an erroneous use of the test equipment or the performance of a nonsense action. If the subject immediately recognizes his error and corrects it, it is not recorded. Any other pertinent actions which the subject performs and any pertinent remarks concerning the subject s behavior are recorded in the space labeled "Misc. and Remarks," Information thus recorded on Observation Records provided the basic data for the examination of the hypotheses concerning differences in the problem-solving processes employed by radio maintenance mechanics. Interpretation of Trouble-Shooting Behavior The information contained in the completed Observation Record is in such a form that it is not easily amenable to analytical abstraction and analysis. Hence, it is transcribed to the Analysis Record Sheet which is presented here as Figure 2. This sheet contains several columns and space for identifying information. When completed it yields an abstracted record of the overt behaviors of the subject in the sequential order in which these behaviors were observed to occur. It, hence, provides an easily read protocol of the subjects' behavior. It is filled out as follows. At the top of this sheet is recorded (a) the subject's name, (b) the base at which the test was administered, (c) the date, (d) the problem number, and (e) the examiner's name. If the subject passed the problem, located the defective component, within the twenty- minute time limit, the total elapsed time is recorded beside "No. Min. Pass." If the subject failed the problem, the total elapsed time is recorded beside "No. Min. Fail.' - ' Normally, this is twenty minutes, unless the subject gave up and refused to continue working before the time limit had expired. In column (l), Step, is the sequence number of the described check. In column (2), Type, is entered a symbol which identifies the kind of check that was performed. The symbols which are used are as follows: SI - signal injection using signal generator SI-T - signal injection by touching part R - resistance check E - voltage check C - condenser shunted S - tube substitution -36- ANALYSIS RECORD SHEET Name Base Problem Examiner No. Min. Pass No. Min. Fail Date (1) Step (2) Type (3) Location (4) Error (5) Dup. (6) Clue (7) Area (8) Interp. (9) Comments / Figure 2. The Analysis Record Sheet Used to Record Interpretation of Trouble-Shooting Behavior. -37- F - tube felt V - volume control varied T - tune control varied G - gain control varied Column (3), Location, refers to the location of the check in the receiver. When a check was made on a tube, the tube symbol used on the Observation Record is entered and is followed by a letter indicating the exact point of the check, as follows: P - plate K - cathode G - grid S - screen The locations of other checks are identified by the symbols for the compo- ents at which the checks were performed. In this column opposite the appropriate step number is also noted the subject's statement of the symptom. The exact statement may be entered at the bottom of the sheet. Any additional remarks or miscellaneous checks performed by the subject are noted here and may be recorded at the bottom of the sheet. Column (4), Error, is entered with an X for each check that has been performed improperly. If necessary, the type of error that was made is briefly described under column (9), Comments. The remaining columns of the Analysis Record Sheet are filled in by abstrating pertinent information from the previous columns. They, thus, contain information which is not explicitly recorded on the Observation Record . For the present study they comprised the systematic interpretation of aspects of the subject's trouble- shooting process and provided the final form of the protocol which was subjected to analysis for the purposes of testing hypotheses concerning problem- solving process variables. Column (5), Duplication, is used to identify all checks which are identical to previously performed checks. Entries are made in terms of the step number of the check repeated. Although a particular measurement might be made by two physically different check procedures, only checks which are repeated in a strictly physical sense are considered to be duplications and are recorded. Column (6), Clues, is entered whenever the check which the mechanic has performed provides information which should clearly assist in isolating the area of the set which contains the defective component or in isolating the specific component itself. The checks which are considered to constitute "clues" are specifically defined for each problem on an a priori basis. Appendix B contains the list of checks which are so defined. - The se clue checks were determined by both a logical analysis of the operation of -38- the receiver with the trouble inserted and by actually inserting the trouble and performing checks and measurements. "General Clues" in Appendix B are checks which provide specific information which, if properly interpreted, aids in isolating the area of the receiver in which the faulty component lies. "Specific Clues," if properly interpreted, provide information which aids in isolating the faulty component itself. In column (6), whenever a clue check has been performed and succeed- ing checks indicate that the subject had correctly perceived and used the information he obtained, a plus (+) entry is made in the appropriate space. If succeeding checks indicate that he has failed to recognize and use this information, a minus (-) entry is made. If it is not possible to determine whether the clue has been used or not, a "C" is recorded. Column (7), Area, is a record of the instances during the trouble- shooting process at which the subject changed fron one area of the set to another in performing specific checks . A plus (+) is entered if the subject changed from a non-trouble area to the trouble area , The trouble area is defined for each problem. It is that section of the receiver, or group of components, which contains the defective component and is most directly affected by its malfunctioning. Appendix B contains the specifications for the trouble areas as these are defined for the eight performance test problems. A minus (-) is entered in Column (7) if the subject changed from one non-trouble area to another non-trouble area. A double minus (- -) is entered if he changed from the trouble area to any other area of the receiver. Column (8), Interpretation, consists of a final classification of the specific check procedures followed. Several different types of entries are made in this column according to nature of the check performed. These types are as follows: 1. Orientation checks. These are checks which are performed before the statement of the symptom in attempting to discover what the symptom of the trouble is. An 0+ is entered if these checks result in a correct statement of the symptom. An O- is entered if these checks result in an incorrect statement of the symptom. 2» General checks. These are checks which provide information concerning the functioning of relatively broad areas of the set. A G+ is entered if the general checks are followed by specific checks in the trouble area. A G- is entered if they are followed by specific checks in any non-trouble area. 3. Specific checks. These are the first checks on specific components or on relatively small groups of components following either orientation or general checks or following an area change. AnS+ is entered if the check is in the trouble area. An_£U is entered if the check is in any non-trouble area. •39° 4. Persistence checks. These are all checks on specific components or small component groups which are made in succession in a single area of the receiver and following the initial "specific check." A P+ is entered if the check is in the trouble area. A P- is entered if it is in a non-trouble area. 5. Fiddle checks . An F is entered for those steps at which the subject varied the volume, gain, or tune control with what seemed to be no purpose other than to keep himself busy or to pass the time. This entry is not made for orientation checks or when there appears to be some purpose for varying these controls, such as, immediately following a tube substitution. 6. Terminating Behavior. Following the final check made by the subject, an X+ is recorded in this column if he had located the trouble or an X- if he had failed to locate the trouble. For the present study Analysis Record Sheets, thus completed, formed the final record of the subjects' problem- solving process. They were then subjected to the analysis and quantification which is reported in the following chapter. The Measure of Basic Electronics Knowledge In order to provide information concerning the contribution of knowledge of electronics facts and principles to trouble- shooting proficiency and to the various aspects of the trouble- shooting process with which this study is concerned, a test of basic electronics knowledge was used. The Basic Electronics Knowledge Test ^ which was used for these purposes is a four response multiple choice test consisting of 130 items. The test is divided into two parts. Part I, consisting of 80 items, was designed to meas- ure knowledge of the theory and operation of radio receivers in general. Part II, consisting of 50 items, was based on the Ul/BER-1 Radio Receiver and was designed to measure the extent to which mechanics possess the specific knowledges which, if known and correctly used, would enable them to solve the trouble- shooting problems of the UI/BER-1 Performance Test. A logical analysis of this test resulted in the following classifications of items according to the nature of the knowledge and abilities required to respond correctly to them. As is usual in attempting to classify items according to types, it was in some cases difficult to make decisions concerning which of two categories a specific item best represented. Hence, the frequencies as reported below should be considered as but approximations. Dora E. Damrin, Floyd M. Gardner, and Ronald F. Tucker. Basic Electronics Knowledge Test- - Radio Receivers. Urbana : Bureau of Educational Research, University of Illinois, 1953. 22 p. (Mimeographed). -40- Part I of the Basic Electronics Knowledge Test was found to contain forty-four items which were deemed to require only knowledge of specific facts. These were divided among those requiring a knowledge of the terminology used with respect to radio and electronics, knowledge of specific facts concerning radio operation in general, and knowledge of conventional circuits used in radio receivers. Examples of items of this type are : 59. The term ''feedback' means that ; a. a portion of the input signal of an amplifier is applied to the output. b. a portion of the output signal of an amplifier is applied back to the input. c. the grid voltage and plate voltage are 180 out of phase. d. the grid voltage and plate voltage are in phase. 31. The purpose of AVC is a superhet receiver is to : , a. prevent local oscillator drift. b. prevent detuning of IF stages. c. provide nearly constant intermediate frequency on all stations. d. provide nearly constant output level on all stations. 29. R-C coupling is generally used in: a. audio stages* b. RF stageso c. IF stages- d. mixer stageso Also contained in Part I were five items which were felt to require knowledge of specific techniques used in performing checks and meas- urements required in maintaining radio equipment. For example : 28. Plate and screen grid voltage may be measured" a. between plate and screen terminals. b. from plate or screen to cathode. c. across the plate load or screen dropping resistor. d. from plate or screen to B+. Six items in Part I v^ere found to involve the ability to interpret or translate information concerning typical electronic circuitry. These items generally required the ability to read and interpret some aspect of a small schematic diagram of an electronic circuitry. In part I were sixteen items which required the application of electronics principles. For example : 46. A 24 volt circuit through which a 3 ampere current is flowing has a total resistance of : -41- a. 3 ohms. b. 8 ohms. c. 24 ohms. d. 72 ohms. The remaining nine items in Part I were deemed to require a higher level of mental ability, that to analyze the relationships in a circuit or to synthesize information presented in order to reach a conclusion concerning some aspect of the operation of an electronic circuitry. These items were typically presented with a small schematic diagram from which the informa- tion was to be abstracted. Part II of this test is oriented specifically to the Ul/BER-1 Radio Receiver and was found to contain but one of the knowledge of specific facts type of items. It contains eleven items of the knowledge of specific techniques types. These items ask what procedures should be used in checking components of the Ul/BER-1. There were found to be eleven items which belonged in the translation and interpretation category. These items generally required that the subject be able to read the schematic diagram of the Ul/BER-1 contained in the Handbook for this receiver. Five items in Part II were classified as requiring the application of electronics principles relative to aspects of the Ul/BER-1. The remaining twenty-two items were felt to require the more intensive analysis and synthesis of the information which they contained to be responded to correctly. These items frequently dealt with probable causes of some type of malfunction in the Ul/BER-1. For example ; 104. The set is dead. The d-c voltage across C24 is zero and across C23 is 430 volts. The most likely cause of this trouble is that; a. the fuse is blown. b. filter choke LI is open circuited,, Co R31 is short circuited,, d. C24 is short circuited. This classification of items according to a logical analysis of the types of knowledges and abilities which they require is summarized in Table 1. The Experimental Procedure The Sample The Ul/BER-1 Performance Test and the Basic Electronics Knowledge Test were administered to forty radio mechanics at an Air Force radio maintenance school. These mechanics were nearing the completion of a twenty-two week training program in basic and applied electronics fundamentals. It will be quickly noticed that this "sample" of subjects was not a random sample from a well defined population in any strict sense of sampling procedure. It is, at best, a sample of opportunity and was selected -42- in such a manner that a given subject would be included merely on the basis of his availability for the testing program. It is felt., however, that the sample was not selected with any deliberate bias in mind ; such as might have occurred if all of the most proficient or all of the least proficient mechanics had been included. Hence, for the purposes of the statistical analyses which are presented in the following chapter and do assume random sampling, this sample or 40 mechanics will be considered as constituting a random sample from a conceptual population in the sense of Walker and Lev. 3 The present conceptual or hypothetical population consists of all radio maintenance mechanics who have had a similar training and background as those of the sample. In other words, it consists of that population from which the present sample could logically be considered to have been randomly drawn. Table 1 Classification of Items Comprising the Basic Electronics Knowledge Test Category Number of Items in Part I Part II Total 44 1 45 5 11 16 6 11 17 16 5 21 9 22 31 Specific Knowledge Knowledge of Technique Translation and Interpretation Application Analysis and Synthesis Total 80 50 130 Administration of the Tests The Ul/BER-1 Performance Test and the Basic Electronics Knowledge Test were administered to the forty mechanics during a seven week period. The time required to take the performance test was such that but two subjects could be tested each day. The examiners were two Air Force electronics technicians. These men had received training by the individuals who had developed these instruments. This training consisted of an intensive study of the Ul/BER-1 receiver and familiarization with the procedures used in administering the performance test and in recording the behaviors of subjects. They also received instruction on the techniques to be used in the group administration of the Basic Electronics Knowledge Test. Helen M. Walker and Joseph Lev, Henry Holt, 1953. p. 9-10. Statistical Inference. New York: -43- The individual testing of subjects on the Ul/BER-1 consisted of three phases. The mechanic was first brought into the testing room which contained a large table. On this table was the equipment used in the performance test. This was (a) the Ul/BER-1 radio receiver and speaker, (b) an enlarged schematic diagram of the Ul/BER-1 , (c) a Heathkit signal generator, (d) a multimeter, (e) miscellaneous tools, including soldering iron, pliers, screwdrivers, test leads, etc., and (f) an assortment of spare parts. During the introductory phase of the experiment the examiner introduced himself and explained completely the purpose of the test. He then made a brief statement concerning the nature and the few unique features of the Ul/BER-1. During the second phase of the testing procedure the subject familiarized himself with the Ul/BER-1 by following a standardized proce- dure contained on an instruction sheet. This Familiarization Procedure sheet is included in Appendix B. The first two parts of this procedure were designed to acquaint the subject with the overall performance of the Ul/BER-1. Part III served the dual purpose of providing the subject with reference information on the operating values of pertinent voltages useful in trouble shooting and of giving him an opportunity to identify and become familiar with the location and wiring of the various circuit elements. Part IV gave him some familiarity with the particular signal generator used and more detailed reference information on the operating charac- teristics of the Ul/BER-1. Throughout this familiarization procedure the receiver was in operating condition. The time required for this phase was approximately one-half hour. The final phase was the performance test proper. Before testing began, the examiner explained that in the subject's absence he would put a trouble into the receiver and that it was the subject's job to locate the trouble. He indicated that the subject should first identify the symptom for the problem and report it as specifically as possible to the examiner. It was explained that there would be a twenty-minute time limit for each problem and that some problems were purposely quite difficult. Hence, the subject would not be expected to solve all the problems. The examiner also explained, both before and after testing, that there were not enough test troubles so that each subject could be given a different set. The cooperation of the subject was solicited in not talking to fellow mechanics concerning the test until the end of the entire testing period. Immediately following the testing of each subject, the examiners transferred the record of the subject's trouble- shooting procedures from the Observation Records to the first four columns of the Analysis Record Sheets. The Analysis Record Sheets were then forwarded to the University of Illinois where electronics technicians filled out the remaining columns in the manner outlined above. Regarding the objectivity of the observation procedure, it should be mentioned that during the early part of the testing period both examiners recorded the behavior of each subject. This was abandoned when it was found that the correspondence between the two protocols thus obtained was nearly perfect. -44- The interpretation of behavior required in filling out columns (5) through (8) of the Analysis Record Sheet was made by three members of the technical staff of the project from which this study received its departure. This resulted in three independently completed copies of each of the 320, eight items and forty subjects, Analysis Record Sheets . For the purposes of (a) preparing a single "final" set of the Analysis Record Sheets and (b) making some estimate of the objectivity with which the interpretations were made, the author and two of the above three electronics technicians re-examined these records. Table 2 presents the proportions of entries in columns (6) and (8) for which there was complete agreement among the three sets of records with respect to interpretation. The entries in column (5) were made in such an automatic fashion that no attempt was made to determine the extent of correspondence among the three sets of records. Column (7) represents, essentially, an abstraction of some of the information contained in column (8). Hence, any errors there also appeared in column (8) and were not noted here. , Table 2 Proportions of Agreement among Three Independent Raters for Interpretations Required in Completing Columns (6) and (8) of the Analysis Record Sheets Problem 1 2 3 4 5 6 7 8 Total .87 .87 The proportions of agreement here reported should be interpreted with care. They probably overestimate the subjective "goodness" of agreement for the three sets of records. This is due to the fact that for many of the checks performed by subjects, the interpretation was unevocable; most checks are obviously not "clues." However, during these examinations of the record sheets, the specifications and directions for their completion were clarified in several respects. It is felt that if the present specifications, as reported in this chapter, were used by two individuals to -45- Column 6 Column 8 Clues Interpretation .81 .91 .86 .90 .93 .89 .85 .93 .89 .87 .90 .71 .80 .95 .88 .80 complete these record sheets, the agreement would be near perfect. The "final" set of 320 Analysis Record Sheets which was used in this study of trouble-shooting behavior resulted from these examinations. Approximately two weeks after a subject had taken the Ul/BER-1 Performance Test he was called back to take the written test of basic electronics knowledge. This test was administered under group testing conditions. Responses were recorded on IBM answer sheets which were scored both by machine and by hand. The subjects were instructed to mark a response for each question in the test, even though this required a calculated guess at best in some cases. Testing time was approximately two and one-half hours, but every subject was allowed to complete the test even though more time than this was required. Part II of the test required that he refer to the Handbook of Maintenance Instructions for Radio Receiver Ul/BER-1 . This Handbook was made available for this part of the test. The testing instruments described in this chapter and the above experimental procedures followed in their administration constituted the groundwork for testing the hypotheses concerning aspects of the problem- solving behavior of radio trouble shooters. The following chapter is devoted to the statistical analysis of these hypotheses and to other pertinent results of the experiment. -46- CHAPTER IV RESULTS The testing of the hypotheses concerning aspects of the trouble- shooting process and stated in Chapter II constituted a major purpose of this study. The present chapter reports the methods used in testing these hypotheses and the results of these tests. Also presented here are some gross results which provide general and descriptive information concerning the Ul/BER-1 Performance Test and the Basic Electronics Knowledge Test. Gross Results Prior to the statistical tests of the specific hypotheses of this study, it was deemed desirable to identify some descriptive characteristics of the instruments used. The following results are introductory in nature to the remainder of the chapter. Table 3 shows the frequency distribution of the number of perform- ance test problems passed for the forty subjects. It will be recalled that a subject was credited with passing a problem if he correctly identified the defective component. This number of problems passed will hereafter be referred to as the subject's pass-fail score . Table 3 Frequency Distribution of Performance Test Pass-Fail Scores Score 8 2 7 4 6 5 5 13 4 4 3 5 2 2 1 5 n = 40 ■47- Table 4 shows the three frequency distributions which resulted from the test of basic electronics knowledge. The first of these is for Part I of this test which is composed of 80 items whose content is general radio and electronics facts and principles. The second distribution in Table 4 is based on the 49 1 items of Part II of this test. These items were designed to measure knowledge and abilities particularly relevant to the Ul/BER-1 Radio Receiver. The final distribution is that of the total score of this test. A subject's score for this test was the number of items to which he responded correctly. Table 4 Frequency Distributions Knowledge Ti of Basic EL ectronics est Scores Part I Part II Total Score f Score f Score f 75-79 1 44-47 6 110-119 6 - 70-74 4 40-43 4 100-109 7 65-69 5 36-39 8 90-99 6 60-64 7 32-35 3 80-80 10 55-59 8 28-31 7 70-79 8 50-54 8 24-27 5 60-69 2 45-49 3 20-23 4 50-59 1 40-44 4 16-19 3 n = 40 n = 40 n - = 40 The means, variances, and standard deviations of these distributions are shown in Table 5. Also presented in this table are reliabilities of the respective scores. These reliabilities, r^ , were determined by means of Kuder-Richardson Formula (20). They are interpreted as indications of the internal consistencies of the several measures. The final entries in Table 5 are the product-moment intercorrelations among the several measures. One item in this part was found to be faulty and was, hence, not scored. G. F. Kuder and M. W. Richardson. 'The Theory of Estimation of Test Reliability." Psychometrika 2:151-60; September 1937. -48- Table 5 Means, Variances, Standard Deviations, Reliabilities, and Intercorrelations of Performance Test Pass-Fail Scores and Basic Electronics Knowledge Test Scores ■tt .51 .55 4.48 3.75 1.94 .63 .79 __a 58.08 78.02 8.83 .8$ __a 32.55 69.80 8.35 .89 90.63 264.88 16.28 .93 Correlations — ? Measure X s BK-1 BK-II BK- Total Performance Test .53 Basic Knowledge, I Basic Knowledge, II Basic Knowledge, Total l Part with total test correlations were not determined. Several conclusions emerged from this descriptive information concerning the testing instruments. It is pertinent that they be noted here. 1. Although the Ul/BER-1 Performance Test was designed and employed in this study primarily as an experimental device, it displayed characteristics of a satisfactory measure of proficiency in terms of the number of problems solved. The nearly optimal range of problems pass.ed and the relatively high coefficient of internal consistency indicate that the test was capable of differen- tiating among the 40 subjects of the present sample. A Kuder- Richardson Formula (20) reliability of .63 for a test of but eight items is usually considered to be highly satisfactory. ■* 2. The Basic Electronics Knowledge Test was relatively easy for the present sample. The mean item difficulty for the total test was .70, for Part I, .73, and for Part II, .66. The coefficient of internal consistency, .93, for the total score on this test is considered to be satisfactory. Furthermore, the logical analysis of the items of this test, discussed in Chapter III, is offered as evidence for the logical validity of the test as a measure of knowledge of basic electronics facts and principles. It is concluded, therefore, that the test has provided a satisfactory measure of basic electronics knowledge. 3 The following section will indicate that it is also highly significant in the statistical sense. -49- Although the purposes of this study did not include an investigation of the components of basic electronics knowledge, it was anticipated that Part II of the basic knowledge measure would prove to be more relevant to the performance measure than would Part I. This, in fact, did not happen to be the case as evidenced by the coefficients of correlation reported in Table 5. The respective parts of the basic knowledge test predicted approximately the same proportion of performance test variance. Furthermore, the correlation between Part I and Part II of the basic knowledge test closely approximates their reliabilities. On the basis of this evidence, it was decided that the two part test scores would be combined and that this total score on the Basic Electronics Knowledge Test would constitute the measure of knowledge of electronics facts and principles to be used in this study. This coefficient of correlation, .55, between the basic knowledge measure and the performance measure is of interest because it provides preliminary evidence with respect to the first hypothesis of this study. It suggests, as was hypothesized, that basic knowl- edge predicts some, but nqt all, of the variance due to individual differences in trouble-shooting proficiency. Even when corrected for lack of internal consistency in the two measures, the coeffi- cient becomes only .72, indicating that fifty percent of the performance variance is predicted by the basic knowledge measure when this source of error is taken into consideration.'* This hypothesis will receive more detailed examination in later portions of this chapter. Statistical Tests Employed The manner in which the hypotheses concerning trouble- shooting behavior were tested made use of several statistical tests. Several of these tests were used in conjunction with more than one hypothesis. Hence, in order to prevent repetition these tests are briefly described as an introduction to their applications. A five percent (.05) critical region was chosen as the basis of rejection for each hypothesis, but statistics falling in one percent (.01) critical regions are also reported. 4 . , The present application of the correction for attenuation is not entirely appropriate in terms of the experimental situation. For example, the form of the basic knowledge test probably creates a source of "true" variance which may be irrelevant to performance, and the internal consistency reliability does not correct for "error" variance which may appear in the original correlation as a result of the differential infesting times. However, the application is considered to be satisfactory for the present introductory purpose. -50- Fisher's "exact test" was used to test hypotheses concerning dependence in two- by-two contingency tables with small cell frequencies. The specific hypothesis which this tool tests is that there is independence in the two systems of classifications. In using it, it is assumed that the marginal frequencies are fixed. Each of the hypotheses for which this test was employed in the present study specified a direction in which the dependence was expected to appear. Hence in the computation of the exact probabilities only that table which occurred and all those more favorable to the hypothesis were included. That is, the tests were all "one-tailed." 6 Another type of hypothesis which was tested is that a given method of scoring performance test behavior is capable of revealing individual differences among subjects on that aspect of behavior involved. The statistical hypothesis which is tested in this case is that ° there are no differences among the means of subjects relative to that aspect of behavior being scored. Hypotheses of this type were tested by means of an analysis of variance suggested by HoytcJ This method assumes that a subject's / score on an item stems from four independent sources. These are (a) a factor common to all individuals and all items, (b) a factor associated with the item, (c) a factor associated with the individual, and (d) the error component. It further assumes that the error components and the means of the individuals are normally distributed. The statistical criterion for the above type of hypothesis is an F-test. The null hypothesis is rejected if Variance among subjects F — ~- ~~~ — — — — — — — — — — ■ Error Variance is greater than some specified value. Because the computation involved in the complete analysis of variance is tedious and because the test was to be repeatedly applied, a simpler criterion was sought. It was found that by using the relation, , • Error Variance r tt - 1 - ~ : — ~~~- ■ ' Variance among subjects a transformation could be made. That is, 5 Ronald A. Fisher. Statistical Methods for Research Workers. (Seventh edition.) Edinburgh" Oliver and Boyd, 1938. p. 100-102. The usually tedious computations of exact probabilities were simplified by the use of: National Bureau of Standards. Tables of the Binomial Probability Distribution, Applied Mathematics Series, No. 6. Washington, D. C: United States Government Printing Office, 1949. 387 p. 7 Cyril Hoyt. "Test Reliability Estimated by Analysis of Variance." Psychometrika 6 :153-60; June 1941. -51- tt 1 = 1 — F and the critical region for F can be described in terms of r.. . For the cases considered in this study F has 39 and 273 degrees of freedom. Hence, the critical region for the hypothesis of no differences among subjects becomes r> .31 for a test at the five percent level and r> .41 for the one percent level. The final simplification employed was the use of the general formula suggested by Cronbach° for the computation of r^. Two types of hypotheses involving correlation were tested. One of these types was tested by the product-moment coefficient. The other required the partial correlation of two variables with the effects of the third held constant. The statistical hypothesis for the former type was that the correlation between two variables is less than or equal to zero. Hence, a large positive coefficient would be required for rejection. That is, the test was "one-tailed." The statistical hypothesis for the latter type stated that the partial correlation coefficient is less than or equal to zero. This also called for a "one-tailed" test and the hypothesis was rejected for large values of the partial coefficient. For both of these types of hypotheses the t-test was used. The formula for t for testing correlations is t = vTT 2 where v is the appropriate degrees of freedom. Again, because these tests were to be applied several times, a transformation was made and the test criterion became r - ■J7 + v The degrees of freedom for the product-moment coefficients were 38 and for the partial coefficient, 37. The rejection region for the former became , then, r > .26 at the five percent level and r>.37 for a one percent test. For the latter, the respective values were .27 and .37. In some cases the hypothesis of relationship could not be tested by using the product-moment correlation coefficient, because the assumption that the two distributions were from a normal bivariate distribution was untenable. In these instances, the data were categorized in a two-by- three contingency table and chi-square with two degrees of freedom was used for the test. The relevant critical values of chi-square are presented in Table 6 which also summarizes the other statistical tests which have been discussed. 8 Lee J. Cronbach. "Coefficient Alpha and the Internal Structure of Tests." Psychometrika 16:297-334; September 1951. -52- Table 6 Critical Values for Statistics Employed in This Study Stat Lstic Degrees freedo: of m Level .05 .01 r tt 39 and 273 .31 .41 r xy r xy. Chi- 38 .26 .37 z ■square 37 2 .27 5.99 .37 9.21 Hypothesis Concerning Knowledge of Facts and Principles The first specific hypothesis of this study was: Knowledge of basic electronics fundamentals is a necessary though not sufficient condition for success in the solution of trouble - shooting problems . The indication of "success" in solving trouble- shooting problems used in this study was the pass-fail score on the performance test. An alternative or supplementary definition of proficiency in trouble- shooting might include speed in locating defective components. Speed is certainly a requisite for proficiency in some situations. It was felt that the twenty- minute time limit imposed for each problem in the performance test made the pass-fail score somewhat dependent on the subject's speed. This supposition and the fact that the pass-fail scores were readily interpretable led to the decision to use it as the criterion of success. The total score on the written test was taken as the measure of "knowledge of basic electronics fundamentals." The correlation between these two measures, .55, has already been discussed with regard to this hypothesis. It is necessary only to add that this coefficient is statistically significant at the one percent level. This fact supports the "necessary" aspect of the hypothesis. It does not, however, clarify the "not sufficient" part. This latter aspect was attacked in conjunction with hypotheses 2 through 8 which concerned specific components of the problem- solving process. The method used was to demonstrate that these components contribute to success in problem solution independently of basic knowledge. The results of these analyses are presented below in conjunction with the specific hypotheses concerned. ■53- Hypotheses Concerning Components of the Problem-Solving Process Hypotheses 2 through 8 of this study had reference to the contributions of certain aspects of the problem- solving process to trouble- shooting suc- cess. These hypotheses possessed a common logic. They were stated in a similar manner and were, hence, tested by similar methods. For these reasons the general methods used for examining them are presented here before results for the particular hypotheses are discussed. Each of these hypotheses was concerned with a particular behavioral aspect of the trouble-shooting process. They were each subdivided into three parts. These parts are stated here in terms of the behaviors which define these components of the process. 1. The behavior is consistent over problems. In other words, there is a tendency for mechanics to be consistent in the degree to which they exhibit the behavior and the performance test is capable of revealing individual differences among subjects with respect to the behavior. 2. The behavior is a contributor to (or detractor from) success in solving individual trouble- shooting problems. 3. There is a positive (or negative) relation between the extent of occurrence of the behavior over several trouble-shooting problems and the number of these problems passed. This has reference to the relationship between the behavior and the performance test as a whole. The first step in examining these seven hypotheses , or more specifically, the sub-hypotheses into which they were divided, was the abstraction of the particular behaviors involved from the records of subjects' trouble- shooting procedures. In this manner the behaviors could be quantified and subjected to analysis. Specifically, the behavior of each of the 40 subjects for each of the eight performance test problems, as this behavior was recorded on the Analysis Record Sheets , was scored. This was done, for example, on the basis of non-occurrence, 0, or occurrence, 1, of the behavior or on the basis of a simple scale: "none," 0; "some," 1; or "much," 2. To derive a subject's total score for the behavior, these individual problem scores or weights were summed over the eight performance test problems. The next step is examining each of these major hypotheses was the computation of r^. This statistic served as the criterion for the first aspect or sub-hypothesis which stated that the behavior was consistent over problems. The normality assumption which is made for this technique will be discussed below in conjunction with the individual hypotheses. To test the second aspect of each of these hypotheses, Fisher's "exact test" was applied to the contingency tables which resulted from classifying the trouble-shooting records for each problem on the basis of passing or failing the problem and some dichotomy of the behavior. The -54- contingency tables which resulted for these tests are presented in Appendix C. The relation between the total behavior score and the total pass-fail score was next examined for each hypothesis. This was done, where the necessary assumptions appeared to be met, by product-moment coefficients of correlation. If the bivariate distribution appeared such that it was untenable to assume that it came from a bivariate normal population, the chi-square test was employed. For these cases, the pass-fail scores were grouped into three classes: four or less problems passed, five problems passed, and six or more problems passed. The total behavior scores were dichotomized such that the dividing point fell as closely as possible to the median of the distribution of these scores. The resulting three by two contingency table was used to compute the chi-square with two degrees of freedom. The final analysis made with respect to hypotheses 2 through 8 had to do with the hypothesis concerning basic knowledge. The logic used here was that if it were to be shown that there existed a relation between the behavioral components of the problem- solving process and problem solution, a relation which was independent of basic knowledge , then the "not sufficient" part of the basic knowledge hypothesis would be supported. The statistic used was the partial correlation coefficient. This gave the correlation between the total behavior scores and the total pass-fail scores with the basic knowledge variable i6 partialed out." Perception of Symptom The second hypothesis of this study had reference to the ability of mechanics to orient successfully to the problem, to perceive the audible symptom of the trouble correctly. It was that % Successful mechanics tend to perceive the symptom of a malfunctioning equipment completely and correctly; whereas, unsuccessful mechanics tend to perceive the symptom incompletely or incompletely or incorrectly. The scoring of the statement of the symptom for each problem was accomplished by simply rating this statement, as it appeared on the Analysis Record Sheets , as correct or incorrect. The frequency distribu- tion of the number of problems for which this statement was correct, the total behavior score for this hypothesis, is shown in Table 7. It is immediately evident from this distribution that there were few incorrect statements of symptoms made. Only 13 of the 40 subjects failed to identify the symptom for at least seven of the test troubles. With respect to the consistency over problems of the ability to state the symptom correctly, the test statistic, r^ s was B 21. The five percent critical value for this statistic is .31. Furthermore, the assumption of normality involved in applying this test might legitimately be questioned. -55- Table 7 Distribution of Number of Correct Statements of Symptom Number f 8 9 7 18 6 9 5 3 4 1 n - = 40 It may be that if the sample did not come from a normal population that the value required for significance would be even larger. Hence, it is concluded that the present experiment was unable to support the position that there exists a general ability to correctly perceive and state the symptoms of troubles. It should be remarked here that an assumption which this experiment forces is that there is a one-to-one correspondence between the mechanic's covert perception and his verbal statement of a symptom. Although the present concern was of necessity with the latter, the former may actually be the crucial aspect of the trouble- shooting process. The fourfold contingency tables which were used to examine the second part of this hypothesis are included in Appendix C. For each problem subjects were classified as having passed or failed the problem and as having made a correct or incorrect statement of the symptom. The exact probabilities associated with these tables are presented in Table 8. These probabilities indicate little support for the hypothesis. Only one problem, problem five, shows a probability of less than .05. Reference to Appendix C will indicate, however, that a cause for the relatively large probabilities obtained was the infrequency of incorrect statements. When any marginal entry in a contingency table is small, only a few combinations of entries are possible. Hence, even the most favorable combination will have associated with it a relatively large probability of occurrence. It must be concluded, however, that this experiment did not provide support for the hypothesis. -56- Table 8 Exact Probabilities for Hypothesis 2 Problem P a 1 2 a 3 -23 4 a 5 .04 6 .74 7 .59 8 .45 i$o incorrect statements It was not possible to use pro duct- moment correlation for testing the relationship between the total behavior score and the past-fail score. The nature of the bivariate distribution of these two variables did not justify the assumption that it came from a normal population. Hence, chi- square was used. The contingency table which resulted for this analysis was the following: Pass-Fe lil Behav ior Score Score 4 - 6 7-8 Total 6-8 5 1 - 4 3 4 6 8 9 10 11 13 16 Total 13 27 40 Chi-square for this table is .34. This does not reach the previously established five percept point of 5.99. Hence, it is concluded that the investigation has not been able to support the hypothesis that the tendency to correctly state the symptom is a correlate of success in trouble- shooting. Because the bivariate distributions involved deviated from normality, the partial correlation of performance test success with orientation behav- ior with the effects of basic knowledge partialed out could not be used. It is unlikely, however, that the relationship would have been appreciable. -57- Tendency to Perform General Checks The third hypothesis of this study was that: Successful mechanics tend to attempt to secure sufficient information before accepting a hypothesis concerning the specific area of the equipment in which the trouble resides; whereas, unsuccessful mechanics frequently accept a hypothesis without attempting to secure such information. In Chapter II, "attempting to secure sufficient information" was interpreted to mean the performance of general checking procedures before beginning to check specific components within given areas of the receiver. The records of performance test behavior clearly indicate whether such general checks were made. Hence, the scoring of this behavior consisted simply in crediting a subject with either having performed general checks immediately following his statement of the symptom or having started instead by making specific checks. Table 9 shows the frequency distribution of the number of problems for which the subject performed general before specific checks. Table 9 Distribution of Number of Problems with Initial General Checks Numbe r 8 9 7 16 6 5 5 7 4 1 3 1 2 1 1 n = 40 Although this distribution is somewhat negatively skewed, it does possess a wide range. That this range is indicative of true individual differences is indicated by r tt which is .58 for these scores. Although this value far exceeds the one percent critical value of .41, the departure of the distribution from -58- normality may limit the certainty with which its significance is accepted. It is felt, however, that even if the parent distribution does depart from normality, the .58 statistic would be well within the appropriate five percent rejection region for the null hypothesis. It is concluded, therefore, that there has been shown to exist individual differences among mechanics in the extent to which they tend to perform general checks. The exact probabilities, indicative of the contributions which initial general checks make to the solution of individual problems, are reported in Table 10. Again it is found that this aspect of the hypothesis received little support. With one exception, the contingency between passing and Table 10 Exact Probabilities for Hypothesis 3 Problem P 1 .70 2 .96 3 .00 4 .55 5 .57 6 .55 7 .25 8 .40 failing a problem and performing initial general checks proved to be insignificant. Hence, it must be concluded that the contribution of initial general checks to success in solving the individual problems has not been demonstrated. Chi- square was used to test the significance of the relationship between the total pass-fail score and the total behavior score or number of items for which initial general checks appear. Following is the contingency table which resulted for applying this test: -59- Pass-Fail Behavior Score Score 1 - 6 7-8 Total 6-8 1 10 11 5 3 10 13 1 - 4 11 5 16 Total 15 25 40 Chi-square for this table is 11.61 which exceeds the 9.21 one percent point. It will be noticed that the relationship expressed by the frequencies in the contingency table is in the predicted direction. Hence, there is no doubt concerning the conclusion. It is that the tendency to perform initial general checks and problem- solving success are positively related. This conclusion may appear contradictory to the previous finding that there was, with one exception, no significant relation with respect to individual items. It appears, however, that the "caution" of the successful subjects cumulated over the eight problems without making substantial contributions to the solution of individual problems. Unfortunately, it was again impossible to examine the relation of this behavior to performance test scores independently of the basic knowl- edge measure. Due to the untenability of its underlying assumptions the partial correlation technique could not be employed for this behavior. First Hypothesis Accepted The fourth hypothesis of this study was that : The first hypothesis accepted by successful mechanics tends to be correct; whereas, for unsuccessful mechanics it tends to be incorrect. In Chapter II, it was indicated that the problem-solving model presented there viewed hypothesis behavior as being evidenced by the performance of what were called "specific checks." The supposition is that when a subject began performing specific checks he had begun to entertain a hypothesis to the effect that the defective component was located in the area in which the checks were made. The scoring of "first hypothesis behavior," therefore, consisted of determining whether the first few specific checks performed by the subject were in the "trouble area" or not. The frequency distribution of the number of problems for which the "first hypothesis was correct" is shown in Table 11. This distribution does not exhibit a large deviation from normality. -60- Table 11 Distribution of Number of Problems for which First Hypothesis was Correct Numbe r 8 2 7 3 6 8 5 10 4 7 3 8 2 2 n = 40 With respect to the consistency of this behavior over items, the statistic, r-j--j-, was .33. This value is not large, but it does fall within the five percent critical region for the hypothesis of no individual differences. It is concluded, therefore, that the experiment was sensitive enough to discriminate among subjects with respect to the tendency for their first hypothesis to be correct. With respect to the contribution which a correct initial hypothesis has to problem solution in trouble shooting, the exact probabilities for the individual problems are shown in Table 12. Four of the eight probabil- ities are less than .05, three of these being less than „01„ This seems to support this aspect of the hypothesis. But what of the other four problems for which it cannot be said that the relation exists ? It would appear that differences among problems have been evidenced. In other words, the nature of the problem situation determines the importance of the mechanic making his initial specific checks in the trouble area. 9 This observation is further supported by the results relative to orientation and tendency to perform general checks . It appeared that each of these types of behavior was crucial with respect to one of the eight test problems. The literature concerning the psychology of problem solving 'The nature of problems 6 and 8 would lead one to expect the result obtained. With the given symptoms, a successful procedure on these problems can be to check capaciters throughout the receiver without attempting to decide on a specific stage. -61- Table 12 Exact Probabilities for Hypothesis 4 Problem P 1 .12 2 .54 3 .01- 4 .00 5 .00 6 .48 7 .01+ 8 .82 contains frequent reference to these differences among problem situations. The present state of knowledge concerning the psychology of problem sol- ving might have induced this study to include a hypothesis concerning differences in problem situations or, even more specifically, differences among the nature of malfunctions, which would affect trouble- shooting behavior. This type of hypothesis was not included, however, because this study was concerned primarily with "modalities" in the problem- solving behavior of mechanics. The other type of question could well be a subject for further inquiry. This concept is discussed here not only because it is of interest in itself, but also because it serves to limit the type of conclusion which can be reached in this study with respect to the importance of particular components of the problem-solving process. Returning to the particular component being discussed, the correla- tion between total pass-fail scores and total behavior scores was .56. This is well within the one percent rejection region for the hypothesis of no correlation. It is therefore concluded that the tendency for the first hypothesis to be correct over a numbex of trouble-shooting problems is 10 q t 1 See, for example Augusta Alpert. The Solution of Problem Situations by Preschool Children. Contributions to Education, No. 323. New York: Bureau of Publications, Teachers College, Columbia University, 1928. 69 p. He stated that: "A child's solving activity was found to be determined more definitely by the problem situation than by any other factor.' (p. 66.) M. R. Marks. "Problem Solving as a Function of the Situation." Journal of Experimental Psychology 41 :74-80; January 1951. Benjamin Burack. ''The Nature and Efficacy of Methods of Attack on Reasoning Problems." Psychological Monographs 64:313:1-26; 1950. -62- a significant contributor to success in trouble shooting. For the first time, the partial correlation technique discussed earlier in this chapter appeared applicable. The partial correlation between the performance pass-fail score and the number of items for which the first hypothesis was correct, the behavior score, with the effect of basic knowledge scores eliminated was .40. This value is beyond the one percent point established for it. It is concluded, therefore, that basic knowledge is not the sole contributor to success in trouble shooting. This result supports the "not sufficient" clause of hypothesis 1. Wrong Hypothesis Behavior The fifth hypothesis stated in Chapter II was that s Unsuccessful mechanics tend to (a) entertain more in- correct hypotheses, and (b) perseverate incorrect hypotheses for a longer period of time than successful mechanics. The testing of this hypothesis was divided into three phases. The first two of these phases connected with the two parts of the hypotheses as it is stated. The final phase was the testing of the total hypothesis by combining these two component aspects of it. These three steps are discussed below in turn. 1. Wrong Hypothesis Entertained. The first part of hypothesis 5 referred to entertaining false hypotheses. In terms of the problem- solving model s a mechanic entertained a false or wrong hypothesis whenever he performed specific checks in a non-trouble area of the receiver. This behavior was scored by simply determining if the subject had or had not performed non-trouble area, specific checks. The fre- quency distribution of the number of items for which the subjects performed non-trouble area checks or, in other words, entertained at least one false hypothesis, is presented in Table 13. The deviation of this distribution from normality is not great. The first question asked about this behavior was if it was consistent over items, if significant individual differences among subjects were found. The statistic, r-fct» was .25. This is in the acceptance region for the hypothesis of no individual differ ences It must, therefore, be concluded that this experiment failed to support the contention that there exist individual differences among the subjects with respect to a tendency to evolve incorrect hypotheses. -63- Table 13 Distribution of Number of Problems for which Incorrect Hypotheses Occurred Numbe r 7 2 6 4 5 8 4 11 3 11 2 1 1 3 n = 40 The exact probabilities associated with the contingencies of this behavior with success in solving individual problems are shown in Table 14. It is evident that considerable suspicion has been cast . Table 14 Exact Probabilities for Hypothesis 5-1 Problem P 1 .08 2 .20 3 .00 4 .00 5 .00 6 .12 7 .05- 8 .20 on the hypothesis that it makes no difference in trouble shooting for a defective component whether wrong hypotheses occur or not. The conclusion is clearly that it does matter. -64- Furthermore, the correlation between the number of items for which wrong hypothesis occurred, the total behavior score, and the pass-fail score was -.54. The negative relationship is what was expected and its magnitude is great enough to reject the hypothesis of no relation. This conclusion appears contradictory to the lack of consistency of wrong hypoth- esis behavior over items. How can a set of nonconsistent scores correlate with another set of scores ? The answer is evident from the relationships implied by Table 14. The relation between success and wrong hypothesis occurrence on individual problems cumulated over problems and found expression in the respective total scores. The partial correlation between pass-fail scores and total behavior scores with the effects of basic knowledge test scores partialed out was, in this case, -.34. This value is of sufficient magnitude for the hypothesis of no independent relation to be rejected. The prior supportive conclusions concerning the "not sufficient 5 ' phrase of hypothesis 1 are further strength- ened. 2. Perseveration. The second part of hypothesis 5 referred to the perseveration of false hypotheses. The problem in quantifying this behavior was in deciding what criterion to use for saying that a hypothesis was retained unusually long. Clearly this would be some function of the time spent or the number of checks performed consecutively in a non-trouble area. The time criterion was ruled out because the method of recording times was not complete enough for the present pruposes. On the basis of an examination of the frequency distribution of the number of P- checks in the longest consecutive "string" of such checks on each of the 320 Analysis Record Sheets , it was decided that six or more consecutive P- checks would be considered as indicating perseveration. Although this decision was of necessity somewhat arbitrary, it does possess logic in that six checks in a particular stage of the Ul/BER -1 Radio Receiver is generally quite adequate for "checking out" that stage or area. Hence, this behavior was scored by determining if the problem- solving process contained a "perseverated hypothesis" or not with the above "six consecutive persistence checks in column (8) of the Analysis Record Sheet" as the criterion for a perseverated hypothesis. Table 15 shows the frequency distribution of the number of problems for which a hypothesis was so perseverated. This distribution obviously shows a distinct deviation from normality. It should be noticed that these scores, due to the nature of their derivation, are dependent to some extent on wrong hypotheses occurring at all. This dependence must be considered when interpreting the statistic r^. The value of r^ was »38. This is a larger value than that obtained for wrong hypothesis occurrence behavior, and hence indicates some degree of consistency for the performance of the 40 subjects with respect to this behavior. In an inferential sense, however, the significance of this additional consistency is in doubt. Furthermore, the non- normality of the distribution of Table 15 decreases the interpretability of the computed rtt with respect to the five percent critical value of .31. Hence, the conclusion that differences in a tendency for perseverative behavior were shown must be held as tentative. 65- Table 15 Distribution of Number of Problems for which a Hypothesis was Perseverated Number 5 1 4 3 3 2 2 7 1 16 11 n = 40 In determining the exact probabilities associated with the rela- tionship between perseveration and success in solving individual problems, this dependence on a wrong hypothesis occurring was removed. The null hypothesis was stated as : given that an incorrect hypothesis was entertained there is no difference between the proportions passing the problem for those mechanics who perseverate and those who do not perseverate a hypothesis. Hence, the total frequencies for the contingency tables in .Appendix C , hypothesis 5-2, correspond to the total number of subjects who entered non-trouble areas of the receiver, who entertained false hypotheses. The exact probabilities for these tables appear in Table 16. These probabilities and the contingency tables from which they were Table 16 Exact Probabilities for Hypothesis 5-2 Problem P 1 .07 2 .15 3 .35 4 .09 5 .07 6 .42 7 .08 8 .22 66- derived indicate that the relation is in the expected direction for each of the problems. Although all of the probabilities are low, none reach the five percent value. This is explained by the marginal entries of these tables, which are in many cases small. For given marginal frequencies so small only a few combinations of entries are possible. Hence, although for five of the eight problems the most favorable arrangement appeared , it was impossible for the probabilities to be below .05. It appears safe, therefore, to conclude, at least tentatively, that perseveration is a strategic factor in trouble shooting individual problems. The distribution of total perseveration scores prohibited the use of the product-moment correlation in testing for relationship between these scores and the pass-fail criterion. Chi-square was therefore used. The following contingency table resulted for this purposes Pass-Fail Behavior Score Total Score - 1 2 - 5 6-8 5 1 - 4 10 10 7 • 1 3 9 11 13 16 Total 27 13 40 Chi-square for this arrangement is 7.39 which with two degrees of freedom falls in the five percent rejection region for the hypothesis of no relation. In as much as the dependence in this table is in the predicted direction, it is concluded that there is a relationship between the two classifications. 3. Total Wrong Hypothesis Behavior . In the following manner the above two behaviors were combined so as to provide overall tests of hypothesis 5. A simple system of weighting was used to derive a total wrong hypothesis behavior score. If, on a problem, a subject had entered no non-trouble areas, it was weighted 0. If he entered a wrong area but performed only one check there, the weight was 1* if he performed two to five consecutive checks there, it was 2; and if he remained there to perform six or more checks, it was weighted 3. The frequency distribution of total wrong hypothesis scores, obtained by summing the weights for individual problems, is shown as Table 17. This distribution does not appear to deviate far from normality. The consistency of wrong hypothesis behavior over problems is represented by r-fct which was .31. This value is in the five percent rejection region for the hypothesis of no individual differences. It can, therefore, be concluded that this wrong hypothesis score differentiates among mechanics. -67- Table 17 Distribution of Total Wrong Hypothesis Behavior Scores Score 18-20 15-17 12-14 9-11 6-8 3-5 0-2 1 3 8 16 9 1 2 n = 40 The contribution of the behavior which is represented by these scores to success on individual problems has been completed in the analysis of this aspect of its two components. It needs no further discus- sion here. The correlation of total wrong hypothesis scores with total pass- fail scores was -.62, a highly significant figure. This result was expected on the basis of the comparable correlations of the two behaviors which compose this one. It forces the acceptance of hypothesis 5 in full. The independence of the contribution of wrong hypothesis behavior to trouble- shooting success was tested by means of the partial correlation coefficient. With the effects of basic knowledge eliminated, the correla- tion was still -.46. Further support hence accrued to the "not sufficient" clause of hypothesis 1. Use of Obtained Information Hypothesis 6 had reference to the mechanic's ability to utilize pertinent information which he received during the course of his trouble- shooting procedures. It was that: Successful mechanics upon obtaining critical information in their checking procedures tend to recognize and use it; whereas, unsuccessful mechanics do not. It will be recalled that the interpretation of trouble- shooting behavior included a consideration of certain "clue" checks which provided definite information concerning the location of the defective component. If the -68- subject, after having performed a "clue" check, performed checks which indicated that he used the information received, it was recorded as a plus (+) clue. If succeeding checks indicated that he missed the information or failed to use it, it was recorded as a minus (-) clue. Because of the nature of the occurrences of plus and minus clues, it was decided to analyze these two aspects of the hypothesis separately. 1. Plus Clue s. One approach to scoring plus clue behavior might be simply to count the number of such checks marked for each problem. The distribution of the numbers of plus clue checks for each problem was, however, extremely skewed. It was, therefore, decided to score this behavior by means of a simple weighting procedure which would not only rectify this distribution to some extent, but also make the scores more manageable. The numbers of plus clue checks for any one problem were divided into four groups. These groups might be called "no plus clue checks," "few plus clue checks, ^ and "many plus clue checks," and "very many plus clue checks." The weights assigned were if there were no such checks, 1 if there were one to four or a "few," 2 if there ' were five to eight or "many," and 3 if there were nine or more, the "very many" classification,, Of the 320 problem records, 92, 88 , 91, and 49 were in the respective groups. Table 18 shows the frequency distribution of the total plus clue scores which resulted when the weights for the individ- ual problems were summed. This distribution exhibits a fair conformity to the normal model. Table 18 Distribution of Total Plus Clue Scores Score 16-17 4 14-15 3 12-13 8 10-11 13 8-9 3 6-7 4 4-5 4 2-3 1 n = 40 The test of consistency of plus clue behavior resulted in an r^ of .65. It can be concluded with assurance that the subjects of this study were differentiated on the basis of the ability to obtain and use pertinent check information,, -69- In order to test the contribution of plus clue behavior to success on individual items, it was necessary to combine categories. The contingency- tables derived by combining categories to arrive at the dichotomy in the behavior scores are included in Appendix C. For the majority of the problems this dichotomy was between "no or a few plus clue checks" and "many or very many plus clue checks." The exact probabilities associated with these tables are shown in Table 19. The conclusion to be reached from Table 19 Exact Probabilities for Hypothesis 6-1 Problem P 1 .02 2 .01 + 3 .00 4 .00 5 .00 6 .00 7 .00 8 .00 an examination of this table is obvious: the relative number of clue checks which provide information which is used is a significant contributor to success on individual problems. The correlation between total plus clue scores and pass-fail scores, .82, was to be expected on the basis of the demonstrated consistency of the behavior and the relationships shown for the individual problems. It is highly significant. The partial correlation between plus clue scores and pass-fail score with the influence of the basic knowledge measure eliminated was .73. It was, hence, demonstrated that another observable aspect of the problem- solving process makes a significant contribution to trouble- shooting success independently of basic knowledge. 2. Minus Clues . Although there were fewer clue checks marked minus on the Analysis Record Sheets , the distribution of such checks was sufficiently positively skewed to suggest that a simple weighting procedure would provide a satisfactory method of scoring this behavior. The following system was used. If no minus clue checks appeared on a problem record, the weight was 0. If one or two or "a few" such checks appeared, the weight was 1. If three or more or "many" appeared, the weight was 2. Of the 320 problem records, 233, 44, and 43, respectively, fell into the three groups. The total minus clue scores were obtained by summing these 70- weights for the eight problems. The frequency distribution for these scores is shown in Table 20. This distribution appears to deviate somewhat from normality. The consistency of behavior which merits minus clue description is indicated by r^ which was .15. It cannot be concluded, therefore, that individual differences in this behavior have been demonstrated. A possible Explanation of this lack of consistency is the fact that in order to merit a clue minus credit, the subject must have first been led to make the clue check. It appears reasonable that there is a negative relation between whatever it is that causes a mechanic to perform clue checks and the ability to interpret and make use of the information he receives from them. Table 20 Distribution of Total Minus Clue Scores Score 8 2 7 2 6 2 5 3 4 7 3 6 2 11 1 5 2 n = 40 In testing for the contribution (or detraction) which minus clue behavior makes to success in trouble- shooting individual problems, the subjects were dichotomized on each problem on the basis of number of minus clues which occurred. The dichotomy was in each case between none and one or more minus clue checks. The exact probabilities for the contingency tables of this dichotomy and the pass-fail criterion are shown in Table 21. The contingency tables are reproduced in Appendix C. The figures in this table generally support the contention that minus clue behavior is associated with failure on individual problems. Notable exceptions are problems 6 and 8. These problems stood out earlier as representing problem situations which failed to conform to general expectations. With respect to minus clue behavior the difference between these two problems and the remaining six probably lies in part in the relative scarcity of check procedures which give clearly defined clues. This seems to have resulted in an extremely small number of the clue .71- minus types of checks. It is impossible for a contingency table with a small marginal frequency to exhibit a decided relation. Table 21 Exact Probabilities for Hypothesis 6-2 Problem P 1 .00 2 .00 3 .17 4 .04 5 .00 6 .76 7 .03 8 .73 The apparent deviation from normality of the total minus clue score distribution forced the use of chi-square as a test of the relation between this total behavior score and the total pass-fail score. The following contingency table formed the basis of this test: Pass-Fail Behav ior Score Total Score - 2 ~J - 8 6-8 8 3 11 5 7 6 13 1 - 4 3 13 16 Total 18 22 40 Chi-square for this table is 8.28. With two degrees of freedom this value falls beyond the five percent critical value. The direction of the relationship expressed in this table is in the expected direction. It can hence be concluded that the inability to make use of obtained information is associated with failure in trouble shooting. The concurrence of this result with the lack of consistency of the behavior over problems can be explained on the bases of the cumulation of the demonstrated relation for the individual -72- items. 11 Errors Hypothesis 7 was that: Successful mechanics tend to make fewer errors in the use of test equipment than do unsuccessful mechanics. It will be recalled from the preceding chapter that the records of trouble- shooting processes included an indication of each check procedure that was incorrectly performed. The most straight-forward method of scoring this behavior might be, therefore, to simply count the number of such errors for each problem. It was again decided, however, that this type of scoring would be unrealistic. This is because many errors were consecutive and stemmed from the same source. Hence, a simple weighting procedure was adopted. The trouble- shooting records were divided into three groups which were characterized respectively by no errors, one or two or a "few" errors, and three or more, or "many' 9 errors. Of the 320 records, 154 were in the first group, 99 in the second, and 67 in the third. The weights assigned these groups were , 1 , and 2 respectively. The frequency distribution of the subjects' total error scores based on these weights is shown as Table 22. Table 22 Distribution of Total Error Scores Score 12-13 1 10-11 5 8-9 7 6-7 5 4-5 13 2-3 5 0-1 4 n = 40 Although the normality assumption was in doubt, the product moment correlation and the partial correlation were computed for this hypothesis as for others. The results were: r( behavior ) ( pass _f ai i) '= -.57, and r( behavior ) (pass-fail), (basic knowledge) = ~ A3 ' I£ the lack of normality does not alter the size of the tests to a great extent, both of these values are significant. -73- This ability to make proper use of test equipment, as here measured in a reverse fashion, was one of the more consistent components of the trouble-shooting process. The statistic, r^, for these scores was .62. This value falls well within the rejection region for the hypotheses of no individual differences. It may, therefore, be concluded that there are individual differences among mechanics in a tendency to make proper use of test equipment. The association of errors with failure on individual problems is shown in the contingency tables for hypothesis seven in Appendix C. To arrive at these tables, it was necessary to combine two of the original groups into which subjects were divided on the basis of number of erroneous checks. Usually, the dichotomy became "no errors" versus "few or many" errors. The exact probabilities associated with these tables are presented in Table 23. The magnitudes of these probabilities cast considerable doubt on the tenability of the hypothesis of no relation. Even though the five Table 23 Exact Probabilities for Hypothesis 7 Problem P 1 .07 2 .05- 3 .06 4 .03 5 .13 6 .05- 7 .01 + 8 .18 percent point was reached for but four of the problems, the remaining values are consistently low. It is considered safe to conclude, therefore, that the improper use of test equipment does contribute to failure on individual trouble- shooting problems. The product-moment correlation between total error scores and total pass-fail scores was -.35. Although this does not indicate a substan- tial relation for the present sample of mechanics, the value is well within the five percent region of rejection for the hypothesis of no relation. It can be concluded that the tendency to make improper use of test equipment is associated with low levels of trouble-shooting performance. The partial correlation between pass-fail scores and error scores, with the influence of basic knowledge test scores eliminated, was -.19. This value is in the region of acceptance for the hypothesis that the partial -74- relationship is zero. Hence, on the basis of this experiment, it cannot be concluded that the contribution of errors to poor performance is independent of basic knowledge. Duplication The eighth hypothesis of the study and the final one concerning compo- nents of the problem- solving process was that: Successful mechanics duplicate the same checks less frequently than do unsuccessful mechanics. In the interpretation of trouble- shooting behavior described in Chapter III, it was indicated that all instances of such duplications were recorded on the Analysis Record Sheets . In a similar fashion to that used with respect to the behaviors involved in hypotheses 6 and 7, duplications were also scored by means of a simple weighting procedure. No or only one duplica- tion for a problem constituted the "few 53 or category, two or three repeated checks, the "several" or 1 category, four to seven repeated checks, the "many" or 2 category, and more than seven repeats, the "very many" or 3 category. Of the 320 problem records, 97, 86, 87, and 50 were in the respective groups. The frequency distribution of total duplication scores, obtained by summing the problem weights is shown in Table 24. This distribution conforms fairly well to the normal model. The consistency of duplication behavior is represented by r^ which was .58. This value indicates a high degree of consistency for this behavior and is the basis for the conclusion that there are individual differences in the tendencies of mechanics to be repetitious in their trouble- shooting procedures. To arrive at the contingency tables in Appendix C for this Table 24 Distribution of Total Duplication Scores Score f 16-18 5 13-15 6 10-12 12 7-9 7 4-6 7 1-3 3 n = 40 -75- hypothesis, it was necessary to combine categories. The manner inwhich this was done is evident from these tables. The results of the test of the hypothesis that duplications are not associated with success or failure on the individual problems are reported in Table 25. For only problems 1 and 2 are these probabilities of such magnitude as to demand rejection of the hypothesis of no contingency. A possible explanation for the deviation of duplication behavior on these problems from the pattern established for Table 25 Exact Probabilities for Hypothesis 8 Problem P 1 .02 2 .00 3 .26 4 .67 5 .46 6 .24 7 .09 8 .28 the remaining six lies in the testing situation. It seems likely that as a result of the novelty of this situation the unsure subjects duplicated more checks on these early problems. This effect would tend to disappear as all of the subjects became adjusted to the situation. If this explanation for the results on problems 1 and 2 are accepted, the conclusion becomes that, although the expected direction of the relation was generally exhibited, a clear relation between tendency to repeat check procedures and failure on individual items was not demonstrated. The product-moment correlation between total duplication scores and total pass-fail scores was -.15. This value has the expected sign but is not of great enough magnitude to enable the hypothesis of no relation to be rejected. Although the assumption underlying this hypothesis was that duplica- tion is a behavioral expression of the subject's confusion or lack of a definite plan of attack, it may also to some extent reflect a degree of caution in trouble shooting. If this should be the case, the lack of relation- ship between duplication and success would become explainable. -76- Hypothesis Concerning Methods of Attack The final specific hypothesis of this study concerned characterizing mechanics on the basis of their overall patterns of response or methods of attack on trouble- shooting problems: It is possible to differentiate among mechanics on the basis of overall methods of attack employed. Further- more, different methods of attack characterize mechanic! at different levels of proficiency with the prototype trouble- shooting process being characteristic of the most success- ful mechanics. The first step in the analysis suggested by this hypothesis was an intensive study of the 320 Analysis Record Sheets . This was done in an attempt to abstract from these trouble- shooting records some number, preferably small, of different sequences of types of checking procedures or objectively different methods of attack. The objective was the definition of categories into which similar patterns of attack could be grouped. This attempt to define categories descriptive of methods of attack was guided by the concepts relative to this aspect of problem solving discussed in Chapter II. It can be said, then, that the following three criteria guided the definition of method of attack categories? 1. An attempt was made to keep the categories small in number without sacrificing the exhaustive criterion of classification. This was an attempt to increase the likelihood of securing appreciable variance in the extent of occurrence of any single category. 2. The descriptions of the patterns of attack which defined the categories were to be specific and descriptive of -the checking procedures as these were recorded on the Analysis Record Sheets. Although this part of the investigation was of necessity somewhat subjective, this was an attempt to minimize this subjectivity. 3. The categories were to have had some foundation in psychological concepts. If the categories were defined in a completely mechan- ical fashion and without regard for their probable psychological basis, it was felt that they would have little ultimate significance. Categories of Methods of Attack The five categories which were finally defined were as follows" A. Logical Elimination . The pattern of trouble shooting which defined this category generally conformed to the prototype trouble- shooting procedure described in Chapter II. Although very few of the 320 processes here under study consisted of the minimum -77- essential checks to logically isolate the defective component, there were many patterns which showed but small deviation from this model. The patterns included in this category consisted, therefore, of several orientation checks, a series of general checks, a not too extended series of specific checks in the trouble area, and a correct solution. This was the only category which was logically dependent on problem solution or failure. B. Persistence . The patterns of attack classified in this category corresponded to the prototype procedure through the general check phase of that procedure. They consisted of several orientation checks, general checks, and finally an extended series of specific checks in the trouble area. The patterns in this category varied with respect to their conclusions. This might be success, failure, or a few checks in other areas of the receiver. The subjects in this category appeared to have formed a correct hypothesis with respect to the area of the receiver which contained the trouble, but did not efficiently localize the specific faulty component within that area. C. Caution. The distinguishing characteristic of the "cautious" method of attack was the prevalence of checks of the general type. In some cases it appeared that the subject was never able to decide on a specific area. In other cases he would enter more than one area of the receiver, but appeared frequently to revert to general checks to guide his selection of an area to examine. D. Random . As is usual in the area of problem solving, the term random may not have been entirely appropriate. It appeared, however, to best describe the pattern of attack in which the subject frequently jumped from one area of the receiver to another in his checking procedures. In this pattern, general checks were employed only at the outset, if at all. This pattern epitomized complete absence of the logical approach. It is possible that this pattern relates to "trouble shooting by probability data." The subject may have been attempting to omit steps in the logical approach by checking components which in his opinion were likely causes of the trouble. E. Perseveration. The patterns which defined this category con- sisted of orientation checks, general checks (usually), and an extended series of specific checks in a single non-trouble area of the receiver. Some deviations from this general pattern were allowed, but the distinguishing characteristic of several patterns was the extended series of non-trouble area checks. Evidently, the subject had formed an incorrect hypothesis concerning the area in which the trouble resided and had been unable to "break out" of this hypothesis even though the information he was receiving demanded him to do so. Consistency in Utilization of Different Methods of Attack After the preliminary examination of the Analysis Record Sheets and the definition of the categories of methods of attack, the actual classification of the 320 records into these categories were accomplished. The results of -78- this classification were the numbers of problems for which each subject was classified into each category. The optimum outcome would have been, that each subject responded to all eight problems with the same characteristic pattern. This, of course, was not the case. In Table 26 are shown the frequency distributions of the numbers of problems which were classified into each category for each subject. Table 26 Distributions of Numbers of Problems Classified into Five Methods of Attack Categories Number Category B C D E 6 5 4 3 2 1 7 2 2 3 4 4 1 8 8 6 4 1 10 11 4 9 3 8 9 14 14 13 5 7 12 8 22 Total 40 40 40 40 40 Although no subject used any one method of attack exclusively, it was possible to estimate the consistency with which subjects followed the respective methods, as shown by this classification. This was done by means of the statistic, r-j-j., which has previously been discus sed. The values of this statistic for each of the five categories were as f ollows i Category r tt A .48 B .29 C .46 D .41 E .33 -79- Categories A, C, and D evidence a high and statistically significant degree of consistency". Values of r-tt are affected by the number of items, in this case problems, upon which it is based and with but eight items, a value of .40 is satisfactory. For the other two categories of methods of attack, the r^ values are somewhat lower. The value for category E, however, exceeds the five percent critical value previously established. Even though the assumption of normality for two of the above five distributions is in doubt, it appears safe to conclude that there do exist individual differences with respect to the tendencies of mechanics to use each of these five methods of attack. Consequently, it is concluded that mechanics differ with respect to their inclinations to respond to trouble- shooting problems with different overall methods of attack. Efficacy of Different Methods of Attack If the above conclusion is acceptable, it would not appear unreasonable to assume that each subject could be characterized by one of the above categories of response types. On the basis of this assumption, the 40 subjects were divided into five groups on the basis of which of the methods of attack occurred with greatest relative frequency over the eight test problems. This was done by transforming each of the distributions in Table 26 to standard scores and then choosing, for each subject, that category for which his standard score was greatest. This was admittedly a somewhat crude procedure because for some of the forty cases the two largest standard scores were of nearly equal value. However, it was felt that in an exploratory study, such as the present one, it is frequently necessary to sacrifice rigor for the possibility of discivering new insights. In the present case the errors that might have occurred in forming the five groups of mechanics would have logically appeared as error in the following analyses and, hence, decreased the likelihood of results favorable to the hypothesis being examined. The results of the classification of subjects into the five methods of attack groups showed ten subjects in group A, ten in group B, nine in group C, seven in group D, and four in group E. The next step was an analysis of variance of the pass-fail performance test scores as a test of the differences in efficiency of the five groups. The results of this analysis of variance are shown in Table 27. The significant value of F indicates that this manner of classifying mechanics according to methods of attack is associated with the efficiency of the mechanics in trouble shooting. With respect to the question of which group was most efficient, the group pass-fail means were : -80- Group Mean A 5.90 B 4.90 C 4.33 D 3.29 E 2.25 Total 4.48 Table 27 Analysis of Variance of Pass-Fail Scores for Methods of Attack Groups Source of Sums of Mean variation df squares squares Total 39 149.98 Between Groups 4 52.00 13.00 4.64** Within Groups 35 97.98 2.80 ^Significant at one percent level Although the analysis of variance does not yield probabilistic evidence with respect to the directions of these differences, inasmuch as the hypothesis specified that group A, the ''logical elimination, subjects," be most efficient, it appears safe to conclude that the statistics support this contention at least. Furthermore, were it possible to ascribe devia- tions from this logical approach to the remaining groups, the rankings of their means would not be unexpected. Hence, it is tentatively concluded that the found ranking of these means is something more than chance. But what of the contribution of basic knowledge ? An analysis of variance on basic knowledge test scores for the five methods of attack groups resulted in an F of 3.52 which with four and thirty-five degrees of freedom is significant at the five percent level. Hence, the groups also differed significantly on this measure. The means for the respective groups on this measure were: -81. Group Mean A 98.90 B 97.20 C 88.89 D 82.43 E 71.75 Total 90.62 The ranking of means here is the same as that for the performance test scores. It must be concluded that the method of attack utilized by a mechanic is to some extent, at least, dependent upon his command of the facts and principles of basic electronics knowledge. In order to discover if it is possible to attribute the differences in efficacy of different methods of attack entirely to differences in basic knowledge, an analysis of covariance was executed. This procedure adjusted the means of the groups on the performance test scores for differences in the group means on the basic knowledge test and provided a test of significance for the differences among these adjusted means. This analysis of covariance is summarized in Table 28. The resulting F of 1.91 has 4 and 34 degrees of freedom and does not reach the five percent level required in this study for significance. Hence, it cannot be concluded on the basis of this analysis that the differences in efficacy associated with the five methods of attack exist independent of basic knowledge. Table 28 Analysis of Covariance for Differences between Pass-Fail Means Adjusted for Differences in Basic Knowledge Means Source of Sums of squares Mean variation df Errors of squares F estimate 1.91 An examination of the values of the adjusted means is, however, of interest. They are, for the respective groups: -82- Total 38 104.37 Adjusted means 4 19.18 4.79 Within groups 34 85.19 2.51 Group Adjusted mean A 5.56 B 4.63 C 4.40 D 3.63 E 3.03 Total 4.48 The ranking of these means corresponds to that of the original means before the adjustment was made. Considering this fact and the crudeness with which the original grouping was done, it would appear that the hypoth- esis that different methods of attack differ in efficacy independent of the effects of basic electronics knowledge should be retained for further experimental investigation. Summary and Conclusions This chapter has presented empirical tests of hypotheses concerning trouble- shooting behavior viewed as a problem- solving situation. Descrip- tive characteristics of the measuring instruments employed were also presented. Hypothesis 1 concerned the relationship between knowledge of basic electronics fundamentals and trouble- shooting success. The significant correlation between the measures of these variables employed in this study supported the contention that basic knowledge was essential for trouble- shooting success. Significant partial correlations between performance test success and various observable components of the problem- solving process with the influence of basic knowledge eliminated supported the contention that basic knowledge is not sufficient for trouble- shooting success. Hypotheses 2 through 8 involved several components of the trouble- shooting situation viewed as a problem- solving task. Specifically tested were (a) the occurrence of individual differences with respect to these components, (b) the contributions of these components to success in solving individual trouble- shooting problems, and (c) the relationships between these components and total success on the performance test. A coefficient of internal consistency, denoted here as ?±±, was described and used for testing the occurrence of individual differences with respect to these compo- ents. Fisher's exact probability method was used to test the contributions of these components to. success on individual problems and the product- moment coefficient of correlations was used, where applicable, to test the relationship between these components and performance test success. Where the latter was inapplicable, chi-square was employed. The results of these tests are summarized in Table 29. -83- Table 29 Summary of Tests of Hypotheses Concerning Components of the Problem-Solving Process Component Exact Probabilities 61 r Chi-Square tt Perception of Symptom Tendency to Perform General Checks .21 .58** ( .34 11.61** First Hypothesis Accepted .33* .56** Wrong Hypothesis Entertained Perseveration Total Wrong Hypothesis Behavior Plus Clues Minus Clues Errors Duplications 25 4 - .54** .38* — 31* __c -.62** 65** 8 .82** 15 5 — 62** 4 -.35* 58** 2 -.15 7.39* 8.28* The entries in this column are the numbers of problems out of the eight for which the exact probabilities were less than .05. r is the correlation between the total score for the component and the total pass-fail performance test score. Chi-square with two degrees of freedom was used if r was inapplicable. Hence, only one of the two figures appears for each component. Exact probabilities were not applicable. j One star indicates significance at the five percent level; two stars indicate significance at the one percent level. -84- Following are the general conclusions concerning these hypotheses: 1. There exist individual differences with respect to the occurrence of each of the components, except perception of symptom , wrong hypotheses entertained , and minus clues. 2. Although there are individual differences among problem situations which must be accounted for, the following components display a contribution to (or detraction from) success in solving individual trouble- shooting problems: first hypothesis accepted, wrong hypotheses entertained , perseveration , plus clues , minus clues , and errors . 3. With the exceptions of perception of symptom and duplications , all of the components are related in the predicted manner to trouble- shooting success. The final hypothesis examined concerned the tendencies of mechanics to utilize characteristic methods of attack on trouble- shooting problems and the relative efficacy of such methods of attack. It was concluded that mechanics do differ with respect to methods of attack employed and that different methods of attack are characteristic of mechanics at differing levels of trouble- shooting proficiency. It remained hypothetical, however, that this latter relation is independent of mechanics' knowledge of basic electronics facts and principles. These conclusions, derived from an empirical analysis of the trouble- shooting processes employed by a sample of radio maintenance mechanics on a particular experimental radio receiver, served as the basis for an analysis of their implications for further research in the area of selection and training of maintenance mechanics. This analysis, which also includes more direct implications for such selection and training, is reported in the following chapter. 15- CHAPTER V IMPLICATIONS One purpose of this study was to illustrate an empirical approach to the determination of some of the important objectives of training programs for maintenance occupations. It was concerned with that class of occupa- tions required in the maintenance of electronic equipment. The particular experiment reported here was related to the task of trouble shooting a particular type of electronic equipment. Prior to a discussion of implica- tions for training programs, it is pertinent that limitations of this study be discussed. Limitations In addition to limitations associated with the design of the present experiment and relating to the generality of its results with respect to the particular class of occupations with which it was concerned, there are limitations which relate to the general applicability of the methodology illustrated. The nature of the task of trouble shooting electronic equipment makes it particularly suited to this empirical approach. This is because of a unique characteristic of such equipment, which is the concept of "information flow" associated with electronic circuitries. As distinct from trouble shooting mechanical devices and more abstract types of problem solving, the problem situations presented by malfunctioning electronic equipment constitute relatively closed systems of elements with precise relationships. This feature of electronic equipment made it possible to formulate definitive hypotheses concerning efficient trouble- shooting procedures. The feasibility of developing such hypotheses concerning efficiency in the execution of other types of tasks would depend, however, upon the nature of the task. It may not be possible to plan an investigation parallel to the present one for tasks which involve situa- tions not possessing the inherent logical structure of electronic circuitries. A further restriction of the applicability of this empirical method for determining training objectives is associated with the criterion of successful performance. The method requires that this criterion be relatively easily defined. By defining trouble-shooting success in terms of solving trouble-shooting problems, it was possible to investigate constituents of trouble- shooting proficiency. This might not be possible for tasks for which the criterion of success possesses a greater degree of ambiguity. With respect to the generality of the results of the present exper- iment and relating to the trouble- shooting task, a major limitation lies in the nature of the sample of mechanics who served as subjects. This limitation was discussed in Chapter III where the sample of forty radio mechanics was described. It is only necessary to repeat here that the certainty with which the conclusions resulting from the statistical analyses should be accepted is somewhat restricted by the characteristics of this sample. Because this investigation has been exploratory in nature, it can be anticipated that further research with more rigid experimental designs -86- and more adequate sampling procedures will provide more definitive ev- idence concerning the importance of aspects of the processes involved in trouble shooting. The most restrictive limitation of the experiment resides in the fact that the Ul/BER-1 Radio Receiver differs in several respects from the electronic equipment with which the research reported here was ultimately concerned. The Ul/BER-1 is an unusual radio receiver, designed especially for experimental purposes. In general, standard radio equipment is (a) much more complex in terms of number and arrangement of electronic circuitries and (b) much more compact in terms of physical fabrication than the Ul/BER-1. It is possible that these electrical and physical dif- ferences between the UI/BER-1 and other radio equipment create essen- tial differences in efficient trouble- shooting procedures. In connection with the larger project of which this study was a part, there is in progress a technical comparison of the Ul/BER-1 Radio Receiver with a particular military transceiver, a radio transmitter and receiver combined. This technical analysis, it is anticipated, will provide information concerning differences in trouble- shooting procedures required for the two pieces of equipment. Although final evidence concern- ing the generality of the conclusions reached in this study concerning trouble shooting the Ul/BER-1 will await further experimentation, prelim- inary results of this technical comparison tend to indicate that there is a logical basis for expecting that this generality does exist. A final limitation of the experiment is not an unusual one. This is the influence of the experimental conditions. The problem of the typicality of behavior observed in an experimental situation always exists. This has reference to such factors as the motivation of the subjects and the degree of tension aroused due to the novelty of the testing situation. In the admin- istration of the performance test, efforts were made to minimize the difference between the behavior exhibited in this situation and that behavior which would occur in trouble shooting on the job. This was done by telling the subjects (a) that they were not expected to solve all the performance test problems, (b) that there were no ''trick' 9 problems, and (c) that the experiment was not for the purpose of evaluation. In general, the interest and cooperation of the subjects were highly satisfactory. Implications for Further Research This investigation has been exploratory in nature. It will have been of value, therefore, if it has been capable of suggesting further research on problems related to the maintenance of electronic equipmento Specif- ically mentioned here are three types of studies which may lead to better understanding of processes involved in trouble shooting and suggest specific practices to be employed in the selection and training of electronics mechanics. -87- Generality of the Ul/BER-1 Performance Test As has already been indicated, research directed toward determining the generality of the Ul/BER-1 as a technique for stimulating trouble- shooting behavior is needed. This research could take one of two specific directions. First, hypotheses such as those that have guided this study could be investigated by means of experiments similar to the present one, but using standard electronic equipment instead of experimental devices, such as the Ul/BER-1 Radio Receiver. This type of research may be capable of identifying strategic aspects of the processes involved in trouble- shooting ^p_ecific equipments. Some of the difficulties which could be expected in using more complex and inaccessible equipment for performance testing purposes were mentioned in Chapter III. These difficulties should not, however, be insurmountable. The second type of investigation of the generality of the Ul/BER-1 as a device for studying trouble-shooting processes would involve perform- ance testing of a single sample of subjects on several different equipments, possibly including one similar to the Ul/BER-1. In this manner the consistency and importance of different aspects of the trouble- shooting process could be investigated with the factor of different types of equipment controlled. If it were to be demonstrated that the Ul/BER-1 or a similar experimental device is comparable with respect to other electronic equip- ment in eliciting typical trouble- shooting procedures, this simpler and more economical device could be used in more definitive investigations with assurance that the results obtained could be generalized. Problem Solving and the Untrained Population Procedures used in the selection of personnel for training in main- tenance occupations might well involve measures of problem- solving abilities. It appears likely that variables other than those currently considered in the untrained population might be capable of predicting success in training for trouble shooting. For example, it would not seem unlikely that general traits which might be called "caution," "flexibility," and "ability to make logical inferences" could be identified with respect to potential trainees and be capable of predicting significant portions of the variance in aptitude for trouble- shooting training. It would again require specific research to determine the usefulness of such an approach to the prediction of success in trouble- shooting train- ing. This research would involve the development of non-equipment oriented tests of problem- solving variables, the application of these tests in a selection battery, and a comparison of the results of these tests with trouble- shooting proficiency at the end of the training program. Experimental Training Programs The usefulness of the problem-solving orientation to the trouble- shooting process could be further investigated by means of experimental training programs based on this conception of the requirements for trouble- shooting proficiency. It is beyond the scope of the present study to analyze current training procedures or to present detailed implications of this orientation for possible revisions of them. The following section does, 88- however, include some implications for training programs based on the results of the experiment reported here and on the assumption that these results can be replicated in more comprehensive investigations. In general, the research suggested here would consist of a comparison of two training programs: one designed specifically to emphasize problem solving and to take into account implications such as those of this study; the other based on current training procedures. The criterion would be trouble- shooting proficiency. Implications for Training Programs Increased knowledge concerning the processes involved in trouble shooting should eventually be reflected in training programs for main- tenance mechanics. The empirical identification of strategic aspects of the trouble- shooting process should supplement, at least, logical analyses of the job requirements for this occupation and assist in the determina- tion of the objectives of such training programs, their content, and teaching methods. The present study appears to indicate that the psychol- ogy of problem solving presents an advantageous orientation to trouble shooting. It is essentially on this basis that the following discussion is offered. It bears repeating , however , that the implications cited here can be only tentative on the basis of the present study. They are offered under the assumption that the results of this study are applicable to a wide variety of electronic equipment, an assumption which is not yet sub- stantiated. Suggested Training Objectives The following list of training objectives appears, on the basis of the empirical evidence of this study, to merit attention in maintenance training programs. The attention any set of educational objectives receives depends upon the school's curriculum and upon individual instructors. Hence, the mere statement of objectives does not guarantee that they serve as guides. However, attempts to attain them are prob- ably more likely to occur in their presence than in their absence. That the following objectives are attainable is suggested by the performance of some of the mechanics involved in the experimental part of this investigation. 1. To develop the ability and predisposition to use logical elimina- tion procedures in trouble shooting. (This objective can be considered to be the core of the list. The remaining ones are, essentially, elabora- tions and extensions of it.) 2 To develop the ability and predisposition to think deductively in the process of trouble shooting, to attack each trouble- shooting problem with a concrete plan of action within which each check performed possesses significance. 3. To develop the ability and predisposition to conceive of electron- ic equipment as systems of "data chains" which control "flow of informa- tion." -89- 4. To develop the ability to analyze circuits in order to determine functional relationships between symptoms and possible causes of malfunctions. 5. To develop the predisposition to utilize "general" checking pro- cedures. 6. To develop the ability to make proper use of test equipment. 7. To develop the particular knowledges of basic electronics facts and principles which are necessary to successful trouble shooting. The second objective of this list has been suggested by Saltz and Moore in connection with an earlier study. On the basis of interviews with trouble shooters, they found that: "The good trouble shooter 'thought out' the problem. "1 The third objective listed above refers to the ability of mechanics to possess an orientation to electronic equipment which will best facilitate logical elimination trouble shooting. The opposite type of conception of such equipment would be that it is composed of a group of relatively unrelated components which have different likelihoods of failure. Objective 4 is included even though the hypothesis that perception of the symptom is an important aspect of trouble shooting was not supported in this study. In interpreting this result it was pointed out that the discrete nature of the sumptoms of the problems in the Ul/BER-1 Performance Test may have been responsible for the lack of demonstrated relation. Furthermore, in their study of trouble shooting, Saltz and Moore did find a significant difference between good and poor trouble shooters with respect to their knowledge of the functioning of the equipment. This knowledge was measured by two tests which concerned the connection between symptoms and possible causes. ^ Objectives 5 and 6 follow directly from the results of the empirical tests of hypotheses 3 and 7 of this study. The final objective of this list relates to hypothesis 1 and implies, furthermore, that there may be some peculiar relationship between basic knowledge and trouble-shooting ability, a relationship that would allow the identification of the particular body of such basic knowledge necessary for successful trouble shooting. Attaining the Objectives In order to attempt to attain training objectives, it is necessary that they serve as guides to procedures used in training programs. The following are a few procedures which may facilitate the attainment of the above type of objectives. Eli Saltz and John V. Moore. A Preliminary Investigation of Trouble Shooting, Technical Report 53-2. San Antonio, Texas: Human Resources Research Center, Lackland Air Force Base, 1953. p. 5. 2 Ibid. -90- In the first place, it would seem that proficiency measurement should become diagnostic with respect to aspects of the trouble-shooting process. Achievement testing which is part of the training program should view that which is called trouble- shooting proficiency as composed of several aspects. Among those would be the components of the problem- solving process identified in this study. In performance testing, the purpose should be to identify inefficient tendencies in the trouble- shooting processes employed by individual students. A simple proficiency score does not serve as a sufficient basis for remedial learning and instruction. The emphasis which in this study has been placed on the trouble- shooting process has not been intended as an argument against the necessity of knowledge of fundamental electronics facts and principles. Conversely, the study has shown that such knowledge is an essential aspect of trouble- shooting proficiency. Hence, the objective stated above indicates that it should receive an appropriate emphasis in maintenance mechanics' train- ing programs. Furthermore, it is recognized that to some extent an , important component of success in trouble shooting any given type of equipment is the mechanic's familiarity with that equipment and his knowl- edge of its peculiar features. Hence, it is expected that emphasis in training programs on aspects of trouble- shooting process will not replace specific familiarization with and training for working with particular types of equipment. However, training in electronics fundamentals and familiarization with specific equipment should be efficient. It may be possible, by means of a study of the trouble -shooting process and other duties required of the maintenance mechanic, to identify the particular knowledges and skills requisite to success in this occupation. This may allow more efficient instruction than occurs when such knowledges are taught in relative isolation of the situations in which they will be used. With respect to the organization of training programs, the above objectives might be implemented by devoting time specifically to the teach- ing of trouble shooting as a generalized process of logical elimination. The implementation of this aspect of training might be a simplified piece of equipment, such as the Ul/BER-1, which is comparable to the types of equipment for which the training is ultimately directed. This generalized trouble- shooting phase of training could not be undertaken until the students possessed some, at least, of the knowledges requisite to working with this vehicle equipment. Specifically, the students should be given opportunity to practice trouble shooting sample malfunctions on this equipment and the instructors should be prepared to observe the procedures used by individual students and be able to point out at what points they deviate from the logical elimination approach. The practice and individual instruction aspects of this suggestion may be critical in the effective teaching of basic principles of efficient trouble- shooting procedures. Another possible advantage of teaching trouble shooting by this "problem- solving method" may be that facts and principles concerning the operation of electronic circuitries would be efficiently made relevant to their application in trouble- shooting tasks. It may even be possible to develop aspects of the basic knowledge requirements most efficiently in this type of context. 91- Later in the training program when the students are studying the operation end unique characteristics of the particular equipment which they will maintain, attention should be given to logical elimination as a trouble- shooting procedure The extent to which this attention need be given would depend upon the extent to which the generalized procedures could be transferred to the specific equipment. Again, the important elements of training would probably be practice in trouble shooting the specific equipment and provision for the assistance of individual trainees, Trouble-Shooting /ids The above discussion which implies that trouble shooting should be taught as a special ability may appear irrelevant in terms of the actual procedures used in maintaining electronic equipment in the field. Accompanying each piece of equipment is a handbook of maintenance instructions which is also called its technical orders. These handbooks generally contain not only information concerning the theory of operation of the equipment, lists of component specifications, and detailed schematic diagrams but also instructions for making alignments, for performing routine checking procedures, and for trouble shooting. The information on trouble shooting frequently contains (a) specific directions for isolating malfunctions in a step-by- step manner and (b) tables showing the relation- ships between possible symptoms and likely causes of trouble. The existence of this information on trouble shooting, however, guarantees neither its intelligibility, its comprehensiveness, or its utilization. Technical orders are usually prepared by engineers. Their style, content, and format seem intended for fellow professional engineers. Theory, and details of construction and equipment action, as well as other kinds of information, surround and interlard specific instructions on what to do and when to do it. Job instructions in these technical orders, although accurate, tend to be incomplete and unclear except perhaps to one who already knows the procedures. 3 It appears then that technical orders do not provide a complete basis for performing trouble-shooting tasks. Furthermore training for trouble shooting, as suggested above, may lead to more efficient practice than training to read and follow technical orders, because the former type of training should produce mechanics who "know what they are doing" even though they find occasion to refer to the handbook for suggestions. The schematic diagrams and lists of component specifications which are contained in these handbooks are indispensable aids to trouble shooting by logical elimination. They contain necessary reference 3 Robert B. Miller. Anticipating Tomorrow's Maintenance job , Research Review 53-1. San Antonio, Texas: Human Resources Research Center , Lackland Air Force Base, 1953. p. 13.. -92- information concerning the arrangements of circuitries and the tolerance values of the various components of the equipment. Suggestions have been made for improving the useability of the handbooks as a whole"* and with special respect to the presentation of this particularly necessary reference information. 5 Summary In part this investigation was intended to illustrate an approach to the determination of certain important objectives of training programs for a particular type of occupation. This approach was empirical and has appeared to be capable of suggesting relevant objectives for programs designed to train maintenance mechanics for trouble- shooting electronic equipment. To the extent that aspects of the trouble- shooting process are contributors to its successful conclusion, it was suggested that they receive attention in the training of potential maintenance mechanics. The discussion of implications of this study with respect to objectives of training programs was prefaced with remarks concerning the limitations of this investigation and with suggestions for further research on problems associated with the need for well trained and efficient maintenance personnel. 4 Ibid. 5 Saltz and Moore, op. cit. -93- CHAPTER VI SUMMARY AND CONCLUSIONS The research described in this dissertation was an attack on problems related to the maintenance of electronic equipment. Because it was ex- ecuted in conjunction with a larger research project supported by the United States Air Force, it was particularly oriented to the maintenance problem of this military establishment. The roles which equipment design and the introduction into general usage of new equipment play in problems of main- tenance were mentioned. Further sources of problems of maintenance were seen to be associated with the selection, training, and proficiency appraisal of maintenance personnel. Trouble shooting, the process of locating sources of trouble in malfunctioning equipment, was identified as constituting one of the most crucial and difficult of the maintenance mechanic's tasks. This trouble- shooting process was taken as the object for this investigation. Specifically, its purposes were (a) to develop a psychological model by which to view trouble-shooting behaviors, (b) to test certain hypotheses taken from this model relative to aspects of the trouble- shooting process, and (c) to illustrate an empirical approach to the identification of important objectives of training programs. The Specific Hypotheses Investigated The objective in trouble shooting is the identification of the particular defective component of a malfunctioning electronic equipment. This is accomplished by means of a series of electrical checks and measurements which provide the mechanic with information concerning the location of the specific defective component. This trouble- shooting process was viewed, in the present study, as a type of diagnostic problem-solving task. The problem-solving orientation to trouble shooting was developed on the basis of a review of selected research and literature on several aspects of problem solving and thinking. This review led to the specification of a hypothetical, model trouble-shooting process. This trouble- shooting prototype was essentially a procedure of logical elimination in which the mechanic first observes the audible manifestation of the trouble, the symptom, and then makes observations on the basis of which to formulate hypotheses concerning the particular area of the equipment in which the trouble lies. After locating the defective area or stage of the equipment, he continues to make observations and formulate more restrictive hypotheses until the specific defective component is located. From this problem-solving orientation to the trouble- shooting pro- cess were taken nine hypotheses concerning aspects of the process. These hypotheses were stated explicitly in Chapter II. The first hypothesis concerned the relationship between mechanics' knowledge of basic electron- ic facts and principles and their trouble-shooting ability. Hypotheses 2 through 8 concerned the contribution or detraction which specific compo- nents of the trouble-shooting process make to success in the trouble- -94- shooting task. The final hypothesis was related to the consistency and efficacy of different overall methods of attack to trouble-shooting prob- lems. Empirical evidence in support of these hypotheses would provide a basis for the acceptance of the logical elimination trouble- shooting prototype as the most efficient procedure. The Method of the Study The manner in which these nine hypotheses were tested involved the administration of a trouble-shooting performance test and a written test of basic electronics knowledge to a sample of forty radio mechanics. The performance test utilized a specially constructed, superheterodyne radio receiver, called the Ul/BER-1. For each of eight performance test problems, the examiner inserts a defective component into the receiver and observes the trouble- shooting behaviors of the subject as he attempts to locate the trouble. There is a twenty minute time limit for each pro- blem. If the subject correctly identifies the faulty component within this time limit he '"passes" the problem. While observing trouble- shooting behavior on the Ul/BER-1 Performance Test, the examiner records the subject's statement of the symptom of the trouble and all of the relevant overt behaviors which he exhibits in his attempt to locate the defective component. The recording of behaviors is facilitated by the use of the Observation Record which is shown as Figure 1 in Chapter III. The information on this Observation Record is transferred to the Analysis Record Sheet which is shown as Figure 2 in Chapter III. These complete records of trouble-shooting behaviors are then subjected to interpretation in such a manner that the final records of trouble- shooting behavior shown on the completed Analysis Record Sheets are amenable to analytical abstraction and quan- tification. For the present study, completed Analysis Record Sheets constituted the records of trouble- shooting processes which were used for testing the several specific hypotheses. The Basic Electronics Knowledge Test consists of 130 multiple- choice items. The test is divided into two parts. The first part consists of eighty items covering general electronic facts and principles common to many types of electronic equipment. The second part consists of fifty items written with specific reference to the circuitry and prin- ciples of operation of the Ul/BER-1 Radio Receiver. The Ul/BER-1 Performance test was administered to a sample of forty students of an Air Force radio maintenance school by two Air Force electronics technicians. The Basic Electronics Knowledge Test was administered approximately two weeks following the administration of the performance test. -95- Testing the Hypotheses and Results In Chapter IV were presented the statistical methods and the results of testing the several specific hypotheses concerning the trouble- shooting process. Also presented there were some gross results descriptive of the instruments used in the investigation. Of particular interest was the internal consistency of the performance test pass-fail score, the number of items which the subject passed. This coefficient was .63, indicating a highly satisfactory degree of internal consistency for a test of but eight items. These preliminary descriptive results also led to the conclusion to utilize the total scores on the Basic Electronics Knowledge Test as the measures of knowledge of basic electronics facts and principles. The first hypothesis tested was that knowledge of basic electronics facts and principles was a necessary, though not sufficient, condition for trouble- shooting success. A product-moment coefficient of correlation of .55, significant at the one percent level, between the total scores on the test of basic electronics knowledge and the pass-fail performance test scores supported the contention that basic knowledge is essential for trouble- shooting success. Several statistically significant coefficients of partial correlation between pass-fail scores and various observable components of the problem- solving process with the influence of basic knowledge test scores removed were taken to be indicative that basic knowledge is not a sufficient condition for success in trouble shooting. Several hypotheses concerning components of the problem- solving process were subjected to empirical tests. These observable components of the process were labeled : perception of symptom , tendency to perform general checks , first hypothesis accepted, wrong hypothesis behavior (which included wrong hypotheses entertained and perseveration ) , use of obtained information (which included plus clues and minus clues) , errors , and duplications . The manner in which the occurrence of the behaviors so labeled were abstracted from the Analysis Record Sheets and quantified was described. With respect to each of these behavioral components of the problem- solving process, three types of statistical tests were made. These tests provided evidence concerning (a) the occurrence of individual differences relative to the behaviors, (b) the contribution, or detraction, which they made to success in solving individual trouble- shooting problems, and (c) the relationships between their occurrence and total success on the performance test problems. A coefficient of internal consistency, based on an analysis of variance and denoted as r^, was described and used as a test of the hypothesis that there exist true individual differences in the extent of occurrence of the various behaviors. Fisher's exact probability method for two-by-two contingency tables was used for testing hypotheses concerning the contribution which the components make to success on individual trouble- shooting problems. Where the requisite assumptions appeared to be met, the product-moment coefficient of correlation was used to test for relationship between total occurrence of the behavioral components and total pass-fail scores. If these assumptions were not met, chi-square was used for this purpose. -96- On the basis of these statistical tests, which are summarized in Table 29 in Chapter IV, the following general conclusions concerning these components of the problem- solving process were reached. 1. There exist individual differences with respect to the occurrence of each of the specified components, except perception of the symptom , wrong hypotheses entertained , and minu s clue behavior . In other words , for seven of the ten identified components there was shown to be a significant degree of consistency in their occurrence over the eight performance test problems. The conclusion is that it is possible to identify and measure these observable aspects of trouble- shooting behavior. 2. Although there are differences among problem situations or "troubles" that affect the importance of the different components, the following ones generally play a strategic role in determining success or failure on individual trouble- shooting problems: first hypothesis accepted r wrong hypotheses entertained , perseveration, plus clues , minus clues, and errors . 3. With the exception of perception of the symptom and duplications , the extent of occurrence of each of the observable components of the prob- lem-solving process are related in the manner predicted to general trouble- shooting success. In other words, eight of the components inves- tigated are strategic with respect to success in trouble- shooting problems. The final hypothesis investigated was that mechanics utilize dif- ferent overall methods of attack to trouble- shooting problems and that these different methods of attack possess differing degrees of efficacy. Five types of overall methods of attack were identified and the problem- solving processes employed by the forty subjects were classified accord- ing to these patterns. These methods of attack were called, logical elimination , persistence , caution, random , and perseveration. Coefficients of internal consistency for the frequency of occurrence of each of these patterns of response indicated that mechanics differ with respect to their tendency to utilize different methods of attack on trouble- shooting problems. On the basis of an analysis of variance of pass- fail scores for the five groups of subjects which resulted when they were classified according to their most characteristic methods of attack, it was concluded that different methods of attack are characteristic of mechanics at different levels of proficiency. An analysis of covariance of pass-fail performance test scores with group means adjusted for differences in basic knowledge test mean scores failed to indicate conclusively that the efficacy of different methods of attack is independent of mechanics' knowl- edge of basic electronic facts and principles. The results did suggest, however, that the independence of this relation of basic knowledge be retained for further investigation. 97- The Study as an Approach to Determining Training Requirements One purpose of this investigation resided in the belief that it rep- resents an empirical approach to the identification of important aspects of the proficiency required for the performance of maintenance tasks. As such, it may constitute a methodology for determining objectives of training programs for maintenance occupations. Implications of the investigation were discussed from this point of view. The analysis of these implications was not extensively elaborated for two reasons. First, the primary purpose of the study, the identification of strategic aspects of trouble-shooting proficiency, did not seem to require a more extended discussion. Furthermore , the limitations associated with the conclusions concerning these aspects of proficiency imposed a restriction upon the validity of a more extended discussion of these implications. Several implied objectives were, however, specifically listed and briefly discussed. In implementing these objectives, practice and individual instruction were suggested. Furthermore, the necessity of diagnostic proficiency measurement was stressed. Other implications mentioned concerned further study on problems related to maintenance tasks. The three types of studies specifically suggested concerned further study of aspects of the trouble- shooting process, the use of diagnostic measurement in the selection of personnel for maintenance training programs, and the exploration of training pro- grams based on the requirements of trouble shooting viewed as a prob- lem-solving task. General Conclusions In general, this investigation was an attempt to gain a better understanding of the task of trouble shooting electronic equipment and, thereby, to provide a foundation upon which future research on this aspect of the maintenance mechanic's job and upon which practices related to the selection, training, and proficiency measurement of electronics maintenance mechanics can be based. The following general conclusions appear to be justified. 1. The trouble-shooting situation can realistically and profitably be viewed as a type of diagnostic problem- solving task which requires (a) knowledge of fundamental electronic facts and principles as a base, and (b) on a specific problem., a course of action guided by an adequate orientation, and by succeedingly more restrictive hypotheses, for- mulated on the basis of careful observation and the logical elimination of possible alternative causes. The final and most specific hypothesis will eventually be shown to be correct. 2. Strategic elements of the requirements for successful trouble- shooting processes can be empirically identified and subjected to analytical investigation. -98- 3. The empirical methodology represented in this investigation is capable of generating hypotheses on which to base further research and practice aimed at the improvement of procedures used in the selection, training, and proficiency testing of members of occupations similar to that concerned with the maintenance of electronic equipment. In a society being transformed by the developments of science and technology, new occupations are being created to meet new requirements. Specific scientific study directed toward the creation of maximum effi- ciency in the execution of duties associated with these occupations is needed. This investigation was an attempt to contribute to the meeting of this need. -99« BIBLIOGRAPHY Alpert , Augusta. The Solving of Problem Situations by Preschool Chil- dren . Contributions to Education, No. 323. New York: Bureau of Publications, Teachers College, Columbia University, 1928. 69 p. Bedell, Ralph C. The Relationship between the Ability to Recall and the Ability to Infer in Specific Learning Situations . Kirksville , Mo. : Journal Printing Co., 1934. 55 p. Billings, Marion L. "Problem-Solving in Different Fields of Endeavor." American Journal of Psychology 46 :2 59-72; April 1934. Bloom, Benjamin S. and Broder , Lois J. Problem Solving Processes of College Students. Chicago: University of Chicago Press, 1950. 109 p. Burack, Benjamin. "Methodological Aspects of Problem-Solving." Progressive Education 30:134-38; March 1953. Burack, Benjamin. "The Nature and Efficacy of Methods of Attack on Reasoning Problems." Psychological Monographs 64:313:1-26; 1950. Cornell, Francis G. and Damrin, Dora E. A Preliminary Report on the Development of a. Functional Knowledge Test Battery for the Measurement of Proficiency of Radar Mechanics , Research Bulletin 52-30. San Antonio, Tex.: Human Resources Research Center, Lackland Air Force Base, 1952. 29 p. Cornell, Francis G. , Damrin, Dora E., and Saupe , Joe L. The AN/APQ-24 Radar Mechanics' Proficiency Testing Study , The Tab Test : A_ Group Test Simulating Performance Behavior for the Measurement of Proficiency in Diagnostic Problem Solving Tasks. Urbana : Bureau of Educational Research, University of Illinois, 1954. 44 p. (Typewritten). Cronbac'h, Lee J. "Coefficient Alpha and the Internal Structure of Tests." Psychometrika 16:297-334; September 1951. Damrin. Dora E., Gardner, Floyd M. , and Tucker, Ronald F. Basic Elec- tronics Knowledge Test- - Radio Receivers . Urbana: Bureau of Educational Research, University of Illinois, 1953. 22 p. (Mimeographed). Dewey, John. How We Think. (Revised Edition.) Boston: D.C. Heath and Co., 1933 . 301 p." " Dunker, Karl. "On Problem Solving." (Translated by Lynne S. Lees.) Psychological Monographs 58:270:1-113; 1945. Dunker, Karl and Krechevsky, Isadore. "On Solution-Achievement." Psycho l ogical Review 46:176-85; March 1939. Durkin, Helen E. "Trial and Error, Gradual Analysis, and Sudden Reorganization, An Experimental Study of Problem Solving." Archives of Psychology 30:210:1-83; May 1937. -100- Evans, Rupert N. and Smith, Lyman J. A Study of Performance Measures of Trouble Shooting Ability on Electronic Equipment. Urbana: College of Education, University of Illinois, 1953. 137 p. Fisher, Ronald A. Statistical Methods for Research Workers. (Seventh Edition) Edinburgh; Oliver and Boyd, 1938. 356 p. Gaier, Eugene L,. "What the Teacher Needs to Know about the Role of Knowledge in Problem-Solving." Progressive Education 30:138-41; March 1953. Glaser, Robert Bo, Damrin, Dora E., and Gardner, Floyd M. The Tab Item : A Technique for the Measurement of Proficiency in Diagnostic Problem Solving Tasks . Urbana; Bureau of Research and Service, University of Illinois, 1952. 13 p. Heidbreder, Edna. "An Experimental Study of Thinking." Archives of Psychology ll:73sl-175; October 1924. Hoyt, Cyril. "Test Reliability Estimated by Analysis of Variance." Psychometrika 6:153-60; June 1941. Johnson, Donald M. " A Modern Account of Problem Solving." Psychol- ogical Bulletin 41:201-29; April 1944. Johnson, Donald M. "Problem Solving and Symbolic Processes." Annual Review of Psychology , Vol. 1. (Edited by Calvin P. Stone.) Stanford: Annual Reviews, Inc., 1950. p. 297-310. Kuder, G. F. and Richardson, M. W. "The theory of Estimation of Test Reliability." Psychometrika 2:151-60; September 1937. Luchins, A. S. "Mechanication in Problem Solving--The Effect of Einstellung." Psychological Monographs 54:6:1-95; 1942. Maier, Norman R. F. "An Aspect of Human Reasoning." British Journal of Psychology 24:144-55; October 1933. Maier, Norman R. F. "Reasoning and Learning/' Psychological Review 38:332-46; July 1931. Maier, Norman R. F. "Reasoning in Humans : I. On Direction." Journal of Comparative Psychology 10:115-43; April 1930. Maier, Norman R. F. "Reasoning in Humans: III. The Mechanisms of Equivalent Stimuli and of Reasoning." Journal of Experimental Psychol- ogy 35:349-60; October 1945. Maier, Norman R. F. "Reasoning in Rats and Human Beings." Psychol - ogical Review 44 :365-78; September 1937. Maier, Norman R. F. "The Behavior Mechanisms Concerned with Prob- lem Solving." Psychological Review 47 :43-58; January 1940. -101. Marks, M. R. "Problem Solving as a Function of the Situation" Journal of Experimental Psychology 41:74-80; January 1951. Miller, Robert B. Anticipating Tomorrow's Maintenance Job , Research Review 53-1. San Antonio , Tex.: Human Resources Research Center, Lackland Air Force Base, 1953. 21 p. Miller, Robert B. and Folley, John D. , Jr . The Validity of Maintenance Job Analysis from the Prototype of an Electronic Equipment , Part I_: AN/APQ-2 4 Radar Set . Pittsburgh: American Institute for Research, 1952. 127 p. Miller, Robert B. , Folley, John D., Jr. and Smith, Philip R. Systematic Trouble Shooting and the Half- Split Technique, Technical Report 53-21. San Antonio, Tex.: Human Resources Research Center , Lackland Air Force Base , 1953. 16 p. National Bureau of Standards. Tables of the B inomial Probability Distri b ution , Applied Mathematics Series, No. 6. Washington, D. C: United States Government Printing Office., 1949. 387 p. Ruger, H. A. "The Psychology of Efficiency."' Archives of Psychology 2:15^-88; June 1910, """""' Saltz , Eli and Moore, John V. A Preliminary Investigation of Trouble Shooting , Technical Report 53-2. San Antonio, Tex.: Human Resources Research Center, Lackland Air Force Base, 1953. 10 p. University of Illinois, Bureau of Educational Research. Research on the Development of a_ Radar Mechanic's Functional Knowledge Test Battery. Quarterly Report No. 10, Air Force Contract No. 33(038)-13236. Urbana° Bureau of Educational Research, University of Illinois, 1953. 34 p. (Mimeographed). Vinacke , William E. The Psychology of Thinking. New York: McGraw- Hill, 1952, 392 p. Walker , Helen M. and Lev, Joseph. Statistical Inference. New York: Henry Holt, 1953. 510 p. -102- APPENDIX A HANDBOOK OF MAINTENANCE INSTRUCTIONS for RADIO RECEIVER UI/BER AF Contract No. 33(038)-13236 (PRELIMINARY BOOKLET - NOT FOR GENERAL DISTRIBUTION) University of Illinois Bureau of Educational Research Urbana, Illinois December - 1952 Section 1 GENERAL DESCRIPTION Radio set Ul/BER-1 is a standard superheterodyne receiver constructed in such a manner as to make it readily usable for performance testing of radio mechanics. The receiver power input is approximately 80 watts at 115 volts, 60 cycles. The frequency range covered is the standard broadcast band, 560 to 1600 kc. The receiver is constructed on a standard 17 x 13" x 3" aluminum chassis. The components and wiring are on top of the chassis to permit ready access and observation. Most components are mounted on terminal strips in order to permit rapid changes when testing. The tube complement is as follows : 1 - 5U4 rectifier. 1 - 6SA7 1st detector and local oscillator. 3 - 6SK7 1 R.F., 2 I.F. amplifiers. 1 - 6H6 2nd detector and AVC diode. 1 - 6J5 1st audio amplifier. 1 - 6K6 audio power output amplifier. Section 2 OPERATION Note : Wait at least one minute after turning set off before turning it on again. If this is not done the fuse is liable to blow because of large charging currents in the filter condensers. (1) Plug the loudspeaker cable into the output jack. (2) Connect a short piece of wire (2 or 3 feet) to the antenna binding post. (3) Insert the power plug into the wall socket. (4) Turn on power switch. (5) Set R.F. gain control for maximum gain (all the way clockwise), -105- (6) Set A.F. gain control for proper volume level. (7) Tune to desired frequency. Because of the open construction, the second and third harmonics of the intermediate frequency can be heard in the output when the input is tuned to them. This is normal behavior for this set ahd should be ignored when encountered. Section 3 THEORY OF OPERATION The Ul/BER-1 is an 8 tube receiver having one r.f. amplifier stage, a frequency converter, two stages of i.f. amplification, a detector - AVC stage, two stages of audio amplification and a rectifier. (See shematic diagram.) The radio frequency band is from 560 to 1600 kc and the intermediate frequency is 455 kc. All circuits involved are almost entirely standard. Therefore, only a few of the circuit details are described here. The frequency converter is a mixer- oscillator combination commonly found in broadcast receivers. The grid leak of the oscillator section is split into two sections: R^ and R7. The larger of the two, R^,, provides most of the grid leak function while R7 is for measuring purposes. A voltmeter placed across R7 will indicate the amount of grid current being drawn. Resistor R5 is used to reduce the feedback so as to obtain the optimum oscillator voltage level. The r.f. gain control simultaneously controls the cathode bias of the r.f. amplifier and the two i.f. amplifiers. For reception of voice signals the r.f. gain control is normally set for maximum gain and the automatic volume control (AVC) controls the gain. The AVC action of the Ul/BER-1 is unusually effective. A change of input signal level from 10 microvolts to 10,000 microvolts results in an output level change of only 8 decibels. This is due to the fact that the AVC is applied to three stages, the r.f. amplifier and both i.f. amplifiers. In order to prevent AVC action on weak signals, an AVC delay is produced by returning the cathode of the AVC diode to a point that is several volts above ground. This delay bias is provided by R26 an d ^19 in the cathode circuit of the first a.f. amplifier. Resistor R£g provides negative feedback around the second audio amplifier. The feedback reduces any distortion generated in this stage and also tends to make the operation of the output stage independent of the load. The r.f., converter, and i.f. stages are very heavily decoupled by RC filters. Each of these stages has a separate filter for its screen, cathode, and AVC circuits. The r.f. amplifier and the converter share 106- one plate circuit filter, and the two i.f. stages share another. No decoupling is necessary in the audio section. This set has unusually high gain for a broadcast receiver. As a result the noise between stations is higher than normally encountered. When improperly tuned, it is possible to hear the 10 kc whistle of two stations beating against each other although the two stations themselves might be inaudible. This too is due to the high gain and disappears with proper tuning . The output jack is of the shorting type. This prevents generation of excessive voltage in the primary of the output transformer in case the loudspeaker plug is removed while the set is in operation. Section 4 R.F. AND I.F. ALIGNMENT PROCEDURE Preliminary . The AVC circuit must be disabled in order to get true output indication. This may be accomplished by connecting a jumper lead from R22» th e 1-megohm AVC decoupling resistor, to ground. The r.f . gain control must be set at a level sufficiently low to insure against blocking of the amplifiers. An A. C. voltmeter or decibel meter should be connected from the plate of the output tube to ground. The speaker plug must be in the jack. Note: If output meter does not have a built in blocking condenser, a condenser must be placed in series with the meter and the plate of the 6K6. I.F. Alignment . The output of a signal generator set at 456 kil- ocycles, with suitable audio modulation is applied to the signal grid of the second id. tube. The signal generator output is increased until a readable deflection is obtained on the output meter. The trimmer condensers (screwdriver adjustment) on the output i.f. can are then tuned for maximum output as indicated by the output meter. The secondary should be tuned first and then the primary. The signal generator output is then moved to the signal grid of the first i.f. tube and the above procedure repeated for the interstage i.f. can. Finally, the signal is fed into the signal grid of the mixer tube and the input i.f. can is tuned. The alignment of the i.f. channel is now complete. -107- R.F. Alignment . The signal generator output is applied to the antenna input. The signal generator is then set at some low frequency- Note : R.F. alignment should not be attempted unless it has been proved necessary. (about 600 kilocycles) and the tuning control of the Ul/BER-1 set to the same frequency. The oscillator padder is then tuned for maximum output. The signal generator and receiver are then tuned to some frequency at the upper end of the band (1400 kilocycles) and the oscillator trimmer condenser (small screwdriver adjustment on third section of main tuning condenser) is tuned for maximum output. The oscillator is now aligned. With the same settings of receiver and signal generator the two remaining trimmer condensers on the main tuning condenser are tuned for maximum output, thereby aligning the r.f. stages. Section 5 TEST EQUIPMENT The test equipment included with radio receiver Ul/BER-1 is: 1 - volt-ohmmeter. 1 - Heathkit signal generator. Assorted test leads. Voltmeter. The voltmeter is a standard multi-test type meter. Signal Generator . The signal generator provided is of the commercial kit - construction type. It has been modified to include blocking condensers in both the audio and r.f. outputs so that signals may be safely applied at points which are at potentials up to 300 volts D. C. Operation of Signal Generator, 1. Plug line cord into 110V 60 cycles A.C. outlet. For audio output: (a) Plug cable into "A.F. output" jack ("A.F. in" is never used). (b) Turn "Modulation" switch to "int". -108- (c) Turn "RF output" clockwise past click. Pilot light lights when power is on. (d) Connect black lead to desired point, stranded lead to ground. (e) Adjust "AF in-out" for desired audio level. For R.F. output: (a) Plug cable into "R.F. out" jack. (b) Turn "Modulation" switch to "int" for 400 cycle modulation; "ext" position gives unmodulated output. (c) Select desired frequency band (A, B, C , D, E) by turning band switch at bottom center. (d) Select desired frequency by means of vernier control on large center dial. (e) Turn power on by turning "R.F. output" control clockwise until pilot light comes on. (f) Connect black lead to desired point, stranded lead to ground. (g) Adjust output level by means of "R.F. steps" control and "R.F. output" control. 109- Section 6 APPROXIMATE VALUES OF PERTINENT VOLTAGES (1000 Ohms/Volt Voltmeter) (R.F. Gain Max. - No Signal In) Tube Point Pin No. or Check Point Voltage Power Supply (5U4) 1st R.F. (6SK7) Mixer (6SA7) Plates of Rectifier Cathode B + Plate Screen Cathode Plate Screen (osc. plate) 4 and 6 2 or 8 Junction R31 and R32 8 6 5 3 4 Voltage Across R 7 (Measures Osc. Grid I.F. (both) (6SK7) 1st Audio (6J5) 2nd Audio (6K6) Plate Screen Cathode Plate Cathode Plate Screen Cathode Current) 8 6 5 3 8 3 4 8 350 V.A.C . +350 V.D.C. +260 V.D.C. +240 V,D.C. + 100 V.D.C* +3 V.D.C. +250 V.D.C. +80 V.D.C. -1 V.D.C. +240 V.D.C. + 100 V.D.C* +3 V.D.C. + 100 V.D.C. +5.5 V.D.C. +260 V.D.C. +265 V.D.C. + 18 V.D.C. *I.F. and R.F. screen voltages as given are nominal, no-signal voltages. These will vary nearly 100 volts as signal is tuned in due to AVC action. Note: (l) AVC voltages can only be measured with a vacuum tube voltmeter. (2) Decreasing R.F. gain control setting will increase most voltages. -110- Section 7 TUBE BASE DIAGRAMS 5U4G 6H6 6SA7 6SK7 H -111- 70 > O O i - > -< 70 o m c= o H m < o m > 70 o XI c > M 2 W m 70 i ;°<=s- (J) T- ^ I©' 3) roi- -=£ o 3 ro T, or i i -J»C o ro O O = "L — ^j 1_ ■o -C_J- -££—*■ - o ©1 ©i A o TO ro o ro O rC=>i o ro ro 4 i 1 00 __J ,__J \ -112- o o 1- o t- * oe o X UJ o o o o t- o UJ o or Q I 1- UJ O t- < H + _i Or < J UJ o O o a o ai o « 7- * — Z o or UJ -1 UJ 111 1 t> o * IE UJ -J X Ul O >- CD * oe -113- APPENDIX B FAMILIARIZATION PROCEDURE USED IN ADMINISTRATION OF UI/BER-1 PERFORMANCE TEST "TROUBLE AREAS" FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS "CLUES" DEFINED FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS FAMILIARIZATION PROCEDURE FOR Ul/BER-1 NOTE: This sheet is for your personal use. Feel free to make any notations on it that you wish, and to refer to it during testing if the need arises. I. General Operation 1 . Turn set on. 2. Set R.F. gain at maximum (full CW). 3. Set A.F. gain at some intermediate point. 4. Tune through several signals and note effect of R.F. and A'.F. gain controls on different signal strengths. II. Check Harmonics 1. Tune set to about 910 kc with full R.F. gain and medium A.F. gain. 2. Note excessive whistle and squeal; this point is the second harmonic of the I.F. 3. Tune set to about 1360 kc. 4. Note excessive whistle and squeal; this point is the third harmonic of the I.F. NOTE : THIS BEHAVIOR IS NORMAL FOR THIS SET AS A RESULT OF ITS OPEN CONSTRUCTION III. Voltage Measurements Measure the following voltages and record them on the line at the right: 1. 2nd A.F.; plate 6. 1st I.F.; plate 2. screen 7. screen 3. cathode 8. cathode 4. 1st A.F. ; plate 9. 2nd I.F-;plate 5. cathode 10. screen -117- FAMILIARIZATION PROCEDURE FOR Ul/BER-1 (continued) 11. cathode 14. cathode 12. R.F. amp.; plate 15. Mixer; plate 13. screen 16. screen 17. Place a check (v) beside all voltages which change with setting of R.F. gain control. 18. Place a cross (X) beside all voltages which change with signal strength. 19- Measure and record the oscillator grid current. (Voltage across R 7 ) 20. Is this affected by gain setting? By signal strength? 21. Measure B+ at output of filter. 22. Measure B+ at cathode of rectifier. 23. Attempt to measure the AVC voltage across one of the AVC filter condensers. Note that the meter loads the circuit excessively and that no meaningful reading can be obtained . IV. Signal Injection 1. Insert audio signal on 2nd audio grid and 1st audio grid. Note gain. 2„ Insert I.F. signal on 2nd I.F. grid, 1st I.F, grid, and mixer grid respectively. 3. Note approximate stage gains. 4. Repeat procedure with AVC disabled and note strong AVC action. 5. Insert R.F. signals at antenna. 6. Note discrepancies between signal generator calibration and dial markings on set. - 118- "TROUBLE AREAS" FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS Problem Trouble Area 1 The entire power supply including the B+ line. Plates of y \, y% > y 3 > V4 > V D anc * ^7 are considered part of the trouble area only when checks on them are immediately followed by checks in power supply or B+ line. 2 The entire 2IF stage including the manual RF gain control. 3 The entire audio section, including voltage and power amplifier stages. i 4 The manual RF gain control extended to the cathodes of Vj, V3 , and V4 and including the cathode bypass condensers of these tubes. 5 The entire 21F stage not including the manual RF gain control. 6 The AVC line including the detector and AVC plates and extending to the grids of Yj , V2, V3 , and V4 and including the AVC filter capacitors in these stages. 7 The entire RF section including the RF amplifier and mixer stages. 8 The entire IF section including the first and second IF stages and the manual RF gain control. -119- "CLUES" FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS Problem General Clues Specific Clues a. Any initial voltage check along the B+ line , immedi- ately following statement of symptom. a. Any plate or screen grid or cathode voltage check. b. Any DC voltage check along the B+ line. a. Signal injection at the detector stage. (IF or AF), b. Signal injection at the 21FG. (IF only). c. Physically touching the entire tube complement or a sizeable portion thereof. a. Signal injection on 2AFG, (AF only). b. Signal injection on 1AFG. (AF only). a. Signal injection at Det Stage- (IF or AF). b. Signal injection at 21FG. (IF only). c. Any DC voltage checks within the power supply filter which bracket R32 d. Any resistance checks within the power supply filter which bracket R3 1 a. Feeling V4 for heat. b. c. d. f. b. a. b. c. d. f. Voltage check on 21F Sg. Voltage check on 21FK. Voltage checks on either side of R15 or Ri£>. Resistance measurement of 2 IF filament, with tube removed from socket. Substitution of V4. All signal injections which precisely bracket C-2Q, i.e., 2AFG, R 2 7-C20 R 29 etc. (AF only). Shunting C20 with a suit- able capacitor. DC voltage check at 21FSg. DC voltage check at HFSg. DC voltage check at RFSg. DC voltage check at 21FK or any DC voltage check in 2 IF cathode line. DC voltage check at 11FK or any DC voltage check in 1 IF cathode line. DC voltage check at RFK or any DC voltage check in RFK cathode line. (continued on next page) -120- CLUES" ' FOR THE EIGHT Ul/BER-1 PERFORMANCE TEST PROBLEMS (continued) Problem General Clues Specific Clues a. a. b. a. b. c. Signal injection at Det stage. (IF or AF). Signal injection at 21FG. (IF only). Shorting out AVC line. Disabling AVC line generally, on AVC plate side of circuit. Signal injection on 11FG. (IF only). Signal injection on MixP. (IF or RF). Signal injection on MixG. (IF or RF). g. DC voltage check in RF gain control circuit. h. All resistance checks in the above circuits to ground. a. All DC voltage checks in 21FSg circuit. , b. All DC resistance checks in 21FSg circuit to ground. a. Shorting junction Rjq.C^ to ground. b. Shunting C9 with suitable capacitor. a. b. c. d. e. f. h. 1. DC voltage checks on RFP. DC voltage checks on RFSg. DC voltage checks on MixP. DC voltage checks on MixSg. DC voltage checks on plate or screen grid circuits of either stage. DC voltage checks on either side of Rg. Resistance checks which clearly indicate open in plate circuits. Resistance checks which point directly to Ro as faulty component, or which closely bracket R9. Shunting R9 with suitable resistance a. Pulling V2 and V3. b. Signal injection on 11FG, 11FP, 21FG, or 21FP. (IF only). a. Shunting C12 with suitable capacitor. 121- APPENDIX C CONTINGENCY TABLES USED FOR TESTING HYPOTHESES CONCERNING THE CONTRIBUTION OF CERTAIN COMPONENTS OF THE PROBLEM- SOLVING PROCESS TO THE SOLUTION OF TROUBLE-SHOOTING PROBLEMS Contingency Tables Hypothesis Prob- 2 lem (Sympt om) 3 (General Che cks) 4 (First Hypothesis) 1. w R 30 10 40 R 24 16 Total 30 10 40 Total 24 16 No Yes Total P F W 7 5 R 23 5 Total P F P F Total 3 1 4 No 27 9 36 Yes 30 10 40 Total 30 10 Total 2. W Total P F 12 W 7 4 28 R 17 12 40 Total P F P F Total 5 1 6 19 15 34 24 16 40 24 16 Total 40 40 Total 11 29 , 40 3. w R Total No Yes Total P F W 2 10 R 18 10 Total P 3 F 6 17 14 31 R 20 20 40 Total P F Total 11 11 No 20 9 29 Yes 20 20 40 Total 20 20 Total 9 4. W Total P F 12 W 4 9 28 R 23 4 40 Total P F 27 13 27 13 P F 1 1 26 12 27 13 27 13 Total 40 40 Total 2 38 40 Total P F 13 W 3 13 27 R 19 5 40 5. W R 12 4 16 R 4 28 Total 22 18 40 Total 5 35 No Yes Total Total P 10 F 14 P F Total 3 3 6 19 15 34 22 18 40 22 18 Total 24 Total 16 W 24 R 40 6. W No Yes Total Total P 1 F 7 P F Total 2 17 3 18 5 35 P F 3 25 2 10 5 35 Total 8 32 40 19 21 40 Total 28 12 40 7. W R 28 9 Total 30 10 No Yes Total P F W 3 5 R 27 5 Total P 2 F 1 P F Total 1 1 30 9 39 30 10 40 30 10 Total 3 37 40 Total 8 W 32 R 40 8. W R Total No Yes Total Total P 2 F 3 19 16 21 19 P F Total 7 8 15 14 11 25 21 19 40 P F 16 13 5 6 21 19 Total 5 35 40 Total 29 11 40 125- Contingency Tables (continued) Hypothesis Prob- lem ("W 'rong 5-1 Hypotheses) 5-2 (Perseveration) 6- (Plus 1 Clues ) 1. No 24 5 Yes 6 5 Total 30 10 P F Total No 5 1 6 Yes 1 4 5 Total 6 5 11 0,1 2, 3 Total P F P 8 F 7 22 3 30 10 Total 29 No 12 5 17 11 Yes 12 11 23 40 Total 24 16 40 Total 15 0. 1 25 2, 3 40 2. P F No JO 6 Yes 2 5 Total 12 11 Total P F P 10 F 13 Total 23 0, 1 P 5 F 19 Total 24 0, 1 P 2 F 7 Total 9 0, 1 P 4 F 18 14 3 17 2, 3 15 1 16 2,-3 25 6 31 2, 3 18 24 16 Total Total lb No 7 Yes 23 Total 40 3. No 17 5 Yes 3 15 Total 20 20 Total P F P F Total 13 10 13 No 5 5 Yes 13 15 18 Total 20 20 Total 4. 22 No 24 4 18 Yes 3 9 40 Total 27 13 40 Total P F P F Total 3 3 6 6 6 3 9 12 27 13 Total 28 No 18 12 Yes 4 18 40 Total 22 18 40 5. No Yes Total Total P F P F 4 8 10 4 18 22 18 Total 18 22 40 Total 12 No 10 Yes 22 Total Total 22 0, 1 P 1 F 33 Total 34 0, 1 18 2, 3 4 2 6 2, 3 40 6. No 3 8 Yes 2 27 Total 5 35 Total P F P F Total 2 17 19 10 10 2 27 29 5 35 Total 11 No 25 5 29 Yes 5 5 40 Total 30 10 40 7. P F P F No 5 2 Yes 3 Total 5 5 Total P 3 F 7 27 3 30 10 Total 30 10 40 Total P F 7 No 15 13 3 Yes 2 5 10 Total 17 18 Total 10 0, 1 P 1 F 13 Total 14 30 2, 3 20 6 26 40 8. P F No 4 1 5 Yes 17 18 35 Total 21 19 40 Total 21 19 Total Total 28 7 35 40 -126- Contingency Tables (continued) Prob- 6-2 lem (Minus Clues) Hypothesis 7 (Errors) (Duplications) 1. 1,2 Total P F 22 8 10 30 10 Total 22 18 40 2. 1,2 Total P F 19 2 5 14 24 16 Total 21 19 40 3. 1,2 Total P F 19 16 1 4 20 20 Total 35 4 40 4. 1, 2 Total P F 18 4 9 9 27 13 Total 22 18 40 5. 1, 2 Total P F 21 7 1 11 22 18 Total 28 12 40 6. 1,2 Total P F 5 33 2 5 35 Total 38 2 40 7. 1,2 Total P F 26 5 4 5 30 10 Total 31 9 40 8. 1, 2 Total P F 19 17 2 2 21 19 Total 36 4 40 1,2 Total P F 19 3 11 7 30 10 Total 22 18 40 1, 2 Total P F 10 2 14 14 24 16 Total 12 28 40 1, 2 Total P F 18 13 2 7 20 20 Total 31 9 40 1, 2 Total P F 16 3 11 10 27 13 Total 19 21 40 1, 2 Total P F 11 5 11 13 22 18 Total 16 24 40 1, 2 Total P F 5 17 18 5 35 Total 22 18 40 0,1 2 Total P F 17 1 13 9 30 10 Total 18 22 40 1, 2 Total P F 15 10 6 9 21 19 Total 25 15 40 0,1 ,2,3 Total P F 22 3 8 7 30 10 Total 25 15 40 1 ,2,3 Total P F 16 3 8 13 24 16 Total 19 n 40 1 ,2,3 Total P F 9 6 11 14 20 20 Total 15 25 40 0,1 2,3 Total P F 10 5 17 8 27 13 Total 15 25 40 0,1 2,3 Total P F 10 7 12 11 22 18 Total 17 23 40 0,1 2,3 Total P F 4 18 1 17 5 35 Total 22 18 40 0,1 2,3 Total P F 21 4 9 6 30 10 Total 25 15 40 0,1 2,3 Total P F 14 10 7 9 21 19 Total 24 16 40 127-