• " : 
 
 SURVEY RELIABILITY 
 AND INTERVIEWER 
 COMPETENCE 
 
LI B HARY 
 
 OF THE 
 
 UNIVERSITY 
 
 Of ILLINOIS 
 
 339.4 
 In82s 
 no .4 
 cop. 2 
 
Return this book on or before the 
 Latest Date stamped below. 
 
 Theft, mutilation, and underlining of books 
 are reasons for disciplinary action and may 
 result in dismissal from the University. 
 University of Illinois Library 
 
 JUN 26 
 
 I 
 
 NOV 191979 
 
 APR 8 
 
 979 
 
 L161— O-1096 
 
Digitized by the Internet Archive 
 
 in 2011 with funding from 
 
 University of Illinois Urbana-Champaign 
 
 http://www.archive.org/details/surveyreliabilit04hauc 
 
SURVEY RELIABILITY AND INTERVIEWER COMPETENCE 
 
Studies in Consumer Savings 
 
 No. 1 Collecting Financial Data by Consumer Panel Tech- 
 niques. Robert Ferber 
 
 No. 2 An Investigation of Response Error. John B. Lansing, 
 Gerald P. Ginsburg, and Kaisa Braaten 
 
 No. 3 The Composition of Consumer Savings Portfolios. 
 Henry J. Claycamp 
 
Studies in Consumer Savings, No. 4 
 
 CONSUMER SAVINGS PROJECT 
 INTER-UNIVERSITY COMMITTEE FOR 
 RESEARCH ON CONSUMER BEHAVIOR 
 
 SURVEY RELIABILITY 
 AND INTERVIEWER 
 COMPETENCE 
 
 Mathew Hauck and Stanley Steinkamp 
 
 PUBLISHED BY THE BUREAU OF ECONOMIC 
 AND BUSINESS RESEARCH • UNIVERSITY 
 OF ILLINOIS • URBANA ■ DECEMBER 1964 
 
$3.00 
 
 © 1964 by the Board of Trustees of the University of Illinois 
 Library of Congress Catalog Card No. 64-64875 
 Manufactured in the United States of America 
 

 PREFACE 
 
 This monograph is the fourth in a series of technical reports presenting 
 findings of the Consumer Savings Project of the Inter-University Com- 
 mittee for Research on Consumer Behavior. The first and second publica- 
 tions in this series presented methodological results of the project, while 
 the third publication related to consumer savings behavior. 
 
 This fourth monograph presents some further methodological findings, 
 focusing on the role of the interviewer in affecting the quality of data 
 obtainable on personal interview surveys. The validation of financial 
 information reported by the respondents represented a unique opportunity 
 to evaluate interviewer effectiveness in obtaining accurate and complete 
 financial data. The selection, training, and supervision procedures em- 
 ployed on the project studies are described and evaluated in this mono- 
 graph, with the validation information being used to provide objective 
 measures of interviewer effectiveness. On the basis of these findings, Dr. 
 Steinkamp and Mr. Hauck present some thought-provoking hypotheses 
 on procedures which may help improve data on future financial surveys. 
 This project has been financed by a grant from the Ford Foundation, 
 with supplemental assistance from the United States Department of Agri- 
 culture. Financial support for additional work currently under way has 
 been provided by the National Science Foundation and by the United 
 States Department of Labor. Robert Ferber, Research Professor of 
 Economics at the University of Illinois, is director of the project. 
 
 The members of the Inter-University Committee for Research on 
 Consumer Behavior are: 
 
 Lincoln Clark, New York University, Secretary-Treasurer 
 
 Robert Ferber, University of Illinois 
 
 George Katona, University of Michigan 
 
 Theodore Newcomb, University of Michigan 
 
 Howard Raiffa, Harvard University 
 
James Tobin, Yale University 
 Guy Orcutt, University of Wisconsin, Chairman 
 Raymond Goldsmith was a member of the Committee until he left in 
 June, 1963, for an OECD assignment in Paris. 
 
 The monographs in this series are research reports. The Inter-Univer- 
 sity Committee, as sponsor of this research, makes every effort to ensure 
 both the quality of the reports and their orientation toward meeting a 
 real need. Nevertheless, the findings reported in this way summarize 
 conclusions arrived at by project staff and do not necessarily represent the 
 individual or collective views of the members of the Inter-University 
 Committee. 
 
 Guy Orcutt, Chairman 
 
 Inter-University Committee 
 
 for Research on Consumer Behavior 
 
ACKNOWLEDGMENTS 
 
 The writers wish to express their gratitude to Henry Claycamp, Ann 
 Harper, and Laura Hauck for reading early drafts of the monograph and 
 for making many helpful comments and suggestions. We also wish to 
 thank the many research assistants who have worked on the Consumer 
 Savings Project and aided in the preparation of materials included in 
 this monograph. The interviewers on whose work the analysis is based 
 aided in the preparation of the monograph through their frank and 
 thoughtful comments on problems encountered in the field. 
 
 Above all, we wish to acknowledge the important contribution of 
 Robert Ferber, Director of the Consumer Savings Project, in offering 
 advice and guidance in the analysis of the material and the writing of the 
 monograph. Finally, we want to acknowledge the contribution of the 
 editor, Mrs. Virginia Speers, in readying the manuscript for publication. 
 
 However, criticism of the monograph or inadequacies in its presenta- 
 tion should be directed toward the authors, who take full responsibility 
 for its contents. 
 
 Mathew Hauck 
 Stanley Steinkamp 
 Champaign, Illinois 
 October 19, 1964 
 
CONTENTS 
 
 I. Background and Introduction 1 
 
 II. Interviewer Variability 11 
 
 III. Interviewer Selection 26 
 
 IV. Interviewer Training 53 
 
 V. Interviewer Supervision 66 
 
 Appendixes 
 
 A. Interviewer Qualifying Examination: Study of 
 
 Farm Family Finances 91 
 
 B. Interviewer Agreement : Study of Farm Family Finances . 103 
 
 C. Interviewer Report Form: Study M3 105 
 
 D. Questionnaire: Study of Farm Family Finances . . . 108 
 
LIST OF FIGURES 
 
 1. Interviewer Application Form: Study M4 32 
 
 2. Evaluation Checklist: Study M4 38 
 
 3. Definition of Variables Available from the Edwards Personal 
 Preference Schedule 40 
 
 4. Reference Evaluation Form: Study M4 45 
 
 5. Interviewer Guide: Study M4 55 
 
 6. Outline of "Pointers on the Interview Situation": Study M4 56 
 
 7. Schedule Outline for Training Sessions: Study M4 58 
 
 8. Reminder Notes for Wave 2: Study M3 69 
 
 9. Editor's Weekly Report: Study M3 70 
 
 10. Spot-check Post Card: Study M3 72 
 
 11. Time and Expense Record : Study M2 83 
 
 LIST OF TABLES 
 
 1. Distribution of Accounts, by Size and by Interviewer 
 
 Group, Study M3 18 
 
 2. Results of Contacts for All Interviewers, by Size of 
 
 Account, Study M3 18 
 
 3. Pick-up Rate (Response Rate X Completeness Rate), by Size of 
 Account and by Interviewer Group, Study M3 19 
 
 4. Reporting of Different Assets and Debts, Study M4 20 
 
 5. Record of Applicants 28 
 
 6. Norms for Selected Populations and Average Interviewer . 
 
 Scores on the Edwards Personal Preference Schedule 42 
 
 7. Distribution of Project Interviewers, by Selected 
 Characteristics and by Study 49 
 
 8. Average Cost per Interview on All Waves of All Studies 85 
 
I. BACKGROUND AND INTRODUCTION 
 
 The survey method is unique in its ability to obtain information con- 
 cerning individual decision-making units. The variety of information 
 obtained about each unit permits the study of relationships between 
 different aspects of financial behavior. Thus, information about income, 
 assets, debts, savings, and the like permits generalizations concerning the 
 economic behavior of the whole population or of groups in the popula- 
 tion. Knowledge so gained gives insights into the nature of aggregate 
 relationships and forms the basis for simulations of the economy. Yet 
 this advantage of survey methods may be largely nullified if the informa- 
 tion obtained is not accurate and reliable. 
 
 One approach to evaluating the accuracy and reliability of survey 
 data is the comparison of aggregates derived from surveys with those 
 derived independently by the Departments of Commerce, Labor, and 
 Agriculture and other sources. Such comparisons indicate underreporting 
 of certain holdings, especially liquid assets and personal debt. 1 However, 
 these comparisons frequently involve heroic efforts through adjustments 
 to make the data comparable, although they offer little insight into the 
 reasons for the discrepancies. 
 
 The accuracy and completeness of sample surveys can also be evalu- 
 ated at the level of the individual sample members. Using institutional 
 records, information about the true holdings of sample members can be 
 compared with the information obtained at the sample addresses. This 
 approach gives insights into the reasons for bias in aggregates prepared 
 from survey data. Such bias may result from failure to obtain an inter- 
 view at the address of the sample member or may result from the persons 
 interviewed giving incomplete or inaccurate information. In addition, 
 bias in the sample variance, which affects the reliability of tests of 
 significance, can be evaluated. 
 
 The Consumer Savings Project 
 
 The Consumer Savings Project was set up primarily to evaluate the 
 reliability of data obtained by sample surveys. The methodological aims 
 
 1 Robert Ferber, The Measurement and Control of Errors: Savings Data, 
 Chapter 3, manuscript in preparation. 
 
2 Survey Reliability and Interviewer Competence 
 
 of the project were first to evaluate the completeness and accuracy of 
 financial survey data, and second to develop methods of improving the 
 completeness and accuracy of information obtained from consumer finan- 
 cial surveys. The project was carried out by means of a series of panel 
 operations conducted in different parts of the country and at different 
 times. 
 
 Some of these operations were conducted by the Survey Research 
 Center of the University of Michigan, and others were conducted under 
 the direct supervision of the project staff at the University of Illinois. 
 Each of the operations had specific objectives relating to some aspect of 
 collecting financial data from consumers. In most instances the accuracy 
 and completeness of the survey data was evaluated in terms of records 
 made available through the cooperation of financial institutions. In 
 many of the studies these records served as a frame for sample selection. 
 
 The reason for the sequential procedure in conducting the panel 
 studies was to permit the design of each succeeding operation to be 
 modified in accordance with the results obtained from the prior studies. 
 In particular, knowledge of the results of the experiments in one study 
 could serve as the basis for new, and, it was hoped, more useful experi- 
 ments in a later study. Results obtained regarding the accuracy of finan- 
 cial survey data are discussed in detail elsewhere. 2 A brief summary of 
 validation results for time deposits in one study is sufficient to provide 
 some idea of the magnitude of the errors which occurred. It represents 
 an extreme though important example of the kinds of errors which may 
 occur. The existence of the validated savings account was ascertained for 
 about 60 percent of the sample addresses where contact was made. In 
 addition, the total amount reported by respondents who mentioned vali- 
 dated accounts was a little over two-fifths of the total holdings of 
 persons at the sample addresses where contact was made. Possibly of 
 greater importance was the finding that the computed sampling variances 
 were biased estimates of the true variances. 3 As a result, tests of signifi- 
 cance had limited meaning. Results of this nature serve to emphasize 
 the importance of the second methodological objective of the project, 
 that of improving the accuracy and reliability of survey data. 
 
 The accuracy and completeness of information obtained by the use of 
 survey techniques can be improved through two interrelated approaches. 
 One is to improve survey methods. This approach emphasizes improved 
 
 2 See J. B. Lansing, G. P. Ginsburg, and Kaisa Braaten, An Investigation of 
 Response Error, Studies in Consumer Savings No. 2 (Urbana: University of Illi- 
 nois, Bureau of Economic and Business Research, 1961); Robert Ferber, The 
 Measurement and Control of Errors: Savings Data, op. cit. 
 
 3 See Robert Ferber, ibid. 
 
Background and Introduction 3 
 
 methods of pre-interview contact; of selecting, training, and super- 
 vising interviewers; and of questionnaire construction. The second ap- 
 proach is to develop techniques for correcting and adjusting information 
 obtained from population samples and validation samples in terms of 
 each other. 4 
 
 The present monograph is concerned with the first of these ap- 
 proaches, specifically with interviewer selection, training, and supervision, 
 and with the extent to which survey reliability appears to depend on the 
 interviewer variability. In the latter respect, the availability of validation 
 data presented a unique opportunity to evaluate interviewer performance 
 and to use an objective measure of the quality of information obtained in 
 the interview as a basis for such an evaluation. 
 
 The Individual Studies 
 
 This monograph is based on four panel studies with which the authors 
 were involved. These studies constituted a major part of the over-all 
 project. 5 
 
 In each of the four studies, the respondents were interviewed five 
 times at regular intervals three or four months apart. In this way, prob- 
 lems of obtaining savings information on a continuing basis could also 
 be studied. 
 
 The unit studied was the savings unit. A savings unit was defined as 
 one or more related persons living in the same dwelling unit who pooled 
 half or more of their income and savings. A dwelling might therefore 
 have had more than one savings unit. Savings of minors were combined 
 with those of their legal guardians whether or not the minors had separate 
 savings. This definition had two implications for the data collected. 
 First, the savings unit was a decision unit, thus facilitating the study of 
 saving behavior. Second, the head of the savings unit or some member 
 of it usually had complete and accurate information about the holdings 
 of the unit as well as knowledge about changes in the holdings of the 
 savings unit. 
 
 The studies were undertaken in four different areas in the Midwest. 
 The first study, referred to as Ml, was made in a large metropolitan 
 center. The second study (M2) was undertaken in another metropolitan 
 center. The third study (M3) was conducted in a smaller urban center, 
 and the last study (M4) was conducted in three farm counties in the 
 Midwest. A brief summary of each of these studies follows. 
 
 4 For further details on this approach, see ibid. 
 
 5 The other studies were carried out by the Survey Research Center, University 
 of Michigan. See J. B. Lansing et al., op. cit. 
 
4 Survey Reliability and Interviewer Competence 
 
 Study Ml 6 
 
 Study Ml was intended to test the feasibility of obtaining income, 
 savings, and other financial data from consumers on a continuing basis. 
 More specifically, the objectives were (1) to estimate costs and response 
 rates on consumer financial panel operations, (2) to gain insights into 
 the problems of obtaining financial data on a continuing basis and to 
 explore different methods of doing so, and (3) to explore the use of 
 internal means of assessing reliability of these data. 
 
 A probability sample of 300 names was selected from the population 
 of the metropolitan area in the fall of 1956. Census tracts were classified 
 into three income strata based on data from the 1950 Census Tracts 
 volume for the area. A systematic selection of census tracts from each 
 stratum was made with a disproportionately high number of tracts from 
 the higher income strata. Names and addresses were selected from each 
 sample tract through the use of voter registration lists and telephone 
 directories. 
 
 Each of the sample families was sent a personal letter on University 
 of Illinois stationery notifying it of the general nature of the survey and 
 the forthcoming call by a named interviewer. Personal advance letters 
 were also sent at the beginning of each succeeding wave of interviews. 
 The letters for subsequent waves differed for some respondents since they 
 were used as a means of answering questions raised during the interviews 
 of the previous wave. 
 
 The actual interviewing was largely subcontracted to two survey or- 
 ganizations. For purposes of obtaining comparative information, some 
 interviews were assigned to two interviewers hired directly by the project. 
 
 Each panel member was asked numerous questions about his financial 
 situation, about record-keeping, and about the use of budgets. In addi- 
 tion, one segment of the sample was asked to keep records or diaries 
 provided by the project. On succeeding interview waves, the panel mem- 
 ber's financial situation was reviewed and changes in it were recorded. 
 The interviews were conducted with the head of the savings unit or with 
 the member of the family who was familiar with the savings unit's 
 finances. 
 
 Study M2 
 
 Study M2 was begun in mid- 1958 with a sample of 308 names. The 
 sample was an area probability sample with disproportionate representa- 
 
 For complete details on this study, see Robert Ferber, Collecting Financial 
 Data by Consumer Panel Techniques, Studies in Consumer Saving No. 1 (Urbana: 
 University of Illinois, Bureau of Economic and Business Research, 1960). 
 
Background and Introduction 5 
 
 tion given to high-income sample members. Over 40 percent of the 
 savings units were estimated to have had incomes in 1959 exceeding 
 $10,000; the corresponding 1950 Census estimate for the sample area was 
 10 percent. 
 
 Respondents were interviewed a total of five times over the period of 
 a year and a half. The respondent for each savings unit was asked ques- 
 tions about all the assets and debts of the savings unit. Included were 
 checking accounts, savings accounts, life insurance holdings, annuities, 
 pensions, brokerage accounts, and the ownership of businesses. The field 
 operation was conducted by interviewers hired, trained, and supervised 
 by a field director. 
 
 A number of experiments relating to the different interview ap- 
 proaches were conducted in the course of the five waves of interviews. 
 When more than one experiment was conducted on a single wave, the 
 sample was divided by systematic selection into as many random segments 
 as there were combinations of experiments and each segment received a 
 different combination — a factorial design. The experiments of the first 
 wave were the following, with the response rates for each experiment in 
 parentheses. 7 
 
 1. Use of a short letter of introduction (76 percent) versus use of a 
 long detailed letter (90 percent). 
 
 2. Use of a highly structured questionnaire (72 percent) versus use of 
 no questionnaire, i.e., unstructured interviews (87 percent). 
 
 3. Request for dollar amounts in the interview (77 percent) versus 
 no request for dollar amounts (81 percent) . 
 
 4. Request for assets first in the interview (72 percent) versus request 
 for debts first (73 percent). 
 
 Two types of financial holdings reported by the persons interviewed 
 were validated, personal debt and life insurance holdings. Secondary-type 
 validation was employed in each case. Holdings reported by the sample 
 members were sent to cooperating financial institutions who reported the 
 correct amount held by the sample members on some specified date, 
 usually the date of the interview. This amount was then used to evaluate 
 the accuracy of the amount reported by the sample member. Secondary 
 validations provided no information about sample addresses where no 
 contact was made or where the respondent refused to be interviewed. 
 In addition, they provided little or no information about holdings which 
 interviewed sample members failed to report. 
 
 7 Further details on the experiments are presented in Robert Ferber, The 
 Measurement and Control of Errors: Savings Data, op. cit., Chapter 13. 
 
6 Survey Reliability and Interviewer Competence 
 
 Study M3 
 
 In interviews undertaken in 1959 in a smaller urban center, panel 
 members in Study M3 were interviewed five times at three-to-four-month 
 intervals. The study began with 316 interviews on Wave 1 and finished 
 with 205 interviews on Wave 5. The following split-run experiments were 
 conducted on the first wave, with the response rates for each experiment 
 shown in parentheses. 8 
 
 1. Use of a regular advance letter (81 percent) versus use of the 
 same letter but including an attempt to involve the panel member by 
 seeking his suggestions and aid in the conduct of the interview (81 per- 
 cent) . 
 
 2. Use of a questionnaire form on which answers were recorded (77 
 percent) versus use of an unstructured type of interview with reminder 
 cards to aid the interviewer (82 percent) . 
 
 3. Request for dollar holdings of assets and debts on the day of the 
 interview (73 percent) versus request for only the changes that had taken 
 place over the past three months (81 percent) . 
 
 4. Request for information on all assets and debts (80 percent) versus 
 request for information on specific assets only (78 percent). 
 
 Experiments conducted on later waves included the following: 
 
 1. Rotation of interviewers among selected panel members on the 
 third wave. 
 
 2. Request to each respondent on the third wave to sign a form 
 authorizing the institution to report the current balance in his savings 
 account. 
 
 3. Use of a mail questionnaire sent to part of the sample on the 
 fourth wave. As with the earlier study, a factorial design was employed. 
 
 The initial sample of 316 persons was selected from the time deposit 
 records of savings institutions in the area. When an interview was ob- 
 tained, the balance on the day of the interview was reported to the project 
 by the cooperating institution. For a sample address where there was no 
 contact or where there was a refusal, a midpoint date for the field opera- 
 tion was selected and the institution reported the person's balance for that 
 day. Interviewer effectiveness was evaluated in terms of these validation 
 data. 
 
 Two secondary validations were also undertaken. One involved life 
 insurance holdings. As in the earlier study, all policies were validated 
 when the respondent mentioned the name of a cooperating institution. 
 The other secondary validation involved demand deposits mentioned by 
 
 8 Ibid. 
 
Background and Introduction 7 
 
 respondents at two of the financial institutions. Because of the volatile 
 nature of demand deposits and because of the existence of float (checks 
 which have not cleared the institution), a balance reported by the 
 validating institution for the date of the interview could not be directly 
 comparable with that reported in the interview. However, corrections 
 made for these differences indicated a fairly good correspondence between 
 reported and true holdings. 
 
 Study M4 (The Farm Study) 
 
 The farm study was begun late in 1960 in three counties in the Mid- 
 west. Half of the roughly 400 farm panel members were interviewed 
 three times at about six-month intervals, and the other half were inter- 
 viewed five times at about three-month intervals. Two experiments were 
 conducted on the first wave and as in earlier studies, a factorial design 
 was used. 9 These experiments and the response rates for each (in 
 parenthesis) were as follows: 
 
 1. Use of a letter of introduction which made a straightforward 
 appeal based on the need for information about how farmers handle 
 their finances (90 percent) versus use of another letter with the same 
 basic approach but in addition an appeal to sample members to aid in 
 improving the data-collection procedure, as in Study M3 (90 percent). 
 
 2. Use of one questionnaire requesting the farmers to report their 
 holdings of assets and debts, including the farm as a business, on the day 
 of the interview (86 percent) versus another questionnaire requesting 
 information about the change in holdings over the previous three months 
 (93 percent) . 
 
 In this study, a somewhat more extensive pretest was used to refine 
 the questionnaires and to evaluate interviewer performance. The sample 
 was selected from institutional records of five different types of assets and 
 debts: time deposits, demand deposits, short-term debt, intermediate- 
 term debt, and long-term debt. As in Study M3, names selected were 
 checked in telephone books and county directories, and only those so 
 located were used. 
 
 It was found that cooperation differed substantially between sample 
 members selected from one validation source and those selected from 
 other sources. As a result, data from individuals selected from different 
 validation sources could be combined only to a limited extent in the same 
 analysis. This severely limited the number of comparable contacts made 
 by each interviewer and accordingly limits the scope of the analysis of 
 these data presented in this monograph. 
 
 9 Ibid. 
 
8 Survey Reliability and Interviewer Competence 
 
 Organization of the Monograph 
 
 Knowledge that each savings unit owned a certain amount of a spe- 
 cific asset or debt on the date of the interview provided several interrelated 
 criteria for evaluating interviewers. Utilizing such data, an attempt is 
 made here to understand and to measure the relationship between inter- 
 viewer effectiveness and techniques of interviewer selection, training, and 
 supervision. In Chapter II interviewer variability and the problems of 
 measuring interviewer performance will be discussed. Interviewer selec- 
 tion methods will be presented in Chapter III. The procedures employed 
 in the project will be reviewed and evaluated, followed by a discussion 
 of a quantitative method used in selecting interviewers. The training of 
 interviewers will be covered in Chapter IV. In Chapter V interviewer 
 supervision and the controls used to maintain high response rates with 
 quality data will be discussed. The identification of effective interviewers, 
 based on their interviewing performance when no validation data are 
 available, will be included in the discussion. 
 
 An effort will be made throughout the monograph to identify effec- 
 tive interviewers, a problem which has been the subject of much pre- 
 vious research. 10 However, a direct comparison of the findings of such 
 research with those discussed here presents one or more of the following 
 problems. 
 
 1. Much of the earlier research was concerned with interviewer effec- 
 tiveness in attitudinal studies. In such studies, interviewer biasing of 
 respondent answers or bias in recording them is of primary importance. 
 In factual studies of the kind reported here, this type of bias is less im- 
 portant and other types become more important. In the absence of 
 tie-in studies, generalizing from one to the other type of study must be 
 limited. 
 
 2. Earlier studies were frequently based on validation information 
 such as age, voter registration, or ownership of a driver's license, about 
 which respondents are less sensitive than about financial holdings. The 
 project studies were based primarily on validated information about which 
 respondents are often highly sensitive. When less sensitive information 
 was validated, it was observed that interviewer variability was much 
 smaller. This suggests that factors influencing interviewer effectiveness 
 in seeking sensitive data may not be so important when less sensitive 
 data are sought. 
 
 10 A good summary of past research is presented in Herbert H. Hyman, Inter- 
 viewing in Social Research (Chicago: University of Chicago Press, 1954). Also, 
 see Robert Ferber and H. G. Wales, A Basic Bibliography on Marketing Research, 
 AM A Bibliography Series No. 2 (rev.) (Chicago: American Marketing Associa- 
 tion, 1963), Section IX. 
 
Background and Introduction 9 
 
 3. The criteria of interviewer effectiveness in many earlier studies 
 were based on the percentage of persons contacted who refused or on the 
 number of times information was not ascertained, refused, or reported as 
 not known in the interview. The present analysis indicates that these 
 show little relationship to the criteria of effectiveness employed here. 
 Other criteria such as ratings by field supervisors, by the field director, or 
 by coders similarly show little correlation with the measures of effective- 
 ness employed. A criterion employed on some studies is interviewer 
 variability relative to the average variability of interviewers. Such analysis- 
 of- variance designs, as is readily admitted by those using them, 11 do not 
 discriminate between good and poor interviewers. 
 
 4. In some cases, the field staffs were very different from those em- 
 ployed in the project research. For example, Caplow points out that the 
 field staffs used in many of the earlier studies were poorly trained, had 
 high turn-over rates, and had limited supervision. 12 The method of 
 selecting the field staff may also have influenced findings. If the inter- 
 viewers selected were homogeneous with respect to some characteristic, 
 that characteristic as well as related characteristics could not have been 
 analyzed. 
 
 5. Quota sampling was frequently employed in these earlier studies. 
 As a result, the nature of the sample may have influenced the factors 
 discovered as well as the weight attached to them. 
 
 In view of these differences, the findings of earlier studies will be 
 reported only when they are relevant in terms of these five dimensions. 
 
 Since 1954 some studies have been made with which the project 
 studies may be directly compared. One of these made in the Netherlands 
 compared the completeness and accuracy with which savings accounts 
 were reported to a staff of interviewers. 13 Another made by the Federal 
 Reserve Board and reported by Broida described the accuracy of reports 
 of new car debt. 14 However, neither of these studies attempted to identify 
 the characteristics of the more effective interviewers. 
 
 11 For example, Robert H. Hanson and Eli S. Marks, "Influence of the Inter- 
 viewer on the Accuracy of Survey Results," Journal of American Statistical Associ- 
 ation, Vol. 53, No. 283 (September, 1958), p. 648. 
 
 12 Theodore Caplow, "The Dynamics of Information Interviewing," American 
 Journal of Sociology, Vol. 62, No. 2 (September, 1956), p. 167. 
 
 13 W. Horn, "Reliability Survey: A Survey on the Reliability of Response to 
 an Interview Survey," Het PTT-bedrijf, Vol. 10, No. 3 (October, 1960), pp. 
 105-56. 
 
 14 Arthur L. Broida, "Consumer Surveys as a Source of Information for Social 
 Accounting: The Problems," in Conference on Research in Income and Wealth, 
 The Flow-of-Funds Approach to Social Accounting: Appraisals, Analysis and 
 Applications, Studies in Income and Wealth, Vol. 26 (Princeton: Princeton 
 University Press for the National Bureau of Economic Research, 1962), pp. 335-81. 
 
10 Survey Reliability and Interviewer Competence 
 
 The findings of the project studies made by the Survey Research 
 Center at the University of Michigan relating to interviewers and of 
 interaction between interviewers and respondents are integrated into the 
 present discussion. 15 
 
 15 J. B. Lansing, et al., op. cit. 
 
II. INTERVIEWER VARIABILITY 
 
 Measurement of Interviewer Performance 
 
 Criteria of interviewer effectiveness employed in surveys for which 
 validation data are not available include the contact rate (percentage of 
 eligible assignments contacted), the response rate (percentage of con- 
 tacts interviewed), measures based on ambiguous answers in the ques- 
 tionnaire, or measures of promptness in completing assignments, coopera- 
 tiveness, and related factors. These criteria, while they may reflect failure 
 to obtain an interview and failure to obtain meaningful answers, do not 
 reflect respondent failure to mention holdings or the accuracy with which 
 they give the amounts of their reported holdings. The latter, however, 
 may introduce as serious a bias into survey data as the former. 
 
 Interviewers may also be evaluated by using an analysis of variance 
 method in which the variation in results obtained by different interviewers 
 is compared with the variation between respondents queried by the same 
 interviewer. 16 However, with this approach "it is not possible to state 
 that one result represents 'good' performance and another result 'bad' 
 performance. We can state that Interviewer A's performance differed 
 from that of Interviewer B, but we do not know whether the direction 
 of difference was towards 'better' or towards 'worse'." 17 In addition, this 
 approach reflects only the differences in the bias of interviewers without 
 saying anything about the amount of bias. For example, "between inter- 
 viewer variance" may be no larger than "within interviewer variance" 
 where all interviewers introduce the same, but large, amount of bias. 
 Without validation information about the true characteristics of the 
 respondents, the amount of bias cannot be known. 
 
 In this chapter, criteria of interviewer effectiveness based on valida- 
 tion data are defined and discussed. Using these criteria, interviewer 
 differences are measured. 
 
 In the present analysis, an interviewer's assignment will be defined 
 as all persons who maintain their home at the sample address. Since the 
 problem of replacing movers in the sample is a sampling problem, this 
 
 10 For example, see Robert H. Hanson and Eli S. Marks, loc. cit., pp. 635-55. 
 17 Ibid., p. 648. 
 
 11 
 
12 Survey Reliability and Interviewer Competence 
 
 definition of assignments is the one applicable for evaluating interviewer 
 effectiveness. 
 
 The interviewer may have failed to obtain information about a sample 
 member for a number of different reasons. First, he may not have made 
 contact with a responsible person at the sample address because some 
 people repeatedly sent children or servants to the door to say that "no 
 one else is at home." 18 A more likely situation was that the sample mem- 
 ber was away from home at the times the interviewer could make a call, 
 especially when the interviewer was not allowed to telephone for appoint- 
 ments as was the case on the first wave of the various studies. Clearly the 
 one situation did not reflect interviewer effectiveness, though the other 
 may have indicated insufficient effort to contact the sample member at 
 different times of the day. 
 
 Second, the interviewer may have made contact with a responsible 
 member of the savings unit but may have failed to obtain information 
 about the savings unit because he was refused an interview. Here, inter- 
 viewer performance became important. It was the interviewer's job to 
 explain the nature of the study, its purpose, its importance, and the 
 confidential treatment of the data and to answer any questions which 
 might arise. 
 
 Third, although an interview was obtained, the information elicited 
 from the respondent may have been incomplete. This may have occurred 
 because the respondent forgot about a holding, because he did not 
 understand what information was wanted, or because he wished to con- 
 ceal the existence of a holding. In each case, the interviewer's perform- 
 ance was likely to be a relevant factor. In the first instance, the 
 interviewer may have been hurrying the respondent or not using sufficient 
 probes. In the second instance, the interviewer may not have been ask- 
 ing the questions in such a way that the respondent understood what was 
 wanted. In the last case, numerous factors relating to interviewer tech- 
 nique and personality may have explained the respondent's withholding 
 of information. 
 
 Fourth, even where the holding was reported there may have been an 
 error in the amount reported. Inaccuracy of report may have occurred 
 because of memory error or because of intentional distortion. Both of 
 these were subject to interviewer influences. Memory error could have 
 been eliminated through the use of records. Even when records were not 
 used, the accuracy of recall was likely to be a function of the motivation 
 of the respondent. 
 
 18 An excellent discussion on this is to be found in H. Lawrence Koss, "The 
 Inaccessible Respondent: A Note on Privacy in City and Country," Public Opinion 
 Quarterly, Vol. 27, No. 2 (Summer, 1963), pp. 269-75. 
 
Interviewer Variability 13 
 
 Interviewer effectiveness in such situations as these four depends 
 somewhat on different aspects of interviewer performance. The following 
 criteria relate to these aspects. 
 
 1. Contact rate (K). The percentage of eligible assignments in which 
 the interviewer makes contact with a responsible person. The inter- 
 viewer's persistence in making a contact is measured by this rate. 
 
 2. Response rate (R). The percentage of times an interviewer 
 
 obtains interviews at sample addresses where contacts are made, i.e.. 
 
 Number of interviews „ . , • , • 
 
 rrz : ? . Since a contact must be either an interview or a 
 
 Number of contacts 
 
 r . . . . , t Number of refusals ,_, . 
 
 refusal, the response rate is also equal to 1 — — = - . Ihis 
 
 Number of contacts 
 
 rate reflects interviewer effectiveness at the door. 
 
 3. Completeness rate (C). The completeness rate is the percentage of 
 interviews in which the validated holding is mentioned by the respondent. 
 This rate reflects in part interviewer effectiveness in the interview. How- 
 ever, it is not independent of the response rate. A low response rate may 
 imply that the respondents interviewed are more likely to be cooperative 
 than is the case with a high response rate. In a sense, the interviewer with 
 a low response rate can be thought of as disposing of his uncooperative 
 sample members at the door and interviewing only the relatively coopera- 
 tively ones, therefore obtaining a higher completeness rate. 
 
 4. Accuracy rate (A). For a savings account that is reported, this 
 rate measures the accuracy with which the amount is reported, arbi- 
 trarily defined as the percentage of respondents who cite amounts within 
 plus or minus 5 percent of the true amount. A range of 10 percentage 
 points was selected because even among the most cooperative project 
 respondents there was a tendency to fail to report interest payments which 
 had not been added to the passbook. 
 
 When change information is requested, the accuracy rate is defined 
 as the percentage of respondents reporting change in the balance of the 
 account within plus or minus 25 percent of the true change. The use of 
 different standards for change and holdings data reflects the fact that 
 change is generally of a smaller magnitude than holdings and thus is 
 likely to yield a larger accuracy ratio for a given amount of error. 19 The 
 accuracy rate is dependent on the response and the completeness rates. 
 The interviewer with low response and completeness rates is likely to 
 
 1S An alternative to using the accuracy rate for evaluating interviewer effective- 
 ness in obtaining accurate data would be the average accuracy ratio, based on 
 cumulation of dollar totals. However, it is not employed because of the dispropor- 
 tionate impact on the average of a single wealthy respondent who reports inac- 
 curately. 
 
14 Survey Reliability and Interviewer Competence 
 
 have only the most cooperative respondents left to report on the amount 
 of their holdings or change. As a result, on a priori grounds, one would 
 expect higher levels of accuracy. 
 
 From these basic rates other rates can be computed. One over-all 
 criterion of interviewer effectiveness could be the product of all four of 
 these measures: K X R X C X A. This rate reflects the percentage of 
 times an interviewer obtains, relative to the number of his total eligible 
 assignments, reports of the amounts of the validated accounts to within 
 plus or minus 5 percent of the true amount. 
 
 The over-all criterion actually selected was determined by the nature 
 of the studies. A policy was employed of requiring each interviewer to 
 make at least five attempts to contact a sample member before turning 
 in a noncontact report. When an interviewer was unable to make contact 
 the sample member was reassigned to a different interviewer. In Study 
 M3, on all except one reassignment, the second interviewer was also 
 unable to make contact. Analysis of the 13 noncontacts in this study 
 revealed that they were evenly distributed among most of the interviewers. 
 For these reasons, it was felt that noncontacts did not reflect interviewer 
 ineffectiveness, and the contact rate should not be included in the cri- 
 terion of effectiveness. Hence, the over-all measure of interviewer effec- 
 tiveness was based on sample addresses where contact was made with a 
 responsible person. 
 
 An experiment undertaken in Study M3 assigned "change" question- 
 naires to half of the sample and "holding" questionnaires to the other 
 half. Subsequent analysis revealed that errors in reporting the amount 
 of change were not comparable with errors in reporting amount of the 
 holding on either an absolute or a relative basis. As a result, a criterion 
 which included the accuracy rate could be applied to one or the other 
 half of the sample at a time, but not to the whole sample. In view of the 
 small average number of contacts (17) made by each interviewer, it was 
 decided to exclude the accuracy rate from the criterion of over-all effec- 
 tiveness. When the contact rate and the accuracy rate are excluded, the 
 criterion of the interviewer effectiveness employed in this analysis is the 
 pick-up rate (P) which is the percentage of contacts in which the in- 
 terviewer picked up the validated account (the product of the response 
 rate and the completeness rate, i.e., P — R X C) . 
 
 It should be stressed that the value of this or related criteria depends 
 on the accuracy of the information obtained from the validation source, 
 which is not necessarily assured. Inaccuracies may have occurred be- 
 cause an employee at the validating institution recorded the balance 
 for the wrong date or recorded it incorrectly for the correct date. A 
 second type of error may have occurred in matching accounts reported 
 
Interviewer Variability 15 
 
 by respondents with those reported by the institution. To reduce this 
 possibility, the names of all financial institutions with whom the respond- 
 ent had assets or debts were requested in the interview. The impression 
 gained from working with the validation process was that very few errors 
 could be attributed to the validating institution or to the matching 
 process. Moreover, errors discovered in the validation procedure on 
 subsequent waves were corrected in the data for earlier waves and any 
 completed analysis was rerun. 
 
 Problems of Generalization 
 
 Interviewer effectiveness in eliciting information about one char- 
 acteristic may not be indicative of effectiveness in obtaining information 
 about other characteristics. For example, interviewers who are effective 
 in eliciting information about the size of the family may be ineffective 
 in obtaining data about the family's savings accounts. In generalizing 
 from a characteristic which was validated to other characteristics which 
 were not validated, the sensitivity of people to different characteristics 
 must be considered. 
 
 Nor is sensitivity a unidimensional concept. People's sensitivity to 
 reporting information may vary from one characteristic to another. It 
 may also vary by the kind of information requested. For example, people 
 appear to be less sensitive about reporting a change in balance than they 
 are about reporting an actual balance. They also appear to be more 
 sensitive about reporting a large holding than about reporting a small 
 holding. Finally, the apparent sensitivity of persons in reporting a 
 characteristic depends on the distribution of the characteristic in the 
 population. For example, people with given wealth or income may be 
 less sensitive about reporting a given amount of stocks and bonds than 
 they are about reporting a similar amount in savings accounts, but if 
 stocks and bonds are predominantly distributed among uncooperative 
 higher-income and wealthier persons, the completeness and accuracy with 
 which they are reported may appear to be lower than is the case for 
 savings accounts. 
 
 Generalizing from interviewer effectiveness with a set of assignments 
 to general effectiveness also necessitates consideration of the following 
 three problems. The first is individual differences. The set of assign- 
 ments received by each interviewer is, in effect, a sample taken from the 
 population studied. Since people in one sample may differ in their willing- 
 ness to cooperate from those in another sample, sampling variation must 
 be considered, particularly when interviewers or interviewer groups are 
 compared. Underlying the statistics for evaluating sampling variation 
 
16 Survey Reliability and Interviewer Competence 
 
 is the assumption of randomization of sample addresses among inter- 
 viewers, which was not a feature of the project studies. 
 
 The second is the likelihood of interaction between persons contacted 
 and interviewers. For example, lower-income sample members may be 
 more effectively interviewed by lower-income interviewers or by inter- 
 viewers with specific personality characteristics. The observed effective- 
 ness of a group of interviewers may then depend on the method of 
 selecting interviewers as well as on that of selecting sample addresses. 
 
 The third problem involves generalizations from the relationships ob- 
 served for the interviewers employed in the project to all interviewers. 
 This type of generalization assumes the random selection of these inter- 
 viewers from some undefined population of interviewers. 
 
 Interviewer Variability: Studies Ml and M2 
 
 Unfortunately, no primary validation was available in either of these 
 studies. As a result, interviewer variability could only be investigated in 
 terms of response rates. On Wave 1 of Study Ml, the response rate was 
 78 percent, with interviewers' response rates ranging from 45 percent to 
 93 percent. The response rates of interviewers on Wave 1 of Study M2 
 ranged from 50 percent to 94 percent with an average of 77 percent. 20 
 
 Interviewer Variability: Study M3 
 
 The interviewer selection procedure was extensive, employing letters 
 of recommendation, applicant interviews, evaluation checklists, and 
 evaluation of pretest interviews. Out of 150 applicants, only 23 were 
 employed. Nevertheless, considerable interviewer differences did appear. 
 
 One difference which became apparent was in the honesty of the 
 interviewers. Three interviewers were detected through the validation 
 data as having falsified some of their interviews. These interviewers were 
 excluded from the analysis. Two interviewers who made less than 10 
 contacts were also excluded. These interviewers had been hired on a 
 stand-by basis to fill in if interviewing help was needed. 
 
 Analysis of the remaining 18 interviewers still indicated much vari- 
 ability. These interviewers made 309 contacts on Wave 1, or an average 
 of about 17 each. The proportion of contacts in which an interviewer 
 ascertained the existence of the validated savings account, the pick-up 
 rate (response rate X completeness rate), ranged from 32 percent to 64 
 percent with an average of 50 percent. 
 
 20 Excluded from the response rates in Study M2 were two interviewers who 
 falsified interviews. Because of the panel nature of the study with repeated inter- 
 views with the same people, an interviewer faking data was likely to be discovered 
 on subsequent waves. 
 
Interviewer Variability 
 
 17 
 
 As indicated earlier, an interviewer might fail to obtain mention of 
 an account because of a refusal or because of the incomplete reporting of 
 savings unit holdings. Response rates of interviewers varied from 46 per- 
 cent to 91 percent, and the completeness rate ranged from 50 percent to 
 90 percent. 
 
 To what extent may field procedures have produced the observed 
 differences between interviewers? In 11 cases, a panel member was re- 
 assigned to a different interviewer before the original interviewer made 
 contact. Such reassignments resulted when an interviewer was delinquent 
 in completing assignments. The exclusion of such reassignments had no 
 effect upon the results presented. 
 
 A second type of reassignment occurred when the person talked to at 
 a sample address refused to be interviewed. In these cases, the initial 
 interviewer was credited with a refusal. The new interviewer was credited 
 with an interview if one was obtained, but if he was refused he was not 
 credited with a refusal. Ten such reassignments occurred and four inter- 
 views were obtained by the interviewers. A review of these four cases 
 showed that they did not significantly influence the measures of inter- 
 viewer effectiveness. 
 
 A possible weakness of the present analysis stems from the process of 
 assigning sample addresses to interviewers. For reasons of economy and 
 convenience, there was a tendency to assign interviewers addresses ranging 
 from the center of the area outward. As a result, interviewers tended to 
 have a pie shaped cross-section of the area involving varying socioeco- 
 nomic levels. However, an examination of the location of the assignments 
 of the more effective interviewers and the location of the assignments of 
 the less effective interviewers did not indicate concentrations for either 
 group in areas with known socioeconomic characteristics. 21 
 
 Interviewers were split into three groups based on the pick-up rate. 
 There tended to be some natural breaks in the rates which, in part, 
 determined the grouping of interviewers. 
 
 Average 
 Com- Number of contacts 
 
 Response pleteness Pick-up inter- per 
 
 Group rate rate rate viewers interviewer 
 
 1 78% 80% 62% 4 19.3 
 
 II 72 72 52 6 17.2 
 
 III 68 50 41 8 16.1 
 
 When over-all effectiveness is broken into its two components, effec- 
 
 21 Further insights into the heterogeneity of interviewer assignments can be 
 gained by comparing the distribution of the size of the validated time deposits of 
 sample members assigned to the interviewers. Presentation of these results will be 
 postponed until after the grouping of interviewers has been discussed. 
 
18 
 
 Survey Reliability and Interviewer Competence 
 
 Table 1. Distribution of Accounts, by Size and 
 
 by Interviewer Group, Study M3 
 
 (Percentages) 
 
 Size of account 
 
 Interviewer group a 
 
 Total 
 
 I 
 
 II 
 
 III 
 
 Under $1 ,000 
 
 34 
 45 
 21 
 
 36 
 43 
 21 
 
 34 
 39 
 27 
 
 35 
 
 $l,000-$4,999 
 
 $5,000 and over 
 
 42 
 23 
 
 Total 
 
 100 
 
 100 
 
 100 
 
 100 
 
 
 77 
 
 103 
 
 129 
 
 309 
 
 
 
 a Group I was more effective than Group II and Group II in turn was more effective than 
 Group III as measured by the pick-up rate. 
 
 tiveness at the door as reflected by the response rate and effectiveness in 
 the interview as reflected by the completeness rate, we observe that inter- 
 viewers in Group I were more effective in both respects than were inter- 
 viewers in the other groups. Those in Group II were more effective than 
 Group III interviewers in both respects. However, the pattern as we 
 move from group to group suggests that the less effective interviewers may 
 have been relatively better at getting past the door than the more effec- 
 tive ones. 
 
 Let us turn now to the question of the heterogeneity of the interviewer 
 assignments. To what extent did the distribution of the size of savings 
 accounts differ by various interviewer groups? All of the differences 
 among interviewer groups as presented in Table 1 could have occurred 
 by chance. Therefore, the assignment procedure employed did not create 
 significant differences in the size-of-account distribution. 
 
 Although the differences in Table 1 are not significant, there was 
 some tendency for the less effective interviewers to receive more assign- 
 
 Table 2. Results of Contacts for All Interviewers, 
 
 by Size of Account, Study M3 
 
 (Percentages) 
 
 Measure 
 
 All 
 accounts 
 
 Under 
 $1,000 
 
 $1,000- 
 $4,999 
 
 $5,000 
 and over 
 
 Response rate 
 
 Completeness rate . 
 Pick-up rate 
 
 Number of contacts 
 
 73 
 69 
 50 
 
 71 
 72 
 51 
 
 82 
 67 
 55 
 
 62 
 64 
 40 
 
 309 
 
 10^ 
 
 130 
 
 72 
 
Interviewer Variability 
 
 19 
 
 Table 3. Pick-up Rate (Response Rate X Completeness Rate) 
 
 by Size of Account and by Interviewer Group, Study M3 
 
 (Percentages) 
 
 Size of account 
 
 Interviewer group* 
 
 I 
 
 II 
 
 III 
 
 Under SI, 000 
 
 65 
 (26)" 
 
 63 
 (35) 
 
 56 
 (16) 
 
 62 
 
 (37) 
 57 
 
 (44) 
 32 
 
 (22) 
 
 34 
 
 $1 ,000-$4,999 
 
 (44) 
 49 
 
 $5,000 and over 
 
 (51) 
 38 
 
 (34) 
 
 All contacts 
 
 62 
 
 (77) 
 
 52 
 (103) 
 
 41 
 (129) 
 
 " Group I was more effective than Group III as measured by the pick-up rate. 
 b Number of contacts on which percentage is based is shown in parentheses. 
 
 merits with savings units having validated savings accounts of $5,000 and 
 over and fewer assignments with those members having accounts of 
 $1,000 to $4,999. As Table 2 shows, savings units with large accounts 
 were the least cooperative, exhibiting significantly lower response rates 
 than all other savings units. Though one would expect larger savings 
 account holders to be more cooperative in the interview on the assump- 
 tion that the uncooperative ones had refused at the door, they were less 
 cooperative. As a result, the over-all cooperativeness of the larger savings 
 account holders as measured by the pick-up rate was also significantly 
 below that of other savings units. 
 
 Table 3 shows how interviewer groups differed in their performance 
 at sample addresses where members had different sizes of validated savings 
 accounts. It brings out a consistent pattern in which Group I interviewers 
 were more effective with holders of all sizes of savings accounts than 
 either Group II or Group III interviewers. They were significantly more 
 effective than Group III interviewers with contacts having under $1,000 
 in the validated account and for all contacts. 
 
 These findings suggest that the differences between the interviewer 
 groups cannot be explained in terms of differences in the assignments 
 received by each interviewer. In the remainder of this monograph it will 
 be assumed that interviewer differences were due to interviewer charac- 
 teristics and interviewer performance at the door and in the interview. 
 
 Interviewer Variability: Study M4 
 
 Experience gained in Study M3 was utilized in planning and carrying 
 out Study M4. However, extensive analysis of the urban interviewers had 
 
20 Survey Reliability and Interviewer Competence 
 
 Table 4. Reporting of Different Assets and Debts, Study M4 
 
 (Percentages) 
 
 Status 
 
 All 
 
 types a 
 
 Savings 
 accounts 
 
 Checking 
 accounts 
 
 Long- 
 term 
 debt 
 
 Inter- 
 mediate- 
 term debt 
 
 Short- 
 term 
 debt 
 
 Interviewed 
 
 Account 
 
 reported 
 
 Account not 
 
 reported 
 
 Refused 
 
 89 
 
 68 
 
 21 
 11 
 
 75 
 
 52 
 
 23 
 25 
 
 92 
 
 89 
 
 3 
 8 
 
 96 
 
 77 
 
 19 
 4 
 
 91 
 
 82 
 
 9 
 9 
 
 91 
 
 58 
 
 33 
 9 
 
 Total 
 
 100 
 
 100 
 
 100 
 
 100 
 
 100 
 
 100 
 
 Number of cases. . . 
 
 326 
 
 64 
 
 38 
 
 69 
 
 32 
 
 116 
 
 Percentage of 
 
 cases 
 
 100 
 
 20 
 
 12 
 
 21 
 
 10 
 
 37 
 
 a Includes seven savings units where two or more assets 
 in breakdown. 
 
 were validated which do not appear 
 
 not been completed and thus did not influence to a large extent the selec- 
 tion and training of interviewers for the farm study. Because of the 
 likelihood that comparisons will be made between these two studies, 
 emphasis in the subsequent discussion will be placed on pointing out 
 major differences between them. 
 
 Farm Validation 
 
 Three types of debt institutions as well as savings institutions were used 
 as sources of sample names in the farm study. About a third of the names 
 were selected from savings institutions, some from time deposit records, 
 and some from demand deposit records. Another third of the farm sample 
 addresses were selected from an institution offering a short-term revolving- 
 type debt, and the final third was selected from two institutions offering 
 longer-term debt. 
 
 The type of sample persons selected and their willingness to discuss the 
 asset or debt validated varied by validation source. As shown in Table 4, 
 the pick-up rate for persons selected from savings account holders was 
 52 percent; for those selected from checking account records it was 89 
 percent; and for those selected from debt validation sources it ranged 
 from 58 percent for short-term debt to 82 percent for intermediate-term 
 debt. 
 
 If the pick-up rate is broken down into its components, the response 
 and completeness rates, further differences appear. Farmers selected from 
 savings account records and those selected from short-term debt records 
 
Interviewer Variability 21 
 
 both had low pick-up rates, but the response rate of the former group 
 was 75 percent in contrast to 91 percent for the latter group. In view of 
 similarity in the pick-up rates of these two groups, this meant that farmers 
 with short-term debt had a lower completeness rate than did farmers with 
 savings accounts. Farmers selected from demand deposit records and from 
 long- and intermediate-term debt records displayed a third pattern of 
 behavior; for them, response rates and completeness rates were both high. 
 
 These results indicated two basic differences in the validation groups 
 used. First, the low response rate for time-deposit holders indicated that 
 they were different. An analysis shows that they were largely older, higher- 
 income farmers or very young farmers. This low response rate probably 
 reflected the opinion expressed by some middle-aged farmers when asked 
 whether they had savings accounts. After saying, "no," they made com- 
 ments such as that made by one farmer, "It doesn't make sense to put 
 money in a savings account when I can invest it here on the farm and get 
 a higher return." 
 
 The second difference was found in the sensitivity of persons about 
 reporting a holding. A comparison of farmers selected from short-term 
 debt records with those selected from other types of debt records revealed 
 similar response rates of approximately 92 percent, but the short-term 
 debt group showed a completeness rate of 63 percent while that for the 
 other debt groups was 85 percent, a difference significant at the 2 per- 
 cent level. Sensitivity about mentioning short-term debt may have arisen 
 because farmers may not like to admit the dependency implied in a short- 
 term loan whose repayment hinges on a good crop in the fall. In contrast, 
 a longer-term loan does not represent a threat to the yearly solvency of 
 the farmer. 
 
 The implications of the different response and completeness rates for 
 the study of interviewer effectiveness are obvious. An attempt will be 
 made to take this factor into account in the following analysis by setting 
 up two groups of assets and debts. The one, called sensitive assets, is 
 made up of time deposits and short-term debt. The other, called less 
 sensitive assets, is composed of demand deposits and long- and interme- 
 diate-term debts. 
 
 Comparison of Interviewer Variability 
 
 The selection of interviewers for the farm study was influenced to 
 some degree by preliminary findings from Study M3. The principal influ- 
 ence was a change in the perceived stereotype of the good interviewer 
 from the socially aggressive type to the more matter-of-fact, pleasant type. 
 This undoubtedly tended to alter the type of persons hired and to create 
 greater homogeneity among the farm interviewers. Twenty interviewers 
 
22 Survey Reliability and Interviewer Competence 
 
 were selected from 139 applicants. However, one of these was suspected 
 on the first wave of falsifying parts of interviews and was excluded from 
 the present analysis. 
 
 As stated earlier, the farm sample was randomly split and half the 
 names were assigned a change questionnaire and the other half a holdings 
 questionnaire. Consistent with the findings in Study M3, a higher re- 
 sponse rate was obtained on the change than on the holding approach 
 (93 percent versus 86 percent). This appears to reflect the attitude of 
 interviewers that the change approach is less prying and less difficult 
 to use. 
 
 Response rates of interviewers ranged from 75 to 100 percent with a 
 median of 93 percent on the farm study, well above the 69 percent of the 
 previous study. This difference undoubtedly reflected the differing situa- 
 tions and types of persons involved. Farmers working by themselves most 
 of the day tended to welcome personal interaction. They were frequently 
 found at the barn or in the fields where a door could not be closed in 
 the interviewer's face. Thus, less difficulty was encountered in obtaining 
 interviews, and door-entrance techniques were of less importance, in 
 marked contrast to Study M3. 
 
 In the farm study limited variability in response rates was observed, 
 but there was considerable variability in completeness rates. Since the 
 farmer by circumstances or nature was not prone to refuse to be inter- 
 viewed, a question arises as to whether his refusals may possibly have come 
 more subtly by failing to report information in the interview. The com- 
 pleteness rat** for all interviews in the farm study was 76 percent. This, 
 however, was a misleading figure for two reasons. First, it included the 
 validation of less sensitive assets and debts for over 45 percent of the 
 farmers. If these were excluded and only sensitive assets and debts were 
 used, the completeness rate would drop to 64 percent, which is below the 
 69 percent on Study M3. However, this would understate the true rate 
 obtained by the farm interviewers because of the second reason, an am- 
 biguity in the change questionnaires. 22 
 
 22 Part of this difference may be due to the questionnaire format for the change 
 approach on the farm study (Appendix D). 
 
 In the section of the change questionnaire dealing with the debt holdings of 
 the farmer, the lead-in question read, "Do you have . . . loans secured by mort- 
 gages, sales contracts, or other loans on . . . :" This was followed by a list of assets 
 on which the farmer might owe money. There was a single box for "no" answers 
 for each asset. The interviewer could interpret such "no" answers in three ways: 
 
 1. The farmer does not own the particular asset. 
 
 2. The farmer owns but has no debt on the asset. 
 
 3. The farmer owns and has debt, but there has been no change in the 
 amount owed. 
 
 The interviewer was instructed in the training sessions to check the "no" box 
 
Interviewer Variability 23 
 
 The change approach for all validations yielded a completeness rate 
 of 65 percent, while the rate for the holdings approach was 87 percent. 
 Comparable rates for Study M3 were 67 percent and 69 percent. 
 
 The pick-up rate on the farm study depended on the base used. The 
 over-all rate was 68 percent, which may have been unduly low because 
 of the possible ambiguity in the change approach. The rate for the 
 holdings approach only was 75 percent, but this included validation of 
 less sensitive assets and debts for almost half of the farmers. Possibly the 
 rate which allowed the most direct comparison with Study M3 was the 
 pick-up rate for the holdings approach on sensitive assets and debts. This 
 figure was 61 percent, compared with 43 percent on Study M3. 
 
 The pick-up rate for the holdings approach on the less sensitive assets 
 was 90 percent, indicating something of the magnitude of differences in 
 cooperation for sensitive and less sensitive assets and debts. 
 
 Limitations of Analysis of Interviewer Effectiveness 
 
 Multiple validation sources, useful for certain aspects of the study, 
 became a handicap in evaluating interviewer effectiveness. Since the pro- 
 portion of holdings to change assignments and of sensitive to less sensitive 
 validation assignments varied from one interviewer to another, meaning- 
 ful criteria could not be developed unless these effects could be removed. 
 Yet when the average of 17 contacts per interviewer was split four ways, 
 there was an average of about four contacts per cell, which was obviously 
 inadequate for evaluating interviewers in view of respondent variability. 
 
 The number of cases could be doubled if only one split was considered 
 at a time. Pick-up rates were computed for the sensitive assets and debts 
 and for the less sensitive ones for each interviewer. Similar rates were 
 prepared on the basis of the holding-change splits. It was hoped that the 
 more effective interviewers would tend to appear near the top of each of 
 these four lists and the less effective ones near the bottom. Except for a 
 few interviewers, this expectation was not met. Coefficients of determina- 
 tion for each combination of pick-up rates were as follows : 
 
 for situation 2 and to write <: not applicable" for situation 1. Situation 3 should 
 not have been elicited until after the name of the person or institution to whom 
 the debt was owed has been obtained and entered on the questionnaire. However, 
 in answering the preceding questions, a respondent may have thought in terms of 
 change and anticipated questions. Thus, a respondent may have said in response 
 to the initial question, "Yes, I owe some money on my tractor, but that hasn't 
 changed." Unless the interviewer understood what was wanted, he might have 
 checked the "no" box and failed to obtain information about the creditor. Such a 
 questionnaire was coded as not mentioning the validated asset or debt. 
 
24 Survey Reliability and Interviewer Competence 
 
 Less 
 
 Sensitive sensitive Holdings Change 
 
 validations validations approach approach 
 
 1. Sensitive validations .003 .004 .053 
 
 2. Less sensitive validations .... .372 .291 
 
 3. Holdings approach .... .... .004 
 
 4. Change approach .... .... .... 
 
 On the whole, the coefficients of determination are small, even among 
 pairs of variables where there is overlap, such as Variables 2 and 3, which 
 have in common all holdings forms which deal with less sensitive valida- 
 tions. Three interrelated explanations may be offered for these low corre- 
 lations. First, distribution of sensitive and less sensitive validation assign- 
 ments varied widely among interviewers, which influenced pick-up rates 
 on the holdings approach. Similarly, the distribution of change and hold- 
 ings approaches varied among interviewers. Second, while the average 
 number of contacts made by each interviewer was 1 7, the numbers ranged 
 from 12 to 22. An interviewer with 12 contacts might only have had 
 3 or 4 contacts involving the holdings approach. Finally, the rates for 
 the less sensitive validations did not distinguish among interviewers. Thus, 
 there were 8 interviewers with 100 percent rates and of the remaining 11, 
 7 had rates of over 80 percent. With the small number of cases, these 
 limited differences are likely to be unreliable measures of interviewer 
 effectiveness. 
 
 Without any reliable measure of interviewer effectiveness in the farm 
 study (M4), a rigorous evaluation of the selection, training, and super- 
 vision methods employed cannot be made. Much of the following evalu- 
 ation of methods is based on the M3 Study. This is regretted because it 
 is felt that there was considerable improvement in the methods used in 
 the farm study over those employed in Study M3. 
 
 Summary 
 
 Considerable interviewer variability on pick-up and completeness rates 
 is evident. This suggests that there was room for improvement in the 
 techniques and methods of selecting, training, and supervising interviewers 
 which were utilized in this study, yet they are representative of the 
 methods employed in most surveys of this type. The following chapters 
 describe the methods used and suggest which were more effective in 
 selecting the better interviewers. Obviously, with only one study from 
 which to draw conclusions, the findings represent hypotheses for future 
 testing. 
 
 Very little information is available about the applicants rejected. As 
 a result, the effectiveness of the methods employed in selecting potentially 
 
Interviewer Variability 25 
 
 better interviewers from among the applicants cannot be determined. 
 This must be kept in mind in evaluating the findings presented. 
 
 The reader must also keep in mind the nature of the asset, savings 
 accounts, which was validated in the M3 Study. Some indication of 
 people's sensitivity to reporting this asset can be gained from the farm 
 study. While the pick-up rate using the same interviewers was 89 percent 
 for checking accounts, it was only 52 percent for savings accounts. This 
 represents differences in similar people's attitudes toward reporting these 
 two types of information to an interviewer. 
 
III. INTERVIEWER SELECTION 
 
 Interviewer selection procedure varies greatly. As a rule, the degree of 
 effort expended on this activity depends in part on the type of study and 
 in part on the researcher himself. In a relatively simple study with 
 structured questionnaires and not many questions, less effort is devoted 
 to interviewer selection. In more complicated studies, greater care is 
 needed to select interviewers who are able to handle the details involved. 
 
 More generally, past experience seems to be the principal guide used 
 in recruiting interviewers. Each organization develops its own approach 
 to this task based to a large extent on its experiences rather than on con- 
 trolled experiments. In relatively small organizations the selection pro- 
 cedure is more informal and is less likely to make use of personality tests 
 and other written examinations. The latter techniques appear to be used 
 extensively by the large survey organizations, although the effectiveness 
 of most of these tests has not been demonstrated. 23 
 
 This chapter reports the methods, procedures, and techniques em- 
 ployed in selecting interviewers for the four project studies. In general, 
 it is felt that these procedures, with a few noted exceptions, were very 
 useful in selecting effective interviewers. However, such subjective evalu- 
 ations are open to possible bias. To complement these evaluations, the 
 selection techniques used in Study M3 were evaluated, where possible, in 
 terms of their relationship to a measure of interviewer effectiveness, the 
 pick-up rate. 
 
 The absence of a significant relationship between these methods, pro- 
 cedures, and techniques and the pick-up rate should not be interpreted as 
 indicating that these methods were of no use in identifying potentially 
 effective interviewers. Most of these variables were employed in deciding 
 
 23 For a description of these testing procedures, see Beatrice F. Dvorak, Frances 
 G. Fox, and Charles Meigh, "Tests for Field Survey Interviewers," Journal of 
 Marketing, Vol. 16, No. 3 (January, 1952), pp. 301-6; Stanley Womer and Harper 
 Boyd, "The Use of a Voice Recorder in the Selection and Training of Field 
 Workers," Public Opinion Quarterly, Vol. 15, No. 2 (Summer, 1951), pp. 358-63; 
 Principal Data Collection Forms and Procedures (Washington: U.S. Bureau of 
 the Census, 1961); H. H. Lamale, Methodology of the Survey of Consumer Ex- 
 penditures in 1950 (Philadelphia: University of Pennsylvania Press, 1959), espe- 
 cially pages 63-65; H. H. Hyman et al., Interviewing in Social Research (Chicago: 
 University of Chicago Press, 1954), especially pages 362-63. 
 
 26 
 
Interviewer Selection 27 
 
 which applicants would be hired. As a result, a variable may have been 
 useful in successfully rejecting undesirable applicants but may have been 
 of no value in discriminating among those hired. Thus, the absence of a 
 relationship may only reflect that the characteristics of interviewers on 
 which the analysis is based were homogeneous with respect to a variable 
 because that variable was employed in selecting them. 
 
 Interviewer Selection Procedures 
 
 The discussion in this chapter refers only to interviewer selection for 
 the panel operations of the Consumer Savings Project. The other studies, 
 which were carried out by the Survey Research Center, utilized their 
 already-established field staff. 
 
 Most of the field work for Study Ml was subcontracted to two well- 
 known interviewing organizations, and the selection of interviewers was 
 the responsibility of these organizations. In addition to their usual selec- 
 tion procedures, both organizations placed stress on procurring inter- 
 viewers with a financial background and with some knowledge of financial 
 terminology. 
 
 In the remaining three panel operations, the field staff was recruited 
 by project personnel in each of the areas being sampled. A review and 
 evaluation of the recruiting procedures used will be given under the 
 following headings : interviewer qualifications, sources of applicants, eval- 
 uation of applicants, selection of interviewer candidates, evaluation of 
 candidates, and final selection. Each of these aspects will be covered 
 separately in the first part of this chapter. The second part will present 
 a quantitative method for selecting interviewers based on information 
 available before applicants are formally hired. 
 
 Interviewer Qualifications 
 
 For the first two studies the qualifications sought in applicants for 
 interviewing positions were based on prior experience of the project per- 
 sonnel. Included were good grooming, pleasant appearance, good hand- 
 writing, age between 25 and 55 years, availability for at least 15 hours a 
 week, availability of an automobile, and permanency in the area. 
 
 The first three of these were obviously necessary qualifications. The 
 age range was based on current marketing literature. "Those who are too 
 young lack tact and the ability to approach people properly. Those who 
 are too old are usually not willing to follow instructions carefully and are 
 not sufficiently aggressive." 24 The availability for at least 15 hours a week 
 
 24 Lyndon O. Brown, Marketing and Distribution Research (3rd ed.; New 
 York: Ronald, 1955), p. 301. 
 
28 Survey Reliability and Interviewer Competence 
 
 Table 5. Record of Applicants 
 
 Item 
 
 Study M3 
 
 Study M4 
 
 Number of Applicants 
 
 141 
 
 136 
 
 Given a personal interview 
 
 Offered training 
 
 Hired as interviewers 
 
 109 
 50 
 23 
 15 
 
 101 
 39 
 20 
 
 Retained on all five waves 
 
 15 
 
 was required so that the time schedule of the field operation could be 
 maintained. Since it would be necessary for the interviewers to travel 
 considerable distances to obtain interviews, a car was essential. 
 
 Permanency in the sample area and willingness to remain on the 
 project staff for the duration of the study were considered particularly 
 important because the same respondents were to be reinterviewed four or 
 five times. In addition, loss of an interviewer could have meant new 
 training problems and could have led to antagonizing the respondent by 
 giving a new interviewer access to confidential data. The success of 
 adhering to this requirement is evidenced in Table 5. On Study M3, 15 
 of the 23 interviewers hired, or 65 percent, remained for all five waves. 
 The retention rate on Study M4 was 75 percent, or 15 of the 20 inter- 
 viewers hired. 
 
 Experience gained in Study Ml indicated that a financial background 
 was not essential. More important was a college education plus the ability 
 to grasp many concepts and new ideas; an interviewer with these qualifi- 
 cations could be taught all the financial terminology that was needed. 
 
 A final prerequisite added in Studies M3 and M4 was confidence that 
 financial data could be obtained in the interview process. The manner 
 in which this characteristic was evaluated is described in a later section 
 of this monograph. 
 
 Sources of Qualified Applicants 
 
 School systems, state employment offices, and local colleges and uni- 
 versities were the primary sources of interviewer applicants. Applicants 
 were also located through such miscellaneous sources as the Farm Bureau, 
 local marketing research organizations, recommendations from distin- 
 guished people in the area, and other previous studies. 
 
 The state employment offices were of assistance in a number of ways. 
 They scanned their files to see if any of the registered job-seekers had the 
 necessary qualifications. Those who did have were asked to come into 
 the employment office for a personal interview. The offices also placed 
 ads in the local newspapers on behalf of the study. In addition, the 
 
Interviewer Selection 29 
 
 offices provided the project staff with desks for personal interviews and 
 telephones so that incoming inquiries could be handled immediately. 
 Considerable time and weeding-out were required to find qualified inter- 
 viewers through this source. For example, the following results were 
 obtained through one employment office: of 27 responding to newspaper 
 ads, 17 were rejected for lack of basic qualifications, 10 were considered 
 for the job, but only 2 were actually hired. 
 
 Local universities were used to locate likely interviewer applicants for 
 Studies M2 and M3. The heads of the departments of psychology, soci- 
 ology, and business administration were asked to post announcements of 
 the positions and to request professors and instructors to make verbal 
 announcements of the positions in their graduate classes. University stu- 
 dent employment offices were also contacted, and a number of graduate 
 students applied for the interviewing positions. A few interviewers were 
 located in this way. 
 
 Locating applicants through the offices of superintendents of schools 
 in the districts and counties where the studies were being made was one 
 of the most productive methods. In all instances, the county superin- 
 tendents went out of their way to find qualified applicants. As one super- 
 intendent remarked, "The experience of being with adults instead of with 
 students would be valuable experience for the teachers and would also 
 give them a chance to earn some needed extra money." The school 
 superintendents were of assistance in numerous other ways. They pro- 
 vided up-to-date listings of teachers' names and addresses so that a post- 
 card describing the interviewing work could be sent to each teacher asking 
 those who were interested to write for further information. In two coun- 
 ties the superintendents of schools sent personal letters informing teachers 
 of the part-time work available. The letter sent by one superintendent of 
 schools asked a subjectively preselected group to return a postcard to him 
 indicating if they were interested in the assignment. Since this was a 
 select group, time was saved in not having to talk to applicants who did 
 not have the basic qualifications. Of 30 teachers to whom letters were 
 sent, 11 indicated interest; 2 of these 11 were selected and became out- 
 standing interviewers. In one county the superintendent included a de- 
 scription of the part-time job in his monthly directive sent to all schools 
 in the county. 
 
 Teachers were also reached through teachers' associations. One state 
 teachers' association was extremely helpful in addressing postcard notices 
 of the job openings. In view of the excellent cooperation obtained from 
 the school systems, it is not surprising to note that on the final two panel 
 studies the majority of the applicants were either school teachers or were 
 connected with the school system, e.g., principals or other administrators. 
 
30 Survey Reliability and Interviewer Competence 
 
 Thus, in Study M3, 76 of the 141 applicants (54 percent) came from the 
 school system, and in Study M4 the ratio was 93 of 136 (68 percent). 
 In addition, much less expense and effort were required in obtaining these 
 candidates than were necessary when using other methods. 
 
 Various other sources also provided interviewer applicants. These 
 included newspaper ads not linked with the state employment service 
 (Studies M2 and M3), lists of interviewers used on related prior studies 
 (Study M3), and for Study M4, county farm advisers. A further source 
 was the applicants themselves. Those who appeared to be good prospects 
 were told that more people with qualifications similar to theirs were 
 needed and they were encouraged to invite any such other people whom 
 they knew to get in touch with the field director. No records were kept 
 of applicants obtained from these miscellaneous sources. 
 
 It is interesting to note that the ratio of applicants to interviewers 
 hired was about 6 to 1 (see Table 5) . 
 
 In terms of retention for the entire operation, the number of appli- 
 cants exceeded the number of final interviewers by roughly 9 to 1. Al- 
 though most of the weeding out occurred at the initial employment 
 interview stage, it is also noteworthy that roughly half of the interviewer 
 candidates did not survive the training sessions, which also served as part 
 of the selection process. This will be discussed in a later section. 
 
 Evaluation of Applicants 
 
 Four tools were used in the evaluation process, each later converted to 
 a weighted numerical score. These four tools were (1) the application 
 form, (2) the personal interview with an evaluation checklist, (3) a 
 personality test, and (4) references. Each of these tools was in turn 
 evaluated from a methodological standpoint, using correlation measures 
 between it and a measure of performance to determine to what extent 
 the applicant's score measured by these tools was related to subsequent 
 performance measured by his pick-up rate as an interviewer. 
 
 Application Form 
 
 The application form served two purposes. It supplied facts about the 
 applicant's age, address, marital status, experience, and so on and indi- 
 cated something about the applicant's ability to follow simple instructions 
 and to do a neat job in completing the application form. 
 
 The application form was an important tool of the selection procedure 
 and merits some description. This discussion is built around the applica- 
 tion form used on Study M4, shown in Figure 1, although most of the 
 questions appeared on application forms used on earlier studies. 
 
 Name in full (please print). The complete name should have been 
 
Interviewer Selection 31 
 
 printed in full with no nicknames or abbreviations unless they were part 
 of the applicant's legal name. 
 
 Address. It was advisable to know where the applicant lived in rela- 
 tion to the need for the interviewers and whether he was centrally enough 
 located so as to keep travel time to a minimum. The full address was 
 needed, including an apartment number, if he had one, and his zone 
 number. 
 
 How long? If the applicant had lived at his present address less than 
 two years, some explanation was desirable. He may not have been perma- 
 nent in the area, or he may not have been a stable individual. This was 
 one measure of the applicant's permanency which was discussed pre- 
 viously. However, no relationship between this and the interview pick-up 
 rate was found for Study M3. 
 
 Home phone. The exchange and number should have been clear and 
 legible, inasmuch as the applicant may have had to be reached quickly to 
 clear up problems if he became an interviewer. 
 
 Office phone. If the answer was "none," this should have been stated. 
 
 Marital status. In all studies of the project, married applicants were 
 given preference over single ones if other qualifications were met. In line 
 with common belief, it was felt that married people would be relatively 
 more stable and more mature and thus make better interviewers. How- 
 ever, the response rates and the pick-up rates did not substantiate this 
 belief, for both groups did equally well. The coefficient of determination 
 did not differ significantly from zero on Study M3. 
 
 Children and ages. If there were any children, there may have been a 
 demand for the applicant at home, thus reducing possible interviewing 
 time. If the applicant was a woman with very young children, there was 
 little likelihood of hiring her unless she had a baby-sitter or unless some 
 other child-care arrangements could be made. Even if arrangements 
 could be made in such a case, it was felt that demands at home might be 
 overwhelming. 
 
 Age. For Study M3 no significant coefficient of determination was 
 found between age and pick-up rate. Because of the basic qualifications 
 requirement, the majority of interviewers selected were between the ages 
 of 25 and 55 — in Study Ml, 93 percent; in Study M2, 96 percent; in 
 Study M3, 87 percent; and in Study M4, 90 percent. 
 
 State of health. Questionable health had to be explained because it 
 was important for an interviewer to stay with the panel study until com- 
 pleted. If the applicant had obvious handicaps or disfigurements, these 
 were noted because they could have caused bias in the information ob- 
 
32 Survey Reliability and Interviewer Competence 
 
 Figure 1. Interviewer Application Form: Study M4 
 
 University of Illinois 
 Bureau of Economic and Business Research 
 
 INTERVIEWER APPLICATION FORM 
 
 (Please print; answer all questions) 
 
 Name in ful l 
 
 Address How long? 
 
 City Home phone Office phone 
 
 □ Single D Married Q Widowed D Divorced 
 
 Number of children Ages Your age State of health 
 
 Education: Check highest level attended. 
 
 r—i Grade r—i High i— i Business r— i Junior r-i College or r-Graduate 
 school school u school ^ college university work 
 
 If attended college, university or business school: 
 
 Dates _, .„ 
 
 attended Degrees, if any 
 
 Name of Institution From To Major field Minor field Name Date rec'd 
 
 Full-time job, if any: D None 
 Type of work , 
 
 Where employed How long?_ 
 
 Part-time jobs, if any: D None 
 
 Type of work 
 
 Where employed How long?_ 
 
 Working days Working hours: From T< 
 
 Have you lived on a farm? Q No D Yes Dates Where 
 
 Have you worked on a farm? D No □ Yes Dates Type of work_ 
 
 Have you held a job involving 
 
 contact with farmers? U No D Yes Dates Type of work 
 
Interviewer Selection 33 
 
 Experience as an interviewer or in related work in past five years: D None 
 
 Name of immediate 
 Employer Location Type of work supervisor 
 
 Number of hours per week that you could devote to interviewing^ 
 
 Are you engaged in any activity (personal or business) which takes you out of 
 the area occasionally? D No Q Yes (Explain) 
 
 Any special time during the day or evening when you would consistently not he 
 available ? 
 
 Any special time of the week when you would consistently not be available? 
 
 Any special time of the year when you would not be available? 
 
 Can you drive? Do you have a car? Is it available at all times? 
 
 Do you know shorthand now? If yes, approximate words per minute? 
 
 Can you speak any foreign languages well? If yes, which ones? 
 
 Why are you applying for this job? 
 
 References: (Names and addresses of three or more persons, not relatives or 
 personal friends, from whom information concerning your quali- 
 fications or interviewing experience may be obtained) 
 
 Circumstances 
 Name Address of relationship 
 
 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 
 
 (Leave blank) 
 Reference results 
 
 Comments 
 
 Final evaluation 
 
34 Survey Reliability and Interviewer Competence 
 
 tained. One of two answers was usually given to this state-of-health 
 question, "excellent" or "good." The more effective interviewers were 
 not concentrated in either group. 
 
 Education. Applicants with college degrees were preferred. There 
 was no requirement as to the type of degree, although it was preferable 
 that the degree be in a field somewhat related to this work: finance, 
 economics, marketing, accounting, sociology, psychology, and so on. Two 
 other factors to be noted were the time elapsed since the applicant 
 received the degree and whether the college education had been uninter- 
 rupted. If it had been interrupted, there should have been some expla- 
 nation. Following are the average response rates for Study M2 inter- 
 viewers compared with the level of education completed. 
 
 High school or less (7 interviewers) 70.3% 
 
 College (6 interviewers) 85.2 
 
 Postgraduate work (3 interviewers) 88.3 
 
 In the light of these findings, applicants with at least a college education 
 were sought for the two subsequent studies. Of the Study M3 inter- 
 viewers, 96 percent had at least a college education and of the Study M4 
 interviewers, 100 percent were college graduates. However, no relation- 
 ship was discovered between the pick-up rates and education of Study M3 
 interviewers. 
 
 Full-time job, if any. It was expected that the applicant would al- 
 ready have a full-time job unless the applicant was a housewife. It was 
 helpful if the full-time job was in a related field, such as finance or ac- 
 counting, or in a position where there was some contact with the general 
 public such as sales, social work, or teaching. 
 
 Where employed? Where the applicant worked in relationship to 
 where he lived might have caused a travel problem with interviewing 
 work. 
 
 How long? This was one measure of the applicant's stability. If he 
 had been employed less than six months, the previous employment was to 
 be specified, together with the reason for leaving the previous job. 
 
 Part-time jobs. All part-time jobs were to be listed since they could 
 have interfered with the applicant's ability to work as an interviewer. 
 In Study M3, no significant coefficient of determination between this 
 variable and the pick-up rate was obtained. 
 
 Farm background. Three questions related to farming were used only 
 for the farm study. Since the interviewer was to be talking to farm owners 
 and operators, some knowledge of farming was essential. If the applicant 
 had no farm background, he was not qualified as an interviewer for this 
 particular study. 
 
Interviewer Selection 35 
 
 Experience. There was overlapping on this question if the applicant 
 had previously recorded experience under full-time or part-time jobs. It 
 was important that the name of the immediate supervisor be listed so the 
 supervisor could be used as a reference if desired. No relationship ap- 
 peared to exist between previous interviewing or related experience and 
 the interviewer's pick-up rate for Study M3. 
 
 Number of hours. The expected number of hours per week was 15 to 
 20 even though the interviewer was more likely to work 10 to 15 hours. 
 After Study M3 was completed, the interviewers were queried on the 
 average number of hours a week they actually worked when interviewing. 
 They reported between 12 and 14 hours a week. Since the interviewing 
 was completed, the interviewers had nothing to lose by being honest. 
 
 In direct contrast to these reports, on Wave 1 of Study M2 the inter- 
 viewers worked an average of 1 7V4 hours a week. This was based on total 
 time actually worked and reported by the interviewers on their time sheets. 
 Since these interviewers were paid by the hour, 25 the results were not 
 very surprising. In addition, the unstructured nature of this study re- 
 quired that the interviewers type up complete details of the interview, 
 thus raising the average hours worked per week. 
 
 If, on the application form, less than 15 hours was reported as avail- 
 able for interviewing, the applicant was usually rejected. A report of 
 20 or more hours a week was considered unrealistic, especially if the appli- 
 cant had a full-time job and a family. 
 
 Some applicants recorded "no limit" or "any amount of time." This 
 was not adequate, and they were pressed for the number of hours they 
 could devote to work outside of their regular activities. When pressured, 
 a few such applicants said "five hours" or "five or ten hours," which was 
 clearly not enough. Some applicants did not answer the question directly, 
 but tried to hedge until they found out the number of hours they were 
 expected to work. The number of hours reported on the application blank 
 did not correlate significantly with the pick-up rate on Study M3. Since 
 the purpose of this question was obvious, an effort was made to estimate 
 how much time the interviewer would have, depending on his existing 
 obligations. For each interviewer in Study M3 an over-all estimate of 
 weekly hours available was calculated. This estimate yielded a significant 
 coefficient of determination (.135) with interviewer response rate, al- 
 though it was not significantly related to pick-up rate. 
 
 Engaged in other activities. Other activities, either personal or busi- 
 ness, which would take the applicant out of the area occasionally could 
 
 25 See the section beginning on page 81 for the results of paying interviewers 
 by the hour compared with paying them by the completed interview. 
 
36 Survey Reliability and Interviewer Competence 
 
 have been detrimental to the completion of field assignments. If the 
 answer was "yes" to this question, a complete explanation was required. 
 
 Special times not available. Three questions were asked relating to 
 any time during the day or evening, week, or year when the applicant 
 would not be available. If the answer was "yes" to any of these, a full 
 explanation was required. If the applicant said "no" to all three ques- 
 tions, he was asked about special obligations (such as clubs, organizations, 
 meetings, and vacations). "No" was an easy (and the expected) answer 
 to this question, so it was probed carefully. Even if an applicant said 
 "yes" to one or more of these questions it did not mean that he should 
 not be selected, although these obligations might have limited his inter- 
 viewing activities. 
 
 Car. As was mentioned previously, a car was necessary for this type 
 of interviewing, and it had to be available at all times so the interviewer 
 could make and keep appointments. If the applicant was a woman, it was 
 better if the car was her own and not the family car, which might have 
 been needed at the same time she had an interviewing appointment. 
 
 Shorthand. Ability to take shorthand was clearly desirable for an 
 interviewer. However, other qualities seemed much more important and 
 this ability was not a prerequisite for hiring and was not taken into con- 
 sideration unless two interviewers were equal in all other respects. 
 
 Foreign language. The purpose of a question on foreign language was 
 to have this information on file in case sample members were encountered 
 who only spoke a foreign language. 
 
 Reasons for applying. This question was asked only on the application 
 forms for Studies M3 and M4. Applicants were separated into three 
 groups on the basis of their answers: financial reasons, social or interest 
 reasons, or a combination of financial and social reasons. Response and 
 pick-up rates for both studies were not significantly related to these three 
 groups of reasons. 
 
 References. The instructions concerning references specified clearly 
 "three or more persons, not relatives or personal friends." Yet many 
 applicants listed only one or two references or listed relatives or personal 
 friends. All references were reviewed with the applicant and the legibility 
 of names and addresses was checked. 26 
 
 The space at the bottom of the page was used for reference results, 
 comments, and final evaluation of the applicant. 
 
 The application was generally completed at the time of the personal 
 interview. The completed form was evaluated and classified into one of 
 
 20 See the section starting on page 44 for further details on the use of refer- 
 ences in the selection of interviewers. 
 
Interviewer Selection 37 
 
 the following categories: (1) good prospect; (2) possibly hire; (3) pos- 
 sibly reject; or (4) reject this prospect. 
 
 The factors given greatest consideration in this subjective decision 
 were neatness, ability to follow instructions, location, length of time at 
 present address, marital status, age, number and ages of children, state of 
 health, education, type of work at present job, length of time at present 
 job, interviewing experience, weekly hours available, other activities, time 
 not available, car, and reasons for applying. 
 
 The value of using an application form was unquestionable. However, 
 no way of compiling answers or of weighting them was discovered which 
 led to a score which was significantly related to interviewer pick-up rates 
 on Study M3. 
 
 Personal Interview 
 
 The purpose of the personal interview was to gather information 
 about the applicant to aid in the selection of interviewer candidates. For 
 the Consumer Savings Project all personal interviews were conducted by 
 the project director, the field director, and a university graduate assistant. 
 An effort was made to standardize the personal interviewing procedures. 
 The interviews were held in semi-private facilities of the state employment 
 offices, chambers of commerce, and hotels. 
 
 The personal interview was comprised of four parts and required 15 
 to 45 minutes, generally averaging about 25 minutes. Included were a 
 review of the application form, a brief description of the job with an 
 effort to evaluate the interviewer's confidence that he could do the work, 
 a personality test, and an evaluation checklist. Each of these will be 
 covered in order. 
 
 The application form was reviewed, with the interviewer probing for 
 further details in areas of interest or wherever the application form had 
 not been filled in adequately. The brief job description covered such 
 subjects as the purpose and objectives of the study, the information being 
 sought, the time and interviewing schedule, the training procedures, the 
 approximate length of the interviews, the number of assignments, the need 
 for confidential treatment of the data, and the compensation. At the same 
 time, in Studies M3 and M4 an attempt was made to ascertain the reac- 
 tion of the applicants to this type of work and whether obtaining personal 
 financial data was something which they felt could be done. They were 
 asked, "Do you think people will give this information?" "If not, why 
 not?" Although no attempt was made to evaluate interviewers on the 
 basis of their answers, pessimistic or negative attitudes of applicants were 
 noted. 
 
 Each applicant was asked to take a personality test. A discussion of 
 
38 
 
 Survey Reliability and Interviewer Competence 
 
 Figure 2. Evaluation Checklist: Study M4 
 
 University of T1.li.nois 
 Bureau of Economic and Business Research 
 
 Applicant Evaluation 
 Personal Interview Check List 
 
 Characteristic 
 
 Superior 
 
 Above 
 average 
 
 Average 
 
 Below 
 average 
 
 Appearance (dress) 
 
 
 
 
 
 Voice 
 
 
 
 
 
 Poise 
 
 
 
 
 
 Self-confidence 
 
 
 
 
 
 Intelligent questions 
 
 
 
 
 
 Intelligent answers 
 
 
 
 
 
 Likable (friendly) 
 
 
 
 
 
 Attitude towards this 
 study- 
 
 
 
 
 
 Ob o'ectivity 
 
 
 
 
 
 Physical appearance 
 
 
 
 
 
 Applicant 
 
 Interviewer 
 
 Date 
 
Interviewer Selection 39 
 
 the evaluation of the results of these personality tests will be covered in 
 the section beginning on "page 49. While the applicant was taking this 
 personality test, the project representative was filling out an evaluation 
 checklist, shown in Figure 2. The purpose of this last part of the inter- 
 view was to call attention to any personal characteristics that might have 
 been overlooked. In addition, if the project representative felt that he 
 did not have enough information about the applicant to rate him on one 
 or more of these characteristics, further questions could be asked after 
 the completion of the personality test. 
 
 Immediately after the applicant had left, the project representative 
 reviewed the evaluation form and assigned it one of the following four 
 ratings: (1) hire this applicant; (2) possibly hire; (3) possibly reject; or 
 (4) reject this applicant. 
 
 The pick-up rates for Study M3 were used to evaluate this tool in the 
 selection process; however, no significant relationship was discovered. In 
 addition to the over-all checklist evaluation, each characteristic was corre- 
 lated with the pick-up rates for Study M3. A significant coefficient of 
 determination was not obtained for any of these characteristics. 
 
 Personality Tests 
 
 For Studies M2 and M3, interviewers were given the Inventory of Per- 
 sonal Attitudes (IPA) test, which consisted of 30 statements for which 
 the interviewer was to indicate whether he would "strongly agree," 
 "agree," "disagree," or "strongly disagree." The test was to cover the 
 following attitudes toward life: future time orientation, desirability of 
 stopping to think, originality, opinion that life (world) is predictable, 
 belief in fate, and willingness to try many actions. This test did not prove 
 useful for identifying good interviewers. 
 
 After Study M2 interviewers had been hired, they were given the 
 Edwards Personal Preference Schedule (EPPS) test, which consisted of 
 225 sets of two statements. The interviewer was to select the statement 
 which he liked the most or disliked the least. The EPPS was developed 
 to provide measures of 15 relatively independent personality variables. 
 A complete description of each of these variables is presented in Figure 3. 
 
 The results of this test were used to aid in the selection of interviewers 
 for Study M4. Each applicant, on the basis of his scores on dominance 
 (high score desirable) and abasement (low score desirable) variables, was 
 put into one of the following four groups: (1) hire this applicant, 
 (2) possibly hire; (3) possibly reject; or (4) reject this applicant. How- 
 ever, this particular semi-subjective approach did not prove an optimal 
 indicator for identifying interviewers with a high pick-up rate. 
 
 One question of interest was whether the interviewer selection pro- 
 
40 
 
 Survey Reliability and Interviewer Competence 
 
 Figure 3. Definitions of Variables Available from the 
 Edwards Personal Preference Schedule 
 
 /. ach Achievement: To do one's 
 best, to be successful, to accomplish 
 tasks requiring skill and effort, to be 
 a recognized authority, to accomplish 
 something of great significance, to do 
 a difficult job well, to solve difficult 
 problems and puzzles, to be able to 
 do things better than others, to write 
 a great novel or play. 
 
 2. def Deference: To get sugges- 
 tions from others, to find out what 
 others think, to follow instructions 
 and do what is expected, to praise 
 others, to tell others that they have 
 done a good job, to accept the leader- 
 ship of others, to read about great 
 men, to conform to custom and avoid 
 the unconventional, to let others 
 make decisions. 
 
 3. ord Order: To have written 
 work neat and organized, to make 
 plans before starting on a difficult 
 task, to have things organized, to 
 keep things neat and orderly, to 
 make advance plans when taking a 
 trip, to organize details of work, to 
 keep letters and files according to 
 some system, to have meals organized 
 and a definite time for eating, to 
 have things arranged so that they run 
 smoothly without change. 
 
 4. exh Exhibition: To say witty 
 and clever things, to tell amusing 
 jokes and stories, to talk about per- 
 sonal adventures and experiences, to 
 have others notice and comment 
 upon one's appearance, to say things 
 just to see what effect it will have on 
 others, to talk about personal achieve- 
 ments, to be the center of attention, 
 to use words that others do not know 
 the meaning of, to ask questions 
 others cannot answer. 
 
 5. aut Autonomy: To be able to 
 come and go as desired, to say what 
 one thinks about things, to be inde- 
 pendent of others in making deci- 
 
 sions, to feel free to do what one 
 wants, to do things that are uncon- 
 ventional, to avoid situations where 
 one is expected to conform, to do 
 things without regard to what others 
 may think, to criticize those in posi- 
 tions of authority, to avoid responsi- 
 bilities and obligations. 
 
 6. aff Affiliation: To be loyal to 
 friends, to participate in friendly 
 groups, to do things for friends, to 
 form new friendships, to make as 
 many friends as possible, to share 
 things with friends, to do things with 
 friends rather than alone, to form 
 strong attachments, to write letters to 
 friends. 
 
 7. int Intraception: To analyze 
 one's motives and feelings, to observe 
 others, to understand how others feel 
 about problems, to put one's self in 
 another's place, to judge people by 
 why they do things rather than by 
 what they do, to analyze the behavior 
 of others, to analyze the motives of 
 others, to predict how others will act. 
 
 8. sue Succorance: To have others 
 provide help when in trouble, to seek 
 encouragement from others, to have 
 others be kindly, to have others be 
 sympathetic and understanding about 
 personal problems, to receive a great 
 deal of affection from others, to have 
 others do favors cheerfully, to be 
 helped by others when depressed, to 
 have others feel sorry when one is 
 sick, to have a fuss made over one 
 when hurt. 
 
 9. dom Dominance: To argue for 
 one's point of view, to be a leader in 
 groups to which one belongs, to be 
 regarded by others as a leader, to be 
 elected or appointed chairman of 
 committees, to make group decisions, 
 to settle arguments and disputes be- 
 tween others, to persuade and influ- 
 ence others to do what one wants, to 
 
Interviewer Selection 
 
 41 
 
 supervise and direct the actions of 
 others, to tell others how to do their 
 jobs. 
 
 10. aba Abasement: To feel guilty 
 when one does something wrong, to 
 accept blame when things do not go 
 right, to feel that personal pain and 
 misery suffered does more good than 
 harm, to feel the need for punish- 
 ment for wrong doing, to feel better 
 when giving in and avoiding a fight 
 than when having one's own way, to 
 feel the need for confession of errors, 
 to feel depressed by inability to han- 
 dle situations, to feel timid in the 
 presence of superiors, to feel inferior 
 to others in most respects. 
 
 11. nur Nurturancc: To help 
 friends when they are in trouble, to 
 assist others less fortunate, to treat 
 others with kindness and sympathy, 
 to forgive others, to do small favors 
 for others, to be generous with others, 
 to sympathize with others who are 
 hurt or sick, to show a great deal of 
 affection toward others, to have oth- 
 ers confide in one about personal 
 problems. 
 
 12. chg Change: To do new and 
 different things, to travel, to meet 
 new people, to experience novelty 
 and change in daily routine, to exper- 
 iment and try new things, to eat in 
 new and different places, to try new 
 and different jobs, to move about the 
 country and live in different places, 
 
 to participate in new fads and 
 fashions. 
 
 13. end Endurance: To keep at a 
 job until it is finished, to complete 
 any job undertaken, to work hard 
 at a task, to keep at a puzzle or prob- 
 lem until it is solved, to work at a 
 single job before taking on others, to 
 stay up late working in order to get 
 a job done, to put in long hours of 
 work without distraction, to stick at 
 a problem even though it may seem 
 as if no progress is being made, to 
 avoid being interrupted while at 
 work. 
 
 14. het Heterosexuality: To go out 
 with members of the opposite sex, to 
 engage in social activities with the 
 opposite sex, to be in love with some- 
 one of the opposite sex, to kiss those 
 of the opposite sex, to be regarded as 
 physically attractive by those of the 
 opposite sex, to participate in discus- 
 sions about sex, to read books and 
 plays involving sex, to listen to or to 
 tell jokes involving sex, to become 
 sexually excited. 
 
 15. agg Aggression: To attack con- 
 trary points of view, to tell others 
 what one thinks about them, to criti- 
 cize others publicly, to make fun of 
 others, to tell others off when disa- 
 greeing with them, to get revenge for 
 insults, to become angry, to blame 
 others when things go wrong, to read 
 newspaper accounts of violence. 
 
 cedure obtained people having EPPS scores different from the adult pop- 
 ulation or from the college population used to set norms for the test. A 
 comparison of Columns 1, 2, and 5 in Table 6 indicates that it did. Inter- 
 viewers selected on Studies M3 27 and M4 showed significantly more 
 "intraception" and "dominance" than the adult population. The manifest 
 needs associated with intraception as shown in Figure 3 are "to analyze 
 one's motives and feelings," "to observe others, and "to put one's self in 
 another's place." Needs associated with dominance are, "to argue for 
 
 27 Two interviewers used on Wave 1 of Study M3 were no longer working 
 when the EPPS was administered. As a result, the averages for Study M3 were 
 based on 16 instead of 18 interviewers. 
 
42 
 
 Survey Reliability and Interviewer Competence 
 
 Table 6. Norms for Selected Populations and 
 
 Average Interviewer Scores on the Edwards 
 
 Personal Preference Schedule 
 
 EPPS 
 variables 
 
 Adult 
 sample 
 
 (1) 
 
 College 
 sample 
 
 (2) 
 
 Study 
 M3 
 
 (3) 
 
 Study 
 M4 
 
 (4) 
 
 Studies 
 
 M3 and 
 
 M4 
 
 (5) 
 
 Study 
 M4 
 
 rejects 
 (6) 
 
 
 14.1 
 
 14.5 
 
 15.2 
 
 12.1 
 
 13. a 
 
 16.3 
 
 14. 8" 
 
 11.9° 
 
 12. 2 C 
 
 15.9° 
 
 17. 2 a 
 
 15.1 
 
 16.7 
 
 10.5 
 
 11.5 
 
 11.5 
 
 14.4 
 
 11.8 
 
 10.2 
 
 14.3 
 
 13. 3 a 
 
 16.2 
 
 16.7° 
 
 11.6° 
 
 15.8 
 
 13.7 
 
 15.2 
 
 16.4 
 
 12.6° 
 
 16. C 
 
 11.7 
 
 11.6 
 
 15.5 
 14.8 
 13.2 
 12.6 
 12.9 
 15.0 
 18.8 
 7.9 
 15.5 
 12.2 
 14.5 
 17.8 
 14.6 
 14.4 
 10.4 
 11.4 
 
 14.5 
 14.6 
 14.3 
 12.7 
 
 9.7 b 
 16.5 
 19.0 
 
 9.2 
 17.2 
 11.9 
 15.9 
 15. 2 b 
 19. 2 b 
 10. 2 b 
 
 9.8 
 11.0 
 
 15.0 
 14.7 
 13.8 
 12.7 
 11.2 
 15.8 
 18.9 
 8.6 
 16.3 
 12.0 
 15.2 
 16.5 
 17.1 
 11.3 
 10.1 
 11.2 
 
 15.2 
 
 Deference 
 
 Order 
 
 14.8 
 13.4 
 14.4 
 
 Autonomy 
 
 10.3 
 16.7 
 
 Intraception 
 
 18.3 
 9.7 
 
 Dominance 
 
 Abasement 
 
 15.7 
 13.3 
 14.8 
 
 Change 
 
 16.2 
 
 Endurance 
 
 Heterosexuality 
 
 Aggression 
 
 Consistency 
 
 16. 7 d 
 
 9.2 
 
 11.6 
 
 11.5 
 
 
 8,963 
 
 1,509 
 
 16 
 
 19 
 
 35 
 
 37 
 
 
 
 a Significant difference at the 5 percent level between Study M4 and Study M3 interviewers 
 and the college and/or adult population as indicated. 
 
 b Significant difference at the 5 percent level between Study M4 and Study M3 interviewers. 
 
 c Significant difference at the 1 percent level between Study M4 and Study M3 interviewers 
 and the college and/or adult population as indicated. 
 
 d The lower endurance scores of the Study M4 rejects as compared with those accepted is 
 significant at the 5 percent level. 
 
 one's point of view," "to be a leader in groups to which one belongs," and 
 "to be regarded by others as a leader." 
 
 The interviewers used on the two studies showed less of the following 
 than the general adult population: autonomy, succorance, abasement, 
 and nurturance. Autonomy is associated with a need "to be independent 
 of others in making decisions," "to feel free to do what one wants," "to 
 criticize those in positions of authority," or "to avoid responsibilities and 
 obligations." Succorance is associated with a need "to seek encouragement 
 from others," "to have others provide help when in trouble," or "to have 
 others feel sorry when one is sick." Needs associated with abasement are, 
 "to feel guilty when one does something wrong," "to accept blame when 
 things do not go right," and "to feel that personal pain and misery suf- 
 fered does more good than harm." Clearly, excessive amounts of these 
 characteristics are not desirable in interviewers. Nurturance is associated 
 with the following needs: "to help friends when they are in trouble," 
 "to treat others with kindness and sympathy," and "to show a great deal 
 of affection toward others." While at first it may be thought that this is a 
 
Interviewer Selection 43 
 
 desirable characteristic for interviewers, there is a possibility that persons 
 with this characteristic are too expressive and have strong needs for 
 nurturance themselves. 
 
 A comparison of the selected interviewers' scores with those of the 
 college population showed that the former had significantly more intra- 
 ception and endurance. Endurance is described as the need "to keep at a 
 job until it is finished," "to work hard at a task," or "to stick at a problem 
 even though it seems as if no progress is being made." The selected inter- 
 viewers showed less of the following than the college population: succor- 
 ance and heterosexuality. The latter is associated with needs "to go out 
 with members of the opposite sex," and "to kiss those of the opposite sex." 
 This attribute probably reflects the stage of the life cycle and, hence, social 
 maturity. 
 
 Thus, the selection process appeared to have obtained persons with 
 desirable interviewer traits. If so, what part of the selection procedure 
 was responsible? Some insight into this question can be gained by com- 
 paring the applicants hired on the farm study with those rejected. A 
 review of Columns 4 and 6 in Table 6 indicates that with the exception 
 of endurance, the applicants rejected did not differ significantly from the 
 applicants hired. 
 
 It appears then that the selection of interviewers who differed from 
 the general adult population in characteristics measured by the EPPS 
 primarily occurred not because of letters of recommendation or personal 
 interview evaluation or other aspects of the formal selection procedure, 
 but because of the nature of the newspaper ad and the ad sent to school 
 teachers, and because a certain type of person tends to apply for inter- 
 viewing jobs. 28 These findings should not be interpreted to mean that the 
 letters of recommendation, evaluations of the applicants in personal inter- 
 views, and test scores were not useful in selecting effective interviewers but 
 rather that they were generally ineffective in further selection based on 
 the characteristics measured by the EPPS. 
 
 A comparison of the characteristics of the Study M4 interviewers with 
 those of the Study M3 interviewers shows that the former had significantly 
 higher scores on the endurance test and lower scores on the autonomy, 
 change, and heterosexuality tests. Of these, only change has not been 
 described. It involves the need "to do new and different things," "to 
 travel," "to meet new people," "to experience novelty and change in daily 
 routine," "to experiment and try new things," and "to participate in new 
 fads and fashions." On the surface, it would seem that these are desirable 
 
 28 This is not completely true since a few applicants were not given the EPPS 
 because they were too young or too old or because they were obviously unfit for 
 the job. 
 
44 Survey Reliability and Interviewer Competence 
 
 characteristics for interviewers. They are continually meeting new people 
 and experiencing new situations. However, change has a second dimen- 
 sion, and this one relates to instability, immaturity, and the like. The 
 absence of the latter characteristics is possibly of greater importance in an 
 interviewer than the presence of the former ones. The characteristics of 
 the Study M4 interviewers when contrasted with those of the Study M3 
 interviewers were not inconsistent with the change in interviewer stereo- 
 type from the more emotional, socially aggressive type of interviewer to 
 the friendly, matter-of-fact type which occurred as a result of the analysis 
 of the interviewers used in Study M3. 
 
 Of the 15 attributes measured by the tests comprising the EPPS, only 
 one, dominance, was significantly related to the pick-up rates of inter- 
 viewers. A coefficient of determination of .36 was obtained. In other 
 words, the dominance scores of the 16 interviewers explained 36 percent 
 of the variance in their pick-up rates. Subsequent analysis, reported later, 
 indicated an interaction between dominance scores and abasement scores. 
 
 References 
 
 All references other than personal friends and relatives were sent a 
 mimeographed one-page letter requesting "a frank statement on the en- 
 closed sheet concerning this person's qualifications for this work with par- 
 ticular reference to the characteristics listed." The sheet enclosed, shown 
 in Figure 4, sought ratings of the applicant with regard to a number of 
 different characteristics that were thought to be related to interviewer 
 ability. In addition, a blank space at the top of the sheet provided room 
 for the reference to write comments in his own words about the appli- 
 cant's fitness for the job. 
 
 The applicant evaluations were scored by assigning numerical values 
 to the ratings for each characteristic, ranging from four for "superior" 
 down to one for "below average." These scores were averaged and each 
 applicant was then assigned to one of the same four groups mentioned in 
 connection with the evaluation which resulted from the personal inter- 
 view. However, no relationship was observed between this measure and 
 the pick-up rate. 
 
 Coefficients of determination were prepared for all the characteristics 
 on the evaluation forms in relation to the pick-up rates for interviewers 
 used on Study M3. Of these, the references' evaluations of self-confidence 
 and of appearance, manner, and poise were significantly related to pick-up 
 rates. The coefficient of determination for self-confidence was .34, indi- 
 cating that 34 percent of the variance in the interviewers' pick-up rates 
 was explained by this variable. The coefficient of determination for ap- 
 pearance was .29. This suggests that further refinements in a reference 
 
Interviewer Selection 
 
 45 
 
 Figure 4. Reference Evaluation Form : Study M4 
 
 University of Illinois 
 Bureau of Economic and Business Research 
 
 APPLICANT EVALUATION 
 
 Personal evaluation of 
 
 A. Information about the applicant which you believe would be valuable in evaluating 
 this individual for work as an interviewer: 
 
 B. Following is a list of characteristics which all 
 For each of these, please indicate how you would 
 
 individuals have in varying degrees, 
 rate this applicant. 
 
 Characteristic 
 
 Superior 
 
 Above 
 average 
 
 Average 
 
 Below 
 average 
 
 Inadequate 
 
 basis for 
 
 judgment 
 
 Ability to make friends 
 
 
 
 
 
 
 Honesty 
 
 
 
 
 
 
 Self-confidence 
 
 
 
 
 
 
 Attention to detail 
 
 
 
 
 
 
 Tendency to procrastinate 
 
 
 
 
 
 
 Ability to think on feet 
 
 
 
 
 
 
 Appearance, manner, and poise 
 
 
 
 
 
 
 Initiative 
 
 
 
 
 
 
 Signed 
 
 Date 
 
46 Survey Reliability and Interviewer Competence 
 
 evaluation form might yield high returns in terms of selecting or identi- 
 fying potentially effective interviewers. 
 
 Selection of Interviewer Candidates 
 
 The selection of interviewer candidates was based on the following 
 scoring system (using each of the four tools employed in the evaluation of 
 applicants) with the following weights: 
 
 Method of Possibly Possibly 
 
 evaluation Hire hire reject Reject 
 
 Application form 8 6 4 2 
 
 Checklist evaluation 16 12 8 4 
 
 Personality test (M4 only) 8 6 4 2 
 
 References 16 12 8 4 
 
 As is evident from this weighting system, at the time the selection was 
 made more reliance was placed on the checklist evaluation and on the 
 information obtained from references than on the personality test or on 
 the application form. The actual procedure consisted of ranking the 
 applicants by their total scores and selecting as many as were needed 
 beginning from the top of the list. At the same time, certain additional 
 factors were taken into consideration, such as the location of the appli- 
 cant relative to the concentration of the sample members. 
 
 Whether this selection procedure did in fact select the most effective 
 interviewers could not be determined. Since no measure of performance 
 was available for those applicants who were rejected, it could not be 
 shown that the best interviewers were selected. 
 
 That this selection procedure was effective could only be measured in 
 the response and completeness rates obtained on the later studies where 
 the selection process was much improved and used effectively. What was 
 apparent from the interviewers selected was that much interviewer varia- 
 bility remained, and thus there was much room for improvement in the 
 selection procedure. 
 
 Once selected, applicants were sent a letter offering them training 29 
 (with pay) and the possibility of being hired if they performed satisfac- 
 torily in the training sessions. A letter was also sent to applicants who 
 were rejected, leaving open the possibility that they might be hired at a 
 future time. 
 
 Evaluation of Candidates Based on Training Experience 
 
 Candidates were evaluated on the basis of their conduct at the train- 
 ing sessions, on their scores on a qualifying examination, and on their 
 
 29 Refer to Table 5 for a breakdown of the numbers offered training on 
 Studies M3 and M4. 
 
Interviewer Selection 47 
 
 performance in trial interviews. Interviewer candidates were expected to 
 attend all training sessions and were penalized for lack of attendance 
 unless unusual circumstances were involved. In addition, each candidate's 
 general conduct was observed and evaluated. Did the candidate dress 
 appropriately, take written notes, listen attentively, ask questions, get 
 along with the other candidates, and prepare for the meetings? No at- 
 tempt was made to score each of these points. However, a candidate's 
 performance in these respects was used as a guide in deciding whether or 
 not to hire him as an interviewer. The specific content of these training 
 sessions will be discussed in the following chapter. 
 
 After the first three training sessions, candidates were given a take- 
 home "Interviewer Qualifying Examination," shown in Appendix A. 30 
 This examination was to be completed and returned to the project office 
 within one week. The examination was designed in such a way that the 
 majority of the questions could be answered directly from the written 
 training materials. However, some of the questions related to situations 
 described verbally in the meetings, and other questions introduced prob- 
 lem situations similar, though not identical, to those covered in the 
 meetings. 
 
 Essentially, the qualifying examination served two purposes. On the 
 one hand, it brought out how well the candidate had familiarized himself 
 with the training materials and had absorbed the discussions at the train- 
 ing sessions, relating not only to interviewing techniques but also to ac- 
 quaintanceship with financial terminology. On the other hand, the 
 examination forced the interviewer to consult the written materials (if he 
 expected to pass) and thus become better acquainted with this material, 
 thereby serving as an additional training device. 
 
 The qualifying examination was scored before the fourth training 
 session and was returned and discussed at that session. The scores served 
 as a basis for weeding out poor candidates as well as a basis for highlight- 
 ing areas in which other candidates were weak. The examination also 
 caused some candidates to drop out of the study because they felt it was 
 too hard. Although the main purpose of the examination was its use as a 
 training device, it was also used in the selection process. For those inter- 
 viewers who were selected for Study M3, correlation was not significant 
 between their test scores and their interviewing ability as measured by 
 their pick-up rates. This, of course, should not discredit the value of the 
 qualifying examination since its main purpose as a teaching tool was 
 satisfied. 
 
 Trial interviews served as an additional means of candidate evalua- 
 
 30 This examination was used in Study M4. Essentially the same examination 
 had been used in Studies M2 and M3. 
 
48 Survey Reliability and Interviewer Competence 
 
 tion. Three to five such interviews were assigned to each candidate at the 
 end of the third training session. The completed interviews were to be 
 returned to the project office within ten days. These interviews were 
 conducted in the same manner as regular interviews, with the same ad- 
 vance letter. After being checked and edited, they were returned to the 
 candidate. They, too, served as a basis for discussion in the fourth train- 
 ing session. 
 
 In Study M2, some of the trial interviews were dummy interviews in 
 the sense that arrangements were made beforehand with people in the 
 sample area to act as respondents. These people were not coached on 
 what answers to give. They were told to act natural and report only what 
 the interviewer requested, if they were so inclined. Following the inter- 
 view, these respondents informed the field director about the information 
 reported to the interviewer. In addition, they reported on the attitude of 
 the interviewer. An evaluation was then made of the interviewer's ability. 
 Although this experiment was adopted only in the pilot study, it proved 
 to be of assistance in selecting interviewers as well as in providing insights 
 into the interview situation from the point of view of the respondent. 
 Numerous problems involved in using the device limited its value. The 
 first problem which really had two aspects, was that of locating respond- 
 ents to play the role and briefing them. Second, and probably more im- 
 portant, was the problem of developing an adequate standard of evalua- 
 tion for the dummy interviews. Two other minor problems were those of 
 time and cost. However, taking these problems into consideration, it was 
 felt that the dummy interviews were a valuable tool in selecting effective 
 interviewers. 
 
 In Studies M3 and M4 no dummy interviews were attempted because 
 time and other restrictions prevented the making of necessary arrange- 
 ments. 
 
 Final Selection of Interviewers 
 
 Candidates were selected as interviewers only if they performed 
 satisfactorily on all three aspects of the evaluation process. In Study M3, 
 23 interviewers were selected from 50 candidates; in Study M4, 20 inter- 
 viewers were selected from 39 candidates (Table 5). The majority of 
 the candidates who were not selected eliminated themselves by quitting 
 because of the pressures of training sessions, qualifying examinations, and 
 trial interviews. 
 
 Once an interviewer was selected, he was asked to read and sign an 
 "Interviewer Agreement," which outlined the nature of the interviewing 
 task, the duties, and the arrangements for compensation (see Appendix 
 B). At the same time, it was made clear verbally that continuation on 
 the project was contingent on satisfactory interviewing performance. 
 
Interviewer Selection 49 
 
 Table 7. Distribution of Project Interviewers 
 
 by Selected Characteristics and by Study 
 
 (Percentages) 
 
 Characteristic category 
 
 Study 
 Ml 
 
 Study 
 
 M2 
 
 Study 
 M3 
 
 Study 
 M4 
 
 Sex 
 
 
 
 
 
 Male 
 
 29 
 
 58 
 
 70 
 
 70 
 
 Female 
 
 71 
 
 42 
 
 30 
 
 30 
 
 Age 
 
 
 
 
 
 Under 25 
 
 7 
 71 
 
 4 
 73 
 
 9 
 
 57 
 
 5 
 
 25-40 
 
 60 
 
 41-55 
 
 22 
 
 23 
 
 30 
 
 30 
 
 56 and over 
 
 
 
 
 
 4 
 
 5 
 
 Education 
 
 
 
 
 
 High school or less 
 
 43 
 
 27 
 
 4 
 
 
 
 College 
 
 50 
 
 62 
 
 44 
 
 80 
 
 Postgraduate 
 
 7 
 
 11 
 
 52 
 
 20 
 
 Occupation 
 
 
 
 
 
 Teacher 
 
 
 
 11 
 
 39 
 
 45 
 
 Other professional 
 
 36 
 
 31 
 
 22 
 
 5 
 
 Housewife 
 
 43 
 
 31 
 
 9 
 
 20 
 
 Other 
 
 21 
 
 27 
 
 30 
 
 30 
 
 Base (number of interviewers) 
 
 14 
 
 26 
 
 23 
 
 20 
 
 Table 7 shows the distribution of interviewers hired for the various 
 studies by selected characteristics. As is evident from this table, the great 
 majority were men (except in Study Ml), between the ages of 25 and 40 
 years of age, college educated (an appreciable number had postgraduate 
 work), and employed in professional occupations. The heavy ratio in 
 favor of men was surprising because no attempt had been made to favor 
 one sex over the other. 
 
 A Quantitative Method for Selecting Interviewers 
 
 To what extent can the information obtained about applicants and 
 trainees be utilized to select the potentially more effective interviewers? 
 The method employed for predicting interviewer effectiveness was linear 
 regression using an equation of the following form: 
 
 y = a + b-tZ-L + b 2 z 2 + • • • + b k z b + u, 
 
 where y was the measure of interviewer effectiveness, the z's were appli- 
 cant data variables, and u was a random variable. The 6's were estimated 
 from the n observations of interviewer characteristics on z t and y. If the 
 us were normally distributed, the least squares method also yielded a 
 maximum likelihood estimate. 
 
50 Survey Reliability and Interviewer Competence 
 
 The analysis employed the data from Study M3. Although it would 
 have been desirable to test the regression equations obtained from the 
 Study M4 data, this was not feasible in view of the difficulty in identifying 
 the effective interviewers on that study, as was previously discussed. In 
 addition, the interviewers were a fairly homogeneous group. Applicants 
 with adverse letters of recommendation or with adverse evaluation were 
 not hired. Applicants under 25 or over 50 and those with less than a 
 college education were rejected unless they had other compensating 
 characteristics. In general, persons employed in social types of work, such 
 as school teachers, were given preference. Those doing very poorly on 
 the take-home qualifying examination were not hired. For these reasons 
 and because of the small number of interviewers employed in Study M3, 
 the equations presented are little more than hypotheses for future testing. 
 
 The following variables, discussed earlier, were evaluated in an at- 
 tempt to find combinations which might prove useful in the selection of 
 interviewers. 
 
 1. Sex. 
 
 2. Age. 
 
 3. Education. 
 
 4. State of health (applicant's evaluation). 
 
 5. Length of residence at present address. 
 
 6. Length of time at present job. 
 
 7. Over- all estimate of number of hours available for interviewing 
 each week. 
 
 8. Financial background of interviewer. 
 
 9. Previous interviewing experience. 
 
 10. Reasons for applying for an interviewing position. 
 
 1 1 . Neatness of the application form and extent to which the applicant 
 followed directions in completing the form. 
 
 12. Each of the 10 scales making up the evaluation checklist form. 
 
 13. Each of the 8 scales making up the reference rating. 
 
 14. The EPPS scores on each of the 15 tests. 
 
 An IBM 1401 was employed to search for the best regression model. 
 Even so, many combinations of variables could not be evaluated. The 
 variables which in the previous section showed a relationship with the 
 pick-up rate were reference evaluation of "self-confidence," the "domi- 
 nance" score on the EPPS, and the checklist evaluation of "likableness." 
 Information on some of these variables was not obtained for all inter- 
 viewers. The first measure was available for all 18 interviewers, the 
 second one for 16 interviewers, and the last one for 11 interviewers. 
 Since most data were available for the 16 interviewers who took the 
 
Interviewer Selection 51 
 
 EPPS, these were used as the basis for the following regression equations. 
 
 Several criteria were employed in selecting the regression equations 
 presented. One was the percentage of variance in the pick-up rates of 
 interviewers explained by the regression equation. This was the coefficient 
 of multiple determination. Another was the ratio of net regression coeffi- 
 cients to their standard errors. This ratio combined with an evaluation 
 of beta coefficients gives some indication of the reliability and importance 
 of specific variables. Finally, the reliability of the estimate was evaluated 
 using the standard error of the estimate. 
 
 It was found that the two variables, self-confidence (references) and 
 dominance (EPPS), had to be present before the equations proved satis- 
 factory on the basis of the criteria applied. When these two variables 
 were combined with the over-all estimate of the number of hours available 
 for interviewing, the following multiple regression equation was obtained. 
 
 Pick-up rate = 2.89 + 4.217 z x + .853 z 2 + .224 z 3 
 
 (1.727) (.281) (.091) 
 R 2 = .71 
 S u = 4.58 
 
 In this equation z t was the reference evaluation of self-confidence of the 
 applicant, z 2 was the score on the EPPS dominance test, z 3 was the over- 
 all estimate of hours available for interviewing, and 2.89 was the constant 
 term. The number in parenthesis below each net regression coefficient 
 is the standard error of the coefficient. The R 2 is the corrected coefficient 
 of multiple determination, and S u is the corrected standard error of the 
 estimate. 31 
 
 The respective beta coefficients were B x = .422, B 2 = .538, and B 3 = 
 .400. 
 
 These variables explained about 71 percent of the variance in the 
 interviewers' pick-up rates. This differed significantly from zero, as did 
 each of the net regression coefficients. 
 
 If the interviewers were divided into thirds on the basis of their pre- 
 dicted pick-up rate, the actual pick-up rate for each group would reflect 
 the effectiveness of the equation in differentiating among interviewers. 
 
 Predicted Completeness Pick-up 
 
 pick-up rate rate rate 
 
 Highest third 79% 58% 
 
 Middle third 66 49 
 
 Lower third 58 41 
 
 All interviewers 68 50 
 
 31 Mordecai Ezekiel and Karl A. Fox, Methods of Correlation and Regression 
 Analysis (New York: Wiley, 1959), pp. 300-2. tmiuFRSfll Of ^ uN °* 5 
 
52 Survey Reliability and Interviewer Competence 
 
 Thus, the pick-up rate for the highest third would be 1 7 percent above 
 that for the lowest third. In addition, the middle third would be about 
 midway between the two extreme groups, suggesting a continuous differ- 
 entiation among interviewers. 
 
 These findings suggest that the selection process may be made more 
 rigorous. However, the findings presented require further validation on 
 larger groups of interviewers. 
 
IV. INTERVIEWER TRAINING 
 
 The amount and intensity of interviewer training required tends to 
 vary with, among other things, the complexity of the study, the famili- 
 arity of the interviewers with the subject matter, the type of question- 
 naire being used, and the availability of time and other resources for 
 adequate instruction. On highly specialized studies, such training may be 
 extensive, especially in the case of a panel operation. Training may be 
 given by mail, by phone, by personal contact with individuals, or in group 
 meetings. As a rule, group meetings are preferable, if feasible, since it is 
 only in this way that standardization of procedures may be achieved. 
 
 The training procedures on the later Consumer Savings Project studies 
 evolved out of the experiences gained on the earlier studies. Although 
 it is felt that these training methods were good, the dimensions of a 
 training program are hard to quantify and evaluate objectively. 
 
 Various goals may be sought in training sessions. The following 
 warrant particular attention, and were also those to receive the most 
 emphasis in the training sessions for the panel studies of the Consumer 
 Savings Project. 
 
 1. To appreciate the value of economic research. 
 
 2. To develop an understanding and appreciation of the particular 
 study. 
 
 3. To develop the ability to obtain interviews, with special emphasis 
 on the possible approaches to reluctant sample members and to those 
 unwilling to cooperate. 
 
 4. To instill an understanding of the need for accurate and complete 
 data and of the techniques and methods by which such information can 
 be obtained. 
 
 5. To familiarize the interviewers fully with the questionnaire, with 
 the purpose and nature of the particular questions, and with dimensions 
 within which answers are to be sought. 
 
 6. To build morale — to convey to the interviewer the feeling that he 
 is a member of a group. 
 
 7. To weed out unqualified interviewers by observing the ability of 
 each candidate to absorb training materials and to conduct satisfactory 
 interviews. 
 
 53 
 
54 Survey Reliability and Interviewer Competence 
 
 8. To promote standardization of procedures and of the type of an- 
 swers to be sought for the different questions. 
 
 A variety of techniques are available for achieving these objectives, 
 ranging from the assignment of background reading material to the con- 
 ducting of trial interviews and assigning interviewers to edit and code 
 questionnaires. 32 A number of these procedures were used in the training 
 sessions of the Consumer Savings Project. 
 
 Interviewer Training for Consumer Financial Studies 
 
 Interviewers employed on the Consumer Savings Project had to be 
 thoroughly trained for two major reasons. First, the study was concerned 
 with savings, assets, and debts, which are subjects about which many 
 respondents are extremely sensitive. Second, because of the experimental 
 nature of the project, its success was highly dependent on good field 
 work. The interviewers had to understand the purpose of the study and 
 had to follow instructions. If they were confronted with new situations 
 not covered in the training, they had to be able to make the correct 
 decision on the basis of their training and understanding of the goals of 
 the project. 
 
 The training sessions for each pilot study incorporated the experience 
 gained from the training sessions for the preceding pilot studies. 
 
 Instructions for most types of situations were covered in the training 
 manual. They were also reviewed verbally and thus reinforced. At the 
 beginning of each study, the interviewers were given initial training 
 which consisted of four meetings 2'/2 to 3 hours in length. A training 
 session was also held before each new wave of interviews, followed on 
 some of the waves of later studies by a "review session" after the inter- 
 viewers had had a chance to complete a few interviews. In addition, 
 meetings were held at the completion of each wave to review experi- 
 ences on that wave. 
 
 Initial Training Sessions 
 
 The initial meetings had two purposes: to train each interviewer in 
 the techniques and methods needed to accomplish the objectives of the 
 
 32 For a discussion of some of these approaches see Robert L. Kahn and 
 Charles F. Cannell, The Dynamics of Interviewing (New York: Wiley, 1957), 
 pp. 241-51; Lester Guest, "A New Training Method for Opinion Interviewers," 
 Public Opinion Quarterly, Vol. 18, No. 3 (Fall, 1954), pp. 287-99; U.S. Bureau 
 of the Census, Principal Data-Collection Forms and Procedures (Washington: 
 U.S. Government Printing Office, 1961); Muriel Harris, compiler, Documents 
 Used During the Selection and Training of Social Survey Interviewers and Se- 
 lected Papers on Interviewers and Interviewing (London: Central Office of Infor- 
 mation, The Social Survey, 1956). 
 
Interviewer Training 55 
 
 Figure 5. Interviewer Guide: Study M4 
 
 Table of Contents 
 
 Page 
 
 A. What This Study Is About 
 
 1. Objectives 1 
 
 2. Sponsorship 1 
 
 3. Reason for Project 1 
 
 4. Selection of Sample 2 
 
 5. Panel Nature of Operation 3 
 
 6. Experimental Aspects 3 
 
 B. Terms You Must Know 3 
 
 1. General Definitions 4 
 
 2. Financial Terms 5 
 
 C. Explanations of Individual Forms 
 
 1. Form IV HI 14 
 
 2. Form IV CI 33 
 
 3. Accuracy Card 42 
 
 4. Interviewer Report Form (IRF) 43 
 
 5. Interview Evaluation Form 47 
 
 study and to weed out those who apparently could not learn to obtain 
 good interviews. 
 
 In addition to the general goals mentioned earlier, these training 
 sessions had the objective of instructing the interviewers in the meanings 
 of financial terms. When the respondent gave information about specific 
 financial holdings (debentures, endowment life insurance, and so forth), 
 the interviewer needed to know what the respondent was talking about. 
 Knowledge of the financial terms used could serve to facilitate communi- 
 cation between the interviewer and the respondent and make the collec- 
 tion of accurate and complete data more probable. 
 
 These objectives were achieved in a number of ways. At the first 
 meeting, each interviewer was given a packet containing an "Interviewer 
 Guide," a set of "Pointers on the Interview Situation," a schedule of 
 material to be covered in all the meetings, and the field forms to be used 
 on the first wave of interviews. Tables of contents of the "Guide" and 
 "Pointers" for Study M4 are shown in Figures 5 and 6; Figure 7 presents 
 the schedule of materials covered in the first three training sessions for 
 that study. 
 
 The "Interviewer Guide" was designed and indexed so that inter- 
 viewers could use it themselves to answer questions after formal training 
 was completed. Its substantive aim was threefold. 
 
 1. To convey a clear explanation of what the study was all about — 
 
56 Survey Reliability and Interviewer Competence 
 
 Figure 6. Outline of "Pointers on The Interview Situation" : 
 
 Study M4 
 
 Page 
 
 A. Securing the Interview 
 
 1. Paving the Way 1 
 
 2. Credentials 1 
 
 3. Contacting the Respondent 1 
 
 4. General Ways of Securing Cooperation 2 
 
 Importance of the Study 2 
 
 Importance of Each Respondent 3 
 
 Confidential Treatment of Data 3 
 
 No Personal Interest in Data 4 
 
 5. Contact Situations and How To Handle 4 
 
 Refusals 4 
 
 Excuses 6 
 
 Sickness or Serious Illness 8 
 
 Appointments 8 
 
 General Problems on Contact Results 9 
 
 6. Noncontact Situations and How to Handle 9 
 
 Moved or Not There 9 
 
 Not Home 9 
 
 B. The Interview 
 
 1. Introduction and Explanation 10 
 
 2. Creating and Maintaining Rapport 1 1 
 
 3. Type of Data Sought 12 
 
 4. Refusals and "Don't Know" Answers 12 
 
 5. Completing the Interview 14 
 
 6. Use of Records 15 
 
 its aims and objectives, the types of data needed, and how the data fitted 
 in with the aims and objectives. 
 
 2. To present a glossary of financial terms. 
 
 3. To discuss the forms to be used in the first wave of interviews, first 
 in general terms to indicate the purpose and functions of each, and 
 then question by question to point out possible sources of misunderstand- 
 ing and to make clear what information was being sought. 
 
 Judging from the interviewers' reactions to the "Guide," it served its 
 practical purpose of helping them obtain complete, accurate information. 
 As one interviewer stated, "Although the initial set of instructions seemed 
 overwhelming, they proved to be highly useful and were of considerable 
 aid." 
 
 The pamphlet, "Pointers on the Interview Situation," was designed to 
 supplement the "Interviewer Guide" by focusing on the procedures to be 
 followed in obtaining an interview with the sample member. In addition 
 to discussing means of approaching the sample member, a substantial 
 
Interviewer Training 57 
 
 portion of this material (Sections A4 to A6 in Figure 6) was devoted to a 
 listing of different contact situations that might be encountered, with 
 specific suggestions for handling each situation. In Studies M3 and M4, 
 these "case situations" were based on actual interview situations experi- 
 enced in earlier studies. Thus, practical examples were undoubtedly 
 helpful in increasing the number of interviews obtained. 
 
 The schedule outlined in Figure 7 indicates the subject matter covered 
 in the first three training sessions and the order of discussion. These 
 sessions were held on successive days (usually in the evening). The first 
 session was devoted to a review of the study, the means of securing the 
 cooperation of the sample member, the time schedule for that wave, and 
 the schedule of interviewer compensation. Special attention was given 
 to the last of these aspects in Studies M3 and M4, payment for which 
 was made on an interview basis geared to an incentive system, as will be 
 explained in Chapter V. 
 
 The main focus of the second session was on the forms to be used, 
 with emphasis on an item-by-item review of the questionnaire and of the 
 interviewer report form. The final part of this session was concerned 
 with a discussion of operating procedures both in contacting respondents 
 and in filling out and returning forms. 
 
 The third session was devoted to a general review of the study objec- 
 tives and questionnaire forms and to role-playing. At the end of that 
 session, candidates were handed copies of the qualifying take-home 
 examination and were given pretest interview assignments. 
 
 The fourth and final session was held about ten days after the first 
 three. In the interim, the qualifying examination and the pretest inter- 
 views had been completed and a decision had been made as to which 
 candidates were to be hired. Only the latter were invited to the fourth 
 session. Even with the care taken at the previous three sessions to give 
 the interviewer a solid background, there were many problems to be 
 discussed at this session. The interviewers benefited from the opportunity 
 to talk over their problems and experiences after they had been exposed 
 to some of the actual field work. 
 
 The first part of the meeting was used to further instruct the inter- 
 viewers in the weak areas indicated by the qualifying examination and 
 the trial interviews. The interviewers were encouraged to discuss the 
 problems they encountered and how they solved them. The corrected 
 copies of the qualifying examination were returned, and there was a 
 review of the most frequently missed questions. Interviewers could also 
 ask about specific questions which were marked wrong but not reviewed. 
 
 Next, the meeting turned to reviewing and correcting the trial inter- 
 views. Each interviewer was given one of his completed assignments 
 
58 Survey Reliability and Interviewer Competence 
 
 Figure 7. Schedule Outline for Training Sessions: Study M4 
 
 I. First Session 
 
 A. Distribution of all materials 
 
 B. Introduction of field staff and interviewers 
 
 C. Discussion of time schedule form 
 
 D. Discussion of interviewer compensation form 
 
 E. Presentation of facts about the study (Interviewer Guide — Part A) 
 
 1. Objectives 
 
 2. Sponsorship 
 
 3. Reason for project 
 
 4. Selection of sample 
 
 5. Panel operation 
 
 6. Experimental aspects 
 
 F. Explanation of terms (Interviewer Guide — Part B) 
 
 G. Suggestions for securing the interview (Pointers on the Interview Situa- 
 
 tion — Part A) 
 
 1. Paving the way 
 
 2. Credentials 
 
 3. Contacting the respondent 
 
 4. General ways of securing cooperation 
 
 5. Contact situations and how to handle 
 
 6. Noncontact situations and how to handle 
 
 H. Discussion of the interview (Pointers on the Interview Situation — 
 Part B) 
 
 1. Introduction and explanation 
 
 2. Creating and maintaining rapport 
 
 3. Type of data sought 
 
 4. Refusals and "don't know" answers 
 
 5. Completing the interview 
 
 6. Use of records 
 
 I. Summary of material covered 
 
 II. Second Session 
 
 A. Question period and discussion of first session 
 
 B. Question-by-question review of Form IV HI (Interviewer Guide — 
 
 PartC-1) 
 
 C. Question-by-question review of Form IV CI (Interviewer Guide — 
 
 Part C-2) 
 
 D. Explanation of the accuracy card (Interviewer Guide — Part C-3) 
 
 E. Discussion of the interviewer report form (Interviewer Guide — Part 
 
 C-4) 
 
 F. Discussion of the interview evaluation form (Interviewer Guide — Part 
 
 C-5) 
 
 G. Explanation of general subjects 
 
 1. Step-by-step interviewing procedures 
 
 2. Spot-checking of interviews 
 
 3. Return of carbon copies of advance letter 
 
 4. General field procedures 
 H. Review of list of assignments 
 
Interviewer Training 59 
 
 III. Third Session 
 
 A. Question period and discussion of second session 
 
 B. Explanation of role-playing (Form IV HI) 
 
 C. Explanation of role-playing (Form IV CI) 
 
 D. Final review 
 
 E. Presentation of pretest assignments 
 
 F. Distribution of copies of take-home qualifying examination 
 
 along with a critique sheet and as he checked over his mistakes, the field 
 director answered the interviewer's questions and explained the reasons 
 for some of the corrections. As each assignment was finished, the inter- 
 viewers were given their other completed assignments one at a time until 
 all were reviewed and corrected. 
 
 The last part of the meeting was devoted to a general discussion of 
 any topic or problem which the interviewers felt they did not under- 
 stand. They were then given two copies of the "Interviewer Agreement," 
 shown in Appendix B. One copy they signed and returned, and the other 
 they kept for their own records. The main purpose of the "Interviewer 
 Agreement" was to outline clearly the working arrangements and the 
 interviewer's job responsibilities. The agreement seemed to have added 
 value in making the interviewers feel obligated to do the job. This last 
 training session was ended by handing out the interviewing assignments. 
 
 All training sessions were held in the city where the field operation 
 was being carried out except in Study M4 when two cities readily acces- 
 sible to the majority of the candidates were used for training sessions. 
 The same material was covered on two consecutive evenings in the two 
 cities; this permitted the interviewer to come to either session. 
 
 As a rule, the size of each group was restricted to 15 or 16 candidates. 
 This number was small enough for individual questions to be answered, 
 and yet large enough to reflect a variety of viewpoints. Groups larger 
 than 15 or 16 were divided into separate groups, with sessions for each 
 group. 
 
 Present at all training sessions were the project director or his deputy; 
 the field director, who presided over the meeting; and field supervisors, 
 if they were being used on that study. The project director or his deputy 
 conducted the first part of the initial session and was present at later 
 sessions to answer any technical questions that might arise. 
 
 The day and time of the training sessions varied with each study. 
 They depended upon the distances the candidates lived from the meeting 
 place, their other activities, when the stores were open in their areas, 
 and so forth. However, the participants generally preferred that the meet- 
 ings be held in the evenings about 7:30 during the middle of the week, 
 
60 Survey Reliability and Interviewer Competence 
 
 although in some areas Friday evenings or Saturday mornings were 
 preferred. The meetings were planned to last no longer than three hours, 
 with a fifteen-minute break in the middle. 
 
 Compensation for the training sessions varied between $1.50 and $1.65 
 an hour, depending upon the going rate in each area. At the end of 
 every meeting the candidates filled out, signed, and submitted a bill for the 
 time spent. No travel time or mileage was paid for attendance at the 
 training sessions. Candidates were also paid for up to two hours of study 
 time on their own and for work on the qualifying examination. 33 
 
 No attempt was made to evaluate the training, but some conclusions 
 were based on the subjective judgment of the people involved. The use 
 of detailed, written instructions with specific examples was an absolute 
 necessity; however, they had to be adequately indexed for quick reference 
 in the field. The value of the qualifying examination as a training tool 
 was not to be overlooked. In fact, any means of exposing the interviewer 
 to the written training material was desirable. Trial interviews, although 
 originally intended as a selection device, proved extremely effective in 
 getting interviewers to put training into practice. The interviewer did, 
 however, have to be thoroughly trained before attempting these trial 
 interviews. 
 
 Two areas of training which it was thought could be of value on 
 future financial studies were role-playing and the further development of 
 the interviewer's confidence in his ability to obtain the personal financial 
 data. 
 
 Inter-wave Sessions 
 
 The purpose of the training sessions before each new wave of inter- 
 views was threefold: first, to maintain and improve the quality and 
 quantity of data obtained on succeeding waves; second, to stress good 
 interviewing practices and correct poor ones; and third, to train inter- 
 viewers on any new procedures, experiments, or forms to be used on the 
 current wave. As a rule, these sessions consisted of just one approximately 
 three-hour meeting, although on several occasions when a fairly large 
 amount of material had to be covered, two meetings were held. 
 
 Each inter-wave training session was scheduled one week before the 
 start of field operations. At this meeting, the response and refusal rates of 
 the previous wave were reported, since interviewers had expressed interest 
 in knowing how they had done. Also presented were statistics on the 
 quality of the work received, number of blanks, number of errors, per- 
 centage of records used, and so forth. This was a logical lead-in to a 
 discussion of how these areas could be improved. 
 
 33 See Chapter V for details on interviewer compensation. 
 
Interviewer Training 61 
 
 The first part of the meeting focused, therefore, on giving the inter- 
 viewers additional instruction in areas of poor performance. Also at this 
 time, there was a discussion and review of the purpose and objectives of 
 the study to reinforce the interviewers' confidence in the study and give 
 them additional material to overcome potential refusals on the present 
 wave. Since panel members tended to forget why they were being inter- 
 viewed and what was being done with the data, the interviewers had to 
 be prepared to answer such questions again, in order to reduce fears and 
 skepticism about the authenticity of the study, the use of the data, or the 
 confidential treatment of answers. The importance of retaining each 
 respondent was emphasized. 
 
 Next to be covered were the different aspects of the present wave of 
 interviews, the experiments being used, a question-by-question review of 
 the questionnaires, and any changes in field procedures from those of the 
 previous wave. The discussion of each question included its purpose, 
 possible answers, problems that might arise, and methods of solving these 
 problems. An attempt was made at this point to relate the answers to 
 later data analysis, showing the need for careful field work and the 
 importance of complete and accurate data. For each wave, the inter- 
 viewers were given new instruction manuals incorporating the procedures 
 to be used on that wave. Verbal presentation supplemented the written 
 manual and was also used to clear up questions raised in the meeting. 
 
 On the farm study (M4), some additional training procedures were 
 used. First, a copy of the new questionnaire for that wave and written 
 instructions were included with the letter to the interviewer informing 
 him of the training session. At the meeting, little time was spent in 
 reviewing the questionnaire. Instead, each interviewer was given a copy 
 of the new questionnaire with fictitious feedback information filled in. 34 
 The field director then gave answers to be filled in for that wave as if he 
 were the respondent. Completed questionnaires were collected and taken 
 back to the office for detailed editing. A record was kept of all mistakes, 
 blanks, and so forth, and memos were sent to all the interviewers pointing 
 out errors and suggesting ways to improve the quality of their work. The 
 filled-in fictitious questionnaires were returned with the memos to the 
 interviewers so they could see their mistakes. 
 
 At the end of this review session, the interviewers were given four or 
 five assignments as a start. These had to be completed and edited 
 
 34 For the regular assignments, each interviewer would receive for each panel 
 member information from the preceding interview. This feedback information was 
 filled in on the new questionnaire form and consisted of holdings reported previ- 
 ously and names of institutions. The interviewer was to obtain the current balance 
 in the account (not filled in from previous wave) and bring other details up to date 
 where relevant. 
 
62 Survey Reliability and Interviewer Competence 
 
 thoroughly by the office before more assignments were given. In this way, 
 many mistakes were caught early in the wave, before they became com- 
 mon practice. 
 
 On such a panel-type operation, it appeared that the inter-wave 
 meetings contributed much to obtaining accurate and complete data. 
 This was especially true when there was a time lapse of a few months 
 between waves. 
 
 End-wave Meetings 
 
 After the completion of each wave, interviewers were requested to 
 attend an end-of-the-wave meeting, which had three purposes: to clear 
 up any field problems remaining from the wave just completed; to 
 determine which areas of the interview and the questionnaire gave the 
 most trouble and how these situations were handled; and to improve 
 interviewer morale through discussion of common problems and the 
 sharing of experiences. An additional purpose was to lay the ground- 
 work for the next wave of interviews to be conducted approximately three 
 months later. 
 
 The meetings were held not later than one week from the time the last 
 interview was completed for that wave, while experiences were still fresh 
 in the interviewers' minds. A permissive atmosphere was maintained, so 
 the interviewers would talk openly about their own personal interviewing 
 problems. These individual problems often proved to be general problems. 
 
 In contrast to the structured training session at the start of the wave, 
 the end-wave meeting took the form of open discussion. Control was 
 maintained by the field director, who presided at these meetings. In 
 addition, a staff analyst was also present to record weak areas of the just- 
 completed wave and to use these experiences to avoid mistakes in design- 
 ing the questionnaire and field procedures for the following wave. 
 
 The end-wave meeting was especially useful for clearing up any re- 
 maining details of the just-completed wave. Problems of completed inter- 
 views with blanks, unclear answers, and so forth could be discussed with 
 the interviewers. The interviewers turned in all remaining assignments 
 which were being held in the field, often with the hope of obtaining 
 interviews with panel members so far not available, temporarily out of 
 town, or busy. At the conclusion of this meeting, the wave was officially 
 closed. 
 
 The final points covered at the end-wave meeting were the tentative 
 plans for the following wave. Subjects included were the approximate 
 date for the start of interviewing, who would be interviewed, (all panel 
 members or only half of the panel members), the approach to be used, 
 and other known experiments and procedures to be followed. The inter- 
 
Interviewer Training 63 
 
 viewers were therefore likely to feel more involved with the study, since 
 they knew what to expect on the next wave. 
 
 Of all the end-wave meetings, the last one at the end of each study 
 proved to be especially rewarding. The interviewers could then feel free 
 to talk about aspects of the study which they may have suppressed in 
 order to maintain good job relations during the study. Some of the more 
 valuable points mentioned by the interviewers at the last meetings follow. 
 
 Study M2 
 
 1. Importance of interviewers' knowledge of financial terms. Possi- 
 bility of having separate briefing sessions with one session on review of 
 terms. 
 
 2. Usefulness of a mid-wave meeting to stimulate morale, especially 
 in the early waves of an operation. 
 
 3. Completion of one or two assignments by an interviewer and no 
 more until these have been checked by the supervisor. 
 
 4. Notification of interviewers as to when to expect pay checks, and 
 promptness in paying them on that date. 
 
 Study M3 
 
 1. Helpfulness of more instructions placed right on the questionnaire 
 forms, such as instructions to cover all members of the savings unit and 
 to probe for additional holdings. 
 
 2. Desirability of more attitudinal questions, since they were extremely 
 useful in maintaining respondent rapport. 
 
 3. Possibility of talking with interviewers individually to solve personal 
 problems instead of having mid-wave meetings. 
 
 4. Appreciation of the initial set of written instructions which proved 
 to be highly useful even though they seemed overwhelming at first. 
 
 Study M4 
 
 1. Desirability of having the interviewers in a future farm study 
 interview an actual farmer during the initial training sessions. 
 
 2. Possible usefulness of advance training on income tax forms and 
 Farm Bureau Management Records. 
 
 3. Damage of some attitudinal questions to rapport. 
 
 4. Resolution of possible personality conflicts by having each inter- 
 viewer rate the cooperativeness of his panel members on the first wave 
 and switching uncooperative members to other interviewers on the next 
 wave. 
 
 At the last meeting of Study M4 interviewers, an additional method 
 of obtaining information from them was used. Enclosed with the letter 
 sent to the interviewers informing them of that meeting was a list of 
 
64 Survey Reliability and Interviewer Competence 
 
 questions about certain aspects of the study. The interviewers were asked 
 to bring their answers with them to the meeting. In this way, they 
 had a chance to give more thought to their answers and to suggest new 
 ideas. 
 
 Recommended Training Procedures 
 
 The training procedures described on the preceding pages appeared 
 to fulfill all the necessary requirements. Indeed, questionnaires distributed 
 to the interviewers at the end of the initial sessions in Studies M3 and M4, 
 and filled in anonymously, indicated general satisfaction with the pro- 
 cedures and with the time allotted to each topic. The only criticism was 
 that some of the interviewers felt that too much time had been wasted 
 by letting interviewers talk about unimportant subjects and personal 
 experiences. Nevertheless, it was felt desirable to allow considerable dis- 
 cussion of this sort rather than to cut off further questions. 
 
 The dummy interviews, the qualifying take-home examination, and 
 the practice completion of questionnaires at the later wave meetings 
 appeared to have been particularly useful devices for training the inter- 
 viewers and for clarifying the type of information being sought. The 
 presence at all sessions of a project analyst, in some cases the project 
 director, also appeared to have been highly worth while, serving to convey 
 to the interviewers more of a feeling of the importance of the study and 
 at the same time making it possible to answer any technical questions 
 that arose. 
 
 On the panel aspects of the operation, the inter-wave training sessions 
 seemed to be essential. Such meetings not only helped make sure that 
 the interviewers were fully cognizant of changes in the questionnaire, but 
 had also not forgotten what the survey was all about and how to handle 
 particular situations as they occurred. These meetings also served to 
 maintain interviewer morale, for which purpose it was found of par- 
 ticular value to review the experiences on the previous wave and to 
 point out errors and omissions that might have been made at that time. 
 
 The end-wave meetings at the completion of each wave also proved 
 to be worth while. Besides giving the interviewers an indication of appre- 
 ciation for their services, these meetings were highly useful as a basis for 
 designing the questionnaire for the following wave and for avoiding 
 earlier mistakes. However, such meetings were not treated as a substitute 
 for a pre-wave meeting to review the new questionnaire and operating 
 procedures. 
 
 Some means of determining the effectiveness of training came from 
 three distinct areas. The response and pick-up rates comprised the first 
 of these. The improvement in training procedures from wave to wave 
 
Interviewer Training 65 
 
 was to some degree reflected in these rates. However, many other vari- 
 ables were operative which may have accounted in part for the improved 
 response and contact rates. Since each study was slightly different, it was 
 not possible to compare response and pick-up rates between studies. 
 
 The second measure of effectiveness was to be found in the amount of 
 supervision required. As training procedures improved from wave to 
 wave and from study to study, the need for supervision decreased ; that is, 
 interviewers tended to become their own supervisors and catch their own 
 errors. 
 
 The third evaluation of training effectiveness came from the inter- 
 viewers themselves. The interviewers' reaction to the training was that it 
 was adequate to good. The few complaints about certain aspects of 
 training were resolved with improvements in the training as the project 
 progressed. 
 
 Needless to say, these recommended training procedures need to be 
 tested under controlled experiments to establish their true value. At the 
 same time, other possible training procedures should be investigated to 
 determine the best training to employ in future financial studies. 
 
V. INTERVIEWER SUPERVISION 
 
 The purpose of interviewer supervision on a personal interview study 
 is to see that complete and accurate information is obtained from all 
 respondents. There are three functions involved. The first is to see that 
 written and verbal instructions to interviewers are carried out in a manner 
 consistent with the original plan of the study. Interpreting these instruc- 
 tions for handling specific, unusual problems is included in this function. 
 
 The second function of supervision is to identify interviewers who are 
 not meeting the standards of completeness and accuracy set for the study. 
 The identification of these interviewers on the basis of their performance 
 has a number of uses. First, more effective interviewers can be selected 
 for succeeding waves on a panel operation. Second, interviewing organi- 
 zations can improve the effectiveness of their permanent staff of inter- 
 viewers. Third, it may prove feasible to set up a sequential analysis to 
 identify ineffective interviewers as interviewing progresses on a single 
 interview study. 35 Ineffective interviewers can then receive further train- 
 ing or can be dismissed. Fourth, when trainees are not formally hired 
 until interviewing performance on pilot studies or pretests is evaluated, 
 it is possible to hire only the more effective trainees. 
 
 The final function of supervision is maintaining good interviewer 
 morale. This function is interrelated with the other two, and all are 
 necessary for effective supervision. 
 
 This chapter will cover these three functions in the following manner. 
 The first part will review the procedures of supervision employed on 
 Study M3 and will present a quantitative method for identifying effective 
 interviewers on the basis of interviewing performance. The second part 
 will cover those aspects of supervision on the other project studies, which 
 differed from the supervision on Study M3. Next, interviewer and super- 
 visor compensation used on the project studies will be discussed. A sum- 
 mary of the findings and recommendations for the future will comprise 
 the last part of this chapter. 
 
 35 A similar type of analysis, though one which is not sequential, is proposed 
 by W. E. Deming in his book, Sample Design in Business Research (New York: 
 Wiley, 1960), p. 250. 
 
 66 
 
Interviewer Supervision 67 
 
 Supervision in Study M3 
 
 A detailed discussion of the procedures of supervision employed on 
 Study M3 will be presented. Unless otherwise indicated, similar pro- 
 cedures and practices were employed on the other project studies. Study 
 M3 was selected because validation data were available which facilitated 
 a more rigorous evaluation of interviewer performance. In addition, 
 Study M3 culminated the work in metropolitan areas since, Study M4 
 was conducted in a farm area. 
 
 Selecting the Supervisor 
 
 The function of a field supervisor in Study M3 was limited to editing 
 questionnaires and resolving questions and problems of the interviewers. 
 This was done because the location of the sample area was relatively close 
 to the project headquarters, enabling the field director to maintain close 
 touch with the interviewers. 
 
 Two supervisors, hereafter referred to as editors, were hired to help 
 with the field supervision of the 19 interviewers. One editor was a 36- 
 year-old woman with considerable market research interviewing and 
 supervisory experience. The other editor was a 42-year-old woman with 
 a full-time job, a business college background and also considerable inter- 
 viewing and supervisory experience. The latter editor did not do any 
 interviewing on Study M3, whereas the former interviewed on all five 
 waves. One of the principal experiments of the study consisted of testing 
 a highly structured approach on half of the sample and a highly un- 
 structured approach on the other half of the sample. Interviewers were 
 given both types of assignments and each editor was trained to cope with 
 the problems arising from each approach. 
 
 Interview Assignments and Reassignments 
 
 Interview assignments were sent from the project office directly to the 
 interviewers in three separate mailings, at one-week intervals. To facili- 
 tate supervision, lists of assignments were sent to the editors who urged 
 the interviewers to make personal contact at these sample addresses within 
 a few days. The purpose of this quick contact was to optimize the value 
 of the advance letter sent to all sample members. 
 
 On the first wave, all noncontacts and refusals were reassigned by the 
 editors to different interviewers. This procedure resulted in obtaining 
 additional interviews, thereby increasing the response and contact rates. 
 Refusals and noncontacts were not reassigned on later waves unless there 
 appeared to be a possibility that a new interviewer could obtain an inter- 
 view. The editors also reassigned sample members if an interviewer quit, 
 
68 Survey Reliability and Interviewer Competence 
 
 was dismissed, or could not complete his assignments by the deadline 
 date for that wave. 
 
 Training and Retraining of Interviewers 
 
 Interviewers were trained primarily by the field director. However, 
 before each training session, the editors were briefed on the content of the 
 
 Figure 8. Reminder Notes for Wave 2 : Study M3 
 
 Checklist for Handling Each Assignment 
 
 A. Before the Interview 
 
 1. Review Wave 1 IRF. 
 
 a. When, where, and who (best time, place, and person to interview). 
 
 b. Attitude of the panel member. 
 
 c. "Summary Comments" and "Interviewer's Notes." 
 
 2. Study "Problems and Comments" sheet if with assignment. 
 
 3. Review "Background Form" (III-2). 
 
 a. Date of last interview. 
 
 b. Family composition. 
 
 4. Look over the "Recording Form" (III HA2, III HS2, III CA2, or III 
 CS2). 
 
 a. Approach being used. 
 
 b. Note if there is any missing information. 
 
 5. Read Wave 2 advance letter. 
 
 a. Correct name and address if in error. 
 
 b. Note any special circumstances in letter. 
 
 6. Have extra cards A, B, and C with you. 
 
 B. During the Interview 
 
 1. Follow instructions for the approach being used. 
 
 2. At end of interview review information given, filling in all blanks. 
 
 C. After the Interview 
 
 1. Correct name and address on carbon of advance letter and mail directly 
 to Champaign. 
 
 2. Edit thoroughly Wave 2 IRF, the Background Form, and the Recording 
 Form. If any information is missing or not clear, correct before sending 
 to the editor. 
 
 3. Send to the editor: 
 
 a. III-IRF-2. 
 
 b. Background Form (III-2). 
 
 c. Recording Form (III HA2, III HS2, III CA2, or III CS2). 
 
 d. Wave 1 IRF. 
 
 e. Problems and comments sheet, if any is included. 
 
 4. Record of your bill : 
 
 a. Date interview completed. 
 
 b. Interviewee number. 
 
 c. Interviewee initials. 
 
 d. Rate from lower right hand corner of Wave 2 IRF. 
 
Interviewer Supervision 69 
 
 session and their reactions to proposed procedures were secured. In addi- 
 tion, both editors had to train interviewers who could not attend some 
 of the later wave sessions, and in some instances they had to give further 
 training to interviewers who were having particular problems. 
 
 As training improved on each wave, a continuing effort was made to 
 have each interviewer be his own editor. For example, on Wave 2 the 
 interviewers were given a one-page instructional sheet called "Reminder 
 Notes," which was a checklist for handling each assignment (see Figure 
 8) . The editors' work was greatly reduced when the interviewers followed 
 the checklist. 
 
 Returning Completed Work 
 
 Completed assignments, regardless of whether they were interviews, 
 refusals, or noncontacts, were to be sent to the editors by 9 : 00 a.m. on the 
 day following their completion. This compelled the interviewers to edit 
 and complete assignments before the details were forgotten. This also 
 made it possible for the editors to reassign noninterviews and for the office 
 to send thank-you letters soon after the interviews were completed. To 
 facilitate mailing, stamped envelopes addressed to the editors were en- 
 closed with the assignments when they were given to the interviewers. 
 If completed assignments, upon receipt by the editors, had no editing 
 corrections, they could be sent to the project office. If there were minor 
 errors, these were usually corrected through phone calls to the inter- 
 viewers. If major omissions of information were involved, the editors 
 returned the assignments to the interviewers for re-editing or for reinter- 
 views with the sample members to obtain the missing data. These proce- 
 dures, it was felt, forced the interviewers to edit their own questionnaires, 
 and this reduced the frequency with which incomplete questionnaires were 
 sent to the editors. 
 
 On Wave 5, the interviewers returned all completed assignments di- 
 rectly to the project office rather than to the field editors. On the previous 
 four waves, the interviewers had been conditioned to look over their work 
 carefully before sending it to the editors. On the last wave, it was felt 
 that the time schedule could be better controlled at the office level, and 
 this method proved very satisfactory. In a meeting at the end of the 
 study, the interviewers expressed the feeling that editors were essential 
 for the first wave and possibly for the second wave, but that there was no 
 real need for them on subsequent waves. 
 
 Identifying Effective Interviewers 
 
 Five methods were used in the identification of good interviewers: 
 ( 1 ) determining the quality of their work from the evaluations of editors 
 
70 
 
 Survey Reliability and Interviewer Competence 
 
 oS -d 
 
 a 
 
 H 
 
 C/3 
 
 H 
 
 Pi 
 O 
 
 w 
 
 5 
 
 w 
 w 
 
 o 
 
 H 
 
 3 
 
 3 
 
 p 
 o 
 
 h-l 
 
 EH 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 % 
 
 •E 
 
 0) 
 H 
 
 >d 
 
 g, 
 
 ■ri 
 
 < 
 
 ■d 
 
 I 
 
 0) 
 
 ■d >d 
 
 8 s 
 
 o> B 
 o 
 
 0) £j 
 
 O 5 
 
 ■d 
 
 0) 
 
 w 
 
 0> 
 K 
 
 •P 
 o 
 ri 
 
 O 
 
 o 
 
 i 
 
 c 
 o 
 a 
 
 1 
 
 o 
 
 ri 
 
 •H 
 •P 
 CQ 
 
Interviewer Supervision 71 
 
 in the field and editors and coders in the office; (2) evaluating their work 
 through the use of "The Editor's Weekly Report"; (3) spot checking to 
 estimate the quantity and quality of their work; (4) changing inter- 
 viewers on a later wave; and (5) employing a quantitative method. The 
 validation data were used to evaluate these five means of identifying 
 effective interviewers. 
 
 Editing 
 
 Editing consisted of a detailed check of the questionnaire and of the 
 Interviewer Report Form (Appendix C) for blanks, unclear answers, 
 obvious inconsistencies, and unclear explanations of unusual situations. 
 The editors noted excessive "don't know" answers and question refusals 
 in the interviewers' work and discussed this problem with them. Formal 
 records of each interviewer's performance were prepared, based on office 
 coding and editing. The field editors were informed of office tabulations 
 so they could watch or retrain poor interviewers. On each succeeding 
 wave, this record of errors was used as a basis for interviewer training. 
 At the end of each wave the interviewers were given a report of their 
 performance on that wave as well as a summary of the performance of 
 all interviewers. 
 
 Editor's Weekly Report 
 
 To identify interviewers who were procrastinating in completing as- 
 signments, the editors were required to prepare and send to the office the 
 "Editor's Weekly Report" (see Figure 9). As its name implies, this 
 report was sent weekly to the field director, and gave a quantitative 
 picture of the field operations as of midnight Saturday. This form was 
 useful in spotting inactive interviewers and in expediting completion of 
 the field work. 
 
 Spot Checking 
 
 On Wave 3, a spot-check procedure was instituted on the basis of 
 validation material. If the known financial holding was listed on the 
 questionnaire, it was assumed that the interviewer had made the inter- 
 view and no check was made. If the known holding in the financial 
 institution was not reported, a post card was sent to the respondent 
 ostensibly seeking information on the length of the interview and on the 
 interviewer's characteristics. Such a post card is shown in Figure 10. 
 A total of 80 cards were sent and 62.5 percent were returned. All of the 
 sample members indicated they were interviewed, although two gave 
 their interviewers a poor rating. Of the 50 returned, 23 gave additional 
 written comments, most of which were favorable. A sample was picked 
 from the 30 who did not return the post card and approximately 15 
 phone calls were made to check if the interview had actually occurred. 
 
72 Survey Reliability and Interviewer Competence 
 
 Figure 10. Spot-check Post Card: Study M3 
 The interview on took No 
 
 ] less than 15 minutes ] 35 to 44 minutes 
 
 ] 15 to 24 minutes ] 45 to 59 minutes 
 
 1 25 to 34 minutes | ] one hour or more 
 
 The interviewer impressed me as follows: 
 
 |~1 Nice person to talk to \~\ Aggressive, pushy 
 
 2 Quiet ] Not very bright 
 l2 Intelligent Q Overly friendly 
 J Neat, well-dressed | ] Talkative 
 
 ! ' ] Tactful i ] Poor appearance 
 
 D- 
 
 Comments . 
 
 No falsification was uncovered by these calls. It was felt that the primary 
 value of the use of the post cards was in forestalling interviewer cheating, 
 as interviewers were informed during the initial training sessions that this 
 type of spot check would be made. 36 
 
 Changing Interviewers 
 
 At the initial training sessions, interviewers were informed that half 
 of their panel members would be reassigned to other interviewers on the 
 third wave. Although the main purpose of this experiment was to explore 
 the possibility that the new interviewer might pick up information missed 
 by the earlier one, the interviewers were undoubtedly aware that this 
 procedure would also serve to uncover falsified interviews. In addition, 
 knowing that this procedure was to be used may well have motivated the 
 interviewers to do a more thorough job of interviewing. No appreciable 
 difference in response and pick-up rates was observed between panel 
 members who were reassigned and those who were not. 
 
 Using a Quantitative Method 
 
 For Study M3, the pick-up rate (response rate times the completeness 
 rate) was employed as the criterion of interviewer effectiveness. Where 
 such validated criteria are not available, other variables must be employed. 
 The present analysis evaluates the extent to which these other variables 
 could have been used to identify interviewers who had a high pick-up 
 rate on Study M3. 
 
 36 Based on a recent experiment, a question may be raised as to the value of 
 postcards to detect interviewer cheating. "In this experiment, ten people selected 
 at random from the Chicago telephone directory were sent thank-you letters, on 
 University of Illinois stationery, for granting an interview on their finances. Al- 
 though these people were not actually interviewed, five returned the postcard indi- 
 cating the length of the interview and their opinion of the interviewer!" Robert 
 Ferber, The Measurement and Control of Errors: Savings Data, op. cit., Chap- 
 ter 13. 
 
Interviewer Supervision 73 
 
 The following variables may be calculated for most survey field 
 studies : 
 
 1. Response rate. 
 
 2. Percentage of ambiguous (unclear) answers. 
 
 3. Percentage of refusals of specific information. 
 
 4. Percentage of respondents using records. 
 
 5. Percentage of questionnaires evaluated as "less than fully complete" 
 (interviewer evaluation) . 
 
 6. Average length of interview. 
 
 7. Variability in length of interview. 
 
 8. Percentage of interviews in which husband and wife both par- 
 ticipated. 
 
 9. Mean number of specific holdings reported. 
 
 The relationship between each of these variables and each inter- 
 viewer's pick-up rate will be evaluated first. Two types of statistical 
 analysis will be employed. The first is correlation analysis, using the 
 coefficient of determination corrected for degrees of freedom (r 2 ). 37 The 
 other test employed is the Mann- Whitney U Test. 38 This is used to test 
 the significance of the difference between the scores of one interviewer 
 group and those of another. Its primary drawback is that the difference 
 between only two groups of interviewers can be tested at a time. 
 
 1. Response rate. The response rate is frequently employed as an indi- 
 cator of interviewer effectiveness, 39 since it is assumed that the interviewer 
 who does well at the door will also be effective in the interview. Although 
 significant, an r 2 of only .18 was obtained between the pick-up rate and 
 the response rate. The direction of this relationship was positive. In 
 addition, an r 2 of .05 was obtained between response rates and complete- 
 ness rates. This did not differ significantly from zero. 
 
 In view of the importance attached to the response rate as a means 
 of evaluating interviewer effectiveness on sample surveys, these findings 
 suggested that a much more vigorous examination of the relationship be- 
 tween response rates and interviewer effectiveness was needed. 
 
 2. Percentage of ambiguous answers. The most frequent type of 
 ambiguous answer occurred when the interviewer left a question blank 
 
 37 Since the sample size is the same for the coefficients of determination pre- 
 sented in this section, it might be noted that an r 2 of .121 differs significantly from 
 zero at the 5 percent level, and an r 2 of .266 is significantly different from zero at 
 the 1 percent level. In this section, all r 2 values are corrected values and, if re- 
 ported significant, are significant at the 5 percent level unless otherwise stated. 
 
 3S See Sidney Siegel, Nonpar ametric Statistics (New York: McGraw-Hill, 
 1956), pp. 116-27. 
 
 39 Donald M. Hobart, Marketing Research Practice (New York: Ronald, 
 1950), p. 102. 
 
74 Survey Reliability and Interviewer Competence 
 
 where some explanation was required. It may have occurred because the 
 respondent did not own an asset or had "no change" in the asset and 
 the interviewer, under the pressure of the interview situation, failed to 
 check the appropriate box. An ambiguous answer may also have occurred 
 because the interviewer failed to do adequate probing. The number of 
 ambiguous answers in a questionnaire tended to indicate something of 
 the interviewer's thoroughness in work habits, either in terms of whether 
 the questionnaire was edited or in terms of the thoroughness of the 
 editing. 
 
 Ambiguities on four types of holdings were counted : savings accounts, 
 checking accounts, life insurance, and debts. The number of these as a 
 percentage of four times the number of interviews taken by an interviewer 
 was the criterion used. The average per interview was 2 percent. 
 
 The corrected coefficient of determination for the percentage of 
 ambiguities was zero for the pick-up rate, which suggested that the use 
 of this variable to indicate over-all interviewer effectiveness was ques- 
 tionable. 
 
 3. Percentage of refusals on specific information. Some respondents 
 granted interviews but then refused to answer certain questions. A 
 "don't know" response was classified for the purposes of this analysis as a 
 refusal. While in some cases a "don't know" answer may have been 
 a truthful one, in others it was another way of refusing information. The 
 measure used was the number of refusals and "don't know" answers 
 recorded by an interviewer under the sections dealing with savings ac- 
 counts, checking accounts, life insurance, and debts as a percentage of 
 four times his number of interviews. A corrected coefficient of determina- 
 tion of zero was obtained between this measure and the pick-up rate. 
 
 4. Percentage of respondents using records. Two measures of the use 
 of records were prepared. One was based on the percentage of reported 
 holdings where records were used, including all types of assets and debts 
 except charge accounts. The measure employed was the percentage of an 
 interviewer's respondents using records on 25 percent or more of their 
 reported holdings. An r 2 of .02 was obtained between this measure and 
 the pick-up rate. This coefficient did not differ significantly from zero. 
 
 Of some interest was the fact that an r 2 of .01 was obtained between 
 the use of records and the percentage of refusals on individual holdings. 
 An r 2 of .01 was also obtained with the percentage of ambiguous answers. 
 
 5. Percentage of questionnaires evaluated as "less than fully com- 
 plete." The measure used was the percentage of respondents whose re- 
 ports of holdings of savings accounts were evaluated by the interviewers as 
 "less than fully complete" (see Appendix C) . An r 2 for this variable with 
 the pick-up rate was zero. 
 
Interviewer Supervision 75 
 
 A corrected coefficient of .33 was obtained between use of records and 
 evaluation of completeness. The regression coefficient was negative. This 
 indicated^ as would have been expected, that the use of records by a 
 respondent influenced the interviewer's evaluation of respondent coopera- 
 tion. 
 
 6. Average length of interview. The length of the interview expressed 
 in minutes was not related to the pick-up rate. The length of interview, 
 however, did show a significant adjusted coefficient of .21 with the re- 
 sponse rate. The regression coefficient was negative, indicating that 
 interviewers with high response rates tended to take a shorter length of 
 time in the interview. This may have reflected the inclusion of less 
 cooperative panel members or basic differences in interviewer character- 
 istics and techniques. 
 
 7. Variability in length of interview. This measure was prepared as 
 follows. The mean length of interview with respondents in four different 
 size categories of total financial holdings was computed for each inter- 
 view. The time required for each interview was subtracted from the 
 average for the size category and squared. These squared deviations 
 were summed for each interviewer across all size-of-holdings groups. This 
 can be summarized as 
 
 EE^-za 2 , 
 
 i=l j=l 
 
 where X\j was the ;th interview in the ith size-of-holdings category, n was 
 the number of interviews taken by the interviewer in the ith size-of- 
 holdings group, and m was 4. 
 
 The coefficient of determination for this variable with pick-up rate 
 did not differ significantly from zero. However, a significant r 2 of .16 
 was obtained between this measure and the percentage of ambiguous 
 answers. 
 
 8. Percentage of interviews in which husband and wife both partic- 
 ipated. Interviewers were instructed to obtain joint interviews with 
 husband and wife whenever possible. It was felt that more accurate 
 reporting would be obtained in this type of interview. The findings from 
 the study indicated that this was, in fact, true. Those interviews in which 
 both husband and wife participated were significantly more accurate than 
 those in which only one member of the savings unit participated. 
 
 The percentage of interviews in which both husband and wife par- 
 ticipated was prepared for each interviewer. However, an r 2 of zero was 
 obtained between this measure and interviewer pick-up rate. 
 
 Interviewer response rate was significantly related to the percentage 
 of interviews in which both husband and wife participated. The r 2 was .22 
 and the regression coefficient was positive. 
 
76 Survey Reliability and Interviewer Competence 
 
 9. Mean number of specific holdings reported. Since one character- 
 istic of the more effective interviewers was their ability to obtain mention 
 of holdings more frequently than the less effective ones, the mean number 
 of holdings reported by their respondents may have been larger. The 
 holdings used were those with a minimum of respondent variability where 
 the number held by a respondent was limited and where most respondents 
 owned holdings. Three types of holdings were included: checking 
 accounts, savings accounts, and non-charge-account debts. An adjusted 
 coefficient of determination of zero was obtained for each of these hold- 
 ings, yet the mean number of each of these holdings reported by respond- 
 ents of the more effective interviewer groups was larger, though not 
 significantly larger, than that for the less effective interviewer groups. 
 
 The criteria of interviewer effectiveness frequently employed on sur- 
 veys for which no validation data are available are response rates, per- 
 centage of ambiguous answers, and percentage of refusals of specific 
 information. The following regression was obtained using these variables. 
 
 Pick-up rate = 8.7 + .621*1 - .285z 2 - .281z 3 
 
 (.245) (.557) (.253) 
 R 2 = .18 
 S u = 8.6 
 
 In this equation Ziwas the response rate, z 2 was the percentage of ambig- 
 uous answers, z 3 was the percentage of refusals of specific information, 
 and 8.7 was the constant term. The number in parentheses below each 
 net regression coefficient is the standard error of the coefficient. The 
 R 2 is the corrected coefficient of multiple determination, and the S u is the 
 corrected standard error of the estimate. 
 
 These variables explained about 18 percent of the variance in the 
 pick-up rate of the interviewers, which was significantly different from 
 zero. The net regression coefficient for response rates was the only one 
 which differed significantly from zero. The reliability of estimates ob- 
 tained with this equation, as reflected by the standard error of the esti- 
 mate, was low when compared with that obtained with subsequent equa- 
 tions employing other variables. 
 
 If the interviewers were divided into thirds on the basis of their pre- 
 dicted pick-up rates, the actual pick-up rates for these groups could be 
 compared. The results of such a grouping are presented below. 
 
 Predicted Completeness Pick-up Number of 
 
 pick-up rate rate rate interviewers 
 
 Highest third 67% 54% 6 
 
 Middle third 67 48 6 
 
 Lowest third 72 47 6 
 
 All interviewers 68 50 18 
 
Interviewer Supervision 77 
 
 While in terms of the pick-up rate there was some tendency to select 
 the most effective interviewers, this equation did not distinguish among 
 the lower two-thirds of the interviewers. In fact, there was a tendency 
 for the equation to eliminate the interviewers who would be more effec- 
 tive in the interview as indicated by the higher completeness rate of the 
 lowest third of the interviewers. 
 
 An interviewer's performance in the interview once he is past the door 
 is frequently evaluated by the percentage of ambiguous answers and the 
 percentage of refusals of specific data. The measure of interviewer effec- 
 tiveness in the interview employed in this study was the completeness rate. 
 To what extent could the percentage of ambiguous answers (z 2 ) and of 
 refusals (z 3 ) be used to identify the interviewers who were effective in the 
 interview? The following equation was obtained. 
 
 Completeness rate = 74.7 — A30zo — .560z 3 
 
 (.663) (.308) 
 R 2 = .08 
 S u = 11.0 
 
 The small amount of variance explained by these two variables indi- 
 cates that studies in which they are used as criteria of interviewer effec- 
 tiveness are measuring something different from what is measured by the 
 completeness rate employed in the present analysis. 
 
 A number of different combinations of the nine variables were tested 
 in an attempt to explain the varying pick-up rates of interviewers. Since 
 the response rate is a major component of the pick-up rate, it seemed 
 logical to include it. The following equation was the best discovered in 
 terms of the criteria presented in Chapter III. 
 
 Pick-up rate = 
 
 20.6 + 1.118 Zl - .450*3 + -165z 4 + .136z 6 - .177z 7 + .231* 8 
 (.219) (.193) (.068) (.059) (.054) (.118) 
 R 2 = .63 
 
 "a — D.O 
 
 In this equation z x was the response rate of each interviewer, z 3 was 
 the percentage refusing specific information, z 4 was the percentage using 
 records, z 6 was the average length of interview, z 7 was the variability in 
 length of interview, and z 8 was the percentage of interviews with both 
 husband and wife. 
 
 The respective beta coefficients were as follows : 
 
 fl x = 1.174, B 3 = .420, B % = .375, B 6 = .479, B 7 = .539, and B a = 
 .379. 
 
 The correlation coefficient was significantly different from zero as was 
 each of the net regression coefficients. The response rate stood out as 
 
78 Survey Reliability and Interviewer Competence 
 
 being of particular importance in selecting effective interviewers. The 
 remaining variables were of about equal importance. 
 
 If the 18 interviewers were ranked on the basis of their computed 
 pick-up rate, the highest third would have an observed pick-up rate 18 
 percentage points higher than the lowest third. In addition, as the fol- 
 lowing tabulation indicates, the middle third would have a rate midway 
 between the two extreme groups, suggesting a continuous differentiation 
 of interviewers by this regression equation. 
 
 Computed Completeness Pick-up 
 
 pick-up rate rate rate 
 
 Highest third 78% 59% 
 
 Middle third 67 50 
 
 Lowest third 59 41 
 
 All interviewers 68 50 
 
 Since the completeness rate of the highest third is 18 percentage 
 points above those of the lowest third, this equation did identify inter- 
 viewers who were more effective in the interview as well as at the door. 
 
 These findings suggest several things. One is that effective interviewers 
 can be identified on the basis of interviewing performance, which in turn 
 suggests that more reliable regression equations might form the basis for 
 a sequential analysis model which would signal when interviewers need 
 additional training or possibly dismissal. Such a model could be useful 
 on a one-time survey as well as on a continuing survey operation. The 
 findings also suggest the need for further research on interviewer effective- 
 ness. Since these equations evolved from Study M3 data, the findings 
 presented here would require independent testing to establish their validity 
 and reliability. 
 
 Interviewer Morale 
 
 An important part of the job of supervision is establishing and main- 
 taining a high level of interviewer morale. Clearly, interviewer. morale is 
 an important factor influencing the completeness and accuracy of infor- 
 mation obtained by interviewers. Yet, because of its subjective nature, 
 it is difficult to identify the dimensions of interviewer morale and the 
 factors which influence it. Further work on this important problem is 
 needed. 
 
 One factor influencing interviewer morale is the interviewer's percep- 
 tion of the importance of the information collected. While the importance 
 of the study can be communicated by the study and field directors, its 
 importance may be indicated with equal or greater force by the actions 
 of supervisors and the standards they set. 
 
 Interviewer morale is also influenced by the interviewer's perception 
 of a respondent's willingness to report the information sought. As was 
 
Interviewer Supervision 79 
 
 indicated earlier, a function of the training sessions in the project studies 
 was to create favorable interviewer attitudes toward interviewing. The 
 supervisor can influence interviewer attitudes through recognition and 
 discussion of interviewer problems. Close contact between supervisor and 
 interviewer, where problems can frequently be discussed, can contribute 
 to favorable interviewer attitudes. 
 
 An interviewer's attitude toward his ability to deal with problems 
 which arise also influences morale. Here again, a part of the supervisor's 
 role can be viewed as an extension of the training program. Inevitably, 
 unforseen problems arise in the field. If a supervisor is available, the 
 interviewer can discuss a problem and resolve it immediately. As a result, 
 on succeeding interviews the interviewer feels that he is able to handle 
 such problems if they arise. In the project studies frequent letters and 
 phone calls from the office were also employed to give the interviewer the 
 feeling that he was being kept up-to-date and that he had expert advice 
 available if problems arose. 
 
 As a further means of maintaining a high morale, response and con- 
 tact rates were discussed at meetings during the wave and at the end of 
 the wave. In this way, interviewers could evaluate their work. At these 
 meetings, interviewers were also encouraged to discuss their problems 
 openly. Not only were answers to these problems worked out, but inter- 
 viewers could see that the other interviewers had similar problems. 
 
 Aspects of Supervision in Other Project Studies 
 Which Differed From Study M3 
 
 Field Supervisors Not Used for Study M4 
 
 On Study M4, the farm study, all supervision was handled directly by 
 the field director, and no field editors or supervisors were hired. The 
 primary reason for this procedure was that the sample members lived in 
 a three-county area covering approximately 3,000 square miles. As a 
 result, interviewers were scattered correspondingly over a wide area to 
 minimize travel time and expense. Since the sample area was close to 
 the project headquarters, it was felt that little saving would have been 
 achieved by having supervisors or editors maintain contact with the 
 interviewers. 
 
 On Study M4, a policy was adopted of giving assignments to the inter- 
 viewers based on how many assignments they were holding at the time. 
 At no time were the interviewers to have a backlog of unworked assign- 
 ments. Previous experience had shown that interviewers with many un- 
 worked assignments were likely either to do a poor job in order to com- 
 plete the interviews or not to complete them by the deadline date. 
 
 To aid the interviewers in the completion of their assignments and to 
 
80 Survey Reliability and Interviewer Competence 
 
 reduce travel time, assignments were given by geographic areas. As had 
 been done in the previous studies, a stamped post card addressed to the 
 office was enclosed with each group of assignments sent to the inter- 
 viewers. The return of this card notified the office that the assignments 
 had been received. To complete the card, interviewers had to look over 
 their material and indicate on the card what had been received. This, it 
 was found, encouraged them to begin interviewing more quickly and, 
 thus, to obtain the benefit of the advance letter. On the later waves, after 
 rapport had been established, contacting the panel member soon after 
 the advance letter had been sent did not seem to be as important as it 
 had been on earlier waves. 
 
 On Waves 4 and 5 a deadline date was set for the completion of each 
 assignment in order to obtain still greater control of the time schedule. 
 For the majority of the assignments, the deadline date was approximately 
 one week after the assignment was sent to the interviewer. This allowed 
 time for the interviewer to get in touch with the panel member, make an 
 appointment if necessary, and obtain the interview. If the interviewer was 
 not able to complete an assignment by its deadline date, he had to call 
 the field director and explain the delay. If the explanation was reason- 
 able and if there was a possibility of an interview, the assignment was 
 left with the interviewer and a new deadline date was set. In a few 
 instances the interviewer was requested to send the incomplete assignment 
 to the field office, and it was then either dropped or assigned to a new 
 interviewer. 
 
 A close record of deadline dates was maintained so that if an inter- 
 viewer neglected to call and explain why an assignment was not com- 
 pleted, he would be called by the field director. This procedure proved 
 worth while and was instrumental in getting the field work done in one 
 month compared with two or more months for previous waves and pre- 
 vious studies. 
 
 Return of Carbon Copy of Advance Letters, Study M4 
 
 On Study M4, the interviewers were requested to return the carbon 
 copy of the advance letter in a separate envelope as soon as the assign- 
 ment was completed. Therefore, when a carbon copy was received, the 
 field director knew that the assignment had been completed and that the 
 questionnaire could be expected in the mail within the next day or two. 
 Interviewers were contacted and asked for an explanation if the com- 
 pleted assignment was not received within three days after this carbon had 
 been returned. 
 
 Interviewer Morale in Study M4 
 
 When interviewers work by themselves without close field supervision, 
 
Interviewer Supervision 81 
 
 there is a good chance that morale will suffer. In Study M4 interviewers 
 were encouraged to get in touch with the field director at any time of the 
 day or night if they had a problem. They, too, were contacted frequently 
 — at least once a week — by phone, by letter, or by a personal visit. 
 Care was taken not to give the interviewers too many assignments at one 
 time, and pressure to complete assignments was kept at a reasonable level. 
 The quantity and quality of each interviewer's work was kept under close 
 observation, and if either appeared to be deteriorating the interviewer was 
 contacted and the problem discussed with him. 
 
 When supervision was handled by the field director, tighter control 
 was possible even though the interviewers worked more independently. 
 The project staff knew what was wanted, and controls were set up so 
 that no time was lost by having the assignments go through field super- 
 visors who were not so familiar with all the needs of the study. Poor- 
 quality field work was caught early and corrected quickly. Time schedules 
 were maintained due to the use of deadline dates for each assignment. In 
 addition, tight control facilitated the prompt payment of interviewers, an 
 important factor in interviewer morale. 
 
 Compensation for Field Workers 
 Supervisors' Compensation 
 
 Since field work was subcontracted on Study Ml and because there 
 were no field supervisors on Study M4, this description of supervisors' 
 compensation will be related only to Studies M2 and M3. 
 
 Supervisors were paid on an hourly basis plus all mileage and expenses. 
 They turned in a weekly time and expense sheet with an itemized record 
 of all miles traveled each day, phone calls, postage, and supplies. They 
 were not reimbursed for meals or other personal expenses. 
 
 For Study M2, the beginning hourly rate was $1.85. This was in- 
 creased 5 cents on each succeeding wave, so that by Wave 5 the super- 
 visors were earning $2.05 per hour. In addition, they were compensated 
 at this same hourly rate for any interviewing they did outside of their 
 supervisory functions. 
 
 For Study M3, a flat rate of $2.00 an hour was paid on all waves. If 
 the supervisors did any personal interviewing besides their supervisory 
 work, they were compensated for this at the same rate per interview as 
 the other interviewers. 
 
 Interviewers' Compensation 
 
 Two very different methods of compensating interviewers were used. 
 On studies Ml and M2, interviewers were paid on an hourly basis with 
 expenses for mileage, phone calls, and miscellaneous items. On Studies 
 
82 Survey Reliability and Interviewer Competence 
 
 M3 and M4, interviewers were paid by the interview supplemented by 
 hourly pay for meetings. 
 
 The hourly payment method was adopted at the beginning of the 
 project in accordance with general survey practice and in accordance with 
 the dictates of much of the literature on the subject. For Study Ml the 
 interviewers were paid on an hourly basis by the interviewing organiza- 
 tions employed. This practice was continued for Study M2. It is gener- 
 ally argued that payment by the hour encourages interviewers to allow 
 sufficient time to conduct the interviews and not to feel that rushing is 
 necessary. To attract good candidates the rate offered was somewhat 
 above the going hourly rate in the area. Interviewers were offered $1.65 
 per hour for all time spent on the study, including meeting and study 
 time. In addition, interviewers were reimbursed for phone calls, postage, 
 and stationery, and received 5 cents per mile for mileage (Figure 11). 
 Furthermore, as an incentive to remain with the study, the hourly rate 
 was raised 5 cents on each wave so that by the fifth wave the inter- 
 viewers were earning $1.85 per hour. 
 
 Experience with this method of compensation brought out various 
 shortcomings. For one thing, high variability in the cost per interview 
 was apparent from one interviewer to another. In part, this may have 
 been due to padding of time sheets by some interviewers. More likely, 
 however, was padding of a different sort; namely, stopping at a sample 
 address if it were near the place the interviewer was going, even though 
 he had not really expected to find the sample member at home. 
 
 Another problem was that the system of compensation failed to pro- 
 vide an incentive for interviewers to complete assignments on time. Fur- 
 thermore, the 5 cent increase per wave seemed to provide virtually no 
 incentive for the interviewers to remain with the study. Interviewers who 
 dropped out as well as interviewers who remained throughout the study 
 felt that the additional pay did not influence their decision. Particularly 
 pertinent in this respect were the comments of many interviewers that 
 being paid on an hourly basis was degrading in view of their professional 
 status in their regular jobs. Their feeling seemed to be that payment for 
 work accomplished would be a more appropriate system. Moreover, com- 
 plaints were registered about the amount of paperwork required to record 
 hours worked and expenses. 
 
 After considering these results, it was decided to pay the interviewers 
 on Study M3 by the completed interview. This approach, it was hoped, 
 would offset the objections raised. Moreover, with strict control over the 
 quality of the interviewing, it was felt that such an approach would 
 enable interviewers to earn a more satisfactory hourly rate than was 
 possible in the previous studies and would at the same time yield more 
 
Interviewer Supervision 
 
 83 
 
 £ 
 
 § 
 
 U 
 
 
 3 
 
 3 
 
 rt 
 
 
 W 
 
 
 M 
 
 
 c 
 
 
 
 
 TJ 
 
 
 C 
 
 
 •T2 ^ 
 
 Q 
 D 
 H 
 1/3 
 
 Q 
 02 
 O 
 O 
 
 <3 
 
 W 
 cc 
 Z 
 
 a 
 
 Oh 
 
 X 
 Q 
 
 :s 
 
 & 
 
 
 Jf.f 
 
 *S 're 
 
 c 
 
 P 
 
 X! 
 < 
 
84 Survey Reliability and Interviewer Competence 
 
 reliable data. The interviewers would not have to spend time on fringe 
 or unproductive activities such as filling in the time and expense record, 
 or making two or three trips to an area for interviews with panel members 
 living close together rather than trying to complete them all on the same 
 trip. In addition it was hoped that they would be more efficient in trying 
 to make contact — going to the back door as well as to the front door 
 in trying to make contact at a sample address. In short, interviewer 
 activities would be directed toward the completion of an interview and 
 the obtaining of accurate and complete information. 
 
 The average rates per interview paid by the hour for Studies Ml and 
 M2 were used as the basis for setting the rates per interview for Study M3. 
 The rates established were $10 for each of the first 10 completed inter- 
 views, $13 each for all interviews over 10, and $15 for each interview 
 obtained from a panel member who had previously refused another inter- 
 viewer. Once the $10 or $13 rate was established, it remained constant 
 for the remaining four waves, regardless of the interviewer. If an inter- 
 viewer persuaded a panel member who had previously refused an inter- 
 view to grant one, the rate for this respondent moved into the $15 
 bracket, where it remained. It was possible for this to happen on any 
 wave. 
 
 These arrangements were made clear to the interviewers at the train- 
 ing sessions and were contained in the "Interviewer Agreement" (Appen- 
 dix B), which each interviewer was required to read and sign. Inter- 
 viewers were also told that payment would be made only after interviews 
 had been completed and the questionnaires had been edited. If a ques- 
 tionnaire was not completed or if it contained many blanks, it was sent 
 back for further work, and the payment of the bill for that interview was 
 deferred. 
 
 From Table 8 it is apparent that the rates established for Study M3 
 were well below the average of Studies Ml and M2. However, the earn- 
 ings of some of the more effective interviewers in Studies Ml and M2 
 were consistently below the rates established for Study M3. As a result, it 
 was felt that the $10 and $13 rates were realistic. On Wave 1 of Study 
 M3, the interviewers' reactions to these rates were closely watched. No 
 apparent loss in morale or drop in quality of interviewing was observed. 
 If the rates had proved to be detrimental, they could have been raised to 
 the necessary levels. However, no such problems occurred. In fact, the 
 interviewers seemed enthusiastic about how much they were earning. 
 Much of the wasted time and motion had been eliminated whereas high 
 response and pick-up rates were being maintained. 
 
 These rates covered all time and expenses required to obtain an inter- 
 view, with the exception that interviewers were reimbursed at the rate of 
 
Interviewer Supervision 
 
 85 
 
 Table 8. Average Cost per Interview on All Waves of All Studies 
 
 (Dollars) 
 
 Wave 
 
 Study 
 Ml 
 
 Study 
 M2 a 
 
 Study 
 M3 
 
 Study 
 
 M4 
 
 1 
 
 2 
 
 3 
 
 4 
 
 5 
 
 18. 87 1 ' 
 
 18.46 
 
 18.23 
 
 18.19 
 
 19.02 
 
 23.11 
 20.61 
 15.42 
 14.41 
 10.03 
 
 16.87 
 13.62 
 13.23 
 12.58 
 13.68 
 
 14.79 
 13.15 
 14.35 
 12.49 
 13.92 
 
 
 18.55 
 
 16.72 
 
 14.00 
 
 13 74 
 
 
 
 a The large decrease in the cost per interview of $13.08 from Wave 1 to Wave 5 was due 
 mainly to the improvement of field procedures and control. This was the first study where the 
 field was under the direct supervision of project personnel. 
 
 b This cost figure is relatively low because this panel operation had been preceded by a 
 so-called ''background interview" with the same panel members about six months earlier, so that 
 much of the original contact work was out of the way by the first wave. 
 
 8 cents per mile for panel members living more than 10 miles from 
 their homes; the rate applied only to mileage over 10 in any direction. 
 Interviewers were also paid for training and study time at the rate of 
 $1.50 per hour. 
 
 The written instructions stated that bills would be processed every 
 three weeks. On those dates, all bills being held were sent to the Univer- 
 sity Business Office for payment. Interviewers were told to call the office 
 if they had a check coming which had not arrived in a week or 10 days 
 after the payment dates. 
 
 The method of payment by interview was also used on Study M4, 
 with two modifications. During each wave the rate per interview was $10 
 for each of the first 10 interviews and $13 for each interview over 10. 
 Once the $15 rate was established for reluctant panel members, it re- 
 mained constant on all succeeding waves. The $13 rate after 10 inter- 
 views had the effect of speeding up the field operation, since interviewers 
 were encouraged to reach the higher rate. The assigning of additional 
 interviews, however, was at the discretion of the field director. There- 
 fore, if an interviewer was doing work that was hurried and of question- 
 able quality, he was not given more assignments. 
 
 A second change was that Study M4 interviewers were paid for excess 
 mileage (over 10 miles) on all assignments worked, regardless of whether 
 the outcome was an interview, a refusal, or noncontact. The purpose of 
 this was to cover some of the interviewer's expenses incurred in attempts 
 to contact panel members living far from the interviewer's home. Reim- 
 bursement for excess mileage on this basis seemed proper because farm 
 respondents might be spread over a wide area. 
 
 All things considered, the procedure of paying by the completed 
 
86 Survey Reliability and Interviewer Competence 
 
 interview seemed to be much superior to that of paying by the hour. 40 In 
 large measure, this was undoubtedly due to the professional background 
 of most of the interviewers and the repeated emphasis placed on the nec- 
 essity of obtaining complete and accurate information. As a result, there 
 was little evidence that interviewers were rushing respondents to shorten 
 the interview. At the same time, there was some incentive for inter- 
 viewers not to prolong the interviews if they were not obtaining relevant 
 information, because they were not then receiving any additional pay. 
 
 In terms of cost per interview, compensation by the interview was 
 more economical than compensation by the hour, as is shown by the 
 schedule of average cost per interview for each wave on each of the four 
 studies in Table 8. 
 
 These data cover all costs connected with the interviews, including 
 compensation for meetings, mileage expenses, and trips made by project 
 staff members to the field. As is evident from this tabulation, the average 
 cost per interview in the last two studies was appreciably below that in 
 the first two. The cost shown for Study Ml is not directly comparable 
 with the costs shown for the other studies because it was subcontracted. 
 Nevertheless, this tabulation does suggest that payment by the interview, 
 even at very respectable rates, is likely to be more economical than pay- 
 ment by the hour. 
 
 Recommendations for the Future 
 
 Continuous contact would appear to be the key to adequate inter- 
 viewer supervision. If feasible, communication should be handled by a 
 field director who is thoroughly conversant with the purpose and objec- 
 tives of the study and with the types of data that are being sought. In 
 large-scale studies this is not possible and field supervisors must be used. 
 In such cases, the principle of continuous contact should still be main- 
 tained with the lines of communication going from the field director to 
 the supervisors, and from the supervisors to the interviewers. • This also 
 indicates the need for extensive and intensive training and involvement 
 of supervisors if they are to perform the functions of the field directors 
 for smaller studies. 
 
 Particularly important to the maintenance of the quality of data de- 
 sired are strict controls at the very beginning of an operation. During 
 the training sessions interviewers should be told of the planned use of such 
 controls, and these controls should be put into effect as soon as the field 
 operation begins. If interviewers are given to understand at the very 
 
 40 For a thorough analysis and comparison of these two methods of interviewer 
 compensation, see Mathew Hauck, "Interviewer Compensation on Consumer Sur- 
 veys." Commentary, The Journal of the Market Research Society, No. 14 (Sum- 
 mer, 1964), pp. 15-18. 
 
Interviewer Supervision 87 
 
 beginning that incomplete or careless work will not be accepted and that 
 they will be remunerated only if the interviews do meet certain standards, 
 and if they see that these principles are enforced at the outset, they will 
 be strongly motivated to carry out their assignments in a thorough manner. 
 
 In addition, with the maintenance of strict controls on the quality of 
 interviewing, payment by the interview seems to be a much more efficient 
 approach than payment by the hour. This is particularly true if profes- 
 sional people are hired as interviewers, for this method of compensation 
 serves to place extra responsibility on the interviewers, while at the same 
 time offering more rewards. 
 
 An important function of proper supervision is to avoid overloading 
 interviewers with assignments. It would seem best to give each interviewer 
 only a few assignments at a time, with a deadline date for these assign- 
 ments. After all of these assignments have been turned in, a new group of 
 assignments may be sent out together with a new deadline date. In this 
 manner, tight control can be maintained over the flow of interviews. 
 Also, this procedure serves to counteract the tendency of many inter- 
 viewers to put off contacting sample members whom they feel may be 
 difficult to interview. It is not unlikely that at times this very habit of 
 tardiness makes a sample member less cooperative, especially when an 
 advance letter had been received many days or weeks earlier. 
 
 If resources permit, a refresher training session after the first few days 
 of interviewing, or after the first five or six interviews, is a very useful 
 device for discovering common problems and for clearing up questions or 
 difficulties that the interviewers may have encountered in the field. Such 
 a meeting can also prove very useful for promoting data standardization. 
 
 The practice of having the interviewers edit their own questionnaires, 
 although it is no substitute for office editing, appears to be of real value 
 in reducing omissions and "don't know" answers. This is particularly true 
 if the interviewers know that unsatisfactory questionnaires will be re- 
 turned to them for additional (noncompensated) work. 
 
 Spot-check post cards and the switching of interviewers' assignments 
 can be useful in reducing interviewer cheating, particularly when inter- 
 viewers are told at the initial training sessions of their planned use. 
 
 Finally, a supervisor must evaluate the effectiveness of each inter- 
 viewer. When interviewers are not meeting project standards of complete- 
 ness and accuracy, they can be dismissed or retrained. 
 
 Thus many different aspects of interviewing performance must be 
 used to obtain a reliable indicator of the completeness and accuracy of the 
 data. The results presented in this chapter suggest that it is possible to 
 develop better means of identifying effective interviewers on the basis of 
 their performance. 
 
APPENDIXES 
 
APPENDIX A 
 
 INTERVIEWER QUALIFYING EXAMINATION 
 Study of Farm Family Finances 
 
 1. In your own words, please explain the objective(s) of this study, i.e., what 
 we hope to gain from it. 
 
 2, Suppose a farmer does not believe it is necessary for him to be interviewed. 
 Which of the following explanations would be appropriate? 
 
 a. The selection was done in a purely random manner. 
 
 b. The refusal of even one respondent lowers the amount 
 of information we have available. 
 
 c. We cannot substitute names. 
 
 d. None of the above. 
 
 3. Currently there are several reliable sources of family savings data, and 
 this project aims to develop another source to be used as a check. T F 
 
 h. In each of the following situations, briefly explain how you would overcome 
 the objections. 
 
 a. A small acreage farmer cannot understand why his name was chosen and not 
 the people down the road. 
 
 b. A wealthy farmer cannot understand why his name was chosen. Besides, he 
 is too busy. 
 
 91 
 
92 Survey Reliability and Interviewer Competence 
 
 A sample member states he does not have much savings and therefore does 
 not have to be interviewed. 
 
 d. The farm operator and his wife are both at home. They refuse to be in- 
 terviewed because there have been several "surveyors" around recently whc 
 have turned out to be salesmen. 
 
 The farm house is run down and poor looking. The farmer refuses to be 
 interviewed because he never did like universities and refuses to have 
 anything to do with them. 
 
 A middle-aged farmer refuses because he believes that the interviewer is 
 just checking on his income tax report. 
 
 A sample member, while being interviewed, refuses to go any further 
 because the questions are too personal. 
 
Appendix A 93 
 
 h. A harried-looking housewife says, "I'm busy. 
 
 5. Which, if any, of the following is the recommended approach in contacting 
 the farmer? 
 
 a. Go to the respondent's house and attempt either to get an 
 
 immediate interview or a firm appointment. 
 
 b. Telephone the respondent and make an appointment to see him. 
 
 c. Go to the respondent's home, fill in the classification data 
 and make an appointment to go back when all the adult family 
 members can be present. 
 
 d. Go to the respondent's house and interview them if the entire 
 adult family is present; otherwise, make an appointment to 
 return when they are all present. 
 
 e. None of the above. 
 
 6. (True or False) The Inter-University Committee for Research on Consumer 
 Behavior is a branch of the Ford Foundation and was organized to conduct 
 this study. T F 
 
 7. In your own words explain why the farm study is being done. 
 
 During the interview the respondent digresses and goes into a long descrip- 
 tion of last year's poor production. You should: 
 
 a. Change the subject quickly back to the interview. 
 
 Say, "Yes, that's very interesting, but I know your time is 
 precious so to get back to the interview..." 
 
 Listen attentively and then ask whatever financial questions 
 may be pertinent to the subject. 
 
 None of the above. 
 
94 Survey Reliability and Interviewer Competence 
 
 9. Briefly explain why records are important and why every attempt should be 
 made to get the farmer to refer to them at all times. 
 
 10. If a respondent tries to avoid being interviewed, which of the following, if 
 any, is (are) the wrong method(s) of proceeding? 
 
 a. Put on a "hard sell." 
 
 b. Say you will call again in two weeks. 
 
 c. Make it clear that you are willing to come at any time that 
 
 is convenient. 
 
 d. Always try to forestall an outright refusal. 
 
 e. None of the above. 
 
 11. Two different experiments are being used on the first wave. List three 
 
 experiments and briefly tell what you must know before attempting to conduct 
 an interview. 
 
 12. What would you say to a farmer if he said this is his busy time and he has 
 no time to stop and talk ? 
 
 13. If a farmer does not comprehend the importance of the study, which of the 
 following, if any, is (are) the correct way(s) to continue? 
 
 a. Explain the fact that this study will bring to light 
 
 important figures on the savings of the farm population. 
 
 b. Restate the fact that their name will not be used. 
 
Appendix A 95 
 
 c. Stress the importance of each farmer in the sample and how he 
 cannot he replaced. 
 
 d« Ignore the questions of the farmer and go on with the inter- 
 view. 
 
 e. None of the ahove. 
 
 Ik. The respondent says he doesn't know how much money he has in his checking 
 account, and it develops that he doesn't know offhand, hut his checkhook 
 is lying on top of the dresser upstairs. You should: 
 
 a. Ask him to give you an estimate. 
 
 h. Tell him that you will call hack later for the information. 
 
 c. Chide him for not heing more cooperative, and try to shame 
 him into looking at his records. 
 
 d. Re-emphasize that the study is of nationwide importance, and 
 that we are seeking complete and accurate information. 
 
 e. None of the ahove. 
 
 15. (True or False) Under no circumstances should the panel nature of the study 
 "be mentioned until the interview is completed. T F 
 
 16. Within a given savings unit we want to interview: 
 a. The oldest person in the savings unit. 
 
 h. The person in the savings unit who has the most money. 
 
 c. The person who makes the decisions on saving and horrowing. 
 
 d. The person who keeps the records on spending and saving. 
 
 e. The person who spends the most money. 
 
 f . None of the ahove. 
 
 17. In connection with Card D (Dollar Brackets for Broad Asset Holdings), we are 
 interested in ohtaining: 
 
 a. Only the holdings of the farm operator. 
 
 h. An exact dollar amount for each holding from his records. 
 
 c. Only in the holdings that the savings unit has. Leave other 
 holdings hlank. 
 
 d. None of the ahove. 
 
96 Survey Reliability and Interviewer Competence 
 
 18. The farm operator is married and has 3 daughters, ages 18, 7 and h. He in- 
 dicates that he has only the farm, a checking account, and some government 
 bonds, and his wife owns some stock. What is the next thing you would 
 probe for? 
 
 19. (True or False) After you find out with Card D what assets the savings 
 unit owns, you will not have to ask about the ones they do not own when you 
 get to detailed asset holdings. T F 
 
 20. (True or False) In conducting an interview, always cover all questions in 
 the order in which they are listed. T F 
 
 The following statements 21-^8 are definitions of some of the terms that you 
 should be familiar with for this study. If the statement is True put a T in the 
 space provided. If the statement is False place an F in the space provided and 
 correct the statement . The question will be counted wrong if it is marked False 
 and not corrected or if the correction is in error. 
 
 21. A mortgage bond is held by an individual or a concern. 
 
 22. Dividends paid on common stock are not fixed and usually depend on the 
 profits of the firm during the previous year. 
 
 23. For recording purposes, we want to separate cash from stock holdings in 
 a brokerage account. 
 
 2k. Preferred stock pay varying dividends just like common stock. It is 
 
 "preferred" because its holder has prior claim against the assets of the 
 firm in case of liquidation. 
 
 25. Public authority bonds are issued by a federal authority. 
 
 26. A contract differs from a mortgage in that the purchaser receives title to 
 the real estate after a specified amount is paid off. 
 
 27. Any interest accruing to a Postal Savings account is added to that account 
 and can be withdrawn when the certificate is cashed. 
 
 28. A dwelling unit is a group of rooms with kitchen facilities. Thus, a 
 dwelling unit may be a house, apartment, trailer, etc. 
 
 29. Mortgage insurance is a form of endowment insurance. 
 
 30. An annuity contract provides for the payment of a sum of money in return for 
 which an individual is guaranteed this sum of money plus a stipulated 
 interest in a lump sum at some future period. 
 
 31. Endowment insurance provides a definite sum of money only to the policy 
 holder's beneficiary after his death. 
 
Appendix A 97 
 
 32. Under straight life insurance, premiums may be payable until death, or 
 premiums may be payable for a specified number of years . 
 
 33. The farm operator is the person earning the largest income in the savings 
 unit. In case of equal earnings, it is the older male in the savings 
 unit. 
 
 3*+. The denomination of United States Government Bonds is the amount re- 
 ceivable when the bond matures. 
 
 35. Series E Bonds may be cashed in only after 10 years if you want to get any 
 interest. If the bond is cashed before 10 years, the holder only gets back 
 what he paid originally. 
 
 36. Savings and loan associations have two kinds of savings accounts. They are 
 (l) savings share accounts, where the interest is mailed to the owner of 
 the account, and (2) investment share accounts, where the interest is added 
 to the account. 
 
 37. Group life insurance is a form of term life insurance. The members of the 
 group usually hold certificates as evidence of their insurance. 
 
 38. Interest on Series H Bonds, United States Treasury Bills, and United States 
 Treasury Notes is sent periodically to the holder. 
 
 39- If "the holder of a savings life insurance policy dies, his beneficiaries 
 receive an amount equal to the balance in the account. 
 
 *+0. A savings unit consists of one or more related persons living in the same 
 dwelling unit who pool half or more of their income and savings. 
 
 hi. A mutual fund sells its stock or shares to investors and then invests the 
 proceeds in stocks and bonds of other corporations. Dividends are paid 
 directly from these corporations to the mutual fund holders just as they 
 pay stockholders direct. 
 
 *+2. An investment club is a club to which a group of people belong into which 
 each member puts a certain sum of money per week, month, etc. The club 
 splits profits and dividends. 
 
 J+3- The face value of a life insurance policy is the amount stated on the front 
 of each policy that will be paid in case of death or at maturity of 
 policy . 
 
 kk. A paid-up life policy is one for which all the premiums have been paid and 
 therefore the policy is no longer in force . 
 
 45. A checking account may be held in either a bank or a savings and loan 
 association. 
 
 U6. Debenture bonds are secured by specific assets of the firm. 
 
 kj. If a savings unit owes money to a credit union, it should be recorded as 
 an asset. 
 
98 Survey Reliability and Interviewer Competence 
 
 48. If a farm operator does not know the value of his machinery, leave the 
 question blank. 
 
 h$, -to your own words, how would you explain to a farmer how he happened to he 
 picked for this study? 
 
 50. (True or False) For savings accounts (holdings and change forms) we want 
 to know, among other things, the date of the last deposit and the date of 
 the last withdrawal. T F 
 
 51. (True or False) Expected total income for i960 is that of the farm 
 operator only. T F 
 
 52. (True or False) We want to know the name of a mortgage holder only if the 
 mortgage is cured to an institution. If a mortgage is held by an individual 
 the name is not necessary — merely "mother-in-law," "uncle," etc., will be 
 sufficient . T F 
 
 53. (True or False) Carrying charges are not to be included in the present 
 amount owed on a debt. T F 
 
 54. In order to keep track of items you may want to recall for the next inter- 
 view, you should: 
 
 . a. Write them under "interviewer's Notes" on the IRF. 
 
 b. Not write them under "interviewer's Notes," because the 
 
 information in this space is to be tabulated. 
 
 c. Not write them under "interviewer's Notes," because the 
 
 IEF will not be returned to you on the next wave. 
 
 d. Keep a separate file of notes on your panel members. 
 
 55. (True or False) A member of the family who is either looking for work or 
 has been temporarily laid off is considered employed. T F 
 
 56. The respondent has a savings account, but knows of another institution 
 where he can get a higher rate. He plans to open up a new account 
 there. How would you record this? 
 
 57. (True or False) The total investment in an annuity is the amount of the 
 down payment. T F 
 
Appendix A 99 
 
 58. (True or False) Only interview sample members who are presently engaged in 
 farming . T F 
 
 59. In your own words, describe the step-by-step procedure from the time you 
 receive your assignments until the completed interview is mailed in: 
 
 60. Explain briefly the meaning of a personal trust. 
 
 61. If a stock does not sound as if it is on a "Listed Exchange," you should: 
 a. Look it up in the paper. 
 
 ______ b. Not record it. 
 
 c. Get the price from the respondent. 
 
 d. Write "not listed" instead of the price per share. 
 
 e. None of the above. 
 
 62. If you find out that & family is away on vacation for the next couple of 
 months, what parts of the Interviewer Report Form do you fill out? 
 
 63. (True or False) The dollar amount the saving unit expects to receive in 
 i960 from all sources of income is to be totaled. T F 
 
100 Survey Reliability and Interviewer Competence 
 
 6k. The farm operator owns five Series E Bonds, two of which axe $50 denomina- 
 tions, one of which is $100, and two of which are $25; he also owns three 
 $100 Treasury Certificates. Show how you would list these "bonds. 
 
 65. (True or False) With regard to insurance premiums, we are interested in 
 what is paid and how often. T F 
 
 66. A checking account had a balance of $1538.25 on July 1, i960, and at the 
 time of the interview has a balance $1^23.85. This would mean the balance 
 on the holding form is: 
 
 a. Up (U) by $lll+.»40. 
 
 b. The same (s). 
 
 c. Down (D) by $11^.^0. 
 
 d. None of the above. 
 
 67. If a family refuses to be interviewed, what parts of the IRF do you fill 
 out? 
 
 68. What does "Atyr" mean, and why is it used? 
 
 69. Explain the differences between accuracy and completeness. 
 
 70. (True or False) The Classification Data (Part 2 of the IRF) can be left 
 blank if an interview is obtained because all of the requested information 
 will be recorded on the other forms. T F 
 
 71. (True or False) Someone in the savings unit has two insurance policies, 
 one with Mutual of Omaha and the other with Mutual of New York. It will 
 be sufficient for our purposes if you record them both as "Mutual." T F 
 
Appendix A 101 
 
 72. Suppose you have to visit a family three times before you obtain an inter- 
 view. The first time, August 23 at 7:00 p.m., no one is at home. The 
 second time, August 25 at U:00 p.m., the farm operator says he is too busy 
 to be interviewed and you make an appointment for August 27 at 10:00 a.m. 
 The third time, August 27 at 10:00 a.m., you obtain the interview from the 
 farm operator and his wife. How do you record this information under 
 "Contact Report" in the IRF? 
 
 Approx. 
 Date Day Time Where Persons(s) talked to age Results of Contact 
 
 73« Briefly tell what we want to know about life insurance on the C Form with 
 
 regard to: Where did the money come from? or What was done with the money? 
 
 7h. The respondent has bought a new insurance policy since July 1. Among other 
 
 things on the change form, we want to know: 
 
 a. Amount received. 
 
 b. Amount that has been piad on the policy since July 1. 
 
 c. Amount the policy is worth with a double indemnity clause. 
 
 d. Face value of the policy. 
 
 e. None of the above. 
 
 75 • In the change approach if a checking account has been opened since July 1 
 we want to know the source of the money. If a checking account has been 
 closed since July 1, we want to know the use of the money. If there has 
 been a change in balance we want to know the source. T F 
 
 76. Consider the following questions as referring to the following family 
 
 situation. The farm operator is a man in his late forties. He operates 210 
 acres. He has a /wife and three children: daughter (19) a college junior, 
 son (17) who is just finishing high school and helps the operator on the 
 farm, and daughter (l 1 *) who just finished her first year of high school. 
 The family lives in a neatly kept nine-room farm house. 
 
 a. On broad asset holdings, the farm operator replies that he and his 
 wife have a checking account. When you are obtaining detailed infor- 
 mation, the farm operator gives you the data on the account. Before 
 going on, and in view of the family composition, what should you query 
 about? 
 
102 Survey Reliability and Interviewer Competence 
 
 h. The farm operator replies that he has no savings accounts and has all 
 his investment tied up in the farm. In view of the family composition., 
 what would he the next question? 
 
 c. List four types of life insurance that the family might own. 
 
 d. The wife says that she has some life insurance that her parents had 
 
 taken out for her hut which she had all hut forgotten. She is not sure 
 that it is still in effect. She says that it is just a nominal amount 
 and is very vague ahout it all. Briefly, what would you do? 
 
 The wife states that she is part owner (with her mother) of some savings 
 honds. She knows the number and denomination. Briefly, what would you 
 do? 
 
 f. If only the wife is mentioned as owning savings honds, what would this 
 prompt you to do? 
 
 g. The farm operator has to leave the interview situation and the wife 
 says she will answer the questions. After a few questions, it appears 
 she does not have the answers. What would you do? 
 
 Date Nam e 
 
 Address 
 
Appendix A 103 
 
 APPENDIX B 
 
 INTERVIEWER AGREEMENT 
 Study of Farm Family Finances 
 
 Background and purpose 
 
 The study in which you have been engaged as an interviewer has as its objective 
 the collection of accurate, up-to-date information on farm savings practices and 
 attitudes. You have undertaken to cooperate in the study and assist in the 
 performance of a very necessary part of it by utilizing your particular skills 
 in an area of interviewing which demands independence of control and an objec- 
 tivity at the interview level. The success of this study will be dependent upon 
 the performance of those skills and their utilization by you; therefore, you 
 will be given a great measure of freedom in the selection of time and method. 
 Tne major. concern of the study will be the results achieved by you, the inter- 
 viewer . 
 
 General arrangements 
 
 You will have submitted to you the names and addresses of approximately fifteen 
 farmers to be interviewed in your interview area. An attempt will be made to 
 restrict interviews to a 10-mile radius of your home. If a sample member re- 
 sides outside the interview area, you will be reimbursed at the rate of 5 cents 
 per mile for every mile beyond the 10-mile radius each way necessary to contact 
 the sample member. 
 
 You will be paid on a contract basis by the completed interview. The pay 
 schedule for completed interviews is as follows: 
 
 a. For each of the first ten completed interviews on each wave, $10.00 
 per interview. 
 
 b. For all completed interviews over ten on each wave, $13.00 per 
 interview. 
 
 c. For interviews on which a previous interviewer has received a refusal, 
 $15.00 per interview. In addition, $15.00 will be paid for interviews 
 with this panel member on each successive wave. 
 
 As an interviewer for this study, you are functioning as an independent contractor, 
 not as an employee of the University of Illinois. Contract payments will be 
 made to you on receipt of bills without any withholding for income tax, retire- 
 ment, insurance, Social Security, or any like or dissimilar deductions. 
 
 It may be necessary from time to time to meet to discuss the review procedures 
 and developments relating to the study. Ordinarily these meetings will take 
 the form of a general briefing session at the outset of each wave of interviews, 
 a review meeting at about the middle of each wave, and a final meeting at the 
 end of each wave of interviews. The meetings will be set at a time and place 
 most convenient to all interviewers. You will receive the sum of $1.65 per hour 
 for attending these meetings. No travel time or mileage payments will be made 
 for attending these meetings . 
 
104 Survey Reliability and Interviewer Competence 
 
 Interviewing 
 
 To give substance to this study and insure that your skills are advantageously 
 utilized, here are a few general guides: 
 
 a. Interviewers are expected to complete about 12 interviews on each wave. 
 In addition, interviewers are expected to attempt to make contact at the 
 designated address within four days after receiving the names and 
 addresses of the interviewees. 
 
 b. To produce the best results, all interviews should be conducted in the 
 home of the sample member, or any other place conducive to privacy and 
 convenient to the sample member. Interviews should not be conducted in 
 a public place nor while neighbors and/ or relatives are present. 
 
 c. A completed interview should cover all questions sought by this study 
 and should contain no unexplained blank spaces. 
 
 d. Completed interviews should be transmitted by mail no later than 9 
 o'clock in the morning of the day following that in which they have 
 been completed. 
 
 e. Information obtained from a sample member must not be discussed with 
 anyone other than project personnnel. 
 
 f . It is essential that interviewers be available to interview on a part- 
 time basis, principally evenings and weekends, for a period of one 
 year. 
 
 I have read and understand the foregoing material included in this Interviewer 
 Agreement . 
 
 (signed) 
 
 Date 
 
Appendix B 
 
 105 
 
106 
 
 Survey Reliability and Interviewer Competence 
 
 ATTITUDE OF PANEL MEMBER 
 
 a. In your opinion, how accurate and how complete is the information that 
 was given for each of the following holdings. 
 
 y Fully so.... 
 
 g Fairly so . . . 
 
 < Not much.. .. 
 
 Very little. 
 
 Checking 
 
 Savings 
 
 Life 
 
 Govern- 
 
 Own 
 
 
 
 accounts 
 
 accounts 
 
 insur- 
 
 ment 
 
 busi- 
 
 
 
 
 
 ance 
 
 bonds 
 □ 
 
 ness 
 □ 
 
 Stock 
 
 D 
 
 Debts 
 
 D 
 
 D 
 
 D 
 
 □ 
 
 D 
 
 D 
 
 D 
 
 □ 
 
 D 
 
 D 
 
 a 
 
 D 
 
 D 
 
 D 
 
 □ 
 
 □ 
 
 D 
 
 a 
 
 D 
 
 □ 
 
 D 
 
 D 
 
 D 
 
 □ 
 
 a 
 
 D 
 
 □ 
 
 □ 
 
 □ 
 
 D 
 
 D 
 
 □ 
 
 □ 
 
 D 
 
 D 
 
 □ 
 
 D 
 
 □ 
 
 □ 
 
 D 
 
 □ 
 
 D 
 
 D 
 
 D 
 
 □ 
 
 D 
 
 D 
 
 D 
 
 D 
 
 D 
 
 □ 
 
 D 
 
 D 
 
 Fully so.... 
 Fairly so... 
 Not much. ... 
 Very little. 
 
 Not applicable. 
 
 □ 
 
 □ 
 
 D 
 
 □ 
 
 □ 
 
 D 
 
 D 
 
 b. What was respondent's attitude toward the use of records in supplying the 
 figures requested? 
 
 Did you have the impression at any time that the respondent did not know 
 the figures too well which he gave you? nYes Q No Q Not sure Explain 
 
 d. Did you have the impression at any time that any figures were deliberately 
 being "doctored"? Q Yes q Nq q Hofc __,, 3-^ 
 
 e. Did you have the impression at any time that any figures were being withheld 
 or suppressed? Dyes Q No DNat suxe Exp i ain _____ 
 
 f . Did you leave an interview evaluation form with the panel member? 
 
 □ Yes []No DNot applicable 
 What was the respondent's reaction? 
 
Appendix C 107 
 
 k. SUMMARY COMMENTS 
 
 Briefly summarize the respondent's over-all reaction to the interview. Please 
 include any information which you feel may he helpful in understanding and 
 analyzing the data of this panel member. 
 
 5. TNTERVTEWSR f S NOTES 
 
 This IRF will he returned to you on the next wave so use this space to record 
 any information you think you may need. Following are suggestions. 
 
 Future contact arrangements 
 
 Who 
 
 When 
 
 Where_ 
 How 
 
 Any information which will make your interview on the next wave easier, touchy 
 data areas or subjects, hobbies or other talking points, personal likes or 
 dislikes, etc. 
 
108 
 
 Survey Reliability and Interviewer Competence 
 
 tf zi 
 
 Rt 
 
 a 
 
 
 d 
 
 CD 
 
 X 
 
 
 l-H 
 
 ^1 
 
 d 
 
 w 
 
 
 
 ffl 
 
 nt 
 
 
 
 •M 
 
 > 
 
 
 
 m 
 
 
 
 
 
 bo 
 
 u 
 
 
 
 X< 
 
 a, 
 
 
 
 d 
 
 a, 
 
 
 
 « <j 
 
 
 
 
 
 «T 
 
 
 10 
 
 rrt 
 
 u 
 
 
 W 
 
 ^ 
 
 • r-l 
 
 
 u 
 
 d 
 
 M 
 
 
 'A 
 
 
 (1) 
 
 
 < 
 
 
 CO 
 
 
 K 
 
 
 Ml 
 
 
 l—i 
 
 
 
 
 [t, 
 
 M 
 
 
 
 
 CO 
 
 CD 
 
 
 
 
 rrt 
 
 <u 
 
 z 
 
 n 
 
 2 
 
 d 
 
 +-» 
 
 d 
 
 <4 
 
 ■i-l 
 
 d 
 
 3 
 
 d 
 
 u 
 
 < 
 
 o 
 
 'd 
 
 o 
 
 In 
 
 00 
 
 < 
 
 
 <o 
 a 
 
 u 
 
 00 
 
 o 
 
 o 
 
 o 
 
 <! 
 
 4J 
 
 d 
 
 P 
 
 o 
 d 
 
 .d 
 
 s 
 
 13 
 
 ■ H 
 
 T) 
 
 !-l 
 
 H 
 
 to 
 
 C 
 
 nt 
 
 CO 
 
 O 
 
 nt 
 
 Pi 
 
 .. 
 
 d 
 
 a) 
 
 P 
 
 H 
 
 JZJ 
 
 
 
 3 
 
 l-H 
 
 h 
 
 CO 
 
 £ 
 
 O 
 >> 
 
 CO 
 
 D 
 
 z 
 
 
 
 
 o 
 
 CO 
 
 h 
 
 
 M 
 
 
 
 
 H 
 
 CD 
 
 V 
 
 
 in 
 
 
 ai 
 
 
 W 
 
 e 
 
 CD 
 
 
 P 
 
 & 
 
 ri 
 
 
 a 
 
 
 
 
 £ 
 
 ^ H 
 
 H 
 
 H 
 
 S fi 
 
 P 
 
 co" 
 
 « en 
 
 d _ 
 
 -d ii 
 
 I 5 
 
 m £ 
 
 <J £ < 
 
 <U M O 
 CO ti CO 
 
 ■g .2 
 
 ^ s 
 
 h £ 
 
 £ ° 
 
 O _d 
 
 u 
 
 - 4) 
 
 O <0 % 
 
 tn S u 
 
 u c$ m 
 
 rt d o 
 
 ^ h d 
 
 ^ > o 
 
 1*1 
 
 3 d ^ 
 
 o o ■£ 
 n o o 
 
 xi ^o <& 
 
 ° J 
 
 ^ -d ^ 
 
 d +* X 
 
 £ «> S 
 >» en & 
 
 - «> O 
 co d 
 
 •r< O S-i 
 
 3 I, 
 
 CD 
 
 o « 
 
 r^ 
 
 
 O 
 
 ■*" 
 
 
 
 d O 
 
 
 O .d 
 
 u 
 
 o> * 
 
 <u 
 
 u "1 
 
 > 
 o 
 u 
 
 ■S b -d 
 
 ^ Z. 
 
 « o 
 
 
 ^ sD 
 
 
 ■w- 
 
 f 
 
 
 
 >n d 
 
 u 
 
 T3 rt 
 
 d 
 
 3 "i 
 
 m 
 
 CD 
 
 CO 
 
 a 
 
 d 
 nt 
 
 6 
 F 
 
 
 ■rH 
 
 DO 
 CO 
 
 n 
 
 >«. 
 
 (U 
 
 
 
 « M 
 
 
 rd 
 
 
 d 
 
 >. 
 
 JS.S 
 
 u 
 
 o 
 
 o 
 
 > 
 
 ai 
 
 
 h 6 
 
 a 
 
 
 
 Pi 
 
 is 
 
 CO 
 
 fe 
 
 a 
 
 •* in vo 
 
Appendix D 
 
 109 
 
 w 
 
 H 
 < 
 H 
 w 
 W 
 
 < 
 
 w 
 3 
 
 Pi 
 w 
 
 X 
 H 
 
 o 
 
 Q 
 
 < 
 
 w 
 
 H 
 
 C ^S r-S 
 
 o 
 K 
 
 D 
 
 ■so 
 
 
 J3 >-l 
 
 so 
 
 ■5 a 
 
 
 •J O. 
 
 •S O 
 
 ?. * 
 
 2 S 
 
 * -5 
 
 
 h 
 
 Uh 
 
 o 
 
 _ 
 
 7J 
 
 <n 
 
 id 
 
 >> 
 
 1-1 
 
 s 
 
 >- 
 
 
 £• S 
 
 * J- 
 
 P-- 
 
 
 
 H 
 
 @ 
 
 d 
 
 o 
 
 
 
 
 13 
 
 
 
 
 XI 
 
 d 
 id 
 
 
 
 
 «J- -W- ■» 
 
 J ^ 
 
 ni 
 
 *«• 
 
 •w- 
 
 «*• 
 
 >» 
 
 
 
 
 ■B S 
 
 
 
 
 • H 
 
 ■w- 
 
 ■ea- 
 
 ■w- 
 
 2 * 
 
 2 j3 
 £ El 
 
 
 g|D □ □ 
 f □ O □ 
 
110 
 
 Survey Reliability and Interviewer Competence 
 
 3 "5 
 
 B g 
 
 2 £ 
 B Z 
 
 >• 
 
 ri 
 
 00 
 
 
 X! 
 
 h 
 
 
 £ 
 
 o 
 
 d 
 
 *— * 
 
 >> 
 
 % 
 
 (0 
 
 CD 
 
 o 
 
 >- 
 
 •H 
 
 >> 
 
 d 
 
 3. 
 
 O 
 
 a 
 
 <! 
 
 
 
 -a iX 
 
 C! ' 
 
 □ □□□□□□□□D 
 
 DDDnnnnnDD 
 DDnnnDDnnn 
 
 mm n n ! .] n !"] □ □ □ 
 
 a □ 
 
 innnnnnnnnn 
 
 »nnnDnnnn 
 
 £3 
 
 o <! 
 
 ° m _ 
 
 +J -H TJ 
 
 U) , fH 
 
 <u £ ni 
 
 > ° u 
 
 TJ u 
 
 a 
 
 4) £ 
 
 
Appendix D 
 
 111 
 
 2 3 
 
 nan 
 
 □ □ □ 
 d a d 
 
 □ D □ 
 
 
 < 
 
 TO 
 
 
 h 
 
 
 o 
 
 g 
 
 o 
 
 
 
 
 
 pd 
 
 >H 
 
 
 
 c~ 
 
 
 0) 
 
 4J 
 
 XI 
 
 
 
 nj 
 
 a 
 
 jC 
 
 rt 
 
 i3 
 
 d 
 
 u 
 
 (J 
 
 
 
 
 
 "3, .O 
 
 c 
 
 
 3 
 
 u 
 
 cu 
 
 
 "a 
 
 XI 
 
 £ 
 
 
 a 
 
 h 
 
 
 
 
 <u 
 
 V 
 
 
 
 _c 
 
 > 
 
 R) 
 
 :* 
 
 to 
 
 
 
 J 
 
 
 
 
 P 
 
 d 
 id 
 
 d 
 
 
 
 >- 
 
 2 
 
 o * 
 
 
 46 6 6 666 6 66 6 66 
 
 dddddddd nn nan 
 
 V) 
 
 
 
 
 
 «j 
 
 
 in 
 
 
 (D 
 
 d 
 
 o 
 
 
 
 B 
 
 M 
 
 d 
 
 ri 
 
 
 
 M 
 
 .^ 
 
 
 
 u 
 
 
 o 
 
 tH 
 
 s 
 
 
 
 .d 
 
 
 
 >• 
 
 
 
 ^3 
 
 u 
 
 XI 
 
 o 
 
 CD 
 
 
 h 
 
 
 a 
 
 CO 
 
 u 
 
 U 
 
 <a 
 
 a 
 
 
 h 
 
 
 
 CO 
 
 d 
 
 d 
 
 
 
 rt 
 
 u 
 
 
 
 
 ,3 
 
 
 & £ 
 
 § £ 
 
 DD □□ □□□ □ 
 
 □ □ DD DDD □ 
 
 DD DD DDD D 
 
 DD DD DDD D 
 
 T3 * 
 
 to. o 
 
 H O H 
 
 -1 <! 
 
 o < o 
 
 66 66 666 6 66 6 66 
 
 DDDDDDDD DD DDD 
 
 D D 
 
 D 
 
 D D 
 
 D D 
 
 D 
 
 D D 
 
 D D 
 
 D 
 
 D D 
 
 D D 
 
 D 
 
 D D 
 
 d a s 
 
 CO rt 
 
 o o 
 
112 Survey Reliability and Interviewer Competence 
 
 .3 
 
 2 cr 
 5 & 
 
 a. - I 
 
 •s sp 
 
 -2 ^ 
 
 •a - - 
 
 ^ 3 
 
 )H O _ 
 
 3 -3 oj 
 
 5 a « 
 
 i <u — 
 
 03 
 
 go" 
 
 •So 
 
 o 
 
 sis 
 
 4J 
 
 3 
 
 cr 
 
 hj O =3 
 
 \ « 9 ^ 
 
 « «; 3 
 
 o 
 
 U JZ* 
 
 
 _M 
 
 <u +■> 
 
 > U .3 ,-3 «J 
 
 S3 rS £ * £ 
 
 3 O &, 2 3 
 
 .3 3 X -3 <u 
 
 I- 1 4J « ^ 
 
 -a 
 
 «J O T3 
 
 t-i -3 -3 *■< 
 
 J _5 
 
 5 I 2 o K 
 
 ^• S s * «*S 
 
 3 
 
 CJ 3 3 g 8 *S 
 
 ,. «* ^ <u < § 
 
 % -a a $ " * 
 
 C w • J -' X 03 as "" 
 
 8 ?s § * ££* 
 
 g ^ ,0 W h^ S a3 
 
 js Sua c/5 g. B 
 
 . -3 £ ^ . -3 
 
 <H 1— I <-3 l-i HH I— I >