AN INVESTIGATION OF RESPONSE ERROR L I B R.AR.Y OF THE UN IVE.RSITY OF ILLINOIS In&2 s ESSSK 'So 8 " Library Materia,8, The ■— «- »* The person charging this material is responsible for ns return to the library from wh.ch it was^.thdrawn on or before the Latest Date stamped below. Theft, mutilation, and underlining of books are reason. #„, M . • ■■ To renew call Telephone Center, 333-8400 UN.VERS.TY OP , LL , N0 ,s LIBRARY AT URBANA , rHflM _ K , FEB FEB 2 Ol WW FEB 1 2 1992 vxc MW'OZ uic-REC'D flcriow L161— O-1096 Digitized by the Internet Archive in 2011 with funding from University of Illinois Urbana-Champaign http://www.archive.org/details/investigationofr02lans AN INVESTIGATION OF RESPONSE ERROR STUDIES IN CONSUMER SAVINGS No. 1 Collecting Financial Data by Consumer Panel Technique. Robert Ferber Studies in Consumer Savings, No. 2 CONSUMER SAVINGS PROJECT INTER-UNIVERSITY COMMITTEE FOR RESEARCH ON CONSUMER BEHAVIOR AN INVESTIGATION OF RESPONSE ERROR John B. Lansing Gerald P. Ginsburg Kaisa Braaten PUBLISHED BY THE BUREAU OF ECONOMIC AND BUSINESS RESEARCH ■ UNIVERSITY OF ILLINOIS • URBANA • JUNE 1961 © 1961 by the Board of Trustees of the University of Illinois 3 3)?- PREFACE This monograph is the second in a series of technical reports presenting findings of the Consumer Savings Project of the Inter- University Committee for Research on Consumer Behavior. It is based on a series of studies undertaken by the Survey Research Center of the University of Michigan and integrates into the findings some results of earlier exploratory investigations conducted by the Center. The monograph throws light on the magnitude and direction of response error that is encountered in consumer financial surveys relating to personal savings accounts, personal cash loans, and automobile debt. The Inter-University Committee is indebted to John B. Lansing for his many valuable contributions to these studies. He gave considerable time and effort to planning the studies and to analyzing the data. Except for the study described in Chapter II, the interview-and- verification experiments described in this monograph were planned and conducted as part of the Consumer Savings Project. This is an integrated project designed to: 1. Determine the reliability of survey techniques for collecting from consumers quantitative financial information, and develop new techniques of improved reliability. 2. Develop procedures for obtaining these data on a current and continuing basis with a known degree of reliability, with sufficient frequency and accuracy for the practical needs of policy-makers in government and business. 3. Begin to collect consumer financial data of a kind which have not hitherto been available but are of strategic importance in study- ing the decisions consumers make about their assets and debts, spending, and saving. The first monograph in this series, entitled "Collecting Financial Data by Consumer Panel Techniques: A Case Study," presented findings obtained in the course of the first panel operation undertaken as part of this project. Some of the field experiments described in the present monograph explore further hypotheses suggested during the course of this panel operation. The present results lead to a number of additional, important conclusions on the nature of response error in financial surveys and on possible means of counteracting these errors in future surveys. Other monographs in this series will present further information on the scope and nature of response errors in consumer financial surveys. In addition, they will focus on methods of utilizing these findings as a basis for developing improved approaches to the col- lection of data on consumer finances. It is anticipated that a summary volume will appear after comple- tion of the various field operations associated with the project. This project is financed by a grant from the Ford Foundation. Robert Ferber, research professor of economics at the University of Illinois, is director of the project. The members of the Inter-University Committee for Research on Consumer Behavior are: Lincoln Clark, New York University, Secretary-Treasurer Robert Ferber, University of Illinois Raymond Goldsmith, New York University George Katona, University of Michigan Theodore Newcomb, University of Michigan James Tobin, Yale University Guy Orcutt, University of Wisconsin, Chairman The monographs in this series are research reports. The Inter- University Committee, as sponsor of this research, makes every effort to ensure both the quality of the reports and their orientation toward meeting a real need. Nevertheless, the findings reported in this way summarize conclusions arrived at by project staff and do not necessarily represent the individual or collective views of the members of the Inter-University Committee. Guy Orcutt, Chairman Inter -University Committee for Research on Consumer Behavior VI ACKNOWLEDGMENTS The project could not have been carried out without the coopera- tion of several financial institutions. The authors are grateful for the interest in the project shown by representatives of these organiza- tions and for their active assistance in carrying out the work. The authors of this report further wish to acknowledge the in- valuable assistance of their colleagues at the Survey Research Center. The study was done in the Economic Behavior Program of the Center, George Katona, Director. The Center itself is directed by Angus Campbell; it is a division of the Institute for Social Re- search, which is directed by Rensis Likert. Dr. Edward S. Bordin, Department of Psychology, University of Michigan, served as a psychological consultant. Dr. Charles F. Cannell, Head of the Field Section, Survey Research Center, played an active role in the development of the field work. Miss Irene Hess, Assistant Head of Sampling Section, Survey Research Center, assumed responsibility for the work at the Center on the design of the required samples. Dr. James N. Morgan, Program Director, Survey Research Center, displayed an active interest in the project from its inception. Dr. Ralph Bristol, now of Rand Corporation, Santa Monica, California, directed much of the work on car debt in Chicago which is reported in Chapter IH. Dr. Frederick May, now of the University of Missouri, arranged the laboratory investigation described in Chapter IV. Dr. Mordechai E. Kreinin, now of Michigan State University, conducted the special check of data on car debt from the Survey of Consumer Finances reported in Chapter IH. This list could easily be extended: it is not possible to mention individually all of the members of the staff of the Survey Research Center who contributed to the work reported here. Without the active cooperation of these individuals this investigation could not have been carried out. vn It is anticipated that a summary volume will appear after comple- tion of the various field operations associated with the project. This project is financed by a grant from the Ford Foundation. Robert Ferber, research professor of economics at the University of Illinois, is director of the project. The members of the Inter-University Committee for Research on Consumer Behavior are: Lincoln Clark, New York University, Secretary-Treasurer Robert Ferber, University of Illinois Raymond Goldsmith, New York University George Katona, University of Michigan Theodore Newcomb, University of Michigan James Tobin, Yale University Guy Orcutt, University of Wisconsin, Chairman The monographs in this series are research reports. The Inter- University Committee, as sponsor of this research, makes every effort to ensure both the quality of the reports and their orientation toward meeting a real need. Nevertheless, the findings reported in this way summarize conclusions arrived at by project staff and do not necessarily represent the individual or collective views of the members of the Inter -University Committee. Guy Orcutt, Chairman Inter-University Committee for Research on Consumer Behavior VI ACKNOWLEDGMENTS The project could not have been carried out without the coopera- tion of several financial institutions. The authors are grateful for the interest in the project shown by representatives of these organiza- tions and for their active assistance in carrying out the work. The authors of this report further wish to acknowledge the in- valuable assistance of their colleagues at the Survey Research Center. The study was done in the Economic Behavior Program of the Center, George Katona, Director. The Center itself is directed by Angus Campbell; it is a division of the Institute for Social Re- search, which is directed by Rensis Likert. Dr. Edward S. Bordin, Department of Psychology, University of Michigan, served as a psychological consultant. Dr. Charles F. Cannell, Head of the Field Section, Survey Research Center, played an active role in the development of the field work. Miss Irene Hess, Assistant Head of Sampling Section, Survey Research Center, assumed responsibility for the work at the Center on the design of the required samples. Dr. James N. Morgan, Program Director, Survey Research Center, displayed an active interest in the project from its inception. Dr. Ralph Bristol, now of Rand Corporation, Santa Monica, California, directed much of the work on car debt in Chicago which is reported in Chapter III. Dr. Frederick May, now of the University of Missouri, arranged the laboratory investigation described in Chapter IV. Dr. Mordechai E. Kreinin, now of Michigan State University, conducted the special check of data on car debt from the Survey of Consumer Finances reported in Chapter III. This list could easily be extended: it is not possible to mention individually all of the members of the staff of the Survey Research Center who contributed to the work reported here. Without the active cooperation of these individuals this investigation could not have been carried out. vn CONTENTS PART ONE. Introduction I. A Statement of the Problem 3 PART TWO. Preliminary Studies II. Preliminary Study of Savings Accounts 11 III. Preliminary Studies of Car Debt 21 IV. Interviews in a Laboratory 43 PART THREE. Three Field Experiments V. Savings Accounts: The First Field Experiment 53 VI. Cash Loans: The Second Field Experiment 91 VII. Savings Accounts: The Third Field Experiment 118 VIII. The Mail Reinterviews 154 PART FOUR. Conclusions DC. Toward a Theory of Response Error 179 Appendixes A. Sampling Error 207 B. Questionnaires Used in Field Experiments 209 C. Results of Studies Using Split Samples on the Survey of Consumer Finances— by Charles Liniger 290 D. Supplementary Tables 297 ix LIST OF TABLES 1. Disposition of Selected Addresses in Preliminary Savings Study 12 2. Response Rate in Preliminary Savings Study 13 3. Estimates of Distribution of Savings Accounts and of Amounts in Savings Accounts in Preliminary Savings Study 14 4. Interview Report on Amount in Savings Account at Time of Interview Compared with Institution Report in Preliminary Savings Study 15 5. Comparison of Groups with Accurate Versus In- accurate Reports on Amounts in Savings Accounts at Time of Interview in Preliminary Savings Study 16 6. Interview Report on Change in Amount in Savings Accounts Compared with Institution Report in Preliminary Savings Study 18 7. Characteristics of Accurate Versus Inaccurate Respondents on Change in Savings Accounts in Preliminary Savings Study 19 8. Distribution of Interviews by Degree of Agreement Between Records and Response 28 9. Distribution of Interviews by Degree of Agreement Between Response and Recorded Loan Within Prob- able Purpose of Loan 30 10. Relation Between Debt According to State or County Records and Respondents' Reports (in dollars) in the 1957 .Chicago Interviews 32 11. Summary of Relation Between Debt According to State or County Records and Respondents' Reports in the 1957 Chicago Interviews 35 12. Interviewers' Impressions of Accuracy of Response by Degree of Agreement for Interviews Where Inter- viewer Did Not Know Details Only 36 13. Relation Between Debt According to State Records and Respondents' Reports (in dollars) in the 1956 Survey of Consumer Finances 39 LIST OF TABLES xi 14. Response Disposition in Relation to Size of Actual Balance in Account in Fall, 1958 58 15. Response Disposition in Fall, 1958, in Relation to Mean Actual Balance in Account in Fall, 1958 60 16. Response Disposition for First Interview (Fall, 1958) in Relation to the Experimental Manipulation 61 17. Response Disposition for Spring Re interview in Relation to Type of Interview Used in Fall 62 18. Response Error in Report of Fall, 1958, Balance in Relation to Experimental Manipulation 64 19. Actual Balances for Fall, 1958, Listed with Fall Reports (in dollars) in Relation to Response Error, with Means , 66 20. Response Error in Best Estimate from Interviews of Fall, 1958, Balance in Relation to Response Error in Fall Report 68 21. Response Error in Best Estimate from Interviews of Fall, 1958, Balance in Relation to Source of Best Estimate 71 22. Reported Number of Changes in Balances from Fall, 1958, to Spring, 1959, in Relation to Actual Changes in Balances 73 23. Response Error in Fall Report of Fall, 1958, Balance in Relation to Size of Actual Balance in the Account, with Means 74 24. Response Error in Best Estimate from Interviews of Fall, 1958, Balance in Relation to Type of Use of This Account 76 25. Response Error in Best Estimate from Interviews of Fall, 1958, Balance in Relation to Size of Change of Balance, Fall, 1958, to Spring, 1959 77 26. Response Error in Best Estimate Report of Fall, 1958, Balance in Relation to Income of Spending Unit 79 27. Response Error in Best Estimate Report of Fall, 1958, Balance in Relation to Education of Respondent 80 28. Response Error in Best Estimate from Both Interviews of Fall, 1958, Balance in Relation to Occupation of Head of Family 81 xii LIST OF TABLES 29. Response Error in Best Estimate Report of Fall, 1958, Balance in Relation to Age of Respondent 82 30. Response Error in Best Estimate Report of Fall, 1958, Balance in Relation to Sex of Respondent 84 31. Response Error in Best Estimate from Both Inter- views of Fall, 1958, Balance in Relation to Whether Respondent Keeps Records of Money Spent 85 32. Response Error in Reports of Balance at Three Points in Time Based on Interviews Taken in Fall, 1958 87 33. Response Error in Reports of Balance at Two Points in Time Based on Reinterviews Taken in Spring, 1959. ... 88 34. Response Error in Spring Report of Spring, 1959, Balance in Relation to Fall Report of Fall, 1958, Balance 89 35. Response Disposition for Persons Known to Have Cash Loans in Relation to Experimental Manipulation of Felt Anonymity by Type of Interview 96 36. Response Error for Cash Loans in Relation to Interview Treatment 97 37. Number of Cash Loans Reported in Relation to Interview Treatment 98 38. Response Error in Dollar Amount of Payment per Payment Period 100 39. Response Error in Purpose of Known Loan 101 40. Response Error in Report of Lending Agency . .. 101 41. Response Error in Total Number of Payments Originally Agreed Upon 102 42. Response Error for Number of Payments Made 102 43. Response Error for Number of Payments Left to Make 103 44. Response Error for Cash Loans in Relation to Income of Family 103 45. Response Error for Cash Loans in Relation to Education of Head of Family 104 46. Response Error for Cash Loans in Relation to Occupation of Head of Family 105 LIST OF TABLES xiii 47. Response Error for Cash Loans in Relation to Age of Head of Family 106 48. Response Error for Cash Loans in Relation to Sex of Respondent 106 49. Response Error for Cash Loans in Relation to Whether Borrower was Respondent 107 50. Response Error for Cash Loans in Relation to Score on Attitudes Toward Borrowing 108 51. Response Error for Cash Loans in Relation to Conformity Score Ill 52. Response Error for Cash Loans in Relation to Personal Effectiveness 113 53. Personal Effectiveness in Relation to Tendency to Conform to Obvious Social Dicta 114 54. Response Error for Cash Loans in Relation to Whether the Respondent Keeps Financial Records 115 55. Response Error for Cash Loans in Relation to Purpose of Loan as Obtained from Lending Records 116 56. Response Disposition in Relation to Size of Actual Balance in Account of January 1, 1959 125 57. Response Disposition in Relation to Mean Actual Balance in Account on January 1, 1959 126 58. Response Disposition in Relation to Preferred Interviewing Technique 127 59. Response Error for January 1, 1959, Balance in Relation to Preferred Interviewing Technique 128 60. Mean Actual Balance and Mean Reported Balance for Each Category of Response Error, January 1, 1959, Account Balance 130 61. Response Error in Relation to Certainty of Identifi- cation of the Selected Account, January 1, 1959, Balance 133 62. Response Error for January 1, 1959, Balance in Relation to Whether the Total Reported Savings Account Holdings for January, 1959, Would Have Been Accepted on Basis of Interview Evidence Only 135 63. Response Error for January 1, 1959, Balance, in Relation to Whether Respondent Gave Information on Savings Accounts in the Financial Form 136 xiv LIST OF TABLES 64. Response Error for January 1, 1959, Balance in Relation to Ownership of Account 137 65. Response Error for January 1, 1959, Balance in Relation to Total Income of Family 138 66. Response Error for January 1, 1959, Balance in Relation to Education of Head of Family Unit 140 67. Response Error for January 1, 1959, Balance in Relation to Occupation of Head of Family 141 68. Response Error for January 1, 1959, Balance in Relation to Age of Head of Family 142 69. Response Error for January 1, 1959, Balance in Relation to Whether Respondent Keeps Records of Money Spent 144 70. Response Error in Reports of Balance at Two Points in Time, Based on Interviews Taken in Fall, 1959 145 71. Response Error for January 1, 1959, Balance in Relation to Evidence of Concealment of Financial Information Among Members of Family 146 72. Response Error for January 1, 1959, Balance in Relation to Age at which Respondent Knew Family Income 147 73. Response Error for January 1, 1959, Balance in Relation to Age Which Children Should Be Before Knowing Family Income and Savings 149 74. Response Error for January 1, 1959, Balance in Relation to Whether Respondent Reported a Round or an exact Figure for Balance 150 75. Response -Error for January 1, 1959, Balance in Relation to Interviewers' Ratings of Accuracy of Respondents' Reports of Bank Accounts 151 76. Reported Change in Balance from January 1, 1958, to January 1, 1959, in Relation to Actual Change in Balance 152 77. Response Disposition to One-Page Mail Questionnaire for Several Studies 156 78. Response Error for Cash Loans Study in Relation to Whether Respondent Returned Mail Question- naire 157 LIST OF TABLES xv 79. Percentage Distributions for Responses to Four Questions on the Mail Questionnaire for Several Studies 159 80. Percentage Distributions for Responses to the Other Four Questions on the Mail Questionnaire for Two Studies 160 81. Response Error for Cash Loans in Relation to Other Factors 161 82. Response Error for January, 1959, Balance in Savings Accounts in Relation to Whether Interviewer Succeeded in Making Clear What Study was About 163 83. Response Error for January, 1959, Balance in Savings Accounts in Relation to Whether Interview was Interesting 164 84. Response Error for January, 1959, Balance in Savings Accounts in Relation to How the Respondent Would Feel About Being Interviewed Again 165 85. Response Error for January, 1959, Balance in Savings Accounts in Relation to How Well the Re- spondent Liked the Interviewer as a Person 166 86. Response Error for January, 1959, Balance in Savings Accounts in Relation to Whether Surveys Like This One Are a Good Idea 167 87. Response Error for January, 1959, Balance in Savings Accounts in Relation to Whether There Were any Questions That Were Too Personal or Prying 168 88. Response Error for January, 1959, Balance in Savings Accounts in Relation to Whether the Re- spondent Had Any Doubts About Giving Information 169 89. Response Error for January, 1959, Balance in Savings Accounts in Relation to Whether People Will Give Us Accurate Information About Their Finances 170 90. Experimental Manipulation in Cash Loans Study in Relation to the Favorableness of Responses to Questions on the Mail Reinterview 172 91. Response Error for Cash Loans Study in Relation to Over-all Score on Favorableness from Mail Questionnaire Items 173 xvi LIST OF TABLES 92. Score on Favorableness Toward the Interviewing Situation in Relation to the Score on Favorableness Toward Accuracy 174 LIST OF CHARTS 1. Total Car Debt Owed 23 2. Best Estimated Reported and Actual Balances, Fall, 1958 70 3. Reported and Actual Balances, January 1, 1959 131 PART ONE INTRODUCTION 0* I. A STATEMENT OF THE PROBLEM The survey method has been used extensively in studies of saving and other aspects of consumer finances, and it may be expected to contribute further in the future. The contribution of the survey technique will be maximized if the errors in estimates of such items as consumer holdings of liquid assets and their indebtedness can be measured and reduced. Three main categories of error can be identified, and each requires a somewhat different approach for cor^ rection. -j-ftiAL--^ ^M" rf>£ The sampling error in estimates of such magnitudes as mean debt per family arising from car purchases can be estimated for a particular probability sample and the size of the error can be con- trolled by changing the characteristics of the sample. It is the error which arises because not every individual in the population under study was included in the sample. A second type of error is that of non-response , where an indivi- dual is included in the sample but the designated respondent is not interviewed. If the non-responses differ from the persons inter- viewed in some systematic way, a bias may be introduced. Errors of non-response are less manageable than sampling errors, because the bias may be unknown in direction or in magnitude, but, at least, the number of non-responses for any survey is usually known. The present study contains some information on the bias of non-response in financial surveys. The present study is primarily concerned, however, with the third main class of error, response error. If the reported value of a statistic differs from the actual value in any interview there is an error of response with respect to the item of information. Response error, thus defined, includes errors made in processing and tabulating completed interviews. This monograph, however, is concerned only with errors made in the interview itself and in the write-up of the interview by the interviewer, and subsequent use of the term "response error" in this report is restricted to such a meaning. The existence and importance of response error in reports of some types of data in financial surveys has become more and more evident and has led to serious concern as to the validity of the data. Critical study of the data has shown that response error is signifi- cant in regard to certain financial topics, but that it is not significant in regard to other topics. Aggregate figures derived from the Surveys of Consumer Fi- nances have been compared with aggregates independently derived from such sources as Census reports, sales, savings institution and insurance company reports, government administrative records, and data gathered by the Departments of Labor, Agriculture, and Commerce. These comparisons suggest that Survey of Consumer 4 AN INVESTIGATION OF RESPONSE ERROR Finances aggregates are very accurate in regard to income, pur- chases of consumer durables, housing tenure, mortgage debt out- standing, and mortgage and rental payments. On the other hand, comparisons of Survey of Consumer Finances personal debt data with Federal Reserve Board estimates based on lender sources display significant underreporting in the former. Data on liquid asset holdings show the same picture. Estimates of aggregate liquid asset holdings from the Surveys of Consumer Finances have been consistently lower than estimates based on institutional records by one -third or more. Financial surveys are subject to significant response error, but such errors are primarily restricted to reports of personal debts and liquid asset holdings. The preceding discussion dealt with aggregates; but it can be argued that the prime value of surveys lies in their ability to ascertain several items of information about the same family or spending unit and, thus, to permit study of economic relationships. Errors may occur in means estimated from surveys, yet the data may be valuable for the study of interrelationships among economic, sociological, and psychological variables. Such an argument, however, certainly does not obviate the necessity for attempting to measure and reduce response error. Response errors limit the contribution to knowledge of the survey technique. The optimal use of the technique requires the sophisticated measurement and control of such errors. The studies discussed in this monograph were attempts to measure and manipulate response error in the areas of personal savings accounts, personal cash loans, and automobile debts. The use of the word "error," in the present context, implies the existence of a standard against which individual answers to particular questions can be checked. The investigator should be able to satisfy himself that he has before him two observations which refer to the same fact. Where the two observations are not identical, he would like to be able to state that one of the two is correct, and that the discrepancy is the result of an error in the other. In order to certify the correct- ness of one of the observations, individuals were selected from in- stitutional and official records (depending on the particular study, this was either a savings account balance or information on an automobile or cash loan). The individuals selected were interviewed in an attempt to elicit by questioning information which could later be verified (with due safeguards regarding the anonymity of the respondents). The interviewers themselves did not know how the sample was selected and usually they had not even been told which financial topic was of principal interest. The pursuit of the twin goals of measuring the error and of re- ducing or controlling it led to forming a judgment about sources of error by careful study of individual interviews, and this approach was A STATEMENT OF THE PROBLEM 5 used extensively in the present research. An additional technique, which was also used, is to introduce an experimental design. That is, an investigator may systemically vary different aspects of the technique of investigation and compare the magnitude of the response error in interviews taken using the different techniques. The measurement and manipulation of response error, as re- ported in the following chapters, turned out to be very difficult but certainly not impossible. The preliminary studies discussed in Chapters II, in, and IV provided a valuable amount of experience for the more intensive field experiments reported in Chapters V, VI, and VII. The first preliminary study (Chapter II) was an attempt to measure the validity of individual reports of amounts in savings accounts and of changes in those amounts. Through the cooperation of savings institutions, the Center interviewed close to 100 persons for whom the balance of a savings account could be verified. The results were disappointing in view of the expectations based upon more general surveys: the response rate was much lower than usual, and response error was significantly evident. The more recent field experiments, however, have shown the low response rate (fluctuating around 70 percent as opposed to the more usual 85 percent in Sur- veys of Consumer Finances) to be characteristic of samples of savings account holders, a finding which is discussed more fully in later chapters. All in all, it was very difficult to assess the ac- curacy of such projects as the Surveys of Consumer Finances on the basis of this early study. Clearly, more work was called for, and this study did provide some hypotheses and insights amenable to systematic incorporation in subsequent designs. Three years later, in 1957, the second preliminary study was undertaken (Chapter III). This study dealt with car debts and at- tempted to measure the accuracy of auto owners' reports against a criterion obtained from official records. It involved three separate stages: exploratory interviews, the main Chicago study, and an analysis of a small number of recent interviews from the Survey of Consumer Finances. Each stage yielded information which proved valuable. An interesting, although tentative, interpretation of some of the car study results has major implications for survey design. The data suggest that relatively high accuracy can be more readily obtained when only a few topics are covered, while to ask for a large number of items of information tends to result in a relatively low level of accuracy for each item. Furthermore, it was becoming very evident that what took place during the actual interview was of great importance to the research. An intense and sensitive scrutiny of the interview situation seemed essential. Just such an intensive scrutiny of the interview situation is dis- cussed in Chapter IV. In that project, a small number of persons were individually interviewed in a room which had a one-way screen 6 AN INVESTIGATION OF RESPONSE ERROR and a one-way audio system, enabling outside observation of the interaction between interviewer and respondent. This observational study directly led to a radical change in questionnaire construction which was later used as part of the experimental design in one of the field studies. With such a background of preliminary studies and knowledge gained from more general surveys, it was felt that the next step had to be field experiments of systematic design. In the fall of 1958, the first of three field experiments was begun. The study (see Chap- ter V) took place in two eastern metropolitan areas and involved a set of interviews in the fall of 1958 and reinterviews of the same people in the spring of 1959. In the fall study, half of the sample was interviewed with a lengthy, structured questionnaire of the type used in the Surveys of Consumer Finances; the other half was inter- viewed with a short, relatively unstructured questionnaire which necessitated more than the usual amount of probing on the part of the interviewers, who had received training to this end. This ex- perimental design was based on the experience of the preliminary studies described previously. The sample was selected from the personal savings account records of cooperating savings institutions and included only persons with large balances ($1,000 and over). The intentions of the researchers, then, were: (1) to measure the fre- quency and magnitude of the discrepancies between reported and actual balances in the relevant account for specified dates, and (2) to compare the magnitudes of the response errors of the structured questionnaire with those of the unstructured interview and to deter- mine the relative efficiencies of the two instruments. The spring reinterview questionnaires were all of one type, but they did provide a comparison with the data obtained in the fall. The use of the reinterview to gather data is discussed in Chapter V. The questionnaires -- both fall and spring -- also included a number of psychological instruments designed to tap the underlying characteristics of the respondents, on the assumption that accurate and inaccurate respondents might well vary along psychological dimensions. If so, then the identification and measurement of such differences might yield insight into the understanding and control of response error. The next field experiment dealt with persons known to have active cash loans from personal loan companies. The primary experimental technique involved manipulating the degree of anonymity associated with the interview situation, ranging from one extreme wherein the interviewer asked for a particular respondent by name to the other extreme where the respondent's name was never used and the fi- nancial data might never be seen by the interviewer. This last was accomplished by a "sealed envelope technique"; a separate form containing answers to the financial questions was placed in an A STATEMENT OF THE PROBLEM 7 envelope, sealed, and mailed directly to Ann Arbor. Some of the psychological instruments used in the savings account studies in the eastern cities were repeated in the cash borrowers' study, al- lowing a comparison between the two groups. In addition, a post- interview mail questionnaire was sent to the respondents, asking for their reactions to the interview and interviewer. The third and final field experiment took place during the fall of 1959 and once again dealt with holders of savings accounts, this time in a third eastern metropolitan area. A two by two experimen- tal design was used: the preferred respondent was either the head of the household or the spouse of the head, and the respondent either was offered $10 for filling out a fairly detailed financial form or was not offered any money for filling it out. Thus, the interview situa- tions could be classified as "Head-Pay," "Head-No Pay," "Wife- Pay," and "Wife -No Pay." Each technique could be compared against the others in terms of the accuracy of the respondents' re- ports of their savings account balances. On the basis of analysis of the field experiments, a number of conclusions have been drawn regarding response error in financial surveys; in addition, certain of the results suggest hypotheses which are clearly testable in future research. In the following chapters of this monograph, the studies and in- terpretations mentioned in summary fashion in this introduction are discussed at length. Part Two (Chapters II-IV) covers the prelimi- nary studies. Part Three (Chapters V-VIII) reports the field ex- periments, including the financial, socio-economic, and psychological correlates of the response errors. Part Four (Chapter IX), after summarizing and integrating the results of the field experiments, presents the theory of response error which has developed on the basis of this over-all project. This chapter includes a brief critical evaluation of the specific techniques used in the various studies and some general observations on the conduct of research on response error. The problem of response error is quite serious, and in part one must learn to live with it; but there are ways of controlling it and researchers certainly can work to minimize it. Research on re- sponse error -- with dual goals of measurement and control -- certainly is not easy; but neither is it impossible nor impractical. The headway made to date indicates that attempts to control the error — and to measure it -- will require a great deal of hard and careful work, and that techniques which are successful in one sit- uation may well be unsuccessful in another. Since this type of re- search is not easy to carry out, this monograph includes discussions of many of the difficulties which arose in the present investigation in the hope that others may be saved trouble and expense in facing similar problems. PART TWO PRELIMINARY STUDIES II. PRELIMINARY STUDY OF SAVINGS ACCOUNTS During the summer of 1954 the staff of the Economic Behavior Program of the Survey Research Center attempted for the first time to measure the validity of individual reports of amounts in savings accounts and changes in those amounts. The project was under the direction of James N. Morgan. It followed projects which were con- cerned with the measurement of response error in reports of the value of owner occupied homes and family income which have been reported elsewhere 1 ' 2 . This chapter is based on Morgan's original memoran- dum describing the results of this preliminary inquiry. Purposes of the Study and Description of Procedures The main purpose of this pilot study was to check the respondents' answers against the true facts in order to improve future research. For this type of check the important consideration was not so much to obtain a sample that would be efficient for quantification of results as to obtain a sample that would be unbiased and randomly selected and would permit the investigation of factors associated with accuracy or inaccuracy of reporting. Since the investigators had only hunches as to possible sources of error, a small-scale test intended to develop hypotheses and to develop a better questionnaire was what was planned. The questionnaire used was a short one which led relatively directly to the subject of savings accounts via questions about how the family was getting along financially, the occupation and income of the family, their debts and savings, various forms of saving, and amounts held in these various forms. Then the number of savings accounts, whether they were single or joint, and whether they were all in the same institution were ascertained. The respondent was asked whether he was close or far off in his report of amounts, and then the interview closed with some standard and easy questions concerning demographic data. 1 Leslie Kish and John B. Lansing, "Response Errors in Es- timating the Value of Homes," Journal of the American Statistical Association (Sept., 1954), pp. 520-38. 2 Monroe G. Sirken, E. Scott Maynes, and John A. Frechtling, "The Survey of Consumer Finances and the Census Quality Check," An Appraisal of the 1950 Census Income Data, Studies in Income and Wealth, Vol. 23, National Bureau of Economic Research (Princeton, New Jersey: Princeton University Press, 1958), pp. 127-69. 11 12 AN INVESTIGATION OF RESPONSE ERROR Table 1. Disposition of Selected Addresses in Preliminary Savings Study Groups of addresses Number of addresses Selected for clustering 862 Selected in clusters 216 Interview or contact attempted 197 Non-sample addresses (moved away, dead, no such address exists) 64 Selected addresses properly in sample 133 The cooperation of a savings institution which already had drawn a sample of savings accounts for its own purposes was secured. The institution selected a sub-sample from the larger sample already drawn, in such a way as to reduce the spread of the sampling ratios for different -sized accounts from 500: 20: 1 to 20: 4: 1. For pur- poses of interpretation, it is important to note the following things about the resulting sample: First, it was designed to give represen- tation to four different counties, three strata of account sizes, and three levels of account activity. Second, losses through inaccurate addresses, accounts that do not belong in a sample of individuals' accounts, and other accounts that could not be checked make the sample inappropriate for making aggregate estimates. One must also keep in mind that this is a small sample and that the distribution of savings accounts by size is highly skewed. The disposition of the selected addresses is shown in the accom- panying table (Table 1). The information in Table 1 is on an unweighted basis. That is, each address is counted as one with no attempt to compensate for the variable sampling fraction. The table shows that only about one ad- dress in four of those originally selected for questioning was actually selected for interviewing. Most of the loss came in the selection of clusters. The purpose of the clustering, of course, was to make it possible for the interviewers to take the hundred interviews planned without travelling all over the four counties in the sample. Error of Non-Response Table 2 shows the disposition between responses and non-res- ponses for the 133 addresses at which interviews should ideally have been taken. The response rate of 71 percent is considerably lower than the average of about 85 percent which interviewers from the Survey Research Center customarily have obtained in the Surveys of PRELIMINARY STUDY OF SAVINGS ACCOUNTS Table 2. Response Rate in Preliminary Savings Study 13 Response disposition Percent Number Interviews Non-interviews Not at home Refusals and other non-response 71 29 11 17 95 38 15 23 Total 100 a 133 a Detail will not add to total because of rounding. Consumer Finances. At the time when the study was completed this low response rate was taken as indication that the field work had been badly managed. In the light of experience with subsequent tests it appears that the low response rate may be the result of difficulty in interviewing this type of person. This point will be discussed more fully in later chapters. Information about the amounts missed and about the accounts missed is shown in Table 3. This table is on a weighted basis. It shows that about 4.2 percent of the accounts were missed in the field because of refusals, because the respondent was ill, because of language difficulty, and so forth. These accounts included 16.9 percent of the total amount in the sample. Hence, it is apparent that the accounts missed for this reason were on the average large ac- counts. It is instructive to know that the cases where the respondent was unknown, dead, moved away, or had no such address account for about the same portion of amounts as they do of accounts. The interviews actually taken account for 60 percent of the accounts but only 36.8 percent of the amounts. In other words, these data show a tendency for the average balance to be larger in the non-responses than among the responses. Further information on the relation be- tween size of account and response disposition will be reported in later chapters. These later results point to different conclusions about the relation between size of balance and whether an interview will be taken. Comments on Response Error The information obtained from this study about response error is summarized in Table 4. This table includes some secondary accounts which could be checked because they were in the same institution as the accounts originally selected. It excludes cases where amounts could not be checked. Altogether a total of 77 accounts could be checked. 14 AN INVESTIGATION OF RESPONSE ERROR tj •£" cd 3 05 CA3 < CO > .3 CO 5 Ph a a 0) ti 3 10 s o o o s cd r/3 w - c co 3 O 0) s rd < H <— 1 O t- ift o o © O ift CD esi © ^» r- in o •h esi o l sj Co cd a S w 03 .5 w «-» a o cd w N 03 5 to o O TJ > « 03 S-, y fcl S ° cd X 03_g 03 U a TJ 0> 4> S-, * T3 3 0) | a o w 8 ft" s -? a o cd w XJ 55 tj g q> cd O 03 ^~. tj to c 5 ' c ■a* > ^ 0) '- W 2 5 TJ °> 1 S 03 c 3 cd O A > U c h cd a; cd 03 > o to cd o xl 0) ^ TJ CO co 3 ° 5 c bo c C ts m S5 a c „ ■"■* to 03 C "*"' -4-> u ID C c 3 o u fcs cd co *J rt "^ cd cd rt 3 ., XI <*-( o ■a TJ cd c u £ cd (13 °. r, n -4-* o s C 0) CO o cd cd co 03 03 CO C 03 o x: ■5 £ ^■o Sid S3 03 X3 X! Cd 0) u to 3 co cd 3 T3 a> U 0) 0) £ oo o cd (h -" o ^ rt 3- -X to 03 W> 0> m s I B* 03 a> u>3 .5 > Si . tO w ? >.X1 U 03 -Jj O > c 6 c u cd 5 <»> 0)Xl x: S > . 3 0) xi £ c5 -SxT ■p-S Q) O ^E 03 O U TJ cd c cd to u 00 id 0) 5 ^-fj a> 03 cd o ^ -5 PRELIMINARY STUDY OF SAVINGS ACCOUNTS 15 >> T3 3 H-> CO a Cfi • H hfl H G • h -r-> > oi rj C/) +j C >> 'J ?H O rrf cj a o •rH < a •rH rfl hO a> r, rH ■T-t rH > a f] Ul ■rH r, -(-> •i-i rH o <1> 2 O Dh a g < o ■i-H -r-> e G o 4-> • rH -M ■M rr> !h ri c ) >—< a cd -G tf +j £ £ CD T? ■ i-i (1) > rH rH rr( 01 -r-> rH a o u ^r 1 £ r| > UJ • <-t rQ > OS 0> H Ch O ,_, rH t- •<# OO o •* (M D- o CO rH rH t- H o & o £ o £ „ o rH H in T3 CM G #3- aS i O OS O OS O OS r. r. rH rH O "*• H CM €/3-#3- H-> g G 1 o O OS a O OS rt O OS rH €«-€«- O O OS a O OS CD O OS rH -^ in O rH PcTrt«" rH £ €«■€«- U CD O OS +J O OS G O OS 1—1 CM -# CSI rH o i— ( i— 1 rH rH O OS O OS lO OS CO (M rH l—t r—t CO &-&- 1 O OS O OS rH rH OS CD o OS > o a OS OS as as CD OS OS as ^ rH OS OS as ■* T3 CTj •N rt rt CM G H-> g OS OS rH ■^ OS #3- rt o o OS OS &- 03- €«- l H •rH +J G -r-> OS ■^ OS 1 1 1 O o OS se- €«- o o o o o rH i o o o o o •rH H-> Cfl €^ o O o o o ^ ^ 1 o o „ „ „ o m e rH CM ITS H CM ITS rH CM r-H O €»- «3- m- €«- €«- #3- #3" v> 16 AN INVESTIGATION OF RESPONSE ERROR a u 3 o i ° « > , , fn T1 ro -M > ctj cd i2 3 CO CO hn 1 ■rH > Jh 11 ^ S G -4-> 3 9 ■rH r: o ° > •rH o ° rt << W MH o £3 bC >> Sh Clj o •rH •r-l 3 H •rH f — 1 Q • rH $H rH ^ O co CD CO •rH T3 O 3 o °l CO rH CD CD SP G CD O rH CD rS Pk U O CD ■i H o rH u ° o M O €«• CO S 11 in ro as t- ^ t- in CO HH CD C *- ( rH o o CO 2 C rH 3 O O < CM i— i i— i cm as ■'tf ■r* CO 00 Tt« CO r-t CO CO CO 73 CD CD • rH > u CD -M rS CO o rH a Owns business Over 45 years old Large account according to institution records ($5,000 or more) Active account (one or more withdrawals January, 1949- December, 1953) Reported within correct bracket 3 Income of $5, 000 or more J3 CO CD CO a o «*H o u CD rQ a s u o a CD rH o +-> CD U £ • rH a <+H T3 a cd „ -t-> CD rW o a rH Q CD s cd CO C • rH o C 3 o t> Sh CD > -Q PRELIMINARY STUDY OF SAVINGS ACCOUNTS 17 Of these 77 cases, 34 were in the correct bracket, 11 were one or more brackets higher in the interview report, and 32 were one or more brackets lower. Thus, underreporting by respondents was ap- proximately three times as frequent as overreporting. Of the 32 reports which were too low, 10 involved an account under $200. Four reports were more than one bracket too high. Twelve reports were too low by more than one bracket, including 8 who reported no savings account when actually they had one. Doubtless, the tendency to under- report was stronger than the tendency to overreport. Table 5 shows some characteristics of those who gave accurate reports within $200 compared with those who did not. Inaccurate reporters were older, had larger accounts, and were more likely to own abusiness. An absolute error of $200 is more probable the larger the account; on a balance of $5, 000 an error of $200 is small in terms of percent while on an account of $100 an error of $200 is very large in terms of percent. Since old people and businessmen tend to have larger balances, it is not surprising to find that they are more likely to give reports in error by $200 or more. It is interesting to note that more of those who were in error by $200 or more had active accounts. Evidently more recent use of the passbook did not improve accuracy of report. Another way to summarize the average error of report is to com- pare the mean balance reported by respondents, $1,464, with the mean balance reported by the institution, which was $2,904. The average difference, thus, was $1,440; and the average balance reported by the respondents was about half that indicated by the institution. Another relevant statistic is the average absolute discrepancy, dis- regarding the sign, which was $1,821. This estimate was dominated by reports from a relatively small number of respondents who made large errors. Of this discrepancy, 47 percent was accounted for by farmers and businessmen in the sample. For many purposes the change in the balance in a savings account is at least as interesting as the absolute amount in the account at the time of interview. Table 6 summarizes the information from this study on the validity of reports of change in amount in savings ac- counts. This table includes only the 68 cases where there seems to be evidence of a match and where all information in amounts "now" and a "a. year ago" was given. Of the 68 responses, 37 were incor- rect by one bracket or more. The remaining 31 were in the correct bracket. These results, of course, depend upon the choice of brackets. Alternatively one may note that of the 68 reports, 38 were in error by $200 or more while 30 were correct within this margin. In some of these cases of large discrepancies the interview made so much sense in terms of internal consistency that it seems highly probable that the respondent had in mind a different savings account which changed in the way described in the interview. 18 AN INVESTIGATION OF RESPONSE ERROR T3 3 ■+-> C/J CO w bfl hfl C r, ■rH •rH > > rt a CO U) >» a u • rH rf -4-> 3 'a o •rH . 1 £h CD ^x rH ^ Ph •i-l G • rH tu bjj O u Oh C3 r! o o • i-H -u +H fn 3 O +J •rH -4-» CD en « r— 1 £ rC cd •iH > ■rH rH CD Ti 0) a . Fi CD o a> CJ rQ w rrj +j H 5h' -1 o o o < rH ei CJ S o o 13 o CO t- "^ OS mc-CMCOO i-H i— 1 CO CO O O rH o o l O -- 1 ,_( C- i-H o l—l O Oi O CT> 1 T-H i-H CM o cr> CM -"^ i CM i-H CM ^ «H O i— l 1 OS i-H i— 1 i i— 1 i-H CM 0) > rH 5 o CM tF O i-H CM 00 i— 1 CM OS l OS i—i i—l se-so- + lO tH CD i O OS O OS CM "* + i— 1 CM i~ 1 ^ l O OS O OS in os + i-H -e- + CM CM 0) bfi C .2 o +J 3 a S o W +j C rH HH O a a> rH Increases $1,000 $500-$999 $200-$499 $1-$199 Decreases $1-$199 $200-$499 $500-$999 $1,000 or more No change r— 1 d +-> o H PRELIMINARY STUDY OF SAVINGS ACCOUNTS 19 CO CD 1* a £ CO W 0) rr; CO bi) (i) G ■u > GO O >» CO ,* ■ g > .3 G CD C _3 -iH o •l-H ^ is s a Up CO > CD £ CO ^ o 2 Ph M CU •^ s U a si m U ^ c 0) cu rt 5 en Cu ff u O JS CO 0) cu u 0) 0. _ (jj ao CO CO O uo eo «-< «h o •"-> in cm t- Cu kl rt en Cu 3 a- tL 5 5 o s- Cu —■CO O- ^j •-" O^-i t~ *~ l CO CM — " ^ 'T —• o o u cu E 3 2 COIT5 O "-I CM OCM CA CM en o o en rt a! ■a cu jm Li =s a> > u cu 0) '3 c o CU cr bi o JS CO 5 u s- CU corr "-"coco coco co co 1/5 *J" rH CO o o cu s 3 55 t— CO CO'-'CM •-« «H O i-l o CO 0) s- rt 5 to cu 3 Cu- be 5 o J .5 o u cu a. moo coot o^> co oo CO CO ■"-' CM o o u cu E 3 2 t-O CM UO •"-" 0«-i CO CM CD CM (0 cu > cu "c < e cu u u cu rf CO OO OS CO "H m CM 00 HO CO CO O o cu a E 3 oo co oococo —"in O t- CM OO c 0) E cu au u sr o cu cu <-. Reported "our" loan Amount times number of payments is within 5 percent of recorded amount times number of payments Amount borrowed is within 5 percent but amount times number of payments is wrong by over 5 percent Amount borrowed is wrong by 5 percent or more by either calculation May or may not have reported "our" loan Amount borrowed is within 5 percent but amount times number of payments is wrong by over 5 percent Amount borrowed is wrong by over 5 percent by either calculation Failed to report "our" loan Does not seem to be discussing "our" loan Questionnaire incomplete "Our" car sold, now clear of debt, and so forth O 01 >•£ tn O m t, i- >. 5 o » I c o o S e jj in £ IS 1 o tz, c 6 *a < - 1- ^ — >> o> ft U O ft tn in 2 g S3 > § > S5 w E tn e en -g 3 c S a 2 O J- £ £ 5 cu O U u o o c 6 u g c O m 6 3 a 3 ■s > tn — . g w cu — o •a *; o CU o 3 O s tn 0) o 0) >> B >-. rd h >1 jg c 2 c o> R ^ b Oi ^_* u h c 3 b c 3 n id o < o a. ir> s * e 5 8. S c 3 Q PRELIMINARY STUDIES OF CAR DEBT 31 for by the inquiry. The sequence of questions used in both the long and the short forms was in fact designed to uncover primary loans. Thus, research leads to a strong recommendation that this question- naire (or any questionnaire in this field) should be designed to cover secondary loans, or else these loans should be excluded from study entirely. The second possibility is that respondents may be reluctant to discuss secondary loans. "Everybody" buys a new car on credit, but not "everybody" has trouble with the payments or has to borrow on his car. It is instructive to note that no respondent in the Chicago sample denied ever having had any car debt. Discrepancies were found with regard to the existence and amount of particular loans, and some said their loans had been paid off; but no respondent in- sisted that he had paid cash for his car and had never borrowed against it. It should also be noted that when respondents did report the existence of "our" loan, they reported the amount involved within a rather close margin of error. As has been previously noted there were 48 interviews in which the respondent did report the existence of what was fairly sure to be a car loan. In 33 of these interviews the product of the amount of the payments by the number of payments was within 5 percent of the corresponding amount from records. There were a few interviews, 8 of the 49, in which the respondent reported the original amount borrowed accurately but seems to have made some error in the report about the payments. That is, the amount borrowed was within 5 percent but the product of the amount by the number of payments was wrong by more than 5 per- cent. In only 8 of the 49 interviews was the amount borrowed wrong by 5 percent or more by both calculations. Table 10 shows the relation between the debt according to state records and according to respondents' reports, interview by inter- view. The first section of Table 10 covers the 36 interviews in which the loan seems to have been a primary debt on the car. For these 36 interviews the mean of the respondents' reports was $2,090 and the mean from the state records was $1,998. The mean deviation was $93. For 37 secondary loans, the mean of the respondents' re- ports was $511 for those cases where the respondent did report or might have been discussing the loan under study. However, in about half the cases the respondent failed to report the loan. The mean for the official records for the entire group was $520 (Table 11). It may also be of some interest to estimate the proportion of the total list of debts which was in fact revealed by the respondents. This statistic is of somewhat doubtful interpretation because of the mixed nature of the list of loans. But for what it is worth, the mean of all records was $1,156 and the mean of the respondents' reports 32 AN INVESTIGATION OF RESPONSE ERROR Table 10. Relation Between Debt According to State or County Records and Respondents' Reports (in dollars) in the 1957 Chicago Interviews Respondents' report (1) State records (2) Deviation (1-2) (3) GROUP I: Primary Loans Respondent seems to be discussing "our" loan 1,955 1,875 80 2,700 2 833 -133 1,512 1 523 - 11 2,000 2 000 2,940 2 952 - 12 2,160 2 158 2 3,276 3 295 - 19 1,600 1 600 2,370 2 384 - 14 2,790 2 723 67 3,240 3 240 2,250 2 231 19 3,600 2 329 1,271 2,424 2 424 1,056 1 056 1,635 1 635 1,680 1 680 816 615 201 2,040 1 980 60 900 1 046 -146 2,797 2 487 310 1,671 1 522 149 3,504 3 512 - 8 1,686 1 686 1,062 885 177 864 742 122 1,560 1,608 - 48 1,464 1,342 122 1,680 2,100 -420 3,360 3,465 -105 1,476 1,403 73 1,380 1,383 - 3 Can't tell whether respondent is discuss ing "our" loan 4,002 3,720 282 2,430 891 1,539 1,440 1,985 -545 1,920 1,612 308 PRELIMINARY STUDIES OF CAR DEBT 33 Table 10 (Continued) Respondents' report (1) State records (2) Deviation (1-2) (3) GROUP II: Secondary Loans Respondent seems to be discussing 'our" loan 656 656 504 504 432 432 504 656 -152 525 525 198 229 - 31 660 656 4 224 210 14 672 660 12 a 217 1,500 J 2,021 -448 150 b 216 - 66 576 578 - 2 648 656 - 8 200 b 336 -136 234 229 5 500 b 656 -156 Can't tell whether respondent is discussing "our" loan 500 b 4,830 629 656 -129 4,174 Respondent does not seem to be discussing "our" loan (2,400) 414 (1, 080) 628 (4,052) 260 (3,090) 336 (3,277) 432 (1,923) 552 (1,620) 656 (1,536) 150 (2,328) 656 (2,880) 552 (2, 155) 280 (1,620) 504 (1,620) 384 (3,660) 300 ( 810) 656 (2,460) 288 (1,824) 485 34 AN INVESTIGATION OF RESPONSE ERROR Table 10 (Continued) Respondents' report State records Deviation (1-2) (1) (2) (3) GROUP II: Secondary Loans (Continued) "Our" loan was not covered by the question asked (1,890) 534 (2,700) 656 (2,610) 656 GROUP III: Loans not Clearly Either Primary or Secondary Respondent does not seem to be discussing "our" loan (3,000) 1,350 (1,200) 738 (1,920) 393 (2,940) 476 (3,300) 1,949 __ d 555 (2,160) 543 (1, 104) 267 (2,820) 329 (2,460) 318 (1, 800) 1,056 (2, 100) 656 (2, 100) 413 "Our" loan not covered by the question asked (2,208) 358 e 552 (2,550) 534 Information as to amount of payments was given in the interview, but not as to number of payments or amount of loan. Respondent reported directly the amount originally borrowed but information on payments was not complete. Numbers in parentheses under "Respondent's report" refer to car loans reported in the interview other than the loan covered in the records. Respondent says he never had such a car. e Information not ascertained in the interview. PRELIMINARY STUDIES OF CAR DEBT 35 Table 11. Summary of Relation Between Debt According to State or County Records and Respondents' Reports in the 1957 Chicago Interviews Respondents' State or county Type of loan reports records Mean Number Mean Number Primary loans $2,090 36 $1,998 36 Secondary loans 511 37 a 520 37 a Loans not clearly either primary or secondary 14 670 14 All loans 965 87 a 1,156 87 a Primary and secondary loans combined Respondent seems to be discussing "our" loan 1,583 53 a 1,543 53 a Respondent failed to report "our" loan 34 512 34 a Excludes respondents who reported the existence of a loan but failed to report the balance. Also excludes 1 respondent where the match between report and records is doubtful. This respondent re- ported a secondary loan of $4, 830; actual amount of the loan was $656. was $965. These means were computed on the assumption that the respondent was not reporting the loan under study, that is, reporting zero for it, in the cases where non-report is indicated as the most probably interpretation in Table 10. Accuracy of Interviewers' Impressions of Accuracy It may be of some interest to compare the interviewers' im- pressions of the accuracy of response with the measure of accuracy of response obtained by comparing debt with the information in the interview. Could the interviewer tell whether she was obtaining complete and accurate information? Table 12 suggests that she could not. This table was prepared, of course, only for those interviews where the interviewer did not know the details about the loan. Of the 12 interviews in which the interviewer expressed the opinion that the response was accurate there were 8 in which the respondent seems to have failed entirely to report the loan under study. 36 AN INVESTIGATION OF RESPONSE ERROR co •i-H 4J > C ^ cd cd a-B (1) •!-! CD «h S-. O <+H CD O ,Q 0) S CD 3 *-i C >> o CO -5 o +2 A CO CO •=< co Q o c 8 « < Q o g o CO CO CD 7? ■•-< CD CD ,-H CO CD CD •2 > H CD o u o u CD b- £ -a o CD CD aj > .Q CD O > CD J3 CD CO C CD a « CO 53 cd 2 bfl to Q £ CD a CD CD 5h b<) cd CD CD S-. bD CD Q o CN CO co p TJ CD -4-> u o o a^ S j- o^ -Q .5 "O -3 -i-i CD o u CO CO CD 'ft g CO « 2 Oj CD g a o co o 0) -M o ® o CD CD | a p m "" +-> o "S a § 3 a a o ■8 5 « I a§ « CO H (J; ■« a 3 .5 » o a 0) CD cd a o 5-. o £ « CO CO 2 o H PRELIMINARY STUDIES OF CAR DEBT 37 Conclusions Most of the errors found in this study were in interviews in which the respondent did not seem to have discussed at all the loan under scrutiny. About one interview in three fell in this group. The results of the experimental manipulation were not conclusive. They do suggest that interviewers obtain more accurate information where they know in advance the details of the financial information. This result is not statistically on firm ground but the difference is in the predicted direction, and the logical presumption in favor of the hypothesis is so strong that we cannot reject it. It is perhaps more remarkable that in 16 interviews where the interviewer knew exactly what she was looking for, she was not able to elicit any men- tion of the loan under study by the respondent. The data also seem to indicate that short questionnaires which focus on a particular topic are more successful than long question- naires. This result, however, can be regarded as no more than sug- gestive if the Chicago investigation is considered in isolation. As has just been indicated, the data cast doubt on the ability of interviewers to tell whether or not they are obtaining complete and accurate information. The evidence also suggests rather strongly a difference in ac- curacy of report between primary and secondary loans. When com- bined with the results for the 25 exploratory interviews, the results in this section tend to support the view that it is possible to obtain data about primary loans on cars with a high degree of accuracy in a survey which concentrates on automobile financing. More difficulty is to be expected in surveys concerned with secondary loans. The fact that it was impossible to distinguish accurately between these two types of loans on the basis of the official records was a serious handicap in the analysis of the data. The difficulties found in using records in county courthouses in Chicago thus contrast with the much more favorable experience in three counties in Michigan. Certainly, any future studies of car debt should be designed to avoid the type of difficulty which was encountered in Chicago. A Special Check Using Data from the 1956 Survey of Consumer Finances Purposes of the Study and Description of Procedures In conducting special interviews for a study such as this one it is difficult to conceal from the interviewers that the study is an in- vestigation of a special sample of people about whom certain infor- mation is known in advance. Interviewers are certain to wonder from what list the sample of names was drawn, and they are likely to 38 AN INVESTIGATION OF RESPONSE ERROR notice that everybody in the sample owns a late model car and reports owing money on it. To reproduce exactly the normal interviewing situation is impossible in such a situation. But the questions remain: "Do interviewers in a study such as the present one behave different- ly from the way in which they behave in a regular survey?" and "Are the errors in such a survey different from those found here?" An attempt has been made to answer these questions by obtaining data from state title offices in those states which maintain such records and were willing to cooperate. Names of respondents who bought new cars in one of the annual Surveys of Consumer Finances were sent to the motor vehicle departments of the states involved, asking for information as to whether there was a lien and as to the amount of the lien. A total of 33 names were found for whom the state involved reported a lien on a purchase of a new car. It is ap- propriate to regard this group of 33 interviews as if they had been selected originally from the records of the several states and then interviewers had visited them. The data which resulted may be com- pared with the data in the exploratory interviews and with the data from the special field experiment in Chicago. The 33 interviews with respondents in the Survey of Consumer Finances represent people who were much more widely scattered geographically, of course, than the respondents in the other studies reported in this chapter. Measures of Error The results of this special check are reported in Table 13. As in the studies of car debt reported earlier in this chapter, the error is the result almost entirely of interviews in which the respondent reported no debt. Of the 33 respondents, 25 reported to the inter- viewer that they had incurred a debt on their car at the time of purchase while 8, or 24 percent, failed to mention such a debt. These figures alone would tend to produce an underreporting of mean car debt for the group as a whole of 24 percent. The mean debt for the sample as a whole from the state records was $1,516, while the mean debt from "the interviewers' reports was $1,059. Thus, the respon- dents' reports represent an understatement of 30 percent. The difference between 24 percent and 30 percent results from a small average understatement of debt in those 25 interviews in which the respondent did report the existence of the debt on the car. Conclusions How do the interviews from the 1956 Survey of Consumer Finances compare with the exploratory interviews and the interviews taken in the field experiment in Chicago? The proper comparison refers to the original loan on the car incurred at the time of purchase, since PRELIMINARY STUDIES OF CAR DEBT 39 Table 13. Relation Between Debt According to State Records and Respondents' Reports (in dollars) in the 1956 Survey of Consumer Finances Respondents' report (1) State records (2) Deviation (1-2) (3) Group I: Reported no debt 2,508 - 2,508 2,250 - 2,250 2,190 - 2,190 1,353 - 1,353 937 - 937 850 850 829 829 521 - 521 Total 11,438 -11,438 Mean 1,428 - 1,428 Group II: Reported debt 3,069 3,174 105 2,595 2,509 86 1,900 2,284 - 384 1,602 2,225 - 623 1,520 2,160 640 2,253 2,114 139 1,656 2,102 446 1,960 1,960 1,830 1,893 63 1,716 1,782 66 1,623 1,623 1,524 1,576 52 1,485 1,485 1,400 1,471 71 1,404 1,404 1,203 1,203 690 1,120 430 1,085 1,085 1,064 1,064 882 882 600 865 265 726 708 18 414 694 280 438 662 224 312 559 247 Total 34,951 38,604 - 3,653 Mean 1,398 1,544 146 Groups I and II combined Total 34,951 Mean 1,059 50,042 1,516 15,091 457 40 AN INVESTIGATION OF RESPONSE ERROR it is only this loan which was covered systematically in all three studies. It will be recalled that in the preliminary interviews in Michigan only one respondent out of 25 failed to report his car debt. Of the respondents in Chicago, there was not one who denied all car debt out of the 92. Yet of the respondents in the 1956 Survey of Consumer Finances, 24 percent failed to report the original debt in- curred on their car when they purchased it. What makes the dif- ference? There are several possible explanations, and it is not possible to rule out the simple explanation of chance fluctuation. After all, the numbers of interviews involved are small. Nevertheless, there are additional data to support some of these results. There is other evidence, for example, that car debt in the Surveys of Consumer Finances is understated by an amount not very different from that found in the special check on the 33 interviews taken in 1956. Why do the results in the other studies seem to be more satisfactory than those in the Surveys of Consumer Finances? One possibility is that the respondents in the special studies were in some way unusual. It is known that the respondents in Chicago included a large group who had incurred not only primary but also secondary debts on their cars. The interpretation already has been suggested that these people may be willing to reveal the primary debts while tending to conceal the secondary loans. It is possible that they are more willing to reveal the primary debt than the population in general. This explanation, however, does not apply to those respondents in Chicago for whom there were only the primary loans. This group represents much less than the full sample, and, indeed, it is not possible to tell exactly how many of the respondents fall in the cate- gory of those who had incurred only a primary loan. They also may be more willing to reveal the primary debt than the population in general. Another possible explanation could be made in terms of some peculiarity of the interviewer involved. A favorable interpretation would be that a particular interviewer was unusually skillful. A more skeptical way of looking at the matter would be to suspect that the interviewer in the exploratory interviews made some special use of her detailed knowledge of the individual loans in such a way as to bias the results of the inquiry. This hypothesis, however, will not explain the results in Chicago. In that city four interviewers were used and they took approximately equal numbers of interviewers. The most probable explanation of the results seems to be that the crucial difference between the Survey of Consumer Finances and the special interviews was that in the latter the interviewer was able to focus her attention upon a single topic, car debt. This interpreta- tion suggests a general proposition: when an interviewer has a clear PRELIMINARY STUDIES OF CAR DEBT 41 grasp of what is wanted in an interview and can focus his attention upon it, he tends to be successful. According to this view, it is important both that the interviewer have a grasp of what is wanted and that he spend time enough on that topic to cover it adequately. He must spend enough time with the respondent to be certain that the respondent understands what the interviewer wants and that he gives a satisfactory answer. When, on the other hand, the questionnaire covers a large number of topics of equal importance, as is the case in the Surveys of Consumer Finances, the interviewer tends to accept the information with less of an attempt to make sure that it is correct. He does not keep the respondent talking about the one topic. He relies on the questionnaire as it is written to do the job. The results may not justify this confidence in the questionnaire. It should be emphasized that this interpretation is tentative. If it is correct, it has major implications for the design of surveys. It implies that the researcher has a choice between a large number of items of information, each with a relatively low level of accuracy, and a small number of items of information, each with a relatively high level of accuracy. IV. INTERVIEWS IN A LABORATORY At this stage in the progress of the research two things were clear to the investigators. In the first place, it was evident that substantial errors of response occurred in many interviews. The evidence from the studies of car debt and from the checks of survey data against outside aggregates made it evident that there was a serious problem of response error. In the second place, it became apparent that what took place during actual interviews was crucial for the research. The investigators felt the need to make an intensive study of a number of individual interviews. They felt that it was important to scrutinize as carefully and as sensitively as possible what took place in the interview situation. Purpose of the Study and Description of Procedures It was with this same general purpose of better observation in mind that the attempt had been made in the exploratory interviews to record on tape actual interviews in the field. As has already been noted, this experiment was less than a perfect success from a technical point of view. The tape recorders did not work well. It seemed probable that further attention to the mechanisms employed could remove this difficulty. The investigators also felt that many cues would be lost if they could not see the interview as well as hear it. It was decided that the most satisfactory arrangement would be to have the interviews take place in a psychological laboratory. For this purpose the investigators were fortunate enough to obtain access to the laboratories maintained by the Research Center for Group Dynamics. These laboratories are so arranged that it was possible for several observers to sit behind a one-way glass screen in a separate observation room in order to observe the interaction be- tween interviewer and respondent. Microphones in the ceiling of the interviewing room attached to an audio system carry sound to the observation room, but the observers there could neither be heard nor seen by the interviewer and the respondent. A tape recorder was placed in the interviewing room so that it was not in the respondent's view, although the respondent was aware of its presence, and the microphone for the tape recorder lay in full view on the interviewing table. The respondents also were informed of the presence of the observers in the next room. These arrangements made possible the most careful observation of the interview itself and thus led to the development of new hy- potheses about the causes and possible cures for response error. 43 44 AN INVESTIGATION OF RESPONSE ERROR S election of Respondents In order to preserve the absolute anonymity of the respondents throughout the entire procedure, respondents unknown to the inter- viewer or the observers were obtained through a contact man living in a community near Ann Arbor. The persons selected were married men with children, employed, with incomes in or near the highest tenth of the income distribution. Also, if possible, the re- spondents were persons with fairly substantial liquid asset holdings. The respondents were told by the contact man that the interview was being conducted for the purpose of testing interviewing techniques and that the information to be asked would be financial. They were asked not to look up any of their records. In the initial interviews it was found that one respondent did not follow these instructions, and in the later interviews respondents were not told in advance the sub- ject matter of the interviews. The respondents were paid $10 if they appeared for the interview and also returned a check sheet after- wards. This payment was arranged through the contact man in order to insure the continued anonymity of the respondents. Post-Mortem Interviews After the conclusion of the regular interview one of the observers entered the interviewing room, was introduced to the respondent, and then interviewed him about the main interview. No fixed question- naire was prepared in advance for the post-mortem interview, but suggestions were made by the various observers during the course of the interview itself as to the questions that might be raised during the post-mortem. Although the post-mortem interview was relatively unstructured, the interviewers did gradually work out a pattern of questions to be asked. A typical series of questions asked in the post-mortem inter- view was as follows: (1) How good a picture did we get, for example, of your financial situation, of your buying plans, of what you expect to do over the. next year? Were there some things we missed? (2) What did you expect that you were going to do when you got here tonight? (3) Were there any of the questions that you were asked that bothered you or you wish somebody hadn't asked you? (4) Was there anywhere in the interview where you thought the interviewer was digging too hard? (5) You appeared to give information that was quite precise. I wondered if you had to bet $25 how accurate would you ex- pect to be on this figure? (6) I was wondering about how well we would have done, for example, if we had come to your house and talked to your wife about these things? INTERVIEWS IN A LABORATORY 45 Special Check After the Interview During the post-mortem interview the respondents were asked to take home the completed interview schedule and to check their re- call statements on liquid asset holdings against their actual records. They were then asked to return the corrected schedules to the Sur- vey Research Center by mail. Development of Hypotheses as the Laboratory Interviews Progressed In the first stage of the work in the laboratory, the questionnaire used was a standard instrument: it was the schedule used in the most recent Survey of Consumer Finances, the 1958 Survey. The inter- viewers were regular members of the field staff of the Center who had used that questionnaire in interviews taken in the normal course of their work a few months earlier. It was hoped in this way to re- produce in the laboratory as closely as possible the conditions of interviews in the field. Important differences, of course, remained between the field situation and the laboratory situation. In particular it was not possible to reproduce the crucial first stages in which the interviewer in the field must obtain the cooperation of the respondent. The respondents in the laboratory already had agreed to cooperate. What it was hoped would be reproduced was the later stages of the interview itself. How successful was this attempt? The observers felt that there was a tendency for both the respondents and the interviewers to be somewhat nervous and ill-at-ease during the session with the regular field interviewers. The post-mortem interviews typically were con- ducted in a more relaxed atmosphere. But the observers felt that the procedure did offer an opportunity to observe the flow of an interview. It was possible to watch as a professional interviewer used a fixed questionnaire and standard probing techniques to obtain data from a respondent. The advantages of a fixed questionnaire are very great. The use of a standardized instrument for data collection makes a major con- tribution to the reproducibility of the research process. The im- portance of conducting research in such a manner that it can be replicated hardly needs emphasis. Yet it was the feeling of the observers in these first interviews that the fixed questionnaire sometimes got in the way of the communi- cation between interviewer and respondent. There were situations in which the rigidity of the Survey of Consumer Finances questionnaire made it more difficult rather than easier for the respondent to give the desired information. The interviewer could not stick to the questionnaire if she wanted to obtain for the analyst a comprehensible picture of the actual financial situation. 46 AN INVESTIGATION OF RESPONSE ERROR The importance of the interviewer in the interviewing situation was noted by the observers. Such characteristics as the inter- viewer's persistence in obtaining a complete answer to the entire question, the confidence with which the interviewer approached particular questions (specifically in one case, the series on liquid assets), the interviewer's ability to enlist the aid of the respondent in helping straighten out difficult financial questions, and the ability to use probes with ability and informality all were noted to play an im- portant role in the progress of the interview. From the point of view of the analyst it is frequently necessary to obtain answers to several questions which may seem similar to the respondent. In some instances respondents appeared bored by what they seemed to feel were repetitive questions. It was also noted that the time spent by the interviewer in taking notes acted as an interruption to the flow of the interview. Re- spondents, of course, realized what was going on, and in at least one instance a respondent reacted to this situation by dictating some of his answers to the interviewer as one dictates to a secretary. After three interviews had been taken using the standard techniques brought in from the field, and after having subjected them to extreme- ly intensive scrutiny, the investigators felt that this particular line of inquiry had reached the point of diminishing returns. The time had come to experiment with alternative techniques of interviewing in the laboratory. Three experimental approaches were tried out. Of these, the first is the most interesting from the point of view of its relation to further research for it led directly into the field experiment con- ducted later in the year. The technique used involved the opposite extreme from the use of a fixed questionnaire. It was developed in part in response to the feeling that the standardized questionnaire had worked only imperfectly in the preliminary interviews just described. In the first attempts at this new technique the interviewer approached the respondent without any fixed questionnaire or answer form. The interviewer did have in mind in some detail the objective of obtaining a report of the respondent's financial affairs similar to that obtained in the course pf the usual interviews in the Surveys of Consumer Finances. But the interviewer entered the interview situation with no fixed ideas as to how the information was to be collected. The underlying idea was to bring the respondent to describe his own financial affairs in his own way. Behind this approach lay the general proposition that people are interested in their own financial affairs. Hence, it was argued that in an interview situation in which anonymity was guaranteed people should find it a pleasant and rewarding experience to discuss their financial affairs. It was hoped to interest the respondent by letting him take his own way through this general area. The goal was a high INTERVIEWS IN A LABORATORY 47 degree of involvement of the respondent in the research process. It was anticipated that under these circumstances, first, he would talk more freely; second, he would remember his affairs more completely and more accurately; and third, he would give the in- vestigators a more complete understanding of how he operated financially. In the laboratory situation the interviews conducted using this approach seemed to work out very well. After some consideration it was decided that this version should be given a trial in a field situation. The attempt was made, with the results which will be described in Chapter V. A second technique which was tried out in the laboratory repre- sents a kind of intermediate step between the structured interview as used in the Survey of Consumer Finances and the completely un- structured approach just described. This technique, like the standard questionnaire in the Survey of Consumer Finances, made use of fixed questions. An attempt was made, however, to organize the fixed questions around the interests and experience of the respondent more closely than is done in the Consumer Finances questionnaire. The new questionnaire, after some preliminaries about the com- position of the household and the occupation of the respondent, approached the topic of his finances by stating, "We're interested in how people handle their money." The respondents were asked how often they got paid, whether they had any other money coming in, who received it, in what form the money was paid to them, and what they did with it. Thus, people told the interviewer whether they de- posited their checks in a savings institution or cashed them else- where. The next sequence concerned who in the family paid the bills, how these bills were handled, what kind of budgeting system the family had, and how well it functioned. From this the question- naire led into a discussion of the regular payments which people make. The next sequence concerned unusual large expenses in the last year or so. The next major section concerned the different bank accounts the family had, what they were used for, and how the family handled its different accounts. For example, respondents were asked questions about whether they kept minimum amounts in their checking accounts, and if so, what these minima might be. These sequences led up to specific questions about total liquid asset holdings. This type of interview seemed also to work successfully in the laboratory situation. It was not chosen for experimentation in the field primarily because it represented less of a departure from the normal interviewing procedure than the previously described techni- que. It was felt that more could be learned from the more radically different questionnaire. The approach, however, influenced the de- sign of the questionnaire used in the study of cash borrowers dis- cussed in Chapter VI. 48 AN INVESTIGATION OF RESPONSE ERROR The third experimental questionnaire tried in the laboratory also represented a more moderate departure from the standard proce- dure. This questionnaire followed the Survey of Consumer Finances approach of first asking for the composition of the household and the economic attitudes of the family. These topics were followed by questions about the occupation and income of the family members taken directly from the Survey of Consumer Finances but asked earlier in the interview than is the standard procedure in that survey. The principal unique feature of this questionnaire was that the next sequence emphasized changes during the previous year. Respondents were not asked to tell the interviewer about the amount in their savings accounts or the amount of their holdings of other forms of assets. They were asked a series of questions about the changes in their balances and the changes in their other assets and the large purchases which they made during the year. They were also asked general questions about how things had worked out for them during the year and any unusual experiences or financial problems that they might have had. This questionnaire also seemed to be successful in the laboratory situation. It was decided to use the first unstructured questionnaire rather than this version in the field experiment for the reason noted earlier: that more might be learned from the use of a dramatically different form of interview than from a relatively moderate change in the research procedure. As it turned out (this will be discussed in more detail in the next chapter) the unstructured interview was not a success in the field situation. This lack of success raises a question about what might have happened if one of the other versions had been developed systematically and tried out on a large scale in a field experiment. The answer, essentially, is that it is impossible to know what might have happened without actually conducting additonal field experi- ments. It would have been necessary to devote an entire experiment to each version of the questionnaire since the hypothesis concerned the result of the development of questionnaires with basically different structures. One cannot try two basic designs of question- naire in the same interview. The investigators possessed only limited resources in terms of the limited possible number of field experiments. They have become increasingly aware of the greater efficiency of developing and testing theory as to the causes of re- sponse error rather than of testing one technique of data collec- tion after another without being able to generalize from one proce- dure to the next. The work described in Chapter VIII and in Part Four reflects this increasing interest in the problem of how to develop a theory of response error. From this point of view two observations may be made about the experience in the laboratory. The first observation is that INTERVIEWS IN A LABORATORY 49 everything seemed to work in the laboratory. All of the varied ap- proaches seemed to bring out the data required. This generalization rests on very few cases and it is possible that respondents seemed more cooperative than they really were, but it does fit the observers' sense of what they saw. If this interpretation is correct, the question arises, "Why did things go better in the laboratory than in the field?" The significant difference may be that in the laboratory the respondents came already committed to help. The second observation is that there was one respondent who cooperated reasonably well during the interview but became quite disturbed about it in retrospect, as the investigators learned through the contact man. The respondent refused to fill in the form calling for a special check on his liquid asset holdings. He was very concerned about what use might be made of the interview. The reason, which became apparent, was that he did not wish some of his transactions to become known for fear that the information might injure him if it came to certain persons. In this individual the de- sire to conceal his financial affairs was a powerful motivating force. The importance of the motivation of respondents will be discussed at more length in Chapter DC. The investigators emerged from this experience with the feeling that careful observation in a laboratory situation could be useful but that after close scrutiny of a dozen or so interviews they had little more to learn from it. After a total of 15 interviews they turned to the planning of field experiments, with the feeling that the laboratory experience had been of assistance and might be useful in other projects, especially in the development of new questionnaires. PART THREE THREE FIELD EXPERIMENTS UHWERSITY OF llUNW* U LIBRARY V. SAVINGS ACCOUNTS: THE FIRST FIELD EXPERIMENT In the fall of 1958 the Survey Research Center undertook the first of a series of three field experiments. These studies are the topics of the next three chapters. After completion of the personal inter- views in two of the studies, respondents were sent a mail question- naire. This reinterview will be reported in Chapter VIII. The first field experiments involved interviews with a group of respondents known to be owners of large savings accounts. It was made possible by the cooperation of a group of financial institutions on the East Coast. Purposes of the Study and Description of Procedures Purposes This study, like other field experiments in this series, had three main purposes: (1) to measure the accuracy of information which people give to interviewers about their finances, (2) to test the dif- ference in response error resulting from the use of different tech- niques of interviewing, and (3) to measure the correlation between the observed response error and other variables. Such correlations, it was hoped, might lead to new and improved interviewing techniques, as well as to indications of the accuracy of comparisons of different sub-groups in the population using existing techniques. The Need for Anonymity When this project was first under discussion, it was agreed by all concerned that a prerequisite for such an investigation was care- ful preservation of the anonymity of the individual respondents. Financial institutions quite properly regard information about the financial affairs of their depositors or clients as confidential. How was it possible, then, to interview a sample of depositors while maintaining complete anonymity for them? The Survey Research Center and other organizations which con- duct sample surveys are basically concerned with protecting the anonymity of their respondents. As a matter of routine every effort is made to make certain that statements made by respondents to interviewers never can be associated with the actual persons who made them. The reports from sample surveys customarily appear in the form of statistical tabulations. It is obviously impossible to identify any individual in such a tabulation. From time to time in the report of a survey a quotation may appear, but such quotations are always attributed to an individual who is described only in the most general terms, as for example, "A truck driver, age between 35 and 44, living in a large city in the northeastern part of the United 53 54 AN INVESTIGATION OF RESPONSE ERROR States." Indeed, the very existence of a sample survey organization depends upon the ability of its interviewers to obtain information from respondents, and this in turn depends in large part upon the interviewers' ability to assure their respondents that the inter- views will be kept confidential. Thus, the problem of insuring the anonymity and confidentiality of financial records about individuals was not actually so difficult to solve as it might appear at first glance. Essentially, what was required was to develop a procedure for transferring information from one confidential file to another. The method actually worked out involved the following steps in the field experiment described in this chapter: (1) The participating savings institutions selected samples of the owners of savings accounts by a method to be described in the following section. (2) The institutions transmitted the names and addresses thus selected to a central association. (3) The names and addresses thus chosen were mixed together for the several cooperating institutions and then transmitted by the association to the Survey Research Center. (4) Interviews were taken with a large fraction of the persons whose names had been transmitted to the Center. The interviewers were not informed as to the origin of the list of names. Many of the interviewers guessed that the list had come from financial institu- tions in some way, but this was only a guess on their part. (5) The Center in turn submitted back through the association to the cooperating institutions a list of the names and addresses of those persons actually interviewed. This list showed the balance in the account in question on certain dates as reported to the interviewer. The Center maintained records arranged by code numbers. (6) The savings institutions sent reports back to the Center, arranged by code numbers. These reports contained indications of the accuracy of the reports. In this way it proved possible to obtain measures of the accuracy of report in the interviews while at the same time giving full pro- tection to the individuals who cooperated in the study. Sample Selection The sample design was intended to yield a sample of owners of large savings accounts selected in accordance with strict probability procedures. The details of the sample selection varied somewhat from one institution to another, depending on differences in the way the files of the institutions were organized, but the essentials of the procedure were as follows. Savings institutions customarily identify savings accounts by SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 55 number. Both the smallest number ever assigned to a savings account and the largest are known. The basic method of sample selection is to select account numbers within this known range. Numbers which refer to accounts which have been closed automatically fall out of the sample. In addition to closed accounts the following types of accounts were dropped out of the sample: (1) Christmas savings accounts, (2) accounts of minors, (3) accounts held in trust, (4) organization accounts, (5) accounts the owners of which live outside the area, and (6) accounts with a current balance at the time of selection of less than $1,000. Thus, the selected accounts consisted of individual or joint accounts owned by one or two adults who lived in the metropolitan area and whose balance was $1,000 or more. This procedure was more exacting and, therefore, more difficult for the cooperating financial institutions to carry out than some more casual method of selecting the sample. It had the merit, however, that the final sample was a probability sample of all depositors with the specified characteristics of the financial institutions in question. Interviewing Procedures As has been mentioned earlier, this field experiment was designed to test two interviewing procedures. These procedures require de- tailed description to be understood. The first was a comparison of structured and unstructured interviews. In one -half of the inter- views the questionnaire used was a modified form of the question- naire used in the Survey of Consumer Finances. This is a highly structured questionnaire, a copy of which appears in Appendix B. The interviewer first fills out a form describing the composition of the dwelling unit, that is, what individuals live there and what their relation is to the head of the dwelling unit. The interviewer deter- mines whether any of these individuals constitute a separate spend- ing unit. The next sequence consists of a page of questions of the fixed-question, free -answer type concerning the individual's general economic attitudes. These questions concern his evaluation of his own personal financial situation and of the general economic situation in the country as a whole. Another page of questions is devoted to the housing arrangements of the unit. Automobile ownership and pur- chases and purchases of other major durables are then covered in some detail. There is a section on personal debt. Careful questions cover in detail the income of the family and the occupations of the various employed members. Finally, the interviewer asks a series of questions covering each of the liquid assets owned by the members of the family, with separate questions about savings bonds, checking accounts, saving accounts in banks, and other types of savings accounts. Following this standard series of questions a special series 56 AN INVESTIGATION OF RESPONSE ERROR was introduced intended to shed some light on problems of the measurement of response error. The results obtained from these questions will be discussed later. In addition, detailed demographic information about the unit interviewed was obtained at the conclusion of the interview. The other half of the respondents were interviewed with an un- structured approach. The interviewer was given a form consisting of four printed pages on which she was asked to record certain specific information. This information included the same amount of detail about liquid asset holdings which was asked for in the structured questionnaire. Otherwise, the information asked for was much less specifically described. For example, one part of the form provided a place to record large expenditures in the last twelve months with room for four items. With regard to income, the interviewer was asked to record the income of the husband, other income of the family, and total family income. A copy of this form also appears in Appendix B. The instructions to the interviewers using the unstructured ap- proach were to ask the respondents to discuss with them their fi- nancial situation. No set questions needed to be asked. What topics were to be covered in detail, the manner in which they were to be covered, and the sequence in which they were to be covered were left up to the respondent. The interviewer's role in the situation was seen essentially as one of encouraging the respondent to talk about how his family handled its financial problems. It was hoped by the in- vestigators that this approach would lead to greater respondent in- terest in the survey, and hence, to greater effort by respondents to explain their financial situations in an intelligible and accurate man- ner, than in the structured interview. The following description of the objectives of this approach was written down by the investigators prior to the interviews themselves: "We may be successful if we indicate to the respondent the general topic we want him to cover and let him take his own way through the facts. Thus, we should ask general, open questions. We should give the respondent time, and let him ramble a bit if he wants to. In a word, we should try for factual data the tech- niques used for attitudes in many surveys. There is some evidence from the psychological studies of testimony which suggests that this approach may yield more accurate data than a succession of specific, detailed questions. "The respondent-oriented approach also has implications for the types of questions asked. Questions, according to this view, should be adapted to the respondent's situation as he thinks about it. To develop this type of question we need to know how respon- dents classify their bank accounts in their own minds. It would be consistent with this approach to let the respondent indicate who in the family is the person best able to give us information on these SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 57 topics. We may also ask about the purposes for which the dif- ferent accounts are kept as the respondent perceives these pur- poses. Knowing more about purposes, we may be able to ask more intelligently about transactions." The second manipulation in this field experiment took place a number of months after the original interviews and involved the reinterviewing of the original respondents. The first wave took place in the fall of 1958, the second, in the spring of 1959. The purposes of this manipulation were described early in 1959 as follows: (a) In some interviews rapport is better the second time. The respondent is convinced of the good faith of the interviewer by such things as receiving in the mail a reprint of an article based on the first survey. (b) The facts are often complex. For example, there may be several accounts, some joint, involving people who are pre- sent in the family at one date but not the other. (c) People may forget information. Frequently their reports are inconsistent because they forget part of the facts at one time and remember them at another. This interpretation leads to the proposal that we should ask for the same information in several ways and ask the respondent to reconcile the discrepancies. For example, we can ask for the same data in three ways as follows: (a) In the first stage of the interview we ask for correct bank account at date (1). (b) Six months or a year later we ask for memory of assets at date (1), for assets at date (2), and for the change. (c) At date (2) we ask for larger transactions involving the bank account between (1) and (2). It was in order to test the usefulness of reinterview in gathering data about the situation at the time of the first interview that the attempt was made to reinterview all of the respondents from the fall survey in the late spring of the next year. The procedure used was that described earlier. It yielded for most interviews three reports of the fall balance: the original report in the fall, a report of that same balance from memory in the spring, and, frequently, a revised report in the spring based on the respondent's resolution of any conflict between the two previous reports. Error of Non-Response The response disposition for the 168 selected addresses at which interviews were attempted is shown in Table 14. The 168 addresses exclude those addresses for which the address reported by the 58 AN INVESTIGATION OF RESPONSE ERROR CD N • rH Ul o 00 -4-> in Oi 1-1 i—l o CO • r-i H-> ■*-> g rt 3 O tf rH • l-H • r-l <+H r- -M O o •1-1 -t-> 3 o o •r-4 •rH CO a CO -t-> •rH rH Q • rH CO CD • rH r CD Ph rrt o 3 -*-» rH CD O Ph CD o £3 rt H rH O 8.1 in iTi o in o CO CO CO CM tH o i—i ■^ U3 €*9- oo in Oi 1 ■ — i O Oi l-H o as O Oi CO c- i— 1 CO o CO I— 1 c- CM CM o CO d i-H fe CO TT fl •rH CD i O CI O Oi O Oi O Oi in ID C- 00 o Oi r— 1 in TT i-H CM o CM d 1—1 rQ CM CM l-H €»-€«- rt 3 H-> o 1 O Oi O Oi ■^ to l-H in o t- rH J-J 3 < s in in O in o CO CO CM i-H o CO o i-H 1—1 rt „ £ CD •rH c r* •rH rH CD Eh -a o 5 •l-H " s +» CO » 2 .rH CO H-> r- CO 3 rS Eh ^ O CO o o <+H CD — ' £ rt CO CD -r-> Qh°> CO " • rH o CO CD > Eh CD +J ■ — i a CO u H Eh CD •rH 1 «rH CD Ph rC H-> o a fn -4-> o 1 SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 59 savings institution was no longer correct, usually because the de- positor had moved. Preliminary weeding out of such addresses was accomplished by eliminating from the sample all cases in which the name and address did not check against the most recent telephone book. A few correct addresses of people with no telephones were eliminated, no doubt, in this process, but this number was probably small. Nobody with less than $1,000 in his account fell in the sample, and it is unlikely that many people with this much money have no telephone. Some further elimination of addresses took place in the field when interviewers found people had moved recently. The final total, as shown, of addresses in the sample at which interviews were attempted was 168. At 65 percent of these addresses interviews were completed. This response rate is substantially lower than the national average of about 85 percent on the Surveys of Consumer Finances. It is also lower than the local response rate in the same area with ap- proximately the same interviewers for the Survey of Consumer Finances, which was 84 percent on the 1957 Survey and 75 percent on the 1958 Survey. Is there a relation between the size of the actual balance in the account and the response rate? Table 14 shows no evidence of such a relationship. The number of observations in the table is not large, but, as far as the data go, the interview rate does not vary systematically with size of account. Table 15 shows the relation between the mean balance in the account and the response disposition. As might be expected in the light of the data in the previous table, there is no evidence of variation (beyond that attributable to chance) in the means for dif- ferent groups of accounts classified by response disposition. It should be kept in mind that all respondents in this sample had a balance of $1,000 or more on the date when the sample was selected. It may be that people with balances at this level are difficult to interview as a group compared with the general population, but that there is no tendency toward further difficulty as the balance goes over that level. This topic will be discussed further in Chapter VII in connection with the interviews taken during the third field ex- periment. There is definite evidence, however, that the response rate for the unstructured interviews was lower than that for the structured interviews. As shown in Table 16, the response rate was 77 percent for the structured interviews and only 54 percent for the unstructured or experimental interviews. Even with the size of sample involved, this difference is significant at the 95 percent level of confidence. This result was a surprise to the investigators. How should it be interpreted? It must be kept in mind that the objectives of the two approaches were similar in terms of the type of data to be gathered. 60 AN INVESTIGATION OF RESPONSE ERROR oo cd rrt c fe o c < c o c • rH •1-1 H-> ■ rH CI) O a CQ ■rH Q o d O) uu c o cd a cj CO CD < K c a £1 cd H rH CD rQ Oi Oi Tt< CM CO CO PI a O in CO CM CD o i— i i-H •rH 3 H-» 55 | CQ s§ CO CO q c- CD CD CM CO 1— 1 < cd CD in CD ■* O CO CO 00 in So o CM rH T-t oo rH H-> rH OS § ■^ ■^ ■^ "^ co •^ OS t— i CD ffr O CD €r> r— i .—4 hfl ° S to o o I— I u O cd Em O rH CD rQ a 5 OS OS ■^ CM CO CO CD a o O r- ( in CO CM CO i-H CJ cd •rH H-> r? Ch ft ^cd Id CO cd o 00 c- CO CO CM CO < 1— 1 00 in Oi CO CO r— t cd CO in in r-t CO c- 5 § CO CO CO CO CD co o €r> cd cd .5 o cri o rH 0) rQ CO in CM o CO i-H ■+-> a O in CO CM CD Sh CTs cd i-\ rH CD 0» CD g 5 CO Ch O a 1"S tH cd CO CM in CM CO 00 3 3 o CD CM m i— 1 CO c- o o CD CO rH i— i in oo -^ S CO CO CO CM CD CO €r> 'R CD •tH c > -rH jL, r- * O fl • rH .i-H 4-> •FH jj S 00 CO ««H •i-i 73 Ch o CD jjj CO """^ £ 00 Q in CO CD •rH CD a o rC CO -4-> 5*°* CO > rt c «3 rH Sh H-> -3 CD CD CO cd u o « J. CD H-> p CD o ■rH > •rH ■HH CD H-> O H-> o cd Id u Ch rH o rH CD H-> o j3 h-H £ o H-> cd o C o CD m% rH CD OS fH CD cd" 3 €*3" -rH VrH Ch r* CJ 3 H cd > o r^ » « H-> ^ CD >> .3 2 cd * T3 CD C bD •rH CO Ch 1 a a rd a •rH •" CD ■d'fl CD CD CO cd CD rQ a a •rH a rH T3 ^3 "c, CD T3 r— 1 o • r-t a X! H-> CD CD >pH rC Jh o .So CO CO CD r3 • rH CD XX H-> cd •rH H-> cd >» a cd > 73 73 C rrt O ™ CD CD H-> cd 73 73 CO CO =1 r-H r3 U -h r* CJ O O j. c c td ,rH T3 •rH •rH OX) rr. CD CD CD • rH CO T3 rH rH H-> CD ® cd E3 i — c CJ CO CD CD > 3 d CD c •rH U CO CQ •rH CD H-> H-> CD CD CO «4H Ch 13 3 rC CD T3 o O o *J rC CJ o CJ O H-> O "^ rH cd CJ cd o cd rH O 5 rH ,_f H-> CD CD r-H cd rH CD bD rH bD u o U cd cd r« ". Id CD »#H i— < rH CQ -H CO c o • rH H-> •rH < rn^o' rnO 3 CO Ch 5 ««- cd o CJ CD o o rQ O Sh •rH H-> •rH , -M rH ^ •* c cd CD a CD H-J Qh o a o rt rH CD H-J a o o g o cd rH C O r* o CO CO < o T— 1 CO o co « <3 co 1— ( cd - CO u rH rH €r> H-> ">li £ o O rH ? •rH CO T3 T3 tJ CD g a g o o C ro ^ „ T3 rl" o y £ 0* Jh o o» O CJ CJ v± •rH SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 61 Table 16. Response Disposition for First Interview (Fall, 1958) in Relation to the Experimental Manipulation Type of interview in fall Response disposition Structured Unstructured for first interview Percent Number Percent Number Interviews Non-interviews 77 29 63 19 54 46 46 40 Total 100 82 100 86 It must also be kept in mind that the decision of the prospective respondent to grant an interview or not to grant one is made before the interview proper begins. Respondents rarely break off an inter- view once it is launched. The difference, then, must be in the ap- proach of the interviewers in the early stages of their relation to the respondents. The interviewers did not feel that the unstructured questionnaire was a success. They reported feeling more confident of themselves with the structured questionnaire. Evidently this difference in con- fidence was somehow communicated to the respondents and led to failure to complete the interview. One view is that the personal financial area, especially with regard to savings accounts, is extremely sensitive and that such topics just are not discussed with strangers under normal circum- stances. In order to effect discussion, the situation must be made unique: it must be non -personal — highly structured in specific ways (so that the respondent is not talking to another person but to an object ). One means of accomplishing this is to use a highly structured questionnaire. Once the interviewer has had a few opportunities to use an unstructured questionnaire and becomes aware of its deleterious effects, her self-confidence is destroyed. The con- sequent lack of self confidence is communicated to later prospec- tive respondents, resulting in unsuccessful approaches. An alternative view is that the difficulty is not so much in the nature of the unstructured approach as in the training of the inter- viewer. The interviewing staff on this project consisted of people with training and experience in working with structured question- naires such as that in the Survey of Consumer Finances. They were given training in working with the unstructured approach but not enough training to handle the situation. If they had started out with more self-confidence, they would have been more successful. A variant of this position is the view that the interviewers 62 AN INVESTIGATION OF RESPONSE ERROR should have been given either less training with unstructured inter- viewing, or much more training. With less training they might have asked structured questions but in an order which varied from one respondent to the next. This procedure might well be less threatening to the respondent than a non-directive approach. With more training-- probably a good deal more training --they might have been able to handle the anxieties which they aroused. As it was, they aroused the anxieties and did not handle them successfully; these anxieties were communicated from respondents to interviewers and then to prospec- tive respondents. It is possible that both of these views are correct. It may be true both that the topic is one not easily handled by an un- structured approach and that with training interviewers can use the unstructured approach with greater success. Since the effect of type of interview on response rate in the fall was so pronounced, the question arises of whether there were any carry-over effects from fall to spring. Of the people who were actually interviewed in the fall, were those who had gone through one type of interview more likely to consent to a reinterview than those who had gone through the other type of interview? Table 17 shows that there was in fact little or no carry-over of this sort. The re- sponse rate in the spring was little influenced by the type of interview in the fall. Response Error The results of the fall interviews in terms of response error are summarized in Table 18. The system of classification of types of Table 17. Response Disposition for Spring Reinterview in Relation to Type of Interview Used in Fall (Percentage distribution of interviews) Response disposition Type of interviews in fall for spring reinterview All interviews Structured Unstructured Reinterviews 73 76 70 Non-reinterviews 27 24 30 Refusal 17 14 20 Other 10 10 10 Total 100 100 100 Number of interviews 109 63 46 SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 63 response error used in this table requires an explanation. The classification was empirically derived. It was based upon the responses to the questionnaire itself and the limited information given to the investigators by the savings institutions. The three major classes into which people could be categorized were exclusive and made conceptual sense relative to the investigators' germinating theoretical views about response error: The respondent failed to report the account at all, he admitted the existence of savings but the relevant balance was not ascertained, or the respondent reported a balance for what appeared to be "our account." It was felt that the dynamics were different for each of the major categories, that the factors operative were common within categories but might be different between them. The subcategories were also empirically derived, but these, too, made conceptual sense. Within the "failed to report" category, there were two sub-classifications possible which described all of the respondents involved; the same thing was true for the "reported account, balance not ascertained" category. (See Table 18). The subcategories for those who reported a balance were selected for different but practical reasons. It might be argued that it would have been more appropriate to work with the percentage error rather than an absolute amount of error. If the investigators had been in a position to make a free choice, they probably would have used percentages. The choice, however, was not free. For most accounts the accuracy could be verified only within $1,000. It is possible to state that a certain report was correct within $1,000, while it is not known exactly what the percentage error is for a given report. Supplementary Tables 1 and 2, Appendix D, show the full details available to the investigators. For most purposes, how- ever, the information shown in Table 18 and the following tables and chart may be sufficient. The main results, then, were as follows. First, 24 percent of the respondents failed to mention the account in question. This type of error is serious since in an investigation with no special check on accuracy these respondents would be assumed to have no such account. (Most of these respondents did mention other savings accounts, as will be discussed in more detail later.) Some of these repondents, about 6 percent of the sample, seem to have failed to report an account which they owned jointly with some third party other than the respondent or his spouse. For example, the account may have been in the name of the wife and her mother. It is probable that in most cases this account was in fact the account of the mother for all practical purposes. In a sense, then, the daughter would have been quite right not to report the account. On the other hand, about three out of four of the missed accounts belonged to the respondent or his spouse or to the two jointly. Second, 19 percent of the respondents reported the existence of 64 AN INVESTIGATION OF RESPONSE ERROR p si .15 CD (1) o oo S LO £ OS ^H ?-( +-> CO •pH rt CD Eh c^ H-> O O O) (!) C- PS c o rt CD r3 CO CD £3 CX) lO OS rH i—( r— 1 rH c •pH cu •I-l > u CD c •rH CO a oo t- i-i -& o -^ t- in cm o CM tH rH CM CM ^ H CO OS OS CO T3 CD rH 3 o rH H-> CO cm os co in o in tf co rH f- CM iH rH rH CO rH rH CO l-HJ o CO CO CO 0) i— i .i-i CU H-> • i-l *ct< oo co os ^f inc-coc-'* CM rH tH i— I in i—l CO O o rH OS o rH -M o CD fn C •rH 2 rH 0) CD CO a a CO CD CD O c i—i Bi rQ CO LO OS 1—1 o Failed to report account Account owned by respondent or spouse or the two jointly Account owned entirely or in part by someone other than respon- dent and spouse Reported account, balance not ascertained Reported account but refused to state balance Reported a total in several accounts but would not break it down Reported balance for account Underreported by $1, 000 or more Overreported by $1, 000 or more Accurate within $1, 000 1* H-> O H CO g > u CD H-J a •1-t trH o u CD rQ a 2 be s •rH a ? o 0) CO o CD rQ H-> c o rH CD a o o SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 65 the account under study, but failed to report the balance. This type of error is serious, but not so serious as the first, since the in- vestigator knows that some data are missing and can make adjust- ments. In a few interviews, 5 percent of the total, the respondents reported a total in several accounts but did not break it down for the individual accounts. This type of response is difficult to handle in a validity check, since it is not possible to make more than a guess as to whether the total is correct. In an ordinary study, however, such a reply may be satisfactory. Ordinarily an investigator is not con- cerned with breaking down a total balance among several accounts. Third, 57 percent of the respondents reported the existence of the account and reported a balance for it. Of these, 16 percent underreported by $1,000 or more, 7 percent overreported by $1,000 or more, and 34 percent reported accurately within those limits. The results were, if anything, worse for the unstructured than for the structured or traditional questionnaire. It is interesting to note that this difference is in the same direction as that found for non- response, but smaller. It hardly needs emphasis that these results show the existence of a serious problem of response error even in the structured inter- view. The most optimistic view would be to accept as accurate those interviews in which the respondents failed to report accounts jointly owned with someone outside the spending unit and also those in which they reported a total but failed to break it down. Even this approach would leave 19 percent of the respondents to the structured interview failing to mention the account and 10 percent refusing to report the balance, in addition to 27 percent whose reports were in error by $1,000 or more. Table 19 shows the actual discrepancies in more detail. For all interviews together the mean actual balance was $3,810, as near as could be estimated from the approximate data made available to the investigators by the financial institutions. The mean reported balance was $2,105. (In preparing the latter estimate, the same mean balance was assigned to those for whom the balance was not ascer- tained as the average for the rest of the sample for whom the balance was not ascertained. Missed accounts, however, were entered with a report of zero.) Thus, an estimate of the aggregate holdings of the sample of 109 based on the interviews would have been 55 percent of the correct total. How successful in reducing this margin of error was the second manipulation, the use of a reinterview to correct the report of the balance in the fall? The results, which are summarized in Table 20, show a modest improvement. In this table the numbers shown are counts of interviews rather than percentages. As shown in the first column, of the 109 respondents in the fall, 80 were successfully reinterviewed, or 73 percent. Was there any relation between accuracy 66 AN INVESTIGATION OF RESPONSE ERROR Table 19. Actual Balances for Fall, 1958, Listed with Fall Reports (in dollars) in Relation to Response Error, with Means Respondent failed to Respondent reported account, but report account a the balance was not ascertained Account owned by respondent or Reported account but refused to spouse or the two jointly state balance Actual balance Actual balance 9,000+ 10,000 7,450 9,000+ 6,600 9,000 6,500 8,500 5,500 8,500 5,500 6,500 5,500 5,500 4,850 4,350 4,500 3,850 4,500 3,250 4,500 2,500 3,700 1,500 3,500 1,500 2,500 650 2,500 550 2,500 Mean = 5,010 2,500 1,550 1,500 1,200 Mean = 4,292 Account owned entirely or in Reported a total in several part by someone other than accounts but would not respondent and spouse break it down Actual balance Actual balance 9,500 5,900 6,500 5,800 5,000 3,350 3,150 3,900 1,850 3,300 1,500 1,050 1,500 Mean = 4,012 Mean = 4,042 Reported balance is zero. SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT Table 19 (Continued) 67 Respondent reported balance Respondent reported balance Actual Reported Actual Reported balance balance Discrepancy balance balance Discrepancy Underreported by $1,000 Accurate within $1,000 or more (Continued) 15,500 10,000 -5,500 3,650 3,000 -650 13,400 5,000 -8,400 3,550 3,400 -150 7,550 1,000 -6,550 3,450 3,100 -350 6,500 3,000 -3,500 3,400 3,000 -400 6,000 350 -5,700 3,200 3,450 300 4,650 1,150 -3,500 2,550 3,000 450 4,500 3,000 -1,500 2,550 3,000 450 3,750 2,200 -1,550 2,500 1,800 -700 3,500 1,000 -2,500 2,350 2,000 -350 3,500 1,000 -2,500 2,150 2,100 - 50 2,800 150 -2,650 1,900 1,000 -900 2,550 300 -2,250 1,750 1,500 -250 2,300 300 -2,000 1,750 2,000 250 2,000 500 -1,500 1,750 1,500 -250 2,000 300 -1,650 1,750 1,250 -500 1,900 400 -1,500 1,650 2,200 550 1,750 400 -1,350 1,550 1,800 250 Mean = Mean = Mean = 1,450 1,000 -450 4,951 1,768 3,183 1,350 1,350 1,350 1,400 1,500 1,200 50 Overrepc >rted by$l 000 or more 150 10,500 15,000 4,500 -150 7,600 10,000 2,400 1,350 1,000 -350 5,500 10,000 4,500 1,300 1,500 200 2,900 12,000 9,100 1,250 1,700 450 1,600 6,100 4,500 1,250 1,300 50 1,600 3,500 1,900 1,200 1,000 -200 1,500 2,600 1,100 1,150 1,100 - 50 1,500 3,000 1,500 1,150 500 -650 Mean = Mean = Mean = 1,050 1,000 - 50 4,086 7,775 3,689 1,050 950 600 200 700 1,000 -850 Accurate within $1,000 -250 10,450 10,000 -450 400 5,400 5,000 -400 5,050 5,300 250 Mean = Mean = Mean = 4,950 5,000 50 2,363 2,236 80 b Taking signs into account, i.e., letting the errors cancel each other. Note: For all interviews, assigning the same mean reported balance for those for which the balance was not ascertained as for those for which the balance was reported, mean actual balance = $3,810 and mean reported balance = $2,105. 68 AN INVESTIGATION OF RESPONSE ERROR u to te aj fe £ Ja c > S ^ -2 o g c is h ,rH ^ fl "H ni W o IS s ? SCO u C CD 23 O X! w 8 I io** 0) o o s §.s -rH -rH -*-» o -2 fi Sn cu '£ cu-s a M a,— 5 o &§ co .2 CD ro K PQ -.2 c 5 « rt CD 5 5 °„ A*8 cu h jg" ■og iio CD < —i" Si h a£° § £ >> % to o 3 CD X CD +• -i-. "o cd y S-, -g CD -w c O 3 co ro ro « 8 £ * ro H CO 0) £ CD x; CD C CD to o c 3 O a a p C CD Q ft to o u T3 CD 3 c o CD s o CD C to T3 c cd 00 X n J3 e o o o < ■a CD a c CD T3 C O ft U o CD CO 3 e o o £ o CO CD O a CD £ CD ro * 2 J o > ~ o> c 5 CD s a &2 CD K eq m m OO CM CO c a h s h CU S > .S 3 CD Si O ft CD CO O « -w * > a v CD T3 3 a a a 3 H O ■a * c ° h poo u o o y -° ra i—l s &•* U CD O CD O C O p O £ CO CD O TO ^ — •° .5 TO - TO £* J? o c ft e & & TO fc T3 S T3 S "to jg 3 & CD T3 £ CD S 5 h a ■? 4_) rr r U H a 1 nj H H tn ,2 S 3 •to< CD TO .^ K CD • J o<; SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 69 of response in the fall and willingness to be interviewed again in the spring? The data point in that direction, but the results are not conclusive. It is at least suggestive that of the 20 respondents who made the "worst" errors by failing to report accounts owned by themselves or their spouses, 8, or 40 percent, dropped out of the spring sample. Of the 37 who gave the "best" answers, only 9, or 24 percent, dropped out. Of the 12 who had failed to report an account and were reinter- viewed, 6 now reported at least the existence of the account, a clear improvement. Of 5 who failed to report an account owned jointly with an outsider and were reinterviewed, all continued to omit mention of the account—as might have been expected if the account was ex- clusively for the benefit of the outsider. Of 12 who reported the account but not the balance and were reinterviewed, 2 now reported a balance. Of 4 who reported only a total, 1 now gave a breakdown, of 14 who underreported, 1 now reported accurately. Of 5 who over- reported, 3 now reported accurately. Of 28 who reported accurately, 2 now slipped into over or underreport, but 26 remained accurate. Altogether, on this calculation, there was an improvement in 13 reports and a loss in only 2. How is it possible for the second interview to make matters worse? This result can follow if, for example, a respondent reports accurately in the fall and inaccurately in the spring, and then insists on the correctness of the spring report. Without independent knowledge of the facts an investigator would have to rely on what was said in the second interview and would have been misled. As has just been noted, this type of unfortunate sequence of events occurred only twice. One type of error in particular was much reduced. Only 4 inter- views showed overreports after the reinterview. This result suggests that overreport is essentially an accident, or, at least, that people are not as strongly motivated to overreport as to underreport. Compared with the magnitude of the original errors, the gains were small. The results are summarized in Table 21, which shows the error in the best estimate of the fall balance on the basis of the two interviews combined. The data are also shown on Chart 2. The parallel lines on that chart show the margin of error of $1,000 used in the tables. Total Holdings in Savings Accounts It was not unusual for a respondent to report to the interviewer one or more savings accounts other than the account under special study. The reported total holdings as of March 1, 1959, for the reinterviews were as follows: 70 AN INVESTIGATION OF RESPONSE ERROR Total reported in all Number of savings accounts interviews None 3 Under $1,000 10 $1,000 - $4,999 29 $5,000 or over 18 Not ascertained 21 Total 81 These results suggest that many respondents compromised by re- vealing partial information about their savings accounts. Only 3 out of 81 (4 percent) stated flatly that they had no savings accounts at all. Chart 2. Best Estimated Reported and Actual Balances, Fall, 1958 BEST ESTIMATED REPORTED BALANCE, FALL, 1958 (THOUSANDS) 15 2 3 4 5 6 7 6 9 10 II 12 ESTIMATED ACTUAL BALANCE, FALL, 1958 (THOUSANDS) 3 14 & OVER SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 71 >?<$ J_l C CD CD o £ X2 o CO CM "*< CO rH LO CO CO Ol OS £.2 a 1— 1 tH CM CD > 2 CD > CD z -!■> cd a a B H-> -rH -i-> C CD c +-> *■* f-> CD ■^ CO C- ■^ o CO CM o o t-H o en CQ fe3 O Sh CD Ph CO CM i—l i-H m t-H t-H CO o t-H m Sh o ^ £ CD 0) £ 5 -Q I-l CD lO •^ t-H CO to os ^ CM o CD -^ •rH > > U U CD a t— 1 i-H i— 1 in t-H CO CO Sh 3 1 o CO 4-> C C ,rH i3 CD +-> r-i *h CD •"^ CO CO CO ■^ ■^ OS ^ IT) o tH d ^ o 1— 1 i-H tH CD CM •^ o fcr3 CD t-H a Ph Sh CD CQ -Q tH ■^ c- CO r}< ■^t* O CM t- t-H OS \ a 1 CM i-H i-H i—l c- CM ■"^ o 1— 1 •rH > $H •r-l CD o> CO CO t- CO •* •* O CO CO o i—l O t-H »—l i-H 1— 1 CD CM CO o < Sh CD ft 1— 1 i—i >>X3 -m as CD d +-> CO CQ 3 CD d d Sh ^_, U CD (1) 1^5 CQ sh o -t-> M r^ "3 £> « -t-> d rj a CD • rH E Sh q. O CQ o c CD o CD CQ o o o d c d o Sh tJ -t-> Sh O u a d o a u o o Sh O a Sh O CD - tH ft, 4-> c 3 o s- Q, -rH w o CD -r ^ A> Sh i— i CD Sh -M B Sh O w CD ,rH CQ ^) o o i-H O o o o o o o Sh O .2 d d Sh CD •' H Sh d^ 2 S_i T» i-H i-H c • rH X! 4-> •rH Sh 'm o o *H ,« o o 4-> rQ ^ cd £3 * Sh T3 CD -4-> O «tH CD O X5 X2 £ Sh CD ,C 3 o o 73 m ® O c t C "£ O o d T3 CD a | Sh c o JE3 ■*-• c d i-H d •a 4-> CD .rH CQ > C Sh O CD CQ S CD U o 1° £ ° O CD fi r. to cd d d -o -a 3 5 5 a CD Sh -t-» Sh O a •5 d r-H d -t-> o +J 3 ® S 1 a § O CQ CQ < CD ™ Sh ^ CD Sh Sh Sh H CD -rH « a O u T3 CD r— 1 •!-H d ° 8. <3 Sh CD &3 £ 3 4-1 Sh o CD CD S3 Sh > o 3 U < =<-( 1*4 tf bJD s ■r-l a o Sh 72 AN INVESTIGATION OF RESPONSE ERROR Excluding the "not ascertained" group, 78 percent of the respon- dents reported total savings account balances of $1,000 or more as of March 1, 1959. Change in Balance Table 22 shows the relation between actual change in balance and reported change in balance from the fall of 1958 to the spring of 1959. This reported change is the difference between the best esti- mate for the fall, based on full use of both interviews, and the balance reported in the spring. A comparison could be made for 49 interviews. Of the 49, 26 actually increased by $100 or more. (Small- er changes typically represented only the accrual of interest.) Of the 26, 5 reported a decrease, 5 no change, and 16 an increase. Of the 9 actual decreases, 3 were reported as such, 4 as no change, and 2 as increases. The low level of accuracy indicated is in part a result of the peculiar structure of the spring interview. The respondents were asked to report in the spring both fall and spring balances. They were then asked to reconcile the two reports of their fall balances. As a rule any changes made in the reconciliation were not carried over to correct the spring balance. Hence, the change, as measured, often was from a corrected fall balance to an uncorrected spring balance. The difference between the two may include both the cor- rection and the reported change. Size of Actual Balance The next step became a search for new light on the causes of response error through study of the factors associated with it. Table 23 shows the relation between the size of the actual balance in the fall of 1958 and the response error in the fall report of the fall balance. The hypothesis originally suggested was that the larger the actual balance the greater the response error would be. The data tend to support the hypothesis. An error of $1,000 may seem less to a respondent on a balance of $5,000 than an error of that amount in reporting a balance of $1,000. But it hardly seems likely that this phenomenon accounts en- tirely for the observed decline in the proportion who reported their balances within $1,000 as the actual balances increased. Of those with a balance under $2,000, 56 percent reported accurately by this measure, compared with 35 percent of those with a balance between $2,000 and $3,499, 18 percent of those with a balance between $3,500 and $5,000, and only 10 percent with a balance of $5,000 or more. It is important to note that the proportion who failed to report the account entirely tended to rise with size of balance. If people failed to SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 73 o O OS O OS O OS O OS O OS O OS O OS O OS O OS o OS o OS IO OS » 6» i o os o OS j I OS I i-H ! 1 O OS O OS •H •>* I h i ! O OS i o os i ITS os l O OS o os O OS ! o o OS o os O OS c a •H ■>}< CO O t— I hiomhm ^ OS OS OS OS OS OS OS OS OS OS OS OS OS OS OS OS ^ rf\ fr\ rf\ ff\ rf\ er\ I I I I I I o o o o o o O O O t-4 o o o O ITS iH CO CM i— I OS OS OS OS OS OS os os os os os os ^* OS OS OS OS h eg in C/J C/J C/J C/J C/J C/J I I I I I I «H O O O O O o o o o o h in o o o^ rifin C/J C/J C/J C/J C/J CO C cd u o 53 O 73 ^j T5 -t-> h 0) O CO a o 3 tu U c o o a O ni .Q a> u CO o si o _T a> £ at > CO ^ U 0) CD o a> ■5 J CM 2 o -X IH 73 +e 0i o en c & £ at b CD ™ o o > o£ U 0) "a co o ■i-t £ o -t-> o CD !> JO > b .5 £ hn ■^H r, !-H u 0) CO C o OS C CM 43 O T3 C cd 2 £ OS co bfl .2 s at c ■" 0) OS 73 «M U5 CO S a os o> .^ ■ Br- S 74 AN INVESTIGATION OF RESPONSE ERROR cd cd .£ "-! CD !Z 00 S -H a .5 •3 _ -O ^ rt S F-i *H Cd O -M _ o .2 g *t3 es CC CJ Si iH g.s -a CM CM CM CO o CM o CO CO o OS co •^< in CO CO CM CM O c- o O CO 03 o CO co ■^< ■* ■* Ln •<*< CO ■^ -* cm' CO s cu 1 %4 o s s . O c- CO en c- CM ,-H in CO co o i CD i—l 1-1 CO Fh O o o o u cu ft c cu o CO co o o C0 C- c- t- o o o in co CM CO CM co o Oi Oi OJ S Fh CU CO CO o CO CM ,H 00 in o co e- 00 3 in 2 ,Q ^ 1 O "cd o r ja M ITS F-l CD ft cu o in in i co CM CD c- oa . co o CU CO* co CO i 1-1 ,H ■* CM 1 1-1 o O C cd "cd X2 OS 1 OS a 3 Fh cu CD -tf CM CO CM rt ^r in ^ CO co CM "cd CO a ,£> ^ €/3- o 1 cu CM CM CO ,— p .Q i o i c CD oo 00 CO o co in o CO O CD cu c- 1— ( in O te- ft u " 03 g u cu 00 o oo o m in t- CO CD Ol CU 3 CM CM CM ' H CD 1-1 co o 3 £ Sh e cu o in 00 c- OS ■* in CO CD t- CO o a cu ft CM in CO o u o cu CU CO 3 o ft CO T3 cu CJ B CO ^3 3 3 j5 a) a o a co CU co c CU 4) 3 cd u o c s CU CJ to DQ O 3 O o Fh O S Fh O a c -° c ^_, T3 cu cd jj Fh Fh O O O o (» >> CU •- 1 o o c O Fh °° Fh rt S en cd 3 "H Q vh a. o to ^ ffi o a 3 o o CJ ni •a B a co cu (h ,0. j>> ° cu >> Fh cu c Fh cd S cu u t3 x; B 0) u B CO cu (H JO c 3 o cu o > T3 CU CO *j 5 -a 5 £ 3 O o CJ cd o 0) u o o o o o o J- 1 ee- B 2 h cu .5 CU S a CJ u O J2 B cu T3 o c o 6 ° o ■W a (V ft 0) Fh is •" o o o CJ CtJ ■a c ed Fh o a Fh a cu cu h o c^ +j o a cu "O cu 5 T3 « cu Fh "cd "id 3 °cf cu U fa 3 °o cu Fh Fh Fh •a a> o cu o CU Fh 3 o CJ XI g s Fh a CU Fh O B 0) > CJ CJ «£ •rf us ttJ 13 O <£ a £> 5n^ - « 3 " •" cd o o cu c CO ^^ CU cd co 5 J3 CO "" CU v. SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 75 report an account because it did not come to mind during the inter- view, one would expect more errors of this type for moderate sized accounts than for large accounts. The data suggest, therefore, that failure to report an account at all is not primarily due to memory error. The proportion of accounts for which the balance was not ascertained also tended to increase somewhat with increasing actual balance. As shown in the last column of the table, the mean actual balance for those for whom the balance was not ascertained was $4,768. For those who did report the balances in their accounts, the mean actual balance was $3,310. Altogether, then, the data in this table clearly point in the direction of increasing response error with increasing actual size of balance. Type of Use of the Account Respondents who reported the existence of an account were asked questions concerning its use. Table 24 shows the relation between the type of use of the account under study as reported by the respon- dents and response error in the best estimate of the Fall, 1958, balance. The hypothesis was that people who use their accounts fre- quently for short-term transactions or for medium-term transactions would have occasion to make deposits and withdrawals often and would tend to know what the balances were in their accounts. It was also expected that people who use their accounts for these rather public reasons would be more willing to discuss the accounts than people who think of their accounts as their major and perhaps secret fi- nancial resource. The data do not indicate any very large differences in the accuracy of reports of people who use their accounts for these different purposes. The only conspicuous difference between the two groups is that those who use their accounts for short -or medium -term transactions did not include any of those who had overreported by $1,000 or more. This result is consistent with the hypothesis mentioned earlier that overreport is essentially an ac- cident. People who use their accounts for short - or medium - term transactions tend not to make this particular mistake. Response Error in Reports of Size of Change in the Balance Another way of testing the same basic hypothesis that people with active accounts will report their balances more precisely is by re- lating the best estimate of the fall balance to the absolute amount of change in the balance from the fall to the spring. The hypothesis was that people with small changes, such as might result simply from the accrual of interest, would be more likely to make mistakes in their reports. The data shown in Table 25 point in that direction but the differences are not large enough to be statistically reliable. 76 AN INVESTIGATION OF RESPONSE ERRORS CO in o> i—i ^ •—i r-H d fe t*H O -i-> rn G" £ 3 o •rH > o Jh < CD -4-> CO £ •rH J3 H-> a CD GO P- W F- -i-> CO O cd -t-> CQ c c o •rH H-> rr| Jh t— i o 0) u Jh Ph w •i-l 0) <]) 10 o o # r-H m -* CM CD r-H rD tt H 5 co O 3 o £ 8 « £ .a ^ CO 0) ■rH > Jh CD CO .3 0) CM CM CO co CM <3> r-( CM 00 CO CM O CM ■^ O CO CO CM ■a CD 73 Of c 10 3 o 73 3 a> a} ap o u > •rH u J-i •rH rH o „ rQ SH -M 3 «J +-> 0) H o H-> o o o o C) O H-> c O flt ci 73 CJ r-H rrt at 73 •a cd ■*-> 111 CJ a 73 CD -t-> 3 o rD 73 CD u oi u CD -t-> o r-H o H-> -4-» Jh o ft CD rD ft CD 3 rD rH O ft CD Ph Ph ft CD Ph Ph Jh o o o o Jh o o o o o o o rD 73 CD H-> Jh O ft CD Jh rH CD 73 C 13 rD 73 CD ■*-> U O ft CD U u CD > o cd Jh CJ cj o H CO CO CM CD CO CO fe CD > CO Jh CD CD > Jh i— i CD t-H ■*-> a CO •rH CD <»H T-S O =1 Jh o CD X rD W H 1 cfl a 73 S Ed ■o CD Jh O ft CD 73 Jh 0) C O •rH rH -(-> CO a D O £ -r-> o -r-> c c 3 n cd u I* o ri jh s 8 CD Jh CO . CD 3 "S ft Jh JD o o p O CO O 73 * q «" S rC 2 > co to « CO CD S CD O SJQ • rH CD xi y Jh SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 77 00 05 tf3 ,-H OS tH bX> r d ft o o >» ., £ CO 0) lO T* OS > i-H DO rH CD - CD ►5 d > a* as c a «|° 55 ^ S Sffl.2 co <4-( 3 O £ CD '2 C CO CQ ^ U cd ft£ co ctf a) r-i 5 | ,a a C— I CT> 8 CD « 5 o CD O 1 a XI x u C CD O CO i CO rH H i © ^ O) CM o CO to i CM CM i C- CM tJH o CO i—i to rH O i—i .Q i3o C CD •^ €/3- CD bO MS? c r- Ol 3 «3 1; ■£3 o J3 u 1 Tt< o ■^ O to m t-O) 1 CO rH i-H c -rt «H 3 CD««- O a cd S CD CD &£>,-*. +-> 3 i— i o u CO o . i— 1 t- "^ t- CO CO cm in tj< co O ■^ ,Q ' CM r-t 7— I CO CM CO O CM < a ««- i— i CO £ (1) r-H -r-i OS CO CD tr- CO <* ^OOCO o Oi < £ 1— 1 i—l i—i i-H CO CM CO o o rH 1—1 CD -t-> C •rH CD i u u CD c a o CD a T5 o a, c CD 73 c a S «3 c ro ^ ■^ -a o 5h CD O -o CD r^ O C cd a rt f-i CO "3 G U CD o u a 2 a C O rH go ° CD . -l-> £ 3 o a CO CD CD U -t-i g o u u ed CD > CD CO o 73 O o o O o o ° w "o <=> TO rH . O CQ U > £ >> 73 £ •rH (I) CQ a ^ o • 1-1 . — 1 ed ■rH -a CD rH ^• r -' r-T o ^^^ ^ rO >> -rt -° -S TO tf * S ^H rH H-> > TO g Jj ** CD •rH > t-l CD ? , CD o CD C o rO CD H-> CD •£ CO > ft CD ° O > in O 5 CD O U X TO ed o ^ a u Jh T3 T3 a io a O CD TO «rH O CD &a o 1* -t-> -4-> CD CD 2 ft+; T5 rH CD rt o O -1-1 ^ m W (H U 9 o CD rH U U E-i u CD "* T3 O CD 8 S-K < cu O o f CD rH ^ CD « CD i— i O X5 O S 3 CD CD « T3 CD CJ 2 C > O rQ a 3 < PS* Cm En tf « a CO H-> c O rt CD O T-J rH O 5 CD rt % & S 2 o> C0 rG O CO 73 ti °5 ro CD ■rH > o CO CD CO ed tn r. CD CI) CO u H-> n^ CJ fl CD ■w-< ■ rH T3 CM > a CI) T3 CD TO ■H t CJ5 s a ■r-< ■rH > a •rA l-H (I) a CD H-> <- a £ •rH CD ro CD rH rH 00 flT ^ CD CO TO u rC CD ft -r-> rH CO o s CJ CD T3 CO C35 to m CD Sh CD - ■& .s *" rH rC ft CJ CO - TO ^CO CCi £ rS rH o t*H CO I ■r-l > CD ■•-< CD CD CO -M CD TO X5 3o CD CJ C ed ed rQ CD rC s • rH T3 a ? o rH CD CO TO o CD J2 H-> CD CJ 'rH CD ft ■ — i CCi H-» o CO CD CO CD u o CD -a CD faao H TO 78 AN INVESTIGATION OF RESPONSE ERROR Income The next series of tables is designed to test the relation between response error and a group of major demographic or economic characteristics of the unit. In Table 26 the response error is shown for groups of spending units in different income classes. The data show a small but statistically not reliable tendency for units with income over $10,000 to be more likely than those with incomes be- low that level to report a balance for the account. This result is the opposite of what might have been predicted from the study of the table showing the relation between actual balance and response error. Since there is a tendency for response error to increase with size of balance, one might have expected a tendency for response error to increase with income. The data do not bear out this relationship. These results suggest that the worst reports may be for large bal- ances owned by people of lower socioeconomic status. Education The relation between the education of the respondent and response error in the report of the fall balance is shown in Table 27. The data in Table 27 suggest a tendency for reports to be more accurate as education increases. The differences, however, are not large enough to be impressive from a statistical point of view. If any- thing, people with more education are more likely to report the existence of an account than those with less education. Occupation In Table 28 the relation between the occupation of the head of the family and response error is shown. The number of interviews permitted only a distinction between white-collar and blue -collar workers. The reports of white-collar workers seem to have been more accurate. This result, of course, is consistent with the pre- vious findings for education and income. Age of the Respondent Table 29 compares the accuracy of response for respondents under 45 with those 45-64 and 65 years of age and over. The table shows little difference among these three groups. What differences do appear may be easily attributable to sampling error. If anything, SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 79 co LO OS (XI " -4-> •1-1 "- 1 g Sh bC. o c tf g a a o .rH CD w tf c 2 .2 =a ■— i faD 0) «J «-> G ID o Sh 0) CD CD co o _ aoft 0) o3 pq CD ■a 2 o o O o lo lo 1— 1 oi OJ CO rH rH H-> O .Q rH i— i CO csj i— 1 CO O cq 3 rH bfi .3 T3 G (l) 1 & O OS XI J3 u O OS O i— i i— i I CO ■^ 1 ■* OS in «M O OS CM i—i rH I CO rH 1 LO OS CO O LO OS 0) €»-€»- a o o fl >-H h s S o CM in o LO CO CM C- ■^ o rH T3 O eg i— i rH CD CVI CO o ■* T-\ 03 £ r— 1 .r-l OS t- CO -* ^ O CD CO o OS ■3 > 1—1 i—i i-H CD cq CO o o < u 1— 1 rH CD H-> c * rH cd o d 3 T3 rt X2 H-> B 0) ■H 3 0) G • rH ei -4-> ?H 0) H-> d CO (0 -£ o CD u CD error in best est of Fall, 1958, bal CD o o O U o u o o CO o 0) o S3 i— i oi T3 0) en G 4H 0) Sh -t-> ,Q h-> G ed cd u CD > CD CO G • |H i-H o +j •i-H 3 H-> g o u o 5h O a u o o o tH &S- >, O a rH o o o 1— 1 #* >> o o o rH «3- co g G o o O o o cd o -t-> 03 rH +J O CD u a 73 CD -M Sh O T3 CD -M rH • iH xi •rH > rH CD -M c •rH a- o -a T3 G 73 a o CD « fH ri -t-> CD H-> 2 £3 CD CD -t-> ccJ rH «rH O o T3 Sh U 3 o T3 CD !h rH u cct H-> CD o o CD rH d +J Si T3 0) •ih -»-> ft a -!-> TS CD o O CD rH a> CD £ Fh C > o H ,Q o ft tf « O r^ O rH CD U ro rt +j o d en hO 03 G r* •rH -a 0> i H o o %-> o <1) CD c 10 +J oi X3 o o CD •rH XJ r* G a CD O rH V) CD §5 G ft 01 O > U o !h I-H rH (I) a* +-i Ui n Ch H-l ■pH lO CSI o X3 T5 G rt cri rH H-l a; G u T7 +J i-H 3 CO CO CJ en CD X CD O W J Q 80 AN INVESTIGATION OF RESPONSE ERROR 00 in os i-H i— i >— i cd In -4-> IH fl o 0) 73 -4-> C P O to o a £ cd cd a; •r-l > Pi 0) HH fl) -4-> o ■*-> cd c a O ««H -»-> O rn +-> W cd o c o -4-> CD % •IH rn •iH o Pi •<-> G CO P o •iH 73 O a> P cd bO 5-! i— i Cd w a> 0) 0) CO c •i-i o Pi (1) a CD o P< CQ a cd m Pi ed CQ t- CM £ J3 1—1 CM 1—1 co CM -*• OS ■^ Pi W o CD cd 73 o Cd CQ 3 Pi CQ 73 hfl 0) OS CM CD 00 00 ITS CO in "^ o 00 W i—i T-l ITS tH CO o CO S Pi r-i bfl O •iH w en £ a> i—l -iH OS c- CO ■^ ■* O CO 00 O OS 1—1 T— 1 1—1 CO CM CO o 1—1 o a» -4-> c •T-i CD CJ 5 cd r— 1 ed -l-> ,a 3 73 CD X2 a> cu +-> CQ * s fl cd -M •i-i -i-> c 3 * cci +-> Pi CD CQ O 3 o o CD P O CD P S3 o CQ 73 o cd a O best 958, cd +-) o CD CQ CD i— c cd P c CQ W -l-> C ■•"• •i-i CO a) 73 C ■a o CJ o cd p O <+-{ CD O cd i— i cd p o o p o Response error in report of Fall, 1 +-> c o cj o Rj ■+-> Pi O a 0) CU a d ed "cd o o o P £ -i-> c O cj o cd 73 o 1— 1 .a T3 CD -i-> P O a o o 1—1 73 CD P O o o 1—1 m- a •r-i js -i-> • rH CD CQ CD •i-i > P CD a •■-I Pi ed CD +-> -M 73 £2 CD P a CD -M cd ^ O O CD P 8 3 T3 CD P p P cd +-> O Pi 3 o p X! +-> a a S -l-» T3 CD CJ CD Pi CD CD £ p a > o a CD r— 1 •l-l O » « a P o < cd CD CD 3 1X4 tf K s? T3 CD a ■H cd -•-> Pi CD CJ CQ cd -l-> o c CQ cd £ -i-> C CD 73 C o a CQ CD P CD A -i-> , • IH r^ o •rH T3 c C o 3 • iH "cd O P o 3 CM n 73 CD CD CD CQ u ^3 CD O rU •iH si -I- 1 Z Pi CD •■-I O P CD CQ *± PV ^ C K ® o CD o 1—1 J a o fi m •pH ■o o 73 CM cd c CQ cd -»-> CD XS u 73 ** Pi a CQ co CJ CQ CD X CD & SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 81 00 in 01 s cd o ^> X! t3 ,2. °> P '-rH 2 ° & o s 13 o cj O .5 3 " p— i . o> 0) co o w c C ret 8. 3 cq CQ DO CO > .3 a o 5 .2 CD bX) c cj Pi O CO V ■* CO o .3 £ PQ u O CO 1 J-l CO o 5 s en co <— I -r-l 0) 6 g ° fl i— i CD d %* & ao £ ID w <» d) iH ■*-> •> CQ £j *h o 2 £ E ® co •£ ® o CO c- i-H CM ITi CM OS i-H OS o in in CO CM CO Tt< Tt« OS CO eg a 3 o u o cd -f-J h o a 0) T3 CD C o c 0) c CCt I— ( d C o u CJ d CD 1 I rt CD cd .a T3 CO CO CO B O o o d 73 co -M o CO tf « 3 ctt 1-8 "3 "^ is a) -M ^ af CO £ S-c o 0) cj c r— < cci CO a CO CO o a o o o o T3 CD -M fn O a CO u u CO T3 c 13 CO o s O g. § CO 'H o a co u u CO 5 p B CQ • fH CO X! -t-> a o CO T3 O X CO co u d o> o u o tn O X2 Cd i— i CO o a CO d CO si CO CQ O £ CQ CO rt 73 CO c •rH cct -M O CQ CXJ . £ S ^ e > 3 tH O CO * 1 CQ 2 %1 o « co cj ^^ co s-, CJ O U «M CO a rH O -M o M c 5 co- co g, (1) 3 ffl 3 fc u 2 o £ CJ O X ed -O y 82 AN INVESTIGATION OF RESPONSE ERROR 00 irs OS fl -M fl) W o c a> ft o ,H § ° o CO b£) S w<3.2 O 3 w O -3 o u W CD CO fl a CO OS CM H . co° fl ->-> •- 1 fl CO 0) fl u ■3 fit 5h CD > O i— t CO CO lO CO CM ■^ i— 1 •^ OS o ■^ T3 CM i-H CM i-H i— t lO CM CM o CM « fi fl i-H -t-> a lO CD CO T3 C o ft CO CD ■^ Sh CO OS •^ LO CM O CM OS CM CO OS o OS T— 1 1—1 i-H l— 1 CO CM CO o ITS <+H LO 1—1 o "tf CD OJD < u JS CD fi "* 1—1 c- Tf i-H i— ( 1 00 C- ■^ c- o • i-l 1— I i— 1 i-H co CM CO o o CO CD a o T5 CD C •i-i CO i— i fl X! fl fl X5 CO o CO >> CD CO rj co i m "5 CO rQ CD -M CO u CD fl -4-> CO fl CD CD CJS i »H Sh fl CJ U CD u CO O CO o o o u +J CO - 1 CD ;z! o -t-> a CD C fl -t-> CJ o fl a o a c fe •1-1 o fl o c CD CJ c CO fl •Q CD CO fl p— 1 fl fl fl fl 5h O o u o si o CD ■H CD > -4-> c fl o o o fl o a CO CD u >? •i-i c fl CL « CO CD 5-i u fl -t-> C CD > CD CO fl •pH -t-i •r-l -a o o o CO u o ■4-1 o o i-H €«■ >. X! o o o 1— 1 o o o 1— 1 a • fH X! CO CD ■r-l CO u fi CD •i-i T3 O 4-> fl fl o CD 5h CD o T3 •Q > 5-t » .5 h CD -r-> fl CJ o X2 c T3 CD o 5 ° A fl o CJ -»-> fl -4-> 5-i O a CD 5-i CD •1-4 ->-> CO H CD fl « s 5h a CD u o £ fl O X! -u +j fl ?h o u fl T3 fl -fl CD fl CD o fl 2 >— 1 fl x> -4-> 5h O a CD CD -4-> fl a ■ r* o ^H -t-> fl ±3 3 CD CD u ^ fl o CD 5h u 5-i 'fl u -fl CD r— 1 O „ O X! -M o O -(-> CD u fl -t-> CD 5 fa C-> "fl 5-1 a a 5-i T3 CD O o Xi o o O O o CD CD o a CD c > O H a •i-H CO ^ all " CO • — < a A .— i -4-> a CO c- CO ■^ tr- o C- o o co a «M i—t 1—1 io CM CO o I— 1 »^ ■a CD o a CO u *4-l o X CD CD m CO CO CO OS i—l CD CM o CM CO IS H I-H 1—1 co CM ■^ o 1—1 co CO 1 — 1 .,— 1 Oi c- CO "^ ■* O CD CO o Oi ■"J > i-H T— 1 rH co CM CO o o C! • i-H CD 3 8 O a g 73 ■+J c B CD _q cd -»-> rt CO -|_> m CO o co a ■>-> o c o c rt CO C 3 -Q S o o O Sh •i-i ^ +-> 73 CO o o a £ -4-> S3 O a u o o o O a O o o o o 1—1 •i-i C ^ o o a a O o o ■+-> B > o 73 CO ^ C ^ ^3 o o O «4H o I— 1 o o 1— 1 CQ •i-i > CO CD ■y 3 rt T3 -4-> u o a u ^2 u !h -u u Q o CJ o 73 O o O c 73 -t-> a O o o 73 CC$ Q « 73 13 T3 a -4-> 'cS •i-i O -(-> tn 0) 5 s u U h t-t 73 I— 1 O O a o a 73 £3 u > 3 o o o a 5 •r-1 a ti PS & O < Ed K tf 25 SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 85 at -l-> i— ( c o r^ a CO cd 0) HH a o o CO S £ VM 0) > CO c ° 5 J3 p5 -rj H-» NH > 5 co ^ PQ &«> rH <*> C g cu .5 h -m ° 0) 03 5 S § g +j to r « o is H ttj to CD £ CD PQ £ bp cd re 5 1 1 . 3 a) g O Sh a> jj co re o % S"" •*• CD o re H c CD 03 >> 0) C O a «(H o en T3 rH o o 0) CO ■a rH o o CD rH O rH CO a cd cd M th co co m o in rr co o rH CQ rH rH rH CO CO rH CNI o o rH as CO « CO T3 rH o o 0) rH CD a o CO CO a CD CD M c- c- lent- eg rt* m i oa 1 r- 1 H C- tH 1 IfJ o o rH rH CO 1— 1 .pH 0) +-> c •rH •^ co co co ^ -rr at -^ in o i-H rH r- ( CO CM *hh u T— I o 1— 1 O 00 CD -t-> cd a • i-H -4-> CO -4-1 CO a> -Q c •rH u o u u CD a> CO c o a CO a> Ph CD O rQ CO m i—i &H CD X! -i-J HH o Failed to report account Account owned by respondent or spouse or the two jointly Account owned entirely or in part by someone other than respondent and spouse Reported account, balance not ascertained Reported account but refused to state balance Reported a total in several accounts but would not break it down Reported balance for account Underreported by $1,000 or more Over reported by $1,000 or more Accurate within $1,000 -M o H CO 1 t-H > u CD H-> a •|H m o u CD rQ a 2 a u o o Rj 73 CD H-> rH a CD rH rH » CD o CO rQ 3 fafl s ■ rH xs o u o CD u a CD r« >> CD X! H-> 73 • rH cd CO o rC CO H-> I go co » o »° "S ^ d rt m i o T3 3 co co T^ co cd O CD O r3 rH Q CD CO 3 u CD rD c CD O rH CD a 86 AN INVESTIGATION OF RESPONSE ERROR Memory Error The two tables which follow were constructed to make possible the comparison of accuracy of response at different dates. Table 32 is based on reports of the balance at three points in time about which information was asked in the fall of 1958. The dates involved were the first of the month in which the interview was taken, six months earlier, and twelve months earlier. The comparison of the three dates cannot be made for all respondents since a number of respon- dents opened their accounts within the period of one year. The data show very little difference in accuracy from one date to the next. No doubt as one goes back farther in time, there would be more departures from perfect accuracy within the group who were approximately accurate. The proportion who were correct within $1,000, however, does not change among the three dates. Memory error does not seem to account for any large part of the errors reported in Table 32. A similar comparison is shown in Table 33 based on data from the reinterviews in the spring of 1959. The first column shows the error in the report of the spring balance, that is, the current balance at the time of interview. The second column shows the report in the spring of the balance in the fall. The second column is based on the statements made by the respondent before he was shown his earlier report of the fall balance. Once again, there is very little evidence of any difference in accuracy between the two reports. It is not surprising that people who failed to report the existence of an account as of a date when they were being interviewed should also fail to report that this account existed half a year earlier. Also, it is not surprising that people who refused to state the balance in an account as of the time of interview should also refuse to state the balance in that account as of some earlier date. No doubt memory is a factor, but its effects are not powerful in explaining the allocation of respondents among the categories shown in Table 33. Classification of Respondents as Accurate Reporters The study design afforded the possibility of independent tests of the accuracy of given individuals as reporters. A respondent might report accurately or inaccurately in the first interview. He might also report his current balance accurately or inaccurately in the second interview. To what extent did people who reported accurately in one interview report accurately in the second? Table 34 was con- structed to shed light on this question. For the most part respondents who reported accurately in the fall persisted in the same tendency in the spring. However, there are a number of examples of people SAVINGS ACCOUNTS: FIRST FIELD EXPERIMENT 87 o Ph cd £°° £ os *3 £ © d CD o «« •£ c e r* ctf 5 ^ CO H » § C^ to ° n M S-i -rj ro w • c a cd 73 o a CO a> 0) o 0) a o CO >> £2 CD H £ cd 2 q. a CO •S-o fa cd CD CO O _ CO O id ^ o o . *j 73 u cd 8.8 ? 5 £*a CD <4H CCi -t_> ■fci cd c x: 3 - O CD o ■«< O X! -t-> 3 CD O Si < o o o cd o o cj Cd 73 T3 CD ■u o a CD O cj CJ cd CD £ > O CD 73 M +, cd > ^ £1 rrt O ^ 43 ^ rt 2 o » "^ CD T3 Fh CD tn -t-> CD 2 £ Jo "S3 CCi !h U U 3 o > CJ o< bo g a T3 CD O CD CO CD x: c o CD & • +■> hC O a a -a a 5 o u T5 s cct cct cd T3 T3 CD CJ CD a CO c CD CJ Jh CO cd -y " CD O T5 O c CD O O c CD u CO g c c O 3 O -»-> O +j nun CD CJ CD 3 «J 3 ^ CD ^ CD CO CD +j j3 +j T3 £ 73 T3 73 aj -t-> O C CO CD O Q CS . CO 5 m 73 +j C O O C CO §)* CO CD cd O CM O XS CM J3 i* en ^ -M CD -M CO fi CD 3 CD 73 7! 73 C S C o « o aw a CO CO CD u CD 88 AN INVESTIGATION OF RESPONSE ERROR CO •l-l o 5 <-> a* d c > i— I .pH ?H cd ,_, « S d TO t-4 bfi 3 -i-H pH o d d rt pO o CO u o a 0) u o h g pH ■M o a ** d T3 O •*< d O CD a o CO >> pQ -i-j CO d Q a a d M ph d O C^ C CD ^ O TJ £ •r-> CD " o g d > | cd 43 O XI rH CD CO O _ S3 CD e *H S d u d -4_> S? £2 O pd u c5 o d 73 ppH 0> 73 -M CD ph ■e o 5 CD Ph d d o o u d CD £ > 5 CD 73 CO ^ d IFH p-h3 d - 4 - 1 pH OpQ 73 A ^ d CD I* Ph CD ph CD O fH s ° 2 a -(-> pH d o sh 8s° o ° ° « CO _j *~1 ^— < ^> "—> X r-t U .0 ^ ^2 t>, _ pO >> -, o 73 *d C CD 73 5 rt 2 S ^ o a o cd ■" cd a^ 73 ph CD d CD pH PH pH ■H cd +-> o S V w , O d > O P4 Oi ITi o> T-l t-H 9 m •pH S pH a CO d d o •pH H-> o Ctj d cd 73 ■ — i cd 73 pQ •i—i 4-> •rH d O o .a a cd CO 73 pd p« 4-> CO cd o pH u o •rH £ a 73 -M d d cd o 73 d d CO 1-5 bi)p2 X d o 3 •rH T3 o d rH Li p3 X S t* -M pH p— 1 ^H « • pH pH O d a « 3 < 3§ d O Co ■pH 73 a, p2 CO 1 •pH -pH +J Q d -d S p* > H-> o PH d rH CO o -a CD O 73 d • pH Sh ■a o o d 5_, o i-i o o a ps -M ** CO PH 73 ph ■M o "S rvi d •rH cd N 73 pH a H-> CO O d -i-i d 73 a PH o C0p3 CO a o O X PhKQW CO -D -J" c O (-. o cu ^ — *> m O 3 CO Bj ■ — ' >-> CD S^ 01 XI 3 « CD -»-> ^ j. *3 33 C0 C 3 tn ° £ 3 3 c § cd S. R O CD ft S co tt ^0°5 Account owned by respondent or spouse > E{ 8 8 Ed XT' 3 * 3 « r-a 5 X! 3 S * o £ S *j o » O CO ni *J ™ ■2 2 73 TO h -5 CD ft S t> o < CD P "S O TO _£ "I CO (3> O S 73 <35 c2 H h •- 1 - O X bfl a z w 5 a CO 90 AN INVESTIGATION OF RESPONSE ERROR who differed in accuracy of report between the two dates. For example, of the 28 respondents who were accurate within $1,000 in the fall, 7 of the 28, or 25 percent, were backsliders. In other words an individual who reports accurately on one occasion probably will do so again, but one cannot have complete confidence that this will be true. This result, combined with the evidence that many respon- dents compromised and reported partial information about savings accounts, suggests that it would not be easy to divide respondents into two groups: accurate reporters and inaccurate reporters. Conclusions This chapter includes evidence that accuracy of response was associated with each of the following: Use of a structured questionnaire Use of re interview Smaller actual balance White-collar occupational status Reporting that the family keep records Accuracy of response in a previous interview There is some doubt as to whether accuracy of response is as- sociated with each of the following: Type of use of the account Change in balance or level of activity of the account Income Education There seems to be little evidence of association between ac- curacy of response and the following: Age Sex of respondent Date for which the balance is asked (current, six months ago, or a year ago) These results will be compared in a later chapter with those from other surveys. VI. REPORTS OF CASH LOANS: THE SECOND FIELD EXPERIMENT In the winter and spring of 1959 the investigators reviewed the results of the fall interviews and made arrangements for two new field experiments. The first of these involved interviews in two large cities with a group of respondents known to have borrowed cash from small loan companies. The results are reported in this chapter. The following chapter reports the results of an additional study of owners of savings accounts which was also planned and carried out during 1959. Purposes of the Study and Description of Procedures Purposes Although further study of owners of savings accounts seemed desirable after completion of the interviews in the first field study, it was decided also to investigate response error in the field of consumer credit. Several factors contributed to this decision. The importance to the economy of the nation of the large and fluctuating volume of consumer debt outstanding is well known. There are indications that the Surveys of Consumer Finances seriously under- state total consumer credit outstanding. Within the field of consumer debt the indications are that debt to cash lenders is the most difficult to ask about in personal interviews. The investigators felt that more might be learned by tackling the difficult topics than the easy ones. The Need for Anonymity In this study as in the others in the series both the investigators and the cooperating financial institutions felt the need to guarantee the confidentiality of data received from individuals in the sample. As before, interviewers were not informed as to the sources of the names and addresses in the sample. In addition, a number of "dud" addresses were included in the sample, names selected from the ap- propriate telephone directories. These interviews have been excluded from the analysis. They did serve the purpose, however, of making certain that even an interviewer who guessed the origin of the sample could never be sure of the fact that any respondent's name had been selected from a list of borrowers. Sample Selection The sample was drawn by selecting every nth name from lists of known borrowers in two large cities. 91 92 AN INVESTIGATION OF RESPONSE ERROR Validation Procedure The procedure adopted, as in the previous study, was for the Center to compare data gained in the survey with data from previous files on known borrowers after the completion of the interviewing (with due safeguards regarding the anonymity of respondents). Experimental Manipulation In planning this project the investigators proceeded by attempting to develop a theory of response error based on the knowledge avail- able to them at this stage. The results of this effort will be incor- porated in Chapter IX. Here it is necessary only to recapitulate briefly the reasoning which led to the particular procedures tried in the field. The investigators had come to believe the most important source of response error in reports of financial data to be the lack of motivation on the part of the respondent to reveal the data. The key question thus became, what motives are the crucial ones? Several suggestions were made. The experimental manipulations were based on the following argument, reproduced here from the statement of plans: "The experience of the survey worker is that accurate infor- mation can be obtained from people when what is required is 'public' information of the kind which people feel others ordi- narily know about them. The difficulties arise only when it is necessary to ask for 'private' information. Very frequently people will reveal 'private' information to interviewers, but what is needed is that everyone or nearly everyone should reveal this information. "Why should people tell an interviewer things they would not reveal in ordinary social intercourse? In order to obtain 'pri- vate' information from respondents, the interviewer must get around the usual standards and norms. The interviewer can do this only if he can so structure the whole situation that the re- spondent sees it as 'different.' "Two things are different about the interviewer: (1) It is anony- mous. Every safeguard must surround the anonymity of the re- spondents. To ask for him by name, as in the first field experi- ment, is a fatal error. He must be given every conceivable assurance that nothing he says will be or can be associated with him in any way. (2) The interview is also unique in the relation- ship developed between interviewer and respondent. People must learn about this relationship from experience: they must be trained to be respondents. Hence, questionnaires should be designed with introductory sequences whose major purpose is to CASH LOANS: SECOND FIELD EXPERIMENT 93 assist the interviewer to convince the respondent that the inter- view is a special situation - - where the norms of reticence do not apply." What the manipulations were can best be conveyed by quoting the actual instructions to the interviewers: "The addresses in the sample have been divided on a random basis into three groups. We hope to compare the accuracy of response from group to group. The groups are as follows: "A. Respondents who fall in the A part of the sample will be approached in the same way as we usually approach respondents on financial surveys. There will be one unusual feature: we will know in advance the name of the person to be interviewed and before leaving the respondent you should check that you have the right respondent by name. You may wait until the end of the interview and ask his name in the context of offering a report and the mention of a mail follow-up by the Ann Arbor office. "B. The procedure here will be identical with that for names in the A part of the sample except that no use should be made of the name of the respondent in the interview . We will rely on the address and his report of how long he has lived there to make sure we have the right respondent. (We will send his copy of the report and any mail follow-up to 'Head of Household.') "You should ask for the male head or female head of the family according to whether the name given you is that of a man or woman. And, if you can, check to see that the name matches the name on a mailbox. But the essential thing is to avoid any use of the name in the interview. Be careful not to let the respondent see the cover sheet with his name on it. "C. The C part of the sample will be similar to the B part in that no use will be made of the name of the respondent , nor will he be asked for his name. The C interview will also differ in that the following techniques will be used here (but not in A or B): (1) The sealed envelope technique: The financial data in the last section of the interview are to be detached from the rest of the interview in the respondent's presence, placed in a stamped envelope addressed to the Survey Research Center, and the envelope sealed. You as interviewer may offer the respondent the option of filling out himself this section of the interview so that you never seethe answers. In any event you should tell the respondent that you will mail the envelope in the nearest letterbox and you should do so. You should invite him to come with you to the letterbox. Do not leave the envelope for him to mail - - people are too forgetful. 94 AN INVESTIGATION OF RESPONSE ERROR (2) Special explanation of how interviews are processed: We are asking you to make to the respondent a special explana- tion of how the factual material in interviews is tabulated. This explanation will involve three 'props': an I.B.M. card, instructions on the last pages of the interview for keypunch- ing the material in the interview onto I.B.M. cards, and a sample report. The idea is to make clear to the respondent the impersonal way in which we actually process such data. How much detail you go into will depend on your assessment of the situation in the interview. (3) Special explanation of purposes of the study and its useful- ness to the respondent: We want to make a special effort to explain the usefulness of the research to the respondent and, if possible, to lead him to see it as contributing directly to something he himself wants. The most hopeful possibility seems to us that he may be interested in how other people handle their financial affairs. (In our pretests, it appeared that most respondents were quite curious about how others handled their finances and how he compared with them. This might be worth playing on.) Do they have a formal budget? How does it work? Do they keep records? How do they handle the problem of living within their income? People who have no problems themselves may be amenable to the idea of passing on to others how they do it. (See the question about advice to a young couple in Schedule E.) We will undertake to prepare a report about these topics which we will send to respondents on this study. (This report may be mentioned to A and B respondents at the close of the inter- view. It should be 'sold' to C respondents.)" The Questionnaire The questionnaire itself appears in Appendix B. It is basically a fixed-question, free-answer schedule of the general type commonly used by the Survey Research Center in economic studies other than the Survey of Consumer Finances. Some use was made of fixed- answer questions. The topics of the questionnaire in the order asked were as follows: Current economic attitudes Housing Children, their education, and how their college education may be financed Financial management Attitudes toward borrowing Sentence completion items on the use of money CASH LOANS: SECOND FIELD EXPERIMENT 95 Fixed alternative questions designed to measure two dimen- sions of personality Personal data Financial information, covering liquid assets, mortgage pay- ments, cash loans, and instalment purchases The financial information was on a separate form suitable for separate mailing in a "sealed envelope." It should be noted that the sequence of questions was designed to follow out the reasoning that people must be trained to be respondents. The sequence on general economic attitudes was one that has been used in many studies by way of introduction. People's houses and their children seemed likely to be interesting topics to respondents and easy for them to talk about. The emphasis on financing education and the extensive discussion of patterns of financial management (budgeting, planning, and so forth) helped to turn the interview toward financial data while avoiding as yet asking for precise and detailed data on financial affairs. The psychological material for the most part served the same purpose, though some items must have seemed like a digression to many respondents. In this way a careful attempt was made to lead up to the significant financial questions. After the interview was over a mail questionnaire was sent to respondents asking for their reactions to the interview. The results of this re interview will appear in Chapter VIII. Error of Non-Response A total of 138 selected addresses fell into the sample. This count excludes those addresses omitted because the selected name and address did not check out against the most recent telephone book or city directory and those addresses at which the interviewer found that the respondent had moved away. Also excluded are the "dud" addresses of persons not known to have a cash loan which were des- cribed in the previous section. The interviewers were successful in obtaining an interview at 73 percent of the selected addresses. No special effort was made to keep down the number of not-at-home and miscellaneous non- responses in this study. The investigators wanted a sample of about 100 persons known to have cash loans, and did not insist on repeated call-backs after that number of interviews had been taken. The pro- portion of refusals is more revealing. It was 9 percent, which is approximately the level in the annual Surveys of Consumer Finances. On the study of owners of savings accounts in the first field study, it will be recalled, the refusal rate was 20 percent. Reports from the interviewers confirm the difference which these statistics suggest. From the interviewers' point of view, the study of cash borrowers went smoothly and even easily, in contrast to the study of owners of 96 AN INVESTIGATION OF RESPONSE ERROR savings accounts. In other words, in the study of cash borrowers the strategy of starting with "easy" topics and working up carefully to the "hard" topics seems to have been reasonably successful. Were there differences in response disposition attributable to the differences in interview treatment? Such variations in refusal rate from one treatment to another as appear in Table 35 are small enough to be attributed to random fluctuations. There was one group of interviews lost subsequent to the com- pletion of interviewing. Respondents were asked about the cash loans on which they were paying as of the first of the month in which the interview was taken. It was planned to proceed rapidly enough from sample selection to interview so that few or none of those selected would have paid off their loans before that date. Actually, 5 percent of the sample was lost in this manner. These respondents are excluded from all but one of the remaining tables in this chap- ter. The main analysis is based on 94 interviews with respondents known actually to have cash loans as of the date discussed in the interview. Table 35. Response Disposition for Persons Known to Have Cash Loans in Relation to Experimental Manipulation of Felt Anonymity by Type of Interview (Percentage distribution of addresses in sample) Response disposition for persons known to have cash loans All interviews Interview treatment a A B C Interview Had loan then Had no loan then Non-interview Refusal Other reason 73 68 5 27 9 17 73 68 5 27 11 16 79 72 6 21 6 15 68 64 4 32 11 21 Total 100 b 100 100 b 100 Number of borrowers 138 44 47 47 A. Interviewer checks before leaving that she has right respond- ent by name. B. No use is made of respondent's name. C. No use is made of respondent's name; sealed envelope tech- nique. Detail will not add to total because of rounding. CASH LOANS: SECOND FIELD EXPERIMENT Response Error 97 The result of this study in terms of response error are sum- marized in Table 36. Of the respondents 39 percent reported the known loan. An additional 49 percent reported other cash loans or instalment purchases but not the loan in question. In some instances it was difficult to ascertain whether the respondent was reporting the selected loan incorrectly or reporting a different obligation. The criteria used in making a judgment on this point will be described later. Finally, 12 percent of the respondents reported no loans. Since the judgment of the investigators entered into the establish- ment of criteria as to whether the known loan was reported, Table 37 was prepared on a different basis. It shows the number of cash loans reported by the respondent. In preparing this table the respon- dent's answers to the questions about cash loans were tabulated with- out any attempt to re -classify his obligations as between instalment purchases and cash loans. Of all respondents, 62 percent reported at least one cash loan. Thus, a number of respondents reported at least one cash loan but failed to report the selected loan. This table includes the respondents who had paid off the selected loan. If they were omitted, the proportion reporting a cash loan would be 65 per- cent. The procedure of indicating the proportion who reported some loans but not the specific cash loan under study is carried through Table 36. Response Error for Cash Loans in Relation to Interview Treatment (Percentage distribution of interviews) Response error for cash loans All interviews Interview treatment a A B C Reported known loan Reported other loans only Reported no loans 39 49 12 30 56 13 38 44 18 50 46 3 Total 100 99 b 100 99 b Number of respondents 94 30 34 30 a A. Interviewer checks before leaving that she has right respond- ent by name. B. No use is made of respondent's name. C. No use is made of respondent's name; sealed envelope tech- nique. b Does not add to 100 percent because of rounding. 98 AN INVESTIGATION OF RESPONSE ERROR Table 37. Number of Cash Loans Reported in Relation to Interview Treatment (Percentage distribution of interviews) Number of cash loans in All interviews Interview treatment a family as of first of month A B C Reported some cash loans 62 56 54 75 One 29 31 19 38 Two 29 19 32 34 Three or more 4 6 3 3 Reported no cash loans 34 34 43 22 Not ascertained whether respondent had cash loans 5 9 3 3 Total 101 b 99 b 100 100 Number of interviews 101 32 37 32 a A. Interviewer checks before leaving that she has right respondent by name. B. No use is made of respondent's name. C. No use is made of respondent's name; sealed envelope technique. b Does not add to 100 percent because of rounding. most of the tables in this chapter. People who admit these other borrowings are cooperating at least partially with the interviewer. It is worth noting that a majority of those who reported any cash loan had more than one such loan at the time of interview. Both Table 36 and Table 37 show an improvement in accuracy from the A interview treatment to the C interview treatment. Of the A respondents, 30 percent reported the known loan, compared with 38 percent of the B respondents and 50 percent of the C respondents (Table 36). Similarly, the proportion who reported that they owed money on a cash loan rises from 56 percent of the A respondents and 54 percent of the B respondents to 75 percent of the C respondents. Whether the differences between treatments are statistically signifi- cant depends on which measure of the difference is selected. The difference in Table 36 between the proportion reporting the known loan for C and the proportion reporting it for A and B combined is not significant (using the "two -tail" test). The difference in the same table between the groups in the proportion reporting no loans is significant. Thus, although the differences are not large from a statistical point of view, they are in the predicted direction, and CASH LOANS: SECOND FIET.D EXPERIMENT 99 they are large enough to suggest strongly that the C technique is a definite improvement on the A technique. Criteria Used in Identifying the Selected Loan Four criteria were used in identifying the selected loan. Perhaps the most satisfactory way to describe these criteria and how they were used is to reproduce the instructions to the coders, which follow: Does any cash loan or instalment purchase sound like "our" loan --that is, the loan reported on the Lender Form? Consider all cash loans and instalment purchases as the same thing, i.e., a loan, for purposes of selecting the loan. Accept as "our" loan a reported loan which "comes close" on any two of the four criteria below, with "comes close" defined as follows: (1) Amount of payment per month (or per payment period); it comes close if it is within + $4 or 20 percent of the actual amount on the Lender Form, whichever gives the nar- rower range. (2) Purpose ; to be close, any specific information given must agree in whole or in part. (3) Lending agency ; to be close means in the same code category. Actual lending agency is always "personal loan company or small loan company." (4) Total number of payments originally agreed upon ; to be close means within 2 payments. (NOTE: If loan has been refinanced or renewed and R confounds the two sets of payments, coder may substitute "Number of Payments Left to Make" with close defined, again, as within 2 payments.) If there is no loan which "comes close" on two of the four criteria, do not accept any loan as "our" loan. If there are two or more loans which could be taken as "our " loan according to the above criteria, select the one which comes closest on criteria (3) (lending agency) and (1) (amount of payment). In practice the most important criteria were the amount of the payment per month and the identification of the type of lending agency. Of the 37 reports of loans accepted as reports of the loan under study, 92 percent were reported as coming from a small loan company or personal loan company (Table 40). It was rare for a re- spondent to report a loan which checked out as the selected loan on two of the other criteria without correctly describing the lender. The agreement on amount of payment per payment period also was close. Of the reports of loans accepted as reports of the loan under study, 65 percent were accurate within one dollar (Table 38). Only 16 percent were in error by $5 or more. 100 AN INVESTIGATION OF RESPONSE ERROR Table 38. Response Error in Dollar Amount of Payment per Payment Period 3 Number Percent Respondent understated by $5 or more Respondent understated by $2 - $4 Respondent was accurate with 1 $1 Respondent overstated by $2 - $4 Respondent overstated by $5 or more Not ascertained from lending records 3 2 24 3 3 2 8 5 65 8 8 5 Number of known loans reported, and total percent 37 99 b a Criterion: with + $4. Does not add to 100 percent because of rounding. A majority of respondents had the total number of payments cor- rect, but many were in error on this point (Table 38). It must be recalled that the count of the number of payments may be complicated by missed payments. By contrast, agreement as to the purpose of the loan was less close. The comparison of purpose could be made only for 37 inter- views; this was true also of the other comparisons in Tables 39, 40, and 41. In nearly half of the cases, however, the purpose of the loan as stated in the interview was different from that reported by the lender. As to the amount of the loan and number of payments, the reports received from the lenders must be presumed to be correct unless a clerical error is involved. The nature of the lending agency also is known. But there is no assurance that the lender is better informed than the interviewer as to the purpose of the loan. Either report may be correct. The accuracy of report of other items of information about the selected loan is indicated in Tables 42 and 43. These tables are based on the same 37 interviews, those in which the respondent reported a loan which was accepted as the selected loan by the pro- cedure described earlier. The two variables shown in the tables were not taken into account in deciding whether the reported loan "came close" enough to be accepted. The first section of the table indicates that respondents have an understandable tendency to overestimate the number of payments which they have already made. This tendency is matched by a tendency to underreport the number of payments still to be made. This comparison is subject to some margin of error since the investigators sometimes were not certain how many pay- ments had fallen due between the date of the known information and the date of interview. It is unlikely, however, that they were often in CASH LOANS: SECOND FIELD EXPERIMENT Table 39. Response Error in Purpose of Known Loan* 101 Number Percent Respondent gave one purpose which was the same as that shown on lending records Respondent gave one purpose which was in same broad category but not exactly the same Respondent gave a purpose that was different from that shown on lending records but could have referred to same financial event Respondent gave a purpose different from that shown on lending records Not ascertained from lending records 8 3 8 16 2 22 8 22 43 5 Number of known loans reported, and total percent 37 100 a Criterion: any specific information given must agree in whole or in part. Table 40. Response Error in Report of Lending Agency Number Percent Correct: From a small loan company or personal loan company Incorrect: From a savings institution From a retailer From a sales finance company 34 1 1 1 92 3 3 3 Number of known loans reported, and total percent 37 101 a Does not add to 100 percent because of rounding. 102 AN INVESTIGATION OF RESPONSE ERROR Table 41. Response Error in Total Number of Payments Originally Agreed Upon 3 Number Percent Respondent underreported 8 22 Respondent reported exactly same total number of payments shown on lending records 21 57 Respondent overreported 4 11 Comparison not possible 4 11 Number of known loans reported, and total percent 37 101 b a Criterion: within t 2 payments. b Does not add to 100 percent because of rounding. error by two or more payments, which is the margin of accuracy shown. Income The relation between income and response error for reports of cash loans is shown in Table 44. Income is significantly and negative- ly related to accuracy of report of cash loans. Of those families with incomes under $5,000, 58 percent reported the known loan; 31 per- cent of the families with incomes between $5,000 and $9,999 reported the known loan; and a mere 11 percent of the families with incomes of $10,000 and over, reported the known loan. Before attempting an interpretation of such figures, it should be noted that income is even more strongly related, but in the opposite direction, to the "reported other loans only" category of response error. This latter error category is characterized by a very high number of reports of instalment purchases, while the "known loan" category pertains only to the cash loan of interest to the investigators (although it might have been reported by the respondent as an instalment purchase). Considering both income effects at once, it is immediately clear that family income is a variable which is proximate and sensitive to Table 42. Response Error for Number of Payments Made Number Percent Respondent underreported by 2 or more payments Respondent was accurate within - 2 payments Respondent overreported by 2 or more payments Comparison not possible 3 19 11 4 8 51 30 11 Number of known loans reported, and total percent 37 100 CASH LOANS: SECOND FIELD EXPERIMENT 103 Table 43. Response Error for Number of Payments Left to Make Number Percent Respondent underreported by 2 or more payments Respondent was accurate within - 2 payments Respondent overreported by 2 or more payments Comparison not possible 15 14 3 5 41 38 8 14 Number of known loans reported, and total percent 37 101 a a Does not add to 100 percent because of rounding. the type of response error under investigation in this study. A more elaborate interpretation would run along the following lines: The self-concept of the adult individual includes a perceived status position, a good part of which is a function of the level of income. The perceived self-status involves a large range of values and behavioral expectations, such values and expectations being generated by the normative behavior of those persons who form the status reference group for the individual. Now, for a per son with a high or moderately high income to admit that he had to borrow cash to meet his expenses would be incongruent with his perceived status position. Such dis- approval would not exist for the very low income families, since cash borrowing is a much more common phenomenon in that group and is logically more congruent with their financial position. On the other hand, instalment purchases do not meet with disapproval in Table 44. Response Error for Cash Loans in Relation to Income of Family (Percentage distribution of interviews) All interviews Income of family 3 Response error for cash loans Under $5,000 $5,000- $9,999 $10,000 and over Reported known loan Reported other loans only Reported no loans 39 49 12 58 28 14 31 58 10 11 78 11 Total 100 100 99 b 100 Number of respondents 94 36 48 9 a Excludes 1 respondent for whom income was not ascertained. He reported other loans only. b Does not add to 100 percent because of rounding. 104 AN INVESTIGATION OF RESPONSE ERROR Table 45. Response Error for Cash Loans in Relation to Education of Head of Family (Percentage distribution of interviews) All interviews Education of head of family Response error for cash loans Eighth grade or less High school College Reported known loan Reported other loans only Reported no loans 39 49 12 33 46 21 41 48 11 47 53 Total 100 100 100 100 Number of respondents 93 a 24 54 15 a Excludes 1 respondent for whom education was not ascertained; he reported other loans only. the higher income groups, and persons in those groups, if they re- port any cash indebtedness, will be much more likely to report instalment purchases. The low proportion of low income respondents reporting "other loans only" (28 percent) is explained by the fact that a majority of those people had already been assigned to the higher priority "reported known loan" category. Education Table 45 shows the relation between education and response error. The table shows that education is positively correlated with reporting the known loan, a surprising result. Income is negatively correlated with reporting the loan, as just discussed, and income is positively correlated with education. Evidently it is specifically high economic status which leads to reluctance to report cash loans rather than socio-economic status in some general sense. It is also possible that people with higher education are more favorably inclined toward social research. Occupation There was no relation between whether the head of the family was employed in a white-collar or blue -collar position and response error in reports of cash loans (Table 46). Exactly 40 percent of those in each occupation group reported the known loan. The result is in con- trast to that found in the study of response error in reports of large savings accounts in the East Coast area. It will be recalled that in CASH LOANS: SECOND FIELD EXPERIMENT 105 Table 46. Response Error for Cash Loans in Relation to Occupation of Head of Family (Percentage distribution of interviews) All interviews Occupation of head of family Response error for cash loans White-collar worker Blue -co liar worker Reported known loan Reported other loans only Reported no loans 39 49 12 40 53 7 40 45 15 Total 100 100 100 Number of respondents 92 a 30 62 Excludes two respondents for whom occupation was not ascertained. that study there was a substantial difference between occupation groups in accuracy of report. This contrast between the studies parallels the contrast in the relation between income and accuracy. This topic will be discussed further in a later chapter. Age of Head With regard to age of head the data from the present study do confirm the tendency which has been found in the earlier parts of the investigation for people under 45 to report more accurately than those in the older age groups. The difference in the proportion of accurate reports, shown in Table 47, is near the margin of statistical signifi- cance. The difference in the proportion who report no loans at all, however, is easily significant at the 95 percent level of confidence. This difference may be the result of differences in attitudes toward borrowing from one age class to the next. Sex of Respondent Most of the respondents in this project were men. The small size of the sample of women respondents makes it impossible to rely on the differences shown in Table 48. The data do point in the direction of more accurate reports from male than from female respondents, but the results are not statistically reliable. Relationship of Borrower to Respondent In this study the interviewers attempted to interview the person who had borrowed the money. They were instructed to interview the 106 AN INVESTIGATION OF RESPONSE ERROR Table 47. Response Error for Cash Loans in Relation to Age of Head of Family (Percentage distribution of interviews) Response error for cash loans 1 ' All interviews Age of head of family 3 Under 45 45 and over Reported known loan Reported other loans only Reported no loans 39 49 12 51 49 31 48 21 Total 100 100 100 Number of respondents 94 41 52 a Excludes 1 interview in which age of the head was not ascertained. head of the family if the name on the list given to them was a man. If the name was that of a woman, they were to interview the wife of the head. The results in Table 49 indicate that in 85 out of 94 of the cases under study the interviewer, in fact, was successful in inter- viewing the borrower. Since there were only 9 interviews in which the borrower was not the respondent, not too much can be made of the comparison of accuracy of response between the interviews where the borrower was the respondent and where the borrower was not the respondent. The data do point in the direction of more accurate Table 48. Response Error for Cash Loans in Relation to Sex of Respondent (Percentage distribution of interviews) Response error for cash loans All interviews Sex of respondent 3 Male Female Reported known loan Reported other loans only Reported no loans 39 49 12 45 47 8 19 50 31 Total 100 100 100 Number of respondents 94 75 16 1 Excludes 3 respondents for whom sex was not recorded. CASH LOANS: SECOND FIELD EXPERIMENT Table 49. Response Error for Cash Loans in Relation to Whether Borrower was Respondent (Percentage distribution of interviews) 107 All interviews Relationship to borrower to respondent Response error for cash loans Borrower is respondent Borrower is not respondent Reported known loan Reported other loans only Reported no loans 39 49 12 42 47 11 11 67 22 Total 100 100 100 Number of respondents 94 85 9 replies when the borrower was himself the respondent in the inter- view. This result, of course, is in the direction that one would antici- pate. Attitudes Toward Borrowing In this study a careful attempt was made to measure the attitudes of the respondents toward borrowing money. The procedure used was to ask the respondents questions of the following type: "How do you feel about borrowing or using credit to cover the expense of a vacation trip? Do you feel it is usually a good idea? Sometimes a good idea? Never a good idea?" This type of question was repeated for the follow- ing other possible purposes of borrowing, as well as for "vacation trips": to pay taxes, to purchase jewelry, for educational purposes, to cover expenses due to illness, to pay bills that have piled up, to purchase a car, and to purchase furniture. A composite score was constructed from the responses to these eight questions. The method was to assign two points for each item to which the respondent said "usually a good idea" and one point for each item to which a re- spondent said "sometimes a good idea." The possible total score thus ranged from to 16. The relation between this score and accuracy of response error is shown in Table 50. The hypothesis was that those individuals with a favorable attitude toward borrowing would be more willing to report any debts which they might have incurred, and, in particular, would be more willing to report the known cash loan. The results tend to support this hypothesis. Of those with unfavorable attitudes toward borrowing, that is, those who score to 5, 30 percent reported the known loan. Of those with the most favorable attitudes toward borrow- ing, that is, a score of 10 to 13, 47 percent reported the known loan. 108 AN INVESTIGATION OF RESPONSE ERROR ti o •i-H d CD Oh hf) ti H ■^H •r-l rn £ ti o (Ti Sh o o Ph -ti rn 73 d O Sh fn o O H O J-i CQ cd 73 ti -I-' •iH -M CD ti n (J o SP CO CD O) CD Sh O O Sh O CD CO Ph Ph o ■8 % „, CO ^ * H « xj A J2 bD 15 2 c- CO CD CD i-H ti Sh ■"■ ^ ■^ CD CM O o cd o E«H O 73 Fh cd ^— ^ £ CD o © ,rt H_> r-i CD T3 s» o to •^ O O CO -3 © ■^ m O LO CD •fh Sh i—l 73 3 § ° -P CO •iH -M -M cd ti o CD Sh O O CO r-H LO .a i cd O Sh O CD > Sh o CD O J2 CD CO CO CO CO CD CM cd O £ CO tD^ CO ^-1 -I-H CD CO CD CM o ^ <3 Sh ■^ i-H o i— 1 CD CD -M Sh ■rH CO c cd O I-H .ti CO cd O >> Sh O Sh SH O CQ -4-> Sh c cd o f— 1 CQ Sh O a CD Sh Cd CQ C d O 73 Sh o ti CD CD CO c O ti Sh CD O a CQ o Sh .ti H-> o 1 — 1 cd CD U 4i 73 o 73 a 73 -t-> O SH O CD CD CD CD Sh CD Ph Sh -t-> Sh -t-> u a O a ,0 s g ^ CD CD Oh Oh Dh 73 73 Sh ti CCi cd Sh CQ "cu 32 > 73 ? ti X3 cd « Cd -m ■=! ■SB, bD-2 ti 3 CG~ CQ CQ Cd CD ti s s ti .iH £ O S © ^S bD ^ •rH ~ <^ Sh 2 » CQ ^ CD CD cq o ; • ti ti » O Cvi & a c o CB ;rH I— j 2 « o h 0 -(->-(-> +-> C ti c c • rH -iH -i-l -rH O O O O ft a a a CM i-H o o cd CD ^2 CD 73 73 •rH O 73 9 o fan CQ d CD ^.§ « S !=> CO cd CD 73 •l-t 73 O O £ bfl O c cd r" > ti CD O !? Q a* CQ I CQ ti O CQ CD & • rH CD CD ,ti .ti O cd CD Sh O CQ CD Sh O O CQ CD .ti CD CO bD C ■rH 73 ti ti O Sh • CD CD CD S U CD cd O w ti ti O CD ** U o ti^ s > 73 CD § cd a ti°o Sh *^ CQ P CD ^ Sh 73 O 73 CD O Cd <-> CQ , , CQ CD a^2 CQ ti CQ CD O CQ O ar CASH LOANS: SECOND FIELD EXPERIMENT 109 On the other hand, of those with unfavorable attitudes toward borrow- ing, 30 percent reported no loans at all. Of those with either middle or favorable attitudes toward borrowing less than 10 or 9 percent reported no loans at all. Evidently, people who are ashamed to bor- row money fail to report their borrowing in an interview situation. The existence of this relationship may have unfortunate implica- tions in some circumstances. Frequently it is important to measure the relationship between people's attitudes and their behavior. These results suggest that a relation between a favorable attitude toward, say, borrowing, and frequency of borrowing may appear in the results of surveys for spurious reasons. A favorable attitude toward the be- havior may lead to more complete reporting of the behavior; those with favorable attitudes will appear to borrow more than those with unfavorable attitudes merely because those favorably disposed re- port their borrowing more completely. These results have implications for the theory of response error and for the design of questionnaires which should be kept in mind in survey work. Conformity Score In studying the results from the interviews taken in the fall of 1958 with owners of large savings accounts, the investigators speculated that conformity tendencies of respondents might be important. They were especially interested in trying to understand why some respondents would fail to report the existence of a financial fact while others would admit the existence of a savings account but openly refuse to tell the interviewer anything about the balance. They speculated that people with a high tendency to conform to social dicta might find it difficult to refuse outright to cooperate with an interviewer in what is, after all, a social situation. In order to measure the importance of this variable, the investi- gators constructed a special conformity score. The items in this score were the following: H2. In a social gathering, if you find you are not dressed like the other people there, does it make you feel uncomfortable or don't you care very much? Makes me feel Don't care uncomfortable very much H3. When people disagree with you, do you sometimes start to wonder whether you're right, or do you nearly always feel sure of yourself even when people disagree with you? Wonder Feel sure 110 AN INVESTIGATION OF RESPONSE ERROR H5. Do you feel that children ought to be brought up to be dif- ferent from their playmates or as much like them as possible? Ought to be As much like them different as possible H7. Do you feel that people are better off if they think and act like the people they associate with or if they stand out as being different? Think and act Stand out like associates as different A score was constructed by adding together the answers to these items. In some interviews respondents insisted on a middle position to one or more items, or said they "didn't know." The scoring proce- dure was to count two for a "conformist" answer, one for a middle position or "don't know," and zero for a "non-conformist" answer. Thus, the low score indicates a low tendency to conformity, and the high score indicates high conformity, with a possible range of 0-8. The relation between this score and response error is shown in Table 51. The proportion who reported the known loan went from 20 percent of those with low conformity to 40 percent of those with low middle conformity, and reached a peak of 71 percent of those in the middle of the range, then fell to 43 percent of those in the high middle range and 18 percent of those with high conformity scores. It is a remarkably symmetrical relationship, with the peak of accuracy in the middle range of the conformity score (Table 51). The investigators cannot claim that they predicted this relation- ship in advance. After the event, of course, it is not impossible to fit a hypothesis to the data. It may be that people with low conformity scores are not susceptible to the influence of the interviewer and do not respond to her request for information. At the other end of the scale, people with high conformity scores may be unwilling to admit to such doubtful behavior as borrowing cash from a small loan company. Those in the middle conformity range, on the other hand, seek the approval of others just as the extreme conformers do, but the middle conformers are less anxious about it and therefore can take the time to allow a definition of what is "right" and "wrong" in the situation to develop on the basis of cues from the interviewer. Thus, they will tend to be more subject to influence by the interviewers than will either the very low or the very high conformers. This result must be taken, for the present, as no more than a suggestive clue for further speculation and research. CASH LOANS: SECOND FIELD EXPERIMENT 111 DO c « ^ r* O -&„_ ^ S . o u S ^ o 2 W 5 s " o fi § -2 aco M ° m "* ■9 - -C go" CO tH CM i-H C- bo „ 1— 1 c- i—l o i— l at: 1— I T3 t3 _ 1". CO CO ■^ o CO "^ rt< i—l o CM ^e 1—1 ba (S •■H a o u 09 •i-i T-l i-H t- ■**» T*< a u t- CM OS i— 1 S o ■a o U i—i T3 ^ ■S co a - o CO ■<* o in •<* IT3 o CM ^ eg 1— ( £ — o J £ ^ o O o o o o « CM ID CO o i—i iJ o 1—1 CO £ Oi OS CM o •* CO Tf i-t o o> (I) -l-> c • t-H CO c rt £ >> ,£3 i—i CO o CO -4-> o o c cct o 1—1 CO a o i—i CO c o !h c a a O u o a X! -4-> o o 13 CO .3 o c -M =*-H T3 T3 TS o Eh o CO u -t-> -M +j O h U h a CO a o O a 3 « « « tf 55 cd u o CO co S3 O > O S -o e ° o ° .SP rt s .a o "S -M C s C3 In h CO CO 2 . o .2 «« S> X> CO K* Cvi * a s o o ■« ■H H fl O « ^ o o O " r ti 5 o o S CO co m a) cct o CQ £ Q B O J3 112 AN INVESTIGATION OF RESPONSE ERROR Personal Effectiveness In addition to the conformity items discussed above, four other forced-choice items were used; they were designed to measure the respondent's perception of the effectiveness of his own plans and actions as follows: HI. Have you usually felt pretty sure your life would work out the way you want it to, or have there been times when you haven't been very sure about it? Pretty sure Sometimes not very sure H4. Do you feel that you are the kind of person that gets his share of bad luck, or do you feel that you have mostly good luck? Mostly good luck Bad luck H6. Would you say that quite often you have trouble making up your mind about important decisions, or don't you feel you ever have much trouble making up your mind on important decisions? Not much trouble Quite often H8. When you make plans ahead do you usually get to carry things out the way you expected, or do things usually come up to make you change your mind? Things work out Have to change as expected plans The items were scaled by a modified Guttman technique and scale scores were assigned to the individual respondents, with a score of indicating a low personal effectiveness self-percept and a score of 4 indicating high personal effectiveness. The relationship of perceived personal effectiveness to response error is shown in Table 52. Of the low effectiveness respondents 71 percent reported the known loan, While only 20 percent of the high effectiveness respondents were accurate. The difference is statistically significant beyond the 95 percent level, and the relationship between response error and effectiveness is negative. The investigators were gratified to find that their hunch about CASH LOANS: SECOND FIELD EXPERIMENT 113 TO a m m a CO £ o a; a> H-l > si > & TO • rH ei o c U CD -i-i o W © u-t ,-H C u rt 2 o C £ u O 3 u CO „Q W H CD Ch^ TO c O 73 o -M CQ 0) 1 SP tf ts £ ^H •rH +J o 4H CO CO CO o CM o CO CM o °tH ■^ "* i— 1 o T-H OS -4-> a •l-H CO a «J O i— i J5 1 X! co c «J o o (h CO" CO CQ O o r— 1 ctf O CO c CD • l-H > U c Oj rH o £ u CD o r-H CD H-> o £! l-H c c -M o cti •i-H r3 o a o Eh «4-l co c 73 CD 73 CD 73 0) o +j h-> +J CD rQ a o Sh (4 fn CO O a o ft OJ CD CD 3 1 P « tf Cd £ > • i-H H-» St <4-H O CD CD I I -M CD S c w " CI* TO w 3 b ^ O 3 ,=2 CD > £ CD 73 ft °> rn ■— ! « TO -G CD CD Jh ^ CD O 5 s TO bD cd "H ai; a •§> CQ 73 CQ C o . c -m J3 co E-, > CT CQ S <*H O ™ °"cS O o • l-H TO a ■a c • l-H ret H-> rH O TO T3 CD CO cj C h 2 2. £ S-H rH o *3 -5 CO i— l C TO § O CQ O 73 S "S rS O " Co w I— 1 U U Oh tf -t-> c hO • i-H •T-I ri «*H O CO c U «4-H ■i-H Tl T5 $H O c CD <*-l u ■H 3 SP +-> u £5 C O O W CO u CO < c o a CO 0) tf ITS IT5 i— i ,Q rt H u o o o o in x! CM co CM o CD i—l ^ - 1 ® CO ro -irt co -a 5-1 o ° m ^ c t3 :3 S ^ i—l T-H l-H ja CD Oi c- o t-l s •l-l 73 rt «J S b £ « ? 52 O co D- O lO a CM c- o l-H i— i o u <4— 1 T3 .Jh - fi ■y fn tie x "S3 O 45 S nJ ^ - -i-i +-> •i-l -M CJ O n? fi to ^ 2 -S (M o 00 O O i-H CO CO CO ci O <£ £ Ctf o 1 — 1 u ° a CO o bD S CO > CD O in CO CO i—l O o i-H CD CO o ® h » S b" a> sh 2 11537 3 — CO J3 £ a "g "c g * ° O ^ 3 i—i ^ ^ "tf tr- l-H i—l Oi i-H CO £ r-H .pH Ol Ol CM O ■^ ^ ^ CO ■^ i— 1 O i-H Ol +-> c •i-H CO CI S3 o r— < >> si CO l—l o CO q CO o T5 o o CO O f-l a d a o u Si o J! ■M o o 'cS CO u M o c o M-l o T3 T3 T3 H CO 5-! c -i-) -*-> +-> a O o O 42 a CO a a a 5 « tf ti PES !? (d xi CASH LOANS: SECOND FIELD EXPERIMENT 117 Interview treatment C ("sealed envelope" technique) Income below $5,000 College education Age below 45 Favorable attitudes toward borrowing Middle position on a conformity scale Low personal effectiveness The evidence is less clear but suggests an association between ac- curacy of report and the following: A male respondent The borrower being the respondent Keeping no records of money spent Purpose of borrowing socially acceptable There is little or no evidence of association between accuracy of report and the white-collar or blue -collar occupation of the head of the family. VII. REPORTS OF SAVINGS ACCOUNTS: THE THIRD FIELD EXPERIMENT The second of the field experiments conducted in 1959 was an additional study of owners of large savings accounts. The investigators were fortunate in obtaining the cooperation of a savings institution in another eastern city. This study was planned and carried out con- currently with the study of cash borrowers reported in the previous chapter. Purposes of the Study and Description of Procedures Purposes It was apparent as the results of the first field study came in that further effort could well be expended in the attempt to understand and, if possible, to reduce response error in reports of savings accounts. For the most part the arrangements for the experiment in the first study were felt to be satisfactory and were carried over into the new investigation. Some changes in the procedures in such matters as sampling and validation were introduced; these will be described later. The major changes, however, were in interviewing technique. Need for Anonymity In this project, as in its predecessors, it was of crucial im- portance to protect the confidentiality of data about the persons to be interviewed. The problem differed slightly from that in the first study since only one financial institution was involved. As was done in the study of cash borrowers, the precaution of including in the sample a number of name sand addresses which were simply selected from the telephone book was adopted here. These "dud" interviews, as they were familiarly called, were excluded from the analysis. Otherwise the procedures used were similar to those in the other field study of savings accounts. Sample Selection The sample was selected by the staff of the savings institution in accordance with plans developed in the sampling section of the Survey Research Center. The purpose of the sampling procedure was to obtain a probability sample of the savings accounts of the institu- tion exclusive of accounts with balances under $500 or over $15,000 and exclusive of accounts owned by minors, by organizations, or by anyone other than adults living in the metropolitan area or its vicini- ty. Account owners were dropped out unless a current address which 118 SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 119 checked with the telephone directory could be found for them. The basic method was to draw a sample of account numbers and drop out account numbers which did not meet the criteria set up. A two-stage sampling procedure was used, in which the first step was the selection of some of the branches of the institution from a list of all branches, and the second step was the selection of accounts within the branches. This procedure had two advantages: it made it unnecessary to set up a clerical operation to select ac- counts in every branch, and it tended to group together some- what the addresses to be visited by the interviewer, thus saving time in the actual interviewing. The procedure had a disadvantage, however: it made necessary the use of a different sampling interval in each branch. The result, however, was a random sample of all accounts which was self-weighting; that is, there was no need to use weights in the analysis to compensate for a variable sampling frac- tion. This procedure, of course, was more convenient for the analyst than that used in the preliminary savings account study reported in Chapter II. To summarize, if the account or the depositor was characterized by one or more of the following, the account and the depositor were dropped from the sample: (1) The account had been closed. (2) This was a Christmas Savings Account. (3) This was the account of a minor. (4) This was an account held in trust. (5) This was an organization account (e.g., religious, fraternal, or educational group). (6) The depositor's address was obviously outside the city. (7) The current balance in the account was less than $500 or ex- ceeded $15,000. Validation Procedure The procedure for validation was similar to that used in the pre- vious study of savings accounts. After completion of the interview- ing, the investigators transmitted a form to the savings institution and obtained information from it which enabled them to carry out this analysis. Experimental Manipulations This project, like the study of cash loans, was planned in the light of a developing theory of response error. The specific line of reasoning on which the experimental manipulations were based was set forth before the questionnaire was developed as follows: "The main difficulty with the interview situation is that the forces on the respondent to communicate are not strong enough. 120 AN INVESTIGATION OF RESPONSE ERROR What is needed is a new and powerful force to communicate. The money may go to them directly or to third parties designated by them. But the fact that the researcher will pay a substantial amount for information has two values. It not only rewards the respondent for his efforts, it indicates to the respondent that the interviewer sets a high value on the information asked for. The effect will be to make the respondent take the inquiry more seriously. "The attempt to increase positive motivation must be accompa- nied by an attempt to reduce forces in the opposite direction. If the procedure results only in an increase in conflict between op- posing forces, the result will be to raise the tension-level for the respondent. The consequences of an increase in tension are likely to be unfortunate; for example, it may lead to aggression toward the interviewer or to an attempt to escape from the situation. Hence, every effort should be made to reduce the threat to the respondent in the interview situation by such measures as as- surance of anonymity." The most satisfactory way to describe the manipulations them- selves may be by repeating the language used in the formal instruc- tion to the interviewers: "The interviews in the sample have been divided on a random basis into four groups. We hope to compare the accuracy of response from group to group. The principal difference from the point of view of interviewing procedure is that respondents in groups A and B will not be paid, while those in groups C and D will be given $10 each. A difference in sampling procedure will be introduced between A and C on the one hand and B and D on the other: in A and C the designated respondent will be the head of the family; if the family is a married couple, the respondent will be the husband. In B and D the designated respondent will be the wife of the head. In summary: Group - Reward Designated Respondent A No payment Head B No payment Wife of head C $10 Head D $10 Wife of head "After the interviews have been taken and analyzed it will be possible to compare the results from interviews with respondents who were paid and respondents who were not paid. The hypothesis is that those who were paid will give more complete and accurate information. "It will also be possible to compare results where the respon- dent is the wife with results where the respondent is the husband. SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 121 To interview the husband costs both extra trouble and expense, and it is proposed to measure the accuracy of the information from the husband to see whether there is a gain in accuracy corresponding to the effort expended. You should proceed as follows: 1. Locate the address on the cover sheet. 2. Locate the right family. The right family is the one which includes the person or persons named on the sticker on the cover sheet. a. Do not use the name on the cover sheet in asking for the respondent unless you are in doubt as to whether you have the right family. (1) If the address is a single family dwelling and the people have lived there a year or more, it is reasonably safe to assume you have the right family. (2) If the address is a multiple family dwelling or apart- ment house and the right name appears by the door- bell, it is reasonably safe to assume you have the right family. (3) If there are several families in the same building and no way to tell them apart, you may need the name. If R asks where you got the name, you may mention that we do sample from lists such as telephone books and city directories but that you do not know what the exact procedure was in this instance. For A and C addresses 3. The designated respondent is the head of the family. a. You should interview the husband if possible. It is better to interview the wife than to take a non-response for the family, however. 122 AN INVESTIGATION OF RESPONSE ERROR For B and D addresses 3. The designated respondent is the wife of the head of the family, if such a person exists. Otherwise, it is the head. a. When the designated R is the wife, you should make every effort to complete the interview with her. It will be ac- ceptable to leave the financial form with her and come back the next day to complete the interview after she had had a chance to consult her husband. Try to avoid coming back in the evening to see the husband yourself. However, if R in- sists that the husband is the one for you to see, you may talk to him. (It is not satisfactory to leave the financial form to be filled in and mailed later by R - - it would be too easy to forget to mail it.)" In these interviews use was made of a financial form which was to be mailed in separately in a sealed envelope. In this respect the procedure resembled that in the C interviews with cash borrowers. The instructions with regard to the use of the form read as follows: "The forms are sufficiently complicated so that we do not think it a good idea as a general rule to hand them to R and let him fill them out all by himself. We would like you to go through the form with him. Usually we expect you will enter the numbers. But if he understands what particular items he should fill in, it is satisfactory to let him enter them and seal the form in the enve- lope without showing the numbers themselves to you." Use of Respondent's Name In the first field study, it will be recalled, the respondent was asked for by name, while use of the name was controlled in the study of cash borrowers as part of the experimental manipulation. As had been indicated previously, the instructions in the present study were to avoid asking for the respondent by name unless there was no other way to resolve doubts as to whether the interviewer was talking to the right person. The interviewers often exercised con- siderable ingenuity in order to make certain that they had the right respondent while avoiding asking for him by name. The following illustrates a procedure that was used to identify a respondent. Faced with a private house with one bell but with five names over the bell, the interviewer noticed that the respondent's name was the fourth, so she rang the bell four times. When a woman came down- stairs, the interviewer explained that she hadn't known "how to get my fifty-eighth dwelling unit, but I wanted the fourth family who lived in this building." The woman answered, "Well, my name is fourth on the list, so I guess you must mean me." Thus, the right respon- dent was found. SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 123 The Questionnaire The questionnaire used in this project differed from that used in its predecessors in one major respect. The investigators believed that respondents should be paid for work done . Thus, they felt no compunction about asking for a fairly complete financial report. It was anticipated that the respondents would react better to the pay- ment if they could see that they were expected to exert themselves in order to earn it. Also, of course, the investigators were interested in a technique for obtaining complete financial reports. The question- naire appears in Appendix B. The topics of the questionnaire in the order asked were as follows: Current economic attitudes Financial management Banking, including a discussion of bank services used, choice of bank, and interest on savings accounts Attitudes toward borrowing from a bank or loan company Personal data Financial form Part 1. Family income (asked in detail) Part 2. Savings (five pages on liquid assets, purchases, debt, and related topics) After the interview a mail questionnaire similar to that used in the study of cash borrowers was sent to respondents. Results are reported in Chapter VIII. Interviewers' Reactions In planning this study the investigators took the position that the reactions of the interviewers to the project as a whole and to the experimental procedures would be of importance. The question- naire to be used was pre -tested by the same interviewers as those who were to participate in the main study. One of the staff spent several days with them in connection with the pretest and later training. The reaction of the interviewers to the idea of paying respondents was less than enthusiastic. As the study progressed they came to feel that the technique was not working well. A suggestion was made for improving the technique, which came, unfortunately, too late to be tried. The difficulty, it was suggested, may be that it is not easy 124 AN INVESTIGATION OF RESPONSE ERROR for the respondent to accept or reject an offer of money in the pres- ence of another person (the interviewer). In fact, the pressure which this puts on the respondent may well cause enough stress to pre- vent him from reacting in the way the investigators desire. It was proposed that the form be left for the respondent to fill out and mail in by himself, together with the request for payment. This arrange- ment would have emphasized the privacy of the information, which the interviewer would never see. It also would have given the idea of payment time to sink in: the respondent would be able to decide at leisure whether or not it would be worth his while to fill out the form and mail it to the University. As was noted earlier, this varia- tion on the procedure was not in fact used since it was not pro- posed until too late. Error of Non-Response In this study a total of 153 accounts fell into the final sample. The "dud" addresses selected from the telephone book were ex- cluded from this total, as were the addresses which proved to be in- correct. The interviewers were successful in obtaining interviews at 58 percent of these addresses as shown in Table 56. They were re- fused interviews at 29 percent of the addresses, and other reasons for not getting an interview accounted for 13 percent of the total. This response rate is extremely low. It is the lowest of the three experimental studies described in this section of the report. It will be recalled that the response rate in the first field study was 65 per- cent, with a refusal rate of 20 percent and other non-responses, 15 percent. The difference, in other words, is in the refusal rate. In the study of cash borrowers the refusal rate was 9 percent. Why were the interviewers so often unsuccessful in the present study? One place to look for an explanation is in the nature of the re- spondents. It may be argued that people with large savings accounts are difficult to interview. This explanation is consistent with the difference between the studies of savings accounts on the one hand and both the study of cash borrowers and the general experience of the Center in financial surveys on the other hand. The investigators are inclined to accept the interpretation as one element in the situation. The hypothesis that people with large savings accounts are hard to interview can not explain the difference in refusal rate between the first study and the third study (the present one). True, there was a difference between the cities in size of account. In the city in- volved in the first study, the mean value of all accounts selected in the sample was $3,478, compared with $2,572 in the present study (Tables 15 and 57). It will be recalled that the sample in the first experiment was selected with a minimum balance of $1,000 or more, SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 125 cd a < CD N •r-H CO cd H Oi in Oi o c o o o < CD co a a 10 0) cd CO 0) 43 tH S3 u ei § cd B CO CD =g o C 0) cd bp ^ ™ re -4-< C cd o !h 0) CM ^ 4) 8 J CM CO CM co o CO in •* CM CM o i—l CM Lf5 , % **c3 in Oi O Oi i—i o o> CM 00 CO m o 1— 1 „ o o> Ifi *# •^1 o i-H CM i—i co"-^ >> €/3-€/3- a g c l cd O Oi •-3 O Oi «tf CO c- Oi o CM o O Oi CO CO CM o i-H CM csTcnT CD m-m- o rt rt i 42 O Oi O Oi 1— 1 Oi CM t- o in Id 5 O Oi i—l i—l in "^ CO i— < o i-H CO o €«■€«- cd o 0) 1 N O Oi CD ■<^ 1—1 CO o CM CO O Oi CO CO CO o CO in Oi i-H #3-€/3- Oi ITi lO Ifi o o o l Oi CO CO 1—1 CM o CM O ■<*< i-H €«•€«- CO -t-> S o 00 CM Oi CO o CO < ITi •^ CM i-H o i-H in i— i o cd c o CO •wf +-> -M S3 •l-l CO o CO 3 o a CO •1-1 1 t—l cd o o cd T3 CD CO c o CO CD CO CD ■f-i > u CD > u CD S3 •t-l 1 a o CO =4-1 CD CD 43 -t-> o o H o u CD a 3 tf 5 fc fc Oi in Oi 3 co cd CD CO 3 oi O CD 43 O CD 43 Sh cd 13 cd ►"D Sh CD -t-> ■a T3 CD S3 CD a o a O o u cd Sh o fa 126 AN INVESTIGATION OF RESPONSE ERROR Table 57. Response Disposition in Relation to Mean Actual Balance in Account on January 1, 1959 a Mean actual Number of Response disposition balance accounts On January ' 1, 1959 Interview $2,394 89 Non-interview 2,819 64 Refusal 2,704 44 Not at home 3,287 12 Other 2,749 8 All accounts 2,572 153 a For accounts opened after January 1. used was July 1, 1959. 1959, the check-date compared with a minimum of $500 in the present study. But the evidence for a relation between size of balance and interview rate is not very powerful, and if anything, Table 56 shows some tendency for the response rate to be lower for accounts over, say, $3,000. The relationship in the first field study was similar. Thus, one would expect a higher refusal rate in the first than in the present study if size of account were the deciding factor, whereas the reverse is what happened. A second possible explanation is in terms of the differences be- tween the interviewing staffs in the cities in which the two studies took place. The staff in the "first" city is larger, and, therefore, more carefully supervised. One of the Center's staff of field super- visors lives in that city and maintains careful control over the field work there. This type of supervision is not carried out in the other city, where the present study took place. The difference in response disposition may reflect that difference. A third possibility is that the difference between the two projects is the result of the experimental manipulations in the present study. This possibility is explored in Table 58. The results in that table show a striking difference between paid and unpaid respondents. The interviewers were successful in obtaining an interview from two-thirds of the unpaid respondents but only from half of the re- spondents who were designated as falling in the part of the sample to be paid. There was no difference between proportions of the sample in which the head was designated as the respondent in contrast to those in which the wife was the designated respondent. SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 127 Table 58. Response Disposition in Relation to Preferred Interviewing Technique (Percentage distribution of accounts in sample) All accounts Pay No pay Response disposition Preferred respondent a Preferred respondent 3 Head Wife Head Wife Interviews Non- interviews Refusal Other 59 41 27 14 57 43 27 16 46 54 35 19 67 33 23 10 66 34 21 13 Total 100 100 100 100 100 Number of accounts 151 b 37 37 39 38 a Interviews were assigned to one of the two (head or wife) respondents by the central office; interviewers, though, were allowed to depart from the preferred respondent if the field situation called for it. b Excludes 2 accounts regarding which the wrong respondents were interviewed. The difference between the paid and unpaid respondents is rem- iniscent of the difference between the structured and unstructured interviews in the first field study. Again, the new technique is associated with a high non-interview rate. This result suggests that interviewers may lack confidence in using any dramatically dif- ferent technique. This lack of confidence may communicate itself to the respondents at the crucial stage when the decision is being made by the respondent concerning whether or not to grant an interview. The difference between the response rates for respondents in the paid and unpaid halves of this sample is by itself enough to indicate that the technique of paying the respondents as it was tried in the present investigation was not a success. If attention is restricted to the standard questionnaire of the first field study and the unpaid respondents in the present study, some difference in response rate in favor of the first study remains. As has been previously suggested, this difference may be a result of closer supervision in the city where the first study took place. Response Error The results of this study in terms of error in response are sum- marized in Tables 59 and 60. As shown in Table 59, about one re- spondent in four failed to report the selected account. Of these, how- ever about half failed to report an account which was owned wholly or in part by someone other than the respondent and his spouse. 128 AN INVESTIGATION OF RESPONSE ERROR cu o I ^ CD .S - H E i—l M bfl 0) >» 3 -a S-l -H S re £ "- 3 CD <+h S t! ° ^ S o =H M ,Q o g£ J-i J" 1 CO (l H O o c CO ^ o CD « ^ K .2 * re — ■ in tf CD i—i ■s re D. O « c cd C a CO 2 CD U tn CD 0) u CD =+h •i-i O CM 000000 1 CSI CO 1 CD CO H i-H i-H i-H 1 in 1 r^ o o i-H T3 re CD m u ■^ O "^ O O l (O^^ffl CSI t-H i— 1 r-t i— 1 I CHHCO O o i-H i— 1 CSI CM c cd o a CO CD u 0) U u cd <4-{ 0) $H Ph CD =4-1 •rH CQ CM O -^ •«*■ I ^tD 1 CO CO H N 1 CO iH 1 ^ o o i-H in CSI T3 re CD u CO H t-t-C- 1 Ift rP ^ C- iH i-l CN CM 1 IC) ^ o o 1—1 CD CSI All inter- views u cd i—i m m in i c» o ^ in CSI i-H i-Hi-Hi-l 1 in iH ^ o o j-4 CO CD t> c TO i— I re £i oT io »— 1 rH >> P. TO TO •-3 O ;-. o s-. u CD CD CO c o a CO CD K Failed to report account Account owned by respondent or spouse or the two jointly Account owned entirely or in part by someone other than respondent and spouse Reported account, balance not ascertained Reported account but refused to state balance Reported a total in several accounts but would not break it down b Reported balance for account Underreported by $1,000 or more Overreported by $1,000 or more Accurate within $1,000 r— < re o CO i •pH > CD •iH o u CD a * 2 cd rS a) co "re _g O S ® X2 CO S CD a CO CD u T3 CD h 5-. CD "ii •*-• CD CD h a o o •g £ ™ j_> <" t! e« a O CD > T3 S O CD +J *• s H-H O O T3 o «a O CD ■o - CD ?! C CD faD > •iH •!-! co bfl as CD CD II: % co, h3 CD « I T3 CD 5h O bdD CD C 43 CD o cm CD CD O (h fn CD CD CO a o a CO CD u O bX) CD -M re . CO 4J c o CO 2 CD ■* re o c CD O Jh CD -M Oh M CO ^ CO s o w SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 129 Of all respondents 15 percent reported the account but failed to report the balance in the account. In this study there were no cases in which a respondent reported a total in several accounts and re- fused explicitly to break down the total. It is possible, of course, that some respondents reported a total but failed to label it as such. Of the respondents 59 percent reported a balance for the selected savings account. Of these 45 percent were accurate within $1,000. This result compares favorably with that for the first field experi- ment where the response error in the report of the balance in the fall of 1958 was, if anything, somewhat higher. Only 34 percent of those respondents were accurate within $1,000. However, in view of the smaller size of the accounts in the present study, this dif- ference should not be taken as important. There was little or no difference between the paid and unpaid portions of the sample in response error. The observed differences in this respect, shown in Table 59, are well within the margin of sampling error. The technique of paying respondents failed to produce an improvement in accuracy in this study. (It is not known, of course, what might have happened if the technique used had been different. Specifically, it is not known what might have taken place if the technique proposed at the end of the previous section had been used.) The data also show little difference in accuracy between those interviews in which the head was the preferred respondent and those in which the wife was the preferred respondent. It will be recalled that the designation of the head as the preferred respondent did not require the interviewer to insist on interviewing the head, and the same was true for the wife. For example, the interviewer was per- mitted to interview the husband if requested to do so by the wife. In terms of the cost of sample surveys it is cheaper to designate the wife than the head as the respondent who is preferred. These data do not indicate that there is any advantage in preferring the husband if the objective of the project is to obtain accurate reports of savings accounts. Table 60 shows the mean actual balance and mean reported balance for groups of respondents corresponding with those shown in Table 58. For all interviews, the mean actual balance was $2,394. The investi- gators estimated a comparable magnitude, the mean reported balance, on the basis of the interviews. In this calculation the balance was entered as zero for those who failed to report the account. For those who did report a balance, the balance as reported was entered. For those who reported that they had an account but did not report the balance, the mean balance for those who did report a balance was assigned. The resulting estimate was $1,753. This amount is 73 per cent of the mean actual balance. In this respect the present study compares favorably with the first study where the same percentage 130 AN INVESTIGATION OF RESPONSE ERROR >> u o bD CD U 43 CJ w o *-. c o i2 CD QD O a 1=! cd o pa o o CD ^ h-> „ a i-T rt ^ m >i £< Jh § cd 3 T3 3 ... *^ 0) 5 cd Fh o OS .^ r-H Cd CQ 3 a -4-> o < a CD O CO a CO CD cd H ^ CM o (M CO CO ^ c- CO i-H Oi CM T-l i—l 1—1 I— 1 lO ■^ 00 3 & T3 CD +j fn d) 2 3 "* CM CO ^ CO & a efl M > •rH 3 CD in J2 (^ cd ■M OJ CO -M CQ CD CO CD i-H H u o a o a a a m O CQ cd o -4-> CD o a CD u o a >> CD ■ph t3 o -t-> u (H 73 a O a CO CD o °» a CO a CD -4-> -4-> a a o u • cd cd 1-3 -M 3 CD CJ c cd i—i a o o o cd o o i-H o o o O O u o <*-H O o o cd ■" a .2 a C Q. CD 57 cd £1 o ^5- ^2 i-H €«- >> i-T a -a -4-> •i-l o ■a o CD -^ M a a o CD O T3 CD XS T3 u $_, o si c a o CJ cd -t-> O a CD CQ CD CD a CD £ cd O J5 53 CD CJ Cd cd T3 O a CD 1 • i-H CO 3 -a 5 ~ -M CO CD -»-> > 5_i C3 O a ^ T3 4-> T3 U CD cd O Cfl -i-> a cd o> S-( CD u u U CD T3 CD O X3 +-> o -M CD u a -i-) O ?H ° rt f-l a 5h 73 CD o a CD o o o o O a CD CD O a CD a > o •rH tf • i-H < s CD 4h ?H O a h . — i cd o c\i rr! rft «»H T3 cd a° CO cd 01 4-1 a CD T3 a CD • i-i CD a CD CD CO co •i-i cd ■*-> r _ 1 efl ja SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 131 was 55 percent. The relation between actual and reported balance is shown also in Chart 3. Why were the results better in the present investigation? The difference is not large enough so that chance can be ruled out as an explanation. It is also possible, however, that the gain is a result of an improvement in techniques. There was one difference between the interviews of the first field study and of the present study: the use of the sealed envelope for reporting the financial data in the present study. This technique may have helped here as well as in the study of cash borrowers even though it received no special emphasis in the instructions to interviewers. Chart 3. Reported and Actual Balances, January 1, 1959 REPORTED BALANCE, JAN. \, 1959 (THOUSANDS) ACTUAL BALANCE, JAN. I, 1959 (THOUSANDS) 132 AN INVESTIGATION OF RESPONSE ERROR Identification of the Selected Account Table 61 shows the relation between the certainty of identification of the selected account and response error. It sometimes happened that the investigators had some doubt as to the match between the account or an account reported in the interview and the selected account. The criteria used for the selection of the designated account were as follows: 1. Ownership information should match (ideally), e.g.: If R reports a joint account with a certain sex distribution of names and this agrees with the savings institution's data, it is a match unless R reports more than one such account. If R reports a single name, and the savings institution reports a single owner, and the sex agrees, it is a match unless R reports more than one such account. 2. If there are two accounts which match on the basis of owner- ship, select the account which comes closest on the January 1, 1959, balance. 3. If the ownership data do not agree as to the second name, i.e., other than R, but R reports the January 1, 1959, balance within a small margin of error, accept the account. a. Consider also, as a favorable indication, agreement as to the change and the January 1, 1958, balance. b. If the internal evidence of financial transaction suggests that the change reported by the savings institution is reasonable, consider this a favorable indication for this account. In 33 interviews the investigators selected an account in the interview and had no doubt in identifying the selected account as that about which information was known. In 24 interviews they selected such an account, but the match was not perfect and there was some doubt in their minds as to whether they identified the account cor- rectly. As Table 61 shows, there was little difference in accuracy of report between these two groups. Among the cases where the analyst did not succeed in selecting an account, there were some in which the existence of an account was known from other indications in the interview. For example, in the earlier part of the interview the respondent might discuss the existence of a savings account in con- nection with the questions about banking. He might then refuse entirely to fill out the financial form. These interviews were coded as cases in which the respondent reported the account but refused to state the balance. SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 133 a o •rH o •rH 1-1 £ « C cd 0) o 2§ >» . CO Is* I Sh iH Sh O CD u-r-g o >>""* -M p>t+H c 3 ° SCO «3 £ £ 1— 1 >~a p4 K .M "JH in OT! o o „, wgf lis 2 o cd CO 0) cd T3 CD 73 0) O xi co 5? 00 co CO 00 -Q _D _C _C en co in co j; X ji n a ^ c !3 o CO CO CO o OS iH 1— ( t- CO iH 00 CO O 10 O 10 CM H tH O O ITS IO CO H *hh cd Sh cd 3 C cd O <*H o Jh h QJ CD Cfl C o & cu K CD CO o ft 09 U o -M c cu c o H-> ft a co 3 CD O U [>. « -° s ■u T5 O Sh ■■-= O C o ? o£ o c ■ M 3 "O o cd o r-H O O CO O _ c3 5 73 3 § g o a ■M PH c o 80 Ph €^ O >> o ^ C CD 73 CD +J O ft CD in CCi ^ rv Co 0? J2 cd O o ft "^ CD 73 ^ CD ti n "S o c ^ .3 CD S tJ ^ CD ca Sh Ph U 3 CD O > o o< c 3 O o o ca CD X! +-> Cfl Z . -M T3 C CD CD CO O 3 t T3 xi CD X! CO H-> X! C ■*-• 3 -M O Pn O O O ft CTJ CD O "•"> +3 o O (« o C ^ ft «5 3 O CD co S Sll ,Q +-> C3 +j X! T5 fl O CD ® *S f O XI fn Sh £ O ^ & 0. C CD 1-1 .rH f_ o rS« rt 52 CD -t-> _• "O -t-> (V) § a rt •" § CO -3 C m CO CO X! CO" g CO CD O j. o O cd 134 AN INVESTIGATION OF RESPONSE ERROR Another way of looking at the data is to ask, would the total reported savings accounts have been accepted on the basis of the evidence within the interview, or would the investigators have believed that the report was inaccurate or incomplete? The in- vestigators judged, as shown in Table 62, that in 66 of the 88 interviews the total would have been accepted, while in 22 it would have been coded as "not ascertained." Of those cases in which the total would have been accepted, 57 percent reported the parti- cular selected account within a margin of $1,000. Table 63 shows the proportion of respondents who gave informa- tion on savings accounts in the financial form. Some information was obtained in 74 of the 89 interviews. In 15 interviews no information about savings appeared on the financial form, but in 6 of the 15, the existence of the selected account was reported earlier in the inter- view. In 25 cases the respondent was the sole owner of the account; in 33 others the respondent was a joint owner with his spouse (Table 64). Thus, in 58 out of the 89 interviews the respondent or his spouse or both were the owners of the account under study. In the remaining 30 cases someone else was involved either as sole owner or joint owner. Of the accounts which belonged to the respondent as sole owner, 68 percent were reported accurately within $1,000. Of those which were owned jointly by the respondent and his spouse, 58 percent were reported accurately. Of the others, the proportion which were accurate was much smaller. The differences were largely due to failure to report at all accounts owned in whole or in part by persons other than the respondent. It is not unreasonable to speculate that respondents may feel free to tell about their own savings and that they may be encouraged in this by careful interviewing techniques; but that these same respondents may feel that accounts belonging to or involving other people are not their business to reveal. At any rate, from Table 64 one may conclude that the best reports are obtained when the owner of a single account is interviewed about it, and the next best reports when the respondent and his spouse own the account. Much less accurate reports are obtained by inter- viewing the spouse of the account owner or by expecting a respon- dent to report accounts between himself and some person other than his spouse. Even when such accounts are reported, it is more likely that the balance will be refused or that the balance reported will be off by more than a thousand dollars. Income The next sequence of tables shows the relation between response error and socioeconomic characteristics of the respondent or of the spending unit. The relation between income and response error is shown in Table 65. SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 135 a> £ us •s >> rt ^ i G c y m o >-i u CJ h <; o <4H co u bD o ?H u G > w CQ CD n co cd n -t-> o u n, o m u (i) cd P(J K _, rrt -i-> CM o CD H a> ^ .Q rt H 'c O - c 0) PQ * r3 U P< CQ i> rrt "3 C S-< •a CD f ) 3 o m (* -*-> o rtf G O H C 2 a> a G « aS CD -2 CD O ,Q H < u o - U Oi U LO CD Oi CD^ (D rt 3 O 01 >> -4-> cd CD O CO o CD O CD U o s o CQ a a G O -t-t CD o CD -* a o CO ^_i Ih ■a *H ? G G G o c 3 o o o cd c o a CO 0) tn >i >> O CD H £ -^ CD O G CD G 3 O CJ o o o o o I— 1 €«• >> X2 73 CD o o o I— 1 o o o i-H £1 c o> £l G <4H >> G o a •a CD o o CD t3 .G CD +-' a o CD CO G O "g 3 O G O o o CD o G ^2 o B <=> c o CJ a cd T3 CD U o a O CD o -l-> CD -M O a CD CJ CD a G .C G CD CO TS G T3 !h CD rt 3 -fcj 5 £ CJ O o CO G 03 CD ^i rt CD U u 'm ci -a O O a CD rt e O CD T3 c CD G o CJ < O e3 < < a CD E a CD D fc CJ K G CO rt CD -G ■n G to CJ ro X a; W J 136 AN INVESTIGATION OF RESPONSE ERROR a <0 X5 CI o ft co CD O O fe -(-> O -pH r3 o 15 c ^ -3 as CO cd a 5: «s CD c > .fa CD fn -ti CD oTH H-> c s.5 •rH tH TO ^ o •—< rr. TO " c CQrt 3 o •rH H-> ,tT O 3 Oi .D. m q s< H-> CO - CO ■•-< tH bfl X5 s CD bC d 5 c CD t» O u ►"3 u . c CD fa o ft O tH vi— i -4-> rt fa s Sh fa S 5 w 1 •— i CD w a? d ^ ftO CO CD Oh CO CO CD 1— 1 rQ rt H ffl CD > be bo c C > .2 rt -(-> co rt o a ^ o ■a •rH _Q _Q _Q _Q o eo t- 2 ' ' ' ' CO CO ON ^ 1 1 1 1 o o r-l IT3 rH C CD CO > S bD > C co -S a g .rH C- CO rH 00 T}< rH IT) 00 r- 1 T—t (OH ^ U Oi OS CO 1 ■ — 1 -|H zi > < u CD -4-> fi •rH co i-i iri in en o ^ m CS1 H tH r-l lO tH ^ O o rH 00 CD O rH Ci O-'rrt tH TO rH^ o - rH 01 rH in O Oi rH CD co _r ft >> 03 fa CD TO « B rt 1-3 Failed to report account Account owned by respondent or spouse or the two jointly Account owned entirely or in part by someone other than respondent and spouse Reported account, balance not ascertained Reported account but refused to state balance Reported balance for account Underreported by $1, 000 or more Overreported by $1,000 or more Accurate within $1, 000 i— 1 a o H CO CD > u CD H-> c • rH tH o rH CD rQ a £ & =« 3 o "H n-f •J3 2 £ O - > •2 r2 X5 CO ^ ^ CD ■•- 1 « > fa CD "g -r-> . c *-< • rt CO CM rt „£ V Sh CD G C rH rH O 3 «« % XJ rH M +J §ft CO S a o ^ 2 _ ^ rH t) O -rH If ■H J3 xs - £ CI • rH — 3 o o -^ S co C r* 6 u tH o • rH CD t+H -rH o CD > H-> CO CD rH CJ 3 X! CD CD rt -4-> -M rC -rH CO o -i-> CD y _ c •rH O rCS rH o H-> r* T3 o CD .3 rt CO - > T3 § CD • rH C > rH u rt CO s • rH > -M CD o rH CD O rH CD ft o o T— 1 rH 3 CO ft o +-> CD CD LO C rH rti X5 « S o fH X5 a to ^ •rH rt -4-> o M rH rH rC d xs S o -M r3 S •rH -l-> CO CO CD §«2 1—1 rt CO c •fH o »"3 Joint account owned by other than respondent and spouse tH CO IT3 ■* to © to ^ ■>^ CO CM CO CM a i-H o i— i i—i Respondent is joint owner with spouse (0 CO CO 1 1*3 CM lO OS CO 1 iH CO i-l 1/3 o o I-H CO CO CO -4-> o o o Ri 0) • 1-1 Someone other than respondent is account owner C- iH CO LO CO CO 1 1 C- CO "^ i-H II o o 1—1 CO 1—1 Respondent is account owner co co i co co co i co iH iH 1 C- 1 CO o o 1— 1 IT3 CM All inter- views irsi-H ^ in oo in m CM »—l i— 1 H CO H ^ o o 1—1 u CO CO CD O o ^ u& o „ J" 1 Ol v CT> CO „ ft >» CO u u CD -4-1 c •rH o u CD a O c CO rt £ rH CD C £ O -M c 3 o o O cd o -l-> -*-> fl CD T3 C O ft CO CD hfl U C =3 MH o c 3 ft o •rH CO «4H n O o CD -t-> CO a. 1 i — i rri CD U U CD •Q C3 •1-1 £1 CD £ a u C CD •■-I c ft •5 11 o CD CD ft o 1—1 o •i-i > S-c CD -t-> |-| s Ti •rH o T! n ai , n -i-> DO CD CO O c CO CD -a CO CD o ■r-l CD O X uj J c -H 0) id n u o CO 138 AN INVESTIGATION OF RESPONSE ERROR a co- in "8 a o o ►3 t— 1 oJ -l-> o H o -•-> g o •iH I — c 0» 1X1 8 ■f-< > c CD +■» •jH C •i-i o «+H G o a i— i r! o • r-( 3 CD ^J - T3 S-t g cti U O <*H rH o u u w CO G o ft CO CD lO CD SP +-> C CD O Sh CD Pm ,-. *h O > CO 00 00 CO 1 tj< m ca i OS o CO T— 1 1 00 rH i CD o rH O ^3 rH 6» rt G ^ •rH a «*H O Oi O OS ■^ oo CD O CD 00 IT5 CO O OO 4H O OJ CM rH rH co IfJ O CO o IT5 OS rH CD a o ¥±1fr rH u 2 5 o D- IT5 CM OO m m 00 CM o CD •o ° CM rH rH CO rH Tt< o CM fl -T rH 3 «3- CO £ CD M r— 1 -|H G •rH T3 CD CD CO fl 3 o a • iH -4-> rH CtJ ■4-1 CO -t-> CO CD u u CO cd +-> o G CD U o u o rH o a u o o o o CD CT> +-> o a CD O a a* •rH o rH ^< o S -M X3 CO ■4H rH c o o o a u o o o o o o %£ G G CO CD CD fl Jh cd rt ■4-1 3 rH rH o co o - CD >> o o o >> -4-> X> G •i-t TJ O CD ^ _, CD 3 s -C Id ^2 S3 O rH o <4H O rH G si • |H > u Hal « S u CD ■■-» cd £3 s CJ C T3 ■>-> o 5 o c o O o CTJ -4-> rH o a ■rH G g i-a CD m o ^ CD £ CD o o 73 y i— 1 cd T3 -4-> rH o a 13 • rH «TH O -*-> ga§ § o a q CO CO CD rH CIS u u rH rH T3 CD .— i •i-i 3 n < -t-> rH o a CD Q. Co CD rO K -4H u o a G 13 u B 3 o o o H X2 a fa BJ Ph g SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 139 There seems to be a gradual increase in the accuracy of report as income increases. This finding is consistent with the results from the first field study and with the general tendency of higher socioeconomic status to be associated with accuracy of report of savings accounts. The higher -income people were less likely to fail to report an account, and they were more likely to be accurate within $1,000 when they did report the account. Of those with incomes under $5,000, 42 percent were accurate within $1,000, while of those with incomes of $10,000 or above, 69 percent were accurate. Taken by themselves the results of this study are not statistically reliable, but taken together with the results of the two earlier studies of savings accounts they show clearly that accuracy of report of savings accounts improves with income. Education There was some evidence in the first field experiment that re- spondents with high education were more likely to report balances for their savings accounts. The data in Table 66 point in the same direction. The proportion of respondents with a college education who reported a balance is 92 percent. Even with the small number of interviews the difference of about 29 percentage points between those with a college education and those with a high school education or less is in itself suggestive. The two studies combined point to the conclusion that education, like income, is positively associated with accuracy of report of savings accounts. Occupation One would expect this difference in education and income to be associated with a difference in occupation with white-collar workers reporting more accurately than blue -collar workers. Such a difference did appear in the first field study. The proportion of blue -collar workers who failed completely to report the account is somewhat higher in Table 67 than the proportion of white-collar workers. The difference is small, however, and consistent with the interpretation that the difference between blue-collar and white-collar status is not of major importance. Age of Head In the first field study of owners of large savings accounts little difference was found between respondents aged under 45 and those aged 45 and over. The results in Table 68 also suggest that the dif- ference from one age group to the next is not large. If anything, younger people are more likely to report accurately. The differences 140 AN INVESTIGATION OF RESPONSE ERROR d o ■i-i -H d i— i 0) PEJ S • l-H CD O C rn i — 1 £ & a CD CQ id •■-1 „p u ^ >> CD • r-t os -rj ^ H «(-( ~ rt o ^ tn c ^o H-> Ctf -rt 3 g § .Q 5 a> •r-l u -M CO •1-1 o ° T3 *h c CD Jh ° of) O £ -t-> Sh O w 5 t3 o CD r£T CO ^ C3 CD O a CQ CD Ph CD CO CD ,Q CC! O -8 w CD bD CD i— i O U OO 1 00 1 CM ITS 1 C- 1 1 OS CM 1 CO o o i-H CM i-H xi o be g •l-H Jj CQ CD 00 00 CO CO 00 lO O i-H CM CD ITS CM O i-H O CD • l-H w OS CO CD f ^t IO ^< IO CM i— 1 H CD tfS o o i-H CD CO All inter- views CD rH ITS lOOSOlO^f CM i-H tH i-H ITS i-H t}< o o i-H U 00 00 i < i < i < < I 1 < i C < CD H .£2 OS D tH a 3^ Q^>> ft H tJ g 1-3 Failed to report account Account owned by respondent or spouse or the two jointly Account owned entirely or in part by someone other than respondent and spouse Reported account, balance not ascertained Reported account but refused to state balance Reported balance for account Underreported by $1, 000 or more Over reported by $1, 000 or more Accurate within $1,000 i— i o H CQ 1 •i-i > u CD 4-> a •l-H "4H O U CD a i SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 141 e o -4-> a ■—1 tf c • rH 0) o c pi 1— 1 cd CO « ►. £ F^-l 0) ~ •<-< •r-i 2^ > a - «*-! •rH ,-) O <(H o Sh ™ c o 3 ffi -M c 3 cd "H ,11 h5 © s-< o 2 co •rH <«-i ^3 73 . cd rH Q, w o 0) fan cd c CD c; 2 o rH M -m (1) o ft a CO a> PCS t-* CO CD r-H rQ a H rH cd r3 5H O ! * PQ u ei 3 rH o a> O rM I rH o 5 * o CD o o CM CO ■^ CO CO o rH « O cd ■" rt rH -Q o „ 0^ CO - cd 1-3 K co o i—l CD 0) ca O a CO u o H-> c 0) 73 C o a CO >> H-J rQ C • rH 73 O CD -i-^ s ° o a o CO £* ^ CO "£ 3 r3 O a co C rrt •rH 73 s * r-H^ o •rH C -s a <" CO •o £ 0) M £ CCJ O rC 73 C •rH cd -r-> rH O CO cd H-> o o c cd cd rQ o c cd r—i cd rQ -M cd CO o -t-> 73 CO a rH -t-> B -4-> c o o y cd o o o cd rH O <*-H o c rQ o S rC fl rH 73 H-> T5 3 ±3 -3 rH O a o a -f-> rH a r*< Oh Dh u o a rH O o o o rQ 73 -t-> rH o a rH rH 73 c rD rH O a u o o o o rQ 13 -M rH O a u rH o o o o rH O t+H rH O rQ cd 1—1 rC a cd t*H . H-> rH «t-l O O rH rH T3 cd _^ m T3 fafl^ S C 73 2 'cd C « +j 3 +- 1 rH 5 - ® ^ « " > CO !h CO o c 0^5 a o o > o rH 73 cd — -~ 0Og Q W > « ja 142 AN INVESTIGATION OF RESPONSE ERROR a re [x, ■s CD a •S a) a re o re C! « O •»H -M 3 5 O) S-. T-l +-> CO •<-\ 1-1 73 >> CD S-l hf) re «i 3 ■s m 0) ►-a (h 0) O Pj, t. o Sh 0) en c o a en a; a 00 CO 1— 1 .a re H 4f T3 re a» lO £ cd o «m re o „ CD '""I A rH a CO £-1 cd re a§ re •-a oo CM CD CQ rj* O lO »-H "^ CM eg Oi oo CD o o 00 o CO o a CO s-i o c a> t3 G o CO -4-> >> fl *~ o -a ■— 5 * +-> CD C .£3 3 * J CD c O CD a o CO >s ,Q CD cia O w a? * a •o £ CD M T3 CD O X! 3 CD O 43 ° n T3 CD CD O CO re -M CD O c re r— I re £5 § o o re CD S-i o ft CD a CD O C re . — i re £5 T3 CD CO a CD -4-» c o o o X5 CD -(-> o a CD a c O o o o CD O c "re ,0 X3 CD -t-l ?-i o a CD a CD Jh O a o o o o €/5- >> 73 CD +-> u o a CD Sh u CD T3 C o o o o o o o €^ €«- T5 CD u a CD fn 6 o H CD X2 a I SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 143 between age groups found in the study of cash borrowers were larger. It may be that the difference in that study is the result of differences between age groups in attitude toward use of instalment credit rather than any inherent difference in willingness to report financial information from one age group to the next. Keeping Records of Money Spent It was found in the first field study that those respondents who keep records of money spent tend to report their savings accounts more accurately. As shown in Table 69 this result was confirmed in the present investigation. There is a difference in accuracy in favor of those who report that they do keep records. As was previously discussed in Chapter VI, however, answers to this question may measure willingness to respond to questions about finances rather than sheer availability of the knowledge to the respondent. Memory Error In Table 70 accuracy of response is compared for two dates, January 1, 1959, and January 1, 1958. The distribution of error is based on a somewhat larger sample for the more recent date, since 12 interviews were with respondents whose accounts had been opened in the interviewing year. There is little or no difference in response error between the dates, to the extent that error is measured in Table 70. This result is similar to that obtained in the first field study. Problems of gradual forgetting as time goes by do not seem to be of importance in explaining the errors shown in Table 70. Family Norms About Discussing Financial Matters In this project the investigators tested a hypothesis, suggested by Dr. John R. P. French, Jr., that the observed differences in response error would be associated with differences in the freedom with which financial matters are discussed within the family. People who discuss financial matters more freely within the family might be expected to be more willing to discuss them with an interviewer. Three tables bear on this hypothesis, Tables 71 to 73. In some interviews there was explicit evidence of concealment of financial information within the family. For example, one re- spondent was a widow living with her children. She told the inter- viewer that she did not know their income but would ask, and re- quested the interviewer to return another day. When the interviewer did so, the respondent reported that her children had refused to tell her their income. There were 12 interviews in which the investigators at least suspected the existence of such concealment. It is not sur- 144 AN INVESTIGATION OF RESPONSE ERROR o • r-H +J CD r- C C m ■ rt ft w CD ■i-H > ffl£ Oi O CD O TO~ § 5 o TO CD •" ^ W "C w O O C C -3 CD T3 « ft ?5 W cd a a CD ft CO >» CD c o a o CO o O CD ft 2 « CO ^ » ?, m CJ CD C CD iH lf3 iTi Ol Ol Ifi Ifi CM H i— 1 H in ^p o o 1—1 U CD 00 J c s c ' f J c ( c ( c i c c p CD O C i—l l> —T 3 § TO i-a Failed to report account Account owned by respondent or spouse or the two jointly Account owned entirely or in part by someone other than respondent and spouse Reported account, balance not ascertained Reported account but refused to state balance Reported balance for account Underreported by $1,000 or more Overreported by $1,000 or more Accurate within $1,000 »— 1 TO -t-> o H CO CD > U CD -4-1 C <4H O u CD a & u CD 43 +-> O CD • i-H X! 73 a CD ft co >> 43 CD C o 43 TO ^' -t-> 'i-* S3 O CD T3 Jh cd O > £ « CD >> TO CD CD g -t-> O U ft O CO ft CD U ^rH ^T3 X! ^ CM O o .- CO r >> CO i— ( o C ft 5 CO 3 TO •° s CO TO tj -t-> (H ^) O w o O CO A t* TO O - ftrt S^ ft^ CO o T3 M O CO O TO >> ^ ^ 2 Susg &S M og . O X! M TO > m 43 a ftjsi:; O r- C C4_l S||« o •■ — "3 S * 3 g O u > ^ CO g TO p O £ w C Q w H HCM 2 CO +j m CO *^ O 5° +J o o ft ft SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 145 Table 70. Response Error in Reports of Balance at Two Points in Time, Based on Interviews Taken in Fall, 1959 (Percentage distribution of interviews) Response error for January 1, 1959, balance Date of balance Balance January 1, 1959 Balance January 1, 1958 Failed to report account 26 28 Account owned by respondent or spouse or the two jointly 11 12 Account owned entirely or in part by someone other than respondent and spouse. 15 16 Reported account, balance not ascertained Reported account but refused to state balance 15 13 Reported balance for account 59 59 Under reported by $1, 000 or more 10 9 Overreported by $1,000 or more 4 9 Accurate within $1, 000 45 41 Total 100 100 Number of interviews 89 77 a a Excludes 12 interviews where accounts were not open on January 1, 1958. prising that in 7 of the 12 the respondent failed to report the known account. In 2 of the 12, the interviewers obtained accurate data. Complete concealment of basic information among adults who live together in a family represents an extreme. Respondents were asked a question intended to be a more sensitive measure of differences in family norms, "How old were you before you knew how much the family income was?" As shown in Table 72, 34 of the 80 respondents (42 percent) said they never knew, 15 were told after they were 16 years of age, 22 were told between the ages of 12 and 15, and 9, before they were 12 years old. Only 11 percent could not or did not 146 AN INVESTIGATION OF RESPONSE ERROR CD O & CD •i-i > >> o a C Eh +3 o CD S-. dB d &p S rt O c -< O 2 -y .a aJ J-l S-i >> O -M 3-5 3 S i— i •*-" «5 nJ O o cd £ c bp O rH ® rH e^, ^ ft h n W -i-> <1> c W r^ -Q a H >p dence lment •r-C i—i 00 CO CD CO o -* OS o c- a d > u CD eg i—i rH CD rH ■^ o rH t- a £ o • i-H rC5 -i-> £ H-> -i-> a C CD 73 CD S3 a C8 _Q £ <•> c- CO -& "^ CO ■^ 1 "* OS c- d ?> a m ■ O CD CD Explici idence ncealm o o O 1 O « O o o m 73 •<-t > CD CM rr ' ^ ' CM CM o rH W > o CD O ■ CO All liter lew CD rH lO Id Oi o "<^ in O OS CM rH tH rH LO tH ^ o H 00 •in f> CD c u o a CD d a T3 'd CD CO o CO CD rQ CD O o a Sh rQ 3 o a CO rH o rQ CD *-< S •rH -4-> u CD O CO Cj CD -t-> d H-> CO o H-> CD rH O a CD U o a u o o ^ ,-rT ^ OS OS -r-> a CD 73 •rH -O O d -t-> o c CD T5 CD CO < HH -r-» PI 3 u o o 3 o a CO CD rH ^ o a d i— i CD rH +-> B -M c o o o u d o o H o o o o o o CO co u 5 H-> rQ r, •rH 73 O .H C « a ® CO rQ CD rQ H-> u o trH CD rQ T3 rH ea- rn • rH rC CD > o H-> d 5h 0) -r^ d) rH S CJ c CD T3 1-3 O fi o « fi o y aJ H-> rH o a, CD CD •r-l fi CD U ■+-> CD 5 rC H-> H-) o CD r-i rQ H-» rH o H-) • rH trH O o S rC 3 H-> C Sh T3 -t-> T3 rH CD aj Rj -4-> 3 O H-> rH S o CD a o t-i O CD rH o a CD T3 CD > o rQ a • rH < •i-t -i-i OS Ct, -i_) d o« ffl 2xs fan w |g cd I* y co op «o CM d -o cd d S S.sk !> xj - h o 3 CD OX) O eg o eg cm oi co o "* CO o eg irj o C- CD w ^ t o co th ira OS CM CO CM 0> o d I— I d oT lO .2 JH d cd •"3 (h O u o cd CD CO C o & cu K CO H ^ CO in h ^ o o CD CO i CO o d CD T3 C o p. CO CD * >. . -rH T3 O CD •■— • go o5 o o CO >> -M CO u d C* O ^ d o ™ *1 s-s c o a CO CD CD d CD O d a < — i re" a * d CD <*h d CD T3 CD C d "d 92 o o d CD OX! o o o d d I— I a xj d § o o d -d cd -t-» ?-< o a CD CD 5h J -t-> c d o o u d T3 CD -!-> o a C d o o o d S-. o <♦-( CD o d CCJ .—I a X5 73 CD -4-> u o a T3 5-1 o a CD u CD T3 d o o o €«- c 3 >> XJ CD d -t-> CD rt o H CD x: H e- co al CD a o o d X! -t-> X! o I o X5 d J* d o • >>>> ■d CD o p< CD CO CD cd o d XJ o CD •>-! " a X5T3 §1 CD O 5H "L L'lT » 0> ^x: "«> d © ..S S « c5 jo ^^ CD " CO O XJ d CD X CO ^ o CD S-i 148 AN INVESTIGATION OF RESPONSE ERROR answer the question. The differences in accuracy of report are striking. Of those who knew the family income before they were 12, not one failed to report the account! Of those who say they were told between 12 and 15, 18 percent failed to report the account. Of those never told, 40 percent failed to report the account. Another related measure was based on the following question: "In your opinion, how old do you think children should be before they know about the family income and savings?" The results are are shown in Table 73. Again, there is clear evidence of a correlation between replies to the question and response error. These results suggest strongly that concealment of financial in- formation is deliberate and that it is associated with norms developed within the family relating to discussion of financial affairs. The im- plications of these results will be discussed at more length in the concluding section of this report. Reporting of "Round Numbers" While it is important to the survey research worker to understand the causes of response error, it may also be helpful to be able to tell which replies are more accurate. Table 74 shows the results of a test of the simple hypothesis that people who report round numbers are likely to be less accurate than those who report unrounded amounts. There is an obvious negative relation between rounding and absolute precision, but the hypothesis is that those who report round numbers are more likely to err by $1,000 or more. The data sup- port the hypothesis. Of those who report a figure rounded to three zeros or more, i.e., to the nearest thousand dollars, only half are correct within $1,000. Of those who rounded to the nearest hundred, two-thirds are correct within $1,000. But of those who report to the nearest $10 or better, almost nine out of ten are correct within $1,000. ' Interviewers' Ratings of Accuracy Another way to estimate the accuracy of a report is to ask the interviewer to make a judgment. The results of such an attempt appear in Table 75. The data indicate that the interviewers did much better than chance. It is not surprising that they knew when the re- spondent failed entirely to report any financial information. They also seem to have had some success in detecting accurate reporting as opposed to underreporting or overreporting. Error in Reports of Change Table 76 shows the relation between actual and reported change for the 48 interviews in which a balance was reported for January SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 149 Sh J3 « CO CD m i-H CO i-H CO | CO o CO 73 > "* i-H CO CM CO 1 CM o i-H ^H CO 1— 1 O H-> S3 CO Fh 73 CO 3 > o XI o O | O c- CO c- CO CO o in CO o &p H-> C xs 73 CM 1 CM c- tH in o tH CO C! PI cd 1-1 c > CO CO O ctJ Fh *3 iH i— i 3 "2 CO s m O i—l CI CO CO CO m OS CO CO o CM J3 1 CM CM i-H i— 1 iH m •^ o 1-t CO ■si i— I Balance mily Inco views) CM Cd u CO 73 C CM CO CO i-l CO i-H CO i-C u i-H CO CO < i-l tH t- i-H to o T-l T-t - S Fh |3 1959 ing F; if inte CO o c >> c o CO u X) 1—1 .1-1 << Fh m o in ■^ o OS m CO Ol CO CM 1-1 i-t i-l CD ■^ O) t- nuar reK ibuti CO H-> •iH cd o u |— CM 4-> "^ CD CO r for Be B ge di CO o e cd i—i ed CO c o CO o £"3 6 Wo 5 CO CO a o CO 73 CO cd ■—I cd m5 (h of <3i § a X CO 'cd ■+-> u CO O s — ' tH CO -i-i CO a o aa CO CO CO 73 tH u o o CO cd o -4-> U o s Fh o Fh O >> Fh cd c CO 73 o a CO CO H >> Fh c o ^ 2-8 C Q. <" to -O CO CO H o e T3 CO CO a CO Fh B a Fh CO 5 Co CO H be I U O P3 8 o o CO o cd i—i cd Xi § o o cd u o o o o o o o o tH #3- o o o i-H €/9- co < Fh o u cd XI c •rH 73 O -m T3 >> 3 •rH > Fh Fh Fh CO hi 3 o c T3 CO o P O P c o o cd Fh O a •i-« H-> CO CO a CO Fh o-2 o o cd cd T3 i—i ,Q -M Fh O i-H C • rH a -i-> CO ^-> a H-» cd o o o &XI C Fh 73 H-> TJ Fh cd H-> o a -(-> 3£ ;3 CO u Fh Fh Fh Fh CO 73 CO I— 1 O . OX! -<-> O -*-> Fh 3 H CO ° r! «•£ Sh a Fh T3 o XI PES o o O O o O C > a ■a < < a CO « a P o < (X4 tf PU z o X! cd £ O c ^ >. -EJ ■+-> Fh O M-l X c o X •1-t H-> T3 CO 3 3 O cr CO si c H-> o Fh H-> T3 i-H •fH 43 CO O o ■a •fH a CO X! Fh H-> 3 • i-H O o C ►* T3 -3 m T3 i— i S co O 3 «» £ F^^ o «*H 43 O bfl rt c CO O o ^ e •i-H a o CO o 2a Fh - r^ S ^ c ^^ J o . nl m £.S ^^ - > cd co" M cd -c F* S CJ Fh a 5^ cd i" •rH C 73 O (DOT31 S £ r, cd^ CO o CJ A -t_> CO O 3 q 4-> c xs CT--H . 3 U) CO Xi >> rn O -ca O X H a rJPW ■ 4h Jj u -a 5 150 AN INVESTIGATION OF RESPONSE ERROR « CD .2 £ -M i— I d rt ■S CQ tf tn r- O .iH CU *^ CO 1 - ei f> OS X J_l £ w 0) Oi H-> iH fi c rt ••H o n Sh 73 rt C 3 3 +j 5 O H Ha •p-i e rt +-> ?-i rf> O 73 •i-H Wtf e CD p. ^ cu . tf S! u t~ Ht* -t-> J2 o CM 1 00 O t- cd rH 1 CO O rH X rH W 73 CU 73 a o 3 n -a o cu CM 1 00 O CO p U N CU H i—l 1 co O i-H 73 ^H _ o a^ Ph •iH CO fa 0) rQ 73 CU 73 'S w CD 2 o -t-> S rH U U p cu t- CM CM tH CO o H N i— 1 H c- O rH Ph rH CU CU cm u rH CU Be-"-' rH •rH 3 fa faD •iH fa 73 CU c o CU 5 Sh I* p CU O O O O O S N a CO CM in O rH rH cu en J-t h 300 • rH fa CO £ CU -c 1— 1 •!-! c- CO m O CO ^3 > rH t- O rH ITS CU -t-> c •iH CU g O O ■H ,2 O -Q rH - i-T >> i-H 03- r-\ CO 0) •iH > IO JO >> • |H rC ^ $2 73 CU 73 cu -t-> C - H-> • rH Respo luary 1 rH a* cu g f-< cu £ a CU "ctf rH -M O <4-l O u cu a § 73 £? 0) H 3 >"3 2 > © O 3 t3 O o C co a H-> rH o o y c cu o rH CU a rC 4-> CO CO cu c o T3 c o Ph CO Cti 0) 73 u r« cu rC O H-> CU rC O J^ CO 73 ctS cu 73 cu CO 3 rH a u • rH 73 a o rH CU CO o cu £2 -t-> rH cu o rH CO Ph o o 73 3 4-> o rH CO CD O Q o a CO cu u c 5 o c o o o CU rC H-> rC o • rH rC c • rH CO cu • rH > rH CU CD CO CO CU O X w SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 151 CO c o 3 ** o a ° iS ^ <^ 5 «g .5 h •-a re^ O O QJ O 3 m 0. $ -a > ■** i—i CM CO N i i eg o i-H H o £ en .S re u 3 re o re 0) -t-> cd a >• u Cd CQ A A u 3 o i-H o re ft CO c 0) ■4-> Oi m to "5 O J3 E* « 2 o O O O o O O l o o o CO ■^ CO tH i-H in cm i co o 1—1 bp t— 1 s -4-> O re re u DQ C >> CO •— ' /!% u 2 « O 3 CO Oi ■^ 05 00 00 CO CM o CM Cd i-H CO i-H i—l CO o CM •r-( 1— I > re o u w re -t-> 5 « 0) ■«-> >> re CC f-i Sh Oi CO CO CO 00 Oi 1 Oi o CO CD 3 00 1 c- o CO > ° i-H re i m •3 > CO i— 1 lO lO UO^ifl o Oi CM i-H iH i-H m i-t ^ o i-H CO ■M rror for 59, balance h-> c o respondent or o jointly irely or in par sr than CO o ft -M o c o c re o T3 cu CO o Si -t-> -4-> d balance for account reported by $1,000 or mor eported by $1,000 or more ate within $1,000 CO re o ft CD o nt owned by ise or the tw nt owned ent lomeone oth< CO c re c a o ft X2 CO n &>> cu re -M C o ^ re .£ o y ft s CD TO O cu o o reiS T3^ -t-> -t-> o > c o ^ -i-> 3 K 3 UJ Sh -(-> J-c U U H u T3 cu f— 1 o « o -° CO cu o re ft +J o CO eport Unde Over Accu 3 •l-l re < < K fc « tf 1 c u h ft q re +j CO CO 152 AN INVESTIGATION OF RESPONSE ERROR 1— 1 in OS tH NH ^c- OtO O CM CO CO OO 00 o H i-H ■^ CO 00 r-* >> O Oi Oi~ in u d 0> 03 O OS in Oi l— 1 i-H CM CO m , >-3 d O Oi i—i O co~ in CD O Oi CM CM Tt< CM co >> 5h o CD Q €«-€«- Cti I d 1-3 Oi Oi CM i-H eo O CO i— t >> #3-€«- o CD „ u d &$ oo o> 3 o o> Oi in o a a x: o 1 o i-H Oi HNN CM i-H 00 oo co >» c ^ Oi i-t ^ .iH c 5P CD #3-€«- o l O Oi i—i ^ CM CO ^ CO CO CM o CM in in CM ° ^ fn d c i *"' 5 3 O Oi 0) y o O Oi CO;. i-H in Oi o m oi o &>-&*■ d o aj CD CO u o i CQ C ^ O .1-1 > O Oi O Oi o.o> i-Ti-T HHN iH in •^ Oi 3 C£ (0 €/3-#3- rt ,- • r-l 1 O Oi bfl d O Oi O Oi^ i-I i-H iH CM T3 0) -t-» a CD cnTcnT €/3-#3- 1— 1 c o a 0) (H O O « cu e X5 o a ° C O 3 aJ » Q < » !S >> ^1 0) a (D CU O 0) fn 0> xs o • r-l xs SAVINGS ACCOUNTS: THIRD FIELD EXPERIMENT 153 1, 1958, and January 1, 1959. There were 31 interviews for which there was an actual increase of $100 or more. Of these, 10 were re- ported correctly; that is, 10 were reported as increases within the correct bracket. An additional 11 were correctly reported as in- creases, but in the wrong interval; 7 were erroneously reported with no change, and 3, with a decrease. Of the 9 actual decreases, 4 were correctly reported. Conclusions To summarize, the following variables seem to be associated with accuracy of response in reports of savings accounts on the basis of data reported in this chapter: Whether the account is the property of the respondent only, or of the respondent and his spouse Income of the family Education of the head of the family Whether records of family expenditures are kept Whether the family norms tend to concealment or free discussion of financial matters Whether the respondent reports exact figures without rounding Interviewers' ratings of accuracy The following do not seem to be associated with error in reports of bank accounts: Whether the respondent is to be paid Whether the preferred respondent is the head or his wife Whether the date about which data are sought is 6 months or 18 months prior to the interview The evidence is doubtful concerning the following: Actual size of balance Age of head Occupation of head VIII. THE MAIL RE INTERVIEWS Purposes of the Study and Description of Procedures It has already been mentioned that when the investigators re- covered from the initial shock of the results from the interviews taken in the fall of 1958, they attempted to develop a more systematic theory of response error than had been available up to that time. A statement of the current state of this theory in the light of the results of the investigation will appear in Chapter IX. Particular lines of reasoning which led to the experimental manipulations in the study of cash borrowers and the study of owners of savings accounts in the third field experiment (Chapter VII) have been described in connection with those studies. The experimental manipulations, however, were designed es- sentially to test particular techniques of investigation rather than to test any theory. It is true that the manipulations were based on particular lines of reasoning, and the failure or success of a technique might lead to a re-evaluation of the reasoning. But the investigators felt that more work might be done to generalize this reasoning and then to check certain aspects of the theory of response error. They were interested in particular in forces growing out of the interpersonal relations between the interviewer and the respondent. Perhaps the most systematic attempt which has been made to develop a theory of interviewing is that reported by R. Kahn and C. Cannell in The Dynamics of Interviewing. 4 - These authors placed particular emphasis on the importance of these forces. It was proposed, there- fore, to attempt to measure them and to investigate the relation be- tween them and response error. Measurement required a separate process of data collection. The interviewer could not ask the respondent, "How do you react to me?" The simplest method of data collection available was a mail reinter- view, and such a reinterview was carried out with the cash borrowers in the second field study and with the owners of savings accounts in the third field study. In these reinterviews use could be made of questions already asked in mail follow-ups in other studies. The possibilities of comparison between the field experiments and regular surveys were attractive. The Questionnaire The forces which it was proposed to measure and the related questions were the following: A. Forces arising out of the interpersonal relation between the respondent and the interviewer. 1. General attitude toward the interview as a whole. Question: How would you feel about being interviewed again? 4 New York: Wiley, 1957. 154 THE MAIL REINTERVIEWS 155 2. Attitude toward the interviewer. Question: How did you like the interviewer as a person? 3. Attitudes toward the content of the interview. Questions : How interesting did you find the interview? How well did the interviewer succeed in making clear to you what the study was about? B. Forces arising out of the relation of the respondent to the project and sponsors and to the University . Questions: How do you feel about whether surveys like this one are a good idea? During the interview did you have any doubts as to whether you should be giving the information? C. Forces arising out of other attitudes of the respondent. Questions: Were there any questions which you thought were too personal or prying? Do you think people will give us accurate information about their finances or not? It is doubtful whether the questions asked were adequate to separate the different categories of forces which the investiga- tors had in mind. Much more information would be needed, for example, to make statements about attitudes of respondents to the University of Michigan or to social research. At minimum it was hoped at least to treat all the questions as aspects of "good rapport" between respondent and interviewer. The relation between rapport and response error could then be examined. Error Non-Response The first concern of anyone using a mail questionnaire is the response rate. In this project it was hoped that the mail question- naire would be returned because of the recent personal interview. The results were encouraging. Of those interviewed in the study of cash borrowers, 59 percent returned the questionnaire (Table 77). Mailing out of the questionnaires was more prompt for the savings account study, and the results were even better. Of the latter, 80 percent were returned. It may be that for many of the respondents the fact that they had been paid contributed to their willingness to respond. These results compare favorably with a return of 75 percent in a comparable reinterview of respondents in a study of community in- tegration and 33 percent on a national Survey of Consumer Finances. There seems to be no relation between response error and re- turning the mail questionnaire, at least for cash borrowers (Table 78). 156 AN INVESTIGATION OF RESPONSE ERROR d 2 SP CO p< CO £ i CD a> •rH c TS r* O 3 +-> rH CD Rj C/J -t-> S3 r— 1 u ctf -4-> c CD o o > CI .1-1 -(-> •rH CO o a CO O o •rH rQ •l-t Jh •!-H H-> p CD 3 CO ■r-t CO c o o H-> a •rH 10 CO c 0) aj 3 CJ O^ rH r- Ph c- i— i .a rt H S hS >, 3 ri T3 lO io o <* g rH 3 3 bD -£ S CD CO t- CM o OS H CO o -a u.2 eM rH " O £ T3 rH .rH r» C CD > 3 C g > co ei o U CO CO CD o o rH o CO 00 rH 3 C -rH I— 1 -3 O r r-1 +2 co o fe g CO rH ^ >> ngs own Lrd itud -Q o O O Oi Savi ount (th: ield s CO CM o 1—1 CO CD «-h O ri 5° ^ ?h -rt Cash rrowe econd Id stu •rH o t—t io "^ o o 1— 1 1— 1 O CO CD ro— £ H_> CTJ ■"* Eh CO g , 1 8 o rt T3 * & CO w S3 r-. ° "3 rH ■a 3 c o B-g - ■a § g £ rH CO 3 "£ 1-^ 3 o CO rH a co *H £ O CO H ^ £ 3 ^ H-> o u > a* ^ cr h u «3 si _^ H-> 73 £) CD a c CtJ rH ctJ s g" H £ 2 c CO rH ^ CO u ■l-l a a c o • rH -M CO 3 a* . i *5 r JC +j •rH rH > CO -t-> • rH C! CD H-> o ^ rH CO *" >i rH -* J »* > rH ^ 3 CO rS £_, ■a 5 1 *- a o o d 3 JC o o H-> -r-> H-> si rW X -t-j '$ £ £ c a rt o CO -4-> H-> •r-i • rH 3 3 rH > o o -H ■+-> -r-> s 3 a c o • r~i CO CO o rH 4C u u •rH +J CO CO o Sh -t-> "a CO U rH •rH J2 3 3 CI CI o o u H-l O a • |-H -rH ^-l ed CO CO CO rH 3 CI 2 cyan > o « J2 o 4-> o THE MAIL RE INTERVIEWS 157 o •r-l ei 0) fl> !h tf s a ^ o CO >> •H r* 'O CO CD CD •r-« > C/l 3 Si 03 (J* CD G •r-H at a o s ?H •|H o rH U W CD 73 T3 CD bJD 0) CQ c o CO CD o rH a CO 0) rn CD CD H-> "*** - ^ CD 00* sg c- ,Q a H CD Fh • rH -(-> d o c c c T3 -rH •rH -t-> T3 03 CD CD ° as CD C "1 CD U Oi ess in c— CO CO rH o o rH rH T3 CD C Si -2 c c Sh O ■rH ■+-> -l-> a 03 o r-< a^ 03 rt in o oo c- co in O O l— 1 O CD 03 % I— 1 •!-< ^^ c •rH c- in T-i t- CO Tt< H O o i— 1 rH o 1— 1 rH «H rg> rH 3 o 5 rH W 0) c 0) ctf 03 O c ^ O rC a 03 03 aJ O Reported known loan Reported other loans only Reported no loans Had no known loan at that time O 03 -r-> C T3 fl O a 03 U > 3 o c ■fj >> rrt ■6 g In 3 s to CD co o to 0) o *o a a CD >>fa ■S 01 C > u o u 3 co 01 £ 3 n s o U 3 >> T3 CO 3 cu a o CO T3 CU a T3 u u a JZ >, n 3 CO Xi en cu ■o d h U u u ■a Q s JO o o a « s ■s h i ^■s ° i § ° 2^>g I*. >> o 11 ^ o t4 > .— w w 3 e si cj o o > -g fa 55 2 -a P s bp .5 ^-s -s !C CO CO 01 CU S-. .3 bD » > o S *. " U > ." -t-> -4-> C 5 o) rt oi o o > > C- i—l CD CM 1 !-H U CU c -H bij s cu fl 3 3 eti CU <2 3 o >i C- lO CM CO CO CM CO 1 T3 C- tH CD CM 1— 1 1 3 O * IS o K 3 •§ s 'c3 C cT spgp he 0) ■a 2 "8 "8 cu 5 s > tl 0) cu CU (J, > c C C » bo o> a, • rf a) o) xi o o -° —I C "^ T3 ■*-* ^H O 3 h S CU 0) rt > Z! i* CO CU S a; •^ *j -3 a) 3 - xj & u - cS 7? s c« •a e ™ ■2 "O "B »< C T3 T3 +j ™ "3 "3 o CU >i TJ o h 3 3 > Q, 3 3 cj o o o s CU P o S^t! Q £ £ 2 fa C3 Q CM Lf3 CO co i-i CXI in be >, a irj ti< h C- CM CM CD o (3 O CO >H CU a o o 0) S-, cu bi 3 O H C35 1 CD CO 1 e- 3 o > u x: CO 3 O "co cu & at cu CO O CM E- CM o co cu cu u cu No (favorable) Yes (unfavorable) Did not answer CO (5 cu -a 3 o a CO cu t< o Sh CU a 1 7^ 5 & CO Rl cu cu J p 160 AN INVESTIGATION OF RESPONSE ERROR cn d> n ■O h p CD T* c >> 03 O £ O £ +J W E-i 5 -a 3 — < h o a> O o •'■" , <+-H o * H CO a xi Fh rn ^ CO -fH rt bC X! c •S -ii c o > CO en CD 3 cy aj § 0) x; c o 09 C o cn o> 3 5^ 3 a> -C u 3 £ " O O T3 Pn fc« f— 1 '- CD 0> -4-> ° £ T3 XI C o CO O & Ph to a> O K co QJ JD eg H ^s 3! 3 CO 0J g £j >> O u (H > .3 -w -M CD ctt rt O O > -g fc £ & £3 I t- I ■* O a) o -a aj <* u s CD g £ — 1 a< c •S^OShCOSh-w "CDOOCDOCDO ^>bD>Q.ft>C 3> Q CNl CM in •* © iH CO » 3 01 oj 55 to Oi 0) CD ctl .3 ^§ £ £+, o •n T3 c cd ctt K K 2 5 S 3 * 5 £ ■a o -5 o cd .a ■S .Q tt Ctt O 2 > ^ * " > g f. c 5 > o cd 3 nj Pm 3> Q THE MAIL REINTERVIEWS 161 a; -a Z, ?> a > « 5 •tft H 0O i-H 73 O o o O t}« 00 CO C- C- m in ■^< to ^1 ITS ■* CD a u K OJ 73 £■0 co f- * s cd £ » 2 73 3 *" P cd > id CD fci |3 t fc £ s* £ :S n o o X K K - o "• •s -C a n O Si CD cd _ ™ CD g — ^. XI 01— Q) Si 513 CD .Q -£ >> CD .O "< c nfi 3 O rt o ^ ° ™ ti !r cd lrco_f-Qj •2 "3 > !=> ^ Pm o cd "S ffeDpo^ti-5 5 * •3 to c -a 3 £ 5 £ O. 2 ^ 01 CD CO 10 c c 3 +J d a cii O O C o o c c 73 73 c n 73 73 ^ a O O £ CD J3 J3 a ■>-> c C C o CD CD CD O 73 73 73 ^ C *h C C o O O o a a a 4J CO CO CO CD CD s *- CTJ CO «~ 1 ,— 1 *j CO CO CO O CD CD CD C 73 73 73 3 3 o> CJ i/i O O O X X X a w J w w 162 AN INVESTIGATION OF RESPONSE ERROR in the mail survey and response error. The tables for the cash bor- rowers are followed by those for the owners of savings accounts (Tables 82 to 89). If the whole set of sixteen relationships is re- garded as a test of one general hypothesis - -that good rapport leads to accurate data --then that hypothesis must be regarded as con- firmed, The relationships are consistently in the direction that favorable answers to the questions on the mail reinterview are as- sociated with accurate responses. It is not easy to compare the closeness of the relationship between the two studies since there is a single measure of success for the cash borrowers, reporting the known loan, while degrees of success are more important for the savings account owners. The data do give the impression, however, that the relation between rapport and accuracy is closer for the cash borrowers. Further investigation would be required to confirm this impression. Which dimensions of rapport, or which of the forces measured, are most important? Again, more elaborate analysis would be re- quired for a precise answer. As far as the study of cash loans is concerned, inspection of the tables suggests that there is not much choice. Considered in isolation, each question seems important. The similarity of results suggests that the questions may in fact be measuring the same thing. The investigators have not conducted a special study of this topic, but they have gone far enough to dis- cover that respondents who give favorable answers to one question do tend to give favorable answers to other questions. Study of the tables for owners of savings accounts shows little relation between rapport and one type of error: failure to report an account wholly or partly owned by someone other than the respon- dent and his spouse. This result is consistent with the interpretation that, psychologically, these accounts typically do not belong to the respondent and that, in terms of understanding his behavior, one should not take them into account. The most explicit kind of refusal, on the other hand, is to report the account .but not the balance in it. One would predict that those who give favorable replies should be less likely to respond in this way. This prediction is supported by the results in all eight of the rele- vant tables. Among those who do report a balance in the selected account one would expect that those who respond favorably to the questions in the mail reinterview should be more likely to report accurately within $1,000. The data do not seem to bear out this prediction. People with poor rapport who do report a balance seem to respond as ac- curately as those with good rapport. This result suggests that rap- port is not the whole story; it emphasizes that other variables also must be taken into consideration to explain differences in ac- curacy of response. THE MAIL RE INTERVIEWS 163 CD r^ rQ -H a ra S3 A O > a t- OS 00 oo m 1 1 in o i— t s of ewer lear w about cm 1—1 1— 1 in l 1 in o i-H i— ( CO -rH O CO CD > d Su int maki: study 3 r\ in o in o ITS i-H OO O o o c o CM i-H 1—1 i-H CO i—l in o CO • i-H > 1-t CO - p •rH Q) cd .3 lO o lO l— 1 •^ O oo i-H o H CM i-H i-H i— 1 CO i-H in o t- 3^ 3.2 1— 1 Sh CD CD G O o a CD cd 2 T3 cd cd CO o CO 0) rQ (1) 3 • rH cd CD CJ o >1 +-> c c a £5 rH a* •-H ^tf CO +j CD 0) CO CD 2-° o -t-> rH CD 5h CO ft o .2 w o CO a o o H-> CD -4-> rH O 2 rH CD rH O 2 m "S T3 a ^ t« rH CO ^r-l CD u c O rH o ~ o o r-?c cd u 3 o o o cd o o o O o o O o CO rH 2 o CO cu £| cd 4-i i-H o o CQ o u >> •3 c rt rH v± i-H i— i r* cd ►-3 o cd rQ -(-> o <*H >> €«- €«- CD • rH •° .2 a> Sq ■. s rQ >» c > T3 O -a ® CU H 3 o CD o TS rQ 2 rH CD H-> o CO -i-s 3 o CJ CD T3 +-> O u CJ cd u cd T3 rQ -r-> rH O a CD u O •rH r* CD •rH >H— 1 O o -r-> ■H CD -M CD CD A -M a f-> TJ H-> T3 rH CD cd o H 3 0) CD rH rH rH rH rH 0) O . o si -4-> O H-> CD rH 3 CD O ° rH o a a> CD rH o CD CD > CJ o rQ 2 p •ft cd G c« -t-> CO g cd •rH Sh CD CD h-> CJ G a d CO d d CQ £ £ CD •r-l Oi > " ^ CD >> -r-> 5 d G ^ g CD c3 xi •-3 CD Jh rC O u g u o w •rH -4-> rt 0» 1—4 Eft CD g Ph o CO £ rH CO 00 CO CD d O > d CO Oi ■"*■ CO in in M 1 o rH CM CD h-> G CM H CM lf3 1 m O rH CM p 73 G G O UH CD CD i— i rQ ctJ rH o .£> > CO O CD CD c- CM ^ rH OS 03 CM t- 1 i-H CO rH »io Oi Tf CD > -4-> d £ rH la £ CD CO > •i-i CD u d •£ CD a e: lO o lO rH "^ O CO 1— 1 o r-t -H CM rH rH rH CO rH IT3 o C- g CD rH •rH O CD G U O •i-i 3 a 5h -t-> CO •rH 73 CD g CD O CD o G bC CD rt -2 a 73 i—i s CD o CD ^2 CO CO G CJ U CD CD O G G G O a rQ CD •rH oi -t-> rH CD H-» Pj, ■rH pj CO -i_i CO CD CO CD o OJ u rH 3 d O O CO o o CD 2«° o -r-> c CD 73 G a a c M H-> H-> 73 CD s O a « Oi •rH rQ O -*-) rH o - CO rH CD Ctf tf 3 rH C O ™ G CD CO G G O a CJ d o o rH o H-> g a r^n CJ G CD rH o o o o O o G o CO CD • rH Bt ■y o C o. d r— < H-> B H-> rH o^ o^ CQ o o d >> -t-> rQ u o rH OS- 1 • rH G rQ fl a> So 0) rH n, G rQ >. es > C\j 1-3 H-> rH a CD U o •rH 73 O -r^ H-> G G CD CJ 73 CD rQ 73 •rH rG -t-> u CD H-) 5 ° IB G -r-> c g O rG -r-> -r-> G rH O o o 73 73 CD H-> d 'd rQ 73 H-> rH o a CD rH CD -t-> O a CD • rH CD 3 -3 G trH O ■*^ G CD CD r4 CD rH u rH 73 CD rH O , O rG H-> o -4-> CD u 3 o CD CJ ^ o o cj O ^H o a CD o a CD 73 G CD > ° Ph rQ a p • rH Ctf •S 4) B oo w .2 _ bo > ■•-I -rH (V) CD +-> ■S 3 ° rt § c CO a o 0) y >r bJD 3 ►£ ctJ q t-3 C . CU c5^ CU o . ^1 fi CO o ft Si ft "-^ O W S « w CO CO ^ c Si a l co ffi a> *2 a H CD a 1 O > « ■i-H 73 of in CO OS in o CO 1 ■^ o CO CM i-H CM m 1 rt< o i-H i-H S3? E | o ^ •a d 42 0> © > XJ ft 3 ® CU +-> -i-i < bfi 3 CD 1— 1 LO c- CO 1—1 ■rt< CO J5 in c '« CM 1—1 1—1 CO I— ( lf5 O in 0) o i-H ^5 > a CO - £ -rH > cu CU O ft CO -° CO -m S3 ^ o CU y CO ct CO CU u o CU u IS O ,Q o a* ft ft CO o a O a O in -t-> c 0) 73 .2 ^ o T5 CU CO CU S3 o 0) Oi CO 1-1 2 >> -M 3 a o ft CO o -t-> >> c 0) ^ ^ c Z! o C ft CU CO CU o O U u o o o H O o o o o CO rt CD 2 tf 1 •-3 O o o -4-> 5-i 2 >? Sh i— 1 +J i-H -4-> c E3 C o o CU 73 1— I >> J2 i-H €«- c CO CU ■l-l > cu o 92 o o =( y c CU T3 -(-> ft CD J-l c 9 a a o y d +-> CU • l-l c B* * j3 05 o CJ 13 i— c a ^2 o ft o CU <*-l +j cu -i-> CU CU ft -t-> o o -4-> C .C C tn T3 +-> T3 u CU a -i-> o H 3 ^ ^ CU f- CU u u u fn 73 0) o . o s: -<-> o -(-> CU $H 3 CU o o o o O ft CU ft CU o ft CU (3 CU > CJ o ^2 a "rt <: < tf p o < fa « tf g tf T5 c s o Jh «M o 0) CO g oJ CJ CU -Q -t-< c CU o Jh CU ft CO o CJ o Sh i-H CU ft o +J in ■n o TS rt d rr( -i-> ,-C o -t-> 1-. CO L0 CO CU CU o .-1 Q 166 AN INVESTIGATION OF RESPONSE ERROR c o • l-l cd I—* « c •i-H W c -4-> o c en =5 u O cd CO c CD CD •i-i CD • i-H > > u nH Eh CD a O n c >— i cd n CD cd x c W -t-> o •rH ^ TS -4-> 42 CT> CO in 44 05 •fH r-H -t-> CO r* 0> „, CO™ cd a-s •-3 CO d) CD n fH Ph Eh U Eh o CD -i-i CD Eh ,_( Jh w CD CD CO n y ffi CO o £ 42 cd -D cd H-> .H CD CO l£ r3 -£ <.s CD u CD O c c •rH (^ rH 15 O 42 rH rH ,-rT CD O^ lO CD "^ CO * H a^ c cd 1-3 CD CD CO CD ■^ O CO H -a CD CD CO o a CO rH O H-> C o a CO rH r£> H-> >» rH ■° o T3 ■>-» o ^ H-> CD C 43 3 < >> CD 42 M H-> 3 rH O Cd ft Oh CO .5 ^ c rH Cd O H-> >> c r-< CD CD rg *-• c £ O C Oh CO r« CD 73 rH CD ^ 6 c £ cd O 43 H-> fl rH 3 CD O 43 ° n T3 rH CD a CO cd ■M o c CD a 5 ed 42 o o o cd T3 CD -4-> U a CD Ph O cd ■ — i cd 42 H-> cd H-> CO o ■*-> T3 CO S3 <+H Eh +-> -t-> C O o o cd H-> Eh O a, Pi O o o cd Eh O «rH o r3 42 •C -4-> Eh O Oh Ph Eh O O O o i— I €«- >> 42 H-> Eh O Eh Eh C rD Eh O O O O 42 T3 -M Eh O a Eh Eh > O o o o 3 c o Eh CO o 42 -4-> c o Eh _ & ° 2 Eh " Oh Q CO *"^-r-> > cd E3 5 c CO CO CO o rJQ o Eh 42 a (0 -Q THE MAIL RE INTERVIEWS 167 CD PES DQ bDT3 2 o ^ torn *° e - > 5 a> sh Fh 3 >> 5f c «J oS > i-a Fh fh w 3 w> e cu o Fh . 0) & Ph £ w 4J CD CO 4j o a CO CD PC, CO a> --< JO in CD -4-> Co u O > ITS CO 1 in 00 CO CO 1 CO 1 CO 1— ( c» CM 1 CM CO CO 1 1 CO O a; m a> OJ xi .2 T3 •a p 3 Hi O O ^J O ,_„ w ^5?* 0) . — 1 CD Fh 03 IT3 1— 1 ■ fa CO ~ £ •<-t Q) o5 .S in O 10 1—1 ^t< O CO 1— 1 O i-H CM T— 1 1—1 1—1 CO 1— 1 10 O t- . i-H <.S a) Fh 8 CD CJ O c a T3 CD o3 CD fi X5 CO CO • l-H a) 3 O a CO >a 0) o3 CD ■rH rt £2 CO $-1 CD O o3 CO CD U O B CD t— 1 O £1 Fh a w CO -4-> O a u 0^ 0) OJ CO rt c „ 13 3 c CD a Q a CO C T3 CD T3 -4-> O d c T3 CD CO "H-H CD U a ni u i-H oS H-> Fh O a CD >, Eh 3 -4-> i-H €6- i-H €/3- co 1 a c c £ O d 03 CD O C a 13 X2 CD -4-> O >> CD • l-H X! +-> > Fh CD H-> a • pH u oi "O -Q a O CD «4-H +J CD -4-> CD o> a +-> O C X! C ^1 T3 -(-> T3 u 0) oi 1— 1 -4-> O 3 ** 3 o> CD Sh CD u Fh Fh Fh O . si -t-> O -M Fh 3 CD 5 O a CD a 0) f-i O X3 CD > O CJ a 3 •I-, d < < K a CD P O < fa PC, tf 15 bX) B C O Fh CD CO 3 o CD Xi a CD +-I O T3 (11 T3 a 03 +J G ^^ oi Is CO •r-l CO -i-i CD CD H-l Q 168 AN INVESTIGATION OF RESPONSE ERROR a o •iH 1— I tf faJD a u § ° ° * < § * 8 * 0-= CO CO rt ^ 3 bp "3 Ct$ 5-i o u «4H 0) sh£ o U 0) 5-. Sh W CD CO C Sh oo CD I— I ■s c CD o S-i CD ft CD i—i Sh ,Q C35 CD CD i-H c- "3< rH c- rH ■^ CM CO O bD O S h-> -ph o > > rt £ CD 5-i Sh A "a T3 p^ 5-i ^ o CO _ H-> 2 § CD -t-> o CD T3 3 CO CO CD 5h 3 CD a 1 a 3 00 (M CO m tr- c«j CM CO O CO • i-H H-> H-> < o > CM i-H T-H ee CO O rH Sh >-> CO 5h O H_> CO Sh 3 CtJ O ft a O CO CO o H-> Sh O a Sh O ^ ,-rT CD 05 H-> -H t3 -t-> o c CO H-> Sh O a CO rH -t-> T3 C o Sh C O C^ o S 2 «*H Sh o u o Sh O o o Sh O O O o o a^ s g •-3 3 o CO 5h p5> ■rH C 13 -t-> rH o rH co •l-t > Sh o H-> 5h a CD -4-» CD *h ^0 4-> -4-> O o O «*H O a c 2 5 c o o o o ctJ i—i -4-> Sh O H-> Sh •iH 5* S3 •|H U O +j CD o X -4-> H-> aJ ,Q a «4-H o o C XI C Sh 13 H-> TJ Sh rt "cJ 3 ** 3 CD Sh Sh Sh Sh Sh T3 CD O , O X! ■*-> O -•-> Sh 3 o <-> fa <-> ^ Sh a Sh 13 eg H ,Q o o O ° a O ft C3 > o a 3 It • pH < < t« ED o < fe « « a SP •i-H T3 § Sh CO CJ ,Q H-> c o Sh ft O O T3 -t-> O c CO o p THE MAIL REINTERVIEWS 169 c a •l-l +J CO § s 8| ■is .si o C CO 3 2 ctS 3 ffl o Q 01 >» S 2< h a c C$ 0) w5 0) Sh CQ 00* CO Eh co • rH n SP C o CU r— i +J rO 3 ■8 § cd o > in o in t- CO CM CM Tt< o i-H CM 1—1 1—1 i— 1 in i—l ■^ o ■^ «»5 i-H ■i-i n co rt •s a 3 *H ■a o o <4H "^ ° bD S > cu 3 ^ -r-l ed t- o t- CO o t- CO o o o bfi u CM 1—1 1— 1 c- CO o CO 0) o i-H fe > cd En CO ^ £ •rt CD OJ .S a E in o in 1—1 ■^ O CO i-H o i-H CM 1— 1 i-H tH CD i-H m o c- r-H 0) T-H r— 1 -r-> CQ O a o CQ >> CI •■-I +-> i—i ci o a •Q $ CJ a c CO -M 3 o CQ Jh ?_, Jh o CQ O Jh O rO o &< CO -t-> O a O a Fh c •5 -a c Jh cd -M o T$ -M Jh cu os m rH o a CU Jh CO 'T! Jh O a 1— 1 &*■ >> Xi 4-> Jh O o i-H +J • rH fs co 1 • rH > Jh H-> •rH < O s -^ c ^ T3 -4-> 73 Jh rt i-H -t-> o 3 -•-> 3 O +J Jh 3 i—i O O T5 > o a •f-H cd < < Ph K 13 o < fe tf tf g 170 AN INVESTIGATION OF RESPONSE ERROR a s .5 d i2'£ Is 80 CO bp 3 _ o > <3 ct o c T1 CD O CJ I— I 0) CO *-! CO Q) faC c CD O (h CD ft O P* ^ ,2 w a o CO ft ft s CO ^ 05 2 Ol OO O ■a u u o CD Ctjr-C CO CD o •5 S .5 CD CD ft CO t _ t fc •r-l rr( • rH a > u < CD c •i-H CD tn CD O c c •rH tf *H 13 O rQ CD O 3 lO CD O 1 CO rH c „ •-3 ^ O CD CO co CM OS OO CO CO CO Oi CO in CM CO O o c o c Cvi CO 3 a o CO T3 1—1 cci rQ O ft >> • rH cti H-> rH H-> CO rQ cd H-> CO £_, H-> CO U o 5-1 3 o rH d o CO o a O -l-> ft ft d H-> a rH O (3 T3 a • i-H CO T3 H-> o 13 CO rH H-) B H-> (3 B o o CvJ X5 -4-> rH o H-> O o o rt +-> u o ft CD U O ft CO Sh >> rQ 73 a o >> +J c •rH o o O >> r-^ r-l • i-H H-> C T3 o S3 T3 S3 o a CO Fn c rt 43 Eh d r— i rQ O u u 3 o o u rH o , rQ T3 H-> u o ft o o o 1—1 &■ >> rQ T3 H-> rH o ft o o o T— 1 r3 -r-> •l-H E5 -M 4-> ■*-> o H-> a 43 s rH •O -t-> •o rH nS 3 +-> ^ u rH rH rH T3 1—1 o o 43 H-> o H-> rH r^ o Jh o -»-> rH ft rH T3 o o o o O O O S3 > u .1-4 < < ft D5 ft 13 o < fH P5 P5 o u rQ s THE MAIL RE INTERVIEWS 171 The Mail Reinterview and the Experimental Manipulations It was found in Chapter VI that the C interview technique tended to produce more accurate responses than the A or B techniques. Were the differences in techniques reflected in the mail reinterview? The data relevant to this question are shown in Table 90. In only one re- spect are there any differences among the techniques. Respondents in the C interviews were the most likely to give a favorable reply to the question, "During the interview did you have any doubts as to whether you should be giving the information?" The C technique was not particularly successful in convincing respondents that the survey was a good idea or in making them feel that they had a clear idea of what the study was about. The use of "props" in the technique, the special explanation of the purposes of the project, and so forth, do not seem to have impressed people with the usefulness of the re- search. Where the technique does seem to have been a success was in the increased confidentiality and anonymity associated with the sealed envelope. The use of the "props" may also have contributed to the respondent's confidence of his anonymity. It will be recalled that the A respondents were given the opposite treatment; they were asked for by name. There seems to have been, as a consequence, a considerable difference between the A and C groups in hesitancy about revealing information. Summary Tables Table 91 summarizes the relation between rapport and response error for the cash borrowers. The score on favorableness was con- structed by adding replies to the eight questions. Of those with the lowest score, only 16 percent responded accurately, compared with 45 percent of those in the middle range and 53 percent of those who scored high. Table 92 shows the relation between two sub-groups of the ques- tions. It illustrates the point that the answers to the different ques- tions are correlated. Conclusions The evidence presented confirms the hypothesis that there is a relationship between high rapport and accurate response. It also indicates, however, that rapport is not the whole story. The implica- tions of these findings will be discussed at more length in Chapter IX. 172 AN INVESTIGATION OF RESPONSE ERROR c o £ CD rr| •i-i •—> > cd U tf CD c •rH •rH CD >> Ph T3 1 3 • r-l -i-> rtf {/J § C a CD o rC HH 1 -M to o cd O en o CD •rH +J CO (D c 5 • r-l rn o d H-> o CO -M tu ed CO c ' -2 •r-t U H-> to .rH A CD S CO faD cd c co a> co Ph . CO 0) Ol 00 o a» oo co CO CO OS CD lf3 t- CD CM CO co in 00 CO as co 00 O 00 oo in c- LO 00 9 ■a a c O- CD -M ■ r4 1 ' d > -a o u 0) XJ cd 0) u o -3 cd CO CTJ +J c ■ r-l CD CO & XI rH cu I • rH > rH CD rH fl CD XS > CD U rC CD is H-> ■ rH CO CD "£ rC rS ** 3 T3 O r-i Q £ Ed 5 « tu cf 3 .3 3 o • rH T3 +-> CO CD U CD T3 CD o >> T3 •rH T3 I CO 2 CD rW CO >> > rH co rH CD X! H-> CD rC O r2 - 'S CD O CD O rC bJD O rC H-> o rCl o •rH rC r* co <>• o 3 3 o >> o o O" rH CD Jh CD CD rH CD co <>■ a a CO .2 H-> -t-> X2 rt S 5 ■« o ^s d ^ CD rC rC g> >> bD T3 CD ^^ ^2 CD 3 •r" o H CO CD ■^ a CD Cd r* rH 3 rH O O CO CD > ; 'Sd' a cd bC -*-> S CD r-H CD s r^ CD +J 3 c cr 1) • l-H n rH rC n O a CD i/j CD CD rH a -r-> o CD •rH rH > a CD T5 rd T3 X CD ti) cd rC CO CD CO -M CD* CD" rC a a cd cd hO c c n CO CO > 4-> H-> n d c LU CD CD ■ ' T3 T3 CD C C U o o o o* a CM CO CO CD CD CD rU u u -a <4H «rH CD o o ^ CD CD T3 T5 r cd cd u a a r-> CO CO UJ cd cd CD H H • rH CD CD r* CO CO CD 3 3 H-) o n r-H g ^ < PQ O THE MAIL REINTERVIEWS 173 CO a co CO co u •i-H a d 5 CU > a •i-i -M co a» CU -M .s 3 <4H o c -.-I cd o •r-t -t-> i 3 a o co S-, •|H CfH •d ro 0) rn bO Q) cd C CO r-H C 0) £) cj m in u 0) o P< > 03 [n c o Sh O _ ra § CO C s - cd Jj P c 3 .3 -4-> C CO o cu cu a* u s^ ■ s O Moo co c- i in •*# i o o i— 1 in cu 1—1 m eg co ^« CO i-H -o CJ CM O 1 ►J o to ct in O o i-H T-H CO -H £ ■3 a» TO tH a > « $h —• CO >— i -t-> «.s Sh t- ^ OS co in o o i-H CD IT5 CO c cd O •—> si CO cd o u o u o u u 0) cu CO o a CO 0) T3 CO Reported known loan Reported other loans only Reported no loans "cd +J o H CO c CU C o a CO cu O CU -a a CO CO T3 Sh CO rH o cu cd e*H CU £5 a cu 3 1 CO cd CO c Jh CO cd o CO o > cd CO Sh ■H r— 1 •i-H pQ CO CU cd si J3 -t-> SCn O a a > 3 o cd e*H •r-l 3 cu Sh o CO si a o CO c T3 o cd CO si ^H CO -t-> cd Sh 3 3 O .Q T3 •i-i CJ CO CO > Sh =3 CO a c •r-l 'fit c CU CO o J3 3 +.1 CO <*H ft bi CO o ( _ s i3 3 cr a CO c •rH T3 i-H ^ o s- 3 • i-H CO • rH cd -I-" o a cu si CO CO 3 Sh <4H CO si CO cr 1 o CO m cu CO CO CO CO 3 c cu cd ^ fl d CJ 3 cu o cu -M CO i-H J2 CO rQ u a c +J o o o •rH cu x: > -I-I CJ £ a cd ^c Sh <*H 3 CO CU CO -t-> P3 o •pH CO o a CO T3 CO Jh Sf •rH -l-> CO CO CO CJ Sh o o 1— 1 o a CO o o CO 1 3 a* CU o -I-I CO u 1 — 1 cd •rH > Jh CO o ■d T3 tHH CO CU -H cd o CO cd CO cu -a CO u CO > o -t-> a •rH CO (h cd si -i-i -i-i o £ u > Sh cu -l-l a co •r-l J2 -I-I CO CO CO cu X w -a -a Sh 5h cu HH & •r-l cd 3 cd CU J3 is o y -o X o o rH 174 AN INVESTIGATION OF RESPONSE ERROR o .1-1 3 % £ S3 .rH (3 o • r-l -t-i C3" 3 >> +-> u •rH cci CO r-l M 3 'to c o £ 'is < •rH 0> • rH X! > > rH CD rH d H-> CD £ s3 -t-> •rH a o H o a> en c Xi -M 73 CO CD S3 o H-> 3 r-l O u CO H o > • rH 73 co cd (1) CO rM hD 0) cci c c H-> CD o CD 2 CD o rt $H rH rH o 0) o o Pj, > CO d fe 0) rC c -t-> o o cd -r-> r-l O o CO rQ cci H 73 O en 3 gen CD Kf rH O > C O a> cd rH rC o + J o CO rC 73 73 "tf £ CO O I rJ O o CO t-H CO CO CO CO r- £ •rH CD CO -rH s M -i a> rH 1) >> r 1 a & cd d Sh rH 3 O a > u a3 cd CM c rH cd <]) rS rH o o u CO o o o o o CO CO iH CO o H <*h « CD O bp^ S-H -t-> rH • 1 ~ l CD >> rH O-rH CD e ft.*3 CD rC -l-> c o 73 CD CO cd rQ CD Sh o CJ CO CD > d C o CO u ft CO 73 c CTJ o o O S3 ■*-• cci CD -2 U "H CD I* -ft CD CO r-J S3 +3 O 5 a o . rt a 4 cci 5 -H "H C s_. -4-> nj h-» O rH 3 "9 2 3 0.5rH ^ -M S3 "O S M > r3 ° 3 SO Cd ■+-> O ft 10 O+J 5 CD a. 3 o* co -2 CO ^ ^ CD CD CD -2 2 o- CD c -e T3 S cd jg 5 CD c - -rj CD ft O rn CD CO CO B %%d CD ^ rl rl (/J O CD > rt « fe 6 CO CD rC! rH CD > „ g ►* SH cn ^ 33 cd rt 2 73 c f* > CO ^ CD CD O cfj ts d O !> -rH CD CO .2 73 S3 ° cn rd .3 H^ .. cd ft-£ C .2 CD Sh +j r5 M bC ,Q rC S3 CD s ^ ** -s S P ctJ cci •" 2 rH rfi H-> CO CD ^^ ^ « O o <" m Sh CD O CD ^^ 2 ^ rQ .2 2 ^ ,2 CD r=l « 73 - ft rv rt o « o ■* ^ CD ^ Sh JH CD O O rH 73 O cd CD S3 CO 2 -= ° ■ lis- 1111 u a 3 XXX XX X XX a 3 'J* m t— 1 CM so ^< tCM l 1 M Tf tM C- lO 1 l M Tt 1 I CM CM CN I I CM CM -a u 0) e nH •a i—( *a< c- t- CO **< l^-t ' i-H t-< CO CM m 00 i-H ' N CO »-l CN Tf N M J < z CM . cu u 00 in en 3 cd -3 cu '~ l 'S U v ^jT o 0) ^_, "cd fa B -3 O cu O 3 >. CD 3 8 s 2 Sh u o ■c 3 3 O 3 3 s o a c 0) co 3 o a CO o o CO >> £3 u cd a 3 '3 cu u cd 0) T3 3 ^3 co ~3 3 O CJ cd ■a 0) I* a cu u U o 3 CD •a c (4 c CU ,3 cd 3 c .3 Eh B o cu "5 (h 3 c o a _>. u 3 u CU > O CJ K fa CD 3 O CO 01 en u iS 3 CU co a u (DO)OJC7JOj01fliOJOiC7iOi010iOiO>0)010iOJ Cfl o u cd J3 !h O < >C71O)O>0iO)C?5O)O)CiO5CT)OlCT)O>O)0)Ol01 3 o u cd (h >> "3 .□ +j 3 OOlOiOlClOiOiOlOCDCOC-CDiO^COCNH a en CD 73 cu -0 3 3 O 3 O ri CU J "^OOir-CDlO'^COCMH cj 1 1 1 1 i 1 1 1 1 1 1 1 1 1 1 1 1 1 a; o a, a> V 3 O 3 c 3 OOOOOOOOOOOOOOOO O O iH 5 o O O cd cd T3 000000000000000000 CJ cd ■a CD T3 £ OOOOOOOOOOJOOC-COlOTfCOCMi-H o c 3 ■a cd OCOD-CDlfi^COCMH 66- 3 3 CU u t- cu cd T3 o o O O u a O a ■- 3 o CJ a cu cu cu O a cu U '3 < < K K < fa K « 298 AN INVESTIGATION OF RESPONSE ERROR ■o 11 u nt _2 .Q .o _Q ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ. ZZ ZZ ZZ o c-icq^ U CD c ,,H -a CL> o Sh _Q _Q J2 i] X i3 J ^ 3 o ft >-( CN1 CN1 t-( ,-1 » 3 H en CD _D _q _o i: i: j: x x o •5 > ■^NnnnNH j ' oinnn ] ' ' ' CD o o £ CD O c ol rt X! co U5 > c .a 2 3 <4-H J. O o' S -£ c OS o 3 ■o ft CD o S *H rt o ft T3 cd ni CD u >M Sh c 3. O at J3 #rH ft tn O u u CO CD pj CD W ►J t-, § < a 0) o^ cn> en en Oi en o^ 05 ►> I < HWco^iocDC-cooiOicnoioicnoioia) o w w H(Mn^U)(D^CO'S > Sh « c c ni CD 3 o a a 3 o a "-HOOOOOOOOOOOOOOOOOO o c oooooooooooooooooo ^HC-gco^ifj^c-coosooooooooo lw HWCO^IOCOC-COO) o a r. c CD o o nt m in 13 CO 3 <" !5 CO CO CO o -^ t- o oocomoc-co ifJcoooj^ONH og2 ■a . o cd eg o inomt-cMcoO ^ft-Oincoincoin CM tT O c- o ■* ^r 1 •^ (inmcDmoo-^ ,o«-Hmai'-icot-'^ Tj< ^ ^ ■ s r- in cm t Ol-HOrHl-lCOOO'-lOOOOr-t^HTHOOO — 3 3 2 id o o o c a> u o o o co rr- co t- CO CM t-H w »h ■y » 01 ft t- en CD OS s CD OOOOOOCMCMCMOOOf-HOOOOOO T* 3 o o in 2 c ■* ^r i CO CO 1 CO 00 co" h CO CM | CM CM 1 in in Cfi- ft o> _. cn CD "3" ai ^-^ -— . ^-v ~~ _ 13 ^o CD -^ CM NH i-H OOOOOOOCOCM'-hOt-hOOO^c^O^^h^h s 3 Pn co ' &*■ 2 i c o 0) uj o o CD f- Oi Oi tf in in rt CM t-H CD .Q CM 13 f/> ft 3 fcl < CD -O m ^ ^ O O^O^O^O OOOOCOOOi-HCMOi-I^HCMCMCM a CO CD 1 «} B o U O CO t- CO O CO t- «/=► t- £ ft h CD 01 js CD Si S 3 2 CM i-H "— - ' CO ^ ^ CO CD OHOHHntMlfiCOHONCOHpg^CNlMM 31 1) c £ 0J o u •h ^ r- o co -^ CM CM ^ *H ^H CD ft >, J3 5 C O ™ O . -w -*-• ft >> C 3 en & "> 3 • u~& O U 3 3 o >> c 2 « J= fc, CD £ i§2 8 *» 13 « o H > O c3 « <" T3 — 1 3 CO ., t4 w o 5 t,0)O)a>0iO3aioicnoJ0)0iaJ0J0icii3iO30) o < o050)0>Oi03QOiaiaioiaso)oi050JO)050) 3 j fjOlOSOlOlOiQOlOlOlCOt-CDlfi'rJ'COWH COC-(Dlfi^CONH OOOOOOOOOOOOOOOOOOi-h R° to (V OS — -a z! -a £ H CD 3 OJ N |i° la *j 3 CO CD o rt rt o ffl-cco c 73 2 2 -a CD fc. t, 3 "K ° 8 O o §■£•! CD CJ 3 s o c ™ ° o °5 O c £ 3 t* *■ 3 "" 3 CD ■a o cp o x: cd a u < oooooooooooooooooo J3 T3 CD ooooooooooscor-cDinTfcocM'-H cncor-cDin^cowH €0- - o« a ° o e3 < < §"« OS Q, ti- OS S 300 AN INVESTIGATION OF RESPONSE ERROR 13 p oj in 3.2 -3 (M co o ir> eo t- lo o m o o o 02 c- co in h cd to r- o c-oom eo t- m co co in co M , , , cq cd t> o , , , , , C*f .-T rH N ,-1 i-l t-I ' ' ' H D- ^' CD' ' GO 69 2 fe u h a) > o a> 1 o" OOO^OOOOOOO^HWrHOOOOO 3 3 "O 3 B o 2 c o o 5h o o m 69 s en aj en OS xi 3 CMOOOOOOOOOOOOOOOOOO p s Ci •* 3 69^ i o 2 - o 0) o o t- o CO CO QJ €» PU OS u „„^^„„^^^^^^^^„„„^^ 3 b en CO a; a iH o'oS'o'-hoo'ooo'o'o'^'o'ooooo CO 3 ««■ 5 cu o c o 2 c 0) u cd o 13 o o CN kl o rt 69- A < OJ OS S S CMCMCNCNCSICM^HOOOCMOO^HOOOOO CT> 3 2 &9- | C O 0) o 69- ft o Sh oa 0) a 3 o ^cMMnnnHOOONHHNOOOOO > 3 CU 2 c c 0) o o o tH 0) Pn >> a o c c 3 3 O rt c-i o a, ai S •a 0) u c h c - —i CO o a, CU O Q. K o OT il — ■ 2 01CJJO)010101010>01C7)CIi0105CBC310)Oi01 i_ a) "re cnoicncjioioioiojoioicjioioicicftcnoicji q 5 J3 a> u ^5 .1 ■° rt" 43 _< — TO ? 3 §33 cu CO B 9 O fit n ho o CU n, 9 o a, c c ^ ■« c M B co bJO rt ■n CU CO CU £^ to 3 x: 9 f J £ O fe ■§ w •-:■.-:-•'•'■ -■'■•■'■v.-.-:. ,; ; .; :, ■; , ,':::■■ .:■'. ■" , .' : : ;: " •;;;■■. '■: v.- v^ :-.■.,: /. •■ ; ™fe--^ UNIVERSITY OF ILLINOIS-URBANA 3 0112 052633424