College and Research Libraries Research Notes The Library Survey: A Research Methodology Rediscovered Helen M. Gothberg Survey research is based on sound theory and the development of hypotheses. It is systematic and controlled; it uses the language of variables and is concerned with relationships between or among them. It seeks answers to the posed hy- potheses and makes use of statistical analyses beyond tabulations. Survey research should not be disregarded as a research methodology because it does not meet all the criteria for em- pirical investigation. The quality of surveys can be improved if researchers understand the basic differences between descriptive surveys and survey research. Some ways of improving the quality of survey research include adequate planning, sampling, data collection, and the use of appropriate statistical analyses. The use of the survey as a methodological approach to the scientific investigation of educational and sociological phenomena was widespread by the end of the 1920s. Librarians' interest in the survey was fos- tered by the publication in 1926-27 of the first comprehensive survey of American libraries. 1 Literature about library sur- veys is not extensive. Several books on the subject have been published over the years, with the last one appearing in 1967 by Maurice B. Line and then revised by Sue Stone in 1982. 2 The few books on li- brary research methods that are in print provide additional information by way of a chapter or two. Such a limited amount of publishing related to library surveys is surprising considering that a search of the ERIC database indicates that between 1966 and March 1990 over 4,600 articles and research reports that were surveys had been entered. This number does not reflect in-house surveys that never made it into a major database. How good are all of these surveys? Rob- ert H. Donahugh suggests that they are time-consuming, costly, and redundant. 3 He pleads with library researchers to exer- cise restraint and to determine whether the survey under consideration would make a significant contribution to the pro- fession. Robert M. Groves observes that survey research is not an academic disci- pline, and that there is a lack of a full un- derstanding of survey quality or agree- ment related to the language of error. 4 Keeping these limitations in mind, it re- mains safe to say that many of the surveys that appear in the library literature seem Helen M. Gothberg is Associate Professor of Library Science at the Graduate Library School, University of Arizona, Tucson, AZ 85719. 553 554 College & Research Libraries more focused on what was found than on the soundness of the survey methodol- ogy. THE PURPOSE OF A SURVEY "Let's do a survey" is a common call. It is frequently answered in the affirmative without a clear understanding of the sig- nificance of the undertaking. Anyone who is considering a library survey should first ask themselves what difference it will make and what action will be taken based on the survey results. Too often surveys are taken only to be put on the shelf and forgotten. There is too much time, effort, and money involved in doing surveys for this to happen. To the clarion call to do a survey, the first response should be, "to what purpose?" and the second, "with what effect?" The reasons for doing a sur- vey are many and varied. In his study of college and university library surveys Ernst Walfred Erickson identified eight reasons why library surveys are under- taken.5 1. To study a library or libraries in depth and with care. 2. To develop long-range planning for a library or group of libraries. 3. To get a perspective of a library in re- lation to the history of the state, region, and university or community. 4. To permit the library to discover ways to improve its administration, orga- nization, and services. 5. To discover ways by which library re- sources can be effectively related and inte- grated with other libraries in the state, re- gion, or the nation. 6. To note any limitations that might prevent the effective operation of the li- brary. 7. To contribute to an increased under- standing of the role of the library, its needs, and its problems. 8. To determine how effective the li- brary is in carrying out its stated mission, goals, and objectives. This list is not exhaustive by any means, but it reflects where the most energy has been focused in the past. Most writers agree that the purpose of the library sur- vey is the systematic collection of data that will shed light on problems and lead to the November 1990 improvement of library services. A good survey also permits analysis that results in a sound interpretation of the data. TYPES OF SURVEYS Broadly speaking, there are two types of surveys, and they serve somewhat differ- ent purposes. The Descriptive Survey. This type of sur- vey can be an aid to library administrators in planning, improving public relations, and even marketing. Most descriptive sur- veys describe one library or a group of li- braries within a system or even a state. Such surveys deal with quantitative data and are used to compare with similar sta- tistics from the previous year, or other li- braries. They may also seek opinions and/ or demographic data about users. These data may be compared with quantitative data about the library by using various kinds of data analyses. The descriptive survey has its place as an aid in helping in- dividual libraries in the decision-making process, or in generating data where none has existed before, but it is limited in scope. A more exacting type of survey is that used in survey research. Survey Research. This type of research may be defined ' as an inquiry that in- volves the systematic collection and sta- tistical analysis of data and that which is based on well thought out hypotheses. Even a descriptive survey needs to be based on sound goals and objectives. A survey that has research as its purpose differs from the descriptive survey in sev- eral ways. First, survey research is global rather than local; that is, the study of one library's users' attitudes is not survey re- search. The study of a group of libraries' users' attitudes may be, depending upon whether the survey meets the remaining criteria. Second, survey research is concerned with theory building. Fred N. Kerlinger defined theory as ''a set of interrelated constructs (concepts), definitions, and propositions that present a systematic view of phenomena by specifying rela- tions among the variables with the pur- pose of explaining and predicting the phe- nomena. " 6 Theory is based on well- established assumptions in the field. For example, studies related to communica- tion skill as an important element in the successful handling of the reference inter- view are based on a well-established body of theory and research in the field of refer- ence librarianship. Theories may or may not be correct, but they form the basis for the development of hypotheses that are tested in many types of research. One of the weaknesses in the field of librarian- ship, like other applied fields, is a lack of a large body of theory to provide the foun- dations for research. Third, survey research is concerned with finding out the truth about a given hypothesis. A hypothesis is a statement that expresses relationships or differences among the phenomena being studied. Kerlinger, citing the American philoso- pher Charles Pierce, described four ways of knowing: tenacity, authority, intuition, and scientific method. 7 Tenacity has to do with the beliefs that people hold onto even when faced with clearly contradictory facts. For example, the belief that all of the public may be reached by libraries if only the right formula can be found is a case in point. Authority is a belief established by the weight of tradition, supported by pub- lic sanction. The belief that university li- braries are important to the prestige of the university community they serve is essen- tially based on the weight of support for this idea. Intuition, or a priori, method rests on the rational approach. University librarians and many other educators would agree that it is reasonable to believe that students who have access to a well- staffed and well-stocked library will get a better education than those who do not. The fourth method of knowing has to do with research and the scientific method. This latter is usually considered more de- pendable in that it appeals to evidence and, as appropriate, meets standards of accountability. To summarize then, the descriptive sur- vey is primarily concerned with the exis- tence of certain phenomena; survey re- search is concerned with the relationships among phenomena. Survey research, however, is not experimental research .. Survey research can be distinguished from experimental or scientific research Research Notes 555 mainly by the fact that in a survey the in- vestigator attempts to measure what would have happened even if the study had not taken place, rather than measur- ing the effects of stimuli that have been de- liberately introduced into the experimen- tal climate. 8 THE SURVEY AS AN EFFECTIVE RESEARCH TOOL Survey research falls somewhere be- tween the scientific and the naturalistic methods. Survey research meets several, but not all of the criteria for scientific re- search. Survey research is based on sound theory; it is systematic and controlled; it uses the language of variables and is con- cerned with relationships between or among them. It seeks answers to posed hypotheses; and it makes use of statistical analyses beyond tabulations. The one cri- terion that survey research does not meet is that of establishing causal relationships. Survey research is like naturalistic study in that it is concerned with field study; it may, through a process of data dredging, search for the truth by uncovering layer upon layer of statistical data, and it does not assume that all of the variables are tightly controlled. It draws inferences from existing variance in samples or popu- lations by a rigorous process of compari- son as opposed to establishing definitive causal relationships as is the case with sci- entific study. 9 Although we find many published stud- ies using the survey technique in the li- brary literature, not a large number ap- pear in Dissertation Abstracts. (Only 1,284 were located up to December 1990.) Some of the reasons may have to do with the prohibitive cost for a student conducting a survey, but another reason may be a mis- trust of the survey method as a reliable re- search tool. Catherine Marsh defended the use of survey method in the investiga- tion of sociological phenomenon. Marsh suggests that the move away from the sur- vey as a research method may be due in part to a II misguided or even malevolent desire to turn [librarianship] into some kind Of technocratiC Science. " 10 Where the study of libraries and their services is in- volved, librarians are for the most part 556 College & Research Libraries dealing with existing sociological phe- nomena. Criticisms of poorly done studies should not damn surveys as a whole; nor should the descriptive type of survey be confused with what is more correctly identified as survey research. IMPROVING THE QUALITY OF SURVEY RESEARCH Librarians have seen that the survey can be a viable tool for research in the field. Understanding the use of the survey in re- search, which has been discussed above, is highly important. A number of common mistakes are made in developing surveys. Four areas are addressed here that will help researchers to improve their survey techniques. They include adequate plan- ning, sample size, data collection, and sta- tistical analysis. PLANNING The researcher must recognize that costs are involved. Doing an adequate sur- vey also takes time. One year should be al- lowed from the inception of the idea to the reproduction of the final report, if it is a funded study, or to the final draft of a pub- lishable article if it is a major study. This timetable assumes that the researcher is involved in other work as well. Time can be shortened considerably by using an outside agency to carry out such an inves- tigation. However, a librarian doing sur- vey research will not be caught short by al- lowing plenty of time to complete the work. If the study can be accomplished in a shorter period of time, no harm is done. Surveys have their own timetables that are determined by the vagaries of data col- lection. Other decisions have to be made in ad- dition to determining costs and time in- volved. Fundamental to any research study is a thorough literature search. With the proliferation of electronic access to bib- liographical databases, there is no excuse for undertaking a study that has already been done. A search of the literature will provide the necessary theoretical back- ground needed for the development of a well thought out hypotheses. Consider- able additional personnel will be needed to carry out the study and to collect the November 1990 data. In most library surveys, the ques- tionnaire is used, but alternative methods may be more suitable. SAMPLE SIZE ''How large should the sample be?'' is a question commonly asked. To some ex- tent the answer is the larger the better; however, reason must prevail. In the most general of terms, the more homogeneous the sample, the less likelihood of sampling error using a smaller sample; the more heterogeneous, the larger a sample should be. In some cases, the entire popu- lation may be surveyed, such as a study of state librarians. In this instance the popu- lation size is smaller than a sample nor- mally would be for survey research. A general rule of thumb for a survey sample size is around 200 with an over 50 percent return required for reliability, and 60 per- cent or above highly desirable. A re- searcher should wonder about the useful- ness of reporting survey results with a 30-40 percent return rate. The size of the sample for a survey can be calculated mathematically, including the use of standardized tables to avoid sampling error. The factors that are neces- sary for using these formulae are, accord- ing to Dennis E. Hinkle and Dale Oliver, as follows: 1. significance level (a) 2. degree of accuracy (effect size) 3. variance in population error {j 2) 11 This procedure is considered appropriate for dichotomous variables as well as those measured by interval and Likert-type scales. The formula for the one-sample case is: n= (significance level t-value)2 (variance) (accuracy level)2 This formula can be used for a two-sample case by multiplying by two. For further ~x­ ploration of this topic and an interesting discussion of sample size plus the sup- porting tables for the formula, the reader should see the Hinkle and Oliver two-part article entitled, ''How Large Should the Sample Be?" A number of other texts on sampling are available as well. DATA COLLECTION Many library surveys are conducted us- ing mailed or hand-distributed question- naires. With the number of surveys on the rise, people are more reluctant to answer them, including our colleagues who may receive seven to ten a week- or more. A researcher only has to look at the prob- lems the U.S. Census Bureau has encoun- tered to realize that collecting survey data from the public using a questionnaire is a difficult task. Many factors can help to in- crease returns. When using a mailed ques- tionnaire, the cover letter needs to be short and to the point. It should include the significance of the study, and the rea- son the respondent's answers are impor- tant. In addition, giving the individual an idea of the amount of time it takes to fill it out and assurances of anonymity are equally important. There is no guarantee that a short questionnaire will be received any more favorably than a long one. Gail F. Munger and Brenda H. Loyd reported on a number of studies in this area and noted that the results were conflicting. u There are problems with long question- naires. A researcher can expect to find more unanswered questions with a long questionnaire because respondents may become bored or tired of answering. One way to deal with this problem is through multiple matrix sampling. This process in- volves administering the longer question- naire to a randomly selected number of re- spondent~ and a randomly selected number of subsets to the remaining groups in the sample. Two studies re- ported success using thb type of sampling procedure in survey research. Munger and Loyd reported more success in gath- ering data using a two-page, twenty- seven item survey than with a five-page, sixty-one item survey. 13In spite of re- ported success in their own and other studies using matrix sampling, Hallie Pre- skill and Tim L. Wentling concluded that more research was needed on this type of sampling approach to surveys. 14 Im- proving survey response can be handled in a number of other ways. Sometimes a monetary or other gift incentive may be ef- fective, but it should be pre-tested to make sure it is appropriate. A second mailing can improve the rate of return, but it has not been this writer's experience that this Research Notes 557 method increases the response rate among professionals to any great degree- 10 to 15 percent is about the maximum. There has also been some indication in re- cent surveys carried out by the writer that people prefer a checklist as opposed to writing out answers. Joseph M .. Viladas claimed that ''repeated follow-up by mail, telegram, telephone, or combinations of these has brought the response up to the 85 to 100 ~ercent range on a number of oc- casions." 5 Others have suggested hand- addressed mailed surveys, but that seems a luxury that few researchers could afford with a large sample or population under study. BarbarA. Zusman and Paul Duby found that the use of monetary incentives could improve the return rate of questionnaires over a second mailing. 16 A single mailing averages about 65 cents per address, in- cluding copying, envelopes, mailing la- bels, and postage. (Postage could make it more if it is a long questionnaire.) A mone- tary enclosure of a dollar is sufficient. This would add $200 to the overall cost of the first mailing of 200 questionnaires. This amount would be reduced by the need to do a second mailing by $65 with a 50 per- cent return. Because many surveys only receive a 30-40 percent return on the first mailing, this could reduce costs to an even greater extent-and save time. Bookmarks could be considered for library surveys as an alternative to cash that has its risks, but weight would need to be taken into con- sideration. The increased costs of single mailings with incentives over repeat mail- ings are not great, and the results could be well worth it. Where data involving opinions are con- cerned, librarians should consider two un- derused methods: face-to-face interviews and the telephone. Interviewers need to be very carefully trained and spot moni- tored, however. One study done in France found that interviewers," ... ignored su- pervisors' instructions and methodolo- gists' dogmas. They didn't act like scien- tists are supposed to act: unbiased and neutral. " 17 STATISTICS The most common form of statistical 558 College & Research Libraries analysis for surveys is frequencies or tabu- lation. When a study is limited to report- ing frequencies, however, it does not tell us a great deal that is useful for survey re- search. A more meaningful statistic is cross-tabs with chi-square. Chi-square is a nonparametric test. Nonparametric tests are generally used with nominal or ordinal scaling or where there is uncertainty about the population or sample. Chi-square may be viewed as a discrepancy statistic. It tells us about significant associations among the variables. For example, it can tell us whether there is a significant difference in the ways female managers answered questions about delegation of authority than males. Chi-square has its own weak- nesses. Two assumptions underlie the test of chi-square: that none of the cells have an expected value of less than one, and that no more than 20 percent of the cells have a value smaller than five. However, this does not mean that the data cannot be used. With today' s computer programs, such as SPSS-x, the number of cells can be reduced by combining row and/or column categories. Hand calculations are no longer necessary. Another useful statistical analysis for some types of surveys is regression analy- sis. Regression permits us to make predic- tions based upon the data. With the aid of a computer program, development in ap- plied statistics makes regression analysis far less cumbersome than in the past. David L. Passmore and Dominic A. Moha- med recommend the use of logistic regres- sion in survey research to avoid trudging through thick summaries of survey results reported in numerous tables. 18 While chi- square permits two-way cross tabulations, regression can show complex relation- ships among variables. An example might be that the graduates from a given library science program are three times as likely to find a job as a librarian as those from an- other accredited program. Another way to reduce the amount of data handled in a survey is to make use of factor analysis. This statistical analysis is one of the oldest and best-known data re- duction techniques that can be used to re- duce a set of variables to a smaller num- November 1990 ber. For a more thorough discussion of factor analysis in relation to survey data, see the article by Linda L. Phillips and Wil- liam Lyons in the previous issue of College & Research Libraries. At least three other statistical tests can be used in survey research. These include cojoint analysis, correlations, and analysis of variance. Cojoint analysis is similar to factor analysis. It is currently a popular technique in marketing research. It is es- pecially useful in attitude studies. Cojoint analysis looks at attributes and benefits of a product that could be adapted to patron attitudes towards the library, how patrons benefit from the product, and the impor- tance of each attribute. Through analysis, the relative importance of each attribute can be measured. Correlation analysis can be used in sur- vey research, but it is a rather weak mea- sure compared to regression, factor, and cojoint analyses. Correlation characterizes the existence of a relationship between variables. Analysis of variance is used when the research hypothesis incorpo- rates two or more population means. Uni- versity librarians have a distinct advan- tage in undertaking studies involving survey research as support groups on the campus can aid in determining statistical methodology. What the researcher needs is an understanding of elementary arithmetic and a fundamental statistics · course. SUMMARY Survey research is a significant way of generating knowledge about libraries, their users or nonusers, library education, and librarians. In designing a survey, care must be taken to understand the differ- ences between the descriptive survey and · survey research. Some ways to improve survey research involve adequate plan- ning that accurately assesses time and costs; determining the adequacy of the size of the survey; and various approaches to data collection, including matrix sam- pling. Various statistical analyses can be used with survey research-some strong- er than others. Selecting the right analysis for a survey should be carried out with the aid of a statistician unless the researcher is well versed in statistical processes. The meaning of the results of a survey will be clarified if a plan for analysis is deter- mined before data collection begins. Research Notes 559 Then, interpretations will more accurately reflect patterns in the data and provide a foundation for better understanding of the library or individuals being surveyed. REFERENCES AND NOTES 1. American Library Association, A Suroey in the United States (Chicago: American Library Assn., 1926-27). 2. Maurice B. Line, Library Suroey, 2d ed., rev. by Sue Stone, 1982 (London: Clive Bingley, 1967). 3. Robert H. Donahugh, "Questioning the Questionnaires," American Libraries 19:402-03 (May 1988). 4. Robert M. Groves, "Research on Survey Data Quality," Public Opinion Quarterly 51:156-62 (Sum- mer 1987). 5. Ernst Walfred Erickson, "College and University Library Surveys, 1938-1952" Ph.D. dissertation, University of illinois, 1958. 6. Fred N. Kerlinger, Foundations of Behavioral Research (New York: Holt, 1973), p.5. 7. Ibid. 8. Herbert Goldhor, An Introduction to Scientific Research in Librarianship (Urbana, ill.: Univ. of illinois, 1972), p.116. ' 9. Egon B. Guba and Yvonna S. Lincoln, Effective Evaluation (San Francisco: Jossey-Bass, 1981), p.61. 10. Catherine Marsh, The Suroey Method: The Contribution ofSuroeys to Sociological Explanation (London: Geroge Allen & Unwin, 1982), p.3. 11. Dennis E. Hinkle and J. Dale Oliver, "How Large Should the Sample Be? A Question With No Simple Answer? Or ... ," Educational and Psychological Measurement 43:1051-60 (Winter 1983); ''How Large Should the Sample Be? Part II-The One-Sample Case for Survey Research,'' Educa- tional and Psychological Measurement 45:271-80 (Summer 1985). 12. Gail F. Munger and Brenda H. Loyd, "The Use of Multiple Matrix Sampling for Survey Re- search," The Journal of Experimental Education 56:187-91 (Summer 1988). 13. Ibid. 14. Hallie Preskill and Tim L. Wentling, "The Application of Matrix Sampling to Paired Comparisons in Survey Research" Journal of Vocational Education Research 9:31-37 (Summer 1984). 15. Joseph M. Viladas, The Book of Suroey Techniques (Greenwich, Conn.: Havemeyer Books, 1982), p.216. 16. Barbara J. Zusman and Paul Duby, "An Evaluation of the Use of Monetary Incentives in Post Sec- ondary Survey Research," Journal of Research and Development in Education 20:73-81 (Summer 1987). 17. Jean Peneff, "The Observers Observed: French Survey Researchers at Work," Social Problems 35:520-35 (Dec. 1988). 18. David L. Passmore and Dominic A. Mohamed, "Application of Logistic Regression Techniques in Survey Research," Journal of Vocational Education Research 9:1-9 (Winter 1984). ANTHROPOLOGYmOLKLORE Power in Performance The Creation of Textual Authority in W eyewa Ritual Speech Joel C. Kuipers Noo. 1990. 230 pp, 18 iUus. Cloth, 8245 .. 0, $24.95 Learning Capitalist Culture Deep in the Heart of Tejas Douglas E. Foley Sept. 1990. 268 pp. Cloth, 8246 .. 9, $31.95; paper, 1314 .. 9, $14.95 Forest Society A Social History of Peten, Guatemala Norman B. Schwartz Dec. 1990. 320 pp, 21 iUus. Cloth, 8248-5, $30.95; paper, 1316 .. 5, $13.95 WOMEN'S STUDIES Women at War The Story of Fifty Military Nurses Who Sen~ed in Vietnam Elizabeth Norman Sept. 1990. 222 pp, 28 iUus. Cloth, 8249 .. 3, $36.95; paper, 1317 .. 3, $14.95 LAW Social Courts in Theory and Practice Yugoslat1 Workers' Courts in Comparatit~e Perspectit~e Robert M. Hayden Jan. 1991. 224 pp. Cloth, 8259-0, $28.95 LITERATURE The Language of Fiction in a World of Pain Reading Politics as Paradox Barbara J. Eckstein Dec. 1990. 220 pp. Cloth, 8254 .. X, $29.95; paper, 1321 .. 1, $13.95 Designs of Darkness in Contemporary American Fiction Arthur M. Saltzman Noo. 1990. 168pp. Cloth, 3051-5,$21.95 A Sociosemiotic Theory of Theatre Jean Alter Jan. 1991. 320 pp. Cloth, 3054 .. X, $32.95 (tent.) Diana's Hunt Caccia ·di Diana Boccaccio' s First Fiction Edited & translated by Anthony K. Cassell & Victoria Kirkham Feb. 1991. 288 pp, 9 iUus. Cloth, 8219 .. 1, $29.95 SCIENCE Molecular Approaches to Supracellular Phenomena Edited by Stephen Roth Oct. 1990. 242 pp, 60 iUus. Cloth, 8251-5, $3;~ns lvAni4 r ~University of Pennsylvania Press Orders taken at: P .0. Box 4836, Hampden Station, Baltimore, MD, 21211 Toll .. free: (800) 445 .. 9880