College and Research Libraries Content Analysis of A~RL Conference Papers Pamela Snelson and S. Anita Talar This study examines the content of papers presented at the second, third, and fourth national conferences of the Association of College and Research Libraries. An analysis of the papers presented at the first national conference revealed that one-third of them were research reports. Given that ACRL has initiated a series of activities designed to encourage the research pursuits of its members in the time period under study, it is our hypothesis that the second, third, and fourth conference papers will contain more research reports than were present in the first set of papers. Using the first study as a baseline against which future conferences can be compared, we examined papers presented at subsequent conferences using the technique developed by Pauline Atherton for analyzing research methods· in the published literature of information science. onference papers are impor- tant to a discipline-they are an index of critical issues, a barometer of a profession's maturity. Most professional organiza- tions have a long history of annual or biennial conferences, some going back more than 100 years. 1 J. C. Row ley suc- cinctly lists many functions of confer- ence proceedings: identification of contributions and activities of individu- als in the field, realization of areas of new research and development, and aware- ness of current problems and possible solutions.2 Relatively new to the aca- demic library profession, the national conference has produced a forum for the presentation of research and conference papers. The Association of College and Re- search Libraries (ACRL) has had five na- tional conferences. An analysis for research content of the papers presented at the first conference (Boston, 1978) re- vealed that fewer than a third of the papers presented could actually be termed re- search.3 That study investigated whether the papers presented at that first confer- ence followed the norms established for scientific or scholarly papers in other disciplines; this study will build on that framework. Four subsequent ACRL na- tional conferences (Minneapolis, 1981; Seattle, 1984; Baltimore, 1986; and Cin- cinnati, 1989) have added significantly to the growing body of conference liter- ature. That scholars will create more re- views of research than actual research is always a danger in a profession. If ACRL conference literature is to make contri- butions similar to those of scholarly con- ference literature in other disciplines, the preponderance of papers at its con- ferences should be reports of original research. This article examines the pa- pers presented at the second, third, and Pamela Snelson is Assistant Director at Drew University Library, Drew University, Madison, New Jersey 07940, and Sister Anita Talar is Reference Librarian and Associate Professor at McLaughlin Library, Seton Hall University, South Orange, New Jersey 07079-2690. 466 fourth ACRL national conferences to de- tennine their research content. LITERATURE REVIEW W. D. Garvey, N. Lin, and C. E. Nelson conducted extensive studies on commu- nication activities in the physical and social sciences and found that most con- ference paper presenters had already re- ported on their work through some other vehicle prior to the meeting.4 While phys- ical scientists seem to disseminate infor- mation most intensively in the shortest period of time, social scientists dissemi- nate more diffusely and over a longer pe- riod of time. In addition, social scientists show greater activity than physical scien- tists, both in contacting authors and pre- senters at the meeting and in later correspondence. Distribution of presenta- tion copies (conference proceedings), par- ticularly among the social sciences, is a major dissemination activity associated with national meetings. The national meeting was an occasion on which social scientists devoted considerable time and energy to interacting with colleagues and to establishing lasting informal con- tacts with conference paper presenters. B. C. Griffith and W. D. Garvey studied the American Psychological Association's (APA) annual convention as an outlet of scientific work in psychology.5 As the largest of the behavioral sciences, psy- chology has one of the most highly at- tended and oldest national meetings. APA conference attenders considered the short speeches, symposia, and papers pre- sented at these annual meetings most im- portant in the dissemination of individual research studies, trends in research, and reports on research programs. The publica- tion of a portion of the contributed papers by APA more than one month before the convention assists the information ex- change. The declared purpose of the Pro- ceedings is "to publish, with minimum delay, studies that would normally be submitted to journals and to offer a better means of disseminating brief research re- ports .. . [leading] to more critical discussion of work within the sessions."6 A review of the literature on research in library science has revealed a varied ACRL Conference Papers 467 and scattered approach. M. H. Harris discussed the problems with research in library science, as did B. C. Peritz, C. C. Williamson, and J. H. Shera.7 Peritz noted an increased output of research papers, indicating real research activity between 1960 and 1975. M. M. Nour's further study of research articles sug- gested that while the number of research articles is increasing, the proportion of research articles in core journals has de- clined since 1975.8 Nour concluded that this difference may reflect changes in editorial policies; increased emphasis on practitioners' publishing, which is less likely to be research; the inclusion of dif- ferent journals in the study; or a stricter definition of research. Peritz, too, found that "factual information on research publications in librarianship is scarce."9 In an issue of Library Trends devoted to this subject, Rose Mary Magrill reiterated that while "research in librarianship has in- creased over the past 20 years ... there is evidence that librarians hip has not yet established effective quality-control procedures." 10 The ACRL conference is not the vehicle of choice for research dissemination. ACRL has initiated a series of activities designed to encourage research within the academic library community. In 1971, Col- lege & Research Libraries started a "Research Notes" column to provide short reports of current research. More recent examples in- clude the "Research Forums" column, begun in mid-1984, in College & Research Libraries News, the research clinic held in July 1985 at the 104th Annual Confer- ence of the American Library Associa- tion, and the newly formed ACRL Committee on Research Development. A final imprimatur on the renewed thrust toward research can be taken from ACRL's strategic plan, which promises to "promote study, research and publica- tion relevant to academic and research librarianship." 11 ACRL members also have the opportunity to participate in 468 College & Research Libraries the activities of ALA's Library Research Round Table. Reviewers of ACRL' s published con- ference proceedings are generally posi- tive in their comments. H. Edelman stated that the papers of the first confer- ence are "a very efficient and useful way of presenting a state of the art ... and a tribute to the depth and quality of aca- demic librarianship in the United States." 12 N. Stevens found the papers of the second conference interesting, "a solid contribution to the literature." 13 Speaking of the third conference, S. Rothstein stated that it was "important as a massive and multifaceted assem- blage of findings, views and proposals regarding college and university librar- ies in North America." 14 Larry Hardesty wrote that qualified progress was made at the third national conference and cited an analysis of this conference prepared by Sharon Rogers and Robert Pesek. 15 Rogers and Pesek noted an improve- ment in the format of the third confer- ence over that of the first and second conferences. The third conference showed an in- creased sophistication evidenced by use of the format "author as analyst" rather than "author as biographer." The latter format describes a system, while in the former the author discusses issues in their complexity, usually involving some primary research, such as a survey instrument, and including a discussion of the literature and the background of the problem. Reviews of the fourth con- ference proceedings were more critical of the conference's lack of research. Wil- liam Miller wrote, "These papers are not of great theoretical import," while Hardesty observed that many papers "de- scribe[d] solutions to particular problems not generalizable beyond a specific li- brary" and asked the question, "Are we going to promote research through the conferences or not?" 16 HYPOTHESES Given that ACRL initiated a series of activities designed to encourage the re- search pursuits of its members in the time period under study, it was our hy- September 1991 pothesis that the second, third, and fourth conference papers contained more research reports than did the first set of conference papers. The null hy- pothesis was that there was no signifi- cant increase in the number of research papers. Our methodology was content analysis of each paper presented at each conference to determine whether or not it was a research report. Apparently, prospective contributors, if given a choice, do not choose to present research. A second hypothesis, given the short history of ACRL conferences, was that the research papers presented differed from scientific papers in adherence to the norms of research papers. These norms include the following standard parts: prob- lem statement, literature review, hypotheses, research methodology, findings, and conclu- sions.17 Methodology for testing this hy- pothesis was content analysis of the conference papers that we categorized as research. METHODOLOGY The first study of ACRL conference papers provided a baseline against which all future conferences can be com- pared. Papers presented at the second, third, and fourth conferences were exam- ined using the same technique Coughlin and Snelson employed to study papers presented at the first conference. Cough- lin and Snelson's definition of catego- ries supplemented a content analysis methodology developed by Pauline Atherton for analyzing research methods in the published literature of informa- tion science. 18 These categories provided the criteria we used to make decisions as to whether an article was a research re- port or a nonresearch report and exam- ines each research paper for selected characteristics, such as the overall goal of the article, the measure on which find- ings were based, the nature of the vari- ables, questions on data collection and ACRL Conference Papers 469 TABLEt RESEARCH CONTENT OF CONFERENCE PAPERS First Conference Second Conference Third Conference Fourth Conference % % Research 30.33 26.79 Nonresearch 69.67 73.21 (N=66) (N=56) analysis, and the objectives and weaknesses of the study. Reports were prepared for each of the 181 papers presented at the three ACRL conferences under discussion. In order to test coder reliability, a small sample of papers was used in a test run and each researcher coded this group. A reliability coefficient of .90 was achieved.19 FINDINGS Analyses of the 181 papers presented at the second, third, and fourth con- ferences indicated that less research appeared in these conference papers than in those of the first conference. While 33.3% of the papers at the first ACRL conference had a research focus, the research content in subsequent con- ferences never achieved 30%, decreasing to a low of 18.3% in papers presented at the fourth ACRL conference (see table 1 ). For the obtained x2 of 5.23 with 3 df, p > .10. We concluded that the null hypoth- esis could not be rejected; there was no significant increase in the number of re- search papers. Therefore, our first hy- pothesis-that the research content of papers presented at ACRL conferences was increasing-was not supported. On the contrary, when the actual number of research papers was examined, we found that the research base was declining. Ex- pert opinion, as evidenced by reviews of the published conference proceedings, lends support to this discovery. The fourth conference contained the lowest percentage of research reported. This particular conference's call for pa- pers clearly gave prospective contribu- tors the opportunity to choose among authoring research reports, position pa- pers, or idea briefs. Apparently, prospec- tive contributors, if given a choice, do not choose to present research. % % 27.70 18.33 72.30 81.67 (N=65) (N=60) The overall goal of most research pa- pers in the second, third, and fourth con- ferences was either evaluative-that is, a comparison or an assessment of issues- or exploratory-that is, a speculative or reflective consideration of issues (see table 2). This contrasts with findings from the first conference, where research paper goals were spread among thecate- gories, with the heaviest concentration having model building as the major goal. Although most researchers paid attention to sampling, they rarely pretested the n?- search instrument. Objectives were often clearly stated and achieved; in many re- search papers, a literature review gener- ated objectives. It was not until the fourth conference that the majority of research papers proposed hypotheses (see table 3). Only in the third conference papers were findings from a majority of research papers integrated into the cur- rent state of the art. The overall research approach used most often in the research papers pre- sented at the second and third conferences was survey research. Data collection for a majority of the research papers presented at these two conferences was done through questionnaires (53.33 percent and 77.78 percent, respectively). In con- trast, the research papers from the fourth conference used a variety of data collec- tion methods-questionnaire, discus- sion, interview, and observation. No method predominated. The overall re- search approach also varied at the fourth conference. In addition to survey re- search, paper presenters used content analysis, regression, and focused inter- views in their research. No one-to-one correlation between any one question on our research report form and the accepted parts of a research paper exists. For example, we did not ask of each 470 College & Research Libraries September 1991 TABLE2 MAJOR GOAL OF RESEARCH PAPERS First Conference Second Conference Third Conference Fourth Conference Evaluative Prescriptive Exploratory Developmental Descriptive Model-building Predictive % 25.00 5.00 10.00 10.00 15.00 30.00 5.00 % 46.67 0.00 33.33 0.00 13.33 6.67 0.00 % % 61.11 45.45 0.00 9.09 22.22 36.36 0.00 0.00 5.56 0.00 0.00 0.00 11.11 9.09 (N=20) (N=15) (N=18) (N=11) TABLE3 RESEARCH PAPER CHARACTERISTICS First Conference Second Conference Third Conference Fourth Conference Careful sampling Instrument pretested Hypothesis stated Objectives clearly stated Objectives achieved Objectives from state of the art % 5.00 5.00 35.00 95.00 80.00 55.00 % 60.00 13.33 6.67 100.00 66.67 46.67 % % 37.50 44.44 16.67 9.09 38.89 63.64 94.44 81.82 83.33 81.82 55.56 54.55 (N=20) (N=15) (N=18) (N=11) research paper, "Is there a problem state- ment?" However, a researcher can look at a conference's totalled response to cer- tain questions on the report form and make observations about adherence to the norm of scientific papers at a given conference. If we consider a positive re- sponse to questions about sampling, instru- ment pretesting, statement of hypothesis, and clearly stated and achieved objectives an indication of a paper's possession of research characteristics, it appears that the research papers presented at the ACRL national conferences do not follow the systematic processes of the scientific method. A majority of the research pa- pers scored negatively in more than half of the questions dealing with standard- ized parts of a research paper, thereby supporting our second hypothesis that the ACRL conference papers differ from the established norm of scientific papers. This is the case in all three sets of confer- ence papers we examined. While papers presented at the fourth conference resem- bled scientific papers no more than those presented at earlier conferences, the meth- ods employed in the research were more sophisticated, moving beyond the ques- tionnaire and survey. In summary, our findings indicate that the research content of the conference papers declined and that few of those papers that can be considered research met the norms of scientific study. CONCLUSIONS Initial publicity on the fourth national conference invited librarians to "listen as your colleagues present position andre- search papers on a range of topics which concern the academic librarian." 20 Clearly, the intent is there. Yet analysis of the 1986 conference papers showed that less ac- tual research was presented than at other conferences, despite an increasing em- phasis or thrust by ACRL to "promote study, research and publication relevant to academic and research librarianship."21 The scientific qualities of research in librarianship have often been called into question. Through the years, various stud- ies of research methods used by library researchers have shown that much of the published research centers around disser- tations. The current increased demand for publication by academic librarians necessi- tates a sound research base for this activity. In the first issue of Library Quarterly, Williamson decried the lack of research in the library profession and wondered about the justification for using the term "library science": If the library is to rise to its opportu- nity as a social and educational force it must ... begin very soon to attack its problems by a thoroughgoing applica- tion of the spirit and methods of re- search that are being found to be effective in every other field . In the natural sciences as well as the human- istic and social sciences, in the applied sciences, in education, in business and industry, in social service-everywhere except in the library field-extensive pro- grams of research are being carried out, highly organized and well financed.22 More than fifty years later, conference proceedings provide little encouraging evidence that research has increased, de- spite the fact that academic librarians now have in proceedings an appropriate and viable tool for the dissemination and publication of research. To compare ACRL ACRL Conference Papers 471 conference proceedings with those of other disciplines is difficult. First, ACRL does not yet have a conference history corresponding to that of long-estab- lished organizations such as the Ameri- can Psychological Association or the American Chemical Society. Second, re- search on conference proceedings in the various disciplines appears to assume a research content for conference papers and measures other variables, such as percentage of association members pre- senting or number of papers contributed per academic institution.23.24 Conference papers should be a vehicle for the dissemination of current research in any scholarly field. Our research showed that the ACRL conference is not the vehicle of choice for research dissem- ination. At least two explanations are possible: academic librarians are not conducting research, or their research is reported outside ACRL conferences. If it is the former, ACRL must increase its conscious efforts to encourage and foster research by academic librarians (e.g., through conferences, workshops, its strategic plan). If it is the latter, which we believe to be the case, efforts must be made to draw academic researchers into that unique channel for the transmission of information-the conference presen- tation. The three-year cycle of ACRL con- ferences is not conducive to timely dissemination of research. If annual ACRL conferences are not feasible at this point, ALA biannual meetings might provide be- tween-conference opportunities for re- search presentation. Additionally, the structure of ACRL conferences should fa- cilitate the communication of research by providing opportunities for informal dis- cussion of research projects, in addition to the formal presentations. REFERENCES 1. Caroline Coughlin and Pamela Snelson, "Searching for Research in the ACRL Confer- ence Papers," Journal of Academic Librarianship 9:21-25 (March 1983). 2. John C. Rowley, "The Conference Literature: Savory or Acrid?" in Conference Literature, Its Role in the Distribution of Information: Proceedings of the Workshop on Conference Literature in Science and Technology May 1-3, 1980, ed., Gloria J. Zamora (Marlton, N.J.: Learned Information, 1981), p.ll-20. 3. Coughlin and Snelson, "Searching for Research," p.21-25. 472 College & Research Libraries September 1991 4. William D. Garvey, Nan Lin, and Camot E. Nelson, "Some Comparisons of Cpmmu- nication Activities in the Physical and Social Sciences," in Communication among Scien- tists and Engineers, eds., Camot E. Nelson and D. K. Pollock (Lexington, Mass.: Heath, 1970), p.61-84. 5. Belver C. Griffith and William D. Garvey, "The National Scientific Meeting in Psychol- ogy as a Changing Social System," American Behavioral Scientist 9:3-8 (Feb. 1966). 6. Ibid., p.8. 7. Michael H. Harris, "The Dialectic of Defeat: Antimonies in Research in Library and · Information Science," Library Trends 34:515-31 (Winter 1986); Bluma C. Peritz, "The Methods of Library Science Research," Library Research 2:251-68 (Fall1980); Charles C. Williamson, "The Place of Research in Library Science," The Library Quarterly 1:1-17 (Jan. 1931); Jesse H. Shera, The Foundations of Education for Librarianship (New York: Becker and Hayes, 1972). 8. Martyvonne M. Nour, "A Quantitative Analysis of the Research Articles Published in Core Library Journals of 1980," Library & Information Science Research 7:261-71 (July /Sept. 1985). 9. Peritz, "The Methods of Library Science Research," p.251. 10. Rose Mary Magrill, ''Publishing Research in Librarianship," Library Trends 32:557-77 (Spring 1984). 11. ACRL Strategic Planning Task Force, "ACRL' s Strategic Plan," College & Research Libraries News 48:21-25 (Jan. 1987). 12. Hendrik Edelman, "New Horizons for Academic Libraries," book review, Library Journal105:1143 (May 15, 1980). 13. Norman Stevens, "Options for the '80s: Proceedings of the Second National Conference of the Association of College and Research Libraries," book review, Wilson Library Bulletin 57:519 (Feb. 1983). 14. Samuel Rothstein," Academic Libraries: Myths and Realities: Proceedings of the Third National Conference of the Association of College and Research Libraries," book review, Canadian Library Journal12:230-31 (Aug. 1985). 15. Larry Hardesty, "Academic Libraries: Myths and Realities; Proceedings of the Third National Conference of the Association of College and Research Libraries," book review, College & Research Libraries 46:441-43 (Sept. 1985). 16. William Miller, "Energies for Transition: Proceedings of the Fourth National Confer- ence of the Association of College and Research Libraries," book review, Journal of Academic Librarianship 12:235 (Sept. 1986); Larry Hardesty, "Energies for Transition: Proceedings of the Fourth National Conference of the Association of College and Research Libraries," book review, College & Research Libraries 47:517-18 (Sept. 1986). 17. Jan C. Robbins, "Social Functions of Scientific Communications," IEEE Transactions on Professional Communication PC-16 :132 (Sept. 1973). 18. Pauline Atherton, "Research in Information Science: An Assessment," in Perspectives in Information Science: Proceedings of the NATO Advanced Study Institute on Perspectives in Information Science, 1973, eds., Anthony Debons and William J. Cameron (Leyden, Netherlands: Nordhoff, 1975), p.665-83. 19. Ole R. Holsti, Content Analysis for the Social Sciences and Humanities (Reading, Mass.: Addison-Wesley, 1969). 20. H. Joanne Harrar, "Invitation to the Baltimore Conference," College & Research Libraries News 47:21 (Jan. 1986). 21. ACRL Strategic Planning Task Force, "ACRL's Strategic Plan," p.22. 22. Williamson, "The Place of Research in Library Science," p.3. 23. Griffith and Garvey, "The National Scientific Meeting." 24. Glenn H. Petry, "A History and Analysis of Scholarly Papers Presented at the Seven Academic Finance Associations from 1939 through 1980," Financial Management 19:93- 104 (1981).