crawford 224 College & Research Libraries May 1999 The Research Literature of Academic Librarianship: A Comparison of College & Research Libraries and Journal of Academic Librarianship Gregory A. Crawford College & Research Libraries (C&RL) and the Journal of Academic Librarianship (JAL) are the two primary journals in the field of academic librarianship. This study examines articles appearing in C&RL and JAL so as to evaluate these journals on the basis of type of articles pub­ lished, structure of the articles, types of statistics used, and data collec­ tion methods used. The results indicate that C&RL publishes significantly more research articles than does JAL. Other results show that surveys are the primary form of data collection and that descriptive statistics are used much more frequently than inferential statistics. The percentage of research articles included in these journals has increased since earlier studies. henever the literature of aca- demic librarianship is dis- cussed, one often hears that the field lacks a good research base. Over time, many authors have called for more research to fill this gap.1 In 1990, ALA published Academic Libraries: Research Perspectives, which examined both basic and applied research on academic librarianship.2 The book also included two chapters that presented views on what re- search is needed by the field.3 The Association of College and Re- search Libraries (ACRL) has gone so far as to publish a list of desired topics for re- search on academic librarianship. In 1995, the ACRL College Libraries Section’s Re- search for College Librarianship Commit- tee presented its “Research Agenda for College Librarianship.”4 The committee developed a list of questions for which re- search related to college librarianship was needed. The goals of the document were “to identify research areas of specific in- terest to college librarianship,” “to offer librarians and graduate students who are new to research an agenda for college librarianship,” and “to provide examples of questions in each research area that are meaningful and that deal with the reali- ties facing college librarianship.”5 The agenda included twenty-nine specific re- search questions that the committee felt needed substantive research. In a recent editorial in JAL, Peter Hernon and several members of the edi- torial board discussed research opportu- nities in the field of library and informa- tion science (LIS).6 The resulting issues in- cluded consortial arrangements, services, Gregory A. Crawford is Head of Public Services in Heindel Library at Penn State Harrisburg; e-mail:gac2@psu.edu. 224 mailto:e-mail:gac2@psu.edu Research Literature of Academic Librarianship 225 decision making, and the position of li- braries within their larger environment. Many called for model building and both basic and applied research. According to various rankings of jour- nals in the field of LIS, the two journals that dominate the field of academic librarianship are C&RL and JAL.7 These journals have as one of their goals the dis- semination of research that addresses problems in academic librarianship. This article explores the research lit- erature of academic librarianship by ex- amining the nature of the articles in- cluded in C&RL and JAL. Specifically, it examines the type of articles published, the structure of the articles, the types of statistics used, and the data collection methods used. Literature Review Examinations of the LIS literature are abundant. Of special concern has been the nature of citation patterns among articles both within the field and in fields outside librarianship, the types of authorship of articles, the content of ar- ticles, and the statistical methods used by authors. The relationship between journals in library and information science and communication has been examined in detail in a series of articles and chap- ters by Christine L. Borgman, Ronald E. Rice, and their colleagues.8 Their ar- ticles show that LIS, as a distinct field, cites other fields regularly, including c o m m u n i c a t i o n . I n c o n t r a s t , o t h e r fields cite LIS only rarely. The content of LIS research articles from thirty-seven journals also has been scrutinized by Kalervo Jarvelin and Pertti Vakkari.9 Their results showed that 54 per- cent of the articles could be classified as research articles, whereas 46 percent were professional articles (including reviews and bibliographies). Jarvelin and Vakkari found that the predominant areas of re- search, accounting for almost 60 percent of the research articles, were LIS activi- ties and information storage and retrieval. In the articles they examined, three main types of research strategies were used: survey, argumentation/criticism, and sys- tems analysis and design. The combining of several data collection methods in one study was rare, as was the use of qualita- tive methods. Lois Buttar studied the content of ar- ticles in sixteen LIS journals.10 She found that over 70 percent of the articles could be classed as “nonresearch.” Surveys were the predominant method of collect- ing data, although documentary evi- dence for historical study, content analy- sis, and citation analysis also were used frequently. She did not examine the type of statistics employed. Kathy B. Enger, Georgia Quirk, and An- drew J. Stewart researched the use of sta- tistics in LIS articles.11 They examined twenty-five core journals published in 1985 and found that academic librarians relied heavily on descriptive statistics, whereas library school faculty often used inferential statistics. They also discovered that less than 21 percent of the articles in the study used descriptive statistics, only 11 percent used inferential statistics, and almost 70 percent used no statistics at all. Barbara C. Brattin examined the types of quantitative methods used in LIS re- search.12 Her study included articles from six journals, including C&RL (but not JAL). Of the 186 articles she studied, sixty-eight (37%) used descriptive statistics and only seventeen (9%) used inferential statistics. According to Brattin’s data, the Journal of the American Society for Information Science had the highest percentage of “research” articles with 71 percent representing either empirical research or model building, whereas 50 percent of the articles in C&RL could be labeled as “research.” In his sta- tistical profile of C&RL, Paul Metz found that articles appearing in recent issues of the journal used quantitative methodolo- gies more frequently, especially for the display of information, when compared to articles appearing in previous de- cades.13 Suhasini Kumar examined 312 articles from ten LIS journals, including JAL.14 Specifically, Kumar studied subject mat- http:cades.13 http:search.12 http:articles.11 http:journals.10 226 College & Research Libraries May 1999 ter, authorship, and research methods used. Most articles (72%) were nonresearch based. Among the research articles, the most frequently used research method was the survey, accounting for 46 percent of the total methods used. Of the articles appearing in JAL during the time period studied, 80 percent were classified as nonresearch based. A variety of articles have examined authorship characteristics of LIS ar- ticles. John Budd and Charles A. Seavey looked at the authorship of articles in thirty-six LIS journals over a five-year period.15 They found that the vast major- ity of authors contributed only one paper over the time period studied. In addition to article content, Buttar examined author gender, geographic location, occupation, and affiliation as well as the number of authors per paper for articles appearing in sixteen LIS journals.16 More recently, James L. Terry updated previous studies by Gloria S. Cline and Metz.17 He studied gender, institutional affiliation, and extent of coauthorship of articles appearing in C&RL for the years 1989–1994. Mickey Zemon and Alice Harrison Bahr studied authorship of articles appearing in both C&RL and JAL between 1986 and 1996.18 They also studied the motivation of publishing. Interestingly, more than half the respondents to their survey rated tenure as the least important motivating factor. Methodology This research sought to determine the differences, if any, between articles ap- pearing in College & Research Libraries and the Journal of Academic Librarianship. The specific focus of the study was type of article (i.e., research or nonresearch), structure of the articles, presentation of graphic information, use of statistics, and data collection methods employed. Ac- cording to its author instructions: College & Research Libraries includes articles in all fields of interest and concern to academic and research li- braries. Well-written manuscripts on all aspects of academic and research librarianship will be considered. Manuscripts may include research studies, case studies, descriptive nar- ratives of successful and unsuccess- ful ventures, thoughtful discussions of issues in librarianship, and other suitable subjects.19 JAL’s guidelines for contributions states: The Journal of Academic Librarianship, an international and refereed jour- nal, publishes articles that focus on problems and issues germane to col- lege and university libraries. JAL provides a forum for authors to present research findings and, where applicable, their practical ap- plications and significance; analyze policies, practices, issues, and trends; speculate about the future of academic librarianship; and present analytical bibliographic essays and philosophical treatises.20 Thus, both journals seek to publish a variety of articles, both research and nonresearch. Both journals are published six times a year. For this study, articles appearing for two years (1996 and 1997) were analyzed. Table 1 provides a breakdown of the num- ber of articles by year for each journal. Only regular peer-reviewed articles were included in the study. Regularly occurring features such as book reviews and col- umns were excluded from consideration. One issue of JAL (November 1997) was ex- cluded from consideration because it was devoted to a symposium on geographic information systems and to regular col- umns and book reviews. Each article was coded for type of ar- ticle; presence of a literature review, meth- odology section, results section, discus- sion section, and conclusion section; use of tables, charts, or graphs; inclusion of a specified hypothesis, null hypothesis, sig- nificance level, independent and depen- dent variables, the study population, and the sampling method used; use of de- http:treatises.20 http:subjects.19 http:journals.16 http:period.15 Research Literature of Academic Librarianship 227 TABLE 1 Number of Articles per Year 1996 1997 Total College & Research Libraries 36 34 70 Journal of Academic Librarianship 28 26 54 Total 64 60 124 square test was not per for med on these data due to the number of cells with a frequency of less than five. The predominant ar- ticle type in C&RL is the quantitative scriptive and inferential statistics; and type of data collection methodology. The coding sheet was developing us- ing previous studies of LIS article con- tent and the Publication Manual of the American Psychological Association.21 The APA manual lists three major types of articles: empirical studies, review ar- ticles, and theoretical articles. The cod- ing sheet provided space for a more de- tailed analysis of both empirical and re- view articles, and added coding for both opinion articles and “other ” for ar- ticles that were difficult to classify. The APA manual also stresses the importance of the manuscript format and calls for spe- cific sections to be included, especially the introduction, method, results, and discus- sion. Also included on the coding sheets were the literature review and the conclu- sion sections. In addition, the coding sheet provided space to list the type of descrip- tive statistics, inferential statistics, and data collection methodologies used in the articles examined. For this research, the null hypothesis states that there is no difference between the two journals in the types of articles be- ing published. All analyses were per- formed using SPSS/PC+ software. To test differences between the two journals, chi-square tests were empirical article, accounting for over 64 percent of articles published during the time period studied. The article type with the second highest total was the “other ” category (14.3%). These articles included discussions of issues of impor- tance to librarians, literature reviews to build models, and critiques and reviews of case studies. In contrast, less than 25 percent of the articles in JAL were classified as quantita- tive empirical and the “other ” category (29.6%) represented the predominant type of article published. Among these ar- ticles were how-to articles, project reports, and model and issue discussions. Opin- ion articles were a close third, account- ing for over 20 percent of the articles published. The preceding data were analyzed af- ter reclassifying the articles into only two categories: research and nonresearch. The results show a significant difference be- tween JAL and C&RL (X2 = 15.8, p<.001). As shown in table 3, C&RL included eigh- teen nonresearch articles and fifty-two re- search articles, compared to JAL with thirty-three nonresearch articles and twenty-one research articles. A literature review is considered to be TABLE 2 used. A significance level of a = .05 was established for de- Article Type C&RL JAL termining the significance of any inferential statistics. Opinion Empirical: Qualitative 4 2 11 5 Empirical: Quantitative 45 13 Results Empirical: Case study 5 3 College & Research Libraries Review: Trend 1 2 and the Journal of Academic Review Policy 2 4 Librarianship have different Review: Bibliographic 1 0 patterns of article content, as Other 10 16 shown in table 2. The chi- http:Association.21 228 College & Research Libraries May 1999 a sine qua non of most research articles. According to the data generated by this study, however, a specific literature review is often missing from the articles in C&RL and JAL, although relevant literature may be reviewed in the introduction and throughout the article. Within C&RL, 46 percent of the articles included a specific literature review (often called something slightly different), whereas only 15 per- cent of articles in JAL included such a re- view. Thus, most of the articles in both journals lacked this “standard” feature (C&RL, 54%; JAL, 85%). The difference be- tween the two journals was significant (X 2= 13.3, p<.001). The APA manual states that a sepa- rate method section is recommended. As with the literature review, there was a significant difference between C&RL and JAL (X2 = 19.3, p<.001). C&RL ar- ticles were much more likely to include a methods section (66%) than were JAL articles (26%). A results section is mandated accord- ing to the APA manual. Several articles in both journals combined the results section with the discussion section. Other articles used slightly different terminology for the results section (e.g., findings) but were coded as having a results section. As with the literature re- view and methods section, there was a significant difference between C&RL and JAL (X2 = 24.7, p<.001). Most ar- ticles in C&RL (67%) contained a spe- cific results section, compared to only 22 percent of the JAL articles. The APA manual stresses the need for a discussion section. Most of the articles in this study (73%) did not provide a distinct discussion section, and there was no sig- nificant difference between the two jour- nals. Although not required by the APA manual, a conclusion or summary section is generally added to most articles, and most articles in the study (73%) did in- clude such a section. There was no signifi- cant difference between C&RL and JAL. The provision of charts, graphs, and tables can be a great help to the readers of articles. Of the articles studied, the ma- jority (67%) provided such assistance. However, the journals differed signifi- cantly in their use of such illustrative mat- ter (X2 = 9.8, p<.01). C&RL articles were more likely to contain tables, charts, or graphs than were JAL articles (79% com- pared to 52%). Hypotheses and null hypotheses form the bases of empirical research. Within these two journals, only ten articles spe- cifically stated their research hypotheses and none gave null hypotheses. Many ar- ticles did provide research questions they were addressing. Hypotheses and null hypotheses form the bases of empirical research. Within these two journals, only ten articles specifically stated their research hypotheses and none gave null hypotheses. Even fewer articles (n = 6, 5%) speci- fied the significance level for any inferen- tial statistics that would be used. Another standard feature of research articles is to specify explicitly any indepen- dent and dependent variables included in the study. There was a significant differ- ence between C&RL and JAL in the speci- fication of variables (X2 = 11.5, p<.001). Of the articles appearing in C&RL, thirty- eight (54%) provided variable specifica- tion whereas only thirteen (24%) of those in JAL did so. This is reflective of the greater number of research-oriented ar- ticles in C&RL. Population and sampling methodology are important features for understanding the relevance of research articles. Among the articles studied, forty-six (66%) in TABLE 3 Article Type C&RL JAL Total Nonresearch 18 33 51 Research 52 21 73 Total 70 54 124 X2=15.8, p<.001 Research Literature of Academic Librarianship 229 C&RL discussed the population of inter- est in the study, but only twelve (22%) in JAL included this information. The differ- ence was significant (X2 = 23.2, p<.001). A similar result was obtained for sam- pling. All the C&RL articles that discussed the population also discussed sampling. For JAL, more articles provided informa- tion on sampling (n = 16, 30%) than on the population as a whole. The difference between the two journals was significant (X2 = 15.9, p<.001). C&RL articles (n = 44, 63%) were much more likely to contain descriptive statis- tics than were JAL articles (n = 16, 30%), resulting in a significant difference (X2 = 13.5, p<.001). The major type of statistic used was a simple reporting of percent- ages, which was provided in forty-nine articles (40%). Means were given in twenty-two articles (18%), and standard deviations were given in only five (4%). One article presented a median, and none gave modes or variances. By far, the most popular methodol- ogy was the survey or questionnaire, which was used by a total of forty- two articles (34%). Inferential statistics were used in only twenty-six (21%) of the articles in the study. There was a significant difference between C&RL and JAL (X2 = 5.6, p<.05). Twenty articles (29%) in C&RL used at least one inferential statistic, compared to only six articles (11%) in JAL. Correlations and chi-squares were the most popular in- ferential statistics, appearing in eight ar- ticles (6.5%) each. ANOVAs were used in seven articles (6%), and t-tests were used in four (3.3%). Multiple regression was used in three articles (2.4%), and a vari- ety of more specialized tests were used in eleven (9%). Data collection methods were speci- fied in a total of seventy-four articles (60%), with a significant difference be- tween the two jour nals ( X 2 = 17.2, p<.001). Of the articles in C&RL, fifty- three (76%) included a description of the data collection methodology. In JAL, twenty-one articles (39%) included this information. By far, the most popular methodology was the survey or ques- tionnaire, which was used by a total of forty-two articles (34%). Documentary evidence was used in twenty-one ar- ticles (17%), and interviews were used in eight (6.5%). Other methods such as the Delphi technique, shelf surveys, and citation data were used in individual ar- ticles. Discussion and Conclusion The data presented in this study indicate that at the current time, C&RL can be con- sidered the primary publisher of research- based articles in academic librarianship. However, this in no way lessens the im- portance and impact of JAL. Both journals publish a mix of articles (research and nonresearch) that can provide both pro- fessional stimulation and practical infor- mation to academic librarians. Moreover, both journals strive to fulfill their goals of providing high-quality articles to their readers. When compared to earlier research, a higher percentage of research articles are now being published (for JAL, 20% versus 39% in this research; for C&RL, 50% versus 74% in the cur rent re- search).22 The growth in the percentage of research articles in these two jour- n a l s t h a t r e p r e s e n t a c a d e m i c librarianship indicates that librarians have heard the call for an increased level of research and are responding positively. Such a result is important to academic librarianship if, indeed, it wishes to be perceived as having a firm foundation based on research. Notes 1. Charles H. Busha, Research Methods in Librarianship (New York: Academic Pr.,1980); Pierce Butler, An Introduction to Library Science (Chicago: Univ. of Chicago Pr., 1933); Herbert Goldhor, http:search).22 230 College & Research Libraries May 1999 An Introduction to Scientific Research in Librarianship (Urbana, Ill.: Univ. of Illinois Pr., 1972); Jesse Shera, “Darwin, Bacon, and Research in Librarianship,” Library Trends 13 (1964): 141–49. 2. Mary Jo Lynch, Academic Libraries: Research Perspectives (Chicago: ALA, 1990). 3. Malcolm Getz, “Analysis and Library Measurement,” in Academic Libraries: Research Per- spectives, ed. Mary Jo Lynch (Chicago: ALA, 1990), 192–214; William G. Potter, “Insurmountable Opportunities: Advanced Technology and the Academic Library,” in Academic Libraries: Research Perspectives, ed. Mary Jo Lynch (Chicago: ALA, 1990), 165–91. 4. Research for College Librarianship Committee, “Research Agenda for College Librarianship,” College & Research Libraries News 56 (1995): 470–71+. 5. Ibid., 470. 6. Peter Hernon, “Editorial: Going beyond ‘Same Old, Same Old,’” Journal of Academic Librarianship 23 (1997): 169–76. 7. David F. Kohl and Charles H. Davis, “Ratings of Journals by ARL Library Directors and Deans of Library and Information Science Schools,” College & Research Libraries 46 (1985): 40–47; Mary K. Sellen, “Bibliometrics in Information Science: A Citation Analysis of Two Academic Li- brary Journals,” College & Research Libraries 45 (1984): 129–32; Robert Swisher and Peggy C. Smith, “Journals Read by ACRL Academic Librarians, 1973 and 1978,” College & Research Libraries 43 (1982): 51–58. 8. Christine L. Borgman and Ronald E. Rice, “The Convergence of Information Science and Communication: A Bibliometric Analysis,” Journal of the American Society for Information Science 43 (1992): 397–411; Ronald E. Rice, “Hierarchies and Clusters in Communication and Library and Information Science Journals, 197–1987,” in Scholarly Communication and Bibliometrics, ed. Chris- tine L. Borgman (Newbury Park, Calif.: Sage, 1990), 138–53; Ronald E. Rice, Christine L. Borgman, D. Bednarski, and P. J. Hart, “Journal-to-Journal Citation Data: Issues of Validity and Reliability,” Scientometrics 15 (1989): 257–82; Ronald E. Rice, Christine L. Borgman, and B. Reeves, “Citation Networks of Communication Journals, 1977–1985: Cliques and Positions, Citations Made and Ci- tations Received, ” Human Communication Research 15 (1988): 256–83; Ronald E. Rice and Gregory A. Crawford, “Context and Content of Citations between Communication and Library and Infor- mation Science Articles,” in Between Communication & Information, ed. Jorge R. Schement and Brent D. Ruben (New Brunswick, N.J.: Transaction, 1993), 189–217. 9. Kalervo Jarvelin and Pertti Vakkari, “Content Analysis of Research Articles in Library and Information Science,” Library and Information Science Research 12 (1990): 395–421. 10. Lois Buttar, “Analyzing the Library Periodical Literature: Content and Authorship,” College & Research Libraries 52 (1991): 38–53. 11. Kathy B. Enger, Georgia Quirk, and Andrew J. Stewart, “Statistical Methods Used by Au- thors of Library and Information Science Journal Articles,” Library and Information Science Research 11 (1989): 37–46. 12. Barbara C. Brattin, “Quantitative Methods in Library and Information Science Literature: Descriptive vs. Inferential Statistics” (Master ’s thesis, Kent State University, 1991). ERIC Docu- ment Reproduction Service No. ED 339 396. 13. Paul Metz, “A Statistical Profile of College & Research Libraries,” College & Research Libraries 50 (1989): 42–47. 14. Suhasini Kumar, “Content Analysis of Journal Literature in Library and Information Sci- ence from June 1994–June 1995” (Master ’s thesis, Kent State University, 1995). ERIC Document Reproduction Service No. ED 401 937. 15. John Budd and Charles A. Seavey, “Characteristics of Journal Authorship by Academic Li- brarians,” College & Research Libraries 51 (1990): 463–70. 16. Buttar, “Analyzing the Library Periodical Literature.” 17. James L. Terry, “Authorship in College & Research Libraries Revisited: Gender, Institutional Affiliation, Collaboration,” College & Research Libraries 57 (1996): 377–83; Gloria S. Cline, “College & Research Libraries: Its First Forty Years,” College & Research Libraries 43 (1982): 208–32; Metz, “Analy- sis and Library Measurement.” 18. Mickey Zemon and Alice Harrison Bahr, “An Analysis of Articles by College Librarians,” College & Research Libraries 59 (1998): 422–32. 19. “About College & Research Libraries,” College & Research Libraries 58 (1997): 105. 20. “Guidelines for Article Contributions,” Journal of Academic Librarianship 23 (1997): 78. 21. American Psychological Association, Publication Manual of the American Psychological Asso- ciation (Washington, D.C.: APA, 1994). 22. Buttar, “Analyzing the Library Periodical Literature;” and Kumar, “Content Analysis of Journal Literature in Library and Information Science” from June 1994—June 1995.”