Evidence Summary

LIS Periodicals Contain a Low Percentage of Articles that Qualify as Research

A Review of:

Turcios, M. E., Agarwal, N. K., & Watkins, L. (2014). How much of library and information science literature qualifies as research? Journal of Academic Librarianship, 40(5), 473-479. http://dx.doi.org/10.1016/j.acalib.2014.06.003

Reviewed by:

Richard Hayman

Assistant Professor & Digital Initiatives Librarian

Mount Royal University

Calgary, Alberta, Canada

Email: rhayman@mtroyal.ca

Received: 29 Apr. 2015 Accepted: 17 Sept. 2015

2015 Hayman. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To determine how much of the literature in a library and information science (LIS) periodical collection qualifies as research.

Design – Content analysis.

Setting – The LIS periodicals collection of an academic library that supports an established LIS graduate program at a college in the United States of America.

Subjects – Of the 177 identified periodicals with LIS content that fell within project scope from the local collection, researchers analyzed 101 journals that include academic/scholarly content and an additional 4 journals with relevant trade content. This study excluded open access (OA) journals.

Methods – Using the most recent issue of each subject journal from the fiscal year 2012-2013, the authors performed a content analysis on all indexed content items, and then classified each content item as research or non-research. For content identified as research, researchers identified the research method (or methods) used. The data collection tool also captured identifying information and keywords for all content.

Main Results – Within the journals meeting the scope of this study, researchers identified 1,880 articles from 105 individual journal issues. Only 16% (n=307) of articles met the authors’ established definition to qualify as research. Within the subset of research articles, the authors further identified 45% (n=139) that used a single research method. An additional 36% (n=112) of identified research articles used two research methods and 15% (n=46) used three methods, with the remainder using four or more methods.

Surveys were the most frequently used research method, accounting for 49% (n=66) of the single method studies. The researchers discovered that surveys remained popular even in mixed-method studies, with 21% (n=117) of all identified research articles using surveys. This is closely followed by 20% (n=109) of studies reported as using the general category of “other” methods, for research that did not meet one of the predefined methods. The next two most popular identified methods were case studies at 13% (n=73), followed by content analyses at 13% (n=71). For the eight other research methods identified, none saw a frequency above 10%. Focus groups and usability studies tied for the least frequently used method among the 307 articles, both at 2% (n=9).

The keyword analysis focused on two categories, one for research article keywords and another for non-research article keywords, for all 1,880 articles identified. Non-research articles had less reliance on keywords, with authors reporting keywords appearing on 73% (n=1156). Within these, authors discovered 120 separate keywords used 10 or more times across non-research articles. The top ten keywords among non-research articles were reported as primarily related to books and publishing, with “non-fiction,” “adult,” and “libraries” as the top three. By comparison, research articles heavily favour the use of keywords, with 94% (n=290) of research articles having keywords. Analysis of the individual keywords found 56 keywords appearing 10 or more times across research articles. The top ten keywords are primarily practice related, with “information,” “libraries,” and “library” being the top three. When comparing shared keywords across both categories, the same top three keywords reported for research in the previous sentence apply to the collective set.

Conclusion – The authors note that the nature and size of the local collection both benefited and limited this study. Compiling and maintaining a comprehensive list of LIS periodicals is a challenging task across a large body of potential sources. Within the resulting periodicals studied, a mere 16% of analyzed LIS literature met the criteria to qualify as research, and that only after the study had eliminated virtually all trade periodicals from the population. Had that trade literature been included, the percentage qualifying as research would have been even lower. The popularity of surveys as a research method among LIS research reflects other recent findings, though the frequency of studies falling into the general “other” category suggests that LIS research is changing. Based on this research, the authors conclude that there is still much to be learned from content analysis of literature published in LIS periodicals. Future analyses could further examine the frequency of research methods used within LIS research.

Commentary

This study identified trends within LIS literature, identifying a number of LIS journals that contain academic/scholarly content as a starting place. By examining content from single periodical issues, the authors achieved a broad, though not deep, analysis of the quantity of LIS research as compared to all LIS literature. A recent comparative example is Chu’s (2015) content analysis study that has depth but not breadth, in that it focuses on just 3 LIS journals, analyzing 1,162 research articles published over a 10-year period.

For researchers interested in replicating this study or employing similar methods for other collections, the authors provide their list of LIS periodicals that include academic content. They also include an example data collection form alongside a coding document that defines different research methods. Together these aid the reader in understanding what content is included in the analysis. Something missing from this documentation and the report is a full explanation of the “other” category of research methods. Given the frequency of these undefined methods, across 20% of all research articles, this study would be strengthened by addressing the various approaches captured in this category. The authors recognize that more detail is required and suggest this as an avenue for further studies. In this reviewer’s opinion the keyword analysis yields little of use to practitioners. The top results within research articles (“information,” “libraries,” and “library”) and within non-research articles (“non-fiction,” “adult,” and “libraries”) are all so generic that they are of little use for comprehensive topic searching.

The study population is tied to a specific library collection that supports an LIS graduate program. The authors infer that the sample population is representative of LIS literature and conclude that their analysis provides “vital statistical data pertaining to the current state of LIS research and periodicals” (p. 479). Yet the authors acknowledge that their sample population focuses on subscription periodicals and does not include open access journals, and this represents a significant lost opportunity. Among others, Yuan and Hua (2011) have illustrated that LIS has fully adopted OA literature as mainstream, so the exclusion of non-subscription content excludes a relevant body of literature from the study population.

Ultimately the low frequency of research in the literature, just 16%, might seem alarming. Based on the research presented here it is possible to infer that the body of LIS literature is crowded with non-research articles, suggesting that there are opportunities for journals to adjust the balance of research versus non-research content. However, we must consider that the authors examined all indexed content within the periodicals under consideration, without accounting for whether the content itself was presented as research. Many publications include non-research content in formats that are indexed: editorials, reviews, commentaries, interviews, and even scholarly evidence summaries are treated as indexed content, but they are not written as, nor typically presented as, original research. A fundamental question, and one needing more study, would be to ask: how much of LIS literature presented as research actually qualifies as research? Further research might help us understand whether journal editors need to prioritize and include more research content in their publications, and also whether the LIS profession itself should be responsible for generating more research overall.

References

Chu, H. (2015). Research methods in library and information science: A content analysis. Library & Information Science Research, 37(1), 36-41. http://dx.doi.org/10.1016/j.lisr.2014.09.003

Yuan, S., & Hua, W. (2011). Scholarly impact measurements of LIS open access journals: Based on citations and links. The Electronic Library, 29(5), 682-697. http://dx.doi.org/10.1108/02640471111177107