EBL 101

 

Research Methods: Systematic Reviews

 

Virginia Wilson

Director, Centre for Evidence Based Library and Information Practice (C-EBLIP)

University Library

University of Saskatchewan

Saskatoon, Saskatchewan, Canada

Email: virginia.wilson@usask.ca

 

Received: 12 Aug. 2013 Accepted: 17 Aug. 2013

 

 

cc-ca_logo_xl 2013 Wilson. This is an Open Access article distributed under the terms of the Creative Commons-Attribution-Noncommercial-Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 

A systematic review is not merely a literature review. While we can be systematic in preparing a literature review, a systematic review is a research method used to address a specific research question. Systematic reviews “present a comprehensive summary of research based knowledge that can aid both practitioners and policy makers in decision making” (Brettle, 2009, p. 43). Meta-analysis, “the use of statistical methods to combine results of individual studies,” may or may not be conducted in a systematic review (The Cochrane Collaboration, 2002). Systematic reviews in LIS research were practically unheard of just over a decade ago. However, as Denise Koufogiannakis found, they “have become an important source of information because they both synthesize the existing research on a topic, as well as critically appraise it and try to draw conclusions from the total body of quality research evidence” (2012, p. 91).

 

The Cochrane Collaboration and the Campbell Collaboration are two organizations which oversee the conducting of systematic reviews based on specific guidelines. The Cochrane Collaboration focuses on the health sciences, while the Campbell Collaboration works in the areas of crime and justice, education, international development, and social welfare. Additionally, an individual researcher or research team can undertake a systematic review to attempt to answer a research question. Khan, Kunz, Kleijnen, andAntes explain that a “review earns the adjective systematic if it is based on a clearly formulated question, identifies relevant studies, appraises their quality and summarizes the evidence by use of explicit methodology. It is the explicit and systematic approach that distinguishes systematic reviews from traditional reviews and commentaries” (2003, p. 118).

 

The process by which to undertake a systematic review has several steps:

 

1.       Develop a research question

 

As with any research project, the development of a clear, explicit, and concise research question is the bedrock upon which the project rests. Spend time thinking and planning.

 

2.       Identify relevant work

 

Do an extensive search for research studies. You may be looking at a particular date range. Once you have exhausted the literature, decide which papers to include and exclude based on criteria that have come out of your research question. You will probably get a lot of results from your initial massive searches and the criteria will help to sort out which studies belong in your systematic review. A test run of the inclusion and exclusion criteria will show if more or is needed or changes need to be made. Record the reasons for inclusion or exclusion.

 

3.       Critically appraise the included studies

 

You need to look for quality and rigour. Quality is difficult to define but thorough critical appraisal can help to determine if the study results are sound. Critical appraisal will also help to determine whether or not meta-analysis will be a part of your systematic review.

 

4.       Extract the data

 

Your research question will guide the types of data you will want to extract from the included studies. Create a standardized data extraction form to keep track of everything you pull from the studies. Methodically review the articles in order to fill out the extraction form.

 

5.       Synthesize and analyze the findings

 

“The goal of data synthesis is to go beyond simply summarizing but to also include ‘an analysis of the relationships within and between studies and an overall assessment of the robustness of the evidence’” (Centre for Reviews and Dissemination, 2009, 48 as quoted in Phelps & Campbell, 2012, p. 13).

 

The final and best thing I can pass along is that Denise Koufogiannakis along with several contributors put together a comprehensive list of all the systematic reviews undertaken in LIS to date. The LIS Systematic Reviews wiki includes the reviews listed alphabetically by author and classified by topic. There is also a nice bibliography where you can find much more information on the systematic review.

 

 

References

 

Brettle, A. (2009). Systematic reviews and evidence based library and information practice. Evidence Based Library and Information Practice, 4(1): 43-50.

 

Khan, K.S., Kunz, R., Kleijnen, J., & Antes, G. (2003). Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 96: 118-121.

 

Koufogiannakis, D. (2012). The LIS systematic review wiki. Retrieved 4 Sept. 2013 from http://lis-systematic-reviews.wikispaces.com/Welcome

 

Koufogiannakis, D. (2012). The state of systematic reviews in library and information studies. Evidence Based Library and Information Practice, 7(2): 91-95.

 

Meta-analysis. (2002). In The Cochrane Collaboration open learning material. Retrieved 4 Sept. 2013 from http://www.cochrane-net.org/openlearning/html/mod12-2.htm   

 

Phelps, S. F., and Campbell, N. (2012). Systematic reviews in theory and practice for library and information studies. Library & Information Research, 36(112): 6-15.