Systematic reviews provide a complete picture of the totality of evidence on a given topic.³ In practice, a summary of evidence is considered to be a systematic review if (at a minimum) the authors conducted a literature search that was comprehensive enough to avoid publication, language and indexing biases; report the criteria used for deciding which studies to include in the review; undertook duplicate study selection and data extraction; and combined data from included studies using reliable methods.⁴

There are numerous examples where failure to prepare timely systematic reviews of existing research evidence resulted in untold and preventable suffering.^5,6 In a recent methodological study, Ochodo and colleagues estimated the frequency of over-interpretation or ‘spin’, defined as:

reporting that distorts study results to make interventions look favourable e.g. an overly optimistic abstract, stronger conclusion in abstract, selective reporting of results in abstract, study conclusions based on selected subgroups, and discrepancy between aim and conclusion.⁷

The authors searched and identified 126 eligible primary diagnostic accuracy studies published between January and June 2010 in PubMed-indexed journals with an impact factor of at least 4. Of these studies, 53 focused on radiological imaging. An analysis of the latter revealed that one-third of the studies contained forms of actual over-interpretation and all contained forms of potential over-interpretation of the diagnostic accuracy of imaging.

In line with this methodological study, I will use two examples to illustrate the importance of systematic reviews in radiology. The first review was conducted by Brealey and colleagues to determine the accuracy of radiographer plain radiograph reporting in clinical practice.⁸ The authors conducted a comprehensive search of numerous peer-reviewed and grey literature sources for studies conducted between 1971 and October 2002. Twelve studies were included in this review, which revealed that the sensitivity and specificity of radiographers’ reports of plain radiographs were 93% and 98% respectively, against a reference standard. The subgroup of studies that focused on accident and emergency settings found no evidence of a difference in reporting accuracy between selectively trained radiographers and radiologists of varying seniority, compared to a reference standard. The authors concluded that flexible teamwork between different professions as to who reports plain radiographs should be promoted. The second example is a comprehensive review of the effectiveness of several radiological techniques in diagnosing occult inguinal hernias, published in 2013.⁹ The authors included 23 studies published since 1950, and found that herniography has a sensitivity of 91% and a specificity of 83%, ultrasound a sensitivity of 86% and a specificity of 77%, and computed tomography a sensitivity of 80% and a specificity of 65% in detecting occult inguinal hernias. The implications of this review for clinical practice, in settings where all three techniques are available, are that herniography should be the initial investigation for occult inguinal hernia. In settings where herniography is not available, ultrasound of the groin should be used, with consideration of computed tomography only in the presence of persistent diagnostic uncertainty.

As illustrated by these systematic reviews, professional good intentions and plausible theories are inadequate criteria for selecting interventions to promote, restore, maintain, protect or monitor the health of human populations. As Iain Chalmers said:

Humility and uncertainty are preconditions for unbiased assessments of the effects of the prescriptions and proscriptions of policy makers and practitioners for other people. We will serve the public more responsibly and ethically when research designed to reduce the likelihood that we will be misled by bias and the play of chance has become an expected element of professional and policymaking practice, not an optional add-on.¹⁰

Universal adoption of systematic reviews in radiology and other healthcare disciplines will ensure that patients benefit from health research and that healthcare resources are used efficiently. Systematic reviews provide a means for decision makers (including policy makers, programme managers and clinicians) to access all available evidence on key questions in a judicious manner¹¹ as well as identify areas where there are knowledge gaps, thus assisting researchers and research funders in setting priorities for new studies¹². Systematic reviews help to increase value and reduce waste in research priority-setting. Therefore:

research funders and regulators should demand that proposals for additional primary research are justified by systematic reviews showing what is already known, and increase funding for the required syntheses of existing evidence.¹³

However, an investigation of the current research landscape paints a dismal picture of how limited research resources continue to be wasted on unnecessary research and needless confusion continues to persist from failure to set new studies in the context of systematic reviews.^13,14 But, every cloud has a silver lining. This could be a golden opportunity for radiologists in South Africa to provide leadership in increasing value and decreasing waste in health research, through routine use of systematic reviews when making healthcare decisions and designing new studies. The ball is in your court – shall you make or mar?

2. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn’t. BMJ. 1996;312:71–72. http://dx.doi.org/10.1136/bmj.312.7023.71

3. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of reporting of meta-analyses. Lancet. 1999;354:1896–900. http://dx.doi.org/10.1016/S0140-6736(99)04149-5

4. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. http://dx.doi.org/10.1186/1471-2288-7-10

5. Murphy C, Hahn S, Volmink J. Reduced osmolarity oral rehydration solution for treating cholera. Cochrane Database Syst Rev. 2004;(4):CD003754.

6. Wiysonge CS, Bradley H, Mayosi BM, et al. Beta-blockers for hypertension. Cochrane Database Syst Rev. 2007;(1):CD002003.

7. Ochodo EA, De Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: Evidence of ‘spin’. Radiology. 2013;267:581–588. http://dx.doi.org/10.1148/radiol.12120527

8. Brealey S, Scally A, Hahn S, Thomas N, Godfrey C, Coomarasamy A. Accuracy of radiographer plain radiograph reporting in clinical practice: A meta‐analysis. Clin Radiol. 2005;60:232–241. http://dx.doi.org/10.1016/j.crad.2004.07.012

9. Robinson A, Light D, Kasim A, Nice C. A systematic review and meta‐analysis of the role of radiology in the diagnosis of occult inguinal hernia. Surg Endosc. 2013;27:11–18. http://dx.doi.org/10.1007/s00464-012-2412-3

10. Chalmers I. Trying to do more good than harm in policy and practice: The role of rigorous, transparent, up-to-date evaluations. Ann Am Acad Polit Soc Sci. 2003;589:22–40. http://dx.doi.org/10.1177/0002716203254762

11. Wiysonge CS, Volmink J. Strengthening research capacity. Lancet. 2002;359:713. http://dx.doi.org/10.1016/S0140-6736(02)07798-X

12. Wiysonge CS, Lavis JN, Volmink J. Make the money work for health in sub-Saharan Africa. Lancet 2009;373:1174. http://dx.doi.org/10.1016/S0140-6736(09)60685-1

13. Chalmers I, Bracken MB, Djulbegovic B, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383:156–165. http://dx.doi.org/10.1016/S0140-6736(13)62229-1

14. Robinson KA, Goodman SN. A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med. 2011;154:50–55. http://dx.doi.org/10.7326/0003-4819-154-1-201101040-00007