Systematic reviews in context: highlighting systematic reviews relevant to Africa in the Pan African Medical Journal
Charles Shey Wiysonge1,2,&, Raoul Kamadjeu3, Landry Tsague3
1Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa, 2Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa, 3The Pan African Medical Journal, Center for Public Health Research and Information, Nairobi, Kenya
Charles Shey Wiysonge, Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Stellenbosch University, Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa
Health research serves to answer questions concerning health and to accumulate facts (evidence) required to guide healthcare policy and practice. However, research designs vary and different types of healthcare questions are best answered by different study designs. For example, qualitative studies are best suited for answering questions about experiences and meaning; cross-sectional studies for questions concerning prevalence; cohort studies for questions regarding incidence and prognosis; and randomised controlled trials for questions on prevention and treatment. In each case, one study would rarely yield sufficient evidence on which to reliably base a healthcare decision.
An unbiased and transparent summary of all existing studies on a given question (i.e. a systematic review) tells a better story than any one of the included studies taken separately. A systematic review enables producers and users of research to gauge what a new study has contributed to knowledge by setting the study’s findings in the context of all previous studies investigating the same question. It is therefore inappropriate to initiate a new study without first conducting a systematic review to find out what can be learnt from existing studies. There is nothing new in taking account of earlier studies in either the design or interpretation of new studies. For example, in the 18th century James Lind conducted a clinical trial followed by a systematic review of contemporary treatments for scurvy; which showed fruits to be an effective treatment for the disease.
However, surveys of the peer-reviewed literature continue to provide empirical evidence that systematic reviews are seldom used in the design and interpretation of the findings of new studies. Such indifference to systematic reviews as a research function is unethical, unscientific, and uneconomical. Without systematic reviews, limited resources are very likely to be squandered on ill-conceived research and policies. In order to contribute in enhancing the value of research in Africa, the Pan African Medical Journal will start a new regular column that will highlight priority systematic reviews relevant to the continent.
When a patient survives a first heart attack, the heart may be so badly damaged that the patient will develop mechanical pump failure. A heart attack may also lead to irregularities in the heart rhythm because of heart muscle damage. Heart attack survivors who develop one or both of these two complications have a higher risk of death than those who do not . In addition, when a patient survives a first heart attack, the chances are that he or she may suffer another heart attack; which will trigger a new set of electrical and mechanical complications. The work done by the left ventricle in pumping blood and the occurrence of heart rhythm irregularities are both increased by endogenous chemical substances known as catecholamines. The action of these substances on beta-adrenergic receptors, part of the sympathetic nervous system that mediates the ‘fight-or-flight’ response, is blocked by a class of drugs called beta-blockers .
On this basis, beta-blockers were proposed for use in people who had survived a heart attack in order to reduce the risk of future cardiac events. The first published randomised controlled trial for this indication was reported in 1972 . A decade later, more trials had been done, but all trials were generally small in size and the individual results were confusing because none of the trial authors had taken a systematic account of previous similar trials. Had they done so, it would have been obvious that enough evidence had accumulated showing a beneficial effect of beta-blocker therapy for secondary prevention after a heart attack [4-6].
Following the publication in 1981 of the seventh study on the topic , a narrative review reported that there was still no convincing evidence that beta-blocker therapy led to long-term survival after heart attacks . However, at that time there was already overwhelming evidence that giving a beta-blocker to heart attack survivors reduced the risk of future cardiac events [5,6]. Yet, over the ensuing decade, trials continued to be done in which beta-blockers were compared to placebo or no treatment . If a systematic review of existing trials had been done and the data pooled using an appropriate statistical method, the combined evidence would have shown that the conduct of further trials was unnecessary. The pointless trials, that continued to be undertaken, deprived half the trial participants of an effective treatment. We see here a failure to boost the significance of existing research and decrease waste in the conduct of new research, because of a failure to set each new study in the context of other relevant studies .
Narrative reviews, such as the one referred
to above , are prone to multiple systematic
errors (or biases). Authors of such reviews typically use subjective
methods to collect and interpret existing data. One source of systematic
error in narrative reviews is publication bias, which occurs when
authors of a review only search for published studies. It is well
documented that studies with statistically significant results
are more likely to be published and more likely to be published
in journals with high citation impact factors than studies with
non-significant results . Therefore, if
a review of treatment effects only considers published studies,
it is very likely to over-estimate the effectiveness of the treatment
under consideration. Another common error in narrative reviews
is language bias. For example, a study of German authors who had
published different trials in both English and German language
journals revealed that statistically significant results were more
likely to be published in English . Thus
a review that searches only for English language publications is
more likely to overestimate treatment effects. Narrative reviews
may also ignore study design in the analysis of existing data.
A systematic review was conducted to assess the effectiveness of
vitamin supplementation in preventing cardiovascular disease .
The authors found a significant association between the use of
vitamin supplements and a lower risk of cardiovascular mortality
in cohort studies. However, randomised controlled trials of the
same supplements failed to demonstrate a consistent effect. Beta-carotene,
for example, significantly reduced cardiovascular mortality in
cohort studies but significantly increased cardiovascular mortality
in randomised controlled trials . When
studying the effects of interventions such as beta-carotene, randomised
trials are better suited than cohort studies because they have
the advantage of taking account of both known and unknown factors
that may influence the risk of having the outcome (such as cardiovascular
mortality in this case). Sources of bias, such as those discussed
above, render the results of narrative reviews unreliable; thus
the need for systematic reviews.
Key characteristics of systematic reviews
The core element of a systematic review is an explicit attempt to collate the totality of relevant existing data on a clearly defined question [14
]. A systematic review is characterised by a well-defined and focused question; pre-defined eligibility criteria for selecting studies; a comprehensive search strategy for identifying all potentially eligible studies; duplicate assessment of the risk of bias and extraction of data from included studies; an appropriate synthesis of data; and a complete presentation of the findings. Statistical aggregation (referred to as meta-analysis) may or may not be used to summarise data from studies included in a systematic review [15
Significance of systematic reviews
Without systematic reviews of previous research, ineffective or even harmful interventions may be used because they are thought to be effective and, conversely, effective interventions may be considered ineffective and withheld [16
]. A systematic review should be the first step when defining questions for new research and when taking decisions about health care. When taking decisions about health care, there should be clear documentation of how relevant systematic reviews were identified and assessed for their quality, local applicability, potential impacts on equity, cost implications, and scaling-up considerations [17
]. When systematic reviews are ignored, it is very likely that limited healthcare resources would be squandered on ill-conceived research and policies, and avoidable confusion would result from failure to set new research in the context of relevant existing research [18
]. There is nothing new about systematic reviews. For example, in 1747 James Lind conducted a systematic review to assess the effects of various contemporary treatments for scurvy [1
]. Another example is the emphasis of the importance of systematic reviews, to the British Association for the Advancement of Science, by Lord Rayleigh in 1884 [19
Highlighting systematic reviews in the Pan African Medical Journal
As shown in the preceding sections, a systematic review of existing research on a given topic paints a better picture of the topic than any one primary study taken separately. In order to increase the use of systematic reviews in Africa, the Pan African Medical Journal will start a new section called “Systematic reviews in context” that will highlight systematic reviews relevant to the continent.
Systematic reviews can help to identify what studies should be replicated, to avoid unnecessary duplication, and result in new studies that address deficits in previous ones. Despite the value of systematic reviews, there is empirical evidence showing that they continue to be ignored in both the design and the interpretation of findings of new research [19
]. This contempt for systematic reviews may occur because researchers are so preoccupied with a misguided philosophy of individual originality that they do not realise that research is a shared scientific activity, in which the community accomplishes more than the sum of the efforts of its members. This misguided philosophy should change, if the efforts of researchers are to yield maximum benefits to the wider community which continues to invest so much in research. We believe that the time for that change is now. This explains why the Pan African Medical Journal plans to regularly highlight systematic reviews relevant to Africa, with the hope that this would contribute to evidence-informed health research, policy, and practice on the continent.
The authors declare no competing interest.
All three authors participated in the conception of the manuscript; Charles Shey Wiysonge wrote the first draft; and all authors critically revised successive versions. All authors have read and agreed to the final version of this manuscript.
Charles Shey Wiysonge's work is supported by Stellenbosch University, the National Research Foundation of South Africa and the Effective Health Care Research Consortium. However, the views expressed in this manuscript are those of the authors and do not necessarily reflect those of the organisations to which the authors are associated.
- Evans I, Thornton H, Chalmers I, Glasziou P. Testing treatments: better research for better healthcare. 2nd edition. London: Pinter & Martin, 2011. Google Scholar
- Wiysonge CS, Volmink J, Opie LH. Beta-blockers and the treatment of hypertension: it is time to move on. Cardiovasc J Afr. 2007 Nov-Dec; 18(6): 351-2. PubMed | Google Scholar
- Reynolds JL, Whitlock RM. Effects of a beta-adrenergic receptor blocker in myocardial infarction treated for one year from onset. Br Heart J. 1972 Mar; 34(3): 252-9. PubMed | Google Scholar
- May GS, Furberg CD, Eberlein KA, Geraci BJ. Secondary prevention after myocardial infarction: a review of short-term acute phase trials. Prog Cardiovasc Dis. 1983 Jan-Feb; 25(4): 335-59. PubMed | Google Scholar
- May GS, Eberlein KA, Furberg CD, Passamani ER, DeMets DL. Secondary prevention after myocardial infarction: a review of long-term trials. Prog Cardiovasc Dis. 1982 Jan-Feb; 24(4): 331-52. PubMed | Google Scholar
- Yusuf S, Peto R, Lewis J, Collins R, Sleight P. Beta blockade during and after myocardial infarction: an overview of the randomized trials. Prog Cardiovasc Dis. 1985 Mar-Apr; 27(5): 335-71. PubMed | Google Scholar
- Norwegian Multicenter Study Group. Timolol-induced reduction in mortality and reinfarction in patients surviving acute myocardial infarction. N Engl J Med. 1981 Apr 2; 304(14): 801-7. PubMed | Google Scholar
- Mitchell JR. Timolol after myocardial infarction: an answer or a new set of questions? Br Med J (Clin Res Ed). 1981 May 16; 282(6276): 1565-70. PubMed | Google Scholar
- Hjalmarson A, Olsson G. Myocardial infarction: effects of beta-blockade. Circulation. 1991; 84(6 Suppl): VI101-7. PubMed | Google Scholar
- Chalmers I, Bracken MB, Djulbegovic B et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014 Jan 11; 383(9912): 156-65. PubMed | Google Scholar
- Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991 Apr 13;337(8746):867-72. PubMed | Google Scholar
- Egger M, Zellweger Z, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet. 1997 Aug 2; 350(9074): 326-9. PubMed | Google Scholar
- Morris CD, Carson S. Routine vitamin supplementation
to prevent cardiovascular disease: a summary of the evidence for the U.S Preventive
Services Task Force. Ann Intern Med. 2003 Jul 1; 139(1): 56-70. PubMed | Google
- Birbeck GL, Wiysonge CS, Mills EJ, Frenk JJ, Zhou XN, Jha P. Global health: the importance of evidence-based medicine. BMC Med. 2013 Oct 16; 11: 223. PubMed | Google Scholar
- Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.1.0[updated March 2011]. Oxford, UK: The Cochrane Collaboration, 2011. Available from http://handbook.cochrane.org/ (accessed 18 June 2016). Google Scholar
- Chalmers I. Trying to do more good than harm in policy
and practice: the role of rigorous, transparent, up-to-date evaluations. Ann
Am Acad Polit Soc Sci. 2003; 589(1): 22-40. PubMed | Google
- Wiysonge CS, Lavis JN, Volmink J. Make the money work for health in sub-Saharan Africa. Lancet. 2009 Apr 4; 373(9670): 1174. PubMed | Google Scholar
- Wiysonge CS, Volmink J. Strengthening research capacity. Lancet. 2002 Feb 23; 359(9307): 713. PubMed | Google Scholar
- Clarke M, Brice A, Chalmers I. Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS ONE. 2014; 9(7): e102670. PubMed | Google Scholar
- Robinson KA, Goodman SN. A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med. 2011 Jan 4; 154(1): 50-5. PubMed | Google Scholar