Información de la revista
Vol. 105. Núm. 3.
Páginas 295-299 (Abril 2014)
Visitas
5135
Vol. 105. Núm. 3.
Páginas 295-299 (Abril 2014)
Original Article
Acceso a texto completo
The Use of Systematic Reviews in Clinical Trials and Narrative Reviews in Dermatology: Is the Best Evidence Being Used?
Empleo de las revisiones sistemáticas en ensayos clínicos y revisiones narrativas en dermatología: ¿se usa la mejor evidencia disponible?
Visitas
5135
A. Conde-Taboadaa,
Autor para correspondencia
condetaboada@aedv.es

Corresponding author.
, B. Araneguib, I. García-Dovalc, P. Dávila-Seijoc, U. González-Castrod
a Servicio de Dermatología, Hospital Clínico San Carlos, Madrid, Spain
b Servicio de Dermatología, Clínica Universitaria de Navarra, Madrid, Spain
c Servicio de Dermatología, Complexo Hospitalario de Pontevedra, Pontevedra, Spain
d Cochrane Skin Group
Este artículo ha recibido
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Figuras (2)
Abstract
Introduction and objectives

Systematic reviews —the most comprehensive type of literature review—should be taken into account before a clinical trial or a narrative review on a topic is undertaken. The objective of this study was to describe the use of systematic reviews in clinical trials and narrative reviews in dermatology.

Material and methods

This was a descriptive cross-sectional study. We selected randomized clinical trials and narrative reviews from the dermatological clinical research journals identified as most important (according to impact factor) and from Actas Dermosifiliográficas, and studied the bibliographies to ascertain whether the authors made reference to existing systematic reviews and Cochrane reviews.

Results

Of the 72 clinical trials for which a systematic review was available, 24 (33.3%) cited at least 1 review; reference was made to relevant Cochrane reviews in 15.6% of cases and to non-Cochrane reviews in 32%. In the case of the 24 narrative reviews for which a review was available, 10 (41.7%) cited at least 1 review; Cochrane reviews were cited in 20% and non-Cochrane reviews in 35.3%.In the case of Actas Dermosifiliográficas, very few clinical trials were found and the findings for narrative review articles were similar to those observed for the other journals.

Conclusions

Systematic reviews are not often taken into account by the authors of clinical trials and narrative reviews and this may lead to redundant studies and publications. Authors appear to use Cochrane reviews even less than non-Cochrane reviews and are therefore ignoring one of the main sources of available evidence.

Keywords:
Evidence-based medicine
Bibliometrics
Dermatology
Review literature as topic
Resumen
Introducción y objetivos

Las revisiones sistemáticas son la forma de revisión más exhaustiva y deberían ser consideradas antes de realizar un ensayo clínico o revisión sobre un tema. El objetivo de este estudio es describir la utilización de revisiones sistemáticas en los ensayos clínicos y revisiones narrativas publicadas en dermatología.

Material y métodos

Se diseñó un estudio descriptivo transversal. Se seleccionaron ensayos clínicos aleatorizados y revisiones narrativas de las revistas más relevantes (por factor de impacto) de investigación clínica de dermatología y de Actas Dermosifiliográficas y se evaluó si hacían referencia a revisiones sistemáticas y Cochrane (en caso de existir) en la bibliografía.

Resultados

En el grupo de ensayos clínicos se hacía referencia a alguna de las revisiones sistemáticas existentes en el 33,3% de los artículos (15,6% de las revisiones Cochrane que existían y 32,2% de las no Cochrane sobre el tema). En el grupo de revisiones narrativas alguna de las revisiones sistemáticas existentes eran referenciadas en el 41,7% de los trabajos (20% Cochrane y 35,3% no Cochrane). En Actas Dermosifiliográficas existían muy pocos ensayos clínicos publicados; las revisiones narrativas reproducían lo que se observaba en el resto de revistas.

Conclusiones

Las revisiones sistemáticas son poco tenidas en cuenta en la realización de ensayos clínicos y revisiones narrativas, lo que puede llevar a estudios y publicaciones redundantes. Además las revisiones Cochrane parecen ser incluso menos empleadas, obviando así los autores una de las principales fuentes de evidencia existentes.

Palabras clave:
Medicina basada en evidencia
Bibliometría
Dematología
Literatura de revisión como asunto
Texto completo
Introduction

An accurate account of existing knowledge is essential when planning a new study or compiling information that is already known. Authors or researchers who embark on a new study or review of a topic without searching for and analyzing pertinent systematic reviews run the risk of looking for an answer that is already known. Systematic reviews are the most effective means of identifying and appraising available evidence.1 Traditional or narrative reviews have been found to reach conclusions of little value, or even erroneous conclusions based on personal opinions and experiences, while systematic reviews produce results based on the best available evidence.2

The aim of this study was to analyze the percentage of clinical trial reports and narrative reviews in dermatology that use information from systematic reviews on the subject under investigation.

Material and Methods

We conducted a cross-sectional descriptive study to investigate whether clinical trial reports and narrative reviews in the field of dermatology cite relevant Cochrane and non-Cochrane systematic reviews in the bibliography. We included randomized controlled trial (RCT) reports addressing treatment interventions published in a selection of dermatology journals in 2010 and 2011 and narrative reviews of treatments published in the same group of journals in 2011.

The journals chosen were the British Journal of Dermatology, the Journal of the American Academy of Dermatology, and Archives of Dermatology. The selection was determined by impact factor and a predominant focus on clinical research. Because of its relevance in Spain, we also performed an identical analysis of Actas Dermo-Sifiliográficas.

We collected all publications from the study period that were indexed in PubMed as “clinical trial” or “review” (narrative, i.e., not systematic). The articles retrieved were distributed among the authors, who recorded the following information: type of article, date of publication, citation of a Cochrane or non-Cochrane systematic review in the bibliography, existence in the literature of a Cochrane or non-Cochrane systematic review on the topic under investigation and the date of its publication. We used the Cochrane Library3 (http://www.thecochranelibrary.com) to search for Cochrane systematic reviews and the Centre for Reviews and Dissemination4 (part of the UK National Institute for Health Research) to search for non-Cochrane systematic reviews. Systematic reviews published at least 1 year before the publication of the article being analyzed were considered citable.

In each group, we excluded articles that were not RCT reports or narrative reviews, RCTs and narrative reviews that did not address a treatment, and duplicate publications.

We calculated the percentage of systematic reviews (Cochrane and non-Cochrane) cited in each group of articles, and also calculated separate figures for each type of review.

Before choosing the target journals and the years to be analyzed, we calculated the sample size needed to detect a margin of error of up to 25% with a statistical power of 0.9 and an alpha level of 0.5; the rate of citation of systematic reviews was estimated at 15%. The data used to calculate the number of years and journals to include in the study were taken from a pilot sample of dermatology journals.

The SPSS software package, version 16.0 (IBM), was used for data analysis. The flow of articles throughout the process was charted.

ResultsRandomized Clinical Trials

The flowchart for the inclusion/exclusion of articles reporting RCTs is shown in Fig. 1.

Figure 1.

Flowchart of RCT articles. RCT indicates randomized controlled trial; SR, systematic review.

(0,29MB).

Not counting data for Actas Dermo-sifiliográficas, we retrieved 137 articles, of which 111 (81%; 95% CI, 73%-87%) met the inclusion criteria for the study. Pertinent systematic reviews existed for 72 RCT publications (64.9%; 95% CI, 55.2%-73.7%) and were cited in 24 of these (33.3%; 95% CI, 22.7%-45.4%).

Cochrane reviews existed for 32 publications (28.8%; 95% CI, 20.6%-38.2%) and were cited in 5 cases (15.6%; 95% CI, 5.2%-32.8%). Non-Cochrane systematic reviews existed for 59 articles (53.2%; 95% CI, 42.4%-62.7%) and were cited in 19 of these (32.2%; 95% CI, 20.6%-45.6%).

Three articles were retrieved from PubMed for Actas Dermo-Sifiliográficas and 2 met the inclusion criteria. There were no systematic reviews on the topics addressed by these articles.

Narrative Reviews

The flowchart showing the selection of narrative reviews is shown in Fig. 2.

Figure 2.

Flowchart of NRs. NR indicates narrative review; SR, systematic review.

(0,34MB).

Our search retrieved 123 narrative reviews, not counting articles from Actas Dermo-Sifiliográficas. Of these, 40 (32.5%; 95% CI, 24.1%-41.6%) met the inclusion criteria. Pertinent systematic reviews existed for 24 of the reviews (60%; 95% CI, 43.3%-75.1%) and were cited in 10 of these (41.7%; 95% CI, 22.1%-63.4%).

Cochrane reviews existed for 20 publications (50%; 95% CI, 33.8%-63.2%) and were cited in 4 (20%; 95% CI, 5.7%-43.7%). Non-Cochrane systematic reviews existed for 17 articles (42.5%; 95% CI, 27%-59.1%) and were cited in 6 cases (35.3%; 95% CI, 14.2%-61.7%).

Thirty-four narrative reviews were retrieved in Actas Dermo-Sifiliográicas, and of these 9 (26.5%; 95% CI, 12.9%-44.4%) fulfilled the study inclusion criteria. Citable systematic reviews were identified for 4 articles (44.4%; 95% CI, 13.7%-78.8%). These reviews were cited in 2 cases (50%; 95% CI, 7.6%-93.2%). Pertinent Cochrane reviews existed for 3 articles (33.3%; 95% CI, 7.5%-70.1%) and were cited in 1 of these (33.3%; 95% CI, 0.8%-90.6%). Finally, non-Cochrane reviews existed for 4 articles (44.4%; 95% CI, 13.7%-78.8%) and were cited in 2 of these (50%; 95% CI, 6.8%-93.2%).

The sum of articles with Cochrane and non-Cochrane systematic reviews does not match the total number of articles because more than 1 citable systematic review (of either type) existed for some articles.

Discussion

Systematic reviews were only taken into account by 1 in every 3 RCT reports (33%) published in the top clinical research journals in the field of dermatology. In the case of narrative reviews, the percentage was 41.7%.

The Consolidated Standards of Reporting Trials (CONSORT) statement specifies the minimum information that should be included when reporting the results of a clinical trial. The statement also recommends that the results of a clinical trial be related to results from previous trials and suggests including systematic reviews to facilitate this comparison.5 Nonetheless, studies indicate that adherence to the CONSORT statement and similar reporting guidelines is suboptimal, although more recent articles seem to be faring better.6

Clarke et al.7,8 collected data on clinical trial reports published in a small group of top medical journals in different years (1997, 2001, 2005, and 2009) and showed that very few reports cited systematic reviews. They also found that the information from the systematic reviews was not adequately linked to the results obtained. There was no evidence of an improvement over the years, although in 2009, there did appear to be a greater presence of systematic reviews.7,8

In our study, we distinguished between Cochrane and non-Cochrane reviews and found that the former were cited less often in both RCT reports (15.6% vs 32.2%) and narrative reviews (20% vs 42.5%). An earlier study of systematic reviews reported that non-Cochrane reviews were twice as likely as Cochrane reviews to reach a favorable conclusion on the intervention being studied, a tendency that was attributed to the fact that Cochrane reviewers apply stricter criteria and only report favorable results when there is no room for doubt.9 One study of dermatology reviews found that approximately 40% of Cochrane reviews find sufficient evidence to recommend clinical intervention.10 This percentage is sufficiently high to warrant consideration of these studies. Additionally, while systematic reviews might not find sufficient evidence to answer a clinical question, they do serve to identify research gaps.11 Conducting a systematic review can be a good first step towards identifying exact needs before launching a clinical trial. The results of this trial can then be used to update and improve the systematic review, leading to a cycle of “systematic reviews followed by clinical trials, followed by systematic reviews.”12

Just 2 RCT reports in Actas Dermo-Sifiliográficas met the inclusion criteria in our study and we are therefore unable to draw any conclusions. However, in the case of narrative reviews, the citation pattern appeared to be similar to that observed for the other journals.

One limitation of our study is that we did not assess the quality of the systematic reviews analyzed, and it is therefore possible that some authors decided not to include what they considered to be reviews of suboptimal quality. However, we believe that systematic reviews always provide information, even if only to identify existing literature. Furthermore, Cochrane reviews are underpinned by a complex editorial system that guarantees that the conclusions reported will be relevant. Also, because we did not conduct an exhaustive search, we may have overlooked some systematic reviews. However, even if this were the case, it would not alter our conclusions as the percentage of “citing” articles would be even lower. Finally, it may be questioned whether the journals selected for our study are the most representative journals. We included journals with the highest impact factors and excluded those focusing primarily on basic research. If we accept that impact factor is a marker of journal and publication quality, we can probably assume that the citation rates we report are higher than true rates for dermatology journals in general.

In conclusion, we believe that systematic reviews are not sufficiently taken into account in RCT reports or narrative reviews and this may result in redundant studies and publications. Furthermore, Cochrane reviews appear to be cited even less frequently than non-Cochrane systematic reviews, indicating that authors are ignoring one of the strongest sources of evidence available.

Ethical DisclosuresProtection of humans and animals

The authors declare that no tests were carried out in humans or animals for the purpose of this study.

Confidentiality of data

The authors declare that they have followed their hospital's protocol on the publication of data concerning patients and that all patients included in the study have received sufficient information and have given their written informed consent to participate in the study.

Right to privacy and informed consent

The authors declare that no private patient data appear in this article.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

References
[1]
M. Clarke.
Doing new research? Don’t forget the old.
[2]
S. Ladhani, H.C. Williams.
The management of established postherpetic neuralgia: A comparison of the quality and content of traditional vs systematic reviews.
Br J Dermatol, 139 (1998), pp. 66-72
[3]
The Cochrane Library [accessed 20 May 2013]. Available at: http://www.thecochranelibrary.com/view/0/index.html
[4]
Centre for Reviews and Dissemination (NHS) [accessed 20 May 2013]. Available at: http://www.crd.york.ac.uk/crdweb/
[5]
K.F. Schulz, D.G. Altman, D. Moher.
CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials.
BMJ, 340 (2010), pp. c332
[6]
Z. Samaan, L. Mbuagbaw, D. Kosa, V. Borg Debono, R. Dillenburg, S. Zhang, et al.
A systematic scoping review of adherence to reporting guidelines in health care literature.
J Multidiscip Healthc, 6 (2013), pp. 169-188
[7]
M. Clarke, S. Hopewell, I. Chalmers.
Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting.
[8]
M. Clarke, S. Hopewell, I. Chalmers.
Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: A status report.
J R Soc Med, 100 (2007), pp. 187-190
[9]
A.C. Tricco, J. Tetzlaff, B. Pham, J. Brehaut, D. Moher.
Non-Cochrane vs Cochrane reviews were twice as likely to have positive conclusion statements: Cross-sectional study.
J Clin Epidemiol, 62 (2009), pp. 380-3860
[10]
E.R. Parker, L.M. Schilling, V. Diba, H.C. Williams, R.P. Dellavalle.
What is the point of databases of reviews for dermatology if all they compile is insufficient evidence.
J Am Acad Dermatol, 50 (2004), pp. 635-639
[11]
H.C. Williams, R.P. Dellavalle.
The growth of clinical trials and systematic reviews in informing dermatological patient care.
J Invest Dermatol, 132 (2012), pp. 1008-1017
[12]
A.N. Atallah.
Systematic reviews followed by clinical trials, followed by systematic reviews: This is how the uncertainties in medicine are reduced.
Sao Paulo Med J, 125 (2007), pp. 71-72

Please cite this article as: Conde-Taboada A, Aranegui B, García-Doval I, Dávila-Seijo P, González-Castro U. Empleo de las revisiones sistemáticas en ensayos clínicos y revisiones narrativas en dermatología: ¿se usa la mejor evidencia disponible?. Actas Dermosifiliogr. 2014. 105:295–299.

Copyright © 2013. Elsevier España, S.L. and AEDV
Idiomas
Actas Dermo-Sifiliográficas
Opciones de artículo
Herramientas
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?