Elsevier

Journal of Dentistry

Volume 38, Issue 9, September 2010, Pages 713-721
Journal of Dentistry

An assessment of quality characteristics of randomised control trials published in dental journals

https://doi.org/10.1016/j.jdent.2010.05.014Get rights and content

Abstract

Objectives

The purpose of this study was to investigate the quality of reporting of randomised clinical trials (RCTs) published in dental specialty journals.

Methods

The journals possessing the highest impact factor (2008 data) in the six major dental specialties were included in the study. The contents of the 24 most recent issues of each journal were hand-searched and research articles identified as randomised controlled trials (RCTs) were selected. Quality evaluation was performed using the modified Consolidated Standards of Reporting Trials (CONSORT) statement checklist. The data were analysed using descriptive statistics followed by univariate and multivariate examination of statistical associations (α = 0.05).

Results

Ninety-five RCTs were identified with generally suboptimal scores on quality reporting on key CONSORT areas. Significant differences were found among journals with the Journal of Clinical Periodontology achieving the highest score, followed by the American Journal of Orthodontics and Dentofacial Orthopedics. There was a positive association between quality score and number of authors, involvement of statistician/epidemiologist, and multicentre trials.

Conclusions

The quality scores of RCTs in major dental journals are considered suboptimal in key CONSORT areas. This receives critical importance considering that improved quality of RCTs is a fundamental prerequisite for improved dental care.

Introduction

Dental research aims at evaluating patient interventions with the objective to reach valid conclusions about current and future treatment modalities. The trustworthiness of the conclusions depends greatly on what is called “internal validity” which pertains to the qualitative characteristics of the design and methodology of the particular trial. An experiment conducted following appropriate methodology is more likely to give results that are considered reliable and may positively impact clinical practice.[1], [2]

It has been reported that lack of a number of factors including control, randomisation, allocation concealment, blinding, and accounting for loss to follow-up in trials, may introduce bias and thus render the study results invalid.[1], [3] Among the various study designs the randomised controlled trial is considered the design that has the potential to provide the highest quality evidence,1 however, there is substantial evidence in the biomedical literature that RCTs’ quality is suboptimal.[4], [5], [6], [7]

In an effort to standardise and guide researchers in conducting and reporting clinical trials the Consolidated Standards of Reporting Trials (CONSORT) guidelines have been developed which consist of 25 items that cover all key aspects of clinical trials and set standards on how to design, conduct, analyse and report such studies.1 Numerous studies in the past have assessed quality of randomised controlled trials using various methods,[3], [4], [8], [9], [10], [11] and recently studies have evaluated the RCTs’ quality using the CONSORT guidelines.[6], [7]

According to the CONSORT guidelines some of the important features that experimental design and reporting must account for are:

  • (a)

    Sample size calculation and power analysis, since adequate sample size and increased power reduce the chance of type II errors (false negative results).[11], [12], [13], [14]

  • (b)

    Randomisation, which facilitates a fair comparison of treatments and reduces bias by guarding against participants being allocated treatment according to some systematic arrangement or by the judgement of the investigator.15

  • (c)

    Blinding; in the absence of which the results favour the new treatment compared to the standard therapy.[16], [17], [18], [19]

  • (d)

    Reporting of effect size, confidence intervals, and statistical significance (p-values).[20], [21], [22], [23], [24], [25], [26], [27]

  • (e)

    Subgroup analyses; as more data are analysed there is an increased risk of finding “positive” results by chance and the danger that these results may be selectively emphasised.[28], [29], [30], [31], [32]

  • (f)

    Confounding/stratification (more applicable to epidemiology studies). Associations found in trials may be attributed to true effect, chance, bias, or confounding.33 Confounding, unlike bias, can be evaluated quantitatively and controlled for during the design of the trial by assuring equal distribution of risk factors and during the analysis of the study through stratification and multivariate analysis.34

A small number of studies have reviewed quality reporting of randomised controlled trials in Periodontics,[35], [36] Prosthodontics,[37], [38], [39] Implantology,40 Orthodontics,41 and Dentistry in general.42 However, there is a lack of studies comparatively evaluating the quality of randomised controlled trials in dental research using the CONSORT guidelines. Clinical trials are important methods for evaluating treatment modalities and their impact on current and future clinical practice is great. It is essential that clinical trials are properly conducted in order for their results to be valid, and it is essential for the dental research audience to know the level of quality of the published clinical trials.

In lieu of the above, the objective of this study was to evaluate the quality of randomised clinical trials using the CONSORT guidelines in the dental journals with the highest impact factor, in six key specialty areas. Additionally, the aim was to determine whether specific study characteristics such as dental subspecialty, country of origin, number of authors, single or multicentre involvement, significance of results and ethics committee approval were related to reporting quality.

Section snippets

Materials and methods

The following six dental journals, which presented the highest impact factor for each of dental specialty in 2009, were included in the study:

  • 1.

    American Journal of Orthodontics and Dentofacial Orthopedics (AJODO).

  • 2.

    International Journal of Prosthetic Dentistry (IJPD).

  • 3.

    Journal of Clinical Periodontology (JCP).

  • 4.

    Journal of Endodontics (JOE).

  • 5.

    Journal of Oral and Maxillofacial Surgery (JOMS).

  • 6.

    Journal of Pediatric Dentistry (JPD).

The content of the most recent 24 issues per journal published up to July 2009

Results

The results of this study are summarised in Table 1, Table 2, Table 3, Table 4.

Table 1 displays the characteristics of all 95 dental randomised controlled trials by dental journal, country or origin, number of authors, involvement of statistician/epidemiologist, single or multicentre study, ethics committee approval and statistical significance of main finding. The number of articles (N) and corresponding frequencies are given per characteristic.

Table 2 presents the overall modified CONSORT

Discussion

The results from RCTs provide the highest quality of evidence and therefore it is of paramount importance to secure that they are designed, conducted and reported by adhering to the strictest standards. The validity of the RCTs result would influence decision making in today's clinical practice and in the future since RCTs are the backbone of systematic reviews.

Overall, the results of this study showed a quality score in a percentage scale ranging from 56.1% to 69% indicating a rather low

References (56)

  • P. Juni et al.

    Systematic reviews in health care: assessing the quality of controlled clinical trials

    British Medical Journal

    (2001)
  • K.F. Schulz et al.

    Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals

    Journal of the American Medical Association

    (1994)
  • S.P. Balasubramanian et al.

    Standards of reporting of randomized controlled trials in general surgery: can we do better?

    Annals of Surgery

    (2006)
  • D. Moher et al.

    Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation

    Journal of the American Medical Association

    (2001)
  • C.A. Junker

    Adherence to published standards of reporting: a comparison of placebo-controlled trials published in English or German

    Journal of the American Medical Association

    (1998)
  • C.Y. Ko et al.

    Reporting randomized, controlled trials: where quality of reporting may be improved

    Diseases of the Colon and Rectum

    (2002)
  • D. Moher et al.

    Statistical power, sample size, and their reporting in randomized controlled trials

    Journal of the American Medical Association

    (1994)
  • D.G. Altman et al.

    Absence of evidence is not evidence of absence

    British Medical Journal

    (1995)
  • J.A. Freiman et al.

    The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 “negative” trials

    New England Journal of Medicine

    (1978)
  • T.C. Chalmers et al.

    Meta-analysis of clinical trials as a scientific discipline. I: control of bias and comparison with large co-operative trials

    Statistics in Medicine

    (1987)
  • J.H. Noseworthy et al.

    The impact of blinding on the results of a randomized, placebo controlled multiple sclerosis clinical trial

    Neurology

    (1994)
  • D.A. Grimes

    Randomized controlled trials: “it ain’t necessarily so

    Obstetrics and Gynecology

    (1991)
  • K.F. Schulz et al.

    Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials

    Journal of the American Medical Association

    (1995)
  • K.J. Rothman

    A show of confidence

    New England Journal of Medicine

    (1978)
  • K.J. Rothman

    Significance questing

    Annals of Internal Medicine

    (1986)
  • M.J. Gardner et al.

    Confidence intervals rather than p values: estimation rather than hypothesis testing

    British Medical Journal (Clinical Research Education)

    (1986)
  • J.C. Bailar et al.

    Guidelines for statistical reporting in articles for medical journals. Amplifications and explanations

    Annals of Internal Medicine

    (1988)
  • M.L. Barnett et al.

    Tyranny of the p-value: the conflict between statistical significance and common sense

    Journal of Dental Research

    (1997)
  • Cited by (78)

    • CONSORT Compliance in Randomized Clinical Trials of Regenerative Endodontic Treatments of Necrotic Immature Teeth: A Scoping Review

      2021, Journal of Endodontics
      Citation Excerpt :

      This type of failure in the description of allocation concealment was identified in 78% of the RCTs in journals in the field of dentistry. The present review detected this flaw in 80% of the studies included48. In pulp regeneration studies, the blinding steps can be performed at least with the patient and his or her guardian and with the outcome assessor because it may be impossible to the blinding of the operator in this type of endodontic intervention.

    View all citing articles on Scopus
    View full text