Reported quality of randomized controlled trials of physiotherapy interventions has improved over time

doi:10.1016/j.jclinepi.2010.08.009

Journal of Clinical Epidemiology

Volume 64, Issue 6, June 2011, Pages 594-601

https://doi.org/10.1016/j.jclinepi.2010.08.009 Get rights and content

Abstract

Objective

To describe the change with time of the reported methodological quality of randomized controlled trials of physiotherapy interventions.

Study Design and Setting

For all trials of physiotherapy interventions indexed on the Physiotherapy Evidence Database (PEDro), year of publication, and methodological quality scores (11-item PEDro scale and total PEDro score [range, 0–10]) were extracted. The relationship between trial quality and time was evaluated using regression analyses for the PEDro total score and individual quality items. The study was carried out in a university research center.

Results

Data from 10,025 trials published since 1960 were analyzed. The total PEDro score was related to time (year of publication), with the total score increasing by an average of ∼0.6 points each decade between 1960 and 2009. The reported use of eight of the 11 individual items from the PEDro scale (intention-to-treat analysis, concealed allocation, groups similar at baseline, reporting of results of between-group statistical comparisons, point measures and measures of variability reported, subjects randomly allocated to groups, eligibility criteria specified, and blinding of assessors) also improved with time.

Conclusion

The reported methodological quality of randomized controlled trials of physiotherapy interventions has improved over time. Further improvement is still necessary.

Introduction

What is new?

Key finding

The reported quality of randomized controlled trials of physiotherapy interventions has improved over time.

What this adds to what was known?

Because of limitations in the samples and analysis methods of previous studies, trends in the quality of randomized controlled trials in physiotherapy were previously unclear. This study clearly identifies improvements in overall quality and individual quality items in randomized controlled trials in physiotherapy, including intention-to-treat analysis, concealed allocation, groups similar at baseline, reporting of results of between-group statistical comparisons, point measures and measures of variability reported, subjects randomly allocated to groups, eligibility criteria specified, and blinding of assessors.

What is the implication, what should change now?

There is scope for further improvement in the conduct and reporting of randomized trials in physiotherapy. Studies of the literature in a scientific discipline need to be based on large representative samples of that literature.

The first randomized controlled trials of physiotherapy interventions were published around 1930 [1], [2]. Since then, growth in the number of randomized controlled trials of physiotherapy interventions has been exponential [3], [4] and by September 2007, there were over 9,000 trials of physiotherapy interventions published in peer-reviewed journals [4]. Like other areas of health care, the methodological quality of randomized controlled trials in physiotherapy is variable, with 52% of trials being of nominally moderate to high quality [3].

During this period of rapid growth in randomized controlled trials of physiotherapy interventions, there has also been an increased awareness of evidence-based practice, advances in the training of physiotherapists, and advances in the design and reporting of clinical trials. Contemporary physiotherapy aims to combine the best available research evidence with patient preferences and clinical experience to guide clinical practice [5]. This shift to evidence-based practice includes developing knowledge and skills to critically appraise reports of research [5]. Physiotherapy entry-level training has changed from diploma vocational courses to university-based baccalaureate and postbaccalaureate degrees [6], [7], with a growing number of physiotherapists undertaking formal postgraduate research training [8]. The importance of several trial design features has been identified and quantified, including random allocation to groups [9], concealed allocation [9], [10], blinding of subjects, therapists and assessors [10], and intention-to-treat analysis [11]. The importance of reporting these key methodological features in reports of randomized controlled trials was emphasized with the development of the Consolidated Standards of Reporting Trials (CONSORT) Statement [12]. Given these advances in the conduct and reporting of clinical trials, training of physiotherapists, and an evidence-based approach to treatment, one would anticipate that the quality of randomized controlled trials would have improved with time. Preliminary analyses suggest that there may have been an increase in the reported methodological quality of randomized controlled trials of physiotherapy interventions [3], [4], with the average quality score for reports of trials published from 1929 to 1979 being 3.6 of 10 (standard deviation [SD], 1.5) compared with an average score of 5.4 of 10 (SD, 1.6) for reports published from 2005 to 2007 [4].

Two recent investigations concluded that there was little change in the quality of randomized controlled trials in physiotherapy with time [13], [14]. A systematic review of the trends in methodological quality [14] based its conclusion on two reviews of 89 trials [15] and eight trials [16] of physiotherapy interventions. The other investigation was a survey of 200 reports of randomized controlled trials of physiotherapy interventions [13]. These small samples are probably unrepresentative of the published reports of randomized controlled trials of physiotherapy interventions [3], [4]. For other areas of health care, the review concluded that overall quality scores have increased with time (13 of 26 reviews) [14]. An examination of reports of trials indexed on PubMed (predominantly drug and surgical interventions) indicates that reporting of the sequence generation (risk ratio, 1.62; 95% confidence interval [CI], 1.32, 1.97) and allocation concealment (risk ratio, 1.40; 95% CI, 1.11–1.76) improved with time but blinding did not (risk ratio, 0.91; 95% CI, 0.75–1.10) [17].

The present study aims to describe changes in the quality of reporting of randomized controlled trials of physiotherapy interventions over time. Specifically, the study will examine the change in overall quality (i.e., total quality score) and in 11 individual quality features over time. A secondary aim was to examine the impact of the CONSORT Statement on the quality of reports of trials of physiotherapy interventions.

Section snippets

Data source

The PEDro database (www.pedro.org.au) provides an index of randomized trials, systematic reviews, and evidence-based practice guidelines in physiotherapy. Trials are located using sensitive search strategies of seven databases (Allied and Complementary Medicine Database, Cumulative Index to Nursing and Allied Health Literature, Cochrane CENTRAL, Database of Abstracts of Reviews of Effects, Embase, Medline, and PsycINFO) and citation tracking of systematic reviews that are indexed on PEDro.

Results

There were 13,659 records indexed on PEDro as on November 3, 2008. Of these, 11,291 were randomized controlled trials, 10,038 had complete consensus ratings, and 10,025 were published in or after 1960. These trials were published in 1,306 different journals.

Total PEDro score increased with time. The relationship between the total PEDro score and time (year of publication) is illustrated in the bubble plot in Fig. 1. The weighted linear regression accounted for 8.9% of the variance. The equation

Discussion

The main finding is that the methodological quality of randomized controlled trials of physiotherapy interventions has improved over the past 5 decades. On average, the total PEDro score improved by ∼0.6 points each decade. Eight of the 11 items on the PEDro scale were satisfied more often in more recent trials. They were intention-to-treat analysis, concealed allocation, groups similar at baseline, reporting of results of between-group statistical comparisons, point measures and measures of

Acknowledgments

Contributions: A.M.M., R.D.H., C.G.M., C.S., and M.R.E. are directors of the Centre for Evidence-Based Physiotherapy, who are responsible for developing and maintaining the Physiotherapy Evidence Database (PEDro). A.M.M., R.D.H., C.G.M., C.S., and M.R.E. devised the idea for this article. A.M.M. extracted and processed the data. R.D.H. and A.M.M performed the statistical analyses. All authors contributed to the writing of the article. A.M.M. is the guarantor.

Funding: PEDro is funded by a grant

References (33)

A.M. Moseley et al.
Evidence for physiotherapy practice: a survey of the Physiotherapy Evidence Database (PEDro)
Aust J Physiother
(2002)
N.A. de Morton
The PEDro scale is a valid measure of the methodological quality of clinical trials: a demographic study
Aust J Physiother
(2009)
A.P. Verhagen et al.
The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus
J Clin Epidemiol
(1998)
L.G. Macedo et al.
There was evidence of convergent and construct validity of Physiotherapy Evidence Database quality scale for physiotherapy trials
J Clin Epidemiol
(2010)
C. Han et al.
The impact of the CONSORT Statement on reporting of randomized clinical trials in psychiatry
Contemp Clin Trials
(2009)
A. Hróbjartsson et al.
Reporting on blinding in trial protocols and corresponding publications was often inadequate but rarely contradictory
J Clin Epidemiol
(2009)
D. Colebrook
Irradiation and health. A. ultra-violet irradiation of school children
Spec Rep Ser Med Res Counc (G B)
(1929)
J.A. Doull et al.
The effect of irradiation with ultra-violet light on the frequency of attacks of upper respiratory disease (common colds)
Am J Hyg
(1931)
C.G. Maher et al.
A description of the trials, reviews, and practice guidelines indexed in the PEDro database
Phys Ther
(2008)
R. Herbert et al.
Practical evidence-based physiotherapy
(2005)

J. McMeeken

Physiotherapy education in Australia

Phys Ther Rev

(2007)

D. Redenbach et al.

Canadian physiotherapy education: the University of British Columbia example

Phys Ther Rev

(2007)

H.J.C.G. Coury et al.

Profile of the Brazilian physical therapy researcher

Rev Bras Fisioter

(2009)

T.C. Chalmers et al.

Bias in treatment assignment in controlled clinical trials

N Engl J Med

(1983)

K.F. Schulz et al.

Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials

JAMA

(1995)

Y.J. Lee et al.

Analysis of clinical trials by treatment actually received: is it really an option?

Stat Med

(1991)

Cited by (94)

Effects of acupuncture or similar needling therapy on pain, proprioception, balance, and self-reported function in individuals with chronic ankle instability: A systematic review and meta-analysis
2023, Complementary Therapies in Medicine
Acupuncture or similar needling therapy has long been used to improve well-being, but its effectiveness in management of chronic ankle instability (CAI) is unclear. To investigate the efficacy of acupuncture or similar needling therapy on pain, proprioception, balance, and self-reported function in individuals with CAI.
Nine databases (PubMed, Embase, Cochrane Library, Web of Science, EBSCO, PEDro, CNKI, WanFang, and CQVIP) were systematically searched from inception to April 2023. This study included randomized controlled trials involving acupuncture or similar needling therapy as an intervention for individuals with CAI. Data were extracted independently by two assessors using a standardized form. Literature quality and risk bias were assessed by using the PEDro scale.
Twelve trials (n = 571) were found, of which the final meta-analysis was conducted with eight. Different studies employ varying treatments, including specific needle types, techniques, and therapeutic frameworks. Compared to control without acupuncture or similar needling therapy, acupuncture or similar needling intervention resulted in improved pain (WMD 1.33, 95 % CI 0.14–2.52, I²=90 %, p = 0.03), proprioception (active joint position sense, WMD 1.71, 95 % CI 0.95–2.48, I²=0 %, p < 0.0001), balance (SMD 0.54, 95 % CI 0.03–1.04, I²=46 %, p = 0.04), and self-reported function (Cumberland Ankle Instability Tool (WMD 2.92, 95 % CI 0.94–4.90, I²=78 %, p = 0.004); American Orthopedic Foot and Ankle Society (WMD 9.36, 95 % CI 6.57–12.15, I²=0 %, p < 0.001); Foot and Ankle Ability Measure: activities of daily living (WMD 5.09, 95 % CI 1.74–8.44, I²=0 %, p = 0.003)) for individuals with CAI.
The available evidence suggests that acupuncture or similar needling therapy may improve pain, proprioception, balance, and self-reported function in individuals with CAI, but more trials are needed to verify these findings. Furthermore, various needles and techniques using in different studies have resulted in methodologic limitations that should be addressed in the future.
Effects of constraint induced movement therapy in patients with multiple sclerosis: A systematic review
2023, Multiple Sclerosis and Related Disorders
Background: Multiple sclerosis (MS) is a chronic neurodegenerative disorder of the central nervous system (CNS) that commonly affects young and middle-aged adults. Neurodegeneration of the CNS affects its functions such as sensorimotor, autonomic and cognitive functions. Affectation of motor function can result in disability in performance of daily life activities. Thus, effective rehabilitation interventions are needed to help prevent disability in patients with MS. One of these interventions is the constraint induced movement therapy (CIMT). The CIMT is used to improve motor function in patients with stroke and other neurological conditions. Recently, its use in patients with MS is gaining ground. The aim of this study is to carry out a systematic review and meta-analysis to determine from the literature, the effects of CIMT on upper limb function in patients with MS.
Methods: PubMED, Embase, Web of Science (WoS), PEDro, and CENTRAL were searched until October 2022. Randomized controlled trials in patients with MS who were 18 years and above were included. Data on the characteristics of the study participants such as disease duration, type of MS, the mean scores of the outcomes of interest such as motor function and use of the arm in daily activities, and white matter integrity were extracted. Methodological quality and risks of bias of the included studies were assessed using PEDro scale and Cochrane risks of bias tool. The data was analysed using both narrative and quantitative syntheses. In the quantitative synthesis, random effect model meta-analysis of the mean and standard deviation of the scores on the outcomes of interest and the study sample size (for both the CIMT and the control group) post intervention was carried out. In addition, percentage of variation across the studies due to heterogeneity (I²) was considered significant when it is between 50% and 90% at p < 0.05.
Results: Two studies comprising of 4 published articles with good methodological quality were included in the study. The results showed that, CIMT is safe and improved white matter integrity, motor function, muscle strength, dexterity, real-world arm use and biomechanical parameters post intervention. However, although there was a trend towards better improvement in the CIMT group in all the outcomes, there was no statistically significant difference between groups in motor function (SMD=0.44, 95% CI=-0.20 to 1.07, p = 0.18) and quality of movement (SMD=0.96, -1.15 to 3.07, p = 0.37).
Conclusion: CIMT can be used in patients with MS since it is safe as well as effective at improving functional outcomes. However, more studies are required to confirm its safety and effectiveness.
Almost one in five physiotherapy trials excluded people due to lack of language proficiency: A meta-epidemiological study
2022, Journal of Clinical Epidemiology
The objective of the study was to examine the characteristics of randomized controlled trials (RCTs) evaluating physiotherapy interventions for low back pain (LBP) that specified a language-grounded eligibility criterion and the proportion of people being excluded consequently.
This is a meta-epidemiological study of RCTs evaluating at least one type of physiotherapy intervention for treatment or prevention of LBP. Records were retrieved from Physiotherapy Evidence Database (PEDro), LILACS, and SciELO from inception to May 2021. We retrieved metadata of each record from PEDro and extracted from included studies: country of recruitment, language-grounded eligibility criterion, and the number of consequent exclusions (if specified).
This study included 2,555 trials. A language-grounded eligibility criterion was specified in 463 trials (18.1%); the proportion was higher in trials conducted in North America and Europe, published after 2000, investigating cognitive and behavioral interventions, and including large sample size. Of these 463 trials, 75 trials (16.2%) reported a total number of 2,152 people being excluded due to lack of language proficiency, equivalent to 12.5% of randomized participants.
Nearly one in five physiotherapy clinical trials on LBP excludes people based on language proficiency, compromising the evidence to manage LBP in minority populations.
Can Therapeutic Exercises Improve Proprioception in Chronic Ankle Instability? A Systematic Review and Network Meta-analysis
2022, Archives of Physical Medicine and Rehabilitation
To assess exercise therapies that aim to enhance proprioception in individuals with chronic ankle instability (CAI).
Five databases (PubMed, Embase, Cochrane Library, Web of Science, and EBSCO) were searched in October 2021.
Randomized controlled trials involving exercise therapy conducted on individuals with CAI were included.
Data were extracted by 2 independent reviewers using a standardized form. Methodological quality and risk of bias were assessed with the Physiotherapy Evidence Database scale.
The end trial weighted mean difference and standard deviations were analyzed, and the synthetic value for the improvement in error scores of ankle joint position sense in multiple directions was evaluated.
Eleven trials with 333 participants were eligible for inclusion in this systematic review and were included in the network meta-analysis. Foot and ankle muscle strengthening exercise showed the highest probability of being among the best treatments (surface under the cumulative ranking [SUCRA]=74.6%). The next 2 were static balance exercise only (SUCRA=67.9%) and corrective exercise (SUCRA=56.1%). The SUCRA values of proprioceptive exercise, dynamic balance exercise only, aquatic exercise, rehabilitation exercise with brace, mixed static/dynamic balance exercise, and control were at relatively low levels and scored at 49.6%, 48.8%, 47.8%, 47.7%, 44.0%, and 13.5%, respectively.
Foot and ankle muscle strengthening exercise may have a good effect when used to improve joint position sense in individuals with CAI. The more complex balance exercise intervention becomes, the less effective the proprioceptive outcome.
Reporting results in manual therapy clinical trials: A need for improvement
2021, International Journal of Osteopathic Medicine
The number of randomized clinical trials (RCTs) for manual therapy (MT) has increased exponentially in recent years but the quality of reporting is heterogeneous.
To assess the quality of the reporting of results in RCTs manual therapy, both in the text and in the graphs.
Methodological review.
We reviewed a random sample of 120 RCTs in MT published between 2000 and 2020 in indexed journals. We identified the primary outcome for each trial, and evaluated the completeness and correctness of reporting of results in the text and in the graphs.
Forty per cent of the RCTs explicitly identified the primary outcome and 47.5% reported a sample size calculation. In 46.7% of the trials, the reporting of between groups comparisons was complete (including effect size and precision). Only 29.2% used the confidence interval as a measure of precision. Fifty-eight per cent of the trials reported significant differences in the results, and 30.8% reported a value of clinical relevance for at least one variable of the study. Forty-seven per cent reported the primary outcome graphically but only 19.6% of the graphs were self explanatory and 66.1% had problems of visual clarity.
Our findings suggest that the reporting of the results in MT trials is generally incomplete and graphics are often poor. These shortcomings could affect the interpretation of the results and their application in clinical practice. Improvements are needed in the reporting of results in order to advance clinical practice and research in manual therapy.
A systematic review highlights the need to improve the quality and applicability of trials of physical therapy interventions for low back pain
2020, Journal of Clinical Epidemiology
Citation Excerpt :
However, reports of blinding of participants, therapists, and assessors have not improved. The difficulties of blinding participants, therapists, and at times, assessors in RCTs of physical therapy interventions have been acknowledged [24–26]. However, allocation concealment and reporting of intention-to-treat effects are usually straightforward and could be easily implemented [24,25,27].
The objective of this study was to review and assess the methodological quality of randomized controlled trials that test physical therapy interventions for low back pain.
This is a systematic review of trials of physical therapy interventions to prevent or treat low back pain (of any duration or type) in participants of any age indexed on the Physiotherapy Evidence Database (PEDro). Existing PEDro scale ratings were used to evaluate methodological quality.
This review identified 2,215 trials. The majority of trials were for adults (n = 2136, 96.4%), low back pain without specific etiology (n = 1,863, 84.1%), and chronic duration (n = 947, 42.8%). The quality of trials improved over time; however, most were at risk of bias. Less than half of the trials concealed allocation to intervention (n = 813, 36.7%), used intention-to-treat principles (n = 778, 35.1%), and blinded assessors (n = 810, 36.6%), participants (n = 174, 7.9%), and therapists (n = 39, 1.8%). These findings did not vary by the type of therapy.
Most trials that test physical therapy interventions for low back pain have methodological limitations that could bias treatment effect estimates. Greater attention to methodological features, such as allocation concealment and the reporting of intention-to-treat effects, would improve the quality of trials testing physical therapy interventions for low back pain.

View all citing articles on Scopus

View full text

Review ArticleReported quality of randomized controlled trials of physiotherapy interventions has improved over time

Abstract

Objective

Study Design and Setting

Results

Conclusion

Introduction

Section snippets

Data source

Results

Discussion

Acknowledgments

Aust J Physiother

Aust J Physiother

J Clin Epidemiol

J Clin Epidemiol

Contemp Clin Trials

J Clin Epidemiol

Irradiation and health. A. ultra-violet irradiation of school children

Spec Rep Ser Med Res Counc (G B)

The effect of irradiation with ultra-violet light on the frequency of attacks of upper respiratory disease (common colds)

Am J Hyg

A description of the trials, reviews, and practice guidelines indexed in the PEDro database

Phys Ther

Practical evidence-based physiotherapy

Physiotherapy education in Australia

Phys Ther Rev

Canadian physiotherapy education: the University of British Columbia example

Phys Ther Rev

Profile of the Brazilian physical therapy researcher

Rev Bras Fisioter

Bias in treatment assignment in controlled clinical trials

N Engl J Med

Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials

JAMA

Analysis of clinical trials by treatment actually received: is it really an option?

Stat Med

Review Article
Reported quality of randomized controlled trials of physiotherapy interventions has improved over time