Article Text
PDF
Abstract
Introduction Consistent evidence shows pathology services are overused worldwide and that about one-third of testing is unnecessary. Audit and feedback (AF) is effective for improving care but few trials evaluating AF to reduce pathology test requesting in primary care have been conducted. The aim of this trial is to estimate the effectiveness of AF for reducing requests for commonly overused pathology test combinations by high-requesting Australian general practitioners (GPs) compared with no intervention control. A secondary aim is to evaluate which forms of AF are most effective.
Methods and analysis This is a factorial cluster randomised trial conducted in Australian general practice. It uses routinely collected Medicare Benefits Schedule data to identify the study population, apply eligibility criteria, generate the interventions and analyse outcomes. On 12 May 2022, all eligible GPs were simultaneously randomised to either no intervention control or to one of eight intervention groups. GPs allocated to an intervention group received individualised AF on their rate of requesting of pathology test combinations compared with their GP peers. Three separate elements of the AF intervention will be evaluated when outcome data become available on 11 August 2023: (1) invitation to participate in continuing professional development-accredited education on appropriate pathology requesting, (2) provision of cost information on pathology test combinations and (3) format of feedback. The primary outcome is the overall rate of requesting of any of the displayed combinations of pathology tests of GPs over 6 months following intervention delivery. With 3371 clusters, assuming no interaction and similar effects for each intervention, we anticipate over 95% power to detect a difference of 4.4 requests in the mean rate of pathology test combination requests between the control and intervention groups.
Ethics and dissemination Ethics approval was received from the Bond University Human Research Ethics Committee (#JH03507; approved 30 November 2021). The results of this study will be published in a peer-reviewed journal and presented at conferences. Reporting will adhere to Consolidated Standards of Reporting Trials.
Trial registration number ACTRN12622000566730.
- PRIMARY CARE
- Adult pathology
- Quality in health care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This is the first nationwide factorial cluster randomised controlled trial testing variations in the design and delivery of audit and feedback to reduce pathology test combination requesting in primary care.
It includes general practitioners who request commonly overused combinations of pathology tests more than 90% of their peers.
The interventions were designed and refined with input from practising general practitioners prior to evaluation in the trial to optimise their acceptability and potential usefulness.
The trial uses routinely collected administrative data to identify the study population, apply eligibility criteria, generate individualised feedback and analyse trial outcomes.
A limitation is that volume not appropriateness of pathology test requesting is evaluated.
Introduction
Consistent evidence suggests there is overuse of pathology services worldwide and that approximately 30% of testing is unnecessary.1 In Australia, the volume and cost of pathology services has increased dramatically over the last decade, with over 150 million pathology services costing Medicare US$3.6 billion in 2020–2021 alone, with the majority of requests made in general practice.2 On top of the increased costs and time spent by practitioners, patients and laboratory staff, unnecessary pathology testing increases the likelihood of false positive results which can lead to unnecessary additional tests, incorrect diagnoses and unnecessary treatment with associated risk of patient harm.
Audit and feedback, which involves the provision of clinical performance data to healthcare providers with the aim of improving quality of care,3 represents a potentially low cost, scalable and sustainable intervention for reducing overuse of pathology services. To our knowledge, only three randomised trials have evaluated audit and feedback interventions for reducing overuse of pathology test requesting by general practitioners (GPs) with mixed effects.4–6
Baker et al found providing practice level (and, where available, individualised) audit and feedback on requesting of five pathology tests (thyroid function tests (TFTs), rheumatoid factor, urine culture, serum lipids and viscosity) at 3-month intervals over 12 months accompanied by guidelines to GPs in 33 practices in the UK did not significantly alter test requesting compared with control.5 However, Verstappen et al showed a multicomponent intervention comprising individualised peer comparison audit and feedback on pathology requesting for various conditions (eg, cardiovascular disease/hypertension, upper and lower abdominal complaints, chronic obstructive pulmonary disease/asthma, fatigue, degenerative joint complaints) provided to GPs from 26 practices in the Netherlands at 2-month intervals over 6 months, accompanied by guidelines and three face-to-face small group meetings focused on identifying barriers to change and developing action plans, led to a 9% relative reduction in the total number of tests requested per practitioner over 6 months compared with control.4
Thomas et al showed providing practice-level audit and feedback on the requesting rates of nine pathology tests (autoantibody screen, carbohydrate antigen-125, carcino embryonic antigen, ferritin, follicle-stimulating hormone, Helicobacter pylori serum, IgE, thyroid-stimulating hormone (TSH), vitamin B12) by GPs from 85 practices in Scotland at 3-month intervals over 12 months, accompanied by brief educational reminder messages added to pathology test result reports, led to a 22% relative reduction in request rates compared with control.6 Audit and feedback alone led to greater reductions in request rates compared with reminders alone although the model-based analyses suggested similar effects (OR for feedback=0.87, 95% CI 0.81 to 0.94, OR for reminders=0.89, 95% CI 0.83 to 0.93).6
While the latter two studies report benefit with audit and feedback, the approaches are resource intensive (ie, feedback provided on multiple occasions, accompanied by small group educational meetings/reminder messages on targeted test reports) and may not be feasible to implement at scale in many jurisdictions.4 6
The current study builds on our previous successful factorial cluster randomised controlled trial of audit and feedback on musculoskeletal diagnostic imaging in Australian general practice.7 It showed individualised audit and feedback provided to GPs known to request musculoskeletal imaging at much higher rates than their peers significantly decreased their requesting rate over 12 months compared with no intervention. It also showed 2 rounds of feedback (vs 1) and an enhanced visual display directing GPs attention to ordering rates greater than 80% of peer rates (vs standard display) led to further modest statistically significant reductions in request rates.
The primary objective of this trial is to estimate the effectiveness of individualised audit and feedback for reducing overall requests for 10 commonly overused combinations of pathology tests by high-requesting GPs in Australia compared with no intervention control. This includes requests for any combination of two or three pathology tests for iron studies, TSH, TFT, vitamin D and vitamin B12. A secondary objective is to evaluate which forms of audit and feedback are most effective in reducing requests for the pathology test combinations and to estimate their effects.
This trial is a collaboration between the Australian Government Department of Health and Aged Care, and Wiser Healthcare, a collaboration of researchers investigating the causes of, and solutions to, overtesting, overdiagnosis and overtreatment in healthcare.8
Methods
Trial design
This is a 9-arm 2×2×2 factorial cluster randomised controlled trial testing variations in the design and delivery of audit and feedback for reducing overall requests for 10 commonly overused combinations of pathology tests in Australian general practice. On 12 May 2022, clusters of general practices based on geographical location with at least one GP who was in the top 10% of requesters for 10 targeted combinations of pathology tests and for at least 2 of the individual pathology test combinations within the 24-month period from 1 July 2019 to 30 June 2021, were simultaneously randomised either to no intervention control or to 1 of 8 individualised written audit and feedback intervention groups.
Within those allocated to audit and feedback, GPs were first randomly allocated to receive an invitation to continuing professional development (CPD)-accredited education (yes vs no) (factor 1), then provision of cost information about pathology test combinations (yes vs no) (factor 2) and then feedback format (pamphlet vs letter) (factor 3).
The protocol received ethics approval from the Bond University Human Research Ethics Committee and the trial was prospectively registered on the Australian New Zealand Clinical Trials Registry (ACTRN12622000566730).
Setting
General practices and included GPs, located in any state or territory of Australia.
Eligibility and recruitment
Inclusion criteria
We included GPs practising in Australia who were (1) in the top 10% of GP referrers for the 10 targeted combinations of pathology tests overall and (2) in the top 10% of GP referrers for at least two of the individual pathology test combinations within the 24-month period from 1 July 2019 to 30 June 2021. The 10 targeted combinations of pathology tests are: (1) iron studies, TSH and vitamin D, (2) iron studies, vitamin D and vitamin B12, (3) iron studies, TSH and vitamin B12, (4) iron studies, TFT and vitamin B12, (5) iron studies, TFT and vitamin D, (6) TSH, vitamin D and vitamin B12, (7) iron studies and vitamin D, (8) iron studies and vitamin B12, (9) iron studies and TFT and (10) TSH and vitamin D. These pathology test combinations were selected for inclusion in the trial in consultation with stakeholders and the Department of Health and Aged Care and reflect commonly overused combinations of pathology tests in Australian general practice. Only pathology tests which originated with a GP request, were rendered by a pathologist and were claimed from the Australian Medicare Benefits Schedule (MBS) (ie, cost covered by the public health system) were included. Pathology tests that were initiated by a pathologist or were not claimed on the MBS were not considered. Under the MBS, vitamin B12 may be requested as a serum (MBS item 66838) or quantification (MBS item 66839). While GPs may request either of these MBS items, pathologists can choose which item to use based on their professional judgement. As such, a request for either vitamin B12 item was coded as a request for vitamin B12 for the purposes of this trial.
Exclusion criteria
We excluded GPs who: (1) did not request the targeted pathology test combinations within the nominated 24-month period from 1 July 2019 to 30 June 2021; (2) had less than 1000 category 1 consultations either between 1 July 2019 and 30 June 2020 or between 1 July 2020 and 30 June 2021. A category 1 consultation refers to a professional attendance by a GP for which benefits are paid under the Australian MBS; (3) participated in qualitative interviews with members of the research team to user test the interventions; (4) were involved in an Australian Government Department of Health and Aged Care compliance activity within the past 12 months and (5) had their primary practice address in a remote or very remote geographical area, as determined by the Modified Monash Model (MMM) classification 6–7, or had a hospital as their primary practice address.
Randomisation and allocation concealment
Clusters of general practices based on geographical location, with at least one eligible GP, were simultaneously randomised either to no intervention control or to one of eight intervention groups at baseline on 12 May 2022. The randomisation sequence was generated using a computer-generated randomisation algorithm in the statistical programme R Studio. Randomisation of clusters was stratified by geographic region (urban, regional/rural-remote: MMM 1, MMM 2–5) to ensure a balance of practices by geographical region across control and intervention groups.
Blinding
Trial participants (ie, GPs) were not blinded to group allocation but the risk of performance bias is considered to be minimal as GPs were not aware of the variations of audit and feedback being tested nor the outcome measures and analytical approach. The statistical analysis plan has been developed by the trial statistician blinded to group allocation.9 Analyses will be independently conducted by two statisticians using randomly shuffled group allocations. Real allocations will only be revealed once analyses are completed and agreement between the two statisticians is reached.
Interventions
GPs were allocated to a no intervention control group or one of eight intervention groups.
GPs allocated to an intervention group received individualised written audit and feedback on their requesting of pathology test combinations from the Chief Medical Officer of Australia. The targeted combinations of pathology tests are: (1) iron studies, TSH and vitamin D, (2) iron studies, vitamin D and vitamin B12, (3) iron studies, TSH and vitamin B12, (4) iron studies, TFT and vitamin B12, (5) iron studies, TFT and vitamin D, (6) TSH, vitamin D and vitamin B12, (7) iron studies and vitamin D, (8) iron studies and vitamin B12, (9) iron studies and TFT and (10) TSH and vitamin D.
The feedback was delivered by postal mail and presented the recipients’ rate of requesting of a minimum of two and maximum of three targeted pathology test combinations for which they were above the 90th percentile compared with the median request rate of their GP peers in a similar geographic region, determined using the MMM. It described the benefits of reducing unnecessary pathology testing (eg, reducing the potential for false positive results) and invited recipients to access best practice resources on a website, reflect on their requesting data and limit their future pathology requesting to only those clinical situations where it had a reasonable likelihood of altering patient management. The feedback provided to all intervention groups was consistent in content and length other than three separate elements of the audit and feedback intervention evaluated in the trial: (1) invitation to participate in CPD-accredited education aimed at improving pathology requesting (yes vs no); (2) provision of cost information on pathology test combinations (yes vs no) and (3) format of feedback (pamphlet vs letter) (online supplemental additional file 1).
Supplemental material
Factor 1: invitation to participate in CPD-accredited education (yes vs no)
Participants randomised to receive an invitation to participate in CPD-accredited education received a weblink and QR code to register and attend a 1-hour online case-based webinar conducted by GPs and aimed at improving appropriate pathology requesting for selected tests (participants earned two CPD points) and to complete a self-directed review of their pathology requesting (participants earned 40 CPD points). The webinar was conducted 1 month following feedback delivery. A previous trial has shown that providing small group face-to-face education to primary care physicians with audit and feedback improves pathology requesting.4 To our knowledge, no studies have investigated the combination of audit and feedback with a scalable online educational webinar aimed at improving knowledge about appropriate pathology requesting compared with audit and feedback alone.
Factor 2: provision of cost information on pathology test combinations (yes vs no)
Participants randomised to receive costing information received data on the cost per displayed pathology test combination as well as the cost of an individual iron studies test and the total cost of all iron studies conducted in Australia over the 2-year period from 1 July 2019 to 30 June 2021. A hyperlink to a website containing the individual and cumulative costs of pathology tests was also provided. While cost information was included with requesting data, the design of the feedback emphasised use of clinical guidelines to inform appropriate pathology test requesting. Previous trials have shown that combining information about the cost of laboratory tests with performance feedback to physicians in hospital and outpatient medical settings can reduce request rates, although these studies included small numbers of physicians and clusters.10 11
Factor 3: feedback format (pamphlet vs letter)
Participants randomised to receive the pamphlet format received the same content as per the standard feedback letter, except in a booklet format with coloured subheadings and text boxes. A more visually appealing feedback format was hypothesised to increase recipients’ interest and engagement with the feedback compared with the standard feedback format.12
The interventions were designed and refined in user testing with practising GPs prior to use in the trial.
Data collection
The trial uses routinely collected Australian MBS administrative data to identify the study population, apply eligibility criteria, generate individualised feedback for the interventions and analyse trial outcomes. The MBS administrative data records the details of all claims made to Medicare. For example, it includes all pathology test requests that are rendered by a pathologist and claimed from Medicare, including details of the patient, requesting practitioner, pathologist, date of request, date of service, and the items requested and rendered. The Australian Government Department of Health and Aged Care will extract the relevant pathology test requesting data for GPs in the trial.
Trial outcomes
The primary outcome is the overall rate of requesting of any of the displayed combinations of pathology tests (listed below) by each GP per 1000 category 1 consultations over the 6 months following intervention delivery, rendered by a pathologist and assessed using MBS data. Targeted test pathology combinations are: (1) iron studies, TSH and vitamin D, (2) iron studies, vitamin D and vitamin B12, (3) iron studies, TSH and vitamin B12, (4) iron studies, TFT and vitamin B12, (5) iron studies, TFT and vitamin D, (6) TSH, vitamin D and vitamin B12, (7) iron studies and vitamin D, (8) iron studies and vitamin B12, (9) iron studies and TFT and (10) TSH and vitamin D. Secondary outcomes include (1) the overall requesting rate of any of the displayed pathology test combinations at other time points, (2) the requesting rates of displayed individual pathology test combinations, (3) the requesting rates of individual pathology tests (ie, iron studies, TSH, vitamin D, vitamin B12, TFT, ferritin), (4) the overall requesting rate of any of the 10 aforementioned targeted pathology test combinations, (5) the requesting rate of any of the pathology test combinations that were not displayed in recipients’ feedback and (6) the estimated number of requests for the targeted pathology test combinations saved as a result of any feedback intervention compared with control over 6 and 12 months (table 1). Rate of requests will be expressed per 1000 category 1 consultations rendered by a pathologist and assessed using MBS as the data source.
Outcome measures
The following baseline data will also be collected: age, sex, geographical location of primary practice address (metropolitan vs other), state or territory of Australia, years practising as GP, total category 1 patient consultations provided during the baseline period (12 months prior to intervention delivery), total number of requests overall for the displayed pathology test combinations, and rates of pathology test combination requests for primary and secondary outcomes at baseline.
Analysis
The main analysis will consist of comparing the overall rate of pathology test combination requests between the control and all eight intervention groups combined. Data will be aggregated at the GP level and analysed using generalised linear regression. The dependent variable for the regression will, therefore, be the individual pathology request rate of each GP.
The analysis will be performed using multilevel mixed effect generalised linear regression model adjusted for the baseline rate of pathology requests of each GP as well as remoteness and years of practice. To remove skewness and potential heteroscedasticity, we will apply a natural log-transformation to the rate (dependent variable) as well as to the baseline rate included as a covariate. Resulting estimates and CIs will be back transformed to the original scale. Clustering of GPs by area-specific clusters will be accounted for by including a random intercept by statistical area level 1 classification. The effect of the intervention will be estimated as the mean difference in the rate of pathology test combination requests between the intervention and control group together with its 95% CI.
Using the same regression model, we will also estimate the differential effect of separate elements of the intervention: (1) invitation for CPD-accredited education (yes vs no); (2) provision of cost information on pathology test combinations (yes vs no) and (3) format of feedback (pamphlet vs letter). Bonferroni correction will be used to account for multiplicity. We will also test for an interaction between the three elements of the intervention. A similar approach will be used to analyse the secondary outcomes.
Additional models with additional baseline covariates will be considered together with a limited number of subgroup analyses to identify potential differences in intervention effects for factors that have been prespecified in the statistical analysis plan.9 We expect the MBS data to capture all services rendered during the study period and will therefore assume no missing data. Analyses will be conducted using R or Stata software.
Power
The primary endpoint is the overall rate of requests of any of the displayed pathology test combinations for each GP per 1000 category 1 consultations measured over 6 months from intervention delivery. A total of 5964 GPs from 3371 area-specific clusters were identified as eligible for inclusion in the study. This sample size provides over 95% power to detect a 10% rate reduction (ie, a difference of 4.4 requests per 1000 category 1 consultations) in the mean rate of targeted pathology test combination requests between intervention and control, assuming 8:1 randomisation ratio between intervention and control groups, mean (SD) baseline pathology request rate of 44.4 (17.4), variable cluster size (mean cluster size is 1.7, range 1 to 11), intracluster correlation of 0.2 and a two-sided type-I error rate of 5%.
This sample size also provides over 85% power to detect a 5% rate reduction (ie, a difference of 2.2 requests per 1000 consultations) in the mean rate of pathology test combination requests between (1) GPs invited to CPD-accredited education versus not, (2) GPs receiving pathology test cost information versus not and (3) GPs receiving pamphlet format vs those receiving letter format, assuming a two-sided type-I error rate of 1.67% to control for multiplicity.
Patient and public involvement
Wiser Healthcare has a consumer advisory panel which includes members from the peak national health consumer organisation in Australia as well as state-based health consumer organisations. This advisory panel has operated for over 6 years providing high-level advice on Wiser Healthcare research projects. Patients and/or the public were not directly involved in the design of the interventions or the trial design, however, the Wiser Healthcare Consumer Panel will advise on, and assist with, dissemination of the study findings through their networks.
Trial status
The trial began on 12 May 2022 with 5964 GPs from 3371 clusters of general practices randomly allocated on a single occasion to 1 of 8 intervention arms and one control arm. The trial is ongoing. The relevant pathology test requesting outcome data will be available for extraction from the Australian MBS administrative database for statistical analysis on 11 August 2023.
Ethics and dissemination
Ethics approval for this trial was obtained from the Bond University Human Research Ethics Committee (#JH03507; approved 30 November 2021). The investigators will ensure the trial is conducted in compliance with this protocol and the Australian National Statement on Ethical Conduct in Human Research.13 Any modification to the protocol will be approved by the Bond University Human Research Ethics Committee prior to implementation. A waiver for participant consent was approved by the Bond University Human Research Ethics Committee. This was on the basis that involvement in the research carries no more than low risk to participants, the benefits from the research justify any risk of harm associated with not seeking consent and it was impractical to obtain consent, and there are robust measures in place to protect participants’ privacy and maintain the confidentiality of data.
The investigators will be responsible for ensuring the results of the trial are published in a peer-reviewed journal and presented at conferences within a reasonable time frame after conclusion of the trial. The results from the trial will be published regardless of the outcome. Reporting of this trial will adhere to the relevant, and most up to date, Consolidated Standards of Reporting Trials statements at the time of submission.14 This protocol (V.2, 14 September 2022) adheres to the Standard Protocol Items for Randomised Trials (SPIRIT) statement (online supplemental additional files 2 and 3).15
Supplemental material
Ethics statements
Patient consent for publication
Acknowledgments
The Australian Government Department of Health and Aged Care and Wiser Healthcare.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @rthomasEBP
Contributors DAO'C led the development of the protocol and all aspects of the study design. DAO'C, DS and AE designed the interventions with input from PG, RB, KM and RT. AG led the design of the statistical analysis plan with input from DAO'C, DS and AE. DAO'C and PG prepared the ethics submission. DAO'C registered the trial. All authors read and approved the final manuscript.
Funding This study is funded by the Australian Government Department of Health and Aged Care and supported by an Australian National Health and Medical Research Council (NHMRC) Centres of Research Excellence Grant on ‘Wiser healthcare: better value care for all Australians' (APP2006545). PG, KM and RB are supported by Australian NHMRC Investigator Fellowships.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.