Article Text

Download PDFPDF

Communication
Registry randomised trials: a methodological perspective
  1. Dorota A Doherty1,2,
  2. Steven Y C Tong3,4,
  3. Jennifer Reilly5,6,
  4. Jane Shrapnel7,
  5. Stephen McDonald8,9,
  6. Susannah Ahern10,
  7. Ian Harris11,
  8. Charmaine S Tam12,
  9. Angela L Brennan10,
  10. Carol Hodgson10,
  11. Leonie Wilcox13,
  12. Anitha Balagurunathan14,
  13. Belinda E Butcher15,16,
  14. Christopher M Reid10,17
  1. 1UWA Medical School, The University of Western Australia, Perth, Western Australia, Australia
  2. 2Biostatistics and Research Design Unit, King Edward Memorial Hospital for Women Perth, Subiaco, Western Australia, Australia
  3. 3The Peter Doherty Institute for Infection and Immunity, Melbourne, Victoria, Australia
  4. 4Menzies School of Health Research, Casuarina, Nothern Territory, Australia
  5. 5Anaesthesiology & Perioperative Medicine, Alfred Hospital, Melbourne, Victoria, Australia
  6. 6Department of Anaesthesia and Perioperative Medicine, Monash University, Melbourne, Victoria, Australia
  7. 7The Sydney Children's Hospitals Network, Westmead, New South Wales, Australia
  8. 8Central Northern Adelaide Dialysis, The University of Adelaide, Adelaide, South Australia, Australia
  9. 9Australia and New Zealand Dialysis and Transplant Registry, South Australian Health and Medical Research Institute Limited, Adelaide, South Australia, Australia
  10. 10School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
  11. 11Ingham Institute for Applied Medical Research, South Western Sydney Clinical School, UNSW, Sydney, New South Wales, Australia
  12. 12Northern Clinical School, Centre for Translational Data Science, The University of Sydney, Sydney, New South Wales, Australia
  13. 13Australasian Bone Marrow Transplant Recipient Registry, St Vincent's Health Australia Ltd, Sydney, New South Wales, Australia
  14. 14Epidemiology, Alfred Health, Melbourne, Victoria, Australia
  15. 15WriteSource Medical Pty Ltd, Lane Cove, New South Wales, Australia
  16. 16University of New South Wales Faculty of Medicine, Sydney, New South Wales, Australia
  17. 17School of Population Health, Curtin University, Perth, Western Australia, Australia
  1. Correspondence to Professor Christopher M Reid; christopher.reid{at}curtin.edu.au

Abstract

Registry randomised clinical trials (RRCTs) have the potential to provide pragmatic answers to important clinical questions. RRCTs can be embedded into large population-based registries or smaller single site registries to provide timely answers at a reduced cost compared with traditional randomised controlled trials. RRCTs can take a number of forms in addition to the traditional individual-level randomised trial, including parallel group trials, platform or adaptive trials, cluster randomised trials and cluster randomised stepped-wedge trials. From an implementation perspective, initially it is advantageous to embed RRCT into well-established registries as these have typically already overcome any issues with end point validation and adjudication. With advances in data linkage and data quality, RRCTs can play an important role in answering clinical questions in a pragmatic, cost-effective way.

  • statistics & research methods
  • public health
  • biotechnology & bioinformatics
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

In Australia, clinical quality registries are encouraged by the Australian Commission on Safety and Quality in Healthcare to identify benchmarks and variation in clinical outcomes, feeding back information to healthcare providers, patients and government to inform clinical practice.1 Clinical quality registries have matured over the past two decades and guidance for their establishment in Australia now aligns with the Framework for Australian Clinical Quality Registries (2014).1 In early 2021, the Australian Government released a national strategy for clinical quality registries and virtual registries.2 Outside of the quality framework, clinical registries may also be set up to collect safety data, disease or procedure information and to measure translation of evidence-based medicine into practice.

Unlike Australia, a number of countries have well-established clinical registries and, for more than a decade, have developed the capability to undertake embedded randomised trials across a variety of clinical disciplines.3–6 A well-conducted scoping review identified 17 published trials using disease, procedure or health services registries.7 One of the early demonstrations of the registry randomised clinical trials (RRCTs) was the Thrombus Aspiration in ST-Elevation Myocardial Infarction (TASTE) rial tundertaken in the SWEDEHEART clinical registry demonstrating no benefit of thrombus aspiration prior to percutaneous coronary intervention for improving clinical outcomes.8 Heralded as the ‘next disruptive technology’ for undertaking randomised trials,9 the SWEDEHEART registry has continued to perform a number of important comparative effectiveness trials and proposing international registry-based randomised trials.

This review considers the benefits of RRCT, the types of questions they can answer and some practical tips on how to successfully embed registry randomised trials into the Australian healthcare setting. It is based on a series of workshops held by the Australian Clinical Trials Alliance in May 2020. A glossary of terms used throughout is provided as table 1.

Table 1

Glossary of terms

Development of clinical quality registries in Australia

Registries may have a large and broad target population, established to monitor high-level activity and outcomes on a population basis; or may have a much smaller reach (eg, a single hospital, or several hospitals within a single state, or a niche area of investigation such as a disease, a treatment or a device), but with much deeper data capture. Clinical registries allow collection of ‘real-world’ data from patients in a clinical setting, many of whom would be excluded from randomised clinical trials.10 There are six pillars underpinning clinical quality registries (see box 1).

Box 1

Six pillars underpinning clinical quality registries in Australia

1. Patient-centred healthcare

Registries can help to identify variability in patient-reported outcomes; support clinicians to tailor care to individual needs and preferences; support equity of healthcare. Datasets should therefore contain a combination of clinician and patient-derived data; and should have clinician oversight.

2. Improved clinical practice care and patient outcomes

Datasets should have mechanisms for ‘benchmarking’ where clinicians, health service and other stakeholders are provided with feedback on their care provision ‘benchmarking’.

3. Quality, efficiency and cost-effectiveness

Improve the quality and efficiency of data collection. Improve governance and allow data sharing: ‘collect one, use multiple times principle’. This requires data linkage, where possible, to reduce burden. Data collection should be standardised, and follow national health data and terminology standards and definitions.

4. Financial sustainability

Sufficient, sustainable funding is required. The funding model may include partnerships with multiple beneficiaries. In this context, funding via clinical trials might also be appropriate.

5. Transparency and access

Timely provision of tailored information to patients, hospitals, jurisdictions, governments, funders, private health insurers, researchers and other stakeholders, while upholding patient privacy.

6. Data linkage, integration and interoperability

By improving linkage, a more comprehensive, longitudinal picture of patient treatment and outcomes than is currently available will be possible. This will also allow for increased analytical power and provide more cost-effective clinical trials and more comprehensive postmarket surveillance of devices and medicines.

Clinical registries positively impact the quality of patient healthcare and health outcomes.11 12 An Australian evaluation reported that registries improve the value of healthcare delivery at a relatively low cost, therefore producing high returns on investment.13 While registries are often designed for such quality and safety purposes, they can also provide a platform to answer pragmatic questions. RRCTs can be embedded into both large population-based registries (eg, health services registries) and smaller registries (eg, disease or procedure registries).

RRCT design considerations

RRCTs complement more traditional RCTs. While RCTs remain the gold standard for demonstrating efficacy, they are limited by the time they take, their costs and their limited external validity.7 One of the main problems with conventional RCTs is often restrictive eligibility criteria, which limits the generalisability between clinical trial populations and the target population.14 Although RRCTs can reduce the problem of generalisability, the extent to which this occurs is dependent both on characteristics of the registry and design of the embedded clinical trial.14 The advantages and disadvantages of RRCT are listed in table 2.

Table 2

Advantages and disadvantages of RRCT

By using existing infrastructure, RRCTs may deliver answers to key clinical questions efficiently and at a lesser cost; have the potential to engage a broad range of stakeholders; have an inbuilt ability to collect long-term follow-up data and can improve generalisability of results.7 15 Furthermore, given the cost of running a RRCT is significantly less than the traditional RCT model, RRCTs may have a key role in evaluating important clinical questions where funding is difficult to access, for example, evaluation of generic pharmacotherapies,16 medical devices and clinical procedures.15

Trial population representativeness

An added benefit of RRCTs relate to the ability to address some of the concerns of the conventional RCTs, including the inadequate representativeness of trial populations.17 Embedding trials in clinical registries provide increased opportunity to systematically offer trial participation to ‘real-world patients’ rather than opportunistically identifying potential trial participants. Studies comparing baseline characteristics of RCT trial populations with registry samples have identified lower risk profiles, with frequent exclusion of elderly patients and those with comorbidities.18 Trial designs that recruit from real-world populations are likely to improve the external validity of the trial findings, providing physicians with appropriate evidence on which to base clinical decisions.19 However, the population coverage and representativeness of the clinical registry used for a RRCT also needs to be considered when generalising from such trials.

Randomisation and treatment exposure assessment in RRCTs

Randomisation can be readily achieved with web-based randomisation modules that can be linked to registry databases. Non-commercial, smartphone-accessible applications can enable rapid, accurate randomisation at the bedside making them highly suitable for adoption into registry-based trials.20 Assuring adequate treatment exposure in RRCTs remains a similar challenge to conventional RCTs. Depending on the trial design, individuals or groups of patient’s treatment allocation will be determined at the point of randomisation. In procedural registries, where the actual procedure to be undertaken varies, routine registry data collection should identify the procedural activity and highlight protocol deviations. In disease and health service registries, drug allocation, treatment compliance and persistence monitoring are required to ensure adequate treatment exposure—similar to conventional RCTs. The efficiency gain in RRCTs relies on the information being collected as part of routine registry follow-up data collection, but does not exclude other data being collected, such as data relevant to treatment compliance.

Outcome information and end point validation

End point validation is an important consideration, particularly where data are collected from different institutions: there must be consistency in data definition and data collection. A fundamental difference from clinical trials end points, which are chosen or designed to meet the needs of the intervention, registry-based end points may have been designed for vastly different purposes. The accuracy of clinical end point determination using registry data as compared with active source data collection, follow-up and clinical adjudication is currently unknown. Some registry outcomes may be linked or aligned to International Classification of Diseases 10th Revision (ICD-10) codes. Internationally, Australia is unique in its adoption of ICD-10 coding for hospital reimbursement, and coding standards differ between states and territories. There is some evidence to suggest poor agreement between ICD-10 coding and clinical audit.21 Adjudication of events within registry trials may therefore be necessary to ensure the quality of risk factor and outcomes data.7 One approach is having a Clinical Event Adjudication Committee adjudicate a subset of randomly identified events. Linking data to other datasets (eg, National Death Index) can also be used for validation, where such datasets are available.

What designs are available for RRCT?

RRCTs are particularly useful when assessing real-world implementation of interventions.7 RRCTs can take a number of designs, including individual-level RCTs, parallel group trials, platform or adaptive trials, cluster randomised trials (CRT) and stepped-wedge CRT (SW-CRT). Adaptive randomisation may occur within prespecified subgroups. While randomisation at the individual level has been more commonly used in RRCTs to date,7 cluster randomisation is increasingly reported,7 22 and has several distinct advantages, including overcoming administrative barriers and reducing costs.22

Several types of CRT design may be used in RRCTs (see figure 1). In each of these study types, clusters (eg, hospitals, general practitioner practices, etc) are randomised rather than individual patients. Similar to other trial designs, in CRT the clustering effects need to be considered. For example, we might expect that mortality risk would vary across intensive care units, but patients within the same intensive care unit are likely to have similar mortality risk. This is called ‘within-cluster correlation’, and as such, the information per patient is not independent. There is some loss of statistical information in CRT which leads to increased sample sizes requirements, however, this is typically offset by the ease of identifying and recruiting patients.

Figure 1

Possible designs for registry randomised controlled trials. ICU, intensive care unit; RCT, randomised controlled trial.

Parallel cluster RCTs are similar to individual patient RCTs, except that randomisation occurs at the level of the cluster rather than the patient. Each cluster is randomly allocated to an intervention and remains with that intervention for the duration of the trial. In these studies, the information per patient is not independent leading to a loss of information, counterbalanced by greater number of recruited patients. However, these studies are relatively simple to analyse and interpret.

In the cluster crossover design, clusters switch between interventions, and the effect of the intervention is estimated by comparisons within each cluster, removing the between-cluster variability. Thus, this design requires fewer clusters and fewer patients than the parallel design. However, in this design, within-period correlations and between-period correlations must both be considered, and the design necessitates that the between-period correlation is smaller than the within-period correlation, because their relative size determines the value of the crossover. Not all individually randomised trials are suitable for conduct as a cluster crossover trial: treatments must be able to be implemented and withdrawn easily; carryover effects must be avoided and all patients must be recruited from the registry.

SW-CRT designs are beneficial where there is a risk of individuals in the control arm being accidentally exposed to the intervention. They are particularly useful in the general practice setting, or when implementing guidelines, training or system changes. In a SW-CRT design, all clusters start in the control phase and randomisation determines the order in which the intervention is implemented. Clusters (or groups of clusters) are randomly assigned to a time point when they crossover from control to intervention phase (step/sequence/arm). The SW-CRT can be designed with data collected cross-sectionally from different samples of individuals at each time point. Alternatively, data may be collected from a closed cohort, where individuals are followed longitudinally over the entire period of the trial and repeated measures are taken on the same individual at each time point. No new individuals join after the study starts. In an open cohort, data are collected on the same individuals over time, but new individuals can join over the study duration. At the end of the trial all clusters are in the intervention phase. Clusters are followed-up longitudinally, with outcomes/end points usually measured at discrete time points on individuals.

In SW-CRT, the sample size calculations need to allow for the effects of randomising clusters instead of individuals, those attending the same institution are more likely to have similar results than those attending elsewhere. The positive correlation of individuals within the same cluster is quantified with the intracluster correlation (ICC). The ICC measures the proportion of the total variance attributable to the variance between clusters. The extra variability between clusters in CRT has implications for the sample size and analysis. SW-CRT assume the full effect of the intervention occurs at the same time interval when intervention is introduced. A delay of intervention effect reduces the study power given a fixed number of cluster and participants. One approach to ensure that the required power is maintained is to add additional measurement periods.

Advantages and disadvantages of various RRCT designs are summarised in table 3.

Table 3

Advantages and disadvantages of various registry randomised controlled trial designs

Extracting data from the electronic medical record to develop virtual registries

Electronic data capture and integration with the electronic medical record has the potential to improve data validity and the efficiency of data collection, both of critical importance for clinical trials.12 Using routinely collected medical record data in an automated fashion for determining clinical trial eligibility according to inclusion and exclusion criteria could greatly facilitate trial recruitment. Using routinely collected electronic medical record data, entered by clinicians at the time of diagnosis and treatment, for automated outcome ascertainment may also reduce time and costs and efficiency in conducting trials. There has been interest in using medical records as a data source for performing clinical analytics as early as the 1960s.23 Medical records contain a tremendous accumulation of data, and it was hoped that electronic data processing systems would allow for organised, chronological records of patient information that could be used to facilitate research and hospital reporting.23 It has recently been suggested that linkage of electronic medical records can be successfully used to provide near real-time clinical audit with feedback to clinicians, and provide a framework for clinical decision support.24

Overcoming the technical and legal issues associated with data linkage to the electronic medical record can be a barrier12; not the least that there are a large number of different electronic medical record (eMR) platforms currently in use. Currently, the majority of data existing in the electronic medical record is free text, requiring careful mapping and validation. Text mining and natural language processing approaches to electronic medical records may assist in accurate patient identification and data collection. The adoption of universal definitions of clinical events coded into eMRs would be an important development in the use of these systems for RRCTs. This requires a collaborative approach including the eMR developer, data architect, data scientist, data analyst and clinicians. There are already successful examples of combining data from registries with the electronic health record.25 The development of privacy preserved record linkage capabilities will further facilitate the extended linking of administrative and clinical trial datasets for monitoring of health outcomes.26 This approach to data linkage has been highlighted as a priority area for clinical quality registries in order to facilitate their use for research purposes.2

Embedding trials into registries

In order to embed trials into registries, trialists must reach a compromise between a ‘broad but shallow’ data collection methodology typical of many registries, and the ‘narrow but deep’ approach for trial-related data collection, often needing to accept simpler accountability than seen in more traditional RCT approaches. In countries with well-established national registries, with standardised end points and little missing data, RRCTs offer a viable alternative to RCTs for generating high-quality clinical evidence.7 By addressing issues of end point validity and adjudication, and decreasing the proportion of missing data, smaller disease or procedure-focused registries might be able to improve the quality of their evidence, and in turn become a viable alternative platform than more costly RCTs.7 For this reason, it may be advantageous to initially embed RRCTs into registries that are already well established.7 Internationally, registry data are becoming increasingly important in regulatory assessments, especially for postmarketing safety and effectiveness studies.27 The key pillars when considering embedding a RRCT are outlined in figure 2.

Figure 2

Key pillars when considering embedding a registry randomised clinical trial.27 QA, quality assurance.

Best practice requires registries to be adequately resourced, so that data quality is maximised. Should the RRCT be a feasible option for a given registry and for a clinical questions, careful delineation of responsibilities regarding randomisation, missing data, handling of data queries, data quality, data extraction and management of serious adverse event information need to be considered. We suggest that the first two are the responsibility of the registry, the last is the responsibility of the trialist and data queries could be attended to by either the registry or trialist. This requires adequate funding both of the registry itself and the RCT embedded within it. However, the benefits may far outweigh the cost. RRCTs allow for potential collaboration between clinical trial networks and clinical quality registries in related disciplines. The shared data management responsibilities between these potentially avoids data wastage for once-only use in more traditional clinical trials, and also improves the quality of data available within the registry. In some cases, RRCTs may not the best approach, such as in earlier phase II or phase III clinical trials.

One of the key benefits of embedding clinical trials into registries is that following the trial’s conclusion, the translation of evidence generated within that trial can then be assessed using the ongoing clinical registry. This addresses one of the key drawbacks of traditional randomised trials—there is no direct way to measure whether or not their findings have been implemented, and whether they translate to real-world practice.

Finally, there is also increasing interest in facilitating long-term follow-up post-RCT using linked administrative and registry data.28 A number of large-scale clinical trials have used this method to report of long-term observational clinical outcomes following the short-term observation of the clinical trials.29–31 This strategy is valuable for mandatory reporting registries, such as cancer and death registries and provides valuable information in relation to long-term outcomes following a particular intervention or treatment. However, it has also proven valuable for trials of acute interventions and short-term follow-up in COVID-19 treatment trials.32

Conclusion

Registries offer a unique platform within which to conduct RCTs. With appropriate registry selection and clinical trial design, and advances in data linkage and data quality, RRCTs can play an important role in answering clinical questions in a pragmatic, cost-effective way.

Ethics approval

Not applicable.

References

Footnotes

  • Twitter @JenReilly78, @profcmreid

  • Contributors As per ICMJE criteria for authorship, all authors made substantial contributions to the conception of this work, drafting the manuscript or revising it critically for important intellectual content and approved the final version to be published. All authors agreed to be accountable for all aspects of the work in ensuring that questions raised relating to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Funding This work was supported by an Australian Government Medical Research Future Fund grant through the 2017 Lifting Clinical Trials and Registries Capacity—Clinical Trials Networks Programme.

  • Competing interests CMR is funded through an NHMRC Principal Research Fellowship (GHT1136372). CH is funded by a National Heart Foundation Fellowship and an NHMRC Investigator Grant. JR is funded by an Australian Government Research Training Programme (RTP) Scholarship and a Monash University Graduate Excellence Scholarship.

  • Provenance and peer review Not commissioned; externally peer reviewed.