Article Text
PDF
Abstract
Objectives In an effort to mitigate COVID-19 related challenges for clinical research, the US Food and Drug Administration (FDA) issued new guidance for the conduct of ‘virtual’ clinical trials in late March 2020. This study documents trends in the use of connected digital products (CDPs), tools that enable remote patient monitoring and telehealth consultation, in clinical trials both before and after the onset of the pandemic.
Design We applied a comprehensive text search algorithm to clinical trial registry data to identify trials that use CDPs for remote monitoring or telehealth. We compared CDP use in the months before and after the issuance of FDA guidance facilitating virtual clinical trials.
Setting All trials registered on ClinicalTrials.gov with start dates from May 2019 through February 2021.
Outcome measures The primary outcome measure was the overall percentage of CDP use in clinical trials started in the 10 months prior to the pandemic onset (May 2019–February 2020) compared with the 10 months following (May 2020–February 2021). Secondary outcome measures included CDP usage by trial type (interventional, observational), funder type (industry, non-industry) and diagnoses (COVID-19 or non-COVID-19 participants).
Results CDP usage in clinical trials increased by only 1.65 percentage points, from 14.19% (n=23 473) of all trials initiated in the 10 months prior to the pandemic onset to 15.84% (n=26 009) of those started in the 10 months following (p<0.01). The increase occurred primarily in observational studies and non-industry funded trials and was driven entirely by CDP usage in trials for COVID-19.
Conclusions These findings suggest that in the short-term, new options created by regulatory guidance to stimulate telehealth and remote monitoring were not widely incorporated into clinical research. In the months immediately following the pandemic onset, CDP adoption increased primarily in observational and non-industry funded studies where virtual protocols are likely medically necessary due to the participants’ COVID-19 diagnosis.
- COVID-19
- clinical trials
- telemedicine
- information technology
- general medicine (see internal medicine)
- health economics
Data availability statement
Data are available in a public, open access repository. At the time of publication, the authors will share data from the study on the public GitHub repository https://github.com/arieldora/Covid19digitaltrials.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- COVID-19
- clinical trials
- telemedicine
- information technology
- general medicine (see internal medicine)
- health economics
Strengths and limitations of this study
This study is the first to quantify postpandemic trends in the use of telehealth and remote patient monitoring technology within the clinical research setting.
A comprehensive text search algorithm was created and used to identify telehealth and remote monitoring use in clinical trials.
Detailed, manual review of 500 trial records validated the search term list and algorithm, confirming a high degree of sensitivity and specificity.
The limited postperiod analysis window (10 months) precludes the ability to draw implications for the pandemic’s longer-term impact on the conduct of clinical research.
The identification of connected digital products in clinical trials is dependent on the trial sponsor entering information about remote monitoring and/or telehealth delivery when creating or updating the ClinicalTrials.gov record.
Introduction
When the WHO declared COVID-19 a global pandemic in March 2020, many aspects of healthcare were rapidly upended including the traditional conduct of clinical research. Preliminary evidence has documented the suspension of critical clinical trials and postponement of new trial start dates due to enrolment challenges and site closures.1–3
In an effort to keep clinical research moving forward at a time when visits to in-person sites were challenging and risky, the US Food and Drug Administration (FDA) issued new, comprehensive guidance for the conduct of clinical trials, with the aim of encouraging virtual patient consultations and remote collection of clinical outcomes assessments.4
Connected digital products (CDPs) are portable software and sensor-based technologies that are designed for patient use with little to no clinician involvement. These products collect health-related measurements from patients and thus enable remote monitoring and virtual consultation in the clinical research setting. Examples of CDPs include wearable activity trackers and heart rate monitors, mobile apps used for data delivery and collection, ingestible sensors, and health assessments conducted via a mobile platform.5 In the years prior to COVID-19, a dramatic increase in the use of CDPs has been documented in clinical trials.5 Though the appeal of CDPs is clear6 7—both for the pandemic duration and beyond—evidence describing their adoption in the clinical research setting following COVID-19 is lacking.
In an effort to understand recent implementation of remote monitoring and telehealth in clinical research, this study asks whether CDP use in clinical trials increased following the FDA’s March 2020 issuance of guidance for the conduct of virtual trials. The study documents trends in the use of CDPs in clinical trials that started both before and after the emergence of pandemic-related challenges for traditional protocols. Trends in CDP adoption data within the clinical research setting are particularly relevant in light of the pandemic-induced broader telehealth-related policy changes, such as the rapid movement of payers to cover a broad range of virtual care delivery services and particular generosity in the coverage of remote patient monitoring technologies.8 9
Methods
We applied a text search algorithm to the full set of trial records downloaded from ClinicalTrials.gov to identify trials that used CDPs.10 ClinicalTrials.gov is a publicly available database that is considered comprehensive, as trial registration is required (1) when a study is being used to support the regulatory approval of a new therapeutic product and/or (2) when it is intended for publication in one of the International Committee of Medical Journal Editors’ member journals.11 12 To conduct the search, we adapted the set of CDP search terms published in Marra et al5 to include additional terms specific to remote monitoring and telehealth delivery (online supplemental file includes the list of search terms). We used an automated algorithm to search for each term in several relevant ClinicalTrials.gov fields, such as “official title”, “brief summary”, “detailed description”, “interventions”, “study arms” and “outcome measures”. We validated our search protocol with a manual review of 500 randomly selected trials (table 1 includes examples of qualified CDP trials).
Supplemental material
Examples of connected digital product usage identified in clinical trials
Our sample period includes trials with start dates between 1 May 2019 and 28 February 2021. We divided the first 10 months of the sample into prepandemic onset (‘preperiod’) and the last 10 months of the sample into postpandemic onset (‘postperiod’). We excluded trials with start dates in March and April of 2020, as FDA guidance on the conduct of clinical trials and other public policies encouraging telehealth use were introduced during this time, and trial sponsors would have needed this window to make protocol adjustments in response to new policies (table 2 presents a brief timeline of key events). For analysis purposes, trials were segmented into COVID-19 or non-COVID trials based on diagnoses listed in the ‘conditions’ field. Trial type (observational or interventional) and funder type (industry or non-industry) were collected from standardised ClinicalTrials.gov fields associated with each trial.13 To test whether the proportion of trials using CDPs was statistically different in the preperiod and postperiod, we performed two-sided z-tests using the means and standard deviation from the 10-month observation periods in each group. The analysis was conducted using STATA V.15.
Timeline of selected COVID-19 related regulatory guidance and policy changes encouraging the use of telehealth and remote monitoring in the USA
Patient and public involvement statement
Neither patients nor the public were involved in this research.
Results
Over the 22-month period from May 2019 to February 2021, the overall use of CDPs in clinical trials remained relatively stable, accounting for between 12% and 19% of all newly started trials in each month of our dataset. Trials using CDPs accounted for 14.19% (3330/23 473) of all trials started in the preperiod and 15.84% (4121/26 009) of trials started in the postperiod. This 1.65 percentage point increase represents a small but statistically significant uptick in CDP usage for trials with start dates after the pandemic onset (p<0.01) (figure 1).
Proportion of clinical trials started monthly that use connected digital products for telehealth delivery or remote monitoring. The preperiod and postperiod analysis windows include the 10 months before and after the COVID-19 pandemic onset. Percentages on the left and right represent the average percentage of trials using connected digital products during the preperiod and postperiod, respectively. *Statistical significance in the mean difference between the preperiods and postperiods using two-sided proportion tests. Source: authors’ analysis of ClinicalTrials.gov data.
Among trials labelled as observational, there was a larger average increase in trials using CDPs from 11.7% (569/4863) preperiod to 14.6% (799/5490) postperiod (p<0.001). There was also an increase in the percentage of trials using CDPs among studies funded by non-industry parties, such as hospitals, academic institutions and government (16.2%–18.3%, p<0.01). The use of CDPs in industry-funded trials was notably lower than the sample mean both before and after the FDA’s guidance was issued in response to the pandemic onset (7.9% vs 8.9%, p=0.17), although this share also experienced a small overall increase, on average.
Trials with start dates in the postperiod were segmented into COVID-19 (n=3294) and non-COVID (n=22 715) diagnoses. Among trials conducted in patients with COVID-19, use of CDPs was significantly higher than the preperiod average (20.0% vs 14.2%, p<0.0001). However, among trials unrelated to COVID-19 (‘non-COVID’) that were started in the postperiod, the change in CDP usage compared with the preperiod was very small and not statistically significant (15.0% vs 14.2%, p=0.19). As such, the observed increase in CDP usage among observational and non-industry funded trials appears to have been driven by studies related directly to COVID-19 (figure 2).
Proportion of clinical trials started that use connected digital products for telehealth delivery or remote monitoring, by study type and funder type. ‘Preperiod’ and ‘postperiod’ refer to the 10 months immediately before and after COVID-19 onset in the USA. March and April 2020 are excluded because policy changes and guidance documents were issued during that time. Trials started in the postperiod are divided into ‘COVID-19’ trials and ‘non-COVID’ trials based on their diagnoses in the ClinicalTrials.gov conditions label. *Statistical significance in the mean difference between the preperiods and postperiods using two-sided proportion tests. Source: authors’ analysis of ClinicalTrials.gov data.
To further validate the results, we performed a series of robustness checks. First, we removed trials where the recruitment status suggested that the study had not begun recruiting participants even though the start date had technically passed.14 Within the subset of trials where commencement of enrolment had been confirmed (n=19 785 preperiod, 18 426 postperiod), there was no statistically significant difference in overall CDP usage between the preperiod and postperiod (15.1%–15.9%, p=0.38). Differences seen in observational and COVID-specific trials were similar to those observed in the overall sample. Second, we removed trials conducted outside of the USA because the guidance encouraging virtual trials was issued by the US FDA. Though we find a somewhat higher rate of CDP adoption among US trials (between 15.7% and 24.3% per month), the average increase in CDP usage among US trials following the pandemic onset was the same as the average seen for all trials in the sample (1.7 percentage points from preperiod to postperiod) (online supplemental file).
Discussion
Overall, the use of CDPs in clinical trials increased only very modestly in trials with start dates occurring after COVID-19 was declared a global pandemic. This finding is surprising in light of new regulatory guidance introduced in late March 2020, which aimed to bolster the use of CDPs to sustain clinical research. Moreover, these modest increases stand in sharp contrast to the multiple-order-of-magnitude increases in the use of telemedicine for routine healthcare delivery that were observed over the same period of time.15 Though it may still be too early to observe the full effect of new regulatory initiatives, our findings suggest that the growing enthusiasm for telehealth and remote patient monitoring in mitigating pandemic-related healthcare challenges has yet to be realised in the clinical research setting.
There are several possible explanations for why CDP usage in the clinical research setting did not increase as dramatically as expected in the months following the pandemic onset. On one hand, the increase in CDP usage documented among observational trials (rather than interventional trials) suggests the relative ease of incorporating new remote monitoring technology into studies that do not evaluate therapeutic benefit. Implementation of protocol changes may be more accessible in the context of observational analysis that does not require rigorous prior planning and approval to incorporate into a study design. On the other hand, the consistently lower adoption of CDPs by industry funders reinforces that firms may continue to perceive substantial risk associated with reliance on technology for trials that require regulatory review or may be reluctant to augment trial protocols and operations—even after the FDA explicitly encouraged protocol modifications to enable virtual trials. Further, the extent to which FDA guidance should elicit a change in protocol design by trial sponsors should be most acutely observed in trials with US sites. However, we find the same percentage point increase in CDP usage following the pandemic onset for US trials as was seen in all trials. It is also possible that clinical trial investigators do not yet see value in adding CDPs to their trials or lack the technical expertise to do so, despite regulatory encouragement.
Additionally, the fact that COVID-specific trials drove the observed increase in CDP usage suggests that the changes in FDA guidance may have primarily impacted trials where virtual protocols were most necessary for the health and safety of clinicians and other patients (in this case, given the trial participants’ diagnosis). Though an increase in CDP usage among non-COVID related trials may still materialise in the future, there has yet to be any evidence of massive take-up on the scale of other services, such as telemedicine. Though several factors could add to investigator and trial sponsor hesitancy in the adoption of remote monitoring and telehealth technology, the findings of this study suggest that trial sponsors may be unwilling or unable to adopt CDPs in their research design until there is either medical necessity or formalised regulatory requirements to do so.
Finally, robustness checks find a very small but non-statistically significant increase in CDP usage among the subset of trials where commencement of participant recruitment could be confirmed, suggesting that at least some of the small overall increase in CDP usage may be attributable to trials that were planned to start within the postperiod analysis window (1 May 2020–28 February 2021) but may have experienced delays.
The study’s primary limitation is its somewhat short analysis period. The trends identified are only representative of CDP usage in the 10 months immediately following issuance of the FDA’s guidance for conducting virtual protocols (May 2020–February 2021) and implications for long-term trends cannot yet be drawn. Other limitations include reliance on data entered by trial sponsors and a risk of incomplete data since trials can be retrospectively registered after their start date. We have attempted to mitigate the latter issue by waiting 5 weeks beyond the end of the analysis period to download records from ClinicalTrials.gov.
Conclusion
Though the pandemic may lead to sustained changes in how healthcare is delivered, the longer-term implications for the use of connected technology in clinical research are less clear. Despite regulatory guidance encouraging remote monitoring and telehealth use in clinical trials, we did not find a meaningful increase in the usage of CDPs in trials started after the pandemic onset. Trends in the use of CDPs will be important to monitor as COVID-19 continues to impact investigators’ ability to conduct clinical research and further policies around telehealth are introduced.
Data availability statement
Data are available in a public, open access repository. At the time of publication, the authors will share data from the study on the public GitHub repository https://github.com/arieldora/Covid19digitaltrials.
Ethics statements
Ethics approval
This study analyses publicly available clinical trial protocols that have been registered in the freely accessible ClinicalTrials.gov database provided by the National Library of Medicine.
Acknowledgments
The authors would like to thank Lila Kelso and Melissa Ouellet for excellent research assistance.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @arieldora
Contributors The study was designed by CM and ADS. CM conducted the analysis in consultation with ADS and WJG. All authors contributed to the writing of the manuscript.
Funding WJG acknowledges research funding from IBM and consulting income from the Office of the National Coordinator for Health Information Technology, both unrelated to this topic. ADS acknowledges research funding from the Kauffman Junior Faculty Fellowship and acknowledges consulting income from the US Department of Health and Human Services unrelated to this topic.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.