Objective To determine levels of public registration for a cohort of clinical trials reviewed and given a favourable opinion by research ethics committees in the United Kingdom.
Study design Audit of records.
Setting Clinical trials receiving a favourable ethics opinion between 1 January 2016 and 30 June 2016.
Main outcome measures Correlation between trials on the UK research ethics committee database and any primary registry entry on the WHO International Clinical Trials Registry Platform or clinicaltrials.gov as of 29 August 2017 (14 to 20 months after the favourable ethics committee opinion).
Results Over the study period 1014 trials received a favourable ethics opinion, with 397 (39%) registered on the European Union Drug Regulating Authorities Clinical Trials database, and 18 with an agreed clinical trial registration deferral. Excluding these trials, the total number subsequently requiring registration was 599, and of these 405 (40% of total) were found to be registered. Follow-up with the 194 investigators or sponsors of trials not found to be registered produced 121 responses with a further 10 (1%) trials having already registered, 55 commitments to register and a variety of other responses. The overall registration rate was therefore 80%.
Conclusions Despite researchers and sponsors being reminded that registration of clinical trials is a condition of the research ethics committee (REC) favourable opinion, one-fifth of clinical trials either had not been registered, or their registration could not easily be found, 14 to 20 months after receiving the favourable opinion letter. The methodology trialled here proved effective, and although there are positive indications of a culture change towards greater registration, our results show that more still needs to be done to increase trial registration.
- research registry
- research ethics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Registration of clinical trials on publicly accessible research registries is a matter of good research ethics. If clinical trials are not registered, research organisations can hide results or trials that they do not like.
Since 2013 UK policy has required registration of all clinical trials as a condition of research ethics committee’s (REC) favourable opinion.
By comparing the REC records with publicly accessible research registries, we have been able to accurately determine clinical trial registration rates.
By comparing records held by a regulator with publicly accessible registries we have for the first time produced a ‘true’ trial registration rate for the UK.
A limitation comes from the use of only a subset of records rather than the whole REC database.
As of 30 September 2013, it has been a UK policy condition of a favourable research ethics committee (REC) opinion that all clinical trials are registered on a publicly accessible database (see box 1 for the wording from the favourable opinion letter provided to researchers).1 This should ideally occur before the first participant is recruited in accordance with the Declaration of Helsinki,2 or no later than 6 weeks after recruitment of the first participant. The requirement was a response to calls by groups such as the Cochrane Collaboration,3 the AllTrials campaign4 and the WHO5 who have argued convincingly for transparency around clinical trials in order to ensure that valuable research is not lost, and also to prevent unscrupulous researchers or investors hiding clinically or scientifically relevant results for commercial reasons.6 Trial registration has been required for certain types of trials since 2004 by the European Union (EU)7 and since 2007 by the USFood and Drug Administration,8 but in the latter case the full policy is not being enforced9 even though overall more trials are being registered.10 Non-regulatory attempts are being made by organisations such as the International Committee of Medical Journal Editors (ICMJE) who are making registration a requirement for publication,11 but national regulatory environments also seem to be important.12 13 Box 1 provides an extract of the trial registration wording from the research ethics committee’s (REC) favourable opinion letter that all researchers receive when a clinical trial is approved in the UK.
Applications to National Health Service (NHS) RECs are made using the online Integrated Research Application System (IRAS),14 which includes a filter question (see box 2) asking researchers to define the type of study or trial. In addition to the UK policy requirement for trial registration, there is a legal obligation for registration placed on Clinical Trials of Investigational Medicinal Products (CTIMPs) under the current European and UK clinical trials legislation.1 All trials with a Clinical Trials Authorisation (CTA) have an entry on the European Union Drug Regulating Authorities Clinical Trials database (EudraCT), which is used to populate the publicly accessible EU Clinical Trial Register. However, the EU legislation has a specific registration exemption for phase I trials involving healthy volunteers,15 while other types of clinical trials are also not covered by the legal requirement to register. In order to determine compliance with its registration policy for these other types of trials, the Health Research Authority (HRA) conducted an audit in early 2016 looking specifically at phase I, device and ‘other’ trial registration (N.B. although most of the HRA’s functions apply to research undertaken in England, the HRA also works closely with the other countries in the UK (Scotland, Wales and Northern Ireland) to provide a UK-wide system including a research ethics service, so was able to audit UK-wide records). The results were published on the HRA website in response to questions raised by a UK government inquiry into research integrity.1 The audit authors concluded that more was needed to be done to highlight the registration requirements to sponsors, with subsequent HRA efforts centred around improved training events and updating the wording on the application form. This paper now describes a second, more systematic attempt to determine registration rates for phase I, medical devices and ‘other’ clinical trials receiving a favourable opinion from RECs 3 years after the registration requirement came into force.
Extract from the Favourable Opinion letter received by all investigators
Registration of Clinical Trials
All clinical trials (defined as the first four categories on the IRAS filter page) must be registered on a publicly accessible database. This should be before the first participant is recruited but no later than 6 weeks after recruitment of the first participant.
There is no requirement to separately notify the REC but you should do so at the earliest opportunity for example, when submitting an amendment. We will audit the registration details as part of the annual progress reporting process.
To ensure transparency in research, we strongly recommend that all research is registered but for non-clinical trials this is not currently mandatory.
If a sponsor wishes to request a deferral for study registration within the required timeframe, they should contact email@example.com. The expectation is that all clinical trials will be registered, however, in exceptional circumstances non-registration may be permissible with prior agreement from the HRA. Guidance on where to register is provided on the HRA website.
Filter question 2 of the Integrated Research Application System (IRAS) form
2. Select one category from the list below:
Clinical trial of an investigational medicinal product
Clinical investigation or other study of a medical device
Combined trial of an investigational medicinal product and an investigational medical device
Other clinical trial to study a novel intervention or randomised clinical trial to compare interventions in clinical practice
Basic science study involving procedures with human participants
Study administering questionnaires/interviews for quantitative analysis, or using mixed quantitative/qualitative methodology
Study involving qualitative methods only
Study limited to working with human tissue samples (or other human biological samples) and data (specific project only)
Study limited to working with data (specific project only)
Research tissue bank
If your work does not fit any of these categories, select the option below:
This study included all applications to UK RECs during the period 1 January 2016 to 30 June 2016 where the investigator or research team had selected one of the first four options in the IRAS (Integrated Research Application System) filter question 2 (defining the work as a clinical trial), and the trials had then ultimately received a favourable opinion from a UK REC. Studies with a legal requirement for a public registration on EudraCT (mainly phases II, III and IV CTIMPs) were marked as already registered.
Extracting data from the HRA Assessment Review Portal
A management information report was extracted from the HRA Assessment Review Portal (HARP) database16 to identify trials within the scope of the study. There are specific data fields on HARP recording the research reference numbers including registration number for trials registered on EudraCT, clinicaltrials.gov and/or the International Standard Randomised Controlled Trial Number Registry (ISRCTN), as well as an ‘other reference numbers’ field. This information is populated on HARP either through direct import from the IRAS application (data collected via question A5-1 of application prepared in IRAS) or as manual input by the REC Manager when they are advised of registration.
Initial trial registration searches
For trials without a registration number logged on HARP, a registration search was conducted in August 2017 using the full trial title, and if the trial could not be located with this, the short title and REC Reference number. The manual searches via the Google search engine sought to locate the clinical trial on a publicly accessible registry. For the purposes of this search the standard applied was registration in any primary registry on the WHO International Clinical Trials Registry Platform,17 or clinicaltrials.gov (this is not a primary WHO registry but is an International Committee of Medical Journal Editors acceptable registry).18 If the trial was located, the registration details (name of registry and registration number) were logged and HARP updated (for future reference if required).
Follow-up for trials not found
If a registry entry could not be located using either the HARP record or manual searches then the Chief Investigator and Sponsor were contacted via email (for phase I trials the Chief Investigator only was contacted) and asked to provide registration details or a reason for registration having not taken place. Only a single email was sent with no reminders. Responses obtained over the following couple of weeks were recorded and HARP was updated where registration information was provided. Responses were reviewed and categorised to determine broad themes. If an email exchange was held with the applicant the response category was updated to reflect the final response (eg, if an applicant initially thought there was no requirement to register their trial but then did agree to register after receiving further guidance from the HRA then the trial was recorded as ‘will register’).
Patient and public involvement
The need for the audit described here was discussed at the HRA’s partner ‘Transparency Forum’ whose aim is to promote research transparency and understand opportunities, obstacles and levers.19 Preliminary results were made available on the HRA website in response to questions from the UK House of Commons Science and Technology Select Committee.20
A total of 1014 trials were initially identified using HARP, of which 397 were CTIMPs (phases II, III and IV) already registered through EudraCT. Of the remaining 617 trials, 18 were trials with an agreed registration deferral. This deferral is allowed by the HRA in instances where public details of a mainly phase I but occasionally device trial might be considered commercially confidential, although there is still the expectation that the trial will be registered on a publicly accessible registry when the reason for the deferral is no longer valid, or immediately should the trial be terminated early for safety reasons. After these exclusions, 599 trials remained. Registration records could be found on either HARP or through the manual search for 405 trials, leaving 194 unregistered trials (in addition to the 18 with deferrals). Data is summarised in figure 1, tables 1 and 2.
Eighty-four phase I trials were identified of which 17 had an agreed clinical trial registration deferral. Of the registered phase I trials (n=58), most were registered on clinicaltrials.gov and only one in the ISRCTN registry. Eight trials were identified as being registered through the HARP data export and a further 50 were identified through manual searches.
A total of 206 device trials were identified with one having an agreed clinical trial registration deferral in place. Of the device trials registered (n=138), the majority were again registered on clinicaltrials.gov with 10% on the ISRCTN registry. One trial was registered on the EU Clinical Trials Register (this is unusual for a device trial). Fifty-nine registrations were identified through the HARP data export and an additional 79 were located through manual searches.
A total of 327 ‘other’ clinical trials were included. This category includes surgery, radiotherapy, imaging investigations, mental health investigations or therapies, physiological investigations, trials of products not defined as medicines or medical devices (eg, nutritional) and complementary or alternative therapies.21 None of these had a registration deferral in place. Of those registered (n=209) just over 50% were on clinicaltrials.gov, and just under half (46%) were on the ISRCTN registry. A small proportion of this trial type (1.4%) were registered on The Australian New Zealand Clinical Trials Registry, and the German Clinical Trials Register. Forty-three registrations were found through the HARP data export and a further 166 through manual searches.
A total of 194 follow-up emails were sent to the Chief Investigators/Sponsors for trials where we could not find registration details forto request confirmation of whether the trial has been registered and if not, what the reason for this was. One hundred and twenty-one responses were received and categorised (table 3). Some respondents queried the requirement to register so a reply was sent to clarify the UK policy position on trial registration, and a number of further responses were received. The email responses identified a further 10 trial that had been registered giving a final total of 812 with a valid registration (80%) out of the total 1014 trials in the cohort.
Nine Chief Investigators of phase I trials were contacted to request confirmation of whether the trial has been registered and if not, what the reason for this was. Five responses were received. Of these, two trials were reported to be registered but were not identified through the initial search (this is likely due to variations in the trial title on HARP or the registry), two trials were reported to have not proceeded and one trial reported to have registered through the EudraCT database and the results been posted there. This respondent also referenced the trial details being publicly available on the HRA website.
Sixty-seven Chief Investigators and Sponsors were contacted for device trials after their trial could not be located on a registry. Forty-one responses were received. Nearly 20% responded to say that they would register the trial, most commonly specifying clinicaltrials.gov or the ISRCTN Registry. Twelve per cent of respondents (n=5) advised that they had registered and were awaiting the registration number. 12% of respondents also stated that their trial was registered (despite not being found on HARP or through our initial manual search) and provided valid registration details. Three of these were registered on the ‘Research Registry’. Although this registry is not a primary registry in the WHO registry network it is listed on the research transparency page of the HRA website as a useful link under research registries. Two respondents reported to have registered on the NIHR portfolio and another respondent advised that their trial was not yet registered but ‘intended to follow normal guidance from NIHR about public accessibility’. At least seven respondents initially claimed that their trial was not a clinical trial (eg, their response stated that the trial was an observation or feasibility trial and therefore they did not consider it as a clinical trial). One respondent noted that their local R&D team advised that registration was not necessary as the trial was not a clinical trial. Of the respondents that initially claimed their trial was not a clinical trial, only two respondents did not send a further email to confirm that they would register the trial. Four respondents replied with names of websites/databases as to where their trial was registered (box 3).
Alternative databases or registries named by correspondents
Aberystwyth University’s online research repository/database, CADAIR.
Clinical research network portfolio of stroke projects.
HRA Research Summaries website (http://www.hra.nhs.uk/news/research-summaries/).
University of Sheffield post-graduate research database (https://www.sheffield.ac.uk/medicine/prospectivepg/taught/mmedsci/currentresearch).
Scottish Pulmonary Vascular Unit (www.spvu.co.uk).
Open science framework website (https://osf.io/sd4yh/) (log in details required).
Public Health Wales Research and Development Activity webpage.
The Health Foundation (www.health.org.uk).
Various Trusts/intranet R&D pages.
One hundred and eighteen Chief Investigators and Sponsors of ‘other’ clinical trials were contacted after their trial could not be found on a registry. Seventy-five responses were received. Over one-third of replies advised that they would register the trial. One respondent advised that they would ‘review their sponsorship processes to ensure that a check on clinical trial registration is built into our sponsorship workflows’. Five respondents reported to have registered on the NIHR Clinical Research Network Portfolio. Six respondents replied with names of websites/databases as to where their trial was registered (box 3). Three respondents advised that their trial was already registered (these were not found through the initial manual search) and provided registration details. Two of these were on the ISRCTN Registry and two on the Research Registry. At least 20 respondents initially claimed that their trial was not a clinical trial. Examples of trial types where the applicant claimed their trial was not a clinical trial included single case design student projects, a feasibility study, a small single arm observational and qualitative interview study. A number of respondents advised that the trial was a pilot with small sample size and did not regard it necessary to register the trial. One respondent reported that they decided not to register after discussions at the REC meeting. A small proportion of responses claimed they had inadvertently selected the incorrect study type on the IRAS application form. One response stated that they selected ‘Other CT’ as it was the least inappropriate category on the IRAS filter page. One respondent who claimed that their study was not a clinical trial advised that they had ‘received confirmation from the MHRA (UK Medicines and Healthcare Products Regulatory Agency) that it is not a CTIMP and does not require a CTA.’ Two respondents questioned whether it was worthwhile registering retrospectively, with one individual noting ‘This would seem to defeat the purpose of pre-registration.’ Of the respondents who initially claimed their study was not a clinical trial, over half of respondents subsequently confirmed that they would register the trial or had since registered. A number of respondents asked for additional guidance on how to register and which registries were appropriate for their trial type. Some respondents were under the impression that the HRA Research Summary webpage was a form of trial registration. For example one respondent queried, ‘If we register this study on www.clinicaltrials.gov then do we need to register this on HRA website too?’
Comparison to previous HRA audit
Compared with the HRA’s initial registration audit in early 2016, we found phase I registration rates up from 63% to 87%, medical device trials up from 48% to 67% and ‘other’ trials up from 48% to 64%. Following identifying the further 10 registered trials through our email contact with the investigators (not found in our manual search due to discrepancies in study titles and errors in reference numbers), the final registration rates were 90% for phase I, 70% for medical devices and 65% for ‘other’. These figures represent the registration rate at the time of this study being started and do not include subsequent registrations that occurred after the sponsors and investigators had been reminded by email. The previous HRA audit found registrations increased up to 77% (phase I), 85% (devices) and 80% (other) after email follow-up, while the approximately equivalent statistics from our study were 91%, 77% and 87% (figure 2).
Including the 397 trials registered through EudraCT the overall registration rate for the studies included in our search criteria was 80%. For the purpose of calculating this percentage we decided to classify both the 18 studies with valid deferrals and the nine that had not yet started as ‘not registered’ (although we acknowledge that these 27 studies have no policy requirement for registration). This 80% figure is broadly consistent with other studies,12 22 but is the first time this has been calculated for studies having been reviewed by RECs in the UK. This is significant because clinical trials conducted in the UK fall under legislation or policy requiring them to be reviewed by RECs and, as a result the REC records contain the only complete record of all clinical trials. Previously it has been very difficult for researchers and systematic reviewers to discover whether trials have even occurred as often the only public record is the registry itself. By auditing confidential data held by a regulator, and then comparing it with the public registry entries, the numbers reported here represent the first ‘true’ registration rate (certainly at a national level).
The increase in registration rates compared with the 2016 HRA audit is encouraging as they show there is an upward trend in registration for all types of clinical trials examined here, suggesting a possible cultural shift within the trialist community. For instance, it is likely that the phase I registrations are higher because of awareness among industry sponsors of the legal obligation to publicly register phases II, III and IV clinical trials, and thus the inclusion of public trial registration as a standard function of contract research organisations tasked with overseeing the governance aspects of trial preparation. Likewise a number of phone calls were made to the HRA following the email contact in the previous audit to query what was being asked, whereas none were reported during the course of this study. It is interesting to note that the phase I registrations and ‘other’ registration rates were higher compared with the first audit, but the device registrations were down slightly. While it is encouraging that more registrations occur following a simple email contact, the ambition is not to have to follow-up in this way.
The response rate from investigators and sponsors was also encouraging especially as most responses were received within a week. Overall the responses and reasons given for not registering were in line with other studies.13 It was concerning that 20 of the 194 emails sent were undeliverable, indicating out of date contact information in the HARP database. A number of respondents also claimed that they had picked the wrong box on the application form, again showing that the information contained within HARP is not always accurate. Of the other respondents, 65 trials were either registered or committed to register soon after receiving our email, while another 18 thought they had registered (although these were not on the approved registries), making up 42% of the studies contacted by email. Unfortunately no answer was received for 38% (73 trials) contacted by email and 8% (15 trials) thought that they no longer needed to register as the trial was not eventually conducted. While we agree that the ethical argument for registering a trial might not be as strong for trials that were never started, we do think that such trials should still be registered with a brief explanation as to why the trial was not conducted so as to avoid future researchers or systematic reviewers trying to track down trial results that never existed. However, of most concern was the study that stated it would not register; this is an issue that probably should have been discussed by the ethics committee when they originally reviewed the trial.
A wider issue of note concerns the 10 studies that investigators claimed were not clinical trials and therefore did not require registration. Although it is not currently a UK policy that studies not in the top four categories of the IRAS filter question two are registered, it is difficult to see how this can be justified ethically. While clinical trials and especially CTIMPs represent the most medically risky studies as far as participants are concerned, research money and effort can also be wasted by not adequately reporting the existence of other types of studies as well.23–25 This is an issue that needs further consideration, and here it is encouraging that organisations such as the ‘Research Registry’26 exist that enable research of any type to be registered.27
The HARP database includes full copies of the REC application form filled out by the research team with two key questions regarding research registration. Question A5-1 asks for research reference numbers including ‘registry reference numbers’ and gives a variety of options and types of reference numbers along with the text:
The UK Policy Framework for Health and Social Care Research sets out the principle of making information about research publicly available. Furthermore Article 19 of the World Medical Association Declaration of Helsinki adopted in 2008 states that ‘every clinical trial must be registered on a publicly accessible database before recruitment of the first subject’; and the International Committee of Medical Journal Editors (ICMJE) will consider a clinical trial for publication only if it has been registered in an appropriate registry
The same text is replicated many pages later in the ‘Publication and Dissemination’ section of the form where question A50 asks ‘Will the research be registered on a public database?” provides ‘yes’ and ‘no’ boxes, and then asks for details especially if the ‘no’ box is checked. Although the guidance notes do remind researchers to also add registry numbers in section A5-1, the replication in IRAS is perhaps unfortunate. RECs have been encouraged to pay special attention to these sections and explicitly ask researchers about registration, but given that these sections were empty or included a variety of ambiguous numbers including local reference numbers and insurance numbers, this may not be happening. Indeed the fact that a manual search needed to be used alongside the information contained within these sections for 295 studies demonstrate that this data field is not being appropriately populated within the HARP database.
Putting the full trial title, or often the abbreviated trial title, into the Google search engine proved surprisingly effective for identifying registered studies, and there were only 10 cases where this did not work (and registration was subsequently confirmed by email). This is again a positive finding as it means that trial details can be found by non-expert searchers using a popular and accessible search engine. However, it is concerning that this search helped to identify 36% (295 out of 802) of registered trials with incorrect or absent registration numbers in the HARP database. Although registration numbers may not legitimately be available at the time of REC review, it would be fairly trivial to update RECs either in the response letter to the REC review, or through a subsequent minor amendment. This is an area of improvement that could be looked at, perhaps by requiring an amendment once the registration is confirmed.
One limitation of this audit was only including clinical trials that had been approved in a 6 month time frame. This was based on a pragmatic attempt to limit the audit to about 1000 clinical trials in order to determine the practicality of the method and produce a baseline figure. If this audit is to be repeated on a regular basis more resources would be needed to deal with the couple of thousand clinical trials that are reviewed by UK RECs each year. The incompleteness of HARP records coupled with the presence of invalid email addresses also limited the information that could be obtained on each trial, but future work could attempt to determine alternative contacts within sponsoring organisations to obtain definitive data on each and every trial. A further analysis of the unregistered trials could also be interesting as a way of determining whether there are any specific types of trials that are more likely not to register.
The study reported here represents the first systematic attempt to compare records of clinical trials held by a national regulator with publicly accessible trial registries. Registration rates have improved from initial audit figures provided by the HRA (figure 2), and it is heartening to see more evidence of a cultural change within the trialist community towards greater registration.10 However, to date, the research ethics service has adopted the approach of encouraging greater trial registration through education rather than sanctioning Chief Investigators or Sponsors who do not register trials. It is likely that this situation may soon change based on recommendations made by the UK House of Commons Science and Technology Select Committee (Commons) in their report on Clinical Trials Transparency published in October 2018.20 The committee recommended that measures be put in place to ensure 100% of clinical trials get registered. It is difficult to see how this target can be achieved without a more complete audit modelled on the one described here, followed by organisations such as the HRA considering the use of sanctions with sponsors or investigators who are found not to have registered their studies.
Contributors CD and SB designed and conducted the audit and provided an initial report to the HRA. SEK re-analysed the results and wrote the paper.
Funding The initial audits were funded by the Health Research Authority.
Competing interests CD and SB were employed by the HRA during the audit, although CD has since left the HRA. SEK chairs an HRA research ethics committee, is a member of the HRA’s Confidentiality Advisory Group and is an academic member of both the HRA’s National Research Ethics Committee Advisors Panel (NREAP) and its Transparency Forum. He is on the board of advisors to the Research Registry.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The initial audit reports that this study is based on are available on the HRA website.
Patient consent for publication Not required.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.