Article Text
PDF
Abstract
Objectives To analyse the completeness of reporting of blinding in randomised controlled trials (RCTs) of interventions in anaesthesiology, the actual blinding status of various persons associated with an RCT and trial authors’ interpretation of blinding terminology related to RCTs.
Methods This was a methodological study and a cross-sectional survey. We analysed reporting related to blinding in published RCTs of interventions published in seven highly cited anaesthesiology journals from 2014 to 2016 and registered protocols in ClinicalTrials.gov. We surveyed corresponding authors of included RCTs about their definitions of blinding. The primary outcome was the number of RCTs that explicitly described who was blinded in a trial. Secondary outcomes were definitions of blinding terminology in the trials; trial authors’ interpretation of blinding terminology; discrepancies in the blinding description within registered protocols and between registered protocols and publications.
Results Out of 622 analysed RCTs, 38% were not explicitly described as either open label or blinded studies and 10% did not report any information about blinding or lack of blinding. Only one manuscript fully reported the status of blinding for various individuals that may be involved with a trial. The most common descriptor was that a trial was double-blind. We found discrepant information regarding blinding in the majority of registered protocols. Even when there were no discrepancies in the registration, we found discrepancies in the reporting of blinding between the majority of registered protocols and published manuscripts. The survey of authors (40 responses from 231 eligible authors; 17% response rate) of analysed RCTs showed that they differed in how they defined different levels of blinding in trials.
Conclusions Reporting of the blinding status of key individuals involved in analysed anaesthesiology RCTs was insufficient. For reporting guidelines, peer reviewers and editors should insist on clear information on who was blinded in a trial instead of using the term ‘double-blind’ for different blinding practices.
- Clinical trials
- statistics & research methods
- epidemiology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We analysed a large group of randomised controlled trials published in seven high-impact anaesthesiology journals.
Reporting about blinding was insufficient in the majority of analysed trials.
The term ‘double-blind’ is ambiguous and researchers interpret it in different ways.
We focused on anaesthesiology trials as a specific research field.
Future studies should monitor whether reporting about blinding in trials will improve.
Introduction
The available evidence about the efficacy and safety of interventions in medicine includes observational studies and studies with a randomised experimental design, that is, randomised controlled trials (RCTs).1 RCTs are considered the highest level of primary evidence for analysing medical interventions because randomisation ensures the allocation of participants by chance, and not by choice, which minimises bias.2 Apart from the method of allocating participants into study arms, there are other methodological aspects of an RCT that can contribute to a risk of bias, including performance bias (blinding of participants and personnel) and detection bias (blinding of outcome assessors).3
Blinding is very important for the validity of RCT results. Multiple studies reported significant differences in the effect size of treatment estimates depending on the use of blinding of key individuals.4–8
The term ‘double-blind’ is sometimes taken as a token of the validity of an RCT.9 However, it has been shown that the definitions of blinding may vary. Devereaux et al showed that both physicians and textbooks provided highly inconsistent interpretations of the blinding terminology used in clinical trials and suggested that the current ambiguous terminology should be replaced by specific descriptions of who was blinded.10 According to the Consolidated Standards of Reporting Trial (CONSORT) checklist for reporting RCTs, the following should be reported for blinding: ‘If done, who was blinded after assignment to interventions (eg, participants, care providers and those assessing outcomes) and how’.11
The aim of this study was to analyse the completeness of the reporting of blinding in RCTs of interventions in the field of anaesthesiology, the actual blinding status of the various persons involved in an RCT, information about the blinding reported in the registered trial protocol and the trial authors’ interpretation of the blinding terminology related to RCTs. The study focused specifically on the field of anaesthesiology because pain is a subjective outcome and there is no objective method for analysing pain. The analgesic placebo effect has been well documented.12 In the area of placebo effects, expectations play a major role, triggering a cascade of endogenous opioids and non-opioids, and altering the experience of pain.12 For this reason, the field of anaesthesiology and pain research is likely biased if blinding is not used adequately.
Methods
Study design
We conducted a methodological cross-sectional study of published RCTs and registered data from their protocols in public trial registries. Additionally, we surveyed the trials’ corresponding authors.
Ethics
The research protocol for the survey of trials’ corresponding authors was approved by the Ethics Committee of the University of Split, School of Medicine (Klasa: 003-08/18-03/0001, Ur. br.: 2181-198-03-04-18-0049; date: October 12, 2018). All invited study participants received information that the study was approved by the Ethics Committee. The participants’ continuation of the survey from the invitation email was considered as consent to participate in the study. The data were fully anonymised for analysis.
Inclusion criteria
We included RCTs of interventions published from January 2014 to December 2016 in the seven first-quartile journals from the Anaesthesiology category in Journal Citation Reports (JCR). Based on the 2015 JCR Journal Impact Factor (JIF), those seven journals were (in alphabetic order) Anaesthesia, Anesthesia & Analgesia, Anesthesiology, British Journal of Anaesthesia, European Journal of Anaesthesiology, Pain, and Regional Anesthesia and Pain Medicine. We excluded RCTs published as letters to the editor.
Search
We searched PubMed by combining the journal name, the 2014–2016 time frame, and the publication type filter for RCTs. We exported titles and abstracts into the EndNote reference management software (Clarivate Analytics, Boston, Massachusetts, USA). Two authors independently analysed titles and abstracts to screen them for eligibility. We retrieved potentially eligible full-text manuscripts and two authors screened them independently. We resolved any disagreements through discussion.
Outcomes
The primary outcome was the number of RCTs that explicitly described who (which individuals or group/s of individuals) was blinded in a trial. Secondary outcomes were the definitions of the blinding terminology in the trials (how the authors defined‘single-blinding’, ‘double-blinding’ and ‘triple-blinding’ in the manuscript), trial authors’ interpretation of the blinding terminology related to RCTs, as well as discrepancies in the blinding description within registered protocols, and between registered protocols and publications.
Data extraction
We extracted data into a Microsoft Excel sheet (Microsoft, Redmond, Washington, USA) after the data extraction table was piloted with 10 studies. Data from each trial were extracted independently by two authors (three authors participated in data extraction: AP, KB and MK). Independent extractions were compared for consistency and discrepancies were resolved via consensus.
For each publication, we analysed the abstract, the full body of the manuscript (and online supplementary files if the authors indicated that additional details about the methods were in online supplementary files) to see whether and in which part of the manuscript, there was an explicit mention that the trial was blinded/masked. We then analysed the Methods section of each included RCT and extracted information about the blinding terminology (‘single-blind’, ‘double-blind’, ‘triple-blind’ or no such definition) and whether the trial authors explicitly elaborated on who was blinded in a trial.
Supplemental material
Supplemental material
Supplemental material
Supplemental material
We checked whether the RCTs explicitly mentioned using the CONSORT checklist for reporting or mentioned CONSORT as a reference. We analysed whether there were any discrepancies in the reporting of blinding between abstracts and the body of the manuscript. If the abstract described the study as ‘double-blind’, but the body of the manuscript provided more details about who was blinded, we considered this a discrepancy. We also checked for the occurrence of the term ‘double-dummy’ in the RCTs to assess the blinding in comparisons of drugs that are administered by different routes.13
We analysed whether the RCTs mentioned trial protocol registration, and, if yes, then we recorded the protocol registration number and the name of the registry. We excluded trials registered after the trial was open for recruitment from further analysis. For trials registered before participants’ recruitment, we analysed whether there was any description of blinding in the registered protocol. If we found a description of blinding, we compared whether there were any discrepancies between the details about blinding reported in the registered protocol and the published manuscript(s). We also searched whether the RCTs mentioned a protocol as an online supplementary file or as a publication in a scholarly journal.
Author survey
For the author survey, we extracted email addresses of corresponding authors. We contacted corresponding authors of the included trials via email, wherein they received information about the study and a list of questions. One author (DB) sent all the emails. After the initial email invitation, potential participants received four subsequent reminders 1 week apart if they did not respond or indicated that they did not wish to participate. All surveys and reminders were sent between November 1, 2018 and January 9, 2019.
We conducted the survey based on the questions described previously.14 Participants received the following questions together with the title of their trial:
For authors who only indicated that the study was blinded, double-blind, single-blind, triple-blind or provided no further details: ‘Who exactly was blinded in the trial (six categories of participants, as used by Haahr and Hróbjartsson14: (i) participants, (ii) healthcare providers such as physicians or nurses taking care of a participant, (iii) data collectors, (iv) outcome assessors, (v) data analysts, (vi) manuscript writers)’—categorical answers.
What were the authors’ personal opinions about the definitions of single-blind, double-blind and triple-blind by providing them the same categorical answers from above, as well as the response of ‘any two types of trial participants’.14
The authors were asked to assess the frequency of the publication of double-blind trials in the literature adhering to their preferred definition. For example, if a participant characterised ‘double-blind’ to represent blinding of participants and personnel, then author was subsequently asked to assess the frequency of ‘double-blind’ trials in which participants and personnel were blinded. The question did not refer to the publications of the participants, but their estimation of the use of the term in the literature.
We emphasised to the survey participants that only anonymised data would be published. The exact text of the survey is presented in the online supplementary file 1. In cases when we had identical corresponding authors for more than one manuscript included in the study, we contacted them for the most recent manuscript only. Two authors (DB, LP) independently categorised the responses obtained via the survey.
Analysis
We used descriptive statistics including frequencies and percentages, as well as medians and 95% (CI) confidence intervals. Data were analysed using MedCalc statistical software, V.15.2.1 (MedCalc Software bvba, Ostend, Belgium).
Results
Descriptors about blinding
We analysed 622 RCTs; the study flowchart is shown in figure 1. The list of included studies is presented in the online supplementary file 2. In the description of the study design, 29 (4.7%) trials were explicitly described as non-blinded or open label, whereas 357 (57%) were explicitly described as blinded, using terms ‘blinded’, ‘single-blinded’, ‘double-blinded’, ‘triple-blinded’ or ‘masked’ in the description of study design. The study design of the remaining 236 (38%) trials was not explicitly described as blinded or non-blinded.
Study inclusion flowchart. RCT, randomised controlled trial.
The information about the blinding status was found in 116 (19%) titles; 283 (45%) abstracts and 470 (76%) methods sections. The most common descriptor for blinding in titles (table 1), abstracts (table 2) and methods sections (table 3) was ‘double-blind’.
Descriptors for blinding status of the study in titles of analysed trials (N=116)
Descriptors about blinding found in abstracts of analysed trials (N=294 descriptors from 283 abstracts)
Categorised statements about blinding found in the methods section of analysed trials (N=470)
Regarding the blinding versus masking terminology, the most common term used in the analysed trials was ‘blinding’ (n=477, 76%). Masking as a term alone was used in 9 (1.4%) trials, whereas in 45 (7.2%) trials, both blinding and masking terms were used. In 91 (15%) trials, neither the word blinding nor masking was used. The term ‘double-dummy’ was found in seven trials and ‘triple-dummy’ in one trial.
Blinding of individuals involved in trials
We analysed 531 trials to determine who was blinded in the trials. Concerning these trials, we found at least some descriptors for blinding, whereas the remaining trials were excluded from this analysis because they were described as open label or were not described as either blinded or open-label. Table 4 presents the prevalence of the blinding of various individuals that may be involved in a trial. For all categories of individuals in the majority of trials, it was unclear whether these individuals were blinded.
Prevalence of blinding of individuals involved in trials described as blinded (N=622)
In 89 (17%) of 531 trials (14% from the overall sample of 622 trials), it was explicitly reported whether the three key groups of individuals, that is, the participants, healthcare providers and data collectors, were blinded or not. This percentage seemed to improve over the analysed time frame, as it was 10% in 2014 (25/203), 15% in 2015 (31/203) and 26% in 2016 (33/125).
Among the analysed trials, 47 (8%) explicitly mentioned that they used the CONSORT checklist for reporting or cited CONSORT as a reference.
Discrepancies between abstracts and the body of the manuscript
We found discrepancies in the reporting of blinding between abstracts and the body of the manuscript in a single trial, which reported in the abstract that the study was double-blind, but that is was single-blind in the body of the manuscript.
Reporting of blinding in registered protocols
There were 384 (62%) trials that reported trial registration and a unique registration identifier. After searching for those registered protocols online, we were able to find 315 of those 384 (82%), we were unable to retrieve the rest from our search using the reported trial registry identifier. Of the 315 registered protocols we managed to access, 174 (55%) were registered prospectively and as many as 141 (45%) were registered after the study start date reported in a clinical trial registry.
We conducted a detailed analysis of 174 prospectively registered trials in registries. The majority of the trial protocols were registered in ClinicalTrials.gov (n=157; 90%). Two studies from our cohort were registered on ClinicalTrials.gov as observational studies and did not have information about blinding. The remaining 155 trials registered on ClinicalTrials.gov had 14 different categories describing the design characteristics of the study in the Study design section, which contains the mandatory registration field for the selection of the masking type (table 5).
Descriptors in the study design/masking field in the analysed registered trial protocols on ClinicalTrials.gov (N=157)
In the registered protocols on ClinicialTrials.gov, the most common descriptor in the Masking subsection was ‘Quadruple (participant, care provider, investigator, outcomes assessor)’ (table 5). Besides this field, we also analysed the full text of the registered protocol to find any other descriptions of blinding/masking. Among 157 registered protocols, 92 did not have any other terminology related to blinding/masking other than the information in the mandatory masking field, and therefore no within-protocol discrepancies. In the remaining 65 registered trial protocols, 18 protocols had no discrepancies, whereas 47 (72%) registered protocols had additional information about the blinding that was different from the descriptor provided in the masking field. Registered protocols that described the trial as quadruple-blind or triple-blind in the Masking registration field accounted for the majority of discrepancies, but the description of the trial as double-blind was also in another part of the registered protocol.
In the 17 protocols registered in other trial registries (table 6), we found only one discrepancy in a trial registered in the Netherlands Trial Register (NTR) which reported both ‘single masking’ and ‘double-blind’ (table 6). The remaining 16 registered protocols did not have any discrepancies related to blinding.
Descriptors of blinding/masking in clinical trial registries other than ClinicalTrials.gov
Among 126 registered trials that did not have discrepancies in the registered protocols, for 88 (70%) trials, we found differences between the registered protocol and published manuscript in the description of blinding. The most common differences were in the description of who was blinded, the extent of blinding and whether there was any blinding at all (table 7). We did not find trial protocols published as online supplementary files or as a manuscript in a scholarly journal.
Differences between the registered protocol and manuscript in the group of trials that did not have discrepancies within the registered protocol itself (trials: N=88)
Survey results
For 362 trials that were described as double-blind, there were 275 unique corresponding authors and 264 email addresses available. There were 33 undelivered emails because the addresses were no longer in use. From the remaining 231 corresponding authors, we received 40 responses (17% response rate). Authors’ responses regarding individuals that were blinded in their trials, as well as their opinions about who was blinded in single-blinded, double-blinded and triple-blinded trials are shown in table 8. One participant who answered the question about blinding in their trial provided a personal definition for the single-blind trial, but stated the following for the remaining questions: ‘I cannot answer the rest of the questions. I don't use “double-blind”or “single-blind” any longer, as I think they are ambiguous. I adhere to Annals of Internal Medicine guidelines that advises authors to avoid those terms, but to specifically note who was blinded’.
Surveyed corresponding authors’ responses regarding who was blinded in their trials of those described as double-blinded and what is their personal definition of who is blinded in a single-blind, double-blind and triple-blind trial (total participants: N=40)
Whereas 34 of 40 (85%) authors responded that participants were blinded in their own double-blinded trial, all authors who responded to the second question stated that participants are blinded in a double-blind trial. Table 8 shows the summary of responses, and the online supplementary file 3 contains detailed anonymised responses. For a single-blind trial, surveyed authors chose at least one from five of six individuals as targets for blinding; only manuscript writers were never considered as individuals who could be blinded in a trial. When asked about their personal definition, respondents chose all six types of individuals given choices of blinded individuals in double-blind and triple-blind trials. Outcome assessors, data analysts and manuscript writers were more commonly mentioned as individuals who could be blinded in a triple-blind trial (table 8).
In addition, 25 participants estimated the percentage of how often their personal definition of a double-blind trial is used in trials they published: this percentage ranged from 20% to 100%, with a median of 80% (95% CI: 60% to 80%).
Reporting checklist
This study was reported in line with the Strengthening the Reporting of Observational studies in Epidemiology checklist, which is enclosed as an online supplementary file 4.
Discussion
We found that only a quarter of analysed trials reported explicitly whether three key groups of individuals were blinded. One-fifth reported information about blinding or the lack of blinding in the title. The most common description related to blinding, both in the title and in the manuscript, was that the study was double-blind. Regarding terminology, ‘blinding’ as a term was used in the majority of trials, whereas only a few trials used the term masking. In the majority of trials described as blinded, it was unclear whether participants and other relevant individuals were blinded or not. For less than half of the trials, we identified that they registered their protocols in public trial registries, and in the majority of those registered protocols, we found discrepant information about blinding—both in different fields in the registered protocol and between the registered protocol and published manuscript.
In 2006, Haahr and Hróbjartsson analysed 200 random RCTs and showed that the blinding status of participants, healthcare providers and data collectors was explicitly reported in 2% of those trials.14 In our study this finding was better; we found that the blinding status of these three groups of individuals was explicitly reported in 14% of the analysed trials.
Twelve years ago, it was indicated that 26% of trials reported no information relevant to blinding beyond indicating that the trial was ‘double-blind’.14 We found that 57% of analysed trials were explicitly described as blinded, using the terms ‘blinded’, ‘single-blinded’, ‘double-blinded’, ‘triple-blinded’ or ‘masked’. Our study shows that warnings aimed towards journal editors15 were not successful in the efforts to reduce the use of the ambiguous term double-blind.
Discrepancies in blinding descriptors within registered trial protocols and between registered protocols and published manuscripts
One of the striking findings of our study was related to the discrepancies in blinding/masking information within different registration fields for registered trial protocols and between those registered protocols and published manuscripts.
First, we were able to identify registered protocols for only about half of the trials in our cohort, either because the authors did not report trial registration in their manuscripts or because the reported registered protocol was not retrievable. Since 2005, the International Committee of Medical Journal Editors (ICMJE) recommends that journals require prospective registration of clinical trials, that is, before or at the time of first patient enrolment.16 Therefore, it is unfortunate that so many recently published trials do not report trial registration in the manuscript, but this is not an isolated finding. In our recent study of osteoarthritis trials published between 2012 and 2017, we found that 57% of 334 analysed trials reported their registration.17 Another recent study showed that 28% of trials published in six high-impact general medicine journals did not comply with the ICMJE policy on prospective trial registration.18 Therefore, even though we analysed high-impact journals in the field, our results are similar to previous findings, which showed that many trials still do not report trial registration and adhere to ICMJE policy, even though they were published recently and in high-impact journals.
Furthermore, the analysis of registered trial protocols that we were able to identify showed that 45% of them were registered after the study has already started, judging by the study start date that was reported in the trial registry. When we analysed the information about blinding/masking, we found that the majority of registered protocols had discrepant information about the blinding of the study in different registration fields. Our analysis of trials in which we did not find discrepancies between registration fields showed that the majority of those trials had differences between the reporting of blinding in a registered protocol and a published manuscript.
ClinicalTrials.gov was the most commonly used clinical trial registry. This is in line with the finding that ClinicalTrials.gov contains the majority of global trial registrations.19 Within the registry, the mandatory field called ‘masking’ offers study authors a choice of five options in a menu. The first four options in the menu are individuals involved in a trial: participant, care provider, investigator and outcomes assessor. The fifth option is ‘none (open label)’. Study authors are invited to ‘check all roles that are masked or check none (open label)’. Below this menu is a blank field called ‘masking description’, in which trialists can fill out additional information about blinding. Based on the number of participants who are checked as blinded, a trial will get a descriptor about being single-blinded, double-blinded, triple-blinded or quadruple-blinded. The most common discrepancy within registered protocols was that the study was described as quadruple or triple blind by ClinicalTrials.gov, but trialists themselves called the study ‘double-blind’.
The meaning of ‘double-blind’
The suggestion by Montori et al that the vague term ‘double-blind’ should be abandoned15 was also repeated later in the literature by other authors.14 Even before that, Devereaux et al reported that hospital physicians used 17 different definitions of the meaning of double-blind.10
Our survey showed that trial authors have varying definitions of who should be blinded in a single-blind, double-blind and triple-blind trial. Authors’ response about who was blinded in their own double-blind trial did not always match their response about who should be blinded in a double-blind trial generally. Based on the participants’ responses that the majority of published trials correspond to their personal definition of a double-blind trial, it appears that the surveyed researchers are not aware of the high heterogeneity of the definitions of blinding in clinical trials. This is an additional argument for ceasing the use of ambiguous terminology such as ‘single-blind’, ‘double-blind’ and ‘triple-blind’, and to require that authors should report specifically who was blinded in their study.
Limitations
Our study had several limitations. We limited our analysis to a recent period and to seven high-ranking journals. We did not scrutinise in detail additional online material, such as supplements and appendices, where, theoretically, additional details about blinding could have been located. But we did analyse all mentions of blinding in the analysed manuscripts and we did not find a single case where the authors indicated that additional details about blinding were available in a supplement.
We used PubMed to search for articles instead of journals’ websites and we used a simple search consisting of the journal name, publication type and chosen time frame. We used the JIF to define high-impact journals. The use of the JIF as a measure of ‘quality’ has been criticised in the research community, specifically due to the asymmetry between the numerator and the denominator, differences across various disciplines, the insufficient citation window, the skewness of the underlying citation distributions and the opaque method of its calculation.20 However, there is no perfect bibliometric indicator and we used it because the JIF is highly prevalent, and most researchers are familiar with this metric.21 Also, we did not analyse all medical fields but focused on one field of interest. Our survey had a response rate of 17%, which carries a high risk of non-responder bias.
Another thing that would be worthwhile to explore is the journals’ instructions for authors in terms of reporting requirements. However, the only proper way to study the influence of instructions for authors on reporting in a manuscript would be to conduct it prospectively, having in mind the current instructions for authors and current submissions.
It could be argued that perhaps we should not have judged an abstract as having a discrepancy that reported therein that a study was ‘double-blind’, but the body of the manuscript reported more details about who was blinded. However, the reporting extension to CONSORT for abstracts explicitly requires authors to report in the abstract (quote): ‘Whether or not participants, care givers, and these assessing the outcomes were blinded to group assignment’.
Therefore, if authors want to report their study adequately, adherence to the relevant reporting guideline would also preclude the usage of the term ‘double-blind’ in an abstract.22
Recommendations
Authors of RCTs should stop using the ambiguous term ‘double-blind’, and editors and peer reviewers can help in achieving this by requiring clear reporting about the blinding. Additionally, editors and peer reviewers can require authors to provide a CONSORT checklist for both the abstract (in line with the CONSORT extension for abstracts) and the body of the manuscript. In this way, authors of RCTs may be compelled to invest more attention to proper reporting.
Conclusions
Explicit and complete reporting of the blinding status of key individuals involved in RCTs is suboptimal. Anaesthesiology trials rarely report who was blinded and mostly use the ambiguous term double-blind, which is not sufficiently informative for readers. Furthermore, we found many discrepancies about reporting of blinding within registered trial protocols, and between registered protocols and published manuscripts. Our survey of authors showed that authors differed in their opinion of who was blinded in their own trial and who could be blinded in trials with different levels of blinding. Thus, poor reporting and heterogeneous definitions of ambiguous terminology hinder a proper risk of bias assessment and analysis of the impact of blinding of certain groups of participants on the study outcomes. Interventions for improving the reporting about blinding are necessary.
Acknowledgments
The authors are grateful to all the participants who took part in this study. They are also very grateful to Professor Shelly Pranic, PhD, for proofreading final version of the manuscript.
References
Footnotes
Twitter @ana_marusic, @liviapuljak
Contributors Study design; data analysis and interpretation; writing manuscript: LP, AM. Data acquisition: AP, DB, KB, MK, LP. Agreeing to submission of the manuscript; agree to be accountable for all aspects of the work: AP, DB, KB, MK, AM, LP.
Funding This study was funded by the Croatian Science Foundation, grant No. IP-2014-09-7672 ‘Professionalism in Healthcare’, awarded to Professor Ana Marušić.
Disclaimer The funder had no role in the design of this study, during its execution and data interpretation.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data collected during the study are available from the corresponding author on reasonable request.