Follow-up by mail in clinical trials: does questionnaire length matter?
Introduction
To reliably detect modest but nevertheless plausible and clinically important treatment effects, clinical trials must enroll many thousands of patients [1]. In large trials where outcome assessment is possible using a questionnaire, it may be more cost-effective to mail them to patients than to conduct interviews in-person. However, nonresponse to mailed questionnaires reduces the effective sample size and can introduce bias [2].
The Medical Research Council (MRC) Corticosteroid Randomization After Significant Head Injury (CRASH) Trial aims to enroll and follow-up 20,000 head-injured patients over 5 years [3]. Long-term outcome is assessed using the Glasgow Outcome Scale (GOS), the most commonly used measure in head injury trials [4]. The GOS is an ordinal outcome scale with five levels: good recovery, moderate disability, severe disability, vegetative state and dead. An extended version of the GOS has also been developed that includes eight levels [5]. During the pilot phase of the CRASH Trial, questionnaires were developed for the GOS and the extended GOS that could be self-administered or completed by a caregiver [6]. Both versions were shown to be reliable in test–retest studies and in comparison with telephone administration by trained nurses. The questionnaire for the GOS comprises seven questions on a single page, and the questionnaire for the extended GOS comprises 15 questions over three pages. Having the choice of two questionnaires raised an important question—when mailed, which questionnaire will provide the highest response rate, thereby maintaining sample size and reducing bias?
To inform the choice and design of follow-up questionnaires for large clinical trials, we conducted a systematic review and meta-analysis of randomized controlled trials evaluating the effect of questionnaire length on response rate, with the aim of describing this relationship. We hypothesized that the propensity to respond to a questionnaire is inversely related to its length. The focus of this paper is on the results of the systematic review. They may be relevant to other research in which outcomes can be assessed using mailed questionnaires.
Section snippets
Systematic review
To examine the effect of questionnaire length on response rate, we conducted a systematic review and meta-analysis of randomized controlled trials of shorter versus longer questionnaires. A systematic search was initially made for all randomized controlled trials of any method to influence response to a postal (mailed) questionnaire [7]. There was no restriction by language, questionnaire topic or study population. We searched 14 electronic bibliographic databases (Table 1), the reference lists
Results
The systematic search for randomized controlled trials of methods to influence response to mailed questionnaires yielded 26,937 records of potentially relevant reports. After screening records and obtaining copies of the reports considered to be relevant for further inspection, a total of 251 reports were found to contain one or more such randomized controlled trials [7].
Of the 251 reports, 27 were about trials evaluating the effect of questionnaire length on response rates. Eight reports
Discussion
This systematic review and meta-analysis of randomized controlled trials evaluating the effect of questionnaire length on response supports the hypothesis that response is inversely related to length. It also suggests that the effect on response is much greater for very short questionnaires. Before we consider the implications of these results on the choice and design of follow-up questionnaires in clinical trials, several methodological issues with a bearing on the validity of the results must
Conclusion
Response can be increased by using a shorter questionnaire. Moderate changes to the length of shorter questionnaires will be more effective than moderate changes to the length of longer questionnaires. If a choice of follow-up questionnaire exists for a clinical trial, the shorter one should be used. If a new follow-up questionnaire is to be designed, it should be made as short as possible without compromising the data collection requirements of the trial. Within health research, further trials
Acknowledgments
We are grateful to Paul Dorman, Eliv Lund and Matthew Murawski for providing additional information about their studies. We would also like to thank the referees and Graham Try whose comments helped to improve this paper.
References (16)
- et al.
Meta-analysis in clinical trials
Control. Clin. Trials
(1986) - et al.
Why do we need some large, simple randomized trials?
Stat. Med.
(1984) - et al.
Principles of exposure measurement in epidemiology
- et al.
Corticosteroids in head injury—the CRASH trial
J. Accid. Emerg. Med.
(1999) - et al.
Size and quality of randomised controlled trials in head injury: review of published studies
BMJ
(2000) - et al.
Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use
J. Neurotrauma.
(1998) - et al.
Reliability of postal questionnaires for the Glasgow Outcome Scale
J. Neurotrauma.
(2002 (Sep)) - et al.
Increasing response rates to postal questionnaires: systematic review
BMJ
(2002)
Cited by (138)
Adaptive Measurement of Change: A Novel Method to Reduce Respondent Burden and Detect Significant Individual-Level Change in Patient-Reported Outcome Measures
2022, Archives of Physical Medicine and RehabilitationIt is a match! How donors and nonprofit organizations come together on a matching donation platform
2024, Journal of Philanthropy and MarketingMethods to increase response to postal and electronic questionnaires
2023, Cochrane Database of Systematic Reviews