Abstract
Purpose
The aim of this study was to reduce respondent burden and decrease data errors in patient-reported outcomes (PROs) intended for a postal survey measuring health and lifestyle factors that may affect quality of life in older people with multiple sclerosis (MS).
Methods
Participants (n = 18) were recruited from a database of outpatient visits. Using the qualitative diagnostic method, cognitive debriefing, participants completed five standardized questionnaires; Frenchay Activities Index, Barthel Index, Simple Lifestyle Indicator Questionnaire, EuroQoL EQ-5D and Personal Resources Questionnaire 2000. PRO item issues and respondent behaviors such as skipping items were recorded. Data collection was an iterative process whereby difficulties experienced by 2–3 subjects were used to modify the survey for following respondents until data saturation was reached.
Results
Most respondents had serious difficulties with at least one PRO item. Response errors fell into three main categories: (1) respondents did not read instructions and completed the item incorrectly, (2) respondents did not understand the question and required examples to clarify and (3) respondents felt that the pre-determined response options did not apply to them. PRO reformatting, minor modifications to item wording and addition of item examples improved precision and reduced respondent burden.
Conclusion
Our findings support the notion that methods such as cognitive debriefing help improve precision of self-reported measures in a special population such as ours.
Abbreviations
- BI:
-
Barthel Index
- FAI:
-
Frenchay Activities Index
- MS:
-
Multiple Sclerosis
- PRQ2000:
-
Personal Resources Questionnaire version 2000
- PROs:
-
Patient-Reported Outcomes
- QoL:
-
Quality of Life
- SLIQ:
-
Simple Lifestyle Indicator Questionnaire
References
Leidy, N. K., & Vernon, M. (2008). Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics, 26(5), 363–370.
Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., et al. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value in Health, 12(4), 419–429.
United States Federal Drug Administration Agency. (2006). Guideline for Industry. Patient reported outcomes measures: Use in medical product development to support labelling claims. Federal Register.
Mullin, P. A., Lohr, K. N., Bresnahan, B. W., & McNulty, P. (2000). Applying cognitive design principles to formatting HRQOL instruments. Quality of Life Research, 9(1), 13–27.
Dillman, D. A., Sinclair, M. D., & Clark, J. R. (1993). Effect of questionnaire length, respondent-friendly design, and a difficult question on response rates for occupant-addressed census mail surveys. Public Opinion Quarterly, 57, 289–304.
O’Sullivan, S. B. (2007). Multiple sclerosis. In S. B. O’Sullivan & T. J. Schmitz (Eds.), Physical rehabilitation. Assessment and treatment. Philadelphia: F.A. Davis Company.
Mallinson, S. (2002). Listening to respondents: A qualitative assessment of the Short-Form 36 Health Status Questionnaire. Social Science and Medicine, 54(1), 11–21.
Hak, T., Willems, D., van der Wal, G., & Visser, F. (2004). A qualitative validation of the Minnesota living with heart failure questionnaire. Quality of Life Research, 13(2), 417–426.
Fisk, J. D., Brown, M. G., Sketris, I. S., Metz, L. M., Murray, T. J., & Stadnyk, K. J. (2005). A comparison of health utility measures for the evaluation of multiple sclerosis treatments. Journal of Neurology, Neurosurgery and Psychiatry, 76(1), 58–63.
Clarke, H. H., & Schober, M. F. (1992). Asking questions and influencing answers. In J. M. Tanur (Ed.), Questions about questions Inquiries into the cognitive bases of surveys. New York: Russell Sage Foundation.
Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.
McColl, E., Meadows, K., & Barofsky, I. (2003). Cognitive aspects of survey methodology and quality of life assessment. Quality of Life Research, 12(3), 217–218.
Turnbull, J. C., Kersten, P., Habib, M., McLellan, L., Mulee, M. A., & George, S. (2000). Validation of the Frenchay Activities Index in a general population aged 16 years and older. Archives of Physical Medicine and Rehabilitation, 81(8), 1034–1038.
Yeo, D., Faleiro, R., & Lincoln, N. B. (1995). Barthel ADL index: A comparison of administration methods. Clinical Rehabilitation, 9(1), 34–39.
Godwin, M., Streight, S., Dyachuk, E., van den Hooven, E. C., Ploemacher, J., Seguin, R., et al. (2008). Testing the Simple Lifestyle Indicator Questionnaire: Initial psychometric study. Canadian Family Physician, 54(1), 76–77.
Brazier, J., Jones, N., & Kind, P. (1993). Testing the validity of the Euroqol and comparing it with the SF-36 health survey questionnaire. Quality of Life Research, 2(3), 169–180.
Weinert, C., & Brandt, P. A. (1987). Measuring social support with the Personal Resource Questionnaire. Western Journal of Nursing Research, 9(4), 589–602.
Korner-Bitensky, N., Wood-Dauphine, S., Siemiatyki, J., Shapire, S., & Becker, R. (1994). Health-related information postdischarge: Telephone versus face-to- face interviewing. Archives of Physical Medicine and Rehabilitation, 75(12), 1287–1296.
Groenvold, M., Klee, M. C., Sprangers, M. A., & Aaronson, N. K. (1997). Validation of the EORTC QLQ-C30 quality of life questionnaire through combined qualitative and quantitative assessment of patient-observer agreement. Journal of Clinical Epidemiology, 50(4), 441–450.
Turner, R. R., Quittner, A. L., Parasuraman, B. M., Kallich, J. D., Cleeland, C. S. FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Instrument development and selection issues. Value in Health, 10(SUPPL. 2), S86–S93.
Ojanen, V., & Gogates, G. (2006). A briefing on cognitive debriefing. Good Clinical Practice Journal, 13(12), 25–29.
Acknowledgments
Research support was provided by the Eastern Health Authority, the Memorial University Dr. W. Ingram Award and Faculty of Medicine Neurology Research Fund.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Ploughman, M., Austin, M., Stefanelli, M. et al. Applying cognitive debriefing to pre-test patient-reported outcomes in older people with multiple sclerosis. Qual Life Res 19, 483–487 (2010). https://doi.org/10.1007/s11136-010-9602-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-010-9602-z