Skip to main content
SpringerLink
Log in

On assessing responsiveness of health-related quality of life instruments: Guidelines for instrument evaluation

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

A lack of clarity exists about the definition and adequate approach for evaluating responsiveness. An overview is presented of different categories of definitions and methods used for calculating responsiveness identified through a literature search. Twenty-five definitions and 31 measures were found. When applied to a general and a disease-specific quality of life questionnaire large variation in results was observed, partly explained by different goals of existing methods. Four major issues are considered to claim the usefulness of an evaluative health-related quality of life (HRQL) instrument. Their relation with responsiveness is discussed. The confusion about responsiveness arises mostly from a lack of distinction between cross-sectional and longitudinal validity and from a lack of distinction between responsiveness defined as the effect of treatment and responsiveness defined as the correlation of changes in the instrument with changes in other measures. All measures of what is currently called responsiveness can be looked at as measures of longitudinal validity or as measures of treatment effect. The latter ones tell us little about how well the instrument serves its purpose and are only of use in interpreting score changes. We therefore argue that the concept of responsiveness can be rejected as a separate measurement property of an evaluative instrument.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Deyo RA, Inui TS. Toward clinical applications of health status measures: Sensitivity of scales to clinically important changes. Health Serv Res 1984; 19: 275–289.

    Google Scholar 

  2. Kirschner B, Guyatt G. A methodological framework for assessing health indices. J Chron Dis 1985; 38: 27–36.

    Google Scholar 

  3. Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. CAMJ 1986; 134: 889–895.

    Google Scholar 

  4. Guyatt G, Walter S, Norman G. Measuring change over time: Assessing the usefulness of evaluative instruments. J Chron Dis 1987; 40: 171–178.

    Google Scholar 

  5. Guyatt GH, Deyo RA, Charlson M, Levine MN, Mitchell A. Responsiveness and validity in health status measurement: A clarification. J Clin Epidemiol 1989; 42: 403–408.

    Google Scholar 

  6. Fitzpatrick R, Ziebland S, Jenkinson C, Mowat A. Importance of sensitivity to change as a criterion for selecting health status measures. Qual Health Care 1992; 1: 89–93.

    Google Scholar 

  7. Fitzpatrick R, Fletcher A, Gore S, Jones D, Spiegelhalter D, Cox D. Quality of life measures in health care. I Applications and issues in assessment. Br Med J 1992; 305: 1074–1077.

    Google Scholar 

  8. Guyatt GH, Kirshner B, Jaeschke R. Measuring health status: What are the necessary measurement properties? J Clin Epidemiol 1992; 45: 1341–1345.

    Google Scholar 

  9. Fletcher A. Quality-of-life measurements in the evaluation of treatment: Proposed guidelines. Br J Clin Pharmac 1995; 39: 217–222.

    Google Scholar 

  10. Juniper EF, Guyatt GH, Jaeschke R. How to develop and validate a new health-related quality of life instrument. In: Spilker B (ed), Quality of Life and Pharmacoeconomics in Clinical Trials, 2nd ed. Philadelphia: Lippincott-Raven Publishers, 1996; 49–56.

    Google Scholar 

  11. Lohr KN, Aaronson NK, Alonso J, et al. Evaluating quality of life and health status instruments: Development of scientific review criteria. Clinical Therapeutics 1996; 18: 979–992.

    Google Scholar 

  12. Testa MA, Simonson DC. Assessment of quality of life outcomes. N Engl J Med 1996; 334: 835–840.

    Google Scholar 

  13. Guyatt GH, Naylor D, Juniper E, Heyland DK, Jaeschke R, Cook DJ. Users' guides to the medical literature. XII How to use articles about health-related quality of life? JAMA 1997; 277: 1232–1237.

    Google Scholar 

  14. Dijkers M. Measuring quality of life. Am J Phys Med Rehabil 1999; 78: 286–300.

    Google Scholar 

  15. Guyatt GH, Jaeschke R, Feeny DH, Patrick DL. Measurement in clinical trials: Choosing the right approach. In: Spilker B (ed), Quality of Life and Pharmacoeconomics in Clinical Trials, 2nd ed. Philadelphia: Lippincott-Raven Publishers, 1996; 41–48.

    Google Scholar 

  16. Williams JL, Naylor CD. How should health status measures be assessed? Cautionary notes on procrustean frameworks. J Clin Epidemiol 1992; 45: 1347–1351.

    Google Scholar 

  17. Stratford PW, Binkley JM, Riddle DL. Health status measures: Strategies and analyticmethods for assessing change scores. Phys Ther 1996; 76: 1109–1123.

    Google Scholar 

  18. Hays RD, Hadorn D. Responsiveness to change: An aspect of validity, not a separate dimension. Qual Life Res 1992; 1: 73–75.

    Google Scholar 

  19. Hays RD, Anderson RT, Revicki D. Assessing reliability and validity of measurement in clinical trials. In: Staquet MJ, Hays RD, Fayers PM (eds), Quality of Life Assessment in Clinical Trials. Methods and Practice, Oxford: Oxford University Press, 1998; 169–182.

    Google Scholar 

  20. Beaton DE, Hogg-Johnson S, Bombardier C. Evaluating changes in health status: Reliability and responsiveness of five generic health status measures in workers with musculoskeletal disorders. J Clin Epidemiol 1997; 50: 79–93.

    Google Scholar 

  21. Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use. New York: Oxford University Press, 1989.

    Google Scholar 

  22. Tuley MR, Mulrow CD, McMahan CA. Estimating and testing an index of responsiveness and the relationship of the index to power. J Clin Epidemiol 1991; 44: 417–421.

    Google Scholar 

  23. Murawski MM, Miederhoff PA. On the generalizibility of statistical expressions of health-related quality of life instrument responsiveness: A data synthesis. Qual Life Res 1998; 7: 11–22.

    Google Scholar 

  24. McHorney CA. Methodological inquiries in health status assessment. Med Care 1998; 36: 445–448.

    Google Scholar 

  25. Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach. J Clin Epidemiol 1997; 50: 869–879.

    Google Scholar 

  26. Wright JG, Young NL. A comparison of different indices of responsiveness. J Clin Epidemiol 1997; 50: 239–246.

    Google Scholar 

  27. Beaton DE, Bombardier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol 2001; 54: 1204–1217.

    Google Scholar 

  28. Stockler MR, Osoba D, Goodwin P, Corey P, Tannock IF. Responsiveness to change in health-related quality of life in a randomized clinical trial: A comparison of the Prostate Cancer Specific Quality of Life Instrument (PROSQOLI) with analogous scales from the EORTC QLQ-C30 and a trial specific module. J Clin Epidemiol 1998; 51: 137–145.

    Google Scholar 

  29. Kirkley A, Griffin S, McLintock H, Ng L. The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability. Am J Sports Med 1998; 26: 764–772.

    Google Scholar 

  30. Stadnyk K, Calder J, Rockwood K. Testing the measurement properties of the Short Form-36 Health Survey in a frail elderly population. J Clin Epidemiol 1998; 51: 827–835.

    Google Scholar 

  31. O'Keeffe ST, Lye M, Donnellan C, Carmichael DN. Reproducibility and responsiveness of quality of life assessment and six minute walk test in elderly heart failure patients. Heart 1998; 80: 377–382.

    Google Scholar 

  32. Garratt AM, Ruta DA, Abdalla MI, Russell IT. Responsiveness of the SF-36 and a condition-specific measure of health for patients with varicose veins. Qual Life Res 1996; 5: 223–234.

    Google Scholar 

  33. Mohtadi N. Development and validation of the quality of life outcome measure (Questionnaire) for chronic anterior cruciate ligament deficiency. Am J Sports Med 1998; 26: 350–359.

    Google Scholar 

  34. Windt van der DAWM, Heijden GJMGvd, Winter AFd, Koes BW, Devillé W, Bouter LM. The responsiveness of the Shoulder Disability Questionnaire. Ann Rheum Dis 1998; 57: 82–87.

    Google Scholar 

  35. Liang MH, Larson MG, Cullen KE, Schwartz JA. Comparitive measurement efficiency and sensitivity of five health status instruments for arthritis research. Arthritis Rheum 1985; 28: 542–547.

    Google Scholar 

  36. Giraudeau B, Ravaud P, Chastang C. Importance of reproducibility in responsiveness issues. Biomet J 1998; 40: 685–701.

    Google Scholar 

  37. Heald SL, Riddle DL, Lamb RL. The shoulder pain and disability index: The construct validity and responsiveness of a region-specific disability measure. Phys Ther 1997; 77: 1079–1089.

    Google Scholar 

  38. Bessette L, Sangha O, Kuntz KM, et al. Comparitive responsiveness of genericversus disease-specificand weighted versus unweighted health status measures in carpal tunnel syndrome. Med Care 1998; 36: 491–502.

    Google Scholar 

  39. Reilly MC, Zbrozek AS. Assessing the responsiveness of a quality of life instrument and the measurement of symptom severity in essential hypertension. Pharmacoeconomics 1992; 2: 54–66.

    Google Scholar 

  40. Partick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care 1989; 27: S217–S232.

    Google Scholar 

  41. Patrick DL, Martin ML, Bushnell DM, Yalcin I, Wagner TH, Buesching DP. Quality of life of women with urinary incontinence: Further development of the Incontinence Quality of life Instrument (I-QOL). Urology 1999; 53: 71–76.

    Google Scholar 

  42. Deyo RA. Measuring functional outcomes in therapeutic trials for chronic disease. Control Clin Trials 1984; 5: 223–240.

    Google Scholar 

  43. Feinstein AR, Josephy BR, Wells CK. Scientific and clinical problems in indexes of functional disability. Ann Intern Med 1986; 105: 413–420.

    Google Scholar 

  44. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Control Clin Trials 1991; 12: 142S–158S.

    Google Scholar 

  45. Guyatt GH, Feeny DH, Patrick DL. Measuring health related quality of life. Ann Intern Med 1993; 118: 622–629.

    Google Scholar 

  46. Flemons WW, Reimer MA. Measurement properties of the Cagalry Sleep Apnea Quality of Life Index. Am J Respir Crit Care Med 2002; 165: 159–164.

    Google Scholar 

  47. Anderson R, Rajagopalan R. Responsiveness of the dermatology-specific quality of life (DSQL) instrument to treatment for acne vulgaris in a placebo-controlled clinical trial. Qual Life Res 1998; 7: 723–734.

    Google Scholar 

  48. de Bruin AF, Diederiks JPM, de Witte LP, Stevens FCJ, Philipsen H. Assessing responsiveness of a functional status measure: The Sickness Impact Profile versus the SIP68. J Clin Epidemiol 1997; 50: 529–540.

    Google Scholar 

  49. Badia X, Podzamczer D, Casado A, Lopez-Lavid CC, Garcia M. Evaluating changes in health status in HIV-infected patients: Medical Outcomes Study-HIV and Multidimensional Quality of Life-HIV quality of life questionnaires. AIDS 2000; 14: 1439–1447.

    Google Scholar 

  50. Parkerson GR, Willke RJ, Hays RD. An international comparison of the reliability and responsiveness of the Duke Health Profile for measuring health-related quality of life of patients treated with alprostadil for erectile dysfunction. Med Care 1999; 37: 56–67.

    Google Scholar 

  51. Conner-Spady B, Cumming C, Nabholtz JM, Jacobs P, Stewart D. Responsiveness of the EuroQol in breast cancer patients undergoing high dose chemotherapy. Qual Life Res 2001; 10: 479–486.

    Google Scholar 

  52. Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostictest performance. J Chron Dis 1986; 39: 897–906.

    Google Scholar 

  53. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: A critical review and recommendations. J Clin Epidemiol 2000; 53: 459–468.

    Google Scholar 

  54. Liang M. Longitudinal construct validity. Establishment of clinical meaning in patient evaluative instruments. Med Care 2000; 38: II-84–II-90.

    Google Scholar 

  55. Fletcher AE, Ellwein LB, Selvaraj S, Vijaykumar V, Rahmathullah R, Thulasiraj RD. Measurement of visual function and quality of life in patients with cataracts in southern India. Arch Ophthalmol 1997; 115: 767–774.

    Google Scholar 

  56. Moayyedi P, Duffett S, Braunholtz D, et al. The Leeds Dyspepsia Questionnaire: A valid tool for measuring the presence and severity of dyspepsia. Aliment Pharmacol Ther 1998; 12: 1257–1262.

    Google Scholar 

  57. Ware JE, Kemp JP, Buchner DA, Singer AE, Nolop KB, Goss TF. The responsiveness of disease-specific and generic health measures to changes in the severity of asthma among adults. Qual Life Res 1998; 7: 235–244.

    Google Scholar 

  58. Guyatt GH, King DR, Feeny DH, Stubbing D, Godlstein RS. Genericand specificmeasureme nt of health-related quality of life in a clinical trial of respiratory rehabilitation. J Clin Epidemiol 1999; 52: 187–192.

    Google Scholar 

  59. Anderson JJ, Firschein HE, Meenam RF. Sensitivity of a health status measure to short-term clinical changes in arthritis. Arthritis Rheum 1989; 32: 844–850.

    Google Scholar 

  60. Mangione CM, Goldman L, Orav EJ, et al. Health-related quality of life after elective surgery. Measurement of longitudinal changes. J Gen Intern Med 1997; 12: 686–697.

    Google Scholar 

  61. Stofmeel MA, Post MW, Kelder JC, Grobbee DE, van Hemel NM. Changes in quality-of-life after pacemaker implantation: Responsiveness of the Aquarel questionnaire. Pacing Clin Electrophysiol 2001; 24: 288–295.

    Google Scholar 

  62. Lipsey MW. A scheme for assessing measurement sensitivity in program evaluation and other applied research. Psychol Bull 1983; 94: 152–165.

    Google Scholar 

  63. Hemingway H, Stafford M, Stansfeld S, Shipley M, Marmot M. Is the SF-36 a valid measure of change in population health? Results from the Whitehall II study. Br Med J 1997; 315: 1273–1279.

    Google Scholar 

  64. Clark JA, Rieker P, Propert KJ, Talcott JA. Changes in quality of life following treatment for early prostate cancer. Urology 1999; 53: 161–168.

    Google Scholar 

  65. Wiebe S, Rose K, Derry P, McLachlan R. Outcome assessment in epilepsy: Comparative responsiveness of quality of life and psychosocial instruments. Epilepsia 1997; 38: 430–438.

    Google Scholar 

  66. Liang MH, Fossel AH, Larson MG. Comparisons of five health status instruments for orthopedicevaluation. Med Care 1990; 28: 632–642.

    Google Scholar 

  67. Shields RK, Ruhland JL, Ross MA, Saehler MM, Smith KB, Heffner ML. Analysis of health-related quality of life and muscle impairment in individuals with amyotrophic lateral sclerosis using the Medical Outcome Survey and the Tufts Quantitative Neuromuscular Exam. Arch Phys Med Rehabil 1998; 79: 855–862.

    Google Scholar 

  68. Chren MM, Lasek RJ, Quinn LM, Mostow EN, Zyzanski SJ. Skindex, a quality of life measure for patients with skin disease: Reliability, validity, and responsiveness. J Invest Dermatol 1996; 107: 707–713.

    Google Scholar 

  69. Perpina M, Belloch A, Marks GB, Martinez-Morag¢n E, Pascual LM, Compte L. Assessment of the reliability, validity, and responsiveness of a Spanish asthma quality of life questionnaire. J Asthma 1998; 35: 513–521.

    Google Scholar 

  70. Hjortswang H, Strom M, Almeida RT, Almer S. Evaluation of the RFIPC, a disease-specific health-related quality of life questionnaire, in Swedish patients with ulcerative colitis. Scand J Gastroenterol 1997; 32: 1235–1240.

    Google Scholar 

  71. Norman GR. Issues in the use of change scores in randomized trials. J Clin Epidemiol 1989; 42: 1097–1105.

    Google Scholar 

  72. Shaw JW, Coons SJ, Foster SA, Leischow SJ, Hays RD. Responsiveness of the Smoking Cessation Quality of Life (SCQoL) questionnaire. Clin Ther 2001; 23: 957–969.

    Google Scholar 

  73. Meenan RF, Anderson JJ, Kazis LE, et al. Outcome assessment in clinical trials: Evidence for the sensitivity of a health status measure. Arthritis Rheum 1984; 27: 1344–1352.

    Google Scholar 

  74. Stucki G, Liang MH, Fossel AH, Katz JN. Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol 1995; 48: 1369–1378.

    Google Scholar 

  75. Terwee CB, Gerding MN, Dekker FW, Prummel MF, Wiersinga WM. Development of a disease-specific quality of life questionnaire for patients with Graves' ophthalmopathy: The GO-QOL. Br J Ophthalmol 1998; 82: 773–779.

    Google Scholar 

  76. Terwee CB, Gerding MN, Dekker FW, Prummel MF, van der Pol JP, Wiersinga WM. Test-retest reliability of the GO-QOL: A disease-specific quality of life questionnaire for patients with Graves' ophthalmopathy. J Clin Epidemiol 1999; 52: 875–884.

    Google Scholar 

  77. Ware JE, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36). I Conceptual framework and item selection. Med Care 1992; 30: 473–483.

    Google Scholar 

  78. Ware JE, Koskinski M, Keller SD. SF-36 Physical and Mental Health Summary Scales: A User's Manual, 2nd ed. Boston, MA: The Health Institute, 1994.

    Google Scholar 

  79. Staquet MJ, Hays RD, Fayers PM. Quality of Life Assessment in Clinical Trials. Methods and Practice. New York: Oxford University Press, 1998.

    Google Scholar 

  80. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 1989; 10: 407–415.

    Google Scholar 

  81. Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific quality of life questionnaire. J Clin Epidemiol 1994; 47: 81–87.

    Google Scholar 

  82. Ravaud P, Giraudeau B, Auleley GR, Edouard-No Ël R, Dougados M, Chastang Cl. Assessing smallest detectable change over time in continuous structural outcome measures: Application to radiological change in knee osteoarthritis. J Clin Epidemiol 1999; 52: 1225–1230.

    Google Scholar 

  83. Jacobson NS, Truax P. Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psych 1991; 59: 12–19.

    Google Scholar 

  84. Erickson P. Assessment of the evaluative properties of health status instruments. Med Care 2000; 38: II-95–II-99.

    Google Scholar 

  85. Moran LA, Guyatt GH, Norman GR. Establishing the minimal number of items for a responsive, valid, healthrelated quality of life instrument. J Clin Epidemiol 2001; 54: 571–579.

    Google Scholar 

  86. Lydick E, Epstein RS. Interpretation of quality of life changes. Qual Life Res 1993; 2: 221–226.

    Google Scholar 

  87. Lydick EG, Epstein RS. Clinical significance of quality of life data. In: Spilker B (ed), Quality of Life and Pharmacoeconomics in Clinical Trials. 2nd ed. Philadelphia: Lippincott-Raven Publishers, 1996: 461–465.

    Google Scholar 

  88. Testa MA. Interpretation of quality of life outcomes. Issues that affect magnitude and meaning. Med Care 2000; 38: II-166–II-174.

    Google Scholar 

  89. Lohr KN. Health outcomes methodology symposium. Summary and recommendations. Med Care 2000; 38: II-194–II-208.

    Google Scholar 

  90. Beaton DE. Understanding the relevance of measures change through studies of responsiveness. Spine 2000; 25: 3192–3199.

    Google Scholar 

  91. Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press, 1977.

    Google Scholar 

  92. Norman GR, Gwadry Sridhar F, Guyatt GH, Walter SD. Relation of distribution-and anchor-based approaches in interpretation of changes in health-related quality of life. Med Care 2001; 39: 1039–1047.

    Google Scholar 

  93. Katz JN, Larson MG, Phillps CB, Fossel AH, Liang MH. Comparative measurement senistivity of short and longer health status instruments. Med Care 1992; 30: 917–925.

    Google Scholar 

  94. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care 1989; 27: S178–S189.

    Google Scholar 

  95. Guyatt GH. Making sense of quality of life data. Med Care 2000; 38: II-175–II-179.

    Google Scholar 

  96. Terwee CB, Dekker FW, Mourits MPh, Gerding MN, Baldeschi L, Kalmann R, Prummel MF, Wiersinga WM. Interpretation and validity of changes in scores on the Graves' opthalmopathy quality of life questionnaire (GOQOL) after different treatments. Clin Endocrinol 2001; 54: 391–398.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Research in Extramural Medicine, VU University Medical Center, The Netherlands

    C.B. Terwee

  2. Department of Clinical Epidemiology, Leiden University Medical Center, The Netherlands

    F.W. Dekker

  3. Departments of Endocrinology and Metabolism, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

    W.M. Wiersinga & M.F. Prummel

  4. Clinical Epidemiology and Biostatistics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

    P.M.M. Bossuyt

Authors
  1. C.B. Terwee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F.W. Dekker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. W.M. Wiersinga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M.F. Prummel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. P.M.M. Bossuyt
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Terwee, C., Dekker, F., Wiersinga, W. et al. On assessing responsiveness of health-related quality of life instruments: Guidelines for instrument evaluation. Qual Life Res 12, 349–362 (2003). https://doi.org/10.1023/A:1023499322593

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023499322593

Search

Navigation