Skip to main content

Advertisement

SpringerLink
Log in

Testing the measurement equivalence of paper and interactive voice response system versions of the EORTC QLQ-C30

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

The objective of this study was to evaluate the measurement equivalence of an interactive voice response system (IVRS) version and the original paper-based version of the EORTC QLQ-C30.

Methods

The QLQ-C30 is a cancer-specific, health-related quality of life questionnaire consisting of nine multi-item scales (physical, role, emotional, cognitive and social functioning, fatigue, nausea/vomiting, pain, and quality of life) and six single item measures (dyspnea, insomnia, appetite loss, constipation, diarrhea, and financial problems). This study utilized a crossover design with subjects randomly assigned to one of two assessment orders: (1) paper then IVRS or (2) IVRS then paper. Equivalence between the two administration modes was established by comparing the 95 % lower confidence interval (CI) of the intraclass correlation coefficients (ICCs) for each scale, with a critical value of 0.70.

Results

The ICCs for the nine multi-item scales were all above 0.79, ranging from 0.791 to 0.899 (ICC 95 % lower CI range 0.726–0.865) and significantly different from our threshold reliability of 0.70. The ICCs for the six single items ranged from 0.689 to 0.896 (ICC 95 % lower CI range 0.611–0.888). Two of the items, insomnia and appetite loss, were not statistically different from 0.70. When considered together, the per-protocol analysis results support the equivalence of the paper and IVRS versions of the QLQ-C30 for 13 of the 15 scores.

Conclusion

This analysis provides evidence that the scores obtained from the IVRS version of the QLQ-C30 are equivalent to those obtained with the original paper version except for the insomnia and appetite loss items.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  1. US Food and Drug Administration (2009). Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. Available at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf.

  2. Lipscomb, J., Reeve, B. B., Clauser, S. B., et al. (2007). Patient-reported outcome assessment in cancer trials: Taking stock, moving forward. Journal of Clinical Oncology, 25, 5133–5140.

    Article  PubMed  Google Scholar 

  3. Osoba, D., Aaronson, N. K., Zee, B., et al. (1997). Modification of the EORTC QLQ-C30 (version 2.0) based on content validity and reliability testing in large samples of patients with cancer. Quality of Life Research, 6, 103–108.

    Article  CAS  PubMed  Google Scholar 

  4. Bottomley, A., & Aaronson, N. K. (2007). International perspective on health-related quality of life research in cancer clinical trials: The European Organization for Research and Treatment of Cancer experience. Journal of Clinical Oncology, 25, 5082–5086.

    Article  PubMed  Google Scholar 

  5. National Cancer Institute (2002). A manual for participants in clinical trials of investigational agents sponsored by DCTD, NCI. Available at: http://ctep.cancer.gov/investigatorResources/docs/hndbk.pdf. Accessed September 5, 2011.

  6. Bloom, D. E. (1998). Technology, experimentation, and the quality of survey data. Science, 280, 847–848.

    Article  CAS  PubMed  Google Scholar 

  7. Taenzer, P. A., Speca, M., Atkinson, M. J., et al. (1997). Computerized quality-of-life screening in an oncology clinic. Cancer Practice, 5, 168–175.

    CAS  PubMed  Google Scholar 

  8. Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275–304.

    Article  Google Scholar 

  9. Velikova, G., Wright, E. P., Smith, A. B., et al. (1999). Automated collection of quality-of-life data: A comparison of paper and computer touch-screen questionnaires. Journal of Clinical Oncology, 17, 998–1007.

    CAS  PubMed  Google Scholar 

  10. Bushnell, D. M., Reilly, M. C., et al. (2006). Validation of electronic data capture of the irritable bowel syndrome—Quality of life measure, the work productivity and activity impairment questionnaire for irritable bowel syndrome and the EuroQol. Value Health, 9, 98–105.

    Article  PubMed  Google Scholar 

  11. Stone, A. A., Shiffman, S., Schwartz, J. E., et al. (2002). Patient noncompliance with paper diaries. BMJ, 324, 1193–1194.

    Article  PubMed  Google Scholar 

  12. Coons, S. J., Gwaltney, C. J., Hays, R. D., et al. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health, 12(4), 419–429.

    Article  PubMed  Google Scholar 

  13. Crow, J. T. (1986). Receptive vocabulary acquisition for reading comprehension. Modern Languages Journal, 70, 242–250.

    Article  Google Scholar 

  14. Henriksen, B. (1999). Three dimensions of vocabulary development. Studies in Second Language Acquisition, 21, 303–317.

    Article  Google Scholar 

  15. Abu-Hasaballah, K., James, A., & Aseltine, R. H, Jr. (2007). Lessons and pitfalls of interactive voice response in medical research. Contemporary Clinical Trials, 28, 593–602.

    Article  PubMed  Google Scholar 

  16. Corkrey, R., & Parkinson, L. (2002). Interactive voice response: review of studies 1989–2000. Behavior Research Methods, Instruments and Computers, 34(3), 342–353.

    Article  Google Scholar 

  17. Alemi, F., Stephens, R., Parran, T., et al. (1994). Automated monitoring of outcomes: Application to the treatment of drug abuse. Medical Decision Making, 14, 180–187.

    Article  CAS  PubMed  Google Scholar 

  18. Agel, J., Greist, J. H., Rockwood, T., et al. (2001). Comparison of interactive voice response and written self-administered patient surveys for clinical research. Orthopedics, 24, 1155–1157.

    CAS  PubMed  Google Scholar 

  19. Juniper, E. F., Scatz, M., Juniper, B. A., & Zeiger, R. (2008). Evaluation of 2 interactive voice-response telephone versions of health-related quality-of-life questionnaires. Journal of Allergy and Clinical Immunology, 122, 654–655.

    Article  PubMed  Google Scholar 

  20. Lauritsen, K., Degl’ Innocenti, A., Hendel, L., Praest, J., Lytje, M. F., Clemmensen-Rotne, K., et al. (2004). Symptom recording in a randomized clinical trial: Paper diaries vs. electronic or telephone data capture. Controlled Clinical Trials, 25, 585–597.

    Article  PubMed  Google Scholar 

  21. Weiler, K., Christ, A. M., Woodworth, G. G., Weiler, R. L., & Weiler, J. M. (2004). Quality of patient-reported outcome data captured using paper and interactive voice response diaries in an allergic rhinitis study: Is electronic data capture really better? Annals of Allergy, Asthma & Immunology, 92, 335–339.

    Article  Google Scholar 

  22. Wilson, S. J., Rich, A. S., Rich, N. C., Potokar, J., & Nutt, D. J. (2004). Evaluation of actigraphy and automated telephoned questionnaires to assess hypnotic effects in insomnia. International Clinical Psychopharmacology, 19, 77–84.

    Article  CAS  PubMed  Google Scholar 

  23. Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., et al. (1993). A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376.

    Article  CAS  PubMed  Google Scholar 

  24. Bjordal, K., de Graeff, A., Fayers, P. M., Hammerlind, E., et al. (2000). A 12 country field study of the EORTC QLQ-C30 (version 3.0) and the head and neck cancer specific module (EORTC QLQ-H&N35) in head and neck patients. European Journal of Cancer, 36, 1796–1807.

    Article  CAS  PubMed  Google Scholar 

  25. Gundy, C.M., Aaronson, N.K. (2010). Effects of mode of administration (MOA) on the measurement properties of the EORTC QLQ-C30: A randomized study. Health and Quality of Life Outcomes 8:35. Available at: http://www.hqlo.com/content/8/1/35.

  26. Lundy, J. J., & Coons, S. J. (2011). Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample. Value Health, 14, 867–871.

    Article  PubMed  Google Scholar 

  27. Hills, M., & Armitage, P. (1979). The two period cross-over clinical trial. British Journal of Clinical Pharmacology, 8, 7–20.

    Article  CAS  PubMed  Google Scholar 

  28. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

    Article  CAS  PubMed  Google Scholar 

  29. McGraw, K.O., Wong, S.P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46 (Correction: Vol. 1, No. 4, 390).

    Google Scholar 

  30. Streiner, D. L., & Norman, G. R. (2003). Health measurement scales: A practical guide to their development and use (3rd ed.). New York: Oxford University Press.

    Google Scholar 

Download references

Acknowledgments

The data used for this research were collected as part of a study funded by ClinPhone Plc (now Perceptive Informatics). Additional support was provided by the University of Arizona Cancer Center Support Grant CA023074 from the National Cancer Institute. The authors gratefully acknowledge the staff and facility support provided by the University of Arizona College of Pharmacy and the University of Arizona Cancer Center’s Behavioral Measurements Shared Service. JJL and SJC were employed by the University of Arizona at the time this study was conducted. The authors have no financial interest in Perceptive Informatics or ClinPhone PLC.

Author information

Authors and Affiliations

  1. Patient-Reported Outcome Consortium, Critical Path Institute, 1730 E River Road, Tucson, AZ, 85718-5893, USA

    J. Jason Lundy & Stephen Joel Coons

  2. Division of Psychosocial Research and Epidemiology, Department of Psychosocial Research, The Netherlands Cancer Institute, 121 Plesmanlaan, 1066 CX, Amsterdam, The Netherlands

    Neil K. Aaronson

Authors
  1. J. Jason Lundy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Joel Coons
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neil K. Aaronson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Jason Lundy.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lundy, J.J., Coons, S.J. & Aaronson, N.K. Testing the measurement equivalence of paper and interactive voice response system versions of the EORTC QLQ-C30. Qual Life Res 23, 229–237 (2014). https://doi.org/10.1007/s11136-013-0454-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-013-0454-1

Keywords

Search

Navigation