Skip to main content

Advertisement

SpringerLink
Log in

Content validity in the PROMIS social-health domain: a qualitative analysis of focus-group data

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Our aim was to assess the content validity of the Patient-Reported Outcomes Measurement Information System (PROMIS) social health item banks by comparing a prespecified conceptual model with concepts that focus-group participants identified as important social-health-related outcomes. These data will inform the process of improving health-related quality-of-life measures.

Methods

Twenty-five patients with a range of social limitations due to chronic health conditions were recruited at two sites; four focus groups were conducted. Raters independently classified participants’ statements using a hierarchical, nested schema that included health-related outcomes, role performance, role satisfaction, family/friends, work, and leisure.

Results

Key themes that emerged were fulfilling both family and work responsibilities and the distinction between activities done out of responsibility versus enjoyment. Although focus-group participants identified volunteerism and pet ownership as important social-health-related concepts, these were not in our original conceptual model. The concept of satisfaction was often found to overlap with the concept of performance.

Conclusion

Our conceptual model appears comprehensive but is being further refined to more appropriately (a) distinguish between responsibilities versus discretionary activities, and (b) situate the outcome of satisfaction as it relates to impairment in social and other domains of health.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Abbreviations

F:

Family and friends

HRO:

Health-related outcome(s)

HRQoL:

Health-related quality of life

L:

Leisure

NIH:

National Institutes of Health

PRO(s):

Patient-reported outcome(s)

PROMIS:

Patient-Reported Outcomes Measurement Information System

RPP:

Role participation performance

RPS:

Role participation satisfaction

UNC:

University of North Carolina at Chapel Hill

W:

Work

WHO:

World Health Organization

WPIC:

Western Psychiatric Institute and Clinic (University of Pittsburgh)

References

  1. Food and Drug Administration. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims (DRAFT). US Department of Health and Human Services Food and Drug Administration; Center for Drug Evaluation and Research (CDER); Center for Biologics Evaluation and Research (CBER); Center for Devices and Radiological Health (CDRH), February 2006. Available from: http://www.fda.gov/CDER/GUIDANCE/5460dft.pdf. Accessed December 18, 2007.

  2. Cella, D., Yount, S., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.

    Article  PubMed  Google Scholar 

  3. DeWalt, D. A., Rothrock, N., et al. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(5 Suppl 1), S12–S21.

    Article  PubMed  Google Scholar 

  4. World Health Organization. (1946). Constitution of the World Health Organization. Geneva: World Health Organization.

    Google Scholar 

  5. World Health Organization. (2001). ICF: International Classification of Functioning, Disability and Health. Geneva: World Health Organization.

    Google Scholar 

  6. Reeve, B. B., Hays, R. D., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–S31.

    Article  PubMed  Google Scholar 

  7. Birchwood, M., Smith, J., et al. (1990). The Social functioning scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. The British Journal of Psychiatry, 157, 853–859.

    Article  PubMed  CAS  Google Scholar 

  8. Dijkers, M. P., Whiteneck, G., et al. (2000). Measures of social outcomes in disability research. Archives of Physical Medicine and Rehabilitation, 81(12 Suppl 2), S63–S80.

    Article  PubMed  CAS  Google Scholar 

  9. Eisen, S. V., Normand, S. L. T., et al. (1994). BASIS-32 and the Revised Behavioral Symptom Identification Scale (BASIS-R). In M. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 759–790). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  10. Horowitz, L. M., Rosenberg, S. E., et al. (1988). Inventory of interpersonal problems: psychometric properties and clinical applications. Journal of Consulting and Clinical Psychology, 56(6), 885–892.

    Article  PubMed  CAS  Google Scholar 

  11. Weissman, M. M., & Bothwell, S. (1976). Assessment of social adjustment by patient self-report. Archives of General Psychiatry, 33(9), 1111–1115.

    PubMed  CAS  Google Scholar 

  12. World Health Organization. (2000). Disability Assessment Schedule II (WHO DAS II) Interviewer’s training manual. Geneva.

  13. McDowell, I., & Newell, C. (1996). Measuring health: A guide to rating scales and questionnaires (2nd ed.). New York: Oxford University Press.

    Google Scholar 

  14. Cohen, S., & Syme, S. L. (1985). Issues in the study, application of social support. In S. Cohen & S. L. Syme (Eds.), Social support and health (pp. 3–22). Orlando, FL: Academic Press, Inc.

    Google Scholar 

  15. McCampbell, C., & Helmer, O. (1993). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467.

    Google Scholar 

  16. Hahn, E., Cella, D., et al. (1991). Social well-being: The forgotten health status measure. Quality of Life Research, 14(9).

  17. Curtis, E., & Redmond, R. (2007). Focus groups in nursing research. Nurse Res, 14(2), 25–37.

    PubMed  Google Scholar 

  18. Krueger, R. (1994). Focus groups: A practical guide for applied research (2nd ed). Thousand Oaks, CA: Sage.

    Google Scholar 

  19. Morgan, D. (1988). Focus groups as qualitative research. Qualitative research methods (Vol 16). Newbury Park: Sage.

    Google Scholar 

  20. Morgan, D. (Ed.). (1993). Successful focus groups: advancing the state of the art. Newbury Park: Sage.

  21. Stewart, D., Shamdasani, P., et al. (2007). Focus groups: Theory and practice. Thousand Oaks: Sage.

    Google Scholar 

  22. Kitzinger, J. (1995). Qualitative research. Introducing focus groups. BMJ, 311(7000), 299–302.

    PubMed  CAS  Google Scholar 

  23. Bender, D., & Ewbank, D. (1994). The focus group as a tool for health research: Issues in design and analysis. Health Transition Review, 4(1), 63–79.

    PubMed  CAS  Google Scholar 

  24. Muhr, T. (2004). User’s manual for ATLAS.ti 5.0. GmbH, Berlin: ATLAS.ti Scientific Software Development.

    Google Scholar 

  25. Carey, J. W., Morgan, M., et al. (1996). Intercoder agreement in analysis of responses to open-ended interview questions: Examples from tuberculosis research. Field Methods, 8(3), 1–5.

    Article  Google Scholar 

  26. Gorden, R. L. (1992). Basic interviewing skills. Long Grove: Waveland Press.

    Google Scholar 

  27. Kupper, L. L., & Hafner, K. B. (1989). On assessing interrater agreement for multiple attribute responses. Biometrics, 45(3), 957–967.

    Article  PubMed  CAS  Google Scholar 

  28. Herrald, M. M., Herrald, M. M., et al. (2002). Pet ownership predicts adherence to cardiovascular rehabilitation. Journal of Applied Social Psychology, 32(6), 1107–1123.

    Article  Google Scholar 

  29. Frost, M. H., Reeve, B. B., et al. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health, 10 (Suppl 2), S94–S105.

    Article  PubMed  Google Scholar 

  30. Ustun, T., Chatterji, S., et al. (Eds.). (2001). Disability and culture: Universalism and diversity. Gottingen: World Health Organization.

Download references

Acknowledgements

The Patient-Reported Outcomes Measurement Information System (PROMIS) is a National Institutes of Health (NIH) Roadmap initiative to develop a computerized system measuring patient-reported outcomes in respondents with a wide range of chronic diseases and demographic characteristics. PROMIS was funded by cooperative agreements to a Statistical Coordinating Center (Evanston Northwestern Healthcare, PI: David Cella, PhD, U01AR52177) and six Primary Research Sites (Duke University, PI: Kevin Weinfurt, PhD, U01AR52186; University of North Carolina, PI: Darren DeWalt, MD, MPH, U01AR52181; University of Pittsburgh, PI: Paul A. Pilkonis, PhD, U01AR52155; Stanford University, PI: James Fries, MD, U01AR52158; Stony Brook University, PI: Arthur Stone, PhD, U01AR52170; and University of Washington, PI: Dagmar Amtmann, PhD, U01AR52171). NIH Science Officers on this project are William Riley, Ph.D., Susan Czajkowski, PhD, Lawrence Fine, MD, DrPH, Louis Quatrano, PhD, Bryce Reeve, PhD, James Witter, MD, and Susana Serrate-Sztein, PhD. This manuscript was reviewed by the PROMIS Publications Subcommittee prior to external peer review. See the Web site at www.nihpromis.org for additional information on the PROMIS cooperative group. This work was also supported by AHRQ National Research Service Award Research Training Grant T32 HS000032–17.

We gratefully acknowledge the editorial assistance provided by David Cella of Evanston Northwestern Healthcare, as well as the data coding efforts of the three raters at the UNC site: Katherine Buysse, Jessica Dilday, and Ashley Hink. Significant acknowledgements at the Pittsburgh site include Emily Huisman, Catherine Maihoefer, and Nathan Dodds.

Author information

Authors and Affiliations

  1. Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, 725 Martin Luther King Jr. Blvd, CB 7590, Chapel Hill, NC, 27599, USA

    Liana D. Castel & Darren A. DeWalt

  2. School of Social Work Ph.D. Program, University of North Carolina at Chapel Hill, CB 3550, Chapel Hill, NC, USA

    Kelly A. Williams

  3. Duke University Medical Center, 2424 Hock Plaza, Suite 1105, Durham, NC, USA

    Hayden B. Bosworth

  4. Department of Health Policy & Management, Boston University School of Public Health and Center for Health Quality, Outcomes & Economic Research, ENRM Veterans Hospital, 200 Springs Road (152), Bedford, MA, USA

    Susan V. Eisen

  5. Center on Outcomes, Research and Education (CORE), Evanston Northwestern Healthcare, 1001 University Place, Suite 100, Evanston, IL, USA

    Elizabeth A. Hahn

  6. Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA

    Elizabeth A. Hahn

  7. Institute for Healthcare Studies, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA

    Elizabeth A. Hahn

  8. Department of Epidemiology, University of North Carolina at Chapel Hill, CB 7590, Chapel Hill, NC, USA

    Debra E. Irwin

  9. Western Psychiatric Institute & Clinic, University of Pittsburgh Medical Center, 3811 O’Hara Street, Pittsburgh, PA, USA

    Morgen A. R. Kelly, Jennifer Morse & Angela Stover

  10. Division of General Medicine and Clinical Epidemiology, University of North Carolina at Chapel Hill, 5039 Old Clinic, CB 7110, Chapel Hill, NC, USA

    Darren A. DeWalt

  11. Department of Health Behavior and Health Education, School of Public Health, University of North Carolina at Chapel Hill, Thurston Arthritis Research Center, CB 7330, Chapel Hill, NC, USA

    Robert F. DeVellis

Authors
  1. Liana D. Castel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kelly A. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hayden B. Bosworth
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susan V. Eisen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elizabeth A. Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Debra E. Irwin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Morgen A. R. Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jennifer Morse
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Angela Stover
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Darren A. DeWalt
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Robert F. DeVellis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liana D. Castel.

Additional information

L. D. Castel, R. F. DeVellis, D. A. DeWalt, D. E. Irwin, and K. Williams conceived of and designed the study. L. D. Castel, D. E. Irwin, A. Stover, and K. A. Williams conducted the data collection. H. B. Bosworth, L. D. Castel, D. A. DeWalt, E. A. Hahn, S. V. Eisen, M. A. R. Kelly, D. E. Irwin, A. Stover, J. Morse, and K. A. Williams carried out analysis and interpretation. L. D. Castel created the initial draft of the article. H. B. Bosworth, R. F. DeVellis, D. A. DeWalt, E. A. Hahn, S. V. Eisen, D. E. Irwin, M. A. R. Kelly, A. Stover, J. Morse, and K. A. Williams aided in subsequent drafting and critical revision. D. A. DeWalt and R. F. DeVellis obtained funding. L. D. Castel held overall responsibility for this work. All authors read and approved the final manuscript.

Appendix

Appendix

Focus-group discussion guide for social-health focus groups

  1. 1.

    [Introductions] Let’s begin by going around the room and introducing ourselves to the group. First, tell us your name, then, tell us an example of something you do because you are expected to, and then follow-up with an example of something you do just because you like doing it. Who would like to start?

    • Follow-up: Let’s make these examples more specific. Thinking about your paid work or things you have done as a homemaker over the past 30 days, what are some things you were expected to do and things you just enjoyed doing?

    • Follow-up: Thinking about the things you have done with family and friends in the past 30 days, what were some things you were expected to do and things you just enjoyed doing?

    • Follow-up: Thinking about the things you have done for leisure and recreation in the past 30 days, what were some things you were expected to do and things you just enjoyed doing?

    • Follow-up: So far, we have discussed things we feel we should do or things we like to do that fall into the category of work, family, or leisure. What other categories of activities, if any, should be added to this list?

  2. 2.

    [Recent health impacts] If we have health problems, it can sometimes be hard to do the things we think we should do or the things we just like to do. Thinking about your own lives over the past 30 days, what are some ways, if any, that your health problems have made it harder for you to do things you just enjoyed doing?

    • Follow-up: Thinking again about the past 30 days, what are some ways, if any, that your health problems made it harder for you to do things you just enjoyed doing?

    • Follow-up: How about with any roles not yet mentioned?

  3. 3.

    [Long-term health impacts] Let’s shift topics a bit. Let’s talk about ways that the things you do now have changed, if at all, from the way they were before you began having to cope with your chronic health condition. As before, we will focus separately on work, family, and friends, and then leisure.

    • Follow-up: How have things changed, if at all, with the things you do at work, or in carrying out your homemaking responsibilities?

    • Follow-up: How have things changed, if at all, with the things you do with your family and friends?

    • Follow-up: Has your chronic health problem affected how you get along with your family or friends? If so, how?

    • Follow-up: How have things changed, if at all, with the things you do for leisure time or recreation?

  4. 4.

    [Coping] Take a moment now to remember how life was before you started to have chronic health problems and the changes you have had to make to cope with them.

    • Follow-up: Which of the changes in your work, relationships, or leisure activities caused by your health condition have been the most disruptive or unpleasant?

    • Follow-up: Which changes have been harder for you: the changes you have been forced to make in the things you’re expected to do or those changes you have made in things you do just for enjoyment?

    • Follow-up: What has made one kind of change harder than the other?

  5. 5.

    [Other health impacts] We have just discussed a number of ways that your chronic health conditions have affected your lives. In the past 30 days, have those health conditions limited any of you in areas other than those we discussed today? If so, how?

  6. 6.

    [Time frame – 30 days vs. 7 days] We have just discussed a number of ways that chronic health conditions have affected your life over the past 30 days. If we had asked you about the impact of chronic health conditions on your life over the past 7 days, how would your responses have been different?

  7. 7.

    [Close] We are coming to the end of this session. I have asked you a lot of questions about how chronic illnesses affect your lives. What other questions, if any, should we be asking?

    • Follow-up: And, finally, what final thoughts or comments would you like to add before we close?

Rights and permissions

Reprints and permissions

About this article

Cite this article

Castel, L.D., Williams, K.A., Bosworth, H.B. et al. Content validity in the PROMIS social-health domain: a qualitative analysis of focus-group data. Qual Life Res 17, 737–749 (2008). https://doi.org/10.1007/s11136-008-9352-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-008-9352-3

Keywords

Search

Navigation