Article Text

Download PDFPDF

Patient-reported outcomes in clinical HIV care: protocol for a single-centre, multistage, mixed-methods study in Denmark
  1. Olivia Borchmann1,
  2. Nina Weis1,2,
  3. Ann-Brit Eg Hansen1,2,
  4. Merete Storgaard3,
  5. Michael D Fetters4,5,
  6. P Paul Chandanabhumma4,5,
  7. Ellen Moseholm1,6
  1. 1Department of Infectious Diseases, Hvidovre Hospital, Copenhagen University Hospital, Hvidovre, Denmark
  2. 2Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Kobenhavn, Denmark
  3. 3Department of Infectious Diseases, Aarhus Universitetshospital, Aarhus, Denmark
  4. 4Department of Family Medicine, University of Michigan Medical School, Ann Arbor, Michigan, USA
  5. 5Mixed Methods Program, University of Michigan, Ann Arbor, Michigan, USA
  6. 6Department of Public Health, Faculty of Health and Medical Sciences, University of Copenhagen, Kobenhavn, Denmark
  1. Correspondence to Olivia Borchmann; oliviaborchmann{at}


Introduction People with HIV-1 (PWH) have worse health-related quality of life (HRQoL) compared with the general population. Using patient-reported outcomes (PROs) may help reorient the focus of HIV care towards improving HRQoL. This study aims to develop, implement and evaluate the use of PROs in HIV care.

Methods and analysis This is a Danish single-centre, multistage mixed-methods study consisting of four substudies (studies I–IV). Study I is a qualitative focus group interview study aiming to identify relevant PRO domains, and barriers and benefits to PRO use. Participants are 5–10 PWH and 5–10 HIV healthcare providers (HCPs). Data are thematically analysed. Results will guide the design of a PRO measure (PROM). Study II is a quantitative study aiming to assess PWH’s willingness and ability to engage with PRO. All PWH are consecutively invited to complete the PROMs before their next consultations. Demographic data are collected at enrolment. Differences between PWH who do/do not complete the PROMs are assessed. Study III is a quantitative before-and-after study aiming to assess the impact of PRO use on HCP awareness. Participants are all who complete the PROMs in Study II. In contrast to study II, HCPs are notified of the PROM results. The number of problems documented by the HCP in patients’ medical records during studies II and III are compared using χ2 tests. Multiple regression models are used to identify factors associated with HCP awareness. Study IV is a qualitative study aiming to explore PWH and HCP experiences of using PROs. Participants are 15–20 PWH and 10–15 HCP. Data are collected from participant observation of PRO consultations and individual interviews. Data are analysed thematically.

Ethics and dissemination This study is approved by the Danish Data Protection Agency. Participants will provide written consent prior to participation. Results will be published in peer-reviewed journals.

  • HIV & AIDS
  • Patient Reported Outcome Measures
  • Quality of Life

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • The use of a multistage, mixed-methods design will provide comprehensive and nuanced information on the potential impact of using patient-reported outcomes (PROs) in clinical HIV care in high-income settings with free access and excellent adherence to antiretroviral therapy.

  • Both people with HIV (PWH) and HIV healthcare providers (HCPs) will be included in the development of clinically relevant PRO measures.

  • An advisory board of relevant concerned parties including PWH, HCP and representatives from HIV non-governmental organisations (NGOs) will monitor the project and provide input and feedback throughout the project. This will help ensure engagement from concerned parties.

  • The study is limited to a single centre and findings may not necessarily be applicable to other settings that do not resemble the participating site—particularly low-income settings or settings that do not have comparable access to antiretroviral therapy.

  • It is not feasible to conduct a randomised study. We will try to overcome this limitation by choosing a before-and after-study setup to assess the impact of PRO use in study III.


Approximately 6800 people are living with HIV-1 (PWH) in Denmark.1 The introduction of antiretroviral therapy (ART) has dramatically improved the prognosis of PWH, with life expectancies approaching those of the general population.2 3 However, PWH have worse health-related quality of life (HRQoL) compared with the general population despite being on effective ART.4 5 Many PWH live under precarious material circumstances and contend with high burdens of complex psychosocial and sexual issues as well as comorbidities, which affect both their HIV-specific and their overall health and well-being.6–11 HRQoL has become a main focal point for improving successful HIV care.12–14 However, many of the issues affecting HRQoL remain underaddressed in HIV care12 13 15 16 and HIV healthcare providers (HCPs) have been shown to underestimate the extend of several of these issues such as drug use, mental health and adherence problems.17

Using patient-reported outcomes (PROs) could represent a key first step in reorienting the focus of HIV care towards improving HRQoL. PROs can be defined as any report of the status of a patient’s health condition that comes directly from the patient, without interpretation by a clinician or anyone.18 PROs often provide valuable insights that cannot otherwise easily be observed or measured.19

There is good evidence, especially within oncology, that routine use of PROs in clinical practice can improve clinical decision-making20 and patient–HCP communication.21 22 Within the HIV field, the use of PROs has been shown to be feasible and acceptable to both HCP and PWH.23–26 There is some evidence, mainly from the USA, showing that PRO use can improve care for PWH,12 23 27 including enhancing HCP awareness of symptoms and behaviours.17 24 25 28 This is also supported by a Danish quality assurance project.29 However, PROs are not well integrated into routine clinical HIV care.19 The task of implementing PROs is challenging, as effective implementation depends on context, involvement of concerned parties and successful use of clinically relevant PROs.23

This research project titled the PRO-CARE study is initiated as a combined research and implementation approach to PROs, focusing on patient involvement, HCP awareness and patient–HCP communication, to facilitate the successful use of PROs in clinical HIV care in a Danish setting. Thus, the aim of this study is to develop, implement and evaluate the use of PROs in the clinical care of PWH.

Methods and analysis

Study design

The PRO-CARE study is a single-centre, multistage combined implementation and evaluation project using mixed methods (MM) (figure 1). MM is defined as a research approach where both quantitative and qualitative data are collected, analysed and integrated.30 31 The rationale behind using MM is to draw on the strengths of both quantitative and qualitative methods to provide a nuanced and evidence-based understanding of the utility and implementation of PROs in clinical HIV care. Due to its multistage design, the PRO-CARE study consists of four substudies (studies I–IV) detailed separately below. The four studies are linked through building, where the results of one study informs the next.31 The study includes two MM core designs. It starts with an exploratory sequential design,30 where initial qualitative data (study I) will be used to build a PRO measure (PROM) for the subsequent quantitative studies (studies II and III). A final qualitative analysis (study IV) will complete the evaluation of the use of PROs, thus constituting an explanatory sequential design.30 Research findings from all four substudies will be further synthesised to construct a model for the long-term sustained integration of PROs in clinical HIV care.

Figure 1

Study diagram. PROs, patient-reported outcomes.


The overall single-centre study is conducted at a Department of Infectious Diseases at a University Hospital in Copenhagen, Denmark. The department treats approximately 1 500 PWH. Most PWH are well treated with biannual outpatient clinic consultations. At a standard visit in the outpatient clinic, PWH will meet an infectious disease specialist for approximately 15 min followed by a nurse for approximately 15 min.

Implementation framework

The RE-AIM framework, an established and widely used (including within healthcare) evaluation framework, will be used to guide the implementation and evaluation of PROs in clinical HIV care. This framework provides a method to assess the implementation strategy on multiple levels and allows for assessment at both the individual (ie, patient and HCP) and organisation level.32 RE-AIM is an acronym for the framework’s five outcome domains: Reach, Effectiveness, Adoption, Implementation and Maintenance.32 33 The RE-AIM dimensions, how they are operationalised, and the sources of data are presented in table 1.

Table 1

Summary of studies, including operationalisation of RE-AIM domains

Study I: patient and HIV HCP perspectives on PRO use in HIV care


Study I is a qualitative study aiming to explore the views, experiences and preferences of HCP and PWH regarding the problems and concerns of PWH, and to assess barriers and benefits perceived by HCP and PWH to the use of PROs in clinical HIV care.


Study participants will be recruited from the participating site using purposeful sampling. For PWH, we will prioritise maximum sample variation across key demographic characteristics and clinical criteria. Priority will be given to representation of socially marginalised PWH including those who use drugs, women, LGBTIQ (lesbian, gay, bisexual, transgender, intersex, queer) persons and migrants as well as treatment experienced individuals and those with many comorbidities. The inclusion criteria are living with HIV-1, receiving care at the participating study site, ≥18 years of age and Danish speaking (for the focus group discussions (FGDs)). HCP will be purposefully sampled by profession, that is, nurses and physicians. We aim for 5–10 participants in each FGD. This relatively small number is chosen to optimise rapport setting and facilitate common discussion between all participants.34

Data collection

FGDs will be held with separate groups of PWH and HCP between May 2023 and August 2023. The rationale behind the choice of FGD is to allow collective reflections and discussions between the participants to enhance data output.35 The methodological framework of this study will be interpretative description (ID), an applied, inductive research methodology emphasising the significance of performing research arising from clinical practice to generate new insights.35 We will conduct 2–4 moderated FGDs split into groups of PWH and HCP following an interview guide with open-ended questions to facilitate reflections and group discussions. The FGDs will be moderated by OB and observed by EM. Both OB and EM are employed at the study site, and thus know the HCP study participants prior to the study, but not the PWH participants. The FGDs will be held at the study site. If potential participants, who otherwise would not participate (due to, eg, language barriers or privacy concerns), request individual interviews, this will be considered as a possibility following ID framework of allowing complementary data sources to inform the analysis and interpretation.35 These individual interviews can be conducted in Danish or English. All interviews will be audiorecorded and transcribed verbatim. Observational notes will be taken during the FGDs.


Data collection and data analysis will be performed simultaneously as an iterative process following the steps described by ID.35 Both OB and EM will conduct the data analysis. The first step of the analysis will focus on familiarisation and developing a sense of the whole beyond the immediate impression of what data covers. The text will then be grouped and structured into broad generic codes reflecting what the text is about. The next step will be to find ways in which the broad codes relate to each other or differ by looking for meaningful patterns and variations, identifying first themes and then subthemes. In the final phase of the analysis, multiple iterations of the themes will be constructed by adding, reorganising and merging several themes. Each step will be revisited several times as described by ID.35 Both interview content and observational notes taken during the FGDs will contribute to the interpretation. QRS NVivo 12 software will be used to support the process and enhance transparency.

Study II: patients’ willingness and ability to use PROs


Study II is a quantitative observational study aiming to assess PWHs’ willingness and ability to successfully complete PROMs during routine care and to characterise PWH who were unable to or chose not to complete the PROMs.


Study participants will be recruited between November 2023 and May 2024. We plan to include all PWH attending the outpatient clinic at the study site, who meet the inclusion criteria. Participants will be consecutively invited to participate in the study by being asked to complete PROMs before their next routine consultations. The inclusion criteria are living with HIV-1, receiving care at the participating study site, ≥18 years old, English or Danish speaking and reading, able to use the patient online version of the electronic medical record system. Potential participants will be excluded if they are unable to give informed consent.

PRO measure

An HIV PROM will be developed based on a literature review identifying already validated PROMs, the results from study I, and a workshop with relevant concerned parties (detailed under Advisory board and workshop with concerned parties).

Data collection

PWH who agree to participate will receive an invitation via the Danish official secure email (‘e-Boks’) to complete the HIV PROM at home within one week before their next scheduled visit to the outpatient clinic. The PROMs will be collected and stored via the electronic medical record system ‘Sundhedsplatformen’. The treating HCP will not be notified of or have access to the PROM data. Quantitative demographic data on PWH who complete and PWH who do not complete the PROMs will be collected anonymously on a screening form at enrolment and in the PROM itself.


The proportion of PWH completing the PROMs will be calculated and differences in characteristics between PWH who complete and PWH who do not complete the PROMs will be assessed using the Pearson’s χ2 test, Student’s unpaired t-test or the Wilcoxon rank-sum test, as appropriate. R statistical software will be used.

Study III: impact of PRO use on HIV HCP awareness


Study III is a quantitative before-and-after study aiming to assess the impact of PRO use in routine HIV care on HCP awareness.


All PWH who complete the PROMs in study II will also be included in study III.

Data collection

The PROMs will be collected and stored as described in study II. However, in this study, the results of the PROMs will be summarised in a report available in the patients’ medical records (ie, ‘Sundhedsplatformen’). The treating HCP will be notified and encouraged to actively use the PROM data in the consultations.


The primary outcome is HCP awareness defined as the number of patient symptoms and psychosocial problems documented by the HCP in patients’ medical records. A study-specific outcome protocol will be developed to ensure consistency in the data collection. The primary analysis will compare the number of symptoms and problems documented by HCP during study II and during study III (ie, before and after HCP notification of PROM results) using χ2 tests. Multiple regression models will be used to identify factors associated with HCP awareness. The analysis will be done in collaboration with an experienced biostatistician. R statistical software will be used.

Study IV: patient and HIV HCP perception of PRO in clinical HIV care


Study IV is a qualitative observational and interview study aiming to explore how PWH and HCP experience the use of PROs in routine HIV care and to explore the effect of PRO use on patient–HCP interaction during consultations. As with study I, ID will be used as the methodological approach.35


Study participants will be recruited from among the participants of study III. If any specific patterns emerge in study II or III that require further in-depth investigation, we will use purposeful sampling. Otherwise, random sampling will be used. HCP (nurses and physicians) will be asked to participate if their patients accept participation. Approximately 15–20 PWH and 10–15 HCP will be included. Recruitment of PWH will continue until no new major themes emerge in the analysis which, following ID framework, will be performed simultaneously with data collection in an iterative process.

Data collection and analysis

Data will be collected from the following sources by OB and EM:

  • Participant observations of consultations where PWH have completed PROMs prior to their visits. Field notes will be taken using a semistructured observation guide to promote consistency.

  • Individual semistructured interviews with PWH. Interviews will be conducted 1–2 weeks following the PRO consultation. Using an interview guide, the focus of the interview will be the clinical use of PROs in general, patient–HCP interaction, and specific situations highlighted in the observed consultations.

  • Individual semistructured interview with HCP. Interviews will be conducted within the week of the PRO consultation. Using an interview guide, the focus in the interview will be on the use of PROs in clinical HIV care specifically related to the consultation and in general, and patient–HCP interaction.

Data analysis will be performed as described in study I using the NVivo 12 software.

MMs integration

In addition to the separate analyses of qualitative and quantitative data, the MM design allows us to gain deeper insights by integrating data. Integration in MM research is defined as an intentional process by which the researcher brings qualitative and quantitative data together in one study.30 Integration in this study occurs at both the design, methods and interpretation levels.31 After the completion of all four substudies, research findings will be further analysed to construct a model for the sustained implementation of PROs in clinical HIV care. The included data and specific analysis will depend on the results from the four substudies. Joint displays will be used to guide the process of understanding how the quantitative and qualitative findings relate to each other, and the drawing of interpretations across the findings, that is, making meta-inferences.30 The integration of data will provide insights beyond the quantitative and the qualitative results alone, thus providing a more nuanced understanding of the utility of PROs in clinical HIV care. These results will be highly relevant when assessing the extent to which PROs are implemented into clinical practice and the extent to which the use of PROs can be sustained over time.

Advisory board and workshop with concerned parties

An advisory board including PWH, representatives from the two largest HIV NGOs in Denmark, HIV Denmark and AIDS-Fondet Denmark, and HCP will monitor the project and provide input and feedback throughout the project. Additionally, the first stage of the study will include a workshop with all relevant concerned parties including department managers, physicians, nurses, PWH and representatives from AIDS-Fondet Denmark and HIV Denmark. The results from study I will be presented at the workshop, and the goal of the workshop is to discuss and identify relevant PRO domains, to identify and address benefits and concerns around the use of PROs, and to help tailor information and secure valuable support.

Patient and public involvement

PWH and HIV NGOs will be involved in providing critical reflections and feedback throughout the whole project in the advisory board as well as in the workshop. Both will also include other relevant concerned parties as described above.


The preparation phase was initiated on 1 March 2023. The recruitment for study I and the first data collection (the FGDs) were initiated in May 2023 and are planned to finish in August 2023. The whole PRO-CARE Study is a 4-year project intended to finish by the end of 2026.

Ethics and dissemination

This study is approved by the Danish Data Protection Agency (Project ID: P-2023-242). All participating HCP and PWH will provide written informed consent before any data collection.

Results from the PRO-CARE study will be published in international, peer-reviewed journals and communicated at relevant scientific meetings and conferences. Authorship will be allocated using the International Committee of Medical Journal Editors guidelines for authorship.


This MM study draws on the strengths of both qualitative and quantitative approaches to provide a nuanced and evidence-based understanding of the utility and implementation of PROs in clinical HIV care. We publish this protocol with the intend of making our methodological choices explicit and transparent. However, a certain flexibility of methods must be present during the course of the study to ensure that the multistage MM design realises its full potential by allowing knowledge gained at one stage to inform the next stage.30

Research integrity to validity concerns

There are several considerations to ensure validity in both exploratory and explanatory sequential core MM designs as highlighted by Fetters et al.31 In exploratory sequential designs, the first consideration relates to ensuring that the quantitative features are built based on the qualitative results. The second consideration relates to ensuring that rigorous quantitative features are developed. In our study, we will try to meet these two considerations by ensuring that the domains uncovered in the FGDs in study I will guide the choice between already validated PROMs with good psychometric properties in the following substudies. The third consideration relates to overlapping participants between the qualitative and quantitative samples. We will meet this consideration by having a large sample size in the subsequent quantitative substudies.

In explanatory sequential designs, the considerations relate to ensuring that important quantitative results, including those that are surprising or contradictory, are identified and sought explained in the subsequent qualitative study. We will try to consider all possible explanations for our results using stratified analyses and considering statistically non-significant results as well, and let these results guide our choice of questions in the subsequent semistructured interviews. The last consideration relates to connecting the quantitative results with the qualitative follow-up by sampling qualitative participants purposefully among the quantitative participants.

Strengths and limitations

The major strength of this study is the use of a multistage MM design, which allows us to draw on the strengths of both quantitative and qualitative approaches and to draw meta-inferences from the integration of these two types of data. This will provide detailed and nuanced information about the utility of PROs in clinical HIV care. This has not been previously explored in a high income setting such as Denmark, where medical treatment of HIV is highly successful. Moreover, the involvement of PWH and HCP in the development of PROs will hopefully help ensure their clinical relevance. Combined with the creation of an advisory board, which will monitor the project and provide feedback, this will enhance the possibility of successful PRO implementation, which will create the best basis for assessing the utility of PROs in clinical HIV care.

The study has some limitations. First, the study is limited to a single centre. Thus, our findings may not all be applicable to other settings with different PWH populations or to low-income settings or settings without good access to ART. Another limitation is the non-randomised setup. It would not be feasible to randomise PWH to either having HCP be aware of their PROM results or not due to the overlapping HCP, which would influence the results. We will try to overcome this limitation by choosing a before-and-after design. The PROMs will only be available in Danish and English. This limitation may reduce the number of migrant participants and thus limit the study results’ generalisability to that subpopulation.

Potential implications of the study

The results of this project will have a considerable impact on the future implementation and maintenance of PROs in clinical HIV care, which will be highly relevant not only at the participating site but to all infectious disease departments throughout Denmark and to similar settings internationally. Finally, reorienting the HIV field to focus more holistically on the well-being of PWH rather than merely their viral loads could potentially support better outcomes across the HIV care cascade.



  • Deceased Died July 16th 2023

  • Contributors EM conceived the idea. OB, NW, A-BEH, MS, MDF, PPC and EM contributed to the design of the study. OB, NW, A-BEH and EM will contribute to future data collection. OB, NW, A-BEH, MS, PPC and EM will contribute to future data analysis. OB and EM wrote the first draft of the manuscript. NW, A-BEH, MS, MDF and PPC revised the manuscript critically for important intellectual content. All authors approved the final version of the manuscript.

  • Funding This work was supported by the Independent Research Fund Denmark, grant number (2096-00190B).

  • Competing interests OB has received financial support from GSK to participate in an HIV conference. NW reports personal fees from AbbVie, Merck Sharp Dohme, outside the submitted work; honorarium paid to her institution, fees from Novo Nordisk and unrestricted grants for research from The Novo Nordisk Foundation, outside the submitted work. A-BEH reports personal fees from Pfizer, outside the submitted work. EM reports personal fees from Bristol Myers Squibb and GSK, outside the submitted work; honorarium paid to her institution and unrestricted grants paid to her institution from The Novo Nordisk Foundation and Gilead, outside the submitted work. All other authors declare no competing interests.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.