Methods, procedures and instruments

As a preparation for the relevance assessment, the first step will be to translate the integrative model of PC14 into a plain language version to make it more easily comprehensible for patients. Two researchers will independently translate the integrative model of PC into a plain language version. Next, they will generate a consensus version through discussion with two other members of the research team. Then, we will use cognitive interviewing with patients to test comprehensibility of the plain language model. Participants will be asked to read the plain language model step by step and to paraphrase in their own words what they have read. The cognitive interviews will be audio-recorded, transcribed and analysed independently by two researchers regarding misunderstandings as well as expressed difficulties with the text and suggestions for improvement. Based on the results, the plain language version will be revised in a group discussion between three researchers. Depending on the extent of revision necessary, a further round of cognitive interviews might be conducted, with a maximum of three rounds.

To assess the patient-reported relevance of the different dimensions of PC, a Delphi study will be conducted. Relevance in this case means the subjective importance of the different dimensions from the patients’ view. In addition, we will assess to what extent those different aspects are already implemented in the German healthcare system according to the patients’ subjective view. In a first round, participating patients will receive the revised plain language model as well as a questionnaire to rate the relevance and implementation of each dimension. In a second round, they will receive the median ratings of all participants as well as their own ratings from the first round. They will again be asked to rate the relevance and implementation. This procedure makes it possible to foster exchange and consensus within a large group of patients. The anonymity of the Delphi method helps to give all opinions an equivalent weight without dominance of certain persons, as might be the case in the personal contact within focus groups.38 39

For all patient participants, we will collect demographic data (eg, age, gender) with a short questionnaire. Furthermore, participants of the cognitive interviews will be asked to answer the European Health Literacy Questionnaire HLS-EU-Q16. This questionnaire was developed and used in The European Health Literacy Project (HLS-EU) in eight different European countries.40 41 The German version was tested with n=4952 respondents. It showed an internal consistency of Cronbach’s α=0.9. The questionnaire is used, because health literacy might influence the ability to comprehend the plain language version. Thus, we want to make sure that we included some patients with a low health literacy as well.

Recruitment/sample size

Patients for phase 1 will be recruited through a range of community-based strategies (including media campaigns using newspaper announcements and flyers, as well as recruitment through patient organisations throughout Germany). This approach was chosen, as it has been shown to be less resource intensive than recruitment through healthcare institutions.42–44

For each round of cognitive interviews, approximately 10 to 15 participants are expected to be necessary to reach theoretical saturation.45 46

For the Delphi study, we aim to collect data from a total sample of 200 participants. This allows for calculating descriptive parameters with high precision for the total sample and warrants a sufficient precision of estimates for disease subgroups as well. To reach 200 participants in the second round, more participants need to be included in the first round. Former Delphi studies showed dropout rates from round 1 to round 2 ranging from 10% to 30%.15 47–49 We expect a rather small dropout rate of approximately 15%, as patients will be reminded personally, and patients who participate in both rounds will receive a financial compensation. We therefore aim to recruit 236 patients for the first round.

Data analysis

We will analyse all sociodemographic data, health literacy scores and ratings from the Delphi survey descriptively (eg, frequencies, means, SD) for all participants as well as for subgroups according to the diseases of the participants.

Methods, procedures and instruments

Based on the relevance ratings of phase 1, items for the core set of PREMs will be developed using multiple sources. First, we will conduct a literature search to identify existing international PREMs on PC. This search will include an electronic search as well as scanning of existing reviews on measures that assess dimensions of PC.32 34 35 50 We will translate suitable items into German using a team translation process. This process consists of the following steps: (1) translation from original language to German by two independent translators, (2) suggestion of consensus version by third person and (3) final consensus team discussion. Furthermore, we will conduct focus groups and key informant interviews with patients as well as with experts (eg, clinicians, health services researchers, representatives from professional associations, sickness funds and health policies). Participants will be asked to generate items, which are suitable to measure PC based on the model from phase 1.

All generated items will be tested using cognitive interviews with patients and revised if necessary to ensure that all items are comprehensible for the target population. Again, in this step, we will use the HLS-EU-Q16 to check the variance of health literacy within our sample.

In a second step, the number of items will be reduced if necessary. Experts and patients will be asked to rate the relevance of all generated items for measuring PC. In addition, they will be asked whether any important questions are missing. This allows to establish content validity by ensuring that all items are relevant as well as that they are reflecting PC comprehensively.51 Items rated as not relevant or less relevant will be dropped.

Recruitment/sample size

We plan to include 50 participants (patients and experts) in the focus groups and key informant interviews for item generation. For cognitive interviewing, we will recruit between 10 and 15 patients per round, with a maximum of three rounds. Content validity will be assessed by 5 to 10 experts as well as 5 to 10 patients, which corresponds to common recommendations.46 52 53

Again, we will recruit patients via community-based strategies in the Hamburg area. Experts other than patients will be invited from our network of collaborating project partners and from the large regional Network for Healthcare Research Hamburg (HAM-NET; http://ham-net.de/en/).

Data analysis

The key informant interviews and focus groups will be recorded, transcribed and analysed via qualitative content analysis.54 Two researchers will conduct the analyses. Regular meetings with a third researcher will be held to ensure that coding schemes and analyses are comparable.

For content validity rating, a four-point Likert scale (1=not relevant, 2=somewhat relevant, 3=quite relevant and 4=very relevant) will be used.46 52 53 We will examine frequencies and median of the content validity ratings. Furthermore, we will calculate the Content Validity Index46 52 53 to explore inter-rater agreement of the relevance ratings for each item.

Methods, procedures and instruments

After the development of the preliminary item set, its psychometric characteristics will be tested. Patients will be asked to fill in a survey. The survey will include the developed items as well as selected validated measures that test related constructs to establish construct validity. For example, we will use the first item of the German version of 12-Item Short Form Survey55 to explore discriminant validity. This item explores the general perception of health status and is expected to show a low correlation <0.3.27 For convergent validity, we will for example assess satisfaction with the treatment by using the German version of the 8-item Client Satisfaction Questionnaire (ZUF-8).56 For satisfaction, we expect a high correlation >0.5.27

The decision whether further measures will be used to test construct validity will be part of the preparation for phase 2b and can only be determined once the content of the item set emerges from the data. The core set of PREMS will be revised according to the results of phase 2b. We will delete items according to common thresholds in floor and ceiling effects, intraclass correlations or Kappa and loadings in factor analysis (eg, items with more than 20% of the replies at one end of the scale and items with intraclass correlations or Kappa coefficients below 0.4057).

Recruitment/sample size

Patients will be recruited in healthcare institutions in the metropolitan area of Hamburg, Germany (eg, academic cancer centre, academic heart centre, psychosomatic clinic, primary care practices). Consecutive sampling will be used in participating outpatient and inpatient healthcare institutions of primary and secondary care. This way, only patients that are currently being treated in a specific healthcare institution will be included in this study phase.

To examine the factorial structure of items and to perform analyses of associations among variables with sufficient precision and power, an approximate sample size of 250 participants per group will be targeted. Overall, there will be eight groups as we collect data from each of the four disease groups in two settings (inpatient and outpatient). Therefore, we aim for a total sample size of 2000 participants. This allows for the use of maximum likelihood or weighted least squared estimators in the investigation of moderately complex models and for the detection of weak but meaningful correlations (r=0.20) with an alpha error probability of α=0.05 and a power of 0.90. The exact methods of assessment, statistical analyses and exact necessary sample sizes will be determined once phase 2a is completed, and the exact number of items and measures of the core set has been decided on.

Data analysis

We will calculate several psychometric indices. We will examine internal consistency using Cronbach’s α or the Kuder Richardson 20 coefficient (depending on the response format of items). For further item reduction and assessment of structural validity, we will analyse item characteristics (discrimination and difficulty) and perform exploratory factor analysis. To assess construct validity, we will calculate correlations. We will use t-tests and Mann-Whitney U-tests (depending on the distribution of the outcome) to assess discriminant validity. We will examine floor and ceiling effects using descriptive analyses by comparing the distribution of the observed values to the theoretically possible range of values. As a proxy of acceptability, we will examine if there are systematic patterns of missing data or low response rates.58 We do not examine retest reliability and inter-rater reliability, as we do not expect the measured construct to remain stable over time or between people.

Methods, procedures and instruments

Collaborating healthcare organisations will be offered to use the final core set of PREMs in their routine practice. As a first step, onsite meetings with staff of participating healthcare institutions will be held. Those have the purpose to come to a shared understanding on how to best administer the core set of PREMs within the current clinical workflow of the respective institution. In these meetings, we will, for example, discuss who should give the core set to the patients, at what time point exactly and how the replies should be collected. In addition, the participating institutions will be able to decide whether they want to receive the results of the PREMs core set in order to use them as performance feedback. These onsite meetings will be followed by a phase of exploratory use of several weeks. After this phase of exploratory use, we will conduct interviews. Specifically, we will ask the participating staff how they organised the implementation and whether they made adaptions to better fit the setting requirement. Furthermore, we will ask which barriers and facilitators they experienced and which steps need to be taken in order to further use the PREMs core set in their routine care.

As a second step of phase 3, we will conduct an expert workshop on implementation of the core set of PREMS in routine care. This workshop aims at (1) presenting the results of the psychometric testing and the assessment of PC in routine practice, and (2) discussing opportunities for the implementation of the core set in routine practice after testing it in the phase of exploratory use. This discussion aims to bring together the results from the semistructured interviews in the pilot phase as well as the knowledge and experience of the participating experts. It will cover both the implementation at individual healthcare organisations as well as the implementation as a potential quality indicator for purposes of accountability and quality improvement at the health system level.25

Recruitment/sample size

For the pilot test, we aim to interview 10 to 15 clinicians and administrative staff from our collaborating healthcare organisations where the core set of PREMs was tested.

We plan to conduct the workshop with 25 to 30 experts. This will allow inviting a range of experts with different backgrounds and areas of expertise (eg, clinicians, health services researchers, representatives from professional associations, sickness funds and health policy). At the same time, it is a sample size feasible to allow the realisation of discussions in small groups and in the full group. They will be invited from our network of collaborating project partners and HAM-NET.

Data analysis

The interviews and focus groups will be recorded, transcribed and analysed via qualitative content analysis.54 The analyses will be conducted by two researchers. Regular meetings with a third researcher will be held to ensure that coding schemes and analyses are comparable.