Article Text
PDF
Abstract
Objective Measuring the quality of care as experienced by patients is increasingly recognised as a way of improving healthcare services. However, disease-specific measures that take the patient journey into account are needed. This paper presents the development of such a measure for patients with heart disease and details the psychometric evaluation.
Design The questionnaire was developed based on a literature review, qualitative interviews and a pilot-test. The psychometric evaluation of the measure was assessed using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), Cronbach’s alpha coefficient and differential item functioning analysis with data from a population-based survey.
Setting Denmark in 2013–2014.
Study participants Nineteen heart patients, four relatives and eight health professionals participated in qualitative interviews in the development phase, and 15 patients participated in the pilot-test. The questionnaire was subsequently sent to a random sample of 5000 heart patients who were diagnosed in 2013.
Results The comprehensive development phase and pilot-testing contributed to high content validity of the questionnaire. Eligible questionnaire responses were received from 2496 patients. EFA indicated a nine-factor model: communication at the hospital, communication with the general practitioner, information on disease and treatment, information on psychosocial aspects, rehabilitation/support, organisation, medication, involvement of relatives and consideration of comorbidity. CFA confirmed the proposed factor structure (eg, goodness-of-fit index=0.88, adjusted goodness-of-fit index=0.86, root mean square error of approximation=0.05), and Cronbach’s alpha coefficient revealed good internal consistency of the factors (range: 0.69–0.93).
Conclusions The results suggest that this disease-specific patient-reported experience measure is of good quality when measuring the quality of care among heart patients. The inclusion of patients in the development phase contributed to high content validity, and subsequent psychometric evaluation found high construct validity and internal consistency. This measure may be especially relevant when seeking information about which aspects of care require improvement and the impact on health outcomes.
- quality of care
- patient-reported experience
- heart disease
- factor analysis
- psychometrics.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
A main strength of this study was the thorough psychometric evaluation of the patient-reported experience measure.
The survey was nationwide and based on a population-based, random sample of 5000 heart patients, reducing the risk of selection bias.
The inclusion of a patient journey provides an important basis to explore the relationship between patients’ experiences with the quality of healthcare and long-term health outcomes and to discover which aspects of care to address to improve patients’ experiences across sectors.
Although this measure could be used in similar settings and patient groups, wider generalisation, across countries and patient groups, may be restricted.
This study is limited by non-responders and item non-response.
Introduction
Due to the impact on daily life and the high mortality rates, heart diseases constitute a major public health concern in both Denmark and worldwide.1 2 Consideration of patients’ experiences are increasingly recognised as essential in achieving high quality healthcare provision. In early 2000, the WHO and the Institute of Medicine (IOM) stressed the importance of patients’ perspectives for assessing and improving healthcare.3 4 WHO introduced the concept of responsiveness and IOM identified the provision of patient-centred care as one of six objectives for improving healthcare. The care should be ‘respectful of and responsive to individual patient preferences, needs and values and ensuring that patient values guide all clinical decisions’.4 Accordingly, enhancing patients’ experience with the quality of healthcare, which includes aspects such as treating patients with dignity and respect and meeting the patients information and communication needs5 has become a legitimate independent goal for healthcare services. Moreover, increasing evidence suggests that good experiences are positively associated with clinical effectiveness, including adherence to treatment, lower length of stay and decreased mortality in various settings and patient groups.6–9
As a result, surveys quantifying patients’ experiences with the quality of healthcare have become widespread and commonly include patient-reported experience measures (PREM).10–12 However, several questions remain unanswered. There is no agreement about what constitutes quality of healthcare from the perspective of patients, resulting in several multidimensional theoretical frameworks and a range of PREM instruments.10 According to Beattie et al, the purpose and context of the survey is important when choosing the PREM, particularly in relation to content validity.10 Also, what is important to patients may differ between countries, patient groups and healthcare settings, evolving over time. Existing instruments have been specific to hospital inpatients or outpatients, primary care patients and often relate to a single contact. However, to truly represent the perspective of patients, it can be argued that we need to take their experiences with a full patient journey into account. This would provide important information on which aspects of the care patients feel require improvement and provide a unique basis to examine the relationship between patients’ experiences and health outcomes, such as readmissions and death, including the relationship with the single patient experience dimensions. The impact on health outcomes of patients’ experience with a patient journey across sectors is expected to be greater than their experiences with a single consultation or healthcare setting. Information about the impact of patients’ experience is scarce, particularly for the dimensions of emotional support, coordination of care, continuity and transition and involvement of relatives.9 13 Thus, there is a need to develop context-specific PREM measures which include a patient journey with the perspective of long-term follow-up. The aim of this paper was twofold: first, to present the development of a national questionnaire measuring patient-reported experience of healthcare quality following heart disease, covering aspects of the patient journey from first contact with the healthcare system to the inpatient and outpatient treatment and rehabilitation and next, to perform a psychometric evaluation of a measure covering these aspects.
Method
Study design and setting
The questionnaire development was based on a literature review, qualitative focus group interviews and individual interviews. It was pilot-tested using cognitive interviewing techniques. The psychometric evaluation was assessed using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), Cronbach’s alpha coefficient and differential item functioning (DIF) analysis with data from a population-based survey conducted in Denmark in 2014. The Danish healthcare system is publicly financed with the goal of free and equal access for all 5.6 million inhabitants. The Danish health system is divided into two sectors: primary healthcare that are provided by general practitioners (GPs) and the municipal health services, where responsibilities include disease prevention, health promotion and rehabilitation outside of the hospital and the secondary (hospital) care being responsible for more specialised medical treatment (inpatient and outpatient care).14
Questionnaire development
The questionnaire was developed between September 2013 and August 2014. An overview of the stages in the development of the questionnaire is available (see online supplementary figure 1). We carried out a literature review and a qualitative explorative investigation. The aim was to develop a context-specific measure with high content validity to examine experiences with the Danish healthcare system among patients with a heart disease and to include the most important measurements of their health status. The patient experience measure was intended to cover aspects of the patient journey from first contact with the healthcare system to the inpatient and outpatient treatment and rehabilitation. The literature search was conducted in the databases PubMed and PsycINFO and included reviews (<15 years) and quantitative and qualitative studies (<10 years). The review was predefined to include 50–100 articles and the following criteria were used to select them: They should cover the four heart diseases of interest (ischaemic heart disease, atrial fibrillation, heart failure and heart valve disease), different topics and different study types. Based on titles, 300 studies were selected by one researcher (LZ), which was reduced by half by another researcher (TH) after reading the abstracts. Through a consensus meeting, the two researchers agreed on appraisal of 86 studies: 23 reviews, 29 quantitative studies and 34 qualitative studies. We systematically extracted information of each eligible paper independently using two data charting forms developed by LZ and TH, one reporting on the material and methods of each study and one reporting the results of the studies. The literature review was used as a supplement to the qualitative investigation and as a basis to develop the interview guide. The qualitative investigation consisted of three focus group interviews with 19 heart patients in total, one focus group interview with four relatives of heart patients and eight individual interviews with health professionals working with heart patients. Patients and relatives were recruited through the Danish Heart Foundations’ membership newsletter to which 34 patients and 8 relatives responded. They were selected based on their gender and their address to enable focus groups of at least four persons within a reasonable driving distance. The Danish Heart Foundations’ counselling centre recruited another four patients and two relatives to ensure the groups were large enough. The interviews were conducted in three different parts of Denmark. The patients included 9 men and 10 women with different educational background and different heart diseases. The relatives included two wives, one husband and a daughter of heart patients (further characteristics of the participants are available in the online supplementary table 1). The health professionals to be interviewed were selected to achieve a group representing different health professions (eg, physician, nurse, psychologist), health sectors (eg, hospital, general practice) and stages of care (eg, acute medical treatment, rehabilitation). The interviews were conducted at professionals’ work places or as telephone interviews. All interviews were conducted by LZ and TH. The semistructured interview protocols (see online supplementary table 2) were developed through discussions in the project group. The group included professionals from public health (TH, LZ), anthropology (MH), sociology (PJ) and cardiology (AZ). It focused on what was perceived by the patients to be important for the quality of healthcare, including areas they found problematic or insufficient and on the challenges they experienced living with heart disease. The interview protocols were modified for patients, relatives and experts, and the interviewers were open and flexible so as to include any issues the interviewees brought up themselves. The focus group interviews lasted approximately 2 hours and the individual interviews lasted between 1–1½ hours. All interviews were audiorecorded, and by listening to these recordings LZ and TH condensed the material, meaning that the experiences and opinions expressed by the interviewees were provided in a shorter form. The analysis conducted by TH and LZ was data-driven and guided by conventional thematic analysis strategies allowing us to specify major themes and subthemes in the material.15 They both read all summaries to get an overall impression of the material and identified themes and subthemes in each interview (half each). After this, they met to discuss, refine and reduce the themes into major themes with subthemes. The results of the exploratory preinvestigation have been published in Danish separately elsewhere.16
Supplemental material
Supplemental material
Supplemental material
A reference group comprising eight health professionals participated together in a consensus workshop to ensure the questionnaire covered the results from the preinvestigation. This process resulted in seven themes concerning patient experiences with the quality of care: information, communication, organisation, psychosocial aspects, rehabilitation/support, medication and involvement of relatives. The group was also consulted to develop specific questionnaire items within their area of expertise and review the first version of the questionnaire. The reference group comprised the same health professionals who took part in the individual interviews, besides three replacements who were researchers within the field. We used existing Danish patient experience surveys, as inspiration and as a first basis for creating the item wordings of PREM.17 The questionnaire was pilot-tested through five individual interviews and two focus groups with 10 heart patients, based on cognitive interviewing techniques.18 The questionnaire was further tested by two researchers with expertise in constructing surveys. Minor adjustments to the questionnaire were made.
The questionnaire
The final questionnaire consisted of 48 PREM items covering the seven themes of patient experience with the healthcare system identified in the preinvestigation: information, communication, organisation, psychosocial aspects, rehabilitation/support, medication and involvement of relatives. They covered aspects of the patient journey, from first contact with the healthcare system to the treatment and rehabilitation in the hospital, in the municipality and by the GP. The questions were measured on a four-point Likert-type scale from ‘yes, to a great degree’ to ‘no, not at all’, though a few questions were binary (yes/no). Further reply options were ‘do not know’ and ‘not relevant’ (primarily managed by filter questions ‘if you answered…, go to question.’). Besides this, the questionnaire also included measures of health status, including physical status, emotional status, anxiety and depression, sexual health, quality of life, overall health and adherence to medication. This information uncovers patient’s needs and provides information on whether healthcare goals have been achieved as most healthcare goals relate to improvement in these areas, besides basic survival.19 Patient characteristic items including measures of patient activation/engagement, decision-making preferences, comorbidity, education and employment status were also included. This information was primarily measured by internationally validated questionnaire scales. An overview of the full content of the questionnaire which took approximately 30–45 min to answer is available (see online supplementary table 3) and the original version is available online.20
Supplemental material
Sample
A random population-based sample of 5000 patients diagnosed with incident ischaemic heart disease, atrial fibrillation, heart failure or heart valve surgery in 2013 was selected from the Danish National Patient Register (NPR). NPR includes information on all inpatient and outpatient contacts in all public and private Danish hospitals.21 Disproportionate stratified random sampling was used to reach at least 400 respondents for each of the four diagnostic groups (taking previous response percentages among heart patients into account) in order to ensure that results could be presented for each group within reasonable resources. The sampling was based on the incidences of disease in Denmark in 2012. Patients were included if they had none of the specified heart diseases in the previous 5 years (2008–2012), were ≥35 years, Danish residents and alive. They were selected based on their diagnosis at discharge according to ICD-10 codes, except for heart valve patients, who were identified using surgical codes according to the Danish version of the Nordic Classification of Surgical Procedures (NCSP-D).22 If they had more than one heart disease in 2013, they were selected based on their first contact. The sample consisted of 2116 patients with ischaemic heart disease (ICD-10: I20, I21, I23, I24,I25), 1695 patients with atrial fibrillation (ICD-10: I48), 642 patients with heart failure (ICD-10: I11.0, I13.0, I13.2, I42.0, I150) and 547 patients who had undergone heart valve surgery (NCSP-D: KFG, KFK, KFM, KFJE, KFJF) (these four diagnoses are referred to as patient groups). The selection criteria were established in consultation with two cardiologists (AZ and the research director at the Danish Heart Foundation) and based on former register-based studies within cardiovascular diseases.23 24 Further information is available in (see online supplementary table 4).
Supplemental material
Data collection
The survey was conducted from October to December 2014. A paper questionnaire, a covering letter and a prepaid return envelope were posted to the patient sample group. The questionnaire could be completed on paper or online by using a unique password. If no response was received after 2 weeks, another questionnaire was sent and after 4 weeks they were contacted by telephone. Addresses were obtained from the Danish Civil Registration System (CRS) with information about all persons with a residence in Denmark, including name, date of birth, place of birth, place of residence and so on,25 and the company NN Markedsdata linked publicly available phones numbers to addresses.
Statistical analysis
Sociodemographic characteristics of respondents and non-respondents were compared according to gender, age (at diagnosis), diagnosis, education, income, region, civil status, ethnic background and time from diagnosis to the distribution of questionnaires to examine whether the non-respondents deviated from the respondents. School and business education was combined into a single measure, according to the International Standard Classification of Education.26 Average disposable income (after tax and interests) was calculated for the period 2008–2012 (100 DKK=13.44 EUR). Ethnic background was based on Statistics Denmark classification: persons of Danish origin (people born in Denmark with at least one Danish-born parent with Danish citizenship), immigrants (people born abroad with no Danish-born parents with Danish citizenship) and descendants (people born in Denmark with no Danish-born parents and without Danish citizenship). The information was obtained from CRS, NPR, the Population’s Education Register and the Income Statistics Register.27
We conducted factor analysis to explore if our data measured the dimensions (factors) identified in the preinvestigation (construct validity). EFA was used as preliminary analysis to explore the structure of relationships between the items using both varimax rotation (orthogonal) and promax rotation (oblique). A priori, seven factors were identified. We expected their respective items to have relatively high loadings on their specific factor and low loadings on other factors. A few items could theoretically enter more than one of the predefined factors and EFA provided insight into which factor was the best fit. To assess whether data were appropriate for factor analysis, the Kaiser-Meier-Olkin measure of sampling adequacy was used and a value above 0.6 is considered tolerable.28 To extract factors, we used the eigenvalue-greater-than-one rule, the scree plot and the proportion of variance accounted for. Further, the items that load on a factor should share a conceptual meaning and the items that load on different factors should appear to measure different constructs.28 Items were excluded when EFA showed low factor loadings (<0.4) and when Cronbach’s alpha increased markedly without the item. Cronbach’s alpha measures internal consistency (reliability) with a coefficient ≥0.7 considered to be good.29 Each dimension was also analysed separately, pairwise and so on, and ‘do not know’ responses were recoded to ‘no, not at all’ in a sensitivity analysis of the full model to allow most valid data to be used. The primary investigator (LZ) and a statistician (AKE) together compared the results from all sensitivity analyses with the main analyses and evaluated whether they pointed in the same direction.
To confirm the proposed factor structure by the EFA, CFA was performed. The overall adequacy of fit for the model was evaluated using a χ² test describing the similarity of the observed and expected matrices and by a number of goodness-of-fit indices: the goodness-of-fit index (GFI) with values>0.90 indicating a good model fit, the adjusted (for df) goodness-of fit index (AGFI) with values>0.80 indicating a good model fit, the standardised root mean square residual (SRMR) with a cut-off value close to 0.08, the root mean square error of approximation (RMSEA) with a small value indicating better model fit, typically 0.06 or less and the comparative fit index (CFI) with a larger CFI value indicating better model fit, typically 0.95 or greater.28 30 Nevertheless, the cut-off values are arbitrary but the measurements together provide a solid basis to evaluate the model fit. To include respondents with missing values, we used the full information maximum likelihood (FIML) method, which is suggested to be the most appropriate way to handle missing data in CFA.31 As a sensitivity analysis, we performed complete case analyses using the method ML and robust ML, the latter being robust to non-normality. Furthermore, ‘do not know’ responses were recoded to missing in an additional FIML analysis (otherwise ‘do not know’ responses were excluded). Finally, we excluded the largest patient group (ischaemic heart disease) from the main analysis.
Differential item functioning (DIF) (both uniform and non-uniform) was examined to evaluate whether men versus women and patients with ischaemic heart disease versus the three other patient groups respond differently to the measurement items using ordinal logistic regression,32 for example, if men and women responded differently to the item ‘Do you feel informed about how the disease may affect your future?’ after accounting for their overall level of information (uniform DIF) and if the direction of the difference changed as their overall level of information changed (non-uniform DIF). The likelihood ratio test was used to compare the goodness-of-fit of the models (with and without DIF). A significance level of 0.01 was used because the large number of significance tests increases the risk of type 1 error and the large sample size increases the risk of detecting unimportant, but statistically significant DIF.32 When DIF was statistical significant, the measure of effect size R2 was used to evaluate the clinical or practical significance using an R2 difference cut-off level of 0.02.32
Data were analysed using SAS V.9.3 (SAS Institute, Cary, North Carolina, USA).
Ethics
The survey with individual-linked information to registries is approved by the University of Southern Denmark (2015-57-0008, no. 17/329 and 17/8592) according to the Permission of the Danish Data Protection Agency. All participants were informed about the aim of the study and were assured that participation was voluntary and results would be anonymised. Verbal consent was obtained from individuals participating in the interviews, and consent was assumed for survey participants when they returned the questionnaire. This type of study does not require further formal ethical approval, according to Danish law.27 33 The method of surveying a sample identified in NPR entails ethical considerations. Some of the selected heart patients may not recognise their own condition, especially patients with asymptomatic atrial fibrillation, due to the character of the disease. Furthermore, register information is not always erased, even if subsequent examinations show that the patient does not have the disease. This can be due to administrative errors. We, therefore, informed all patients that they had been selected to receive a questionnaire because, according to register information, they had been examined/treated for a heart disease in 2013. Each patient was also asked whether they had been diagnosed with a heart disease. If not, they were asked to return the questionnaire without answering further questions.
Results
Study population
The questionnaire was returned by 3029 patients (61% of the invited population). Of these, 533 stated they did not have a heart disease in 2013, and they were excluded from the sample (adjusted sample 4467). A total of 2496 patients were eligible for inclusion (56%) (see figure 1). A total of 43% of the respondents had ischaemic heart disease, 11% heart failure, 30% atrial fibrillation and 16% had undergone heart valve surgery and 35% of the patients were women. Sociodemographic characteristics of the respondents are shown in table 1.
Flow diagram of the process from the sample to the final study population.
Sociodemographic characteristics of the respondents and non-respondents
Non-respondents analyses
The respondents differed from the non-respondents according to sociodemographic characteristics (table 1). Patients who were either male, married, had a heart valve disease or a higher education seemed to be over-represented in the response group. The difference in percentage points was below 10, with more than half below five (civil status ‘married’ did however have a 16.1 points difference and educational level ‘basic school’ a 10.9 points difference).
Validation of the PREM measurement
The predefined PREM dimensions from the preinvestigation, their items, responses and proportion of missing values are presented in table 2. The mean item-non-response was 12.1%.
Predefined dimensions from the preinvestigation, their items, responses (valid responses, don’t know, not relevant, missing) and proportion of missing values
Based on an overall picture of several analyses all pointing in the same direction (including subanalyses and sensitivity analyses) the EFA provided support for the predefined dimensions. The exceptions to this were the dimension ‘communication’ that was split in two and the dimension ‘consideration of comorbidity’ was found to be an independent dimension separated from ‘organisation’. Thus, it resulted in a nine-factor model: (1) communication at the hospital, (2) communication with the GP, (3) information on disease and treatment, (4) information on psychosocial aspects, (5) rehabilitation/support, (6) organisation, (7) medication, (8) involvement of relatives and (9) consideration of comorbidity, with a total of 41 items. Four items (E2, E5, B7, D5) were found to fit another dimension, rather than their predefined dimension, and seven items (A1, B6, D6, E1, E8, E9, F1) were excluded because they did not load substantially on any factors. Item E3 ‘Do you think a doctor or a nurse at the hospital have had an overview of the course of treatment of your heart disease at all times?’ was the only item with cross-loading. Based on theoretical considerations, it was placed within the dimension ‘organisation’ although it also loaded onto the dimension ‘communication at the hospital’. When the final nine dimensions were analysed separately, to allow most valid data to be used, EFA suggested one factor for each of them. The factor loadings were between 0.49 and 0.95, and the dimensions revealed good internal consistency with Cronbach’s alpha coefficients ranging from 0.69 to 0.93 (table 3). Results with all dimensions analysed simultaneously using promax rotation are presented in the online supplementary table 5. The results were similar regardless of rotation type (promax/varimax).
Supplemental material
Results from the final exploratory factor analysis exploring each dimension separately
The CFA supported the nine-factor model because it provided an acceptable overall fit: GFI=0.88, AGFI=0.86 SRMR=0.06, RMSEA=0.05 and CFI=0.93. The χ² test did though indicate a poor fit (p<0.001), but this test is limited by its sensitivity to sample size and model complexity. Thus, the model was expected to be rejected because of our large sample size and complex model, and therefore more emphasis was put on the other fit indices. The correlation between the factors (range=0.02–0.80, 54% below 0.50), the factor loadings for each item (range=0.43–0.95, 75% above 0.6) and the proportion of the variance for each item that could be explained by the factor (range=0.19–0.90, 58% above 0.50) are detailed in figure 2. All the sensitivity analyses confirmed the results. Statistical significant uniform DIF was detected in two items for gender (A2 and B1), for patient groups in four items (B3, B4, C3 and D5) and non-uniform DIF in one item (F3). Importantly, none of the effect sizes were clinical significant. The full text of the final nine-factor model with 41 items constituting the patient-reported instrument that measures experiences of healthcare quality in Denmark are available in the online supplementary table 6.
Supplemental material
![[SP7.jpg]](https://bmjopen.bmj.com/content/bmjopen/7/10/e016234/DC3/embed/inline-supplementary-material-3.jpg?download=true){kind=link}
Results from the confirmatory factor analysis , using the full information maximum likelihood method. N=814; χ2=15674, p=<0.0001; GFI=0.88; AGFI=0.86; CFI=0.93; RMSEA=0.05; SRMR=0.06. *N=507; χ2=12531, p=<0.001; GFI=0.87; AGFI=0.85; CFI=0.93; RMSEA=0.05; SRMR=0.06. *Complete case analysis, using the robust maximum likelihood method. Abbreviations: AGFI, the adjusted goodness-of fit index; CFI, comparative fit index; GFI, goodness-of-fit index; GP, general practitioner; RMSEA, root mean square error of approximation; SRMR, standardised root mean square residual. Note: The coefficients associated to the left are the correlations between the latent factors. The coefficients associated with arrows leading from the latent factors to the item products show the factor loadings for each item product. The coefficients to the right of the item products are the proportion of the variance for the item product that could be explained by the latent factor. n/a: Items excluded from the analysis, because they were irrelevant for a large proportion of respondents, reducing the data substantial: C1, E7, F5 and the factor consideration to comorbidity (E10 and E11). Subanalyses and sensitivity analyses supported the inclusion of these items.
Discussion
This study conducted a psychometric evaluation of a national questionnaire measuring patient-reported experience with the quality of patient care following a heart disease, covering aspects of the patient journey from first contact with the healthcare system to the inpatient and outpatient treatment and rehabilitation and presented the development of the questionnaire. The results suggest that the instrument is of good quality. The inclusion of heart patients in the comprehensive development phase and pilot testing of the questionnaire contributed to high content validity, and the subsequent psychometric evaluation found high construct validity and internal consistency. EFA indicated a nine-factor model: communication at the hospital, communication with the GP, information on disease and treatment, information on psychosocial aspects, rehabilitation/support, organisation, medication, involvement of relatives and consideration of comorbidity with a total of 41 items, and CFA confirmed this structure provided a good fit to the data suggesting high construct validity. Furthermore, Cronbach’s alpha coefficient revealed good internal consistency of the dimensions, and no clinical significant DIF was detected.
Some challenges regarding the measure were observed. CFA suggested that the factors organisation and communication at the hospital were highly overlapping as the correlation between the factors was 0.80. A factor correlation that exceeds 0.80 or 0.85 is often used as the criterion to define poor discriminant validity.31 We kept the factors separated, because they are conceptually different, another issue important to consider before collapsing factors.31 In future, it may be valuable to refine or expand the questions intended to capture the dimension organisation. The mean item-non-response was 12.1%, indicating a reasonable acceptance of the questionnaire. Even so, the question on smoking cessation guidance, if side-effects were taken seriously and the two questions regarding consideration of comorbidity had a proportion of missing values above 20%. These questions were irrelevant for a large proportion of the respondents and they might have skipped them without noticing the ‘not relevant’ response option. Simple filter questions may work better. The high proportion of ‘do not apply to me responses’ in certain items indicates a challenge with developing questionnaires to a wide group of heart patients. We chose to maintain such items to ensure that all aspects of importance to heart patients are represented in the questionnaire.
Implications
We believe this is the first PREM instrument among heart patients, covering aspects of the patient journey from first contact with the healthcare system to the inpatient and outpatient treatment and rehabilitation. This makes it particularly relevant when seeking information on aspects of care requiring improvement and when exploring long-term health outcomes. This new measure and the possibility of linking individual level patient data from Danish registries would allow future follow-up studies to examine the relationship between patients’ experiences with the quality of care and health outcomes such as readmissions and mortality. The measure would also be a suitable national surveillance tool to track performance of the healthcare system over time, as seen from the perspective of patients with heart disease. However, in acknowledging that what is important to patients may shift over time,10 we suggest refining the measurement on a regular basis. The measure is not appropriate for benchmarking hospital performance because several questions span across different sectors. Other measures are available for this purpose.10
Strengths and limitations
A main strength of this study was the thorough psychometric evaluation of the measure, which was based on a large population-based random sample of patients with heart disease limiting introduction of selection bias. However, a number of limitations should be considered. Missing-item non-response, ‘do not know’ and ‘do not apply to me’ responses reduced the number of valid responses and the data for the factor analysis substantially, even though the numerous subanalysis and sensitivity analysis all confirmed the results. The results of this study may not be generalisable to other patient groups; including patients with heart diseases other than those included in this study, patients below 35 years of age and patients only treated at their GP, since these patient groups are not represented in the survey. Nevertheless, it included the four most common heart diseases and the age groups where heart disease is most frequent.
Only half of the invited patients completed the survey, which is in accordance with other patient experience surveys,12 34 and the small differences between non-respondents and respondents indicate only minor implications. Also, no noteworthy differences were found according to age, educational level and region when comparing the respondents to the total population of heart patients (new cases in 2013),35 although men were over-represented (data not shown). The respondents were diagnosed between 10 and 22 months prior to the data collection and the delay might make it difficult for patients to remember their first encounters with the healthcare system, but it might also help to distinguish between minor irritations and more serious failings.5 Importantly, it allows for the inclusion of a patient journey from first contact with the healthcare system to the inpatient and outpatient treatment and rehabilitation. Also, the variation in time since diagnosis may have influenced the patients’ answers. To our knowledge, no studies have considered the possible effects of time on PREM. The instrument is developed among relatively newly diagnosed patients, and even though the instrument is developed to cover aspects of the patient journey, it does only give an indication of their experience until that point in time. Nevertheless, as long as it is possible to identify and specify which course of disease is under consideration, we believe the questionnaire can be used by anybody with a heart disease. However, this assumption has not been tested. This measure is recommended for similar settings and patient groups; however, wider generalisation across countries may be restricted due to differences in the organisation of healthcare systems and so on. Moreover, generalisation to other patient groups must be done with caution, because what is important to diverse patient groups may differ.10 Also, we acknowledge the importance of conducting qualitative studies to complement patient experience surveys and vice versa.
Conclusion
The results suggest that the developed PREM is of good quality for measuring the quality of care among patients with heart disease covering aspects of the patient journey from first contact with the healthcare system to the inpatient and outpatient treatment and rehabilitation. The involvement of heart patients in the development phase contributed to high content validity, and subsequent psychometric evaluation found high construct validity and internal consistency. The inclusion of a patient journey provides an important basis on which to explore the relationship between patients’ experiences with the quality of healthcare and long-term health outcomes and which aspects of care to address to improve patients’ experiences.
Acknowledgments
We sincerely extend our appreciation to all patients who took the time to complete this survey and to the patients, caregivers and health professionals who participated in the qualitative interviews as well as the reference group who participated in the development of the questionnaire ‘Living with a heart disease’. Further, we acknowledge the work of our collaborators at the National Institute of Public Health.
References
Footnotes
Contributors LZ, TH, AOZ, PDJ, NS and MHR designed the study and constructed the questionnaire. NS, TH, LZ and MHR collected the data. LZ, AKE, TH, NS and AOZ were involved in analysing the data. LZ, AKE, TH, AOZ, NS and HT interpreted the data. LZ drafted the manuscript. All authors contributed to the writing of the manuscript, critically revised the manuscript and approved the final manuscript.
Funding This work was supported by the Danish Heart Foundation grant number 15-R99-A5908-22941; The Danish Center of Rehabilitation and Palliative care; University of Southern Denmark and The National Institute of Public Health. The survey was supported by the Danish Heart Foundation.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.