Article Text
PDF
Abstract
Objective To investigate whether accreditation of general practice in Denmark promotes patient-reported quality of care and patient satisfaction.
Design A national cluster randomised case control study based on an online version of the Danish Patients Evaluate Practice questionnaire. Mixed effects ordered logit regression models taking account of clustering of patients in different municipalities were used in the analyses.
Setting General practice in Denmark.
Participants A representative sample of the Danish population.
Primary and secondary outcome measures The primary outcome measure was patient-reported quality of care, and patient satisfaction with general practice and patient satisfaction with the general practitioner served as secondary outcome measures.
Results In total, 3609 respondents answered the survey. We found no statistically significant relationships between patient-reported quality of care and practice accreditation (2016: OR=0.89, 95% CI 0.73 to 1.07 and 2017: OR=0.85, 95% CI 0.71 to 1.02) and between patient satisfaction with the general practitioner and accreditation (2016: OR=0.93, 95% CI 0.76 to 1.13 and 2017: OR=0.86, 95% CI 0.70 to 1.04). However, there was a statistically significant negative relationship between patient satisfaction with the general practice and recent practice accreditation compared with satisfaction with practices not yet accredited (OR=0.81, 95% CI 0.67 to 0.97) but no significant relationship between patient satisfaction with the general practice and previous accreditation (OR=0.91, 95% CI 0.76 to 1.09).
Conclusion Accreditation does not promote patient-reported quality of care or patient satisfaction. On the contrary, patient satisfaction with the general practice decreases when general practice is recently accredited.
- change management
- health policy
- organisational development
- organisation of health services
- quality in health care
- health & safety
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- change management
- health policy
- organisational development
- organisation of health services
- quality in health care
- health & safety
Strengths and limitations of this study
The target sample size of 1085 patients from each of the three groups of accreditation years was reached.
The test for successful randomisation between the three groups of patients showed that randomisation was successful on all variables.
Some patients residing in a municipality where general practice was recently accredited in 2017 may have visited the general practitioner (GP) before accreditation took place resulting in a conservative estimate of the true effect of accreditation.
It is not possible with our study design to link patients with their specific GP, and a few patients may therefore be listed with a GP located in another municipality which could be allocated to another year of accreditation.
We are not able to control for any GP and practice characteristics.
Introduction
In recent years, several countries worldwide have applied external interventions, financial and non-financial, to their healthcare systems in order to regulate and allocate the existing resources properly and improve quality of care for the patients.1 Health plans, report cards, certification and accreditation are examples of external interventions characterised by standardisation of various parts of healthcare in order to monitor and improve quality. Initially, these quality improvement programmes were targeted hospitals, but eventually, the concept was applied within the framework of primary care as well.2
Standardisation may promote the overall quality of care, and the criteria of success can be measured in several ways.3 Donabedian has argued that quality can be measured by seven attributes of healthcare. One of these is acceptability which refers to the accordance between the patients’ preferences and provision of care measured on accessibility, the relation between patient and practitioner, the amenities, effects of healthcare and the healthcare costs. Hence, the patients’ perceptions of aspects of healthcare are important parameters that should be taken into consideration when measuring quality of care in healthcare systems. These types of parameters are often referred to as patient-reported experience measures (PREMs).4
Patient-reported quality of care is normally part of an overall assessment of the quality of care in healthcare systems, and many countries have integrated patient evaluations in their national accreditation programmes, either as a criterium in the accreditation decision or as a potential quality improvement tool.5 In Denmark, one aim of accreditation of general practice is to promote patient-reported quality of care.6 However, there is a lack of studies investigating the relationship between accreditation and PREM.
Patient-perceived quality of care has, however, been used as the primary outcome variable in several studies investigating organisational changes as a way to ensure patient-centred care.7 8 Rodriguez et al explored organisational influences on physician performance on patient experience measures and found no associations between use of patient experience improvement strategies and the perceived physician performance.7 In fact, increased emphasis on productivity and efficiency criteria was associated with lower patient perceived access to care. Elmore et al investigated the relation between the length of consultations and patient-perceived quality of care but found no associations either.8
There are also studies investigating a relation between the introduction of financial incentives and PREM.9–13 However, the results of the studies are inconclusive. van den Hombergh et al investigated the influence of organisational changes in Dutch general practice including increased reimbursement related to the performance of general practitioners (GPs) and practices.9 They found that the number of patients with positive experiences of their GP and their practice increased by 4.8% and 6.6%, respectively, from 2007 to 2012. Maeng et al explored how a newly introduced patient-centred primary care model improved patient experienced care.11 The healthcare sites enrolled in the programmes received fee for service as in traditional primary care, and additionally, they were reimbursed according to their performance.12 They found that patients receiving care in sites using this model were significantly more likely to report positive changes in their care experience and quality. However, Campbell et al examined changes in patient-reported quality of care for communication, nursing care, coordination and overall satisfaction in relation to introduction of a pay-for-performance scheme in the UK but found no significant results.10
Aim
The overall aim of this study is to investigate whether accreditation of general practice in Denmark promotes patient-reported quality of care and patient satisfaction.
Setting
Danish general practice
There are approximately 3600 GPs distributed on 2200 practices covering the Danish population of around 5.6 million inhabitants.14 All Danish citizens are entitled to have most of their healthcare financed through taxes and to be listed with a GP if they choose one within the municipality in which they live.15 A GP can, however, accept to enrol a patient on his/her list even though the patient lives in another municipality. A total of 98% of the Danish citizens are enlisted with a GP.
Mandatory accreditation of general practice
Accreditation as a tool for quality assessment is commonly used in healthcare systems worldwide, and considerable resources have been spent in development and implementation of accreditation schemes.16 In Australia, accreditation was established in primary care more than two decades ago.17 National accreditation standards in several countries have ever since been developed for primary care—primarily resting on the principle of voluntariness. To succeed with their accreditation, a healthcare organisation has to meet a number of predetermined criteria of an accreditation scheme defined by a professional accrediting agency.18 19
The organisation of GPs in Denmark agreed as part of the collective agreement with the Danish regions to undergo a compulsory accreditation programme in general practice from January 2016 to December 2018. The design is a stepwise cluster randomised process where randomisation of the 98 municipalities in Denmark determines the allocation of general practices to accreditation year (2016, 2017 or 2018). Finally, randomisation of the order of practice accreditation within the municipalities was randomised as well.20
A national accreditation agency, The Danish Institute for Quality and Accreditation in Health Care (IKAS), has developed the accreditation programme and manages it continuously. The programme covers 16 standards each including several indicators which the general practices are obliged to fulfil in order to be accredited. Originally, 19 standards were developed based on expert consensus. These standards were pilot tested in 10 general practices. Based on this pilot test, the standards were reduced to the 16 standards included in the final accreditation scheme.21 The 16 standards are distributed on 4 themes which are: (1) quality and patient safety, (2) patient safety critical standards, (3) good patient continuity of care and (4) management and organisation. For an overview of the 16 standards, see box 1.
The 16 accreditation standards for general practice
Quality and patient safety
The professional quality
Application of good clinical practice
Adverse events
Patient evaluations
Patient safety critical standards
Prevention of mix-up of patient identities #
Medicine prescribing and renewal of prescriptions #
Paraclinical examinations #
Emergency standby and heart failure #
The patient chart, data privacy and confidence #
Good patient continuity of care
Accessibility
Referral
Coordination of continuity of care
Acquisition, storage and disposal of clinical utensils and medicines/vaccines
Management and organisation
Hygiene
Management and service
Recruitment, introduction or competency development
In order to fulfil the 16 standards listed in box 1, a number of indicators within these standards should be taken into consideration, and the GPs should document how they complied with them. Moreover, the practices should outline detailed procedures for each of the indicators, for example, written procedure on renewal of medicine. This written documentation was sent to the surveyor team prior to their visit to the practice. During the visit, the surveyors could ask further questions about the fulfilment of the indicators. Hence, the practices were free to choose the process in order to fulfil the standards. However, the accreditation institute did produce support function in the form of a web page with guidance, document management tools, good examples of guidelines and so on. Further, the practices could access quality data on their own performance.
The overall goal of the accreditation programme is to promote the clinical, organisational and patient experienced quality into general practice,6 the latter being the focus area of this study.
None of the standards in the accreditation scheme are directly related to patient-perceived quality of care or patient satisfaction, and therefore, one might not expect to find an effect of accreditation on these measures. However, some of the standards are more likely to be influential than others, for instance, ensuring accessibility, continuity of care and performing patient evaluations. The standard related to patient evaluations is a process outcome, that is, whether an evaluation was performed, and does not take into consideration the results of the evaluation. The Danish accreditation programme prescribes that a patient evaluation is conducted at least every third year using the Danish Patients Evaluate Practice (DanPEP) questionnaire and that the GPs follow up on the results of it.
To check whether practices complied with the accreditation standards or not, they were given a visit by a surveyor team. For each indicator in a standard, the surveyors assessed it to be either met, largely met, partially met or not met. Based on this assessment, the Accreditation Award Committee awarded one of three accreditation statuses: accredited, accredited with remarks or not accredited.22 Accreditation was a relatively weak incentive as there were no sanctions given if a practice was not accredited. Practices that were not given the award of accreditation would, however, suffer reputationally as all assessments are publicly available (https://www.ikas.dk/afg%C3%B8relser/almen-praksis/).
Methods
Survey design and outcome measures
The study was based on the validated DanPEP questionnaire concerning patient-perceived quality of care provided by the patient’s regular GP and was developed based on the European Project on Patient Evaluation of General Practice Care questionnaire (EUROPEP)23 which has been recommended as the best tool for measuring this perspective on care in general practice.5
Hence, DanPEP is a tool for measuring aspects of quality of care in relation to the patient’s regular GP. However, according to the outlined aim, we developed and added three questions to the questionnaire concerning the respondents’ overall assessment of the GP, the general practice and the care in the practice. The latter was phrased to precisely reflect the wording of the aim of accreditation as formulated by IKAS6 and served as our primary outcome variable. The three outcome measures were measured on a 5-point Likert scale in accordance with the original response categories in the DanPEP questionnaire.
Patient and public involvement
This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.
Data collection
The online survey was sent out by a Danish data collection agency in January 2018 to an adult panel. Panel members have consented in writing to participate in the agency’s surveys. The gross sample of respondents was stratified on age, gender (also within each age group) and education. Quotas were set on municipality level to ensure that the number of respondents from each municipality reflected the relative size of the municipality. Respondents were screened based on how many times they had visited their GP during the past year. Only patients with one or more visits were included in the final sample to enable analyses of PREMs based on current experience.
The target sample size was 3255 patients, corresponding to 1085 patients from municipalities randomised to each of the three accreditation years (previously accredited in 2016, recently accredited in 2017 and not yet accredited). This target was set based on a one-way analysis of variance power calculation using data collected from a similar question on a comparable sample from the Danish population in 2010.24 The significance level was set to 0.05, power was 80% and the probability of a type II error 20%.
As approximately 87% of the Danish population visit their GP at least once a year,25 the agency collected 4315 responses to reach the target sample size. In total, 3609 respondents (corresponding to 84% of the surveyed patients) had visited their GP at least once within the past year, and these were included in our sample. Of these, 1094 patients resided in a municipality where general practice was previously accredited in 2016, 1327 patients resided in a municipality where general practice was recently accredited in 2017 and 1188 patients lived in a municipality where general practice was not yet accredited but was randomised to be so in 2018. See figures 1 and 2 for timeline of the project and a flow chart.
Timeline.
Flow chart. GP, general practitioner.
Statistical analyses
Due to the cluster randomised design of the accreditation intervention and the timing of data collection, it is possible to compare the outcome measures from patients residing in municipalities where general practice has been previously or recently accredited (2016 and 2017) with patients residing in municipalities where general practice was not yet accredited (2018). In 17 of the 98 municipalities in Denmark, the different general practices in the municipality were split between accreditation years. For these 17 municipalities, we set the accreditation year to the year where most general practices were accredited. As a robustness check, we also ran the analyses excluding these 17 municipalities.
As general practices were randomised to accreditation year at municipality level, we assumed that patients residing in a given municipality were also allocated to accreditation year at random. However, we tested for successful randomisation of respondents between accreditation year on a number of variables by means of χ2 test statistics to check for allocation bias in our design.
The effect of accreditation on our primary outcome variable—patient-reported quality of care—and our two secondary outcome measures—patient-reported satisfaction with the general practice and patient-reported satisfaction with the GP—was estimated by means of mixed effects ordered logit regression models taking account of clustering of patients in different municipalities. These models are preferred to maintain the level of detail in our variables and to be able to incorporate the clustering of patients in municipalities. As randomisation was performed at the municipality level, it is not possible to include municipality fixed effects. All standard errors were specified as robust and allow for intragroup correlation at the highest level in the multilevel models. The use of robust standard errors together with random effects ensures that the standard errors are also robust if there is heteroscedasticity beyond the clustering.
Accreditation in 2016 or 2017 was our key explanatory variables. These were included as dummy variables in the models with accreditation in 2018 as our reference level. In all models, we controlled for age, gender, education, number of visits to the GP within the last year, years listed with current GP, self-assessed health status and knowledge of accreditation to take account of any remaining differences between groups. All control variables were dummy coded. As a robustness check, we also ran the models without control variables. To reduce information loss, ‘cannot answer/not relevant’, responses to the three outcome measures were imputed using (1) exact mean values and (2) integer mean values. As results were robust to the two different imputation strategies, we only reported results where integer values were used. However, a sensitivity analysis was performed excluding respondents who answered, cannot answer/not relevant.
Results
Test for successful randomisation and descriptive statistics
The test for successful randomisation between our three groups of patients allocated to accreditation year shows that randomisation is successful on all variables (table 1).
Test for successful randomisation between groups (n=3609)
Our descriptive results in table 2 show that the target sample size of 1085 patients from each group was reached. Most patients have had 1–2 consultations with their GP within the last year, and almost two-thirds of the patients have been listed with their current GP for 6 years or longer. Most of the patients stated that they had not heard about accreditation (83.2%), while 8% reported to have heard about it and 8% did not know. The patient-reported quality of care and satisfaction is rather high. For the primary outcome measure, two-thirds of the patients assessed the care in their general practice as either good or excellent while only 10% assessed it to be fair or poor. For our two secondary outcome measures on patient satisfaction, two-thirds of the patients assessed the general practice and the GP as good or excellent. This level is similar to what has previously been found in a study using the DanPEP questionnaire.23 Online supplementary file 1 shows the patient-reported quality of care and satisfaction split on accreditation years.
Supplemental material
Descriptives (N=3609)
Patient-reported quality of care
The results for our primary outcome measure, patient-reported quality of care, are not statistically significant for neither of the accreditation years 2016 and 2017 compared with the accreditation year 2018 (table 3). When we exclude the 17 municipalities with varying accreditation years, patient-reported quality of care becomes statistically significantly lower for patients residing in municipalities where general practice was recently accredited in 2017, compared with practices not yet accredited in 2018. See online supplementary file 2. Hence, we find no evidence that accreditation promotes patient-reported quality of care.
Supplemental material
Mixed effects ordered logit regression analyses (N=3609)
Patient-reported satisfaction measures
The results for our secondary outcome measure on patient-reported satisfaction with the general practice show that there is no statistically significant relationship between the satisfaction with the general practice and residing in a municipality where general practice was previously accredited (2016). However, there is a negative statistically significant relationship between the satisfaction with general practice and residing in a municipality where general practice was recently accredited (2017) compared with the satisfaction of patients residing in municipalities where general practice was not yet accredited (OR=0.81, 95% CI 0.67 to 0.97; table 3). This finding is robust when we exclude the 17 split municipalities (see online supplementary file 1).
With respect to the other secondary outcome measure on patient-reported satisfaction with the GP, there are no statistically significant relationships with neither of the two accomplished accreditation years, 2016 and 2017, compared with accreditation year 2018 (table 3). However, when excluding the split municipalities, we also see a statistically significantly negative relationship between patient satisfaction with the GP residing in municipalities where general practice was recently accredited in 2017 compared with 2018 (see online supplementary file 2).
The results for our control variables show that patient-reported quality of care and satisfaction are positively associated with being older, having three or more consultations per year, being listed with the current GP 6 years or longer, having an excellent or very good self-assessed health and having heard of accreditation of general practice.
When running the models without control variables (see online supplementary file 3), the results are robust to the two different specifications. Moreover, our results are insensitive to excluding respondents who answered, cannot answer/not relevant (see online supplementary file 4).
Supplemental material
Supplemental material
Discussion
The aim of accreditation of general practice is, among other things, to promote patient-experienced quality of care. However, we find no evidence that this aim is fulfilled, neither for patients residing in municipalities where general practice was previously accredited in 2016 or recently accredited in 2017. On the contrary, we see a statistically significant negative effect on patient-reported satisfaction with the general practice for patients residing in municipalities where general practice was recently accredited. This effect is not found in practices previously accredited in 2016, indicating that it is only temporary. Generally, when we exclude the 17 split municipalities, we find a significant negative association between our PREMs and residing in a municipality where practices were recently accredited.
The fact that we find no positive associations between patient-reported quality of care and accreditation may be due to the content of the 16 accreditation standards as they are not only related to quality of care but also contain indicators on other things in general practice, for example, management and organisation. A narrative synthesis of health service accreditation literature also suggests that accreditation targets or influences aspects of health service function and delivery that are less visible to patients.26 This result gives rise to the idea that new accreditation standards closely related to this aspect of care could be identified and added to the programme to achieve this goal. Other possibilities are that the standards have no bearing on the patients’ experiences of quality of care at all or that the practices already performed well on the standards before the programme was initiated.
There could be several reasons why we found a negative effect on patient-reported satisfaction with recently accredited general practices and not on practices which were previously accredited in 2016. First, as accreditation is a rather resource-demanding programme to implement, patients may have experienced a temporary decrease in quality of care, while their practice focused on things related to the fulfilment of the standards, thereby influencing provision of care. Second, the patients might have experienced constraints in relation to implementation of the accreditation standards. For instance, to fulfil the patient safety standard, general practice could decide to discontinue renewal of addictive medicine or otherwise unsafe medicine, for example, non-steroid anti-inflammatory drugs to the elderly. Hence, the standards do not necessarily relate to the promotion of patient satisfaction but could also in some cases thwart satisfaction. Third, the results could also be explained by another scenario grounded in organisation theory.
In 1947, the German-American psychologist, Kurt Lewin, developed the ‘Unfreeze—Change—Refreeze-model’.27 In this model, he theorised about three steps every organisation must go through in order to be successful with a planned organisational change and sustain the results of it. At the unfreeze stage, the planned organisational changes are presented to the employees, and this causes frustrations and reluctance. At the change stage, the planned changes are to be implemented, but for this to happen, the culture of the organisation is essential and should be worked on intensively at the previous unfreeze stage. Otherwise, they are more inclined to relapse into old habits and rituals at the refreeze stage where the changes are to be sustained.
Hence, if there is a lack of a strong culture supporting the changes, this could explain why practices accredited in 2016 did not show any significant results as the practice after completing the process may have relapsed into the previous way of working. The applicability of the theory in the context of accreditation of general practice is supported by the study by Teodoro et al showing that successful accreditation is associated with a strong culture in an organisation.28 Moreover, a baseline survey among Danish GPs showing that as many as 45% had negative a priori attitudes towards the accreditation programme may further underpin this explanation of the results.29
Our study results are in contrast to the findings of van den Hombergh et al who found significant associations between accreditation in Dutch general practice and the number of patients having positive experiences with GPs and especially general practice from 2007 to 2012.9 This difference may occur because van den Hombergh et al linked the patients to the individual GP whereas we linked patients to municipalities. Due to the cluster randomised design, this enabled a rigorous test of the effect of accreditation. Another explanation could be that many other organisational changes were taking place in Dutch general practice during this period and that the Dutch accreditation model was voluntary and partly incentivised unlike the Danish mandatory and non-financial accreditation model.
The results by van den Homberg et al are supported by the study by Maeng et al who found that patients receiving care in sites using a new patient-centred primary care model were significantly more likely to report positive changes in their care experience and quality than patients receiving traditional care.11 Like the Dutch accreditation programme, the model explored by Maeng et al was incentivised.
However, other studies reporting on incentivised improvements strategies did not find any positive associations between those and the patient-reported experience of quality of care.7 10 13 For instance, Rodriguez et al who found no associations between use of incentivised patient experience improvement strategies and the perceived physician performance but instead found associations between increased emphasis on productivity and efficiency criteria and lower patient perceived access to care.7 This may indicate that the reason for the differences between results of studies reporting on primary care improvement strategies should be found elsewhere. As in the study by van den Hombergh et al, a variety of organisational changes were applied to the primary care setting under investigation by Maeng et al enhancing the probability of this explanation.
There are limitations to this study. First, some patients residing in a municipality where general practice was recently accredited in 2017 may have visited the GP before accreditation took place as it is an ongoing process. This could dilute our results for the 2017 group. If this is the case, our statistically significant results may be a conservative estimate of the true effect of accreditation. However, as GPs are notified about their accreditation date 1 year in advance, it is highly likely that the GPs were in the process of working with the accreditation standards during the whole year of 2017. Hence, we do not see this as a major drawback of the study.
Second, it is not possible with our study design to link patients with their specific GP. A few patients may therefore be listed with a GP located in another municipality which could be allocated to another year of accreditation. However, due to the randomised design, this should not skew our results. The assumption that patients residing in a given municipality are also allocated to accreditation year at random is deemed valid as it is highly unlikely that patients have selected their place of residence based on the allocation of accreditation year to the municipality and their GP. Also, only 8% of our sample reported to have heard of accreditation in general practice, and, importantly, our test for successful randomisation on observable variables showed that patients were randomly allocated.
Third, we are not able to control for any GP and practice characteristics. However, as practices are randomised to accreditation, this should not affect our results. Hence, the combination of the design of the accreditation programme and our statistical approach enhances our contribution.
Fourth, spillover effects between practices allocated to different accreditation years are a possibility, which could explain the lack of significant differences between years. However, as practices were notified about the date for accreditation 1 year in advance, it seems unlikely that the practices would start working on fulfilling the accreditation standards before the date of the surveyor visit is disclosed. Moreover, it also seems unlikely that the practices would have been neither ready, nor willing to implement all the changes several months in advance of the scheduled visit. Finally, it may take more than a few years before the effect of accreditation fully appears. If this is the case, accreditation may positively affect PREMs in the future. In relation to this, the cultural changes that might take place during an accreditation process could be interesting to capture and use strategically in general practice prospectively. Future studies could investigate this.
Conclusion
We find no evidence that mandatory accreditation of general practice in Denmark promotes patient-reported quality of care or patient satisfaction. On the contrary, patient satisfaction with general practice decreases when general practice is recently accredited.
Acknowledgments
Thanks to Sonja Wehberg for assistance on power calculations and to Kirubakaran Balasubramaniam for fruitful comments on the project.
References
Footnotes
Contributors LBP conducted the data analyses. HR and LBP interpreted the results of the data analyses and wrote the draft article. All participated in planning the design of the study and took part in the revisions and methodological discussions as well. All of the authors read and approved the final manuscript.
Funding This research received an unrestricted grant from the Danish Institute for Quality and Accreditation in Healthcare (IKAS) and the Danish Research Foundation for General Practice.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Ethics approval The project received approval from the Danish Data Protection Agency (file number 2016-41-4579).The survey on which the project is based was sent out by a Danish data collection agency to members of an online adult panel. Panel members have consented in writing to participate in the agency’s surveys.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available. The data are stored at the Research Unit for General Practice, University of Southern Denmark, but are not available due to Danish legislations.