Article Text
PDF
Abstract
Objectives This study is a process evaluation of the Quality Improvement in Chronic Kidney Disease (QICKD) study, comparing audit-based education (ABE) and sending clinical guidelines and prompts (G&P) with usual practice, in improving systolic blood pressure control in primary care. This evaluation aimed to explore how far clinical staff in participating practices were aware of the intervention, and why change in practice might have taken place.
Setting 4 primary care practices in England: 2 received ABE, and 2 G&P. We purposively selected 1 northern/southern/city and rural practice from each study arm (from a larger pool of 132 practices as part of the QICKD trial).
Participants The 4 study practices were purposively sampled, and focus groups conducted with staff from each. All staff members were invited to attend.
Interventions Focus groups in each of 4 practices, at the mid-study point and at the end. 4 additional trial practices not originally selected for in-depth process evaluation took part in end of trial focus groups, to a total of 12 focus groups. These were recorded, transcribed and analysed using the framework approach.
Results 5 themes emerged: (1) involvement in the study made participants more positive about the CKD register; (2) clinicians did not always explain to patients that they had CKD; (3) while practitioners improved their monitoring of CKD, many were sceptical that it improved care and were more motivated by pay-for-performance measures; (4) the impact of study interventions on practice was generally positive, particularly the interaction with specialists, included in ABE; (5) the study stimulated ideas for future clinical practice.
Conclusions Improving quality in CKD is complex. Lack of awareness of clinical guidelines and scepticism about their validity are barriers to change. While pay-for-performance incentives are the main driver for change, quality improvement interventions can have a complementary influence.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This study provides important analysis of qualitative factors that affect chronic kidney disease management in primary care, beyond pay-for-performance strategies.
Involvement in the study was rewarding, made participants more positive about CKD, and they perceived improve care.
Many were sceptical that identifying CKD improved care and pay-for-performance was a greater motivator.
The study assessed a small number of practices (four in each arm, eight in total), and results may have been strongly influenced by opinion leaders within practices.
Background
Chronic kidney disease (CKD) is a common condition and an increasingly important public health issue.1 It is estimated that CKD amounts to approximately 1.3% of annual health service spending in England.2 CKD is an established, independent risk factor for cardiovascular disease;3 significantly more patients with CKD die from cardiovascular complications than progress to end-stage kidney disease.4 ,5 Most patients with mild-to-moderate CKD are treated in primary care.6 General practice is therefore an important service sector to target for initiatives to improve CKD care, principally by lowering blood pressure (BP), and to slow progression of renal disease7 and reduce cardiovascular risk.8 ,9
There is limited evidence about how best to improve the quality of CKD management in primary care.10 The Quality Improvement in Chronic Kidney Disease (QICKD) study was designed to address this question. QICKD was a three-arm cluster randomised control study comparing the effect of two recognised quality improvement (QI) interventions with usual practice (UP) in improving systolic BP control in CKD, in primary care settings across England.11 The two QI interventions were audit-based education (ABE) and guidelines and prompts (G&P). ABE is a QI intervention developed over the past 15 years, which provides education, peer support, and uses audit to document the gap between achievement and guidelines. This is a complex, non-judgemental, educational intervention underpinned by the use of IT to extract and make comparisons between practices and against evidence-based guidelines (box1).12 In observational studies, ABE improved the quality of cardiovascular disease (including CKD) management,13 as well as in the QICKD trial.14 G&P is a much less elaborate intervention and the most commonly used QI intervention in primary care.15 This usually involves distribution of clinical guidelines, generally on paper, sometimes with postal reminders as well as internet resources.
Components of audit-based education (ABE)
ABE is an intervention developed over 10 years ago; its aim is to provide feedback about performance against guidance. ABE includes feedback about quality compared with peers in a workshop setting usually led by a local general practitioner (GP) with a specialist (nephrology consultant or nurse) available as an expert resource, and also supported by academic detailing. Usually, ABE also identified lists of patients within the practices needing intervention. It consists of:
Anonymised extraction of the data set required to report whether there was any quality improvement. The usual components are:
Denominator to allow standardisation of prevalence,
Subset of people with the target condition—to create a virtual disease register,
Clinically relevant comorbidities, risk factors and treatment.
Processing these data listed above to make them informative to the target audience. In addition providing comparative feedback combined with academic detailing. A key feature is presenting comparative feedback comparing practices at twice yearly meetings held within a locality/primary care organisation. These meetings are called Data Quality Workshops (DQW), generally locally led with a consultant of the relevant discipline attending as a specialist resource.
In addition to the presentation at the DQW, practices are provided two additional printed aids:
‘Laminate’—a single laminated A4 page summary of the practice demographics and case ascertainment compared with others who attended the DQW. This is for the practice notice board or other prominent location (we recommend wherever they take their breaks).
Workbook—a slide by slide explanation of the DQW presentation—and what the data mean for their practice, compared with their peers and any evidence-based guidance.
Running local searches in the practices to provide lists of patients that need to be targeted for intervention. These lists are usually generated by an individual GP. Experiential learning is that audit lists of up to 150 per 10 000 registered patients result in change.
Supporting education about the evidence base and providing coding or other computerised medical record system training is provided as required.
Participants have been encouraged to contribute to the future development of the ABE programme.
The QICKD trial reported a modest but statistically significant improvement in systolic BP in people with CKD exposed to ABE compared with UP (OR of 1.24 of achieving at least a 5 mm Hg reduction in systolic BP), whereas G&P showed no significant difference.14
Given the complex nature of QI interventions such as G&P and ABE, we carried out a process evaluation to explore practitioners' perceptions of the extent to which they were exposed to the intervention, the nature of that exposure and the degree to which the trial interventions might have influenced practice. A secondary aim of the study was to identify what factors might have influenced the management of CKD in community settings over the course of the trial.
During the course of the trial, there were a number of changes in primary care that affected CKD management. Pay-for-performance (P4P) was introduced in April 2004, mainly targeted on vascular disease, with a CKD domain added in 2006. This scheme uses routine data to determine the level of case ascertainment, on a disease register, and sets financially incentivised quality indicators; practitioners are asked to maintain a list of patients reaching the criteria for a diagnosis of CKD and ensure that they meet certain treatment targets. The CKD indicator includes a treatment target of keeping BP below 140/85 mm Hg preferentially using angiotensin-modulating drugs in the presence of proteinuria. However, national evidence-based guidance proposes different thresholds and targets for treatment. P4P guidance sets less challenging BP levels than national evidence-based guidelines (table 1).
Guidance introduction before and during the QICKD trial, including variation in BP target and proteinuria measures
Methods
Overview: We identified four practices, two from each active study arm for in-depth process evaluation. These practices undertook two focus groups: one at the midpoint during the study and another at the end of the study. In addition, another four practices undertook a focus group at the end of the study. The focus groups were recorded, transcribed and analysed using the framework approach.16
Description of the QICKD trial practices
We recruited 132 practices, with 16 dropping out before randomisation. There were 30, 32 and 31 practices allocated to the ABE, G&P and UP arms, respectively. In addition, we recruited a further 10 per arm who also received a questionnaire about their confidence and competence in managing CKD (but due to an allocation error, there were actually 29). This was carried out to test whether QI might be secondary to improving confidence, knowledge and competence in managing CKD.17 We identified four practices, two from each active study arm for in-depth process evaluation using focus groups.
Participants and recruitment
At the start, four general practices were selected to participate in the process evaluation, in parallel to the randomised control study. Two practices from this group received ABE, and the other two G&P. These practices participated in focus groups at the midpoint and at the end of the trial period. We purposively selected one from the north, one from the south, and one city and one non-urban practice from each active study arm. Another four practices were involved in a focus group at the end of the project (again, two received ABE and the other two G&P). The additional focus groups were undertaken in order to increase the representativeness of the findings (12 groups in total). All focus groups were attended by the 6–9 members of the multiprofessional team (general practitioner (GP), practice nurses, healthcare assistants and practice manager). The focus groups were undertaken by a member of the research team and an experienced qualitative researcher.
Data collection
Focus groups are advantageous in their ability to extract the attitudes and beliefs of participants, and are more likely to do so when compared with individual interviews or observation.18 ,19 The focus groups comprised 6–9 health professionals; GPs were a majority in all, and other professions were practice manager and practice nurse. A non-prescriptive interview guide was developed by the research team in order to guide the interaction and prompt reflection, but no attempt was made to suggest responses or limit discussion, thus allowing maximal interaction between participants and an opportunity for related concepts to emerge. The four areas explored were:
Understanding the effects of the two QI interventions;
Explore the extent to which practices were aware of the interventions and their impact;
Understanding how CKD management had changed since P4P was introduced;
Identifying any other factors which may have impacted on the quality of CKD management during the study.
Data analysis
Focus groups data were transcribed and analysed using a multistep iterative process known as framework analysis. The framework method was selected because it is a method that is considered suitable for use in teams where not all members have experience of qualitative research.20 The framework approach consists of an initial period of familiarisation with the data; verbatim transcripts were created by a specialist clerical officer at the University of Surrey, noting recurrent or intensely expressed issues and themes.21 We subsequently developed a thematic framework using the issues and themes noted in the first stage and their relevance to the four areas intended to be explored in the focus groups. These themes were then indexed and charted according to themes and subthemes. The final step was a review of these charted themes in order to identify associations, detect anomalies and refine the thematic framework. Thus, a set of major themes is outlined in the Results section, with associated subthemes.
Ethical considerations
The study, including its process evaluation, was approved by a research ethics committee prior to the start of the trial.
Findings
Overview of the findings
Overall, the practitioners found that participating in the trial improves their understanding of CKD. However, the concept of CKD itself, how risk is stratified in CKD and changes in management made it a challenging condition to explain to patients. Practitioners perceived that P4P targets may have had a greater impact on CKD management than the study intervention. They enjoyed and found the face-to-face education useful, but some found the paper feedback they received confusing.
Improving the validity of the CKD disease register
Practitioners were positive about the concept of a CKD disease register. Several participants acknowledged that initially they were not aware that having two estimated glomerular filtration rate (eGFR) readings, at least 3 months apart, was needed to meet the diagnostic criteria for CKD; creation of a CKD register was considered useful in limiting overdiagnosis of CKD.We used to put our patients on [the CKD disease register] straight away if they only came once in for eGFR measurement which was below 60, or it might be 58 and they ended up going on the register…the register has maybe got a bit smaller after that and the reason for that was actually cleaning it up and making sure patients are there appropriately. (FG2)
Nevertheless, there was variation between practices in how the register was maintained and new patients added. Some practices were systematic and used audit tools to identify cases, while others opportunistically identified patients with CKD in routine primary care consultations, or looked for cases proactively, in chronic disease management clinics (eg, practice hypertension clinics).
Explanation of CKD diagnosis
Many practitioners commented on the difficult balance between avoiding unnecessary anxiety and social stigma by giving patients what they considered a potentially confusing diagnosis. They were concerned about labelling patients with a chronic disease label, especially older people in whom CKD is extremely common. At the same time, they did not want to conceal information from them. It was noted that patients often required a great deal of reassurance and explanation of the condition, even with early-stage disease—particularly given the often silent nature of the condition:But sometimes it’'s difficult, especially with CKD 3 for example you just kind of, you don’t want to label them as chronic kidney disease and it’s difficult for them to kind of come to terms with it. (FG1)
People automatically think dialysis, don't they? So ‘You've got kidney disease’ and they're like, ‘Oh, my god. (FG4)
Although some participants did acknowledge that providing the diagnosis of CKD is necessary for patient concordance with treatment, most chose to use terms such as ‘kidney strain’, as opposed to the words ‘chronic kidney disease’, and discuss it as a function of age rather than active disease. Since a CKD diagnosis was based on laboratory results, often the diagnosis was added to a patient's records when results were reviewed rather than in a face-to-face contact. It was often not practical with the business of the day and the number of results to inform all patients and to cross-check if there had been a proteinuria test and if it were positive. Both of these factors probably contribute to the observation that not all patients with CKD were informed of their diagnosis.
Meeting CKD targets
A key issue raised was the contradictory BP targets advised for patients with CKD in national guidance and the P4P scheme. The BP guidance set out by the Royal College of Physicians and Renal Association was different from that set within the P4P indicators for primary care, which were again different from that set by National Institute for Health and Care Excellence (NICE), the national body that develops evidence guidelines (table 1).22 ,23 While many practitioners chose to follow the P4P BP target, others altered the aggressiveness of their management dependent on the patient group:I tend to go for younger patients particularly. If they are 38 and they have got hypertension and we are treating them, the QOF [P4P] target is 150/90 and if we are getting 148/88, it's okay but they have got another thirty years of that. (FG1)
Guidance from NICE was felt to be too lengthy and confusion was frequently reported with the interpretation of the newly introduced proteinuria measure, albumin:creatinine ratio (ACR), both in its measurement and its effect on patient management:I think the confusion is what level of ACR you act on. I think it's slightly clearer with the NICE guidelines, basically if you're talking about ACRs of more than 30, 70 or 30 if they've got blood, and I think the little bit of uncertainty is in diabetics an ACR of more than 3.5 is classed as microalbuminuria but in non-diabetics what do you do with ACRs between 3 and 30? I think that's the slight area of uncertainty that I feel. (FG6)
The impact of the study interventions
The question of what impact study interventions (ABE and guidelines with prompts) had on clinical practice drew mixed responses. Two of the four practices felt guidelines were useful both to base their local policy on, and more commonly as a reference for clarification:It was shown very clear when to refer, when you’ve got proteinuria when to refer, when, so that not everyone with proteinuria had to be referred and so the guidelines I thought were very clear and good. (FG1)
Practices responding in this fashion had generally been proactive in seeking out and using local guidelines, and implementing new policy such as ACR measurement. They also commented on the usefulness of the educational meetings, particularly where their representative could discuss concerns with a consultant nephrologist and subsequently drive change within their group.
However, the other two practices felt that the written information provided, both in the form of audit data and prompts, was not especially useful:And there was a whole load of audit stuff that went in that I just looked through, but again it was quite complicated, involved I should say, involved, so I'm afraid we didn't take any notice of that either. (FG8)
A theme throughout all focus groups was the difficulty in interpreting the effect of study interventions given the P4P introduction in tandem, and the expected familiarity that develops with new policy implementation over time. While some noted that interventions provided more background and understanding of the evidence behind national guidelines (particularly soon after their initial introduction), P4P targets remain the driving force behind local practice:So I think that's universally accepted, that if you keep the QOF [P4P] up-to-date you'll get money, you'll get points, so I think you can't beat on that one. (FG5)
Ideas for future clinical practice
Ideas for future practice were largely clustered around tools to make the process of updating the CKD register or accessing guidelines more efficient, as well as to improve access to specialist advice. A number of practices identified the scope for guidelines with prompts to be incorporated into their current electronic patient management systems:I think one of the [ideas], is to develop some sort of software which then integrates the clinical system so when you see patients [with] CKD it actually pops up a window saying you haven’t done x, y, z. (FG1)
Others noted that workshops or ‘virtual clinics’ where practice members had direct access to specialists proved very useful:We had quite a useful exercise where we had, it wasn't actually for CKD but it could apply to CKD, we had a diabetic endocrinologist come and a diabetic specialist nurse and they sat down and they did a virtual clinic involving diabetics who had HbA1c under ten and they said what to do about them. (FG2)
Summary
Five main themes emerged from this study (table 2). First, the need to improve the accuracy and understanding of the CKD disease register; most importantly, many people were included who may not have had two qualifying eGFR measurements at least 3 months apart. Second, a diagnosis of CKD was hard for some professionals to share with patients, who they felt needed a lot of reassurance. Some clinicians avoided using the term. Third, practitioners were more driven to change practice by P4P indicators than they were by evidence-based guidance; they tended to use the former to drive their practice. Fourth, the study provided an opportunity to develop practice guidance and the presence of a specialist nephrology doctor or nurse was useful, whereas much of the study information was less so. Finally, participants could see how prompts, role play through virtual clinics, workshops and updates might be useful tools for keeping up to date.
Conceptual framework listing main themes and subthemes derived from the process evaluation
Discussion
Principal findings
The process evaluation practices in the QICKD trial were exposed to the intervention as planned; those exposed to ABE had the chance to test some of their uncertainties about CKD and its management. Reaction towards the study interventions and P4P varied both between and within individual practices. However, five major themes consistently emerged in analysis of focus group discussions. The introduction of a nationally defined CKD register was generally viewed as a positive step in formalising diagnostic criteria, reducing ‘false additions’ and facilitating audit. Practices were encouraging about the effects of study interventions, particularly ABE, in improving awareness and understanding of CKD, and would largely support similar interventions in future. The ABE practices found the interaction with a specialist advising about how to implement guidance particularly useful. However, P4P measures were regarded as the main driving force behind changes in practice.
Concerns persist about the difficulty in explaining the diagnosis of CKD to patients, the stratification of risk and the inconsistency in updating the CKD register. Additionally, many participants questioned the usefulness of conflicting BP guidelines from NICE and P4P, which added to what was perceived to be unnecessary complexity (table 1).
Practice implications
QI interventions such as ABE and G&P appear to have some positive influence on the primary care management of CKD. Practitioners reported that they found the interactive sessions where their data were presented and they could interact with a specialist nephrologist most useful. Receiving information on paper was remembered but perceived to be confusing. The costs of providing such sessions are obviously high but could be used where achieving change was a high priority and such sessions could be integrated into other locality-based educational interventions. In contrast, providing written information did not appear to have an impact, and maybe those looking to improve quality by mailing out to practices should think carefully about whether this is likely to be effective, especially where there is complexity.
However, over the period of this study, the major impetus for improving quality appeared to be exposure to P4P. Where QI interventions appear to be particularly useful is in providing further guidance, in conjunction with newly introduced P4P targets, and in clarifying any inconsistencies or confusion. QI interventions can support introduction of new guidelines by familiarising practices with new evidence and offering support and constructive feedback for implementation, which would not otherwise be available. Interaction with a specialist, as well as consulting about the level of care in the practice, as part of the ABE process, rather than about individual patients, was a different but useful experience.
Greater appreciation of the benefits of identifying people with CKD and a fuller understanding of appropriate CKD management beyond the mechanistic application of P4P indicators is needed. This evaluation revealed inconsistency in application and gaps in clinicians' understanding. These findings are also replicated in other studies.24 Concurrent introduction of QI interventions with future P4P targets and national guidance plus continuing education would avoid confusion over their use and could increase speed of uptake and achievement of targets. This study has shown that primary care practitioners are generally supportive of such measures and found them useful in guiding their own practice's adoption of guidelines. However, despite this, practitioners reported difficulty in explaining a diagnosis of CKD to their patients. There appears to be a gap in current guidance, namely a lack of explanation about how healthcare professionals could best communicate the nuances of a condition such as a CKD diagnosis to patients, a problem reported by other studies.25 ,26
Comparison with the literature
Much research has focused on the extent to which P4P affects clinical practice in primary care, much less about what constitutes effective QI, or what promotes effective management of CKD in primary care. There is evidence that P4P does indeed result in acceleration of target achievement, such as a progressive reduction in BP8 or improved diabetic control.27 However, this improvement appears to slow once targets are reached, and quality of care can decline for non-incentivised criteria.28 New P4P guidelines are not always immediately accepted and embraced by practices, and low practitioner confidence in management of CKD (particularly high-risk patients, such as those with proteinuria) provides a plausible explanation as to why they might have a negative effect on target attainment, even where P4P is in place.17 There may be a gradual, underlying diffusion of knowledge and management of CKD.29 There appeared a greater acceptance among healthcare professionals of CKD as a relevant diagnosis, compared with previous studies,23 but this may simply be due to the length of time that has now elapsed since the introduction of P4P (2006) and NICE guidance (2008, table 1).
It has been shown that QI strategies can be useful in improving specific targets in CKD, such as BP14 ,30 and requirement for renal replacement therapy.31
Limitations
This study examined a small number of practices; we purposively sampled two each from the two study intervention arms and from different parts of the country. The practices contained a larger number of individuals and it is possible that opinion leaders within those practices, or practices and localities within which they were based, may have adopted procedures and processes that were not representative of the study practices as a whole. Additionally, these individuals may have unduly dominated the focus group discussion, compared with more junior practice members. These limitations are all possible, but the study team which conducted these focus groups was also exposed to the other practices involved in the trial.
Additionally, QI interventions such as ABE or G&P can vary in design between studies, with no common detailed approach for their implementation. In systems such as the National Health Service (NHS), data are readily accessible, and creating personalised feedback for practices should be relatively straightforward and is already done in a number of areas such as prescribing. However, meeting time for practitioners is expensive, especially where in ABE a facilitator (to explain the data) and a specialist physician or nurse are brought along too. While these roles might be combined, ABE is an expensive interaction, an issue raised in a commentary on the QICKD trial.32
Further research
While there has undoubtedly been engagement with CKD through the P4P process, the case for P4P in CKD remains unproven.33 ,34 Future research should seek to determine whether improvements identified following QI interventions can be generalised. We need to have a greater understanding about why clinicians found this a difficult diagnosis to explain to patients. We also need to test whether computerised medical record systems could improve the way they display information; in the case of CKD, a combination of finding two renal function tests at least 3 months apart, whether there is a significant level of proteinuria, a relevant comorbidity, and if BP is correctly controlled for this level of risk are all recorded in different places (table 1). We also need to know whether ABE is cost-effective. It is only likely to be so if embedded into existing educational processes carried out on a locality basis.
Conclusions
P4P incentives were the major driver of improved management of CKD in primary care (box 2); however, QI interventions can have a complementary role. This process evaluation demonstrates the place for a QI intervention alongside the introduction of P4P guidelines. The focus groups reported positive interactions from ABE, particularly with a kidney doctor or nurse. It is plausible that the interaction involved in ABE contributed towards the modest but significantly greater reduction in systolic BP in this arm compared with sending out G&Ps.
Summary conclusions
Chronic kidney disease (CKD) is a difficult condition to explain to patients, risk stratification is complex, and the relevant data are dispersed through computerised medical record systems.
Pay-for-performance (P4P) incentives are the major driver for change in the management of CKD in primary care.
Quality improvement interventions may have a complementary role with future P4P initiatives.
Audit-based education was positively received by practices, but further research is needed to demonstrate whether it is cost-effective.
Acknowledgments
The QICKD trial was the result of a collaboration of participating practices and their patients; and investigators, in local and in a central lead group.
References
Footnotes
Contributors AN took the transcripts and an initial analysis of the findings and wrote the paper, under SdL's supervision. SdL conceived and designed the QICKD trial in collaboration with the other protocol authors, and supported and iterated the development of the paper with AN. NT is the project coordinator and organised the ABE intervention in the southern localities; she undertook the focus groups and reviewed and edited the paper. MAT and HG were involved in the review and editing of the paper.
Funding QICKD was funded by the Health Foundation and Edith Murphy Foundation. The National Institute for Health Research (NIHR) research portfolio scheme supported participant practices; the Joint Research Office at St George's, University of London and St George's Healthcare Trust for supported the application and sponsored the research. Kidney Research UK coordinated the trial. AN is supported by the London Deanery.
Competing interests SdL is the principal investigator of the QICKD Trial. SdL led the group that provided the clinical input into the development of the CKD pay-for-performance indicator. NT is the deputy on the NICE guideline development group for CKD (2008). MAT is the QICKD investigator and led the feedback to some practices. HG co-designed the QICKD study and is the senior investigator.
Ethics approval Oxford Research Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.