Objectives Tackling the harm associated with acute kidney injury (AKI) is a global priority. In England, a national computerised AKI algorithm is being introduced across the National Health Service (NHS) to drive this change. The study sought to maximise its clinical utility and minimise the potential for burden on clinicians and patients in primary care.
Design An appropriateness ratings evaluation using the RAND/UCLA Appropriateness Method.
Setting Clinical scenarios were developed to test the timeliness in (1) communication of AKI warning stage test results from clinical pathology services to primary care, and (2) primary care clinician response to an AKI warning stage test result.
Participants A 10-person panel was purposively sampled with representation from clinical biochemistry, acute and emergency medicine and general practice. General practitioners (GPs) represented typical practice in relation to rural and urban practice, out of hours care, GP commissioning and those interested in reducing the impact of medicalisation and ‘overdiagnosis’.
Results There was agreement that delivery of AKI warning stage test results through interruptive methods of communication (ie, telephone) from laboratories to primary care was the appropriate next step for patients with an AKI warning stage 3 test result. In the context of acute illness, waiting up to 72 hours to respond to an AKI warning stage test result was deemed an inappropriate action in 62 out of the 65 (94.5%) cases. There was agreement that a clinician response was required within 6 hours, or less, in 39 out of 40 (97.5%) clinical cases relating AKI warning stage test results in the presence of moderate hyperkalaemia.
Conclusions The study has informed national guidance to support a timely and calibrated response to AKI warning stage test results for adults in primary care. Further research is needed to support effective implementation, with a view to examine the effect on health outcomes and costs.
Statistics from Altmetric.com
Strengths and limitations of this study
The study used systematic RAND/UCLA consensus methods to optimise clinician response to acute kidney injury (AKI) warning stage test results for adults in primary care.
Efforts to maximise the utility of AKI alerts and minimise the potential for burden included engagement with general practitioners who represented concerns about medicalisation and overdiagnosis.
Although a range of clinically relevant scenarios were considered, in terms of burden on RAND/UCLA panellists, it was not feasible to test all major comorbidity groups (eg, patients with type 2 diabetes).
The study sought clarity on ‘necessary’ actions required under ideal conditions and further studies are needed to understand the implementation of AKI alerts and guidance in routine clinical practice.
Maintenance of kidney health is central to addressing two global health priorities: the prevention of cardiovascular disease and support for people vulnerable to complications during episodes of acute illness.1–4 The international 0by25 initiative (http://www.0by25.org/) aims to prevent avoidable harm associated with acute kidney injury (AKI).2 In a similar vein, the National Health Service (NHS) Five Year Forward View emphasises tackling AKI as a priority to improve patient safety and health outcomes across the NHS.5
AKI is a clinical syndrome characterised by a sudden reduction in kidney function.4 The diagnosis of AKI and its staging is based on acute changes in serum creatinine and/or a reduction in urine output.4 There are many causes for AKI though it is most commonly associated with episodes of acute illness on the background of increased risk.4 ,6–8 AKI is associated with frailty and is more common in older people living with complex comorbidities.4 ,6 ,8–12 These include people with pre-existing chronic kidney disease (CKD), diabetes and heart failure as well as those with underlying cognitive decline, who may be reliant on a carer for adequate fluid intake during episodes of acute illness.6
AKI is associated with ∼1 in 5 unplanned hospital admissions, with almost two-thirds of these patients with AKI having developed it in the community.13 It is associated with increased length of hospital admission and need for renal replacement therapy.4 AKI is also associated with poorer health outcomes including increased risk of progression of CKD and increased mortality both in the short and long terms.4 Care of people with AKI costs around 1% of the NHS budget, reflecting the fact that AKI is a strong marker for severity of acute illness.14
In a major NHS England patient safety initiative, a computerised AKI algorithm has been established, based on the Kidney Disease Improving Global Outcomes (KDIGO) classification system for AKI, to automatically detect biochemical changes in kidney function and identify potential cases of AKI.4 ,15 ,16 The test result is named an ‘AKI warning stage’ and is categorised according to severity (ie, AKI stage 1, 2 or 3).15 A level 3 NHS England Patient Safety Notice made it mandatory that all NHS Acute Trusts and Foundation Trusts implement the use of the algorithm within laboratory information systems, providing warning stage results to users in secondary care, by March 2015. This is combined with a mandatory submission of a data set to a national registry allowing audit of the frequency of alerts.16 The next phase of AKI alerting involves direct communication of AKI warning stage test results to primary care in England, from April 2016.15 ,16
National clinical guidelines for AKI recommend ‘identifying AKI in patients with acute illness’ who are deemed at risk of AKI.6 At present, while a significant proportion of episodes of illness complicated by AKI start in the community, AKI tends to be identified on hospital admission.17 As a potential marker of the ‘sick patient’, this new test could support earlier detection and prompt management of acute illness in primary care.15 ,18 However, trade-offs exist between their clinical utility and the potential for burden on clinical care.19 ,20
First, kidney function tests are taken in primary care for many reasons both as part of routine disease management monitoring as well as assessment of urgent care. Current data suggest that only a small proportion of all primary care serum creatinine requests generate an AKI alert.21 In primary care, a decision on whether or not to check kidney function needs to be tailored to the individual circumstance.6 Consideration needs to be given as to whether this will support clinical management. For example, blood tests are not necessary when immediate admission is required (eg, there is evidence of sepsis).22 ,23 At the other end of the spectrum, checking kidney function tests may also not be necessary for patients presenting with a minor self-limiting acute illness, such as a single episode of diarrhoea or vomiting.
Second, ‘false-positive’ test results may occur. For example, a change in serum creatinine may be due to progression of CKD rather than represent an acute deterioration in kidney function.15 ,19 ,20 Third, there is also need to address the risks associated with a test result being communicated from laboratories to out of hours services, requiring a clinician who may have limited access to clinical information to act on the result. Fourth, there is evidence to suggest that the introduction of ‘e-alerts’ in isolation are unlikely to improve health outcomes.24 ,25 In particular, ‘alert fatigue’ is a recognised problem limiting implementation and effectiveness.26 ,27 If implemented well, the release of AKI warning stage test results to primary care has the potential to generate major improvements in outcomes. However, experience of other ‘top-down’ initiatives intended to improve kidney outcomes in primary care has been mixed.28
In order to maximise their clinical utility and minimise burden on health professionals and patients, guidance is required to ensure a timely response and for test results to be considered with an understanding of the clinical context in which the blood test was taken.
The RAND/UCLA Appropriateness Method (RAM) was used to help develop guidance on the response to AKI warning stage test results in primary care.29 RAM is an established consensus method to generate appropriateness ratings.29 It is a systematic approach to address specific dilemmas in clinical practice in which clinical decisions are required but where ‘robust scientific evidence’ about the benefits is lacking.29–31 The methodology combines ‘best available scientific evidence with the collective judgement of experts’ in order to provide a statement regarding the appropriateness of undertaking a particular action ‘at the level of patient-specific symptoms, medical history and tests results’.29–31 As such, it is a rigorous methodology congruent with a need to place response to AKI warning stage test results in clinical context. Findings from the process are deemed to have face validity and provide a platform for prospective development and evaluation of guidance to support clinical decision-making.32
Developing clinical scenarios for appropriateness rating
Through NHS England's Think Kidneys Programme, a working group was established to develop clinical scenarios for appropriateness rating.33 In order to ensure scenarios were informed from a range of perspectives, the group comprised general practitioners (GPs), a nephrologist, a chemical pathologist with a GP background and a methodologist with expertise in the RAM as well as patient safety in primary care. The group identified national AKI clinical guidelines and invited hospital trusts, clinical commissioning groups and strategic clinical networks throughout the country to send guidance documents in use, or being developed, to support primary care in responding to AKI warning stage test results. These documents were reviewed to identify key elements of care relating to AKI, which through group discussion led to the generation of a list of clinical cases, with a focus on rating the timeliness in response to each case.
Clinical scenarios were categorised according to the following clinical characteristics: level of AKI warning stage test result (AKI warning stage 1, 2 and 3); medical history (episode of acute illness, history of chronic heart failure, CKD stage 4 or 5, or history of renal transplant, recent change in dose of a diuretic, ACE inhibitor or angiotensin receptor blocker); and evidence of complicating factors (poor fluid intake/urine output, hyperkalaemia, both mild (K 5.5–5.9 mmol/L) or moderate (K 6.0–6.4 mmol/L), evidence of risk of urinary tract obstruction or intrinsic renal disease). Scenarios were not tested for severe hyperkalaemia as UK Renal Association Clinical Practice Guidelines recommend emergency assessment and treatment of severe hyperkalaemia (K+≥6.5 mmol/L).34 In order to reduce research burden on panel members and keep the process salient, a decision was made to restrict the number of ratings for AKI warning stage 3 test results in which timeliness in management was deemed less controversial.4 Also, the clinical scenarios considered during the consensus process were not specific to either in hours or out of hours care. Other categories considered in scenarios presented were: age (adult, or children and young person); and whether a patient was receiving end of life care. These clinical characteristics were then used to develop 163 clinical cases, with four responses (next steps) requiring rating for each clinical case (ie, 163×4=652 clinical scenarios).
Each clinical case represented a separate testing indication consisting of variable combinations of the clinical characteristics. Clinical cases were independently linked to two main areas of decision-making:
Based on knowledge of an AKI warning stage test result and potassium level, clinical pathology service/laboratory staff need to make one of four decisions:
▸ Send an AKI warning stage result via the data transfer service without comment (non-interruptive communication).
▸ Send an AKI warning stage result via the data transfer service with comment (non-interruptive communication).
▸ Send an AKI warning stage result to an NHS email address that is known to be monitored regularly during working hours (non-interruptive communication).
▸ Send an AKI warning stage result by telephone call to GP/practice/out of hours service provider (interruptive communication).
Based on the knowledge of the presenting AKI warning stage test result, clinical history including complications, GPs (responsible primary care team, ie, either in hours or out of hours service provider) will need to make one of four decisions:
▸ Seek immediate admission.
▸ Respond to the AKI warning stage result within 6 hours.
▸ Respond to the AKI warning stage result within 24 hours.
▸ Respond to the AKI warning stage result within 72 hours.
An established rating scale was used to rate each potential decision for each clinical scenario on a nine-point integer scale, with 1 indicating an ‘extremely inappropriate next step’ and 9 indicating an ‘extremely appropriate next step’:29 ,31
Scores 1–3: inappropriate next step (ie, no benefit, possible harms).
Scores 4–6: uncertainty about next step (ie, when harms and benefits are judged as approximately equal, or when the best available evidence does not support a judgement either way).
Scores 7–9: Appropriate next step (ie, benefits judged to outweigh harms).
Panel membership and rating
Panel members were purposively sampled and recruited to ensure that a range of perspectives were considered and ensure that the appropriateness ratings were valid for routine clinical general practice. A 10-person panel was identified with representation from clinical biochemistry (n=2), acute and emergency medicine (n=2) and general practice (GP principals, n=5; salaried GP, n=1). Individuals with an expressed interest and expertise in AKI through membership of the NHS England Think Kidneys Programme were invited as well as GPs who represented typical practice in relation to out of hours care, rural and urban practice, GP commissioning and those with an interest in reducing medicalisation and ‘overdiagnosis’.33 ,35 The panel was co-chaired by one researcher (SC), who is expert in the RAM, and a nephrologist (CT) who is expert in AKI, neither of whom contributed to the RAND/UCLA decision-making scores.
The RAND process entailed two rounds of rating:
Round 1 was conducted by email with the panel members rating the clinical scenarios individually at home, with no discussion between panellists. In order to help rate the appropriateness of each decision for every scenario, panel members were sent an ‘instructions’ document and a ‘context’ document that provided an overview of the literature with a focus on the rationale for the clinical characteristics chosen. The evaluation aimed to gain clarity on ‘necessary’ processes of care under ideal conditions. When scoring each scenario, panellists were instructed to assume that they had available all the clinical information presented.
Supplementary Instructions for panel members
Supplementary Context Document
Supplementary RAND example scenarios
Round 2 entailed a 1-day face-to-face meeting held in September 2015. Panel members were presented with their own data from round 1 as well as data outlining the distribution of all panel members' ratings. This summary was used to stimulate discussion with a focus on areas of disagreement. Panel members then rated each scenario on their own individual-blinded rating sheets. Panel members were not required to reach consensus.
Data entry and analysis
RAMs were used to conduct analysis.29 Data from round 2 were collated and analysed to determine the level of agreement within the panel for a proposed action for each clinical scenario. Agreement was defined by 8 out the 10 (80%) panel members rating the same three-point region on the nine-point integer scale (ie, 1–3, 4–6, 7–9). Disagreement was defined to exist where ≥30% of panel members rated a scenario in the 1–3 range and where ≥30% rated the same scenario in the 7–9 range on the nine-point integer scale. A proposed action was then categorised as an ‘appropriate’ next step if a scenario rated 7–9 without disagreement and a rating of 1–3 without disagreement was deemed to be an ‘inappropriate’ next step. Ratings of clinical scenarios without consensus (either ‘agreement’ or ‘disagreement’) were considered as equivocal.
The results are presented in tables 1⇓⇓–4 for round 2 ratings. Table 1 summarises the ratings across the 652 clinical scenarios, with 415 (63.7%) achieving agreement; 32 (4.9%) rated with disagreement and 205 (31.4%) with an equivocal rating. A proposed action was deemed the appropriate next step in 137 (21.0%) out of the 652 scenarios. Two hundred and ten (32.2%) of the proposed actions were deemed inappropriate, and in 68 (10.4%) of the proposed actions for a scenario, there was agreement of uncertain benefit.
The study was primarily undertaken to support the development of new guidance on the response to AKI warning stage test results for adults in primary care. Tables 2⇑–4 and the Results section focus on presenting data for adults. Supplementary tables present data for a child or young person as well as for adults with chronic heart failure on an end of life care pathway.
Communication of an AKI warning stage test result to primary care
Table 2 presents the data for the timeliness in communication of AKI warning stage test results for adults by clinical pathology services to primary care. There was agreement that delivery of AKI warning stage test results through interruptive methods of communication (ie, telephone result) from laboratories to primary care was the appropriate next step for patients with: an AKI warning stage 3 test result irrespective of potassium level; an AKI warning stage 2 test result in the presence of mild (K+ 5.5–5.9 mmol/L) or moderate hyperkalaemia (K+ 6.0–6.4 mmol/L); and an AKI warning stage 1 test result in the presence of moderate hyperkalaemia (K+ 6.0–6.4 mmol/L).
Timeliness in clinician response to an AKI warning stage test result
Data on the timeliness in clinician response to AKI warning stage test results for adults in primary care are presented in table 3 (in the context of acute illness) and table 4 (no history of acute illness). Agreement was reached on an appropriate next step for the majority of clinical cases presented to panel members: 48 out of the 65 (73.8%) clinical cases relating to AKI test results in the context of acute illness and 58 out of 72 (80.6%) clinical cases for AKI test results with no history of acute illness.
In the context of acute illness, there was a greater level of uncertainty on the appropriate action for clinical cases relating to AKI warning stage 1 test results, particularly in the context of patients with chronic heart failure. In the absence of acute illness, there was a greater level of uncertainty on the appropriate next step for clinical cases relating to AKI warning stage 2 test results.
In the context of acute illness, waiting up to 72 hours to respond to an AKI warning stage test result was deemed an inappropriate action in 62 out of 65 (94.5%) clinical cases tested. In the absence of acute illness, waiting up to 72 hours was deemed inappropriate in 56 out of the 72 (77.8%) clinical cases, with uncertainty for this timeframe tending to relate to AKI warning stage 1 test results.
There was agreement that a clinician response was required within 6 hours, or less, in 39 out of 40 (97.5%) clinical cases relating AKI warning stage test results in the presence of moderate hyperkalaemia (K+ 6.0–6.4 mmol/L). These findings included agreement to seek immediate admission if moderate hyperkalaemia was present in the context of AKI warning stage 3 results, AKI warning stage 2 results in the context of acute illness and AKI warning stage 1 results in the context of acute illness associated with poor oral intake/urine output.
There was uncertainty on the appropriate response to AKI warning stage test results generated for patients with chronic heart failure on an end of life care pathway, though waiting up to 72 hours to respond to these test results was deemed an inappropriate next step.
The introduction of warning stage test results based on a national AKI algorithm is a major patient safety initiative across the NHS in England.15 However, its implementation in primary care is at a period in time where clinical workload is deemed to be reaching saturation point.36 The study is an important starting point in terms of navigating the challenges of addressing the harm and costs associated with AKI while avoiding the creation of additional workload for professionals and unnecessary burden on patients.37 Through use of rigorous consensus methods, the findings highlight clinical scenarios where interruptive methods of communication (ie, telephoning a test result) by clinical pathology services to primary care are considered to be appropriate. There was also agreement in the majority of clinical cases in terms of determining the timeliness in response to an AKI warning stage test result by a clinician in primary care. In the main, waiting up to 72 hours was deemed an inappropriate response. The findings from the study have informed the development and recent publication of NHS England Think Kidneys guidance (https://http://www.thinkkidneys.nhs.uk/aki/resources/primary-care/).33
Strengths and weaknesses of this study
The study used systematic RAND/UCLA consensus methods to optimise the development of guidance for clinicians on how to respond to AKI warning stage test results for adults in primary care.29 Investment in ‘overcoming questions about the legitimacy of new guidelines’ is important to ensure successful implementation.38 A key feature of the study was to maximise the utility of AKI alerts and minimise the potential for burden on patients and health professionals. Efforts to achieve this included engaging primary care clinicians who represented concerns about medicalisation and overdiagnosis.35
Multiple scenarios were tested through use of RAMs.29 Key features of these scenarios were identified through identification and comparison of existing relevant guidance documents. Cross-referencing and integrating different guidelines may address challenges of clinical decision-making in primary care, particularly for patients with multimorbidity.39 For example, rather than treating results in isolation, the methodological approach enabled AKI warning stage test results to be considered in the context of hyperkalaemia. The results offer a platform to inform alignment in guidance developed by NHS England Think Kidneys Programme, the Royal College of Pathologists and the UK Renal Association.33 ,34
AKI is a clinical syndrome and AKI warning stage test results need to be considered with an understanding of the clinical context in which they were taken.4 The UCLA/RAND consensus process aimed to provide face validity regarding the timeliness in response to AKI warning stage test results by clinicians based on the assumption that all the relevant clinical information was available. There is evidence of the predictive validity of quality indicators developed through the RAM.32 ,40 In terms of developing ‘timeliness in response’ to AKI test results as an indicator of patient safety in primary care, future research needs to provide evidence of acceptability, feasibility, reliability, sensitivity to change and predictive validity.32
Although a range of clinically relevant scenarios were considered, in terms of burden on RAND/UCLA panellists, it was not feasible to test all major comorbidity groups (eg, patients with type 2 diabetes, patients receiving cancer treatment and not on an end of life care pathway). In addition, developing scenarios for working hours and out of hours may have produced greater clarity on timeliness in response though would have also lengthened the ratings process for panel members with the potential for ‘cognitive strain’.41 On balance, the chosen response times for rating (ie, seek immediate admission; respond within 6 hours; respond within 24 hours; respond within 72 hours) sought to evaluate appropriate actions taking into account key time frames in current clinical practice (eg, respond within 6 hours in order to determine whether appropriate to be seen within working hours, or before next day if knowledge of test result during out of hours; respond within 72 hours to determine if appropriate to wait until Monday if knowledge of a test result on Friday afternoon/evening).
Though the study sought clarity on ‘necessary’ next steps conducted under ideal conditions, it is hoped that the subsequent published guidance will seek to drive system change to better support clinician decision-making, both for in and out of hours care. Developing robust systems for managing laboratory results is deemed key to ensuring patient safety in general practice.42
Comparison with other studies
The introduction of AKI as a new classification system has both the potential to structure and constrain the delivery of acute illness care.4 ,43 Although AKI alerts are a potential driver for improving patient safety, AKI is a clinical diagnosis and diagnostic accuracy is dependent on the clinician interpreting the test result in the context of an individual patient. This includes recognising the potential for ‘false positives’ such as cases of misclassified CKD.15 ,19 ,20 An assessment of the sensitivity of the NHS England AKI algorithm has been carried out in secondary care but so far, there remain little data on either its specificity or its real-time use in primary care.19 A retrospective review of electronic medical records would help estimate the diagnostic AKI error rate as well as determine the incidence and factors associated with missed diagnostic and management opportunities.44
In the majority of UCLA/RAND clinical cases tested through a multidisciplinary professional panel, a wait of up to 72 hours to respond to an AKI warning stage test result was considered to be inappropriate. This is a potentially important marker of patient safety and requires further evaluation. Caution is required interpreting an AKI warning stage test result in primary care as AKI diagnostic and staging criteria are based on changes in serum creatinine by >26 µmol/L within the previous 48 hours, or an increase in serum creatinine by ≥1.5 times baseline that is known or presumed to have occurred within the prior 7 days.4 ,15 Currently, unlike the hospital setting, ‘baseline’ creatinine levels for comparison are less likely to have taken place within these time periods, reiterating the importance that AKI is a clinical (not merely a biochemical) diagnosis that requires clinician review in order to place a test result in context.
Though there is evidence that AKI is associated with adverse clinical outcomes, the majority of the studies informing the development of the KDIGO classification system for AKI were based on critical care databases.45 As such, its application may not necessarily be generalisable to the primary care setting.45 However, irrespective of whether an AKI warning stage test result leads to confirmation of AKI or alternatively flags CKD progression, there is evidence to indicate that the e-alerts are still likely to ‘identify patients at risk of poor outcomes’ in terms of non-recovery, chronic renal replacement therapy and long-term mortality.17 ,46
Implications for clinicians, policymakers and future research
There was greater uncertainty in how to respond to AKI warning stage test results for people with chronic heart failure, a population who have an increased morbidity and mortality and who account for ∼5% of emergency hospital admissions.47 This highlights a need for better evidence to support response to changes in serum creatinine for these patients.48 ,49 As indicated by the European Society of Cardiology Heart Failure (ESC) guidelines (2016), increases in serum creatinine are not always the main clinical priority, ‘especially when they are accompanied by appropriate decongestion, diuresis and haemoconcentration’49—although this statement was in the context of acute heart failure in hospital; data on the prognostic importance of changes in kidney function during treatment of heart failure in primary care are lacking. The ESC guidelines recommend that ‘when large increases in serum creatinine occur, care should be taken to evaluate the patient thoroughly and should include assessment of a possible renal artery stenosis, excessive hypervolaemia or hypovolaemia, concomitant medication and hyperkalaemia, which frequently coincides with worsening renal function’.49 This again highlights the importance of ‘treating the patient and not the test result’ and of comparing the test result with current and previous creatinine levels.33 ,48 Knowing the reason for why kidney function tests were taken is of primary importance. Were they taken in the context of: routine chronic disease monitoring, pharmacotherapy monitoring (eg, dose adjustment) or to support decision-making for patients presenting with a history of acute illness?33
Effective minimally disruptive medicine is a key principle underpinning care delivery.37 Resourcing initiatives to address the growth in demand for urgent care services is a priority, particularly with an ageing population with increasing care needs.50 ,51 Rather than being an additional burden, targeting AKI may provide an approach to better assessment and management of acute illness across the primary/secondary care interface.52 As stated by Guthrie et al,39 people with complex comorbidities ‘have patterns of illness that do not neatly fit disease categories, in which case syndrome focused guidance or tools will often be relevant’.39 Aligned with the implementation of the national AKI algorithm and professional guidance, resourcing patient and public involvement is critical to the development of a more responsive urgent care service.50 ,51 However, while dialogue about the importance of kidney health may provide a platform to support self-management, this currently remains limited.33 ,53–56 Evidence exists that only one in two people are aware that the kidneys make urine and that only 12% of the population have knowledge that the kidneys have a role in processing medicines.57
Tackling the harm associated with care complicated by AKI is a global priority.2 In England, a national AKI algorithm is being introduced across the to drive this change.15 However, in a context where clinical workload in English general practice is reaching saturation point,36 there is a need to maximise its clinical utility and minimise the potential for burden on clinicians and patients.37 The study sought to address this challenge and the findings have informed the development of guidance to support a timely and calibrated response to AKI warning stage test results for adults in primary care. Further research is needed to support effective implementation, with a view to examine the effect on health outcomes and costs.
The following colleagues are from the NHS England Think Kidneys Programme: Richard Fluck, Ron Cullen, Mike Jones, Karen Thomas, Julie Slevin, Annie Taylor, Teresa Wallace and Melanie Dillon. TB is funded by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Greater Manchester. The work outlined in this document may be considered to be affiliated to the work of the NIHR CLAHRC Greater Manchester. DL is funded by the NIHR Oxford Biomedical Research Centre.
Twitter Follow Tom Blakeman at @TomBlakeman
Contributors TB, CT and SC designed the study. TB, KG, DL, BL, SC and CT developed the clinical scenarios for appropriateness rating and organised panel membership for round 1 and 2 ratings. SC, CT and TB conducted the round 2 meeting. TB, JYT and CT conducted the analysis and SC provided methodological expertise. All authors contributed to the interpretation of the results, commented on draft manuscripts and have given their approval for publication.
Funding This work was supported by Think Kidneys. Think Kidneys is a national programme led by NHS England in partnership with UK Renal Registry.
Disclaimer The views expressed in this document are those of the author(s) and not necessarily those of the NHS, NIHR or the Department of Health.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.