Article Text
PDF
Abstract
Background Many studies identify factors that contribute to renal prescribing errors, but few examine how healthcare professionals (HCPs) detect and recover from an error or potential patient safety concern. Knowledge of this information could inform advanced error detection systems and decision support tools that help prevent prescribing errors.
Objective To examine the cognitive strategies that HCPs used to recognise and manage medication-related problems for patients with renal insufficiency.
Design HCPs submitted documentation about medication-related incidents. We then conducted cognitive task analysis interviews. Qualitative data were analysed inductively.
Setting Inpatient and outpatient facilities at a major US Veterans Affairs Medical Centre.
Participants Physicians, nurses and pharmacists who took action to prevent or resolve a renal-drug problem in patients with renal insufficiency.
Outcomes Emergent themes from interviews, as related to recognition of renal-drug problems and decision-making processes.
Results We interviewed 20 HCPs. Results yielded a descriptive model of the decision-making process, comprised of three main stages: detect, gather information and act. These stages often followed a cyclical path due largely to the gradual decline of patients’ renal function. Most HCPs relied on being vigilant to detect patients’ renal-drug problems rather than relying on systems to detect unanticipated cues. At each stage, HCPs relied on different cognitive cues depending on medication type: for renally eliminated medications, HCPs focused on gathering renal dosing guidelines, while for nephrotoxic medications, HCPs investigated the need for particular medication therapy, and if warranted, safer alternatives.
Conclusions Our model is useful for trainees so they can gain familiarity with managing renal-drug problems. Based on findings, improvements are warranted for three aspects of healthcare systems: (1) supporting the cyclical nature of renal-drug problem management via longitudinal tracking mechanisms, (2) providing tools to alleviate HCPs’ heavy reliance on vigilance and (3) supporting HCPs’ different decision-making needs for renally eliminated versus nephrotoxic medications.
- nephrology
- medical education and training
- health and safety
- qualitative research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Cognitive task analysis (CTA) was utilised to uncover healthcare professionals’ internal decision-making process and cues they used to recognise and manage renal-drug problems.
Multiple methods were used to aid participants’ recall accuracy about the incidents, using an adapted critical decision method in which participants were (a) asked to document important details as soon as they identified a renal-drug problem and (b) encouraged to access patient’s electronic health record during CTA interviews to aid recall of the incident.
Safety incidents were selected and rigorously analysed by an interdisciplinary team that included physicians, two human factors experts, a pharmacist researcher and a clinical pharmacist.
Healthcare professionals voluntarily chose which incidents to submit to the research team for consideration, and may have submitted incidents in which they felt most confident about their actions. This could have influenced the type of incidents we studied.
We intended to compare cognitive cues used by prescribers versus non-prescribers; however, due to the wide variation in incidents, we could not directly compare them and no notable differences were found.
Introduction
Many research studies have adequately assessed prevalence and factors contributing to medication errors,1–4 but very few examined how healthcare professionals detect and manage safety concerns.5 Such knowledge would inform more effective error detection systems and decision support tools, which can aid error prevention.5 Renal-drug problems in particular would benefit from such knowledge due to their known risks for patient harm. Medications that are renally eliminated or nephrotoxic require dose adjustments and substitutions to prevent drug accumulation, renal injury and adverse effects for patients with renal insufficiency.6 Nonetheless, rates of inappropriate prescribing range from 9.4% to 81.1% in this patient population across various healthcare settings.1 2 7 8 The clinical implications of such errors are serious. For example, 10% of patients with renal insufficiency experienced an adverse drug event (ADE).9 Of these ADEs, 91% were considered preventable and 51% were serious.9 Despite the availability of renal function estimators in electronic health records (EHRs), patients’ renal impairment is often overlooked when making medication-related decisions.3 4 10–13 Lack of knowledge about the need to adjust a medication’s dosage is a common cause of prescribing errors in patients with renal insufficiency.14–16 Why these errors persist, and what can be done to prevent them, are important questions. Part of the answer relies on understanding how healthcare professionals (HCPs) recognise and address safety situations that involve renal-drug problems.
The cognitive strategies that HCPs use in decision-making are ill-defined due to their implicit nature. Experts in the medical research field have shown that if such strategies are made explicit, errors might be dramatically reduced.5 17 Human factors experts have also emphasised the importance of identifying cognitive cues to be able to design systems that account for human limitations, enhance human performance and improve safety.18 One underused approach to understanding HCPs’ decision-making strategies is cognitive task analysis (CTA).5 17 19 20 The objective of this research was to examine the cognitive strategies that HCPs use to identify and manage renal-drug problems among patients with renal insufficiency. This research is part of a larger study that examined decision-making for two other types of medication incidents: adverse drug reactions and drug-drug interactions. Previously, we published a methods paper that describes how we adapted CTA for this research.19 The present article presents results for HCPs’ decision-making regarding renal-drug problems.
Methods
Setting
This study was conducted at a major US Veterans Affairs (VA) Medical Centre. The VA’s computerised physician order entry (CPOE) system provides an alert about patients’ renal impairment and requires HCPs to click ‘OK’ to acknowledge the content of the alert, but, except for metformin, does not offer real-time renal-drug alerts for specific medication orders. When patient laboratory results are available in VA’s EHR, serum creatinine and estimated glomerular filtration rate (eGFR) are displayed automatically on a ‘labs tab’ and the latter is based on standardised body surface area with units of ml/min/1.73 m2. The eGFR reported to prescribers is based on the modification of diet in renal disease equation and is adjusted for isotope dilution mass spectrometry traceable assays.21
Study design
We used CTA methods, which in other industries, are commonly used to identify strategies and cognitive information cues that people use to solve problems.22 23 We used the CTA approach to uncover HCPs’ internalised thinking, renal-drug problem recognition and clinical management processes that would typically be implicit and otherwise unarticulated.24–26 As described elsewhere, we adapted and used a specific type of CTA - the critical decision method -19 a specialised interview technique that is commonly used in CTA studies.27 28 We adapted this method by (1) asking HCPs to record key details on a standardised ‘incident card’ as soon as they became aware of a renal-drug problem and (2) encouraging HCPs to access EHR during CTA interview, to aid their recall of the incident.
Participant recruitment and sampling
All inpatient and outpatient practising physicians, nurse practitioners and pharmacists involved in the clinical care of patients were invited to participate via emails, flyers and follow-up phone calls. A research team member, who had no clinical or managerial roles, was responsible for recruitment. We did not seek to study nor quantify the incidence of renal-drug problems, and instead focused on critical incidents where HCPs detected a safety concern and took action to manage the renal-drug problem. Thus, consistent with the critical decision method technique for CTA,19 we purposely sampled from HCPs who encountered and acted on a renal-drug safety problem. Our goal sample size was 20 HCPs, which is considered a relatively large sample in qualitative studies.29 A study by Fusch and Ness found that qualitative data saturation is often reached within 12 participants,29 but we intentionally sampled more because we expected a variety of renal-drug incidents.19 We stratified recruitment to include the same number of prescribers (physicians and nurse practitioners) and pharmacists.
Participants completed a voluntary, written consent prior to any study data collection: consent occurred before participants submitted any incidents.
Data collection
Capture of renal-drug problems
Participants completed incident cards19 when they encountered a renal-drug issue that warranted their attention. This card helped collect key details to aid their recall during the interview. For example, the card captured how the participant first became aware of the problem, what types of electronic resources or clinical consultations were used to help with decision-making and what actions were taken to mitigate safety risks. HCPs submitted incident cards via printed copy or secure email.
Problem selection
A physician, pharmacist and human factors expert on the research team reviewed each incident card to determine eligibility for an interview with the HCP. The team selected incidents in which the issue required great expertise, could potentially cause patient harm or may be more challenging for trainees to resolve. Further details about interview eligibility and case selection is provided in the methods article by Russ et al.19
Interview process
Interviews were conducted within 2 to 4 weeks after incidence occurrence.19 We sought to aid HCPs’ recall of the incident by having participants: (1) document important details in writing as soon as they encountered a renal-drug incident, then refer to that document during the interview and (2) access the patient’s EHR during interviews. Semi-structured interviews were scheduled for 60 min. Critical decision method interviews were conducted in four phases: (1) collect a brief summary of the incident, (2) construct a broad timeline of events, (3) ask about details that influenced the participant’s decision-making process and (4) ask hypothetical (‘what if’) questions to gain further insights regarding the significance of cognitive cues and strategies.28 The interview guide is available elsewhere.19 Interviews were audio recorded and transcribed. Example questions were:
When did you first recognise a problem?
What information did you use to assess renal function?
What factors influenced your choice of (medication name) as an alternative medication?
Under what circumstances, if any, would you have discontinued (medication name) for this patient, rather than reducing the dose?
Data analysis
Goals of our analysis were to identify information cues that HCPs used to recognise a potential problem related to renally eliminated or nephrotoxic medications and select actions to help prevent or address the problem. For each interview, we used an inductive, qualitative approach in two stages.30 First, a human factors expert and pharmacist independently reviewed each transcript to create a decision-requirements table for each incident.28 Decision requirements tables are often used to analyse CTA interviews.27 28 Analysts met and discussed each table for each incident until reaching consensus. (The table template is found in the online supplementary table A but to maintain participant confidentiality, individual, completed tables are not shown). Second, a team consisting of two human factors experts, a clinical pharmacist and a pharmacist researcher identified initial emergent themes, and discussed new themes until reaching consensus. Individuals coded data independently and any disagreements were resolved through consensus.31–33 As in prior studies,34 35 20% of interview transcripts were analysed by all four individuals; the remaining transcripts were analysed by a human factors expert and a pharmacist on the team. Data were double-coded if they corresponded to two distinct themes.36 37 No new codes were identified after analysing 13 cases, which indicates adequate data saturation. For this stage, we used an inductive coding approach,33 along with NVivo software (QSR International, V.11), to facilitate data management and analysis.
Supplementary file 1
Patient and public involvement
This study was conducted to assess renal-medication problems, specifically, to understand the decision-making processes that healthcare professionals use to detect and respond to these patient safety concerns. These types of medication problems are mostly preventable and often serious, which makes finding solutions a priority to improve patient care. Patients were not directly involved in this study nor were they recruited; instead, our participants were HCPs who addressed a renal-drug problem for one of their patients. Our results can inform future enhancements of healthcare systems to provide safer care for patients.
Results
Participants and incidents
Twenty HCPs participated in 21 interviews (table 1). Interviews captured renal-drug problems related to 24 distinct medications as some incidents involved two drugs.
Characteristics of study participants (n=20)
Model of decision-making processes
CTA analyses informed the development of a descriptive model with three overarching stages for HCPs’ decision-making: detect a renal-drug problem, gather information about the problem, then act to resolve the problem (figure 1). Three important aspects of this model were evident. First, these stages often followed a cyclical path. The cyclical phenomenon was evident due to recurrence of renal-drug problems even after HCPs took action to ‘resolve’ a previous problem. The longer the duration of use of a renally eliminated medication in the setting of renal insufficiency, the greater the likelihood that multiple dosage adjustments or substitutions will be necessary to ensure patient safety. For example, a pharmacist reduced a gabapentin dose to account for a patient’s renal insufficiency, but described planning to: monitor adverse effects due to drug accumulation and check the patient’s renal function periodically. At times, gaps in organisation and continuity of clinical care appeared to contribute to the cycles of detection of incidents, gathering of information and acting on findings. For example, a levofloxacin dose was adjusted due to a patient’s renal insufficiency, but in an emergency-department encounter a few days later, the patient was ordered a higher, renally inappropriate dose of levofloxacin. Second, the cognitive needs of HCPs for each stage differed slightly depending on whether the medication was renally eliminated or nephrotoxic (figure 2). Third, a cognitive discordance was evident in our sample where HCPs described greater trust in the safety of a medication if the patient had used it for a long duration. This phenomenon was exemplified by one of the physicians who stated that he/she ‘didn’t think HCTZ/lisinopril was the cause (of renal insufficiency) at first because…patient had been on it for a long time, (at the) same dose for about 4 years’. Within our sample, we examined findings from prescribers versus non-prescribers, but did not find evidence that they relied on substantially different cognitive strategies to detect and manage renal-drug problems; instead, interview data indicated that their strategies were similar.
Descriptive model of HCPs’ decision-making process for renal-drug problems. This depiction is derived from 21 renal-drug incidents. This model illustrates the cyclical nature of stages that HCPs used to manage patients with renal insufficiency. The larger circle of the ‘detect’ stage reflects its important contribution to one of the three main study findings (ie, HCPs reliance on their own vigilance).
Description of how the three stages of healthcare professionals’ renal-drug decision-making differed depending on whether the medication was renally eliminated or nephrotoxic. 1From 15 incidents. 2From six incidents. 3All 20 HCPs used more than one cue to detect each renal-drug problem. HCPs, healthcare professionals; HCTZ, hydrochlorothiazide.
Stage 1: detect
HCPs’ vigilance was an important strategy for detection, where HCPs were ‘on the lookout’ for potential renal insufficiency and problematical medications. Specifically, they conduct ongoing and deliberate review of patients’ information, rather than relying on systems to saliently highlight unanticipated cues and bring renal-drug concerns to their attention. Thirteen (65%) of the 20 HCPs detected a problem via their own vigilance and monitoring. As one pharmacist stated, ‘there’s just certain medications that [I] always know to look for, so that’s just on the radar list’. (See online supplementary table B for additional quotes). Their vigilance strategy often involved reviews of medication and laboratory tests.
Depending on the drug type, the cognitive cues for detection differed (figure 2). HCPs used more cues to detect renally eliminated drugs compared with cues to detect nephrotoxic drugs (figure 2). For renally eliminated drugs, a frequent cue that helped HCPs detect an inappropriate medication was recognising a high medication dose. A pharmacist stated, “(the) patient (was) on 300 mg three times daily gabapentin, and my reaction was, ‘I’m sure that that’s probably too much for him’”. Additional cues included risk factors known to worsen renal function, such as older age, signs of dehydration, recent surgery or dialysis treatment. With one exception, all nephrotoxic drugs were detected by noticing an abnormal laboratory test result. In particular, extremely high serum creatinine (SCr), a SCr value higher than the patient’s baseline, or an increase in SCr over a short period, prompted HCPs to look for the cause of patients’ decline in renal function.
Stage 2: gather information
In almost all incidents (20/21, 95%), HCPs gathered additional information about patients’ medications (figure 2 and online supplementary table C) using their clinical experience and knowledge, reviewing drug information references, consulting a pharmacist, reading EHR notes or discussing directly with the patient. An important challenge that HCPs encountered while gathering information was the lack of consistency in drug resources. For example, a pharmacist participant found two different recommendations for dosing colchicine in dialysis patients, one reference recommended 0.3 mg twice a week and the other recommended 0.6 mg and not repeating for 2 weeks. HCPs sometimes checked to confirm the indication for a renally problematical drug. For example, for antibiotics, they reviewed microbial cultures, and for gabapentin, they looked for reasons for the original prescription. The information the HCPs gathered differed depending on the type of drug. For renally eliminated medications, HCPs sometimes investigated whether the patient was experiencing a side effect as a result of drug accumulation; for example, ‘patients can get rhabdomyolysis if [they] overdose on fenofibrate, which is serious’, and ‘both nitrofurantoin (in presence of renal insufficiency) and UTI [urinary tract infection] can independently cause an older patient to be disoriented’. HCPs also gathered information about renal clearance thresholds and renal doses to assess whether the patient’s medication dose needed adjustment. On the other hand, for nephrotoxic drugs, HCPs tended to gather information about the need for continued therapy and about safer treatment alternatives, as necessary.
In 95% (20/21) of incidents, HCPs also gathered information about patients’ renal function: they usually reviewed SCr or renal function estimates (eg, eGFR, estimated creatinine clearance (eCrCl)) provided in the EHR (n=16 incidents), or they calculated estimated renal function (n=4 incidents). Looking up patients’ renal function was especially important to HCPs if patients were older, on dialysis, or taking a renally eliminated medication that might require a dose adjustment. HCPs also checked for particular patterns in SCr changes or trends over the last three to four laboratory tests (eg, ‘almost doubled’, ‘big change’, ‘going up’). Several pharmacists (n=3) expressed that CrCL estimates were most useful, and perceived them as a ‘standard’ for deciding if medication dose adjustment is needed and for selecting appropriate doses. As one pharmacist explained, ‘[I] calculate CrCL because when you look up drug information like in Micromedex or like Lexicomp for the package insert, when they (provide guidelines) about renal adjustment, it usually goes off a creatinine clearance, not so much a serum Creatinine, or eGFR’. Thus, HCPs always took the time to calculate CrCL estimates if the patient was on medication that required renal adjustment. HCPs explained that estimating renal function can get ‘tricky’ in older patients, underweight patients and in amputees.
Stage 3: act
HCPs’ medication-related actions to resolve renal-drug problems included reducing the medication dose or frequency, discontinuing the medication or substituting the medication with an alternative (table 2). Other actions, that did not involve medication changes, that HCPs took were to continue the medication, follow-up on the patient’s case (eg, monitor renal function, co-morbid diseases) or document communications with other HCPs. (See online supplementary table C for additional quotes). HCPs’ descriptions of the factors that led to these actions are described in table 3. Actions differed depending on the drug type (figure 2).
Medication-related actions that HCPs made to help address a renal-drug problem (n=21 incidents)
Summary of actions and decisions taken by HCPs and the factors associated with these decisions
Discussion
Our results identified three promising areas for system enhancements. First, the cyclical nature of the decision-making process (figure 1) highlights the importance of tools to track and detect renal-drug problems for patients longitudinally. This cyclical phenomenon found in our study was, in part, due to the progressive nature of patients’ renal dysfunction, which often continues to decline, with medications requiring adjustment over time. HCPs' actions, such as dose adjustments, can be falsely reassuring, because they may not lead to sustained, safe medical treatment for this patient population. Our study elucidated this safety cognitive discordance, and we did not find this articulated by other literature, suggesting this finding is a novel contribution of our research. We also did not find evidence in the literature of prescribing-related interventions (technology based or otherwise) that systematically accounts for the cyclical nature of renal-drug problems. Our results indicate that efforts to support the cyclical aspect of renal-drug decision-making are likely to improve patient safety. Although we examined a specific problem, renal-drug safety applies to a large patient population. More than 15% of all adults in the USA (30 million people) have chronic kidney disease.38 Additionally, we expect that our findings can inform other chronic diseases (eg, diabetes, cardiovascular diseases, chronic obstructive pulmonary disease) that do not necessarily have a cyclical nature but often worsen with patients’ age and require regular monitoring and medication adjustments over time. Our findings may also inform more effective software solutions to aid attention, memory and perception, and thus provide cognitive support to help prevent prescription errors more generally.
Second, HCPs often relied on their own vigilance to detect a renal-drug problem. However, vigilance is recognised by the scientific community as a fallible, tedious and difficult strategy for individuals to effectively maintain to monitor and detect patient safety related problems.39 Moreover, vigilance is especially inadequate if cues lack sufficient salience.39 HCPs divert their mental energy to find renal-drug problems, while their cognitive capacity is needed for other complex clinical tasks. The detection stage of renal-drug decision-making is crucial (figure 1), because there is no way to prevent or mitigate a renal-drug problem if the issue remains unnoticed. Thus, there is a need for more robust, system-level solutions to aid HCP’s detection of these problems. Much research in the literature has focused on developing more accurate measures of renal function,40–42 but our findings, along with reports of high rates for inappropriate prescribing,1 7 9 indicate that much more effort is needed to develop systematic interventions to help HCPs detect renal-drug problems when it is already established that the patient has renal insufficiency. This could include improved EHR visualisations and information representation, which could shift focus from standard numerical values to depicting graphical trends and/or the magnitude of change over time for the patients’ renal function.43 In addition to EHR medication alerts, which generally appear during prescribing processes,44 safety for this patient population may be improved via mechanisms that support more regular, focused follow-up and monitoring. Specifically, monitoring is warranted to detect emergent renal-drug problems for renally eliminated and nephrotoxic medications that occur between prescribing intervals, since during those times, renal function may decline below important medication safety thresholds. Additionally, to aid safety, it may be warranted to restrict renally eliminated and nephrotoxic medication prescriptions to reduced, limited quantities and fewer number of refills for patients with known renal impairment. Our results indicate that efforts to further reduce reliance on HCPs’ vigilance (beyond alert interventions), are likely to improve safety for patients with renal impairment. We did not find alert fatigue to be a notable factor for renal-drug problems, most likely because VA’s CPOE system provides only limited renal alerts.21 Additionally, our study examined decision-making for renal-drug problems broadly, regardless of whether or not CPOE alerts were involved in the incident. In the incidents we collected, HCPs detected problems not just during prescribing, but at other points of the medication use process, which typical CPOE alerts cannot detect. Our findings are important because they provide evidence that can inform future enhancements of alerts. Future studies could also build on our findings and implement patient education programmes to empower patients with knowledge on how to detect potential renal-drug problems and the importance of regular follow-up.
Third, computerised decision support features in EHRs, such as medication alerts, should account for differences in HCPs’ cognitive needs for renally eliminated versus nephrotoxic medications (figure 2). One study implemented alerts in a hospital’s CPOE system in an effort to improve prescribing for patients with renal impairment.45 The researchers used a semi-automated process where pharmacists received alerts and then could enter recommendations into a progress note in the EHR. However, they did not articulate or discuss the different decision-making or alert content needs between the two types of renal-drug problems.45 Similarly, a pilot study conducted in three types of clinical settings (geriatrics, internal medicine and outpatient care) implemented an alert system to reduce renally inappropriate prescriptions.46 Their alerts provided recommendations for renal dosing and discontinuing drugs, which were perceived as useful by physicians. However, the study did not indicate if those alerts and associated recommendations differed according to the type of renal-drug problem.46 Our study suggests that for renally eliminated medications, providers need information on dosage reductions; for nephrotoxic medications, providers need information on safe drug alternatives. Future, tailored approaches to clinical decision support that distinguishes between these two types of renal-drug problems may substantially reduce inappropriate prescriptions for patients with renal insufficiency.
Our study has limitations. First, it was conducted at one large VA Medical Centre, so our findings may not always generalise to other healthcare settings, although we have no evidence that our findings or HCP’s cognitive needs are unique to this healthcare organisation. Second, the VA’s EHR system is capable of alerting HCPs about the patient’s renal impairment at the start of medication ordering process,21 but does not offer real-time renal alerts for specific medication orders other than metformin, a state that likely led to less problem detection via computerised alerts in our sample. Our broader focus on HCPs’ cognitive strategies and cues rather than studying alert design and technical aspects of the VA’s EHR, however, makes this limitation less of a concern. Third, HCPs submitted renal-drug incidents voluntarily, and since the study focused on medication problems, some individuals might have avoided participation even though we examined cases where participants intervened. Thus, the voluntary nature of this study might have influenced the sample of participants and type of incidents we captured. For instance, HCPs might have submitted incidents for which they felt the most confident that they had taken the best course of action for the patient. However, this sampling strategy was specifically selected as it aligns with our objective of examining HCPs cognitive strategies for situations where errors and potential safety concerns were caught and addressed. Fourth, although we intended to compare cognitive cues used by prescribers versus non-prescribers, we could not find differences in their decision-making processes, perhaps due to the wide variation in patient cases. Finally, recall bias might have occurred for HCPs, since we conducted follow-up interviews after the incident. The adaptations that we made to the CTA, however, were intended to stimulate recall and aid data collection accuracy.19
Conclusion
This study yielded a descriptive model of the decision-making process that HCPs used to manage renal-drug problems in patients with renal insufficiency. This model is expected to be useful for medical, pharmacy and nursing trainees and residents to help them gain familiarity with HCPs’ processes for managing renal-drug problems. Based on the findings, improvements are warranted for three areas of healthcare systems: (1) support the cyclical nature of the three main stages of renal-drug problem management via more robust, longitudinal safety mechanisms for individual patients, (2) provide tools to alleviate HCPs’ heavy reliance on their own vigilance to detect patients’ renal-drug problems, which is mentally demanding and potentially error-prone and (3) develop systems to support HCPs’ distinct decision-making needs for renally eliminated versus nephrotoxic medications. Our findings can inform systems to mitigate renal-drug problems; thus providing safer care for patients.
Acknowledgments
Views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Veterans Affairs or the US government. Rachel Dismore assisted with IRB paperwork, participant recruitment, incident card collection, descriptive statistics and data in Tables 1 and 2. Dr Weiner is Chief of Health Services Research and Development at the Richard L. Roudebush Veterans Affairs Medical Center in Indianapolis, Indiana. We would like to thank all of the physicians, nurses and pharmacists who participated.
References
Footnotes
Contributors NE participated in data analysis, interpretation, model development, drafted the initial manuscript and led the finalisation of manuscript content. AI participated in review of cognitive task analysis interviews for trends, assisted with drafting pieces of the initial manuscript, provided critical edits/input to the manuscript and approved of the final manuscript. JD participated in data analysis, interpretation, model development, provided critical edits/input to the manuscript and approved of the final manuscript. KJA screened submitted cases for clinical appropriateness/interview selection, participated in review of cognitive task analysis interviews for trends, provided critical edits/input to the manuscript and approved of the final manuscript. MW contributed to study conception and design, data interpretation, provided critical edits/input to the manuscript and approved of the final manuscript. LGM contributed to the design of the study, aided in interpreting findings, provided critical edits/input to the manuscript and approved of the final manuscript. PAG contributed to study conception and design, data interpretation, provided critical edits/input to the manuscript and approved of the final manuscript. AJZ contributed to the design of the study, aided in interpreting findings, provided critical edits/input to the manuscript and approved of the final manuscript. AR conceived and designed the study, wrote the funded grant, conducted all cognitive task analysis interviews, aided in data analysis and interpretation, provided critical edits/input to the manuscript and approved of the final manuscript.
Funding This work was supported by VA HSR&D Career Development Award 11-214 (PI: Russ), along with the Centre for Health Information and Communication, US Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, CIN 13-416.
Disclaimer Views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Veterans Affairs or the US government. Rachel Dismore assisted with IRB paperwork, participant recruitment, incident card collection, descriptive statistics and data in Tables 1 and 2. Dr Weiner is Chief of Health Services Research and Development at the Richard L. Roudebush Veterans Affairs Medical Center in Indianapolis, Indiana. We would like to thank all of the physicians, nurses and pharmacists who participated.
Competing interests LGM is co-owner of Applied Decision Science, LLC, a company that studies decision-making in complex environments and utilises the critical decision method. MW has stock in Allscripts and Express Scripts Holding Company. All other authors report that they have no competing interests.
Ethics approval This study was approved by the Indiana University Institutional Review Board and the Richard L. Roudebush VA Research and Development Committee (IRB study #1301010433).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement A sample of the qualitative data from participants' interviews is available in the online supplemental document. For additional inquiries, please contact the Principal Investigator, Dr. Alissa L. Russ.
Patient consent for publication Not required.