Article Text
PDF
Abstract
Objective Examine clinical reasoning and decision making in an out of hours (OOH) primary care setting to gain insights into how general practitioners (GPs) make clinical decisions and manage risk in this environment.
Design Semi-structured interviews using open-ended questions.
Setting A 2-month qualitative interview study conducted in Oxfordshire, UK.
Participants 21 GPs working in OOH primary care.
Results The most powerful themes to emerge related to dealing with urgent potentially high-risk cases, keeping patients safe and responding to their needs, while trying to keep patients out of hospital and the concept of ‘fire fighting’. There were a number of well-defined characteristics that GPs reported making presentations easy or difficult to deal with. Severely ill patients were straightforward, while the older people, with complex multisystem diseases, were often difficult. GPs stopped collecting clinical information and came to clinical decisions when high-risk disease and severe illness requiring hospital attention has been excluded; they had responded directly to the patient's needs and there was a reliable safety net in place. Learning points that GPs identified as important for trainees in the OOH setting included the importance of developing rapport in spite of time pressures, learning to deal with uncertainty and learning about common presentations with a focus on critical cues to exclude severe illness.
Conclusions The findings support suggestions that improvements in primary care OOH could be achieved by including automated and regular timely feedback system for GPs and individual peer and expert clinician support for GPs with regular meetings to discuss recent cases. In addition, trainee support and mentoring to focus on clinical skills, knowledge and risk management issues specific to OOH is currently required. Investigating the stopping rules used for diagnostic closure may provide new insights into the root causes of clinical error in such a high-risk setting.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
Article summary
Article focus
Clinical reasoning and decision making in an out of hours (OOH) primary care setting.
The aim is to gain insights into how general practitioners (GPs) make clinical decisions and manage risk in this environment.
Implications for system changes and training.
Key messages
Clinical decision making in OOH is dominated by rule-out strategies for severe illness or potentially high-risk diseases.
GPs use three main criteria to determine diagnostic closure: global wellness with rule-outs, responded to patient needs, presence of a reliable safety net.
Improvements to clinical decision making could be achieved by providing routine feedback to clinical staff working in OOH, building in systems to support reflection on clinical cases and more tailored GP training.
Strengths and limitations of this study
The design of the study is based on a strong theoretical framework provided by the dual theory of cognition.
Face validity through using recently seen cases.
Limitations relate primarily to sampling, participants consisting of self-selected individuals.
Introduction
Primary care in the UK is provided during weekends, evenings and overnight by various types of out of hours (OOH) services, and it has become a core component of round-the-clock primary care in many countries. Accessible and high-quality OOH services provide an important mechanism to reduce service pressures and associated costs from inappropriate use of Emergency Department services.1 The majority of OOH care in the UK is provided by general practitioners (GPs), often in conjunction with emergency medical and nursing practitioners, and organised by private providers or primary care trusts. Patients typically initiate contact through a telephone triage system. Those requiring face-to-face consultations, but not urgent hospital referral, are seen by a GP at a local clinic or in the patient's home.
Patients who contact OOH services are more likely to have acute problems than those who are seen in daytime primary care. Therefore, GPs working in OOH are likely to meet a higher number of acutely ill patients than in their routine practice. Mostly these are ‘new’ patients, that is, presenting to GPs who are unlikely to know them and with little or no access to their GP or hospital records. The combination of lack of continuity and high incidence of acute illness provides a potentially hazardous practice environment, in which errors can occur.2 Although the incidence of significant clinical error is unknown, high profile adverse incidents have received significant media attention3 and prompted a fundamental review of OOH service provision in the UK.4 Recommendations focused on clinical governance, more effective performance management, better team work and making patient records more accessible. In addition, they have highlighted staff development issues such as the possible use of checklists, regular feedback, improving GP training for the specific demands of OOH and individualised audits.
However, the ways in which GPs make clinical decisions in OOH has received little attention. Yet, how clinical decisions are made under these circumstances is currently unknown, and little is known of the factors that affect safety and referral patterns. We therefore examined clinical reasoning (CR) and decision making in an OOH setting, with the aim of gaining insights into how GPs manage risk while keeping patients safe in this environment and focusing on factors they may improve the delivery of services.
Methodology
Studies of CR describe the reasoning processes employed during a consultation.5 We have previously published a model of CR based on our work with GPs.6 The assumption behind the model was that understanding the reasoning process was essential for fruitful reflection on clinical practice. The model highlights key areas of knowledge and critical value judgements that are used in the clinical encounter. We used this model, which is derived from the dual theory of cognition,7 as the framework for a qualitative study of OOH practice.8 We are not aware of other studies that have tested the applicability of similar theory-based models to real-life consultations, though generic theoretical models have been proposed.9
The dual theory posits two systems interacting with each other. The non-analytic system 1 provides the fast response, while the reflective system 2 is slow and may override the original response. There is rapid recognition of a salient feature of the case,10 which may stem from the context provided by what is already known about the patient, physical appearance or critical cues specific to a presentation. This links to pre-existing mental representations, based on theory and experience built up of multiple exposures to similar cases. While some aspects of the salient feature can be described by the clinician, others may appear no more than a ‘sense of alarm or reassurance’, described as ‘gut feelings’ by Stolper et al,11 which impact on the reasoning and decision-making process of GPs. This combination of recognition and gut feelings leads to a limited search for more cues. System 2 represents reasoning through the initial response vis-à-vis the rules provided by the theory of the profession and may lead to correcting or overriding system 1. In cases where there are no salient features perceived, system 2 takes over in place of the absent fast response.7 The initial judgements provide the frame for the direction of the search for further information. Drivers prioritise the search, generally aimed at risk avoidance with the focus on ruling diagnostic options out or in, through searching for a few highly critical cues that support or negate the original frame. A necessary aspect of this kind of reasoning is a stopping rule for data collection, but we have not been able to find any research that describes them in a clinical setting.12
Setting and cohort
We conducted a 2-month study in Oxfordshire, UK, where OOH care is provided by a combination of different services. Most weekend, evening and holiday care is provided by GPs employed by the primary care trust. Overnight services are provided by a private provider. Initially, we invited all 62 GPs who regularly worked in the OOH service organised by the primary care trust. We asked them to participate in 30 min interviews about two recently seen OOH cases. We chose 30 min to minimise the pressure on busy work schedules. From past experience, most GPs would be interviewed before or after consulting and often during their brief lunch breaks. The time allocated was deemed sufficient to capture the data on the issues on which we were focused. Information about the study was kept to a minimum to reduce respondent bias. We obtained positive responses from 29 (47%) GPs and interviewed 21 (34%) GPs, who provided a total of 42 cases. Four GPs had been in practice <5 years, nine for 5–20 years and eight for >20 years (range <1–37 years). In our experience and in reports in the literature, busy professionals are relatively difficult to persuade to find time for an interview. Therefore, one is generally unlikely to find random cohorts, the participants coming through word of mouth and professional contacts. The only variable we would expect to have an impact for this study was length of experience, and in this respect, our cohort was relatively well represented.
Design and data collection
To provide face validity to our findings, we used recently seen cases, rather than case vignettes in a laboratory setting. Participants were asked to describe the presentation of two new patients who had consulted with them during their most recent OOH session and interviewed within 48 h of their session. They were asked to discuss a straightforward case and a more demanding one. After the first few interviews, it became apparent all straightforward cases offered were young children with diarrhoea or raised temperature. Subsequent interviews excluded this group of presentations. Most GPs referred to computer printouts of the cases during the interviews. All interviews, conducted by a single researcher (JB), were analysed, audiotaped and transcribed. The GPs were asked to describe the presentation of the patient and told that there would be interruptions to clarify what went on in their mind during the consultation. The semi-structured interviews used open-ended questions and focused on each step of the model (appendix 1).6
The literature suggests that the greatest impact on decision making occurs at the first step which frames the direction of the process and at the final step of diagnostic closure when immediate data collection stops and where most of the errors are likely to occur.13 14 Our focus for the first step was to what extent GPs could describe the salient feature that would lead to recognition of the case. We specifically did not attempt to dig deeper or look for gut feelings, at the same time acknowledging their prominence in the diagnostic process. On the other hand, we explored these instinctive aspects at the stage of closure. This relates to our particular interest in this step. The scope of the study would not allow us a full exploration of all facets of each step.
Data analysis
The audiotapes were used to develop brief typed summaries of each case to gain a sense of the processes; these were then used to build a structure for the analysis. This was carried out with NVivo software. Two researchers searched the text independently for underlying themes in relation to our conceptual framework. Categories were coded according to emerging themes and added to or changed as new concepts emerged.15 Any differences in interpretation were discussed, and consensus was reached on clustering to common themes.
We divide the results into five areas: (1) the participants' perceptions of the OOH environment and its impact on practice, (2) characteristics of a case to make it straightforward or demanding, (3) the process of reasoning in this context, (4) prerequisites for diagnostic closure and (5) learning points from OOH consultations for GP trainees. Within each area, we report on common themes raised by the participants.
Results
All participants indicated that our questions were easy to understand and many remarked that it was a good way for them to reflect on their practice. The analysis was consistent with previous findings on the dual process of reasoning, where instant recognition, followed by reasoning and value judgements lead to a search for a limited number of cues prior to diagnostic closure. (Direct quotes from the interviews appear in italics.)
Key perceptions of OOH environment and its impact on practice
The most powerful themes to emerge related to dealing with urgent potentially high-risk cases, trying to keep patients out of hospital, the concept of ‘fire fighting’, time pressures and working in an unfamiliar environment (table 1). This challenging environment impacted on their practice in a number of ways. It was often difficult to work out whose agenda they had to deal with, the patient's or concerned relatives or carers who had contacted the OOH service. Their approach to the patient was to deal with immediate problems only, rather than the usual holistic approach, as described by one GP: basically you've got to decide whether this chap is safe. Lack of feedback and follow-up was the rule: you don't really tend to get any feedback at all unless something has gone horrendously wrong (table 2).
Participants' perceptions of out of hours (OOH) environment (quotes in italics, numeric refers to interviewee ID). Emerging themes to question: How is the OOH environment different from normal practice and how does it impact on your practice? Frequently these comments came up spontaneously
Impact of out of hours (OOH) environment on practice
What makes a case straightforward or demanding?
There were a number of well-defined characteristics thath made presentations easy or difficult to deal with. The focus was very much on illness severity, and a severely ill patient was seen as straightforward, as they just required hospital transfer: It's easy when somebody is terribly unwell…going to admit them to hospital. Oldery patients were often difficult because of the complex nature of multiple pre-existing diseases in the absence of background information. Children and young adults were seen as straightforward with well-defined single conditions: Easy cases tend to be younger people with less complicated medical histories and where serious things are less common. Familiarity with the condition helped, while falling between the two extremes tended to lead to lack of confidence (table 3).
Key themes to differentiate between straightforward and demanding presentations. Question: Can you explain why you felt that Case 1 was straightforward and second one more demanding or difficult? (S: straightforward, D: demanding)
The process of CR
In most cases, there were instantaneous formulations, for instance, on seeing a 2-year-old child with 2 days of fever, runny nose, coughing and pulling her left ear: the likelihood is that it was probably a upper respiratory tract infection…with otitis media. The salient features that brought these to mind were generally recognised: had all the signs of a grizzly toddler tugging at the left ear, yep have to exclude an otitis media. An inability to make such instantaneous judgement made the case difficult and required data clarification, which was not always achieved. Once presentation issues were clarified, early formulations were associated with brief lists, generally headed by the most high-risk clinical conditions. This drove the search for one or two critical cues to rule out these. There was a particular focus on red flags which potentially indicated severe illness: she's quite dry…her oxygen saturation is a little bit low… I'm really thinking about, is this lady compromised because of her diarrhoea? (table 4).
Process of reasoning in this context. Question: Can you describe what was the main driver of the process? Often these statements were made spontaneously during the interview. (Digit after stop refers to 1st or 2nd case for interviewee.)
Prerequisites for diagnostic closure
Diagnostic closure occurs when the clinician stops immediate data collection and makes a management decision based on the available information. A rule of thumb used by one GP defined the issue: dividing people into two groups…on the basis of one question: is it serious or potentially serious or not? Confidence to stop and make a decision was then assessed using a number of criteria as in table 5.
Prerequisites for closure of diagnostic process. Question: Can you try and explain what made you stop the process at this particular stage?
Confidence that appropriate threshold had been reached was rarely stated numerically by participants, and when asked to assess this way, it was generally over 90%, but some were not willing to go beyond a descriptive assessment. As a prerequisite to closure, GPs depended on a combination of three criteria being met: (1) Confidence in ruling out a list of diseases, with clarity about illness severity: I'd reached the end of what I could do. (2) Meeting patient's needs: I'd relieved the patient's anxiety that was the main issue at stake. (3) An ability to set up a good safety net: a gold star safety net in this case. This is described as particularly important in emergency settings where diagnostic uncertainty is prominent and the patient's condition may deteriorate unexpectedly.16 It was consistently stressed during our interviews: mum was,…supportive,…safety netting was pretty major, and as a consequence, GPs went to great lengths to set one up if necessary.
Learning points for GP trainees
This was the last question we asked about each case. It turned out to be particularly useful as it gave GPs time to pause and reflect on what they regarded as particularly significant training issues relating to that consultation. Emerging themes divided into (1) training needs on how to deal with patients, stressing the need to develop rapport despite the intense time pressures; (2) learning to deal with uncertainty: think about how certain am I, how certain do I need to be to send him home and (3) learning about common presentations with a focus on critical cues to exclude severe illness and a few high-risk diseases: what are the danger signs you would look out for; specific symptom questions you want to ask (table 6).
Trainee take home messages. Answers in response to question: What sort of message would you like your trainee to take home from this consultation?
Discussion
To our knowledge, this is the first study to show how the context of OOH practice impacts on the CR process in primary care. Participants were aware that their behaviours and decisions were affected by the environment. In line with the perceived purpose of OOH care, objectives were to keep patients safe, respond to their needs and reduce unnecessary hospital attendances. Their expectation was to deal with urgent and potentially high-risk issues only, avoiding hospital admissions unless patients were at unacceptable risk.
GPs found practice in OOH rewarding but identified a number of issues that made this a high-risk clinical environment. The most prominent of these include a lack of background information in the form of medical records and complex multisystem chronic disease presentations where it was difficult to define the pertinent issues. Success depended on clinical skills to develop rapport with patients and recognise their concerns. Lack of regular feedback on clinical performance was raised repeatedly.
Our findings were consistent with the dual theory of cognition. In line with system 1, there was instant recognition on finding salient features of the presentation. Even before GPs knew the presenting complaint, they were constantly on the lookout for severe illness (tables 1 and 3), using red flags (table 4) and well-tried markers of systemic illness (table 5). Lists and markers were derived from personal experience, and the novice would struggle in their absence (table 6). These early cues would frame the immediate search for further cues. The major driver was to exclude serious disease and severe illness (table 4). Ruling out is commonly used in everyday practice,17 but it was practically uniform in OOH. It was surprising how often this was seen as ‘working backwards rather than forwards’. System 2 that represents correcting or overriding system 1 was used in many but not all cases. For instance, ruling in first was used only occasionally, in straightforward familiar presentations, and then followed by ruling out in most cases. Once serious disease, severe illness or the risk of abrupt deterioration were excluded, the precise diagnosis became less important and could be left for another day. Such decisions, as they often required reflection, represented system 2. Diagnostic closure that occurs after a clinician has weighed up all the options and needs to choose a specific course of action, in all but the most clear-cut presentation would involve such reflection (table 5). This is the point where clinical errors crystallise, and therefore, understanding the rules and thresholds for stopping provides insights into the causes of diagnostic error. Little is known about how diagnostic closure occurs in most clinical specialties and settings, including primary care. The OOH setting provided a unique opportunity to gain insights into diagnostic closure, as clinical care is associated with clear-cut end points, not often seen in other practice settings. The stopping rules used by GPs involved three criteria: first, have high-risk conditions and severe illness that require urgent hospital attention been excluded? Second, have I responded directly to the patient's needs? Third, is there a reliable safety net in place, given the high-risk environment where error may be more likely than in a familiar situation?
The basis of individualised judgements about acceptable confidence levels for the stopping rules were difficult to define and as one GP put it ‘feels like a nebulous thing’. This is consistent with a previous study, which used an end point of variation in hospital referral rates and concluded that the individual nature of learning from experience leads to a variation in approach to risk tolerance and associated confidence levels for decision making.18 We believe that our findings are consistent with the work of Gabbay and le May,19 who suggested that multiple exposures to clinical experience, embedded in a social context lead to a set of ‘internalised, collectively reinforced and often tacit guidelines…malleable to deal both with individual patients' needs and the multifarious factors that come into reckoning when making decisions.’
Strengths and limitations of the study
This is the first study that we are aware of that has explicitly explored the diagnostic reasoning that occurs in OOH primary care and is likely to be relevant to other countries with similar systems of primary care. The design of the study is based on a strong theoretical framework provided by the dual theory of cognition. This has been shown to have generalisability to human reasoning irrespective of the domain.7 There are also good theoretical models suggesting that clinical thinking can be modelled within this framework.9 The study also has face validity using recently seen cases, where participants were not made aware of the content and purpose of the interviews.
Limitations relate to a number of issues. (1) Sampling: participants were not chosen at random but consisted of self-selected individuals who made up a third of a cohort we approached. This is an almost universal problem with studies that involve busy and senior clinicians. We asked participants to select the two most recently seen patients who met the selection criteria. We cannot be certain if these rules were always applied. We have not performed similar patient-based studies in other settings in the UK or in countries with different health systems, and to this stage, the research has not included other specialty groups. (2) There are concerns regarding the validity of think-aloud protocols, but they have been considered appropriate for studies of cognition in natural settings as used here.20 (3) We designed the study to allow us to focus on specified steps of the CR process, for example, closure. This means that some steps could not have full coverage of issues, such as some of the crucial intuitive aspects of the early stages of diagnosis. This will require further qualitative work. (4) Another issue relates to triangulation. These qualitative studies lead to the development of inventories suitable for triangulation through quantitative studies.
We believe that the dual theory of cognition is the closest model to how people actually think in real-life scenarios and this makes it an appropriate way to study reasoning in a clinical setting.13 On account of the strong theoretical base, our findings, albeit with different emphases, are likely to have some degree of generalisability to other specialties and health systems.
Implications for practice
High-quality OOH care is important to the functioning of the health service and particularly the interface between primary care, emergency care and acute hospital care. While the results of our investigation provide a unique insight into the clinical decision making that occurs in OOH, we have several practical recommendations based on our research which could potentially be implemented with minimal additional resources.
Recognising the limitations of our study, we also find that our conclusions are consistent with the literature on the importance of feedback on performance and settings for fruitful reflection on experience. We believe that given this congruence between the relevant literature and our proposals, our recommendations merit cautious acceptance and evaluation.
First, at the organisational level, an automated, regular and timely system of feedback to GPs and other clinical staff practising in the OOH setting is essential. Providing clinical care in an environment almost devoid of clinical feedback (apart from when there are complaints or serious events) is contrary to the type of practice that GPs are trained to work in. One of the mainstays of primary care practice is the longitudinal relationship with patients, so that information gathering and clinical decision making occur over several encounters. In the OOH setting, this process becomes compressed and is compounded by high patient flow. Since the vast majority of OOH care is provided by local GPs and other staff, an initial step would be to provide routine feedback on the outcomes of their clinical decisions. Since most OOH services are now computerised, this could be implemented with little additional resource. These systems need to be implemented with sensitivity, recognising the risk of creating anxiety and dislike of being monitored. Feedback needs to be seen as non-judgemental and to be used by the recipient and trusted colleagues only. The importance of regular feedback on performance to gain expertise is well accepted in the literature.21
Second, for the individual, in line with the principles of fruitful learning from experience, we suggest that peer and expert clinician support with regular meetings and feedback on recent cases should be implemented. The educational literature provides good evidence that practice with feedback enhances deep learning, which goes with improved understanding of subject matter.22 GPs would then be in a position to reflect, calibrate their decision making and update their mental constructs, all necessary for good practice. Some GPs noted that they previously had used informal coffee breaks to do this, but the increasing numbers of patients attending OOH now precluded this. However, such informal systems are probably inadequate, and we propose that OOH practice should experiment with formal opportunities for reflection in a safe and supportive environment.
Third, our research has implications for training GPs, whose requirement for OOH training and experience is fairly modest. Individual mentoring and group discussions need to focus on (1) how to develop rapport with new and anxious patients; (2) how to deal with uncertainty and (3) common presentations with a focus on critical cues to exclude severe illness and the relatively few high-risk diseases. We believe that the model of deliberate practice, as described by Ericsson,23 is likely to be a suitable model for training. The clinician seeks out cases in areas of perceived weakness, and practice is associated with immediate individual feedback and group discussions. To this stage, excellent improvements have been demonstrated in interventional specialties but not in primary care. However, in view of the confluence of the principles of deliberate practice and deep learning approaches, we recommend the evaluation of deliberate practice training models for trainees.
We also draw attention to the ongoing difficulties in accessing GP or hospital records from OOH settings; a situation that we acknowledge is unlikely to change without the continued significant investment in the NHS IT infrastructure.
Implications for future research
We recognise that our study is a starting point for further research in this area. To explore the generalisability of our findings to other OOH or alternative emergency settings, our studies will need to be replicated with appropriate cohorts of GPs and other clinicians working in such environments. We also propose to develop a quantitative questionnaire based on our qualitative studies and on leads that we find in the literature. Such studies, extending beyond our cohort, may then provide data to allow greater generalisability of the results. We do not know if the practice changes that we propose will lead to better clinical care and this needs separate evaluation. Finally, investigation of the rules clinicians use for diagnostic closure may give us insights into the root causes of clinical error. We are at this stage in the process of developing a qualitative study, examining diagnostic error in general practice.
Conclusions
This study provides the first evidence of how working in an OOH environment impacts on CR and decision making. GPs identify OOH as a potentially high-risk clinical environment and adapt their practice approach to this setting. Lack of access to GP or hospital records is a major concern. Our study provides further support to the literature on reflective practice and educational programmes in clinical settings. We suggest that improving feedback to GPs about their clinical decisions and providing opportunities for reflection on OOH practice may be valuable for ongoing review and improvement of clinical practice. The findings provide guidance to the development of GP training programmes to cover the specific needs of OOH. Exploring further the rules and criteria that clinicians use for diagnostic closure may give us insights into the root causes of clinical error.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data Supplement - Manuscript file of format pdf
- Data Supplement - Manuscript file of format pdf
Footnotes
To cite: Balla J, Heneghan C, Thompson M, et al. Clinical decision making in a high-risk primary care environment: a qualitative study in the UK. BMJ Open 2012;2:e000414. doi:10.1136/bmjopen-2011-000414
Contributors JB designed the study, conducted the interviews, analysed the data and produced the first draft to final document. CH and MT conceived the project and were involved with design and all stages of writing the drafts and final document. MB participated in the interviews and analysis as well as reviewing the drafts and final document.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None.
Ethics approval This is a quality improvement study involving volunteer general practitioners who cannot be identified.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The original transcripts are available.
Linked Articles
- Miscellaneous