Article Text

Original research
WHY STOP? A prospective observational vignette-based study to determine the cognitive-behavioural effects of rapid diagnostic PCR-based point-of-care test results on antibiotic cessation in ICU infections
  1. Suveer Singh1,2,
  2. Martine Nurek3,
  3. Sonia Mason4,
  4. Luke SP Moore5,6,
  5. Nabeela Mughal5,6,
  6. Marcela P Vizcaychipi7,8
  1. 1Faculty of Medicine, Imperial College London, London, UK
  2. 2Respiratory and Intensive Care Medicine, Chelsea and Westminster Hospital NHS Foundation Trust, London, UK
  3. 3Surgery and Cancer, Imperial College London, London, UK
  4. 4Guy's and St Thomas' Hospitals NHS Trust, London, UK
  5. 5Imperial College London, London, UK
  6. 6Chelsea and Westminster Hospital NHS Foundation Trust, London, UK
  7. 7APMIC, Imperial College London, London, UK
  8. 8Magill Department of Anaesthesia and Intensive Care Medicine, Chelsea and Westminster Healthcare NHS Trust, London, UK
  1. Correspondence to Professor Suveer Singh; suveer.singh{at}imperial.ac.uk

Abstract

Objectives Point-of-care tests (POCTs) for infection offer accurate rapid diagnostics but do not consistently improve antibiotic stewardship (ASP) of suspected ventilator-associated pneumonia. We aimed to measure the effect of a negative PCR-POCT result on intensive care unit (ICU) clinicians’ antibiotic decisions and the additional effects of patient trajectory and cognitive-behavioural factors (clinician intuition, dis/interest in POCT, risk averseness).

Design Observational cohort simulation study.

Setting ICU.

Participants 70 ICU consultants/trainees working in UK-based teaching hospitals.

Methods Clinicians saw four case vignettes describing patients who had completed a course of antibiotics for respiratory infection. Vignettes comprised clinical and biological data (ie, white cell count, C reactive protein), varied to create four trajectories: clinico-biological improvement (the ‘improvement’ case), clinico-biological worsening (‘worsening’), clinical improvement/biological worsening (‘discordant clin better’), clinical worsening/biological improvement (‘discordant clin worse’). Based on this, clinicians made an initial antibiotics decision (stop/continue) and rated confidence (6-point Likert scale). A PCR-based POCT was then offered, which clinicians could accept or decline. All clinicians (including those who declined) were shown the result, which was negative. Clinicians updated their antibiotics decision and confidence.

Measures Antibiotics decisions and confidence were compared pre-POCT versus post-POCT, per vignette.

Results A negative POCT result increased the proportion of stop decisions (54% pre-POCT vs 70% post-POCT, χ2(1)=25.82, p<0.001, w=0.32) in all vignettes except improvement (already high), most notably in discordant clin worse (49% pre-POCT vs 74% post-POCT). In a linear regression, factors that significantly reduced clinicians’ inclination to stop antibiotics were a worsening trajectory (b=−0.73 (−1.33, –0.14), p=0.015), initial confidence in continuing (b=0.66 (0.56, 0.76), p<0.001) and involuntary receipt of POCT results (clinicians who accepted the POCT were more inclined to stop than clinicians who declined it, b=1.30 (0.58, 2.02), p<0.001). Clinician risk averseness was not found to influence antibiotic decisions (b=−0.01 (−0.12, 0.10), p=0.872).

Conclusions A negative PCR-POCT result can encourage antibiotic cessation in ICU, notably in cases of clinical worsening (where the inclination might otherwise be to continue). This effect may be reduced by high clinician confidence to continue and/or disinterest in POCT, perhaps due to low trust/perceived utility. Such cognitive-behavioural and trajectorial factors warrant greater consideration in future ASP study design.

  • microbiology
  • infectious diseases
  • diagnostic microbiology
  • adult intensive & critical care
  • intensive & critical care
  • molecular diagnostics

Data availability statement

Data are available in a public, open access repository. The data are publicly available in The Open Science Framework: https://osf.io/bgp84/.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • Use of carefully constructed and controlled clinical vignettes allowed us to isolate and quantify the effects of specific variables on antibiotic decision making, while holding others constant.

  • However, the vignettes were simplistic, which could limit the generalisability of the findings to clinical practice.

  • Relatedly, participants were not given the option to change/de-escalate antibiotics (lest it become the ‘safest’ and, therefore, ‘default’ option).

  • Finally, we chose not to examine the effect of a positive (vs negative) point-of-care test result, expecting that this would inhibit antibiotic stop decisions (the phenomenon of interest).

Introduction

Antimicrobial resistance is a developing global threat. A crisis of uncontrollable infection has been predicted.1 That said, prescribing decisions in critically ill patients with nosocomial infection are often subject to diagnostic uncertainty; with the risk of negative consequences of undertreatment/overtreatment.2 Antibiotic stewardship (ASP) is used as a strategy to improve appropriate prescribing; sometimes successfully implemented at the organisational hospital level.3 The intensive care unit (ICU) is an important setting for such ASPs. Not only is there a high burden of antimicrobial use,4 but the illness severity naturally dictates a tendency to longer courses. Despite this, ASP-driven reductions in antibiotic duration have not shown association with worse in-hospital mortality.3 The use of biomarker surrogates of infection such as procalcitonin (added to the conventional markers of white cell count (WCC) and C reactive protein (CRP)) may be successful in reducing unnecessary antibiotic course lengths, in randomised controlled clinical studies.5 Yet, at a patient-clinician level, there is more uncertainty about the benefits of these strategies. Evidence exists regarding the benefits of certain ASP strategies such as frequent microbiologist input at ward rounds.6 Other factors are likely important such as an organisational culture (ie, restriction and enablement) and prescribing guidelines.3 However, autonomous individual decision-making is variable, often widely so among clinicians. Indeed, the same clinician may have a different judgement in a similar scenario at different time points.

Unwanted variability in decision-making has been termed ‘noise’.7 Various sources of such variability can influence antibiotic stop decisions, including system noise (eg, organisational variability, casemix, prevalence of infection/resistance, prescribing policies), pattern level noise (ie, interclinician variation due to risk aversion, experience) and occasion noise (intraclinician variability). Guidelines and protocols can reduce system level noise, but less so inter/intra clinician variability.7

Point-of-care tests (POCTs), using molecular platforms such as PCR,8 are emerging as a potentially valuable tool in rapid diagnostics. They have been an important strategy during the COVID-19 pandemic for identification of infection by SARS-CoV-2.9

Notably, POCTs for rapid diagnosis or exclusion of infection may be indirect biomarkers (eg, surrogates of the inflammatory effect of an infectious agent, ie, IL1, IL8) or they may directly identify an infective agent (ie, 16s or 23s ribosomes, PCR or RT-PCR).10 11 Here, we refer to the second (ie, infection-identifying POCT), for which a number of commercially available PCR molecular platforms are in use mainly to determine the presence of infective organisms.

In the context of bacterial infection, POCTs have been treated more as an antibiotic start/stop trigger, where other indicators of infection may be less certain. Specifically, in suspected ventilator-associated pneumonia (VAP) studies, using biomarker combinations such as IL1/8 are highly accurate in ruling out respiratory infection.12 Yet this efficacious ‘rule out’ test has not led to more antibiotic free days.13 Thus, the utility of POCT in decision-making strategies to reduce antibiotic prescribing has not been demonstrable.13 Their effect is speculated to be diminished by competing factors (cognitive, behavioural and/or situational), producing unwanted variation in judgements. These probably override clinical information available at the time of the antibiotic stop decision. Yet, little research has been performed to identify, quantify and modify these factors.14

So, we sought to understand what factors, and to what extent they influence clinicians’ antibiotic stop decision-making when presented with scenarios of common ICU-related respiratory infection and varying degrees of apparent uncertainty at the point of the decision making. The focus on stopping antibiotics was because the threshold for resolution of an infection is poorly defined. This uncertainty may lead to variability in the decision to stop antibiotics. With this in mind, we sought to:

  1. Quantify the effect of negative POCT results on antibiotic stop decisions, in situations of uncertainty for resolving infection.

  2. Determine the effect that factors which can ‘compete’ with negative POCT results (and prevent stopping) had on stop decisions.

  3. Determine the effect of defined clinician characteristics on antibiotic stop decisions.

We expected that a negative POCT result would increase stop decisions (hypothesis 1), while the following factors would reduce it: an ambiguous/worsening clinico-biological trajectory (hypothesis 2), clinicians’ first impressions (specifically, high confidence that antibiotics are still needed, hypothesis 3) and disinterest in POCT (rejection of the test, when offered, hypothesis 4). We also expected that less experienced clinicians would be less inclined to stop (due to lower confidence, hypothesis 5), as would those higher in risk averseness (hypothesis 6). Further details are available in online supplemental material 1.

Methods

Participants

Consultants and trainees in intensive care medicine (3+ months continuous experience in ICU) currently working in UK-based university teaching hospitals were invited to take part. Participants were invited in one of two ways. First, we made use of existing clinical and critical care research networks by posting advertisements in closed social media groups that are exclusive to ICU consultants and trainees. Second, eligible participants known to the author group were invited via direct email or word-of-mouth. Interested participants were sent a hyperlink to the study website, which contained a link to the online survey (hosted by Qualtrics). On accessing the survey, participants read an information sheet and a consent form. The consent form ended with a tickbox, stating ‘I consent to participate in this study.’ Participants could not proceed with the survey unless they ticked this box (ie, provided informed consent). The survey remained open May–September 2021. Participants were given the opportunity to join the WHY STOP Consortium (https://why-stop.wixsite.com/itu-decision-making), and/or to be acknowledged within the manuscript (all optional).

Materials

Four simulated vignettes depicting ICU patients with respiratory infection were presented. Each vignette used clinical and biological data (ie, WCC, CRP) to describe the patient’s trajectory after a course of antibiotics (table 1). The full scenarios can be found in online supplemental material 2.

Table 1

The four clinical vignettes used in this study

The scenarios were constructed following an iterative process of piloting and revision until saturation. Pilot participants (colleagues) were not eligible to participate in the study proper. Further details of the piloting process are provided in online supplemental material 3.

These vignettes were thought to accurately represent patient cases commonly seen in the ICU, and the varying degrees of diagnostic un/certainty encountered. Two of the vignettes (consistent clinico-biological improvement or worsening) functioned as controls, in that they clearly supported a decision to stop (improvement) or continue (worsening) antibiotics. The remaining two vignettes—hereafter termed disc clin better (clinical improvement/biological decline) and disc clin worse (clinical decline/biological improvement)—presented a greater diagnostic challenge, where the appropriate course of action was less clear. These discordant scenarios were intended to simulate situations in which there is equipoise in the inclination to stop/continue, because one of the factors key to decision making is deteriorating. The purpose of the two discordant scenarios was to explore the relative importance of clinical and lab-based (biological) trajectories in the inclination to stop antibiotics.

Each scenario offered a highly accurate infection-detecting PCR-based POCT with at least 95% sensitivity and specificity. This POCT provides rapid diagnostics for a named panel of bacteria and viruses. Whereas clinical and lab-based findings are surrogates for presence of infection, this POCT marks the actual presence of an infective organism. The POCT result was always negative (ie, implying no active lung infection). In one scenario (improvement), clinicians were subsequently told that the negative POCT result was erroneous (a laboratory error), and retesting gave a positive result. This explored the effect of a positive result on seemingly clear STOP judgements.

Procedures

Following informed consent, clinicians responded to all four vignettes (order randomised except for improvement; always presented last to preserve laboratory and POCT credibility). For each vignette (table 1), clinicians made an initial antibiotic decision (stop/continue), rated their confidence in this decision (1=not at all to 6=extremely confident) and selected reason/s for their decision (online supplemental material 3). Participants were then offered the POCT (yes/no), and selected reason/s for their choice (table 2).

Table 2

List of reasons presented to clinicians that accepted (left) versus rejected (right) the POCT Reasons were presented in a random order and clinicians could select as many as needed

Irrespective of their decision to perform the POCT or not, the result was presented (always negative). Clinicians were then asked to update their original antibiotic decision (stop/continue) and their confidence (range 1–6).

In the improvement scenario only (always presented last), clinicians were then informed of the laboratory error and presented with a new, positive POCT result; in response, they were asked to update their antibiotic decision and confidence.

After completing all four vignettes, participants completed Grol et al’s Attitudes to Risk-Taking in Medical Decision Making questionnaire,15 adapted to ICU (online supplemental material 4).

Statistical analysis

The study was powered to detect an effect of medium size (f2=0.15) in a two-tailed linear regression of final inclination-to-stop on patient trajectory, initial inclination-to-stop and in/voluntary POCT. Using G*Power V.3.1, 55 responses were sought to detect this effect, given power=80% and alpha=0.05. This sample size calculation was validated using pilot data (see online supplemental material 5).

Cluster-adjusted χ2 analysis was used to compare the proportion of clinicians that stopped antibiotics before vs after a negative POCT (hypothesis 1). For a more sensitive measure of the inclination to stop antibiotics, clinicians self-rated (scale 1–6) their initial and final confidence in accord with the corresponding choice; that is, positive (+) if the decision was to stop antibiotics and negative (−) if to continue. This returned a continuous measure of each clinician’s initial and final ‘inclination-to-stop’ (−6=minimal, 6=maximal).

Inclination-to-stop was compared before versus after the negative POCT result, using mixed-effects linear regression with a per-participant random intercept.

To test the effects of clinical trajectory (hypothesis 2), initial inclination-to-stop (hypothesis 3) and in/voluntary POCT (hypothesis 4) on final inclination-to-stop antibiotics, regression analysis was performed of final inclination-to-stop (−6 to 6) on patient trajectory (1=improvement, 2=disc clin better/disc clin worse, 3=worsening), initial inclination-to-stop (−6 to 6) and in/voluntary POCT (1=rejected, 2=requested), using mixed effects linear regression with a per-participant random intercept.

Risk inclination scores (the per-participant sum to Grol et al’s questionnaire) and level of experience (0=trainee, 1=consultant) were subsequently added to this model, to explore the effects of experience (hypothesis 5) and risk inclination (hypothesis 6) on final inclination-to-stop. Statistical analysis was performed using SPSS V.26 (IBM) and Stata/MP V.13.1 (StataCorp).

The manuscript adheres to The checklist for reporting result of internet e-surveys (CHERRIES) guidance for reporting e-survey results.16

Patient and public involvement

None.

Results

Demographic data

As participants were recruited via social media, an exact response rate ratio cannot be reliably calculated. However, 74 clinicians accessed the survey; of these, 4 (6%) did not complete a single patient scenario (their data were excluded from the analysis). Of the remaining 70 respondents, 62 (89%) completed all 4 scenarios, 2 (3%) completed 2 scenarios and 6 (8%) completed 1 scenario; yielding 258 scenario responses. Sample characteristics are displayed in online supplemental material 6.

The effect of a negative POCT on antibiotic stop decisions (hypothesis 1)

Prior to receipt of a negative POCT result, an antibiotic stop decision occurred 54% of the time (138/258; clinicians’ reasons are presented in online supplemental material 7). Following receipt of the negative POCT result, this increased to 70% (180/258; χ2(1)=25.82, p<0.001, w=0.32). This trend was consistent across scenarios, with varying magnitude (figure 1).

Figure 1

Number and % of clinicians that chose to STOP antibiotics, before and after a negative POCT result, per scenario. From left to right, the scenarios represent improvement (n=62 responses); disc clin better (n=66 responses); disc clin worse (n=65 responses); worsening (n=65 responses). *Difference in proportions (pre-POCT vs post-POCT) significant at p<0.01. POCT, point-of-care test.

In the improvement scenario (figure 1, panel 1), there was a high initial inclination to stop. The proportion of participants willing to stop antibiotics changed from 87% pre-POCT to 90% post-POCT (χ2(1)=0.61, p=0.437, w=0.09). Thus, a negative POCT did not change their inclination to stop antibiotics (ie, ceiling effect). In the disc clin better scenario (figure 1, panel 2), the initial proportion willing to stop was lower (73% pre-POCT) but the effect of the POCT on stop decisions was larger (88% post-POCT); this absolute increase of 15% was significant (χ2(1)=7.41, p=0.006, w=0.34). In the disc clin worse scenario (figure 1, panel 3), the initial proportion willing to stop was lower still (49% pre-POCT) and the effect of a negative POCT greater still (74% post-POCT): an absolute increase of 25% (χ2(1)=16.06, p<0.001, w=0.50). In the worsening scenario (figure 1, panel 4), the initial tendency to stop was expectedly lowest (6% pre-POCT) yet the effect of a negative POCT remained high (28% post-POCT): an absolute increase of 22% (χ2(1)=54.23, p<0.001, w=0.93). Even so, the proportion of participants willing to stop in this scenario remained a minority (28%).

Commonly selected reasons for stopping antibiotics were ‘continuing antibiotics is not clinically necessary based on the information provided’ (83%, 114/138) and AMR concerns (73%, 100/138). This did not vary substantially across scenarios (see online supplemental material 7). Commonly selected reasons for continuing antibiotics were ‘stopping is inappropriate based on the clinical information provided’ (77%, 92/120) and ‘disapproval from colleagues’ in the improving and discordant scenarios (56%–79%)—particularly disc clin worse (79%, 26/33), where it was the main reason for continuing (see online supplemental material 7).

Clinicians’ mean inclination-to-stop pre-POCT vs post-POCT, per scenario (figure 2) demonstrated that a negative POCT increased clinicians’ inclination-to-stop in all scenarios except improvement (b=0.69 (0.62–0.77), p<0.001). Again, the biggest change was observed in the disc clin worse scenario, where pre-POCT inclination-to-stop was statistically 0 (indicating equivalence), but post-POCT inclination-to stop-was reliably positive (favouring stopping).

Figure 2

Mean inclination-to-stop antibiotics, before versus after a negative POCT result, per scenario. POCT, point-of-care test.

The two discordant scenarios allowed exploration of the hierarchy of importance between clinical and biological data in antibiotic stop decisions. We note, therefore, that mean inclination-to-stop (both pre-POCT and post-POCT) was lower in the disc clin worse scenario (figure 2, panel 3) than the disc clin better scenario (figure 2, panel 2), suggesting that clinical factors were more influential than surrogate laboratory-based data. However, a negative POCT changed the influence of clinical deterioration on inclination-to-stop, from equivocal to favourable, and STOP proportions to a level comparable to a scenario of a clinical improvement (disc clin better).

After learning that the POCT result had changed from negative to positive (improvement only), the rate of stopping decreased significantly (56/62, 90% vs 38/62, 61%; χ2(1)=56.78, p<0.001, w=0.97), as did inclination-to-stop (M=4.19, SD=2.82 vs M=0.95, SD=4.35; t(61)=6.90, p<0.001, d=0.88).

The effect of patient trajectory, pre-POCT inclination-to-stop and in/voluntary POCT on STOP decisions (hypotheses 2–4)

POCT was requested 65% of the time (167/258), the proportions differing by scenario (χ2 (3)=33.80, p<0.001, w=0.36; improvement=39%, disc clin better=59%, disc clin worse=85%, worsening=75%) and by the initial antibiotics decision (χ2(1)=19.32, p<0.001, w=0.28; stop=52%, continue=79%). Figure 3 displays the number (proportion) of times that POCT was requested (green) versus rejected (red), per scenario and initial decision. POCT was requested most frequently in the two scenarios with clinical worsening (disc clin worse=85%, worsening=75%), especially in those who initially chose to continue (disc clin worse=94%, worsening=75%; figure 3).

Figure 3

Number and proportion of times that POCT was requested (green) versus rejected (red) by those who initially elected to stop (left panel) versus continue (right panel) antibiotics, per scenario. POCT, point-of-care test

As hypothesised, final inclination-to-stop antibiotics was a function of the patient’s trajectory: clinicians were significantly less inclined to stop antibiotics (despite receiving a negative POCT) when the patient’s trajectory was ambiguous (discordant scenarios) or worsening, as opposed to improving (b=−0.73 (–1.33, –0.14), p=0.015). Final inclination-to-stop was also a function of the clinician’s initial leaning, with high (low) inclination-to-stop pre-POCT predicting high (low) inclination-to-stop post-POCT (b=0.66 (0.56, 0.76), p<0.001). Finally, clinicians that actively requested (vs passively received) POCT results were significantly more inclined to stop (b=1.30 (0.58, 2.02), p<0.001). Indeed, of those who initially elected to ‘continue’ and requested the POCT (n=95), 43% changed their minds (ie, switched to stop, n=41); of those who initially elected to ‘continue’ and did not want the POCT (n=25), only 8% changed their minds (n=2). As such, the vast majority of those who changed their minds (from ‘continue’ to ‘stop’) had requested the POCT (95%, 41/43).

Variability in clinicians’ judgements (ie, SD) was reduced by negative POCT results in all but one scenario (worsening) (see SM8).

Reasons for requesting/rejecting POCT are presented in online supplemental material 8. The most common indication for requesting POCT was ‘to supplement my clinical judgement’ (95%, 159/167), while the most common indication for rejecting POCT was ‘the test is unnecessary in this case’ (81%, 74/91).

Effect of clinician characteristics on inclination-to-stop (hypotheses 5–6)

The clinician’s grade (trainee or consultant) did not influence inclination-to-stop antibiotics (b=0.15 (−0.63, 0.93), p=0.699), nor did attitudes towards risk-taking (b=−0.01 (−0.12, 0.10), p=0.872; see also SM10online supplemental material 10). However, the study was not powered to detect these effects; therefore, we cannot confidently rule out the possible influence of seniority and/or risk appetite.

Finally, clinicians were deliberately not given the opportunity to de-escalate (rather than stop) antibiotics. Recognising this potential limitation, we asked clinicians whether they would have de-escalated, had the option been available. Most clinicians said yes (74%, 52/70), a minority said no (14%) and 11% did not answer.

Discussion

This prospective observational study determined the effect of infection PCR POCT results on antibiotic stop decisions in ICU. Thus, when the availability of a PCR-based POCT result suggested the absence of infection. Further, the study determined the effect cognitive-behavioural factors had on these decisions. Other studies have identified cognitive-behavioural drivers of stop decisions,3 and influential factors in such decisions. None has quantitatively determined the amount to which they influence the proportion of stop decisions, nor the degree to which that decision may change in scenarios where the clinical and biological biomarker trajectories were conflicting.

What is the effect of a negative POCT on antibiotic stop decisions?

Having a negative POCT result significantly increased clinicians’ inclination to stop antibiotics. A negative POCT did not trigger stop decisions indiscriminately; rather, it appeared to operate as one input to clinicians’ decisions, increasing their inclination to stop by degree. A negative POCT thus appears to operate as a prompt or nudge for antibiotic stop decisions.

The efficacy of this nudge—that is, the extent to which it brings about stop decisions—varied by scenario.17 18 It was most striking in an ambiguous scenario featuring clinical deterioration but biological improvement (disc clin worse), where a negative POCT result shifted the majority decision from ‘uncertain’ to ‘stop’ (49% vs 74% stopped). Indeed, the post-POCT stop rate in this scenario began to approach the pre-POCT stop rate in scenarios featuring clinical improvement (improvement, disc clin better), suggesting that a negative POCT led clinicians to reinterpret clinical deterioration. A psychological explanation for this is the ‘plausible coherence’ hypothesis; the human mind tries to resolve conflicting information into an acceptable story.7 A negative POCT result may have enabled this; clinicians with uncertainty about stopping may have been ‘nudged along’ to a comfortable stop decision by a negative POCT. This was not however sufficient when the patient’s clinical and biological trajectory were declining (the worsening scenario): while a negative POCT result did increase stopping significantly in this scenario, the majority decision was ultimately to continue. In this scenario, it would seem implausible to argue against the clinico-biological data. POCT may therefore be most useful as an influencing strategy in ambiguous scenarios.

Why do clinicians not stop antibiotics when POCT data suggests that they can?

As hypothesised, three factors were found to ‘compete’ with (ie, reduce or moderate) the effect of a negative POCT on antibiotic stop decisions. The first was the patient’s clinico-biological trajectory. As the trajectory deteriorated, so too did clinicians’ inclination to stop, despite receipt of a negative POCT. This speaks to plausible coherence; POCT is merely one input to clinicians’ decisions and will be considered in light of all the available data. Interestingly, we observed a hierarchy in this ‘available data’: clinicians were more inclined to stop antibiotics when there was clinical improvement with biological worsening (disc clin better) versus clinical worsening with biological improvement (disc clin worse). This confirms that clinical factors are more influential than surrogate laboratory-based data,19 and reflects the historical evolution of clinical diagnosis. Laboratory-based findings were only incorporated into clinical practice later, in William Osler’s era.20

The second factor ‘competing’ with a negative POCT was strength of initial inclination. This determined the final inclination to stop, irrespective of a negative POCT.21 High confidence in the initial decision to continue antibiotics might not be swayed by a negative POCT. This could explain the failure of negative POCT to increase stopping in trials such as VAP-rapid.13

The third ‘competing’ factor might be termed ‘disinterest’ in POCT. Clinicians who actively requested the POCT result were significantly more likely to act on its results (ie, stop antibiotics) than those who involuntarily received it. There are several reasons why clinicians might decline a POCT. The most commonly selected reason in the present study was that ‘the test is unnecessary in this case’, indicating conviction in their decision. Prior work has recognised lack of trust in the validity of POCT among clinicians22 23; presently, only a minority of POCT-rejectors indicated lack of trust, but this may be due to our deliberate presentation of the POCT as reliable. Whatever clinicians’ reasons for rejecting POCT, unregulated POCT without appropriate guidance will reduce its utility. The most promising route forward might be a combined approach, including POCT prompts (eg, algorithmic triggers to flag cases where POCT is most useful and nudge POCT requests) as well as active attempts to increase clinicians’ appreciation of POCT as potentially influential in antibiotic stop decision-making. Future work might also identify the factors that influence clinicians’ inclination/decision to request more information—be they clinical (eg, patient trajectory), organisational (eg, hospital culture) or individual (eg, confidence, open-mindedness).

Inclination for POCT may be the result of uncertainty (‘I’m not sure what to do, I need more information’) or affirmation of a draft judgement (‘I think there is [not] an infection, but I want proof’). Presently, POCT was most requested and apparently most useful in situations of clinical deterioration (disc clin worse, worsening), where the inclination to stop after a completed course may be hindered by concerns about the possibility of ongoing and unidentified infection. High demand for POCT in the worsening scenario is interesting and could represent an attempt to seek new information on suspected bacteria, or to counter the proposition that a clinical deterioration was due to infection, to inform the antibiotic stop decision better. The necessity concerns framework details the benefit–risk relationship to treatment decision.24 In a previous study (also using vignette-based interviews about nosocomial infection), ICU clinicians described ‘erring on the side of caution’; clinicians viewed antibiotics’ necessity (ie, protection for their patients and themselves) as outweighing concerns about antibiotic toxicity and AMR.25

Do clinician characteristics (experience and risk-taking) influence antibiotic stop decision-making?

We found no evidence to suggest that clinician experience (trainee to consultant) or risk averseness influenced willingness to stop antibiotics.15 However, the role of experience and expertise warrants further investigation. As the study was not powered to detect a difference in antibiotic decisions related to clinician experience, it is possible that a true difference may have been missed in this study. Future work might also investigate the role of systemic factors such as hospital/ward culture and baseline prevalence of infection.

Strengths and limitations

A negative POCT result facilitated antibiotic stop decisions; there lies justification for the use of POCT. However, there remains the question of whether this simulated effect can be used to effect change in real life (eg, VAPrapid). Perhaps it is too easy to stop antibiotics in a hypothetical patient scenario and certain factors are given disproportionate weight. Furthermore, it is unlikely that all influencing factors from this study are made explicit and/or considered simultaneously at the point of decision in reality. Our vignettes were simplistic and may have omitted other relevant factors. The model’s simplicity was intentional; only through careful control of the scenarios’ components (clinical and biological) could we assess their effects on stop decision making; indeed, the predictive accuracy of simple models is usually reduced by adding complexity.26 The present findings would benefit from replication in the context of (a) more detailed vignettes and (b) simulated clinical settings.

Relatedly, we did not offer participants the option to change/de-escalate antibiotics (lest it become the ‘safest’ and therefore default option—particularly in ambiguous scenarios). We are aware that this limits the generalisability of our findings. It could also explain some of our results, such as the substantial shift towards ‘stop’ following a negative POCT in the worsening scenario: it is possible that participants were not electing to stop per se, but rather trying to express that they would not continue along the same path. Our study also may not have fully accounted for consideration of a non-pulmonary cause to explain clinical worsening, that is, (1) declining the POCT (due to the belief that lungs that are not the issue) and/or (2) requesting the POCT in the worsening scenario (to determine bacteria that may influence a change/escalation-decision rather than a stop-decision). Follow-up studies should manipulate the likelihood of alternative sources of infection and allow for de-escalation/escalation.

The present study used a convenience sampling method. We cannot exclude the possibility that a different (ie, probabilistic) form of sampling may have returned different results. However, participants were recruited via a wide range of technological portals spanning numerous hospitals/ICUs and regions that had predominantly general medicosurgical ICUs. Moreover, our vignettes described common patient presentations that most (if not all) clinicians would encounter as part of routine practice, irrespective of region or ICU type (eg, general or specialist).

Finally, we chose not to examine the effect of a positive POCT result (suggesting infection), presuming that it would inhibit stop decisions (the very phenomenon that we wished to study).

With that, the present work helps to specify conditions for targeted and effective use of POCT as part of ASP strategies. In so doing, it may shed light on the low success of VAPrapid and similar initiatives. It identifies specific situations in which POCT might be most gainfully used, which could inform algorithms and/or guidelines as to selected deployment of POCT for maximum advantage.

Conclusion

This is the first study to generate and quantitatively test hypotheses regarding specific cognitive-behavioural factors that might influence antibiotic stop decisions, using quantitative experimental methods. A negative POCT result increased clinicians’ inclination to stop antibiotics in most scenarios; there lies justification for the use of POCT. However, clinical deterioration, an intuitive unwillingness to stop, and disinterest in POCT (ie, failure to request) reduced its effect. We conclude that judicious use of POCT to reduce uncertainty can improve antibiotic STOP decisions in those prepared to request it. These and other identified behavioural factors must be signposted in ASPs, with the necessary prompts. Such work might inform future ASP study.

Data availability statement

Data are available in a public, open access repository. The data are publicly available in The Open Science Framework: https://osf.io/bgp84/.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and the study was approved by the Imperial College Research Ethics Committee (ref 201C6499). Participants gave informed consent to participate in the study before taking part.

Acknowledgments

All clinicians who agreed to participate in this study are acknowledged with appreciation, in particular those who assisted in the piloting of vignettes: Dr James Mcintee, Dr Frederick Hill, Dr Kaladheran Abogantaen. Members of the WHY STOP Consortium are also acknowledged for providing valuable comments on the manuscript: Abbas, Madiha; Alger, Laura; Cabaret, Cyrille; Chana, Sanjeet; Christie, Linsey Emma; Daly, Stephen; Davies, Roger; El Ghazali, Sally; Erridge, Simon; Gallagher, Susan; Hayes, Michelle; Hewitt, Sam; Hill, Alfred; Hill, Frederick; Imtiaz, Sana; Jabble, Navdeep; Jhanji, Shaman; Kam, Elisa; Keays, Richard; Khaladeran, Agbontaen; Knudsen, Rasmus; Kotecha, Shaan; Lau, Timothy; Lees, Nicholas J; Little, James; Mandrekar, Ameya; Mason, Sonia; Mehta, Sachin; Millette, Ben; Misri, Sahil; Moore, Luke; Mughal, Nabeela; Muhammad, Saad; Nunn, Lois; Nurek, Martine; Patel, Manan; Poon, Sharon; Popescu, Monica; Porter, John; Rostron, Anthony; Siddiqui, Aliya; Singh, Suveer; Singhal, Archit; Sisson, Alice; Sokhi, Jagdish; Soni, Sanooj; Strul, Jo-Anne; Tausan, Matija; Trainer, Charlotte; Trainer, Harris; Valliani, Dilshad; Vinnakota, Krishna; Vizcaychipi, Marcela; Walton, Victoria; Woods, David.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @respitudoc, @mvizcayc

  • SS and MN contributed equally.

  • Contributors SS conceived the idea. SS, MN, SM and MPV developed the protocol. SM and MN developed the data collection tool. SM created the study website. MN analysed the data. SS, MN, SM, LSPM and NM interpreted the results. SS wrote the first and subsequent drafts with MN. SS, MN, SM, MPV and WHY STOP Consortium members reviewed the manuscript. SS is guarantor responsible for the overall content of the study.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.