Article Text
PDF
Abstract
Objective To evaluate service use, clinical outcomes and user experience related to telephone-based digital triage in urgent care.
Design Systematic review and narrative synthesis.
Data sources Medline, Embase, CINAHL, Web of Science and Scopus were searched for literature published between 1 March 2000 and 1 April 2020.
Eligibility criteria for selecting studies Studies of any design investigating patterns of triage advice, wider service use, clinical outcomes and user experience relating to telephone based digital triage in urgent care.
Data extraction and synthesis Two reviewers extracted data and conducted quality assessments using the mixed methods appraisal tool. Narrative synthesis was used to analyse findings.
Results Thirty-one studies were included, with the majority being UK based; most investigated nurse-led digital triage (n=26). Eight evaluated the impact on wider healthcare service use following digital triage implementation, typically reporting reduction or no change in service use. Six investigated patient level service use, showing mixed findings relating to patients’ adherence with triage advice. Evaluation of clinical outcomes was limited. Four studies reported on hospitalisation rates of digitally triaged patients and highlighted potential triage errors where patients appeared to have not been given sufficiently high urgency advice. Overall, service users reported high levels of satisfaction, in studies of both clinician and non-clinician led digital triage, but with some dissatisfaction over the relevance and number of triage questions.
Conclusions Further research is needed into patient level service use, including patients’ adherence with triage advice and how this influences subsequent use of services. Further evaluation of clinical outcomes using larger datasets and comparison of different digital triage systems is needed to explore consistency and safety. The safety and effectiveness of non-clinician led digital triage also needs evaluation. Such evidence should contribute to improvement of digital triage tools and service delivery.
PROSPERO registration number CRD42020178500.
- health services administration & management
- quality in health care
- organisation of health services
- qualitative research
- public health
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Relevant data are included in online supplemental tables.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- health services administration & management
- quality in health care
- organisation of health services
- qualitative research
- public health
Strengths and limitations of this study
This is the first systematic review to focus on the use of telephone based digital triage in urgent care.
This comprehensive, mixed-methods review covers a 20-year period, enabling evaluation of older literature prior to shifts of some services to non-clinician led models of service delivery.
Outcomes relating to cost-effectiveness, and staff focused outcomes were not within the review scope.
The review was limited to studies published in English, which may have led to some evidence being overlooked.
Background
Telephone based digital triage is widely used in urgent care.1 2 Urgent care is the ‘the range of responses that health and care services provide to people who require—or who perceive the need for—urgent advice, treatment or diagnosis’,3 and includes national or regional help-lines, out of hours centres and emergency care providers.
Digital triage involves a call handler or clinician using a digital triage tool to generate advice based on an assessment of a patient’s symptoms. Advice typically takes the form of signposting within defined levels of urgency to specific local services, such as an emergency department (ED), out of hours centre or general practice (GP) appointment; in some cases self-care advice is given.
Digital triage service delivery models vary widely. In England and Scotland digital triage is delivered by non-clinical call handlers, for example, through the 111 service, which operates 24/7, while in most other countries it is predominantly clinician (nurse) led.4–9 In part, digital triage has been implemented in response to increasing demand on primary care and EDs in the last several decades.10
Despite wide adoption over the last several decades, there is limited evaluation of its impact on wider healthcare service use, clinical outcomes and user experience. No previous systematic reviews have focused solely on services that use digital triage; instead reviewing telephone consultation and triage more broadly, including services that use digital triage and those that are not digitally supported.1 10 11
One review indicated that 50% of calls in the general healthcare setting (with studies predominantly conducted in primary care settings) could be handled completely over the telephone, showing the potential of telephone triage to manage face to face care demand.10 However, there are mixed findings relating to wider healthcare service use and very limited investigation of clinical outcomes.10 A previous review reported a high level of user satisfaction,10 while another highlighted that satisfaction with advice related to improved compliance with advice.11
Given technological development and, in some cases, the reorganisation of services in recent years,2 systematic reviews conducted several years ago (between 2005 and 2012)1 10–13 may have limited relevance to today’s services.
This review addresses the need for an up-to date evaluation of telephone-based digital triage within urgent care. It aims to evaluate wider healthcare service use, clinical outcomes and user experience in a range of in hours and out of hours urgent care settings in order to identify areas for improvement and the need for further research.
Method
This review uses a mixed-methods design and is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework.14 See online supplemental appendix 1 for the PRISMA checklist.15
Supplemental material
Patient and public involvement
No patient and public involvement (PPI) directly fed into the development or conduct of this review.
Eligibility criteria
Eligibility criteria have been developed using the population, interventions, comparators, outcomes and study designs principle16:
Population: studies that evaluated digital triage in the general population or within population subgroups (eg, older people).
Interventions: studies that assessed telephone based digital triage, which met all of the below criteria:
In services providing urgent care (excluding in-hours GP)
That was used by the general population (not condition specific services).
That result in signposting advice (referral to a local service, such as ED, GP, ambulance dispatch and in some cases self-care advice).
Outcomes: studies that evaluated at least one of the following: characteristics of service users and triage advice; healthcare service use following triage; clinical outcomes (including hospitalisations and mortality) and service user experience.
All empirical study types published between 1 March 2000 and 1 April 2020 in English were included: qualitative, quantitative and mixed-methods studies.
Search strategy
The search strategy was designed with support from a librarian. Searches were conducted in Medline, Embase, CINAHL, Web of Science and Scopus. Terms relating to digital triage and urgent care settings (excluding in-hours GP) were used. See Medline search terms in online supplemental appendix 2. The search was restricted to studies published in English, including electronically published (Epub) studies ahead of print. Reference handsearches were conducted for all included full texts.
Supplemental material
Study selection and data extraction
Articles were deduplicated ahead of study selection. Two reviewers screened studies independently at title and abstract stage and at full text stage using Covidence software. Any disagreements were resolved through discussion between the reviewers; where necessary a third reviewer was consulted. A PRISMA flow chart was is presented in the results.
A data extraction form was developed and initially piloted on three studies to confirm that key elements of studies were captured. See online supplemental appendix 3 for data extraction fields. Data were extracted independently by two reviewers, and any discrepancies were resolved through discussion with a third reviewer. Study authors were contacted in cases where clarifications regarding study conduct were required.
Supplemental material
Quality assessment
Quality assessment, including risk of bias, was conducted by two reviewers using the mixed methods appraisal tool (MMAT),17 which enables the assessment of mixed study types. The assessment was used to provide context, rather than to exclude studies.18 Based on the number of MMAT criteria met, studies were categorised as high (if all five MMAT criteria were met), medium (if three or four criteria were met) or low quality (if two or less criteria were met).
Data synthesis
Narrative synthesis18 was used due to the diversity of designs in the included studies. This included: generating a preliminary synthesis, exploring relationships in findings across studies, assessing the robustness of the evidence and summarising findings.18 Statistical meta-analysis was not possible due to the heterogeneity of the included studies. Key findings within and between studies were grouped by outcome and visually summarised using a subgroup analyses method,18 which we modified to additionally present the strength of evidence. Where a visual summary was not possible due to heterogeneity of outcomes, findings were summarised in text.
Results
The search resulted in 6921 records, after duplicates were removed, there were 5010 records to screen at title and abstract level; 102 records were included for full-text screening, out of which 31 studies were included. See figure 1 for PRISMA flow chart.
PRISMA flow chart. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Most included studies were of quantitative design (n=25)5 7 19–41 including: routine data analyses(n=16),5 7 19–25 27 29 34 35 37–39 surveys(n=6),26 28 31 33 40 41 controlled trials (n=2)30 36 and a quantitative descriptive study (n=1).32 There were fewer qualitative (n=4)42–45 and mixed-methods studies (n=2).6 46
Studies were mainly from the UK (n=17),5 6 20 21 23 26–29 32 36–38 40 42 43 46 with small numbers from Sweden (n=4),41 44 45 47 Australia (n=4),30 31 34 39 USA (n=3),7 19 22 Netherlands (n=2),25 33 Japan (n=1)35 and Portugal (n=1).24 Most included the full range of service users (n=24),5 6 19 21–26 28 30 32–36 38–41 43–46 but some focused on subsets: older adults,21 24 younger age groups,20 37 parents of children,31 men42 or adults with limited English proficiency (LEP).7
Most studies evaluated digital triage conducted by nurses (n=26),5 7 19–34 37 39 41–46 but some included non-clinicians (n=3),6 38 40 nurses and paramedics (n=1)36 or nurses and non-clinical call handler (n=1).35
Most studies were of identifiable call centre-based services: England’s former National Health Service (NHS) Direct20 21 23 26 28 29 37 42–44 46 and current NHS 111 service,38 40 Scotland’s NHS24,5 6 USA’s MayoClinic,7 19 22 Portugal’s Linha Saude 24,24 Swedish Health Direct,41 44 45 Australia’s Health Direct.34 A few involved smaller scale ‘unnamed’ implementations30 39 or GP cooperatives.25 32 33 Two were based in the emergency setting, one within an English ambulance service36 and one within an emergency telephone service in Japan.35 Table 1 shows characteristics of studies.
Characteristics of included studies (31 studies)
Nineteen studies were rated as being of high quality,5–7 21 23–26 29 33 34 36–39 42–45 11 medium19 20 22 27 28 30–32 35 40 41 and 1 was low.46 Qualitative studies tended to be of higher quality, while quantitative studies were more variable. Reasons for lower quality among quantitative studies included inadequate description of accounting for confounders28 30 34 35 and risk of non-response bias.31 32 40 41 One mixed-methods study did not adequately describe integration of qualitative and quantitative components.46 In two of the qualitative studies details about how the findings were derived from the data could have been expanded.43 45 The quality assessment results are included in online supplemental appendix 4.
Supplemental material
Patterns of use
Nine studies focused on patterns of triage advice; all used routine datasets.5 7 19–25 Key findings are summarised below; detailed findings from studies are in online supplemental table 1.
Supplemental material
Characteristics of patients and callers
Presenting symptoms with highest frequency among patients, included: abdominal or digestive problems, 6.8%–12.2% of calls5 19 22 24 39 and respiratory problems, 11.3%–11.9%39 ,24 of calls. The majority of calls were made by women (range: 59%–72%).5 19 22–24 39
Calls about patients in younger age groups22 23 made up a comparatively high proportions of calls; 24% of calls were for 0–5 years in one study23 and another reported 15% of out of hours calls being for 0–4 years.5
User characteristics and triage advice urgency
Factors associated with triage advice urgency included:
Patient’s age: Two studies reported urgency to be lower in children and younger age groups23 ,20, one study reported a high proportion (47%) of calls about children aged (0–15) were resolved through self-care advice or health information.20 Two studies reported that urgency increased with age.19 24
Sex: Two studies reported women were more likely to receive lower urgency advice as compared with men; however, neither controlled for age or presenting symptoms,21 23 one suggested this may be explained by women seeking care advice earlier, before their symptoms progress and become more urgent.21
Symptoms: Two studies reported symptoms associated with higher urgency advice20 25; for example, calls about children with respiratory problems were more likely to be referred to emergency care as compared with other symptom types.20
Caller language proficiency: One case–control study reported that adults with LEP were more likely to receive higher urgency advice (ambulance, immediate ED attendance or urgent visit) (49.4% vs 39.0%; p<0.0004)7; groups in this study were balanced based on age and sex and comorbidities were controlled for.7
Service use and clinical outcomes following triage
Change in service use following digital triage implementation
Eight studies reported on change in wider healthcare service use (primary care, ED use, ambulance use and emergency admissions) following implementation of digital triage.28–30 32 35 36 38 46 Of these, one investigated non-clinician led triage.38 Comparators included: rates of service use in patients receiving usual care (eg, GP referral) in comparison to those who were digitally triaged32 36; service use rates prior to implementation28 30 35 46; comparator regions with no digital triage implementation29 38 and national service use comparator.30
There were mixed findings across studies, as visually summarised in figure 2. Most reported reduction or no change in wider service use after implementation; there were two exceptions, which both evaluated clinician (nurse) led digital triage: one (rated as being a lower quality study) reported an increase in ED use.46 The other reported some increase in out of hours service use (GP clinic use and home visits) related to ‘standalone’ digital triage call centres in comparison to national comparator; however, this study differed to the other studies as it utilised household surveys to capture service use.30
Findings from studies of out of hours (OOH) service use after digital triage implementation. ED, emergency department.
Online supplemental table 2 presents detailed findings from studies.
Supplemental material
Patient level service use and adherence with advice
Six studies reported varying patient adherence to triage advice through evaluation of patients’ subsequent ED attendance.26 27 31 34 37 39 Four used routine data and data linkage with sample sizes ranging from: 3312 to 13 019 triage calls. Of these, three studies reported 60%–70% of patients who were advised to attend ED followed this advice27 34 37 ; one reported a range of 29%–69%, with higher compliance when ambulance was advised (53%–69%) and lowest compliance when self-transport to ED was recommended (29%).37
One small survey of 268 callers reported high levels of adherence with advice to attend ED (96%; 49 of 51 calls), to contact a GP (92%; 133 of 144) and to self care (93%; 64 of 69).26
Four studies reported proportions of patients who attended ED after receiving alternative triage advice (other than attending ED): 2.4%,27 9%34 37 and 22%.31 The latter included 51 of 1150 parents who had remained worried after calling the digital triage service.31 Results are showed in online supplemental table 3.
Supplemental material
Safety
Four studies highlighted potential triage errors based on hospital admission rates.27 34 36 37 These mainly related to potential ‘undertriage’, where the advice was considered to be at too low a level of urgency in relation to clinical need. However, these findings were peripheral to the main aims of these studies.27 34 36 37
One study reported similar hospitalisation rates between patients attending ED who had been directed to ‘immediate or prompt’ care and ‘non-urgent’ care: immediate or prompt: 38%(n=261), 95% CI 34 to 41 vs non-urgent: 37% (n=56), 95% CI 30 to 44).34 Another reported 15% (n=71) of paediatric cases attending ED after being triaged were admitted; of these, 37 had been advised to attend ED and 34 were given other lower urgency advice.37
Another study reported 15% (n=15) of patients given advice that was lower urgency than ED attendance, (such as urgent or routine GP appointment or self care), attended ED following their triage call and were admitted.27 One study reported 9.2% (n=30) of patients triaged as not requiring ambulance dispatch were subsequently admitted.27 36
One qualitative study described users reporting not having received appropriate triage advice for symptoms which later turned out to be more serious.44
Service user experience
Seven studies focused on user experience and satisfaction.6 40–45 Three studies reported a high level of satisfaction among users.6 31 40 Two studies reported higher satisfaction among those who received higher urgency advice.40 41 Two studies reported dissatisfaction relating to the relevance and number of triage questions.6 40Three studies highlighted that callers felt they needed to be assertive in order to receive the expected care advice.42 44 45 For example, a user’s post to an online forum:
If you need help and advice you can always call the healthcare advice line, if you think they’re giving you the ‘wrong’ advice, tell them, and maybe you’ll get better help.44
Two studies reported that users felt that the nurses using digital triage gave them time, conducted ‘thorough’ assessments and felt reassured.43 45
In contrast, one study of users who posted to an online forum reported feeling scrutinised by the nurses questioning their symptoms and need for care.44 Some expressed doubts about nurses’ advice, competency and credibility.44
Integrated services made for a smoother patient care journey. One study based on an online forum described the experience of poor integration:
They send you to the ER where they yell at you for being stupid enough to listen to them (SHD). SHD is a big problem and seems to be at war with the ER.44
In contrast, there was high satisfaction in 71%, of users where the service provider was able to book an appointment at a local service on behalf of the patient.40
See figure 3 for a visual summary of findings across studies and table 2 for detailed findings.
Findings from studies that investigated user experience and satisfaction
Key themes from studies of user experience.
Discussion
This systematic review has evaluated the evidence on how telephone-based digital triage affects wider healthcare service use, clinical outcomes and user experience in urgent care. Thirty-one studies were included, covering a range of different designs, settings, populations and digital triage systems. Studies typically showed no change or a reduction in wider healthcare service use following the implementation of digital triage. They reported varied levels of caller adherence to the triage advice provided. There was very limited evidence on clinical outcomes; however four studies reported some findings on hospitalisation rates that highlighted potential safety concerns relating to under-triage.
Overall user satisfaction with telephone based digital triage appears to be high, but there was some evidence of poorer user experience relating to the length and relevance of triage questioning, and perceptions of ‘undertriage’. Users sometimes felt the need for assertiveness during calls when their expectations were not being met; however, this is unlikely to be specific to digital triage and has been reported in telephone-based consultation more widely.48
There was considerable heterogeneity across studies in terms of types of setting, types of participants, study designs and ‘digital triage’ systems. ‘Digital triage’ is a complex intervention with outcomes that may be influenced by multiple factors due to varying healthcare systems, local service configuration, staff training and an evolving landscape in the use of digital technologies to allow patients to seek urgent care, for example, through the use of digital self-triage tools. Hence, there needs to be caution in the interpretation of the applicability of findings. Additionally, strength of evidence differed between studies, as demonstrated by the visual tables of key findings; these differences fed into the narrative synthesis of this review.
Many of the studies that investigated service use following digital triage implementation reported no change in wider healthcare service use. In one context, for example, following the replacement of a nurse-led service with a non-clinician led service this may be seen as a success,38 but this may not be applicable to all healthcare settings. One study of ‘standalone’ digital triage implementation showed an increase in GP clinic use,30 which was in contrast to other studies in this review; this may be because this service was less embedded within the healthcare system, but could also have been a methodological consequence of using household surveys to gather service use data.30
Strengths and limitations
This is the first systematic review to focus on the use of telephone based digital triage in urgent care. It covered a 20-year period, during which some services have started to shift towards non-clinician-led models of service delivery. This review enabled evaluation of a broad range of service models and settings. However, it was limited to studies published in English, and this may have led to important evidence being overlooked.
This review used a comprehensive mixed-methods approach and evaluated quality of studies using the MMAT tool. While this tool worked well for many studies in this review, an acknowledged limitation49 is the applicability of its criteria for assessing studies that are cross-sectional in nature (where there are not necessarily defined groups with an intervention or exposure); this is applicable to some of the studies included in this review
There was limited evaluation of non-clinician led models of digital triage, with only one study evaluating service use following implementation and no studies of clinical outcomes. Another limitation is the scope of the included outcomes; outcomes relating to broad utilisation of services that use digital triage (such as call volumes, call lengths and caller characteristics alone), cost-effectiveness and staff focused outcomes were not covered.
While PPI did not directly feed into this review, this forms the first stage of a wider project investigating user outcomes related to digital triage. For the wider project, has been sought in the project design, and a panel has been selected to aid the interpretation of results and dissemination of findings.
Comparison with other literature
This review’s focus is narrower, in terms of intervention and setting, compared with previous reviews which evaluated telephone triage more broadly, including services that were not digitally supported.1 10 Bunn et al’s review evaluated telephone triage in comparison to usual care.10 They similarly reported no significant change in wider healthcare use (ED visits, routine GP visits and hospitalisations) associated with telephone triage. Other reviews found that user satisfaction is generally high when comparing telephone consultation with other forms of care,10 but lower satisfaction was described when patients’ initial expectations were not met.48
Our review highlights the limited evaluation of clinical outcomes. A previous review of telephone triage reported limited and inconclusive findings on mortality rates (with no mortalities occurring in some studies that sought to investigate this outcome), and rates of undertriage and subsequent hospitalisation ranging from 0.2% to 5.25%.1
Although our review did not include broad utilisation outcomes related to digital triage, a previous study reported lower than expected use by some ethnic minority groups.50 Our review found that no studies to date have reported on patterns of advice, user experience, service use or clinical outcomes in ethnic minority groups; this may have been limited by our exclusion of studies that were not published in English.
We found that patients’ adherence with advice varied by setting and study design. While very high adherence was reported in one survey based study,26 this may be an overestimate due to response bias in comparison to other studies that evaluated adherence based on routine data. Similar observations in higher adherence rates in self-reported service use were reported by two reviews.11 13
Implications for service delivery and future research
The review has identified several gaps in the literature, particularly a need for evaluation of patient level service use and clinical outcomes. Further analysis of large patient level datasets (particularly those that are linked with subsequent service use and clinical outcomes data) will help to gain a better understanding of who does and does not adhere to advice and help to evaluate safety concerns relating to under triage within particular patient subgroups.
In the absence of comparative studies, it is unclear how patient satisfaction and outcomes are affected by the design of services, the staff groups involved and how they are trained and managed, and the type of digital triage system deployed. Further evaluation of non-clinician led digital triage may help policy-makers and service commissioners to adopt the most efficient and safe digital triage systems.
While not a key aim, this review highlights that associations between factors (such as age, gender, ethnicity) and urgency of advice have not been explored in depth. The granular demographic and symptom data captured by digital triage tools gives opportunity to explore these associations which will likely provide insight into how services are used by different groups and form the basis for generating hypotheses within particular groups.
Many studies in this review were undertaken when digital triage was first being implemented. However, like any significant service change, digital triage services will take a significant period of time to become established and performing optimally within urgent care services that have been used to working in another way. To date, no studies have involved longitudinal data collection to evidence the extent to which this occurs. Longer-term evaluation studies are needed to explore how the safety and effectiveness of services changes over time. In addition, telephone-based approaches to seeking care have been critical during the COVID-19 pandemic and are likely to be more widely adopted in the long term51; therefore, evaluation of how these services have functioned during and after the pressures of a pandemic is also important.
Lastly, this review highlights limited qualitative and mixed-methods approaches to date. Integrating findings from routine data with qualitative research will help to better understand user experiences and care needs of particular patients groups in more depth. These could feed into targeted support for these groups within or outside of digital triage services, and ultimately improved delivery of these services which are key to a well functioning healthcare system.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Relevant data are included in online supplemental tables.
Ethics statements
Patient consent for publication
Acknowledgments
The authors would like to thank Samantha Johnson (Academic Support Librarian, University of Warwick) for support with developing the search strategy. Patients and or public were not involved directly in the conduct of this review.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @h_atherton
Contributors VS developed the review protocol, with the support of HA and JD. VS conducted searches. VS, CB, ES and JWNB conducted screening, data extraction and quality assessment. VS conducted the narrative synthesis with support from CB and HA. HA and JD reviewed and revised manuscript and approved the final version. VS is the guarantor for the review.
Funding This systematic review is part of a PhD that is funded through University of Warwick in collaboration with an industrial partner: Advanced (https://www.oneadvanced.com/).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.