Article Text
PDF
Abstract
Objectives Assess the impact of text-based electronic notifications on improving clinic attendance, in relation to study quality (according to risk of bias), and to assess simple ways in which notifications can be optimised (ie, impact of multiple notifications).
Design Systematic review, study quality appraisal assessing risk of bias, data synthesised in meta-analyses.
Data sources MEDLINE, EMBASE, PsycINFO, Web of Science and Cochrane Database of Systematic Reviews (01.01.05 until 25.4.15). A systematic search to discover all studies containing quantitative data for synthesis into meta-analyses.
Eligibility criteria Studies examining the effect of text-based electronic notifications on prescheduled appointment attendance in healthcare settings. Primary analysis included experimental studies where randomisation was used to define allocation to intervention and where a control group consisting of ‘no reminders’ was used. Secondary meta-analysis included studies comparing text reminders with voice reminders. Studies lacking sufficient information for inclusion (after attempting to contact study authors) were excluded.
Outcome measures Primary outcomes were rate of attendance/non-attendance at healthcare appointments. Secondary outcome was rate of rescheduled and cancelled appointments.
Results 26 articles were included. 21 included in the primary meta-analysis (8345 patients receiving electronic text notifications, 7731 patients receiving no notifications). Studies were included from Europe (9), Asia (7), Africa (2), Australia (2) and America (1). Patients who received notifications were 23% more likely to attend clinic than those who received no notification (risk ratio=1.23, 67% vs 54%). Those receiving notifications were 25% less likely to ‘no show’ for appointments (risk ratio=.75, 15% vs 21%). Results were similar when accounting for risk of bias, region and publication year. Multiple notifications were significantly more effective at improving attendance than single notifications. Voice notifications appeared more effective than text notifications at improving attendance.
Conclusions Electronic text notifications improve attendance and reduce no shows across healthcare settings. Sending multiple notifications could improve attendance further.
- Reminders
- clinic attendance
- no shows
- healthcare efficiency
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Updates and appraises the evidence for how electronic text notifications impact on appointment attendance.
Assesses study quality using a risk of bias framework.
Large number of participants means that the impact of high quality studies can be considered.
Tests the effect of multiple notifications, one way in which notifications can be optimised.
Introduction
Reducing the number of missed healthcare appointments improves the efficiency of health services. Missing healthcare appointments without cancelling in advance results in a ‘no show’, a vacant appointment slot that cannot be offered to others. In 2015, the UK Secretary of State for Health estimated that missed general practitioner (GP) and hospital appointments cost the National Health Service (NHS) an estimated £912m per year1 and most appointments are missed due to simple reasons such as forgetfulness.2 ,3 Missed appointments and no shows are more problematic in some areas of healthcare than others, for example, attendance is poor in community mental health settings which can have subsequent effects on care.4 ,5
‘No shows’ can be reduced by reminding patients about their appointment in advance. The simplest way to do this is through electronic text notifications to patients' phones. Currently, there are as many mobile subscriptions as people in the world.6 In 2014, 93% of UK adults owned a mobile phone, with 61% having a smartphone (a 10% increase from 2013; Ofcom, 2014). In the last two decades, the service of sending text messages from mobile phones has dramatically changed the way in which people communicate. The number of messages sent has increased threefold to >150 billion between 2006 and 2011.7 This form of communication is acceptable to the public and has been harnessed by healthcare providers to remind patients about their appointments.
For the purposes of this review, the term ‘electronic text notifications’ refers to written messages sent from a service provider to a patient, in order to help patients remember, cancel or reschedule healthcare appointments. Notifications can be sent to patients' phones by text message, email or instant messaging applications. They cost little and can be delivered almost instantly.8 Unlike voice notifications, patients are able to reread and refer back to text notifications at their own convenience, and they may be perceived as less intrusive.9 They are used throughout the world across healthcare settings, with studies and reviews demonstrating increased appointment attendance.10–13 A recent meta-analysis and systematic review showed 50% improvements in attendance (relative to when no notification was provided);14 since this review was published, the use of technology is even more prevalent, with the use of smartphones almost doubling in the USA, (from 35% to 64% among adults).15
No-one has yet assessed the effectiveness of the intervention with regard to study quality/risk of bias. The large number of studies now available also allows an exploration of other potential predictive variables such as year of publication and geographical region. Similarly, there is little evidence on how to optimise electronic text notifications, specifically, whether the effect of multiple notifications is greater than the effect of a single notification and whether text notifications are as effective as voice notifications. This paper reviews and critically appraises the updated evidence for electronic text notifications and begins to answer such questions.
Aim
This review explores how much electronic text notifications improve attendance at healthcare appointments.
Methods
Types of studies
We included all experimental studies containing quantitative data to be synthesised into a meta-analysis, in which randomisation was used to define allocation to the intervention. We included all studies published in the last 10 years (January 2005 until April 2015), including data from conference presentations where full published studies were unavailable. Where the same data had been published in two publications, the article with complete data was favoured (usually the later publication). Studies published prior to 2005 were excluded. The rationale is that mobile phone ownership was limited (and thus unrepresentative) prior to this date. No published protocol exists for this review.
Types of participants
We included all participants in studies which contained data measuring the effect of electronic text notifications on scheduled appointment attendance in any healthcare setting.
Types of interventions
We included studies examining the effect of electronic text notifications on the attendance of prescheduled healthcare appointments. Studies were only included in the primary analysis if they included a control group which received ‘no notifications’. In cases where studies had multiple comparison groups (eg, electronic text notifications vs voice notifications vs no notifications), the data from alternative intervention groups were included in a secondary analysis.
We excluded:
Data relating to patients attending non-scheduled drop-in clinics or where patients were reminded to book future appointments, or health outcomes other than clinic attendance (eg, adherence to medication).
Studies not published in the peer-reviewed literature or presented at academic conferences or which lacked sufficient information to be included in the meta-analysis after contacting study authors (ie, studies failing to report the number of patients allocated to receive an electronic text notifications intervention).
Outcomes
Primary outcomes were the rate of attendance/non-attendance at healthcare appointments. The secondary outcome was the rate of rescheduling/cancellation of appointments (as opposed to ‘no show’ appointments, where the patient does not attend or cancel).
In addition to the effects on the primary and secondary outcomes, we investigated (i) whether potential predictive variables such as study quality, year of publication or geographical region affected the results, (ii) whether the number and timing of notifications affected the outcome, (iii) whether notifications had any effect in mental health settings which generally have the lowest attendance rates and (iv) how effective text reminders were in comparison to voice reminders (in studies which compared the two).
Information sources
The following bibliographic databases were searched (25.4.15): MEDLINE, EMBASE, PsycINFO, Web of Science and The Cochrane Database of Systematic Reviews (from January 2005 until the search date). A hand search was also conducted of the reference lists of included studies, which identified two additional studies. The key terms used in the electronic searches for each of the databases are shown in online supplementary appendices 1–5. Authors of studies were contacted for further information when it was not present in the published data, for example, to clarify the patient groups they had included in their study.
supplementary appendices
Data collection process
Two reviewers (SS and JR) independently screened all the papers against the inclusion criteria. For the papers that met the inclusion criteria, the reviewers independently extracted information on the geographical location, clinic type, sample size, interventions and controls, study design, and comparison outcomes of attendance and non-attendance rates. Disagreements throughout this process were resolved by arbitration with a third reviewer (DR).
Classification of data
Articles were included and interpreted based on the outcome measures used. This fell into three categories; attendance in clinic, ‘no show’ rates in clinic and cancellations/rescheduled appointments in clinic. Attendance rate and ‘no show’ rate were examined separately. Although there are many similarities between these outcomes, they cannot be assumed as equivalent as some unattended appointments may be cancelled or rescheduled in advance, in which case they are not classifiable as ‘no shows’. For those studies that measured attendance as a primary outcome measure, it was not possible to separate the proportion of ‘no shows’ and the proportion of cancellations.
Assessment of risk of bias
Studies were appraised using Cochrane Handbook for Systematic Reviews of Interventions.16 For each study which was to be included in the primary analysis, two reviewers (SS and JR) independently assessed the risk of bias. Each domain was judged as ‘low’, ‘high’ or ‘unclear’ risk (when insufficient information was provided to permit judgement). The agreement rate between the raters was 79% (κ=0.58), a moderate level of agreement. Discrepancies between ratings were resolved by discussion with the third reviewer (DR).
All authors were contacted and asked for comments or clarifications on the risk of bias rating. The authors of 14 studies responded; changes were discussed with the reviewers. One or more changes were made to the ratings of seven studies. The most common reason for changing the rating was gaining access to study protocols. Studies were classified as either ‘at risk’ of bias or ‘at low risk’ of bias.
Summary measures
The principle summary measure was risk difference between those who attended appointments compared with those who missed their appointment (expressed as a percentage); risk ratios were also calculated for the primary outcome. We compared those groups in which patients had received an electronic text notification with groups who received no notification.
We combined the results of participants receiving electronic text notification ‘intervention’ from all included studies. These were compared against ‘control’ participants who received no notification. The percentages of the primary outcome measure (attendance, ‘no shows’ and appointment cancellation) for all known intervention groups and control groups were extracted. Some studies presented data from multiple intervention groups (eg, from different clinics). In these cases, the intervention groups were the unit of analysis rather than the study itself, for example.17 A secondary meta-analysis pooled the data from studies comparing electronic text notifications against voice reminder notifications.
Synthesis of results
Meta-analyses were conducted to determine the pooled effect size relating to intervention versus control groups using a random effects model. This is more realistic than fixed-effect meta-analyses in this situation due to the variety of populations and settings between studies.18 The primary meta-analysis was split by the three possible outcomes (i) attendance, (ii) ‘no shows’ and (iii) appointment cancellation. Pooled relative rates with exact Clopper-Pearson 95% confidence limits are presented. The risk ratio and the risk difference were calculated for included studies along with their CIs (at 95%), in order to calculate overall effect sizes for the intervention group and control group. Here, the risk ratio is the ratio of the probability of a positive event occurring in the intervention group to the probability of the event occurring in the control group. Interstudy heterogeneity was calculated using the I2 statistic (≥50% indicated heterogeneity). Heterogeneity was investigated by conducting meta-regressions to examine the influence of risk of bias, year of publication and geographical region. We also investigated the impact of multiple notifications in comparison to single notifications.
We investigated whether any study had a large influence on the pooled estimate in sensitivity analyses by re-estimating meta-analysis omitting each study in turn using Stata's (V.11.2, StataCorp, College Station, Texas, the USA) ‘metainf’ command.19
Publication bias was assessed by visual inspections of funnel plots, Egger's test and using a non-parametric ‘trim and fill’ method.20 If the conclusion of the meta-analysis remains unchanged following adjustment for the publication bias using the trim and fill method, the results can be considered as robust, excluding publication bias. All analyses were carried out in Stata (V.11.2, StataCorp).
Results
Study selection
After duplicates had been removed, 3981 articles were screened. Of these, 3910 articles were excluded based on the abstract alone; these studies clearly did not meet the inclusion criteria. A further 45 articles were excluded after reading full-text articles. The primary reason for exclusion in each case were as follows: non-randomised study (n=23), examined the effects of electronic text notifications on reminding patients to take medication or to make appointments (rather than to attend) (n=3), only collecting data on preference for notifications rather than information about their effect on attendance (n=9), only providing secondary data or where the same data had been assimilated in another included paper (n=6), exclusively using other notification systems such automated call backs (n=2) or lacking sufficient information to be included in the meta-analysis after we attempted to contact authors (n=2). In total, 26 articles met the study inclusion criteria and were included in the systematic review. Of these, five studies were excluded from the primary meta-analysis because they lacked a ‘no intervention’ control group. Instead, they were included in the secondary meta-analysis comparing electronic notifications against voice notifications. The Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) flow chart (figure 1)21 describes the process in which studies were included and excluded:
PRISMA flow chart. PRISMA, Preferred Reporting Items for Systematic review and Meta-Analysis.
Study characteristics for primary meta-analysis
Of all data included in the primary meta-analysis, 12 studies used attendances as an outcome measure, 16 measured ‘no show’ rates and 3 measured cancellation. A total of 8345 patients received electronic text notifications, and 7731 patients received no notification. Randomised studies typically compared attendance or ‘no show’ rates of those who received electronic text notifications with those who did not receive notifications.
Studies spanned Europe (nine), Asia (seven), America (one), Africa (two) and Australia (two). The most common study context was primary care/general physical healthcare (11); followed by sexual health (3), dental care (2), mental health (2), and paediatrics (2). Other types of health intervention included paediatrics, postnatal care, blood donation, optometry, chronic illness, allergies and mixed contexts.
All included studies used short message service (SMS) notifications; one study used this in conjunction with follow-up phone calls and postal notifications. A typical example of an electronic text notification informed the patient of the time and date of the appointment and asked them to respond if they could not come to the appointment, for example: ‘You have an appointment on… (date) at … (time) with Dr … (name) Please answer NO if you do not intend to come’.
Studies differed in the number and in the timing of notifications sent to patients. The majority of studies (n=13) sent only one notification. Two notifications were sent in two studies; more than two notifications were sent in three studies, one study used voice notifications followed up with electronic text notifications (this was classified as a study in which ‘more than one notification’ was sent). One study did not provide this information. In one anomalous study, notifications were sent on every day for 30 days prior to an appointment. Regarding timing of messages, in nine studies notifications were sent 48 hours (or less) before the appointment. In three studies, notifications were sent over 48 hours before the appointment. In one study, participants were sent a notification 8 weeks prior to their appointment. Of the eight studies where two or more notifications were sent, the majority (n=5) reminded patients before and after the 48 hour mark. Full details of individual studies are presented in table 1.
Studies included in meta-analyses
Study characteristics for secondary meta-analysis
The secondary meta-analysis included six studies (only one of which had been included in the primary meta-analysis). Attendance rates were measured in three studies, ‘no show’ rates were measured in four. A total of 9885 patients received electronic text notifications, and 5076 patients received voice notifications. The studies were conducted in America (four), Europe (one) and Asia (one). Context included were as follows: primary care/general healthcare (two), sexual health (one), dental paediatrics (one), blood donation (one) and chronic illness (one).
Risk of bias within and across studies
The risk of bias within individual studies is presented in online supplementary appendix 6. We used items of Cochrane's framework in judging the quality of the studies. The corresponding author of each article was sent their assessment to check and suggest revisions if necessary. Biases relating to blinding were considered of lesser importance in context of the intervention; participants cannot be blinded to a notification intervention, and outcome assessment is objective (ie, the participant either attended appointment or not). Random sequence generation and incomplete outcome data were considered important potential biases. The most common reason for ‘unclear’ bias was the unavailability of protocols.
Primary meta-analysis results: main outcomes
The pooled attendance rate was 67% (N=13, CI 53% to 82%) for intervention groups and 54% (N=13, CI 37% to 70%) for control groups. The risk ratio was 1.23 (CI 1.10 to 1.38; N=13, p<0.01, I2=83%), the risk difference was 13% in favour of the intervention group (95% CI 6% to 19%; N=13, p<0.01, I2=82%). The pooled ‘no show’ rate was 15% (N=16, CI 10% to 19%) for intervention groups and 21% (N=16, CI 16% to 26%) for control groups. The risk ratio was 0.75 (CI 0.68 to 0.82; N=16, p<0.01, I2=21%), the risk difference was 5% in favour of the intervention group (95% CI −7% to −3%; N=16, p<0.01, I2=31%). The percentage difference between intervention and control groups for each study is shown in figures 2 and 3.
Effect of notifications on attendance rates. RD, risk difference.
Effect of notifications on ‘no show’ rates. RD, risk difference.
The pooled cancellation rates were 11% for intervention (N=3, CI −2% to 19%) and 8% control (N=3, CI −1% to 14%) groups. The risk ratio was non-significant at 1.37 (N=3, p=0.34, I2<1) as was the 2% risk difference at (N=3, p=0.4, I2<1).
Visual inspection of funnel plots (see figures 4 and 5, see online supplementary appendix 7) revealed evidence of potential publication bias in attendance but little evidence of publication bias in ‘no shows’. The trim and fill method revealed no missing studies. Egger's test was not significant in the meta-analyses.
Funnel plot for attendance rates.
Funnel plot for ‘no show’ rates.
Assessing and identifying study heterogeneity
Meta-regression compared the impact of the following potential predictive variables: risk of bias (high bias, low bias), number of notifications (one, multiple), year of publication (2005–2010, 2011–2015) and geographic region (Europe, Asia, other), where the ‘other’ category was created because there were few studies in the other continents.
Of these variables, the only significant finding was the effect of multiple reminders on appointment attendance (shown in table 2). Multiple notifications increased the risk of patients attending appointments by 25% (compared with 6% for patients receiving one notification), but multiple reminders did not make a significant difference in reducing ‘no shows’. No significant effects were found for risk of bias (p=0.88 for attendance, p=0.68 for ‘no shows’), age of study (p=0.16 for attendance, p=0.38 for ‘no shows’) or geographic region (F=0.11, p=0.9 for attendance; F=1.6, p=0.23 for ‘no shows’). No significant associations were found when all variables were pooled (F=2.35 p=0.15 for attendance; F=0.66, p=0.66 for ‘no shows’).
Effect of notification frequency
Sensitivity analyses
In a sensitivity analysis in which the meta-analyses were repeated, excluding one study at a time to investigate the influence of each individual study on the overall meta-analysis summary did not reveal that any single study significantly affected the results.
What happens in mental health?
We pooled the results of three studies (182 participants received notifications, 197 did not). Two studies measured attendance and three measured ‘no shows’. One took place in psychosis services, one in a university therapeutic context and one at a youth clinic. The pooled attendance rates were 85% for intervention (N=2, CI 78% to 91%) and 87% for control (N=2, CI 81% to 93%). The risk ratio was 1.01 (CI 0.85 to 1.2; N=2, p=.92, I2=30%), the risk difference was <1% (95% CI −11% to 12%; N=2, p=0.93, I2=25%). The pooled rate of ‘no shows’ was 7% for intervention (N=3, CI 3% to 11%) and 13% for control (N=3, CI 4% to 22%). The risk ratio was 0.61 (CI 0.29 to 1.29; N=3, p=0.2, I2=21%), the risk difference was 5% (95% CI −14% to 4%; N=3, p=0.26, I2=52%).
How do electronic text notifications compare to voice notifications? (secondary meta-analysis)
The pooled attendance rate for the electronic text notifications was 74% (N=3, CI 60% to 88%) and 74% for voice notifications (N=3, CI 51% to 97%). This difference was significant; however, the risk ratio was 0.90 (CI 0.82 to 0.98; N=3, p=0.01, I2<1%), the risk difference was 8% in favour of voice notifications (95% CI −16% to 0.1%; N=3, p=0.05, I2=6%). Pooled ‘no show’ rates were 15% for electronic text notifications (N=4, CI 11% to 20%) and 13% for voice notifications (N=4, CI 7% to 18%). The risk ratio was 1.12 (CI 0.90 to 1.38; N=4, p=0.32, I2<73%), the risk difference was 1% (95% CI −2% to 4%; N=4, p=0.35, I2=70%).
Discussion
This review and meta-analysis demonstrates that electronic text notifications improve appointment attendance and reduce ‘no shows’. Notifications improve attendance and reduce ‘no shows’. These findings replicate earlier ones,14 but we can have more confidence in the results because they were stable even after removing the influence of studies which were at risk of bias. A novel finding is that two or more notifications increased attendance by as much as 19% over and above sending one notification, and voice notifications may offer slight improvements over text notifications for increasing attendance.
Taking the UK Secretary of State's estimates literally, a 5% reduction in ‘no shows’ across the National Health Service (NHS) GPs and hospitals would save the NHS >£45 million. There may be additional savings gained by sending multiple (as opposed to single) notifications. Almost all NHS services have an electronic text notification system already; these could be adapted to provide an extra notification at little extra cost to accommodate this change.
Some areas, such as mental health, have historically reported high rates of missed appointments,5 where people with severe mental illness may miss up to 45% of scheduled appointments in primary care.46 The studies reviewed here suggested that attendance rates for mental health settings were not dissimilar to those in other settings. These studies do not, therefore, reflect the ‘normal’ clinic attendance known to be lower and therefore suggests that more studies reflecting usual practice are needed. For those clinical areas with poor attendance, text messages may not act in the same way and may need to be adapted. But currently, we do not have any evidence to draw any conclusion.
Strengths and limitations
The main strength of this review is the synthesis of large data sets. We were able to distinguish between different types of outcomes (attendance and ‘no shows’). The study provides basic information about how to optimise notifications, but more is needed in this area.
Future research directions
This review confirms the importance of electronic text notifications in healthcare. To make the notifications even more effective we need to examine the tone and content of notifications, and whether the effects differ between client groups, and what preferences patients have for receiving notifications.
Conclusions
Electronic text notifications increase attendance and reduce ‘no shows’. Multiple notifications add significantly to the effectiveness. The large number of ‘no shows’ in health services means any successful intervention to reduce them will have cost implications.
References
Footnotes
Contributors DR, JR and TW designed the study and formulated the clinical question. JR and SS performed the literature search and, with DR, reviewed the search results for study inclusion. DR, JR and SS designed the data extraction form, extracted the data and assessed risk of bias. DS developed the analysis strategy, DR and DS performed the statistical analyses. DR led the writing of the final manuscript with support from SS and JR. All authors critically revised drafts and the final manuscript. All authors approved the final manuscript for submission.
Funding The authors would like to acknowledge the support the National Institute for Health Research (NIHR) Mental Health Biomedical Research Centre at South London and Maudsley National Health Service (NHS) Foundation Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests All authors have completed the International Committee of Medical Journal Editors (ICMJE) uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Transparency declaration The lead author affirms that this manuscript is an honest, accurate and transparent account of the review being reported, that no important aspects of the study have been omitted and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.