Table 1

Studies of symptom checkers as a self-contained intervention

ReferenceStudy designSystem typeComparatorPopulation/sample
Babylon Health23
UK
· Uncontrolled observational
No control group but some comparison with NHS111 telephone data
· Digital
Smartphone app
· Health professional performance on real-world data
· Other
NHS111 data for 12 months from February 2017
· General population
Participants in the London pilot evaluation of ‘digital 111’ services
Berry et al15
USA
· Simulation
Evaluation of symptom checker performance on clinical vignettes
· Online
17 symptom checkers
· None· Specific condition(s)
Gastrointestinal symptoms
Berry et al38
USA
· Controlled observational· Online
Three online symptom checkers (WebMD, iTriage and FreeMD)
· Health professional performance on real-world data· Specific condition(s)
Patients with a cough presenting to an internal medicine clinic
Berry et al16
USA
· Controlled observational· Online
Three online symptom checkers (WebMD, iTriage, FreeMD)
· Health professional performance on real-world data· Specific condition(s)
Abdominal pain
Kellermann et al12
USA
· Simulation
The developed algorithm was tested against past patient records.
· Online
Strategy for Off-Site Rapid Triage (SORT) was available on two interactive websites
· Health professional performance on real-world data
The algorithm was tested against clinicians’ decision on past patient records.
· Specific condition(s)
Influenza symptoms
Little et al13
UK
· Experimental
Randomised controlled trial
· Online
‘Internet Doctor’ website
· Other
Usual GP care without access to the Internet Doctor website
· Specific condition(s)
Respiratory infections and associated symptoms
Luger et al33
USA
· Simulation
Described as ‘human–computer interaction study’ using think-aloud protocols.
· Online
Google and WebMD
· Other
Comparing two internet health tools.
· General population
Older adults (50 years or older)
Marco-Ruiz et al 34
Norway
· Qualitative
Qualitative element
· Other
1. Online evaluation by users (problem detection) 2. Think aloud technique by smaller sample of participants (usability)
· Online
Erdusyk
· None· General population
Internet tool users
Middleton et al17
UK
· Simulation· Digital
‘babylon check’ automatic triage system
· Health professional performance on test/simulation
Twelve ‘ clinicians’ (doctors) and 17 nurses
· General population
Nagykaldi et al35
USA
· Uncontrolled observational· Online
Customised practice website including a bilingual influenza self-triage module, a downloadable influenza toolkit and electronic messaging capability. A bilingual seasonal influenza telephone hotline was available as an alternative.
· None· Specific condition(s)
Influenza
Nijland et al20
Netherlands
· Uncontrolled observational
Retrospective analysis of 15 months’ data
· Online
Web-based triage system (http://www.dokterdokter.nl)
· None· General population
Poote et al18
UK
· Uncontrolled observational· Online
Prototype self-assessment triage system
· Health professional performance on real-world data
GPs triage rating was compared with rating from the self-assessment system
· General population
Students attending a University Student Health Centre with new acute symptoms
Price et al22
USA
· Uncontrolled observational· Online
A web-based decision support tool—SORT for kids designed to help parents and adult caregivers decide whether a child with possible influenza symptoms needs to visit the emergency department (ED) for immediate care.
· Health professional performance on real-world data
The sensitivity of the algorithm was compared with a gold standard—evidence form child’s medical records that they received one or more of five ED-specific interventions.
· Specific condition(s)
Influenza in children
Semigran et al6
N/A
· Experimental
Described as an audit study
· Multiple
23 symptom checkers were evaluated. Symptom checkers available as apps (via the App Store and Google Play) were identified through searching for ‘symptom checker’ and ‘medical diagnosis’ and screened the first 240 results. Symptom checkers available online were identified through searching Google and Google Scholar for symptom checker and medical diagnosis and screened the first 300 results.
· Other
Vignettes had a diagnosis and triage attached to them and these were compared against the symptom checker advice.
· General population
Where a single class of illness was examined by the symptom checker, the symptom checker was excluded from the study.
Semigran et al21
USA
· Experimental
Comparison of physician and symptom checker diagnoses based on clinical vignettes
· Multiple
‘Human Dx is a web-based and app-based platform’
· Health professional performance on test/simulation
Clincial vignettes—a comparison of 23 symptom checkers with physician diagnosis for 45 vignettes
· General population
Of the 45 condition vignettes—there were 15 low, 15 medium and 15 high acuity vignettes—there were 26 common and 19 uncommon condition vignettes
Sole et al19
USA
· Uncontrolled observational
Descriptive comparative study
· Online
A web-based triage system (24/7 WebMed)
· Health professional performance on real-world data
Data were evaluated from students who had used the web-based triage and then requested an appointment via email (so triage data were available for comparison).
· General population
Yardley et al14
UK
· Experimental
Exploratory randomised trial
· Online
Internet Doctor website
· Other
Self-care information provided as a static web page with no symptom checker or triage advice
· Specific condition(s)
Minor respiratory symptoms, for example, cough, sore throat, fever, runny nose
  • GP, general practitioner; N/A, not applicable; NHS, National Health Service.