Table 3

Overall strength of evidence by outcome

OutcomeRelevant studiesEvidence statementStrength of evidenceComments
Safety =Kellermann et al 12
=Little et al 13
=Middleton et al 17
=Poote et al 18
=Price et al 22
Madan26
No evidence of a difference in risk between health professionals and symptom checkersWeakerRating changed from stronger based on study numbers and design to weaker because of low numbers of adverse events reported
Clinical effectiveness −Little et al 13
?Madan26
Insufficient evidence to draw any firm conclusionsVery limited
Costs/cost effectiveness+Babylon Health23
±Cowie et al 25
+Madan26
Insufficient evidence to draw any firm conclusionsInconsistent
Diagnostic accuracy?Berry et al 15
−Berry et al 38
Berry et al 16
− Price et al 22
?Semigran et al 6
−Semigran et al  21
=Sole et al 19
Symptom checkers appear inferior to health professionals in terms of diagnostic accuracyStrongerMainly for specific conditions or preprepared vignettes
Disposition accuracy=Babylon Health23
Berry et al 16
=Middleton et al 17
?Nijland et al 20
− Poote et al 18
±Semigran et al 6
±NHS England 7
Inconsistent findings on accuracy of dispositionInconsistentPerformance variable between different systems
Service use/diversion ?Kellermann et al 12
±Little et al 13
±Poote et al 18
?Carter et al 24
?Cowie et al 25
+Madan26
±NHS England 7
+Babylon Health23
−Luger et al 33
Inconsistent findings on effects on service useInconsistent
Compliance?Nijland et al 20
?NHS England7
No comparative data on complianceVery limited
Patient/carer satisfaction?Nagykaldi et al 35
?Nijland et al 20
?Price et al 22
+Yardley et al 14
?Carter et al 24
?Cowie et al 25
?Madan26
?NHS England7
?Lanseng and Andreassen30
Most studies report high rates of patient satisfaction with symptom checkers and e-consultation systems generallyWeakerFew studies with comparator data
  • Controlled studies in bold; =means no significant difference in outcomes; +means better outcome with symptom checker; ±varying results within study; ? results difficult to interpret in comparative terms.