The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia
Medical Journal of Australia
School of Medical and Health Sciences
Objectives To investigate the quality of diagnostic and triage advice provided by free website and mobile application symptom checkers (SC s) accessible in Australia. Design 36 SC s providing medical diagnosis or triage advice were tested with 48 medical condition vignettes (1170 diagnosis vignette tests, 688 triage vignette tests). Main outcome measures Correct diagnosis advice (provided in first, the top three or top ten diagnosis results); correct triage advice (appropriate triage category recommended). Results The 27 diagnostic SC s listed the correct diagnosis first in 421 of 1170 SC vignette tests (36%; 95% CI , 31–42%), among the top three results in 606 tests (52%; 95% CI , 47–59%), and among the top ten results in 681 tests (58%; 95% CI , 53–65%). SC s using artificial intelligence algorithms listed the correct diagnosis first in 46% of tests (95% CI , 40–57%), compared with 32% (95% CI , 26–38%) for other SC s. The mean rate of first correct results for individual SC s ranged between 12% and 61%. The 19 triage SC s provided correct advice for 338 of 688 vignette tests (49%; 95% CI , 44–54%). Appropriate triage advice was more frequent for emergency care (63%; 95% CI , 52–71%) and urgent care vignette tests (56%; 95% CI , 52–75%) than for non‐urgent care (30%; 95% CI , 11–39%) and self‐care tests (40%; 95% CI , 26–49%). Conclusion The quality of diagnostic advice varied between SC s, and triage advice was generally risk‐averse, often recommending more urgent care than appropriate.