AI-Based Symptom Checkers in Medical Self-Diagnosis
Medical self-diagnostic tools like AI-based symptom checkers gain popularity, but are they a secure alternative for identifying medical conditions?
Medical Symptom Checkers Definition and Use
Using the internet for self-diagnosis is common among patients. In Australia, for example, 34% of patients research their symptoms online before visiting an emergency department [1]. Self-diagnosis can influence patient decisions regarding whether to seek medical treatment and the timing of such care [2].
Apart from searching for diagnostic information online with their computers or smartphones, patients can use self-diagnosis tools such as medical symptom checkers to obtain insights based on their symptoms. These applications pose targeted questions to analyze and evaluate users’ medical conditions [3]. In the past, these symptom checkers have shown mixed results and were generally cautious, often encouraging patients to seek further medical treatment [4]. Whether new technologies, such as
Artificial Intelligence
(AI),
can enhance diagnostic accuracy remains an open question [5].
Efficiency Analysis of Medical Symptom Checkers
In
their study
, Hammoud et al. analyzed the diagnostic performance of symptom checkers compared to experienced medical doctors, using a multi-stage vignette approach (see Figure 1). The vignettes were designed to provide medical information for the symptom checkers and included gold-standard main diagnosis and differential diagnoses information, ranked by relevance [6].
Figure 1: Multi-stage study approach
After performing the testing, Hammoud et al. compared the diagnoses made by the checkers and the doctors to the gold-standard diagnoses described in the vignettes. They evaluated whether the correct main diagnosis, as well as the correct differential diagnoses were identified [6].
Additionally, they looked at the precision of the diagnosis, counting the number of vignettes where the main diagnosis was listed as the top 1 diagnosis, or within the top 3, or top 5 differential diagnoses [6].
AI as an Efficiency Driver for Medical Symptom Checkers
Prior studies indicated that symptom checkers could not outperform health professionals in terms of diagnostic accuracy [7]. However, Hammoud et al. show that the use of
AI
has significant potential in supporting medical decision-making. In their study, the two AI-based symptom checkers consistently performed the best among all six symptom checkers [6].
One medical doctor constantly performed best among all medical doctors [6], showing that the induvial experience of a medical doctor has great impact in formulating the correct diagnosis.
Comparing the symptom checkers to the medical doctors, the best-performing symptom checker, on average, outperformed the doctors. Still, the medical doctors were better at listing the correct main diagnosis as their top 1 diagnosis or among the top 3 of their differential diagnoses list, showing that they had a higher precision in finding the correct diagnosis. However, the best-performing symptom checkers were better at listing all potential differential diagnoses [6].
Future Implications regarding AI Self-Diagnosis Tools
The performance of symptom checkers varies significantly, with AI-based systems delivering better results.
As diagnostic precision continues to improve, these systems could serve a dual purpose: guiding patients to appropriate healthcare providers based on the suggested diagnoses and empowering them to take greater control over their health, while also offering medical professionals a systematic approach to identifying potential diagnoses.
Furthermore, advanced symptom checkers could be integrated into Health Information Exchanges (HIE). By allowing patients to use these applications prior to consultations, the results could be automatically leveraged during the anamnesis process, streamlining preparation for both healthcare providers and patients.
This integration would be particularly valuable in addressing complex diagnostic tasks [8].
References
- Cocco AM, Zordan R, Taylor DM, Weiland TJ, Dilley SJ, Kant J, et al. Dr Google in the ED: searching for online health information by adult emergency department patients. Med J Aust. 2018;209:342–7. doi:10.5694/mja17.00889.
- Mueller J, Jay C, Harper S, Davies A, Vega J, Todd C. Web Use for Symptom Appraisal of Physical Health Conditions: A Systematic Review. J Med Internet Res. 2017;19:e202. doi:10.2196/jmir.6755.
- Hill MG, Sim M, Mills B. The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia. Med J Aust. 2020;212:514–9. doi:10.5694/mja2.50600.
- Semigran HL, Linder JA, Gidengil C, Mehrotra A. Evaluation of symptom checkers for self diagnosis and triage: audit study. BMJ. 2015;351:h3480. doi:10.1136/bmj.h3480.
- Levine DM, Mehrotra A. Assessment of Diagnosis and Triage in Validated Case Vignettes Among Nonphysicians Before and After Internet Search. JAMA Netw Open. 2021;4:e213287. doi:10.1001/jamanetworkopen.2021.3287.
- Hammoud M, Douglas S, Darmach M, Alawneh S, Sanyal S, Kanbour Y. Evaluating the Diagnostic Performance of Symptom Checkers: Clinical Vignette Study. JMIR AI. 2024;3:e46875. doi:10.2196/46875.
- Gilbert S, Mehl A, Baluch A, Cawley C, Challiner J, Fraser H, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020;10:e040269. doi:10.1136/bmjopen-2020-040269.
- Wiedermann CJ, Mahlknecht A, Piccoliori G, Engl A. Redesigning Primary Care: The Emergence of Artificial-Intelligence-Driven Symptom Diagnostic Tools. J Pers Med 2023. doi:10.3390/jpm13091379
Also read:
Revolutionizing Nursing Education: AI’s Emergent Role in Virtual Reality Simulations
