Google Research: SymptomAI, a Conversational AI Agent, Outperforms Clinicians in Over 50% of Symptom Assessments

Google Research introduced SymptomAI — a conversational AI agent, that is, a system that gathers information about symptoms through dialogue with the user and suggests possible explanations, built on the Gemini Flash 2.0 model, designed for everyday symptom assessment. Note: this is research, not a medical product.

How well does SymptomAI assess symptoms compared to clinicians?

In a nationwide study with 13,917 participants, clinical experts ranked SymptomAI’s recommendations as better in more than 50% of cases compared to recommendations from other clinicians. In addition, SymptomAI achieved higher top-5 accuracy — the share of cases in which the actual diagnosis appears among the five most likely options offered — than the comparison clinicians.

Five prompting strategies and the advantage of an agent-driven approach

The researchers compared five prompting strategies, that is, five different ways of structuring queries to the model, to determine which approach best leverages the model’s capabilities for symptom assessment. All five agent-driven strategies, in which the agent actively leads the conversation and asks follow-up questions, significantly outperformed the baseline strategy without such guidance. SymptomAI’s advantage proved greatest precisely in situations where the clinicians themselves were uncertain in their assessment.

What role do Fitbit biosignal data play?

Data from Fitbit devices showed physiological changes correlating with SymptomAI’s diagnoses of infectious diseases, and this occurred before participants even reported symptoms. This opens the possibility of earlier disease detection based on objective biosignals, rather than relying solely on a subjective description of complaints.

Conclusion

The results show that an agent-driven approach to conversational symptom assessment outperforms both the baseline AI strategy and, according to clinical expert rankings, more than half of the comparisons with human clinicians — with the caveat that this is a research stage, not a finished medical tool.