Anthropic: VirBench benchmark and gget virus tool raise AI agent accuracy in biology to 99.7%

Anthropic researchers published on June 8, 2026 the paper Paving the way for agents in biology (lead author Laura Luebbert, with co-authors Ferdous Nasri, Saru Gurev, Patrick Varilly and others). The paper, also available as an arXiv preprint (arXiv:2606.06749), addresses a fundamental problem in applying AI agents to biology: the reproducibility of data retrieval from public databases.

VirBench: a new standard for evaluating biological agents

VirBench is a set of 120 real-world bioinformatics queries built around 40 pathogens, each with a verified correct set of viral sequences from NCBI databases. Unlike synthetic tests, VirBench replicates real research scenarios — from identifying reference strains to comparing genomes of epidemiologically relevant viruses.

Why are errors in sequence retrieval dangerous?

Incorrect input sequence sets cascade into corrupted downstream analyses. Researchers documented a concrete case: incorrectly retrieved sequences for one pathogen yielded epidemic origin dates ranging from 1922 to April 2014 — versus the verified date of January 2014. Moreover, the same model run multiple times on the same query returned different sets of sequences, eliminating research reproducibility.

Without tools vs. with a tool: from 16.9% to 99.7%

Without specialized tools, SOTA models achieved only 16.9–91.3% accuracy on NCBI Virus queries — far below the ~100% needed for reproducible biology. The new tool gget virus, developed in collaboration with NCBI researchers, provides deterministic API coordination and complex sequence filtering by taxonomy, date, and strain. With it, accuracy rises above 90% for all tested models and reaches 99.7% for GPT-5.5.

Context engines as an infrastructure response

The paper advocates introducing context engines — deterministic, agent-accessible API layers that separate data retrieval from the model’s creative reasoning. Biological databases such as NCBI contain vast structured datasets, but without an intermediary layer agents interpret them non-deterministically and without any guarantee of reproducibility. The authors call for building such layers as public infrastructure for reliable bioinformatics.