arXiv: ABC-Bench — AI agents outperform expert biologists on all biosecurity tasks

A research team has published ABC-Bench (Agentic Bio-Capabilities Benchmark) — the first standardized benchmark designed for systematic measurement of AI agent capabilities on biosecurity-relevant tasks. Authors Andrew Bo Liu, Samira Nedungadi, Bryce Cai, Alex Kleinman, Harmon Bhasin, and Seth Donoughe developed the benchmark to fill a critical gap: until now, no empirical tool existed for assessing the dual-use risks of AI agents in biological research. The 18-page paper was accepted at ICML 2026.

Three categories of biosecurity tasks

ABC-Bench covers three task categories, selected because they represent real competencies that previously required experienced biologists with years of laboratory training:

1. Programming liquid-handling robots — agents write scripts for automated laboratory equipment, specifically OpenTrons robotic pipettes. The task requires precise knowledge of laboratory protocols and the robot’s programming interface (API).

2. DNA fragment design for in-silico assembly — agents design DNA fragment sequences for computational assembly of specific genetic constructs. The task combines bioinformatics and molecular biology knowledge.

3. Evading DNA synthesis screening — agents attempt to generate sequences that would pass through the safety filters of commercial DNA synthesis providers. This is a directly biosecurity-relevant category with a clear misuse dimension.

Can AI agents replace expert biologists on high-risk tasks?

The key finding of the ABC-Bench study is unambiguous: all tested LLM agents outperformed the median expert biologist on all three task categories. The authors emphasize that agents performed especially well on tasks based on published knowledge and documented protocols — areas where language models have a structural advantage thanks to the breadth of data they were trained on.

On the other hand, performance was weaker on tasks requiring novel bioinformatics reasoning beyond the reach of existing literature — situations where an experienced researcher retains an advantage rooted in intuition gained through experimental work.

This finding is not an argument that AI replaces biologists in a holistic sense. But it indicates that specific high-risk competencies — which until now were protected by the complexity of specialized knowledge — are now accessible via AI agents that can operate without formal academic or institutional expertise.

Wet-lab validation

A particularly important aspect of the study is the wet-lab validation. When the model OpenAI o4-mini-high generated scripts for the OpenTrons robot, those scripts were tested in a real laboratory environment. The result: the scripts successfully assembled DNA sequences with the expected results.

This wet-lab confirmation distinguishes ABC-Bench from purely software-based benchmarks. It was not merely a theoretical ability to generate biologically correct code — it was a demonstrated, physically verified ability of an AI agent to carry out a laboratory procedure with tangible outcomes. This elevates the benchmark to the level of actual, rather than merely hypothetical, risk.

Dual use and implications for biosecurity policy

The authors explicitly frame ABC-Bench within the dual-use context: the same capabilities that help legitimate researchers accelerate experimental protocols also represent potential misuse pathways. The benchmark is not designed to create an attack tool — quite the opposite, it is intended to create an empirical basis for risk assessment.

The paper is conceived as a resource for AI biosecurity policymakers and AI system evaluation bodies. Without a standardized, empirically verified benchmark, policies remain grounded in assumptions about model capabilities rather than measured data. Publication under CC BY 4.0 means that researchers, regulators, and AI company security teams can freely use and extend ABC-Bench for their own evaluations.

ABC-Bench as a foundation for future evaluation standards

ABC-Bench arrives at a moment when the biosecurity dimensions of AI systems are attracting increasing attention from regulators. The paper offers a concrete evaluation framework that could become a reference point for assessing new models and agentic systems.

If AI agents can reliably outperform expert biologists on tasks with misuse potential, then policies based solely on restricting access to the models themselves are insufficient. Dynamic verification mechanisms, user intent detection, and structured collaboration between AI companies and biosecurity institutions are needed.