LangChain: Cheaper verification of legal AI agents cuts cost tenfold

LangChain Labs, in collaboration with Harvey, has published research on how to verify the outputs of legal AI agents more cheaply. Verifying complex legal work using frontier models is a cost bottleneck, which the researchers set out to address.

How did they reduce the cost?

The team tested two approaches. The first is batching verifiers, that is, combining multiple evaluation criteria into a single API call instead of separate requests. The second is using cheaper open models, such as DeepSeek, together with batching. The result is a reduction in cost per criterion by what they describe as an order of magnitude, that is roughly ten times, while preserving quality.

How extensive was the testing?

The research was conducted on 40 public legal tasks with a total of 2,348 individual evaluation criteria graded as pass or fail. The areas covered included corporate mergers and acquisitions, tax, growth companies and venture capital, and estate administration. The models tested included GPT-5.5, Sonnet 4.6, Haiku 4.5, DeepSeek v4 Flash, and Claude Opus 4.7.

Which models performed best?

DeepSeek stood out as a strong alternative, reaching accuracy comparable to Opus at a dramatically lower cost. Cheaper models such as Haiku showed problematic false-pass rates, as high as 48.4 percent per criterion, which is undesirable in a legal context. Through targeted prompt optimization, the team lowered DeepSeek’s error rate from 10.7 to 9.5 percent per criterion, and from 15.6 to 14.2 percent in batch mode.