arXiv:2606.26649: Agent instructions converted into formally verified policy-as-code

The gap between promises and guarantees

Most of today’s safety measures for AI agents rest on probabilistic guardrails — constraints imposed through system prompts and fine-tuning, without formal guarantees. Mondl, Maisel, and Brock are not satisfied with that. Their work, accepted at the AIWILD workshop at ICML 2026, introduces an automated pipeline that translates natural-language policy documents, prompts, and MCP tool descriptions into Cedar Policy Language — a formal language for verified rules that Amazon uses for cloud access control.

How does the generator-critic loop work?

An LLM generator proposes a Cedar policy from the source text, while a second LLM critic checks coverage and consistency. The loop repeats until the policy satisfies all conditions. The result is machine-verifiable code, not text that an agent can ignore.

What do the results on MedAgentBench show?

On the medical agent MedAgentBench, the autoformalized policies covered a substantially greater share of the original NL specification than manually coded symbolic enforcement from prior work. The authors do not publish exact percentages in the abstract, but the qualitative difference is described as “substantially greater coverage” — which is precisely the point of failure where the manual approach breaks down when scaling to more complex domains.

Why is Cedar the right choice?

Cedar Policy Language is designed for expressiveness and fast formal reasoning. Unlike Python scripts or regex filters, Cedar policies allow static analysis: you can prove that an agent will never call a specific tool without appropriate authorization, without running a single real invocation.