arXiv:2605.20173: 6 Architectural Patterns for Production LLM Agents
A new arXiv paper introduces the stochastic-deterministic boundary as the foundational design principle for production LLM agents and defines 6 composable runtime patterns — from hierarchical delegation to human-in-the-loop — selected according to three architectural concerns: coordination, state, and control.
This article was generated using artificial intelligence from primary sources.
Building an LLM agent that works in the lab is relatively easy. Building an agent that reliably works in production — that is a different story. A new arXiv paper (Vasundra Srinivasan, 2605.20173) offers the first systematic methodology for that distinction.
What is the stochastic-deterministic boundary?
The central insight of the paper is the concept of the stochastic-deterministic boundary (SDB) — the line between non-deterministic LLM inference and deterministically executed system actions. The SDB functions as a four-part contract: the proposer (LLM) suggests an action, the verifier checks whether predefined rules are met, the commit step records the decision as irreversible, and the reject signal returns control with diagnostic information.
Without an explicit SDB, the system does not know where stochasticity ends and accountability begins — which is the root cause of most production incidents.
Why does the architectural pattern become more important as models improve?
The paper demystifies reliability into two components: per-call model variance and architectural momentum. A counterintuitive conclusion: as models become more accurate, the choice of architectural pattern becomes proportionally more critical. Less model noise means that systemic failures in coordination and control dominate the overall error rate.
Six composable patterns are organized according to three concerns — coordination, state, and control — and can be combined. The reference implementation shows a 90-day contract-renewal agent combining scatter-gather plus saga with a human-in-the-loop gate.
Replay divergence: the silent error that destroys audit trails
A particularly valuable warning: replay divergence — the situation where the same deterministic event log yields different outputs after a model update or prompt change. In regulated industries (finance, law, healthcare) this can retroactively invalidate an entire audit trail.
The paper provides diagnostic procedures for error attribution and a five-step pattern selection methodology. For teams transitioning from prototype to production — required reading.
Frequently Asked Questions
- What is the stochastic-deterministic boundary?
- It is a four-part contract between a proposer, verifier, commit step, and reject signal that determines when and how LLM output becomes an irreversible system action.
- What are the 6 runtime patterns from the paper?
- Hierarchical delegation, scatter-gather plus saga, event-driven sequencing, shared state machine, supervisor plus gate, and human-in-the-loop — each addressing a different combination of coordination, state, and control.
- What is replay divergence and why is it dangerous?
- An error in which an agent consuming a deterministic event log produces inconsistent outputs after a model or prompt change — a serious problem for audit trails, compliance, and reproducibility in production systems.