AWS: Strands Evals SDK Automates AI Agent Failure Detection and Root Cause Analysis

AWS has introduced Strands Evals, an SDK that automates the detection of AI agent failures and their root cause analysis, addressing one of the hardest parts of working with agents in production.

How does Strands Evals detect agent failures?

Strands Evals works in two phases. In the first phase it detects failures across nine categories, including hallucinations, wrong actions, orchestration errors, context errors, and unnecessary repetitions. The analysis is based on reviewing agent execution traces using a language model that recognizes failure patterns that are hard to catch with classical rules.

What does the root cause analysis deliver?

In the second phase the tool performs root cause analysis by building a causal chain and classifying contributions as PRIMARY, SECONDARY, or TERTIARY. Rather than simply reporting that the agent failed, Strands Evals points to the most likely source of the problem and suggests concrete fixes such as SYSTEM_PROMPT_FIX or TOOL_DESCRIPTION_FIX. AWS says this cuts diagnostics “from hours to minutes.”

How does it fit into the development and production workflow?

Strands Evals provides a DiagnosisConfig with two triggering modes: ON_FAILURE for CI/CD pipelines and ALWAYS for audit purposes. Through CloudWatchProvider it can analyze production sessions from Amazon CloudWatch logs. The SDK requires Python 3.10 or later and integrates with Amazon Bedrock.

Why does this matter for agent development?

As agentic systems enter production, diagnosing why an agent went wrong becomes a bottleneck. Automated failure detection and root cause analysis with concrete repair suggestions shifts part of that work from manual investigation to a tool, speeding up iteration and improving agent reliability.