Amazon Nova Sonic + WebRTC: real-time voice agents with Kinesis Video Streams and async tool calling for RAG/MCP

Amazon Web Services published on May 13, 2026 an architecture that combines the Nova Sonic speech-to-speech model with the Kinesis Video Streams WebRTC pipeline — a reference blueprint for real-time voice agent applications with async tool calling to MCP servers and RAG systems.

How do Nova Sonic and WebRTC share responsibility?

The architecture introduces a speech-to-speech event processor that “orchestrates input and output events” between the WebRTC stream and the Nova Sonic model. Communication is split into media events (audio through WebRTC) and text data (through data channels). WebRTC establishes peer-to-peer links through Kinesis Video Streams signaling channels, enabling bidirectional audio/video transmission with adaptive bitrate control and forward error correction.

What does server-side VAD contribute?

Voice Activity Detection (VAD) uses the Python WebRTCVAD library on the server side. Detection suppresses noise and reduces audio token volume before the stream reaches Nova Sonic. The approach has two benefits: it lowers inference costs (fewer tokens = lower Bedrock price) and improves latency because Nova Sonic does not have to process long silence segments.

How does Nova Sonic invoke tools during a conversation?

Nova Sonic supports asynchronous tool callouts to MCP servers, Strands agents or RAG systems during a voice session. A user can ask “what is the current temperature in the garage?” mid-conversation, and the agent simultaneously calls an MCP server that returns the sensor reading without interrupting the conversation. The async approach is critical because the voice latency budget (250–500 ms) does not allow for a synchronous RAG lookup pause.

What are the first demonstration use cases?

AWS showcases two scenarios. Smart home: voice commands control IoT devices through MQTT, integrated with Amazon Bedrock Knowledge Base and an MCP server — the agent knows device state and can control them. Connected vehicles: real-time driver monitoring detects phone-use behaviors, while a voice assistant confirms safety status through independent monitoring streams — turning the voice agent into a safety tool, not just an entertainment interface.

WebRTC delivers the lowest latency among media streaming protocols (RTMP, RTSP, HLS, MPEG-DASH) — critical for voice agents where delays above 500 ms degrade perceived conversation quality.