Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI

This article demonstrates how to reduce latency in AI voice agents by streaming text to a Kokoro-82M text-to-speech model on Amazon SageMaker as sentences are generated, enabling audio playback to begin before the full response is complete.

• Deploy Kokoro-82M on a SageMaker real-time endpoint using a custom container with bidirectional WebSocket streaming
• Handle binary WebSocket frames correctly by accepting both text and bytes message types in Starlette handlers
• Keep bidirectional sessions open across multiple messages to avoid reconnection overhead between sentences
• Use an IncrementalSentenceChunker class to normalize streaming text and emit complete sentences immediately
• Multiplex raw binary audio and JSON control frames over the same bidirectional session instead of base64-encoding
• Achieved 52% reduction in time to first audio for TTS alone, and 2.4-3.5x improvement in end-to-end agent scenarios

Streaming synthesis alongside generation instead of waiting for complete responses significantly reduces perceived latency in voice agents while keeping inference costs low on mid-tier GPU instances.