ModelRefs / Speech Pipeline Stack — Architecture Pattern
Speech Pipeline Stack — Architecture Pattern
ASR → LLM → TTS with streaming, barge-in, and end-of-utterance detection.
Overview
Speech-in / speech-out pipeline: streaming ASR, LLM reasoning, streaming TTS, with end-of-utterance detection and barge-in support for natural turn-taking.
When to use it: You need voice-first conversational AI with low perceived latency.
Pattern details
| Pattern class | multimodal |
|---|---|
| Difficulty | advanced |
| Topology | pipeline |
| Also known as | voice ai stack, asr-llm-tts |
| Last reviewed | 2026-06-07 |
Known failure modes
- Latency budget — Total round-trip exceeds 800ms. Mitigation: Stream every layer; cache TTS prefixes.
- Barge-in lag — TTS keeps speaking after user starts. Mitigation: Wire VAD events directly to TTS muter.
When not to use it
- Non-streaming TTS in realtime voice.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Speech Pipeline Stack — Architecture Pattern.
Frequently asked questions
When should I adopt the Speech Pipeline Stack?
You need voice-first conversational AI with low perceived latency.
What are common failure modes of Speech Pipeline Stack?
Latency budget • Barge-in lag
Is Speech Pipeline Stack production-ready?
Yes when paired with the safety controls and observability hooks documented on the pattern page.