End of Speech Detection (EOS)

End of Speech (EOS) detection determines when a caller has finished their turn and the assistant should start responding. It is the single most impactful setting for perceived conversation latency — a fast EOS means the assistant responds quickly, but a premature EOS means the assistant cuts the caller off mid-sentence. EOS works downstream of VAD. While VAD detects whether the caller is actively making sound (frame-by-frame), EOS decides whether a period of silence means “I’m done talking” or “I’m pausing to think”.

---

​

Providers

Rapida supports three EOS providers, ranging from a simple silence timer to ML-powered turn detection models.

---

​

Silence-Based EOS

The simplest approach: after the last speech activity, wait for a fixed duration of silence, then trigger end-of-speech. No ML model, no inference overhead. Why choose Silence-Based:

• Zero additional compute — no model to load or run
• Predictable, deterministic behaviour — the timeout is exactly what you configure
• Works with any language, any accent, any audio quality
• Easiest to reason about and debug

When to use: Most deployments, especially when starting out. Silence-based EOS is the default and works well for the majority of voice AI use cases. It is the right choice when you want simplicity and predictability, or when your callers speak in short, clear turns (IVR menus, yes/no questions, appointment booking). When it falls short: Callers who pause mid-sentence (e.g., “I’d like to book a flight to… hmm… London”) will be interrupted if the pause exceeds the timeout. This is where model-based EOS providers add value.

​

Parameters

---

​

Pipecat Smart Turn EOS

Pipecat Smart Turn uses a Whisper-based audio model (~8 MB) to predict whether the caller has finished their turn directly from the speech audio waveform. Unlike silence-based detection, it understands prosodic cues — falling intonation, slowing speech rate, and other acoustic signals that indicate turn completion. Why choose Pipecat Smart Turn:

• Detects turn completion from audio features, not just silence — catches prosodic cues like falling intonation at the end of a sentence
• ~10 ms inference time per prediction — negligible latency impact
• Supports 23 languages out of the box
• Small model size (~8 MB ONNX)
• Uses a rolling audio buffer (~5 seconds) for context — doesn’t need the full conversation history

When to use: Conversations where callers frequently pause mid-sentence. Pipecat Smart Turn significantly reduces premature turn-taking compared to silence-based detection because it can distinguish between a “thinking pause” (flat or rising intonation) and a “finished speaking” pause (falling intonation, complete sentence prosody). Best for: customer support, complex information gathering (addresses, travel bookings), multilingual deployments where pause patterns vary by language. How it works:

1. Audio from the caller is accumulated in a rolling buffer (max ~5 seconds at 16 kHz)
2. When a final STT transcript arrives, the model runs inference on the buffered audio
3. The model outputs a probability between 0 and 1 indicating likelihood of turn completion
4. If probability >= threshold → use quick_timeout (short wait, then fire)
5. If probability < threshold → use silence_timeout (long wait, keep listening)
6. Interim STT transcripts reset the timer with the fallback_timeout

​

Parameters

---

​

LiveKit Turn Detector EOS

The LiveKit Turn Detector uses a language model to predict turn completion from transcribed text combined with conversation history. Unlike Pipecat (which analyzes audio), LiveKit analyzes the linguistic content of what was said to determine if the caller is done. Why choose LiveKit Turn Detector:

• Context-aware — uses conversation history (up to 6 turns by default) to make better predictions. If the assistant asked “What is your address?”, the model knows the caller is likely still speaking during a pause after saying “123 Main Street”
• Text-based analysis — catches semantic cues that audio models miss. For example, “My address is 123” is clearly incomplete, regardless of intonation
• Reduces false triggers on addresses, phone numbers, and lists — the model understands that these naturally contain pauses between segments
• Available in two model variants: English-only (66 MB, optimized) and Multilingual (378 MB, 14 languages)

When to use: Conversations with structured data collection where callers frequently pause mid-answer. The LiveKit model excels at preventing premature turn-taking during:

• Address dictation (“123 Main Street… apartment 4B… New York”)
• Phone numbers (“area code 212… 555… 1234”)
• Lists or multi-part answers
• Complex questions requiring thought

How it works:

1. Final STT transcripts are accumulated into the current user turn
2. When a final transcript arrives, the model builds a chat template from conversation history + current text
3. The model predicts an end-of-utterance probability
4. If probability >= threshold → use quick_timeout
5. If probability < threshold → use silence_timeout
6. Assistant responses (from LLMResponseDonePacket) are recorded in history for context

​

Parameters

---

​

Choosing a provider

​

Decision guide

---

​

How EOS providers interact with VAD

VAD and EOS work together but serve different purposes: The VAD continuously sends speech activity heartbeats while the caller is speaking. These heartbeats reset the EOS silence timer, preventing the EOS from firing while the caller is actively speaking. When speech stops, the VAD stops sending heartbeats, and the EOS timer begins counting down. For the model-based EOS providers (Pipecat and LiveKit), the EOS also receives the final STT transcript. On receiving a final transcript, the model runs inference to decide whether to use the quick timeout (turn complete) or extended timeout (still speaking).

---

​

Next steps