> ## Documentation Index
> Fetch the complete documentation index at: https://doc.rapida.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Listen

> Configure how Rapida captures, cleans, detects, and transcribes user speech.

Listen settings control how Rapida handles user audio before the assistant responds. Use this section for `listen.*` runtime options and deployment settings related to speech-to-text, noise cancellation, voice activity detection, and end-of-speech.

```mermaid theme={null}
flowchart LR
    A["User audio"] --> B["Noise cancellation"]
    B --> C["Voice Activity Detection"]
    C --> D["Speech-to-Text"]
    D --> E["End of Speech"]
    E --> F["LLM + tools"]
```

<Info>
  Listen settings are used by voice-capable deployments: Phone Call, Web Widget with microphone input, and Web App / SDK with microphone input. Text-only channels do not use listen configuration.
</Info>

## Configure it

Open your assistant, select **Configure Assistant**, then open **Deployments**. Listen settings appear in the **Voice Input** step for each deployment that supports microphone or phone audio.

| Area                     | What it controls                                                   |
| ------------------------ | ------------------------------------------------------------------ |
| Speech-to-Text           | Provider, credential, model, language, and transcription behavior. |
| Noise cancellation       | Background noise removal before VAD and STT.                       |
| Voice Activity Detection | Speech start/stop detection and barge-in sensitivity.              |
| End of Speech            | Turn completion detection and silence timeout behavior.            |

## Configuration pages

<CardGroup cols={2}>
  <Card title="Speech-to-Text" icon="mic" href="/assistants/speech-to-text">
    Choose the provider, credential, model, and language used to transcribe user speech.
  </Card>

  <Card title="Noise Cancellation" icon="audio-waveform" href="/assistants/noise-cancellation">
    Clean background noise before VAD and STT process the user's audio.
  </Card>

  <Card title="Voice Activity Detection" icon="activity" href="/assistants/voice-activity-detection">
    Tune speech detection, silence frames, and barge-in sensitivity.
  </Card>

  <Card title="End of Speech Detection" icon="clock" href="/assistants/end-of-speech">
    Decide when the user has finished a turn and the assistant should respond.
  </Card>
</CardGroup>

## Recommended starting point

| Area               | Start with                                                                                 |
| ------------------ | ------------------------------------------------------------------------------------------ |
| STT                | A streaming provider and model that matches your channel audio.                            |
| Noise cancellation | RNNoise enabled for phone calls and noisy browser environments.                            |
| VAD                | Silero VAD.                                                                                |
| EOS                | Pipecat Smart Turn for natural conversations, or Silence-Based for simple IVR-style flows. |

<Tip>
  Tune listen settings from real conversation logs. If a caller gets cut off, start with EOS and VAD. If transcription is wrong, check language, audio quality, noise cancellation, and STT model.
</Tip>

## Runtime overrides

You can override listen settings for a single outbound phone call by passing `listen.*` keys in `CreatePhoneCallRequest.options`. Runtime overrides do not update the saved deployment.

```json theme={null}
{
  "options": {
    "listen.language": "en",
    "listen.model": "nova-3",
    "listen.threshold": 0.75,
    "listen.audio.encoding": "mulaw",
    "listen.audio.sample_rate": 8000,
    "listen.smart_format": true,
    "listen.filler_words": false,
    "listen.endpointing": "300"
  }
}
```

| Option key                 | Type                  | Description                                                                           |
| -------------------------- | --------------------- | ------------------------------------------------------------------------------------- |
| `listen.language`          | `string`              | Primary transcription language or provider language code.                             |
| `listen.model`             | `string`              | Provider-specific STT model for this call.                                            |
| `listen.threshold`         | `number`              | Speech detection threshold used by supported providers.                               |
| `listen.audio.encoding`    | `string`              | Input audio encoding, such as `mulaw` or `pcm_s16le`, when supported by the provider. |
| `listen.audio.sample_rate` | `number`              | Input audio sample rate, such as `8000` for telephony audio.                          |
| `listen.region`            | `string`              | Provider region for services that support regional endpoints.                         |
| `listen.smart_format`      | `boolean`             | Enables provider smart formatting when supported.                                     |
| `listen.filler_words`      | `boolean`             | Controls filler-word transcription when supported.                                    |
| `listen.vad_events`        | `boolean`             | Enables provider VAD events when supported.                                           |
| `listen.endpointing`       | `string`              | Provider endpointing or silence setting.                                              |
| `listen.multichannel`      | `boolean`             | Enables multichannel transcription when supported.                                    |
| `listen.keyword`           | `string` / `string[]` | Provider keyword boosting terms.                                                      |
| `listen.operating_point`   | `string`              | Provider accuracy/latency tier, such as Speechmatics operating point.                 |
| `listen.query_params`      | JSON object string    | Custom STT provider query parameters.                                                 |
| `listen.request_rules`     | JSON array string     | Custom STT request DSL rules.                                                         |
| `listen.response_rules`    | JSON array string     | Custom STT response DSL rules.                                                        |

## Troubleshooting map

| Symptom                                            | First place to look                                                                                                       |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Assistant responds before the user is done         | [End of Speech Detection](/assistants/end-of-speech)                                                                      |
| Assistant interrupts on coughs or background noise | [Voice Activity Detection](/assistants/voice-activity-detection) and [Noise Cancellation](/assistants/noise-cancellation) |
| Transcript is wrong or incomplete                  | [Speech-to-Text](/assistants/speech-to-text)                                                                              |
| Phone calls behave differently from web sessions   | Deployment-level Voice Input settings                                                                                     |

## Related

<CardGroup cols={2}>
  <Card title="Experience" icon="sliders-horizontal" href="/assistants/configuration/experience">
    Configure greeting, idle timeout, error message, and session duration.
  </Card>

  <Card title="Speak" icon="volume-2" href="/assistants/configuration/speak">
    Configure text-to-speech and spoken output.
  </Card>

  <Card title="Phone Call Deployment" icon="phone" href="/voice-deployment-options/phone">
    Configure required voice input for phone calls.
  </Card>

  <Card title="Web App / SDK Deployment" icon="monitor" href="/voice-deployment-options/web-app">
    Configure optional voice input for custom apps.
  </Card>
</CardGroup>