> ## Documentation Index
> Fetch the complete documentation index at: https://doc.rapida.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom TTS

> Configure a custom WebSocket text-to-speech provider in Rapida.

Custom TTS lets you connect Rapida to any WebSocket-based speech synthesis service without writing a new Go transformer. The UI stores provider credentials and assistant options; `assistant-api` reads them through the `custom-tts` transformer and maps LLM text packets and provider audio frames with WebSocket DSL rules.

**Provider identifier:** `custom-tts`

***

## Where it is configured

**1. Create the provider credential**

In the Rapida dashboard, open **Integrations > Models**, choose **Custom TTS**, and create a credential.

| Credential field | Required | Value |
| - | -: | - |
| `apiCompatibility` | No | `websocket_v1`. Defaults to `websocket_v1` when omitted. |
| `baseUrl` | Yes | WebSocket URL for your TTS service, for example `wss://tts.example.com/v1/speak` |
| `headers` | No | Header map sent on the WebSocket handshake, for example `{"Authorization":"Bearer sk_..."}` |

The backend also accepts snake case keys: `api_compatibility` and `base_url`.

**2. Select Custom TTS on the assistant**

Open the assistant voice settings and select **Custom TTS** as the text-to-speech provider.

**3. Fill the TTS arguments**

Set the audio format, query parameters, request rules, and response rules. The argument reference below maps to the UI fields and the transformer option keys.

***

## TTS arguments

The custom TTS transformer reads the option keys shown below. The current UI exposes the audio and DSL fields; `speak.voice.id`, `speak.model`, and `speak.language` are optional API/import metadata keys that request rules and query parameters can read when present. The UI may preserve other speaker-level settings under `speaker.*`, but the custom TTS DSL fields are unprefixed `speak.*` keys.

| Option key | Required | Default | Description |
| - | -: | - | - |
| `speak.voice.id` | No | Empty | Optional provider voice identifier. Reference it in DSL as `voice_id` or `config.voice.id`. |
| `speak.model` | No | Empty | Optional provider model identifier. Reference it in DSL as `model` or `config.model`. |
| `speak.language` | No | Empty | Optional provider language code. Reference it in DSL as `language` or `config.language`. |
| `speak.audio.encoding` | Yes | `LINEAR16` | Audio encoding expected back from the provider. Supported values: `LINEAR16`, `MuLaw8`. |
| `speak.audio.sample_rate` | Yes | `16000` | Audio sample rate expected back from the provider. Supported UI values are `8000`, `16000`, `22050`, `24000`, `32000`, `44100`, and `48000`. |
| `speak.query_params` | No | `{}` | Flat JSON object appended to `baseUrl` as query parameters. Values can use DSL expressions. |
| `speak.request_rules` | Yes | See example below | Ordered JSON array that tells Rapida what to send for each outbound packet. Must contain at least one `text` rule. |
| `speak.response_rules` | Yes | None | Ordered JSON array that tells Rapida how to parse provider WebSocket frames into audio, done, or error events. |

***

## Query parameters

Use `speak.query_params` when your provider expects configuration in the WebSocket URL.

Supported variables:

| Variable | Source |
| - | - |
| `message_id` | Current synthesis message ID |
| `voice_id` | `speak.voice.id` |
| `model` | `speak.model` |
| `language` | `speak.language` |
| `encoding` | `speak.audio.encoding` |
| `sample_rate` | `speak.audio.sample_rate` |

```json theme={null}
{
  "voice": { "$var": "voice_id" },
  "model": { "$var": "model" },
  "language": { "$var": "language" },
  "sample_rate": {
    "$cast": "number",
    "value": { "$var": "sample_rate" }
  }
}
```

> [!NOTE]
>
> If your provider needs a voice, model, or language and those values are not present as metadata, set them as static values inside `speak.request_rules`.

***

## Request rules

Request rules are evaluated for normalized TTS packets produced by Rapida.

| Packet | When it is sent | Available paths |
| - | - | - |
| `text` | LLM text is ready for synthesis | `packet.kind`, `packet.message_id`, `packet.text`, `config.voice.id`, `config.model`, `config.language`, `config.audio.encoding`, `config.audio.sample_rate` |
| `done` | The LLM response is complete | `packet.kind`, `packet.message_id`, `packet.text`, `config.*` |
| `interrupt` | User interruption is detected | `packet.kind`, `packet.message_id`, `packet.text`, `config.*` |

Supported outbound frame types are `binary`, `json`, and `text`.

> [!WARNING]
>
> Add an `interrupt` rule if your provider needs an explicit cancel/clear message. Without it, queued provider audio can continue after the user starts speaking.

### One-shot synthesis

Use this when the provider synthesizes each text packet immediately.

```json theme={null}
[
  {
    "when": { "packet": "text" },
    "send": {
      "frame": "json",
      "body": {
        "text": { "$path": "packet.text" },
        "voice_id": "narrator-1",
        "message_id": { "$path": "packet.message_id" },
        "model": "sonic-2",
        "language": "en-US",
        "audio": {
          "encoding": { "$path": "config.audio.encoding" },
          "sample_rate": {
            "$cast": "number",
            "value": { "$path": "config.audio.sample_rate" }
          }
        }
      }
    }
  }
]
```

### Two-step synthesis with done

Use this when the provider expects text first and a final flush/done packet.

```json theme={null}
[
  {
    "when": { "packet": "text" },
    "send": {
      "frame": "json",
      "body": {
        "text": { "$path": "packet.text" },
        "voice_id": "narrator-1",
        "message_id": { "$path": "packet.message_id" }
      }
    }
  },
  {
    "when": { "packet": "done" },
    "send": {
      "frame": "json",
      "body": {
        "type": "done",
        "message_id": { "$path": "packet.message_id" }
      }
    }
  },
  {
    "when": { "packet": "interrupt" },
    "send": {
      "frame": "json",
      "body": {
        "type": "interrupt",
        "message_id": { "$path": "packet.message_id" }
      }
    }
  }
]
```

***

## Response rules

Response rules parse provider WebSocket frames into Rapida audio packets.

Supported inbound frame types:

| Frame | Use when |
| - | - |
| `binary` | The provider streams raw audio frames. |
| `json` | The provider returns JSON messages with base64 audio or status fields. |

Supported emit keys:

| Emit key | Type | Description |
| - | - | - |
| `audio` | Binary bytes or base64-decoded bytes | Audio chunk to play. |
| `message_id` | `string` | Optional message ID associated with the audio or done event. |
| `done` | `boolean` | `true` when synthesis for the message is complete. |
| `error` | Any | Provider error value. When present, Rapida treats the frame as an error. |

### Binary audio responses

```json theme={null}
[
  {
    "when": { "frame": "binary" },
    "emit": {
      "audio": { "$frame": "binary" }
    }
  },
  {
    "when": { "frame": "json", "path": "type", "equals": "done" },
    "emit": {
      "message_id": { "$path": "message_id" },
      "done": true
    }
  },
  {
    "when": { "frame": "json", "path": "type", "equals": "error" },
    "emit": {
      "message_id": { "$path": "message_id" },
      "error": { "$path": "error.message" },
      "done": true
    }
  }
]
```

### JSON base64 audio responses

```json theme={null}
[
  {
    "when": { "frame": "json", "path": "type", "equals": "chunk" },
    "emit": {
      "audio": {
        "$decode": "base64",
        "value": { "$path": "audio" }
      },
      "message_id": { "$path": "message_id" }
    }
  },
  {
    "when": { "frame": "json", "path": "type", "equals": "done" },
    "emit": {
      "message_id": { "$path": "message_id" },
      "done": true
    }
  }
]
```

***

## TTS DSL design

Custom TTS uses a JSON-template DSL with three sections: query parameters, request rules, and response rules. The DSL is intentionally small. It does not run scripts, call functions, concatenate strings, perform regex matching, or read environment variables.

### TTS frame support

| Frame type | Outbound request | Inbound response | Notes |
| - | -: | -: | - |
| `binary` | Yes | Yes | Use inbound binary frames for raw provider audio chunks. |
| `json` | Yes | Yes | Use for provider request payloads, base64 audio chunks, done events, and errors. |
| `text` | Yes | No | Use for outbound provider control frames when required. |

Inbound parsing rules:

* WebSocket message type `2` is treated as `binary`.
* Non-binary messages are parsed as JSON when they contain exactly one valid JSON value.
* Non-JSON messages are treated as `text`, but TTS response rules do not support text response frames.

### TTS operators

Every operator object must contain only that operator and its required field.

| Operator | Where supported | Description |
| - | - | - |
| `$var` | Query parameters | Reads `message_id`, `voice_id`, `model`, `language`, `encoding`, or `sample_rate`. |
| `$path` | Request rules, response rules | Reads a dot path from request scope or a JSON response frame. |
| `$cast` | Query parameters, request rules, response rules | Casts to `string`, `number`, or `boolean`. |
| `$frame` | Response rules | Reads the full current `binary` response frame. |
| `$decode` | Response rules | Decodes a base64 string into bytes. Only `base64` is supported. |

Unsupported TTS operators:

* `$frame: "text"` and `$frame: "json"` are not supported for TTS emit rules.
* `$decode` supports only `base64`.

### Cast behavior

| Cast | Behavior |
| - | - |
| `string` | Converts strings, bytes, numbers, booleans, and null to string form. |
| `number` | Converts JSON numbers, numeric values, or numeric strings to an integer or float. |
| `boolean` | Converts booleans, boolean strings, and numeric values. JSON numbers are accepted as `0` or `1`; typed numeric values use zero as `false` and non-zero as `true`. |

### JSON path behavior

`$path` uses dot-separated paths.

```json theme={null}
{ "$path": "packet.text" }
```

Objects are traversed by key. Arrays are traversed by numeric index.

```json theme={null}
{ "$path": "chunks.0.audio" }
```

Limits:

* Keys containing a literal dot are not addressable.
* Request rules can only read from `config` and `packet`.
* Response rules can use `$path` only with JSON response frames.
* A missing path in `when.path` means the rule does not match.
* A missing path in `emit` or `send.body` is an error.

### Query parameter rules

`speak.query_params` must be a flat JSON object. In the UI, each value must resolve to a string, number, or boolean. The backend also accepts null values and skips them when building the URL. Nested objects and arrays are rejected unless the object is a DSL expression.

```json theme={null}
{
  "voice": { "$var": "voice_id" },
  "model": { "$var": "model" },
  "sample_rate": {
    "$cast": "number",
    "value": { "$var": "sample_rate" }
  }
}
```

The rendered query parameters are appended to `baseUrl`. Existing query parameters in `baseUrl` are preserved unless the same key is rendered by `speak.query_params`.

> [!NOTE]
>
> TTS request rules can read `packet.text`, but `text` is not a supported query parameter variable.

### Request rule shape

`speak.request_rules` is an ordered JSON array. Every matching rule is sent, so one packet can produce multiple WebSocket messages.

| Field | Required | Description |
| - | -: | - |
| `when.packet` | Yes | `text`, `done`, or `interrupt`. |
| `send.frame` | Yes | `binary`, `json`, or `text`. |
| `send.body` | Yes | Static value or DSL expression tree. |

Body validation depends on `send.frame`:

| `send.frame` | `send.body` must resolve to |
| - | - |
| `binary` | Bytes or string. |
| `json` | Valid JSON value. Byte arrays are not valid JSON bodies. |
| `text` | Value convertible to string. |

### TTS request scope

The request scope is the data available to `$path` in request rules.

```json theme={null}
{
  "config": {
    "voice": {
      "id": "voice_123"
    },
    "model": "model-a",
    "language": "en-US",
    "audio": {
      "encoding": "LINEAR16",
      "sample_rate": 16000
    }
  },
  "packet": {
    "kind": "text",
    "message_id": "msg_123",
    "text": "Hello world"
  }
}
```

For `done` and `interrupt`, `packet.text` is present but may be empty.

### Response rule shape

`speak.response_rules` is an ordered JSON array. The first matching rule is evaluated; later rules are skipped for that frame.

| Field | Required | Description |
| - | -: | - |
| `when.frame` | Yes | `binary` or `json`. |
| `when.path` | No | Dot path inside a JSON frame. Must be paired with `when.equals`. |
| `when.equals` | No | Primitive value compared against `when.path`. |
| `emit` | Yes | Object containing supported TTS emit keys. |

Match behavior:

| Frame | Match behavior |
| - | - |
| `binary` | Matches by frame type only. `when.path` and `when.equals` are not allowed. |
| `json` | If `when.path` and `when.equals` are omitted, matches any JSON frame. If provided, both fields are required and compared exactly. |

### TTS emit keys

| Emit key | Type after evaluation | Effect |
| - | - | - |
| `audio` | bytes or string | Emits a TTS audio chunk. |
| `message_id` | string | Associates audio, error, or done with a message. Falls back to the current context ID when omitted. |
| `done` | boolean | Ends synthesis for the message, closes the connection, and emits a TTS end packet. |
| `error` | string | Emits a TTS error. |

### Binary audio response

Use this for providers that stream raw audio as binary WebSocket frames.

```json theme={null}
[
  {
    "when": { "frame": "binary" },
    "emit": {
      "audio": { "$frame": "binary" }
    }
  }
]
```

### JSON base64 audio response

Use `$decode` when the provider returns base64-encoded audio inside JSON.

```json theme={null}
[
  {
    "when": { "frame": "json", "path": "type", "equals": "chunk" },
    "emit": {
      "audio": {
        "$decode": "base64",
        "value": { "$path": "audio" }
      },
      "message_id": { "$path": "request_id" }
    }
  },
  {
    "when": { "frame": "json", "path": "type", "equals": "done" },
    "emit": {
      "message_id": { "$path": "request_id" },
      "done": true
    }
  },
  {
    "when": { "frame": "json", "path": "type", "equals": "error" },
    "emit": {
      "message_id": { "$path": "request_id" },
      "error": { "$path": "error.message" },
      "done": true
    }
  }
]
```

### Text, done, and interrupt recipe

Use this pattern when the provider expects text payloads, an explicit final message, and an explicit cancel message.

```json theme={null}
[
  {
    "when": { "packet": "text" },
    "send": {
      "frame": "json",
      "body": {
        "type": "speak",
        "text": { "$path": "packet.text" },
        "voice": { "$path": "config.voice.id" },
        "request_id": { "$path": "packet.message_id" },
        "audio": {
          "encoding": { "$path": "config.audio.encoding" },
          "sample_rate": {
            "$cast": "number",
            "value": { "$path": "config.audio.sample_rate" }
          }
        }
      }
    }
  },
  {
    "when": { "packet": "done" },
    "send": {
      "frame": "json",
      "body": {
        "type": "done",
        "request_id": { "$path": "packet.message_id" }
      }
    }
  },
  {
    "when": { "packet": "interrupt" },
    "send": {
      "frame": "json",
      "body": {
        "type": "interrupt",
        "request_id": { "$path": "packet.message_id" }
      }
    }
  }
]
```

### Runtime behavior

* The connection URL is built from `baseUrl` and `speak.query_params`.
* Headers are copied from the credential and are not templated.
* The transformer opens a connection per active message/context. A new context closes the previous connection.
* `text` packets open the WebSocket connection if needed.
* `done` and `interrupt` request rules are optional. If no rule exists for that packet, nothing is sent.
* On interruption, Rapida sends the optional `interrupt` rule first, then closes the connection.
* Audio returned by the provider is interpreted as `speak.audio.encoding` and `speak.audio.sample_rate`, then resampled to Rapida's internal audio format when needed.
* If no response rule matches an inbound frame, the frame is ignored.
* If a response emits `error`, Rapida emits a TTS error packet.
* If a response emits `done`, Rapida closes the connection and emits a TTS end packet.

### Current TTS limits

* No regex, contains, starts-with, greater-than, or compound match conditions.
* No string interpolation or concatenation.
* No fallback values inside expressions.
* No dynamic headers or dynamic URL path segments.
* No text response handling for TTS.
* No `$frame: "json"` selector in emit rules.
* `$decode` supports only base64.

***

## Backend mapping

`assistant-api` resolves `custom-tts` in `api/assistant-api/internal/transformer/transformer.go`, then dispatches to the WebSocket v1 implementation in `api/assistant-api/internal/transformer/custom/tts_websocket_v1`.

The WebSocket v1 transformer validates:

* `baseUrl` is present in the credential.
* `speak.audio.encoding` is not empty.
* `speak.audio.sample_rate` is positive.
* `speak.request_rules` contains at least one `text` packet rule.
* `speak.response_rules` contains at least one rule.

> [card] [Custom STT](https://doc.rapida.ai/opensource/services/assistant-api/stt/custom.md)
>
> Configure WebSocket speech-to-text.

> [card] [TTS overview](https://doc.rapida.ai/opensource/services/assistant-api/tts/overview.md)
>
> Transformer interface and supported providers.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.