> ## Documentation Index
> Fetch the complete documentation index at: https://doc.rapida.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom LLM

> Connect Rapida to an OpenAI-compatible or custom LLM endpoint.

Use **Custom LLM** when your model is served from your own endpoint or from a provider that follows an OpenAI-compatible API shape.

## Setup

<Steps>
  <Step title="Create the credential">
    Open **Integrations > Models**, select **Custom LLM**, and create a credential.

    | Field              | Required | Description                                                                                                        |
    | ------------------ | -------: | ------------------------------------------------------------------------------------------------------------------ |
    | `apiCompatibility` |      Yes | Request format. Use `openai_chat_completions`, `openai_compatible`, or `openai_responses` for active chat support. |
    | `baseUrl`          |      Yes | API root for your model server, for example `https://llm.example.com/v1`.                                          |
    | `headers`          |       No | Headers sent with each request, such as `Authorization: Bearer ...`.                                               |
  </Step>

  <Step title="Select the model">
    Open the assistant model settings, select **Custom LLM**, and enter the model ID your server expects.
  </Step>

  <Step title="Configure model parameters">
    Use **Model Parameters** for provider-specific JSON such as temperature, token limits, or extra server options.
  </Step>
</Steps>

## Supported compatibility values

| Value                     | Use for                                                                        |
| ------------------------- | ------------------------------------------------------------------------------ |
| `openai_chat_completions` | Servers compatible with `/v1/chat/completions`.                                |
| `openai_compatible`       | OpenAI-compatible local/model servers such as vLLM, Ollama, LM Studio, or TGI. |
| `openai_responses`        | Servers compatible with `/v1/responses`.                                       |
| `anthropic_messages`      | Credential option only; chat execution is not implemented yet.                 |
| `gemini_generate_content` | Credential option only; chat execution is not implemented yet.                 |

## Example

```json theme={null}
{
  "apiCompatibility": "openai_compatible",
  "baseUrl": "http://localhost:8000/v1",
  "headers": {
    "Authorization": "Bearer local-dev-key"
  }
}
```

```json theme={null}
{
  "model.id": "meta-llama/Llama-3.1-8B-Instruct",
  "model.name": "meta-llama/Llama-3.1-8B-Instruct",
  "model.parameters": {
    "temperature": 0.4,
    "max_tokens": 512,
    "top_p": 0.9
  }
}
```

<Card title="Full Custom LLM reference" icon="network" href="/opensource/services/integration-api/llm/custom">
  Credential fields, model arguments, compatibility notes, and backend mapping.
</Card>