Integration API

Purpose

The integration-api is the single point of contact with every external AI provider in the platform. It:

Stores provider API keys encrypted at rest using AES-256-GCM
Decrypts credentials in-memory per request (never written to logs or disk)
Executes LLM streaming and non-streaming inference
Generates embeddings for RAG search
Verifies provider credentials before saving them

Port

9004 — HTTP · gRPC (cmux)

Language

Go 1.25 Gin (REST) + gRPC

Storage

PostgreSQL integration_db Redis (provider cache)

integration-api is the only service that decrypts and uses provider API keys. Keys are decrypted in-memory per request and are never written to logs, forwarded to other services, or stored in plaintext anywhere.

Credential Encryption Flow

Supported Providers

LLM
STT
TTS
Telephony

Provider	Notes
OpenAI	GPT-4o, GPT-4, GPT-3.5 · Function calling · Streaming
Anthropic	Claude 3.5 Sonnet, Claude 3 · Tool use · Streaming
Google Gemini	Gemini Pro · Flash · Streaming
Google Vertex AI	Enterprise Gemini deployment
Azure OpenAI	Enterprise GPT with custom endpoint
AWS Bedrock	Llama, Titan, Mistral via AWS
Cohere	Command R+ · Streaming
Mistral	Mistral Large · Small · Streaming
HuggingFace	Inference API
Replicate	Model hosting
VoyageAI	Embeddings and reranking

Provider	Notes
Google Cloud STT	Streaming, 100+ languages
Azure Cognitive Services	Neural Speech, real-time
Deepgram	Nova-2 / Nova-3, low-latency streaming
AssemblyAI	Streaming and batch
Cartesia	Real-time
Sarvam AI	Indian languages

Provider	Notes
Google Cloud TTS	WaveNet / Neural2
Azure Cognitive Services	Neural voices, 140+ languages
ElevenLabs	High-fidelity voice cloning
Deepgram Aura	Fast synthesis
Cartesia	Streaming synthesis
Sarvam AI	Indian languages

Provider	Notes
Twilio	Programmable Voice, Media Streams, bulk calling
Vonage	Voice API, WebSocket audio
Exotel	Cloud telephony — India / SEA

Caller Architecture

Each LLM provider lives under api/integration-api/internal/caller/<provider>/. The directory structure is:

api/integration-api/internal/caller/
├── callers.go              ← interfaces + factory
├── chat_options.go         ← ChatCompletionOptions, ToolDefinition
├── embedding_options.go    ← EmbeddingOptions
├── reranking_options.go    ← RerankerOptions
├── anthropic/
├── azure/
├── cohere/
├── gemini/
├── huggingface/
├── mistral/
├── openai/                 ← reference implementation
├── replicate/
├── vertexai/
└── voyageai/

Each provider package contains:

File	Purpose
`<provider>.go`	Client initialization, credential binding
`llm.go`	`LargeLanguageCaller` implementation
`embedding.go`	`EmbeddingCaller` (where supported)
`verify-credential.go`	Pre-storage credential validation

Running

Docker Compose
From Source

make up-integration

make logs-integration

make rebuild-integration

Requires Go 1.25, PostgreSQL 15, and Redis 7 running locally.

export $(grep -v '^#' docker/integration-api/.integration.env | xargs)
export POSTGRES__HOST=localhost
export REDIS__HOST=localhost
export WEB_HOST=localhost:9001

go run cmd/integration/integration.go

Health Endpoints

Endpoint	Purpose
`GET /readiness/`	Service ready (DB + Redis connected)
`GET /healthz/`	Liveness probe

curl http://localhost:9004/readiness/

Next Steps

Configuration

Environment variables including INTEGRATION_CRYPTO_KEY.

Adding Providers

LargeLanguageCaller interface and how to add a new LLM provider.

Assistant API

How assistant-api calls integration-api during a live voice conversation.

Architecture

Full system topology.

Getting Started

Dashboard (UI)

Web API

Assistant API

Endpoint API

Document API

Integration API

Purpose

Port

Language

Storage

Credential Encryption Flow

Supported Providers

Caller Architecture

Running

Health Endpoints

Next Steps

Configuration

Adding Providers

Assistant API

Architecture

​Purpose

Port

Language

Storage

​Credential Encryption Flow

​Supported Providers

​Caller Architecture

​Running

​Health Endpoints

​Next Steps

Configuration

Adding Providers

Assistant API

Architecture

Purpose

Credential Encryption Flow

Supported Providers

Caller Architecture

Running

Health Endpoints

Next Steps