Talaria Documentation

Learn how to integrate Talaria's intelligent model routing engine into your AI stack.

Quickstart — Get started in 5 minutes

This guide walks you through your first Talaria route call. By the end you will have a working integration that selects the optimal LLM for any task.

1. Sign up and get your API key

Visit the registration page and create an account. Once signed in, navigate to the Keys page from the dashboard sidebar. Click Create API Key and copy the generated key — it is shown only once.

2. Connect with the OpenAI client

Talaria exposes a standard OpenAI-compatible API. No special SDK needed — just point any OpenAI client at Talaria's endpoint:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8200/v1",
    api_key="YOUR_TALARIA_KEY"
)

3. Make your first route call

Talaria automatically selects the best model for your task. Just send a standard chat completion request:

response = client.chat.completions.create(
    model="talaria",  # Special model name — triggers routing
    messages=[
        {"role": "user", "content": "Summarize this quarterly report in 3 bullet points."}
    ]
)

print(response.choices[0].message.content)

4. (Optional) Use the HTTP API directly

If you prefer raw HTTP, send a POST request:

curl -X POST http://localhost:8200/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TALARIA_KEY" \
  -d '{
    "model": "talaria",
    "messages": [{"role": "user", "content": "Explain quantum computing simply."}]
  }'

BYOM Registration Guide — Bring Your Own Model

Talaria supports any model that exposes an OpenAI-compatible chat completions API. Whether you are running a local Ollama instance, have a private Anthropic endpoint, or have deployed a fine-tuned model on your own infrastructure, you can register it with Talaria in minutes.

What you need

• Model endpoint URL (must support /v1/chat/completions)
• API key for the provider (if applicable)
• Model identifier string used by the provider
• Pricing information (cost per 1M input tokens and 1M output tokens)

Register via the Dashboard

1. Navigate to Admin → Models in the sidebar.
2. Click Add Model.
3. Fill in the model details: ID, provider name, endpoint URL, API key, and cost data.
4. Optionally set a label for display purposes.
5. Click Save. The model is immediately available for routing.

Register via the Admin API

For programmatic registration, use the admin API endpoint:

POST /admin/models
Content-Type: application/json

{
  "model_id": "my-custom-llama",
  "provider": "custom",
  "api_model_id": "llama-3.1-70b",
  "endpoint": "https://my-llm.internal/v1/chat/completions",
  "api_key": "sk-...",
  "cost_per_1m_in": 0.25,
  "cost_per_1m_out": 1.00
}

Supported Providers

API Reference

Talaria exposes an OpenAI-compatible API at /v1/chat/completions plus several Talaria-specific endpoints for routing and statistics.

Authentication

All API requests require authentication via an API key. Pass the key using the standard Bearer token in the Authorization header:

Authorization: Bearer talaria_xxxxxxxxxxxxxxxxxxxx

Alternatively, you can use the X-Talaria-Key header for compatibility with older integrations. API keys are created and managed in the Talaria dashboard under Keys.

POST /v1/chat/completions

The primary endpoint for model routing. Accepts the same request body as OpenAI's chat completions endpoint, with a special model field:

• model: "talaria" — Triggers automatic routing to the best model for the task.
• model: "talaria-{mode}" — Use a specific routing mode, e.g., talaria-heuristics for fast heuristic-only routing, talaria-llm-classify for LLM-based classification.
• model: "provider/model-id" — Directly invoke a specific model, bypassing routing entirely (e.g., deepseek/deepseek-chat).

Request Parameters

Response

Returns a standard OpenAI chat completion response object. The response includes a model field indicating which model was selected by the router, and a usage field with token counts and cost estimates.

POST /v1/route

The Talaria-specific routing endpoint. Returns a routing decision without executing the model:

POST /v1/route
Content-Type: application/json

{
  "task": "Write a poem about the ocean.",
  "mode": "auto"
}

Response:

{
  "model_id": "deepseek/deepseek-chat",
  "provider": "deepseek",
  "confidence": 0.92,
  "reasoning": "Creative writing task — DeepSeek V3 provides excellent quality at lowest cost",
  "cost_estimate": 0.00042
}

GET /v1/models

List all available models and their capabilities. Returns an array of model objects with provider, cost, and status information.

GET /v1/stats

Per-key usage statistics. Returns total calls, cost, and routing decisions attributed to the API key used for authentication.

Self-Hosted Deployment

Deploy Talaria behind your own firewall for full data sovereignty and compliance. The self-hosted version includes all features of the cloud offering plus on-premise model routing, custom authentication, and direct database access.

Docker Quickstart

docker run -d \
  --name talaria \
  -p 8200:8200 \
  -e TALARIA_ADMIN_PASSWORD="your-admin-password" \
  -e TALARIA_AUTH_DB="/data/auth.db" \
  -e TALARIA_TRAINING_DB="/data/training.db" \
  -v talaria-data:/data \
  talaria/talaria:latest

License Activation

Self-hosted deployments require a license key. After purchasing a Pro or Business plan, you will receive a license key via email. Activate it by setting the TALARIA_LICENSE_KEY environment variable:

-e TALARIA_LICENSE_KEY="tlr_xxxxxxxxxxxxxxxxxxxx"

Without a license key, Talaria runs in evaluation mode with reduced limits (2 models, 500 requests/day).

Environment Variables

Docker Compose

For production deployments, use the provided docker-compose.yml:

version: '3.8'
services:
  talaria:
    image: talaria/talaria:latest
    ports:
      - "8200:8200"
    volumes:
      - talaria-data:/data
    environment:
      - TALARIA_ADMIN_PASSWORD=${TALARIA_ADMIN_PASSWORD}
      - TALARIA_LICENSE_KEY=${TALARIA_LICENSE_KEY}
      - STRIPE_SECRET_KEY=${STRIPE_SECRET_KEY}
volumes:
  talaria-data:

Topology Guide — Parallel Merge and Sequential Routing

Talaria supports two advanced routing topologies beyond single-model dispatch. These topologies let you combine outputs from multiple models for better results.

Parallel Merge

In parallel_merge mode, Talaria dispatches the same task to multiple models simultaneously, then synthesizes the best response from all results. This is ideal for tasks where consensus or diverse perspectives improve quality — brainstorming, content generation, complex analysis.

When to use parallel_merge:

• Creative writing and content generation where variety matters
• Fact-checking and verification — compare responses from different models
• Code review — get suggestions from multiple LLMs and merge the best practices
• Translation — compare translations and select the most natural one

Sequential

In sequential mode, Talaria chains models together: the output of one model becomes input to the next. Each step can use a different model optimized for that specific subtask. This is ideal for multi-step reasoning pipelines.

When to use sequential:

• Research pipelines — summarize a document, then analyze the summary, then generate recommendations
• Code generation — generate code with one model, then review and improve it with another
• Data extraction — extract entities with a fast model, then validate with a more capable model
• Content moderation — classify with a fast model, escalate borderline cases to a stronger model

Admin Guide — Running a Talaria Instance

A walkthrough of every page in the Talaria dashboard. You'll learn how to manage API keys, set budgets, test models, review spend, and keep things running smoothly.

You need an admin account on a Talaria instance. If you're self-hosting, the bootstrap admin password is set via the TALARIA_ADMIN_PASSWORD environment variable when you first launch. Open your browser to your Talaria instance and log in.

Navigation

Dashboard

The dashboard is your home base. It shows what Talaria is doing right now. At the top you'll see stat cards for Routed Spend, Ambient Spend, Registered Models, and live Provider Balances (DeepSeek, Kimi, Anthropic, OpenAI where available).

Click the range buttons (Today, 7d, 30d, 90d, All) to filter spend data. Below the stats, a table breaks down spend by model — calls, cost, percentage, and pass rate. The Quick Route Tester at the bottom lets you type a task and see routing decisions without making actual calls.

API Keys

Go to Settings → Users & Keys. Create keys from the form — pick an owner name, org, and tier. The full key is shown once; copy it immediately. Keys use the format tl_<org>_<owner>_<random> so you can tell at a glance who a key belongs to. Revoke keys to disable them instantly.

Users

The Users page lists everyone who can log into the dashboard — owner name, org, tier, role, and status. Admins create and manage users from here.

Organizations

The Orgs page manages tenant isolation. Each org gets its own models, API keys, providers, and budget. Click into an org to manage its models (add from catalog, set per-model API keys, rate limits, cost overrides, enable/disable) and providers (register custom endpoints with automatic validation).

Budgets

Set spending limits per org to prevent surprise bills. Each budget rule has a hard cap (routing blocked when exceeded), soft cap (warning only), and optional webhook URL for alerts. Spend is tracked in-memory with 5-minute alert deduplication. No budget configured means all requests pass — backward compatible.

Model Catalog

Your registry of available LLMs at Admin → Models. Each entry has a model ID, provider, pricing, complexity range, capabilities, and context window. Add models via the form — they're immediately available for routing and org assignment.

Playground

Test single-model routing interactively. Type a prompt, Talaria shows the routing decision (model, confidence, cost), then click Run to execute. You can also manually pick any registered model instead of auto-routing.

Synthesize

Dispatch the same prompt to multiple models in parallel, then merge results through a synthesizer model. Pick 2+ workers and an optional merge model. See per-worker latency and status, plus the synthesized final output.

Eval

Run test suites against your models to measure quality. Create suites with test cases (prompt + expected output), then run them with a grader: exact_match, contains, json_schema, or llm_judge. Compare multiple models side-by-side with pass rates and per-case breakdowns.

Analytics

Cost and usage insights: summary cards, cost-over-time charts, usage heatmaps, per-key and per-user breakdowns, and spend forecasts. Use the date range picker to zoom in or out.

Logs

The Log Explorer is a searchable, filterable view of every routed request. Filter by date, model, provider, outcome, or text search. Click any row for full request detail including tags and guardrail actions. Paginate with 25/50/100/200 per page.

Prompts

Store and version prompt templates with placeholders. Each edit creates a new version — view history, set the active version, and test-render with variable values. Callers reference prompts by name for consistent system prompts.

Batch

Async model inference. Submit jobs, they run in the background through queued → running → completed (or failed/cancelled). Max 100 concurrent jobs, auto-pruned after 24 hours, in-memory only.

Cache

Two-tier response caching: exact match (same messages + model + temperature) and semantic match (trigram similarity above 95%). Configure via TALARIA_CACHE_TTL (default 3600s) and TALARIA_CACHE_MAXSIZE (default 1000). Streaming responses are not cached.

Guardrails

Safety filters disabled by default — enable with TALARIA_GUARDRAILS=enabled. Includes PII redaction (emails, phones, SSNs, API keys), jailbreak detection (15 regex patterns), topic blocking (malware, ransomware, exploits), and content policy checks on responses. All actions are logged.

Health

Circuit breaker tracking per provider. Three consecutive failures mark a provider unhealthy, with 60-second cooldown before re-checking and sliding window latency tracking. Unhealthy providers trigger automatic fallback to the next best model.

Shadow

Canary traffic — send a percentage of live requests to a new model without affecting users. Configure the shadow model, percentage, and exclusions. After each successful proxy call, a fire-and-forget shadow request runs. Ring buffer stores last 1000 results with match rate and latency stats.

Spend

Detailed per-model spend breakdown with optimization tips — Talaria flags opportunities to save money by routing low-complexity tasks to cheaper models. Shows routed vs ambient spend split.

Quick reference

Frequently Asked Questions