AI Gateway
The AI Gateway is Exemplar’s multi-provider LLM control plane. Point any OpenAI-compatible SDK at one org endpoint, authenticate with an eis_* platform key, and route to OpenAI, Anthropic, Gemini, Groq, and more—with shared analytics, rate limits, failover, and optional response cache.
Marketing capability page: exemplar.dev/marshal/gateway . In the console: AI Gateway (analytics + Integration Guide) and AI Gateway → Management (providers, routes, cache).
Chat Completions (POST /gateway/v1/chat/completions) is the recommended universal surface. Responses (POST /gateway/v1/responses) is a supported peer for OpenAI-shaped agentic flows. Prefer Chat for frameworks and SDKs unless you specifically need Responses.
When to use what
| Goal | Surface | Base |
|---|---|---|
| One OpenAI-compatible client → many providers | Unified API | …/gateway/v1 · model provider/model |
| Anthropic SDK / Claude Code / Claude Agent SDK | Anthropic passthrough | …/gateway/anthropic · bare claude-… |
| Cursor Ask/Plan override | Cursor adapter | …/gateway/cursor |
| OpenAI-only SDK / Agents SDK / Agno OpenAI / ADK→OpenAI | OpenAI passthrough | …/gateway/openai/v1 · bare OpenAI id |
| IDE shell / MCP / prompt policy | Relay (not a model base) | Relay Connect |
Full comparison (failover, model-id rules, mistakes): Choose your surface.
Bases and auth
| Environment | Gateway origin (no trailing slash) | OpenAI-compatible root |
|---|---|---|
| Production | https://production-api.exemplar.dev/gateway | …/gateway/v1 |
Platform key (required) — org API key starting with eis_. Create under Tokens and API keys.
export EXEMPLAR_API_KEY=eis_… # required
export EXEMPLAR_GATEWAY_URL=https://production-api.exemplar.dev/gateway/v1Send the key as Authorization: Bearer $EXEMPLAR_API_KEY or X-API-Key: $EXEMPLAR_API_KEY. Org is resolved from the key.
How a request flows
Rate limits run after auth and before cache and failover. A gateway 429 is terminal—it never triggers provider fallbacks. Upstream 429s remain failover-retryable.
Topics
Related
- Tokens and API keys — create
eis_*keys - Guardrails — budgets, breakers, policy gates beside gateway RPM/TPM
- Tools & MCP — connector tools beside LLM routing
- Relay — Cursor / Claude Code hook governance (shell, MCP, prompts)—pair with
/gateway/cursoror/gateway/anthropicfor model routing - Frameworks with Marshal SDK —
harness.tools()for LangChain, Agno, Google ADK - Marshal SDK — memory, prompts, skills, and agents on the same platform
- Console AI Gateway → Integration Guide — endpoint for this console + short quickstart; full guides on these pages