Skip to Content
AI GatewayUnified API

Unified API

The unified surface lives under /gateway/v1. Chat Completions is the recommended path for frameworks and SDKs. Responses is a supported peer for OpenAI-shaped agentic flows.

Interactive OpenAPI: https://production-api.exemplar.dev/gateway/docs.

When to use Unified vs passthrough

Use Unified /gateway/v1 when…Prefer passthrough / Cursor when…
One client should reach many providersThe client is Anthropic-native (Claude Code, Anthropic SDK) → /gateway/anthropic
Frameworks speak Chat Completions (provider/model ids)The client is Cursor Override Base URL → /gateway/cursor
You want Console failover routes, shared analytics, and one EXEMPLAR_GATEWAY_URLYou must keep a single-vendor native path (e.g. OpenAI-only Agents SDK) → /gateway/openai/v1

End-user decision guide: Choose your surface. Passthrough examples (Agents SDK, Agno, ADK): Provider passthrough.

Routes

RouteRole
POST /gateway/v1/chat/completionsRecommended universal Chat Completions
POST /gateway/v1/responsesResponses API (peer)
POST /gateway/v1/embeddingsEmbeddings
GET /gateway/v1/modelsModel list (?provider=openai optional)
POST /gateway/v1/responses/input_tokensToken count (OpenAI/Azure native)
POST /gateway/v1/responses/compactContext compaction (OpenAI/Azure native)
curl https://production-api.exemplar.dev/gateway/v1/chat/completions \
  -H "Authorization: Bearer $EXEMPLAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hello from the Exemplar gateway"}]
  }'

Same endpoint works across providers—change only model (e.g. anthropic/claude-sonnet-4-6, groq/llama-3.3-70b-versatile).

Responses API (peer)

Use when you want OpenAI Responses-shaped requests (input, max_output_tokens, etc.). Prefer Chat for universal SDK compatibility.

curl https://production-api.exemplar.dev/gateway/v1/responses \
  -H "Authorization: Bearer $EXEMPLAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "input": "Write a one-paragraph incident update",
    "max_output_tokens": 256
  }'
res = gateway.responses.create(
    model="openai/gpt-4o-mini",
    input="Write a one-paragraph incident update",
    max_output_tokens=256,
)
print(getattr(res, "output_text", res))

On OpenAI Responses, prefer max_output_tokens ≥ 16. Very small values can be rejected upstream as invalid requests. Streaming is supported ("stream": true); durable stream failover/cache is covered under streaming resilience work.

Streaming

Set "stream": true on Chat or Responses. Clients receive text/event-stream.

curl -N https://production-api.exemplar.dev/gateway/v1/chat/completions \
  -H "Authorization: Bearer $EXEMPLAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "stream": true,
    "messages": [{"role": "user", "content": "Count to five"}]
  }'

Streaming always bypasses the response cache. Rate-limit TPM is still debited at stream end from captured usage events—see Rate limits.

Embeddings and models

# Embeddings
curl https://production-api.exemplar.dev/gateway/v1/embeddings \
  -H "Authorization: Bearer $EXEMPLAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/text-embedding-3-small","input":"hello"}'
 
# Model list (optional provider filter)
curl "https://production-api.exemplar.dev/gateway/v1/models?provider=openai" \
  -H "Authorization: Bearer $EXEMPLAR_API_KEY"

Errors

Expect JSON error bodies with upstream status codes (4xx/5xx). A plain Cloudflare error code: 502 without JSON usually means edge/origin failure—not a normal provider validation error.

Gateway-enforced rate limits return 429 with Retry-After and do not enter failover—see Rate limits.

Last updated on
Choose your surfaceProvider passthrough