
kaeso
app.kaeso/kaeso
Manage a self-hosted Kaeso AI gateway: projects, API keys, models, routes, limits and usage.
Documentation
The open-source AI gateway you run yourself
One OpenAI-, Anthropic-, Gemini- and ElevenLabs-compatible API in front of every provider and your own model servers, for chat, embeddings, reranking, decisions, images, video, speech, transcription and music. Single sign-on, SCIM, roles, budgets, guardrails, alias routes, key pools, request logs, a playground and an MCP portal come with it, under the MIT license, with no paid edition.
Website · Documentation (Deutsch, 简体中文) · Live demo · Articles · Changelog
Install
On a Linux server or your computer with Docker and Compose 2.24 or newer:
curl -fsSL https://kaeso.app/install.sh | sh
It installs Kaeso with PostgreSQL, ClickHouse and Valkey into ./kaeso, starts it at https://localhost and prints the link that creates the first user. Add -s -- --domain kaeso.example.com to serve your own domain. The quick start continues from there, and install by hand sets up the same Compose package step by step.
On Kubernetes, each release publishes a Helm chart next to its images (Kubernetes):
helm install kaeso oci://ghcr.io/logicplanes/charts/kaeso \
--namespace kaeso --create-namespace --values values.yaml --wait
On a cloud or a hosting platform, the platform templates run the same release:
Render · Railway · DigitalOcean · Fly.io · AWS · Azure · Google Cloud · Coolify · Dokploy · CapRover · Kubernetes
Updates install from the console's Updates page or with the installer's --update, and check the install's settings before they switch anything (upgrades).
Send your first request
Point the OpenAI SDK at Kaeso and call any model in your project, Claude included:
from openai import OpenAI
client = OpenAI(base_url="https://kaeso.example.com/v1", api_key="sk-kso-…")
print(client.responses.create(model="claude-haiku-5-5", input="Hello").output_text)
The Anthropic, Google Gen AI, ElevenLabs and Cohere SDKs work the same way with their base URL pointed at Kaeso. Kaeso translates the request and the reply between the formats (calling models).
Built in are Amazon Bedrock, Anthropic, AssemblyAI, Azure, Baidu Qianfan, Baseten, Cartesia, Cerebras, Cloudflare AI Gateway, Cohere, Deepgram, DeepInfra, DeepSeek, ElevenLabs, fal, Fireworks AI, Gemini Enterprise Agent Platform, Google AI Studio, Groq, Hugging Face, IONOS AI Model Hub, Jina AI, MiniMax, Mistral, Moonshot, Nebius Token Factory, Novita AI, NVIDIA, Ollama Cloud, OpenAI, OpenRouter, OVHcloud AI Endpoints, Perplexity, QwenCloud, Replicate, SambaNova, Scaleway, SiliconFlow, SpaceX (xAI), Standard Compute, StepFun, Tencent TokenHub, Together AI, TypeSafe, Upstage, Venice, Vercel, Volcengine Ark, Voyage AI and Z.ai (supported providers). A server of your own, such as vLLM, Ollama, LM Studio or llama.cpp, joins as a custom provider.
What you get
| One format in, any provider out Apps speak OpenAI Chat Completions or Responses, Anthropic Messages, Gemini or ElevenLabs; Kaeso encodes each request for the provider and translates the reply and the stream back (format translation). | More than chat Embeddings, reranking in Cohere's format, decision models that answer typed questions with probabilities, images, video, speech, transcription, sound effects, music and OpenAI-format batches, with the same keys, limits and logs (API reference). |
| Routes that hold up Alias routes with conditions, weighted splits and fallbacks, retries after dropped connections and 5xx answers, key pools that spread a project over several keys of one provider, and a response cache (alias routes). | Model lifecycle Replies from retiring models carry Deprecation and Sunset headers, projects see who still calls them, aliases move to the replacement, and model lines such as claude-opus-latest follow the newest model (model lifecycle). |
| People and access Single sign-on per organization over OpenID Connect for any number of people, roles and projects from your identity provider's groups, SCIM 2.0, organization and project roles with single rights, and audit logs (members and roles). | Spend under control Hard or soft budgets and rate limits per project, member, service account or key, checked before the provider is called; prices from KPMR, shown in your currency; alerts by email or signed webhook (limits and budgets). |
| Guardrails Built-in rules find keys, tokens, email addresses, IBANs and card numbers and mask or block them, and your own HTTP services can allow, block or rewrite a request before the model sees it (guardrails). | Logs you decide on Every request with its model, status, tokens, cost, timing and caller; content only where the organization, the project or the request allows it; OpenTelemetry traces and Prometheus metrics for your own tools (request logs). |
| An MCP portal per project Connect your MCP servers once, review their tools, and let claude.ai, ChatGPT, Cursor or any MCP client call them through one signed-in portal (MCP portal). | Verifiable releases Every image, chart and release file carries signed build provenance that gh attestation verify checks, and the updater installs nothing whose signatures don't match (upgrades). |
New in 0.7
Decision models, reranking, guardrails, webhooks, SCIM, key pools, batches and the Helm chart arrived in 0.7, with new providers from Amazon Bedrock and Azure to Deepgram and fal. The release article walks through them, and the changelog lists every change.
|
|
|
|
|
|
The console
The pages below show the console with sample data. The live demo is a read-only install to try it yourself.
Analytics |
Request logs |
Models |
Alias routes |
Limits and budgets |
Logging |
How it works
flowchart LR
accTitle: How Kaeso sits between apps and providers
accDescr: Apps, SDKs and coding tools call the gateway in the OpenAI, Anthropic, Gemini or ElevenLabs format, and MCP clients call its MCP portal. The gateway resolves the model, checks rights, limits and guardrails, calls the provider in its own format, and writes usage and logs to PostgreSQL, ClickHouse and Valkey, which the console reads.
apps[Apps, SDKs and coding tools]:::outside
mcp[MCP clients]:::outside
subgraph install [Your Kaeso install]
direction LR
gateway[Gateway]:::accent
console[Console]
data[(PostgreSQL, ClickHouse, Valkey)]
gateway --- data
console --- gateway
end
guardrails[Your guardrail services]:::outside
providers[Model providers]:::outside
servers[Your MCP servers]:::outside
apps -- OpenAI, Anthropic, Gemini or ElevenLabs format --> gateway
mcp -- MCP --> gateway
gateway -- signed HTTP --> guardrails
gateway -- the provider's format --> providers
gateway -- MCP --> servers
The gateway is one Rust process with no garbage collector between your app and the provider. Run several behind Caddy or a load balancer and they share everything a request depends on (multiple gateways). The gateway benchmark in this repository measures the time and memory Kaeso adds, so you can run it on your own hardware.
How it compares
These tables compare Kaeso with the gateways people most often weigh it against, as their own documentation, pricing pages and licenses describe them. They were read on 21 November 2026 and every entry is sourced below. Products change: open an issue when an entry is out of date, and we'll correct it.
Yes means the free or open-source version has it. Paid means a paid plan or enterprise license has it. Partial is explained in a note. – means we didn't find it in their documentation.
Running it yourself
| On your servers | License | Written in | Web console | Docker Compose | Helm chart | |
|---|---|---|---|---|---|---|
| Kaeso | Yes | MIT | Rust | Yes | Yes | Yes |
| LiteLLM | Yes | MIT, except enterprise/1 | Python2 | Yes | Yes | Yes |
| Bifrost | Yes3 | Apache 2.0 | Go | Yes | Yes | Yes |
| Portkey | Yes | MIT | TypeScript | Logs only | Yes | Paid4 |
| Helicone | Partial5 | Apache 2.0 | TypeScript | Yes | Yes | Yes |
| Kong AI Gateway | Yes | Apache 2.0, AI plugins paid6 | Lua | Yes | Yes | Yes |
| Agent Router (was Envoy AI Gateway) | Yes | Apache 2.0 | Go | – | Yes | Yes |
| OpenRouter | Hosted only | – | – | Hosted | – | – |
| Cloudflare AI Gateway | Hosted only | – | – | Hosted | – | – |
Teams and control
| Single sign-on | SCIM | Roles | Audit log | Budgets | PII and secret rules | Your own guardrail service | |
|---|---|---|---|---|---|---|---|
| Kaeso | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| LiteLLM | Up to 5 users | Paid | Partial7 | Paid | Yes | Partial8 | Yes |
| Bifrost | Paid | Paid | Paid | Paid | Yes | Paid | Partial9 |
| Portkey | Paid | Paid | Paid | Paid | Paid | Paid | Yes |
| Helicone | Paid | – | – | – | Partial10 | – | – |
| Kong AI Gateway | Paid | Paid | Paid | Paid | Paid | Paid | Paid |
| Agent Router | – | – | – | – | Partial11 | – | Partial12 |
| OpenRouter | Paid | Paid | Partial13 | Paid | Yes14 | Yes | – |
| Cloudflare AI Gateway | Yes | Paid | Partial15 | Yes | Yes | Partial16 | – |
APIs your apps can call
| OpenAI Chat Completions and Responses | Anthropic Messages | Gemini | ElevenLabs | Rerank | OpenAI batches | |
|---|---|---|---|---|---|---|
| Kaeso | Yes | Yes | Yes | Yes | Yes | Partial17 |
| LiteLLM | Yes | Yes | Yes | – | Yes | Yes |
| Bifrost | Yes | Yes | Yes | Yes | Yes | Yes |
| Portkey | Yes | Yes | Partial18 | Partial19 | Yes | Paid |
| Helicone | Yes | Partial20 | Partial20 | – | – | – |
| Kong AI Gateway | Partial21 | Paid21 | Yes | – | Paid21 | Paid21 |
| Agent Router | Yes | Yes | – | – | Yes | – |
| OpenRouter | Yes | Yes | – | – | Yes | Yes |
| Cloudflare AI Gateway | Yes | Yes | Yes22 | Yes22 | Partial23 | – |
Media
| Images | Video | Speech | Transcription | Music | |
|---|---|---|---|---|---|
| Kaeso | Yes | Yes | Yes | Yes | Yes |
| LiteLLM | Yes | Yes | Yes | Yes | Yes |
| Bifrost | Yes | Yes | Yes | Yes | – |
| Portkey | Yes | Yes | Yes | Yes | – |
| Helicone | Partial24 | – | – | – | – |
| Kong AI Gateway | Paid21 | Paid21 | Paid21 | Paid21 | – |
| Agent Router | Yes | – | – | Yes | – |
| OpenRouter | Yes | Yes | Yes | Yes | – |
| Cloudflare AI Gateway | Yes | Yes | Yes | Yes | – |
Routing and operations
| Several keys per provider | Fallbacks | Retries | Model retirements | MCP gateway | Logs without content | |
|---|---|---|---|---|---|---|
| Kaeso | Yes, weighted | Yes | Yes | Yes25 | Yes | Yes |
| LiteLLM | Yes | Yes | Yes | Partial26 | Yes | Yes |
| Bifrost | Yes, weighted | Yes | Yes | Partial27 | Yes | Yes |
| Portkey | Yes, weighted | Yes | Yes | – | Partial28 | Paid |
| Helicone | – | Yes | Yes | – | – | Yes |
| Kong AI Gateway | – | Paid | Paid | – | Paid | Yes |
| Agent Router | Partial29 | Yes | Yes | – | Yes | Yes |
| OpenRouter | Partial30 | Yes | Yes | Partial31 | – | Yes |
| Cloudflare AI Gateway | Partial32 | Yes | Yes | – | –33 | Yes |
Paid plans, as published
| Product | Paid plans |
|---|---|
| Kaeso | None. Everything above is in the MIT-licensed release; there is no paid edition or support contract. |
| LiteLLM | Enterprise license, priced on request. |
| Bifrost | Enterprise, custom pricing. |
| Portkey | Pro at $49 a month and Enterprise at custom pricing; Portkey is now offered by Palo Alto Networks as Prisma AIRS AI Gateway. |
| Helicone | Pro at $79 and Team at $799 a month, Enterprise on request. |
| Kong AI Gateway | Konnect Plus from $25 a month plus usage, with AI model proxies at $100 a month each; Enterprise at custom pricing. |
| Agent Router | None; an Apache 2.0 project. |
| OpenRouter | A 5.5% fee on credits on Standard and 8% on Business, Enterprise on request. |
| Cloudflare AI Gateway | Core features free; a 5% fee on Unified Billing credits; logs billed as Workers Logs. |
When another tool fits better
Kaeso runs as containers next to PostgreSQL, ClickHouse and Valkey, on at least 2 vCPUs and 4 GB of memory (requirements), and it has no hosted version and no paid support. If you want a hosted service with nothing to run, OpenRouter or Cloudflare AI Gateway fit better, and OpenRouter bills hundreds of models through one account. If your platform team already runs Kong or Envoy Gateway, Kong AI Gateway or Agent Router put model traffic under the same policies and Kubernetes resources. LiteLLM serves more kinds of endpoints, such as realtime, OCR and vector stores, and comes as a Python SDK as well as a proxy. Helicone goes further into observability, with sessions, evaluations, datasets and prompt management. If you need a vendor contract, LiteLLM, Bifrost, Portkey, Helicone and Kong sell enterprise plans.
TensorZero is not in the tables: its website says it is no longer maintained.
Notes
- The repository's license is MIT except for
enterprise/folders, whose code needs a BerriAI Enterprise license in production. - GitHub lists Python as the repository's main language; its description reads "Rust core with Python SDK".
- Running several open-source Bifrost nodes against one PostgreSQL is not supported; clustering is part of Enterprise.
- The official chart pulls enterprise images with credentials Portkey shares.
- The self-hosted stack forwards OpenAI and Anthropic traffic; Vertex AI, Bedrock and Azure OpenAI are not supported there, and the AI gateway is documented as a cloud endpoint.
- Kong Gateway is Apache 2.0; most AI plugins, such as AI Proxy Advanced, AI Sanitizer and AI MCP Proxy, are part of AI Gateway Enterprise.
- Global proxy roles carry no enterprise label; roles scoped to organizations or teams are a premium feature.
- PII masking with Presidio is in the open-source version; secret detection and redaction are enterprise-only.
- No webhook guardrail in the open-source version; Go or WASM plugins can inspect requests, and named guardrail vendors are Enterprise.
- Cost-based rate limits per user or property, set with a request header.
- Token quotas per model, not money.
- Envoy's external authorization hook, not a guardrail feature of its own.
- Organization admins and members; workspace roles need a higher plan.
- The workspace budgets page says budgets are on every plan; the pricing page lists them as not included on Free.
- Cloudflare account roles and API token permissions; no role for AI Gateway members alone.
- Data loss prevention flags or blocks prompts and replies; two predefined profiles without a Zero Trust subscription, no masking documented.
- For providers whose batch API takes OpenAI's batches as they are, today OpenAI.
- Google Gen AI SDK requests pass through for Gemini models on Vertex AI only.
- Speech and transcription go through OpenAI-style
/v1/audioendpoints; other ElevenLabs APIs through a custom host. - Through a provider-specific proxy that Helicone maintains but no longer develops.
- Documented for AI Proxy from Kong Gateway 3.10 or 3.11; the open-source releases on GitHub are 3.9. The open-source AI Proxy takes Chat Completions.
- The provider's own API, passed through to that provider only.
- Cohere's API passes through to Cohere.
- One image model, through Google AI Studio.
Deprecation,SunsetandLinkheaders, 410 after the shutdown, and aliases that move to the replacement.- Deprecation reminders 30 days and 1 day before a provider's date, with a recommended replacement.
- Requests to a retired model are recognized and handled like other failures; no warnings or move to a replacement are documented.
- In Portkey's hosted and enterprise platform, not in the open-source gateway.
- Weighted backends, each with its own credentials.
- Several of your own keys, tried in the order you set; no weights.
- The models API returns a model endpoint's deprecation date.
- Several keys per provider, chosen per request by an alias header.
- MCP server portals are a separate Cloudflare One product.
Sources, all read on 21 November 2026
Kaeso: license, installation, Kubernetes, single sign-on and SCIM, roles, audit log, budgets, guardrails, model API, ElevenLabs API, rerank, batches, images, videos, audio, key pools, fallbacks and retries, model lifecycle, MCP portal, keeping content out of the log, requirements.
LiteLLM: license, enterprise license, repository, pricing, enterprise features, Docker, Helm, SSO, SCIM, roles, audit logs, budgets, PII masking, generic guardrail API, supported endpoints, Responses, Anthropic format, generateContent, pass-through, rerank, batches, images, speech, transcription, music, load balancing, fallbacks and retries, deprecation alerts, MCP gateway, logging.
Bifrost: license, repository, pricing, setup, multiple nodes, Helm, SSO and SCIM, RBAC, audit logs, budgets, guardrails, plugins, OpenAI, Anthropic, Google GenAI, ElevenLabs, Cohere, batches, media, video, keys, retries and fallbacks, MCP, content logging.
Portkey: license, repository, deployment, Helm chart, plans and features, SSO, audit logs, budgets, PII redaction, webhook guardrails, Responses, Messages, Vertex AI, ElevenLabs, rerank, batches, images, video, speech, transcription, load balancing, fallbacks, retries, MCP gateway, request logging, Prisma AIRS AI Gateway, where portkey.ai/pricing leads.
Helicone: license, repository, pricing, self-hosting with Docker, Kubernetes, custom rate limits, gateway, Responses, Anthropic proxy, Gemini proxy, gateway OpenAPI, batches, images, provider routing, retries, MCP, headers.
Kong AI Gateway: license, repository, Helm charts, pricing, AI Proxy, AI Proxy Advanced, open-source AI Proxy formats, AI Rate Limiting Advanced, AI Sanitizer, AI Custom Guardrail, load balancing, AI MCP Proxy, payload logging.
Agent Router: license, repository, installation, CLI, security, quota policy, supported endpoints, API, provider fallback, MCP, tracing. The former address, aigateway.envoyproxy.io, redirects there.
OpenRouter: pricing, FAQ, SSO, SCIM and audit log, workspaces, spend controls, sensitive information, guardrails, compatibility, Anthropic Messages, rerank, your own keys, fallbacks, provider selection, models API, MCP server, data collection.
Cloudflare AI Gateway: pricing, dashboard SSO, SCIM, roles, audit logs, spend limits, guardrails, REST API, Google AI Studio, ElevenLabs, Cohere, your own keys, fallbacks, request handling, MCP server portals, logging.
TensorZero: website, repository, archived on GitHub.
Works with
| Tool | How |
|---|---|
| Claude Code and Codex | Kaeso is their model provider, so every session is logged and budgeted |
| The Kaeso plugins for Claude Code and Cursor | Manage Kaeso through its MCP server, from keys to limits; the Claude Code plugin's skill connects the server for you |
| claude.ai, ChatGPT and any MCP client | Call your MCP servers' tools through a project's MCP portal |
| The OpenAI, Anthropic, Google Gen AI and ElevenLabs SDKs, in any language, and anything with a base URL setting | Point the base URL at Kaeso; supported providers lists what each provider can do through each API |
| OpenTelemetry collectors and Prometheus | Receive a trace per request and the gateway's metrics (monitoring) |
In this repository
| Part | What it is |
|---|---|
crates/ | The gateway, the updater and KPP, the protocol every request is decoded to; kaeso-kpp and kaeso-codecs are on crates.io |
apps/console | The web console |
packages/ui and packages/tokens | The UI library and design tokens behind it, on npm, with a showcase |
apps/docs | The documentation at docs.kaeso.app, in English, German and Chinese |
deploy/, render.yaml, .railway/ and .do/ | The Compose package, the Helm chart, the container images and the templates for Render, Railway, DigitalOcean, Fly.io, AWS, Azure, Google Cloud, Coolify, Dokploy and CapRover |
plugins/ | The Claude Code and Cursor plugins for Kaeso's MCP server |
Architecture explains how the parts fit, and contributing how to work in them.
Community
- Discussions for questions, ideas and what you built.
- Articles on kaeso.app for releases and guides, and @kaesoapp on X for news.
- Good first issues to start contributing, with the contributing guide.
License
MIT. The Kaeso name and logo are not covered by the license. All trademarks belong to their owners; Kaeso is not affiliated with any provider, client or product it names, including those in the comparison. Third-party notices, trademarks included, are in NOTICE.md.
No installation packages available.