An LLM gateway puts a single proxy in front of every model provider so you get unified keys, fallbacks, caching, and cost tracking in one place. OpenRouter is the fastest on-ramp: one account, one bill, 300+ models with no infrastructure to run. LiteLLM is the right answer when data residency or full control matters, and you are willing to operate Postgres and Redis. Portkey sits in the middle: open-source gateway with a polished managed layer and the deepest guardrails story. The key decision is managed-vs-self-hosted, not which one has the longest feature list.
One API in front of every model: the 8 gateways ranked by routing intelligence, cost control, and self-host depth
Every LLM integration eventually hits the same wall: three different SDKs, four sets of API keys, no shared rate-limit budget, and no single place to see what a week of inference actually cost. Toolradar data: only 298 of the 1,627 developer tools we track are paid-only, under one in five.
An LLM gateway, also called an AI gateway, is a reverse proxy that absorbs that complexity. Your application calls one endpoint; the gateway handles provider auth, routing, retries, semantic caching, and spend tracking on the other side. When a provider goes down at 2 a.m., the gateway fails over silently. When two teams both call GPT-5.5 for the same query, the cache returns the result for free.
The eight tools in this guide represent the serious options in 2026. They differ on three axes that matter: managed cloud versus self-hosted open source, aggregator billing (you pay the gateway, not each provider) versus bring-your-own-keys, and pure routing versus routing-plus-observability. Get those three answers right and the shortlist becomes obvious.
Top Picks
Picked by editorial review, informed by G2 and Capterra review volume and rating and by media mentions, the signals behind our category rankings. How we rate
One API key, one bill, access to 300+ models from every major provider
Starting price
Custom
3.0 (62 reviews)
Best for
Teams that want instant multi-provider access without managing provider accounts or running any infrastructure
Open-source proxy that calls 100+ LLM APIs in OpenAI format, self-hosted on your own infrastructure
Starting price
Free plan
Best for
Engineering teams with strict data residency requirements who are comfortable operating a containerized service with Postgres and Redis
Production-grade AI gateway with built-in guardrails, full observability, and an open-source self-host option
Starting price
From $49 /mo
4.6 (18 reviews)
Best for
Teams that need gateway plus observability in one product and want the option to self-host the routing layer under Apache 2.0
| Tool | Starting price | Rating | Best for |
|---|---|---|---|
| OpenRouter | Custom | 3.0 62 reviews | Teams that want instant multi-provider access without managing provider accounts or running any infrastructure |
| LiteLLM | Free plan | n/a | Engineering teams with strict data residency requirements who are comfortable operating a containerized service with Postgres and Redis |
| Portkey | From $49/mo | 4.6 18 reviews | Teams that need gateway plus observability in one product and want the option to self-host the routing layer under Apache 2.0 |
| Kong AI Gateway | Free plan | 4.2 349 reviews | Enterprises that already operate Kong for API management and want to route LLM traffic through the same platform and governance layer |
| Cloudflare AI Gateway | From $25/mo | 4.3 79 reviews | Teams already building on Cloudflare Workers or Pages who want observability, caching, and rate limiting with minimal setup and no added latency |
| Unify | From $40/mo | 4.7 43 reviews | AI engineering teams running high volumes who want automated model selection rather than manually tuned routing rules |
| Eden AI | Custom | 3.0 7 reviews | Teams building pipelines that combine text generation with OCR, speech-to-text, translation, or image analysis and want one vendor relationship for all of it |
| Helicone | From $100/mo | n/a | Teams already using Helicone who need its observability depth and are comfortable with a platform that is no longer receiving new features |
OpenRouter
One API key, one bill, access to 300+ models from every major provider
- 1.7 on Trustpilot (41 reviews)
- 4.6 on G2 (21 reviews)
- Starting price
- Custom
- Free plan
- No
- Rating
- 3.0 (62 reviews)
Best for: Teams that want instant multi-provider access without managing provider accounts or running any infrastructure
- Pro: 300+ models available immediately including Claude, GPT, Gemini, DeepSeek, Llama, and Grok, all through one OpenAI-compatible endpoint
- Pro: Aggregated billing means one credit balance and one invoice across all providers, which simplifies finance and procurement
- Pro: Dozens of free models with no per-token cost, useful for development and low-volume workloads
- Con: All request data transits OpenRouter's cloud, so it does not satisfy strict data residency or air-gapped compliance requirements
- Con: Pricing markup over direct provider APIs varies by model and is not always transparent at a glance
OpenRouter's 'Pay as you go' model is inherently fair and generous, as users only pay for what they consume, with no upfront costs or commitments.
LiteLLM
Open-source proxy that calls 100+ LLM APIs in OpenAI format, self-hosted on your own infrastructure
- Starting price
- Free plan
- Free plan
- Yes
Best for: Engineering teams with strict data residency requirements who are comfortable operating a containerized service with Postgres and Redis
- Pro: MIT-licensed software means zero platform fee and full code transparency, which matters for compliance and audit trails
- Pro: Supports 100+ providers including Bedrock, Azure, VertexAI, and local models via a single OpenAI-compatible interface
- Pro: Built-in virtual key management, per-key budget caps, load balancing, and OpenTelemetry-compatible logging
- Con: You own the entire operational burden: provisioning Redis for caching, Postgres for spend logs, container orchestration, and upgrades
- Con: Enterprise features like SSO, RBAC, and team-level budget enforcement require a paid enterprise license rather than the open-source build
LiteLLM's pricing is extremely generous, offering a fully-featured 'Open Source' tier for $0.
- Starting price
- From $49/mo
- Free plan
- Yes
- Rating
- 4.6 (18 reviews)
Best for: Teams that need gateway plus observability in one product and want the option to self-host the routing layer under Apache 2.0
- Pro: Routing to 1,600+ models combined with guardrails (PII redaction, jailbreak detection, content moderation) at the proxy layer, not bolted on afterward
- Pro: Full observability suite including request traces, cost dashboards, session analytics, and latency percentiles without a separate tool
- Pro: Gateway core is open-source under Apache 2.0 since March 2026, so you can self-host routing without paying the managed platform fee
- Con: The managed free tier is limited to prototyping scale; production use requires a paid subscription
- Con: The breadth of features (gateway plus guardrails plus observability) means a steeper initial configuration than a pure proxy like Cloudflare AI Gateway
Portkey's pricing is fair, offering a generous free Developer tier with 10k requests/month.
Kong AI Gateway
Enterprise API gateway extended to LLM traffic, built on the proven Kong plugin architecture
- 4.4 on G2 (347 reviews)
- 3.4 on Trustpilot (2 reviews)
- Starting price
- Free plan
- Free plan
- Yes
- Rating
- 4.2 (349 reviews)
Best for: Enterprises that already operate Kong for API management and want to route LLM traffic through the same platform and governance layer
- Pro: Builds on Kong's mature plugin ecosystem so existing API policies (auth, rate limiting, analytics) apply to LLM routes without new tooling
- Pro: Semantic caching, semantic routing, and PII sanitization are first-class plugins, not afterthoughts
- Pro: Agent-to-agent traffic support added in 2026 positions Kong for agentic workloads where LLM calls chain across services
- Con: Konnect cloud pricing scales steeply: each LLM provider integration counts as a Gateway Service, and the per-service fees add up quickly for teams with many model endpoints
- Con: Configuration complexity is high; Kong's plugin and declarative config model has a steeper learning curve than purpose-built LLM gateways
Kong's pricing is very generous, offering a fully functional open-source 'Free' tier.
Cloudflare AI Gateway
Zero-infrastructure AI gateway on Cloudflare's global edge network, free to start with one line of code
- 4.3 on PeerSpot (79 reviews)
- Starting price
- From $25/mo
- Free plan
- Yes
- Rating
- 4.3 (79 reviews)
Best for: Teams already building on Cloudflare Workers or Pages who want observability, caching, and rate limiting with minimal setup and no added latency
- Pro: Core features including caching, rate limiting, analytics, and content moderation are free with no usage caps on the gateway layer itself
- Pro: Integration is a single base URL change; no new SDK, no new container, no new infrastructure to operate
- Pro: Global edge network means the gateway hop adds negligible latency compared to routing traffic through a centralized cloud proxy
- Con: Does not provide aggregated billing across providers; you still maintain separate accounts and API keys for each model provider
- Con: Routing intelligence is basic compared to dedicated tools; there is no intelligent per-query model selection or cost-optimized routing
Cloudflare's Free plan is the most generous in the CDN industry, unlimited bandwidth, DDoS protection, and a global CDN at $0.
- Starting price
- From $40/mo
- Free plan
- Yes
- Rating
- 4.7 (43 reviews)
Best for: AI engineering teams running high volumes who want automated model selection rather than manually tuned routing rules
- Pro: Neural routing analyzes each prompt and routes to the model that meets your declared cost, latency, or quality target rather than round-robin or fixed priority
- Pro: Benchmarking tools let you compare endpoints across providers on your actual workloads before committing to a routing policy
- Pro: Multimodal routing support including vision and text-to-image tasks added in 2026 extends the optimizer beyond text-only LLMs
- Con: The routing logic (the neural scorer) is proprietary and not inspectable, which raises concerns for teams that need full auditability of routing decisions
- Con: No official npm package as of 2026 limits adoption in pure JavaScript or TypeScript stacks
Unify's pricing model is fair, offering a generous free 'Personal' tier.
Eden AI
Multi-modal AI aggregator with a unified API across 500+ models including OCR, speech, translation, and vision alongside text LLMs
- 2.4 on Trustpilot (6 reviews)
- 4.5 on G2 (1 reviews)
- Starting price
- Custom
- Free plan
- No
- Rating
- 3.0 (7 reviews)
Best for: Teams building pipelines that combine text generation with OCR, speech-to-text, translation, or image analysis and want one vendor relationship for all of it
- Pro: 500+ models spanning text, OCR, document parsing, speech, translation, and computer vision in a single API, which is broader than any other gateway in this list
- Pro: Pure usage-based pricing with a 5.5% platform fee and no monthly subscription since January 2025, making cost fully proportional to usage
- Pro: GDPR-native with EU data residency by default and a Data Processing Agreement included as standard, which simplifies compliance in European markets
- Con: The 5.5% platform fee on top of provider costs adds up at high token volumes compared to direct API access or a zero-fee gateway like Cloudflare
- Con: Routing intelligence for text LLMs is less sophisticated than dedicated tools like Unify; the focus is breadth of modalities rather than per-query optimization
Eden AI's pricing model, with its 'Pay as you go' for the AI API Gateway, is fair and flexible for developers and small teams.
- Starting price
- From $100/mo
- Free plan
- Yes
Best for: Teams already using Helicone who need its observability depth and are comfortable with a platform that is no longer receiving new features
- Pro: Proxy-layer caching, rate limiting, and request routing work without any SDK changes, just a base URL swap
- Pro: Observability depth is strong: cost breakdowns, latency percentiles, session analytics, and user-level tracking out of the box
- Pro: Open-source under MIT license so existing deployments can fork or maintain the code without depending on the vendor
- Con: Helicone was acquired by Mintlify in March 2026 and is now in active maintenance mode with no new features planned, which is a significant risk for new production deployments
- Con: Pro tier at $79 per month is comparable to Portkey cloud but without the active development trajectory
Helicone's pricing is quite fair, offering a generous Free tier for initial exploration and a well-priced Pro tier at $100/month for significant production use.
Watch out
Potential overage fees if exceeding Pro tier limits
What Is an LLM Gateway?
An LLM gateway (or AI gateway) is a unified proxy layer that sits between your application and one or more model providers. You point your HTTP client at the gateway instead of at OpenAI or Anthropic directly. The gateway normalizes request and response formats, manages provider credentials, and adds production infrastructure your application should not have to build itself.
The category includes several distinct archetypes:
- Aggregator-with-billing (OpenRouter, Eden AI): you pay the gateway, which pays the providers. One account, one invoice, access to hundreds of models.
- Self-hosted open-source proxy (LiteLLM, Kong): you run the software on your own infrastructure. No data leaves your network. You own operational burden.
- Managed gateway with observability (Portkey, Helicone): the gateway layer is paired with a full tracing and analytics platform. Closer to LLMOps than pure routing.
- Infrastructure-native (Cloudflare AI Gateway): bolted onto an existing global network. Near-zero latency overhead, free tier, but no aggregated billing.
- Routing optimizer (Unify): the differentiator is intelligent model selection per query rather than simple round-robin or priority fallback.
Why It Matters
Without a gateway, each team in your organization maintains its own provider keys, its own retry logic, and its own cost spreadsheet. A gateway centralizes all three. Budget limits, per-user or per-team rate caps, and real-time cost dashboards become possible without touching application code. Semantic caching alone typically cuts inference spend by 20 to 40 percent on workloads with repeated or similar queries. Fallback routing means a provider outage becomes a blip in your logs rather than a page to your on-call engineer. As model choice proliferates, a gateway is the practical answer to the question of how you stay provider-agnostic without rewriting your integration every quarter.
Key Features to Look For
Multi-provider routing and fallbacks (Essential)
The gateway should route to at least the major providers (OpenAI, Anthropic, Google, AWS Bedrock, Azure) and automatically retry a failed request on a fallback provider without changing your application code.
Unified API key and credential management (Essential)
A single outward-facing key that your applications use, with provider credentials managed centrally in the gateway. This decouples secret rotation from application deploys.
Cost tracking and budget enforcement (Essential)
Per-key, per-team, or per-user spend dashboards and hard budget caps. Without this you are flying blind on what inference actually costs at scale.
Semantic and exact caching (Important)
Serves cached responses for identical or semantically similar prompts. Reduces both latency and cost on high-repetition workloads without application changes.
Guardrails and content moderation (Important)
PII redaction, jailbreak detection, and prompt/response scanning at the proxy layer. Enforces safety policies without coupling them to application logic.
Self-host or on-premises deployment (Nice to have)
The option to run the gateway inside your own VPC or on-premises so no request data transits a third-party cloud. Critical for regulated industries and strict data residency requirements.
How to Choose
-
Decide self-hosted versus managed first. If your compliance team requires that prompts never leave your network, the shortlist is LiteLLM and Kong. Everything else is a managed cloud service.
-
Check whether you want aggregated billing. OpenRouter and Eden AI let you pay one bill for all providers. Everyone else requires you to maintain direct accounts and keys with each provider.
-
Match routing intelligence to your use case. Simple priority fallback is built into almost every option. Intelligent per-query routing that picks the cheapest model meeting a quality threshold is a Unify specialty.
-
Do not conflate gateway with observability. Portkey and Helicone do both, but you can also run LiteLLM for routing and Langfuse for tracing as separate concerns.
-
Check the operational surface. Managed tools (OpenRouter, Cloudflare, Portkey cloud) are a URL change. Self-hosted tools (LiteLLM, Kong) require Postgres, Redis, and ongoing ops.
-
Verify the maintenance trajectory before committing. Helicone was acquired and entered maintenance mode in March 2026. Always check a project's recent commit velocity and roadmap before building on it.
Evaluation Checklist
Confirm whether requests must stay inside your own network. If yes, only LiteLLM and Kong OSS qualify without managed-cloud involvement.
Count how many model providers you need on day one and in 12 months. Aggregators with 300+ models save procurement time; self-hosted gateways require you to configure each provider manually.
Test caching hit rates on a sample of your real workloads before committing. Semantic caching savings vary widely by use case.
Check the routing fallback behavior under a simulated provider outage. Automatic retry with exponential backoff and cross-provider failover is the minimum acceptable behavior for production.
Review the project's recent commit history and acquisition or funding status. Maintenance-mode tools (Helicone) and lightly staffed open-source projects carry continuity risk.
Calculate total cost of ownership including infrastructure, subscriptions, and engineering time to operate. A free open-source license does not mean free to run.
Pricing Overview
Free / open-source
Self-hosted deployments (LiteLLM, Kong OSS) and Cloudflare AI Gateway for projects already on Cloudflare Workers
$0 platform fee
Pay-as-you-go aggregator
Teams that want one account and one bill across all providers without running infrastructure (OpenRouter, Eden AI)
Provider token rates plus a small markup or percentage fee
Managed platform subscription
Roughly $49 to $100 per month for production tiers
Enterprise / custom
Large organizations needing SSO, RBAC, dedicated instances, SLAs, compliance certifications, or on-premises deployment
Custom contract
Pricing Comparison
| Tool | Open source | Free tier | Pricing model |
|---|---|---|---|
No | Yes | Pay per token + 5.5% fee | |
Yes | Yes | Free OSS; Enterprise from $250/mo | |
Portkey | Yes | Yes | $49/mo paid tier; free self-host |
Yes | Yes | Free OSS; Konnect from $105/service/mo | |
No | Yes | Free tier; Workers Paid from $5/mo | |
Unify | No | Yes | $99/mo; limited free tier |
No | Yes | Free $10 credit; 5.5% fee on usage | |
Helicone | Yes | Yes | Free 10k req/mo; Pro $79/mo |
Pricing as of June 2026; check each vendor for current rates.
Mistakes to Avoid
- ×
Choosing a gateway based on the longest model list rather than on operational fit. A 300-model aggregator is useless if your compliance team requires on-premises routing.
- ×
Conflating the gateway layer with the observability layer and buying a combined tool when separate best-in-class tools would be cheaper and more flexible.
- ×
Treating a base URL swap as the complete integration and skipping budget caps, rate limits, and fallback configuration. The swap is the start, not the finish.
- ×
Running a self-hosted gateway without Redis and Postgres properly sized for your request volume, then blaming the gateway when it performs poorly under load.
- ×
Locking into a single gateway vendor without verifying that your application can switch to another endpoint within a day if the vendor has an outage or raises prices.
Expert Tips
- →
Start with a managed gateway for speed, but architect your application so the base URL is an environment variable. That one discipline makes it trivial to swap gateways or go direct to a provider later.
- →
Enable semantic caching from day one even at low volume. The hit rate data you collect over the first month will tell you whether investing in a more sophisticated caching strategy is worth it.
- →
Set hard per-key budget caps before you give API access to any third-party code or external contributor. Cost blowouts from a misconfigured loop are much easier to prevent than to recover from.
- →
Run a monthly routing audit: pull your token spend by model and check whether the models getting the most traffic are actually the best fit for those tasks, or just the defaults you set at launch.
- →
If you are choosing between two gateways and one is in active development and the other is in maintenance mode, the maintenance-mode tool needs to offer a uniquely irreplaceable feature to justify the adoption risk.
Red Flags to Watch For
-
A gateway that markets 'zero latency overhead' without publishing independent benchmark methodology. Every proxy adds some latency; the question is how much.
-
No mention of fallback behavior or what happens when a provider returns a 5xx. A gateway without automatic retry and failover is not production-ready.
-
Proprietary routing logic with no explainability or audit trail. If you cannot see why a request was sent to a particular model, you cannot debug unexpected costs or quality regressions.
-
A platform in maintenance mode or with stalled development (check the GitHub commit history and changelog before building on it).
-
Aggregated billing with opaque markup rates. You should be able to verify what you are paying per model versus the provider's published rate.
The Bottom Line
OpenRouter is the right starting point for most teams: one account, one bill, 300+ models, zero infrastructure. When data residency or full control is non-negotiable, LiteLLM is the mature self-hosted answer. Portkey is the strongest choice for teams that want gateway plus observability plus production guardrails without running everything themselves. Cloudflare AI Gateway is the obvious pick for teams already on the Cloudflare platform who want caching and analytics at no extra cost. Kong is a serious option only if you are already standardized on Kong for API management. Unify is worth evaluating for high-volume workloads where intelligent per-query routing can cut costs materially. Eden AI stands out when your pipelines span multiple modalities beyond text. Helicone is a proven tool now in maintenance mode: existing users have no urgency to migrate, but new deployments should look elsewhere.
Frequently Asked Questions
What is the best LLM gateway in 2026?
It depends on your primary constraint. OpenRouter is the best option for teams that want immediate access to hundreds of models with one API key and no infrastructure to run. LiteLLM is the best self-hosted option when data must stay inside your own network. Portkey is the strongest managed gateway when you also need observability and guardrails in the same product. There is no single best gateway because the right answer depends on whether you need aggregated billing, self-hosting, routing intelligence, or deep observability.
What is the difference between an LLM gateway and an AI gateway?
The terms are used interchangeably in 2026. Both describe a proxy layer that sits in front of one or more model providers, normalizes the API, and adds routing, caching, rate limiting, and cost tracking. Some vendors use 'AI gateway' to signal support for non-LLM modalities like image generation, speech, or OCR (Eden AI is a clear example), but there is no standardized distinction.
Is LiteLLM free?
The open-source software is MIT-licensed and free. However, running it in production requires infrastructure: a Postgres instance for spend logs and key management, a Redis instance for caching and rate limit counters, and the operational cost of maintaining container deployments. Enterprise features like SSO and RBAC require a paid commercial license. The total cost is infrastructure and engineering time, not a platform subscription.
Can I use an LLM gateway without sending my data to a third party?
Yes. LiteLLM and Kong both run entirely on your own infrastructure. Your application sends requests to your own proxy, which forwards them to model providers. The gateway vendor never sees your traffic. Cloudflare AI Gateway routes through Cloudflare's edge network, which is not on-premises but is a large, established cloud infrastructure provider with a strong trust posture. Fully air-gapped deployments require LiteLLM or Kong.
What happened to Helicone?
Helicone was acquired by Mintlify in March 2026 and entered maintenance mode. Existing features continue to work and the open-source repository remains available, but no new features are being developed. Teams evaluating LLM gateways for new production deployments should consider Portkey or LiteLLM instead, both of which have active development.
Cite this page: Toolradar, "Best LLM Gateways in 2026", updated June 2026, https://toolradar.com/guides/best-llm-gateways
Sources
Prices and plan details on this page come from each vendor's own pricing page, re-checked by the Toolradar pricing tracker:
- OpenRouter pricing, checked
- LiteLLM pricing, checked
- Portkey pricing, checked
- Kong AI Gateway pricing, checked
- Cloudflare AI Gateway pricing, checked
- Unify pricing, checked
- Eden AI pricing, checked
- Helicone pricing, checked
Related Guides
From the team behind Toolradar
Reddit management for B2B tech
Authentic Reddit presence in the subreddits dev-tool buyers actually live in.
See how we workSome offers on this page may be paid placements or contain affiliate links.