Category guide
What is a model router?
A model router is a single endpoint that sits between agents and the model providers they call. It presents one API shape, chooses which model serves each request, injects the provider credential, and reports usage consistently across providers.
Why Dome
Why use Dome for a model router?
If one application needs to reach several models, a router is the right size of answer. Dome is for the enterprise AI platform team that owns model access for the whole business, and has to account for every token.
Which means the model is not the only thing under governance. Spend attributes to the agent rather than the key. The person the agent acts for is part of the authorization decision. Tool calls run under the same rules and land in the same audit trail. One object model, one rule language, one record. How that works.
The problem
One provider is a decision you make once
Calling a provider API directly is the right first move. It stops being right at a predictable point.
The first pressure is availability. A provider rate limit becomes your outage, and the fix has to live somewhere other than in every agent. The second is cost: token spend arrives as one bill with no way to attribute it to the team, the agent, or the workflow that caused it.
The third is choice. Models change every few months, and the good answer differs by task. A router turns each of those from a code change into a configuration change, and gives you one place to see what is actually being spent.
Capabilities
What a model router does
Provider abstraction
One request shape in front of many providers. The agent speaks a single API and the router translates, so changing provider is a configuration change rather than a code change.
Routing strategy
Choosing which model serves a request: cheapest that will do, fastest available, pinned to a region, or escalating to a stronger model only when the cheap one fails.
Failover
Providers have outages and rate limits. Grouping models into a pool lets a request fall through to the next candidate without the agent knowing anything happened.
Credential isolation
Provider API keys live at the router and are injected at egress. Agents never hold them, which is what makes rotation and revocation tractable.
Spend control
Token and cost ceilings, ideally per agent and per workspace rather than per API key. This is where most of an agent estate's cost actually accrues.
Normalized telemetry
Token counts and latency reported the same way regardless of provider, so usage across a mixed estate can be compared and charged back.
Choosing
Where model routers differ
The capability lists converge. The differences that matter are underneath them.
The landscape
Who builds model routers
A short, honest map of the category. Products are described by shape rather than by feature list, because shape is what stays true.
| Product | What it is | Best suited to |
|---|---|---|
| LiteLLM | The open-source default. Broad provider coverage, library-shaped, self-hostable. | Teams who want provider abstraction in their own stack and are happy to operate it. |
| Prisma AIRS | Formerly Portkey, now part of Palo Alto Networks. A routing layer inside a security suite. | Organizations consolidating on Palo Alto for AI runtime security. |
| Kong AI Gateway | API-management lineage extended to model traffic, on the same routes and plugin chain. | Organizations already running Kong who want models on that fabric. |
| OpenRouter | Aggregation-first. One account and one endpoint in front of a very wide model catalog. | Reaching many models quickly without negotiating provider contracts. |
| Dome | An agent operations platform. The model router is one control point of three, alongside tool traffic and agent identity. | Enterprise platform teams running agent services for the whole business: many agents, many teams, models and tools both in scope, one audit trail. |
Last updated 11 August 2026, based on publicly available information about these products. Spot something out of date? Tell us and we will correct it.
Evaluating
Questions worth asking
- 01Can a spend cap bind to an agent, or only to an API key?
- 02Can a rule say which agents may call which models, evaluated per call?
- 03Who governs the tool calls the same agents are making?
- 04Does the audit record name the agent and the person it acted for?
- 05On failover, is the fallback model authorized in its own right?
- 06If semantic caching is on, whose response can another caller receive?
How Dome approaches it
Routing in service of governance
Most products in this category began as routing and added governance afterward. Dome began at the other end, and it changes what the router is for.
A router that exists to route treats spend and access as settings on a key. A router that exists to govern treats the model as one object among several that the same rules have to reach. That is the difference between metering usage and being able to say which agent spent it, on whose behalf, and under which rule.
So Dome’s model router, the Model Broker, is one control point of three. Spend attributes to the agent, not the key. Every failover candidate is authorized in its own right. The same rules that decide which model an agent may call decide which tools it may reach. The agent’s call shape never changes. What changes is that somebody can finally answer for it.
Quotas bind to identity
Spend and token ceilings attach to the agent and the workspace, not to an API key. Attribution and containment stop being a reconciliation exercise across two systems.
Failover is re-authorized
Each candidate in a pool is authorized in its own right rather than inheriting the permission of the first choice. A fallback cannot quietly widen what an agent may reach.
One record with tool calls
Model calls and tool calls share an audit vocabulary, so the question of what an agent did has one answer in one place rather than two partial ones.
Agents Need to Talk to Models is a 20-minute field guide to the Model Broker: connecting providers, building pools, expressing model choice as routing policy, and capping spend. No form, no gate.
FAQ
Common questions
What is a model router?
A model router is a single endpoint that sits between applications or agents and the model providers they call. It presents one API shape, chooses which provider and model serves each request, injects the provider credential, handles failover, and reports usage consistently across providers.
What is the difference between a model router and an LLM gateway?
In practice the terms are used interchangeably. Where people do draw a line, routing emphasizes choosing between models on cost, latency or capability, while gateway emphasizes the control functions around the call: authentication, limits, filtering and audit. Most products in the category do both.
Why not call the provider API directly?
Direct calls mean every agent holds provider keys, model choice is hard-coded, an outage at one provider is an outage for you, and spend is visible only as one large bill per provider rather than per agent or per team.
Does a model router govern tool calls?
Usually not. Model routers govern the model channel. Tool calls, which is where an agent takes actions with real-world consequences, typically pass through a separate MCP gateway with its own rules and its own logs.
Is Dome a model router?
Dome's Model Broker is a model router, and is part of the overall Dome platform. It accepts OpenAI- and Anthropic-compatible calls, routes to a provider or pool, and injects credentials at egress. It also governs tool calls, agent identity and the caller an agent acts for, under one rule language and one audit trail.
Which model router should I choose?
Model routers tend to have similar capabilities but targeted at specific users and focus only on model routing. Dome is for enterprise AI platform teams building agent services for the rest of the business: dozens of agents across many teams, model spend that has to be attributed to the agent rather than the key, tool calls governed by the same rules, and a security function that expects one answer to who did what, on whose behalf, and under which rule.