Dashboard →

Methodology

Routing savings

The Savings section on the Routing page estimates what routing saved against the model each request would otherwise have used. It is an estimate, not an invoice. This page states how it is calculated and, just as important, what it does not claim.

The calculation

For each completed routed request, the same tokens (input, output, cache read, cache write, reasoning, and any non-token units) are priced twice at the time of the request: once on the baseline model and once on the model that actually served it. Both prices use your workspace's rates, with a negotiated rate taking precedence over list price. The saving is the baseline cost minus the served cost. A request that cost more than its baseline is a negative saving and is never rounded up to zero.

Worked example

A request uses 1,000 input and 200 output tokens. The baseline model costs $15 per million input tokens and $75 per million output tokens: 1,000 × $15 / 1M + 200 × $75 / 1M = $0.030. The model that served it costs $1 and $5 per million: $0.002. The saving is $0.028, which is 93.3% of the baseline cost.

The headline figure sums the saving over priced decisions. The percentage is that net saving divided by what the baseline models would have cost over the same decisions, including decisions where routing left the model unchanged (their saving is zero), so a mostly-unchanged workload shows a small percentage.

The baseline

  • A concrete model the caller named is its own baseline: the saving compares what the caller asked for with what was served.
  • Tiered routing, or a placeholder such as auto: the router chose the model, so the baseline is the premium model your workspace declared in Routing settings. If none is declared, those requests are left unpriced rather than guessed.
  • Model names are matched through the same alias table used for metered usage, so a dated model name and its canonical name count as the same model.

Changing the baseline restates history

The baseline model is applied when the figures are read, not stored with each request. Declaring or changing it updates every past figure in the window, including requests that were previously unpriced. Each change is recorded in the audit log with the previous and new value.

What counts as a saving

  • Model switch. A policy or tiering sent the request to a different model than the baseline. Counted.
  • Batch arbitrage. The router ran a request through a provider batch at the batch price. The baseline is the same model at the synchronous price. Counted.
  • Evaluation canary. A live evaluation campaign sent the request to a candidate model. It is measured against the model the policy would otherwise have used, shown separately, and not counted: it is an experiment, not a routing choice.
  • No change. The served model is the baseline. Included in the percentage denominator, saving zero.
  • Fallbacks. The primary model failed and a configured fallback served the request. Shown separately and not counted: that is resilience, not a routing choice.
  • Budget downgrades. A monthly budget cap forced a cheaper model. Shown separately and not counted as a routing saving.

Headline and estimated lines. The headline counts only decisions whose baseline provider was recorded when the request was routed, or whose baseline is your declared default. Older decisions made before that was recorded rely on a provider inferred from the model name. They are reported on a separate estimated line and are never added into the headline.

What is excluded or unpriced

Some decisions cannot be priced. They are excluded from the dollar figures, counted in the coverage shown beside every number (for example “44 of 48 priced”), and never treated as free. The Routing page lists the reasons:

  • No baseline model is declared for a tiered or placeholder request.
  • The baseline or the served model has no price in the catalog.
  • The baseline model has no price for a token class the request used. The common case is cache writes on a request served by one provider and measured against another provider's model that has no cache-write price. These are listed separately because they make cross-provider switches less likely to be priced.
  • The usage was partial, for example a stream that was interrupted.
  • The usage has not settled yet.

Not counted at all: the router's own internal classifier, guardrail and judge calls, responses served from cache (no provider spend, so no model price to compare), and requests that failed or were rejected.

Quality evidence

A cheaper model is only a saving if it did the job. Beside the figure, the page compares the success rate of decisions where routing changed the model with decisions left on the requested model, using the outcomes you report against routed requests. A decision counts as successful when every success outcome linked to it is true. The share of decisions that carry any label is always shown, and below 30 labelled switched decisions the page says the saving is unproven. The two groups are not like for like: routing sends easier work to cheaper models, so the switched group is expected to succeed more often even if the cheaper model is worse. Only a controlled comparison, such as an evaluation campaign, shows whether a cheaper model is as good.

What this does not claim

This is a counterfactual estimate

The baseline never ran, so its cost is computed, not observed. The figure assumes the baseline model would have used the same tokens. Models differ in how much they write and how often they need a retry, so the true difference can be larger or smaller.
  • It ignores prompt-cache effects: a baseline that would have hit a warm cache would have cost less.
  • It does not measure output quality. The quality evidence above is the only check, and it is often sparse.
  • It prices at your current catalog and negotiated rates. A rate correction changes past figures.
  • It is not an invoice and does not reconcile to provider billing. Metered spend and provider-billed spend remain separate.
  • It does not include the optional token savers (concise-output instruction, log compression, Recall). Those are reported separately, in tokens, because their dollar effect depends on prompt-cache behavior. The concise-output instruction has a known cost, the input tokens it adds to every request, which is shown; its benefit is not measured.
  • Team breakdowns group teams with fewer than 5 attributed people into “Other (small teams)”, so a figure cannot identify an individual. A team with no member attribution is grouped the same way.

Where the numbers come from

Everything is computed when you load the page from metered usage events and the current price catalog, over whole UTC days (the selected window ends today). Nothing is stored as a dollar figure, so a corrected price or a newly declared baseline is reflected immediately. Usage that has not settled yet is unpriced until it does. The formal definition is also on the metric definitions page.