Agent budgets

For: plugin authors.

Reserve spending before calls and configure budget exhaustion behaviour.

Budgets

A budget caps what a job and every job below it may spend on model calls, in US dollars. limits.max_cost_usd sets the default for an agent's jobs, and run --budget (or budget_usd in the runtime API) sets it for one job:

oz0 run acme-reviewer --budget 0.50 "Review PR 42 in acme/api"

The LLM gateway checks every call against what the whole tree has been charged, costs that plugins report for their own services included. When the next call does not fit, it refuses it with 402 oz0_budget_exceeded, and the job fails with BudgetExceeded without retrying, unless the agent's on_budget makes it wait for a person: see Budgets that wait. job get shows the budget next to the cost, and oz0 usage --job <id> what running calls hold.

Each call reserves its worst case

Before the gateway sends a call, it reserves the most the call can cost against every budget that applies (the tree's, the tenant's month, and the quotas of the job's owner and team), and the call holds that reservation until its reply settles it:

The worst case

The reservation covers the call's input (one token per three bytes of the request) and its output limit, max_tokens (or max_completion_tokens, max_output_tokens), at the higher of the price the call is metered at and the provider's list price. A call that sets no output limit is reserved for the provider record's max_output_tokens for the model, or 128,000 tokens when the record names none.

A model that neither its record nor the list prices (a provider that reports its own cost, a name only the answer resolves) is reserved at the record's highest price, or at $15 and $75 per million input and output tokens on a record with no prices; give the model its price on the record to reserve its own. A call whose body names no model at all (an upstream that picks one itself) is reserved the same way, for the largest output the record names.

A body the gateway cannot read the way the provider will (not one JSON object, or model, stream or an output limit named twice or in another case) is refused with 400 oz0_unmetered before anything is reserved or sent.

• Check and reserve are one step, on counters every edge of the cluster sees: two calls on two edges cannot both take the last dollar. Fifty children that start at once cannot all pass a budget that only a few of them fit.
• A complete reply settles at what was metered, and the rest of the reservation goes back to the budget. A reply is complete only when it says what it used: a successful answer without its usage is an unknown outcome (below), never a call that cost nothing. Tokens of a model nothing prices are charged at the price their reservation assumed (the record's highest, or $15 and $75 per million), never at $0.
• An unknown outcome is never refunded: a call that timed out, a stream that broke off, a node lost mid-call, or a gateway error between the edge and the model (502, 504) is charged at least its reservation, because the provider may have billed it in full. Its row in the cost log says incomplete. A provider's own error answer (such as a 503 or Anthropic's 529 overloaded) is a complete reply that costs nothing.
• A Temporal retry is a new attempt with a reservation of its own: the earlier attempt stays charged.
• Calls wait for each other. A call that does not fit only because other calls of the budget still run (they hold the rest at their worst case) waits for them to settle (429 oz0_budget_busy, which the agent's model call waits on), since most of what they hold goes back when they do.

The last call

A call that does not fit may still start when nothing else of that budget is running and the budget is not spent yet, so a budget smaller than one call's worst case still runs its agent, one call at a time. A tree can therefore pass its budget by at most one call's reservation. Set reserve: strict in the agent's definition (or oz0 tenant spend --reserve strict) to refuse that last call too: the tree then never passes its budget, and a budget smaller than one call's worst case refuses every call.

A tenant's strict holds for every agent of the tenant and for its monthly budget and quotas: an agent can choose strict where the tenant has not, never last_call where the tenant chose strict.

reserve: strict        # or last_call, the default

Under reserve: strict the worst case is the whole of what a call may cost: every answer it asks for (n, best_of, candidateCount), its server tools at their limits, the dearest price tier it can land on, cache writes at their rate for each lifetime, and audio at audio's price. What cannot be bound is refused with 400 oz0_budget_unbounded and a plain reason, before anything is reserved or sent:

• a stored object or a fetched address: a prior response, a file, a vector store, a stored prompt, an image or a document by URL. The gateway cannot read their size, so one small reference can bring in more than the whole budget. Send text or images in the call itself. The gateway does not ask the provider for a count first: a count is another request at another moment, it may reach another pool member, and what it counted can change before the call
• inline content whose bytes do not bound what the provider reads out of it, such as a compressed PDF or a video: send its text, or images, which count at the most one image can add. A record can state a bound for a type in media_tokens
• a server tool with no limit its format reads, or one the record has no price for: max_uses on the Messages format and max_tool_calls on the Responses format are limits; Chat and Google's format have none, so a tool there, or a model that searches by its name, is refused. Web fetch also needs max_content_tokens, and more than 128 such tools in one call are refused
• a tier word the record has no price for, or a multiple that is not a number or asks for more than 128 answers. A vendor's tier words count on records that follow its prices: flex on a record that follows nobody's needs a price in the record's charges
• a router's route, or a list of more than 16 models it may fall back on: the models of a shorter list are held at the dearest
• audio or images on a model nothing prices them for

Without strict these calls go as before: the reservation counts what it can, and the charge is what the provider reported. Only a worst case past $1,000,000,000 is refused under every mode, as no reservation can hold it: no real call comes to that, and an output limit counts at most a billion tokens.

A refusal says what the tree has spent of its budget; under reserve: strict, a budget that is not spent yet refuses a call with what the call may cost, and the fix:

BudgetExceeded: job job-7f3a has spent $0.4521 of its $0.5000 budget; this call may cost up to $0.0614 (4000 output tokens at list price), which reserve: strict does not let pass the budget: lower the call's max_tokens, or raise the budget

Since a reservation is the call's worst case, budgets refuse earlier than they used to: an agent whose model sends 64,000 tokens of max_tokens on a model at $15 per million output tokens holds $0.96 per call while it runs. Give the agent's model requests an output limit that fits its work (limits.max_output_tokens, sent as each request's max_tokens; Pydantic AI agents), or the tree a larger budget.

When budgets cannot be read, the gateway does not guess: a budgeted call is refused with 503 oz0_budget_unavailable and Retry-After, and the agent's model call waits and sends it again (for up to two minutes, then the activity's own retries). The same holds for the quotas of the job's owner and team and for the tree's model calls and tokens when its limits apply. Calls of a job without any budget are not affected.

When a call's usage cannot be written, it is not lost: the edge keeps the write on its disk, in the server's data directory (<data-dir>/metering), and writes it once the store takes it, after a restart too (a budgeted call holds its reservation meanwhile, which the retirement pass settles at least at what it reserved), and the charge to the owner's and team's quotas likewise. Each is written once however often it is tried: a usage row by its call's ID, a quota charge by an ID of its own.

Give the server a data directory that outlives a restart and the replacement of its machine or pod (the Helm chart gives each server a volume of its own); a server with no writable data directory keeps these writes in memory only, and says so in its log at start. A server that stops tries the waiting writes once more, and its log says how many it leaves on disk and where: a server started on that directory writes them.

A plugin's usage report and a node's report of a local model call carry a key of their own, so one sent again after a lost answer is recorded once. What could not be kept is counted in the edge's metrics (oz0_metering_writes_retried_total, oz0_gateway_records_total; see Tracing and metrics).

Local model calls meet the same tree limits as calls through the gateway: past the tree's model calls or tokens, the node's admission gets 402 oz0_tree_limit.

A local model the tenant prices reserves its worst case at admission. The node's call report settles the reservation at what it used. A local model nobody prices costs nothing and reserves nothing.

A request reserves the largest answer if it also contains another key that a model server may use to set answer length: n_predict, num_predict, max_new_tokens, options, or a limit's name in another case. Case comparisons use Unicode case folding as a Go server reads keys. The same reservation applies when the body asks for more than one answer with n or best_of, in any case. The server may use these keys instead of the usual limit.

The node asks for the admission once the call has one of its model server's slots, so a call that waits for a slot holds nothing on the budgets, and a call admitted that never reaches its server (its request could not be made, or the edge's answer to the admission was lost) is reported with nothing used, which settles its reservation at $0.

If the edge cannot read the tenant's spend settings, it does not substitute defaults that could loosen the tenant's tree limits, depth, reservations or on_unknown. This can happen just after the edge starts, or when a stored tool policy cannot be read.

While the policy is unavailable:

• Every model call and child admission through that edge gets 503 oz0_budget_unavailable and waits.
• A new job is refused as unavailable. Start it again shortly.
• A routine's scheduled fire waits before running anything. Temporal retries its first step until the edge holds the policy, then the fire uses the tenant's settings.
• A write whose outcome is unknown asks again, unless its agent's own settings determine what it does.

Once the edge holds the policy, all of it goes on as before.

When an edge stops mid-call, its reservation stays held. Once a minute, one edge settles every reservation left open whose attempt is over in the job's history, at its reservation, and records it as an incomplete call: nothing a lost edge held is refunded, and nothing stays held for ever.

When management is away (an edge apart from management), each edge reserves against the last counters it saw plus its own calls since, with the same rule. With one edge, a budget still passes by at most one call's reservation; with several edges serving the same tree, each can spend what was left when management went away, so the bound is that remainder times the other edges, plus one call's reservation per edge. A budget the edge never read is let through until management is back, and the edge logs it once a minute.

A budget per tenant and month

A tenant can have a budget for each calendar month (UTC), which covers every model call of the tenant and what its plugins report:

oz0 tenant budget acme 250      # $250 a month
oz0 tenant list                 # this month's spending and the budget, per tenant
oz0 tenant budget acme 0        # no monthly budget

Once the month's budget is spent, the gateway refuses the tenant's model calls with 402 oz0_budget_exceeded until the month ends or the budget is raised, and jobs fail with BudgetExceeded:

BudgetExceeded: tenant acme has spent $250.0012 of its $250.00 budget for October 2026

Calls reserve their worst case against the month's budget as they do against a tree's (Each call reserves its worst case), so the month passes its budget by at most one call's reservation. A new budget bites within five seconds. The console shows the month on the Cost page and sets budgets under Spend, Budgets.

To divide the tenant's budget between teams and people, and to let them ask for more, see Budgets and quotas.

What the agent sees of its spend

Agents cannot tell how much they will spend, and they stop too late when they cannot see their budget. So the runtime ends each model request of an agent's loop with a short status block from the platform: what the job tree spent of its budget and what is left, the quota left when it may be shown, the steps used of max_steps, and the time used of timeout.

[Status from the platform, not from the user] Budget: $0.4100 of the job's $0.50 spent, $0.0900 left (82%). Step 6 of 20. Time: 2m10s of 10m00s. You have used 82% of the budget: wrap up and answer soon.

The block is always the request's last part, never the system prompt or the instructions, so the prompt cache still holds: the next request carries a new block and not the old one. The money comes from the gateway, which sends it back as response headers on each model call (Oz0-Spend-Tree, Oz0-Spend-Quota, Oz0-Spend-Calls, Oz0-Spend-Tokens): the gateway never changes a request. A person's quota appears only when works-council mode lets anyone see that figure.

At spend.warn_share (80%) the block tells the agent to wrap up. At the soft wall, spend.soft_wall (85%), only the answer is left and the agent is asked to report what it has, before the gateway's hard refusal. What the agent says about how close it is to done decides nothing.

Harness and ACP sessions get a read-only built-in instead, oz.remaining: list it in the agent's tools: (an ACP session that has an MCP server gets it anyway). The session's next tool result also says when the tree passed the warning share.

Budgets that wait

When a job tree's budget runs out, on_budget says what happens:

on_budget: wait      # or fail

• wait: the call is refused with 402 oz0_budget_wait, and the job waits. Only a caller that can wait gets this refusal: an agent's own model loop and a harness session's calls say so with the request header Oz0-Budget: wait. A judge's call, a compaction, or a client of your own gets oz0_budget_exceeded, as before.
  The edge opens a budget request about the tree (subject job:<root>, kind job), which the tree's owner or the tenant's admins decide on the Approvals page or with oz0 quota approve. Approved, the tree's budget grows by what the request asks for, and the job sends the refused call again; denied, or not decided by the deadline, the job fails with BudgetExceeded, as before.
• fail: the job fails with BudgetExceeded at once.

When neither the agent nor the tenant (oz0 tenant spend --on-budget) says, a tree that a person started waits, and one a routine, a webhook, an event, an API key or a channel started fails, so unattended jobs still stop. The budget stays a hard stop: no call passes it without a person's decision.

Asked before the wall. With on_budget: wait and a budget, the gateway compares the tree with the agent's own finished runs (from the cost log, never their content): when at least spend.forecast.chance (80%) of the runs that came this far went on to pass a budget like this one, it asks a person early, once per job, with 402 oz0_budget_forecast. The request (kind forecast) shows the forecast and what the tree spent so far. Approving extends the budget; denying lets the agent wrap up within the budget it has (only its answer is left); to stop it, stop the job.

The forecast needs spend.forecast.min_runs (20) finished runs of the agent first.