Review runtime behaviour changes

For: cluster operators.

Follow each section's upgrade order for runtime egress, budgets, timings and failure kinds.

Tenant entries need a runtime from the same release

A node binary from this release tells its runtime which egress entries are the tenant's, so they may reach private networks. A runtime from before it does not, and on such a node the tenant's intranet hosts are refused inside sandbox tier 1 until the runtime is upgraded: upgrade the runtime with the node binary. An older node binary under a new runtime works as before, without the new limit on the other lists' entries.

Strict budgets count more of a call

A budget now holds a call's whole worst case, under every mode: several answers, server tools, the dearest tier the call can land on, cache writes at their rate and audio at its price.

Output limits count only in the field the call's format reads: max_tokens on a Responses call, or at the top of a generateContent call, no longer bounds anything. So calls hold more while they run, and on small budgets oz0_budget_busy and a refusal come sooner.

A call that names no tier on OpenAI's records holds twice its standard price until it settles; naming service_tier: default holds the standard price. On Anthropic's records it holds 1.1 times, a workspace's default region being able to cost that, and inference_geo: global holds the standard price.

flex and batch no longer lower a reservation, and lower a charge only when the answer names them.

A model with its own rate for images or audio holds that rate unless the call restricts it to text, a router's models list holds its dearest model, and predicted output holds its bytes as output.

Under reserve: strict, calls that were sent before are refused with 400 oz0_budget_unbounded: a stored object or a URL the provider fetches, a compressed file sent inline such as a PDF or a video, a server tool with no limit its format reads or no price, a web fetch with no max_content_tokens, a router's route, a tier no record prices.

Tier words belong to their vendor: on a record that follows neither OpenAI's nor Anthropic's prices, flex, priority, speed and inference_geo are such words until its charges price them. max_tool_calls counts on the Responses format only, and max_uses on the Messages format only. Check agents with strict budgets that use them: the list says what to do with each.

Charges change where they were low: cache writes on the Responses API are charged at their rate, a record's price without a cache-write price charges writes by the cache's multiplier rather than at nothing, the longer lifetime is charged without the prompt cache's hooks too, and audio is charged at its own price. Cache writes that a Chat answer reports are charged like those on the Responses API. A tool's fee that is no web search counts in the cost's unsplit part. A reported cost is no longer raised by the fees it already holds.

Timings on model calls, and per-second prices from the send

From this release every model call records its timings: its queue, its time to first token, and its prompt and decode speed. What you may notice:

• A local model's per-second price drops for calls that queued. A tenant that prices a node's local model per second was charged from the moment the node's proxy took the call, before the edge's admission and the wait for a free slot; it is now charged from the upstream send to the end.
  A provider record priced per second through the gateway counts from its upstream send too, so the edge's checks and a pool's moves are not charged. Token prices do not change. A call that waited too long on the fleet's model queue and reached no model server is not charged any seconds; until this release it was charged for its whole wait.
• Reprices count seconds the same way. A governance reprice with a per-second price charges a call from its upstream send, as the call was charged, so the same price sheet gives the same cost. In the usage export, a row's duration_ms is still the call's whole time, so charges.seconds of a call that queued is less than duration_ms times the per-second price.
• Calls from before have no timings: their columns stay empty (unknown, never 0), and means over a period count only the calls that measured each figure.
• Upgrade the edges before the nodes. A node from this release reports its local calls' timings with them; an edge from before refuses a report with a field it does not know, and the node sends the report again without the timings, so the call is recorded and only its timings are lost. An older node under a new edge reports no timings and is charged its whole duration, as before.
• New metrics: oz0_llm_first_token_seconds, oz0_llm_decode_tokens_per_second and oz0_llm_queue_seconds, histograms with a label set each per tenant, provider, model and, for a node's own model servers, node. Scrapers that limit series per target count them.

Failure kinds

From this release a failed job carries a failure kind. What you may notice:

• Jobs that started before the upgrade fail as they did, also on replay: a delegation whose child fails still fails its parent, and a flow step's failure still ends its flow. Jobs that start after it read a failed child as an answer and run a flow's error branches. Their failure on the runtime API is filled for old jobs too, from their error's type.
• Upgrade the edges with the runtime. An edge from before shows no failure and no declared kinds in the catalog; the jobs run the same.
• An update that adds a failure kind is a breaking change, and so is one that turns a kind's retryable off or its partial or person on: give it a new major version, or install it with --allow-breaking.
• harness-claude 0.1.4 offers oz.fail to its sessions; an older one does not, so its agents cannot end a job with a kind of their own (their jobs still get the platform's kinds).