Explain prices and cache costs

For: tenant administrators and cost owners.

Inspect the price source and the cause and cost of prompt cache misses.

How prices are worked out

The gateway prices each call at the time of the call, from the usage the provider reports in the response or the stream: at the provider record's own price for the model, at the provider's reported cost where its record says so, or with genai-prices; and adds $0.01 for each of Anthropic's web searches.

Calls to local models are reported by the node that ran them, as provider local, at $0 unless the tenant priced them. Calls the gateway refused for a data rule cost nothing and are counted as refused, in usage and on the Cost page. Usage rows are kept in the management database and outlive the jobs' history in Temporal.

Reprice recorded spend

What would last month's work have cost under another price sheet, at another model's prices, or without a prompt cache? A reprice prices the recorded calls' own tokens again, part by part, exact to the cent, with no model call:

oz0 replay reprice --prices sheet.json --since 30d --wait
oz0 replay reprice --model anthropic/claude-haiku-4-5 --since 30d --wait
oz0 replay reprice --cache none --since 30d --wait

Another model's prices on the same tokens are labelled same tokens, another price: quality not measured: the figure says what the same tokens cost there, not what that model would have done. Per agent, model and team (teams through works-council mode), effective and at the list price.

The prompt cache

Where a provider caches prompts, the gateway records for every call how much of its input was read from the cache, what that saved, and, for a call that read less than it could have, why and what it cost. Calls before this was recorded are not in the figures.

• Hit rate: the tokens read from the cache over the cacheable input (new input, cache reads and cache writes) of calls to providers that cache.
• Saved: the cache reads at the input price, minus what they cost, minus the write premium (cache writes at their price minus the input price; writes with the longer lifetime at theirs). A call that only wrote saves less than nothing; the next one that reads makes it up.
• Missed: for a call that read less than its job's previous call could have left it, the tokens it could have read, at what they were charged minus the read price.

Each miss has one cause, checked in this order:

The gateway compares each call with the job's previous one by hashes of its tools, its system prompt and its messages, salted per tenant; it never keeps their text.

oz0 usage cache --since 7d
oz0 usage cache --causes
oz0 usage cache --by-team --json

Prompt cache: auto; figures since 2026-10-13T18:32:05Z
AGENT       TEAM  MODEL                        CALLS  HIT RATE  READ  WRITTEN  SAVED    MISSED
waiter      -     anthropic/claude-sonnet-5-5  5      18.4%     848   1746     $0.0006  $0.0038

The figures are per agent and model, or per team with --by-team; a team smaller than works-council mode shows is folded into "other teams", and nothing is shown per person. The Cost page shows the same as its Prompt cache card, and an agent's scorecard (on its page and in oz0 insights agent <agent>) its hit rate and saving beside the period before. To measure what the cache does for an agent on your own cases, run a comparison with --variant-cache off.

Prompts with their stable parts first

A provider's cache reads a prompt from its start up to the first byte that changed. So the runtime puts what stays the same within a job first and what changes last, in every adapter:

1. The agent's instructions, then the skills index, then plugins' start-up text (such as accepted memory decisions), each read once when the job starts and the same bytes in every request of the job. The skills index changes only when a grant of another plugin's skill is given or withdrawn during the job.
2. The tool definitions, in the order the agent lists them, then the platform's built-ins in a fixed order; a tool withdrawn for a request (a routine's list, finish-only) leaves the others in their places.
3. The conversation's history, in order.
4. What changes, last: the status block of what is left of the budget, attached to the newest message and never kept in the history, and the newest tool results. No date or time is put into a prompt.

A harness session gets the instructions, the skills index and the start-up text once, in that order, when it starts; what the harness puts before them (its own system prompt) is the harness's.

The finding. An agent whose jobs still show instructions_changed within one job, or tools_changed more than once, has something changing at the start of its prompt: a date or a counter in its instructions file, a tool description that changes, tools that come and go.

The platform never changes an agent's instructions within a job, so one change of them counts; a single change of tools is left out, since that is the platform's own, such as a spend wall leaving the model only its answer.

oz0 doctor and the console's health page list such agents per tenant from the last 7 days ("prompt prefix"), by agent and never by person. Instructions are not templated by the platform, so the cause is in the agent's own files or in its tools' listings.

Large tool results reach the model as a stub above the job's threshold (large results), which keeps a long page from filling every later request. Whether that lowers what a task costs depends on the agent: it may read the result back in parts. Measure it on your own cases with a comparison and --variant-offload off; no saving is claimed here.