Set reasoning effort and cascades

For: tenant administrators and cost owners.

Set a thinking level or start another model round after a failed check. Both remain within the job's budget.

Reasoning effort

How hard a model thinks is a setting of its own beside which model it is. An agent asks for one with model.effort, and the tenant decides which model it runs on:

model:
  use: smart          # which model: the tenant's alias
  needs: [tools]
  effort: high        # how hard it thinks: low, medium, high, xhigh or max

Unset, the model runs at its own default, which differs by model: Claude Opus 5.5 and Haiku 5.5 run at medium and Claude Sonnet 5.5 at high, so moving an alias from one to another changes how hard every agent on it thinks. Set an effort where it matters, and measure it: as a start, low for routing, triage and classification, medium for summaries and writing, high or xhigh for coding and long tool use, and high for judges and reviewers. An effort name never picks a model: a stronger model at a lower effort is often the better trade.

Effort precedence

When a job starts, the edge decides one level for each model the agent resolved to, the later winning:

Supported effort levels

Which levels a model takes and its default are model data: the release knows the standard aliases' models and older Claude and OpenAI models, and a provider record's model may say its own with oz0 provider model <provider> <model> --effort-levels low,medium,high --effort-default high.

A level a model lacks is rounded to the nearest one it has (the lower on a tie, never above the ceiling); a model that takes none (Claude Sonnet 4.5 and Haiku 4.5), or whose levels nobody gives, is sent none.

Under a ceiling, a model whose levels are known but whose own default is not, and that has levels above the ceiling, is sent the ceiling, since nothing says it would stay below it by itself.

Each of these is the finding effort_adjusted (info) on the job; it never fails the job.

Wire format

The runtime sends the level as Anthropic's output_config.effort, Chat Completions' reasoning_effort or the Responses API's reasoning.effort, per model, so a fallback of another vendor gets its own; the Claude harness passes it to the Agent SDK, and an ACP agent that offers a thought level gets it there. A thinking token budget is never sent. Fallbacks, cascade steps, judges and best-of candidates get the job's effort decided for their own model.

Enforce an effort ceiling

The ceiling is a governance setting: lowering or setting it applies at once, raising or removing it takes a reason and, where four eyes are on, another admin.

A job that starts under it resolves to the ceiling at most.

The gateway holds it for every call, whoever sends it: a call that asks for more is lowered to the ceiling (rounded to a level the model takes) and goes on, and a call that names none, on a model whose own default is above the ceiling, is sent the ceiling (not for Gemini's generateContent, which is only lowered).

So a job that was already running when the ceiling was set or lowered keeps running, its calls at the ceiling from then on, and so does a harness that sends an effort of its own (Claude Code sends medium when it is told none).

Each lowered call's answer carries the header Oz0-Effort-Lowered: max to medium (or default to medium), and the edge logs it.

The job's trace still says the level the job started with.

The gateway reads only where each format puts the effort (Messages' output_config.effort, Chat Completions' reasoning_effort or reasoning.effort, Responses' reasoning.effort, Gemini's thinkingLevel or thinking_level), whatever else the body holds, matching keys in any case.

A body that names an effort there that the gateway cannot read, such as a number, a list or a body that is no JSON object, is refused with 403 oz0_effort_ceiling; nothing is passed on above the ceiling.

Thinking budgets (budget_tokens, Gemini's thinkingBudget) are not held by the ceiling.

Inspect the chosen effort

oz0 model resolve smart --effort xhigh prints each model's level and what was adjusted; oz0 job get and the job's page name the model and its effort; each model step of oz0 job trace says the effort it was sent.

A change of an alias, a rule or an agent applies to jobs that start after it: a running job keeps the level it started with, also on a retry on another node, and only a ceiling below it lowers its calls at the gateway.

On the Configure tab, a rule's effort sits beside each agent's model, and a change offers the plugin's evals and a paired comparison at the new level first, as a model change does.

A cascade: the next model only when a round fails

A cascade runs a round on a small model first, and restarts it on a larger one only when its answer fails a check. You choose the models and the check; nothing picks a model for you.

model:
  primary: anthropic/claude-haiku-4-5        # or use: fast
  fallback: [mistral/mistral-small]           # the primary's, for provider errors
  cascade: [anthropic/claude-sonnet-5-5]      # one to three models, aliases or <provider>/<model>
outcome:
  grader:                                     # checks each round's answer, with no model call
    contains: ["## Summary"]
    cel: 'size(string(output)) < 4000'

What fails a round, and so starts the next model:

• the agent's outcome check: its grader, its judge, its reviewer, or a harness agent's verify script;
• its output contract, once the answer has broken the schema after its own two retries.

A provider's error, a timeout, a quota or a budget wait never does: those are what fallback is for. A cascade without an outcome check or an output schema is refused by oz0 plugin lint, since nothing could fail a round.

How another round starts

The next model does not take over the conversation. It starts a new one with the task, the last answer, why it failed, and every tool call the earlier rounds made with what came back. Those calls already ran, and the model is told so: it should use their results rather than make a write again, and the platform never runs them again.

Each result is cut at 2,000 characters and the list at 24,000; the oldest calls are left out first, and the prompt says how many.

Instructions, tools, policies, approvals and the budget are the job's, unchanged. Once the last model has run, any rounds outcome.max_rounds still allows revise on that model, as an outcome check always does; with a cascade, max_rounds defaults to one round per model.

Each step resolves when the job starts, as an alias does, and the job keeps those models even if the tenant changes an alias meanwhile. The first round's model is named in model.primary or model.use, also for a harness agent.

An agent with data: local names only local/ and test/ models in its cascade. The nodes that run a cascading agent must serve every local/ model it names, whatever its primary model is, since a restarted round runs on the job's node.

pydantic-ai agents and harness agents can cascade (a harness agent's next round is a new session on the next model); openai-agents and remote agents cannot yet, and both lint and the job refuse them with CascadeUnsupported. A cascade does not go with model.placement: fleet.

oz0 job get job-7f3a

Outcome: passed in round 2 of 2 (grader: fail, pass)
Cascade: round 1 on anthropic/claude-haiku-4-5, round 2 on anthropic/claude-sonnet-5-5 (after the grader); anthropic/claude-sonnet-5-5 answered

Every round is metered and counted against the job's budget like any other call, so a job that needs the larger model pays for both rounds. Whether a cascade costs less than the larger model alone depends on how often the small model passes: measure it with a measured-savings comparison of the agent against the variant {model: <the larger model>}, on the same cases with the same pass rule, before you count on a saving.

Both arms run with the agent's outcome.max_rounds, one round per model unless you set it, so the larger model alone gets as many rounds as the cascade does.