Jobs and durability
For: cluster operators and plugin authors.
A job is one run of an agent, implemented as a Temporal AgentWorkflow. Its job ID is the workflow ID. Temporal holds the job's status, input, answer and complete history; no separate system stores the job.
What durable means here
- The agent's loop is workflow code. Temporal records every step in the job's history.
- Every model call is an activity. It streams, has a ten-minute timeout and up to five attempts, and heartbeats while it runs. A crashed node loses at most the call in flight.
Temporal is its only retry layer: the provider SDKs do not retry on their own, so an attempt is one call to the provider, and it waits as long as the provider'sretry-afterasks. An error no retry can fix, such as a bad request or a missing key, fails the job at once.
A spent budget waits for a person's decision when a person started the job, and fails at once otherwise, unless the agent'son_budgetsays differently (budgets that wait). - Every tool call is an activity on the home node's queue. Its result is recorded, so it is not called again on replay. Tools can have side effects, so a failed tool call is not retried, unless its function is read-only (MCP's
readOnlyHint), or idempotent and its server honours the call's idempotency key: those run again, up to three times (tools). Otherwise the error goes back to the model, which decides what to do. - Workflow code reads nothing of the world itself. Network, files, the clock, randomness and the environment are reached only from activities, whose results the history records, so a replay on any node makes the same decisions. The project's lint checks this on every change, and the workflows run in Temporal's workflow sandbox.
A runtime upgrade does not break the jobs already running: every new step is added behind a patch marker, and histories recorded during development, for every kind of job a node runs (agent, flow, tool-call, judge, event, continuation and shadow jobs), are replayed in the project's tests, which fail on a step added without one.
The lint cannot tell a new step from an old one by itself; that half rests on those replay tests, and holds for the steps their histories reach. - Stopping the client does not stop the job.
oz0 runwaits for the answer, but if you press Ctrl+C the job keeps running. Follow it withjob get.
If the server restarts, Temporal replays the recorded workflow history. In-flight activities follow their retry or recovery rules. Tool activities run on the job's home node and wait if it is unavailable; recovery of a lost harness session can select a new home.
A lost harness session does not resume its conversation. If the agent is restart-safe, a fresh session starts from job input; otherwise the job waits for a person's restart approval. Review may be needed when the lost session could have made effects the platform cannot verify. See restart waits and harness restart safety.
IDs and trees
Top-level jobs get an ID such as job-bb270cf1-b194-4bf1-a0e3-267e061e416a, or the one you pass with --id. The same request under the same ID answers with the job it started instead of starting a new one, which makes retries from apps safe; any other start under that ID is refused.
Children carry their parent's ID: <parent>/<agent>-<8 hex>. That is how job get finds a job's tree and how cost is added up per branch.
Limits and contracts
The platform enforces limits and contracts in workflow code, so a model cannot talk its way around them:
max_steps,max_tokensandtimeoutend a job when reached.- An input that does not match the agent's
input_schemafails the job before it starts. - An answer that does not match
output_schemagoes back to the model with the reason, at most twice.
See Contracts and limits.
History
Every job has a full history in Temporal: each model call with its request and response, each tool call and each child. The edge reads it as the job's steps, with what each one cost: oz0 job trace, and the job's page in the console (Job history). In dev mode the Temporal UI shows the history itself in a browser. History stays as long as the namespace keeps finished workflows; usage rows for cost stay longer.
Payloads larger than 256 KiB, such as a large input or a long conversation that every model call carries, do not go into the history itself: the node stores them on the edge, which keeps them per tenant in the database, and the history holds a reference. So a job can take inputs of many megabytes, up to 32 MiB a payload, where Temporal alone would refuse anything over 2 MB. The edge deletes a payload 90 days after a run last stored it. The Temporal UI shows such a payload as its reference.