---
title: "How an improvement runs"
description: "Follow evidence collection, candidate checks and the work after a merge."
canonical_url: "https://docs.oz0.ai/build/plugins/verified/plugin-improver/job-lifecycle"
---
# How an improvement runs

> Follow evidence collection, candidate checks and the work after a merge.

For: tenant administrators and plugin users.

Follow evidence collection, candidate checks and the work after a merge.

## What a job does

`improve-plugin` takes one target plugin and a reason, such as `{"target": "notes", "task": "The date format case fails"}`, and goes through these steps. Each check happens before the next step, and a job that stops says why.

### 1. Select the target plugin

The target must be one of the plugins `PLUGIN_IMPROVER_TARGETS` names, the setting where an admin
says where each plugin's source is. The repository and the folder come from that setting, never from the request or
the model. Orchestrator Zero's own repositories and the improver itself are refused, whatever the setting says.

### 2. Resolve its commit

`oz0_github.resolve_ref` turns the target's branch (the repository's default branch unless the
setting names one) into the full SHA of the commit it is at, and `oz0_github.download_archive` downloads that commit.
A private repository is read by the connector, with your connection: the session never holds a credential.

### 3. Verify ownership

First the platform's own record: a flow step reads where the target is installed from with [`oz.evidence`](https://docs.oz0.ai/improve/evidence-for-agents#where-a-target-is-installed-from) (kind `installed`), answered with the permissions of the person who granted the improver the target's results.

The target must be installed in your tenant from the repository and folder `PLUGIN_IMPROVER_TARGETS` names, on the host `PLUGIN_IMPROVER_FORGE_HOST` names: one installed from another host, repository or folder, from an uploaded bundle or an OCI image, from a hub listing of another publisher than `PLUGIN_IMPROVER_PUBLISHER`, or from a repository its forge says is in other hands than the setting's owner (a custody your tenant did not start) is refused, and so is a target the improver was not granted.

Then the downloaded commit: the target's folder must hold a manifest of the target's name, use none of the names only Orchestrator Zero's own plugins may (`oz0_` tool IDs, `oz0-` agent names in the folders its manifest lists), and, when it carries a hub listing, name your own publisher. Anything else is another publisher's plugin, and the job ends with "Not changed". The manifest and the listing are read as YAML; one that does not parse is refused too. Connect GitHub as the [GitHub App](https://docs.oz0.ai/build/plugins/github): its token reaches only the repositories its installation covers.

### 4. Prepare the change

The coding agent of your harness works on a copy of the repository at that commit, with the build kit's skills as its own. It reads and edits files and has a shell to run the target's own tests while it works (Read, Write, Edit, Glob, Grep and Bash on the Claude Agent SDK; ACP's read, edit, delete, search, think and execute on OpenCode and goose; no web fetch, and no move).

A session that can write or run a shell runs only in [sandbox tier 1](https://docs.oz0.ai/concepts/security/sandbox-tiers#sandboxes), whoever started the job, and the tenant's [command rules](https://docs.oz0.ai/govern/command-rules) decide each command: a destructive one waits for a person, with no guardrails plugin installed. Where a tenant admin opted a node out of the tier, that node runs such sessions at tier 0 and the job's page says so: keep the improver off such nodes with its install's node selector.

The session follows the kit's [where a fix belongs](https://docs.oz0.ai/improve/where-a-fix-belongs) ladder: a skill first, then a script or a tool, a tool's manual, delegation, a hook, an outcome check, and the agent's instructions last; when it adds instructions it cuts as many words and says both counts. It says which of the target's tests cover the change and how they went.

### 5. Run checks outside the coding session

The following checks run before anyone is asked to approve the changeset.

#### Verify the changed files

The platform pins a verify script from the improver definition when the job starts. It runs against the changeset tree in the session sandbox and compares file contents with the downloaded commit. File timestamps cannot hide changes, and dot-prefixed folders are included.

Each plugin containing a changed file must still load. Its manifest must parse, contain `apiVersion: oz0/v1`, a name and a version, and name folders that exist. The script also checks that:

- Changed agent, flow and eval definitions parse.
- Each changed `SKILL.md` names its folder.
- Changed JSON files read by OZ parse, including `schemas/`, `evals/`, `tools.*.json` and manifest-listed folders.

Existing problems outside the change do not refuse it. A deletion-only change still has its plugin manifest checked. A failed verification gets one more coding session; another failure ends the job without offering a changeset.

#### Report the target's offline tests

The verify script also reports the touched plugin's own tests where they can run offline: pytest, or `python3 -m unittest` without pytest, for Python plugins with a `tests/` folder; `go test ./...` for a Go module; and `npm test` when `node_modules` exists. Otherwise it reports why they could not run.

The approval's changeset shows the command, exit status, counts and last output lines. A failed test is evidence for the reviewer, not an automatic refusal.

#### Judge the reviewer-facing answer

A judge using the `fast` alias checks the agent's answer against the [default deliverable rubric](https://docs.oz0.ai/improve/outcome-checks/judges#the-default-rubric).

#### Test a private candidate

The change is built and linted as an installation of the target. Unless it widens access, it becomes a [private candidate](https://docs.oz0.ai/build/plugins/develop-and-test/candidate-checks#a-changesets-candidate-check) on a tier-1 node open to candidates. The platform [compares it](https://docs.oz0.ai/improve/comparisons-of-a-changed-agent#a-candidate-check) with the installed version using the changed agents' eval cases.

An install-check refusal returns to the coding agent as a failed round, just like failed verification. Other findings appear beside the approval. See [the candidate check](https://docs.oz0.ai/build/plugins/verified/plugin-improver/job-lifecycle#the-candidate-check).

#### Enforce protected paths and change budgets

A change to the target's evals, rubrics, outcome checks, policy files, CODEOWNERS or CI workflows requires a second approval from a person with the governance permission. No agent, key or policy supplies that approval. See [protected paths](https://docs.oz0.ai/govern/reviewed-changes/protected-paths#protected-paths).

If the target's [change budget](https://docs.oz0.ai/govern/reviewed-changes/protected-paths#a-change-budget-per-plugin) is spent, the changeset is refused before anyone is asked and a `change_budget` finding opens.

### 6. Review the changeset

The changeset waits as a changes approval on the Approvals page, with its diff,
[where it came from](https://docs.oz0.ai/govern/reviewed-changes/changesets#where-a-change-came-from) (the job, the agent and the improver's
version, the model alias, the target and its version, and the evidence the request cited and the check's
comparisons), what the candidate check found, the session's sandbox tier and the target's test report. A yes
applies exactly that patch: `push_changes` opens a draft pull request on a branch `oz0/improver-<...>`, and the
commit carries the same record as `OZ-` trailers. A no ends the job; nothing is written.

### 7. Read the final account

The answer is the agent's account for the reviewer: what failed, which step of the ladder the fix is at, the
files it changed, the tests that cover the change and what it left alone. `oz0 changes show <job>` on the change
step's job shows the changeset and the check.

### The candidate check

Before the changes approval, the platform checks the changeset as a candidate of its target. It applies to a changeset
whose target is a plugin your tenant installed and the improver holds a results grant for; any other changeset goes on
as before.

- Exactly the change. The session packs the target's folder after the pinned check, and the edge takes that bundle
only when every file is the downloaded commit's or what the changeset's patch made of it, and nothing else: the
candidate is the code a person will approve, never other code a node sends. An earlier check answers only for the same
plugin at the same installed commit, the same folder, bundle and patch: the same patch against anything else is
checked again.
- Linted as an install. The server builds it as it builds any install's bytes, and runs the install's checks:
lint, the agents' contracts, names and tool IDs, the secrets it may get. A refusal goes back to the agent.

#### Never what widens

A change that would let the target reach or hold more (an egress host, a use of another plugin's part, a secret or a setting, a tool's command, package or address, an agent's tools) is not placed: the approval says what it widens, and an install of it meets the reviews an install does.

Agents, flows and harnesses are compared whole, apart from their descriptions and an agent's instructions file: a change to a harness option (its approvals, its servers), the approvals an agent waits for or a flow's steps (an approval step taken out) is listed as the definition that changed, as is a field added in a later release.

#### Where it runs

The check runs as a private candidate owned by the job, only on nodes labelled `candidates=allow` that have sandbox tier 1, never the job's own node, with only the secrets the installed version gets. Its runs are side-effect free, as every comparison's, and carry the job's labels: its taint, and the compartments and level it holds, those it was started holding included; a job whose labels cannot be read is refused the check. Without such a node, the approval says "not run: no tier-1 node open to private candidates"; it is never skipped silently.

Nor is a check that could not be asked for: an edge the job's node could not reach, a node too old to forward it, an answer the job could not read, or a target folder the session could not pack (more files than a plugin's bundle holds, a link) shows on the approval card and the job page as "not run", with the reason.

- What it compares. One comparison per changed agent (at most three), the installed version against the
candidate, on the installed version's first 20 eval cases, one run per arm and case, graded by each case's `expect`.
A target without evals is linted only, which the approval says. Each comparison may spend at most $1, and no more
than the job's own budget has left; with that spent, it does not run.
- Bounds. One check per job (a later round's replaces it), two running per tenant, at most 45 minutes each, then
"not finished in time". A check that ends however it ends, replaced by a later round's included, takes its
candidate with it on management's next pass, and the comparisons of a candidate that is gone stop.
- Never an approval. Nothing on this path installs, promotes or approves anything: the candidate leaves with the
check, and the installed version, its versions and its rollouts are as they were. "Same" or "better" says how the
eval cases went; a person still reads the change.

After a person merges the pull request, the target is updated the way any plugin is: see [After a merge](https://docs.oz0.ai/build/plugins/verified/plugin-improver/job-lifecycle#after-a-merge).

## After a merge

When the improver's pull request is closed, the `after-merge.yaml` routine (or
`oz0 run improver-after-merge '{"repository": "acme/notes", "number": 7}'`) reads the pull request and the job the
platform recorded in its body (the job its commit's `OZ-Job` trailer names), with no model:

- It reports how the change ended on that job with [`oz.report_outcome`](https://docs.oz0.ai/improve/outcome-checks/rollouts#downstream-results):
`merged` (a success) or `closed_unmerged` (a failure). The platform records it only for a finished job of the
improver's own plugin, once per name, within 14 days; the results count in the quality gates of the improver's own
versions. The target's update meets the target's own gates at its rollout.
- After a merge, it proposes the target's update with `oz.propose_update`, as a
[change request a job files](https://docs.oz0.ai/govern/reviewed-changes/proposals#change-requests-a-job-files) that a person who may install
plugins approves.<br />

The platform writes the request's lines from its own records: how the target is installed (its
source, ref, folder, node selector, repository secret and grants, kept as `oz0 plugin update` keeps them), the commit
its installed ref resolves to now, and the rollout from `PLUGIN_IMPROVER_CANARY`, `PLUGIN_IMPROVER_SOAK` and
`PLUGIN_IMPROVER_SHADOW_RUNS`.<br />

The flow's note ("Pull request #7 of acme/notes was merged into main, now at
8e7d6c5b4a3f.") is shown as the job's words. The request says that an outside event started the job, and the job's
taint.

```text
Reported merged on job improve-plugin-7f3a/improver-1c2d3e4f, the change's own job. Change request cr-0193a3c2-11
proposes updating notes to 8e7d6c5b4a3f. Nothing changes until a person who may install plugins approves it: in the
console's Reviewed changes, or oz0 change show cr-0193a3c2-11 --tenant acme.
```

Only a person approves it, never an API key, an app or a policy, and the person needs `plugins.install`. The approval runs the update as them: the update's lint, the target's evals on a test node ([the quality gate](https://docs.oz0.ai/improve/evals/quality-gates#quality-gates)), the custody check (an update from new hands files a change of its own), and the [rollout](https://docs.oz0.ai/build/plugins/install/rollouts#new-versions-roll-out-to-canaries-first) to canary nodes first, behind the rollout's health and quality gates, which stop it or roll it back.

It updates to exactly the commit the request shows: when the target's ref resolves to another commit by then, or how it is installed changed, nothing is updated and the request says so. What the update widens is the approver's own update's: refused unless granted, or a change request of its own that, under `second_admin` or four eyes, another admin decides.

When the target follows the branch the pull request was merged into, the update moves it to the merged commit. When it is installed at a tag or a fixed commit, its ref still resolves to the installed commit, and no update is proposed ("nothing to update"): tag a release that holds the merge, or update it as you release. When it was installed without `--ref`, it follows the repository's highest `vX.Y.Z` release tag (its default branch only when it has none).

`oz0 rollout abort` rolls a rollout back by hand, and a rollback starts the target's cooldown for agent-made changes. `oz0 changes rollback <job>` proposes a pull request that reverts the change, reviewed the same way.


## Sitemap

See the full [sitemap](https://docs.oz0.ai/sitemap.md) for all pages.