How an improvement runs

For: tenant administrators and plugin users.

Follow evidence collection, candidate checks and the work after a merge.

What a job does

improve-plugin takes one target plugin and a reason, such as {"target": "notes", "task": "The date format case fails"}, and goes through these steps. Each check happens before the next step, and a job that stops says why.

1. Select the target plugin

The target must be one of the plugins PLUGIN_IMPROVER_TARGETS names, the setting where an admin says where each plugin's source is. The repository and the folder come from that setting, never from the request or the model. Orchestrator Zero's own repositories and the improver itself are refused, whatever the setting says.

2. Resolve its commit

oz0_github.resolve_ref turns the target's branch (the repository's default branch unless the setting names one) into the full SHA of the commit it is at, and oz0_github.download_archive downloads that commit. A private repository is read by the connector, with your connection: the session never holds a credential.

3. Verify ownership

First the platform's own record: a flow step reads where the target is installed from with oz.evidence (kind installed), answered with the permissions of the person who granted the improver the target's results.

The target must be installed in your tenant from the repository and folder PLUGIN_IMPROVER_TARGETS names, on the host PLUGIN_IMPROVER_FORGE_HOST names: one installed from another host, repository or folder, from an uploaded bundle or an OCI image, from a hub listing of another publisher than PLUGIN_IMPROVER_PUBLISHER, or from a repository its forge says is in other hands than the setting's owner (a custody your tenant did not start) is refused, and so is a target the improver was not granted.

Then the downloaded commit: the target's folder must hold a manifest of the target's name, use none of the names only Orchestrator Zero's own plugins may (oz0_ tool IDs, oz0- agent names in the folders its manifest lists), and, when it carries a hub listing, name your own publisher. Anything else is another publisher's plugin, and the job ends with "Not changed". The manifest and the listing are read as YAML; one that does not parse is refused too. Connect GitHub as the GitHub App: its token reaches only the repositories its installation covers.

4. Prepare the change

The coding agent of your harness works on a copy of the repository at that commit, with the build kit's skills as its own. It reads and edits files and has a shell to run the target's own tests while it works (Read, Write, Edit, Glob, Grep and Bash on the Claude Agent SDK; ACP's read, edit, delete, search, think and execute on OpenCode and goose; no web fetch, and no move).

A session that can write or run a shell runs only in sandbox tier 1, whoever started the job, and the tenant's command rules decide each command: a destructive one waits for a person, with no guardrails plugin installed. Where a tenant admin opted a node out of the tier, that node runs such sessions at tier 0 and the job's page says so: keep the improver off such nodes with its install's node selector.

The session follows the kit's where a fix belongs ladder: a skill first, then a script or a tool, a tool's manual, delegation, a hook, an outcome check, and the agent's instructions last; when it adds instructions it cuts as many words and says both counts. It says which of the target's tests cover the change and how they went.

5. Run checks outside the coding session

The following checks run before anyone is asked to approve the changeset.

Verify the changed files

The platform pins a verify script from the improver definition when the job starts. It runs against the changeset tree in the session sandbox and compares file contents with the downloaded commit. File timestamps cannot hide changes, and dot-prefixed folders are included.

Each plugin containing a changed file must still load. Its manifest must parse, contain apiVersion: oz0/v1, a name and a version, and name folders that exist. The script also checks that:

• Changed agent, flow and eval definitions parse.
• Each changed SKILL.md names its folder.
• Changed JSON files read by OZ parse, including schemas/, evals/, tools.*.json and manifest-listed folders.

Existing problems outside the change do not refuse it. A deletion-only change still has its plugin manifest checked. A failed verification gets one more coding session; another failure ends the job without offering a changeset.

Report the target's offline tests

The verify script also reports the touched plugin's own tests where they can run offline: pytest, or python3 -m unittest without pytest, for Python plugins with a tests/ folder; go test ./... for a Go module; and npm test when node_modules exists. Otherwise it reports why they could not run.

The approval's changeset shows the command, exit status, counts and last output lines. A failed test is evidence for the reviewer, not an automatic refusal.

Judge the reviewer-facing answer

A judge using the fast alias checks the agent's answer against the default deliverable rubric.

Test a private candidate

The change is built and linted as an installation of the target. Unless it widens access, it becomes a private candidate on a tier-1 node open to candidates. The platform compares it with the installed version using the changed agents' eval cases.

An install-check refusal returns to the coding agent as a failed round, just like failed verification. Other findings appear beside the approval. See the candidate check.

Enforce protected paths and change budgets

A change to the target's evals, rubrics, outcome checks, policy files, CODEOWNERS or CI workflows requires a second approval from a person with the governance permission. No agent, key or policy supplies that approval. See protected paths.

If the target's change budget is spent, the changeset is refused before anyone is asked and a change_budget finding opens.

6. Review the changeset

The changeset waits as a changes approval on the Approvals page, with its diff, where it came from (the job, the agent and the improver's version, the model alias, the target and its version, and the evidence the request cited and the check's comparisons), what the candidate check found, the session's sandbox tier and the target's test report. A yes applies exactly that patch: push_changes opens a draft pull request on a branch oz0/improver-<...>, and the commit carries the same record as OZ- trailers. A no ends the job; nothing is written.

7. Read the final account

The answer is the agent's account for the reviewer: what failed, which step of the ladder the fix is at, the files it changed, the tests that cover the change and what it left alone. oz0 changes show <job> on the change step's job shows the changeset and the check.

The candidate check

Before the changes approval, the platform checks the changeset as a candidate of its target. It applies to a changeset whose target is a plugin your tenant installed and the improver holds a results grant for; any other changeset goes on as before.

• Exactly the change. The session packs the target's folder after the pinned check, and the edge takes that bundle only when every file is the downloaded commit's or what the changeset's patch made of it, and nothing else: the candidate is the code a person will approve, never other code a node sends. An earlier check answers only for the same plugin at the same installed commit, the same folder, bundle and patch: the same patch against anything else is checked again.
• Linted as an install. The server builds it as it builds any install's bytes, and runs the install's checks: lint, the agents' contracts, names and tool IDs, the secrets it may get. A refusal goes back to the agent.

Never what widens

A change that would let the target reach or hold more (an egress host, a use of another plugin's part, a secret or a setting, a tool's command, package or address, an agent's tools) is not placed: the approval says what it widens, and an install of it meets the reviews an install does.

Agents, flows and harnesses are compared whole, apart from their descriptions and an agent's instructions file: a change to a harness option (its approvals, its servers), the approvals an agent waits for or a flow's steps (an approval step taken out) is listed as the definition that changed, as is a field added in a later release.

Where it runs

The check runs as a private candidate owned by the job, only on nodes labelled candidates=allow that have sandbox tier 1, never the job's own node, with only the secrets the installed version gets. Its runs are side-effect free, as every comparison's, and carry the job's labels: its taint, and the compartments and level it holds, those it was started holding included; a job whose labels cannot be read is refused the check. Without such a node, the approval says "not run: no tier-1 node open to private candidates"; it is never skipped silently.

Nor is a check that could not be asked for: an edge the job's node could not reach, a node too old to forward it, an answer the job could not read, or a target folder the session could not pack (more files than a plugin's bundle holds, a link) shows on the approval card and the job page as "not run", with the reason.

• What it compares. One comparison per changed agent (at most three), the installed version against the candidate, on the installed version's first 20 eval cases, one run per arm and case, graded by each case's expect. A target without evals is linted only, which the approval says. Each comparison may spend at most $1, and no more than the job's own budget has left; with that spent, it does not run.
• Bounds. One check per job (a later round's replaces it), two running per tenant, at most 45 minutes each, then "not finished in time". A check that ends however it ends, replaced by a later round's included, takes its candidate with it on management's next pass, and the comparisons of a candidate that is gone stop.
• Never an approval. Nothing on this path installs, promotes or approves anything: the candidate leaves with the check, and the installed version, its versions and its rollouts are as they were. "Same" or "better" says how the eval cases went; a person still reads the change.

After a person merges the pull request, the target is updated the way any plugin is: see After a merge.

After a merge

When the improver's pull request is closed, the after-merge.yaml routine (or oz0 run improver-after-merge '{"repository": "acme/notes", "number": 7}') reads the pull request and the job the platform recorded in its body (the job its commit's OZ-Job trailer names), with no model:

• It reports how the change ended on that job with oz.report_outcome: merged (a success) or closed_unmerged (a failure). The platform records it only for a finished job of the improver's own plugin, once per name, within 14 days; the results count in the quality gates of the improver's own versions. The target's update meets the target's own gates at its rollout.
• After a merge, it proposes the target's update with oz.propose_update, as a change request a job files that a person who may install plugins approves.
  The platform writes the request's lines from its own records: how the target is installed (its source, ref, folder, node selector, repository secret and grants, kept as oz0 plugin update keeps them), the commit its installed ref resolves to now, and the rollout from PLUGIN_IMPROVER_CANARY, PLUGIN_IMPROVER_SOAK and PLUGIN_IMPROVER_SHADOW_RUNS.
  The flow's note ("Pull request #7 of acme/notes was merged into main, now at 8e7d6c5b4a3f.") is shown as the job's words. The request says that an outside event started the job, and the job's taint.

Reported merged on job improve-plugin-7f3a/improver-1c2d3e4f, the change's own job. Change request cr-0193a3c2-11
proposes updating notes to 8e7d6c5b4a3f. Nothing changes until a person who may install plugins approves it: in the
console's Reviewed changes, or oz0 change show cr-0193a3c2-11 --tenant acme.

Only a person approves it, never an API key, an app or a policy, and the person needs plugins.install. The approval runs the update as them: the update's lint, the target's evals on a test node (the quality gate), the custody check (an update from new hands files a change of its own), and the rollout to canary nodes first, behind the rollout's health and quality gates, which stop it or roll it back.

It updates to exactly the commit the request shows: when the target's ref resolves to another commit by then, or how it is installed changed, nothing is updated and the request says so. What the update widens is the approver's own update's: refused unless granted, or a change request of its own that, under second_admin or four eyes, another admin decides.

When the target follows the branch the pull request was merged into, the update moves it to the merged commit. When it is installed at a tag or a fixed commit, its ref still resolves to the installed commit, and no update is proposed ("nothing to update"): tag a release that holds the merge, or update it as you release. When it was installed without --ref, it follows the repository's highest vX.Y.Z release tag (its default branch only when it has none).

oz0 rollout abort rolls a rollout back by hand, and a rollback starts the target's cooldown for agent-made changes. oz0 changes rollback <job> proposes a pull request that reverts the change, reviewed the same way.