Lint and test a plugin

For: plugin authors.

Check plugin definitions and run their declared tests.

Lint

oz0 plugin lint            # or: oz0 plugin lint ./my-plugin
oz0 plugin lint --json     # the checked bundle and its problems, for CI

lint runs the same checks the edge runs before it installs a plugin: the manifest (unknown fields are errors), the agents it lists, its skills and the files they refer to. Each finding says how to fix it. It exits non-zero on errors; warnings, such as an unpinned package, do not fail it. Run it in your plugin's CI.

Two warnings of lint are errors at install. A Python project without uv.lock, or a Node.js project without package-lock.json, is refused by the edge (lockfiles); oz0 dev and plugin lint only warn, so you can work before the first lock. And an npm range such as npm:x@^1.2 is an error everywhere: pin one version.

With a measured listing, lint also prints what each agent's tool definitions add to every model request against its budget, and fails when an agent passes its limits.tool_tokens (8,000 unless it says) or all the plugin's functions pass the manifest's tool_tokens (16,000), naming the biggest functions (tools.budget, the size budget).

plugin test checks the listing it has just measured the same way, and plugin pack refuses to pack past a budget, so a CI job that runs lint, test or pack fails when a change makes the tools too big. The studio shows the same estimate and budget at each agent's tools:.

Tool budget (estimate): agent clerk, about 254 of 8,000 tokens per model request (lazy: the discovery tools and its built-ins; about 2,386 eager)
Tool budget (estimate): plugin, 40 functions, about 2,386 of 16,000 tokens

error: agents/triage.yaml: input: declare the agent's input: input: text, or input_schema: <file> with a JSON Schema; an error from v1.0 (installs still accept it for now)
  fix: add `input: text` to agents/triage.yaml, or `input_schema: <file>` for JSON input
warning: oz0-plugin.yaml: tools[0].id: tool "search" is a generic name that other plugins want too, and one tenant holds it for one plugin only
  fix: rename it to acme_triage_search (tools[].id)
1 of these can be fixed for you: oz0 plugin lint --fix .

Every finding has a fix: line. Lint knows the exact fix of the findings it meets most: a misspelt field (rename instructons to instructions, or move max_cost under limits: as max_cost_usd), a file that does not exist (it points at the nearest file in that folder), a negative limit, a value that must be one of a few (set runtime to pydantic-ai), a version, a duration (set limits.timeout to 10m for 10 min), a name with capitals or spaces, a model without its provider, a tool the plugin does not have, and YAML that does not parse.

The checks also fix mechanical problems where they find them:

• Clamp a number to the nearest value in its range: an eval's threshold, container limits, max_body_kb, a signature's max_age_s, colleagues.max_questions and colleagues.timeout.
• Remove a field that has no effect there: a token's encoding or message, or permissions on an ACP harness.
• Add a secret required by a header or event source to requires.secrets, and remove duplicate entries.
• Write ${secret} in a header in capitals.
• Add oz.ask_colleague to an agent that sets colleagues.
• Add status: failed to an eval case with error_contains.
• Add protocol: acp to a harness with acp: settings.
• Remove the ${{ }} wrapper from a when expression or event filter.

For the rest, the fix line names the page of these docs that says what the field takes.

--fix applies the fixes that are edits of the plugin's YAML files, such as input: text on an agent that declares no input, a renamed field, the nearest prompt file or a field taken out, keeping each file's comments and layout, and checks again. A fix can bring the next finding to light, since an agent with an unknown field does not load until it is renamed: run it again until it says nothing is left. The others need you: a rename of a tool changes the code and the agents that call it.

fixed: agents/triage.yaml: renamed instructons to instructions
fixed: agents/triage.yaml: limits.timeout: set limits.timeout to "10m"

Since v1.0, an agent that declares neither its input nor its output is an error in lint. An install still takes it, so plugins written before can be updated in the meantime; oz0 plugin lint --fix adds input: text and output: text.

Names of your own

A tenant holds each tool ID and each agent name for one plugin only, so lint warns about names that would clash with other plugins':

• A tool ID or agent name that starts with neither the plugin's name nor its first dash-separated segment. Plugin acme-risk may have tools acme_risk and acme_fetch (a - in the name is a _ in tool IDs) and agents acme-risk and acme-reviewer.
• A generic name: github, slack, discord, search, fetch, web, http, git, shell, files or reviewer.
• The prefixes oz0_ and oz0-, which belong to the verified plugins.

They are warnings, each with the new name to give it.

Test it

oz0 plugin test runs the plugin's evals on a dev server and a node of its own, with model answers recorded once (--record) and replayed after, so tests are fast, free and repeatable, in CI too. Since it starts your servers, it also writes their measured listing, tools.listing.json; commit it. --report <file> writes a report of every eval of exactly these files, which an index can attach to your listing so the version shows as validated:

oz0 plugin test ./my-plugin --record   # once, with your API key
oz0 plugin test ./my-plugin            # after: no key, no tokens

What your tools' answers cost

After the evals, plugin test probes each function of the plugin's tools that says it only reads and stays inside the tenant (readOnlyHint: true, openWorldHint: false) for cost amplification: it calls the function, on the test node, with a small input, a large one (4 KB strings, big numbers, lists of 100) and the small one twice more, and compares the answers' sizes. A function whose answer grows ten times or more with its input, or with each call, can blow up an agent's context and bill, and keep a looping agent busy.

The results go to tools.amplification.json next to the manifest, rewritten only when they change; commit it, and plugin lint, plugin inspect and installs warn about such a function:

  probe my_tool.search: x410.0 (small 10 B, large 4100 B, again [10 10] B): its answers grow with its input or each call; an agent's context and bill grow with them

--probe all calls every function, also those that write or reach outside services (a web search, a Git host's API) or declare no hints, so it can call those services with your keys; --probe off skips the probe. It is evidence for the people who install the plugin, not a gate.