--- title: "Lint and test a plugin" description: "Check plugin definitions and run their declared tests." canonical_url: "https://docs.oz0.ai/build/plugins/develop-and-test/lint-and-tests" --- # Lint and test a plugin > Check plugin definitions and run their declared tests. For: plugin authors. Check plugin definitions and run their declared tests. ## Lint ```bash [Terminal] oz0 plugin lint # or: oz0 plugin lint ./my-plugin oz0 plugin lint --json # the checked bundle and its problems, for CI ``` `lint` runs the same checks the edge runs before it installs a plugin: the manifest (unknown fields are errors), the agents it lists, its skills and the files they refer to. Each finding says how to fix it. It exits non-zero on errors; warnings, such as an unpinned package, do not fail it. Run it in your plugin's CI. Two warnings of lint are errors at install. A Python project without `uv.lock`, or a Node.js project without `package-lock.json`, is refused by the edge ([lockfiles](https://docs.oz0.ai/build/plugins/packages#lockfiles)); `oz0 dev` and `plugin lint` only warn, so you can work before the first lock. And an npm range such as `npm:x@^1.2` is an error everywhere: pin one version. With a [measured listing](https://docs.oz0.ai/build/parts/tools/snapshots#the-measured-listing), lint also prints what each agent's tool definitions add to every model request against its budget, and fails when an agent passes its `limits.tool_tokens` (8,000 unless it says) or all the plugin's functions pass the manifest's `tool_tokens` (16,000), naming the biggest functions (`tools.budget`, [the size budget](https://docs.oz0.ai/build/parts/tools/definitions#the-size-budget)). `plugin test` checks the listing it has just measured the same way, and `plugin pack` refuses to pack past a budget, so a CI job that runs `lint`, `test` or `pack` fails when a change makes the tools too big. The studio shows the same estimate and budget at each agent's `tools:`. ```text Tool budget (estimate): agent clerk, about 254 of 8,000 tokens per model request (lazy: the discovery tools and its built-ins; about 2,386 eager) Tool budget (estimate): plugin, 40 functions, about 2,386 of 16,000 tokens ``` ```text error: agents/triage.yaml: input: declare the agent's input: input: text, or input_schema: with a JSON Schema; an error from v1.0 (installs still accept it for now) fix: add `input: text` to agents/triage.yaml, or `input_schema: ` for JSON input warning: oz0-plugin.yaml: tools[0].id: tool "search" is a generic name that other plugins want too, and one tenant holds it for one plugin only fix: rename it to acme_triage_search (tools[].id) 1 of these can be fixed for you: oz0 plugin lint --fix . ``` Every finding has a `fix:` line. Lint knows the exact fix of the findings it meets most: a misspelt field (`rename instructons to instructions`, or `move max_cost under limits: as max_cost_usd`), a file that does not exist (it points at the nearest file in that folder), a negative limit, a value that must be one of a few (`set runtime to pydantic-ai`), a version, a duration (`set limits.timeout to 10m` for `10 min`), a name with capitals or spaces, a model without its provider, a tool the plugin does not have, and YAML that does not parse. The checks also fix mechanical problems where they find them: - Clamp a number to the nearest value in its range: an eval's `threshold`, container limits, `max_body_kb`, a signature's `max_age_s`, `colleagues.max_questions` and `colleagues.timeout`. - Remove a field that has no effect there: a token's `encoding` or `message`, or `permissions` on an ACP harness. - Add a secret required by a header or event source to `requires.secrets`, and remove duplicate entries. - Write `${secret}` in a header in capitals. - Add `oz.ask_colleague` to an agent that sets `colleagues`. - Add `status: failed` to an eval case with `error_contains`. - Add `protocol: acp` to a harness with `acp:` settings. - Remove the `${{ }}` wrapper from a `when` expression or event filter. For the rest, the fix line names the page of these docs that says what the field takes. `--fix` applies the fixes that are edits of the plugin's YAML files, such as `input: text` on an agent that declares no input, a renamed field, the nearest prompt file or a field taken out, keeping each file's comments and layout, and checks again. A fix can bring the next finding to light, since an agent with an unknown field does not load until it is renamed: run it again until it says nothing is left. The others need you: a rename of a tool changes the code and the agents that call it. ```text fixed: agents/triage.yaml: renamed instructons to instructions fixed: agents/triage.yaml: limits.timeout: set limits.timeout to "10m" ``` Since v1.0, an agent that declares neither its input nor its output is an error in `lint`. An install still takes it, so plugins written before can be updated in the meantime; `oz0 plugin lint --fix` adds `input: text` and `output: text`. ### Names of your own A tenant holds each tool ID and each agent name for one plugin only, so `lint` warns about names that would clash with other plugins': - A tool ID or agent name that starts with neither the plugin's name nor its first dash-separated segment. Plugin `acme-risk` may have tools `acme_risk` and `acme_fetch` (a `-` in the name is a `_` in tool IDs) and agents `acme-risk` and `acme-reviewer`. - A generic name: `github`, `slack`, `discord`, `search`, `fetch`, `web`, `http`, `git`, `shell`, `files` or `reviewer`. - The prefixes `oz0_` and `oz0-`, which belong to the verified plugins. They are warnings, each with the new name to give it. ## Test it `oz0 plugin test` runs the plugin's [evals](https://docs.oz0.ai/improve/evals) on a dev server and a node of its own, with model answers recorded once (`--record`) and replayed after, so tests are fast, free and repeatable, in CI too. Since it starts your servers, it also writes their [measured listing](https://docs.oz0.ai/build/parts/tools/snapshots#the-measured-listing), `tools.listing.json`; commit it. `--report ` writes a report of every eval of exactly these files, which an index can attach to your listing so the version shows as [validated](https://docs.oz0.ai/build/plugins/indexes/version-records#version-states): ```bash [Terminal] oz0 plugin test ./my-plugin --record # once, with your API key oz0 plugin test ./my-plugin # after: no key, no tokens ``` ### What your tools' answers cost After the evals, `plugin test` probes each function of the plugin's tools that says it only reads and stays inside the tenant (`readOnlyHint: true`, `openWorldHint: false`) for cost amplification: it calls the function, on the test node, with a small input, a large one (4 KB strings, big numbers, lists of 100) and the small one twice more, and compares the answers' sizes. A function whose answer grows ten times or more with its input, or with each call, can blow up an agent's context and bill, and keep a looping agent busy. The results go to `tools.amplification.json` next to the manifest, rewritten only when they change; commit it, and `plugin lint`, `plugin inspect` and installs warn about such a function: ```text probe my_tool.search: x410.0 (small 10 B, large 4100 B, again [10 10] B): its answers grow with its input or each call; an agent's context and bill grow with them ``` `--probe all` calls every function, also those that write or reach outside services (a web search, a Git host's API) or declare no hints, so it can call those services with your keys; `--probe off` skips the probe. It is evidence for the people who install the plugin, not a gate. ## Sitemap See the full [sitemap](https://docs.oz0.ai/sitemap.md) for all pages.