Threat model

Origins, sequences and tool results

Origins, sequences and tool results for cluster operators and plugin authors.

For: cluster operators and plugin authors.

Origin checks and sequence rules

Prompt injection works by getting a model to use something it read as if a person had asked for it. For the arguments a tool marks as sensitive (a recipient, a payee, an amount, a host, a path, a repository), each job checks where the value came from before the call is made, without a model: the person's request, the starter's verified identity, a tenant record a tenant admin vouched for, or anything else the job read (tool policy).

Sequence rules decide a call by what came before it in the job or its tree, such as sending data out after something confidential was read. The edge writes each call's audit entry, with these verdicts, before the call is scheduled, and refuses to schedule it when it cannot; oz0 evidence verify decides the verdicts again from the exported trail.

What these checks stop: a value that only a page, a mail, a ticket or another tool result put in front of the model, used where the tool says a person's word is needed; a call a tenant's rule says must not follow another, in any job of the tree.

What they do not stop:

A person acting on misleading text

Text that misleads a person. A request a person types, or approves, on the strength of something misleading is a request: the check trusts it.

Trusted tenant records

A value placed in a tenant record, when the tenant vouches for that record's function: the check trusts the record.

Arguments without a role marking

Arguments a tool does not mark. Only role-marked arguments with a floor are checked; the rest go by the other rules.

Uses of permitted values

What a call does with a value it is allowed to use, and values that are near but not equal to a trusted one: values are compared exactly, so a near miss asks rather than passes.

A runtime that lies

A runtime that lies, in every way. The runtime decides each call and reports its verdicts. The edge records them, decides tree sequence rules again, refuses a call a refusing origin rule names when it carries no receipt or its own verdict fails the floor, and refuses to schedule a call without its audit entry.

The server records the digests of each call's sensitive arguments as the history holds them, and the verifier re-derives every reported verdict from the server's own reading of the history and flags a receipt that leaves an argument out, checked other values, differs from the copy the call's input carried, or is missing; it finds each call by its run, its activity and the digest of its input, as the edge and the history saw them, never by a name or call ID the node wrote.

The server cannot check these decisions fully:

  • A decision honestly reported by the runtime but approved by a person, or an argument marked differently in the node's plugin copy than in the approved snapshot read by the server, such as after a plugin update during the job.
  • Tree rules fired by a job's results, which only its runtime sees. An unreported fired rule can let a sibling's call through at the edge, and the verifier cannot see it either.
  • Values beyond the first 20 in a list argument. The call record keeps none of those, and the verifier bounds its verdict check to the first 20.

A node that runs modified code can still make a call an asking rule should have stopped; the edge only refuses, and the verifier shows it afterwards, when the exported segment holds the call's record (a note says when it does not).

How origin digests are protected

Values are kept as digests of their normalised form. In the job's history they are salted with the job tree's root; that salt is no protection against someone who knows the root and tries candidates, which is easy for an amount, a phone number or an address of the tenant's own directory.

So every digest the tenant-wide audit trail and the evidence export record is keyed once more, with a key the cluster derives per tenant from its master key: equal values still match, which is all the verifier needs, and nobody without that key can test a guess. Reading the job's history needs the content permission.

Checks on what tools return

ai.oz0/guardrails adds hooks that read every tool's result and every job's output: they redact personal data and secrets by pattern, send a result that reads like an injection to a person, and, when a tenant turns it on, ask a model whether each call fits what the job was asked to do.

They are checks, not a guarantee, and no detection rate is claimed for them. Injection defences that do well on fixed benchmarks often fail against an attacker who adapts to them, and their measures are only worth something against such attacks on a real model; none has been run for these checks yet.

What they catch: the formats their packs list, checked by a checksum where the format has one (payment cards, IBANs, national identity numbers); the tenant's own secrets wherever they appear in a result, by fingerprint; common key formats; instruction overrides, chat-template markers, forged tool calls, exfiltration phrasing and invisible text in a result.

What they miss:

  • Meaning. Names, addresses and other personal data that is not a format; an injection that persuades rather than commands, is in another language, or was written around the patterns by someone who read them.
  • Other encodings. Values in base64 or URL encoding, split across fields, or in images and files.
  • An intent check that is fooled. The intent check reads the arguments it checks, which an attacker may have written. So it is an escalate-only hook, which may only send a call to a person: a fooled check lets a call on to the rules after it, which still decide it, or asks a person too often. It sends the arguments of every call to the model it asks.
  • What hooks do not see: the model's context, a harness session's own tools after their call, and what a tool sends elsewhere by itself.
  • Commands no harness asks about. The tenant's command rules run on every node by default, with nothing installed. They read a harness session's shell command before it runs and send a catastrophic one (rm -rf ~, mkfs, a force push to main, curl ... | sh), or one they cannot read with certainty, to a person; only a rule the tenant wrote refuses one.
    They see only the commands a harness asks about: Codex's sandboxed commands, Gemini CLI's read-only tools and every call of a harness without permissions run without them, and so does whatever a script, a program or another tool of the session does once it runs.
    They catch a model's mistake and the plain forms of an attack, not someone who writes around them. The sandbox bounds what such a session reaches; the rules do not.
  • What the platform keeps. Redaction changes what the model reads and the job's output. The tool's result stays as it was in the job's history and trace, for those who may read the job's content, and a person asked about an escalation reads the value unredacted, which the approval keeps.
  • Text written to be slow. A result can be written so that a pattern takes long to scan. The plugin refuses pack patterns of the shapes known to be slow, and stops any check after 3 seconds. A stopped check fails closed, so such a result costs a person's time, not a missed check.

Keep the tool policy's rules, origin checks and taint and labels, which limit what a fooled model can do whatever these checks find.

Copyright © 2026