Retain records and restore backups

For: tenant admins and governance reviewers.

Keep what a law asks you to keep

Some records must outlive an erasure: bookkeeping and anti-money-laundering rules make a business keep them for years. An agent's keep_for keeps its jobs at least that long:

name: bookkeeper
runtime: pydantic-ai
keep_for: 7y

A tenant's admins can set or override it per agent or flow, with a reason:

oz0 retention keep-for
oz0 retention keep-for bookkeeper 10y --basis bookkeeping-act --reason "bookkeeping act, chapter 7"
oz0 retention keep-for bookkeeper --clear --reason "back to the definition"

A period is 30d, 12w or 7y (a year counts 366 days, so a period is never shorter than the one meant), at most 30 years. It counts from the job's start.

The platform suggests no period: which one applies to your records is yours to decide.

Lengthening a rule tightens; shortening or clearing it loosens and waits for another admin under four eyes (retention.keep_for).

--basis records the legal basis you keep the jobs under, in your own words: an ID from your tenant's own list of legal bases, the one purposes take theirs from (oz0 inventory purposes set --legal-basis bookkeeping-act="..."). The platform ships no basis and names none by default. Recording one where there was none tightens; changing or removing it loosens, as a purpose's basis does, and a basis a rule records stays on the list until the rule changes.

A definition's keep_for applies as soon as its plugin version is installed, so installing a version that brings a period, changes it or drops it warns the installer (plugin install prints it with the version's other warnings) and records retention.keep_for.plugin with the agent, the old and new period and any tenant rule that applies instead.

Read it as you would a new permission: a plugin's period holds people's data that long and raises how long the namespace keeps histories. A tenant rule overrides the definition.

Then an erasure erases everything else at once and defers the jobs of agents with a period until it ends.

One subject.erase.deferred entry per agent says which agent, which period, until when and how many jobs, with the reason keep_for and, when the agent's rule records one, the legal basis you recorded (basis).

The platform names no article or basis itself: whether a period lets an erasure wait, and under which law, is yours to decide.

Entries written before this release carry the label GDPR Art. 17(3)(b) they were written with.

When the period ends, the eraser erases those jobs by itself, and subject.erase.done closes the erasure.

A legal hold stops that too, for the held tenant only. subject find and the console's Privacy page show what is deferred and until when.

A period is also a minimum retention: the tenant's retention keeps the agent's jobs at least that long, and the tenant's Temporal namespace keeps its histories at least the longest period of its agents.

A namespace keeps one number for all its jobs, so one agent with a ten-year period keeps every job's history ten years on a self-hosted Temporal.

Where the backend keeps less (Temporal Cloud keeps 90 days at most), or the period is longer than the edge's 90 days for stored payloads, the edge copies the agent's closed jobs' histories, with their stored payloads, as held copies that retention deletes when the period ends. A delegated child job is kept as long as its root job's agent says.

Backups

A management backup (server backup) holds the wrapped keys. Restoring a backup taken before an erasure would bring the erased data back, so management records every erasure in an erasure ledger, a file outside the database (--erasure-ledger, by default erasure-ledger.jsonl in the server's data directory; keep it on storage apart from the database's backups). Each line names the erasure, the tenant and the person's hash, and is authenticated with a key the master key derives and chained to the line before, so a forged or removed line is found.

An erasure is recorded or it is not done. Management writes and syncs the ledger line in the transaction that applies the erasure, before it commits, and the database keeps which line holds it.

When the ledger cannot record it (a full disk, a file that cannot be written, a ledger that does not verify, or one that lost lines from its end since the database last anchored one), nothing is erased and subject erase (or the approval of a second-person erasure) fails and says that a cluster admin finds why in oz0 doctor; ask for the erasure again once the ledger is fixed.

A line that an interrupted write left cut short belongs to an erasure that was not applied, and the next erasure cuts it off.

A line written by an erasure whose commit then failed stays in the ledger, and a restore applies it: it erases a person someone asked to erase.

To start a new ledger, give --erasure-ledger another file and keep the old one for restores.

A restore applies again every erasure the ledgers recorded after the backup was taken, before any server takes a job:

OZ0_DB="$DB" oz0 server restore backup.tar.gz --master-key-file master.key --erasure-ledger /mnt/ledger/erasure-ledger.jsonl

What holds until then: between the restore of an older backup and the re-application, and for every restore made without the ledger, the backup holds the erased people's data for whoever holds it and the master key.

A backup of Temporal's own databases holds encrypted histories whose keys an erasure destroyed, unless the management backup restored with them is older than the erasure and no ledger re-applies it.

Whether that meets your deadline for erasure is a legal question; keep backups no longer than you need, and restore with the ledger.

oz0 doctor reads the whole ledger, as a restore would, and fails its erasure ledger check when a line does not verify, when the file cannot be read, or when it no longer holds the newest erasure the database says it recorded; in a cluster it warns when a management server records erasures in no ledger.