Back up a cluster
For: cluster operators.
Back up
server backup writes one archive of the management database while the servers keep running. It reads every table in one snapshot, and for a dev server it copies the embedded Temporal's database too:
oz0 server backup --out oz0-backup.tar.gz # a cluster: OZ0_DB or --db-file
oz0 server backup --dev --data-dir .oz0/dev --out oz0-backup.tar.gz
The archive holds tenants, nodes and join tokens, plugin records, sealed secrets, accounts and memberships, usage, the audit log and job summaries. It includes artifact bytes and large job payloads only when those bytes are stored in the management database being backed up.
A separate SQL payload database and an S3 payload bucket need separate backups. An S3-backed artifact's database row is in the archive; its bytes remain in the bucket. Preserve a consistent database and payload-store recovery point. See payload storage and backups.
The archive also leaves out:
- the master key. Keep it apart, and give it to
server restore; - what describes the moment: live servers, nodes' connections and sign-in sessions;
- the dist directory, which
make distorserver dist pullfills again.
| What | Why | How |
|---|---|---|
| The master key file | Without it, the CAs, secrets and payload keys cannot be unsealed, encrypted histories can never be read again, and the cluster is lost | Copy it somewhere safe and offline once, after server init, then server master-key confirm-backup |
| The secrets key (roles apart) | Without it, tenant secrets sealed to it are lost | Keep it with the master key's copy, then server secrets-key confirm-backup |
| The management database | Everything in the list above | server backup on a schedule, or PostgreSQL's own backups for point-in-time recovery |
| A separate SQL payload database | Artifact bytes and large payloads stored outside the management database | Back up the database named by --payload-db-file with PostgreSQL's tools; coordinate its recovery point with the management database |
| An S3 payload bucket | Artifact bytes and large payloads stored in objects | Use the bucket provider's backup tools, such as versioning and replication; preserve a recoverable copy matching the database backup |
| Temporal's two databases (a cluster) | Running jobs and their recent histories | PostgreSQL's own backups: the archive does not hold them |
| Operator contexts | Operator certificates | Renew them with operator renew. If one is lost or expired, issue new credentials on a server with server operator issue. Keep them private |
| The erasure ledger | Erasures made after a backup was taken, which a restore applies again (backups and erasure) | Each management server appends to --erasure-ledger (default erasure-ledger.jsonl in its data directory; the Helm chart and docker compose put it on a volume of its own): keep it on storage that outlives the server and apart from the database's backups. It holds hashes, never a person |
Nodes hold nothing you need to back up: their plugins come from the edge, and a lost node can simply join again.
A backup is not a tenant's export. A backup restores this cluster, for every tenant at once, with the master key. To take one tenant elsewhere, or to keep its records outside the cluster in documented formats without any sealed value, use oz0 tenant export.
Backups and payload keys. Each tenant's payload keys are in the management database, sealed with the master key. A history written with a key made after your newest backup cannot be opened after a database loss, on any Temporal backend: on Temporal Cloud the history survives, as ciphertext only those keys open.
server backup records when it ran, and oz0 doctor warns (database backup) while the newest one is older than the newest payload key. A backup made with PostgreSQL's own tools is recorded by hand: oz0 server backup record --taken-at 2026-10-08T03:00:00Z. Both confirmations and every recorded backup are in the audit log (key.backup.confirmed, backup.recorded).
Restore a saved backup
Use the restore procedure to recover into an empty database.