---
title: "Releases"
description: "What a release holds, how to install from one, and how to check its signatures, SBOMs and provenance."
canonical_url: "https://docs.oz0.ai/fleet/releases"
---
# Releases

> What a release holds, how to install from one, and how to check its signatures, SBOMs and provenance.

For: cluster operators.

Choose the channel and platform artifact you need, then verify it before installation. Use the upgrade guide when replacing an existing deployment.

<note>

There is no public release yet. Everything on this page works with a release candidate you build yourself with `make release-candidate`, signed with your own key, on the build machine or offline.

</note>

## What a release holds

A release is a flat list of files, as a GitHub release holds them:

<table>
<thead>
  <tr>
    <th>
      File
    </th>
    
    <th>
      What it is
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        orchestrator-zero_&lt;v>_&lt;os>_&lt;arch>.tar.gz
      </code>
      
       (<code>
        .zip
      </code>
      
       on Windows)
    </td>
    
    <td>
      The server: <code>
        bin/orchestrator-zero
      </code>
      
      , <code>
        LICENSE
      </code>
      
       and <code>
        README.md
      </code>
      
      , for Linux and macOS on amd64 and arm64 and Windows on amd64 and arm64
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        orchestrator-zero-node_&lt;v>_&lt;os>_&lt;arch>
      </code>
    </td>
    
    <td>
      The node binary, for Linux and macOS on amd64 and arm64 and Windows on amd64
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        orchestrator-zero-dist_&lt;v>.tar.gz
      </code>
    </td>
    
    <td>
      The folder the edge serves to new nodes: the node binaries, the runtime and their signed manifest. It goes next to the server's <code>
        bin/
      </code>
      
       folder
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        orchestrator-zero-runtime_&lt;v>.tar.gz
      </code>
    </td>
    
    <td>
      The Python runtime nodes install
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        orchestrator-zero-toolchain_&lt;v>_&lt;os>_&lt;arch>.tar.gz
      </code>
    </td>
    
    <td>
      Optional, one per node platform, about 150 MB each: what nodes there install before their runtime (uv, Python, the runtime's packages, Node.js), so that they reach only the edge (<a href="https://docs.oz0.ai/concepts/outbound-connections">
        Outbound connections
      </a>
      
      )
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        orchestrator-zero_&lt;v>_image_linux_&lt;arch>.tar.gz
      </code>
      
      , <code>
        orchestrator-zero-node_…
      </code>
    </td>
    
    <td>
      The server and node container images; <code>
        docker load -i
      </code>
      
       them
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        install.sh
      </code>
    </td>
    
    <td>
      Installs the server or a node from the release
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        *.cdx.json
      </code>
    </td>
    
    <td>
      A CycloneDX SBOM for each server archive, node binary, runtime, toolchain archive and image
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        release.json
      </code>
    </td>
    
    <td>
      What each file is, the version, the commit and the <a href="https://docs.oz0.ai/fleet/releases#release-channels">
        release channel
      </a>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        provenance.intoto.jsonl
      </code>
    </td>
    
    <td>
      SLSA v1 provenance for every file: what was built, from which commit and channel, with which tools
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        SHA256SUMS
      </code>
      
       and its signatures
    </td>
    
    <td>
      Every file's SHA-256, signed with the release key
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        release-sign_&lt;v>_linux_&lt;arch>
      </code>
    </td>
    
    <td>
      In a release signed offline: the signing tool itself, for linux/amd64 and linux/arm64, with its SHA-256 in the release notes
    </td>
  </tr>
</tbody>
</table>

The binaries trust the release key: an edge serves a dist folder's release only once its signature verifies, and a node checks the key's signature again before it installs a new binary or runtime from its edge ([upgrades](https://docs.oz0.ai/fleet/upgrades)).

## Release channels

Every release is on one channel, which its signed `manifest.json` (in the dist archive) and `release.json` both name, so an edge cannot change it without breaking the signature:

<table>
<thead>
  <tr>
    <th>
      Channel
    </th>
    
    <th>
      What it carries
    </th>
    
    <th>
      A cluster on it takes
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        stable
      </code>
    </td>
    
    <td>
      Releases: <code>
        v1.2.0
      </code>
    </td>
    
    <td>
      Stable releases
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        preview
      </code>
    </td>
    
    <td>
      Release candidates: <code>
        v1.3.0-rc.1
      </code>
    </td>
    
    <td>
      Release candidates and stable releases, so it never falls behind stable
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        lts-&lt;major>.&lt;minor>
      </code>
      
      , such as <code>
        lts-1.0
      </code>
    </td>
    
    <td>
      One long-term support line: <code>
        v1.0.Z
      </code>
      
      , built from the line's branch with backported fixes; never a release candidate
    </td>
    
    <td>
      Only that line's releases
    </td>
  </tr>
</tbody>
</table>

A cluster follows one channel. `oz0 server init` (and a dev server's first start) picks the channel of the release in the server's own dist folder, so a server installed from an LTS release starts on its line, and stable when there is none; `--release-channel` sets another. From then on:

- `oz0 server dist pull` mirrors only a release of the cluster's channel, and never an older node version over a newer one on the same channel;
- `oz0 node upgrade --binary` tells nodes only about a release of that channel, and refuses to send a node back to an older version;
- each node checks the same itself, against the signed manifest, before it replaces its binary.

Versions are ordered as [Semantic Versioning](https://semver.org/spec/v2.0.0.html) says: `v1.0.0-rc.2` comes before `v1.0.0`, which comes before `v1.0.1`. A build of a commit that is not a release (`make dist` from a working tree: a commit hash, or `git describe` output such as `v1.0.0-5-gabcdef1`) has no place in that order: a node that runs a release does not move to one unless you allow it, and a node that runs such a build may move to anything. Going back, or to another channel, is always an explicit act with a reason: see [Switch channels](https://docs.oz0.ai/fleet/upgrades/release-verification#switch-release-channels).

`oz0 server dist status` shows the release a dist folder serves, its channel and whether its signature verifies; with the database (`OZ0_DB`, `--db-file` or `--dev`) also the channel the cluster follows. `oz0 doctor` warns when the two disagree.

A build's flavour, such as a FIPS build, is a property beside the channel: no channel makes a cluster take a build of another flavour.

### LTS lines

An LTS line starts from a release tag: its branch `lts/<major>.<minor>` is cut from `v<major>.<minor>.0`, and its releases are `v<major>.<minor>.Z` built with `make release-candidate CHANNEL=lts-<major>.<minor>`.

Fixes go back to the line as cherry-picks that name the commit they came from, never with a database migration (a fix that needs one reaches the line's clusters with the next minor release); security fixes reach the main branch and every supported release branch no later than any LTS line. The backported commits are part of the core source. How long each line is supported is published with its first release.

With repository access, you can build a line from its branch and sign it with your own key; the binaries then trust that key, as with any build ([upgrades](https://docs.oz0.ai/fleet/upgrades/release-verification#releases-and-their-signatures)). Public source downloads are coming soon.

## Install from a release

On Linux or macOS, the server:

```bash
curl -fsSL <release-url>/install.sh | sh -s - server
oz0 server start --dev
```

As root it goes to `/opt/orchestrator-zero/<version>` with a link in `/usr/local/bin`; otherwise to `~/.local/share/orchestrator-zero` and `~/.local/bin`. `--prefix` and `--bin-dir` change both. The dist folder comes with it, so the edge can serve nodes at once.

For nodes that should reach nothing but the edge, unpack the toolchain archive of each of your nodes' platforms into the same folder: it adds `dist/toolchain/` with that platform's files and manifest (`dist/toolchain/platforms/<os>-<arch>.json`), the edge serves every platform it finds there, and nodes run with `oz0-node run --offline` (or `OZ0_NODE_OFFLINE=1`) never fall back to the internet:

```bash
tar -xzf orchestrator-zero-toolchain_<v>_linux_amd64.tar.gz -C /opt/orchestrator-zero/<version>
```

A node, joined to a cluster and run [as a service](https://docs.oz0.ai/fleet/services):

```bash
curl -fsSL <release-url>/install.sh | OZ0_JOIN_TOKEN='K10…' sh -s - node --server https://oz0-edge:7443
```

`--foreground` runs it in the terminal instead (and so does a machine without systemd, such as a container), `--no-run` joins without starting, `--service-name` and `--no-linger` pass to `service install`, `--no-sandbox-setup` skips the [sandbox setup](https://docs.oz0.ai/getting-started/install/linux-sandbox#sandbox-tier-1-on-linux) it runs as root on Linux (bubblewrap, and Ubuntu's own AppArmor profile for it), `--data-dir` and `--name` work as for `oz0-node join`, and `--from` installs from another URL or from a folder, for machines without internet access. The [one-line install from the console](https://docs.oz0.ai/fleet/join-nodes) is still the quickest way to add a node to a running cluster: it installs from your edge.

`--channel stable|preview|lts-<major>.<minor>` makes the script refuse a release that a cluster on that channel would not take, as the release's signed `release.json` says, before it installs anything: `--channel preview` takes a stable release too.

The script checks every file against `SHA256SUMS`, and `SHA256SUMS` against the release key it carries, with `openssl` (OpenSSL 3 checks ed25519 signatures; macOS's LibreSSL and OpenSSL before 3 do not). It installs only once openssl says the signature verifies. Where there is no openssl that can check it, it stops and says so: install OpenSSL 3, or check `SHA256SUMS` with [cosign](https://docs.oz0.ai/fleet/releases#check-a-release) yourself and pass `--digests-only`, which checks each file against `SHA256SUMS` alone and says so. Any other answer of openssl stops the install, with or without the flag.

It is tested on Ubuntu 24.04, Debian 13, Fedora 42 and Alpine 3.22. On Alpine (musl) the server runs and the node binary installs and joins, but the node's Python runtime does not install yet, so `oz0-node run` stops: one of its dependencies, `cel-expr-python`, publishes no musl wheel. Use a glibc distribution for nodes.

## Check a release

Get the release's public key from somewhere other than the release itself, then:

```bash
sha256sum -c SHA256SUMS
cosign verify-blob --key release.pub --bundle SHA256SUMS.sigstore.json --insecure-ignore-tlog=true SHA256SUMS
cosign verify-blob-attestation --key release.pub --bundle provenance.sigstore.json \
  --type slsaprovenance1 --insecure-ignore-tlog=true orchestrator-zero-node_<v>_linux_amd64
```

The project's releases are signed by release key A, or by its standby B, on a machine that never goes online; every official binary trusts both, so a release signed by B installs and updates like one signed by A. Their public halves, to check `release.pub` against:

<table>
<thead>
  <tr>
    <th>
      Key
    </th>
    
    <th>
      Public half (Ed25519, base64)
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Release key A
    </td>
    
    <td>
      <code>
        xRl/UKsJS2EALhhHB3lbVsk/pJI1LNNA7lOQhMEuqVw=
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Release key B (standby)
    </td>
    
    <td>
      <code>
        IEl7naTT+Y8zBsVMFlhjKFuLctFiuTS0eiydkTpJcM4=
      </code>
    </td>
  </tr>
</tbody>
</table>

`release.json`'s `key` names the one that signed; the commands above are the same whichever it is. The signatures are made with the release key and recorded in no transparency log, so `--insecure-ignore-tlog=true` is expected. cosign reads at most 128 MiB of a file it checks against the provenance, and the toolchain archives are larger: set `COSIGN_MAX_ATTACHMENT_SIZE=512MiB` for them. From a checkout of the source, `go run ./scripts/releasetool verify <folder> release.pub` checks all of it, and also that the binaries trust the key, that each SBOM belongs to its file and that the dist archive serves the release's own node binaries.

Every file but the images is reproducible: building the same commit with `make release-candidate` gives the same bytes. It refuses a tree with uncommitted changes or untracked files, since those would be built in too. With `ALLOW_DIRTY=1` it builds one as a trial: the version ends in `-dirty`, `release.json` and the provenance say `"dirty": true`, and verify prints a warning.

### Sign your own releases offline

A cluster that builds its own releases can keep its key off the build machine too. `make release-candidate SIGN=offline RELEASE_PUB=<your key's public half>` builds with no private key, reads none and makes none, and prints a folder to carry to the machine that holds the key.

There, `release-sign sign-bundle --key <seed file, or - for stdin> <folder>` checks the folder against the digests it lists, refuses a key other than the one it asks for, and writes the signatures into the folder; `release-sign show <folder>` first says what it is asked to sign.

It signs only files it can name (the dist manifest, and in Enterprise a sector pack's payload) and refuses any text it would print with a control character in it, so what the terminal shows is what gets signed.

Bring the folder back and run the same `make` command with `SIGNED=<folder>`.

A release takes two such trips: the first signs the dist manifest, which the dist archive and the server image hold, and the second the provenance and `SHA256SUMS`.

The build machine refuses anything that is not what it wrote, signed by that key, and the result checks with the same commands as above.

`release-sign` runs without Go: the release holds it, built static for Linux, and `make release-signer` builds the same bytes on another machine to compare, without reading or making a key there.

## What the provenance says

Each release carries an [SLSA v1 provenance](https://slsa.dev/spec/v1.0/provenance) in `provenance.sigstore.json`, signed with the release key. Its `buildType` and its builder's `id` are links to the two sections below, which say what they mean.

### Release candidate v1

`https://docs.oz0.ai/fleet/releases#release-candidate-v1` is the `buildType` of a release built with `make release-candidate`: it builds the server and node binaries, the runtime bundle, the toolchain archives and the images from one commit, writes their SBOMs and `release.json`, and signs `SHA256SUMS` and the provenance with the release key. Its fields are:

<table>
<thead>
  <tr>
    <th>
      Field
    </th>
    
    <th>
      What it holds
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        externalParameters.version
      </code>
    </td>
    
    <td>
      The release's version, as <code>
        release.json
      </code>
      
       names it
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        externalParameters.source
      </code>
    </td>
    
    <td>
      The repository and the commit the tree was built from
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        externalParameters.dirty
      </code>
    </td>
    
    <td>
      <code>
        true
      </code>
      
       only when the tree had uncommitted changes (<code>
        ALLOW_DIRTY=1
      </code>
      
      )
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        externalParameters.channel
      </code>
    </td>
    
    <td>
      The release channel, when the release names one
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        externalParameters.*
      </code>
    </td>
    
    <td>
      The build's other settings, such as the platforms and the image architectures, and the Go FIPS module for the FIPS flavour
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        internalParameters.SOURCE_DATE
      </code>
    </td>
    
    <td>
      The timestamp written into every file, which makes the build reproducible
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        resolvedDependencies
      </code>
    </td>
    
    <td>
      The source commit and the build's other inputs, its lock files and base images, each with its digest
    </td>
  </tr>
</tbody>
</table>

`go run ./scripts/releasetool verify` refuses a provenance of any other `buildType`. A candidate built before the `buildType` was this link (it pointed at a file in the source repository) does not verify: build it again. No release was published with the earlier value.

### Local builder

`https://docs.oz0.ai/fleet/releases#local-builder` is the builder `id` of a release candidate built on a person's machine. `builder.version` names the version of each tool the build used. SLSA counts such a build as [Build L1](https://slsa.dev/spec/v1.0/levels#build-l1): the provenance exists and is signed, but nothing stronger than the release key vouches for the machine that built it. Rebuilding the same commit and comparing the bytes is how anyone checks it further.

## Two kinds of SBOM

Releases and plugins each have their own SBOMs, both CycloneDX:

<table>
<thead>
  <tr>
    <th>
      
    </th>
    
    <th>
      Release SBOMs
    </th>
    
    <th>
      Plugin SBOMs
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Describe
    </td>
    
    <td>
      Each release file: the server and node binaries, the runtime, the toolchain archives, the images
    </td>
    
    <td>
      One plugin version: the packages its lockfiles and the edge's hash pins name, and its tools' images
    </td>
  </tr>
  
  <tr>
    <td>
      Made by
    </td>
    
    <td>
      syft at <code>
        make release-candidate
      </code>
      
      , normalised so a rebuild gives the same bytes
    </td>
    
    <td>
      Orchestrator Zero itself from the lockfiles, at <code>
        plugin pack
      </code>
      
      , at a hub's inspect and for an installed version
    </td>
  </tr>
  
  <tr>
    <td>
      Bound to
    </td>
    
    <td>
      The file's SHA-256 (<code>
        metadata.component
      </code>
      
      )
    </td>
    
    <td>
      The SHA-256 of the plugin's other files
    </td>
  </tr>
  
  <tr>
    <td>
      Signed
    </td>
    
    <td>
      In <code>
        SHA256SUMS
      </code>
      
      , with the release key
    </td>
    
    <td>
      In the plugin's bundle, and its digest in a signed hub listing
    </td>
  </tr>
  
  <tr>
    <td>
      Where
    </td>
    
    <td>
      <code>
        *.cdx.json
      </code>
      
       in the release
    </td>
    
    <td>
      <code>
        sbom.cdx.json
      </code>
      
       in the bundle, <code>
        oz0 plugin sbom
      </code>
      
      , the Plugins page, <a href="https://docs.oz0.ai/govern/audit-trail/evidence-bundles#evidence-bundles">
        evidence bundles
      </a>
    </td>
  </tr>
</tbody>
</table>

See [SBOMs](https://docs.oz0.ai/build/plugins/indexes/version-records#sboms) for the plugins'.

For a build of your own from source, `make sbom` writes the same kind of SBOMs into `dist/sbom/`: one each for the server and node binaries in `bin/`, one for the runtime bundle (with the bundle as one archive beside it), and one for each plugin in the repository's `plugins/` folder, as `plugin pack` writes it.

It uses the release's own pinned syft, downloaded once into `~/.cache/oz0-tools` and checked against its published SHA-256, and the release's own normalising, so each SBOM is bound to the SHA-256 of the file it describes and the same build gives the same SBOM.

`RUNTIME=0` and `PLUGINS=0` leave those out, and `SBOM_OUT` names another folder (a new or empty one, or one an earlier run wrote: it never clears a folder that holds anything else); images and toolchain archives are only built, with their SBOMs, by `make release-candidate`.

## The FIPS flavour

A release is built in one flavour. The [FIPS flavour](https://docs.oz0.ai/fleet/fips)'s release candidate is `make release-candidate FLAVOUR=fips`: the same files, every Go binary built against Go's FIPS 140-3 module snapshot (`FIPS140_MODULE`, default `v1.26.0`), in `dist/release/<v>-fips`. Its `release.json` and the dist folder's `manifest.json` say `"flavour": "fips"`, and `release.json`'s parameters name the snapshot (`gofips140`).

It has no container images yet. Its `install.sh` downloads from the tag `<v>-fips` unless `RELEASE_URL` names another place, and, like the default release's, installs only binaries of its own flavour: the two releases name their files alike and are signed with the same key. An edge serves, `oz0 server dist pull` mirrors and a node installs only a release of its own flavour, so a FIPS cluster never upgrades to a default build.

## Upgrade from an earlier version

Each release is tested by upgrading the data of earlier versions to it, on SQLite and PostgreSQL, with a node that replaces its own binary across the same versions.

Install the new release next to the old one, stop the server, and start the new one on the same data: it migrates the database. `install.sh server` puts the new dist folder next to the new binary, so the edge serves the new node binaries and runtime.

An edge on another machine takes the release from a folder: unpack `orchestrator-zero-dist_<version>.tar.gz` there and run `oz0 server dist pull --from <the unpacked dist folder>` (or serve the folder over HTTP and give `oz0 server dist pull` its URL).

Either way the signature is checked first, and the toolchain comes along when the folder has one. A cluster with no way out takes it from an [offline folder](https://docs.oz0.ai/fleet/offline-install).

Then upgrade the nodes from the console or with `oz0 node upgrade --binary`. See [upgrades](https://docs.oz0.ai/fleet/upgrades).


## Sitemap

See the full [sitemap](https://docs.oz0.ai/sitemap.md) for all pages.