govoplan/docs/RELEASE_CONSOLE.md

# GovOPlaN Release Console

The release console is a local operator tool for planning and executing
GovOPlaN releases. It belongs to the `govoplan` meta repository because it works
across all local checkouts, release scripts, module manifests, migration audits,
catalog files, Git state, and signing keys.

The current implementation has a read-only dashboard, preview-only legacy
controls, and a bounded durable executor for release steps whose inputs and
effects can be verified safely:

- inspect repositories from `repositories.json`
- show dirty, ahead, behind, missing, no-HEAD, and tag state
- show local package catalog and keyring state
- optionally run release/dev migration audits
- propose next actions without executing them
- compare local catalog/keyring JSON with the published channel and public
  keyring when online checks are enabled
- configure target versions per release unit in the web UI
- select the repositories that should advance through checkboxes
- generate dry-run selective release plans for independently versioned packages
- freeze a selective plan as a durable, resumable local release run
- durably preflight a creation-time-bound repository, create its annotated tag,
  and publish its branch/tag pair atomically
- build selected Python wheels and generate a private, signed, receipt-bound
  catalog candidate
- publish that exact candidate through a verified website commit and immutable
  tag after explicit confirmation

Start it from the meta repository:

```sh
./.venv/bin/python tools/release/release-console.py
```

The launcher accepts only a numeric loopback address, binds to `127.0.0.1` by
default, always creates a fresh API token, and prints that token only in the URL
fragment. Open that URL in a browser on the same machine. A deliberately
tokenless embedded app is read-only: non-GET `/api/` requests fail with `403`.
Non-loopback operation needs a separately deployed authenticated TLS boundary;
the local launcher will not expose the mutation API that way.

Run durable release mutation only against an operator-private workspace.
Repository roots, every path ancestor, critical and nested Git metadata, and
tracked worktree files must be owned by the console process UID (or root where
appropriate) and must not be group/world writable. Symlink/special Git
metadata, object alternates, and grafts are rejected. The console pins
`/usr/bin/git`, `/usr/bin/ssh`, a fixed system `PATH`, disabled hooks, isolated
Git configuration, and no replace objects. Shared or `nfsnobody`-owned
checkouts remain usable for read-only planning, but every durable executor
fails closed there; clone the registered origins into a private workspace
before releasing.

The runtime itself is part of the authority boundary. Durable run creation
verifies the meta checkout, release/check tooling, repository registry, Python
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
executables. It then binds a clean meta-repository HEAD and named branch that
exactly match the registered `origin`. Every execution and reconciliation
rechecks that receipt. Permission checks cannot prove that code was safe before
it entered a writable tree, so release from a fresh operator-private clone or a
separately verified installed console artifact.

The web UI starts with a repository table. Each repository can be checked
independently and assigned its own target version. Repositories without version
metadata remain visible so they can be planned as initial releases. `Build Plan`
shows the dry-run commands for the selected rows, and `Generate Candidate`
creates a signed catalog candidate that advances only selected repositories that
already have a catalog entry.

`Build Plan` also returns structured release-gate findings for each selected
repository. The plan names the recommended next action and gives an explicit
remediation for source-version, lockfile, Core WebUI composition, Git state, and
worktree findings. A target version that has not yet been applied consistently
to the selected source tree is therefore explained before the source-tag
preflight rather than appearing only as an error after the operator tries to
tag. `source_preflight_ready` means that the plan-visible source gates pass; the
non-mutating `Preview Tag + Publish` remains mandatory for remote, manifest, and
immutable-tag checks.

## Durable release runs

The **Durable Run State** card turns the current repository/version selection
into a versioned local run record. The server rebuilds the selective plan and
requires the plan to resolve exactly the requested repositories and target
versions; the browser cannot submit or replace the plan snapshot. The input and
plan are then immutable and covered by a canonical SHA-256 integrity digest.
Every executable repository step also carries its creation-time full HEAD,
branch, worktree state, target tag, and a SHA-256 over both the fetch and push
URLs of `origin`. Both URLs must exactly equal the remote registered in
`repositories.json`; a changed HEAD, branch, worktree, remote, or push URL
requires a new run rather than silently retargeting the frozen compatibility
decision.
The complete record also has a checksum so a valid-looking manual edit to its
mutable state fails closed. File permissions remain the authority boundary;
these digests detect accidental or manual corruption, not an attacker who can
replace the private record and recompute its checksums. Changing a target,
channel, or gate input requires a new run.

Creation requires a caller-generated `request_id`. Its SHA-256 fingerprint is
the private, workspace-scoped durable mapping to exactly one run; the raw
identifier is never persisted. Repeating the same identifier with the same
immutable inputs returns that run without rebuilding the plan while it remains
inside the bounded local retention window, even if the live dashboard has
since drifted. Reusing it with different inputs fails closed. The browser keeps
an uncertain create identifier in session storage,
replays it after reload, and selects the known run returned by the server. A
successful create remains shown as saved if only the subsequent list refresh
fails.

Run records survive console restarts, but remain local operator state rather
than a signed release artifact or the system audit log. The default location is
`$XDG_STATE_HOME/govoplan/release-console/workspace-<sha256>/release-runs/`, or
`~/.local/state/...` when `XDG_STATE_HOME` is unset or relative. It is outside
the source checkout so filesystems without enforceable POSIX modes cannot
silently weaken the journal. Newly created state directories use mode `0700`
and records use `0600`. Writes use a cross-process lock, a same-directory
temporary file, `fsync`, atomic replacement, and directory/parent `fsync`.
Symbolic-link paths, untrusted owners or writable ancestry, overly broad record
modes, malformed schemas, unknown fields, invalid state combinations,
oversized files, and digest mismatches fail closed. A bad record is neither
rewritten nor automatically quarantined.

The store retains at most 512 workspace-scoped run records created through the
console. On creation at that limit it removes only the oldest fully completed,
integrity-verified record. Running, planned, attention, blocked, foreign, and
unreadable records are never deleted implicitly; if no completed record is
available, creation fails closed with a retention remediation. Unavailable
records still consume the bound and remain visible in cursor-paginated lists
so corruption cannot be hidden by normal turnover.

The full resolved-workspace SHA-256 is part of both the private storage
namespace and immutable input snapshot. Every list, read, and state transition
checks it. An alternate workspace therefore cannot list or resume another
workspace's runs. This remains true for an embedding/test `run_state_root`
override: the server always appends
`workspace-<full-sha256>/release-runs/` rather than treating the override as a
shared record directory. Durable candidates use its private sibling
`release-candidates/` directory, never a checkout-local runtime path. A corrupt
record from one workspace therefore cannot leak even its identifier or an
integrity error into another workspace.

Each frozen plan step has an explicit `pending`, `running`, `succeeded`,
`failed`, or `interrupted` state. Plan order remains a prerequisite: a later
step is unavailable until earlier steps have succeeded. Exact attempt and
resume/retry/reconciliation request identifiers are fingerprinted so delayed
repetitions remain idempotent for the retained run's lifetime. These fingerprints are
never evicted: the store fails closed before accepting more than 2,048 commands
or 2,048 attempts and asks the operator to create a fresh run. The display
record keeps at most 256 server-generated state events. Events contain only an
enum event type, timestamp, step identifier, and bounded result code—never
commands, process output, confirmation text, credentials, bearer tokens, or
signing material.

Before a step can enter `running`, the store reserves the two command-ledger
slots needed for worst-case recovery. It also projects the serialized record
through start, finish/failure, resume, and the required terminal reconciliation
or read-only retry; the start is rejected unless every required atomic write
fits the record-size bound. An explicit resume consumes one slot and is
accepted only while a persisted attempt is actually running. A mutating attempt
always retains its final `effect_absent` or `effect_succeeded` slot;
`unresolved` may be recorded at most once for that attempt and only when an
additional ledger slot and serialized terminal-write capacity are available.
Read-only interruption and known failure retain the slot and byte capacity
needed to prepare a retry. Capacity exhaustion is therefore detected before
starting an effect rather than stranding an ambiguous attempt.

An explicit resume after a process restart converts every persisted `running`
step to `interrupted`; it never guesses whether an external effect happened.
An interrupted read-only step can be prepared for retry. An interrupted
mutating step remains unavailable until the operator independently reconciles
local and remote state, selects `effect_absent`, `effect_succeeded`, or
`unresolved`, and types `RECONCILE`. `effect_absent` prepares a safe new
attempt, `effect_succeeded` advances the run without repeating the external
effect, and `unresolved` keeps the run blocked. Each outcome emits a bounded,
code-only state event. A known failed attempt can likewise be prepared for
retry. The UI keeps unavailable controls visible and disabled.

Supported executors durably claim the step before invoking an effect. Exact
attempt replays return the recorded outcome and never invoke the executor a
second time. Successful repository preflight, tag, and push steps persist a
bounded repository-state receipt. Tag reconciliation independently requires an
annotated local tag at the frozen HEAD; push reconciliation additionally
requires both the remote annotated tag object and remote branch to match.
Catalog generation persists
only its server-issued opaque candidate ID and canonical catalog SHA-256, then
re-resolves and re-hashes that private candidate before publication.

The safe baseline is intentionally narrower than the complete dry-run plan.
Dirty worktrees and version changes still show commit/version steps, but those
steps have no durable executor because the frozen run does not bind the exact
proposed file content. A run selecting Core together with one or more modules
begins with a disabled dependency-ordering barrier: local module tags, Core
release-lock regeneration/commit, Core tagging, alignment verification, and
pushes need an explicit DAG executor before that composition can mutate.
Operators must prepare and review those changes outside the console and create
a new run from the resulting clean HEAD. The console never skips these steps or
claims an end-to-end release succeeded.

The browser likewise retains the request identifier for an uncertain
resume/retry/reconciliation response and replays it after reload. A successful
replay selects the returned run state. Transport and server failures retain the
identifier; a deterministic `4xx` rejection clears it so a stale command cannot
poison a later attempt.

The run API is covered by the same local console token middleware as every
other `/api/` route:

- `POST /api/release-runs` requires `request_id`, then idempotently rebuilds and
  freezes a selective plan only when that creation is not already known.
- `GET /api/release-runs` lists bounded summaries ordered by immutable
  `created_at` and run identifier. `next_cursor` advances a stable descending
  traversal even while older runs are updated, without offset duplicates or
  skips. Unreadable entries
  remain a deterministic final section and are therefore reachable through
  pagination instead of displacing newer verified runs.
- `GET /api/release-runs/{run_id}` reads and verifies one exact record.
- `POST /api/release-runs/{run_id}/resume` records explicit recovery.
- `POST /api/release-runs/{run_id}/steps/{step_id}/retry` prepares a failed or
  read-only interrupted step for another attempt.
- `POST /api/release-runs/{run_id}/steps/{step_id}/reconcile` records a
  confirmed observed outcome for an interrupted mutating step.
- `POST /api/release-runs/{run_id}/steps/{step_id}/execute` claims and invokes
  only the narrow executor declared by the immutable plan step. Durable release
  execution accepts only the registered `origin`, never a caller-selected
  remote.
- `POST /api/release-runs/{run_id}/steps/{step_id}/preview` provides the
  non-mutating preview for receipt-bound catalog publication.

Run-storage errors are confined to the Durable Run State card; dashboard and
release-preview collection continue and show the bounded storage remediation.

The run record is execution evidence only for a supported step whose durable
claim and bounded result receipt were persisted. The console never infers
success from a button click or process exit alone. An executor exception or a
lost result write leaves the attempt interrupted and non-retriable until
explicit recovery. Catalog publication persists the candidate and keyring
hashes, exact website commit, annotated tag object and peeled commit, branch,
tag name, and registered-origin digest. A successful reconciliation revalidates
the candidate against the trust anchor in the frozen website parent, requires
the exact deterministic catalog/keyring/module blobs and full commit delta, a
sole frozen parent, and matching local and remote branch/tag identities. If any
part cannot be proved, the run remains interrupted and may only be recorded as
`unresolved`.

Dashboard collection is also fail closed. Unreadable Core version metadata or a
malformed module contract is returned as a bounded `collection_errors` entry
with a remediation, marks the dashboard blocked, and becomes a structured
blocker in both full and selective release plans. The console does not silently
omit a contract or turn these source errors into an HTTP 500.

The release-control area above the repository table is read-only and is meant
to become the central release cockpit. It shows:

- local and published channel health
- catalog/keyring drift
- signature and trusted-key status
- published module versions and refs
- local checkout version drift against the catalog
- catalog-declared interface compatibility

The target-version control can generate the next major, minor, or subversion
from the current base version. Manual target input accepts explicit versions
such as `0.2.0` or `0.2.0-alpha1`, but requires the first three version numbers
to move forward.

Plain repository pushes are separate from catalog publication. `Preview Push`
shows the selected repository push commands. `Push Selected` requires `PUSH` in
the repository push confirmation field.

The source release panel retains `Preview Tag + Publish` as a non-mutating
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
but disabled; the corresponding mutation endpoint rejects apply requests.
Creation and atomic branch/tag publication use the `TAG` and `PUBLISH`
confirmations on the durable run steps. The gate requires an aligned target
version, a clean named branch with a HEAD, and a checkout that is not behind. Existing
local or remote tags must resolve to the selected HEAD and are never moved.
The local and remote annotated tag objects must also be identical, not merely
point at the same commit. Before any source tag is created, the console loads
the cross-repository module registry. This release gate also rejects every
user-facing workflow documentation topic that has no scope condition, or has
an alternative condition without `required_scopes` or `any_scopes`.

The catalog workflow panel can also operate on the same selected rows for
generation and preview. Its legacy apply/push controls stay disabled; durable
publication consumes only the candidate receipt recorded by that run:

- `Generate` creates a signed candidate in the operator's private XDG state
  directory and records only an opaque candidate ID plus the canonical catalog
  SHA-256 in durable run state.
- `Preview` validates a candidate and shows what would be copied into the
  website repository.
- `Apply + Website Tag` remains visible but disabled outside a durable run.
- `Push Website Release` remains visible but disabled outside a durable run.

Source release tags belong to Core or module repositories. Website catalog
publication creates a separate catalog tag in the website repository; the UI
names these independently to avoid confusing the two immutable references.

Candidate signing is fail-closed: every Core and module source ref in the
complete post-update catalog must already have the requested annotated tag both
locally and on the configured source remote. Both tags must be the same
annotation object with version-aligned tagged package metadata. Selected tags
must additionally resolve to the selected clean, version-aligned HEAD, and the
signed `selected_units` record captures the peeled commit and tag-object
identifiers. Create and publish the source tags before using `Generate`.

Catalog publication repeats that check for selected units and also verifies
every Core and module source ref in the complete candidate, including entries
preserved from the previous catalog. Historical refs need not be at the current
worktree HEAD, but their local and remote annotated tags must match and their
tagged installable package and module-manifest metadata must declare the catalog
version. New selected releases additionally pass the stricter current-source
alignment gate, including public runtime version declarations. A candidate
cannot be applied or published while any referenced source tag is absent or
inconsistent; the preview and API response list each repository and missing or
invalid ref so the operator can repair the exact source releases first.

Every selected Python release must also supply its exact built wheel. Durable
generation first verifies that the receipt-bound annotated tag is the same
object locally and on the registered origin. It clones an isolated checkout at
the receipt's exact commit and builds from that checkout, never from the mutable
live worktree. Every authenticated base-catalog source is likewise cloned from
its registered origin at an annotated release tag; only selected repositories
contribute synthesized fields. The base catalog and keyring are read as one
authenticated, exact private snapshot before synthesis.

Python wheels are built by a required Bubblewrap worker with no network,
private temporary/home directories, a read-only exact source mount, no host
home or file keys, cleared environment, fixed system tools, and CPU/address
space/process/file-size limits. If a trusted Bubblewrap launcher is unavailable,
generation fails closed. The worker must return one bounded regular wheel; the
console copies it through no-follow descriptors into a fresh private file,
`fsync`s it, then validates its package identity. Generation computes the
archive SHA-256 and an install-stable
payload identity from one bounded, regular-file descriptor and signs those
values into `release.artifacts`. Updating a Python version without a matching
wheel removes the stale identity. A source-only preview can still explain the
gap, but apply, commit, tag, and push fail closed until every selected Python
unit has a matching built-artifact identity. Repositories selected only for a
meta/source tag do not need a Python artifact.

Candidate directories and files are operator-owned `0700`/`0600` state. Before
any later release step consumes one, the executor re-resolves the opaque handle
below the configured root and re-hashes the signed catalog against its persisted
receipt. Shared roots, symlinks, channel path fragments, altered candidates, and
unowned files are rejected.

The current same-host worker is a containment baseline, not the final hostile
build-service boundary. `RLIMIT_FSIZE` is per file, and the worker does not yet
have a size-limited filesystem/cgroup quota, a fresh kernel keyring, or a
seccomp profile denying keyctl/request-key/ptrace/mount operations. A malicious
backend could therefore exhaust scratch disk/inodes or target same-UID kernel
facilities. Closing that denial-of-service/isolation gap requires a dedicated
quota/cgroup worker with a fresh keyring and seccomp policy.

The server-owned wheel builds remain under the private candidate's `artifacts/`
directory. This slice signs their identities but does **not** upload the wheel or
add a download URL to the public catalog; the existing Git `python_ref` is source
provenance and rebuilding it is not an equivalent artifact. Keep the private
candidate until the exact wheels have been transferred through an approved
deployment channel, verified against `archive_sha256`, consumed by the installer,
and covered by its signed receipt. Public artifact transport and receipt-aware
candidate cleanup remain separate release-lifecycle work.

Read-only provenance checks derive a same-host HTTPS Git URL from conventional
SSH remotes when possible, with a non-interactive check of the configured remote
as fallback. Source tag creation and atomic publication always use the explicit
configured Git remote.

The default signing key is
`$HOME/.config/govoplan/release-keys/release-key-1.pem` when no signing key is
entered in the UI. Signing-key files must be regular, owned by the operator, and
inaccessible to group/other users. The browser sends a configured key path only
on the initial execution request; it never persists signing material in session
storage, and request-ID recovery replays no key path.

Generate a selective release plan from the terminal:

```sh
./.venv/bin/python tools/release/release-plan.py \
  --repo govoplan-files \
  --target-version 0.1.9 \
  --channel stable \
  --online
```

`--online` compares the local catalog/keyring with the published channel and
published keyring. Use `--remote-tags` only when the plan also needs to check
Git remotes for tag existence; that can be slower across the full repository
set.

Build a signed selective catalog candidate:

```sh
KEY_DIR="$HOME/.config/govoplan/release-keys"
ARTIFACT_DIR="$(mktemp -d)"

../govoplan-files/.venv/bin/python -m pip wheel \
  --no-deps \
  --no-build-isolation \
  --wheel-dir "$ARTIFACT_DIR" \
  ../govoplan-files

./.venv/bin/python tools/release/release-catalog.py selective \
  --repo-version govoplan-files=0.1.9 \
  --python-artifact \
    "govoplan-files=$ARTIFACT_DIR/govoplan_files-0.1.9-py3-none-any.whl" \
  --channel stable \
  --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem"
```

This writes candidate catalog/keyring files below
`$XDG_STATE_HOME/govoplan/release-candidates/` (or
`~/.local/state/govoplan/release-candidates/`), generates the browsable module
directory below `modules/`, validates the signed candidate with the module
installer validator, and reports whether the candidate catalog/keyring match the
currently published channel. It does not publish to the website repository.

Regenerate the browsable module directory from an existing catalog/keyring:

```sh
./.venv/bin/python tools/release/release-catalog.py module-directory \
  --catalog ../addideas-govoplan-website/public/catalogs/v1/channels/stable.json \
  --keyring ../addideas-govoplan-website/public/catalogs/v1/keyring.json \
  --output-dir runtime/module-directory-preview \
  --channel stable
```

Preview publication of a reviewed candidate:

```sh
CANDIDATE_ROOT="${XDG_STATE_HOME:-$HOME/.local/state}/govoplan/release-candidates"

./.venv/bin/python tools/release/release-catalog.py publish-candidate \
  --candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
  --channel stable
```

Apply the reviewed candidate into the website repository without pushing:

```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
  --candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
  --channel stable \
  --apply \
  --commit \
  --tag
```

Push is a separate explicit flag:

```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
  --candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
  --channel stable \
  --apply \
  --commit \
  --tag \
  --push
```

Publication validates the same in-memory catalog object that it writes; it does
not copy a path that can change after validation. On commit, the console reads
the catalog, keyring, and complete module directory back from the immutable Git
tree and requires byte-for-byte equality with those validated objects. Tags and
remote branch updates then reference that exact commit SHA rather than the
mutable worktree `HEAD`.

Published channels are expected below the public catalog base URL:

- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
- `https://govoplan.add-ideas.de/catalogs/v1/keyring.json`

The console treats channels as release artifacts. A package can advance without
forcing every repository to the same tag, but channel publication must preserve
the unchanged package versions, validate interface compatibility, sign the
updated catalog, and keep the published keyring healthy.

When a selected module exposes a WebUI package, its requested version must also
match Core's `webui/package.release.json` input and the resolved
`package-lock.release.json` entry. The source-tag preflight, selective plan, and
catalog-candidate writer all enforce this composition boundary. Pins for modules
that are not part of the selective release remain unchanged.

Release integration also enforces repository and composition version alignment
and generates a CycloneDX SBOM from the resolved Python environment and the
release WebUI lockfile. Catalog publication should attach that immutable SBOM
and its digest to the corresponding Core/composition release.

Addresses and Notifications have current WebUI source contributions but are
not part of the pinned v0.1.8 WebUI release composition because their v0.1.8
tags predate those packages. They re-enter the release composition only through
new, immutable module tags whose backend, manifest, frontend, and lock metadata
pass the alignment gate.

Candidate publication verifies signatures against the already published
keyring, not against keys supplied only by the candidate. A changed keyring
must have its canonical SHA-256 embedded in the signed catalog. The public
module-directory files are regenerated from that verified catalog and keyring
at publication time; candidate-supplied directory files are never copied as
authoritative provenance.

## Published Module Directory

The target release repository is an online, browsable module directory. The
catalog remains the machine-readable channel entry point, but the published
space should also expose a tree that operators and installations can inspect:

- `/catalogs/v1/channels/<channel>.json` describes the active channel state.
- `/catalogs/v1/keyring.json` publishes trusted release signing keys.
- `/catalogs/v1/modules/<module>/<version>/manifest.json` describes one
  published module version, including repository refs, package refs, contracts,
  compatibility windows, signatures, and available artifacts.
- `/catalogs/v1/modules/<module>/index.json` lists available versions for
  one module.
- `/catalogs/v1/modules/index.json` lists all published modules.

Gitea tags/releases remain the source release anchors. The public GovOPlaN
catalog directory becomes the installation-facing release repository that points
to those anchors and carries structured compatibility information.

Selective catalog candidates now include this directory tree. Publishing a
candidate copies the channel catalog, keyring, and `modules/` tree into the
website repository together.

Selective catalog generation can add a module that is not yet represented in
the source channel. Initial entries are synthesized from the selected
repository's `[project]` metadata, `govoplan.modules` entry point, and runtime
`ModuleManifest`; there is no separate hand-maintained initial-entry list. The
release gate requires the entry point to resolve inside the selected checkout,
exact package/manifest/frontend version agreement, verified source-tag
provenance, a non-conflicting module id, and complete required dependency and
interface closure in the resulting candidate. Classification beyond the
manifest-derived `official` tag remains an explicit later catalog curation
step rather than an inferred business/service label.

## Direction

The console should grow in slices:

1. Read-only dashboard and next-action suggestions.
2. Release plan builder that writes explicit JSON plans.
3. Dry-run executor that shows exact commands and expected file changes.
4. Apply executor for commit, tag, push, artifact build, signing, and catalog
   publication.
5. Compatibility planner for manifest contracts and version ranges.
6. Install/test workflow that validates a release from tags or catalog entries.

All mutation must stay explicit: plan first, dry run second, apply only after a
clear confirmation.