From 75bca04dfe7076f0a24015bc3ffd7362c4fe974b Mon Sep 17 00:00:00 2001 From: zemion Date: Thu, 9 Jul 2026 13:18:57 +0200 Subject: [PATCH] Sync Repo-docs-RELEASE-DEPENDENCIES from project files --- Repo-docs-RELEASE-DEPENDENCIES.-.md | 629 ++++++++++++++++++++++++++++ 1 file changed, 629 insertions(+) create mode 100644 Repo-docs-RELEASE-DEPENDENCIES.-.md diff --git a/Repo-docs-RELEASE-DEPENDENCIES.-.md b/Repo-docs-RELEASE-DEPENDENCIES.-.md new file mode 100644 index 0000000..597f120 --- /dev/null +++ b/Repo-docs-RELEASE-DEPENDENCIES.-.md @@ -0,0 +1,629 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# GovOPlaN Release Dependencies + +This document owns release package composition, signed package catalogs, +license checks, catalog publishing, migration baselines, and the final release +checklist. + +Operator runtime configuration and module install/uninstall execution live in +`DEPLOYMENT_OPERATOR_GUIDE.md`. + +## Backend Packages + +Release installs must not depend on sibling checkout paths. Local development +can keep editable installs and `file:` WebUI links, but release packaging must +resolve modules from tagged git refs or from a package registry. + +Local development: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m pip install -r requirements-dev.txt +``` + +Release install from a core checkout plus tagged module repositories: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m pip install -r requirements-release.txt +``` + +`.[server]` is resolved relative to the current working directory. If you +create the virtualenv elsewhere, still run the install command from the core +checkout: + +```bash +cd /mnt/DATA/git/govoplan-core +/tmp/govoplan-release-test/bin/python -m pip install -r requirements-release.txt +``` + +`requirements-release.txt` pins the module repositories to the release tag. +Update those refs when cutting a release: + +```text +govoplan-access git@git.add-ideas.de:add-ideas/govoplan-access.git v0.1.6 +govoplan-admin git@git.add-ideas.de:add-ideas/govoplan-admin.git v0.1.6 +govoplan-tenancy git@git.add-ideas.de:add-ideas/govoplan-tenancy.git v0.1.6 +govoplan-policy git@git.add-ideas.de:add-ideas/govoplan-policy.git v0.1.6 +govoplan-audit git@git.add-ideas.de:add-ideas/govoplan-audit.git v0.1.6 +govoplan-files git@git.add-ideas.de:add-ideas/govoplan-files.git v0.1.6 +govoplan-mail git@git.add-ideas.de:add-ideas/govoplan-mail.git v0.1.6 +govoplan-campaign git@git.add-ideas.de:add-ideas/govoplan-campaign.git v0.1.6 +govoplan-calendar git@git.add-ideas.de:add-ideas/govoplan-calendar.git v0.1.6 +``` + +## WebUI Packages + +Local development uses `webui/package.json`, which may point at sibling module +checkouts while active development is happening. + +Release WebUI installs should use `webui/package.release.json`. It points +module dependencies at the same tagged git repositories. After the module tags +referenced there exist, generate the committed release lockfile without +touching the development package files: + +```bash +cd /mnt/DATA/git/govoplan-core +scripts/generate-release-lock.sh +cd webui +PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build +``` + +The module repositories include root-level npm package manifests so git +installs can resolve `@govoplan/access-webui`, `@govoplan/admin-webui`, +`@govoplan/files-webui`, `@govoplan/mail-webui`, +`@govoplan/campaign-webui`, and `@govoplan/calendar-webui` from repository +roots even though their source lives below `webui/src`. + +### Release Lockfile Strategy + +The supported release composition currently is the full GovOPlaN product: core +plus access, admin, tenancy, policy, audit, files, mail, campaign, and +calendar. Keep one committed full-product release lockfile at +`webui/package-lock.release.json`, generated from +`webui/package.release.json` in a clean release workspace. Development +`package-lock.json` may continue to point at local `file:` dependencies. + +Frontend module permutations are regression-tested through +`GOVOPLAN_WEBUI_MODULE_PACKAGES` and temporary build output, not through +committed lockfiles for every possible combination. If a smaller composition +becomes a separately shipped product, add an explicit release manifest and +lockfile pair for that product, for example +`package.release.files-mail.json` and `package-lock.release.files-mail.json`, +generated in a clean release workspace from tagged git dependencies. + +## Release Tag Script + +The normal release path is automated by `scripts/push-release-tag.sh`: it bumps +or accepts the target version, updates Python/WebUI/module manifest versions, +commits/tags/pushes the module repositories first, regenerates +`webui/package-lock.release.json`, and then commits/tags/pushes core. If the +working tree has already been bumped, pass the current version explicitly: + +```bash +cd /mnt/DATA/git/govoplan-core +scripts/push-release-tag.sh --version 0.1.6 +``` + +The script also includes GovOPlaN roadmap/scaffold module repositories that do +not yet have package metadata. Those repositories are committed, tagged, and +pushed with the same release tag, but they are tag-only until they contain +`pyproject.toml`, module manifests, or WebUI packages. Tag-only repositories +are not listed in `requirements-release.txt` or `webui/package.release.json`. + +Current tag-only module repositories: + +- `govoplan-addresses` +- `govoplan-appointments` +- `govoplan-cases` +- `govoplan-connectors` +- `govoplan-dms` +- `govoplan-erp` +- `govoplan-fit-connect` +- `govoplan-forms` +- `govoplan-identity-trust` +- `govoplan-idm` +- `govoplan-ledger` +- `govoplan-notifications` +- `govoplan-ops` +- `govoplan-payments` +- `govoplan-portal` +- `govoplan-reporting` +- `govoplan-scheduling` +- `govoplan-search` +- `govoplan-tasks` +- `govoplan-templates` +- `govoplan-workflow` +- `govoplan-xoev` +- `govoplan-xrechnung` +- `govoplan-xta-osci` + +## Catalog Trust And Licensing + +GovOPlaN module install and uninstall must remain operator-controlled. The +running server may plan and validate package changes, but package mutation is +performed by the separate installer daemon or an operator shell during +maintenance mode. + +`govoplan-web` is the public static distribution surface for official catalog +resources: + +- signed module package catalogs, grouped by release channel +- public catalog keyrings +- public license verification keyrings +- examples and operator-facing download paths + +`govoplan-core` is the verifier and orchestrator: + +- fetches a local or remote module catalog +- verifies catalog signatures against configured trusted keys +- enforces approved release channels +- rejects expired or not-yet-valid catalogs +- records accepted catalog sequence numbers for replay protection +- checks catalog entry license feature requirements before planning installs +- writes installer plans and request records + +Feature and platform modules own their package artifacts, manifests, migration +metadata, retirement providers, and optional lifecycle behavior. + +Core accepts either a local catalog file or a remote URL: + +```bash +GOVOPLAN_MODULE_PACKAGE_CATALOG=/srv/govoplan/catalogs/stable.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.example/catalogs/v1/channels/stable.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/catalog-cache/stable.json +``` + +If both file and URL are set, the URL wins. The cache is used when a remote +fetch fails, so an operator can still inspect the last known catalog. A cached +catalog must still pass signature, freshness, channel, and replay validation. + +An official catalog is a JSON object with: + +- `catalog_version` +- `channel` +- `sequence` +- `generated_at` +- `not_before` when delayed activation is needed +- `expires_at` +- `modules` +- `signatures` + +Each module entry can declare: + +- backend package name and pinned install reference +- WebUI package name and pinned install reference +- display metadata and tags +- `license_features`, the feature entitlements required to plan that install + +The signature is Ed25519 over canonical JSON with both `signature` and +`signatures` removed. Core accepts the legacy single `signature` field and the +new `signatures` array. + +Trusted catalog keys are configured locally: + +```bash +GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/catalog-keyring.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS='{"release-key-1":""}' +``` + +For development or tightly controlled deployments, a keyring can be read from a +URL and cached: + +```bash +GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_URL=https://govoplan.example/catalogs/v1/keyring.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_CACHE=/srv/govoplan/runtime/catalog-cache/keyring.json +``` + +Production installations should pin the trusted keyring locally or ship it +through deployment configuration. Fetching trusted keys from the same public +origin as the catalog is convenient, but that origin must not become the only +trust root. + +## Dependency Audits + +Dependency vulnerability checks are documented in +[`DEPENDENCY_AUDITS.md`](DEPENDENCY_AUDITS.md). The local audit runner is: + +```bash +cd /mnt/DATA/git/govoplan-core +scripts/check-dependency-audits.sh +``` + +The Gitea workflow in `.gitea/workflows/dependency-audit.yml` runs the same +check against release dependency refs on pushes, pull requests, and a weekly +schedule. + +Keyring entries support: + +- `key_id` +- `public_key` or `public_key_base64` +- `status`: `active`, `next`, `retired`, `revoked`, or `disabled` +- `not_before` +- `not_after` + +Rotation process: + +1. Add the next public key to the local trusted keyring with status `next`. +2. Publish catalogs signed by both current and next keys. +3. Upgrade installations so the next key is locally trusted. +4. Promote the next key to `active`. +5. Retire the old key only after every supported installation trusts the new + key. +6. Mark a compromised key `revoked` and publish a higher sequence catalog + signed by an uncompromised key. + +Use replay state in production: + +```bash +GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-sequences.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true +``` + +Core records the accepted sequence per channel after a catalog entry is planned +from the admin interface. With strict sequence enforcement, a previously +accepted sequence is rejected; without strict enforcement, only older sequences +are rejected. Catalogs should always expire. + +The sequence state file is operational state, not a trust root. Keep it on +persistent storage and include it in normal backups: + +```json +{ + "channels": { + "stable": { + "last_sequence": 42, + "accepted_at": "2026-07-07T12:00:00Z", + "key_id": "release-key-1", + "source": "https://govoplan.example/catalogs/v1/channels/stable.json" + } + } +} +``` + +If the file is lost, restore it from backup. If no backup exists, reconstruct +each channel from the highest sequence already accepted in installer run +records, release records, or the currently deployed module package set. Do not +lower `last_sequence` to make an older catalog pass; publish a new higher +sequence catalog when the accepted point is uncertain. + +If the file is corrupted, copy it aside for incident review, validate the +current signed catalog with channel and freshness enforcement, then rewrite the +state with the known accepted sequence. Keep +`GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true` and approved-channel +checks enabled during recovery. Temporarily disabling +`GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE` allows revalidating the same +sequence, but older sequences remain rejected once the reconstructed +`last_sequence` is in place. + +Approved channels are deployment policy: + +```bash +GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable,lts +GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true +``` + +The admin UI can display other catalog metadata, but core rejects catalogs from +unapproved channels when validation is configured. + +Catalog entries can require license features: + +```json +"license_features": ["module.mail", "support.standard"] +``` + +Core checks those requirements against an offline license file before allowing +the entry into the install plan. + +```bash +GOVOPLAN_LICENSE_FILE=/srv/govoplan/license.json +GOVOPLAN_LICENSE_ENFORCEMENT=true +GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE=/srv/govoplan/trust/license-keyring.json +``` + +License files are JSON objects with: + +- `license_id` +- `subject` +- `features` +- `valid_from` +- `valid_until` +- `signature` + +Issue or renew a license from an operator/release shell that has the Ed25519 +private key: + +```bash +govoplan-module-installer \ + --issue-license /srv/govoplan/license.json \ + --license-id customer-2026-07 \ + --license-subject "Example Municipality" \ + --license-feature module.mail \ + --license-feature support.standard \ + --license-valid-until 2027-07-31T23:59:59Z \ + --license-signing-key-id license-issuer-1 \ + --license-signing-private-key /srv/govoplan/secrets/license-issuer-1.pem \ + --format json +``` + +Validate an imported license without exposing secrets: + +```bash +govoplan-module-installer \ + --validate-license /srv/govoplan/license.json \ + --license-trusted-key license-issuer-1="" \ + --require-trusted-license \ + --license-required-feature module.mail \ + --format json +``` + +The CLI and admin module catalog panel report the license id, subject, +validity window, signing key id, signed/trusted state, available features, and +missing entitlements for the configured package catalog. They do not expose +private signing material. + +License enforcement can run in observe-only mode by leaving +`GOVOPLAN_LICENSE_ENFORCEMENT` unset. In that mode, missing or invalid license +data is surfaced as a warning but does not block planning. + +Renewal is an ordinary re-issuance with a new `license_id`, extended +`valid_until`, and the full intended feature set. Import the renewed JSON to +`GOVOPLAN_LICENSE_FILE`, keep the previous file for audit, and validate it +before setting enforcement. + +Revocation is handled through the trusted license keyring. Mark a compromised +or invalid issuer key as `revoked` or `disabled`, publish or deploy the updated +keyring, then reissue affected licenses with an active key. Installations that +run with `GOVOPLAN_LICENSE_ENFORCEMENT=true` reject licenses signed only by a +revoked key after the local keyring is updated. + +Emergency fallback is deliberately explicit. Operators can temporarily unset +`GOVOPLAN_LICENSE_ENFORCEMENT` to keep package planning observable while a +license or keyring is recovered. Record the change in the operational incident +log, keep catalog signature and channel enforcement enabled, and restore +license enforcement after a trusted renewal validates successfully. + +Licensing is intentionally separate from open-source code licensing. The +catalog/license mechanism can govern support channels, official release +eligibility, hosted update access, professional support, or commercial +entitlements without changing the source license of the repositories. + +Production-grade distribution still needs remote registry/git artifact +resolution before package-manager apply, a hardened catalog publishing pipeline +in `govoplan-web`, and automated key rotation and emergency revocation drills. + +## Release Catalog Publishing + +GovOPlaN release catalogs are published by `govoplan-web` as static JSON and +verified by `govoplan-core` before installer plans are accepted. Private signing +keys must stay outside all git repositories. Public keyrings are published with +the website. + +Create the first catalog signing key on the release machine: + +```bash +cd /mnt/DATA/git/govoplan-core +KEY_DIR="$HOME/.config/govoplan/release-keys" +mkdir -p "$KEY_DIR" +./.venv/bin/python scripts/generate-catalog-keypair.py \ + --key-id release-key-1 \ + --private-key "$KEY_DIR/release-key-1.pem" \ + --public-key "$KEY_DIR/release-key-1.pub" \ + --keyring "$KEY_DIR/catalog-keyring.json" +``` + +Keep `release-key-1.pem` private. The generated keyring contains only public +material. + +Generate the signed catalog into `govoplan-web`: + +```bash +cd /mnt/DATA/git/govoplan-core +KEY_DIR="$HOME/.config/govoplan/release-keys" +scripts/publish-release-catalog.sh \ + --version \ + --sequence 202607071340 \ + --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ + --build-web +``` + +This writes: + +- `/mnt/DATA/git/govoplan-web/public/catalogs/v1/channels/stable.json` +- `/mnt/DATA/git/govoplan-web/public/catalogs/v1/keyring.json` + +The wrapper validates the catalog with core using the generated public keyring. + +For normal module/core releases, first audit and record migration baselines, +then tag and push the module/core repos. Finally publish the website catalog: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python scripts/release-migration-audit.py --strict +``` + +```bash +cd /mnt/DATA/git/govoplan-core +KEY_DIR="$HOME/.config/govoplan/release-keys" +scripts/publish-release-catalog.sh \ + --version \ + --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ + --build-web \ + --commit \ + --tag \ + --push +``` + +The website tag is `catalog-v`. The public URL is: + +```text +https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json +``` + +The public keyring URL is: + +```text +https://govoplan.add-ideas.de/catalogs/v1/keyring.json +``` + +`scripts/push-release-tag.sh` can publish the web catalog after module and core +tags have been pushed. It runs the migration release audit in automatic mode: +warning-only before the first recorded migration baseline, strict after a +baseline exists. Add `--strict-migration-audit` when you want to force strict +mode explicitly: + +```bash +cd /mnt/DATA/git/govoplan-core +KEY_DIR="$HOME/.config/govoplan/release-keys" +scripts/push-release-tag.sh \ + --bump subversion \ + --strict-migration-audit \ + --publish-web-catalog \ + --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ + --build-web-catalog +``` + +Use `--catalog-signing-key` more than once during a key rotation window. The +catalog will contain multiple signatures and the public keyring will include the +corresponding public keys. + +On a GovOPlaN installation that should consume the official stable catalog: + +```bash +GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/catalog-cache/stable.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true +GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable +GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/catalog-keyring.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-sequences.json +GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true +``` + +For production, copy the public keyring into deployment configuration and pin it +locally. Do not rely on a URL-fetched keyring as the only trust root. + +`stable.json` includes a top-level `core_release` section for operator/update +tooling. Core is intentionally not listed as a normal module entry because it +must not be added to saved enabled-module state. Core upgrades should remain an +operator-supervised package update with restart and health checks. + +Key rotation for published catalogs: + +1. Generate the next private key outside git. +2. Run `publish-release-catalog.sh` with both signing keys. +3. Publish the web catalog/keyring. +4. Roll the new public keyring into installations. +5. Stop signing with the old key after the supported fleet trusts the new key. +6. Mark compromised keys as revoked in the public keyring and publish a higher + sequence catalog signed by a trusted uncompromised key. + +## PostgreSQL Release Check + +Release candidates should pass a disposable PostgreSQL migration and startup +smoke check before tagging or publishing catalogs. Start the local testbed, +then run the permutation check from the core checkout: + +```bash +cd /mnt/DATA/git/govoplan-core/dev/postgres +cp .env.example .env +docker compose --env-file .env up -d + +cd /mnt/DATA/git/govoplan-core +set -a +. dev/postgres/.env +set +a +./.venv/bin/python scripts/postgres-integration-check.py \ + --database-url "$GOVOPLAN_POSTGRES_DATABASE_URL" \ + --reset-schema +``` + +The script checks migrations and `/health` startup for core-only, files-only, +mail-only, campaign-only, campaign+files, campaign+mail, and full-product +module sets. `--reset-schema` is destructive and must only be used against a +throwaway database. + +## Migration Baselines + +Development migrations may be small and numerous while a feature is moving. +Before a stable release, unreleased migrations may be rewritten or squashed into +a release-level baseline or release-to-release upgrade migration. After a +release tag has shipped, released migration revision IDs are immutable. + +The release policy is: + +- unreleased migrations may be folded before release; +- released migrations are never rewritten or deleted; +- each stable release records the public migration head revisions in + `docs/migration-release-baselines.json`; +- fresh installations should apply release-level baselines/upgrades, not + unreleased create-then-rename churn; +- release-to-release schema changes should be folded into one reviewed + migration per migration owner where practical. + +Audit the current graph during release preparation: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python scripts/release-migration-audit.py +``` + +Generate the reviewed/manual squash checklist: + +```bash +./.venv/bin/python scripts/release-migration-audit.py --squash-plan +``` + +After the release migrations have been reviewed and the graph is final, record +the release baseline: + +```bash +./.venv/bin/python scripts/release-migration-audit.py --record-release +``` + +Use strict mode to verify that the current heads are recorded: + +```bash +./.venv/bin/python scripts/release-migration-audit.py --strict +``` + +`scripts/push-release-tag.sh` runs the audit by default in automatic mode: +non-strict while no release baseline exists, strict after the first baseline is +recorded. Pass `--warn-migration-audit` for an explicit non-strict audit, +`--strict-migration-audit` to force strict mode, or `--skip-migration-audit` +only for emergency/manual release work. + +Before the first stable release, fold the current development chain into the +first public baseline and record that baseline in +`docs/migration-release-baselines.json`. The tracking issue is +`add-ideas/govoplan-core#223`. + +## Related Operator Documents + +- `DEPLOYMENT_OPERATOR_GUIDE.md`: runtime environment, explicit migrations, + backup/restore commands, module installer daemon/supervisor operation, and + rollback drills. +- `REMOTE_WEBUI_BUNDLES.md`: experimental browser-loaded module bundles for + controlled deployments; normal releases use package builds. + +## Release Checklist + +- Keep Python package versions, WebUI package versions, and git tags aligned. +- Tag core, access, admin, tenancy, policy, audit, files, mail, campaign, + calendar, and scaffold module repositories together. +- Update `requirements-release.txt` and `webui/package.release.json` when the + release tag changes. +- Generate the committed full-product release lockfile from + `package.release.json` with `scripts/generate-release-lock.sh`. +- Run `scripts/release-migration-audit.py --strict` after recording a release + baseline. +- Run the PostgreSQL release check against a disposable database. +- Publish the signed catalog through the release catalog publishing flow above. +- Add separate release manifest/lockfile pairs only for module compositions + that are shipped as their own products. +- Do not commit local sibling paths into release manifests.