From 40a5ff2aebbf33f138e3c4801fba789f9a7b8e8c Mon Sep 17 00:00:00 2001 From: Albrecht Degering Date: Thu, 6 Aug 2026 22:52:03 +0200 Subject: [PATCH] Sync Repo-docs-PACKAGE-REGISTRY-RELEASES from project files --- Repo-docs-PACKAGE-REGISTRY-RELEASES.-.md | 265 +++++++++++++++++++++++ 1 file changed, 265 insertions(+) create mode 100644 Repo-docs-PACKAGE-REGISTRY-RELEASES.-.md diff --git a/Repo-docs-PACKAGE-REGISTRY-RELEASES.-.md b/Repo-docs-PACKAGE-REGISTRY-RELEASES.-.md new file mode 100644 index 0000000..4144352 --- /dev/null +++ b/Repo-docs-PACKAGE-REGISTRY-RELEASES.-.md @@ -0,0 +1,265 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan/docs/PACKAGE_REGISTRY_RELEASES.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Package Registry Releases + +GovOPlaN publishes reusable module artifacts through Gitea's native PyPI and +npm registries. These packages improve developer installation, release +resolution, cacheability, and artifact inspection. They do not replace the +signed runtime distribution: the signed manifest and digest-pinned OCI images +remain the production deployment authority. + +## Publication boundary + +Every repository with a `pyproject.toml` contains +`.gitea/workflows/module-package-release.yml`. The meta repository owns the +canonical template and installs it with: + +```bash +python tools/repo/sync-module-package-workflows.py --write +python tools/repo/sync-module-package-workflows.py --check +``` + +The workflow runs for `v*` tags and may be dispatched manually for an existing +tag. The organization preflight verifies that every package repository protects +the `v*` namespace. Before building, the workflow itself verifies that: + +- the tagged commit is contained in `main`; +- the tag, Python project version, and optional WebUI package version agree; +- package names remain in the `govoplan-*` and `@govoplan/*-webui` namespaces. + +The workflow binds the repository explicitly from the Gitea Actions context. +Do not rely on GitHub-compatible environment variables being injected by the +runner image; Gitea runners may expose only the context values. Gitea 1.24 job +tokens cannot read repository tag-protection settings, so package jobs must not +receive a broad administrator token merely to repeat the organization preflight. +Run the following before the first publication and after repository or tag-rule +changes: + +```bash +python tools/gitea/gitea-configure-package-releases.py +``` + +Preview and dispatch the exact wheel/WebUI versions selected by the developer +meta-package with: + +```bash +python tools/gitea/gitea-dispatch-package-set.py \ + --env-file ~/.config/gitea/gitea.env +python tools/gitea/gitea-dispatch-package-set.py \ + --env-file ~/.config/gitea/gitea.env \ + --apply +``` + +The dispatcher reads exact versions from `packages/govoplan-meta/pyproject.toml`, +inspects the selected tag to determine whether a WebUI package is expected, +skips complete registry pairs and does not duplicate an active workflow. Use +`--repository govoplan-core` for a bounded dispatch or `--verify-existing` to +rebuild and hash-verify versions already present in both registries. + +For coordinated lockstep tags, `push-release-tag.sh` pushes module tags first, +Core next, and the meta tag last. This is a dependency guarantee for a +single-capacity Actions runner: the developer package cannot run before its +exact Core and module versions have entered the queue. + +The same release entry point first validates the migration graph, then records +the reviewed current Alembic heads under the target release version and reruns +the strict migration audit before it changes package versions, commits, or +tags. The default preflight intentionally does not require those heads to exist +in the previous release baseline. A failed candidate-baseline check therefore +cannot produce a protected package release. + +The source gate validates `pyproject.toml`, the module version declaration +(`MODULE_VERSION` or the top-level `ModuleManifest.version`), public package +`__version__`, and WebUI metadata before creating tags. Release-tag artifact +checks run only after the candidate tags and immutable WebUI lock have been +created locally. + +Release-lock regeneration resolves a fresh immutable lock from the reviewed +candidate manifests; it does not seed resolution from the previous release +lock. This prevents removed transitive packages and stale peer metadata from +blocking or contaminating the new release. Candidate resolution also uses an +isolated temporary npm cache, so a locally replaced tag cannot reuse metadata +from a failed, unpushed release attempt. + +Modules that retain the same WebUI package identity in both a root publish +manifest and `webui/package.json` use the WebUI manifest as the canonical peer +contract. The coordinated release synchronizes `peerDependencies` and +`peerDependenciesMeta` into the publish manifest before creating the module +tag, then synchronizes each lockfile root from the final package metadata. A +distinct root package remains independent. + +It builds one wheel and, where applicable, one npm tarball. The workflow records +the source tag, source commit, filename, size, and SHA-256 in +`package-artifacts.json` before publishing. Gitea rejects a second upload of the +same package version, so correction requires a new version rather than artifact +replacement. + +A retry after partial publication is safe. Before upload, the workflow reads the +native package registry file record and compares its SHA-256 with the artifact +rebuilt from the protected tag. An exact existing artifact is skipped; a +same-version artifact with another digest or an unexpected file set fails +closed. This permits a failed npm publication to resume without weakening +package immutability or accepting `--skip-existing` blindly. + +The npm tarball is always published through an explicit local `./dist/...` +path. Without that prefix, npm may interpret a relative tarball name as a Git +package shorthand before it ever contacts the configured registry. + +Published WebUI packages contain registry-compatible dependencies only. The +workflow converts an internal dependency pinned to a protected `vX.Y.Z` Git tag +into the exact `X.Y.Z` registry version and rejects unresolved `file:` or Git +dependencies. Repository development metadata may therefore keep local or Git +references without leaking them into the published package contract. +Historical `add-ideas` and current `GovOPlaN` organization URLs are accepted +for immutable tagged releases; both normalize to the same exact registry +dependency and no branch or unversioned Git reference is accepted. + +## One-time Gitea setup + +Protect `v*` tags in every package repository and the meta repository. Allow +only the `Owners` team to create or delete those tags. + +```bash +set -a +. ~/.config/gitea/gitea.env +set +a +python tools/gitea/gitea-configure-package-releases.py --apply +``` + +Create a dedicated personal access token with only `write:package` scope and +store these organization-level Actions secrets on `GovOPlaN`: + +- `GOVOPLAN_PACKAGE_USERNAME`: account owning the package token; +- `GOVOPLAN_PACKAGE_TOKEN`: dedicated package-write token. + +Do not use an administrator or general release token. Gitea 1.24 does not grant +package publication to the automatic Actions job token. Organization secrets +allow the same least-privilege credential to serve every module workflow. + +## Exact release consumption + +`tools/release/generate-release-package-set.py` supports two explicit package +profiles. `base` translates the reviewed roots in `requirements-release.txt`; +`full` reads the exact `govoplan[full]` dependency set from the developer +meta-package. Both profiles resolve every version tag to its commit and verify +the package metadata from that exact Git tree. The official module directory +and immutable runtime distribution use `full`, so every publicly released +module can be discovered without rebuilding the application image. + +`tools/release/resolve-package-artifacts.py` then downloads exactly those wheel +and WebUI versions from Gitea. It reads the identity embedded in every wheel and +npm tarball, rejects missing, duplicate, unexpected, or oversized artifacts, +and writes `package-artifacts.lock.json` with credential-free HTTPS download +URLs, SHA-256 values, and npm registry integrity values. The resolver verifies +that the bytes downloaded by `npm pack` match the registry's own integrity +record. Credentials are accepted only through environment variables and are +never written to the lock. Python resolution ignores ambient pip configuration +and extra indexes for GovOPlaN roots, preventing an internal package name from +being selected from an undeclared registry. + +The runtime distribution workflow uses the verified full-profile wheelhouse +directly and installs every selected module WebUI tarball only after matching +it to the lock. It publishes the package set, package lock, and hash-locked +requirements as release assets. +The WebUI installer receives the absolute runtime-build interpreter path so its +directory changes cannot escape the isolated release environment. +Gitea 1.24 dispatches this workflow from a branch, but that branch is only the +workflow implementation. The job fetches and peels the protected `v` +tag explicitly and materializes both `requirements-release.txt` and the +developer meta-package from that Git tree. It then binds the signed distribution +source and Gitea release assets to the same exact commit. A post-tag workflow +repair can therefore retry publication without changing the released package +composition or relabelling the later branch commit as released source. +The package-lock SHA-256 is part of the signed distribution manifest. Runtime +finalization also requires the lock's package versions and hashes to match the +wheel composition embedded in the images. OCI assembly remains network-free +after package and third-party dependency resolution. + +The source refs remain in the module catalog for source provenance and release +planning. Production installation consumes the signed runtime images rather +than invoking `pip`, `npm`, or Git on the target host. + +## Public module directory + +`tools/release/publish-release-catalog.sh` resolves the selected package set and +registry lock before it creates a catalog. Catalog entries are synthesized from +the exact tagged module manifests, never from a hand-maintained module list or +the current workspace. Each entry binds its Python wheel and optional WebUI +tarball to the registry URL, filename, size, SHA-256, package identity, source +tag, and source commit before the complete catalog is signed. + +The same publication transaction regenerates and prunes the browsable static +directory under `public/catalogs/v1/modules/`. It writes a global +`modules/index.json`, one `/index.json`, and one +`//manifest.json` for every entry in the signed channel. +These files are derived from that exact signed payload and keyring; stale JSON +from an older partial catalog is removed while unrelated static assets are left +untouched. The signed channel remains the trust anchor, while the module +directory provides stable discovery URLs for browsers and external tooling. + +Official GovOPlaN modules are open-source directory entries and do not require +license entitlements. The generic `license_features` contract remains available +for third-party package directories, support/configuration packages, or future +deployment-specific presets. A catalog entry is gated only when that entry +explicitly declares such features. + +Core carries the public stable catalog URL and its independently pinned trust +anchor. In the absence of an operator-configured catalog, Admin discovers the +official directory automatically. Selecting an entry creates a reviewed +install/update plan; the trusted installer downloads the exact signed artifacts +into a private digest cache, verifies size and hash, and installs only from that +cache. A saved plan is rejected if any package ref, artifact identity, catalog +channel, sequence, or signing-key identity differs from the currently validated +catalog. + +The Admin directory can be searched by module, package, repository, or tag and +filtered by available, installed, update, and blocked/withdrawn states. It +shows the source revision, artifact digest, release notes, and configuration +requirements. Missing dependency/interface providers and unsupported update +windows are surfaced before an operator adds the entry to a plan; installer +preflight remains authoritative. + +Package lifecycle and availability are intentionally separate: + +- install, update, and uninstall change the instance-wide package composition; +- enable and disable change the active instance runtime graph; +- tenant module entitlements define unavailable, available, and forced modules; +- group/user presentation is governed through Views and Policy; and +- enabling a capability module does not opt data into that capability. + +Single-process or single-host installations may execute a supervised package +plan locally. Shared-state and Kubernetes profiles reject node-local package +mutation: operators compose and roll out a new signed full-profile runtime image +instead. This prevents replicas from drifting while retaining the same Admin +catalog and preflight experience. + +## Developer meta-package + +`packages/govoplan-meta` builds the optional `govoplan` package. Its default +dependencies mirror the reviewed runtime roots; `govoplan[full]` adds all +currently packageable workspace modules. Regenerate it after changing release +requirements or package versions: + +```bash +python tools/release/generate-developer-meta-package.py +python tools/release/generate-developer-meta-package.py --check +``` + +`push-release-tag.sh` performs this synchronization before release commits and +tags. The meta-package is for editable/developer setup and composition tests. It +does not enable modules, apply migrations, provision services, or establish +backup and recovery evidence. + +If the tag-triggered developer meta-package job fails before publication, rerun +`publish-developer-meta-package.yml` with the existing protected version. The +manual path validates that tag against `main`, checks out its exact commit, and +publishes only when the registry does not already contain the same wheel hash. + +Generic Packages are intentionally not used. Add that transport only when a +consumer needs an artifact format unsupported by PyPI, npm, Gitea Releases, or +the OCI registry.