10 Commits
Author SHA1 Message Date
zemion a24c94435e Release v0.1.15
Dependency Audit / dependency-audit (push) Failing after 1m49s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m50s
Developer Meta-package Release / publish-package (push) Failing after 4s
2026-08-04 15:20:50 +02:00
zemion 774793976c Support legacy module version declarations
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Failing after 1m42s
Security Audit / security-audit (push) Successful in 10m43s
2026-08-04 15:08:46 +02:00
zemion 492449a4e2 Align all release version declarations
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 10m42s
2026-08-04 15:06:36 +02:00
zemion 8e890b37ed Validate candidate migration baseline during release
Dependency Audit / dependency-audit (push) Failing after 1m42s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m38s
2026-08-04 15:01:50 +02:00
zemion 077735bc24 Bind migration heads to coordinated releases
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Failing after 1m44s
Security Audit / security-audit (push) Successful in 10m36s
2026-08-04 14:55:58 +02:00
zemion d9522d3cc4 Publish release tags in dependency order
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m53s
2026-08-04 14:54:54 +02:00
zemion bad0ea37a7 Automate exact package-set publication
Dependency Audit / dependency-audit (push) Failing after 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m46s
2026-08-04 14:32:06 +02:00
zemion 9ffd46fe22 Make package publication retries hash-safe
Dependency Audit / dependency-audit (push) Failing after 1m48s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m35s
2026-08-04 14:18:49 +02:00
zemion be51a9c347 Document production target evidence handoff
Dependency Audit / dependency-audit (push) Failing after 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m40s
2026-08-04 14:00:45 +02:00
zemion e36a6573bf Harden package release workflows for Gitea 2026-08-04 14:00:32 +02:00
18 changed files with 1376 additions and 137 deletions
@@ -8,6 +8,8 @@ on:
jobs:
publish-package:
runs-on: ubuntu-latest
env:
GITEA_REPOSITORY: ${{ gitea.repository }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
@@ -16,17 +18,12 @@ jobs:
with:
python-version: "3.12"
- name: Validate protected release tag and package version
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
python - <<'PY'
import fnmatch
import json
import os
from pathlib import Path
import subprocess
import tomllib
import urllib.request
tag = os.environ["GITEA_REF_NAME"]
project = tomllib.loads(Path("packages/govoplan-meta/pyproject.toml").read_text(encoding="utf-8"))["project"]
@@ -34,14 +31,6 @@ jobs:
raise SystemExit("meta-package version does not match the release tag")
if subprocess.run(["git", "merge-base", "--is-ancestor", "HEAD", "origin/main"]).returncode:
raise SystemExit("release tag is not contained in main")
request = urllib.request.Request(
f"{os.environ['GITEA_API_URL']}/repos/{os.environ['GITEA_REPOSITORY']}/tag_protections",
headers={"Authorization": f"token {os.environ['GITEA_TOKEN']}"},
)
with urllib.request.urlopen(request, timeout=30) as response:
protections = json.load(response)
if not any(fnmatch.fnmatchcase(tag, item.get("name_pattern", "")) for item in protections):
raise SystemExit("release tag is not protected")
PY
- name: Build and publish developer package
env:
+77 -2
View File
@@ -18,24 +18,99 @@ python tools/repo/sync-module-package-workflows.py --check
```
The workflow runs for `v*` tags and may be dispatched manually for an existing
tag. Before building, it verifies that:
tag. The organization preflight verifies that every package repository protects
the `v*` namespace. Before building, the workflow itself verifies that:
- the selected tag is covered by repository tag protection;
- the tagged commit is contained in `main`;
- the tag, Python project version, and optional WebUI package version agree;
- package names remain in the `govoplan-*` and `@govoplan/*-webui` namespaces.
The workflow binds the repository explicitly from the Gitea Actions context.
Do not rely on GitHub-compatible environment variables being injected by the
runner image; Gitea runners may expose only the context values. Gitea 1.24 job
tokens cannot read repository tag-protection settings, so package jobs must not
receive a broad administrator token merely to repeat the organization preflight.
Run the following before the first publication and after repository or tag-rule
changes:
```bash
python tools/gitea/gitea-configure-package-releases.py
```
Preview and dispatch the exact wheel/WebUI versions selected by the developer
meta-package with:
```bash
python tools/gitea/gitea-dispatch-package-set.py \
--env-file ~/.config/gitea/gitea.env
python tools/gitea/gitea-dispatch-package-set.py \
--env-file ~/.config/gitea/gitea.env \
--apply
```
The dispatcher reads exact versions from `packages/govoplan-meta/pyproject.toml`,
inspects the selected tag to determine whether a WebUI package is expected,
skips complete registry pairs and does not duplicate an active workflow. Use
`--repository govoplan-core` for a bounded dispatch or `--verify-existing` to
rebuild and hash-verify versions already present in both registries.
For coordinated lockstep tags, `push-release-tag.sh` pushes module tags first,
Core next, and the meta tag last. This is a dependency guarantee for a
single-capacity Actions runner: the developer package cannot run before its
exact Core and module versions have entered the queue.
The same release entry point first validates the migration graph, then records
the reviewed current Alembic heads under the target release version and reruns
the strict migration audit before it changes package versions, commits, or
tags. The default preflight intentionally does not require those heads to exist
in the previous release baseline. A failed candidate-baseline check therefore
cannot produce a protected package release.
The source gate validates `pyproject.toml`, the module version declaration
(`MODULE_VERSION` or the top-level `ModuleManifest.version`), public package
`__version__`, and WebUI metadata before creating tags. Release-tag artifact
checks run only after the candidate tags and immutable WebUI lock have been
created locally.
Release-lock regeneration resolves a fresh immutable lock from the reviewed
candidate manifests; it does not seed resolution from the previous release
lock. This prevents removed transitive packages and stale peer metadata from
blocking or contaminating the new release. Candidate resolution also uses an
isolated temporary npm cache, so a locally replaced tag cannot reuse metadata
from a failed, unpushed release attempt.
Modules that retain the same WebUI package identity in both a root publish
manifest and `webui/package.json` use the WebUI manifest as the canonical peer
contract. The coordinated release synchronizes `peerDependencies` and
`peerDependenciesMeta` into the publish manifest before creating the module
tag, then synchronizes each lockfile root from the final package metadata. A
distinct root package remains independent.
It builds one wheel and, where applicable, one npm tarball. The workflow records
the source tag, source commit, filename, size, and SHA-256 in
`package-artifacts.json` before publishing. Gitea rejects a second upload of the
same package version, so correction requires a new version rather than artifact
replacement.
A retry after partial publication is safe. Before upload, the workflow reads the
native package registry file record and compares its SHA-256 with the artifact
rebuilt from the protected tag. An exact existing artifact is skipped; a
same-version artifact with another digest or an unexpected file set fails
closed. This permits a failed npm publication to resume without weakening
package immutability or accepting `--skip-existing` blindly.
The npm tarball is always published through an explicit local `./dist/...`
path. Without that prefix, npm may interpret a relative tarball name as a Git
package shorthand before it ever contacts the configured registry.
Published WebUI packages contain registry-compatible dependencies only. The
workflow converts an internal dependency pinned to a protected `vX.Y.Z` Git tag
into the exact `X.Y.Z` registry version and rejects unresolved `file:` or Git
dependencies. Repository development metadata may therefore keep local or Git
references without leaking them into the published package contract.
Historical `add-ideas` and current `GovOPlaN` organization URLs are accepted
for immutable tagged releases; both normalize to the same exact registry
dependency and no branch or unversioned Git reference is accepted.
## One-time Gitea setup
+182
View File
@@ -0,0 +1,182 @@
# Production Target And Independent Evidence Handoff
This runbook identifies the external inputs needed to finish
[GovOPlaN #27](https://git.add-ideas.de/GovOPlaN/govoplan/issues/27) and
[GovOPlaN #37](https://git.add-ideas.de/GovOPlaN/govoplan/issues/37). The
repository can render, inspect and sign evidence for a target, but it cannot
manufacture an independent failure domain or an independent approval authority.
## GovOPlaN #27: real two-node target
The bounded acceptance target is two independently schedulable worker nodes.
The API and WebUI must each have ready replicas on both nodes, all Deployments
must be available, the active module composition and software versions must be
consistent, every configured queue must have a worker, and the database
connection budget must pass. The validation then deletes one ready API pod and
requires replacement without an observed readiness outage.
Two virtual machines on different physical hosts or availability zones meet the
failure-domain intent. Two containers, VMs or Kubernetes nodes on one physical
host are useful development targets but do not close #27. A two-worker cluster
also does not prove control-plane high availability. For a self-managed
production cluster, use three control-plane nodes plus at least two workers; a
managed control plane plus two workers is the shorter path.
### What the target owner must provide
Provide these through a secure handoff, not an issue, chat message or Git:
1. A kubeconfig path with access to the target, for example
`~/.config/govoplan/targets/<target>.kubeconfig`, mode `0600`.
2. A stable installation ID, public HTTPS hostname, namespace, ingress class and
TLS-secret or certificate-manager arrangement.
3. Two independently schedulable workers and permission to place API and WebUI
replicas on both.
4. External, logically shared PostgreSQL, Redis and S3 endpoints with trusted
CA material and network reachability from every worker. Do not co-locate the
only copies of these services on the two workers used for the failure drill.
5. The six runtime secret values required by the generated manifest:
`MASTER_KEY_B64`, `DATABASE_URL`, `GOVOPLAN_DATABASE_URL_PGTOOLS`,
`REDIS_URL`, `FILE_STORAGE_S3_ACCESS_KEY_ID` and
`FILE_STORAGE_S3_SECRET_ACCESS_KEY`.
6. A short-lived GovOPlaN API key limited to `ops:operations:read`, supplied in
`GOVOPLAN_OPS_API_KEY` only for evidence collection.
7. An approved drill window and permission to delete one API pod.
If no Kubernetes target exists, provide hostnames/IP addresses for the machines,
an SSH user and key path, the internal/external DNS plan, and the permitted
firewall ports. Those inputs are sufficient to provision a k3s target. They are
not sufficient to claim control-plane HA unless three control-plane failure
domains are present.
### Separate deployment and evidence authorities
The deployment identity may create and update the namespace, Secret,
ConfigMap, Deployments, Services, Jobs, PodDisruptionBudgets and Ingress. The
evidence collector only needs:
- cluster scope: `get` and `list` for `nodes`;
- target namespace: `get` and `list` for `pods` and `deployments`;
- target namespace during the approved drill: `delete` for `pods`.
Use separate kubeconfig contexts or service accounts when the same person does
not hold both roles.
### Render, apply and verify
Use the signed, digest-pinned installation bundle selected for the target:
```bash
export KUBECONFIG="$HOME/.config/govoplan/targets/<target>.kubeconfig"
python tools/deployment/govoplan-deploy.py render-kubernetes \
--directory /srv/govoplan/<installation-id> \
--namespace govoplan \
--secret-name govoplan-runtime \
--tls-secret-name govoplan-tls \
--ingress-class-name nginx \
--output /srv/govoplan/<installation-id>/kubernetes.json
kubectl apply -f /srv/govoplan/<installation-id>/kubernetes.json
kubectl -n govoplan wait --for=condition=available deployment --all --timeout=10m
export GOVOPLAN_OPS_API_KEY="$(cat /run/secrets/govoplan-ops-evidence-key)"
python tools/deployment/govoplan-deploy.py verify-kubernetes \
--directory /srv/govoplan/<installation-id> \
--namespace govoplan \
--exercise-api-pod-loss \
--output /srv/govoplan/<installation-id>/evidence/kubernetes-multi-host.json
unset GOVOPLAN_OPS_API_KEY
```
The verifier emits sanitized JSON and exits nonzero if the topology, runtime,
queue, connection-budget or pod-loss checks fail. Preserve the private cluster
logs and manifest alongside the sanitized result in the controlled evidence
store.
## GovOPlaN #37: controlled signed target evidence
Yes, collection, review and signing can run in containers. A container provides
repeatability and process isolation; it does not create independent authority.
The production approver must control a different private key from the target
operator/assessor and must review the evidence before signing the
`production_approval` scope.
Use at least these three key boundaries:
1. **Installer authority:** signs installed-release-origin receipts only.
2. **Target assessment authority:** signs the permitted target, accessibility,
privacy, security, operations and recovery scopes.
3. **Production approval authority:** independently signs only
`production_approval` after reviewing the other evidence.
Do not reuse release-catalog keys for any of these roles. Keep private Ed25519
keys outside Git, Gitea, GovOPlaN application storage and chat. Publish only the
public keyrings. The proof issuer already rejects key reuse across release,
installer and proof trust domains.
### Generate independently held keys
Each authority runs this command in its own `0700` directory. The generator
refuses existing output paths and writes both files as `0600`:
```bash
install -d -m 0700 "$HOME/.config/govoplan/authority-keys"
python tools/assessments/generate-authority-keypair.py \
--purpose proof \
--key-id authority:target-2026 \
--scope target_environment \
--scope accessibility \
--scope privacy \
--scope security \
--scope operations \
--scope recovery \
--private-key "$HOME/.config/govoplan/authority-keys/target-2026.pem" \
--keyring "$HOME/.config/govoplan/authority-keys/target-2026-public.json"
```
The independent production approver generates another key with only
`--scope production_approval`. An installer authority uses `--purpose installer`
and no `--scope`. Merge public key entries into the separately controlled
keyrings only after the responsible authorities verify fingerprints out of
band.
### Container boundary
Use two one-shot jobs or containers:
- **Collector/assessor:** network access, read-only source and trust mounts,
read/write private evidence output, and the narrowly scoped kubeconfig. It
must not receive the production-approval private key.
- **Production approver:** `--network none`, read-only assessment/evidence/trust
mounts, a read-only secret mount containing only the approval key, and a
separate output mount. It must not receive deployment credentials.
Build or select the assessment image by digest and record that digest in the
evidence log. A representative runtime shape is:
```bash
docker run --rm --network none --read-only --tmpfs /tmp \
--user "$(id -u):$(id -g)" \
--mount type=bind,src="$PWD/evidence",dst=/evidence,readonly \
--mount type=bind,src="$PWD/trust",dst=/trust,readonly \
--mount type=bind,src="$HOME/.config/govoplan/authority-keys",dst=/run/keys,readonly \
--mount type=bind,src="$PWD/approved",dst=/output \
<assessment-image>@sha256:<digest> \
<assessment command>
```
The current evidence commands and required scopes are documented in
[`TARGET_MATURITY_EVIDENCE_RUNBOOK.md`](TARGET_MATURITY_EVIDENCE_RUNBOOK.md).
The final proof must cover `target_environment`, `accessibility`, `privacy`,
`security`, `operations`, `recovery` and independent `production_approval`, and
must bind to the verified installed composition and installer receipt.
## Completion boundary
#27 can close after the real target produces a passing pod-loss result. #37 can
close after an independently approved, schema-valid proof is generated for that
same installed composition and the public authority keyrings, proof and private
evidence custody references are recorded. Neither issue should close from a
single-host simulation or a self-approved signature.
@@ -1,5 +1,9 @@
# Scaling And Multi-Host Deployment
For the exact external handoff, least-privilege collector permissions and live
two-node acceptance procedure, see
[`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md).
## Implemented Contract
GovOPlaN now supports a stateless application tier backed by logically shared
+4
View File
@@ -1,5 +1,9 @@
# Target Maturity Evidence Runbook
For authority-key generation, container isolation and the concrete inputs that
must be supplied by the target owner and independent production approver, see
[`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md).
This runbook turns retained target-environment results into a sanitized,
signed GovOPlaN capability-fit proof. It does not make a deployment suitable,
certified, supported, or production-approved by itself. The proof records what
+67 -67
View File
@@ -4,82 +4,82 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan"
version = "0.1.14"
version = "0.1.15"
description = "Developer convenience package for a versioned GovOPlaN composition"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" }
dependencies = [
"govoplan-core[server]==0.1.14",
"govoplan-tenancy==0.1.8",
"govoplan-organizations==0.1.8",
"govoplan-identity==0.1.8",
"govoplan-idm==0.1.8",
"govoplan-access==0.1.8",
"govoplan-admin==0.1.8",
"govoplan-policy==0.1.8",
"govoplan-audit==0.1.8",
"govoplan-dashboard==0.1.8",
"govoplan-files==0.1.8",
"govoplan-mail==0.1.10",
"govoplan-campaign==0.1.11",
"govoplan-calendar==0.1.8",
"govoplan-docs==0.1.8",
"govoplan-ops==0.1.8",
"govoplan-core[server]==0.1.15",
"govoplan-tenancy==0.1.15",
"govoplan-organizations==0.1.15",
"govoplan-identity==0.1.15",
"govoplan-idm==0.1.15",
"govoplan-access==0.1.15",
"govoplan-admin==0.1.15",
"govoplan-policy==0.1.15",
"govoplan-audit==0.1.15",
"govoplan-dashboard==0.1.15",
"govoplan-files==0.1.15",
"govoplan-mail==0.1.15",
"govoplan-campaign==0.1.15",
"govoplan-calendar==0.1.15",
"govoplan-docs==0.1.15",
"govoplan-ops==0.1.15",
]
[project.optional-dependencies]
full = [
"govoplan-addresses==0.1.9",
"govoplan-approvals==0.1.14",
"govoplan-assets==0.1.8",
"govoplan-booking==0.1.8",
"govoplan-cases==0.1.8",
"govoplan-certificates==0.1.8",
"govoplan-committee==0.1.8",
"govoplan-connectors==0.1.14",
"govoplan-consultation==0.1.8",
"govoplan-contracts==0.1.8",
"govoplan-dataflow==0.1.14",
"govoplan-datasources==0.1.14",
"govoplan-decisions==0.1.14",
"govoplan-dist-lists==0.1.14",
"govoplan-encryption==0.1.14",
"govoplan-evaluation==0.1.8",
"govoplan-facilities==0.1.8",
"govoplan-forms==0.1.14",
"govoplan-forms-runtime==0.1.14",
"govoplan-grants==0.1.8",
"govoplan-helpdesk==0.1.8",
"govoplan-identity-trust==0.1.14",
"govoplan-inspections==0.1.8",
"govoplan-learning==0.1.8",
"govoplan-mandates==0.1.14",
"govoplan-notifications==0.1.8",
"govoplan-parties==0.1.14",
"govoplan-permits==0.1.8",
"govoplan-poll==0.1.11",
"govoplan-portal==0.1.8",
"govoplan-postbox==0.1.2",
"govoplan-procurement==0.1.8",
"govoplan-projects==0.1.14",
"govoplan-records==0.1.8",
"govoplan-reporting==0.1.14",
"govoplan-resources==0.1.8",
"govoplan-rest==0.1.7",
"govoplan-risk-compliance==0.1.8",
"govoplan-scheduling==0.1.11",
"govoplan-search==0.1.14",
"govoplan-services==0.1.14",
"govoplan-soap==0.1.7",
"govoplan-templates==0.1.14",
"govoplan-tickets==0.1.8",
"govoplan-transparency==0.1.8",
"govoplan-views==0.1.0",
"govoplan-voting==0.1.14",
"govoplan-wiki==0.1.14",
"govoplan-workflow==0.1.14",
"govoplan-workflow-engine==0.1.14",
"govoplan-addresses==0.1.15",
"govoplan-approvals==0.1.15",
"govoplan-assets==0.1.15",
"govoplan-booking==0.1.15",
"govoplan-cases==0.1.15",
"govoplan-certificates==0.1.15",
"govoplan-committee==0.1.15",
"govoplan-connectors==0.1.15",
"govoplan-consultation==0.1.15",
"govoplan-contracts==0.1.15",
"govoplan-dataflow==0.1.15",
"govoplan-datasources==0.1.15",
"govoplan-decisions==0.1.15",
"govoplan-dist-lists==0.1.15",
"govoplan-encryption==0.1.15",
"govoplan-evaluation==0.1.15",
"govoplan-facilities==0.1.15",
"govoplan-forms==0.1.15",
"govoplan-forms-runtime==0.1.15",
"govoplan-grants==0.1.15",
"govoplan-helpdesk==0.1.15",
"govoplan-identity-trust==0.1.15",
"govoplan-inspections==0.1.15",
"govoplan-learning==0.1.15",
"govoplan-mandates==0.1.15",
"govoplan-notifications==0.1.15",
"govoplan-parties==0.1.15",
"govoplan-permits==0.1.15",
"govoplan-poll==0.1.15",
"govoplan-portal==0.1.15",
"govoplan-postbox==0.1.15",
"govoplan-procurement==0.1.15",
"govoplan-projects==0.1.15",
"govoplan-records==0.1.15",
"govoplan-reporting==0.1.15",
"govoplan-resources==0.1.15",
"govoplan-rest==0.1.15",
"govoplan-risk-compliance==0.1.15",
"govoplan-scheduling==0.1.15",
"govoplan-search==0.1.15",
"govoplan-services==0.1.15",
"govoplan-soap==0.1.15",
"govoplan-templates==0.1.15",
"govoplan-tickets==0.1.15",
"govoplan-transparency==0.1.15",
"govoplan-views==0.1.15",
"govoplan-voting==0.1.15",
"govoplan-wiki==0.1.15",
"govoplan-workflow==0.1.15",
"govoplan-workflow-engine==0.1.15",
]
[project.urls]
+15 -15
View File
@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published.
../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.8
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.10
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.11
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.8
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.15
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.15
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.15
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.15
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.15
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.15
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.15
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.15
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.15
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.15
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.15
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.15
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.15
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.15
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.15
+120
View File
@@ -0,0 +1,120 @@
from __future__ import annotations
import base64
import json
from pathlib import Path
import stat
import subprocess
import sys
import tempfile
import unittest
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from jsonschema import Draft202012Validator, FormatChecker
META_ROOT = Path(__file__).resolve().parents[1]
GENERATOR = META_ROOT / "tools" / "assessments" / "generate-authority-keypair.py"
class AssessmentAuthorityKeypairTests(unittest.TestCase):
def test_generates_schema_valid_scoped_proof_authority(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
output_dir = Path(temp_dir)
output_dir.chmod(0o700)
private_path = output_dir / "target.pem"
keyring_path = output_dir / "target.json"
result = subprocess.run(
(
sys.executable,
str(GENERATOR),
"--purpose",
"proof",
"--key-id",
"authority:target-2026",
"--scope",
"target_environment",
"--scope",
"operations",
"--private-key",
str(private_path),
"--keyring",
str(keyring_path),
),
check=False,
capture_output=True,
text=True,
)
self.assertEqual(0, result.returncode, result.stderr)
self.assertEqual(0o600, stat.S_IMODE(private_path.stat().st_mode))
self.assertEqual(0o600, stat.S_IMODE(keyring_path.stat().st_mode))
keyring = json.loads(keyring_path.read_text(encoding="utf-8"))
schema = json.loads(
(
META_ROOT
/ "docs"
/ "capability-fit-proof-authority-keyring.schema.json"
).read_text(encoding="utf-8")
)
errors = tuple(
Draft202012Validator(
schema, format_checker=FormatChecker()
).iter_errors(keyring)
)
self.assertEqual((), errors)
self.assertEqual(
["target_environment", "operations"],
keyring["keys"][0]["allowed_scopes"],
)
private_key = serialization.load_pem_private_key(
private_path.read_bytes(), password=None
)
self.assertIsInstance(private_key, Ed25519PrivateKey)
public_key = base64.b64encode(
private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
).decode("ascii")
self.assertEqual(public_key, keyring["keys"][0]["public_key"])
def test_installer_authority_uses_fixed_scope_and_refuses_overwrite(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
output_dir = Path(temp_dir)
output_dir.chmod(0o700)
private_path = output_dir / "installer.pem"
keyring_path = output_dir / "installer.json"
command = (
sys.executable,
str(GENERATOR),
"--purpose",
"installer",
"--key-id",
"authority:installer-2026",
"--private-key",
str(private_path),
"--keyring",
str(keyring_path),
)
first = subprocess.run(
command, check=False, capture_output=True, text=True
)
second = subprocess.run(
command, check=False, capture_output=True, text=True
)
self.assertEqual(0, first.returncode, first.stderr)
self.assertNotEqual(0, second.returncode)
keyring = json.loads(keyring_path.read_text(encoding="utf-8"))
self.assertEqual(
["installed_release_origin"],
keyring["keys"][0]["allowed_scopes"],
)
if __name__ == "__main__":
unittest.main()
+16 -2
View File
@@ -17,15 +17,22 @@ class ModulePackageWorkflowTests(unittest.TestCase):
META_ROOT / "tools/repo/templates/module-package-release.yml"
).read_text(encoding="utf-8")
self.assertIn("tag_protections", workflow)
self.assertIn("GITEA_REPOSITORY: ${{ gitea.repository }}", workflow)
self.assertNotIn("tag_protections", workflow)
self.assertNotIn("secrets.GITEA_TOKEN", workflow)
self.assertIn("git merge-base --is-ancestor", workflow)
self.assertIn("does not match", workflow)
self.assertIn("package-artifacts.json", workflow)
self.assertIn("api/packages/GovOPlaN/pypi", workflow)
self.assertIn("api/packages/GovOPlaN/npm", workflow)
self.assertIn('npm publish "./${webui_packages[0]}"', workflow)
self.assertIn("Check immutable registry state", workflow)
self.assertIn('files[0].get("sha256") != expected_sha256', workflow)
self.assertIn('if [[ "$PUBLISH_PYPI" == 1 ]]', workflow)
self.assertIn('[[ "$PUBLISH_NPM" == 1 ]]', workflow)
self.assertIn("GOVOPLAN_PACKAGE_TOKEN", workflow)
self.assertIn("must resolve to an exact registry version", workflow)
self.assertIn("git\\\\.add-ideas\\\\.de/GovOPlaN", workflow)
self.assertIn("git\\\\.add-ideas\\\\.de/(?:GovOPlaN|add-ideas)", workflow)
self.assertIn("release package identity does not match", workflow)
self.assertNotIn("Generic", workflow)
@@ -52,6 +59,10 @@ class ModulePackageWorkflowTests(unittest.TestCase):
"@govoplan/access-webui": (
"git+ssh://git@git.add-ideas.de/GovOPlaN/"
"govoplan-access.git#v0.1.11"
),
"@govoplan/admin-webui": (
"git+ssh://git@git.add-ideas.de/add-ideas/"
"govoplan-admin.git#v0.1.8"
)
},
}
@@ -73,6 +84,9 @@ class ModulePackageWorkflowTests(unittest.TestCase):
self.assertEqual(
"0.1.11", package["dependencies"]["@govoplan/access-webui"]
)
self.assertEqual(
"0.1.8", package["dependencies"]["@govoplan/admin-webui"]
)
def test_sync_script_only_targets_packageable_govoplan_repositories(self) -> None:
namespace: dict[str, object] = {
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
TOOLS_ROOT = META_ROOT / "tools" / "gitea"
if str(TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(TOOLS_ROOT))
SCRIPT = TOOLS_ROOT / "gitea-dispatch-package-set.py"
SPEC = importlib.util.spec_from_file_location("gitea_dispatch_package_set", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
MODULE = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = MODULE
SPEC.loader.exec_module(MODULE)
class PackageSetDispatchTests(unittest.TestCase):
def test_meta_package_resolves_to_exact_tagged_repository_targets(self) -> None:
targets = MODULE.package_targets()
self.assertEqual(66, len(targets))
self.assertEqual(66, len({target.distribution for target in targets}))
by_name = {target.distribution: target for target in targets}
self.assertEqual("v0.1.14", by_name["govoplan-core"].tag)
self.assertEqual("v0.1.8", by_name["govoplan-access"].tag)
self.assertTrue(by_name["govoplan-core"].tag_exists)
self.assertTrue(by_name["govoplan-access"].has_webui)
self.assertEqual(
"@govoplan/access-webui",
by_name["govoplan-access"].webui_package,
)
if __name__ == "__main__":
unittest.main()
+55
View File
@@ -29,17 +29,72 @@ class ReleaseEntrypointGateTests(unittest.TestCase):
workflow = script[confirm:]
source_gate = workflow.index("run_version_alignment_gate source")
baseline = workflow.index("record_migration_release_baseline")
first_commit = workflow.index('run git -C "$repo" commit')
lock_generation = workflow.index("generate_release_lock")
full_gate = workflow.index("run_version_alignment_gate", source_gate + 1)
first_push = workflow.index('run git -C "$repo" push')
self.assertLess(baseline, source_gate)
self.assertLess(source_gate, first_commit)
self.assertLess(first_commit, lock_generation)
self.assertLess(lock_generation, full_gate)
self.assertLess(full_gate, first_push)
self.assertLess(manifest_gate, confirm)
def test_lockstep_release_pushes_meta_package_after_core(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
module_push = script.index('for repo in "${MODULE_REPOS[@]}"; do\n run git -C "$repo" push')
core_push = script.index('run git -C "$ROOT" push', module_push)
support_push = script.index('for repo in "${SUPPORT_REPOS[@]}"; do\n run git -C "$repo" push', core_push)
self.assertLess(module_push, core_push)
self.assertLess(core_push, support_push)
def test_default_migration_preflight_accepts_new_release_heads(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
audit_function = script[
script.index("run_migration_release_audit()") :
script.index("record_migration_release_baseline()")
]
self.assertNotIn("--strict-if-baseline", audit_function)
self.assertIn('command+=("--strict")', audit_function)
def test_source_gate_does_not_require_tags_before_they_are_created(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
gate = script[
script.index("run_version_alignment_gate()") :
script.index("run_manifest_shape_gate()")
]
self.assertIn('command+=(--source-metadata-only)', gate)
self.assertIn('else\n command+=(--release-composition)', gate)
def test_version_updater_targets_canonical_runtime_declarations(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
self.assertIn("^manifest\\s*=\\s*ModuleManifest", script)
self.assertIn("could not update module version declaration", script)
self.assertIn("update_package_init_versions", script)
self.assertIn("synchronize-webui-package-metadata.py", script)
self.assertIn('"peerDependenciesMeta",', script)
self.assertLess(
script.index('synchronize-webui-package-metadata.py" --repo "$repo"'),
script.index('synchronize_lockfile_root "$package_path"', script.index('synchronize-webui-package-metadata.py" --repo "$repo"')),
)
self.assertNotIn("could not update ModuleManifest.version", script)
def test_release_lock_refreshes_candidate_govoplan_metadata(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-lock.sh").read_text()
self.assertEqual(2, script.count('"npm_config_cache=$TMP_DIR/npm-cache"'))
self.assertNotIn(
'cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"',
script,
)
self.assertIn('cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"', script)
def test_source_catalog_generator_enforces_explicit_repo_versions(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text()
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "release" / "synchronize-webui-package-metadata.py"
class SynchronizeWebuiPackageMetadataTests(unittest.TestCase):
def test_copies_peer_contract_without_changing_publish_paths(self) -> None:
with tempfile.TemporaryDirectory() as directory:
repo = Path(directory)
(repo / "webui").mkdir()
(repo / "package.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"exports": {".": "./webui/src/index.ts"},
"peerDependencies": {"vite": "^6"},
}
)
)
(repo / "webui" / "package.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"peerDependencies": {"vite": "^7"},
"peerDependenciesMeta": {"vite": {"optional": True}},
}
)
)
subprocess.run(
[sys.executable, str(SCRIPT), "--repo", str(repo)],
check=True,
capture_output=True,
text=True,
)
package = json.loads((repo / "package.json").read_text())
self.assertEqual({"vite": "^7"}, package["peerDependencies"])
self.assertEqual({"vite": {"optional": True}}, package["peerDependenciesMeta"])
self.assertEqual({".": "./webui/src/index.ts"}, package["exports"])
def test_leaves_distinct_root_and_webui_packages_separate(self) -> None:
with tempfile.TemporaryDirectory() as directory:
repo = Path(directory)
(repo / "webui").mkdir()
(repo / "package.json").write_text(json.dumps({"name": "@govoplan/one"}))
(repo / "webui" / "package.json").write_text(json.dumps({"name": "@govoplan/two"}))
subprocess.run(
[sys.executable, str(SCRIPT), "--repo", str(repo)],
check=True,
capture_output=True,
text=True,
)
root = json.loads((repo / "package.json").read_text())
self.assertEqual("@govoplan/one", root["name"])
self.assertNotIn("peerDependencies", root)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,194 @@
#!/usr/bin/env python3
"""Generate an independently held Ed25519 assessment-authority keypair."""
from __future__ import annotations
import argparse
import base64
from datetime import UTC, datetime, timedelta
import json
import os
from pathlib import Path
import re
import stat
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
KEY_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,159}$")
PROOF_SCOPES = (
"target_environment",
"external_providers",
"accessibility",
"privacy",
"security",
"operations",
"recovery",
"production_approval",
)
PURPOSES = {
"proof": (
"govoplan.capability-fit-proof-authorities",
"./capability-fit-proof-authority-keyring.schema.json",
),
"installer": (
"govoplan.installer-receipt-authorities",
"./installer-receipt-authority-keyring.schema.json",
),
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--purpose", choices=tuple(PURPOSES), required=True)
parser.add_argument("--key-id", required=True)
parser.add_argument(
"--scope",
action="append",
choices=PROOF_SCOPES,
default=[],
help="Authorized proof scope; repeat as needed. Not used for installer keys.",
)
parser.add_argument("--private-key", type=Path, required=True)
parser.add_argument("--keyring", type=Path, required=True)
parser.add_argument(
"--valid-days",
type=int,
default=365,
help="Validity from generation time (default: 365 days).",
)
parser.add_argument(
"--status",
choices=("active", "next"),
default="active",
)
args = parser.parse_args(argv)
if not KEY_ID_PATTERN.fullmatch(args.key_id):
parser.error("--key-id must be a valid opaque identifier")
if args.valid_days < 1 or args.valid_days > 3660:
parser.error("--valid-days must be between 1 and 3660")
scopes = _resolve_scopes(parser, purpose=args.purpose, scopes=args.scope)
private_path = args.private_key.expanduser().resolve()
keyring_path = args.keyring.expanduser().resolve()
_require_fresh_output(parser, private_path, label="private key")
_require_fresh_output(parser, keyring_path, label="keyring")
_require_private_directory(parser, private_path.parent)
_require_output_directory(parser, keyring_path.parent)
private_key = Ed25519PrivateKey.generate()
private_bytes = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
public_bytes = private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
public_base64 = base64.b64encode(public_bytes).decode("ascii")
now = datetime.now(UTC).replace(microsecond=0)
not_after = now + timedelta(days=args.valid_days)
purpose, schema = PURPOSES[args.purpose]
keyring = {
"$schema": schema,
"schema_version": "0.1.0",
"purpose": purpose,
"keys": [
{
"key_id": args.key_id,
"status": args.status,
"public_key": public_base64,
"allowed_scopes": scopes,
"not_before": _rfc3339(now),
"not_after": _rfc3339(not_after),
}
],
}
_write_new_private_file(private_path, private_bytes)
try:
_write_new_private_file(
keyring_path,
(json.dumps(keyring, indent=2, sort_keys=True) + "\n").encode("utf-8"),
)
except BaseException:
private_path.unlink(missing_ok=True)
keyring_path.unlink(missing_ok=True)
raise
print(f"private_key={private_path}")
print(f"keyring={keyring_path}")
print(f"key_id={args.key_id}")
print(f"allowed_scopes={','.join(scopes)}")
return 0
def _resolve_scopes(
parser: argparse.ArgumentParser, *, purpose: str, scopes: list[str]
) -> list[str]:
if purpose == "installer":
if scopes:
parser.error("installer authorities do not accept --scope")
return ["installed_release_origin"]
unique = list(dict.fromkeys(scopes))
if not unique:
parser.error("proof authorities require at least one --scope")
return unique
def _require_fresh_output(
parser: argparse.ArgumentParser, path: Path, *, label: str
) -> None:
if path.exists() or path.is_symlink():
parser.error(f"{label.capitalize()} output already exists: {path}")
def _require_private_directory(
parser: argparse.ArgumentParser, directory: Path
) -> None:
_require_output_directory(parser, directory)
mode = stat.S_IMODE(directory.stat().st_mode)
if mode & (stat.S_IRWXG | stat.S_IRWXO):
parser.error(
"Private-key parent directory must not be accessible by group or others"
)
def _require_output_directory(
parser: argparse.ArgumentParser, directory: Path
) -> None:
try:
metadata = directory.lstat()
except OSError as exc:
parser.error(f"Output parent directory is unavailable: {directory}")
raise AssertionError from exc
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):
parser.error(f"Output parent must be a real directory: {directory}")
def _write_new_private_file(path: Path, payload: bytes) -> None:
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
descriptor = os.open(path, flags, 0o600)
try:
with os.fdopen(descriptor, "wb", closefd=False) as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
metadata = os.fstat(descriptor)
if not stat.S_ISREG(metadata.st_mode) or stat.S_IMODE(metadata.st_mode) != 0o600:
raise OSError("Authority output could not be secured")
finally:
os.close(descriptor)
def _rfc3339(value: datetime) -> str:
return value.isoformat().replace("+00:00", "Z")
if __name__ == "__main__":
raise SystemExit(main())
+302
View File
@@ -0,0 +1,302 @@
#!/usr/bin/env python3
"""Dispatch protected package releases required by the govoplan meta-package."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import json
from pathlib import Path
import re
import subprocess
import sys
import tomllib
from gitea_common import (
GiteaClient,
GiteaError,
RepoTarget,
load_dotenv,
org_path,
quote_path,
repo_path,
require_token,
)
META_ROOT = Path(__file__).resolve().parents[2]
META_PROJECT = META_ROOT / "packages" / "govoplan-meta" / "pyproject.toml"
WORKFLOW_ID = "module-package-release.yml"
EXACT_REQUIREMENT = re.compile(
r"^(?P<name>govoplan-[a-z0-9-]+)(?:\[[a-z0-9_,.-]+\])?==(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$"
)
ACTIVE_STATES = {"queued", "waiting", "in_progress", "running"}
@dataclass(frozen=True, slots=True)
class PackageTarget:
distribution: str
version: str
repository: str
tag_exists: bool
has_webui: bool
@property
def tag(self) -> str:
return f"v{self.version}"
@property
def webui_package(self) -> str | None:
if not self.has_webui:
return None
return f"@govoplan/{self.distribution.removeprefix('govoplan-')}-webui"
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--url", default="https://git.add-ideas.de")
parser.add_argument("--owner", default="GovOPlaN")
parser.add_argument("--env-file", type=Path)
parser.add_argument(
"--repository",
action="append",
default=[],
help="Limit dispatch to one repository; repeat as needed.",
)
parser.add_argument(
"--verify-existing",
action="store_true",
help="Also rerun exact versions already present in both registries.",
)
parser.add_argument("--apply", action="store_true")
return parser
def package_targets(project_path: Path = META_PROJECT) -> tuple[PackageTarget, ...]:
project = tomllib.loads(project_path.read_text(encoding="utf-8"))["project"]
requirements = list(project.get("dependencies") or [])
requirements.extend(project.get("optional-dependencies", {}).get("full") or [])
parsed: dict[str, str] = {}
for requirement in requirements:
match = EXACT_REQUIREMENT.fullmatch(str(requirement))
if match is None:
raise ValueError(
f"Meta-package requirement is not an exact GovOPlaN version: {requirement!r}"
)
name = match.group("name")
version = match.group("version")
previous = parsed.setdefault(name, version)
if previous != version:
raise ValueError(f"Meta-package selects conflicting versions for {name}")
targets: list[PackageTarget] = []
for distribution, version in sorted(parsed.items()):
repository = distribution
repository_root = META_ROOT.parent / repository
if not (repository_root / ".git").is_dir():
raise ValueError(f"Package repository is not checked out: {repository}")
tag = f"v{version}"
tag_exists = _tag_exists(repository_root, tag)
has_webui = (
_tag_has_path(repository_root, tag, "webui/package.json")
if tag_exists
else False
)
targets.append(
PackageTarget(
distribution=distribution,
version=version,
repository=repository,
tag_exists=tag_exists,
has_webui=has_webui,
)
)
return tuple(targets)
def _tag_exists(repository: Path, tag: str) -> bool:
result = subprocess.run(
(
"git",
"-C",
str(repository),
"rev-parse",
"--verify",
"--quiet",
f"refs/tags/{tag}",
),
check=False,
capture_output=True,
text=True,
)
if result.returncode not in {0, 1}:
raise ValueError(
f"Could not inspect {repository.name}:{tag}: {result.stderr.strip()}"
)
return result.returncode == 0
def _tag_has_path(repository: Path, tag: str, path: str) -> bool:
result = subprocess.run(
("git", "-C", str(repository), "cat-file", "-e", f"{tag}:{path}"),
check=False,
capture_output=True,
text=True,
)
if result.returncode not in {0, 128}:
raise ValueError(
f"Could not inspect {repository.name}:{tag}:{path}: {result.stderr.strip()}"
)
return result.returncode == 0
def _published_packages(
client: GiteaClient, *, owner: str, package_type: str
) -> set[tuple[str, str]]:
values = client.paginate(
f"/packages/{quote_path(owner)}",
query={"type": package_type, "q": "govoplan"},
)
return {
(str(item.get("name") or ""), str(item.get("version") or ""))
for item in values
if item.get("type") == package_type
}
def _has_active_run(
client: GiteaClient, *, owner: str, repository: str
) -> bool:
payload = client.request_json(
"GET",
repo_path(
owner,
repository,
f"/actions/workflows/{quote_path(WORKFLOW_ID)}/runs",
),
query={"limit": 10},
)
runs = payload.get("workflow_runs") if isinstance(payload, dict) else None
return isinstance(runs, list) and any(
isinstance(run, dict) and str(run.get("status") or "") in ACTIVE_STATES
for run in runs
)
def dispatch(
client: GiteaClient,
*,
owner: str,
targets: tuple[PackageTarget, ...],
published_pypi: set[tuple[str, str]],
published_npm: set[tuple[str, str]],
verify_existing: bool,
apply: bool,
) -> tuple[int, int, int]:
dispatched = 0
active = 0
complete = 0
for target in targets:
wheel_exists = (target.distribution, target.version) in published_pypi
npm_exists = target.webui_package is None or (
target.webui_package,
target.version,
) in published_npm
if wheel_exists and npm_exists and not verify_existing:
complete += 1
print(f"complete {target.repository}:{target.tag}")
continue
if _has_active_run(client, owner=owner, repository=target.repository):
active += 1
print(f"active {target.repository}:{target.tag}")
continue
action = "dispatching" if apply else "would dispatch"
print(
f"{action} {target.repository}:{target.tag} "
f"(wheel={'present' if wheel_exists else 'missing'}, "
f"webui={'present' if npm_exists else 'missing'})"
)
if apply:
client.request_json(
"POST",
repo_path(
owner,
target.repository,
f"/actions/workflows/{quote_path(WORKFLOW_ID)}/dispatches",
),
body={"ref": "main", "inputs": {"release_tag": target.tag}},
)
dispatched += 1
return dispatched, active, complete
def main() -> int:
args = build_parser().parse_args()
try:
load_dotenv(args.env_file)
token = require_token()
targets = package_targets()
selected = set(args.repository)
if selected:
known = {target.repository for target in targets}
unknown = sorted(selected - known)
if unknown:
raise ValueError(
"Unknown meta-package repositories: " + ", ".join(unknown)
)
targets = tuple(
target for target in targets if target.repository in selected
)
missing_tags = [
f"{target.repository}:{target.tag}"
for target in targets
if not target.tag_exists
]
if missing_tags:
raise ValueError(
"Meta-package release tags are missing: " + ", ".join(missing_tags)
)
target = RepoTarget(base_url=args.url, owner=args.owner, repo="govoplan")
with GiteaClient(target, token) as client:
secrets = client.request_json(
"GET", org_path(args.owner, "/actions/secrets"), query={"limit": 50}
)
secret_names = {
str(item.get("name") or "")
for item in secrets
if isinstance(item, dict)
}
required = {"GOVOPLAN_PACKAGE_USERNAME", "GOVOPLAN_PACKAGE_TOKEN"}
if not required <= secret_names:
raise ValueError(
"Organization package publisher secrets are not configured"
)
published_pypi = _published_packages(
client, owner=args.owner, package_type="pypi"
)
published_npm = _published_packages(
client, owner=args.owner, package_type="npm"
)
counts = dispatch(
client,
owner=args.owner,
targets=targets,
published_pypi=published_pypi,
published_npm=published_npm,
verify_existing=args.verify_existing,
apply=args.apply,
)
action = "dispatched" if args.apply else "planned"
print(
f"Package set {action}: {counts[0]}; active: {counts[1]}; "
f"already complete: {counts[2]}."
)
return 0
except (GiteaError, OSError, ValueError, json.JSONDecodeError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+8 -5
View File
@@ -94,9 +94,6 @@ cleanup() {
trap cleanup EXIT
cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"
if [[ -f "$WEBUI/package-lock.release.json" ]]; then
cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"
fi
echo "Generating release lockfile from $WEBUI/package.release.json"
echo "Temporary workspace: $TMP_DIR"
@@ -120,7 +117,10 @@ GIT_ENV+=("GIT_CONFIG_COUNT=$git_config_count")
(
cd "$TMP_DIR"
"${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" install --package-lock-only --ignore-scripts
"${GIT_ENV[@]}" \
"npm_config_cache=$TMP_DIR/npm-cache" \
PATH="$(dirname "$NPM_BIN"):$PATH" \
"$NPM_BIN" install --package-lock-only --ignore-scripts
mapfile -t GIT_PACKAGES < <(
PATH="$(dirname "$NODE_BIN"):$PATH" "$NODE_BIN" <<'NODE'
const fs = require("fs");
@@ -136,7 +136,10 @@ NODE
)
if [[ "${#GIT_PACKAGES[@]}" -gt 0 ]]; then
echo "Refreshing git package lock entries: ${GIT_PACKAGES[*]}"
"${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
"${GIT_ENV[@]}" \
"npm_config_cache=$TMP_DIR/npm-cache" \
PATH="$(dirname "$NPM_BIN"):$PATH" \
"$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
fi
)
+77 -7
View File
@@ -333,20 +333,53 @@ path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(\s*version=)["\'][^"\']+["\'](,?\s*)$',
r'(?m)^(MODULE_VERSION\s*=\s*)["\'][^"\']+["\'](\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count == 0:
text, count = re.subn(
r'(?m)^(MODULE_VERSION\s*=\s*)["\'][^"\']+["\'](\s*)$',
r'(?ms)(^manifest\s*=\s*ModuleManifest\(.*?^\s*version\s*=\s*)["\'][^"\']+["\'](,?\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update ModuleManifest.version in {path}")
raise SystemExit(f"could not update module version declaration in {path}")
path.write_text(text)
PYCODE
done
}
update_package_init_versions() {
local repo="$1"
local version="$2"
local package_init=""
for package_init in "$repo"/src/*/__init__.py; do
[[ -f "$package_init" ]] || continue
if ! grep -q '^__version__\s*=' "$package_init"; then
continue
fi
"$PYTHON" - "$package_init" "$version" <<'PYCODE'
from __future__ import annotations
import pathlib
import re
import sys
path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(__version__\s*=\s*)["\'][^"\']+["\'](\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update __version__ in {path}")
path.write_text(text)
PYCODE
done
@@ -380,6 +413,11 @@ if project_name != "govoplan-core":
peers["@govoplan/core-webui"] = f"^{new_version}"
path.write_text(json.dumps(data, indent=2) + "\n")
PYCODE
done
"$PYTHON" "$META_ROOT/tools/release/synchronize-webui-package-metadata.py" --repo "$repo"
for package_path in "$repo/package.json" "$repo/webui/package.json"; do
[[ -f "$package_path" ]] || continue
synchronize_lockfile_root "$package_path" "${package_path%package.json}package-lock.json"
done
@@ -428,7 +466,19 @@ if not isinstance(version, str) or not version:
lock["version"] = version
packages = lock.get("packages")
if isinstance(packages, dict) and isinstance(packages.get(""), dict):
packages[""]["version"] = version
root = packages[""]
root["version"] = version
for group in (
"dependencies",
"devDependencies",
"optionalDependencies",
"peerDependencies",
"peerDependenciesMeta",
):
if group in package:
root[group] = package[group]
else:
root.pop(group, None)
lock_path.write_text(json.dumps(lock, indent=2) + "\n")
PYCODE
}
@@ -470,6 +520,7 @@ update_version_files() {
update_pyproject "$repo" "$version"
update_manifest_version "$repo" "$project_name" "$version"
update_package_init_versions "$repo" "$version"
update_webui_package "$repo" "$project_name" "$version"
}
@@ -500,10 +551,11 @@ run_version_alignment_gate() {
"$PYTHON"
"$META_ROOT/tools/checks/check-version-alignment.py"
--workspace-root "$PARENT"
--release-composition
)
if [[ "$mode" == "source" ]]; then
command+=(--source-metadata-only)
else
command+=(--release-composition)
fi
local repo
for repo in "${PACKAGE_REPOS[@]}"; do
@@ -533,7 +585,8 @@ run_migration_release_audit() {
command+=("--strict")
;;
auto)
command+=("--strict-if-baseline")
# A coordinated release creates a new baseline after confirmation. The
# preflight validates the graph; strictness applies to that new baseline.
;;
warn)
;;
@@ -546,6 +599,18 @@ run_migration_release_audit() {
run "${command[@]}"
}
record_migration_release_baseline() {
local audit_script="$META_ROOT/tools/release/release-migration-audit.py"
[[ -f "$audit_script" ]] || fail "missing migration audit helper: $audit_script"
run "$PYTHON" "$audit_script" \
--track release \
--record-release "$TARGET_VERSION"
if [[ "$DRY_RUN" -eq 0 ]]; then
"$PYTHON" "$audit_script" --track release --strict
fi
}
print_command() {
printf '+'
printf ' %q' "$@"
@@ -872,6 +937,8 @@ run_manifest_shape_gate
confirm_release
record_migration_release_baseline
for repo in "${PACKAGE_REPOS[@]}"; do
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "Would update version files in $repo to $TARGET_VERSION"
@@ -938,10 +1005,13 @@ fi
run git -C "$ROOT" commit -m "$COMMIT_MESSAGE"
run git -C "$ROOT" tag -a "$TAG" -m "$TAG_MESSAGE"
for repo in "${PRE_CORE_REPOS[@]}"; do
for repo in "${MODULE_REPOS[@]}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
run git -C "$ROOT" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$ROOT]}" "refs/tags/$TAG"
for repo in "${SUPPORT_REPOS[@]}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
if [[ "$PUBLISH_WEB_CATALOG" -eq 1 ]]; then
CATALOG_ARGS=(
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Synchronize duplicated publish and development WebUI package contracts."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
SYNCHRONIZED_KEYS = ("peerDependencies", "peerDependenciesMeta")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--repo", type=Path, required=True)
args = parser.parse_args()
root_path = args.repo / "package.json"
webui_path = args.repo / "webui" / "package.json"
if not root_path.exists() or not webui_path.exists():
return 0
root = _load(root_path)
webui = _load(webui_path)
root_name = root.get("name")
webui_name = webui.get("name")
if not isinstance(root_name, str) or root_name != webui_name:
return 0
changed = False
for key in SYNCHRONIZED_KEYS:
if key in webui:
value = webui[key]
if root.get(key) != value:
root[key] = value
changed = True
elif key in root:
del root[key]
changed = True
if changed:
root_path.write_text(json.dumps(root, indent=2) + "\n")
print(f"Synchronized WebUI peer metadata in {root_path}")
return 0
def _load(path: Path) -> dict[str, object]:
payload = json.loads(path.read_text())
if not isinstance(payload, dict):
raise SystemExit(f"package metadata must be an object: {path}")
return payload
if __name__ == "__main__":
raise SystemExit(main())
+83 -22
View File
@@ -14,6 +14,8 @@ on:
jobs:
publish-packages:
runs-on: ubuntu-latest
env:
GITEA_REPOSITORY: ${{ gitea.repository }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
@@ -29,7 +31,6 @@ jobs:
env:
REQUESTED_TAG: ${{ inputs.release_tag }}
TRIGGER_TAG: ${{ gitea.ref_name }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
set -euo pipefail
tag="${REQUESTED_TAG:-$TRIGGER_TAG}"
@@ -43,24 +44,6 @@ jobs:
echo "Release tag is not contained in main" >&2
exit 1
}
python - "$tag" <<'PY'
import fnmatch
import json
import os
import sys
import urllib.request
tag = sys.argv[1]
repository = os.environ["GITEA_REPOSITORY"]
request = urllib.request.Request(
f"{os.environ['GITEA_API_URL']}/repos/{repository}/tag_protections",
headers={"Authorization": f"token {os.environ['GITEA_TOKEN']}"},
)
with urllib.request.urlopen(request, timeout=30) as response:
protections = json.load(response)
if not any(fnmatch.fnmatchcase(tag, item.get("name_pattern", "")) for item in protections):
raise SystemExit(f"Release tag {tag!r} is not covered by repository tag protection")
PY
git checkout --detach "$tag"
printf 'RELEASE_TAG=%s\n' "$tag" >> "$GITEA_ENV"
printf 'SOURCE_DATE_EPOCH=%s\n' "$(git show -s --format=%ct HEAD)" >> "$GITEA_ENV"
@@ -130,7 +113,7 @@ jobs:
const escapedRepository = repository.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const gitTag = specifier.match(
new RegExp(
`^git\\+(?:ssh://git@|https://)git\\.add-ideas\\.de/GovOPlaN/${escapedRepository}\\.git#v([0-9]+\\.[0-9]+\\.[0-9]+)$`,
`^git\\+(?:ssh://git@|https://)git\\.add-ideas\\.de/(?:GovOPlaN|add-ideas)/${escapedRepository}\\.git#v([0-9]+\\.[0-9]+\\.[0-9]+)$`,
),
);
if (gitTag) {
@@ -180,6 +163,78 @@ jobs:
with:
name: module-packages-${{ gitea.ref_name }}
path: dist/package-artifacts.json
- name: Check immutable registry state
shell: bash
env:
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_TOKEN"
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import tomllib
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
api_root = "https://git.add-ideas.de/api/v1/packages/GovOPlaN"
token = os.environ["PACKAGE_TOKEN"]
def should_publish(kind, name, version, path):
package_url = "/".join(
(api_root, kind, quote(name, safe=""), quote(version, safe=""), "files")
)
request = Request(
package_url,
headers={"Accept": "application/json", "Authorization": f"token {token}"},
)
try:
with urlopen(request, timeout=30) as response:
files = json.load(response)
except HTTPError as exc:
if exc.code == 404:
print(f"{kind} package {name}=={version} is not published yet")
return True
raise
if not isinstance(files, list) or len(files) != 1:
raise SystemExit(
f"immutable {kind} package {name}=={version} has an unexpected file set"
)
expected_sha256 = hashlib.sha256(path.read_bytes()).hexdigest()
if files[0].get("sha256") != expected_sha256:
raise SystemExit(
f"immutable {kind} package {name}=={version} already exists with a different SHA-256"
)
print(f"verified existing {kind} package {name}=={version} ({expected_sha256})")
return False
project = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))["project"]
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("release build must contain exactly one wheel")
publish_pypi = should_publish(
"pypi", str(project["name"]), str(project["version"]), wheels[0]
)
tarballs = tuple(Path("dist").glob("*.tgz"))
if len(tarballs) > 1:
raise SystemExit("release build must contain at most one npm package")
publish_npm = False
if tarballs:
webui = json.loads(
Path(".package-webui/package.json").read_text(encoding="utf-8")
)
publish_npm = should_publish(
"npm", str(webui["name"]), str(webui["version"]), tarballs[0]
)
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"PUBLISH_PYPI={int(publish_pypi)}\n")
env_file.write(f"PUBLISH_NPM={int(publish_npm)}\n")
PY
- name: Publish wheel and WebUI package
shell: bash
env:
@@ -189,13 +244,17 @@ jobs:
set -euo pipefail
test -n "$PACKAGE_USERNAME"
test -n "$PACKAGE_TOKEN"
if [[ "$PUBLISH_PYPI" == 1 ]]; then
TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \
python -m twine upload --non-interactive \
--repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \
dist/*.whl
else
echo "Exact wheel is already present; skipping immutable retry."
fi
shopt -s nullglob
webui_packages=(dist/*.tgz)
if (( ${#webui_packages[@]} )); then
if (( ${#webui_packages[@]} )) && [[ "$PUBLISH_NPM" == 1 ]]; then
npmrc="$(mktemp)"
trap 'rm -f "$npmrc"' EXIT
chmod 600 "$npmrc"
@@ -203,7 +262,9 @@ jobs:
'@govoplan:registry=https://git.add-ideas.de/api/packages/GovOPlaN/npm/' \
"//git.add-ideas.de/api/packages/GovOPlaN/npm/:_authToken=$PACKAGE_TOKEN" \
> "$npmrc"
NPM_CONFIG_USERCONFIG="$npmrc" npm publish "${webui_packages[0]}" \
NPM_CONFIG_USERCONFIG="$npmrc" npm publish "./${webui_packages[0]}" \
--ignore-scripts --access public \
--registry https://git.add-ideas.de/api/packages/GovOPlaN/npm/
elif (( ${#webui_packages[@]} )); then
echo "Exact WebUI package is already present; skipping immutable retry."
fi