24 Commits
Author SHA1 Message Date
zemion 7ec0bbb826 Release v0.1.17
Dependency Audit / dependency-audit (push) Failing after 1m50s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m34s
Developer Meta-package Release / publish-package (push) Successful in 10s
2026-08-05 20:34:17 +02:00
zemion 3ca068f76a Release v0.1.16
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m18s
Dependency Audit / dependency-audit (push) Failing after 1m47s
Developer Meta-package Release / publish-package (push) Successful in 11s
2026-08-05 19:52:32 +02:00
zemion 61463a24cb Bind runtime releases to protected source tags [skip ci] 2026-08-04 16:43:54 +02:00
zemion 8262215fcd Keep runtime builder path stable [skip ci] 2026-08-04 16:34:07 +02:00
zemion 8aba74e01e Make meta-package release retries tag-safe [skip ci] 2026-08-04 16:27:47 +02:00
zemion a24c94435e Release v0.1.15
Dependency Audit / dependency-audit (push) Failing after 1m49s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m50s
Developer Meta-package Release / publish-package (push) Failing after 4s
2026-08-04 15:20:50 +02:00
zemion 774793976c Support legacy module version declarations
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Failing after 1m42s
Security Audit / security-audit (push) Successful in 10m43s
2026-08-04 15:08:46 +02:00
zemion 492449a4e2 Align all release version declarations
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 10m42s
2026-08-04 15:06:36 +02:00
zemion 8e890b37ed Validate candidate migration baseline during release
Dependency Audit / dependency-audit (push) Failing after 1m42s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m38s
2026-08-04 15:01:50 +02:00
zemion 077735bc24 Bind migration heads to coordinated releases
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Failing after 1m44s
Security Audit / security-audit (push) Successful in 10m36s
2026-08-04 14:55:58 +02:00
zemion d9522d3cc4 Publish release tags in dependency order
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m53s
2026-08-04 14:54:54 +02:00
zemion bad0ea37a7 Automate exact package-set publication
Dependency Audit / dependency-audit (push) Failing after 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m46s
2026-08-04 14:32:06 +02:00
zemion 9ffd46fe22 Make package publication retries hash-safe
Dependency Audit / dependency-audit (push) Failing after 1m48s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m35s
2026-08-04 14:18:49 +02:00
zemion be51a9c347 Document production target evidence handoff
Dependency Audit / dependency-audit (push) Failing after 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m40s
2026-08-04 14:00:45 +02:00
zemion e36a6573bf Harden package release workflows for Gitea 2026-08-04 14:00:32 +02:00
zemion 629bfec1f1 Cover datasource publication change events
Dependency Audit / dependency-audit (push) Failing after 1m40s
Deployment Installer / deployment-installer (push) Successful in 5s
Security Audit / security-audit (push) Successful in 10m48s
2026-08-04 12:05:48 +02:00
zemion 9e956eec6f Classify first-run bootstrap endpoints
Dependency Audit / dependency-audit (push) Failing after 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m52s
2026-08-04 10:07:24 +02:00
zemion 9bb2c808a6 Enforce declared platform interface inventory
Dependency Audit / dependency-audit (push) Failing after 1m36s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m2s
2026-08-04 05:20:47 +02:00
zemion f7590a7b8b Add registry-backed module package releases
Dependency Audit / dependency-audit (push) Failing after 1m37s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m21s
2026-08-04 04:14:28 +02:00
zemion 1a68565ba0 Reconcile encryption interface surfaces
Dependency Audit / dependency-audit (push) Failing after 1m40s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m33s
2026-08-04 01:28:03 +02:00
zemion d9f67b5c26 Reconcile implemented platform interface surfaces
Dependency Audit / dependency-audit (push) Failing after 1m39s
Deployment Installer / deployment-installer (push) Successful in 5s
Security Audit / security-audit (push) Successful in 10m32s
2026-08-04 01:04:40 +02:00
zemion 1cfdaec250 inventory: mark campaign archive UI reachable
Dependency Audit / dependency-audit (push) Successful in 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m54s
2026-08-03 20:39:16 +02:00
zemion 62501d399a ci: enforce endpoint inventory independently
Dependency Audit / dependency-audit (push) Successful in 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m43s
2026-08-03 20:29:14 +02:00
zemion ce5528e3b8 Record first verified runtime distribution 2026-08-03 20:17:29 +02:00
85 changed files with 9849 additions and 385 deletions
+2 -2
View File
@@ -55,9 +55,9 @@ jobs:
- name: Install WebUI release dependencies with test scripts
working-directory: govoplan
run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
- name: Validate platform endpoint surface declarations
- name: Validate platform interface and endpoint declarations
working-directory: govoplan
run: .venv/bin/python tools/inventory/platform-interface-inventory.py --strict
run: .venv/bin/python tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
- name: Validate Search against PostgreSQL
working-directory: govoplan
env:
@@ -0,0 +1,178 @@
name: Developer Meta-package Release
on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
version:
description: Existing protected release version without leading v
required: true
type: string
jobs:
publish-package:
runs-on: ubuntu-latest
env:
GITEA_REPOSITORY: ${{ gitea.repository }}
TRIGGER_TAG: ${{ gitea.ref_name }}
REQUESTED_VERSION: ${{ inputs.version }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: "3.12"
- name: Validate protected release tag and package version
run: |
python - <<'PY'
import os
from pathlib import Path
import subprocess
import tomllib
requested_version = os.environ.get("REQUESTED_VERSION", "").strip()
tag = f"v{requested_version}" if requested_version else os.environ["TRIGGER_TAG"]
if not tag.startswith("v") or not tag[1:]:
raise SystemExit("release tag is missing")
project_text = subprocess.check_output(
["git", "show", f"{tag}:packages/govoplan-meta/pyproject.toml"],
text=True,
)
project = tomllib.loads(project_text)["project"]
if tag != f"v{project['version']}":
raise SystemExit("meta-package version does not match the release tag")
tag_commit = subprocess.check_output(
["git", "rev-parse", f"refs/tags/{tag}^{{commit}}"], text=True
).strip()
if subprocess.run(
["git", "merge-base", "--is-ancestor", tag_commit, "origin/main"]
).returncode:
raise SystemExit("release tag is not contained in main")
if not requested_version:
head_commit = subprocess.check_output(
["git", "rev-parse", "HEAD"], text=True
).strip()
if head_commit != tag_commit:
raise SystemExit("tag-triggered checkout does not match the release tag")
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"RELEASE_TAG={tag}\n")
subprocess.run(["git", "checkout", "--detach", tag_commit], check=True)
PY
- name: Build developer package
run: |
set -euo pipefail
python -m pip install --disable-pip-version-check build==1.5.0 twine==7.0.0
python -m build --wheel --outdir dist packages/govoplan-meta
python -m twine check dist/*.whl
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import subprocess
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("meta release must contain exactly one wheel")
wheel = wheels[0]
evidence = {
"schema_version": "1",
"repository": os.environ["GITEA_REPOSITORY"],
"tag": os.environ["RELEASE_TAG"],
"commit": subprocess.check_output(
["git", "rev-parse", "HEAD"], text=True
).strip(),
"artifacts": [
{
"filename": wheel.name,
"sha256": hashlib.sha256(wheel.read_bytes()).hexdigest(),
"size": wheel.stat().st_size,
}
],
}
Path("dist/package-artifacts.json").write_text(
json.dumps(evidence, indent=2, sort_keys=True) + "\n", encoding="utf-8"
)
PY
- name: Retain package hash evidence
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: developer-meta-package
path: dist/package-artifacts.json
- name: Check immutable registry state
env:
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_TOKEN"
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import tomllib
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
project = tomllib.loads(
Path("packages/govoplan-meta/pyproject.toml").read_text(encoding="utf-8")
)["project"]
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("meta release must contain exactly one wheel")
wheel = wheels[0]
digest = hashlib.sha256(wheel.read_bytes()).hexdigest()
package_url = "/".join(
(
"https://git.add-ideas.de/api/v1/packages/GovOPlaN",
"pypi",
quote(str(project["name"]), safe=""),
quote(str(project["version"]), safe=""),
"files",
)
)
request = Request(
package_url,
headers={
"Accept": "application/json",
"Authorization": f"token {os.environ['PACKAGE_TOKEN']}",
},
)
publish = True
try:
with urlopen(request, timeout=30) as response:
files = json.load(response)
except HTTPError as exc:
if exc.code != 404:
raise
else:
if not isinstance(files, list) or len(files) != 1:
raise SystemExit("immutable meta-package has an unexpected file set")
if files[0].get("sha256") != digest:
raise SystemExit(
"immutable meta-package already exists with a different SHA-256"
)
publish = False
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"PUBLISH_PYPI={int(publish)}\n")
PY
- name: Publish developer package
env:
PACKAGE_USERNAME: ${{ secrets.GOVOPLAN_PACKAGE_USERNAME }}
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_USERNAME"
test -n "$PACKAGE_TOKEN"
if [[ "$PUBLISH_PYPI" == 1 ]]; then
TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \
python -m twine upload --non-interactive \
--repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \
dist/*.whl
else
echo "Exact developer meta-package is already present; skipping immutable retry."
fi
+6
View File
@@ -25,6 +25,12 @@ jobs:
- name: Bootstrap GovOPlaN repositories
working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --reuse-checkout-auth --exclude-repo addideas-govoplan-website
- name: Validate package publication contracts
working-directory: govoplan
run: |
python tools/repo/sync-module-package-workflows.py --check
python tools/release/generate-developer-meta-package.py --check
python -m unittest tests.test_module_package_workflows tests.test_package_registry_release
- name: Install backend release integration dependencies
working-directory: govoplan
run: |
+34 -6
View File
@@ -92,6 +92,15 @@ jobs:
if image_pattern.fullmatch(os.environ[name]) is None:
raise SystemExit(f"{name} must be an exact sha256 image reference")
PY
- name: Resolve immutable release source
working-directory: govoplan
env:
VERSION: ${{ inputs.version }}
run: |
git fetch --force --no-tags origin "refs/tags/v$VERSION:refs/tags/v$VERSION"
mkdir -p runtime-output
git rev-parse "v$VERSION^{commit}" > runtime-output/release-source-commit
grep -Eq '^[0-9a-f]{40}$' runtime-output/release-source-commit
- name: Use HTTPS for GovOPlaN repositories
run: |
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
@@ -101,12 +110,27 @@ jobs:
run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --reuse-checkout-auth --exclude-repo addideas-govoplan-website
- name: Build release wheel roots and WebUI
working-directory: govoplan
env:
VERSION: ${{ inputs.version }}
GOVOPLAN_PACKAGE_USERNAME: ${{ secrets.GOVOPLAN_PACKAGE_USERNAME }}
GOVOPLAN_PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
python -m venv .runtime-build
.runtime-build/bin/python -m pip install --upgrade pip wheel cryptography
mkdir -p runtime-output/local-wheels
.runtime-build/bin/python -m pip wheel --no-deps --wheel-dir runtime-output/local-wheels --requirement requirements-release.txt
bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
.runtime-build/bin/python -m pip install --upgrade pip cryptography
.runtime-build/bin/python tools/release/generate-release-package-set.py \
--version "$VERSION" \
--output runtime-output/release-packages.json
.runtime-build/bin/python tools/release/resolve-package-artifacts.py \
--package-set runtime-output/release-packages.json \
--wheelhouse runtime-output/local-wheels \
--webui-packages runtime-output/webui-packages \
--lock-output runtime-output/package-artifacts.lock.json \
--requirements-output runtime-output/requirements-release.packages.txt \
--python .runtime-build/bin/python
PYTHON="$PWD/.runtime-build/bin/python" \
GOVOPLAN_WEBUI_PACKAGE_LOCK="$PWD/runtime-output/package-artifacts.lock.json" \
GOVOPLAN_WEBUI_PACKAGE_DIR="$PWD/runtime-output/webui-packages" \
bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
npm --prefix ../govoplan-core/webui run build
.runtime-build/bin/python tools/release/prepare-runtime-context.py \
--wheelhouse runtime-output/local-wheels \
@@ -240,7 +264,6 @@ jobs:
working-directory: govoplan
env:
VERSION: ${{ inputs.version }}
SOURCE_COMMIT: ${{ gitea.sha }}
SIGNING_KEY: ${{ secrets.RUNTIME_DISTRIBUTION_SIGNING_KEY }}
SIGNING_KEY_ID: ${{ secrets.RUNTIME_DISTRIBUTION_SIGNING_KEY_ID }}
TRUSTED_KEYRING: ${{ secrets.RUNTIME_DISTRIBUTION_KEYRING }}
@@ -251,6 +274,7 @@ jobs:
GARAGE_IMAGE: ${{ inputs.garage_image }}
TEST_MAIL_IMAGE: ${{ inputs.test_mail_image }}
run: |
SOURCE_COMMIT="$(cat runtime-output/release-source-commit)"
test -n "$SIGNING_KEY"
test -n "$SIGNING_KEY_ID"
test -n "$TRUSTED_KEYRING"
@@ -264,6 +288,7 @@ jobs:
--web-metadata runtime-output/web-metadata.json \
--deployer runtime-output/govoplan-deploy.pyz \
--deployer-url "$ARTIFACT_BASE/govoplan-deploy.pyz" \
--package-lock runtime-output/package-artifacts.lock.json \
--artifact-base-url "$ARTIFACT_BASE" \
--source-commit "$SOURCE_COMMIT" \
--version "$VERSION" \
@@ -347,9 +372,9 @@ jobs:
working-directory: govoplan
env:
VERSION: ${{ inputs.version }}
SOURCE_COMMIT: ${{ gitea.sha }}
GITEA_RELEASE_TOKEN: ${{ secrets.GOVOPLAN_RELEASE_TOKEN }}
run: |
SOURCE_COMMIT="$(cat runtime-output/release-source-commit)"
python tools/release/publish-runtime-release.py \
--tag "v$VERSION" \
--target-commit "$SOURCE_COMMIT" \
@@ -361,6 +386,9 @@ jobs:
--asset runtime-output/distribution-manifest.json.sha256 \
--asset runtime-output/distribution-keyring.json \
--asset runtime-output/context-amd64/composition.json \
--asset runtime-output/release-packages.json \
--asset runtime-output/package-artifacts.lock.json \
--asset runtime-output/requirements-release.packages.txt \
--asset runtime-output/evidence/api-sbom.cdx.json \
--asset runtime-output/evidence/web-sbom.cdx.json \
--asset runtime-output/evidence/api-provenance.json \
+3
View File
@@ -10,6 +10,9 @@ tools/release/runtime/*
!tools/release/runtime/Dockerfile.web
!tools/release/runtime/nginx.conf
__pycache__/
build/
dist/
*.egg-info/
audit-reports/
coverage/
htmlcov/
+15
View File
@@ -113,6 +113,18 @@ Generate the CycloneDX dependency inventory from a resolved release environment:
./.venv/bin/python tools/release/generate-release-sbom.py --python ./.venv/bin/python
```
Synchronize module package workflows and inspect the registry release contract:
```sh
./.venv/bin/python tools/repo/sync-module-package-workflows.py --check
./.venv/bin/python tools/release/generate-release-package-set.py \
--output /tmp/govoplan-release-packages.json
```
Package publication, exact artifact locking, and the optional `govoplan`
developer meta-package are documented in
[Package Registry Releases](docs/PACKAGE_REGISTRY_RELEASES.md).
For reproducible release artifacts, set `SOURCE_DATE_EPOCH` to the release
commit timestamp (or pass an explicit timezone-qualified `--timestamp`):
@@ -161,6 +173,9 @@ The current executable slice and remaining production gates are documented in
[Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md).
Same-host replica balancing and the multi-host promotion boundary are documented
in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
Create, update, pause, resume, verify and remove a local or multi-hypervisor K3s
VM target with the guarded lifecycle documented in
[Kubernetes VM Test Lab](docs/KUBERNETES_TEST_LAB.md).
The recovery state machine, migration rollback boundary, and required restore
drills are documented in
[Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md).
+167
View File
@@ -0,0 +1,167 @@
# Assisted and Non-Digital Channels
## Purpose
GovOPlaN must support people who cannot or do not use a self-service portal.
Telephone, paper, in-person service, authorized representation, mobile staff,
interpreters, and temporary offline work are not exceptional side systems.
They are governed channels into the same service, case, workflow, record, and
decision.
The goal is equivalent institutional treatment, not forced channel identity.
The system preserves which channel was used and which evidence is available
without giving digitally confident users stronger substantive rights.
The first end-to-end journey is tracked in
[GovOPlaN #42](https://git.add-ideas.de/GovOPlaN/govoplan/issues/42).
## Actor Model
Every assisted interaction distinguishes:
- the affected person or organization;
- the real staff member or external helper entering information;
- the represented party and representation basis;
- an interpreter, witness, guardian, or support person where relevant;
- the responsible institutional function;
- the channel and location;
- the person who reviewed or confirmed the captured information.
"Entered by" is not "declared by". "Declared by" is not "verified by".
Authentication assurance, representation authority, and evidence quality are
separate fields.
## Channel-Neutral Intake Contract
All channels create the same versioned service/form submission contract with
additional provenance:
- service, form, schema, language, and accessibility version;
- valid and recorded time;
- channel (`portal`, `counter`, `telephone`, `paper`, `email`, `mobile`,
`representative`, `offline_import`, or configured extension);
- affected and represented parties;
- capture actor and responsible function;
- consent, notice, purpose, legal basis, and information source;
- field-level source and confidence where staff transcribed or inferred data;
- attachments, scans, originals, signatures, recordings, and attestations as
governed evidence references;
- read-back/confirmation result and correction path;
- receipt and chosen return channels;
- duplicate/matching assessment and any manual resolution.
Forms Runtime owns the submission lifecycle. Parties owns procedural capacity
and representation. Identity/Addresses own subject and contact references.
Cases owns the matter. Records owns filing and retention. Audit preserves the
action/effect evidence.
## Assisted Session
An assisted session is a resumable work item, not a privileged bypass. It:
1. selects service, language, channel, affected party, and represented capacity;
2. shows the staff member only fields and evidence relevant to the service;
3. explains why sensitive data is requested and what evidence quality is
required;
4. records source per value when information comes from speech, paper, an
existing register, or staff observation;
5. validates and previews consequences before submission;
6. supports read-back, correction, confirmation, and a second-person check
where policy requires it;
7. generates an accessible receipt through the requested channel;
8. creates follow-up tasks when original documents, signatures, translation,
or verification remain outstanding.
The helper's normal account and represented function remain in the audit
chain. Assistance never grants access to unrelated records about the person.
## Paper And Scanning
- Register receipt before scanning so custody and deadlines do not depend on
successful OCR.
- Store the original scan or external archive reference with digest, pages,
capture device/provider, time, operator, and quality assessment.
- Treat OCR and extracted fields as derived data with confidence and source
coordinates. A person confirms consequential values.
- Support separation, ordering, missing-page, duplicate, malware, and
readability review.
- File the resulting document and submission into the appropriate eAkte;
retain or return the physical original according to policy.
- Produce cover sheets, barcodes, and return instructions through Templates,
not a separate print domain.
## Telephone And In-Person Handling
- Show a scripted but adaptable interview from the same Form definition.
- Record how identity and representation were checked; do not equate caller ID
with identity proof.
- Require explicit confirmation of consequential declarations and capture the
method (read-back, signed summary, one-time code, witness, later letter).
- Record call audio only when a lawful, declared profile permits it; an
interaction note is the default.
- Make interrupted sessions resumable without exposing prior answers to an
unauthorized caller or visitor.
## Offline And Mobile Work
Offline packages are encrypted, device-bound, time-limited, purpose-limited,
and contain only the required forms/reference data. Synchronization uses
idempotent intents and exposes conflicts rather than last-write-wins. Device
loss, expiry, revocation, duplicate submission, clock drift, and outcome
unknown have explicit recovery paths.
## Outbound Non-Digital Delivery
Campaign and Postbox model one delivery intent with channel choices and policy:
- portal/postbox delivery;
- email;
- print and postal fulfillment through a managed provider or local handoff;
- in-person collection;
- telephone notification followed by durable confirmation;
- accessible or language-specific variants.
Distribution preferences are purpose- and service-specific, effective-dated,
and may be overridden only by a documented legal or urgent-delivery rule. A
fallback occurs only before a channel has accepted the effect unless policy
explicitly authorizes duplicate delivery. Receipts distinguish creation,
provider acceptance, dispatch, delivery, return, and acknowledgement.
## Accessibility And Equality
- The person can request language, easy-language, large-print, screen-reader,
sign-language, relay, interpreter, or representative support without those
preferences becoming a general-purpose profile visible everywhere.
- Staff interfaces support keyboard-only capture, clear focus, error summary,
read-back, and printable/offline alternatives.
- Channel choice and need for assistance must not be used as an adverse risk
signal.
- Reports compare completion, wait, correction, abandonment, and outcome by
channel only under a declared equality/service-quality purpose and with
privacy thresholds.
## Security And Abuse Controls
- purpose-aware field access and session timeout;
- current authority checks for every read and effect;
- dual control for high-risk identity, payment, address, or representation
changes;
- immutable source/attestation evidence and correction history;
- rate and anomaly controls that do not silently reject a person;
- explicit safe handling of domestic-abuse, protected-address, witness, or
sealed-record cases;
- no secret answers or full documents in ordinary operational logs.
## First Reference Journey
Implement the permit-to-payment/service-to-decision journey through three
equivalent starts:
1. self-service portal submission;
2. staff-assisted counter/telephone submission;
3. paper receipt, scan, extraction, confirmation, and filing.
All three must create the same Case and Workflow contract, preserve different
provenance, support correction, produce a receipt, file an eAkte, reach the same
decision rules, and prove accessibility, privacy, recovery, and channel
fallback in browser and operator tests.
+8 -2
View File
@@ -1,5 +1,11 @@
# GovOPlaN Capability and IT-Infrastructure Fit Assessment
> **Pinned historical evidence:** This document assesses the exact 2026-07-22
> Campaign composition below. It is intentionally not updated to describe later
> main-branch work. Use [Strategy Status](STRATEGY_STATUS.md) for the current
> cross-product reconciliation and create a new dated fit assessment for a new
> target composition.
## Assessment record
| Field | Value |
@@ -19,8 +25,8 @@
Datasources, Dataflow, Search, encryption contracts, and other later main-branch
work must not be inferred into this evidence record. The current product
direction and implemented-state reconciliation are documented separately in
the
[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
the [Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md)
and [Strategy Status](STRATEGY_STATUS.md).
This is a fit assessment, not a production approval or security certification.
It deliberately does not infer implementation from a repository, issue, or
+11 -71
View File
@@ -18,7 +18,8 @@ Read it together with:
- the [institutional governance target architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md)
- the [selected reference-journey program](REFERENCE_JOURNEY_PROGRAM.md)
- the [current capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md)
- the [current strategy status](STRATEGY_STATUS.md)
- the [pinned Campaign capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md)
- the [interface pattern language](INTERFACE_PATTERN_LANGUAGE.md)
- the [interface surface inventory](INTERFACE_SURFACE_INVENTORY.md)
- the [module contract and install model](MODULE_CONTRACTS_AND_INSTALLS.md)
@@ -41,8 +42,8 @@ Read it together with:
- Use [Near-term portfolio order](#near-term-portfolio-order) for the bridge to
implementation and [Product decisions](#product-decisions-to-make-progressively)
for choices that can remain deferred.
- Use the [dated snapshot appendix](#snapshot-appendix-2026-07-20) only to
understand which live backlog and release facts informed this revision.
- Use the [dated strategic review](STRATEGIC_REVIEW_2026-08-05.md) to understand
why the current convergence and reference-journey order was chosen.
### Planning ownership
@@ -50,7 +51,7 @@ Read it together with:
| --- | --- |
| What product should GovOPlaN become, for whom, in which configurations, and through which outcome horizons? | This meta roadmap |
| Which module owns a capability, which technical wave should deliver it, and what implementation gates apply? | The Core master roadmap and owning-module concepts |
| What is actively planned, blocked, implemented, or closed now? | Gitea issues |
| What is actively planned, blocked, implemented, or closed now? | Gitea issues and the dated reconciliation in `STRATEGY_STATUS.md` |
| What can a named composition credibly claim in a target environment? | A dated capability/infrastructure fit assessment |
The horizons and near-term order below express product outcomes and portfolio
@@ -1351,71 +1352,10 @@ language, what service it configured, who can act, which systems participate,
what happens when they fail, how a decision can be reviewed, and where the
evidence remains—and the product can prove that explanation at runtime.
## Snapshot appendix: 2026-07-20
## Dated Context
This appendix records volatile facts that informed this revision. It is not a
second source of truth and should be refreshed or removed when a later roadmap
review uses a new release/backlog snapshot.
### Composition and release snapshot
The cross-repository contract scan found 43 module manifest contracts, 29
provided interface names, 16 requirements, and no contract error across 65
scanned repositories. That is meaningful composition evidence, but the release
metadata trailed the integrated code: Core, Policy, Poll, and Scheduling
declared `0.1.9` while the whole-product release requirements remained on
module tag `v0.1.8`; the root self-hosted `.env.example` and release smoke
composition did not yet exercise all installed release modules. Other
development compositions already included some of those modules. This was a
release/composition gap, not evidence that the underlying slices did not exist.
### Backlog snapshot
The Gitea audit found 206 open issues across 36 of 66 catalogued repositories
and 362 closed issues. Campaign had 51 open issues and Core 44; together they
held 46% of current work. This reflected substantial completed kernel,
security, and platform work and a deliberate concentration on the first usable
vertical, but also risked crowding out production evidence and the shared
process spine.
The issue workflow needed a reconciliation pass before another delivery
program could be inferred from labels: 119 open issues remained in triage, 116
had no milestone, and several recently pushed Calendar, Scheduling, Poll,
Campaign, and Files slices still described themselves as local or awaiting
integration. Conversely, 30 repositories had no open issue; for many
later-wave modules this meant no implementation program had been opened, not
that the capability was complete.
[Poll #2](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/2) was a clear
tracker-drift example: its configurable transition engine, agreed transition
matrix/history, idempotent keyed retries, re-decision audit, archive/unarchive,
and preservation behavior were implemented and pushed while the issue still
reported `needs-info`.
Issue anchors that informed the bridge from the baseline into this roadmap:
- [Meta #10](https://git.add-ideas.de/GovOPlaN/govoplan/issues/10) for the
capability/infrastructure assessment and its target proof;
- [Meta #11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) for the
universal interface and focused-view direction;
- [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225) for
guided, safe configuration;
- [Core #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29) for the
backup/restore production gate;
- [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263) and
[Campaign #63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63),
[#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62),
[#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65), and
[#69](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/69) for the
reference interface/delivery vocabulary and behavior;
- [Poll #1](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/1) for the
database-enforced respondent invariant exposed by Scheduling;
- [Connectors #6](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/6)
for the governed connector configuration/simulation foundation;
- [Meta #9](https://git.add-ideas.de/GovOPlaN/govoplan/issues/9) for the first
permit-to-payment reference process; and
- [Meta #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12) for the
deliberately deferred, consumer-independent export-control story.
Live Gitea issue state remains canonical. These dated facts explain the roadmap
sequence only.
The volatile release and backlog appendix that originally accompanied this
roadmap has been removed so the durable direction cannot become a competing
status source. The [Strategic Review 2026-08-05](STRATEGIC_REVIEW_2026-08-05.md)
retains the dated assessment and reasoning. Current reconciliation belongs in
[Strategy Status](STRATEGY_STATUS.md), and live work state belongs in Gitea.
+155
View File
@@ -0,0 +1,155 @@
# Federated GovOPlaN Architecture
## Purpose
Federation lets autonomous GovOPlaN installations exchange data,
configuration, work, messages, records, and evidence without sharing a database
or surrendering local policy. It is institution-to-institution cooperation,
not multi-tenancy across an untrusted network.
The first implementation should prove a bounded exchange between two
installations. A new federation module is not justified until the shared
protocol has at least two independent consumers. Core owns neutral envelopes
and trust contracts; Connectors owns transport providers; domain modules own
the objects and effects they exchange.
Implementation is tracked in
[GovOPlaN #41](https://git.add-ideas.de/GovOPlaN/govoplan/issues/41).
## Invariants
1. Every installation remains authoritative for its tenants, identities,
policies, keys, records, and local mappings.
2. A remote identity or permission never becomes a local authorization claim.
3. Every exchange declares purpose, legal/organizational basis, classification,
minimization, retention expectation, and permitted onward use.
4. Every object reference identifies origin instance, owner tenant, object type,
object ID, exact revision, and source-authority mode.
5. Payloads and receipts are signed; sensitive transports use mutually
authenticated encrypted channels.
6. Acceptance, rejection, outcome unknown, retry, revocation, correction, and
reconciliation are durable states.
7. Local policy may reject or narrow a remote request. It cannot silently claim
to have accepted an effect that did not occur.
8. Federation works asynchronously and can exchange signed offline bundles
where continuous connectivity is unavailable.
## Trust Domains
An instance publishes a signed, versioned federation descriptor containing:
- stable instance and operator identity;
- supported protocol and schema versions;
- signing and transport key identifiers with rotation history;
- accepted object and exchange profiles;
- endpoint locations and size/rate limits;
- support, incident, revocation, and data-protection contacts;
- evidence and conformance references.
Pairing is a two-sided administrative workflow. Each side verifies the other,
maps the remote institution to a local trusted-party record, selects permitted
profiles and purposes, sets policy ceilings, and records approvals. Trust is
directional and profile-specific; trusting signed Postbox delivery does not
automatically permit case transfer or configuration import.
## Exchange Envelope
Every request, response, receipt, correction, and revocation uses one neutral
envelope with:
- message ID, correlation ID, causation ID, creation and expiry;
- origin and destination instance/institution/tenant references;
- real actor and represented institutional capacity where disclosure is
permitted;
- exchange profile and semantic schema version;
- exact domain object references and content digests;
- purpose, legal basis, classification, data categories, retention expectation,
onward-transfer constraint, and subject notice status;
- requested action and idempotency key;
- encryption recipients and signature chain;
- attachment/object manifests rather than unbounded embedded blobs;
- previous-envelope references for correction, replacement, or revocation.
The envelope is evidence, not a universal domain object. Each owner validates
and imports or links its own payload.
## Exchange Profiles
| Profile | First owners | Behavior |
| --- | --- | --- |
| Postbox delivery | Postbox, Campaign, Notifications | Address or derive a remote function-bound postbox, obtain acceptance receipt, and track acknowledgement where permitted |
| Case handoff | Cases, Parties, Services, Workflow Engine | Offer exact context and evidence; destination accepts into a new local case and returns the mapping |
| Record transfer | Records, Files, DMS, Audit | Transfer or offer a signed record package with file-plan, metadata, content digests, holds, and disposition constraints |
| Decision/evidence reference | Decisions, Committee, Audit | Publish a protected exact outcome or verifiable reference without transferring unrelated case content |
| Data product publication | Datasources, Dataflow, Reporting | Publish immutable governed materializations with schema, quality, freshness, lineage, and use constraints |
| Configuration package | Core, Policy, Views, Workflow, Forms, Templates | Exchange signed definitions; destination assesses compatibility, maps values, derives locally, and never imports secrets |
| Search discovery | Search and domain providers | Return permission-filtered metadata or a handoff link; never expose raw remote indexes as local authority |
## State Machine
```text
draft -> authorized -> queued -> transmitted -> received
| |
v v
outcome_unknown rejected
|
received -> validating -> accepted -> applied -> acknowledged
| | |
v v v
rejected accepted_ reconciled
pending
```
Acceptance means the destination durably owns the received intent. It does not
mean the requested domain effect completed. Receipts distinguish transport,
validation, acceptance, application, and human acknowledgement.
## Conflict And Autonomy
- Incoming native objects become local references, mirrors, or newly owned
objects according to the profile. They do not overwrite local authority by
ID coincidence.
- Local mappings are effective-dated and auditable.
- Corrections create a linked revision. They do not erase what the destination
previously observed.
- Revocation is a request and evidence event; the destination applies its own
legal and retention rules.
- Configuration imports use assessment and derivation. A remote package cannot
weaken local policy or install code implicitly.
- A disconnected partner remains a visible pending/failed state; work can be
rerouted through an approved alternative channel.
## Security And Privacy
- Use mTLS for paired online transports and signed envelopes for end-to-end
origin evidence.
- Encrypt payload objects for the destination, with key rotation and outcome-
unknown recovery; transport encryption alone is insufficient for queued
bundles.
- Do not put bearer credentials, local permission scopes, or reusable secrets
in an exchange.
- Rate-limit and size-bound discovery and transfer; quarantine unknown schemas
and active content.
- Evaluate current local authorization at every effect even when the envelope
describes historical authority.
- Log metadata separately from protected content so operators can reconcile
without broad content access.
- Subject access, correction, restriction, legal hold, and deletion requests
become federated workflows with local decisions and receipts, not remote
direct database operations.
## First Reference Proof
1. Pair two disposable installations with independent tenants, keys, and
policies.
2. Exchange signed descriptors and approve only the Postbox delivery profile.
3. Deliver one Campaign message to a remote function-bound Postbox.
4. Prove replay safety, rejection, timeout/outcome unknown, retry,
acknowledgement, correction, key rotation, and revoked trust.
5. Export the complete evidence bundle and restore both sides from backup.
6. Add configuration-package exchange only after the delivery proof passes.
The result is a provider-neutral federation contract. A future dedicated
module becomes appropriate only when pairing, trust administration, exchange
queues, and evidence have a lifecycle independent of Connectors and the first
domain owner.
@@ -221,26 +221,39 @@ references and archive hashes; mutable tags or incomplete bundles are rejected.
## Current Production Gates
The tool deliberately reports blockers instead of pretending the source tree is
a production distribution:
The first immutable production-distribution baseline is published as
[`v0.1.14`](https://git.add-ideas.de/GovOPlaN/govoplan/releases/tag/v0.1.14)
from source commit `1f039dd39c1ce2672f4978c8abc6dff862ef1445`. Runtime
Distribution [run #459](https://git.add-ideas.de/GovOPlaN/govoplan/actions/runs/459)
proved migrations, schema compatibility, non-root API/Web readiness, and worker
delivery/shutdown on both `linux/amd64` and `linux/arm64`. Its signed manifest
has SHA-256
`d703267e01855dee63200cb20921c91c3f95fbff550c8ca76e9a35cba3f69109`
and pins these runtime indexes:
1. **First publication.** The protected workflow and fail-closed artifact
contracts are implemented, but a release operator must configure the Gitea
registry/release tokens and runtime Ed25519 key, publish the first pinned
release, and retain its amd64/arm64 readiness evidence.
3. **First administrator.** Production needs a one-time, restricted enrollment
- API: `git.add-ideas.de/govoplan/runtime-api@sha256:197ed01790986f2bc927eaa5d8348fa118702e5d2dc05feb851fc2643c23764a`
- WebUI: `git.add-ideas.de/govoplan/runtime-web@sha256:e936cca124f1fad29a067834cf17627d4c236410fdc3fa129e0ccb26b8193812`
The signed bootstrap has SHA-256
`1ff946fba82b0895d153b23352d06e30fe18388450dfd37fed6fb9912310efc5`
and key id `runtime-distribution-2026-01`. The managed-ingress boundary passed
the same publication run and the independently dispatchable Runtime Ingress
Drill [run #458](https://git.add-ideas.de/GovOPlaN/govoplan/actions/runs/458).
Every later release must renew this evidence; the following target-specific
gates remain:
1. **First administrator.** Production needs a one-time, restricted enrollment
identity. The development bootstrap must not be enabled in production.
4. **Image/module composition.** The deployer now enforces the signed
2. **Image/module composition.** The deployer enforces the signed
composition. A selected module not shipped by that release cannot be
enabled.
5. **Deployment agent.** Web updates need a separate privileged reconciler with
3. **Deployment agent.** Web updates need a separate privileged reconciler with
a typed command allowlist. The API and browser must never receive the Docker
socket or arbitrary shell access.
6. **Ingress reachability evidence.** Managed Caddy ingress and the
4. **Target reachability evidence.** Managed Caddy ingress and the
existing-proxy contract are implemented. A production claim still requires
running `doctor` from the target host after public DNS/firewall changes and
retaining the first successful container drill and public TLS/readiness
evidence.
retaining public TLS/readiness evidence for that deployment.
`apply --allow-unverified-images` is therefore restricted to the evaluation
profile. It explicitly acknowledges both mutable image identities and
+149
View File
@@ -0,0 +1,149 @@
# Institutional Digital Twin
## Definition
The institutional digital twin is a governed, time-aware projection of how an
institution is constituted and operates. It connects structure, authority,
services, work, information, technology, obligations, controls, evidence, and
outcomes without becoming a second source of truth.
The twin is not one editable graph database and not an employee-surveillance
system. Domain modules and external systems keep ownership. The twin stores or
materializes exact references, declared relationships, provenance, confidence,
and projection versions. Changes flow through owner actions.
Implementation is tracked in
[GovOPlaN #43](https://git.add-ideas.de/GovOPlaN/govoplan/issues/43).
## Questions It Should Answer
- Which unit and function is responsible for a service, decision, record,
system, dataset, control, or risk at a given valid and recorded time?
- Which mandates and policies permit or constrain an action?
- Which processes, providers, staff capacities, data sources, and records are
required to deliver a service?
- What is affected if a system, provider, organizational unit, role, package,
or legal rule changes?
- Where are responsibilities missing, conflicting, expired, or concentrated?
- Which controls are evidenced, stale, failed, or dependent on an unverified
assertion?
- How do actual process traces differ from defined workflows?
- Which public outcomes can be explained from protected internal evidence?
## Projection Planes
| Plane | Meaning |
| --- | --- |
| Current | Valid now, reconstructed from owner projections and current provider state |
| Historical | Valid at and recorded by selected instants, with present-day security enforced |
| Planned | Approved or proposed future structures, services, policies, projects, and package changes |
| Observed | Events, process traces, service measures, incidents, effects, and evidence actually recorded |
| Scenario | Non-authoritative simulation of a proposed change and its estimated consequences |
The UI must label these planes unambiguously. Scenario output never becomes an
institutional fact until an authorized owner action accepts it.
## Canonical Graph
Nodes are stable institutional references, including institution, tenant,
unit, function, assignment, mandate, jurisdiction, service, case, party, task,
workflow, approval, decision, record, file, message, appointment, dataset,
report, provider, system, control, risk, project, asset, and configuration
package.
Edges have:
- owner and source authority;
- relationship type and direction;
- valid-from/valid-to and recorded/superseded times;
- exact source revision and evidence digest;
- institution/tenant boundary;
- purpose and visibility classification;
- confidence and derivation method for inferred relationships;
- correction and replacement references.
Inferred edges are never displayed as owner assertions. They remain
explainable analytical products with source lineage.
## Ownership And Implementation
- Core owns neutral institutional references, temporal context, provider
registration, and graph projection contracts.
- Domain modules publish bounded nodes and edges through provider interfaces.
- Search indexes discoverable identities and links.
- Reporting materializes governed analytical projections.
- Dataflow computes derived relationships, quality checks, and scenarios.
- Policy evaluates visibility, purpose, retention, and allowed scenario/action
transitions.
- Audit supplies observed events and evidence references.
- Projects supplies planned change and benefit relationships.
- Views renders role- and task-focused twin perspectives.
- Workflow Engine coordinates accepted changes but does not edit owner tables.
No new digital-twin module is required for the first slice. A dedicated owner
is justified later if persisted scenario models, graph revisions, and
cross-domain projection lifecycle become independent product objects.
## Beyond The Current Platform
### Continuous assurance
Controls become versioned assertions with evidence requirements, evaluation
frequency, responsible function, exception workflow, and freshness. Dataflow
and provider checks evaluate them continuously; Policy decides whether a stale
or failed control advises, requires review, or blocks an effect.
### Process mining and conformance
Governed event histories can derive actual paths, wait times, rework, and
exceptions. Comparison to Workflow definitions should improve procedures, not
rank individuals. Access to personal or small-cohort detail is purpose-limited
and separately governed.
### Change-impact simulation
A proposed organizational, provider, policy, or package change can be assessed
against dependencies, mandates, open work, records, controls, capacity, and
recovery plans before activation. Results identify uncertainty rather than
inventing precision.
### Federated institutional models
Installations can exchange signed public or partner-specific subsets of their
service, mandate, provider, and evidence graph. Every side maps the references
locally and retains autonomy. Federation does not create one supranational
master graph.
### Accountable assistance
Assistance may summarize context, identify missing evidence, draft a decision
or workflow, propose mappings, and explain policy. Every output records model,
inputs, constraints, uncertainty, human review, and accepted edits. Assistance
does not become the acting authority.
### Public evidence chains
Transparency packages can publish a minimized chain from rule and aggregate
facts to decision and observed outcome, with digests proving relation to
protected evidence. Public verification does not require disclosure of the
underlying personal data.
## Guardrails
- Do not infer competence, misconduct, intent, or personal performance from
graph proximity or incomplete events.
- Do not centralize protected content merely to make graph queries easier.
- Do not use historical authorization to expose data now prohibited.
- Do not let a scenario engine write domain state directly.
- Do not hide source authority, freshness, uncertainty, or missing evidence.
- Do not retain analytical detail longer than the declared purpose requires.
## Delivery Slices
1. Publish exact institutional reference/edge providers for the service-to-
decision and monthly-data journeys.
2. Build a current/historical dependency explorer with source and access
explanations.
3. Add planned Project/package changes and bounded impact reports.
4. Add control evidence/freshness and process conformance for one journey.
5. Prove a minimized federated projection and a public evidence package.
@@ -9,13 +9,17 @@ concepts prepared outside the repositories:
- `software_big_picture.md`
The source concepts describe GovOPlaN as an operational governance platform for
public institutions. This document merges that direction with the implemented
platform state as of 2026-08-01. It is the canonical repository version of the
direction. Gitea issues remain the source of truth for delivery state.
public institutions. This document is the canonical repository version of that
durable architectural direction. Its implementation table records the accepted
2026-08-01 baseline; it is not a rolling status report. Current reconciliation
lives in [Strategy Status](STRATEGY_STATUS.md), and Gitea issues remain the
source of truth for delivery state.
Read this together with:
- [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md)
- [Platform Core Ideas](PLATFORM_CORE_IDEAS.md)
- [Strategy Status](STRATEGY_STATUS.md)
- [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md)
- [Module Contracts and Install Boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md)
@@ -70,7 +74,12 @@ compositions described here are now implemented. Subsequent work is
**product depth and stronger maturity evidence**, not another runtime rewrite
or an unimplemented architecture boundary.
## Implementation status (2026-08-01)
## Accepted implementation baseline (2026-08-01)
This section is retained as the dated baseline against which the architecture
decision was accepted. Later implementation must be reconciled in
`STRATEGY_STATUS.md` rather than editing individual rows here into a competing
status report.
The architecture contract is implemented as a bounded, executable vertical
slice. The portfolio declarations and provider governance gates apply to the
@@ -79,7 +88,7 @@ were proven now have independent persistent owners:
| Area | Implemented state | Remaining rollout |
| --- | --- | --- |
| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for all 62 source manifests. Focused and release checks enforce `--require-architecture`; a new module cannot enter the workspace without truthful declaration and repository-local evidence. |
| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for the source manifests in the 2026-08-01 snapshot. Focused and release checks enforce `--require-architecture`; current portfolio counts belong in `STRATEGY_STATUS.md`. |
| External providers | Core validates provider objects/field groups, operations, integration maturity, source authority, bounded reads, freshness/health, idempotency, conflicts, outcome-unknown handling, evidence, correction, reconciliation, outage, classification, purpose, retention, and secret handling. Addresses/CardDAV, Files remote storage, Mail SMTP/IMAP, Calendar CalDAV/ICS/Graph/EWS, and Connectors tabular/sanctions providers declare the contract and tenant-bounded secret-free runtime state. | Registry validation rejects any declared external provider without a sanitized state provider. Future adapters must cross the same gate before activation. |
| Institutional context | Core provides versioned temporal, actor/representation, institution/unit/function/task/mandate/jurisdiction/service/case/party/work-item/workflow/approval/decision/record, legal-basis, evidence, information-governance, external-source, presentation, and geographic references. Events, automation actions, audit records, and the transactional Audit outbox preserve the envelope. | Owning modules must progressively require the relevant subset for consequential operations. |
| Semantic provider contracts | Provider-neutral DTOs and protocols cover Mandate resolution, versioned Service definitions, procedure Parties/representation, and formal Decisions. `govoplan-mandates`, `govoplan-services`, `govoplan-parties`, and `govoplan-decisions` now persist immutable revisions behind those contracts with tenant isolation, bounded reads, replay safety, OCC, migrations, uninstall guards, permissions, APIs, capability documentation, and recovery documentation. | The owners are deliberately headless. Procedure-specific UI remains with consuming modules. |
@@ -391,7 +400,9 @@ submodule, configuration fragment, package, or profile.
- This reconciliation is canonical in the meta repository and mirrored to the
Gitea wiki.
- All 62 source manifests carry validated evidence-based architecture metadata.
- All source manifests in the accepted 2026-08-01 baseline carried validated
evidence-based architecture metadata; current counts belong in
`STRATEGY_STATUS.md`.
- External-reference, action/effect, operational-health, ownership, policy,
audit, and documentation primitives compose into one enforced provider
declaration and sanitized runtime-state contract.
@@ -446,7 +457,7 @@ submodule, configuration fragment, package, or profile.
recovery, accessibility, privacy, security, and operator evidence. This is a
maturity gate, not missing architecture implementation.
## What remains after the executable architecture slice
## What remains within the accepted 2026-08-01 architecture slice
The remaining work is not another Core or cross-module architecture rewrite.
It falls into two explicitly different categories, neither of which can be
@@ -487,12 +498,38 @@ persistence, migrations, recovery/disable semantics, documentation and focused
tests. Their remaining tickets concern concrete providers, deeper adapters and
target evidence, not an unresolved institutional architecture boundary.
Everything else described as architecture in this document now has a
Everything else described in the accepted baseline of this document now has a
repository owner, versioned contract, bounded implementation, migration and
recovery boundary where state exists, documentation, and executable evidence.
Further work in those modules is product breadth, UX depth, provider adoption,
and evidence renewal.
## Strategic extensions accepted after the baseline
The completed baseline does not imply that institutional product architecture
can no longer grow. The 2026-08-05 strategic review accepted four extensions
that consume the existing contracts without reopening the kernel or moving
domain ownership into Core:
- [Product Experience and Module Boundaries](PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md)
separates technical package topology from stable task/object/product
surfaces; implementation is tracked in Core #283.
- [Federated GovOPlaN Architecture](FEDERATED_GOVOPLAN_ARCHITECTURE.md)
defines governed exchange between autonomous installations; implementation
is tracked in GovOPlaN #41.
- [Assisted and Non-Digital Channels](ASSISTED_AND_NON_DIGITAL_CHANNELS.md)
makes channel inclusion part of the service-to-decision journey; the first
reference proof is tracked in GovOPlaN #42.
- [Institutional Digital Twin](INSTITUTIONAL_DIGITAL_TWIN.md) defines a
time-aware, policy-filtered projection over owner data; implementation is
tracked in GovOPlaN #43.
The eAkte depth required by those journeys is owned by Records and specified in
`govoplan-records/docs/EAKTE_ARCHITECTURE.md`, tracked in Records #1. These are
new product-depth programs with bounded contracts and acceptance journeys, not
evidence that the original institutional semantics or module architecture
failed.
## Delivery tracking
The completed cross-repository architecture epic is
+54 -36
View File
@@ -58,12 +58,12 @@ Inventory states:
| Surface | Owner and code evidence | Audience/access evidence | Primary task and target archetype | Audit / rollout |
| --- | --- | --- | --- | --- |
| Public landing and login | `govoplan-core` `PublicLandingPage`; rendered while no authenticated principal exists | Unauthenticated; maintenance and backend-reachability context are shell inputs | Understand the service and authenticate; public entry | Unreviewed; later public-entry audit |
| Session/bootstrap state | `govoplan-core` `App.tsx` and `AppShell` | All browser sessions during bootstrap | Understand that session/platform state is loading; state contract | Unreviewed; core shell |
| `/` authenticated redirect | `govoplan-core` chooses the first visible navigation destination | Authenticated; result depends on visible nav contributions | Enter the actor's first accessible service area; navigation behavior, not a content page | Unreviewed; focused-view/default-route work must preserve this fallback |
| `/dashboard` fallback | `govoplan-core` `DashboardPage` only when the Dashboard module is absent | Authenticated; no route-specific scope in core | Cross-module starting point; dashboard | Unreviewed; compare with module dashboard before shared changes |
| Public landing and login | `govoplan-core` `PublicLandingPage`; rendered while no authenticated principal exists | Unauthenticated; maintenance and backend-reachability context are shell inputs | Understand the service and authenticate; public entry | Core shell contract complete under Core #227: semantic entry/login, uniform reachable/offline/maintenance feedback, keyboard focus, responsive layout and privacy-safe pre-authentication state |
| Session/bootstrap state | `govoplan-core` `App.tsx` and `AppShell` | All browser sessions during bootstrap | Understand that session/platform state is loading; state contract | Core shell contract complete: loading, unreachable, maintenance, authentication-required and module-load failure states use shared status/alert boundaries without erasing the shell |
| `/` authenticated redirect | `govoplan-core` chooses the first visible navigation destination | Authenticated; result depends on visible nav contributions | Enter the actor's first accessible service area; navigation behavior, not a content page | Core route/module-permutation contract complete; permission, module, View and fallback filtering precede navigation and do not execute a domain action |
| `/dashboard` fallback | `govoplan-core` `DashboardPage` only when the Dashboard module is absent | Authenticated; no route-specific scope in core | Cross-module starting point; dashboard | Core fallback and Dashboard module permutations complete; fallback remains usable without the optional Dashboard module |
| `/settings` | `govoplan-core` `SettingsPage` | Authenticated; contributed sections and integrations filter internally | Profile, UI/workspace preference, local connection, and user-scoped integration settings; configuration | Core-owned pattern migration complete in [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225), commit `fa32cca` |
| Shell chrome | `AppShell`, `Titlebar`, `IconRail`, `BreadcrumbBar`, `HelpMenu`, language menu, unsaved-change provider | Public/authenticated variants; nav filtered later | Tenant/actor context, global navigation, help, language, session and maintenance state | Unreviewed; platform-owned prerequisite for focused views |
| Shell chrome | `AppShell`, `Titlebar`, `IconRail`, `BreadcrumbBar`, `HelpMenu`, language menu, unsaved-change provider | Public/authenticated variants; nav filtered later | Tenant/actor context, global navigation, help, language, session and maintenance state | Core shell contract complete under Core #227/#225 and Views #2: semantic global controls, scroll-safe rail, visible maintenance state, guarded navigation, configured Docs fallback, optional Search, responsive/theme/i18n checks and module permutations |
## Direct Module Route Contributions
@@ -78,15 +78,15 @@ semantics as authenticated navigation routes.
| `/address-book` | Addresses | `addresses:contact:read` | Governed source directory, contact/list detail, external-provider operation, governance facts, and reversible correction | Addresses pattern migration complete in [Addresses #23](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/23), commit `f9a7185` |
| `/approvals` | Approvals | `approvals:workspace:read` | Work queue/guided decision | Approvals pattern migration complete in [Approvals #3](https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/3), commit `24e9559` |
| `/calendar` | Calendar | `calendar:event:read` | Full-height calendar workspace with filterable collection/agenda sidebar, continuous and bounded date views, guarded VEVENT and source editors, synchronized-source status, durable outbox recovery, and destructive remote-move evidence | Calendar pattern migration complete in [Calendar #22](https://git.add-ideas.de/GovOPlaN/govoplan-calendar/issues/22), commit `d7fd944` |
| `/campaigns`, `/campaigns/:campaignId/*`, `/campaigns/queue`, `/campaigns/reports`, `/templates` | Campaign | Campaign read/report/control scopes; template route has no route guard | List-detail, guided review, monitoring, reporting | [Campaign #74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74) |
| `/operator` | Campaign | Campaign read plus queue/control scope | Compatibility redirect to `/campaigns/queue` | Campaign #74; retire under the compatibility policy |
| `/campaigns`, `/campaigns/:campaignId/*`, `/campaigns/queue`, `/campaigns/reports` | Campaign | Campaign read/report/control scopes | List-detail, guided review, monitoring, reporting | Campaign pattern pilot complete in [Campaign #74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74); bounded product features such as watched-folder policy remain independently tracked |
| `/operator` | Campaign | Campaign read plus queue/control scope | Compatibility redirect to `/campaigns/queue` | Campaign #74 complete; redirect remains declared for saved links and is retired under the compatibility policy rather than through the UI migration |
| `/cases`, `/cases/:caseId` | Cases | `cases:case:read` | Governed case directory and detail workspace with guarded OCC lifecycle editor, provider-owned references, immutable timeline/history, and confirmed object-access editor | Cases pattern migration complete in [Cases #4](https://git.add-ideas.de/GovOPlaN/govoplan-cases/issues/4), commit `43b4cc8` |
| `/committee` | Committee | `committee:workspace:read` | Governed workspace | Committee pattern migration complete in [Committee #2](https://git.add-ideas.de/GovOPlaN/govoplan-committee/issues/2), commit `e64af30` |
| `/dashboard` | Dashboard | No route-specific scope | View-specific personal workspace with module/permission-filtered widget library, guarded four-column composition, nested widget settings, server/browser fallback, and optimistic layout persistence | Dashboard pattern migration complete in [Dashboard #3](https://git.add-ideas.de/GovOPlaN/govoplan-dashboard/issues/3), commit `da3947f` |
| `/dataflow` | Dataflow | Pipeline read/admin | Governed library, guarded graph/constrained-SQL definition editor, typed node inspector, bounded intermediate preview, automation triggers, and durable run/deployment evidence | Dataflow pattern migration complete in [Dataflow #20](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/20), commit `109ddcd` |
| `/datasources` | Datasources | Catalogue read/source admin | Governed catalogue, staging preflight, optional-origin directory, authority editor, and immutable evidence | Datasources pattern migration complete in [Datasources #7](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/7), commit `6406ce7` |
| `/distribution-lists` | Distribution Lists | List read/write/admin | Governed directory, immutable-revision editor, expansion preview, and evidence register | Distribution Lists pattern migration complete in [Distribution Lists #8](https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists/issues/8), commit `6cdd804` |
| `/docs` | Docs | Documentation or settings read | Documentation/reference | [Docs #15](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/15) |
| `/docs` | Docs | Documentation or settings read | Documentation/reference | Configured-system workflow/reference/pattern help complete in [Docs #15](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/15), commit `abe2f78` |
| `/files` | Files | `files:file:read` | Directory/explorer | Files pattern migration complete in [Files #42](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/42), commit `d8ae506` |
| `/forms` | Forms | `forms:definition:read` | Definition library/editor | Forms pattern migration complete in [Forms #4](https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/4), commit `e505536` |
| `/forms-runtime`, `/forms-runtime/:instanceId` | Forms Runtime | Participate or workspace read | Guided form execution | Forms Runtime pattern migration complete in [Forms Runtime #5](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/5), commit `07dd35b` |
@@ -95,17 +95,36 @@ semantics as authenticated navigation routes.
| `/notifications` | Notifications | `notifications:notification:read` | Inbox/list-detail with guarded recipient state, confirmed local cancellation/dispatch, and sanitized delivery evidence | Notifications pattern migration complete in [Notifications #4](https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/4), commit `ad6a31f` |
| `/ops` | Ops | Operations or settings read | Monitoring/evidence with contextual run, drain, readiness-blocker, and recovery guidance | Ops pattern migration complete in [Ops #4](https://git.add-ideas.de/GovOPlaN/govoplan-ops/issues/4), commit `2b32643` |
| `/organizations` | Organizations | Model/unit/function or settings read | Directory/hierarchy editor | Organizations pattern migration complete in [Organizations #7](https://git.add-ideas.de/GovOPlaN/govoplan-organizations/issues/7), commit `97acfcb` |
| `/portal` | Portal | `portal:service:read` | Service portal | [Portal #2](https://git.add-ideas.de/GovOPlaN/govoplan-portal/issues/2) |
| `/portal` | Portal | `portal:service:read` | Explained service directory and governed handoff | Portal pattern migration complete in [Portal #2](https://git.add-ideas.de/GovOPlaN/govoplan-portal/issues/2); durable evidence in `govoplan-portal/docs/INTERFACE_PATTERN_MIGRATION.md` |
| `/postbox` | Postbox | `postbox:postbox:read` | Inbox/list-detail | Postbox pattern migration complete in [Postbox #26](https://git.add-ideas.de/GovOPlaN/govoplan-postbox/issues/26), commit `a97eb3b` |
| `/projects` | Projects | `projects:project:read` | List-detail/project workspace | [Projects #2](https://git.add-ideas.de/GovOPlaN/govoplan-projects/issues/2) |
| `/reporting`, `/reports` | Reporting | `reporting:definition:read` | Reporting/definition library | [Reporting #8](https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/8) |
| `/projects` | Projects | `projects:project:read` | Revisioned list-detail/project workspace | Projects pattern migration complete in [Projects #2](https://git.add-ideas.de/GovOPlaN/govoplan-projects/issues/2); durable evidence in `govoplan-projects/docs/INTERFACE_PATTERN_MIGRATION.md` |
| `/reporting`, `/reports` | Reporting | `reporting:definition:read` | Governed report catalogue, analytical workspace and evidence | Reporting pattern migration complete in [Reporting #8](https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/8); durable evidence in `govoplan-reporting/docs/INTERFACE_PATTERN_MIGRATION.md` |
| `/risk-compliance` | Risk Compliance | Workspace or sanctions read | Immutable source evidence, version-pinned screening, list-detail review, and revisioned assurance graph with explicit blockers and consequences | Risk Compliance pattern migration complete in [Risk Compliance #8](https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance/issues/8), commit `24d80a6` |
| `/scheduling` | Scheduling | `scheduling:schedule:read` | List-detail/guided decision | Scheduling pattern migration complete in [Scheduling #8](https://git.add-ideas.de/GovOPlaN/govoplan-scheduling/issues/8), commit `c17cbda` |
| `/scheduling/public/:requestId/:token` | Scheduling | Public signed token | Public participation | Scheduling #8 complete in `c17cbda` |
| `/search` | Search | `search:result:read` | Search overlay/results | [Search #4](https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/4) |
| `/search` | Search | `search:result:read` | Keyboard-first global/context overlay and full results fallback | Search pattern migration complete in [Search #4](https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/4); durable evidence in `govoplan-search/docs/INTERFACE_PATTERN_MIGRATION.md` |
| `/templates` | Templates | Template read/write/publish/render/admin | Governed library, immutable-revision editor, compatibility preview, and render evidence | Templates pattern migration complete in [Templates #5](https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/5), commit `72fafa2` |
| `/voting` | Voting | `voting:ballot:read` | Governed ballot workspace | Voting pattern migration complete in [Voting #1](https://git.add-ideas.de/GovOPlaN/govoplan-voting/issues/1), commit `2625990` |
| `/workflow` | Workflow | Definition read or instance admin | Graph editor/execution evidence | [Workflow #15](https://git.add-ideas.de/GovOPlaN/govoplan-workflow/issues/15) |
| `/workflow` | Workflow | Definition read or instance admin | Native BPMN editor, governed revision actions and execution evidence | Workflow pattern migration complete in [Workflow #15](https://git.add-ideas.de/GovOPlaN/govoplan-workflow/issues/15); durable evidence in `govoplan-workflow/docs/INTERFACE_PATTERN_MIGRATION.md` |
## Final Module Closure Evidence
The final five module-owned work packages complete the 2026-08-03 rollout
snapshot. Their module documents are the durable detailed inventories; the
table below records the cross-product closure evidence.
| Owner | Dominant archetype and consequential boundary | Focused evidence |
| --- | --- | --- |
| Workflow | Definition list-detail plus specialized native BPMN editor; save/activate/archive/delete/reset and instance transitions remain revisioned, confirmed, and Engine-owned | Shared dialogs/status/alerts/help, dirty-navigation guard, keyboard palette insertion, edge inspector alternative, responsive/reduced-motion contract, TypeScript and focused structure test |
| Search | Focus-contained global/context overlay plus URL-stable full results; filters only narrow permission-aware source results | F3/Ctrl/Cmd+K, listbox keyboard navigation, provider-partial diagnostics, shared controls/help, narrow layout and focused overlay/interface tests |
| Reporting | Three-region governed analytical workspace; runs, schedules, exports and publications retain purpose, permission, source and policy provenance | Shared grid/dialog/status/help, keyboard-explainable Run blockers, responsive task order, provider/semantic backend tests and focused interface test |
| Projects | Revisioned list-detail planning workspace; visibility and saves are ACL/OCC-governed and retain a change reason | Shared dialog/status/help/field labels, save errors attached to the editor, semantic list controls, responsive/focus contract and focused interface test |
| Portal | Explained service directory and exact-revision provider handoff; Portal never owns the launched case/form/workflow effect | Shared status/alert/toggle/help/blocker controls, stable disabled Open action with actor/action/destination, guarded navigation, responsive layout and focused interface test |
Future WebUI modules and newly added routes are not grandfathered by this
snapshot. They must meet the same surface definition of done in their owning
feature issue and pass the source/runtime inventory gates; they do not reopen
this finite migration program unless the pattern contract itself changes.
## Manifest And Runtime Route Alignment
@@ -155,10 +174,10 @@ enabled and the actor passes the declared filters.
| `/settings` | Core host | Profile; interface; workspace; local connection | Personal configuration with adaptive forms and immediate feedback | Pattern migration complete in Core #225 (`fa32cca`) |
| `/settings` | Files and Mail named capabilities | User-scoped file connections and mail profiles/policy | Optional integration regions disappear cleanly when capability absent | Files #42, Mail #20 and Core #225 complete |
| `/admin` and `/settings` | `govoplan-views` `admin.sections`, `settings.sections`, and `views.runtime` | System/tenant definition and assignment editors, personal/group editors, global selector | Versioned presentation projection with inheritance, lockout safeguards, optional directory targets, and no authorization effect | Pattern migration, contextual help, localized selector/editor, guarded drafts, explained inherited/permission/capability states, and focused evidence complete in Views #2 (`c125f33`) |
| `/settings` | `govoplan-notifications` `settings.sections` | Notification preferences | Personal configuration | Unreviewed |
| `/dashboard` | Dashboard host and `dashboard.widgets` | Installed-modules widget; Ops health widget when Ops contributes it | Widget ordering, staleness, permissions, destination behavior | Unreviewed |
| `/settings` | `govoplan-notifications` `settings.sections` | Notification preferences | Personal configuration | Pattern migration, contextual help, permission/target explanation, typed toggles and focused evidence complete in Notifications #4 (`ad6a31f`) |
| `/dashboard` | Dashboard host and `dashboard.widgets` | Installed-modules widget; Ops health widget when Ops contributes it | Widget ordering, staleness, permissions, destination behavior | Pattern migration, view-aware composition, keyboard/drag alternatives, responsive packing, module filtering and focused evidence complete in Dashboard #3 (`da3947f`) |
| `/organizations` | IDM `organizations.functionActions` | Action leading to assignment view filtered by IDM scopes | Cross-module context action through explicit capability | IDM pattern migration complete in IDM #12 (`d864317`) |
| Campaign attachments/import | Files `files.fileExplorer` | Folder tree, managed chooser, file listing/pattern resolution/sharing | Optional domain composition without sibling-private imports | Pilot audit under Campaign #74 |
| Campaign attachments/import | Files `files.fileExplorer` | Folder tree, managed chooser, file listing/pattern resolution/sharing | Optional domain composition without sibling-private imports | Campaign #74 pilot complete; watched-folder and duplicate-attachment product policy remain independent Campaign #60/#61 features |
| Campaign review/send | Mail runtime `mail.devMailbox` | Mock-mail verification when backend advertises runtime capability | Optional review stage with unavailable/optional states | Explicit intervention and review-progress vocabulary delivered in Campaign #63; send modes/progress delivered in #62/#79 |
Other named capability exports (`files.connectors`, `organizations.functionPicker`,
@@ -262,26 +281,25 @@ prove that the composition or states satisfy the pattern.
| Surface / code evidence | Primary task | Target pattern | Material consequence/state | Known issue / rollout |
| --- | --- | --- | --- | --- |
| Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | Audit in [#74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74); guided entry [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) |
| Overview (`CampaignOverviewPage`) | Understand/edit campaign identity, version, access, lifecycle | Object overview plus adaptive edit | Lock/archive/delete/access changes need real consequence and reversibility wording | #74 remaining audit |
| Fields (`CampaignFieldsPage`) | Define recipient/template field schema | Structured editor | Schema changes can invalidate recipient/template data | #74 audit |
| Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74; attachment-detail [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) |
| Recipients (`RecipientDataPage`) | Select/import/map/edit recipients, address fields and per-recipient values/files | Import/mapping plus list-detail editor | Personal data, validation, bulk activation, file links | Consolidated editor delivered in #67; #74 remaining audit and guided entry #35 |
| Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74; stable overlay [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) |
| Mail settings (`MailSettingsPage` settings view) | Select/configure campaign mail transport | Adaptive configuration | Credentials, SMTP/IMAP destinations, test outcomes | #74; align with Core #225 mail pattern |
| Campaign settings (`GlobalSettingsPage` settings view) | Configure campaign behavior | Adaptive configuration | Can alter validation/build/send behavior | #74 audit |
| Mail policy (`MailSettingsPage` policy view) | Inspect/override effective mail policy | Effective policy/provenance editor | Inheritance and locks affect allowed delivery | #74; Core #225 policy pattern |
| Campaign policy (`GlobalSettingsPage` policy view) | Inspect/override campaign policy | Effective policy/provenance editor | Inheritance, actor authority, and blocked edits | #74; Core #225 policy pattern |
| Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, bounded synchronous execution, persisted queue mode, partial effects, retries, evidence | Blocking/non-blocking interventions and reviewed/remaining evidence delivered in [#63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63); bounded synchronous and explicit/persisted queued modes delivered in [#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62) and [#79](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/79); #74 audit remains |
| Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | #74 and guided entry [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) complete |
| Overview (`CampaignOverviewPage`) | Understand/edit campaign identity, version, access, lifecycle | Object overview plus adaptive edit | Lock/archive/delete/access changes expose consequence, reversibility, owner/access and lifecycle evidence | #74 complete; lifecycle policy is independently extended in Campaign #26 |
| Fields (`CampaignFieldsPage`) | Define recipient/template field schema | Structured editor | Schema changes can invalidate recipient/template data | #74 complete |
| Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74 and attachment-detail [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) complete |
| Recipients (`RecipientDataPage`) | Select/import/map/edit recipients, address fields and per-recipient values/files | Import/mapping plus list-detail editor | Personal data, validation, bulk activation, file links | Consolidated editor #67, guided entry #35 and #74 audit complete; independent bulk action #68 remains product scope |
| Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74 and stable overlay [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) complete |
| Mail settings (`MailSettingsPage` settings view) | Select/configure campaign mail transport | Adaptive configuration | Credentials, SMTP/IMAP destinations, test outcomes | #74 and Core #225 shared mail pattern complete; final credential hierarchy remains Mail #10 |
| Campaign settings (`GlobalSettingsPage` settings view) | Configure campaign behavior | Adaptive configuration | Can alter validation/build/send behavior | #74 complete |
| Mail policy (`MailSettingsPage` policy view) | Inspect/override effective mail policy | Effective policy/provenance editor | Inheritance and locks affect allowed delivery | #74 and Core #225 effective-policy pattern complete |
| Campaign policy (`GlobalSettingsPage` policy view) | Inspect/override campaign policy | Effective policy/provenance editor | Inheritance, actor authority, and blocked edits | #74 and Core #225 effective-policy pattern complete |
| Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, bounded synchronous execution, persisted queue mode, partial effects, retries, evidence | Interventions #63, send/progress #62/#79 and #74 wording/accessibility audit complete |
| Message and attachment detail overlays | Inspect one built/mock message and its attachment links | Stable detail/review dialog | Personal data, exact outbound content, reviewed state | Delivered and verified in [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) and [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) |
| Campaign report (`CampaignReportPage`) | Filter and inspect delivery outcomes | Reporting/list-detail | Partial, failed, explicitly excluded/skipped, SMTP/IMAP outcomes and retries | Server-owned filtering and counts delivered in [#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65) with the full-result DataGrid contract from [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263); excluded semantics in [#66](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/66) |
| Audit (`CampaignAuditPage`) | Inspect campaign evidence/history | Provenance timeline/report | Actor/action/effect trace | #74 audit |
| JSON (`CampaignJsonView`) | Inspect expert representation | Advanced diagnostics/reference | Raw data may contain personal/configuration values; not a primary editor | #74 privacy/redaction audit |
| Audit (`CampaignAuditPage`) | Reach campaign evidence/history | Explained provenance handoff | Campaign emits platform evidence; Audit owns reading, retention and bundles | #74 complete as an explicit Audit handoff; object-scoped projection may follow Audit #3 without a sibling-private import |
| JSON (`CampaignJsonView`) | Inspect/download expert representation | Advanced diagnostics/reference | Full authorized configuration may contain personal data but no inline transport secrets | #74 privacy audit complete with explicit sensitivity warning and campaign-read boundary |
| Create wizard (`CreateWizard`) | Seed a campaign through basics, sender, fields, recipients, template, attachments, review, send | Guided setup | Current steps mix creation and later consequential delivery; completion semantics need audit | Guided first campaign [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) |
| Review/send wizard routes | Alternate guided review/send shells | Guided review | Tracked routes exist; implementation relationship to `ReviewSendPage` must be established, not guessed | #74 inventory decision |
| Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Campaign/version/job identity, historical active-version discovery, fixed action positions, authority-aware disabled states, exact non-overlapping queue counts, server-paged jobs, bounded refresh, retry/queue/reconcile per version, campaign-wide pause/resume/cancel, and leave/return progress | Durable operator controls delivered in [#78](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/78); #74 wording/accessibility audit remains |
| Review/send wizard routes | Focus the canonical review or send stage | Guided review | Thin wrappers render the same `ReviewSendPage` with a stable initial stage; no parallel workflow state exists | #74 inventory decision complete |
| Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Campaign/version/job identity, historical active-version discovery, fixed action positions, authority-aware disabled states, exact non-overlapping queue counts, server-paged jobs, bounded refresh, retry/queue/reconcile per version, campaign-wide pause/resume/cancel, and leave/return progress | Durable controls #78 and #74 wording/accessibility audit complete |
| Aggregate reports (`AggregateReportsPage`) | Compare cross-campaign delivery outcomes | Privacy-preserving aggregate reporting | Tenant/campaign ACL, deployment/tenant small-cell policy, complementary and overlapping-cell suppression, explicit denominator, and no recipient detail/diagnostics/export/drill-down | Separate aggregate-reader surface delivered in [#80](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/80); not parity with the permission-gated per-campaign detail report |
| Templates route (`TemplatesPage`) | Browse template records | Directory/list-detail | Template availability and later generated outputs | #74 audit; verify missing route guard intent |
The five review stages currently named in code are `Validate and inspect`,
`Build and review`, `Mock send and verify`, `Confirm and send`, and `Delivery
@@ -313,7 +331,7 @@ make every module symmetrical.
| Order | Scope | Current evidence | Target | Owner / issue | Verification gate | Status |
| --- | --- | --- | --- | --- | --- | --- |
| 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) | Docs links/diff checks; issue/wiki sync after integration | Initial slice in this document |
| 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) | Reviewed route/component inventory, module documents, manifest shapes and focused contracts | Complete 2026-08-03 |
| 1 | Campaign baseline integration | Recipient-editor WIP and tracker state have been reconciled with remote `main` | Integrated, testable baseline before migration claims | Campaign #67 and tracker cleanup | Backend and focused WebUI suites; issue evidence | Complete 2026-07-22 |
| 2 | Campaign previews/details | Stable shared dialog with bounded scrolling and fixed responsive preview workspace | Stable header/body/footer, accessible long-content detail | Campaign #59 and #73 | Review-preview and overlay structure tests | Complete 2026-07-22 |
| 3 | Campaign review/interventions | Five domain-owned stages use central blocker and guided-review primitives; validation/build warnings name action, actor, and destination; hard blockers, individual review, and group review remain distinct; reviewed/remaining counts survive reload through build-bound review evidence | Clear stages, outcomes, blockers, next actor/action, reviewed evidence | Campaign #63 | `reviewProgress` state tests, shared-component structure contract, TypeScript build, configured-system help topic, and Campaign documentation tests | Complete 2026-08-03 (`d635f3a`; Core primitives and contextual help `b823a22`) |
@@ -322,12 +340,12 @@ make every module symmetrical.
| 6 | Campaign operator recovery | A durable campaign/version queue page exposes historical work, exact non-overlapping state counts, persisted mode, permission-safe controls, server-paged job evidence, bounded refresh and active-state recovery | Fixed-position actions, disabled explanations, leave/return state, version-scoped retry/queue/reconcile and explicit campaign-wide pause/resume/cancel | Campaign #78 | Queue model/structure, historical-version, permission, paging, recovery-control, stale-response and delta tests | Complete 2026-07-22 (`21f3014`, `99d44ee`, `735e874`) |
| 7 | Campaign aggregate reports | A separate aggregate-reader projection and UI expose only policy-suppressed business totals with a stable status domain | Explicit denominator and exclusions, deployment floor plus tenant-strengthened small-cell threshold, complementary and overlapping-cell suppression, no detail/export/diagnostics | Campaign #80 | Aggregate query, cross-metric suppression, route/role/ACL, stable filter and UI structure tests | Complete 2026-07-22 (`06125cc`, `fc36aee`, `8ee87b7`, `ac3329c`, `1225802`) |
| 8 | Campaign excluded outcomes | Excluded build rows become explicit skipped transport outcomes and remain protected from queue/cancel/retry ambiguity | One durable source-to-job-to-report meaning with guarded historical normalization | Campaign #66 | Builder/persistence, migration, query/count, queue-control and report-explanation tests | Complete 2026-07-22 (`7229fb8`) |
| 9 | Guided first campaign | Existing wizard routes and ordinary workspace overlap | Task-oriented entry that hands off clearly to normal editing/review | Campaign #35 | First-run flow, resume/back, validation, optional modules, no implicit send | P1 after core pilot patterns stabilize |
| 9 | Guided first campaign | Eight-stage creation flow persists current step/draft and hands off to ordinary review/delivery preparation | Task-oriented entry that hands off clearly to normal editing/review | Campaign #35 | First-run flow, resume/back, partial validation, immutable-history and optional-module behavior, no implicit send | Complete 2026-07-30 |
| 10 | Prove/extract generic primitives | Shared consequence, focus, help, blocker, unsaved-change, confirmation, connection-tree and effective-policy contracts now have Core and multiple module consumers | Keep Core behavior-only and leave domain composition in owning modules | Core #225 plus bounded follow-ups | Core behavior/accessibility tests and module-permutation tests | Complete 2026-08-03 (`fa32cca`; Files `d8ae506`; Mail `7844d9c`) |
| 11 | Configured-system pattern help | Role/config-aware workflow, reference, pattern, and system topics are projected by Docs; shared route, field, blocker, and action links resolve to configured Docs or the hosted fallback | Stable configured-system guidance without feature-to-Docs imports | Docs #15 | Docs suite, shared component tests, Campaign review tests, 46 module permutations, full-product bundle budget | Complete 2026-08-03 (Docs `abe2f78`; Core `b823a22`; Campaign `d635f3a`) |
| 12 | Admin/configuration family | Core host/settings/credential/retention contracts, shared primitives, module lifecycle, Files, Mail, Policy, Access, Admin, Tenancy, Views, and Organizations are integrated and verified | Continue the same consequence/provenance grammar only through bounded module-owned migrations | Core #225 and module children | Per-surface state/accessibility/consequence evidence | Core #225 complete `fa32cca`; Access `1409dbf`; Files `d8ae506`; Mail `7844d9c`; Policy `f964ed7`; Admin `d428f33`; Tenancy `e76fe16`; Views `c125f33`; Organizations `97acfcb` |
| 13 | Remaining module surfaces | 33 bounded module-owned issues cover every WebUI contributor not already tracked by Campaign #74 or completed Docs #15 | Per-module audit and migration, ordered by user task and consequence rather than a bulk rewrite | Issues linked in the direct-route and composed-surface sections | Module-focused tests, manifest shapes, contextual Docs, and applicable definition-of-done gates | Scheduling `c17cbda`, Audit `6d3fcc1`, Access `1409dbf`, Files `d8ae506`, Mail `7844d9c`, Policy `f964ed7`, Admin `d428f33`, Tenancy `e76fe16`, Views `c125f33`, Organizations `97acfcb`, Postbox `a97eb3b`, IDM `d864317`, Committee `e64af30`, Approvals `24e9559`, Forms Runtime `07dd35b`, Forms `e505536`, Voting `2625990`, Distribution Lists `6cdd804`, Templates `72fafa2`, Addresses `f9a7185`, Datasources `6406ce7`, Dataflow `109ddcd`, Dashboard `da3947f`, Cases `43b4cc8`, Calendar `d7fd944`, Ops `2b32643`, Notifications `ad6a31f`, and Risk Compliance `24d80a6` complete |
| 14 | Manifest/runtime alignment | Several executable routes are absent from manifest metadata | Declared alignment or explicit validated exception | Core contract issue to create | Automated manifest/module route check and configured Docs verification | Discovery follow-up |
| 13 | Remaining module surfaces | 33 bounded module-owned issues cover every WebUI contributor not already tracked by Campaign #74 or completed Docs #15 | Per-module audit and migration, ordered by user task and consequence rather than a bulk rewrite | Issues linked in the direct-route and composed-surface sections | Module-focused tests, manifest shapes, contextual Docs, and applicable definition-of-done gates | Complete: prior 28 recorded commits plus Workflow #15, Search #4, Reporting #8, Projects #2 and Portal #2 verified 2026-08-03 |
| 14 | Manifest/runtime alignment | Authenticated canonical routes align; public signed-token and compatibility routes are explicit exceptions | Stable declarations reconcile with source and any effective runtime module combination | [Meta #25](https://git.add-ideas.de/GovOPlaN/govoplan/issues/25) | Strict duplicate/stale/undeclared declaration CI, per-module digests, and authorized read-only runtime inventory | Complete 2026-08-04 |
Workflow remains outside this rollout matrix because it has its own runtime and
editor workstream, not because it is postponed. Focused views can be specified,
+340
View File
@@ -0,0 +1,340 @@
# Kubernetes VM Test Lab
`tools/lab/govoplan-lab.py` creates and operates an amd64 Ubuntu/K3s test
environment on local or SSH-accessible libvirt hypervisors. It provides the
commands requested for the complete VM lifecycle:
| Command | Effect |
| --- | --- |
| `doctor` | Validate the strict inventory and, with `--online`, every hypervisor. |
| `create --apply` | Download checksum-pinned cloud images, create VM overlays and boot the declared VMs. |
| `deploy --apply` | Verify the signed GovOPlaN release, deploy shared state, install pinned K3s and apply GovOPlaN. |
| `update --apply` | Pull newly pinned state images, update K3s serially and roll the selected GovOPlaN release. |
| `status` | Show libvirt VM state, Kubernetes nodes and GovOPlaN pods. |
| `pause --apply` | Gracefully shut down workers, control planes and shared state while retaining disks. |
| `resume --apply` | Start the retained environment in dependency order and wait for readiness. |
| `verify` | Collect sanitized live-cluster evidence and optionally perform the API-pod-loss drill. |
| `destroy --apply --confirm <lab>` | Delete only the lab-owned domains and overlays; local evidence is retained by default. |
Every mutating command is a dry run unless `--apply` is present. Destruction
also requires the exact lab name. Generated credentials, CA keys, manifests and
evidence are written below the configured `state_directory` with owner-only
permissions. Keep that directory outside the repository and include it in the
workstation backup policy. Existing domains are reused or removed only when
their GovOPlaN ownership description and both expected lab disk paths match.
## What The Lab Proves
The supplied inventories describe two different assurance levels:
- `tools/lab/govoplan-lab.example.toml` creates four VMs on one libvirt host.
It is suitable for development, deployment rehearsal, migration testing,
application-pod replacement and recovery-tool exercises. It cannot close
GovOPlaN #27 because one physical host remains one failure domain.
- `tools/lab/govoplan-lab.acceptance.example.toml` places the two workers on
different hypervisors and puts the control and state VMs on a third. It can
produce the bounded stateless application-tier evidence required by #27 when
the declared hypervisors are genuinely independent physical failure domains.
Both examples use one control-plane VM and one state VM. This keeps the bounded
#27 target economical, but it does not prove control-plane or state-service
high availability. For control-plane failover, declare exactly three control
nodes on independent hosts. PostgreSQL, Redis and object-storage failover must
be tested against independently operated HA services; the lab's single state
VM is intentionally a replaceable integration fixture.
Approximate minimum capacity for the four-VM profile is 10 vCPUs, 16 GiB RAM
and 192 GiB of thin-provisioned disk. A six-VM profile with three controls needs
additional capacity. Do not overcommit memory on an acceptance target.
## 1. Prepare The Hypervisors
On each Ubuntu/Debian libvirt host:
```bash
sudo apt-get update
sudo apt-get install -y \
qemu-kvm libvirt-daemon-system libvirt-clients virtinst cloud-image-utils curl
sudo systemctl enable --now libvirtd
```
Use a dedicated lab-administration account. Remote hypervisors are managed over
SSH and the lifecycle invokes `sudo -n` there, so that account needs bounded
non-interactive permission for libvirt, image and cloud-init operations.
`NOPASSWD: ALL` is acceptable only on isolated lab hypervisors.
On a local hypervisor, put the workstation account in the `libvirt` group and
point `vm_image_directory` at a directory writable by that account and
traversable by `libvirt-qemu`. The lifecycle connects explicitly to
`qemu:///system` and does not require passwordless local sudo. Log out and back
in after a new group assignment before running `doctor --online`. Create the
configured image directory before running the doctor; it deliberately rejects
a missing or non-writable storage root instead of silently falling back to a
different filesystem.
Create a dedicated SSH key on the management workstation:
```bash
ssh-keygen -t ed25519 -f "$HOME/.ssh/govoplan-lab" \
-C "GovOPlaN Kubernetes lab"
```
Install its public key for every remote hypervisor account. The same public key
is injected into the VMs. The lifecycle keeps its own `ssh_known_hosts` file,
uses `accept-new` for first contact, and rejects changed host keys until a
lab-owned VM is deliberately recreated.
### Network contract
The configured `bridge` must exist on every selected hypervisor. All VM
addresses are static. Reserve them outside DHCP allocation and ensure that the
management workstation can route directly to every VM address; the lifecycle
does not tunnel VM traffic through the hypervisor SSH connection.
Permit only these flows inside the lab network:
| Port | Source and destination | Purpose |
| --- | --- | --- |
| TCP 22 | management workstation to every VM/hypervisor | Provisioning and evidence collection |
| TCP 6443 | all K3s nodes and management path to controls | Kubernetes API |
| UDP 8472 | K3s node to K3s node | Default Flannel VXLAN; never expose publicly |
| TCP 10250 | K3s node to K3s node | Kubelet metrics and API |
| TCP 2379-2380 | control to control, only with three controls | Embedded etcd |
| TCP 80/443 | test clients to K3s nodes | Traefik/ServiceLB ingress |
| TCP 5432/6379/9443 | K3s nodes to the state VM | PostgreSQL, Redis and TLS-protected Garage S3 |
| TCP 3025/3143 | approved test clients/workers to the state VM | GreenMail SMTP/IMAP test endpoints |
The official
[K3s networking requirements](https://docs.k3s.io/installation/requirements#networking)
remain authoritative. Restrict state ports to the lab network even though the
generated integration stack binds them on the state VM.
## 2. Create The Inventory
Start with the one-host rehearsal:
```bash
install -d -m 0700 "$HOME/.config/govoplan/labs"
cp tools/lab/govoplan-lab.example.toml \
"$HOME/.config/govoplan/labs/development.toml"
chmod 0600 "$HOME/.config/govoplan/labs/development.toml"
```
Edit at least the bridge, network, static addresses and SSH key paths. For a
multi-host run, copy the acceptance example and replace every example hostname,
failure-domain declaration and network value. Strict parsing rejects unknown
keys, mutable HTTP inputs, malformed checksums, duplicate addresses/MACs and an
acceptance inventory that collapses workers onto one declared hypervisor or
failure domain.
Cloud image, K3s binary, K3s installer and GovOPlaN release inputs are URL plus
SHA-256 pairs. Updating means changing those reviewed pins and then running the
`update` command; the tool deliberately does not follow `latest` aliases.
The one-host example uses the dedicated `govoplan-lab` NAT network. Its DHCP
pool ends at `192.168.123.99`; the static lab addresses start at
`192.168.123.201`. Define and start it once on the local hypervisor:
```bash
virsh --connect qemu:///system net-define \
tools/lab/libvirt/govoplan-lab-network.xml
virsh --connect qemu:///system net-autostart govoplan-lab
virsh --connect qemu:///system net-start govoplan-lab
```
Re-running those commands is unnecessary when `virsh net-info govoplan-lab`
already reports an active, persistent network. The lab destroy command leaves
this reusable network in place.
## 3. Validate And Create The VMs
```bash
LAB="$HOME/.config/govoplan/labs/development.toml"
PYTHON="/mnt/DATA/git/govoplan/.venv/bin/python"
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" doctor --online
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" create
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" create --apply
```
The preview is safe to run repeatedly. Creation reuses a domain whose exact
lab-owned name already exists and otherwise creates a thin qcow2 overlay under
`vm_image_directory/<lab>/<node>`.
## 4. Deploy GovOPlaN
If `git.add-ideas.de` requires authentication for release images, export a
read-only package/container-registry identity for this shell. A Gitea package
token can be used as the password:
```bash
export GOVOPLAN_LAB_REGISTRY_USERNAME='package-reader'
read -r -s GOVOPLAN_LAB_REGISTRY_PASSWORD
export GOVOPLAN_LAB_REGISTRY_PASSWORD
```
Then preview and apply:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" deploy
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" deploy --apply
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" status
unset GOVOPLAN_LAB_REGISTRY_PASSWORD
```
Deployment verifies the downloaded release manifest and keyring by pinned
digest and by the existing GovOPlaN signature policy. It deploys PostgreSQL,
Redis, single-node Garage and GreenMail on the state VM. The API, WebUI, workers
and scheduler run in K3s from digest-pinned release images. A private lab CA
protects both ingress and S3; backend pods receive only the CA Secret and keep
TLS verification enabled.
The final output identifies two local files below `state_directory`:
- `hosts` maps the public GovOPlaN and S3 test names to their VM addresses;
- `pki/ca.crt` is the private lab CA certificate.
Add the host mappings to the test client's resolver and trust the CA only on
devices used for this lab. On Debian/Ubuntu:
```bash
STATE="$HOME/.local/share/govoplan/labs/govoplan-k8s-lab"
cat "$STATE/hosts"
sudo install -m 0644 "$STATE/pki/ca.crt" \
/usr/local/share/ca-certificates/govoplan-k8s-lab.crt
sudo update-ca-certificates
```
Review mappings before adding them to `/etc/hosts`; the lifecycle does not edit
the workstation's trust or resolver configuration.
### Enroll the first administrator
The production runtime does not create a default password. Issue one expiring,
single-use first-administrator credential inside an API pod and copy its
owner-only artifact out immediately:
```bash
KUBECTL="$STATE/bin/kubectl"
POD="$($KUBECTL -n govoplan get pods \
-l app.kubernetes.io/component=api \
-o jsonpath='{.items[0].metadata.name}')"
ARTIFACT="$STATE/first-admin-enrollment.json"
umask 077
$KUBECTL -n govoplan exec "$POD" -- \
python -m govoplan_core.commands.first_admin issue \
--reason 'initial Kubernetes lab enrollment' \
--output /tmp/first-admin-enrollment.json
$KUBECTL -n govoplan exec "$POD" -- \
cat /tmp/first-admin-enrollment.json > "$ARTIFACT"
$KUBECTL -n govoplan exec "$POD" -- \
rm -f /tmp/first-admin-enrollment.json
chmod 0600 "$ARTIFACT"
```
Submit the token from that artifact once to
`/api/v1/bootstrap/first-admin` with the administrator email, display name,
password, tenant slug and tenant name. The password must contain at least 12
characters. The lab command performs that exchange without placing either the
token or password in process arguments, rejects redirects, and removes the
artifact only after HTTP 201:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" enroll-admin \
--email 'owner@example.org' \
--display-name 'System Owner' \
--tenant-slug default \
--tenant-name 'Default Tenant'
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" enroll-admin \
--email 'owner@example.org' \
--display-name 'System Owner' \
--tenant-slug default \
--tenant-name 'Default Tenant' \
--apply
```
The public lab hostname must already resolve on the management workstation;
the command verifies TLS through the generated private CA directly.
## 5. Collect #27 Evidence
Create a short-lived API key authorized to read the Ops status endpoint. In the
current Access administration UI, open **Tenant API keys** and select only
**View tenant settings** (`admin:settings:read`); the Ops endpoint explicitly
accepts that compatibility scope. A dedicated operator credential may instead
use `ops:operations:read`. Then run:
```bash
export GOVOPLAN_OPS_API_KEY='short-lived-value'
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" verify \
--exercise-api-pod-loss
unset GOVOPLAN_OPS_API_KEY
```
The verifier requires ready API and WebUI pods across at least two Kubernetes
nodes, all Deployments available, consistent runtime composition, queue
coverage and a valid database-connection budget. During the optional drill it
deletes one ready API pod, probes public readiness and waits for replacement.
It writes sanitized output to
`state_directory/evidence/kubernetes-multi-host.json` and never stores the API
key. A rehearsal inventory prints an explicit warning that its result is not
independent-failure-domain evidence.
Retain these private artifacts together for review:
1. `inventory.json` and the reviewed inventory TOML;
2. the adopted release manifest/keyring and installation receipt;
3. `kubernetes.json`;
4. the Kubernetes verifier output;
5. private cluster logs for the approved drill window;
6. the operator's out-of-band evidence that the worker hypervisors are
independent physical hosts or availability zones.
GovOPlaN #37 additionally requires independent assessment and production
approval keys. Running its evidence jobs in containers is supported, but a
container does not create an independent authority. Follow
`TARGET_MATURITY_EVIDENCE_RUNBOOK.md` after the #27 drill passes.
## 6. Update, Pause, Resume And Remove
After reviewing and changing pinned image/K3s/release values in the inventory:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" update
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" update --apply
```
Workers are cordoned, drained, updated and uncordoned one at a time. K3s
controls are reconciled serially. The release-specific migration Job remains
subject to GovOPlaN's signed backup-evidence gate. The lab update command is not
a substitute for creating recovery evidence before a destructive state-schema
change.
To stop compute use without deleting disks:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" pause --apply
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" resume --apply
```
To remove VM resources while preserving local evidence:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" destroy
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" destroy \
--apply --confirm govoplan-k8s-lab
```
Add `--purge-local-state` only after evidence and recovery material have been
retained elsewhere. That option deletes the generated local CA, secrets,
manifests and evidence as well as the VMs.
## Acceptance Boundary
This tool supplies reproducible infrastructure and executes the bounded
stateless-node drill. It does not certify the truth of operator-entered failure
domains, provide HA PostgreSQL/Redis/Garage, create production backup evidence,
or approve its own results. Those boundaries are deliberate: #27 can close
after a passing run on independently controlled hosts; broader production
maturity remains governed by #35, #37 and the target evidence runbook.
+197
View File
@@ -0,0 +1,197 @@
# Package Registry Releases
GovOPlaN publishes reusable module artifacts through Gitea's native PyPI and
npm registries. These packages improve developer installation, release
resolution, cacheability, and artifact inspection. They do not replace the
signed runtime distribution: the signed manifest and digest-pinned OCI images
remain the production deployment authority.
## Publication boundary
Every repository with a `pyproject.toml` contains
`.gitea/workflows/module-package-release.yml`. The meta repository owns the
canonical template and installs it with:
```bash
python tools/repo/sync-module-package-workflows.py --write
python tools/repo/sync-module-package-workflows.py --check
```
The workflow runs for `v*` tags and may be dispatched manually for an existing
tag. The organization preflight verifies that every package repository protects
the `v*` namespace. Before building, the workflow itself verifies that:
- the tagged commit is contained in `main`;
- the tag, Python project version, and optional WebUI package version agree;
- package names remain in the `govoplan-*` and `@govoplan/*-webui` namespaces.
The workflow binds the repository explicitly from the Gitea Actions context.
Do not rely on GitHub-compatible environment variables being injected by the
runner image; Gitea runners may expose only the context values. Gitea 1.24 job
tokens cannot read repository tag-protection settings, so package jobs must not
receive a broad administrator token merely to repeat the organization preflight.
Run the following before the first publication and after repository or tag-rule
changes:
```bash
python tools/gitea/gitea-configure-package-releases.py
```
Preview and dispatch the exact wheel/WebUI versions selected by the developer
meta-package with:
```bash
python tools/gitea/gitea-dispatch-package-set.py \
--env-file ~/.config/gitea/gitea.env
python tools/gitea/gitea-dispatch-package-set.py \
--env-file ~/.config/gitea/gitea.env \
--apply
```
The dispatcher reads exact versions from `packages/govoplan-meta/pyproject.toml`,
inspects the selected tag to determine whether a WebUI package is expected,
skips complete registry pairs and does not duplicate an active workflow. Use
`--repository govoplan-core` for a bounded dispatch or `--verify-existing` to
rebuild and hash-verify versions already present in both registries.
For coordinated lockstep tags, `push-release-tag.sh` pushes module tags first,
Core next, and the meta tag last. This is a dependency guarantee for a
single-capacity Actions runner: the developer package cannot run before its
exact Core and module versions have entered the queue.
The same release entry point first validates the migration graph, then records
the reviewed current Alembic heads under the target release version and reruns
the strict migration audit before it changes package versions, commits, or
tags. The default preflight intentionally does not require those heads to exist
in the previous release baseline. A failed candidate-baseline check therefore
cannot produce a protected package release.
The source gate validates `pyproject.toml`, the module version declaration
(`MODULE_VERSION` or the top-level `ModuleManifest.version`), public package
`__version__`, and WebUI metadata before creating tags. Release-tag artifact
checks run only after the candidate tags and immutable WebUI lock have been
created locally.
Release-lock regeneration resolves a fresh immutable lock from the reviewed
candidate manifests; it does not seed resolution from the previous release
lock. This prevents removed transitive packages and stale peer metadata from
blocking or contaminating the new release. Candidate resolution also uses an
isolated temporary npm cache, so a locally replaced tag cannot reuse metadata
from a failed, unpushed release attempt.
Modules that retain the same WebUI package identity in both a root publish
manifest and `webui/package.json` use the WebUI manifest as the canonical peer
contract. The coordinated release synchronizes `peerDependencies` and
`peerDependenciesMeta` into the publish manifest before creating the module
tag, then synchronizes each lockfile root from the final package metadata. A
distinct root package remains independent.
It builds one wheel and, where applicable, one npm tarball. The workflow records
the source tag, source commit, filename, size, and SHA-256 in
`package-artifacts.json` before publishing. Gitea rejects a second upload of the
same package version, so correction requires a new version rather than artifact
replacement.
A retry after partial publication is safe. Before upload, the workflow reads the
native package registry file record and compares its SHA-256 with the artifact
rebuilt from the protected tag. An exact existing artifact is skipped; a
same-version artifact with another digest or an unexpected file set fails
closed. This permits a failed npm publication to resume without weakening
package immutability or accepting `--skip-existing` blindly.
The npm tarball is always published through an explicit local `./dist/...`
path. Without that prefix, npm may interpret a relative tarball name as a Git
package shorthand before it ever contacts the configured registry.
Published WebUI packages contain registry-compatible dependencies only. The
workflow converts an internal dependency pinned to a protected `vX.Y.Z` Git tag
into the exact `X.Y.Z` registry version and rejects unresolved `file:` or Git
dependencies. Repository development metadata may therefore keep local or Git
references without leaking them into the published package contract.
Historical `add-ideas` and current `GovOPlaN` organization URLs are accepted
for immutable tagged releases; both normalize to the same exact registry
dependency and no branch or unversioned Git reference is accepted.
## One-time Gitea setup
Protect `v*` tags in every package repository and the meta repository. Allow
only the `Owners` team to create or delete those tags.
```bash
set -a
. ~/.config/gitea/gitea.env
set +a
python tools/gitea/gitea-configure-package-releases.py --apply
```
Create a dedicated personal access token with only `write:package` scope and
store these organization-level Actions secrets on `GovOPlaN`:
- `GOVOPLAN_PACKAGE_USERNAME`: account owning the package token;
- `GOVOPLAN_PACKAGE_TOKEN`: dedicated package-write token.
Do not use an administrator or general release token. Gitea 1.24 does not grant
package publication to the automatic Actions job token. Organization secrets
allow the same least-privilege credential to serve every module workflow.
## Exact release consumption
`tools/release/generate-release-package-set.py` translates the reviewed Git
source refs in `requirements-release.txt` into an exact registry package set.
It resolves each version tag to its commit and verifies the package metadata in
that tag.
`tools/release/resolve-package-artifacts.py` then downloads exactly those wheel
and WebUI versions from Gitea. It reads the identity embedded in every wheel and
npm tarball, rejects missing, duplicate, unexpected, or oversized artifacts,
and writes `package-artifacts.lock.json` with SHA-256 values and npm integrity
values. Credentials are accepted only through environment variables and are
never written to the lock. Python resolution ignores ambient pip configuration
and extra indexes for GovOPlaN roots, preventing an internal package name from
being selected from an undeclared registry.
The runtime distribution workflow uses the verified wheelhouse directly and
installs module WebUI tarballs only after matching them to the lock. It publishes
the package set, package lock, and hash-locked requirements as release assets.
The WebUI installer receives the absolute runtime-build interpreter path so its
directory changes cannot escape the isolated release environment.
Gitea 1.24 dispatches this workflow from a branch, but that branch is only the
workflow implementation. The job fetches and peels the protected `v<version>`
tag explicitly, then binds both the signed distribution source and the Gitea
release assets to that exact commit. A post-tag workflow repair can therefore
retry publication without relabelling the later branch commit as released
source.
The package-lock SHA-256 is part of the signed distribution manifest. Runtime
finalization also requires the lock's package versions and hashes to match the
wheel composition embedded in the images. OCI assembly remains network-free
after package and third-party dependency resolution.
The source refs remain in the module catalog for source provenance and release
planning. Production installation consumes the signed runtime images rather
than invoking `pip`, `npm`, or Git on the target host.
## Developer meta-package
`packages/govoplan-meta` builds the optional `govoplan` package. Its default
dependencies mirror the reviewed runtime roots; `govoplan[full]` adds all
currently packageable workspace modules. Regenerate it after changing release
requirements or package versions:
```bash
python tools/release/generate-developer-meta-package.py
python tools/release/generate-developer-meta-package.py --check
```
`push-release-tag.sh` performs this synchronization before release commits and
tags. The meta-package is for editable/developer setup and composition tests. It
does not enable modules, apply migrations, provision services, or establish
backup and recovery evidence.
If the tag-triggered developer meta-package job fails before publication, rerun
`publish-developer-meta-package.yml` with the existing protected version. The
manual path validates that tag against `main`, checks out its exact commit, and
publishes only when the registry does not already contain the same wheel hash.
Generic Packages are intentionally not used. Add that transport only when a
consumer needs an artifact format unsupported by PyPI, npm, Gitea Releases, or
the OCI registry.
+68 -15
View File
@@ -25,6 +25,7 @@ releases, module boundaries, migrations, and security controls.
| Labels and translations | Generated translation catalogs plus source usage |
| Fields and help coverage | Shared form components plus generated TypeScript AST inventory |
| API use by the WebUI | Typed API clients plus generated static reference inventory |
| Stable platform interface IDs | Typed manifest/WebUI declarations plus line-independent source anchors for low-level controls |
| Effective configuration | Owning module data plus Policy provenance |
Runtime introspection is authoritative for an installed system. Static source
@@ -45,9 +46,13 @@ The command writes:
- `audit-reports/platform-inventory/platform-interface-inventory.json`
- `audit-reports/platform-inventory/platform-interface-inventory.md`
Use `--strict` in CI. In addition to translation coverage, strict mode requires
every backend endpoint without a statically visible WebUI path to have an exact
entry in
Use `--strict` for the combined translation, endpoint, and declaration audit.
Use `--strict-declarations` for duplicate/stale/undeclared interface checks
without making existing translation coverage a release blocker. Use
`--strict-endpoints` in the endpoint-surface CI gate so unrelated translation
catalog work cannot disable route classification enforcement. Both strict modes
require every backend endpoint without a statically visible WebUI path to have
an exact entry in
`tools/inventory/endpoint-surface-declarations.json`. The registry is keyed by
repository, HTTP method, and canonical version-independent path. It accepts:
@@ -73,6 +78,16 @@ It combines:
2. TypeScript AST extraction of fields, label attributes, visible text,
translations, frontend routes, navigation, capabilities, and API references
3. Python AST extraction of FastAPI route decorators and router prefixes
4. normalized runtime declarations from every loaded `ModuleManifest`
The declaration set covers routes, navigation, View surfaces, fields, actions,
help references, translations, admin/settings sections, widgets, search
objects, permissions, provided interfaces, and backend capabilities. Typed
module contributions keep their declared IDs. Shared controls may declare
`interfaceId` and `helpTopicId`; otherwise the extractor assigns a deterministic
source anchor based on repository, file, component context, control type, and
semantic label rather than a line number. The JSON records which identity
source was used.
The JSON includes exact repository, file, and line evidence. A missing-help
entry is a review candidate because dynamic parent components may supply help.
@@ -80,9 +95,40 @@ A backend route without a static frontend reference is also a review candidate:
public APIs, workers, callbacks, health checks, connectors, and dynamic URL
assembly are valid explanations.
`--strict` currently enforces only translation-catalog completeness. Endpoint
and help classifications need narrow reviewed baselines before they can become
release gates.
The module matrix enforces endpoint and interface declarations with
`--strict-endpoints --strict-declarations`.
Combined `--strict` additionally fails when used translation keys are absent
from generated locale catalogs. Help-text findings remain review candidates
rather than a release gate because dynamic parent components can supply help.
## Runtime Comparison
Core exposes a sanitized read-only catalog at
`GET /api/v1/platform/interface-catalog`. Access requires
`admin:module:read` or `system:settings:read`. Tenant module entitlements are
applied before serialization, so the response describes only the effective
installed combination. It contains IDs, paths, authorization metadata,
versions, counts, and canonical digests; it excludes factories, callbacks,
credentials, and mutable runtime state.
Capture and compare a running installation:
```bash
curl --fail --silent \
-H "Authorization: Bearer $GOVOPLAN_ACCESS_TOKEN" \
"$GOVOPLAN_URL/api/v1/platform/interface-catalog" \
> /tmp/govoplan-runtime-interface.json
./.venv/bin/python tools/inventory/platform-interface-inventory.py \
--runtime-snapshot /tmp/govoplan-runtime-interface.json \
--strict-declarations \
--strict-endpoints
```
The comparison accepts any installed subset. Every module present in the
runtime response must have the same contract version, module version, and
declaration digest as the static release inventory. Unknown, duplicate, or
mismatched runtime modules fail strict declaration mode.
## Admin Information Architecture
@@ -136,13 +182,20 @@ Custom code, new routes, arbitrary SQL, and executable workflow nodes remain
release artifacts. Modeling them as ordinary configuration would create an
unreviewed code-execution and migration channel.
## Next Enforcement Slices
## Enforced Contract
1. Require every WebUI module route and admin/settings contribution to have
matching manifest metadata or a reviewed exception.
2. Add stable field IDs and optional help-topic IDs to shared field components.
3. Classify each statically unreferenced backend endpoint by consumer type.
4. Compare a running installation's OpenAPI and module registry against the
release inventory.
5. Publish the sanitized installed-system structure through Ops/Docs for
authorized administrators.
1. Public WebUI routes and View surfaces must reconcile with runtime manifest
metadata; stale runtime routes and source-only public surfaces fail CI.
2. Duplicate stable IDs fail CI. Shared controls support explicit field/action
and help-topic identities; fallback anchors remain visible review evidence.
3. Every statically unreferenced backend endpoint has an exact reviewed
consumer classification, and stale classifications fail CI.
4. Runtime module combinations can be compared exactly with static release
evidence through versioned per-module digests.
5. Runtime introspection is authorized, tenant-filtered, and read-only. It is
safe for Ops/Docs projection but is not a generic configuration or code
mutation channel.
Generated JSON and Markdown remain build/audit artifacts. Do not hand-edit or
use them as a backlog; change the owning manifest, typed WebUI contribution,
translation/help declaration, or exact endpoint classification instead.
+173
View File
@@ -0,0 +1,173 @@
# GovOPlaN Platform Core Ideas
## Purpose
GovOPlaN is an institutional governance and operations layer. Its central
promise is:
> Model the institution, orchestrate its work, connect its systems, and
> preserve why and under whose authority it acted.
The platform should let people complete a real task without understanding its
repository or module graph. It should let institutions retain control over
their data, procedures, providers, and deployment while still sharing
interoperable definitions and evidence.
This document is the stable summary of the ideas that every product package,
module, interface, and integration must preserve. Current implementation state
lives in [Strategy Status](STRATEGY_STATUS.md).
## Ten Core Ideas
### 1. Institutional context before application context
Work happens for a tenant, institution, organizational unit, function,
mandate, jurisdiction, service, case, and represented party. The real actor
and represented capacity remain distinct. Application permissions alone do not
prove institutional competence.
### 2. Governance is executable
Policy is not explanatory prose around an operation. Consequential actions
must expose applicable rules, authority, purpose, expected effects, review
requirements, recovery behavior, and evidence. Inheritance may tighten a rule
but must not silently loosen an upstream constraint.
### 3. Time has two independent meanings
Valid time answers when a fact applied. Recorded time answers what the system
knew at a point in history. Historical browsing changes the business-data
projection, never the current authorization context. Corrections and
supersession remain visible rather than rewriting history.
### 4. One context, many owners
Cases, tasks, decisions, records, messages, files, appointments, reports, and
external objects remain owned by their domain modules or source systems. Stable
references create one navigable context without a universal copied master
record or cross-module table access.
### 5. Native and connected operation are peers
For every integration, GovOPlaN states whether it is authoritative, mirrors an
external source, synchronizes governed fields, adds a governance overlay, or
keeps a link only. An external system can be used today and replaced later
without losing provenance or institutional control.
### 6. Human work is a first-class system object
An intake becomes owned, reviewable work. A person can see the current context,
next responsible action, reason, deadline, consequence, and completion
evidence. Workflow Engine coordinates machine and human transitions; focused
views guide people through the relevant platform surfaces.
### 7. Views reduce complexity without changing authority
The interface is a task- and role-sensitive projection of installed
capabilities. Views, dashboards, search, documentation, and workflow-guided
surfaces may hide irrelevant functions, but they never grant access. Users can
escape a focused mode when policy permits and can always understand why
something is unavailable.
### 8. Evidence and recovery are part of the operation
Intent, exact input versions, approvals, external effects, receipts,
outcome-unknown states, reconciliation, corrections, retention, and recovery
belong to one evidence chain. A retry must be idempotent; rollback claims must
distinguish reversible local state from effects already observed elsewhere.
### 9. Inclusion is multi-channel, not portal-only
Public portal, postbox, mail, telephone, paper, in-person assistance, APIs, and
external systems are channels around the same governed work. Assisted entry
records who entered information, for whom, from which source, with which
attestation, and how the affected person receives a usable receipt and
correction path.
### 10. Successful configurations are portable products
Modules are ingredients. A usable product is a signed configuration package
with terminology, forms, policies, workflows, views, reports, provider
profiles, documentation, migration rules, and evidence. Institutions derive
local packages without forking code or weakening inherited constraints.
## Platform Planes
The planes below are ownership lenses, not navigation groups or mandatory
deployment tiers.
| Plane | Responsibility |
| --- | --- |
| Experience | Shell, views, dashboard, search, help, accessibility, and task-focused composition |
| Participation and channels | Portal, postbox, mail, campaigns, calendar, scheduling, consultation, and assisted channels |
| Human work and procedure | Services, forms/runtime, cases, tasks, approvals, workflow execution, and domain procedures |
| Content, records, and evidence | Files, templates, DMS, eAkte/records, audit, reporting, transparency, and publication |
| Institutional governance | Identity, access, tenancy, organizations, functions, mandates, policy, trust, and formal decisions |
| Data and integration | Connectors, datasources, dataflow, search, external references, provider health, and reconciliation |
| Runtime and assurance | Module composition, operations, deployment, recovery, security evidence, and signed packages |
## Canonical Distinctions
The platform must not collapse these pairs:
- identity vs account vs represented capacity;
- role/permission vs function/mandate/competence;
- valid time vs recorded time;
- purpose for use vs general technical access;
- document content vs managed file bytes vs institutional record;
- task vs workflow definition vs workflow instance;
- approval vs formal decision;
- message intent vs transport delivery vs recipient acknowledgement;
- source authority vs connector maturity;
- current state vs historical evidence;
- correction/compensation vs erasure of an observed effect;
- a module boundary vs a user-visible product boundary.
## Product Experience Rule
The normal user interface speaks in services, work, records, messages,
meetings, decisions, and outcomes. Module names, provider IDs, capability names,
package coordinates, and schema details are technical provenance. They are
visible to administrators and in expandable diagnostics, but they are not the
primary information architecture for ordinary work.
## Maturity Rule
A repository, route, model, or unit test does not make a capability complete.
Claims advance only with evidence appropriate to the claim:
1. `scaffold`: boundary and documentation exist;
2. `vertical_slice`: useful behavior has focused tests;
3. `reference_ready`: an end-to-end reference journey passed target,
accessibility, privacy, security, operations, and recovery evidence;
4. `supported`: upgrades, interoperability, support procedures, and release
guarantees are defined;
5. `lts`: compatibility and maintenance windows are contractual.
## Deliberate Non-Goals
GovOPlaN does not aim to:
- replace every specialist system, ERP, DMS, groupware, or data tool;
- make one database authoritative for every connected fact;
- expose every installed capability to every person;
- infer authority from organizational membership alone;
- make historical browsing weaken current security;
- treat AI output as an unaccountable institutional decision;
- create a repository for every noun in the information model;
- claim production maturity from local development evidence.
## Decision Test
A proposed feature fits the platform when it improves at least one real
institutional journey and can answer:
1. Who owns the object and source of truth?
2. In which institutional and temporal context does it apply?
3. For which declared purpose may it be used?
4. Which policy and authority permit the action?
5. What effect, evidence, retention, and recovery behavior result?
6. How can it operate with an external owner without losing autonomy?
7. How will a person discover and complete it without learning the module
graph?
+190
View File
@@ -0,0 +1,190 @@
# Production Target And Independent Evidence Handoff
This runbook identifies the external inputs needed to finish
[GovOPlaN #27](https://git.add-ideas.de/GovOPlaN/govoplan/issues/27) and
[GovOPlaN #37](https://git.add-ideas.de/GovOPlaN/govoplan/issues/37). The
repository can render, inspect and sign evidence for a target, but it cannot
manufacture an independent failure domain or an independent approval authority.
## GovOPlaN #27: real two-node target
The bounded acceptance target is two independently schedulable worker nodes.
The API and WebUI must each have ready replicas on both nodes, all Deployments
must be available, the active module composition and software versions must be
consistent, every configured queue must have a worker, and the database
connection budget must pass. The validation then deletes one ready API pod and
requires replacement without an observed readiness outage.
Two virtual machines on different physical hosts or availability zones meet the
failure-domain intent. Two containers, VMs or Kubernetes nodes on one physical
host are useful development targets but do not close #27. A two-worker cluster
also does not prove control-plane high availability. For a self-managed
production cluster, use three control-plane nodes plus at least two workers; a
managed control plane plus two workers is the shorter path.
### What the target owner must provide
Provide these through a secure handoff, not an issue, chat message or Git:
1. A kubeconfig path with access to the target, for example
`~/.config/govoplan/targets/<target>.kubeconfig`, mode `0600`.
2. A stable installation ID, public HTTPS hostname, namespace, ingress class and
TLS-secret or certificate-manager arrangement.
3. Two independently schedulable workers and permission to place API and WebUI
replicas on both.
4. External, logically shared PostgreSQL, Redis and S3 endpoints with trusted
CA material and network reachability from every worker. Do not co-locate the
only copies of these services on the two workers used for the failure drill.
5. The six runtime secret values required by the generated manifest:
`MASTER_KEY_B64`, `DATABASE_URL`, `GOVOPLAN_DATABASE_URL_PGTOOLS`,
`REDIS_URL`, `FILE_STORAGE_S3_ACCESS_KEY_ID` and
`FILE_STORAGE_S3_SECRET_ACCESS_KEY`.
6. A short-lived GovOPlaN API key limited to `ops:operations:read`, supplied in
`GOVOPLAN_OPS_API_KEY` only for evidence collection.
7. An approved drill window and permission to delete one API pod.
If no Kubernetes target exists, provide hostnames/IP addresses for the machines,
an SSH user and key path, the internal/external DNS plan, and the permitted
firewall ports. Those inputs are sufficient to provision a k3s target. They are
not sufficient to claim control-plane HA unless three control-plane failure
domains are present.
The repository now supplies the strict libvirt/K3s lifecycle and example
inventories for this handoff in
[`KUBERNETES_TEST_LAB.md`](KUBERNETES_TEST_LAB.md). Its `acceptance` mode
rejects a declared topology unless the workers and shared-state fixture occupy
different hypervisor and failure-domain identifiers. Reviewers must still
verify that those identifiers correspond to genuinely independent hosts.
### Separate deployment and evidence authorities
The deployment identity may create and update the namespace, Secret,
ConfigMap, Deployments, Services, Jobs, PodDisruptionBudgets and Ingress. The
evidence collector only needs:
- cluster scope: `get` and `list` for `nodes`;
- target namespace: `get` and `list` for `pods` and `deployments`;
- target namespace during the approved drill: `delete` for `pods`.
Use separate kubeconfig contexts or service accounts when the same person does
not hold both roles.
### Render, apply and verify
Use the signed, digest-pinned installation bundle selected for the target:
```bash
export KUBECONFIG="$HOME/.config/govoplan/targets/<target>.kubeconfig"
python tools/deployment/govoplan-deploy.py render-kubernetes \
--directory /srv/govoplan/<installation-id> \
--namespace govoplan \
--secret-name govoplan-runtime \
--tls-secret-name govoplan-tls \
--s3-ca-secret-name govoplan-s3-ca \
--ingress-class-name nginx \
--output /srv/govoplan/<installation-id>/kubernetes.json
kubectl apply -f /srv/govoplan/<installation-id>/kubernetes.json
kubectl -n govoplan wait --for=condition=available deployment --all --timeout=10m
export GOVOPLAN_OPS_API_KEY="$(cat /run/secrets/govoplan-ops-evidence-key)"
python tools/deployment/govoplan-deploy.py verify-kubernetes \
--directory /srv/govoplan/<installation-id> \
--namespace govoplan \
--exercise-api-pod-loss \
--output /srv/govoplan/<installation-id>/evidence/kubernetes-multi-host.json
unset GOVOPLAN_OPS_API_KEY
```
The verifier emits sanitized JSON and exits nonzero if the topology, runtime,
queue, connection-budget or pod-loss checks fail. Preserve the private cluster
logs and manifest alongside the sanitized result in the controlled evidence
store.
## GovOPlaN #37: controlled signed target evidence
Yes, collection, review and signing can run in containers. A container provides
repeatability and process isolation; it does not create independent authority.
The production approver must control a different private key from the target
operator/assessor and must review the evidence before signing the
`production_approval` scope.
Use at least these three key boundaries:
1. **Installer authority:** signs installed-release-origin receipts only.
2. **Target assessment authority:** signs the permitted target, accessibility,
privacy, security, operations and recovery scopes.
3. **Production approval authority:** independently signs only
`production_approval` after reviewing the other evidence.
Do not reuse release-catalog keys for any of these roles. Keep private Ed25519
keys outside Git, Gitea, GovOPlaN application storage and chat. Publish only the
public keyrings. The proof issuer already rejects key reuse across release,
installer and proof trust domains.
### Generate independently held keys
Each authority runs this command in its own `0700` directory. The generator
refuses existing output paths and writes both files as `0600`:
```bash
install -d -m 0700 "$HOME/.config/govoplan/authority-keys"
python tools/assessments/generate-authority-keypair.py \
--purpose proof \
--key-id authority:target-2026 \
--scope target_environment \
--scope accessibility \
--scope privacy \
--scope security \
--scope operations \
--scope recovery \
--private-key "$HOME/.config/govoplan/authority-keys/target-2026.pem" \
--keyring "$HOME/.config/govoplan/authority-keys/target-2026-public.json"
```
The independent production approver generates another key with only
`--scope production_approval`. An installer authority uses `--purpose installer`
and no `--scope`. Merge public key entries into the separately controlled
keyrings only after the responsible authorities verify fingerprints out of
band.
### Container boundary
Use two one-shot jobs or containers:
- **Collector/assessor:** network access, read-only source and trust mounts,
read/write private evidence output, and the narrowly scoped kubeconfig. It
must not receive the production-approval private key.
- **Production approver:** `--network none`, read-only assessment/evidence/trust
mounts, a read-only secret mount containing only the approval key, and a
separate output mount. It must not receive deployment credentials.
Build or select the assessment image by digest and record that digest in the
evidence log. A representative runtime shape is:
```bash
docker run --rm --network none --read-only --tmpfs /tmp \
--user "$(id -u):$(id -g)" \
--mount type=bind,src="$PWD/evidence",dst=/evidence,readonly \
--mount type=bind,src="$PWD/trust",dst=/trust,readonly \
--mount type=bind,src="$HOME/.config/govoplan/authority-keys",dst=/run/keys,readonly \
--mount type=bind,src="$PWD/approved",dst=/output \
<assessment-image>@sha256:<digest> \
<assessment command>
```
The current evidence commands and required scopes are documented in
[`TARGET_MATURITY_EVIDENCE_RUNBOOK.md`](TARGET_MATURITY_EVIDENCE_RUNBOOK.md).
The final proof must cover `target_environment`, `accessibility`, `privacy`,
`security`, `operations`, `recovery` and independent `production_approval`, and
must bind to the verified installed composition and installer receipt.
## Completion boundary
#27 can close after the real target produces a passing pod-loss result. #37 can
close after an independently approved, schema-valid proof is generated for that
same installed composition and the public authority keyrings, proof and private
evidence custody references are recorded. Neither issue should close from a
single-host simulation or a self-approved signature.
@@ -0,0 +1,147 @@
# Product Experience and Module Boundaries
## Problem
GovOPlaN's runtime modularity is a strength, but the implementation structure
is exposed too directly in the product. Ordinary users encounter module names,
one top-level route per module, one navigation item per repository, package and
provider identifiers, and errors framed as missing modules. This makes the
system look like a toolbox of adjacent applications instead of one operating
environment for institutional work.
The correction is not a monolithic frontend and not hidden provenance. It is a
separate product information architecture assembled from typed module
contributions.
Implementation is tracked in
[Core #283](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/283).
## Current Exposure Inventory
| Surface | Direct exposure | Appropriate audience | Product-facing alternative |
| --- | --- | --- | --- |
| Side rail | One icon and route for many installed modules | Administrators and power users | Work areas, services, inboxes, records, communication, data and assurance |
| Route paths | Technical owners such as `/dataflow`, `/forms`, or `/postbox` | Deep links and diagnostics | Stable product aliases and journey routes that resolve to owner surfaces |
| Dashboard | Installed module count and module-owned widget library | Operators | Outcome, obligation, work, exception, and service widgets |
| Administration | Package names, database state, capabilities, providers | Module and system administrators | Guided product/package configuration with technical details on demand |
| Errors | "Module/capability not installed" | Diagnostics | Explain the unavailable outcome, responsible administrator, and enabling path |
| Documentation | Topics grouped primarily by module | Administrators | Task, role, service, and object documentation with module provenance secondary |
| Permissions | Module-namespaced scopes | Access administrators | Human-readable responsibility bundles; exact scopes remain inspectable |
| Search | Provider/module as a result facet | Advanced filtering | Object type, institution, time, purpose, case/service, and source authority |
| Workflow | Steps can expose target route/module details | Workflow designers | User-facing action and expected result; technical binding in definition details |
| Connector state | Provider IDs and source types | Integration owners | Named source, authority, freshness, health, last effect, and recovery state |
## Boundary Decision
Three layers remain distinct:
1. **Technical module layer:** package ownership, dependencies, capabilities,
permissions, migrations, routes, and provider identifiers.
2. **Product composition layer:** work areas, object types, journeys, commands,
inboxes, configuration packages, and role-based defaults.
3. **Presentation projection:** active view, tenant policy, current task,
temporal context, language, accessibility preferences, and device layout.
Modules own implementation and contribute typed product metadata. Core
assembles it. Views filters it. Policy constrains it. Access authorizes the
underlying actions. No consumer imports another optional module's UI directly.
## Product Surface Contract
Each WebUI module should be able to announce:
- `product_areas`: stable areas to which a route, command, widget, or object
belongs;
- `object_types`: user-facing nouns, icons, search context, detail route, and
owner provenance;
- `work_item_sources`: open work, exceptions, deadlines, and responsible
capacity;
- `journey_actions`: launch, resume, review, correct, decide, publish, and
reconcile commands;
- `workspace_surfaces`: embeddable but owner-rendered list, detail, editor, and
status surfaces;
- `configuration_contributions`: guided settings with consequence and
prerequisite metadata;
- `help_contexts`: user/admin documentation for the product identity as well as
the technical owner;
- `technical_provenance`: module, interface version, capability, and provider
identifiers shown only in details and evidence.
The contract references surfaces. It does not permit Core or a product package
to import their implementation.
## Navigation Model
The default shell should prioritize:
1. global search and create/resume commands;
2. personal and function-bound work;
3. configured product areas;
4. pinned user destinations;
5. administration and technical module inspection when authorized.
A module route remains a valid deep link. A product area may combine links and
owner-rendered surfaces from several modules. When a required contribution is
absent, the area explains the missing outcome rather than rendering a broken
placeholder.
Views remain the projection mechanism. They may select product areas, routes,
sections, commands, widgets, and fields. A view must not grant a permission or
change data semantics. Policy can force, allow, or prohibit a surface at system,
tenant, group, or user scope.
## Error And Provenance Language
Normal errors answer:
- what the person was trying to achieve;
- why it is unavailable or failed;
- whether data was saved or an external effect may have occurred;
- who can resolve it and where;
- the correlation/evidence reference.
An expandable technical section may then identify the module, capability,
provider, request, and version. This keeps the product intelligible without
hiding operational truth.
## Migration
### Slice 1: inventory and aliases
- classify every route, navigation item, widget, setting, search object, and
help context by product area and object type;
- add product aliases without removing existing deep links;
- flag raw module IDs in ordinary-user labels and errors.
### Slice 2: work-first shell
- provide a generic work/exception/deadline aggregation capability;
- make work areas and configured packages the default navigation;
- move the complete module catalogue to administration and an optional power-
user surface.
### Slice 3: composite journeys
- let product packages define journey launch/resume actions and default views;
- let Workflow Engine activate a view and focus an owner surface without
controlling authorization;
- expose provider provenance and technical bindings on demand.
### Slice 4: enforceability
- make product classification mandatory for user-visible manifest surfaces;
- reject duplicate product identities and missing owner routes in CI;
- add browser tests proving that reference users can complete a journey without
knowing module names.
## Acceptance Criteria
- An ordinary user can describe every primary navigation item as work or an
institutional object, not as a package.
- A product package can remove irrelevant navigation while retaining deep-link
and help integrity.
- Missing optional modules produce an actionable product explanation.
- Administrators can still inspect exact module, capability, provider, schema,
and evidence provenance.
- Module permutation tests prove that no product surface assumes an optional
owner is installed.
+75
View File
@@ -0,0 +1,75 @@
# GovOPlaN Documentation Map
This directory contains cross-repository product, architecture, release, and
operational documentation. The map below defines which document answers which
question. A document not listed as the current status source must not present
volatile repository, issue, release, or maturity counts as current facts.
## Strategy
| Question | Canonical source |
| --- | --- |
| What are the stable ideas and boundaries of the platform? | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) |
| What product outcomes should GovOPlaN pursue? | [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) |
| Which institutional concepts and owners form the target architecture? | [Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md) |
| Which end-to-end proofs should guide implementation? | [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md) |
| What is the reconciled state now? | [Strategy Status](STRATEGY_STATUS.md) |
The dated [Strategic Review](STRATEGIC_REVIEW_2026-08-05.md) explains why the
current reset and sequencing were chosen. It is an assessment record, not a
second live status page.
## Product Architecture
| Topic | Canonical source |
| --- | --- |
| Product-facing experience and hiding technical module boundaries | [Product Experience and Module Boundaries](PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md) |
| Federation between autonomous installations | [Federated GovOPlaN Architecture](FEDERATED_GOVOPLAN_ARCHITECTURE.md) |
| Institutional digital twin and continuous assurance | [Institutional Digital Twin](INSTITUTIONAL_DIGITAL_TWIN.md) |
| Assisted and non-digital channels | [Assisted and Non-Digital Channels](ASSISTED_AND_NON_DIGITAL_CHANNELS.md) |
| Cross-module temporal, purpose, retention, and institutional-context adoption | `govoplan-core/docs/INFORMATION_GOVERNANCE_ADOPTION.md` |
| eAkte and digital-record ownership | `govoplan-records/docs/EAKTE_ARCHITECTURE.md` |
| Data source, definition, and transformation graph | [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md) |
| Focused task views | [Views Architecture](VIEWS_ARCHITECTURE.md) |
| Shared interface patterns | [Interface Pattern Language](INTERFACE_PATTERN_LANGUAGE.md) |
## Runtime And Delivery
- [Module Contracts and Installs](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Platform Control Plane](PLATFORM_CONTROL_PLANE.md)
- [Installation and Deployment Architecture](INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md)
- [Kubernetes VM Test Lab](KUBERNETES_TEST_LAB.md)
- [Scaling and Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md)
- [Recovery and Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md)
- [Recovery Ledger Adoption](RECOVERY_LEDGER_ADOPTION.md)
- [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md)
## Evidence And Snapshots
These documents are intentionally dated or pinned. They may remain useful even
after the product changes, but they do not override `STRATEGY_STATUS.md`.
- [Capability and Infrastructure Fit Assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md), pinned to the 2026-07-22 Campaign composition
- [Strategic Review 2026-08-05](STRATEGIC_REVIEW_2026-08-05.md)
- [Backup and Restore Evidence](BACKUP_AND_RESTORE_EVIDENCE.md)
- [Production Target Handoff](PRODUCTION_TARGET_HANDOFF.md)
- [Target Maturity Evidence Runbook](TARGET_MATURITY_EVIDENCE_RUNBOOK.md)
Machine-readable schemas and evidence files belong beside the document that
defines them. Generated inventories belong in `audit-reports/` and should not
be edited manually.
## Maintenance Rules
1. Gitea issues are the only live work-state source.
2. `STRATEGY_STATUS.md` is the only prose reconciliation of current portfolio
state. Refresh it from manifests, inventories, tests, and Gitea; do not copy
its counts into durable architecture pages.
3. Durable documents state decisions, invariants, ownership, and acceptance
gates. They link to status and issues for implementation depth.
4. Dated assessments retain their original composition and conclusion. Add a
snapshot notice rather than silently updating their claims.
5. Module-specific behavior and user/admin documentation remain in the owning
repository. Meta documentation defines cross-module outcomes and contracts.
6. A new strategy document must replace, narrow, or link an existing source;
it must not introduce a parallel roadmap.
+33
View File
@@ -18,6 +18,27 @@ delivered as small, reviewable, green increments and is complete only when its
user journey, failure behavior, documentation, and operator evidence work in a
pinned composition.
## 2026 outcome reset
Repository completion is not product completion. From 2026-08-05 onward, work
is accepted primarily through three maintained real-life journeys:
1. **Governed communication:** select accountable recipients, prepare content
and attachments, approve, deliver through Mail and/or a function-bound
Postbox, reconcile uncertain outcomes, and file the evidence.
2. **Inclusive service-to-decision:** accept a request through a digital or
assisted channel, establish identity and purpose, guide the case through
human and automatic work, decide, notify, and file the resulting eAkte.
3. **Monthly data and sanctions:** acquire immutable source snapshots, validate
and reconcile them interactively, preserve decisions and lineage, produce
reports and files, and deliver the accepted result through Campaign.
The staged program below remains the architectural build order. These journeys
are the acceptance lens across those stages. Every significant feature should
identify the journey it improves, or provide security, operability, recovery,
accessibility, or usability evidence that those journeys require. Work that
does neither stays in the backlog until a concrete consumer exists.
## Why this sequence
The sequence grows one connected product rather than advancing repositories in
@@ -86,6 +107,12 @@ journey needs and supplies contracts shared by all five stages.
execution. Database, broker, cache, and worker channels are constrained by
deployment network policy and authenticated transport rather than treated as
tenant connector profiles.
10. **Information governance.** Temporal browsing, purpose-aware access,
retention/legal-hold behavior, and institutional acting context are applied
to every owned object type. Historical reads use current authorization.
Module manifests state `contract_only`, `partial`, `enforced`, or
`not_applicable` adoption with evidence; supported maturity is blocked until
every applicable dimension is enforced.
## Documentation contract for every reference stage
@@ -108,6 +135,9 @@ Every demonstrated journey provides:
provenance, evidence, retention, and destructive actions.
- **Acceptance view:** runnable examples, expected results, failure injection,
and release gates.
- **Channel and records view:** assisted/non-digital intake and output,
representation, provenance, filing, retention, legal hold, and archive
consequences where the journey creates evidence or a record.
The Docs module selects and links these views according to installed
capabilities and actor context. Feature repositories remain the source of
@@ -436,6 +466,9 @@ or the external editor the document-lifecycle owner.
link, callback, webhook, file, identity, or data row.
- Do not claim a stage complete from local unit tests. Use pinned composition,
target integration, failure drills, adaptive docs, and operator evidence.
- Do not claim a module complete while its relevant information-governance
dimensions remain `contract_only` or while the reference journey lacks an
assisted-channel and records outcome where those are applicable.
- A later stage may prototype contracts while the preceding gate is being
proven, but it may not redefine an owning module's boundary by convenience.
+7
View File
@@ -76,6 +76,13 @@ one place. If a deployment profile later needs pinned SHAs for every repository,
generate that lock as a release artifact instead of making day-to-day
development depend on submodule updates.
Module release tags also publish wheels and WebUI tarballs to the organization
PyPI/npm registries. The meta release resolves exact versions into a hash-bound
package lock before producing the signed OCI runtime. See
`docs/PACKAGE_REGISTRY_RELEASES.md`. Git tags remain source provenance; package
registries are reusable artifact transport; the signed runtime manifest and
digest-pinned images remain production authority.
## Docker Placement
Whole-product Docker and production-like deployment composition belongs in
+25 -4
View File
@@ -1,5 +1,11 @@
# Scaling And Multi-Host Deployment
For the exact external handoff, least-privilege collector permissions and live
two-node acceptance procedure, see
[`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md).
For a reproducible local or multi-hypervisor libvirt/K3s target, use
[`KUBERNETES_TEST_LAB.md`](KUBERNETES_TEST_LAB.md).
## Implemented Contract
GovOPlaN now supports a stateless application tier backed by logically shared
@@ -70,6 +76,7 @@ python tools/deployment/govoplan-deploy.py render-kubernetes \
--namespace govoplan \
--secret-name govoplan-runtime \
--tls-secret-name govoplan-tls \
--s3-ca-secret-name govoplan-s3-ca \
--ingress-class-name nginx \
--output /srv/govoplan/default/kubernetes.json
```
@@ -86,11 +93,25 @@ command prints the exact required key contract. Review the generated
`FORWARDED_ALLOW_IPS` value and replace it with the exact ingress-proxy network
before production use.
When an external S3 endpoint is signed by a private CA, create the optional CA
Secret with a `ca.crt` key and pass `--s3-ca-secret-name`. The renderer mounts
that Secret read-only and sets `AWS_CA_BUNDLE` for API, worker, scheduler,
migration and database-wait containers. It does not disable certificate
verification or replace the WebUI trust store.
The generated containers run as non-root with a read-only root filesystem and
an ephemeral `/tmp`. Runtime Deployments wait for the exact configured database
migration heads before starting. The API exposes `/health/ready`, which fails
while that API node is draining or cannot prove its runtime-coordination
heartbeat.
an ephemeral `/tmp`. Celery Beat keeps its replaceable schedule database there;
durable schedule definitions remain in shared state. The WebUI resolves its
configured API Service when the container starts, so Kubernetes deployments do
not inherit the Compose-only `load-balancer` hostname. Runtime Deployments wait
for the exact dependency-resolved database migration heads before starting.
The API exposes `/health/ready`, which fails while that API node is draining or
cannot prove its runtime-coordination heartbeat.
Replicated API, WebUI, and worker Deployments use a hard hostname-spread
constraint scoped to the current pod-template hash. A rollout therefore keeps
each replica set distributed across independently schedulable nodes instead of
allowing all replacement pods to settle on one node after the old set exits.
## Runtime Coordination
+126
View File
@@ -0,0 +1,126 @@
# Strategic Review - 2026-08-05
## Assessment
GovOPlaN has not lost its central direction. The architecture now expresses a
coherent institutional governance platform, but architecture and repository
breadth have advanced faster than complete, usable outcomes. The immediate
need is convergence: fewer simultaneous fronts, stronger cross-cutting
adoption, and end-to-end reference journeys that non-developers can complete.
This is a dated review. Current status belongs in
[Strategy Status](STRATEGY_STATUS.md); stable direction belongs in
[Platform Core Ideas](PLATFORM_CORE_IDEAS.md).
## What Is Already Strong
- A modular runtime with manifests, capabilities, interfaces, migrations,
optional integrations, signed releases, and permutation checks.
- Explicit institutional semantics for identity, representation,
organization, function, mandate, service, case, party, approval, decision,
evidence, and record references.
- Governed communication foundations spanning Campaign, Mail, Files, Postbox,
Addresses, Distribution Lists, Templates, Audit, and Policy.
- Governed data foundations spanning Connectors, Datasources, Dataflow,
Reporting, Search, and immutable provenance.
- Bitemporal browsing, views, contextual documentation, action/effect
contracts, event delivery, recovery ledgers, and stateless deployment
contracts.
- A credible deployment and release foundation with signed artifacts and
reproducible composition evidence.
## Where The Program Veered
### Repository breadth preceded product proof
Logical modularity often became a repository before a reference journey proved
that an independent release boundary was required. Scaffolds are useful as
ownership markers, but their number makes the product appear broader and more
complete than its supported outcomes.
### Foundations outran reference gates
Later-stage contracts such as federation, encryption, formal governance,
deployment evidence, and broad module metadata were developed while basic
human-work and records journeys remained incomplete. Those foundations are not
wasted; they now need to be consumed by a small number of demonstrable
products.
### The module graph leaked into the experience
Navigation, routes, administration, errors, documentation, and configuration
often present module names and package structure directly. This is appropriate
for operators, but ordinary users should see work, services, records, and
outcomes.
### Status became duplicated
Roadmaps, target architecture, fit assessments, issue comments, and release
documents each contained partial implementation snapshots. Their stable
decisions remain valuable, but volatile counts and maturity claims diverged.
### Too much work remained active simultaneously
The issue portfolio had many high-priority and in-progress items without
milestones. This reduces the signal of both labels and roadmap order and makes
completion harder to demonstrate.
## Where GovOPlaN Has Not Gone Far Enough
1. No composition has yet crossed the full `reference_ready` gate.
2. The human-work spine is incomplete: work queues, tasks, handoffs, deadlines,
reminders, escalation, and resumption need a coherent user experience.
3. Records and document management remain too shallow for a public-sector
operating platform.
4. Real target integrations and GovOPlaN-to-GovOPlaN federation are not yet
proven.
5. Temporal browsing, purpose-aware access, retention, and institutional
context exist as contracts but are not adopted uniformly by domain reads
and effects.
6. German completeness, contextual help, accessibility, responsive behavior,
and browser-level journey testing are not yet release gates everywhere.
7. Multi-host, backup/restore, provider interoperability, and independent
signed target evidence still require real environments and operators.
## Important Omissions
- a named first institution, bounded users, volumes, and operating constraints;
- measurable usability outcomes, not only functional tests;
- installable sector packages and migration/exit demonstrations;
- support, upgrade, deprecation, and LTS promises;
- complete assisted, paper, telephone, and in-person channel handling;
- a native eAkte/records model that can also overlay an external DMS or archive.
## Opportunities Beyond The Original Idea
- an institutional digital twin that exposes responsibilities, dependencies,
obligations, services, work, data, controls, and change impact over time;
- continuous assurance that evaluates controls and evidence as work happens;
- process mining and conformance analysis over governed event histories;
- federated product packages and inter-institution case/evidence exchange;
- accountable assistance that drafts and explains without obscuring authority;
- public evidence chains that disclose decisions and provenance without
exposing protected source data.
## Recommended Reset
1. Freeze new repositories unless a real journey proves an independent owner,
release lifecycle, security boundary, or optional installation need.
2. Use one generated maturity/status dashboard and one current status document.
3. Complete governed communication and function-bound Postbox against a real
target.
4. Complete the monthly-data journey, then sanctions screening on the same
data foundations.
5. Complete one browser-driven service-to-decision journey, including assisted
intake and records.
6. Make eAkte/records the next major product-depth program.
7. Tie feature work to a reference journey, a security/recovery gate, or a
measured usability defect.
## Success Criterion
The reset succeeds when a public institution can install a signed composition,
configure a named procedure, complete it through digital and assisted channels,
connect an external source, reconstruct the authority and evidence, recover it
after failure, and transfer or retire it without custom code.
+118
View File
@@ -0,0 +1,118 @@
# GovOPlaN Strategy Status
## Status Record
| Field | Value |
| --- | --- |
| Reconciled on | 2026-08-05 |
| Source scope | Local workspace manifests, source inventory, focused journey checks, signed release evidence, and live Gitea issue state |
| Stable direction | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) and [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) |
| Delivery source | Gitea issues |
This is the only prose source for current cross-product status. It is a
reconciliation, not a release certification. Module manifests and target
evidence remain authoritative for specific maturity claims.
## Portfolio Snapshot
- 65 source module manifests were loadable and architecture-declared.
- 47 modules declared `vertical_slice`; 18 declared `scaffold`.
- No module declared `reference_ready`, `supported`, or `lts`.
- The live portfolio had 133 open issues, including 39 priority-P1 items.
- 117 open issues had no milestone, so issue labels do not yet express a
reliable completion sequence on their own.
- Three product package manifests existed: governed communication, governed
data and assurance, and service to decision. None had crossed the complete
target-evidence gate.
These counts are dated. Refresh them rather than copying them into another
document.
## Interface And Contract Evidence
The 2026-08-05 source inventory found:
- 1,247 UI fields and 1,220 UI actions;
- 7,929 stable interface declarations with no duplicate IDs;
- 39 frontend routes and 872 backend endpoints;
- no public WebUI surfaces missing runtime declarations;
- no stale runtime route declarations;
- no unclassified endpoint without a static UI reference;
- all 1,247 fields with a resolvable F1 context; 1,087 remain candidates for
richer field-specific content beyond page/module fallback;
- German (`de`) as the complete reference locale and no used key missing from
the required German or English catalogs;
- 260 module information-governance dimensions classified as `contract_only`.
This is an honest platform-wide baseline, not a claim that temporal,
purpose, retention, and institutional-context adoption is complete.
## Credible Current Outcomes
### Platform foundation
Module discovery, optional dependency validation, migrations, shared WebUI,
tenant and access foundations, signed catalogs/packages, event delivery,
recovery contracts, contextual help, views, temporal titlebar context, and
stateless-runtime patterns are implemented and tested at varying depths.
### Governed communication
Campaign authoring, recipient data, attachments, templates, mail profiles,
mock/real delivery paths, audit evidence, reporting, distribution-list
composition, and optional Postbox delivery form the deepest product cluster.
Target provider, accessibility, recovery, and high-volume evidence still
prevent a reference-ready claim.
### Institutional service and decision
Services, Forms, Forms Runtime, Cases, Parties, Mandates, Approvals, Committee,
Voting, Decisions, Portal, Postbox, and Audit have an executable service-to-
decision fixture. Browser-complete assisted intake, production identity,
records, delivery, and target evidence remain.
### Governed data and assurance
Connectors, Datasources, Dataflow, Reporting, Search, Policy, Risk Compliance,
and Workflow provide source governance, immutable snapshots, transformation,
quality, semantic reporting, and provenance foundations. The monthly-data and
sanctions journeys still need real connectors, complete interactive
reconciliation, publication/export, and guided handoff evidence.
## Material Gaps
| Gap | Consequence | Next proof |
| --- | --- | --- |
| No reference-ready product package | The platform cannot yet make a bounded supported-product claim | Complete one named target composition and evidence bundle |
| Human-work spine incomplete | Users still navigate modules and remember unfinished work | Task/work inbox, resumable guided journey, deadlines and handoffs |
| Records/eAkte shallow | Institutional memory and disposition remain fragmented | Native record lifecycle plus external DMS/archive overlay |
| Cross-cutting governance adoption uneven | Historical and purpose-sensitive behavior varies by module | Enforced adoption declarations and route/query/effect migration |
| Explicit help/accessibility depth incomplete | German/reference and F1 association gates now pass, but generic fallback remains too common | High-risk German help content and browser/a11y matrix |
| Real federation absent | Cross-institution exchange remains connector-specific | Paired-instance signed exchange and reconciliation proof |
| External production evidence incomplete | Scale, restore, interoperability and custody claims remain conditional | Real target drills and independent signed evidence |
## Active Strategic Order
1. Establish German, help, temporal, purpose, retention, and institutional
context as enforceable platform quality contracts.
2. Complete governed communication and Postbox against a named target.
3. Complete the monthly-data flow and use it as the data foundation for
sanctions screening.
4. Complete one digital and assisted service-to-decision journey with an eAkte.
5. Add native PostgreSQL search coverage for the objects used by those
journeys; keep OpenSearch optional.
6. Prove one external product connector and one GovOPlaN federation exchange.
7. Finish multi-host, restore, provider, accessibility, and independent signed
target evidence before increasing maturity claims.
## Refresh Procedure
Refresh this page only from evidence:
1. run `tools/checks/check-manifest-shapes.py`;
2. run `tools/inventory/platform-interface-inventory.py --strict
--strict-declarations --strict-endpoints`;
3. run the selected reference-journey checks;
4. inspect signed release and target evidence;
5. query live Gitea issue/milestone state;
6. update the dated values and material gaps here;
7. retain prior assessments as dated evidence rather than rewriting them.
@@ -141,12 +141,16 @@ The canonical backlog item is
Implementation status as of the current source tree:
- Slice 1 now has the source-controlled production artifact boundary: offline
per-architecture wheel resolution, non-root API/Web image definitions,
multi-architecture OCI publication, signed composition/SBOM/provenance,
immutable Gitea assets, a signed one-file deployer, and fail-closed manifest
adoption. The first real published release and cross-architecture runtime
evidence remain release-operator work rather than source-code claims.
- Slice 1 has a published production-artifact baseline. Immutable
[`v0.1.14`](https://git.add-ideas.de/GovOPlaN/govoplan/releases/tag/v0.1.14)
binds source commit `1f039dd39c1ce2672f4978c8abc6dff862ef1445`, a signed
one-file deployer, exact API/Web and managed-dependency image digests,
composition, SBOMs, and provenance. Runtime Distribution
[run #459](https://git.add-ideas.de/GovOPlaN/govoplan/actions/runs/459)
passed migrations, schema checks, non-root API/Web readiness, and worker
delivery/shutdown on both amd64 and arm64. Each future release must renew the
evidence, and a real installation must still produce topology-specific
ingress, failover, backup, and recovery receipts.
- Slice 6 has a working application-tier foundation: state profiles, shared
object storage, runtime node registration/heartbeats/drain, fenced scheduler,
migration serialization, exact-head startup waiting, Ops visibility, and a
+4
View File
@@ -1,5 +1,9 @@
# Target Maturity Evidence Runbook
For authority-key generation, container isolation and the concrete inputs that
must be supplied by the target owner and independent production approver, see
[`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md).
This runbook turns retained target-environment results into a sanitized,
signed GovOPlaN capability-fit proof. It does not make a deployment suitable,
certified, supported, or production-approved by itself. The proof records what
+6
View File
@@ -353,6 +353,12 @@
"description": "GovOPlaN Projects module behavior or integration.",
"exclusive": false
},
{
"name": "module/records",
"color": "0052cc",
"description": "GovOPlaN Records and eAkte lifecycle behavior or integration.",
"exclusive": false
},
{
"name": "module/reporting",
"color": "c2e0c6",
@@ -13,6 +13,7 @@
"expires_at",
"revoked",
"deployer",
"package_lock",
"images",
"dependencies",
"composition",
@@ -27,6 +28,7 @@
"expires_at": { "type": "string", "format": "date-time" },
"revoked": { "const": false },
"deployer": { "$ref": "#/$defs/artifact" },
"package_lock": { "$ref": "#/$defs/artifact" },
"images": {
"type": "object",
"additionalProperties": false,
+11
View File
@@ -0,0 +1,11 @@
# GovOPlaN developer meta-package
`govoplan` is an optional convenience package for local development and
composition tests. The default dependency set matches the reviewed runtime
release roots; `govoplan[full]` adds every packageable module present in the
workspace at generation time.
This package is not a production deployment artifact. Production installations
consume the signed runtime distribution manifest and digest-pinned OCI images.
The package does not enable modules, apply migrations, choose infrastructure,
or replace installation and recovery evidence.
+90
View File
@@ -0,0 +1,90 @@
[build-system]
requires = ["setuptools>=69", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "govoplan"
version = "0.1.17"
description = "Developer convenience package for a versioned GovOPlaN composition"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" }
dependencies = [
"govoplan-core[server]==0.1.17",
"govoplan-tenancy==0.1.17",
"govoplan-organizations==0.1.17",
"govoplan-identity==0.1.17",
"govoplan-idm==0.1.17",
"govoplan-access==0.1.17",
"govoplan-admin==0.1.17",
"govoplan-policy==0.1.17",
"govoplan-audit==0.1.17",
"govoplan-dashboard==0.1.17",
"govoplan-files==0.1.17",
"govoplan-mail==0.1.17",
"govoplan-campaign==0.1.17",
"govoplan-calendar==0.1.17",
"govoplan-docs==0.1.17",
"govoplan-ops==0.1.17",
]
[project.optional-dependencies]
full = [
"govoplan-addresses==0.1.17",
"govoplan-approvals==0.1.17",
"govoplan-assets==0.1.17",
"govoplan-booking==0.1.17",
"govoplan-cases==0.1.17",
"govoplan-certificates==0.1.17",
"govoplan-committee==0.1.17",
"govoplan-connectors==0.1.17",
"govoplan-consultation==0.1.17",
"govoplan-contracts==0.1.17",
"govoplan-dataflow==0.1.17",
"govoplan-datasources==0.1.17",
"govoplan-decisions==0.1.17",
"govoplan-dist-lists==0.1.17",
"govoplan-encryption==0.1.17",
"govoplan-evaluation==0.1.17",
"govoplan-facilities==0.1.17",
"govoplan-forms==0.1.17",
"govoplan-forms-runtime==0.1.17",
"govoplan-grants==0.1.17",
"govoplan-helpdesk==0.1.17",
"govoplan-identity-trust==0.1.17",
"govoplan-inspections==0.1.17",
"govoplan-learning==0.1.17",
"govoplan-mandates==0.1.17",
"govoplan-notifications==0.1.17",
"govoplan-parties==0.1.17",
"govoplan-permits==0.1.17",
"govoplan-poll==0.1.17",
"govoplan-portal==0.1.17",
"govoplan-postbox==0.1.17",
"govoplan-procurement==0.1.17",
"govoplan-projects==0.1.17",
"govoplan-records==0.1.17",
"govoplan-reporting==0.1.17",
"govoplan-resources==0.1.17",
"govoplan-rest==0.1.17",
"govoplan-risk-compliance==0.1.17",
"govoplan-scheduling==0.1.17",
"govoplan-search==0.1.17",
"govoplan-services==0.1.17",
"govoplan-soap==0.1.17",
"govoplan-templates==0.1.17",
"govoplan-tickets==0.1.17",
"govoplan-transparency==0.1.17",
"govoplan-views==0.1.17",
"govoplan-voting==0.1.17",
"govoplan-wiki==0.1.17",
"govoplan-workflow==0.1.17",
"govoplan-workflow-engine==0.1.17",
]
[project.urls]
Repository = "https://git.add-ideas.de/GovOPlaN/govoplan"
Documentation = "https://govoplan.add-ideas.de"
[tool.setuptools.packages.find]
where = ["src"]
@@ -0,0 +1,12 @@
"""Metadata helpers for the optional GovOPlaN developer composition."""
from importlib.metadata import PackageNotFoundError, version
try:
__version__ = version("govoplan")
except PackageNotFoundError: # pragma: no cover - source checkout only
__version__ = "0+unknown"
__all__ = ["__version__"]
+15 -15
View File
@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published.
../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.8
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.10
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.11
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.8
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.17
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.17
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.17
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.17
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.17
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.17
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.17
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.17
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.17
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.17
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.17
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.17
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.17
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.17
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.17
+120
View File
@@ -0,0 +1,120 @@
from __future__ import annotations
import base64
import json
from pathlib import Path
import stat
import subprocess
import sys
import tempfile
import unittest
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from jsonschema import Draft202012Validator, FormatChecker
META_ROOT = Path(__file__).resolve().parents[1]
GENERATOR = META_ROOT / "tools" / "assessments" / "generate-authority-keypair.py"
class AssessmentAuthorityKeypairTests(unittest.TestCase):
def test_generates_schema_valid_scoped_proof_authority(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
output_dir = Path(temp_dir)
output_dir.chmod(0o700)
private_path = output_dir / "target.pem"
keyring_path = output_dir / "target.json"
result = subprocess.run(
(
sys.executable,
str(GENERATOR),
"--purpose",
"proof",
"--key-id",
"authority:target-2026",
"--scope",
"target_environment",
"--scope",
"operations",
"--private-key",
str(private_path),
"--keyring",
str(keyring_path),
),
check=False,
capture_output=True,
text=True,
)
self.assertEqual(0, result.returncode, result.stderr)
self.assertEqual(0o600, stat.S_IMODE(private_path.stat().st_mode))
self.assertEqual(0o600, stat.S_IMODE(keyring_path.stat().st_mode))
keyring = json.loads(keyring_path.read_text(encoding="utf-8"))
schema = json.loads(
(
META_ROOT
/ "docs"
/ "capability-fit-proof-authority-keyring.schema.json"
).read_text(encoding="utf-8")
)
errors = tuple(
Draft202012Validator(
schema, format_checker=FormatChecker()
).iter_errors(keyring)
)
self.assertEqual((), errors)
self.assertEqual(
["target_environment", "operations"],
keyring["keys"][0]["allowed_scopes"],
)
private_key = serialization.load_pem_private_key(
private_path.read_bytes(), password=None
)
self.assertIsInstance(private_key, Ed25519PrivateKey)
public_key = base64.b64encode(
private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
).decode("ascii")
self.assertEqual(public_key, keyring["keys"][0]["public_key"])
def test_installer_authority_uses_fixed_scope_and_refuses_overwrite(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
output_dir = Path(temp_dir)
output_dir.chmod(0o700)
private_path = output_dir / "installer.pem"
keyring_path = output_dir / "installer.json"
command = (
sys.executable,
str(GENERATOR),
"--purpose",
"installer",
"--key-id",
"authority:installer-2026",
"--private-key",
str(private_path),
"--keyring",
str(keyring_path),
)
first = subprocess.run(
command, check=False, capture_output=True, text=True
)
second = subprocess.run(
command, check=False, capture_output=True, text=True
)
self.assertEqual(0, first.returncode, first.stderr)
self.assertNotEqual(0, second.returncode)
keyring = json.loads(keyring_path.read_text(encoding="utf-8"))
self.assertEqual(
["installed_release_origin"],
keyring["keys"][0]["allowed_scopes"],
)
if __name__ == "__main__":
unittest.main()
+178
View File
@@ -158,6 +158,81 @@ class DeploymentInstallerTests(unittest.TestCase):
evidence["snapshot"]["ready_node_names"],
)
def test_kubernetes_api_loss_uses_a_non_json_mutation_command(self) -> None:
initial_pods = [
_kubernetes_test_pod("api-a", "api", "node-a"),
_kubernetes_test_pod("api-b", "api", "node-b"),
_kubernetes_test_pod("web-a", "web", "node-a"),
_kubernetes_test_pod("web-b", "web", "node-b"),
]
replacement_pods = [
_kubernetes_test_pod("api-b", "api", "node-b"),
_kubernetes_test_pod("api-c", "api", "node-a"),
]
deleted = False
actions: list[tuple[str, ...]] = []
def run(arguments):
if "nodes" in arguments:
return {
"items": [
{
"metadata": {"name": name},
"spec": {},
"status": {
"conditions": [
{"type": "Ready", "status": "True"}
]
},
}
for name in ("node-a", "node-b")
]
}
if "deployments" in arguments:
return {
"items": [
_kubernetes_test_deployment("api", 2),
_kubernetes_test_deployment("web", 2),
]
}
return {"items": replacement_pods if deleted else initial_pods}
def act(arguments):
nonlocal deleted
actions.append(tuple(arguments))
deleted = True
evidence = collect_kubernetes_evidence(
installation_id="govoplan-cluster",
namespace="govoplan",
ops_url="https://govoplan.example.test/api/v1/ops/status",
api_key="not-retained",
exercise_api_pod_loss=True,
command_runner=run,
action_runner=act,
json_fetcher=lambda _url, _key: {
"readiness": {"ready": True},
"runtime_cluster": {
"composition": {"skewed": False},
"software_versions": {"skewed": False},
"queues": {"missing": []},
},
"checks": [
{
"id": "database_capacity",
"state": "ok",
"detail": "Within budget",
}
],
},
)
self.assertEqual("passed", evidence["api_pod_loss"]["state"])
self.assertEqual(1, len(actions))
self.assertIn("delete", actions[0])
self.assertNotIn("-o", actions[0])
self.assertNotIn("not-retained", json.dumps(evidence))
def test_pre_migration_failure_restores_checksum_verified_applied_bundle(
self,
) -> None:
@@ -347,6 +422,10 @@ class DeploymentInstallerTests(unittest.TestCase):
"template"
]["spec"]["containers"][0]["command"]
self.assertIn("govoplan_core.commands.fenced_run", scheduler_command)
self.assertEqual(
["--schedule", "/tmp/celerybeat-schedule"],
scheduler_command[-2:],
)
api_init_command = deployments["govoplan-cluster-api"]["spec"]["template"][
"spec"
]["initContainers"][0]["command"]
@@ -385,6 +464,12 @@ class DeploymentInstallerTests(unittest.TestCase):
"spec"
]["containers"][0]["command"]
self.assertIn("--concurrency", worker_command)
for deployment in deployments.values():
spread = deployment["spec"]["template"]["spec"][
"topologySpreadConstraints"
][0]
self.assertEqual("DoNotSchedule", spread["whenUnsatisfiable"])
self.assertEqual(["pod-template-hash"], spread["matchLabelKeys"])
self.assertEqual(
"62",
manifest["metadata"]["annotations"][
@@ -392,6 +477,99 @@ class DeploymentInstallerTests(unittest.TestCase):
],
)
def test_kubernetes_export_mounts_an_optional_s3_ca_on_backend_roles(
self,
) -> None:
spec = default_spec(
installation_id="govoplan-cluster",
postgres_mode="external",
redis_mode="external",
storage_mode="s3",
api_replicas=2,
web_replicas=2,
worker_replicas=2,
api_image="registry.example.test/govoplan-api@sha256:" + "a" * 64,
web_image="registry.example.test/govoplan-web@sha256:" + "b" * 64,
)
environment = initial_secrets(
spec,
supplied={
"DATABASE_URL": "postgresql+psycopg://user:secret@postgres.example.test/govoplan",
"GOVOPLAN_DATABASE_URL_PGTOOLS": "postgresql://user:secret@postgres.example.test/govoplan",
"REDIS_URL": "rediss://:secret@redis.example.test/0",
"FILE_STORAGE_S3_ENDPOINT_URL": "https://s3.example.test",
"FILE_STORAGE_S3_REGION": "eu-test-1",
"FILE_STORAGE_S3_ACCESS_KEY_ID": "object-key",
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": "object-secret",
"FILE_STORAGE_S3_BUCKET": "govoplan",
"GOVOPLAN_DB_CONNECTION_LIMIT": "100",
},
)
manifest = render_kubernetes(
spec,
environment,
s3_ca_secret_name="govoplan-s3-ca",
backup_required=False,
)
backend_pods = [
item["spec"]["template"]["spec"]
for item in manifest["items"]
if item["kind"] in {"Deployment", "Job"}
and item["metadata"]["labels"].get("app.kubernetes.io/component")
in {"api", "worker", "scheduler", "migration"}
]
web = next(
item
for item in manifest["items"]
if item["kind"] == "Deployment"
and item["metadata"]["labels"].get("app.kubernetes.io/component")
== "web"
)
self.assertTrue(backend_pods)
for pod in backend_pods:
self.assertIn(
{
"name": "s3-ca",
"secret": {
"secretName": "govoplan-s3-ca",
"items": [{"key": "ca.crt", "path": "s3-ca.crt"}],
},
},
pod["volumes"],
)
for container in [*pod.get("initContainers", []), *pod["containers"]]:
self.assertIn(
{
"name": "AWS_CA_BUNDLE",
"value": "/etc/govoplan/trust/s3-ca.crt",
},
container["env"],
)
self.assertIn(
{
"name": "s3-ca",
"mountPath": "/etc/govoplan/trust",
"readOnly": True,
},
container["volumeMounts"],
)
self.assertNotIn(
"s3-ca",
{
volume["name"]
for volume in web["spec"]["template"]["spec"]["volumes"]
},
)
with self.assertRaisesRegex(ValueError, "S3 CA secret"):
render_kubernetes(
spec,
environment,
s3_ca_secret_name="INVALID_NAME",
backup_required=False,
)
def test_kubernetes_export_splits_worker_queues_and_rejects_capacity_overrun(
self,
) -> None:
+239
View File
@@ -0,0 +1,239 @@
from __future__ import annotations
from contextlib import redirect_stdout
import io
import json
from pathlib import Path
import stat
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import MagicMock
META_ROOT = Path(__file__).resolve().parents[1]
LAB_TOOLS = META_ROOT / "tools" / "lab"
if str(LAB_TOOLS) not in sys.path:
sys.path.insert(0, str(LAB_TOOLS))
from govoplan_lab.cli import main # noqa: E402
from govoplan_lab.config import LabConfigError, load_config # noqa: E402
from govoplan_lab.lifecycle import ( # noqa: E402
CommandRunner,
LabOperationError,
_assert_domain_owned,
_domain_description,
_render_kubectl_wrapper,
destroy,
)
from govoplan_lab.render import ( # noqa: E402
render_k3s_config,
render_registry_config,
render_state_compose,
write_private,
)
REHEARSAL_CONFIG = LAB_TOOLS / "govoplan-lab.example.toml"
ACCEPTANCE_CONFIG = LAB_TOOLS / "govoplan-lab.acceptance.example.toml"
class KubernetesLabTests(unittest.TestCase):
def test_example_inventories_describe_their_evidence_boundary(self) -> None:
rehearsal = load_config(REHEARSAL_CONFIG)
acceptance = load_config(ACCEPTANCE_CONFIG)
self.assertEqual("rehearsal", rehearsal.mode)
self.assertFalse(rehearsal.evidence_capable)
self.assertEqual(2, len(rehearsal.workers))
self.assertEqual("acceptance", acceptance.mode)
self.assertTrue(acceptance.evidence_capable)
self.assertEqual(3, len({node.hypervisor for node in acceptance.nodes}))
self.assertEqual(3, len({node.failure_domain for node in acceptance.nodes}))
def test_acceptance_inventory_rejects_collapsed_worker_failure_domains(self) -> None:
source = ACCEPTANCE_CONFIG.read_text(encoding="utf-8")
collapsed = source.replace(
'hypervisor = "lab-admin@hypervisor-b.example.org"',
'hypervisor = "lab-admin@hypervisor-a.example.org"',
).replace('failure_domain = "rack-b"', 'failure_domain = "rack-a"')
with tempfile.TemporaryDirectory(prefix="govoplan-lab-config-") as directory:
path = Path(directory) / "lab.toml"
path.write_text(collapsed, encoding="utf-8")
with self.assertRaisesRegex(LabConfigError, "acceptance mode"):
load_config(path)
def test_create_without_apply_is_a_non_mutating_preview(self) -> None:
output = io.StringIO()
with redirect_stdout(output):
exit_code = main(["--config", str(REHEARSAL_CONFIG), "create"])
self.assertEqual(0, exit_code)
self.assertIn("Dry run: create", output.getvalue())
self.assertIn("Re-run with --apply", output.getvalue())
def test_local_hypervisor_uses_system_libvirt_without_sudo(self) -> None:
config = load_config(REHEARSAL_CONFIG)
runner = CommandRunner(config)
runner.run = MagicMock(
return_value=subprocess.CompletedProcess([], 0, stdout=b"", stderr=b"")
)
runner.hypervisor(config.nodes[0], ["virsh", "dominfo", "test-domain"])
runner.run.assert_called_once_with(
[
"virsh",
"--connect",
"qemu:///system",
"dominfo",
"test-domain",
],
capture=False,
check=True,
timeout=None,
)
def test_local_hypervisor_file_operations_do_not_use_sudo(self) -> None:
config = load_config(REHEARSAL_CONFIG)
runner = CommandRunner(config)
runner.run = MagicMock(
return_value=subprocess.CompletedProcess([], 0, stdout=b"", stderr=b"")
)
runner.hypervisor(config.nodes[0], ["install", "-d", "/tmp/lab"])
runner.run.assert_called_once_with(
["install", "-d", "/tmp/lab"],
capture=False,
check=True,
timeout=None,
)
def test_kubectl_wrapper_quotes_remote_arguments(self) -> None:
wrapper = _render_kubectl_wrapper(
["ssh", "-i", "/tmp/lab key", "govoplan@example.test"]
)
self.assertIn("shlex.join(_REMOTE)", wrapper)
self.assertIn('["sudo", "--", "k3s", "kubectl", *sys.argv[1:]]', wrapper)
self.assertNotIn('kubectl \"$@\"', wrapper)
def test_destroy_requires_the_exact_lab_name(self) -> None:
config = load_config(REHEARSAL_CONFIG)
with self.assertRaisesRegex(LabOperationError, "--confirm"):
destroy(
config,
apply=True,
confirmation="wrong-lab",
purge_local_state=False,
)
def test_enroll_admin_without_apply_is_a_non_mutating_preview(self) -> None:
output = io.StringIO()
with redirect_stdout(output):
exit_code = main(
[
"--config",
str(REHEARSAL_CONFIG),
"enroll-admin",
"--email",
"owner@example.test",
]
)
self.assertEqual(0, exit_code)
self.assertIn("Dry run: enroll-admin", output.getvalue())
self.assertIn("owner@example.test", output.getvalue())
def test_domain_ownership_requires_marker_and_expected_disks(self) -> None:
config = load_config(REHEARSAL_CONFIG)
node = config.nodes[0]
node_directory = f"{config.vm_image_directory}/{config.name}/{node.name}"
runner = MagicMock()
runner.hypervisor.side_effect = [
subprocess.CompletedProcess(
[],
0,
stdout=(_domain_description(config, node) + "\n").encode(),
stderr=b"",
),
subprocess.CompletedProcess(
[],
0,
stdout=(
f"file disk vda {node_directory}/root.qcow2\n"
f"file cdrom sda {node_directory}/seed.img\n"
).encode(),
stderr=b"",
),
]
_assert_domain_owned(config, runner, node)
runner.hypervisor.side_effect = [
subprocess.CompletedProcess(
[],
0,
stdout=b"unrelated domain\n",
stderr=b"",
)
]
with self.assertRaisesRegex(LabOperationError, "ownership marker"):
_assert_domain_owned(config, runner, node)
def test_state_compose_uses_only_supplied_pinned_images(self) -> None:
names = ("postgres", "redis", "garage", "managed_ingress", "test_mail")
images = {
name: f"registry.example.test/{name}@sha256:{index:064x}"
for index, name in enumerate(names, start=1)
}
compose = json.loads(render_state_compose(images))
self.assertEqual(images["postgres"], compose["services"]["postgres"]["image"])
self.assertEqual(images["redis"], compose["services"]["redis"]["image"])
self.assertEqual(images["garage"], compose["services"]["garage"]["image"])
self.assertEqual(
images["managed_ingress"], compose["services"]["s3-tls"]["image"]
)
self.assertEqual(
images["test_mail"], compose["services"]["test-mail"]["image"]
)
def test_k3s_workers_join_the_primary_control_and_receive_failure_labels(
self,
) -> None:
config = load_config(ACCEPTANCE_CONFIG)
worker = config.workers[0]
rendered = render_k3s_config(config, worker, cluster_token="test-token")
self.assertIn(f'server: "https://{config.primary_control.address}:6443"', rendered)
self.assertIn(
f'topology.govoplan.add-ideas.de/failure-domain={worker.failure_domain}',
rendered,
)
self.assertNotIn("cluster-init", rendered)
def test_registry_credentials_are_all_or_nothing(self) -> None:
self.assertEqual("", render_registry_config("", ""))
with self.assertRaisesRegex(ValueError, "supplied together"):
render_registry_config("publisher", "")
rendered = render_registry_config("publisher", "secret")
self.assertIn('"git.add-ideas.de"', rendered)
self.assertIn("publisher", rendered)
def test_private_writer_enforces_owner_only_permissions(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-lab-private-") as directory:
path = Path(directory) / "nested" / "secret.txt"
write_private(path, "secret\n")
self.assertEqual(0o600, stat.S_IMODE(path.stat().st_mode))
if __name__ == "__main__":
unittest.main()
+113
View File
@@ -0,0 +1,113 @@
from __future__ import annotations
import json
from pathlib import Path
import shutil
import subprocess
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
class ModulePackageWorkflowTests(unittest.TestCase):
def test_template_enforces_tag_version_hash_and_registry_contract(self) -> None:
workflow = (
META_ROOT / "tools/repo/templates/module-package-release.yml"
).read_text(encoding="utf-8")
self.assertIn("GITEA_REPOSITORY: ${{ gitea.repository }}", workflow)
self.assertNotIn("tag_protections", workflow)
self.assertNotIn("secrets.GITEA_TOKEN", workflow)
self.assertIn("git merge-base --is-ancestor", workflow)
self.assertIn("does not match", workflow)
self.assertIn("package-artifacts.json", workflow)
self.assertIn("api/packages/GovOPlaN/pypi", workflow)
self.assertIn("api/packages/GovOPlaN/npm", workflow)
self.assertIn('npm publish "./${webui_packages[0]}"', workflow)
self.assertIn("Check immutable registry state", workflow)
self.assertIn('files[0].get("sha256") != expected_sha256', workflow)
self.assertIn('if [[ "$PUBLISH_PYPI" == 1 ]]', workflow)
self.assertIn('[[ "$PUBLISH_NPM" == 1 ]]', workflow)
self.assertIn("GOVOPLAN_PACKAGE_TOKEN", workflow)
self.assertIn("must resolve to an exact registry version", workflow)
self.assertIn("git\\\\.add-ideas\\\\.de/(?:GovOPlaN|add-ideas)", workflow)
self.assertIn("release package identity does not match", workflow)
self.assertNotIn("Generic", workflow)
@unittest.skipUnless(shutil.which("node"), "Node.js is required")
def test_webui_publication_normalizes_internal_git_dependencies(self) -> None:
workflow = (
META_ROOT / "tools/repo/templates/module-package-release.yml"
).read_text(encoding="utf-8")
marker = " node <<'NODE'\n"
script = workflow.split(marker, 1)[1].split("\n NODE", 1)[0]
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
package_dir = root / ".package-webui"
package_dir.mkdir()
package_path = package_dir / "package.json"
package_path.write_text(
json.dumps(
{
"name": "@govoplan/core-webui",
"version": "0.1.14",
"private": True,
"dependencies": {
"@govoplan/access-webui": (
"git+ssh://git@git.add-ideas.de/GovOPlaN/"
"govoplan-access.git#v0.1.11"
),
"@govoplan/admin-webui": (
"git+ssh://git@git.add-ideas.de/add-ideas/"
"govoplan-admin.git#v0.1.8"
)
},
}
),
encoding="utf-8",
)
subprocess.run(
["node"],
input=script,
cwd=root,
check=True,
text=True,
capture_output=True,
)
package = json.loads(package_path.read_text(encoding="utf-8"))
self.assertNotIn("private", package)
self.assertEqual(
"0.1.11", package["dependencies"]["@govoplan/access-webui"]
)
self.assertEqual(
"0.1.8", package["dependencies"]["@govoplan/admin-webui"]
)
def test_sync_script_only_targets_packageable_govoplan_repositories(self) -> None:
namespace: dict[str, object] = {
"__file__": str(META_ROOT / "tools/repo/sync-module-package-workflows.py"),
"__name__": "test_sync_module_package_workflows",
}
script = (META_ROOT / "tools/repo/sync-module-package-workflows.py").read_text(
encoding="utf-8"
)
exec(compile(script, str(namespace["__file__"]), "exec"), namespace)
with tempfile.TemporaryDirectory() as temporary:
parent = Path(temporary)
package_repositories = namespace["package_repositories"]
# The production inventory is authoritative, so a temporary parent
# only exposes matching paths that are present in that inventory.
known = parent / "govoplan-core"
known.mkdir()
(known / "pyproject.toml").write_text("[project]\n", encoding="utf-8")
self.assertEqual(package_repositories(parent), (known,))
if __name__ == "__main__":
unittest.main()
+180
View File
@@ -0,0 +1,180 @@
from __future__ import annotations
from io import BytesIO
import importlib.util
import json
from pathlib import Path
import sys
import tarfile
import tempfile
import tomllib
import unittest
import zipfile
ROOT = Path(__file__).resolve().parents[1]
def _load(name: str, path: Path):
spec = importlib.util.spec_from_file_location(name, path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
return module
PACKAGE_SET = _load(
"generate_release_package_set",
ROOT / "tools/release/generate-release-package-set.py",
)
ARTIFACTS = _load(
"resolve_package_artifacts",
ROOT / "tools/release/resolve-package-artifacts.py",
)
class PackageRegistryReleaseTests(unittest.TestCase):
def test_current_release_sources_form_a_hash_bound_package_set(self) -> None:
core_version = tomllib.loads(
(ROOT.parent / "govoplan-core/pyproject.toml").read_text(encoding="utf-8")
)["project"]["version"]
payload = PACKAGE_SET.generate_package_set(
core_version=core_version,
requirements=ROOT / "requirements-release.txt",
workspace=ROOT.parent,
)
self.assertEqual("1", payload["schema_version"])
self.assertEqual("govoplan-core", payload["python"][0]["name"])
self.assertIn(
"@govoplan/core-webui",
{item["name"] for item in payload["webui"]},
)
unsigned = dict(payload)
digest = unsigned.pop("package_set_sha256")
self.assertEqual(ARTIFACTS._canonical_sha256(unsigned), digest)
def test_wheel_and_webui_artifacts_are_verified_by_embedded_identity(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-package-artifacts-") as value:
root = Path(value)
wheels = root / "wheels"
webui = root / "webui"
wheels.mkdir()
webui.mkdir()
wheel = wheels / "govoplan_demo-1.2.3-py3-none-any.whl"
with zipfile.ZipFile(wheel, "w") as archive:
archive.writestr(
"govoplan_demo-1.2.3.dist-info/METADATA",
"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: 1.2.3\n",
)
package_json = json.dumps(
{"name": "@govoplan/demo-webui", "version": "1.2.3"}
).encode("utf-8")
npm = webui / "govoplan-demo-webui-1.2.3.tgz"
with tarfile.open(npm, "w:gz") as archive:
member = tarfile.TarInfo("package/package.json")
member.size = len(package_json)
archive.addfile(member, BytesIO(package_json))
source = {
"version": "1.2.3",
"repository": "govoplan-demo",
"tag": "v1.2.3",
"commit": "1" * 40,
}
python_rows = ARTIFACTS._verify_wheels(
({"name": "govoplan-demo", "extras": ["server"], **source},), wheels
)
webui_rows = ARTIFACTS._verify_webui(
({"name": "@govoplan/demo-webui", **source},), webui
)
self.assertEqual("govoplan-demo", python_rows[0]["name"])
self.assertEqual(["server"], python_rows[0]["extras"])
self.assertEqual("@govoplan/demo-webui", webui_rows[0]["name"])
self.assertTrue(str(webui_rows[0]["integrity"]).startswith("sha512-"))
def test_package_set_rejects_argument_shaped_package_names(self) -> None:
payload = {
"schema_version": "1",
"release_version": "1.2.3",
"registries": {
"python": "https://packages.example.test/pypi/simple",
"npm": "https://packages.example.test/npm/",
},
"python": [
{
"name": "--index-url",
"version": "1.2.3",
"repository": "govoplan-demo",
"extras": [],
"tag": "v1.2.3",
"commit": "1" * 40,
}
],
"webui": [
{
"name": "@govoplan/demo-webui",
"version": "1.2.3",
"repository": "govoplan-demo",
"tag": "v1.2.3",
"commit": "1" * 40,
}
],
}
payload["package_set_sha256"] = ARTIFACTS._canonical_sha256(payload)
with tempfile.TemporaryDirectory() as value:
path = Path(value) / "packages.json"
path.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(
ARTIFACTS.PackageArtifactError, "invalid identity"
):
ARTIFACTS._load_package_set(path)
def test_runtime_workflow_consumes_registry_artifacts_and_publishes_lock(self) -> None:
workflow = (ROOT / ".gitea/workflows/runtime-distribution.yml").read_text(
encoding="utf-8"
)
self.assertIn("resolve-package-artifacts.py", workflow)
self.assertIn("package-artifacts.lock.json", workflow)
self.assertIn(
"--package-lock runtime-output/package-artifacts.lock.json",
workflow,
)
self.assertIn('PYTHON="$PWD/.runtime-build/bin/python"', workflow)
self.assertNotIn(
"pip wheel --no-deps --wheel-dir runtime-output/local-wheels",
workflow,
)
def test_developer_meta_package_matches_workspace_versions(self) -> None:
script = _load(
"generate_developer_meta_package",
ROOT / "tools/release/generate-developer-meta-package.py",
)
expected = script.render(
workspace=ROOT.parent,
requirements=ROOT / "requirements-release.txt",
)
actual = (ROOT / "packages/govoplan-meta/pyproject.toml").read_text(
encoding="utf-8"
)
self.assertEqual(expected, actual)
def test_meta_package_workflow_supports_hash_safe_tag_retry(self) -> None:
workflow = (
ROOT / ".gitea/workflows/publish-developer-meta-package.yml"
).read_text(encoding="utf-8")
self.assertIn("workflow_dispatch:", workflow)
self.assertIn("TRIGGER_TAG: ${{ gitea.ref_name }}", workflow)
self.assertNotIn("GITEA_REF_NAME", workflow)
self.assertIn('refs/tags/{tag}^{{commit}}', workflow)
self.assertIn("already exists with a different SHA-256", workflow)
self.assertIn("PUBLISH_PYPI", workflow)
if __name__ == "__main__":
unittest.main()
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
TOOLS_ROOT = META_ROOT / "tools" / "gitea"
if str(TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(TOOLS_ROOT))
SCRIPT = TOOLS_ROOT / "gitea-dispatch-package-set.py"
SPEC = importlib.util.spec_from_file_location("gitea_dispatch_package_set", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
MODULE = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = MODULE
SPEC.loader.exec_module(MODULE)
class PackageSetDispatchTests(unittest.TestCase):
def test_meta_package_resolves_to_exact_tagged_repository_targets(self) -> None:
targets = MODULE.package_targets()
self.assertEqual(66, len(targets))
self.assertEqual(66, len({target.distribution for target in targets}))
by_name = {target.distribution: target for target in targets}
self.assertEqual("v0.1.14", by_name["govoplan-core"].tag)
self.assertEqual("v0.1.8", by_name["govoplan-access"].tag)
self.assertTrue(by_name["govoplan-core"].tag_exists)
self.assertTrue(by_name["govoplan-access"].has_webui)
self.assertEqual(
"@govoplan/access-webui",
by_name["govoplan-access"].webui_package,
)
if __name__ == "__main__":
unittest.main()
+136
View File
@@ -141,6 +141,34 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
self.assertEqual(1, result["summary"]["stale_endpoint_declarations"])
self.assertIsNone(result["api"]["backend_endpoints"][0]["surface"])
def test_endpoint_only_strict_mode_does_not_fail_on_translation_debt(
self,
) -> None:
result = {
"translation_health": {"missing_catalog_entries": ["missing.key"]},
"api": {
"unclassified_endpoints": [],
"stale_endpoint_declarations": [],
},
}
self.assertEqual(
[],
inventory._strict_failures(
result,
check_translations=False,
check_endpoints=True,
),
)
self.assertEqual(
["used translation keys are missing from generated catalogs"],
inventory._strict_failures(
result,
check_translations=True,
check_endpoints=True,
),
)
def test_fastapi_route_scanner_includes_router_prefix(self) -> None:
tree = ast.parse(
"""
@@ -171,6 +199,114 @@ def read_item(item_id: str):
},
)
def test_source_declarations_normalize_stable_control_and_contribution_ids(
self,
) -> None:
webui = {
"fields": [
{
"repository": "govoplan-example",
"file": "webui/src/Example.tsx",
"line": 12,
"column": 3,
"id": "govoplan-example.field.example.name.abc123",
"idSource": "source_anchor",
"explicitId": None,
"context": "Example",
"helpId": "govoplan-example.field.example.name.abc123.help",
"helpDynamic": False,
}
],
"actions": [],
"contributions": [
{
"repository": "govoplan-example",
"file": "webui/src/module.ts",
"line": 20,
"column": 5,
"kind": "frontend_route",
"id": "/examples/:exampleId",
"path": "/examples/:exampleId",
}
],
"translationCatalog": {"en": {}, "de": {}},
}
manifests = [{"repository": "govoplan-example", "id": "examples"}]
declarations = inventory._source_interface_declarations(webui, manifests)
keys = {item["key"] for item in declarations}
self.assertIn("field:examples.field.example.name.abc123", keys)
self.assertIn(
"help:examples.field.example.name.abc123.help",
keys,
)
self.assertIn(
"frontend_route:examples.route.examples.exampleid",
keys,
)
def test_declaration_health_rejects_duplicate_and_undeclared_source_ids(
self,
) -> None:
declaration = {
"key": "frontend_route:example.route.unlisted",
"id": "example.route.unlisted",
"module_id": "example",
"kind": "frontend_route",
"origin": "webui_contribution",
}
manifests = [
{
"id": "example",
"repository": "govoplan-example",
"interface_catalog": {"declarations": []},
}
]
health = inventory._declaration_health(
[declaration, dict(declaration)],
manifests,
)
self.assertEqual(1, len(health["duplicate_ids"]))
self.assertEqual(1, len(health["undeclared_source_surfaces"]))
def test_runtime_snapshot_comparison_accepts_an_installed_subset(self) -> None:
manifests = [
{
"id": "one",
"interface_catalog": {
"contract_version": "1",
"module_id": "one",
"module_version": "1.0.0",
"digest": "sha256:one",
},
},
{
"id": "two",
"interface_catalog": {
"contract_version": "1",
"module_id": "two",
"module_version": "1.0.0",
"digest": "sha256:two",
},
},
]
snapshot = {
"contract_version": "1",
"modules": [dict(manifests[1]["interface_catalog"])],
}
comparison = inventory._compare_runtime_snapshot(snapshot, manifests)
self.assertEqual(["two"], comparison["matched_modules"])
self.assertEqual([], comparison["mismatches"])
snapshot["modules"][0]["digest"] = "sha256:changed"
comparison = inventory._compare_runtime_snapshot(snapshot, manifests)
self.assertEqual("digest_mismatch", comparison["mismatches"][0]["reason"])
if __name__ == "__main__":
unittest.main()
+20 -3
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import sys
import tomllib
import unittest
from pathlib import Path
@@ -26,6 +27,13 @@ from govoplan_release.catalog_entry_synthesis import ( # noqa: E402
from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402
def repository_version(name: str) -> str:
payload = tomllib.loads(
(META_ROOT.parent / name / "pyproject.toml").read_text(encoding="utf-8")
)
return str(payload["project"]["version"])
class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
def test_catalog_entry_preserves_architecture_and_provider_declarations(
self,
@@ -92,6 +100,12 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
)
self.assertEqual("vertical_slice", entry["architecture"]["maturity"])
self.assertEqual(
"contract_only",
entry["information_governance"]["dimensions"]["retention"][
"adoption"
],
)
self.assertEqual(
"external_mirror",
entry["external_providers"][0]["objects"][0][
@@ -116,9 +130,12 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
changes = apply_repo_updates(
payload,
repo_versions={
"govoplan-addresses": "0.1.9",
"govoplan-poll": "0.1.11",
"govoplan-scheduling": "0.1.11",
name: repository_version(name)
for name in (
"govoplan-addresses",
"govoplan-poll",
"govoplan-scheduling",
)
},
repo_contracts={},
repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN",
+55
View File
@@ -29,17 +29,72 @@ class ReleaseEntrypointGateTests(unittest.TestCase):
workflow = script[confirm:]
source_gate = workflow.index("run_version_alignment_gate source")
baseline = workflow.index("record_migration_release_baseline")
first_commit = workflow.index('run git -C "$repo" commit')
lock_generation = workflow.index("generate_release_lock")
full_gate = workflow.index("run_version_alignment_gate", source_gate + 1)
first_push = workflow.index('run git -C "$repo" push')
self.assertLess(baseline, source_gate)
self.assertLess(source_gate, first_commit)
self.assertLess(first_commit, lock_generation)
self.assertLess(lock_generation, full_gate)
self.assertLess(full_gate, first_push)
self.assertLess(manifest_gate, confirm)
def test_lockstep_release_pushes_meta_package_after_core(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
module_push = script.index('for repo in "${MODULE_REPOS[@]}"; do\n run git -C "$repo" push')
core_push = script.index('run git -C "$ROOT" push', module_push)
support_push = script.index('for repo in "${SUPPORT_REPOS[@]}"; do\n run git -C "$repo" push', core_push)
self.assertLess(module_push, core_push)
self.assertLess(core_push, support_push)
def test_default_migration_preflight_accepts_new_release_heads(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
audit_function = script[
script.index("run_migration_release_audit()") :
script.index("record_migration_release_baseline()")
]
self.assertNotIn("--strict-if-baseline", audit_function)
self.assertIn('command+=("--strict")', audit_function)
def test_source_gate_does_not_require_tags_before_they_are_created(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
gate = script[
script.index("run_version_alignment_gate()") :
script.index("run_manifest_shape_gate()")
]
self.assertIn('command+=(--source-metadata-only)', gate)
self.assertIn('else\n command+=(--release-composition)', gate)
def test_version_updater_targets_canonical_runtime_declarations(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
self.assertIn("^manifest\\s*=\\s*ModuleManifest", script)
self.assertIn("could not update module version declaration", script)
self.assertIn("update_package_init_versions", script)
self.assertIn("synchronize-webui-package-metadata.py", script)
self.assertIn('"peerDependenciesMeta",', script)
self.assertLess(
script.index('synchronize-webui-package-metadata.py" --repo "$repo"'),
script.index('synchronize_lockfile_root "$package_path"', script.index('synchronize-webui-package-metadata.py" --repo "$repo"')),
)
self.assertNotIn("could not update ModuleManifest.version", script)
def test_release_lock_refreshes_candidate_govoplan_metadata(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-lock.sh").read_text()
self.assertEqual(2, script.count('"npm_config_cache=$TMP_DIR/npm-cache"'))
self.assertNotIn(
'cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"',
script,
)
self.assertIn('cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"', script)
def test_source_catalog_generator_enforces_explicit_repo_versions(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text()
+8
View File
@@ -93,6 +93,14 @@ class RuntimeDistributionTests(unittest.TestCase):
with self.assertRaisesRegex(DistributionError, "active trusted key"):
verify_manifest(unknown, self.keyring, now=self.now)
with self.assertRaisesRegex(DistributionError, "expected 'candidate'"):
verify_manifest(
self._manifest(),
self.keyring,
expected_channel="candidate",
now=self.now,
)
def test_offline_image_index_is_complete_and_digest_bound(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-offline-images-") as value:
root = Path(value)
+62 -2
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import argparse
import hashlib
import importlib.util
import json
import os
@@ -120,6 +121,22 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
):
self.assertIn(temporary_path, nginx)
def test_web_runtime_resolves_the_configured_api_service_at_startup(self) -> None:
nginx = (ROOT / "tools/release/runtime/nginx.conf").read_text(
encoding="utf-8"
)
dockerfile = (ROOT / "tools/release/runtime/Dockerfile.web").read_text(
encoding="utf-8"
)
entrypoint = (ROOT / "tools/release/runtime/web-entrypoint.sh").read_text(
encoding="utf-8"
)
self.assertIn("proxy_pass ${GOVOPLAN_API_UPSTREAM};", nginx)
self.assertIn("nginx.conf.template", dockerfile)
self.assertIn("govoplan-web-entrypoint", dockerfile)
self.assertIn("envsubst '${GOVOPLAN_API_UPSTREAM}'", entrypoint)
def test_workflow_verifies_portable_bootstrap_artifacts_before_execution(
self,
) -> None:
@@ -162,7 +179,7 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
)
self.assertNotIn(".platforms[\\\"linux/amd64\\\"]", workflow)
def test_workflow_binds_the_release_tag_to_the_workflow_commit(self) -> None:
def test_workflow_binds_distribution_to_the_peeled_release_tag(self) -> None:
workflow = (ROOT / ".gitea/workflows/runtime-distribution.yml").read_text(
encoding="utf-8"
)
@@ -170,7 +187,21 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
encoding="utf-8"
)
self.assertIn("SOURCE_COMMIT: ${{ gitea.sha }}", workflow)
self.assertIn(
'git fetch --force --no-tags origin "refs/tags/v$VERSION:refs/tags/v$VERSION"',
workflow,
)
self.assertIn(
'git rev-parse "v$VERSION^{commit}" > runtime-output/release-source-commit',
workflow,
)
self.assertEqual(
2,
workflow.count(
'SOURCE_COMMIT="$(cat runtime-output/release-source-commit)"'
),
)
self.assertNotIn("SOURCE_COMMIT: ${{ gitea.sha }}", workflow)
self.assertIn('--target-commit "$SOURCE_COMMIT"', workflow)
self.assertIn('"target_commitish": target_commit', publisher)
self.assertIn("self._resolve_commit(tag) != target_commit", publisher)
@@ -298,12 +329,37 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
(root / "web.json").write_text(json.dumps(web_metadata))
deployer = root / "govoplan-deploy.pyz"
deployer.write_bytes(b"zipapp")
package_lock = root / "package-artifacts.lock.json"
package_lock_value = {
"schema_version": "1",
"release_version": "1.2.3",
"python": [
{
"name": "govoplan-core",
"version": "1.2.3",
"sha256": "8" * 64,
}
],
"webui": [],
}
package_lock_value["lock_sha256"] = hashlib.sha256(
json.dumps(
package_lock_value,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
).hexdigest()
package_lock.write_text(
json.dumps(package_lock_value) + "\n",
encoding="utf-8",
)
args = argparse.Namespace(
composition=root / "composition.json",
api_metadata=root / "api.json",
web_metadata=root / "web.json",
deployer=deployer,
deployer_url="https://downloads.example/govoplan-deploy.pyz",
package_lock=package_lock,
artifact_base_url="https://downloads.example/runtime/v1.2.3",
source_commit="f" * 40,
version="1.2.3",
@@ -327,6 +383,10 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
)
self.assertTrue((root / "evidence/api-sbom.cdx.json").is_file())
self.assertTrue((root / "evidence/web-provenance.json").is_file())
self.assertEqual(
hashlib.sha256(package_lock.read_bytes()).hexdigest(),
descriptor["package_lock"]["sha256"],
)
def test_rejects_incomplete_oci_index(self) -> None:
with self.assertRaisesRegex(ValueError, "linux/amd64 and linux/arm64"):
+6
View File
@@ -75,6 +75,12 @@ class RuntimeImageContextTests(unittest.TestCase):
).read_text()
)
self.assertEqual(composition, published)
self.assertEqual(
(
SCRIPT.parent / "runtime" / "web-entrypoint.sh"
).read_bytes(),
(root / "context" / "web-entrypoint.sh").read_bytes(),
)
def test_rejects_missing_required_module(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-runtime-context-") as value:
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "release" / "synchronize-webui-package-metadata.py"
class SynchronizeWebuiPackageMetadataTests(unittest.TestCase):
def test_copies_peer_contract_without_changing_publish_paths(self) -> None:
with tempfile.TemporaryDirectory() as directory:
repo = Path(directory)
(repo / "webui").mkdir()
(repo / "package.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"exports": {".": "./webui/src/index.ts"},
"peerDependencies": {"vite": "^6"},
}
)
)
(repo / "webui" / "package.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"peerDependencies": {"vite": "^7"},
"peerDependenciesMeta": {"vite": {"optional": True}},
}
)
)
subprocess.run(
[sys.executable, str(SCRIPT), "--repo", str(repo)],
check=True,
capture_output=True,
text=True,
)
package = json.loads((repo / "package.json").read_text())
self.assertEqual({"vite": "^7"}, package["peerDependencies"])
self.assertEqual({"vite": {"optional": True}}, package["peerDependenciesMeta"])
self.assertEqual({".": "./webui/src/index.ts"}, package["exports"])
def test_leaves_distinct_root_and_webui_packages_separate(self) -> None:
with tempfile.TemporaryDirectory() as directory:
repo = Path(directory)
(repo / "webui").mkdir()
(repo / "package.json").write_text(json.dumps({"name": "@govoplan/one"}))
(repo / "webui" / "package.json").write_text(json.dumps({"name": "@govoplan/two"}))
subprocess.run(
[sys.executable, str(SCRIPT), "--repo", str(repo)],
check=True,
capture_output=True,
text=True,
)
root = json.loads((repo / "package.json").read_text())
self.assertEqual("@govoplan/one", root["name"])
self.assertNotIn("peerDependencies", root)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,194 @@
#!/usr/bin/env python3
"""Generate an independently held Ed25519 assessment-authority keypair."""
from __future__ import annotations
import argparse
import base64
from datetime import UTC, datetime, timedelta
import json
import os
from pathlib import Path
import re
import stat
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
KEY_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,159}$")
PROOF_SCOPES = (
"target_environment",
"external_providers",
"accessibility",
"privacy",
"security",
"operations",
"recovery",
"production_approval",
)
PURPOSES = {
"proof": (
"govoplan.capability-fit-proof-authorities",
"./capability-fit-proof-authority-keyring.schema.json",
),
"installer": (
"govoplan.installer-receipt-authorities",
"./installer-receipt-authority-keyring.schema.json",
),
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--purpose", choices=tuple(PURPOSES), required=True)
parser.add_argument("--key-id", required=True)
parser.add_argument(
"--scope",
action="append",
choices=PROOF_SCOPES,
default=[],
help="Authorized proof scope; repeat as needed. Not used for installer keys.",
)
parser.add_argument("--private-key", type=Path, required=True)
parser.add_argument("--keyring", type=Path, required=True)
parser.add_argument(
"--valid-days",
type=int,
default=365,
help="Validity from generation time (default: 365 days).",
)
parser.add_argument(
"--status",
choices=("active", "next"),
default="active",
)
args = parser.parse_args(argv)
if not KEY_ID_PATTERN.fullmatch(args.key_id):
parser.error("--key-id must be a valid opaque identifier")
if args.valid_days < 1 or args.valid_days > 3660:
parser.error("--valid-days must be between 1 and 3660")
scopes = _resolve_scopes(parser, purpose=args.purpose, scopes=args.scope)
private_path = args.private_key.expanduser().resolve()
keyring_path = args.keyring.expanduser().resolve()
_require_fresh_output(parser, private_path, label="private key")
_require_fresh_output(parser, keyring_path, label="keyring")
_require_private_directory(parser, private_path.parent)
_require_output_directory(parser, keyring_path.parent)
private_key = Ed25519PrivateKey.generate()
private_bytes = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
public_bytes = private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
public_base64 = base64.b64encode(public_bytes).decode("ascii")
now = datetime.now(UTC).replace(microsecond=0)
not_after = now + timedelta(days=args.valid_days)
purpose, schema = PURPOSES[args.purpose]
keyring = {
"$schema": schema,
"schema_version": "0.1.0",
"purpose": purpose,
"keys": [
{
"key_id": args.key_id,
"status": args.status,
"public_key": public_base64,
"allowed_scopes": scopes,
"not_before": _rfc3339(now),
"not_after": _rfc3339(not_after),
}
],
}
_write_new_private_file(private_path, private_bytes)
try:
_write_new_private_file(
keyring_path,
(json.dumps(keyring, indent=2, sort_keys=True) + "\n").encode("utf-8"),
)
except BaseException:
private_path.unlink(missing_ok=True)
keyring_path.unlink(missing_ok=True)
raise
print(f"private_key={private_path}")
print(f"keyring={keyring_path}")
print(f"key_id={args.key_id}")
print(f"allowed_scopes={','.join(scopes)}")
return 0
def _resolve_scopes(
parser: argparse.ArgumentParser, *, purpose: str, scopes: list[str]
) -> list[str]:
if purpose == "installer":
if scopes:
parser.error("installer authorities do not accept --scope")
return ["installed_release_origin"]
unique = list(dict.fromkeys(scopes))
if not unique:
parser.error("proof authorities require at least one --scope")
return unique
def _require_fresh_output(
parser: argparse.ArgumentParser, path: Path, *, label: str
) -> None:
if path.exists() or path.is_symlink():
parser.error(f"{label.capitalize()} output already exists: {path}")
def _require_private_directory(
parser: argparse.ArgumentParser, directory: Path
) -> None:
_require_output_directory(parser, directory)
mode = stat.S_IMODE(directory.stat().st_mode)
if mode & (stat.S_IRWXG | stat.S_IRWXO):
parser.error(
"Private-key parent directory must not be accessible by group or others"
)
def _require_output_directory(
parser: argparse.ArgumentParser, directory: Path
) -> None:
try:
metadata = directory.lstat()
except OSError as exc:
parser.error(f"Output parent directory is unavailable: {directory}")
raise AssertionError from exc
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):
parser.error(f"Output parent must be a real directory: {directory}")
def _write_new_private_file(path: Path, payload: bytes) -> None:
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
descriptor = os.open(path, flags, 0o600)
try:
with os.fdopen(descriptor, "wb", closefd=False) as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
metadata = os.fstat(descriptor)
if not stat.S_ISREG(metadata.st_mode) or stat.S_IMODE(metadata.st_mode) != 0o600:
raise OSError("Authority output could not be secured")
finally:
os.close(descriptor)
def _rfc3339(value: datetime) -> str:
return value.isoformat().replace("+00:00", "Z")
if __name__ == "__main__":
raise SystemExit(main())
@@ -15,6 +15,7 @@ from govoplan_core.core.dataflows import (
dataflow_run_lifecycle,
)
from govoplan_core.core.automation import AutomationPrincipalResolution
from govoplan_core.core.change_sequence import ChangeSequenceEntry
from govoplan_core.core.access import (
CAPABILITY_AUTH_AUTOMATION_PRINCIPAL_PROVIDER,
)
@@ -79,6 +80,7 @@ def main() -> int:
Base.metadata.create_all(
engine,
tables=[
ChangeSequenceEntry.__table__,
DistributedLease.__table__,
RecoveryOperation.__table__,
RecoveryCheckpoint.__table__,
+4
View File
@@ -40,6 +40,10 @@ GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
cd "$META_ROOT"
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
"$PYTHON" tools/repo/sync-module-package-workflows.py --check
"$PYTHON" tools/release/generate-developer-meta-package.py --check
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
"$PYTHON" -m unittest tests.test_deployment_installer
"$PYTHON" -m unittest tests.test_capability_fit_evidence
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
+54
View File
@@ -128,6 +128,13 @@ def main() -> int:
manifest=manifest,
)
)
errors.extend(
_information_governance_evidence_errors(
repository_name=repository_name,
repository_root=repository_root,
manifest=manifest,
)
)
manifests.append(manifest)
@@ -153,6 +160,20 @@ def main() -> int:
f"Architecture declaration coverage: {declared}/{len(manifests)} modules "
f"({(declared / len(manifests) * 100):.1f}%)."
)
governance_counts: dict[str, int] = {}
for manifest in manifests:
for dimension in manifest.information_governance.dimensions.values():
governance_counts[dimension.adoption] = (
governance_counts.get(dimension.adoption, 0) + 1
)
print(
"Information-governance adoption: "
+ ", ".join(
f"{status}={count}"
for status, count in sorted(governance_counts.items())
)
+ "."
)
return 0
@@ -201,6 +222,39 @@ def _architecture_evidence_errors(
return errors
def _information_governance_evidence_errors(
*,
repository_name: str,
repository_root: Path,
manifest: object,
) -> list[str]:
declaration = getattr(manifest, "information_governance", None)
if declaration is None:
return [
f"{repository_name}: module has no information-governance declaration"
]
errors: list[str] = []
for dimension_name, dimension in declaration.dimensions.items():
for reference in dimension.evidence:
if not _looks_like_repository_reference(reference):
continue
candidate = (repository_root / reference).resolve()
try:
candidate.relative_to(repository_root.resolve())
except ValueError:
errors.append(
f"{repository_name}: {dimension_name} evidence escapes the "
f"repository: {reference!r}"
)
continue
if not candidate.exists():
errors.append(
f"{repository_name}: {dimension_name} evidence does not exist: "
f"{reference!r}"
)
return errors
def _looks_like_repository_reference(reference: str) -> bool:
normalized = reference.strip()
if not normalized or "://" in normalized:
@@ -182,6 +182,11 @@ run_step "Validate installed module manifests and registry"
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \
--requirements "$META_ROOT/requirements-release.txt"
run_step "Validate platform interface and endpoint declarations"
"$PYTHON" "$META_ROOT/tools/inventory/platform-interface-inventory.py" \
--strict-declarations \
--strict-endpoints
run_step "Generate release dependency provenance"
"$PYTHON" "$META_ROOT/tools/release/generate-release-sbom.py" \
--python "$PYTHON" \
+14 -2
View File
@@ -219,6 +219,13 @@ def build_parser() -> argparse.ArgumentParser:
kubernetes.add_argument("--namespace", default="govoplan")
kubernetes.add_argument("--secret-name", default="govoplan-runtime")
kubernetes.add_argument("--tls-secret-name", default="govoplan-tls")
kubernetes.add_argument(
"--s3-ca-secret-name",
help=(
"Optional Secret containing ca.crt for the external S3 endpoint; "
"mounted read-only into backend runtime roles."
),
)
kubernetes.add_argument("--ingress-class-name")
kubernetes.add_argument(
"--output",
@@ -239,7 +246,10 @@ def build_parser() -> argparse.ArgumentParser:
verify_kubernetes.add_argument(
"--api-key-env",
default="GOVOPLAN_OPS_API_KEY",
help="Environment variable containing an API key with Ops read scope.",
help=(
"Environment variable containing an API key authorized to read "
"Ops status."
),
)
verify_kubernetes.add_argument(
"--exercise-api-pod-loss",
@@ -1146,6 +1156,7 @@ def _render_kubernetes(args: argparse.Namespace) -> int:
namespace=args.namespace,
secret_name=args.secret_name,
tls_secret_name=args.tls_secret_name,
s3_ca_secret_name=args.s3_ca_secret_name,
ingress_class_name=args.ingress_class_name,
backup_required=backup_required,
backup_evidence=backup_summary,
@@ -1170,7 +1181,8 @@ def _verify_kubernetes(args: argparse.Namespace) -> int:
api_key = str(os.environ.get(args.api_key_env) or "").strip()
if not api_key:
raise ValueError(
f"{args.api_key_env} must contain an API key with Ops read scope"
f"{args.api_key_env} must contain an API key authorized to read "
"Ops status"
)
ops_url = args.ops_url or (spec.public_url.rstrip("/") + "/api/v1/ops/status")
evidence = collect_kubernetes_evidence(
@@ -14,6 +14,7 @@ from urllib.request import Request, urlopen
JsonObject = dict[str, Any]
CommandRunner = Callable[[Sequence[str]], JsonObject]
ActionRunner = Callable[[Sequence[str]], None]
JsonFetcher = Callable[[str, str], JsonObject]
@@ -26,11 +27,13 @@ def collect_kubernetes_evidence(
exercise_api_pod_loss: bool = False,
timeout_seconds: float = 180.0,
command_runner: CommandRunner | None = None,
action_runner: ActionRunner | None = None,
json_fetcher: JsonFetcher | None = None,
) -> JsonObject:
"""Inspect a live cluster and optionally exercise one API pod replacement."""
run_json = command_runner or _kubectl_json
run_action = action_runner or _kubectl_action
fetch_json = json_fetcher or _fetch_json
nodes = run_json(("get", "nodes", "-o", "json"))
pods = run_json(
@@ -76,6 +79,7 @@ def collect_kubernetes_evidence(
initial_pods=pods,
timeout_seconds=timeout_seconds,
run_json=run_json,
run_action=run_action,
fetch_json=fetch_json,
)
evidence = {
@@ -211,6 +215,7 @@ def _exercise_api_pod_loss(
initial_pods: Mapping[str, Any],
timeout_seconds: float,
run_json: CommandRunner,
run_action: ActionRunner,
fetch_json: JsonFetcher,
) -> JsonObject:
candidates = [
@@ -228,7 +233,7 @@ def _exercise_api_pod_loss(
victim = sorted(candidates, key=lambda item: item["name"])[0]
initial_uids = {item["uid"] for item in candidates}
desired_ready = len(candidates)
run_json(
run_action(
(
"-n",
namespace,
@@ -236,8 +241,6 @@ def _exercise_api_pod_loss(
"pod",
victim["name"],
"--wait=false",
"-o",
"json",
)
)
deadline = time.monotonic() + timeout_seconds
@@ -309,6 +312,22 @@ def _kubectl_json(arguments: Sequence[str]) -> JsonObject:
return payload
def _kubectl_action(arguments: Sequence[str]) -> None:
kubectl = shutil.which("kubectl")
if kubectl is None:
raise ValueError("kubectl is required for Kubernetes evidence collection")
result = subprocess.run(
(kubectl, *arguments),
check=False,
capture_output=True,
text=True,
timeout=60,
)
if result.returncode:
detail = result.stderr.strip() or result.stdout.strip()
raise ValueError(f"kubectl failed: {detail}")
def _fetch_json(url: str, api_key: str) -> JsonObject:
request = Request(
url,
@@ -167,6 +167,7 @@ def validate_manifest(
"composition",
"signatures",
},
optional={"package_lock"},
label="distribution manifest",
)
if payload.get("schema_version") != "1":
@@ -200,6 +201,12 @@ def validate_manifest(
_https_url(deployer.get("url"), "deployer.url")
_sha256(deployer.get("sha256"), "deployer.sha256")
if "package_lock" in payload:
package_lock = _object(payload.get("package_lock"), "package_lock")
_exact_keys(package_lock, required={"url", "sha256"}, label="package_lock")
_https_url(package_lock.get("url"), "package_lock.url")
_sha256(package_lock.get("sha256"), "package_lock.sha256")
images = _object(payload.get("images"), "images")
if set(images) != {"api", "web"}:
raise DistributionError("images must contain exactly api and web")
@@ -630,10 +637,12 @@ def _exact_keys(
value: Mapping[str, Any],
*,
required: set[str],
optional: set[str] | None = None,
label: str,
) -> None:
optional = optional or set()
missing = sorted(required - set(value))
extra = sorted(set(value) - required)
extra = sorted(set(value) - required - optional)
if missing or extra:
detail = []
if missing:
+89 -5
View File
@@ -76,13 +76,20 @@ def render_kubernetes(
namespace: str = "govoplan",
secret_name: str = "govoplan-runtime",
tls_secret_name: str = "govoplan-tls",
s3_ca_secret_name: str | None = None,
ingress_class_name: str | None = None,
backup_required: bool = True,
backup_evidence: Mapping[str, object] | None = None,
) -> dict[str, Any]:
"""Render runtime roles only; shared state services stay externally managed."""
_validate_cluster_profile(spec, environment, namespace, secret_name)
_validate_cluster_profile(
spec,
environment,
namespace,
secret_name,
s3_ca_secret_name=s3_ca_secret_name,
)
worker_pools = _worker_pools(spec, environment)
database_capacity = _database_capacity(spec, environment, worker_pools)
name = _resource_name(spec.installation_id)
@@ -155,6 +162,7 @@ def render_kubernetes(
),
config_name=config_name,
secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account,
container_port=8000,
readiness_path="/health/ready",
@@ -179,6 +187,7 @@ def render_kubernetes(
command=(),
config_name=None,
secret_name=None,
s3_ca_secret_name=None,
service_account=service_account,
container_port=8080,
extra_environment={"GOVOPLAN_API_UPSTREAM": f"http://{name}-api:8000"},
@@ -198,6 +207,7 @@ def render_kubernetes(
image=spec.release.api_image,
config_name=config_name,
secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account,
database_environment=_role_database_environment(
environment,
@@ -237,6 +247,7 @@ def render_kubernetes(
),
config_name=config_name,
secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account,
extra_environment={
**_role_database_environment(environment, "WORKER"),
@@ -287,9 +298,12 @@ def render_kubernetes(
"beat",
"--loglevel",
"INFO",
"--schedule",
"/tmp/celerybeat-schedule",
),
config_name=config_name,
secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account,
extra_environment=_role_database_environment(
environment,
@@ -318,6 +332,7 @@ def render_kubernetes(
"annotations": {
"govoplan.add-ideas.de/profile": "stateless-shared-state",
"govoplan.add-ideas.de/secret-contract": ",".join(_SECRET_KEYS),
"govoplan.add-ideas.de/s3-ca-secret": s3_ca_secret_name or "",
"govoplan.add-ideas.de/database-connection-peak": str(
database_capacity["peak"]
),
@@ -342,9 +357,13 @@ def _validate_cluster_profile(
environment: Mapping[str, str],
namespace: str,
secret_name: str,
*,
s3_ca_secret_name: str | None,
) -> None:
if not _DNS_LABEL.fullmatch(namespace) or not _DNS_LABEL.fullmatch(secret_name):
raise ValueError("Kubernetes namespace and secret names must be DNS labels")
if s3_ca_secret_name is not None and not _DNS_LABEL.fullmatch(s3_ca_secret_name):
raise ValueError("Kubernetes S3 CA secret name must be a DNS label")
if spec.installation_id == "govoplan-local":
raise ValueError(
"Kubernetes export requires a non-default stable installation id"
@@ -633,6 +652,7 @@ def _deployment(
command: tuple[str, ...],
config_name: str | None,
secret_name: str | None,
s3_ca_secret_name: str | None,
service_account: str,
container_port: int | None = None,
readiness_path: str | None = None,
@@ -660,6 +680,10 @@ def _deployment(
)
if secret_name:
environment.extend(_secret_environment(secret_name))
if s3_ca_secret_name:
environment.append(
{"name": "AWS_CA_BUNDLE", "value": "/etc/govoplan/trust/s3-ca.crt"}
)
container: dict[str, Any] = {
"name": role,
"image": image,
@@ -704,12 +728,16 @@ def _deployment(
{
"maxSkew": 1,
"topologyKey": "kubernetes.io/hostname",
"whenUnsatisfiable": "ScheduleAnyway",
"whenUnsatisfiable": "DoNotSchedule",
"matchLabelKeys": ["pod-template-hash"],
"labelSelector": {"matchLabels": role_labels},
}
],
}
container["volumeMounts"] = [{"name": "tmp", "mountPath": "/tmp"}]
if s3_ca_secret_name:
pod_spec["volumes"].append(_s3_ca_volume(s3_ca_secret_name))
container["volumeMounts"].append(_s3_ca_volume_mount())
if config_name:
pod_spec["containers"][0]["envFrom"] = [{"configMapRef": {"name": config_name}}]
if config_name and secret_name:
@@ -731,6 +759,16 @@ def _deployment(
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration-wait"},
{"name": "GOVOPLAN_DB_POOL_SIZE", "value": "1"},
{"name": "GOVOPLAN_DB_MAX_OVERFLOW", "value": "0"},
*(
[
{
"name": "AWS_CA_BUNDLE",
"value": "/etc/govoplan/trust/s3-ca.crt",
}
]
if s3_ca_secret_name
else []
),
*_secret_environment(secret_name),
],
"securityContext": {
@@ -738,7 +776,10 @@ def _deployment(
"capabilities": {"drop": ["ALL"]},
"readOnlyRootFilesystem": True,
},
"volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}],
"volumeMounts": [
{"name": "tmp", "mountPath": "/tmp"},
*([_s3_ca_volume_mount()] if s3_ca_secret_name else []),
],
}
]
return {
@@ -769,6 +810,7 @@ def _migration_job(
image: str,
config_name: str,
secret_name: str,
s3_ca_secret_name: str | None,
service_account: str,
database_environment: Mapping[str, str],
backup_required: bool,
@@ -832,6 +874,16 @@ def _migration_job(
"env": [
{"name": "TMPDIR", "value": "/tmp"},
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration"},
*(
[
{
"name": "AWS_CA_BUNDLE",
"value": "/etc/govoplan/trust/s3-ca.crt",
}
]
if s3_ca_secret_name
else []
),
*(
{"name": key, "value": value}
for key, value in sorted(
@@ -851,10 +903,24 @@ def _migration_job(
"capabilities": {"drop": ["ALL"]},
"readOnlyRootFilesystem": True,
},
"volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}],
"volumeMounts": [
{"name": "tmp", "mountPath": "/tmp"},
*(
[_s3_ca_volume_mount()]
if s3_ca_secret_name
else []
),
],
}
],
"volumes": [{"name": "tmp", "emptyDir": {}}],
"volumes": [
{"name": "tmp", "emptyDir": {}},
*(
[_s3_ca_volume(s3_ca_secret_name)]
if s3_ca_secret_name
else []
),
],
},
},
},
@@ -873,6 +939,24 @@ def _secret_environment(secret_name: str) -> list[dict[str, Any]]:
]
def _s3_ca_volume(secret_name: str) -> dict[str, Any]:
return {
"name": "s3-ca",
"secret": {
"secretName": secret_name,
"items": [{"key": "ca.crt", "path": "s3-ca.crt"}],
},
}
def _s3_ca_volume_mount() -> dict[str, Any]:
return {
"name": "s3-ca",
"mountPath": "/etc/govoplan/trust",
"readOnly": True,
}
def _service(
*,
name: str,
@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""Configure and verify protected GovOPlaN package-release boundaries."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
from gitea_common import (
GiteaClient,
GiteaError,
RepoTarget,
load_dotenv,
org_path,
quote_path,
repo_path,
require_token,
)
META_ROOT = Path(__file__).resolve().parents[2]
REQUIRED_SECRETS = {"GOVOPLAN_PACKAGE_USERNAME", "GOVOPLAN_PACKAGE_TOKEN"}
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--url", default="https://git.add-ideas.de")
parser.add_argument("--owner", default="GovOPlaN")
parser.add_argument("--team", default="Owners")
parser.add_argument("--pattern", default="v*")
parser.add_argument("--env-file", type=Path)
parser.add_argument("--apply", action="store_true")
return parser
def package_repositories() -> tuple[str, ...]:
inventory = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
values = ["govoplan"]
for item in inventory["repositories"]:
name = str(item["name"])
repository = META_ROOT.parent / str(item["path"])
if name.startswith("govoplan-") and (repository / "pyproject.toml").is_file():
values.append(name)
return tuple(sorted(set(values)))
def configure(
client: GiteaClient,
*,
owner: str,
team: str,
pattern: str,
apply: bool,
) -> tuple[str, ...]:
missing: list[str] = []
expected = {
"name_pattern": pattern,
"whitelist_teams": [team],
"whitelist_usernames": [],
}
for repository in package_repositories():
path = repo_path(owner, repository, "/tag_protections")
protections = client.request_json("GET", path)
matching = [
item
for item in protections
if isinstance(item, dict) and item.get("name_pattern") == pattern
]
if len(matching) == 1 and _matches(matching[0], expected):
print(f"protected {repository}:{pattern}")
continue
missing.append(repository)
if not apply:
print(f"would protect {repository}:{pattern}")
continue
if len(matching) == 1:
protection_id = matching[0].get("id")
client.request_json(
"PATCH",
f"{path}/{quote_path(str(protection_id))}",
body=expected,
)
print(f"updated {repository}:{pattern}")
elif not matching:
client.request_json("POST", path, body=expected)
print(f"created {repository}:{pattern}")
else:
raise GiteaError(f"{repository} has duplicate {pattern!r} tag protections")
return tuple(missing)
def _matches(value: dict[str, object], expected: dict[str, object]) -> bool:
return (
value.get("name_pattern") == expected["name_pattern"]
and sorted(value.get("whitelist_teams") or []) == expected["whitelist_teams"]
and sorted(value.get("whitelist_usernames") or []) == expected["whitelist_usernames"]
)
def main() -> int:
args = build_parser().parse_args()
try:
load_dotenv(args.env_file)
token = require_token()
target = RepoTarget(base_url=args.url, owner=args.owner, repo="govoplan")
with GiteaClient(target, token) as client:
mismatches = configure(
client,
owner=args.owner,
team=args.team,
pattern=args.pattern,
apply=args.apply,
)
secrets = client.request_json(
"GET", org_path(args.owner, "/actions/secrets"), query={"limit": 50}
)
names = {
str(item.get("name") or "")
for item in secrets
if isinstance(item, dict)
}
missing_secrets = sorted(REQUIRED_SECRETS - names)
if missing_secrets:
print(
"Missing organization Actions secrets: " + ", ".join(missing_secrets),
file=sys.stderr,
)
unresolved = (bool(mismatches) and not args.apply) or bool(missing_secrets)
if unresolved:
return 1
print("Package release protection and credential names are configured.")
return 0
except (GiteaError, OSError, ValueError, json.JSONDecodeError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+302
View File
@@ -0,0 +1,302 @@
#!/usr/bin/env python3
"""Dispatch protected package releases required by the govoplan meta-package."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import json
from pathlib import Path
import re
import subprocess
import sys
import tomllib
from gitea_common import (
GiteaClient,
GiteaError,
RepoTarget,
load_dotenv,
org_path,
quote_path,
repo_path,
require_token,
)
META_ROOT = Path(__file__).resolve().parents[2]
META_PROJECT = META_ROOT / "packages" / "govoplan-meta" / "pyproject.toml"
WORKFLOW_ID = "module-package-release.yml"
EXACT_REQUIREMENT = re.compile(
r"^(?P<name>govoplan-[a-z0-9-]+)(?:\[[a-z0-9_,.-]+\])?==(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$"
)
ACTIVE_STATES = {"queued", "waiting", "in_progress", "running"}
@dataclass(frozen=True, slots=True)
class PackageTarget:
distribution: str
version: str
repository: str
tag_exists: bool
has_webui: bool
@property
def tag(self) -> str:
return f"v{self.version}"
@property
def webui_package(self) -> str | None:
if not self.has_webui:
return None
return f"@govoplan/{self.distribution.removeprefix('govoplan-')}-webui"
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--url", default="https://git.add-ideas.de")
parser.add_argument("--owner", default="GovOPlaN")
parser.add_argument("--env-file", type=Path)
parser.add_argument(
"--repository",
action="append",
default=[],
help="Limit dispatch to one repository; repeat as needed.",
)
parser.add_argument(
"--verify-existing",
action="store_true",
help="Also rerun exact versions already present in both registries.",
)
parser.add_argument("--apply", action="store_true")
return parser
def package_targets(project_path: Path = META_PROJECT) -> tuple[PackageTarget, ...]:
project = tomllib.loads(project_path.read_text(encoding="utf-8"))["project"]
requirements = list(project.get("dependencies") or [])
requirements.extend(project.get("optional-dependencies", {}).get("full") or [])
parsed: dict[str, str] = {}
for requirement in requirements:
match = EXACT_REQUIREMENT.fullmatch(str(requirement))
if match is None:
raise ValueError(
f"Meta-package requirement is not an exact GovOPlaN version: {requirement!r}"
)
name = match.group("name")
version = match.group("version")
previous = parsed.setdefault(name, version)
if previous != version:
raise ValueError(f"Meta-package selects conflicting versions for {name}")
targets: list[PackageTarget] = []
for distribution, version in sorted(parsed.items()):
repository = distribution
repository_root = META_ROOT.parent / repository
if not (repository_root / ".git").is_dir():
raise ValueError(f"Package repository is not checked out: {repository}")
tag = f"v{version}"
tag_exists = _tag_exists(repository_root, tag)
has_webui = (
_tag_has_path(repository_root, tag, "webui/package.json")
if tag_exists
else False
)
targets.append(
PackageTarget(
distribution=distribution,
version=version,
repository=repository,
tag_exists=tag_exists,
has_webui=has_webui,
)
)
return tuple(targets)
def _tag_exists(repository: Path, tag: str) -> bool:
result = subprocess.run(
(
"git",
"-C",
str(repository),
"rev-parse",
"--verify",
"--quiet",
f"refs/tags/{tag}",
),
check=False,
capture_output=True,
text=True,
)
if result.returncode not in {0, 1}:
raise ValueError(
f"Could not inspect {repository.name}:{tag}: {result.stderr.strip()}"
)
return result.returncode == 0
def _tag_has_path(repository: Path, tag: str, path: str) -> bool:
result = subprocess.run(
("git", "-C", str(repository), "cat-file", "-e", f"{tag}:{path}"),
check=False,
capture_output=True,
text=True,
)
if result.returncode not in {0, 128}:
raise ValueError(
f"Could not inspect {repository.name}:{tag}:{path}: {result.stderr.strip()}"
)
return result.returncode == 0
def _published_packages(
client: GiteaClient, *, owner: str, package_type: str
) -> set[tuple[str, str]]:
values = client.paginate(
f"/packages/{quote_path(owner)}",
query={"type": package_type, "q": "govoplan"},
)
return {
(str(item.get("name") or ""), str(item.get("version") or ""))
for item in values
if item.get("type") == package_type
}
def _has_active_run(
client: GiteaClient, *, owner: str, repository: str
) -> bool:
payload = client.request_json(
"GET",
repo_path(
owner,
repository,
f"/actions/workflows/{quote_path(WORKFLOW_ID)}/runs",
),
query={"limit": 10},
)
runs = payload.get("workflow_runs") if isinstance(payload, dict) else None
return isinstance(runs, list) and any(
isinstance(run, dict) and str(run.get("status") or "") in ACTIVE_STATES
for run in runs
)
def dispatch(
client: GiteaClient,
*,
owner: str,
targets: tuple[PackageTarget, ...],
published_pypi: set[tuple[str, str]],
published_npm: set[tuple[str, str]],
verify_existing: bool,
apply: bool,
) -> tuple[int, int, int]:
dispatched = 0
active = 0
complete = 0
for target in targets:
wheel_exists = (target.distribution, target.version) in published_pypi
npm_exists = target.webui_package is None or (
target.webui_package,
target.version,
) in published_npm
if wheel_exists and npm_exists and not verify_existing:
complete += 1
print(f"complete {target.repository}:{target.tag}")
continue
if _has_active_run(client, owner=owner, repository=target.repository):
active += 1
print(f"active {target.repository}:{target.tag}")
continue
action = "dispatching" if apply else "would dispatch"
print(
f"{action} {target.repository}:{target.tag} "
f"(wheel={'present' if wheel_exists else 'missing'}, "
f"webui={'present' if npm_exists else 'missing'})"
)
if apply:
client.request_json(
"POST",
repo_path(
owner,
target.repository,
f"/actions/workflows/{quote_path(WORKFLOW_ID)}/dispatches",
),
body={"ref": "main", "inputs": {"release_tag": target.tag}},
)
dispatched += 1
return dispatched, active, complete
def main() -> int:
args = build_parser().parse_args()
try:
load_dotenv(args.env_file)
token = require_token()
targets = package_targets()
selected = set(args.repository)
if selected:
known = {target.repository for target in targets}
unknown = sorted(selected - known)
if unknown:
raise ValueError(
"Unknown meta-package repositories: " + ", ".join(unknown)
)
targets = tuple(
target for target in targets if target.repository in selected
)
missing_tags = [
f"{target.repository}:{target.tag}"
for target in targets
if not target.tag_exists
]
if missing_tags:
raise ValueError(
"Meta-package release tags are missing: " + ", ".join(missing_tags)
)
target = RepoTarget(base_url=args.url, owner=args.owner, repo="govoplan")
with GiteaClient(target, token) as client:
secrets = client.request_json(
"GET", org_path(args.owner, "/actions/secrets"), query={"limit": 50}
)
secret_names = {
str(item.get("name") or "")
for item in secrets
if isinstance(item, dict)
}
required = {"GOVOPLAN_PACKAGE_USERNAME", "GOVOPLAN_PACKAGE_TOKEN"}
if not required <= secret_names:
raise ValueError(
"Organization package publisher secrets are not configured"
)
published_pypi = _published_packages(
client, owner=args.owner, package_type="pypi"
)
published_npm = _published_packages(
client, owner=args.owner, package_type="npm"
)
counts = dispatch(
client,
owner=args.owner,
targets=targets,
published_pypi=published_pypi,
published_npm=published_npm,
verify_existing=args.verify_existing,
apply=args.apply,
)
action = "dispatched" if args.apply else "planned"
print(
f"Package set {action}: {counts[0]}; active: {counts[1]}; "
f"already complete: {counts[2]}."
)
return 0
except (GiteaError, OSError, ValueError, json.JSONDecodeError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+174 -148
View File
@@ -155,44 +155,39 @@
"repository": "govoplan-access"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/admin/service-accounts",
"rationale": "Service-account lifecycle is implemented but its administration UI is tracked separately.",
"repository": "govoplan-access",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/18"
"rationale": "Access administration lists service accounts and opens their lifecycle and credential manager.",
"repository": "govoplan-access"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/admin/service-accounts",
"rationale": "Service-account lifecycle is implemented but its administration UI is tracked separately.",
"repository": "govoplan-access",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/18"
"rationale": "Access administration creates service accounts through the governed editor.",
"repository": "govoplan-access"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/admin/service-accounts/{}",
"rationale": "Service-account lifecycle is implemented but its administration UI is tracked separately.",
"repository": "govoplan-access",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/18"
"rationale": "Access administration refreshes service-account state before governed mutations.",
"repository": "govoplan-access"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "PATCH",
"path": "/admin/service-accounts/{}",
"rationale": "Service-account lifecycle is implemented but its administration UI is tracked separately.",
"repository": "govoplan-access",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/18"
"rationale": "Access administration edits, activates, and deactivates service accounts with revision checks.",
"repository": "govoplan-access"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/admin/service-accounts/{}/retire",
"rationale": "Service-account lifecycle is implemented but its administration UI is tracked separately.",
"repository": "govoplan-access",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/18"
"rationale": "Access administration exposes separately confirmed retirement and credential revocation.",
"repository": "govoplan-access"
},
{
"category": "intentionally_headless",
@@ -237,44 +232,39 @@
"repository": "govoplan-admin"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/approvals/templates",
"rationale": "Approval-template and escalation administration UI is tracked separately.",
"repository": "govoplan-approvals",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/2"
"rationale": "Approval administration lists immutable template revisions.",
"repository": "govoplan-approvals"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/approvals/templates",
"rationale": "Approval-template and escalation administration UI is tracked separately.",
"repository": "govoplan-approvals",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/2"
"rationale": "Approval administration creates validated draft templates.",
"repository": "govoplan-approvals"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "PUT",
"path": "/approvals/templates/{}",
"rationale": "Approval-template and escalation administration UI is tracked separately.",
"repository": "govoplan-approvals",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/2"
"rationale": "Approval administration revises templates through optimistic immutable revisions.",
"repository": "govoplan-approvals"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/approvals/templates/{}/publish",
"rationale": "Approval-template and escalation administration UI is tracked separately.",
"repository": "govoplan-approvals",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/2"
"rationale": "Approval administration publishes a draft only after an explicit confirmation.",
"repository": "govoplan-approvals"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/approvals/{}/escalate",
"rationale": "Approval-template and escalation administration UI is tracked separately.",
"repository": "govoplan-approvals",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/2"
"rationale": "The request dialog exposes escalation only for due pending requests and authorized administrators.",
"repository": "govoplan-approvals"
},
{
"category": "ui_reachable",
@@ -353,6 +343,20 @@
"rationale": "Published integration, interoperability, public-participant, or health endpoint.",
"repository": "govoplan-calendar"
},
{
"category": "intentionally_headless",
"method": "POST",
"path": "/bootstrap/first-admin",
"rationale": "The restricted first-run endpoint is consumed by the local bootstrap handoff and can only create the first durable administrator.",
"repository": "govoplan-core"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/bootstrap/status",
"rationale": "The local bootstrap handoff reads only minimum first-run readiness before a normal authenticated shell exists.",
"repository": "govoplan-core"
},
{
"category": "public_integration",
"method": "GET",
@@ -368,12 +372,11 @@
"repository": "govoplan-calendar"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/campaigns/{}/archive",
"rationale": "Campaign archive lifecycle UI remains tracked by the archive policy issue.",
"repository": "govoplan-campaign",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/26"
"rationale": "The Campaign overview exposes the governed archive action with permission checks and an evidence-retention confirmation.",
"repository": "govoplan-campaign"
},
{
"category": "intentionally_headless",
@@ -389,6 +392,13 @@
"rationale": "Campaign delivery diagnostic/status API is retained for bounded support and automation consumers.",
"repository": "govoplan-campaign"
},
{
"category": "worker_internal",
"method": "POST",
"path": "/campaigns/operations/artifacts/reconcile",
"rationale": "Privileged, bounded artifact recovery operation used by operators and recovery automation rather than an end-user surface.",
"repository": "govoplan-campaign"
},
{
"category": "ui_reachable",
"method": "PUT",
@@ -571,6 +581,13 @@
"rationale": "The shared ownership UI uses a dynamic transfer-action path that the static string scan cannot resolve.",
"repository": "govoplan-core"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/platform/interface-catalog",
"rationale": "Authorized module and system administrators consume the read-only control-plane inventory directly or through Ops/Docs projections.",
"repository": "govoplan-core"
},
{
"category": "compatibility",
"method": "GET",
@@ -656,156 +673,172 @@
"repository": "govoplan-docs"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/encryption/disable-preflight",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "The Encryption administration panel displays disable readiness and bounded blocking envelope references.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "POST",
"path": "/encryption/envelopes",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Feature modules register envelopes while retaining content ownership; the operator UI must not construct feature content envelopes.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/encryption/envelopes",
"rationale": "The Encryption administration panel consumes the bounded, secret-free envelope summary contract.",
"repository": "govoplan-encryption"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/encryption/envelopes/{}",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Owning modules resolve a specific full envelope through the capability/API contract; the operator UI uses the secret-free summary projection.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/encryption/migrations",
"rationale": "The Encryption administration panel displays bounded migration state and evidence counts.",
"repository": "govoplan-encryption"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/migrations",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Encryption custodians can authorize a two-phase migration from a bounded envelope summary.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "POST",
"path": "/encryption/migrations/{}/outcome",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Only the owning module or governed worker can attest the durable content outcome and exact target envelope.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/migrations/{}/reconcile",
"rationale": "Encryption custodians can re-read recorded provider migration state without declaring an outcome.",
"repository": "govoplan-encryption"
},
{
"category": "ui_reachable",
"method": "GET",
"path": "/encryption/recoveries",
"rationale": "The Encryption administration panel displays bounded recovery requests, quorum, expiry, and state.",
"repository": "govoplan-encryption"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/recoveries",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Authorized custodians can request an expiring high-assurance recovery ceremony.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/recoveries/{}/decision",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "The recovery table exposes explicit approve/reject decisions with assurance, reason, and optimistic revision.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/encryption/vaults",
"rationale": "The Encryption administration panel consumes the bounded, secret-free vault summary contract.",
"repository": "govoplan-encryption"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/vaults",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Encryption custodians can create a governed vault without entering or receiving raw key material.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "GET",
"path": "/encryption/vaults/{}",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "Capability consumers may resolve the complete vault reference; the operator UI uses the secret-free summary projection.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/vaults/{}/destruction",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "The vault action group schedules destructive key lifecycle operations with explicit consequences.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/vaults/{}/reconcile",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "The vault action group reconciles an outcome-unknown provider operation.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/vaults/{}/revoke",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "The vault action group revokes the current key with policy, assurance, reason, and revision evidence.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/encryption/vaults/{}/rotate",
"rationale": "Encryption custodian, lifecycle, and recovery administration UI is tracked separately.",
"repository": "govoplan-encryption",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-encryption/issues/4"
"rationale": "The vault action group rotates the current key with policy, assurance, reason, and revision evidence.",
"repository": "govoplan-encryption"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/files/integrity/findings/{}/cleanup",
"rationale": "File-integrity operations UI is tracked separately.",
"repository": "govoplan-files",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/40"
"rationale": "The Files integrity panel requires a dry-run preview and separate confirmation before cleanup.",
"repository": "govoplan-files"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/files/integrity/findings/{}/recheck",
"rationale": "File-integrity operations UI is tracked separately.",
"repository": "govoplan-files",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/40"
"rationale": "The Files integrity panel rechecks findings against their displayed revision.",
"repository": "govoplan-files"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/files/integrity/scans",
"rationale": "File-integrity operations UI is tracked separately.",
"repository": "govoplan-files",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/40"
"rationale": "The Files integrity administration panel lists bounded reconciliation scans.",
"repository": "govoplan-files"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/files/integrity/scans",
"rationale": "File-integrity operations UI is tracked separately.",
"repository": "govoplan-files",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/40"
"rationale": "Authorized Files operators can create a bounded integrity scan from administration.",
"repository": "govoplan-files"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/files/integrity/scans/{}/findings",
"rationale": "File-integrity operations UI is tracked separately.",
"repository": "govoplan-files",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/40"
"rationale": "The Files integrity panel displays findings and blocker evidence for the selected scan.",
"repository": "govoplan-files"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/files/integrity/scans/{}/run",
"rationale": "File-integrity operations UI is tracked separately.",
"repository": "govoplan-files",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/40"
"rationale": "The Files integrity panel runs and resumes bounded batches with stale-action protection.",
"repository": "govoplan-files"
},
{
"category": "compatibility",
@@ -857,60 +890,53 @@
"repository": "govoplan-identity"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "POST",
"path": "/identity-trust/assurance/check",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Assurance checks are capability operations performed by protected module workflows rather than direct user commands.",
"repository": "govoplan-identity-trust"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "POST",
"path": "/identity-trust/assurance/evidence",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Trusted assurance providers record evidence through this capability endpoint; users inspect the resulting projection.",
"repository": "govoplan-identity-trust"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "GET",
"path": "/identity-trust/device-keys",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Identity Trust settings and administration list permission-filtered public device keys.",
"repository": "govoplan-identity-trust"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "POST",
"path": "/identity-trust/device-keys",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Device onboarding registers public key material through the trust capability; the UI manages registered keys without handling private material.",
"repository": "govoplan-identity-trust"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/identity-trust/device-keys/{}/revoke",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Authorized users and trust officers revoke public device keys through a consequence-aware confirmation.",
"repository": "govoplan-identity-trust"
},
{
"category": "missing_ui",
"category": "ui_reachable",
"method": "POST",
"path": "/identity-trust/epochs/rotate",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Identity Trust administration exposes governed epoch rotation with history and consequence explanations.",
"repository": "govoplan-identity-trust"
},
{
"category": "missing_ui",
"category": "intentionally_headless",
"method": "POST",
"path": "/identity-trust/key-access/decide",
"rationale": "Identity-assurance and device-key administration UI is tracked separately.",
"repository": "govoplan-identity-trust",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust/issues/2"
"rationale": "Protected modules request immutable key-access decisions through the capability; trust officers inspect the resulting decision projection.",
"repository": "govoplan-identity-trust"
},
{
"category": "ui_reachable",
+251 -7
View File
@@ -2,6 +2,7 @@
import fs from "node:fs";
import path from "node:path";
import { createHash } from "node:crypto";
import { pathToFileURL } from "node:url";
const [metaRootArgument] = process.argv.slice(2);
@@ -60,9 +61,19 @@ const helpAttributes = new Set([
"helperText",
"helpText"
]);
const actionComponentPattern = /(?:Action|Button|Link)$/;
const contributionTypes = new Map([
["AdminSectionsUiCapability", "admin_section"],
["DashboardWidgetsUiCapability", "widget"],
["OrganizationFunctionActionsUiCapability", "action"],
["SearchContextsUiCapability", "search_object"],
["SettingsSectionsUiCapability", "setting"],
["WizardDirectoriesUiCapability", "workflow_directory"]
]);
const result = {
fields: [],
actions: [],
labels: [],
visibleText: [],
translationCatalog: {},
@@ -71,7 +82,8 @@ const result = {
routes: [],
navigation: [],
frontendApiReferences: [],
uiCapabilities: []
uiCapabilities: [],
contributions: []
};
for (const repository of repositoryCatalog.repositories) {
@@ -115,6 +127,7 @@ function inspectSource(repository, sourceRoot, sourcePath) {
sourcePath.endsWith(".tsx") ? ts.ScriptKind.TSX : ts.ScriptKind.TS
);
const relativeFile = path.relative(path.join(workspaceRoot, repository), sourcePath);
const identityCounters = new Map();
function location(node) {
const position = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile));
@@ -178,6 +191,7 @@ function inspectSource(repository, sourceRoot, sourcePath) {
const isField =
fieldComponents.has(component) ||
(fieldComponentPattern.test(component) && component !== "FormField");
inspectAction(node, component, attributes, locate);
if (!isField) return;
const parentFormField = nearestFormField(node);
@@ -191,8 +205,25 @@ function inspectSource(repository, sourceRoot, sourcePath) {
null;
const help = firstAttribute(attributes, helpAttributes) ??
firstAttribute(parentAttributes, helpAttributes);
const hasHelp = hasAnyAttribute(attributes, helpAttributes) ||
hasAnyAttribute(parentAttributes, helpAttributes);
const explicitId = firstAttribute(
attributes,
new Set(["interfaceId", "data-interface-id", "id", "name", "field"])
);
const context = nearestNamedContext(node);
const stableId = sourceIdentity(
"field",
node,
component,
explicitId ?? label ?? attributes.get("placeholder") ?? "field"
);
result.fields.push({
...locate(node),
id: stableId,
explicitId,
idSource: explicitId === null ? "source_anchor" : "explicit",
context,
component,
name:
attributes.get("name") ??
@@ -202,8 +233,102 @@ function inspectSource(repository, sourceRoot, sourcePath) {
label,
placeholder: attributes.get("placeholder") ?? null,
help: help ?? null,
helpCandidate: help === null
helpId: hasHelp ? `${stableId}.help` : null,
helpDynamic: hasHelp && help === null,
helpCandidate: !hasHelp
});
}
function inspectAction(node, component, attributes, locate) {
const lowerComponent = component.toLowerCase();
const inputType = attributes.get("type")?.toLowerCase();
const isAction = lowerComponent === "button" ||
lowerComponent === "a" ||
actionComponentPattern.test(component) ||
(lowerComponent === "input" && ["button", "reset", "submit"].includes(inputType));
if (!isAction) return;
const label = attributes.get("aria-label") ??
attributes.get("title") ??
attributes.get("label") ??
staticJsxChildText(node) ??
null;
const explicitId = firstAttribute(
attributes,
new Set(["interfaceId", "data-interface-id", "id", "name"])
);
const context = nearestNamedContext(node);
result.actions.push({
...locate(node),
id: sourceIdentity(
"action",
node,
component,
explicitId ?? label ?? "action"
),
explicitId,
idSource: explicitId === null ? "source_anchor" : "explicit",
context,
component,
label
});
}
function nearestNamedContext(node) {
let current = node.parent;
while (current) {
if (ts.isFunctionDeclaration(current) && current.name) {
return current.name.text;
}
if (ts.isMethodDeclaration(current) && current.name) {
return current.name.getText(sourceFile);
}
if (
(ts.isArrowFunction(current) || ts.isFunctionExpression(current)) &&
ts.isVariableDeclaration(current.parent) &&
ts.isIdentifier(current.parent.name)
) {
return current.parent.name.text;
}
if (
(ts.isArrowFunction(current) || ts.isFunctionExpression(current)) &&
ts.isPropertyAssignment(current.parent)
) {
return propertyNameText(current.parent.name) ?? "anonymous";
}
current = current.parent;
}
return path.basename(relativeFile).replace(/\.[^.]+$/, "");
}
function sourceIdentity(kind, node, component, semantic) {
const context = nearestNamedContext(node);
const normalizedSemantic = slug(String(semantic));
const counterKey = `${kind}:${context}:${component}:${normalizedSemantic}`;
const occurrence = (identityCounters.get(counterKey) ?? 0) + 1;
identityCounters.set(counterKey, occurrence);
const anchor = [relativeFile, context, component, normalizedSemantic, occurrence].join(":");
const digest = createHash("sha256").update(anchor).digest("hex").slice(0, 12);
return `${repository}.${kind}.${slug(context)}.${normalizedSemantic}.${digest}`;
}
function staticJsxChildText(node) {
if (!ts.isJsxOpeningElement(node) || !ts.isJsxElement(node.parent)) return null;
const values = [];
for (const child of node.parent.children) {
if (ts.isJsxText(child)) {
const value = child.getText(sourceFile).replace(/\s+/g, " ").trim();
if (value) values.push(value);
} else if (
ts.isJsxExpression(child) &&
child.expression &&
ts.isStringLiteralLike(child.expression)
) {
values.push(child.expression.text);
}
}
return values.length > 0 ? values.join(" ") : null;
}
function nearestFormField(node) {
@@ -275,22 +400,103 @@ function inspectSource(repository, sourceRoot, sourcePath) {
function inspectProperty(node, locate) {
const propertyName = propertyNameText(node.name);
if (isDirectUiCapabilityProperty(node) && typeof propertyName === "string") {
result.uiCapabilities.push({ ...locate(node), name: propertyName });
result.contributions.push({
...locate(node),
kind: "ui_capability",
id: propertyName
});
}
const value = staticExpressionText(node.initializer);
if (value === null) return;
if (propertyName === "path" && value.startsWith("/") && !value.includes("/api/")) {
result.routes.push({ ...locate(node), path: value });
const routeCollection = nearestCollectionProperty(node);
if (routeCollection === "routes" || routeCollection === "publicRoutes") {
result.contributions.push({
...locate(node),
kind: routeCollection === "publicRoutes" ? "public_route" : "frontend_route",
id: value,
path: value
});
}
}
if (propertyName === "to" && value.startsWith("/")) {
result.navigation.push({ ...locate(node), path: value });
if (nearestCollectionProperty(node) === "navItems") {
result.contributions.push({
...locate(node),
kind: "navigation",
id: value,
path: value
});
}
}
if (
ancestorPropertyName(node, "uiCapabilities") &&
typeof propertyName === "string"
) {
result.uiCapabilities.push({ ...locate(node), name: propertyName });
if (propertyName === "id") {
const contributionKind = contributionKindFor(node);
if (contributionKind !== null) {
result.contributions.push({
...locate(node),
kind: contributionKind,
id: value
});
}
}
}
function contributionKindFor(node) {
const collection = nearestCollectionProperty(node);
if (collection === "viewSurfaces") return "view_surface";
if (collection === "widgets") return "widget";
if (collection === "contexts") return "search_object";
if (collection === "actions") return "action";
if (collection === "directories") return "workflow_directory";
if (collection !== "sections") return null;
const variableType = nearestVariableType(node);
for (const [typeName, kind] of contributionTypes) {
if (variableType.includes(typeName)) return kind;
}
return ancestorPropertyName(node, "admin.sections")
? "admin_section"
: ancestorPropertyName(node, "settings.sections")
? "setting"
: "section";
}
function nearestCollectionProperty(node) {
let current = node.parent;
while (current) {
if (
ts.isArrayLiteralExpression(current) &&
ts.isPropertyAssignment(current.parent)
) {
return propertyNameText(current.parent.name);
}
current = current.parent;
}
return null;
}
function nearestVariableType(node) {
let current = node.parent;
while (current) {
if (ts.isVariableDeclaration(current)) {
return current.type?.getText(sourceFile) ?? "";
}
current = current.parent;
}
return "";
}
function isDirectUiCapabilityProperty(node) {
const parent = node.parent;
const uiCapabilities = parent?.parent;
return ts.isObjectLiteralExpression(parent) &&
ts.isPropertyAssignment(uiCapabilities) &&
propertyNameText(uiCapabilities.name) === "uiCapabilities";
}
function inspectTranslationProperty(node, locate) {
const key = propertyNameText(node.name);
if (!key?.startsWith("i18n:")) return;
@@ -315,11 +521,32 @@ function inspectSource(repository, sourceRoot, sourcePath) {
) {
return propertyNameText(current.name);
}
if (
ts.isVariableDeclaration(current) &&
ts.isIdentifier(current.name) &&
(current.name.text === "en" || current.name.text === "de") &&
current.initializer &&
isCatalogObject(current.initializer)
) {
return current.name.text;
}
current = current.parent;
}
return null;
}
function isCatalogObject(node) {
let current = node;
while (
ts.isAsExpression(current) ||
ts.isSatisfiesExpression(current) ||
ts.isParenthesizedExpression(current)
) {
current = current.expression;
}
return ts.isObjectLiteralExpression(current);
}
function ancestorPropertyName(node, expected) {
let current = node.parent;
while (current) {
@@ -343,6 +570,23 @@ function firstAttribute(attributes, names) {
return null;
}
function hasAnyAttribute(attributes, names) {
for (const name of names) {
if (attributes.has(name)) return true;
}
return false;
}
function slug(value) {
const normalized = value
.toLowerCase()
.replace(/^i18n:/, "")
.replace(/\.[a-f0-9]{8}$/, "")
.replace(/[^a-z0-9]+/g, ".")
.replace(/^\.+|\.+$/g, "");
return (normalized || "unnamed").slice(0, 72);
}
function propertyNameText(name) {
if (
ts.isIdentifier(name) ||
+488 -20
View File
@@ -31,6 +31,8 @@ ENDPOINT_SURFACE_CATEGORIES = {
DEFAULT_ENDPOINT_DECLARATIONS = (
META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json"
)
REQUIRED_LOCALES = ("de", "en")
REFERENCE_LOCALE = "de"
def main() -> int:
@@ -45,6 +47,28 @@ def main() -> int:
action="store_true",
help="Fail on missing translations or incomplete endpoint-surface declarations.",
)
parser.add_argument(
"--strict-endpoints",
action="store_true",
help="Fail only on incomplete or stale endpoint-surface declarations.",
)
parser.add_argument(
"--strict-declarations",
action="store_true",
help=(
"Fail on duplicate stable IDs, WebUI surfaces absent from runtime "
"metadata, or stale runtime route declarations."
),
)
parser.add_argument(
"--runtime-snapshot",
type=Path,
help=(
"Compare a saved /api/v1/platform/interface-catalog response with "
"the static manifest inventory. Any installed module combination "
"is accepted; every module present in the snapshot must match."
),
)
parser.add_argument(
"--endpoint-declarations",
type=Path,
@@ -66,6 +90,11 @@ def main() -> int:
backend_endpoints=backend_endpoints,
manifests=manifests,
endpoint_declarations=endpoint_declarations,
runtime_snapshot=(
_load_runtime_snapshot(args.runtime_snapshot.resolve())
if args.runtime_snapshot is not None
else None
),
)
output_dir = args.output_dir.resolve()
@@ -80,20 +109,13 @@ def main() -> int:
print(f"Platform inventory JSON: {json_path}")
print(f"Platform inventory summary: {markdown_path}")
if args.strict:
failures: list[str] = []
if inventory["translation_health"]["missing_catalog_entries"]:
failures.append("used translation keys are missing from generated catalogs")
if inventory["api"]["unclassified_endpoints"]:
failures.append(
f"{len(inventory['api']['unclassified_endpoints'])} backend "
"endpoints have no WebUI evidence or surface declaration"
)
if inventory["api"]["stale_endpoint_declarations"]:
failures.append(
f"{len(inventory['api']['stale_endpoint_declarations'])} "
"endpoint declarations do not match a backend endpoint"
)
if args.strict or args.strict_endpoints or args.strict_declarations:
failures = _strict_failures(
inventory,
check_translations=args.strict,
check_endpoints=args.strict or args.strict_endpoints,
check_declarations=args.strict or args.strict_declarations,
)
if failures:
print(
"Strict platform inventory failed: " + "; ".join(failures) + ".",
@@ -103,6 +125,58 @@ def main() -> int:
return 0
def _strict_failures(
inventory: dict[str, Any],
*,
check_translations: bool,
check_endpoints: bool,
check_declarations: bool = False,
) -> list[str]:
failures: list[str] = []
if (
check_translations
and inventory["translation_health"]["missing_catalog_entries"]
):
failures.append("used translation keys are missing from generated catalogs")
if check_endpoints and inventory["api"]["unclassified_endpoints"]:
failures.append(
f"{len(inventory['api']['unclassified_endpoints'])} backend "
"endpoints have no WebUI evidence or surface declaration"
)
if check_endpoints and inventory["api"]["stale_endpoint_declarations"]:
failures.append(
f"{len(inventory['api']['stale_endpoint_declarations'])} "
"endpoint declarations do not match a backend endpoint"
)
declaration_health = inventory.get("declaration_health", {})
if check_declarations and declaration_health.get("duplicate_ids"):
failures.append(
f"{len(declaration_health['duplicate_ids'])} platform interface "
"IDs are declared more than once"
)
if check_declarations and declaration_health.get("undeclared_source_surfaces"):
failures.append(
f"{len(declaration_health['undeclared_source_surfaces'])} public "
"WebUI surfaces have no runtime manifest declaration"
)
if check_declarations and declaration_health.get("stale_runtime_routes"):
failures.append(
f"{len(declaration_health['stale_runtime_routes'])} runtime route "
"declarations have no WebUI implementation"
)
runtime_comparison = inventory.get("runtime_comparison")
if (
check_declarations
and runtime_comparison is not None
and runtime_comparison.get("mismatches")
):
failures.append(
f"{len(runtime_comparison['mismatches'])} runtime catalog entries "
"do not match the release inventory"
)
return failures
def _resolve_workspace_root(catalog: dict[str, Any]) -> Path:
sibling_root = META_ROOT.parent.resolve()
configured_root = Path(str(catalog["default_parent"])).expanduser().resolve()
@@ -243,6 +317,10 @@ def _extract_manifests(
if (workspace_root / repository["path"] / "src").is_dir()
]
sys.path[:0] = [str(path) for path in source_roots]
from govoplan_core.core.platform_interfaces import ( # noqa: PLC0415
manifest_interface_catalog,
)
manifests: list[dict[str, Any]] = []
for repository in catalog["repositories"]:
source_root = workspace_root / repository["path"] / "src"
@@ -277,15 +355,32 @@ def _extract_manifests(
}
for permission in manifest.permissions
],
"architecture": (
manifest.architecture.to_dict()
if manifest.architecture is not None
else None
),
"information_governance": (
manifest.information_governance.to_dict()
),
"interface_catalog": manifest_interface_catalog(manifest),
"frontend": (
{
"package": frontend.package_name,
"routes": [
_plain_value(route) for route in frontend.routes
],
"public_routes": [
_plain_value(route)
for route in frontend.public_routes
],
"nav_items": [
_plain_value(item) for item in frontend.nav_items
],
"settings_routes": [
_plain_value(route)
for route in frontend.settings_routes
],
"view_surfaces": [
_plain_value(surface)
for surface in frontend.view_surfaces
@@ -305,6 +400,7 @@ def _assemble_inventory(
backend_endpoints: list[dict[str, Any]],
manifests: list[dict[str, Any]],
endpoint_declarations: dict[tuple[str, str, str], dict[str, Any]],
runtime_snapshot: dict[str, Any] | None = None,
) -> dict[str, Any]:
frontend_refs = webui["frontendApiReferences"]
frontend_paths = {
@@ -361,38 +457,60 @@ def _assemble_inventory(
usages = {item["key"] for item in webui["translationUsages"]}
catalogs = webui["translationCatalog"]
catalog_keys = {locale: set(entries) for locale, entries in catalogs.items()}
expected_locales = sorted(catalog_keys)
expected_locales = sorted(set(catalog_keys) | set(REQUIRED_LOCALES))
missing_catalog_entries = [
{
"key": key,
"missing_locales": [
locale for locale in expected_locales if key not in catalog_keys[locale]
locale
for locale in expected_locales
if key not in catalog_keys.get(locale, set())
],
}
for key in sorted(usages)
if any(key not in catalog_keys[locale] for locale in expected_locales)
if any(key not in catalog_keys.get(locale, set()) for locale in expected_locales)
]
fields = webui["fields"]
help_candidates = [field for field in fields if field["helpCandidate"]]
dynamic_help = [field for field in fields if field.get("helpDynamic")]
governance_adoption = Counter(
dimension["adoption"]
for manifest in manifests
for dimension in manifest["information_governance"]["dimensions"].values()
)
source_declarations = _source_interface_declarations(webui, manifests)
declaration_health = _declaration_health(source_declarations, manifests)
runtime_comparison = (
_compare_runtime_snapshot(runtime_snapshot, manifests)
if runtime_snapshot is not None
else None
)
return {
"schema_version": 1,
"schema_version": 2,
"scope": {
"source": "local GovOPlaN repository catalog",
"limitations": [
"Static extraction cannot resolve runtime-computed labels, routes, or API paths.",
"A backend endpoint without a static WebUI reference may intentionally serve public clients, workers, connectors, or external integrations.",
"A field marked as a help candidate may receive contextual help from a surrounding dynamic component.",
"Low-level field and action IDs use line-independent source anchors unless an explicit interfaceId, DOM id, or name is declared.",
],
},
"modules": manifests,
"interface_declarations": source_declarations,
"declaration_health": declaration_health,
"runtime_comparison": runtime_comparison,
"ui": {
"fields": fields,
"actions": webui.get("actions", []),
"labels": webui["labels"],
"visible_text": webui["visibleText"],
"routes": webui["routes"],
"navigation": webui["navigation"],
"ui_capabilities": webui["uiCapabilities"],
"help_candidates": help_candidates,
"dynamic_help": dynamic_help,
"contributions": webui.get("contributions", []),
},
"translations": {
"catalog": catalogs,
@@ -401,9 +519,26 @@ def _assemble_inventory(
},
"translation_health": {
"locales": expected_locales,
"reference_locale": REFERENCE_LOCALE,
"reference_locale_entries": len(catalog_keys.get(REFERENCE_LOCALE, set())),
"reference_locale_complete": not any(
REFERENCE_LOCALE in item["missing_locales"]
for item in missing_catalog_entries
),
"used_keys": len(usages),
"missing_catalog_entries": missing_catalog_entries,
},
"information_governance_health": {
"dimensions": len(manifests) * 4,
"adoption_counts": dict(sorted(governance_adoption.items())),
"modules": [
{
"module_id": manifest["id"],
"dimensions": manifest["information_governance"]["dimensions"],
}
for manifest in manifests
],
},
"api": {
"backend_endpoints": classified_endpoints,
"frontend_references": frontend_refs,
@@ -416,7 +551,18 @@ def _assemble_inventory(
"modules": len(manifests),
"ui_fields": len(fields),
"ui_fields_with_static_help": len(fields) - len(help_candidates),
"ui_fields_with_resolvable_f1_context": len(fields),
"help_review_candidates": len(help_candidates),
"dynamic_help_references": len(dynamic_help),
"ui_actions": len(webui.get("actions", [])),
"interface_declarations": len(source_declarations),
"duplicate_interface_ids": len(declaration_health["duplicate_ids"]),
"undeclared_source_surfaces": len(
declaration_health["undeclared_source_surfaces"]
),
"stale_runtime_routes": len(
declaration_health["stale_runtime_routes"]
),
"label_attributes": len(webui["labels"]),
"visible_text_nodes": len(webui["visibleText"]),
"frontend_routes": len(webui["routes"]),
@@ -425,10 +571,309 @@ def _assemble_inventory(
"backend_endpoints_without_static_webui_reference": len(unreferenced),
"unclassified_backend_endpoints": len(unclassified),
"stale_endpoint_declarations": len(stale_declarations),
"information_governance_dimensions": len(manifests) * 4,
"information_governance_enforced": governance_adoption["enforced"],
"information_governance_partial": governance_adoption["partial"],
"information_governance_contract_only": governance_adoption[
"contract_only"
],
},
}
def _source_interface_declarations(
webui: dict[str, Any],
manifests: list[dict[str, Any]],
) -> list[dict[str, Any]]:
module_by_repository = {
str(manifest["repository"]): str(manifest["id"])
for manifest in manifests
}
declarations: list[dict[str, Any]] = []
def module_id(repository: str) -> str:
if repository in module_by_repository:
return module_by_repository[repository]
if repository == "govoplan-core":
return "core"
return repository.removeprefix("govoplan-").replace("-", "_")
def source_evidence(item: dict[str, Any]) -> dict[str, Any]:
return {
key: item[key]
for key in ("repository", "file", "line", "column")
if key in item
}
for kind, items in (
("field", webui["fields"]),
("action", webui.get("actions", [])),
):
for item in items:
repository = str(item["repository"])
owner = module_id(repository)
raw_id = str(item["id"])
stable_id = (
f"{owner}.{raw_id[len(repository) + 1:]}"
if raw_id.startswith(f"{repository}.")
else _namespaced_interface_id(owner, kind, raw_id)
)
declarations.append(
{
"key": f"{kind}:{stable_id}",
"id": stable_id,
"module_id": owner,
"kind": kind,
"origin": "webui_source",
"id_source": item.get("idSource", "source_anchor"),
"explicit_id": item.get("explicitId"),
"context": item.get("context"),
**source_evidence(item),
}
)
if kind == "field" and item.get("helpId"):
help_raw_id = str(item["helpId"])
help_id = (
f"{owner}.{help_raw_id[len(repository) + 1:]}"
if help_raw_id.startswith(f"{repository}.")
else _namespaced_interface_id(owner, "help", help_raw_id)
)
declarations.append(
{
"key": f"help:{help_id}",
"id": help_id,
"module_id": owner,
"kind": "help",
"origin": "webui_source",
"field_id": stable_id,
"dynamic": bool(item.get("helpDynamic")),
**source_evidence(item),
}
)
for item in webui.get("contributions", []):
repository = str(item["repository"])
owner = module_id(repository)
kind = str(item["kind"])
raw_id = str(item["id"])
path = item.get("path")
if kind == "frontend_route" and isinstance(path, str):
stable_id = f"{owner}.route.{_surface_slug(path)}"
elif kind == "public_route" and isinstance(path, str):
stable_id = f"{owner}.public.{_surface_slug(path)}"
elif kind == "navigation" and isinstance(path, str):
stable_id = f"{owner}.nav.{_surface_slug(path)}"
else:
stable_id = _namespaced_interface_id(owner, kind, raw_id)
declarations.append(
{
"key": f"{kind}:{stable_id}",
"id": stable_id,
"module_id": owner,
"kind": kind,
"origin": "webui_contribution",
"declared_value": raw_id,
**({"path": path} if isinstance(path, str) else {}),
**source_evidence(item),
}
)
translation_entries: dict[tuple[str, str], dict[str, Any]] = {}
for locale, entries in webui["translationCatalog"].items():
for key, item in entries.items():
repository = str(item["repository"])
owner = module_id(repository)
declaration_key = (owner, str(key))
declaration = translation_entries.setdefault(
declaration_key,
{
"key": f"translation:{key}",
"id": key,
"module_id": owner,
"kind": "translation",
"origin": "translation_catalog",
"locales": [],
**source_evidence(item),
},
)
declaration["locales"].append(locale)
declarations.extend(translation_entries.values())
return sorted(
declarations,
key=lambda item: (
item["module_id"],
item["kind"],
item["id"],
item.get("file", ""),
item.get("line", 0),
),
)
def _declaration_health(
source_declarations: list[dict[str, Any]],
manifests: list[dict[str, Any]],
) -> dict[str, Any]:
grouped: dict[tuple[str, str, str], list[dict[str, Any]]] = {}
for declaration in source_declarations:
key = (
str(declaration["module_id"]),
str(declaration["kind"]),
str(declaration["id"]),
)
grouped.setdefault(key, []).append(declaration)
duplicate_ids = [
{
"module_id": key[0],
"kind": key[1],
"id": key[2],
"evidence": values,
}
for key, values in sorted(grouped.items())
if len(values) > 1
]
comparable_kinds = {
"frontend_route",
"navigation",
"public_route",
"view_surface",
}
source_surfaces = {
(str(item["module_id"]), str(item["kind"]), str(item["id"])): item
for item in source_declarations
if item["origin"] == "webui_contribution"
and item["kind"] in comparable_kinds
}
runtime_surfaces: dict[tuple[str, str, str], dict[str, Any]] = {}
for manifest in manifests:
catalog = manifest["interface_catalog"]
for declaration in catalog["declarations"]:
if declaration["kind"] not in comparable_kinds:
continue
key = (
str(manifest["id"]),
str(declaration["kind"]),
str(declaration["id"]),
)
runtime_surfaces[key] = {
"repository": manifest["repository"],
**declaration,
}
surface_id = declaration.get("metadata", {}).get("surface_id")
if (
declaration["kind"]
in {"frontend_route", "navigation", "settings_route"}
and isinstance(surface_id, str)
and surface_id
):
runtime_surfaces[
(str(manifest["id"]), "view_surface", surface_id)
] = {
"repository": manifest["repository"],
"key": f"view_surface:{surface_id}",
"id": surface_id,
"module_id": manifest["id"],
"kind": "view_surface",
"metadata": {
"derived_from": declaration["kind"],
"path": declaration.get("path"),
},
}
undeclared_source_surfaces = [
source_surfaces[key]
for key in sorted(source_surfaces.keys() - runtime_surfaces.keys())
]
route_kinds = {"frontend_route", "public_route"}
stale_runtime_routes = [
runtime_surfaces[key]
for key in sorted(runtime_surfaces.keys() - source_surfaces.keys())
if key[1] in route_kinds
]
return {
"duplicate_ids": duplicate_ids,
"undeclared_source_surfaces": undeclared_source_surfaces,
"stale_runtime_routes": stale_runtime_routes,
"source_declaration_count": len(source_declarations),
"runtime_declaration_count": sum(
len(manifest["interface_catalog"]["declarations"])
for manifest in manifests
),
}
def _compare_runtime_snapshot(
snapshot: dict[str, Any],
manifests: list[dict[str, Any]],
) -> dict[str, Any]:
static_by_module = {
str(manifest["id"]): manifest["interface_catalog"]
for manifest in manifests
}
modules = snapshot.get("modules")
if not isinstance(modules, list):
raise ValueError("Runtime interface snapshot must contain a modules list.")
mismatches: list[dict[str, Any]] = []
seen: set[str] = set()
matched: list[str] = []
for item in modules:
if not isinstance(item, dict) or not isinstance(item.get("module_id"), str):
raise ValueError("Runtime interface snapshot has an invalid module entry.")
module_id = item["module_id"]
if module_id in seen:
mismatches.append({"module_id": module_id, "reason": "duplicate_module"})
continue
seen.add(module_id)
expected = static_by_module.get(module_id)
if expected is None:
mismatches.append({"module_id": module_id, "reason": "unknown_module"})
continue
for field in ("contract_version", "module_version", "digest"):
if item.get(field) != expected.get(field):
mismatches.append(
{
"module_id": module_id,
"reason": f"{field}_mismatch",
"expected": expected.get(field),
"actual": item.get(field),
}
)
if not any(
mismatch["module_id"] == module_id for mismatch in mismatches
):
matched.append(module_id)
return {
"contract_version": snapshot.get("contract_version"),
"matched_modules": sorted(matched),
"mismatches": mismatches,
}
def _load_runtime_snapshot(path: Path) -> dict[str, Any]:
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise ValueError(f"Runtime interface snapshot does not exist: {path}") from exc
except json.JSONDecodeError as exc:
raise ValueError(f"Runtime interface snapshot is invalid JSON: {exc}") from exc
if not isinstance(payload, dict):
raise ValueError("Runtime interface snapshot must be a JSON object.")
return payload
def _namespaced_interface_id(module_id: str, kind: str, value: str) -> str:
if value.startswith(f"{module_id}."):
return value
return f"{module_id}.{kind}.{_surface_slug(value)}"
def _surface_slug(value: str) -> str:
normalized = re.sub(r"[^a-z0-9]+", ".", value.strip().lower()).strip(".")
return normalized or "root"
def _render_markdown(inventory: dict[str, Any]) -> str:
summary = inventory["summary"]
missing = inventory["translation_health"]["missing_catalog_entries"]
@@ -450,8 +895,15 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
"",
f"- Modules: {summary['modules']}",
f"- UI fields: {summary['ui_fields']}",
f"- UI actions: {summary['ui_actions']}",
f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}",
f"- Fields with a resolvable F1 context: {summary['ui_fields_with_resolvable_f1_context']}",
f"- Fields with dynamic help references: {summary['dynamic_help_references']}",
f"- Help review candidates: {summary['help_review_candidates']}",
f"- Stable interface declarations: {summary['interface_declarations']}",
f"- Duplicate interface IDs: {summary['duplicate_interface_ids']}",
f"- WebUI surfaces missing runtime declarations: {summary['undeclared_source_surfaces']}",
f"- Runtime routes missing WebUI implementations: {summary['stale_runtime_routes']}",
f"- Label attributes: {summary['label_attributes']}",
f"- Frontend routes: {summary['frontend_routes']}",
f"- Backend endpoints: {summary['backend_endpoints']}",
@@ -462,7 +914,12 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
),
f"- Unclassified backend endpoints: {summary['unclassified_backend_endpoints']}",
f"- Stale endpoint declarations: {summary['stale_endpoint_declarations']}",
f"- Used translation keys missing from a locale catalog: {len(missing)}",
f"- Reference locale: `{inventory['translation_health']['reference_locale']}`",
f"- Reference locale complete: `{str(inventory['translation_health']['reference_locale_complete']).lower()}`",
f"- Used translation keys missing from a required locale catalog: {len(missing)}",
f"- Information-governance dimensions enforced: {summary['information_governance_enforced']}",
f"- Information-governance dimensions partial: {summary['information_governance_partial']}",
f"- Information-governance dimensions contract-only: {summary['information_governance_contract_only']}",
"",
"## Help Review Candidates",
"",
@@ -505,13 +962,24 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
)
lines.extend(
[
"",
"## Declaration Reconciliation",
"",
"Routes, navigation, View surfaces, fields, actions, help references,",
"translations, settings, widgets, search objects, and backend",
"capabilities use normalized stable IDs. CI rejects duplicate IDs,",
"WebUI public surfaces absent from runtime metadata, and runtime routes",
"without a WebUI implementation.",
"",
"",
"## Interpretation",
"",
"Use the JSON artifact for exact file and line evidence. Missing help is",
"a triage list, not an automatic defect. Endpoint coverage requires an",
"owner classification before enforcement. Runtime-computed structures",
"need explicit manifest metadata to become canonically visible.",
"need explicit manifest or typed PlatformWebModule metadata to become",
"canonically visible. The generated files are release evidence, not an",
"editable source of platform behavior.",
"",
]
)
@@ -0,0 +1,85 @@
schema_version = 1
name = "govoplan-k8s-acceptance"
mode = "acceptance"
state_directory = "~/.local/share/govoplan/labs/govoplan-k8s-acceptance"
vm_image_directory = "/var/lib/libvirt/images/govoplan-labs"
ssh_user = "govoplan"
ssh_private_key = "~/.ssh/govoplan-lab"
ssh_public_key = "~/.ssh/govoplan-lab.pub"
namespace = "govoplan"
public_host = "govoplan.acceptance.example.org"
s3_host = "s3.govoplan.acceptance.example.org"
ingress_class = "traefik"
module_set = "base"
api_replicas = 2
web_replicas = 2
worker_replicas = 2
db_connection_limit = 100
[network]
prefix_length = 24
gateway = "10.77.10.1"
dns_servers = ["10.77.10.1", "1.1.1.1"]
bridge = "br0"
[image]
url = "https://cloud-images.ubuntu.com/releases/noble/release-20260801/ubuntu-24.04-server-cloudimg-amd64.img"
sha256 = "0533b0655c32e68b31d792ecd6ccfca95abdbc536c4446874fe0513bd4140ffe"
[k3s]
version = "v1.36.1+k3s1"
binary_url = "https://github.com/k3s-io/k3s/releases/download/v1.36.1%2Bk3s1/k3s"
binary_sha256 = "a443db3fe9820cd93617ae67e4386d87c1514c1e96ceb30f4c2791c39065653c"
install_script_url = "https://raw.githubusercontent.com/k3s-io/k3s/v1.36.1%2Bk3s1/install.sh"
install_script_sha256 = "46177d4c99440b4c0311b67233823a8e8a2fc09693f6c89af1a7161e152fbfad"
[release]
manifest_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-manifest.json"
manifest_sha256 = "09ac1ade6ede4958bab0dfb7fd8f99246f4d991846308db1f410b25b46267840"
keyring_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-keyring.json"
keyring_sha256 = "92a9f8e3bac0ef525ad9a063c76faa36233a9070cd4a5db9ee9b3f92323b602f"
channel = "stable"
[[nodes]]
name = "control-1"
role = "control"
address = "10.77.10.21"
hypervisor = "lab-admin@hypervisor-state.example.org"
failure_domain = "rack-c"
mac_address = "52:54:00:68:01:01"
cpus = 2
memory_mib = 4096
disk_gib = 32
[[nodes]]
name = "worker-1"
role = "worker"
address = "10.77.10.22"
hypervisor = "lab-admin@hypervisor-a.example.org"
failure_domain = "rack-a"
mac_address = "52:54:00:68:01:02"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "worker-2"
role = "worker"
address = "10.77.10.23"
hypervisor = "lab-admin@hypervisor-b.example.org"
failure_domain = "rack-b"
mac_address = "52:54:00:68:01:03"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "state-1"
role = "state"
address = "10.77.10.24"
hypervisor = "lab-admin@hypervisor-state.example.org"
failure_domain = "rack-c"
mac_address = "52:54:00:68:01:04"
cpus = 4
memory_mib = 8192
disk_gib = 120
+85
View File
@@ -0,0 +1,85 @@
schema_version = 1
name = "govoplan-k8s-lab"
mode = "rehearsal"
state_directory = "~/.local/share/govoplan/labs/govoplan-k8s-lab"
vm_image_directory = "/var/lib/libvirt/images/govoplan-labs"
ssh_user = "govoplan"
ssh_private_key = "~/.ssh/govoplan-lab"
ssh_public_key = "~/.ssh/govoplan-lab.pub"
namespace = "govoplan"
public_host = "govoplan.lab.test"
s3_host = "s3.govoplan.lab.test"
ingress_class = "traefik"
module_set = "base"
api_replicas = 2
web_replicas = 2
worker_replicas = 2
db_connection_limit = 100
[network]
prefix_length = 24
gateway = "192.168.123.1"
dns_servers = ["192.168.123.1", "1.1.1.1"]
bridge = "virbr-gplab"
[image]
url = "https://cloud-images.ubuntu.com/releases/noble/release-20260801/ubuntu-24.04-server-cloudimg-amd64.img"
sha256 = "0533b0655c32e68b31d792ecd6ccfca95abdbc536c4446874fe0513bd4140ffe"
[k3s]
version = "v1.36.1+k3s1"
binary_url = "https://github.com/k3s-io/k3s/releases/download/v1.36.1%2Bk3s1/k3s"
binary_sha256 = "a443db3fe9820cd93617ae67e4386d87c1514c1e96ceb30f4c2791c39065653c"
install_script_url = "https://raw.githubusercontent.com/k3s-io/k3s/v1.36.1%2Bk3s1/install.sh"
install_script_sha256 = "46177d4c99440b4c0311b67233823a8e8a2fc09693f6c89af1a7161e152fbfad"
[release]
manifest_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-manifest.json"
manifest_sha256 = "09ac1ade6ede4958bab0dfb7fd8f99246f4d991846308db1f410b25b46267840"
keyring_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-keyring.json"
keyring_sha256 = "92a9f8e3bac0ef525ad9a063c76faa36233a9070cd4a5db9ee9b3f92323b602f"
channel = "stable"
[[nodes]]
name = "control-1"
role = "control"
address = "192.168.123.201"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:01"
cpus = 2
memory_mib = 4096
disk_gib = 32
[[nodes]]
name = "worker-1"
role = "worker"
address = "192.168.123.202"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:02"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "worker-2"
role = "worker"
address = "192.168.123.203"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:03"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "state-1"
role = "state"
address = "192.168.123.204"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:04"
cpus = 4
memory_mib = 4096
disk_gib = 80
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env python3
"""Executable entry point for the GovOPlaN Kubernetes VM lab."""
from __future__ import annotations
from pathlib import Path
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
from govoplan_lab.cli import main
if __name__ == "__main__":
raise SystemExit(main())
+5
View File
@@ -0,0 +1,5 @@
"""Reproducible GovOPlaN Kubernetes acceptance lab."""
from .config import LabConfig, LabConfigError, LabNode, load_config
__all__ = ["LabConfig", "LabConfigError", "LabNode", "load_config"]
+153
View File
@@ -0,0 +1,153 @@
"""Command-line interface for the GovOPlaN Kubernetes VM lab."""
from __future__ import annotations
import argparse
from pathlib import Path
from typing import Sequence
from .config import LabConfigError, load_config
from .lifecycle import (
LabOperationError,
create,
deploy,
destroy,
doctor,
enroll_admin,
pause,
resume,
status,
update,
verify,
)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="govoplan-lab",
description="Create and operate a libvirt-backed GovOPlaN Kubernetes test lab.",
)
parser.add_argument(
"--config",
type=Path,
default=Path("govoplan-lab.toml"),
help="Strict TOML lab inventory (default: ./govoplan-lab.toml).",
)
parser.add_argument("--verbose", action="store_true")
subparsers = parser.add_subparsers(dest="command", required=True)
doctor_parser = subparsers.add_parser("doctor", help="Validate inventory and prerequisites.")
doctor_parser.add_argument(
"--online",
action="store_true",
help="Also connect to every hypervisor and verify its toolchain.",
)
subparsers.add_parser("status", help="Show VM, Kubernetes node, and pod state.")
_mutation_parser(subparsers, "create", "Create or reuse all declared VMs.")
_mutation_parser(subparsers, "deploy", "Deploy shared state, K3s, and GovOPlaN.")
_mutation_parser(subparsers, "update", "Reconcile pinned K3s and GovOPlaN inputs serially.")
_mutation_parser(subparsers, "pause", "Gracefully stop the lab while preserving disks.")
_mutation_parser(subparsers, "resume", "Start a paused lab in dependency order.")
destroy_parser = _mutation_parser(
subparsers,
"destroy",
"Destroy lab-owned VMs and disks with an explicit name confirmation.",
)
destroy_parser.add_argument("--confirm", default="")
destroy_parser.add_argument(
"--purge-local-state",
action="store_true",
help="Also delete local secrets, manifests, and evidence after VM teardown.",
)
verify_parser = subparsers.add_parser(
"verify",
help=(
"Collect sanitized live-cluster evidence using an "
"Ops-read-authorized GOVOPLAN_OPS_API_KEY."
),
)
verify_parser.add_argument(
"--exercise-api-pod-loss",
action="store_true",
help="Delete one ready API pod during the bounded availability drill.",
)
enroll_parser = _mutation_parser(
subparsers,
"enroll-admin",
"Securely consume the first-administrator enrollment artifact.",
)
enroll_parser.add_argument("--email", required=True)
enroll_parser.add_argument("--display-name", default=None)
enroll_parser.add_argument("--tenant-slug", default="default")
enroll_parser.add_argument("--tenant-name", default="Default Tenant")
return parser
def _mutation_parser(
subparsers: argparse._SubParsersAction[argparse.ArgumentParser],
name: str,
help_text: str,
) -> argparse.ArgumentParser:
parser = subparsers.add_parser(name, help=help_text)
parser.add_argument(
"--apply",
action="store_true",
help="Perform mutations; without this flag the command is a dry run.",
)
return parser
def main(argv: Sequence[str] | None = None) -> int:
args = build_parser().parse_args(argv)
try:
config = load_config(args.config)
if args.command == "doctor":
return doctor(config, online=args.online, verbose=args.verbose)
if args.command == "status":
return status(config, verbose=args.verbose)
if args.command == "create":
create(config, apply=args.apply, verbose=args.verbose)
elif args.command == "deploy":
deploy(config, apply=args.apply, verbose=args.verbose)
elif args.command == "update":
update(config, apply=args.apply, verbose=args.verbose)
elif args.command == "pause":
pause(config, apply=args.apply, verbose=args.verbose)
elif args.command == "resume":
resume(config, apply=args.apply, verbose=args.verbose)
elif args.command == "destroy":
destroy(
config,
apply=args.apply,
confirmation=args.confirm,
purge_local_state=args.purge_local_state,
verbose=args.verbose,
)
elif args.command == "verify":
verify(
config,
exercise_api_pod_loss=args.exercise_api_pod_loss,
verbose=args.verbose,
)
elif args.command == "enroll-admin":
enroll_admin(
config,
email=args.email,
display_name=args.display_name,
tenant_slug=args.tenant_slug,
tenant_name=args.tenant_name,
apply=args.apply,
)
else:
raise RuntimeError(f"unsupported command: {args.command}")
return 0
except (LabConfigError, LabOperationError, OSError, ValueError) as exc:
print(f"error: {exc}", file=__import__("sys").stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+466
View File
@@ -0,0 +1,466 @@
"""Strict TOML model for the GovOPlaN Kubernetes VM lab."""
from __future__ import annotations
from dataclasses import dataclass
import ipaddress
from pathlib import Path
import re
import tomllib
from typing import Any, Mapping
from urllib.parse import urlsplit
SCHEMA_VERSION = 1
_NAME = re.compile(r"^[a-z][a-z0-9-]{1,47}$")
_HOSTNAME = re.compile(
r"^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)*"
r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$"
)
_SHA256 = re.compile(r"^[0-9a-f]{64}$")
_K3S_VERSION = re.compile(r"^v[0-9]+\.[0-9]+\.[0-9]+\+k3s[0-9]+$")
_SSH_TARGET = re.compile(r"^(?:[A-Za-z0-9_.-]+@)?[A-Za-z0-9_.:-]+$")
_BRIDGE = re.compile(r"^[A-Za-z0-9_.:-]{1,32}$")
_ROLE = {"control", "worker", "state"}
_MODE = {"rehearsal", "acceptance"}
class LabConfigError(ValueError):
"""Raised when the lab inventory cannot be used safely."""
@dataclass(frozen=True, slots=True)
class LabNode:
name: str
role: str
address: ipaddress.IPv4Address
hypervisor: str
failure_domain: str
mac_address: str
cpus: int
memory_mib: int
disk_gib: int
@property
def is_local(self) -> bool:
return self.hypervisor == "local"
@dataclass(frozen=True, slots=True)
class NetworkConfig:
prefix_length: int
gateway: ipaddress.IPv4Address
dns_servers: tuple[ipaddress.IPv4Address, ...]
bridge: str
@dataclass(frozen=True, slots=True)
class ImageConfig:
url: str
sha256: str
@dataclass(frozen=True, slots=True)
class K3sConfig:
version: str
binary_url: str
binary_sha256: str
install_script_url: str
install_script_sha256: str
@dataclass(frozen=True, slots=True)
class ReleaseConfig:
manifest_url: str
manifest_sha256: str
keyring_url: str
keyring_sha256: str
channel: str
@dataclass(frozen=True, slots=True)
class LabConfig:
source: Path
schema_version: int
name: str
mode: str
state_directory: Path
vm_image_directory: str
ssh_user: str
ssh_private_key: Path
ssh_public_key: Path
namespace: str
public_host: str
s3_host: str
ingress_class: str
module_set: str
api_replicas: int
web_replicas: int
worker_replicas: int
db_connection_limit: int
network: NetworkConfig
image: ImageConfig
k3s: K3sConfig
release: ReleaseConfig
nodes: tuple[LabNode, ...]
@property
def controls(self) -> tuple[LabNode, ...]:
return tuple(node for node in self.nodes if node.role == "control")
@property
def workers(self) -> tuple[LabNode, ...]:
return tuple(node for node in self.nodes if node.role == "worker")
@property
def state_node(self) -> LabNode:
return next(node for node in self.nodes if node.role == "state")
@property
def primary_control(self) -> LabNode:
return self.controls[0]
@property
def public_url(self) -> str:
return f"https://{self.public_host}"
@property
def s3_url(self) -> str:
return f"https://{self.s3_host}:9443"
@property
def evidence_capable(self) -> bool:
worker_domains = {node.failure_domain for node in self.workers}
worker_hypervisors = {node.hypervisor for node in self.workers}
return (
self.mode == "acceptance"
and len(worker_domains) == len(self.workers)
and len(worker_hypervisors) == len(self.workers)
and self.state_node.failure_domain not in worker_domains
and self.state_node.hypervisor not in worker_hypervisors
)
def load_config(path: Path) -> LabConfig:
source = path.expanduser().resolve()
try:
raw = tomllib.loads(source.read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise LabConfigError(f"lab configuration does not exist: {source}") from exc
except tomllib.TOMLDecodeError as exc:
raise LabConfigError(f"lab configuration is not valid TOML: {exc}") from exc
root = _mapping(raw, "lab configuration")
_only_keys(
root,
{
"schema_version",
"name",
"mode",
"state_directory",
"vm_image_directory",
"ssh_user",
"ssh_private_key",
"ssh_public_key",
"namespace",
"public_host",
"s3_host",
"ingress_class",
"module_set",
"api_replicas",
"web_replicas",
"worker_replicas",
"db_connection_limit",
"network",
"image",
"k3s",
"release",
"nodes",
},
"lab configuration",
)
schema_version = _integer(root, "schema_version")
if schema_version != SCHEMA_VERSION:
raise LabConfigError(
f"schema_version must be {SCHEMA_VERSION}; found {schema_version}"
)
name = _pattern(root, "name", _NAME)
mode = _choice(root, "mode", _MODE)
base = source.parent
state_directory = _path(root, "state_directory", base)
vm_image_directory = _absolute_posix_path(root, "vm_image_directory")
ssh_user = _pattern(root, "ssh_user", re.compile(r"^[a-z_][a-z0-9_-]{0,31}$"))
ssh_private_key = _path(root, "ssh_private_key", base)
ssh_public_key = _path(root, "ssh_public_key", base)
namespace = _pattern(root, "namespace", _NAME)
public_host = _pattern(root, "public_host", _HOSTNAME)
s3_host = _pattern(root, "s3_host", _HOSTNAME)
if public_host == s3_host:
raise LabConfigError("public_host and s3_host must be different")
ingress_class = _pattern(root, "ingress_class", _NAME)
module_set = _choice(root, "module_set", {"core", "base", "full"})
api_replicas = _bounded_integer(root, "api_replicas", 2, 32)
web_replicas = _bounded_integer(root, "web_replicas", 2, 32)
worker_replicas = _bounded_integer(root, "worker_replicas", 2, 64)
db_connection_limit = _bounded_integer(root, "db_connection_limit", 50, 10000)
network = _parse_network(_mapping(root.get("network"), "network"))
image = _parse_image(_mapping(root.get("image"), "image"))
k3s = _parse_k3s(_mapping(root.get("k3s"), "k3s"))
release = _parse_release(_mapping(root.get("release"), "release"))
raw_nodes = root.get("nodes")
if not isinstance(raw_nodes, list) or not raw_nodes:
raise LabConfigError("nodes must be a non-empty array of tables")
nodes = tuple(_parse_node(item, index=index) for index, item in enumerate(raw_nodes))
config = LabConfig(
source=source,
schema_version=schema_version,
name=name,
mode=mode,
state_directory=state_directory,
vm_image_directory=vm_image_directory.rstrip("/"),
ssh_user=ssh_user,
ssh_private_key=ssh_private_key,
ssh_public_key=ssh_public_key,
namespace=namespace,
public_host=public_host,
s3_host=s3_host,
ingress_class=ingress_class,
module_set=module_set,
api_replicas=api_replicas,
web_replicas=web_replicas,
worker_replicas=worker_replicas,
db_connection_limit=db_connection_limit,
network=network,
image=image,
k3s=k3s,
release=release,
nodes=nodes,
)
_validate_topology(config)
return config
def _parse_network(raw: Mapping[str, Any]) -> NetworkConfig:
_only_keys(raw, {"prefix_length", "gateway", "dns_servers", "bridge"}, "network")
prefix_length = _bounded_integer(raw, "prefix_length", 8, 30)
gateway = _ipv4(raw, "gateway")
dns_raw = raw.get("dns_servers")
if not isinstance(dns_raw, list) or not dns_raw or len(dns_raw) > 4:
raise LabConfigError("network.dns_servers must contain 1-4 IPv4 addresses")
dns_servers = tuple(_ipv4_value(value, "network.dns_servers") for value in dns_raw)
bridge = _pattern(raw, "bridge", _BRIDGE)
return NetworkConfig(prefix_length, gateway, dns_servers, bridge)
def _parse_image(raw: Mapping[str, Any]) -> ImageConfig:
_only_keys(raw, {"url", "sha256"}, "image")
return ImageConfig(
url=_https_url(raw, "url"),
sha256=_pattern(raw, "sha256", _SHA256),
)
def _parse_k3s(raw: Mapping[str, Any]) -> K3sConfig:
_only_keys(
raw,
{
"version",
"binary_url",
"binary_sha256",
"install_script_url",
"install_script_sha256",
},
"k3s",
)
return K3sConfig(
version=_pattern(raw, "version", _K3S_VERSION),
binary_url=_https_url(raw, "binary_url"),
binary_sha256=_pattern(raw, "binary_sha256", _SHA256),
install_script_url=_https_url(raw, "install_script_url"),
install_script_sha256=_pattern(raw, "install_script_sha256", _SHA256),
)
def _parse_release(raw: Mapping[str, Any]) -> ReleaseConfig:
_only_keys(
raw,
{"manifest_url", "manifest_sha256", "keyring_url", "keyring_sha256", "channel"},
"release",
)
return ReleaseConfig(
manifest_url=_https_url(raw, "manifest_url"),
manifest_sha256=_pattern(raw, "manifest_sha256", _SHA256),
keyring_url=_https_url(raw, "keyring_url"),
keyring_sha256=_pattern(raw, "keyring_sha256", _SHA256),
channel=_pattern(raw, "channel", _NAME),
)
def _parse_node(value: object, *, index: int) -> LabNode:
raw = _mapping(value, f"nodes[{index}]")
_only_keys(
raw,
{
"name",
"role",
"address",
"hypervisor",
"failure_domain",
"mac_address",
"cpus",
"memory_mib",
"disk_gib",
},
f"nodes[{index}]",
)
hypervisor = _string(raw, "hypervisor")
if hypervisor != "local" and not _SSH_TARGET.fullmatch(hypervisor):
raise LabConfigError(
f"nodes[{index}].hypervisor must be 'local' or a simple SSH target"
)
mac_address = _string(raw, "mac_address").lower()
try:
octets = mac_address.split(":")
valid_mac = len(octets) == 6 and all(
len(octet) == 2 and 0 <= int(octet, 16) <= 255 for octet in octets
)
except ValueError:
valid_mac = False
if not valid_mac:
raise LabConfigError(f"nodes[{index}].mac_address is not a canonical MAC address")
return LabNode(
name=_pattern(raw, "name", _NAME),
role=_choice(raw, "role", _ROLE),
address=_ipv4(raw, "address"),
hypervisor=hypervisor,
failure_domain=_pattern(raw, "failure_domain", _NAME),
mac_address=mac_address,
cpus=_bounded_integer(raw, "cpus", 1, 64),
memory_mib=_bounded_integer(raw, "memory_mib", 2048, 262144),
disk_gib=_bounded_integer(raw, "disk_gib", 16, 4096),
)
def _validate_topology(config: LabConfig) -> None:
names = [node.name for node in config.nodes]
addresses = [node.address for node in config.nodes]
mac_addresses = [node.mac_address for node in config.nodes]
for label, values in (
("node names", names),
("node addresses", addresses),
("node MAC addresses", mac_addresses),
):
if len(values) != len(set(values)):
raise LabConfigError(f"{label} must be unique")
too_long = [
node.name
for node in config.nodes
if len(f"{config.name}-{node.name}") > 63
]
if too_long:
raise LabConfigError(
"lab name plus node name must fit the 63-character libvirt domain limit: "
+ ", ".join(too_long)
)
if len(config.controls) not in {1, 3}:
raise LabConfigError("the lab requires exactly one or three control-plane nodes")
if len(config.workers) < 2:
raise LabConfigError("the lab requires at least two worker nodes")
if sum(node.role == "state" for node in config.nodes) != 1:
raise LabConfigError("the lab requires exactly one external shared-state node")
network = ipaddress.ip_network(
f"{config.network.gateway}/{config.network.prefix_length}", strict=False
)
if any(node.address not in network for node in config.nodes):
raise LabConfigError("every node address must be in the configured IPv4 network")
if config.network.gateway in addresses:
raise LabConfigError("the network gateway cannot also be a node address")
if config.mode == "acceptance" and not config.evidence_capable:
raise LabConfigError(
"acceptance mode requires each worker and the shared-state node to use "
"distinct hypervisors and failure_domain values"
)
def _mapping(value: object, label: str) -> Mapping[str, Any]:
if not isinstance(value, dict):
raise LabConfigError(f"{label} must be a table")
return value
def _only_keys(raw: Mapping[str, Any], allowed: set[str], label: str) -> None:
unexpected = sorted(set(raw) - allowed)
if unexpected:
raise LabConfigError(f"{label} contains unsupported keys: {', '.join(unexpected)}")
def _string(raw: Mapping[str, Any], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str) or not value.strip():
raise LabConfigError(f"{key} must be a non-empty string")
return value.strip()
def _integer(raw: Mapping[str, Any], key: str) -> int:
value = raw.get(key)
if not isinstance(value, int) or isinstance(value, bool):
raise LabConfigError(f"{key} must be an integer")
return value
def _bounded_integer(raw: Mapping[str, Any], key: str, minimum: int, maximum: int) -> int:
value = _integer(raw, key)
if not minimum <= value <= maximum:
raise LabConfigError(f"{key} must be between {minimum} and {maximum}")
return value
def _pattern(raw: Mapping[str, Any], key: str, pattern: re.Pattern[str]) -> str:
value = _string(raw, key)
if pattern.fullmatch(value) is None:
raise LabConfigError(f"{key} has an unsupported format")
return value
def _choice(raw: Mapping[str, Any], key: str, choices: set[str]) -> str:
value = _string(raw, key)
if value not in choices:
raise LabConfigError(f"{key} must be one of: {', '.join(sorted(choices))}")
return value
def _ipv4(raw: Mapping[str, Any], key: str) -> ipaddress.IPv4Address:
return _ipv4_value(_string(raw, key), key)
def _ipv4_value(value: object, label: str) -> ipaddress.IPv4Address:
if not isinstance(value, str):
raise LabConfigError(f"{label} must contain strings")
try:
parsed = ipaddress.ip_address(value)
except ValueError as exc:
raise LabConfigError(f"{label} contains an invalid IP address") from exc
if not isinstance(parsed, ipaddress.IPv4Address):
raise LabConfigError(f"{label} supports IPv4 only in schema version 1")
return parsed
def _path(raw: Mapping[str, Any], key: str, base: Path) -> Path:
value = Path(_string(raw, key)).expanduser()
return (value if value.is_absolute() else base / value).resolve()
def _absolute_posix_path(raw: Mapping[str, Any], key: str) -> str:
value = _string(raw, key)
if not value.startswith("/") or ".." in Path(value).parts:
raise LabConfigError(f"{key} must be an absolute path without '..'")
return value
def _https_url(raw: Mapping[str, Any], key: str) -> str:
value = _string(raw, key)
parsed = urlsplit(value)
if parsed.scheme != "https" or not parsed.hostname or parsed.username or parsed.password:
raise LabConfigError(f"{key} must be a credential-free HTTPS URL")
if parsed.fragment:
raise LabConfigError(f"{key} must not contain a fragment")
return value
File diff suppressed because it is too large Load Diff
+311
View File
@@ -0,0 +1,311 @@
"""Deterministic configuration rendering for the Kubernetes VM lab."""
from __future__ import annotations
import base64
import json
from pathlib import Path
from typing import Mapping
from .config import LabConfig, LabNode
def render_cloud_init(config: LabConfig, node: LabNode, public_key: str) -> str:
packages = ["ca-certificates", "curl", "qemu-guest-agent"]
if node.role == "state":
packages.extend(["docker.io", "docker-compose-v2", "openssl"])
package_lines = "\n".join(f" - {item}" for item in packages)
return f"""#cloud-config
hostname: {node.name}
manage_etc_hosts: true
package_update: true
package_upgrade: false
packages:
{package_lines}
users:
- default
- name: {config.ssh_user}
groups: [adm, sudo]
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
lock_passwd: true
ssh_authorized_keys:
- {json.dumps(public_key.strip())}
ssh_pwauth: false
disable_root: true
runcmd:
- [systemctl, enable, --now, qemu-guest-agent]
- [sh, -c, "test ! -e /usr/bin/docker || systemctl enable --now docker"]
final_message: "GovOPlaN lab node is ready"
"""
def render_network_config(config: LabConfig, node: LabNode) -> str:
dns = ", ".join(str(item) for item in config.network.dns_servers)
return f"""version: 2
ethernets:
primary:
match:
macaddress: {node.mac_address}
set-name: eth0
addresses:
- {node.address}/{config.network.prefix_length}
routes:
- to: default
via: {config.network.gateway}
nameservers:
addresses: [{dns}]
"""
def render_meta_data(config: LabConfig, node: LabNode) -> str:
return f"instance-id: {config.name}-{node.name}\nlocal-hostname: {node.name}\n"
def render_k3s_config(
config: LabConfig,
node: LabNode,
*,
cluster_token: str,
) -> str:
lines = [
f'node-name: "{node.name}"',
f'node-ip: "{node.address}"',
f'token: "{cluster_token}"',
]
if node.role == "control":
if node == config.primary_control:
lines.append("cluster-init: true")
else:
lines.append(f'server: "https://{config.primary_control.address}:6443"')
lines.extend(
[
'write-kubeconfig-mode: "0600"',
"secrets-encryption: true",
"tls-san:",
f' - "{config.primary_control.address}"',
"node-taint:",
' - "node-role.kubernetes.io/control-plane=true:NoSchedule"',
]
)
elif node.role == "worker":
lines.extend(
[
f'server: "https://{config.primary_control.address}:6443"',
"node-label:",
' - "govoplan.add-ideas.de/runtime=true"',
f' - "topology.govoplan.add-ideas.de/failure-domain={node.failure_domain}"',
]
)
else:
raise ValueError("state nodes do not receive K3s configuration")
return "\n".join(lines) + "\n"
def render_registry_config(username: str, password: str) -> str:
if not username and not password:
return ""
if not username or not password:
raise ValueError("registry username and password must be supplied together")
return (
'mirrors:\n "git.add-ideas.de":\n'
' endpoint:\n - "https://git.add-ideas.de"\n'
'configs:\n "git.add-ideas.de":\n auth:\n'
f" username: {json.dumps(username)}\n"
f" password: {json.dumps(password)}\n"
)
def render_garage_config() -> str:
return """metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
replication_factor = 1
rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage.localhost"
[admin]
api_bind_addr = "[::]:3903"
"""
def render_caddyfile() -> str:
return """:9443 {
tls /etc/caddy/tls/server.crt /etc/caddy/tls/server.key
reverse_proxy garage:3900
}
"""
def render_state_compose(images: Mapping[str, str]) -> str:
required = {"postgres", "redis", "garage", "managed_ingress", "test_mail"}
missing = sorted(required - set(images))
if missing:
raise ValueError("release manifest is missing state images: " + ", ".join(missing))
value = {
"name": "govoplan-lab-state",
"services": {
"postgres": {
"image": images["postgres"],
"restart": "unless-stopped",
"environment": {
"POSTGRES_DB": "${POSTGRES_DB}",
"POSTGRES_USER": "${POSTGRES_USER}",
"POSTGRES_PASSWORD": "${POSTGRES_PASSWORD}",
},
"ports": ["5432:5432"],
"healthcheck": {
"test": [
"CMD-SHELL",
'pg_isready -U "$${POSTGRES_USER}" -d "$${POSTGRES_DB}"',
],
"interval": "5s",
"timeout": "3s",
"retries": 30,
},
"volumes": ["postgres-data:/var/lib/postgresql/data"],
"networks": ["internal"],
},
"redis": {
"image": images["redis"],
"restart": "unless-stopped",
"command": [
"sh",
"-ec",
'exec redis-server --appendonly yes --requirepass "$$REDIS_PASSWORD"',
],
"environment": {"REDIS_PASSWORD": "${REDIS_PASSWORD}"},
"ports": ["6379:6379"],
"healthcheck": {
"test": [
"CMD-SHELL",
'redis-cli -a "$${REDIS_PASSWORD}" --no-auth-warning ping',
],
"interval": "5s",
"timeout": "3s",
"retries": 30,
},
"volumes": ["redis-data:/data"],
"networks": ["internal"],
},
"garage": {
"image": images["garage"],
"restart": "unless-stopped",
"command": ["/garage", "server", "--single-node", "--default-bucket"],
"environment": {
"GARAGE_DEFAULT_ACCESS_KEY": "${GARAGE_DEFAULT_ACCESS_KEY}",
"GARAGE_DEFAULT_SECRET_KEY": "${GARAGE_DEFAULT_SECRET_KEY}",
"GARAGE_DEFAULT_BUCKET": "${GARAGE_DEFAULT_BUCKET}",
"GARAGE_RPC_SECRET": "${GARAGE_RPC_SECRET}",
"GARAGE_ADMIN_TOKEN": "${GARAGE_ADMIN_TOKEN}",
"GARAGE_METRICS_TOKEN": "${GARAGE_METRICS_TOKEN}",
},
"healthcheck": {
"test": ["CMD", "/garage", "status"],
"interval": "10s",
"timeout": "5s",
"retries": 30,
"start_period": "15s",
},
"security_opt": ["no-new-privileges:true"],
"volumes": [
"./garage.toml:/etc/garage.toml:ro",
"garage-meta:/var/lib/garage/meta",
"garage-data:/var/lib/garage/data",
],
"networks": ["internal"],
},
"s3-tls": {
"image": images["managed_ingress"],
"restart": "unless-stopped",
"depends_on": {"garage": {"condition": "service_healthy"}},
"ports": ["9443:9443"],
"volumes": [
"./Caddyfile:/etc/caddy/Caddyfile:ro",
"./tls:/etc/caddy/tls:ro",
],
"security_opt": ["no-new-privileges:true"],
"networks": ["internal"],
},
"test-mail": {
"image": images["test_mail"],
"restart": "unless-stopped",
"environment": {
"GREENMAIL_OPTS": (
"-Dgreenmail.setup.test.smtp -Dgreenmail.setup.test.imap "
"-Dgreenmail.hostname=0.0.0.0"
)
},
"ports": ["3025:3025", "3143:3143"],
"networks": ["internal"],
},
},
"volumes": {
"postgres-data": {},
"redis-data": {},
"garage-meta": {},
"garage-data": {},
},
"networks": {"internal": {"driver": "bridge"}},
}
return json.dumps(value, indent=2, sort_keys=True) + "\n"
def render_state_environment(values: Mapping[str, str]) -> str:
return "".join(f"{key}={_env_quote(value)}\n" for key, value in sorted(values.items()))
def render_secret_manifest(
*, namespace: str, name: str, values: Mapping[str, bytes | str]
) -> bytes:
encoded = {
key: base64.b64encode(value.encode() if isinstance(value, str) else value).decode()
for key, value in sorted(values.items())
}
payload = {
"apiVersion": "v1",
"kind": "Secret",
"metadata": {"name": name, "namespace": namespace},
"type": "Opaque",
"data": encoded,
}
return (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
def render_tls_secret_manifest(
*, namespace: str, name: str, certificate: bytes, private_key: bytes
) -> bytes:
payload = json.loads(
render_secret_manifest(
namespace=namespace,
name=name,
values={"tls.crt": certificate, "tls.key": private_key},
)
)
payload["type"] = "kubernetes.io/tls"
return (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
def render_hosts(config: LabConfig) -> str:
return (
f"{config.primary_control.address} {config.public_host}\n"
f"{config.state_node.address} {config.s3_host}\n"
)
def _env_quote(value: str) -> str:
return json.dumps(value, ensure_ascii=True)
def write_private(path: Path, value: str | bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
data = value.encode() if isinstance(value, str) else value
path.write_bytes(data)
path.chmod(0o600)
@@ -0,0 +1,11 @@
<network>
<name>govoplan-lab</name>
<forward mode="nat"/>
<bridge name="virbr-gplab" stp="on" delay="0"/>
<domain name="lab.test" localOnly="yes"/>
<ip address="192.168.123.1" netmask="255.255.255.0">
<dhcp>
<range start="192.168.123.2" end="192.168.123.99"/>
</dhcp>
</ip>
</network>
@@ -26,6 +26,7 @@ def build_parser() -> argparse.ArgumentParser:
parser.add_argument("--web-metadata", type=Path, required=True)
parser.add_argument("--deployer", type=Path, required=True)
parser.add_argument("--deployer-url", required=True)
parser.add_argument("--package-lock", type=Path, required=True)
parser.add_argument("--artifact-base-url", required=True)
parser.add_argument("--source-commit", required=True)
parser.add_argument("--version", required=True)
@@ -45,6 +46,11 @@ def build_parser() -> argparse.ArgumentParser:
def finalize(args: argparse.Namespace) -> dict[str, Any]:
composition = _json_object(args.composition)
_validate_package_lock(
_json_object(args.package_lock),
version=args.version,
composition=composition,
)
api = _image_metadata(_json_object(args.api_metadata), "api")
web = _image_metadata(_json_object(args.web_metadata), "web")
dependencies = dict(_dependency(value) for value in args.dependency)
@@ -99,6 +105,10 @@ def finalize(args: argparse.Namespace) -> dict[str, Any]:
"url": args.deployer_url,
"sha256": _sha256_file(args.deployer),
},
"package_lock": {
"url": f"{artifact_base}/package-artifacts.lock.json",
"sha256": _sha256_file(args.package_lock),
},
"images": {
"api": {
**api,
@@ -247,6 +257,57 @@ def _json_object(path: Path) -> dict[str, Any]:
return value
def _validate_package_lock(
lock: dict[str, Any],
*,
version: str,
composition: dict[str, Any],
) -> None:
if lock.get("schema_version") != "1" or lock.get("release_version") != version:
raise ValueError("package lock schema or release version does not match")
unsigned = dict(lock)
expected_hash = unsigned.pop("lock_sha256", None)
actual_hash = hashlib.sha256(
json.dumps(unsigned, sort_keys=True, separators=(",", ":")).encode("utf-8")
).hexdigest()
if expected_hash != actual_hash:
raise ValueError("package lock hash does not match its contents")
rows = lock.get("python")
if not isinstance(rows, list):
raise ValueError("package lock Python artifacts are missing")
locked = _package_identities(rows, name_key="name")
composed = _package_identities(
composition["python"]["packages"],
name_key="package",
)
if locked != composed:
raise ValueError("package lock does not match the runtime wheel composition")
def _package_identities(
rows: list[object],
*,
name_key: str,
) -> dict[str, tuple[str, str]]:
identities: dict[str, tuple[str, str]] = {}
for row in rows:
if not isinstance(row, dict):
raise ValueError("package artifact identity is malformed")
name = row.get(name_key)
version = row.get("version")
digest = row.get("sha256")
if (
not isinstance(name, str)
or not isinstance(version, str)
or not isinstance(digest, str)
or SHA256.fullmatch(digest) is None
or name in identities
):
raise ValueError("package artifact identity is malformed or duplicated")
identities[name] = (version, digest)
return identities
def _https_url(value: str, label: str) -> str:
parsed = urlsplit(value)
if parsed.scheme != "https" or not parsed.netloc or parsed.username or parsed.password:
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""Generate the optional GovOPlaN developer convenience meta-package."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
import tomllib
META_ROOT = Path(__file__).resolve().parents[2]
DIRECT = re.compile(r"^(govoplan-[a-z0-9-]+)(?:\[([^]]+)\])?\s+@\s+.*@v([A-Za-z0-9._+!-]+)$")
LOCAL_CORE = re.compile(r"^(?:-e\s+)?\.\./govoplan-core(?:\[([^]]+)\])?$")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--workspace", type=Path, default=META_ROOT.parent)
parser.add_argument(
"--requirements",
type=Path,
default=META_ROOT / "requirements-release.txt",
)
parser.add_argument(
"--output",
type=Path,
default=META_ROOT / "packages" / "govoplan-meta" / "pyproject.toml",
)
parser.add_argument("--check", action="store_true")
return parser
def render(*, workspace: Path, requirements: Path) -> str:
core = tomllib.loads((workspace / "govoplan-core/pyproject.toml").read_text(encoding="utf-8"))["project"]
version = str(core["version"])
base: list[str] = []
for raw in requirements.read_text(encoding="utf-8").splitlines():
line = raw.strip()
if not line or line.startswith("#"):
continue
local = LOCAL_CORE.fullmatch(line)
if local:
extra = f"[{local.group(1)}]" if local.group(1) else ""
base.append(f"govoplan-core{extra}=={version}")
continue
match = DIRECT.fullmatch(line)
if match is None:
raise ValueError(f"unsupported release requirement: {line!r}")
extra = f"[{match.group(2)}]" if match.group(2) else ""
base.append(f"{match.group(1)}{extra}=={match.group(3)}")
base_names = {_requirement_name(item) for item in base}
full: list[str] = []
for project_path in sorted(workspace.glob("govoplan-*/pyproject.toml")):
project = tomllib.loads(project_path.read_text(encoding="utf-8"))["project"]
name = str(project.get("name") or "")
package_version = str(project.get("version") or "")
if name.startswith("govoplan-") and name not in base_names:
full.append(f"{name}=={package_version}")
return "\n".join(
[
"[build-system]",
'requires = ["setuptools>=69", "wheel"]',
'build-backend = "setuptools.build_meta"',
"",
"[project]",
'name = "govoplan"',
f'version = {json.dumps(version)}',
'description = "Developer convenience package for a versioned GovOPlaN composition"',
'readme = "README.md"',
'requires-python = ">=3.12"',
'license = { text = "AGPL-3.0-or-later" }',
"dependencies = [",
*[f" {json.dumps(item)}," for item in base],
"]",
"",
"[project.optional-dependencies]",
"full = [",
*[f" {json.dumps(item)}," for item in full],
"]",
"",
"[project.urls]",
'Repository = "https://git.add-ideas.de/GovOPlaN/govoplan"',
'Documentation = "https://govoplan.add-ideas.de"',
"",
"[tool.setuptools.packages.find]",
'where = ["src"]',
"",
]
)
def _requirement_name(value: str) -> str:
return value.split("[", 1)[0].split("==", 1)[0]
def main() -> int:
args = build_parser().parse_args()
try:
expected = render(
workspace=args.workspace.expanduser().resolve(),
requirements=args.requirements.expanduser().resolve(),
)
except (OSError, KeyError, ValueError, tomllib.TOMLDecodeError) as exc:
print(f"error: {exc}")
return 1
output = args.output.expanduser()
current = output.read_text(encoding="utf-8") if output.is_file() else None
if args.check:
if current != expected:
print(f"error: developer meta-package is stale: {output}")
return 1
print("Developer meta-package is synchronized.")
return 0
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(expected, encoding="utf-8")
print(f"Developer meta-package written to {output}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -332,6 +332,7 @@ def _manifest_catalog_metadata(manifest: ModuleManifest | None) -> dict[str, obj
payload["optional_dependencies"] = list(manifest.optional_dependencies)
if manifest.architecture is not None:
payload["architecture"] = manifest.architecture.to_dict()
payload["information_governance"] = manifest.information_governance.to_dict()
if manifest.external_providers:
payload["external_providers"] = [
declaration.to_dict()
+8 -5
View File
@@ -94,9 +94,6 @@ cleanup() {
trap cleanup EXIT
cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"
if [[ -f "$WEBUI/package-lock.release.json" ]]; then
cp "$WEBUI/package-lock.release.json" "$TMP_DIR/package-lock.json"
fi
echo "Generating release lockfile from $WEBUI/package.release.json"
echo "Temporary workspace: $TMP_DIR"
@@ -120,7 +117,10 @@ GIT_ENV+=("GIT_CONFIG_COUNT=$git_config_count")
(
cd "$TMP_DIR"
"${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" install --package-lock-only --ignore-scripts
"${GIT_ENV[@]}" \
"npm_config_cache=$TMP_DIR/npm-cache" \
PATH="$(dirname "$NPM_BIN"):$PATH" \
"$NPM_BIN" install --package-lock-only --ignore-scripts
mapfile -t GIT_PACKAGES < <(
PATH="$(dirname "$NODE_BIN"):$PATH" "$NODE_BIN" <<'NODE'
const fs = require("fs");
@@ -136,7 +136,10 @@ NODE
)
if [[ "${#GIT_PACKAGES[@]}" -gt 0 ]]; then
echo "Refreshing git package lock entries: ${GIT_PACKAGES[*]}"
"${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
"${GIT_ENV[@]}" \
"npm_config_cache=$TMP_DIR/npm-cache" \
PATH="$(dirname "$NPM_BIN"):$PATH" \
"$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
fi
)
@@ -0,0 +1,196 @@
#!/usr/bin/env python3
"""Generate the exact registry package set for a GovOPlaN runtime release."""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import re
import subprocess
import tomllib
META_ROOT = Path(__file__).resolve().parents[2]
NAME = re.compile(r"^govoplan-[a-z0-9-]+$")
VERSION = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._+!-]{0,127}$")
GIT_REQUIREMENT = re.compile(
r"^(?P<package>govoplan-[a-z0-9-]+)(?:\[(?P<extras>[^]]+)\])?\s+@\s+"
r"(?P<url>git\+[^\s]+/GovOPlaN/(?P<repo>govoplan-[a-z0-9-]+)\.git@v"
r"(?P<version>[A-Za-z0-9._+!-]+))$"
)
LOCAL_CORE = re.compile(r"^(?:-e\s+)?\.\./govoplan-core(?:\[(?P<extras>[^]]+)\])?$")
class PackageSetError(ValueError):
"""Release source references cannot form an immutable package set."""
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--version",
help="Core/meta release version. Defaults to the workspace Core version.",
)
parser.add_argument(
"--requirements",
type=Path,
default=META_ROOT / "requirements-release.txt",
)
parser.add_argument("--workspace", type=Path, default=META_ROOT.parent)
parser.add_argument("--output", type=Path, required=True)
return parser
def parse_release_requirements(path: Path, *, core_version: str) -> tuple[dict[str, object], ...]:
values: list[dict[str, object]] = []
for line_number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
line = raw.strip()
if not line or line.startswith("#"):
continue
local = LOCAL_CORE.fullmatch(line)
if local:
values.append(
{
"name": "govoplan-core",
"version": core_version.removeprefix("v"),
"repository": "govoplan-core",
"extras": _extras(local.group("extras")),
}
)
continue
match = GIT_REQUIREMENT.fullmatch(line)
if match is None:
raise PackageSetError(
f"unsupported release requirement at {path}:{line_number}: {line!r}"
)
values.append(
{
"name": match.group("package"),
"version": match.group("version"),
"repository": match.group("repo"),
"extras": _extras(match.group("extras")),
}
)
names = [str(item["name"]) for item in values]
if not values or names.count("govoplan-core") != 1 or len(names) != len(set(names)):
raise PackageSetError("release requirements must contain one Core and unique packages")
return tuple(values)
def generate_package_set(
*,
core_version: str,
requirements: Path,
workspace: Path,
) -> dict[str, object]:
core_version = core_version.removeprefix("v")
if VERSION.fullmatch(core_version) is None:
raise PackageSetError("release version is invalid")
python_packages: list[dict[str, object]] = []
webui_packages: list[dict[str, object]] = []
seen_webui: set[str] = set()
for requirement in parse_release_requirements(requirements, core_version=core_version):
repository = workspace / str(requirement["repository"])
tag = f"v{requirement['version']}"
if not (repository / ".git").is_dir():
raise PackageSetError(f"release repository is missing: {repository}")
commit = _git(repository, "rev-list", "-n", "1", tag)
if not commit:
raise PackageSetError(f"release tag is missing: {repository.name}@{tag}")
project = tomllib.loads(_git(repository, "show", f"{tag}:pyproject.toml"))["project"]
if project.get("name") != requirement["name"] or project.get("version") != requirement["version"]:
raise PackageSetError(f"tag metadata does not match {repository.name}@{tag}")
entry = {
**requirement,
"tag": tag,
"commit": commit,
}
python_packages.append(entry)
try:
webui_raw = _git(repository, "show", f"{tag}:webui/package.json")
except subprocess.CalledProcessError:
continue
webui = json.loads(webui_raw)
webui_name = webui.get("name")
if (
not isinstance(webui_name, str)
or not webui_name.startswith("@govoplan/")
or webui.get("version") != requirement["version"]
or webui_name in seen_webui
):
raise PackageSetError(f"WebUI tag metadata does not match {repository.name}@{tag}")
seen_webui.add(webui_name)
webui_packages.append(
{
"name": webui_name,
"version": requirement["version"],
"repository": requirement["repository"],
"tag": tag,
"commit": commit,
}
)
payload: dict[str, object] = {
"schema_version": "1",
"release_version": core_version,
"registries": {
"python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
"npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/",
},
"python": python_packages,
"webui": webui_packages,
}
payload["package_set_sha256"] = _canonical_sha256(payload)
return payload
def _extras(value: str | None) -> list[str]:
if not value:
return []
extras = sorted({item.strip() for item in value.split(",") if item.strip()})
if any(re.fullmatch(r"[a-z][a-z0-9_-]*", item) is None for item in extras):
raise PackageSetError("release requirement contains an invalid extra")
return extras
def _git(repository: Path, *arguments: str) -> str:
return subprocess.check_output(
["git", "-C", str(repository), *arguments],
text=True,
stderr=subprocess.DEVNULL,
).strip()
def _canonical_sha256(value: object) -> str:
encoded = json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def main() -> int:
args = build_parser().parse_args()
try:
workspace = args.workspace.expanduser().resolve()
version = args.version
if not version:
core = tomllib.loads(
(workspace / "govoplan-core/pyproject.toml").read_text(encoding="utf-8")
)
version = str(core["project"]["version"])
payload = generate_package_set(
core_version=version,
requirements=args.requirements.expanduser().resolve(),
workspace=workspace,
)
except (PackageSetError, OSError, ValueError, subprocess.CalledProcessError) as exc:
print(f"error: {exc}")
return 1
output = args.output.expanduser()
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print(f"Release package set written to {output}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -167,6 +167,7 @@ def manifest_catalog_entry(
entry["optional_dependencies"] = list(manifest.optional_dependencies)
if manifest.architecture is not None:
entry["architecture"] = manifest.architecture.to_dict()
entry["information_governance"] = manifest.information_governance.to_dict()
if manifest.external_providers:
entry["external_providers"] = [
declaration.to_dict()
@@ -8,6 +8,9 @@ WEBUI_DIR="${1:-$CORE_ROOT/webui}"
WORK_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/govoplan-webui-release-deps.XXXXXXXX")"
GOVOPLAN_DEPS="$WORK_ROOT/govoplan-webui-deps.tsv"
export GOVOPLAN_DEPS
PACKAGE_LOCK="${GOVOPLAN_WEBUI_PACKAGE_LOCK:-}"
PACKAGE_DIR="${GOVOPLAN_WEBUI_PACKAGE_DIR:-}"
PYTHON_BIN="${PYTHON:-python3}"
trap 'rm -rf "$WORK_ROOT"' EXIT
@@ -55,9 +58,69 @@ rm -f package-lock.json
npm cache clean --force
retry npm install --prefer-online
if [[ -n "$PACKAGE_LOCK" || -n "$PACKAGE_DIR" ]]; then
[[ -n "$PACKAGE_LOCK" && -n "$PACKAGE_DIR" ]] || {
echo "GOVOPLAN_WEBUI_PACKAGE_LOCK and GOVOPLAN_WEBUI_PACKAGE_DIR must be set together" >&2
exit 1
}
"$PYTHON_BIN" - "$PACKAGE_LOCK" "$PACKAGE_DIR" "$GOVOPLAN_DEPS" <<'PY'
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import sys
lock_path = Path(sys.argv[1]).resolve()
package_dir = Path(sys.argv[2]).resolve()
output = Path(sys.argv[3])
lock = json.loads(lock_path.read_text(encoding="utf-8"))
if lock.get("schema_version") != "1" or not isinstance(lock.get("webui"), list):
raise SystemExit("WebUI package lock is malformed")
unsigned = dict(lock)
expected_lock_hash = unsigned.pop("lock_sha256", None)
actual_lock_hash = hashlib.sha256(
json.dumps(unsigned, sort_keys=True, separators=(",", ":")).encode("utf-8")
).hexdigest()
if expected_lock_hash != actual_lock_hash:
raise SystemExit("WebUI package lock hash does not match its contents")
rows = {}
for item in lock["webui"]:
if not isinstance(item, dict) or not isinstance(item.get("name"), str):
raise SystemExit("WebUI package lock contains a malformed artifact")
if item["name"] in rows:
raise SystemExit(f"WebUI package lock contains duplicate artifact {item['name']}")
rows[item["name"]] = item
requested = []
for line in output.read_text(encoding="utf-8").splitlines():
if not line:
continue
name, _source_ref = line.split("\t", 1)
row = rows.get(name)
if not isinstance(row, dict):
raise SystemExit(f"WebUI package lock has no artifact for {name}")
filename = row.get("filename")
if not isinstance(filename, str) or Path(filename).name != filename:
raise SystemExit(f"WebUI package lock has an invalid filename for {name}")
artifact = package_dir / filename
if artifact.is_symlink() or not artifact.is_file():
raise SystemExit(f"WebUI package artifact is missing for {name}")
encoded = artifact.read_bytes()
if len(encoded) != row.get("size") or hashlib.sha256(encoded).hexdigest() != row.get("sha256"):
raise SystemExit(f"WebUI package artifact hash does not match for {name}")
requested.append(f"{name}\tfile:{artifact}")
output.write_text("\n".join(requested) + "\n", encoding="utf-8")
PY
fi
module_paths=()
while IFS=$'\t' read -r package_name spec; do
[[ -n "${package_name:-}" ]] || continue
if [[ "$spec" == file:* ]]; then
echo "Installing $package_name from verified package artifact"
module_paths+=("$spec")
continue
fi
git_url="${spec%%#*}"
git_ref="${spec#*#}"
if [[ "$git_url" == "$spec" || -z "$git_ref" ]]; then
+5
View File
@@ -154,6 +154,11 @@ def prepare_context(
output / "nginx.conf",
source_date_epoch=source_date_epoch,
)
_copy_regular(
Path(__file__).resolve().parent / "runtime" / "web-entrypoint.sh",
output / "web-entrypoint.sh",
source_date_epoch=source_date_epoch,
)
return composition
+86 -7
View File
@@ -333,20 +333,53 @@ path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(\s*version=)["\'][^"\']+["\'](,?\s*)$',
r'(?m)^(MODULE_VERSION\s*=\s*)["\'][^"\']+["\'](\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count == 0:
text, count = re.subn(
r'(?m)^(MODULE_VERSION\s*=\s*)["\'][^"\']+["\'](\s*)$',
r'(?ms)(^manifest\s*=\s*ModuleManifest\(.*?^\s*version\s*=\s*)["\'][^"\']+["\'](,?\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update ModuleManifest.version in {path}")
raise SystemExit(f"could not update module version declaration in {path}")
path.write_text(text)
PYCODE
done
}
update_package_init_versions() {
local repo="$1"
local version="$2"
local package_init=""
for package_init in "$repo"/src/*/__init__.py; do
[[ -f "$package_init" ]] || continue
if ! grep -q '^__version__\s*=' "$package_init"; then
continue
fi
"$PYTHON" - "$package_init" "$version" <<'PYCODE'
from __future__ import annotations
import pathlib
import re
import sys
path = pathlib.Path(sys.argv[1])
new_version = sys.argv[2]
text = path.read_text()
text, count = re.subn(
r'(?m)^(__version__\s*=\s*)["\'][^"\']+["\'](\s*)$',
rf'\1"{new_version}"\2',
text,
count=1,
)
if count != 1:
raise SystemExit(f"could not update __version__ in {path}")
path.write_text(text)
PYCODE
done
@@ -380,6 +413,11 @@ if project_name != "govoplan-core":
peers["@govoplan/core-webui"] = f"^{new_version}"
path.write_text(json.dumps(data, indent=2) + "\n")
PYCODE
done
"$PYTHON" "$META_ROOT/tools/release/synchronize-webui-package-metadata.py" --repo "$repo"
for package_path in "$repo/package.json" "$repo/webui/package.json"; do
[[ -f "$package_path" ]] || continue
synchronize_lockfile_root "$package_path" "${package_path%package.json}package-lock.json"
done
@@ -428,7 +466,19 @@ if not isinstance(version, str) or not version:
lock["version"] = version
packages = lock.get("packages")
if isinstance(packages, dict) and isinstance(packages.get(""), dict):
packages[""]["version"] = version
root = packages[""]
root["version"] = version
for group in (
"dependencies",
"devDependencies",
"optionalDependencies",
"peerDependencies",
"peerDependenciesMeta",
):
if group in package:
root[group] = package[group]
else:
root.pop(group, None)
lock_path.write_text(json.dumps(lock, indent=2) + "\n")
PYCODE
}
@@ -456,6 +506,13 @@ path.write_text(updated)
PYCODE
}
update_developer_meta_package() {
"$PYTHON" "$META_ROOT/tools/release/generate-developer-meta-package.py" \
--workspace "$PARENT" \
--requirements "$META_ROOT/requirements-release.txt" \
--output "$META_ROOT/packages/govoplan-meta/pyproject.toml"
}
update_version_files() {
local repo="$1"
local version="$2"
@@ -463,6 +520,7 @@ update_version_files() {
update_pyproject "$repo" "$version"
update_manifest_version "$repo" "$project_name" "$version"
update_package_init_versions "$repo" "$version"
update_webui_package "$repo" "$project_name" "$version"
}
@@ -493,10 +551,11 @@ run_version_alignment_gate() {
"$PYTHON"
"$META_ROOT/tools/checks/check-version-alignment.py"
--workspace-root "$PARENT"
--release-composition
)
if [[ "$mode" == "source" ]]; then
command+=(--source-metadata-only)
else
command+=(--release-composition)
fi
local repo
for repo in "${PACKAGE_REPOS[@]}"; do
@@ -526,7 +585,8 @@ run_migration_release_audit() {
command+=("--strict")
;;
auto)
command+=("--strict-if-baseline")
# A coordinated release creates a new baseline after confirmation. The
# preflight validates the graph; strictness applies to that new baseline.
;;
warn)
;;
@@ -539,6 +599,18 @@ run_migration_release_audit() {
run "${command[@]}"
}
record_migration_release_baseline() {
local audit_script="$META_ROOT/tools/release/release-migration-audit.py"
[[ -f "$audit_script" ]] || fail "missing migration audit helper: $audit_script"
run "$PYTHON" "$audit_script" \
--track release \
--record-release "$TARGET_VERSION"
if [[ "$DRY_RUN" -eq 0 ]]; then
"$PYTHON" "$audit_script" --track release --strict
fi
}
print_command() {
printf '+'
printf ' %q' "$@"
@@ -865,6 +937,8 @@ run_manifest_shape_gate
confirm_release
record_migration_release_baseline
for repo in "${PACKAGE_REPOS[@]}"; do
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "Would update version files in $repo to $TARGET_VERSION"
@@ -875,8 +949,10 @@ done
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "Would update $META_ROOT/requirements-release.txt to $TAG"
echo "Would synchronize packages/govoplan-meta/pyproject.toml"
else
update_release_requirements "$TARGET_VERSION"
update_developer_meta_package
fi
refresh_development_webui_lock
@@ -929,10 +1005,13 @@ fi
run git -C "$ROOT" commit -m "$COMMIT_MESSAGE"
run git -C "$ROOT" tag -a "$TAG" -m "$TAG_MESSAGE"
for repo in "${PRE_CORE_REPOS[@]}"; do
for repo in "${MODULE_REPOS[@]}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
run git -C "$ROOT" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$ROOT]}" "refs/tags/$TAG"
for repo in "${SUPPORT_REPOS[@]}"; do
run git -C "$repo" push --atomic "$REMOTE" "HEAD:refs/heads/${BRANCHES[$repo]}" "refs/tags/$TAG"
done
if [[ "$PUBLISH_WEB_CATALOG" -eq 1 ]]; then
CATALOG_ARGS=(
+361
View File
@@ -0,0 +1,361 @@
#!/usr/bin/env python3
"""Download, verify, and lock exact GovOPlaN registry package artifacts."""
from __future__ import annotations
import argparse
import base64
from email.parser import BytesParser
from email.policy import compat32
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import shutil
import subprocess
import sys
import tarfile
import tempfile
from urllib.parse import quote, urlsplit, urlunsplit
import zipfile
NAME = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
WEBUI_NAME = re.compile(r"^@govoplan/[a-z0-9]+(?:-[a-z0-9]+)*-webui$")
VERSION = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._+!-]{0,127}$")
COMMIT = re.compile(r"^[0-9a-f]{40}$")
MAX_ARTIFACT_BYTES = 512 * 1024 * 1024
class PackageArtifactError(ValueError):
"""Registry artifacts do not match the selected package set."""
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--package-set", type=Path, required=True)
parser.add_argument("--wheelhouse", type=Path, required=True)
parser.add_argument("--webui-packages", type=Path, required=True)
parser.add_argument("--lock-output", type=Path, required=True)
parser.add_argument("--requirements-output", type=Path)
parser.add_argument("--python", default=sys.executable)
parser.add_argument("--npm", default="npm")
return parser
def resolve(args: argparse.Namespace) -> dict[str, object]:
package_set = _load_package_set(args.package_set)
wheelhouse = args.wheelhouse.expanduser().resolve()
webui_packages = args.webui_packages.expanduser().resolve()
_require_empty_destination(wheelhouse)
_require_empty_destination(webui_packages)
wheelhouse.parent.mkdir(parents=True, exist_ok=True)
webui_packages.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="govoplan-package-resolution-") as value:
temporary = Path(value)
wheels = temporary / "wheels"
webui = temporary / "webui"
wheels.mkdir()
webui.mkdir()
_download_wheels(
packages=tuple(package_set["python"]),
destination=wheels,
python=args.python,
index_url=str(package_set["registries"]["python"]),
)
_download_webui(
packages=tuple(package_set["webui"]),
destination=webui,
npm=args.npm,
registry=str(package_set["registries"]["npm"]),
)
python_rows = _verify_wheels(tuple(package_set["python"]), wheels)
webui_rows = _verify_webui(tuple(package_set["webui"]), webui)
lock: dict[str, object] = {
"schema_version": "1",
"release_version": package_set["release_version"],
"package_set_sha256": package_set["package_set_sha256"],
"registries": package_set["registries"],
"python": python_rows,
"webui": webui_rows,
}
lock["lock_sha256"] = _canonical_sha256(lock)
shutil.copytree(wheels, wheelhouse, dirs_exist_ok=True)
shutil.copytree(webui, webui_packages, dirs_exist_ok=True)
args.lock_output.parent.mkdir(parents=True, exist_ok=True)
args.lock_output.write_text(json.dumps(lock, indent=2, sort_keys=True) + "\n", encoding="utf-8")
if args.requirements_output is not None:
_write_requirements(args.requirements_output, python_rows)
return lock
def _load_package_set(path: Path) -> dict[str, object]:
value = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(value, dict) or value.get("schema_version") != "1":
raise PackageArtifactError("package set has an unsupported shape")
expected_hash = value.get("package_set_sha256")
unsigned = dict(value)
unsigned.pop("package_set_sha256", None)
if expected_hash != _canonical_sha256(unsigned):
raise PackageArtifactError("package set hash does not match its contents")
registries = value.get("registries")
if not isinstance(registries, dict) or set(registries) != {"python", "npm"}:
raise PackageArtifactError("package set registries are invalid")
for registry in registries.values():
parsed = urlsplit(str(registry))
if parsed.scheme != "https" or not parsed.netloc or parsed.username or parsed.password:
raise PackageArtifactError("package registries must use credential-free HTTPS URLs")
for group in ("python", "webui"):
packages = value.get(group)
if not isinstance(packages, list) or not packages:
raise PackageArtifactError(f"package set {group} entries are missing")
_validate_package_entries(group, packages)
return value
def _validate_package_entries(group: str, packages: list[object]) -> None:
names: set[str] = set()
for raw in packages:
if not isinstance(raw, dict):
raise PackageArtifactError(f"package set {group} entry is malformed")
name = raw.get("name")
version = raw.get("version")
repository = raw.get("repository")
tag = raw.get("tag")
commit = raw.get("commit")
name_valid = (
isinstance(name, str)
and (NAME.fullmatch(name) if group == "python" else WEBUI_NAME.fullmatch(name))
)
if (
not name_valid
or name in names
or not isinstance(version, str)
or VERSION.fullmatch(version) is None
or not isinstance(repository, str)
or NAME.fullmatch(repository) is None
or tag != f"v{version}"
or not isinstance(commit, str)
or COMMIT.fullmatch(commit) is None
):
raise PackageArtifactError(f"package set {group} entry has an invalid identity")
names.add(name)
if group == "python":
extras = raw.get("extras")
if not isinstance(extras, list) or any(
not isinstance(item, str)
or re.fullmatch(r"[a-z][a-z0-9_-]*", item) is None
for item in extras
):
raise PackageArtifactError("package set Python extras are invalid")
def _download_wheels(
*, packages: tuple[dict[str, object], ...], destination: Path, python: str, index_url: str
) -> None:
requirements = [_python_requirement(item) for item in packages]
environment = dict(os.environ)
environment["PIP_INDEX_URL"] = _authenticated_url(index_url)
environment["PIP_EXTRA_INDEX_URL"] = ""
environment["PIP_CONFIG_FILE"] = os.devnull
environment["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
subprocess.run(
[python, "-m", "pip", "download", "--no-deps", "--only-binary=:all:", "--dest", str(destination), *requirements],
check=True,
env=environment,
)
def _download_webui(
*, packages: tuple[dict[str, object], ...], destination: Path, npm: str, registry: str
) -> None:
environment = dict(os.environ)
npmrc: tempfile.NamedTemporaryFile[bytes] | None = None
token = os.environ.get("GOVOPLAN_PACKAGE_TOKEN", "")
if token:
parsed = urlsplit(registry)
auth_path = f"//{parsed.netloc}{parsed.path}:_authToken={token}\n"
npmrc = tempfile.NamedTemporaryFile(prefix="govoplan-npmrc-", delete=False)
npmrc.write(f"@govoplan:registry={registry}\n{auth_path}".encode("utf-8"))
npmrc.close()
os.chmod(npmrc.name, 0o600)
environment["NPM_CONFIG_USERCONFIG"] = npmrc.name
try:
for item in packages:
subprocess.run(
[npm, "pack", f"{item['name']}@{item['version']}", "--ignore-scripts", "--pack-destination", str(destination), "--registry", registry],
check=True,
env=environment,
)
finally:
if npmrc is not None:
Path(npmrc.name).unlink(missing_ok=True)
def _verify_wheels(packages: tuple[dict[str, object], ...], root: Path) -> list[dict[str, object]]:
expected = {_normalize(str(item["name"])): item for item in packages}
rows: list[dict[str, object]] = []
seen: set[str] = set()
for path in sorted(root.glob("*.whl")):
identity = _wheel_identity(path)
name = str(identity["name"])
package = expected.get(name)
if package is None or identity["version"] != package["version"] or name in seen:
raise PackageArtifactError(f"unexpected wheel artifact: {path.name}")
seen.add(name)
rows.append(_artifact_row(path, package))
if seen != set(expected):
raise PackageArtifactError("registry did not return every selected Python wheel")
return sorted(rows, key=lambda item: str(item["name"]))
def _verify_webui(packages: tuple[dict[str, object], ...], root: Path) -> list[dict[str, object]]:
expected = {str(item["name"]): item for item in packages}
rows: list[dict[str, object]] = []
seen: set[str] = set()
for path in sorted(root.glob("*.tgz")):
identity = _npm_identity(path)
name = str(identity["name"])
package = expected.get(name)
if package is None or identity["version"] != package["version"] or name in seen:
raise PackageArtifactError(f"unexpected WebUI artifact: {path.name}")
seen.add(name)
row = _artifact_row(path, package)
row["integrity"] = "sha512-" + base64.b64encode(hashlib.sha512(path.read_bytes()).digest()).decode("ascii")
rows.append(row)
if seen != set(expected):
raise PackageArtifactError("registry did not return every selected WebUI package")
return sorted(rows, key=lambda item: str(item["name"]))
def _wheel_identity(path: Path) -> dict[str, str]:
_bounded(path)
with zipfile.ZipFile(path) as archive:
metadata = [
item for item in archive.infolist()
if PurePosixPath(item.filename).name == "METADATA"
and PurePosixPath(item.filename).parent.name.endswith(".dist-info")
]
if len(metadata) != 1 or metadata[0].file_size > 1024 * 1024:
raise PackageArtifactError(f"wheel metadata is invalid: {path.name}")
parsed = BytesParser(policy=compat32).parsebytes(archive.read(metadata[0]))
name = _normalize(str(parsed.get("Name") or ""))
version = str(parsed.get("Version") or "")
if NAME.fullmatch(name) is None or VERSION.fullmatch(version) is None:
raise PackageArtifactError(f"wheel identity is invalid: {path.name}")
return {"name": name, "version": version}
def _npm_identity(path: Path) -> dict[str, str]:
_bounded(path)
with tarfile.open(path, mode="r:gz") as archive:
try:
member = archive.getmember("package/package.json")
except KeyError as exc:
raise PackageArtifactError(f"npm package metadata is missing: {path.name}") from exc
if not member.isfile() or member.size > 1024 * 1024:
raise PackageArtifactError(f"npm package metadata is invalid: {path.name}")
extracted = archive.extractfile(member)
if extracted is None:
raise PackageArtifactError(f"npm package metadata cannot be read: {path.name}")
value = json.load(extracted)
name = value.get("name")
version = value.get("version")
if not isinstance(name, str) or not name.startswith("@govoplan/") or not isinstance(version, str) or VERSION.fullmatch(version) is None:
raise PackageArtifactError(f"npm package identity is invalid: {path.name}")
return {"name": name, "version": version}
def _artifact_row(path: Path, package: dict[str, object]) -> dict[str, object]:
row = {
"name": package["name"],
"version": package["version"],
"repository": package["repository"],
"tag": package["tag"],
"commit": package["commit"],
"filename": path.name,
"sha256": hashlib.sha256(path.read_bytes()).hexdigest(),
"size": path.stat().st_size,
}
if "extras" in package:
row["extras"] = package["extras"]
return row
def _write_requirements(path: Path, rows: list[dict[str, object]]) -> None:
lines = ["--no-index", "--find-links ./local-wheels", "--require-hashes"]
for row in rows:
selected_extras = row.get("extras") or []
extras = (
f"[{','.join(str(value) for value in selected_extras)}]"
if selected_extras
else ""
)
lines.append(
f"{row['name']}{extras}=={row['version']} "
f"--hash=sha256:{row['sha256']}"
)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
def _python_requirement(item: dict[str, object]) -> str:
extras = item.get("extras") or []
suffix = f"[{','.join(str(value) for value in extras)}]" if extras else ""
return f"{item['name']}{suffix}=={item['version']}"
def _authenticated_url(url: str) -> str:
token = os.environ.get("GOVOPLAN_PACKAGE_TOKEN", "")
username = os.environ.get("GOVOPLAN_PACKAGE_USERNAME", "")
if not token:
return url
if not username:
raise PackageArtifactError("GOVOPLAN_PACKAGE_USERNAME is required with a package token")
parsed = urlsplit(url)
return urlunsplit(
(
parsed.scheme,
f"{quote(username, safe='')}:{quote(token, safe='')}@{parsed.netloc}",
parsed.path,
parsed.query,
"",
)
)
def _require_empty_destination(path: Path) -> None:
if path.exists() and (not path.is_dir() or any(path.iterdir())):
raise PackageArtifactError(f"output directory must be absent or empty: {path}")
if path.is_symlink():
raise PackageArtifactError(f"output directory must not be a symlink: {path}")
def _bounded(path: Path) -> None:
if path.is_symlink() or not path.is_file() or path.stat().st_size > MAX_ARTIFACT_BYTES:
raise PackageArtifactError(f"package artifact is invalid or too large: {path.name}")
def _normalize(value: str) -> str:
return re.sub(r"[-_.]+", "-", value.strip().lower())
def _canonical_sha256(value: object) -> str:
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8")).hexdigest()
def main() -> int:
args = build_parser().parse_args()
try:
lock = resolve(args)
except (PackageArtifactError, OSError, ValueError, subprocess.CalledProcessError, zipfile.BadZipFile, tarfile.TarError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 1
print(f"Resolved {len(lock['python'])} Python and {len(lock['webui'])} WebUI packages.")
print(f"Package artifact lock written to {args.lock_output}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+6 -4
View File
@@ -11,11 +11,13 @@ LABEL org.opencontainers.image.title="GovOPlaN WebUI runtime" \
USER 0
RUN rm -rf /usr/share/nginx/html/* /etc/nginx/conf.d/*
COPY web-dist/ /usr/share/nginx/html/
COPY nginx.conf /etc/nginx/nginx.conf
COPY nginx.conf /etc/nginx/nginx.conf.template
COPY web-entrypoint.sh /usr/local/bin/govoplan-web-entrypoint
RUN chown -R 101:101 /usr/share/nginx/html \
&& chmod -R a-w /usr/share/nginx/html /etc/nginx/nginx.conf
&& chmod -R a-w /usr/share/nginx/html /etc/nginx/nginx.conf.template \
&& chmod 0555 /usr/local/bin/govoplan-web-entrypoint
USER 101:101
EXPOSE 8080
ENTRYPOINT []
CMD ["nginx", "-g", "daemon off;"]
ENTRYPOINT ["/usr/local/bin/govoplan-web-entrypoint"]
CMD []
+1 -1
View File
@@ -35,7 +35,7 @@ http {
}
location /api/ {
proxy_pass http://load-balancer:8000;
proxy_pass ${GOVOPLAN_API_UPSTREAM};
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Synchronize duplicated publish and development WebUI package contracts."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
SYNCHRONIZED_KEYS = ("peerDependencies", "peerDependenciesMeta")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--repo", type=Path, required=True)
args = parser.parse_args()
root_path = args.repo / "package.json"
webui_path = args.repo / "webui" / "package.json"
if not root_path.exists() or not webui_path.exists():
return 0
root = _load(root_path)
webui = _load(webui_path)
root_name = root.get("name")
webui_name = webui.get("name")
if not isinstance(root_name, str) or root_name != webui_name:
return 0
changed = False
for key in SYNCHRONIZED_KEYS:
if key in webui:
value = webui[key]
if root.get(key) != value:
root[key] = value
changed = True
elif key in root:
del root[key]
changed = True
if changed:
root_path.write_text(json.dumps(root, indent=2) + "\n")
print(f"Synchronized WebUI peer metadata in {root_path}")
return 0
def _load(path: Path) -> dict[str, object]:
payload = json.loads(path.read_text())
if not isinstance(payload, dict):
raise SystemExit(f"package metadata must be an object: {path}")
return payload
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,78 @@
#!/usr/bin/env python3
"""Install or verify the canonical package-release workflow in module repos."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
META_ROOT = Path(__file__).resolve().parents[2]
TEMPLATE = META_ROOT / "tools" / "repo" / "templates" / "module-package-release.yml"
DESTINATION = Path(".gitea/workflows/module-package-release.yml")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--check", action="store_true")
mode.add_argument("--write", action="store_true")
parser.add_argument(
"--parent",
type=Path,
default=META_ROOT.parent,
help="Parent directory containing the repositories.",
)
return parser
def package_repositories(parent: Path) -> tuple[Path, ...]:
inventory = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
repositories: list[Path] = []
for item in inventory["repositories"]:
name = str(item["name"])
if not name.startswith("govoplan-"):
continue
repository = parent / str(item["path"])
if (repository / "pyproject.toml").is_file():
repositories.append(repository)
return tuple(sorted(repositories))
def synchronize(*, parent: Path, write: bool) -> tuple[str, ...]:
expected = TEMPLATE.read_bytes()
mismatches: list[str] = []
for repository in package_repositories(parent):
destination = repository / DESTINATION
current = destination.read_bytes() if destination.is_file() else None
if current == expected:
continue
mismatches.append(repository.name)
if write:
destination.parent.mkdir(parents=True, exist_ok=True)
temporary = destination.with_suffix(destination.suffix + ".tmp")
temporary.write_bytes(expected)
temporary.chmod(0o644)
temporary.replace(destination)
return tuple(mismatches)
def main() -> int:
args = build_parser().parse_args()
mismatches = synchronize(parent=args.parent.expanduser().resolve(), write=args.write)
if args.write:
print(f"Installed package-release workflow in {len(mismatches)} repositories.")
return 0
if mismatches:
print("Package-release workflow is missing or stale in:", file=sys.stderr)
for repository in mismatches:
print(f"- {repository}", file=sys.stderr)
return 1
print("Package-release workflows are synchronized.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,270 @@
name: Module Package Release
on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
release_tag:
description: Existing protected version tag to publish
required: true
type: string
jobs:
publish-packages:
runs-on: ubuntu-latest
env:
GITEA_REPOSITORY: ${{ gitea.repository }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: "3.12"
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "22"
- name: Select and validate protected release tag
shell: bash
env:
REQUESTED_TAG: ${{ inputs.release_tag }}
TRIGGER_TAG: ${{ gitea.ref_name }}
run: |
set -euo pipefail
tag="${REQUESTED_TAG:-$TRIGGER_TAG}"
case "$tag" in
v[0-9]*.[0-9]*.[0-9]*) ;;
*) echo "Release tag must start with a SemVer-shaped vX.Y.Z value" >&2; exit 1 ;;
esac
git fetch --force origin "refs/tags/$tag:refs/tags/$tag" refs/heads/main:refs/remotes/origin/main
tag_commit="$(git rev-list -n 1 "$tag")"
git merge-base --is-ancestor "$tag_commit" refs/remotes/origin/main || {
echo "Release tag is not contained in main" >&2
exit 1
}
git checkout --detach "$tag"
printf 'RELEASE_TAG=%s\n' "$tag" >> "$GITEA_ENV"
printf 'SOURCE_DATE_EPOCH=%s\n' "$(git show -s --format=%ct HEAD)" >> "$GITEA_ENV"
- name: Validate package versions
run: |
python - <<'PY'
import json
from pathlib import Path
import os
import re
import tomllib
tag = os.environ["RELEASE_TAG"]
expected = tag.removeprefix("v")
project = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))["project"]
if project.get("version") != expected:
raise SystemExit(f"pyproject version {project.get('version')!r} does not match {tag}")
if re.fullmatch(r"govoplan-[a-z0-9-]+", str(project.get("name", ""))) is None:
raise SystemExit("Python distribution name must use the govoplan-* namespace")
webui = Path("webui/package.json")
if webui.is_file():
package = json.loads(webui.read_text(encoding="utf-8"))
if package.get("version") != expected:
raise SystemExit(f"WebUI version {package.get('version')!r} does not match {tag}")
if re.fullmatch(r"@govoplan/[a-z0-9-]+-webui", str(package.get("name", ""))) is None:
raise SystemExit("WebUI package name must use the @govoplan/*-webui namespace")
release = Path("webui/package.release.json")
if release.is_file():
release_package = json.loads(release.read_text(encoding="utf-8"))
if (
release_package.get("name") != package.get("name")
or release_package.get("version") != expected
):
raise SystemExit("WebUI release package identity does not match package.json and the release tag")
PY
- name: Build immutable package artifacts
shell: bash
run: |
set -euo pipefail
python -m pip install --disable-pip-version-check build==1.5.0 twine==7.0.0
rm -rf dist .package-webui
python -m build --wheel --outdir dist
python -m twine check dist/*.whl
if [[ -f webui/package.json ]]; then
mkdir .package-webui
cp -a webui/. .package-webui/
rm -rf .package-webui/node_modules .package-webui/dist
if [[ -f .package-webui/package.release.json ]]; then
cp .package-webui/package.release.json .package-webui/package.json
fi
node <<'NODE'
const fs = require("node:fs");
const path = ".package-webui/package.json";
const packageJson = JSON.parse(fs.readFileSync(path, "utf8"));
const groups = ["dependencies", "optionalDependencies", "peerDependencies"];
for (const group of groups) {
for (const [name, specifier] of Object.entries(packageJson[group] || {})) {
if (!name.startsWith("@govoplan/")) continue;
if (typeof specifier !== "string") {
throw new Error(`${group}.${name} must use a string version`);
}
const packageSlug = name.slice("@govoplan/".length);
if (!packageSlug.endsWith("-webui")) {
throw new Error(`${group}.${name} is outside the WebUI package namespace`);
}
const repository = `govoplan-${packageSlug.slice(0, -"-webui".length)}`;
const escapedRepository = repository.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const gitTag = specifier.match(
new RegExp(
`^git\\+(?:ssh://git@|https://)git\\.add-ideas\\.de/(?:GovOPlaN|add-ideas)/${escapedRepository}\\.git#v([0-9]+\\.[0-9]+\\.[0-9]+)$`,
),
);
if (gitTag) {
packageJson[group][name] = gitTag[1];
continue;
}
if (specifier.startsWith("file:") || specifier.startsWith("git+")) {
throw new Error(
`${group}.${name} must resolve to an exact registry version for publication`,
);
}
}
}
delete packageJson.private;
fs.writeFileSync(path, `${JSON.stringify(packageJson, null, 2)}\n`);
NODE
npm pkg delete private --prefix .package-webui
(cd .package-webui && npm pack --ignore-scripts --pack-destination ../dist)
fi
python - <<'PY'
import hashlib
import json
from pathlib import Path
import os
import subprocess
artifacts = []
for path in sorted(Path("dist").iterdir()):
if path.suffix not in {".whl", ".tgz"}:
continue
digest = hashlib.sha256(path.read_bytes()).hexdigest()
artifacts.append({"filename": path.name, "sha256": digest, "size": path.stat().st_size})
payload = {
"schema_version": "1",
"repository": os.environ["GITEA_REPOSITORY"],
"tag": os.environ["RELEASE_TAG"],
"commit": subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip(),
"artifacts": artifacts,
}
Path("dist/package-artifacts.json").write_text(
json.dumps(payload, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
PY
- name: Retain package hash evidence
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: module-packages-${{ gitea.ref_name }}
path: dist/package-artifacts.json
- name: Check immutable registry state
shell: bash
env:
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_TOKEN"
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import tomllib
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
api_root = "https://git.add-ideas.de/api/v1/packages/GovOPlaN"
token = os.environ["PACKAGE_TOKEN"]
def should_publish(kind, name, version, path):
package_url = "/".join(
(api_root, kind, quote(name, safe=""), quote(version, safe=""), "files")
)
request = Request(
package_url,
headers={"Accept": "application/json", "Authorization": f"token {token}"},
)
try:
with urlopen(request, timeout=30) as response:
files = json.load(response)
except HTTPError as exc:
if exc.code == 404:
print(f"{kind} package {name}=={version} is not published yet")
return True
raise
if not isinstance(files, list) or len(files) != 1:
raise SystemExit(
f"immutable {kind} package {name}=={version} has an unexpected file set"
)
expected_sha256 = hashlib.sha256(path.read_bytes()).hexdigest()
if files[0].get("sha256") != expected_sha256:
raise SystemExit(
f"immutable {kind} package {name}=={version} already exists with a different SHA-256"
)
print(f"verified existing {kind} package {name}=={version} ({expected_sha256})")
return False
project = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))["project"]
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("release build must contain exactly one wheel")
publish_pypi = should_publish(
"pypi", str(project["name"]), str(project["version"]), wheels[0]
)
tarballs = tuple(Path("dist").glob("*.tgz"))
if len(tarballs) > 1:
raise SystemExit("release build must contain at most one npm package")
publish_npm = False
if tarballs:
webui = json.loads(
Path(".package-webui/package.json").read_text(encoding="utf-8")
)
publish_npm = should_publish(
"npm", str(webui["name"]), str(webui["version"]), tarballs[0]
)
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"PUBLISH_PYPI={int(publish_pypi)}\n")
env_file.write(f"PUBLISH_NPM={int(publish_npm)}\n")
PY
- name: Publish wheel and WebUI package
shell: bash
env:
PACKAGE_USERNAME: ${{ secrets.GOVOPLAN_PACKAGE_USERNAME }}
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_USERNAME"
test -n "$PACKAGE_TOKEN"
if [[ "$PUBLISH_PYPI" == 1 ]]; then
TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \
python -m twine upload --non-interactive \
--repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \
dist/*.whl
else
echo "Exact wheel is already present; skipping immutable retry."
fi
shopt -s nullglob
webui_packages=(dist/*.tgz)
if (( ${#webui_packages[@]} )) && [[ "$PUBLISH_NPM" == 1 ]]; then
npmrc="$(mktemp)"
trap 'rm -f "$npmrc"' EXIT
chmod 600 "$npmrc"
printf '%s\n' \
'@govoplan:registry=https://git.add-ideas.de/api/packages/GovOPlaN/npm/' \
"//git.add-ideas.de/api/packages/GovOPlaN/npm/:_authToken=$PACKAGE_TOKEN" \
> "$npmrc"
NPM_CONFIG_USERCONFIG="$npmrc" npm publish "./${webui_packages[0]}" \
--ignore-scripts --access public \
--registry https://git.add-ideas.de/api/packages/GovOPlaN/npm/
elif (( ${#webui_packages[@]} )); then
echo "Exact WebUI package is already present; skipping immutable retry."
fi