19 Commits
Author SHA1 Message Date
zemion 612a44bc8e feat(deploy): project infrastructure capabilities
Dependency Audit / dependency-audit (push) Successful in 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m43s
2026-08-07 01:59:45 +02:00
zemion dce725636d feat(release): disclose module permissions in catalog 2026-08-07 01:59:18 +02:00
zemion 78811f7f6e Synchronize public module directory publication
Dependency Audit / dependency-audit (push) Successful in 1m50s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m58s
2026-08-06 22:42:25 +02:00
zemion 09046e6e62 Publish complete signed module catalogs
Dependency Audit / dependency-audit (push) Successful in 1m46s
Deployment Installer / deployment-installer (push) Successful in 5s
Security Audit / security-audit (push) Successful in 11m49s
2026-08-06 21:13:33 +02:00
zemion f3cfd1bccc Integrate Quick Access product presentation
Dependency Audit / dependency-audit (push) Successful in 1m50s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m40s
2026-08-06 19:02:56 +02:00
zemion 241db623c7 Refresh live strategy issue counts
Dependency Audit / dependency-audit (push) Successful in 1m54s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 12m9s
2026-08-06 16:40:46 +02:00
zemion b269791c48 Reconcile product inputs and prove durable work handoffs
Dependency Audit / dependency-audit (push) Successful in 1m48s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 12m2s
2026-08-06 16:06:18 +02:00
zemion 69ba1037bf Prove reference journey compositions
Dependency Audit / dependency-audit (push) Successful in 1m45s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m16s
2026-08-06 12:42:20 +02:00
zemion 8bdf7b5f7e docs(records): record lifecycle and endpoint coverage
Dependency Audit / dependency-audit (push) Successful in 1m38s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m32s
2026-08-06 05:36:29 +02:00
zemion d6fdd7ddf5 chore(inventory): track remaining records administration surfaces
Dependency Audit / dependency-audit (push) Successful in 1m41s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m47s
2026-08-06 01:43:37 +02:00
zemion 7b0ab31adf Document GovOPlaN deployment profiles
Dependency Audit / dependency-audit (push) Successful in 1m45s
Deployment Installer / deployment-installer (push) Successful in 7s
Security Audit / security-audit (push) Successful in 11m33s
2026-08-05 22:42:27 +02:00
zemion 389df7c3d5 Drain API pods before Kubernetes shutdown
Dependency Audit / dependency-audit (push) Successful in 1m51s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m37s
2026-08-05 21:55:32 +02:00
zemion 3b9ae901dd Harden Kubernetes lab CA profile [skip ci] 2026-08-05 21:17:32 +02:00
zemion bf2f02891f Release v0.1.18
Dependency Audit / dependency-audit (push) Successful in 1m48s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m31s
Developer Meta-package Release / publish-package (push) Successful in 10s
2026-08-05 21:07:52 +02:00
zemion 7ec0bbb826 Release v0.1.17
Dependency Audit / dependency-audit (push) Failing after 1m50s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m34s
Developer Meta-package Release / publish-package (push) Successful in 10s
2026-08-05 20:34:17 +02:00
zemion 3ca068f76a Release v0.1.16
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m18s
Dependency Audit / dependency-audit (push) Failing after 1m47s
Developer Meta-package Release / publish-package (push) Successful in 11s
2026-08-05 19:52:32 +02:00
zemion 61463a24cb Bind runtime releases to protected source tags [skip ci] 2026-08-04 16:43:54 +02:00
zemion 8262215fcd Keep runtime builder path stable [skip ci] 2026-08-04 16:34:07 +02:00
zemion 8aba74e01e Make meta-package release retries tag-safe [skip ci] 2026-08-04 16:27:47 +02:00
87 changed files with 8735 additions and 626 deletions
+2 -2
View File
@@ -15,7 +15,7 @@ GOVOPLAN_DB_MAX_OVERFLOW=10
GOVOPLAN_DB_POOL_TIMEOUT_SECONDS=30 GOVOPLAN_DB_POOL_TIMEOUT_SECONDS=30
GOVOPLAN_DB_POOL_RECYCLE_SECONDS=1800 GOVOPLAN_DB_POOL_RECYCLE_SECONDS=1800
ENABLED_MODULES=tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,files,templates,mail,campaigns,calendar,poll,scheduling,connectors,datasources,dataflow,dist_lists,workflow_engine,workflow,views,search,risk_compliance,postbox,notifications,services,parties,mandates,decisions,portal,cases,committee,docs,ops ENABLED_MODULES=tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,files,templates,mail,campaigns,calendar,poll,scheduling,connectors,datasources,dataflow,dist_lists,workflow_engine,workflow,tasks,views,quick_access,search,risk_compliance,postbox,notifications,services,parties,mandates,decisions,portal,cases,committee,docs,ops
CELERY_ENABLED=true CELERY_ENABLED=true
REDIS_URL=redis://127.0.0.1:6379/0 REDIS_URL=redis://127.0.0.1:6379/0
@@ -57,4 +57,4 @@ DEV_MAILBOX_API_ENABLED=false
GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/etc/govoplan/catalog-keyring.json GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/etc/govoplan/catalog-keyring.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL=stable GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable
@@ -4,12 +4,20 @@ on:
push: push:
tags: tags:
- "v*" - "v*"
workflow_dispatch:
inputs:
version:
description: Existing protected release version without leading v
required: true
type: string
jobs: jobs:
publish-package: publish-package:
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
GITEA_REPOSITORY: ${{ gitea.repository }} GITEA_REPOSITORY: ${{ gitea.repository }}
TRIGGER_TAG: ${{ gitea.ref_name }}
REQUESTED_VERSION: ${{ inputs.version }}
steps: steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with: with:
@@ -25,14 +33,134 @@ jobs:
import subprocess import subprocess
import tomllib import tomllib
tag = os.environ["GITEA_REF_NAME"] requested_version = os.environ.get("REQUESTED_VERSION", "").strip()
project = tomllib.loads(Path("packages/govoplan-meta/pyproject.toml").read_text(encoding="utf-8"))["project"] tag = f"v{requested_version}" if requested_version else os.environ["TRIGGER_TAG"]
if not tag.startswith("v") or not tag[1:]:
raise SystemExit("release tag is missing")
project_text = subprocess.check_output(
["git", "show", f"{tag}:packages/govoplan-meta/pyproject.toml"],
text=True,
)
project = tomllib.loads(project_text)["project"]
if tag != f"v{project['version']}": if tag != f"v{project['version']}":
raise SystemExit("meta-package version does not match the release tag") raise SystemExit("meta-package version does not match the release tag")
if subprocess.run(["git", "merge-base", "--is-ancestor", "HEAD", "origin/main"]).returncode: tag_commit = subprocess.check_output(
["git", "rev-parse", f"refs/tags/{tag}^{{commit}}"], text=True
).strip()
if subprocess.run(
["git", "merge-base", "--is-ancestor", tag_commit, "origin/main"]
).returncode:
raise SystemExit("release tag is not contained in main") raise SystemExit("release tag is not contained in main")
if not requested_version:
head_commit = subprocess.check_output(
["git", "rev-parse", "HEAD"], text=True
).strip()
if head_commit != tag_commit:
raise SystemExit("tag-triggered checkout does not match the release tag")
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"RELEASE_TAG={tag}\n")
subprocess.run(["git", "checkout", "--detach", tag_commit], check=True)
PY PY
- name: Build and publish developer package - name: Build developer package
run: |
set -euo pipefail
python -m pip install --disable-pip-version-check build==1.5.0 twine==7.0.0
python -m build --wheel --outdir dist packages/govoplan-meta
python -m twine check dist/*.whl
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import subprocess
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("meta release must contain exactly one wheel")
wheel = wheels[0]
evidence = {
"schema_version": "1",
"repository": os.environ["GITEA_REPOSITORY"],
"tag": os.environ["RELEASE_TAG"],
"commit": subprocess.check_output(
["git", "rev-parse", "HEAD"], text=True
).strip(),
"artifacts": [
{
"filename": wheel.name,
"sha256": hashlib.sha256(wheel.read_bytes()).hexdigest(),
"size": wheel.stat().st_size,
}
],
}
Path("dist/package-artifacts.json").write_text(
json.dumps(evidence, indent=2, sort_keys=True) + "\n", encoding="utf-8"
)
PY
- name: Retain package hash evidence
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: developer-meta-package
path: dist/package-artifacts.json
- name: Check immutable registry state
env:
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_TOKEN"
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import tomllib
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
project = tomllib.loads(
Path("packages/govoplan-meta/pyproject.toml").read_text(encoding="utf-8")
)["project"]
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("meta release must contain exactly one wheel")
wheel = wheels[0]
digest = hashlib.sha256(wheel.read_bytes()).hexdigest()
package_url = "/".join(
(
"https://git.add-ideas.de/api/v1/packages/GovOPlaN",
"pypi",
quote(str(project["name"]), safe=""),
quote(str(project["version"]), safe=""),
"files",
)
)
request = Request(
package_url,
headers={
"Accept": "application/json",
"Authorization": f"token {os.environ['PACKAGE_TOKEN']}",
},
)
publish = True
try:
with urlopen(request, timeout=30) as response:
files = json.load(response)
except HTTPError as exc:
if exc.code != 404:
raise
else:
if not isinstance(files, list) or len(files) != 1:
raise SystemExit("immutable meta-package has an unexpected file set")
if files[0].get("sha256") != digest:
raise SystemExit(
"immutable meta-package already exists with a different SHA-256"
)
publish = False
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"PUBLISH_PYPI={int(publish)}\n")
PY
- name: Publish developer package
env: env:
PACKAGE_USERNAME: ${{ secrets.GOVOPLAN_PACKAGE_USERNAME }} PACKAGE_USERNAME: ${{ secrets.GOVOPLAN_PACKAGE_USERNAME }}
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }} PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
@@ -40,10 +168,11 @@ jobs:
set -euo pipefail set -euo pipefail
test -n "$PACKAGE_USERNAME" test -n "$PACKAGE_USERNAME"
test -n "$PACKAGE_TOKEN" test -n "$PACKAGE_TOKEN"
python -m pip install --disable-pip-version-check build==1.5.0 twine==7.0.0 if [[ "$PUBLISH_PYPI" == 1 ]]; then
python -m build --wheel --outdir dist packages/govoplan-meta
python -m twine check dist/*.whl
TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \ TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \
python -m twine upload --non-interactive \ python -m twine upload --non-interactive \
--repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \ --repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \
dist/*.whl dist/*.whl
else
echo "Exact developer meta-package is already present; skipping immutable retry."
fi
+18 -3
View File
@@ -92,6 +92,17 @@ jobs:
if image_pattern.fullmatch(os.environ[name]) is None: if image_pattern.fullmatch(os.environ[name]) is None:
raise SystemExit(f"{name} must be an exact sha256 image reference") raise SystemExit(f"{name} must be an exact sha256 image reference")
PY PY
- name: Resolve immutable release source
working-directory: govoplan
env:
VERSION: ${{ inputs.version }}
run: |
git fetch --force --no-tags origin "refs/tags/v$VERSION:refs/tags/v$VERSION"
mkdir -p runtime-output
git rev-parse "v$VERSION^{commit}" > runtime-output/release-source-commit
grep -Eq '^[0-9a-f]{40}$' runtime-output/release-source-commit
git show "v$VERSION:requirements-release.txt" > runtime-output/requirements-release.source.txt
git show "v$VERSION:packages/govoplan-meta/pyproject.toml" > runtime-output/govoplan-meta.source.toml
- name: Use HTTPS for GovOPlaN repositories - name: Use HTTPS for GovOPlaN repositories
run: | run: |
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan" git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
@@ -110,6 +121,9 @@ jobs:
.runtime-build/bin/python -m pip install --upgrade pip cryptography .runtime-build/bin/python -m pip install --upgrade pip cryptography
.runtime-build/bin/python tools/release/generate-release-package-set.py \ .runtime-build/bin/python tools/release/generate-release-package-set.py \
--version "$VERSION" \ --version "$VERSION" \
--profile full \
--requirements runtime-output/requirements-release.source.txt \
--meta-package runtime-output/govoplan-meta.source.toml \
--output runtime-output/release-packages.json --output runtime-output/release-packages.json
.runtime-build/bin/python tools/release/resolve-package-artifacts.py \ .runtime-build/bin/python tools/release/resolve-package-artifacts.py \
--package-set runtime-output/release-packages.json \ --package-set runtime-output/release-packages.json \
@@ -118,9 +132,10 @@ jobs:
--lock-output runtime-output/package-artifacts.lock.json \ --lock-output runtime-output/package-artifacts.lock.json \
--requirements-output runtime-output/requirements-release.packages.txt \ --requirements-output runtime-output/requirements-release.packages.txt \
--python .runtime-build/bin/python --python .runtime-build/bin/python
PYTHON=.runtime-build/bin/python \ PYTHON="$PWD/.runtime-build/bin/python" \
GOVOPLAN_WEBUI_PACKAGE_LOCK="$PWD/runtime-output/package-artifacts.lock.json" \ GOVOPLAN_WEBUI_PACKAGE_LOCK="$PWD/runtime-output/package-artifacts.lock.json" \
GOVOPLAN_WEBUI_PACKAGE_DIR="$PWD/runtime-output/webui-packages" \ GOVOPLAN_WEBUI_PACKAGE_DIR="$PWD/runtime-output/webui-packages" \
GOVOPLAN_WEBUI_INSTALL_ALL_PACKAGES=true \
bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
npm --prefix ../govoplan-core/webui run build npm --prefix ../govoplan-core/webui run build
.runtime-build/bin/python tools/release/prepare-runtime-context.py \ .runtime-build/bin/python tools/release/prepare-runtime-context.py \
@@ -255,7 +270,6 @@ jobs:
working-directory: govoplan working-directory: govoplan
env: env:
VERSION: ${{ inputs.version }} VERSION: ${{ inputs.version }}
SOURCE_COMMIT: ${{ gitea.sha }}
SIGNING_KEY: ${{ secrets.RUNTIME_DISTRIBUTION_SIGNING_KEY }} SIGNING_KEY: ${{ secrets.RUNTIME_DISTRIBUTION_SIGNING_KEY }}
SIGNING_KEY_ID: ${{ secrets.RUNTIME_DISTRIBUTION_SIGNING_KEY_ID }} SIGNING_KEY_ID: ${{ secrets.RUNTIME_DISTRIBUTION_SIGNING_KEY_ID }}
TRUSTED_KEYRING: ${{ secrets.RUNTIME_DISTRIBUTION_KEYRING }} TRUSTED_KEYRING: ${{ secrets.RUNTIME_DISTRIBUTION_KEYRING }}
@@ -266,6 +280,7 @@ jobs:
GARAGE_IMAGE: ${{ inputs.garage_image }} GARAGE_IMAGE: ${{ inputs.garage_image }}
TEST_MAIL_IMAGE: ${{ inputs.test_mail_image }} TEST_MAIL_IMAGE: ${{ inputs.test_mail_image }}
run: | run: |
SOURCE_COMMIT="$(cat runtime-output/release-source-commit)"
test -n "$SIGNING_KEY" test -n "$SIGNING_KEY"
test -n "$SIGNING_KEY_ID" test -n "$SIGNING_KEY_ID"
test -n "$TRUSTED_KEYRING" test -n "$TRUSTED_KEYRING"
@@ -363,9 +378,9 @@ jobs:
working-directory: govoplan working-directory: govoplan
env: env:
VERSION: ${{ inputs.version }} VERSION: ${{ inputs.version }}
SOURCE_COMMIT: ${{ gitea.sha }}
GITEA_RELEASE_TOKEN: ${{ secrets.GOVOPLAN_RELEASE_TOKEN }} GITEA_RELEASE_TOKEN: ${{ secrets.GOVOPLAN_RELEASE_TOKEN }}
run: | run: |
SOURCE_COMMIT="$(cat runtime-output/release-source-commit)"
python tools/release/publish-runtime-release.py \ python tools/release/publish-runtime-release.py \
--tag "v$VERSION" \ --tag "v$VERSION" \
--target-commit "$SOURCE_COMMIT" \ --target-commit "$SOURCE_COMMIT" \
+1
View File
@@ -9,6 +9,7 @@ tools/release/runtime/*
!tools/release/runtime/Dockerfile.api !tools/release/runtime/Dockerfile.api
!tools/release/runtime/Dockerfile.web !tools/release/runtime/Dockerfile.web
!tools/release/runtime/nginx.conf !tools/release/runtime/nginx.conf
!tools/release/runtime/web-entrypoint.sh
__pycache__/ __pycache__/
build/ build/
dist/ dist/
+8 -1
View File
@@ -171,8 +171,15 @@ Create and validate a private, declarative installation bundle:
The current executable slice and remaining production gates are documented in The current executable slice and remaining production gates are documented in
[Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md). [Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md).
The canonical distinction between local source development, split source
integration, immutable single-host rehearsal, one-host production and
multi-host Kubernetes production is in
[Deployment Profiles](docs/DEPLOYMENT_PROFILES.md).
Same-host replica balancing and the multi-host promotion boundary are documented Same-host replica balancing and the multi-host promotion boundary are documented
in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md). in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
Create, update, pause, resume, verify and remove a local or multi-hypervisor K3s
VM target with the guarded lifecycle documented in
[Kubernetes VM Test Lab](docs/KUBERNETES_TEST_LAB.md).
The recovery state machine, migration rollback boundary, and required restore The recovery state machine, migration rollback boundary, and required restore
drills are documented in drills are documented in
[Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md). [Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md).
@@ -210,7 +217,7 @@ sequence are documented in the
The selected Campaign-to-Postbox-to-data-to-collaboration implementation path, The selected Campaign-to-Postbox-to-data-to-collaboration implementation path,
including stage gates and shared documentation expectations, is in the including stage gates and shared documentation expectations, is in the
[Reference Journey Program](docs/REFERENCE_JOURNEY_PROGRAM.md). [Reference Journey Program](docs/REFERENCE_JOURNEY_PROGRAM.md).
The administrator journey from Core-only bootstrap through online module The administrator journey from a Core-baseline bootstrap through online module
installation, scale-out, and reversible environment promotion is defined in installation, scale-out, and reversible environment promotion is defined in
[System Administrator Lifecycle User Story](docs/SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md). [System Administrator Lifecycle User Story](docs/SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
The corresponding host deployment compiler, managed/external component choices, The corresponding host deployment compiler, managed/external component choices,
+4
View File
@@ -2,6 +2,10 @@
This profile runs the shared services that production depends on while keeping This profile runs the shared services that production depends on while keeping
API, worker, scheduler, and WebUI code in the editable local repositories. API, worker, scheduler, and WebUI code in the editable local repositories.
It is the **split source integration** profile defined in
[`docs/DEPLOYMENT_PROFILES.md`](../../docs/DEPLOYMENT_PROFILES.md). It does not
exercise signed application images. Use an installer-generated evaluation
Compose bundle for an immutable Dockerized whole-product rehearsal.
It provides: It provides:
+167
View File
@@ -0,0 +1,167 @@
# Assisted and Non-Digital Channels
## Purpose
GovOPlaN must support people who cannot or do not use a self-service portal.
Telephone, paper, in-person service, authorized representation, mobile staff,
interpreters, and temporary offline work are not exceptional side systems.
They are governed channels into the same service, case, workflow, record, and
decision.
The goal is equivalent institutional treatment, not forced channel identity.
The system preserves which channel was used and which evidence is available
without giving digitally confident users stronger substantive rights.
The first end-to-end journey is tracked in
[GovOPlaN #42](https://git.add-ideas.de/GovOPlaN/govoplan/issues/42).
## Actor Model
Every assisted interaction distinguishes:
- the affected person or organization;
- the real staff member or external helper entering information;
- the represented party and representation basis;
- an interpreter, witness, guardian, or support person where relevant;
- the responsible institutional function;
- the channel and location;
- the person who reviewed or confirmed the captured information.
"Entered by" is not "declared by". "Declared by" is not "verified by".
Authentication assurance, representation authority, and evidence quality are
separate fields.
## Channel-Neutral Intake Contract
All channels create the same versioned service/form submission contract with
additional provenance:
- service, form, schema, language, and accessibility version;
- valid and recorded time;
- channel (`portal`, `counter`, `telephone`, `paper`, `email`, `mobile`,
`representative`, `offline_import`, or configured extension);
- affected and represented parties;
- capture actor and responsible function;
- consent, notice, purpose, legal basis, and information source;
- field-level source and confidence where staff transcribed or inferred data;
- attachments, scans, originals, signatures, recordings, and attestations as
governed evidence references;
- read-back/confirmation result and correction path;
- receipt and chosen return channels;
- duplicate/matching assessment and any manual resolution.
Forms Runtime owns the submission lifecycle. Parties owns procedural capacity
and representation. Identity/Addresses own subject and contact references.
Cases owns the matter. Records owns filing and retention. Audit preserves the
action/effect evidence.
## Assisted Session
An assisted session is a resumable work item, not a privileged bypass. It:
1. selects service, language, channel, affected party, and represented capacity;
2. shows the staff member only fields and evidence relevant to the service;
3. explains why sensitive data is requested and what evidence quality is
required;
4. records source per value when information comes from speech, paper, an
existing register, or staff observation;
5. validates and previews consequences before submission;
6. supports read-back, correction, confirmation, and a second-person check
where policy requires it;
7. generates an accessible receipt through the requested channel;
8. creates follow-up tasks when original documents, signatures, translation,
or verification remain outstanding.
The helper's normal account and represented function remain in the audit
chain. Assistance never grants access to unrelated records about the person.
## Paper And Scanning
- Register receipt before scanning so custody and deadlines do not depend on
successful OCR.
- Store the original scan or external archive reference with digest, pages,
capture device/provider, time, operator, and quality assessment.
- Treat OCR and extracted fields as derived data with confidence and source
coordinates. A person confirms consequential values.
- Support separation, ordering, missing-page, duplicate, malware, and
readability review.
- File the resulting document and submission into the appropriate eAkte;
retain or return the physical original according to policy.
- Produce cover sheets, barcodes, and return instructions through Templates,
not a separate print domain.
## Telephone And In-Person Handling
- Show a scripted but adaptable interview from the same Form definition.
- Record how identity and representation were checked; do not equate caller ID
with identity proof.
- Require explicit confirmation of consequential declarations and capture the
method (read-back, signed summary, one-time code, witness, later letter).
- Record call audio only when a lawful, declared profile permits it; an
interaction note is the default.
- Make interrupted sessions resumable without exposing prior answers to an
unauthorized caller or visitor.
## Offline And Mobile Work
Offline packages are encrypted, device-bound, time-limited, purpose-limited,
and contain only the required forms/reference data. Synchronization uses
idempotent intents and exposes conflicts rather than last-write-wins. Device
loss, expiry, revocation, duplicate submission, clock drift, and outcome
unknown have explicit recovery paths.
## Outbound Non-Digital Delivery
Campaign and Postbox model one delivery intent with channel choices and policy:
- portal/postbox delivery;
- email;
- print and postal fulfillment through a managed provider or local handoff;
- in-person collection;
- telephone notification followed by durable confirmation;
- accessible or language-specific variants.
Distribution preferences are purpose- and service-specific, effective-dated,
and may be overridden only by a documented legal or urgent-delivery rule. A
fallback occurs only before a channel has accepted the effect unless policy
explicitly authorizes duplicate delivery. Receipts distinguish creation,
provider acceptance, dispatch, delivery, return, and acknowledgement.
## Accessibility And Equality
- The person can request language, easy-language, large-print, screen-reader,
sign-language, relay, interpreter, or representative support without those
preferences becoming a general-purpose profile visible everywhere.
- Staff interfaces support keyboard-only capture, clear focus, error summary,
read-back, and printable/offline alternatives.
- Channel choice and need for assistance must not be used as an adverse risk
signal.
- Reports compare completion, wait, correction, abandonment, and outcome by
channel only under a declared equality/service-quality purpose and with
privacy thresholds.
## Security And Abuse Controls
- purpose-aware field access and session timeout;
- current authority checks for every read and effect;
- dual control for high-risk identity, payment, address, or representation
changes;
- immutable source/attestation evidence and correction history;
- rate and anomaly controls that do not silently reject a person;
- explicit safe handling of domestic-abuse, protected-address, witness, or
sealed-record cases;
- no secret answers or full documents in ordinary operational logs.
## First Reference Journey
Implement the permit-to-payment/service-to-decision journey through three
equivalent starts:
1. self-service portal submission;
2. staff-assisted counter/telephone submission;
3. paper receipt, scan, extraction, confirmation, and filing.
All three must create the same Case and Workflow contract, preserve different
provenance, support correction, produce a receipt, file an eAkte, reach the same
decision rules, and prove accessibility, privacy, recovery, and channel
fallback in browser and operator tests.
+8 -2
View File
@@ -1,5 +1,11 @@
# GovOPlaN Capability and IT-Infrastructure Fit Assessment # GovOPlaN Capability and IT-Infrastructure Fit Assessment
> **Pinned historical evidence:** This document assesses the exact 2026-07-22
> Campaign composition below. It is intentionally not updated to describe later
> main-branch work. Use [Strategy Status](STRATEGY_STATUS.md) for the current
> cross-product reconciliation and create a new dated fit assessment for a new
> target composition.
## Assessment record ## Assessment record
| Field | Value | | Field | Value |
@@ -19,8 +25,8 @@
Datasources, Dataflow, Search, encryption contracts, and other later main-branch Datasources, Dataflow, Search, encryption contracts, and other later main-branch
work must not be inferred into this evidence record. The current product work must not be inferred into this evidence record. The current product
direction and implemented-state reconciliation are documented separately in direction and implemented-state reconciliation are documented separately in
the the [Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md)
[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md). and [Strategy Status](STRATEGY_STATUS.md).
This is a fit assessment, not a production approval or security certification. This is a fit assessment, not a production approval or security certification.
It deliberately does not infer implementation from a repository, issue, or It deliberately does not infer implementation from a repository, issue, or
+14 -73
View File
@@ -18,7 +18,8 @@ Read it together with:
- the [institutional governance target architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md) - the [institutional governance target architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md)
- the [selected reference-journey program](REFERENCE_JOURNEY_PROGRAM.md) - the [selected reference-journey program](REFERENCE_JOURNEY_PROGRAM.md)
- the [current capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md) - the [current strategy status](STRATEGY_STATUS.md)
- the [pinned Campaign capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md)
- the [interface pattern language](INTERFACE_PATTERN_LANGUAGE.md) - the [interface pattern language](INTERFACE_PATTERN_LANGUAGE.md)
- the [interface surface inventory](INTERFACE_SURFACE_INVENTORY.md) - the [interface surface inventory](INTERFACE_SURFACE_INVENTORY.md)
- the [module contract and install model](MODULE_CONTRACTS_AND_INSTALLS.md) - the [module contract and install model](MODULE_CONTRACTS_AND_INSTALLS.md)
@@ -41,8 +42,8 @@ Read it together with:
- Use [Near-term portfolio order](#near-term-portfolio-order) for the bridge to - Use [Near-term portfolio order](#near-term-portfolio-order) for the bridge to
implementation and [Product decisions](#product-decisions-to-make-progressively) implementation and [Product decisions](#product-decisions-to-make-progressively)
for choices that can remain deferred. for choices that can remain deferred.
- Use the [dated snapshot appendix](#snapshot-appendix-2026-07-20) only to - Use the [dated strategic review](STRATEGIC_REVIEW_2026-08-05.md) to understand
understand which live backlog and release facts informed this revision. why the current convergence and reference-journey order was chosen.
### Planning ownership ### Planning ownership
@@ -50,7 +51,7 @@ Read it together with:
| --- | --- | | --- | --- |
| What product should GovOPlaN become, for whom, in which configurations, and through which outcome horizons? | This meta roadmap | | What product should GovOPlaN become, for whom, in which configurations, and through which outcome horizons? | This meta roadmap |
| Which module owns a capability, which technical wave should deliver it, and what implementation gates apply? | The Core master roadmap and owning-module concepts | | Which module owns a capability, which technical wave should deliver it, and what implementation gates apply? | The Core master roadmap and owning-module concepts |
| What is actively planned, blocked, implemented, or closed now? | Gitea issues | | What is actively planned, blocked, implemented, or closed now? | Gitea issues and the dated reconciliation in `STRATEGY_STATUS.md` |
| What can a named composition credibly claim in a target environment? | A dated capability/infrastructure fit assessment | | What can a named composition credibly claim in a target environment? | A dated capability/infrastructure fit assessment |
The horizons and near-term order below express product outcomes and portfolio The horizons and near-term order below express product outcomes and portfolio
@@ -266,7 +267,7 @@ inspection is authorized.
The complete installation and lifecycle journey is specified in the The complete installation and lifecycle journey is specified in the
[System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md): [System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md):
one-command Core-only bootstrap, signed online module installation and updates, one-command Core-baseline bootstrap, signed online module installation and updates,
stateless scale-out, versioned configuration transfer, undo, and reproducible stateless scale-out, versioned configuration transfer, undo, and reproducible
environment-promotion recipes. environment-promotion recipes.
@@ -976,7 +977,8 @@ Priorities:
1. Deliver the first slices of the 1. Deliver the first slices of the
[System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md): [System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md):
a verified Core-only distribution, first-run control plane, read-only online a verified full-package distribution with only the Core baseline active,
first-run control plane, read-only online
module directory, and durable plan/confirm/install progress. module directory, and durable plan/confirm/install progress.
2. Pin and publish a compatible Core/WebUI/module composition and first 2. Pin and publish a compatible Core/WebUI/module composition and first
reference configuration package. reference configuration package.
@@ -1351,71 +1353,10 @@ language, what service it configured, who can act, which systems participate,
what happens when they fail, how a decision can be reviewed, and where the what happens when they fail, how a decision can be reviewed, and where the
evidence remains—and the product can prove that explanation at runtime. evidence remains—and the product can prove that explanation at runtime.
## Snapshot appendix: 2026-07-20 ## Dated Context
This appendix records volatile facts that informed this revision. It is not a The volatile release and backlog appendix that originally accompanied this
second source of truth and should be refreshed or removed when a later roadmap roadmap has been removed so the durable direction cannot become a competing
review uses a new release/backlog snapshot. status source. The [Strategic Review 2026-08-05](STRATEGIC_REVIEW_2026-08-05.md)
retains the dated assessment and reasoning. Current reconciliation belongs in
### Composition and release snapshot [Strategy Status](STRATEGY_STATUS.md), and live work state belongs in Gitea.
The cross-repository contract scan found 43 module manifest contracts, 29
provided interface names, 16 requirements, and no contract error across 65
scanned repositories. That is meaningful composition evidence, but the release
metadata trailed the integrated code: Core, Policy, Poll, and Scheduling
declared `0.1.9` while the whole-product release requirements remained on
module tag `v0.1.8`; the root self-hosted `.env.example` and release smoke
composition did not yet exercise all installed release modules. Other
development compositions already included some of those modules. This was a
release/composition gap, not evidence that the underlying slices did not exist.
### Backlog snapshot
The Gitea audit found 206 open issues across 36 of 66 catalogued repositories
and 362 closed issues. Campaign had 51 open issues and Core 44; together they
held 46% of current work. This reflected substantial completed kernel,
security, and platform work and a deliberate concentration on the first usable
vertical, but also risked crowding out production evidence and the shared
process spine.
The issue workflow needed a reconciliation pass before another delivery
program could be inferred from labels: 119 open issues remained in triage, 116
had no milestone, and several recently pushed Calendar, Scheduling, Poll,
Campaign, and Files slices still described themselves as local or awaiting
integration. Conversely, 30 repositories had no open issue; for many
later-wave modules this meant no implementation program had been opened, not
that the capability was complete.
[Poll #2](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/2) was a clear
tracker-drift example: its configurable transition engine, agreed transition
matrix/history, idempotent keyed retries, re-decision audit, archive/unarchive,
and preservation behavior were implemented and pushed while the issue still
reported `needs-info`.
Issue anchors that informed the bridge from the baseline into this roadmap:
- [Meta #10](https://git.add-ideas.de/GovOPlaN/govoplan/issues/10) for the
capability/infrastructure assessment and its target proof;
- [Meta #11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) for the
universal interface and focused-view direction;
- [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225) for
guided, safe configuration;
- [Core #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29) for the
backup/restore production gate;
- [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263) and
[Campaign #63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63),
[#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62),
[#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65), and
[#69](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/69) for the
reference interface/delivery vocabulary and behavior;
- [Poll #1](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/1) for the
database-enforced respondent invariant exposed by Scheduling;
- [Connectors #6](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/6)
for the governed connector configuration/simulation foundation;
- [Meta #9](https://git.add-ideas.de/GovOPlaN/govoplan/issues/9) for the first
permit-to-payment reference process; and
- [Meta #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12) for the
deliberately deferred, consumer-independent export-control story.
Live Gitea issue state remains canonical. These dated facts explain the roadmap
sequence only.
+120
View File
@@ -0,0 +1,120 @@
# GovOPlaN Deployment Profiles
## Purpose
GovOPlaN distinguishes how code is executed, where it is placed, and how mature
the target is. These are separate concerns:
- **execution basis:** editable source trees or an immutable signed release;
- **topology:** local processes, one-host containers, or a multi-host
orchestrator;
- **component ownership:** installer-managed or externally supplied state and
infrastructure services; and
- **assurance state:** development, rehearsal/acceptance, or approved
production.
PostgreSQL, Redis, object storage, mail and ingress choices are component
bindings inside a profile. They do not create a new application topology by
themselves.
## Canonical Profiles
| Profile | Entry point | Application execution | State services | Intended use | Explicit boundary |
| --- | --- | --- | --- | --- | --- |
| Local source development | `tools/launch/launch-dev.sh` | Editable Uvicorn/Vite processes with reload | Local development bindings, optionally the shared PostgreSQL helper | Fast module and UI work | No production packaging, isolation, availability or capacity claim |
| Split source integration | `tools/launch/launch-production-like-dev.sh` | Editable API, WebUI, worker and scheduler processes | Containerized PostgreSQL/Redis by default; environment bindings may point at developer-owned services | Queue, migration, Redis and split-role integration while retaining source reload | “Production-like” describes behavior, not immutable artifacts or a production security boundary |
| Immutable single-host rehearsal | `govoplan-deploy init/apply --profile evaluation` | Signed API/WebUI images and generated Compose roles | Bounded managed components or explicit external bindings | Test the downloadable artifacts, installer, migrations, load balancer and component choices | All containers and managed services may share one host and failure domain; evaluation conveniences are not production controls |
| Single-host production | `govoplan-deploy init/apply --profile self-hosted` | Signed API/WebUI images behind generated HAProxy and selected TLS ingress | Durable local/single-node managed services where accepted, or external services | Small and medium installations whose accepted availability boundary is one host | Multiple containers add capacity and rolling-process resilience, but do not survive host loss |
| Multi-host Kubernetes production | `govoplan-deploy render-kubernetes` or the guarded K3s lab/acceptance workflow | Immutable API, WebUI and queue-specific worker Deployments across failure domains | External PostgreSQL, Redis and S3-compatible storage; external secret and ingress control | Institution-scale availability and horizontal application-tier capacity | Production claims require independent nodes, HA state services, load/capacity evidence and signed recovery evidence |
Docker Compose services are containers or replicas, not Kubernetes pods. The
immutable single-host rehearsal is the appropriate Dockerized whole-product
test when source reload is not required.
The K3s VM lab has two modes over the same Kubernetes profile:
- `rehearsal` may place VMs on one physical hypervisor and proves bounded
orchestration behavior;
- `acceptance` requires independently controlled worker failure domains and can
contribute target evidence.
## Module composition and availability
Official immutable API and WebUI images carry the verified `full` package
profile. This is package availability, not runtime activation and not a license
or tenant entitlement. The signed distribution manifest records the complete
package composition; the desired module graph selects which installed modules
are active; tenant module policy applies unavailable/available/forced ceilings;
and Views/Policy control group and user presentation.
Local and single-host profiles may use the supervised installer to download a
signed catalog artifact into a private digest cache and mutate the local package
environment during maintenance. A multi-host/shared-state profile must never
change one replica in place. Its Admin install plan is a composition request:
publish and roll out a new signed image whose package lock contains the target,
then activate the module graph after all replicas report the same composition.
## Component Choices
The installer may manage a component where its bounded profile is appropriate,
or consume an operator-provided service:
| Component | Managed boundary | External/BYO boundary |
| --- | --- | --- |
| PostgreSQL | Single-host Compose database | Stable primary-aware endpoint supplied by a PostgreSQL provider/operator |
| Redis | Single-host persistent Redis | Tested HA Redis endpoint compatible with queues, throttling and coordination |
| File/object storage | Durable local storage or single-node Garage | Shared, redundant S3-compatible storage |
| Mail | Development GreenMail only | Institution/provider SMTP and IMAP services |
| Ingress/TLS | Generated Caddy on one host | Existing reverse proxy or Kubernetes ingress and secret management |
Switching to an external component changes ownership and evidence requirements;
it does not remove GovOPlaN's health, capacity, backup and recovery checks.
## Scaling Responsibilities
GovOPlaN scales application roles, while the infrastructure control plane owns
machines and state-service replication:
| Concern | Scaling model | Owner |
| --- | --- | --- |
| API and WebUI | Increase replicas behind health-aware Services/Ingress | GovOPlaN deployment desired state, reconciled by Compose or Kubernetes |
| Background work | Add queue-specific worker replicas and bounded concurrency | GovOPlaN deployment desired state and worker-pool configuration |
| Scheduler, migrations and module lifecycle | Singleton execution protected by database leases/fencing | GovOPlaN; these roles are never scaled as unfenced active-active workers |
| Kubernetes worker/control nodes | Add, drain, replace and upgrade machines; optionally use a cluster autoscaler | Kubernetes/platform operator, not the GovOPlaN application |
| PostgreSQL | Replication, failover, backups, connection pooling and stable writer endpoint | Database operator/provider; GovOPlaN currently consumes the stable endpoint and does not route arbitrary reads to replicas |
| Redis | Replication/failover, persistence, eviction and TLS/authentication | Redis operator/provider |
| S3-compatible storage | Placement, replication, repair and capacity | Storage operator/provider |
Administrators should eventually be able to review and change permitted
application replica and worker-pool desired state through the Ops surface.
Creating physical machines, database replicas or storage members remains an
orchestrator/provider action. GovOPlaN must observe their health and block unsafe
changes rather than becoming a second infrastructure scheduler.
Every scale change must recalculate the database connection budget, preserve
queue coverage, verify software/module-composition consistency and respect
drain and fencing state.
## What Has Been Proven
The current implementation and the signed `v0.1.18` rehearsal prove that the
application tier can run as stateless API, WebUI and worker replicas against
logically shared state. Two Kubernetes worker VMs hosted API and WebUI replicas,
and an API pod was replaced without an observed public-readiness failure.
This is not yet proof of general “large organization fit.” That claim also
requires:
- representative concurrent-user, dataset, report and background-job load
tests with latency and saturation budgets;
- independent physical failure domains and real ingress/network behavior;
- HA PostgreSQL, Redis and object storage with failover drills;
- session, accepted-job and provider-effect continuity under node and service
loss;
- coordinated backup/isolated restore, measured RTO/RPO and semantic recovery;
- observability, alerting, capacity forecasting and sustained soak evidence.
The profile therefore proves the architecture is horizontally deployable. A
specific institution is production-fit only after its target topology and load
envelope have produced the governed evidence described in
`TARGET_MATURITY_EVIDENCE_RUNBOOK.md`.
+155
View File
@@ -0,0 +1,155 @@
# Federated GovOPlaN Architecture
## Purpose
Federation lets autonomous GovOPlaN installations exchange data,
configuration, work, messages, records, and evidence without sharing a database
or surrendering local policy. It is institution-to-institution cooperation,
not multi-tenancy across an untrusted network.
The first implementation should prove a bounded exchange between two
installations. A new federation module is not justified until the shared
protocol has at least two independent consumers. Core owns neutral envelopes
and trust contracts; Connectors owns transport providers; domain modules own
the objects and effects they exchange.
Implementation is tracked in
[GovOPlaN #41](https://git.add-ideas.de/GovOPlaN/govoplan/issues/41).
## Invariants
1. Every installation remains authoritative for its tenants, identities,
policies, keys, records, and local mappings.
2. A remote identity or permission never becomes a local authorization claim.
3. Every exchange declares purpose, legal/organizational basis, classification,
minimization, retention expectation, and permitted onward use.
4. Every object reference identifies origin instance, owner tenant, object type,
object ID, exact revision, and source-authority mode.
5. Payloads and receipts are signed; sensitive transports use mutually
authenticated encrypted channels.
6. Acceptance, rejection, outcome unknown, retry, revocation, correction, and
reconciliation are durable states.
7. Local policy may reject or narrow a remote request. It cannot silently claim
to have accepted an effect that did not occur.
8. Federation works asynchronously and can exchange signed offline bundles
where continuous connectivity is unavailable.
## Trust Domains
An instance publishes a signed, versioned federation descriptor containing:
- stable instance and operator identity;
- supported protocol and schema versions;
- signing and transport key identifiers with rotation history;
- accepted object and exchange profiles;
- endpoint locations and size/rate limits;
- support, incident, revocation, and data-protection contacts;
- evidence and conformance references.
Pairing is a two-sided administrative workflow. Each side verifies the other,
maps the remote institution to a local trusted-party record, selects permitted
profiles and purposes, sets policy ceilings, and records approvals. Trust is
directional and profile-specific; trusting signed Postbox delivery does not
automatically permit case transfer or configuration import.
## Exchange Envelope
Every request, response, receipt, correction, and revocation uses one neutral
envelope with:
- message ID, correlation ID, causation ID, creation and expiry;
- origin and destination instance/institution/tenant references;
- real actor and represented institutional capacity where disclosure is
permitted;
- exchange profile and semantic schema version;
- exact domain object references and content digests;
- purpose, legal basis, classification, data categories, retention expectation,
onward-transfer constraint, and subject notice status;
- requested action and idempotency key;
- encryption recipients and signature chain;
- attachment/object manifests rather than unbounded embedded blobs;
- previous-envelope references for correction, replacement, or revocation.
The envelope is evidence, not a universal domain object. Each owner validates
and imports or links its own payload.
## Exchange Profiles
| Profile | First owners | Behavior |
| --- | --- | --- |
| Postbox delivery | Postbox, Campaign, Notifications | Address or derive a remote function-bound postbox, obtain acceptance receipt, and track acknowledgement where permitted |
| Case handoff | Cases, Parties, Services, Workflow Engine | Offer exact context and evidence; destination accepts into a new local case and returns the mapping |
| Record transfer | Records, Files, DMS, Audit | Transfer or offer a signed record package with file-plan, metadata, content digests, holds, and disposition constraints |
| Decision/evidence reference | Decisions, Committee, Audit | Publish a protected exact outcome or verifiable reference without transferring unrelated case content |
| Data product publication | Datasources, Dataflow, Reporting | Publish immutable governed materializations with schema, quality, freshness, lineage, and use constraints |
| Configuration package | Core, Policy, Views, Workflow, Forms, Templates | Exchange signed definitions; destination assesses compatibility, maps values, derives locally, and never imports secrets |
| Search discovery | Search and domain providers | Return permission-filtered metadata or a handoff link; never expose raw remote indexes as local authority |
## State Machine
```text
draft -> authorized -> queued -> transmitted -> received
| |
v v
outcome_unknown rejected
|
received -> validating -> accepted -> applied -> acknowledged
| | |
v v v
rejected accepted_ reconciled
pending
```
Acceptance means the destination durably owns the received intent. It does not
mean the requested domain effect completed. Receipts distinguish transport,
validation, acceptance, application, and human acknowledgement.
## Conflict And Autonomy
- Incoming native objects become local references, mirrors, or newly owned
objects according to the profile. They do not overwrite local authority by
ID coincidence.
- Local mappings are effective-dated and auditable.
- Corrections create a linked revision. They do not erase what the destination
previously observed.
- Revocation is a request and evidence event; the destination applies its own
legal and retention rules.
- Configuration imports use assessment and derivation. A remote package cannot
weaken local policy or install code implicitly.
- A disconnected partner remains a visible pending/failed state; work can be
rerouted through an approved alternative channel.
## Security And Privacy
- Use mTLS for paired online transports and signed envelopes for end-to-end
origin evidence.
- Encrypt payload objects for the destination, with key rotation and outcome-
unknown recovery; transport encryption alone is insufficient for queued
bundles.
- Do not put bearer credentials, local permission scopes, or reusable secrets
in an exchange.
- Rate-limit and size-bound discovery and transfer; quarantine unknown schemas
and active content.
- Evaluate current local authorization at every effect even when the envelope
describes historical authority.
- Log metadata separately from protected content so operators can reconcile
without broad content access.
- Subject access, correction, restriction, legal hold, and deletion requests
become federated workflows with local decisions and receipts, not remote
direct database operations.
## First Reference Proof
1. Pair two disposable installations with independent tenants, keys, and
policies.
2. Exchange signed descriptors and approve only the Postbox delivery profile.
3. Deliver one Campaign message to a remote function-bound Postbox.
4. Prove replay safety, rejection, timeout/outcome unknown, retry,
acknowledgement, correction, key rotation, and revoked trust.
5. Export the complete evidence bundle and restore both sides from backup.
6. Add configuration-package exchange only after the delivery proof passes.
The result is a provider-neutral federation contract. A future dedicated
module becomes appropriate only when pairing, trust administration, exchange
queues, and evidence have a lifecycle independent of Connectors and the first
domain owner.
@@ -10,6 +10,10 @@ reconfigures that installation instead of creating unrelated state.
The canonical product journey remains The canonical product journey remains
[System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md). [System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
This document defines the deployer boundary and the first executable slice. This document defines the deployer boundary and the first executable slice.
The execution, topology, component-ownership and assurance modes are defined
canonically in [Deployment Profiles](DEPLOYMENT_PROFILES.md). In particular,
the editable production-like developer launcher is distinct from both an
immutable Compose rehearsal and a supported one-host production deployment.
## First Executable Slice ## First Executable Slice
@@ -95,6 +99,7 @@ The private installation directory contains:
| `existing-proxy.json` | Exact upstream, trusted-source, header, and health contract for an operator-owned proxy | | `existing-proxy.json` | Exact upstream, trusted-source, header, and health contract for an operator-owned proxy |
| `plan.json` | Latest desired-state diff and readiness findings | | `plan.json` | Latest desired-state diff and readiness findings |
| `receipt.json` | Last successfully applied immutable identities | | `receipt.json` | Last successfully applied immutable identities |
| `infrastructure-capabilities.json` | Deterministic non-secret capability states, endpoint metadata, secret references, consumers, and resumable post-install tasks |
| `distribution-manifest.json` | Canonical signed runtime/image selection adopted by the installer | | `distribution-manifest.json` | Canonical signed runtime/image selection adopted by the installer |
| `distribution-keyring.json` | Explicitly installed public trust anchor for runtime releases | | `distribution-keyring.json` | Explicitly installed public trust anchor for runtime releases |
| `backup-evidence.json` | Signed provider-neutral coordinated backup and isolated-restore receipt | | `backup-evidence.json` | Signed provider-neutral coordinated backup and isolated-restore receipt |
@@ -404,16 +409,24 @@ the supported topology and promotion path.
## Reconfiguration Semantics ## Reconfiguration Semantics
`installation.json` is desired state. `receipt.json` is the last successfully `installation.json` is desired state. `receipt.json` is the last successfully
applied state. `plan` compares their canonical hashes and service sets. applied state. `plan` compares their canonical hashes, service sets, and
infrastructure capability projections.
- Adding a managed component creates its service and persistent volume. - Adding a managed component creates its service and persistent volume.
- Removing a component removes its service container on apply. - Removing a component removes its service container on apply.
- Replacing or removing a capability adds a review action that names the prior
and desired state/source plus declared module consumers. This does not claim
that the deployer can inspect module-owned database configuration; the
operator must review that inventory before apply.
- Volumes are retained by default; deleting data requires a separate, - Volumes are retained by default; deleting data requires a separate,
deliberately destructive workflow. deliberately destructive workflow.
- Existing generated credentials are retained unless an explicit future rotate - Existing generated credentials are retained unless an explicit future rotate
operation is requested. operation is requested.
- Private configuration changes are represented by a keyed fingerprint in the - Private configuration changes are represented by a keyed fingerprint in the
plan and receipt; plaintext values are never copied there. plan and receipt; plaintext values are never copied there.
- Capability documents contain sanitized scheme/host/port metadata and stable
`env:` references only. Credential values and secret-bearing URLs remain in
`secrets.env` or module-owned credential envelopes.
- Managed-to-external transitions require the new endpoint in the same - Managed-to-external transitions require the new endpoint in the same
operation. operation.
- Migrations run as a one-shot service before API/worker replacement. - Migrations run as a one-shot service before API/worker replacement.
@@ -426,6 +439,12 @@ applied state. `plan` compares their canonical hashes and service sets.
- Health must recover before a new receipt and applied-state snapshot are - Health must recover before a new receipt and applied-state snapshot are
committed. committed.
Compose mounts the capability document read-only into API and worker runtime
containers. The Kubernetes export projects the same document through a
dedicated ConfigMap and read-only file mount. Ops validates the bounded schema
before displaying configured, externally supplied, available-unconfigured, or
unavailable states and any pending post-install tasks.
Every apply operation is journalled before image pulls or runtime mutation. A Every apply operation is journalled before image pulls or runtime mutation. A
failure before migration may restore a verified previous bundle. Once migration failure before migration may restore a verified previous bundle. Once migration
starts, recovery is forward-only unless an independently verified database starts, recovery is forward-only unless an independently verified database
+149
View File
@@ -0,0 +1,149 @@
# Institutional Digital Twin
## Definition
The institutional digital twin is a governed, time-aware projection of how an
institution is constituted and operates. It connects structure, authority,
services, work, information, technology, obligations, controls, evidence, and
outcomes without becoming a second source of truth.
The twin is not one editable graph database and not an employee-surveillance
system. Domain modules and external systems keep ownership. The twin stores or
materializes exact references, declared relationships, provenance, confidence,
and projection versions. Changes flow through owner actions.
Implementation is tracked in
[GovOPlaN #43](https://git.add-ideas.de/GovOPlaN/govoplan/issues/43).
## Questions It Should Answer
- Which unit and function is responsible for a service, decision, record,
system, dataset, control, or risk at a given valid and recorded time?
- Which mandates and policies permit or constrain an action?
- Which processes, providers, staff capacities, data sources, and records are
required to deliver a service?
- What is affected if a system, provider, organizational unit, role, package,
or legal rule changes?
- Where are responsibilities missing, conflicting, expired, or concentrated?
- Which controls are evidenced, stale, failed, or dependent on an unverified
assertion?
- How do actual process traces differ from defined workflows?
- Which public outcomes can be explained from protected internal evidence?
## Projection Planes
| Plane | Meaning |
| --- | --- |
| Current | Valid now, reconstructed from owner projections and current provider state |
| Historical | Valid at and recorded by selected instants, with present-day security enforced |
| Planned | Approved or proposed future structures, services, policies, projects, and package changes |
| Observed | Events, process traces, service measures, incidents, effects, and evidence actually recorded |
| Scenario | Non-authoritative simulation of a proposed change and its estimated consequences |
The UI must label these planes unambiguously. Scenario output never becomes an
institutional fact until an authorized owner action accepts it.
## Canonical Graph
Nodes are stable institutional references, including institution, tenant,
unit, function, assignment, mandate, jurisdiction, service, case, party, task,
workflow, approval, decision, record, file, message, appointment, dataset,
report, provider, system, control, risk, project, asset, and configuration
package.
Edges have:
- owner and source authority;
- relationship type and direction;
- valid-from/valid-to and recorded/superseded times;
- exact source revision and evidence digest;
- institution/tenant boundary;
- purpose and visibility classification;
- confidence and derivation method for inferred relationships;
- correction and replacement references.
Inferred edges are never displayed as owner assertions. They remain
explainable analytical products with source lineage.
## Ownership And Implementation
- Core owns neutral institutional references, temporal context, provider
registration, and graph projection contracts.
- Domain modules publish bounded nodes and edges through provider interfaces.
- Search indexes discoverable identities and links.
- Reporting materializes governed analytical projections.
- Dataflow computes derived relationships, quality checks, and scenarios.
- Policy evaluates visibility, purpose, retention, and allowed scenario/action
transitions.
- Audit supplies observed events and evidence references.
- Projects supplies planned change and benefit relationships.
- Views renders role- and task-focused twin perspectives.
- Workflow Engine coordinates accepted changes but does not edit owner tables.
No new digital-twin module is required for the first slice. A dedicated owner
is justified later if persisted scenario models, graph revisions, and
cross-domain projection lifecycle become independent product objects.
## Beyond The Current Platform
### Continuous assurance
Controls become versioned assertions with evidence requirements, evaluation
frequency, responsible function, exception workflow, and freshness. Dataflow
and provider checks evaluate them continuously; Policy decides whether a stale
or failed control advises, requires review, or blocks an effect.
### Process mining and conformance
Governed event histories can derive actual paths, wait times, rework, and
exceptions. Comparison to Workflow definitions should improve procedures, not
rank individuals. Access to personal or small-cohort detail is purpose-limited
and separately governed.
### Change-impact simulation
A proposed organizational, provider, policy, or package change can be assessed
against dependencies, mandates, open work, records, controls, capacity, and
recovery plans before activation. Results identify uncertainty rather than
inventing precision.
### Federated institutional models
Installations can exchange signed public or partner-specific subsets of their
service, mandate, provider, and evidence graph. Every side maps the references
locally and retains autonomy. Federation does not create one supranational
master graph.
### Accountable assistance
Assistance may summarize context, identify missing evidence, draft a decision
or workflow, propose mappings, and explain policy. Every output records model,
inputs, constraints, uncertainty, human review, and accepted edits. Assistance
does not become the acting authority.
### Public evidence chains
Transparency packages can publish a minimized chain from rule and aggregate
facts to decision and observed outcome, with digests proving relation to
protected evidence. Public verification does not require disclosure of the
underlying personal data.
## Guardrails
- Do not infer competence, misconduct, intent, or personal performance from
graph proximity or incomplete events.
- Do not centralize protected content merely to make graph queries easier.
- Do not use historical authorization to expose data now prohibited.
- Do not let a scenario engine write domain state directly.
- Do not hide source authority, freshness, uncertainty, or missing evidence.
- Do not retain analytical detail longer than the declared purpose requires.
## Delivery Slices
1. Publish exact institutional reference/edge providers for the service-to-
decision and monthly-data journeys.
2. Build a current/historical dependency explorer with source and access
explanations.
3. Add planned Project/package changes and bounded impact reports.
4. Add control evidence/freshness and process conformance for one journey.
5. Prove a minimized federated projection and a public evidence package.
@@ -9,13 +9,17 @@ concepts prepared outside the repositories:
- `software_big_picture.md` - `software_big_picture.md`
The source concepts describe GovOPlaN as an operational governance platform for The source concepts describe GovOPlaN as an operational governance platform for
public institutions. This document merges that direction with the implemented public institutions. This document is the canonical repository version of that
platform state as of 2026-08-01. It is the canonical repository version of the durable architectural direction. Its implementation table records the accepted
direction. Gitea issues remain the source of truth for delivery state. 2026-08-01 baseline; it is not a rolling status report. Current reconciliation
lives in [Strategy Status](STRATEGY_STATUS.md), and Gitea issues remain the
source of truth for delivery state.
Read this together with: Read this together with:
- [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) - [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md)
- [Platform Core Ideas](PLATFORM_CORE_IDEAS.md)
- [Strategy Status](STRATEGY_STATUS.md)
- [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md) - [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md)
- [Module Contracts and Install Boundaries](MODULE_CONTRACTS_AND_INSTALLS.md) - [Module Contracts and Install Boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md) - [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md)
@@ -70,7 +74,12 @@ compositions described here are now implemented. Subsequent work is
**product depth and stronger maturity evidence**, not another runtime rewrite **product depth and stronger maturity evidence**, not another runtime rewrite
or an unimplemented architecture boundary. or an unimplemented architecture boundary.
## Implementation status (2026-08-01) ## Accepted implementation baseline (2026-08-01)
This section is retained as the dated baseline against which the architecture
decision was accepted. Later implementation must be reconciled in
`STRATEGY_STATUS.md` rather than editing individual rows here into a competing
status report.
The architecture contract is implemented as a bounded, executable vertical The architecture contract is implemented as a bounded, executable vertical
slice. The portfolio declarations and provider governance gates apply to the slice. The portfolio declarations and provider governance gates apply to the
@@ -79,7 +88,7 @@ were proven now have independent persistent owners:
| Area | Implemented state | Remaining rollout | | Area | Implemented state | Remaining rollout |
| --- | --- | --- | | --- | --- | --- |
| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for all 62 source manifests. Focused and release checks enforce `--require-architecture`; a new module cannot enter the workspace without truthful declaration and repository-local evidence. | | Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for the source manifests in the 2026-08-01 snapshot. Focused and release checks enforce `--require-architecture`; current portfolio counts belong in `STRATEGY_STATUS.md`. |
| External providers | Core validates provider objects/field groups, operations, integration maturity, source authority, bounded reads, freshness/health, idempotency, conflicts, outcome-unknown handling, evidence, correction, reconciliation, outage, classification, purpose, retention, and secret handling. Addresses/CardDAV, Files remote storage, Mail SMTP/IMAP, Calendar CalDAV/ICS/Graph/EWS, and Connectors tabular/sanctions providers declare the contract and tenant-bounded secret-free runtime state. | Registry validation rejects any declared external provider without a sanitized state provider. Future adapters must cross the same gate before activation. | | External providers | Core validates provider objects/field groups, operations, integration maturity, source authority, bounded reads, freshness/health, idempotency, conflicts, outcome-unknown handling, evidence, correction, reconciliation, outage, classification, purpose, retention, and secret handling. Addresses/CardDAV, Files remote storage, Mail SMTP/IMAP, Calendar CalDAV/ICS/Graph/EWS, and Connectors tabular/sanctions providers declare the contract and tenant-bounded secret-free runtime state. | Registry validation rejects any declared external provider without a sanitized state provider. Future adapters must cross the same gate before activation. |
| Institutional context | Core provides versioned temporal, actor/representation, institution/unit/function/task/mandate/jurisdiction/service/case/party/work-item/workflow/approval/decision/record, legal-basis, evidence, information-governance, external-source, presentation, and geographic references. Events, automation actions, audit records, and the transactional Audit outbox preserve the envelope. | Owning modules must progressively require the relevant subset for consequential operations. | | Institutional context | Core provides versioned temporal, actor/representation, institution/unit/function/task/mandate/jurisdiction/service/case/party/work-item/workflow/approval/decision/record, legal-basis, evidence, information-governance, external-source, presentation, and geographic references. Events, automation actions, audit records, and the transactional Audit outbox preserve the envelope. | Owning modules must progressively require the relevant subset for consequential operations. |
| Semantic provider contracts | Provider-neutral DTOs and protocols cover Mandate resolution, versioned Service definitions, procedure Parties/representation, and formal Decisions. `govoplan-mandates`, `govoplan-services`, `govoplan-parties`, and `govoplan-decisions` now persist immutable revisions behind those contracts with tenant isolation, bounded reads, replay safety, OCC, migrations, uninstall guards, permissions, APIs, capability documentation, and recovery documentation. | The owners are deliberately headless. Procedure-specific UI remains with consuming modules. | | Semantic provider contracts | Provider-neutral DTOs and protocols cover Mandate resolution, versioned Service definitions, procedure Parties/representation, and formal Decisions. `govoplan-mandates`, `govoplan-services`, `govoplan-parties`, and `govoplan-decisions` now persist immutable revisions behind those contracts with tenant isolation, bounded reads, replay safety, OCC, migrations, uninstall guards, permissions, APIs, capability documentation, and recovery documentation. | The owners are deliberately headless. Procedure-specific UI remains with consuming modules. |
@@ -391,7 +400,9 @@ submodule, configuration fragment, package, or profile.
- This reconciliation is canonical in the meta repository and mirrored to the - This reconciliation is canonical in the meta repository and mirrored to the
Gitea wiki. Gitea wiki.
- All 62 source manifests carry validated evidence-based architecture metadata. - All source manifests in the accepted 2026-08-01 baseline carried validated
evidence-based architecture metadata; current counts belong in
`STRATEGY_STATUS.md`.
- External-reference, action/effect, operational-health, ownership, policy, - External-reference, action/effect, operational-health, ownership, policy,
audit, and documentation primitives compose into one enforced provider audit, and documentation primitives compose into one enforced provider
declaration and sanitized runtime-state contract. declaration and sanitized runtime-state contract.
@@ -446,7 +457,7 @@ submodule, configuration fragment, package, or profile.
recovery, accessibility, privacy, security, and operator evidence. This is a recovery, accessibility, privacy, security, and operator evidence. This is a
maturity gate, not missing architecture implementation. maturity gate, not missing architecture implementation.
## What remains after the executable architecture slice ## What remains within the accepted 2026-08-01 architecture slice
The remaining work is not another Core or cross-module architecture rewrite. The remaining work is not another Core or cross-module architecture rewrite.
It falls into two explicitly different categories, neither of which can be It falls into two explicitly different categories, neither of which can be
@@ -487,12 +498,38 @@ persistence, migrations, recovery/disable semantics, documentation and focused
tests. Their remaining tickets concern concrete providers, deeper adapters and tests. Their remaining tickets concern concrete providers, deeper adapters and
target evidence, not an unresolved institutional architecture boundary. target evidence, not an unresolved institutional architecture boundary.
Everything else described as architecture in this document now has a Everything else described in the accepted baseline of this document now has a
repository owner, versioned contract, bounded implementation, migration and repository owner, versioned contract, bounded implementation, migration and
recovery boundary where state exists, documentation, and executable evidence. recovery boundary where state exists, documentation, and executable evidence.
Further work in those modules is product breadth, UX depth, provider adoption, Further work in those modules is product breadth, UX depth, provider adoption,
and evidence renewal. and evidence renewal.
## Strategic extensions accepted after the baseline
The completed baseline does not imply that institutional product architecture
can no longer grow. The 2026-08-05 strategic review accepted four extensions
that consume the existing contracts without reopening the kernel or moving
domain ownership into Core:
- [Product Experience and Module Boundaries](PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md)
separates technical package topology from stable task/object/product
surfaces; implementation is tracked in Core #283.
- [Federated GovOPlaN Architecture](FEDERATED_GOVOPLAN_ARCHITECTURE.md)
defines governed exchange between autonomous installations; implementation
is tracked in GovOPlaN #41.
- [Assisted and Non-Digital Channels](ASSISTED_AND_NON_DIGITAL_CHANNELS.md)
makes channel inclusion part of the service-to-decision journey; the first
reference proof is tracked in GovOPlaN #42.
- [Institutional Digital Twin](INSTITUTIONAL_DIGITAL_TWIN.md) defines a
time-aware, policy-filtered projection over owner data; implementation is
tracked in GovOPlaN #43.
The eAkte depth required by those journeys is owned by Records and specified in
`govoplan-records/docs/EAKTE_ARCHITECTURE.md`, tracked in Records #1. These are
new product-depth programs with bounded contracts and acceptance journeys, not
evidence that the original institutional semantics or module architecture
failed.
## Delivery tracking ## Delivery tracking
The completed cross-repository architecture epic is The completed cross-repository architecture epic is
+343
View File
@@ -0,0 +1,343 @@
# Kubernetes VM Test Lab
`tools/lab/govoplan-lab.py` creates and operates an amd64 Ubuntu/K3s test
environment on local or SSH-accessible libvirt hypervisors. It provides the
commands requested for the complete VM lifecycle:
| Command | Effect |
| --- | --- |
| `doctor` | Validate the strict inventory and, with `--online`, every hypervisor. |
| `create --apply` | Download checksum-pinned cloud images, create VM overlays and boot the declared VMs. |
| `deploy --apply` | Verify the signed GovOPlaN release, deploy shared state, install pinned K3s and apply GovOPlaN. |
| `update --apply` | Pull newly pinned state images, update K3s serially and roll the selected GovOPlaN release. |
| `status` | Show libvirt VM state, Kubernetes nodes and GovOPlaN pods. |
| `pause --apply` | Gracefully shut down workers, control planes and shared state while retaining disks. |
| `resume --apply` | Start the retained environment in dependency order and wait for readiness. |
| `verify` | Collect sanitized live-cluster evidence and optionally perform the API-pod-loss drill. |
| `destroy --apply --confirm <lab>` | Delete only the lab-owned domains and overlays; local evidence is retained by default. |
Every mutating command is a dry run unless `--apply` is present. Destruction
also requires the exact lab name. Generated credentials, CA keys, manifests and
evidence are written below the configured `state_directory` with owner-only
permissions. Keep that directory outside the repository and include it in the
workstation backup policy. Existing domains are reused or removed only when
their GovOPlaN ownership description and both expected lab disk paths match.
## What The Lab Proves
The supplied inventories describe two different assurance levels:
- `tools/lab/govoplan-lab.example.toml` creates four VMs on one libvirt host.
It is suitable for development, deployment rehearsal, migration testing,
application-pod replacement and recovery-tool exercises. It cannot close
GovOPlaN #27 because one physical host remains one failure domain.
- `tools/lab/govoplan-lab.acceptance.example.toml` places the two workers on
different hypervisors and puts the control and state VMs on a third. It can
produce the bounded stateless application-tier evidence required by #27 when
the declared hypervisors are genuinely independent physical failure domains.
Both examples use one control-plane VM and one state VM. This keeps the bounded
#27 target economical, but it does not prove control-plane or state-service
high availability. For control-plane failover, declare exactly three control
nodes on independent hosts. PostgreSQL, Redis and object-storage failover must
be tested against independently operated HA services; the lab's single state
VM is intentionally a replaceable integration fixture.
Approximate minimum capacity for the four-VM profile is 10 vCPUs, 16 GiB RAM
and 192 GiB of thin-provisioned disk. A six-VM profile with three controls needs
additional capacity. Do not overcommit memory on an acceptance target.
## 1. Prepare The Hypervisors
On each Ubuntu/Debian libvirt host:
```bash
sudo apt-get update
sudo apt-get install -y \
qemu-kvm libvirt-daemon-system libvirt-clients virtinst cloud-image-utils curl
sudo systemctl enable --now libvirtd
```
Use a dedicated lab-administration account. Remote hypervisors are managed over
SSH and the lifecycle invokes `sudo -n` there, so that account needs bounded
non-interactive permission for libvirt, image and cloud-init operations.
`NOPASSWD: ALL` is acceptable only on isolated lab hypervisors.
On a local hypervisor, put the workstation account in the `libvirt` group and
point `vm_image_directory` at a directory writable by that account and
traversable by `libvirt-qemu`. The lifecycle connects explicitly to
`qemu:///system` and does not require passwordless local sudo. Log out and back
in after a new group assignment before running `doctor --online`. Create the
configured image directory before running the doctor; it deliberately rejects
a missing or non-writable storage root instead of silently falling back to a
different filesystem.
Create a dedicated SSH key on the management workstation:
```bash
ssh-keygen -t ed25519 -f "$HOME/.ssh/govoplan-lab" \
-C "GovOPlaN Kubernetes lab"
```
Install its public key for every remote hypervisor account. The same public key
is injected into the VMs. The lifecycle keeps its own `ssh_known_hosts` file,
uses `accept-new` for first contact, and rejects changed host keys until a
lab-owned VM is deliberately recreated.
### Network contract
The configured `bridge` must exist on every selected hypervisor. All VM
addresses are static. Reserve them outside DHCP allocation and ensure that the
management workstation can route directly to every VM address; the lifecycle
does not tunnel VM traffic through the hypervisor SSH connection.
Permit only these flows inside the lab network:
| Port | Source and destination | Purpose |
| --- | --- | --- |
| TCP 22 | management workstation to every VM/hypervisor | Provisioning and evidence collection |
| TCP 6443 | all K3s nodes and management path to controls | Kubernetes API |
| UDP 8472 | K3s node to K3s node | Default Flannel VXLAN; never expose publicly |
| TCP 10250 | K3s node to K3s node | Kubelet metrics and API |
| TCP 2379-2380 | control to control, only with three controls | Embedded etcd |
| TCP 80/443 | test clients to K3s nodes | Traefik/ServiceLB ingress |
| TCP 5432/6379/9443 | K3s nodes to the state VM | PostgreSQL, Redis and TLS-protected Garage S3 |
| TCP 3025/3143 | approved test clients/workers to the state VM | GreenMail SMTP/IMAP test endpoints |
The official
[K3s networking requirements](https://docs.k3s.io/installation/requirements#networking)
remain authoritative. Restrict state ports to the lab network even though the
generated integration stack binds them on the state VM.
## 2. Create The Inventory
Start with the one-host rehearsal:
```bash
install -d -m 0700 "$HOME/.config/govoplan/labs"
cp tools/lab/govoplan-lab.example.toml \
"$HOME/.config/govoplan/labs/development.toml"
chmod 0600 "$HOME/.config/govoplan/labs/development.toml"
```
Edit at least the bridge, network, static addresses and SSH key paths. For a
multi-host run, copy the acceptance example and replace every example hostname,
failure-domain declaration and network value. Strict parsing rejects unknown
keys, mutable HTTP inputs, malformed checksums, duplicate addresses/MACs and an
acceptance inventory that collapses workers onto one declared hypervisor or
failure domain.
Cloud image, K3s binary, K3s installer and GovOPlaN release inputs are URL plus
SHA-256 pairs. Updating means changing those reviewed pins and then running the
`update` command; the tool deliberately does not follow `latest` aliases.
The one-host example uses the dedicated `govoplan-lab` NAT network. Its DHCP
pool ends at `192.168.123.99`; the static lab addresses start at
`192.168.123.201`. Define and start it once on the local hypervisor:
```bash
virsh --connect qemu:///system net-define \
tools/lab/libvirt/govoplan-lab-network.xml
virsh --connect qemu:///system net-autostart govoplan-lab
virsh --connect qemu:///system net-start govoplan-lab
```
Re-running those commands is unnecessary when `virsh net-info govoplan-lab`
already reports an active, persistent network. The lab destroy command leaves
this reusable network in place.
## 3. Validate And Create The VMs
```bash
LAB="$HOME/.config/govoplan/labs/development.toml"
PYTHON="/mnt/DATA/git/govoplan/.venv/bin/python"
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" doctor --online
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" create
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" create --apply
```
The preview is safe to run repeatedly. Creation reuses a domain whose exact
lab-owned name already exists and otherwise creates a thin qcow2 overlay under
`vm_image_directory/<lab>/<node>`.
## 4. Deploy GovOPlaN
If `git.add-ideas.de` requires authentication for release images, export a
read-only package/container-registry identity for this shell. A Gitea package
token can be used as the password:
```bash
export GOVOPLAN_LAB_REGISTRY_USERNAME='package-reader'
read -r -s GOVOPLAN_LAB_REGISTRY_PASSWORD
export GOVOPLAN_LAB_REGISTRY_PASSWORD
```
Then preview and apply:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" deploy
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" deploy --apply
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" status
unset GOVOPLAN_LAB_REGISTRY_PASSWORD
```
Deployment verifies the downloaded release manifest and keyring by pinned
digest and by the existing GovOPlaN signature policy. It deploys PostgreSQL,
Redis, single-node Garage and GreenMail on the state VM. The API, WebUI, workers
and scheduler run in K3s from digest-pinned release images. A private lab CA
protects both ingress and S3; backend pods receive only the CA Secret and keep
TLS verification enabled. The CA profile carries critical `CA:TRUE` and
`keyCertSign,cRLSign` constraints. `deploy` and `update` rotate older lab CAs
that do not satisfy that profile and reissue the ingress/S3 certificate.
The final output identifies two local files below `state_directory`:
- `hosts` maps the public GovOPlaN and S3 test names to their VM addresses;
- `pki/ca.crt` is the private lab CA certificate.
Add the host mappings to the test client's resolver and trust the CA only on
devices used for this lab. On Debian/Ubuntu:
```bash
STATE="$HOME/.local/share/govoplan/labs/govoplan-k8s-lab"
cat "$STATE/hosts"
sudo install -m 0644 "$STATE/pki/ca.crt" \
/usr/local/share/ca-certificates/govoplan-k8s-lab.crt
sudo update-ca-certificates
```
Review mappings before adding them to `/etc/hosts`; the lifecycle does not edit
the workstation's trust or resolver configuration. Reinstall `pki/ca.crt` in
the client trust store after an automatic CA rotation.
### Enroll the first administrator
The production runtime does not create a default password. Issue one expiring,
single-use first-administrator credential inside an API pod and copy its
owner-only artifact out immediately:
```bash
KUBECTL="$STATE/bin/kubectl"
POD="$($KUBECTL -n govoplan get pods \
-l app.kubernetes.io/component=api \
-o jsonpath='{.items[0].metadata.name}')"
ARTIFACT="$STATE/first-admin-enrollment.json"
umask 077
$KUBECTL -n govoplan exec "$POD" -- \
python -m govoplan_core.commands.first_admin issue \
--reason 'initial Kubernetes lab enrollment' \
--output /tmp/first-admin-enrollment.json
$KUBECTL -n govoplan exec "$POD" -- \
cat /tmp/first-admin-enrollment.json > "$ARTIFACT"
$KUBECTL -n govoplan exec "$POD" -- \
rm -f /tmp/first-admin-enrollment.json
chmod 0600 "$ARTIFACT"
```
Submit the token from that artifact once to
`/api/v1/bootstrap/first-admin` with the administrator email, display name,
password, tenant slug and tenant name. The password must contain at least 12
characters. The lab command performs that exchange without placing either the
token or password in process arguments, rejects redirects, and removes the
artifact only after HTTP 201:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" enroll-admin \
--email 'owner@example.org' \
--display-name 'System Owner' \
--tenant-slug default \
--tenant-name 'Default Tenant'
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" enroll-admin \
--email 'owner@example.org' \
--display-name 'System Owner' \
--tenant-slug default \
--tenant-name 'Default Tenant' \
--apply
```
The public lab hostname must already resolve on the management workstation;
the command verifies TLS through the generated private CA directly.
## 5. Collect #27 Evidence
Create a short-lived API key authorized to read the Ops status endpoint. In the
current Access administration UI, open **Tenant API keys** and select only
**View tenant settings** (`admin:settings:read`); the Ops endpoint explicitly
accepts that compatibility scope. A dedicated operator credential may instead
use `ops:operations:read`. Then run:
```bash
export GOVOPLAN_OPS_API_KEY='short-lived-value'
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" verify \
--exercise-api-pod-loss
unset GOVOPLAN_OPS_API_KEY
```
The verifier requires ready API and WebUI pods across at least two Kubernetes
nodes, all Deployments available, consistent runtime composition, queue
coverage and a valid database-connection budget. During the optional drill it
deletes one ready API pod, probes public readiness and waits for replacement.
It writes sanitized output to
`state_directory/evidence/kubernetes-multi-host.json` and never stores the API
key. A rehearsal inventory prints an explicit warning that its result is not
independent-failure-domain evidence.
Retain these private artifacts together for review:
1. `inventory.json` and the reviewed inventory TOML;
2. the adopted release manifest/keyring and installation receipt;
3. `kubernetes.json`;
4. the Kubernetes verifier output;
5. private cluster logs for the approved drill window;
6. the operator's out-of-band evidence that the worker hypervisors are
independent physical hosts or availability zones.
GovOPlaN #37 additionally requires independent assessment and production
approval keys. Running its evidence jobs in containers is supported, but a
container does not create an independent authority. Follow
`TARGET_MATURITY_EVIDENCE_RUNBOOK.md` after the #27 drill passes.
## 6. Update, Pause, Resume And Remove
After reviewing and changing pinned image/K3s/release values in the inventory:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" update
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" update --apply
```
Workers are cordoned, drained, updated and uncordoned one at a time. K3s
controls are reconciled serially. The release-specific migration Job remains
subject to GovOPlaN's signed backup-evidence gate. The lab update command is not
a substitute for creating recovery evidence before a destructive state-schema
change.
To stop compute use without deleting disks:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" pause --apply
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" resume --apply
```
To remove VM resources while preserving local evidence:
```bash
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" destroy
"$PYTHON" tools/lab/govoplan-lab.py --config "$LAB" destroy \
--apply --confirm govoplan-k8s-lab
```
Add `--purge-local-state` only after evidence and recovery material have been
retained elsewhere. That option deletes the generated local CA, secrets,
manifests and evidence as well as the VMs.
## Acceptance Boundary
This tool supplies reproducible infrastructure and executes the bounded
stateless-node drill. It does not certify the truth of operator-entered failure
domains, provide HA PostgreSQL/Redis/Garage, create production backup evidence,
or approve its own results. Those boundaries are deliberate: #27 can close
after a passing run on independently controlled hosts; broader production
maturity remains governed by #35, #37 and the target evidence runbook.
+88 -9
View File
@@ -136,23 +136,38 @@ allow the same least-privilege credential to serve every module workflow.
## Exact release consumption ## Exact release consumption
`tools/release/generate-release-package-set.py` translates the reviewed Git `tools/release/generate-release-package-set.py` supports two explicit package
source refs in `requirements-release.txt` into an exact registry package set. profiles. `base` translates the reviewed roots in `requirements-release.txt`;
It resolves each version tag to its commit and verifies the package metadata in `full` reads the exact `govoplan[full]` dependency set from the developer
that tag. meta-package. Both profiles resolve every version tag to its commit and verify
the package metadata from that exact Git tree. The official module directory
and immutable runtime distribution use `full`, so every publicly released
module can be discovered without rebuilding the application image.
`tools/release/resolve-package-artifacts.py` then downloads exactly those wheel `tools/release/resolve-package-artifacts.py` then downloads exactly those wheel
and WebUI versions from Gitea. It reads the identity embedded in every wheel and and WebUI versions from Gitea. It reads the identity embedded in every wheel and
npm tarball, rejects missing, duplicate, unexpected, or oversized artifacts, npm tarball, rejects missing, duplicate, unexpected, or oversized artifacts,
and writes `package-artifacts.lock.json` with SHA-256 values and npm integrity and writes `package-artifacts.lock.json` with credential-free HTTPS download
values. Credentials are accepted only through environment variables and are URLs, SHA-256 values, and npm registry integrity values. The resolver verifies
that the bytes downloaded by `npm pack` match the registry's own integrity
record. Credentials are accepted only through environment variables and are
never written to the lock. Python resolution ignores ambient pip configuration never written to the lock. Python resolution ignores ambient pip configuration
and extra indexes for GovOPlaN roots, preventing an internal package name from and extra indexes for GovOPlaN roots, preventing an internal package name from
being selected from an undeclared registry. being selected from an undeclared registry.
The runtime distribution workflow uses the verified wheelhouse directly and The runtime distribution workflow uses the verified full-profile wheelhouse
installs module WebUI tarballs only after matching them to the lock. It publishes directly and installs every selected module WebUI tarball only after matching
the package set, package lock, and hash-locked requirements as release assets. it to the lock. It publishes the package set, package lock, and hash-locked
requirements as release assets.
The WebUI installer receives the absolute runtime-build interpreter path so its
directory changes cannot escape the isolated release environment.
Gitea 1.24 dispatches this workflow from a branch, but that branch is only the
workflow implementation. The job fetches and peels the protected `v<version>`
tag explicitly and materializes both `requirements-release.txt` and the
developer meta-package from that Git tree. It then binds the signed distribution
source and Gitea release assets to the same exact commit. A post-tag workflow
repair can therefore retry publication without changing the released package
composition or relabelling the later branch commit as released source.
The package-lock SHA-256 is part of the signed distribution manifest. Runtime The package-lock SHA-256 is part of the signed distribution manifest. Runtime
finalization also requires the lock's package versions and hashes to match the finalization also requires the lock's package versions and hashes to match the
wheel composition embedded in the images. OCI assembly remains network-free wheel composition embedded in the images. OCI assembly remains network-free
@@ -162,6 +177,65 @@ The source refs remain in the module catalog for source provenance and release
planning. Production installation consumes the signed runtime images rather planning. Production installation consumes the signed runtime images rather
than invoking `pip`, `npm`, or Git on the target host. than invoking `pip`, `npm`, or Git on the target host.
## Public module directory
`tools/release/publish-release-catalog.sh` resolves the selected package set and
registry lock before it creates a catalog. Catalog entries are synthesized from
the exact tagged module manifests, never from a hand-maintained module list or
the current workspace. Each entry binds its Python wheel and optional WebUI
tarball to the registry URL, filename, size, SHA-256, package identity, source
tag, and source commit before the complete catalog is signed.
The same publication transaction regenerates and prunes the browsable static
directory under `public/catalogs/v1/modules/`. It writes a global
`modules/index.json`, one `<module>/index.json`, and one
`<module>/<version>/manifest.json` for every entry in the signed channel.
These files are derived from that exact signed payload and keyring; stale JSON
from an older partial catalog is removed while unrelated static assets are left
untouched. The signed channel remains the trust anchor, while the module
directory provides stable discovery URLs for browsers and external tooling.
Official GovOPlaN modules are open-source directory entries and do not require
license entitlements. The generic `license_features` contract remains available
for third-party package directories, support/configuration packages, or future
deployment-specific presets. A catalog entry is gated only when that entry
explicitly declares such features.
Core carries the public stable catalog URL and its independently pinned trust
anchor. In the absence of an operator-configured catalog, Admin discovers the
official directory automatically. Selecting an entry creates a reviewed
install/update plan; the trusted installer downloads the exact signed artifacts
into a private digest cache, verifies size and hash, and installs only from that
cache. A saved plan is rejected if any package ref, artifact identity, catalog
channel, sequence, or signing-key identity differs from the currently validated
catalog.
The Admin directory can be searched by module, package, repository, or tag and
filtered by available, installed, update, and blocked/withdrawn states. It
shows the source revision, artifact digest, release notes, and configuration
requirements. Missing dependency/interface providers and unsupported update
windows are surfaced before an operator adds the entry to a plan; installer
preflight remains authoritative.
Catalog entries also carry the permission definitions declared by the tagged
module manifest. Admin groups and exposes their scopes before an install or
update is planned. This is disclosure only: installing a module does not grant
its permissions to an account, role, group, tenant, or service account.
Package lifecycle and availability are intentionally separate:
- install, update, and uninstall change the instance-wide package composition;
- enable and disable change the active instance runtime graph;
- tenant module entitlements define unavailable, available, and forced modules;
- group/user presentation is governed through Views and Policy; and
- enabling a capability module does not opt data into that capability.
Single-process or single-host installations may execute a supervised package
plan locally. Shared-state and Kubernetes profiles reject node-local package
mutation: operators compose and roll out a new signed full-profile runtime image
instead. This prevents replicas from drifting while retaining the same Admin
catalog and preflight experience.
## Developer meta-package ## Developer meta-package
`packages/govoplan-meta` builds the optional `govoplan` package. Its default `packages/govoplan-meta` builds the optional `govoplan` package. Its default
@@ -179,6 +253,11 @@ tags. The meta-package is for editable/developer setup and composition tests. It
does not enable modules, apply migrations, provision services, or establish does not enable modules, apply migrations, provision services, or establish
backup and recovery evidence. backup and recovery evidence.
If the tag-triggered developer meta-package job fails before publication, rerun
`publish-developer-meta-package.yml` with the existing protected version. The
manual path validates that tag against `main`, checks out its exact commit, and
publishes only when the registry does not already contain the same wheel hash.
Generic Packages are intentionally not used. Add that transport only when a Generic Packages are intentionally not used. Add that transport only when a
consumer needs an artifact format unsupported by PyPI, npm, Gitea Releases, or consumer needs an artifact format unsupported by PyPI, npm, Gitea Releases, or
the OCI registry. the OCI registry.
+198
View File
@@ -0,0 +1,198 @@
# GovOPlaN Platform Core Ideas
## Purpose
GovOPlaN is an institutional governance and operations layer. Its central
promise is:
> Model the institution, orchestrate its work, connect its systems, and
> preserve why and under whose authority it acted.
The platform should let people complete a real task without understanding its
repository or module graph. It should let institutions retain control over
their data, procedures, providers, and deployment while still sharing
interoperable definitions and evidence.
This document is the stable summary of the ideas that every product package,
module, interface, and integration must preserve. Current implementation state
lives in [Strategy Status](STRATEGY_STATUS.md).
## Ten Core Ideas
### 1. Institutional context before application context
Work happens for a tenant, institution, organizational unit, function,
mandate, jurisdiction, service, case, and represented party. The real actor
and represented capacity remain distinct. Application permissions alone do not
prove institutional competence.
### 2. Governance is executable
Policy is not explanatory prose around an operation. Consequential actions
must expose applicable rules, authority, purpose, expected effects, review
requirements, recovery behavior, and evidence. Inheritance may tighten a rule
but must not silently loosen an upstream constraint.
### 3. Time has two independent meanings
Valid time answers when a fact applied. Recorded time answers what the system
knew at a point in history. Historical browsing changes the business-data
projection, never the current authorization context. Corrections and
supersession remain visible rather than rewriting history.
### 4. One context, many owners
Cases, tasks, decisions, records, messages, files, appointments, reports, and
external objects remain owned by their domain modules or source systems. Stable
references create one navigable context without a universal copied master
record or cross-module table access.
### 5. Native and connected operation are peers
For every integration, GovOPlaN states whether it is authoritative, mirrors an
external source, synchronizes governed fields, adds a governance overlay, or
keeps a link only. An external system can be used today and replaced later
without losing provenance or institutional control.
### 6. Human work is a first-class system object
An intake becomes owned, reviewable work. A person can see the current context,
next responsible action, reason, deadline, consequence, and completion
evidence. Workflow Engine coordinates machine and human transitions; focused
views guide people through the relevant platform surfaces.
Tasks owns explicit work items and the unified work inbox. Workflow Engine owns
process execution and resumable handoffs. Notifications attract attention, and
domain modules retain their business objects. These boundaries prevent an
inbox, workflow, or notification from becoming a second copy of institutional
state.
### 7. Views reduce complexity without changing authority
The interface is a task- and role-sensitive projection of installed
capabilities. Views, dashboards, search, documentation, and workflow-guided
surfaces may hide irrelevant functions, but they never grant access. Users can
escape a focused mode when policy permits and can always understand why
something is unavailable.
Configurable product areas organize authorized capabilities around work,
services, records, communication, meetings, data and institutional
responsibility. The optional Quick Access rail presents task-local Work,
Calendar, Messages and Files contributions without merging their owners or
turning presentation settings into permissions.
### 8. Evidence and recovery are part of the operation
Intent, exact input versions, approvals, external effects, receipts,
outcome-unknown states, reconciliation, corrections, retention, and recovery
belong to one evidence chain. A retry must be idempotent; rollback claims must
distinguish reversible local state from effects already observed elsewhere.
### 9. Inclusion is multi-channel, not portal-only
Public portal, postbox, mail, telephone, paper, in-person assistance, APIs, and
external systems are channels around the same governed work. Assisted entry
records who entered information, for whom, from which source, with which
attestation, and how the affected person receives a usable receipt and
correction path.
Responsive, mobile, desktop, and embedded launch surfaces are additional ways
to enter the same governed context, not separate products with weaker authority
or evidence. Common task-local actions may open in bounded overlays while their
owning modules retain validation, policy, and persistence.
### 10. Successful configurations are portable products
Modules are ingredients. A usable product is a signed configuration package
with terminology, forms, policies, workflows, views, reports, provider
profiles, documentation, migration rules, and evidence. Institutions derive
local packages without forking code or weakening inherited constraints.
## Platform Planes
The planes below are ownership lenses, not navigation groups or mandatory
deployment tiers.
| Plane | Responsibility |
| --- | --- |
| Experience | Shell, views, dashboard, search, help, accessibility, and task-focused composition |
| Participation and channels | Portal, postbox, mail, campaigns, calendar, scheduling, consultation, and assisted channels |
| Human work and procedure | Services, forms/runtime, cases, tasks, approvals, workflow execution, and domain procedures |
| Content, records, and evidence | Files, templates, DMS, eAkte/records, audit, reporting, transparency, and publication |
| Institutional governance | Identity, access, tenancy, organizations, functions, mandates, policy, trust, and formal decisions |
| Data and integration | Connectors, datasources, dataflow, search, external references, provider health, and reconciliation |
| Runtime and assurance | Module composition, operations, deployment, recovery, security evidence, and signed packages |
Collected product ideas and normalized actor outcomes are preserved in the
[Product Input Register](PRODUCT_INPUT_REGISTER.md). They enter implementation
only through a named journey, package, or explicit discovery issue.
## Canonical Distinctions
The platform must not collapse these pairs:
- identity vs account vs represented capacity;
- role/permission vs function/mandate/competence;
- valid time vs recorded time;
- purpose for use vs general technical access;
- document content vs managed file bytes vs institutional record;
- task vs workflow definition vs workflow instance;
- approval vs formal decision;
- message intent vs transport delivery vs recipient acknowledgement;
- source authority vs connector maturity;
- current state vs historical evidence;
- correction/compensation vs erasure of an observed effect;
- a module boundary vs a user-visible product boundary.
## Product Experience Rule
The normal user interface speaks in services, work, records, messages,
meetings, decisions, and outcomes. Module names, provider IDs, capability names,
package coordinates, and schema details are technical provenance. They are
visible to administrators and in expandable diagnostics, but they are not the
primary information architecture for ordinary work.
The complete permission-derived tool catalogue remains deliberately available
to power users. Product areas and Quick Access provide sensible system and
tenant defaults plus governed user personalization; they do not make familiar
tools harder to reach merely to conceal modular implementation.
## Maturity Rule
A repository, route, model, or unit test does not make a capability complete.
Claims advance only with evidence appropriate to the claim:
1. `scaffold`: boundary and documentation exist;
2. `vertical_slice`: useful behavior has focused tests;
3. `reference_ready`: an end-to-end reference journey passed target,
accessibility, privacy, security, operations, and recovery evidence;
4. `supported`: upgrades, interoperability, support procedures, and release
guarantees are defined;
5. `lts`: compatibility and maintenance windows are contractual.
## Deliberate Non-Goals
GovOPlaN does not aim to:
- replace every specialist system, ERP, DMS, groupware, or data tool;
- make one database authoritative for every connected fact;
- expose every installed capability to every person;
- infer authority from organizational membership alone;
- make historical browsing weaken current security;
- treat AI output as an unaccountable institutional decision;
- create a repository for every noun in the information model;
- claim production maturity from local development evidence.
## Decision Test
A proposed feature fits the platform when it improves at least one real
institutional journey and can answer:
1. Who owns the object and source of truth?
2. In which institutional and temporal context does it apply?
3. For which declared purpose may it be used?
4. Which policy and authority permit the action?
5. What effect, evidence, retention, and recovery behavior result?
6. How can it operate with an external owner without losing autonomy?
7. How will a person discover and complete it without learning the module
graph?
+8
View File
@@ -49,6 +49,13 @@ firewall ports. Those inputs are sufficient to provision a k3s target. They are
not sufficient to claim control-plane HA unless three control-plane failure not sufficient to claim control-plane HA unless three control-plane failure
domains are present. domains are present.
The repository now supplies the strict libvirt/K3s lifecycle and example
inventories for this handoff in
[`KUBERNETES_TEST_LAB.md`](KUBERNETES_TEST_LAB.md). Its `acceptance` mode
rejects a declared topology unless the workers and shared-state fixture occupy
different hypervisor and failure-domain identifiers. Reviewers must still
verify that those identifiers correspond to genuinely independent hosts.
### Separate deployment and evidence authorities ### Separate deployment and evidence authorities
The deployment identity may create and update the namespace, Secret, The deployment identity may create and update the namespace, Secret,
@@ -74,6 +81,7 @@ python tools/deployment/govoplan-deploy.py render-kubernetes \
--namespace govoplan \ --namespace govoplan \
--secret-name govoplan-runtime \ --secret-name govoplan-runtime \
--tls-secret-name govoplan-tls \ --tls-secret-name govoplan-tls \
--s3-ca-secret-name govoplan-s3-ca \
--ingress-class-name nginx \ --ingress-class-name nginx \
--output /srv/govoplan/<installation-id>/kubernetes.json --output /srv/govoplan/<installation-id>/kubernetes.json
@@ -0,0 +1,173 @@
# Product Experience and Module Boundaries
## Problem
GovOPlaN's runtime modularity is a strength, but the implementation structure
is exposed too directly in the product. Ordinary users encounter module names,
one top-level route per module, one navigation item per repository, package and
provider identifiers, and errors framed as missing modules. This makes the
system look like a toolbox of adjacent applications instead of one operating
environment for institutional work.
The correction is not a monolithic frontend and not hidden provenance. It is a
separate product information architecture assembled from typed module
contributions.
Implementation is tracked in
[Core #283](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/283).
The accepted configurable product-area and task-local tool design is defined
in [Quick Access And Product Areas](QUICK_ACCESS_AND_PRODUCT_AREAS.md).
## Current Exposure Inventory
| Surface | Direct exposure | Appropriate audience | Product-facing alternative |
| --- | --- | --- | --- |
| Side rail | One icon and route for many installed modules | Administrators and power users | Work areas, services, inboxes, records, communication, data and assurance |
| Route paths | Technical owners such as `/dataflow`, `/forms`, or `/postbox` | Deep links and diagnostics | Stable product aliases and journey routes that resolve to owner surfaces |
| Dashboard | Installed module count and module-owned widget library | Operators | Outcome, obligation, work, exception, and service widgets |
| Administration | Package names, database state, capabilities, providers | Module and system administrators | Guided product/package configuration with technical details on demand |
| Errors | "Module/capability not installed" | Diagnostics | Explain the unavailable outcome, responsible administrator, and enabling path |
| Documentation | Topics grouped primarily by module | Administrators | Task, role, service, and object documentation with module provenance secondary |
| Permissions | Module-namespaced scopes | Access administrators | Human-readable responsibility bundles; exact scopes remain inspectable |
| Search | Provider/module as a result facet | Advanced filtering | Object type, institution, time, purpose, case/service, and source authority |
| Workflow | Steps can expose target route/module details | Workflow designers | User-facing action and expected result; technical binding in definition details |
| Connector state | Provider IDs and source types | Integration owners | Named source, authority, freshness, health, last effect, and recovery state |
## Boundary Decision
Three layers remain distinct:
1. **Technical module layer:** package ownership, dependencies, capabilities,
permissions, migrations, routes, and provider identifiers.
2. **Product composition layer:** work areas, object types, journeys, commands,
inboxes, configuration packages, and role-based defaults.
3. **Presentation projection:** active view, tenant policy, current task,
temporal context, language, accessibility preferences, and device layout.
Modules own implementation and contribute typed product metadata. Core
assembles it. Views filters it. Policy constrains it. Access authorizes the
underlying actions. No consumer imports another optional module's UI directly.
## Product Surface Contract
Each WebUI module should be able to announce:
- `product_areas`: stable areas to which a route, command, widget, or object
belongs;
- `object_types`: user-facing nouns, icons, search context, detail route, and
owner provenance;
- `work_item_sources`: open work, exceptions, deadlines, and responsible
capacity;
- `journey_actions`: launch, resume, review, correct, decide, publish, and
reconcile commands;
- `workspace_surfaces`: embeddable but owner-rendered list, detail, editor, and
status surfaces;
- `configuration_contributions`: guided settings with consequence and
prerequisite metadata;
- `help_contexts`: user/admin documentation for the product identity as well as
the technical owner;
- `technical_provenance`: module, interface version, capability, and provider
identifiers shown only in details and evidence.
The contract references surfaces. It does not permit Core or a product package
to import their implementation.
## Navigation Model
The default shell should prioritize:
1. global search and create/resume commands;
2. personal and function-bound work;
3. configured product areas;
4. pinned user destinations;
5. administration and technical module inspection when authorized.
The baseline product areas are Work, Services and Cases, Records and
Documents, Communication, Meetings and Decisions, Data and Assurance, and
People and Responsibility. They are configurable system/tenant defaults and
Views projections, not hard-coded repository groups. Empty areas disappear;
single-destination areas may link directly; familiar tools may remain pinned.
The complete permission-derived module rail remains available as **All
available tools**. Its ability to scroll is useful and is not itself the
product defect. The defect is requiring people to infer a task or outcome from
repository topology.
Task-local Work, Calendar, Messages and Files tools may be contributed to the
optional `govoplan-quick-access` rail. Messages composes Mail, Postbox and
future governed chat presentation without merging their channel semantics or
state.
A module route remains a valid deep link. A product area may combine links and
owner-rendered surfaces from several modules. When a required contribution is
absent, the area explains the missing outcome rather than rendering a broken
placeholder.
Views remain the projection mechanism. They may select product areas, routes,
sections, commands, widgets, and fields. A view must not grant a permission or
change data semantics. Policy can force, allow, or prohibit a surface at system,
tenant, group, or user scope.
## Error And Provenance Language
Normal errors answer:
- what the person was trying to achieve;
- why it is unavailable or failed;
- whether data was saved or an external effect may have occurred;
- who can resolve it and where;
- the correlation/evidence reference.
An expandable technical section may then identify the module, capability,
provider, request, and version. This keeps the product intelligible without
hiding operational truth.
## Migration
Core's product-area and Quick Access contracts, the optional Quick Access
module, the first five providers and immutable View presentation revisions are
implemented. The migration below now concerns broader classification and
product-language adoption; it is not a prerequisite for safely enabling the
first rail slice.
### Slice 1: inventory and aliases
- classify every route, navigation item, widget, setting, search object, and
help context by product area and object type;
- add product aliases without removing existing deep links;
- flag raw module IDs in ordinary-user labels and errors.
### Slice 2: work-first shell
- provide a generic work/exception/deadline aggregation capability;
- make work areas and configured packages the default navigation;
- move the complete module catalogue to administration and an optional power-
user surface.
- implement the configurable Quick Access rail through Core-mediated
contributions, system/tenant/user resolution and View/Policy ceilings.
### Slice 3: composite journeys
- let product packages define journey launch/resume actions and default views;
- let Workflow Engine activate a view and focus an owner surface without
controlling authorization;
- expose provider provenance and technical bindings on demand.
### Slice 4: enforceability
- make product classification mandatory for user-visible manifest surfaces;
- reject duplicate product identities and missing owner routes in CI;
- add browser tests proving that reference users can complete a journey without
knowing module names.
## Acceptance Criteria
- An ordinary user can describe every primary navigation item as work or an
institutional object, not as a package.
- A product package can remove irrelevant navigation while retaining deep-link
and help integrity.
- Missing optional modules produce an actionable product explanation.
- Administrators can still inspect exact module, capability, provider, schema,
and evidence provenance.
- Module permutation tests prove that no product surface assumes an optional
owner is installed.
+203
View File
@@ -0,0 +1,203 @@
# GovOPlaN Product Input Register
## Purpose
This document preserves and normalizes product ideas and user-story notes that
inform GovOPlaN without turning a private note file into a second backlog.
Gitea issues remain the source of live work state; the stable platform direction
remains in [Platform Core Ideas](PLATFORM_CORE_IDEAS.md), the
[Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md),
and the [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md).
The register was reconciled on 2026-08-06 from:
- `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/ideas.md`;
- `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/user_stories.txt`.
The source notes remain useful as the original capture. This maintained version
uses consistent terminology, makes ownership explicit, and records where an
idea enters the product program.
## Product Themes
### Operable and scalable installation
An operator should be able to install, update, reconfigure, scale, back up,
restore, pause, and retire GovOPlaN through one explainable control plane.
Existing infrastructure may be reused or managed components may be provisioned.
The WebUI and CLI must invoke the same governed operations, show the planned and
completed effects, preserve recovery evidence, and never claim rollback for an
external effect that cannot actually be reversed.
This theme is owned by Core, Admin, Ops, Policy, Files, and the signed product
package. It is tracked primarily by GovOPlaN #13 and the production evidence
issues. It advances in parallel with, but does not replace, actor-facing
reference journeys.
### Focused, consistent work
People should see the work and tools relevant to the current task, not the
installed module graph. Views may be defined by administrators, groups, or
users within policy. Workflow instances may pin a governed View. Contextual
help, predictable action placement, consistent central components, visible
intermediate results, and plain institutional terminology are product
requirements.
Small task-local actions such as writing a Mail or Postbox message, completing
a Template, or manipulating Files should be launchable without abandoning the
current context. These actions remain owned by their modules and use bounded
overlays or workspaces; the shell supplies discovery and return context rather
than reimplementing them.
The accepted first presentation is the optional, configurable Quick Access
rail: Work, Calendar, Messages and Files. Messages may compose Mail, Postbox
and future chat contributions while preserving their separate authority and
channel semantics. System and tenant administrators govern availability and
forced entries; users select categories and ordering within those ceilings.
Modules register typed contributions through Core and continue to work when
Quick Access is absent.
This theme is owned by Core experience contracts, Views, Dashboard, Tasks,
Workflow Engine, Quick Access, Docs, and the contributing feature modules. The
first proof is the resumable service-to-decision/eAkte journey in GovOPlaN #42.
### Governed human work
An intake or event becomes owned work with a responsible actor or function,
priority, deadline, current action, consequence, source context, and completion
evidence. Tasks owns explicit work items and the unified work inbox. Workflow
Engine owns process execution, waits, retries, and handoffs. Domain modules own
the business objects and commands. Notifications attract attention but do not
replace durable work state.
This distinction applies to service requests, technical support, approvals,
data reconciliation, campaigns, meetings, decisions, records, and failed
automation. It is the immediate shared implementation priority because users
must be able to leave work and resume it safely.
### Institutional responsibility and workforce context
Organization units, functions, mandates, assignments, delegations, and acting
context determine institutional responsibility. Presence, absence, illness,
availability, and similar status are effective-dated operational facts used to
route work, suppress or redirect notifications, explain planning, and trigger
policy. They are not merely profile decorations and they do not replace the IDM
lifecycle status of an identity or account.
Time recording, absence management, sickness reporting, return-to-work
management, and applicant management form a possible workforce package. The
first implementation must be driven by a real journey and legal/privacy
profile; no new module boundary is implied solely by this register.
### Integration-first and provider-neutral operation
GovOPlaN should integrate tightly with software already used by an institution
and offer native alternatives only where that produces a better governed
outcome. Core-mediated provider contracts expose stable, vendor-neutral
capabilities; adapters encapsulate specific products. Authority, synchronized
fields, conflict behavior, health, credential custody, provenance, and
retirement must be explicit.
The LBV Baden-Wuerttemberg idea is retained as a candidate workforce/payroll
integration profile and as a test of provider-neutral contracts. Desktop and
groupware integration for Microsoft Office, Outlook, LibreOffice, Thunderbird,
file managers, Windows, Unix, and macOS should use standards, deep links,
protocol handlers, synchronization, and governed connectors before custom
desktop software is introduced.
### Inclusive channels and device surfaces
Portal, Postbox, Mail, telephone, paper, in-person assistance, API, desktop,
and mobile are channels around the same governed work. A responsive or native
mobile surface must not create a second authority or data model. Assisted work
records representation, source, attestation, receipt, correction, and delivery
choice. People may opt into permitted distribution channels while policy keeps
mandatory channels and legal delivery requirements explicit.
Video meetings, chat, instant messaging, and forums are retained as governed
collaboration-channel candidates. The default direction is integration with an
established provider through typed message, meeting, participant, evidence, and
retention contracts before building another communications stack.
### Meetings, deliberation, decisions, and voting
An institutional meeting spans scheduling, participants and mandates,
documents, agenda, discussion, formal motions, votes, decisions, minutes,
follow-up work, publication, and eligible expense settlement. Committee owns
the meeting and deliberation semantics while Calendar, Scheduling, Files,
Templates, Decisions, Tasks, Reporting, Ledger, and Voting contribute optional
capabilities.
Voting requiring certified assurance remains a provider program. POLYAS is the
first external profile; a native provider may progress only through the
controlled assurance and certification program already tracked in Voting.
### Controlled data work and understandable reporting
People should manipulate data through immutable inputs, previewed operations,
intermediate materializations, reversible definition changes, durable review
decisions, quality rules, and complete lineage. Reports expose their definitions
and source revisions so controllers can understand and change how a result is
produced. Technical support may package controlled workflows that let
non-technical users safely operate otherwise hidden data.
The monthly-data journey is the first proof. Sanctions screening follows on the
same source, snapshot, transformation, review, reporting, workflow, and
delivery contracts.
### Institutional memory and consequence
Decisions should be prepared, discussed, made, communicated, implemented, and
filed with their authority and consequences visible. A record/eAkte provides
the familiar administrative context across exact source revisions without
copying ownership from Cases, Decisions, Files, Forms, Campaign, Postbox, or
other modules. The institutional digital twin may later use governed
projections to model and simulate organizational change, but simulation output
never becomes authority without an explicit adoption decision.
## Normalized Story Catalogue
The following catalogue preserves the intent of the collected notes. It is an
orientation index, not a completion checklist.
| Actor and desired outcome | Product owner or composition | First proof |
| --- | --- | --- |
| Operator installs, updates, scales, backs up, restores, and rolls back through one explainable workflow | Core, Admin, Ops, signed package | GovOPlaN #13 and target-evidence lane |
| System and tenant module administrators govern module availability and lifecycle | Core, Admin, Policy, Tenancy | Module entitlement and lifecycle composition |
| User works in a decluttered, consistent and task-sensitive interface | Views, Core, Dashboard, Workflow, Docs | Service-to-decision workspace |
| Policy maker defines inherited, explainable and enforced rules | Policy plus every consequential owner | Information-governance adoption gate |
| Controller and auditor reconstruct results, rules, evidence and correction paths | Audit, Reporting, Records, Dataflow | Monthly-data and eAkte journeys |
| User sees institutional terminology, current progress, intermediate results and consequences | Domain owner, Tasks, Workflow, Views | All reference journey acceptance tests |
| Voting body and voter obtain independently assured democratic voting | Voting, Committee, Identity Trust, Encryption | POLYAS profile and controlled native-provider program |
| Support staff packages safe guided manipulation of hidden data | Workflow, Dataflow, Tasks, Views | Monthly reconciliation workflow |
| Data worker performs controlled, understandable and recoverable transformations | Datasources, Connectors, Dataflow, Reporting | GovOPlaN #8 |
| Decision maker prepares, deliberates, decides, records and follows consequences | Committee, Decisions, Tasks, Records, Reporting | Service-to-decision journey |
| Management delegates responsibility and receives governed activity reports | Organizations, IDM, Access, Policy, Reporting | Function-bound Postbox and work inbox |
| Institution models and simulates organizational change | Organizations, Policy, Dataflow, Reporting, Digital Twin | Later governed digital-twin package |
| Sender distributes generated files to functions without knowing incumbents | Campaign, Distribution Lists, Postbox, Organizations, IDM | Governed communication package |
| Function holder receives current and policy-selected historical work and information | IDM, Access, Postbox, Tasks, Records | Postbox reassignment/history tests |
| Administrative worker accesses one familiar eAkte context across exact owned objects | Records and record-source providers | GovOPlaN #42 and Records #8 |
| User invokes common message, template and file actions without leaving the current task | Core shell, Views, Workflow and contributing modules | Task-local action contract and service workspace |
## Idea Preservation Map
| Original idea cluster | Preserved direction |
| --- | --- |
| Time recording, absence, sickness, reintegration, applicant management | Governed workforce-context journey; effective-dated status and privacy profile before module expansion |
| LBV BW interface | Candidate provider-neutral workforce/payroll connector profile |
| Abstract interfaces | Versioned Core contracts with product adapters and explicit source authority |
| Desktop and groupware integration | Standards, connectors, deep launch and synchronization before custom clients |
| GovOPlaN app/mobile-first pages | Responsive shared semantics; native shell only when a proven journey needs device capabilities |
| Video, chat, instant messaging and forum | Optional governed collaboration providers with retention/evidence contracts |
| Somacos Session-style meeting management | Committee-led meeting composition across Calendar, Files, Decisions, Templates, Tasks, Reporting and Ledger |
| Stronger software integration | Integration-first roadmap rule and first full external product connectors |
## Maintenance
When a source idea becomes actionable:
1. link it to a named reference journey or explicit discovery issue;
2. identify the owning module and external authority;
3. create or update the Gitea issue with acceptance criteria;
4. keep live status out of this document;
5. update this register only when the durable interpretation changes.
+193
View File
@@ -0,0 +1,193 @@
# Quick Access And Product Areas
## Purpose
GovOPlaN presents institutional work without requiring ordinary users to
understand the installed package graph. Two complementary projections provide
that experience:
- **product areas** group destinations, objects, work and actions by the
outcome a person recognizes;
- **Quick Access** keeps a small set of task-local tools available without
leaving the current page, case, record or Workflow context.
Technical modules remain the implementation, release and provenance boundary.
Product areas and Quick Access are presentation contracts over those owners;
they do not copy domain state or bypass authorization.
Implementation is tracked by Core #283 and #285, GovOPlaN's product-experience
umbrella, Views, Policy and `govoplan-quick-access`.
The repository and product name is `govoplan-quick-access`, with module id
`quick_access`. `govoplan-qar` was rejected because the abbreviation hides the
purpose in package catalogues, diagnostics, permissions and operations.
## Implementation Status
The first production-shaped slice is implemented:
- Core validates and publishes versioned `product_areas` and
`quick_access_tools` manifest contracts;
- `govoplan-quick-access` derives its live catalogue from installed modules,
persists optimistic-concurrency-protected system, tenant and user profiles,
and resolves blocked, forced, ordered and stale preferences;
- the shell hosts the optional right rail and one composed drawer with keyboard
dismissal, focus return, responsive mobile behavior and full-page fallbacks;
- Tasks, Calendar, Mail, Postbox and Files contribute the first owner-rendered
tools; Mail and Postbox remain separate sections inside Messages;
- immutable View revisions now carry grouped/flat navigation, product-area
order and optional labels. Scoped Views therefore configure product
presentation for system, tenant, group, user and Workflow contexts;
- the expanded left rail groups classified destinations while retaining
Dashboard and every authorized unclassified destination under More tools.
The remaining rollout is classification rather than a missing boundary: other
user-facing modules must announce their product areas and future bounded tools,
reference journeys need browser accessibility evidence, and richer active-object
context should be added only through a separately versioned launch-context
contract. Until classification is complete, authorized unclassified routes
remain visible rather than disappearing.
## Quick Access Boundary
Core owns a versioned contribution contract. Feature modules may register a
tool when they have a useful bounded surface. They do not import Quick Access.
`govoplan-quick-access` owns configuration, effective resolution, ordering,
the right-side rail and its drawer. Views may narrow tools for the current
task. Policy may constrain availability and customization. Access and each
owner's backend remain authoritative.
The initial categories are:
| Category | Typical contributions |
| --- | --- |
| Work | Explicit Tasks, Workflow handoffs, approvals, deadlines and exceptions |
| Calendar | Today/upcoming agenda, event creation and scheduling launch |
| Messages | Mail, function-bound Postbox messages and future governed chat providers |
| Files | Contextual/recent files, attachment selection and upload |
Messages is one shell category but not one data model. Mail, Postbox and future
chat providers retain their channel semantics, custody, policy, audit and
delivery behavior. The drawer identifies the channel where that distinction
matters.
## Contribution Contract
A Quick Access contribution declares:
- a stable id, category and human label;
- icon, order and optional badge/summary provider;
- required permissions and optional dependencies;
- accepted context references and produced return references;
- an owner-rendered bounded WebUI surface and full-page fallback route;
- View surface, help context and availability explanation;
- whether the contribution supports preview, create, select or resume.
The shell passes only bounded references: tenant, acting context, temporal
read context, active task/Workflow, current institutional object, selected
resources and a safe return location. The owner reauthorizes every read and
effect. Credentials, protected content and permission decisions are never
embedded in launch context.
## Effective Configuration
The effective rail is resolved from:
1. installed and enabled modules and their registered contributions;
2. system availability, forced entries and ordering defaults;
3. tenant availability, forced entries and ordering defaults;
4. group and user View/Policy ceilings where configured;
5. the user's enabled categories, entries and ordering;
6. the active View and optional Workflow-step narrowing overlay;
7. current authorization and contribution availability.
Lower scopes may narrow or reorder allowed entries but cannot enable a tool
blocked above them. A forced entry cannot be removed below its source. User
configuration stores stable contribution ids; unavailable or retired ids are
retained as explained stale preferences without rendering broken controls.
Configuration screens derive their available choices from the live registry.
Installing or enabling a contributing module adds its permitted choices;
disabling it removes the runtime tool while preserving harmless preferences.
If Quick Access is absent, contributors behave exactly as before.
## Interaction Model
Desktop uses a narrow right-side rail with at most four initial category
buttons and an overflow when an administrator or user adds more categories.
Selecting a category opens one fixed, owner-neutral drawer. Contributions are
shown inside that drawer as tabs, sections or commands according to the
category contract. The default drawer overlays content so DataGrid and fixed
workspace layouts do not resize unexpectedly; a later explicit pinned mode may
reserve layout width on sufficiently wide screens.
The drawer preserves host-page state, has a deterministic focus return, closes
with Escape, supports keyboard traversal, and provides an explicit full-page
open action. Mobile and narrow layouts use the same category/configuration
semantics in a bottom sheet or compact menu.
## Product Areas
Product areas are stable configurable identities, not repositories. The
recommended baseline is:
- Work;
- Services and Cases;
- Records and Documents;
- Communication;
- Meetings and Decisions;
- Data and Assurance;
- People and Responsibility.
Modules contribute routes, objects, actions, widgets, work sources and help to
one or more areas. Product packages and administrators may define sensible
system and tenant defaults. Views select, order, rename or narrow allowed
areas, and users may personalize them within Policy ceilings. An empty area is
omitted. An area with one destination may open it directly. A multi-destination
area provides a useful work/recent/action surface rather than another menu.
Familiar product nouns such as Calendar, Mail or Files may remain directly
pinned. The objective is not to hide every module name; it is to prevent
repository topology from determining a person's workflow.
## Full Access And Provenance
The existing permission-derived module rail remains available as **All
available tools** for power users and deliberate escape from a focused View.
It contains only currently authorized destinations. Technical module,
capability, provider and package provenance remains visible in administration,
diagnostics, evidence and expandable details.
Search, deep links and help distinguish three states:
- available in the active View;
- authorized but outside the active View, with a temporary escape or View
switch;
- unavailable because of authorization, Policy, configuration or a missing
capability, with an actionable explanation.
## Delivery Order
1. Define Core product-area and Quick Access contracts and validation.
2. Implement `govoplan-quick-access` configuration, effective resolution and
shell capability.
3. Contribute Work, Calendar, Messages and Files bounded surfaces.
4. Add configurable product-area defaults through Views and product packages.
5. Migrate navigation, breadcrumbs, search, errors, documentation, dashboard
and administration toward product terminology.
6. Prove keyboard, focus, responsive, optional-module and reference-journey
behavior before making it the ordinary-user default.
## Acceptance Criteria
- A user can configure allowed Quick Access categories and ordering without
gaining authority.
- System and tenant administrators can make entries available, forced or
unavailable with provenance.
- Mail, Postbox and another future channel can share Messages presentation
while retaining independent state and channel semantics.
- A reference journey can use a bounded tool and return without losing host
state or Workflow context.
- Product areas remain useful under sparse and rich permission sets and under
optional-module permutations.
- All available tools and technical provenance remain deliberately reachable.
+80
View File
@@ -0,0 +1,80 @@
# GovOPlaN Documentation Map
This directory contains cross-repository product, architecture, release, and
operational documentation. The map below defines which document answers which
question. A document not listed as the current status source must not present
volatile repository, issue, release, or maturity counts as current facts.
## Strategy
| Question | Canonical source |
| --- | --- |
| What are the stable ideas and boundaries of the platform? | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) |
| What product outcomes should GovOPlaN pursue? | [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) |
| Which institutional concepts and owners form the target architecture? | [Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md) |
| Which end-to-end proofs should guide implementation? | [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md) |
| What is the reconciled state now? | [Strategy Status](STRATEGY_STATUS.md) |
| Which collected ideas and user stories inform the product direction? | [Product Input Register](PRODUCT_INPUT_REGISTER.md) |
The dated [Strategic Review](STRATEGIC_REVIEW_2026-08-05.md) explains why the
current reset and sequencing were chosen. It is an assessment record, not a
second live status page.
## Product Architecture
| Topic | Canonical source |
| --- | --- |
| Product-facing experience and hiding technical module boundaries | [Product Experience and Module Boundaries](PRODUCT_EXPERIENCE_AND_MODULE_BOUNDARIES.md) |
| Configurable product areas and task-local tools | [Quick Access and Product Areas](QUICK_ACCESS_AND_PRODUCT_AREAS.md) |
| Federation between autonomous installations | [Federated GovOPlaN Architecture](FEDERATED_GOVOPLAN_ARCHITECTURE.md) |
| Institutional digital twin and continuous assurance | [Institutional Digital Twin](INSTITUTIONAL_DIGITAL_TWIN.md) |
| Assisted and non-digital channels | [Assisted and Non-Digital Channels](ASSISTED_AND_NON_DIGITAL_CHANNELS.md) |
| Cross-module temporal, purpose, retention, and institutional-context adoption | `govoplan-core/docs/INFORMATION_GOVERNANCE_ADOPTION.md` |
| eAkte and digital-record ownership | `govoplan-records/docs/EAKTE_ARCHITECTURE.md` |
| Data source, definition, and transformation graph | [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md) |
| Focused task views | [Views Architecture](VIEWS_ARCHITECTURE.md) |
| Shared interface patterns | [Interface Pattern Language](INTERFACE_PATTERN_LANGUAGE.md) |
## Runtime And Delivery
- [Module Contracts and Installs](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Platform Control Plane](PLATFORM_CONTROL_PLANE.md)
- [Installation and Deployment Architecture](INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md)
- [Kubernetes VM Test Lab](KUBERNETES_TEST_LAB.md)
- [Deployment Profiles](DEPLOYMENT_PROFILES.md)
- [Scaling and Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md)
- [Recovery and Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md)
- [Recovery Ledger Adoption](RECOVERY_LEDGER_ADOPTION.md)
- [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md)
## Evidence And Snapshots
These documents are intentionally dated or pinned. They may remain useful even
after the product changes, but they do not override `STRATEGY_STATUS.md`.
- [Capability and Infrastructure Fit Assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md), pinned to the 2026-07-22 Campaign composition
- [Strategic Review 2026-08-05](STRATEGIC_REVIEW_2026-08-05.md)
- [Backup and Restore Evidence](BACKUP_AND_RESTORE_EVIDENCE.md)
- [Production Target Handoff](PRODUCTION_TARGET_HANDOFF.md)
- [Target Maturity Evidence Runbook](TARGET_MATURITY_EVIDENCE_RUNBOOK.md)
Machine-readable schemas and evidence files belong beside the document that
defines them. Generated inventories belong in `audit-reports/` and should not
be edited manually.
## Maintenance Rules
1. Gitea issues are the only live work-state source.
2. `STRATEGY_STATUS.md` is the only prose reconciliation of current portfolio
state. Refresh it from manifests, inventories, tests, and Gitea; do not copy
its counts into durable architecture pages.
3. Durable documents state decisions, invariants, ownership, and acceptance
gates. They link to status and issues for implementation depth.
4. Dated assessments retain their original composition and conclusion. Add a
snapshot notice rather than silently updating their claims.
5. Module-specific behavior and user/admin documentation remain in the owning
repository. Meta documentation defines cross-module outcomes and contracts.
6. A new strategy document must replace, narrow, or link an existing source;
it must not introduce a parallel roadmap.
7. The Product Input Register preserves external idea and story notes, but only
Gitea issues carry live priority, ownership, and implementation state.
+57
View File
@@ -18,6 +18,40 @@ delivered as small, reviewable, green increments and is complete only when its
user journey, failure behavior, documentation, and operator evidence work in a user journey, failure behavior, documentation, and operator evidence work in a
pinned composition. pinned composition.
## 2026 outcome reset
Repository completion is not product completion. From 2026-08-05 onward, work
is accepted primarily through three maintained real-life journeys:
1. **Governed communication:** select accountable recipients, prepare content
and attachments, approve, deliver through Mail and/or a function-bound
Postbox, reconcile uncertain outcomes, and file the evidence.
2. **Inclusive service-to-decision:** accept a request through a digital or
assisted channel, establish identity and purpose, guide the case through
human and automatic work, decide, notify, and file the resulting eAkte.
3. **Monthly data and sanctions:** acquire immutable source snapshots, validate
and reconcile them interactively, preserve decisions and lineage, produce
reports and files, and deliver the accepted result through Campaign.
The staged program below remains the architectural build order. These journeys
are the acceptance lens across those stages. Every significant feature should
identify the journey it improves, or provide security, operability, recovery,
accessibility, or usability evidence that those journeys require. Work that
does neither stays in the backlog until a concrete consumer exists.
The Records vertical now supplies the journey's native file plan, immutable
record and item revisions, chronology, close/reopen, retention calculation,
holds, appraisal, independent disposition approval, recovery-ledger evidence,
and archive-neutral package simulation. Forms Runtime, Cases, and Decisions
expose exact, permission-rechecked source revisions for explicit filing, and
all three contribute metadata-only native Search projections that can be
rebuilt from authoritative state. The executable fixtures prove those native
transitions without claiming archival custody. A persisted Workflow Engine
handoff is now reloaded through the Tasks aggregation surface and remains
visible until the authoritative Workflow transition completes. The journey
still needs pinned-composition reconstruction evidence and one target-tested
archive profile.
## Why this sequence ## Why this sequence
The sequence grows one connected product rather than advancing repositories in The sequence grows one connected product rather than advancing repositories in
@@ -86,6 +120,23 @@ journey needs and supplies contracts shared by all five stages.
execution. Database, broker, cache, and worker channels are constrained by execution. Database, broker, cache, and worker channels are constrained by
deployment network policy and authenticated transport rather than treated as deployment network policy and authenticated transport rather than treated as
tenant connector profiles. tenant connector profiles.
10. **Information governance.** Temporal browsing, purpose-aware access,
retention/legal-hold behavior, and institutional acting context are applied
to every owned object type. Historical reads use current authorization.
Module manifests state `contract_only`, `partial`, `enforced`, or
`not_applicable` adoption with evidence; supported maturity is blocked until
every applicable dimension is enforced.
11. **Durable human work.** Tasks aggregates explicit work and module-owned
attention items; Workflow Engine persists process state and handoffs;
Notifications attracts attention; Views focuses the relevant surfaces.
Leaving or refreshing the browser never becomes the only record that work
remains unfinished.
12. **Task-local tools.** Mail, Postbox, Templates, Files, and other common
actions may contribute bounded launch surfaces with return context. The
shell and Workflow compose them without copying their data or validation.
The optional Quick Access module presents configurable Work, Calendar,
Messages and Files categories; system/tenant/user settings and View/Policy
ceilings resolve their availability and ordering.
## Documentation contract for every reference stage ## Documentation contract for every reference stage
@@ -108,6 +159,9 @@ Every demonstrated journey provides:
provenance, evidence, retention, and destructive actions. provenance, evidence, retention, and destructive actions.
- **Acceptance view:** runnable examples, expected results, failure injection, - **Acceptance view:** runnable examples, expected results, failure injection,
and release gates. and release gates.
- **Channel and records view:** assisted/non-digital intake and output,
representation, provenance, filing, retention, legal hold, and archive
consequences where the journey creates evidence or a record.
The Docs module selects and links these views according to installed The Docs module selects and links these views according to installed
capabilities and actor context. Feature repositories remain the source of capabilities and actor context. Feature repositories remain the source of
@@ -436,6 +490,9 @@ or the external editor the document-lifecycle owner.
link, callback, webhook, file, identity, or data row. link, callback, webhook, file, identity, or data row.
- Do not claim a stage complete from local unit tests. Use pinned composition, - Do not claim a stage complete from local unit tests. Use pinned composition,
target integration, failure drills, adaptive docs, and operator evidence. target integration, failure drills, adaptive docs, and operator evidence.
- Do not claim a module complete while its relevant information-governance
dimensions remain `contract_only` or while the reference journey lacks an
assisted-channel and records outcome where those are applicable.
- A later stage may prototype contracts while the preceding gate is being - A later stage may prototype contracts while the preceding gate is being
proven, but it may not redefine an owning module's boundary by convenience. proven, but it may not redefine an owning module's boundary by convenience.
+25 -4
View File
@@ -3,6 +3,8 @@
For the exact external handoff, least-privilege collector permissions and live For the exact external handoff, least-privilege collector permissions and live
two-node acceptance procedure, see two-node acceptance procedure, see
[`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md). [`PRODUCTION_TARGET_HANDOFF.md`](PRODUCTION_TARGET_HANDOFF.md).
For a reproducible local or multi-hypervisor libvirt/K3s target, use
[`KUBERNETES_TEST_LAB.md`](KUBERNETES_TEST_LAB.md).
## Implemented Contract ## Implemented Contract
@@ -74,6 +76,7 @@ python tools/deployment/govoplan-deploy.py render-kubernetes \
--namespace govoplan \ --namespace govoplan \
--secret-name govoplan-runtime \ --secret-name govoplan-runtime \
--tls-secret-name govoplan-tls \ --tls-secret-name govoplan-tls \
--s3-ca-secret-name govoplan-s3-ca \
--ingress-class-name nginx \ --ingress-class-name nginx \
--output /srv/govoplan/default/kubernetes.json --output /srv/govoplan/default/kubernetes.json
``` ```
@@ -90,11 +93,25 @@ command prints the exact required key contract. Review the generated
`FORWARDED_ALLOW_IPS` value and replace it with the exact ingress-proxy network `FORWARDED_ALLOW_IPS` value and replace it with the exact ingress-proxy network
before production use. before production use.
When an external S3 endpoint is signed by a private CA, create the optional CA
Secret with a `ca.crt` key and pass `--s3-ca-secret-name`. The renderer mounts
that Secret read-only and sets `AWS_CA_BUNDLE` for API, worker, scheduler,
migration and database-wait containers. It does not disable certificate
verification or replace the WebUI trust store.
The generated containers run as non-root with a read-only root filesystem and The generated containers run as non-root with a read-only root filesystem and
an ephemeral `/tmp`. Runtime Deployments wait for the exact configured database an ephemeral `/tmp`. Celery Beat keeps its replaceable schedule database there;
migration heads before starting. The API exposes `/health/ready`, which fails durable schedule definitions remain in shared state. The WebUI resolves its
while that API node is draining or cannot prove its runtime-coordination configured API Service when the container starts, so Kubernetes deployments do
heartbeat. not inherit the Compose-only `load-balancer` hostname. Runtime Deployments wait
for the exact dependency-resolved database migration heads before starting.
The API exposes `/health/ready`, which fails while that API node is draining or
cannot prove its runtime-coordination heartbeat.
Replicated API, WebUI, and worker Deployments use a hard hostname-spread
constraint scoped to the current pod-template hash. A rollout therefore keeps
each replica set distributed across independently schedulable nodes instead of
allowing all replacement pods to settle on one node after the old set exits.
## Runtime Coordination ## Runtime Coordination
@@ -253,6 +270,10 @@ record under the installation evidence directory and never retains the API key.
Use `--exercise-api-pod-loss` in an approved drill window to delete one API pod, Use `--exercise-api-pod-loss` in an approved drill window to delete one API pod,
observe the public readiness path continuously, and record its replacement. observe the public readiness path continuously, and record its replacement.
Generated API workloads use a ten-second pre-stop drain so Kubernetes can remove
the terminating endpoint from ingress and service routing before Uvicorn exits.
Do not remove or shorten this drain without repeating the public-path pod-loss
test against the target ingress controller and network implementation.
This proves the bounded stateless-node-loss slice only. Session continuity, This proves the bounded stateless-node-loss slice only. Session continuity,
accepted-job redelivery, state-service failover, and coordinated restore remain accepted-job redelivery, state-service failover, and coordinated restore remain
separate target exercises whose signed evidence is governed by separate target exercises whose signed evidence is governed by
+132
View File
@@ -0,0 +1,132 @@
# Strategic Review - 2026-08-05
## Assessment
GovOPlaN has not lost its central direction. The architecture now expresses a
coherent institutional governance platform, but architecture and repository
breadth have advanced faster than complete, usable outcomes. The immediate
need is convergence: fewer simultaneous fronts, stronger cross-cutting
adoption, and end-to-end reference journeys that non-developers can complete.
This is a dated review. Current status belongs in
[Strategy Status](STRATEGY_STATUS.md); stable direction belongs in
[Platform Core Ideas](PLATFORM_CORE_IDEAS.md).
## What Is Already Strong
- A modular runtime with manifests, capabilities, interfaces, migrations,
optional integrations, signed releases, and permutation checks.
- Explicit institutional semantics for identity, representation,
organization, function, mandate, service, case, party, approval, decision,
evidence, and record references.
- Governed communication foundations spanning Campaign, Mail, Files, Postbox,
Addresses, Distribution Lists, Templates, Audit, and Policy.
- Governed data foundations spanning Connectors, Datasources, Dataflow,
Reporting, Search, and immutable provenance.
- Bitemporal browsing, views, contextual documentation, action/effect
contracts, event delivery, recovery ledgers, and stateless deployment
contracts.
- A credible deployment and release foundation with signed artifacts and
reproducible composition evidence.
## Where The Program Veered
### Repository breadth preceded product proof
Logical modularity often became a repository before a reference journey proved
that an independent release boundary was required. Scaffolds are useful as
ownership markers, but their number makes the product appear broader and more
complete than its supported outcomes.
### Foundations outran reference gates
Later-stage contracts such as federation, encryption, formal governance,
deployment evidence, and broad module metadata were developed while basic
human-work and records journeys remained incomplete. Those foundations are not
wasted; they now need to be consumed by a small number of demonstrable
products.
### The module graph leaked into the experience
Navigation, routes, administration, errors, documentation, and configuration
often present module names and package structure directly. This is appropriate
for operators, but ordinary users should see work, services, records, and
outcomes.
This is not primarily a rail-length or scrolling problem. Sparse permissions
already reduce navigation and the complete technical rail remains useful for
power users. The correction is configurable product areas, task-focused Views
and a bounded Quick Access rail, while preserving deliberate access to every
authorized tool and technical provenance. The accepted design is maintained in
[Quick Access And Product Areas](QUICK_ACCESS_AND_PRODUCT_AREAS.md).
### Status became duplicated
Roadmaps, target architecture, fit assessments, issue comments, and release
documents each contained partial implementation snapshots. Their stable
decisions remain valuable, but volatile counts and maturity claims diverged.
### Too much work remained active simultaneously
The issue portfolio had many high-priority and in-progress items without
milestones. This reduces the signal of both labels and roadmap order and makes
completion harder to demonstrate.
## Where GovOPlaN Has Not Gone Far Enough
1. No composition has yet crossed the full `reference_ready` gate.
2. The human-work spine is incomplete: work queues, tasks, handoffs, deadlines,
reminders, escalation, and resumption need a coherent user experience.
3. Records and document management remain too shallow for a public-sector
operating platform.
4. Real target integrations and GovOPlaN-to-GovOPlaN federation are not yet
proven.
5. Temporal browsing, purpose-aware access, retention, and institutional
context exist as contracts but are not adopted uniformly by domain reads
and effects.
6. German completeness, contextual help, accessibility, responsive behavior,
and browser-level journey testing are not yet release gates everywhere.
7. Multi-host, backup/restore, provider interoperability, and independent
signed target evidence still require real environments and operators.
## Important Omissions
- a named first institution, bounded users, volumes, and operating constraints;
- measurable usability outcomes, not only functional tests;
- installable sector packages and migration/exit demonstrations;
- support, upgrade, deprecation, and LTS promises;
- complete assisted, paper, telephone, and in-person channel handling;
- a native eAkte/records model that can also overlay an external DMS or archive.
## Opportunities Beyond The Original Idea
- an institutional digital twin that exposes responsibilities, dependencies,
obligations, services, work, data, controls, and change impact over time;
- continuous assurance that evaluates controls and evidence as work happens;
- process mining and conformance analysis over governed event histories;
- federated product packages and inter-institution case/evidence exchange;
- accountable assistance that drafts and explains without obscuring authority;
- public evidence chains that disclose decisions and provenance without
exposing protected source data.
## Recommended Reset
1. Freeze new repositories unless a real journey proves an independent owner,
release lifecycle, security boundary, or optional installation need.
2. Use one generated maturity/status dashboard and one current status document.
3. Complete governed communication and function-bound Postbox against a real
target.
4. Complete the monthly-data journey, then sanctions screening on the same
data foundations.
5. Complete one browser-driven service-to-decision journey, including assisted
intake and records.
6. Make eAkte/records the next major product-depth program.
7. Tie feature work to a reference journey, a security/recovery gate, or a
measured usability defect.
## Success Criterion
The reset succeeds when a public institution can install a signed composition,
configure a named procedure, complete it through digital and assisted channels,
connect an external source, reconstruct the authority and evidence, recover it
after failure, and transfer or retire it without custom code.
+132
View File
@@ -0,0 +1,132 @@
# GovOPlaN Strategy Status
## Status Record
| Field | Value |
| --- | --- |
| Reconciled on | 2026-08-06 |
| Source scope | Local workspace manifests, source inventory, focused journey checks, signed release evidence, and live Gitea issue state |
| Stable direction | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) and [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) |
| Collected product input | [Product Input Register](PRODUCT_INPUT_REGISTER.md) |
| Delivery source | Gitea issues |
This is the only prose source for current cross-product status. It is a
reconciliation, not a release certification. Module manifests and target
evidence remain authoritative for specific maturity claims.
## Portfolio Snapshot
- 66 source module manifests were loadable and architecture-declared.
- 48 modules declared `vertical_slice`; 18 declared `scaffold`.
- No module declared `reference_ready`, `supported`, or `lts`.
- The live portfolio had 133 open issues, including 37 priority-P1 items.
- 118 open issues had no milestone, so issue labels do not yet express a
reliable completion sequence on their own.
- Three product package manifests existed: governed communication, governed
data and assurance, and service to decision. None had crossed the complete
target-evidence gate.
These counts are dated. Refresh them rather than copying them into another
document.
## Interface And Contract Evidence
The 2026-08-06 source inventory found:
- 1,307 UI fields and 1,291 UI actions;
- 8,157 stable interface declarations with no duplicate IDs;
- 43 frontend routes and 920 backend endpoints;
- no public WebUI surfaces missing runtime declarations;
- no stale runtime route declarations;
- no unclassified endpoint without a static UI reference;
- all 1,307 fields with a resolvable F1 context; 1,143 remain candidates for
richer field-specific content beyond page/module fallback;
- German (`de`) as the complete reference locale and no used key missing from
the required German or English catalogs;
- 3 module information-governance dimensions classified as `enforced`, 1 as
`partial`, and 260 as `contract_only`.
This is an honest platform-wide baseline, not a claim that temporal,
purpose, retention, and institutional-context adoption is complete.
## Credible Current Outcomes
### Platform foundation
Module discovery, optional dependency validation, migrations, shared WebUI,
tenant and access foundations, signed catalogs/packages, event delivery,
recovery contracts, contextual help, views, temporal titlebar context, and
stateless-runtime patterns are implemented and tested at varying depths.
### Governed communication
Campaign authoring, recipient data, attachments, templates, mail profiles,
mock/real delivery paths, audit evidence, reporting, distribution-list
composition, and optional Postbox delivery form the deepest product cluster.
Target provider, accessibility, recovery, and high-volume evidence still
prevent a reference-ready claim.
### Institutional service and decision
Services, Forms, Forms Runtime, Cases, Parties, Mandates, Approvals, Committee,
Voting, Decisions, Portal, Postbox, and Audit have an executable service-to-
decision fixture. Public and invitation intake can retain Files-backed
evidence; Forms submissions, Cases, and formal Decisions can be explicitly
filed as exact eAkte source revisions and reconstructed through permission-
rechecked native Search projections. A durable Workflow Engine handoff now
survives session restart and appears through the Tasks work inbox until the
authoritative transition completes. Browser-complete assisted intake, broader
work projections and escalation, production identity and delivery, a named
archive profile, and target evidence remain.
### Governed data and assurance
Connectors, Datasources, Dataflow, Reporting, Search, Policy, Risk Compliance,
and Workflow provide source governance, immutable snapshots, transformation,
quality, semantic reporting, and provenance foundations. The monthly-data and
sanctions compositions now prove immutable connector snapshots, pinned
Dataflow publication, Risk Compliance review, and rescreening in process. The
journeys still need target connector profiles, complete interactive
reconciliation, governed export/delivery, and browser-level handoff evidence.
## Material Gaps
| Gap | Consequence | Next proof |
| --- | --- | --- |
| No reference-ready product package | The platform cannot yet make a bounded supported-product claim | Complete one named target composition and evidence bundle |
| Human-work spine is only an MVP | Tasks aggregates explicit work plus Workflow, Approval, and unread Postbox projections, but broad domain coverage, deadline escalation, assignment lifecycle, and focused product UX remain | Extend source providers through the three reference journeys and prove overdue/reassignment behavior in browser tests |
| Records/eAkte target integration incomplete | Native lifecycle, retention, holds, approval, recovery, and transfer simulation are implemented, but real custody is not proved | Target-test one archive/xdomea profile and complete the assisted reference journey |
| Cross-cutting governance adoption uneven | Historical and purpose-sensitive behavior varies by module | Enforced adoption declarations and route/query/effect migration |
| Explicit help/accessibility depth incomplete | German/reference and F1 association gates now pass, but generic fallback remains too common | High-risk German help content and browser/a11y matrix |
| Real federation absent | Cross-institution exchange remains connector-specific | Paired-instance signed exchange and reconciliation proof |
| External production evidence incomplete | Scale, restore, interoperability and custody claims remain conditional | Real target drills and independent signed evidence |
## Active Strategic Order
1. Establish German, help, temporal, purpose, retention, and institutional
context as enforceable platform quality contracts.
2. Complete governed communication and Postbox against a named target.
3. Complete the monthly-data flow and use it as the data foundation for
sanctions screening.
4. Complete the browser and resumable-work proof for the digital and assisted
service-to-decision journey with its existing exact eAkte filing contracts.
5. Complete native PostgreSQL search coverage for remaining journey-owned
objects and prove reauthorization and reindex operations at target volume;
keep OpenSearch optional. Communication, Records, service-to-decision,
Dataflow, Reporting, Risk Compliance, and Datasource catalogue sources now
exist.
6. Prove one external product connector and one GovOPlaN federation exchange.
7. Finish multi-host, restore, provider, accessibility, and independent signed
target evidence before increasing maturity claims.
## Refresh Procedure
Refresh this page only from evidence:
1. run `tools/checks/check-manifest-shapes.py`;
2. run `tools/inventory/platform-interface-inventory.py --strict
--strict-declarations --strict-endpoints`;
3. run the selected reference-journey checks;
4. inspect signed release and target evidence;
5. query live Gitea issue/milestone state;
6. update the dated values and material gaps here;
7. retain prior assessments as dated evidence rather than rewriting them.
@@ -4,7 +4,8 @@
> As a system administrator, I can execute one shell command that downloads a > As a system administrator, I can execute one shell command that downloads a
> verified GovOPlaN distribution and starts a completely configured Core control > verified GovOPlaN distribution and starts a completely configured Core control
> plane without optional modules. In the WebUI I can browse compatible signed > plane with the official package directory available but only the protected
> baseline active. In the WebUI I can browse compatible signed
> module releases, select the modules for this installation, and follow every > module releases, select the modules for this installation, and follow every
> download, validation, migration, installation, activation, and health-check > download, validation, migration, installation, activation, and health-check
> step. When an update is available, I can review its impact and confirm it. > step. When an update is available, I can review its impact and confirm it.
@@ -25,7 +26,8 @@ The canonical backlog item is
- **Core control plane:** the smallest bootable distribution: Core API, Core - **Core control plane:** the smallest bootable distribution: Core API, Core
WebUI, PostgreSQL, Redis, installer worker, migration runner, and durable WebUI, PostgreSQL, Redis, installer worker, migration runner, and durable
storage configuration. No optional GovOPlaN module package is installed. storage configuration. An immutable image may carry the full verified package
profile, but optional modules are not active or tenant-entitled by implication.
- **Bootstrap administrator:** a single-use, time-limited installation identity - **Bootstrap administrator:** a single-use, time-limited installation identity
that may access only first-run and module-lifecycle functions. It is retired that may access only first-run and module-lifecycle functions. It is retired
when the selected identity/access configuration becomes healthy. when the selected identity/access configuration becomes healthy.
@@ -55,7 +57,9 @@ The canonical backlog item is
5. It prints the local URL and one-time bootstrap credential. Re-running the 5. It prints the local URL and one-time bootstrap credential. Re-running the
command is idempotent and shows or repairs the existing installation rather command is idempotent and shows or repairs the existing installation rather
than creating another identity or database. than creating another identity or database.
6. No optional module is installed or enabled at this point. 6. Only the protected baseline is enabled. Installed package availability does
not grant permissions, tenant entitlement, View visibility, or capability
opt-in.
### Module selection, installation, and update ### Module selection, installation, and update
@@ -160,19 +164,24 @@ Implementation status as of the current source tree:
ledger and deployment operation journal. Automatic database backup and broad ledger and deployment operation journal. Automatic database backup and broad
adoption by module-owned external effects remain open work. adoption by module-owned external effects remain open work.
1. **Reproducible Core-only distribution.** Publish pinned multi-architecture 1. **Reproducible Core-baseline distribution.** Publish pinned multi-architecture
images, signed distribution manifest, Core-only Compose profile, bootstrap full-package images, signed distribution manifest, Core-baseline Compose profile, bootstrap
preflight, generated secrets, readiness, and idempotent rerun/repair. preflight, generated secrets, readiness, and idempotent rerun/repair.
2. **First-run control plane.** Add the restricted bootstrap administrator, 2. **First-run control plane.** Add the restricted bootstrap administrator,
one-time enrollment, initial catalog/keyring configuration, and retirement one-time enrollment, initial catalog/keyring configuration, and retirement
after durable administrator access is established. after durable administrator access is established.
3. **Read-only online module directory.** Move the existing catalog and module 3. **Read-only online module directory (implemented foundation).** Admin falls
directory contracts into the installed Core WebUI with compatibility, back to the signed public stable directory, presents installed/update state,
provenance, release-note, and update-state presentation. searchable availability/blocker filters, immutable source/artifact
4. **Durable module plan and install.** Reuse the existing installer queue, provenance, configuration requirements, release notes, and technical
locks, signed-package validator, rollback drill, and run evidence behind a compatibility. Withdrawn releases remain visible but cannot be planned.
plan/confirm/progress UI. Add initial catalog-entry synthesis and artifact Operator-configured catalogs remain an explicit override.
acquisition where the current release console still assumes local sources. 4. **Durable module plan and install (implemented local boundary).** Catalog
selection creates a reviewed plan; the installer queue, lock, preflight,
maintenance gate, digest-verified artifact cache, rollback drill, and run
evidence remain separate from the API process. Shared deployments convert
the same intent into a new immutable release composition instead of mutating
one replica.
5. **Safe module update.** Add drain/maintenance coordination, backup gate, 5. **Safe module update.** Add drain/maintenance coordination, backup gate,
migration compatibility window, reconnectable progress, health verification, migration compatibility window, reconnectable progress, health verification,
retry/recovery, and update notification. retry/recovery, and update notification.
@@ -194,7 +203,8 @@ Implementation status as of the current source tree:
## Explicit non-goals for the first distribution slice ## Explicit non-goals for the first distribution slice
- Shipping optional modules in the Core image. - Activating, tenant-entitling, or exposing optional modules merely because the
immutable image carries their verified packages.
- Exporting secrets or production business data with configuration. - Exporting secrets or production business data with configuration.
- Pretending every schema migration can be reversed automatically. - Pretending every schema migration can be reversed automatically.
- Building a proprietary orchestrator instead of supporting Compose and a - Building a proprietary orchestrator instead of supporting Compose and a
+16
View File
@@ -7,6 +7,11 @@ responsibility, or workflow step. A View can reduce the visible modules,
navigation entries, routes, page sections, and commands to the interface navigation entries, routes, page sections, and commands to the interface
needed for the current job. needed for the current job.
Views also project configurable product areas and Quick Access contributions.
They may select, order, rename or hide permitted presentation identities but
do not move ownership or merge Mail, Postbox, Files, Calendar, Tasks or other
domain state.
Views are optional. If `govoplan-views` is not installed or enabled, the normal Views are optional. If `govoplan-views` is not installed or enabled, the normal
permission-derived interface remains unchanged. permission-derived interface remains unchanged.
@@ -119,6 +124,12 @@ Implemented in the initial Views slice:
prevention prevention
- surface declarations for every currently installed module that contributes a - surface declarations for every currently installed module that contributes a
WebUI, including finer-grained shared administration and settings surfaces WebUI, including finer-grained shared administration and settings surfaces
- immutable presentation settings for grouped or flat navigation, product-area
order and product-area labels; the shell resolves these settings through the
same system, tenant, group, user and Workflow-aware View projection
- live product-area identities from module manifests, with authorized
unclassified destinations retained under More tools during incremental
adoption
Still intentionally separate: Still intentionally separate:
@@ -127,6 +138,11 @@ Still intentionally separate:
- read-only and layout-replacement projections beyond the version `1` - read-only and layout-replacement projections beyond the version `1`
visible/hidden contract visible/hidden contract
Quick Access ordering and availability remain owned by
`govoplan-quick-access`; Views only narrow its declared surfaces for the active
task. Neither contract permits arbitrary layout or styling. See
`docs/QUICK_ACCESS_AND_PRODUCT_AREAS.md` in the meta repository.
## Gitea Work Packages ## Gitea Work Packages
- `govoplan#17`: task-focused Views user story - `govoplan#17`: task-focused Views user story
+12
View File
@@ -353,6 +353,12 @@
"description": "GovOPlaN Projects module behavior or integration.", "description": "GovOPlaN Projects module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/records",
"color": "0052cc",
"description": "GovOPlaN Records and eAkte lifecycle behavior or integration.",
"exclusive": false
},
{ {
"name": "module/reporting", "name": "module/reporting",
"color": "c2e0c6", "color": "c2e0c6",
@@ -365,6 +371,12 @@
"description": "GovOPlaN Risk Compliance module behavior or integration.", "description": "GovOPlaN Risk Compliance module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/quick-access",
"color": "c5def5",
"description": "GovOPlaN configurable task-local Quick Access behavior and integrations.",
"exclusive": false
},
{ {
"name": "module/search", "name": "module/search",
"color": "bfdadc", "color": "bfdadc",
+69 -67
View File
@@ -4,82 +4,84 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "govoplan" name = "govoplan"
version = "0.1.15" version = "0.1.18"
description = "Developer convenience package for a versioned GovOPlaN composition" description = "Developer convenience package for a versioned GovOPlaN composition"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" } license = { text = "AGPL-3.0-or-later" }
dependencies = [ dependencies = [
"govoplan-core[server]==0.1.15", "govoplan-core[server]==0.1.18",
"govoplan-tenancy==0.1.15", "govoplan-tenancy==0.1.18",
"govoplan-organizations==0.1.15", "govoplan-organizations==0.1.18",
"govoplan-identity==0.1.15", "govoplan-identity==0.1.18",
"govoplan-idm==0.1.15", "govoplan-idm==0.1.18",
"govoplan-access==0.1.15", "govoplan-access==0.1.18",
"govoplan-admin==0.1.15", "govoplan-admin==0.1.18",
"govoplan-policy==0.1.15", "govoplan-policy==0.1.18",
"govoplan-audit==0.1.15", "govoplan-audit==0.1.18",
"govoplan-dashboard==0.1.15", "govoplan-dashboard==0.1.18",
"govoplan-files==0.1.15", "govoplan-files==0.1.18",
"govoplan-mail==0.1.15", "govoplan-mail==0.1.18",
"govoplan-campaign==0.1.15", "govoplan-campaign==0.1.18",
"govoplan-calendar==0.1.15", "govoplan-calendar==0.1.18",
"govoplan-docs==0.1.15", "govoplan-docs==0.1.18",
"govoplan-ops==0.1.15", "govoplan-ops==0.1.18",
] ]
[project.optional-dependencies] [project.optional-dependencies]
full = [ full = [
"govoplan-addresses==0.1.15", "govoplan-addresses==0.1.18",
"govoplan-approvals==0.1.15", "govoplan-approvals==0.1.18",
"govoplan-assets==0.1.15", "govoplan-assets==0.1.18",
"govoplan-booking==0.1.15", "govoplan-booking==0.1.18",
"govoplan-cases==0.1.15", "govoplan-cases==0.1.18",
"govoplan-certificates==0.1.15", "govoplan-certificates==0.1.18",
"govoplan-committee==0.1.15", "govoplan-committee==0.1.18",
"govoplan-connectors==0.1.15", "govoplan-connectors==0.1.18",
"govoplan-consultation==0.1.15", "govoplan-consultation==0.1.18",
"govoplan-contracts==0.1.15", "govoplan-contracts==0.1.18",
"govoplan-dataflow==0.1.15", "govoplan-dataflow==0.1.18",
"govoplan-datasources==0.1.15", "govoplan-datasources==0.1.18",
"govoplan-decisions==0.1.15", "govoplan-decisions==0.1.18",
"govoplan-dist-lists==0.1.15", "govoplan-dist-lists==0.1.18",
"govoplan-encryption==0.1.15", "govoplan-encryption==0.1.18",
"govoplan-evaluation==0.1.15", "govoplan-evaluation==0.1.18",
"govoplan-facilities==0.1.15", "govoplan-facilities==0.1.18",
"govoplan-forms==0.1.15", "govoplan-forms==0.1.18",
"govoplan-forms-runtime==0.1.15", "govoplan-forms-runtime==0.1.18",
"govoplan-grants==0.1.15", "govoplan-grants==0.1.18",
"govoplan-helpdesk==0.1.15", "govoplan-helpdesk==0.1.18",
"govoplan-identity-trust==0.1.15", "govoplan-identity-trust==0.1.18",
"govoplan-inspections==0.1.15", "govoplan-inspections==0.1.18",
"govoplan-learning==0.1.15", "govoplan-learning==0.1.18",
"govoplan-mandates==0.1.15", "govoplan-mandates==0.1.18",
"govoplan-notifications==0.1.15", "govoplan-notifications==0.1.18",
"govoplan-parties==0.1.15", "govoplan-parties==0.1.18",
"govoplan-permits==0.1.15", "govoplan-permits==0.1.18",
"govoplan-poll==0.1.15", "govoplan-poll==0.1.18",
"govoplan-portal==0.1.15", "govoplan-portal==0.1.18",
"govoplan-postbox==0.1.15", "govoplan-postbox==0.1.18",
"govoplan-procurement==0.1.15", "govoplan-procurement==0.1.18",
"govoplan-projects==0.1.15", "govoplan-projects==0.1.18",
"govoplan-records==0.1.15", "govoplan-quick-access==0.1.18",
"govoplan-reporting==0.1.15", "govoplan-records==0.1.19",
"govoplan-resources==0.1.15", "govoplan-reporting==0.1.18",
"govoplan-rest==0.1.15", "govoplan-resources==0.1.18",
"govoplan-risk-compliance==0.1.15", "govoplan-rest==0.1.18",
"govoplan-scheduling==0.1.15", "govoplan-risk-compliance==0.1.18",
"govoplan-search==0.1.15", "govoplan-scheduling==0.1.18",
"govoplan-services==0.1.15", "govoplan-search==0.1.18",
"govoplan-soap==0.1.15", "govoplan-services==0.1.18",
"govoplan-templates==0.1.15", "govoplan-soap==0.1.18",
"govoplan-tickets==0.1.15", "govoplan-tasks==0.1.19",
"govoplan-transparency==0.1.15", "govoplan-templates==0.1.18",
"govoplan-views==0.1.15", "govoplan-tickets==0.1.18",
"govoplan-voting==0.1.15", "govoplan-transparency==0.1.18",
"govoplan-wiki==0.1.15", "govoplan-views==0.1.18",
"govoplan-workflow==0.1.15", "govoplan-voting==0.1.18",
"govoplan-workflow-engine==0.1.15", "govoplan-wiki==0.1.18",
"govoplan-workflow==0.1.18",
"govoplan-workflow-engine==0.1.18",
] ]
[project.urls] [project.urls]
@@ -28,5 +28,7 @@ The artifact remains a `product` package. Promotion to `reference` requires:
access; and access; and
- version-pinned user and administrator documentation. - version-pinned user and administrator documentation.
Optional Notifications, Portal, Reporting, and Workflow Engine integrations do Optional Notifications, Portal, Reporting, Tasks, and Workflow Engine
not change the package boundary when absent. integrations do not change the package boundary when absent. When Tasks is
present, acknowledgement, reconciliation, and operator intervention remain
owned by their source modules and are projected into the common work inbox.
@@ -21,6 +21,7 @@
{"module_id": "notifications"}, {"module_id": "notifications"},
{"module_id": "portal"}, {"module_id": "portal"},
{"module_id": "reporting"}, {"module_id": "reporting"},
{"module_id": "tasks"},
{"module_id": "workflow_engine"} {"module_id": "workflow_engine"}
], ],
"evidence": [ "evidence": [
@@ -12,7 +12,7 @@ review.
1. Register a typed datasource with source authority, purpose, classification, 1. Register a typed datasource with source authority, purpose, classification,
owner, freshness, and correction policy. owner, freshness, and correction policy.
2. Acquire or upload an immutable source state. 2. Acquire or upload an immutable source state.
3. execute a versioned flow and retain intermediate materializations and 3. Execute a versioned flow and retain intermediate materializations and
provenance; provenance;
4. publish a report or decision input against exact source and flow revisions; 4. publish a report or decision input against exact source and flow revisions;
5. link obligation, governed object, risk, control, evidence, finding, 5. link obligation, governed object, risk, control, evidence, finding,
@@ -37,5 +37,28 @@ The artifact remains a `product` package. Promotion to `reference` requires:
aggregate disclosure; and aggregate disclosure; and
- version-pinned user and administrator documentation. - version-pinned user and administrator documentation.
Optional Connectors, Files, Notifications, and Workflow Engine integrations Optional Connectors, Files, Notifications, Tasks, and Workflow Engine
must remain capability-based and absence-safe. integrations must remain capability-based and absence-safe. Tasks may present
review and recovery handoffs, but Dataflow and Risk Compliance remain the
authoritative owners of run and screening state.
## Executable evidence
- `tools/checks/check-datasource-composition.py` composes Connector snapshots,
governed Datasources, queued Dataflow execution, frozen publication,
idempotent replay, and recovery evidence.
- `govoplan-dataflow/fixtures/golden/monthly-reconciliation` pins synthetic
monthly inputs, stable reconciliation hashes, reviewed decisions, expected
output, source fingerprints, and output hashes.
- `tools/checks/check-sanctions-screening-composition.py` composes an immutable
Connector acquisition, idempotent Risk Compliance import and screening,
independent disposition, a cleared gate, changed-source invalidation, and
the rescreening queue through the registered versioned capabilities.
- `govoplan-dataflow/fixtures/golden/sanctions-screening` independently proves
the deterministic normalization and matching graph with exact expected
output.
These checks use synthetic data and run without network access. They prove the
module contracts and durable state transitions; they do not replace the
deployment, security, privacy, accessibility, and operator evidence still
listed above.
@@ -21,6 +21,7 @@
{"module_id": "connectors"}, {"module_id": "connectors"},
{"module_id": "files"}, {"module_id": "files"},
{"module_id": "notifications"}, {"module_id": "notifications"},
{"module_id": "tasks"},
{"module_id": "workflow_engine"} {"module_id": "workflow_engine"}
], ],
"evidence": [ "evidence": [
@@ -28,7 +29,17 @@
"kind": "documentation", "kind": "documentation",
"reference": "packages/product/governed-data-assurance/README.md", "reference": "packages/product/governed-data-assurance/README.md",
"summary": "Defines the package boundary, provenance chain, and reference-readiness gates." "summary": "Defines the package boundary, provenance chain, and reference-readiness gates."
},
{
"kind": "target_test",
"reference": "tools/checks/check-datasource-composition.py",
"summary": "Proves governed Connector acquisition, Datasource registration, queued Dataflow execution, frozen publication, idempotency, and recovery evidence."
},
{
"kind": "target_test",
"reference": "tools/checks/check-sanctions-screening-composition.py",
"summary": "Proves immutable sanctions acquisition, import, screening replay, independent review, freshness gates, and rescreening across module capabilities."
} }
], ],
"tags": ["datasources", "dataflow", "reporting", "assurance"] "tags": ["datasources", "dataflow", "reporting", "sanctions", "assurance"]
} }
+24 -5
View File
@@ -24,19 +24,38 @@ grant cross-module table access and can omit optional presentation, work,
deliberation, delivery, or records modules while retaining explicit references deliberation, delivery, or records modules while retaining explicit references
to externally performed steps. to externally performed steps.
When Records is present, Forms Runtime, Cases, and Decisions expose exact,
digest-bound source snapshots for explicit filing. The source module rechecks
current access, Records chooses the destination and preserves chronology, and
the filed reference never becomes an editable copy. When Search is present,
the same three owners contribute rebuildable metadata-only projections. Form
values, evidence payloads, Decision reasoning, operative results, and
conditions are excluded; every candidate is authorized again before it is
shown.
When Tasks is present, explicit work and source-owned Workflow handoffs appear
in one resumable inbox with typed account, group, role, function, or assignment
responsibility. Workflow Engine retains process state and completion commands;
Tasks retains only explicit tasks and the aggregation surface.
## Security And Recovery ## Security And Recovery
Every provider is tenant-bound. Missing or conflicting authority fails closed. Every provider is tenant-bound. Missing or conflicting authority fails closed.
Protected Decision content has a separate permission. Writes are replay-safe Protected Decision content has a separate permission. Writes are replay-safe
and OCC-guarded. Database restore is the semantic-state recovery unit; file and and OCC-guarded. Database restore is the semantic-state recovery unit; file and
communication effects remain governed by their owning providers and are linked communication effects remain governed by their owning providers and are linked
through requested/observed effect, evidence, and audit references. through requested/observed effect, evidence, and audit references. Search is a
derived recovery unit and can be rebuilt from authoritative module state.
The executable fixture in The executable fixture in
`tests/test_institutional_governance_journey.py` proves SQL-backed Service, `tests/test_institutional_governance_journey.py` proves SQL-backed Service,
Case, Party, Mandate, Committee meeting/agendum/vote/minute, and Decision state. Case, Party, Mandate, Committee meeting/agendum/vote/minute, and Decision state.
`tests/test_institutional_service_journey.py` separately proves exact Portal `tests/test_institutional_service_journey.py` separately proves exact Portal
Form launch, persisted submission provenance, and idempotent replay. Form launch, persisted submission provenance, idempotent replay, and a durable
Target-environment accessibility, security, operator, privacy, Workflow handoff that remains visible through Tasks after the database session
delivery-provider, and recovery evidence are still required before this product is reopened and disappears only after the Workflow Engine records completion.
package may claim `reference_ready` maturity. Module-level Records source tests prove exact Form submission, Case revision,
and Decision revision filing. Target-environment browser accessibility,
production identity and delivery, a named archive profile, and recovery evidence
are still required before this product package may claim `reference_ready`
maturity.
@@ -35,6 +35,7 @@
{"module_id": "forms_runtime"}, {"module_id": "forms_runtime"},
{"module_id": "postbox"}, {"module_id": "postbox"},
{"module_id": "records"}, {"module_id": "records"},
{"module_id": "search"},
{"module_id": "tasks"}, {"module_id": "tasks"},
{"module_id": "workflow_engine"} {"module_id": "workflow_engine"}
], ],
+1
View File
@@ -58,6 +58,7 @@
{"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-postbox.git", "path": "govoplan-postbox"}, {"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-postbox.git", "path": "govoplan-postbox"},
{"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-procurement.git", "path": "govoplan-procurement"}, {"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-procurement.git", "path": "govoplan-procurement"},
{"name": "govoplan-projects", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-projects.git", "path": "govoplan-projects"}, {"name": "govoplan-projects", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-projects.git", "path": "govoplan-projects"},
{"name": "govoplan-quick-access", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-quick-access.git", "path": "govoplan-quick-access"},
{"name": "govoplan-records", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-records.git", "path": "govoplan-records"}, {"name": "govoplan-records", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-records.git", "path": "govoplan-records"},
{"name": "govoplan-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-reporting.git", "path": "govoplan-reporting"}, {"name": "govoplan-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-reporting.git", "path": "govoplan-reporting"},
{"name": "govoplan-resources", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-resources.git", "path": "govoplan-resources"}, {"name": "govoplan-resources", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-resources.git", "path": "govoplan-resources"},
+2
View File
@@ -35,6 +35,8 @@
-e ../govoplan-dataflow -e ../govoplan-dataflow
-e ../govoplan-workflow-engine -e ../govoplan-workflow-engine
-e ../govoplan-workflow -e ../govoplan-workflow
-e ../govoplan-tasks
-e ../govoplan-quick-access
-e ../govoplan-views -e ../govoplan-views
-e ../govoplan-voting -e ../govoplan-voting
-e ../govoplan-search -e ../govoplan-search
+15 -15
View File
@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags. # Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published. # Only add a module after its referenced tag has been published.
../govoplan-core[server] ../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.15 govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.18
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.15 govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.18
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.15 govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.18
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.15 govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.18
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.15 govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.18
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.15 govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.18
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.15 govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.18
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.15 govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.18
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.15 govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.18
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.15 govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.18
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.15 govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.18
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.15 govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.18
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.15 govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.18
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.15 govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.18
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.15 govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.18
+310 -2
View File
@@ -33,6 +33,10 @@ from govoplan_deploy.bundle import ( # noqa: E402
) )
from govoplan_deploy.cli import _receipt_uses_direct_web_port, main # noqa: E402 from govoplan_deploy.cli import _receipt_uses_direct_web_port, main # noqa: E402
import govoplan_deploy.cli as deployment_cli # noqa: E402 import govoplan_deploy.cli as deployment_cli # noqa: E402
from govoplan_deploy.capabilities import ( # noqa: E402
capability_change_impacts,
infrastructure_capability_document,
)
from govoplan_deploy.cluster_evidence import ( # noqa: E402 from govoplan_deploy.cluster_evidence import ( # noqa: E402
collect_kubernetes_evidence, collect_kubernetes_evidence,
) )
@@ -158,6 +162,81 @@ class DeploymentInstallerTests(unittest.TestCase):
evidence["snapshot"]["ready_node_names"], evidence["snapshot"]["ready_node_names"],
) )
def test_kubernetes_api_loss_uses_a_non_json_mutation_command(self) -> None:
initial_pods = [
_kubernetes_test_pod("api-a", "api", "node-a"),
_kubernetes_test_pod("api-b", "api", "node-b"),
_kubernetes_test_pod("web-a", "web", "node-a"),
_kubernetes_test_pod("web-b", "web", "node-b"),
]
replacement_pods = [
_kubernetes_test_pod("api-b", "api", "node-b"),
_kubernetes_test_pod("api-c", "api", "node-a"),
]
deleted = False
actions: list[tuple[str, ...]] = []
def run(arguments):
if "nodes" in arguments:
return {
"items": [
{
"metadata": {"name": name},
"spec": {},
"status": {
"conditions": [
{"type": "Ready", "status": "True"}
]
},
}
for name in ("node-a", "node-b")
]
}
if "deployments" in arguments:
return {
"items": [
_kubernetes_test_deployment("api", 2),
_kubernetes_test_deployment("web", 2),
]
}
return {"items": replacement_pods if deleted else initial_pods}
def act(arguments):
nonlocal deleted
actions.append(tuple(arguments))
deleted = True
evidence = collect_kubernetes_evidence(
installation_id="govoplan-cluster",
namespace="govoplan",
ops_url="https://govoplan.example.test/api/v1/ops/status",
api_key="not-retained",
exercise_api_pod_loss=True,
command_runner=run,
action_runner=act,
json_fetcher=lambda _url, _key: {
"readiness": {"ready": True},
"runtime_cluster": {
"composition": {"skewed": False},
"software_versions": {"skewed": False},
"queues": {"missing": []},
},
"checks": [
{
"id": "database_capacity",
"state": "ok",
"detail": "Within budget",
}
],
},
)
self.assertEqual("passed", evidence["api_pod_loss"]["state"])
self.assertEqual(1, len(actions))
self.assertIn("delete", actions[0])
self.assertNotIn("-o", actions[0])
self.assertNotIn("not-retained", json.dumps(evidence))
def test_pre_migration_failure_restores_checksum_verified_applied_bundle( def test_pre_migration_failure_restores_checksum_verified_applied_bundle(
self, self,
) -> None: ) -> None:
@@ -339,6 +418,27 @@ class DeploymentInstallerTests(unittest.TestCase):
self.assertNotIn("db-secret", rendered) self.assertNotIn("db-secret", rendered)
self.assertNotIn("redis-secret", rendered) self.assertNotIn("redis-secret", rendered)
self.assertNotIn("object-secret", rendered) self.assertNotIn("object-secret", rendered)
capability_config = next(
item
for item in manifest["items"]
if item["kind"] == "ConfigMap"
and item["metadata"]["name"].endswith("infrastructure-capabilities")
)
capability_payload = json.loads(
capability_config["data"]["infrastructure-capabilities.json"]
)
self.assertEqual(1, capability_payload["schema_version"])
self.assertNotIn("db-secret", json.dumps(capability_payload))
api_container = deployments["govoplan-cluster-api"]["spec"]["template"]["spec"]["containers"][0]
self.assertIn(
{
"name": "deployment-capabilities",
"mountPath": "/etc/govoplan/deployment/infrastructure-capabilities.json",
"subPath": "infrastructure-capabilities.json",
"readOnly": True,
},
api_container["volumeMounts"],
)
self.assertNotIn("PersistentVolumeClaim", kinds) self.assertNotIn("PersistentVolumeClaim", kinds)
self.assertNotIn("StatefulSet", kinds) self.assertNotIn("StatefulSet", kinds)
self.assertEqual(3, deployments["govoplan-cluster-api"]["spec"]["replicas"]) self.assertEqual(3, deployments["govoplan-cluster-api"]["spec"]["replicas"])
@@ -347,6 +447,10 @@ class DeploymentInstallerTests(unittest.TestCase):
"template" "template"
]["spec"]["containers"][0]["command"] ]["spec"]["containers"][0]["command"]
self.assertIn("govoplan_core.commands.fenced_run", scheduler_command) self.assertIn("govoplan_core.commands.fenced_run", scheduler_command)
self.assertEqual(
["--schedule", "/tmp/celerybeat-schedule"],
scheduler_command[-2:],
)
api_init_command = deployments["govoplan-cluster-api"]["spec"]["template"][ api_init_command = deployments["govoplan-cluster-api"]["spec"]["template"][
"spec" "spec"
]["initContainers"][0]["command"] ]["initContainers"][0]["command"]
@@ -381,10 +485,32 @@ class DeploymentInstallerTests(unittest.TestCase):
"containers" "containers"
][0]["readinessProbe"]["httpGet"]["httpHeaders"], ][0]["readinessProbe"]["httpGet"]["httpHeaders"],
) )
api_pod_spec = deployments["govoplan-cluster-api"]["spec"]["template"][
"spec"
]
self.assertEqual(30, api_pod_spec["terminationGracePeriodSeconds"])
self.assertEqual(
["/bin/sh", "-c", "sleep 10"],
api_pod_spec["containers"][0]["lifecycle"]["preStop"]["exec"][
"command"
],
)
self.assertNotIn(
"lifecycle",
deployments["govoplan-cluster-worker"]["spec"]["template"]["spec"][
"containers"
][0],
)
worker_command = deployments["govoplan-cluster-worker"]["spec"]["template"][ worker_command = deployments["govoplan-cluster-worker"]["spec"]["template"][
"spec" "spec"
]["containers"][0]["command"] ]["containers"][0]["command"]
self.assertIn("--concurrency", worker_command) self.assertIn("--concurrency", worker_command)
for deployment in deployments.values():
spread = deployment["spec"]["template"]["spec"][
"topologySpreadConstraints"
][0]
self.assertEqual("DoNotSchedule", spread["whenUnsatisfiable"])
self.assertEqual(["pod-template-hash"], spread["matchLabelKeys"])
self.assertEqual( self.assertEqual(
"62", "62",
manifest["metadata"]["annotations"][ manifest["metadata"]["annotations"][
@@ -392,6 +518,99 @@ class DeploymentInstallerTests(unittest.TestCase):
], ],
) )
def test_kubernetes_export_mounts_an_optional_s3_ca_on_backend_roles(
self,
) -> None:
spec = default_spec(
installation_id="govoplan-cluster",
postgres_mode="external",
redis_mode="external",
storage_mode="s3",
api_replicas=2,
web_replicas=2,
worker_replicas=2,
api_image="registry.example.test/govoplan-api@sha256:" + "a" * 64,
web_image="registry.example.test/govoplan-web@sha256:" + "b" * 64,
)
environment = initial_secrets(
spec,
supplied={
"DATABASE_URL": "postgresql+psycopg://user:secret@postgres.example.test/govoplan",
"GOVOPLAN_DATABASE_URL_PGTOOLS": "postgresql://user:secret@postgres.example.test/govoplan",
"REDIS_URL": "rediss://:secret@redis.example.test/0",
"FILE_STORAGE_S3_ENDPOINT_URL": "https://s3.example.test",
"FILE_STORAGE_S3_REGION": "eu-test-1",
"FILE_STORAGE_S3_ACCESS_KEY_ID": "object-key",
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": "object-secret",
"FILE_STORAGE_S3_BUCKET": "govoplan",
"GOVOPLAN_DB_CONNECTION_LIMIT": "100",
},
)
manifest = render_kubernetes(
spec,
environment,
s3_ca_secret_name="govoplan-s3-ca",
backup_required=False,
)
backend_pods = [
item["spec"]["template"]["spec"]
for item in manifest["items"]
if item["kind"] in {"Deployment", "Job"}
and item["metadata"]["labels"].get("app.kubernetes.io/component")
in {"api", "worker", "scheduler", "migration"}
]
web = next(
item
for item in manifest["items"]
if item["kind"] == "Deployment"
and item["metadata"]["labels"].get("app.kubernetes.io/component")
== "web"
)
self.assertTrue(backend_pods)
for pod in backend_pods:
self.assertIn(
{
"name": "s3-ca",
"secret": {
"secretName": "govoplan-s3-ca",
"items": [{"key": "ca.crt", "path": "s3-ca.crt"}],
},
},
pod["volumes"],
)
for container in [*pod.get("initContainers", []), *pod["containers"]]:
self.assertIn(
{
"name": "AWS_CA_BUNDLE",
"value": "/etc/govoplan/trust/s3-ca.crt",
},
container["env"],
)
self.assertIn(
{
"name": "s3-ca",
"mountPath": "/etc/govoplan/trust",
"readOnly": True,
},
container["volumeMounts"],
)
self.assertNotIn(
"s3-ca",
{
volume["name"]
for volume in web["spec"]["template"]["spec"]["volumes"]
},
)
with self.assertRaisesRegex(ValueError, "S3 CA secret"):
render_kubernetes(
spec,
environment,
s3_ca_secret_name="INVALID_NAME",
backup_required=False,
)
def test_kubernetes_export_splits_worker_queues_and_rejects_capacity_overrun( def test_kubernetes_export_splits_worker_queues_and_rejects_capacity_overrun(
self, self,
) -> None: ) -> None:
@@ -529,6 +748,10 @@ class DeploymentInstallerTests(unittest.TestCase):
compose["services"]["load-balancer"]["ports"], compose["services"]["load-balancer"]["ports"],
) )
self.assertNotIn("ports", compose["services"]["web"]) self.assertNotIn("ports", compose["services"]["web"])
self.assertIn(
"./infrastructure-capabilities.json:/etc/govoplan/deployment/infrastructure-capabilities.json:ro",
compose["services"]["api"]["volumes"],
)
self.assertEqual(1, compose["services"]["api"]["scale"]) self.assertEqual(1, compose["services"]["api"]["scale"])
self.assertEqual(1, compose["services"]["web"]["scale"]) self.assertEqual(1, compose["services"]["web"]["scale"])
@@ -754,6 +977,63 @@ class DeploymentInstallerTests(unittest.TestCase):
reconciled["GARAGE_RPC_SECRET"], reconciled["GARAGE_RPC_SECRET"],
) )
def test_infrastructure_capability_document_exposes_refs_not_secrets(self) -> None:
spec = default_spec(
installation_id="govoplan-shared",
postgres_mode="external",
redis_mode="external",
storage_mode="s3",
mail_mode="external-relay",
module_set="full",
)
values = initial_secrets(
spec,
supplied={
"DATABASE_URL": "postgresql+psycopg://user:database-secret@db.example.test/govoplan",
"REDIS_URL": "rediss://:redis-secret@redis.example.test/0",
"FILE_STORAGE_S3_ENDPOINT_URL": "https://s3.example.test",
"FILE_STORAGE_S3_REGION": "eu-test-1",
"FILE_STORAGE_S3_ACCESS_KEY_ID": "object-key",
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": "object-secret",
"FILE_STORAGE_S3_BUCKET": "govoplan",
},
)
document = infrastructure_capability_document(spec, values)
rendered = json.dumps(document, sort_keys=True)
capabilities = {item["id"]: item for item in document["capabilities"]}
self.assertNotIn("database-secret", rendered)
self.assertNotIn("redis-secret", rendered)
self.assertNotIn("object-secret", rendered)
self.assertNotIn("object-key", rendered)
self.assertEqual("externally_supplied", capabilities["database.postgresql"]["state"])
self.assertEqual("db.example.test", capabilities["database.postgresql"]["endpoint"]["host"])
self.assertEqual(["env:DATABASE_URL"], capabilities["database.postgresql"]["secret_refs"])
self.assertEqual("available_unconfigured", capabilities["mail.smtp"]["state"])
self.assertEqual("mail.smtp-profile", document["post_install_tasks"][0]["id"])
def test_capability_impact_detects_external_endpoint_rebinding(self) -> None:
spec = default_spec(postgres_mode="external", module_set="full")
previous = infrastructure_capability_document(
spec,
{"DATABASE_URL": "postgresql://user:old-secret@old-db.example.test/govoplan"},
)
desired = infrastructure_capability_document(
spec,
{"DATABASE_URL": "postgresql://user:new-secret@new-db.example.test/govoplan"},
)
impacts = {
item.capability_id: item
for item in capability_change_impacts(previous, desired)
}
self.assertEqual("reconfigure", impacts["database.postgresql"].action)
self.assertIn("changed endpoint binding", impacts["database.postgresql"].detail)
self.assertNotIn("old-secret", impacts["database.postgresql"].detail)
self.assertNotIn("new-secret", impacts["database.postgresql"].detail)
def test_replica_counts_drive_compose_and_load_balancer_discovery(self) -> None: def test_replica_counts_drive_compose_and_load_balancer_discovery(self) -> None:
spec = default_spec( spec = default_spec(
storage_mode="garage", storage_mode="garage",
@@ -883,10 +1163,14 @@ class DeploymentInstallerTests(unittest.TestCase):
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory: with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
paths = bundle_paths(Path(directory)) paths = bundle_paths(Path(directory))
paths.root.chmod(0o700) paths.root.chmod(0o700)
first_spec = default_spec(mail_mode="test-mail") first_spec = default_spec(mail_mode="test-mail", module_set="full")
first_environment = initial_secrets(first_spec) first_environment = initial_secrets(first_spec)
write_env(paths.env, first_environment) write_env(paths.env, first_environment)
first_plan = build_plan(first_spec, paths, include_host_checks=False) first_plan = build_plan(first_spec, paths, include_host_checks=False)
first_capabilities = infrastructure_capability_document(
first_spec,
first_environment,
)
atomic_write( atomic_write(
paths.receipt, paths.receipt,
canonical_json( canonical_json(
@@ -897,12 +1181,17 @@ class DeploymentInstallerTests(unittest.TestCase):
first_plan.desired_environment_fingerprint first_plan.desired_environment_fingerprint
), ),
"services": list(render_compose(first_spec)["services"]), "services": list(render_compose(first_spec)["services"]),
"infrastructure_capabilities": first_capabilities,
} }
), ),
mode=0o600, mode=0o600,
) )
second_spec = default_spec(redis_mode="disabled", mail_mode="disabled") second_spec = default_spec(
redis_mode="disabled",
mail_mode="disabled",
module_set="full",
)
write_env( write_env(
paths.env, paths.env,
reconcile_runtime_environment(second_spec, first_environment), reconcile_runtime_environment(second_spec, first_environment),
@@ -918,6 +1207,20 @@ class DeploymentInstallerTests(unittest.TestCase):
{"redis", "worker", "scheduler", "test-mail"}, {"redis", "worker", "scheduler", "test-mail"},
removed, removed,
) )
impacts = {
item.capability_id: item
for item in second_plan.capability_impacts
}
self.assertEqual("remove", impacts["coordination.redis"].action)
self.assertEqual("remove", impacts["mail.smtp"].action)
self.assertIn("mail", impacts["mail.smtp"].dependent_modules)
self.assertTrue(
any(
check.id == "capability.change.mail.smtp"
and check.level == "warning"
for check in second_plan.checks
)
)
def test_secret_change_is_planned_without_exposing_secret_values(self) -> None: def test_secret_change_is_planned_without_exposing_secret_values(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory: with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
@@ -1286,6 +1589,11 @@ class DeploymentInstallerTests(unittest.TestCase):
receipt["listen"], receipt["listen"],
) )
self.assertNotIn("installer", receipt["services"]) self.assertNotIn("installer", receipt["services"])
self.assertEqual(
1,
receipt["infrastructure_capabilities"]["schema_version"],
)
self.assertTrue((root / "infrastructure-capabilities.json").is_file())
def test_installation_root_symlink_is_rejected(self) -> None: def test_installation_root_symlink_is_rejected(self) -> None:
if not hasattr(Path, "symlink_to"): if not hasattr(Path, "symlink_to"):
+238 -8
View File
@@ -2,11 +2,14 @@ from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
from types import SimpleNamespace
import unittest import unittest
from sqlalchemy import create_engine from sqlalchemy import create_engine
from sqlalchemy.orm import Session from sqlalchemy.orm import Session, sessionmaker
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.institutional import ( from govoplan_core.core.institutional import (
CAPABILITY_FORM_DEFINITIONS, CAPABILITY_FORM_DEFINITIONS,
CAPABILITY_SERVICE_DEFINITIONS, CAPABILITY_SERVICE_DEFINITIONS,
@@ -18,6 +21,17 @@ from govoplan_core.core.institutional import (
TemporalRevision, TemporalRevision,
service_launch_capability, service_launch_capability,
) )
from govoplan_core.core.runtime_coordination import (
DistributedLease,
RuntimeIdentity,
bind_process_runtime_identity,
)
from govoplan_core.core.tasks import (
RegisteredWorkItemProvider,
WorkItemProviderRegistration,
WorkItemQuery,
)
from govoplan_core.core.recovery import RecoveryCheckpoint, RecoveryOperation
from govoplan_cases.backend.service_intake import ( from govoplan_cases.backend.service_intake import (
CAPABILITY_CASES_SERVICE_INTAKE, CAPABILITY_CASES_SERVICE_INTAKE,
CaseServiceIntake, CaseServiceIntake,
@@ -37,6 +51,34 @@ from govoplan_forms_runtime.backend.service import (
FormsServiceLauncher, FormsServiceLauncher,
) )
from govoplan_portal.backend.service_directory import PortalServiceDirectory from govoplan_portal.backend.service_directory import PortalServiceDirectory
from govoplan_tasks.backend.aggregation import aggregate_work_items
from govoplan_workflow_engine.backend.db.models import (
WorkflowDefinition,
WorkflowDefinitionRevision,
WorkflowInstance,
WorkflowInstanceEvent,
WorkflowInstanceStep,
WorkflowTrigger,
WorkflowTriggerDelivery,
WorkflowWaitState,
)
from govoplan_workflow_engine.backend.instance_service import (
resolve_step,
start_instance,
)
from govoplan_workflow_engine.backend.schemas import (
WorkflowDefinitionCreateRequest,
WorkflowEdge,
WorkflowGraph,
WorkflowInstanceStartRequest,
WorkflowNode,
WorkflowStepActionRequest,
)
from govoplan_workflow_engine.backend.service import (
activate_definition,
create_definition,
)
from govoplan_workflow_engine.backend.work_items import WorkflowWorkItemProvider
NOW = datetime(2026, 8, 1, 10, 0, tzinfo=UTC) NOW = datetime(2026, 8, 1, 10, 0, tzinfo=UTC)
@@ -74,10 +116,14 @@ class _Provider:
def __init__(self, definition: ServiceDefinition) -> None: def __init__(self, definition: ServiceDefinition) -> None:
self.definition = definition self.definition = definition
def get_service_definition(self, session, principal, *, reference, effective_at=None): def get_service_definition(
self, session, principal, *, reference, effective_at=None
):
return self.definition return self.definition
def list_service_definitions(self, session, principal, *, tenant_id, query="", limit=100): def list_service_definitions(
self, session, principal, *, tenant_id, query="", limit=100
):
return (self.definition,) return (self.definition,)
@@ -111,17 +157,58 @@ class _Principal:
class _FormRegistry(_Registry): class _FormRegistry(_Registry):
def __init__(self, definition: ServiceDefinition) -> None: def __init__(self, definition: ServiceDefinition) -> None:
super().__init__(definition) super().__init__(definition)
self.capabilities[CAPABILITY_FORM_DEFINITIONS] = ( self.capabilities[CAPABILITY_FORM_DEFINITIONS] = SqlFormDefinitionProvider()
SqlFormDefinitionProvider() self.capabilities[service_launch_capability("form")] = FormsServiceLauncher(
) self
self.capabilities[service_launch_capability("form")] = (
FormsServiceLauncher(self)
) )
def has(self, module_id: str) -> bool: def has(self, module_id: str) -> bool:
return module_id in {"portal", "forms", "forms_runtime"} return module_id in {"portal", "forms", "forms_runtime"}
class _WorkflowTaskRegistry:
def __init__(self) -> None:
self.provider = WorkflowWorkItemProvider(registry=self)
self.registered = RegisteredWorkItemProvider(
module_id="workflow_engine",
registration=WorkItemProviderRegistration(
id="workflow_engine.handoffs",
factory=lambda _context: self.provider,
order=20,
),
)
def has_capability(self, _name: str) -> bool:
return False
def capability(self, name: str) -> object:
raise KeyError(name)
def work_item_providers(self):
return ((self.registered, self.provider),)
def _workflow_principal() -> ApiPrincipal:
return ApiPrincipal(
principal=PrincipalRef(
account_id="account-1",
membership_id="membership-1",
tenant_id="tenant-1",
scopes=frozenset(
{
"tasks:item:read",
"workflow:definition:read",
"workflow:instance:read",
"workflow:instance:start",
"workflow:instance:transition",
}
),
),
account=SimpleNamespace(id="account-1"),
user=SimpleNamespace(id="membership-1"),
)
class InstitutionalServiceJourneyTests(unittest.TestCase): class InstitutionalServiceJourneyTests(unittest.TestCase):
def test_one_service_version_drives_portal_and_case_intake(self) -> None: def test_one_service_version_drives_portal_and_case_intake(self) -> None:
definition = _service() definition = _service()
@@ -266,6 +353,149 @@ class InstitutionalServiceJourneyTests(unittest.TestCase):
session.close() session.close()
engine.dispose() engine.dispose()
def test_workflow_handoff_survives_session_reopen_and_projects_into_tasks(
self,
) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
tables = (
DistributedLease.__table__,
RecoveryOperation.__table__,
RecoveryCheckpoint.__table__,
WorkflowDefinition.__table__,
WorkflowDefinitionRevision.__table__,
WorkflowInstance.__table__,
WorkflowInstanceStep.__table__,
WorkflowInstanceEvent.__table__,
WorkflowTrigger.__table__,
WorkflowTriggerDelivery.__table__,
WorkflowWaitState.__table__,
)
for table in tables:
table.create(engine)
sessions = sessionmaker(bind=engine)
registry = _WorkflowTaskRegistry()
principal = _workflow_principal()
bind_process_runtime_identity(
RuntimeIdentity(
installation_id="service-journey",
node_id="journey-node",
incarnation="journey-run",
role="web",
software_version="test",
composition_hash="c" * 64,
)
)
try:
with sessions() as session:
definition = create_definition(
session,
tenant_id="tenant-1",
actor_id="account-1",
payload=WorkflowDefinitionCreateRequest(
name="Permit decision",
graph=WorkflowGraph(
nodes=[
WorkflowNode(
id="start",
type="workflow.start.manual",
),
WorkflowNode(
id="review",
type="workflow.activity",
config={
"title": "Decide the permit application",
"instructions": "Review the filed evidence and record the decision.",
"assignee": "account:account-1",
"due_after": "2d",
},
),
WorkflowNode(
id="done",
type="workflow.end.completed",
),
],
edges=[
WorkflowEdge(
id="start-review",
source="start",
target="review",
),
WorkflowEdge(
id="review-done",
source="review",
target="done",
),
],
),
execution_mode="guided",
),
)
activate_definition(
session,
tenant_id="tenant-1",
definition_id=definition.id,
actor_id="account-1",
)
instance, replayed = start_instance(
session,
tenant_id="tenant-1",
definition_id=definition.id,
actor_id="account-1",
principal=principal,
registry=registry,
payload=WorkflowInstanceStartRequest(
idempotency_key="permit-decision-1",
input={"case_id": "case-1"},
correlation_id="case-1",
),
)
self.assertFalse(replayed)
session.commit()
instance_id = instance.id
step_id = instance.current_step_id
with sessions() as reopened:
work = aggregate_work_items(
registry,
reopened,
principal,
query=WorkItemQuery(tenant_id="tenant-1"),
)
self.assertEqual(1, work.total)
self.assertEqual(step_id, work.items[0].id)
self.assertEqual(
"Decide the permit application",
work.items[0].title,
)
self.assertTrue(work.items[0].action_url.startswith("/workflow?"))
self.assertIn(f"run={instance_id}", work.items[0].action_url)
resolve_step(
reopened,
tenant_id="tenant-1",
instance_id=instance_id,
step_id=step_id,
actor_id="account-1",
principal=principal,
registry=registry,
payload=WorkflowStepActionRequest(action="complete"),
)
reopened.commit()
with sessions() as verified:
self.assertEqual(
0,
aggregate_work_items(
registry,
verified,
principal,
query=WorkItemQuery(tenant_id="tenant-1"),
).total,
)
finally:
bind_process_runtime_identity(None)
engine.dispose()
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+273
View File
@@ -0,0 +1,273 @@
from __future__ import annotations
from contextlib import redirect_stdout
from dataclasses import replace
import io
import json
from pathlib import Path
import shutil
import stat
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import MagicMock
META_ROOT = Path(__file__).resolve().parents[1]
LAB_TOOLS = META_ROOT / "tools" / "lab"
if str(LAB_TOOLS) not in sys.path:
sys.path.insert(0, str(LAB_TOOLS))
from govoplan_lab.cli import main # noqa: E402
from govoplan_lab.config import LabConfigError, load_config # noqa: E402
from govoplan_lab.lifecycle import ( # noqa: E402
CommandRunner,
LabOperationError,
_assert_domain_owned,
_domain_description,
_ensure_certificates,
_render_kubectl_wrapper,
destroy,
)
from govoplan_lab.render import ( # noqa: E402
render_k3s_config,
render_registry_config,
render_state_compose,
write_private,
)
REHEARSAL_CONFIG = LAB_TOOLS / "govoplan-lab.example.toml"
ACCEPTANCE_CONFIG = LAB_TOOLS / "govoplan-lab.acceptance.example.toml"
class KubernetesLabTests(unittest.TestCase):
def test_example_inventories_describe_their_evidence_boundary(self) -> None:
rehearsal = load_config(REHEARSAL_CONFIG)
acceptance = load_config(ACCEPTANCE_CONFIG)
self.assertEqual("rehearsal", rehearsal.mode)
self.assertFalse(rehearsal.evidence_capable)
self.assertEqual(2, len(rehearsal.workers))
self.assertEqual("acceptance", acceptance.mode)
self.assertTrue(acceptance.evidence_capable)
self.assertEqual(3, len({node.hypervisor for node in acceptance.nodes}))
self.assertEqual(3, len({node.failure_domain for node in acceptance.nodes}))
def test_acceptance_inventory_rejects_collapsed_worker_failure_domains(self) -> None:
source = ACCEPTANCE_CONFIG.read_text(encoding="utf-8")
collapsed = source.replace(
'hypervisor = "lab-admin@hypervisor-b.example.org"',
'hypervisor = "lab-admin@hypervisor-a.example.org"',
).replace('failure_domain = "rack-b"', 'failure_domain = "rack-a"')
with tempfile.TemporaryDirectory(prefix="govoplan-lab-config-") as directory:
path = Path(directory) / "lab.toml"
path.write_text(collapsed, encoding="utf-8")
with self.assertRaisesRegex(LabConfigError, "acceptance mode"):
load_config(path)
def test_create_without_apply_is_a_non_mutating_preview(self) -> None:
output = io.StringIO()
with redirect_stdout(output):
exit_code = main(["--config", str(REHEARSAL_CONFIG), "create"])
self.assertEqual(0, exit_code)
self.assertIn("Dry run: create", output.getvalue())
self.assertIn("Re-run with --apply", output.getvalue())
def test_local_hypervisor_uses_system_libvirt_without_sudo(self) -> None:
config = load_config(REHEARSAL_CONFIG)
runner = CommandRunner(config)
runner.run = MagicMock(
return_value=subprocess.CompletedProcess([], 0, stdout=b"", stderr=b"")
)
runner.hypervisor(config.nodes[0], ["virsh", "dominfo", "test-domain"])
runner.run.assert_called_once_with(
[
"virsh",
"--connect",
"qemu:///system",
"dominfo",
"test-domain",
],
capture=False,
check=True,
timeout=None,
)
def test_local_hypervisor_file_operations_do_not_use_sudo(self) -> None:
config = load_config(REHEARSAL_CONFIG)
runner = CommandRunner(config)
runner.run = MagicMock(
return_value=subprocess.CompletedProcess([], 0, stdout=b"", stderr=b"")
)
runner.hypervisor(config.nodes[0], ["install", "-d", "/tmp/lab"])
runner.run.assert_called_once_with(
["install", "-d", "/tmp/lab"],
capture=False,
check=True,
timeout=None,
)
def test_kubectl_wrapper_quotes_remote_arguments(self) -> None:
wrapper = _render_kubectl_wrapper(
["ssh", "-i", "/tmp/lab key", "govoplan@example.test"]
)
self.assertIn("shlex.join(_REMOTE)", wrapper)
self.assertIn('["sudo", "--", "k3s", "kubectl", *sys.argv[1:]]', wrapper)
self.assertNotIn('kubectl \"$@\"', wrapper)
def test_destroy_requires_the_exact_lab_name(self) -> None:
config = load_config(REHEARSAL_CONFIG)
with self.assertRaisesRegex(LabOperationError, "--confirm"):
destroy(
config,
apply=True,
confirmation="wrong-lab",
purge_local_state=False,
)
def test_enroll_admin_without_apply_is_a_non_mutating_preview(self) -> None:
output = io.StringIO()
with redirect_stdout(output):
exit_code = main(
[
"--config",
str(REHEARSAL_CONFIG),
"enroll-admin",
"--email",
"owner@example.test",
]
)
self.assertEqual(0, exit_code)
self.assertIn("Dry run: enroll-admin", output.getvalue())
self.assertIn("owner@example.test", output.getvalue())
def test_domain_ownership_requires_marker_and_expected_disks(self) -> None:
config = load_config(REHEARSAL_CONFIG)
node = config.nodes[0]
node_directory = f"{config.vm_image_directory}/{config.name}/{node.name}"
runner = MagicMock()
runner.hypervisor.side_effect = [
subprocess.CompletedProcess(
[],
0,
stdout=(_domain_description(config, node) + "\n").encode(),
stderr=b"",
),
subprocess.CompletedProcess(
[],
0,
stdout=(
f"file disk vda {node_directory}/root.qcow2\n"
f"file cdrom sda {node_directory}/seed.img\n"
).encode(),
stderr=b"",
),
]
_assert_domain_owned(config, runner, node)
runner.hypervisor.side_effect = [
subprocess.CompletedProcess(
[],
0,
stdout=b"unrelated domain\n",
stderr=b"",
)
]
with self.assertRaisesRegex(LabOperationError, "ownership marker"):
_assert_domain_owned(config, runner, node)
def test_state_compose_uses_only_supplied_pinned_images(self) -> None:
names = ("postgres", "redis", "garage", "managed_ingress", "test_mail")
images = {
name: f"registry.example.test/{name}@sha256:{index:064x}"
for index, name in enumerate(names, start=1)
}
compose = json.loads(render_state_compose(images))
self.assertEqual(images["postgres"], compose["services"]["postgres"]["image"])
self.assertEqual(images["redis"], compose["services"]["redis"]["image"])
self.assertEqual(images["garage"], compose["services"]["garage"]["image"])
self.assertEqual(
images["managed_ingress"], compose["services"]["s3-tls"]["image"]
)
self.assertEqual(
images["test_mail"], compose["services"]["test-mail"]["image"]
)
def test_k3s_workers_join_the_primary_control_and_receive_failure_labels(
self,
) -> None:
config = load_config(ACCEPTANCE_CONFIG)
worker = config.workers[0]
rendered = render_k3s_config(config, worker, cluster_token="test-token")
self.assertIn(f'server: "https://{config.primary_control.address}:6443"', rendered)
self.assertIn(
f'topology.govoplan.add-ideas.de/failure-domain={worker.failure_domain}',
rendered,
)
self.assertNotIn("cluster-init", rendered)
def test_registry_credentials_are_all_or_nothing(self) -> None:
self.assertEqual("", render_registry_config("", ""))
with self.assertRaisesRegex(ValueError, "supplied together"):
render_registry_config("publisher", "")
rendered = render_registry_config("publisher", "secret")
self.assertIn('"git.add-ideas.de"', rendered)
self.assertIn("publisher", rendered)
def test_private_writer_enforces_owner_only_permissions(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-lab-private-") as directory:
path = Path(directory) / "nested" / "secret.txt"
write_private(path, "secret\n")
self.assertEqual(0o600, stat.S_IMODE(path.stat().st_mode))
@unittest.skipUnless(shutil.which("openssl"), "openssl is required")
def test_generated_lab_ca_passes_strict_chain_validation(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-lab-pki-") as directory:
config = replace(
load_config(REHEARSAL_CONFIG),
state_directory=Path(directory),
)
_ensure_certificates(config, CommandRunner(config))
ca_certificate = config.state_directory / "pki" / "ca.crt"
server_certificate = config.state_directory / "pki" / "server.crt"
result = subprocess.run(
[
"openssl",
"verify",
"-x509_strict",
"-CAfile",
str(ca_certificate),
str(server_certificate),
],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
self.assertEqual(
0,
result.returncode,
(result.stdout + result.stderr).decode(errors="replace"),
)
if __name__ == "__main__":
unittest.main()
+56
View File
@@ -46,6 +46,7 @@ class PackageRegistryReleaseTests(unittest.TestCase):
) )
self.assertEqual("1", payload["schema_version"]) self.assertEqual("1", payload["schema_version"])
self.assertEqual("base", payload["profile"])
self.assertEqual("govoplan-core", payload["python"][0]["name"]) self.assertEqual("govoplan-core", payload["python"][0]["name"])
self.assertIn( self.assertIn(
"@govoplan/core-webui", "@govoplan/core-webui",
@@ -55,6 +56,43 @@ class PackageRegistryReleaseTests(unittest.TestCase):
digest = unsigned.pop("package_set_sha256") digest = unsigned.pop("package_set_sha256")
self.assertEqual(ARTIFACTS._canonical_sha256(unsigned), digest) self.assertEqual(ARTIFACTS._canonical_sha256(unsigned), digest)
def test_full_profile_is_derived_from_the_developer_meta_package(self) -> None:
selected = PACKAGE_SET.parse_meta_package(
ROOT / "packages/govoplan-meta/pyproject.toml",
core_version="0.1.18",
)
by_name = {item["name"]: item for item in selected}
self.assertIn("govoplan-core", by_name)
self.assertIn("govoplan-records", by_name)
self.assertEqual("0.1.19", by_name["govoplan-tasks"]["version"])
payload = PACKAGE_SET.generate_package_set(
core_version="0.1.18",
requirements=ROOT / "requirements-release.txt",
workspace=ROOT.parent,
profile="full",
meta_package=ROOT / "packages/govoplan-meta/pyproject.toml",
)
self.assertEqual("full", payload["profile"])
self.assertEqual(len(selected), len(payload["python"]))
self.assertIn(
"@govoplan/records-webui",
{item["name"] for item in payload["webui"]},
)
def test_python_registry_artifact_url_is_immutable_and_credential_free(self) -> None:
url = ARTIFACTS._python_artifact_url(
"https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
package={"name": "govoplan-files", "version": "0.1.18"},
filename="govoplan_files-0.1.18-py3-none-any.whl",
)
self.assertEqual(
"https://git.add-ideas.de/api/packages/GovOPlaN/pypi/files/govoplan-files/0.1.18/govoplan_files-0.1.18-py3-none-any.whl",
url,
)
def test_wheel_and_webui_artifacts_are_verified_by_embedded_identity(self) -> None: def test_wheel_and_webui_artifacts_are_verified_by_embedded_identity(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-package-artifacts-") as value: with tempfile.TemporaryDirectory(prefix="govoplan-package-artifacts-") as value:
root = Path(value) root = Path(value)
@@ -138,11 +176,17 @@ class PackageRegistryReleaseTests(unittest.TestCase):
) )
self.assertIn("resolve-package-artifacts.py", workflow) self.assertIn("resolve-package-artifacts.py", workflow)
self.assertIn("--profile full", workflow)
self.assertIn('git show "v$VERSION:requirements-release.txt"', workflow)
self.assertIn('git show "v$VERSION:packages/govoplan-meta/pyproject.toml"', workflow)
self.assertIn("--meta-package runtime-output/govoplan-meta.source.toml", workflow)
self.assertIn("GOVOPLAN_WEBUI_INSTALL_ALL_PACKAGES=true", workflow)
self.assertIn("package-artifacts.lock.json", workflow) self.assertIn("package-artifacts.lock.json", workflow)
self.assertIn( self.assertIn(
"--package-lock runtime-output/package-artifacts.lock.json", "--package-lock runtime-output/package-artifacts.lock.json",
workflow, workflow,
) )
self.assertIn('PYTHON="$PWD/.runtime-build/bin/python"', workflow)
self.assertNotIn( self.assertNotIn(
"pip wheel --no-deps --wheel-dir runtime-output/local-wheels", "pip wheel --no-deps --wheel-dir runtime-output/local-wheels",
workflow, workflow,
@@ -162,6 +206,18 @@ class PackageRegistryReleaseTests(unittest.TestCase):
) )
self.assertEqual(expected, actual) self.assertEqual(expected, actual)
def test_meta_package_workflow_supports_hash_safe_tag_retry(self) -> None:
workflow = (
ROOT / ".gitea/workflows/publish-developer-meta-package.yml"
).read_text(encoding="utf-8")
self.assertIn("workflow_dispatch:", workflow)
self.assertIn("TRIGGER_TAG: ${{ gitea.ref_name }}", workflow)
self.assertNotIn("GITEA_REF_NAME", workflow)
self.assertIn('refs/tags/{tag}^{{commit}}', workflow)
self.assertIn("already exists with a different SHA-256", workflow)
self.assertIn("PUBLISH_PYPI", workflow)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+46 -4
View File
@@ -1,10 +1,11 @@
from __future__ import annotations from __future__ import annotations
import sys import sys
import tomllib
import unittest import unittest
from pathlib import Path from pathlib import Path
from govoplan_core.core.modules import ModuleManifest from govoplan_core.core.modules import ModuleManifest, PermissionDefinition
from govoplan_core.core.provider_governance import ( from govoplan_core.core.provider_governance import (
ExternalProviderDeclaration, ExternalProviderDeclaration,
ModuleArchitectureDeclaration, ModuleArchitectureDeclaration,
@@ -26,6 +27,13 @@ from govoplan_release.catalog_entry_synthesis import ( # noqa: E402
from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402 from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402
def repository_version(name: str) -> str:
payload = tomllib.loads(
(META_ROOT.parent / name / "pyproject.toml").read_text(encoding="utf-8")
)
return str(payload["project"]["version"])
class ReleaseCatalogEntrySynthesisTests(unittest.TestCase): class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
def test_catalog_entry_preserves_architecture_and_provider_declarations( def test_catalog_entry_preserves_architecture_and_provider_declarations(
self, self,
@@ -80,6 +88,18 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
id="example", id="example",
name="Example", name="Example",
version="1.2.3", version="1.2.3",
permissions=(
PermissionDefinition(
scope="example:records:read",
label="Read records",
description="Read example records.",
category="Records",
level="tenant",
module_id="example",
resource="records",
action="read",
),
),
architecture=architecture, architecture=architecture,
external_providers=(provider,), external_providers=(provider,),
), ),
@@ -92,12 +112,31 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
) )
self.assertEqual("vertical_slice", entry["architecture"]["maturity"]) self.assertEqual("vertical_slice", entry["architecture"]["maturity"])
self.assertEqual(
"contract_only",
entry["information_governance"]["dimensions"]["retention"][
"adoption"
],
)
self.assertEqual( self.assertEqual(
"external_mirror", "external_mirror",
entry["external_providers"][0]["objects"][0][ entry["external_providers"][0]["objects"][0][
"default_authority_mode" "default_authority_mode"
], ],
) )
self.assertEqual(
{
"scope": "example:records:read",
"label": "Read records",
"description": "Read example records.",
"category": "Records",
"level": "tenant",
"resource": "records",
"action": "read",
"deprecated": False,
},
entry["permissions"][0],
)
def test_selective_update_synthesizes_initial_entries_from_package_manifests(self) -> None: def test_selective_update_synthesizes_initial_entries_from_package_manifests(self) -> None:
payload: dict[str, object] = { payload: dict[str, object] = {
@@ -116,9 +155,12 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
changes = apply_repo_updates( changes = apply_repo_updates(
payload, payload,
repo_versions={ repo_versions={
"govoplan-addresses": "0.1.9", name: repository_version(name)
"govoplan-poll": "0.1.11", for name in (
"govoplan-scheduling": "0.1.11", "govoplan-addresses",
"govoplan-poll",
"govoplan-scheduling",
)
}, },
repo_contracts={}, repo_contracts={},
repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN", repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN",
+9 -2
View File
@@ -95,14 +95,21 @@ class ReleaseEntrypointGateTests(unittest.TestCase):
) )
self.assertIn('cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"', script) self.assertIn('cp "$WEBUI/package.release.json" "$TMP_DIR/package.json"', script)
def test_source_catalog_generator_enforces_explicit_repo_versions(self) -> None: def test_catalog_generator_validates_registry_package_set_before_writing(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text() script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text()
gate = script.index("selected_repository_version_issues(") gate = script.index("_validate_release_inputs(package_set, package_lock")
write = script.index("output.write_text(") write = script.index("output.write_text(")
self.assertLess(gate, write) self.assertLess(gate, write)
def test_full_catalog_publication_synchronizes_browsable_module_directory(self) -> None:
publisher = (META_ROOT / "tools" / "release" / "publish-release-catalog.sh").read_text()
self.assertIn('--module-directory-output "$WEB_ROOT/public/catalogs/v1"', publisher)
self.assertIn('git -C "$WEB_ROOT" add -A', publisher)
self.assertIn('"$MODULE_DIRECTORY_PATH"', publisher)
def test_candidate_publication_uses_existing_keyring_as_trust_anchor(self) -> None: def test_candidate_publication_uses_existing_keyring_as_trust_anchor(self) -> None:
publisher = (META_ROOT / "tools" / "release" / "govoplan_release" / "publisher.py").read_text() publisher = (META_ROOT / "tools" / "release" / "govoplan_release" / "publisher.py").read_text()
+75
View File
@@ -1,7 +1,9 @@
from __future__ import annotations from __future__ import annotations
import json
from pathlib import Path from pathlib import Path
import sys import sys
import tempfile
import unittest import unittest
from unittest import mock from unittest import mock
@@ -14,6 +16,7 @@ if str(RELEASE_TOOLS_ROOT) not in sys.path:
from govoplan_release.module_directory import ( # noqa: E402 from govoplan_release.module_directory import ( # noqa: E402
module_directory_payloads, module_directory_payloads,
safe_path_part, safe_path_part,
write_module_directory,
) )
@@ -77,6 +80,78 @@ class ReleaseModuleDirectoryTests(unittest.TestCase):
catalog_payload={}, keyring_payload={}, channel="../stable" catalog_payload={}, keyring_payload={}, channel="../stable"
) )
def test_prune_removes_stale_json_but_keeps_unrelated_assets(self) -> None:
catalog = {
"generated_at": "2026-08-06T12:00:00Z",
"sequence": 8,
"modules": [
{
"module_id": "files",
"name": "Files",
"version": "1.2.3",
"python_package": "govoplan-files",
"source": {
"repository": "govoplan-files",
"tag": "v1.2.3",
"commit": "a" * 40,
},
"artifact_integrity": {
"python": {"sha256": "b" * 64},
},
}
],
}
with tempfile.TemporaryDirectory() as value:
output_root = Path(value)
stale = output_root / "modules" / "legacy" / "0.1.0" / "manifest.json"
stale.parent.mkdir(parents=True)
stale.write_text("{}\n", encoding="utf-8")
unrelated = output_root / "modules" / "README.txt"
unrelated.write_text("keep\n", encoding="utf-8")
written = write_module_directory(
catalog_payload=catalog,
keyring_payload={},
output_root=output_root,
channel="stable",
prune=True,
)
self.assertFalse(stale.exists())
self.assertTrue(unrelated.exists())
self.assertEqual(3, len(written))
manifest = json.loads(
(output_root / "modules" / "files" / "1.2.3" / "manifest.json").read_text()
)
self.assertEqual("govoplan-files", manifest["module"]["repo"])
self.assertEqual("v1.2.3", manifest["module"]["python_tag"])
self.assertEqual("b" * 64, manifest["module"]["artifact_integrity"]["python"]["sha256"])
def test_writer_refuses_nested_symlink_targets(self) -> None:
catalog = {
"modules": [{"module_id": "files", "version": "1.2.3"}],
}
with tempfile.TemporaryDirectory() as value:
root = Path(value)
output_root = root / "public"
external = root / "external"
(output_root / "modules").mkdir(parents=True)
external.mkdir()
(output_root / "modules" / "files").symlink_to(
external,
target_is_directory=True,
)
with self.assertRaisesRegex(ValueError, "symlinks"):
write_module_directory(
catalog_payload=catalog,
keyring_payload={},
output_root=output_root,
channel="stable",
)
self.assertEqual([], list(external.iterdir()))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+32 -2
View File
@@ -121,6 +121,22 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
): ):
self.assertIn(temporary_path, nginx) self.assertIn(temporary_path, nginx)
def test_web_runtime_resolves_the_configured_api_service_at_startup(self) -> None:
nginx = (ROOT / "tools/release/runtime/nginx.conf").read_text(
encoding="utf-8"
)
dockerfile = (ROOT / "tools/release/runtime/Dockerfile.web").read_text(
encoding="utf-8"
)
entrypoint = (ROOT / "tools/release/runtime/web-entrypoint.sh").read_text(
encoding="utf-8"
)
self.assertIn("proxy_pass ${GOVOPLAN_API_UPSTREAM};", nginx)
self.assertIn("nginx.conf.template", dockerfile)
self.assertIn("govoplan-web-entrypoint", dockerfile)
self.assertIn("envsubst '${GOVOPLAN_API_UPSTREAM}'", entrypoint)
def test_workflow_verifies_portable_bootstrap_artifacts_before_execution( def test_workflow_verifies_portable_bootstrap_artifacts_before_execution(
self, self,
) -> None: ) -> None:
@@ -163,7 +179,7 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
) )
self.assertNotIn(".platforms[\\\"linux/amd64\\\"]", workflow) self.assertNotIn(".platforms[\\\"linux/amd64\\\"]", workflow)
def test_workflow_binds_the_release_tag_to_the_workflow_commit(self) -> None: def test_workflow_binds_distribution_to_the_peeled_release_tag(self) -> None:
workflow = (ROOT / ".gitea/workflows/runtime-distribution.yml").read_text( workflow = (ROOT / ".gitea/workflows/runtime-distribution.yml").read_text(
encoding="utf-8" encoding="utf-8"
) )
@@ -171,7 +187,21 @@ class RuntimeDistributionBuildTests(unittest.TestCase):
encoding="utf-8" encoding="utf-8"
) )
self.assertIn("SOURCE_COMMIT: ${{ gitea.sha }}", workflow) self.assertIn(
'git fetch --force --no-tags origin "refs/tags/v$VERSION:refs/tags/v$VERSION"',
workflow,
)
self.assertIn(
'git rev-parse "v$VERSION^{commit}" > runtime-output/release-source-commit',
workflow,
)
self.assertEqual(
2,
workflow.count(
'SOURCE_COMMIT="$(cat runtime-output/release-source-commit)"'
),
)
self.assertNotIn("SOURCE_COMMIT: ${{ gitea.sha }}", workflow)
self.assertIn('--target-commit "$SOURCE_COMMIT"', workflow) self.assertIn('--target-commit "$SOURCE_COMMIT"', workflow)
self.assertIn('"target_commitish": target_commit', publisher) self.assertIn('"target_commitish": target_commit', publisher)
self.assertIn("self._resolve_commit(tag) != target_commit", publisher) self.assertIn("self._resolve_commit(tag) != target_commit", publisher)
+6
View File
@@ -75,6 +75,12 @@ class RuntimeImageContextTests(unittest.TestCase):
).read_text() ).read_text()
) )
self.assertEqual(composition, published) self.assertEqual(composition, published)
self.assertEqual(
(
SCRIPT.parent / "runtime" / "web-entrypoint.sh"
).read_bytes(),
(root / "context" / "web-entrypoint.sh").read_bytes(),
)
def test_rejects_missing_required_module(self) -> None: def test_rejects_missing_required_module(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-runtime-context-") as value: with tempfile.TemporaryDirectory(prefix="govoplan-runtime-context-") as value:
+2
View File
@@ -96,6 +96,7 @@ PY
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow-engine/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow-engine/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-quick-access/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests
@@ -109,6 +110,7 @@ PY
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py "$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py" "$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count "$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
+54
View File
@@ -128,6 +128,13 @@ def main() -> int:
manifest=manifest, manifest=manifest,
) )
) )
errors.extend(
_information_governance_evidence_errors(
repository_name=repository_name,
repository_root=repository_root,
manifest=manifest,
)
)
manifests.append(manifest) manifests.append(manifest)
@@ -153,6 +160,20 @@ def main() -> int:
f"Architecture declaration coverage: {declared}/{len(manifests)} modules " f"Architecture declaration coverage: {declared}/{len(manifests)} modules "
f"({(declared / len(manifests) * 100):.1f}%)." f"({(declared / len(manifests) * 100):.1f}%)."
) )
governance_counts: dict[str, int] = {}
for manifest in manifests:
for dimension in manifest.information_governance.dimensions.values():
governance_counts[dimension.adoption] = (
governance_counts.get(dimension.adoption, 0) + 1
)
print(
"Information-governance adoption: "
+ ", ".join(
f"{status}={count}"
for status, count in sorted(governance_counts.items())
)
+ "."
)
return 0 return 0
@@ -201,6 +222,39 @@ def _architecture_evidence_errors(
return errors return errors
def _information_governance_evidence_errors(
*,
repository_name: str,
repository_root: Path,
manifest: object,
) -> list[str]:
declaration = getattr(manifest, "information_governance", None)
if declaration is None:
return [
f"{repository_name}: module has no information-governance declaration"
]
errors: list[str] = []
for dimension_name, dimension in declaration.dimensions.items():
for reference in dimension.evidence:
if not _looks_like_repository_reference(reference):
continue
candidate = (repository_root / reference).resolve()
try:
candidate.relative_to(repository_root.resolve())
except ValueError:
errors.append(
f"{repository_name}: {dimension_name} evidence escapes the "
f"repository: {reference!r}"
)
continue
if not candidate.exists():
errors.append(
f"{repository_name}: {dimension_name} evidence does not exist: "
f"{reference!r}"
)
return errors
def _looks_like_repository_reference(reference: str) -> bool: def _looks_like_repository_reference(reference: str) -> bool:
normalized = reference.strip() normalized = reference.strip()
if not normalized or "://" in normalized: if not normalized or "://" in normalized:
@@ -0,0 +1,306 @@
#!/usr/bin/env python3
"""Prove the governed Connectors -> Risk Compliance sanctions journey."""
from __future__ import annotations
from types import SimpleNamespace
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_connectors.backend.db.models import (
ConnectorSanctionsAcquisitionRun,
ConnectorSanctionsSnapshot,
)
from govoplan_connectors.backend.sanctions_sources import (
SANCTIONS_READ_SCOPE as CONNECTOR_SANCTIONS_READ_SCOPE,
SANCTIONS_REFRESH_SCOPE,
SYNTHETIC_PROVIDER_ID,
)
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.modules import ModuleContext
from govoplan_core.core.recovery import RecoveryCheckpoint, RecoveryOperation
from govoplan_core.core.runtime_coordination import (
DistributedLease,
RuntimeIdentity,
bind_process_runtime_identity,
)
from govoplan_core.core.sanctions import (
SanctionsScreeningFreshnessRequest,
SanctionsScreeningPolicy,
SanctionsScreeningRequest,
SanctionsScreeningSubject,
sanctions_screening_provider,
sanctions_snapshot_provider,
)
from govoplan_core.db.base import Base
from govoplan_core.server.registry import build_platform_registry
from govoplan_risk_compliance.backend.db.models import (
RiskAssuranceEdge,
RiskAssuranceNode,
RiskSanctionsAddress,
RiskSanctionsAlias,
RiskSanctionsDate,
RiskSanctionsEntry,
RiskSanctionsIdentifier,
RiskSanctionsListSnapshot,
RiskScreeningCandidate,
RiskScreeningDisposition,
RiskScreeningException,
RiskScreeningRun,
RiskScreeningSubjectSnapshot,
)
from govoplan_risk_compliance.backend.permissions import (
SANCTIONS_ADMIN_SCOPE,
SANCTIONS_READ_SCOPE,
SANCTIONS_REVIEW_SCOPE,
SANCTIONS_SCREEN_SCOPE,
)
from govoplan_risk_compliance.backend.review import (
DispositionInput,
record_disposition,
)
from govoplan_risk_compliance.backend.sanctions_catalog import (
import_connector_snapshot,
)
from govoplan_risk_compliance.backend.screening import (
get_screening_run,
list_rescreening_requirements,
)
TABLES = (
DistributedLease.__table__,
RecoveryOperation.__table__,
RecoveryCheckpoint.__table__,
ConnectorSanctionsAcquisitionRun.__table__,
ConnectorSanctionsSnapshot.__table__,
RiskAssuranceNode.__table__,
RiskAssuranceEdge.__table__,
RiskSanctionsListSnapshot.__table__,
RiskSanctionsEntry.__table__,
RiskSanctionsAlias.__table__,
RiskSanctionsIdentifier.__table__,
RiskSanctionsDate.__table__,
RiskSanctionsAddress.__table__,
RiskScreeningSubjectSnapshot.__table__,
RiskScreeningRun.__table__,
RiskScreeningCandidate.__table__,
RiskScreeningDisposition.__table__,
RiskScreeningException.__table__,
)
def main() -> int:
registry = build_platform_registry(("connectors", "risk_compliance"))
registry.configure_capability_context(
ModuleContext(registry=registry, settings=object())
)
snapshot_provider = sanctions_snapshot_provider(registry)
screening_provider = sanctions_screening_provider(registry)
refresh_source = getattr(snapshot_provider, "refresh_source", None)
if snapshot_provider is None or not callable(refresh_source):
raise RuntimeError("Connectors sanctions acquisition is unavailable.")
if screening_provider is None:
raise RuntimeError("Risk Compliance sanctions screening is unavailable.")
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(engine, tables=TABLES)
bind_process_runtime_identity(
RuntimeIdentity(
installation_id="sanctions-composition-check",
node_id="sanctions-worker",
incarnation="sanctions-worker-incarnation",
role="worker",
software_version="test",
composition_hash="d" * 64,
)
)
try:
with Session(engine) as session:
operator = _principal("operator-1", operational=True)
reviewer = _principal("reviewer-1", operational=False)
acquired = refresh_source(
session,
operator,
provider_id=SYNTHETIC_PROVIDER_ID,
idempotency_key="synthetic-sanctions-2026-08-01",
)
replay = refresh_source(
session,
operator,
provider_id=SYNTHETIC_PROVIDER_ID,
idempotency_key="synthetic-sanctions-2026-08-01",
)
if acquired.status != "succeeded" or acquired.snapshot is None:
raise RuntimeError(f"Synthetic acquisition failed: {acquired!r}")
if (
replay.run_id != acquired.run_id
or replay.snapshot is None
or replay.snapshot.ref != acquired.snapshot.ref
or replay.snapshot.sha256 != acquired.snapshot.sha256
):
raise RuntimeError("Acquisition idempotency did not replay exact evidence.")
imported, created = import_connector_snapshot(
session,
operator,
registry=registry,
connector_snapshot_ref=acquired.snapshot.ref,
)
imported_replay, replay_created = import_connector_snapshot(
session,
operator,
registry=registry,
connector_snapshot_ref=acquired.snapshot.ref,
)
if not created or replay_created or imported_replay.id != imported.id:
raise RuntimeError("Risk Compliance snapshot import is not idempotent.")
subject = SanctionsScreeningSubject(
subject_type="person",
primary_name="Alex Example",
subject_ref="party:fixture-person-1",
)
policy = SanctionsScreeningPolicy(failure_policy="block")
request = SanctionsScreeningRequest(
list_snapshot_id=imported.id,
idempotency_key="fixture-party-screening-1",
subject=subject,
policy=policy,
)
screened = screening_provider.request_screening(
session,
operator,
request,
)
screened_replay = screening_provider.request_screening(
session,
operator,
request,
)
if not screened.created or screened_replay.created:
raise RuntimeError("Screening request idempotency is not stable.")
if screened.evidence.ref != screened_replay.evidence.ref:
raise RuntimeError("Screening replay returned different evidence.")
if screened.evidence.outcome != "potential" or screened.evidence.candidate_count != 1:
raise RuntimeError(f"Synthetic match was not reviewable: {screened.evidence!r}")
run = get_screening_run(
session,
operator,
run_id=screened.evidence.run_id,
)
candidate, disposition = record_disposition(
session,
reviewer,
candidate_id=run.candidates[0].id,
disposition=DispositionInput(
decision="false_positive",
reason="Independent fixture evidence excludes the screened party.",
evidence_refs=(acquired.snapshot.raw_evidence_ref,),
),
)
if candidate.review_status != "false_positive":
raise RuntimeError("Independent review did not resolve the candidate.")
if disposition.separation_status != "independent":
raise RuntimeError("Reviewer separation evidence was not retained.")
cleared = screening_provider.check_freshness(
session,
operator,
SanctionsScreeningFreshnessRequest(
evidence_ref=screened.evidence.ref,
current_subject=subject,
expected_list_snapshot_id=imported.id,
policy=policy,
),
)
if not cleared.fresh or cleared.gate_decision != "allow":
raise RuntimeError(f"Reviewed evidence did not clear the gate: {cleared!r}")
# Acquisition and review are separate durable commands in production.
session.commit()
session.expire_all()
refreshed = refresh_source(
session,
operator,
provider_id=SYNTHETIC_PROVIDER_ID,
idempotency_key="synthetic-sanctions-2026-08-02",
)
if refreshed.snapshot is None or refreshed.snapshot.ref == acquired.snapshot.ref:
raise RuntimeError("A new acquisition did not create new immutable evidence.")
current, current_created = import_connector_snapshot(
session,
operator,
registry=registry,
connector_snapshot_ref=refreshed.snapshot.ref,
)
if not current_created:
raise RuntimeError("The refreshed list state was not imported separately.")
stale = screening_provider.check_freshness(
session,
operator,
SanctionsScreeningFreshnessRequest(
evidence_ref=screened.evidence.ref,
current_subject=subject,
expected_list_snapshot_id=current.id,
policy=policy,
),
)
if stale.fresh or stale.gate_decision != "block":
raise RuntimeError(f"Changed source evidence did not close the gate: {stale!r}")
if "source_snapshot_changed" not in stale.reasons:
raise RuntimeError("Source change provenance was not reported.")
requirements = list_rescreening_requirements(session, operator)
if screened.evidence.run_id not in {item.run.id for item in requirements}:
raise RuntimeError("The stale screening is absent from the rescreening queue.")
session.commit()
if session.query(RecoveryOperation).count() != 2:
raise RuntimeError("Connector acquisition recovery evidence is incomplete.")
if session.query(RiskScreeningDisposition).count() != 1:
raise RuntimeError("Disposition evidence was duplicated or lost.")
finally:
bind_process_runtime_identity(None)
engine.dispose()
print(
"Connectors -> immutable sanctions snapshot -> Risk Compliance review "
"and rescreening composition passed."
)
return 0
def _principal(account_id: str, *, operational: bool) -> ApiPrincipal:
scopes = {
SANCTIONS_READ_SCOPE,
SANCTIONS_REVIEW_SCOPE,
}
if operational:
scopes.update(
{
CONNECTOR_SANCTIONS_READ_SCOPE,
SANCTIONS_REFRESH_SCOPE,
SANCTIONS_ADMIN_SCOPE,
SANCTIONS_SCREEN_SCOPE,
}
)
return ApiPrincipal(
principal=PrincipalRef(
account_id=account_id,
membership_id=f"membership-{account_id}",
tenant_id="tenant-1",
scopes=frozenset(scopes),
),
account=SimpleNamespace(id=account_id),
user=SimpleNamespace(id=f"membership-{account_id}"),
)
if __name__ == "__main__":
raise SystemExit(main())
+9 -1
View File
@@ -26,6 +26,7 @@ CADDY_CONFIG_FILENAME = "Caddyfile"
EXISTING_PROXY_FILENAME = "existing-proxy.json" EXISTING_PROXY_FILENAME = "existing-proxy.json"
PLAN_FILENAME = "plan.json" PLAN_FILENAME = "plan.json"
RECEIPT_FILENAME = "receipt.json" RECEIPT_FILENAME = "receipt.json"
CAPABILITIES_FILENAME = "infrastructure-capabilities.json"
MANIFEST_FILENAME = "distribution-manifest.json" MANIFEST_FILENAME = "distribution-manifest.json"
KEYRING_FILENAME = "distribution-keyring.json" KEYRING_FILENAME = "distribution-keyring.json"
BACKUP_EVIDENCE_FILENAME = "backup-evidence.json" BACKUP_EVIDENCE_FILENAME = "backup-evidence.json"
@@ -88,6 +89,7 @@ RUNTIME_ENV_KEYS = (
"DEV_BOOTSTRAP_ENABLED", "DEV_BOOTSTRAP_ENABLED",
"GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE", "GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE",
"GOVOPLAN_DEPLOYMENT_SPEC_PATH", "GOVOPLAN_DEPLOYMENT_SPEC_PATH",
"GOVOPLAN_DEPLOYMENT_CAPABILITIES_PATH",
"FILE_STORAGE_BACKEND", "FILE_STORAGE_BACKEND",
"FILE_STORAGE_LOCAL_ROOT", "FILE_STORAGE_LOCAL_ROOT",
"FILE_STORAGE_S3_ENDPOINT_URL", "FILE_STORAGE_S3_ENDPOINT_URL",
@@ -113,6 +115,7 @@ class BundlePaths:
existing_proxy: Path existing_proxy: Path
plan: Path plan: Path
receipt: Path receipt: Path
capabilities: Path
manifest: Path manifest: Path
keyring: Path keyring: Path
backup_evidence: Path backup_evidence: Path
@@ -137,6 +140,7 @@ def bundle_paths(root: Path) -> BundlePaths:
existing_proxy=resolved / EXISTING_PROXY_FILENAME, existing_proxy=resolved / EXISTING_PROXY_FILENAME,
plan=resolved / PLAN_FILENAME, plan=resolved / PLAN_FILENAME,
receipt=resolved / RECEIPT_FILENAME, receipt=resolved / RECEIPT_FILENAME,
capabilities=resolved / CAPABILITIES_FILENAME,
manifest=resolved / MANIFEST_FILENAME, manifest=resolved / MANIFEST_FILENAME,
keyring=resolved / KEYRING_FILENAME, keyring=resolved / KEYRING_FILENAME,
backup_evidence=resolved / BACKUP_EVIDENCE_FILENAME, backup_evidence=resolved / BACKUP_EVIDENCE_FILENAME,
@@ -296,6 +300,7 @@ def reconcile_runtime_environment(
"DEV_AUTO_MIGRATE_ENABLED": "false", "DEV_AUTO_MIGRATE_ENABLED": "false",
"DEV_BOOTSTRAP_ENABLED": "false", "DEV_BOOTSTRAP_ENABLED": "false",
"GOVOPLAN_DEPLOYMENT_SPEC_PATH": "/etc/govoplan/deployment/installation.json", "GOVOPLAN_DEPLOYMENT_SPEC_PATH": "/etc/govoplan/deployment/installation.json",
"GOVOPLAN_DEPLOYMENT_CAPABILITIES_PATH": "/etc/govoplan/deployment/infrastructure-capabilities.json",
} }
) )
if redis.mode == "disabled": if redis.mode == "disabled":
@@ -370,7 +375,10 @@ def render_compose(spec: InstallationSpec) -> dict[str, object]:
deployment_mount = ( deployment_mount = (
f"./{SPEC_FILENAME}:/etc/govoplan/deployment/installation.json:ro" f"./{SPEC_FILENAME}:/etc/govoplan/deployment/installation.json:ro"
) )
data_mounts = [deployment_mount] capabilities_mount = (
f"./{CAPABILITIES_FILENAME}:/etc/govoplan/deployment/infrastructure-capabilities.json:ro"
)
data_mounts = [deployment_mount, capabilities_mount]
if spec.components.storage.mode == "local": if spec.components.storage.mode == "local":
data_mounts.append("files-data:/var/lib/govoplan/files") data_mounts.append("files-data:/var/lib/govoplan/files")
@@ -0,0 +1,489 @@
"""Non-secret infrastructure capability projection and change impact."""
from __future__ import annotations
from dataclasses import asdict, dataclass
from typing import Mapping
from urllib.parse import urlsplit
from .model import InstallationSpec
CAPABILITY_DOCUMENT_SCHEMA_VERSION = 1
CAPABILITY_STATES = frozenset(
{
"configured",
"available_unconfigured",
"externally_supplied",
"unavailable",
}
)
@dataclass(frozen=True, slots=True)
class InfrastructureCapability:
id: str
label: str
state: str
source: str
detail: str
endpoint: Mapping[str, object]
secret_refs: tuple[str, ...]
dependent_modules: tuple[str, ...]
def to_dict(self) -> dict[str, object]:
value = asdict(self)
value["endpoint"] = dict(self.endpoint)
value["secret_refs"] = list(self.secret_refs)
value["dependent_modules"] = list(self.dependent_modules)
return value
@dataclass(frozen=True, slots=True)
class CapabilityChangeImpact:
capability_id: str
action: str
previous_state: str
desired_state: str
previous_source: str
desired_source: str
dependent_modules: tuple[str, ...]
detail: str
required_action: str
def to_dict(self) -> dict[str, object]:
value = asdict(self)
value["dependent_modules"] = list(self.dependent_modules)
return value
def infrastructure_capability_document(
spec: InstallationSpec,
environment: Mapping[str, str],
) -> dict[str, object]:
"""Project installer choices without copying credentials or secret URLs."""
capabilities = tuple(
sorted(
(
_postgres_capability(spec, environment),
_redis_capability(spec, environment),
_mail_capability(spec),
_storage_capability(spec, environment),
_load_balancer_capability(spec),
_ingress_capability(spec),
),
key=lambda item: item.id,
)
)
tasks = _post_install_tasks(spec, capabilities)
return {
"schema_version": CAPABILITY_DOCUMENT_SCHEMA_VERSION,
"installation_id": spec.installation_id,
"profile": spec.profile,
"capabilities": [item.to_dict() for item in capabilities],
"post_install_tasks": tasks,
}
def capability_change_impacts(
previous_document: object,
desired_document: Mapping[str, object],
) -> tuple[CapabilityChangeImpact, ...]:
previous = _capability_map(previous_document)
desired = _capability_map(desired_document)
if not previous:
return ()
impacts: list[CapabilityChangeImpact] = []
for capability_id in sorted(set(previous) | set(desired)):
before = previous.get(capability_id)
after = desired.get(capability_id)
if before is None or after is None:
continue
previous_state = str(before.get("state") or "unavailable")
desired_state = str(after.get("state") or "unavailable")
previous_source = str(before.get("source") or "unknown")
desired_source = str(after.get("source") or "unknown")
previous_endpoint = _endpoint_signature(before.get("endpoint"))
desired_endpoint = _endpoint_signature(after.get("endpoint"))
previous_secret_refs = tuple(
sorted(_string_items(before.get("secret_refs")))
)
desired_secret_refs = tuple(
sorted(_string_items(after.get("secret_refs")))
)
if (
previous_state == desired_state
and previous_source == desired_source
and previous_endpoint == desired_endpoint
and previous_secret_refs == desired_secret_refs
):
continue
action = (
"remove"
if previous_state != "unavailable" and desired_state == "unavailable"
else "replace"
if previous_source != desired_source
else "reconfigure"
)
dependents = tuple(
sorted(
{
*(_string_items(before.get("dependent_modules"))),
*(_string_items(after.get("dependent_modules"))),
}
)
)
dependent_label = ", ".join(dependents) or "no declared module consumers"
binding_change = _binding_change_label(
previous_endpoint,
desired_endpoint,
previous_secret_refs,
desired_secret_refs,
)
impacts.append(
CapabilityChangeImpact(
capability_id=capability_id,
action=action,
previous_state=previous_state,
desired_state=desired_state,
previous_source=previous_source,
desired_source=desired_source,
dependent_modules=dependents,
detail=(
f"{capability_id} changes from {previous_state}/{previous_source} "
f"to {desired_state}/{desired_source}{binding_change}; "
f"declared consumers: {dependent_label}."
),
required_action=(
"Review module-owned configuration and data migration or recovery evidence before apply."
),
)
)
return tuple(impacts)
def _postgres_capability(
spec: InstallationSpec,
environment: Mapping[str, str],
) -> InfrastructureCapability:
managed = spec.components.postgres.mode == "managed"
endpoint = (
{"scheme": "postgresql", "host": "postgres", "port": 5432}
if managed
else _redacted_endpoint(environment.get("DATABASE_URL", ""), default_port=5432)
)
return InfrastructureCapability(
id="database.postgresql",
label="PostgreSQL database",
state="configured" if managed else "externally_supplied",
source="installer-managed" if managed else "operator-supplied",
detail=(
"The installer manages the database service."
if managed
else "The deployment binds an externally operated PostgreSQL service."
),
endpoint=endpoint,
secret_refs=("env:POSTGRES_PASSWORD",) if managed else ("env:DATABASE_URL",),
dependent_modules=("core", *tuple(sorted(spec.enabled_modules))),
)
def _redis_capability(
spec: InstallationSpec,
environment: Mapping[str, str],
) -> InfrastructureCapability:
mode = spec.components.redis.mode
consumers = _enabled_consumers(
spec,
{
"campaigns",
"dataflow",
"files",
"mail",
"notifications",
"scheduling",
"workflow_engine",
},
include_core=True,
)
if mode == "disabled":
return InfrastructureCapability(
id="coordination.redis",
label="Redis coordination and queues",
state="unavailable",
source="disabled",
detail="Distributed queues and coordination are disabled.",
endpoint={},
secret_refs=(),
dependent_modules=consumers,
)
managed = mode == "managed"
endpoint = (
{"scheme": "redis", "host": "redis", "port": 6379}
if managed
else _redacted_endpoint(environment.get("REDIS_URL", ""), default_port=6379)
)
return InfrastructureCapability(
id="coordination.redis",
label="Redis coordination and queues",
state="configured" if managed else "externally_supplied",
source="installer-managed" if managed else "operator-supplied",
detail=(
"The installer manages the Redis service."
if managed
else "The deployment binds an externally operated Redis service."
),
endpoint=endpoint,
secret_refs=("env:REDIS_PASSWORD",) if managed else ("env:REDIS_URL",),
dependent_modules=consumers,
)
def _mail_capability(spec: InstallationSpec) -> InfrastructureCapability:
mode = spec.components.mail.mode
consumers = _enabled_consumers(
spec,
{"campaigns", "mail", "notifications"},
)
if mode == "disabled":
return InfrastructureCapability(
id="mail.smtp",
label="SMTP delivery",
state="unavailable",
source="disabled",
detail="No SMTP infrastructure was selected.",
endpoint={},
secret_refs=(),
dependent_modules=consumers,
)
if mode == "test-mail":
return InfrastructureCapability(
id="mail.smtp",
label="SMTP delivery",
state="available_unconfigured",
source="installer-managed-test",
detail="GreenMail is reachable, but Mail still owns profile and credential configuration.",
endpoint={"scheme": "smtp", "host": "test-mail", "port": 3025},
secret_refs=(),
dependent_modules=consumers,
)
return InfrastructureCapability(
id="mail.smtp",
label="SMTP delivery",
state="available_unconfigured",
source="operator-supplied",
detail="An external relay was selected; Mail still needs a reviewed server and credential binding.",
endpoint={},
secret_refs=(),
dependent_modules=consumers,
)
def _storage_capability(
spec: InstallationSpec,
environment: Mapping[str, str],
) -> InfrastructureCapability:
mode = spec.components.storage.mode
consumers = _enabled_consumers(
spec,
{"campaigns", "files", "records", "templates"},
)
if mode == "local":
return InfrastructureCapability(
id="files.storage",
label="Managed file content storage",
state="configured",
source="host-local",
detail="Files use the installer-managed local persistent volume.",
endpoint={"kind": "filesystem", "reference": "volume:files-data"},
secret_refs=(),
dependent_modules=consumers,
)
if mode == "garage":
return InfrastructureCapability(
id="files.storage",
label="Managed file content storage",
state="configured",
source="installer-managed-garage",
detail="Files use the installer-managed single-node Garage service.",
endpoint={"scheme": "http", "host": "garage", "port": 3900},
secret_refs=(
"env:FILE_STORAGE_S3_ACCESS_KEY_ID",
"env:FILE_STORAGE_S3_SECRET_ACCESS_KEY",
"env:GARAGE_RPC_SECRET",
),
dependent_modules=consumers,
)
return InfrastructureCapability(
id="files.storage",
label="Managed file content storage",
state="externally_supplied",
source="operator-supplied-s3",
detail="Files use an externally operated S3-compatible service.",
endpoint=_redacted_endpoint(
environment.get("FILE_STORAGE_S3_ENDPOINT_URL", ""),
default_port=443,
),
secret_refs=(
"env:FILE_STORAGE_S3_ACCESS_KEY_ID",
"env:FILE_STORAGE_S3_SECRET_ACCESS_KEY",
),
dependent_modules=consumers,
)
def _load_balancer_capability(spec: InstallationSpec) -> InfrastructureCapability:
return InfrastructureCapability(
id="runtime.load_balancing",
label="Application load balancing",
state="configured",
source="installer-managed",
detail=(
f"HAProxy balances {spec.replicas.web} WebUI and {spec.replicas.api} API replica(s)."
),
endpoint={"scheme": "http", "host": "load-balancer", "port": 8080},
secret_refs=(),
dependent_modules=("core", "ops"),
)
def _ingress_capability(spec: InstallationSpec) -> InfrastructureCapability:
mode = spec.ingress.mode
endpoint = _redacted_endpoint(spec.public_url, default_port=443)
if mode == "unconfigured":
state = "unavailable"
source = "unconfigured"
detail = "No supported public ingress boundary is configured."
elif mode == "existing-proxy":
state = "externally_supplied"
source = "operator-supplied-proxy"
detail = "An externally operated reverse proxy provides public ingress."
else:
state = "configured"
source = "installer-managed" if mode == "managed" else "host-local"
detail = "The installer has a bounded public ingress configuration."
return InfrastructureCapability(
id="network.ingress",
label="Public HTTP ingress",
state=state,
source=source,
detail=detail,
endpoint=endpoint,
secret_refs=(),
dependent_modules=("core", "ops"),
)
def _post_install_tasks(
spec: InstallationSpec,
capabilities: tuple[InfrastructureCapability, ...],
) -> list[dict[str, object]]:
by_id = {item.id: item for item in capabilities}
tasks: list[dict[str, object]] = []
mail = by_id["mail.smtp"]
if mail.state == "available_unconfigured" and "mail" in spec.enabled_modules:
tasks.append(
{
"id": "mail.smtp-profile",
"resume_key": f"{spec.installation_id}:mail.smtp-profile:v1",
"capability_id": mail.id,
"state": "pending",
"owner_module": "mail",
"summary": "Create or select a Mail SMTP server and credential envelope.",
"required_inputs": [
"server endpoint",
"transport security policy",
"credential envelope reference when authentication is required",
],
"secret_boundary": "credential-envelope-reference-only",
}
)
ingress = by_id["network.ingress"]
if ingress.state == "unavailable":
tasks.append(
{
"id": "network.configure-ingress",
"resume_key": f"{spec.installation_id}:network.configure-ingress:v1",
"capability_id": ingress.id,
"state": "pending",
"owner_module": "ops",
"summary": "Select managed ingress or bind an existing reverse proxy.",
"required_inputs": ["public URL", "TLS and proxy trust boundary"],
"secret_boundary": "no-secret-material",
}
)
return tasks
def _enabled_consumers(
spec: InstallationSpec,
candidates: set[str],
*,
include_core: bool = False,
) -> tuple[str, ...]:
consumers = candidates.intersection(spec.enabled_modules)
if include_core:
consumers.add("core")
return tuple(sorted(consumers))
def _redacted_endpoint(value: str, *, default_port: int) -> dict[str, object]:
try:
parsed = urlsplit(value)
host = parsed.hostname
port = parsed.port or default_port
except ValueError:
return {"reference": "unresolved"}
if not parsed.scheme or not host:
return {"reference": "unresolved"}
return {"scheme": parsed.scheme, "host": host, "port": port}
def _capability_map(value: object) -> dict[str, Mapping[str, object]]:
if not isinstance(value, Mapping):
return {}
raw_items = value.get("capabilities")
if not isinstance(raw_items, list):
return {}
result: dict[str, Mapping[str, object]] = {}
for item in raw_items:
if not isinstance(item, Mapping):
continue
capability_id = str(item.get("id") or "").strip()
state = str(item.get("state") or "").strip()
if capability_id and state in CAPABILITY_STATES:
result[capability_id] = item
return result
def _string_items(value: object) -> tuple[str, ...]:
if not isinstance(value, list):
return ()
return tuple(str(item).strip() for item in value if str(item).strip())
def _endpoint_signature(value: object) -> tuple[tuple[str, str], ...]:
if not isinstance(value, Mapping):
return ()
return tuple(
sorted(
(str(key), str(raw))
for key, raw in value.items()
if isinstance(key, str) and isinstance(raw, (str, int, bool))
)
)
def _binding_change_label(
previous_endpoint: tuple[tuple[str, str], ...],
desired_endpoint: tuple[tuple[str, str], ...],
previous_secret_refs: tuple[str, ...],
desired_secret_refs: tuple[str, ...],
) -> str:
changes: list[str] = []
if previous_endpoint != desired_endpoint:
changes.append("endpoint binding")
if previous_secret_refs != desired_secret_refs:
changes.append("secret-reference binding")
return f" with changed {' and '.join(changes)}" if changes else ""
+24 -2
View File
@@ -45,6 +45,7 @@ from .bundle import (
service_names, service_names,
write_env, write_env,
) )
from .capabilities import infrastructure_capability_document
from .cluster_evidence import collect_kubernetes_evidence from .cluster_evidence import collect_kubernetes_evidence
from .distribution import ( from .distribution import (
MAX_KEYRING_BYTES, MAX_KEYRING_BYTES,
@@ -219,6 +220,13 @@ def build_parser() -> argparse.ArgumentParser:
kubernetes.add_argument("--namespace", default="govoplan") kubernetes.add_argument("--namespace", default="govoplan")
kubernetes.add_argument("--secret-name", default="govoplan-runtime") kubernetes.add_argument("--secret-name", default="govoplan-runtime")
kubernetes.add_argument("--tls-secret-name", default="govoplan-tls") kubernetes.add_argument("--tls-secret-name", default="govoplan-tls")
kubernetes.add_argument(
"--s3-ca-secret-name",
help=(
"Optional Secret containing ca.crt for the external S3 endpoint; "
"mounted read-only into backend runtime roles."
),
)
kubernetes.add_argument("--ingress-class-name") kubernetes.add_argument("--ingress-class-name")
kubernetes.add_argument( kubernetes.add_argument(
"--output", "--output",
@@ -239,7 +247,10 @@ def build_parser() -> argparse.ArgumentParser:
verify_kubernetes.add_argument( verify_kubernetes.add_argument(
"--api-key-env", "--api-key-env",
default="GOVOPLAN_OPS_API_KEY", default="GOVOPLAN_OPS_API_KEY",
help="Environment variable containing an API key with Ops read scope.", help=(
"Environment variable containing an API key authorized to read "
"Ops status."
),
) )
verify_kubernetes.add_argument( verify_kubernetes.add_argument(
"--exercise-api-pod-loss", "--exercise-api-pod-loss",
@@ -1146,6 +1157,7 @@ def _render_kubernetes(args: argparse.Namespace) -> int:
namespace=args.namespace, namespace=args.namespace,
secret_name=args.secret_name, secret_name=args.secret_name,
tls_secret_name=args.tls_secret_name, tls_secret_name=args.tls_secret_name,
s3_ca_secret_name=args.s3_ca_secret_name,
ingress_class_name=args.ingress_class_name, ingress_class_name=args.ingress_class_name,
backup_required=backup_required, backup_required=backup_required,
backup_evidence=backup_summary, backup_evidence=backup_summary,
@@ -1170,7 +1182,8 @@ def _verify_kubernetes(args: argparse.Namespace) -> int:
api_key = str(os.environ.get(args.api_key_env) or "").strip() api_key = str(os.environ.get(args.api_key_env) or "").strip()
if not api_key: if not api_key:
raise ValueError( raise ValueError(
f"{args.api_key_env} must contain an API key with Ops read scope" f"{args.api_key_env} must contain an API key authorized to read "
"Ops status"
) )
ops_url = args.ops_url or (spec.public_url.rstrip("/") + "/api/v1/ops/status") ops_url = args.ops_url or (spec.public_url.rstrip("/") + "/api/v1/ops/status")
evidence = collect_kubernetes_evidence( evidence = collect_kubernetes_evidence(
@@ -1275,6 +1288,10 @@ def _deployment_receipt(
"agent": "cli", "agent": "cli",
"web_updates": False, "web_updates": False,
}, },
"infrastructure_capabilities": infrastructure_capability_document(
spec,
secrets,
),
} }
@@ -1452,6 +1469,11 @@ def _write_bundle(
runtime_environment.update(_backup_runtime_environment(spec, paths)) runtime_environment.update(_backup_runtime_environment(spec, paths))
atomic_write(paths.spec, canonical_json(spec.to_dict()), mode=0o600) atomic_write(paths.spec, canonical_json(spec.to_dict()), mode=0o600)
write_env(paths.env, runtime_environment) write_env(paths.env, runtime_environment)
atomic_write(
paths.capabilities,
canonical_json(infrastructure_capability_document(spec, runtime_environment)),
mode=0o644,
)
atomic_write(paths.compose, canonical_json(render_compose(spec)), mode=0o600) atomic_write(paths.compose, canonical_json(render_compose(spec)), mode=0o600)
atomic_write( atomic_write(
paths.load_balancer_config, paths.load_balancer_config,
@@ -14,6 +14,7 @@ from urllib.request import Request, urlopen
JsonObject = dict[str, Any] JsonObject = dict[str, Any]
CommandRunner = Callable[[Sequence[str]], JsonObject] CommandRunner = Callable[[Sequence[str]], JsonObject]
ActionRunner = Callable[[Sequence[str]], None]
JsonFetcher = Callable[[str, str], JsonObject] JsonFetcher = Callable[[str, str], JsonObject]
@@ -26,11 +27,13 @@ def collect_kubernetes_evidence(
exercise_api_pod_loss: bool = False, exercise_api_pod_loss: bool = False,
timeout_seconds: float = 180.0, timeout_seconds: float = 180.0,
command_runner: CommandRunner | None = None, command_runner: CommandRunner | None = None,
action_runner: ActionRunner | None = None,
json_fetcher: JsonFetcher | None = None, json_fetcher: JsonFetcher | None = None,
) -> JsonObject: ) -> JsonObject:
"""Inspect a live cluster and optionally exercise one API pod replacement.""" """Inspect a live cluster and optionally exercise one API pod replacement."""
run_json = command_runner or _kubectl_json run_json = command_runner or _kubectl_json
run_action = action_runner or _kubectl_action
fetch_json = json_fetcher or _fetch_json fetch_json = json_fetcher or _fetch_json
nodes = run_json(("get", "nodes", "-o", "json")) nodes = run_json(("get", "nodes", "-o", "json"))
pods = run_json( pods = run_json(
@@ -76,6 +79,7 @@ def collect_kubernetes_evidence(
initial_pods=pods, initial_pods=pods,
timeout_seconds=timeout_seconds, timeout_seconds=timeout_seconds,
run_json=run_json, run_json=run_json,
run_action=run_action,
fetch_json=fetch_json, fetch_json=fetch_json,
) )
evidence = { evidence = {
@@ -211,6 +215,7 @@ def _exercise_api_pod_loss(
initial_pods: Mapping[str, Any], initial_pods: Mapping[str, Any],
timeout_seconds: float, timeout_seconds: float,
run_json: CommandRunner, run_json: CommandRunner,
run_action: ActionRunner,
fetch_json: JsonFetcher, fetch_json: JsonFetcher,
) -> JsonObject: ) -> JsonObject:
candidates = [ candidates = [
@@ -228,7 +233,7 @@ def _exercise_api_pod_loss(
victim = sorted(candidates, key=lambda item: item["name"])[0] victim = sorted(candidates, key=lambda item: item["name"])[0]
initial_uids = {item["uid"] for item in candidates} initial_uids = {item["uid"] for item in candidates}
desired_ready = len(candidates) desired_ready = len(candidates)
run_json( run_action(
( (
"-n", "-n",
namespace, namespace,
@@ -236,8 +241,6 @@ def _exercise_api_pod_loss(
"pod", "pod",
victim["name"], victim["name"],
"--wait=false", "--wait=false",
"-o",
"json",
) )
) )
deadline = time.monotonic() + timeout_seconds deadline = time.monotonic() + timeout_seconds
@@ -309,6 +312,22 @@ def _kubectl_json(arguments: Sequence[str]) -> JsonObject:
return payload return payload
def _kubectl_action(arguments: Sequence[str]) -> None:
kubectl = shutil.which("kubectl")
if kubectl is None:
raise ValueError("kubectl is required for Kubernetes evidence collection")
result = subprocess.run(
(kubectl, *arguments),
check=False,
capture_output=True,
text=True,
timeout=60,
)
if result.returncode:
detail = result.stderr.strip() or result.stdout.strip()
raise ValueError(f"kubectl failed: {detail}")
def _fetch_json(url: str, api_key: str) -> JsonObject: def _fetch_json(url: str, api_key: str) -> JsonObject:
request = Request( request = Request(
url, url,
+161 -5
View File
@@ -11,6 +11,7 @@ from typing import Any, Mapping
from urllib.parse import urlsplit from urllib.parse import urlsplit
from .bundle import BACKUP_RUNTIME_ENV_KEYS from .bundle import BACKUP_RUNTIME_ENV_KEYS
from .capabilities import infrastructure_capability_document
from .model import InstallationSpec, image_is_digest_pinned from .model import InstallationSpec, image_is_digest_pinned
@@ -76,19 +77,27 @@ def render_kubernetes(
namespace: str = "govoplan", namespace: str = "govoplan",
secret_name: str = "govoplan-runtime", secret_name: str = "govoplan-runtime",
tls_secret_name: str = "govoplan-tls", tls_secret_name: str = "govoplan-tls",
s3_ca_secret_name: str | None = None,
ingress_class_name: str | None = None, ingress_class_name: str | None = None,
backup_required: bool = True, backup_required: bool = True,
backup_evidence: Mapping[str, object] | None = None, backup_evidence: Mapping[str, object] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Render runtime roles only; shared state services stay externally managed.""" """Render runtime roles only; shared state services stay externally managed."""
_validate_cluster_profile(spec, environment, namespace, secret_name) _validate_cluster_profile(
spec,
environment,
namespace,
secret_name,
s3_ca_secret_name=s3_ca_secret_name,
)
worker_pools = _worker_pools(spec, environment) worker_pools = _worker_pools(spec, environment)
database_capacity = _database_capacity(spec, environment, worker_pools) database_capacity = _database_capacity(spec, environment, worker_pools)
name = _resource_name(spec.installation_id) name = _resource_name(spec.installation_id)
public_host = urlsplit(spec.public_url).hostname or "localhost" public_host = urlsplit(spec.public_url).hostname or "localhost"
labels = {"app.kubernetes.io/name": "govoplan", "app.kubernetes.io/instance": name} labels = {"app.kubernetes.io/name": "govoplan", "app.kubernetes.io/instance": name}
config_name = f"{name}-runtime" config_name = f"{name}-runtime"
capabilities_config_name = f"{name}-infrastructure-capabilities"
service_account = f"{name}-runtime" service_account = f"{name}-runtime"
config = { config = {
key: str(environment[key]) key: str(environment[key])
@@ -135,6 +144,22 @@ def render_kubernetes(
"metadata": {"name": config_name, "namespace": namespace, "labels": labels}, "metadata": {"name": config_name, "namespace": namespace, "labels": labels},
"data": dict(sorted(config.items())), "data": dict(sorted(config.items())),
}, },
{
"apiVersion": "v1",
"kind": "ConfigMap",
"metadata": {
"name": capabilities_config_name,
"namespace": namespace,
"labels": labels,
},
"data": {
"infrastructure-capabilities.json": json.dumps(
infrastructure_capability_document(spec, environment),
sort_keys=True,
separators=(",", ":"),
)
},
},
_deployment( _deployment(
name=f"{name}-api", name=f"{name}-api",
namespace=namespace, namespace=namespace,
@@ -154,12 +179,15 @@ def render_kubernetes(
"--proxy-headers", "--proxy-headers",
), ),
config_name=config_name, config_name=config_name,
capabilities_config_name=capabilities_config_name,
secret_name=secret_name, secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account, service_account=service_account,
container_port=8000, container_port=8000,
readiness_path="/health/ready", readiness_path="/health/ready",
liveness_path="/health", liveness_path="/health",
probe_host=public_host, probe_host=public_host,
graceful_shutdown_seconds=10,
extra_environment=_role_database_environment(environment, "API"), extra_environment=_role_database_environment(environment, "API"),
), ),
_service( _service(
@@ -178,7 +206,9 @@ def render_kubernetes(
image=spec.release.web_image, image=spec.release.web_image,
command=(), command=(),
config_name=None, config_name=None,
capabilities_config_name=None,
secret_name=None, secret_name=None,
s3_ca_secret_name=None,
service_account=service_account, service_account=service_account,
container_port=8080, container_port=8080,
extra_environment={"GOVOPLAN_API_UPSTREAM": f"http://{name}-api:8000"}, extra_environment={"GOVOPLAN_API_UPSTREAM": f"http://{name}-api:8000"},
@@ -198,6 +228,7 @@ def render_kubernetes(
image=spec.release.api_image, image=spec.release.api_image,
config_name=config_name, config_name=config_name,
secret_name=secret_name, secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account, service_account=service_account,
database_environment=_role_database_environment( database_environment=_role_database_environment(
environment, environment,
@@ -236,7 +267,9 @@ def render_kubernetes(
"INFO", "INFO",
), ),
config_name=config_name, config_name=config_name,
capabilities_config_name=capabilities_config_name,
secret_name=secret_name, secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account, service_account=service_account,
extra_environment={ extra_environment={
**_role_database_environment(environment, "WORKER"), **_role_database_environment(environment, "WORKER"),
@@ -287,9 +320,13 @@ def render_kubernetes(
"beat", "beat",
"--loglevel", "--loglevel",
"INFO", "INFO",
"--schedule",
"/tmp/celerybeat-schedule",
), ),
config_name=config_name, config_name=config_name,
capabilities_config_name=capabilities_config_name,
secret_name=secret_name, secret_name=secret_name,
s3_ca_secret_name=s3_ca_secret_name,
service_account=service_account, service_account=service_account,
extra_environment=_role_database_environment( extra_environment=_role_database_environment(
environment, environment,
@@ -318,6 +355,7 @@ def render_kubernetes(
"annotations": { "annotations": {
"govoplan.add-ideas.de/profile": "stateless-shared-state", "govoplan.add-ideas.de/profile": "stateless-shared-state",
"govoplan.add-ideas.de/secret-contract": ",".join(_SECRET_KEYS), "govoplan.add-ideas.de/secret-contract": ",".join(_SECRET_KEYS),
"govoplan.add-ideas.de/s3-ca-secret": s3_ca_secret_name or "",
"govoplan.add-ideas.de/database-connection-peak": str( "govoplan.add-ideas.de/database-connection-peak": str(
database_capacity["peak"] database_capacity["peak"]
), ),
@@ -342,9 +380,13 @@ def _validate_cluster_profile(
environment: Mapping[str, str], environment: Mapping[str, str],
namespace: str, namespace: str,
secret_name: str, secret_name: str,
*,
s3_ca_secret_name: str | None,
) -> None: ) -> None:
if not _DNS_LABEL.fullmatch(namespace) or not _DNS_LABEL.fullmatch(secret_name): if not _DNS_LABEL.fullmatch(namespace) or not _DNS_LABEL.fullmatch(secret_name):
raise ValueError("Kubernetes namespace and secret names must be DNS labels") raise ValueError("Kubernetes namespace and secret names must be DNS labels")
if s3_ca_secret_name is not None and not _DNS_LABEL.fullmatch(s3_ca_secret_name):
raise ValueError("Kubernetes S3 CA secret name must be a DNS label")
if spec.installation_id == "govoplan-local": if spec.installation_id == "govoplan-local":
raise ValueError( raise ValueError(
"Kubernetes export requires a non-default stable installation id" "Kubernetes export requires a non-default stable installation id"
@@ -632,12 +674,15 @@ def _deployment(
image: str, image: str,
command: tuple[str, ...], command: tuple[str, ...],
config_name: str | None, config_name: str | None,
capabilities_config_name: str | None,
secret_name: str | None, secret_name: str | None,
s3_ca_secret_name: str | None,
service_account: str, service_account: str,
container_port: int | None = None, container_port: int | None = None,
readiness_path: str | None = None, readiness_path: str | None = None,
liveness_path: str | None = None, liveness_path: str | None = None,
probe_host: str | None = None, probe_host: str | None = None,
graceful_shutdown_seconds: int = 0,
extra_environment: Mapping[str, str] | None = None, extra_environment: Mapping[str, str] | None = None,
selector_labels: Mapping[str, str] | None = None, selector_labels: Mapping[str, str] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
@@ -660,6 +705,17 @@ def _deployment(
) )
if secret_name: if secret_name:
environment.extend(_secret_environment(secret_name)) environment.extend(_secret_environment(secret_name))
if capabilities_config_name:
environment.append(
{
"name": "GOVOPLAN_DEPLOYMENT_CAPABILITIES_PATH",
"value": "/etc/govoplan/deployment/infrastructure-capabilities.json",
}
)
if s3_ca_secret_name:
environment.append(
{"name": "AWS_CA_BUNDLE", "value": "/etc/govoplan/trust/s3-ca.crt"}
)
container: dict[str, Any] = { container: dict[str, Any] = {
"name": role, "name": role,
"image": image, "image": image,
@@ -691,6 +747,18 @@ def _deployment(
container_port or 8000, container_port or 8000,
host=probe_host, host=probe_host,
) )
if graceful_shutdown_seconds:
container["lifecycle"] = {
"preStop": {
"exec": {
"command": [
"/bin/sh",
"-c",
f"sleep {graceful_shutdown_seconds}",
]
}
}
}
pod_spec: dict[str, Any] = { pod_spec: dict[str, Any] = {
"serviceAccountName": service_account, "serviceAccountName": service_account,
"automountServiceAccountToken": False, "automountServiceAccountToken": False,
@@ -704,12 +772,44 @@ def _deployment(
{ {
"maxSkew": 1, "maxSkew": 1,
"topologyKey": "kubernetes.io/hostname", "topologyKey": "kubernetes.io/hostname",
"whenUnsatisfiable": "ScheduleAnyway", "whenUnsatisfiable": "DoNotSchedule",
"matchLabelKeys": ["pod-template-hash"],
"labelSelector": {"matchLabels": role_labels}, "labelSelector": {"matchLabels": role_labels},
} }
], ],
} }
if graceful_shutdown_seconds:
pod_spec["terminationGracePeriodSeconds"] = max(
30,
graceful_shutdown_seconds + 20,
)
container["volumeMounts"] = [{"name": "tmp", "mountPath": "/tmp"}] container["volumeMounts"] = [{"name": "tmp", "mountPath": "/tmp"}]
if s3_ca_secret_name:
pod_spec["volumes"].append(_s3_ca_volume(s3_ca_secret_name))
container["volumeMounts"].append(_s3_ca_volume_mount())
if capabilities_config_name:
pod_spec["volumes"].append(
{
"name": "deployment-capabilities",
"configMap": {
"name": capabilities_config_name,
"items": [
{
"key": "infrastructure-capabilities.json",
"path": "infrastructure-capabilities.json",
}
],
},
}
)
container["volumeMounts"].append(
{
"name": "deployment-capabilities",
"mountPath": "/etc/govoplan/deployment/infrastructure-capabilities.json",
"subPath": "infrastructure-capabilities.json",
"readOnly": True,
}
)
if config_name: if config_name:
pod_spec["containers"][0]["envFrom"] = [{"configMapRef": {"name": config_name}}] pod_spec["containers"][0]["envFrom"] = [{"configMapRef": {"name": config_name}}]
if config_name and secret_name: if config_name and secret_name:
@@ -731,6 +831,16 @@ def _deployment(
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration-wait"}, {"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration-wait"},
{"name": "GOVOPLAN_DB_POOL_SIZE", "value": "1"}, {"name": "GOVOPLAN_DB_POOL_SIZE", "value": "1"},
{"name": "GOVOPLAN_DB_MAX_OVERFLOW", "value": "0"}, {"name": "GOVOPLAN_DB_MAX_OVERFLOW", "value": "0"},
*(
[
{
"name": "AWS_CA_BUNDLE",
"value": "/etc/govoplan/trust/s3-ca.crt",
}
]
if s3_ca_secret_name
else []
),
*_secret_environment(secret_name), *_secret_environment(secret_name),
], ],
"securityContext": { "securityContext": {
@@ -738,7 +848,10 @@ def _deployment(
"capabilities": {"drop": ["ALL"]}, "capabilities": {"drop": ["ALL"]},
"readOnlyRootFilesystem": True, "readOnlyRootFilesystem": True,
}, },
"volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}], "volumeMounts": [
{"name": "tmp", "mountPath": "/tmp"},
*([_s3_ca_volume_mount()] if s3_ca_secret_name else []),
],
} }
] ]
return { return {
@@ -769,6 +882,7 @@ def _migration_job(
image: str, image: str,
config_name: str, config_name: str,
secret_name: str, secret_name: str,
s3_ca_secret_name: str | None,
service_account: str, service_account: str,
database_environment: Mapping[str, str], database_environment: Mapping[str, str],
backup_required: bool, backup_required: bool,
@@ -832,6 +946,16 @@ def _migration_job(
"env": [ "env": [
{"name": "TMPDIR", "value": "/tmp"}, {"name": "TMPDIR", "value": "/tmp"},
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration"}, {"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration"},
*(
[
{
"name": "AWS_CA_BUNDLE",
"value": "/etc/govoplan/trust/s3-ca.crt",
}
]
if s3_ca_secret_name
else []
),
*( *(
{"name": key, "value": value} {"name": key, "value": value}
for key, value in sorted( for key, value in sorted(
@@ -851,10 +975,24 @@ def _migration_job(
"capabilities": {"drop": ["ALL"]}, "capabilities": {"drop": ["ALL"]},
"readOnlyRootFilesystem": True, "readOnlyRootFilesystem": True,
}, },
"volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}], "volumeMounts": [
{"name": "tmp", "mountPath": "/tmp"},
*(
[_s3_ca_volume_mount()]
if s3_ca_secret_name
else []
),
],
} }
], ],
"volumes": [{"name": "tmp", "emptyDir": {}}], "volumes": [
{"name": "tmp", "emptyDir": {}},
*(
[_s3_ca_volume(s3_ca_secret_name)]
if s3_ca_secret_name
else []
),
],
}, },
}, },
}, },
@@ -873,6 +1011,24 @@ def _secret_environment(secret_name: str) -> list[dict[str, Any]]:
] ]
def _s3_ca_volume(secret_name: str) -> dict[str, Any]:
return {
"name": "s3-ca",
"secret": {
"secretName": secret_name,
"items": [{"key": "ca.crt", "path": "s3-ca.crt"}],
},
}
def _s3_ca_volume_mount() -> dict[str, Any]:
return {
"name": "s3-ca",
"mountPath": "/etc/govoplan/trust",
"readOnly": True,
}
def _service( def _service(
*, *,
name: str, name: str,
@@ -54,7 +54,9 @@ FULL_MODULES = (
"dataflow", "dataflow",
"workflow_engine", "workflow_engine",
"workflow", "workflow",
"tasks",
"views", "views",
"quick_access",
"search", "search",
"risk_compliance", "risk_compliance",
"postbox", "postbox",
@@ -35,6 +35,11 @@ from .bundle import (
render_existing_proxy_contract, render_existing_proxy_contract,
service_names, service_names,
) )
from .capabilities import (
CapabilityChangeImpact,
capability_change_impacts,
infrastructure_capability_document,
)
from .distribution import ( from .distribution import (
MAX_KEYRING_BYTES, MAX_KEYRING_BYTES,
MAX_MANIFEST_BYTES, MAX_MANIFEST_BYTES,
@@ -83,6 +88,8 @@ class DeploymentPlan:
desired_environment_fingerprint: str desired_environment_fingerprint: str
actions: tuple[PlanAction, ...] actions: tuple[PlanAction, ...]
checks: tuple[Check, ...] checks: tuple[Check, ...]
infrastructure_capabilities: Mapping[str, object]
capability_impacts: tuple[CapabilityChangeImpact, ...]
@property @property
def blocked(self) -> bool: def blocked(self) -> bool:
@@ -97,6 +104,8 @@ class DeploymentPlan:
"blocked": self.blocked, "blocked": self.blocked,
"actions": [action.to_dict() for action in self.actions], "actions": [action.to_dict() for action in self.actions],
"checks": [check.to_dict() for check in self.checks], "checks": [check.to_dict() for check in self.checks],
"infrastructure_capabilities": dict(self.infrastructure_capabilities),
"capability_impacts": [item.to_dict() for item in self.capability_impacts],
} }
@@ -122,6 +131,14 @@ def build_plan(
spec_digest = digest_json(spec.to_dict()) spec_digest = digest_json(spec.to_dict())
compose_digest = digest_json(compose) compose_digest = digest_json(compose)
environment_digest = environment_fingerprint(read_env(paths.env)) environment_digest = environment_fingerprint(read_env(paths.env))
infrastructure_capabilities = infrastructure_capability_document(
spec,
read_env(paths.env),
)
capability_impacts = capability_change_impacts(
previous.get("infrastructure_capabilities"),
infrastructure_capabilities,
)
actions: list[PlanAction] = [] actions: list[PlanAction] = []
if not previous: if not previous:
@@ -166,12 +183,21 @@ def build_plan(
"Remove the service container; retained volumes are not deleted.", "Remove the service container; retained volumes are not deleted.",
) )
) )
for impact in capability_impacts:
actions.append(
PlanAction(
"review",
f"capability:{impact.capability_id}",
impact.detail,
)
)
if ( if (
previous previous
and previous_spec_digest == spec_digest and previous_spec_digest == spec_digest
and previous_compose_digest == compose_digest and previous_compose_digest == compose_digest
and previous_environment_fingerprint == environment_digest and previous_environment_fingerprint == environment_digest
and previous_services == desired_services and previous_services == desired_services
and not capability_impacts
): ):
actions.append( actions.append(
PlanAction( PlanAction(
@@ -180,6 +206,15 @@ def build_plan(
) )
checks = list(static_checks(spec, paths)) checks = list(static_checks(spec, paths))
checks.extend(
Check(
id=f"capability.change.{impact.capability_id}",
level="warning",
message=impact.detail,
action=impact.required_action,
)
for impact in capability_impacts
)
if include_host_checks: if include_host_checks:
checks.extend(host_checks(spec, paths, command_runner=command_runner)) checks.extend(host_checks(spec, paths, command_runner=command_runner))
return DeploymentPlan( return DeploymentPlan(
@@ -189,6 +224,8 @@ def build_plan(
desired_environment_fingerprint=environment_digest, desired_environment_fingerprint=environment_digest,
actions=tuple(actions), actions=tuple(actions),
checks=tuple(checks), checks=tuple(checks),
infrastructure_capabilities=infrastructure_capabilities,
capability_impacts=capability_impacts,
) )
@@ -33,6 +33,7 @@ _BUNDLE_FILES = (
"backup-keyring.json", "backup-keyring.json",
"backup-verification.json", "backup-verification.json",
"receipt.json", "receipt.json",
"infrastructure-capabilities.json",
) )
@@ -798,6 +798,13 @@
"rationale": "The vault action group rotates the current key with policy, assurance, reason, and revision evidence.", "rationale": "The vault action group rotates the current key with policy, assurance, reason, and revision evidence.",
"repository": "govoplan-encryption" "repository": "govoplan-encryption"
}, },
{
"category": "public_integration",
"method": "POST",
"path": "/files/form-evidence/upload",
"rationale": "A short-lived purpose-bound bearer grant lets the public Forms Runtime surface stream one attachment directly to Files without granting general Files access.",
"repository": "govoplan-files"
},
{ {
"category": "ui_reachable", "category": "ui_reachable",
"method": "POST", "method": "POST",
@@ -1335,6 +1342,83 @@
"rationale": "The module WebUI constructs this endpoint through a mounted router prefix, generic action, or provider path.", "rationale": "The module WebUI constructs this endpoint through a mounted router prefix, generic action, or provider path.",
"repository": "govoplan-projects" "repository": "govoplan-projects"
}, },
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/appraise",
"rationale": "The Records lifecycle panel invokes the record appraisal action through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/close",
"rationale": "The Records lifecycle panel invokes the record closure action through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/dispositions",
"rationale": "The Records lifecycle panel proposes a governed disposition through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/dispositions/{}/finalize",
"rationale": "The Records lifecycle panel finalizes an approved disposition through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/dispositions/{}/withdraw",
"rationale": "The Records lifecycle panel withdraws a pending disposition through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/holds",
"rationale": "The Records lifecycle panel applies a hold through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/holds/{}/release",
"rationale": "The Records lifecycle panel releases a hold through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/reopen",
"rationale": "The Records lifecycle panel invokes the record reopen action through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/transfer-packages",
"rationale": "The Records lifecycle panel prepares an archive-neutral transfer package through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/transfer-packages/{}/dispatch",
"rationale": "The Records lifecycle panel runs an explicitly simulated transfer through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/volumes",
"rationale": "The Records filing dialog creates explicit record volumes through a dynamically constructed record path.",
"repository": "govoplan-records"
},
{ {
"category": "missing_ui", "category": "missing_ui",
"method": "GET", "method": "GET",
@@ -1735,6 +1819,13 @@
"rationale": "IDM lacks the typed-group membership explanation surface for this existing API.", "rationale": "IDM lacks the typed-group membership explanation surface for this existing API.",
"repository": "govoplan-idm", "repository": "govoplan-idm",
"tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-idm/issues/11" "tracking_issue": "https://git.add-ideas.de/GovOPlaN/govoplan-idm/issues/11"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/tasks/{}",
"rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.",
"repository": "govoplan-tasks"
} }
], ],
"schema_version": 1 "schema_version": 1
@@ -521,11 +521,32 @@ function inspectSource(repository, sourceRoot, sourcePath) {
) { ) {
return propertyNameText(current.name); return propertyNameText(current.name);
} }
if (
ts.isVariableDeclaration(current) &&
ts.isIdentifier(current.name) &&
(current.name.text === "en" || current.name.text === "de") &&
current.initializer &&
isCatalogObject(current.initializer)
) {
return current.name.text;
}
current = current.parent; current = current.parent;
} }
return null; return null;
} }
function isCatalogObject(node) {
let current = node;
while (
ts.isAsExpression(current) ||
ts.isSatisfiesExpression(current) ||
ts.isParenthesizedExpression(current)
) {
current = current.expression;
}
return ts.isObjectLiteralExpression(current);
}
function ancestorPropertyName(node, expected) { function ancestorPropertyName(node, expected) {
let current = node.parent; let current = node.parent;
while (current) { while (current) {
@@ -31,6 +31,8 @@ ENDPOINT_SURFACE_CATEGORIES = {
DEFAULT_ENDPOINT_DECLARATIONS = ( DEFAULT_ENDPOINT_DECLARATIONS = (
META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json" META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json"
) )
REQUIRED_LOCALES = ("de", "en")
REFERENCE_LOCALE = "de"
def main() -> int: def main() -> int:
@@ -353,6 +355,14 @@ def _extract_manifests(
} }
for permission in manifest.permissions for permission in manifest.permissions
], ],
"architecture": (
manifest.architecture.to_dict()
if manifest.architecture is not None
else None
),
"information_governance": (
manifest.information_governance.to_dict()
),
"interface_catalog": manifest_interface_catalog(manifest), "interface_catalog": manifest_interface_catalog(manifest),
"frontend": ( "frontend": (
{ {
@@ -447,20 +457,27 @@ def _assemble_inventory(
usages = {item["key"] for item in webui["translationUsages"]} usages = {item["key"] for item in webui["translationUsages"]}
catalogs = webui["translationCatalog"] catalogs = webui["translationCatalog"]
catalog_keys = {locale: set(entries) for locale, entries in catalogs.items()} catalog_keys = {locale: set(entries) for locale, entries in catalogs.items()}
expected_locales = sorted(catalog_keys) expected_locales = sorted(set(catalog_keys) | set(REQUIRED_LOCALES))
missing_catalog_entries = [ missing_catalog_entries = [
{ {
"key": key, "key": key,
"missing_locales": [ "missing_locales": [
locale for locale in expected_locales if key not in catalog_keys[locale] locale
for locale in expected_locales
if key not in catalog_keys.get(locale, set())
], ],
} }
for key in sorted(usages) for key in sorted(usages)
if any(key not in catalog_keys[locale] for locale in expected_locales) if any(key not in catalog_keys.get(locale, set()) for locale in expected_locales)
] ]
fields = webui["fields"] fields = webui["fields"]
help_candidates = [field for field in fields if field["helpCandidate"]] help_candidates = [field for field in fields if field["helpCandidate"]]
dynamic_help = [field for field in fields if field.get("helpDynamic")] dynamic_help = [field for field in fields if field.get("helpDynamic")]
governance_adoption = Counter(
dimension["adoption"]
for manifest in manifests
for dimension in manifest["information_governance"]["dimensions"].values()
)
source_declarations = _source_interface_declarations(webui, manifests) source_declarations = _source_interface_declarations(webui, manifests)
declaration_health = _declaration_health(source_declarations, manifests) declaration_health = _declaration_health(source_declarations, manifests)
runtime_comparison = ( runtime_comparison = (
@@ -502,9 +519,26 @@ def _assemble_inventory(
}, },
"translation_health": { "translation_health": {
"locales": expected_locales, "locales": expected_locales,
"reference_locale": REFERENCE_LOCALE,
"reference_locale_entries": len(catalog_keys.get(REFERENCE_LOCALE, set())),
"reference_locale_complete": not any(
REFERENCE_LOCALE in item["missing_locales"]
for item in missing_catalog_entries
),
"used_keys": len(usages), "used_keys": len(usages),
"missing_catalog_entries": missing_catalog_entries, "missing_catalog_entries": missing_catalog_entries,
}, },
"information_governance_health": {
"dimensions": len(manifests) * 4,
"adoption_counts": dict(sorted(governance_adoption.items())),
"modules": [
{
"module_id": manifest["id"],
"dimensions": manifest["information_governance"]["dimensions"],
}
for manifest in manifests
],
},
"api": { "api": {
"backend_endpoints": classified_endpoints, "backend_endpoints": classified_endpoints,
"frontend_references": frontend_refs, "frontend_references": frontend_refs,
@@ -517,6 +551,7 @@ def _assemble_inventory(
"modules": len(manifests), "modules": len(manifests),
"ui_fields": len(fields), "ui_fields": len(fields),
"ui_fields_with_static_help": len(fields) - len(help_candidates), "ui_fields_with_static_help": len(fields) - len(help_candidates),
"ui_fields_with_resolvable_f1_context": len(fields),
"help_review_candidates": len(help_candidates), "help_review_candidates": len(help_candidates),
"dynamic_help_references": len(dynamic_help), "dynamic_help_references": len(dynamic_help),
"ui_actions": len(webui.get("actions", [])), "ui_actions": len(webui.get("actions", [])),
@@ -536,6 +571,12 @@ def _assemble_inventory(
"backend_endpoints_without_static_webui_reference": len(unreferenced), "backend_endpoints_without_static_webui_reference": len(unreferenced),
"unclassified_backend_endpoints": len(unclassified), "unclassified_backend_endpoints": len(unclassified),
"stale_endpoint_declarations": len(stale_declarations), "stale_endpoint_declarations": len(stale_declarations),
"information_governance_dimensions": len(manifests) * 4,
"information_governance_enforced": governance_adoption["enforced"],
"information_governance_partial": governance_adoption["partial"],
"information_governance_contract_only": governance_adoption[
"contract_only"
],
}, },
} }
@@ -856,6 +897,7 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
f"- UI fields: {summary['ui_fields']}", f"- UI fields: {summary['ui_fields']}",
f"- UI actions: {summary['ui_actions']}", f"- UI actions: {summary['ui_actions']}",
f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}", f"- Fields with statically associated help: {summary['ui_fields_with_static_help']}",
f"- Fields with a resolvable F1 context: {summary['ui_fields_with_resolvable_f1_context']}",
f"- Fields with dynamic help references: {summary['dynamic_help_references']}", f"- Fields with dynamic help references: {summary['dynamic_help_references']}",
f"- Help review candidates: {summary['help_review_candidates']}", f"- Help review candidates: {summary['help_review_candidates']}",
f"- Stable interface declarations: {summary['interface_declarations']}", f"- Stable interface declarations: {summary['interface_declarations']}",
@@ -872,7 +914,12 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
), ),
f"- Unclassified backend endpoints: {summary['unclassified_backend_endpoints']}", f"- Unclassified backend endpoints: {summary['unclassified_backend_endpoints']}",
f"- Stale endpoint declarations: {summary['stale_endpoint_declarations']}", f"- Stale endpoint declarations: {summary['stale_endpoint_declarations']}",
f"- Used translation keys missing from a locale catalog: {len(missing)}", f"- Reference locale: `{inventory['translation_health']['reference_locale']}`",
f"- Reference locale complete: `{str(inventory['translation_health']['reference_locale_complete']).lower()}`",
f"- Used translation keys missing from a required locale catalog: {len(missing)}",
f"- Information-governance dimensions enforced: {summary['information_governance_enforced']}",
f"- Information-governance dimensions partial: {summary['information_governance_partial']}",
f"- Information-governance dimensions contract-only: {summary['information_governance_contract_only']}",
"", "",
"## Help Review Candidates", "## Help Review Candidates",
"", "",
@@ -0,0 +1,85 @@
schema_version = 1
name = "govoplan-k8s-acceptance"
mode = "acceptance"
state_directory = "~/.local/share/govoplan/labs/govoplan-k8s-acceptance"
vm_image_directory = "/var/lib/libvirt/images/govoplan-labs"
ssh_user = "govoplan"
ssh_private_key = "~/.ssh/govoplan-lab"
ssh_public_key = "~/.ssh/govoplan-lab.pub"
namespace = "govoplan"
public_host = "govoplan.acceptance.example.org"
s3_host = "s3.govoplan.acceptance.example.org"
ingress_class = "traefik"
module_set = "base"
api_replicas = 2
web_replicas = 2
worker_replicas = 2
db_connection_limit = 100
[network]
prefix_length = 24
gateway = "10.77.10.1"
dns_servers = ["10.77.10.1", "1.1.1.1"]
bridge = "br0"
[image]
url = "https://cloud-images.ubuntu.com/releases/noble/release-20260801/ubuntu-24.04-server-cloudimg-amd64.img"
sha256 = "0533b0655c32e68b31d792ecd6ccfca95abdbc536c4446874fe0513bd4140ffe"
[k3s]
version = "v1.36.1+k3s1"
binary_url = "https://github.com/k3s-io/k3s/releases/download/v1.36.1%2Bk3s1/k3s"
binary_sha256 = "a443db3fe9820cd93617ae67e4386d87c1514c1e96ceb30f4c2791c39065653c"
install_script_url = "https://raw.githubusercontent.com/k3s-io/k3s/v1.36.1%2Bk3s1/install.sh"
install_script_sha256 = "46177d4c99440b4c0311b67233823a8e8a2fc09693f6c89af1a7161e152fbfad"
[release]
manifest_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-manifest.json"
manifest_sha256 = "09ac1ade6ede4958bab0dfb7fd8f99246f4d991846308db1f410b25b46267840"
keyring_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-keyring.json"
keyring_sha256 = "92a9f8e3bac0ef525ad9a063c76faa36233a9070cd4a5db9ee9b3f92323b602f"
channel = "stable"
[[nodes]]
name = "control-1"
role = "control"
address = "10.77.10.21"
hypervisor = "lab-admin@hypervisor-state.example.org"
failure_domain = "rack-c"
mac_address = "52:54:00:68:01:01"
cpus = 2
memory_mib = 4096
disk_gib = 32
[[nodes]]
name = "worker-1"
role = "worker"
address = "10.77.10.22"
hypervisor = "lab-admin@hypervisor-a.example.org"
failure_domain = "rack-a"
mac_address = "52:54:00:68:01:02"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "worker-2"
role = "worker"
address = "10.77.10.23"
hypervisor = "lab-admin@hypervisor-b.example.org"
failure_domain = "rack-b"
mac_address = "52:54:00:68:01:03"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "state-1"
role = "state"
address = "10.77.10.24"
hypervisor = "lab-admin@hypervisor-state.example.org"
failure_domain = "rack-c"
mac_address = "52:54:00:68:01:04"
cpus = 4
memory_mib = 8192
disk_gib = 120
+85
View File
@@ -0,0 +1,85 @@
schema_version = 1
name = "govoplan-k8s-lab"
mode = "rehearsal"
state_directory = "~/.local/share/govoplan/labs/govoplan-k8s-lab"
vm_image_directory = "/var/lib/libvirt/images/govoplan-labs"
ssh_user = "govoplan"
ssh_private_key = "~/.ssh/govoplan-lab"
ssh_public_key = "~/.ssh/govoplan-lab.pub"
namespace = "govoplan"
public_host = "govoplan.lab.test"
s3_host = "s3.govoplan.lab.test"
ingress_class = "traefik"
module_set = "base"
api_replicas = 2
web_replicas = 2
worker_replicas = 2
db_connection_limit = 100
[network]
prefix_length = 24
gateway = "192.168.123.1"
dns_servers = ["192.168.123.1", "1.1.1.1"]
bridge = "virbr-gplab"
[image]
url = "https://cloud-images.ubuntu.com/releases/noble/release-20260801/ubuntu-24.04-server-cloudimg-amd64.img"
sha256 = "0533b0655c32e68b31d792ecd6ccfca95abdbc536c4446874fe0513bd4140ffe"
[k3s]
version = "v1.36.1+k3s1"
binary_url = "https://github.com/k3s-io/k3s/releases/download/v1.36.1%2Bk3s1/k3s"
binary_sha256 = "a443db3fe9820cd93617ae67e4386d87c1514c1e96ceb30f4c2791c39065653c"
install_script_url = "https://raw.githubusercontent.com/k3s-io/k3s/v1.36.1%2Bk3s1/install.sh"
install_script_sha256 = "46177d4c99440b4c0311b67233823a8e8a2fc09693f6c89af1a7161e152fbfad"
[release]
manifest_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-manifest.json"
manifest_sha256 = "09ac1ade6ede4958bab0dfb7fd8f99246f4d991846308db1f410b25b46267840"
keyring_url = "https://git.add-ideas.de/GovOPlaN/govoplan/releases/download/v0.1.15/distribution-keyring.json"
keyring_sha256 = "92a9f8e3bac0ef525ad9a063c76faa36233a9070cd4a5db9ee9b3f92323b602f"
channel = "stable"
[[nodes]]
name = "control-1"
role = "control"
address = "192.168.123.201"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:01"
cpus = 2
memory_mib = 4096
disk_gib = 32
[[nodes]]
name = "worker-1"
role = "worker"
address = "192.168.123.202"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:02"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "worker-2"
role = "worker"
address = "192.168.123.203"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:03"
cpus = 2
memory_mib = 4096
disk_gib = 40
[[nodes]]
name = "state-1"
role = "state"
address = "192.168.123.204"
hypervisor = "local"
failure_domain = "local-host"
mac_address = "52:54:00:67:01:04"
cpus = 4
memory_mib = 4096
disk_gib = 80
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env python3
"""Executable entry point for the GovOPlaN Kubernetes VM lab."""
from __future__ import annotations
from pathlib import Path
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
from govoplan_lab.cli import main
if __name__ == "__main__":
raise SystemExit(main())
+5
View File
@@ -0,0 +1,5 @@
"""Reproducible GovOPlaN Kubernetes acceptance lab."""
from .config import LabConfig, LabConfigError, LabNode, load_config
__all__ = ["LabConfig", "LabConfigError", "LabNode", "load_config"]
+153
View File
@@ -0,0 +1,153 @@
"""Command-line interface for the GovOPlaN Kubernetes VM lab."""
from __future__ import annotations
import argparse
from pathlib import Path
from typing import Sequence
from .config import LabConfigError, load_config
from .lifecycle import (
LabOperationError,
create,
deploy,
destroy,
doctor,
enroll_admin,
pause,
resume,
status,
update,
verify,
)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="govoplan-lab",
description="Create and operate a libvirt-backed GovOPlaN Kubernetes test lab.",
)
parser.add_argument(
"--config",
type=Path,
default=Path("govoplan-lab.toml"),
help="Strict TOML lab inventory (default: ./govoplan-lab.toml).",
)
parser.add_argument("--verbose", action="store_true")
subparsers = parser.add_subparsers(dest="command", required=True)
doctor_parser = subparsers.add_parser("doctor", help="Validate inventory and prerequisites.")
doctor_parser.add_argument(
"--online",
action="store_true",
help="Also connect to every hypervisor and verify its toolchain.",
)
subparsers.add_parser("status", help="Show VM, Kubernetes node, and pod state.")
_mutation_parser(subparsers, "create", "Create or reuse all declared VMs.")
_mutation_parser(subparsers, "deploy", "Deploy shared state, K3s, and GovOPlaN.")
_mutation_parser(subparsers, "update", "Reconcile pinned K3s and GovOPlaN inputs serially.")
_mutation_parser(subparsers, "pause", "Gracefully stop the lab while preserving disks.")
_mutation_parser(subparsers, "resume", "Start a paused lab in dependency order.")
destroy_parser = _mutation_parser(
subparsers,
"destroy",
"Destroy lab-owned VMs and disks with an explicit name confirmation.",
)
destroy_parser.add_argument("--confirm", default="")
destroy_parser.add_argument(
"--purge-local-state",
action="store_true",
help="Also delete local secrets, manifests, and evidence after VM teardown.",
)
verify_parser = subparsers.add_parser(
"verify",
help=(
"Collect sanitized live-cluster evidence using an "
"Ops-read-authorized GOVOPLAN_OPS_API_KEY."
),
)
verify_parser.add_argument(
"--exercise-api-pod-loss",
action="store_true",
help="Delete one ready API pod during the bounded availability drill.",
)
enroll_parser = _mutation_parser(
subparsers,
"enroll-admin",
"Securely consume the first-administrator enrollment artifact.",
)
enroll_parser.add_argument("--email", required=True)
enroll_parser.add_argument("--display-name", default=None)
enroll_parser.add_argument("--tenant-slug", default="default")
enroll_parser.add_argument("--tenant-name", default="Default Tenant")
return parser
def _mutation_parser(
subparsers: argparse._SubParsersAction[argparse.ArgumentParser],
name: str,
help_text: str,
) -> argparse.ArgumentParser:
parser = subparsers.add_parser(name, help=help_text)
parser.add_argument(
"--apply",
action="store_true",
help="Perform mutations; without this flag the command is a dry run.",
)
return parser
def main(argv: Sequence[str] | None = None) -> int:
args = build_parser().parse_args(argv)
try:
config = load_config(args.config)
if args.command == "doctor":
return doctor(config, online=args.online, verbose=args.verbose)
if args.command == "status":
return status(config, verbose=args.verbose)
if args.command == "create":
create(config, apply=args.apply, verbose=args.verbose)
elif args.command == "deploy":
deploy(config, apply=args.apply, verbose=args.verbose)
elif args.command == "update":
update(config, apply=args.apply, verbose=args.verbose)
elif args.command == "pause":
pause(config, apply=args.apply, verbose=args.verbose)
elif args.command == "resume":
resume(config, apply=args.apply, verbose=args.verbose)
elif args.command == "destroy":
destroy(
config,
apply=args.apply,
confirmation=args.confirm,
purge_local_state=args.purge_local_state,
verbose=args.verbose,
)
elif args.command == "verify":
verify(
config,
exercise_api_pod_loss=args.exercise_api_pod_loss,
verbose=args.verbose,
)
elif args.command == "enroll-admin":
enroll_admin(
config,
email=args.email,
display_name=args.display_name,
tenant_slug=args.tenant_slug,
tenant_name=args.tenant_name,
apply=args.apply,
)
else:
raise RuntimeError(f"unsupported command: {args.command}")
return 0
except (LabConfigError, LabOperationError, OSError, ValueError) as exc:
print(f"error: {exc}", file=__import__("sys").stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+466
View File
@@ -0,0 +1,466 @@
"""Strict TOML model for the GovOPlaN Kubernetes VM lab."""
from __future__ import annotations
from dataclasses import dataclass
import ipaddress
from pathlib import Path
import re
import tomllib
from typing import Any, Mapping
from urllib.parse import urlsplit
SCHEMA_VERSION = 1
_NAME = re.compile(r"^[a-z][a-z0-9-]{1,47}$")
_HOSTNAME = re.compile(
r"^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)*"
r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$"
)
_SHA256 = re.compile(r"^[0-9a-f]{64}$")
_K3S_VERSION = re.compile(r"^v[0-9]+\.[0-9]+\.[0-9]+\+k3s[0-9]+$")
_SSH_TARGET = re.compile(r"^(?:[A-Za-z0-9_.-]+@)?[A-Za-z0-9_.:-]+$")
_BRIDGE = re.compile(r"^[A-Za-z0-9_.:-]{1,32}$")
_ROLE = {"control", "worker", "state"}
_MODE = {"rehearsal", "acceptance"}
class LabConfigError(ValueError):
"""Raised when the lab inventory cannot be used safely."""
@dataclass(frozen=True, slots=True)
class LabNode:
name: str
role: str
address: ipaddress.IPv4Address
hypervisor: str
failure_domain: str
mac_address: str
cpus: int
memory_mib: int
disk_gib: int
@property
def is_local(self) -> bool:
return self.hypervisor == "local"
@dataclass(frozen=True, slots=True)
class NetworkConfig:
prefix_length: int
gateway: ipaddress.IPv4Address
dns_servers: tuple[ipaddress.IPv4Address, ...]
bridge: str
@dataclass(frozen=True, slots=True)
class ImageConfig:
url: str
sha256: str
@dataclass(frozen=True, slots=True)
class K3sConfig:
version: str
binary_url: str
binary_sha256: str
install_script_url: str
install_script_sha256: str
@dataclass(frozen=True, slots=True)
class ReleaseConfig:
manifest_url: str
manifest_sha256: str
keyring_url: str
keyring_sha256: str
channel: str
@dataclass(frozen=True, slots=True)
class LabConfig:
source: Path
schema_version: int
name: str
mode: str
state_directory: Path
vm_image_directory: str
ssh_user: str
ssh_private_key: Path
ssh_public_key: Path
namespace: str
public_host: str
s3_host: str
ingress_class: str
module_set: str
api_replicas: int
web_replicas: int
worker_replicas: int
db_connection_limit: int
network: NetworkConfig
image: ImageConfig
k3s: K3sConfig
release: ReleaseConfig
nodes: tuple[LabNode, ...]
@property
def controls(self) -> tuple[LabNode, ...]:
return tuple(node for node in self.nodes if node.role == "control")
@property
def workers(self) -> tuple[LabNode, ...]:
return tuple(node for node in self.nodes if node.role == "worker")
@property
def state_node(self) -> LabNode:
return next(node for node in self.nodes if node.role == "state")
@property
def primary_control(self) -> LabNode:
return self.controls[0]
@property
def public_url(self) -> str:
return f"https://{self.public_host}"
@property
def s3_url(self) -> str:
return f"https://{self.s3_host}:9443"
@property
def evidence_capable(self) -> bool:
worker_domains = {node.failure_domain for node in self.workers}
worker_hypervisors = {node.hypervisor for node in self.workers}
return (
self.mode == "acceptance"
and len(worker_domains) == len(self.workers)
and len(worker_hypervisors) == len(self.workers)
and self.state_node.failure_domain not in worker_domains
and self.state_node.hypervisor not in worker_hypervisors
)
def load_config(path: Path) -> LabConfig:
source = path.expanduser().resolve()
try:
raw = tomllib.loads(source.read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise LabConfigError(f"lab configuration does not exist: {source}") from exc
except tomllib.TOMLDecodeError as exc:
raise LabConfigError(f"lab configuration is not valid TOML: {exc}") from exc
root = _mapping(raw, "lab configuration")
_only_keys(
root,
{
"schema_version",
"name",
"mode",
"state_directory",
"vm_image_directory",
"ssh_user",
"ssh_private_key",
"ssh_public_key",
"namespace",
"public_host",
"s3_host",
"ingress_class",
"module_set",
"api_replicas",
"web_replicas",
"worker_replicas",
"db_connection_limit",
"network",
"image",
"k3s",
"release",
"nodes",
},
"lab configuration",
)
schema_version = _integer(root, "schema_version")
if schema_version != SCHEMA_VERSION:
raise LabConfigError(
f"schema_version must be {SCHEMA_VERSION}; found {schema_version}"
)
name = _pattern(root, "name", _NAME)
mode = _choice(root, "mode", _MODE)
base = source.parent
state_directory = _path(root, "state_directory", base)
vm_image_directory = _absolute_posix_path(root, "vm_image_directory")
ssh_user = _pattern(root, "ssh_user", re.compile(r"^[a-z_][a-z0-9_-]{0,31}$"))
ssh_private_key = _path(root, "ssh_private_key", base)
ssh_public_key = _path(root, "ssh_public_key", base)
namespace = _pattern(root, "namespace", _NAME)
public_host = _pattern(root, "public_host", _HOSTNAME)
s3_host = _pattern(root, "s3_host", _HOSTNAME)
if public_host == s3_host:
raise LabConfigError("public_host and s3_host must be different")
ingress_class = _pattern(root, "ingress_class", _NAME)
module_set = _choice(root, "module_set", {"core", "base", "full"})
api_replicas = _bounded_integer(root, "api_replicas", 2, 32)
web_replicas = _bounded_integer(root, "web_replicas", 2, 32)
worker_replicas = _bounded_integer(root, "worker_replicas", 2, 64)
db_connection_limit = _bounded_integer(root, "db_connection_limit", 50, 10000)
network = _parse_network(_mapping(root.get("network"), "network"))
image = _parse_image(_mapping(root.get("image"), "image"))
k3s = _parse_k3s(_mapping(root.get("k3s"), "k3s"))
release = _parse_release(_mapping(root.get("release"), "release"))
raw_nodes = root.get("nodes")
if not isinstance(raw_nodes, list) or not raw_nodes:
raise LabConfigError("nodes must be a non-empty array of tables")
nodes = tuple(_parse_node(item, index=index) for index, item in enumerate(raw_nodes))
config = LabConfig(
source=source,
schema_version=schema_version,
name=name,
mode=mode,
state_directory=state_directory,
vm_image_directory=vm_image_directory.rstrip("/"),
ssh_user=ssh_user,
ssh_private_key=ssh_private_key,
ssh_public_key=ssh_public_key,
namespace=namespace,
public_host=public_host,
s3_host=s3_host,
ingress_class=ingress_class,
module_set=module_set,
api_replicas=api_replicas,
web_replicas=web_replicas,
worker_replicas=worker_replicas,
db_connection_limit=db_connection_limit,
network=network,
image=image,
k3s=k3s,
release=release,
nodes=nodes,
)
_validate_topology(config)
return config
def _parse_network(raw: Mapping[str, Any]) -> NetworkConfig:
_only_keys(raw, {"prefix_length", "gateway", "dns_servers", "bridge"}, "network")
prefix_length = _bounded_integer(raw, "prefix_length", 8, 30)
gateway = _ipv4(raw, "gateway")
dns_raw = raw.get("dns_servers")
if not isinstance(dns_raw, list) or not dns_raw or len(dns_raw) > 4:
raise LabConfigError("network.dns_servers must contain 1-4 IPv4 addresses")
dns_servers = tuple(_ipv4_value(value, "network.dns_servers") for value in dns_raw)
bridge = _pattern(raw, "bridge", _BRIDGE)
return NetworkConfig(prefix_length, gateway, dns_servers, bridge)
def _parse_image(raw: Mapping[str, Any]) -> ImageConfig:
_only_keys(raw, {"url", "sha256"}, "image")
return ImageConfig(
url=_https_url(raw, "url"),
sha256=_pattern(raw, "sha256", _SHA256),
)
def _parse_k3s(raw: Mapping[str, Any]) -> K3sConfig:
_only_keys(
raw,
{
"version",
"binary_url",
"binary_sha256",
"install_script_url",
"install_script_sha256",
},
"k3s",
)
return K3sConfig(
version=_pattern(raw, "version", _K3S_VERSION),
binary_url=_https_url(raw, "binary_url"),
binary_sha256=_pattern(raw, "binary_sha256", _SHA256),
install_script_url=_https_url(raw, "install_script_url"),
install_script_sha256=_pattern(raw, "install_script_sha256", _SHA256),
)
def _parse_release(raw: Mapping[str, Any]) -> ReleaseConfig:
_only_keys(
raw,
{"manifest_url", "manifest_sha256", "keyring_url", "keyring_sha256", "channel"},
"release",
)
return ReleaseConfig(
manifest_url=_https_url(raw, "manifest_url"),
manifest_sha256=_pattern(raw, "manifest_sha256", _SHA256),
keyring_url=_https_url(raw, "keyring_url"),
keyring_sha256=_pattern(raw, "keyring_sha256", _SHA256),
channel=_pattern(raw, "channel", _NAME),
)
def _parse_node(value: object, *, index: int) -> LabNode:
raw = _mapping(value, f"nodes[{index}]")
_only_keys(
raw,
{
"name",
"role",
"address",
"hypervisor",
"failure_domain",
"mac_address",
"cpus",
"memory_mib",
"disk_gib",
},
f"nodes[{index}]",
)
hypervisor = _string(raw, "hypervisor")
if hypervisor != "local" and not _SSH_TARGET.fullmatch(hypervisor):
raise LabConfigError(
f"nodes[{index}].hypervisor must be 'local' or a simple SSH target"
)
mac_address = _string(raw, "mac_address").lower()
try:
octets = mac_address.split(":")
valid_mac = len(octets) == 6 and all(
len(octet) == 2 and 0 <= int(octet, 16) <= 255 for octet in octets
)
except ValueError:
valid_mac = False
if not valid_mac:
raise LabConfigError(f"nodes[{index}].mac_address is not a canonical MAC address")
return LabNode(
name=_pattern(raw, "name", _NAME),
role=_choice(raw, "role", _ROLE),
address=_ipv4(raw, "address"),
hypervisor=hypervisor,
failure_domain=_pattern(raw, "failure_domain", _NAME),
mac_address=mac_address,
cpus=_bounded_integer(raw, "cpus", 1, 64),
memory_mib=_bounded_integer(raw, "memory_mib", 2048, 262144),
disk_gib=_bounded_integer(raw, "disk_gib", 16, 4096),
)
def _validate_topology(config: LabConfig) -> None:
names = [node.name for node in config.nodes]
addresses = [node.address for node in config.nodes]
mac_addresses = [node.mac_address for node in config.nodes]
for label, values in (
("node names", names),
("node addresses", addresses),
("node MAC addresses", mac_addresses),
):
if len(values) != len(set(values)):
raise LabConfigError(f"{label} must be unique")
too_long = [
node.name
for node in config.nodes
if len(f"{config.name}-{node.name}") > 63
]
if too_long:
raise LabConfigError(
"lab name plus node name must fit the 63-character libvirt domain limit: "
+ ", ".join(too_long)
)
if len(config.controls) not in {1, 3}:
raise LabConfigError("the lab requires exactly one or three control-plane nodes")
if len(config.workers) < 2:
raise LabConfigError("the lab requires at least two worker nodes")
if sum(node.role == "state" for node in config.nodes) != 1:
raise LabConfigError("the lab requires exactly one external shared-state node")
network = ipaddress.ip_network(
f"{config.network.gateway}/{config.network.prefix_length}", strict=False
)
if any(node.address not in network for node in config.nodes):
raise LabConfigError("every node address must be in the configured IPv4 network")
if config.network.gateway in addresses:
raise LabConfigError("the network gateway cannot also be a node address")
if config.mode == "acceptance" and not config.evidence_capable:
raise LabConfigError(
"acceptance mode requires each worker and the shared-state node to use "
"distinct hypervisors and failure_domain values"
)
def _mapping(value: object, label: str) -> Mapping[str, Any]:
if not isinstance(value, dict):
raise LabConfigError(f"{label} must be a table")
return value
def _only_keys(raw: Mapping[str, Any], allowed: set[str], label: str) -> None:
unexpected = sorted(set(raw) - allowed)
if unexpected:
raise LabConfigError(f"{label} contains unsupported keys: {', '.join(unexpected)}")
def _string(raw: Mapping[str, Any], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str) or not value.strip():
raise LabConfigError(f"{key} must be a non-empty string")
return value.strip()
def _integer(raw: Mapping[str, Any], key: str) -> int:
value = raw.get(key)
if not isinstance(value, int) or isinstance(value, bool):
raise LabConfigError(f"{key} must be an integer")
return value
def _bounded_integer(raw: Mapping[str, Any], key: str, minimum: int, maximum: int) -> int:
value = _integer(raw, key)
if not minimum <= value <= maximum:
raise LabConfigError(f"{key} must be between {minimum} and {maximum}")
return value
def _pattern(raw: Mapping[str, Any], key: str, pattern: re.Pattern[str]) -> str:
value = _string(raw, key)
if pattern.fullmatch(value) is None:
raise LabConfigError(f"{key} has an unsupported format")
return value
def _choice(raw: Mapping[str, Any], key: str, choices: set[str]) -> str:
value = _string(raw, key)
if value not in choices:
raise LabConfigError(f"{key} must be one of: {', '.join(sorted(choices))}")
return value
def _ipv4(raw: Mapping[str, Any], key: str) -> ipaddress.IPv4Address:
return _ipv4_value(_string(raw, key), key)
def _ipv4_value(value: object, label: str) -> ipaddress.IPv4Address:
if not isinstance(value, str):
raise LabConfigError(f"{label} must contain strings")
try:
parsed = ipaddress.ip_address(value)
except ValueError as exc:
raise LabConfigError(f"{label} contains an invalid IP address") from exc
if not isinstance(parsed, ipaddress.IPv4Address):
raise LabConfigError(f"{label} supports IPv4 only in schema version 1")
return parsed
def _path(raw: Mapping[str, Any], key: str, base: Path) -> Path:
value = Path(_string(raw, key)).expanduser()
return (value if value.is_absolute() else base / value).resolve()
def _absolute_posix_path(raw: Mapping[str, Any], key: str) -> str:
value = _string(raw, key)
if not value.startswith("/") or ".." in Path(value).parts:
raise LabConfigError(f"{key} must be an absolute path without '..'")
return value
def _https_url(raw: Mapping[str, Any], key: str) -> str:
value = _string(raw, key)
parsed = urlsplit(value)
if parsed.scheme != "https" or not parsed.hostname or parsed.username or parsed.password:
raise LabConfigError(f"{key} must be a credential-free HTTPS URL")
if parsed.fragment:
raise LabConfigError(f"{key} must not contain a fragment")
return value
File diff suppressed because it is too large Load Diff
+311
View File
@@ -0,0 +1,311 @@
"""Deterministic configuration rendering for the Kubernetes VM lab."""
from __future__ import annotations
import base64
import json
from pathlib import Path
from typing import Mapping
from .config import LabConfig, LabNode
def render_cloud_init(config: LabConfig, node: LabNode, public_key: str) -> str:
packages = ["ca-certificates", "curl", "qemu-guest-agent"]
if node.role == "state":
packages.extend(["docker.io", "docker-compose-v2", "openssl"])
package_lines = "\n".join(f" - {item}" for item in packages)
return f"""#cloud-config
hostname: {node.name}
manage_etc_hosts: true
package_update: true
package_upgrade: false
packages:
{package_lines}
users:
- default
- name: {config.ssh_user}
groups: [adm, sudo]
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
lock_passwd: true
ssh_authorized_keys:
- {json.dumps(public_key.strip())}
ssh_pwauth: false
disable_root: true
runcmd:
- [systemctl, enable, --now, qemu-guest-agent]
- [sh, -c, "test ! -e /usr/bin/docker || systemctl enable --now docker"]
final_message: "GovOPlaN lab node is ready"
"""
def render_network_config(config: LabConfig, node: LabNode) -> str:
dns = ", ".join(str(item) for item in config.network.dns_servers)
return f"""version: 2
ethernets:
primary:
match:
macaddress: {node.mac_address}
set-name: eth0
addresses:
- {node.address}/{config.network.prefix_length}
routes:
- to: default
via: {config.network.gateway}
nameservers:
addresses: [{dns}]
"""
def render_meta_data(config: LabConfig, node: LabNode) -> str:
return f"instance-id: {config.name}-{node.name}\nlocal-hostname: {node.name}\n"
def render_k3s_config(
config: LabConfig,
node: LabNode,
*,
cluster_token: str,
) -> str:
lines = [
f'node-name: "{node.name}"',
f'node-ip: "{node.address}"',
f'token: "{cluster_token}"',
]
if node.role == "control":
if node == config.primary_control:
lines.append("cluster-init: true")
else:
lines.append(f'server: "https://{config.primary_control.address}:6443"')
lines.extend(
[
'write-kubeconfig-mode: "0600"',
"secrets-encryption: true",
"tls-san:",
f' - "{config.primary_control.address}"',
"node-taint:",
' - "node-role.kubernetes.io/control-plane=true:NoSchedule"',
]
)
elif node.role == "worker":
lines.extend(
[
f'server: "https://{config.primary_control.address}:6443"',
"node-label:",
' - "govoplan.add-ideas.de/runtime=true"',
f' - "topology.govoplan.add-ideas.de/failure-domain={node.failure_domain}"',
]
)
else:
raise ValueError("state nodes do not receive K3s configuration")
return "\n".join(lines) + "\n"
def render_registry_config(username: str, password: str) -> str:
if not username and not password:
return ""
if not username or not password:
raise ValueError("registry username and password must be supplied together")
return (
'mirrors:\n "git.add-ideas.de":\n'
' endpoint:\n - "https://git.add-ideas.de"\n'
'configs:\n "git.add-ideas.de":\n auth:\n'
f" username: {json.dumps(username)}\n"
f" password: {json.dumps(password)}\n"
)
def render_garage_config() -> str:
return """metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
replication_factor = 1
rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage.localhost"
[admin]
api_bind_addr = "[::]:3903"
"""
def render_caddyfile() -> str:
return """:9443 {
tls /etc/caddy/tls/server.crt /etc/caddy/tls/server.key
reverse_proxy garage:3900
}
"""
def render_state_compose(images: Mapping[str, str]) -> str:
required = {"postgres", "redis", "garage", "managed_ingress", "test_mail"}
missing = sorted(required - set(images))
if missing:
raise ValueError("release manifest is missing state images: " + ", ".join(missing))
value = {
"name": "govoplan-lab-state",
"services": {
"postgres": {
"image": images["postgres"],
"restart": "unless-stopped",
"environment": {
"POSTGRES_DB": "${POSTGRES_DB}",
"POSTGRES_USER": "${POSTGRES_USER}",
"POSTGRES_PASSWORD": "${POSTGRES_PASSWORD}",
},
"ports": ["5432:5432"],
"healthcheck": {
"test": [
"CMD-SHELL",
'pg_isready -U "$${POSTGRES_USER}" -d "$${POSTGRES_DB}"',
],
"interval": "5s",
"timeout": "3s",
"retries": 30,
},
"volumes": ["postgres-data:/var/lib/postgresql/data"],
"networks": ["internal"],
},
"redis": {
"image": images["redis"],
"restart": "unless-stopped",
"command": [
"sh",
"-ec",
'exec redis-server --appendonly yes --requirepass "$$REDIS_PASSWORD"',
],
"environment": {"REDIS_PASSWORD": "${REDIS_PASSWORD}"},
"ports": ["6379:6379"],
"healthcheck": {
"test": [
"CMD-SHELL",
'redis-cli -a "$${REDIS_PASSWORD}" --no-auth-warning ping',
],
"interval": "5s",
"timeout": "3s",
"retries": 30,
},
"volumes": ["redis-data:/data"],
"networks": ["internal"],
},
"garage": {
"image": images["garage"],
"restart": "unless-stopped",
"command": ["/garage", "server", "--single-node", "--default-bucket"],
"environment": {
"GARAGE_DEFAULT_ACCESS_KEY": "${GARAGE_DEFAULT_ACCESS_KEY}",
"GARAGE_DEFAULT_SECRET_KEY": "${GARAGE_DEFAULT_SECRET_KEY}",
"GARAGE_DEFAULT_BUCKET": "${GARAGE_DEFAULT_BUCKET}",
"GARAGE_RPC_SECRET": "${GARAGE_RPC_SECRET}",
"GARAGE_ADMIN_TOKEN": "${GARAGE_ADMIN_TOKEN}",
"GARAGE_METRICS_TOKEN": "${GARAGE_METRICS_TOKEN}",
},
"healthcheck": {
"test": ["CMD", "/garage", "status"],
"interval": "10s",
"timeout": "5s",
"retries": 30,
"start_period": "15s",
},
"security_opt": ["no-new-privileges:true"],
"volumes": [
"./garage.toml:/etc/garage.toml:ro",
"garage-meta:/var/lib/garage/meta",
"garage-data:/var/lib/garage/data",
],
"networks": ["internal"],
},
"s3-tls": {
"image": images["managed_ingress"],
"restart": "unless-stopped",
"depends_on": {"garage": {"condition": "service_healthy"}},
"ports": ["9443:9443"],
"volumes": [
"./Caddyfile:/etc/caddy/Caddyfile:ro",
"./tls:/etc/caddy/tls:ro",
],
"security_opt": ["no-new-privileges:true"],
"networks": ["internal"],
},
"test-mail": {
"image": images["test_mail"],
"restart": "unless-stopped",
"environment": {
"GREENMAIL_OPTS": (
"-Dgreenmail.setup.test.smtp -Dgreenmail.setup.test.imap "
"-Dgreenmail.hostname=0.0.0.0"
)
},
"ports": ["3025:3025", "3143:3143"],
"networks": ["internal"],
},
},
"volumes": {
"postgres-data": {},
"redis-data": {},
"garage-meta": {},
"garage-data": {},
},
"networks": {"internal": {"driver": "bridge"}},
}
return json.dumps(value, indent=2, sort_keys=True) + "\n"
def render_state_environment(values: Mapping[str, str]) -> str:
return "".join(f"{key}={_env_quote(value)}\n" for key, value in sorted(values.items()))
def render_secret_manifest(
*, namespace: str, name: str, values: Mapping[str, bytes | str]
) -> bytes:
encoded = {
key: base64.b64encode(value.encode() if isinstance(value, str) else value).decode()
for key, value in sorted(values.items())
}
payload = {
"apiVersion": "v1",
"kind": "Secret",
"metadata": {"name": name, "namespace": namespace},
"type": "Opaque",
"data": encoded,
}
return (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
def render_tls_secret_manifest(
*, namespace: str, name: str, certificate: bytes, private_key: bytes
) -> bytes:
payload = json.loads(
render_secret_manifest(
namespace=namespace,
name=name,
values={"tls.crt": certificate, "tls.key": private_key},
)
)
payload["type"] = "kubernetes.io/tls"
return (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
def render_hosts(config: LabConfig) -> str:
return (
f"{config.primary_control.address} {config.public_host}\n"
f"{config.state_node.address} {config.s3_host}\n"
)
def _env_quote(value: str) -> str:
return json.dumps(value, ensure_ascii=True)
def write_private(path: Path, value: str | bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
data = value.encode() if isinstance(value, str) else value
path.write_bytes(data)
path.chmod(0o600)
@@ -0,0 +1,11 @@
<network>
<name>govoplan-lab</name>
<forward mode="nat"/>
<bridge name="virbr-gplab" stp="on" delay="0"/>
<domain name="lab.test" localOnly="yes"/>
<ip address="192.168.123.1" netmask="255.255.255.0">
<dhcp>
<range start="192.168.123.2" end="192.168.123.99"/>
</dhcp>
</ip>
</network>
+1 -1
View File
@@ -51,7 +51,7 @@ set +a
export APP_ENV="${APP_ENV:-staging}" export APP_ENV="${APP_ENV:-staging}"
export GOVOPLAN_INSTALL_PROFILE="${GOVOPLAN_INSTALL_PROFILE:-production-like}" export GOVOPLAN_INSTALL_PROFILE="${GOVOPLAN_INSTALL_PROFILE:-production-like}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,connectors,datasources,dataflow,dist_lists,templates,workflow_engine,workflow,views,search,risk_compliance,notifications,docs,ops}" export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,connectors,datasources,dataflow,dist_lists,templates,workflow_engine,workflow,tasks,views,quick_access,search,risk_compliance,notifications,docs,ops}"
export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}" export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}" export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}" export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}"
+1 -1
View File
@@ -66,7 +66,7 @@ export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-po
export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}" export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}" export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}"
export CELERY_ENABLED="${CELERY_ENABLED:-true}" export CELERY_ENABLED="${CELERY_ENABLED:-true}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,connectors,datasources,dataflow,dist_lists,templates,workflow_engine,workflow,views,search,risk_compliance,notifications,docs,ops}" export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,connectors,datasources,dataflow,dist_lists,templates,workflow_engine,workflow,tasks,views,quick_access,search,risk_compliance,notifications,docs,ops}"
export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}" export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}"
export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}" export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}"
export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}" export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}"
+229 -296
View File
@@ -1,192 +1,55 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Generate and sign a GovOPlaN module package release catalog.""" """Generate a signed registry-backed GovOPlaN module package catalog."""
from __future__ import annotations from __future__ import annotations
import argparse import argparse
import base64 import base64
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
import hashlib
import json import json
import os import os
from pathlib import Path from pathlib import Path
import re
import sys import sys
from typing import Any from typing import Any
from urllib.parse import urlsplit
from cryptography.hazmat.primitives import serialization from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[2] META_ROOT = Path(__file__).resolve().parents[2]
CORE_ROOT = Path(os.environ.get("GOVOPLAN_CORE_ROOT", META_ROOT.parent / "govoplan-core")).resolve() CORE_ROOT = Path(os.environ.get("GOVOPLAN_CORE_ROOT", META_ROOT.parent / "govoplan-core")).resolve()
sys.path.insert(0, str(CORE_ROOT / "src")) sys.path.insert(0, str(CORE_ROOT / "src"))
sys.path.insert(0, str(META_ROOT / "tools" / "release")) sys.path.insert(0, str(META_ROOT / "tools" / "release"))
from govoplan_core.core.modules import ModuleManifest # noqa: E402 from govoplan_release.catalog_entry_synthesis import ( # noqa: E402
from govoplan_core.server.registry import available_module_manifests # noqa: E402 synthesize_repository_catalog_entries,
from govoplan_release.version_alignment import selected_repository_version_issues # noqa: E402 validate_initial_entry_closure,
GITEA_BASE = "git+ssh://git@git.add-ideas.de/GovOPlaN"
@dataclass(frozen=True, slots=True)
class CatalogModule:
module_id: str
repo: str
python_package: str
name: str
description: str
tags: tuple[str, ...]
webui_package: str | None = None
provides_interfaces: tuple[dict[str, object], ...] = ()
requires_interfaces: tuple[dict[str, object], ...] = ()
CATALOG_MODULES = (
CatalogModule(
module_id="tenancy",
repo="govoplan-tenancy",
python_package="govoplan-tenancy",
name="Tenancy",
description="Tenant registry, tenant settings, and tenant resolution platform module.",
tags=("official", "platform-module"),
webui_package="@govoplan/tenancy-webui",
),
CatalogModule(
module_id="organizations",
repo="govoplan-organizations",
python_package="govoplan-organizations",
name="Organizations",
description="Organization units, functions, and account-held function assignments.",
tags=("official", "platform-module"),
),
CatalogModule(
module_id="identity",
repo="govoplan-identity",
python_package="govoplan-identity",
name="Identity",
description="Canonical identities and links between identities and platform accounts.",
tags=("official", "platform-module"),
),
CatalogModule(
module_id="access",
repo="govoplan-access",
python_package="govoplan-access",
name="Access",
description="Authentication, accounts, users, groups, roles, API keys, and access capabilities.",
tags=("official", "platform-module"),
webui_package="@govoplan/access-webui",
),
CatalogModule(
module_id="admin",
repo="govoplan-admin",
python_package="govoplan-admin",
name="Admin",
description="System settings, governance templates, module management, and admin shell contributions.",
tags=("official", "platform-module"),
webui_package="@govoplan/admin-webui",
),
CatalogModule(
module_id="policy",
repo="govoplan-policy",
python_package="govoplan-policy",
name="Policy",
description="Policy and governance capability module.",
tags=("official", "platform-module"),
webui_package="@govoplan/policy-webui",
),
CatalogModule(
module_id="audit",
repo="govoplan-audit",
python_package="govoplan-audit",
name="Audit",
description="Audit-log storage and audit administration routes.",
tags=("official", "platform-module"),
webui_package="@govoplan/audit-webui",
),
CatalogModule(
module_id="dashboard",
repo="govoplan-dashboard",
python_package="govoplan-dashboard",
name="Dashboard",
description="Configurable user home assembled from module-provided dashboard widgets.",
tags=("official", "platform-module"),
webui_package="@govoplan/dashboard-webui",
),
CatalogModule(
module_id="addresses",
repo="govoplan-addresses",
python_package="govoplan-addresses",
name="Addresses",
description="Reusable address directories, recipient sources, consent metadata, and address quality workflows.",
tags=("official", "business-module"),
webui_package="@govoplan/addresses-webui",
),
CatalogModule(
module_id="files",
repo="govoplan-files",
python_package="govoplan-files",
name="Files",
description="Managed file spaces and campaign attachment integration.",
tags=("official", "service-module"),
webui_package="@govoplan/files-webui",
),
CatalogModule(
module_id="mail",
repo="govoplan-mail",
python_package="govoplan-mail",
name="Mail",
description="SMTP/IMAP profile management, credential policy, and read-only mailbox access.",
tags=("official", "service-module"),
webui_package="@govoplan/mail-webui",
),
CatalogModule(
module_id="campaigns",
repo="govoplan-campaign",
python_package="govoplan-campaign",
name="Campaigns",
description="Campaign authoring, validation, queueing, delivery control, and reports.",
tags=("official", "business-module"),
webui_package="@govoplan/campaign-webui",
),
CatalogModule(
module_id="calendar",
repo="govoplan-calendar",
python_package="govoplan-calendar",
name="Calendar",
description="Calendar collections, events, CalDAV sources, and calendar WebUI routes.",
tags=("official", "service-module"),
webui_package="@govoplan/calendar-webui",
),
CatalogModule(
module_id="docs",
repo="govoplan-docs",
python_package="govoplan-docs",
name="Docs",
description="Configured-system documentation and evidence-aware help surfaces.",
tags=("official", "platform-module"),
webui_package="@govoplan/docs-webui",
),
CatalogModule(
module_id="ops",
repo="govoplan-ops",
python_package="govoplan-ops",
name="Ops",
description="Runtime health, deployment profile, worker split, and sizing visibility.",
tags=("official", "platform-module"),
webui_package="@govoplan/ops-webui",
),
) )
from govoplan_release.module_directory import write_module_directory # noqa: E402
SHA256 = re.compile(r"^[0-9a-f]{64}$")
def main() -> int: def main() -> int:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--version", required=True, help="GovOPlaN release version, without leading v.") parser.add_argument("--version", required=True, help="Core release version, without leading v.")
parser.add_argument("--package-set", type=Path, required=True)
parser.add_argument("--package-lock", type=Path, required=True)
parser.add_argument("--workspace", type=Path, default=META_ROOT.parent)
parser.add_argument("--channel", default="stable") parser.add_argument("--channel", default="stable")
parser.add_argument("--sequence", type=int, help="Monotonic channel sequence. Defaults to UTC timestamp.") parser.add_argument("--sequence", type=int, help="Monotonic channel sequence. Defaults to UTC timestamp.")
parser.add_argument("--expires-days", type=int, default=90) parser.add_argument("--expires-days", type=int, default=90)
parser.add_argument("--catalog-output", type=Path, required=True) parser.add_argument("--catalog-output", type=Path, required=True)
parser.add_argument("--keyring-output", type=Path) parser.add_argument("--keyring-output", type=Path)
parser.add_argument(
"--module-directory-output",
type=Path,
help="Catalog v1 root under which the browsable modules directory is synchronized.",
)
parser.add_argument( parser.add_argument(
"--catalog-signing-key", "--catalog-signing-key",
action="append", action="append",
@@ -195,102 +58,165 @@ def main() -> int:
help="Ed25519 private key used to sign the catalog; may be repeated for rotation.", help="Ed25519 private key used to sign the catalog; may be repeated for rotation.",
) )
parser.add_argument("--public-base-url", default="https://govoplan.add-ideas.de") parser.add_argument("--public-base-url", default="https://govoplan.add-ideas.de")
parser.add_argument("--repository-base", default=GITEA_BASE)
args = parser.parse_args() args = parser.parse_args()
try:
version = args.version.removeprefix("v") version = args.version.removeprefix("v")
version_issues = selected_repository_version_issues( package_set = _read_hashed_json(args.package_set, hash_field="package_set_sha256")
repo_versions={ package_lock = _read_hashed_json(args.package_lock, hash_field="lock_sha256")
"govoplan-core": version, _validate_release_inputs(package_set, package_lock, core_version=version)
**{module.repo: version for module in CATALOG_MODULES}, signing_keys = [_parse_signing_key(value) for value in args.catalog_signing_key]
},
workspace=CORE_ROOT.parent,
)
if version_issues:
details = "; ".join(
f"{issue.repo}: {issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
for issue in version_issues
)
parser.error(f"version alignment gate failed: {details}")
tag = f"v{version}"
generated_at = datetime.now(tz=UTC) generated_at = datetime.now(tz=UTC)
sequence = args.sequence if args.sequence is not None else int(generated_at.strftime("%Y%m%d%H%M")) sequence = args.sequence if args.sequence is not None else int(generated_at.strftime("%Y%m%d%H%M"))
expires_at = generated_at + timedelta(days=args.expires_days)
signing_keys = [_parse_signing_key(value) for value in args.catalog_signing_key]
catalog = _catalog_payload( catalog = _catalog_payload(
version=version, package_set=package_set,
tag=tag, package_lock=package_lock,
channel=args.channel, channel=args.channel,
sequence=sequence, sequence=sequence,
generated_at=generated_at, generated_at=generated_at,
expires_at=expires_at, expires_at=generated_at + timedelta(days=args.expires_days),
repository_base=args.repository_base.rstrip("/"), workspace=args.workspace.expanduser().resolve(),
public_base_url=args.public_base_url.rstrip("/"), public_base_url=args.public_base_url.rstrip("/"),
) )
if signing_keys: if signing_keys:
catalog["signatures"] = [_signature(catalog, key_id=key_id, private_key=private_key) for key_id, private_key in signing_keys] catalog["signatures"] = [
_signature(catalog, key_id=key_id, private_key=private_key)
for key_id, private_key in signing_keys
]
except (KeyError, OSError, ValueError, json.JSONDecodeError) as exc:
parser.error(str(exc))
output = args.catalog_output.expanduser() output = args.catalog_output.expanduser()
output.parent.mkdir(parents=True, exist_ok=True) output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(catalog, indent=2, sort_keys=True) + "\n", encoding="utf-8") output.write_text(json.dumps(catalog, indent=2, sort_keys=True) + "\n", encoding="utf-8")
keyring = _keyring(signing_keys=signing_keys, generated_at=generated_at)
if args.keyring_output is not None: if args.keyring_output is not None:
keyring_output = args.keyring_output.expanduser() keyring_output = args.keyring_output.expanduser()
keyring_output.parent.mkdir(parents=True, exist_ok=True) keyring_output.parent.mkdir(parents=True, exist_ok=True)
keyring_output.write_text( keyring_output.write_text(
json.dumps(_keyring(signing_keys=signing_keys, generated_at=generated_at), indent=2, sort_keys=True) + "\n", json.dumps(keyring, indent=2, sort_keys=True) + "\n",
encoding="utf-8", encoding="utf-8",
) )
module_directory_files: tuple[Path, ...] = ()
if args.module_directory_output is not None:
module_directory_files = write_module_directory(
catalog_payload=catalog,
keyring_payload=keyring,
output_root=args.module_directory_output.expanduser(),
channel=args.channel,
public_base_url=args.public_base_url,
prune=True,
)
print(f"catalog={output}") print(f"catalog={output}")
if args.keyring_output is not None: if args.keyring_output is not None:
print(f"keyring={args.keyring_output.expanduser()}") print(f"keyring={args.keyring_output.expanduser()}")
if args.module_directory_output is not None:
print(f"module_directory={args.module_directory_output.expanduser()}")
print(f"module_directory_files={len(module_directory_files)}")
print(f"channel={args.channel}") print(f"channel={args.channel}")
print(f"sequence={sequence}") print(f"sequence={sequence}")
print(f"version={version}") print(f"version={version}")
print(f"profile={package_set.get('profile', 'base')}")
return 0 return 0
def _catalog_payload( def _catalog_payload(
*, *,
version: str, package_set: dict[str, Any],
tag: str, package_lock: dict[str, Any],
channel: str, channel: str,
sequence: int, sequence: int,
generated_at: datetime, generated_at: datetime,
expires_at: datetime, expires_at: datetime,
repository_base: str, workspace: Path,
public_base_url: str, public_base_url: str,
) -> dict[str, Any]: ) -> dict[str, Any]:
manifests = _discovered_catalog_manifests() python_lock = _rows_by_name(package_lock, "python")
modules: list[dict[str, Any]] = [] webui_lock = _rows_by_repository(package_lock, "webui")
for module in CATALOG_MODULES: modules: list[dict[str, object]] = []
manifest = manifests.get(module.module_id) core_release: dict[str, object] | None = None
module_version = manifest.version if manifest is not None else version selected_units: list[dict[str, str]] = []
module_tag = f"v{module_version.removeprefix('v')}"
entry: dict[str, Any] = { for package in package_set["python"]:
"module_id": module.module_id, name = str(package["name"])
"name": module.name, version = str(package["version"])
"description": module.description, repository = str(package["repository"])
"version": module_version, selected_units.append(
"action": "install", {
"python_package": module.python_package, "repo": repository,
"python_ref": f"{module.python_package} @ {repository_base}/{module.repo}.git@{module_tag}", "version": version,
"license_features": [f"module.{module.module_id}"], "tag": str(package["tag"]),
"tags": list(module.tags), "commit": str(package["commit"]),
} }
if module.webui_package: )
entry["webui_package"] = module.webui_package python_artifact = python_lock[name]
entry["webui_ref"] = f"{repository_base}/{module.repo}.git#{module_tag}" webui_artifact = webui_lock.get(repository)
manifest_metadata = _manifest_catalog_metadata(manifest) if name == "govoplan-core":
entry.update(manifest_metadata) core_release = {
if module.provides_interfaces: "name": "GovOPlaN Core",
entry["provides_interfaces"] = [dict(item) for item in module.provides_interfaces] "version": version,
if module.requires_interfaces: "python_package": name,
entry["requires_interfaces"] = [dict(item) for item in module.requires_interfaces] "python_ref": _python_ref(name, python_artifact, extras=tuple(package.get("extras") or ())),
"artifact_integrity": {
"python": _artifact_integrity(python_artifact, ref=_python_ref(name, python_artifact, extras=tuple(package.get("extras") or ())))
},
}
if webui_artifact is not None:
webui_ref = _artifact_url(webui_artifact)
core_release.update(
{
"webui_package": webui_artifact["name"],
"webui_ref": webui_ref,
}
)
core_release["artifact_integrity"]["webui"] = _artifact_integrity(webui_artifact, ref=webui_ref)
continue
entries = synthesize_repository_catalog_entries(
repo=repository,
version=version,
workspace=workspace,
repository_base="git+https://git.add-ideas.de/GovOPlaN",
source_ref=str(package["tag"]),
)
for entry in entries:
python_ref = _python_ref(name, python_artifact)
entry["python_ref"] = python_ref
repository_url = f"https://git.add-ideas.de/GovOPlaN/{repository}"
source_commit = str(package["commit"])
entry["source"] = {
"repository": repository,
"tag": package["tag"],
"commit": source_commit,
"repository_url": repository_url,
"revision_url": f"{repository_url}/commit/{source_commit}",
}
entry["availability"] = "available"
entry["release_notes_url"] = f"{repository_url}/releases/tag/{package['tag']}"
integrity: dict[str, object] = {
"python": _artifact_integrity(python_artifact, ref=python_ref),
}
if entry.get("webui_package"):
if webui_artifact is None or webui_artifact.get("name") != entry["webui_package"]:
raise ValueError(f"Package lock has no matching WebUI artifact for {repository}.")
webui_ref = _artifact_url(webui_artifact)
entry["webui_ref"] = webui_ref
integrity["webui"] = _artifact_integrity(webui_artifact, ref=webui_ref)
else:
entry.pop("webui_ref", None)
entry["artifact_integrity"] = integrity
modules.append(entry) modules.append(entry)
if core_release is None:
raise ValueError("Package set does not contain govoplan-core.")
validate_initial_entry_closure(
catalog_modules=modules,
initial_module_ids={str(item["module_id"]) for item in modules},
)
release_version = str(package_set["release_version"])
return { return {
"catalog_version": "1", "catalog_version": "1",
"channel": channel, "channel": channel,
@@ -298,96 +224,103 @@ def _catalog_payload(
"generated_at": _json_datetime(generated_at), "generated_at": _json_datetime(generated_at),
"expires_at": _json_datetime(expires_at), "expires_at": _json_datetime(expires_at),
"release": { "release": {
"version": version, "version": release_version,
"tag": tag, "tag": f"v{release_version}",
"profile": package_set.get("profile", "base"),
"catalog_url": f"{public_base_url}/catalogs/v1/channels/{channel}.json", "catalog_url": f"{public_base_url}/catalogs/v1/channels/{channel}.json",
"keyring_url": f"{public_base_url}/catalogs/v1/keyring.json", "keyring_url": f"{public_base_url}/catalogs/v1/keyring.json",
"package_set_sha256": package_set["package_set_sha256"],
"package_lock_sha256": package_lock["lock_sha256"],
"selected_units": sorted(selected_units, key=lambda item: item["repo"]),
}, },
"core_release": { "core_release": core_release,
"name": "GovOPlaN Core", "modules": sorted(modules, key=lambda item: str(item["module_id"])),
"version": version,
"python_package": "govoplan-core",
"python_ref": f"govoplan-core[server] @ {repository_base}/govoplan-core.git@{tag}",
"webui_package": "@govoplan/core-webui",
"webui_ref": f"{repository_base}/govoplan-core.git#{tag}",
},
"modules": modules,
} }
def _discovered_catalog_manifests() -> dict[str, ModuleManifest]: def _read_hashed_json(path: Path, *, hash_field: str) -> dict[str, Any]:
try: payload = json.loads(path.expanduser().read_text(encoding="utf-8"))
return available_module_manifests(ignore_load_errors=True) if not isinstance(payload, dict):
except Exception: raise ValueError(f"{path} must contain a JSON object.")
return {} expected = payload.get(hash_field)
unsigned = dict(payload)
unsigned.pop(hash_field, None)
def _manifest_catalog_metadata(manifest: ModuleManifest | None) -> dict[str, object]: if not isinstance(expected, str) or expected != _canonical_sha256(unsigned):
if manifest is None: raise ValueError(f"{path} {hash_field} does not match its contents.")
return {}
payload: dict[str, object] = {}
if manifest.dependencies:
payload["dependencies"] = list(manifest.dependencies)
if manifest.optional_dependencies:
payload["optional_dependencies"] = list(manifest.optional_dependencies)
if manifest.architecture is not None:
payload["architecture"] = manifest.architecture.to_dict()
if manifest.external_providers:
payload["external_providers"] = [
declaration.to_dict()
for declaration in manifest.external_providers
]
if manifest.migration_spec is not None:
payload["migration_safety"] = "requires_review"
payload["migration_notes"] = "Module owns database migrations; review release notes and migration output before activation."
if manifest.migration_spec.migration_after:
payload["migration_after"] = list(manifest.migration_spec.migration_after)
if manifest.migration_spec.migration_before:
payload["migration_before"] = list(manifest.migration_spec.migration_before)
if manifest.migration_spec.migration_tasks:
tasks: list[dict[str, object]] = []
for task in manifest.migration_spec.migration_tasks:
task_payload: dict[str, object] = {
"task_id": task.task_id,
"phase": task.phase,
"summary": task.summary,
"task_version": task.task_version,
"safety": task.safety,
"idempotent": task.idempotent,
}
if task.timeout_seconds is not None:
task_payload["timeout_seconds"] = task.timeout_seconds
tasks.append(task_payload)
payload["migration_tasks"] = tasks
if manifest.provides_interfaces:
payload["provides_interfaces"] = [
{"name": item.name, "version": item.version}
for item in manifest.provides_interfaces
]
if manifest.requires_interfaces:
requirements: list[dict[str, object]] = []
for item in manifest.requires_interfaces:
requirement: dict[str, object] = {
"name": item.name,
"optional": item.optional,
}
if item.version_min is not None:
requirement["version_min"] = item.version_min
if item.version_max_exclusive is not None:
requirement["version_max_exclusive"] = item.version_max_exclusive
requirements.append(requirement)
payload["requires_interfaces"] = requirements
return payload return payload
def _validate_release_inputs(package_set: dict[str, Any], package_lock: dict[str, Any], *, core_version: str) -> None:
if package_set.get("schema_version") != "1" or package_lock.get("schema_version") != "1":
raise ValueError("Package set and lock must use schema version 1.")
if package_set.get("release_version") != core_version or package_lock.get("release_version") != core_version:
raise ValueError("Package set and lock release versions must match --version.")
if package_lock.get("package_set_sha256") != package_set.get("package_set_sha256"):
raise ValueError("Package lock does not belong to the selected package set.")
if package_lock.get("profile", "base") != package_set.get("profile", "base"):
raise ValueError("Package set and lock profiles do not match.")
for group in ("python", "webui"):
selected = {(item.get("name"), item.get("version"), item.get("repository")) for item in package_set.get(group, ()) if isinstance(item, dict)}
locked = {(item.get("name"), item.get("version"), item.get("repository")) for item in package_lock.get(group, ()) if isinstance(item, dict)}
if not selected or selected != locked:
raise ValueError(f"Package lock does not contain the exact {group} package set.")
for item in package_lock[group]:
_artifact_url(item)
if SHA256.fullmatch(str(item.get("sha256") or "")) is None:
raise ValueError(f"Package lock has an invalid {group} artifact digest.")
def _rows_by_name(payload: dict[str, Any], group: str) -> dict[str, dict[str, object]]:
return {str(item["name"]): item for item in payload[group]}
def _rows_by_repository(payload: dict[str, Any], group: str) -> dict[str, dict[str, object]]:
result: dict[str, dict[str, object]] = {}
for item in payload[group]:
repository = str(item["repository"])
if repository in result:
raise ValueError(f"Package lock contains multiple {group} artifacts for {repository}.")
result[repository] = item
return result
def _artifact_url(artifact: dict[str, object]) -> str:
value = str(artifact.get("url") or "")
parsed = urlsplit(value)
if parsed.scheme != "https" or not parsed.netloc or parsed.username or parsed.password or parsed.fragment:
raise ValueError(f"Package artifact has an unsafe download URL: {value!r}.")
return value
def _python_ref(name: str, artifact: dict[str, object], *, extras: tuple[object, ...] = ()) -> str:
extra = f"[{','.join(str(item) for item in extras)}]" if extras else ""
return f"{name}{extra} @ {_artifact_url(artifact)}#sha256={artifact['sha256']}"
def _artifact_integrity(artifact: dict[str, object], *, ref: str) -> dict[str, object]:
result: dict[str, object] = {
"ref": ref,
"url": _artifact_url(artifact),
"filename": artifact["filename"],
"sha256": artifact["sha256"],
"size": artifact["size"],
"registry_identity": f"{artifact['name']}@{artifact['version']}",
"git_ref": artifact["tag"],
"source_commit": artifact["commit"],
}
if artifact.get("integrity"):
result["integrity"] = artifact["integrity"]
return result
def _parse_signing_key(value: str) -> tuple[str, Ed25519PrivateKey]: def _parse_signing_key(value: str) -> tuple[str, Ed25519PrivateKey]:
key_id, separator, path_text = value.partition("=") key_id, separator, path_text = value.partition("=")
if not separator or not key_id.strip() or not path_text.strip(): if not separator or not key_id.strip() or not path_text.strip():
raise SystemExit("--catalog-signing-key must use KEY_ID=/path/to/private.pem") raise ValueError("--catalog-signing-key must use KEY_ID=/path/to/private.pem")
path = Path(path_text).expanduser() path = Path(path_text).expanduser()
private_key = serialization.load_pem_private_key(path.read_bytes(), password=None) private_key = serialization.load_pem_private_key(path.read_bytes(), password=None)
if not isinstance(private_key, Ed25519PrivateKey): if not isinstance(private_key, Ed25519PrivateKey):
raise SystemExit(f"Catalog signing key must be an Ed25519 private key: {path}") raise ValueError(f"Catalog signing key must be an Ed25519 private key: {path}")
return key_id.strip(), private_key return key_id.strip(), private_key
@@ -395,11 +328,10 @@ def _signature(payload: dict[str, Any], *, key_id: str, private_key: Ed25519Priv
signature_payload = dict(payload) signature_payload = dict(payload)
signature_payload.pop("signature", None) signature_payload.pop("signature", None)
signature_payload.pop("signatures", None) signature_payload.pop("signatures", None)
signature = private_key.sign(_canonical_bytes(signature_payload))
return { return {
"algorithm": "ed25519", "algorithm": "ed25519",
"key_id": key_id, "key_id": key_id,
"value": base64.b64encode(signature).decode("ascii"), "value": base64.b64encode(private_key.sign(_canonical_bytes(signature_payload))).decode("ascii"),
} }
@@ -412,7 +344,12 @@ def _keyring(*, signing_keys: list[tuple[str, Ed25519PrivateKey]], generated_at:
{ {
"key_id": key_id, "key_id": key_id,
"status": "active", "status": "active",
"public_key": _public_key_base64(private_key), "public_key": base64.b64encode(
private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
).decode("ascii"),
"not_before": generated_at.date().isoformat() + "T00:00:00Z", "not_before": generated_at.date().isoformat() + "T00:00:00Z",
} }
for key_id, private_key in signing_keys for key_id, private_key in signing_keys
@@ -420,18 +357,14 @@ def _keyring(*, signing_keys: list[tuple[str, Ed25519PrivateKey]], generated_at:
} }
def _public_key_base64(private_key: Ed25519PrivateKey) -> str:
public_bytes = private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
return base64.b64encode(public_bytes).decode("ascii")
def _canonical_bytes(payload: object) -> bytes: def _canonical_bytes(payload: object) -> bytes:
return json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8") return json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _canonical_sha256(payload: object) -> str:
return hashlib.sha256(_canonical_bytes(payload)).hexdigest()
def _json_datetime(value: datetime) -> str: def _json_datetime(value: datetime) -> str:
return value.astimezone(UTC).isoformat().replace("+00:00", "Z") return value.astimezone(UTC).isoformat().replace("+00:00", "Z")
+64 -1
View File
@@ -21,6 +21,10 @@ GIT_REQUIREMENT = re.compile(
r"(?P<version>[A-Za-z0-9._+!-]+))$" r"(?P<version>[A-Za-z0-9._+!-]+))$"
) )
LOCAL_CORE = re.compile(r"^(?:-e\s+)?\.\./govoplan-core(?:\[(?P<extras>[^]]+)\])?$") LOCAL_CORE = re.compile(r"^(?:-e\s+)?\.\./govoplan-core(?:\[(?P<extras>[^]]+)\])?$")
EXACT_PACKAGE = re.compile(
r"^(?P<package>govoplan(?:-[a-z0-9-]+)?)(?:\[(?P<extras>[^]]+)\])?=="
r"(?P<version>[A-Za-z0-9._+!-]+)$"
)
class PackageSetError(ValueError): class PackageSetError(ValueError):
@@ -38,6 +42,17 @@ def build_parser() -> argparse.ArgumentParser:
type=Path, type=Path,
default=META_ROOT / "requirements-release.txt", default=META_ROOT / "requirements-release.txt",
) )
parser.add_argument(
"--profile",
choices=("base", "full"),
default="base",
help="Base runtime roots or every package selected by govoplan[full].",
)
parser.add_argument(
"--meta-package",
type=Path,
default=META_ROOT / "packages" / "govoplan-meta" / "pyproject.toml",
)
parser.add_argument("--workspace", type=Path, default=META_ROOT.parent) parser.add_argument("--workspace", type=Path, default=META_ROOT.parent)
parser.add_argument("--output", type=Path, required=True) parser.add_argument("--output", type=Path, required=True)
return parser return parser
@@ -79,11 +94,46 @@ def parse_release_requirements(path: Path, *, core_version: str) -> tuple[dict[s
return tuple(values) return tuple(values)
def parse_meta_package(path: Path, *, core_version: str) -> tuple[dict[str, object], ...]:
project = tomllib.loads(path.read_text(encoding="utf-8")).get("project")
if not isinstance(project, dict):
raise PackageSetError("developer meta-package has no [project] table")
if str(project.get("name") or "") != "govoplan":
raise PackageSetError("developer meta-package has an unexpected project name")
if str(project.get("version") or "").removeprefix("v") != core_version.removeprefix("v"):
raise PackageSetError("developer meta-package version does not match Core")
optional = project.get("optional-dependencies")
full = optional.get("full") if isinstance(optional, dict) else None
dependencies = project.get("dependencies")
if not isinstance(dependencies, list) or not isinstance(full, list):
raise PackageSetError("developer meta-package must declare dependencies and the full extra")
values: list[dict[str, object]] = []
for raw in (*dependencies, *full):
if not isinstance(raw, str) or (match := EXACT_PACKAGE.fullmatch(raw.strip())) is None:
raise PackageSetError(f"developer meta-package requirement is not exact: {raw!r}")
package = match.group("package")
repository = "govoplan-core" if package == "govoplan-core" else package
values.append(
{
"name": package,
"version": match.group("version"),
"repository": repository,
"extras": _extras(match.group("extras")),
}
)
names = [str(item["name"]) for item in values]
if names.count("govoplan-core") != 1 or len(names) != len(set(names)):
raise PackageSetError("developer meta-package must contain one Core and unique packages")
return tuple(values)
def generate_package_set( def generate_package_set(
*, *,
core_version: str, core_version: str,
requirements: Path, requirements: Path,
workspace: Path, workspace: Path,
profile: str = "base",
meta_package: Path | None = None,
) -> dict[str, object]: ) -> dict[str, object]:
core_version = core_version.removeprefix("v") core_version = core_version.removeprefix("v")
if VERSION.fullmatch(core_version) is None: if VERSION.fullmatch(core_version) is None:
@@ -91,7 +141,17 @@ def generate_package_set(
python_packages: list[dict[str, object]] = [] python_packages: list[dict[str, object]] = []
webui_packages: list[dict[str, object]] = [] webui_packages: list[dict[str, object]] = []
seen_webui: set[str] = set() seen_webui: set[str] = set()
for requirement in parse_release_requirements(requirements, core_version=core_version): if profile not in {"base", "full"}:
raise PackageSetError(f"unsupported release profile: {profile}")
selected = (
parse_meta_package(
meta_package or META_ROOT / "packages" / "govoplan-meta" / "pyproject.toml",
core_version=core_version,
)
if profile == "full"
else parse_release_requirements(requirements, core_version=core_version)
)
for requirement in selected:
repository = workspace / str(requirement["repository"]) repository = workspace / str(requirement["repository"])
tag = f"v{requirement['version']}" tag = f"v{requirement['version']}"
if not (repository / ".git").is_dir(): if not (repository / ".git").is_dir():
@@ -134,6 +194,7 @@ def generate_package_set(
payload: dict[str, object] = { payload: dict[str, object] = {
"schema_version": "1", "schema_version": "1",
"release_version": core_version, "release_version": core_version,
"profile": profile,
"registries": { "registries": {
"python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple", "python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
"npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/", "npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/",
@@ -181,6 +242,8 @@ def main() -> int:
core_version=version, core_version=version,
requirements=args.requirements.expanduser().resolve(), requirements=args.requirements.expanduser().resolve(),
workspace=workspace, workspace=workspace,
profile=args.profile,
meta_package=args.meta_package.expanduser().resolve(),
) )
except (PackageSetError, OSError, ValueError, subprocess.CalledProcessError) as exc: except (PackageSetError, OSError, ValueError, subprocess.CalledProcessError) as exc:
print(f"error: {exc}") print(f"error: {exc}")
@@ -10,6 +10,8 @@ import os
from pathlib import Path from pathlib import Path
import subprocess import subprocess
import sys import sys
import tarfile
import tempfile
import tomllib import tomllib
from types import ModuleType from types import ModuleType
from typing import Iterator from typing import Iterator
@@ -30,13 +32,14 @@ def synthesize_repository_catalog_entries(
version: str, version: str,
workspace: Path, workspace: Path,
repository_base: str, repository_base: str,
source_ref: str | None = None,
) -> tuple[dict[str, object], ...]: ) -> tuple[dict[str, object], ...]:
"""Build install entries from tagged, preflighted local source metadata. """Build install entries from tagged, preflighted local source metadata.
The caller owns source-tag and worktree provenance checks. This function Distribution metadata identifies the runtime entry point and the runtime
accepts no hand-maintained module catalog registry: distribution metadata ``ModuleManifest`` is the authoritative dependency/interface/frontend
identifies the runtime entry point and the runtime ``ModuleManifest`` is description. When ``source_ref`` is supplied, metadata is read from that
the authoritative dependency/interface/frontend description. immutable Git tree rather than from the current checkout.
""" """
if os.getenv(_INSPECTION_CHILD) == "1": if os.getenv(_INSPECTION_CHILD) == "1":
@@ -45,6 +48,7 @@ def synthesize_repository_catalog_entries(
version=version, version=version,
workspace=workspace, workspace=workspace,
repository_base=repository_base, repository_base=repository_base,
source_ref=source_ref,
) )
command = ( command = (
sys.executable, sys.executable,
@@ -59,6 +63,8 @@ def synthesize_repository_catalog_entries(
"--repository-base", "--repository-base",
repository_base, repository_base,
) )
if source_ref:
command = (*command, "--source-ref", source_ref)
environment = os.environ.copy() environment = os.environ.copy()
environment[_INSPECTION_CHILD] = "1" environment[_INSPECTION_CHILD] = "1"
release_root = str(Path(__file__).resolve().parents[1]) release_root = str(Path(__file__).resolve().parents[1])
@@ -98,12 +104,14 @@ def synthesize_repository_catalog_entries_in_process(
version: str, version: str,
workspace: Path, workspace: Path,
repository_base: str, repository_base: str,
source_ref: str | None = None,
) -> tuple[dict[str, object], ...]: ) -> tuple[dict[str, object], ...]:
specs = {item.name: item for item in load_repository_specs(include_website=False)} specs = {item.name: item for item in load_repository_specs(include_website=False)}
spec = specs.get(repo) spec = specs.get(repo)
if spec is None: if spec is None:
raise ValueError(f"Cannot synthesize {repo}: repository is not registered.") raise ValueError(f"Cannot synthesize {repo}: repository is not registered.")
root = resolve_repo_path(spec, workspace) checkout_root = resolve_repo_path(spec, workspace)
with materialized_source_tree(checkout_root, source_ref=source_ref) as root:
project = read_project_metadata(root / "pyproject.toml") project = read_project_metadata(root / "pyproject.toml")
package = required_text(project, "name", source=f"{repo}/pyproject.toml") package = required_text(project, "name", source=f"{repo}/pyproject.toml")
project_version = required_text(project, "version", source=f"{repo}/pyproject.toml").removeprefix("v") project_version = required_text(project, "version", source=f"{repo}/pyproject.toml").removeprefix("v")
@@ -156,8 +164,7 @@ def manifest_catalog_entry(
"action": "install", "action": "install",
"python_package": package, "python_package": package,
"python_ref": f"{package} @ {repository_base}/{repo}.git@{tag}", "python_ref": f"{package} @ {repository_base}/{repo}.git@{tag}",
"license_features": [f"module.{manifest.id}"], "tags": ["official", "open-source"],
"tags": ["official"],
} }
if description: if description:
entry["description"] = description entry["description"] = description
@@ -165,8 +172,26 @@ def manifest_catalog_entry(
entry["dependencies"] = list(manifest.dependencies) entry["dependencies"] = list(manifest.dependencies)
if manifest.optional_dependencies: if manifest.optional_dependencies:
entry["optional_dependencies"] = list(manifest.optional_dependencies) entry["optional_dependencies"] = list(manifest.optional_dependencies)
if manifest.permissions:
entry["permissions"] = [
{
"scope": permission.scope,
"label": permission.label,
"description": permission.description,
"category": permission.category,
"level": permission.level,
"resource": permission.resource,
"action": permission.action,
"deprecated": permission.deprecated,
}
for permission in sorted(
manifest.permissions,
key=lambda item: item.scope,
)
]
if manifest.architecture is not None: if manifest.architecture is not None:
entry["architecture"] = manifest.architecture.to_dict() entry["architecture"] = manifest.architecture.to_dict()
entry["information_governance"] = manifest.information_governance.to_dict()
if manifest.external_providers: if manifest.external_providers:
entry["external_providers"] = [ entry["external_providers"] = [
declaration.to_dict() declaration.to_dict()
@@ -217,6 +242,51 @@ def manifest_catalog_entry(
return entry return entry
@contextmanager
def materialized_source_tree(root: Path, *, source_ref: str | None) -> Iterator[Path]:
if not source_ref:
yield root
return
if not (root / ".git").exists():
raise ValueError(f"Cannot inspect {source_ref!r}: {root} is not a Git checkout.")
with tempfile.TemporaryDirectory(prefix="govoplan-catalog-source-") as value:
temporary = Path(value)
archive_path = temporary / "source.tar"
source_root = temporary / "source"
source_root.mkdir()
result = subprocess.run(
[
"git",
"-C",
str(root),
"archive",
"--format=tar",
f"--output={archive_path}",
source_ref,
],
check=False,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
text=True,
timeout=30,
)
if result.returncode != 0:
detail = result.stderr.strip() or "Git archive failed"
raise ValueError(f"Cannot inspect {source_ref!r} in {root.name}: {detail}")
if archive_path.stat().st_size > 256 * 1024 * 1024:
raise ValueError(f"Cannot inspect {source_ref!r} in {root.name}: source archive exceeds 256 MiB.")
with tarfile.open(archive_path, mode="r:") as archive:
members = archive.getmembers()
if len(members) > 50_000:
raise ValueError(f"Cannot inspect {source_ref!r} in {root.name}: source archive has too many entries.")
for member in members:
path = Path(member.name)
if path.is_absolute() or ".." in path.parts or member.issym() or member.islnk() or member.isdev():
raise ValueError(f"Cannot inspect {source_ref!r} in {root.name}: source archive contains an unsafe entry.")
archive.extractall(source_root, members=members, filter="data")
yield source_root
def validate_initial_entry_closure( def validate_initial_entry_closure(
*, *,
catalog_modules: list[object], catalog_modules: list[object],
@@ -366,6 +436,7 @@ def main() -> int:
parser.add_argument("--version", required=True) parser.add_argument("--version", required=True)
parser.add_argument("--workspace", type=Path, required=True) parser.add_argument("--workspace", type=Path, required=True)
parser.add_argument("--repository-base", required=True) parser.add_argument("--repository-base", required=True)
parser.add_argument("--source-ref")
args = parser.parse_args() args = parser.parse_args()
try: try:
entries = synthesize_repository_catalog_entries( entries = synthesize_repository_catalog_entries(
@@ -373,6 +444,7 @@ def main() -> int:
version=args.version, version=args.version,
workspace=args.workspace.resolve(), workspace=args.workspace.resolve(),
repository_base=args.repository_base, repository_base=args.repository_base,
source_ref=args.source_ref,
) )
except ValueError as exc: except ValueError as exc:
print(str(exc), file=sys.stderr) print(str(exc), file=sys.stderr)
@@ -20,6 +20,7 @@ def write_module_directory(
output_root: Path, output_root: Path,
channel: str, channel: str,
public_base_url: str = DEFAULT_PUBLIC_BASE_URL, public_base_url: str = DEFAULT_PUBLIC_BASE_URL,
prune: bool = False,
) -> tuple[Path, ...]: ) -> tuple[Path, ...]:
payloads = module_directory_payloads( payloads = module_directory_payloads(
catalog_payload=catalog_payload, catalog_payload=catalog_payload,
@@ -29,13 +30,62 @@ def write_module_directory(
) )
written: list[Path] = [] written: list[Path] = []
for relative_path, payload in payloads: for relative_path, payload in payloads:
path = output_root / relative_path path = _prepare_output_path(output_root=output_root, relative_path=relative_path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8") path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
written.append(path) written.append(path)
if prune:
_prune_stale_module_directory_files(
output_root=output_root,
expected={relative_path for relative_path, _payload in payloads},
)
return tuple(written) return tuple(written)
def _prepare_output_path(*, output_root: Path, relative_path: Path) -> Path:
if output_root.is_symlink():
raise ValueError("module-directory output root must not be a symlink")
output_root.mkdir(parents=True, exist_ok=True)
current = output_root
for part in relative_path.parent.parts:
current = current / part
if current.is_symlink():
raise ValueError("module-directory path must not contain symlinks")
if current.exists():
if not current.is_dir():
raise ValueError("module-directory parent path must be a directory")
continue
current.mkdir()
path = output_root / relative_path
if path.is_symlink() or (path.exists() and not path.is_file()):
raise ValueError("module-directory output path must be a regular file")
return path
def _prune_stale_module_directory_files(
*,
output_root: Path,
expected: set[Path],
) -> None:
module_root = output_root / "modules"
if module_root.is_symlink():
raise ValueError("module-directory root must not be a symlink")
if not module_root.exists():
return
for path in module_root.rglob("*.json"):
relative_path = path.relative_to(output_root)
if relative_path not in expected:
path.unlink()
for path in sorted(
(item for item in module_root.rglob("*") if item.is_dir()),
key=lambda item: len(item.parts),
reverse=True,
):
try:
path.rmdir()
except OSError:
pass
def module_directory_payloads( def module_directory_payloads(
*, *,
catalog_payload: dict[str, Any], catalog_payload: dict[str, Any],
@@ -185,17 +185,24 @@ def module_rows(payload: object) -> list[dict[str, object]]:
for item in raw_modules: for item in raw_modules:
if not isinstance(item, dict): if not isinstance(item, dict):
continue continue
repo = module_repo(item) source = item.get("source") if isinstance(item.get("source"), dict) else {}
repo = string(source.get("repository")) or module_repo(item)
source_tag = string(source.get("tag"))
modules.append( modules.append(
{ {
"module_id": string(item.get("module_id")), "module_id": string(item.get("module_id")),
"name": string(item.get("name")), "name": string(item.get("name")),
"description": string(item.get("description")),
"version": string(item.get("version")), "version": string(item.get("version")),
"repo": repo, "repo": repo,
"source": dict(source),
"python_ref": string(item.get("python_ref")), "python_ref": string(item.get("python_ref")),
"python_tag": ref_tag(string(item.get("python_ref"))), "python_tag": source_tag or ref_tag(string(item.get("python_ref"))),
"webui_ref": string(item.get("webui_ref")), "webui_ref": string(item.get("webui_ref")),
"webui_tag": ref_tag(string(item.get("webui_ref"))), "webui_tag": source_tag or ref_tag(string(item.get("webui_ref"))),
"artifact_integrity": dict(item.get("artifact_integrity")) if isinstance(item.get("artifact_integrity"), dict) else {},
"dependencies": tuple(str(value) for value in item.get("dependencies", ()) if isinstance(value, str)) if isinstance(item.get("dependencies"), list) else (),
"optional_dependencies": tuple(str(value) for value in item.get("optional_dependencies", ()) if isinstance(value, str)) if isinstance(item.get("optional_dependencies"), list) else (),
"provides_interfaces": interface_list(item.get("provides_interfaces")), "provides_interfaces": interface_list(item.get("provides_interfaces")),
"requires_interfaces": requirement_list(item.get("requires_interfaces")), "requires_interfaces": requirement_list(item.get("requires_interfaces")),
"migration_safety": string(item.get("migration_safety")), "migration_safety": string(item.get("migration_safety")),
@@ -158,6 +158,7 @@ def build_selective_catalog_candidate(
repository_base=repository_base.rstrip("/"), repository_base=repository_base.rstrip("/"),
workspace=workspace, workspace=workspace,
) )
changes.extend(remove_official_license_requirements(candidate))
changes.extend( changes.extend(
apply_python_artifact_identities( apply_python_artifact_identities(
candidate, candidate,
@@ -728,6 +729,32 @@ def module_entry_repo(entry: dict[str, Any]) -> str | None:
return str(package).split("[", 1)[0] if isinstance(package, str) and package.startswith("govoplan-") else None return str(package).split("[", 1)[0] if isinstance(package, str) and package.startswith("govoplan-") else None
def remove_official_license_requirements(payload: dict[str, Any]) -> list[CatalogEntryChange]:
"""Official open-source modules never require commercial entitlements."""
modules = payload.get("modules")
if not isinstance(modules, list):
return []
changes: list[CatalogEntryChange] = []
for entry in modules:
if not isinstance(entry, dict):
continue
tags = entry.get("tags")
if not isinstance(tags, list) or "official" not in tags or "license_features" not in entry:
continue
before = entry.pop("license_features")
changes.append(
CatalogEntryChange(
repo=module_entry_repo(entry) or "unknown",
module_id=str(entry.get("module_id") or "") or None,
field="license_features",
before=json.dumps(before, sort_keys=True),
after=None,
)
)
return changes
def apply_python_artifact_identities( def apply_python_artifact_identities(
payload: dict[str, Any], payload: dict[str, Any],
*, *,
@@ -68,6 +68,7 @@ from __future__ import annotations
import hashlib import hashlib
import json import json
import os
from pathlib import Path from pathlib import Path
import sys import sys
@@ -91,11 +92,18 @@ for item in lock["webui"]:
if item["name"] in rows: if item["name"] in rows:
raise SystemExit(f"WebUI package lock contains duplicate artifact {item['name']}") raise SystemExit(f"WebUI package lock contains duplicate artifact {item['name']}")
rows[item["name"]] = item rows[item["name"]] = item
install_all = os.environ.get("GOVOPLAN_WEBUI_INSTALL_ALL_PACKAGES", "").strip().lower() in {"1", "true", "yes", "on"}
names = (
sorted(name for name in rows if name != "@govoplan/core-webui")
if install_all
else [
line.split("\t", 1)[0]
for line in output.read_text(encoding="utf-8").splitlines()
if line
]
)
requested = [] requested = []
for line in output.read_text(encoding="utf-8").splitlines(): for name in names:
if not line:
continue
name, _source_ref = line.split("\t", 1)
row = rows.get(name) row = rows.get(name)
if not isinstance(row, dict): if not isinstance(row, dict):
raise SystemExit(f"WebUI package lock has no artifact for {name}") raise SystemExit(f"WebUI package lock has no artifact for {name}")
+5
View File
@@ -154,6 +154,11 @@ def prepare_context(
output / "nginx.conf", output / "nginx.conf",
source_date_epoch=source_date_epoch, source_date_epoch=source_date_epoch,
) )
_copy_regular(
Path(__file__).resolve().parent / "runtime" / "web-entrypoint.sh",
output / "web-entrypoint.sh",
source_date_epoch=source_date_epoch,
)
return composition return composition
+31 -4
View File
@@ -15,6 +15,7 @@ Options:
--channel <name> Catalog channel. Defaults to stable. --channel <name> Catalog channel. Defaults to stable.
--sequence <number> Monotonic channel sequence. Defaults to UTC timestamp. --sequence <number> Monotonic channel sequence. Defaults to UTC timestamp.
--expires-days <days> Catalog expiry window. Defaults to 90. --expires-days <days> Catalog expiry window. Defaults to 90.
--profile <base|full> Package composition to publish. Defaults to full.
--catalog-signing-key <key-id=/path/private.pem> --catalog-signing-key <key-id=/path/private.pem>
Ed25519 private key. May be repeated for rotation. Ed25519 private key. May be repeated for rotation.
--core-root <path> govoplan-core checkout. Defaults to ../govoplan-core. --core-root <path> govoplan-core checkout. Defaults to ../govoplan-core.
@@ -46,6 +47,7 @@ VERSION=""
CHANNEL="stable" CHANNEL="stable"
SEQUENCE="" SEQUENCE=""
EXPIRES_DAYS="90" EXPIRES_DAYS="90"
PROFILE="full"
PUBLIC_BASE_URL="https://govoplan.add-ideas.de" PUBLIC_BASE_URL="https://govoplan.add-ideas.de"
REMOTE="origin" REMOTE="origin"
BRANCH="" BRANCH=""
@@ -95,6 +97,11 @@ while [[ $# -gt 0 ]]; do
EXPIRES_DAYS="$2" EXPIRES_DAYS="$2"
shift 2 shift 2
;; ;;
--profile)
[[ $# -ge 2 ]] || fail "missing value for $1"
PROFILE="$2"
shift 2
;;
--catalog-signing-key) --catalog-signing-key)
[[ $# -ge 2 ]] || fail "missing value for $1" [[ $# -ge 2 ]] || fail "missing value for $1"
SIGNING_KEYS+=("$2") SIGNING_KEYS+=("$2")
@@ -166,14 +173,12 @@ done
[[ -n "$VERSION" ]] || fail "--version is required" [[ -n "$VERSION" ]] || fail "--version is required"
[[ "$VERSION" =~ ^[0-9]+[.][0-9]+[.][0-9]+$ ]] || fail "version must be x.y.z: $VERSION" [[ "$VERSION" =~ ^[0-9]+[.][0-9]+[.][0-9]+$ ]] || fail "version must be x.y.z: $VERSION"
[[ "$PROFILE" == "base" || "$PROFILE" == "full" ]] || fail "profile must be base or full: $PROFILE"
[[ ${#SIGNING_KEYS[@]} -gt 0 ]] || fail "at least one --catalog-signing-key is required" [[ ${#SIGNING_KEYS[@]} -gt 0 ]] || fail "at least one --catalog-signing-key is required"
[[ -d "$CORE_ROOT/.git" ]] || fail "not a govoplan-core git repo: $CORE_ROOT" [[ -d "$CORE_ROOT/.git" ]] || fail "not a govoplan-core git repo: $CORE_ROOT"
[[ -d "$WEB_ROOT/.git" ]] || fail "not an addideas-govoplan-website git repo: $WEB_ROOT" [[ -d "$WEB_ROOT/.git" ]] || fail "not an addideas-govoplan-website git repo: $WEB_ROOT"
command -v "$PYTHON" >/dev/null 2>&1 || fail "Python not found: $PYTHON" command -v "$PYTHON" >/dev/null 2>&1 || fail "Python not found: $PYTHON"
if [[ "$BUILD_WEB" -eq 1 ]]; then
command -v "$NPM_BIN" >/dev/null 2>&1 || fail "npm not found: $NPM_BIN" command -v "$NPM_BIN" >/dev/null 2>&1 || fail "npm not found: $NPM_BIN"
fi
if [[ -z "$BRANCH" ]]; then if [[ -z "$BRANCH" ]]; then
BRANCH="$(git -C "$WEB_ROOT" symbolic-ref --quiet --short HEAD || true)" BRANCH="$(git -C "$WEB_ROOT" symbolic-ref --quiet --short HEAD || true)"
@@ -182,7 +187,12 @@ fi
CATALOG_PATH="$WEB_ROOT/public/catalogs/v1/channels/$CHANNEL.json" CATALOG_PATH="$WEB_ROOT/public/catalogs/v1/channels/$CHANNEL.json"
KEYRING_PATH="$WEB_ROOT/public/catalogs/v1/keyring.json" KEYRING_PATH="$WEB_ROOT/public/catalogs/v1/keyring.json"
MODULE_DIRECTORY_PATH="$WEB_ROOT/public/catalogs/v1/modules"
TAG_NAME="catalog-v$VERSION" TAG_NAME="catalog-v$VERSION"
TEMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/govoplan-release-catalog.XXXXXXXX")"
PACKAGE_SET="$TEMP_ROOT/release-packages.json"
PACKAGE_LOCK="$TEMP_ROOT/package-artifacts.lock.json"
trap 'rm -rf "$TEMP_ROOT"' EXIT
run() { run() {
printf '+' printf '+'
@@ -197,10 +207,14 @@ GEN_ARGS=(
env "GOVOPLAN_CORE_ROOT=$CORE_ROOT" env "GOVOPLAN_CORE_ROOT=$CORE_ROOT"
"$PYTHON" "$META_ROOT/tools/release/generate-release-catalog.py" "$PYTHON" "$META_ROOT/tools/release/generate-release-catalog.py"
--version "$VERSION" --version "$VERSION"
--package-set "$PACKAGE_SET"
--package-lock "$PACKAGE_LOCK"
--workspace "$PARENT"
--channel "$CHANNEL" --channel "$CHANNEL"
--expires-days "$EXPIRES_DAYS" --expires-days "$EXPIRES_DAYS"
--catalog-output "$CATALOG_PATH" --catalog-output "$CATALOG_PATH"
--keyring-output "$KEYRING_PATH" --keyring-output "$KEYRING_PATH"
--module-directory-output "$WEB_ROOT/public/catalogs/v1"
--public-base-url "$PUBLIC_BASE_URL" --public-base-url "$PUBLIC_BASE_URL"
) )
if [[ -n "$SEQUENCE" ]]; then if [[ -n "$SEQUENCE" ]]; then
@@ -210,6 +224,18 @@ for signing_key in "${SIGNING_KEYS[@]}"; do
GEN_ARGS+=(--catalog-signing-key "$signing_key") GEN_ARGS+=(--catalog-signing-key "$signing_key")
done done
run "$PYTHON" "$META_ROOT/tools/release/generate-release-package-set.py" \
--version "$VERSION" \
--profile "$PROFILE" \
--workspace "$PARENT" \
--output "$PACKAGE_SET"
run "$PYTHON" "$META_ROOT/tools/release/resolve-package-artifacts.py" \
--package-set "$PACKAGE_SET" \
--wheelhouse "$TEMP_ROOT/wheelhouse" \
--webui-packages "$TEMP_ROOT/webui-packages" \
--lock-output "$PACKAGE_LOCK" \
--python "$PYTHON" \
--npm "$NPM_BIN"
run "${GEN_ARGS[@]}" run "${GEN_ARGS[@]}"
if [[ "$DRY_RUN" -eq 0 ]]; then if [[ "$DRY_RUN" -eq 0 ]]; then
@@ -228,7 +254,7 @@ if [[ "$BUILD_WEB" -eq 1 ]]; then
fi fi
if [[ "$COMMIT" -eq 1 ]]; then if [[ "$COMMIT" -eq 1 ]]; then
run git -C "$WEB_ROOT" add "$CATALOG_PATH" "$KEYRING_PATH" run git -C "$WEB_ROOT" add -A "$CATALOG_PATH" "$KEYRING_PATH" "$MODULE_DIRECTORY_PATH"
if [[ "$DRY_RUN" -eq 0 ]]; then if [[ "$DRY_RUN" -eq 0 ]]; then
if git -C "$WEB_ROOT" diff --cached --quiet; then if git -C "$WEB_ROOT" diff --cached --quiet; then
echo "No addideas-govoplan-website catalog changes to commit." echo "No addideas-govoplan-website catalog changes to commit."
@@ -258,4 +284,5 @@ fi
echo "Catalog ready:" echo "Catalog ready:"
echo " $CATALOG_PATH" echo " $CATALOG_PATH"
echo " $KEYRING_PATH" echo " $KEYRING_PATH"
echo " $MODULE_DIRECTORY_PATH"
echo " URL: $PUBLIC_BASE_URL/catalogs/v1/channels/$CHANNEL.json" echo " URL: $PUBLIC_BASE_URL/catalogs/v1/channels/$CHANNEL.json"
+1
View File
@@ -114,6 +114,7 @@ PACKAGE_MODULE_REPOS=(
"$PARENT/govoplan-poll" "$PARENT/govoplan-poll"
"$PARENT/govoplan-portal" "$PARENT/govoplan-portal"
"$PARENT/govoplan-procurement" "$PARENT/govoplan-procurement"
"$PARENT/govoplan-quick-access"
"$PARENT/govoplan-records" "$PARENT/govoplan-records"
"$PARENT/govoplan-resources" "$PARENT/govoplan-resources"
"$PARENT/govoplan-risk-compliance" "$PARENT/govoplan-risk-compliance"
+80 -8
View File
@@ -64,17 +64,26 @@ def resolve(args: argparse.Namespace) -> dict[str, object]:
python=args.python, python=args.python,
index_url=str(package_set["registries"]["python"]), index_url=str(package_set["registries"]["python"]),
) )
_download_webui( webui_registry_metadata = _download_webui(
packages=tuple(package_set["webui"]), packages=tuple(package_set["webui"]),
destination=webui, destination=webui,
npm=args.npm, npm=args.npm,
registry=str(package_set["registries"]["npm"]), registry=str(package_set["registries"]["npm"]),
) )
python_rows = _verify_wheels(tuple(package_set["python"]), wheels) python_rows = _verify_wheels(
webui_rows = _verify_webui(tuple(package_set["webui"]), webui) tuple(package_set["python"]),
wheels,
registry=str(package_set["registries"]["python"]),
)
webui_rows = _verify_webui(
tuple(package_set["webui"]),
webui,
registry_metadata=webui_registry_metadata,
)
lock: dict[str, object] = { lock: dict[str, object] = {
"schema_version": "1", "schema_version": "1",
"release_version": package_set["release_version"], "release_version": package_set["release_version"],
"profile": package_set.get("profile", "base"),
"package_set_sha256": package_set["package_set_sha256"], "package_set_sha256": package_set["package_set_sha256"],
"registries": package_set["registries"], "registries": package_set["registries"],
"python": python_rows, "python": python_rows,
@@ -169,7 +178,7 @@ def _download_wheels(
def _download_webui( def _download_webui(
*, packages: tuple[dict[str, object], ...], destination: Path, npm: str, registry: str *, packages: tuple[dict[str, object], ...], destination: Path, npm: str, registry: str
) -> None: ) -> dict[str, dict[str, str]]:
environment = dict(os.environ) environment = dict(os.environ)
npmrc: tempfile.NamedTemporaryFile[bytes] | None = None npmrc: tempfile.NamedTemporaryFile[bytes] | None = None
token = os.environ.get("GOVOPLAN_PACKAGE_TOKEN", "") token = os.environ.get("GOVOPLAN_PACKAGE_TOKEN", "")
@@ -182,7 +191,31 @@ def _download_webui(
os.chmod(npmrc.name, 0o600) os.chmod(npmrc.name, 0o600)
environment["NPM_CONFIG_USERCONFIG"] = npmrc.name environment["NPM_CONFIG_USERCONFIG"] = npmrc.name
try: try:
metadata: dict[str, dict[str, str]] = {}
for item in packages: for item in packages:
view = subprocess.run(
[npm, "view", f"{item['name']}@{item['version']}", "dist", "--json", "--registry", registry],
check=True,
env=environment,
text=True,
stdout=subprocess.PIPE,
)
dist = json.loads(view.stdout)
if not isinstance(dist, dict):
raise PackageArtifactError(f"npm registry returned no distribution metadata for {item['name']}")
tarball = dist.get("tarball")
integrity = dist.get("integrity")
parsed = urlsplit(str(tarball or ""))
if (
parsed.scheme != "https"
or not parsed.netloc
or parsed.username
or parsed.password
or not isinstance(integrity, str)
or not integrity.startswith("sha512-")
):
raise PackageArtifactError(f"npm registry returned unsafe distribution metadata for {item['name']}")
metadata[str(item["name"])] = {"url": str(tarball), "integrity": integrity}
subprocess.run( subprocess.run(
[npm, "pack", f"{item['name']}@{item['version']}", "--ignore-scripts", "--pack-destination", str(destination), "--registry", registry], [npm, "pack", f"{item['name']}@{item['version']}", "--ignore-scripts", "--pack-destination", str(destination), "--registry", registry],
check=True, check=True,
@@ -191,9 +224,15 @@ def _download_webui(
finally: finally:
if npmrc is not None: if npmrc is not None:
Path(npmrc.name).unlink(missing_ok=True) Path(npmrc.name).unlink(missing_ok=True)
return metadata
def _verify_wheels(packages: tuple[dict[str, object], ...], root: Path) -> list[dict[str, object]]: def _verify_wheels(
packages: tuple[dict[str, object], ...],
root: Path,
*,
registry: str | None = None,
) -> list[dict[str, object]]:
expected = {_normalize(str(item["name"])): item for item in packages} expected = {_normalize(str(item["name"])): item for item in packages}
rows: list[dict[str, object]] = [] rows: list[dict[str, object]] = []
seen: set[str] = set() seen: set[str] = set()
@@ -204,13 +243,21 @@ def _verify_wheels(packages: tuple[dict[str, object], ...], root: Path) -> list[
if package is None or identity["version"] != package["version"] or name in seen: if package is None or identity["version"] != package["version"] or name in seen:
raise PackageArtifactError(f"unexpected wheel artifact: {path.name}") raise PackageArtifactError(f"unexpected wheel artifact: {path.name}")
seen.add(name) seen.add(name)
rows.append(_artifact_row(path, package)) row = _artifact_row(path, package)
if registry:
row["url"] = _python_artifact_url(registry, package=package, filename=path.name)
rows.append(row)
if seen != set(expected): if seen != set(expected):
raise PackageArtifactError("registry did not return every selected Python wheel") raise PackageArtifactError("registry did not return every selected Python wheel")
return sorted(rows, key=lambda item: str(item["name"])) return sorted(rows, key=lambda item: str(item["name"]))
def _verify_webui(packages: tuple[dict[str, object], ...], root: Path) -> list[dict[str, object]]: def _verify_webui(
packages: tuple[dict[str, object], ...],
root: Path,
*,
registry_metadata: dict[str, dict[str, str]] | None = None,
) -> list[dict[str, object]]:
expected = {str(item["name"]): item for item in packages} expected = {str(item["name"]): item for item in packages}
rows: list[dict[str, object]] = [] rows: list[dict[str, object]] = []
seen: set[str] = set() seen: set[str] = set()
@@ -222,7 +269,13 @@ def _verify_webui(packages: tuple[dict[str, object], ...], root: Path) -> list[d
raise PackageArtifactError(f"unexpected WebUI artifact: {path.name}") raise PackageArtifactError(f"unexpected WebUI artifact: {path.name}")
seen.add(name) seen.add(name)
row = _artifact_row(path, package) row = _artifact_row(path, package)
row["integrity"] = "sha512-" + base64.b64encode(hashlib.sha512(path.read_bytes()).digest()).decode("ascii") integrity = "sha512-" + base64.b64encode(hashlib.sha512(path.read_bytes()).digest()).decode("ascii")
row["integrity"] = integrity
metadata = (registry_metadata or {}).get(name)
if metadata:
if metadata.get("integrity") != integrity:
raise PackageArtifactError(f"npm registry integrity does not match downloaded package: {name}")
row["url"] = metadata["url"]
rows.append(row) rows.append(row)
if seen != set(expected): if seen != set(expected):
raise PackageArtifactError("registry did not return every selected WebUI package") raise PackageArtifactError("registry did not return every selected WebUI package")
@@ -283,6 +336,25 @@ def _artifact_row(path: Path, package: dict[str, object]) -> dict[str, object]:
return row return row
def _python_artifact_url(
registry: str,
*,
package: dict[str, object],
filename: str,
) -> str:
parsed = urlsplit(registry.rstrip("/"))
path = parsed.path.rstrip("/")
if not path.endswith("/simple"):
raise PackageArtifactError("Python registry URL must end in /simple to derive immutable artifacts")
artifact_path = (
f"{path.removesuffix('/simple')}/files/"
f"{quote(str(package['name']), safe='')}/"
f"{quote(str(package['version']), safe='')}/"
f"{quote(filename, safe='')}"
)
return urlunsplit((parsed.scheme, parsed.netloc, artifact_path, "", ""))
def _write_requirements(path: Path, rows: list[dict[str, object]]) -> None: def _write_requirements(path: Path, rows: list[dict[str, object]]) -> None:
lines = ["--no-index", "--find-links ./local-wheels", "--require-hashes"] lines = ["--no-index", "--find-links ./local-wheels", "--require-hashes"]
for row in rows: for row in rows:
+6 -4
View File
@@ -11,11 +11,13 @@ LABEL org.opencontainers.image.title="GovOPlaN WebUI runtime" \
USER 0 USER 0
RUN rm -rf /usr/share/nginx/html/* /etc/nginx/conf.d/* RUN rm -rf /usr/share/nginx/html/* /etc/nginx/conf.d/*
COPY web-dist/ /usr/share/nginx/html/ COPY web-dist/ /usr/share/nginx/html/
COPY nginx.conf /etc/nginx/nginx.conf COPY nginx.conf /etc/nginx/nginx.conf.template
COPY web-entrypoint.sh /usr/local/bin/govoplan-web-entrypoint
RUN chown -R 101:101 /usr/share/nginx/html \ RUN chown -R 101:101 /usr/share/nginx/html \
&& chmod -R a-w /usr/share/nginx/html /etc/nginx/nginx.conf && chmod -R a-w /usr/share/nginx/html /etc/nginx/nginx.conf.template \
&& chmod 0555 /usr/local/bin/govoplan-web-entrypoint
USER 101:101 USER 101:101
EXPOSE 8080 EXPOSE 8080
ENTRYPOINT [] ENTRYPOINT ["/usr/local/bin/govoplan-web-entrypoint"]
CMD ["nginx", "-g", "daemon off;"] CMD []
+1 -1
View File
@@ -35,7 +35,7 @@ http {
} }
location /api/ { location /api/ {
proxy_pass http://load-balancer:8000; proxy_pass ${GOVOPLAN_API_UPSTREAM};
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Host $host;
+11
View File
@@ -0,0 +1,11 @@
#!/bin/sh
set -eu
: "${GOVOPLAN_API_UPSTREAM:=http://load-balancer:8000}"
export GOVOPLAN_API_UPSTREAM
envsubst '${GOVOPLAN_API_UPSTREAM}' \
< /etc/nginx/nginx.conf.template \
> /tmp/nginx.conf
exec nginx -c /tmp/nginx.conf -g 'daemon off;'