Compare commits

...
109 Commits
Author SHA1 Message Date
zemion be8ba10ae3 Wire institutional governance modules and checks
Dependency Audit / dependency-audit (push) Failing after 9s
Deployment Installer / deployment-installer (push) Successful in 7s
Security Audit / security-audit (push) Failing after 7s
2026-08-01 20:57:27 +02:00
zemion d78b13f9d3 feat: implement institutional governance and recovery architecture
Dependency Audit / dependency-audit (push) Failing after 10s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 9s
2026-08-01 17:46:53 +02:00
zemion 3c658fa32d Harden worker queue and restore drill defaults
Dependency Audit / dependency-audit (push) Successful in 2m23s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m22s
2026-07-31 22:48:06 +02:00
zemion efd734fad2 build: integrate workflow engine repository
Dependency Audit / dependency-audit (push) Successful in 2m26s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 10m0s
2026-07-31 16:59:08 +02:00
zemion 186aa104ce Align release harness with optional modules [skip ci] 2026-07-31 06:37:56 +02:00
zemion ff9d2404ab Propagate CI safety settings into nested checks
Deployment Installer / deployment-installer (push) Successful in 7s
Dependency Audit / dependency-audit (push) Successful in 2m37s
Security Audit / security-audit (push) Successful in 10m59s
2026-07-31 06:15:56 +02:00
zemion ba82a85547 Repair Gitea Actions bootstrap and module matrix
Dependency Audit / dependency-audit (push) Successful in 2m28s
Deployment Installer / deployment-installer (push) Successful in 7s
Security Audit / security-audit (push) Failing after 10m36s
2026-07-31 05:57:33 +02:00
zemion f1fd143ef5 Complete guided release console workflow
Dependency Audit / dependency-audit (push) Successful in 2m35s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 15s
2026-07-31 05:46:50 +02:00
zemion b4248a849e chore: register tenancy WebUI in release catalog
Dependency Audit / dependency-audit (push) Successful in 2m27s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 14s
2026-07-31 04:21:34 +02:00
zemion ff47659899 Provision Python for security audit workflow
Dependency Audit / dependency-audit (push) Failing after 7s
Security Audit / security-audit (push) Failing after 6s
Deployment Installer / deployment-installer (push) Failing after 4s
2026-07-31 02:53:37 +02:00
zemion 908090dd0f Add scalable deployment planning and guided releases
Security Audit / security-audit (push) Failing after 4s
Dependency Audit / dependency-audit (push) Failing after 7s
Deployment Installer / deployment-installer (push) Failing after 4s
2026-07-31 02:49:03 +02:00
zemion 3864ce28b1 Delete .gitea/workflows/runner_test.yml
Dependency Audit / dependency-audit (push) Failing after 6s
Deployment Installer / deployment-installer (push) Failing after 4s
Security Audit / security-audit (push) Failing after 4s
2026-07-31 02:24:30 +02:00
zemion 857dbe55f7 Add .gitea/workflows/runner_test.yml
Dependency Audit / dependency-audit (push) Failing after 6s
Deployment Installer / deployment-installer (push) Failing after 4s
Security Audit / security-audit (push) Failing after 5s
2026-07-31 02:15:18 +02:00
zemion c9fcdc90c1 feat: add platform interface inventory and encryption registry
Dependency Audit / dependency-audit (push) Failing after 24s
Deployment Installer / deployment-installer (push) Failing after 4s
Security Audit / security-audit (push) Failing after 5s
2026-07-30 17:42:11 +02:00
zemion 82e836b720 feat(deploy): add declarative installation workflow
Deployment Installer / deployment-installer (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
Dependency Audit / dependency-audit (push) Has been cancelled
2026-07-30 15:36:37 +02:00
zemion fcb8296812 test: dispatch queued dataflow composition runs
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-30 02:33:16 +02:00
zemion f2e2eb5517 ci: enforce complete security audit coverage
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-30 01:40:36 +02:00
zemion 1aea3e7c4f fix: repair dev modules and scoped release git trust
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 20:54:58 +02:00
zemion 3f9567af18 Gate Search on PostgreSQL in module CI
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 20:37:08 +02:00
zemion de16f11ce8 test(datasources): include immutable payload schema
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 20:13:49 +02:00
zemion 9d6cdff4b8 CSS changes for release console
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 19:35:49 +02:00
zemion aa4050c0ca feat: expose focused backend reload mode
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 18:52:55 +02:00
zemion d3cdbd8c7a feat: include risk compliance in dev composition
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 18:46:53 +02:00
zemion 7115c4711d chore: register search wiki projects and tickets
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 15:50:35 +02:00
zemion a3beca6fc5 chore: harden audit checks and issue taxonomy
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 14:16:27 +02:00
zemion 11d45bce25 Register governed Views module
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 21:04:55 +02:00
zemion 7b6135b89b Document governed Views architecture
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 19:05:37 +02:00
zemion 5b79e7d377 test: verify pinned dataflow publication composition
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 13:48:11 +02:00
zemion 6afb8fea76 fix: exclude generated artifacts from wiki sync
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 12:52:46 +02:00
zemion 163b35c0af feat: integrate datasource and workflow modules
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 12:45:05 +02:00
zemion 603e07cec5 Integrate Connectors and Dataflow validation
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 11:14:38 +02:00
zemion 97dfd333c6 Register Dataflow module and work taxonomy
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 01:34:06 +02:00
zemion ba88c574b9 chore: move repositories to GovOPlaN organization
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-27 15:46:51 +02:00
zemion 8d292184d4 ci: execute tagged module pytest suites
Security Audit / security-audit (push) Successful in 4m3s
Security Audit Toolbox Update / toolbox-update (push) Successful in 4s
Dependency Audit / dependency-audit (push) Successful in 1m51s
2026-07-23 01:18:47 +02:00
zemion 52bd3527cd ci: install release test harness dependencies
Dependency Audit / dependency-audit (push) Successful in 1m37s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 01:15:07 +02:00
zemion ff49dabf8f ci: retain installed artifact checks in matrix
Dependency Audit / dependency-audit (push) Successful in 1m42s
Security Audit / security-audit (push) Successful in 4m7s
2026-07-23 00:57:24 +02:00
zemion cd15aa514e ci: run artifact-aware module matrix tests
Dependency Audit / dependency-audit (push) Successful in 1m36s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 00:53:16 +02:00
zemion a042baa1d3 fix(release): verify tags in mounted workspaces 2026-07-23 00:53:16 +02:00
zemion e80de00f29 chore(release): align Mail with Campaign contract
Dependency Audit / dependency-audit (push) Successful in 1m37s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 00:47:46 +02:00
zemion c69acf0dee ci: use Gitea-compatible artifact upload
Dependency Audit / dependency-audit (push) Successful in 1m39s
Security Audit / security-audit (push) Successful in 4m11s
2026-07-23 00:36:59 +02:00
zemion 8b93bbc6b6 ci: mount audit workspace safely
Dependency Audit / dependency-audit (push) Successful in 1m43s
Security Audit / security-audit (push) Failing after 4m10s
2026-07-23 00:30:28 +02:00
zemion 3fc17701df ci: fix action repository access
Dependency Audit / dependency-audit (push) Successful in 1m45s
Security Audit / security-audit (push) Failing after 3m33s
2026-07-23 00:10:29 +02:00
zemion 9788bdde0c ci: harden repository bootstrap transport 2026-07-23 00:09:43 +02:00
zemion a10a01c903 docs: surface system action status
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 23:45:42 +02:00
zemion e005e54333 Seal catalog publication Git metadata
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:31:58 +02:00
zemion 76e4baa76e Define the catalog publication identity
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:23:01 +02:00
zemion b6e9a9bd81 Pin the website publication toolchain
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:20:09 +02:00
zemion dc46bda224 Validate complete Core release bundle sources
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:10:35 +02:00
zemion b00d4e55ee Harden isolated release builds in Flatpak
Dependency Audit / dependency-audit (push) Failing after 16s
Security Audit / security-audit (push) Failing after 15s
2026-07-22 21:48:48 +02:00
zemion 76f19dc602 Lock down the local release console 2026-07-22 21:37:33 +02:00
zemion 5381e37a9e Execute receipt-bound durable release runs 2026-07-22 21:37:18 +02:00
zemion 7ecf1f17b0 Harden release source execution 2026-07-22 21:37:14 +02:00
zemion 349a099e5a Harden catalog publication transaction 2026-07-22 21:20:08 +02:00
zemion fdee766993 Authenticate selective catalog base pairs 2026-07-22 20:59:00 +02:00
zemion 47b9c05bde chore(release): align Campaign release dependency 2026-07-22 20:44:43 +02:00
zemion 4e42911477 Document artifact origin and installer receipts 2026-07-22 20:42:35 +02:00
zemion 9c0650b2ed Bind installed releases to signed artifact receipts 2026-07-22 20:42:05 +02:00
zemion 4dc0c8d013 Harden release artifact publication 2026-07-22 20:41:42 +02:00
zemion 35c346a1fa Add private release candidate handles 2026-07-22 20:41:36 +02:00
zemion 4edbc11fe5 feat(release): bound durable run retention 2026-07-22 20:16:19 +02:00
zemion ddee5c00dc fix(assessment): keep release origin unproven
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 14s
2026-07-22 19:55:35 +02:00
zemion 4c16069f88 fix(release): preserve durable recovery capacity 2026-07-22 19:48:55 +02:00
zemion 22f5c2ff4e fix(release): make durable runs recovery safe 2026-07-22 19:30:12 +02:00
zemion 19c4b63ade fix(assessment): harden capability evidence trust 2026-07-22 19:21:06 +02:00
zemion 1dc9148ec3 fix(release): harden durable run recovery 2026-07-22 18:58:54 +02:00
zemion ead697d049 fix(assessment): constrain installed record reads 2026-07-22 18:36:12 +02:00
zemion 65aa8e0b7c docs(assessment): define external proof boundaries 2026-07-22 18:34:06 +02:00
zemion 6c0b003dee feat(assessment): verify installed composition evidence 2026-07-22 18:34:02 +02:00
zemion d0dc916837 feat(release): persist resumable run state 2026-07-22 18:32:19 +02:00
zemion bd715a8473 fix(assessment): reject ambiguous trust keyrings
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 17:17:34 +02:00
zemion ad8dd4f319 fix(assessment): establish independent release trust
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 17:09:35 +02:00
zemion 753dd2a3ee fix(release): fail closed on dashboard parse errors 2026-07-22 17:07:30 +02:00
zemion b6bef410d6 fix(release): fail closed on malformed WebUI metadata 2026-07-22 16:55:06 +02:00
zemion 72f697c341 fix(release): harden review gate diagnostics 2026-07-22 16:52:27 +02:00
zemion 36f662c56f test(release): isolate manifest gate remote 2026-07-22 16:34:39 +02:00
zemion 25fbbaf5ad feat(assessment): gate signed release drift 2026-07-22 16:31:28 +02:00
zemion bcab7095c3 feat(release): guide selective gate remediation 2026-07-22 16:28:13 +02:00
zemion 99534c0251 fix(release): align Campaign package refs
Dependency Audit / dependency-audit (push) Failing after 18s
Security Audit / security-audit (push) Failing after 15s
2026-07-22 15:30:43 +02:00
zemion 71e74601cf test(release): require retirement atomicity proof 2026-07-22 15:30:16 +02:00
zemion c29f1e9977 docs: refresh capability release baseline 2026-07-22 15:21:27 +02:00
zemion 5447299289 Block mismatched selective WebUI releases
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 10:37:19 +02:00
zemion 568ea6059a docs(ui): record completed Campaign pilot evidence
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 14s
2026-07-22 09:40:32 +02:00
zemion 8b6bcb7a4d docs(ui): record delivered Campaign pilot slices 2026-07-22 08:49:57 +02:00
zemion 2fb0a71bfb docs(ui): reconcile Campaign pilot rollout 2026-07-22 08:09:06 +02:00
zemion 8ecf8770ab feat(release): synthesize initial catalog entries
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 12s
2026-07-22 04:56:36 +02:00
zemion d53db2da06 fix(audit): bridge Docker from Flatpak sandboxes 2026-07-22 04:48:10 +02:00
zemion 8906704dc0 feat(release): gate tags on scoped workflows
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 12s
2026-07-22 01:56:18 +02:00
zemion 30f9ed152a feat(release): publish verified source tags
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 12s
2026-07-22 01:17:07 +02:00
zemion eca75cbc93 fix(dev): resolve stale editable packages together 2026-07-22 00:58:16 +02:00
zemion a8abb85676 docs(roadmap): refine postbox and university BI paths
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 12s
2026-07-21 20:48:55 +02:00
zemion 097f9ca2f4 docs(ui): codify universal interaction patterns 2026-07-21 20:48:55 +02:00
zemion 89771d96fb docs: define connector secret retirement
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 16:00:29 +02:00
zemion 492fec46b6 docs: separate installer artifact trust boundary 2026-07-21 15:55:30 +02:00
zemion a863767233 docs: select connected reference journey program
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 15:41:54 +02:00
zemion 99e255cf81 fix(release): separate artifact and source contracts
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 13:24:37 +02:00
zemion d9819c4aad test(release): validate installed module artifacts 2026-07-21 13:24:02 +02:00
zemion 78b601a2e5 fix(release): reject unpackageable tagged artifacts 2026-07-21 12:58:15 +02:00
zemion a18c7ea040 fix(release): validate independent module versions 2026-07-21 12:51:25 +02:00
zemion b193d4555f fix(audit): accept empty OSV source reports 2026-07-21 12:51:19 +02:00
zemion a81dc21e90 chore: clean release tooling imports 2026-07-21 12:39:36 +02:00
zemion 676030b993 Harden security audit result contracts 2026-07-21 12:29:53 +02:00
zemion a15a74c54c feat(release): enforce aligned trusted publications 2026-07-21 12:25:06 +02:00
zemion c34892f6ea feat(release): generate reproducible CycloneDX SBOM 2026-07-21 12:24:49 +02:00
zemion 397c87a012 docs: define system administrator lifecycle story 2026-07-21 12:24:32 +02:00
zemion f1b07354a0 Surface connector deployment allowlists 2026-07-21 12:10:44 +02:00
zemion 17f8036bdc Harden release console token bootstrap
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 03:16:57 +02:00
zemion fcc5885dcf Validate manifest module imports 2026-07-21 03:16:57 +02:00
zemion 0b1506f860 Cover live worktrees in security audits 2026-07-21 03:16:57 +02:00
zemion 295b49bb04 docs: add connected governance platform roadmap
Dependency Audit / dependency-audit (push) Failing after 13s
Security Audit / security-audit (push) Failing after 13s
2026-07-20 20:48:41 +02:00
161 changed files with 49116 additions and 1069 deletions
+29 -2
View File
@@ -2,20 +2,47 @@
# Copy to a deployment-local .env or secret store. Do not commit populated secrets. # Copy to a deployment-local .env or secret store. Do not commit populated secrets.
APP_ENV=production APP_ENV=production
# Live graph changes are useful in development. Production should apply saved
# module state through a coordinated restart of all API and worker processes.
GOVOPLAN_MODULE_LIVE_APPLY_ENABLED=
GOVOPLAN_INSTALL_PROFILE=self-hosted GOVOPLAN_INSTALL_PROFILE=self-hosted
MASTER_KEY_B64=<generate-with-govoplan-config-env-template-generate-secrets> MASTER_KEY_B64=<generate-with-govoplan-config-env-template-generate-secrets>
DATABASE_URL=postgresql+psycopg://govoplan:change-me@127.0.0.1:5432/govoplan DATABASE_URL=postgresql+psycopg://govoplan:change-me@127.0.0.1:5432/govoplan
GOVOPLAN_DATABASE_URL_PGTOOLS=postgresql://govoplan:change-me@127.0.0.1:5432/govoplan GOVOPLAN_DATABASE_URL_PGTOOLS=postgresql://govoplan:change-me@127.0.0.1:5432/govoplan
GOVOPLAN_DB_POOL_SIZE=5
GOVOPLAN_DB_MAX_OVERFLOW=10
GOVOPLAN_DB_POOL_TIMEOUT_SECONDS=30
GOVOPLAN_DB_POOL_RECYCLE_SECONDS=1800
ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,audit,files,mail,campaigns,calendar,docs,ops ENABLED_MODULES=tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,files,mail,campaigns,calendar,poll,scheduling,connectors,datasources,dataflow,workflow_engine,workflow,views,search,risk_compliance,postbox,notifications,services,parties,mandates,decisions,portal,cases,committee,docs,ops
CELERY_ENABLED=true CELERY_ENABLED=true
REDIS_URL=redis://127.0.0.1:6379/0 REDIS_URL=redis://127.0.0.1:6379/0
CELERY_QUEUES=send_email,append_sent,notifications,calendar,default CELERY_QUEUES=send_email,append_sent,notifications,calendar,dataflow,events,default
CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90 CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=false
GOVOPLAN_CONNECTOR_MAX_STRUCTURED_RESPONSE_BYTES=16777216
GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES=536870912
GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST=
GOVOPLAN_CONNECTOR_CA_BUNDLE_ALLOWLIST=
GOVOPLAN_HTTP_MAX_REQUEST_BODY_BYTES=536870912
GOVOPLAN_HTTP_HSTS_SECONDS=31536000
AUTH_LOGIN_THROTTLE_ENABLED=true
AUTH_ACTIVITY_TOUCH_INTERVAL_SECONDS=300
AUTH_LOGIN_THROTTLE_IDENTITY_LIMIT=10
AUTH_LOGIN_THROTTLE_CLIENT_LIMIT=100
AUTH_LOGIN_THROTTLE_WINDOW_SECONDS=900
AUTH_LOGIN_THROTTLE_REDIS_RETRY_SECONDS=30
# Production startup fails without Redis unless this explicit single-process
# risk acknowledgement is enabled.
GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE=false
CORS_ORIGINS=https://govoplan.example.org CORS_ORIGINS=https://govoplan.example.org
GOVOPLAN_TRUSTED_HOSTS=govoplan.example.org
FORWARDED_ALLOW_IPS=127.0.0.1
AUTH_COOKIE_SECURE=true AUTH_COOKIE_SECURE=true
AUTH_COOKIE_SAMESITE=lax AUTH_COOKIE_SAMESITE=lax
AUTH_COOKIE_DOMAIN= AUTH_COOKIE_DOMAIN=
+4 -14
View File
@@ -21,23 +21,13 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with: with:
node-version: "22" node-version: "22"
- name: Configure SSH for release dependencies - name: Use anonymous HTTPS for public GovOPlaN repositories
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
run: | run: |
mkdir -p ~/.ssh git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
chmod 700 ~/.ssh git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend dev audit dependencies - name: Install backend dev audit dependencies
working-directory: govoplan working-directory: govoplan
run: | run: |
+30
View File
@@ -0,0 +1,30 @@
name: Deployment Installer
on:
pull_request:
push:
branches:
- main
workflow_dispatch:
jobs:
deployment-installer:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
path: govoplan
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: "3.12"
- name: Compile deployment tooling
working-directory: govoplan
run: python -m py_compile tools/deployment/govoplan-deploy.py tools/deployment/govoplan_deploy/*.py
- name: Test declarative deployment bundle
working-directory: govoplan
run: python -m unittest -v tests.test_deployment_installer
- name: Build single-file deployer artifact
working-directory: govoplan
run: |
python tools/deployment/build-deployer-zipapp.py --output /tmp/govoplan-deploy.pyz
python /tmp/govoplan-deploy.pyz --help
+32 -14
View File
@@ -7,6 +7,18 @@ on:
jobs: jobs:
module-matrix: module-matrix:
runs-on: ubuntu-latest runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_DB: govoplan_test
POSTGRES_USER: govoplan
POSTGRES_PASSWORD: govoplan_test
options: >-
--health-cmd "pg_isready -U govoplan -d govoplan_test"
--health-interval 5s
--health-timeout 5s
--health-retries 20
steps: steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with: with:
@@ -17,32 +29,38 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with: with:
node-version: "22" node-version: "22"
- name: Configure SSH for release dependencies - name: Use anonymous HTTPS for public GovOPlaN repositories
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
run: | run: |
mkdir -p ~/.ssh git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
chmod 700 ~/.ssh git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend release dependencies - name: Install backend release dependencies
working-directory: govoplan working-directory: govoplan
run: | run: |
python -m venv .venv python -m venv .venv
.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip .venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip
.venv/bin/python -m pip install '../govoplan-core[dev]' .venv/bin/python -m pip install '../govoplan-core[dev]'
.venv/bin/python -m pip install --no-deps ../govoplan-search
- name: Install WebUI release dependencies with test scripts - name: Install WebUI release dependencies with test scripts
working-directory: govoplan working-directory: govoplan
run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
- name: Validate Search against PostgreSQL
working-directory: govoplan
env:
GOVOPLAN_SEARCH_POSTGRES_URL: postgresql+psycopg://govoplan:govoplan_test@postgres:5432/govoplan_test
run: |
GOVOPLAN_CORE_ROOT="$PWD/../govoplan-core" .venv/bin/python tools/checks/postgres-integration-check.py \
--database-url "$GOVOPLAN_SEARCH_POSTGRES_URL" \
--module-set search=tenancy,access,search \
--reset-schema \
--skip-retirement-atomicity
PYTHONPATH="$PWD/../govoplan-search/src:$PWD/../govoplan-core/src" \
.venv/bin/python -m unittest discover \
-s ../govoplan-search/tests \
-p test_postgres_search.py \
-v
- name: Run module matrix and contract tests - name: Run module matrix and contract tests
working-directory: govoplan working-directory: govoplan
run: GOVOPLAN_CORE_ROOT="$PWD/../govoplan-core" PYTHON="$PWD/.venv/bin/python" bash tools/checks/check-module-matrix.sh run: GOVOPLAN_CORE_ROOT="$PWD/../govoplan-core" PYTHON="$PWD/.venv/bin/python" bash tools/checks/check-module-matrix.sh
+5 -15
View File
@@ -16,29 +16,19 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with: with:
node-version: "22" node-version: "22"
- name: Configure SSH for release dependencies - name: Use anonymous HTTPS for public GovOPlaN repositories
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
run: | run: |
mkdir -p ~/.ssh git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
chmod 700 ~/.ssh git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend release integration dependencies - name: Install backend release integration dependencies
working-directory: govoplan working-directory: govoplan
run: | run: |
python -m venv .venv python -m venv .venv
.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip .venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip
.venv/bin/python -m pip install '../govoplan-core[dev]' .venv/bin/python -m pip install -r requirements-release-tests.txt '../govoplan-core[dev]'
- name: Install WebUI release dependencies - name: Install WebUI release dependencies
working-directory: govoplan working-directory: govoplan
run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
+12 -19
View File
@@ -1,7 +1,6 @@
name: Security Audit name: Security Audit
on: on:
pull_request:
push: push:
branches: branches:
- main - main
@@ -13,37 +12,31 @@ jobs:
security-audit: security-audit:
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
SECURITY_AUDIT_MODE: ci SECURITY_AUDIT_MODE: full
SECURITY_AUDIT_SCOPE: govoplan SECURITY_AUDIT_SCOPE: govoplan
SECURITY_AUDIT_FAIL_ON_FINDINGS: "0" SECURITY_AUDIT_FAIL_ON_FINDINGS: "0"
SECURITY_AUDIT_REQUIRE_TOOLS: "1" SECURITY_AUDIT_REQUIRE_TOOLS: "1"
GIT_CONFIG_COUNT: "2"
GIT_CONFIG_KEY_0: url.https://git.add-ideas.de/GovOPlaN/govoplan.insteadOf
GIT_CONFIG_VALUE_0: git@git.add-ideas.de:GovOPlaN/govoplan
GIT_CONFIG_KEY_1: url.https://git.add-ideas.de/GovOPlaN/govoplan.insteadOf
GIT_CONFIG_VALUE_1: ssh://git@git.add-ideas.de/GovOPlaN/govoplan
steps: steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with: with:
path: govoplan path: govoplan
- name: Configure SSH for repository bootstrap - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
env: with:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }} python-version: "3.12"
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh repository bootstrap."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Run security audit - name: Run whole-system security audit
working-directory: govoplan working-directory: govoplan
run: tools/checks/security-audit/run.sh --mode "$SECURITY_AUDIT_MODE" --scope "$SECURITY_AUDIT_SCOPE" --reports-dir audit-reports run: tools/checks/security-audit/run.sh --mode "$SECURITY_AUDIT_MODE" --scope "$SECURITY_AUDIT_SCOPE" --reports-dir audit-reports
- name: Upload audit reports - name: Upload audit reports
if: always() if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with: with:
name: security-audit-reports name: security-audit-reports
path: govoplan/audit-reports path: govoplan/audit-reports
+21
View File
@@ -0,0 +1,21 @@
# GovOPlaN Workspace Codex Guide
## Scope
This repository coordinates the GovOPlaN workspace, release catalog, shared
checks, development environment, and cross-repository automation. Business and
platform behavior remains owned by the corresponding module repository.
## Documentation Contract
- Treat documentation as part of every behavior change. Update the owning module's manifest-driven `DocumentationTopic` contributions for each affected user and administrator workflow, setting, permission, limitation, and operational consequence.
- Keep feature content in the owning module. The optional `govoplan-docs` module projects module contributions and must not import feature internals.
- Every module manifest must provide a static user and administrator baseline, even when `documentation_providers` add configured-state details.
- Run `tools/checks/check-manifest-shapes.py` after module behavior or manifest changes. Run `tools/checks/check-focused.sh` for cross-module changes.
## Working Rules
- Treat Gitea issues as the canonical backlog and state log.
- Preserve optional module boundaries and use Core contracts or capabilities for integrations.
- Prefer targeted checks before full workspace scans.
- Do not start persistent development servers unless requested.
+83 -1
View File
@@ -4,6 +4,12 @@
**Repository type:** system (meta). **Repository type:** system (meta).
<!-- govoplan-repository-type:end --> <!-- govoplan-repository-type:end -->
[![Module Matrix](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/module-matrix.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=module-matrix.yml&actor=0&status=0)
[![Release Integration](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/release-integration.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=release-integration.yml&actor=0&status=0)
[![Deployment Installer](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/deployment-installer.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=deployment-installer.yml&actor=0&status=0)
[![Dependency Audit](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/dependency-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=dependency-audit.yml&actor=0&status=0)
[![Security Audit](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/security-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=security-audit.yml&actor=0&status=0)
This is the GovOPlaN meta repository. It is the operator entry point for This is the GovOPlaN meta repository. It is the operator entry point for
whole-product development, release orchestration, repository bootstrap, and whole-product development, release orchestration, repository bootstrap, and
system-level Docker composition. system-level Docker composition.
@@ -36,6 +42,16 @@ Open the WebUI in a browser after launch only when explicitly requested:
GOVOPLAN_OPEN_BROWSER=1 ./tools/launch/launch-dev.sh GOVOPLAN_OPEN_BROWSER=1 ./tools/launch/launch-dev.sh
``` ```
Limit backend reload triggers during focused module work without changing the
enabled module graph:
```sh
GOVOPLAN_BACKEND_RELOAD_MODULES=calendar,campaign ./tools/launch/launch-dev.sh
```
Set `GOVOPLAN_BACKEND_RELOAD_MODULES=none` to watch only core/config sources.
Leaving it unset keeps the broad default and watches all enabled modules.
Start the shared development PostgreSQL service: Start the shared development PostgreSQL service:
```sh ```sh
@@ -78,6 +94,27 @@ Run the static cross-repository module contract check:
./tools/checks/check-contracts.sh ./tools/checks/check-contracts.sh
``` ```
Require backend, manifest, frontend, lockfile, and release-composition versions
to agree before a release:
```sh
./.venv/bin/python tools/checks/check-version-alignment.py --release-composition
```
Generate the CycloneDX dependency inventory from a resolved release environment:
```sh
./.venv/bin/python tools/release/generate-release-sbom.py --python ./.venv/bin/python
```
For reproducible release artifacts, set `SOURCE_DATE_EPOCH` to the release
commit timestamp (or pass an explicit timezone-qualified `--timestamp`):
```sh
SOURCE_DATE_EPOCH="$(git -C ../govoplan-core show -s --format=%ct HEAD)" \
./.venv/bin/python tools/release/generate-release-sbom.py --python ./.venv/bin/python
```
Run the consolidated focused verification suite: Run the consolidated focused verification suite:
```sh ```sh
@@ -105,6 +142,23 @@ Start the local release console:
./.venv/bin/python tools/release/release-console.py ./.venv/bin/python tools/release/release-console.py
``` ```
Create and validate a private, declarative installation bundle:
```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py init \
--directory ~/.local/share/govoplan/installations/default
./.venv/bin/python tools/deployment/govoplan-deploy.py doctor \
--directory ~/.local/share/govoplan/installations/default
```
The current executable slice and remaining production gates are documented in
[Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md).
Same-host replica balancing and the multi-host promotion boundary are documented
in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
The recovery state machine, migration rollback boundary, and required restore
drills are documented in
[Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md).
## Configuration ## Configuration
The repository root `.env.example` is the self-hosted operator template for a The repository root `.env.example` is the self-hosted operator template for a
@@ -125,8 +179,36 @@ Meta ownership and module install/contract boundaries are documented in
`docs/META_REPO_SCAN.md` and `docs/MODULE_CONTRACTS_AND_INSTALLS.md`. `docs/META_REPO_SCAN.md` and `docs/MODULE_CONTRACTS_AND_INSTALLS.md`.
Frontend layout principles for module pages are documented in Frontend layout principles for module pages are documented in
`docs/FRONTEND_LAYOUT_PRINCIPLES.md`. `docs/FRONTEND_LAYOUT_PRINCIPLES.md`.
The provider-neutral datasource boundary and reusable Dataflow/Workflow graph
contract are documented in
`docs/DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md`.
The cross-product destination, stakeholder visions, configuration archetypes,
connected outcome stories, and capability horizons are documented in
the [Connected Governance Platform Roadmap](docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md).
The reconciled institutional semantics, source-authority modes, module layers,
candidate Mandates/Services/Parties/Decisions boundaries, and migration
sequence are documented in the
[Institutional Governance Target Architecture](docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
The selected Campaign-to-Postbox-to-data-to-collaboration implementation path,
including stage gates and shared documentation expectations, is in the
[Reference Journey Program](docs/REFERENCE_JOURNEY_PROGRAM.md).
The administrator journey from Core-only bootstrap through online module
installation, scale-out, and reversible environment promotion is defined in
[System Administrator Lifecycle User Story](docs/SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
The corresponding host deployment compiler, managed/external component choices,
reconfiguration semantics, and safe Web update boundary are defined in
[Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md).
The concrete replica, worker-node, load-balancer, and shared-state topology is
defined in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
Durable deployment journals, Core recovery evidence, and the distinction
between pre-migration configuration restore and post-migration forward recovery
are defined in
[Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md).
The first Campaign-centric capability and infrastructure fit assessment is in The first Campaign-centric capability and infrastructure fit assessment is in
`docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md`. `docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md`. Its rerun tooling can collect and
verify a bounded installed composition; target, provider and production claims
remain separate, expiring attestations signed by independently scoped proof
authorities.
# GovOPlaN Docker # GovOPlaN Docker
+6 -1
View File
@@ -117,7 +117,12 @@ tools/checks/postgres-integration-check.py \
``` ```
`--reset-schema` drops and recreates the `public` schema before every module `--reset-schema` drops and recreates the `public` schema before every module
set. Use it only against this disposable database. set. Use it only against this disposable database. Before those permutations,
the check runs the Files credential-retirement atomicity proof in random,
test-owned schemas. The proof does not modify `public` and fails the release
gate if its PostgreSQL or full-stack dependencies are unavailable. The
`--skip-retirement-atomicity` option is only for bounded migration/smoke
diagnosis; do not use it for release evidence.
Stop the testbed: Stop the testbed:
+16
View File
@@ -19,8 +19,24 @@ CELERY_ENABLED=true
CELERY_QUEUES=send_email,append_sent,notifications,calendar,default CELERY_QUEUES=send_email,append_sent,notifications,calendar,default
CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90 CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=true
GOVOPLAN_CONNECTOR_MAX_STRUCTURED_RESPONSE_BYTES=16777216
GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES=536870912
GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST=
GOVOPLAN_CONNECTOR_CA_BUNDLE_ALLOWLIST=
GOVOPLAN_HTTP_MAX_REQUEST_BODY_BYTES=536870912
GOVOPLAN_HTTP_HSTS_SECONDS=0
AUTH_LOGIN_THROTTLE_ENABLED=true
AUTH_LOGIN_THROTTLE_IDENTITY_LIMIT=10
AUTH_LOGIN_THROTTLE_CLIENT_LIMIT=100
AUTH_LOGIN_THROTTLE_WINDOW_SECONDS=900
AUTH_LOGIN_THROTTLE_REDIS_RETRY_SECONDS=30
ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops
CORS_ORIGINS=http://127.0.0.1:5173,http://localhost:5173 CORS_ORIGINS=http://127.0.0.1:5173,http://localhost:5173
GOVOPLAN_TRUSTED_HOSTS=127.0.0.1,localhost,testserver
FORWARDED_ALLOW_IPS=127.0.0.1
AUTH_COOKIE_SECURE=false AUTH_COOKIE_SECURE=false
FILE_STORAGE_BACKEND=local FILE_STORAGE_BACKEND=local
FILE_STORAGE_LOCAL_ROOT=runtime/production-like/files FILE_STORAGE_LOCAL_ROOT=runtime/production-like/files
+5
View File
@@ -7,6 +7,11 @@ Current shared profiles:
- `govoplan/dev/postgres` - `govoplan/dev/postgres`
- `govoplan/dev/production-like` - `govoplan/dev/production-like`
The generated whole-product Compose profile is owned by
`tools/deployment/govoplan-deploy.py`. It renders a deployment-specific
`compose.json` from a versioned installation specification; generated files and
secrets remain outside the repository.
Module-specific Docker test beds remain in their owning repositories: Module-specific Docker test beds remain in their owning repositories:
- `govoplan-campaign/dev/mail-testbed` - `govoplan-campaign/dev/mail-testbed`
+360 -65
View File
@@ -4,27 +4,35 @@
| Field | Value | | Field | Value |
| --- | --- | | --- | --- |
| Assessment ID | `campaign-reference-2026-07-20` | | Assessment ID | `campaign-reference-2026-07-22` |
| Schema | `govoplan.fit-assessment/0.1.0` | | Schema | `govoplan.fit-assessment/0.1.0` |
| Assessed on | 2026-07-20 | | Assessed on | 2026-07-22 |
| Product snapshot | Untagged workspace at meta commit `05ae81d64195`; this is not a release artifact | | Product snapshot | Live Ed25519-signed stable catalog sequence `202607220843`: Core `v0.1.13` and Campaign `v0.1.10` |
| Configuration basis | Root `.env.example` and the production-like development profile | | Configuration basis | Root `.env.example` and the production-like development profile |
| Scope | Campaign-centric internal pilot and small-production candidate | | Scope | Campaign-centric internal pilot and small-production candidate |
| Explicitly postponed | Workflow and workflow-driven user stories | | Explicitly postponed | Workflow and workflow-driven user stories |
| Machine-readable companion | [`capability-fit-current.json`](capability-fit-current.json) | | Machine-readable companion | [`capability-fit-current.json`](capability-fit-current.json) |
| Input schema | [`capability-fit.schema.json`](capability-fit.schema.json) | | Input schema | [`capability-fit.schema.json`](capability-fit.schema.json) |
**Snapshot notice:** this assessment remains valid only for the pinned
2026-07-22 composition above. Workflow Engine, the optional Workflow editor,
Datasources, Dataflow, Search, encryption contracts, and other later main-branch
work must not be inferred into this evidence record. The current product
direction and implemented-state reconciliation are documented separately in
the
[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
This is a fit assessment, not a production approval or security certification. This is a fit assessment, not a production approval or security certification.
It deliberately does not infer implementation from a repository, issue, or It deliberately does not infer implementation from a repository, issue, or
manifest existing. A conclusion needs code plus a route/contract, test, manifest existing. A conclusion needs code plus a route/contract, test,
operational drill, configured example, or explicit evidence that the capability operational drill, configured example, or explicit evidence that the capability
is absent. is absent.
The checkout contained unrelated uncommitted work when this assessment was The release baseline below is reproducible at its tagged source and package
made. A dirty repository is recorded below, but local-only changes do not count references. Later main-branch commits do not become release capabilities merely
as an integrated release capability. Tests run against the dirty workspace are because they are committed, pushed, or code-tested. This assessment therefore
useful implementation evidence and are labelled as such; they do not turn the distinguishes code-tested, package-integrated, target-tested, and
workspace into a reproducible release. production-approved evidence explicitly.
## Controlled status vocabulary ## Controlled status vocabulary
@@ -51,12 +59,13 @@ non-production SMTP/IMAP service. The module contract, Campaign authoring and
validation paths, managed attachments, mail-profile boundary, local audit validation paths, managed attachments, mail-profile boundary, local audit
records, and operator status surface all have direct code/test evidence. records, and operator status surface all have direct code/test evidence.
It is not yet possible to call the current workspace a supported small-production The signed catalog resolves the earlier source/package reproducibility gap, but
distribution. The production-like profile intentionally runs only PostgreSQL it does not make the composition production-approved. The production-like
and Redis in containers; API, WebUI, worker, and scheduler processes still run profile intentionally runs only PostgreSQL and Redis in containers; API,
from editable source trees. A target deployment must supply TLS termination, WebUI, worker, and scheduler processes still run from editable source trees. A
process supervision, secret injection, monitoring, backup storage, and recovery target deployment must supply TLS termination, process supervision, secret
procedures. The open [Core backup/restore issue #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29) injection, monitoring, backup storage, and recovery procedures. The open
[Core backup/restore issue #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29)
is a production gate. Identity federation, external audit export, and a tested is a production gate. Identity federation, external audit export, and a tested
disaster-recovery plan are also not complete. disaster-recovery plan are also not complete.
@@ -64,43 +73,48 @@ The current recommendation is therefore:
- **Pilot:** fit with conditions and a bounded proof of concept. - **Pilot:** fit with conditions and a bounded proof of concept.
- **Small production:** partial fit; do not approve until the proof checks and - **Small production:** partial fit; do not approve until the proof checks and
operational gates below pass against a clean, pinned release candidate. operational gates below pass against the signed release and target
environment.
- **Workflow:** planned and postponed; it is not part of either recommended - **Workflow:** planned and postponed; it is not part of either recommended
composition in this assessment. composition in this assessment.
## Pinned composition ## Pinned composition
The configured reference composition comes from the meta repository's The configured reference composition comes from the meta repository's
`ENABLED_MODULES`. Module versions come from the actual manifests. `Dirty` `ENABLED_MODULES`. Versions and refs below are pinned by the live stable catalog;
means that repository had uncommitted changes at assessment time. commits are the commits peeled from those annotated tags.
| Module | Repository commit | Manifest version | Dirty | Role in this assessment | | Module | Tagged source commit | Catalog version/ref | Role in this assessment |
| --- | --- | ---: | :---: | --- | | --- | --- | --- | --- |
| Core runtime | `govoplan-core@e6f7c45f0a95` | server `0.3.0` | yes | API, registry, migration orchestration, shared WebUI, sessions and kernel contracts | | Core runtime | `govoplan-core@d487726f4d2c` | `0.1.13` / `v0.1.13` | API, registry, migration orchestration, shared WebUI, sessions and kernel contracts |
| `tenancy` | `govoplan-tenancy@1dde03854733` | 0.1.8 | no | tenant context and lifecycle | | `tenancy` | `govoplan-tenancy@efbec827616b` | `0.1.8` / `v0.1.8` | tenant context and lifecycle |
| `organizations` | `govoplan-organizations@45cda1a33f18` | 0.1.8 | no | organization model | | `organizations` | `govoplan-organizations@39c081c4fb8f` | `0.1.8` / `v0.1.8` | organization model |
| `identity` | `govoplan-identity@7a1710af896f` | 0.1.8 | yes | normalized internal identity directory | | `identity` | `govoplan-identity@7a1710af896f` | `0.1.8` / `v0.1.8` | normalized internal identity directory |
| `access` | `govoplan-access@ab07075a679b` | 0.1.8 | yes | local authentication, sessions, API keys and RBAC | | `access` | `govoplan-access@f1d64d247e12` | `0.1.11` / `v0.1.11` | local authentication, sessions, API keys and RBAC |
| `admin` | `govoplan-admin@c9e1fb287f50` | 0.1.8 | no | administration surfaces | | `admin` | `govoplan-admin@11ecf362a36d` | `0.1.8` / `v0.1.8` | administration surfaces |
| `dashboard` | `govoplan-dashboard@a315a7c62b1d` | 0.1.8 | no | module-aware home surface | | `dashboard` | `govoplan-dashboard@4b960ad37f0d` | `0.1.8` / `v0.1.8` | module-aware home surface |
| `policy` | `govoplan-policy@2511fbb5a864` | 0.1.8 | no | policy explanation/configuration boundary | | `policy` | `govoplan-policy@1063622d311a` | `0.1.9` / `v0.1.9` | policy explanation/configuration boundary |
| `audit` | `govoplan-audit@5e4f84a789ea` | 0.1.8 | no | database audit records and retrying audit outbox | | `audit` | `govoplan-audit@d3d2c60d7dc1` | `0.1.8` / `v0.1.8` | database audit records and retrying audit outbox |
| `campaigns` | `govoplan-campaign@2e593b7fa412` | 0.1.8 | yes | Campaign authoring, build, delivery control and reporting | | `campaigns` | `govoplan-campaign@735e874bd03c` | `0.1.10` / `v0.1.10` | Campaign authoring, build, delivery control and reporting |
| `files` | `govoplan-files@f3210234d35a` | 0.1.8 | yes | managed files and Campaign attachments | | `files` | `govoplan-files@2b34f6e30578` | `0.1.9` / `v0.1.9` | managed files and Campaign attachments |
| `mail` | `govoplan-mail@b0337b2d261a` | 0.1.8 | yes | SMTP/IMAP profiles and transports | | `mail` | `govoplan-mail@3e2302909022` | `0.1.10` / `v0.1.10` | SMTP/IMAP profiles and transports |
| `calendar` | `govoplan-calendar@371a00aff51c` | 0.1.8 | yes | optional calendar; not needed by the Campaign pilot | | `calendar` | `govoplan-calendar@9bcf41bb1fbb` | `0.1.8` / `v0.1.8` | optional calendar; not needed by the Campaign pilot |
| `docs` | `govoplan-docs@f2ade1d62475` | 0.1.8 | yes | configured-system documentation | | `docs` | `govoplan-docs@be52b716caed` | `0.1.10` / `v0.1.10` | configured-system documentation |
| `ops` | `govoplan-ops@341773a4ff8a` | 0.1.8 | no | readiness and deployment-profile visibility | | `ops` | `govoplan-ops@341773a4ff8a` | `0.1.8` / `v0.1.8` | readiness and deployment-profile visibility |
No tagged release or configuration package is pinned. That is a reproducibility Stable catalog sequence `202607220843` is valid, signed by trusted key
gap, not a harmless omission. The static contract scan found 43 interface `release-key-1`, and pins matching Python and WebUI refs where applicable. This
contracts, 29 providers, and no contract errors in this workspace. This proves proves release source/package selection and provenance, not behavior in a target
the scanned manifest graph is internally consistent; it does not prove the environment. No configuration revision/package is pinned yet, so deployment
behavior of every repository in that graph. configuration remains a separate reproducibility and promotion gate. The static
contract scan found 43 interface contracts, 33 providers, and no contract errors
in the current workspace. That proves the scanned manifest graph is internally
consistent; it does not prove every provider or production journey.
`govoplan-addresses@f19350e65d76` (manifest 0.1.8, dirty) is an optional extension `govoplan-addresses@93dddbb8c52a` (`0.1.9` / `v0.1.9`) is an optional catalogued
for reusable recipient sources. It is not enabled in the pinned root profile and extension for reusable recipient sources. It is not enabled in the pinned root
must be added explicitly when the pilot needs address lists or CardDAV. profile and must be added explicitly when the pilot needs address lists or
CardDAV.
## Recommended scenarios ## Recommended scenarios
@@ -128,36 +142,36 @@ persistence, and durable shared or object storage. Run one scheduler only when a
selected module needs scheduled recovery. Multiple scheduler replicas require selected module needs scheduled recovery. Multiple scheduler replicas require
leader election or an external lock, which is not established by this report. leader election or an external lock, which is not established by this report.
This topology is a recommendation from the [Ops scalability profiles](https://git.add-ideas.de/add-ideas/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md), This topology is a recommendation from the [Ops scalability profiles](https://git.add-ideas.de/GovOPlaN/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md),
not a currently shipped production Compose/Kubernetes/systemd package. not a currently shipped production Compose/Kubernetes/systemd package.
## Functional capability matrix ## Functional capability matrix
| Capability | Status | Evidence and scope | Conditions, gaps, or exclusions | | Capability | Status | Evidence and scope | Conditions, gaps, or exclusions |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Module-aware API/WebUI composition | `verified` | Core registry tests cover installed manifests, route contribution, missing modules, and module permutations; the static contract scan passed. | The pinned workspace is dirty and untagged; repeat on a clean release candidate. | | Module-aware API/WebUI composition | `verified` | Core registry tests cover installed manifests, route contribution, missing modules, and module permutations; the static contract scan passed; stable catalog sequence `202607220843` has a valid trusted signature and matching package refs. | Package integration is verified; repeat the module-permutation checks on the installed target composition. |
| Local tenants, accounts, sessions, API keys and RBAC | `verified` | Access/tenancy manifests, auth dependency tests, cookie/CSRF API smoke tests, and role/permission contracts. | External identity providers are not included in this conclusion. | | Local tenants, accounts, sessions, API keys and RBAC | `verified` | Access/tenancy manifests, auth dependency tests, cookie/CSRF API smoke tests, and role/permission contracts. | External identity providers are not included in this conclusion. |
| Campaign draft authoring, validation, recipient import, build, execution snapshot and mock send | `verified` | Core API smoke tests exercise create/validate/build/mock-send, frozen delivery configuration, policy gates, job deltas and reconciliation; Campaign-focused tests passed in this workspace. | Current Campaign UI/code also contains substantial unintegrated WIP; usability and release integration need separate acceptance. | | Campaign draft authoring, validation, recipient import, build, execution snapshot and mock send | `verified` | Core API smoke tests exercise create/validate/build/mock-send, frozen delivery configuration, policy gates, job deltas and reconciliation; Campaign-focused tests passed; Campaign `v0.1.10` and Core `v0.1.13` are package-integrated in the signed stable catalog. | Code and package integration are verified; usability and target-provider acceptance remain separate. |
| Managed Campaign attachments and local file storage | `verified` | Files capability/route tests, Campaign attachment-build tests, and local-storage backend code. | The deployment must provide a durable path and include files in database/storage backup plans. | | Managed Campaign attachments and local file storage | `verified` | Files capability/route tests, Campaign attachment-build tests, and local-storage backend code. | The deployment must provide a durable path and include files in database/storage backup plans. |
| SMTP send, IMAP append and mailbox/profile policy | `available_unconfigured` | Mail adapters, encrypted profile model, helper tests, Campaign mail-policy smoke tests, and a GreenMail-oriented runbook/testbed. | No target mail server, TLS chain, throttling, sender policy, bounce/reply process, or real delivery receipt was exercised here. | | SMTP send, IMAP append and mailbox/profile policy | `available_unconfigured` | Mail adapters, encrypted profile model, helper tests, Campaign mail-policy smoke tests, and a GreenMail-oriented runbook/testbed. | No target mail server, TLS chain, throttling, sender policy, bounce/reply process, or real delivery receipt was exercised here. |
| Delivery retry, uncertainty and reconciliation | `partial` | Campaign API smoke tests include worker-loss/outcome-unknown reconciliation and frozen job evidence. | A target-provider failure drill and operational alerting are still required; local Campaign audit-proof work is not release-integrated. | | Delivery retry, uncertainty and reconciliation | `partial` | Campaign API smoke tests include worker-loss/outcome-unknown reconciliation and frozen job evidence; those paths are present in catalogued Campaign `v0.1.10`. | A target-provider failure drill and operational alerting are still required; package integration does not prove provider behavior. |
| Reusable address lists and Campaign recipient-source integration | `available_unconfigured` | Address list/recipient capability and Campaign optional lookup tests passed; manifests expose optional interfaces. | `addresses` is not in the pinned profile. Configure permissions and data governance before use. | | Reusable address lists and Campaign recipient-source integration | `available_unconfigured` | Address list/recipient capability and Campaign optional lookup tests passed; manifests expose optional interfaces. | `addresses` is not in the pinned profile. Configure permissions and data governance before use. |
| CardDAV address synchronization | `available_unconfigured` | Discovery, preview/conflict, two-way create/update/delete and disconnect tests passed in the current Addresses workspace. | Requires target-server interoperability, credentials, CA/TLS, rate-limit, conflict and restore drills. | | CardDAV address synchronization | `available_unconfigured` | Discovery, preview/conflict, two-way create/update/delete and disconnect tests passed in the current Addresses workspace. | Requires target-server interoperability, credentials, CA/TLS, rate-limit, conflict and restore drills. |
| File connectors (WebDAV/Nextcloud/Seafile/SMB/S3 import) | `partial` | Provider descriptors and browse/import helper tests exist; S3 connector tests passed. | Coverage varies by provider and optional dependency. No target content system was exercised; connector egress and provenance policy need review. | | File connectors (WebDAV/Nextcloud/Seafile/SMB/S3 import) | `partial` | Provider descriptors and browse/import helper tests exist; S3 connector tests passed. | Coverage varies by provider and optional dependency. No target content system was exercised; connector egress and provenance policy need review. |
| Local audit log and governed-event retry outbox | `verified` | Audit module contract plus enqueue/dispatch/failure-for-retry tests passed. | Central audit/SIEM export, retention enforcement and operational monitoring are not verified. | | Local audit log and governed-event retry outbox | `verified` | Audit module contract plus enqueue/dispatch/failure-for-retry tests passed. | Central audit/SIEM export, retention enforcement and operational monitoring are not verified. |
| Configured-system documentation and Ops status pages | `verified` | Docs/Ops manifests contribute protected routes and WebUI packages; Ops code checks database, Redis, workers, storage and deployment-security settings. | This does not replace external monitoring or a target runbook. | | Configured-system documentation and Ops status pages | `verified` | Docs/Ops manifests contribute protected routes and WebUI packages; Ops code checks database, Redis, workers, storage and deployment-security settings. | This does not replace external monitoring or a target runbook. |
| Calendar/CalDAV integration | `partial` | Calendar storage/sync tests exist; durable external-write outbox, worker recovery and retention are local WIP in this snapshot. | Not required for the Campaign pilot and must not be treated as release-integrated yet. Bulk synchronized-calendar migration semantics remain separate work. | | Calendar/CalDAV integration | `partial` | Calendar `v0.1.8` supplies the catalogued storage/sync foundation. The durable external-write outbox, worker recovery, reconciliation, and retention work is committed, tested, and pushed on Calendar `main` after that tag. | The post-tag outbox work is remote-integrated source, not local-only WIP, but it is not in the signed stable package baseline and has not passed a target CalDAV drill. Bulk synchronized-calendar migration semantics remain separate work. |
| External LDAP/AD, OIDC/SAML or SCIM identity integration | `scaffold` | IDM owns normalized assignment APIs and documents connector boundaries. | Provider connectors, login callback flow and target directory reconciliation are not an implemented end-to-end capability. Use local accounts for this pilot. | | External LDAP/AD, OIDC/SAML or SCIM identity integration | `scaffold` | IDM owns normalized assignment APIs and documents connector boundaries. | Provider connectors, login callback flow and target directory reconciliation are not an implemented end-to-end capability. Use local accounts for this pilot. |
| Export-control/embargo-list screening | `planned` | Product-level [GovOPlaN #12](https://git.add-ideas.de/add-ideas/govoplan/issues/12) defines the consumer-independent user story. | No screening provider, list provenance, matching policy, review flow or legal evidence exists in this composition. | | Export-control/embargo-list screening | `planned` | Product-level [GovOPlaN #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12) defines the consumer-independent user story. | No screening provider, list provenance, matching policy, review flow or legal evidence exists in this composition. |
| Workflow-driven journeys and views | `planned` | Workflow contracts/concepts exist outside this assessment. | Explicitly postponed. Do not include Workflow in pilot or production claims from this report. | | Workflow-driven journeys and views | `planned` | Workflow contracts/concepts exist outside this assessment. | Explicitly postponed. Do not include Workflow in pilot or production claims from this report. |
## Infrastructure matrix ## Infrastructure matrix
| Component | Status | Current evidence | Target requirement / gap | | Component | Status | Current evidence | Target requirement / gap |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| WebUI and API | `verified` | FastAPI app, module routes, shared WebUI host, health endpoint, dev smoke and module-permutation tests. | Build immutable, matching frontend/backend artifacts and supervise them; no production image/service bundle is supplied here. | | WebUI and API | `verified` | FastAPI app, module routes, shared WebUI host, health endpoint, dev smoke and module-permutation tests; the catalog pins matching Core/Campaign backend and WebUI refs. | Materialize and supervise immutable artifacts in the target; no production image/service bundle is supplied here. |
| Background workers | `available_unconfigured` | Celery app, Campaign queues, Redis configuration and production-like launcher. | Run target broker/worker heartbeat and failure drills; split queues when provider limits or load justify it. | | Background workers | `available_unconfigured` | Celery app, Campaign queues, Redis configuration and production-like launcher. | Run target broker/worker heartbeat and failure drills; split queues when provider limits or load justify it. |
| Scheduler | `partial` | A separately launched Celery beat process and Calendar recovery schedule exist in local WIP. | Keep single-instance until leader election/locking is proved; add process supervision and missed-schedule alerts. | | Scheduler | `partial` | A separately launched Celery beat process exists; Calendar recovery scheduling is committed and pushed after the catalogued `v0.1.8` tag. | The Calendar recovery slice is not stable-package-integrated. Keep single-instance until leader election/locking is proved; add process supervision and missed-schedule alerts. |
| PostgreSQL | `verified` | Production target in settings/operator docs, migration tracks and a disposable integration-check harness. | Target HA, patching, connection limits, WAL/archive policy and restore time are external deployment decisions. | | PostgreSQL | `verified` | Production target in settings/operator docs, migration tracks and a disposable integration-check harness. | Target HA, patching, connection limits, WAL/archive policy and restore time are external deployment decisions. |
| Redis/queues | `available_unconfigured` | Redis-backed Celery configuration, health ping and append-only production-like container. | Target availability, persistence, eviction, authentication/TLS and queue-loss behavior were not assessed. | | Redis/queues | `available_unconfigured` | Redis-backed Celery configuration, health ping and append-only production-like container. | Target availability, persistence, eviction, authentication/TLS and queue-loss behavior were not assessed. |
| Local file storage | `verified` | Local backend, configured root, fallback-root tests and Ops writability check. | Suitable only for a durable single-node/shared path with backup; it blocks independent API scaling when node-local. | | Local file storage | `verified` | Local backend, configured root, fallback-root tests and Ops writability check. | Suitable only for a durable single-node/shared path with backup; it blocks independent API scaling when node-local. |
@@ -171,13 +185,13 @@ not a currently shipped production Compose/Kubernetes/systemd package.
| Health/readiness | `verified` | `/health`, protected `/health/details`, and Ops checks for DB, Redis, workers, storage, maintenance and cookie/CORS posture. | Add external probes and distinguish liveness from dependency readiness for the chosen orchestrator. | | Health/readiness | `verified` | `/health`, protected `/health/details`, and Ops checks for DB, Redis, workers, storage, maintenance and cookie/CORS posture. | Add external probes and distinguish liveness from dependency readiness for the chosen orchestrator. |
| Metrics, logs and alerting | `partial` | Correlation IDs, slow-request/query metrics in logs, worker inspection and operator status are implemented. | No bundled metrics exporter, log collector, dashboards, queue-depth alerts, pager route or SLO is verified. | | Metrics, logs and alerting | `partial` | Correlation IDs, slow-request/query metrics in logs, worker inspection and operator status are implemented. | No bundled metrics exporter, log collector, dashboards, queue-depth alerts, pager route or SLO is verified. |
| Audit | `partial` | Local audit tables and retry outbox are verified. | Retention, tamper-evident export, privileged access review and SIEM integration are unproved. | | Audit | `partial` | Local audit tables and retry outbox are verified. | Retention, tamper-evident export, privileged access review and SIEM integration are unproved. |
| Backup and restore | `partial` | Operator guide and installer hooks describe `pg_dump`/`pg_restore`; SQLite and simulated installer rollback drills exist. | [Core #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29) remains open. No target PostgreSQL + files + secrets restore drill or measured RTO/RPO exists. | | Backup and restore | `partial` | Operator guide and installer hooks describe `pg_dump`/`pg_restore`; SQLite and simulated installer rollback drills exist. | [Core #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29) remains open. No target PostgreSQL + files + secrets restore drill or measured RTO/RPO exists. |
| Disaster recovery | `not_assessed` | The Ops guide asks for RPO/RTO and restore drills. | No agreed RPO/RTO, off-site copy, failover topology, dependency recovery order, communications plan or exercise evidence was supplied. | | Disaster recovery | `not_assessed` | The Ops guide asks for RPO/RTO and restore drills. | No agreed RPO/RTO, off-site copy, failover topology, dependency recovery order, communications plan or exercise evidence was supplied. |
The scalability and sizing documentation delivered the documentation portions The scalability and sizing documentation delivered the documentation portions
of [Core #217](https://git.add-ideas.de/add-ideas/govoplan-core/issues/217) and of [Core #217](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/217) and
[Core #219](https://git.add-ideas.de/add-ideas/govoplan-core/issues/219). [Core #219](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/219).
[Core #28](https://git.add-ideas.de/add-ideas/govoplan-core/issues/28) records the [Core #28](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/28) records the
operator-documentation slice. These closed tickets are evidence of documented operator-documentation slice. These closed tickets are evidence of documented
models, not evidence that an organization's production environment has passed models, not evidence that an organization's production environment has passed
them. them.
@@ -209,7 +223,8 @@ Facts:
- The production-like profile is a development validation composition, not a - The production-like profile is a development validation composition, not a
production deployment artifact. production deployment artifact.
- The assessed workspace is untagged and several key repositories are dirty. - The assessed package baseline is the trusted, signed stable catalog sequence
`202607220843`; it pins Core `v0.1.13` and Campaign `v0.1.10`.
- Workflow is postponed. - Workflow is postponed.
- The default composition does not enable Addresses. - The default composition does not enable Addresses.
- Health/readiness checks exist; an external monitoring stack does not. - Health/readiness checks exist; an external monitoring stack does not.
@@ -225,22 +240,25 @@ Assumptions that require confirmation:
Highest risks: Highest risks:
1. A dirty, untagged workspace cannot be reproduced or promoted safely. 1. A signed, reproducible package selection can still be installed or configured
incorrectly and has not been accepted in the target environment.
2. Real SMTP/IMAP behavior, throttling and ambiguous outcomes have not been 2. Real SMTP/IMAP behavior, throttling and ambiguous outcomes have not been
proven against the target provider. proven against the target provider.
3. Backup/restore and disaster recovery are not demonstrated across database, 3. Backup/restore and disaster recovery are not demonstrated across database,
files, keys and deployment configuration. files, keys and deployment configuration.
4. External identity, monitoring, secret-store and reverse-proxy controls are 4. External identity, monitoring, secret-store and reverse-proxy controls are
deployment gaps rather than GovOPlaN-delivered components. deployment gaps rather than GovOPlaN-delivered components.
5. Local Campaign and Calendar WIP can look complete in the checkout while 5. Post-tag Calendar work is committed and pushed but remains outside the signed
remaining absent from the remote release baseline. stable package baseline; branch integration must not be mistaken for a
release.
6. Data classification, retention, recipient consent/legal basis and external 6. Data classification, retention, recipient consent/legal basis and external
disclosure rules are not assessed for a concrete organization. disclosure rules are not assessed for a concrete organization.
## Proof checks before promotion ## Proof checks before promotion
1. Create a clean release candidate that pins every module/package and WebUI 1. Materialize the signed catalog into an isolated installation and rerun the
artifact; rerun the contract, module-permutation and focused test gates. contract, migration, module-permutation, and focused acceptance gates against
the installed artifacts.
2. Run a target-like Campaign from import through validation, attachment build, 2. Run a target-like Campaign from import through validation, attachment build,
queue, SMTP acceptance, IMAP append, reporting and audit using safe data. queue, SMTP acceptance, IMAP append, reporting and audit using safe data.
3. Drill transient SMTP failure, accepted-but-local-commit-lost uncertainty, 3. Drill transient SMTP failure, accepted-but-local-commit-lost uncertainty,
@@ -320,20 +338,284 @@ assumptions, gaps, risks, recommendation and proof check. Reassessment must pin
new composition and review any row whose code, evidence, configuration or target new composition and review any row whose code, evidence, configuration or target
requirement changed. requirement changed.
### Repeatable release rerun
The release-aware rerun tool validates the machine-readable assessment against
its JSON Schema, verifies the catalog's Ed25519 signature against an
independently pinned local trust keyring, and separately verifies the canonical
hash of the published or candidate keyring pinned by the signed catalog. The
downloaded keyring is therefore never its own sole trust root. The tool compares
catalog sequence and module versions and—when local checkouts are available—checks
annotated release tags against assessed commits. For repositories represented in
signed `release.selected_units`, both the peeled commit and annotated tag-object
ID must match exactly; re-annotating an unchanged commit requires review.
Release drift produces a machine-readable list of affected capability and
infrastructure conclusions instead of silently carrying their prior status
forward:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--public \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json
```
Use `--catalog CATALOG --keyring PUBLISHED_OR_CANDIDATE_KEYRING
--trusted-keyring PINNED_LOCAL_KEYRING` for an offline or release-candidate
rerun. `GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE` may provide the
independently managed local trust-root path instead. During key rotation this
local file may contain both current and `next` keys within their validity
windows. Only `active` and `next` entries are trusted; unknown statuses,
duplicate key IDs and malformed structured keyrings fail closed, while revoked,
disabled and retired entries are ignored. Do not establish this trust file by
downloading it in the same rerun. Public JSON responses are bounded to 16 MiB.
Use `--json` or `--output PATH` for automation. Evidence and report files are
written through a bounded same-directory atomic replacement, with mode `0600`
and file and directory `fsync`. All parent directories must already exist, no
parent component may be a symlink, and an existing symlink or non-regular output
target is rejected.
Exit status `0` means the assessed release metadata is current, `2` means
explicit review is required, and `1` means the schema or trust checks are
blocked. The machine report distinguishes independent signature trust,
published-keyring hash agreement, and local-tag proof. Local-tag proof is only
marked checked when every composition entry is available for comparison and was
attempted. A successful rerun proves only the assessment schema, signed catalog
metadata, published-keyring integrity and optional local tag provenance; it does
not prove installed artifacts, target providers, target infrastructure, or
production fitness. Those remain separate proof checks above.
### Installed-composition evidence
The same rerun can inspect the Python environment in which it executes and bind
that observation to the assessment and signed catalog:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--public \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--collect-installed-evidence /var/tmp/govoplan-installed-evidence.json \
--output /var/tmp/govoplan-fit-review.json
```
The collector follows the strict version `0.4.0`
[`installed-composition-evidence.schema.json`](installed-composition-evidence.schema.json)
contract. It enumerates all installed distributions whose normalized name starts
with `govoplan-`, compares the enabled assessed package and module-manifest
versions, and identifies missing, duplicate and extra GovOPlaN distributions.
Those differences produce stable review targets for the affected composition
entries and evidence-backed conclusions.
For each distribution, the collector also:
- reads the installed wheel `RECORD` file directly, with a 4 MiB metadata bound,
rather than relying on an interpreted distribution file list; missing files,
duplicate or malformed declarations, mismatched declared sizes and hashes all
prevent a `verified` result;
- verifies every supported SHA-256 wheel-payload declaration in that local
metadata, within fixed limits of 256 GovOPlaN distributions, 10,000 files and
512 MiB hashed per distribution, 50,000 files and 2 GiB hashed for one
collection, and 64 MiB for any single file;
- accepts package-owned data-file targets below the installation prefix and
declared `console_scripts`, while rejecting other traversal, undeclared
scripts, paths outside the prefix and leaf symlinks before opening a file;
- reports pip-generated, unhashed `__pycache__/*.pyc` entries separately only
when their derived source is a hashed payload entry, including generated
files below a confined package-owned runtime-data prefix;
- labels every PEP 610 observation with the explicit
`local-pep610-metadata` basis and distinguishes coherent declared Git commits
and archive hashes from editable installs, local directories,
package-index/unknown origins and malformed metadata;
- compares a non-editable VCS commit with the assessment commit and, when one is
present, the signed catalog `selected_units` commit;
- derives two content identities from bytes it actually verifies: an
install-stable identity for immutable wheel-declared files and a full
installed-RECORD identity, while retaining only SHA-256 digests and counts;
- records only package/module identifiers, versions, bounded counters, hashes
and stable error codes. It never writes direct URLs, paths, hostnames,
usernames, exception text or file contents.
`RECORD` `verified` means complete matching coverage of the wheel payload
declared by the local metadata, not every runtime byte and not a release-origin
binding. The proof scope exposes the aggregate count of generated unhashed
bytecode; runtime activation remains unchecked. `RECORD` agreement proves that
payload files agree with installed metadata, but does not make self-consistent
metadata a trusted release origin. Similarly, a version string is not artifact
integrity. PEP 610 VCS metadata is accepted only for
`vcs: git`, a full 4064 hexadecimal commit, one mutually exclusive provenance
form and a coherent bounded URL shape. Editable and directory-backed installs
remain mutable even when their small editable-install `RECORD` is valid. Archive
or index artifacts without a hash anchored by the assessed release remain
unanchored. A matching declaration is reported under
`installed_source_provenance` as local consistency only, with
`release_origin_bound: false` until the separate origin proof succeeds. A signed
catalog can contain `release.artifacts` identities computed directly from built,
bounded wheel files. Wheels with installer-transformed scripts or headers are
marked `requires_installer_receipt`; catalog metadata alone cannot attest those
post-install bytes. A role-scoped installer receipt instead binds the exact
installed-evidence digest, full installed payload identities, catalog archive
digests and canonical signed-catalog digest. The tool never accepts an installed
hash merely because the installed evidence asserted it.
Only evidence produced in-process by `--collect-installed-evidence` is a local
observation, and it must be no more than five minutes old (with at most 30
seconds of future clock skew) at the selected verification time. Use
`--installed-evidence PATH` to compare evidence collected in a separate
environment. Imported JSON without a valid independent installer receipt
carries a blocker: it can identify differences but cannot establish checked or
valid installed proof. A receipt signed by a separately provisioned installer
authority authenticates that exact imported evidence across the process
boundary. The proof scope records observation mode, collection/evaluation time,
receipt authentication, freshness, and whether evaluation used current time or
an explicit historical override. Evidence input and output are bounded to 4
MiB. Collection loads the `govoplan.modules` entry-point factories in order to
read module IDs and manifest versions; that executes installed GovOPlaN manifest
code. Run it only inside the installation being assessed and with the same
isolation expected for other installed-artifact acceptance checks. This
collector does not observe
which modules a running service activated, migrations, configuration, health,
or reference-journey behavior. Runtime activation therefore remains an explicit
unchecked boundary.
### Reference-readiness, provider and production proof boundary
Installed evidence cannot establish target acceptance, accessibility, privacy,
security, operations, recovery, an external provider, or production use. These
scopes use a separate, expiring
[`capability-fit-boundary-evidence.schema.json`](capability-fit-boundary-evidence.schema.json)
bundle. The bundle is bound to the assessment ID, assessment release and exact
installed-evidence SHA-256 digest. It contains only opaque subject/control/result
IDs and content hashes, not endpoints, credentials, people or raw result files.
Boundary evidence is accepted only when at least one Ed25519 signature validates
against a separately provisioned
[`capability-fit-proof-authority-keyring.schema.json`](capability-fit-proof-authority-keyring.schema.json).
Each authority key explicitly lists the scopes it may attest. Target,
accessibility, privacy, security, operations, recovery, and provider claims use
`passed` or `failed`; production claims use `approved` or `rejected`.
One claim per scope, unique control/artifact IDs, `issued_at < expires_at`, current
validity and exact digest binding are mandatory. Any schema, binding, time,
signature or authority blocker leaves every supplied boundary claim unchecked;
one malformed authority member or invalid validity interval invalidates the
supplied authority set as a whole. An authorized negative result is checked but
requires review only after every prerequisite, including installed
release-origin binding, is established.
Signatures cover UTF-8 JSON with the `signatures` member omitted, object keys
sorted, compact `,`/`:` separators and non-ASCII characters escaped, matching
the tool's deterministic canonicalization.
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--public \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--installed-evidence /srv/govoplan/evidence/installed.json \
--boundary-evidence /srv/govoplan/evidence/target-proof.json \
--boundary-authority-keyring /srv/govoplan/trust/proof-authorities.json \
--expected-external-provider-subject provider-production
```
With `--installed-evidence`, this command performs comparison and proof-binding
diagnostics. Without an installer receipt, the imported document remains
unsigned, so neither it nor the boundary claim becomes accepted proof. Direct
in-process collection can match catalog-anchored artifact identities for wheels
without installer-transformed files. The installer-receipt flow below
authenticates a durable cross-process observation and is required for transformed
script/header payloads.
Issue a receipt only in the installation process performing the live collection.
The command refuses evidence supplied from a file and first validates the
assessment, independently trusted signed catalog, composition and RECORD bytes:
```bash
./.venv/bin/python tools/assessments/installer-receipt.py \
--assessment /srv/govoplan/assessment.json \
--catalog /srv/govoplan/catalogs/stable.json \
--keyring /srv/govoplan/catalogs/keyring.json \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--receipt-id install:production:20260722 \
--signing-key installer-production=/run/secrets/installer-ed25519.pem \
--python-artifact govoplan-core=/srv/govoplan/staged/govoplan_core-0.1.13-py3-none-any.whl \
--python-artifact govoplan-campaign=/srv/govoplan/staged/govoplan_campaign-0.1.10-py3-none-any.whl \
--evidence-output /srv/govoplan/evidence/installed.json \
--receipt-output /srv/govoplan/evidence/installer-receipt.json
```
Repeat `--python-artifact PACKAGE=/exact/consumed.whl` for every enabled
GovOPlaN distribution. The issuer reopens each exact wheel, verifies its archive
and payload identities against the signed catalog, and refuses a different
build with the same name and version. Receipt issuance must follow collection
within five minutes, and live verification must still fall inside that bounded
window; retain the pair for a reproducible historical review at its explicit
verification time.
Verify that durable pair with its separately managed trust root:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--catalog /srv/govoplan/catalogs/stable.json \
--keyring /srv/govoplan/catalogs/keyring.json \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--installed-evidence /srv/govoplan/evidence/installed.json \
--installer-receipt /srv/govoplan/evidence/installer-receipt.json \
--installer-authority-keyring /srv/govoplan/trust/installer-authorities.json
```
Target-environment and production-approval claims must use the assessment's
`deployment_profile.id` as `subject_id`. External-provider claims require the
operator to supply a bounded opaque expected subject with
`--expected-external-provider-subject`; without it, such a claim remains
unchecked and blocks. Expected and observed IDs are retained in proof scope.
Accessibility, privacy, security, operations, and recovery claims use the same
deployment subject. The report emits a cumulative `reference_readiness` verdict
only when all six required scopes are checked and positive. This verdict remains
separate from production approval and from provider-specific acceptance.
Both authority keyrings are governance trust roots. Installer receipt keys use
the strict
[`installer-receipt-authority-keyring.schema.json`](installer-receipt-authority-keyring.schema.json)
contract and may attest only `installed_release_origin`; their public material
must not be reused by catalog or boundary-proof authorities. Do not download or
generate them from the proof bundle being checked. The checker rejects
proof-authority public keys reused by either the published or independently
trusted catalog keyring, and rejects the same proof public-key material assigned
to multiple authority IDs. A malformed key, an invalid or empty validity
interval, or ambiguous key
identity invalidates the supplied authority set. Authority `not_after` is
exclusive. A target operator or approver must validate the referenced
drill/result artifacts before signing. The rerun verifies the
attestation and its bindings; it does not fetch or reinterpret those artifacts.
`--verification-time` exists only for reproducible historical review. Supplying
it always marks the machine and human report as a **HISTORICAL override**; callers
cannot relabel it as current. Live admission must omit it and use the actual
current time.
No boundary bundle or production authority has been supplied for this current
assessment. Reference-readiness, provider, and production proof therefore
remain explicitly unchecked rather than inferred from the local GreenMail
journey, source tests, or signed release metadata.
## Evidence used in this slice ## Evidence used in this slice
- [Production-like profile](../dev/production-like/README.md) and - [Production-like profile](../dev/production-like/README.md) and
[Compose dependencies](../dev/production-like/docker-compose.yml) [Compose dependencies](../dev/production-like/docker-compose.yml)
- [Module contracts and install boundaries](MODULE_CONTRACTS_AND_INSTALLS.md) - [Module contracts and install boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Core deployment operator guide](https://git.add-ideas.de/add-ideas/govoplan-core/src/branch/main/docs/DEPLOYMENT_OPERATOR_GUIDE.md) - [Core deployment operator guide](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/branch/main/docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [Ops scalability profiles](https://git.add-ideas.de/add-ideas/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md) - [Ops scalability profiles](https://git.add-ideas.de/GovOPlaN/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md)
- Actual module manifests in the pinned repositories and the static contract - Actual module manifests in the pinned repositories and the static contract
checker in this meta repository checker in this meta repository
- Core module-system/API smoke/auth/install-config tests, plus focused Campaign, - Core module-system/API smoke/auth/install-config tests, plus focused Campaign,
Files, Mail, Audit and Addresses tests Files, Mail, Audit and Addresses tests
- [Campaign delivery runbook](https://git.add-ideas.de/add-ideas/govoplan-campaign/src/branch/main/docs/CAMPAIGN_DELIVERY_RUNBOOK.md) - [Campaign delivery runbook](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/src/branch/main/docs/CAMPAIGN_DELIVERY_RUNBOOK.md)
- [Live signed stable catalog](https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json)
and [published keyring](https://govoplan.add-ideas.de/catalogs/v1/keyring.json),
verified against a separately provisioned local trust keyring
- Annotated source tags `govoplan-core/v0.1.13` and
`govoplan-campaign/v0.1.10`, including their catalogued Python and WebUI refs
- Installed-composition, boundary-proof and independently scoped proof-authority
schemas plus their deterministic review tests; no current target or production
proof bundle is asserted
Checks run for this first slice against the current workspace: Checks retained from the first assessment slice against its then-current
workspace:
- static manifest/interface contract scan: 43 contracts, 29 providers, no issues - static manifest/interface contract scan: 43 contracts, 29 providers, no issues
- selected Core module-composition, auth/CSRF, health, Campaign journey, - selected Core module-composition, auth/CSRF, health, Campaign journey,
@@ -345,5 +627,18 @@ Checks run for this first slice against the current workspace:
- Addresses tests: 14 passed (one non-failing SQLite resource warning) - Addresses tests: 14 passed (one non-failing SQLite resource warning)
- JSON Schema validation of the machine-readable assessment: passed - JSON Schema validation of the machine-readable assessment: passed
Refresh checks on 2026-07-22:
- live stable catalog sequence `202607220843`: valid Ed25519 signature trusted
through `release-key-1`, no validator warnings
- current static contract scan: 43 modules, 33 providers, 19 requirements, no
contract errors
- catalog-selected commits: Core `d487726f4d2c` at `v0.1.13` and Campaign
`735e874bd03c` at `v0.1.10`; local and remote release refs agree
- Campaign's immutable `v0.1.10` tag object and peeled commit exactly match the
remote tag and catalog record; Calendar durable outbox work is committed and
pushed after catalogued `v0.1.8`
- JSON Schema validation of the refreshed machine-readable assessment: passed
These checks are evidence for the rows above; they are not a substitute for the These checks are evidence for the rows above; they are not a substitute for the
clean-release and target-environment proof checks. installed-release and target-environment proof checks.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,90 @@
# Datasource And Definition Graph Architecture
## Two-Layer Data Boundary
GovOPlaN separates governed data identity from external acquisition:
| Layer | Owner | Responsibilities |
| --- | --- | --- |
| Datasource layer | `govoplan-datasources` | Governed data/register catalogue, tenant visibility, live/cached/static mode, source authority, staging, immutable materializations, frozen states, schema, quality/freshness policy, institutional provenance, dependencies, and bounded reads |
| Connector layer | `govoplan-connectors` and protocol/provider modules | External protocols, endpoints, connection profiles, credentials, discovery, provider maturity/authority support, health, source-side filtering, query pushdown, and effect reconciliation |
Connectors publish versioned datasource origins. Datasources registers those
origins and presents one stable capability to Dataflow, Workflow, Reporting,
Risk Compliance, and other consumers. Consumers must not import connector
implementations or retain credentials.
The initial provider path is:
1. Connectors imports a bounded JSON/CSV snapshot and exposes it as an origin.
2. Datasources registers it as live or cached, or accepts a direct static upload
through staging.
3. Cached refreshes and static promotions append immutable materializations.
4. Any datasource may expose a frozen state for reproducible execution evidence.
5. Dataflow stores an opaque datasource reference, state policy, and expected
fingerprint.
6. A pinned Dataflow run may publish a complete bounded result as a new
immutable materialization through an idempotent Datasources capability.
Database, REST/HTTP, LDAP/directory, managed file, watched-directory, feed, and
stream providers fit behind the same origin contract. Provider-specific
configuration remains in Connectors.
## Shared Definition Graph
Core owns domain-neutral graph primitives:
- nodes, typed ports, edges, and configuration field descriptors;
- node libraries and category labels;
- graph size, connectivity, cycle, and node-count constraints;
- shared backend validation and frontend connection checks.
Domain modules own their semantics:
- Dataflow provides load, combine, filter, transform, and output nodes. Its
graph is acyclic and has one output.
- Workflow Engine provides trigger, activity, review, decision, wait,
module-action, Dataflow, and outcome semantics. It permits governed loops and
has exactly one trigger plus one or more outcomes. The optional Workflow
module supplies the editor over the same native graph/BPMN language.
This division permits a shared editor shell without making Workflow a special
kind of Dataflow or leaking either module into Core.
## Current Implementation
- Core graph and datasource contracts are versioned at `0.1.0`.
- Workflow Engine owns tenant-isolated definitions, immutable revisions,
activation pinning, module-contributed versioned baselines, runtime instances,
governed action/effect execution, retries, waits, and reconciliation. The
optional Workflow module exposes the reusable native BPMN graph editor.
- Datasources exposes catalogue, origins, staging, promotion, preview,
materialization history, refresh, freeze, retirement, and producer
publication APIs.
- Datasources WebUI exposes all current lifecycle views.
- Connectors adapts existing tabular snapshots to datasource origins.
- Dataflow consumes only Datasources catalogue/lifecycle capabilities and can
request current, live, or latest-frozen state.
- Dataflow exposes typed graph/IR, registry-driven validation/execution/SQL
compilation, expressions and reusable subflows, a pinned run-lifecycle
capability, production worker boundary, and Run/Publish surface. Runs record
lineage and intermediate artifacts and publish only complete bounded results.
- The focused composition check proves Connector origin -> Datasource ->
pinned Dataflow run -> frozen published materialization, including replay.
## Next Slices
1. Add the provider declaration and source-authority binding used consistently
by Connectors, Datasources, configuration packages, Ops, and Docs.
2. Add typed datasource owner/steward, legal/purpose, quality/freshness,
classification, correction, service/process, and downstream dependency
metadata under
[Datasources #6](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/6).
3. Add SQL database and governed REST origin providers with credential-envelope
references and bounded pushdown.
4. Add managed-file and directory origins.
5. Complete datasource quality rules, schema compatibility policy, retention, and
promotion approvals.
6. Complete scheduled/event/API/chained Dataflow trigger governance, reusable
template inheritance, Reporting publication, human reconciliation transforms,
and target resource/recovery evidence for large runs.
+12
View File
@@ -23,6 +23,10 @@ Principles:
panels, calendar list panels, or mailbox folder panels. panels, calendar list panels, or mailbox folder panels.
- Keep collection navigation and collection-level actions close to the relevant - Keep collection navigation and collection-level actions close to the relevant
pane header. pane header.
- In a list-detail workspace such as Scheduling, keep related lists stacked in
the left pane and use the remaining main pane for view/create/edit. A single
Add action stays in the relevant list-pane header and opens the common main
editor; it does not create an additional menu or launcher.
- Use bounded widths for navigation/list panes and let the main detail/content - Use bounded widths for navigation/list panes and let the main detail/content
pane take the remaining space. pane take the remaining space.
- Keep filtering controls inside the pane they affect. - Keep filtering controls inside the pane they affect.
@@ -63,3 +67,11 @@ components directly.
When a module-specific component becomes generally useful, promote it to core When a module-specific component becomes generally useful, promote it to core
with a parameterized API before reusing it elsewhere. with a parameterized API before reusing it elsewhere.
Non-self-explanatory fields use Core `FieldLabel`; documented omissions must
name their accessible-label source. Explicit Discard and dirty navigation use
the same Core unsaved-changes dialog. Table action sets retain unavailable row
actions as disabled controls and reserve empty-state slots so Add remains
aligned. Use central feedback/dialog components; `window.alert` is not an
authorized product surface unless a product-owner-approved exception is first
recorded in the Core decision ledger.
+3 -3
View File
@@ -8,11 +8,11 @@ The same pattern is reusable outside GovOPlaN for any project where Codex works
The repository contains Gitea issue templates in `.gitea/ISSUE_TEMPLATE`, a pull request template in `.gitea/PULL_REQUEST_TEMPLATE.md`, and the label taxonomy in `docs/gitea-labels.json`. The repository contains Gitea issue templates in `.gitea/ISSUE_TEMPLATE`, a pull request template in `.gitea/PULL_REQUEST_TEMPLATE.md`, and the label taxonomy in `docs/gitea-labels.json`.
The scripts infer this repository from `origin` (`git@git.add-ideas.de:add-ideas/govoplan.git`). Override inference when needed: The scripts infer this repository from `origin` (`git@git.add-ideas.de:GovOPlaN/govoplan.git`). Override inference when needed:
```bash ```bash
export GITEA_URL=https://git.add-ideas.de export GITEA_URL=https://git.add-ideas.de
export GITEA_OWNER=add-ideas export GITEA_OWNER=GovOPlaN
export GITEA_REPO=govoplan export GITEA_REPO=govoplan
export GITEA_TOKEN=... export GITEA_TOKEN=...
``` ```
@@ -23,7 +23,7 @@ The API scripts also read `GITEA_*` values from the target repository's `.env` f
GITEA_TOKEN=... GITEA_TOKEN=...
# Optional if origin inference is not enough: # Optional if origin inference is not enough:
GITEA_URL=https://git.add-ideas.de GITEA_URL=https://git.add-ideas.de
GITEA_OWNER=add-ideas GITEA_OWNER=GovOPlaN
GITEA_REPO=govoplan GITEA_REPO=govoplan
``` ```
@@ -0,0 +1,387 @@
# Installation And Deployment Architecture
## Goal
A supported GovOPlaN installation starts with one downloaded, verified
bootstrap artifact. The administrator answers a bounded set of questions and
receives a working base system. Re-running the same tool repairs or
reconfigures that installation instead of creating unrelated state.
The canonical product journey remains
[System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
This document defines the deployer boundary and the first executable slice.
## First Executable Slice
`tools/deployment/govoplan-deploy.py` is a standard-library-only deployment
compiler and reconciler. It can be tested without installing GovOPlaN itself.
It currently supports:
- evaluation and self-hosted profiles;
- managed or external PostgreSQL;
- managed, external, or evaluation-only disabled Redis;
- disabled mail, an external relay declaration, or an evaluation-only
GreenMail service;
- durable local file storage, managed single-node Garage S3, or external
S3-compatible storage;
- an explicit HAProxy service that load-balances configured WebUI and API
replicas without access to the Docker socket;
- declarative API, WebUI, and worker replica counts while keeping migrations
and the scheduler singleton;
- Core, base, or full initial module selections;
- deterministic Compose JSON accepted by Compose v2;
- generated secrets stored in a private `0600` file;
- service-specific environment allowlists so infrastructure containers do not
receive unrelated application credentials;
- plan, render, doctor, status, apply, Kubernetes export, operation history,
and bounded recovery commands;
- an installation lock, migration-before-start ordering, readiness polling,
and an applied-state receipt;
- a durable hash-chained deployment journal captured before runtime mutation;
- PostgreSQL advisory serialization for Core and module migrations;
- runtime initialization that waits for exact configured migration heads
without mutating schema;
- runtime node registration, heartbeats, drain state, and a fenced scheduler;
- idempotent reconfiguration that preserves generated secrets;
- a keyed environment fingerprint that detects private binding changes without
writing secret values to plans or receipts;
- host CPU, memory, disk, entropy, architecture, Docker daemon, Compose,
listen-port, and external endpoint preflight checks;
- service removal without implicit data-volume deletion.
Create a local evaluation bundle:
```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py init \
--directory /tmp/govoplan-evaluation \
--profile evaluation \
--postgres managed \
--redis managed \
--storage garage \
--mail test-mail \
--api-replicas 2 \
--web-replicas 2 \
--worker-replicas 2 \
--module-set base
```
Inspect the generated intent and host requirements:
```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py doctor \
--directory /tmp/govoplan-evaluation
```
Change a component without rotating existing generated secrets:
```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py configure \
--directory /tmp/govoplan-evaluation \
--redis external \
--redis-url 'rediss://:password@redis.example.org:6379/0'
```
The private installation directory contains:
| File | Purpose |
| --- | --- |
| `installation.json` | Versioned, non-secret desired state |
| `secrets.env` | Deployment-local secrets and external service bindings |
| `compose.json` | Deterministic generated Compose definition |
| `garage.toml` | Non-secret managed Garage server configuration |
| `load-balancer.cfg` | Non-secret HAProxy WebUI/API discovery configuration |
| `plan.json` | Latest desired-state diff and readiness findings |
| `receipt.json` | Last successfully applied immutable identities |
| `applied-state/` | Checksum-verified snapshot of the last healthy deployment bundle |
| `operations/<id>/` | Private hash-chained deployment progress and recovery evidence |
| `kubernetes.json` | Optional stateless multi-host Kubernetes export |
| `.deployment.lock` | Same-host operation exclusion |
The specification contract is
[`installation-spec.schema.json`](installation-spec.schema.json).
Build the same dependency-free tool as one downloadable artifact:
```sh
./.venv/bin/python tools/deployment/build-deployer-zipapp.py \
--output /tmp/govoplan-deploy.pyz
python /tmp/govoplan-deploy.pyz --help
```
To exercise reconciliation with locally available evaluation images:
```sh
python /tmp/govoplan-deploy.pyz init \
--non-interactive \
--directory /tmp/govoplan-evaluation \
--profile evaluation \
--api-image local/govoplan-api:test \
--web-image local/govoplan-web:test
python /tmp/govoplan-deploy.pyz apply \
--directory /tmp/govoplan-evaluation \
--allow-unverified-images \
--skip-pull
```
Those images must already contain the selected module set. The override exists
only to exercise local orchestration before release artifacts exist; it is
rejected for `self-hosted`.
## Current Production Gates
The tool deliberately reports blockers instead of pretending the source tree is
a production distribution:
1. **OCI release artifacts.** The release pipeline does not yet publish pinned
multi-architecture API and WebUI images.
2. **Signed distribution manifest.** A channel manifest must bind exact image
digests, Compose compatibility, SBOM/provenance references, and revocation
state. Recording a URL and checksum is not signature verification.
3. **First administrator.** Production needs a one-time, restricted enrollment
identity. The development bootstrap must not be enabled in production.
4. **Image/module composition.** The selected module set must be proven present
in the exact image or installed from verified offline artifacts before it is
enabled.
5. **Deployment agent.** Web updates need a separate privileged reconciler with
a typed command allowlist. The API and browser must never receive the Docker
socket or arbitrary shell access.
6. **Ingress and certificates.** The managed HAProxy service provides HTTP
load balancing inside the deployment boundary; it does not issue or renew
certificates. A self-hosted profile still needs an explicit choice
between an existing reverse proxy and a supported managed ingress, including
trusted-proxy boundaries, TLS certificate issuance, renewal, and health
probing through the public route.
`apply --allow-unverified-images` is therefore restricted to the evaluation
profile. It explicitly acknowledges both mutable image identities and
unverified image/module composition. It is a local test escape hatch, not a
production setting.
## Component Choices
### PostgreSQL
`managed` creates a persistent PostgreSQL container and private generated
credentials. `external` requires an explicit `DATABASE_URL`; switching from
managed to external cannot reuse the old `postgres` Docker hostname
accidentally.
Interactive entry hides external URLs because they commonly contain
credentials. For unattended automation, provide them through a protected
operator mechanism and avoid storing secret-bearing flags in shell history.
Production policy should support external managed databases and local managed
PostgreSQL equally at the application boundary. Backup, point-in-time recovery,
high availability, and major-version upgrades remain deployment properties.
### Redis
`managed` creates an authenticated, append-only Redis container. `external`
requires an explicit `REDIS_URL`. `disabled` is evaluation-only and disables
workers while recording the single-process login-throttle risk acknowledgement.
`doctor` performs a bounded TCP connection check for external PostgreSQL,
Redis, and S3 endpoints. This verifies DNS, routing, and that the port accepts a
connection; it is not an authentication or semantic health check.
Production base installations include Redis because durable queues, distributed
throttling, notifications, scheduled work, and transactional event delivery
must survive API restarts.
### Mail
The first slice distinguishes:
- `disabled`;
- `external-relay`, which records the infrastructure decision but leaves Mail
server/credential creation as a visible post-install task;
- `test-mail`, an evaluation-only GreenMail service.
A bundled production mail server is intentionally not a default. Operating one
requires DNS, reverse DNS, TLS, DKIM, SPF, DMARC, reputation, abuse handling,
queue monitoring, and upgrade policy. A later profile may support an
operator-selected MTA/relay, but it must expose these requirements rather than
presenting a container as a complete mail service.
### File Storage
`local` uses a durable Compose volume and is appropriate for one-host
installations. `garage` provisions Garage 2.3 in its supported single-node
bootstrap mode, generates a private application key and bucket, and connects
the Files S3 backend to the exact installer-owned internal endpoint. The
managed trust marker cannot authorize another S3 host.
Garage metadata and object data use separate persistent volumes. `s3` requires
an external endpoint, region, access key, secret key, and bucket values.
Self-hosted external S3 endpoints must be clean HTTPS origins. The generated
runtime explicitly sets `FILE_STORAGE_S3_ENDPOINT_TRUSTED=true` for that
operator-selected endpoint. The trust flag is not accepted for local storage
and cannot be combined with installer-managed Garage trust.
Local storage must be included in backup and restore drills. Horizontal API or
worker scale-out requires shared/object storage. The managed Garage profile is
persistent but has no data redundancy; availability-sensitive installations
must use a tested multi-node Garage cluster or another external S3 service.
### Load Balancing And Replicas
The generated Compose topology publishes only `load-balancer`. HAProxy uses
Docker DNS service discovery to distribute public traffic across WebUI replicas
and WebUI API proxy traffic across API replicas. The WebUI and API services do
not publish host ports. HAProxy has no Docker socket and discovers only the
bounded replica slots rendered into `load-balancer.cfg`.
Replica counts are desired state:
```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py configure \
--directory /tmp/govoplan-evaluation \
--api-replicas 3 \
--web-replicas 2 \
--worker-replicas 4
./.venv/bin/python tools/deployment/govoplan-deploy.py apply \
--directory /tmp/govoplan-evaluation
```
Workers are queue consumers, so they are scaled through Redis rather than put
behind an HTTP load balancer. Migrations are serialized with a deployment-wide
PostgreSQL advisory lock. The Celery scheduler is run under a renewable,
fencing-token lease. Multiple API replicas are rejected when Redis is disabled
because distributed throttling and queued work cannot then be shared correctly.
This is same-host scaling. Docker Compose uses a bridge network and does not
place containers on another machine. See
[Scaling And Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md) for
the supported topology and promotion path.
## Reconfiguration Semantics
`installation.json` is desired state. `receipt.json` is the last successfully
applied state. `plan` compares their canonical hashes and service sets.
- Adding a managed component creates its service and persistent volume.
- Removing a component removes its service container on apply.
- Volumes are retained by default; deleting data requires a separate,
deliberately destructive workflow.
- Existing generated credentials are retained unless an explicit future rotate
operation is requested.
- Private configuration changes are represented by a keyed fingerprint in the
plan and receipt; plaintext values are never copied there.
- Managed-to-external transitions require the new endpoint in the same
operation.
- Migrations run as a one-shot service before API/worker replacement.
- API, worker, and scheduler start commands wait for exact configured migration
heads; only the migration command is permitted to change schema.
- API and worker replicas register their software/module composition and
heartbeat in PostgreSQL. Ops can request and cancel a node drain.
- The first upgrade from a direct WebUI host port stops that legacy WebUI
container immediately before HAProxy claims the same endpoint.
- Health must recover before a new receipt and applied-state snapshot are
committed.
Every apply operation is journalled before image pulls or runtime mutation. A
failure before migration may restore a verified previous bundle. Once migration
starts, recovery is forward-only unless an independently verified database
backup is restored. See
[Recovery And Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md).
Production updates still need an operator-provided database backup/restore
gate, database compatibility declaration, image signature verification, and
deployment-specific drain policy. The deployment journal proves its own
actions; it does not manufacture backup evidence.
## Stateless Kubernetes Runtime
`render-kubernetes` exports the application tier for a standard orchestrator.
It requires external PostgreSQL, Redis, and S3 and emits no stateful service or
secret value:
```sh
python tools/deployment/govoplan-deploy.py render-kubernetes \
--directory /srv/govoplan/default \
--namespace govoplan \
--secret-name govoplan-runtime
```
The output includes a release-specific migration Job, database-head wait init
containers, API readiness/liveness probes, rolling Deployments, Services, Pod
disruption budgets, a tokenless ServiceAccount, and one fenced scheduler. Apply
the named Secret through the cluster's secret manager and review ingress proxy
CIDRs before deployment. Detailed rollout and scaling rules live in
[Scaling And Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
## Recovery Commands
List durable deployment operations:
```sh
python tools/deployment/govoplan-deploy.py operations \
--directory /srv/govoplan/default
```
Recover a selected failed operation after reviewing its stage evidence:
```sh
python tools/deployment/govoplan-deploy.py recover \
--directory /srv/govoplan/default \
--operation-id <operation-id>
```
The command reports whether it restored the pre-migration applied bundle,
requires forward recovery, or needs manual intervention. Add `--apply` only
after that decision has been reviewed.
## Web Update Boundary
The intended update path is:
1. Ops reads the non-secret installation receipt and reports management mode,
current release, component health, and update availability.
2. An authorized administrator asks Core to create a typed deployment request,
for example `reconcile_release` or `rollback_release`.
3. Core persists the reviewed immutable plan, actor, expected current receipt,
and idempotency key.
4. A separately deployed, narrow deployment agent claims the request.
5. The agent verifies signatures/digests, acquires a fenced deployment lock,
backs up, pulls, migrates, reconciles, probes health, and writes evidence.
6. Ops presents durable progress and the resulting receipt.
The agent owns container-runtime access. It accepts no command strings from the
browser and has no domain-data permissions. Installations managed by Kubernetes,
systemd, or another external orchestrator expose read-only status and an export
of the reviewed update recipe instead of a non-functional update button.
## Distribution Workflow
The downloadable entry point should eventually be:
```sh
curl --proto '=https' --tlsv1.2 --fail --location \
https://govoplan.add-ideas.de/install/v1/bootstrap.pyz \
--output govoplan-bootstrap.pyz
python3 govoplan-bootstrap.pyz init
```
The published documentation must include an independent checksum/signature
verification command before execution. The zipapp then downloads only a signed
distribution manifest, verifies it against an embedded or explicitly installed
keyring, and renders the same installation contract implemented here.
The source-tree script is the test harness for that future zipapp. It is not yet
the internet bootstrap artifact.
## Verification
Run the focused tests:
```sh
./.venv/bin/python -m unittest -v tests.test_deployment_installer
```
The tests cover profile restrictions, secret persistence, external endpoint
requirements, managed Garage bootstrap, S3 policy, replica validation, HAProxy
discovery configuration, Compose service selection, secret non-disclosure,
service-specific environment isolation, private file modes, external endpoint
preflight, first-plan generation, apply ordering, receipt idempotency,
hash-chained recovery journals, migration recovery boundaries, and stateless
Kubernetes rendering.
@@ -0,0 +1,575 @@
# Institutional Governance Target Architecture
## Status and sources
This document is the accepted architectural reconciliation of two product
concepts prepared outside the repositories:
- `govoplan_concept_dev.md`
- `software_big_picture.md`
The source concepts describe GovOPlaN as an operational governance platform for
public institutions. This document merges that direction with the implemented
platform state as of 2026-08-01. It is the canonical repository version of the
direction. Gitea issues remain the source of truth for delivery state.
Read this together with:
- [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md)
- [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md)
- [Module Contracts and Install Boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md)
- [Capability and Infrastructure Fit](CAPABILITY_AND_INFRASTRUCTURE_FIT.md)
- [Core Module Architecture](../../govoplan-core/docs/MODULE_ARCHITECTURE.md)
- [Core External References and Integration Maturity](../../govoplan-core/docs/EXTERNAL_REFERENCES_AND_INTEGRATION_MATURITY.md)
- [Core Action, Effect, and Automation Layer](../../govoplan-core/docs/ACTION_EFFECT_AUTOMATION_LAYER.md)
## Decision
GovOPlaN is a configurable **institutional governance and operations layer** for
public institutions. It should model the institution, coordinate its work,
connect its specialist systems, and preserve why and under whose authority an
action occurred.
GovOPlaN is not intended to become one universal ERP, DMS, groupware suite,
workflow editor, or specialist procedure. It should own the governance concepts
that must remain understandable across those systems and support native,
external, mirrored, synchronized, overlay, and link-only operation explicitly.
This changes product emphasis, not the modular architecture:
1. The current kernel and optional-module model remains.
2. Existing domain owners keep their data and behavior.
3. Cross-module semantics become explicit, versioned contracts.
4. Successful compositions become product and sector packages, not forks or
monolithic replacement applications.
5. Repository creation follows a proof threshold; a noun in the information
model does not automatically require a module.
## What recent work already supersedes
The source concepts predate several implemented foundations. These items are
accepted as the current baseline and must not be reopened as greenfield work.
| Concept requirement | Reconciled current state |
| --- | --- |
| Slim kernel plus installable modules | Implemented through entry-point discovery, `ModuleManifest`, migrations, capabilities, interfaces, WebUI contributions, and permutation checks. |
| Versioned cross-module contracts | Implemented through named interface ranges, capability protocols, static workspace graph checks, activation validation, and release checks. |
| Separate headless workflow runtime and editor | Implemented as `govoplan-workflow-engine` and optional `govoplan-workflow`. Module-owned workflow baselines are versioned and reconciled without replacing local overrides. |
| Provider-neutral external references | Implemented in Core with stable external identity and cumulative integration maturity from discovery through replacement. |
| Governed asynchronous effects | Implemented foundations include the action/effect contract, transactional platform event outbox, module outboxes, idempotency, outcome-unknown states, reconciliation, and worker health. Coverage still varies by provider. |
| Acting identity, function assignment, mandate, and ownership recovery | Implemented foundations span Identity, Organizations, IDM, Access, Mandates, generic ownership transfer/recovery, and audit provenance. Effective competence now resolves through a tenant-bound Mandate capability. |
| Governed data foundations | Connectors, Datasources, Dataflow, Reporting, and Search now exist. Datasources already provides live/cached/static modes, staging, immutable materializations, and publication contracts. |
| Task-focused projections and configured documentation | Views, view-surface declarations, configurable dashboards, and manifest-driven user/admin documentation exist. Rollout and content depth remain incremental. |
| Encryption as an optional capability | Core defines provider-neutral contracts; `govoplan-identity-trust` persists public device keys, key epochs and assurance evidence; and `govoplan-encryption` persists opaque vault/key lifecycle, versioned protection envelopes, quorum recovery authorization, outcome-unknown reconciliation, and disable preflight. No concrete cipher/provider or E2EE claim is bundled. |
| Search without mandatory OpenSearch | PostgreSQL-backed, permission-aware search and module provider contracts exist; OpenSearch remains an optional adapter. |
| Scale-out and recovery architecture | Stateless API/worker, shared database/object storage, event delivery, deployment, and recovery contracts are documented and partly exercised. Production profiles and drills remain active work. |
The institutional semantics, provider declaration gate, and first product
compositions described here are now implemented. Subsequent work is
**product depth and stronger maturity evidence**, not another runtime rewrite
or an unimplemented architecture boundary.
## Implementation status (2026-08-01)
The architecture contract is implemented as a bounded, executable vertical
slice. The portfolio declarations and provider governance gates apply to the
whole workspace, while the four semantic domains whose repository thresholds
were proven now have independent persistent owners:
| Area | Implemented state | Remaining rollout |
| --- | --- | --- |
| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for all 62 source manifests. Focused and release checks enforce `--require-architecture`; a new module cannot enter the workspace without truthful declaration and repository-local evidence. |
| External providers | Core validates provider objects/field groups, operations, integration maturity, source authority, bounded reads, freshness/health, idempotency, conflicts, outcome-unknown handling, evidence, correction, reconciliation, outage, classification, purpose, retention, and secret handling. Addresses/CardDAV, Files remote storage, Mail SMTP/IMAP, Calendar CalDAV/ICS/Graph/EWS, and Connectors tabular/sanctions providers declare the contract and tenant-bounded secret-free runtime state. | Registry validation rejects any declared external provider without a sanitized state provider. Future adapters must cross the same gate before activation. |
| Institutional context | Core provides versioned temporal, actor/representation, institution/unit/function/task/mandate/jurisdiction/service/case/party/work-item/workflow/approval/decision/record, legal-basis, evidence, information-governance, external-source, presentation, and geographic references. Events, automation actions, audit records, and the transactional Audit outbox preserve the envelope. | Owning modules must progressively require the relevant subset for consequential operations. |
| Semantic provider contracts | Provider-neutral DTOs and protocols cover Mandate resolution, versioned Service definitions, procedure Parties/representation, and formal Decisions. `govoplan-mandates`, `govoplan-services`, `govoplan-parties`, and `govoplan-decisions` now persist immutable revisions behind those contracts with tenant isolation, bounded reads, replay safety, OCC, migrations, uninstall guards, permissions, APIs, capability documentation, and recovery documentation. | The owners are deliberately headless. Procedure-specific UI remains with consuming modules. |
| Formal-outcome proof | Committee persists bodies, meetings, agenda items, minutes, lifecycle events, and an optional protected local Decision projection. Voting separately owns immutable ballot definitions, frozen electorates, recorded casting/replacement, deterministic tally, certification, challenge, annulment, and provider-backed assurance profiles. Committee consumes `voting.ballots` and retains only deliberation linkage and verified aggregate outcome. | Native recorded ballots are reconstructable, not secret. A concrete confidential/secret/certified provider and deployment assurance still require protocol/custody/legal decisions and target evidence. |
| Service-to-case proof | Services owns the persistent exact definitions consumed by Portal discovery and Cases intake. Forms owns immutable multi-page/conditional/localized schemas, accessibility assessment and package fragments. Forms Runtime owns definition-aware drafts, validation, submission receipts, status/evidence history, and durable native Case/Workflow handoffs with intent-before-effect and outcome-unknown reconciliation. Portal delegates URL, Case, Form, or Workflow launch to an installed owner while retaining exact Service/Form provenance. | Anonymous intake and concrete attachment/signature providers remain product depth. Portal and Runtime fail closed and explain any absent launcher, target capability, or provider prerequisite. |
| Generic approvals and process execution | Approvals persists exact-subject chains, delegation, separation of duties, quorum, signatures-as-evidence, escalation, OCC, and replay-safe decisions. Campaign proves an exact-version delivery gate. Workflow Engine owns immutable definitions/instances plus API, schedule, event and parent triggers, durable timer/event waits, scale-out claims, current-authority rechecks, and idempotent starts independently of the optional editor. | Policy-authored Approval template selection, concrete signature providers, cron adapters, and broader BPMN execution profiles are product/provider depth on explicit contracts. |
| Device trust and content protection | Identity Trust separates public device keys, epochs, assurance and key-access decisions from login and Access. Encryption separates resource ownership from opaque provider key custody, versioned envelopes, migration evidence, quorum recovery authorization and uninstall proof. Files/Postbox fixtures prove the optional boundary. | Concrete reviewed KMS/HSM/client providers, feature adapters, backup/restore/key-loss drills, and E2EE interoperability/certification remain required before a deployment protection claim. |
| Procedure-party proof | Parties persists effective procedure roles, frozen contact snapshots, and representation powers. Existing powers cannot disappear or be silently rewritten; explicit OCC-guarded revocation is required. Cases resolves the provider capability and excludes expired/revoked authority from downstream delivery. | Procedure modules still decide which contextual fields and actions to present. |
| Integrated institutional journey | The executable `product.service-to-decision` fixture uses real SQL-backed Services, Cases, Parties, Mandates, Committee, and Decisions providers. It carries one exact Service version through persisted Case intake, representation and frozen delivery authority, effective Mandate resolution, a body/meeting/agendum/vote/minute sequence, a persisted formal Decision, confirmed Postbox effect, Audit/record evidence, remedy/review, and protected reconstruction. A second executable path proves Portal to exact Form revision, persisted submission, and idempotent replay. | This is architecture and composition evidence. Signed, release-bound target accessibility, privacy, security, operator, delivery-provider, and recovery-drill evidence is still required before the package may claim `reference_ready`. |
| Governed data catalogue | Datasources stores typed governance metadata, exposes bounded tenant-scoped filters and update APIs/UI, carries governance through staging, and snapshots it into immutable materializations. Reporting now persists immutable dataset, semantic-model, report, quality-plan, saved-view, and schedule revisions; executes typed semantic queries with quality gates, access checks, replay, pivoting, export/import assessment, and provenance; and exposes the governed analytical WebUI. | Rich dependency/impact traversal, additional expression functions, and policy-specific field visibility can grow on the established contracts without moving connector, transformation, or source ownership. |
| Portfolio and change governance | Projects now persists tenant-safe, immutable portfolio/project/milestone revisions with OCC, replay, lifecycle rules, restricted memberships, Search ACL indexing, outcomes, benefits, dependencies, capacity assumptions, change impact, and institutional references. Its WebUI exposes the planning catalogue and core planning fields. | Advanced planning structures already accepted by the API can receive deeper specialized editors without creating a second Policy, Reporting, Resources, or Goals owner. |
| Product/package governance | Signed configuration packages distinguish reference, product, sector, deployment, and integration classes; preserve parent/evidence provenance; prevent derived packages from loosening constraints; and preflight provider authority, maturity, exact binding, health, freshness, and recovery expectations. Executable product manifests now exist for governed communication and governed data/assurance and are checked in the module matrix. | Both artifacts deliberately remain product-class until target, accessibility, privacy, security, operations, and recovery evidence justifies reference readiness. |
| Projection and release | Platform metadata, signed module catalogs, release synthesis, Ops, and role-aware Docs retain and display architecture/provider declarations. Module-owned state providers add bounded configured/active, authority, health, freshness, conflict, recovery, and observation state; ordinary-user Docs omits binding detail. Static checks validate evidence paths, and the WebUI build verifies consuming types. | Runtime-state adoption and broader portfolio presentation follow truthful provider declaration rollout. |
The implementation deliberately keeps shared reference contracts in Core and
domain tables in their owners. It does not claim unsupported release maturity:
the four extracted owners and package remain `vertical_slice`/`product` until
target evidence supports a stronger claim. Gitea remains authoritative for
feature depth beyond this architecture contract.
## Target capability layers
The layers describe ownership and dependency direction. They are not navigation
groups and do not imply that every installation exposes every module.
| Layer | Responsibility | Current owners and declared directions |
| --- | --- | --- |
| 0. Runtime and meta | Composition, release, migrations, shared contracts, operations, deployment | Core, meta repository, Admin, Ops |
| 1. Institutional foundation | Institution, tenant, identity, organization, function, authority, access, trust | Tenancy, Identity, Organizations, IDM, Access, Identity Trust, Encryption, Mandates |
| 2. Governance and accountability | Policy, audit, risk, control, explainability, configured projection | Policy, Audit, Risk Compliance, Docs, Views, Search, Decisions |
| 3. Human work and procedure | Intake, cases, tasks, approvals, process execution and editing | Services, Forms, Forms Runtime, Cases, Parties, Tasks, Approvals, Workflow Engine, Workflow, Tickets |
| 4. Communication and participation | Delivery, participation, scheduling, channels, consultation | Portal, Postbox, Notifications, Mail, Campaign, Calendar, Scheduling, Poll, Appointments, Booking, Consultation, Committee, Addresses, Distribution Lists |
| 5. Content, records, and evidence | Managed content, templates, records, knowledge, disclosure | Files, Templates, DMS, Records, Wiki, Transparency, Certificates |
| 6. Data, reporting, and integration | Source access, staging, transformation, search, analytics, protocols | Connectors, Datasources, Dataflow, Reporting, Dashboard, REST, SOAP, XOE/V, XTA/OSCI, FIT-Connect, XRechnung, ERP adapters |
| 7. Domain capabilities | Reusable public-sector subject matter | Projects, Procurement, Contracts, Grants, Resources, Assets, Facilities, Learning, Payments, Ledger, Permits, Inspections, Evaluation, Helpdesk |
| 8. Product and sector packages | Versioned compositions, terminology, forms, processes, controls, reports, integration profiles | Signed configuration packages and reference packages; not runtime modules by default |
## Canonical institutional semantics
The connected model must keep these concepts distinct even where one UI
combines them.
| Concept | Canonical answer | Owner or direction |
| --- | --- | --- |
| Institution and tenant | In which governed installation and tenant does work occur? | Tenancy and Organizations |
| Organization and unit | Where is responsibility situated? | Organizations |
| Function | Which named organizational responsibility can an incumbent hold? | Organizations |
| Identity and account | Who is the person or machine, and through which account do they act? | Identity and Access |
| Function assignment | Who holds or represents a function, for which interval and source? | IDM |
| Role and permission | What application behavior may the acting principal perform? | Access, constrained by Policy |
| Mandate and jurisdiction | Why is an institution, unit, or function competent to act on this subject, territory, population, or interval? | Mandates |
| Service | What governed promise can an institution offer, to whom, under which prerequisites, evidence, channel, deadline, and responsibility? | Services; Portal presents it |
| Case | Which concrete administrative matter is being handled? | Cases |
| Party | In what procedural capacity does a person or organization participate, and who may represent or receive for it? | Parties; Identity/Organizations remain the subject owners |
| Work item | What must a responsible actor do next? | Tasks and domain modules |
| Workflow | How is work coordinated, including waits, human hand-offs, and governed actions? | Workflow Engine; Workflow is the optional editor |
| Approval | Has a proposed action passed a configured review or separation-of-duties gate? | Approvals |
| Decision | What formal institutional outcome was reached, by which competent authority, on which facts, rules, evidence, reasoning, and review path? | Decisions |
| Evidence and record | What proves the input, state, action, effect, correction, and retained institutional memory? | Domain owner, Files/DMS/Records, and Audit |
### Extracted semantic modules
Four horizontal concepts passed the repository proof threshold. Their Core
DTOs and provider protocols remain neutral; their persistent data, lifecycle,
security, APIs, migrations, and recovery behavior now live in independent
repositories.
#### Mandates
Mandates should own public or internal tasks, jurisdiction, responsibility,
decision/signature authority, legal or organizational basis, and effective
history. Organizations continues to own structures and functions; IDM owns
incumbency; Access owns permissions; Policy owns constraints.
`govoplan-mandates` answers: *Was this function competent to act for this case
at the relevant time, and on what basis?* Its resolver evaluates effective
time, task, authority, unit, function, jurisdiction, subject, conflicts, legal
basis, and evidence deterministically. Missing or ambiguous authority fails
closed.
#### Services
Services should own versioned service definitions: audience, prerequisites,
legal basis, evidence, fees, deadlines, channels, responsible unit/function,
jurisdiction, forms, case/workflow/result bindings, remedies, service levels,
and publication status. Portal presents and starts services but should not own
their institutional definition.
`govoplan-services` now owns those exact versioned definitions. Portal is the
first presentation consumer and Cases freezes the selected revision into its
intake context. Availability is an independent capability so publication does
not imply that all runtime prerequisites are satisfied.
#### Parties
Parties should own procedure-local roles and relationships: applicant,
respondent, beneficiary, representative, joint applicant, delivery recipient,
power or authority to represent, and permitted/preferred channels for the
matter. Identity answers who the subject is; Organizations answers which
institutional unit it is; Addresses owns contact points; Parties answers how
the subject participates here.
`govoplan-parties` owns the shared effective-dated lifecycle. Cases retains a
bounded compatibility projection only when the module is absent; that fallback
contains no representation lifecycle and cannot silently become a second
authority source.
#### Decisions
Decisions should own formal outcomes: subject, type, competent authority,
facts, evidence, applicable rule versions, reasoning, operative result,
conditions, effect, delivery/publication, remedy/review, correction, revocation,
and links to observed effects. Approvals own review gates; Poll owns response
collection; Committee owns deliberation, meetings, and votes; Workflow owns
coordination.
`govoplan-decisions` owns the persistent lifecycle and protected reconstruction
surface. Committee supplies deliberation context and records through the
provider capability. Consumers retain exact Decision references without
gaining table access.
## Source authority and integration maturity
Two independent dimensions must be recorded. They must not be collapsed into a
single `sync` flag.
### Source-authority mode
| Mode | Meaning |
| --- | --- |
| `native_authoritative` | GovOPlaN owns the authoritative object and lifecycle. |
| `external_authoritative` | The external system owns the object; GovOPlaN reads or acts through it. |
| `external_mirror` | The external system is authoritative and GovOPlaN keeps a governed local projection or immutable snapshots. |
| `governed_sync` | Both sides may change supported fields under explicit conflict and reconciliation rules. |
| `governance_overlay` | GovOPlaN owns policy, responsibility, evidence, or coordination around an externally executed object. |
| `linked_reference` | GovOPlaN keeps only a stable link and minimal display/provenance metadata. |
Authority may be declared per tenant, organization, service, object type,
object, field group, or process step. A broad default must not hide a narrower
override.
### Integration maturity
The implemented maturity ladder remains `discover`, `link`, `search`, `read`,
`publish`, `synchronize`, `migrate`, and `replace`. Maturity says what an
adapter can do. Source-authority mode says who owns truth in a particular
configuration. For example, a connector may support `synchronize`, while a
tenant deliberately configures it as `external_mirror`.
### Provider declaration
Every provider that reads or causes external effects must declare:
- owned object and field groups;
- supported source-authority modes and integration maturity;
- read, write, delete, search, preview, and dry-run operations;
- revision/concurrency tokens, freshness, health, and bounded-read limits;
- idempotency, retry, timeout, conflict, and outcome-unknown behavior;
- evidence, audit, correction, rollback/compensation, and reconciliation paths;
- degraded and outage behavior;
- classification, purpose, retention, and secret-handling requirements.
The common provider declaration composes the external-reference, action/effect,
connector-lifecycle, capability, operational-check, and documentation
contracts. Core, release tooling, Ops, Docs, and configuration-package
preflight validate it; Registry refuses to activate a declared external
provider without bounded, sanitized runtime state.
## Cross-cutting contracts
The following contracts are mandatory for consequential domain objects. They
should be shared reference DTOs and provider protocols, not shared domain
tables in Core.
1. **Time and history:** valid-from/to, recorded-at, superseded-at, revision,
change reason, and stable identity.
2. **Actor and representation:** real account/identity, system or service
account, represented account/function/party, delegation or power, and
mandate reference.
3. **Institutional context:** tenant, institution, organization unit, function,
task/mandate, jurisdiction, service, case, and decision references.
4. **Legal and policy basis:** typed, versioned references to rules,
obligations, policies, exceptions, and the effective decision source.
5. **Requested and observed effect:** intent, approval, dispatch, possible
execution, confirmation, reconciliation, correction, and terminal evidence.
6. **Evidence and provenance:** source, version, checksum, derivation,
responsible actor, timestamps, and inspection links.
7. **Information governance:** classification, purpose, legal basis, retention,
hold, minimization, and disclosure state.
8. **External source:** system/profile/object identity, authority mode,
maturity, version, freshness, health, and conflict state.
9. **Presentation:** language, accessibility, channel, explanation, and
configured availability.
Existing contracts already cover substantial parts of items 1, 2, 5, 6, 8,
and 9. New work should extend those contracts instead of creating parallel DTO
families.
## Existing module direction changes
### Datasources becomes the governed data and register catalogue
The implemented live/cached/static, staging, immutable materialization, and
publication model includes typed governance metadata for owner/steward,
authoritative source and authority mode, legal basis and purpose, semantic
definition, quality and freshness policy, classification, transfer agreement,
correction process, affected services/processes, and dependent flows,
reports, controls, and decisions. Connector credentials and protocol behavior
remain outside Datasources.
### Projects grows into portfolio and change governance
The Projects boundary already includes portfolios and goals. Extend it through
versioned objectives/outcomes, dependencies, capacity, benefits, change impact,
and links to mandates, services, risks, contracts, resources, and indicators.
Do not create a separate Goals module before more than one domain proves an
independent goal lifecycle.
### Reporting becomes evidence-backed institutional measurement
Every report, measure, and indicator should explain the institutional question
or obligation it serves, owner, source/materialization and flow revision,
freshness/quality, calculation version, visibility/purpose limits, publication,
and decisions or actions that consumed it. Reporting owns presentation and
execution; source and transformation owners retain their domains.
### Risk Compliance becomes the horizontal assurance model
Sanctions screening remains a complete vertical slice. The broader reusable
model is:
```text
Obligation -> governed object -> risk -> control -> evidence -> finding -> measure -> effectiveness review
```
Risk Compliance now persists that effective-dated, immutable-revision assurance
graph, exposes bounded tenant-safe traversal/search/editing, and projects each
completed sanctions run into it idempotently. Policy
owns enforceable rules and decisions; Audit owns immutable event evidence;
domain modules own the governed objects and corrective actions.
### Connectors exposes authority and effect behavior
Connector direction (`consume`, `publish`, `bidirectional`) remains useful but
is not enough. Profiles and bindings need the source-authority mode and
provider declaration above. ERP remains an integration family: finance,
workforce, procurement, asset, or other domain modules own semantics while
connectors own transport and source interaction.
### Geography starts as a reference contract
Before adding a `govoplan-geo` module, define a common reference shape for
coordinates, geometry, administrative area, address/location, CRS, source,
accuracy, validity, and external GIS identity. Create a repository only when
GovOPlaN must own spatial datasets, topology, or independent geospatial
lifecycles rather than link to an external GIS.
## Product and sector packages
A module says what capability can exist. A product package says how capabilities
work together for a bounded outcome. A sector package specializes vocabulary,
forms, rules, process baselines, controls, reports, and integration profiles
without forking the platform.
The signed configuration-package mechanism distinguishes:
- **reference package:** tested composition proving a journey and its recovery
behavior;
- **product package:** reusable operating capability such as governed
communication, service-to-decision, procurement/contracts, or governed BI;
- **sector package:** institutional specialization such as municipality,
university/research, ministry/program, regulator, grants authority, or
committee/council;
- **deployment profile:** supported infrastructure and operational topology;
- **integration profile:** supported set of external systems, authority modes,
bindings, and health expectations.
Packages may require modules and capabilities, but package definitions remain
configuration and evidence. They do not gain access to module-owned tables.
## Module portfolio metadata
Repository category is not capability maturity. The runtime manifest, release
catalog, Docs projection, and meta repository inventory use one
machine-readable declaration with at least:
- architecture layer and module kind;
- lifecycle/maturity claim: `concept`, `scaffold`, `vertical_slice`,
`reference_ready`, `supported`, or `lts`;
- evidence supporting the claim and known limits;
- supported source-authority modes;
- owned and explicitly non-owned concepts;
- provided/required capabilities and interfaces;
- reference packages and target-tested providers;
- migration, upgrade, recovery, security, and operations documentation.
Maturity is a release claim and must be checked against evidence. A manifest
must not become “supported” merely because a maintainer changes one string.
Create a repository only when the capability has distinct data ownership,
independent installability, technical assets, a security/lifecycle profile, a
release reason, more than one consumer or a proven reference process, and tests
that justify the boundary. Otherwise use a shared DTO, provider capability,
submodule, configuration fragment, package, or profile.
## Implemented migration sequence
### 0. Align the portfolio and contracts - complete
- This reconciliation is canonical in the meta repository and mirrored to the
Gitea wiki.
- All 62 source manifests carry validated evidence-based architecture metadata.
- External-reference, action/effect, operational-health, ownership, policy,
audit, and documentation primitives compose into one enforced provider
declaration and sanitized runtime-state contract.
- Institutional context, legal basis, evidence, presentation, external source,
information governance, temporal revision, and geo references are shared
Core DTOs rather than shared domain tables.
### 1. Prove responsibility and formal outcome - complete
- Mandate and Decision contracts, deterministic resolution, lifecycle
transitions, persistence providers, APIs, permissions, migrations, recovery,
and tests are implemented.
- Committee and the SQL-backed institutional fixture prove effective-time
authority, persisted meeting/agendum/vote/minute context, approval context,
reasoning, evidence, observed effect, correction/revision rules, protected
reconstruction, and review references.
- The independent Mandates and Decisions repositories were created only after
persistence and reuse passed the repository threshold.
### 2. Separate service and party semantics - complete
- Portal remains the presentation surface while Services owns reusable,
versioned definitions and explainable availability.
- Parties owns procedure roles, contact snapshots, and append-only
representation/revocation authority; Cases consumes the common resolver.
- `product.service-to-decision` proves both through a portable administrative
service composition.
- Forms owns immutable, versioned schemas while Forms Runtime owns drafts,
server validation, submission receipts, status/evidence history, and exact
Service/Form provenance. Portal delegates Form launch through the runtime
capability and fails closed when it is unavailable.
### 3. Complete governed data, portfolio, and assurance - vertical slices complete
- Datasources carries typed governance through staging and immutable
materializations, with bounded catalogue filters and dependency references.
- Reporting owns immutable semantic definitions, safe execution, quality gates,
provenance, schedules, saved views, pivoting, and export/import assessment
without taking source or transformation ownership.
- Risk Compliance persists the horizontal obligation/risk/control/evidence/
finding/measure graph and projects sanctions runs idempotently.
- Projects persists portfolio/outcome/change-governance revisions as a
consuming domain without becoming a second policy or reporting engine.
### 4. Package repeatable public-sector outcomes - complete at product maturity
- Governed communication, governed data/assurance, and service-to-decision are
portable product package manifests with repository-local evidence.
- Package preflight enforces module, capability, provider authority, health,
freshness, and recovery expectations without cross-module table access.
- Sector and `reference_ready` claims remain gated on target-environment,
recovery, accessibility, privacy, security, and operator evidence. This is a
maturity gate, not missing architecture implementation.
## What remains after the executable architecture slice
The remaining work is not another Core or cross-module architecture rewrite.
It falls into two explicitly different categories, neither of which can be
truthfully completed by adding generic platform code:
1. **Concrete provider packages:** the Committee ballot adapter contract is
complete, but a real secret/electronic ballot provider requires a selected
protocol and product decisions for voter eligibility, custody, secrecy,
recount, challenge, retention, and operational assurance. Equivalent future
adapters must satisfy the declared provider and recovery gates.
Provider selection and certification are tracked in
[Committee #1](https://git.add-ideas.de/GovOPlaN/govoplan-committee/issues/1).
2. **Target-produced maturity evidence:** `reference_ready`, `supported`, and
`lts` cannot be generated from source code. An exact release and deployment
must produce signed, expiring accessibility, privacy, security, operator,
provider, backup/restore, rollback, and recovery-drill evidence. The verifier
and schemas are implemented; the actual claims require those real runs.
The pinned-release evidence run is tracked in
[GovOPlaN #37](https://git.add-ideas.de/GovOPlaN/govoplan/issues/37).
Forms and Forms Runtime no longer constitute an architecture gap. Conditional
multi-page/localized authoring, package-fragment import, and durable native
Case/Workflow handoffs are implemented. Remaining depth is limited to
anonymous/public identity profiles, concrete file/signature providers, and
additional handoff target adapters. Those use the implemented immutable
definition, runtime, policy, evidence, service-launch, and domain-owner
boundaries rather than requiring another split. Public/provider decisions stay
tracked in Forms Runtime
[#2](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/2) and
[#3](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/3).
Approvals, Voting, Workflow trigger/wait dispatch, Identity Trust, and
Encryption now likewise have repository owners, neutral Core contracts,
persistence, migrations, recovery/disable semantics, documentation and focused
tests. Their remaining tickets concern concrete providers, deeper adapters and
target evidence, not an unresolved institutional architecture boundary.
Everything else described as architecture in this document now has a
repository owner, versioned contract, bounded implementation, migration and
recovery boundary where state exists, documentation, and executable evidence.
Further work in those modules is product breadth, UX depth, provider adoption,
and evidence renewal.
## Delivery tracking
The completed cross-repository architecture epic is
[GovOPlaN #29](https://git.add-ideas.de/GovOPlaN/govoplan/issues/29).
Its implementation work packages and resulting owners are:
- [Core #279](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/279):
validated module architecture and provider authority declarations;
- [Core #280](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/280):
shared institutional-context and governed reference primitives;
- [GovOPlaN #30](https://git.add-ideas.de/GovOPlaN/govoplan/issues/30) and
[govoplan-mandates](https://git.add-ideas.de/GovOPlaN/govoplan-mandates):
Mandates semantics and persistent resolver;
- [GovOPlaN #31](https://git.add-ideas.de/GovOPlaN/govoplan/issues/31) and
[govoplan-services](https://git.add-ideas.de/GovOPlaN/govoplan-services):
Services semantics, catalogue, and availability;
- [GovOPlaN #32](https://git.add-ideas.de/GovOPlaN/govoplan/issues/32) and
[govoplan-parties](https://git.add-ideas.de/GovOPlaN/govoplan-parties):
Parties and representation semantics and resolver;
- [GovOPlaN #33](https://git.add-ideas.de/GovOPlaN/govoplan/issues/33) and
[govoplan-decisions](https://git.add-ideas.de/GovOPlaN/govoplan-decisions):
formal Decisions semantics and registry;
- [Datasources #6](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/6):
governed data/register catalogue;
- [Risk Compliance #7](https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance/issues/7):
horizontal assurance graph;
- [GovOPlaN #34](https://git.add-ideas.de/GovOPlaN/govoplan/issues/34):
product and sector package classes; and
- [Docs #19](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/19):
configured architecture, maturity, and source-authority explanations;
- [Forms #2](https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/2):
immutable reusable definitions and the designer surface; and
- [Forms Runtime #1](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/1):
definition-aware submissions and Portal service launch;
- [Forms #3](https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/3) and
[Forms Runtime #4](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/4):
conditional/localized definition depth and governed native handoffs;
- [Approvals #1](https://git.add-ideas.de/GovOPlaN/govoplan-approvals/issues/1)
and [Campaign #22](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/22):
generic exact-subject approval chains and one consequential delivery gate;
- `govoplan-voting`: governed recorded ballots plus fail-closed provider-backed
assurance profiles consumed by Committee; and
- Identity Trust #1 and Encryption #1-#3: public device trust, provider-neutral
key/protection lifecycle, recovery authorization and disable proof, while
concrete provider conformance remains separately gated.
Existing Projects #1, Reporting #4, Portal #1, Cases #1, Datasources #1,
Risk Compliance #2, GovOPlaN #14, and GovOPlaN #19 carry product-depth and
reference-readiness work instead of duplicating the completed architecture
contract.
## Completion evidence
The architecture direction is established by the following executable and
machine-enforced evidence:
- the `product.service-to-decision` composition and SQL-backed golden fixture
retain institutional context from service entry through a persisted case,
party, authority/work context, committee deliberation, decision, observed
communication effect, minute/record, and review references;
- the Portal/Form journey retains the exact published Service and Form
revisions through persisted draft state, validates on the server, and returns
the same submission on an idempotent launch replay;
- the system can answer who acted, for whom, in which function, under which
mandate and jurisdiction, using which rule and evidence versions;
- every implemented external binding declares authority mode, maturity,
operations, health, freshness, conflict, and recovery behavior, and Registry
rejects a declaration without sanitized runtime state;
- every material report or decision can be reconstructed from governed source
and transformation versions;
- product/package manifests are portable without cross-module table access or
code forks, while future sector packages inherit the same signed-package
constraints; and
- documentation and Ops explain the configured composition and its limits to
users, administrators, operators, and auditors.
These criteria complete the architecture contract at `vertical_slice` and
`product` maturity. They do not waive the separately enforced evidence needed
for a module or package to claim `reference_ready`, `supported`, or `lts`.
The capability-fit verifier now computes that cumulative readiness gate from
independently signed, expiring claims bound to the exact assessed release,
installed payload, deployment subject, controls, and artifact hashes. Actual
target runs and recovery drills remain operator-produced evidence.
+47 -17
View File
@@ -110,7 +110,10 @@ one.
upper-right of the page heading. `Discard` comes before `Save …`, with the upper-right of the page heading. `Discard` comes before `Save …`, with the
primary save action at the far right. Keep both controls in the same place primary save action at the far right. Keep both controls in the same place
across validation, loading, and saved states; guard unsaved work when the across validation, loading, and saved states; guard unsaved work when the
user discards or navigates away. user discards or navigates away. Explicit Discard and dirty in-application
navigation use the same central unsaved-changes dialog and registered
save/discard callbacks. A browser-controlled tab/window unload warning is the
only unavoidable different surface.
- A page or panel has one visually primary action for its current state. Put - A page or panel has one visually primary action for its current state. Put
secondary actions beside it. Separate destructive actions and name their real secondary actions beside it. Separate destructive actions and name their real
effect. effect.
@@ -125,10 +128,16 @@ one.
- Row actions in tables are icon-only controls in a stable rightmost action - Row actions in tables are icon-only controls in a stable rightmost action
column. Order them by intent: inspect/open, edit, copy/duplicate, column. Order them by intent: inspect/open, edit, copy/duplicate,
transfer/share/download, retry/restore, then remove/delete last. Omit actions transfer/share/download, retry/restore, then remove/delete last. Omit actions
that do not apply without closing the gap or changing the relative order. only when they are structurally irrelevant to the entire table. An action
Every icon has a translated accessible name and matching tooltip. Separate that belongs to the table but is unavailable for one row remains in its
destructive actions visually, and use a named confirmation/review surface normal position and is disabled; when the reason is not obvious from row
when the consequence cannot be understood from the icon and row context. state, provide it through the central focusable disabled-action explanation.
It does not disappear. Every icon has a translated accessible name and matching tooltip.
Separate destructive actions visually, and use a named confirmation/review
surface when the consequence cannot be understood from the icon and row
context. In an empty editable table, place Add in the same left-most action
slot it occupies in a populated row and reserve the remaining slots so the
column geometry does not move.
- Transient feedback should not shift the workspace. Durable failures, partial - Transient feedback should not shift the workspace. Durable failures, partial
results, and blockers remain attached to the affected item or operation. results, and blockers remain attached to the affected item or operation.
@@ -136,7 +145,12 @@ one.
- Keep the selected object's identity and material status visible while its - Keep the selected object's identity and material status visible while its
detail changes. detail changes.
- Short field help sits with the label. Longer "Why?", policy source, - Every non-self-explanatory field uses the central `FieldLabel`; short field
help sits with that label. A field without `FieldLabel` is an explicit
documented omission whose register names the field, rationale, and accessible
label source. Users may hide inline help markers with their persisted
interface preference; the visible/accessibility label and validation remain.
Longer "Why?", policy source,
diagnostics, or provenance belongs in an expandable area, detail panel, or diagnostics, or provenance belongs in an expandable area, detail panel, or
review step. review step.
- Empty space is not an error. An empty state states what is empty, why that can - Empty space is not an error. An empty state states what is empty, why that can
@@ -199,6 +213,11 @@ Errors should identify the affected object and whether saved state or external
effects may already exist. Never expose raw exception text as the only user effects may already exist. Never expose raw exception text as the only user
message. message.
Use the central dialog, confirmation, alert, and attached-error components for
feedback. `window.alert` and the global `alert` function are prohibited. A
genuinely unavoidable exception requires explicit product-owner authorization
and an entry in the Core alert exception register before implementation.
## State Contract ## State Contract
Every surface implements the states it can reach; it does not render a blank Every surface implements the states it can reach; it does not render a blank
@@ -355,11 +374,18 @@ and appearance.
The Scheduling request surface is the first explicit reference composition for The Scheduling request surface is the first explicit reference composition for
these rules: these rules:
- The `Scheduling requests` page heading owns one `Add` action. - The persistent left panel contains `My scheduling requests` and `Scheduling
- The left panel is shown for the creation view; it is not a second permanent requests for me` as two stacked lists. It preserves list context like mailbox
creation launcher beside the request list. folders but does not invent folders.
- The left pane's `Scheduling requests` heading owns one short `Add` action. It
opens a new request in the right pane without an extra menu or duplicate
launcher.
- The right main pane is the stable view/create/edit surface. Selecting a list
item opens its details; Add opens the same editor composition used for edit.
- `Basic information`, `Calendar integration`, `Candidate slots`, and - `Basic information`, `Calendar integration`, `Candidate slots`, and
`Participants` are logical sections rendered with the central `Card`. `Participants` are logical sections rendered with the central `Card`.
- Privacy and participation behavior is a separate settings `Card`; dependent
number/password fields are disclosed by their central `ToggleSwitch`.
- Candidate slots and participants are row collections rendered with the - Candidate slots and participants are row collections rendered with the
central `DataGrid`, including its stable action column. central `DataGrid`, including its stable action column.
- Calendar integration is a boolean choice rendered with the central - Calendar integration is a boolean choice rendered with the central
@@ -368,6 +394,10 @@ these rules:
ordered row actions. An address-parsing text area is not the ordinary editor; ordered row actions. An address-parsing text area is not the ordinary editor;
parsing pasted address lists belongs only in an explicitly designed bulk parsing pasted address lists belongs only in an explicitly designed bulk
import flow. import flow.
- View mode shows participation statistics and only state-valid quick actions.
Scheduling owns those current domain actions; a future Workflow module may
coordinate them through stable action contracts but is not a runtime
dependency of the surface.
Apply the underlying placement and component rules to equivalent collection and Apply the underlying placement and component rules to equivalent collection and
create/edit surfaces throughout the system; the Scheduling domain names are an create/edit surfaces throughout the system; the Scheduling domain names are an
@@ -414,25 +444,25 @@ reason to infer that a pattern is satisfied.
and does not duplicate a central component. and does not duplicate a central component.
- Behavioral/accessibility evidence is linked from the rollout matrix and issue. - Behavioral/accessibility evidence is linked from the rollout matrix and issue.
- Configured-system help can reach the applicable pattern or reference topic - Configured-system help can reach the applicable pattern or reference topic
when [Docs #15](https://git.add-ideas.de/add-ideas/govoplan-docs/issues/15) when [Docs #15](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/15)
supplies that experience. supplies that experience.
## First Pilot: Campaign ## First Pilot: Campaign
[Campaign #74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74) [Campaign #74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74)
is the first full-domain audit and migration. It should prove patterns before is the first full-domain audit and migration. It should prove patterns before
generic extraction: generic extraction:
- [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) and - [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) and
[#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73): stable, [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73): stable,
accessible preview and attachment-detail overlays accessible preview and attachment-detail overlays
- [#63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63): review - [#63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63): review
stages, outcomes, blockers, and intervention vocabulary stages, outcomes, blockers, and intervention vocabulary
- [#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62): explicit - [#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62): explicit
synchronous/asynchronous send mode and durable delivery progress synchronous/asynchronous send mode and durable delivery progress
- [#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65): one - [#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65): one
coherent report filtering and count-affordance model coherent report filtering and count-affordance model
- [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35): guided - [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35): guided
first-campaign entry first-campaign entry
These slices do not depend on the Workflow runtime. Campaign's current These slices do not depend on the Workflow runtime. Campaign's current
+54 -47
View File
@@ -9,7 +9,13 @@ The applicable design contract is
## Snapshot And Method ## Snapshot And Method
Snapshot date: 2026-07-20. The source-derived inventory command is documented in
[`PLATFORM_CONTROL_PLANE.md`](PLATFORM_CONTROL_PLANE.md). It produces
machine-readable field, label, translation, route, API-reference, and module
manifest evidence. This hand-maintained document remains the reviewed product
interpretation and rollout ledger; generated evidence does not replace it.
Snapshot refreshed: 2026-07-22.
Evidence was read from tracked Git `HEAD` in the local GovOPlaN checkouts: Evidence was read from tracked Git `HEAD` in the local GovOPlaN checkouts:
@@ -20,10 +26,10 @@ Evidence was read from tracked Git `HEAD` in the local GovOPlaN checkouts:
- Campaign nested routes in its tracked `CampaignWorkspace.tsx` and - Campaign nested routes in its tracked `CampaignWorkspace.tsx` and
`SectionSidebar.tsx` `SectionSidebar.tsx`
Tracked `HEAD` is used as the integrated baseline. Dirty worktree changes are Tracked commits are used as the integrated baseline. Dirty worktree changes
not treated as delivered behavior. One material exception is called out in the are not treated as delivered behavior. The former Campaign recipient-editor
Campaign inventory: local WIP folds the tracked `recipient-data` surface into and preview WIP is integrated in tracked commits; completed work is no longer
`recipients`, but that change is not yet part of the baseline. described as a local exception.
Runtime visibility remains conditional on the module being packaged and Runtime visibility remains conditional on the module being packaged and
enabled, server metadata, installed optional capabilities, the authenticated enabled, server metadata, installed optional capabilities, the authenticated
@@ -59,16 +65,16 @@ The access guard column reports only the route-level declaration in
| Route | Owner / render evidence | Route-level access evidence | Primary task | Target archetype | Status / priority | | Route | Owner / render evidence | Route-level access evidence | Primary task | Target archetype | Status / priority |
| --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- |
| `/admin` | `govoplan-access` `AdminPage` | Any core `adminReadScopes` | Administer system and tenant concerns assembled from module sections | Administration/configuration | Contributed; unreviewed; P1 under [Core #225](https://git.add-ideas.de/add-ideas/govoplan-core/issues/225) | | `/admin` | `govoplan-access` `AdminPage` | Any core `adminReadScopes` | Administer system and tenant concerns assembled from module sections | Administration/configuration | Contributed; unreviewed; P1 under [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225) |
| `/address-book` | `govoplan-addresses` `AddressBookPage` | `addresses:contact:read` | Browse and manage contacts, address books, and lists | Directory/list-detail | Contributed; unreviewed; P2 after Campaign | | `/address-book` | `govoplan-addresses` `AddressBookPage` | `addresses:contact:read` | Browse and manage contacts, address books, and lists | Directory/list-detail | Contributed; unreviewed; P2 after Campaign |
| `/calendar` | `govoplan-calendar` `CalendarPage` | `calendar:event:read` | Browse calendars/events and act on calendar data | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign | | `/calendar` | `govoplan-calendar` `CalendarPage` | `calendar:event:read` | Browse calendars/events and act on calendar data | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/campaigns` | `govoplan-campaign` `CampaignListPage` | `campaigns:campaign:read` | Find, compare, create, and open campaigns | List-detail entry | Pilot; P1 [Campaign #74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74) | | `/campaigns` | `govoplan-campaign` `CampaignListPage` | `campaigns:campaign:read` | Find, compare, create, and open campaigns | List-detail entry | Pilot; P1 [Campaign #74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74) |
| `/campaigns/:campaignId/*` | `govoplan-campaign` `CampaignResourceRoute` and `CampaignWorkspace` | `campaigns:campaign:read`, plus resource probe | Configure, review, send, and inspect one campaign/version | List-detail workspace containing edit, review, monitoring, and evidence surfaces | Pilot; P1 Campaign #74 | | `/campaigns/:campaignId/*` | `govoplan-campaign` `CampaignResourceRoute` and `CampaignWorkspace` | `campaigns:campaign:read`, plus resource probe | Configure, review, send, and inspect one campaign/version | List-detail workspace containing edit, review, monitoring, and evidence surfaces | Pilot; P1 Campaign #74 |
| `/operator` | `govoplan-campaign` `OperatorQueuePage` | Any campaign queue/review/send scope declared by the module | Monitor and intervene in campaign jobs | Monitoring/work queue | Pilot; P1 Campaign #74 | | `/operator` | `govoplan-campaign` `OperatorQueuePage` | `campaigns:campaign:read` and any of queue, control, retry, or reconcile | Monitor and intervene in campaign jobs through authority-specific controls | Monitoring/work queue | Pilot; durable queue controls delivered in [Campaign #78](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/78); #74 audit remains |
| `/reports` | `govoplan-campaign` inline `ReportsPage` | `campaigns:report:read` | Reach campaign reporting; current component identifies itself as prepared for later functionality | Reporting | Pilot inventory; implementation maturity explicitly incomplete; P2 after campaign report pilot | | `/reports` | `govoplan-campaign` `AggregateReportsPage` | `campaigns:report:read` | Compare privacy-protected cross-campaign outcome totals without recipient detail, diagnostics, export, or drill-down | Aggregate reporting | Pilot; aggregate-reader surface delivered in [Campaign #80](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/80); #74 audit remains |
| `/templates` | `govoplan-campaign` `TemplatesPage` | No route guard declared in `module.ts` | Browse/manage campaign templates | Directory/list-detail | Pilot audit; permission intent must be verified; P2 | | `/templates` | `govoplan-campaign` `TemplatesPage` | No route guard declared in `module.ts` | Browse/manage campaign templates | Directory/list-detail | Pilot audit; permission intent must be verified; P2 |
| `/dashboard` | `govoplan-dashboard` `DashboardPage` | No route-specific scope | Assemble module-provided actionable widgets | Dashboard | Contributed; unreviewed; P2 | | `/dashboard` | `govoplan-dashboard` `DashboardPage` | No route-specific scope | Assemble module-provided actionable widgets | Dashboard | Contributed; unreviewed; P2 |
| `/docs` | `govoplan-docs` `DocsPage` | Docs read or system/tenant settings read scopes | Read configured, available, and evidence-aware documentation | Documentation directory/reference | Contributed; unreviewed; P1 [Docs #15](https://git.add-ideas.de/add-ideas/govoplan-docs/issues/15) after initial pattern content | | `/docs` | `govoplan-docs` `DocsPage` | Docs read or system/tenant settings read scopes | Read configured, available, and evidence-aware documentation | Documentation directory/reference | Contributed; unreviewed; P1 [Docs #15](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/15) after initial pattern content |
| `/files` | `govoplan-files` `FilesPage` | `files:file:read` | Browse folders/files and perform managed-file work | Directory/explorer | Contributed; metadata gap; unreviewed; P2 after Campaign | | `/files` | `govoplan-files` `FilesPage` | `files:file:read` | Browse folders/files and perform managed-file work | Directory/explorer | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/idm` | `govoplan-idm` `IdmPage` | Any IDM assignment/write or organization function-assign scope | Inspect and govern identity/function assignments | List-detail/configuration | Contributed; unreviewed; P2 | | `/idm` | `govoplan-idm` `IdmPage` | Any IDM assignment/write or organization function-assign scope | Inspect and govern identity/function assignments | List-detail/configuration | Contributed; unreviewed; P2 |
| `/mail` | `govoplan-mail` `MailboxPage` | `mail:mailbox:read` | Browse mailboxes and messages | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign | | `/mail` | `govoplan-mail` `MailboxPage` | `mail:mailbox:read` | Browse mailboxes and messages | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign |
@@ -137,41 +143,38 @@ above; they are not independent routes.
## Campaign Pilot Surface Map ## Campaign Pilot Surface Map
Campaign is detailed first because it exercises almost every archetype. The Campaign is detailed first because it exercises almost every archetype. The
tracked baseline still contains a dedicated `recipient-data` section. The dirty recipient-data editor is now consolidated into the `recipients` section on
local WIP redirects that path into `recipients` and removes it from the sidebar; remote `main`; [Campaign #67](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/67)
do not mark that consolidation delivered until records the accepted and verified integration boundary.
[Campaign #67](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/67)
is integrated or the issue state is corrected.
Campaign already consumes core primitives including `ModuleSubnav`, `Card`, Campaign already consumes core primitives including `ModuleSubnav`, `Card`,
`PageTitle`, `Button`, `LoadingFrame`, `DismissibleAlert`, `FormField`, `PageTitle`, `Button`, `LoadingFrame`, `DismissibleAlert`, `FormField`,
`StatusBadge`, `MetricCard`, `Dialog`, `ConfirmDialog`, `FileDropZone`, `StatusBadge`, `MetricCard`, `DataGrid`, `TableActionGroup`, `Dialog`,
`MessageDisplayPanel`, policy components, access/module capabilities, and `ConfirmDialog`, `FileDropZone`, `MessageDisplayPanel`, policy components,
unsaved-navigation guards. Reuse alone does not prove that the composition or access/module capabilities, and unsaved-navigation guards. Reuse alone does not
states satisfy the pattern. prove that the composition or states satisfy the pattern.
| Surface / code evidence | Primary task | Target pattern | Material consequence/state | Known issue / rollout | | Surface / code evidence | Primary task | Target pattern | Material consequence/state | Known issue / rollout |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | Audit in [#74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74); guided entry [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35) | | Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | Audit in [#74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74); guided entry [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) |
| Overview (`CampaignOverviewPage`) | Understand/edit campaign identity, version, access, lifecycle | Object overview plus adaptive edit | Lock/archive/delete/access changes need real consequence and reversibility wording | #74; integrate/reconcile #67 before migration claims | | Overview (`CampaignOverviewPage`) | Understand/edit campaign identity, version, access, lifecycle | Object overview plus adaptive edit | Lock/archive/delete/access changes need real consequence and reversibility wording | #74 remaining audit |
| Fields (`CampaignFieldsPage`) | Define recipient/template field schema | Structured editor | Schema changes can invalidate recipient/template data | #74 audit | | Fields (`CampaignFieldsPage`) | Define recipient/template field schema | Structured editor | Schema changes can invalidate recipient/template data | #74 audit |
| Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74; attachment-detail [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) | | Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74; attachment-detail [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) |
| Recipients (`RecipientDataPage`) | Select/import/map/edit recipients and per-recipient values/files | Import/mapping plus list-detail editor | Personal data, validation, bulk activation, file links | #74; guided entry #35 | | Recipients (`RecipientDataPage`) | Select/import/map/edit recipients, address fields and per-recipient values/files | Import/mapping plus list-detail editor | Personal data, validation, bulk activation, file links | Consolidated editor delivered in #67; #74 remaining audit and guided entry #35 |
| Recipient data (`RecipientDetailsPage` in tracked baseline) | Inspect/edit a dedicated recipient-data view | List-detail editor | Personal data and send eligibility | Local WIP removes this separate surface; #67 integration state must decide baseline | | Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74; stable overlay [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) |
| Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74; stable overlay [#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73) |
| Mail settings (`MailSettingsPage` settings view) | Select/configure campaign mail transport | Adaptive configuration | Credentials, SMTP/IMAP destinations, test outcomes | #74; align with Core #225 mail pattern | | Mail settings (`MailSettingsPage` settings view) | Select/configure campaign mail transport | Adaptive configuration | Credentials, SMTP/IMAP destinations, test outcomes | #74; align with Core #225 mail pattern |
| Campaign settings (`GlobalSettingsPage` settings view) | Configure campaign behavior | Adaptive configuration | Can alter validation/build/send behavior | #74 audit | | Campaign settings (`GlobalSettingsPage` settings view) | Configure campaign behavior | Adaptive configuration | Can alter validation/build/send behavior | #74 audit |
| Mail policy (`MailSettingsPage` policy view) | Inspect/override effective mail policy | Effective policy/provenance editor | Inheritance and locks affect allowed delivery | #74; Core #225 policy pattern | | Mail policy (`MailSettingsPage` policy view) | Inspect/override effective mail policy | Effective policy/provenance editor | Inheritance and locks affect allowed delivery | #74; Core #225 policy pattern |
| Campaign policy (`GlobalSettingsPage` policy view) | Inspect/override campaign policy | Effective policy/provenance editor | Inheritance, actor authority, and blocked edits | #74; Core #225 policy pattern | | Campaign policy (`GlobalSettingsPage` policy view) | Inspect/override campaign policy | Effective policy/provenance editor | Inheritance, actor authority, and blocked edits | #74; Core #225 policy pattern |
| Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, partial effects, retries, evidence | Intervention vocabulary [#63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63); sync/async progress [#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62) | | Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, bounded synchronous execution, persisted queue mode, partial effects, retries, evidence | Bounded synchronous and explicit/persisted queued modes delivered in [#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62) and [#79](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/79); [#63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63) wording and #74 audit remain |
| Message and attachment detail overlays | Inspect one built/mock message and its attachment links | Stable detail/review dialog | Personal data, exact outbound content, reviewed state | Implement/verify [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) and [#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73) together | | Message and attachment detail overlays | Inspect one built/mock message and its attachment links | Stable detail/review dialog | Personal data, exact outbound content, reviewed state | Delivered and verified in [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) and [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) |
| Campaign report (`CampaignReportPage`) | Filter and inspect delivery outcomes | Reporting/list-detail | Partial, failed, skipped, SMTP/IMAP outcomes and retries | Filtering [#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65), dependent on [Core #263](https://git.add-ideas.de/add-ideas/govoplan-core/issues/263) | | Campaign report (`CampaignReportPage`) | Filter and inspect delivery outcomes | Reporting/list-detail | Partial, failed, explicitly excluded/skipped, SMTP/IMAP outcomes and retries | Server-owned filtering and counts delivered in [#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65) with the full-result DataGrid contract from [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263); excluded semantics in [#66](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/66) |
| Audit (`CampaignAuditPage`) | Inspect campaign evidence/history | Provenance timeline/report | Actor/action/effect trace | #74 audit | | Audit (`CampaignAuditPage`) | Inspect campaign evidence/history | Provenance timeline/report | Actor/action/effect trace | #74 audit |
| JSON (`CampaignJsonView`) | Inspect expert representation | Advanced diagnostics/reference | Raw data may contain personal/configuration values; not a primary editor | #74 privacy/redaction audit | | JSON (`CampaignJsonView`) | Inspect expert representation | Advanced diagnostics/reference | Raw data may contain personal/configuration values; not a primary editor | #74 privacy/redaction audit |
| Create wizard (`CreateWizard`) | Seed a campaign through basics, sender, fields, recipients, template, attachments, review, send | Guided setup | Current steps mix creation and later consequential delivery; completion semantics need audit | Guided first campaign [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35) | | Create wizard (`CreateWizard`) | Seed a campaign through basics, sender, fields, recipients, template, attachments, review, send | Guided setup | Current steps mix creation and later consequential delivery; completion semantics need audit | Guided first campaign [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) |
| Review/send wizard routes | Alternate guided review/send shells | Guided review | Tracked routes exist; implementation relationship to `ReviewSendPage` must be established, not guessed | #74 inventory decision | | Review/send wizard routes | Alternate guided review/send shells | Guided review | Tracked routes exist; implementation relationship to `ReviewSendPage` must be established, not guessed | #74 inventory decision |
| Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Retry/pause/reconcile and system actor/effect evidence | #74, then #62 vocabulary | | Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Campaign/version/job identity, historical active-version discovery, fixed action positions, authority-aware disabled states, exact non-overlapping queue counts, server-paged jobs, bounded refresh, retry/queue/reconcile per version, campaign-wide pause/resume/cancel, and leave/return progress | Durable operator controls delivered in [#78](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/78); #74 wording/accessibility audit remains |
| Top-level reports placeholder | Enter cross-campaign reports | Reporting | Component explicitly says functionality is prepared for later passes | Keep marked incomplete; do not infer parity with per-campaign report | | Aggregate reports (`AggregateReportsPage`) | Compare cross-campaign delivery outcomes | Privacy-preserving aggregate reporting | Tenant/campaign ACL, deployment/tenant small-cell policy, complementary and overlapping-cell suppression, explicit denominator, and no recipient detail/diagnostics/export/drill-down | Separate aggregate-reader surface delivered in [#80](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/80); not parity with the permission-gated per-campaign detail report |
| Templates route (`TemplatesPage`) | Browse template records | Directory/list-detail | Template availability and later generated outputs | #74 audit; verify missing route guard intent | | Templates route (`TemplatesPage`) | Browse template records | Directory/list-detail | Template availability and later generated outputs | #74 audit; verify missing route guard intent |
The five review stages currently named in code are `Validate and inspect`, The five review stages currently named in code are `Validate and inspect`,
@@ -189,7 +192,7 @@ The following local repositories contain a backend manifest but no
`govoplan-contracts`, `govoplan-dist-lists`, `govoplan-evaluation`, `govoplan-contracts`, `govoplan-dist-lists`, `govoplan-evaluation`,
`govoplan-facilities`, `govoplan-forms-runtime`, `govoplan-grants`, `govoplan-facilities`, `govoplan-forms-runtime`, `govoplan-grants`,
`govoplan-helpdesk`, `govoplan-identity`, `govoplan-inspections`, `govoplan-helpdesk`, `govoplan-identity`, `govoplan-inspections`,
`govoplan-issue-reporting`, `govoplan-learning`, `govoplan-permits`, `govoplan-tickets`, `govoplan-learning`, `govoplan-permits`,
`govoplan-poll`, `govoplan-procurement`, `govoplan-records`, `govoplan-poll`, `govoplan-procurement`, `govoplan-records`,
`govoplan-resources`, `govoplan-rest`, `govoplan-risk-compliance`, `govoplan-resources`, `govoplan-rest`, `govoplan-risk-compliance`,
`govoplan-soap`, `govoplan-tenancy`, and `govoplan-transparency`. `govoplan-soap`, `govoplan-tenancy`, and `govoplan-transparency`.
@@ -202,23 +205,27 @@ backend-only modules may remain intentionally headless.
| Order | Scope | Current evidence | Target | Owner / issue | Verification gate | Status | | Order | Scope | Current evidence | Target | Owner / issue | Verification gate | Status |
| --- | --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- | --- |
| 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/add-ideas/govoplan/issues/11) | Docs links/diff checks; issue/wiki sync after integration | Initial slice in this document | | 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) | Docs links/diff checks; issue/wiki sync after integration | Initial slice in this document |
| 1 | Campaign baseline integration | Closed work and local WIP can differ from tracked baseline | Integrated, testable baseline before migration claims | Campaign #67 and tracker cleanup | Clean scoped diff, behavior tests, build, issue evidence | Must precede claiming Campaign migration complete | | 1 | Campaign baseline integration | Recipient-editor WIP and tracker state have been reconciled with remote `main` | Integrated, testable baseline before migration claims | Campaign #67 and tracker cleanup | Backend and focused WebUI suites; issue evidence | Complete 2026-07-22 |
| 2 | Campaign previews/details | Existing message/attachment overlays and known overflow/geometry issues | Stable header/body/footer, accessible long-content detail | Campaign #59 and #73 | Keyboard/focus/scroll behavior, long content, responsive and translated text | P1 ready | | 2 | Campaign previews/details | Stable shared dialog with bounded scrolling and fixed responsive preview workspace | Stable header/body/footer, accessible long-content detail | Campaign #59 and #73 | Review-preview and overlay structure tests | Complete 2026-07-22 |
| 3 | Campaign review/interventions | Five domain-owned stages with unresolved intervention language | Clear stages, outcomes, blockers, next actor/action, reviewed evidence | Campaign #63 | State matrix behavior/accessibility tests and agreed vocabulary | P1 needs product wording decision | | 3 | Campaign review/interventions | Five domain-owned stages with unresolved intervention language | Clear stages, outcomes, blockers, next actor/action, reviewed evidence | Campaign #63 | State matrix behavior/accessibility tests and agreed vocabulary | P1 needs product wording decision |
| 4 | Campaign send/progress | Synchronous/asynchronous behavior is not explicit enough | Pre-send mode/consequence plus durable leave/return progress, retry and reconciliation | Campaign #62 | Sync and async behavior, partial/failure/retry, reload/return tests | P1 ready after stage vocabulary | | 4 | Campaign send/progress | A hard deployment ceiling bounds synchronous delivery; the selected synchronous, worker-queue, or database-queue mode is explicit and persisted; progress and recovery survive navigation; immediate-send response and audit evidence are allowlisted | Pre-send mode/consequence plus durable leave/return progress, retry and reconciliation without recipient/provider leakage | Campaign #62 and #79 | Boundary/concurrency/preflight, async selection, persisted mode, sanitized response/audit, partial/failure/retry and reload/return tests | Complete 2026-07-22 (`7e16603`, `60efd1c`, `62a6879`, `b0282eb`, `f095a3e`) |
| 5 | Campaign report filtering | Duplicate/inconsistent filter models are reported | One shared status/list/filter/count model | Campaign #65 and Core #263 | Query/filter behavior, keyboard, counts, large result set | P1 blocked on shared DataGrid direction | | 5 | Campaign report filtering | Core DataGrid distinguishes client/full-result from server-owned queries; Campaign applies filter/sort/count before pagination and synchronizes count shortcuts with the grid query | One shared server-owned status/list/filter/count model | Campaign #65 and Core #263 | DataGrid contract/build tests plus exact shortcut/query/filter/count and large-result behavior | Complete 2026-07-22 (`e6062fe`, `cece71d`, `aa4ec66`, `4eb651c`) |
| 6 | Guided first campaign | Existing wizard routes and ordinary workspace overlap | Task-oriented entry that hands off clearly to normal editing/review | Campaign #35 | First-run flow, resume/back, validation, optional modules, no implicit send | P1 after core pilot patterns stabilize | | 6 | Campaign operator recovery | A durable campaign/version queue page exposes historical work, exact non-overlapping state counts, persisted mode, permission-safe controls, server-paged job evidence, bounded refresh and active-state recovery | Fixed-position actions, disabled explanations, leave/return state, version-scoped retry/queue/reconcile and explicit campaign-wide pause/resume/cancel | Campaign #78 | Queue model/structure, historical-version, permission, paging, recovery-control, stale-response and delta tests | Complete 2026-07-22 (`21f3014`, `99d44ee`, `735e874`) |
| 7 | Prove/extract generic primitives | Core already exports many primitives; Campaign composition still unreviewed | Extract only contracts with a second consumer or clear platform ownership | Core #225 plus bounded follow-ups | Core behavior/accessibility tests and module-permutation tests | After Campaign proof | | 7 | Campaign aggregate reports | A separate aggregate-reader projection and UI expose only policy-suppressed business totals with a stable status domain | Explicit denominator and exclusions, deployment floor plus tenant-strengthened small-cell threshold, complementary and overlapping-cell suppression, no detail/export/diagnostics | Campaign #80 | Aggregate query, cross-metric suppression, route/role/ACL, stable filter and UI structure tests | Complete 2026-07-22 (`06125cc`, `fc36aee`, `8ee87b7`, `ac3329c`, `1225802`) |
| 8 | Configured-system pattern help | Docs route and classification exist | Role/config-aware pattern and route/field/blocker help | Docs #15 | Topic grouping, audience filtering, stable links/anchors | P1 after initial pattern IDs stabilize | | 8 | Campaign excluded outcomes | Excluded build rows become explicit skipped transport outcomes and remain protected from queue/cancel/retry ambiguity | One durable source-to-job-to-report meaning with guarded historical normalization | Campaign #66 | Builder/persistence, migration, query/count, queue-control and report-explanation tests | Complete 2026-07-22 (`7229fb8`) |
| 9 | Admin/configuration family | Phase inventory and connector primitives exist in the ledger | Apply the pattern to files, mail, policy, retention, packages, modules, API keys, settings | Core #225 and module children | Per-surface state/accessibility/consequence evidence | Parallel where independent of Campaign shared decisions | | 9 | Guided first campaign | Existing wizard routes and ordinary workspace overlap | Task-oriented entry that hands off clearly to normal editing/review | Campaign #35 | First-run flow, resume/back, validation, optional modules, no implicit send | P1 after core pilot patterns stabilize |
| 10 | Remaining direct routes | Routes are contributed; most are unreviewed | Per-module bounded audit and migration plan | New module issues derived from this inventory | Applicable definition-of-done gates | P2 after Campaign, not a bulk rewrite | | 10 | Prove/extract generic primitives | Core already exports many primitives; Campaign composition still unreviewed | Extract only contracts with a second consumer or clear platform ownership | Core #225 plus bounded follow-ups | Core behavior/accessibility tests and module-permutation tests | After Campaign proof |
| 11 | Manifest/runtime alignment | Several executable routes are absent from manifest metadata | Declared alignment or explicit validated exception | Core contract issue to create | Automated manifest/module route check and configured Docs verification | Discovery follow-up | | 11 | Configured-system pattern help | Docs route and classification exist | Role/config-aware pattern and route/field/blocker help | Docs #15 | Topic grouping, audience filtering, stable links/anchors | P1 after initial pattern IDs stabilize |
| 12 | Admin/configuration family | Phase inventory and connector primitives exist in the ledger | Apply the pattern to files, mail, policy, retention, packages, modules, API keys, settings | Core #225 and module children | Per-surface state/accessibility/consequence evidence | Parallel where independent of Campaign shared decisions |
| 13 | Remaining direct routes | Routes are contributed; most are unreviewed | Per-module bounded audit and migration plan | New module issues derived from this inventory | Applicable definition-of-done gates | P2 after Campaign, not a bulk rewrite |
| 14 | Manifest/runtime alignment | Several executable routes are absent from manifest metadata | Declared alignment or explicit validated exception | Core contract issue to create | Automated manifest/module route check and configured Docs verification | Discovery follow-up |
Workflow/user-story implementation is postponed. It is not on the critical path Workflow remains outside this rollout matrix because it has its own runtime and
for rows 0 through 10. Focused views can be specified, manually selected, and editor workstream, not because it is postponed. Focused views can be specified,
tested through core composition contracts; a later workflow step may become one manually selected, and tested through core composition contracts today.
activation source without changing the proven surface patterns. Workflow steps may activate those views through the same contract without
changing the proven surface patterns.
## Inventory Maintenance ## Inventory Maintenance
+4 -1
View File
@@ -114,7 +114,10 @@ For release validation:
```sh ```sh
./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python ./.venv/bin/python ./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python ./.venv/bin/python
./.venv/bin/python -m pip install -r requirements-release-tests.txt
``` ```
That install is necessary because the release environment intentionally resolves That install is necessary because the release environment intentionally resolves
tagged package refs, not local editable source trees. tagged package refs, not local editable source trees. The second requirements
file contains only the harness needed to execute tests from those immutable
source tags; it is not part of the deployable release dependency set.
+126
View File
@@ -0,0 +1,126 @@
# Platform Control Plane And Self-Description
## Objective
GovOPlaN should be able to describe its installed structure without becoming a
self-modifying application. The platform model is a declarative control plane:
module manifests, UI contributions, schemas, policy provenance, runtime
capabilities, and generated source evidence describe what can be configured.
Ordinary administrators edit validated data through those contracts; they do
not edit Python, TypeScript, routes, or database code from the product UI.
This distinction provides the requested overview while preserving reviewable
releases, module boundaries, migrations, and security controls.
## Canonical Sources
| Concern | Canonical source |
| --- | --- |
| Installed modules and dependency graph | Runtime `ModuleManifest` registry |
| Backend routes | Registered FastAPI application; Python AST is build-time evidence |
| Frontend routes and navigation | `PlatformWebModule` contributions |
| View-filterable regions | Versioned `viewSurfaces` declarations |
| Admin sections and module settings | `admin.sections`, including `moduleId`, `kind`, scope group, permission guards, and surface ID |
| User settings | `settings.sections` and core settings schemas |
| Labels and translations | Generated translation catalogs plus source usage |
| Fields and help coverage | Shared form components plus generated TypeScript AST inventory |
| API use by the WebUI | Typed API clients plus generated static reference inventory |
| Effective configuration | Owning module data plus Policy provenance |
Runtime introspection is authoritative for an installed system. Static source
inventory is authoritative evidence for a checkout or release candidate. The
two should be compared in CI and by Ops, not conflated.
## Generated Inventory
Run:
```bash
cd /mnt/DATA/git/govoplan
./.venv/bin/python tools/inventory/platform-interface-inventory.py
```
The command writes:
- `audit-reports/platform-inventory/platform-interface-inventory.json`
- `audit-reports/platform-inventory/platform-interface-inventory.md`
It combines:
1. loaded module manifests
2. TypeScript AST extraction of fields, label attributes, visible text,
translations, frontend routes, navigation, capabilities, and API references
3. Python AST extraction of FastAPI route decorators and router prefixes
The JSON includes exact repository, file, and line evidence. A missing-help
entry is a review candidate because dynamic parent components may supply help.
A backend route without a static frontend reference is also a review candidate:
public APIs, workers, callbacks, health checks, connectors, and dynamic URL
assembly are valid explanations.
`--strict` currently enforces only translation-catalog completeness. Endpoint
and help classifications need narrow reviewed baselines before they can become
release gates.
## Admin Information Architecture
The Admin host uses a tree because system, tenant, group, user, and module
settings form a hierarchy rather than one flat list. Every contributed section
can identify:
- its owning `moduleId`
- whether it is `management` or `settings`
- its system/tenant/group/user scope group
- an optional future `parentId`
- permission and View visibility requirements
Existing panels remain their own render owners. The tree only changes discovery
and grouping. A later embedded-settings contract may add named slots inside an
owning page; it must not allow one module to import another module's private
component.
## Navigation And Workflow
The intended maximum visible navigation stack is:
1. global shell context
2. one task/object navigation surface
3. one workflow stage surface when a workflow is active
Workflow instance pages should reuse the Campaign stage language: clear stage
state, optional/skipped/blocked semantics, partial progress, and a stable current
step. Workflow definition pages remain graph editors. Views may activate a
focused workflow view that suppresses unrelated shell and module surfaces while
retaining an explicit way out.
Nested module submenus should not be added merely because a data hierarchy
exists. Prefer a tree inside configuration/directory surfaces, tabs for sibling
views, and the workflow stage rail for ordered work.
## Safe Meta-Configuration
The platform can eventually render many configuration editors from versioned
JSON Schema and UI Schema supplied by modules. Generated editors remain bounded
by:
- explicit typed schemas and migrations
- module-owned validation and preview
- Policy locks and provenance
- permission and View filtering
- preflight, consequence, and rollback information
- auditable apply operations
Custom code, new routes, arbitrary SQL, and executable workflow nodes remain
release artifacts. Modeling them as ordinary configuration would create an
unreviewed code-execution and migration channel.
## Next Enforcement Slices
1. Require every WebUI module route and admin/settings contribution to have
matching manifest metadata or a reviewed exception.
2. Add stable field IDs and optional help-topic IDs to shared field components.
3. Classify each statically unreferenced backend endpoint by consumer type.
4. Compare a running installation's OpenAPI and module registry against the
release inventory.
5. Publish the sanitized installed-system structure through Ops/Docs for
authorized administrators.
+147
View File
@@ -0,0 +1,147 @@
# Recovery And Rollback Guarantees
## Principle
GovOPlaN must prove recovery claims with durable state recorded before and
after side effects. A failed operation is not automatically rolled back merely
because the previous application image still exists. Database schema and
external effects may make release rollback unsafe.
Core therefore distinguishes five recovery modes:
| Mode | Meaning |
| --- | --- |
| `atomic` | One database transaction either commits or rolls back. No external effect is claimed. |
| `compensation` | Durable evidence identifies explicit inverse actions for completed effects. |
| `snapshot_restore` | A separately verified backup reference and restore procedure exist. |
| `forward_recovery` | Repair or resume the current version; reverting code/configuration is not claimed safe. |
| `irreversible` | No automated recovery is claimed and an approval reference is mandatory. |
An operation plan must include verification steps. Compensation requires named
compensation steps, snapshot restore requires a verified backup reference,
forward recovery requires repair steps, and irreversible work requires explicit
approval.
## Core Recovery Ledger
Core stores recovery operations and append-only, hash-chained checkpoints in
PostgreSQL. The contract provides:
- installation/module/resource identity;
- an idempotency key bound to a canonical request hash;
- recovery mode, preconditions, verification steps, and references;
- optional runtime lease holder and fencing token;
- explicit planned, prepared, running, recovery-required, recovering,
succeeded, recovered, failed, outcome-unknown, and manual-intervention states;
- an evidence-chain head and sequence count;
- rejection of plaintext secrets in metadata or evidence.
Preparation cannot succeed without durable precondition evidence. A non-atomic
operation cannot hide a partial effect by transitioning directly from running
to failed. Success and recovery require explicit verification evidence with at
least one check. The ledger verifies its hash chain before evidence is trusted.
This is a platform contract, not an assertion that every existing module
operation has adopted it. Module operations with external or multi-resource
effects must be migrated to the ledger before claiming these guarantees.
## Deployment Journal
Every `govoplan-deploy apply` begins an operation journal before it pulls images
or mutates runtime state. The private installation directory records:
```text
operations/<operation-id>/operation.json
operations/<operation-id>/before/
applied-state/
```
Each stage is hash-chained. The previous applied bundle is copied with per-file
SHA-256 evidence. Applied state is replaced atomically after health verification;
an interrupted replacement restores its previous directory.
New journals also bind the complete desired deployment plan, snapshot
availability, failure summary, recovery mode, and terminal status to the
evidence chain. Recovery verifies every snapshot entry and checksum before it
changes any live bundle file, then replaces each live file atomically. A crash
between file replacements is recoverable by rerunning the same idempotent
recovery command under the deployment lock.
Inspect operations:
```sh
python tools/deployment/govoplan-deploy.py operations \
--directory /srv/govoplan/default
```
Recover the latest failed operation, or provide its identifier:
```sh
python tools/deployment/govoplan-deploy.py recover \
--directory /srv/govoplan/default \
--operation-id 20260801T120000Z-1234abcd
```
Add `--apply` only after reviewing the reported action.
## Migration Boundary
Before database migration starts, a failed deployment with a verified prior
applied snapshot may restore its prior release/configuration bundle and
reconcile that desired state.
As soon as migration starts, the journal permanently changes to
`forward_recovery`. It will not restore old application configuration because
old code may not understand the new schema. Recovery then means one of:
1. fix and re-run the current release;
2. deploy a newer compatible repair release;
3. restore a separately verified, coordinated database/object/key backup and
then deploy the matching release.
The deployment tool does not create or validate that database backup. A
`backup-required` annotation on the Kubernetes migration Job is an operator
gate, not backup evidence. Production automation must provide a backup hook or
external backup controller whose artifact, timestamp, scope, encryption key,
and restore test can be referenced from the recovery record.
## Scaled Nodes
Recovery actions must be safe across replicas:
- drain affected API and worker nodes before incompatible changes;
- use the deployment-wide PostgreSQL advisory lock for schema migration;
- use distributed leases and fencing tokens for singleton or externally
visible effects;
- use idempotency keys for retried commands and jobs;
- retain shared object keys and database references until deletion succeeds;
- classify uncertain external outcomes instead of retrying blindly;
- verify the exact software/module composition after replacement.
Campaign generated-message objects now follow this model: object writes are
compensated when a build fails before database commit, workers verify stored
size and digest before delivery, and retention keeps the database reference
when storage deletion fails. A hard process loss between object creation and
database commit can still leave an orphan object; an inventory reconciler is a
separate operational slice and must use the build-specific object prefix.
## Required Drills
Record evidence for at least these scenarios before production acceptance:
1. Kill an API replica and verify traffic continues without session loss.
2. Drain and replace a worker while work is queued and while one job is active.
3. Start two migration jobs and verify only one mutates schema.
4. Kill the fenced scheduler and verify one replacement acquires a higher
fencing token.
5. Fail deployment before migration and restore the prior applied bundle.
6. Fail deployment after migration and verify old configuration is not
restored.
7. Restore PostgreSQL, object storage, and encryption keys to one coordinated
recovery point and verify representative object hashes.
8. Interrupt object storage during Campaign build and retention and verify
compensation/reference-preservation behavior.
9. Tamper with a deployment or Core recovery checkpoint and verify chain
validation rejects it.
No runbook, status badge, or green health endpoint substitutes for a dated,
repeatable restore drill against the actual deployment topology.
+460
View File
@@ -0,0 +1,460 @@
# GovOPlaN Reference Journey Program
## Status
This is the selected product-development sequence, originally chosen on
2026-07-21 and reconciled with the implemented platform on 2026-07-31. It turns
the long-term connected-platform roadmap into five demonstrable journeys.
Workflow Engine and the optional Workflow editor now exist, but they are used
by a stage only when its package explicitly composes and proves them; Workflow
is not an automatic dependency of every journey.
The institutional semantics and source-authority model applied to these stages
are defined in the
[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
The stages are ordered, but they are not monolithic releases. Each stage is
delivered as small, reviewable, green increments and is complete only when its
user journey, failure behavior, documentation, and operator evidence work in a
pinned composition.
## Why this sequence
The sequence grows one connected product rather than advancing repositories in
isolation:
```text
Campaign reference
-> function-bound Postbox delivery
-> data-backed templates and reports
-> governed analytical data products
-> collaborative document lifecycle
```
Each stage reuses the preceding contracts:
- Campaign proves governed recipient selection, files, profiles, preview,
durable delivery, retry, reconciliation, audit, and role-specific UI.
- Postbox generalizes delivery from an email address to an institutional
responsibility that survives personnel changes.
- Templates and Reporting turn governed source data into reproducible outputs
and provide a safe launch boundary for specialist systems such as HIS.
- The BI path scales those source, transformation, quality, lineage, and report
contracts from one document to reusable analytical data products.
- DMS collaboration turns generated or uploaded artifacts into governed living
documents without confusing storage, editing, approval, or records
ownership.
## Continuous foundation lane
This lane is not a sixth product stage. It advances only as far as the next
journey needs and supplies contracts shared by all five stages.
1. **Governed integrations.** Connector profiles declare endpoint, authority,
source-of-truth mode, capabilities, health, limits, credential references,
and lifecycle. Network destinations are resolved and pinned at connection
time; private-network access is deployment-controlled and unpinned private
transports fail closed. Deleting a connector or provider immediately
deletes every secret it owns and records a non-secret audit event in the
same lifecycle action; an unavailable external secret provider blocks the
deletion or retirement rather than leaving an unaudited orphan.
2. **External effects.** Requested action, durable command, observed effect,
unknown outcome, retry, reconciliation, compensation/correction, and manual
intervention remain distinct and auditable.
3. **Institutional identity.** Identity owns subjects, Organizations owns units
and functions, IDM owns identity-to-function assignments and upstream
synchronization, and Access owns permission and acting-context decisions.
4. **Stable composition.** Modules cooperate through versioned Core-mediated
capabilities and typed references. Optional sibling modules are never
imported as implementation dependencies.
5. **Evidence and provenance.** Every imported datum, generated artifact,
delivery, transformation, and later correction can identify its source,
version, actor or system trigger, policy, and relevant execution evidence.
6. **Focused experience.** Central components and the interface pattern
language are mandatory. Views present only what the current task and actor
need while preserving an explainable escape to the configured full system.
7. **Adaptive documentation.** The owning module contributes canonical subject
matter; GovOPlaN Docs composes it for the installed modules, enabled
configuration, role, route, and task.
8. **Release proof.** Version alignment, clean install/upgrade, module
permutations, backup/restore, degraded-provider behavior, security checks,
and target-environment acceptance gate product claims.
9. **Artifact and infrastructure egress.** Tenant connectors use the pinned
connector boundary. Installer artifacts use a separate deployment-owned
trust path: approved origins, bounded pinned download or operator prefetch,
signature/checksum verification, safe extraction, and offline package-manager
execution. Database, broker, cache, and worker channels are constrained by
deployment network policy and authenticated transport rather than treated as
tenant connector profiles.
## Documentation contract for every reference stage
Documentation is one connected knowledge model rendered through different
perspectives, not separately maintained manuals that drift.
Every demonstrated journey provides:
- **Task guidance:** “I want to …” instructions using visible UI wording.
- **Process view:** actors, inputs, states, decisions, handoffs, exceptions,
completion, correction, and retention.
- **Concept view:** domain language and ownership boundaries.
- **Administrator view:** permissions, policies, profiles, configuration,
module requirements, migrations, and audit effects.
- **Operator view:** health, queues, retries, reconciliation, backup/restore,
incident handling, and target-provider checks.
- **Integration view:** capabilities, DTOs, APIs, events, idempotency,
compatibility, deep-link/launch contracts, and example payloads.
- **Security, privacy, and audit view:** threats, disclosure rules, secrets,
provenance, evidence, retention, and destructive actions.
- **Acceptance view:** runnable examples, expected results, failure injection,
and release gates.
The Docs module selects and links these views according to installed
capabilities and actor context. Feature repositories remain the source of
truth for their behavior; Docs owns indexing, conditions, safe disclosure, and
presentation. Generated API/contract reference may supplement, but never
replace, the maintained behavioral explanation.
## Stage 1: Campaign as the demonstration module
### Product promise
> As a communications team, we can compose a campaign from governed recipient
> data, managed files, reusable mail configuration, and policy; preview the
> exact effect; authorize delivery; and resolve every recipient to a known or
> explicitly uncertain outcome without exposing transport internals to normal
> readers.
Campaign is the first reference composition, not an all-purpose mail client or
contact database.
### Ownership and composition
- Campaign owns campaign intent, content, campaign-local recipient snapshots,
personalization, execution state, and delivery evidence.
- Mail owns reusable profiles, credentials, protocol adapters, profile policy,
send/append execution, and provider-facing diagnostics. Campaign persists
only a stable Mail profile reference, never SMTP/IMAP settings or secrets.
- Files owns stored objects, provider connectors, file policy, and provenance.
Campaign persists stable attachment references and frozen execution evidence.
- Addresses or another directory may supply recipients, but Campaign freezes
the execution-time recipient and source evidence needed to explain the send.
- Policy, Access, Audit, Docs, Notifications, Organizations, Identity, IDM, and
Postbox contribute only through declared capabilities when installed.
### Implementation slices
1. Enforce the Mail-profile-only boundary and reject inline transport material
on authoring, import, update, build, and delivery paths. Legacy records are
preserved but fail closed until explicitly migrated.
2. Finish the central-component UI pass and consistent task-focused author,
reviewer, operator, and reader surfaces. Normal readers receive business
state, not paths, backend keys, worker claims, or raw provider diagnostics.
3. Prove test, single-send, resend, batch, append-to-Sent, retry,
reconciliation, quarantine, and manual intervention semantics against a
target SMTP/IMAP environment.
4. Complete adaptive Campaign, Mail, and Files documentation under the shared
documentation contract, including runnable reference campaigns and failure
drills.
5. Package the composition with sample data, policies, roles, configured views,
preflight, health checks, and pinned compatible versions.
### Gate
A clean installation can run the maintained examples through target providers;
no Campaign payload contains transport credentials; worker/provider failures do
not cause blind duplicate delivery; each supported role sees an appropriate
surface and explanation; install, upgrade, restore, and release evidence are
repeatable.
## Stage 2: Postbox and delivery to institutional responsibility
### Product promise
> As a sender, I can deliver a message to a named function in an organizational
> unit without knowing which individual currently performs it. As the current
> function holder or authorized delegate, I can discover, read, and act on that
> message; when assignments change, access follows current responsibility while
> the delivery and access history remains explainable.
### Ownership and composition
- Postbox owns addressable in-platform postboxes, messages, bindings,
participants, attachment references, access-sensitive events, and retention
inputs.
- Organizations defines units and functions. Identity defines subjects. IDM
assigns identities to functions and reconciles upstream directories. Access
resolves roles, delegations, acting context, and permissions.
- Mail is an optional external channel bridge. A Postbox is not an SMTP/IMAP
account and is never owned by one login credential.
- Campaign consumes a generic delivery-target capability. It may target an
email recipient or a function-bound Postbox without importing either
module's internals.
- A Postbox exists independently of its current holders. Vacancy, one
incumbent, several incumbents, and time-bounded delegation are valid states;
none turns the Postbox into a personal account.
- The E2EE target uses per-content keys and function/postbox key epochs.
Assignment grants device-bound access to policy-selected history, while
hand-over, revocation, and delegation expiry rotate or withdraw future key
access without claiming to erase plaintext already obtained.
- Content and signed manifests are immutable. Correction or replacement is a
linked new object/version, never silent substitution.
### Implementation slices
1. Stabilize Postbox manifest, permissions, DTOs, migrations, directory,
binding administration, access decisions, message create/list/read, and
audit events.
2. Define a typed delivery target and receipt contract for a Postbox id or an
organization-unit/function address. Resolution never copies current holders
into permanent mailbox ownership.
3. Complete the OrganizationsIdentityIDMAccess function-assignment and
delegation path, including provenance and compatibility migration from
legacy projections.
4. Define the E2EE key-grant/epoch profile for vacancy, multiple incumbents,
hand-over, history access, time-bounded delegation, recovery, and audit.
5. Add Campaign delivery to a function-bound Postbox, file attachment
references, optional external-mail bridge, and notification attention
signals through capabilities.
6. Document reassignment, delegation, vacancy, expired assignment, unavailable
optional module, retention, and future E2EE limitations honestly.
### Gate
A campaign message can be delivered once to a function-bound Postbox; an
authorized holder can act in an explicit context; a reassignment changes future
access without moving or rewriting the message. A vacant Postbox remains
addressable, retains the delivery, and visibly reports that no holder can
decrypt or act until assignment; multiple incumbents and delegates receive
independent key/access evidence. Delivery, key access, and later corrections
remain auditable.
## Stage 3: Templates, reports, data sources, and deep launches
### Product promise
> As a staff member arriving from a specialist system such as HIS, I can open a
> GovOPlaN report or document task with a governed source context already
> selected, review the effective data and template version, generate the
> document, and return or link the immutable result without re-entering data.
### Ownership and composition
- Templates owns reusable definitions, versions, merge schemas, localization,
render profiles, preview, and rendering.
- Reporting owns report definitions, parameters, curated data selection,
execution history, dashboards, sharing, scheduling, publication, and export.
- Connectors own protocol-specific access. A governed data-source catalog owns
connection profiles, source contracts, freshness, and health if repeated
use proves that a separate module is warranted.
- Files stores generated bytes. DMS owns a generated artifact once it becomes a
governed living document. Records owns later retention/archive semantics.
### Safe launch and generation contract
1. The source program opens a stable GovOPlaN route with an opaque, short-lived,
single-purpose launch reference—not credentials, arbitrary SQL, or trusted
personal data in the URL.
2. GovOPlaN authenticates the actor, authorizes the requested report/template
and source object, resolves the launch context server-side, and shows source,
freshness, purpose, and any blockers.
3. A curated connector/read-model capability returns a versioned input snapshot
that satisfies the template or report schema.
4. Generation records template/report definition version, parameters, input
snapshot or reproducible source references, transformation version, output
checksum, actor, and policy.
5. The result is stored or handed back through an explicit callback/reference
contract. Retries are idempotent; expired or replayed launch references fail
safely.
### Implementation slices
1. Implement the first Templates manifest, version/schema DTOs, safe preview,
deterministic render, package fragments, and Files-backed output.
2. Implement the first Reporting manifest, parameter/source DTOs, one
module-owned read model, execution evidence, and downloadable export.
3. Define the shared data-source/profile and launch-context contracts with one
read-only target connector and a mock HIS-style launch producer.
4. Deliver one end-to-end reference document and one analytical report from the
same governed source contract.
5. Add role-specific UI/docs, source freshness and validation states, deep-link
integration examples, and target-system acceptance tests.
### Gate
A user can follow a signed or server-side launch reference from a target system
to a prefilled GovOPlaN task, verify the source and freshness, render an output,
and reproduce why that exact artifact was produced. The browser cannot inject
arbitrary queries or transport secrets, and authorization is re-evaluated in
GovOPlaN.
## Stage 4: Governed BI and reporting
### Product promise
> As an institutional analyst, I can combine approved operational sources into
> versioned, quality-checked analytical data products, define transparent
> measures and reports, drill from aggregates where policy permits, and explain
> the source, transformation, effective organizational hierarchy, and reporting
> date behind every result.
### Lessons taken from SuperX
SuperX is useful inspiration because it combines many university sources in a
modular data warehouse, loads and transforms them on a schedule, keeps
historical or reporting-date views, supplies prepared subject modules, and
serves different aggregation levels through a common reporting surface. Its
official overview also emphasizes a lower operational-data layer, validation
logs, configurable load routines, organizational hierarchies, and transparent
report calculations. See the
[SuperX project overview](https://www.superx-projekt.de/__index.htm),
[administration and module architecture](https://www.superx-projekt.de/doku/kern_modul/admin/__index.htm),
and [cross-institution indicator module](https://www.superx-projekt.de/doku/kenn_modul/benutzer/__index.htm).
GovOPlaN should adopt the principles, not clone the implementation. Its distinct
value is connecting analytical results to governed work, responsibility,
evidence, policy, and correction across independently installable modules.
### Architecture layers
1. **Source catalog:** governed profiles, owners, purpose, schema/version,
classification, credentials, health, extraction mode, and source-of-truth
declaration.
2. **Ingestion and staging:** immutable run evidence, source snapshots or
watermarks, schema-drift detection, quarantine, replay, and personal-data
minimization.
3. **Transformation and quality:** versioned transformations, dependencies,
tests, validation findings, correction policy, and lineage. Silent repair of
source facts is not acceptable.
4. **Semantic products:** documented dimensions, measures, official key
mappings, organizational hierarchies, time/stichtag semantics, ownership,
access policy, and version.
5. **Consumption:** Reporting definitions, dashboards, drill-down, scheduled
outputs, APIs, open-data handoffs, and template generation.
6. **Operations:** scheduling, backfill, freshness objectives, observability,
cost/resource limits, retention, backup/restore, promotion between systems,
and reproducible release recipes.
### Selected first subject-area direction
- Start with read-only HIS and CampusOnline source profiles and produce the
institution's internal student-statistics data product plus reports derived
from it.
- Model reference/effective date separately from extraction, validation, and
freeze dates. For example, a semester state may describe `1 September`, be
accepted and frozen on `12 December`, and remain the reproducible source for
that semester's official statistics. Later corrections create a superseding
version; they never rewrite the frozen state silently.
- Make a graphical, typed data-flow editor the primary analyst surface. Nodes
represent governed sources/snapshots, validation, mapping, transformation,
aggregation, a list report/data product, template rendering, and publication.
The graph is versioned, testable, previewable, acyclic, and shows schema,
lineage, quality findings, policy, and pinned upstream versions.
- A list report may become a governed source for another flow. This enables
`source + transformations/aggregation -> list report/data product -> template
-> publishable report` without copying the calculation or losing lineage.
### Implementation slices
1. Bound the first HIS/CampusOnline student-statistics dataset and internal
report; define its legal/policy basis, source owners, official keys,
reference/freeze semantics, privacy level, and acceptance calculation.
2. Generalize the Stage 3 source profile into catalog, ingestion-run, staged
dataset, validation, transformation, lineage, and data-product contracts.
3. Implement one scheduled load and explicit frozen-state lifecycle with
quarantine/replay and a fully transparent transformation into the internal
student-statistics product.
4. Implement the first graphical typed flow editor over those proven nodes,
including version/diff, validation, preview, cycle prevention, and pinned
upstream product versions.
5. Serve the product through Reporting with policy-aware aggregate and
drill-down behavior; export reproducible evidence and calculation metadata.
6. Chain one list report as a source into a template-backed publishable report.
7. Package mappings, transformations, validations, reports, documentation, and
tests so they can be promoted from development to test to production.
### Gate
One internal student-statistics product can be rebuilt from declared HIS and/or
CampusOnline snapshots, preserves its reference and freeze dates, passes
explicit quality tests, explains every measure and mapping, enforces disclosure
policy, survives schema/freshness failures visibly, and produces the same
accepted list and template-backed report across promoted environments.
## Stage 5: Collaborative document editing
### Product promise
> As an authorized group, we can turn an uploaded or generated artifact into a
> shared document, edit it concurrently or through controlled check-out,
> comment, review, approve, compare, and recover versions, then freeze the
> accepted rendition as evidence or a record without losing who changed what.
### Ownership and composition
- Files owns bytes, checksums, storage providers, upload/download, and low-level
file permissions.
- DMS owns document identity, versions, renditions, editing sessions, locks,
comments, reviews, approvals, signatures, external DMS references, and
document-level evidence.
- A collaboration connector owns provider-specific Collabora, OnlyOffice,
Nextcloud/OpenDesk, or another office-suite protocol. DMS owns the session
lifecycle and authorization even when editing occurs externally.
- Templates may create the first version. Cases, Campaign, Postbox, Reporting,
and Workflow may reference documents. Records owns classification, legal
hold, archive handoff, and disposal after the document enters that lifecycle.
### Implementation slices
1. Implement DMS document/version/reference DTOs and one Files-backed version
path, including optimistic concurrency, checksums, locks, and audit.
2. Add comments, review requests, approval state, immutable accepted
renditions, comparison, and recovery of prior versions.
3. Define a provider-neutral editing-session capability with short-lived
grants, callback authentication, save idempotency, health, and reconciliation.
4. Integrate one external collaborative editor, including loss of provider,
concurrent save, stale callback, permission revocation, and restore tests.
5. Connect generated reports/templates, Postbox/Campaign sharing, focused
document views, configured docs, and Records handoff.
### Gate
Multiple authorized actors can edit a document without silent lost updates;
every accepted version has stable content and provenance; provider callbacks
cannot bypass current authorization; degraded or ambiguous saves are visible
and reconcilable; the approved rendition can enter records without making Files
or the external editor the document-lifecycle owner.
## Program-wide acceptance and pause rules
- Do not start a broad generic platform abstraction until two concrete stages
demonstrate the repeated contract.
- Do not make Workflow a dependency of these stages. Manual transitions,
explicit actions, and focused views should become stable providers that a
later transition engine can coordinate.
- Do not treat a remote system as trusted merely because it supplied a deep
link, callback, webhook, file, identity, or data row.
- Do not claim a stage complete from local unit tests. Use pinned composition,
target integration, failure drills, adaptive docs, and operator evidence.
- A later stage may prototype contracts while the preceding gate is being
proven, but it may not redefine an owning module's boundary by convenience.
## Decisions intentionally deferred
The sequence itself is selected. These bounded choices remain for the stage
that first needs them:
- first target SMTP/IMAP, file/directory, identity, and deployment profile;
- first external IDM source and identity conflict/disable policy;
- first trusted Postbox assurance/recovery/history-access profile and
external-mail bridge;
- first HIS or other launch producer and callback/reference contract;
- exact HIS/CampusOnline source endpoints, student-statistics keys and accepted
calculation, freeze/correction policy, privacy profile, and permitted
drill-down level;
- datasource provider selection and quality/promotion policy; the architecture
now separates `govoplan-datasources` lifecycle from `govoplan-connectors`
acquisition and `govoplan-dataflow` transformation;
- first collaborative editor/provider and whether the first UX is concurrent
editing, controlled check-out, or both; and
- first Records/archive target and approval/signature assurance level.
+415 -22
View File
@@ -5,8 +5,9 @@ GovOPlaN releases. It belongs to the `govoplan` meta repository because it works
across all local checkouts, release scripts, module manifests, migration audits, across all local checkouts, release scripts, module manifests, migration audits,
catalog files, Git state, and signing keys. catalog files, Git state, and signing keys.
The current implementation has a read-only dashboard plus guarded local The current implementation has a read-only dashboard, preview-only legacy
candidate/publish actions: controls, and a bounded durable executor for release steps whose inputs and
effects can be verified safely:
- inspect repositories from `repositories.json` - inspect repositories from `repositories.json`
- show dirty, ahead, behind, missing, no-HEAD, and tag state - show dirty, ahead, behind, missing, no-HEAD, and tag state
@@ -18,8 +19,20 @@ candidate/publish actions:
- configure target versions per release unit in the web UI - configure target versions per release unit in the web UI
- select the repositories that should advance through checkboxes - select the repositories that should advance through checkboxes
- generate dry-run selective release plans for independently versioned packages - generate dry-run selective release plans for independently versioned packages
- generate signed catalog candidates for the selected release units - freeze a selective plan as a durable, resumable local release run
- preview/apply/push reviewed catalog candidates behind explicit confirmations - durably preflight a creation-time-bound repository, create its annotated tag,
and publish its branch/tag pair atomically
- deterministically update recognized package/manifest version declarations and
commit only the receipt-bound metadata paths
- order selected module providers before consumers, create module tags before
Core, regenerate Core's selected WebUI release lock, and re-run alignment
before any remote push
- build selected Python wheels and generate a private, signed, receipt-bound
catalog candidate
- publish that exact candidate through a verified website commit and immutable
tag after explicit confirmation
- install selected candidate wheels into a private no-network/no-dependency
target and verify their installed metadata against the frozen plan
Start it from the meta repository: Start it from the meta repository:
@@ -27,8 +40,32 @@ Start it from the meta repository:
./.venv/bin/python tools/release/release-console.py ./.venv/bin/python tools/release/release-console.py
``` ```
The server binds to `127.0.0.1` by default and prints a URL containing a local The launcher accepts only a numeric loopback address, binds to `127.0.0.1` by
API token. Open that URL in a browser on the same machine. default, always creates a fresh API token, and prints that token only in the URL
fragment. Open that URL in a browser on the same machine. A deliberately
tokenless embedded app is read-only: non-GET `/api/` requests fail with `403`.
Non-loopback operation needs a separately deployed authenticated TLS boundary;
the local launcher will not expose the mutation API that way.
Run durable release mutation only against an operator-private workspace.
Repository roots, every path ancestor, critical and nested Git metadata, and
tracked worktree files must be owned by the console process UID (or root where
appropriate) and must not be group/world writable. Symlink/special Git
metadata, object alternates, and grafts are rejected. The console pins
`/usr/bin/git`, `/usr/bin/ssh`, a fixed system `PATH`, disabled hooks, isolated
Git configuration, and no replace objects. Shared or `nfsnobody`-owned
checkouts remain usable for read-only planning, but every durable executor
fails closed there; clone the registered origins into a private workspace
before releasing.
The runtime itself is part of the authority boundary. Durable run creation
verifies the meta checkout, release/check tooling, repository registry, Python
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
executables. It then binds a clean meta-repository HEAD and named branch that
exactly match the registered `origin`. Every execution and reconciliation
rechecks that receipt. Permission checks cannot prove that code was safe before
it entered a writable tree, so release from a fresh operator-private clone or a
separately verified installed console artifact.
The web UI starts with a repository table. Each repository can be checked The web UI starts with a repository table. Each repository can be checked
independently and assigned its own target version. Repositories without version independently and assigned its own target version. Repositories without version
@@ -37,6 +74,217 @@ shows the dry-run commands for the selected rows, and `Generate Candidate`
creates a signed catalog candidate that advances only selected repositories that creates a signed catalog candidate that advances only selected repositories that
already have a catalog entry. already have a catalog entry.
The full-width **Release Workflow** guide projects the server state into seven
operator phases: Inspect, Targets, Validate, Source, Package, Publish, and
Verify. It does not maintain a second workflow state. Completed, current,
blocked, locked, and unavailable phases are derived from the dashboard,
selective plan, and durable run record. The next-action panel opens the exact
section or durable step that needs attention. Changing a channel, target
version, repository selection, or release gate detaches the browser from the
current run and invalidates the draft plan; the persisted run remains available
from the saved-run selector. Problems in unselected repositories remain visible
as workspace notices but do not lock an unrelated release; the selective plan
is the authority for blockers in the selected repository set.
Installation verification is an explicit durable step after catalog
publication. It verifies the exact candidate receipt, installs every selected
Python wheel into a temporary target with network and dependency resolution
disabled, and compares installed names and versions with the frozen plan.
Deployment startup, database upgrades, and module-combination smoke tests remain
release-integration CI gates; the console does not represent its local wheel
check as a production deployment.
`Build Plan` also returns structured release-gate findings for each selected
repository. The plan names the recommended next action and gives an explicit
remediation for source-version, lockfile, Core WebUI composition, Git state, and
worktree findings. A target version that differs from internally consistent
source metadata becomes a bounded `UPDATE` step. Unsupported, missing, or
internally inconsistent declarations remain a blocker with exact remediation.
`source_preflight_ready` means that plan-visible source gates pass or have a
bounded deterministic mutation; the non-mutating `Preview Tag + Publish`
remains mandatory for remote, manifest, and immutable-tag checks.
## Durable release runs
The **Durable Run State** card turns the current repository/version selection
into a versioned local run record. The server rebuilds the selective plan and
requires the plan to resolve exactly the requested repositories and target
versions; the browser cannot submit or replace the plan snapshot. The input and
plan are then immutable and covered by a canonical SHA-256 integrity digest.
Every executable repository step also carries its creation-time full HEAD,
branch, target tag, a SHA-256 over both the fetch and push URLs of `origin`, and
a SHA-256 over bounded dirty-path names and bytes. Both URLs must exactly equal
the remote registered in `repositories.json`; a changed HEAD, branch, worktree,
remote, push URL, or metadata byte requires a new run or explicit
interrupted-step reconciliation rather than silently retargeting the frozen
compatibility decision.
The complete record also has a checksum so a valid-looking manual edit to its
mutable state fails closed. File permissions remain the authority boundary;
these digests detect accidental or manual corruption, not an attacker who can
replace the private record and recompute its checksums. Changing a target,
channel, or gate input requires a new run.
Creation requires a caller-generated `request_id`. Its SHA-256 fingerprint is
the private, workspace-scoped durable mapping to exactly one run; the raw
identifier is never persisted. Repeating the same identifier with the same
immutable inputs returns that run without rebuilding the plan while it remains
inside the bounded local retention window, even if the live dashboard has
since drifted. Reusing it with different inputs fails closed. The browser keeps
an uncertain create identifier in session storage,
replays it after reload, and selects the known run returned by the server. A
successful create remains shown as saved if only the subsequent list refresh
fails.
Run records survive console restarts, but remain local operator state rather
than a signed release artifact or the system audit log. The default location is
`$XDG_STATE_HOME/govoplan/release-console/workspace-<sha256>/release-runs/`, or
`~/.local/state/...` when `XDG_STATE_HOME` is unset or relative. It is outside
the source checkout so filesystems without enforceable POSIX modes cannot
silently weaken the journal. Newly created state directories use mode `0700`
and records use `0600`. Writes use a cross-process lock, a same-directory
temporary file, `fsync`, atomic replacement, and directory/parent `fsync`.
Symbolic-link paths, untrusted owners or writable ancestry, overly broad record
modes, malformed schemas, unknown fields, invalid state combinations,
oversized files, and digest mismatches fail closed. A bad record is neither
rewritten nor automatically quarantined.
The store retains at most 512 workspace-scoped run records created through the
console. On creation at that limit it removes only the oldest fully completed,
integrity-verified record. Running, planned, attention, blocked, foreign, and
unreadable records are never deleted implicitly; if no completed record is
available, creation fails closed with a retention remediation. Unavailable
records still consume the bound and remain visible in cursor-paginated lists
so corruption cannot be hidden by normal turnover.
The full resolved-workspace SHA-256 is part of both the private storage
namespace and immutable input snapshot. Every list, read, and state transition
checks it. An alternate workspace therefore cannot list or resume another
workspace's runs. This remains true for an embedding/test `run_state_root`
override: the server always appends
`workspace-<full-sha256>/release-runs/` rather than treating the override as a
shared record directory. Durable candidates use its private sibling
`release-candidates/` directory, never a checkout-local runtime path. A corrupt
record from one workspace therefore cannot leak even its identifier or an
integrity error into another workspace.
Each frozen plan step has an explicit `pending`, `running`, `succeeded`,
`failed`, or `interrupted` state. Plan order remains a prerequisite: a later
step is unavailable until earlier steps have succeeded. Exact attempt and
resume/retry/reconciliation request identifiers are fingerprinted so delayed
repetitions remain idempotent for the retained run's lifetime. These fingerprints are
never evicted: the store fails closed before accepting more than 2,048 commands
or 2,048 attempts and asks the operator to create a fresh run. The display
record keeps at most 256 server-generated state events. Events contain only an
enum event type, timestamp, step identifier, and bounded result code—never
commands, process output, confirmation text, credentials, bearer tokens, or
signing material.
Before a step can enter `running`, the store reserves the two command-ledger
slots needed for worst-case recovery. It also projects the serialized record
through start, finish/failure, resume, and the required terminal reconciliation
or read-only retry; the start is rejected unless every required atomic write
fits the record-size bound. An explicit resume consumes one slot and is
accepted only while a persisted attempt is actually running. A mutating attempt
always retains its final `effect_absent` or `effect_succeeded` slot;
`unresolved` may be recorded at most once for that attempt and only when an
additional ledger slot and serialized terminal-write capacity are available.
Read-only interruption and known failure retain the slot and byte capacity
needed to prepare a retry. Capacity exhaustion is therefore detected before
starting an effect rather than stranding an ambiguous attempt.
An explicit resume after a process restart converts every persisted `running`
step to `interrupted`; it never guesses whether an external effect happened.
An interrupted read-only step can be prepared for retry. An interrupted
mutating step remains unavailable until the operator independently reconciles
local and remote state, selects `effect_absent`, `effect_succeeded`, or
`unresolved`, and types `RECONCILE`. `effect_absent` prepares a safe new
attempt, `effect_succeeded` advances the run without repeating the external
effect, and `unresolved` keeps the run blocked. Each outcome emits a bounded,
code-only state event. A known failed attempt can likewise be prepared for
retry. The UI keeps unavailable controls visible and disabled.
Supported executors durably claim the step before invoking an effect. Exact
attempt replays return the recorded outcome and never invoke the executor a
second time. Successful repository preflight, version, commit, Core-bundle,
tag, and push steps persist a bounded repository-state receipt. Version
reconciliation requires aligned declarations in the same commit; commit
reconciliation requires the expected single-parent release commit and only
recognized metadata paths. Tag reconciliation independently requires an
annotated local tag at the frozen HEAD; push reconciliation additionally
requires both the remote annotated tag object and remote branch to match.
Catalog generation persists
only its server-issued opaque candidate ID and canonical catalog SHA-256, then
re-resolves and re-hashes that private candidate before publication.
Repository capabilities are frozen into each plan unit (`python-package`,
`webui-package`, `module-manifest`, `database-migrations`, `documentation`,
`core-release-bundle`, and the universal `git-source`) and determine which
steps appear. Internally aligned version changes are rendered deterministically
from recognized TOML, JSON, lockfile, manifest, and package declarations.
Pre-existing dirty worktrees remain visible but have no commit executor; the
console never absorbs unrelated operator changes.
For mixed releases, module interface providers are ordered before consumers
and Core is tagged last. The durable sequence creates and commits module
metadata, creates local module tags, updates Core's selected WebUI references,
regenerates the release lock against those local tags, commits/tags Core, and
runs a receipt-bound alignment gate before exposing any atomic branch/tag push.
A failed step stops later steps while preserving prior receipts for explicit
retry or reconciliation.
The browser likewise retains the request identifier for an uncertain
resume/retry/reconciliation response and replays it after reload. A successful
replay selects the returned run state. Transport and server failures retain the
identifier; a deterministic `4xx` rejection clears it so a stale command cannot
poison a later attempt.
The run API is covered by the same local console token middleware as every
other `/api/` route:
- `POST /api/release-runs` requires `request_id`, then idempotently rebuilds and
freezes a selective plan only when that creation is not already known.
- `GET /api/release-runs` lists bounded summaries ordered by immutable
`created_at` and run identifier. `next_cursor` advances a stable descending
traversal even while older runs are updated, without offset duplicates or
skips. Unreadable entries
remain a deterministic final section and are therefore reachable through
pagination instead of displacing newer verified runs.
- `GET /api/release-runs/{run_id}` reads and verifies one exact record.
- `POST /api/release-runs/{run_id}/resume` records explicit recovery.
- `POST /api/release-runs/{run_id}/steps/{step_id}/retry` prepares a failed or
read-only interrupted step for another attempt.
- `POST /api/release-runs/{run_id}/steps/{step_id}/reconcile` records a
confirmed observed outcome for an interrupted mutating step.
- `POST /api/release-runs/{run_id}/steps/{step_id}/execute` claims and invokes
only the narrow executor declared by the immutable plan step. Durable release
execution accepts only the registered `origin`, never a caller-selected
remote.
- `POST /api/release-runs/{run_id}/steps/{step_id}/preview` provides the
non-mutating preview for receipt-bound catalog publication.
Run-storage errors are confined to the Durable Release Run section; dashboard
and release-preview collection continue and the workflow guide points to the
bounded storage remediation.
The run record is execution evidence only for a supported step whose durable
claim and bounded result receipt were persisted. The console never infers
success from a button click or process exit alone. An executor exception or a
lost result write leaves the attempt interrupted and non-retriable until
explicit recovery. Catalog publication persists the candidate and keyring
hashes, exact website commit, annotated tag object and peeled commit, branch,
tag name, and registered-origin digest. A successful reconciliation revalidates
the candidate against the trust anchor in the frozen website parent, requires
the exact deterministic catalog/keyring/module blobs and full commit delta, a
sole frozen parent, and matching local and remote branch/tag identities. If any
part cannot be proved, the run remains interrupted and may only be recorded as
`unresolved`.
Dashboard collection is also fail closed. Unreadable Core version metadata or a
malformed module contract is returned as a bounded `collection_errors` entry
with a remediation, marks the dashboard blocked, and becomes a structured
blocker in both full and selective release plans. The console does not silently
omit a contract or turn these source errors into an HTTP 500.
The release-control area above the repository table is read-only and is meant The release-control area above the repository table is read-only and is meant
to become the central release cockpit. It shows: to become the central release cockpit. It shows:
@@ -56,17 +304,112 @@ Plain repository pushes are separate from catalog publication. `Preview Push`
shows the selected repository push commands. `Push Selected` requires `PUSH` in shows the selected repository push commands. `Push Selected` requires `PUSH` in
the repository push confirmation field. the repository push confirmation field.
The catalog workflow panel can also operate on the same selected rows: The source release panel retains `Preview Tag + Publish` as a non-mutating
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
but disabled; the corresponding mutation endpoint rejects apply requests.
Creation and atomic branch/tag publication use the `TAG` and `PUBLISH`
confirmations on the durable run steps. The gate requires an aligned target
version, a clean named branch with a HEAD, and a checkout that is not behind. Existing
local or remote tags must resolve to the selected HEAD and are never moved.
The local and remote annotated tag objects must also be identical, not merely
point at the same commit. Before any source tag is created, the console loads
the cross-repository module registry. This release gate also rejects every
user-facing workflow documentation topic that has no scope condition, or has
an alternative condition without `required_scopes` or `any_scopes`.
- `Generate` creates a signed candidate below `runtime/release-candidates/`. The catalog workflow panel can also operate on the same selected rows for
generation and preview. Its legacy apply/push controls stay disabled; durable
publication consumes only the candidate receipt recorded by that run:
- `Generate` creates a signed candidate in the operator's private XDG state
directory and records only an opaque candidate ID plus the canonical catalog
SHA-256 in durable run state.
- `Preview` validates a candidate and shows what would be copied into the - `Preview` validates a candidate and shows what would be copied into the
website repository. website repository.
- `Apply + Tag` requires `APPLY` in the confirmation field. - `Apply + Website Tag` remains visible but disabled outside a durable run.
- `Push` requires `PUSH` in the confirmation field. - `Push Website Release` remains visible but disabled outside a durable run.
Source release tags belong to Core or module repositories. Website catalog
publication creates a separate catalog tag in the website repository; the UI
names these independently to avoid confusing the two immutable references.
Candidate signing is fail-closed: every Core and module source ref in the
complete post-update catalog must already have the requested annotated tag both
locally and on the configured source remote. Both tags must be the same
annotation object with version-aligned tagged package metadata. Selected tags
must additionally resolve to the selected clean, version-aligned HEAD, and the
signed `selected_units` record captures the peeled commit and tag-object
identifiers. Create and publish the source tags before using `Generate`.
Catalog publication repeats that check for selected units and also verifies
every Core and module source ref in the complete candidate, including entries
preserved from the previous catalog. Historical refs need not be at the current
worktree HEAD, but their local and remote annotated tags must match and their
tagged installable package and module-manifest metadata must declare the catalog
version. New selected releases additionally pass the stricter current-source
alignment gate, including public runtime version declarations. A candidate
cannot be applied or published while any referenced source tag is absent or
inconsistent; the preview and API response list each repository and missing or
invalid ref so the operator can repair the exact source releases first.
Every selected Python release must also supply its exact built wheel. Durable
generation first verifies that the receipt-bound annotated tag is the same
object locally and on the registered origin. It clones an isolated checkout at
the receipt's exact commit and builds from that checkout, never from the mutable
live worktree. Every authenticated base-catalog source is likewise cloned from
its registered origin at an annotated release tag; only selected repositories
contribute synthesized fields. The base catalog and keyring are read as one
authenticated, exact private snapshot before synthesis.
Python wheels are built by a required Bubblewrap worker with no network,
private temporary/home directories, a read-only exact source mount, no host
home or file keys, cleared environment, fixed system tools, and CPU/address
space/process/file-size limits. If a trusted Bubblewrap launcher is unavailable,
generation fails closed. The worker must return one bounded regular wheel; the
console copies it through no-follow descriptors into a fresh private file,
`fsync`s it, then validates its package identity. Generation computes the
archive SHA-256 and an install-stable
payload identity from one bounded, regular-file descriptor and signs those
values into `release.artifacts`. Updating a Python version without a matching
wheel removes the stale identity. A source-only preview can still explain the
gap, but apply, commit, tag, and push fail closed until every selected Python
unit has a matching built-artifact identity. Repositories selected only for a
meta/source tag do not need a Python artifact.
Candidate directories and files are operator-owned `0700`/`0600` state. Before
any later release step consumes one, the executor re-resolves the opaque handle
below the configured root and re-hashes the signed catalog against its persisted
receipt. Shared roots, symlinks, channel path fragments, altered candidates, and
unowned files are rejected.
The current same-host worker is a containment baseline, not the final hostile
build-service boundary. `RLIMIT_FSIZE` is per file, and the worker does not yet
have a size-limited filesystem/cgroup quota, a fresh kernel keyring, or a
seccomp profile denying keyctl/request-key/ptrace/mount operations. A malicious
backend could therefore exhaust scratch disk/inodes or target same-UID kernel
facilities. Closing that denial-of-service/isolation gap requires a dedicated
quota/cgroup worker with a fresh keyring and seccomp policy.
The server-owned wheel builds remain under the private candidate's `artifacts/`
directory. This slice signs their identities but does **not** upload the wheel or
add a download URL to the public catalog; the existing Git `python_ref` is source
provenance and rebuilding it is not an equivalent artifact. Keep the private
candidate until the exact wheels have been transferred through an approved
deployment channel, verified against `archive_sha256`, consumed by the installer,
and covered by its signed receipt. Public artifact transport and receipt-aware
candidate cleanup remain separate release-lifecycle work.
Read-only provenance checks derive a same-host HTTPS Git URL from conventional
SSH remotes when possible, with a non-interactive check of the configured remote
as fallback. Source tag creation and atomic publication always use the explicit
configured Git remote.
The default signing key is The default signing key is
`$HOME/.config/govoplan/release-keys/release-key-1.pem` when no signing key is `$HOME/.config/govoplan/release-keys/release-key-1.pem` when no signing key is
entered in the UI. entered in the UI. Signing-key files must be regular, owned by the operator, and
inaccessible to group/other users. The browser sends a configured key path only
on the initial execution request; it never persists signing material in session
storage, and request-ID recovery replays no key path.
Generate a selective release plan from the terminal: Generate a selective release plan from the terminal:
@@ -87,18 +430,28 @@ Build a signed selective catalog candidate:
```sh ```sh
KEY_DIR="$HOME/.config/govoplan/release-keys" KEY_DIR="$HOME/.config/govoplan/release-keys"
ARTIFACT_DIR="$(mktemp -d)"
../govoplan-files/.venv/bin/python -m pip wheel \
--no-deps \
--no-build-isolation \
--wheel-dir "$ARTIFACT_DIR" \
../govoplan-files
./.venv/bin/python tools/release/release-catalog.py selective \ ./.venv/bin/python tools/release/release-catalog.py selective \
--repo-version govoplan-files=0.1.9 \ --repo-version govoplan-files=0.1.9 \
--python-artifact \
"govoplan-files=$ARTIFACT_DIR/govoplan_files-0.1.9-py3-none-any.whl" \
--channel stable \ --channel stable \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem"
``` ```
This writes candidate catalog/keyring files below `runtime/release-candidates/`, This writes candidate catalog/keyring files below
generates the browsable module directory below `modules/`, validates the signed `$XDG_STATE_HOME/govoplan/release-candidates/` (or
candidate with the module installer validator, and reports whether the candidate `~/.local/state/govoplan/release-candidates/`), generates the browsable module
catalog/keyring match the currently published channel. It does not publish to directory below `modules/`, validates the signed candidate with the module
the website repository. installer validator, and reports whether the candidate catalog/keyring match the
currently published channel. It does not publish to the website repository.
Regenerate the browsable module directory from an existing catalog/keyring: Regenerate the browsable module directory from an existing catalog/keyring:
@@ -113,8 +466,10 @@ Regenerate the browsable module directory from an existing catalog/keyring:
Preview publication of a reviewed candidate: Preview publication of a reviewed candidate:
```sh ```sh
CANDIDATE_ROOT="${XDG_STATE_HOME:-$HOME/.local/state}/govoplan/release-candidates"
./.venv/bin/python tools/release/release-catalog.py publish-candidate \ ./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir runtime/release-candidates/stable-YYYYMMDD-HHMMSS \ --candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable --channel stable
``` ```
@@ -122,7 +477,7 @@ Apply the reviewed candidate into the website repository without pushing:
```sh ```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \ ./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir runtime/release-candidates/stable-YYYYMMDD-HHMMSS \ --candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable \ --channel stable \
--apply \ --apply \
--commit \ --commit \
@@ -133,7 +488,7 @@ Push is a separate explicit flag:
```sh ```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \ ./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir runtime/release-candidates/stable-YYYYMMDD-HHMMSS \ --candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable \ --channel stable \
--apply \ --apply \
--commit \ --commit \
@@ -141,6 +496,13 @@ Push is a separate explicit flag:
--push --push
``` ```
Publication validates the same in-memory catalog object that it writes; it does
not copy a path that can change after validation. On commit, the console reads
the catalog, keyring, and complete module directory back from the immutable Git
tree and requires byte-for-byte equality with those validated objects. Tags and
remote branch updates then reference that exact commit SHA rather than the
mutable worktree `HEAD`.
Published channels are expected below the public catalog base URL: Published channels are expected below the public catalog base URL:
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json` - `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
@@ -151,6 +513,30 @@ forcing every repository to the same tag, but channel publication must preserve
the unchanged package versions, validate interface compatibility, sign the the unchanged package versions, validate interface compatibility, sign the
updated catalog, and keep the published keyring healthy. updated catalog, and keep the published keyring healthy.
When a selected module exposes a WebUI package, its requested version must also
match Core's `webui/package.release.json` input and the resolved
`package-lock.release.json` entry. The source-tag preflight, selective plan, and
catalog-candidate writer all enforce this composition boundary. Pins for modules
that are not part of the selective release remain unchanged.
Release integration also enforces repository and composition version alignment
and generates a CycloneDX SBOM from the resolved Python environment and the
release WebUI lockfile. Catalog publication should attach that immutable SBOM
and its digest to the corresponding Core/composition release.
Addresses and Notifications have current WebUI source contributions but are
not part of the pinned v0.1.8 WebUI release composition because their v0.1.8
tags predate those packages. They re-enter the release composition only through
new, immutable module tags whose backend, manifest, frontend, and lock metadata
pass the alignment gate.
Candidate publication verifies signatures against the already published
keyring, not against keys supplied only by the candidate. A changed keyring
must have its canonical SHA-256 embedded in the signed catalog. The public
module-directory files are regenerated from that verified catalog and keyring
at publication time; candidate-supplied directory files are never copied as
authoritative provenance.
## Published Module Directory ## Published Module Directory
The target release repository is an online, browsable module directory. The The target release repository is an online, browsable module directory. The
@@ -174,9 +560,16 @@ Selective catalog candidates now include this directory tree. Publishing a
candidate copies the channel catalog, keyring, and `modules/` tree into the candidate copies the channel catalog, keyring, and `modules/` tree into the
website repository together. website repository together.
Current limitation: selective catalog generation updates existing catalog Selective catalog generation can add a module that is not yet represented in
entries. Initial catalog entry synthesis for brand-new modules is still a the source channel. Initial entries are synthesized from the selected
separate backend slice. repository's `[project]` metadata, `govoplan.modules` entry point, and runtime
`ModuleManifest`; there is no separate hand-maintained initial-entry list. The
release gate requires the entry point to resolve inside the selected checkout,
exact package/manifest/frontend version agreement, verified source-tag
provenance, a non-conflicting module id, and complete required dependency and
interface closure in the resulting candidate. Classification beyond the
manifest-derived `official` tag remains an explicit later catalog curation
step rather than an inferred business/service label.
## Direction ## Direction
+77 -65
View File
@@ -6,81 +6,93 @@ Generated from `repositories.json`. Use that JSON file as the machine-readable s
| Repository | Subtype | Local path | Gitea | | Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `govoplan` | `meta` | `../govoplan` | [govoplan](https://git.add-ideas.de/add-ideas/govoplan) | | `govoplan` | `meta` | `../govoplan` | [govoplan](https://git.add-ideas.de/GovOPlaN/govoplan) |
| `govoplan-core` | `kernel` | `../govoplan-core` | [govoplan-core](https://git.add-ideas.de/add-ideas/govoplan-core) | | `govoplan-core` | `kernel` | `../govoplan-core` | [govoplan-core](https://git.add-ideas.de/GovOPlaN/govoplan-core) |
## Module ## Module
| Repository | Subtype | Local path | Gitea | | Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `govoplan-access` | `platform` | `../govoplan-access` | [govoplan-access](https://git.add-ideas.de/add-ideas/govoplan-access) | | `govoplan-access` | `platform` | `../govoplan-access` | [govoplan-access](https://git.add-ideas.de/GovOPlaN/govoplan-access) |
| `govoplan-addresses` | `domain` | `../govoplan-addresses` | [govoplan-addresses](https://git.add-ideas.de/add-ideas/govoplan-addresses) | | `govoplan-addresses` | `domain` | `../govoplan-addresses` | [govoplan-addresses](https://git.add-ideas.de/GovOPlaN/govoplan-addresses) |
| `govoplan-admin` | `platform` | `../govoplan-admin` | [govoplan-admin](https://git.add-ideas.de/add-ideas/govoplan-admin) | | `govoplan-admin` | `platform` | `../govoplan-admin` | [govoplan-admin](https://git.add-ideas.de/GovOPlaN/govoplan-admin) |
| `govoplan-appointments` | `domain` | `../govoplan-appointments` | [govoplan-appointments](https://git.add-ideas.de/add-ideas/govoplan-appointments) | | `govoplan-appointments` | `domain` | `../govoplan-appointments` | [govoplan-appointments](https://git.add-ideas.de/GovOPlaN/govoplan-appointments) |
| `govoplan-approvals` | `domain` | `../govoplan-approvals` | [govoplan-approvals](https://git.add-ideas.de/add-ideas/govoplan-approvals) | | `govoplan-approvals` | `domain` | `../govoplan-approvals` | [govoplan-approvals](https://git.add-ideas.de/GovOPlaN/govoplan-approvals) |
| `govoplan-assets` | `domain` | `../govoplan-assets` | [govoplan-assets](https://git.add-ideas.de/add-ideas/govoplan-assets) | | `govoplan-assets` | `domain` | `../govoplan-assets` | [govoplan-assets](https://git.add-ideas.de/GovOPlaN/govoplan-assets) |
| `govoplan-audit` | `platform` | `../govoplan-audit` | [govoplan-audit](https://git.add-ideas.de/add-ideas/govoplan-audit) | | `govoplan-audit` | `platform` | `../govoplan-audit` | [govoplan-audit](https://git.add-ideas.de/GovOPlaN/govoplan-audit) |
| `govoplan-booking` | `domain` | `../govoplan-booking` | [govoplan-booking](https://git.add-ideas.de/add-ideas/govoplan-booking) | | `govoplan-booking` | `domain` | `../govoplan-booking` | [govoplan-booking](https://git.add-ideas.de/GovOPlaN/govoplan-booking) |
| `govoplan-calendar` | `domain` | `../govoplan-calendar` | [govoplan-calendar](https://git.add-ideas.de/add-ideas/govoplan-calendar) | | `govoplan-calendar` | `domain` | `../govoplan-calendar` | [govoplan-calendar](https://git.add-ideas.de/GovOPlaN/govoplan-calendar) |
| `govoplan-campaign` | `domain` | `../govoplan-campaign` | [govoplan-campaign](https://git.add-ideas.de/add-ideas/govoplan-campaign) | | `govoplan-campaign` | `domain` | `../govoplan-campaign` | [govoplan-campaign](https://git.add-ideas.de/GovOPlaN/govoplan-campaign) |
| `govoplan-cases` | `domain` | `../govoplan-cases` | [govoplan-cases](https://git.add-ideas.de/add-ideas/govoplan-cases) | | `govoplan-cases` | `domain` | `../govoplan-cases` | [govoplan-cases](https://git.add-ideas.de/GovOPlaN/govoplan-cases) |
| `govoplan-certificates` | `domain` | `../govoplan-certificates` | [govoplan-certificates](https://git.add-ideas.de/add-ideas/govoplan-certificates) | | `govoplan-certificates` | `domain` | `../govoplan-certificates` | [govoplan-certificates](https://git.add-ideas.de/GovOPlaN/govoplan-certificates) |
| `govoplan-committee` | `domain` | `../govoplan-committee` | [govoplan-committee](https://git.add-ideas.de/add-ideas/govoplan-committee) | | `govoplan-committee` | `domain` | `../govoplan-committee` | [govoplan-committee](https://git.add-ideas.de/GovOPlaN/govoplan-committee) |
| `govoplan-consultation` | `domain` | `../govoplan-consultation` | [govoplan-consultation](https://git.add-ideas.de/add-ideas/govoplan-consultation) | | `govoplan-consultation` | `domain` | `../govoplan-consultation` | [govoplan-consultation](https://git.add-ideas.de/GovOPlaN/govoplan-consultation) |
| `govoplan-contracts` | `domain` | `../govoplan-contracts` | [govoplan-contracts](https://git.add-ideas.de/add-ideas/govoplan-contracts) | | `govoplan-contracts` | `domain` | `../govoplan-contracts` | [govoplan-contracts](https://git.add-ideas.de/GovOPlaN/govoplan-contracts) |
| `govoplan-dashboard` | `platform` | `../govoplan-dashboard` | [govoplan-dashboard](https://git.add-ideas.de/add-ideas/govoplan-dashboard) | | `govoplan-dashboard` | `platform` | `../govoplan-dashboard` | [govoplan-dashboard](https://git.add-ideas.de/GovOPlaN/govoplan-dashboard) |
| `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/add-ideas/govoplan-dms) | | `govoplan-dataflow` | `platform` | `../govoplan-dataflow` | [govoplan-dataflow](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow) |
| `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/add-ideas/govoplan-dist-lists) | | `govoplan-datasources` | `platform` | `../govoplan-datasources` | [govoplan-datasources](https://git.add-ideas.de/GovOPlaN/govoplan-datasources) |
| `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/add-ideas/govoplan-docs) | | `govoplan-decisions` | `domain` | `../govoplan-decisions` | [govoplan-decisions](https://git.add-ideas.de/GovOPlaN/govoplan-decisions) |
| `govoplan-erp` | `domain` | `../govoplan-erp` | [govoplan-erp](https://git.add-ideas.de/add-ideas/govoplan-erp) | | `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/GovOPlaN/govoplan-dms) |
| `govoplan-evaluation` | `domain` | `../govoplan-evaluation` | [govoplan-evaluation](https://git.add-ideas.de/add-ideas/govoplan-evaluation) | | `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists) |
| `govoplan-facilities` | `domain` | `../govoplan-facilities` | [govoplan-facilities](https://git.add-ideas.de/add-ideas/govoplan-facilities) | | `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/GovOPlaN/govoplan-docs) |
| `govoplan-files` | `domain` | `../govoplan-files` | [govoplan-files](https://git.add-ideas.de/add-ideas/govoplan-files) | | `govoplan-encryption` | `platform` | `../govoplan-encryption` | [govoplan-encryption](https://git.add-ideas.de/GovOPlaN/govoplan-encryption) |
| `govoplan-forms` | `domain` | `../govoplan-forms` | [govoplan-forms](https://git.add-ideas.de/add-ideas/govoplan-forms) | | `govoplan-erp` | `domain` | `../govoplan-erp` | [govoplan-erp](https://git.add-ideas.de/GovOPlaN/govoplan-erp) |
| `govoplan-forms-runtime` | `platform` | `../govoplan-forms-runtime` | [govoplan-forms-runtime](https://git.add-ideas.de/add-ideas/govoplan-forms-runtime) | | `govoplan-evaluation` | `domain` | `../govoplan-evaluation` | [govoplan-evaluation](https://git.add-ideas.de/GovOPlaN/govoplan-evaluation) |
| `govoplan-grants` | `domain` | `../govoplan-grants` | [govoplan-grants](https://git.add-ideas.de/add-ideas/govoplan-grants) | | `govoplan-facilities` | `domain` | `../govoplan-facilities` | [govoplan-facilities](https://git.add-ideas.de/GovOPlaN/govoplan-facilities) |
| `govoplan-helpdesk` | `domain` | `../govoplan-helpdesk` | [govoplan-helpdesk](https://git.add-ideas.de/add-ideas/govoplan-helpdesk) | | `govoplan-files` | `domain` | `../govoplan-files` | [govoplan-files](https://git.add-ideas.de/GovOPlaN/govoplan-files) |
| `govoplan-identity` | `platform` | `../govoplan-identity` | [govoplan-identity](https://git.add-ideas.de/add-ideas/govoplan-identity) | | `govoplan-forms` | `domain` | `../govoplan-forms` | [govoplan-forms](https://git.add-ideas.de/GovOPlaN/govoplan-forms) |
| `govoplan-identity-trust` | `platform` | `../govoplan-identity-trust` | [govoplan-identity-trust](https://git.add-ideas.de/add-ideas/govoplan-identity-trust) | | `govoplan-forms-runtime` | `platform` | `../govoplan-forms-runtime` | [govoplan-forms-runtime](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime) |
| `govoplan-idm` | `platform` | `../govoplan-idm` | [govoplan-idm](https://git.add-ideas.de/add-ideas/govoplan-idm) | | `govoplan-grants` | `domain` | `../govoplan-grants` | [govoplan-grants](https://git.add-ideas.de/GovOPlaN/govoplan-grants) |
| `govoplan-inspections` | `domain` | `../govoplan-inspections` | [govoplan-inspections](https://git.add-ideas.de/add-ideas/govoplan-inspections) | | `govoplan-helpdesk` | `domain` | `../govoplan-helpdesk` | [govoplan-helpdesk](https://git.add-ideas.de/GovOPlaN/govoplan-helpdesk) |
| `govoplan-issue-reporting` | `domain` | `../govoplan-issue-reporting` | [govoplan-issue-reporting](https://git.add-ideas.de/add-ideas/govoplan-issue-reporting) | | `govoplan-identity` | `platform` | `../govoplan-identity` | [govoplan-identity](https://git.add-ideas.de/GovOPlaN/govoplan-identity) |
| `govoplan-learning` | `domain` | `../govoplan-learning` | [govoplan-learning](https://git.add-ideas.de/add-ideas/govoplan-learning) | | `govoplan-identity-trust` | `platform` | `../govoplan-identity-trust` | [govoplan-identity-trust](https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust) |
| `govoplan-ledger` | `domain` | `../govoplan-ledger` | [govoplan-ledger](https://git.add-ideas.de/add-ideas/govoplan-ledger) | | `govoplan-idm` | `platform` | `../govoplan-idm` | [govoplan-idm](https://git.add-ideas.de/GovOPlaN/govoplan-idm) |
| `govoplan-mail` | `domain` | `../govoplan-mail` | [govoplan-mail](https://git.add-ideas.de/add-ideas/govoplan-mail) | | `govoplan-inspections` | `domain` | `../govoplan-inspections` | [govoplan-inspections](https://git.add-ideas.de/GovOPlaN/govoplan-inspections) |
| `govoplan-notifications` | `platform` | `../govoplan-notifications` | [govoplan-notifications](https://git.add-ideas.de/add-ideas/govoplan-notifications) | | `govoplan-learning` | `domain` | `../govoplan-learning` | [govoplan-learning](https://git.add-ideas.de/GovOPlaN/govoplan-learning) |
| `govoplan-ops` | `platform` | `../govoplan-ops` | [govoplan-ops](https://git.add-ideas.de/add-ideas/govoplan-ops) | | `govoplan-ledger` | `domain` | `../govoplan-ledger` | [govoplan-ledger](https://git.add-ideas.de/GovOPlaN/govoplan-ledger) |
| `govoplan-organizations` | `platform` | `../govoplan-organizations` | [govoplan-organizations](https://git.add-ideas.de/add-ideas/govoplan-organizations) | | `govoplan-mail` | `domain` | `../govoplan-mail` | [govoplan-mail](https://git.add-ideas.de/GovOPlaN/govoplan-mail) |
| `govoplan-payments` | `domain` | `../govoplan-payments` | [govoplan-payments](https://git.add-ideas.de/add-ideas/govoplan-payments) | | `govoplan-mandates` | `domain` | `../govoplan-mandates` | [govoplan-mandates](https://git.add-ideas.de/GovOPlaN/govoplan-mandates) |
| `govoplan-permits` | `domain` | `../govoplan-permits` | [govoplan-permits](https://git.add-ideas.de/add-ideas/govoplan-permits) | | `govoplan-notifications` | `platform` | `../govoplan-notifications` | [govoplan-notifications](https://git.add-ideas.de/GovOPlaN/govoplan-notifications) |
| `govoplan-policy` | `platform` | `../govoplan-policy` | [govoplan-policy](https://git.add-ideas.de/add-ideas/govoplan-policy) | | `govoplan-ops` | `platform` | `../govoplan-ops` | [govoplan-ops](https://git.add-ideas.de/GovOPlaN/govoplan-ops) |
| `govoplan-poll` | `domain` | `../govoplan-poll` | [govoplan-poll](https://git.add-ideas.de/add-ideas/govoplan-poll) | | `govoplan-organizations` | `platform` | `../govoplan-organizations` | [govoplan-organizations](https://git.add-ideas.de/GovOPlaN/govoplan-organizations) |
| `govoplan-portal` | `domain` | `../govoplan-portal` | [govoplan-portal](https://git.add-ideas.de/add-ideas/govoplan-portal) | | `govoplan-payments` | `domain` | `../govoplan-payments` | [govoplan-payments](https://git.add-ideas.de/GovOPlaN/govoplan-payments) |
| `govoplan-postbox` | `domain` | `../govoplan-postbox` | [govoplan-postbox](https://git.add-ideas.de/add-ideas/govoplan-postbox) | | `govoplan-parties` | `domain` | `../govoplan-parties` | [govoplan-parties](https://git.add-ideas.de/GovOPlaN/govoplan-parties) |
| `govoplan-procurement` | `domain` | `../govoplan-procurement` | [govoplan-procurement](https://git.add-ideas.de/add-ideas/govoplan-procurement) | | `govoplan-permits` | `domain` | `../govoplan-permits` | [govoplan-permits](https://git.add-ideas.de/GovOPlaN/govoplan-permits) |
| `govoplan-records` | `domain` | `../govoplan-records` | [govoplan-records](https://git.add-ideas.de/add-ideas/govoplan-records) | | `govoplan-policy` | `platform` | `../govoplan-policy` | [govoplan-policy](https://git.add-ideas.de/GovOPlaN/govoplan-policy) |
| `govoplan-reporting` | `domain` | `../govoplan-reporting` | [govoplan-reporting](https://git.add-ideas.de/add-ideas/govoplan-reporting) | | `govoplan-poll` | `domain` | `../govoplan-poll` | [govoplan-poll](https://git.add-ideas.de/GovOPlaN/govoplan-poll) |
| `govoplan-resources` | `domain` | `../govoplan-resources` | [govoplan-resources](https://git.add-ideas.de/add-ideas/govoplan-resources) | | `govoplan-portal` | `domain` | `../govoplan-portal` | [govoplan-portal](https://git.add-ideas.de/GovOPlaN/govoplan-portal) |
| `govoplan-risk-compliance` | `domain` | `../govoplan-risk-compliance` | [govoplan-risk-compliance](https://git.add-ideas.de/add-ideas/govoplan-risk-compliance) | | `govoplan-postbox` | `domain` | `../govoplan-postbox` | [govoplan-postbox](https://git.add-ideas.de/GovOPlaN/govoplan-postbox) |
| `govoplan-scheduling` | `domain` | `../govoplan-scheduling` | [govoplan-scheduling](https://git.add-ideas.de/add-ideas/govoplan-scheduling) | | `govoplan-procurement` | `domain` | `../govoplan-procurement` | [govoplan-procurement](https://git.add-ideas.de/GovOPlaN/govoplan-procurement) |
| `govoplan-search` | `platform` | `../govoplan-search` | [govoplan-search](https://git.add-ideas.de/add-ideas/govoplan-search) | | `govoplan-projects` | `domain` | `../govoplan-projects` | [govoplan-projects](https://git.add-ideas.de/GovOPlaN/govoplan-projects) |
| `govoplan-tasks` | `domain` | `../govoplan-tasks` | [govoplan-tasks](https://git.add-ideas.de/add-ideas/govoplan-tasks) | | `govoplan-records` | `domain` | `../govoplan-records` | [govoplan-records](https://git.add-ideas.de/GovOPlaN/govoplan-records) |
| `govoplan-templates` | `domain` | `../govoplan-templates` | [govoplan-templates](https://git.add-ideas.de/add-ideas/govoplan-templates) | | `govoplan-reporting` | `domain` | `../govoplan-reporting` | [govoplan-reporting](https://git.add-ideas.de/GovOPlaN/govoplan-reporting) |
| `govoplan-tenancy` | `platform` | `../govoplan-tenancy` | [govoplan-tenancy](https://git.add-ideas.de/add-ideas/govoplan-tenancy) | | `govoplan-resources` | `domain` | `../govoplan-resources` | [govoplan-resources](https://git.add-ideas.de/GovOPlaN/govoplan-resources) |
| `govoplan-transparency` | `domain` | `../govoplan-transparency` | [govoplan-transparency](https://git.add-ideas.de/add-ideas/govoplan-transparency) | | `govoplan-risk-compliance` | `domain` | `../govoplan-risk-compliance` | [govoplan-risk-compliance](https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance) |
| `govoplan-workflow` | `platform` | `../govoplan-workflow` | [govoplan-workflow](https://git.add-ideas.de/add-ideas/govoplan-workflow) | | `govoplan-scheduling` | `domain` | `../govoplan-scheduling` | [govoplan-scheduling](https://git.add-ideas.de/GovOPlaN/govoplan-scheduling) |
| `govoplan-search` | `platform` | `../govoplan-search` | [govoplan-search](https://git.add-ideas.de/GovOPlaN/govoplan-search) |
| `govoplan-services` | `domain` | `../govoplan-services` | [govoplan-services](https://git.add-ideas.de/GovOPlaN/govoplan-services) |
| `govoplan-tasks` | `domain` | `../govoplan-tasks` | [govoplan-tasks](https://git.add-ideas.de/GovOPlaN/govoplan-tasks) |
| `govoplan-templates` | `domain` | `../govoplan-templates` | [govoplan-templates](https://git.add-ideas.de/GovOPlaN/govoplan-templates) |
| `govoplan-tenancy` | `platform` | `../govoplan-tenancy` | [govoplan-tenancy](https://git.add-ideas.de/GovOPlaN/govoplan-tenancy) |
| `govoplan-tickets` | `domain` | `../govoplan-tickets` | [govoplan-tickets](https://git.add-ideas.de/GovOPlaN/govoplan-tickets) |
| `govoplan-transparency` | `domain` | `../govoplan-transparency` | [govoplan-transparency](https://git.add-ideas.de/GovOPlaN/govoplan-transparency) |
| `govoplan-views` | `platform` | `../govoplan-views` | [govoplan-views](https://git.add-ideas.de/GovOPlaN/govoplan-views) |
| `govoplan-voting` | `domain` | `../govoplan-voting` | [govoplan-voting](https://git.add-ideas.de/GovOPlaN/govoplan-voting) |
| `govoplan-wiki` | `domain` | `../govoplan-wiki` | [govoplan-wiki](https://git.add-ideas.de/GovOPlaN/govoplan-wiki) |
| `govoplan-workflow` | `platform` | `../govoplan-workflow` | [govoplan-workflow](https://git.add-ideas.de/GovOPlaN/govoplan-workflow) |
| `govoplan-workflow-engine` | `platform` | `../govoplan-workflow-engine` | [govoplan-workflow-engine](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine) |
## Connector ## Connector
| Repository | Subtype | Local path | Gitea | | Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `govoplan-connectors` | `connector-hub` | `../govoplan-connectors` | [govoplan-connectors](https://git.add-ideas.de/add-ideas/govoplan-connectors) | | `govoplan-connectors` | `connector-hub` | `../govoplan-connectors` | [govoplan-connectors](https://git.add-ideas.de/GovOPlaN/govoplan-connectors) |
| `govoplan-fit-connect` | `standard` | `../govoplan-fit-connect` | [govoplan-fit-connect](https://git.add-ideas.de/add-ideas/govoplan-fit-connect) | | `govoplan-fit-connect` | `standard` | `../govoplan-fit-connect` | [govoplan-fit-connect](https://git.add-ideas.de/GovOPlaN/govoplan-fit-connect) |
| `govoplan-rest` | `protocol` | `../govoplan-rest` | [govoplan-rest](https://git.add-ideas.de/add-ideas/govoplan-rest) | | `govoplan-rest` | `protocol` | `../govoplan-rest` | [govoplan-rest](https://git.add-ideas.de/GovOPlaN/govoplan-rest) |
| `govoplan-soap` | `protocol` | `../govoplan-soap` | [govoplan-soap](https://git.add-ideas.de/add-ideas/govoplan-soap) | | `govoplan-soap` | `protocol` | `../govoplan-soap` | [govoplan-soap](https://git.add-ideas.de/GovOPlaN/govoplan-soap) |
| `govoplan-xoev` | `standard` | `../govoplan-xoev` | [govoplan-xoev](https://git.add-ideas.de/add-ideas/govoplan-xoev) | | `govoplan-xoev` | `standard` | `../govoplan-xoev` | [govoplan-xoev](https://git.add-ideas.de/GovOPlaN/govoplan-xoev) |
| `govoplan-xrechnung` | `standard` | `../govoplan-xrechnung` | [govoplan-xrechnung](https://git.add-ideas.de/add-ideas/govoplan-xrechnung) | | `govoplan-xrechnung` | `standard` | `../govoplan-xrechnung` | [govoplan-xrechnung](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung) |
| `govoplan-xta-osci` | `standard` | `../govoplan-xta-osci` | [govoplan-xta-osci](https://git.add-ideas.de/add-ideas/govoplan-xta-osci) | | `govoplan-xta-osci` | `standard` | `../govoplan-xta-osci` | [govoplan-xta-osci](https://git.add-ideas.de/GovOPlaN/govoplan-xta-osci) |
## Website ## Website
+183
View File
@@ -0,0 +1,183 @@
# Scaling And Multi-Host Deployment
## Implemented Contract
GovOPlaN now supports a stateless application tier backed by logically shared
state services. The runtime roles are independently replaceable API, WebUI,
worker, and scheduler processes. Every replica in one installation must use the
same immutable release composition and the same:
- `GOVOPLAN_INSTALLATION_ID`;
- PostgreSQL database;
- Redis broker and coordination service;
- `MASTER_KEY_B64` and deployment secret references;
- enabled-module graph;
- S3-compatible object-storage namespace.
The application tier must not use node-local durable business data in a
multi-host deployment. Files owns managed file metadata while Core provides the
storage-backend contract. Campaign build artifacts are stored under opaque
object keys and workers read those objects from the shared backend. Temporary
build and materialization directories may remain node-local because they are
discardable.
Core validates three explicit state profiles:
| Profile | Supported shape | Storage rule |
| --- | --- | --- |
| `local` | One API and one worker process for development | Local filesystem permitted. |
| `host-shared` | Multiple processes on one Docker host | A shared host volume is permitted; PostgreSQL and Redis are required. |
| `shared` | Multiple independent hosts | PostgreSQL, Redis, and S3-compatible object storage are required. |
`shared` also requires a stable installation identifier. Module package
mutation is blocked in this profile: build and verify a new immutable release,
then roll the complete cluster to it.
## Same-Host Compose
The generated Compose bundle provides:
```text
client -> TLS proxy -> HAProxy -> WebUI replicas -> HAProxy -> API replicas
API/worker/scheduler -> PostgreSQL
-> Redis
-> local volume, managed Garage, or external S3
```
HAProxy discovers Compose replicas through Docker DNS and performs health-aware
balancing without mounting the Docker socket. This improves concurrency and
permits process replacement, but the Docker host and installer-managed stateful
services remain single failure domains. Generated Compose therefore declares
the `host-shared` state profile even when its shared storage happens to be an
external S3 service. Its API backend checks `/health/ready`, so drain or
coordination loss removes a replica from rotation. Container, load-balancer,
and Kubernetes probes send the configured public host explicitly, keeping
readiness compatible with strict trusted-host validation.
Managed Garage is a convenient single-node S3-compatible service. It is not a
multi-host storage cluster. Use an independently operated Garage cluster or
another S3-compatible service for the `shared` profile.
## Kubernetes Export
The deployment compiler exports a stateless Kubernetes runtime when PostgreSQL,
Redis, and S3 are all external:
```sh
python tools/deployment/govoplan-deploy.py render-kubernetes \
--directory /srv/govoplan/default \
--namespace govoplan \
--secret-name govoplan-runtime \
--tls-secret-name govoplan-tls \
--ingress-class-name nginx \
--output /srv/govoplan/default/kubernetes.json
```
The export contains a Namespace, tokenless ServiceAccount, non-secret
ConfigMap, API/WebUI/worker/scheduler Deployments, Services, Pod disruption
budgets, Ingress, and a release-specific migration Job. It deliberately emits
no Secret values, persistent volume, PostgreSQL, Redis, or object-store
deployment. Export is rejected unless both release images use immutable
`image@sha256:...` references.
Create the named Secret through the cluster's secret-management path. The
command prints the exact required key contract. Review the generated
`FORWARDED_ALLOW_IPS` value and replace it with the exact ingress-proxy network
before production use.
The generated containers run as non-root with a read-only root filesystem and
an ephemeral `/tmp`. Runtime Deployments wait for the exact configured database
migration heads before starting. The API exposes `/health/ready`, which fails
while that API node is draining or cannot prove its runtime-coordination
heartbeat.
## Runtime Coordination
Each API and worker incarnation registers in PostgreSQL with its role, software
version, module-composition hash, queue set, and heartbeat. Ops shows active,
draining, stopped, and stale nodes and compares active counts with configured
replica expectations.
An operator may request or cancel drain from Ops:
- API readiness becomes unavailable on the next heartbeat so the load balancer
stops assigning new requests.
- A worker stops consuming its configured queues and may finish work already
claimed by that process.
- A stale process incarnation cannot overwrite a replacement incarnation's
heartbeat.
- A coordination outage removes API readiness and cancels worker consumers;
the existing incarnation must heartbeat successfully before either resumes.
Singleton work uses PostgreSQL-backed leases with monotonically increasing
fencing tokens. The generated scheduler runs Celery beat through
`govoplan_core.commands.fenced_run`; loss of its lease terminates the child and
returns a distinct failure code. A fenced business operation must validate the
same lease token immediately before committing its effect.
## Release Ordering
Use this order for every multi-replica rollout:
1. Verify immutable image identities, module composition, external state
reachability, backup evidence, and the generated plan.
2. Drain application replicas when the migration compatibility declaration
requires it.
3. Run the release-specific migration Job exactly once. PostgreSQL advisory
locking serializes all Core and module migration tasks across competing
deployment jobs.
4. Let runtime init containers run `wait_for_database`. They wait for exact
configured Alembic heads and never mutate schema.
5. Roll API, workers, scheduler, and WebUI using health-aware replacement.
6. Verify runtime composition, expected replica counts, queue consumers,
object-storage round trips, and recovery status in Ops.
Applying the complete generated manifest is fail-closed: runtime pods remain in
their init phase until the migration Job reaches the expected heads. A second
release may be submitted concurrently, but advisory locking prevents concurrent
schema mutation and each release has a distinct migration Job name.
## Storage Trust Boundary
Installer-managed Garage uses its exact generated endpoint. An arbitrary
external S3 endpoint is accepted only when the deployment explicitly sets
`FILE_STORAGE_S3_ENDPOINT_TRUSTED=true`; that endpoint must be a clean HTTPS
origin without embedded credentials, query, fragment, or path. This is an
operator trust declaration, not a user-controlled connector bypass. Operators
remain responsible for DNS, certificate, network-egress, bucket-policy,
versioning, and lifecycle controls.
## Capacity
- Scale API replicas only within the PostgreSQL connection budget.
- Scale workers by queue, with upper bounds based on external provider limits.
- Keep one fenced scheduler rather than load-balancing schedulers.
- Increase WebUI replicas for asset/proxy capacity.
- Measure request latency, database query time and locks, active connections,
queue age, retry rate, storage latency, and provider throttling before adding
replicas.
Workers compete for Redis-backed work and are not placed behind a load balancer.
SMTP, IMAP, directory, connector, workflow, dataflow, and reporting queues often
hit external-system limits before host CPU is exhausted.
## What This Does Not Claim
The implemented contract provides stateless runtime placement, shared artifact
access, node visibility, drain controls, migration serialization, and scheduler
fencing. It does not by itself provide:
- a highly available PostgreSQL, Redis, or object-store deployment;
- automatic PostgreSQL backup, point-in-time recovery, or restore verification;
- autoscaling policy;
- central logs, metrics, traces, or alert routing;
- managed ingress certificates;
- automatic reconciliation of every possible module side effect;
- a service-level availability guarantee.
Those are deployment and module-adoption requirements. Before claiming high
availability, drill replica loss, rolling replacement, session continuity, job
redelivery, scheduler failover, migration exclusion, object-store outage, and a
coordinated database/object/key restore. Recovery rules and evidence are
defined in [Recovery And Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md).
+57 -13
View File
@@ -33,17 +33,41 @@ cd /mnt/DATA/git/govoplan
tools/checks/security-audit/run.sh --mode full --scope govoplan tools/checks/security-audit/run.sh --mode full --scope govoplan
``` ```
Reports are written to `audit-reports/`, which is intentionally ignored by git. When invoked from a Flatpak development environment without a sandbox-local
Docker CLI, the wrapper automatically uses `flatpak-spawn --host docker`. The
host account must still be allowed to open the Docker daemon socket. For a
conventional rootful installation this commonly means membership in the
`docker` group followed by a complete logout/login; that membership is
root-equivalent, so rootless Docker is preferable where the deployment policy
requires a smaller privilege boundary.
The wrapper tags the toolbox image by a fingerprint of the Dockerfile and Reports are written to `audit-reports/`, which is intentionally ignored by git.
`requirements-audit.txt`. If those inputs have not changed, subsequent runs reuse Each run records tool versions, report checksums, and start/end repository
the existing local image instead of reinstalling all tools. The stable alias is revision plus worktree fingerprints. A repository change during scanning makes
`govoplan/security-audit:local` unless `SECURITY_AUDIT_IMAGE` is set. the run fail so a mixed code snapshot cannot be reported as a valid audit.
Step exit codes are recorded separately from findings: report-only mode may
accept findings, but scanner execution errors and malformed JSON/SARIF reports
always fail the run. The manifest lists the expected, present, and missing
reports for that invocation. Validation and checksums use that explicit set, so
reusing a report directory cannot make stale output look like part of a new run.
It also contains `coverage_status` and structured `scanner_coverage` entries.
Every required scanner is recorded as `no-findings`, `findings`,
`scanner-failure`, or `skipped`; this makes an incomplete local run visible
without treating it as a clean audit.
The wrapper tags the toolbox image by a fingerprint of the Dockerfile,
`requirements-audit.txt`, and the Semgrep smoke-test inputs. If those inputs have
not changed, subsequent runs reuse the existing local image instead of
reinstalling all tools. The stable alias is `govoplan/security-audit:local`
unless `SECURITY_AUDIT_IMAGE` is set.
Semgrep is installed separately in the toolbox image because current Semgrep Semgrep is installed separately in the toolbox image because current Semgrep
packages pin Click to an affected `8.1.x` line. The image upgrades Click after packages pin affected Click and MCP versions. The image upgrades both after
Semgrep installation and fails during build if Semgrep no longer starts with the Semgrep installation, runs an actual local-rule scan, and audits the final
patched Click version. toolbox Python environment during the image build. The compatibility releases
are pinned exactly to keep the tested override reproducible. Remove either
compatibility override only after Semgrep's own dependency bounds include a
fixed version.
Force a cached rebuild: Force a cached rebuild:
@@ -70,6 +94,16 @@ tools/checks/security-audit/run.sh --mode quick --scope current --update --build
- `ci`: quick plus Semgrep public registry rulesets, Trivy, pip-audit, npm audit. - `ci`: quick plus Semgrep public registry rulesets, Trivy, pip-audit, npm audit.
- `full`: ci plus OSV-Scanner, jscpd, Radon, and Xenon. - `full`: ci plus OSV-Scanner, jscpd, Radon, and Xenon.
The Gitea workflow uses `full` mode so its coverage contract includes every
scanner above. Missing scanners fail strict runs and all Actions runs, even
while actual findings remain report-only. Local report-only runs may finish
with missing tools for diagnostics, but their manifest is marked
`coverage_status: incomplete`.
Semgrep and Trivy are invoked with finding-sensitive exit codes. Their exit 1
is therefore a finding under the wrapper contract; higher exit codes, missing
output, invalid JSON/SARIF, and scanner error payloads are execution failures.
Bandit and Ruff security reports are split by source kind. Production code under Bandit and Ruff security reports are split by source kind. Production code under
`src/` is written to `bandit.json` and `ruff-security.json` and controls strict `src/` is written to `bandit.json` and `ruff-security.json` and controls strict
mode. Test code under `tests/` is still scanned for visibility, but its findings mode. Test code under `tests/` is still scanned for visibility, but its findings
@@ -79,7 +113,7 @@ baseline.
## Gating ## Gating
The initial Gitea workflow runs in report-only mode: The Gitea workflow currently runs findings in report-only mode:
```bash ```bash
SECURITY_AUDIT_FAIL_ON_FINDINGS=0 SECURITY_AUDIT_FAIL_ON_FINDINGS=0
@@ -95,13 +129,13 @@ SECURITY_AUDIT_FAIL_ON_FINDINGS=1
or run locally with: or run locally with:
```bash ```bash
tools/checks/security-audit/run.sh --mode ci --scope current --strict tools/checks/security-audit/run.sh --mode full --scope govoplan --strict
``` ```
## Audit Burndown Workflow ## Audit Burndown Workflow
Treat Gitea issues as the active audit state. A full GovOPlaN audit should Treat Gitea issues as the active audit state. A full GovOPlaN audit should
produce one tracker issue in `add-ideas/govoplan` and child issues in the produce one tracker issue in `GovOPlaN/govoplan` and child issues in the
repository that owns each fix. repository that owns each fix.
Use the tracker issue for: Use the tracker issue for:
@@ -138,7 +172,14 @@ clusters that cross module ownership or make behavior harder to change safely.
The regular `Security Audit` workflow reuses the fingerprinted toolbox image The regular `Security Audit` workflow reuses the fingerprinted toolbox image
when the Docker daemon is persistent, which is the normal case for the when the Docker daemon is persistent, which is the normal case for the
self-hosted Gitea runner using the host Docker socket. The separate self-hosted Gitea runner using the host Docker socket. Trusted push, schedule,
and manual runs scan all registered repositories; authenticated SSH is used
only for the private website repository. The wrapper inspects the Actions job
mount table and forwards only the narrowest writable mount covering the audit
scope; it never inherits the job's Docker socket or unrelated runner mounts.
Pull-request audit runs stay disabled while the audit runner exposes its host
Docker socket: PR-controlled audit code must run on a disposable or rootless
runner without host-socket access. The separate
`Security Audit Toolbox Update` workflow runs weekly with `Security Audit Toolbox Update` workflow runs weekly with
`SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the `SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the
allowed tool version ranges into a refreshed local image. allowed tool version ranges into a refreshed local image.
@@ -153,7 +194,10 @@ cd /mnt/DATA/git/govoplan
python -m venv .venv python -m venv .venv
./.venv/bin/python -m pip install -r requirements-audit.txt ./.venv/bin/python -m pip install -r requirements-audit.txt
./.venv/bin/python -m pip install 'semgrep>=1.140,<2' ./.venv/bin/python -m pip install 'semgrep>=1.140,<2'
./.venv/bin/python -m pip install --upgrade --no-deps 'click>=8.3.3' ./.venv/bin/python -m pip install --upgrade --no-deps 'click==8.3.3'
./.venv/bin/python -m pip install --upgrade --no-deps 'mcp==1.28.1'
./.venv/bin/semgrep scan --metrics=off --config tools/checks/security-audit/semgrep-govoplan.yml tools/checks/check-version-alignment.py
./.venv/bin/pip-audit --progress-spinner off
``` ```
Then install the non-Python tools (`gitleaks`, `trivy`, `osv-scanner`, `jscpd`) Then install the non-Python tools (`gitleaks`, `trivy`, `osv-scanner`, `jscpd`)
@@ -0,0 +1,192 @@
# System Administrator Lifecycle User Story
## Story
> As a system administrator, I can execute one shell command that downloads a
> verified GovOPlaN distribution and starts a completely configured Core control
> plane without optional modules. In the WebUI I can browse compatible signed
> module releases, select the modules for this installation, and follow every
> download, validation, migration, installation, activation, and health-check
> step. When an update is available, I can review its impact and confirm it.
>
> I can add API or worker replicas on containers or other servers without
> copying mutable local state. I can operate development, test, and production
> systems, transfer a versioned configuration through the WebUI, undo an applied
> configuration version, test an update in one environment, and then apply the
> same immutable update recipe to another environment.
This is a product-level story owned by the GovOPlaN platform rather than by an
individual domain module. It joins installation, module lifecycle, operations,
configuration packages, and release provenance into one administrator journey.
The canonical backlog item is
[GovOPlaN #13](https://git.add-ideas.de/GovOPlaN/govoplan/issues/13).
## Terms
- **Core control plane:** the smallest bootable distribution: Core API, Core
WebUI, PostgreSQL, Redis, installer worker, migration runner, and durable
storage configuration. No optional GovOPlaN module package is installed.
- **Bootstrap administrator:** a single-use, time-limited installation identity
that may access only first-run and module-lifecycle functions. It is retired
when the selected identity/access configuration becomes healthy.
- **Module release:** an immutable, signed package plus manifest, compatibility
contract, migrations, WebUI contribution, checksums, and SBOM references.
- **Configuration revision:** an immutable, schema-versioned export of
non-secret system/module settings, policies, compositions, and secret
references. Secret values are never exported.
- **Update recipe:** the reviewed, immutable plan containing exact Core/module
versions, configuration revision, migration order, preflight results,
maintenance/drain requirements, health checks, and permitted rollback or
forward-recovery actions.
## Acceptance journeys
### One-command first installation
1. The administrator runs one documented command on a supported host.
2. The bootstrapper verifies a signed distribution manifest before executing or
starting downloaded artifacts.
3. It checks container/runtime, ports, storage, entropy, memory, architecture,
and connectivity requirements; generates deployment-local secrets with
restrictive permissions; and never prints them.
4. It starts PostgreSQL, Redis, Core API/WebUI, one installer worker, and the
migration runner. Readiness does not pass until migrations and durable
dependencies are healthy.
5. It prints the local URL and one-time bootstrap credential. Re-running the
command is idempotent and shows or repairs the existing installation rather
than creating another identity or database.
6. No optional module is installed or enabled at this point.
### Module selection, installation, and update
1. Core reads an approved signed catalog and trusted keyring through the shared
outbound-network policy.
2. The WebUI shows available, installed, compatible, blocked, withdrawn, and
update-available releases with their channel, provenance, contracts,
migrations, permissions, configuration requirements, and release notes.
3. Selecting modules produces a dependency/compatibility plan before any
mutation. The administrator can amend the selection or confirm the plan.
4. Installation executes durably in a worker. The UI receives persisted step
state and can reconnect without losing progress.
5. Package signature/checksum, version alignment, contract closure, migration
graph, configuration schema, and health checks are mandatory gates.
6. Remote artifacts are acquired by a deployment-owned downloader that permits
approved origins, pins the validated connection peer, enforces size and time
limits, and verifies signature/checksum before making a local immutable
artifact available. `pip`, `npm`, archive tools, and any compatibility Git
importer then run without unrestricted network access. A restricted-network
profile can satisfy the same contract through operator-prefetched artifacts.
7. A failed or interrupted operation reaches a visible retry, forward-recovery,
rollback, or manual-intervention state. It never reports success merely
because the initiating request returned.
8. An available update follows the same preview and confirmation path. Exact
repeated requests are idempotent.
### Horizontal scaling
1. API, WebUI, installer, scheduler, and ordinary worker roles are stateless
with respect to local container disks. Durable state uses PostgreSQL, Redis,
and configured shared file/object storage.
2. A documented command can add API or queue-specific worker replicas. The
default Compose profile supports local scale-out; an orchestrator profile
supplies equivalent health/readiness probes and rolling replacement.
3. Only one migration or module-lifecycle mutation may own the deployment lock,
while any healthy replica can serve read and normal domain traffic.
4. Workers announce identity, queues, software composition, heartbeat, and
drain state. Operators can see skew and safely retire a replica.
5. Sessions, throttling, idempotency, jobs, installer progress, and scheduled
work remain correct when requests move between replicas.
### Development, test, and production promotion
1. The administrator exports a configuration revision from development through
the WebUI. The package is versioned, checksummed, attributable, and contains
secret references or required-secret declarations, never secret values.
2. Test imports the package into a preview area. Core reports environment-bound
values, missing capabilities/secrets, compatibility changes, and the exact
apply plan.
3. Applying creates a new revision; the previous effective revision remains
addressable. Undo is a new audited revision that restores the earlier
configuration where contracts permit it.
4. After tests and health checks pass, Core emits an update recipe with exact
immutable release and configuration identifiers.
5. Production validates the recipe against its own environment, requires a new
confirmation, and executes the same ordered plan. Environment-specific
secret bindings and endpoints remain local.
6. Promotion never copies tenant/business data implicitly. Data migration,
anonymized fixtures, and backup/restore are separate explicit operations.
## Safety and governance requirements
- Catalogs, artifacts, recipes, and configuration revisions have signatures,
checksums, provenance, expiry/revocation semantics, and audit evidence.
- Catalog and artifact retrieval is not delegated to an unrestricted package
manager. Approved-origin and peer-pinned download, response bounds,
signature/checksum verification, extraction safety, offline installation, and
deployment egress policy form separate defenses.
- The installer has a dedicated narrow authority; normal domain permissions do
not imply host/package-management access.
- The UI distinguishes reversible configuration rollback from database or
package migrations that require forward recovery.
- Cluster-wide mutations use fencing/leases so an expired worker cannot later
commit a stale result.
- Update plans declare availability impact, worker draining, backup/restore
prerequisites, database compatibility windows, and post-change probes.
- Deployment profiles set secure headers, trusted proxies/hosts, body limits,
outbound-network policy, storage, TLS/cookie posture, and observability.
- Release publication requires aligned backend/frontend/manifest versions and
machine-readable dependency/SBOM provenance.
## Implementation slices
Implementation status as of the current source tree:
- Slice 6 has a working application-tier foundation: state profiles, shared
object storage, runtime node registration/heartbeats/drain, fenced scheduler,
migration serialization, exact-head startup waiting, Ops visibility, and a
Kubernetes export. Production acceptance still requires topology-specific
failover and restore drills.
- The recovery foundation for slices 4 and 5 is implemented as a Core recovery
ledger and deployment operation journal. Automatic database backup and broad
adoption by module-owned external effects remain open work.
1. **Reproducible Core-only distribution.** Publish pinned multi-architecture
images, signed distribution manifest, Core-only Compose profile, bootstrap
preflight, generated secrets, readiness, and idempotent rerun/repair.
2. **First-run control plane.** Add the restricted bootstrap administrator,
one-time enrollment, initial catalog/keyring configuration, and retirement
after durable administrator access is established.
3. **Read-only online module directory.** Move the existing catalog and module
directory contracts into the installed Core WebUI with compatibility,
provenance, release-note, and update-state presentation.
4. **Durable module plan and install.** Reuse the existing installer queue,
locks, signed-package validator, rollback drill, and run evidence behind a
plan/confirm/progress UI. Add initial catalog-entry synthesis and artifact
acquisition where the current release console still assumes local sources.
5. **Safe module update.** Add drain/maintenance coordination, backup gate,
migration compatibility window, reconnectable progress, health verification,
retry/recovery, and update notification.
6. **Stateless replica profile.** Continue module adoption and operational
proof for the implemented role commands, shared-state validation, runtime
registration/drain, fenced scheduler, and Kubernetes application-tier
export. Prove multiple API and worker replicas against the target shared
dependencies.
7. **Configuration revision model.** Define provider export/import schemas,
canonical serialization, secret references, validation/diff, immutable
revision storage, audit, apply, and undo-as-new-revision.
8. **Environment promotion and recipes.** Add source/target fingerprints,
preview, environment bindings, acceptance evidence, exact recipe generation,
signed transfer, and independently confirmed application.
9. **Operational proof.** Exercise interrupted installs, stale locks, unavailable
catalogs, revoked keys, failed migrations, replica loss, configuration undo,
and development-to-test-to-production promotion in release CI and target
drills.
## Explicit non-goals for the first distribution slice
- Shipping optional modules in the Core image.
- Exporting secrets or production business data with configuration.
- Pretending every schema migration can be reversed automatically.
- Building a proprietary orchestrator instead of supporting Compose and a
standard cluster scheduler through the same role/readiness contracts.
- Allowing the browser process to execute arbitrary shell commands.
+137
View File
@@ -0,0 +1,137 @@
# GovOPlaN Views Architecture
## Purpose
GovOPlaN Views are governed presentation projections for a task,
responsibility, or workflow step. A View can reduce the visible modules,
navigation entries, routes, page sections, and commands to the interface
needed for the current job.
Views are optional. If `govoplan-views` is not installed or enabled, the normal
permission-derived interface remains unchanged.
## Security Boundary
A View is not an authorization mechanism.
- Access, tenant isolation, resource guards, and backend permission checks
remain authoritative.
- A View may hide an interface surface that the actor is otherwise allowed to
use.
- A View can never expose a route, action, tenant, or resource that normal
authorization denies.
- An authorized deep link outside the current View should offer an explicit
temporary escape or View switch. It must not be presented as a permission
denial.
This boundary lets Views improve focus without creating a second, weaker RBAC
system.
## Ownership
Core owns the versioned, module-neutral surface contract and WebUI runtime
hooks. Modules declare stable surfaces and use shared hooks to respect the
effective projection. Modules do not import `govoplan-views`.
`govoplan-views` owns:
- draft and immutable published View revisions
- system, tenant, group, and user assignments
- default, mandatory, and user-selectable Views
- active per-user View state
- effective projection resolution and provenance
- the View editor, preview, validation, and stale-surface diagnostics
Policy optionally owns inherited ceilings and explainable decisions. Workflow
optionally references a pinned View revision for an instance or step and may
narrow it further.
## Surface Contract
Modules announce only useful, semantic surfaces:
- module
- navigation item
- route or workspace
- section or panel
- command or action
Each descriptor has a stable namespaced id, parent id, kind, label, default
visibility, ordering, and dependency metadata where needed. Surface ids are
public module contracts, not CSS selectors, component paths, or arbitrary DOM
fragments.
The first release supports visible or hidden. Read-only states, layout
replacement, visual emphasis, and arbitrary styling are separate concerns and
are deferred.
## Effective Resolution
The effective interface is the intersection of:
1. installed and enabled modules
2. actor permissions and resource access
3. administrator and Policy ceilings
4. an assigned or user-selected View
5. an optional workflow instance or step overlay
Lower scopes and workflow overlays may narrow inherited visibility but cannot
broaden it. Every inherited, locked, hidden, unavailable, or stale choice
should carry provenance that the editor and runtime can explain.
Published View revisions are immutable. Active workflow instances pin the
revision they use. Unknown or retired surface ids produce diagnostics rather
than breaking startup. If no valid effective View can be resolved, the system
uses the last valid projection or the normal authorized interface and reports
the configuration problem to administrators.
## Workflow Behavior
A workflow definition may reference a View for the whole instance or a
particular step. Starting, resuming, or advancing the workflow activates the
appropriate projection. Users can intentionally leave focused mode and return
from an open-work widget or notification without losing workflow state.
Module handoffs carry the workflow and View context through Core contracts.
Workflow does not import the target module or the Views implementation.
## Delivery Order
1. Define the Core surface registry and runtime hooks.
2. Initialize `govoplan-views` and persist versioned definitions.
3. Add assignment, selection, resolution, provenance, and the editor.
4. Add Policy inheritance and administrator ceilings.
5. Add Workflow instance and step activation.
6. Adopt semantic section/action descriptors module by module.
## Implementation Status
Implemented in the initial Views slice:
- Core contract version `1`, stable module/navigation/route identifiers, custom
section/action descriptors, manifest validation, and platform API metadata
- shell navigation, route-boundary, settings, administration, dashboard-widget,
embedded-capability, and organization-action filtering
- `govoplan-views` definitions, immutable revisions, system/tenant/group/user
assignments, user selection, provenance, and stale-surface recovery
- a system and tenant administration editor with unsaved-change protection,
publish/archive controls, assignment management, and server-enforced lockout
prevention
- surface declarations for every currently installed module that contributes a
WebUI, including finer-grained shared administration and settings surfaces
Still intentionally separate:
- Policy-owned inherited ceilings and policy decision provenance
- workflow-instance and workflow-step activation of pinned View revisions
- read-only and layout-replacement projections beyond the version `1`
visible/hidden contract
## Gitea Work Packages
- `govoplan#17`: task-focused Views user story
- `govoplan#16`: initialize and implement `govoplan-views`
- `govoplan-core#271`: versioned surface and runtime contracts
- `govoplan-policy#9`: inheritance, ceilings, and provenance
- `govoplan-workflow#7`: workflow instance and step activation
- `govoplan-workflow#3`: focused workflow mode user story
@@ -0,0 +1,153 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/capability-fit-boundary-evidence.schema.json",
"title": "GovOPlaN externally issued capability-fit boundary evidence",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"proof_id",
"assessment_id",
"assessment_release",
"installed_evidence_sha256",
"issued_at",
"expires_at",
"claims",
"signatures"
],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.1.0"
},
"evidence_kind": {
"const": "govoplan.capability-fit-boundary-proof"
},
"proof_id": {
"$ref": "#/$defs/opaque_id"
},
"assessment_id": {
"$ref": "#/$defs/opaque_id"
},
"assessment_release": {
"$ref": "#/$defs/opaque_id"
},
"installed_evidence_sha256": {
"$ref": "#/$defs/sha256"
},
"issued_at": {
"type": "string",
"format": "date-time"
},
"expires_at": {
"type": "string",
"format": "date-time"
},
"claims": {
"type": "array",
"minItems": 1,
"maxItems": 8,
"items": {
"$ref": "#/$defs/claim"
}
},
"signatures": {
"type": "array",
"minItems": 1,
"maxItems": 16,
"items": {
"$ref": "#/$defs/signature"
}
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"claim": {
"type": "object",
"additionalProperties": false,
"required": ["scope", "result", "subject_id", "control_ids", "artifacts"],
"properties": {
"scope": {
"enum": [
"target_environment",
"external_providers",
"accessibility",
"privacy",
"security",
"operations",
"recovery",
"production_approval"
]
},
"result": {
"enum": ["passed", "failed", "approved", "rejected"]
},
"subject_id": {
"$ref": "#/$defs/opaque_id"
},
"control_ids": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"uniqueItems": true,
"items": {
"$ref": "#/$defs/opaque_id"
}
},
"artifacts": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"items": {
"$ref": "#/$defs/evidence_artifact"
}
}
}
},
"evidence_artifact": {
"type": "object",
"additionalProperties": false,
"required": ["artifact_id", "sha256"],
"properties": {
"artifact_id": {
"$ref": "#/$defs/opaque_id"
},
"sha256": {
"$ref": "#/$defs/sha256"
}
}
},
"signature": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "key_id", "value"],
"properties": {
"algorithm": {
"const": "ed25519"
},
"key_id": {
"$ref": "#/$defs/opaque_id"
},
"value": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
}
}
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}
+73 -63
View File
@@ -1,8 +1,8 @@
{ {
"$schema": "./capability-fit.schema.json", "$schema": "./capability-fit.schema.json",
"schema_version": "0.1.0", "schema_version": "0.1.0",
"assessment_id": "campaign-reference-2026-07-20", "assessment_id": "campaign-reference-2026-07-22",
"assessed_at": "2026-07-20", "assessed_at": "2026-07-22",
"scope": { "scope": {
"title": "Campaign-centric internal pilot and small-production candidate", "title": "Campaign-centric internal pilot and small-production candidate",
"reference_journeys": [ "reference_journeys": [
@@ -14,30 +14,31 @@
] ]
}, },
"release": { "release": {
"kind": "workspace_snapshot", "kind": "tagged_release",
"ref": "workspace-2026-07-20", "ref": "stable-catalog-202607220843",
"meta_commit": "05ae81d64195", "meta_commit": "5447299289a1",
"reproducible": false, "reproducible": true,
"configuration_packages": [], "configuration_packages": [],
"notes": [ "notes": [
"The snapshot is untagged.", "The live stable catalog has a valid Ed25519 signature trusted through release-key-1.",
"Dirty repositories are recorded and local-only work is not treated as release-integrated." "Core v0.1.13 and Campaign v0.1.10 are tagged and package-integrated; this is not target-environment or production approval.",
"No configuration revision or configuration package is pinned yet."
] ]
}, },
"composition": [ "composition": [
{ {
"module_id": "core", "module_id": "core",
"repository": "govoplan-core", "repository": "govoplan-core",
"commit": "e6f7c45f0a95", "commit": "d487726f4d2c",
"manifest_version": "server-0.3.0", "manifest_version": "0.1.13",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "API, registry, migrations, sessions, kernel contracts and shared WebUI" "role": "API, registry, migrations, sessions, kernel contracts and shared WebUI"
}, },
{ {
"module_id": "tenancy", "module_id": "tenancy",
"repository": "govoplan-tenancy", "repository": "govoplan-tenancy",
"commit": "1dde03854733", "commit": "efbec827616b",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": false, "dirty": false,
@@ -46,7 +47,7 @@
{ {
"module_id": "organizations", "module_id": "organizations",
"repository": "govoplan-organizations", "repository": "govoplan-organizations",
"commit": "45cda1a33f18", "commit": "39c081c4fb8f",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": false, "dirty": false,
@@ -58,22 +59,22 @@
"commit": "7a1710af896f", "commit": "7a1710af896f",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "Normalized internal identity directory" "role": "Normalized internal identity directory"
}, },
{ {
"module_id": "access", "module_id": "access",
"repository": "govoplan-access", "repository": "govoplan-access",
"commit": "ab07075a679b", "commit": "f1d64d247e12",
"manifest_version": "0.1.8", "manifest_version": "0.1.11",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "Local authentication, sessions, API keys and RBAC" "role": "Local authentication, sessions, API keys and RBAC"
}, },
{ {
"module_id": "admin", "module_id": "admin",
"repository": "govoplan-admin", "repository": "govoplan-admin",
"commit": "c9e1fb287f50", "commit": "11ecf362a36d",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": false, "dirty": false,
@@ -82,7 +83,7 @@
{ {
"module_id": "dashboard", "module_id": "dashboard",
"repository": "govoplan-dashboard", "repository": "govoplan-dashboard",
"commit": "a315a7c62b1d", "commit": "4b960ad37f0d",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": false, "dirty": false,
@@ -91,8 +92,8 @@
{ {
"module_id": "policy", "module_id": "policy",
"repository": "govoplan-policy", "repository": "govoplan-policy",
"commit": "2511fbb5a864", "commit": "1063622d311a",
"manifest_version": "0.1.8", "manifest_version": "0.1.9",
"enabled": true, "enabled": true,
"dirty": false, "dirty": false,
"role": "Policy explanation and configuration boundary" "role": "Policy explanation and configuration boundary"
@@ -100,7 +101,7 @@
{ {
"module_id": "audit", "module_id": "audit",
"repository": "govoplan-audit", "repository": "govoplan-audit",
"commit": "5e4f84a789ea", "commit": "d3d2c60d7dc1",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": false, "dirty": false,
@@ -109,46 +110,46 @@
{ {
"module_id": "campaigns", "module_id": "campaigns",
"repository": "govoplan-campaign", "repository": "govoplan-campaign",
"commit": "2e593b7fa412", "commit": "735e874bd03c",
"manifest_version": "0.1.8", "manifest_version": "0.1.10",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "Campaign authoring, build, delivery control and reporting" "role": "Campaign authoring, build, delivery control and reporting"
}, },
{ {
"module_id": "files", "module_id": "files",
"repository": "govoplan-files", "repository": "govoplan-files",
"commit": "f3210234d35a", "commit": "2b34f6e30578",
"manifest_version": "0.1.8", "manifest_version": "0.1.9",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "Managed files and Campaign attachments" "role": "Managed files and Campaign attachments"
}, },
{ {
"module_id": "mail", "module_id": "mail",
"repository": "govoplan-mail", "repository": "govoplan-mail",
"commit": "b0337b2d261a", "commit": "3e2302909022",
"manifest_version": "0.1.8", "manifest_version": "0.1.10",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "SMTP and IMAP profiles and transports" "role": "SMTP and IMAP profiles and transports"
}, },
{ {
"module_id": "calendar", "module_id": "calendar",
"repository": "govoplan-calendar", "repository": "govoplan-calendar",
"commit": "371a00aff51c", "commit": "9bcf41bb1fbb",
"manifest_version": "0.1.8", "manifest_version": "0.1.8",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "Optional calendar outside the Campaign pilot minimum" "role": "Optional calendar outside the Campaign pilot minimum"
}, },
{ {
"module_id": "docs", "module_id": "docs",
"repository": "govoplan-docs", "repository": "govoplan-docs",
"commit": "f2ade1d62475", "commit": "be52b716caed",
"manifest_version": "0.1.8", "manifest_version": "0.1.10",
"enabled": true, "enabled": true,
"dirty": true, "dirty": false,
"role": "Configured-system documentation" "role": "Configured-system documentation"
}, },
{ {
@@ -163,10 +164,10 @@
{ {
"module_id": "addresses", "module_id": "addresses",
"repository": "govoplan-addresses", "repository": "govoplan-addresses",
"commit": "f19350e65d76", "commit": "93dddbb8c52a",
"manifest_version": "0.1.8", "manifest_version": "0.1.9",
"enabled": false, "enabled": false,
"dirty": true, "dirty": false,
"role": "Optional reusable recipient sources and CardDAV" "role": "Optional reusable recipient sources and CardDAV"
} }
], ],
@@ -293,20 +294,20 @@
"kind": "contract", "kind": "contract",
"scope": "current_workspace", "scope": "current_workspace",
"locator": "govoplan/tools/checks/check-contracts.py", "locator": "govoplan/tools/checks/check-contracts.py",
"note": "43 contracts, 29 providers, no issues" "note": "43 modules, 33 providers, 19 requirements, no issues"
} }
], ],
"conditions": [ "conditions": [
"Repeat checks on a clean pinned release candidate." "Package integration is verified; repeat checks on the installed target composition."
], ],
"gaps": [ "gaps": [
"The current snapshot is dirty and untagged." "No target deployment acceptance is recorded."
], ],
"risks": [ "risks": [
"A working module graph can still be unreproducible." "A reproducible module graph can still be installed or configured incorrectly."
], ],
"recommendation": "Use the minimal Campaign composition and pin all artifacts before promotion.", "recommendation": "Use the signed stable catalog and verify the minimal Campaign composition after installation.",
"proof_check": "Run contract, migration, API and WebUI module-permutation gates on the release candidate." "proof_check": "Run contract, migration, API and WebUI module-permutation gates on the installed release."
}, },
{ {
"id": "access.local", "id": "access.local",
@@ -348,21 +349,27 @@
}, },
{ {
"kind": "test", "kind": "test",
"scope": "current_workspace", "scope": "committed_source",
"locator": "govoplan-campaign/tests", "locator": "govoplan-campaign/tests",
"note": "14 tests passed" "note": "Campaign v0.1.10 is exactly the catalog-selected tagged source"
},
{
"kind": "configuration",
"scope": "committed_source",
"locator": "https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json#sequence-202607220843",
"note": "Core v0.1.13 and Campaign v0.1.10 have matching catalogued Python and WebUI refs"
} }
], ],
"conditions": [ "conditions": [
"This verifies implementation paths, not target-provider delivery." "This verifies implementation paths, not target-provider delivery."
], ],
"gaps": [ "gaps": [
"Substantial Campaign UI/code WIP is not release-integrated." "Usability and target-provider acceptance remain separate."
], ],
"risks": [ "risks": [
"Local WIP can make the checkout look more complete than the release baseline." "Package integration does not prove provider behavior or production operations."
], ],
"recommendation": "Integrate and retest Campaign work before usability or production acceptance.", "recommendation": "Use the catalogued Campaign release for usability and target-provider acceptance.",
"proof_check": "Run the complete journey with safe data and the target-like mail service." "proof_check": "Run the complete journey with safe data and the target-like mail service."
}, },
{ {
@@ -500,7 +507,7 @@
{ {
"kind": "issue", "kind": "issue",
"scope": "documented_model", "scope": "documented_model",
"locator": "https://git.add-ideas.de/add-ideas/govoplan/issues/12" "locator": "https://git.add-ideas.de/GovOPlaN/govoplan/issues/12"
} }
], ],
"conditions": [], "conditions": [],
@@ -554,7 +561,7 @@
} }
], ],
"conditions": [ "conditions": [
"Build immutable matching artifacts for promotion." "Materialize the matching catalogued artifacts in the target."
], ],
"gaps": [ "gaps": [
"No production image or service bundle is supplied by the profile." "No production image or service bundle is supplied by the profile."
@@ -562,7 +569,7 @@
"risks": [ "risks": [
"Editable source processes are unsuitable as a production artifact." "Editable source processes are unsuitable as a production artifact."
], ],
"recommendation": "Package and supervise WebUI/API from one release candidate.", "recommendation": "Install matching catalogued WebUI/API refs and supervise them as immutable artifacts.",
"proof_check": "Deploy the built artifacts and run health/module-route checks." "proof_check": "Deploy the built artifacts and run health/module-route checks."
}, },
{ {
@@ -595,12 +602,13 @@
"evidence": [ "evidence": [
{ {
"kind": "test", "kind": "test",
"scope": "current_workspace", "scope": "committed_source",
"locator": "govoplan-core/tests/test_calendar_outbox_worker.py" "locator": "govoplan-calendar/tests/test_outbox.py",
"note": "Committed and pushed after the catalogued Calendar v0.1.8 tag"
} }
], ],
"conditions": [ "conditions": [
"Relevant Calendar work is local WIP." "Calendar outbox and recovery work is remote-integrated source but not stable-package-integrated."
], ],
"gaps": [ "gaps": [
"No distributed leader election or target supervision is established." "No distributed leader election or target supervision is established."
@@ -785,7 +793,7 @@
{ {
"kind": "issue", "kind": "issue",
"scope": "documented_model", "scope": "documented_model",
"locator": "https://git.add-ideas.de/add-ideas/govoplan-core/issues/29" "locator": "https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29"
} }
], ],
"conditions": [], "conditions": [],
@@ -928,9 +936,9 @@
"risks": [ "risks": [
{ {
"id": "risk.reproducibility", "id": "risk.reproducibility",
"statement": "The dirty, untagged workspace cannot be reproduced as a release.", "statement": "The signed package selection is reproducible but has not been accepted as an installed target composition.",
"impact": "Uncertain deployed behavior and unsafe promotion or rollback.", "impact": "Installation or configuration drift can still produce uncertain deployed behavior.",
"treatment": "Integrate scoped work and create a clean pinned release candidate.", "treatment": "Materialize the signed catalog in an isolated target and run installed-artifact acceptance gates.",
"owner": null, "owner": null,
"residual_risk": "Module and environment differences still require release-environment verification." "residual_risk": "Module and environment differences still require release-environment verification."
}, },
@@ -955,14 +963,16 @@
"Proceed only with a controlled internal Campaign pilot after the bounded proof checks pass.", "Proceed only with a controlled internal Campaign pilot after the bounded proof checks pass.",
"Use the minimal composition and enable Addresses only for an explicit reusable-recipient journey.", "Use the minimal composition and enable Addresses only for an explicit reusable-recipient journey.",
"Do not claim Workflow, export-control screening, identity federation or production DR as implemented.", "Do not claim Workflow, export-control screening, identity federation or production DR as implemented.",
"Treat a clean release candidate, target mail proof, monitoring and coherent restore drill as production gates." "Treat installed-release acceptance, target mail proof, monitoring and a coherent restore drill as production gates."
], ],
"proof_checks": [ "proof_checks": [
"Pin and build a clean matching backend/WebUI release candidate and rerun cross-repository gates.", "Materialize the signed catalog into an isolated installation and rerun contract, migration and module-permutation gates against the installed artifacts.",
"Collect the isolated installation with the bounded installed-composition evidence contract; require exact enabled package/module versions, complete RECORD verification and immutable provenance anchored to this assessment.",
"Run a safe target-like Campaign through SMTP acceptance, IMAP append, reporting and audit.", "Run a safe target-like Campaign through SMTP acceptance, IMAP append, reporting and audit.",
"Drill worker, Redis and ambiguous-delivery failures without duplicate sends.", "Drill worker, Redis and ambiguous-delivery failures without duplicate sends.",
"Restore PostgreSQL, managed files, configuration and encrypted credentials and measure RPO/RTO.", "Restore PostgreSQL, managed files, configuration and encrypted credentials and measure RPO/RTO.",
"Validate proxy/TLS, cookies/CORS, account bootstrap, secret redaction, monitoring and alert delivery.", "Validate proxy/TLS, cookies/CORS, account bootstrap, secret redaction, monitoring and alert delivery.",
"Measure representative Campaign/file/queue/database load and external throttling." "Measure representative Campaign/file/queue/database load and external throttling.",
"Require separately issued, expiring and independently scope-authorized evidence before marking target environment, external provider or production approval proof as checked."
] ]
} }
@@ -0,0 +1,80 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/capability-fit-proof-authority-keyring.schema.json",
"title": "GovOPlaN capability-fit proof authority keyring",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "purpose", "keys"],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.1.0"
},
"purpose": {
"const": "govoplan.capability-fit-proof-authorities"
},
"keys": {
"type": "array",
"maxItems": 64,
"items": {
"$ref": "#/$defs/key"
}
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"key": {
"type": "object",
"additionalProperties": false,
"required": ["key_id", "status", "public_key", "allowed_scopes"],
"properties": {
"key_id": {
"$ref": "#/$defs/opaque_id"
},
"status": {
"enum": ["active", "next", "revoked", "disabled", "retired"]
},
"public_key": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
},
"allowed_scopes": {
"type": "array",
"minItems": 1,
"maxItems": 8,
"uniqueItems": true,
"items": {
"enum": [
"target_environment",
"external_providers",
"accessibility",
"privacy",
"security",
"operations",
"recovery",
"production_approval"
]
}
},
"not_before": {
"type": "string",
"format": "date-time"
},
"not_after": {
"type": "string",
"format": "date-time"
}
}
}
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"$schema": "https://json-schema.org/draft/2020-12/schema", "$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit.schema.json", "$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/capability-fit.schema.json",
"title": "GovOPlaN capability and infrastructure fit assessment", "title": "GovOPlaN capability and infrastructure fit assessment",
"type": "object", "type": "object",
"additionalProperties": false, "additionalProperties": false,
+108
View File
@@ -113,6 +113,12 @@
"description": "GovOPlaN Appointments module behavior or integration.", "description": "GovOPlaN Appointments module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/approvals",
"color": "d93f0b",
"description": "GovOPlaN Approvals module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/audit", "name": "module/audit",
"color": "0e8a16", "color": "0e8a16",
@@ -143,30 +149,72 @@
"description": "GovOPlaN Connectors module behavior or integration.", "description": "GovOPlaN Connectors module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/committee",
"color": "5319e7",
"description": "GovOPlaN Committee module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/core", "name": "module/core",
"color": "0052cc", "color": "0052cc",
"description": "GovOPlaN core runner, shared primitives, shell, or extension points.", "description": "GovOPlaN core runner, shared primitives, shell, or extension points.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/dashboard",
"color": "1d76db",
"description": "GovOPlaN Dashboard module behavior or integration.",
"exclusive": false
},
{
"name": "module/dataflow",
"color": "1d76db",
"description": "GovOPlaN Dataflow module behavior or integration.",
"exclusive": false
},
{
"name": "module/datasources",
"color": "006b75",
"description": "GovOPlaN governed datasource contracts, catalogs, and integrations.",
"exclusive": false
},
{ {
"name": "module/dms", "name": "module/dms",
"color": "c5def5", "color": "c5def5",
"description": "GovOPlaN Dms module behavior or integration.", "description": "GovOPlaN Dms module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/docs",
"color": "c5def5",
"description": "GovOPlaN Docs module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/dist-lists", "name": "module/dist-lists",
"color": "0e8a16", "color": "0e8a16",
"description": "GovOPlaN Distribution Lists module behavior or integration.", "description": "GovOPlaN Distribution Lists module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/decisions",
"color": "d93f0b",
"description": "GovOPlaN formal Decisions module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/erp", "name": "module/erp",
"color": "fef2c0", "color": "fef2c0",
"description": "GovOPlaN Erp module behavior or integration.", "description": "GovOPlaN Erp module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/encryption",
"color": "b60205",
"description": "GovOPlaN Encryption key custody, cryptographic policy, and E2EE integration.",
"exclusive": false
},
{ {
"name": "module/evaluation", "name": "module/evaluation",
"color": "bfdadc", "color": "bfdadc",
@@ -191,6 +239,18 @@
"description": "GovOPlaN Forms module behavior or integration.", "description": "GovOPlaN Forms module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/forms-runtime",
"color": "f9d0c4",
"description": "GovOPlaN Forms Runtime module behavior or integration.",
"exclusive": false
},
{
"name": "module/helpdesk",
"color": "c2e0c6",
"description": "GovOPlaN Helpdesk module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/identity-trust", "name": "module/identity-trust",
"color": "d4c5f9", "color": "d4c5f9",
@@ -221,6 +281,12 @@
"description": "GovOPlaN mail module behavior or integration.", "description": "GovOPlaN mail module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/mandates",
"color": "006b75",
"description": "GovOPlaN Mandates, jurisdiction, responsibility, and authority behavior or integration.",
"exclusive": false
},
{ {
"name": "module/notifications", "name": "module/notifications",
"color": "d876e3", "color": "d876e3",
@@ -245,6 +311,12 @@
"description": "GovOPlaN Payments module behavior or integration.", "description": "GovOPlaN Payments module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/parties",
"color": "bfd4f2",
"description": "GovOPlaN procedure Parties, representation, and delivery-authority behavior or integration.",
"exclusive": false
},
{ {
"name": "module/permits", "name": "module/permits",
"color": "fbca04", "color": "fbca04",
@@ -275,12 +347,24 @@
"description": "GovOPlaN Postbox module behavior or integration.", "description": "GovOPlaN Postbox module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/projects",
"color": "5319e7",
"description": "GovOPlaN Projects module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/reporting", "name": "module/reporting",
"color": "c2e0c6", "color": "c2e0c6",
"description": "GovOPlaN Reporting module behavior or integration.", "description": "GovOPlaN Reporting module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/risk-compliance",
"color": "b60205",
"description": "GovOPlaN Risk Compliance module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/search", "name": "module/search",
"color": "bfdadc", "color": "bfdadc",
@@ -293,6 +377,12 @@
"description": "GovOPlaN Scheduling module behavior or integration.", "description": "GovOPlaN Scheduling module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/services",
"color": "1d76db",
"description": "GovOPlaN versioned institutional Services behavior or integration.",
"exclusive": false
},
{ {
"name": "module/tasks", "name": "module/tasks",
"color": "c5def5", "color": "c5def5",
@@ -311,6 +401,24 @@
"description": "GovOPlaN Tenancy module behavior or integration.", "description": "GovOPlaN Tenancy module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/tickets",
"color": "0e8a16",
"description": "GovOPlaN Tickets module behavior or integration.",
"exclusive": false
},
{
"name": "module/views",
"color": "c5def5",
"description": "GovOPlaN governed task views, interface projections, and workflow view integration.",
"exclusive": false
},
{
"name": "module/wiki",
"color": "006b75",
"description": "GovOPlaN Wiki module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/workflow", "name": "module/workflow",
"color": "f9d0c4", "color": "f9d0c4",
+320
View File
@@ -0,0 +1,320 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://govoplan.add-ideas.de/schemas/installation-spec-v1.json",
"title": "GovOPlaN installation specification",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"installation_id",
"profile",
"public_url",
"listen",
"network_subnet",
"release",
"components",
"enabled_modules"
],
"properties": {
"schema_version": {
"const": 1
},
"installation_id": {
"type": "string",
"pattern": "^[a-z][a-z0-9-]{1,47}$"
},
"profile": {
"enum": [
"evaluation",
"self-hosted"
]
},
"public_url": {
"type": "string",
"format": "uri",
"pattern": "^https?://"
},
"listen": {
"type": "object",
"additionalProperties": false,
"required": [
"address",
"port"
],
"properties": {
"address": {
"type": "string"
},
"port": {
"type": "integer",
"minimum": 1,
"maximum": 65535
}
}
},
"network_subnet": {
"type": "string"
},
"release": {
"type": "object",
"additionalProperties": false,
"required": [
"channel",
"version",
"manifest_url",
"manifest_sha256",
"api_image",
"web_image"
],
"properties": {
"channel": {
"type": "string",
"pattern": "^[a-z][a-z0-9-]{1,31}$"
},
"version": {
"type": "string",
"minLength": 1,
"maxLength": 80
},
"manifest_url": {
"type": "string"
},
"manifest_sha256": {
"type": "string",
"pattern": "^$|^[0-9a-f]{64}$"
},
"api_image": {
"type": "string",
"minLength": 1,
"maxLength": 300
},
"web_image": {
"type": "string",
"minLength": 1,
"maxLength": 300
}
}
},
"components": {
"type": "object",
"additionalProperties": false,
"required": [
"postgres",
"redis",
"mail",
"storage"
],
"properties": {
"postgres": {
"$ref": "#/$defs/postgres"
},
"redis": {
"$ref": "#/$defs/redis"
},
"mail": {
"$ref": "#/$defs/mail"
},
"storage": {
"type": "object",
"additionalProperties": false,
"required": [
"mode"
],
"properties": {
"mode": {
"enum": [
"local",
"garage",
"s3"
]
},
"image": {
"type": "string",
"maxLength": 300
}
}
},
"load_balancer": {
"$ref": "#/$defs/load_balancer"
}
}
},
"replicas": {
"type": "object",
"additionalProperties": false,
"required": [
"api",
"web",
"worker"
],
"properties": {
"api": {
"type": "integer",
"minimum": 1,
"maximum": 64
},
"web": {
"type": "integer",
"minimum": 1,
"maximum": 64
},
"worker": {
"type": "integer",
"minimum": 0,
"maximum": 128
}
}
},
"enabled_modules": {
"type": "array",
"uniqueItems": true,
"items": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]{1,63}$"
}
}
},
"$defs": {
"service": {
"type": "object",
"additionalProperties": false,
"required": [
"mode",
"image",
"url_env"
],
"properties": {
"mode": {
"type": "string"
},
"image": {
"type": "string"
},
"url_env": {
"type": "string"
}
}
},
"postgres": {
"allOf": [
{
"$ref": "#/$defs/service"
},
{
"properties": {
"mode": {
"enum": [
"managed",
"external"
]
},
"url_env": {
"const": "DATABASE_URL"
}
}
}
]
},
"redis": {
"allOf": [
{
"$ref": "#/$defs/service"
},
{
"properties": {
"mode": {
"enum": [
"managed",
"external",
"disabled"
]
},
"url_env": {
"const": "REDIS_URL"
}
}
}
]
},
"mail": {
"allOf": [
{
"$ref": "#/$defs/service"
},
{
"properties": {
"mode": {
"enum": [
"disabled",
"external-relay",
"test-mail"
]
},
"url_env": {
"const": ""
}
}
}
]
},
"load_balancer": {
"allOf": [
{
"$ref": "#/$defs/service"
},
{
"properties": {
"mode": {
"const": "managed"
},
"url_env": {
"const": ""
}
}
}
]
}
},
"allOf": [
{
"if": {
"properties": {
"profile": {
"const": "self-hosted"
}
}
},
"then": {
"properties": {
"public_url": {
"pattern": "^https://"
},
"components": {
"properties": {
"redis": {
"properties": {
"mode": {
"enum": [
"managed",
"external"
]
}
}
},
"mail": {
"properties": {
"mode": {
"enum": [
"disabled",
"external-relay"
]
}
}
}
}
}
}
}
}
]
}
@@ -0,0 +1,335 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/installed-composition-evidence.schema.json",
"title": "GovOPlaN installed composition evidence",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"assessment_id",
"assessment_release",
"collected_at",
"scope",
"artifacts",
"collection_issues"
],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.4.0"
},
"evidence_kind": {
"const": "govoplan.installed-composition"
},
"assessment_id": {
"$ref": "#/$defs/opaque_id"
},
"assessment_release": {
"$ref": "#/$defs/opaque_id"
},
"collected_at": {
"type": "string",
"format": "date-time"
},
"scope": {
"const": "current-python-environment.govoplan-distributions"
},
"artifacts": {
"type": "array",
"maxItems": 256,
"items": {
"$ref": "#/$defs/artifact"
}
},
"collection_issues": {
"type": "array",
"maxItems": 256,
"items": {
"$ref": "#/$defs/collection_issue"
}
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"package_name": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
},
"version": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._+!-]*$"
},
"artifact": {
"type": "object",
"additionalProperties": false,
"required": [
"package_name",
"package_version",
"modules",
"source_provenance",
"record_integrity"
],
"properties": {
"package_name": {
"$ref": "#/$defs/package_name"
},
"package_version": {
"$ref": "#/$defs/version"
},
"modules": {
"type": "array",
"maxItems": 16,
"items": {
"$ref": "#/$defs/module"
}
},
"source_provenance": {
"$ref": "#/$defs/source_provenance"
},
"record_integrity": {
"$ref": "#/$defs/record_integrity"
}
}
},
"module": {
"type": "object",
"additionalProperties": false,
"required": ["module_id", "manifest_version"],
"properties": {
"module_id": {
"$ref": "#/$defs/opaque_id"
},
"manifest_version": {
"$ref": "#/$defs/version"
}
}
},
"source_provenance": {
"oneOf": [
{
"type": "object",
"additionalProperties": false,
"required": ["basis", "kind", "commit"],
"properties": {
"basis": {
"const": "local-pep610-metadata"
},
"kind": {
"const": "vcs-commit"
},
"commit": {
"type": "string",
"pattern": "^[0-9a-f]{40,64}$"
}
}
},
{
"type": "object",
"additionalProperties": false,
"required": ["basis", "kind", "sha256"],
"properties": {
"basis": {
"const": "local-pep610-metadata"
},
"kind": {
"const": "archive"
},
"sha256": {
"$ref": "#/$defs/sha256"
}
}
},
{
"type": "object",
"additionalProperties": false,
"required": ["basis", "kind"],
"properties": {
"basis": {
"const": "local-pep610-metadata"
},
"kind": {
"enum": [
"editable-local",
"local-directory",
"index-or-unknown",
"malformed-direct-url"
]
}
}
}
]
},
"record_integrity": {
"type": "object",
"additionalProperties": false,
"required": [
"status",
"hashed_file_count",
"permitted_unhashed_file_count",
"generated_unhashed_file_count",
"unverifiable_file_count",
"missing_file_count",
"mismatched_file_count",
"artifact_payload_identity",
"installed_payload_identity"
],
"properties": {
"status": {
"enum": [
"verified",
"partial",
"mismatch",
"unavailable",
"limit-exceeded"
]
},
"hashed_file_count": {
"$ref": "#/$defs/count"
},
"permitted_unhashed_file_count": {
"$ref": "#/$defs/count"
},
"generated_unhashed_file_count": {
"$ref": "#/$defs/count"
},
"unverifiable_file_count": {
"$ref": "#/$defs/count"
},
"missing_file_count": {
"$ref": "#/$defs/count"
},
"mismatched_file_count": {
"$ref": "#/$defs/count"
},
"artifact_payload_identity": {
"oneOf": [
{ "$ref": "#/$defs/artifact_payload_identity" },
{ "type": "null" }
]
},
"installed_payload_identity": {
"oneOf": [
{ "$ref": "#/$defs/installed_payload_identity" },
{ "type": "null" }
]
}
},
"allOf": [
{
"if": { "properties": { "status": { "const": "verified" } } },
"then": {
"properties": {
"hashed_file_count": { "minimum": 1 },
"permitted_unhashed_file_count": { "minimum": 1 },
"unverifiable_file_count": { "const": 0 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 },
"artifact_payload_identity": { "$ref": "#/$defs/artifact_payload_identity" },
"installed_payload_identity": { "$ref": "#/$defs/installed_payload_identity" }
}
}
},
{
"if": { "properties": { "status": { "const": "partial" } } },
"then": {
"properties": {
"hashed_file_count": { "minimum": 1 },
"unverifiable_file_count": { "minimum": 1 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 }
}
}
},
{
"if": { "properties": { "status": { "const": "mismatch" } } },
"then": {
"anyOf": [
{ "properties": { "missing_file_count": { "minimum": 1 } } },
{ "properties": { "mismatched_file_count": { "minimum": 1 } } }
]
}
},
{
"if": { "properties": { "status": { "const": "unavailable" } } },
"then": {
"properties": {
"hashed_file_count": { "const": 0 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 }
}
}
}
]
},
"collection_issue": {
"type": "object",
"additionalProperties": false,
"required": ["code", "package_name"],
"properties": {
"code": {
"enum": [
"distribution-metadata-invalid",
"duplicate-distribution",
"module-entry-point-load-failed",
"module-entry-point-invalid",
"module-entry-point-limit-exceeded",
"collection-limit-exceeded"
]
},
"package_name": {
"$ref": "#/$defs/package_name"
}
}
},
"artifact_payload_identity": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-wheel-declared-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
},
"installed_payload_identity": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-installed-record-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
},
"count": {
"type": "integer",
"minimum": 0,
"maximum": 1000000
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}
@@ -0,0 +1,53 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/installer-receipt-authority-keyring.schema.json",
"title": "GovOPlaN installer receipt authority keyring",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "purpose", "keys"],
"properties": {
"$schema": { "type": "string", "format": "uri-reference" },
"schema_version": { "const": "0.1.0" },
"purpose": { "const": "govoplan.installer-receipt-authorities" },
"keys": {
"type": "array",
"minItems": 1,
"maxItems": 64,
"items": { "$ref": "#/$defs/key" }
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"key": {
"type": "object",
"additionalProperties": false,
"required": ["key_id", "status", "public_key", "allowed_scopes"],
"properties": {
"key_id": { "$ref": "#/$defs/opaque_id" },
"status": {
"enum": ["active", "next", "revoked", "disabled", "retired"]
},
"public_key": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
},
"allowed_scopes": {
"type": "array",
"minItems": 1,
"maxItems": 1,
"uniqueItems": true,
"items": { "const": "installed_release_origin" }
},
"not_before": { "type": "string", "format": "date-time" },
"not_after": { "type": "string", "format": "date-time" }
}
}
}
}
+122
View File
@@ -0,0 +1,122 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/installer-receipt.schema.json",
"title": "GovOPlaN signed installer receipt",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"receipt_id",
"assessment_id",
"assessment_release",
"installed_evidence_sha256",
"catalog",
"issued_at",
"artifacts",
"signatures"
],
"properties": {
"$schema": { "type": "string", "format": "uri-reference" },
"schema_version": { "const": "0.1.0" },
"evidence_kind": { "const": "govoplan.installer-receipt" },
"receipt_id": { "$ref": "#/$defs/opaque_id" },
"assessment_id": { "$ref": "#/$defs/opaque_id" },
"assessment_release": { "$ref": "#/$defs/opaque_id" },
"installed_evidence_sha256": { "$ref": "#/$defs/sha256" },
"catalog": {
"type": "object",
"additionalProperties": false,
"required": ["channel", "sequence", "sha256"],
"properties": {
"channel": {
"type": "string",
"pattern": "^[a-z][a-z0-9_-]{0,63}$"
},
"sequence": { "type": "integer", "minimum": 1 },
"sha256": { "$ref": "#/$defs/sha256" }
}
},
"issued_at": { "type": "string", "format": "date-time" },
"artifacts": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"items": { "$ref": "#/$defs/artifact" }
},
"signatures": {
"type": "array",
"minItems": 1,
"maxItems": 16,
"items": { "$ref": "#/$defs/signature" }
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"package_name": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
},
"version": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._+!-]*$"
},
"artifact": {
"type": "object",
"additionalProperties": false,
"required": [
"package_name",
"package_version",
"catalog_archive_sha256",
"installed_payload"
],
"properties": {
"package_name": { "$ref": "#/$defs/package_name" },
"package_version": { "$ref": "#/$defs/version" },
"catalog_archive_sha256": { "$ref": "#/$defs/sha256" },
"installed_payload": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-installed-record-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
}
}
},
"signature": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "key_id", "value"],
"properties": {
"algorithm": { "const": "ed25519" },
"key_id": { "$ref": "#/$defs/opaque_id" },
"value": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
}
}
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}
@@ -0,0 +1,32 @@
# Governed Communication Product Package
This package composes Campaign, Mail, Postbox, Files, Templates, Policy, Audit,
and Access into one governed-delivery capability. Modules retain their own
tables and lifecycle. The package declares installation and configuration
expectations; it does not import module internals.
## Reference journey
1. Select governed recipient and attachment data.
2. Build and review an immutable Campaign version.
3. Apply attachment, access, and delivery policy.
4. Deliver through Mail and/or function-bound Postbox targets.
5. Reconcile outcomes, acknowledgements, correction, and reports against Audit
evidence.
## Reference-readiness gates
The artifact remains a `product` package. Promotion to `reference` requires:
- target-environment delivery and reconciliation tests, including an
outcome-unknown transport result;
- backup/restore and interrupted-dispatch recovery evidence;
- security evidence for recipient, attachment, and postbox isolation;
- operator runbooks for queue health, retry, reconciliation, and retirement;
- keyboard, focus, screen-reader, and responsive workflow evidence;
- privacy evidence for minimization, purpose, retention, redaction, and report
access; and
- version-pinned user and administrator documentation.
Optional Notifications, Portal, Reporting, and Workflow Engine integrations do
not change the package boundary when absent.
@@ -0,0 +1,34 @@
{
"package_id": "product.governed-communication",
"name": "Governed Communication",
"version": "0.1.0",
"package_class": "product",
"description": "Compose evidence-backed campaigns, mail and function-bound postboxes without merging their domain ownership.",
"publisher": "GovOPlaN",
"category": "communication",
"license": "AGPL-3.0-or-later",
"required_modules": [
{"module_id": "access"},
{"module_id": "audit"},
{"module_id": "campaign"},
{"module_id": "files"},
{"module_id": "mail"},
{"module_id": "policy"},
{"module_id": "postbox"},
{"module_id": "templates"}
],
"optional_modules": [
{"module_id": "notifications"},
{"module_id": "portal"},
{"module_id": "reporting"},
{"module_id": "workflow_engine"}
],
"evidence": [
{
"kind": "documentation",
"reference": "packages/product/governed-communication/README.md",
"summary": "Defines the package boundary, journey, and reference-readiness gates."
}
],
"tags": ["campaign", "mail", "postbox", "public-sector"]
}
@@ -0,0 +1,41 @@
# Governed Data and Assurance Product Package
This package composes Datasources, Dataflow, Reporting, Search, Risk
Compliance, Policy, Audit, and Access. Connectors may acquire external data,
but Datasources owns the governed catalogue and immutable snapshots; Dataflow
owns transformation definitions and runs; Reporting owns measures and
presentation; Risk Compliance owns controls, findings, and effectiveness
review.
## Reference journey
1. Register a typed datasource with source authority, purpose, classification,
owner, freshness, and correction policy.
2. Acquire or upload an immutable source state.
3. execute a versioned flow and retain intermediate materializations and
provenance;
4. publish a report or decision input against exact source and flow revisions;
5. link obligation, governed object, risk, control, evidence, finding,
corrective measure, and effectiveness review; and
6. search current authorized objects while preserving ownership and access
rechecks.
## Reference-readiness gates
The artifact remains a `product` package. Promotion to `reference` requires:
- a target-tested monthly-data and sanctions-screening fixture with expected
outputs and complete provenance;
- database/object-storage backup and restore plus interrupted-run recovery;
- security evidence for datasource credentials, staged data, intermediate
states, search documents, and assurance references;
- operator runbooks for stale sources, failed runs, index rebuilds, and graph
reconciliation;
- accessibility evidence for the catalogue, graph editors, result inspection,
reports, and assurance graph;
- privacy evidence for minimization, purpose, retention, field visibility, and
aggregate disclosure; and
- version-pinned user and administrator documentation.
Optional Connectors, Files, Notifications, and Workflow Engine integrations
must remain capability-based and absence-safe.
@@ -0,0 +1,34 @@
{
"package_id": "product.governed-data-assurance",
"name": "Governed Data and Assurance",
"version": "0.1.0",
"package_class": "product",
"description": "Compose governed sources, transformations, reports, search, controls and assurance evidence through stable contracts.",
"publisher": "GovOPlaN",
"category": "data-governance",
"license": "AGPL-3.0-or-later",
"required_modules": [
{"module_id": "access"},
{"module_id": "audit"},
{"module_id": "dataflow"},
{"module_id": "datasources"},
{"module_id": "policy"},
{"module_id": "reporting"},
{"module_id": "risk_compliance"},
{"module_id": "search"}
],
"optional_modules": [
{"module_id": "connectors"},
{"module_id": "files"},
{"module_id": "notifications"},
{"module_id": "workflow_engine"}
],
"evidence": [
{
"kind": "documentation",
"reference": "packages/product/governed-data-assurance/README.md",
"summary": "Defines the package boundary, provenance chain, and reference-readiness gates."
}
],
"tags": ["datasources", "dataflow", "reporting", "assurance"]
}
@@ -0,0 +1,42 @@
# Governed Service To Decision
This product package composes independently owned institutional semantics into
one reconstructable administrative journey:
```text
Service discovery -> Case intake -> Party and representation -> Mandate
resolution -> approval/deliberation -> formal Decision -> observed delivery
effect -> record and review references
```
An installed Forms and Forms Runtime pair adds an alternative governed entry
path before case/workflow handoff:
```text
Service discovery -> exact Form revision -> validated draft/submission
-> receipt and handoff evidence -> Case or Workflow owner
```
Services, Cases, Parties, Mandates, Committee, and Decisions retain immutable
provider-owned revisions for the parts they own. Portal, Cases, and Committee
consume capabilities for cross-module semantics only. The package does not
grant cross-module table access and can omit optional presentation, work,
deliberation, delivery, or records modules while retaining explicit references
to externally performed steps.
## Security And Recovery
Every provider is tenant-bound. Missing or conflicting authority fails closed.
Protected Decision content has a separate permission. Writes are replay-safe
and OCC-guarded. Database restore is the semantic-state recovery unit; file and
communication effects remain governed by their owning providers and are linked
through requested/observed effect, evidence, and audit references.
The executable fixture in
`tests/test_institutional_governance_journey.py` proves SQL-backed Service,
Case, Party, Mandate, Committee meeting/agendum/vote/minute, and Decision state.
`tests/test_institutional_service_journey.py` separately proves exact Portal
Form launch, persisted submission provenance, and idempotent replay.
Target-environment accessibility, security, operator, privacy,
delivery-provider, and recovery evidence are still required before this product
package may claim `reference_ready` maturity.
@@ -0,0 +1,59 @@
{
"package_id": "product.service-to-decision",
"name": "Governed Service To Decision",
"version": "0.1.0",
"package_class": "product",
"description": "Carry one exact institutional context from service discovery and case intake through parties, authority, formal outcome, communication evidence, and review.",
"publisher": "GovOPlaN",
"category": "institutional-governance",
"license": "AGPL-3.0-or-later",
"required_modules": [
{"module_id": "audit"},
{"module_id": "cases"},
{"module_id": "decisions"},
{"module_id": "mandates"},
{"module_id": "parties"},
{"module_id": "policy"},
{"module_id": "portal"},
{"module_id": "services"}
],
"required_capabilities": [
"cases.party_context",
"cases.service_intake",
"decisions.registry",
"mandates.resolver",
"parties.resolver",
"portal.service_directory",
"services.availability",
"services.definitions"
],
"optional_modules": [
{"module_id": "approvals"},
{"module_id": "committee"},
{"module_id": "files"},
{"module_id": "forms"},
{"module_id": "forms_runtime"},
{"module_id": "postbox"},
{"module_id": "records"},
{"module_id": "tasks"},
{"module_id": "workflow_engine"}
],
"evidence": [
{
"kind": "documentation",
"reference": "packages/product/service-to-decision/README.md",
"summary": "Defines the package boundary, authority path, recovery contract, and known operational limits."
},
{
"kind": "target_test",
"reference": "tests/test_institutional_governance_journey.py",
"summary": "Executes the SQL-backed provider composition from service discovery through persisted formal Decision reconstruction."
},
{
"kind": "target_test",
"reference": "tests/test_institutional_service_journey.py",
"summary": "Executes Portal delegation from an exact Service revision through an exact immutable Form revision to a replay-safe persisted submission."
}
],
"tags": ["service", "case", "mandate", "party", "decision", "public-sector"]
}
+79 -67
View File
@@ -1,73 +1,85 @@
{ {
"version": 1, "version": 1,
"organization": "add-ideas", "organization": "GovOPlaN",
"default_parent": "/mnt/DATA/git", "default_parent": "/mnt/DATA/git",
"repositories": [ "repositories": [
{"name": "govoplan", "category": "system", "subtype": "meta", "remote": "git@git.add-ideas.de:add-ideas/govoplan.git", "path": "govoplan"}, {"name": "govoplan", "category": "system", "subtype": "meta", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan.git", "path": "govoplan"},
{"name": "govoplan-core", "category": "system", "subtype": "kernel", "remote": "git@git.add-ideas.de:add-ideas/govoplan-core.git", "path": "govoplan-core"}, {"name": "govoplan-core", "category": "system", "subtype": "kernel", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-core.git", "path": "govoplan-core"},
{"name": "govoplan-access", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-access.git", "path": "govoplan-access"}, {"name": "govoplan-access", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-access.git", "path": "govoplan-access"},
{"name": "govoplan-addresses", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-addresses.git", "path": "govoplan-addresses"}, {"name": "govoplan-addresses", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-addresses.git", "path": "govoplan-addresses"},
{"name": "govoplan-admin", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-admin.git", "path": "govoplan-admin"}, {"name": "govoplan-admin", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-admin.git", "path": "govoplan-admin"},
{"name": "govoplan-appointments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-appointments.git", "path": "govoplan-appointments"}, {"name": "govoplan-appointments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-appointments.git", "path": "govoplan-appointments"},
{"name": "govoplan-approvals", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-approvals.git", "path": "govoplan-approvals"}, {"name": "govoplan-approvals", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-approvals.git", "path": "govoplan-approvals"},
{"name": "govoplan-assets", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-assets.git", "path": "govoplan-assets"}, {"name": "govoplan-assets", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-assets.git", "path": "govoplan-assets"},
{"name": "govoplan-audit", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-audit.git", "path": "govoplan-audit"}, {"name": "govoplan-audit", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-audit.git", "path": "govoplan-audit"},
{"name": "govoplan-booking", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-booking.git", "path": "govoplan-booking"}, {"name": "govoplan-booking", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-booking.git", "path": "govoplan-booking"},
{"name": "govoplan-calendar", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-calendar.git", "path": "govoplan-calendar"}, {"name": "govoplan-calendar", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-calendar.git", "path": "govoplan-calendar"},
{"name": "govoplan-campaign", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-campaign.git", "path": "govoplan-campaign"}, {"name": "govoplan-campaign", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-campaign.git", "path": "govoplan-campaign"},
{"name": "govoplan-cases", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-cases.git", "path": "govoplan-cases"}, {"name": "govoplan-cases", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-cases.git", "path": "govoplan-cases"},
{"name": "govoplan-certificates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-certificates.git", "path": "govoplan-certificates"}, {"name": "govoplan-certificates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-certificates.git", "path": "govoplan-certificates"},
{"name": "govoplan-committee", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-committee.git", "path": "govoplan-committee"}, {"name": "govoplan-committee", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-committee.git", "path": "govoplan-committee"},
{"name": "govoplan-connectors", "category": "connector", "subtype": "connector-hub", "remote": "git@git.add-ideas.de:add-ideas/govoplan-connectors.git", "path": "govoplan-connectors"}, {"name": "govoplan-connectors", "category": "connector", "subtype": "connector-hub", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-connectors.git", "path": "govoplan-connectors"},
{"name": "govoplan-consultation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-consultation.git", "path": "govoplan-consultation"}, {"name": "govoplan-consultation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-consultation.git", "path": "govoplan-consultation"},
{"name": "govoplan-contracts", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-contracts.git", "path": "govoplan-contracts"}, {"name": "govoplan-contracts", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-contracts.git", "path": "govoplan-contracts"},
{"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dashboard.git", "path": "govoplan-dashboard"}, {"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dashboard.git", "path": "govoplan-dashboard"},
{"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dms.git", "path": "govoplan-dms"}, {"name": "govoplan-dataflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dataflow.git", "path": "govoplan-dataflow"},
{"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dist-lists.git", "path": "govoplan-dist-lists"}, {"name": "govoplan-datasources", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-datasources.git", "path": "govoplan-datasources"},
{"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-docs.git", "path": "govoplan-docs"}, {"name": "govoplan-decisions", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-decisions.git", "path": "govoplan-decisions"},
{"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-erp.git", "path": "govoplan-erp"}, {"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dms.git", "path": "govoplan-dms"},
{"name": "govoplan-evaluation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-evaluation.git", "path": "govoplan-evaluation"}, {"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dist-lists.git", "path": "govoplan-dist-lists"},
{"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-facilities.git", "path": "govoplan-facilities"}, {"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-docs.git", "path": "govoplan-docs"},
{"name": "govoplan-files", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-files.git", "path": "govoplan-files"}, {"name": "govoplan-encryption", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-encryption.git", "path": "govoplan-encryption"},
{"name": "govoplan-fit-connect", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-fit-connect.git", "path": "govoplan-fit-connect"}, {"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-erp.git", "path": "govoplan-erp"},
{"name": "govoplan-forms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-forms.git", "path": "govoplan-forms"}, {"name": "govoplan-evaluation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-evaluation.git", "path": "govoplan-evaluation"},
{"name": "govoplan-forms-runtime", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-forms-runtime.git", "path": "govoplan-forms-runtime"}, {"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-facilities.git", "path": "govoplan-facilities"},
{"name": "govoplan-grants", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-grants.git", "path": "govoplan-grants"}, {"name": "govoplan-files", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-files.git", "path": "govoplan-files"},
{"name": "govoplan-helpdesk", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-helpdesk.git", "path": "govoplan-helpdesk"}, {"name": "govoplan-fit-connect", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-fit-connect.git", "path": "govoplan-fit-connect"},
{"name": "govoplan-identity", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-identity.git", "path": "govoplan-identity"}, {"name": "govoplan-forms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-forms.git", "path": "govoplan-forms"},
{"name": "govoplan-identity-trust", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-identity-trust.git", "path": "govoplan-identity-trust"}, {"name": "govoplan-forms-runtime", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-forms-runtime.git", "path": "govoplan-forms-runtime"},
{"name": "govoplan-idm", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-idm.git", "path": "govoplan-idm"}, {"name": "govoplan-grants", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-grants.git", "path": "govoplan-grants"},
{"name": "govoplan-inspections", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-inspections.git", "path": "govoplan-inspections"}, {"name": "govoplan-helpdesk", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-helpdesk.git", "path": "govoplan-helpdesk"},
{"name": "govoplan-issue-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-issue-reporting.git", "path": "govoplan-issue-reporting"}, {"name": "govoplan-identity", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-identity.git", "path": "govoplan-identity"},
{"name": "govoplan-learning", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-learning.git", "path": "govoplan-learning"}, {"name": "govoplan-identity-trust", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-identity-trust.git", "path": "govoplan-identity-trust"},
{"name": "govoplan-ledger", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-ledger.git", "path": "govoplan-ledger"}, {"name": "govoplan-idm", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-idm.git", "path": "govoplan-idm"},
{"name": "govoplan-mail", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-mail.git", "path": "govoplan-mail"}, {"name": "govoplan-inspections", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-inspections.git", "path": "govoplan-inspections"},
{"name": "govoplan-notifications", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-notifications.git", "path": "govoplan-notifications"}, {"name": "govoplan-learning", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-learning.git", "path": "govoplan-learning"},
{"name": "govoplan-ops", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-ops.git", "path": "govoplan-ops"}, {"name": "govoplan-ledger", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-ledger.git", "path": "govoplan-ledger"},
{"name": "govoplan-organizations", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-organizations.git", "path": "govoplan-organizations"}, {"name": "govoplan-mail", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-mail.git", "path": "govoplan-mail"},
{"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-payments.git", "path": "govoplan-payments"}, {"name": "govoplan-mandates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-mandates.git", "path": "govoplan-mandates"},
{"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-permits.git", "path": "govoplan-permits"}, {"name": "govoplan-notifications", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-notifications.git", "path": "govoplan-notifications"},
{"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-policy.git", "path": "govoplan-policy"}, {"name": "govoplan-ops", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-ops.git", "path": "govoplan-ops"},
{"name": "govoplan-poll", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-poll.git", "path": "govoplan-poll"}, {"name": "govoplan-organizations", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-organizations.git", "path": "govoplan-organizations"},
{"name": "govoplan-portal", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-portal.git", "path": "govoplan-portal"}, {"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-payments.git", "path": "govoplan-payments"},
{"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-postbox.git", "path": "govoplan-postbox"}, {"name": "govoplan-parties", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-parties.git", "path": "govoplan-parties"},
{"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-procurement.git", "path": "govoplan-procurement"}, {"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-permits.git", "path": "govoplan-permits"},
{"name": "govoplan-records", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-records.git", "path": "govoplan-records"}, {"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-policy.git", "path": "govoplan-policy"},
{"name": "govoplan-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-reporting.git", "path": "govoplan-reporting"}, {"name": "govoplan-poll", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-poll.git", "path": "govoplan-poll"},
{"name": "govoplan-resources", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-resources.git", "path": "govoplan-resources"}, {"name": "govoplan-portal", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-portal.git", "path": "govoplan-portal"},
{"name": "govoplan-rest", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:add-ideas/govoplan-rest.git", "path": "govoplan-rest"}, {"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-postbox.git", "path": "govoplan-postbox"},
{"name": "govoplan-risk-compliance", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-risk-compliance.git", "path": "govoplan-risk-compliance"}, {"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-procurement.git", "path": "govoplan-procurement"},
{"name": "govoplan-scheduling", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-scheduling.git", "path": "govoplan-scheduling"}, {"name": "govoplan-projects", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-projects.git", "path": "govoplan-projects"},
{"name": "govoplan-search", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-search.git", "path": "govoplan-search"}, {"name": "govoplan-records", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-records.git", "path": "govoplan-records"},
{"name": "govoplan-soap", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:add-ideas/govoplan-soap.git", "path": "govoplan-soap"}, {"name": "govoplan-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-reporting.git", "path": "govoplan-reporting"},
{"name": "govoplan-tasks", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-tasks.git", "path": "govoplan-tasks"}, {"name": "govoplan-resources", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-resources.git", "path": "govoplan-resources"},
{"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-templates.git", "path": "govoplan-templates"}, {"name": "govoplan-rest", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-rest.git", "path": "govoplan-rest"},
{"name": "govoplan-tenancy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-tenancy.git", "path": "govoplan-tenancy"}, {"name": "govoplan-risk-compliance", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-risk-compliance.git", "path": "govoplan-risk-compliance"},
{"name": "govoplan-transparency", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-transparency.git", "path": "govoplan-transparency"}, {"name": "govoplan-scheduling", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-scheduling.git", "path": "govoplan-scheduling"},
{"name": "addideas-govoplan-website", "category": "website", "subtype": "public-site", "remote": "git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git", "path": "addideas-govoplan-website"}, {"name": "govoplan-search", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-search.git", "path": "govoplan-search"},
{"name": "govoplan-workflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-workflow.git", "path": "govoplan-workflow"}, {"name": "govoplan-services", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-services.git", "path": "govoplan-services"},
{"name": "govoplan-xoev", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xoev.git", "path": "govoplan-xoev"}, {"name": "govoplan-soap", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-soap.git", "path": "govoplan-soap"},
{"name": "govoplan-xrechnung", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xrechnung.git", "path": "govoplan-xrechnung"}, {"name": "govoplan-tasks", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tasks.git", "path": "govoplan-tasks"},
{"name": "govoplan-xta-osci", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xta-osci.git", "path": "govoplan-xta-osci"} {"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-templates.git", "path": "govoplan-templates"},
{"name": "govoplan-tenancy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tenancy.git", "path": "govoplan-tenancy"},
{"name": "govoplan-tickets", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tickets.git", "path": "govoplan-tickets"},
{"name": "govoplan-transparency", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-transparency.git", "path": "govoplan-transparency"},
{"name": "govoplan-views", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-views.git", "path": "govoplan-views"},
{"name": "govoplan-voting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-voting.git", "path": "govoplan-voting"},
{"name": "govoplan-wiki", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-wiki.git", "path": "govoplan-wiki"},
{"name": "addideas-govoplan-website", "category": "website", "subtype": "public-site", "remote": "git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git", "path": "addideas-govoplan-website", "bootstrap_transport": "registered"},
{"name": "govoplan-workflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-workflow.git", "path": "govoplan-workflow"},
{"name": "govoplan-workflow-engine", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-workflow-engine.git", "path": "govoplan-workflow-engine"},
{"name": "govoplan-xoev", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-xoev.git", "path": "govoplan-xoev"},
{"name": "govoplan-xrechnung", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-xrechnung.git", "path": "govoplan-xrechnung"},
{"name": "govoplan-xta-osci", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-xta-osci.git", "path": "govoplan-xta-osci"}
] ]
} }
+25
View File
@@ -12,24 +12,49 @@
-e ../govoplan-admin -e ../govoplan-admin
-e ../govoplan-policy -e ../govoplan-policy
-e ../govoplan-audit -e ../govoplan-audit
-e ../govoplan-approvals
-e ../govoplan-dashboard -e ../govoplan-dashboard
-e ../govoplan-addresses -e ../govoplan-addresses
-e ../govoplan-dist-lists -e ../govoplan-dist-lists
-e ../govoplan-files -e ../govoplan-files
-e ../govoplan-forms
-e ../govoplan-forms-runtime
-e ../govoplan-mail -e ../govoplan-mail
-e ../govoplan-campaign -e ../govoplan-campaign
-e ../govoplan-calendar -e ../govoplan-calendar
-e ../govoplan-committee
-e ../govoplan-cases
-e ../govoplan-portal
-e ../govoplan-services
-e ../govoplan-parties
-e ../govoplan-mandates
-e ../govoplan-decisions
-e ../govoplan-connectors
-e ../govoplan-datasources
-e ../govoplan-dataflow
-e ../govoplan-workflow-engine
-e ../govoplan-workflow
-e ../govoplan-views
-e ../govoplan-voting
-e ../govoplan-search
-e ../govoplan-risk-compliance
-e ../govoplan-postbox
-e ../govoplan-poll -e ../govoplan-poll
-e ../govoplan-scheduling -e ../govoplan-scheduling
-e ../govoplan-notifications -e ../govoplan-notifications
-e ../govoplan-evaluation -e ../govoplan-evaluation
-e ../govoplan-docs -e ../govoplan-docs
-e ../govoplan-encryption
-e ../govoplan-identity-trust
-e ../govoplan-ops -e ../govoplan-ops
httpx==0.28.1 httpx==0.28.1
httpx2>=2.5,<3 httpx2>=2.5,<3
filelock>=3.20.3 filelock>=3.20.3
idna>=3.15 idna>=3.15
jsonschema>=4,<5
pip>=26.1.2 pip>=26.1.2
pip-audit>=2.9,<3 pip-audit>=2.9,<3
pytest>=9.0.3,<10
pygments>=2.20,<3
python-multipart>=0.0.31 python-multipart>=0.0.31
ruff>=0.14,<1 ruff>=0.14,<1
+5
View File
@@ -0,0 +1,5 @@
# Test-harness dependencies used against immutable release source tags.
# Keep these separate from requirements-release.txt so they are not part of the
# deployable product dependency set.
pytest>=9.0.3,<10
pygments>=2.20,<3
+17 -23
View File
@@ -1,24 +1,18 @@
# Whole-product release install from tagged module repositories. # Whole-product release install from immutable, independently versioned module tags.
# Update GOVOPLAN_RELEASE_TAG together with pyproject/package versions when # Only add a module after its referenced tag has been published.
# cutting a release.
../govoplan-core[server] ../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-tenancy.git@v0.1.8 govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-organizations.git@v0.1.8 govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-identity.git@v0.1.8 govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-idm.git@v0.1.8 govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-access.git@v0.1.8 govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-admin.git@v0.1.8 govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-policy.git@v0.1.8 govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-audit.git@v0.1.8 govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git@v0.1.8 govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.8
govoplan-addresses @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-addresses.git@v0.1.8 govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.8 govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.10
govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.8 govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.11
govoplan-campaign @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-campaign.git@v0.1.8 govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.8
govoplan-calendar @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-calendar.git@v0.1.8 govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.8
govoplan-poll @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-poll.git@v0.1.8 govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.8
govoplan-scheduling @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-scheduling.git@v0.1.8
govoplan-notifications @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-notifications.git@v0.1.8
govoplan-evaluation @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-evaluation.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-ops.git@v0.1.8
+190
View File
@@ -0,0 +1,190 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import stat
import sys
import tempfile
import unittest
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
for tools_root in (ASSESSMENT_TOOLS_ROOT, RELEASE_TOOLS_ROOT):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
class CapabilityFitAtomicIOTests(unittest.TestCase):
def test_writes_private_json_with_expected_content(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
target = Path(temporary_directory) / "evidence.json"
payload = {"z": [1, 2], "message": "Grüße"}
created_in: list[Path] = []
original_mkstemp = tempfile.mkstemp
def observed_mkstemp(*args, **kwargs):
created_in.append(Path(kwargs["dir"]))
return original_mkstemp(*args, **kwargs)
with (
mock.patch(
"govoplan_assessment.atomic_io.tempfile.mkstemp",
side_effect=observed_mkstemp,
),
mock.patch(
"govoplan_assessment.atomic_io.os.fsync",
wraps=os.fsync,
) as fsync,
):
atomic_write_json(target, payload, max_bytes=1024)
self.assertEqual(payload, json.loads(target.read_text(encoding="utf-8")))
self.assertTrue(target.read_bytes().endswith(b"\n"))
self.assertEqual(0o600, stat.S_IMODE(target.stat().st_mode))
self.assertEqual([target.parent], created_in)
self.assertEqual(2, fsync.call_count)
def test_replaces_existing_regular_file_and_secures_mode(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
target = Path(temporary_directory) / "report.json"
target.write_text('{"old": true}\n', encoding="utf-8")
target.chmod(0o644)
old_handle = target.open("rb")
self.addCleanup(old_handle.close)
atomic_write_json(target, {"new": True}, max_bytes=1024)
self.assertEqual(b'{"old": true}\n', old_handle.read())
self.assertEqual({"new": True}, json.loads(target.read_text("utf-8")))
self.assertEqual(0o600, stat.S_IMODE(target.stat().st_mode))
def test_size_bound_leaves_existing_target_unchanged(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
original = b'{"original": true}\n'
target.write_bytes(original)
with self.assertRaisesRegex(AtomicJsonWriteError, "size limit"):
atomic_write_json(target, {"large": "x" * 100}, max_bytes=32)
self.assertEqual(original, target.read_bytes())
self.assertEqual([], list(directory.glob(".govoplan-json-*.tmp")))
@unittest.skipUnless(hasattr(os, "symlink"), "symbolic links are unavailable")
def test_rejects_symlink_without_modifying_link_or_referent(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
referent = directory / "outside.json"
referent.write_bytes(b'{"keep": true}\n')
target = directory / "evidence.json"
target.symlink_to(referent.name)
with self.assertRaisesRegex(AtomicJsonWriteError, "symbolic link"):
atomic_write_json(target, {"replace": True}, max_bytes=1024)
self.assertTrue(target.is_symlink())
self.assertEqual(b'{"keep": true}\n', referent.read_bytes())
self.assertEqual([], list(directory.glob(".govoplan-json-*.tmp")))
@unittest.skipUnless(hasattr(os, "symlink"), "symbolic links are unavailable")
def test_rejects_symlinked_parent_component(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
real_parent = directory / "real-parent"
real_parent.mkdir()
linked_parent = directory / "linked-parent"
linked_parent.symlink_to(real_parent, target_is_directory=True)
target = linked_parent / "new" / "evidence.json"
with self.assertRaisesRegex(AtomicJsonWriteError, "parent path"):
atomic_write_json(target, {"unsafe": True}, max_bytes=1024)
self.assertFalse((real_parent / "new").exists())
def test_requires_existing_parent_without_creating_directories(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
missing_parent = directory / "missing" / "nested"
target = missing_parent / "evidence.json"
with self.assertRaisesRegex(AtomicJsonWriteError, "already exist"):
atomic_write_json(target, {"safe": True}, max_bytes=1024)
self.assertFalse(missing_parent.exists())
def test_rejects_post_replace_mode_or_identity_change(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
original_replace = os.replace
def insecure_replace(source, destination):
original_replace(source, destination)
Path(destination).chmod(0o644)
with mock.patch(
"govoplan_assessment.atomic_io.os.replace",
side_effect=insecure_replace,
):
with self.assertRaisesRegex(
AtomicJsonWriteError,
"permissions changed and were restored",
):
atomic_write_json(target, {"safe": True}, max_bytes=1024)
self.assertEqual(0o600, stat.S_IMODE(target.stat().st_mode))
self.assertEqual({"safe": True}, json.loads(target.read_text("utf-8")))
def test_does_not_unlink_unknown_post_replace_inode(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
original_replace = os.replace
def replaced_again(source, destination):
original_replace(source, destination)
Path(destination).unlink()
Path(destination).write_bytes(b"unknown replacement\n")
with mock.patch(
"govoplan_assessment.atomic_io.os.replace",
side_effect=replaced_again,
):
with self.assertRaisesRegex(
AtomicJsonWriteError,
"did not preserve",
):
atomic_write_json(target, {"secret": True}, max_bytes=1024)
self.assertEqual(b"unknown replacement\n", target.read_bytes())
def test_replace_failure_removes_private_temporary_file(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
with mock.patch(
"govoplan_assessment.atomic_io.os.replace",
side_effect=OSError("simulated replacement failure"),
):
with self.assertRaisesRegex(
AtomicJsonWriteError, "could not be written atomically"
):
atomic_write_json(target, {"safe": True}, max_bytes=1024)
self.assertFalse(target.exists())
self.assertEqual([], list(directory.glob(".govoplan-json-*.tmp")))
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large Load Diff
+700
View File
@@ -0,0 +1,700 @@
from __future__ import annotations
import base64
from copy import deepcopy
from datetime import UTC, datetime, timedelta
import hashlib
import importlib.util
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
from unittest import mock
import unittest
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
for tools_root in (ASSESSMENT_TOOLS_ROOT, RELEASE_TOOLS_ROOT):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from govoplan_assessment.capability_fit import ( # noqa: E402
local_tag_provenance,
render_review,
review_capability_fit,
trusted_keys_from_keyring,
)
class CapabilityFitReviewTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.assessment = json.loads(
(META_ROOT / "docs" / "capability-fit-current.json").read_text(
encoding="utf-8"
)
)
cls.schema = json.loads(
(META_ROOT / "docs" / "capability-fit.schema.json").read_text(
encoding="utf-8"
)
)
def test_matching_signed_catalog_is_current(self) -> None:
catalog, keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("current", report["status"])
self.assertEqual([], report["changes"])
self.assertFalse(report["proof_scope"]["installed_artifacts"]["checked"])
self.assertFalse(report["proof_scope"]["target_environment"]["checked"])
self.assertFalse(report["proof_scope"]["local_tag_provenance"]["checked"])
repeated = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual(
json.dumps(report, sort_keys=True), json.dumps(repeated, sort_keys=True)
)
def test_release_drift_identifies_affected_conclusions(self) -> None:
catalog, keyring = signed_catalog(
self.assessment, versions={"campaigns": "0.1.11"}, sequence=202607230001
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("review_required", report["status"])
target_ids = {item["id"] for item in report["review_targets"]}
self.assertIn("campaign.journey", target_ids)
self.assertIn("composition.campaigns", target_ids)
self.assertIn("assessment.release", target_ids)
self.assertIn(
"composition_version_changed", {item["code"] for item in report["findings"]}
)
def test_contradictory_signed_selected_unit_metadata_blocks_rerun(self) -> None:
core = next(
item
for item in self.assessment["composition"]
if item["module_id"] == "core"
)
catalog, keyring = signed_catalog(
self.assessment,
selected_units=[
{
"repo": core["repository"],
"version": core["manifest_version"],
"tag": "v999.0.0",
"commit_sha": "0" * 40,
"tag_object_sha": "1" * 40,
}
],
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("blocked", report["status"])
codes = {item["code"] for item in report["findings"]}
self.assertIn("catalog_release_metadata", codes)
self.assertIn("composition_commit_changed", codes)
self.assertTrue(report["proof_scope"]["catalog_signature_and_keyring"]["valid"])
self.assertFalse(report["proof_scope"]["release_metadata"]["valid"])
def test_signed_selected_unit_commit_drift_requires_review(self) -> None:
core = next(
item
for item in self.assessment["composition"]
if item["module_id"] == "core"
)
catalog, keyring = signed_catalog(
self.assessment,
selected_units=[
{
"repo": core["repository"],
"version": core["manifest_version"],
"tag": f"v{core['manifest_version']}",
"commit_sha": "0" * 40,
"tag_object_sha": "1" * 40,
}
],
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("review_required", report["status"])
self.assertIn(
"composition_commit_changed", {item["code"] for item in report["findings"]}
)
self.assertIn(
"composition.core", {item["id"] for item in report["review_targets"]}
)
self.assertFalse(report["proof_scope"]["release_metadata"]["valid"])
def test_missing_duplicate_and_malformed_selected_unit_provenance_blocks(
self,
) -> None:
core = next(
item
for item in self.assessment["composition"]
if item["module_id"] == "core"
)
valid_unit = {
"repo": core["repository"],
"version": core["manifest_version"],
"tag": f"v{core['manifest_version']}",
"commit_sha": (str(core["commit"]) + "0" * 40)[:40],
"tag_object_sha": "1" * 40,
}
cases = (
("missing", None, False, "catalog_source_provenance"),
(
"duplicate",
[valid_unit, deepcopy(valid_unit)],
True,
"catalog_release_metadata",
),
(
"malformed",
[{**valid_unit, "commit_sha": "not-a-git-object"}],
True,
"catalog_source_provenance",
),
)
for label, selected_units, include_selected_units, expected_code in cases:
with self.subTest(label=label):
catalog, keyring = signed_catalog(
self.assessment,
selected_units=selected_units,
include_selected_units=include_selected_units,
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("blocked", report["status"])
self.assertIn(
expected_code, {item["code"] for item in report["findings"]}
)
self.assertFalse(report["proof_scope"]["release_metadata"]["valid"])
def test_catalog_validation_ignores_and_preserves_installer_replay_state(
self,
) -> None:
catalog, keyring = signed_catalog(self.assessment)
with tempfile.TemporaryDirectory() as temp_dir:
state_path = Path(temp_dir) / "catalog-sequence.json"
state_path.write_text(
json.dumps(
{"channels": {"stable": {"last_sequence": catalog["sequence"]}}}
),
encoding="utf-8",
)
replay_environment = {
"GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE": str(state_path),
"GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE": "true",
}
with mock.patch.dict(os.environ, replay_environment, clear=False):
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual(
replay_environment,
{key: os.environ[key] for key in replay_environment},
)
self.assertEqual("current", report["status"])
self.assertTrue(report["proof_scope"]["catalog_signature_and_keyring"]["valid"])
def test_self_consistent_substitution_fails_independent_trust_root(self) -> None:
_, trusted_keyring = signed_catalog(self.assessment)
catalog, published_keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=published_keyring,
trusted_keyring=trusted_keyring,
)
self.assertEqual("blocked", report["status"])
codes = {item["code"] for item in report["findings"]}
self.assertIn("catalog_trust", codes)
self.assertNotIn("catalog_keyring_hash_mismatch", codes)
self.assertTrue(report["proof_scope"]["assessment_schema"]["valid"])
self.assertFalse(
report["proof_scope"]["catalog_signature_and_trusted_keyring"]["valid"]
)
self.assertTrue(report["proof_scope"]["published_keyring_hash"]["valid"])
def test_locally_pinned_rotation_keys_fail_closed(self) -> None:
keys = trusted_keys_from_keyring(
{
"keys": [
{"key_id": "active", "status": "active", "public_key": "a"},
{"key_id": "next", "status": "next", "public_key": "b"},
{"key_id": "retired", "status": "retired", "public_key": "c"},
]
}
)
self.assertEqual({"active": "a", "next": "b"}, keys)
self.assertEqual(
{"legacy": "base64-key"},
trusted_keys_from_keyring({"legacy": "base64-key"}),
)
self.assertEqual(
{},
trusted_keys_from_keyring(
{
"keys": [
{
"key_id": "typo",
"status": "retierd",
"public_key": "unsafe",
}
]
}
),
)
self.assertEqual(
{},
trusted_keys_from_keyring(
{
"keys": [
{"key_id": "duplicate", "public_key": "a"},
{"key_id": "duplicate", "public_key": "b"},
]
}
),
)
self.assertEqual({}, trusted_keys_from_keyring({"keys": "invalid"}))
self.assertEqual(
{},
trusted_keys_from_keyring(
{"keys": [{"key_id": " ", "public_key": "unsafe"}]}
),
)
def test_schema_error_blocks_before_comparison(self) -> None:
assessment = deepcopy(self.assessment)
assessment.pop("scope")
catalog, keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=assessment,
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("blocked", report["status"])
self.assertEqual(
{"assessment_schema"}, {item["code"] for item in report["findings"]}
)
self.assertFalse(report["proof_scope"]["assessment_schema"]["valid"])
self.assertFalse(
report["proof_scope"]["catalog_signature_and_keyring"]["checked"]
)
self.assertIsNone(
report["proof_scope"]["catalog_signature_and_keyring"]["valid"]
)
def test_human_report_states_proof_limit(self) -> None:
catalog, keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
rendered = render_review(report)
self.assertIn("Capability fit rerun: current", rendered)
self.assertIn(
"No installed-composition, installed-release-origin, target-environment, external-provider, production-approval proof",
rendered,
)
def test_public_fetch_failure_is_generic_and_blocking(self) -> None:
script = META_ROOT / "tools" / "assessments" / "capability-fit.py"
spec = importlib.util.spec_from_file_location("capability_fit_cli", script)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
with tempfile.TemporaryDirectory() as temp_dir:
trusted_path = Path(temp_dir) / "trusted-keyring.json"
trusted_path.write_text('{"keys": []}\n', encoding="utf-8")
with mock.patch.object(
module,
"fetch_json",
return_value={
"ok": False,
"error": "Authorization: secret at https://internal.invalid",
},
):
with self.assertRaisesRegex(
SystemExit, r"^Could not fetch fixed public catalog endpoint[.]$"
):
module.main(
[
"--public",
"--trusted-keyring",
str(trusted_path),
"--skip-tag-provenance",
]
)
def test_cli_requires_an_independent_local_trust_root(self) -> None:
script = META_ROOT / "tools" / "assessments" / "capability-fit.py"
spec = importlib.util.spec_from_file_location(
"capability_fit_cli_trust", script
)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
with mock.patch.dict(
os.environ,
{module.TRUSTED_KEYRING_FILE_ENV: ""},
clear=False,
):
with self.assertRaisesRegex(SystemExit, r"--trusted-keyring .* required"):
module.main(["--public", "--skip-tag-provenance"])
def test_local_selected_tag_commit_and_object_must_match_exactly(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
commit_sha, tag_object_sha = create_tagged_repository(
workspace=workspace,
repository="govoplan-core",
version="0.1.0",
)
assessment = deepcopy(self.assessment)
core = next(
deepcopy(item)
for item in assessment["composition"]
if item["module_id"] == "core"
)
core["manifest_version"] = "0.1.0"
core["commit"] = commit_sha[:12]
assessment["composition"] = [core]
selected_unit = {
"repo": "govoplan-core",
"version": "0.1.0",
"tag": "v0.1.0",
"commit_sha": commit_sha,
"tag_object_sha": tag_object_sha,
}
catalog, keyring = signed_catalog(
assessment,
selected_units=[selected_unit],
)
baseline = review_capability_fit(
assessment=deepcopy(assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
workspace_root=workspace,
)
self.assertEqual("current", baseline["status"])
self.assertTrue(baseline["proof_scope"]["local_tag_provenance"]["valid"])
forged_commit = commit_sha[:12] + ("0" * 28)
if forged_commit == commit_sha:
forged_commit = commit_sha[:12] + ("f" * 28)
forged_catalog, forged_keyring = signed_catalog(
assessment,
selected_units=[{**selected_unit, "commit_sha": forged_commit}],
)
commit_mismatch = review_capability_fit(
assessment=deepcopy(assessment),
schema=self.schema,
catalog=forged_catalog,
published_keyring=forged_keyring,
trusted_keyring=forged_keyring,
workspace_root=workspace,
)
self.assertEqual("review_required", commit_mismatch["status"])
self.assertIn(
"tag_signed_commit_changed",
{item["kind"] for item in commit_mismatch["changes"]},
)
repo = workspace / "govoplan-core"
git_text(repo, "tag", "-d", "v0.1.0")
git_text(repo, "tag", "-a", "v0.1.0", "-m", "Re-annotated release")
reannotated = review_capability_fit(
assessment=deepcopy(assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
workspace_root=workspace,
)
self.assertEqual("review_required", reannotated["status"])
self.assertIn(
"tag_object_changed",
{item["kind"] for item in reannotated["changes"]},
)
self.assertTrue(reannotated["proof_scope"]["local_tag_provenance"]["checked"])
self.assertFalse(reannotated["proof_scope"]["local_tag_provenance"]["valid"])
def test_missing_composition_entry_does_not_claim_complete_tag_proof(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
core_commit, core_tag_object = create_tagged_repository(
workspace=workspace,
repository="govoplan-core",
version="0.1.0",
)
tenancy_commit, _ = create_tagged_repository(
workspace=workspace,
repository="govoplan-tenancy",
version="0.1.0",
)
reviewed_assessment = deepcopy(self.assessment)
core = next(
deepcopy(item)
for item in reviewed_assessment["composition"]
if item["module_id"] == "core"
)
tenancy = next(
deepcopy(item)
for item in reviewed_assessment["composition"]
if item["module_id"] == "tenancy"
)
core.update(manifest_version="0.1.0", commit=core_commit[:12])
tenancy.update(manifest_version="0.1.0", commit=tenancy_commit[:12])
catalog_assessment = deepcopy(reviewed_assessment)
catalog_assessment["composition"] = [core]
reviewed_assessment["composition"] = [core, tenancy]
catalog, keyring = signed_catalog(
catalog_assessment,
selected_units=[
{
"repo": "govoplan-core",
"version": "0.1.0",
"tag": "v0.1.0",
"commit_sha": core_commit,
"tag_object_sha": core_tag_object,
}
],
)
report = review_capability_fit(
assessment=reviewed_assessment,
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
workspace_root=workspace,
)
proof = report["proof_scope"]["local_tag_provenance"]
self.assertEqual("review_required", report["status"])
self.assertFalse(proof["checked"])
self.assertIsNone(proof["valid"])
self.assertEqual(2, proof["attempted_count"])
self.assertEqual(2, proof["expected_count"])
self.assertNotIn(
"tag_provenance_changed", {item["code"] for item in report["findings"]}
)
def test_repository_path_traversal_is_not_resolved(self) -> None:
with mock.patch("govoplan_assessment.capability_fit.subprocess.run") as run:
result = local_tag_provenance(
workspace_root=META_ROOT.parent,
repository="../outside",
version="0.1.0",
assessed_commit="0123456",
)
self.assertEqual("tag_provenance_unavailable", result["kind"])
self.assertIn("invalid repository identifier", result["message"])
run.assert_not_called()
def create_tagged_repository(
*, workspace: Path, repository: str, version: str
) -> tuple[str, str]:
repo = workspace / repository
git_text(workspace, "init", "-b", "main", str(repo))
git_text(repo, "config", "user.name", "Capability Fit Test")
git_text(repo, "config", "user.email", "capability-fit@example.invalid")
(repo / "README.md").write_text(f"{repository}\n", encoding="utf-8")
git_text(repo, "add", "README.md")
git_text(repo, "commit", "-m", "Initial release fixture")
git_text(repo, "tag", "-a", f"v{version}", "-m", f"Release v{version}")
return (
git_text(repo, "rev-parse", "HEAD"),
git_text(repo, "rev-parse", f"refs/tags/v{version}"),
)
def git_text(cwd: Path, *args: str) -> str:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
capture_output=True,
text=True,
)
if result.returncode != 0:
raise AssertionError(result.stderr or result.stdout)
return result.stdout.strip()
def signed_catalog(
assessment: dict[str, object],
*,
versions: dict[str, str] | None = None,
sequence: int = 202607220843,
selected_units: list[dict[str, object]] | None = None,
include_selected_units: bool = True,
release_artifacts: list[dict[str, object]] | None = None,
) -> tuple[dict[str, object], dict[str, object]]:
versions = versions or {}
private_key = Ed25519PrivateKey.generate()
public_key = base64.b64encode(
private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
).decode("ascii")
keyring: dict[str, object] = {
"keyring_version": "1",
"purpose": "test",
"generated_at": "2026-01-01T00:00:00Z",
"keys": [{"key_id": "test-key", "status": "active", "public_key": public_key}],
}
now = datetime.now(tz=UTC)
modules: list[dict[str, object]] = []
core_release: dict[str, object] | None = None
for component in assessment["composition"]:
module_id = str(component["module_id"])
repository = str(component["repository"])
version = versions.get(module_id, str(component["manifest_version"]))
entry: dict[str, object] = {
"module_id": module_id,
"name": module_id,
"version": version,
"python_package": repository,
"python_ref": f"{repository} @ git+ssh://git@example.invalid/example/{repository}.git@v{version}",
}
if module_id == "core":
core_release = entry
else:
modules.append(entry)
assert core_release is not None
if selected_units is None:
selected_units = [
{
"repo": component["repository"],
"version": versions.get(
str(component["module_id"]), str(component["manifest_version"])
),
"tag": "v"
+ versions.get(
str(component["module_id"]), str(component["manifest_version"])
),
"commit_sha": (str(component["commit"]) + "0" * 40)[:40],
"tag_object_sha": hashlib.sha256(
f"{component['repository']}:{component['manifest_version']}:tag".encode()
).hexdigest()[:40],
}
for component in assessment["composition"]
]
release: dict[str, object] = {"keyring_sha256": canonical_hash(keyring)}
if include_selected_units:
release["selected_units"] = selected_units
if release_artifacts is not None:
release["artifacts"] = release_artifacts
catalog: dict[str, object] = {
"catalog_version": "1",
"channel": "stable",
"sequence": sequence,
"generated_at": now.isoformat().replace("+00:00", "Z"),
"expires_at": (now + timedelta(days=30)).isoformat().replace("+00:00", "Z"),
"core_release": core_release,
"modules": modules,
"release": release,
}
signature_payload = json.dumps(
catalog, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
catalog["signatures"] = [
{
"algorithm": "ed25519",
"key_id": "test-key",
"value": base64.b64encode(private_key.sign(signature_payload)).decode(
"ascii"
),
}
]
return catalog, keyring
def canonical_hash(payload: object) -> str:
encoded = json.dumps(
payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,79 @@
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import unittest
from govoplan_core.core.configuration_packages import (
ConfigurationPackageManifest,
ConfigurationPreflightContext,
configuration_package_claim_issues,
dry_run_configuration_package,
)
ROOT = Path(__file__).resolve().parents[1]
PACKAGE_ROOT = ROOT / "packages"
class ConfigurationPackageArtifactTests(unittest.TestCase):
def test_product_package_manifests_are_portable_and_evidence_backed(self) -> None:
paths = tuple(sorted(PACKAGE_ROOT.glob("*/*/package.json")))
self.assertGreaterEqual(len(paths), 2)
package_ids: set[str] = set()
for path in paths:
manifest = ConfigurationPackageManifest.from_mapping(
json.loads(path.read_text(encoding="utf-8"))
)
self.assertNotIn(manifest.package_id, package_ids)
package_ids.add(manifest.package_id)
self.assertEqual((), configuration_package_claim_issues(manifest))
for evidence in manifest.evidence:
evidence_path = ROOT / evidence.reference
self.assertTrue(
evidence_path.is_file(),
f"Missing evidence {evidence.reference} for {manifest.package_id}",
)
if evidence.checksum is not None:
self.assertEqual(
evidence.checksum,
"sha256:" + hashlib.sha256(evidence_path.read_bytes()).hexdigest(),
f"Stale evidence checksum for {manifest.package_id}: {evidence.reference}",
)
for requirement in (
*manifest.required_modules,
*manifest.optional_modules,
):
repository = ROOT.parent / (
"govoplan-" + requirement.module_id.replace("_", "-")
)
self.assertTrue(
repository.is_dir(),
f"Missing repository for {requirement.module_id}",
)
result = dry_run_configuration_package(
manifest,
(),
ConfigurationPreflightContext(
installed_modules={
item.module_id: item.version or "workspace"
for item in manifest.required_modules
},
capabilities=frozenset(manifest.required_capabilities),
),
)
self.assertFalse(
any(item.severity == "blocker" for item in result.diagnostics),
tuple(item.to_dict() for item in result.diagnostics),
)
self.assertIn("product.governed-communication", package_ids)
self.assertIn("product.governed-data-assurance", package_ids)
self.assertIn("product.service-to-decision", package_ids)
if __name__ == "__main__":
unittest.main()
+8
View File
@@ -88,9 +88,17 @@ class DependencyBoundaryDiscoveryTests(unittest.TestCase):
"govoplan-calendar", "govoplan-calendar",
"govoplan-poll", "govoplan-poll",
"govoplan-scheduling", "govoplan-scheduling",
"govoplan-workflow-engine",
): ):
self.assertIn(owner, boundary.REPOS) self.assertIn(owner, boundary.REPOS)
def test_transitional_allowlist_entries_have_a_removal_target(self) -> None:
boundary = load_boundary_module()
for item in boundary.ALLOWLIST:
self.assertIn("remove", item.reason.casefold())
self.assertRegex(item.reason, r"#[0-9]+")
def test_webui_discovery_is_fail_closed(self) -> None: def test_webui_discovery_is_fail_closed(self) -> None:
boundary = load_boundary_module() boundary = load_boundary_module()
with tempfile.TemporaryDirectory(prefix="govoplan-boundary-webui-") as directory: with tempfile.TemporaryDirectory(prefix="govoplan-boundary-webui-") as directory:
File diff suppressed because it is too large Load Diff
+49
View File
@@ -0,0 +1,49 @@
from __future__ import annotations
import importlib.util
import pathlib
import sys
import tempfile
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
SCRIPT = ROOT / "tools" / "gitea" / "gitea-sync-wiki.py"
TOOLS = SCRIPT.parent
if str(TOOLS) not in sys.path:
sys.path.insert(0, str(TOOLS))
SPEC = importlib.util.spec_from_file_location("gitea_sync_wiki", SCRIPT)
assert SPEC and SPEC.loader
wiki_sync = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = wiki_sync
SPEC.loader.exec_module(wiki_sync)
class WikiSourceDiscoveryTests(unittest.TestCase):
def test_generated_and_incidental_govoplan_files_are_not_docs(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary)
paths = {
"readme": root / "README.md",
"architecture": root / "docs" / "DATASOURCE_ARCHITECTURE.md",
"plan": root / "workflow-plan.md",
"audit": root / "audit-reports" / "full" / "manifest.json",
"runtime": root / "runtime" / "release" / "manifest.json",
"incidental": root / "tools" / "semgrep" / "govoplan.yml",
"manifest": root / "manifest.json",
}
for path in paths.values():
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("content\n", encoding="utf-8")
self.assertTrue(wiki_sync.is_repo_doc(root, paths["readme"]))
self.assertTrue(wiki_sync.is_repo_doc(root, paths["architecture"]))
self.assertTrue(wiki_sync.is_repo_doc(root, paths["plan"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["audit"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["runtime"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["incidental"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["manifest"]))
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,591 @@
from __future__ import annotations
from dataclasses import dataclass, replace
from datetime import UTC, datetime, timedelta
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_core.core.institutional import (
ActorRepresentationReference,
DecisionEffectReference,
EvidenceReference,
InformationGovernanceReference,
InstitutionalReference,
LegalBasisReference,
MandateDefinition,
PartyRepresentation,
PartySubjectReference,
ProcedureParty,
ServiceBinding,
ServiceDefinition,
TemporalRevision,
service_launch_capability,
)
from govoplan_cases.backend.party_context import CasePartyContext
from govoplan_cases.backend.db.models import (
CaseAccessGrant,
CaseIdentity,
CaseRecordRevision,
CaseStatusDefinition,
CaseTimelineEntry,
CaseTypeDefinition,
)
from govoplan_cases.backend.service import (
create_case_from_intake,
get_case,
upsert_case_status,
upsert_case_type,
)
from govoplan_cases.backend.service_intake import CaseServiceIntake
from govoplan_committee.backend.db.models import (
CommitteeDecisionProjection,
CommitteeWorkspaceEvent,
CommitteeWorkspaceRevision,
)
from govoplan_committee.backend.decision_path import (
CommitteeDecisionPath,
CommitteeDecisionProposal,
)
from govoplan_committee.backend.workspace import (
CommitteeWorkspaceRecord,
SqlCommitteeWorkspace,
get_workspace_object,
record_workspace_object,
)
from govoplan_portal.backend.service_directory import PortalServiceDirectory
from govoplan_decisions.backend.db.models import FormalDecisionRevision
from govoplan_decisions.backend.service import SqlDecisionRegistry
from govoplan_mandates.backend.db.models import MandateRevision
from govoplan_mandates.backend.service import SqlMandateResolver, record_mandate
from govoplan_parties.backend.db.models import ProcedurePartyRevision
from govoplan_parties.backend.service import SqlPartyResolver, record_procedure_party
from govoplan_services.backend.db.models import ServiceDefinitionRevision
from govoplan_services.backend.service import SqlServiceDefinitionProvider, record_service_definition
NOW = datetime(2026, 8, 1, 10, 0, tzinfo=UTC)
def _reference(
kind: str,
object_id: str,
*,
owner: str,
version: str = "1",
) -> InstitutionalReference:
return InstitutionalReference(
kind=kind, # type: ignore[arg-type]
owner_module=owner,
object_id=object_id,
tenant_id="tenant-1",
version=version,
valid_at=NOW,
)
def _evidence(kind: str, evidence_id: str, owner: str) -> EvidenceReference:
return EvidenceReference(
kind=kind, # type: ignore[arg-type]
owner_module=owner,
evidence_id=evidence_id,
tenant_id="tenant-1",
version="1",
checksum=f"sha256:{evidence_id}",
captured_at=NOW,
)
@dataclass
class _Principal:
tenant_id: str = "tenant-1"
account_id: str = "account-1"
class _Registry:
def __init__(self, capabilities: dict[str, object]) -> None:
self.capabilities = capabilities
def has(self, module_id: str) -> bool:
return module_id in {
"portal",
"cases",
"committee",
"postbox",
"services",
"parties",
"mandates",
"decisions",
}
def has_capability(self, name: str) -> bool:
return name in self.capabilities
def capability(self, name: str) -> object:
return self.capabilities[name]
class InstitutionalGovernanceJourneyTests(unittest.TestCase):
def test_service_to_formal_outcome_retains_governed_context(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
for table in (
ServiceDefinitionRevision.__table__,
CaseStatusDefinition.__table__,
CaseTypeDefinition.__table__,
CaseIdentity.__table__,
CaseRecordRevision.__table__,
CaseAccessGrant.__table__,
CaseTimelineEntry.__table__,
ProcedurePartyRevision.__table__,
MandateRevision.__table__,
FormalDecisionRevision.__table__,
CommitteeWorkspaceRevision.__table__,
CommitteeWorkspaceEvent.__table__,
CommitteeDecisionProjection.__table__,
):
table.create(engine)
session = Session(engine)
principal = _Principal()
self.addCleanup(engine.dispose)
self.addCleanup(session.close)
organization = _reference(
"organization_unit",
"permit-office",
owner="organizations",
)
function = _reference("function", "permit-officer", owner="organizations")
jurisdiction = _reference(
"jurisdiction",
"city-1",
owner="organizations",
)
mandate_ref = _reference(
"mandate",
"permit-mandate",
owner="mandates",
version="4",
)
legal_basis = LegalBasisReference(
kind="law",
authority="Example legislature",
reference="permit-law:3",
version="2026-01",
effective_from=NOW - timedelta(days=100),
)
service = ServiceDefinition(
reference=_reference(
"service",
"permit-service",
owner="services",
version="5",
),
key="permit.apply",
temporal=TemporalRevision(
revision="5",
valid_from=NOW - timedelta(days=1),
valid_to=NOW + timedelta(days=30),
recorded_at=NOW - timedelta(days=2),
change_reason="Service published.",
),
title="Apply for a permit",
audience=("resident",),
legal_bases=(legal_basis,),
required_evidence_types=("application", "identity"),
channels=("portal", "postbox"),
responsible_organization_ref=organization,
responsible_function_ref=function,
mandate_ref=mandate_ref,
jurisdiction_refs=(jurisdiction,),
bindings=(
ServiceBinding("case", "permit-application"),
ServiceBinding("workflow", "workflow:permit-review"),
ServiceBinding("result", "decision:permit"),
),
remedy_refs=("review:administrative-court",),
publication_state="published",
)
record_service_definition(session, principal, definition=service)
service_registry = _Registry(
{
"services.definitions": SqlServiceDefinitionProvider(),
service_launch_capability("case"): object(),
}
)
entry = PortalServiceDirectory(service_registry).list_entries(
session,
principal,
tenant_id="tenant-1",
effective_at=NOW,
audiences=("resident",),
)[0]
self.assertTrue(entry.available)
intake = CaseServiceIntake().plan(
entry.definition,
case_id="case-1",
effective_at=NOW,
)
application_evidence = _evidence("document", "application-1", "files")
upsert_case_status(
session,
principal,
status_key="intake",
label="Intake",
)
upsert_case_status(
session,
principal,
status_key="decided",
label="Decided",
category="decided",
)
upsert_case_type(
session,
principal,
type_key="permit-application",
label="Permit application",
initial_status_key="intake",
allowed_status_keys=("intake", "decided"),
)
case_record = create_case_from_intake(
session,
principal,
plan=intake,
case_number="PERMIT-2026-0001",
title="Permit application",
status_key=None,
opened_at=NOW,
recorded_at=NOW,
change_reason="Portal application received.",
idempotency_key="journey-case-create",
evidence_refs=(application_evidence,),
deadline_at=NOW + timedelta(days=30),
)
applicant = _reference("party", "applicant", owner="parties")
representative = _reference("party", "representative", owner="parties")
address_evidence = _evidence(
"snapshot",
"address-snapshot-1",
"addresses",
)
representative_party = ProcedureParty(
reference=representative,
procedure_ref=case_record.reference,
role="representative",
subject=PartySubjectReference(
kind="identity",
provider="identity",
subject_id="identity-2",
tenant_id="tenant-1",
version="2",
),
temporal=TemporalRevision(
revision="1",
valid_from=NOW - timedelta(days=1),
recorded_at=NOW - timedelta(days=1),
change_reason="Representative added to the procedure.",
),
permitted_channels=("postbox",),
preferred_channels=("postbox",),
delivery_recipient=True,
representations=(
PartyRepresentation(
representative_party_ref=representative,
represented_party_ref=applicant,
power_ref="power-1",
permitted_actions=("submit", "receive"),
temporal=TemporalRevision(
revision="1",
valid_from=NOW - timedelta(days=1),
recorded_at=NOW - timedelta(days=1),
change_reason="Representation power recorded.",
),
evidence=(address_evidence,),
),
),
contact_snapshot_refs=("addresses:snapshot-1",),
evidence=(address_evidence,),
)
record_procedure_party(session, principal, party=representative_party)
party_context = CasePartyContext(
_Registry({"parties.resolver": SqlPartyResolver()})
)
parties = party_context.resolve(
session,
principal,
case_ref=case_record.reference,
effective_at=NOW,
)
delivery_target = party_context.delivery_targets(
parties,
channel="postbox",
)[0]
mandate = MandateDefinition(
reference=mandate_ref,
temporal=TemporalRevision(
revision="4",
valid_from=NOW - timedelta(days=30),
valid_to=NOW + timedelta(days=30),
recorded_at=NOW - timedelta(days=31),
change_reason="Permit authority delegated.",
),
task_types=("committee.formal_decision",),
authority_types=("permit",),
organization_unit_refs=(organization,),
function_refs=(function,),
jurisdiction_refs=(jurisdiction,),
legal_bases=(legal_basis,),
evidence=(_evidence("record", "mandate-record-4", "mandates"),),
authority_ceiling="permit:standard",
)
record_mandate(session, principal, definition=mandate)
workspace = SqlCommitteeWorkspace()
body = CommitteeWorkspaceRecord(
tenant_id="tenant-1",
object_kind="body",
object_id="permit-board",
revision=1,
state="active",
title="Permit board",
recorded_at=NOW,
change_reason="Permit board configured.",
attributes={
"organization_unit_ref": organization.to_dict(),
"function_refs": [function.to_dict()],
"quorum": {"minimum_count": 1},
},
)
meeting = CommitteeWorkspaceRecord(
tenant_id="tenant-1",
object_kind="meeting",
object_id="meeting-1",
revision=1,
state="open",
title="Permit board meeting",
parent_id=body.object_id,
recorded_at=NOW,
change_reason="Meeting opened.",
attributes={
"starts_at": NOW.isoformat(),
"ends_at": (NOW + timedelta(hours=1)).isoformat(),
},
context=case_record.context,
)
agenda = CommitteeWorkspaceRecord(
tenant_id="tenant-1",
object_kind="agenda_item",
object_id="item-1",
revision=1,
state="deliberating",
title="Permit application",
parent_id=meeting.object_id,
recorded_at=NOW,
change_reason="Agenda item entered deliberation.",
attributes={
"position": 1,
"subject_refs": [case_record.reference.to_dict()],
},
context=case_record.context,
evidence=(application_evidence,),
)
approval_ref = _reference("approval", "approval-1", owner="approvals")
vote = CommitteeWorkspaceRecord(
tenant_id="tenant-1",
object_kind="vote",
object_id="vote-1",
revision=1,
state="closed",
title="Vote on permit application",
parent_id=agenda.object_id,
recorded_at=NOW,
change_reason="Vote result accepted.",
attributes={
"method": "recorded",
"choices": ["yes", "no"],
"eligible_count": 3,
"cast_count": 3,
"counts": {"yes": 3, "no": 0},
"quorum_met": True,
"approval_ref": approval_ref.to_dict(),
},
context=case_record.context,
evidence=(_evidence("record", "vote-result-1", "committee"),),
)
for record, key in (
(body, "journey-body"),
(meeting, "journey-meeting"),
(agenda, "journey-agenda"),
(vote, "journey-vote"),
):
record_workspace_object(
session,
principal,
record=record,
idempotency_key=key,
)
decision_registry = SqlDecisionRegistry()
decision_result = CommitteeDecisionPath(
_Registry(
{
"mandates.resolver": SqlMandateResolver(),
"decisions.registry": decision_registry,
"committee.workspace": workspace,
}
)
).decide(
session,
principal,
proposal=CommitteeDecisionProposal(
tenant_id="tenant-1",
decision_id="decision-1",
revision="1",
effective_at=NOW,
meeting_ref="meeting-1",
agenda_item_ref="item-1",
decision_type="permit",
subject_refs=(case_record.reference,),
organization_unit_ref=organization,
function_ref=function,
actor=ActorRepresentationReference(
tenant_id="tenant-1",
account_id="account-1",
identity_id="identity-1",
represented_function_ref=function,
mandate_ref=mandate_ref,
),
approval_refs=(
approval_ref,
),
fact_evidence=(application_evidence, address_evidence),
legal_bases=(legal_basis,),
operative_result="Permit granted.",
reasoning="The application satisfies the effective rule.",
case_ref=case_record.reference,
jurisdiction_refs=(jurisdiction,),
party_refs=(applicant, representative),
record_refs=(
_reference("record", "record-1", owner="audit"),
),
remedy_refs=service.remedy_refs,
review_refs=("review:administrative-court",),
information_governance=InformationGovernanceReference(
classification="restricted",
purposes=("permit-decision", "party-delivery"),
legal_basis_refs=("permit-law:3@2026-01",),
retention_policy_ref="records:permit",
disclosure_state="partly_disclosable",
),
),
observed_effects=(
DecisionEffectReference(
effect_key="postbox.deliver_decision",
state="confirmed",
resource_refs=(
f"postbox:{delivery_target.party_ref.object_id}",
),
audit_event_refs=("audit:delivery-1",),
evidence_refs=(address_evidence.evidence_id,),
),
),
)
decided_agenda = replace(
agenda,
revision=2,
state="decided",
recorded_at=NOW + timedelta(minutes=1),
change_reason="Formal Decision recorded.",
attributes={
**dict(agenda.attributes),
"decision_ref": decision_result.decision.reference.to_dict(),
},
)
record_workspace_object(
session,
principal,
record=decided_agenda,
expected_revision=1,
idempotency_key="journey-agenda-decided",
)
closed_meeting = replace(
meeting,
revision=2,
state="closed",
recorded_at=NOW + timedelta(hours=1),
change_reason="All agenda items completed.",
)
record_workspace_object(
session,
principal,
record=closed_meeting,
expected_revision=1,
idempotency_key="journey-meeting-closed",
)
minute = CommitteeWorkspaceRecord(
tenant_id="tenant-1",
object_kind="minute",
object_id="minute-1",
revision=1,
state="accepted",
title="Accepted permit board minutes",
parent_id=meeting.object_id,
recorded_at=NOW + timedelta(hours=2),
change_reason="Minutes approved.",
attributes={
"content_ref": _reference(
"record",
"meeting-minutes-1",
owner="records",
).to_dict(),
"approval_ref": _reference(
"approval",
"minutes-approval-1",
owner="approvals",
).to_dict(),
},
context=case_record.context,
evidence=(_evidence("record", "minutes-proof-1", "records"),),
)
record_workspace_object(
session,
principal,
record=minute,
idempotency_key="journey-minute",
)
reconstruction = decision_result.reconstruction_payload()["decision"]
persisted = decision_registry.get_decision(
session,
principal,
reference=decision_result.decision.reference,
)
self.assertTrue(decision_result.persisted_by_decision_registry)
self.assertEqual(decision_result.decision, persisted)
self.assertEqual(
case_record,
get_case(session, principal, case_id="case-1"),
)
self.assertEqual(
"decided",
get_workspace_object(
session,
principal,
object_kind="agenda_item",
object_id="item-1",
).state,
)
self.assertEqual(service.reference, intake.context.service_ref)
self.assertEqual("applicant", delivery_target.represented_party_refs[0].object_id)
self.assertEqual("permit-mandate", reconstruction["authority_context"]["mandate_ref"]["object_id"])
self.assertEqual("city-1", reconstruction["authority_context"]["jurisdiction_refs"][0]["object_id"])
self.assertEqual("confirmed", reconstruction["observed_effects"][0]["state"])
self.assertEqual("audit:delivery-1", reconstruction["observed_effects"][0]["audit_event_refs"][0])
self.assertEqual("application-1", reconstruction["fact_evidence"][0]["evidence_id"])
self.assertEqual("The application satisfies the effective rule.", reconstruction["reasoning"])
self.assertEqual("review:administrative-court", reconstruction["review_refs"][0])
if __name__ == "__main__":
unittest.main()
+271
View File
@@ -0,0 +1,271 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_core.core.institutional import (
CAPABILITY_FORM_DEFINITIONS,
CAPABILITY_SERVICE_DEFINITIONS,
FormDefinition,
FormFieldDefinition,
InstitutionalReference,
ServiceBinding,
ServiceDefinition,
TemporalRevision,
service_launch_capability,
)
from govoplan_cases.backend.service_intake import (
CAPABILITY_CASES_SERVICE_INTAKE,
CaseServiceIntake,
)
from govoplan_forms.backend.db.models import FormDefinitionRevision
from govoplan_forms.backend.service import (
SqlFormDefinitionProvider,
record_form_definition,
)
from govoplan_forms_runtime.backend.db.models import (
FormInstanceEvent,
FormInstanceIdentity,
FormInstanceRevision,
)
from govoplan_forms_runtime.backend.service import (
FormRuntimeService,
FormsServiceLauncher,
)
from govoplan_portal.backend.service_directory import PortalServiceDirectory
NOW = datetime(2026, 8, 1, 10, 0, tzinfo=UTC)
def _service() -> ServiceDefinition:
return ServiceDefinition(
reference=InstitutionalReference(
kind="service",
owner_module="portal",
object_id="permit",
tenant_id="tenant-1",
version="5",
),
key="permit.apply",
temporal=TemporalRevision(
revision="5",
valid_from=NOW - timedelta(days=1),
valid_to=NOW + timedelta(days=1),
recorded_at=NOW - timedelta(days=2),
),
title="Apply for a permit",
audience=("resident",),
required_evidence_types=("application",),
bindings=(
ServiceBinding("capability", CAPABILITY_CASES_SERVICE_INTAKE),
ServiceBinding("case", "permit-application"),
ServiceBinding("workflow", "workflow:permit-review"),
),
publication_state="published",
)
class _Provider:
def __init__(self, definition: ServiceDefinition) -> None:
self.definition = definition
def get_service_definition(self, session, principal, *, reference, effective_at=None):
return self.definition
def list_service_definitions(self, session, principal, *, tenant_id, query="", limit=100):
return (self.definition,)
class _Registry:
def __init__(self, definition: ServiceDefinition) -> None:
self.capabilities = {
CAPABILITY_SERVICE_DEFINITIONS: _Provider(definition),
CAPABILITY_CASES_SERVICE_INTAKE: CaseServiceIntake(),
service_launch_capability("case"): object(),
}
def has(self, module_id: str) -> bool:
return module_id in {"portal", "cases"}
def has_capability(self, name: str) -> bool:
return name in self.capabilities
def capability(self, name: str) -> object:
return self.capabilities[name]
def require_capability(self, name: str) -> object:
return self.capabilities[name]
@dataclass
class _Principal:
tenant_id: str = "tenant-1"
account_id: str = "account-1"
class _FormRegistry(_Registry):
def __init__(self, definition: ServiceDefinition) -> None:
super().__init__(definition)
self.capabilities[CAPABILITY_FORM_DEFINITIONS] = (
SqlFormDefinitionProvider()
)
self.capabilities[service_launch_capability("form")] = (
FormsServiceLauncher(self)
)
def has(self, module_id: str) -> bool:
return module_id in {"portal", "forms", "forms_runtime"}
class InstitutionalServiceJourneyTests(unittest.TestCase):
def test_one_service_version_drives_portal_and_case_intake(self) -> None:
definition = _service()
registry = _Registry(definition)
entries = PortalServiceDirectory(registry).list_entries(
None,
None,
tenant_id="tenant-1",
effective_at=NOW,
audiences=("resident",),
)
plan = registry.capability(CAPABILITY_CASES_SERVICE_INTAKE).plan(
entries[0].definition,
case_id="case-1",
effective_at=NOW,
)
self.assertTrue(entries[0].available)
self.assertIs(definition, entries[0].definition)
self.assertEqual(definition.reference, plan.service_ref)
self.assertEqual("5", plan.context.service_ref.version)
self.assertEqual("workflow:permit-review", plan.workflow_refs[0])
def test_portal_launches_exact_form_revision_and_persists_submission(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
for table in (
FormDefinitionRevision.__table__,
FormInstanceIdentity.__table__,
FormInstanceRevision.__table__,
FormInstanceEvent.__table__,
):
table.create(engine)
session = Session(engine)
principal = _Principal()
try:
form = record_form_definition(
session,
principal,
definition=FormDefinition(
reference=InstitutionalReference(
kind="form",
owner_module="forms",
object_id="permit-application",
tenant_id="tenant-1",
version="3",
),
key="permit-application",
temporal=TemporalRevision(
revision="3",
valid_from=NOW - timedelta(days=1),
valid_to=NOW + timedelta(days=1),
recorded_at=NOW - timedelta(days=2),
change_reason="Publish the permit application.",
),
title="Permit application",
fields=(
FormFieldDefinition(
key="applicant_name",
label="Applicant name",
required=True,
constraints={"min_length": 2},
),
),
publication_state="published",
allow_drafts=True,
handoff_kinds=("case",),
),
)
binding = ServiceBinding(
"form",
f"{form.reference.object_id}/{form.reference.version}",
)
service = ServiceDefinition(
reference=InstitutionalReference(
kind="service",
owner_module="services",
object_id="permit",
tenant_id="tenant-1",
version="6",
),
key="permit.apply",
temporal=TemporalRevision(
revision="6",
valid_from=NOW - timedelta(days=1),
valid_to=NOW + timedelta(days=1),
recorded_at=NOW - timedelta(days=2),
),
title="Apply for a permit",
audience=("public",),
bindings=(binding,),
publication_state="published",
)
registry = _FormRegistry(service)
directory = PortalServiceDirectory(registry)
launched = directory.launch_service(
session,
principal,
reference=service.reference,
requested_at=NOW,
idempotency_key="portal-form-launch-1",
parameters={"applicant_name": "Ada Lovelace"},
)
replay = directory.launch_service(
session,
principal,
reference=service.reference,
requested_at=NOW,
idempotency_key="portal-form-launch-1",
parameters={"applicant_name": "Ada Lovelace"},
)
instance = FormRuntimeService(registry).get_instance(
session,
principal,
instance_id=str(launched.metadata["form_instance_id"]),
)
self.assertEqual("form_submission", launched.target_ref.kind)
self.assertEqual(service.reference, launched.service_ref)
self.assertEqual("3", launched.metadata["form_definition_revision"])
self.assertTrue(replay.replayed)
self.assertEqual(launched.target_ref, replay.target_ref)
self.assertEqual(
(
form.reference.owner_module,
form.reference.object_id,
form.reference.tenant_id,
form.reference.version,
),
(
instance.definition_ref.owner_module,
instance.definition_ref.object_id,
instance.definition_ref.tenant_id,
instance.definition_ref.version,
),
)
self.assertEqual(NOW, instance.definition_ref.valid_at)
self.assertEqual(service.reference, instance.service_ref)
self.assertEqual("Ada Lovelace", instance.values["applicant_name"])
finally:
session.close()
engine.dispose()
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,66 @@
from __future__ import annotations
import ast
import importlib.util
from pathlib import Path
import unittest
SCRIPT = (
Path(__file__).resolve().parents[1]
/ "tools"
/ "inventory"
/ "platform-interface-inventory.py"
)
SPEC = importlib.util.spec_from_file_location("platform_interface_inventory", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
inventory = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(inventory)
class PlatformInterfaceInventoryTests(unittest.TestCase):
def test_canonical_api_path_normalizes_versions_and_parameters(self) -> None:
self.assertEqual(
inventory.canonical_api_path(
"http://localhost/api/v1/campaigns/${campaignId}?limit=10"
),
"/campaigns/{}",
)
self.assertEqual(
inventory.canonical_api_path("/api/v2/campaigns/{campaign_id}"),
"/campaigns/{}",
)
def test_fastapi_route_scanner_includes_router_prefix(self) -> None:
tree = ast.parse(
"""
from fastapi import APIRouter
router = APIRouter(prefix="/api/v1/items")
@router.get("/{item_id}")
def read_item(item_id: str):
return item_id
"""
)
prefixes = inventory._router_prefixes(tree)
function = next(
node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)
)
route = inventory._endpoint_from_decorator(
function.decorator_list[0],
prefixes=prefixes,
)
self.assertEqual(
route,
{
"method": "GET",
"path": "/api/v1/items/{item_id}",
"router": "router",
},
)
if __name__ == "__main__":
unittest.main()
+75
View File
@@ -0,0 +1,75 @@
from __future__ import annotations
import importlib.util
import os
from pathlib import Path
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "checks" / "postgres-integration-check.py"
def _load_script():
spec = importlib.util.spec_from_file_location(
"postgres_integration_check",
SCRIPT,
)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class PostgresIntegrationCheckTests(unittest.TestCase):
def test_staging_check_has_explicit_runtime_safety_settings(self) -> None:
module = _load_script()
with patch.dict(os.environ, {}, clear=True):
env = module._check_env(
database_url="postgresql+psycopg://user:password@postgres/db",
modules="tenancy,access,search",
app_env="staging",
)
self.assertEqual(
env["CORS_ORIGINS"],
"http://127.0.0.1:5173,http://localhost:5173",
)
self.assertEqual(
env["GOVOPLAN_TRUSTED_HOSTS"],
"127.0.0.1,localhost,testserver",
)
self.assertEqual(
env["GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS"],
"false",
)
self.assertEqual(
env["GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE"],
"true",
)
def test_existing_runtime_safety_settings_are_preserved(self) -> None:
module = _load_script()
configured = {
"CORS_ORIGINS": "https://govoplan.example.test",
"GOVOPLAN_TRUSTED_HOSTS": "govoplan.example.test",
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "true",
"GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE": "false",
}
with patch.dict(os.environ, configured, clear=True):
env = module._check_env(
database_url="postgresql+psycopg://user:password@postgres/db",
modules="tenancy,access,search",
app_env="staging",
)
for key, value in configured.items():
self.assertEqual(env[key], value)
if __name__ == "__main__":
unittest.main()
+152
View File
@@ -0,0 +1,152 @@
from __future__ import annotations
import hashlib
import importlib.util
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "repo" / "sync-python-environment.py"
def load_sync_module():
spec = importlib.util.spec_from_file_location("sync_python_environment", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class PythonEnvironmentSyncTests(unittest.TestCase):
def test_stale_local_projects_share_one_resolver_transaction(self) -> None:
sync = load_sync_module()
with tempfile.TemporaryDirectory(prefix="govoplan-python-sync-") as directory:
root = Path(directory)
requirements = root / "requirements-dev.txt"
requirements.write_text("-e ./govoplan-core\n-e ./govoplan-access\n", encoding="utf-8")
for project in ("govoplan-core", "govoplan-access"):
project_root = root / project
project_root.mkdir()
(project_root / "pyproject.toml").write_text(
f'[project]\nname = "{project}"\nversion = "0.1.10"\n',
encoding="utf-8",
)
local_requirements = sync.local_requirement_entries(requirements)
fingerprint = sync.build_fingerprint(
requirements=requirements,
python="/test/venv/bin/python",
local_requirements=local_requirements,
)
requirements_digest = hashlib.sha256(requirements.read_bytes()).hexdigest()
previous = {
"version": sync.STAMP_VERSION,
"python": "/test/venv/bin/python",
"inputs": [
{"path": str(requirements), "sha256": requirements_digest},
*(
{"path": entry.pyproject, "sha256": "stale"}
for entry in local_requirements
),
],
"requirements_entries": [
entry.as_dict() for entry in sync.parse_requirement_entries(requirements)
],
}
plan = sync.build_install_plan(
previous=previous,
fingerprint=fingerprint,
requirements=requirements,
python="/test/venv/bin/python",
local_requirements=local_requirements,
force=False,
)
self.assertEqual(plan.mode, "Selective Python environment sync")
self.assertIn("one resolver transaction", plan.reason)
self.assertEqual(len(plan.commands), 1)
command = plan.commands[0]
self.assertEqual(command[:5], ("/test/venv/bin/python", "-m", "pip", "install", "-e"))
self.assertEqual(command.count("-e"), 2)
self.assertIn(str(root / "govoplan-core"), command)
self.assertIn(str(root / "govoplan-access"), command)
def test_missing_editable_distribution_is_not_hidden_by_current_stamp(self) -> None:
sync = load_sync_module()
with tempfile.TemporaryDirectory(prefix="govoplan-python-sync-") as directory:
root = Path(directory)
project_root = root / "govoplan-probe-missing"
project_root.mkdir()
(project_root / "pyproject.toml").write_text(
"\n".join(
(
"[project]",
'name = "govoplan-probe-definitely-not-installed"',
'version = "0.1.0"',
"",
'[project.entry-points."govoplan.modules"]',
'probe_missing = "govoplan_probe.backend.manifest:get_manifest"',
"",
)
),
encoding="utf-8",
)
requirements = root / "requirements-dev.txt"
requirements.write_text("-e ./govoplan-probe-missing\n", encoding="utf-8")
entries = sync.local_requirement_entries(requirements)
validation = sync.validate_local_installations(sys.executable, entries)
plan = sync.build_environment_repair_plan(
python=sys.executable,
stale_requirements=validation.stale_requirements,
)
self.assertFalse(validation.current)
self.assertEqual(validation.stale_requirements, entries)
self.assertIn("distribution is not installed", validation.issues[0])
self.assertEqual(plan.mode, "Selective Python environment repair")
self.assertEqual(plan.commands[0][-2:], ("-e", str(project_root)))
def test_declared_module_entry_points_are_part_of_environment_validation(self) -> None:
sync = load_sync_module()
with tempfile.TemporaryDirectory(prefix="govoplan-python-sync-") as directory:
root = Path(directory)
project_root = root / "govoplan-probe"
project_root.mkdir()
(project_root / "pyproject.toml").write_text(
"\n".join(
(
"[project]",
'name = "govoplan-probe"',
'version = "0.1.0"',
"",
'[project.entry-points."govoplan.modules"]',
'probe = "govoplan_probe.backend.manifest:get_manifest"',
"",
)
),
encoding="utf-8",
)
requirements = root / "requirements-dev.txt"
requirements.write_text("-e ./govoplan-probe\n", encoding="utf-8")
expectations = sync.local_installation_expectations(
sync.local_requirement_entries(requirements)
)
self.assertEqual(len(expectations), 1)
self.assertEqual(expectations[0].distribution, "govoplan-probe")
self.assertEqual(
expectations[0].entry_points,
(("govoplan.modules", "probe", "govoplan_probe.backend.manifest:get_manifest"),),
)
if __name__ == "__main__":
unittest.main()
+277
View File
@@ -0,0 +1,277 @@
from __future__ import annotations
import csv
from datetime import UTC, datetime
import io
from pathlib import Path
import sys
import tempfile
import unittest
from unittest import mock
import zipfile
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.artifact_identity import ( # noqa: E402
ArtifactIdentityError,
inspect_python_wheel,
selected_artifact_identity_issues,
)
from govoplan_release.selective_catalog import ( # noqa: E402
apply_python_artifact_identities,
)
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
if str(ASSESSMENT_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(ASSESSMENT_TOOLS_ROOT))
from govoplan_assessment.installer_receipt import issue_installer_receipt # noqa: E402
class ReleaseArtifactIdentityTests(unittest.TestCase):
def test_built_wheel_bytes_become_catalog_identity(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
wheel = self._wheel(Path(temp_dir), console_script=True)
payload = self._catalog_payload()
changes = apply_python_artifact_identities(
payload,
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={"govoplan-demo": wheel},
)
identity = payload["release"]["artifacts"][0]
self.assertEqual("govoplan-demo", identity["package_name"])
self.assertEqual("1.2.3", identity["package_version"])
self.assertRegex(identity["archive_sha256"], r"^[0-9a-f]{64}$")
self.assertTrue(identity["requires_installer_receipt"])
self.assertEqual("release.artifact", changes[0].field)
self.assertEqual((), selected_artifact_identity_issues(payload))
def test_version_update_without_wheel_removes_stale_identity_and_blocks_publish(self) -> None:
payload = self._catalog_payload()
payload["release"]["artifacts"] = [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-demo",
"package_version": "1.2.2",
"archive_sha256": "a" * 64,
"archive_size": 10,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "b" * 64,
"file_count": 1,
},
"requires_installer_receipt": False,
}
]
apply_python_artifact_identities(
payload,
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={},
)
self.assertNotIn("artifacts", payload["release"])
self.assertIn(
"no built artifact identity",
" ".join(selected_artifact_identity_issues(payload)),
)
def test_tag_only_selection_needs_no_python_artifact_mapping(self) -> None:
payload = self._catalog_payload()
payload["release"]["selected_units"].append(
{"repo": "govoplan", "version": "1.2.3"}
)
apply_python_artifact_identities(
payload,
repo_versions={"govoplan": "1.2.3"},
python_artifacts={},
)
self.assertIn("selected_units", payload["release"])
def test_unsafe_or_mismatched_wheel_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
unsafe = root / "govoplan_demo-1.2.3-py3-none-any.whl"
with zipfile.ZipFile(unsafe, "w") as archive:
archive.writestr("../escape", b"bad")
archive.writestr(
"govoplan_demo-1.2.3.dist-info/METADATA",
"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: 1.2.3\n",
)
wrong = self._wheel(root, version="9.9.9")
with self.assertRaisesRegex(ArtifactIdentityError, "unsafe"):
inspect_python_wheel(unsafe)
with self.assertRaisesRegex(ValueError, "expected govoplan-demo 1.2.3"):
apply_python_artifact_identities(
self._catalog_payload(),
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={"govoplan-demo": wrong},
)
def test_receipt_issuer_hashes_exact_consumed_wheel(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
original = self._wheel(root / "original", value=b"VALUE = 1\n")
different = self._wheel(root / "different", value=b"VALUE = 2\n")
identity = inspect_python_wheel(original)
catalog = self._catalog_payload()
catalog["channel"] = "stable"
catalog["sequence"] = 1
catalog["release"]["artifacts"] = [identity.catalog_payload()]
assessment = {
"assessment_id": "assessment:test",
"release": {"ref": "stable-catalog-1"},
"composition": [{"module_id": "demo", "enabled": True}],
}
installed_payload = {
"algorithm": "govoplan-installed-record-payload-v1",
"sha256": "c" * 64,
"file_count": identity.payload_file_count,
}
evidence = {
"assessment_id": "assessment:test",
"assessment_release": "stable-catalog-1",
"collected_at": "2026-07-22T12:00:00Z",
"artifacts": [
{
"package_name": "govoplan-demo",
"package_version": "1.2.3",
"record_integrity": {
"status": "verified",
"hashed_file_count": identity.payload_file_count,
"permitted_unhashed_file_count": 1,
"generated_unhashed_file_count": 0,
"unverifiable_file_count": 0,
"missing_file_count": 0,
"mismatched_file_count": 0,
"artifact_payload_identity": catalog["release"]["artifacts"][0]["installed_payload"],
"installed_payload_identity": installed_payload,
},
}
],
}
key = Ed25519PrivateKey.generate()
receipt = issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": original},
issued_at=datetime(2026, 7, 22, 12, 1, tzinfo=UTC),
same_process_observation=True,
)
with self.assertRaisesRegex(ValueError, "differs from the signed"):
issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": different},
issued_at=datetime(2026, 7, 22, 12, 1, tzinfo=UTC),
same_process_observation=True,
)
with self.assertRaisesRegex(ValueError, "within five minutes"):
issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": original},
issued_at=datetime(2026, 7, 22, 12, 6, tzinfo=UTC),
same_process_observation=True,
)
self.assertEqual(identity.archive_sha256, receipt["artifacts"][0]["catalog_archive_sha256"])
def test_path_replacement_during_one_descriptor_inspection_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
target = self._wheel(root / "target", value=b"VALUE = 1\n")
replacement = self._wheel(root / "replacement", value=b"VALUE = 2\n")
real_zip = zipfile.ZipFile
def swap_path(file_or_path, *args, **kwargs):
target.unlink()
replacement.rename(target)
return real_zip(file_or_path, *args, **kwargs)
with (
mock.patch(
"govoplan_release.artifact_identity.zipfile.ZipFile",
side_effect=swap_path,
),
self.assertRaisesRegex(ArtifactIdentityError, "changed"),
):
inspect_python_wheel(target)
@staticmethod
def _catalog_payload() -> dict[str, object]:
return {
"core_release": {},
"modules": [
{
"module_id": "demo",
"version": "1.2.3",
"python_package": "govoplan-demo",
"python_ref": "govoplan-demo @ git+ssh://git@example.test/acme/govoplan-demo.git@v1.2.3",
}
],
"release": {
"selected_units": [
{"repo": "govoplan-demo", "version": "1.2.3"}
]
},
}
@staticmethod
def _wheel(
root: Path,
*,
version: str = "1.2.3",
console_script: bool = False,
value: bytes = b"VALUE = 1\n",
) -> Path:
root.mkdir(parents=True, exist_ok=True)
wheel = root / f"govoplan_demo-{version}-py3-none-any.whl"
dist_info = f"govoplan_demo-{version}.dist-info"
files: dict[str, bytes] = {
"govoplan_demo.py": value,
f"{dist_info}/METADATA": (
f"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: {version}\n"
).encode(),
f"{dist_info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
}
if console_script:
files[f"{dist_info}/entry_points.txt"] = (
b"[console_scripts]\ngovoplan-demo = govoplan_demo:main\n"
)
record_buffer = io.StringIO()
writer = csv.writer(record_buffer, lineterminator="\n")
for name, encoded in files.items():
writer.writerow((name, "", len(encoded)))
writer.writerow((f"{dist_info}/RECORD", "", ""))
files[f"{dist_info}/RECORD"] = record_buffer.getvalue().encode()
with zipfile.ZipFile(wheel, "w", compression=zipfile.ZIP_DEFLATED) as archive:
for name, encoded in files.items():
archive.writestr(name, encoded)
return wheel
if __name__ == "__main__":
unittest.main()
+184
View File
@@ -0,0 +1,184 @@
from __future__ import annotations
import json
from datetime import UTC, datetime
from pathlib import Path
import sys
import tempfile
import unittest
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
candidate_keyring_from_authenticated_base,
load_authenticated_catalog_base,
public_key_base64,
signature,
)
class ReleaseBaseCatalogTrustTests(unittest.TestCase):
def test_exact_signed_base_pair_is_loaded_once_and_carried_in_memory(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
authenticated = load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={"release-key": public_key_base64(private_key)},
)
keyring.unlink()
self.assertEqual("release-key", authenticated.keyring["keys"][0]["key_id"])
self.assertEqual(
canonical_hash(authenticated.keyring),
authenticated.catalog["release"]["keyring_sha256"],
)
def test_deliberate_old_to_new_signer_rotation_is_carried_forward(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
new_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
configured = {
"release-key": public_key_base64(private_key),
"release-key-next": public_key_base64(new_key),
}
authenticated = load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys=configured,
)
candidate = candidate_keyring_from_authenticated_base(
authenticated.keyring,
signer_public_keys=configured,
generated_at=datetime(2026, 7, 22, 12, tzinfo=UTC),
)
self.assertEqual(
{"release-key", "release-key-next"},
{item["key_id"] for item in candidate["keys"]},
)
def test_injected_extra_key_without_catalog_authorization_is_rejected(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
extra_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
payload = json.loads(keyring.read_text(encoding="utf-8"))
payload["keys"].append(
{
"key_id": "injected-key",
"status": "active",
"public_key": public_key_base64(extra_key),
}
)
keyring.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
def test_unpinned_or_symlinked_base_keyring_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
payload = json.loads(keyring.read_text(encoding="utf-8"))
payload["generated_at"] = "changed"
keyring.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
keyring.unlink()
outside = root / "outside.json"
outside.write_text("{}", encoding="utf-8")
keyring.symlink_to(outside)
with self.assertRaisesRegex(ValueError, "opened safely"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
@staticmethod
def _write_pair(
root: Path, *, private_key: Ed25519PrivateKey
) -> tuple[Path, Path]:
keys = [
{
"key_id": "release-key",
"status": "active",
"public_key": public_key_base64(private_key),
}
]
keyring_payload = {
"keyring_version": "1",
"purpose": "govoplan module package catalog signatures",
"keys": keys,
}
catalog_payload = {
"channel": "stable",
"sequence": 1,
"core_release": {"version": "1.0.0"},
"modules": [],
"release": {"keyring_sha256": canonical_hash(keyring_payload)},
}
catalog_payload["signatures"] = [
signature(
catalog_payload,
key_id="release-key",
private_key=private_key,
)
]
catalog = root / "stable.json"
keyring = root / "keyring.json"
catalog.write_text(json.dumps(catalog_payload), encoding="utf-8")
keyring.write_text(json.dumps(keyring_payload), encoding="utf-8")
return catalog, keyring
if __name__ == "__main__":
unittest.main()
+153
View File
@@ -0,0 +1,153 @@
from __future__ import annotations
import json
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.candidate_artifact import ( # noqa: E402
CandidateArtifactError,
candidate_output_path,
issue_candidate_id,
issue_candidate_receipt,
verify_candidate_receipt,
validate_release_channel,
)
class CandidateArtifactTests(unittest.TestCase):
def test_channel_is_one_canonical_basename(self) -> None:
self.assertEqual("stable_1", validate_release_channel("stable_1"))
for value in ("../stable", "/stable", ".", "..", "Stable", "stable/name"):
with self.subTest(value=value), self.assertRaises(CandidateArtifactError):
validate_release_channel(value)
def test_handle_is_deterministic_and_can_precede_output(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "not-created" / "release-candidates"
candidate_id = issue_candidate_id("rr-123", "attempt-456")
first = candidate_output_path(root, candidate_id)
second = candidate_output_path(root, candidate_id)
self.assertEqual(first, second)
self.assertEqual(candidate_id, first.name)
self.assertRegex(candidate_id, r"^candidate-[0-9a-f]{32}$")
def test_receipt_rejects_catalog_drift(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
catalog = self._write_candidate(root / candidate_id)
receipt = issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
resolved = verify_candidate_receipt(
root=root,
candidate_id=receipt.candidate_id,
catalog_sha256=receipt.catalog_sha256,
channel="stable",
)
catalog.write_text(
json.dumps({"channel": "stable", "sequence": 2, "signatures": [{}]}),
encoding="utf-8",
)
self.assertEqual(root / candidate_id, resolved)
with self.assertRaisesRegex(CandidateArtifactError, "no longer matches"):
verify_candidate_receipt(
root=root,
candidate_id=receipt.candidate_id,
catalog_sha256=receipt.catalog_sha256,
channel="stable",
)
def test_unissued_ids_traversal_and_symlinks_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
root.mkdir(mode=0o700)
candidate_id = issue_candidate_id("run", "attempt")
outside = Path(temp_dir) / "outside"
outside.mkdir()
(root / candidate_id).symlink_to(outside, target_is_directory=True)
with self.assertRaises(CandidateArtifactError):
candidate_output_path(root, "../candidate-" + "0" * 32)
with self.assertRaisesRegex(CandidateArtifactError, "real directory"):
candidate_output_path(root, candidate_id)
def test_catalog_and_channel_components_must_not_be_symlinks(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
candidate = root / candidate_id
target = Path(temp_dir) / "target"
target.mkdir()
(target / "stable.json").write_text(
json.dumps({"signatures": [{}]}), encoding="utf-8"
)
root.mkdir(mode=0o700)
candidate.mkdir(mode=0o700)
(candidate / "channels").symlink_to(target, target_is_directory=True)
with self.assertRaisesRegex(CandidateArtifactError, "real directory"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
def test_shared_candidate_roots_and_catalog_files_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
root.mkdir(mode=0o755)
candidate_id = issue_candidate_id("run", "attempt")
with self.assertRaisesRegex(CandidateArtifactError, "another user"):
candidate_output_path(root, candidate_id)
root.chmod(0o700)
catalog = self._write_candidate(root / candidate_id)
catalog.chmod(0o640)
with self.assertRaisesRegex(CandidateArtifactError, "another user"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
def test_receipt_rejects_catalog_with_inconsistent_channel(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
catalog = self._write_candidate(root / candidate_id)
catalog.write_text(
json.dumps({"channel": "next", "signatures": [{}]}),
encoding="utf-8",
)
with self.assertRaisesRegex(CandidateArtifactError, "does not match"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
@staticmethod
def _write_candidate(candidate: Path) -> Path:
candidate.parent.mkdir(mode=0o700, exist_ok=True)
candidate.mkdir(mode=0o700)
channels = candidate / "channels"
channels.mkdir(mode=0o700)
catalog = channels / "stable.json"
catalog.write_text(
json.dumps({"channel": "stable", "sequence": 1, "signatures": [{}]}),
encoding="utf-8",
)
catalog.chmod(0o600)
return catalog
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,168 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
from govoplan_core.core.modules import ModuleManifest
from govoplan_core.core.provider_governance import (
ExternalProviderDeclaration,
ModuleArchitectureDeclaration,
ModuleMaturityEvidence,
ProviderBehaviorDeclaration,
ProviderObjectDeclaration,
)
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.catalog_entry_synthesis import ( # noqa: E402
manifest_catalog_entry,
validate_initial_entry_closure,
)
from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402
class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
def test_catalog_entry_preserves_architecture_and_provider_declarations(
self,
) -> None:
provider = ExternalProviderDeclaration(
id="example.records",
module_id="example",
label="Example records",
maturity="read",
operations=("read",),
objects=(
ProviderObjectDeclaration(
object_type="record",
field_groups=("identity", "content"),
authority_modes=("external_mirror",),
default_authority_mode="external_mirror",
),
),
behavior=ProviderBehaviorDeclaration(
freshness="Reports the acquisition timestamp.",
health="Reports source and parser health.",
max_read_items=100,
classifications=("internal",),
purposes=("release contract test",),
retention="The owning package retention policy applies.",
),
)
architecture = ModuleArchitectureDeclaration(
layer="data_reporting_integration",
kind="integration",
maturity="vertical_slice",
evidence=(
ModuleMaturityEvidence(
kind="test",
reference="tests/test_release_catalog_entry_synthesis.py",
summary="Proves catalog metadata preservation.",
),
ModuleMaturityEvidence(
kind="documentation",
reference="docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md",
summary="Defines the provider declaration contract.",
),
),
known_limits=("Test-only provider.",),
supported_authority_modes=("external_mirror",),
owned_concepts=("example transport",),
target_tested_providers=(provider.id,),
)
entry = manifest_catalog_entry(
manifest=ModuleManifest(
id="example",
name="Example",
version="1.2.3",
architecture=architecture,
external_providers=(provider,),
),
repo="govoplan-example",
package="govoplan-example",
version="1.2.3",
description=None,
root=META_ROOT,
repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN",
)
self.assertEqual("vertical_slice", entry["architecture"]["maturity"])
self.assertEqual(
"external_mirror",
entry["external_providers"][0]["objects"][0][
"default_authority_mode"
],
)
def test_selective_update_synthesizes_initial_entries_from_package_manifests(self) -> None:
payload: dict[str, object] = {
"core_release": {},
"release": {},
"modules": [
{
"module_id": "access",
"version": "0.1.11",
"python_package": "govoplan-access",
"python_ref": "govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.11",
}
],
}
changes = apply_repo_updates(
payload,
repo_versions={
"govoplan-addresses": "0.1.9",
"govoplan-poll": "0.1.11",
"govoplan-scheduling": "0.1.11",
},
repo_contracts={},
repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN",
workspace=META_ROOT.parent,
)
modules = {
item["module_id"]: item
for item in payload["modules"] # type: ignore[index]
}
self.assertEqual({"access", "addresses", "poll", "scheduling"}, set(modules))
self.assertEqual("@govoplan/addresses-webui", modules["addresses"]["webui_package"])
self.assertIn(
{"name": "addresses.people_search", "version": "0.1.0"},
modules["addresses"]["provides_interfaces"],
)
self.assertEqual("govoplan-poll", modules["poll"]["python_package"])
self.assertEqual(["poll"], modules["scheduling"]["dependencies"])
self.assertEqual("@govoplan/scheduling-webui", modules["scheduling"]["webui_package"])
self.assertEqual("requires_review", modules["scheduling"]["migration_safety"])
self.assertTrue(
any(
item["name"] == "poll.governed_participation" and item["optional"] is False
for item in modules["scheduling"]["requires_interfaces"]
)
)
self.assertEqual(
{"govoplan-addresses", "govoplan-poll", "govoplan-scheduling"},
{change.repo for change in changes},
)
self.assertEqual({"catalog_entry"}, {change.field for change in changes})
def test_initial_required_dependency_must_be_represented(self) -> None:
with self.assertRaisesRegex(ValueError, "requires catalog module 'poll'"):
validate_initial_entry_closure(
catalog_modules=[
{
"module_id": "scheduling",
"dependencies": ["poll"],
}
],
initial_module_ids={"scheduling"},
)
if __name__ == "__main__":
unittest.main()
+796
View File
@@ -0,0 +1,796 @@
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.publisher import ( # noqa: E402
FrozenPublicationRemote,
bind_publication_remote,
commit_publication_tree,
publication_mutation_trust_issues,
publish_catalog_candidate,
remote_publication_identity,
run_checked,
verify_committed_publication,
verify_remote_branch_head,
verify_remote_publication,
_git_bytes,
_sanitized_git_environment,
_seal_git_metadata_file,
_trusted_npm_command,
)
class ReleaseCatalogPublicationTests(unittest.TestCase):
def test_publication_git_writes_ignore_permissive_operator_umask(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
repository = Path(temp_dir) / "website"
repository.mkdir()
self._git(repository, "init", "--quiet")
payload_number = 0
while True:
payload = f"private publication object {payload_number}\n".encode()
header = f"blob {len(payload)}\0".encode()
expected_object = hashlib.sha1(
header + payload,
usedforsecurity=False,
).hexdigest()
object_parent = repository / ".git" / "objects" / expected_object[:2]
if not object_parent.exists():
break
payload_number += 1
previous_umask = os.umask(0o002)
try:
object_id = (
_git_bytes(
repository,
"hash-object",
"-w",
"--stdin",
input_bytes=payload,
)
.decode("ascii")
.strip()
)
finally:
os.umask(previous_umask)
object_path = object_parent / expected_object[2:]
self.assertEqual(expected_object, object_id)
self.assertEqual(os.geteuid(), object_parent.stat().st_uid)
self.assertEqual(0o700, object_parent.stat().st_mode & 0o777)
self.assertEqual(os.geteuid(), object_path.stat().st_uid)
self.assertEqual(0o400, object_path.stat().st_mode & 0o777)
def test_publication_git_metadata_is_sealed_after_git_writes(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
metadata = Path(temp_dir) / "index"
metadata.write_bytes(b"index")
metadata.chmod(0o664)
_seal_git_metadata_file(metadata, label="test index")
self.assertEqual(0o600, metadata.stat().st_mode & 0o777)
def test_publication_git_identity_is_fixed_and_non_personal(self) -> None:
with patch.dict(
os.environ,
{
"GIT_AUTHOR_NAME": "Attacker",
"GIT_AUTHOR_EMAIL": "attacker@example.test",
"GIT_COMMITTER_NAME": "Attacker",
"GIT_COMMITTER_EMAIL": "attacker@example.test",
},
clear=False,
):
environment = _sanitized_git_environment()
self.assertEqual(
"GovOPlaN Release Automation",
environment["GIT_AUTHOR_NAME"],
)
self.assertEqual(
"release@govoplan.invalid",
environment["GIT_AUTHOR_EMAIL"],
)
self.assertEqual(
environment["GIT_AUTHOR_NAME"],
environment["GIT_COMMITTER_NAME"],
)
self.assertEqual(
environment["GIT_AUTHOR_EMAIL"],
environment["GIT_COMMITTER_EMAIL"],
)
def test_build_command_uses_its_pinned_node_directory(self) -> None:
completed = subprocess.CompletedProcess(
["/trusted/node/bin/npm"],
0,
stdout=b"",
stderr=b"",
)
with patch(
"govoplan_release.publisher.subprocess.run",
return_value=completed,
) as runner:
run_checked(
["/trusted/node/bin/npm", "run", "build"],
cwd=Path("/trusted/website"),
)
environment = runner.call_args.kwargs["env"]
self.assertEqual(
"/trusted/node/bin:/usr/bin:/bin",
environment["PATH"],
)
def test_npm_command_requires_trusted_npm_and_sibling_node(self) -> None:
with (
patch(
"govoplan_release.publisher.shutil.which",
return_value="/trusted/node/bin/npm",
),
patch(
"govoplan_release.publisher._trusted_runtime_executable_issue",
side_effect=(None, None),
) as trust_check,
):
self.assertEqual(
"/trusted/node/bin/npm",
_trusted_npm_command("npm"),
)
self.assertEqual(Path("/trusted/node/bin/npm"), trust_check.call_args_list[0].args[0])
self.assertEqual(Path("/trusted/node/bin/node"), trust_check.call_args_list[1].args[0])
def test_npm_command_rejects_caller_relative_path(self) -> None:
with (
patch("govoplan_release.publisher.shutil.which") as which,
self.assertRaisesRegex(RuntimeError, "absolute path or the bare name"),
):
_trusted_npm_command("./npm")
which.assert_not_called()
def test_committed_publication_must_match_validated_blobs_exactly(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
web_root = Path(tmp) / "website"
module_root = web_root / "public" / "catalogs" / "v1" / "modules"
module_root.mkdir(parents=True)
catalog = (
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
)
keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
catalog.parent.mkdir(parents=True)
expected = {
catalog.relative_to(web_root).as_posix(): b'{"catalog":true}\n',
keyring.relative_to(web_root).as_posix(): b'{"keys":[]}\n',
(module_root / "index.json")
.relative_to(web_root)
.as_posix(): b'{"modules":[]}\n',
}
for relative, encoded in expected.items():
target = web_root / relative
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(encoded)
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "publication")
commit_sha = self._git(web_root, "rev-parse", "HEAD").strip()
verify_committed_publication(
web_root=web_root,
commit_sha=commit_sha,
expected_blobs=expected,
module_root=module_root,
)
changed = dict(expected)
changed[catalog.relative_to(web_root).as_posix()] = b'{"catalog":false}\n'
with self.assertRaisesRegex(RuntimeError, "differs"):
verify_committed_publication(
web_root=web_root,
commit_sha=commit_sha,
expected_blobs=changed,
module_root=module_root,
)
catalog_path = catalog.relative_to(web_root).as_posix()
self._git(web_root, "update-index", "--chmod=+x", catalog_path)
self._git(web_root, "commit", "--quiet", "-m", "unsafe mode")
executable_commit = self._git(web_root, "rev-parse", "HEAD").strip()
with self.assertRaisesRegex(RuntimeError, "regular blob"):
verify_committed_publication(
web_root=web_root,
commit_sha=executable_commit,
expected_blobs=expected,
module_root=module_root,
)
def test_private_index_commit_has_one_parent_and_only_computed_delta(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
web_root = root / "website"
module_root = web_root / "public" / "catalogs" / "v1" / "modules"
channel = (
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
)
keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
old_module = module_root / "obsolete.json"
unrelated = web_root / "unrelated.txt"
for path, encoded in (
(channel, b'{"old":true}\n'),
(keyring, b'{"keys":[]}\n'),
(old_module, b'{"obsolete":true}\n'),
(unrelated, b"keep\n"),
):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(encoded)
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
frozen_head = self._git(web_root, "rev-parse", "HEAD").strip()
branch = self._git(web_root, "branch", "--show-current").strip()
malicious = web_root / "not-in-publication.txt"
malicious.write_text("staged but excluded\n", encoding="utf-8")
self._git(web_root, "add", malicious.name)
marker = root / "reference-hook-ran"
hook = web_root / ".git" / "hooks" / "reference-transaction"
hook.write_text(f"#!/bin/sh\ntouch {marker}\n", encoding="utf-8")
hook.chmod(0o755)
expected = {
channel.relative_to(web_root).as_posix(): b'{"new":true}\n',
keyring.relative_to(web_root).as_posix(): b'{"keys":["release"]}\n',
(module_root / "index.json")
.relative_to(web_root)
.as_posix(): b'{"modules":[]}\n',
}
redirected = root / "attacker-index"
with patch.dict(
os.environ,
{
"GIT_DIR": str(root / "attacker-git-dir"),
"GIT_INDEX_FILE": str(redirected),
"GIT_OBJECT_DIRECTORY": str(root / "attacker-objects"),
"GIT_CONFIG_GLOBAL": str(root / "attacker-config"),
},
):
commit_sha = commit_publication_tree(
web_root=web_root,
frozen_head=frozen_head,
branch=branch,
expected_blobs=expected,
module_root=module_root,
message="Exact publication",
)
parents = self._git(
web_root, "rev-list", "--parents", "-n", "1", commit_sha
).split()
changed = set(
filter(
None,
self._git(
web_root,
"diff-tree",
"--no-commit-id",
"--name-only",
"-r",
frozen_head,
commit_sha,
).splitlines(),
)
)
hook_ran = marker.exists()
inherited_index_used = redirected.exists()
commit_paths = self._git(
web_root, "ls-tree", "-r", "--name-only", commit_sha
)
self.assertEqual([commit_sha, frozen_head], parents)
self.assertEqual(
{
"public/catalogs/v1/channels/stable.json",
"public/catalogs/v1/keyring.json",
"public/catalogs/v1/modules/index.json",
"public/catalogs/v1/modules/obsolete.json",
},
changed,
)
self.assertFalse(hook_ran)
self.assertFalse(inherited_index_used)
self.assertNotIn("not-in-publication.txt", commit_paths)
def test_remote_binding_and_annotated_publication_identity_are_exact(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
remote_root = root / "website.git"
remote_root.mkdir()
self._git(remote_root, "init", "--bare", "--quiet")
web_root = root / "website"
web_root.mkdir()
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "branch", "-M", "main")
web_root.joinpath("base.txt").write_text("base\n", encoding="utf-8")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
self._git(web_root, "remote", "add", "origin", str(remote_root))
self._git(web_root, "push", "--quiet", "-u", "origin", "main")
base_commit = self._git(web_root, "rev-parse", "HEAD").strip()
frozen = bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
self.assertIsInstance(frozen, FrozenPublicationRemote)
verify_remote_branch_head(
web_root=web_root,
remote_url=frozen.url,
branch="main",
expected_commit=base_commit,
)
web_root.joinpath("catalog.json").write_text("{}\n", encoding="utf-8")
self._git(web_root, "add", "catalog.json")
self._git(web_root, "commit", "--quiet", "-m", "publication")
commit_sha = self._git(web_root, "rev-parse", "HEAD").strip()
tag_name = "catalog-test"
self._git(web_root, "tag", "-a", tag_name, "-m", "publication")
tag_object = self._git(
web_root, "rev-parse", f"refs/tags/{tag_name}"
).strip()
self._git(
web_root,
"push",
"--quiet",
"--atomic",
"origin",
f"{commit_sha}:refs/heads/main",
f"{tag_object}:refs/tags/{tag_name}",
)
identity = remote_publication_identity(
web_root=web_root,
remote_url=frozen.url,
branch="main",
tag_name=tag_name,
)
verified = verify_remote_publication(
web_root=web_root,
remote_url=frozen.url,
branch="main",
tag_name=tag_name,
expected_commit=commit_sha,
expected_tag_object=tag_object,
)
self._git(
web_root,
"remote",
"set-url",
"--add",
"--push",
"origin",
str(root / "other.git"),
)
with self.assertRaisesRegex(RuntimeError, "do not match"):
bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
self.assertEqual(identity, verified)
self.assertEqual(commit_sha, identity["publication_commit_sha"])
self.assertEqual(tag_object, identity["publication_tag_object_sha"])
self.assertEqual(commit_sha, identity["publication_tag_commit_sha"])
def test_mutation_rejects_writable_website_and_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
web_root = root / "website"
web_root.mkdir()
self._git(web_root, "init", "--quiet")
candidate = self._candidate(root, key="trusted-key")
target_root = web_root / "public" / "catalogs" / "v1"
target_catalog = target_root / "channels" / "stable.json"
target_keyring = target_root / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text("{}\n", encoding="utf-8")
web_root.chmod(0o777)
root_issues = publication_mutation_trust_issues(
web_root=web_root,
candidate_catalog=candidate / "channels" / "stable.json",
candidate_keyring=candidate / "keyring.json",
target_catalog=target_catalog,
target_keyring=target_keyring,
target_modules=target_root / "modules",
)
web_root.chmod(0o755)
config = web_root / ".git" / "config"
config.chmod(0o666)
config_issues = publication_mutation_trust_issues(
web_root=web_root,
candidate_catalog=candidate / "channels" / "stable.json",
candidate_keyring=candidate / "keyring.json",
target_catalog=target_catalog,
target_keyring=target_keyring,
target_modules=target_root / "modules",
)
self.assertIn("website root is writable by another user", root_issues)
self.assertIn("website Git config is writable by another user", config_issues)
def test_push_returns_only_independently_verified_publication_receipt(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog["sequence"] = 7
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
],
"artifacts": [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-core",
"package_version": "1.2.3",
"archive_sha256": "c" * 64,
"archive_size": 100,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "d" * 64,
"file_count": 1,
},
"requires_installer_receipt": True,
}
],
}
catalog["signatures"] = [{}]
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
remote_root = root / "website.git"
remote_root.mkdir()
self._git(remote_root, "init", "--bare", "--quiet")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "branch", "-M", "main")
self._git(web_root, "remote", "add", "origin", str(remote_root))
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
self._git(web_root, "push", "--quiet", "-u", "origin", "main")
frozen_head = self._git(web_root, "rev-parse", "HEAD").strip()
frozen_remote = bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.publication_runtime_trust_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.registered_website_remote",
return_value=str(remote_root),
),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
push=True,
branch="main",
tag_name="catalog-test",
expected_website_head=frozen_head,
expected_website_branch="main",
expected_remote_sha256=frozen_remote.sha256,
)
remote_identity = remote_publication_identity(
web_root=web_root,
remote_url=str(remote_root),
branch="main",
tag_name="catalog-test",
)
self.assertEqual("published", result.status)
self.assertEqual("origin", result.remote)
self.assertEqual(
remote_identity["publication_commit_sha"], result.publication_commit_sha
)
self.assertEqual(
remote_identity["publication_tag_object_sha"],
result.publication_tag_object_sha,
)
self.assertEqual(
remote_identity["publication_tag_commit_sha"],
result.publication_tag_commit_sha,
)
def test_apply_writes_validated_objects_even_if_candidate_path_changes(
self,
) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog.update({"channel": "stable", "sequence": 1})
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
],
"artifacts": [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-core",
"package_version": "1.2.3",
"archive_sha256": "c" * 64,
"archive_size": 100,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "d" * 64,
"file_count": 1,
},
"requires_installer_receipt": True,
}
],
}
catalog["signatures"] = [{}]
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
registered_remote = "ssh://example.test/release/website.git"
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "remote", "add", "origin", registered_remote)
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
def validate_and_swap(*args, **kwargs):
del args, kwargs
catalog_path.write_text(
json.dumps(
{
"channel": "stable",
"sequence": 999,
"malicious": True,
"signatures": [{}],
}
),
encoding="utf-8",
)
return {"valid": True, "warnings": [], "error": None}
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
side_effect=validate_and_swap,
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch("govoplan_release.publisher.website_dirty", return_value=False),
patch(
"govoplan_release.publisher.publication_runtime_trust_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.registered_website_remote",
return_value=registered_remote,
),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
allow_dirty_website=True,
)
published = json.loads(
(
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
).read_text(encoding="utf-8")
)
self.assertEqual("applied", result.status)
self.assertEqual(1, published["sequence"])
self.assertNotIn("malicious", published)
def test_apply_blocks_selected_python_release_without_built_identity(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
]
}
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
(web_root / ".git").mkdir()
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch("govoplan_release.publisher.website_dirty", return_value=False),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
)
self.assertEqual("blocked", result.status)
self.assertIn(
"selected Python repository govoplan-core has no built artifact identity",
" ".join(result.notes),
)
def test_publication_requires_an_existing_trust_anchor(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="candidate-key")
web_root = root / "website"
web_root.mkdir()
with patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
)
self.assertEqual("blocked", result.status)
self.assertIn("publication trust anchor is missing", " ".join(result.notes))
def test_publication_rejects_rebinding_an_existing_key_id(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="replacement-key")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
with patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
)
self.assertEqual("blocked", result.status)
self.assertIn("changes the public key", " ".join(result.notes))
@staticmethod
def _candidate(root: Path, *, key: str) -> Path:
candidate = root / "candidate"
channel = candidate / "channels"
channel.mkdir(parents=True)
channel.joinpath("stable.json").write_text(
json.dumps(
{
"channel": "stable",
"core_release": {
"version": "1.2.3",
"python_ref": (
"govoplan-core @ git+ssh://git@example.test/acme/"
"govoplan-core.git@v1.2.3"
),
},
"modules": [],
}
)
)
candidate.joinpath("keyring.json").write_text(
json.dumps(ReleaseCatalogPublicationTests._keyring(key))
)
return candidate
@staticmethod
def _keyring(key: str) -> dict[str, object]:
return {
"keys": [
{
"key_id": "release-key",
"status": "active",
"public_key": key,
}
]
}
@staticmethod
def _git(root: Path, *args: str) -> str:
return subprocess.run(
["git", *args],
cwd=root,
check=True,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
).stdout
if __name__ == "__main__":
unittest.main()
+66
View File
@@ -0,0 +1,66 @@
from __future__ import annotations
import subprocess
import sys
import unittest
from pathlib import Path
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from server.app import create_app # noqa: E402
class ReleaseConsoleSecurityTests(unittest.TestCase):
def test_api_token_is_accepted_only_from_header(self) -> None:
with TestClient(create_app(workspace_root=META_ROOT, token="test-console-token")) as client:
query_response = client.get("/api/health?token=test-console-token")
header_response = client.get(
"/api/health",
headers={"X-Release-Console-Token": "test-console-token"},
)
self.assertEqual(query_response.status_code, 401)
self.assertEqual(header_response.status_code, 200)
def test_bootstrap_token_uses_and_clears_url_fragment(self) -> None:
launcher = (RELEASE_ROOT / "release-console.py").read_text(encoding="utf-8")
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
self.assertIn("#token={token}", launcher)
self.assertNotIn("?token={token}", launcher)
self.assertIn("window.location.hash.slice(1)", webui)
self.assertIn("history.replaceState", webui)
def test_tokenless_embedding_is_read_only(self) -> None:
with TestClient(create_app(workspace_root=META_ROOT)) as client:
read_response = client.get("/api/health")
write_response = client.post("/api/selective-plan", json={})
self.assertEqual(200, read_response.status_code)
self.assertEqual(403, write_response.status_code)
self.assertIn("read-only", write_response.json()["detail"])
def test_launcher_rejects_non_loopback_and_tokenless_modes(self) -> None:
launcher = RELEASE_ROOT / "release-console.py"
for arguments in (("--host", "0.0.0.0"), ("--no-token",)):
with self.subTest(arguments=arguments):
result = subprocess.run(
(sys.executable, str(launcher), *arguments),
cwd=META_ROOT,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=10,
)
self.assertEqual(2, result.returncode)
if __name__ == "__main__":
unittest.main()
+177
View File
@@ -0,0 +1,177 @@
from __future__ import annotations
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from server.app import create_app # noqa: E402
class ReleaseDashboardDiagnosticsTests(unittest.TestCase):
def test_malformed_core_version_is_bounded_and_blocks_api_plans(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
core = workspace / "govoplan-core"
core.mkdir()
(core / "pyproject.toml").write_text("[project\n", encoding="utf-8")
with TestClient(create_app(workspace_root=workspace)) as client:
dashboard_response = client.get(
"/api/dashboard", params={"public_catalog": "false"}
)
selective_response = client.get(
"/api/selective-plan",
params={
"repos": "govoplan-core",
"repo_versions": "govoplan-core:1.2.4",
"public_catalog": "false",
},
)
full_plan_response = client.get(
"/api/plan", params={"public_catalog": "false"}
)
self.assertEqual(200, dashboard_response.status_code)
dashboard = dashboard_response.json()
self.assertEqual("blocked", dashboard["summary"]["status"])
error = next(
item
for item in dashboard["collection_errors"]
if item["code"] == "core_version_metadata_unreadable"
)
self.assertEqual("govoplan-core/pyproject.toml", error["source"])
self.assertIn("TOMLDecodeError", error["message"])
self.assertIn("Repair govoplan-core/pyproject.toml", error["remediation"])
self.assertLess(len(error["message"]), 200)
self.assertNotIn(str(workspace), error["message"])
self.assertEqual(200, selective_response.status_code)
selective_plan = selective_response.json()
self.assertEqual("blocked", selective_plan["status"])
self.assertFalse(selective_plan["source_preflight_ready"])
finding = next(
item
for item in selective_plan["gate_findings"]
if item["code"] == "core_version_metadata_unreadable"
)
self.assertEqual(error["remediation"], finding["remediation"])
self.assertEqual(
"resolve_release_gate", selective_plan["recommended_action"]["id"]
)
self.assertEqual(200, full_plan_response.status_code)
full_plan = full_plan_response.json()
self.assertEqual("blocked", full_plan["status"])
self.assertTrue(
any(
action["id"].startswith("dashboard:core_version_metadata_unreadable:")
for action in full_plan["actions"]
)
)
def test_malformed_manifest_is_not_silently_omitted_from_selected_plan(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
core = workspace / "govoplan-core"
core.mkdir()
(core / "pyproject.toml").write_text(
'[project]\nname = "govoplan-core"\nversion = "0.1.13"\n',
encoding="utf-8",
)
files = workspace / "govoplan-files"
manifest = files / "src" / "govoplan_files" / "backend" / "manifest.py"
manifest.parent.mkdir(parents=True)
(files / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
manifest.write_text("def broken(:\n", encoding="utf-8")
initialize_repository(files)
with TestClient(create_app(workspace_root=workspace)) as client:
dashboard_response = client.get(
"/api/dashboard", params={"public_catalog": "false"}
)
selective_response = client.get(
"/api/selective-plan",
params={
"repos": "govoplan-files",
"repo_versions": "govoplan-files:1.2.4",
"public_catalog": "false",
},
)
self.assertEqual(200, dashboard_response.status_code)
dashboard = dashboard_response.json()
error = next(
item
for item in dashboard["collection_errors"]
if item["code"] == "module_contract_unreadable"
)
self.assertEqual("govoplan-files", error["repo"])
self.assertEqual(
"govoplan-files/src/govoplan_files/backend/manifest.py",
error["source"],
)
self.assertIn("SyntaxError", error["message"])
self.assertIn("Repair the manifest", error["remediation"])
self.assertEqual(200, selective_response.status_code)
plan = selective_response.json()
self.assertEqual("blocked", plan["status"])
self.assertFalse(plan["source_preflight_ready"])
self.assertIn(
"module_contract_unreadable",
{finding["code"] for finding in plan["gate_findings"]},
)
self.assertEqual("resolve_release_gate", plan["recommended_action"]["id"])
def initialize_repository(path: Path) -> None:
subprocess.run(
["git", "init", "--initial-branch=main", str(path)],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
subprocess.run(
["git", "-C", str(path), "add", "."],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
subprocess.run(
[
"git",
"-C",
str(path),
"-c",
"user.name=Release Test",
"-c",
"user.email=release@example.invalid",
"-c",
"commit.gpgsign=false",
"commit",
"-m",
"Initial fixture",
],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
if __name__ == "__main__":
unittest.main()
+16
View File
@@ -51,6 +51,22 @@ class ReleaseDoctorTests(unittest.TestCase):
self.assertEqual("repo-dirty", findings[0].check_id) self.assertEqual("repo-dirty", findings[0].check_id)
self.assertIn("git status --short", [item.command for item in findings[0].commands]) self.assertIn("git status --short", [item.command for item in findings[0].commands])
def test_repo_version_mismatch_blocks_release(self) -> None:
doctor = load_doctor_module()
repo = doctor.RepoState(
name="govoplan-files",
path="/tmp/govoplan-files",
exists=True,
pyproject_version="0.1.9",
package_version="0.1.8",
webui_package_version="0.1.9",
)
findings = doctor.repo_findings((repo,), target_version="0.1.9", target_tag="v0.1.9")
mismatch = next(item for item in findings if item.check_id == "repo-version-mismatch")
self.assertEqual("blocker", mismatch.severity)
def test_dubious_ownership_errors_suggest_safe_directory_command(self) -> None: def test_dubious_ownership_errors_suggest_safe_directory_command(self) -> None:
doctor = load_doctor_module() doctor = load_doctor_module()
repo = doctor.RepoState( repo = doctor.RepoState(
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
import json
from pathlib import Path
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
class ReleaseEntrypointGateTests(unittest.TestCase):
def test_lockstep_release_includes_every_registered_module_and_connector(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
registry = json.loads((META_ROOT / "repositories.json").read_text())
expected = {
item["name"]
for item in registry["repositories"]
if item["category"] not in {"system", "website"}
}
missing = sorted(repo for repo in expected if f'$PARENT/{repo}"' not in script)
self.assertEqual([], missing)
def test_lockstep_release_runs_source_and_full_gates_before_remote_push(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
manifest_gate = script.index("run_manifest_shape_gate\n\nconfirm_release")
confirm = script.index("\nconfirm_release\n", manifest_gate)
workflow = script[confirm:]
source_gate = workflow.index("run_version_alignment_gate source")
first_commit = workflow.index('run git -C "$repo" commit')
lock_generation = workflow.index("generate_release_lock")
full_gate = workflow.index("run_version_alignment_gate", source_gate + 1)
first_push = workflow.index('run git -C "$repo" push')
self.assertLess(source_gate, first_commit)
self.assertLess(first_commit, lock_generation)
self.assertLess(lock_generation, full_gate)
self.assertLess(full_gate, first_push)
self.assertLess(manifest_gate, confirm)
def test_source_catalog_generator_enforces_explicit_repo_versions(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text()
gate = script.index("selected_repository_version_issues(")
write = script.index("output.write_text(")
self.assertLess(gate, write)
def test_candidate_publication_uses_existing_keyring_as_trust_anchor(self) -> None:
publisher = (META_ROOT / "tools" / "release" / "govoplan_release" / "publisher.py").read_text()
self.assertIn("candidate_catalog_version_issues(candidate_payload)", publisher)
self.assertIn("trusted_keys_from_keyring(\n target_keyring_payload", publisher)
self.assertIn("trusted_keys=publication_trust", publisher)
self.assertNotIn("trusted_keys=candidate_keys", publisher)
self.assertIn("module_directory_payloads(", publisher)
self.assertIn("verify_committed_publication(", publisher)
self.assertNotIn("shutil.copytree(candidate_modules", publisher)
def test_release_integration_honors_independent_module_versions(self) -> None:
script = (META_ROOT / "tools" / "checks" / "check-release-integration.sh").read_text()
self.assertNotIn('expected_tag = os.environ["RELEASE_TAG"]', script)
self.assertIn('dependency_version = tag_match.group(1)', script)
self.assertIn('repo_tag="$(release_tag_for_package "$repo")"', script)
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large Load Diff
+67
View File
@@ -0,0 +1,67 @@
from __future__ import annotations
from pathlib import Path
import subprocess
import sys
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release import git_state # noqa: E402
class ReleaseGitStateTests(unittest.TestCase):
def test_manifest_version_does_not_confuse_interface_versions(self) -> None:
from tempfile import TemporaryDirectory
with TemporaryDirectory() as directory:
root = Path(directory) / "govoplan-example"
manifest = root / "src" / "govoplan_example" / "backend" / "manifest.py"
manifest.parent.mkdir(parents=True)
manifest.write_text(
"\n".join(
(
'MODULE_VERSION = "1.2.3"',
"manifest = ModuleManifest(",
' id="example",',
" version=MODULE_VERSION,",
" provides_interfaces=(",
' ModuleInterfaceProvider(name="example.items", version="9.8.7"),',
" ),",
")",
"",
)
),
encoding="utf-8",
)
versions = git_state.read_manifest_versions(root)
self.assertEqual(("1.2.3",), versions)
def test_git_trusts_only_the_resolved_repository_for_each_command(self) -> None:
repository = Path("/workspace/../workspace/govoplan-core")
completed = subprocess.CompletedProcess([], 0, "", "")
with patch.object(git_state.subprocess, "run", return_value=completed) as run:
result = git_state.git(repository, "status", "--porcelain")
self.assertEqual(result.returncode, 0)
command = run.call_args.args[0]
self.assertIn(f"safe.directory={repository.resolve()}", command)
self.assertNotIn("safe.directory=*", command)
self.assertEqual(run.call_args.kwargs["cwd"], repository)
self.assertEqual(
run.call_args.kwargs["env"]["GIT_CONFIG_GLOBAL"],
"/dev/null",
)
if __name__ == "__main__":
unittest.main()
+72
View File
@@ -0,0 +1,72 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.http_fetch import ( # noqa: E402
DEFAULT_MAX_RESPONSE_BYTES,
fetch_http,
)
class ReleaseHttpFetchTests(unittest.TestCase):
def test_content_length_over_default_limit_blocks_before_read(self) -> None:
response = FakeResponse(
body=b"ignored",
headers={"Content-Length": str(DEFAULT_MAX_RESPONSE_BYTES + 1)},
)
with mock.patch(
"govoplan_release.http_fetch.urllib.request.urlopen",
return_value=response,
):
with self.assertRaisesRegex(ValueError, "exceeds the configured size"):
fetch_http("https://example.invalid/catalog.json", timeout=1)
self.assertEqual([], response.read_sizes)
def test_streamed_body_is_capped_when_length_is_absent(self) -> None:
response = FakeResponse(body=b"x" * 9, headers={})
with mock.patch(
"govoplan_release.http_fetch.urllib.request.urlopen",
return_value=response,
):
with self.assertRaisesRegex(ValueError, "exceeds the configured size"):
fetch_http(
"https://example.invalid/catalog.json",
timeout=1,
max_response_bytes=8,
)
self.assertEqual([9], response.read_sizes)
class FakeResponse:
def __init__(self, *, body: bytes, headers: dict[str, str]) -> None:
self.status = 200
self.headers = headers
self.body = body
self.read_sizes: list[int] = []
def __enter__(self) -> FakeResponse:
return self
def __exit__(self, *args: object) -> None:
return None
def read(self, size: int) -> bytes:
self.read_sizes.append(size)
return self.body[:size]
if __name__ == "__main__":
unittest.main()
+167
View File
@@ -0,0 +1,167 @@
from __future__ import annotations
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from tools.checks.release_integration import (
SOURCE_COUPLED_CORE_TESTS,
InstalledModuleArtifact,
artifact_contract_issues,
filter_test_suite,
release_package_names,
run_module_release_tests,
)
def _named_test(test_id: str) -> unittest.TestCase:
class NamedTest(unittest.TestCase):
def id(self) -> str:
return test_id
def runTest(self) -> None:
return None
return NamedTest()
class ReleaseIntegrationTests(unittest.TestCase):
def test_release_requirement_parser_selects_only_immutable_module_refs(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
requirements = Path(temp_dir) / "requirements-release.txt"
requirements.write_text(
"\n".join(
(
"../govoplan-core[server]",
"govoplan-idm @ git+ssh://git@example.test/GovOPlaN/govoplan-idm.git@v0.1.8",
"govoplan-campaign @ git+ssh://git@example.test/GovOPlaN/govoplan-campaign.git@v0.1.8",
"govoplan-idm @ git+ssh://git@example.test/GovOPlaN/govoplan-idm.git@v0.1.8",
"other-package==1.0",
)
),
encoding="utf-8",
)
self.assertEqual(
release_package_names(requirements),
("govoplan-idm", "govoplan-campaign"),
)
def test_artifact_contracts_compare_distribution_entry_point_and_discovery_versions(self) -> None:
artifacts = (
InstalledModuleArtifact("govoplan-access", "0.1.8", "access", "0.1.8"),
InstalledModuleArtifact("govoplan-idm", "0.1.8", "idm", "0.1.7"),
InstalledModuleArtifact("govoplan-shadow-idm", "0.1.8", "idm", "0.1.8"),
)
issues = artifact_contract_issues(
artifacts,
release_packages=("govoplan-access", "govoplan-idm", "govoplan-shadow-idm", "govoplan-campaign"),
available_manifest_versions={"access": "0.1.8", "idm": "0.1.6"},
)
self.assertEqual(len(issues), 5)
self.assertTrue(any("metadata version '0.1.8' does not match" in issue for issue in issues))
self.assertTrue(any("does not publish a govoplan.modules entry point" in issue for issue in issues))
self.assertTrue(any("multiple release packages" in issue for issue in issues))
def test_release_suite_excludes_only_named_source_coupled_tests(self) -> None:
retained_id = "tests.test_core_events.CoreEventTests.test_event_delivery"
nested = unittest.TestSuite(
(
unittest.TestSuite(_named_test(test_id) for test_id in SOURCE_COUPLED_CORE_TESTS),
unittest.TestSuite((_named_test(retained_id),)),
)
)
filtered, removed = filter_test_suite(nested, tuple(SOURCE_COUPLED_CORE_TESTS))
self.assertEqual(set(removed), set(SOURCE_COUPLED_CORE_TESTS))
self.assertEqual([test.id() for test in filtered], [retained_id])
self.assertEqual(len(SOURCE_COUPLED_CORE_TESTS), 5)
def test_release_entrypoint_uses_artifact_checks_and_filtered_core_suite(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
script = (meta_root / "tools" / "checks" / "check-release-integration.sh").read_text(encoding="utf-8")
artifact_check = script.index('"$META_ROOT/tools/checks/release_integration.py" artifacts')
core_tests = script.index('"$META_ROOT/tools/checks/release_integration.py" core-tests')
module_tests = script.index('run_step "Run cloned module backend tests"')
self.assertLess(artifact_check, core_tests)
self.assertLess(core_tests, module_tests)
self.assertNotIn("unittest discover", script)
self.assertNotIn('"$PYTHON" -m unittest \\\n tests.test_module_system', script)
def test_tagged_module_runner_executes_pytest_functions_from_module_root(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-release-module-tests-") as temp_dir:
module_root = Path(temp_dir)
tests_root = module_root / "tests"
tests_root.mkdir()
marker = module_root / "pytest-function-ran"
(tests_root / "test_probe.py").write_text(
"""from pathlib import Path
import os
def test_pytest_style_function():
expected_root = Path(os.environ[\"GOVOPLAN_TEST_MODULE_ROOT\"])
assert Path.cwd() == expected_root
Path(os.environ[\"GOVOPLAN_TEST_MARKER\"]).write_text(\"ran\", encoding=\"utf-8\")
""",
encoding="utf-8",
)
with patch.dict(
os.environ,
{
"GOVOPLAN_TEST_MODULE_ROOT": str(module_root),
"GOVOPLAN_TEST_MARKER": str(marker),
},
):
status = run_module_release_tests(module_root)
self.assertEqual(status, 0)
self.assertEqual(marker.read_text(encoding="utf-8"), "ran")
def test_tagged_module_runner_skips_missing_test_tree(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-release-module-tests-") as temp_dir:
self.assertEqual(run_module_release_tests(Path(temp_dir)), 0)
def test_module_matrix_uses_artifact_aware_core_suite(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
script = (meta_root / "tools" / "checks" / "check-module-matrix.sh").read_text(encoding="utf-8")
artifact_check = script.index('"$META_ROOT/tools/checks/release_integration.py" artifacts')
core_tests = script.index('"$META_ROOT/tools/checks/release_integration.py" core-tests')
self.assertLess(artifact_check, core_tests)
self.assertIn('--requirements "$META_ROOT/requirements-release.txt"', script)
self.assertIn('--core-root "$ROOT"', script)
self.assertNotIn('"$PYTHON" -m unittest tests.test_module_system', script)
def test_module_matrix_installs_search_before_postgres_validation(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
workflow = (meta_root / ".gitea" / "workflows" / "module-matrix.yml").read_text(encoding="utf-8")
install = workflow.index("pip install --no-deps ../govoplan-search")
validation = workflow.index("Validate Search against PostgreSQL")
self.assertLess(install, validation)
def test_release_workflow_installs_tagged_source_test_harness(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
workflow = (meta_root / ".gitea" / "workflows" / "release-integration.yml").read_text(encoding="utf-8")
requirements = (meta_root / "requirements-release-tests.txt").read_text(encoding="utf-8")
install = workflow.index("pip install -r requirements-release-tests.txt")
checks = workflow.index("bash tools/checks/check-release-integration.sh")
self.assertLess(install, checks)
self.assertIn("pytest>=9.0.3,<10", requirements)
self.assertNotIn("requirements-release-tests.txt", (meta_root / "requirements-release.txt").read_text(encoding="utf-8"))
if __name__ == "__main__":
unittest.main()
+32
View File
@@ -47,6 +47,38 @@ branch_labels: Union[str, Sequence[str], None] = None
self.assertEqual(migration.depends_on, ("core",)) self.assertEqual(migration.depends_on, ("core",))
self.assertEqual(migration.branch_labels, ()) self.assertEqual(migration.branch_labels, ())
def test_development_wrapper_uses_release_revision_metadata(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
versions = root / "versions"
development = root / "dev_versions"
versions.mkdir()
development.mkdir()
release = versions / "1234_example.py"
release.write_text(
'revision = "1234"\n'
'down_revision = "base"\n'
'depends_on = "core"\n'
"branch_labels = None\n",
encoding="utf-8",
)
wrapper = development / release.name
wrapper.write_text(
"revision = _migration.revision\n"
"down_revision = _migration.down_revision\n"
"depends_on = _migration.depends_on\n"
"branch_labels = _migration.branch_labels\n",
encoding="utf-8",
)
migration = audit.parse_migration_file("govoplan-core", wrapper)
self.assertIsNotNone(migration)
self.assertEqual("1234", migration.revision)
self.assertEqual(("base",), migration.down_revisions)
self.assertEqual(("core",), migration.depends_on)
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None: def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
audit = load_audit_module() audit = load_audit_module()
migrations = [ migrations = [
+82
View File
@@ -0,0 +1,82 @@
from __future__ import annotations
from pathlib import Path
import sys
import unittest
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.module_directory import ( # noqa: E402
module_directory_payloads,
safe_path_part,
)
class ReleaseModuleDirectoryTests(unittest.TestCase):
def test_signed_catalog_timestamp_makes_derived_files_reproducible(self) -> None:
catalog = {
"generated_at": "2026-07-22T12:00:00Z",
"sequence": 7,
"modules": [],
}
first = module_directory_payloads(
catalog_payload=catalog,
keyring_payload={},
channel="stable",
)
second = module_directory_payloads(
catalog_payload=catalog,
keyring_payload={},
channel="stable",
)
self.assertEqual(first, second)
self.assertEqual(
"2026-07-22T12:00:00Z",
first[-1][1]["generated_at"],
)
def test_dot_segments_and_noncanonical_ids_never_become_paths(self) -> None:
for value in (".", "..", "../escape", "/absolute", "module/child"):
with self.subTest(value=value), self.assertRaises(ValueError):
safe_path_part(value)
for module_id in ("..", "../escape", "UPPER", "bad.id"):
with self.subTest(module_id=module_id), mock.patch(
"govoplan_release.module_directory.module_rows",
return_value=[
{
"module_id": module_id,
"version": "1.2.3",
}
],
):
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={},
keyring_payload={},
channel="stable",
)
def test_noncanonical_version_and_channel_fail_before_paths(self) -> None:
with mock.patch(
"govoplan_release.module_directory.module_rows",
return_value=[{"module_id": "demo", "version": "../1.2.3"}],
):
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={}, keyring_payload={}, channel="stable"
)
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={}, keyring_payload={}, channel="../stable"
)
if __name__ == "__main__":
unittest.main()
+316
View File
@@ -0,0 +1,316 @@
from __future__ import annotations
import sys
import tempfile
import unittest
from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.model import ( # noqa: E402
CatalogSnapshot,
DashboardSummary,
ReleaseDashboard,
RepositorySnapshot,
RepositorySpec,
VersionSnapshot,
)
from govoplan_release.selective_planner import build_selective_release_plan # noqa: E402
class ReleasePlanGuidanceTests(unittest.TestCase):
def test_unprepared_target_gets_bounded_version_and_commit_steps(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.3"\n',
encoding="utf-8",
)
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.3"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertEqual("attention", plan.status)
self.assertTrue(plan.source_preflight_ready)
self.assertFalse(
any(
item.code == "repository_version_alignment"
for item in plan.gate_findings
)
)
steps = {step.id: step for step in plan.dry_run_steps}
self.assertEqual("planned", steps["govoplan-files:version"].status)
self.assertEqual("planned", steps["govoplan-files:commit"].status)
self.assertIn("pyproject.toml", steps["govoplan-files:version"].detail)
self.assertTrue(
any(
"version metadata will be updated" in warning
for warning in plan.units[0].warnings
)
)
self.assertIn("python-package", plan.units[0].capabilities)
def test_aligned_target_recommends_non_mutating_tag_preview(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.4"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertTrue(plan.source_preflight_ready)
self.assertEqual("preview_source_release", plan.recommended_action.id)
self.assertIn("Preview Tag + Publish", plan.recommended_action.remediation)
def test_mixed_plan_tags_modules_before_core_and_aligns_before_push(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
for repo_name in ("govoplan-core", "govoplan-files"):
repo_path = workspace / repo_name
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
f'[project]\nname = "{repo_name}"\nversion = "1.2.3"\n',
encoding="utf-8",
)
repositories = tuple(
RepositorySnapshot(
spec=RepositorySpec(
name=repo_name,
category="core" if repo_name == "govoplan-core" else "module",
subtype="platform" if repo_name == "govoplan-core" else "infrastructure",
remote=f"git@example.test:GovOPlaN/{repo_name}.git",
path=repo_name,
),
absolute_path=str(workspace / repo_name),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject="1.2.3"),
local_target_tag_exists=False,
)
for repo_name in ("govoplan-core", "govoplan-files")
)
base = dashboard(workspace=workspace, version="1.2.3")
mixed = ReleaseDashboard(
generated_at=base.generated_at,
meta_root=base.meta_root,
workspace_root=base.workspace_root,
target_version=None,
target_tag=None,
online=False,
include_migrations=False,
summary=DashboardSummary(
repository_count=2,
missing_count=0,
dirty_count=0,
ahead_count=0,
behind_count=0,
no_head_count=0,
error_count=0,
safe_directory_count=0,
local_target_tag_missing_count=0,
status="ready",
),
repositories=repositories,
catalog=base.catalog,
)
plan = build_selective_release_plan(
mixed,
selected_repos=("govoplan-core", "govoplan-files"),
repo_versions={
"govoplan-core": "1.2.3",
"govoplan-files": "1.2.3",
},
)
self.assertEqual(
["govoplan-files", "govoplan-core"],
[unit.repo for unit in plan.units],
)
step_ids = [step.id for step in plan.dry_run_steps]
self.assertLess(
step_ids.index("govoplan-files:tag"),
step_ids.index("govoplan-core:tag"),
)
self.assertLess(
step_ids.index("govoplan-core:tag"),
step_ids.index("release:alignment"),
)
self.assertLess(
step_ids.index("release:alignment"),
step_ids.index("govoplan-files:push"),
)
self.assertEqual("release:install-verify", step_ids[-1])
core = next(unit for unit in plan.units if unit.repo == "govoplan-core")
self.assertIn("core-release-bundle", core.capabilities)
def test_malformed_version_metadata_is_a_structured_blocker(self) -> None:
cases = (
("package.json", "{not-json\n", "JSONDecodeError"),
("pyproject.toml", "[project\n", "TOMLDecodeError"),
)
for relative_path, malformed, error_name in cases:
with self.subTest(relative_path=relative_path):
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
(repo_path / relative_path).write_text(malformed, encoding="utf-8")
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.4"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertEqual("blocked", plan.status)
finding = next(
item
for item in plan.gate_findings
if item.code == "repository_version_metadata_unreadable"
)
self.assertEqual("govoplan-files", finding.repo)
self.assertIn(error_name, finding.message)
self.assertIn("Repair the malformed TOML or JSON", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
def test_malformed_webui_metadata_is_a_structured_blocker(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
(repo_path / "webui").mkdir(parents=True)
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
(repo_path / "webui" / "package.json").write_text(
"{not-json\n", encoding="utf-8"
)
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.4"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertEqual("blocked", plan.status)
codes = {item.code for item in plan.gate_findings}
self.assertIn("repository_version_metadata_unreadable", codes)
finding = next(
item
for item in plan.gate_findings
if item.code == "release_webui_metadata_unreadable"
)
self.assertIn("JSONDecodeError", finding.message)
self.assertIn("Repair malformed JSON", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
def test_webui_renders_recommendation_and_remediation(self) -> None:
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
self.assertIn("plan.gate_findings", webui)
self.assertIn("plan.recommended_action", webui)
self.assertIn("recommended next", webui)
self.assertIn('plan.status === "blocked" ? "block"', webui)
self.assertIn('pill("recommended next", recommendationKind)', webui)
self.assertIn("<strong>Remediation:</strong>", webui)
def test_webui_projects_release_state_into_a_guided_workflow(self) -> None:
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
for phase in (
"Inspect",
"Targets",
"Validate",
"Source",
"Package",
"Publish",
"Verify",
):
self.assertIn(f'label: "{phase}"', webui)
self.assertIn("releaseWorkflowPhases", webui)
self.assertIn("workflowPrimaryAction", webui)
self.assertIn("data-run-step-id", webui)
self.assertIn("invalidateReleaseDraft", webui)
self.assertIn("Installation Verification", webui)
self.assertIn("receipt-bound local gate", webui)
self.assertIn('step.id === "release:install-verify"', webui)
self.assertIn('const inspectionNotices = (summary.missing_count || 0)', webui)
self.assertIn('(summary.repository_count || 0) === 0', webui)
def dashboard(*, workspace: Path, version: str) -> ReleaseDashboard:
repo = RepositorySnapshot(
spec=RepositorySpec(
name="govoplan-files",
category="module",
subtype="infrastructure",
remote="git@example.test:GovOPlaN/govoplan-files.git",
path="govoplan-files",
),
absolute_path=str(workspace / "govoplan-files"),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject=version),
local_target_tag_exists=False,
)
return ReleaseDashboard(
generated_at="2026-07-22T00:00:00Z",
meta_root=str(workspace / "govoplan"),
workspace_root=str(workspace),
target_version=None,
target_tag=None,
online=False,
include_migrations=False,
summary=DashboardSummary(
repository_count=1,
missing_count=0,
dirty_count=0,
ahead_count=0,
behind_count=0,
no_head_count=0,
error_count=0,
safe_directory_count=0,
local_target_tag_missing_count=0,
status="ready",
),
repositories=(repo,),
catalog=CatalogSnapshot(
channel="stable",
website_path="",
catalog_path="",
catalog_exists=False,
keyring_path="",
keyring_exists=False,
),
)
if __name__ == "__main__":
unittest.main()
+440
View File
@@ -0,0 +1,440 @@
from __future__ import annotations
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.repository_tag import tag_repositories # noqa: E402
from server.app import create_app # noqa: E402
class ReleaseRepositoryTagTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.workspace = self.root / "workspace"
self.workspace.mkdir()
self.repo, self.remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-core",
version="0.1.10",
)
self.manifest_repo, self.manifest_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-access",
version="0.1.10",
)
add_scoped_workflow_manifest(self.manifest_repo)
def tearDown(self) -> None:
self.temporary.cleanup()
def test_core_preview_is_non_mutating_and_publish_is_idempotent(self) -> None:
preview = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=False,
push=True,
)
self.assertEqual(preview["status"], "planned")
row = preview["repositories"][0]
self.assertEqual(row["status"], "planned")
self.assertIn("git tag -a v0.1.10", row["create_command"])
self.assertIn("git push --atomic origin", row["publish_command"])
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.remote, "refs/tags/v0.1.10"))
published = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
head = git_text(self.repo, "rev-parse", "HEAD")
self.assertEqual(published["status"], "published")
self.assertEqual(git_text(self.repo, "cat-file", "-t", "refs/tags/v0.1.10"), "tag")
self.assertEqual(git_text(self.repo, "rev-parse", "refs/tags/v0.1.10^{commit}"), head)
self.assertEqual(git_text(self.remote, "rev-parse", "refs/tags/v0.1.10^{commit}"), head)
repeated = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual(repeated["status"], "published")
self.assertIn("already published", repeated["repositories"][0]["detail"])
self.assertEqual(git_text(self.remote, "rev-parse", "refs/tags/v0.1.10^{commit}"), head)
def test_core_alignment_and_clean_worktree_are_release_gates(self) -> None:
mismatch = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.11"},
workspace_root=self.workspace,
)
self.assertEqual(mismatch["status"], "blocked")
self.assertIn("version alignment gate failed", mismatch["repositories"][0]["detail"])
(self.repo / "uncommitted.txt").write_text("not reviewed\n", encoding="utf-8")
dirty = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
)
self.assertEqual(dirty["status"], "blocked")
self.assertIn("worktree is not clean", dirty["repositories"][0]["detail"])
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
def test_user_workflow_scope_contract_blocks_source_tagging(self) -> None:
replace_with_unscoped_workflow_manifest(self.manifest_repo)
result = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
)
self.assertEqual(result["status"], "blocked")
self.assertIn("scope-conditioned alternative", result["repositories"][0]["detail"])
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
def test_empty_manifest_workspace_blocks_source_tagging(self) -> None:
workspace = self.root / "empty-manifest-workspace"
remote_root = self.root / "empty-manifest-remotes"
workspace.mkdir()
remote_root.mkdir()
core, _ = create_release_repo(
root=remote_root,
workspace=workspace,
name="govoplan-core",
version="0.1.10",
)
result = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=workspace,
)
self.assertEqual(result["status"], "blocked")
self.assertIn("No module manifests found", result["repositories"][0]["detail"])
self.assertFalse(ref_exists(core, "refs/tags/v0.1.10"))
def test_batch_preflight_blocks_every_mutation_when_a_later_repo_is_dirty(self) -> None:
access = self.manifest_repo
access_remote = self.manifest_remote
(access / "uncommitted.txt").write_text("not reviewed\n", encoding="utf-8")
result = tag_repositories(
repos=("govoplan-core", "govoplan-access"),
repo_versions={"govoplan-core": "0.1.10", "govoplan-access": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual(result["status"], "blocked")
self.assertIn("no selected repository was mutated", result["detail"])
self.assertEqual(result["repositories"][0]["status"], "skipped")
self.assertEqual(result["repositories"][1]["status"], "blocked")
for repository in (self.repo, self.remote, access, access_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_batch_preflight_blocks_selected_webui_version_missing_from_core_bundle(self) -> None:
campaign, campaign_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-campaign",
version="0.1.10",
)
campaign_webui = campaign / "webui"
campaign_webui.mkdir()
(campaign_webui / "package.json").write_text(
'{"name":"@govoplan/campaign-webui","version":"0.1.10"}\n',
encoding="utf-8",
)
git(campaign, "add", "webui/package.json")
git(campaign, "commit", "-m", "Add Campaign WebUI package")
git(campaign, "tag", "-a", "v0.1.9", "-m", "Prior Campaign release")
git(campaign, "push", "origin", "main", "refs/tags/v0.1.9")
campaign_head = git_text(campaign, "rev-parse", "HEAD")
core_webui = self.repo / "webui"
core_webui.mkdir()
old_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v0.1.9"
(core_webui / "package.release.json").write_text(
json.dumps(
{
"name": "@govoplan/core-webui",
"version": "0.1.10",
"dependencies": {"@govoplan/campaign-webui": old_ref},
}
)
+ "\n",
encoding="utf-8",
)
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {
"name": "@govoplan/core-webui",
"version": "0.1.10",
"dependencies": {"@govoplan/campaign-webui": old_ref},
},
"node_modules/@govoplan/campaign-webui": {
"version": "0.1.9",
"resolved": f"git+ssh://git@example.test/acme/govoplan-campaign.git#{campaign_head}",
},
}
}
)
+ "\n",
encoding="utf-8",
)
git(self.repo, "add", "webui/package.release.json", "webui/package-lock.release.json")
git(self.repo, "commit", "-m", "Add release WebUI composition")
git(self.repo, "push", "origin", "main")
result = tag_repositories(
repos=("govoplan-core", "govoplan-campaign"),
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual("blocked", result["status"])
self.assertIn("no selected repository was mutated", result["detail"])
self.assertEqual("skipped", result["repositories"][0]["status"])
self.assertEqual("blocked", result["repositories"][1]["status"])
self.assertIn("release WebUI composition gate failed", result["repositories"][1]["detail"])
for repository in (self.repo, self.remote, campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_api_keeps_legacy_source_release_preview_only(self) -> None:
with TestClient(
create_app(workspace_root=self.workspace, token="token")
) as client:
headers = {"X-Release-Console-Token": "token"}
rejected = client.post(
"/api/repositories/tag",
headers=headers,
json={
"repos": ["govoplan-core"],
"repo_versions": {"govoplan-core": "0.1.10"},
"apply": True,
"push": True,
"confirm": "TAG",
},
)
preview = client.post(
"/api/repositories/tag",
headers=headers,
json={
"repos": ["govoplan-core"],
"repo_versions": {"govoplan-core": "0.1.10"},
"apply": False,
"push": True,
},
)
legacy_push = client.post(
"/api/repositories/push",
headers=headers,
json={
"repos": ["govoplan-core"],
"apply": True,
"confirm": "PUSH",
},
)
legacy_sync = client.post(
"/api/repositories/sync",
headers=headers,
json={
"repos": ["govoplan-core"],
"apply": True,
"confirm": "SYNC",
},
)
ui = client.get("/")
self.assertEqual(rejected.status_code, 409)
self.assertIn("durable release run", rejected.json()["detail"])
self.assertEqual(preview.status_code, 200)
self.assertEqual(409, legacy_push.status_code)
self.assertEqual(409, legacy_sync.status_code)
self.assertIn("durable", legacy_push.json()["detail"])
self.assertIn("durable", legacy_sync.json()["detail"])
self.assertEqual(preview.json()["repositories"][0]["repo"], "govoplan-core")
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
self.assertIn('id="publishReleaseTags"', ui.text)
self.assertIn(
'id="publishReleaseTags" data-release-disabled="true" disabled',
ui.text,
)
self.assertIn('postJson("/api/repositories/tag"', ui.text)
self.assertIn(
'id="syncRepos" disabled data-release-disabled="true"', ui.text
)
self.assertIn(
'id="pushRepos" disabled data-release-disabled="true"', ui.text
)
self.assertIn("Signed Website Catalog", ui.text)
self.assertIn("Apply + Website Tag", ui.text)
def git(cwd: Path, *args: str) -> None:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
def create_release_repo(*, root: Path, workspace: Path, name: str, version: str) -> tuple[Path, Path]:
remote = root / f"{name}.git"
repo = workspace / name
git(root, "init", "--bare", str(remote))
git(workspace, "init", "-b", "main", str(repo))
git(repo, "config", "user.name", "GovOPlaN Release Test")
git(repo, "config", "user.email", "release-test@example.invalid")
(repo / "pyproject.toml").write_text(
f'[project]\nname = "{name}"\nversion = "{version}"\n',
encoding="utf-8",
)
git(repo, "add", "pyproject.toml")
git(repo, "commit", "-m", f"{name} {version}")
git(repo, "remote", "add", "origin", str(remote))
git(repo, "push", "-u", "origin", "main")
return repo, remote
def add_scoped_workflow_manifest(repo: Path) -> None:
package = repo / "src" / "govoplan_access"
backend = package / "backend"
backend.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8")
(backend / "__init__.py").write_text("", encoding="utf-8")
(backend / "manifest.py").write_text(
"""from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition
def get_manifest():
return ModuleManifest(
id="access",
name="Access",
version="0.1.10",
permissions=(
PermissionDefinition(
scope="access:item:read",
module_id="access",
resource="item",
action="read",
label="Read items",
description="Read example items.",
category="Example",
level="tenant",
),
),
documentation=(
DocumentationTopic(
id="access.workflow.scoped",
title="Scoped task",
summary="This is a valid release fixture.",
documentation_types=("user",),
conditions=(DocumentationCondition(required_scopes=("access:item:read",)),),
metadata={"kind": "workflow"},
),
),
)
""",
encoding="utf-8",
)
with (repo / "pyproject.toml").open("a", encoding="utf-8") as handle:
handle.write(
'\n[project.entry-points."govoplan.modules"]\n'
'access = "govoplan_access.backend.manifest:get_manifest"\n'
)
git(repo, "add", "pyproject.toml", "src")
git(repo, "commit", "-m", "Add scoped workflow manifest")
git(repo, "push", "origin", "main")
def replace_with_unscoped_workflow_manifest(repo: Path) -> None:
manifest = repo / "src" / "govoplan_access" / "backend" / "manifest.py"
manifest.write_text(
"""from govoplan_core.core.modules import DocumentationTopic, ModuleManifest
def get_manifest():
return ModuleManifest(
id="access",
name="Access",
version="0.1.10",
documentation=(
DocumentationTopic(
id="access.workflow.unscoped",
title="Unscoped task",
summary="This must block release.",
documentation_types=("user",),
metadata={"kind": "workflow"},
),
),
)
""",
encoding="utf-8",
)
git(repo, "add", str(manifest.relative_to(repo)))
git(repo, "commit", "-m", "Replace scoped workflow with unscoped workflow")
git(repo, "push", "origin", "main")
def git_text(cwd: Path, *args: str) -> str:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
return result.stdout.strip()
def ref_exists(cwd: Path, ref: str) -> bool:
return subprocess.run(
("git", "rev-parse", "--verify", ref),
cwd=cwd,
check=False,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
).returncode == 0
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large Load Diff
+126
View File
@@ -0,0 +1,126 @@
from __future__ import annotations
import base64
from datetime import UTC, datetime
import hashlib
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.sbom import build_release_sbom, render_sbom, sbom_sha256, validate_sbom # noqa: E402
class ReleaseSbomTests(unittest.TestCase):
@staticmethod
def _build_at(timestamp: datetime) -> dict:
return build_release_sbom(
product_version="1.2.3",
pip_inspect={
"version": "1",
"installed": [{"metadata": {"name": "FastAPI", "version": "0.100.0"}}],
},
npm_lock={"packages": {"": {"name": "govoplan", "version": "1.2.3"}}},
timestamp=timestamp,
)
def test_builds_valid_deduplicated_python_and_npm_components(self) -> None:
integrity = base64.b64encode(b"example digest").decode("ascii")
payload = build_release_sbom(
product_version="1.2.3",
pip_inspect={
"version": "1",
"installed": [
{"metadata": {"name": "FastAPI", "version": "0.100.0"}},
{"metadata": {"name": "FastAPI", "version": "0.100.0"}},
],
},
npm_lock={
"packages": {
"": {"name": "govoplan", "version": "1.2.3"},
"node_modules/@scope/example": {
"name": "@scope/example",
"version": "4.5.6",
"integrity": f"sha512-{integrity}",
},
}
},
)
validate_sbom(payload)
self.assertEqual(2, len(payload["components"]))
npm = next(item for item in payload["components"] if item["name"] == "@scope/example")
self.assertEqual("SHA-512", npm["hashes"][0]["alg"])
self.assertIn("pkg:npm/%40scope/example@4.5.6", npm["purl"])
self.assertEqual(
hashlib.sha256(render_sbom(payload).encode("utf-8")).hexdigest(),
sbom_sha256(payload),
)
def test_rejects_empty_component_inventory(self) -> None:
payload = build_release_sbom(
product_version="1.2.3",
pip_inspect={"version": "1", "installed": []},
npm_lock={"packages": {"": {"version": "1.2.3"}}},
)
with self.assertRaisesRegex(ValueError, "no dependency components"):
validate_sbom(payload)
def test_fixed_timestamp_produces_identical_bytes_and_digest(self) -> None:
timestamp = datetime(2026, 7, 21, 8, 30, tzinfo=UTC)
first = self._build_at(timestamp)
second = self._build_at(timestamp)
self.assertEqual(render_sbom(first), render_sbom(second))
self.assertEqual(sbom_sha256(first), sbom_sha256(second))
def test_timestamp_changes_serial_and_digest(self) -> None:
first = self._build_at(datetime(2026, 7, 21, 8, 30, tzinfo=UTC))
second = self._build_at(datetime(2026, 7, 21, 8, 31, tzinfo=UTC))
self.assertNotEqual(first["serialNumber"], second["serialNumber"])
self.assertNotEqual(sbom_sha256(first), sbom_sha256(second))
def test_local_and_credential_bearing_distribution_urls_are_omitted(self) -> None:
payload = build_release_sbom(
product_version="1.2.3",
pip_inspect={
"version": "1",
"installed": [
{
"metadata": {"name": "LocalPackage", "version": "1.0.0"},
"direct_url": {"url": "file:///private/build/govoplan-core"},
},
{
"metadata": {"name": "PrivatePackage", "version": "1.0.0"},
"direct_url": {"url": "https://user:secret@example.test/private.whl"},
},
{
"metadata": {"name": "PublicPackage", "version": "1.0.0"},
"direct_url": {"url": "https://example.test/public.whl"},
},
],
},
npm_lock={"packages": {"": {"name": "govoplan", "version": "1.2.3"}}},
timestamp=datetime(2026, 7, 21, 8, 30, tzinfo=UTC),
)
by_name = {component["name"]: component for component in payload["components"]}
self.assertNotIn("externalReferences", by_name["LocalPackage"])
self.assertNotIn("externalReferences", by_name["PrivatePackage"])
self.assertEqual(
[{"type": "distribution", "url": "https://example.test/public.whl"}],
by_name["PublicPackage"]["externalReferences"],
)
if __name__ == "__main__":
unittest.main()
+373
View File
@@ -0,0 +1,373 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.publisher import publish_catalog_candidate # noqa: E402
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
build_selective_catalog_candidate,
enforce_selected_source_provenance,
parse_signing_key,
public_key_base64,
signature,
)
from govoplan_release.source_provenance import ( # noqa: E402
catalog_source_selection,
read_only_https_remote,
source_tag_provenance_issues,
tagged_source_version_issues,
)
from server.app import create_app # noqa: E402
class ReleaseSourceProvenanceTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.workspace = self.root / "workspace"
self.workspace.mkdir()
def tearDown(self) -> None:
self.temporary.cleanup()
def test_selected_source_requires_local_and_remote_annotated_tag_at_clean_head(self) -> None:
repo, remote = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
missing = source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
)
self.assertIn("missing from the local source repository", messages(missing))
self.assertIn("missing from remote 'origin'", messages(missing))
git(repo, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(repo, "push", "origin", "refs/tags/v1.2.3")
self.assertEqual(
(),
source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
),
)
git(remote, "config", "user.name", "Different Release Actor")
git(remote, "config", "user.email", "different@example.invalid")
git(remote, "tag", "-f", "-a", "v1.2.3", git_text(repo, "rev-parse", "HEAD"), "-m", "Different annotation")
inconsistent = source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
)
self.assertIn("local and remote annotated tag objects differ", messages(inconsistent))
(repo / "dirty.txt").write_text("not reviewed\n", encoding="utf-8")
dirty = source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
)
self.assertIn("selected source worktree is not clean", messages(dirty))
self.assertEqual(git_text(remote, "rev-parse", "refs/tags/v1.2.3^{commit}"), git_text(repo, "rev-parse", "HEAD"))
def test_selected_source_gate_is_fail_closed_before_signing(self) -> None:
make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
with self.assertRaisesRegex(ValueError, "Source tag provenance gate failed") as error:
enforce_selected_source_provenance(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
)
self.assertIn("v1.2.3", str(error.exception))
self.assertIn("missing", str(error.exception))
def test_historical_tag_uses_installable_and_manifest_version_metadata(self) -> None:
repo, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
package = repo / "src" / "govoplan_core"
package.mkdir(parents=True)
(package / "__init__.py").write_text('__version__ = "1.2.2"\n', encoding="utf-8")
git(repo, "add", "src/govoplan_core/__init__.py")
git(repo, "commit", "-m", "Keep legacy runtime version string")
git(repo, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
issues = tagged_source_version_issues(
path=repo,
repo="govoplan-core",
tag="v1.2.3",
expected_version="1.2.3",
)
self.assertEqual((), issues)
def test_signed_candidate_records_selected_commit_and_tag_object(self) -> None:
core, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
git(core, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(core, "push", "origin", "refs/tags/v1.2.3")
private_key = Ed25519PrivateKey.generate()
keyring = {
"keys": [
{
"key_id": "test-key",
"status": "active",
"public_key": public_key_base64(private_key),
}
]
}
base_keyring = self.root / "base-keyring.json"
base_keyring.write_text(json.dumps(keyring), encoding="utf-8")
base_catalog = self.root / "base.json"
base_payload = {
"channel": "stable",
"sequence": 1,
"core_release": {
"version": "1.2.2",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.2",
},
"modules": [],
"release": {
"version": "1.2.2",
"tag": "v1.2.2",
"keyring_sha256": canonical_hash(keyring),
},
}
base_payload["signatures"] = [
signature(base_payload, key_id="test-key", private_key=private_key)
]
base_catalog.write_text(json.dumps(base_payload), encoding="utf-8")
key_path = self.root / "release.pem"
key_path.write_bytes(
private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
key_path.chmod(0o600)
output = self.root / "candidate"
with patch(
"govoplan_release.selective_catalog.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = build_selective_catalog_candidate(
repo_versions={"govoplan-core": "1.2.3"},
workspace_root=self.workspace,
output_dir=output,
base_catalog=base_catalog,
base_keyring=base_keyring,
signing_keys=(f"test-key={key_path}",),
check_public=False,
)
payload = json.loads(Path(result.catalog_path).read_text(encoding="utf-8"))
selected = payload["release"]["selected_units"][0]
self.assertEqual(git_text(core, "rev-parse", "refs/tags/v1.2.3^{commit}"), selected["commit_sha"])
self.assertEqual(git_text(core, "rev-parse", "refs/tags/v1.2.3"), selected["tag_object_sha"])
self.assertEqual("test-key", payload["signatures"][0]["key_id"])
def test_catalog_signing_key_must_be_operator_private(self) -> None:
private_key = Ed25519PrivateKey.generate()
key_path = self.root / "release.pem"
key_path.write_bytes(
private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
key_path.chmod(0o644)
with self.assertRaisesRegex(ValueError, "inaccessible to other users"):
parse_signing_key(f"test-key={key_path}")
key_path.chmod(0o600)
key_id, _parsed = parse_signing_key(f"test-key={key_path}")
self.assertEqual("test-key", key_id)
def test_catalog_publication_checks_every_preserved_source_ref(self) -> None:
core, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
git(core, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(core, "push", "origin", "refs/tags/v1.2.3")
make_repo(self.workspace, self.root, "govoplan-access", "1.1.0")
candidate = self.root / "candidate"
(candidate / "channels").mkdir(parents=True)
payload = catalog_payload(
commit_sha="f" * 40,
tag_object_sha=git_text(core, "rev-parse", "refs/tags/v1.2.3"),
)
(candidate / "channels" / "stable.json").write_text(json.dumps(payload), encoding="utf-8")
keyring = {"keys": [{"key_id": "release-key", "status": "active", "public_key": "trusted"}]}
(candidate / "keyring.json").write_text(json.dumps(keyring), encoding="utf-8")
web_root = self.root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(keyring), encoding="utf-8")
with patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = publish_catalog_candidate(
candidate_dir=candidate,
workspace_root=self.workspace,
web_root=web_root,
)
detail = " ".join(result.notes)
self.assertEqual("blocked", result.status)
self.assertIn("govoplan-access v1.1.0", detail)
self.assertIn("missing from the local source repository", detail)
self.assertIn("missing from remote 'origin'", detail)
self.assertIn("no longer matches signed commit_sha", detail)
self.assertNotIn("govoplan-core v1.2.3: annotated tag is missing", detail)
def test_catalog_selection_includes_core_and_all_modules(self) -> None:
selection = catalog_source_selection(catalog_payload())
self.assertEqual(
{"govoplan-core": "1.2.3", "govoplan-access": "1.1.0"},
selection.all_versions,
)
self.assertEqual({"govoplan-core": "1.2.3"}, selection.selected_versions)
self.assertEqual((), selection.issues)
def test_console_renders_publication_provenance_notes(self) -> None:
ui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
self.assertIn("Array.isArray(result.notes)", ui)
self.assertIn("Validation and provenance details", ui)
def test_console_disables_unclaimed_legacy_candidate_signing(self) -> None:
with TestClient(
create_app(workspace_root=self.workspace, token="token")
) as client:
response = client.post(
"/api/catalog-candidates",
headers={"X-Release-Console-Token": "token"},
json={
"repo_versions": {"govoplan-core": "1.2.3"},
"signing_keys": ["test=/unused/key.pem"],
},
)
self.assertEqual(409, response.status_code)
self.assertIn("durable release run", response.json()["detail"])
def test_read_only_ref_checks_can_reuse_the_same_gitea_https_endpoint(self) -> None:
self.assertEqual(
"https://git.example.test/acme/govoplan-core.git",
read_only_https_remote("git@git.example.test:acme/govoplan-core.git"),
)
self.assertEqual(
"https://git.example.test/acme/govoplan-core.git",
read_only_https_remote("ssh://git@git.example.test/acme/govoplan-core.git"),
)
self.assertIsNone(read_only_https_remote("ssh://git@git.example.test:2222/acme/govoplan-core.git"))
self.assertIsNone(read_only_https_remote("/srv/git/govoplan-core.git"))
def catalog_payload(
*,
commit_sha: str = "0" * 40,
tag_object_sha: str = "1" * 40,
) -> dict[str, object]:
return {
"channel": "stable",
"core_release": {
"version": "1.2.3",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.3",
},
"modules": [
{
"module_id": "access",
"version": "1.1.0",
"python_package": "govoplan-access",
"python_ref": "govoplan-access @ git+ssh://git@example.test/acme/govoplan-access.git@v1.1.0",
}
],
"release": {
"version": "1.2.3",
"tag": "v1.2.3",
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": commit_sha,
"tag_object_sha": tag_object_sha,
},
],
},
}
def make_repo(workspace: Path, root: Path, name: str, version: str) -> tuple[Path, Path]:
remote = root / f"{name}.git"
repo = workspace / name
git(root, "init", "--bare", str(remote))
git(workspace, "init", "-b", "main", str(repo))
git(repo, "config", "user.name", "GovOPlaN Release Test")
git(repo, "config", "user.email", "release-test@example.invalid")
(repo / "pyproject.toml").write_text(
f'[project]\nname = "{name}"\nversion = "{version}"\n',
encoding="utf-8",
)
git(repo, "add", "pyproject.toml")
git(repo, "commit", "-m", f"{name} {version}")
git(repo, "remote", "add", "origin", str(remote))
git(repo, "push", "-u", "origin", "main")
return repo, remote
def messages(issues: object) -> str:
return " ".join(issue.message for issue in issues) # type: ignore[attr-defined]
def git(cwd: Path, *args: str) -> None:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
def git_text(cwd: Path, *args: str) -> str:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
return result.stdout.strip()
if __name__ == "__main__":
unittest.main()
+105
View File
@@ -0,0 +1,105 @@
from __future__ import annotations
import json
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.version_metadata import ( # noqa: E402
apply_version_metadata_mutations,
version_metadata_mutations,
)
class ReleaseVersionMetadataTests(unittest.TestCase):
def test_updates_recognized_metadata_without_changing_interface_versions(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
package = root / "src" / "govoplan_example"
backend = package / "backend"
webui = root / "webui"
backend.mkdir(parents=True)
webui.mkdir()
(root / "pyproject.toml").write_text(
'[project]\nname = "govoplan-example"\nversion = "1.2.3"\n',
encoding="utf-8",
)
(root / "package.json").write_text(
'{"name":"root","version":"1.2.3"}\n',
encoding="utf-8",
)
(webui / "package.json").write_text(
'{"name":"@govoplan/example-webui","version":"1.2.3"}\n',
encoding="utf-8",
)
(webui / "package-lock.json").write_text(
json.dumps(
{
"name": "@govoplan/example-webui",
"version": "1.2.3",
"packages": {
"": {
"name": "@govoplan/example-webui",
"version": "1.2.3",
}
},
}
)
+ "\n",
encoding="utf-8",
)
(backend / "manifest.py").write_text(
"def get_manifest():\n"
" return ModuleManifest(\n"
' id="example",\n'
' version="1.2.3",\n'
" provides_interfaces=(\n"
' ModuleInterfaceProvider(name="example.api", version="4.0"),\n'
" ),\n"
" )\n",
encoding="utf-8",
)
(package / "__init__.py").write_text(
'__version__ = "1.2.3"\n',
encoding="utf-8",
)
preview = version_metadata_mutations(
root,
target_version="1.2.4",
)
changed = apply_version_metadata_mutations(
root,
target_version="1.2.4",
)
self.assertEqual(
{mutation.path for mutation in preview},
set(changed),
)
self.assertEqual(
"1.2.4",
json.loads((webui / "package-lock.json").read_text())[
"packages"
][""]["version"],
)
manifest = (backend / "manifest.py").read_text(encoding="utf-8")
self.assertIn('version="1.2.4"', manifest)
self.assertIn('version="4.0"', manifest)
self.assertEqual(
'__version__ = "1.2.4"\n',
(package / "__init__.py").read_text(encoding="utf-8"),
)
if __name__ == "__main__":
unittest.main()
+393
View File
@@ -0,0 +1,393 @@
from __future__ import annotations
import importlib.util
import json
from pathlib import Path
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "repo" / "bootstrap-repositories.py"
def load_bootstrap_module():
spec = importlib.util.spec_from_file_location("bootstrap_repositories", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class RepositoryBootstrapTests(unittest.TestCase):
def test_public_https_transport_rewrites_registered_gitea_remotes(self) -> None:
bootstrap = load_bootstrap_module()
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
bootstrap.clone_remote(
"git@git.add-ideas.de:GovOPlaN/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
bootstrap.clone_remote(
"ssh://git@git.add-ideas.de/GovOPlaN/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
bootstrap.clone_remote(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
def test_registered_transport_preserves_the_manifest_remote(self) -> None:
bootstrap = load_bootstrap_module()
remote = "git@example.test:private/repository.git"
self.assertEqual(
remote,
bootstrap.clone_remote(
remote,
transport=bootstrap.REGISTERED_TRANSPORT,
),
)
def test_public_https_transport_fails_closed_for_other_hosts(self) -> None:
bootstrap = load_bootstrap_module()
unsafe_remotes = (
"git@example.test:GovOPlaN/govoplan-core.git",
"https://token@git.add-ideas.de/GovOPlaN/govoplan-core.git",
"https://git.add-ideas.de:443/GovOPlaN/govoplan-core.git",
"https://git.add-ideas.de/add-ideas/../govoplan-core.git",
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git?ref=main",
"ssh://root@git.add-ideas.de/GovOPlaN/govoplan-core.git",
)
for remote in unsafe_remotes:
with self.subTest(remote=remote), self.assertRaises(ValueError):
bootstrap.clone_remote(
remote,
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
)
def test_main_clones_missing_repositories_over_public_https(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(0, status)
runner.assert_called_once_with(
[
"git",
"-c",
"credential.helper=",
"clone",
"--",
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
str(parent / "govoplan-core"),
],
check=True,
env=runner.call_args.kwargs["env"],
)
environment = runner.call_args.kwargs["env"]
self.assertEqual("/bin/false", environment["GIT_ASKPASS"])
self.assertEqual("0", environment["GIT_TERMINAL_PROMPT"])
def test_main_validates_every_remote_before_cloning(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
},
{
"name": "unsafe",
"path": "unsafe",
"remote": "git@example.test:private/unsafe.git",
},
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
self.assertRaises(ValueError),
):
bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
runner.assert_not_called()
def test_main_preserves_an_explicit_private_repository_transport(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "website",
"path": "website",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/addideas-govoplan-website.git"
),
"bootstrap_transport": (
bootstrap.REGISTERED_TRANSPORT
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(0, status)
command = runner.call_args.args[0]
self.assertEqual(
"git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git",
command[-2],
)
def test_main_limits_bootstrap_to_selected_repositories(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": name,
"path": name,
"remote": (
"git@git.add-ideas.de:GovOPlaN/"
f"{name}.git"
),
}
for name in ("govoplan-core", "govoplan-poll")
],
}
),
encoding="utf-8",
)
for repository_filter in (
["--repo", "govoplan-core"],
["--exclude-repo", "govoplan-poll"],
):
with (
self.subTest(repository_filter=repository_filter),
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
*repository_filter,
]
)
self.assertEqual(0, status)
runner.assert_called_once()
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
runner.call_args.args[0][-2],
)
def test_main_rejects_unknown_or_conflicting_repository_filters(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
with self.assertRaisesRegex(ValueError, "unknown registered"):
bootstrap.main(["--repo", "govoplan-missing"])
with self.assertRaisesRegex(ValueError, "selected and excluded"):
bootstrap.main(
[
"--repo",
"govoplan-core",
"--exclude-repo",
"govoplan-core",
]
)
runner.assert_not_called()
def test_check_reports_missing_without_cloning(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--check",
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(1, status)
runner.assert_not_called()
def test_anonymous_ci_bootstrap_excludes_registered_transport_repositories(
self,
) -> None:
manifest = json.loads(
(META_ROOT / "repositories.json").read_text(encoding="utf-8")
)
registered_only = {
entry["name"]
for entry in manifest["repositories"]
if entry.get("bootstrap_transport") == "registered"
}
self.assertTrue(registered_only)
for workflow in sorted(
(META_ROOT / ".gitea" / "workflows").glob("*.yml")
):
contents = workflow.read_text(encoding="utf-8")
if (
"bootstrap-repositories.py" not in contents
or "--transport public-https" not in contents
):
continue
with self.subTest(workflow=workflow.name):
for repository in registered_only:
self.assertIn(
f"--exclude-repo {repository}",
contents,
)
if __name__ == "__main__":
unittest.main()
+248
View File
@@ -0,0 +1,248 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RESOLVER = (
META_ROOT / "tools" / "checks" / "security-audit" / "resolve_workspace_mount.py"
)
def load_resolver_module():
spec = importlib.util.spec_from_file_location(
"security_audit_mount_resolver",
RESOLVER,
)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {RESOLVER}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class SecurityAuditMountResolverTests(unittest.TestCase):
def setUp(self) -> None:
self.resolver = load_resolver_module()
self.root = "/workspace/GovOPlaN/govoplan/govoplan"
self.workspace = "/workspace/GovOPlaN/govoplan"
def test_resolves_only_the_named_workspace_volume_for_both_scopes(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
{
"Type": "volume",
"Name": "actions-workspace",
"Source": "/var/lib/docker/volumes/actions-workspace/_data",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "volume",
"Name": "actions-environment",
"Source": "/var/lib/docker/volumes/actions-environment/_data",
"Destination": "/var/run/act",
"RW": True,
},
]
for scope in ("current", "govoplan"):
with self.subTest(scope=scope):
self.assertEqual(
(f"type=volume,source=actions-workspace,target={self.workspace}"),
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope=scope,
reports_dir="audit-reports",
),
)
def test_resolves_a_workspace_bind_without_other_mounts(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/srv/gitea/actions/task-123",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
]
self.assertEqual(
(f"type=bind,source=/srv/gitea/actions/task-123,target={self.workspace}"),
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
),
)
def test_nested_repository_mount_is_valid_only_for_current_scope(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "all-repositories",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "volume",
"Name": "current-repository",
"Destination": self.root,
"RW": True,
},
]
self.assertIn(
"source=current-repository",
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="current",
reports_dir="audit-reports",
),
)
with self.assertRaisesRegex(
self.resolver.MountResolutionError,
"nested job-container mounts",
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
def test_rejects_unsafe_or_unusable_mount_layouts(self) -> None:
cases = {
"missing": [],
"path-boundary": [
{
"Type": "volume",
"Name": "wrong-workspace",
"Destination": "/workspace/GovOPlaN/govoplan-other",
"RW": True,
}
],
"read-only": [
{
"Type": "volume",
"Name": "actions-workspace",
"Destination": self.workspace,
"RW": False,
}
],
"ambiguous": [
{
"Type": "volume",
"Name": name,
"Destination": self.workspace,
"RW": True,
}
for name in ("workspace-one", "workspace-two")
],
"scope-too-narrow": [
{
"Type": "volume",
"Name": "repository-only",
"Destination": self.root,
"RW": True,
}
],
}
for name, mounts in cases.items():
with (
self.subTest(name=name),
self.assertRaises(self.resolver.MountResolutionError),
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
def test_rejects_reports_outside_the_selected_workspace_mount(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "actions-workspace",
"Destination": self.workspace,
"RW": True,
}
]
with self.assertRaisesRegex(
self.resolver.MountResolutionError,
"reports path .* outside",
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="current",
reports_dir="/tmp/audit-reports",
)
def test_rejects_broad_or_sensitive_workspace_bind_sources(self) -> None:
for source in (
"/",
"/home",
"/var/run/docker.sock",
"/srv/../etc/shadow",
):
with (
self.subTest(source=source),
self.assertRaisesRegex(
self.resolver.MountResolutionError,
"too broad or sensitive",
),
):
self.resolver.resolve_workspace_mount(
[
{
"Type": "bind",
"Source": source,
"Destination": self.workspace,
"RW": True,
}
],
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
with self.assertRaises(self.resolver.MountResolutionError):
self.resolver.resolve_workspace_mount(
[
{
"Type": "bind",
"Source": "//var/lib/workspace",
"Destination": self.workspace,
"RW": True,
}
],
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
if __name__ == "__main__":
unittest.main()
+666
View File
@@ -0,0 +1,666 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import textwrap
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
AUDIT_SCRIPT = META_ROOT / "tools" / "checks" / "check-security-audit.sh"
CONTAINER_RUNNER = META_ROOT / "tools" / "checks" / "security-audit" / "run.sh"
class SecurityAuditWrapperTests(unittest.TestCase):
def setUp(self) -> None:
self._temporary_directory = tempfile.TemporaryDirectory()
self.temporary_root = Path(self._temporary_directory.name)
self.checkout = self.temporary_root / "checkout"
checks = self.checkout / "tools" / "checks"
checks.mkdir(parents=True)
shutil.copy2(AUDIT_SCRIPT, checks / AUDIT_SCRIPT.name)
(self.checkout / "README.md").write_text("audit fixture\n", encoding="utf-8")
subprocess.run(["git", "init", "-q", str(self.checkout)], check=True)
subprocess.run(
[
"git",
"-C",
str(self.checkout),
"config",
"user.email",
"audit@example.invalid",
],
check=True,
)
subprocess.run(
["git", "-C", str(self.checkout), "config", "user.name", "Audit Test"],
check=True,
)
subprocess.run(["git", "-C", str(self.checkout), "add", "."], check=True)
subprocess.run(
["git", "-C", str(self.checkout), "commit", "-qm", "audit fixture"],
check=True,
)
self.stub_bin = self.temporary_root / "bin"
self.stub_bin.mkdir()
self._write_stub(
"semgrep",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'semgrep stub 1.0'
exit 0
fi
if [[ " $* " != *" --error "* ]]; then
echo 'missing --error' >&2
exit 3
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output' ]]; then
output="$2"
shift 2
else
shift
fi
done
case "${STUB_SEMGREP_REPORT:-valid}" in
valid)
printf '%s\n' '{"version":"2.1.0","runs":[]}' > "$output"
;;
malformed)
printf '%s\n' '{' > "$output"
;;
invalid-sarif)
printf '%s\n' '{}' > "$output"
;;
missing)
;;
esac
if [[ -n "${STUB_MUTATE_PATH:-}" ]]; then
printf '%s\n' 'changed during audit' >> "$STUB_MUTATE_PATH"
fi
exit "${STUB_SEMGREP_EXIT:-0}"
""",
)
self._write_stub(
"gitleaks",
r"""
if [[ "${1:-}" == '--version' || "${1:-}" == 'version' ]]; then
echo 'gitleaks stub 1.0'
exit 0
fi
if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--report-path' ]]; then
output="$2"
shift 2
else
shift
fi
done
mkdir -p "$(dirname "$output")"
printf '%s\n' '[]' > "$output"
exit "${STUB_GITLEAKS_EXIT:-0}"
""",
)
for tool in ("bandit", "ruff"):
self._write_stub(
tool,
f"""
if [[ " $* " == *" --version "* ]]; then
echo '{tool} stub 1.0'
fi
exit 0
""",
)
def tearDown(self) -> None:
self._temporary_directory.cleanup()
def _write_stub(self, name: str, body: str) -> None:
path = self.stub_bin / name
path.write_text(
"#!/usr/bin/env bash\nset -euo pipefail\n" + textwrap.dedent(body).lstrip(),
encoding="utf-8",
)
path.chmod(0o755)
def _install_full_mode_stubs(self) -> None:
self._write_stub(
"trivy",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'trivy stub 1.0'
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output' ]]; then
output="$2"
shift 2
else
shift
fi
done
printf '%s\n' '{"version":"2.1.0","runs":[]}' > "$output"
""",
)
self._write_stub(
"osv-scanner",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'osv-scanner stub 1.0'
exit 0
fi
if [[ " $* " == *" scan --help "* ]]; then
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output-file' ]]; then
output="$2"
shift 2
else
shift
fi
done
printf '{"results": %s}\n' "${STUB_OSV_RESULTS:-null}" > "$output"
""",
)
self._write_stub(
"jscpd",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'jscpd stub 1.0'
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output' ]]; then
output="$2"
shift 2
else
shift
fi
done
mkdir -p "$output"
printf '%s\n' '{"duplicates":[],"statistics":{}}' > "$output/jscpd-report.json"
""",
)
for tool in ("pip-audit", "npm", "radon", "xenon"):
self._write_stub(
tool,
f"""
if [[ " $* " == *" --version "* ]]; then
echo '{tool} stub 1.0'
exit 0
fi
exit 0
""",
)
def _run(
self, *arguments: str, mode: str = "quick", **environment: str
) -> tuple[subprocess.CompletedProcess[str], Path]:
reports = self.temporary_root / "reports"
env = os.environ.copy()
env.update(
{
"PATH": f"{self.stub_bin}:{env['PATH']}",
"SECURITY_AUDIT_REQUIRE_TOOLS": "1",
"SECURITY_AUDIT_TOOLBOX_FINGERPRINT": 'stub-"safe',
**environment,
}
)
result = subprocess.run(
[
"bash",
str(self.checkout / "tools" / "checks" / AUDIT_SCRIPT.name),
"--mode",
mode,
"--scope",
"current",
"--reports-dir",
str(reports),
*arguments,
],
cwd=self.checkout,
env=env,
check=False,
capture_output=True,
text=True,
)
return result, reports
@staticmethod
def _manifest(reports: Path) -> dict[str, object]:
return json.loads((reports / "manifest.json").read_text(encoding="utf-8"))
def test_report_only_findings_are_nonfatal_and_strict_findings_fail(self) -> None:
result, reports = self._run("--report-only", STUB_SEMGREP_EXIT="1")
self.assertEqual(0, result.returncode, result.stderr)
manifest = self._manifest(reports)
self.assertEqual(0, manifest["overall_status"])
self.assertEqual(1, manifest["finding_status"])
self.assertEqual(0, manifest["execution_error_status"])
self.assertEqual("complete", manifest["coverage_status"])
scanner_results = {
result["id"]: result["status"]
for result in manifest["scanner_coverage"]["results"]
}
self.assertEqual("findings", scanner_results["semgrep"])
self.assertIn(
"Semgrep SAST\t1\tfindings", (reports / "step-status.tsv").read_text()
)
strict_result, strict_reports = self._run("--strict", STUB_SEMGREP_EXIT="1")
self.assertEqual(1, strict_result.returncode)
self.assertEqual(1, self._manifest(strict_reports)["overall_status"])
def test_scanner_execution_error_is_fatal_in_report_only_mode(self) -> None:
result, reports = self._run("--report-only", STUB_SEMGREP_EXIT="2")
self.assertEqual(1, result.returncode)
manifest = self._manifest(reports)
self.assertEqual(1, manifest["overall_status"])
self.assertEqual(1, manifest["execution_error_status"])
self.assertIn(
"Semgrep SAST\t2\texecution-error",
(reports / "step-status.tsv").read_text(),
)
def test_malformed_or_missing_machine_report_is_fatal(self) -> None:
for report_kind in ("malformed", "invalid-sarif", "missing"):
with self.subTest(report_kind=report_kind):
result, reports = self._run(
"--report-only",
STUB_SEMGREP_REPORT=report_kind,
)
self.assertEqual(1, result.returncode)
manifest = self._manifest(reports)
self.assertEqual(1, manifest["execution_error_status"])
if report_kind == "missing":
self.assertIn("semgrep.sarif", manifest["missing_reports"])
self.assertIn(
"Validate machine-readable audit reports\t2\texecution-error",
(reports / "step-status.tsv").read_text(),
)
def test_osv_null_results_are_valid_but_wrong_shapes_fail(self) -> None:
self._install_full_mode_stubs()
result, reports = self._run("--report-only", mode="full")
self.assertEqual(0, result.returncode, result.stderr)
osv_report = json.loads(
(reports / "osv-scanner-checkout.json").read_text(encoding="utf-8")
)
self.assertIsNone(osv_report["results"])
invalid_result, invalid_reports = self._run(
"--report-only",
mode="full",
STUB_OSV_RESULTS="{}",
)
self.assertEqual(1, invalid_result.returncode)
self.assertIn(
"invalid OSV-Scanner report",
invalid_result.stderr,
)
self.assertEqual(1, self._manifest(invalid_reports)["execution_error_status"])
def test_missing_tool_is_machine_readable_and_strictly_enforced(self) -> None:
gitleaks_stub = self.stub_bin / "gitleaks"
disabled_stub = self.stub_bin / "gitleaks.disabled"
gitleaks_stub.rename(disabled_stub)
try:
result, reports = self._run(
"--report-only",
SECURITY_AUDIT_REQUIRE_TOOLS="0",
CI="false",
GITEA_ACTIONS="false",
)
self.assertEqual(0, result.returncode, result.stderr)
manifest = self._manifest(reports)
self.assertEqual("incomplete", manifest["coverage_status"])
self.assertEqual(["gitleaks"], manifest["scanner_coverage"]["incomplete"])
scanner_results = {
item["id"]: item for item in manifest["scanner_coverage"]["results"]
}
self.assertEqual("skipped", scanner_results["gitleaks"]["status"])
self.assertEqual(127, scanner_results["gitleaks"]["exit_code"])
self.assertIn(
"gitleaks\tmissing",
(reports / "tool-versions.txt").read_text(encoding="utf-8"),
)
strict_result, strict_reports = self._run(
"--strict",
SECURITY_AUDIT_REQUIRE_TOOLS="0",
CI="false",
GITEA_ACTIONS="false",
)
self.assertEqual(1, strict_result.returncode)
self.assertEqual(
"incomplete",
self._manifest(strict_reports)["coverage_status"],
)
ci_result, ci_reports = self._run(
"--report-only",
SECURITY_AUDIT_REQUIRE_TOOLS="0",
CI="true",
GITEA_ACTIONS="false",
)
self.assertEqual(1, ci_result.returncode)
self.assertEqual(
"incomplete",
self._manifest(ci_reports)["coverage_status"],
)
finally:
disabled_stub.rename(gitleaks_stub)
def test_full_mode_records_every_required_scanner(self) -> None:
self._install_full_mode_stubs()
result, reports = self._run("--report-only", mode="full")
self.assertEqual(0, result.returncode, result.stderr)
manifest = self._manifest(reports)
self.assertEqual("complete", manifest["coverage_status"])
required = manifest["scanner_coverage"]["required"]
results = manifest["scanner_coverage"]["results"]
self.assertEqual(
{
"semgrep",
"bandit",
"ruff-security",
"gitleaks",
"trivy",
"pip-audit",
"npm-audit",
"osv-scanner",
"jscpd",
"radon",
"xenon",
},
set(required),
)
self.assertEqual(set(required), {item["id"] for item in results})
self.assertTrue(all(item["status"] == "no-findings" for item in results))
def test_workspace_mutation_invalidates_the_audit(self) -> None:
result, reports = self._run(
"--report-only",
STUB_MUTATE_PATH=str(self.checkout / "README.md"),
)
self.assertEqual(1, result.returncode)
manifest = self._manifest(reports)
self.assertFalse(manifest["workspace_unchanged"])
self.assertEqual(1, manifest["overall_status"])
self.assertNotEqual(
(reports / "workspace-state-start.tsv").read_text(),
(reports / "workspace-state-end.tsv").read_text(),
)
def test_manifest_and_checksums_cover_only_current_run_reports(self) -> None:
reports = self.temporary_root / "reports"
reports.mkdir()
(reports / "stale-malformed.json").write_text("{", encoding="utf-8")
result, reports = self._run("--report-only")
self.assertEqual(0, result.returncode, result.stderr)
manifest = self._manifest(reports)
self.assertEqual('stub-"safe', manifest["toolbox_fingerprint"])
self.assertNotIn("stale-malformed.json", manifest["reports"])
self.assertEqual([], manifest["missing_reports"])
checksums = (reports / "report-sha256.txt").read_text(encoding="utf-8")
self.assertNotIn("stale-malformed.json", checksums)
verification = subprocess.run(
["sha256sum", "--check", "report-sha256.txt"],
cwd=reports,
check=False,
capture_output=True,
text=True,
)
self.assertEqual(0, verification.returncode, verification.stderr)
checksummed_paths = {
line.split(" ", 1)[1] for line in checksums.splitlines() if " " in line
}
self.assertEqual(set(manifest["reports"]), checksummed_paths)
class SecurityAuditContainerRunnerTests(unittest.TestCase):
def setUp(self) -> None:
self._temporary_directory = tempfile.TemporaryDirectory(
prefix="govoplan-audit-runner-"
)
root = Path(self._temporary_directory.name)
self.stub_bin = root / "bin"
self.stub_bin.mkdir()
self.docker_log = root / "docker.jsonl"
docker_stub = self.stub_bin / "docker"
docker_stub.write_text(
textwrap.dedent(
"""\
#!/usr/bin/env python3
import json
import os
from pathlib import Path
import sys
arguments = sys.argv[1:]
with Path(os.environ["DOCKER_STUB_LOG"]).open(
"a", encoding="utf-8"
) as handle:
handle.write(json.dumps(arguments) + "\\n")
if arguments and arguments[0] == "version":
print("26.1.0")
if arguments[:2] == ["container", "inspect"]:
print(os.environ["DOCKER_STUB_MOUNTS"])
"""
),
encoding="utf-8",
)
docker_stub.chmod(0o755)
self.environment = os.environ.copy()
self.environment.update(
{
"DOCKER_STUB_LOG": str(self.docker_log),
"DOCKER_STUB_MOUNTS": "[]",
"PATH": f"{self.stub_bin}:{self.environment['PATH']}",
}
)
def tearDown(self) -> None:
self._temporary_directory.cleanup()
def _run(
self,
*,
scope: str,
actions_mounts: object | None = None,
**environment_overrides: str,
) -> tuple[subprocess.CompletedProcess[str], list[list[str]]]:
self.docker_log.write_text("", encoding="utf-8")
environment = self.environment.copy()
environment.update(environment_overrides)
if actions_mounts is None:
environment.pop("GITEA_ACTIONS", None)
else:
environment["GITEA_ACTIONS"] = "true"
environment["DOCKER_STUB_MOUNTS"] = json.dumps(actions_mounts)
result = subprocess.run(
[
"bash",
str(CONTAINER_RUNNER),
"--mode",
"quick",
"--scope",
scope,
],
cwd=META_ROOT,
env=environment,
check=False,
capture_output=True,
text=True,
)
commands = [
json.loads(line)
for line in self.docker_log.read_text(encoding="utf-8").splitlines()
]
return result, commands
def test_gitea_job_shares_only_its_workspace_mount(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
{
"Type": "volume",
"Name": "govoplan-actions-workspace",
"Destination": str(META_ROOT.parent),
"RW": True,
},
{
"Type": "volume",
"Name": "govoplan-actions-environment",
"Destination": "/var/run/act",
"RW": True,
},
]
container_id = subprocess.run(
["hostname"],
check=True,
capture_output=True,
text=True,
).stdout.strip()
for scope in ("current", "govoplan"):
with self.subTest(scope=scope):
result, commands = self._run(scope=scope, actions_mounts=mounts)
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(
command for command in commands if command[0] == "run"
)
inspect_command = next(
command
for command in commands
if command[:2] == ["container", "inspect"]
)
self.assertEqual(container_id, inspect_command[-1])
mount_index = run_command.index("--mount")
self.assertEqual(
(
"type=volume,source=govoplan-actions-workspace,"
f"target={META_ROOT.parent}"
),
run_command[mount_index + 1],
)
self.assertNotIn("--volumes-from", run_command)
self.assertNotIn("-v", run_command)
self.assertNotIn("/var/run/docker.sock", " ".join(run_command))
self.assertNotIn("/var/run/act", " ".join(run_command))
repository_roots = [
value
for value in run_command
if value.startswith("GOVOPLAN_REPOS_ROOT=")
]
expected_roots = (
[f"GOVOPLAN_REPOS_ROOT={META_ROOT.parent}"]
if scope == "govoplan"
else []
)
self.assertEqual(expected_roots, repository_roots)
self.assertIn("SECURITY_AUDIT_REQUIRE_TOOLS=1", run_command)
def test_gitea_job_passes_only_public_git_url_rewrites(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "govoplan-actions-workspace",
"Destination": str(META_ROOT.parent),
"RW": True,
}
]
git_config = {
"GIT_CONFIG_COUNT": "2",
"GIT_CONFIG_KEY_0": (
"url.https://git.add-ideas.de/GovOPlaN/govoplan.insteadOf"
),
"GIT_CONFIG_VALUE_0": "git@git.add-ideas.de:GovOPlaN/govoplan",
"GIT_CONFIG_KEY_1": (
"url.https://git.add-ideas.de/GovOPlaN/govoplan.insteadOf"
),
"GIT_CONFIG_VALUE_1": (
"ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
),
}
result, commands = self._run(
scope="govoplan",
actions_mounts=mounts,
**git_config,
)
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(command for command in commands if command[0] == "run")
for key, value in git_config.items():
self.assertIn(f"{key}={value}", run_command)
def test_container_runner_rejects_non_url_git_configuration(self) -> None:
result, _commands = self._run(
scope="current",
GIT_CONFIG_COUNT="1",
GIT_CONFIG_KEY_0="credential.helper",
GIT_CONFIG_VALUE_0="!print-secret",
)
self.assertEqual(1, result.returncode)
self.assertIn(
"Only credential-free HTTPS Git insteadOf rewrites",
result.stderr,
)
def test_security_workflow_configures_nested_public_git_resolution(
self,
) -> None:
workflow = (
META_ROOT / ".gitea" / "workflows" / "security-audit.yml"
).read_text(encoding="utf-8")
self.assertIn('GIT_CONFIG_COUNT: "2"', workflow)
self.assertIn(
"url.https://git.add-ideas.de/GovOPlaN/govoplan.insteadOf",
workflow,
)
def test_non_actions_runner_keeps_the_scoped_bind_mount(self) -> None:
result, commands = self._run(scope="govoplan")
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(command for command in commands if command[0] == "run")
bind_index = run_command.index("-v")
self.assertEqual(
f"{META_ROOT.parent}:/workspace",
run_command[bind_index + 1],
)
self.assertNotIn("--mount", run_command)
if __name__ == "__main__":
unittest.main()
+467
View File
@@ -0,0 +1,467 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.version_alignment import ( # noqa: E402
candidate_catalog_version_issues,
release_composition_issues,
selected_release_webui_bundle_issues,
repository_version_issues,
selected_repository_version_issues,
)
from govoplan_release.git_state import sanitized_git_environment # noqa: E402
from govoplan_release.model import ( # noqa: E402
CatalogSnapshot,
DashboardSummary,
ReleaseDashboard,
RepositorySnapshot,
RepositorySpec,
VersionSnapshot,
)
from govoplan_release.selective_planner import build_selective_release_plan, build_unit # noqa: E402
from govoplan_release.selective_catalog import enforce_selected_version_alignment # noqa: E402
class VersionAlignmentTests(unittest.TestCase):
def test_candidate_catalog_refs_and_selected_units_must_match_versions(self) -> None:
payload = {
"core_release": {
"version": "1.2.3",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.3",
},
"modules": [
{
"module_id": "files",
"version": "1.2.3",
"python_ref": "govoplan-files @ git+ssh://git@example.test/acme/govoplan-files.git@v1.2.4",
"webui_package": "@govoplan/files-webui",
"webui_ref": "git+ssh://git@example.test/acme/govoplan-files.git#v1.2.3",
}
],
"release": {
"selected_units": [
{"repo": "govoplan-files", "version": "1.2.5", "tag": "v1.2.6"},
]
},
}
issues = candidate_catalog_version_issues(payload)
messages = {issue.message for issue in issues}
self.assertIn("Python ref tag must match the catalog entry version", messages)
self.assertIn("selected-unit tag must match its version", messages)
self.assertIn("selected-unit version must match the catalog entry", messages)
def test_catalog_candidate_gate_rejects_selected_repo_drift(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
repo = workspace / "govoplan-files"
(repo / "webui").mkdir(parents=True)
(repo / "pyproject.toml").write_text('[project]\nname="govoplan-files"\nversion="1.2.3"\n')
(repo / "webui" / "package.json").write_text('{"version":"1.2.4"}\n')
with self.assertRaisesRegex(ValueError, "Version alignment gate failed"):
enforce_selected_version_alignment(
repo_versions={"govoplan-files": "1.2.5"},
workspace=workspace,
)
def test_catalog_candidate_gate_rejects_requested_version_that_differs_from_source(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
repo = workspace / "govoplan-files"
(repo / "webui").mkdir(parents=True)
(repo / "pyproject.toml").write_text('[project]\nname="govoplan-files"\nversion="1.2.3"\n')
(repo / "webui" / "package.json").write_text('{"version":"1.2.3"}\n')
with self.assertRaisesRegex(ValueError, "source version must match the requested release version"):
enforce_selected_version_alignment(
repo_versions={"govoplan-files": "1.2.4"},
workspace=workspace,
)
def test_selected_webui_release_must_match_core_bundle_input_and_lock(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
core_webui = workspace / "govoplan-core" / "webui"
campaign_webui = workspace / "govoplan-campaign" / "webui"
core_webui.mkdir(parents=True)
campaign_webui.mkdir(parents=True)
dependency = "@govoplan/campaign-webui"
old_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v1.2.2"
(campaign_webui / "package.json").write_text(
json.dumps({"name": dependency, "version": "1.2.3"})
)
(core_webui / "package.release.json").write_text(
json.dumps({"dependencies": {dependency: old_ref}})
)
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"dependencies": {dependency: old_ref}},
f"node_modules/{dependency}": {"version": "1.2.2"},
}
}
)
)
issues = selected_release_webui_bundle_issues(
repo_versions={"govoplan-core": "2.0.0", "govoplan-campaign": "1.2.3"},
workspace=workspace,
)
with self.assertRaisesRegex(ValueError, "Core release WebUI dependency"):
enforce_selected_version_alignment(
repo_versions={"govoplan-campaign": "1.2.3"},
workspace=workspace,
)
self.assertEqual(2, len(issues))
self.assertTrue(all(issue.repo == "govoplan-campaign" for issue in issues))
self.assertEqual(
{
"Core release WebUI dependency must match the selected module version",
"Core release lock must resolve the selected module version",
},
{issue.message for issue in issues},
)
def test_selected_webui_gate_ignores_unselected_module_pins(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
core_webui = workspace / "govoplan-core" / "webui"
campaign_webui = workspace / "govoplan-campaign" / "webui"
core_webui.mkdir(parents=True)
campaign_webui.mkdir(parents=True)
campaign_package = "@govoplan/campaign-webui"
files_package = "@govoplan/files-webui"
campaign_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v1.2.3"
files_ref = "git+ssh://git@example.test/acme/govoplan-files.git#v0.9.0"
(campaign_webui / "package.json").write_text(
json.dumps({"name": campaign_package, "version": "1.2.3"})
)
dependencies = {campaign_package: campaign_ref, files_package: files_ref}
(core_webui / "package.release.json").write_text(json.dumps({"dependencies": dependencies}))
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"dependencies": dependencies},
f"node_modules/{campaign_package}": {"version": "1.2.3"},
f"node_modules/{files_package}": {"version": "0.9.0"},
}
}
)
)
issues = selected_release_webui_bundle_issues(
repo_versions={"govoplan-campaign": "1.2.3"},
workspace=workspace,
)
self.assertEqual((), issues)
def test_selective_plan_blocks_mismatched_selected_webui_bundle(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
core_webui = workspace / "govoplan-core" / "webui"
campaign_webui = workspace / "govoplan-campaign" / "webui"
core_webui.mkdir(parents=True)
campaign_webui.mkdir(parents=True)
package_name = "@govoplan/campaign-webui"
old_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v1.2.2"
(campaign_webui / "package.json").write_text(
json.dumps({"name": package_name, "version": "1.2.3"})
)
(core_webui / "package.release.json").write_text(
json.dumps({"dependencies": {package_name: old_ref}})
)
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"dependencies": {package_name: old_ref}},
f"node_modules/{package_name}": {"version": "1.2.2"},
}
}
)
)
campaign = RepositorySnapshot(
spec=RepositorySpec(
name="govoplan-campaign",
category="module",
subtype="domain",
remote="git@example.test:acme/govoplan-campaign.git",
path="govoplan-campaign",
),
absolute_path=str(workspace / "govoplan-campaign"),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject="1.2.3", webui_package="1.2.3"),
)
dashboard = ReleaseDashboard(
generated_at="2026-01-01T00:00:00Z",
meta_root=str(workspace / "govoplan"),
workspace_root=str(workspace),
target_version=None,
target_tag=None,
online=False,
include_migrations=False,
summary=DashboardSummary(
repository_count=1,
missing_count=0,
dirty_count=0,
ahead_count=0,
behind_count=0,
no_head_count=0,
error_count=0,
safe_directory_count=0,
local_target_tag_missing_count=0,
status="ready",
),
repositories=(campaign,),
catalog=CatalogSnapshot(
channel="stable",
website_path="",
catalog_path="",
catalog_exists=False,
keyring_path="",
keyring_exists=False,
),
)
plan = build_selective_release_plan(
dashboard,
selected_repos=("govoplan-campaign",),
repo_versions={"govoplan-campaign": "1.2.3"},
)
self.assertEqual("blocked", plan.status)
self.assertEqual("blocked", plan.units[0].status)
self.assertTrue(any("Core release WebUI dependency" in item for item in plan.units[0].blockers))
finding = next(
item
for item in plan.gate_findings
if item.code == "release_webui_composition_alignment"
)
self.assertIn("package.release.json", finding.source)
self.assertIn("regenerate webui/package-lock.release.json", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
def test_selected_repository_gate_reports_missing_version_metadata(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
(workspace / "govoplan-files").mkdir()
issues = selected_repository_version_issues(
repo_versions={"govoplan-files": "1.2.3"},
workspace=workspace,
)
self.assertEqual(1, len(issues))
self.assertEqual("repository has no version metadata", issues[0].message)
def test_selective_release_unit_is_blocked_by_version_drift(self) -> None:
repo = RepositorySnapshot(
spec=RepositorySpec(
name="govoplan-example",
category="module",
subtype="domain",
remote="git@example.test:acme/govoplan-example.git",
path="govoplan-example",
),
absolute_path="/tmp/govoplan-example",
exists=True,
is_git=True,
has_head=True,
versions=VersionSnapshot(pyproject="1.2.3", webui_package="1.2.4", manifests=("1.2.3",)),
)
unit = build_unit(repo, target_version="1.2.5", contracts=None)
self.assertEqual("blocked", unit.status)
self.assertIn("version metadata is not aligned", unit.blockers[0])
def test_repository_versions_and_lockfiles_must_match(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp) / "govoplan-example"
(root / "src" / "govoplan_example" / "backend").mkdir(parents=True)
(root / "webui").mkdir()
(root / "pyproject.toml").write_text('[project]\nname = "govoplan-example"\nversion = "1.2.3"\n')
(root / "package.json").write_text('{"name":"@govoplan/example","version":"1.2.3"}\n')
(root / "webui" / "package.json").write_text('{"name":"@govoplan/example-webui","version":"1.2.4"}\n')
(root / "webui" / "package-lock.json").write_text(
json.dumps({"packages": {"": {"version": "1.2.2"}}})
)
(root / "src" / "govoplan_example" / "backend" / "manifest.py").write_text(
'manifest = ModuleManifest(\n version="1.2.3",\n)\n'
)
issues = repository_version_issues(root)
self.assertEqual(2, len(issues))
self.assertEqual({"webui/package.json", "webui/package-lock.json"}, {issue.source for issue in issues})
def test_public_runtime_version_must_match_package_metadata(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp) / "govoplan-example"
package = root / "src" / "govoplan_example"
package.mkdir(parents=True)
(root / "pyproject.toml").write_text(
'[project]\nname = "govoplan-example"\nversion = "1.2.3"\n'
)
(package / "__init__.py").write_text('__version__ = "1.2.2"\n')
issues = repository_version_issues(root)
self.assertEqual(1, len(issues))
self.assertEqual("package __init__.py 1", issues[0].source)
self.assertEqual("1.2.3", issues[0].expected)
self.assertEqual("1.2.2", issues[0].actual)
def test_release_backend_and_frontend_refs_must_match(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
meta = Path(tmp) / "govoplan"
core = Path(tmp) / "govoplan-core"
meta.mkdir()
(core / "webui").mkdir(parents=True)
(meta / "requirements-release.txt").write_text(
"govoplan-example @ git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3\n"
)
(core / "webui" / "package.release.json").write_text(
json.dumps(
{
"dependencies": {
"@govoplan/example-webui": (
"git+ssh://git@example.test/acme/govoplan-example.git#v1.2.4"
)
}
}
)
)
issues = release_composition_issues(meta, core_root=core)
self.assertEqual(1, len(issues))
self.assertEqual("1.2.3", issues[0].expected)
self.assertEqual("1.2.4", issues[0].actual)
def test_release_refs_must_point_at_installable_versioned_artifacts(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
module = workspace / "govoplan-example"
meta.mkdir()
(core / "webui").mkdir(parents=True)
module.mkdir()
(meta / "requirements-release.txt").write_text(
"govoplan-example @ git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3\n"
)
(core / "webui" / "package.release.json").write_text(
json.dumps(
{
"dependencies": {
"@govoplan/example-webui": (
"git+ssh://git@example.test/acme/govoplan-example.git#v1.2.3"
)
}
}
)
)
subprocess.run(["git", "init", "-q", str(module)], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.email", "test@example.test"], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.name", "Test"], check=True)
(module / "README.md").write_text("uninstallable release\n")
subprocess.run(["git", "-C", str(module), "add", "README.md"], check=True)
subprocess.run(["git", "-C", str(module), "commit", "-qm", "release"], check=True)
subprocess.run(["git", "-C", str(module), "tag", "v1.2.3"], check=True)
issues = release_composition_issues(meta, core_root=core)
self.assertEqual(2, len(issues))
self.assertEqual(
{"v1.2.3:pyproject.toml", "v1.2.3:webui/package.json"},
{issue.source for issue in issues},
)
self.assertTrue(all("must contain" in issue.message for issue in issues))
def test_annotated_release_tags_work_with_sanitized_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
module = workspace / "govoplan-example"
meta.mkdir()
(core / "webui").mkdir(parents=True)
(module / "webui").mkdir(parents=True)
backend_ref = "git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3"
webui_ref = "git+ssh://git@example.test/acme/govoplan-example.git#v1.2.3"
(meta / "requirements-release.txt").write_text(f"govoplan-example @ {backend_ref}\n")
(module / "pyproject.toml").write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
(module / "webui" / "package.json").write_text(
'{"name":"@govoplan/example-webui","version":"1.2.3"}\n'
)
subprocess.run(["git", "init", "-q", str(module)], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.email", "test@example.test"], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.name", "Test"], check=True)
subprocess.run(["git", "-C", str(module), "add", "pyproject.toml", "webui/package.json"], check=True)
subprocess.run(["git", "-C", str(module), "commit", "-qm", "release"], check=True)
subprocess.run(
["git", "-C", str(module), "tag", "-a", "v1.2.3", "-m", "Release v1.2.3"],
check=True,
)
tagged_commit = subprocess.run(
["git", "-C", str(module), "rev-parse", "refs/tags/v1.2.3^{commit}"],
check=True,
text=True,
stdout=subprocess.PIPE,
).stdout.strip()
dependencies = {"@govoplan/example-webui": webui_ref}
(core / "pyproject.toml").write_text('[project]\nname="govoplan-core"\nversion="2.0.0"\n')
(core / "webui" / "package.release.json").write_text(
json.dumps({"version": "2.0.0", "dependencies": dependencies})
)
(core / "webui" / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"version": "2.0.0", "dependencies": dependencies},
"node_modules/@govoplan/example-webui": {
"version": "1.2.3",
"resolved": f"git+ssh://git@example.test/acme/govoplan-example.git#{tagged_commit}",
},
}
}
)
)
git_environment = sanitized_git_environment()
git_environment["GIT_TEST_ASSUME_DIFFERENT_OWNER"] = "1"
with patch(
"govoplan_release.version_alignment.sanitized_git_environment",
return_value=git_environment,
):
composition_issues = release_composition_issues(meta, core_root=core)
core_issues = repository_version_issues(core)
self.assertEqual((), composition_issues)
self.assertEqual((), core_issues)
if __name__ == "__main__":
unittest.main()
+451
View File
@@ -0,0 +1,451 @@
#!/usr/bin/env python3
"""Re-run a capability-fit assessment against a trusted release catalog."""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import re
import sys
META_ROOT = Path(__file__).resolve().parents[2]
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
for tools_root in (ASSESSMENT_TOOLS_ROOT, RELEASE_TOOLS_ROOT):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from govoplan_assessment import ( # noqa: E402
collect_installed_composition,
render_review,
review_capability_fit,
)
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
from govoplan_assessment.evidence import validate_payload # noqa: E402
from govoplan_release.catalog import DEFAULT_PUBLIC_BASE_URL, fetch_json # noqa: E402
TRUSTED_KEYRING_FILE_ENV = "GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE"
MAX_ASSESSMENT_INPUT_BYTES = 16 * 1024 * 1024
MAX_EVIDENCE_INPUT_BYTES = 4 * 1024 * 1024
MAX_REPORT_OUTPUT_BYTES = 16 * 1024 * 1024
OPAQUE_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,159}$")
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Validate a fit assessment and identify conclusions needing review after release drift."
)
parser.add_argument(
"--assessment",
type=Path,
default=META_ROOT / "docs" / "capability-fit-current.json",
)
parser.add_argument(
"--schema",
type=Path,
default=META_ROOT / "docs" / "capability-fit.schema.json",
)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument(
"--public",
action="store_true",
help="Use the fixed public GovOPlaN stable catalog and keyring URLs.",
)
source.add_argument(
"--catalog", type=Path, help="Read a catalog from a local file."
)
parser.add_argument(
"--keyring",
type=Path,
help="Published or candidate keyring whose hash is pinned by --catalog.",
)
parser.add_argument(
"--trusted-keyring",
type=Path,
help=(
"Independently pinned local keyring used as the catalog signature trust "
f"root; defaults to ${TRUSTED_KEYRING_FILE_ENV}."
),
)
parser.add_argument(
"--workspace-root",
type=Path,
default=META_ROOT.parent,
help="Workspace containing repository checkouts for local tag provenance.",
)
parser.add_argument(
"--skip-tag-provenance",
action="store_true",
help="Compare signed metadata without checking local annotated tags.",
)
installed = parser.add_mutually_exclusive_group()
installed.add_argument(
"--installed-evidence",
type=Path,
help="Validate an existing bounded installed-composition evidence document.",
)
installed.add_argument(
"--collect-installed-evidence",
type=Path,
help=(
"Collect the current interpreter's GovOPlaN distributions, write the "
"bounded evidence document, and include it in this review. Loading "
"module entry points executes installed GovOPlaN manifest factories."
),
)
parser.add_argument(
"--installed-evidence-schema",
type=Path,
default=META_ROOT / "docs" / "installed-composition-evidence.schema.json",
)
parser.add_argument(
"--installer-receipt",
type=Path,
help="Signed installer receipt binding installed payloads to this catalog.",
)
parser.add_argument(
"--installer-receipt-schema",
type=Path,
default=META_ROOT / "docs" / "installer-receipt.schema.json",
)
parser.add_argument(
"--installer-authority-keyring",
type=Path,
help=(
"Independently provisioned, role-scoped trust root for installer "
"receipt signatures."
),
)
parser.add_argument(
"--installer-authority-keyring-schema",
type=Path,
default=META_ROOT
/ "docs"
/ "installer-receipt-authority-keyring.schema.json",
)
parser.add_argument(
"--boundary-evidence",
type=Path,
help=(
"Externally issued target/provider/production proof bound to the "
"installed-evidence digest."
),
)
parser.add_argument(
"--boundary-evidence-schema",
type=Path,
default=META_ROOT / "docs" / "capability-fit-boundary-evidence.schema.json",
)
parser.add_argument(
"--boundary-authority-keyring",
type=Path,
help=(
"Independently provisioned keyring authorizing proof signers for "
"specific target/provider/production scopes."
),
)
parser.add_argument(
"--boundary-authority-keyring-schema",
type=Path,
default=META_ROOT
/ "docs"
/ "capability-fit-proof-authority-keyring.schema.json",
)
parser.add_argument(
"--verification-time",
help="Optional ISO-8601 time for reproducible boundary-proof verification.",
)
parser.add_argument(
"--expected-external-provider-subject",
help=(
"Opaque provider-environment ID that external-provider proof must match; "
"never supply a URL, credential, or endpoint identifier."
),
)
parser.add_argument(
"--json", action="store_true", help="Print the machine-readable review report."
)
parser.add_argument(
"--output",
type=Path,
help="Also write the machine-readable review report to this path.",
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
if args.catalog is not None and args.keyring is None:
raise SystemExit("--keyring is required with --catalog")
if args.public and args.keyring is not None:
raise SystemExit("--keyring cannot be combined with --public")
if (args.boundary_evidence is None) != (args.boundary_authority_keyring is None):
raise SystemExit(
"--boundary-evidence and --boundary-authority-keyring are required together"
)
if (args.installer_receipt is None) != (
args.installer_authority_keyring is None
):
raise SystemExit(
"--installer-receipt and --installer-authority-keyring are required together"
)
if args.installer_receipt is not None and (
args.installed_evidence is None and args.collect_installed_evidence is None
):
raise SystemExit(
"--installer-receipt requires --installed-evidence or --collect-installed-evidence"
)
if args.boundary_evidence is not None and (
args.installed_evidence is None and args.collect_installed_evidence is None
):
raise SystemExit(
"--boundary-evidence requires --installed-evidence or --collect-installed-evidence"
)
if (
args.expected_external_provider_subject is not None
and OPAQUE_ID_PATTERN.fullmatch(args.expected_external_provider_subject) is None
):
raise SystemExit(
"--expected-external-provider-subject must be a bounded opaque ID"
)
if (
args.collect_installed_evidence is not None
and args.output is not None
and args.collect_installed_evidence.resolve() == args.output.resolve()
):
raise SystemExit("Evidence and review output paths must differ")
configured_trusted_keyring = os.getenv(TRUSTED_KEYRING_FILE_ENV, "").strip()
trusted_keyring_path = args.trusted_keyring or (
Path(configured_trusted_keyring) if configured_trusted_keyring else None
)
if trusted_keyring_path is None:
raise SystemExit(
f"--trusted-keyring or ${TRUSTED_KEYRING_FILE_ENV} is required"
)
assessment = read_object(
args.assessment,
label="assessment",
max_bytes=MAX_ASSESSMENT_INPUT_BYTES,
)
schema = read_object(
args.schema,
label="schema",
max_bytes=MAX_ASSESSMENT_INPUT_BYTES,
)
if args.public:
catalog = fetch_public_json(
f"{DEFAULT_PUBLIC_BASE_URL}/catalogs/v1/channels/stable.json",
label="catalog",
)
published_keyring = fetch_public_json(
f"{DEFAULT_PUBLIC_BASE_URL}/catalogs/v1/keyring.json",
label="published keyring",
)
else:
catalog = read_object(
args.catalog, label="catalog", max_bytes=MAX_ASSESSMENT_INPUT_BYTES
)
published_keyring = read_object(
args.keyring,
label="published keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
trusted_keyring = read_object(
trusted_keyring_path,
label="trusted keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installed_evidence = None
installed_evidence_schema = None
if (
args.installed_evidence is not None
or args.collect_installed_evidence is not None
):
installed_evidence_schema = read_object(
args.installed_evidence_schema,
label="installed evidence schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
if args.installed_evidence is not None:
installed_evidence = read_object(
args.installed_evidence,
label="installed evidence",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
else:
installed_evidence = collect_installed_composition(assessment=assessment)
collection_schema_errors = validate_payload(
payload=installed_evidence,
schema=installed_evidence_schema,
)
if collection_schema_errors:
raise SystemExit(
"Collected installed evidence did not satisfy its bounded schema"
)
write_object(
args.collect_installed_evidence,
installed_evidence,
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
label="installed evidence",
)
boundary_evidence = None
boundary_evidence_schema = None
boundary_authority_keyring = None
boundary_authority_keyring_schema = None
if args.boundary_evidence is not None:
boundary_evidence = read_object(
args.boundary_evidence,
label="boundary evidence",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
boundary_evidence_schema = read_object(
args.boundary_evidence_schema,
label="boundary evidence schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
boundary_authority_keyring = read_object(
args.boundary_authority_keyring,
label="boundary authority keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
boundary_authority_keyring_schema = read_object(
args.boundary_authority_keyring_schema,
label="boundary authority keyring schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_receipt = None
installer_receipt_schema = None
installer_authority_keyring = None
installer_authority_keyring_schema = None
if args.installer_receipt is not None:
installer_receipt = read_object(
args.installer_receipt,
label="installer receipt",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_receipt_schema = read_object(
args.installer_receipt_schema,
label="installer receipt schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_authority_keyring = read_object(
args.installer_authority_keyring,
label="installer authority keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_authority_keyring_schema = read_object(
args.installer_authority_keyring_schema,
label="installer authority keyring schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
verification_time = parse_datetime(args.verification_time)
report = review_capability_fit(
assessment=assessment,
schema=schema,
catalog=catalog,
published_keyring=published_keyring,
trusted_keyring=trusted_keyring,
workspace_root=None
if args.skip_tag_provenance
else args.workspace_root.resolve(),
installed_evidence=installed_evidence,
installed_evidence_schema=installed_evidence_schema,
installer_receipt=installer_receipt,
installer_receipt_schema=installer_receipt_schema,
installer_authority_keyring=installer_authority_keyring,
installer_authority_keyring_schema=installer_authority_keyring_schema,
boundary_evidence=boundary_evidence,
boundary_evidence_schema=boundary_evidence_schema,
boundary_authority_keyring=boundary_authority_keyring,
boundary_authority_keyring_schema=boundary_authority_keyring_schema,
verification_time=verification_time,
installed_evidence_mode=(
"direct_local"
if args.collect_installed_evidence is not None
else "imported_unsigned"
),
expected_external_provider_subject=args.expected_external_provider_subject,
)
encoded = json.dumps(report, indent=2, sort_keys=True) + "\n"
if args.output is not None:
write_object(
args.output,
report,
max_bytes=MAX_REPORT_OUTPUT_BYTES,
label="review report",
)
sys.stdout.write(encoded if args.json else render_review(report))
return {"current": 0, "blocked": 1, "review_required": 2}[report["status"]]
def read_object(path: Path | None, *, label: str, max_bytes: int) -> dict[str, object]:
if path is None:
raise SystemExit(f"Missing {label} path")
try:
if path.stat().st_size > max_bytes:
raise SystemExit(
f"{label.capitalize()} exceeds the {max_bytes}-byte input limit"
)
with path.open("rb") as handle:
encoded = handle.read(max_bytes + 1)
if len(encoded) > max_bytes:
raise SystemExit(
f"{label.capitalize()} exceeds the {max_bytes}-byte input limit"
)
payload = json.loads(encoded.decode("utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise SystemExit(f"Could not read {label} {path}: {exc}") from exc
if not isinstance(payload, dict):
raise SystemExit(f"{label.capitalize()} must be a JSON object: {path}")
return payload
def write_object(
path: Path | None,
payload: dict[str, object],
*,
max_bytes: int,
label: str,
) -> None:
if path is None:
raise SystemExit(f"Missing {label} output path")
try:
atomic_write_json(path, payload, max_bytes=max_bytes)
except AtomicJsonWriteError as exc:
raise SystemExit(f"Could not securely write {label}") from exc
def parse_datetime(value: str | None):
if value is None:
return None
from datetime import datetime
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise SystemExit("--verification-time must be an ISO-8601 timestamp") from exc
if parsed.tzinfo is None:
raise SystemExit("--verification-time must include a timezone")
return parsed
def fetch_public_json(url: str, *, label: str) -> dict[str, object]:
result = fetch_json(url)
if result.get("ok") is not True or not isinstance(result.get("payload"), dict):
raise SystemExit(f"Could not fetch fixed public {label} endpoint.")
return result["payload"]
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,10 @@
"""Repeatable GovOPlaN product-assessment tooling."""
from .capability_fit import review_capability_fit, render_review
from .evidence import collect_installed_composition
__all__ = (
"collect_installed_composition",
"render_review",
"review_capability_fit",
)
@@ -0,0 +1,230 @@
"""Bounded, private, atomic JSON output for assessment evidence."""
from __future__ import annotations
import json
import os
from pathlib import Path
import stat
import tempfile
from typing import Any
class AtomicJsonWriteError(RuntimeError):
"""Raised when JSON output cannot be written with the required guarantees."""
def atomic_write_json(
path: str | os.PathLike[str],
payload: Any,
*,
max_bytes: int,
) -> None:
"""Write bounded JSON through a private same-directory temporary file.
A successful return means the file contents and the containing directory
entry have both been flushed. If flushing the directory fails after the
atomic replacement, the replacement may be visible but its crash
durability is unknown, so the function reports failure. Every parent must
already exist as a real directory; symbolic-link components are rejected.
"""
if isinstance(max_bytes, bool) or not isinstance(max_bytes, int) or max_bytes <= 0:
raise ValueError("max_bytes must be a positive integer")
encoded = _encode_bounded_json(payload, max_bytes=max_bytes)
target = Path(os.path.abspath(os.fspath(path)))
parent = target.parent
descriptor = -1
verification_descriptor = -1
temporary_path: Path | None = None
temporary_identity: tuple[int, int] | None = None
try:
_assert_real_parent(parent)
_assert_replaceable_target(target)
try:
descriptor, temporary_name = tempfile.mkstemp(
dir=parent,
prefix=".govoplan-json-",
suffix=".tmp",
)
temporary_path = Path(temporary_name)
os.fchmod(descriptor, 0o600)
metadata = os.fstat(descriptor)
if (
not stat.S_ISREG(metadata.st_mode)
or stat.S_IMODE(metadata.st_mode) != 0o600
):
raise AtomicJsonWriteError(
"Atomic JSON temporary file could not be secured."
)
temporary_identity = (metadata.st_dev, metadata.st_ino)
verification_descriptor = os.dup(descriptor)
handle = os.fdopen(descriptor, "wb")
descriptor = -1
with handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
# Recheck after creating and flushing the temporary file so an
# unsafe target discovered before replacement is never followed.
_assert_real_parent(parent)
_assert_replaceable_target(target)
os.replace(temporary_path, target)
temporary_path = None
_assert_installed_target(
target,
expected_identity=temporary_identity,
recovery_descriptor=verification_descriptor,
)
_fsync_directory(parent)
finally:
try:
if descriptor >= 0:
os.close(descriptor)
finally:
try:
if verification_descriptor >= 0:
os.close(verification_descriptor)
finally:
if temporary_path is not None:
try:
temporary_path.unlink()
except FileNotFoundError:
pass
except AtomicJsonWriteError:
raise
except OSError as exc:
raise AtomicJsonWriteError(
"JSON output could not be written atomically."
) from exc
def _encode_bounded_json(payload: Any, *, max_bytes: int) -> bytes:
encoder = json.JSONEncoder(
allow_nan=False,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
chunks: list[bytes] = []
encoded_size = 1 # The document always ends with one newline.
for chunk in encoder.iterencode(payload):
encoded_chunk = chunk.encode("utf-8")
encoded_size += len(encoded_chunk)
if encoded_size > max_bytes:
raise AtomicJsonWriteError("JSON output exceeds its size limit.")
chunks.append(encoded_chunk)
return b"".join(chunks) + b"\n"
def _assert_replaceable_target(path: Path) -> None:
try:
metadata = path.lstat()
except FileNotFoundError:
return
except OSError as exc:
raise AtomicJsonWriteError(
"JSON output target could not be inspected safely."
) from exc
if stat.S_ISLNK(metadata.st_mode):
raise AtomicJsonWriteError("JSON output target must not be a symbolic link.")
if not stat.S_ISREG(metadata.st_mode):
raise AtomicJsonWriteError("JSON output target must be a regular file.")
def _assert_real_parent(path: Path) -> None:
current = Path(path.anchor)
for part in path.parts[1:]:
current /= part
try:
metadata = current.lstat()
except FileNotFoundError as exc:
raise AtomicJsonWriteError(
"JSON output parent directory must already exist."
) from exc
except OSError as exc:
raise AtomicJsonWriteError(
"JSON output parent directory could not be inspected safely."
) from exc
if stat.S_ISLNK(metadata.st_mode):
raise AtomicJsonWriteError(
"JSON output parent path must not contain symbolic links."
)
if not stat.S_ISDIR(metadata.st_mode):
raise AtomicJsonWriteError(
"JSON output parent path must contain only directories."
)
def _assert_installed_target(
path: Path,
*,
expected_identity: tuple[int, int] | None,
recovery_descriptor: int,
) -> None:
try:
metadata = path.lstat()
except OSError as exc:
raise AtomicJsonWriteError(
"Atomic JSON replacement could not be verified."
) from exc
observed_identity = (metadata.st_dev, metadata.st_ino)
if (
expected_identity is None
or observed_identity != expected_identity
or not stat.S_ISREG(metadata.st_mode)
):
raise AtomicJsonWriteError(
"Atomic JSON replacement did not preserve the secured regular file."
)
if stat.S_IMODE(metadata.st_mode) == 0o600:
return
try:
recovery_metadata = os.fstat(recovery_descriptor)
if (
not stat.S_ISREG(recovery_metadata.st_mode)
or (recovery_metadata.st_dev, recovery_metadata.st_ino) != expected_identity
):
raise AtomicJsonWriteError(
"Atomic JSON replacement recovery descriptor is invalid."
)
os.fchmod(recovery_descriptor, 0o600)
os.fsync(recovery_descriptor)
recovered_metadata = os.fstat(recovery_descriptor)
installed_metadata = path.lstat()
except OSError as exc:
raise AtomicJsonWriteError(
"Atomic JSON replacement permissions could not be restored."
) from exc
if (
stat.S_IMODE(recovered_metadata.st_mode) != 0o600
or (installed_metadata.st_dev, installed_metadata.st_ino) != expected_identity
or not stat.S_ISREG(installed_metadata.st_mode)
or stat.S_IMODE(installed_metadata.st_mode) != 0o600
):
raise AtomicJsonWriteError(
"Atomic JSON replacement permissions could not be restored."
)
_fsync_directory(path.parent)
raise AtomicJsonWriteError(
"Atomic JSON replacement permissions changed and were restored."
)
def _fsync_directory(path: Path) -> None:
flags = os.O_RDONLY
if hasattr(os, "O_DIRECTORY"):
flags |= os.O_DIRECTORY
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
descriptor = os.open(path, flags)
try:
if not stat.S_ISDIR(os.fstat(descriptor).st_mode):
raise AtomicJsonWriteError("JSON output parent path must be a directory.")
os.fsync(descriptor)
finally:
os.close(descriptor)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,204 @@
"""Issuance of role-scoped receipts from same-process installed observations."""
from __future__ import annotations
import base64
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, Mapping
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from .capability_fit import canonical_hash
from .evidence import (
MAX_LOCAL_OBSERVATION_AGE,
OPAQUE_ID_PATTERN,
_catalog_artifact_identities,
_record_integrity_semantics_valid,
canonical_bytes,
canonical_sha256,
normalize_package_name,
)
from govoplan_release.artifact_identity import inspect_python_wheel
def issue_installer_receipt(
*,
assessment: Mapping[str, Any],
catalog: Mapping[str, Any],
installed_evidence: dict[str, Any],
receipt_id: str,
key_id: str,
private_key: Ed25519PrivateKey,
consumed_artifacts: Mapping[str, Path | str] | None = None,
issued_at: datetime | None = None,
same_process_observation: bool = False,
) -> dict[str, Any]:
"""Sign a receipt only for a fresh observation collected by this process.
File-based evidence is intentionally not an admitted issuance input. The CLI
collector calls this function directly with its in-memory observation.
"""
if same_process_observation is not True:
raise ValueError("installer receipts require a same-process observation")
if OPAQUE_ID_PATTERN.fullmatch(receipt_id) is None:
raise ValueError("receipt ID must be a bounded opaque ID")
if OPAQUE_ID_PATTERN.fullmatch(key_id) is None:
raise ValueError("installer signing key ID must be a bounded opaque ID")
if not isinstance(consumed_artifacts, Mapping) or not consumed_artifacts:
raise ValueError("installer receipt issuance requires exact consumed wheels")
release = catalog.get("release")
raw_artifacts = release.get("artifacts") if isinstance(release, Mapping) else None
catalog_artifacts, artifact_findings = _catalog_artifact_identities(raw_artifacts)
if artifact_findings:
raise ValueError("signed catalog artifact manifest is invalid")
evidence_rows = installed_evidence.get("artifacts")
if not isinstance(evidence_rows, list) or not evidence_rows:
raise ValueError("installed evidence has no artifact observations")
expected_packages = _expected_packages(assessment=assessment, catalog=catalog)
receipt_rows: list[dict[str, Any]] = []
seen: set[str] = set()
for artifact in evidence_rows:
if not isinstance(artifact, dict):
raise ValueError("installed evidence contains a malformed artifact")
package_name = normalize_package_name(str(artifact.get("package_name") or ""))
package_version = str(artifact.get("package_version") or "")
if package_name in seen:
raise ValueError("installed evidence contains duplicate packages")
seen.add(package_name)
catalog_artifact = catalog_artifacts.get(package_name)
record = artifact.get("record_integrity")
installed_payload = (
record.get("installed_payload_identity")
if isinstance(record, Mapping)
else None
)
if (
catalog_artifact is None
or catalog_artifact.get("package_version") != package_version
or not isinstance(record, Mapping)
or record.get("status") != "verified"
or not _record_integrity_semantics_valid(record)
or not isinstance(installed_payload, Mapping)
):
raise ValueError(
"installed payload is not a verified match for a catalog artifact"
)
artifact_path = consumed_artifacts.get(package_name)
if artifact_path is None:
raise ValueError("every installed package requires its exact consumed wheel")
consumed = inspect_python_wheel(artifact_path)
if (
consumed.package_name != package_name
or consumed.package_version != package_version
or consumed.archive_sha256 != catalog_artifact.get("archive_sha256")
or consumed.catalog_payload().get("installed_payload")
!= catalog_artifact.get("installed_payload")
or record.get("artifact_payload_identity")
!= catalog_artifact.get("installed_payload")
):
raise ValueError(
"consumed wheel or observed wheel payload differs from the signed catalog identity"
)
receipt_rows.append(
{
"package_name": package_name,
"package_version": package_version,
"catalog_archive_sha256": catalog_artifact["archive_sha256"],
"installed_payload": dict(installed_payload),
}
)
if seen != expected_packages:
raise ValueError(
"installed evidence must contain exactly the enabled assessed catalog packages"
)
if set(consumed_artifacts) != expected_packages:
raise ValueError("consumed wheel set must exactly match enabled packages")
receipt_rows.sort(key=lambda item: (item["package_name"], item["package_version"]))
observed_at = issued_at or datetime.now(tz=UTC)
if observed_at.tzinfo is None:
raise ValueError("installer receipt issuance time must include a timezone")
collected_at = _datetime(installed_evidence.get("collected_at"))
if (
collected_at is None
or observed_at < collected_at
or observed_at - collected_at > MAX_LOCAL_OBSERVATION_AGE
):
raise ValueError(
"installer receipt issuance must follow live collection within five minutes"
)
assessment_release = assessment.get("release")
assessment_release_ref = (
assessment_release.get("ref")
if isinstance(assessment_release, Mapping)
else None
)
receipt: dict[str, Any] = {
"$schema": "./installer-receipt.schema.json",
"schema_version": "0.1.0",
"evidence_kind": "govoplan.installer-receipt",
"receipt_id": receipt_id,
"assessment_id": assessment.get("assessment_id"),
"assessment_release": assessment_release_ref,
"installed_evidence_sha256": canonical_sha256(installed_evidence),
"catalog": {
"channel": catalog.get("channel"),
"sequence": catalog.get("sequence"),
"sha256": canonical_hash(catalog),
},
"issued_at": observed_at.astimezone(UTC).isoformat().replace("+00:00", "Z"),
"artifacts": receipt_rows,
}
receipt["signatures"] = [
{
"algorithm": "ed25519",
"key_id": key_id,
"value": base64.b64encode(
private_key.sign(canonical_bytes(receipt))
).decode("ascii"),
}
]
return receipt
def _expected_packages(
*, assessment: Mapping[str, Any], catalog: Mapping[str, Any]
) -> set[str]:
entries: dict[str, Mapping[str, Any]] = {}
core = catalog.get("core_release")
if isinstance(core, Mapping):
entries["core"] = core
modules = catalog.get("modules")
if isinstance(modules, list):
for entry in modules:
if isinstance(entry, Mapping) and isinstance(entry.get("module_id"), str):
entries[str(entry["module_id"])] = entry
expected: set[str] = set()
composition = assessment.get("composition")
if not isinstance(composition, list):
raise ValueError("assessment composition is unavailable")
for component in composition:
if not isinstance(component, Mapping) or component.get("enabled") is not True:
continue
entry = entries.get(str(component.get("module_id") or ""))
package = entry.get("python_package") if isinstance(entry, Mapping) else None
if not isinstance(package, str):
raise ValueError("enabled assessment component has no catalog package")
normalized = normalize_package_name(package)
if normalized in expected:
raise ValueError("enabled assessment packages are ambiguous")
expected.add(normalized)
if not expected:
raise ValueError("assessment has no enabled catalog packages")
return expected
def _datetime(value: object) -> datetime | None:
if not isinstance(value, str):
return None
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return None
return parsed.astimezone(UTC) if parsed.tzinfo is not None else None
+170
View File
@@ -0,0 +1,170 @@
#!/usr/bin/env python3
"""Collect live installed payloads and issue an independently signed receipt."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
META_ROOT = Path(__file__).resolve().parents[2]
for tools_root in (META_ROOT / "tools" / "assessments", META_ROOT / "tools" / "release"):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from cryptography.hazmat.primitives import serialization # noqa: E402
from cryptography.hazmat.primitives.asymmetric.ed25519 import ( # noqa: E402
Ed25519PrivateKey,
)
from govoplan_assessment import collect_installed_composition # noqa: E402
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
from govoplan_assessment.capability_fit import ( # noqa: E402
canonical_hash,
review_capability_fit,
)
from govoplan_assessment.evidence import validate_payload # noqa: E402
from govoplan_assessment.installer_receipt import ( # noqa: E402
issue_installer_receipt,
)
MAX_INPUT_BYTES = 16 * 1024 * 1024
MAX_OUTPUT_BYTES = 4 * 1024 * 1024
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--assessment", type=Path, required=True)
parser.add_argument("--assessment-schema", type=Path, default=META_ROOT / "docs" / "capability-fit.schema.json")
parser.add_argument("--catalog", type=Path, required=True)
parser.add_argument("--keyring", type=Path, required=True)
parser.add_argument("--trusted-keyring", type=Path, required=True)
parser.add_argument("--receipt-id", required=True)
parser.add_argument("--signing-key", required=True, metavar="KEY_ID=/PATH/PRIVATE.pem")
parser.add_argument(
"--python-artifact",
action="append",
default=[],
required=True,
metavar="PACKAGE=/PATH/TO/CONSUMED.whl",
help="Exact wheel consumed by this installer; repeat for every package.",
)
parser.add_argument("--evidence-output", type=Path, required=True)
parser.add_argument("--receipt-output", type=Path, required=True)
parser.add_argument("--installed-evidence-schema", type=Path, default=META_ROOT / "docs" / "installed-composition-evidence.schema.json")
parser.add_argument("--receipt-schema", type=Path, default=META_ROOT / "docs" / "installer-receipt.schema.json")
args = parser.parse_args(argv)
if args.evidence_output.resolve() == args.receipt_output.resolve():
parser.error("evidence and receipt output paths must differ")
assessment = read_object(args.assessment, label="assessment")
assessment_schema = read_object(args.assessment_schema, label="assessment schema")
catalog = read_object(args.catalog, label="catalog")
keyring = read_object(args.keyring, label="published keyring")
trusted_keyring = read_object(args.trusted_keyring, label="trusted keyring")
evidence_schema = read_object(args.installed_evidence_schema, label="installed evidence schema")
receipt_schema = read_object(args.receipt_schema, label="installer receipt schema")
release = catalog.get("release")
if not isinstance(release, dict) or release.get("keyring_sha256") != canonical_hash(keyring):
parser.error("catalog does not pin the supplied published keyring")
evidence = collect_installed_composition(assessment=assessment)
if validate_payload(payload=evidence, schema=evidence_schema):
parser.error("live installed observation does not satisfy its bounded schema")
report = review_capability_fit(
assessment=assessment,
schema=assessment_schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=trusted_keyring,
workspace_root=None,
installed_evidence=evidence,
installed_evidence_schema=evidence_schema,
installed_evidence_mode="direct_local",
)
required_scopes = (
"catalog_signature_and_keyring",
"catalog_signature_and_trusted_keyring",
"published_keyring_hash",
"release_metadata",
"installed_artifacts",
"installed_record_integrity",
)
if any(report["proof_scope"].get(scope, {}).get("valid") is not True for scope in required_scopes):
parser.error("trusted catalog and live installed-composition checks must pass before receipt issuance")
key_id, private_key = read_signing_key(args.signing_key)
consumed_artifacts = parse_package_paths(tuple(args.python_artifact))
receipt = issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id=args.receipt_id,
key_id=key_id,
private_key=private_key,
consumed_artifacts=consumed_artifacts,
same_process_observation=True,
)
if validate_payload(payload=receipt, schema=receipt_schema):
parser.error("issued installer receipt does not satisfy its bounded schema")
write_object(args.evidence_output, evidence, label="installed evidence")
write_object(args.receipt_output, receipt, label="installer receipt")
print(json.dumps({"receipt_id": args.receipt_id, "evidence_output": str(args.evidence_output), "receipt_output": str(args.receipt_output)}, sort_keys=True))
return 0
def read_object(path: Path, *, label: str) -> dict[str, object]:
try:
if path.stat().st_size > MAX_INPUT_BYTES:
raise ValueError("input exceeds size limit")
encoded = path.read_bytes()
payload = json.loads(encoded.decode("utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError, ValueError) as exc:
raise SystemExit(f"Could not read {label}") from exc
if not isinstance(payload, dict):
raise SystemExit(f"{label.capitalize()} must be a JSON object")
return payload
def read_signing_key(value: str) -> tuple[str, Ed25519PrivateKey]:
key_id, separator, path_text = value.partition("=")
if not separator or not key_id or not path_text:
raise SystemExit("--signing-key must use KEY_ID=/path/to/private.pem")
try:
key = serialization.load_pem_private_key(Path(path_text).expanduser().read_bytes(), password=None)
except (OSError, ValueError) as exc:
raise SystemExit("Could not read installer signing key") from exc
if not isinstance(key, Ed25519PrivateKey):
raise SystemExit("Installer signing key must be Ed25519")
return key_id, key
def parse_package_paths(values: tuple[str, ...]) -> dict[str, Path]:
result: dict[str, Path] = {}
for value in values:
package, separator, path_text = value.partition("=")
package = package.strip()
path_text = path_text.strip()
if not separator or not package or not path_text or package in result:
raise SystemExit(
"--python-artifact must uniquely use PACKAGE=/path/to/consumed.whl"
)
result[package] = Path(path_text).expanduser()
return result
def write_object(path: Path, payload: dict[str, object], *, label: str) -> None:
try:
atomic_write_json(path, payload, max_bytes=MAX_OUTPUT_BYTES)
except AtomicJsonWriteError as exc:
raise SystemExit(f"Could not securely write {label}") from exc
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,328 @@
#!/usr/bin/env python3
"""Exercise connector -> datasource -> dataflow publication capabilities."""
from __future__ import annotations
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_connectors.backend.db.models import ConnectorTabularSource
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.dataflows import (
DataflowPublicationTarget,
DataflowRunRequest,
dataflow_run_lifecycle,
)
from govoplan_core.core.automation import AutomationPrincipalResolution
from govoplan_core.core.access import (
CAPABILITY_AUTH_AUTOMATION_PRINCIPAL_PROVIDER,
)
from govoplan_core.core.datasources import (
DatasourceReadRequest,
datasource_catalogue,
datasource_lifecycle,
datasource_publication,
)
from govoplan_core.core.modules import ModuleContext
from govoplan_core.core.tabular_sources import (
TabularSnapshotInput,
tabular_snapshot_writer,
)
from govoplan_core.db.base import Base
from govoplan_core.server.registry import build_platform_registry
from govoplan_dataflow.backend.schemas import (
GraphEdge,
GraphNode,
GraphPosition,
PipelineGraph,
PipelineCreateRequest,
PipelinePreviewRequest,
)
from govoplan_dataflow.backend.db.models import (
DataflowPipeline,
DataflowPipelineRevision,
DataflowRun,
)
from govoplan_dataflow.backend.service import create_pipeline, preview_pipeline
from govoplan_dataflow.backend.run_worker import SqlDataflowRunWorker
from govoplan_datasources.backend.db.models import (
DatasourceMaterializationRecord,
DatasourcePayloadRecord,
DatasourcePayloadRowRecord,
DatasourcePublicationRecord,
DatasourceRecord,
DatasourceStageRecord,
)
def main() -> int:
registry = build_platform_registry(
(
"connectors",
"datasources",
"dataflow",
"workflow_engine",
"workflow",
)
)
registry.configure_capability_context(
ModuleContext(registry=registry, settings=object())
)
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(
engine,
tables=[
ConnectorTabularSource.__table__,
DatasourceRecord.__table__,
DatasourcePayloadRecord.__table__,
DatasourcePayloadRowRecord.__table__,
DatasourceMaterializationRecord.__table__,
DatasourceStageRecord.__table__,
DatasourcePublicationRecord.__table__,
DataflowPipeline.__table__,
DataflowPipelineRevision.__table__,
DataflowRun.__table__,
],
)
session_factory = sessionmaker(bind=engine)
with session_factory() as session:
principal = _principal()
writer = tabular_snapshot_writer(registry)
lifecycle = datasource_lifecycle(registry)
catalogue = datasource_catalogue(registry)
publisher = datasource_publication(registry)
runner = dataflow_run_lifecycle(registry)
if (
writer is None
or lifecycle is None
or catalogue is None
or publisher is None
or runner is None
):
raise RuntimeError("Datasource composition capabilities are incomplete.")
origin = writer.create_snapshot(
session,
principal,
snapshot=TabularSnapshotInput(
name="Monthly cases",
source_name="connector_monthly_cases",
rows=(
{"id": 1, "amount": 5},
{"id": 2, "amount": 15},
),
),
)
datasource = lifecycle.register_origin(
session,
principal,
origin_ref=origin.ref,
name="Monthly cases cache",
source_name="monthly_cases",
mode="cached",
)
result = preview_pipeline(
session,
tenant_id="tenant-1",
actor_id="account-1",
payload=PipelinePreviewRequest(
graph=_graph(
datasource_ref=datasource.ref,
fingerprint=datasource.fingerprint,
),
row_limit=100,
),
principal=principal,
registry=registry,
)
expected_rows = [
{"id": 1, "amount": 5},
{"id": 2, "amount": 15},
]
if result.status != "succeeded":
raise RuntimeError(f"Dataflow preview failed: {result.diagnostics}")
if result.rows != expected_rows:
raise RuntimeError(f"Unexpected Dataflow rows: {result.rows!r}")
if result.source_fingerprints[0]["source_ref"] != datasource.ref:
raise RuntimeError("Dataflow lineage did not retain the datasource reference.")
pipeline = create_pipeline(
session,
tenant_id="tenant-1",
actor_id="account-1",
payload=PipelineCreateRequest(
name="Monthly case output",
status="active",
graph=_graph(
datasource_ref=datasource.ref,
fingerprint=datasource.fingerprint,
),
editor_mode="graph",
),
)
run_request = DataflowRunRequest(
pipeline_ref=f"pipeline:{pipeline.id}",
revision=1,
idempotency_key="composition-run-1",
publication=DataflowPublicationTarget(
name="Monthly case result",
source_name="monthly_case_result",
freeze=True,
frozen_label="Composition evidence",
),
)
published = runner.start_run(
session,
principal,
request=run_request,
)
replayed = runner.start_run(
session,
principal,
request=run_request,
)
if published.status != "queued":
raise RuntimeError(
f"Dataflow run was not queued: {published.status}"
)
worker = SqlDataflowRunWorker(
registry=_AutomationRegistry(registry, principal)
)
worker_result = worker.dispatch_pending(
session,
worker_id="composition-worker",
)
if worker_result["succeeded"] != 1:
raise RuntimeError(
f"Dataflow worker failed: {worker_result!r}"
)
completed = runner.get_run(
session,
principal,
run_ref=published.ref,
)
if completed is None:
raise RuntimeError("Dataflow run evidence disappeared.")
published = completed
if published.status != "succeeded":
raise RuntimeError(f"Dataflow publication failed: {published.error}")
if replayed.ref != published.ref or not replayed.replayed:
raise RuntimeError("Dataflow run idempotency did not replay the prior run.")
if (
not published.output_datasource_ref
or not published.output_materialization_ref
):
raise RuntimeError("Dataflow publication did not retain output references.")
output = catalogue.read_datasource(
session,
principal,
request=DatasourceReadRequest(
datasource_ref=published.output_datasource_ref,
),
)
if list(output.rows) != expected_rows:
raise RuntimeError(
f"Unexpected published Dataflow rows: {list(output.rows)!r}"
)
if (
output.materialization is None
or output.materialization.ref != published.output_materialization_ref
or output.materialization.frozen_at is None
):
raise RuntimeError(
"Published Datasource materialization is not pinned and frozen."
)
engine.dispose()
print(
"Connector -> Datasources -> pinned Dataflow publication composition passed."
)
return 0
class _AutomationProvider:
def __init__(self, principal: ApiPrincipal) -> None:
self.principal = principal
def resolve_automation_principal(self, _session, *, request):
return AutomationPrincipalResolution(
allowed=True,
principal=self.principal,
granted_scopes=request.grant_scopes,
provenance={"status": "composition_recheck"},
)
class _AutomationRegistry:
def __init__(self, registry, principal: ApiPrincipal) -> None:
self.registry = registry
self.provider = _AutomationProvider(principal)
def has_capability(self, name: str) -> bool:
return (
name == CAPABILITY_AUTH_AUTOMATION_PRINCIPAL_PROVIDER
or self.registry.has_capability(name)
)
def capability(self, name: str):
if name == CAPABILITY_AUTH_AUTOMATION_PRINCIPAL_PROVIDER:
return self.provider
return self.registry.capability(name)
def _principal() -> ApiPrincipal:
return ApiPrincipal(
principal=PrincipalRef(
account_id="account-1",
membership_id="membership-1",
tenant_id="tenant-1",
scopes=frozenset(
{
"connectors:source:read",
"connectors:source:write",
"datasources:catalogue:read",
"datasources:source:write",
"datasources:stage:write",
"dataflow:pipeline:run",
}
),
),
account=object(),
user=object(),
)
def _graph(*, datasource_ref: str, fingerprint: str) -> PipelineGraph:
return PipelineGraph(
nodes=[
GraphNode(
id="source",
type="source.reference",
label="Cases",
position=GraphPosition(x=0, y=0),
config={
"source_ref": datasource_ref,
"source_name": "monthly_cases",
"expected_fingerprint": fingerprint,
"consistency": "current",
},
),
GraphNode(
id="output",
type="output",
label="Output",
position=GraphPosition(x=200, y=0),
config={},
),
],
edges=[
GraphEdge(
id="source-output",
source="source",
target="output",
)
],
)
if __name__ == "__main__":
raise SystemExit(main())
+52 -1
View File
@@ -24,11 +24,28 @@ export NPM_CONFIG_USERCONFIG="$NPM_USERCONFIG"
export GOVOPLAN_NPM_USERCONFIG="$NPM_USERCONFIG" export GOVOPLAN_NPM_USERCONFIG="$NPM_USERCONFIG"
unset npm_config_tmp NPM_CONFIG_TMP unset npm_config_tmp NPM_CONFIG_TMP
# Validate the current sibling checkouts even when a newly added module has not
# yet been installed into an existing development virtualenv.
SOURCE_PYTHONPATH=""
for source_dir in "$META_ROOT"/../govoplan*/src; do
[ -d "$source_dir" ] || continue
SOURCE_PYTHONPATH="${SOURCE_PYTHONPATH:+$SOURCE_PYTHONPATH:}$source_dir"
done
export PYTHONPATH="${SOURCE_PYTHONPATH}${PYTHONPATH:+:$PYTHONPATH}"
cd "$ROOT" cd "$ROOT"
GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh" GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh"
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact "$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
cd "$META_ROOT"
"$PYTHON" -m unittest tests.test_deployment_installer
"$PYTHON" -m unittest tests.test_capability_fit_evidence
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
"$PYTHON" -m unittest tests.test_institutional_governance_journey
"$PYTHON" -m unittest tests.test_institutional_service_journey
cd "$ROOT"
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
import ast import ast
@@ -69,6 +86,25 @@ PY
"$PYTHON" -c 'import govoplan_core.db.bootstrap; import govoplan_access.backend.admin.service; import govoplan_addresses.backend.manifest; import govoplan_files.backend.router; import govoplan_mail.backend.sending.imap; print("targeted backend imports passed")' "$PYTHON" -c 'import govoplan_core.db.bootstrap; import govoplan_access.backend.admin.service; import govoplan_addresses.backend.manifest; import govoplan_files.backend.router; import govoplan_mail.backend.sending.imap; print("targeted backend imports passed")'
"$META_ROOT/tools/checks/check_dependency_boundaries.py" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
"$PYTHON" -m unittest tests.test_module_system "$PYTHON" -m unittest tests.test_module_system
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow-engine/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms-runtime/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-cases/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-committee/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-voting/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-approvals/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count "$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
@@ -77,6 +113,21 @@ cd "$ROOT/webui"
"$NPM" run test:module-capabilities "$NPM" run test:module-capabilities
"$NPM" run test:module-permutations "$NPM" run test:module-permutations
cd /mnt/DATA/git/govoplan-dataflow/webui
"$NPM" run test:structure
cd /mnt/DATA/git/govoplan-datasources/webui
"$NPM" run typecheck
cd /mnt/DATA/git/govoplan-workflow/webui
"$NPM" run typecheck
cd /mnt/DATA/git/govoplan-dashboard/webui
"$NPM" run test:dashboard-layout
cd /mnt/DATA/git/govoplan-postbox/webui
"$NPM" run test:ui-structure
cd /mnt/DATA/git/govoplan-mail/webui cd /mnt/DATA/git/govoplan-mail/webui
"$NPM" run test:mail-ui "$NPM" run test:mail-ui
+104 -1
View File
@@ -6,12 +6,17 @@ from __future__ import annotations
import argparse import argparse
import importlib import importlib
import json import json
import re
import sys import sys
import tomllib import tomllib
from pathlib import Path from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[2] META_ROOT = Path(__file__).resolve().parents[2]
MODULE_NAME_PATTERN = re.compile(
r"[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*"
)
REQUIRED_DOCUMENTATION_TYPES = frozenset({"admin", "user"})
def main() -> int: def main() -> int:
@@ -22,6 +27,14 @@ def main() -> int:
default=None, default=None,
help="Directory containing the GovOPlaN repositories. Defaults to repositories.json default_parent.", help="Directory containing the GovOPlaN repositories. Defaults to repositories.json default_parent.",
) )
parser.add_argument(
"--require-architecture",
action="store_true",
help=(
"Reject every module that has not adopted the versioned architecture "
"declaration. GovOPlaN release and focused checks enable this gate."
),
)
args = parser.parse_args() args = parser.parse_args()
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8")) catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
@@ -56,8 +69,15 @@ def main() -> int:
errors: list[str] = [] errors: list[str] = []
for repository_name, source_root, manifest_path in manifest_sources: for repository_name, source_root, manifest_path in manifest_sources:
module_name = ".".join(manifest_path.relative_to(source_root).with_suffix("").parts) module_name = ".".join(manifest_path.relative_to(source_root).with_suffix("").parts)
if MODULE_NAME_PATTERN.fullmatch(module_name) is None:
errors.append(
f"{repository_name}: manifest path does not map to a safe Python module name: {module_name!r}"
)
continue
try: try:
loaded_module = importlib.import_module(module_name) # Module names are derived from repository-owned manifest paths and
# constrained to canonical Python identifiers immediately above.
loaded_module = importlib.import_module(module_name) # nosemgrep: python.lang.security.audit.non-literal-import.non-literal-import
get_manifest = getattr(loaded_module, "get_manifest") get_manifest = getattr(loaded_module, "get_manifest")
manifest = get_manifest() manifest = get_manifest()
except Exception as exc: # pragma: no cover - rendered as a check failure except Exception as exc: # pragma: no cover - rendered as a check failure
@@ -79,6 +99,36 @@ def main() -> int:
) )
continue continue
documented_types = {
documentation_type
for topic in manifest.documentation
for documentation_type in topic.documentation_types
}
missing_documentation_types = sorted(REQUIRED_DOCUMENTATION_TYPES - documented_types)
if missing_documentation_types:
errors.append(
f"{repository_name}: module {manifest.id!r} is missing static documentation for "
f"{', '.join(missing_documentation_types)}; add manifest DocumentationTopic entries "
"even when runtime documentation providers are registered"
)
continue
repository_root = manifest_path.parents[3]
if manifest.architecture is None:
if args.require_architecture:
errors.append(
f"{repository_name}: module {manifest.id!r} has no architecture declaration"
)
continue
else:
errors.extend(
_architecture_evidence_errors(
repository_name=repository_name,
repository_root=repository_root,
manifest=manifest,
)
)
manifests.append(manifest) manifests.append(manifest)
if errors: if errors:
@@ -98,6 +148,11 @@ def main() -> int:
f"Manifest registry check passed: {len(snapshot.manifests)} manifests " f"Manifest registry check passed: {len(snapshot.manifests)} manifests "
f"from {len(manifest_sources)} source files." f"from {len(manifest_sources)} source files."
) )
declared = sum(manifest.architecture is not None for manifest in manifests)
print(
f"Architecture declaration coverage: {declared}/{len(manifests)} modules "
f"({(declared / len(manifests) * 100):.1f}%)."
)
return 0 return 0
@@ -109,5 +164,53 @@ def _module_entry_points(pyproject_path: Path) -> dict[str, str]:
return {str(name): str(target) for name, target in values.items()} return {str(name): str(target) for name, target in values.items()}
def _architecture_evidence_errors(
*,
repository_name: str,
repository_root: Path,
manifest: object,
) -> list[str]:
architecture = getattr(manifest, "architecture", None)
if architecture is None:
return []
errors: list[str] = []
references = [
(f"{item.kind} evidence", item.reference)
for item in architecture.evidence
]
for category in ("migration", "upgrade", "recovery", "security", "operations"):
references.extend(
(f"{category} documentation", reference)
for reference in getattr(architecture.documentation, category)
)
for label, reference in references:
if not _looks_like_repository_reference(reference):
continue
candidate = (repository_root / reference).resolve()
try:
candidate.relative_to(repository_root.resolve())
except ValueError:
errors.append(
f"{repository_name}: {label} escapes the repository: {reference!r}"
)
continue
if not candidate.exists():
errors.append(
f"{repository_name}: {label} does not exist: {reference!r}"
)
return errors
def _looks_like_repository_reference(reference: str) -> bool:
normalized = reference.strip()
if not normalized or "://" in normalized:
return False
return (
"/" in normalized
or normalized.startswith("README")
or normalized.endswith((".md", ".py", ".json", ".yaml", ".yml"))
)
if __name__ == "__main__": if __name__ == "__main__":
raise SystemExit(main()) raise SystemExit(main())
+11 -1
View File
@@ -19,9 +19,19 @@ fi
cd "$ROOT" cd "$ROOT"
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact "$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
"$PYTHON" -m unittest tests.test_module_system tests.test_access_contracts "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \
--requirements "$META_ROOT/requirements-release.txt"
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" core-tests \
--core-root "$ROOT"
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py" "$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
cd "$META_ROOT"
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
PYTHONPATH="$META_ROOT/../govoplan-portal/src:$META_ROOT/../govoplan-forms/src:$META_ROOT/../govoplan-forms-runtime/src:$META_ROOT/../govoplan-cases/src:$ROOT/src${PYTHONPATH:+:$PYTHONPATH}" \
"$PYTHON" -m unittest tests.test_institutional_service_journey
PYTHONPATH="$META_ROOT/../govoplan-portal/src:$META_ROOT/../govoplan-cases/src:$META_ROOT/../govoplan-committee/src:$META_ROOT/../govoplan-services/src:$META_ROOT/../govoplan-parties/src:$META_ROOT/../govoplan-mandates/src:$META_ROOT/../govoplan-decisions/src:$ROOT/src${PYTHONPATH:+:$PYTHONPATH}" \
"$PYTHON" -m unittest tests.test_institutional_governance_journey
cd "$ROOT/webui" cd "$ROOT/webui"
PATH="$ROOT/webui/node_modules/.bin:$NODE_BIN:$PATH" "$NPM" run test:module-capabilities PATH="$ROOT/webui/node_modules/.bin:$NODE_BIN:$PATH" "$NPM" run test:module-capabilities
PATH="$ROOT/webui/node_modules/.bin:$NODE_BIN:$PATH" "$NPM" run test:module-permutations PATH="$ROOT/webui/node_modules/.bin:$NODE_BIN:$PATH" "$NPM" run test:module-permutations
+48 -70
View File
@@ -32,9 +32,8 @@ with (Path.cwd() / "pyproject.toml").open("rb") as handle:
print(tomllib.load(handle)["project"]["version"]) print(tomllib.load(handle)["project"]["version"])
PY PY
)" )"
RELEASE_TAG="${GOVOPLAN_RELEASE_TAG:-v$RELEASE_VERSION}"
export RELEASE_TAG RELEASE_VERSION export RELEASE_VERSION
export META_ROOT export META_ROOT
export GOVOPLAN_CORE_SOURCE_ROOT="$ROOT" export GOVOPLAN_CORE_SOURCE_ROOT="$ROOT"
export GOVOPLAN_MIGRATION_TRACK=release export GOVOPLAN_MIGRATION_TRACK=release
@@ -61,24 +60,6 @@ retry() {
done done
} }
run_discovered_tests() {
local suite_dir="$1"
local status
if ! find "$suite_dir" -type f -name 'test*.py' -print -quit | grep -q .; then
echo "No unittest test files found under $suite_dir; skipping."
return 0
fi
set +e
"$PYTHON" -m unittest discover -s "$suite_dir"
status=$?
set -e
if [[ "$status" == 5 ]]; then
echo "No unittest tests discovered under $suite_dir; skipping."
return 0
fi
return "$status"
}
install_cloned_webui_dependencies() { install_cloned_webui_dependencies() {
local webui_dir="$1" local webui_dir="$1"
if [[ ! -f "$webui_dir/package.json" ]]; then if [[ ! -f "$webui_dir/package.json" ]]; then
@@ -91,7 +72,32 @@ install_cloned_webui_dependencies() {
) )
} }
release_tag_for_package() {
local package_name="$1"
"$PYTHON" - "$META_ROOT/requirements-release.txt" "$package_name" <<'PY'
from __future__ import annotations
from pathlib import Path
import re
import sys
requirements_path = Path(sys.argv[1])
package_name = sys.argv[2]
pattern = re.compile(
rf"^{re.escape(package_name)}\s+@\s+git\+ssh://.+?\.git@(v[0-9]+\.[0-9]+\.[0-9]+)$"
)
for line in requirements_path.read_text(encoding="utf-8").splitlines():
match = pattern.match(line.strip())
if match:
print(match.group(1))
raise SystemExit(0)
raise SystemExit(f"No immutable release reference found for {package_name}")
PY
}
run_step "Validate release refs and installed package metadata" run_step "Validate release refs and installed package metadata"
"$PYTHON" "$META_ROOT/tools/checks/check-version-alignment.py" --release-composition
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact "$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
from __future__ import annotations from __future__ import annotations
@@ -108,7 +114,6 @@ root = Path.cwd()
release_requirements = Path(os.environ["META_ROOT"]) / "requirements-release.txt" release_requirements = Path(os.environ["META_ROOT"]) / "requirements-release.txt"
release_webui = root / "webui" / "package.release.json" release_webui = root / "webui" / "package.release.json"
expected_version = os.environ["RELEASE_VERSION"] expected_version = os.environ["RELEASE_VERSION"]
expected_tag = os.environ["RELEASE_TAG"]
errors: list[str] = [] errors: list[str] = []
python_requirements: list[tuple[str, str, str]] = [] python_requirements: list[tuple[str, str, str]] = []
@@ -121,8 +126,6 @@ if not python_requirements:
errors.append("No GovOPlaN git+ssh release requirements found.") errors.append("No GovOPlaN git+ssh release requirements found.")
for package_name, repo_url, tag in python_requirements: for package_name, repo_url, tag in python_requirements:
if tag != expected_tag:
errors.append(f"{package_name} release requirement uses {tag!r}, expected {expected_tag!r}.")
version = metadata.version(package_name) version = metadata.version(package_name)
if version != tag.removeprefix("v"): if version != tag.removeprefix("v"):
errors.append(f"{package_name} installed version {version!r} does not match {tag!r}.") errors.append(f"{package_name} installed version {version!r} does not match {tag!r}.")
@@ -146,15 +149,18 @@ for package_name, spec in sorted(webui_dependencies.items()):
if not package_name.startswith("@govoplan/"): if not package_name.startswith("@govoplan/"):
continue continue
tag = spec.rsplit("#", 1)[-1] if "#" in spec else "" tag = spec.rsplit("#", 1)[-1] if "#" in spec else ""
if tag != expected_tag: tag_match = re.fullmatch(r"v([0-9]+\.[0-9]+\.[0-9]+)", tag)
errors.append(f"{package_name} release dependency uses {tag!r}, expected {expected_tag}.") if tag_match is None:
errors.append(f"{package_name} release dependency has invalid immutable tag {tag!r}.")
continue
dependency_version = tag_match.group(1)
package_json = root / "webui" / "node_modules" / package_name / "package.json" package_json = root / "webui" / "node_modules" / package_name / "package.json"
if not package_json.exists(): if not package_json.exists():
errors.append(f"{package_name} is missing from release node_modules.") errors.append(f"{package_name} is missing from release node_modules.")
continue continue
installed = json.loads(package_json.read_text(encoding="utf-8")) installed = json.loads(package_json.read_text(encoding="utf-8"))
if installed.get("version") != expected_version: if installed.get("version") != dependency_version:
errors.append(f"{package_name} installed version {installed.get('version')!r}, expected {expected_version!r}.") errors.append(f"{package_name} installed version {installed.get('version')!r}, expected {dependency_version!r} from {tag!r}.")
shared_peers = ("lucide-react", "react", "react-dom", "react-router-dom") shared_peers = ("lucide-react", "react", "react-dom", "react-router-dom")
root_peers = release_package.get("peerDependencies", {}) root_peers = release_package.get("peerDependencies", {})
@@ -173,40 +179,14 @@ print(f"Release refs and metadata passed for {len(python_requirements)} Python p
PY PY
run_step "Validate installed module manifests and registry" run_step "Validate installed module manifests and registry"
"$PYTHON" - <<'PY' "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \
from __future__ import annotations --requirements "$META_ROOT/requirements-release.txt"
import sys run_step "Generate release dependency provenance"
from govoplan_core.server.registry import available_module_manifests, build_platform_registry "$PYTHON" "$META_ROOT/tools/release/generate-release-sbom.py" \
--python "$PYTHON" \
expected = { --core-root "$ROOT" \
"access", --output "$WORK_ROOT/govoplan-sbom.cdx.json"
"admin",
"audit",
"calendar",
"campaigns",
"dashboard",
"docs",
"files",
"identity",
"idm",
"mail",
"ops",
"organizations",
"policy",
"tenancy",
}
manifests = available_module_manifests()
missing = sorted(expected - set(manifests))
if missing:
print(f"Missing installed module manifests: {', '.join(missing)}", file=sys.stderr)
raise SystemExit(1)
registry = build_platform_registry(tuple(sorted(expected)))
snapshot = registry.validate()
print("Registry modules:", ", ".join(manifest.id for manifest in snapshot.manifests))
PY
run_step "Run SQLite migration smoke for release modules" run_step "Run SQLite migration smoke for release modules"
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
@@ -245,22 +225,20 @@ PY
run_step "Clone release module test sources" run_step "Clone release module test sources"
for repo in govoplan-mail govoplan-calendar govoplan-campaign; do for repo in govoplan-mail govoplan-calendar govoplan-campaign; do
git clone --depth 1 --branch "$RELEASE_TAG" "git@git.add-ideas.de:add-ideas/${repo}.git" "$WORK_ROOT/$repo" repo_tag="$(release_tag_for_package "$repo")"
git clone --depth 1 --branch "$repo_tag" "git@git.add-ideas.de:GovOPlaN/${repo}.git" "$WORK_ROOT/$repo"
done done
run_step "Run core backend release tests" run_step "Run core backend release tests"
"$PYTHON" -m unittest \ "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" core-tests \
tests.test_module_system \ --core-root "$ROOT"
tests.test_access_contracts \
tests.test_identity_organization_contracts \
tests.test_core_events \
tests.test_policy_contracts
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py" "$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
run_step "Run cloned module backend tests" run_step "Run cloned module backend tests"
run_discovered_tests "$WORK_ROOT/govoplan-mail/tests" for repo in govoplan-mail govoplan-calendar govoplan-campaign; do
run_discovered_tests "$WORK_ROOT/govoplan-calendar/tests" "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" module-tests \
run_discovered_tests "$WORK_ROOT/govoplan-campaign/tests" --module-root "$WORK_ROOT/$repo"
done
run_step "Run core WebUI release tests and build" run_step "Run core WebUI release tests and build"
cd "$ROOT/webui" cd "$ROOT/webui"
+661 -64
View File
@@ -77,6 +77,46 @@ if [[ "$REPORTS_DIR" != /* ]]; then
REPORTS_DIR="$ROOT/$REPORTS_DIR" REPORTS_DIR="$ROOT/$REPORTS_DIR"
fi fi
mkdir -p "$REPORTS_DIR" mkdir -p "$REPORTS_DIR"
REPORTS_DIR="$(cd "$REPORTS_DIR" && pwd -P)"
for flag_name in FAIL_ON_FINDINGS REQUIRE_TOOLS; do
flag_value="${!flag_name}"
if [[ "$flag_value" != "0" && "$flag_value" != "1" ]]; then
echo "$flag_name must be 0 or 1, got: $flag_value" >&2
exit 2
fi
done
# A strict audit is only meaningful when every scanner in the selected mode is
# available. CI must enforce the same coverage even while findings are
# temporarily report-only.
if [[ "$FAIL_ON_FINDINGS" == "1" \
|| "${CI:-false}" == "true" \
|| "${GITEA_ACTIONS:-false}" == "true" ]]; then
REQUIRE_TOOLS=1
fi
declare -a REQUIRED_SCANNERS=(
semgrep
bandit
ruff-security
gitleaks
)
if [[ "$MODE" != "quick" ]]; then
REQUIRED_SCANNERS+=(
trivy
pip-audit
npm-audit
)
fi
if [[ "$MODE" == "full" ]]; then
REQUIRED_SCANNERS+=(
osv-scanner
jscpd
radon
xenon
)
fi
declare -a REPOS=() declare -a REPOS=()
if [[ "$SCOPE" == "govoplan" ]]; then if [[ "$SCOPE" == "govoplan" ]]; then
@@ -118,7 +158,7 @@ for repo in "${REPOS[@]}"; do
done done
safe_name() { safe_name() {
basename "$1" | tr -c 'A-Za-z0-9_.-' '_' printf '%s' "$(basename "$1")" | tr -c 'A-Za-z0-9_.-' '_'
} }
has_tool() { has_tool() {
@@ -128,65 +168,461 @@ has_tool() {
overall_status=0 overall_status=0
finding_status=0 finding_status=0
missing_status=0 missing_status=0
execution_status=0
coverage_complete=true
audit_started_at="$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
audit_completed_at=""
workspace_unchanged="unknown"
audit_toolbox_fingerprint="${SECURITY_AUDIT_TOOLBOX_FINGERPRINT:-direct-host}"
declare -A REPORT_FILES=()
declare -A MACHINE_REPORTS=()
declare -A SCANNER_TERMINAL_STATUSES=()
run_step() { register_report() {
local name="$1" local path="$1"
local report_kind="${2:-text}"
case "$path" in
"$REPORTS_DIR"/*) ;;
*)
echo "Refusing to register an audit report outside $REPORTS_DIR: $path" >&2
return 2
;;
esac
REPORT_FILES["$path"]=1
if [[ "$report_kind" == "machine" ]]; then
MACHINE_REPORTS["$path"]=1
fi
}
prepare_machine_report() {
local path="$1"
register_report "$path" machine || return 2
if [[ -e "$path" || -L "$path" ]]; then
rm -f -- "$path" || return 2
fi
}
run_step_internal() {
local scanner_id="$1"
shift shift
local name="$1"
local exit_contract="$2"
shift 2
echo echo
echo "==> $name" echo "==> $name"
"$@" "$@"
local status=$? local status=$?
if [[ "$status" -ne 0 ]]; then local outcome="passed"
echo "Audit step reported findings or failed: $name (exit $status)" >&2 local scanner_outcome="no-findings"
if [[ "$status" -eq 0 ]]; then
:
elif [[ "$exit_contract" == "findings-exit-one" && "$status" -eq 1 ]]; then
outcome="findings"
scanner_outcome="findings"
echo "Audit step reported findings: $name (exit $status)" >&2
finding_status=1 finding_status=1
if [[ "$FAIL_ON_FINDINGS" == "1" ]]; then if [[ "$FAIL_ON_FINDINGS" == "1" ]]; then
overall_status=1 overall_status=1
fi fi
else
outcome="execution-error"
scanner_outcome="scanner-failure"
echo "Audit step failed to execute successfully: $name (exit $status)" >&2
execution_status=1
overall_status=1
fi fi
printf '%s\t%s\t%s\n' "$name" "$status" "$outcome" >> "$REPORTS_DIR/step-status.tsv"
if [[ -n "$scanner_id" ]]; then
SCANNER_TERMINAL_STATUSES["$scanner_id"]="$scanner_outcome"
if [[ "$scanner_outcome" == "scanner-failure" ]]; then
coverage_complete=false
fi
printf '%s\t%s\t%s\t%s\n' \
"$scanner_id" \
"$name" \
"$status" \
"$scanner_outcome" \
>> "$REPORTS_DIR/scanner-status.tsv"
fi
}
run_step() {
run_step_internal "" "$@"
}
run_scanner_step() {
local scanner_id="$1"
shift
run_step_internal "$scanner_id" "$@"
} }
skip_or_fail_missing() { skip_or_fail_missing() {
local tool="$1" local scanner_id="$1"
local tool="$2"
echo "Skipping $tool: command not found. Use tools/checks/security-audit/run.sh for the containerized toolbox." >&2 echo "Skipping $tool: command not found. Use tools/checks/security-audit/run.sh for the containerized toolbox." >&2
missing_status=1
coverage_complete=false
SCANNER_TERMINAL_STATUSES["$scanner_id"]="skipped"
if [[ "$REQUIRE_TOOLS" == "1" ]]; then if [[ "$REQUIRE_TOOLS" == "1" ]]; then
missing_status=1
overall_status=1 overall_status=1
fi fi
printf '%s\t127\tmissing\n' "$tool" >> "$REPORTS_DIR/step-status.tsv"
printf '%s\t%s\t127\tskipped\n' \
"$scanner_id" \
"$tool" \
>> "$REPORTS_DIR/scanner-status.tsv"
}
validate_scanner_status_contract() {
local scanner_id
local scanner_status
local invalid_status=0
for scanner_id in "${REQUIRED_SCANNERS[@]}"; do
scanner_status="${SCANNER_TERMINAL_STATUSES[$scanner_id]:-}"
case "$scanner_status" in
no-findings|findings) ;;
scanner-failure|skipped)
coverage_complete=false
;;
"")
echo "Required scanner did not record a terminal status: $scanner_id" >&2
coverage_complete=false
invalid_status=2
;;
*)
echo "Required scanner recorded an invalid status: $scanner_id ($scanner_status)" >&2
coverage_complete=false
invalid_status=2
;;
esac
done
return "$invalid_status"
} }
write_manifest() { write_manifest() {
{ local -a present_reports=()
printf '{\n' local -a expected_reports=()
printf ' "mode": "%s",\n' "$MODE" local path
printf ' "scope": "%s",\n' "$SCOPE" while IFS= read -r path; do
printf ' "fail_on_findings": "%s",\n' "$FAIL_ON_FINDINGS" expected_reports+=("${path#"$REPORTS_DIR"/}")
printf ' "repositories": [\n' if [[ -f "$path" ]]; then
local index=0 present_reports+=("${path#"$REPORTS_DIR"/}")
for repo in "${REPOS[@]}"; do fi
[[ "$index" -gt 0 ]] && printf ',\n' done < <(printf '%s\n' "${!REPORT_FILES[@]}" | sort)
printf ' "%s"' "$repo"
index=$((index + 1)) python3 - \
done "$REPORTS_DIR/manifest.json" \
printf '\n ]\n' "$MODE" \
printf '}\n' "$SCOPE" \
} > "$REPORTS_DIR/manifest.json" "$FAIL_ON_FINDINGS" \
"$audit_started_at" \
"$audit_completed_at" \
"$workspace_unchanged" \
"$overall_status" \
"$finding_status" \
"$execution_status" \
"$missing_status" \
"$coverage_complete" \
"$audit_toolbox_fingerprint" \
"${#REQUIRED_SCANNERS[@]}" \
"${REQUIRED_SCANNERS[@]}" \
"${#REPOS[@]}" \
"${REPOS[@]}" \
"${#present_reports[@]}" \
"${present_reports[@]}" \
"${expected_reports[@]}" <<'PY'
from __future__ import annotations
import json
from pathlib import Path
import sys
(
manifest_path,
mode,
scope,
fail_on_findings,
started_at,
completed_at,
workspace_unchanged,
overall_status,
finding_status,
execution_status,
missing_status,
coverage_complete,
toolbox_fingerprint,
*remaining,
) = sys.argv[1:]
required_scanner_count = int(remaining[0])
required_scanners = remaining[1 : required_scanner_count + 1]
remaining = remaining[required_scanner_count + 1 :]
repository_count = remaining[0]
remaining = remaining[1:]
repo_count = int(repository_count)
repositories = remaining[:repo_count]
remaining = remaining[repo_count:]
report_count = int(remaining[0])
reports = remaining[1 : report_count + 1]
expected_reports = remaining[report_count + 1 :]
scanner_results = []
scanner_status_path = Path(manifest_path).with_name("scanner-status.tsv")
if scanner_status_path.is_file():
for line_number, raw_line in enumerate(
scanner_status_path.read_text(encoding="utf-8").splitlines(),
start=1,
):
fields = raw_line.split("\t")
if len(fields) != 4:
raise SystemExit(
f"{scanner_status_path}:{line_number}: expected four fields"
)
scanner_id, name, exit_code, status = fields
scanner_results.append(
{
"id": scanner_id,
"name": name,
"exit_code": int(exit_code),
"status": status,
}
)
result_ids = {result["id"] for result in scanner_results}
incomplete_scanners = sorted(
{
result["id"]
for result in scanner_results
if result["status"] in {"scanner-failure", "skipped"}
}
| (set(required_scanners) - result_ids)
)
payload = {
"mode": mode,
"scope": scope,
"fail_on_findings": fail_on_findings == "1",
"started_at_utc": started_at,
"completed_at_utc": completed_at,
"toolbox_fingerprint": toolbox_fingerprint,
"workspace_unchanged": workspace_unchanged == "true",
"overall_status": int(overall_status),
"finding_status": int(finding_status),
"execution_error_status": int(execution_status),
"missing_tool_status": int(missing_status),
"coverage_status": "complete" if coverage_complete == "true" else "incomplete",
"scanner_coverage": {
"required": required_scanners,
"results": scanner_results,
"incomplete": incomplete_scanners,
},
"tool_versions": "tool-versions.txt",
"workspace_state_start": "workspace-state-start.tsv",
"workspace_state_end": "workspace-state-end.tsv",
"report_checksums": "report-sha256.txt",
"step_status": "step-status.tsv",
"repositories": repositories,
"reports": reports,
"expected_reports": expected_reports,
"missing_reports": sorted(set(expected_reports) - set(reports)),
}
Path(manifest_path).write_text(
json.dumps(payload, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
PY
}
accumulate_exit_status() {
local -n aggregate="$1"
local child_status="$2"
if [[ "$child_status" -gt 1 ]]; then
aggregate=2
elif [[ "$child_status" -eq 1 && "$aggregate" -eq 0 ]]; then
aggregate=1
fi
}
repository_fingerprint() {
local repo="$1"
(
set -e
cd "$repo" || exit 1
git rev-parse HEAD
git status --porcelain=v1 -z
git diff --binary --no-ext-diff HEAD --
while IFS= read -r -d '' path; do
printf '%s\0' "$path"
if [[ -L "$path" ]]; then
printf 'symlink\0'
readlink -- "$path"
else
sha256sum -- "$path"
fi
done < <(git ls-files --others --exclude-standard -z)
) | sha256sum | cut -d' ' -f1
}
write_workspace_state() {
local destination="$1"
local head
local fingerprint
: > "$destination"
for repo in "${REPOS[@]}"; do
if ! head="$(git -C "$repo" rev-parse HEAD)"; then
return 2
fi
if ! fingerprint="$(repository_fingerprint "$repo")"; then
return 2
fi
printf '%s\t%s\t%s\n' \
"$repo" \
"$head" \
"$fingerprint" \
>> "$destination"
done
}
write_tool_versions() {
local destination="$REPORTS_DIR/tool-versions.txt"
local status=0
local version
: > "$destination"
local tool
for tool in semgrep bandit ruff gitleaks trivy pip-audit npm osv-scanner jscpd radon xenon; do
if ! has_tool "$tool"; then
printf '%s\t%s\n' "$tool" "missing" >> "$destination"
continue
fi
if [[ "$tool" == "gitleaks" ]]; then
version="$("$tool" version 2>&1)"
else
version="$("$tool" --version 2>&1)"
fi
local version_status=$?
version="${version%%$'\n'*}"
if [[ "$version_status" -ne 0 || -z "$version" ]]; then
printf '%s\t%s\n' "$tool" "version-command-failed:$version_status" >> "$destination"
status=2
else
printf '%s\t%s\n' "$tool" "$version" >> "$destination"
fi
done
return "$status"
}
write_report_checksums() {
local path
local relative_path
: > "$REPORTS_DIR/report-sha256.txt"
while IFS= read -r path; do
[[ -f "$path" ]] || continue
relative_path="${path#"$REPORTS_DIR"/}"
(cd "$REPORTS_DIR" && sha256sum -- "$relative_path") \
>> "$REPORTS_DIR/report-sha256.txt" || return 2
done < <(printf '%s\n' "${!REPORT_FILES[@]}" | sort)
}
validate_machine_reports() {
python3 - "$REPORTS_DIR" "${!MACHINE_REPORTS[@]}" <<'PY'
from __future__ import annotations
import json
from pathlib import Path
import sys
root = Path(sys.argv[1]).resolve()
errors: list[str] = []
validated = 0
for raw_path in sorted(set(sys.argv[2:])):
path = Path(raw_path)
try:
relative_path = path.resolve().relative_to(root)
except (OSError, ValueError) as exc:
errors.append(f"{path}: invalid report path: {exc}")
continue
if not path.is_file():
errors.append(f"{relative_path}: expected report was not produced")
continue
if path.stat().st_size == 0:
errors.append(f"{relative_path}: report is empty")
continue
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
errors.append(f"{relative_path}: invalid JSON: {exc}")
continue
validated += 1
if path.suffix == ".sarif" and (
not isinstance(payload, dict)
or not isinstance(payload.get("version"), str)
or not isinstance(payload.get("runs"), list)
):
errors.append(f"{relative_path}: invalid SARIF document")
continue
report_name = path.name
if report_name.startswith("bandit") and (
not isinstance(payload, dict)
or not isinstance(payload.get("errors"), list)
or not isinstance(payload.get("metrics"), dict)
or not isinstance(payload.get("results"), list)
):
errors.append(f"{relative_path}: invalid Bandit report")
elif report_name.startswith("ruff-security") and not isinstance(payload, list):
errors.append(f"{relative_path}: invalid Ruff report")
elif report_name.startswith("gitleaks-") and not isinstance(payload, list):
errors.append(f"{relative_path}: invalid Gitleaks report")
elif report_name.startswith("pip-audit-") and (
not isinstance(payload, dict)
or not isinstance(payload.get("dependencies"), list)
):
errors.append(f"{relative_path}: invalid pip-audit report")
elif report_name.startswith("npm-audit-") and (
not isinstance(payload, dict)
or payload.get("error")
or not isinstance(payload.get("metadata"), dict)
or not isinstance(payload.get("vulnerabilities"), dict)
):
errors.append(f"{relative_path}: npm audit returned an invalid or error report")
elif report_name.startswith("osv-scanner-") and (
not isinstance(payload, dict)
or "results" not in payload
or (payload["results"] is not None and not isinstance(payload["results"], list))
):
errors.append(f"{relative_path}: invalid OSV-Scanner report")
elif report_name == "jscpd-report.json" and (
not isinstance(payload, dict)
or not isinstance(payload.get("duplicates"), list)
or not isinstance(payload.get("statistics"), dict)
):
errors.append(f"{relative_path}: invalid jscpd report")
if errors:
print("\n".join(errors), file=sys.stderr)
raise SystemExit(2)
print(f"Validated {validated} JSON/SARIF reports.")
PY
} }
run_semgrep() { run_semgrep() {
local report="$REPORTS_DIR/semgrep.sarif" local report="$REPORTS_DIR/semgrep.sarif"
prepare_machine_report "$report" || return 2
local config_args=(--config "$ROOT/tools/checks/security-audit/semgrep-govoplan.yml") local config_args=(--config "$ROOT/tools/checks/security-audit/semgrep-govoplan.yml")
if [[ "$MODE" != "quick" ]]; then if [[ "$MODE" != "quick" ]]; then
config_args+=(--config p/default --config p/owasp-top-ten) config_args+=(--config p/default --config p/owasp-top-ten)
fi fi
semgrep scan \ semgrep scan \
--metrics=off \ --metrics=off \
--error \
--no-git-ignore \
--sarif \ --sarif \
--output "$report" \ --output "$report" \
--exclude .git \
--exclude node_modules \ --exclude node_modules \
--exclude .venv \ --exclude .venv \
--exclude dist \ --exclude dist \
--exclude build \ --exclude build \
--exclude runtime \ --exclude runtime \
--exclude audit-reports \
--exclude '**/.*-test-build/**' \
"${config_args[@]}" \ "${config_args[@]}" \
"${REPOS[@]}" "${REPOS[@]}"
} }
@@ -194,10 +630,20 @@ run_semgrep() {
run_bandit() { run_bandit() {
local status=0 local status=0
if [[ "${#PY_PROD_ROOTS[@]}" -gt 0 ]]; then if [[ "${#PY_PROD_ROOTS[@]}" -gt 0 ]]; then
bandit -r "${PY_PROD_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit.json" || status=1 prepare_machine_report "$REPORTS_DIR/bandit.json" || return 2
bandit -r "${PY_PROD_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit.json"
accumulate_exit_status status "$?"
fi fi
if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then
bandit -r "${PY_TEST_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit-tests.json" || true prepare_machine_report "$REPORTS_DIR/bandit-tests.json" || return 2
# Tests intentionally use assertions and synthetic credentials. Keep the
# separate test scan useful by excluding only those test-specific signals.
bandit -r "${PY_TEST_ROOTS[@]}" \
--skip B101,B105,B106,B107 \
-f json \
-o "$REPORTS_DIR/bandit-tests.json"
local test_status=$?
[[ "$test_status" -le 1 ]] || status=2
fi fi
return "$status" return "$status"
} }
@@ -205,10 +651,19 @@ run_bandit() {
run_ruff_security() { run_ruff_security() {
local status=0 local status=0
if [[ "${#PY_PROD_ROOTS[@]}" -gt 0 ]]; then if [[ "${#PY_PROD_ROOTS[@]}" -gt 0 ]]; then
ruff check --select S --output-format json "${PY_PROD_ROOTS[@]}" > "$REPORTS_DIR/ruff-security.json" || status=1 prepare_machine_report "$REPORTS_DIR/ruff-security.json" || return 2
ruff check --select S --output-format json "${PY_PROD_ROOTS[@]}" > "$REPORTS_DIR/ruff-security.json"
accumulate_exit_status status "$?"
fi fi
if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then
ruff check --select S --output-format json "${PY_TEST_ROOTS[@]}" > "$REPORTS_DIR/ruff-security-tests.json" || true prepare_machine_report "$REPORTS_DIR/ruff-security-tests.json" || return 2
ruff check \
--select S \
--ignore S101,S105,S106,S107 \
--output-format json \
"${PY_TEST_ROOTS[@]}" > "$REPORTS_DIR/ruff-security-tests.json"
local test_status=$?
[[ "$test_status" -le 1 ]] || status=2
fi fi
return "$status" return "$status"
} }
@@ -219,19 +674,34 @@ run_gitleaks() {
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
if gitleaks git --help >/dev/null 2>&1; then if gitleaks git --help >/dev/null 2>&1; then
prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2
prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2
gitleaks git \ gitleaks git \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
--report-path "$REPORTS_DIR/gitleaks-$name.json" \ --report-path "$REPORTS_DIR/gitleaks-history-$name.json" \
--no-banner \ --no-banner \
"$repo" || status=1 "$repo"
accumulate_exit_status status "$?"
# History scanning does not include new or modified working-tree files.
# Scan the directory as well so pre-commit audits cover the exact code
# under review, while retaining the history scan above.
gitleaks dir \
--config "$ROOT/.gitleaks.toml" \
--report-format json \
--report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \
--no-banner \
"$repo"
accumulate_exit_status status "$?"
else else
prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2
gitleaks detect \ gitleaks detect \
--source "$repo" \ --source "$repo" \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
--report-path "$REPORTS_DIR/gitleaks-$name.json" \ --report-path "$REPORTS_DIR/gitleaks-$name.json" \
--no-banner || status=1 --no-banner
accumulate_exit_status status "$?"
fi fi
done done
return "$status" return "$status"
@@ -242,6 +712,7 @@ run_trivy() {
for repo in "${REPOS[@]}"; do for repo in "${REPOS[@]}"; do
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
prepare_machine_report "$REPORTS_DIR/trivy-$name.sarif" || return 2
trivy fs \ trivy fs \
--scanners vuln,secret,misconfig \ --scanners vuln,secret,misconfig \
--skip-dirs node_modules \ --skip-dirs node_modules \
@@ -249,8 +720,10 @@ run_trivy() {
--skip-dirs dist \ --skip-dirs dist \
--skip-dirs build \ --skip-dirs build \
--format sarif \ --format sarif \
--exit-code 1 \
--output "$REPORTS_DIR/trivy-$name.sarif" \ --output "$REPORTS_DIR/trivy-$name.sarif" \
"$repo" || status=1 "$repo"
accumulate_exit_status status "$?"
done done
return "$status" return "$status"
} }
@@ -258,10 +731,39 @@ run_trivy() {
run_pip_audit_manifests() { run_pip_audit_manifests() {
local status=0 local status=0
for repo in "${REPOS[@]}"; do for repo in "${REPOS[@]}"; do
[[ -f "$repo/requirements.txt" ]] || continue local repo_name
local name repo_name="$(safe_name "$repo")"
name="$(safe_name "$repo")" while IFS= read -r -d '' requirements_file; do
pip-audit -r "$repo/requirements.txt" --progress-spinner off --format json --output "$REPORTS_DIR/pip-audit-$name.json" || status=1 local manifest_name report_path
manifest_name="$(safe_name "$requirements_file")"
report_path="$REPORTS_DIR/pip-audit-$repo_name-$manifest_name.json"
prepare_machine_report "$report_path" || return 2
# Requirements may contain project-relative entries such as `.[server]`.
# Resolve them from the directory containing the manifest.
(
cd "$(dirname "$requirements_file")" &&
pip-audit \
-r "$(basename "$requirements_file")" \
--progress-spinner off \
--format json \
--output "$report_path"
)
accumulate_exit_status status "$?"
done < <(
find "$repo" \
-type d \( \
-name .git \
-o -name .venv \
-o -name node_modules \
-o -name dist \
-o -name build \
-o -name runtime \
-o -name audit-reports \
-o -name '.*-test-build' \
\) -prune \
-o -type f -name 'requirements*.txt' -print0 |
sort -z
)
done done
return "$status" return "$status"
} }
@@ -269,10 +771,44 @@ run_pip_audit_manifests() {
run_npm_audit_manifests() { run_npm_audit_manifests() {
local status=0 local status=0
for repo in "${REPOS[@]}"; do for repo in "${REPOS[@]}"; do
[[ -f "$repo/webui/package-lock.json" ]] || continue local repo_name
local name repo_name="$(safe_name "$repo")"
name="$(safe_name "$repo")" while IFS= read -r -d '' lock_file; do
(cd "$repo/webui" && npm audit --omit=dev --json > "$REPORTS_DIR/npm-audit-$name.json") || status=1 local lock_dir lock_name runtime_report all_report
lock_dir="$(dirname "$lock_file")"
lock_name="$(
printf '%s' "${lock_file#"$repo"/}" |
tr -c 'A-Za-z0-9_.-' '_'
)"
runtime_report="$REPORTS_DIR/npm-audit-runtime-$repo_name-$lock_name.json"
all_report="$REPORTS_DIR/npm-audit-all-$repo_name-$lock_name.json"
prepare_machine_report "$runtime_report" || return 2
(
cd "$lock_dir" &&
npm audit --omit=dev --json > "$runtime_report"
)
accumulate_exit_status status "$?"
prepare_machine_report "$all_report" || return 2
(
cd "$lock_dir" &&
npm audit --json > "$all_report"
)
accumulate_exit_status status "$?"
done < <(
find "$repo" \
-type d \( \
-name .git \
-o -name .venv \
-o -name node_modules \
-o -name dist \
-o -name build \
-o -name runtime \
-o -name audit-reports \
-o -name '.*-test-build' \
\) -prune \
-o -type f -name package-lock.json -print0 |
sort -z
)
done done
return "$status" return "$status"
} }
@@ -283,15 +819,24 @@ run_osv_scanner() {
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
if osv-scanner scan --help >/dev/null 2>&1; then if osv-scanner scan --help >/dev/null 2>&1; then
osv-scanner scan -r --format json --output "$REPORTS_DIR/osv-scanner-$name.json" "$repo" || status=1 prepare_machine_report "$REPORTS_DIR/osv-scanner-$name.json" || return 2
osv-scanner scan -r \
--allow-no-lockfiles \
--format json \
--output-file "$REPORTS_DIR/osv-scanner-$name.json" \
"$repo"
accumulate_exit_status status "$?"
else else
osv-scanner -r --format json --output "$REPORTS_DIR/osv-scanner-$name.json" "$repo" || status=1 prepare_machine_report "$REPORTS_DIR/osv-scanner-$name.json" || return 2
osv-scanner -r --format json --output "$REPORTS_DIR/osv-scanner-$name.json" "$repo"
accumulate_exit_status status "$?"
fi fi
done done
return "$status" return "$status"
} }
run_jscpd() { run_jscpd() {
prepare_machine_report "$REPORTS_DIR/jscpd/jscpd-report.json" || return 2
local ignored_paths local ignored_paths
ignored_paths=( ignored_paths=(
"**/.git/**" "**/.git/**"
@@ -326,97 +871,149 @@ run_jscpd() {
--output "$REPORTS_DIR/jscpd" \ --output "$REPORTS_DIR/jscpd" \
--ignore "$ignored_path_pattern" \ --ignore "$ignored_path_pattern" \
"${REPOS[@]}" "${REPOS[@]}"
local jscpd_status=$?
[[ "$jscpd_status" -eq 0 ]] || return 2
python3 - "$REPORTS_DIR/jscpd/jscpd-report.json" <<'PY'
from __future__ import annotations
import json
from pathlib import Path
import sys
report = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
raise SystemExit(1 if report.get("duplicates") else 0)
PY
} }
run_radon() { run_radon() {
[[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0 [[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0
register_report "$REPORTS_DIR/radon-cc.txt" || return 2
radon cc -s -a "${PY_ROOTS[@]}" > "$REPORTS_DIR/radon-cc.txt" radon cc -s -a "${PY_ROOTS[@]}" > "$REPORTS_DIR/radon-cc.txt"
} }
run_xenon() { run_xenon() {
[[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0 [[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0
register_report "$REPORTS_DIR/xenon.txt" || return 2
xenon --max-absolute C --max-modules B --max-average A "${PY_ROOTS[@]}" > "$REPORTS_DIR/xenon.txt" 2>&1 xenon --max-absolute C --max-modules B --max-average A "${PY_ROOTS[@]}" > "$REPORTS_DIR/xenon.txt" 2>&1
} }
write_manifest register_report "$REPORTS_DIR/step-status.tsv"
register_report "$REPORTS_DIR/scanner-status.tsv"
register_report "$REPORTS_DIR/tool-versions.txt"
register_report "$REPORTS_DIR/workspace-state-start.tsv"
register_report "$REPORTS_DIR/workspace-state-end.tsv"
: > "$REPORTS_DIR/step-status.tsv"
: > "$REPORTS_DIR/scanner-status.tsv"
if ! write_workspace_state "$REPORTS_DIR/workspace-state-start.tsv"; then
echo "Could not capture the repository state before the audit." >&2
exit 1
fi
run_step "Record audit tool versions" execution-only write_tool_versions
if has_tool semgrep; then if has_tool semgrep; then
run_step "Semgrep SAST" run_semgrep run_scanner_step semgrep "Semgrep SAST" findings-exit-one run_semgrep
else else
skip_or_fail_missing semgrep skip_or_fail_missing semgrep semgrep
fi fi
if has_tool bandit; then if has_tool bandit; then
run_step "Bandit Python security scan" run_bandit run_scanner_step bandit "Bandit Python security scan" findings-exit-one run_bandit
else else
skip_or_fail_missing bandit skip_or_fail_missing bandit bandit
fi fi
if has_tool ruff; then if has_tool ruff; then
run_step "Ruff flake8-bandit security rules" run_ruff_security run_scanner_step ruff-security "Ruff flake8-bandit security rules" findings-exit-one run_ruff_security
else else
skip_or_fail_missing ruff skip_or_fail_missing ruff-security ruff
fi fi
if has_tool gitleaks; then if has_tool gitleaks; then
run_step "Gitleaks secret scan" run_gitleaks run_scanner_step gitleaks "Gitleaks secret scan" findings-exit-one run_gitleaks
else else
skip_or_fail_missing gitleaks skip_or_fail_missing gitleaks gitleaks
fi fi
if [[ "$MODE" != "quick" ]]; then if [[ "$MODE" != "quick" ]]; then
if has_tool trivy; then if has_tool trivy; then
run_step "Trivy filesystem vulnerability/secret/misconfig scan" run_trivy run_scanner_step trivy "Trivy filesystem vulnerability/secret/misconfig scan" findings-exit-one run_trivy
else else
skip_or_fail_missing trivy skip_or_fail_missing trivy trivy
fi fi
if has_tool pip-audit; then if has_tool pip-audit; then
run_step "pip-audit requirements scan" run_pip_audit_manifests run_scanner_step pip-audit "pip-audit requirements scan" findings-exit-one run_pip_audit_manifests
else else
skip_or_fail_missing pip-audit skip_or_fail_missing pip-audit pip-audit
fi fi
if has_tool npm; then if has_tool npm; then
run_step "npm audit lockfile scan" run_npm_audit_manifests run_scanner_step npm-audit "npm audit lockfile scan" findings-exit-one run_npm_audit_manifests
else else
skip_or_fail_missing npm skip_or_fail_missing npm-audit npm
fi fi
fi fi
if [[ "$MODE" == "full" ]]; then if [[ "$MODE" == "full" ]]; then
if has_tool osv-scanner; then if has_tool osv-scanner; then
run_step "OSV-Scanner recursive dependency scan" run_osv_scanner run_scanner_step osv-scanner "OSV-Scanner recursive dependency scan" findings-exit-one run_osv_scanner
else else
skip_or_fail_missing osv-scanner skip_or_fail_missing osv-scanner osv-scanner
fi fi
if has_tool jscpd; then if has_tool jscpd; then
run_step "jscpd duplicate code scan" run_jscpd run_scanner_step jscpd "jscpd duplicate code scan" findings-exit-one run_jscpd
else else
skip_or_fail_missing jscpd skip_or_fail_missing jscpd jscpd
fi fi
if has_tool radon; then if has_tool radon; then
run_step "Radon complexity report" run_radon run_scanner_step radon "Radon complexity report" execution-only run_radon
else else
skip_or_fail_missing radon skip_or_fail_missing radon radon
fi fi
if has_tool xenon; then if has_tool xenon; then
run_step "Xenon complexity threshold scan" run_xenon run_scanner_step xenon "Xenon complexity threshold scan" findings-exit-one run_xenon
else else
skip_or_fail_missing xenon skip_or_fail_missing xenon xenon
fi fi
fi fi
run_step "Validate required scanner coverage records" execution-only validate_scanner_status_contract
run_step "Validate machine-readable audit reports" execution-only validate_machine_reports
echo echo
echo "Security audit reports written to: $REPORTS_DIR" echo "Security audit reports written to: $REPORTS_DIR"
if [[ "$finding_status" -ne 0 && "$FAIL_ON_FINDINGS" != "1" ]]; then if [[ "$finding_status" -ne 0 && "$FAIL_ON_FINDINGS" != "1" ]]; then
echo "Findings were reported, but this run is report-only. Re-run with --strict to fail on findings." echo "Findings were reported, but this run is report-only. Re-run with --strict to fail on findings."
fi fi
if [[ "$missing_status" -ne 0 ]]; then if [[ "$missing_status" -ne 0 ]]; then
echo "One or more required tools were missing." >&2 echo "One or more audit tools were missing." >&2
fi
if ! write_workspace_state "$REPORTS_DIR/workspace-state-end.tsv"; then
workspace_unchanged=false
execution_status=1
overall_status=1
echo "Could not capture the repository state after the audit." >&2
elif cmp -s "$REPORTS_DIR/workspace-state-start.tsv" "$REPORTS_DIR/workspace-state-end.tsv"; then
workspace_unchanged=true
else
workspace_unchanged=false
overall_status=1
echo "Repository state changed during the audit; discard this mixed snapshot and rerun." >&2
fi
audit_completed_at="$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
if ! write_report_checksums; then
execution_status=1
overall_status=1
echo "Could not generate report checksums." >&2
fi
if ! write_manifest; then
echo "Could not write the audit manifest." >&2
exit 1
fi fi
exit "$overall_status" exit "$overall_status"
+130
View File
@@ -0,0 +1,130 @@
#!/usr/bin/env python3
"""Fail when backend, manifest, frontend, lock, or release-ref versions drift."""
from __future__ import annotations
import argparse
from dataclasses import asdict
import json
from pathlib import Path
import sys
META_ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(META_ROOT / "tools" / "release"))
from govoplan_release.version_alignment import ( # noqa: E402
release_composition_issues,
repository_version_issues,
selected_repository_version_issues,
)
from govoplan_release.workspace import ( # noqa: E402
load_repository_specs,
resolve_repo_path,
resolve_workspace_root,
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--workspace-root", type=Path, default=None)
parser.add_argument("--repo", action="append", default=[], help="Repository name to check; may be repeated.")
parser.add_argument(
"--repo-version",
action="append",
default=[],
metavar="REPO=VERSION",
help="Require a repository's aligned source metadata to equal VERSION; may be repeated.",
)
parser.add_argument(
"--release-composition",
action="store_true",
help="Also compare tagged backend and WebUI references in the release composition.",
)
parser.add_argument(
"--source-metadata-only",
action="store_true",
help="Skip lockfile checks for the pre-commit source gate; never use for publication.",
)
parser.add_argument("--json", action="store_true", help="Print machine-readable output.")
args = parser.parse_args()
workspace_root = resolve_workspace_root(args.workspace_root)
expected_versions = _parse_repo_versions(parser, args.repo_version)
selected = {*args.repo, *expected_versions}
known = {spec.name for spec in load_repository_specs()}
unknown = sorted(selected - known)
if unknown:
parser.error(f"unknown repository name(s): {', '.join(unknown)}")
checked: list[str] = []
issues = list(
selected_repository_version_issues(
repo_versions=expected_versions,
workspace=workspace_root,
include_lockfiles=not args.source_metadata_only,
)
)
for spec in load_repository_specs():
if selected and spec.name not in selected:
continue
repo_path = resolve_repo_path(spec, workspace_root)
if not repo_path.exists():
continue
checked.append(spec.name)
if spec.name in expected_versions:
continue
issues.extend(
repository_version_issues(
repo_path,
include_lockfiles=not args.source_metadata_only,
)
)
if args.release_composition:
issues.extend(
release_composition_issues(
META_ROOT,
core_root=workspace_root / "govoplan-core",
)
)
payload = {
"workspace_root": str(workspace_root),
"repositories_checked": checked,
"release_composition_checked": args.release_composition,
"issues": [asdict(issue) for issue in issues],
}
if args.json:
print(json.dumps(payload, indent=2, sort_keys=True))
elif issues:
print("Version alignment failed:", file=sys.stderr)
for issue in issues:
print(
f"- {issue.repo}: {issue.source}: {issue.actual!r}; expected {issue.expected!r} "
f"({issue.message})",
file=sys.stderr,
)
else:
suffix = " including release composition" if args.release_composition else ""
print(f"Version alignment passed for {len(checked)} repositories{suffix}.")
return 1 if issues else 0
def _parse_repo_versions(parser: argparse.ArgumentParser, values: list[str]) -> dict[str, str]:
result: dict[str, str] = {}
for value in values:
if "=" not in value:
parser.error(f"--repo-version must use REPO=VERSION: {value}")
repo, version = (item.strip() for item in value.split("=", 1))
version = version.removeprefix("v")
if not repo or not version:
parser.error(f"--repo-version must use REPO=VERSION: {value}")
if repo in result and result[repo] != version:
parser.error(f"conflicting versions requested for {repo}")
result[repo] = version
return result
if __name__ == "__main__":
raise SystemExit(main())

Some files were not shown because too many files have changed in this diff Show More