Compare commits

...

87 Commits

Author SHA1 Message Date
cd15aa514e ci: run artifact-aware module matrix tests
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m36s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 00:53:16 +02:00
a042baa1d3 fix(release): verify tags in mounted workspaces 2026-07-23 00:53:16 +02:00
e80de00f29 chore(release): align Mail with Campaign contract
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m37s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 00:47:46 +02:00
c69acf0dee ci: use Gitea-compatible artifact upload
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m39s
Security Audit / security-audit (push) Successful in 4m11s
2026-07-23 00:36:59 +02:00
8b93bbc6b6 ci: mount audit workspace safely
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m43s
Security Audit / security-audit (push) Failing after 4m10s
2026-07-23 00:30:28 +02:00
3fc17701df ci: fix action repository access
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m45s
Security Audit / security-audit (push) Failing after 3m33s
2026-07-23 00:10:29 +02:00
9788bdde0c ci: harden repository bootstrap transport 2026-07-23 00:09:43 +02:00
a10a01c903 docs: surface system action status
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 23:45:42 +02:00
e005e54333 Seal catalog publication Git metadata
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:31:58 +02:00
76e4baa76e Define the catalog publication identity
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:23:01 +02:00
b6e9a9bd81 Pin the website publication toolchain
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:20:09 +02:00
dc46bda224 Validate complete Core release bundle sources
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:10:35 +02:00
b00d4e55ee Harden isolated release builds in Flatpak
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 16s
Security Audit / security-audit (push) Failing after 15s
2026-07-22 21:48:48 +02:00
76f19dc602 Lock down the local release console 2026-07-22 21:37:33 +02:00
5381e37a9e Execute receipt-bound durable release runs 2026-07-22 21:37:18 +02:00
7ecf1f17b0 Harden release source execution 2026-07-22 21:37:14 +02:00
349a099e5a Harden catalog publication transaction 2026-07-22 21:20:08 +02:00
fdee766993 Authenticate selective catalog base pairs 2026-07-22 20:59:00 +02:00
47b9c05bde chore(release): align Campaign release dependency 2026-07-22 20:44:43 +02:00
4e42911477 Document artifact origin and installer receipts 2026-07-22 20:42:35 +02:00
9c0650b2ed Bind installed releases to signed artifact receipts 2026-07-22 20:42:05 +02:00
4dc0c8d013 Harden release artifact publication 2026-07-22 20:41:42 +02:00
35c346a1fa Add private release candidate handles 2026-07-22 20:41:36 +02:00
4edbc11fe5 feat(release): bound durable run retention 2026-07-22 20:16:19 +02:00
ddee5c00dc fix(assessment): keep release origin unproven
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 14s
2026-07-22 19:55:35 +02:00
4c16069f88 fix(release): preserve durable recovery capacity 2026-07-22 19:48:55 +02:00
22f5c2ff4e fix(release): make durable runs recovery safe 2026-07-22 19:30:12 +02:00
19c4b63ade fix(assessment): harden capability evidence trust 2026-07-22 19:21:06 +02:00
1dc9148ec3 fix(release): harden durable run recovery 2026-07-22 18:58:54 +02:00
ead697d049 fix(assessment): constrain installed record reads 2026-07-22 18:36:12 +02:00
65aa8e0b7c docs(assessment): define external proof boundaries 2026-07-22 18:34:06 +02:00
6c0b003dee feat(assessment): verify installed composition evidence 2026-07-22 18:34:02 +02:00
d0dc916837 feat(release): persist resumable run state 2026-07-22 18:32:19 +02:00
bd715a8473 fix(assessment): reject ambiguous trust keyrings
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 17:17:34 +02:00
ad8dd4f319 fix(assessment): establish independent release trust
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 17:09:35 +02:00
753dd2a3ee fix(release): fail closed on dashboard parse errors 2026-07-22 17:07:30 +02:00
b6bef410d6 fix(release): fail closed on malformed WebUI metadata 2026-07-22 16:55:06 +02:00
72f697c341 fix(release): harden review gate diagnostics 2026-07-22 16:52:27 +02:00
36f662c56f test(release): isolate manifest gate remote 2026-07-22 16:34:39 +02:00
25fbbaf5ad feat(assessment): gate signed release drift 2026-07-22 16:31:28 +02:00
bcab7095c3 feat(release): guide selective gate remediation 2026-07-22 16:28:13 +02:00
99534c0251 fix(release): align Campaign package refs
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 18s
Security Audit / security-audit (push) Failing after 15s
2026-07-22 15:30:43 +02:00
71e74601cf test(release): require retirement atomicity proof 2026-07-22 15:30:16 +02:00
c29f1e9977 docs: refresh capability release baseline 2026-07-22 15:21:27 +02:00
5447299289 Block mismatched selective WebUI releases
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 10:37:19 +02:00
568ea6059a docs(ui): record completed Campaign pilot evidence
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 14s
2026-07-22 09:40:32 +02:00
8b6bcb7a4d docs(ui): record delivered Campaign pilot slices 2026-07-22 08:49:57 +02:00
2fb0a71bfb docs(ui): reconcile Campaign pilot rollout 2026-07-22 08:09:06 +02:00
8ecf8770ab feat(release): synthesize initial catalog entries
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 12s
2026-07-22 04:56:36 +02:00
d53db2da06 fix(audit): bridge Docker from Flatpak sandboxes 2026-07-22 04:48:10 +02:00
8906704dc0 feat(release): gate tags on scoped workflows
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 12s
2026-07-22 01:56:18 +02:00
30f9ed152a feat(release): publish verified source tags
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 12s
2026-07-22 01:17:07 +02:00
eca75cbc93 fix(dev): resolve stale editable packages together 2026-07-22 00:58:16 +02:00
a8abb85676 docs(roadmap): refine postbox and university BI paths
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 12s
2026-07-21 20:48:55 +02:00
097f9ca2f4 docs(ui): codify universal interaction patterns 2026-07-21 20:48:55 +02:00
89771d96fb docs: define connector secret retirement
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 16:00:29 +02:00
492fec46b6 docs: separate installer artifact trust boundary 2026-07-21 15:55:30 +02:00
a863767233 docs: select connected reference journey program
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 15:41:54 +02:00
99e255cf81 fix(release): separate artifact and source contracts
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 13:24:37 +02:00
d9819c4aad test(release): validate installed module artifacts 2026-07-21 13:24:02 +02:00
78b601a2e5 fix(release): reject unpackageable tagged artifacts 2026-07-21 12:58:15 +02:00
a18c7ea040 fix(release): validate independent module versions 2026-07-21 12:51:25 +02:00
b193d4555f fix(audit): accept empty OSV source reports 2026-07-21 12:51:19 +02:00
a81dc21e90 chore: clean release tooling imports 2026-07-21 12:39:36 +02:00
676030b993 Harden security audit result contracts 2026-07-21 12:29:53 +02:00
a15a74c54c feat(release): enforce aligned trusted publications 2026-07-21 12:25:06 +02:00
c34892f6ea feat(release): generate reproducible CycloneDX SBOM 2026-07-21 12:24:49 +02:00
397c87a012 docs: define system administrator lifecycle story 2026-07-21 12:24:32 +02:00
f1b07354a0 Surface connector deployment allowlists 2026-07-21 12:10:44 +02:00
17f8036bdc Harden release console token bootstrap
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-21 03:16:57 +02:00
fcc5885dcf Validate manifest module imports 2026-07-21 03:16:57 +02:00
0b1506f860 Cover live worktrees in security audits 2026-07-21 03:16:57 +02:00
295b49bb04 docs: add connected governance platform roadmap
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 13s
Security Audit / security-audit (push) Failing after 13s
2026-07-20 20:48:41 +02:00
65cc099839 chore(meta): register current workspace modules
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 12s
2026-07-20 20:01:35 +02:00
1ce3d02239 fix(dev): make browser launch opt in 2026-07-20 20:01:27 +02:00
9805d085fa test(release): keep release tooling coverage with its owner 2026-07-20 20:01:19 +02:00
2e21397868 chore(checks): discover workspace module boundaries 2026-07-20 20:01:14 +02:00
4b217c7aba docs(ui): require central component reuse 2026-07-20 17:56:14 +02:00
8b80afbd60 feat(calendar): run scheduled outbox recovery
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 18s
Security Audit / security-audit (push) Failing after 13s
2026-07-20 17:05:17 +02:00
8255665349 chore(gitea): add the user-story issue type 2026-07-20 17:03:23 +02:00
2fc3b66c07 docs: add capability and infrastructure fit assessment 2026-07-20 16:50:06 +02:00
9e7103f613 docs: define the interface pattern language 2026-07-20 16:49:30 +02:00
05ae81d641 intermediate commit
Some checks failed
Security Audit / security-audit (push) Failing after 11s
Security Audit Toolbox Update / toolbox-update (push) Failing after 4s
Dependency Audit / dependency-audit (push) Failing after 17s
2026-07-14 13:32:09 +02:00
dd796b4d3c Register poll and evaluation repositories
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 12s
2026-07-12 09:23:18 +02:00
27ea3c82d4 Reuse Gitea API connections in maintenance scripts
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 25s
Security Audit / security-audit (push) Failing after 22s
2026-07-12 00:18:40 +02:00
342582645b Harden audit tooling and release helpers
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 1m14s
Security Audit / security-audit (push) Failing after 31s
2026-07-11 18:49:36 +02:00
54ab1945ff Update release refs for v0.1.8 2026-07-11 17:22:53 +02:00
134 changed files with 44935 additions and 405 deletions

View File

@@ -12,9 +12,26 @@ ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,aud
CELERY_ENABLED=true CELERY_ENABLED=true
REDIS_URL=redis://127.0.0.1:6379/0 REDIS_URL=redis://127.0.0.1:6379/0
CELERY_QUEUES=send_email,append_sent,default CELERY_QUEUES=send_email,append_sent,notifications,calendar,default
CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=false
GOVOPLAN_CONNECTOR_MAX_STRUCTURED_RESPONSE_BYTES=16777216
GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES=536870912
GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST=
GOVOPLAN_CONNECTOR_CA_BUNDLE_ALLOWLIST=
GOVOPLAN_HTTP_MAX_REQUEST_BODY_BYTES=536870912
GOVOPLAN_HTTP_HSTS_SECONDS=31536000
AUTH_LOGIN_THROTTLE_ENABLED=true
AUTH_LOGIN_THROTTLE_IDENTITY_LIMIT=10
AUTH_LOGIN_THROTTLE_CLIENT_LIMIT=100
AUTH_LOGIN_THROTTLE_WINDOW_SECONDS=900
AUTH_LOGIN_THROTTLE_REDIS_RETRY_SECONDS=30
CORS_ORIGINS=https://govoplan.example.org CORS_ORIGINS=https://govoplan.example.org
GOVOPLAN_TRUSTED_HOSTS=govoplan.example.org
FORWARDED_ALLOW_IPS=127.0.0.1
AUTH_COOKIE_SECURE=true AUTH_COOKIE_SECURE=true
AUTH_COOKIE_SAMESITE=lax AUTH_COOKIE_SAMESITE=lax
AUTH_COOKIE_DOMAIN= AUTH_COOKIE_DOMAIN=

View File

@@ -21,29 +21,18 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with: with:
node-version: "22" node-version: "22"
- name: Configure SSH for release dependencies - name: Use anonymous HTTPS for public GovOPlaN repositories
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
run: | run: |
mkdir -p ~/.ssh git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "git@git.add-ideas.de:add-ideas/govoplan"
chmod 700 ~/.ssh git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "ssh://git@git.add-ideas.de/add-ideas/govoplan"
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend dev audit dependencies - name: Install backend dev audit dependencies
working-directory: govoplan working-directory: govoplan
run: | run: |
python -m venv .venv python -m venv .venv
.venv/bin/python -m pip install --upgrade pip .venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip
.venv/bin/python -m pip install -r requirements-release.txt
.venv/bin/python -m pip install 'pip-audit>=2.9,<3' .venv/bin/python -m pip install 'pip-audit>=2.9,<3'
- name: Install WebUI release dependencies - name: Install WebUI release dependencies
working-directory: govoplan working-directory: govoplan

View File

@@ -17,29 +17,18 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with: with:
node-version: "22" node-version: "22"
- name: Configure SSH for release dependencies - name: Use anonymous HTTPS for public GovOPlaN repositories
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
run: | run: |
mkdir -p ~/.ssh git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "git@git.add-ideas.de:add-ideas/govoplan"
chmod 700 ~/.ssh git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "ssh://git@git.add-ideas.de/add-ideas/govoplan"
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend release dependencies - name: Install backend release dependencies
working-directory: govoplan working-directory: govoplan
run: | run: |
python -m venv .venv python -m venv .venv
.venv/bin/python -m pip install --upgrade pip .venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip
.venv/bin/python -m pip install -r requirements-release.txt
.venv/bin/python -m pip install '../govoplan-core[dev]' .venv/bin/python -m pip install '../govoplan-core[dev]'
- name: Install WebUI release dependencies with test scripts - name: Install WebUI release dependencies with test scripts
working-directory: govoplan working-directory: govoplan

View File

@@ -16,29 +16,18 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with: with:
node-version: "22" node-version: "22"
- name: Configure SSH for release dependencies - name: Use anonymous HTTPS for public GovOPlaN repositories
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
run: | run: |
mkdir -p ~/.ssh git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "git@git.add-ideas.de:add-ideas/govoplan"
chmod 700 ~/.ssh git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "ssh://git@git.add-ideas.de/add-ideas/govoplan"
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend release integration dependencies - name: Install backend release integration dependencies
working-directory: govoplan working-directory: govoplan
run: | run: |
python -m venv .venv python -m venv .venv
.venv/bin/python -m pip install --upgrade pip .venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip
.venv/bin/python -m pip install -r requirements-release.txt
.venv/bin/python -m pip install '../govoplan-core[dev]' .venv/bin/python -m pip install '../govoplan-core[dev]'
- name: Install WebUI release dependencies - name: Install WebUI release dependencies
working-directory: govoplan working-directory: govoplan

View File

@@ -1,7 +1,6 @@
name: Security Audit name: Security Audit
on: on:
pull_request:
push: push:
branches: branches:
- main - main
@@ -21,6 +20,10 @@ jobs:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with: with:
path: govoplan path: govoplan
- name: Use anonymous HTTPS for public GovOPlaN repositories
run: |
git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "git@git.add-ideas.de:add-ideas/govoplan"
git config --global --add url."https://git.add-ideas.de/add-ideas/govoplan".insteadOf "ssh://git@git.add-ideas.de/add-ideas/govoplan"
- name: Configure SSH for repository bootstrap - name: Configure SSH for repository bootstrap
env: env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }} GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
@@ -37,13 +40,13 @@ jobs:
chmod 600 ~/.ssh/known_hosts chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories - name: Bootstrap GovOPlaN repositories
working-directory: govoplan working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent .. run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https
- name: Run security audit - name: Run whole-system security audit
working-directory: govoplan working-directory: govoplan
run: tools/checks/security-audit/run.sh --mode "$SECURITY_AUDIT_MODE" --scope "$SECURITY_AUDIT_SCOPE" --reports-dir audit-reports run: tools/checks/security-audit/run.sh --mode "$SECURITY_AUDIT_MODE" --scope "$SECURITY_AUDIT_SCOPE" --reports-dir audit-reports
- name: Upload audit reports - name: Upload audit reports
if: always() if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with: with:
name: security-audit-reports name: security-audit-reports
path: govoplan/audit-reports path: govoplan/audit-reports

View File

@@ -3,7 +3,6 @@ title = "GovOPlaN secret scanning"
[allowlist] [allowlist]
description = "Repository examples and documented development placeholders" description = "Repository examples and documented development placeholders"
regexes = [ regexes = [
'''dev-multimailer-api-key''',
'''example(\.invalid|\.org|\.com)''', '''example(\.invalid|\.org|\.com)''',
'''postgresql(\+psycopg)?://[^:@/\s]+:\*\*\*@''', '''postgresql(\+psycopg)?://[^:@/\s]+:\*\*\*@''',
'''<redacted>''', '''<redacted>''',

View File

@@ -4,6 +4,11 @@
**Repository type:** system (meta). **Repository type:** system (meta).
<!-- govoplan-repository-type:end --> <!-- govoplan-repository-type:end -->
[![Module Matrix](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/module-matrix.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=module-matrix.yml&actor=0&status=0)
[![Release Integration](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/release-integration.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=release-integration.yml&actor=0&status=0)
[![Dependency Audit](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/dependency-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=dependency-audit.yml&actor=0&status=0)
[![Security Audit](https://git.add-ideas.de/add-ideas/govoplan/actions/workflows/security-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/add-ideas/govoplan/actions?workflow=security-audit.yml&actor=0&status=0)
This is the GovOPlaN meta repository. It is the operator entry point for This is the GovOPlaN meta repository. It is the operator entry point for
whole-product development, release orchestration, repository bootstrap, and whole-product development, release orchestration, repository bootstrap, and
system-level Docker composition. system-level Docker composition.
@@ -17,8 +22,7 @@ Create the whole-product development virtualenv in this meta repository:
```sh ```sh
python3 -m venv .venv python3 -m venv .venv
./.venv/bin/python -m pip install --upgrade pip ./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-dev.txt --python ./.venv/bin/python --upgrade-pip
./.venv/bin/python -m pip install -r requirements-dev.txt
``` ```
The meta venv is the default Python environment for launch, check, release, and The meta venv is the default Python environment for launch, check, release, and
@@ -31,6 +35,12 @@ Start the development stack through the meta repository:
./tools/launch/launch-dev.sh ./tools/launch/launch-dev.sh
``` ```
Open the WebUI in a browser after launch only when explicitly requested:
```sh
GOVOPLAN_OPEN_BROWSER=1 ./tools/launch/launch-dev.sh
```
Start the shared development PostgreSQL service: Start the shared development PostgreSQL service:
```sh ```sh
@@ -55,6 +65,45 @@ Update generated repository type notes in all READMEs:
./tools/repo/update-repository-type-notes.py ./tools/repo/update-repository-type-notes.py
``` ```
Regenerate the human-readable repository link index:
```sh
./tools/repo/generate-repository-index.py
```
Synchronize the Python environment after package metadata changes:
```sh
./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-dev.txt --python ./.venv/bin/python
```
Run the static cross-repository module contract check:
```sh
./tools/checks/check-contracts.sh
```
Require backend, manifest, frontend, lockfile, and release-composition versions
to agree before a release:
```sh
./.venv/bin/python tools/checks/check-version-alignment.py --release-composition
```
Generate the CycloneDX dependency inventory from a resolved release environment:
```sh
./.venv/bin/python tools/release/generate-release-sbom.py --python ./.venv/bin/python
```
For reproducible release artifacts, set `SOURCE_DATE_EPOCH` to the release
commit timestamp (or pass an explicit timezone-qualified `--timestamp`):
```sh
SOURCE_DATE_EPOCH="$(git -C ../govoplan-core show -s --format=%ct HEAD)" \
./.venv/bin/python tools/release/generate-release-sbom.py --python ./.venv/bin/python
```
Run the consolidated focused verification suite: Run the consolidated focused verification suite:
```sh ```sh
@@ -76,6 +125,12 @@ Run installer rollback drills:
Release, catalog, Gitea, security-audit, and cross-repository maintenance Release, catalog, Gitea, security-audit, and cross-repository maintenance
commands should also be called from this repository through `tools/`. commands should also be called from this repository through `tools/`.
Start the local release console:
```sh
./.venv/bin/python tools/release/release-console.py
```
## Configuration ## Configuration
The repository root `.env.example` is the self-hosted operator template for a The repository root `.env.example` is the self-hosted operator template for a
@@ -89,7 +144,27 @@ such as `~/.config/gitea/gitea.env` and be passed with `--env-file`.
The repository categories are documented in The repository categories are documented in
`docs/REPOSITORY_STRUCTURE.md`. The machine-readable list lives in `docs/REPOSITORY_STRUCTURE.md`. The machine-readable list lives in
`repositories.json`. `repositories.json`; the clickable human-readable index is
`docs/REPOSITORY_INDEX.md`.
Meta ownership and module install/contract boundaries are documented in
`docs/META_REPO_SCAN.md` and `docs/MODULE_CONTRACTS_AND_INSTALLS.md`.
Frontend layout principles for module pages are documented in
`docs/FRONTEND_LAYOUT_PRINCIPLES.md`.
The cross-product destination, stakeholder visions, configuration archetypes,
connected outcome stories, and capability horizons are documented in
the [Connected Governance Platform Roadmap](docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md).
The selected Campaign-to-Postbox-to-data-to-collaboration implementation path,
including stage gates and shared documentation expectations, is in the
[Reference Journey Program](docs/REFERENCE_JOURNEY_PROGRAM.md).
The administrator journey from Core-only bootstrap through online module
installation, scale-out, and reversible environment promotion is defined in
[System Administrator Lifecycle User Story](docs/SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
The first Campaign-centric capability and infrastructure fit assessment is in
`docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md`. Its rerun tooling can collect and
verify a bounded installed composition; target, provider and production claims
remain separate, expiring attestations signed by independently scoped proof
authorities.
# GovOPlaN Docker # GovOPlaN Docker

View File

@@ -80,6 +80,12 @@ cd /mnt/DATA/git/govoplan
tools/launch/launch-dev.sh tools/launch/launch-dev.sh
``` ```
The launcher does not open a browser by default. To opt in:
```bash
GOVOPLAN_OPEN_BROWSER=1 tools/launch/launch-dev.sh
```
To force the old SQLite fallback for a disposable local run: To force the old SQLite fallback for a disposable local run:
```bash ```bash
@@ -111,7 +117,12 @@ tools/checks/postgres-integration-check.py \
``` ```
`--reset-schema` drops and recreates the `public` schema before every module `--reset-schema` drops and recreates the `public` schema before every module
set. Use it only against this disposable database. set. Use it only against this disposable database. Before those permutations,
the check runs the Files credential-retirement atomicity proof in random,
test-owned schemas. The proof does not modify `public` and fails the release
gate if its PostgreSQL or full-stack dependencies are unavailable. The
`--skip-retirement-atomicity` option is only for bounded migration/smoke
diagnosis; do not use it for release evidence.
Stop the testbed: Stop the testbed:

View File

@@ -16,10 +16,27 @@ DATABASE_URL=postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan
GOVOPLAN_DATABASE_URL_PGTOOLS=postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan GOVOPLAN_DATABASE_URL_PGTOOLS=postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan
REDIS_URL=redis://127.0.0.1:56379/0 REDIS_URL=redis://127.0.0.1:56379/0
CELERY_ENABLED=true CELERY_ENABLED=true
CELERY_QUEUES=send_email,append_sent,default CELERY_QUEUES=send_email,append_sent,notifications,calendar,default
CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=true
GOVOPLAN_CONNECTOR_MAX_STRUCTURED_RESPONSE_BYTES=16777216
GOVOPLAN_CONNECTOR_MAX_FILE_TRANSFER_BYTES=536870912
GOVOPLAN_CONNECTOR_SECRET_ENV_ALLOWLIST=
GOVOPLAN_CONNECTOR_CA_BUNDLE_ALLOWLIST=
GOVOPLAN_HTTP_MAX_REQUEST_BODY_BYTES=536870912
GOVOPLAN_HTTP_HSTS_SECONDS=0
AUTH_LOGIN_THROTTLE_ENABLED=true
AUTH_LOGIN_THROTTLE_IDENTITY_LIMIT=10
AUTH_LOGIN_THROTTLE_CLIENT_LIMIT=100
AUTH_LOGIN_THROTTLE_WINDOW_SECONDS=900
AUTH_LOGIN_THROTTLE_REDIS_RETRY_SECONDS=30
ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops
CORS_ORIGINS=http://127.0.0.1:5173,http://localhost:5173 CORS_ORIGINS=http://127.0.0.1:5173,http://localhost:5173
GOVOPLAN_TRUSTED_HOSTS=127.0.0.1,localhost,testserver
FORWARDED_ALLOW_IPS=127.0.0.1
AUTH_COOKIE_SECURE=false AUTH_COOKIE_SECURE=false
FILE_STORAGE_BACKEND=local FILE_STORAGE_BACKEND=local
FILE_STORAGE_LOCAL_ROOT=runtime/production-like/files FILE_STORAGE_LOCAL_ROOT=runtime/production-like/files

View File

@@ -1,7 +1,7 @@
# Production-Like Development Profile # Production-Like Development Profile
This profile runs the shared services that production depends on while keeping This profile runs the shared services that production depends on while keeping
API, worker, and WebUI code in the editable local repositories. API, worker, scheduler, and WebUI code in the editable local repositories.
It provides: It provides:
@@ -10,6 +10,7 @@ It provides:
- explicit `ENABLED_MODULES` - explicit `ENABLED_MODULES`
- local durable file storage under `runtime/production-like/files` - local durable file storage under `runtime/production-like/files`
- a Celery worker process using the same queues as the API - a Celery worker process using the same queues as the API
- a Celery beat process for durable retry and recovery schedules
Start it from the meta repository: Start it from the meta repository:
@@ -37,7 +38,7 @@ password changes:
cp dev/production-like/.env.example dev/production-like/.env cp dev/production-like/.env.example dev/production-like/.env
``` ```
The API and worker use: The API, worker, and scheduler use:
```text ```text
DATABASE_URL=postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan DATABASE_URL=postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan
@@ -45,7 +46,7 @@ REDIS_URL=redis://127.0.0.1:56379/0
CELERY_ENABLED=true CELERY_ENABLED=true
``` ```
Stop the launched API/WebUI/worker with `Ctrl+C`. The PostgreSQL and Redis Stop the launched API/WebUI/worker/scheduler with `Ctrl+C`. The PostgreSQL and Redis
containers keep running by default so the next launch is fast. To stop them too: containers keep running by default so the next launch is fast. To stop them too:
```bash ```bash

View File

@@ -0,0 +1,630 @@
# GovOPlaN Capability and IT-Infrastructure Fit Assessment
## Assessment record
| Field | Value |
| --- | --- |
| Assessment ID | `campaign-reference-2026-07-22` |
| Schema | `govoplan.fit-assessment/0.1.0` |
| Assessed on | 2026-07-22 |
| Product snapshot | Live Ed25519-signed stable catalog sequence `202607220843`: Core `v0.1.13` and Campaign `v0.1.10` |
| Configuration basis | Root `.env.example` and the production-like development profile |
| Scope | Campaign-centric internal pilot and small-production candidate |
| Explicitly postponed | Workflow and workflow-driven user stories |
| Machine-readable companion | [`capability-fit-current.json`](capability-fit-current.json) |
| Input schema | [`capability-fit.schema.json`](capability-fit.schema.json) |
This is a fit assessment, not a production approval or security certification.
It deliberately does not infer implementation from a repository, issue, or
manifest existing. A conclusion needs code plus a route/contract, test,
operational drill, configured example, or explicit evidence that the capability
is absent.
The release baseline below is reproducible at its tagged source and package
references. Later main-branch commits do not become release capabilities merely
because they are committed, pushed, or code-tested. This assessment therefore
distinguishes code-tested, package-integrated, target-tested, and
production-approved evidence explicitly.
## Controlled status vocabulary
| Status | Meaning |
| --- | --- |
| `verified` | The stated capability is implemented and directly exercised by evidence appropriate to the stated scope. Conditions and target-environment proof may still remain. |
| `available_unconfigured` | An implementation and supporting evidence exist, but the target endpoint, credentials, policy, or deployment component has not been configured and exercised. |
| `partial` | A useful subset exists, but one or more material parts of the stated journey or operational requirement are missing or unproved. |
| `scaffold` | Contracts, models, routes, or UI structure exist, but the end-to-end capability is not yet usable. |
| `external_system` | GovOPlaN expects the deployment or another system to supply this capability; integration requirements must still be assessed. |
| `planned` | The capability is represented only by a concept, backlog item, or design direction. |
| `not_fit` | Evidence shows that the current composition cannot meet the stated requirement. |
| `not_assessed` | The requirement or target environment is not sufficiently known to reach a conclusion. |
Status alone is not enough. Every row also states its evidence scope and
conditions. For example, a unit-tested SMTP adapter is
`available_unconfigured`, not `verified` for an organization's mail system.
## Executive conclusion
GovOPlaN is a credible candidate for a controlled, Campaign-centric internal
pilot using local accounts, PostgreSQL, durable single-node file storage, and a
non-production SMTP/IMAP service. The module contract, Campaign authoring and
validation paths, managed attachments, mail-profile boundary, local audit
records, and operator status surface all have direct code/test evidence.
The signed catalog resolves the earlier source/package reproducibility gap, but
it does not make the composition production-approved. The production-like
profile intentionally runs only PostgreSQL and Redis in containers; API,
WebUI, worker, and scheduler processes still run from editable source trees. A
target deployment must supply TLS termination, process supervision, secret
injection, monitoring, backup storage, and recovery procedures. The open
[Core backup/restore issue #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29)
is a production gate. Identity federation, external audit export, and a tested
disaster-recovery plan are also not complete.
The current recommendation is therefore:
- **Pilot:** fit with conditions and a bounded proof of concept.
- **Small production:** partial fit; do not approve until the proof checks and
operational gates below pass against the signed release and target
environment.
- **Workflow:** planned and postponed; it is not part of either recommended
composition in this assessment.
## Pinned composition
The configured reference composition comes from the meta repository's
`ENABLED_MODULES`. Versions and refs below are pinned by the live stable catalog;
commits are the commits peeled from those annotated tags.
| Module | Tagged source commit | Catalog version/ref | Role in this assessment |
| --- | --- | --- | --- |
| Core runtime | `govoplan-core@d487726f4d2c` | `0.1.13` / `v0.1.13` | API, registry, migration orchestration, shared WebUI, sessions and kernel contracts |
| `tenancy` | `govoplan-tenancy@efbec827616b` | `0.1.8` / `v0.1.8` | tenant context and lifecycle |
| `organizations` | `govoplan-organizations@39c081c4fb8f` | `0.1.8` / `v0.1.8` | organization model |
| `identity` | `govoplan-identity@7a1710af896f` | `0.1.8` / `v0.1.8` | normalized internal identity directory |
| `access` | `govoplan-access@f1d64d247e12` | `0.1.11` / `v0.1.11` | local authentication, sessions, API keys and RBAC |
| `admin` | `govoplan-admin@11ecf362a36d` | `0.1.8` / `v0.1.8` | administration surfaces |
| `dashboard` | `govoplan-dashboard@4b960ad37f0d` | `0.1.8` / `v0.1.8` | module-aware home surface |
| `policy` | `govoplan-policy@1063622d311a` | `0.1.9` / `v0.1.9` | policy explanation/configuration boundary |
| `audit` | `govoplan-audit@d3d2c60d7dc1` | `0.1.8` / `v0.1.8` | database audit records and retrying audit outbox |
| `campaigns` | `govoplan-campaign@735e874bd03c` | `0.1.10` / `v0.1.10` | Campaign authoring, build, delivery control and reporting |
| `files` | `govoplan-files@2b34f6e30578` | `0.1.9` / `v0.1.9` | managed files and Campaign attachments |
| `mail` | `govoplan-mail@3e2302909022` | `0.1.10` / `v0.1.10` | SMTP/IMAP profiles and transports |
| `calendar` | `govoplan-calendar@9bcf41bb1fbb` | `0.1.8` / `v0.1.8` | optional calendar; not needed by the Campaign pilot |
| `docs` | `govoplan-docs@be52b716caed` | `0.1.10` / `v0.1.10` | configured-system documentation |
| `ops` | `govoplan-ops@341773a4ff8a` | `0.1.8` / `v0.1.8` | readiness and deployment-profile visibility |
Stable catalog sequence `202607220843` is valid, signed by trusted key
`release-key-1`, and pins matching Python and WebUI refs where applicable. This
proves release source/package selection and provenance, not behavior in a target
environment. No configuration revision/package is pinned yet, so deployment
configuration remains a separate reproducibility and promotion gate. The static
contract scan found 43 interface contracts, 33 providers, and no contract errors
in the current workspace. That proves the scanned manifest graph is internally
consistent; it does not prove every provider or production journey.
`govoplan-addresses@93dddbb8c52a` (`0.1.9` / `v0.1.9`) is an optional catalogued
extension for reusable recipient sources. It is not enabled in the pinned root
profile and must be added explicitly when the pilot needs address lists or
CardDAV.
## Recommended scenarios
### Campaign pilot
Use one internal tenant or office, controlled operators, local GovOPlaN accounts,
and non-critical recipient volumes. Enable:
```text
tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,docs,ops
```
Add `addresses` only when reusable address books/lists or CardDAV are in scope.
Leave Calendar and Workflow out unless the pilot has an explicit journey for
them. The minimum topology is one supervised API process, one built WebUI, one
PostgreSQL database, durable local file storage, and one Redis/Celery worker when
asynchronous delivery is enabled. Use a dedicated non-production SMTP/IMAP
account and a restricted recipient allow-list.
### Small-production candidate
Use a reverse proxy/TLS endpoint, built WebUI assets, separately supervised API
and worker processes, PostgreSQL with measured backup/restore, Redis with
persistence, and durable shared or object storage. Run one scheduler only when a
selected module needs scheduled recovery. Multiple scheduler replicas require
leader election or an external lock, which is not established by this report.
This topology is a recommendation from the [Ops scalability profiles](https://git.add-ideas.de/add-ideas/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md),
not a currently shipped production Compose/Kubernetes/systemd package.
## Functional capability matrix
| Capability | Status | Evidence and scope | Conditions, gaps, or exclusions |
| --- | --- | --- | --- |
| Module-aware API/WebUI composition | `verified` | Core registry tests cover installed manifests, route contribution, missing modules, and module permutations; the static contract scan passed; stable catalog sequence `202607220843` has a valid trusted signature and matching package refs. | Package integration is verified; repeat the module-permutation checks on the installed target composition. |
| Local tenants, accounts, sessions, API keys and RBAC | `verified` | Access/tenancy manifests, auth dependency tests, cookie/CSRF API smoke tests, and role/permission contracts. | External identity providers are not included in this conclusion. |
| Campaign draft authoring, validation, recipient import, build, execution snapshot and mock send | `verified` | Core API smoke tests exercise create/validate/build/mock-send, frozen delivery configuration, policy gates, job deltas and reconciliation; Campaign-focused tests passed; Campaign `v0.1.10` and Core `v0.1.13` are package-integrated in the signed stable catalog. | Code and package integration are verified; usability and target-provider acceptance remain separate. |
| Managed Campaign attachments and local file storage | `verified` | Files capability/route tests, Campaign attachment-build tests, and local-storage backend code. | The deployment must provide a durable path and include files in database/storage backup plans. |
| SMTP send, IMAP append and mailbox/profile policy | `available_unconfigured` | Mail adapters, encrypted profile model, helper tests, Campaign mail-policy smoke tests, and a GreenMail-oriented runbook/testbed. | No target mail server, TLS chain, throttling, sender policy, bounce/reply process, or real delivery receipt was exercised here. |
| Delivery retry, uncertainty and reconciliation | `partial` | Campaign API smoke tests include worker-loss/outcome-unknown reconciliation and frozen job evidence; those paths are present in catalogued Campaign `v0.1.10`. | A target-provider failure drill and operational alerting are still required; package integration does not prove provider behavior. |
| Reusable address lists and Campaign recipient-source integration | `available_unconfigured` | Address list/recipient capability and Campaign optional lookup tests passed; manifests expose optional interfaces. | `addresses` is not in the pinned profile. Configure permissions and data governance before use. |
| CardDAV address synchronization | `available_unconfigured` | Discovery, preview/conflict, two-way create/update/delete and disconnect tests passed in the current Addresses workspace. | Requires target-server interoperability, credentials, CA/TLS, rate-limit, conflict and restore drills. |
| File connectors (WebDAV/Nextcloud/Seafile/SMB/S3 import) | `partial` | Provider descriptors and browse/import helper tests exist; S3 connector tests passed. | Coverage varies by provider and optional dependency. No target content system was exercised; connector egress and provenance policy need review. |
| Local audit log and governed-event retry outbox | `verified` | Audit module contract plus enqueue/dispatch/failure-for-retry tests passed. | Central audit/SIEM export, retention enforcement and operational monitoring are not verified. |
| Configured-system documentation and Ops status pages | `verified` | Docs/Ops manifests contribute protected routes and WebUI packages; Ops code checks database, Redis, workers, storage and deployment-security settings. | This does not replace external monitoring or a target runbook. |
| Calendar/CalDAV integration | `partial` | Calendar `v0.1.8` supplies the catalogued storage/sync foundation. The durable external-write outbox, worker recovery, reconciliation, and retention work is committed, tested, and pushed on Calendar `main` after that tag. | The post-tag outbox work is remote-integrated source, not local-only WIP, but it is not in the signed stable package baseline and has not passed a target CalDAV drill. Bulk synchronized-calendar migration semantics remain separate work. |
| External LDAP/AD, OIDC/SAML or SCIM identity integration | `scaffold` | IDM owns normalized assignment APIs and documents connector boundaries. | Provider connectors, login callback flow and target directory reconciliation are not an implemented end-to-end capability. Use local accounts for this pilot. |
| Export-control/embargo-list screening | `planned` | Product-level [GovOPlaN #12](https://git.add-ideas.de/add-ideas/govoplan/issues/12) defines the consumer-independent user story. | No screening provider, list provenance, matching policy, review flow or legal evidence exists in this composition. |
| Workflow-driven journeys and views | `planned` | Workflow contracts/concepts exist outside this assessment. | Explicitly postponed. Do not include Workflow in pilot or production claims from this report. |
## Infrastructure matrix
| Component | Status | Current evidence | Target requirement / gap |
| --- | --- | --- | --- |
| WebUI and API | `verified` | FastAPI app, module routes, shared WebUI host, health endpoint, dev smoke and module-permutation tests; the catalog pins matching Core/Campaign backend and WebUI refs. | Materialize and supervise immutable artifacts in the target; no production image/service bundle is supplied here. |
| Background workers | `available_unconfigured` | Celery app, Campaign queues, Redis configuration and production-like launcher. | Run target broker/worker heartbeat and failure drills; split queues when provider limits or load justify it. |
| Scheduler | `partial` | A separately launched Celery beat process exists; Calendar recovery scheduling is committed and pushed after the catalogued `v0.1.8` tag. | The Calendar recovery slice is not stable-package-integrated. Keep single-instance until leader election/locking is proved; add process supervision and missed-schedule alerts. |
| PostgreSQL | `verified` | Production target in settings/operator docs, migration tracks and a disposable integration-check harness. | Target HA, patching, connection limits, WAL/archive policy and restore time are external deployment decisions. |
| Redis/queues | `available_unconfigured` | Redis-backed Celery configuration, health ping and append-only production-like container. | Target availability, persistence, eviction, authentication/TLS and queue-loss behavior were not assessed. |
| Local file storage | `verified` | Local backend, configured root, fallback-root tests and Ops writability check. | Suitable only for a durable single-node/shared path with backup; it blocks independent API scaling when node-local. |
| S3-compatible object storage | `partial` | S3 backend and connector code plus mocked browse/import tests. | Exercise the chosen service, credentials, CA, versioning, lifecycle, multipart/size behavior, restore and consistency expectations. |
| Reverse proxy and TLS | `external_system` | Core validates explicit CORS and secure-cookie posture; Ops reports unsafe production settings. | Deployment must provide certificates, proxy/header policy, request limits, logs and renewal monitoring. |
| Local identity and authorization | `verified` | Password/session/API-key/RBAC capabilities and tests. | Establish account lifecycle, MFA expectation, protected bootstrap and break-glass procedure. |
| Federated identity | `scaffold` | IDM boundary and assignment APIs. | Select and implement/test the actual OIDC/SAML/LDAP/SCIM path before requiring federation. |
| Application secret encryption | `verified` | Stable `MASTER_KEY_B64` contract and encrypted mail-credential paths. | Rotation and loss-recovery procedure need target validation. |
| Secret store and injection | `external_system` | Environment/file references are supported and templates avoid populated secrets. | Choose Vault/KMS/Kubernetes/systemd/environment mechanism, restrict access, rotate credentials and prevent secret exposure in logs/backups. |
| SMTP/IMAP and other connectors | `available_unconfigured` | Protocol adapters and development testbeds. | Target endpoints, egress, DNS, CA, throttling, service accounts and data-processing terms remain environment-specific. |
| Health/readiness | `verified` | `/health`, protected `/health/details`, and Ops checks for DB, Redis, workers, storage, maintenance and cookie/CORS posture. | Add external probes and distinguish liveness from dependency readiness for the chosen orchestrator. |
| Metrics, logs and alerting | `partial` | Correlation IDs, slow-request/query metrics in logs, worker inspection and operator status are implemented. | No bundled metrics exporter, log collector, dashboards, queue-depth alerts, pager route or SLO is verified. |
| Audit | `partial` | Local audit tables and retry outbox are verified. | Retention, tamper-evident export, privileged access review and SIEM integration are unproved. |
| Backup and restore | `partial` | Operator guide and installer hooks describe `pg_dump`/`pg_restore`; SQLite and simulated installer rollback drills exist. | [Core #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29) remains open. No target PostgreSQL + files + secrets restore drill or measured RTO/RPO exists. |
| Disaster recovery | `not_assessed` | The Ops guide asks for RPO/RTO and restore drills. | No agreed RPO/RTO, off-site copy, failover topology, dependency recovery order, communications plan or exercise evidence was supplied. |
The scalability and sizing documentation delivered the documentation portions
of [Core #217](https://git.add-ideas.de/add-ideas/govoplan-core/issues/217) and
[Core #219](https://git.add-ideas.de/add-ideas/govoplan-core/issues/219).
[Core #28](https://git.add-ideas.de/add-ideas/govoplan-core/issues/28) records the
operator-documentation slice. These closed tickets are evidence of documented
models, not evidence that an organization's production environment has passed
them.
## Trust, data-flow and network boundaries
| Boundary / flow | Data and trust | Required controls |
| --- | --- | --- |
| Browser -> reverse proxy -> WebUI/API | Session and CSRF cookies, campaign content, recipient personal data and managed files cross the public/client boundary. | HTTPS, exact origins, secure cookies, request/body limits, tenant/RBAC enforcement, security headers and access logs. |
| API -> PostgreSQL | Tenants, identities, permissions, campaign drafts/snapshots/jobs, connector metadata and audit evidence enter the primary trusted data store. | Dedicated DB identity, encrypted transport where networked, least privilege, migration control, backup/restore and retention. |
| API/worker -> file/object storage | Campaign attachments and generated evidence may contain personal or confidential content. | Private buckets/paths, encryption, scoped credentials, malware/content policy where required, lifecycle and coordinated restore. |
| API -> Redis -> worker | Queue messages and job identifiers cross from request processing to an asynchronous trust zone. PostgreSQL remains the durable business-state authority. | Private/authenticated broker, bounded payloads, idempotent claims, queue monitoring and worker isolation. |
| Worker -> SMTP/IMAP | Recipient addresses, message bodies and attachments leave GovOPlaN; IMAP append stores a sent copy externally. | Approved service account, TLS/CA policy, recipient/sender policy, rate limits, outcome reconciliation and disclosure/legal basis. |
| Connector worker -> CardDAV/WebDAV/Seafile/SMB/S3/CalDAV | Address data, files or calendars cross an organizational/system boundary in both directions depending on connector mode. | Explicit direction, scoped credentials, endpoint allow-list, provenance, conflict policy, retry/reconciliation and target interoperability test. |
| Operator -> installer/catalog/migration plane | A highly privileged process can mutate packages, schemas and desired module state. | Separate operator identity, signed/pinned catalogs, maintenance mode, immutable run records, backups, rollback checks and restricted network access. |
| API/audit -> monitoring or SIEM | Operational metadata and potentially personal audit context may leave GovOPlaN. | Data minimization, retention/access policy, authenticated transport, integrity and documented processor/location. No sink is verified today. |
Typical network requirements are inbound HTTPS to the deployment proxy and
private connectivity from API/workers to PostgreSQL and Redis. Outbound access
may be required to SMTP submission, IMAP, HTTPS-based connectors/object storage,
DNS, NTP, certificate validation, monitoring and the signed module catalog.
Ports and destinations must come from the selected infrastructure; common
defaults such as PostgreSQL 5432, Redis 6379, SMTP 465/587 and IMAP 993 are not
an allow-list.
## Known assumptions, gaps and risks
Facts:
- The production-like profile is a development validation composition, not a
production deployment artifact.
- The assessed package baseline is the trusted, signed stable catalog sequence
`202607220843`; it pins Core `v0.1.13` and Campaign `v0.1.10`.
- Workflow is postponed.
- The default composition does not enable Addresses.
- Health/readiness checks exist; an external monitoring stack does not.
Assumptions that require confirmation:
- The pilot can use local accounts and one internal tenant/office.
- A controlled non-production SMTP/IMAP account and safe recipients are
available.
- Campaign volume is below the measured limits of a single worker and database.
- Local durable storage is acceptable for the pilot and all recipient data has
an approved legal basis and retention policy.
Highest risks:
1. A signed, reproducible package selection can still be installed or configured
incorrectly and has not been accepted in the target environment.
2. Real SMTP/IMAP behavior, throttling and ambiguous outcomes have not been
proven against the target provider.
3. Backup/restore and disaster recovery are not demonstrated across database,
files, keys and deployment configuration.
4. External identity, monitoring, secret-store and reverse-proxy controls are
deployment gaps rather than GovOPlaN-delivered components.
5. Post-tag Calendar work is committed and pushed but remains outside the signed
stable package baseline; branch integration must not be mistaken for a
release.
6. Data classification, retention, recipient consent/legal basis and external
disclosure rules are not assessed for a concrete organization.
## Proof checks before promotion
1. Materialize the signed catalog into an isolated installation and rerun the
contract, migration, module-permutation, and focused acceptance gates against
the installed artifacts.
2. Run a target-like Campaign from import through validation, attachment build,
queue, SMTP acceptance, IMAP append, reporting and audit using safe data.
3. Drill transient SMTP failure, accepted-but-local-commit-lost uncertainty,
worker restart, Redis interruption and manual reconciliation without a
duplicate send.
4. Restore PostgreSQL, managed files, configuration and encrypted credentials
into an isolated environment; measure RPO and RTO.
5. Validate proxy/TLS, CORS, cookies, headers, body limits, account bootstrap,
maintenance access and secret redaction.
6. Measure representative recipient/file volume, queue age, database growth,
send rate and provider throttling; set capacity and alert thresholds.
7. Confirm audit/retention/privacy/legal requirements and any SIEM or archive
export before using production personal data.
## Reusable assessment questionnaire
Answers should contain no secrets and no unnecessary personal data. Each answer
must be marked `answered`, `assumed`, `not_applicable`, or `not_assessed` and may
link evidence.
### Scope and outcomes
- What outcome and reference journey must GovOPlaN support?
- Which users, roles, tenants, organization units and delegated functions take
part?
- Which journey steps are mandatory, optional, manual, external, or explicitly
postponed?
- What constitutes pilot success and production acceptance?
### Data, privacy, records and policy
- Which data classes enter database, files, messages, calendars, addresses,
logs and audit evidence?
- What are the legal basis, purpose, minimization, access, residency, retention,
deletion, archive and legal-hold requirements?
- Which external systems/processors receive data, and in which jurisdictions?
- Which decisions require four-eyes approval, explainability or immutable
evidence?
### Identity and integrations
- Are local accounts acceptable, or are OIDC, SAML, LDAP/AD or SCIM mandatory?
- What are the MFA, joiner/mover/leaver, service-account and break-glass rules?
- Which SMTP/IMAP, file/DMS, CardDAV/CalDAV, ERP, API or event endpoints are in
scope? Record protocol/version, direction, auth, CA, rate limit and owner.
- Which integrations may be unavailable, and what degraded/manual behavior is
acceptable?
### Workload and growth
- Tenants, named/active/concurrent users and peak requests?
- Campaigns per period, recipients per Campaign, send window, attachment sizes,
import size and retry peak?
- Managed files, database and audit volume now and over the retention window?
- Connector batches, queue depth/age, scheduled jobs and external rate limits?
### Availability, hosting and operations
- Required service hours, planned maintenance, availability, RPO and RTO?
- Hosting, network zones, egress/proxy/DNS/NTP/CA, data-residency and
air-gap constraints?
- Who operates PostgreSQL, Redis, storage, TLS, identity, secrets, monitoring,
backups and incident response?
- What deployment, patch, migration, rollback, restore and DR drills must pass?
### Procurement and decisions
- Required open-source/license, support, accessibility, security, certification,
interoperability and procurement conditions?
- Which requirements are blockers, accepted risks, residual risks or later
roadmap work?
- Who owns each decision, evidence item and next review date?
For every capability and infrastructure conclusion, record the requirement,
status from the controlled vocabulary, evidence scope, evidence links,
assumptions, gaps, risks, recommendation and proof check. Reassessment must pin a
new composition and review any row whose code, evidence, configuration or target
requirement changed.
### Repeatable release rerun
The release-aware rerun tool validates the machine-readable assessment against
its JSON Schema, verifies the catalog's Ed25519 signature against an
independently pinned local trust keyring, and separately verifies the canonical
hash of the published or candidate keyring pinned by the signed catalog. The
downloaded keyring is therefore never its own sole trust root. The tool compares
catalog sequence and module versions and—when local checkouts are available—checks
annotated release tags against assessed commits. For repositories represented in
signed `release.selected_units`, both the peeled commit and annotated tag-object
ID must match exactly; re-annotating an unchanged commit requires review.
Release drift produces a machine-readable list of affected capability and
infrastructure conclusions instead of silently carrying their prior status
forward:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--public \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json
```
Use `--catalog CATALOG --keyring PUBLISHED_OR_CANDIDATE_KEYRING
--trusted-keyring PINNED_LOCAL_KEYRING` for an offline or release-candidate
rerun. `GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE` may provide the
independently managed local trust-root path instead. During key rotation this
local file may contain both current and `next` keys within their validity
windows. Only `active` and `next` entries are trusted; unknown statuses,
duplicate key IDs and malformed structured keyrings fail closed, while revoked,
disabled and retired entries are ignored. Do not establish this trust file by
downloading it in the same rerun. Public JSON responses are bounded to 16 MiB.
Use `--json` or `--output PATH` for automation. Evidence and report files are
written through a bounded same-directory atomic replacement, with mode `0600`
and file and directory `fsync`. All parent directories must already exist, no
parent component may be a symlink, and an existing symlink or non-regular output
target is rejected.
Exit status `0` means the assessed release metadata is current, `2` means
explicit review is required, and `1` means the schema or trust checks are
blocked. The machine report distinguishes independent signature trust,
published-keyring hash agreement, and local-tag proof. Local-tag proof is only
marked checked when every composition entry is available for comparison and was
attempted. A successful rerun proves only the assessment schema, signed catalog
metadata, published-keyring integrity and optional local tag provenance; it does
not prove installed artifacts, target providers, target infrastructure, or
production fitness. Those remain separate proof checks above.
### Installed-composition evidence
The same rerun can inspect the Python environment in which it executes and bind
that observation to the assessment and signed catalog:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--public \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--collect-installed-evidence /var/tmp/govoplan-installed-evidence.json \
--output /var/tmp/govoplan-fit-review.json
```
The collector follows the strict version `0.4.0`
[`installed-composition-evidence.schema.json`](installed-composition-evidence.schema.json)
contract. It enumerates all installed distributions whose normalized name starts
with `govoplan-`, compares the enabled assessed package and module-manifest
versions, and identifies missing, duplicate and extra GovOPlaN distributions.
Those differences produce stable review targets for the affected composition
entries and evidence-backed conclusions.
For each distribution, the collector also:
- reads the installed wheel `RECORD` file directly, with a 4 MiB metadata bound,
rather than relying on an interpreted distribution file list; missing files,
duplicate or malformed declarations, mismatched declared sizes and hashes all
prevent a `verified` result;
- verifies every supported SHA-256 wheel-payload declaration in that local
metadata, within fixed limits of 256 GovOPlaN distributions, 10,000 files and
512 MiB hashed per distribution, 50,000 files and 2 GiB hashed for one
collection, and 64 MiB for any single file;
- accepts package-owned data-file targets below the installation prefix and
declared `console_scripts`, while rejecting other traversal, undeclared
scripts, paths outside the prefix and leaf symlinks before opening a file;
- reports pip-generated, unhashed `__pycache__/*.pyc` entries separately only
when their derived source is a hashed payload entry, including generated
files below a confined package-owned runtime-data prefix;
- labels every PEP 610 observation with the explicit
`local-pep610-metadata` basis and distinguishes coherent declared Git commits
and archive hashes from editable installs, local directories,
package-index/unknown origins and malformed metadata;
- compares a non-editable VCS commit with the assessment commit and, when one is
present, the signed catalog `selected_units` commit;
- derives two content identities from bytes it actually verifies: an
install-stable identity for immutable wheel-declared files and a full
installed-RECORD identity, while retaining only SHA-256 digests and counts;
- records only package/module identifiers, versions, bounded counters, hashes
and stable error codes. It never writes direct URLs, paths, hostnames,
usernames, exception text or file contents.
`RECORD` `verified` means complete matching coverage of the wheel payload
declared by the local metadata, not every runtime byte and not a release-origin
binding. The proof scope exposes the aggregate count of generated unhashed
bytecode; runtime activation remains unchecked. `RECORD` agreement proves that
payload files agree with installed metadata, but does not make self-consistent
metadata a trusted release origin. Similarly, a version string is not artifact
integrity. PEP 610 VCS metadata is accepted only for
`vcs: git`, a full 4064 hexadecimal commit, one mutually exclusive provenance
form and a coherent bounded URL shape. Editable and directory-backed installs
remain mutable even when their small editable-install `RECORD` is valid. Archive
or index artifacts without a hash anchored by the assessed release remain
unanchored. A matching declaration is reported under
`installed_source_provenance` as local consistency only, with
`release_origin_bound: false` until the separate origin proof succeeds. A signed
catalog can contain `release.artifacts` identities computed directly from built,
bounded wheel files. Wheels with installer-transformed scripts or headers are
marked `requires_installer_receipt`; catalog metadata alone cannot attest those
post-install bytes. A role-scoped installer receipt instead binds the exact
installed-evidence digest, full installed payload identities, catalog archive
digests and canonical signed-catalog digest. The tool never accepts an installed
hash merely because the installed evidence asserted it.
Only evidence produced in-process by `--collect-installed-evidence` is a local
observation, and it must be no more than five minutes old (with at most 30
seconds of future clock skew) at the selected verification time. Use
`--installed-evidence PATH` to compare evidence collected in a separate
environment. Imported JSON without a valid independent installer receipt
carries a blocker: it can identify differences but cannot establish checked or
valid installed proof. A receipt signed by a separately provisioned installer
authority authenticates that exact imported evidence across the process
boundary. The proof scope records observation mode, collection/evaluation time,
receipt authentication, freshness, and whether evaluation used current time or
an explicit historical override. Evidence input and output are bounded to 4
MiB. Collection loads the `govoplan.modules` entry-point factories in order to
read module IDs and manifest versions; that executes installed GovOPlaN manifest
code. Run it only inside the installation being assessed and with the same
isolation expected for other installed-artifact acceptance checks. This
collector does not observe
which modules a running service activated, migrations, configuration, health,
or reference-journey behavior. Runtime activation therefore remains an explicit
unchecked boundary.
### Target, provider and production proof boundary
Installed evidence cannot approve a target environment, an external provider,
or production use. These scopes use a separate, expiring
[`capability-fit-boundary-evidence.schema.json`](capability-fit-boundary-evidence.schema.json)
bundle. The bundle is bound to the assessment ID, assessment release and exact
installed-evidence SHA-256 digest. It contains only opaque subject/control/result
IDs and content hashes, not endpoints, credentials, people or raw result files.
Boundary evidence is accepted only when at least one Ed25519 signature validates
against a separately provisioned
[`capability-fit-proof-authority-keyring.schema.json`](capability-fit-proof-authority-keyring.schema.json).
Each authority key explicitly lists the scopes it may attest. Target and provider
claims use `passed` or `failed`; production claims use `approved` or `rejected`.
One claim per scope, unique control/artifact IDs, `issued_at < expires_at`, current
validity and exact digest binding are mandatory. Any schema, binding, time,
signature or authority blocker leaves every supplied boundary claim unchecked;
one malformed authority member or invalid validity interval invalidates the
supplied authority set as a whole. An authorized negative result is checked but
requires review only after every prerequisite, including installed
release-origin binding, is established.
Signatures cover UTF-8 JSON with the `signatures` member omitted, object keys
sorted, compact `,`/`:` separators and non-ASCII characters escaped, matching
the tool's deterministic canonicalization.
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--public \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--installed-evidence /srv/govoplan/evidence/installed.json \
--boundary-evidence /srv/govoplan/evidence/target-proof.json \
--boundary-authority-keyring /srv/govoplan/trust/proof-authorities.json \
--expected-external-provider-subject provider-production
```
With `--installed-evidence`, this command performs comparison and proof-binding
diagnostics. Without an installer receipt, the imported document remains
unsigned, so neither it nor the boundary claim becomes accepted proof. Direct
in-process collection can match catalog-anchored artifact identities for wheels
without installer-transformed files. The installer-receipt flow below
authenticates a durable cross-process observation and is required for transformed
script/header payloads.
Issue a receipt only in the installation process performing the live collection.
The command refuses evidence supplied from a file and first validates the
assessment, independently trusted signed catalog, composition and RECORD bytes:
```bash
./.venv/bin/python tools/assessments/installer-receipt.py \
--assessment /srv/govoplan/assessment.json \
--catalog /srv/govoplan/catalogs/stable.json \
--keyring /srv/govoplan/catalogs/keyring.json \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--receipt-id install:production:20260722 \
--signing-key installer-production=/run/secrets/installer-ed25519.pem \
--python-artifact govoplan-core=/srv/govoplan/staged/govoplan_core-0.1.13-py3-none-any.whl \
--python-artifact govoplan-campaign=/srv/govoplan/staged/govoplan_campaign-0.1.10-py3-none-any.whl \
--evidence-output /srv/govoplan/evidence/installed.json \
--receipt-output /srv/govoplan/evidence/installer-receipt.json
```
Repeat `--python-artifact PACKAGE=/exact/consumed.whl` for every enabled
GovOPlaN distribution. The issuer reopens each exact wheel, verifies its archive
and payload identities against the signed catalog, and refuses a different
build with the same name and version. Receipt issuance must follow collection
within five minutes, and live verification must still fall inside that bounded
window; retain the pair for a reproducible historical review at its explicit
verification time.
Verify that durable pair with its separately managed trust root:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--catalog /srv/govoplan/catalogs/stable.json \
--keyring /srv/govoplan/catalogs/keyring.json \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--installed-evidence /srv/govoplan/evidence/installed.json \
--installer-receipt /srv/govoplan/evidence/installer-receipt.json \
--installer-authority-keyring /srv/govoplan/trust/installer-authorities.json
```
Target-environment and production-approval claims must use the assessment's
`deployment_profile.id` as `subject_id`. External-provider claims require the
operator to supply a bounded opaque expected subject with
`--expected-external-provider-subject`; without it, such a claim remains
unchecked and blocks. Expected and observed IDs are retained in proof scope.
Both authority keyrings are governance trust roots. Installer receipt keys use
the strict
[`installer-receipt-authority-keyring.schema.json`](installer-receipt-authority-keyring.schema.json)
contract and may attest only `installed_release_origin`; their public material
must not be reused by catalog or boundary-proof authorities. Do not download or
generate them from the proof bundle being checked. The checker rejects
proof-authority public keys reused by either the published or independently
trusted catalog keyring, and rejects the same proof public-key material assigned
to multiple authority IDs. A malformed key, an invalid or empty validity
interval, or ambiguous key
identity invalidates the supplied authority set. Authority `not_after` is
exclusive. A target operator or approver must validate the referenced
drill/result artifacts before signing. The rerun verifies the
attestation and its bindings; it does not fetch or reinterpret those artifacts.
`--verification-time` exists only for reproducible historical review. Supplying
it always marks the machine and human report as a **HISTORICAL override**; callers
cannot relabel it as current. Live admission must omit it and use the actual
current time.
No boundary bundle or production authority has been supplied for this current
assessment. Target environment, provider and production proof therefore remain
explicitly unchecked rather than inferred from the local GreenMail journey,
source tests or signed release metadata.
## Evidence used in this slice
- [Production-like profile](../dev/production-like/README.md) and
[Compose dependencies](../dev/production-like/docker-compose.yml)
- [Module contracts and install boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Core deployment operator guide](https://git.add-ideas.de/add-ideas/govoplan-core/src/branch/main/docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [Ops scalability profiles](https://git.add-ideas.de/add-ideas/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md)
- Actual module manifests in the pinned repositories and the static contract
checker in this meta repository
- Core module-system/API smoke/auth/install-config tests, plus focused Campaign,
Files, Mail, Audit and Addresses tests
- [Campaign delivery runbook](https://git.add-ideas.de/add-ideas/govoplan-campaign/src/branch/main/docs/CAMPAIGN_DELIVERY_RUNBOOK.md)
- [Live signed stable catalog](https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json)
and [published keyring](https://govoplan.add-ideas.de/catalogs/v1/keyring.json),
verified against a separately provisioned local trust keyring
- Annotated source tags `govoplan-core/v0.1.13` and
`govoplan-campaign/v0.1.10`, including their catalogued Python and WebUI refs
- Installed-composition, boundary-proof and independently scoped proof-authority
schemas plus their deterministic review tests; no current target or production
proof bundle is asserted
Checks retained from the first assessment slice against its then-current
workspace:
- static manifest/interface contract scan: 43 contracts, 29 providers, no issues
- selected Core module-composition, auth/CSRF, health, Campaign journey,
reconciliation and install-configuration tests: 14 passed
- Campaign tests: 14 passed
- Files tests: 14 passed
- Mail tests: 22 passed
- Audit tests: 5 passed
- Addresses tests: 14 passed (one non-failing SQLite resource warning)
- JSON Schema validation of the machine-readable assessment: passed
Refresh checks on 2026-07-22:
- live stable catalog sequence `202607220843`: valid Ed25519 signature trusted
through `release-key-1`, no validator warnings
- current static contract scan: 43 modules, 33 providers, 19 requirements, no
contract errors
- catalog-selected commits: Core `d487726f4d2c` at `v0.1.13` and Campaign
`735e874bd03c` at `v0.1.10`; local and remote release refs agree
- Campaign's immutable `v0.1.10` tag object and peeled commit exactly match the
remote tag and catalog record; Calendar durable outbox work is committed and
pushed after catalogued `v0.1.8`
- JSON Schema validation of the refreshed machine-readable assessment: passed
These checks are evidence for the rows above; they are not a substitute for the
installed-release and target-environment proof checks.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,77 @@
# GovOPlaN Frontend Layout Principles
GovOPlaN modules should choose their page layout by the kind of work the user is
doing, not by the repository that owns the feature.
These concise layout choices are one canonical input to the broader
[`INTERFACE_PATTERN_LANGUAGE.md`](INTERFACE_PATTERN_LANGUAGE.md). The current
route and rollout evidence lives in
[`INTERFACE_SURFACE_INVENTORY.md`](INTERFACE_SURFACE_INVENTORY.md).
## Structured Data Directories
Use a full-available-space workspace for structured data directories: files,
addresses, calendars, records, mailboxes, document stores, and similar domains
where the primary task is browsing, selecting, filtering, inspecting, and acting
on related objects.
Principles:
- The module route should use the full available content area.
- Do not add a separate page heading row above the main workspace.
- Prefer persistent navigation panes, such as tree panels, source panels, folder
panels, calendar list panels, or mailbox folder panels.
- Keep collection navigation and collection-level actions close to the relevant
pane header.
- In a list-detail workspace such as Scheduling, keep related lists stacked in
the left pane and use the remaining main pane for view/create/edit. A single
Add action stays in the relevant list-pane header and opens the common main
editor; it does not create an additional menu or launcher.
- Use bounded widths for navigation/list panes and let the main detail/content
pane take the remaining space.
- Keep filtering controls inside the pane they affect.
- Use overlays, toasts, or floating alerts for transient messages so the
workspace height does not change.
This pattern is appropriate when the user is working inside one coherent data
domain and needs spatial continuity.
## Workflow And Configuration Surfaces
Use the standard heading/menu/card visual language for workflow structures,
settings, administration, dashboards, and pages that collect essentially
unrelated areas.
Principles:
- A page heading and subnavigation are appropriate when the page explains a
task, workflow stage, or administrative area.
- Cards are appropriate for repeated independent panels, settings groups,
summaries, and dashboard widgets.
- Collapsible panels and segmented controls are appropriate when a dense
configuration area needs controlled disclosure.
- A collapsible card whose sole content is a table gives that table the full
available card body; avoid nested cards, duplicate padding, inner max-widths,
and nested scrolling.
- Avoid forcing workflow/configuration pages into a file-explorer style unless
the primary interaction is genuinely directory browsing.
This pattern is appropriate when the user is comparing or configuring separate
concerns rather than navigating one structured object space.
## Shared Components
Reusable layout components belong in `govoplan-core` WebUI. Modules may consume
shared components from core, but must not import another module's private UI
components directly.
When a module-specific component becomes generally useful, promote it to core
with a parameterized API before reusing it elsewhere.
Non-self-explanatory fields use Core `FieldLabel`; documented omissions must
name their accessible-label source. Explicit Discard and dirty navigation use
the same Core unsaved-changes dialog. Table action sets retain unavailable row
actions as disabled controls and reserve empty-state slots so Add remains
aligned. Use central feedback/dialog components; `window.alert` is not an
authorized product surface unless a product-owner-approved exception is first
recorded in the Core decision ledger.

View File

@@ -140,6 +140,7 @@ Use one `type/*` label:
- `type/bug` - `type/bug`
- `type/feature` - `type/feature`
- `type/user-story`
- `type/task` - `type/task`
- `type/debt` - `type/debt`
- `type/docs` - `type/docs`

View File

@@ -0,0 +1,479 @@
# GovOPlaN Interface Pattern Language
This document is the cross-repository pattern language for GovOPlaN user
interfaces. It turns the existing ethical doctrine, binding UI/UX decisions,
layout principles, and module boundary into a common composition and review
grammar. It does not replace those sources.
The companion [interface surface inventory](INTERFACE_SURFACE_INVENTORY.md)
records which surfaces the current code contributes and where each surface
enters the rollout.
## Source Of Truth And Precedence
Use the narrowest owning document when changing a rule:
1. `govoplan-core/docs/INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` owns why a
consequential interface must preserve context, decision, consequence,
responsibility, contestability, and traceability.
2. `govoplan-core/docs/UI_UX_DECISION_LEDGER.md` owns accepted product decisions
such as progressive disclosure, adaptive forms, blocker language, guided
operations, and the platform theme contract.
3. `docs/FRONTEND_LAYOUT_PRINCIPLES.md` owns the high-level choice between a
full-space structured-data workspace and a heading/menu/card workflow or
configuration surface.
4. `govoplan-core/docs/MODULE_ARCHITECTURE.md` owns the shell, route, navigation,
UI-capability, and shared-component boundaries.
5. This document owns the common pattern names, placement grammar, wording and
state conventions, focused-view composition, and definition of done across
those sources.
If two rules appear to conflict, do not create a third local convention. Record
the conflict in the owning decision ledger, resolve it there, and update the
affected patterns and surfaces together.
## Product Contract
GovOPlaN should feel calm because it shows what is relevant to the task, not
because it hides authority, risk, or evidence. Every surface follows these
rules:
- Start with the user's current object and task.
- Show common actions before advanced controls.
- Keep context, status, problems, and the next action spatially connected.
- Make consequential effects explicit before execution and observed effects
inspectable afterwards.
- Treat permissions, policy, privacy, and module availability as behavior, not
decoration.
- Keep optional modules optional. Compose through core route and UI-capability
contracts, never sibling-private components.
- Use the centrally exported core components wherever a matching contract
exists. A module-local replacement is not an implementation choice: it is a
product exception that requires explicit product-owner authorization.
- Preserve a stable way back to the containing object and the broader system.
- Do not let navigation, selection, or a view switch imply consent.
## Surface Archetypes
Choose an archetype from the task, then specialize it for the domain. A route
may contain more than one bounded archetype, but it should have one dominant
one.
| Archetype | Use when | Standard anatomy | Do not use when |
| --- | --- | --- | --- |
| Directory or explorer | Users browse hierarchical collections such as files, mailboxes, calendars, addresses, or records. | Collection/source pane, collection actions, filter in the pane it affects, main list/content pane, optional detail pane. | The content is a set of unrelated settings or workflow stages. |
| List-detail workspace | Users repeatedly find objects, inspect one, and act without losing list context. | Search/filter/list, persistent selected-object context, detail/actions, stable selection and URL. | A single guided operation is the primary task. |
| Focused task view | Only a bounded composition is needed to complete one task. | Task identity and reason, selected object, necessary module regions/actions, progress or status, obvious exit to the full system. | Hiding a surface would obscure a consequence, blocker, or required evidence. |
| Create or edit | Users change one coherent object state. | Adaptive typed form, field-level validation, advanced section, save/cancel, unsaved-change guard. | Discovery-heavy setup or a broad consequential change needs staged review. |
| Guided setup or import | The user must discover, upload, map, test, or preflight before a safe result exists. | Named steps, current progress, preserved inputs, validation/problem list, review, resumable completion where work is durable. | An ordinary edit can be understood as one coherent form. |
| Review or decision | A person must inspect evidence and deliberately approve, reject, send, publish, or otherwise commit. | Decision context, evidence/problems, consequence and reversibility, authority/provenance, explicit action, resulting record. | The interaction is passive inspection. |
| Monitoring, progress, or report | Users observe asynchronous or aggregate state and intervene when needed. | Summary, filters, durable job/item state, last update, retry/reconcile/intervention, detail and evidence. | A toast is sufficient for a short, non-durable local action. |
| Administration or configuration | Users compare and configure separate concerns. | Heading and scope, grouped subnavigation, overview/list plus selected details, adaptive editor or guided risky operation, effective policy and source. | The primary task is browsing one structured object space. |
| Dashboard | Users need a task-oriented starting point across modules. | Prioritized actionable widgets, scoped status, clear destination per widget, explicit refresh/staleness. | It merely duplicates every module navigation item or metric. |
| Public service or entry | An unauthenticated or external participant starts or resumes a service. | Service identity, eligibility/context, privacy and evidence expectations, accessible form/task, save/resume or handoff. | The actor is performing internal administration. |
## Placement Grammar
### Shell And Navigation
- The global title bar and rail belong to core. A module contributes routes,
navigation metadata, and explicit UI capabilities; it does not reproduce the
shell.
- Global navigation answers "which service area?" Local subnavigation answers
"which stable facet of this object or area?" A progress indicator answers
"where am I in this operation?" Do not use those three controls
interchangeably.
- Keep the current tenant, actor, object, and selected version or scope stable
across local navigation. Guard unsaved work before navigation.
- Permission and capability filtering happens before view composition. A
missing menu item is not evidence that an actor lacks backend access, and a
visible item is never authorization by itself.
- Route, selection, panel, and view changes must not execute consequential
actions.
### Page And Workspace
- Structured directories use the full available content space and persistent
panes. They do not add a decorative heading row that reduces working height.
- Workflow, configuration, dashboard, and explanatory pages may use a heading.
The heading names the task or scoped object and contains only route-level
actions.
- Put a collection-wide create action in the heading of the collection it
affects. Use a short, specific label such as `Add` when the heading already
names the object. Do not duplicate that action in a permanently visible side
panel. A side panel used as the creation surface appears for creation and is
otherwise absent or returns to its documented non-creation purpose.
- Put filters beside the list or pane they affect. Put bulk actions immediately
above or beside the current selection. Put object actions with the object
detail, not in the global title bar.
- Full-page create and edit surfaces put their persistent action cluster in the
upper-right of the page heading. `Discard` comes before `Save …`, with the
primary save action at the far right. Keep both controls in the same place
across validation, loading, and saved states; guard unsaved work when the
user discards or navigates away. Explicit Discard and dirty in-application
navigation use the same central unsaved-changes dialog and registered
save/discard callbacks. A browser-controlled tab/window unload warning is the
only unavoidable different surface.
- A page or panel has one visually primary action for its current state. Put
secondary actions beside it. Separate destructive actions and name their real
effect.
- Dialog actions use a stable footer: cancel/back first, then the primary action
at the end. Header and footer stay fixed while long bodies scroll.
- Use cards for independent groups, summaries, and settings blocks. Do not wrap
every region in a card or nest cards merely to create spacing.
- When a collapsible card contains one table and no other content, the table
uses the card's full available width and body height. The card/table region
owns overflow; do not add an inner max-width, decorative wrapper, duplicate
padding, or nested scroll container that reduces the working area.
- Row actions in tables are icon-only controls in a stable rightmost action
column. Order them by intent: inspect/open, edit, copy/duplicate,
transfer/share/download, retry/restore, then remove/delete last. Omit actions
only when they are structurally irrelevant to the entire table. An action
that belongs to the table but is unavailable for one row remains in its
normal position and is disabled; when the reason is not obvious from row
state, provide it through the central focusable disabled-action explanation.
It does not disappear. Every icon has a translated accessible name and matching tooltip.
Separate destructive actions visually, and use a named confirmation/review
surface when the consequence cannot be understood from the icon and row
context. In an empty editable table, place Add in the same left-most action
slot it occupies in a populated row and reserve the remaining slots so the
column geometry does not move.
- Transient feedback should not shift the workspace. Durable failures, partial
results, and blockers remain attached to the affected item or operation.
### Detail And Explanation
- Keep the selected object's identity and material status visible while its
detail changes.
- Every non-self-explanatory field uses the central `FieldLabel`; short field
help sits with that label. A field without `FieldLabel` is an explicit
documented omission whose register names the field, rationale, and accessible
label source. Users may hide inline help markers with their persisted
interface preference; the visible/accessibility label and validation remain.
Longer "Why?", policy source,
diagnostics, or provenance belongs in an expandable area, detail panel, or
review step.
- Empty space is not an error. An empty state states what is empty, why that can
happen, and the permitted next action. Do not show creation actions to actors
who cannot create.
## Visual Grammar
- Core owns the appearance contract and shared CSS tokens. Modules use core
colors, spacing, radii, shadows, focus treatment, status colors, and disabled
treatment; module CSS may specialize layout only.
- Establish hierarchy through spacing, typography, grouping, and placement
before adding borders or color.
- Color never carries status or required action alone. Pair it with text and,
where useful, an icon.
- Icons support recognition but do not replace accessible names. Use the core
icon-name mapping for navigation.
- Keep list columns, tree indentation, headers, dialog dimensions, and action
positions stable as content changes.
- Density is a user preference, not a license to remove labels, focus targets,
explanations, or consequences.
- Respect system/light/dark themes and reduced-motion preferences through the
core contract. Do not build module-local theme systems.
## Wording Grammar
Use the same noun for the same domain object in navigation, headings, fields,
actions, states, API-facing explanations, and documentation. Prefer the most
specific user-facing noun: "Recipients" rather than "Data", "Delivery job"
rather than "Process", and "Mail profile" rather than "Configuration" when
that is what the user is acting on.
Actions use a verb plus the object or consequence:
- Prefer `Save campaign`, `Review messages`, `Queue delivery`, `Retry failed
deliveries`, or `Delete calendar`.
- Avoid `Submit`, `OK`, `Continue`, or `Execute` when a more precise action is
available.
- Use `Continue` only when it advances a reversible guided flow without
committing the final effect.
- Do not say `Undo` when the system can only cancel future work, create a
correction, supersede a record, or request retraction.
State text describes observed state, not optimism. Use stable shared terms where
they fit: `Draft`, `Ready for review`, `Blocked`, `Queued`, `Running`,
`Retry scheduled`, `Partially completed`, `Completed`, `Failed`, and
`Cancelled`. Domain-specific states may refine these terms but should not give a
shared term a contradictory meaning.
Blocked and failed actions use the structured language from DUE-005:
- what is unavailable or failed
- why, in plain language
- what must happen next
- who can do it
- where to go
- optional technical details behind deliberate disclosure
Errors should identify the affected object and whether saved state or external
effects may already exist. Never expose raw exception text as the only user
message.
Use the central dialog, confirmation, alert, and attached-error components for
feedback. `window.alert` and the global `alert` function are prohibited. A
genuinely unavoidable exception requires explicit product-owner authorization
and an entry in the Core alert exception register before implementation.
## State Contract
Every surface implements the states it can reach; it does not render a blank
region while waiting or collapse distinct outcomes into a generic error.
| State | Required treatment |
| --- | --- |
| Loading | Keep the stable shell and context visible. Name what is loading; preserve usable prior data when safe. |
| Empty | State the scope and reason, then show only permitted next actions. |
| Validation problem | Attach the problem to the field/item and provide a navigable summary when problems span regions or steps. |
| Permission denied | Name the unavailable action or object, the required actor/role where safe, and a valid exit. Do not leak protected data. |
| Capability unavailable | Distinguish not installed, disabled, not configured, unhealthy, and not permitted when the actor may know. Give the responsible actor and target. |
| Offline or unreachable | Preserve local context and unsaved input, show last-known/stale state, and offer a safe retry. Do not present network absence as an authentication failure. |
| Stale or conflicting | Show which data changed, preserve both values where feasible, and offer reload, merge, or explicit overwrite according to policy. |
| Partial result | Show completed and incomplete effects separately. Never label a partial operation successful without qualification. |
| Asynchronous work | Show durable job identity, queued/running/retry/block/final state, last update, progress if meaningful, and leave/return behavior. |
| Success | State the resulting object/effect and provide its evidence or destination. Use a transient toast only when the result is already visible and durable elsewhere. |
| Destructive or corrective action | Preview scope, downstream effects, reversibility limit, evidence, and required confirmation or approval. |
Long-running or external work must expose retry and reconciliation as observable
states. The UI must not imply that a request and its external effect were one
atomic success when an outbox, worker, or remote system sits between them.
## Consequence And Provenance
Before an action affects records, rights, policy, retention, communications,
money, external systems, or workflow state, its action surface must answer the
decision-surface questions in the interface doctrine. At minimum show:
- affected object and scope
- acting identity or system actor and relevant authority
- immediate and possible downstream effects
- whether the operation is reversible, cancellable, corrective, or final
- blockers and their resolution path
- audit/evidence that will be created
- effective policy or configuration source when it changes the decision
After execution, users must be able to reach the command/job, observed effects,
policy result, failures, retries, reconciliation outcome, actor, time, and source
data that explain the result. Provenance may be quiet by default, but it must not
be absent.
Privacy follows the same rule: lists, previews, logs, notifications, and
diagnostics show only the personal or secret data needed for the actor's task.
Redaction must be explicit enough that users do not mistake a redacted value for
missing source data.
## Focused Views
A focused view is a declarative UI composition for a task. It selects the
routes, local regions, navigation entries, and actions relevant to that task
after installed-module, capability, permission, and policy filtering. It does
not change backend authorization or domain state.
A view definition must be able to explain:
- its stable identifier, label, and task purpose
- the current object/scope and default destination
- included navigation and contributed regions/actions, with deterministic order
- the visible escape to the containing module and full system
- why the view is active and how the user may switch when switching is allowed
- what happens to unsaved work when entering, leaving, or switching views
Focused views follow these invariants:
- Do not hide a blocker, material consequence, provenance, or required review
merely to make the screen quieter.
- Preserve the global tenant/actor context and provide an obvious exit.
- Filter unavailable contributions without leaving broken separators, empty
groups, or dead destinations.
- A manual or automatic switch is navigation, not consent. Guard unsaved work
and never execute a domain action as a side effect.
- Display why a non-manual default was selected, for example a role or task
default, without exposing protected policy details.
- Treat an unknown or invalid view as a recoverable fallback to the normal
module surface.
When more than one source proposes a focused view, use this precedence:
1. a manual view pinned for the current user session
2. a current-task suggestion, including a future workflow-step suggestion
3. the user's saved default
4. the role or tenant default
5. the normal full interface
Show the active source and an escape to the full interface. A task or workflow
suggestion is never an authorization change and never locks the user into the
composition; tenant policy may constrain which views are selectable, but it
must not hide required evidence or remove that escape. Workflow implementation
is explicitly postponed and is not a prerequisite for defining, manually
selecting, testing, or piloting focused views. A future workflow module may
request a view through a core contract; it must not own the view composition
implementation.
## Accessibility And Responsive Contract
- Every action is reachable and operable by keyboard in a logical order.
- Use semantic headings, landmarks, labels, tables/lists, and native controls
before adding ARIA. Icon-only controls need stable accessible names.
- Focus is visible. Dialogs trap focus, announce their name, and return focus to
the control that opened them. Validation moves or links focus to the first
relevant problem without losing the problem summary.
- Loading, saved, failed, queued, progress, and externally updated states are
announced without repeatedly interrupting the user.
- Disabled primary actions need a focusable explanation; a pointer-only tooltip
is insufficient.
- Do not rely on color, hover, drag-and-drop, pointer precision, or animation as
the only interaction. Provide keyboard and explicit-control equivalents.
- At narrow widths and high zoom, preserve task order and action access. Collapse
secondary panes into an explicit drawer/step and never move a destructive
action into the primary position.
- Honor reduced motion. Avoid motion that implies progress when the operation is
merely waiting.
- Truncation has an accessible full-value path. Personal or secret values remain
redacted according to permission and policy in that path.
## Component Ownership
Core already exports shell, navigation, access-boundary, form, dialog, loading,
status, policy/provenance, blocker, review, table, tree, message-display, and
unsaved-change primitives. These centrally exported components are mandatory
across GovOPlaN wherever their contract covers the interaction. In particular,
use the core `Card` for logical sections, `DataGrid` for tabular collections and
row actions, and `ToggleSwitch` (the standard Toggle control) for boolean
settings. Styling a native element or a module-local component to imitate one
of these controls is duplication, not reuse.
A route or domain composition assembled from central primitives is not a custom
control. Any new reusable UI control, presentation primitive, or module-local
substitute is a custom component and requires explicit product-owner
authorization before implementation. Record the authorization in the owning
decision or issue together with:
- the narrowly defined purpose and consumers
- why no central component or composition satisfies the need
- the exact scope in which the exception may be used
- its accessibility, state, theme, and test contract
- whether it should remain domain-specific or later become a core component
An authorized custom component serves only that specific purpose. It must not
duplicate, fork, restyle into a substitute for, or silently broaden beyond a
central component. Code review convenience, an existing local implementation,
or a small visual difference is not authorization. When core gains the required
contract, migrate the exception unless the product owner explicitly retains it.
Do not promote a component only because two screens look similar. Promote it to
`@govoplan/core-webui` after a second consumer or a clear platform contract has
proved shared behavior, accessibility, state, and extension needs. Modules own
domain composition, wording, and policy semantics; core owns generic contracts
and appearance.
### Scheduling Request Composition Reference
The Scheduling request surface is the first explicit reference composition for
these rules:
- The persistent left panel contains `My scheduling requests` and `Scheduling
requests for me` as two stacked lists. It preserves list context like mailbox
folders but does not invent folders.
- The left pane's `Scheduling requests` heading owns one short `Add` action. It
opens a new request in the right pane without an extra menu or duplicate
launcher.
- The right main pane is the stable view/create/edit surface. Selecting a list
item opens its details; Add opens the same editor composition used for edit.
- `Basic information`, `Calendar integration`, `Candidate slots`, and
`Participants` are logical sections rendered with the central `Card`.
- Privacy and participation behavior is a separate settings `Card`; dependent
number/password fields are disclosed by their central `ToggleSwitch`.
- Candidate slots and participants are row collections rendered with the
central `DataGrid`, including its stable action column.
- Calendar integration is a boolean choice rendered with the central
`ToggleSwitch`; dependent calendar controls are disclosed only when enabled.
- Each participant is one structured row containing name, email address, and
ordered row actions. An address-parsing text area is not the ordinary editor;
parsing pasted address lists belongs only in an explicitly designed bulk
import flow.
- View mode shows participation statistics and only state-valid quick actions.
Scheduling owns those current domain actions; a future Workflow module may
coordinate them through stable action contracts but is not a runtime
dependency of the surface.
Apply the underlying placement and component rules to equivalent collection and
create/edit surfaces throughout the system; the Scheduling domain names are an
example, not a module-local convention.
## Test Expectations
For every changed surface, select tests from each applicable layer:
- route and permission tests: module enabled/disabled permutations, route guard,
contribution filtering, fallback, and direct-link behavior
- behavior tests: primary task, validation, unsaved-change guard, confirmation,
retry/reconcile, partial result, and leave/return behavior
- accessibility tests: semantic names/roles, keyboard order, focus entry/return,
live-state announcements, and non-color status meaning
- state tests: loading, empty, denied, capability-missing, stale/conflict,
offline, partial, success, and destructive/corrective outcomes as applicable
- composition tests: absent optional module, duplicate/unknown contribution,
deterministic ordering, and focused-view fallback
- presentation checks: supported widths/zoom, light/dark/system theme, reduced
motion, comfortable/compact density, and long translated text
- i18n checks: user-facing strings owned by the rendering package and structural
audits passing
- visual regression: useful for geometry and hierarchy after behavior and
accessibility assertions exist; never the only evidence
If the current harness cannot automate a required check, record the gap and the
manual evidence in the owning issue. "Not tested" is an inventory state, not a
reason to infer that a pattern is satisfied.
## Definition Of Done For A Surface
- The dominant task and archetype are recorded in the inventory.
- Placement, action hierarchy, wording, and every reachable state follow this
pattern language or an explicit ledger exception.
- Permission, capability, privacy, and redaction behavior are verified.
- Consequence, reversibility, authority, evidence, and provenance are present
where applicable.
- Keyboard, focus, announcement, responsive, theme, density, motion, and i18n
behavior are covered in proportion to the surface.
- Optional modules remain optional and no sibling-private UI import was added.
- Every matching central component is reused. Any custom-component exception
has recorded product-owner authorization, narrow scope, rationale, and tests,
and does not duplicate a central component.
- Behavioral/accessibility evidence is linked from the rollout matrix and issue.
- Configured-system help can reach the applicable pattern or reference topic
when [Docs #15](https://git.add-ideas.de/add-ideas/govoplan-docs/issues/15)
supplies that experience.
## First Pilot: Campaign
[Campaign #74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74)
is the first full-domain audit and migration. It should prove patterns before
generic extraction:
- [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) and
[#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73): stable,
accessible preview and attachment-detail overlays
- [#63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63): review
stages, outcomes, blockers, and intervention vocabulary
- [#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62): explicit
synchronous/asynchronous send mode and durable delivery progress
- [#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65): one
coherent report filtering and count-affordance model
- [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35): guided
first-campaign entry
These slices do not depend on the Workflow runtime. Campaign's current
domain-owned review/send state is enough to prove layout, wording, focused-view,
progress, intervention, and evidence patterns.
## Revision Procedure
1. Record a changed product decision in the core UI/UX decision ledger.
2. Update the relevant pattern here without copying the full owning doctrine.
3. Update the surface inventory and rollout owner/issues.
4. Change shared components only where the proven contract belongs to core.
5. Migrate and test affected module surfaces.
6. Publish configured-system pattern/reference help through the Docs module.

View File

@@ -0,0 +1,236 @@
# GovOPlaN Interface Surface Inventory And Rollout
This is the initial evidence inventory for the product-wide interface pattern
language. It records code contributions, not an assertion that every listed
surface is complete, enabled in a deployment, usable, or compliant.
The applicable design contract is
[`INTERFACE_PATTERN_LANGUAGE.md`](INTERFACE_PATTERN_LANGUAGE.md).
## Snapshot And Method
Snapshot refreshed: 2026-07-22.
Evidence was read from tracked Git `HEAD` in the local GovOPlaN checkouts:
- core routes and fallback behavior in `govoplan-core/webui/src/App.tsx`
- every present `govoplan-*/webui/src/module.ts`
- the matching backend `src/*/backend/manifest.py`
- named UI capabilities and contribution identifiers in each `module.ts`
- Campaign nested routes in its tracked `CampaignWorkspace.tsx` and
`SectionSidebar.tsx`
Tracked commits are used as the integrated baseline. Dirty worktree changes
are not treated as delivered behavior. The former Campaign recipient-editor
and preview WIP is integrated in tracked commits; completed work is no longer
described as a local exception.
Runtime visibility remains conditional on the module being packaged and
enabled, server metadata, installed optional capabilities, the authenticated
actor, tenant context, route permission guards, and inner-surface permission
checks. A route in this inventory therefore means "the code contributes this
route when its module is active", not "every user sees it".
Inventory states:
- **Contributed**: a route or named UI capability exists in tracked source.
- **Metadata gap**: frontend runtime source contributes a route but the backend
manifest does not describe the same route/navigation surface.
- **Unreviewed**: the surface has not yet completed the pattern, state,
accessibility, privacy, and consequence audit. This is the default unless an
issue supplies verification evidence.
- **Pilot**: the surface is in the Campaign-first rollout.
## Core Shell Surfaces
| Surface | Owner and code evidence | Audience/access evidence | Primary task and target archetype | Audit / rollout |
| --- | --- | --- | --- | --- |
| Public landing and login | `govoplan-core` `PublicLandingPage`; rendered while no authenticated principal exists | Unauthenticated; maintenance and backend-reachability context are shell inputs | Understand the service and authenticate; public entry | Unreviewed; later public-entry audit |
| Session/bootstrap state | `govoplan-core` `App.tsx` and `AppShell` | All browser sessions during bootstrap | Understand that session/platform state is loading; state contract | Unreviewed; core shell |
| `/` authenticated redirect | `govoplan-core` chooses the first visible navigation destination | Authenticated; result depends on visible nav contributions | Enter the actor's first accessible service area; navigation behavior, not a content page | Unreviewed; focused-view/default-route work must preserve this fallback |
| `/dashboard` fallback | `govoplan-core` `DashboardPage` only when the Dashboard module is absent | Authenticated; no route-specific scope in core | Cross-module starting point; dashboard | Unreviewed; compare with module dashboard before shared changes |
| `/settings` | `govoplan-core` `SettingsPage` | Authenticated; contributed sections and integrations filter internally | Profile, UI/workspace preference, local connection, and user-scoped integration settings; configuration | Unreviewed; Core #225 program |
| Shell chrome | `AppShell`, `Titlebar`, `IconRail`, `BreadcrumbBar`, `HelpMenu`, language menu, unsaved-change provider | Public/authenticated variants; nav filtered later | Tenant/actor context, global navigation, help, language, session and maintenance state | Unreviewed; platform-owned prerequisite for focused views |
## Direct Module Route Contributions
The access guard column reports only the route-level declaration in
`module.ts`. Inner APIs and controls may impose additional checks.
| Route | Owner / render evidence | Route-level access evidence | Primary task | Target archetype | Status / priority |
| --- | --- | --- | --- | --- | --- |
| `/admin` | `govoplan-access` `AdminPage` | Any core `adminReadScopes` | Administer system and tenant concerns assembled from module sections | Administration/configuration | Contributed; unreviewed; P1 under [Core #225](https://git.add-ideas.de/add-ideas/govoplan-core/issues/225) |
| `/address-book` | `govoplan-addresses` `AddressBookPage` | `addresses:contact:read` | Browse and manage contacts, address books, and lists | Directory/list-detail | Contributed; unreviewed; P2 after Campaign |
| `/calendar` | `govoplan-calendar` `CalendarPage` | `calendar:event:read` | Browse calendars/events and act on calendar data | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/campaigns` | `govoplan-campaign` `CampaignListPage` | `campaigns:campaign:read` | Find, compare, create, and open campaigns | List-detail entry | Pilot; P1 [Campaign #74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74) |
| `/campaigns/:campaignId/*` | `govoplan-campaign` `CampaignResourceRoute` and `CampaignWorkspace` | `campaigns:campaign:read`, plus resource probe | Configure, review, send, and inspect one campaign/version | List-detail workspace containing edit, review, monitoring, and evidence surfaces | Pilot; P1 Campaign #74 |
| `/operator` | `govoplan-campaign` `OperatorQueuePage` | `campaigns:campaign:read` and any of queue, control, retry, or reconcile | Monitor and intervene in campaign jobs through authority-specific controls | Monitoring/work queue | Pilot; durable queue controls delivered in [Campaign #78](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/78); #74 audit remains |
| `/reports` | `govoplan-campaign` `AggregateReportsPage` | `campaigns:report:read` | Compare privacy-protected cross-campaign outcome totals without recipient detail, diagnostics, export, or drill-down | Aggregate reporting | Pilot; aggregate-reader surface delivered in [Campaign #80](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/80); #74 audit remains |
| `/templates` | `govoplan-campaign` `TemplatesPage` | No route guard declared in `module.ts` | Browse/manage campaign templates | Directory/list-detail | Pilot audit; permission intent must be verified; P2 |
| `/dashboard` | `govoplan-dashboard` `DashboardPage` | No route-specific scope | Assemble module-provided actionable widgets | Dashboard | Contributed; unreviewed; P2 |
| `/docs` | `govoplan-docs` `DocsPage` | Docs read or system/tenant settings read scopes | Read configured, available, and evidence-aware documentation | Documentation directory/reference | Contributed; unreviewed; P1 [Docs #15](https://git.add-ideas.de/add-ideas/govoplan-docs/issues/15) after initial pattern content |
| `/files` | `govoplan-files` `FilesPage` | `files:file:read` | Browse folders/files and perform managed-file work | Directory/explorer | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/idm` | `govoplan-idm` `IdmPage` | Any IDM assignment/write or organization function-assign scope | Inspect and govern identity/function assignments | List-detail/configuration | Contributed; unreviewed; P2 |
| `/mail` | `govoplan-mail` `MailboxPage` | `mail:mailbox:read` | Browse mailboxes and messages | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/notifications` | `govoplan-notifications` `NotificationCenterPage` | `notifications:notification:read` | Inspect and acknowledge notification state | List-detail/inbox | Contributed without a nav item or backend frontend metadata; navigation intent unknown; P2 discovery |
| `/ops` | `govoplan-ops` `OpsPage` | Ops read or system/tenant settings read scopes | Inspect runtime health and readiness | Monitoring | Contributed; unreviewed; P2 |
| `/organizations` | `govoplan-organizations` `OrganizationsPage` | Organization model/unit/function or admin settings read scopes | Model and inspect organizational structures/functions | Directory/list-detail | Contributed; unreviewed; P2 |
| `/scheduling` | `govoplan-scheduling` `SchedulingPage` | `scheduling:schedule:read` | Plan and decide scheduling requests and availability | List-detail/guided decision | Contributed; metadata gap; unreviewed; P2 |
## Manifest And Runtime Route Alignment
Backend route metadata lets operators, Docs, release tooling, and remote bundle
loading reason about the configured interface without executing module UI code.
`module.ts` remains the executable local route/render source. Missing metadata
is recorded here as an evidence gap; this inventory does not infer whether each
gap is intentional.
| Module | `module.ts` routes | Backend manifest frontend routes | Backend nav alignment | Result |
| --- | --- | --- | --- | --- |
| Access | `/admin` | `/admin` | Aligned | Described |
| Addresses | `/address-book` | `/address-book` | Aligned | Described |
| Admin | No direct route; `admin.sections` | None | Not applicable | Composed surface |
| Audit | No direct route; `admin.sections` | None | Not applicable | Composed surface |
| Calendar | `/calendar` | None | `/calendar` nav exists | Metadata gap |
| Campaign | Five routes | None | Four top-level nav items exist | Metadata gap; wildcard resource route is also undescribed |
| Dashboard | `/dashboard` | `/dashboard` | Aligned | Described |
| Docs | `/docs` | `/docs` | Aligned | Described |
| Files | `/files` | None | `/files` nav exists | Metadata gap |
| IDM | `/idm` | `/idm` | Aligned | Described |
| Mail | `/mail` | None | `/mail` nav exists | Metadata gap |
| Notifications | `/notifications` | No frontend metadata | No nav item | Metadata and discovery gap |
| Ops | `/ops` | `/ops` | Aligned | Described |
| Organizations | `/organizations` | `/organizations` | Aligned | Described |
| Policy | No direct route; `admin.sections` | None | Not applicable | Composed surface |
| Scheduling | `/scheduling` | None | `/scheduling` nav exists | Metadata gap |
Before a release claims a complete configured-system route inventory, add a
contract check or explicit exceptions so executable routes and manifest
metadata cannot silently diverge.
## Composed Surfaces And Extension Points
These surfaces are active only when the host and contributing modules are
enabled and the actor passes the declared filters.
| Host surface | Contributor and evidence | Contributed regions/actions | Pattern implication | Audit |
| --- | --- | --- | --- | --- |
| `/admin` | Access host (`AdminPage`) | System tenants/users/roles, tenant users/groups/roles/API keys/settings, function-role mappings, user/group mail and file connector scopes | One stable admin information architecture must contain both host-owned and contributed sections | Unreviewed; P1 Core #225 |
| `/admin` | `govoplan-admin` `admin.sections` | Overview; system settings; configuration changes; configuration packages; role/group templates; module management | Configuration, guided operations, review/preflight, consequence | In progress under Core #225; surface-level evidence still needed |
| `/admin` | `govoplan-audit` `admin.sections` | System audit; tenant audit | Evidence/provenance and reporting | Unreviewed |
| `/admin` | `govoplan-files` `admin.sections` and `files.connectors` | System and tenant file connections plus scoped connector managers used by Access | Adaptive configuration, discovery/test, policy and credentials | First migration family in Core #225; verification incomplete in this inventory |
| `/admin` | `govoplan-organizations` `admin.sections` | Tenant organization settings | Configuration/list-detail | Unreviewed |
| `/admin` | `govoplan-policy` `admin.sections` | System, tenant, group, and user retention | Effective value, source/provenance, consequential configuration | Unreviewed; Core #225 phase 4 |
| `/admin` and `/settings` | `govoplan-mail` `mail.profiles` | System/tenant/group/user mail profile and policy managers | Same server/credential/policy grammar as file connectors | Unreviewed; Core #225 mail migration |
| `/settings` | Core host | Profile; interface; workspace; local connection | Personal configuration with adaptive forms and immediate feedback | Unreviewed |
| `/settings` | Files and Mail named capabilities | User-scoped file connections and mail profiles/policy | Optional integration regions disappear cleanly when capability absent | Unreviewed |
| `/settings` | `govoplan-notifications` `settings.sections` | Notification preferences | Personal configuration | Unreviewed |
| `/dashboard` | Dashboard host and `dashboard.widgets` | Installed-modules widget; Ops health widget when Ops contributes it | Widget ordering, staleness, permissions, destination behavior | Unreviewed |
| `/organizations` | IDM `organizations.functionActions` | Action leading to assignment view filtered by IDM scopes | Cross-module context action through explicit capability | Unreviewed |
| Campaign attachments/import | Files `files.fileExplorer` | Folder tree, managed chooser, file listing/pattern resolution/sharing | Optional domain composition without sibling-private imports | Pilot audit under Campaign #74 |
| Campaign review/send | Mail runtime `mail.devMailbox` | Mock-mail verification when backend advertises runtime capability | Optional review stage with unavailable/optional states | Pilot audit under Campaign #63/#62 |
Other named capability exports (`files.connectors`, `organizations.functionPicker`,
and mail profile validation) are contracts consumed inside the composed surfaces
above; they are not independent routes.
## Campaign Pilot Surface Map
Campaign is detailed first because it exercises almost every archetype. The
recipient-data editor is now consolidated into the `recipients` section on
remote `main`; [Campaign #67](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/67)
records the accepted and verified integration boundary.
Campaign already consumes core primitives including `ModuleSubnav`, `Card`,
`PageTitle`, `Button`, `LoadingFrame`, `DismissibleAlert`, `FormField`,
`StatusBadge`, `MetricCard`, `DataGrid`, `TableActionGroup`, `Dialog`,
`ConfirmDialog`, `FileDropZone`, `MessageDisplayPanel`, policy components,
access/module capabilities, and unsaved-navigation guards. Reuse alone does not
prove that the composition or states satisfy the pattern.
| Surface / code evidence | Primary task | Target pattern | Material consequence/state | Known issue / rollout |
| --- | --- | --- | --- | --- |
| Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | Audit in [#74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74); guided entry [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35) |
| Overview (`CampaignOverviewPage`) | Understand/edit campaign identity, version, access, lifecycle | Object overview plus adaptive edit | Lock/archive/delete/access changes need real consequence and reversibility wording | #74 remaining audit |
| Fields (`CampaignFieldsPage`) | Define recipient/template field schema | Structured editor | Schema changes can invalidate recipient/template data | #74 audit |
| Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74; attachment-detail [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) |
| Recipients (`RecipientDataPage`) | Select/import/map/edit recipients, address fields and per-recipient values/files | Import/mapping plus list-detail editor | Personal data, validation, bulk activation, file links | Consolidated editor delivered in #67; #74 remaining audit and guided entry #35 |
| Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74; stable overlay [#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73) |
| Mail settings (`MailSettingsPage` settings view) | Select/configure campaign mail transport | Adaptive configuration | Credentials, SMTP/IMAP destinations, test outcomes | #74; align with Core #225 mail pattern |
| Campaign settings (`GlobalSettingsPage` settings view) | Configure campaign behavior | Adaptive configuration | Can alter validation/build/send behavior | #74 audit |
| Mail policy (`MailSettingsPage` policy view) | Inspect/override effective mail policy | Effective policy/provenance editor | Inheritance and locks affect allowed delivery | #74; Core #225 policy pattern |
| Campaign policy (`GlobalSettingsPage` policy view) | Inspect/override campaign policy | Effective policy/provenance editor | Inheritance, actor authority, and blocked edits | #74; Core #225 policy pattern |
| Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, bounded synchronous execution, persisted queue mode, partial effects, retries, evidence | Bounded synchronous and explicit/persisted queued modes delivered in [#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62) and [#79](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/79); [#63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63) wording and #74 audit remain |
| Message and attachment detail overlays | Inspect one built/mock message and its attachment links | Stable detail/review dialog | Personal data, exact outbound content, reviewed state | Delivered and verified in [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) and [#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73) |
| Campaign report (`CampaignReportPage`) | Filter and inspect delivery outcomes | Reporting/list-detail | Partial, failed, explicitly excluded/skipped, SMTP/IMAP outcomes and retries | Server-owned filtering and counts delivered in [#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65) with the full-result DataGrid contract from [Core #263](https://git.add-ideas.de/add-ideas/govoplan-core/issues/263); excluded semantics in [#66](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/66) |
| Audit (`CampaignAuditPage`) | Inspect campaign evidence/history | Provenance timeline/report | Actor/action/effect trace | #74 audit |
| JSON (`CampaignJsonView`) | Inspect expert representation | Advanced diagnostics/reference | Raw data may contain personal/configuration values; not a primary editor | #74 privacy/redaction audit |
| Create wizard (`CreateWizard`) | Seed a campaign through basics, sender, fields, recipients, template, attachments, review, send | Guided setup | Current steps mix creation and later consequential delivery; completion semantics need audit | Guided first campaign [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35) |
| Review/send wizard routes | Alternate guided review/send shells | Guided review | Tracked routes exist; implementation relationship to `ReviewSendPage` must be established, not guessed | #74 inventory decision |
| Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Campaign/version/job identity, historical active-version discovery, fixed action positions, authority-aware disabled states, exact non-overlapping queue counts, server-paged jobs, bounded refresh, retry/queue/reconcile per version, campaign-wide pause/resume/cancel, and leave/return progress | Durable operator controls delivered in [#78](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/78); #74 wording/accessibility audit remains |
| Aggregate reports (`AggregateReportsPage`) | Compare cross-campaign delivery outcomes | Privacy-preserving aggregate reporting | Tenant/campaign ACL, deployment/tenant small-cell policy, complementary and overlapping-cell suppression, explicit denominator, and no recipient detail/diagnostics/export/drill-down | Separate aggregate-reader surface delivered in [#80](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/80); not parity with the permission-gated per-campaign detail report |
| Templates route (`TemplatesPage`) | Browse template records | Directory/list-detail | Template availability and later generated outputs | #74 audit; verify missing route guard intent |
The five review stages currently named in code are `Validate and inspect`,
`Build and review`, `Mock send and verify`, `Confirm and send`, and `Delivery
results`. Campaign #63 owns the intervention and status vocabulary; Workflow is
not required to define or implement it.
## Repositories Without A WebUI Route Contribution
The following local repositories contain a backend manifest but no
`webui/src/module.ts` at this snapshot:
`govoplan-approvals`, `govoplan-assets`, `govoplan-booking`,
`govoplan-certificates`, `govoplan-committee`, `govoplan-consultation`,
`govoplan-contracts`, `govoplan-dist-lists`, `govoplan-evaluation`,
`govoplan-facilities`, `govoplan-forms-runtime`, `govoplan-grants`,
`govoplan-helpdesk`, `govoplan-identity`, `govoplan-inspections`,
`govoplan-issue-reporting`, `govoplan-learning`, `govoplan-permits`,
`govoplan-poll`, `govoplan-procurement`, `govoplan-records`,
`govoplan-resources`, `govoplan-rest`, `govoplan-risk-compliance`,
`govoplan-soap`, `govoplan-tenancy`, and `govoplan-transparency`.
This is only negative route evidence. It does not classify the backend module's
maturity or decide that it needs a WebUI. Connector-only, capability-only, or
backend-only modules may remain intentionally headless.
## Rollout Matrix
| Order | Scope | Current evidence | Target | Owner / issue | Verification gate | Status |
| --- | --- | --- | --- | --- | --- | --- |
| 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/add-ideas/govoplan/issues/11) | Docs links/diff checks; issue/wiki sync after integration | Initial slice in this document |
| 1 | Campaign baseline integration | Recipient-editor WIP and tracker state have been reconciled with remote `main` | Integrated, testable baseline before migration claims | Campaign #67 and tracker cleanup | Backend and focused WebUI suites; issue evidence | Complete 2026-07-22 |
| 2 | Campaign previews/details | Stable shared dialog with bounded scrolling and fixed responsive preview workspace | Stable header/body/footer, accessible long-content detail | Campaign #59 and #73 | Review-preview and overlay structure tests | Complete 2026-07-22 |
| 3 | Campaign review/interventions | Five domain-owned stages with unresolved intervention language | Clear stages, outcomes, blockers, next actor/action, reviewed evidence | Campaign #63 | State matrix behavior/accessibility tests and agreed vocabulary | P1 needs product wording decision |
| 4 | Campaign send/progress | A hard deployment ceiling bounds synchronous delivery; the selected synchronous, worker-queue, or database-queue mode is explicit and persisted; progress and recovery survive navigation; immediate-send response and audit evidence are allowlisted | Pre-send mode/consequence plus durable leave/return progress, retry and reconciliation without recipient/provider leakage | Campaign #62 and #79 | Boundary/concurrency/preflight, async selection, persisted mode, sanitized response/audit, partial/failure/retry and reload/return tests | Complete 2026-07-22 (`7e16603`, `60efd1c`, `62a6879`, `b0282eb`, `f095a3e`) |
| 5 | Campaign report filtering | Core DataGrid distinguishes client/full-result from server-owned queries; Campaign applies filter/sort/count before pagination and synchronizes count shortcuts with the grid query | One shared server-owned status/list/filter/count model | Campaign #65 and Core #263 | DataGrid contract/build tests plus exact shortcut/query/filter/count and large-result behavior | Complete 2026-07-22 (`e6062fe`, `cece71d`, `aa4ec66`, `4eb651c`) |
| 6 | Campaign operator recovery | A durable campaign/version queue page exposes historical work, exact non-overlapping state counts, persisted mode, permission-safe controls, server-paged job evidence, bounded refresh and active-state recovery | Fixed-position actions, disabled explanations, leave/return state, version-scoped retry/queue/reconcile and explicit campaign-wide pause/resume/cancel | Campaign #78 | Queue model/structure, historical-version, permission, paging, recovery-control, stale-response and delta tests | Complete 2026-07-22 (`21f3014`, `99d44ee`, `735e874`) |
| 7 | Campaign aggregate reports | A separate aggregate-reader projection and UI expose only policy-suppressed business totals with a stable status domain | Explicit denominator and exclusions, deployment floor plus tenant-strengthened small-cell threshold, complementary and overlapping-cell suppression, no detail/export/diagnostics | Campaign #80 | Aggregate query, cross-metric suppression, route/role/ACL, stable filter and UI structure tests | Complete 2026-07-22 (`06125cc`, `fc36aee`, `8ee87b7`, `ac3329c`, `1225802`) |
| 8 | Campaign excluded outcomes | Excluded build rows become explicit skipped transport outcomes and remain protected from queue/cancel/retry ambiguity | One durable source-to-job-to-report meaning with guarded historical normalization | Campaign #66 | Builder/persistence, migration, query/count, queue-control and report-explanation tests | Complete 2026-07-22 (`7229fb8`) |
| 9 | Guided first campaign | Existing wizard routes and ordinary workspace overlap | Task-oriented entry that hands off clearly to normal editing/review | Campaign #35 | First-run flow, resume/back, validation, optional modules, no implicit send | P1 after core pilot patterns stabilize |
| 10 | Prove/extract generic primitives | Core already exports many primitives; Campaign composition still unreviewed | Extract only contracts with a second consumer or clear platform ownership | Core #225 plus bounded follow-ups | Core behavior/accessibility tests and module-permutation tests | After Campaign proof |
| 11 | Configured-system pattern help | Docs route and classification exist | Role/config-aware pattern and route/field/blocker help | Docs #15 | Topic grouping, audience filtering, stable links/anchors | P1 after initial pattern IDs stabilize |
| 12 | Admin/configuration family | Phase inventory and connector primitives exist in the ledger | Apply the pattern to files, mail, policy, retention, packages, modules, API keys, settings | Core #225 and module children | Per-surface state/accessibility/consequence evidence | Parallel where independent of Campaign shared decisions |
| 13 | Remaining direct routes | Routes are contributed; most are unreviewed | Per-module bounded audit and migration plan | New module issues derived from this inventory | Applicable definition-of-done gates | P2 after Campaign, not a bulk rewrite |
| 14 | Manifest/runtime alignment | Several executable routes are absent from manifest metadata | Declared alignment or explicit validated exception | Core contract issue to create | Automated manifest/module route check and configured Docs verification | Discovery follow-up |
Workflow/user-story implementation is postponed. It is not on the critical path
for this rollout matrix. Focused views can be specified, manually selected, and
tested through core composition contracts; a later workflow step may become one
activation source without changing the proven surface patterns.
## Inventory Maintenance
When a route, nav item, named UI capability, host section, or Campaign workspace
surface changes:
1. Update the owner, evidence, task, archetype, and consequence here.
2. Link the implementation issue and verification evidence.
3. Keep "unreviewed" until state, permission/privacy, consequence/provenance,
accessibility, responsive, theme, i18n, and applicable async behavior have
been checked.
4. Re-scan both `module.ts` and the backend manifest; do not infer one from the
other.
5. Recreate the inventory from a clean release lockfile before using it as
release evidence.

83
docs/META_REPO_SCAN.md Normal file
View File

@@ -0,0 +1,83 @@
# Meta Repository Scan
Scan date: 2026-07-13.
This scan checked local repositories under `/mnt/DATA/git` listed in
`repositories.json`.
## Repository Inventory
`repositories.json` already lists every checked-out GovOPlaN repository.
Checked-out repositories not listed in `repositories.json`: none.
Repositories listed in `repositories.json` but not checked out locally: none.
The human-readable link index is `docs/REPOSITORY_INDEX.md`; the JSON file
remains the machine-readable source of truth.
## Meta-Owned Content
These items are correctly owned by the meta repository:
- `.gitea/workflows`: cross-repository CI and audit workflows.
- `.gitleaks.toml`: whole-workspace secret scan policy.
- `audit-reports`: whole-workspace audit output.
- `dev/postgres`: shared development PostgreSQL service.
- `dev/production-like`: production-like product validation composition.
- `tools/checks`: whole-product checks and audit launchers.
- `tools/gitea`: issue/wiki/label/backlog tooling.
- `tools/launch`: product launch entry points.
- `tools/release`: release catalog, lock, push, and release console tooling.
- `tools/repo`: repository bootstrap, status, and metadata tooling.
## Module-Local Content That Should Stay Local
These paths look operational, but they are tied to one module's protocol or
transport behavior and should stay in the owning repository for now:
- `govoplan-campaign/dev/mail-testbed`: mail transport testbed for campaign.
- `govoplan-files/dev/connectors`: WebDAV, Nextcloud, and SMB connector smoke
environment for files.
The meta repo can later wrap these with aggregate commands, but the test-bed
definitions should remain close to the module code unless they become a shared
product deployment profile.
## Generated Or Local-Only Content
These should not move to the meta repo; they should be ignored or cleaned:
- `__pycache__` directories under `tools/gitea` and module test beds.
- `.venv` and `.ruff_cache` directories.
- Populated `.env` files such as `govoplan-campaign/dev/mail-testbed/.env`.
## Repo-Local Workflow Files
Most module repositories have `.gitea/ISSUE_TEMPLATE` installed. These are
repo-local generated copies of the shared workflow templates and are fine to
keep in each repository.
Repositories currently missing a `.gitea` directory in the local checkout:
- `govoplan-dashboard`
- `govoplan-evaluation`
- `govoplan-poll`
- `govoplan-rest`
- `govoplan-soap`
If those repositories should use the shared issue templates, run the meta repo
installer from `govoplan/tools/gitea/gitea-install-workflow.py`.
## Do Not Move
The following stay in each module repository:
- `pyproject.toml` and package dependency metadata.
- Backend manifests and migrations.
- Module-owned tests.
- Module READMEs and module-specific docs.
- WebUI package manifests and module frontend source.
Those files describe or implement the module itself. The meta repo should
catalog and orchestrate them, not become the owner of module code.

View File

@@ -0,0 +1,120 @@
# Module Contracts and Install Boundaries
## Why Some Changes Require `pip install`
GovOPlaN discovers runtime modules through Python package entry points in the
`govoplan.modules` group. Core reads those entry points with
`importlib.metadata.entry_points()`, imports the configured manifest factory, and
then builds the module registry from the returned `ModuleManifest`.
That means the Python environment must know that a package exists before core can
discover it.
In development, `requirements-dev.txt` installs modules with `-e
../govoplan-module`. With editable installs:
- normal Python source changes are picked up after the process reloads;
- manifest code changes are picked up after the process reloads;
- new imports inside an already installed package are picked up after reload.
`pip install` is still needed when package metadata changes:
- a repository was not installed in the environment before;
- a `pyproject.toml` entry point is added, renamed, or removed;
- package dependencies change;
- optional extras change;
- package names or import roots change;
- console scripts or other installed metadata change;
- release installs need a different tag, wheel, or source ref.
The same principle applies to WebUI packages: source changes are local during
development, but `package.json` dependency or export changes require an install
step so the consuming app sees the correct package metadata.
## Current Contract Mechanism
Modules already announce contracts through `ModuleManifest`:
- `dependencies`
- `optional_dependencies`
- `required_capabilities`
- `optional_capabilities`
- `provides_interfaces`
- `requires_interfaces`
- `capability_factories`
- permissions and role templates
- route factories, migrations, docs, lifecycle hooks, and frontend metadata
Core validates the active manifest graph when it builds the registry. Release
tooling can inspect those manifests to calculate compatibility and migration
impact.
## What Core Can and Cannot Pick Up Automatically
Core can pick up contract changes automatically after reload when the changed
module package is already installed and importable.
Core cannot discover a new module or new entry point that has not been installed
into the environment, because there is no distribution metadata to enumerate.
Core also cannot notify every module about a contract change at edit time by
itself. The runtime registry is built from installed/importable packages. The
right place for cross-module announcement is the meta repo tooling and CI:
- scan manifests across all repositories;
- build an impact graph from provided/required interfaces and capabilities;
- run affected module tests;
- post Gitea issue/release notes for affected modules;
- block releases when a required interface is missing or incompatible.
## Mitigation Strategy
Use three layers:
1. Editable development environment.
Keep `requirements-dev.txt` in the meta repo as the one workspace installer.
Source edits then need process reloads, not repeated full installs.
2. Versioned runtime contracts.
Keep adding and tightening `provides_interfaces`, `requires_interfaces`, and
capability protocols. Treat interface names and versions as public module
contracts.
3. Meta-level contract audit.
The meta repo statically reads `src/**/backend/manifest.py` files across all
repositories and validates provided/required interface ranges without
importing the packages. CI blocks missing or incompatible required
interfaces before release installs are attempted.
Run the static graph check with:
```sh
./tools/checks/check-contracts.sh
```
Use `--json` when the release console or another automation needs structured
provider/consumer impact data.
## Practical Rule
Do not run `pip install` for every code edit. Run it when package metadata or the
set of installed packages changes.
For normal development:
```sh
./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-dev.txt --python ./.venv/bin/python
```
Then restart the server when Python code or manifests change. The development
launcher runs this helper automatically by default; set
`GOVOPLAN_AUTO_SYNC_PYTHON=0` to disable that preflight.
For release validation:
```sh
./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python ./.venv/bin/python
```
That install is necessary because the release environment intentionally resolves
tagged package refs, not local editable source trees.

View File

@@ -0,0 +1,452 @@
# GovOPlaN Reference Journey Program
## Status
This is the selected product-development sequence as of 2026-07-21. It turns
the long-term connected-platform roadmap into five demonstrable journeys while
the Workflow program remains deliberately deferred.
The stages are ordered, but they are not monolithic releases. Each stage is
delivered as small, reviewable, green increments and is complete only when its
user journey, failure behavior, documentation, and operator evidence work in a
pinned composition.
## Why this sequence
The sequence grows one connected product rather than advancing repositories in
isolation:
```text
Campaign reference
-> function-bound Postbox delivery
-> data-backed templates and reports
-> governed analytical data products
-> collaborative document lifecycle
```
Each stage reuses the preceding contracts:
- Campaign proves governed recipient selection, files, profiles, preview,
durable delivery, retry, reconciliation, audit, and role-specific UI.
- Postbox generalizes delivery from an email address to an institutional
responsibility that survives personnel changes.
- Templates and Reporting turn governed source data into reproducible outputs
and provide a safe launch boundary for specialist systems such as HIS.
- The BI path scales those source, transformation, quality, lineage, and report
contracts from one document to reusable analytical data products.
- DMS collaboration turns generated or uploaded artifacts into governed living
documents without confusing storage, editing, approval, or records
ownership.
## Continuous foundation lane
This lane is not a sixth product stage. It advances only as far as the next
journey needs and supplies contracts shared by all five stages.
1. **Governed integrations.** Connector profiles declare endpoint, authority,
source-of-truth mode, capabilities, health, limits, credential references,
and lifecycle. Network destinations are resolved and pinned at connection
time; private-network access is deployment-controlled and unpinned private
transports fail closed. Deleting a connector or provider immediately
deletes every secret it owns and records a non-secret audit event in the
same lifecycle action; an unavailable external secret provider blocks the
deletion or retirement rather than leaving an unaudited orphan.
2. **External effects.** Requested action, durable command, observed effect,
unknown outcome, retry, reconciliation, compensation/correction, and manual
intervention remain distinct and auditable.
3. **Institutional identity.** Identity owns subjects, Organizations owns units
and functions, IDM owns identity-to-function assignments and upstream
synchronization, and Access owns permission and acting-context decisions.
4. **Stable composition.** Modules cooperate through versioned Core-mediated
capabilities and typed references. Optional sibling modules are never
imported as implementation dependencies.
5. **Evidence and provenance.** Every imported datum, generated artifact,
delivery, transformation, and later correction can identify its source,
version, actor or system trigger, policy, and relevant execution evidence.
6. **Focused experience.** Central components and the interface pattern
language are mandatory. Views present only what the current task and actor
need while preserving an explainable escape to the configured full system.
7. **Adaptive documentation.** The owning module contributes canonical subject
matter; GovOPlaN Docs composes it for the installed modules, enabled
configuration, role, route, and task.
8. **Release proof.** Version alignment, clean install/upgrade, module
permutations, backup/restore, degraded-provider behavior, security checks,
and target-environment acceptance gate product claims.
9. **Artifact and infrastructure egress.** Tenant connectors use the pinned
connector boundary. Installer artifacts use a separate deployment-owned
trust path: approved origins, bounded pinned download or operator prefetch,
signature/checksum verification, safe extraction, and offline package-manager
execution. Database, broker, cache, and worker channels are constrained by
deployment network policy and authenticated transport rather than treated as
tenant connector profiles.
## Documentation contract for every reference stage
Documentation is one connected knowledge model rendered through different
perspectives, not separately maintained manuals that drift.
Every demonstrated journey provides:
- **Task guidance:** “I want to …” instructions using visible UI wording.
- **Process view:** actors, inputs, states, decisions, handoffs, exceptions,
completion, correction, and retention.
- **Concept view:** domain language and ownership boundaries.
- **Administrator view:** permissions, policies, profiles, configuration,
module requirements, migrations, and audit effects.
- **Operator view:** health, queues, retries, reconciliation, backup/restore,
incident handling, and target-provider checks.
- **Integration view:** capabilities, DTOs, APIs, events, idempotency,
compatibility, deep-link/launch contracts, and example payloads.
- **Security, privacy, and audit view:** threats, disclosure rules, secrets,
provenance, evidence, retention, and destructive actions.
- **Acceptance view:** runnable examples, expected results, failure injection,
and release gates.
The Docs module selects and links these views according to installed
capabilities and actor context. Feature repositories remain the source of
truth for their behavior; Docs owns indexing, conditions, safe disclosure, and
presentation. Generated API/contract reference may supplement, but never
replace, the maintained behavioral explanation.
## Stage 1: Campaign as the demonstration module
### Product promise
> As a communications team, we can compose a campaign from governed recipient
> data, managed files, reusable mail configuration, and policy; preview the
> exact effect; authorize delivery; and resolve every recipient to a known or
> explicitly uncertain outcome without exposing transport internals to normal
> readers.
Campaign is the first reference composition, not an all-purpose mail client or
contact database.
### Ownership and composition
- Campaign owns campaign intent, content, campaign-local recipient snapshots,
personalization, execution state, and delivery evidence.
- Mail owns reusable profiles, credentials, protocol adapters, profile policy,
send/append execution, and provider-facing diagnostics. Campaign persists
only a stable Mail profile reference, never SMTP/IMAP settings or secrets.
- Files owns stored objects, provider connectors, file policy, and provenance.
Campaign persists stable attachment references and frozen execution evidence.
- Addresses or another directory may supply recipients, but Campaign freezes
the execution-time recipient and source evidence needed to explain the send.
- Policy, Access, Audit, Docs, Notifications, Organizations, Identity, IDM, and
Postbox contribute only through declared capabilities when installed.
### Implementation slices
1. Enforce the Mail-profile-only boundary and reject inline transport material
on authoring, import, update, build, and delivery paths. Legacy records are
preserved but fail closed until explicitly migrated.
2. Finish the central-component UI pass and consistent task-focused author,
reviewer, operator, and reader surfaces. Normal readers receive business
state, not paths, backend keys, worker claims, or raw provider diagnostics.
3. Prove test, single-send, resend, batch, append-to-Sent, retry,
reconciliation, quarantine, and manual intervention semantics against a
target SMTP/IMAP environment.
4. Complete adaptive Campaign, Mail, and Files documentation under the shared
documentation contract, including runnable reference campaigns and failure
drills.
5. Package the composition with sample data, policies, roles, configured views,
preflight, health checks, and pinned compatible versions.
### Gate
A clean installation can run the maintained examples through target providers;
no Campaign payload contains transport credentials; worker/provider failures do
not cause blind duplicate delivery; each supported role sees an appropriate
surface and explanation; install, upgrade, restore, and release evidence are
repeatable.
## Stage 2: Postbox and delivery to institutional responsibility
### Product promise
> As a sender, I can deliver a message to a named function in an organizational
> unit without knowing which individual currently performs it. As the current
> function holder or authorized delegate, I can discover, read, and act on that
> message; when assignments change, access follows current responsibility while
> the delivery and access history remains explainable.
### Ownership and composition
- Postbox owns addressable in-platform postboxes, messages, bindings,
participants, attachment references, access-sensitive events, and retention
inputs.
- Organizations defines units and functions. Identity defines subjects. IDM
assigns identities to functions and reconciles upstream directories. Access
resolves roles, delegations, acting context, and permissions.
- Mail is an optional external channel bridge. A Postbox is not an SMTP/IMAP
account and is never owned by one login credential.
- Campaign consumes a generic delivery-target capability. It may target an
email recipient or a function-bound Postbox without importing either
module's internals.
- A Postbox exists independently of its current holders. Vacancy, one
incumbent, several incumbents, and time-bounded delegation are valid states;
none turns the Postbox into a personal account.
- The E2EE target uses per-content keys and function/postbox key epochs.
Assignment grants device-bound access to policy-selected history, while
hand-over, revocation, and delegation expiry rotate or withdraw future key
access without claiming to erase plaintext already obtained.
- Content and signed manifests are immutable. Correction or replacement is a
linked new object/version, never silent substitution.
### Implementation slices
1. Stabilize Postbox manifest, permissions, DTOs, migrations, directory,
binding administration, access decisions, message create/list/read, and
audit events.
2. Define a typed delivery target and receipt contract for a Postbox id or an
organization-unit/function address. Resolution never copies current holders
into permanent mailbox ownership.
3. Complete the OrganizationsIdentityIDMAccess function-assignment and
delegation path, including provenance and compatibility migration from
legacy projections.
4. Define the E2EE key-grant/epoch profile for vacancy, multiple incumbents,
hand-over, history access, time-bounded delegation, recovery, and audit.
5. Add Campaign delivery to a function-bound Postbox, file attachment
references, optional external-mail bridge, and notification attention
signals through capabilities.
6. Document reassignment, delegation, vacancy, expired assignment, unavailable
optional module, retention, and future E2EE limitations honestly.
### Gate
A campaign message can be delivered once to a function-bound Postbox; an
authorized holder can act in an explicit context; a reassignment changes future
access without moving or rewriting the message. A vacant Postbox remains
addressable, retains the delivery, and visibly reports that no holder can
decrypt or act until assignment; multiple incumbents and delegates receive
independent key/access evidence. Delivery, key access, and later corrections
remain auditable.
## Stage 3: Templates, reports, data sources, and deep launches
### Product promise
> As a staff member arriving from a specialist system such as HIS, I can open a
> GovOPlaN report or document task with a governed source context already
> selected, review the effective data and template version, generate the
> document, and return or link the immutable result without re-entering data.
### Ownership and composition
- Templates owns reusable definitions, versions, merge schemas, localization,
render profiles, preview, and rendering.
- Reporting owns report definitions, parameters, curated data selection,
execution history, dashboards, sharing, scheduling, publication, and export.
- Connectors own protocol-specific access. A governed data-source catalog owns
connection profiles, source contracts, freshness, and health if repeated
use proves that a separate module is warranted.
- Files stores generated bytes. DMS owns a generated artifact once it becomes a
governed living document. Records owns later retention/archive semantics.
### Safe launch and generation contract
1. The source program opens a stable GovOPlaN route with an opaque, short-lived,
single-purpose launch reference—not credentials, arbitrary SQL, or trusted
personal data in the URL.
2. GovOPlaN authenticates the actor, authorizes the requested report/template
and source object, resolves the launch context server-side, and shows source,
freshness, purpose, and any blockers.
3. A curated connector/read-model capability returns a versioned input snapshot
that satisfies the template or report schema.
4. Generation records template/report definition version, parameters, input
snapshot or reproducible source references, transformation version, output
checksum, actor, and policy.
5. The result is stored or handed back through an explicit callback/reference
contract. Retries are idempotent; expired or replayed launch references fail
safely.
### Implementation slices
1. Implement the first Templates manifest, version/schema DTOs, safe preview,
deterministic render, package fragments, and Files-backed output.
2. Implement the first Reporting manifest, parameter/source DTOs, one
module-owned read model, execution evidence, and downloadable export.
3. Define the shared data-source/profile and launch-context contracts with one
read-only target connector and a mock HIS-style launch producer.
4. Deliver one end-to-end reference document and one analytical report from the
same governed source contract.
5. Add role-specific UI/docs, source freshness and validation states, deep-link
integration examples, and target-system acceptance tests.
### Gate
A user can follow a signed or server-side launch reference from a target system
to a prefilled GovOPlaN task, verify the source and freshness, render an output,
and reproduce why that exact artifact was produced. The browser cannot inject
arbitrary queries or transport secrets, and authorization is re-evaluated in
GovOPlaN.
## Stage 4: Governed BI and reporting
### Product promise
> As an institutional analyst, I can combine approved operational sources into
> versioned, quality-checked analytical data products, define transparent
> measures and reports, drill from aggregates where policy permits, and explain
> the source, transformation, effective organizational hierarchy, and reporting
> date behind every result.
### Lessons taken from SuperX
SuperX is useful inspiration because it combines many university sources in a
modular data warehouse, loads and transforms them on a schedule, keeps
historical or reporting-date views, supplies prepared subject modules, and
serves different aggregation levels through a common reporting surface. Its
official overview also emphasizes a lower operational-data layer, validation
logs, configurable load routines, organizational hierarchies, and transparent
report calculations. See the
[SuperX project overview](https://www.superx-projekt.de/__index.htm),
[administration and module architecture](https://www.superx-projekt.de/doku/kern_modul/admin/__index.htm),
and [cross-institution indicator module](https://www.superx-projekt.de/doku/kenn_modul/benutzer/__index.htm).
GovOPlaN should adopt the principles, not clone the implementation. Its distinct
value is connecting analytical results to governed work, responsibility,
evidence, policy, and correction across independently installable modules.
### Architecture layers
1. **Source catalog:** governed profiles, owners, purpose, schema/version,
classification, credentials, health, extraction mode, and source-of-truth
declaration.
2. **Ingestion and staging:** immutable run evidence, source snapshots or
watermarks, schema-drift detection, quarantine, replay, and personal-data
minimization.
3. **Transformation and quality:** versioned transformations, dependencies,
tests, validation findings, correction policy, and lineage. Silent repair of
source facts is not acceptable.
4. **Semantic products:** documented dimensions, measures, official key
mappings, organizational hierarchies, time/stichtag semantics, ownership,
access policy, and version.
5. **Consumption:** Reporting definitions, dashboards, drill-down, scheduled
outputs, APIs, open-data handoffs, and template generation.
6. **Operations:** scheduling, backfill, freshness objectives, observability,
cost/resource limits, retention, backup/restore, promotion between systems,
and reproducible release recipes.
### Selected first subject-area direction
- Start with read-only HIS and CampusOnline source profiles and produce the
institution's internal student-statistics data product plus reports derived
from it.
- Model reference/effective date separately from extraction, validation, and
freeze dates. For example, a semester state may describe `1 September`, be
accepted and frozen on `12 December`, and remain the reproducible source for
that semester's official statistics. Later corrections create a superseding
version; they never rewrite the frozen state silently.
- Make a graphical, typed data-flow editor the primary analyst surface. Nodes
represent governed sources/snapshots, validation, mapping, transformation,
aggregation, a list report/data product, template rendering, and publication.
The graph is versioned, testable, previewable, acyclic, and shows schema,
lineage, quality findings, policy, and pinned upstream versions.
- A list report may become a governed source for another flow. This enables
`source + transformations/aggregation -> list report/data product -> template
-> publishable report` without copying the calculation or losing lineage.
### Implementation slices
1. Bound the first HIS/CampusOnline student-statistics dataset and internal
report; define its legal/policy basis, source owners, official keys,
reference/freeze semantics, privacy level, and acceptance calculation.
2. Generalize the Stage 3 source profile into catalog, ingestion-run, staged
dataset, validation, transformation, lineage, and data-product contracts.
3. Implement one scheduled load and explicit frozen-state lifecycle with
quarantine/replay and a fully transparent transformation into the internal
student-statistics product.
4. Implement the first graphical typed flow editor over those proven nodes,
including version/diff, validation, preview, cycle prevention, and pinned
upstream product versions.
5. Serve the product through Reporting with policy-aware aggregate and
drill-down behavior; export reproducible evidence and calculation metadata.
6. Chain one list report as a source into a template-backed publishable report.
7. Package mappings, transformations, validations, reports, documentation, and
tests so they can be promoted from development to test to production.
### Gate
One internal student-statistics product can be rebuilt from declared HIS and/or
CampusOnline snapshots, preserves its reference and freeze dates, passes
explicit quality tests, explains every measure and mapping, enforces disclosure
policy, survives schema/freshness failures visibly, and produces the same
accepted list and template-backed report across promoted environments.
## Stage 5: Collaborative document editing
### Product promise
> As an authorized group, we can turn an uploaded or generated artifact into a
> shared document, edit it concurrently or through controlled check-out,
> comment, review, approve, compare, and recover versions, then freeze the
> accepted rendition as evidence or a record without losing who changed what.
### Ownership and composition
- Files owns bytes, checksums, storage providers, upload/download, and low-level
file permissions.
- DMS owns document identity, versions, renditions, editing sessions, locks,
comments, reviews, approvals, signatures, external DMS references, and
document-level evidence.
- A collaboration connector owns provider-specific Collabora, OnlyOffice,
Nextcloud/OpenDesk, or another office-suite protocol. DMS owns the session
lifecycle and authorization even when editing occurs externally.
- Templates may create the first version. Cases, Campaign, Postbox, Reporting,
and Workflow may reference documents. Records owns classification, legal
hold, archive handoff, and disposal after the document enters that lifecycle.
### Implementation slices
1. Implement DMS document/version/reference DTOs and one Files-backed version
path, including optimistic concurrency, checksums, locks, and audit.
2. Add comments, review requests, approval state, immutable accepted
renditions, comparison, and recovery of prior versions.
3. Define a provider-neutral editing-session capability with short-lived
grants, callback authentication, save idempotency, health, and reconciliation.
4. Integrate one external collaborative editor, including loss of provider,
concurrent save, stale callback, permission revocation, and restore tests.
5. Connect generated reports/templates, Postbox/Campaign sharing, focused
document views, configured docs, and Records handoff.
### Gate
Multiple authorized actors can edit a document without silent lost updates;
every accepted version has stable content and provenance; provider callbacks
cannot bypass current authorization; degraded or ambiguous saves are visible
and reconcilable; the approved rendition can enter records without making Files
or the external editor the document-lifecycle owner.
## Program-wide acceptance and pause rules
- Do not start a broad generic platform abstraction until two concrete stages
demonstrate the repeated contract.
- Do not make Workflow a dependency of these stages. Manual transitions,
explicit actions, and focused views should become stable providers that a
later transition engine can coordinate.
- Do not treat a remote system as trusted merely because it supplied a deep
link, callback, webhook, file, identity, or data row.
- Do not claim a stage complete from local unit tests. Use pinned composition,
target integration, failure drills, adaptive docs, and operator evidence.
- A later stage may prototype contracts while the preceding gate is being
proven, but it may not redefine an owning module's boundary by convenience.
## Decisions intentionally deferred
The sequence itself is selected. These bounded choices remain for the stage
that first needs them:
- first target SMTP/IMAP, file/directory, identity, and deployment profile;
- first external IDM source and identity conflict/disable policy;
- first trusted Postbox assurance/recovery/history-access profile and
external-mail bridge;
- first HIS or other launch producer and callback/reference contract;
- exact HIS/CampusOnline source endpoints, student-statistics keys and accepted
calculation, freeze/correction policy, privacy profile, and permitted
drill-down level;
- whether repeated data-source and dataflow contracts justify separate
`govoplan-datasources` and `govoplan-dataflow` modules after the Stage 3 proof;
- first collaborative editor/provider and whether the first UX is concurrent
editing, controlled check-out, or both; and
- first Records/archive target and approval/signature assurance level.

550
docs/RELEASE_CONSOLE.md Normal file
View File

@@ -0,0 +1,550 @@
# GovOPlaN Release Console
The release console is a local operator tool for planning and executing
GovOPlaN releases. It belongs to the `govoplan` meta repository because it works
across all local checkouts, release scripts, module manifests, migration audits,
catalog files, Git state, and signing keys.
The current implementation has a read-only dashboard, preview-only legacy
controls, and a bounded durable executor for release steps whose inputs and
effects can be verified safely:
- inspect repositories from `repositories.json`
- show dirty, ahead, behind, missing, no-HEAD, and tag state
- show local package catalog and keyring state
- optionally run release/dev migration audits
- propose next actions without executing them
- compare local catalog/keyring JSON with the published channel and public
keyring when online checks are enabled
- configure target versions per release unit in the web UI
- select the repositories that should advance through checkboxes
- generate dry-run selective release plans for independently versioned packages
- freeze a selective plan as a durable, resumable local release run
- durably preflight a creation-time-bound repository, create its annotated tag,
and publish its branch/tag pair atomically
- build selected Python wheels and generate a private, signed, receipt-bound
catalog candidate
- publish that exact candidate through a verified website commit and immutable
tag after explicit confirmation
Start it from the meta repository:
```sh
./.venv/bin/python tools/release/release-console.py
```
The launcher accepts only a numeric loopback address, binds to `127.0.0.1` by
default, always creates a fresh API token, and prints that token only in the URL
fragment. Open that URL in a browser on the same machine. A deliberately
tokenless embedded app is read-only: non-GET `/api/` requests fail with `403`.
Non-loopback operation needs a separately deployed authenticated TLS boundary;
the local launcher will not expose the mutation API that way.
Run durable release mutation only against an operator-private workspace.
Repository roots, every path ancestor, critical and nested Git metadata, and
tracked worktree files must be owned by the console process UID (or root where
appropriate) and must not be group/world writable. Symlink/special Git
metadata, object alternates, and grafts are rejected. The console pins
`/usr/bin/git`, `/usr/bin/ssh`, a fixed system `PATH`, disabled hooks, isolated
Git configuration, and no replace objects. Shared or `nfsnobody`-owned
checkouts remain usable for read-only planning, but every durable executor
fails closed there; clone the registered origins into a private workspace
before releasing.
The runtime itself is part of the authority boundary. Durable run creation
verifies the meta checkout, release/check tooling, repository registry, Python
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
executables. It then binds a clean meta-repository HEAD and named branch that
exactly match the registered `origin`. Every execution and reconciliation
rechecks that receipt. Permission checks cannot prove that code was safe before
it entered a writable tree, so release from a fresh operator-private clone or a
separately verified installed console artifact.
The web UI starts with a repository table. Each repository can be checked
independently and assigned its own target version. Repositories without version
metadata remain visible so they can be planned as initial releases. `Build Plan`
shows the dry-run commands for the selected rows, and `Generate Candidate`
creates a signed catalog candidate that advances only selected repositories that
already have a catalog entry.
`Build Plan` also returns structured release-gate findings for each selected
repository. The plan names the recommended next action and gives an explicit
remediation for source-version, lockfile, Core WebUI composition, Git state, and
worktree findings. A target version that has not yet been applied consistently
to the selected source tree is therefore explained before the source-tag
preflight rather than appearing only as an error after the operator tries to
tag. `source_preflight_ready` means that the plan-visible source gates pass; the
non-mutating `Preview Tag + Publish` remains mandatory for remote, manifest, and
immutable-tag checks.
## Durable release runs
The **Durable Run State** card turns the current repository/version selection
into a versioned local run record. The server rebuilds the selective plan and
requires the plan to resolve exactly the requested repositories and target
versions; the browser cannot submit or replace the plan snapshot. The input and
plan are then immutable and covered by a canonical SHA-256 integrity digest.
Every executable repository step also carries its creation-time full HEAD,
branch, worktree state, target tag, and a SHA-256 over both the fetch and push
URLs of `origin`. Both URLs must exactly equal the remote registered in
`repositories.json`; a changed HEAD, branch, worktree, remote, or push URL
requires a new run rather than silently retargeting the frozen compatibility
decision.
The complete record also has a checksum so a valid-looking manual edit to its
mutable state fails closed. File permissions remain the authority boundary;
these digests detect accidental or manual corruption, not an attacker who can
replace the private record and recompute its checksums. Changing a target,
channel, or gate input requires a new run.
Creation requires a caller-generated `request_id`. Its SHA-256 fingerprint is
the private, workspace-scoped durable mapping to exactly one run; the raw
identifier is never persisted. Repeating the same identifier with the same
immutable inputs returns that run without rebuilding the plan while it remains
inside the bounded local retention window, even if the live dashboard has
since drifted. Reusing it with different inputs fails closed. The browser keeps
an uncertain create identifier in session storage,
replays it after reload, and selects the known run returned by the server. A
successful create remains shown as saved if only the subsequent list refresh
fails.
Run records survive console restarts, but remain local operator state rather
than a signed release artifact or the system audit log. The default location is
`$XDG_STATE_HOME/govoplan/release-console/workspace-<sha256>/release-runs/`, or
`~/.local/state/...` when `XDG_STATE_HOME` is unset or relative. It is outside
the source checkout so filesystems without enforceable POSIX modes cannot
silently weaken the journal. Newly created state directories use mode `0700`
and records use `0600`. Writes use a cross-process lock, a same-directory
temporary file, `fsync`, atomic replacement, and directory/parent `fsync`.
Symbolic-link paths, untrusted owners or writable ancestry, overly broad record
modes, malformed schemas, unknown fields, invalid state combinations,
oversized files, and digest mismatches fail closed. A bad record is neither
rewritten nor automatically quarantined.
The store retains at most 512 workspace-scoped run records created through the
console. On creation at that limit it removes only the oldest fully completed,
integrity-verified record. Running, planned, attention, blocked, foreign, and
unreadable records are never deleted implicitly; if no completed record is
available, creation fails closed with a retention remediation. Unavailable
records still consume the bound and remain visible in cursor-paginated lists
so corruption cannot be hidden by normal turnover.
The full resolved-workspace SHA-256 is part of both the private storage
namespace and immutable input snapshot. Every list, read, and state transition
checks it. An alternate workspace therefore cannot list or resume another
workspace's runs. This remains true for an embedding/test `run_state_root`
override: the server always appends
`workspace-<full-sha256>/release-runs/` rather than treating the override as a
shared record directory. Durable candidates use its private sibling
`release-candidates/` directory, never a checkout-local runtime path. A corrupt
record from one workspace therefore cannot leak even its identifier or an
integrity error into another workspace.
Each frozen plan step has an explicit `pending`, `running`, `succeeded`,
`failed`, or `interrupted` state. Plan order remains a prerequisite: a later
step is unavailable until earlier steps have succeeded. Exact attempt and
resume/retry/reconciliation request identifiers are fingerprinted so delayed
repetitions remain idempotent for the retained run's lifetime. These fingerprints are
never evicted: the store fails closed before accepting more than 2,048 commands
or 2,048 attempts and asks the operator to create a fresh run. The display
record keeps at most 256 server-generated state events. Events contain only an
enum event type, timestamp, step identifier, and bounded result code—never
commands, process output, confirmation text, credentials, bearer tokens, or
signing material.
Before a step can enter `running`, the store reserves the two command-ledger
slots needed for worst-case recovery. It also projects the serialized record
through start, finish/failure, resume, and the required terminal reconciliation
or read-only retry; the start is rejected unless every required atomic write
fits the record-size bound. An explicit resume consumes one slot and is
accepted only while a persisted attempt is actually running. A mutating attempt
always retains its final `effect_absent` or `effect_succeeded` slot;
`unresolved` may be recorded at most once for that attempt and only when an
additional ledger slot and serialized terminal-write capacity are available.
Read-only interruption and known failure retain the slot and byte capacity
needed to prepare a retry. Capacity exhaustion is therefore detected before
starting an effect rather than stranding an ambiguous attempt.
An explicit resume after a process restart converts every persisted `running`
step to `interrupted`; it never guesses whether an external effect happened.
An interrupted read-only step can be prepared for retry. An interrupted
mutating step remains unavailable until the operator independently reconciles
local and remote state, selects `effect_absent`, `effect_succeeded`, or
`unresolved`, and types `RECONCILE`. `effect_absent` prepares a safe new
attempt, `effect_succeeded` advances the run without repeating the external
effect, and `unresolved` keeps the run blocked. Each outcome emits a bounded,
code-only state event. A known failed attempt can likewise be prepared for
retry. The UI keeps unavailable controls visible and disabled.
Supported executors durably claim the step before invoking an effect. Exact
attempt replays return the recorded outcome and never invoke the executor a
second time. Successful repository preflight, tag, and push steps persist a
bounded repository-state receipt. Tag reconciliation independently requires an
annotated local tag at the frozen HEAD; push reconciliation additionally
requires both the remote annotated tag object and remote branch to match.
Catalog generation persists
only its server-issued opaque candidate ID and canonical catalog SHA-256, then
re-resolves and re-hashes that private candidate before publication.
The safe baseline is intentionally narrower than the complete dry-run plan.
Dirty worktrees and version changes still show commit/version steps, but those
steps have no durable executor because the frozen run does not bind the exact
proposed file content. A run selecting Core together with one or more modules
begins with a disabled dependency-ordering barrier: local module tags, Core
release-lock regeneration/commit, Core tagging, alignment verification, and
pushes need an explicit DAG executor before that composition can mutate.
Operators must prepare and review those changes outside the console and create
a new run from the resulting clean HEAD. The console never skips these steps or
claims an end-to-end release succeeded.
The browser likewise retains the request identifier for an uncertain
resume/retry/reconciliation response and replays it after reload. A successful
replay selects the returned run state. Transport and server failures retain the
identifier; a deterministic `4xx` rejection clears it so a stale command cannot
poison a later attempt.
The run API is covered by the same local console token middleware as every
other `/api/` route:
- `POST /api/release-runs` requires `request_id`, then idempotently rebuilds and
freezes a selective plan only when that creation is not already known.
- `GET /api/release-runs` lists bounded summaries ordered by immutable
`created_at` and run identifier. `next_cursor` advances a stable descending
traversal even while older runs are updated, without offset duplicates or
skips. Unreadable entries
remain a deterministic final section and are therefore reachable through
pagination instead of displacing newer verified runs.
- `GET /api/release-runs/{run_id}` reads and verifies one exact record.
- `POST /api/release-runs/{run_id}/resume` records explicit recovery.
- `POST /api/release-runs/{run_id}/steps/{step_id}/retry` prepares a failed or
read-only interrupted step for another attempt.
- `POST /api/release-runs/{run_id}/steps/{step_id}/reconcile` records a
confirmed observed outcome for an interrupted mutating step.
- `POST /api/release-runs/{run_id}/steps/{step_id}/execute` claims and invokes
only the narrow executor declared by the immutable plan step. Durable release
execution accepts only the registered `origin`, never a caller-selected
remote.
- `POST /api/release-runs/{run_id}/steps/{step_id}/preview` provides the
non-mutating preview for receipt-bound catalog publication.
Run-storage errors are confined to the Durable Run State card; dashboard and
release-preview collection continue and show the bounded storage remediation.
The run record is execution evidence only for a supported step whose durable
claim and bounded result receipt were persisted. The console never infers
success from a button click or process exit alone. An executor exception or a
lost result write leaves the attempt interrupted and non-retriable until
explicit recovery. Catalog publication persists the candidate and keyring
hashes, exact website commit, annotated tag object and peeled commit, branch,
tag name, and registered-origin digest. A successful reconciliation revalidates
the candidate against the trust anchor in the frozen website parent, requires
the exact deterministic catalog/keyring/module blobs and full commit delta, a
sole frozen parent, and matching local and remote branch/tag identities. If any
part cannot be proved, the run remains interrupted and may only be recorded as
`unresolved`.
Dashboard collection is also fail closed. Unreadable Core version metadata or a
malformed module contract is returned as a bounded `collection_errors` entry
with a remediation, marks the dashboard blocked, and becomes a structured
blocker in both full and selective release plans. The console does not silently
omit a contract or turn these source errors into an HTTP 500.
The release-control area above the repository table is read-only and is meant
to become the central release cockpit. It shows:
- local and published channel health
- catalog/keyring drift
- signature and trusted-key status
- published module versions and refs
- local checkout version drift against the catalog
- catalog-declared interface compatibility
The target-version control can generate the next major, minor, or subversion
from the current base version. Manual target input accepts explicit versions
such as `0.2.0` or `0.2.0-alpha1`, but requires the first three version numbers
to move forward.
Plain repository pushes are separate from catalog publication. `Preview Push`
shows the selected repository push commands. `Push Selected` requires `PUSH` in
the repository push confirmation field.
The source release panel retains `Preview Tag + Publish` as a non-mutating
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
but disabled; the corresponding mutation endpoint rejects apply requests.
Creation and atomic branch/tag publication use the `TAG` and `PUBLISH`
confirmations on the durable run steps. The gate requires an aligned target
version, a clean named branch with a HEAD, and a checkout that is not behind. Existing
local or remote tags must resolve to the selected HEAD and are never moved.
The local and remote annotated tag objects must also be identical, not merely
point at the same commit. Before any source tag is created, the console loads
the cross-repository module registry. This release gate also rejects every
user-facing workflow documentation topic that has no scope condition, or has
an alternative condition without `required_scopes` or `any_scopes`.
The catalog workflow panel can also operate on the same selected rows for
generation and preview. Its legacy apply/push controls stay disabled; durable
publication consumes only the candidate receipt recorded by that run:
- `Generate` creates a signed candidate in the operator's private XDG state
directory and records only an opaque candidate ID plus the canonical catalog
SHA-256 in durable run state.
- `Preview` validates a candidate and shows what would be copied into the
website repository.
- `Apply + Website Tag` remains visible but disabled outside a durable run.
- `Push Website Release` remains visible but disabled outside a durable run.
Source release tags belong to Core or module repositories. Website catalog
publication creates a separate catalog tag in the website repository; the UI
names these independently to avoid confusing the two immutable references.
Candidate signing is fail-closed: every Core and module source ref in the
complete post-update catalog must already have the requested annotated tag both
locally and on the configured source remote. Both tags must be the same
annotation object with version-aligned tagged package metadata. Selected tags
must additionally resolve to the selected clean, version-aligned HEAD, and the
signed `selected_units` record captures the peeled commit and tag-object
identifiers. Create and publish the source tags before using `Generate`.
Catalog publication repeats that check for selected units and also verifies
every Core and module source ref in the complete candidate, including entries
preserved from the previous catalog. Historical refs need not be at the current
worktree HEAD, but their local and remote annotated tags must match and their
tagged installable package and module-manifest metadata must declare the catalog
version. New selected releases additionally pass the stricter current-source
alignment gate, including public runtime version declarations. A candidate
cannot be applied or published while any referenced source tag is absent or
inconsistent; the preview and API response list each repository and missing or
invalid ref so the operator can repair the exact source releases first.
Every selected Python release must also supply its exact built wheel. Durable
generation first verifies that the receipt-bound annotated tag is the same
object locally and on the registered origin. It clones an isolated checkout at
the receipt's exact commit and builds from that checkout, never from the mutable
live worktree. Every authenticated base-catalog source is likewise cloned from
its registered origin at an annotated release tag; only selected repositories
contribute synthesized fields. The base catalog and keyring are read as one
authenticated, exact private snapshot before synthesis.
Python wheels are built by a required Bubblewrap worker with no network,
private temporary/home directories, a read-only exact source mount, no host
home or file keys, cleared environment, fixed system tools, and CPU/address
space/process/file-size limits. If a trusted Bubblewrap launcher is unavailable,
generation fails closed. The worker must return one bounded regular wheel; the
console copies it through no-follow descriptors into a fresh private file,
`fsync`s it, then validates its package identity. Generation computes the
archive SHA-256 and an install-stable
payload identity from one bounded, regular-file descriptor and signs those
values into `release.artifacts`. Updating a Python version without a matching
wheel removes the stale identity. A source-only preview can still explain the
gap, but apply, commit, tag, and push fail closed until every selected Python
unit has a matching built-artifact identity. Repositories selected only for a
meta/source tag do not need a Python artifact.
Candidate directories and files are operator-owned `0700`/`0600` state. Before
any later release step consumes one, the executor re-resolves the opaque handle
below the configured root and re-hashes the signed catalog against its persisted
receipt. Shared roots, symlinks, channel path fragments, altered candidates, and
unowned files are rejected.
The current same-host worker is a containment baseline, not the final hostile
build-service boundary. `RLIMIT_FSIZE` is per file, and the worker does not yet
have a size-limited filesystem/cgroup quota, a fresh kernel keyring, or a
seccomp profile denying keyctl/request-key/ptrace/mount operations. A malicious
backend could therefore exhaust scratch disk/inodes or target same-UID kernel
facilities. Closing that denial-of-service/isolation gap requires a dedicated
quota/cgroup worker with a fresh keyring and seccomp policy.
The server-owned wheel builds remain under the private candidate's `artifacts/`
directory. This slice signs their identities but does **not** upload the wheel or
add a download URL to the public catalog; the existing Git `python_ref` is source
provenance and rebuilding it is not an equivalent artifact. Keep the private
candidate until the exact wheels have been transferred through an approved
deployment channel, verified against `archive_sha256`, consumed by the installer,
and covered by its signed receipt. Public artifact transport and receipt-aware
candidate cleanup remain separate release-lifecycle work.
Read-only provenance checks derive a same-host HTTPS Git URL from conventional
SSH remotes when possible, with a non-interactive check of the configured remote
as fallback. Source tag creation and atomic publication always use the explicit
configured Git remote.
The default signing key is
`$HOME/.config/govoplan/release-keys/release-key-1.pem` when no signing key is
entered in the UI. Signing-key files must be regular, owned by the operator, and
inaccessible to group/other users. The browser sends a configured key path only
on the initial execution request; it never persists signing material in session
storage, and request-ID recovery replays no key path.
Generate a selective release plan from the terminal:
```sh
./.venv/bin/python tools/release/release-plan.py \
--repo govoplan-files \
--target-version 0.1.9 \
--channel stable \
--online
```
`--online` compares the local catalog/keyring with the published channel and
published keyring. Use `--remote-tags` only when the plan also needs to check
Git remotes for tag existence; that can be slower across the full repository
set.
Build a signed selective catalog candidate:
```sh
KEY_DIR="$HOME/.config/govoplan/release-keys"
ARTIFACT_DIR="$(mktemp -d)"
../govoplan-files/.venv/bin/python -m pip wheel \
--no-deps \
--no-build-isolation \
--wheel-dir "$ARTIFACT_DIR" \
../govoplan-files
./.venv/bin/python tools/release/release-catalog.py selective \
--repo-version govoplan-files=0.1.9 \
--python-artifact \
"govoplan-files=$ARTIFACT_DIR/govoplan_files-0.1.9-py3-none-any.whl" \
--channel stable \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem"
```
This writes candidate catalog/keyring files below
`$XDG_STATE_HOME/govoplan/release-candidates/` (or
`~/.local/state/govoplan/release-candidates/`), generates the browsable module
directory below `modules/`, validates the signed candidate with the module
installer validator, and reports whether the candidate catalog/keyring match the
currently published channel. It does not publish to the website repository.
Regenerate the browsable module directory from an existing catalog/keyring:
```sh
./.venv/bin/python tools/release/release-catalog.py module-directory \
--catalog ../addideas-govoplan-website/public/catalogs/v1/channels/stable.json \
--keyring ../addideas-govoplan-website/public/catalogs/v1/keyring.json \
--output-dir runtime/module-directory-preview \
--channel stable
```
Preview publication of a reviewed candidate:
```sh
CANDIDATE_ROOT="${XDG_STATE_HOME:-$HOME/.local/state}/govoplan/release-candidates"
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable
```
Apply the reviewed candidate into the website repository without pushing:
```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable \
--apply \
--commit \
--tag
```
Push is a separate explicit flag:
```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable \
--apply \
--commit \
--tag \
--push
```
Publication validates the same in-memory catalog object that it writes; it does
not copy a path that can change after validation. On commit, the console reads
the catalog, keyring, and complete module directory back from the immutable Git
tree and requires byte-for-byte equality with those validated objects. Tags and
remote branch updates then reference that exact commit SHA rather than the
mutable worktree `HEAD`.
Published channels are expected below the public catalog base URL:
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
- `https://govoplan.add-ideas.de/catalogs/v1/keyring.json`
The console treats channels as release artifacts. A package can advance without
forcing every repository to the same tag, but channel publication must preserve
the unchanged package versions, validate interface compatibility, sign the
updated catalog, and keep the published keyring healthy.
When a selected module exposes a WebUI package, its requested version must also
match Core's `webui/package.release.json` input and the resolved
`package-lock.release.json` entry. The source-tag preflight, selective plan, and
catalog-candidate writer all enforce this composition boundary. Pins for modules
that are not part of the selective release remain unchanged.
Release integration also enforces repository and composition version alignment
and generates a CycloneDX SBOM from the resolved Python environment and the
release WebUI lockfile. Catalog publication should attach that immutable SBOM
and its digest to the corresponding Core/composition release.
Addresses and Notifications have current WebUI source contributions but are
not part of the pinned v0.1.8 WebUI release composition because their v0.1.8
tags predate those packages. They re-enter the release composition only through
new, immutable module tags whose backend, manifest, frontend, and lock metadata
pass the alignment gate.
Candidate publication verifies signatures against the already published
keyring, not against keys supplied only by the candidate. A changed keyring
must have its canonical SHA-256 embedded in the signed catalog. The public
module-directory files are regenerated from that verified catalog and keyring
at publication time; candidate-supplied directory files are never copied as
authoritative provenance.
## Published Module Directory
The target release repository is an online, browsable module directory. The
catalog remains the machine-readable channel entry point, but the published
space should also expose a tree that operators and installations can inspect:
- `/catalogs/v1/channels/<channel>.json` describes the active channel state.
- `/catalogs/v1/keyring.json` publishes trusted release signing keys.
- `/catalogs/v1/modules/<module>/<version>/manifest.json` describes one
published module version, including repository refs, package refs, contracts,
compatibility windows, signatures, and available artifacts.
- `/catalogs/v1/modules/<module>/index.json` lists available versions for
one module.
- `/catalogs/v1/modules/index.json` lists all published modules.
Gitea tags/releases remain the source release anchors. The public GovOPlaN
catalog directory becomes the installation-facing release repository that points
to those anchors and carries structured compatibility information.
Selective catalog candidates now include this directory tree. Publishing a
candidate copies the channel catalog, keyring, and `modules/` tree into the
website repository together.
Selective catalog generation can add a module that is not yet represented in
the source channel. Initial entries are synthesized from the selected
repository's `[project]` metadata, `govoplan.modules` entry point, and runtime
`ModuleManifest`; there is no separate hand-maintained initial-entry list. The
release gate requires the entry point to resolve inside the selected checkout,
exact package/manifest/frontend version agreement, verified source-tag
provenance, a non-conflicting module id, and complete required dependency and
interface closure in the resulting candidate. Classification beyond the
manifest-derived `official` tag remains an explicit later catalog curation
step rather than an inferred business/service label.
## Direction
The console should grow in slices:
1. Read-only dashboard and next-action suggestions.
2. Release plan builder that writes explicit JSON plans.
3. Dry-run executor that shows exact commands and expected file changes.
4. Apply executor for commit, tag, push, artifact build, signing, and catalog
publication.
5. Compatibility planner for manifest contracts and version ranges.
6. Install/test workflow that validates a release from tags or catalog entries.
All mutation must stay explicit: plan first, dry run second, apply only after a
clear confirmation.

89
docs/REPOSITORY_INDEX.md Normal file
View File

@@ -0,0 +1,89 @@
# GovOPlaN Repository Index
Generated from `repositories.json`. Use that JSON file as the machine-readable source of truth; this page is the human-readable link index.
## System
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `govoplan` | `meta` | `../govoplan` | [govoplan](https://git.add-ideas.de/add-ideas/govoplan) |
| `govoplan-core` | `kernel` | `../govoplan-core` | [govoplan-core](https://git.add-ideas.de/add-ideas/govoplan-core) |
## Module
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `govoplan-access` | `platform` | `../govoplan-access` | [govoplan-access](https://git.add-ideas.de/add-ideas/govoplan-access) |
| `govoplan-addresses` | `domain` | `../govoplan-addresses` | [govoplan-addresses](https://git.add-ideas.de/add-ideas/govoplan-addresses) |
| `govoplan-admin` | `platform` | `../govoplan-admin` | [govoplan-admin](https://git.add-ideas.de/add-ideas/govoplan-admin) |
| `govoplan-appointments` | `domain` | `../govoplan-appointments` | [govoplan-appointments](https://git.add-ideas.de/add-ideas/govoplan-appointments) |
| `govoplan-approvals` | `domain` | `../govoplan-approvals` | [govoplan-approvals](https://git.add-ideas.de/add-ideas/govoplan-approvals) |
| `govoplan-assets` | `domain` | `../govoplan-assets` | [govoplan-assets](https://git.add-ideas.de/add-ideas/govoplan-assets) |
| `govoplan-audit` | `platform` | `../govoplan-audit` | [govoplan-audit](https://git.add-ideas.de/add-ideas/govoplan-audit) |
| `govoplan-booking` | `domain` | `../govoplan-booking` | [govoplan-booking](https://git.add-ideas.de/add-ideas/govoplan-booking) |
| `govoplan-calendar` | `domain` | `../govoplan-calendar` | [govoplan-calendar](https://git.add-ideas.de/add-ideas/govoplan-calendar) |
| `govoplan-campaign` | `domain` | `../govoplan-campaign` | [govoplan-campaign](https://git.add-ideas.de/add-ideas/govoplan-campaign) |
| `govoplan-cases` | `domain` | `../govoplan-cases` | [govoplan-cases](https://git.add-ideas.de/add-ideas/govoplan-cases) |
| `govoplan-certificates` | `domain` | `../govoplan-certificates` | [govoplan-certificates](https://git.add-ideas.de/add-ideas/govoplan-certificates) |
| `govoplan-committee` | `domain` | `../govoplan-committee` | [govoplan-committee](https://git.add-ideas.de/add-ideas/govoplan-committee) |
| `govoplan-consultation` | `domain` | `../govoplan-consultation` | [govoplan-consultation](https://git.add-ideas.de/add-ideas/govoplan-consultation) |
| `govoplan-contracts` | `domain` | `../govoplan-contracts` | [govoplan-contracts](https://git.add-ideas.de/add-ideas/govoplan-contracts) |
| `govoplan-dashboard` | `platform` | `../govoplan-dashboard` | [govoplan-dashboard](https://git.add-ideas.de/add-ideas/govoplan-dashboard) |
| `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/add-ideas/govoplan-dms) |
| `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/add-ideas/govoplan-dist-lists) |
| `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/add-ideas/govoplan-docs) |
| `govoplan-erp` | `domain` | `../govoplan-erp` | [govoplan-erp](https://git.add-ideas.de/add-ideas/govoplan-erp) |
| `govoplan-evaluation` | `domain` | `../govoplan-evaluation` | [govoplan-evaluation](https://git.add-ideas.de/add-ideas/govoplan-evaluation) |
| `govoplan-facilities` | `domain` | `../govoplan-facilities` | [govoplan-facilities](https://git.add-ideas.de/add-ideas/govoplan-facilities) |
| `govoplan-files` | `domain` | `../govoplan-files` | [govoplan-files](https://git.add-ideas.de/add-ideas/govoplan-files) |
| `govoplan-forms` | `domain` | `../govoplan-forms` | [govoplan-forms](https://git.add-ideas.de/add-ideas/govoplan-forms) |
| `govoplan-forms-runtime` | `platform` | `../govoplan-forms-runtime` | [govoplan-forms-runtime](https://git.add-ideas.de/add-ideas/govoplan-forms-runtime) |
| `govoplan-grants` | `domain` | `../govoplan-grants` | [govoplan-grants](https://git.add-ideas.de/add-ideas/govoplan-grants) |
| `govoplan-helpdesk` | `domain` | `../govoplan-helpdesk` | [govoplan-helpdesk](https://git.add-ideas.de/add-ideas/govoplan-helpdesk) |
| `govoplan-identity` | `platform` | `../govoplan-identity` | [govoplan-identity](https://git.add-ideas.de/add-ideas/govoplan-identity) |
| `govoplan-identity-trust` | `platform` | `../govoplan-identity-trust` | [govoplan-identity-trust](https://git.add-ideas.de/add-ideas/govoplan-identity-trust) |
| `govoplan-idm` | `platform` | `../govoplan-idm` | [govoplan-idm](https://git.add-ideas.de/add-ideas/govoplan-idm) |
| `govoplan-inspections` | `domain` | `../govoplan-inspections` | [govoplan-inspections](https://git.add-ideas.de/add-ideas/govoplan-inspections) |
| `govoplan-issue-reporting` | `domain` | `../govoplan-issue-reporting` | [govoplan-issue-reporting](https://git.add-ideas.de/add-ideas/govoplan-issue-reporting) |
| `govoplan-learning` | `domain` | `../govoplan-learning` | [govoplan-learning](https://git.add-ideas.de/add-ideas/govoplan-learning) |
| `govoplan-ledger` | `domain` | `../govoplan-ledger` | [govoplan-ledger](https://git.add-ideas.de/add-ideas/govoplan-ledger) |
| `govoplan-mail` | `domain` | `../govoplan-mail` | [govoplan-mail](https://git.add-ideas.de/add-ideas/govoplan-mail) |
| `govoplan-notifications` | `platform` | `../govoplan-notifications` | [govoplan-notifications](https://git.add-ideas.de/add-ideas/govoplan-notifications) |
| `govoplan-ops` | `platform` | `../govoplan-ops` | [govoplan-ops](https://git.add-ideas.de/add-ideas/govoplan-ops) |
| `govoplan-organizations` | `platform` | `../govoplan-organizations` | [govoplan-organizations](https://git.add-ideas.de/add-ideas/govoplan-organizations) |
| `govoplan-payments` | `domain` | `../govoplan-payments` | [govoplan-payments](https://git.add-ideas.de/add-ideas/govoplan-payments) |
| `govoplan-permits` | `domain` | `../govoplan-permits` | [govoplan-permits](https://git.add-ideas.de/add-ideas/govoplan-permits) |
| `govoplan-policy` | `platform` | `../govoplan-policy` | [govoplan-policy](https://git.add-ideas.de/add-ideas/govoplan-policy) |
| `govoplan-poll` | `domain` | `../govoplan-poll` | [govoplan-poll](https://git.add-ideas.de/add-ideas/govoplan-poll) |
| `govoplan-portal` | `domain` | `../govoplan-portal` | [govoplan-portal](https://git.add-ideas.de/add-ideas/govoplan-portal) |
| `govoplan-postbox` | `domain` | `../govoplan-postbox` | [govoplan-postbox](https://git.add-ideas.de/add-ideas/govoplan-postbox) |
| `govoplan-procurement` | `domain` | `../govoplan-procurement` | [govoplan-procurement](https://git.add-ideas.de/add-ideas/govoplan-procurement) |
| `govoplan-records` | `domain` | `../govoplan-records` | [govoplan-records](https://git.add-ideas.de/add-ideas/govoplan-records) |
| `govoplan-reporting` | `domain` | `../govoplan-reporting` | [govoplan-reporting](https://git.add-ideas.de/add-ideas/govoplan-reporting) |
| `govoplan-resources` | `domain` | `../govoplan-resources` | [govoplan-resources](https://git.add-ideas.de/add-ideas/govoplan-resources) |
| `govoplan-risk-compliance` | `domain` | `../govoplan-risk-compliance` | [govoplan-risk-compliance](https://git.add-ideas.de/add-ideas/govoplan-risk-compliance) |
| `govoplan-scheduling` | `domain` | `../govoplan-scheduling` | [govoplan-scheduling](https://git.add-ideas.de/add-ideas/govoplan-scheduling) |
| `govoplan-search` | `platform` | `../govoplan-search` | [govoplan-search](https://git.add-ideas.de/add-ideas/govoplan-search) |
| `govoplan-tasks` | `domain` | `../govoplan-tasks` | [govoplan-tasks](https://git.add-ideas.de/add-ideas/govoplan-tasks) |
| `govoplan-templates` | `domain` | `../govoplan-templates` | [govoplan-templates](https://git.add-ideas.de/add-ideas/govoplan-templates) |
| `govoplan-tenancy` | `platform` | `../govoplan-tenancy` | [govoplan-tenancy](https://git.add-ideas.de/add-ideas/govoplan-tenancy) |
| `govoplan-transparency` | `domain` | `../govoplan-transparency` | [govoplan-transparency](https://git.add-ideas.de/add-ideas/govoplan-transparency) |
| `govoplan-workflow` | `platform` | `../govoplan-workflow` | [govoplan-workflow](https://git.add-ideas.de/add-ideas/govoplan-workflow) |
## Connector
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `govoplan-connectors` | `connector-hub` | `../govoplan-connectors` | [govoplan-connectors](https://git.add-ideas.de/add-ideas/govoplan-connectors) |
| `govoplan-fit-connect` | `standard` | `../govoplan-fit-connect` | [govoplan-fit-connect](https://git.add-ideas.de/add-ideas/govoplan-fit-connect) |
| `govoplan-rest` | `protocol` | `../govoplan-rest` | [govoplan-rest](https://git.add-ideas.de/add-ideas/govoplan-rest) |
| `govoplan-soap` | `protocol` | `../govoplan-soap` | [govoplan-soap](https://git.add-ideas.de/add-ideas/govoplan-soap) |
| `govoplan-xoev` | `standard` | `../govoplan-xoev` | [govoplan-xoev](https://git.add-ideas.de/add-ideas/govoplan-xoev) |
| `govoplan-xrechnung` | `standard` | `../govoplan-xrechnung` | [govoplan-xrechnung](https://git.add-ideas.de/add-ideas/govoplan-xrechnung) |
| `govoplan-xta-osci` | `standard` | `../govoplan-xta-osci` | [govoplan-xta-osci](https://git.add-ideas.de/add-ideas/govoplan-xta-osci) |
## Website
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `addideas-govoplan-website` | `public-site` | `../addideas-govoplan-website` | [addideas-govoplan-website](https://git.add-ideas.de/add-ideas/addideas-govoplan-website) |

View File

@@ -33,12 +33,37 @@ cd /mnt/DATA/git/govoplan
tools/checks/security-audit/run.sh --mode full --scope govoplan tools/checks/security-audit/run.sh --mode full --scope govoplan
``` ```
Reports are written to `audit-reports/`, which is intentionally ignored by git. When invoked from a Flatpak development environment without a sandbox-local
Docker CLI, the wrapper automatically uses `flatpak-spawn --host docker`. The
host account must still be allowed to open the Docker daemon socket. For a
conventional rootful installation this commonly means membership in the
`docker` group followed by a complete logout/login; that membership is
root-equivalent, so rootless Docker is preferable where the deployment policy
requires a smaller privilege boundary.
The wrapper tags the toolbox image by a fingerprint of the Dockerfile and Reports are written to `audit-reports/`, which is intentionally ignored by git.
`requirements-audit.txt`. If those inputs have not changed, subsequent runs reuse Each run records tool versions, report checksums, and start/end repository
the existing local image instead of reinstalling all tools. The stable alias is revision plus worktree fingerprints. A repository change during scanning makes
`govoplan/security-audit:local` unless `SECURITY_AUDIT_IMAGE` is set. the run fail so a mixed code snapshot cannot be reported as a valid audit.
Step exit codes are recorded separately from findings: report-only mode may
accept findings, but scanner execution errors and malformed JSON/SARIF reports
always fail the run. The manifest lists the expected, present, and missing
reports for that invocation. Validation and checksums use that explicit set, so
reusing a report directory cannot make stale output look like part of a new run.
The wrapper tags the toolbox image by a fingerprint of the Dockerfile,
`requirements-audit.txt`, and the Semgrep smoke-test inputs. If those inputs have
not changed, subsequent runs reuse the existing local image instead of
reinstalling all tools. The stable alias is `govoplan/security-audit:local`
unless `SECURITY_AUDIT_IMAGE` is set.
Semgrep is installed separately in the toolbox image because current Semgrep
packages pin affected Click and MCP versions. The image upgrades both after
Semgrep installation, runs an actual local-rule scan, and audits the final
toolbox Python environment during the image build. The compatibility releases
are pinned exactly to keep the tested override reproducible. Remove either
compatibility override only after Semgrep's own dependency bounds include a
fixed version.
Force a cached rebuild: Force a cached rebuild:
@@ -65,6 +90,17 @@ tools/checks/security-audit/run.sh --mode quick --scope current --update --build
- `ci`: quick plus Semgrep public registry rulesets, Trivy, pip-audit, npm audit. - `ci`: quick plus Semgrep public registry rulesets, Trivy, pip-audit, npm audit.
- `full`: ci plus OSV-Scanner, jscpd, Radon, and Xenon. - `full`: ci plus OSV-Scanner, jscpd, Radon, and Xenon.
Semgrep and Trivy are invoked with finding-sensitive exit codes. Their exit 1
is therefore a finding under the wrapper contract; higher exit codes, missing
output, invalid JSON/SARIF, and scanner error payloads are execution failures.
Bandit and Ruff security reports are split by source kind. Production code under
`src/` is written to `bandit.json` and `ruff-security.json` and controls strict
mode. Test code under `tests/` is still scanned for visibility, but its findings
are written separately to `bandit-tests.json` and `ruff-security-tests.json` so
fixture passwords, assertions, and temporary paths do not hide the production
baseline.
## Gating ## Gating
The initial Gitea workflow runs in report-only mode: The initial Gitea workflow runs in report-only mode:
@@ -115,11 +151,25 @@ Keep active implementation status in issues instead of committing generated
audit reports. `audit-reports/` is ignored; quote the report directory and the audit reports. `audit-reports/` is ignored; quote the report directory and the
important scanner counts in the tracker issue. important scanner counts in the tracker issue.
The jscpd step is intentionally scoped to application and test source. It
excludes documentation snippets, package manifests, generated translations,
public SVG assets, workflow YAML, and declarative backend schema JSON because
those reports produce metadata or asset repetition rather than actionable source
duplication. Keep exclusions narrow and create child issues for source-code
clusters that cross module ownership or make behavior harder to change safely.
## Image Freshness ## Image Freshness
The regular `Security Audit` workflow reuses the fingerprinted toolbox image The regular `Security Audit` workflow reuses the fingerprinted toolbox image
when the Docker daemon is persistent, which is the normal case for the when the Docker daemon is persistent, which is the normal case for the
self-hosted Gitea runner using the host Docker socket. The separate self-hosted Gitea runner using the host Docker socket. Trusted push, schedule,
and manual runs scan all registered repositories; authenticated SSH is used
only for the private website repository. The wrapper inspects the Actions job
mount table and forwards only the narrowest writable mount covering the audit
scope; it never inherits the job's Docker socket or unrelated runner mounts.
Pull-request audit runs stay disabled while the audit runner exposes its host
Docker socket: PR-controlled audit code must run on a disposable or rootless
runner without host-socket access. The separate
`Security Audit Toolbox Update` workflow runs weekly with `Security Audit Toolbox Update` workflow runs weekly with
`SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the `SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the
allowed tool version ranges into a refreshed local image. allowed tool version ranges into a refreshed local image.
@@ -133,6 +183,11 @@ tools first:
cd /mnt/DATA/git/govoplan cd /mnt/DATA/git/govoplan
python -m venv .venv python -m venv .venv
./.venv/bin/python -m pip install -r requirements-audit.txt ./.venv/bin/python -m pip install -r requirements-audit.txt
./.venv/bin/python -m pip install 'semgrep>=1.140,<2'
./.venv/bin/python -m pip install --upgrade --no-deps 'click==8.3.3'
./.venv/bin/python -m pip install --upgrade --no-deps 'mcp==1.28.1'
./.venv/bin/semgrep scan --metrics=off --config tools/checks/security-audit/semgrep-govoplan.yml tools/checks/check-version-alignment.py
./.venv/bin/pip-audit --progress-spinner off
``` ```
Then install the non-Python tools (`gitleaks`, `trivy`, `osv-scanner`, `jscpd`) Then install the non-Python tools (`gitleaks`, `trivy`, `osv-scanner`, `jscpd`)

View File

@@ -0,0 +1,179 @@
# System Administrator Lifecycle User Story
## Story
> As a system administrator, I can execute one shell command that downloads a
> verified GovOPlaN distribution and starts a completely configured Core control
> plane without optional modules. In the WebUI I can browse compatible signed
> module releases, select the modules for this installation, and follow every
> download, validation, migration, installation, activation, and health-check
> step. When an update is available, I can review its impact and confirm it.
>
> I can add API or worker replicas on containers or other servers without
> copying mutable local state. I can operate development, test, and production
> systems, transfer a versioned configuration through the WebUI, undo an applied
> configuration version, test an update in one environment, and then apply the
> same immutable update recipe to another environment.
This is a product-level story owned by the GovOPlaN platform rather than by an
individual domain module. It joins installation, module lifecycle, operations,
configuration packages, and release provenance into one administrator journey.
The canonical backlog item is
[GovOPlaN #13](https://git.add-ideas.de/add-ideas/govoplan/issues/13).
## Terms
- **Core control plane:** the smallest bootable distribution: Core API, Core
WebUI, PostgreSQL, Redis, installer worker, migration runner, and durable
storage configuration. No optional GovOPlaN module package is installed.
- **Bootstrap administrator:** a single-use, time-limited installation identity
that may access only first-run and module-lifecycle functions. It is retired
when the selected identity/access configuration becomes healthy.
- **Module release:** an immutable, signed package plus manifest, compatibility
contract, migrations, WebUI contribution, checksums, and SBOM references.
- **Configuration revision:** an immutable, schema-versioned export of
non-secret system/module settings, policies, compositions, and secret
references. Secret values are never exported.
- **Update recipe:** the reviewed, immutable plan containing exact Core/module
versions, configuration revision, migration order, preflight results,
maintenance/drain requirements, health checks, and permitted rollback or
forward-recovery actions.
## Acceptance journeys
### One-command first installation
1. The administrator runs one documented command on a supported host.
2. The bootstrapper verifies a signed distribution manifest before executing or
starting downloaded artifacts.
3. It checks container/runtime, ports, storage, entropy, memory, architecture,
and connectivity requirements; generates deployment-local secrets with
restrictive permissions; and never prints them.
4. It starts PostgreSQL, Redis, Core API/WebUI, one installer worker, and the
migration runner. Readiness does not pass until migrations and durable
dependencies are healthy.
5. It prints the local URL and one-time bootstrap credential. Re-running the
command is idempotent and shows or repairs the existing installation rather
than creating another identity or database.
6. No optional module is installed or enabled at this point.
### Module selection, installation, and update
1. Core reads an approved signed catalog and trusted keyring through the shared
outbound-network policy.
2. The WebUI shows available, installed, compatible, blocked, withdrawn, and
update-available releases with their channel, provenance, contracts,
migrations, permissions, configuration requirements, and release notes.
3. Selecting modules produces a dependency/compatibility plan before any
mutation. The administrator can amend the selection or confirm the plan.
4. Installation executes durably in a worker. The UI receives persisted step
state and can reconnect without losing progress.
5. Package signature/checksum, version alignment, contract closure, migration
graph, configuration schema, and health checks are mandatory gates.
6. Remote artifacts are acquired by a deployment-owned downloader that permits
approved origins, pins the validated connection peer, enforces size and time
limits, and verifies signature/checksum before making a local immutable
artifact available. `pip`, `npm`, archive tools, and any compatibility Git
importer then run without unrestricted network access. A restricted-network
profile can satisfy the same contract through operator-prefetched artifacts.
7. A failed or interrupted operation reaches a visible retry, forward-recovery,
rollback, or manual-intervention state. It never reports success merely
because the initiating request returned.
8. An available update follows the same preview and confirmation path. Exact
repeated requests are idempotent.
### Horizontal scaling
1. API, WebUI, installer, scheduler, and ordinary worker roles are stateless
with respect to local container disks. Durable state uses PostgreSQL, Redis,
and configured shared file/object storage.
2. A documented command can add API or queue-specific worker replicas. The
default Compose profile supports local scale-out; an orchestrator profile
supplies equivalent health/readiness probes and rolling replacement.
3. Only one migration or module-lifecycle mutation may own the deployment lock,
while any healthy replica can serve read and normal domain traffic.
4. Workers announce identity, queues, software composition, heartbeat, and
drain state. Operators can see skew and safely retire a replica.
5. Sessions, throttling, idempotency, jobs, installer progress, and scheduled
work remain correct when requests move between replicas.
### Development, test, and production promotion
1. The administrator exports a configuration revision from development through
the WebUI. The package is versioned, checksummed, attributable, and contains
secret references or required-secret declarations, never secret values.
2. Test imports the package into a preview area. Core reports environment-bound
values, missing capabilities/secrets, compatibility changes, and the exact
apply plan.
3. Applying creates a new revision; the previous effective revision remains
addressable. Undo is a new audited revision that restores the earlier
configuration where contracts permit it.
4. After tests and health checks pass, Core emits an update recipe with exact
immutable release and configuration identifiers.
5. Production validates the recipe against its own environment, requires a new
confirmation, and executes the same ordered plan. Environment-specific
secret bindings and endpoints remain local.
6. Promotion never copies tenant/business data implicitly. Data migration,
anonymized fixtures, and backup/restore are separate explicit operations.
## Safety and governance requirements
- Catalogs, artifacts, recipes, and configuration revisions have signatures,
checksums, provenance, expiry/revocation semantics, and audit evidence.
- Catalog and artifact retrieval is not delegated to an unrestricted package
manager. Approved-origin and peer-pinned download, response bounds,
signature/checksum verification, extraction safety, offline installation, and
deployment egress policy form separate defenses.
- The installer has a dedicated narrow authority; normal domain permissions do
not imply host/package-management access.
- The UI distinguishes reversible configuration rollback from database or
package migrations that require forward recovery.
- Cluster-wide mutations use fencing/leases so an expired worker cannot later
commit a stale result.
- Update plans declare availability impact, worker draining, backup/restore
prerequisites, database compatibility windows, and post-change probes.
- Deployment profiles set secure headers, trusted proxies/hosts, body limits,
outbound-network policy, storage, TLS/cookie posture, and observability.
- Release publication requires aligned backend/frontend/manifest versions and
machine-readable dependency/SBOM provenance.
## Implementation slices
1. **Reproducible Core-only distribution.** Publish pinned multi-architecture
images, signed distribution manifest, Core-only Compose profile, bootstrap
preflight, generated secrets, readiness, and idempotent rerun/repair.
2. **First-run control plane.** Add the restricted bootstrap administrator,
one-time enrollment, initial catalog/keyring configuration, and retirement
after durable administrator access is established.
3. **Read-only online module directory.** Move the existing catalog and module
directory contracts into the installed Core WebUI with compatibility,
provenance, release-note, and update-state presentation.
4. **Durable module plan and install.** Reuse the existing installer queue,
locks, signed-package validator, rollback drill, and run evidence behind a
plan/confirm/progress UI. Add initial catalog-entry synthesis and artifact
acquisition where the current release console still assumes local sources.
5. **Safe module update.** Add drain/maintenance coordination, backup gate,
migration compatibility window, reconnectable progress, health verification,
retry/recovery, and update notification.
6. **Stateless replica profile.** Remove remaining local-runtime assumptions,
expose role-specific commands/images, implement worker registration/drain,
and prove multiple API and worker replicas against shared dependencies.
7. **Configuration revision model.** Define provider export/import schemas,
canonical serialization, secret references, validation/diff, immutable
revision storage, audit, apply, and undo-as-new-revision.
8. **Environment promotion and recipes.** Add source/target fingerprints,
preview, environment bindings, acceptance evidence, exact recipe generation,
signed transfer, and independently confirmed application.
9. **Operational proof.** Exercise interrupted installs, stale locks, unavailable
catalogs, revoked keys, failed migrations, replica loss, configuration undo,
and development-to-test-to-production promotion in release CI and target
drills.
## Explicit non-goals for the first distribution slice
- Shipping optional modules in the Core image.
- Exporting secrets or production business data with configuration.
- Pretending every schema migration can be reversed automatically.
- Building a proprietary orchestrator instead of supporting Compose and a
standard cluster scheduler through the same role/readiness contracts.
- Allowing the browser process to execute arbitrary shell commands.

View File

@@ -0,0 +1,148 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit-boundary-evidence.schema.json",
"title": "GovOPlaN externally issued capability-fit boundary evidence",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"proof_id",
"assessment_id",
"assessment_release",
"installed_evidence_sha256",
"issued_at",
"expires_at",
"claims",
"signatures"
],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.1.0"
},
"evidence_kind": {
"const": "govoplan.capability-fit-boundary-proof"
},
"proof_id": {
"$ref": "#/$defs/opaque_id"
},
"assessment_id": {
"$ref": "#/$defs/opaque_id"
},
"assessment_release": {
"$ref": "#/$defs/opaque_id"
},
"installed_evidence_sha256": {
"$ref": "#/$defs/sha256"
},
"issued_at": {
"type": "string",
"format": "date-time"
},
"expires_at": {
"type": "string",
"format": "date-time"
},
"claims": {
"type": "array",
"minItems": 1,
"maxItems": 3,
"items": {
"$ref": "#/$defs/claim"
}
},
"signatures": {
"type": "array",
"minItems": 1,
"maxItems": 16,
"items": {
"$ref": "#/$defs/signature"
}
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"claim": {
"type": "object",
"additionalProperties": false,
"required": ["scope", "result", "subject_id", "control_ids", "artifacts"],
"properties": {
"scope": {
"enum": [
"target_environment",
"external_providers",
"production_approval"
]
},
"result": {
"enum": ["passed", "failed", "approved", "rejected"]
},
"subject_id": {
"$ref": "#/$defs/opaque_id"
},
"control_ids": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"uniqueItems": true,
"items": {
"$ref": "#/$defs/opaque_id"
}
},
"artifacts": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"items": {
"$ref": "#/$defs/evidence_artifact"
}
}
}
},
"evidence_artifact": {
"type": "object",
"additionalProperties": false,
"required": ["artifact_id", "sha256"],
"properties": {
"artifact_id": {
"$ref": "#/$defs/opaque_id"
},
"sha256": {
"$ref": "#/$defs/sha256"
}
}
},
"signature": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "key_id", "value"],
"properties": {
"algorithm": {
"const": "ed25519"
},
"key_id": {
"$ref": "#/$defs/opaque_id"
},
"value": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
}
}
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}

View File

@@ -0,0 +1,978 @@
{
"$schema": "./capability-fit.schema.json",
"schema_version": "0.1.0",
"assessment_id": "campaign-reference-2026-07-22",
"assessed_at": "2026-07-22",
"scope": {
"title": "Campaign-centric internal pilot and small-production candidate",
"reference_journeys": [
"Internal operator authors, validates, builds, queues, sends and reconciles an email Campaign with managed attachments",
"Operator inspects delivery and audit evidence"
],
"postponed": [
"Workflow and workflow-driven user stories"
]
},
"release": {
"kind": "tagged_release",
"ref": "stable-catalog-202607220843",
"meta_commit": "5447299289a1",
"reproducible": true,
"configuration_packages": [],
"notes": [
"The live stable catalog has a valid Ed25519 signature trusted through release-key-1.",
"Core v0.1.13 and Campaign v0.1.10 are tagged and package-integrated; this is not target-environment or production approval.",
"No configuration revision or configuration package is pinned yet."
]
},
"composition": [
{
"module_id": "core",
"repository": "govoplan-core",
"commit": "d487726f4d2c",
"manifest_version": "0.1.13",
"enabled": true,
"dirty": false,
"role": "API, registry, migrations, sessions, kernel contracts and shared WebUI"
},
{
"module_id": "tenancy",
"repository": "govoplan-tenancy",
"commit": "efbec827616b",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Tenant context and lifecycle"
},
{
"module_id": "organizations",
"repository": "govoplan-organizations",
"commit": "39c081c4fb8f",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Organization model"
},
{
"module_id": "identity",
"repository": "govoplan-identity",
"commit": "7a1710af896f",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Normalized internal identity directory"
},
{
"module_id": "access",
"repository": "govoplan-access",
"commit": "f1d64d247e12",
"manifest_version": "0.1.11",
"enabled": true,
"dirty": false,
"role": "Local authentication, sessions, API keys and RBAC"
},
{
"module_id": "admin",
"repository": "govoplan-admin",
"commit": "11ecf362a36d",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Administration surfaces"
},
{
"module_id": "dashboard",
"repository": "govoplan-dashboard",
"commit": "4b960ad37f0d",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Module-aware home surface"
},
{
"module_id": "policy",
"repository": "govoplan-policy",
"commit": "1063622d311a",
"manifest_version": "0.1.9",
"enabled": true,
"dirty": false,
"role": "Policy explanation and configuration boundary"
},
{
"module_id": "audit",
"repository": "govoplan-audit",
"commit": "d3d2c60d7dc1",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Database audit records and retrying audit outbox"
},
{
"module_id": "campaigns",
"repository": "govoplan-campaign",
"commit": "735e874bd03c",
"manifest_version": "0.1.10",
"enabled": true,
"dirty": false,
"role": "Campaign authoring, build, delivery control and reporting"
},
{
"module_id": "files",
"repository": "govoplan-files",
"commit": "2b34f6e30578",
"manifest_version": "0.1.9",
"enabled": true,
"dirty": false,
"role": "Managed files and Campaign attachments"
},
{
"module_id": "mail",
"repository": "govoplan-mail",
"commit": "3e2302909022",
"manifest_version": "0.1.10",
"enabled": true,
"dirty": false,
"role": "SMTP and IMAP profiles and transports"
},
{
"module_id": "calendar",
"repository": "govoplan-calendar",
"commit": "9bcf41bb1fbb",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Optional calendar outside the Campaign pilot minimum"
},
{
"module_id": "docs",
"repository": "govoplan-docs",
"commit": "be52b716caed",
"manifest_version": "0.1.10",
"enabled": true,
"dirty": false,
"role": "Configured-system documentation"
},
{
"module_id": "ops",
"repository": "govoplan-ops",
"commit": "341773a4ff8a",
"manifest_version": "0.1.8",
"enabled": true,
"dirty": false,
"role": "Readiness and deployment-profile visibility"
},
{
"module_id": "addresses",
"repository": "govoplan-addresses",
"commit": "93dddbb8c52a",
"manifest_version": "0.1.9",
"enabled": false,
"dirty": false,
"role": "Optional reusable recipient sources and CardDAV"
}
],
"deployment_profile": {
"id": "production-like-dev",
"status": "partial",
"description": "PostgreSQL and Redis run in containers while API, WebUI, worker and scheduler run from editable source trees.",
"evidence": [
{
"kind": "configuration",
"scope": "current_workspace",
"locator": "govoplan/dev/production-like/docker-compose.yml"
},
{
"kind": "documentation",
"scope": "documented_model",
"locator": "govoplan/dev/production-like/README.md"
}
]
},
"questionnaire": {
"scope_outcomes": [
{
"id": "outcome.reference_journey",
"question": "Which journey is assessed?",
"state": "answered",
"answer": "An internal operator authors, validates, builds, queues, sends and reconciles a Campaign with managed attachments.",
"evidence": []
},
{
"id": "outcome.workflow",
"question": "Is Workflow in scope?",
"state": "answered",
"answer": "No; Workflow is planned and explicitly postponed.",
"evidence": []
}
],
"data_policy": [
{
"id": "data.classification",
"question": "Which data classes and legal bases apply?",
"state": "not_assessed",
"answer": null,
"evidence": []
},
{
"id": "data.retention",
"question": "What retention, deletion, archive and legal-hold rules apply?",
"state": "not_assessed",
"answer": null,
"evidence": []
}
],
"identity_integrations": [
{
"id": "identity.pilot",
"question": "May the pilot use local GovOPlaN accounts?",
"state": "assumed",
"answer": "Yes; federation is outside the verified composition.",
"evidence": []
},
{
"id": "integration.mail",
"question": "Which target SMTP/IMAP service and policy apply?",
"state": "not_assessed",
"answer": null,
"evidence": []
}
],
"workload_growth": [
{
"id": "workload.campaign",
"question": "What are Campaign frequency, recipients per Campaign, send window, import size and attachment volume?",
"state": "not_assessed",
"answer": null,
"evidence": []
},
{
"id": "workload.platform",
"question": "What are tenant, user, concurrency, file, database, queue and audit growth assumptions?",
"state": "not_assessed",
"answer": null,
"evidence": []
}
],
"availability_operations": [
{
"id": "availability.rto_rpo",
"question": "What availability, RPO and RTO are required?",
"state": "not_assessed",
"answer": null,
"evidence": []
},
{
"id": "operations.ownership",
"question": "Who operates database, queue, storage, TLS, secrets, monitoring, backup and incident response?",
"state": "not_assessed",
"answer": null,
"evidence": []
}
],
"procurement_decisions": [
{
"id": "procurement.constraints",
"question": "Which licensing, accessibility, security, certification, support and procurement conditions are mandatory?",
"state": "not_assessed",
"answer": null,
"evidence": []
}
]
},
"capabilities": [
{
"id": "platform.composition",
"requirement": "Compose enabled backend and WebUI modules without hard optional-module dependencies.",
"status": "verified",
"evidence": [
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-core/tests/test_module_system.py"
},
{
"kind": "contract",
"scope": "current_workspace",
"locator": "govoplan/tools/checks/check-contracts.py",
"note": "43 modules, 33 providers, 19 requirements, no issues"
}
],
"conditions": [
"Package integration is verified; repeat checks on the installed target composition."
],
"gaps": [
"No target deployment acceptance is recorded."
],
"risks": [
"A reproducible module graph can still be installed or configured incorrectly."
],
"recommendation": "Use the signed stable catalog and verify the minimal Campaign composition after installation.",
"proof_check": "Run contract, migration, API and WebUI module-permutation gates on the installed release."
},
{
"id": "access.local",
"requirement": "Provide tenant-scoped local accounts, sessions, API keys and RBAC.",
"status": "verified",
"evidence": [
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-access/tests/test_auth_dependencies.py"
},
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-core/tests/test_api_smoke.py#cookie-session-csrf"
}
],
"conditions": [
"Pilot accepts local accounts."
],
"gaps": [
"MFA and federated lifecycle are not part of this conclusion."
],
"risks": [
"Manual account lifecycle may not satisfy production identity policy."
],
"recommendation": "Use controlled local pilot accounts and define break-glass/bootstrap rules.",
"proof_check": "Exercise joiner, role change, suspension and protected-owner recovery."
},
{
"id": "campaign.journey",
"requirement": "Author, validate, build, queue, send, reconcile and report a Campaign with frozen execution evidence.",
"status": "verified",
"evidence": [
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-core/tests/test_api_smoke.py#campaign-create-validate-build-mock-send"
},
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-campaign/tests",
"note": "Campaign v0.1.10 is exactly the catalog-selected tagged source"
},
{
"kind": "configuration",
"scope": "committed_source",
"locator": "https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json#sequence-202607220843",
"note": "Core v0.1.13 and Campaign v0.1.10 have matching catalogued Python and WebUI refs"
}
],
"conditions": [
"This verifies implementation paths, not target-provider delivery."
],
"gaps": [
"Usability and target-provider acceptance remain separate."
],
"risks": [
"Package integration does not prove provider behavior or production operations."
],
"recommendation": "Use the catalogued Campaign release for usability and target-provider acceptance.",
"proof_check": "Run the complete journey with safe data and the target-like mail service."
},
{
"id": "files.managed_attachments",
"requirement": "Store and resolve managed Campaign attachments on durable storage.",
"status": "verified",
"evidence": [
{
"kind": "test",
"scope": "current_workspace",
"locator": "govoplan-files/tests",
"note": "14 tests passed"
},
{
"kind": "test",
"scope": "current_workspace",
"locator": "govoplan-campaign/tests/test_attachment_building.py"
}
],
"conditions": [
"Deployment provides a durable storage root."
],
"gaps": [
"Target backup and restore are not verified."
],
"risks": [
"Node-local storage prevents safe independent API scaling."
],
"recommendation": "Use durable local storage for the pilot and assess object/shared storage before scaling.",
"proof_check": "Back up and restore files together with database references."
},
{
"id": "mail.smtp_imap",
"requirement": "Send Campaign mail through SMTP and optionally append sent messages through IMAP.",
"status": "available_unconfigured",
"evidence": [
{
"kind": "test",
"scope": "current_workspace",
"locator": "govoplan-mail/tests",
"note": "22 tests passed"
},
{
"kind": "documentation",
"scope": "documented_model",
"locator": "govoplan-campaign/docs/CAMPAIGN_DELIVERY_RUNBOOK.md"
}
],
"conditions": [
"Use a dedicated non-production service account and safe recipients."
],
"gaps": [
"No target provider, TLS chain, throttling or bounce/reply process was exercised."
],
"risks": [
"Ambiguous provider outcomes can cause duplicate-send risk if reconciled incorrectly."
],
"recommendation": "Run target-like interoperability and failure drills before production use.",
"proof_check": "Prove SMTP acceptance, IMAP append, throttling and outcome reconciliation."
},
{
"id": "addresses.recipient_sources",
"requirement": "Select reusable address lists as Campaign recipient sources.",
"status": "available_unconfigured",
"evidence": [
{
"kind": "test",
"scope": "current_workspace",
"locator": "govoplan-addresses/tests",
"note": "14 tests passed"
}
],
"conditions": [
"Enable the Addresses module explicitly."
],
"gaps": [
"Addresses is disabled in the pinned root profile."
],
"risks": [
"Recipient governance may differ between source data and frozen Campaign evidence."
],
"recommendation": "Enable only when reusable lists are a pilot requirement.",
"proof_check": "Build a Campaign from a source list and verify immutable recipient provenance."
},
{
"id": "audit.local",
"requirement": "Retain tenant/system audit evidence and retry governed audit events.",
"status": "verified",
"evidence": [
{
"kind": "test",
"scope": "current_workspace",
"locator": "govoplan-audit/tests",
"note": "5 tests passed"
}
],
"conditions": [
"Conclusion covers local database evidence only."
],
"gaps": [
"No central sink, retention enforcement or tamper-evident archive is verified."
],
"risks": [
"Local audit evidence may not satisfy organizational records or SIEM requirements."
],
"recommendation": "Define retention and export requirements before production approval.",
"proof_check": "Exercise privileged-event review, retention and any required external export."
},
{
"id": "identity.federation",
"requirement": "Integrate external LDAP/AD, OIDC/SAML or SCIM identity infrastructure.",
"status": "scaffold",
"evidence": [
{
"kind": "documentation",
"scope": "documented_model",
"locator": "govoplan-idm/README.md"
}
],
"conditions": [],
"gaps": [
"No end-to-end provider connector or federated login is verified."
],
"risks": [
"Federation-dependent organizations cannot use the current pilot composition without extra implementation."
],
"recommendation": "Use local pilot accounts or assess and implement the selected provider path.",
"proof_check": "Run provider metadata, login/provisioning, deprovisioning and failure tests."
},
{
"id": "compliance.export_control",
"requirement": "Screen persons and organizations against embargo/sanctions lists with review evidence.",
"status": "planned",
"evidence": [
{
"kind": "issue",
"scope": "documented_model",
"locator": "https://git.add-ideas.de/add-ideas/govoplan/issues/12"
}
],
"conditions": [],
"gaps": [
"No provider, list provenance, match policy, review flow or legal evidence exists."
],
"risks": [
"The current composition must not be represented as performing export-control screening."
],
"recommendation": "Keep outside pilot claims until the user story is implemented and legally validated.",
"proof_check": "Validate list ingestion, versioning, matching, false-positive review and audit evidence."
},
{
"id": "workflow",
"requirement": "Orchestrate the journey through Workflow.",
"status": "planned",
"evidence": [
{
"kind": "observation",
"scope": "documented_model",
"locator": "Assessment scope",
"note": "Explicitly postponed"
}
],
"conditions": [],
"gaps": [
"Workflow is outside this assessment."
],
"risks": [
"Including it would overstate the assessed composition."
],
"recommendation": "Do not enable or claim Workflow for this reference pilot.",
"proof_check": "Reassess in a later Workflow-focused composition."
}
],
"infrastructure": [
{
"id": "runtime.web_api",
"requirement": "Serve matching WebUI and API artifacts with health endpoints.",
"status": "verified",
"evidence": [
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-core/tests/test_module_system.py"
},
{
"kind": "route",
"scope": "committed_source",
"locator": "govoplan-core/src/govoplan_core/server/fastapi.py#/health"
}
],
"conditions": [
"Materialize the matching catalogued artifacts in the target."
],
"gaps": [
"No production image or service bundle is supplied by the profile."
],
"risks": [
"Editable source processes are unsuitable as a production artifact."
],
"recommendation": "Install matching catalogued WebUI/API refs and supervise them as immutable artifacts.",
"proof_check": "Deploy the built artifacts and run health/module-route checks."
},
{
"id": "runtime.worker",
"requirement": "Run durable asynchronous Campaign jobs.",
"status": "available_unconfigured",
"evidence": [
{
"kind": "configuration",
"scope": "current_workspace",
"locator": "govoplan/tools/launch/launch-production-like-dev.sh"
}
],
"conditions": [
"Redis and a supervised worker are required when Celery is enabled."
],
"gaps": [
"Target heartbeat, restart and queue-age alerting are not proved."
],
"risks": [
"Queued work can stall silently without monitoring."
],
"recommendation": "Start one worker for the pilot and split queues only after measurement.",
"proof_check": "Interrupt and restart a worker while preserving job/reconciliation safety."
},
{
"id": "runtime.scheduler",
"requirement": "Run periodic recovery and cleanup safely.",
"status": "partial",
"evidence": [
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan-calendar/tests/test_outbox.py",
"note": "Committed and pushed after the catalogued Calendar v0.1.8 tag"
}
],
"conditions": [
"Calendar outbox and recovery work is remote-integrated source but not stable-package-integrated."
],
"gaps": [
"No distributed leader election or target supervision is established."
],
"risks": [
"Multiple schedulers can duplicate periodic dispatch without locking."
],
"recommendation": "Omit from the Campaign-only pilot or run one supervised instance.",
"proof_check": "Prove missed-schedule recovery and single-leader behavior."
},
{
"id": "data.postgresql",
"requirement": "Persist application state in PostgreSQL with explicit migrations.",
"status": "verified",
"evidence": [
{
"kind": "configuration",
"scope": "committed_source",
"locator": "govoplan/dev/postgres"
},
{
"kind": "test",
"scope": "committed_source",
"locator": "govoplan/tools/checks/postgres-integration-check.py"
}
],
"conditions": [
"Target database remains deployment-owned."
],
"gaps": [
"HA, patching, WAL policy and capacity are not assessed."
],
"risks": [
"A single unprotected database is a system-wide failure point."
],
"recommendation": "Use managed or dedicated PostgreSQL with explicit migration and backup controls.",
"proof_check": "Run migrations and restore a target-like database."
},
{
"id": "queue.redis",
"requirement": "Provide the Celery broker and queue persistence.",
"status": "available_unconfigured",
"evidence": [
{
"kind": "configuration",
"scope": "current_workspace",
"locator": "govoplan/dev/production-like/docker-compose.yml#redis"
}
],
"conditions": [],
"gaps": [
"Authentication, TLS, eviction, HA and queue-loss policy are not assessed."
],
"risks": [
"Broker loss or eviction can delay work even when database business state survives."
],
"recommendation": "Configure private persistent Redis and monitor queue age/depth.",
"proof_check": "Exercise broker interruption and worker recovery."
},
{
"id": "storage.local",
"requirement": "Persist managed files on a durable single-node/shared path.",
"status": "verified",
"evidence": [
{
"kind": "contract",
"scope": "committed_source",
"locator": "govoplan-files/src/govoplan_files/backend/storage/backends.py"
}
],
"conditions": [
"Path is durable, private, writable and backed up."
],
"gaps": [
"Node-local storage cannot support independent API replicas."
],
"risks": [
"Files can be lost or become inconsistent with database state."
],
"recommendation": "Use for a bounded pilot only with coordinated backup.",
"proof_check": "Restore files and verify all database references."
},
{
"id": "storage.object",
"requirement": "Use S3-compatible storage for independently scalable file persistence.",
"status": "partial",
"evidence": [
{
"kind": "test",
"scope": "current_workspace",
"locator": "govoplan-files/tests/test_connector_providers.py"
}
],
"conditions": [],
"gaps": [
"No chosen target service or storage-backend interoperability drill."
],
"risks": [
"Provider semantics, CA or lifecycle mismatch can break file access/retention."
],
"recommendation": "Select and exercise the target object store before horizontal scaling.",
"proof_check": "Upload, retrieve, version, back up and restore representative objects."
},
{
"id": "edge.proxy_tls",
"requirement": "Terminate HTTPS and enforce proxy/security policy.",
"status": "external_system",
"evidence": [
{
"kind": "route",
"scope": "committed_source",
"locator": "govoplan-ops/src/govoplan_ops/backend/api/v1/routes.py#deployment-security"
}
],
"conditions": [],
"gaps": [
"No proxy, certificates, renewal, header or request-limit configuration is shipped here."
],
"risks": [
"Incorrect proxy/cookie/CORS configuration can expose sessions or block legitimate use."
],
"recommendation": "Supply and monitor the edge through the target platform.",
"proof_check": "Run external TLS/header/cookie/CORS and upload-limit tests."
},
{
"id": "security.secret_store",
"requirement": "Inject and rotate master, database, mail and connector secrets.",
"status": "external_system",
"evidence": [
{
"kind": "configuration",
"scope": "committed_source",
"locator": "govoplan/.env.example"
}
],
"conditions": [],
"gaps": [
"No target secret manager or rotation drill is selected."
],
"risks": [
"Loss of the master key makes encrypted credentials unavailable; leakage compromises connectors."
],
"recommendation": "Use target-native secret injection and document rotation/recovery.",
"proof_check": "Rotate a non-production credential and recover from a protected backup."
},
{
"id": "operations.monitoring",
"requirement": "Detect API, database, worker, queue, storage and delivery degradation.",
"status": "partial",
"evidence": [
{
"kind": "route",
"scope": "committed_source",
"locator": "govoplan-ops/src/govoplan_ops/backend/api/v1/routes.py#/ops/readiness"
},
{
"kind": "contract",
"scope": "committed_source",
"locator": "govoplan-core/src/govoplan_core/server/fastapi.py#slow-request-logging"
}
],
"conditions": [],
"gaps": [
"No metrics exporter, log collector, dashboards, alert routes or SLO is verified."
],
"risks": [
"Failures and queue backlog can remain unnoticed."
],
"recommendation": "Integrate external monitoring before small production.",
"proof_check": "Trigger each readiness/delivery failure and verify an actionable alert."
},
{
"id": "operations.backup_restore",
"requirement": "Back up and restore database, files, configuration and keys as a coherent service.",
"status": "partial",
"evidence": [
{
"kind": "documentation",
"scope": "documented_model",
"locator": "govoplan-core/docs/DEPLOYMENT_OPERATOR_GUIDE.md"
},
{
"kind": "issue",
"scope": "documented_model",
"locator": "https://git.add-ideas.de/add-ideas/govoplan-core/issues/29"
}
],
"conditions": [],
"gaps": [
"No target full-service restore drill or measured RPO/RTO exists."
],
"risks": [
"Partial restore can produce missing files, unusable secrets or inconsistent evidence."
],
"recommendation": "Treat Core #29 and a target restore drill as a production gate.",
"proof_check": "Restore the whole service into an isolated environment and measure it."
},
{
"id": "operations.disaster_recovery",
"requirement": "Recover the service after site or dependency loss within agreed RPO/RTO.",
"status": "not_assessed",
"evidence": [
{
"kind": "absence",
"scope": "current_workspace",
"locator": "No target DR plan or exercise evidence supplied"
}
],
"conditions": [],
"gaps": [
"RPO/RTO, off-site copies, recovery order, failover, communications and exercise schedule are unknown."
],
"risks": [
"Service and evidence may be unrecoverable after a major incident."
],
"recommendation": "Define and exercise DR before any availability commitment.",
"proof_check": "Run a documented end-to-end recovery exercise."
}
],
"data_flows": [
{
"id": "browser.api",
"from": "User browser",
"to": "Reverse proxy and GovOPlaN WebUI/API",
"data": [
"Session and CSRF cookies",
"Campaign content",
"Recipient personal data",
"Managed files"
],
"trust_boundary": "Client/public to application",
"controls": [
"HTTPS",
"Exact CORS origins",
"Secure cookies",
"Tenant and RBAC enforcement",
"Request limits"
]
},
{
"id": "api.database",
"from": "GovOPlaN API and workers",
"to": "PostgreSQL",
"data": [
"Tenant and identity records",
"Campaign drafts, snapshots and jobs",
"Connector metadata",
"Audit evidence"
],
"trust_boundary": "Application to primary state store",
"controls": [
"Dedicated database identity",
"Private or encrypted transport",
"Migrations",
"Backup and retention"
]
},
{
"id": "api.queue.worker",
"from": "GovOPlaN API",
"to": "Redis and Celery worker",
"data": [
"Job identifiers",
"Queue routing and retry metadata"
],
"trust_boundary": "Request plane to asynchronous processing plane",
"controls": [
"Private authenticated broker",
"Bounded payloads",
"Idempotent claims",
"Queue monitoring"
]
},
{
"id": "worker.mail",
"from": "GovOPlaN Campaign worker",
"to": "External SMTP and IMAP services",
"data": [
"Recipient addresses",
"Message bodies",
"Attachments",
"Sent-message copy"
],
"trust_boundary": "GovOPlaN to external communication provider",
"controls": [
"Scoped service account",
"TLS and CA policy",
"Sender and recipient policy",
"Rate limits",
"Outcome reconciliation"
]
},
{
"id": "worker.connectors",
"from": "GovOPlaN connector worker",
"to": "External address, file, object or calendar service",
"data": [
"Addresses",
"Files and provenance",
"Calendar resources"
],
"trust_boundary": "GovOPlaN to organizational/external content systems",
"controls": [
"Explicit sync direction",
"Scoped credentials",
"Endpoint allow-list",
"Provenance",
"Conflict and reconciliation policy"
]
}
],
"assumptions": [
"The pilot can use local accounts and one internal tenant or office.",
"A dedicated non-production SMTP/IMAP account and safe recipients are available.",
"Pilot load fits one API and one worker until measured otherwise.",
"Durable local storage is acceptable for the pilot."
],
"open_questions": [
"What are the target organization's data classes, legal bases, retention and external-disclosure rules?",
"Which identity, mail, file, address and monitoring systems are mandatory?",
"What are Campaign volume, concurrency, growth, availability, RPO and RTO?",
"Who owns each external runtime component and operational control?",
"Which accessibility, security, support and procurement constraints are mandatory?"
],
"risks": [
{
"id": "risk.reproducibility",
"statement": "The signed package selection is reproducible but has not been accepted as an installed target composition.",
"impact": "Installation or configuration drift can still produce uncertain deployed behavior.",
"treatment": "Materialize the signed catalog in an isolated target and run installed-artifact acceptance gates.",
"owner": null,
"residual_risk": "Module and environment differences still require release-environment verification."
},
{
"id": "risk.delivery_provider",
"statement": "Target SMTP/IMAP behavior and failure modes are unproved.",
"impact": "Failed, delayed or duplicate communication and incomplete evidence.",
"treatment": "Run target-like interoperability, throttling and uncertainty drills.",
"owner": null,
"residual_risk": "External provider outages and ambiguous outcomes remain operational risks."
},
{
"id": "risk.recovery",
"statement": "Backup/restore and disaster recovery are not demonstrated across all state and keys.",
"impact": "Irrecoverable or inconsistent service after loss.",
"treatment": "Complete Core #29 and an isolated full-service restore/DR exercise.",
"owner": null,
"residual_risk": "Recovery time and data loss remain bounded by the selected external infrastructure."
}
],
"recommendations": [
"Proceed only with a controlled internal Campaign pilot after the bounded proof checks pass.",
"Use the minimal composition and enable Addresses only for an explicit reusable-recipient journey.",
"Do not claim Workflow, export-control screening, identity federation or production DR as implemented.",
"Treat installed-release acceptance, target mail proof, monitoring and a coherent restore drill as production gates."
],
"proof_checks": [
"Materialize the signed catalog into an isolated installation and rerun contract, migration and module-permutation gates against the installed artifacts.",
"Collect the isolated installation with the bounded installed-composition evidence contract; require exact enabled package/module versions, complete RECORD verification and immutable provenance anchored to this assessment.",
"Run a safe target-like Campaign through SMTP acceptance, IMAP append, reporting and audit.",
"Drill worker, Redis and ambiguous-delivery failures without duplicate sends.",
"Restore PostgreSQL, managed files, configuration and encrypted credentials and measure RPO/RTO.",
"Validate proxy/TLS, cookies/CORS, account bootstrap, secret redaction, monitoring and alert delivery.",
"Measure representative Campaign/file/queue/database load and external throttling.",
"Require separately issued, expiring and independently scope-authorized evidence before marking target environment, external provider or production approval proof as checked."
]
}

View File

@@ -0,0 +1,75 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit-proof-authority-keyring.schema.json",
"title": "GovOPlaN capability-fit proof authority keyring",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "purpose", "keys"],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.1.0"
},
"purpose": {
"const": "govoplan.capability-fit-proof-authorities"
},
"keys": {
"type": "array",
"maxItems": 64,
"items": {
"$ref": "#/$defs/key"
}
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"key": {
"type": "object",
"additionalProperties": false,
"required": ["key_id", "status", "public_key", "allowed_scopes"],
"properties": {
"key_id": {
"$ref": "#/$defs/opaque_id"
},
"status": {
"enum": ["active", "next", "revoked", "disabled", "retired"]
},
"public_key": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
},
"allowed_scopes": {
"type": "array",
"minItems": 1,
"maxItems": 3,
"uniqueItems": true,
"items": {
"enum": [
"target_environment",
"external_providers",
"production_approval"
]
}
},
"not_before": {
"type": "string",
"format": "date-time"
},
"not_after": {
"type": "string",
"format": "date-time"
}
}
}
}
}

View File

@@ -0,0 +1,298 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit.schema.json",
"title": "GovOPlaN capability and infrastructure fit assessment",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"assessment_id",
"assessed_at",
"scope",
"release",
"composition",
"deployment_profile",
"questionnaire",
"capabilities",
"infrastructure",
"data_flows",
"assumptions",
"open_questions",
"risks",
"recommendations",
"proof_checks"
],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.1.0"
},
"assessment_id": {
"$ref": "#/$defs/non_empty_string"
},
"assessed_at": {
"type": "string",
"format": "date"
},
"scope": {
"type": "object",
"additionalProperties": false,
"required": ["title", "reference_journeys", "postponed"],
"properties": {
"title": { "$ref": "#/$defs/non_empty_string" },
"reference_journeys": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/non_empty_string" }
},
"postponed": {
"type": "array",
"items": { "$ref": "#/$defs/non_empty_string" }
}
}
},
"release": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "ref", "meta_commit", "reproducible", "configuration_packages"],
"properties": {
"kind": {
"enum": ["tagged_release", "release_candidate", "workspace_snapshot"]
},
"ref": { "$ref": "#/$defs/non_empty_string" },
"meta_commit": { "$ref": "#/$defs/non_empty_string" },
"reproducible": { "type": "boolean" },
"configuration_packages": {
"type": "array",
"items": { "$ref": "#/$defs/non_empty_string" }
},
"notes": {
"type": "array",
"items": { "$ref": "#/$defs/non_empty_string" }
}
}
},
"composition": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/module" }
},
"deployment_profile": {
"type": "object",
"additionalProperties": false,
"required": ["id", "status", "description", "evidence"],
"properties": {
"id": { "$ref": "#/$defs/non_empty_string" },
"status": { "$ref": "#/$defs/status" },
"description": { "$ref": "#/$defs/non_empty_string" },
"evidence": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/evidence" }
}
}
},
"questionnaire": {
"type": "object",
"additionalProperties": false,
"required": [
"scope_outcomes",
"data_policy",
"identity_integrations",
"workload_growth",
"availability_operations",
"procurement_decisions"
],
"properties": {
"scope_outcomes": { "$ref": "#/$defs/answers" },
"data_policy": { "$ref": "#/$defs/answers" },
"identity_integrations": { "$ref": "#/$defs/answers" },
"workload_growth": { "$ref": "#/$defs/answers" },
"availability_operations": { "$ref": "#/$defs/answers" },
"procurement_decisions": { "$ref": "#/$defs/answers" }
}
},
"capabilities": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/assessed_item" }
},
"infrastructure": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/assessed_item" }
},
"data_flows": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/data_flow" }
},
"assumptions": { "$ref": "#/$defs/string_list" },
"open_questions": { "$ref": "#/$defs/string_list" },
"risks": {
"type": "array",
"items": { "$ref": "#/$defs/risk" }
},
"recommendations": { "$ref": "#/$defs/string_list" },
"proof_checks": { "$ref": "#/$defs/string_list" }
},
"$defs": {
"non_empty_string": {
"type": "string",
"minLength": 1
},
"string_list": {
"type": "array",
"items": { "$ref": "#/$defs/non_empty_string" }
},
"status": {
"enum": [
"verified",
"available_unconfigured",
"partial",
"scaffold",
"external_system",
"planned",
"not_fit",
"not_assessed"
]
},
"evidence": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "scope", "locator"],
"properties": {
"kind": {
"enum": [
"test",
"route",
"contract",
"drill",
"documentation",
"configuration",
"issue",
"absence",
"observation"
]
},
"scope": {
"enum": [
"committed_source",
"current_workspace",
"documented_model",
"target_environment"
]
},
"locator": { "$ref": "#/$defs/non_empty_string" },
"note": { "type": "string" }
}
},
"module": {
"type": "object",
"additionalProperties": false,
"required": [
"module_id",
"repository",
"commit",
"manifest_version",
"enabled",
"dirty",
"role"
],
"properties": {
"module_id": { "$ref": "#/$defs/non_empty_string" },
"repository": { "$ref": "#/$defs/non_empty_string" },
"commit": {
"type": "string",
"pattern": "^[0-9a-f]{7,40}$"
},
"manifest_version": { "$ref": "#/$defs/non_empty_string" },
"enabled": { "type": "boolean" },
"dirty": { "type": "boolean" },
"role": { "$ref": "#/$defs/non_empty_string" }
}
},
"answers": {
"type": "array",
"items": { "$ref": "#/$defs/answer" }
},
"answer": {
"type": "object",
"additionalProperties": false,
"required": ["id", "question", "state", "answer", "evidence"],
"properties": {
"id": { "$ref": "#/$defs/non_empty_string" },
"question": { "$ref": "#/$defs/non_empty_string" },
"state": {
"enum": ["answered", "assumed", "not_applicable", "not_assessed"]
},
"answer": {
"type": ["string", "array", "null"],
"items": { "type": "string" }
},
"evidence": {
"type": "array",
"items": { "$ref": "#/$defs/evidence" }
}
}
},
"assessed_item": {
"type": "object",
"additionalProperties": false,
"required": [
"id",
"requirement",
"status",
"evidence",
"conditions",
"gaps",
"risks",
"recommendation",
"proof_check"
],
"properties": {
"id": { "$ref": "#/$defs/non_empty_string" },
"requirement": { "$ref": "#/$defs/non_empty_string" },
"status": { "$ref": "#/$defs/status" },
"evidence": {
"type": "array",
"items": { "$ref": "#/$defs/evidence" }
},
"conditions": { "$ref": "#/$defs/string_list" },
"gaps": { "$ref": "#/$defs/string_list" },
"risks": { "$ref": "#/$defs/string_list" },
"recommendation": { "type": "string" },
"proof_check": { "type": "string" }
}
},
"data_flow": {
"type": "object",
"additionalProperties": false,
"required": ["id", "from", "to", "data", "trust_boundary", "controls"],
"properties": {
"id": { "$ref": "#/$defs/non_empty_string" },
"from": { "$ref": "#/$defs/non_empty_string" },
"to": { "$ref": "#/$defs/non_empty_string" },
"data": { "$ref": "#/$defs/string_list" },
"trust_boundary": { "$ref": "#/$defs/non_empty_string" },
"controls": { "$ref": "#/$defs/string_list" }
}
},
"risk": {
"type": "object",
"additionalProperties": false,
"required": ["id", "statement", "impact", "treatment", "owner", "residual_risk"],
"properties": {
"id": { "$ref": "#/$defs/non_empty_string" },
"statement": { "$ref": "#/$defs/non_empty_string" },
"impact": { "$ref": "#/$defs/non_empty_string" },
"treatment": { "$ref": "#/$defs/non_empty_string" },
"owner": { "type": ["string", "null"] },
"residual_risk": { "$ref": "#/$defs/non_empty_string" }
}
}
}
}

View File

@@ -11,6 +11,12 @@
"description": "New user-visible behavior or platform capability.", "description": "New user-visible behavior or platform capability.",
"exclusive": true "exclusive": true
}, },
{
"name": "type/user-story",
"color": "1d76db",
"description": "End-to-end user journey or real-world process story used to steer product slices.",
"exclusive": true
},
{ {
"name": "type/task", "name": "type/task",
"color": "1d76db", "color": "1d76db",
@@ -149,12 +155,24 @@
"description": "GovOPlaN Dms module behavior or integration.", "description": "GovOPlaN Dms module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/dist-lists",
"color": "0e8a16",
"description": "GovOPlaN Distribution Lists module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/erp", "name": "module/erp",
"color": "fef2c0", "color": "fef2c0",
"description": "GovOPlaN Erp module behavior or integration.", "description": "GovOPlaN Erp module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/evaluation",
"color": "bfdadc",
"description": "GovOPlaN Evaluation module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/files", "name": "module/files",
"color": "006b75", "color": "006b75",
@@ -227,12 +245,24 @@
"description": "GovOPlaN Payments module behavior or integration.", "description": "GovOPlaN Payments module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/permits",
"color": "fbca04",
"description": "GovOPlaN Permits module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/policy", "name": "module/policy",
"color": "d93f0b", "color": "d93f0b",
"description": "GovOPlaN Policy module behavior or integration.", "description": "GovOPlaN Policy module behavior or integration.",
"exclusive": false "exclusive": false
}, },
{
"name": "module/poll",
"color": "e4e669",
"description": "GovOPlaN Poll module behavior or integration.",
"exclusive": false
},
{ {
"name": "module/portal", "name": "module/portal",
"color": "1d76db", "color": "1d76db",

View File

@@ -0,0 +1,335 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/installed-composition-evidence.schema.json",
"title": "GovOPlaN installed composition evidence",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"assessment_id",
"assessment_release",
"collected_at",
"scope",
"artifacts",
"collection_issues"
],
"properties": {
"$schema": {
"type": "string",
"format": "uri-reference"
},
"schema_version": {
"const": "0.4.0"
},
"evidence_kind": {
"const": "govoplan.installed-composition"
},
"assessment_id": {
"$ref": "#/$defs/opaque_id"
},
"assessment_release": {
"$ref": "#/$defs/opaque_id"
},
"collected_at": {
"type": "string",
"format": "date-time"
},
"scope": {
"const": "current-python-environment.govoplan-distributions"
},
"artifacts": {
"type": "array",
"maxItems": 256,
"items": {
"$ref": "#/$defs/artifact"
}
},
"collection_issues": {
"type": "array",
"maxItems": 256,
"items": {
"$ref": "#/$defs/collection_issue"
}
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"package_name": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
},
"version": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._+!-]*$"
},
"artifact": {
"type": "object",
"additionalProperties": false,
"required": [
"package_name",
"package_version",
"modules",
"source_provenance",
"record_integrity"
],
"properties": {
"package_name": {
"$ref": "#/$defs/package_name"
},
"package_version": {
"$ref": "#/$defs/version"
},
"modules": {
"type": "array",
"maxItems": 16,
"items": {
"$ref": "#/$defs/module"
}
},
"source_provenance": {
"$ref": "#/$defs/source_provenance"
},
"record_integrity": {
"$ref": "#/$defs/record_integrity"
}
}
},
"module": {
"type": "object",
"additionalProperties": false,
"required": ["module_id", "manifest_version"],
"properties": {
"module_id": {
"$ref": "#/$defs/opaque_id"
},
"manifest_version": {
"$ref": "#/$defs/version"
}
}
},
"source_provenance": {
"oneOf": [
{
"type": "object",
"additionalProperties": false,
"required": ["basis", "kind", "commit"],
"properties": {
"basis": {
"const": "local-pep610-metadata"
},
"kind": {
"const": "vcs-commit"
},
"commit": {
"type": "string",
"pattern": "^[0-9a-f]{40,64}$"
}
}
},
{
"type": "object",
"additionalProperties": false,
"required": ["basis", "kind", "sha256"],
"properties": {
"basis": {
"const": "local-pep610-metadata"
},
"kind": {
"const": "archive"
},
"sha256": {
"$ref": "#/$defs/sha256"
}
}
},
{
"type": "object",
"additionalProperties": false,
"required": ["basis", "kind"],
"properties": {
"basis": {
"const": "local-pep610-metadata"
},
"kind": {
"enum": [
"editable-local",
"local-directory",
"index-or-unknown",
"malformed-direct-url"
]
}
}
}
]
},
"record_integrity": {
"type": "object",
"additionalProperties": false,
"required": [
"status",
"hashed_file_count",
"permitted_unhashed_file_count",
"generated_unhashed_file_count",
"unverifiable_file_count",
"missing_file_count",
"mismatched_file_count",
"artifact_payload_identity",
"installed_payload_identity"
],
"properties": {
"status": {
"enum": [
"verified",
"partial",
"mismatch",
"unavailable",
"limit-exceeded"
]
},
"hashed_file_count": {
"$ref": "#/$defs/count"
},
"permitted_unhashed_file_count": {
"$ref": "#/$defs/count"
},
"generated_unhashed_file_count": {
"$ref": "#/$defs/count"
},
"unverifiable_file_count": {
"$ref": "#/$defs/count"
},
"missing_file_count": {
"$ref": "#/$defs/count"
},
"mismatched_file_count": {
"$ref": "#/$defs/count"
},
"artifact_payload_identity": {
"oneOf": [
{ "$ref": "#/$defs/artifact_payload_identity" },
{ "type": "null" }
]
},
"installed_payload_identity": {
"oneOf": [
{ "$ref": "#/$defs/installed_payload_identity" },
{ "type": "null" }
]
}
},
"allOf": [
{
"if": { "properties": { "status": { "const": "verified" } } },
"then": {
"properties": {
"hashed_file_count": { "minimum": 1 },
"permitted_unhashed_file_count": { "minimum": 1 },
"unverifiable_file_count": { "const": 0 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 },
"artifact_payload_identity": { "$ref": "#/$defs/artifact_payload_identity" },
"installed_payload_identity": { "$ref": "#/$defs/installed_payload_identity" }
}
}
},
{
"if": { "properties": { "status": { "const": "partial" } } },
"then": {
"properties": {
"hashed_file_count": { "minimum": 1 },
"unverifiable_file_count": { "minimum": 1 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 }
}
}
},
{
"if": { "properties": { "status": { "const": "mismatch" } } },
"then": {
"anyOf": [
{ "properties": { "missing_file_count": { "minimum": 1 } } },
{ "properties": { "mismatched_file_count": { "minimum": 1 } } }
]
}
},
{
"if": { "properties": { "status": { "const": "unavailable" } } },
"then": {
"properties": {
"hashed_file_count": { "const": 0 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 }
}
}
}
]
},
"collection_issue": {
"type": "object",
"additionalProperties": false,
"required": ["code", "package_name"],
"properties": {
"code": {
"enum": [
"distribution-metadata-invalid",
"duplicate-distribution",
"module-entry-point-load-failed",
"module-entry-point-invalid",
"module-entry-point-limit-exceeded",
"collection-limit-exceeded"
]
},
"package_name": {
"$ref": "#/$defs/package_name"
}
}
},
"artifact_payload_identity": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-wheel-declared-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
},
"installed_payload_identity": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-installed-record-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
},
"count": {
"type": "integer",
"minimum": 0,
"maximum": 1000000
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}

View File

@@ -0,0 +1,53 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/installer-receipt-authority-keyring.schema.json",
"title": "GovOPlaN installer receipt authority keyring",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "purpose", "keys"],
"properties": {
"$schema": { "type": "string", "format": "uri-reference" },
"schema_version": { "const": "0.1.0" },
"purpose": { "const": "govoplan.installer-receipt-authorities" },
"keys": {
"type": "array",
"minItems": 1,
"maxItems": 64,
"items": { "$ref": "#/$defs/key" }
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"key": {
"type": "object",
"additionalProperties": false,
"required": ["key_id", "status", "public_key", "allowed_scopes"],
"properties": {
"key_id": { "$ref": "#/$defs/opaque_id" },
"status": {
"enum": ["active", "next", "revoked", "disabled", "retired"]
},
"public_key": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
},
"allowed_scopes": {
"type": "array",
"minItems": 1,
"maxItems": 1,
"uniqueItems": true,
"items": { "const": "installed_release_origin" }
},
"not_before": { "type": "string", "format": "date-time" },
"not_after": { "type": "string", "format": "date-time" }
}
}
}
}

View File

@@ -0,0 +1,122 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/installer-receipt.schema.json",
"title": "GovOPlaN signed installer receipt",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"receipt_id",
"assessment_id",
"assessment_release",
"installed_evidence_sha256",
"catalog",
"issued_at",
"artifacts",
"signatures"
],
"properties": {
"$schema": { "type": "string", "format": "uri-reference" },
"schema_version": { "const": "0.1.0" },
"evidence_kind": { "const": "govoplan.installer-receipt" },
"receipt_id": { "$ref": "#/$defs/opaque_id" },
"assessment_id": { "$ref": "#/$defs/opaque_id" },
"assessment_release": { "$ref": "#/$defs/opaque_id" },
"installed_evidence_sha256": { "$ref": "#/$defs/sha256" },
"catalog": {
"type": "object",
"additionalProperties": false,
"required": ["channel", "sequence", "sha256"],
"properties": {
"channel": {
"type": "string",
"pattern": "^[a-z][a-z0-9_-]{0,63}$"
},
"sequence": { "type": "integer", "minimum": 1 },
"sha256": { "$ref": "#/$defs/sha256" }
}
},
"issued_at": { "type": "string", "format": "date-time" },
"artifacts": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"items": { "$ref": "#/$defs/artifact" }
},
"signatures": {
"type": "array",
"minItems": 1,
"maxItems": 16,
"items": { "$ref": "#/$defs/signature" }
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"package_name": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
},
"version": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._+!-]*$"
},
"artifact": {
"type": "object",
"additionalProperties": false,
"required": [
"package_name",
"package_version",
"catalog_archive_sha256",
"installed_payload"
],
"properties": {
"package_name": { "$ref": "#/$defs/package_name" },
"package_version": { "$ref": "#/$defs/version" },
"catalog_archive_sha256": { "$ref": "#/$defs/sha256" },
"installed_payload": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-installed-record-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
}
}
},
"signature": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "key_id", "value"],
"properties": {
"algorithm": { "const": "ed25519" },
"key_id": { "$ref": "#/$defs/opaque_id" },
"value": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
}
}
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}

View File

@@ -23,8 +23,10 @@
{"name": "govoplan-contracts", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-contracts.git", "path": "govoplan-contracts"}, {"name": "govoplan-contracts", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-contracts.git", "path": "govoplan-contracts"},
{"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dashboard.git", "path": "govoplan-dashboard"}, {"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dashboard.git", "path": "govoplan-dashboard"},
{"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dms.git", "path": "govoplan-dms"}, {"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dms.git", "path": "govoplan-dms"},
{"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dist-lists.git", "path": "govoplan-dist-lists"},
{"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-docs.git", "path": "govoplan-docs"}, {"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-docs.git", "path": "govoplan-docs"},
{"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-erp.git", "path": "govoplan-erp"}, {"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-erp.git", "path": "govoplan-erp"},
{"name": "govoplan-evaluation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-evaluation.git", "path": "govoplan-evaluation"},
{"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-facilities.git", "path": "govoplan-facilities"}, {"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-facilities.git", "path": "govoplan-facilities"},
{"name": "govoplan-files", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-files.git", "path": "govoplan-files"}, {"name": "govoplan-files", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-files.git", "path": "govoplan-files"},
{"name": "govoplan-fit-connect", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-fit-connect.git", "path": "govoplan-fit-connect"}, {"name": "govoplan-fit-connect", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-fit-connect.git", "path": "govoplan-fit-connect"},
@@ -46,6 +48,7 @@
{"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-payments.git", "path": "govoplan-payments"}, {"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-payments.git", "path": "govoplan-payments"},
{"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-permits.git", "path": "govoplan-permits"}, {"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-permits.git", "path": "govoplan-permits"},
{"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-policy.git", "path": "govoplan-policy"}, {"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-policy.git", "path": "govoplan-policy"},
{"name": "govoplan-poll", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-poll.git", "path": "govoplan-poll"},
{"name": "govoplan-portal", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-portal.git", "path": "govoplan-portal"}, {"name": "govoplan-portal", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-portal.git", "path": "govoplan-portal"},
{"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-postbox.git", "path": "govoplan-postbox"}, {"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-postbox.git", "path": "govoplan-postbox"},
{"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-procurement.git", "path": "govoplan-procurement"}, {"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-procurement.git", "path": "govoplan-procurement"},
@@ -61,7 +64,7 @@
{"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-templates.git", "path": "govoplan-templates"}, {"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-templates.git", "path": "govoplan-templates"},
{"name": "govoplan-tenancy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-tenancy.git", "path": "govoplan-tenancy"}, {"name": "govoplan-tenancy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-tenancy.git", "path": "govoplan-tenancy"},
{"name": "govoplan-transparency", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-transparency.git", "path": "govoplan-transparency"}, {"name": "govoplan-transparency", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-transparency.git", "path": "govoplan-transparency"},
{"name": "addideas-govoplan-website", "category": "website", "subtype": "public-site", "remote": "git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git", "path": "addideas-govoplan-website"}, {"name": "addideas-govoplan-website", "category": "website", "subtype": "public-site", "remote": "git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git", "path": "addideas-govoplan-website", "bootstrap_transport": "registered"},
{"name": "govoplan-workflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-workflow.git", "path": "govoplan-workflow"}, {"name": "govoplan-workflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-workflow.git", "path": "govoplan-workflow"},
{"name": "govoplan-xoev", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xoev.git", "path": "govoplan-xoev"}, {"name": "govoplan-xoev", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xoev.git", "path": "govoplan-xoev"},
{"name": "govoplan-xrechnung", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xrechnung.git", "path": "govoplan-xrechnung"}, {"name": "govoplan-xrechnung", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xrechnung.git", "path": "govoplan-xrechnung"},

View File

@@ -1,4 +1,5 @@
bandit>=1.8,<2 bandit>=1.8,<2
click>=8.3.3
filelock>=3.20.3 filelock>=3.20.3
idna>=3.15 idna>=3.15
pip>=26.1.2 pip>=26.1.2
@@ -6,5 +7,4 @@ pip-audit>=2.9,<3
python-multipart>=0.0.31 python-multipart>=0.0.31
radon>=6,<7 radon>=6,<7
ruff>=0.14,<1 ruff>=0.14,<1
semgrep>=1.140,<2
xenon>=0.9,<1 xenon>=0.9,<1

View File

@@ -7,21 +7,30 @@
-e ../govoplan-tenancy -e ../govoplan-tenancy
-e ../govoplan-organizations -e ../govoplan-organizations
-e ../govoplan-identity -e ../govoplan-identity
-e ../govoplan-idm
-e ../govoplan-access -e ../govoplan-access
-e ../govoplan-admin -e ../govoplan-admin
-e ../govoplan-policy -e ../govoplan-policy
-e ../govoplan-audit -e ../govoplan-audit
-e ../govoplan-dashboard -e ../govoplan-dashboard
-e ../govoplan-addresses
-e ../govoplan-dist-lists
-e ../govoplan-files -e ../govoplan-files
-e ../govoplan-mail -e ../govoplan-mail
-e ../govoplan-campaign -e ../govoplan-campaign
-e ../govoplan-calendar -e ../govoplan-calendar
-e ../govoplan-poll
-e ../govoplan-scheduling
-e ../govoplan-notifications
-e ../govoplan-evaluation
-e ../govoplan-docs -e ../govoplan-docs
-e ../govoplan-ops -e ../govoplan-ops
httpx==0.28.1 httpx==0.28.1
httpx2>=2.5,<3 httpx2>=2.5,<3
filelock>=3.20.3 filelock>=3.20.3
idna>=3.15 idna>=3.15
jsonschema>=4,<5
pip>=26.1.2 pip>=26.1.2
pip-audit>=2.9,<3 pip-audit>=2.9,<3
python-multipart>=0.0.31 python-multipart>=0.0.31
ruff>=0.14,<1

View File

@@ -1,19 +1,18 @@
# Whole-product release install from tagged module repositories. # Whole-product release install from immutable, independently versioned module tags.
# Update GOVOPLAN_RELEASE_TAG together with pyproject/package versions when # Only add a module after its referenced tag has been published.
# cutting a release.
../govoplan-core[server] ../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-tenancy.git@v0.1.6 govoplan-tenancy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-organizations.git@v0.1.6 govoplan-organizations @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-identity.git@v0.1.6 govoplan-identity @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-idm.git@v0.1.6 govoplan-idm @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-access.git@v0.1.6 govoplan-access @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-admin.git@v0.1.6 govoplan-admin @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-policy.git@v0.1.6 govoplan-policy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-audit.git@v0.1.6 govoplan-audit @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git@v0.1.6 govoplan-dashboard @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.6 govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.8
govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.6 govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.10
govoplan-campaign @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-campaign.git@v0.1.6 govoplan-campaign @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-campaign.git@v0.1.11
govoplan-calendar @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-calendar.git@v0.1.6 govoplan-calendar @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-calendar.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-docs.git@v0.1.6 govoplan-docs @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-ops.git@v0.1.6 govoplan-ops @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-ops.git@v0.1.8

View File

@@ -0,0 +1,190 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import stat
import sys
import tempfile
import unittest
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
for tools_root in (ASSESSMENT_TOOLS_ROOT, RELEASE_TOOLS_ROOT):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
class CapabilityFitAtomicIOTests(unittest.TestCase):
def test_writes_private_json_with_expected_content(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
target = Path(temporary_directory) / "evidence.json"
payload = {"z": [1, 2], "message": "Grüße"}
created_in: list[Path] = []
original_mkstemp = tempfile.mkstemp
def observed_mkstemp(*args, **kwargs):
created_in.append(Path(kwargs["dir"]))
return original_mkstemp(*args, **kwargs)
with (
mock.patch(
"govoplan_assessment.atomic_io.tempfile.mkstemp",
side_effect=observed_mkstemp,
),
mock.patch(
"govoplan_assessment.atomic_io.os.fsync",
wraps=os.fsync,
) as fsync,
):
atomic_write_json(target, payload, max_bytes=1024)
self.assertEqual(payload, json.loads(target.read_text(encoding="utf-8")))
self.assertTrue(target.read_bytes().endswith(b"\n"))
self.assertEqual(0o600, stat.S_IMODE(target.stat().st_mode))
self.assertEqual([target.parent], created_in)
self.assertEqual(2, fsync.call_count)
def test_replaces_existing_regular_file_and_secures_mode(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
target = Path(temporary_directory) / "report.json"
target.write_text('{"old": true}\n', encoding="utf-8")
target.chmod(0o644)
old_handle = target.open("rb")
self.addCleanup(old_handle.close)
atomic_write_json(target, {"new": True}, max_bytes=1024)
self.assertEqual(b'{"old": true}\n', old_handle.read())
self.assertEqual({"new": True}, json.loads(target.read_text("utf-8")))
self.assertEqual(0o600, stat.S_IMODE(target.stat().st_mode))
def test_size_bound_leaves_existing_target_unchanged(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
original = b'{"original": true}\n'
target.write_bytes(original)
with self.assertRaisesRegex(AtomicJsonWriteError, "size limit"):
atomic_write_json(target, {"large": "x" * 100}, max_bytes=32)
self.assertEqual(original, target.read_bytes())
self.assertEqual([], list(directory.glob(".govoplan-json-*.tmp")))
@unittest.skipUnless(hasattr(os, "symlink"), "symbolic links are unavailable")
def test_rejects_symlink_without_modifying_link_or_referent(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
referent = directory / "outside.json"
referent.write_bytes(b'{"keep": true}\n')
target = directory / "evidence.json"
target.symlink_to(referent.name)
with self.assertRaisesRegex(AtomicJsonWriteError, "symbolic link"):
atomic_write_json(target, {"replace": True}, max_bytes=1024)
self.assertTrue(target.is_symlink())
self.assertEqual(b'{"keep": true}\n', referent.read_bytes())
self.assertEqual([], list(directory.glob(".govoplan-json-*.tmp")))
@unittest.skipUnless(hasattr(os, "symlink"), "symbolic links are unavailable")
def test_rejects_symlinked_parent_component(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
real_parent = directory / "real-parent"
real_parent.mkdir()
linked_parent = directory / "linked-parent"
linked_parent.symlink_to(real_parent, target_is_directory=True)
target = linked_parent / "new" / "evidence.json"
with self.assertRaisesRegex(AtomicJsonWriteError, "parent path"):
atomic_write_json(target, {"unsafe": True}, max_bytes=1024)
self.assertFalse((real_parent / "new").exists())
def test_requires_existing_parent_without_creating_directories(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
missing_parent = directory / "missing" / "nested"
target = missing_parent / "evidence.json"
with self.assertRaisesRegex(AtomicJsonWriteError, "already exist"):
atomic_write_json(target, {"safe": True}, max_bytes=1024)
self.assertFalse(missing_parent.exists())
def test_rejects_post_replace_mode_or_identity_change(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
original_replace = os.replace
def insecure_replace(source, destination):
original_replace(source, destination)
Path(destination).chmod(0o644)
with mock.patch(
"govoplan_assessment.atomic_io.os.replace",
side_effect=insecure_replace,
):
with self.assertRaisesRegex(
AtomicJsonWriteError,
"permissions changed and were restored",
):
atomic_write_json(target, {"safe": True}, max_bytes=1024)
self.assertEqual(0o600, stat.S_IMODE(target.stat().st_mode))
self.assertEqual({"safe": True}, json.loads(target.read_text("utf-8")))
def test_does_not_unlink_unknown_post_replace_inode(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
original_replace = os.replace
def replaced_again(source, destination):
original_replace(source, destination)
Path(destination).unlink()
Path(destination).write_bytes(b"unknown replacement\n")
with mock.patch(
"govoplan_assessment.atomic_io.os.replace",
side_effect=replaced_again,
):
with self.assertRaisesRegex(
AtomicJsonWriteError,
"did not preserve",
):
atomic_write_json(target, {"secret": True}, max_bytes=1024)
self.assertEqual(b"unknown replacement\n", target.read_bytes())
def test_replace_failure_removes_private_temporary_file(self) -> None:
with tempfile.TemporaryDirectory() as temporary_directory:
directory = Path(temporary_directory)
target = directory / "evidence.json"
with mock.patch(
"govoplan_assessment.atomic_io.os.replace",
side_effect=OSError("simulated replacement failure"),
):
with self.assertRaisesRegex(
AtomicJsonWriteError, "could not be written atomically"
):
atomic_write_json(target, {"safe": True}, max_bytes=1024)
self.assertFalse(target.exists())
self.assertEqual([], list(directory.glob(".govoplan-json-*.tmp")))
if __name__ == "__main__":
unittest.main()

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,700 @@
from __future__ import annotations
import base64
from copy import deepcopy
from datetime import UTC, datetime, timedelta
import hashlib
import importlib.util
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
from unittest import mock
import unittest
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
for tools_root in (ASSESSMENT_TOOLS_ROOT, RELEASE_TOOLS_ROOT):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from govoplan_assessment.capability_fit import ( # noqa: E402
local_tag_provenance,
render_review,
review_capability_fit,
trusted_keys_from_keyring,
)
class CapabilityFitReviewTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.assessment = json.loads(
(META_ROOT / "docs" / "capability-fit-current.json").read_text(
encoding="utf-8"
)
)
cls.schema = json.loads(
(META_ROOT / "docs" / "capability-fit.schema.json").read_text(
encoding="utf-8"
)
)
def test_matching_signed_catalog_is_current(self) -> None:
catalog, keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("current", report["status"])
self.assertEqual([], report["changes"])
self.assertFalse(report["proof_scope"]["installed_artifacts"]["checked"])
self.assertFalse(report["proof_scope"]["target_environment"]["checked"])
self.assertFalse(report["proof_scope"]["local_tag_provenance"]["checked"])
repeated = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual(
json.dumps(report, sort_keys=True), json.dumps(repeated, sort_keys=True)
)
def test_release_drift_identifies_affected_conclusions(self) -> None:
catalog, keyring = signed_catalog(
self.assessment, versions={"campaigns": "0.1.11"}, sequence=202607230001
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("review_required", report["status"])
target_ids = {item["id"] for item in report["review_targets"]}
self.assertIn("campaign.journey", target_ids)
self.assertIn("composition.campaigns", target_ids)
self.assertIn("assessment.release", target_ids)
self.assertIn(
"composition_version_changed", {item["code"] for item in report["findings"]}
)
def test_contradictory_signed_selected_unit_metadata_blocks_rerun(self) -> None:
core = next(
item
for item in self.assessment["composition"]
if item["module_id"] == "core"
)
catalog, keyring = signed_catalog(
self.assessment,
selected_units=[
{
"repo": core["repository"],
"version": core["manifest_version"],
"tag": "v999.0.0",
"commit_sha": "0" * 40,
"tag_object_sha": "1" * 40,
}
],
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("blocked", report["status"])
codes = {item["code"] for item in report["findings"]}
self.assertIn("catalog_release_metadata", codes)
self.assertIn("composition_commit_changed", codes)
self.assertTrue(report["proof_scope"]["catalog_signature_and_keyring"]["valid"])
self.assertFalse(report["proof_scope"]["release_metadata"]["valid"])
def test_signed_selected_unit_commit_drift_requires_review(self) -> None:
core = next(
item
for item in self.assessment["composition"]
if item["module_id"] == "core"
)
catalog, keyring = signed_catalog(
self.assessment,
selected_units=[
{
"repo": core["repository"],
"version": core["manifest_version"],
"tag": f"v{core['manifest_version']}",
"commit_sha": "0" * 40,
"tag_object_sha": "1" * 40,
}
],
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("review_required", report["status"])
self.assertIn(
"composition_commit_changed", {item["code"] for item in report["findings"]}
)
self.assertIn(
"composition.core", {item["id"] for item in report["review_targets"]}
)
self.assertFalse(report["proof_scope"]["release_metadata"]["valid"])
def test_missing_duplicate_and_malformed_selected_unit_provenance_blocks(
self,
) -> None:
core = next(
item
for item in self.assessment["composition"]
if item["module_id"] == "core"
)
valid_unit = {
"repo": core["repository"],
"version": core["manifest_version"],
"tag": f"v{core['manifest_version']}",
"commit_sha": (str(core["commit"]) + "0" * 40)[:40],
"tag_object_sha": "1" * 40,
}
cases = (
("missing", None, False, "catalog_source_provenance"),
(
"duplicate",
[valid_unit, deepcopy(valid_unit)],
True,
"catalog_release_metadata",
),
(
"malformed",
[{**valid_unit, "commit_sha": "not-a-git-object"}],
True,
"catalog_source_provenance",
),
)
for label, selected_units, include_selected_units, expected_code in cases:
with self.subTest(label=label):
catalog, keyring = signed_catalog(
self.assessment,
selected_units=selected_units,
include_selected_units=include_selected_units,
)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("blocked", report["status"])
self.assertIn(
expected_code, {item["code"] for item in report["findings"]}
)
self.assertFalse(report["proof_scope"]["release_metadata"]["valid"])
def test_catalog_validation_ignores_and_preserves_installer_replay_state(
self,
) -> None:
catalog, keyring = signed_catalog(self.assessment)
with tempfile.TemporaryDirectory() as temp_dir:
state_path = Path(temp_dir) / "catalog-sequence.json"
state_path.write_text(
json.dumps(
{"channels": {"stable": {"last_sequence": catalog["sequence"]}}}
),
encoding="utf-8",
)
replay_environment = {
"GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE": str(state_path),
"GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE": "true",
}
with mock.patch.dict(os.environ, replay_environment, clear=False):
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual(
replay_environment,
{key: os.environ[key] for key in replay_environment},
)
self.assertEqual("current", report["status"])
self.assertTrue(report["proof_scope"]["catalog_signature_and_keyring"]["valid"])
def test_self_consistent_substitution_fails_independent_trust_root(self) -> None:
_, trusted_keyring = signed_catalog(self.assessment)
catalog, published_keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=published_keyring,
trusted_keyring=trusted_keyring,
)
self.assertEqual("blocked", report["status"])
codes = {item["code"] for item in report["findings"]}
self.assertIn("catalog_trust", codes)
self.assertNotIn("catalog_keyring_hash_mismatch", codes)
self.assertTrue(report["proof_scope"]["assessment_schema"]["valid"])
self.assertFalse(
report["proof_scope"]["catalog_signature_and_trusted_keyring"]["valid"]
)
self.assertTrue(report["proof_scope"]["published_keyring_hash"]["valid"])
def test_locally_pinned_rotation_keys_fail_closed(self) -> None:
keys = trusted_keys_from_keyring(
{
"keys": [
{"key_id": "active", "status": "active", "public_key": "a"},
{"key_id": "next", "status": "next", "public_key": "b"},
{"key_id": "retired", "status": "retired", "public_key": "c"},
]
}
)
self.assertEqual({"active": "a", "next": "b"}, keys)
self.assertEqual(
{"legacy": "base64-key"},
trusted_keys_from_keyring({"legacy": "base64-key"}),
)
self.assertEqual(
{},
trusted_keys_from_keyring(
{
"keys": [
{
"key_id": "typo",
"status": "retierd",
"public_key": "unsafe",
}
]
}
),
)
self.assertEqual(
{},
trusted_keys_from_keyring(
{
"keys": [
{"key_id": "duplicate", "public_key": "a"},
{"key_id": "duplicate", "public_key": "b"},
]
}
),
)
self.assertEqual({}, trusted_keys_from_keyring({"keys": "invalid"}))
self.assertEqual(
{},
trusted_keys_from_keyring(
{"keys": [{"key_id": " ", "public_key": "unsafe"}]}
),
)
def test_schema_error_blocks_before_comparison(self) -> None:
assessment = deepcopy(self.assessment)
assessment.pop("scope")
catalog, keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=assessment,
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
self.assertEqual("blocked", report["status"])
self.assertEqual(
{"assessment_schema"}, {item["code"] for item in report["findings"]}
)
self.assertFalse(report["proof_scope"]["assessment_schema"]["valid"])
self.assertFalse(
report["proof_scope"]["catalog_signature_and_keyring"]["checked"]
)
self.assertIsNone(
report["proof_scope"]["catalog_signature_and_keyring"]["valid"]
)
def test_human_report_states_proof_limit(self) -> None:
catalog, keyring = signed_catalog(self.assessment)
report = review_capability_fit(
assessment=deepcopy(self.assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
)
rendered = render_review(report)
self.assertIn("Capability fit rerun: current", rendered)
self.assertIn(
"No installed-composition, installed-release-origin, target-environment, external-provider, production-approval proof",
rendered,
)
def test_public_fetch_failure_is_generic_and_blocking(self) -> None:
script = META_ROOT / "tools" / "assessments" / "capability-fit.py"
spec = importlib.util.spec_from_file_location("capability_fit_cli", script)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
with tempfile.TemporaryDirectory() as temp_dir:
trusted_path = Path(temp_dir) / "trusted-keyring.json"
trusted_path.write_text('{"keys": []}\n', encoding="utf-8")
with mock.patch.object(
module,
"fetch_json",
return_value={
"ok": False,
"error": "Authorization: secret at https://internal.invalid",
},
):
with self.assertRaisesRegex(
SystemExit, r"^Could not fetch fixed public catalog endpoint[.]$"
):
module.main(
[
"--public",
"--trusted-keyring",
str(trusted_path),
"--skip-tag-provenance",
]
)
def test_cli_requires_an_independent_local_trust_root(self) -> None:
script = META_ROOT / "tools" / "assessments" / "capability-fit.py"
spec = importlib.util.spec_from_file_location(
"capability_fit_cli_trust", script
)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
with mock.patch.dict(
os.environ,
{module.TRUSTED_KEYRING_FILE_ENV: ""},
clear=False,
):
with self.assertRaisesRegex(SystemExit, r"--trusted-keyring .* required"):
module.main(["--public", "--skip-tag-provenance"])
def test_local_selected_tag_commit_and_object_must_match_exactly(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
commit_sha, tag_object_sha = create_tagged_repository(
workspace=workspace,
repository="govoplan-core",
version="0.1.0",
)
assessment = deepcopy(self.assessment)
core = next(
deepcopy(item)
for item in assessment["composition"]
if item["module_id"] == "core"
)
core["manifest_version"] = "0.1.0"
core["commit"] = commit_sha[:12]
assessment["composition"] = [core]
selected_unit = {
"repo": "govoplan-core",
"version": "0.1.0",
"tag": "v0.1.0",
"commit_sha": commit_sha,
"tag_object_sha": tag_object_sha,
}
catalog, keyring = signed_catalog(
assessment,
selected_units=[selected_unit],
)
baseline = review_capability_fit(
assessment=deepcopy(assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
workspace_root=workspace,
)
self.assertEqual("current", baseline["status"])
self.assertTrue(baseline["proof_scope"]["local_tag_provenance"]["valid"])
forged_commit = commit_sha[:12] + ("0" * 28)
if forged_commit == commit_sha:
forged_commit = commit_sha[:12] + ("f" * 28)
forged_catalog, forged_keyring = signed_catalog(
assessment,
selected_units=[{**selected_unit, "commit_sha": forged_commit}],
)
commit_mismatch = review_capability_fit(
assessment=deepcopy(assessment),
schema=self.schema,
catalog=forged_catalog,
published_keyring=forged_keyring,
trusted_keyring=forged_keyring,
workspace_root=workspace,
)
self.assertEqual("review_required", commit_mismatch["status"])
self.assertIn(
"tag_signed_commit_changed",
{item["kind"] for item in commit_mismatch["changes"]},
)
repo = workspace / "govoplan-core"
git_text(repo, "tag", "-d", "v0.1.0")
git_text(repo, "tag", "-a", "v0.1.0", "-m", "Re-annotated release")
reannotated = review_capability_fit(
assessment=deepcopy(assessment),
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
workspace_root=workspace,
)
self.assertEqual("review_required", reannotated["status"])
self.assertIn(
"tag_object_changed",
{item["kind"] for item in reannotated["changes"]},
)
self.assertTrue(reannotated["proof_scope"]["local_tag_provenance"]["checked"])
self.assertFalse(reannotated["proof_scope"]["local_tag_provenance"]["valid"])
def test_missing_composition_entry_does_not_claim_complete_tag_proof(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
core_commit, core_tag_object = create_tagged_repository(
workspace=workspace,
repository="govoplan-core",
version="0.1.0",
)
tenancy_commit, _ = create_tagged_repository(
workspace=workspace,
repository="govoplan-tenancy",
version="0.1.0",
)
reviewed_assessment = deepcopy(self.assessment)
core = next(
deepcopy(item)
for item in reviewed_assessment["composition"]
if item["module_id"] == "core"
)
tenancy = next(
deepcopy(item)
for item in reviewed_assessment["composition"]
if item["module_id"] == "tenancy"
)
core.update(manifest_version="0.1.0", commit=core_commit[:12])
tenancy.update(manifest_version="0.1.0", commit=tenancy_commit[:12])
catalog_assessment = deepcopy(reviewed_assessment)
catalog_assessment["composition"] = [core]
reviewed_assessment["composition"] = [core, tenancy]
catalog, keyring = signed_catalog(
catalog_assessment,
selected_units=[
{
"repo": "govoplan-core",
"version": "0.1.0",
"tag": "v0.1.0",
"commit_sha": core_commit,
"tag_object_sha": core_tag_object,
}
],
)
report = review_capability_fit(
assessment=reviewed_assessment,
schema=self.schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=keyring,
workspace_root=workspace,
)
proof = report["proof_scope"]["local_tag_provenance"]
self.assertEqual("review_required", report["status"])
self.assertFalse(proof["checked"])
self.assertIsNone(proof["valid"])
self.assertEqual(2, proof["attempted_count"])
self.assertEqual(2, proof["expected_count"])
self.assertNotIn(
"tag_provenance_changed", {item["code"] for item in report["findings"]}
)
def test_repository_path_traversal_is_not_resolved(self) -> None:
with mock.patch("govoplan_assessment.capability_fit.subprocess.run") as run:
result = local_tag_provenance(
workspace_root=META_ROOT.parent,
repository="../outside",
version="0.1.0",
assessed_commit="0123456",
)
self.assertEqual("tag_provenance_unavailable", result["kind"])
self.assertIn("invalid repository identifier", result["message"])
run.assert_not_called()
def create_tagged_repository(
*, workspace: Path, repository: str, version: str
) -> tuple[str, str]:
repo = workspace / repository
git_text(workspace, "init", "-b", "main", str(repo))
git_text(repo, "config", "user.name", "Capability Fit Test")
git_text(repo, "config", "user.email", "capability-fit@example.invalid")
(repo / "README.md").write_text(f"{repository}\n", encoding="utf-8")
git_text(repo, "add", "README.md")
git_text(repo, "commit", "-m", "Initial release fixture")
git_text(repo, "tag", "-a", f"v{version}", "-m", f"Release v{version}")
return (
git_text(repo, "rev-parse", "HEAD"),
git_text(repo, "rev-parse", f"refs/tags/v{version}"),
)
def git_text(cwd: Path, *args: str) -> str:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
capture_output=True,
text=True,
)
if result.returncode != 0:
raise AssertionError(result.stderr or result.stdout)
return result.stdout.strip()
def signed_catalog(
assessment: dict[str, object],
*,
versions: dict[str, str] | None = None,
sequence: int = 202607220843,
selected_units: list[dict[str, object]] | None = None,
include_selected_units: bool = True,
release_artifacts: list[dict[str, object]] | None = None,
) -> tuple[dict[str, object], dict[str, object]]:
versions = versions or {}
private_key = Ed25519PrivateKey.generate()
public_key = base64.b64encode(
private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
).decode("ascii")
keyring: dict[str, object] = {
"keyring_version": "1",
"purpose": "test",
"generated_at": "2026-01-01T00:00:00Z",
"keys": [{"key_id": "test-key", "status": "active", "public_key": public_key}],
}
now = datetime.now(tz=UTC)
modules: list[dict[str, object]] = []
core_release: dict[str, object] | None = None
for component in assessment["composition"]:
module_id = str(component["module_id"])
repository = str(component["repository"])
version = versions.get(module_id, str(component["manifest_version"]))
entry: dict[str, object] = {
"module_id": module_id,
"name": module_id,
"version": version,
"python_package": repository,
"python_ref": f"{repository} @ git+ssh://git@example.invalid/example/{repository}.git@v{version}",
}
if module_id == "core":
core_release = entry
else:
modules.append(entry)
assert core_release is not None
if selected_units is None:
selected_units = [
{
"repo": component["repository"],
"version": versions.get(
str(component["module_id"]), str(component["manifest_version"])
),
"tag": "v"
+ versions.get(
str(component["module_id"]), str(component["manifest_version"])
),
"commit_sha": (str(component["commit"]) + "0" * 40)[:40],
"tag_object_sha": hashlib.sha256(
f"{component['repository']}:{component['manifest_version']}:tag".encode()
).hexdigest()[:40],
}
for component in assessment["composition"]
]
release: dict[str, object] = {"keyring_sha256": canonical_hash(keyring)}
if include_selected_units:
release["selected_units"] = selected_units
if release_artifacts is not None:
release["artifacts"] = release_artifacts
catalog: dict[str, object] = {
"catalog_version": "1",
"channel": "stable",
"sequence": sequence,
"generated_at": now.isoformat().replace("+00:00", "Z"),
"expires_at": (now + timedelta(days=30)).isoformat().replace("+00:00", "Z"),
"core_release": core_release,
"modules": modules,
"release": release,
}
signature_payload = json.dumps(
catalog, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
catalog["signatures"] = [
{
"algorithm": "ed25519",
"key_id": "test-key",
"value": base64.b64encode(private_key.sign(signature_payload)).decode(
"ascii"
),
}
]
return catalog, keyring
def canonical_hash(payload: object) -> str:
encoded = json.dumps(
payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,144 @@
from __future__ import annotations
import importlib.util
import json
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "checks" / "check_dependency_boundaries.py"
def load_boundary_module():
spec = importlib.util.spec_from_file_location("check_dependency_boundaries", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class DependencyBoundaryDiscoveryTests(unittest.TestCase):
def test_editable_install_metadata_is_not_treated_as_a_source_package(self) -> None:
boundary = load_boundary_module()
with tempfile.TemporaryDirectory(prefix="govoplan-boundary-discovery-") as directory:
root = Path(directory)
package = root / "govoplan-example" / "src" / "govoplan_example"
package.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8")
(package.parent / "govoplan_example.egg-info").mkdir()
repos, prefixes, errors = boundary._discover_python_repos(root)
self.assertEqual(errors, ())
self.assertEqual(repos, {"govoplan-example": package})
self.assertEqual(prefixes, {"govoplan-example": "govoplan_example"})
def test_ambiguous_or_missing_source_packages_fail_discovery(self) -> None:
boundary = load_boundary_module()
with tempfile.TemporaryDirectory(prefix="govoplan-boundary-discovery-") as directory:
root = Path(directory)
missing_source = root / "govoplan-missing" / "src"
missing_source.mkdir(parents=True)
(missing_source / "govoplan_missing.egg-info").mkdir()
ambiguous_source = root / "govoplan-ambiguous" / "src"
for name in ("govoplan_alpha", "govoplan_beta"):
package = ambiguous_source / name
package.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8")
repos, prefixes, errors = boundary._discover_python_repos(root)
self.assertEqual(repos, {})
self.assertEqual(prefixes, {})
self.assertEqual(len(errors), 2)
self.assertTrue(any("govoplan-missing" in error and "found 0" in error for error in errors))
self.assertTrue(any("govoplan-ambiguous" in error and "found 2" in error for error in errors))
def test_duplicate_python_package_ownership_fails_discovery(self) -> None:
boundary = load_boundary_module()
with tempfile.TemporaryDirectory(prefix="govoplan-boundary-discovery-") as directory:
root = Path(directory)
for owner in ("govoplan-first", "govoplan-second"):
package = root / owner / "src" / "govoplan_shared"
package.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8")
repos, prefixes, errors = boundary._discover_python_repos(root)
self.assertEqual(set(repos), {"govoplan-first"})
self.assertEqual(prefixes, {"govoplan-first": "govoplan_shared"})
self.assertEqual(len(errors), 1)
self.assertIn("already owned", errors[0])
def test_workspace_discovery_covers_changed_capability_repositories(self) -> None:
boundary = load_boundary_module()
self.assertEqual(boundary.PYTHON_DISCOVERY_ERRORS, ())
self.assertGreaterEqual(len(boundary.REPOS), 40)
for owner in (
"govoplan-core",
"govoplan-identity",
"govoplan-idm",
"govoplan-calendar",
"govoplan-poll",
"govoplan-scheduling",
):
self.assertIn(owner, boundary.REPOS)
def test_webui_discovery_is_fail_closed(self) -> None:
boundary = load_boundary_module()
with tempfile.TemporaryDirectory(prefix="govoplan-boundary-webui-") as directory:
root = Path(directory)
good = root / "govoplan-good" / "webui"
good.mkdir(parents=True)
(good / "package.json").write_text(
json.dumps({"name": "@govoplan/good-webui"}),
encoding="utf-8",
)
missing = root / "govoplan-missing" / "webui"
missing.mkdir(parents=True)
invalid = root / "govoplan-invalid" / "webui"
invalid.mkdir(parents=True)
(invalid / "package.json").write_text("{", encoding="utf-8")
unnamed = root / "govoplan-unnamed" / "webui"
unnamed.mkdir(parents=True)
(unnamed / "package.json").write_text("{}", encoding="utf-8")
duplicate = root / "govoplan-zduplicate" / "webui"
duplicate.mkdir(parents=True)
(duplicate / "package.json").write_text(
json.dumps({"name": "@govoplan/good-webui"}),
encoding="utf-8",
)
repos, packages, errors = boundary._discover_webui_repos(root)
self.assertEqual(repos, {"govoplan-good": good})
self.assertEqual(packages, {"govoplan-good": "@govoplan/good-webui"})
self.assertEqual(len(errors), 4)
self.assertTrue(any("govoplan-missing" in error and "no package.json" in error for error in errors))
self.assertTrue(any("govoplan-invalid" in error and "invalid" in error for error in errors))
self.assertTrue(any("govoplan-unnamed" in error and "package name" in error for error in errors))
self.assertTrue(any("govoplan-zduplicate" in error and "already owned" in error for error in errors))
def test_workspace_webui_discovery_covers_composition_repositories(self) -> None:
boundary = load_boundary_module()
self.assertEqual(boundary.WEBUI_DISCOVERY_ERRORS, ())
self.assertGreaterEqual(len(boundary.WEBUI_REPOS), 17)
for owner in (
"govoplan-core",
"govoplan-calendar",
"govoplan-scheduling",
"govoplan-notifications",
):
self.assertIn(owner, boundary.WEBUI_REPOS)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,82 @@
from __future__ import annotations
import hashlib
import importlib.util
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "repo" / "sync-python-environment.py"
def load_sync_module():
spec = importlib.util.spec_from_file_location("sync_python_environment", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class PythonEnvironmentSyncTests(unittest.TestCase):
def test_stale_local_projects_share_one_resolver_transaction(self) -> None:
sync = load_sync_module()
with tempfile.TemporaryDirectory(prefix="govoplan-python-sync-") as directory:
root = Path(directory)
requirements = root / "requirements-dev.txt"
requirements.write_text("-e ./govoplan-core\n-e ./govoplan-access\n", encoding="utf-8")
for project in ("govoplan-core", "govoplan-access"):
project_root = root / project
project_root.mkdir()
(project_root / "pyproject.toml").write_text(
f'[project]\nname = "{project}"\nversion = "0.1.10"\n',
encoding="utf-8",
)
local_requirements = sync.local_requirement_entries(requirements)
fingerprint = sync.build_fingerprint(
requirements=requirements,
python="/test/venv/bin/python",
local_requirements=local_requirements,
)
requirements_digest = hashlib.sha256(requirements.read_bytes()).hexdigest()
previous = {
"version": sync.STAMP_VERSION,
"python": "/test/venv/bin/python",
"inputs": [
{"path": str(requirements), "sha256": requirements_digest},
*(
{"path": entry.pyproject, "sha256": "stale"}
for entry in local_requirements
),
],
"requirements_entries": [
entry.as_dict() for entry in sync.parse_requirement_entries(requirements)
],
}
plan = sync.build_install_plan(
previous=previous,
fingerprint=fingerprint,
requirements=requirements,
python="/test/venv/bin/python",
local_requirements=local_requirements,
force=False,
)
self.assertEqual(plan.mode, "Selective Python environment sync")
self.assertIn("one resolver transaction", plan.reason)
self.assertEqual(len(plan.commands), 1)
command = plan.commands[0]
self.assertEqual(command[:5], ("/test/venv/bin/python", "-m", "pip", "install", "-e"))
self.assertEqual(command.count("-e"), 2)
self.assertIn(str(root / "govoplan-core"), command)
self.assertIn(str(root / "govoplan-access"), command)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,277 @@
from __future__ import annotations
import csv
from datetime import UTC, datetime
import io
from pathlib import Path
import sys
import tempfile
import unittest
from unittest import mock
import zipfile
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.artifact_identity import ( # noqa: E402
ArtifactIdentityError,
inspect_python_wheel,
selected_artifact_identity_issues,
)
from govoplan_release.selective_catalog import ( # noqa: E402
apply_python_artifact_identities,
)
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
if str(ASSESSMENT_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(ASSESSMENT_TOOLS_ROOT))
from govoplan_assessment.installer_receipt import issue_installer_receipt # noqa: E402
class ReleaseArtifactIdentityTests(unittest.TestCase):
def test_built_wheel_bytes_become_catalog_identity(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
wheel = self._wheel(Path(temp_dir), console_script=True)
payload = self._catalog_payload()
changes = apply_python_artifact_identities(
payload,
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={"govoplan-demo": wheel},
)
identity = payload["release"]["artifacts"][0]
self.assertEqual("govoplan-demo", identity["package_name"])
self.assertEqual("1.2.3", identity["package_version"])
self.assertRegex(identity["archive_sha256"], r"^[0-9a-f]{64}$")
self.assertTrue(identity["requires_installer_receipt"])
self.assertEqual("release.artifact", changes[0].field)
self.assertEqual((), selected_artifact_identity_issues(payload))
def test_version_update_without_wheel_removes_stale_identity_and_blocks_publish(self) -> None:
payload = self._catalog_payload()
payload["release"]["artifacts"] = [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-demo",
"package_version": "1.2.2",
"archive_sha256": "a" * 64,
"archive_size": 10,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "b" * 64,
"file_count": 1,
},
"requires_installer_receipt": False,
}
]
apply_python_artifact_identities(
payload,
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={},
)
self.assertNotIn("artifacts", payload["release"])
self.assertIn(
"no built artifact identity",
" ".join(selected_artifact_identity_issues(payload)),
)
def test_tag_only_selection_needs_no_python_artifact_mapping(self) -> None:
payload = self._catalog_payload()
payload["release"]["selected_units"].append(
{"repo": "govoplan", "version": "1.2.3"}
)
apply_python_artifact_identities(
payload,
repo_versions={"govoplan": "1.2.3"},
python_artifacts={},
)
self.assertIn("selected_units", payload["release"])
def test_unsafe_or_mismatched_wheel_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
unsafe = root / "govoplan_demo-1.2.3-py3-none-any.whl"
with zipfile.ZipFile(unsafe, "w") as archive:
archive.writestr("../escape", b"bad")
archive.writestr(
"govoplan_demo-1.2.3.dist-info/METADATA",
"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: 1.2.3\n",
)
wrong = self._wheel(root, version="9.9.9")
with self.assertRaisesRegex(ArtifactIdentityError, "unsafe"):
inspect_python_wheel(unsafe)
with self.assertRaisesRegex(ValueError, "expected govoplan-demo 1.2.3"):
apply_python_artifact_identities(
self._catalog_payload(),
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={"govoplan-demo": wrong},
)
def test_receipt_issuer_hashes_exact_consumed_wheel(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
original = self._wheel(root / "original", value=b"VALUE = 1\n")
different = self._wheel(root / "different", value=b"VALUE = 2\n")
identity = inspect_python_wheel(original)
catalog = self._catalog_payload()
catalog["channel"] = "stable"
catalog["sequence"] = 1
catalog["release"]["artifacts"] = [identity.catalog_payload()]
assessment = {
"assessment_id": "assessment:test",
"release": {"ref": "stable-catalog-1"},
"composition": [{"module_id": "demo", "enabled": True}],
}
installed_payload = {
"algorithm": "govoplan-installed-record-payload-v1",
"sha256": "c" * 64,
"file_count": identity.payload_file_count,
}
evidence = {
"assessment_id": "assessment:test",
"assessment_release": "stable-catalog-1",
"collected_at": "2026-07-22T12:00:00Z",
"artifacts": [
{
"package_name": "govoplan-demo",
"package_version": "1.2.3",
"record_integrity": {
"status": "verified",
"hashed_file_count": identity.payload_file_count,
"permitted_unhashed_file_count": 1,
"generated_unhashed_file_count": 0,
"unverifiable_file_count": 0,
"missing_file_count": 0,
"mismatched_file_count": 0,
"artifact_payload_identity": catalog["release"]["artifacts"][0]["installed_payload"],
"installed_payload_identity": installed_payload,
},
}
],
}
key = Ed25519PrivateKey.generate()
receipt = issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": original},
issued_at=datetime(2026, 7, 22, 12, 1, tzinfo=UTC),
same_process_observation=True,
)
with self.assertRaisesRegex(ValueError, "differs from the signed"):
issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": different},
issued_at=datetime(2026, 7, 22, 12, 1, tzinfo=UTC),
same_process_observation=True,
)
with self.assertRaisesRegex(ValueError, "within five minutes"):
issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": original},
issued_at=datetime(2026, 7, 22, 12, 6, tzinfo=UTC),
same_process_observation=True,
)
self.assertEqual(identity.archive_sha256, receipt["artifacts"][0]["catalog_archive_sha256"])
def test_path_replacement_during_one_descriptor_inspection_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
target = self._wheel(root / "target", value=b"VALUE = 1\n")
replacement = self._wheel(root / "replacement", value=b"VALUE = 2\n")
real_zip = zipfile.ZipFile
def swap_path(file_or_path, *args, **kwargs):
target.unlink()
replacement.rename(target)
return real_zip(file_or_path, *args, **kwargs)
with (
mock.patch(
"govoplan_release.artifact_identity.zipfile.ZipFile",
side_effect=swap_path,
),
self.assertRaisesRegex(ArtifactIdentityError, "changed"),
):
inspect_python_wheel(target)
@staticmethod
def _catalog_payload() -> dict[str, object]:
return {
"core_release": {},
"modules": [
{
"module_id": "demo",
"version": "1.2.3",
"python_package": "govoplan-demo",
"python_ref": "govoplan-demo @ git+ssh://git@example.test/acme/govoplan-demo.git@v1.2.3",
}
],
"release": {
"selected_units": [
{"repo": "govoplan-demo", "version": "1.2.3"}
]
},
}
@staticmethod
def _wheel(
root: Path,
*,
version: str = "1.2.3",
console_script: bool = False,
value: bytes = b"VALUE = 1\n",
) -> Path:
root.mkdir(parents=True, exist_ok=True)
wheel = root / f"govoplan_demo-{version}-py3-none-any.whl"
dist_info = f"govoplan_demo-{version}.dist-info"
files: dict[str, bytes] = {
"govoplan_demo.py": value,
f"{dist_info}/METADATA": (
f"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: {version}\n"
).encode(),
f"{dist_info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
}
if console_script:
files[f"{dist_info}/entry_points.txt"] = (
b"[console_scripts]\ngovoplan-demo = govoplan_demo:main\n"
)
record_buffer = io.StringIO()
writer = csv.writer(record_buffer, lineterminator="\n")
for name, encoded in files.items():
writer.writerow((name, "", len(encoded)))
writer.writerow((f"{dist_info}/RECORD", "", ""))
files[f"{dist_info}/RECORD"] = record_buffer.getvalue().encode()
with zipfile.ZipFile(wheel, "w", compression=zipfile.ZIP_DEFLATED) as archive:
for name, encoded in files.items():
archive.writestr(name, encoded)
return wheel
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,184 @@
from __future__ import annotations
import json
from datetime import UTC, datetime
from pathlib import Path
import sys
import tempfile
import unittest
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
candidate_keyring_from_authenticated_base,
load_authenticated_catalog_base,
public_key_base64,
signature,
)
class ReleaseBaseCatalogTrustTests(unittest.TestCase):
def test_exact_signed_base_pair_is_loaded_once_and_carried_in_memory(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
authenticated = load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={"release-key": public_key_base64(private_key)},
)
keyring.unlink()
self.assertEqual("release-key", authenticated.keyring["keys"][0]["key_id"])
self.assertEqual(
canonical_hash(authenticated.keyring),
authenticated.catalog["release"]["keyring_sha256"],
)
def test_deliberate_old_to_new_signer_rotation_is_carried_forward(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
new_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
configured = {
"release-key": public_key_base64(private_key),
"release-key-next": public_key_base64(new_key),
}
authenticated = load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys=configured,
)
candidate = candidate_keyring_from_authenticated_base(
authenticated.keyring,
signer_public_keys=configured,
generated_at=datetime(2026, 7, 22, 12, tzinfo=UTC),
)
self.assertEqual(
{"release-key", "release-key-next"},
{item["key_id"] for item in candidate["keys"]},
)
def test_injected_extra_key_without_catalog_authorization_is_rejected(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
extra_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
payload = json.loads(keyring.read_text(encoding="utf-8"))
payload["keys"].append(
{
"key_id": "injected-key",
"status": "active",
"public_key": public_key_base64(extra_key),
}
)
keyring.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
def test_unpinned_or_symlinked_base_keyring_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
payload = json.loads(keyring.read_text(encoding="utf-8"))
payload["generated_at"] = "changed"
keyring.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
keyring.unlink()
outside = root / "outside.json"
outside.write_text("{}", encoding="utf-8")
keyring.symlink_to(outside)
with self.assertRaisesRegex(ValueError, "opened safely"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
@staticmethod
def _write_pair(
root: Path, *, private_key: Ed25519PrivateKey
) -> tuple[Path, Path]:
keys = [
{
"key_id": "release-key",
"status": "active",
"public_key": public_key_base64(private_key),
}
]
keyring_payload = {
"keyring_version": "1",
"purpose": "govoplan module package catalog signatures",
"keys": keys,
}
catalog_payload = {
"channel": "stable",
"sequence": 1,
"core_release": {"version": "1.0.0"},
"modules": [],
"release": {"keyring_sha256": canonical_hash(keyring_payload)},
}
catalog_payload["signatures"] = [
signature(
catalog_payload,
key_id="release-key",
private_key=private_key,
)
]
catalog = root / "stable.json"
keyring = root / "keyring.json"
catalog.write_text(json.dumps(catalog_payload), encoding="utf-8")
keyring.write_text(json.dumps(keyring_payload), encoding="utf-8")
return catalog, keyring
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,153 @@
from __future__ import annotations
import json
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.candidate_artifact import ( # noqa: E402
CandidateArtifactError,
candidate_output_path,
issue_candidate_id,
issue_candidate_receipt,
verify_candidate_receipt,
validate_release_channel,
)
class CandidateArtifactTests(unittest.TestCase):
def test_channel_is_one_canonical_basename(self) -> None:
self.assertEqual("stable_1", validate_release_channel("stable_1"))
for value in ("../stable", "/stable", ".", "..", "Stable", "stable/name"):
with self.subTest(value=value), self.assertRaises(CandidateArtifactError):
validate_release_channel(value)
def test_handle_is_deterministic_and_can_precede_output(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "not-created" / "release-candidates"
candidate_id = issue_candidate_id("rr-123", "attempt-456")
first = candidate_output_path(root, candidate_id)
second = candidate_output_path(root, candidate_id)
self.assertEqual(first, second)
self.assertEqual(candidate_id, first.name)
self.assertRegex(candidate_id, r"^candidate-[0-9a-f]{32}$")
def test_receipt_rejects_catalog_drift(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
catalog = self._write_candidate(root / candidate_id)
receipt = issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
resolved = verify_candidate_receipt(
root=root,
candidate_id=receipt.candidate_id,
catalog_sha256=receipt.catalog_sha256,
channel="stable",
)
catalog.write_text(
json.dumps({"channel": "stable", "sequence": 2, "signatures": [{}]}),
encoding="utf-8",
)
self.assertEqual(root / candidate_id, resolved)
with self.assertRaisesRegex(CandidateArtifactError, "no longer matches"):
verify_candidate_receipt(
root=root,
candidate_id=receipt.candidate_id,
catalog_sha256=receipt.catalog_sha256,
channel="stable",
)
def test_unissued_ids_traversal_and_symlinks_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
root.mkdir(mode=0o700)
candidate_id = issue_candidate_id("run", "attempt")
outside = Path(temp_dir) / "outside"
outside.mkdir()
(root / candidate_id).symlink_to(outside, target_is_directory=True)
with self.assertRaises(CandidateArtifactError):
candidate_output_path(root, "../candidate-" + "0" * 32)
with self.assertRaisesRegex(CandidateArtifactError, "real directory"):
candidate_output_path(root, candidate_id)
def test_catalog_and_channel_components_must_not_be_symlinks(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
candidate = root / candidate_id
target = Path(temp_dir) / "target"
target.mkdir()
(target / "stable.json").write_text(
json.dumps({"signatures": [{}]}), encoding="utf-8"
)
root.mkdir(mode=0o700)
candidate.mkdir(mode=0o700)
(candidate / "channels").symlink_to(target, target_is_directory=True)
with self.assertRaisesRegex(CandidateArtifactError, "real directory"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
def test_shared_candidate_roots_and_catalog_files_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
root.mkdir(mode=0o755)
candidate_id = issue_candidate_id("run", "attempt")
with self.assertRaisesRegex(CandidateArtifactError, "another user"):
candidate_output_path(root, candidate_id)
root.chmod(0o700)
catalog = self._write_candidate(root / candidate_id)
catalog.chmod(0o640)
with self.assertRaisesRegex(CandidateArtifactError, "another user"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
def test_receipt_rejects_catalog_with_inconsistent_channel(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
catalog = self._write_candidate(root / candidate_id)
catalog.write_text(
json.dumps({"channel": "next", "signatures": [{}]}),
encoding="utf-8",
)
with self.assertRaisesRegex(CandidateArtifactError, "does not match"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
@staticmethod
def _write_candidate(candidate: Path) -> Path:
candidate.parent.mkdir(mode=0o700, exist_ok=True)
candidate.mkdir(mode=0o700)
channels = candidate / "channels"
channels.mkdir(mode=0o700)
catalog = channels / "stable.json"
catalog.write_text(
json.dumps({"channel": "stable", "sequence": 1, "signatures": [{}]}),
encoding="utf-8",
)
catalog.chmod(0o600)
return catalog
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,84 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.catalog_entry_synthesis import validate_initial_entry_closure # noqa: E402
from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402
class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
def test_selective_update_synthesizes_initial_entries_from_package_manifests(self) -> None:
payload: dict[str, object] = {
"core_release": {},
"release": {},
"modules": [
{
"module_id": "access",
"version": "0.1.11",
"python_package": "govoplan-access",
"python_ref": "govoplan-access @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-access.git@v0.1.11",
}
],
}
changes = apply_repo_updates(
payload,
repo_versions={
"govoplan-addresses": "0.1.9",
"govoplan-poll": "0.1.11",
"govoplan-scheduling": "0.1.11",
},
repo_contracts={},
repository_base="git+ssh://git@git.add-ideas.de/add-ideas",
workspace=META_ROOT.parent,
)
modules = {
item["module_id"]: item
for item in payload["modules"] # type: ignore[index]
}
self.assertEqual({"access", "addresses", "poll", "scheduling"}, set(modules))
self.assertEqual("@govoplan/addresses-webui", modules["addresses"]["webui_package"])
self.assertIn(
{"name": "addresses.people_search", "version": "0.1.0"},
modules["addresses"]["provides_interfaces"],
)
self.assertEqual("govoplan-poll", modules["poll"]["python_package"])
self.assertEqual(["poll"], modules["scheduling"]["dependencies"])
self.assertEqual("@govoplan/scheduling-webui", modules["scheduling"]["webui_package"])
self.assertEqual("requires_review", modules["scheduling"]["migration_safety"])
self.assertTrue(
any(
item["name"] == "poll.governed_participation" and item["optional"] is False
for item in modules["scheduling"]["requires_interfaces"]
)
)
self.assertEqual(
{"govoplan-addresses", "govoplan-poll", "govoplan-scheduling"},
{change.repo for change in changes},
)
self.assertEqual({"catalog_entry"}, {change.field for change in changes})
def test_initial_required_dependency_must_be_represented(self) -> None:
with self.assertRaisesRegex(ValueError, "requires catalog module 'poll'"):
validate_initial_entry_closure(
catalog_modules=[
{
"module_id": "scheduling",
"dependencies": ["poll"],
}
],
initial_module_ids={"scheduling"},
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,796 @@
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.publisher import ( # noqa: E402
FrozenPublicationRemote,
bind_publication_remote,
commit_publication_tree,
publication_mutation_trust_issues,
publish_catalog_candidate,
remote_publication_identity,
run_checked,
verify_committed_publication,
verify_remote_branch_head,
verify_remote_publication,
_git_bytes,
_sanitized_git_environment,
_seal_git_metadata_file,
_trusted_npm_command,
)
class ReleaseCatalogPublicationTests(unittest.TestCase):
def test_publication_git_writes_ignore_permissive_operator_umask(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
repository = Path(temp_dir) / "website"
repository.mkdir()
self._git(repository, "init", "--quiet")
payload_number = 0
while True:
payload = f"private publication object {payload_number}\n".encode()
header = f"blob {len(payload)}\0".encode()
expected_object = hashlib.sha1(
header + payload,
usedforsecurity=False,
).hexdigest()
object_parent = repository / ".git" / "objects" / expected_object[:2]
if not object_parent.exists():
break
payload_number += 1
previous_umask = os.umask(0o002)
try:
object_id = (
_git_bytes(
repository,
"hash-object",
"-w",
"--stdin",
input_bytes=payload,
)
.decode("ascii")
.strip()
)
finally:
os.umask(previous_umask)
object_path = object_parent / expected_object[2:]
self.assertEqual(expected_object, object_id)
self.assertEqual(os.geteuid(), object_parent.stat().st_uid)
self.assertEqual(0o700, object_parent.stat().st_mode & 0o777)
self.assertEqual(os.geteuid(), object_path.stat().st_uid)
self.assertEqual(0o400, object_path.stat().st_mode & 0o777)
def test_publication_git_metadata_is_sealed_after_git_writes(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
metadata = Path(temp_dir) / "index"
metadata.write_bytes(b"index")
metadata.chmod(0o664)
_seal_git_metadata_file(metadata, label="test index")
self.assertEqual(0o600, metadata.stat().st_mode & 0o777)
def test_publication_git_identity_is_fixed_and_non_personal(self) -> None:
with patch.dict(
os.environ,
{
"GIT_AUTHOR_NAME": "Attacker",
"GIT_AUTHOR_EMAIL": "attacker@example.test",
"GIT_COMMITTER_NAME": "Attacker",
"GIT_COMMITTER_EMAIL": "attacker@example.test",
},
clear=False,
):
environment = _sanitized_git_environment()
self.assertEqual(
"GovOPlaN Release Automation",
environment["GIT_AUTHOR_NAME"],
)
self.assertEqual(
"release@govoplan.invalid",
environment["GIT_AUTHOR_EMAIL"],
)
self.assertEqual(
environment["GIT_AUTHOR_NAME"],
environment["GIT_COMMITTER_NAME"],
)
self.assertEqual(
environment["GIT_AUTHOR_EMAIL"],
environment["GIT_COMMITTER_EMAIL"],
)
def test_build_command_uses_its_pinned_node_directory(self) -> None:
completed = subprocess.CompletedProcess(
["/trusted/node/bin/npm"],
0,
stdout=b"",
stderr=b"",
)
with patch(
"govoplan_release.publisher.subprocess.run",
return_value=completed,
) as runner:
run_checked(
["/trusted/node/bin/npm", "run", "build"],
cwd=Path("/trusted/website"),
)
environment = runner.call_args.kwargs["env"]
self.assertEqual(
"/trusted/node/bin:/usr/bin:/bin",
environment["PATH"],
)
def test_npm_command_requires_trusted_npm_and_sibling_node(self) -> None:
with (
patch(
"govoplan_release.publisher.shutil.which",
return_value="/trusted/node/bin/npm",
),
patch(
"govoplan_release.publisher._trusted_runtime_executable_issue",
side_effect=(None, None),
) as trust_check,
):
self.assertEqual(
"/trusted/node/bin/npm",
_trusted_npm_command("npm"),
)
self.assertEqual(Path("/trusted/node/bin/npm"), trust_check.call_args_list[0].args[0])
self.assertEqual(Path("/trusted/node/bin/node"), trust_check.call_args_list[1].args[0])
def test_npm_command_rejects_caller_relative_path(self) -> None:
with (
patch("govoplan_release.publisher.shutil.which") as which,
self.assertRaisesRegex(RuntimeError, "absolute path or the bare name"),
):
_trusted_npm_command("./npm")
which.assert_not_called()
def test_committed_publication_must_match_validated_blobs_exactly(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
web_root = Path(tmp) / "website"
module_root = web_root / "public" / "catalogs" / "v1" / "modules"
module_root.mkdir(parents=True)
catalog = (
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
)
keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
catalog.parent.mkdir(parents=True)
expected = {
catalog.relative_to(web_root).as_posix(): b'{"catalog":true}\n',
keyring.relative_to(web_root).as_posix(): b'{"keys":[]}\n',
(module_root / "index.json")
.relative_to(web_root)
.as_posix(): b'{"modules":[]}\n',
}
for relative, encoded in expected.items():
target = web_root / relative
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(encoded)
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "publication")
commit_sha = self._git(web_root, "rev-parse", "HEAD").strip()
verify_committed_publication(
web_root=web_root,
commit_sha=commit_sha,
expected_blobs=expected,
module_root=module_root,
)
changed = dict(expected)
changed[catalog.relative_to(web_root).as_posix()] = b'{"catalog":false}\n'
with self.assertRaisesRegex(RuntimeError, "differs"):
verify_committed_publication(
web_root=web_root,
commit_sha=commit_sha,
expected_blobs=changed,
module_root=module_root,
)
catalog_path = catalog.relative_to(web_root).as_posix()
self._git(web_root, "update-index", "--chmod=+x", catalog_path)
self._git(web_root, "commit", "--quiet", "-m", "unsafe mode")
executable_commit = self._git(web_root, "rev-parse", "HEAD").strip()
with self.assertRaisesRegex(RuntimeError, "regular blob"):
verify_committed_publication(
web_root=web_root,
commit_sha=executable_commit,
expected_blobs=expected,
module_root=module_root,
)
def test_private_index_commit_has_one_parent_and_only_computed_delta(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
web_root = root / "website"
module_root = web_root / "public" / "catalogs" / "v1" / "modules"
channel = (
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
)
keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
old_module = module_root / "obsolete.json"
unrelated = web_root / "unrelated.txt"
for path, encoded in (
(channel, b'{"old":true}\n'),
(keyring, b'{"keys":[]}\n'),
(old_module, b'{"obsolete":true}\n'),
(unrelated, b"keep\n"),
):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(encoded)
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
frozen_head = self._git(web_root, "rev-parse", "HEAD").strip()
branch = self._git(web_root, "branch", "--show-current").strip()
malicious = web_root / "not-in-publication.txt"
malicious.write_text("staged but excluded\n", encoding="utf-8")
self._git(web_root, "add", malicious.name)
marker = root / "reference-hook-ran"
hook = web_root / ".git" / "hooks" / "reference-transaction"
hook.write_text(f"#!/bin/sh\ntouch {marker}\n", encoding="utf-8")
hook.chmod(0o755)
expected = {
channel.relative_to(web_root).as_posix(): b'{"new":true}\n',
keyring.relative_to(web_root).as_posix(): b'{"keys":["release"]}\n',
(module_root / "index.json")
.relative_to(web_root)
.as_posix(): b'{"modules":[]}\n',
}
redirected = root / "attacker-index"
with patch.dict(
os.environ,
{
"GIT_DIR": str(root / "attacker-git-dir"),
"GIT_INDEX_FILE": str(redirected),
"GIT_OBJECT_DIRECTORY": str(root / "attacker-objects"),
"GIT_CONFIG_GLOBAL": str(root / "attacker-config"),
},
):
commit_sha = commit_publication_tree(
web_root=web_root,
frozen_head=frozen_head,
branch=branch,
expected_blobs=expected,
module_root=module_root,
message="Exact publication",
)
parents = self._git(
web_root, "rev-list", "--parents", "-n", "1", commit_sha
).split()
changed = set(
filter(
None,
self._git(
web_root,
"diff-tree",
"--no-commit-id",
"--name-only",
"-r",
frozen_head,
commit_sha,
).splitlines(),
)
)
hook_ran = marker.exists()
inherited_index_used = redirected.exists()
commit_paths = self._git(
web_root, "ls-tree", "-r", "--name-only", commit_sha
)
self.assertEqual([commit_sha, frozen_head], parents)
self.assertEqual(
{
"public/catalogs/v1/channels/stable.json",
"public/catalogs/v1/keyring.json",
"public/catalogs/v1/modules/index.json",
"public/catalogs/v1/modules/obsolete.json",
},
changed,
)
self.assertFalse(hook_ran)
self.assertFalse(inherited_index_used)
self.assertNotIn("not-in-publication.txt", commit_paths)
def test_remote_binding_and_annotated_publication_identity_are_exact(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
remote_root = root / "website.git"
remote_root.mkdir()
self._git(remote_root, "init", "--bare", "--quiet")
web_root = root / "website"
web_root.mkdir()
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "branch", "-M", "main")
web_root.joinpath("base.txt").write_text("base\n", encoding="utf-8")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
self._git(web_root, "remote", "add", "origin", str(remote_root))
self._git(web_root, "push", "--quiet", "-u", "origin", "main")
base_commit = self._git(web_root, "rev-parse", "HEAD").strip()
frozen = bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
self.assertIsInstance(frozen, FrozenPublicationRemote)
verify_remote_branch_head(
web_root=web_root,
remote_url=frozen.url,
branch="main",
expected_commit=base_commit,
)
web_root.joinpath("catalog.json").write_text("{}\n", encoding="utf-8")
self._git(web_root, "add", "catalog.json")
self._git(web_root, "commit", "--quiet", "-m", "publication")
commit_sha = self._git(web_root, "rev-parse", "HEAD").strip()
tag_name = "catalog-test"
self._git(web_root, "tag", "-a", tag_name, "-m", "publication")
tag_object = self._git(
web_root, "rev-parse", f"refs/tags/{tag_name}"
).strip()
self._git(
web_root,
"push",
"--quiet",
"--atomic",
"origin",
f"{commit_sha}:refs/heads/main",
f"{tag_object}:refs/tags/{tag_name}",
)
identity = remote_publication_identity(
web_root=web_root,
remote_url=frozen.url,
branch="main",
tag_name=tag_name,
)
verified = verify_remote_publication(
web_root=web_root,
remote_url=frozen.url,
branch="main",
tag_name=tag_name,
expected_commit=commit_sha,
expected_tag_object=tag_object,
)
self._git(
web_root,
"remote",
"set-url",
"--add",
"--push",
"origin",
str(root / "other.git"),
)
with self.assertRaisesRegex(RuntimeError, "do not match"):
bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
self.assertEqual(identity, verified)
self.assertEqual(commit_sha, identity["publication_commit_sha"])
self.assertEqual(tag_object, identity["publication_tag_object_sha"])
self.assertEqual(commit_sha, identity["publication_tag_commit_sha"])
def test_mutation_rejects_writable_website_and_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
web_root = root / "website"
web_root.mkdir()
self._git(web_root, "init", "--quiet")
candidate = self._candidate(root, key="trusted-key")
target_root = web_root / "public" / "catalogs" / "v1"
target_catalog = target_root / "channels" / "stable.json"
target_keyring = target_root / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text("{}\n", encoding="utf-8")
web_root.chmod(0o777)
root_issues = publication_mutation_trust_issues(
web_root=web_root,
candidate_catalog=candidate / "channels" / "stable.json",
candidate_keyring=candidate / "keyring.json",
target_catalog=target_catalog,
target_keyring=target_keyring,
target_modules=target_root / "modules",
)
web_root.chmod(0o755)
config = web_root / ".git" / "config"
config.chmod(0o666)
config_issues = publication_mutation_trust_issues(
web_root=web_root,
candidate_catalog=candidate / "channels" / "stable.json",
candidate_keyring=candidate / "keyring.json",
target_catalog=target_catalog,
target_keyring=target_keyring,
target_modules=target_root / "modules",
)
self.assertIn("website root is writable by another user", root_issues)
self.assertIn("website Git config is writable by another user", config_issues)
def test_push_returns_only_independently_verified_publication_receipt(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog["sequence"] = 7
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
],
"artifacts": [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-core",
"package_version": "1.2.3",
"archive_sha256": "c" * 64,
"archive_size": 100,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "d" * 64,
"file_count": 1,
},
"requires_installer_receipt": True,
}
],
}
catalog["signatures"] = [{}]
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
remote_root = root / "website.git"
remote_root.mkdir()
self._git(remote_root, "init", "--bare", "--quiet")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "branch", "-M", "main")
self._git(web_root, "remote", "add", "origin", str(remote_root))
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
self._git(web_root, "push", "--quiet", "-u", "origin", "main")
frozen_head = self._git(web_root, "rev-parse", "HEAD").strip()
frozen_remote = bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.publication_runtime_trust_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.registered_website_remote",
return_value=str(remote_root),
),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
push=True,
branch="main",
tag_name="catalog-test",
expected_website_head=frozen_head,
expected_website_branch="main",
expected_remote_sha256=frozen_remote.sha256,
)
remote_identity = remote_publication_identity(
web_root=web_root,
remote_url=str(remote_root),
branch="main",
tag_name="catalog-test",
)
self.assertEqual("published", result.status)
self.assertEqual("origin", result.remote)
self.assertEqual(
remote_identity["publication_commit_sha"], result.publication_commit_sha
)
self.assertEqual(
remote_identity["publication_tag_object_sha"],
result.publication_tag_object_sha,
)
self.assertEqual(
remote_identity["publication_tag_commit_sha"],
result.publication_tag_commit_sha,
)
def test_apply_writes_validated_objects_even_if_candidate_path_changes(
self,
) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog.update({"channel": "stable", "sequence": 1})
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
],
"artifacts": [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-core",
"package_version": "1.2.3",
"archive_sha256": "c" * 64,
"archive_size": 100,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "d" * 64,
"file_count": 1,
},
"requires_installer_receipt": True,
}
],
}
catalog["signatures"] = [{}]
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
registered_remote = "ssh://example.test/release/website.git"
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "remote", "add", "origin", registered_remote)
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
def validate_and_swap(*args, **kwargs):
del args, kwargs
catalog_path.write_text(
json.dumps(
{
"channel": "stable",
"sequence": 999,
"malicious": True,
"signatures": [{}],
}
),
encoding="utf-8",
)
return {"valid": True, "warnings": [], "error": None}
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
side_effect=validate_and_swap,
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch("govoplan_release.publisher.website_dirty", return_value=False),
patch(
"govoplan_release.publisher.publication_runtime_trust_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.registered_website_remote",
return_value=registered_remote,
),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
allow_dirty_website=True,
)
published = json.loads(
(
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
).read_text(encoding="utf-8")
)
self.assertEqual("applied", result.status)
self.assertEqual(1, published["sequence"])
self.assertNotIn("malicious", published)
def test_apply_blocks_selected_python_release_without_built_identity(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
]
}
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
(web_root / ".git").mkdir()
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch("govoplan_release.publisher.website_dirty", return_value=False),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
)
self.assertEqual("blocked", result.status)
self.assertIn(
"selected Python repository govoplan-core has no built artifact identity",
" ".join(result.notes),
)
def test_publication_requires_an_existing_trust_anchor(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="candidate-key")
web_root = root / "website"
web_root.mkdir()
with patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
)
self.assertEqual("blocked", result.status)
self.assertIn("publication trust anchor is missing", " ".join(result.notes))
def test_publication_rejects_rebinding_an_existing_key_id(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="replacement-key")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
with patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
)
self.assertEqual("blocked", result.status)
self.assertIn("changes the public key", " ".join(result.notes))
@staticmethod
def _candidate(root: Path, *, key: str) -> Path:
candidate = root / "candidate"
channel = candidate / "channels"
channel.mkdir(parents=True)
channel.joinpath("stable.json").write_text(
json.dumps(
{
"channel": "stable",
"core_release": {
"version": "1.2.3",
"python_ref": (
"govoplan-core @ git+ssh://git@example.test/acme/"
"govoplan-core.git@v1.2.3"
),
},
"modules": [],
}
)
)
candidate.joinpath("keyring.json").write_text(
json.dumps(ReleaseCatalogPublicationTests._keyring(key))
)
return candidate
@staticmethod
def _keyring(key: str) -> dict[str, object]:
return {
"keys": [
{
"key_id": "release-key",
"status": "active",
"public_key": key,
}
]
}
@staticmethod
def _git(root: Path, *args: str) -> str:
return subprocess.run(
["git", *args],
cwd=root,
check=True,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
).stdout
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,66 @@
from __future__ import annotations
import subprocess
import sys
import unittest
from pathlib import Path
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from server.app import create_app # noqa: E402
class ReleaseConsoleSecurityTests(unittest.TestCase):
def test_api_token_is_accepted_only_from_header(self) -> None:
with TestClient(create_app(workspace_root=META_ROOT, token="test-console-token")) as client:
query_response = client.get("/api/health?token=test-console-token")
header_response = client.get(
"/api/health",
headers={"X-Release-Console-Token": "test-console-token"},
)
self.assertEqual(query_response.status_code, 401)
self.assertEqual(header_response.status_code, 200)
def test_bootstrap_token_uses_and_clears_url_fragment(self) -> None:
launcher = (RELEASE_ROOT / "release-console.py").read_text(encoding="utf-8")
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
self.assertIn("#token={token}", launcher)
self.assertNotIn("?token={token}", launcher)
self.assertIn("window.location.hash.slice(1)", webui)
self.assertIn("history.replaceState", webui)
def test_tokenless_embedding_is_read_only(self) -> None:
with TestClient(create_app(workspace_root=META_ROOT)) as client:
read_response = client.get("/api/health")
write_response = client.post("/api/selective-plan", json={})
self.assertEqual(200, read_response.status_code)
self.assertEqual(403, write_response.status_code)
self.assertIn("read-only", write_response.json()["detail"])
def test_launcher_rejects_non_loopback_and_tokenless_modes(self) -> None:
launcher = RELEASE_ROOT / "release-console.py"
for arguments in (("--host", "0.0.0.0"), ("--no-token",)):
with self.subTest(arguments=arguments):
result = subprocess.run(
(sys.executable, str(launcher), *arguments),
cwd=META_ROOT,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=10,
)
self.assertEqual(2, result.returncode)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,177 @@
from __future__ import annotations
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from server.app import create_app # noqa: E402
class ReleaseDashboardDiagnosticsTests(unittest.TestCase):
def test_malformed_core_version_is_bounded_and_blocks_api_plans(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
core = workspace / "govoplan-core"
core.mkdir()
(core / "pyproject.toml").write_text("[project\n", encoding="utf-8")
with TestClient(create_app(workspace_root=workspace)) as client:
dashboard_response = client.get(
"/api/dashboard", params={"public_catalog": "false"}
)
selective_response = client.get(
"/api/selective-plan",
params={
"repos": "govoplan-core",
"repo_versions": "govoplan-core:1.2.4",
"public_catalog": "false",
},
)
full_plan_response = client.get(
"/api/plan", params={"public_catalog": "false"}
)
self.assertEqual(200, dashboard_response.status_code)
dashboard = dashboard_response.json()
self.assertEqual("blocked", dashboard["summary"]["status"])
error = next(
item
for item in dashboard["collection_errors"]
if item["code"] == "core_version_metadata_unreadable"
)
self.assertEqual("govoplan-core/pyproject.toml", error["source"])
self.assertIn("TOMLDecodeError", error["message"])
self.assertIn("Repair govoplan-core/pyproject.toml", error["remediation"])
self.assertLess(len(error["message"]), 200)
self.assertNotIn(str(workspace), error["message"])
self.assertEqual(200, selective_response.status_code)
selective_plan = selective_response.json()
self.assertEqual("blocked", selective_plan["status"])
self.assertFalse(selective_plan["source_preflight_ready"])
finding = next(
item
for item in selective_plan["gate_findings"]
if item["code"] == "core_version_metadata_unreadable"
)
self.assertEqual(error["remediation"], finding["remediation"])
self.assertEqual(
"resolve_release_gate", selective_plan["recommended_action"]["id"]
)
self.assertEqual(200, full_plan_response.status_code)
full_plan = full_plan_response.json()
self.assertEqual("blocked", full_plan["status"])
self.assertTrue(
any(
action["id"].startswith("dashboard:core_version_metadata_unreadable:")
for action in full_plan["actions"]
)
)
def test_malformed_manifest_is_not_silently_omitted_from_selected_plan(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
core = workspace / "govoplan-core"
core.mkdir()
(core / "pyproject.toml").write_text(
'[project]\nname = "govoplan-core"\nversion = "0.1.13"\n',
encoding="utf-8",
)
files = workspace / "govoplan-files"
manifest = files / "src" / "govoplan_files" / "backend" / "manifest.py"
manifest.parent.mkdir(parents=True)
(files / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
manifest.write_text("def broken(:\n", encoding="utf-8")
initialize_repository(files)
with TestClient(create_app(workspace_root=workspace)) as client:
dashboard_response = client.get(
"/api/dashboard", params={"public_catalog": "false"}
)
selective_response = client.get(
"/api/selective-plan",
params={
"repos": "govoplan-files",
"repo_versions": "govoplan-files:1.2.4",
"public_catalog": "false",
},
)
self.assertEqual(200, dashboard_response.status_code)
dashboard = dashboard_response.json()
error = next(
item
for item in dashboard["collection_errors"]
if item["code"] == "module_contract_unreadable"
)
self.assertEqual("govoplan-files", error["repo"])
self.assertEqual(
"govoplan-files/src/govoplan_files/backend/manifest.py",
error["source"],
)
self.assertIn("SyntaxError", error["message"])
self.assertIn("Repair the manifest", error["remediation"])
self.assertEqual(200, selective_response.status_code)
plan = selective_response.json()
self.assertEqual("blocked", plan["status"])
self.assertFalse(plan["source_preflight_ready"])
self.assertIn(
"module_contract_unreadable",
{finding["code"] for finding in plan["gate_findings"]},
)
self.assertEqual("resolve_release_gate", plan["recommended_action"]["id"])
def initialize_repository(path: Path) -> None:
subprocess.run(
["git", "init", "--initial-branch=main", str(path)],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
subprocess.run(
["git", "-C", str(path), "add", "."],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
subprocess.run(
[
"git",
"-C",
str(path),
"-c",
"user.name=Release Test",
"-c",
"user.email=release@example.invalid",
"-c",
"commit.gpgsign=false",
"commit",
"-m",
"Initial fixture",
],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,207 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
SCRIPT = RELEASE_TOOLS_ROOT / "release-doctor.py"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
def load_doctor_module():
spec = importlib.util.spec_from_file_location("release_doctor", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class ReleaseDoctorTests(unittest.TestCase):
def test_release_repo_names_include_catalog_and_migration_baseline_owners(self) -> None:
doctor = load_doctor_module()
names = set(doctor.release_repo_names(META_ROOT.parent))
self.assertIn("govoplan-core", names)
self.assertIn("govoplan-files", names)
self.assertIn("govoplan-idm", names)
def test_repo_findings_detect_dirty_repositories(self) -> None:
doctor = load_doctor_module()
repo = doctor.RepoState(
name="govoplan-files",
path="/tmp/govoplan-files",
exists=True,
dirty_entries=(" M pyproject.toml",),
pyproject_version="0.1.7",
local_tag_exists=True,
)
findings = doctor.repo_findings((repo,), target_version="0.1.7", target_tag="v0.1.7")
self.assertEqual("blocker", findings[0].severity)
self.assertEqual("repo-dirty", findings[0].check_id)
self.assertIn("git status --short", [item.command for item in findings[0].commands])
def test_repo_version_mismatch_blocks_release(self) -> None:
doctor = load_doctor_module()
repo = doctor.RepoState(
name="govoplan-files",
path="/tmp/govoplan-files",
exists=True,
pyproject_version="0.1.9",
package_version="0.1.8",
webui_package_version="0.1.9",
)
findings = doctor.repo_findings((repo,), target_version="0.1.9", target_tag="v0.1.9")
mismatch = next(item for item in findings if item.check_id == "repo-version-mismatch")
self.assertEqual("blocker", mismatch.severity)
def test_dubious_ownership_errors_suggest_safe_directory_command(self) -> None:
doctor = load_doctor_module()
repo = doctor.RepoState(
name="govoplan-files",
path="/tmp/govoplan-files",
exists=True,
safe_directory_required=True,
safe_directory_command="git config --global --add safe.directory /tmp/govoplan-files",
errors=("fatal: detected dubious ownership in repository",),
)
findings = doctor.repo_findings((repo,), target_version="0.1.7", target_tag="v0.1.7")
self.assertEqual("repo-safe-directory", findings[0].check_id)
self.assertEqual("blocker", findings[0].severity)
self.assertTrue(findings[0].commands[0].mutating)
self.assertIn("safe.directory", findings[0].commands[0].command)
def test_dubious_ownership_detection_matches_git_error(self) -> None:
doctor = load_doctor_module()
self.assertTrue(
doctor.is_dubious_ownership_error(
"fatal: detected dubious ownership in repository at '/workspace/repo'\n"
"git config --global --add safe.directory /workspace/repo"
)
)
def test_release_migration_strict_errors_suggest_recording_baseline(self) -> None:
doctor = load_doctor_module()
findings = doctor.migration_findings(
{
"release": {
"ok": True,
"graph_errors": [],
"strict_errors": ["current migration heads are not recorded"],
},
"dev": {"ok": True, "graph_errors": [], "strict_errors": []},
},
target_version="0.2.0",
)
self.assertEqual("migration-release-baseline", findings[0].check_id)
self.assertEqual("blocker", findings[0].severity)
commands = [item.command for item in findings[0].commands]
self.assertTrue(any("--record-release 0.2.0" in item for item in commands))
def test_catalog_findings_detect_missing_signatures(self) -> None:
doctor = load_doctor_module()
findings = doctor.catalog_findings(
{
"catalog_exists": True,
"catalog_path": "/tmp/stable.json",
"core_release_version": "0.1.7",
"signature_count": 0,
"keyring_exists": True,
"key_count": 1,
},
target_version="0.1.7",
online=False,
)
self.assertEqual("catalog-signatures", findings[0].check_id)
self.assertEqual("blocker", findings[0].severity)
def test_suggested_commands_are_deduplicated(self) -> None:
doctor = load_doctor_module()
suggested = doctor.SuggestedCommand(title="Status", command="git status --short", cwd="/tmp/repo")
report = doctor.DoctorReport(
generated_at="2026-07-11T00:00:00Z",
workspace_root="/tmp",
target_version="0.1.7",
target_tag="v0.1.7",
online=False,
overall_status="blocked",
repos=(),
findings=(
doctor.Finding("a", "blocker", "A", "", (suggested,)),
doctor.Finding("b", "warning", "B", "", (suggested,)),
),
)
self.assertEqual((suggested,), doctor.collect_suggested_commands(report))
def test_default_report_is_concise_and_keeps_details_out(self) -> None:
doctor = load_doctor_module()
repo = doctor.RepoState(
name="govoplan-files",
path="/tmp/govoplan-files",
exists=True,
dirty_entries=(" M pyproject.toml",),
)
report = doctor.DoctorReport(
generated_at="2026-07-11T00:00:00Z",
workspace_root="/tmp",
target_version="0.1.7",
target_tag="v0.1.7",
online=False,
overall_status="blocked",
repos=(repo,),
findings=(doctor.Finding("repo-dirty", "blocker", "govoplan-files has uncommitted changes", "M pyproject.toml"),),
)
concise = doctor.render_text_report(report, detailed=False, include_commands=False)
detailed = doctor.render_text_report(report, detailed=True, include_commands=False)
self.assertIn("dirty=1", concise)
self.assertNotIn("M pyproject.toml", concise)
self.assertIn("M pyproject.toml", detailed)
def test_interactive_actions_offer_dirty_bulk_commit_push(self) -> None:
doctor = load_doctor_module()
repo = doctor.RepoState(
name="govoplan-files",
path="/tmp/govoplan-files",
exists=True,
dirty_entries=(" M pyproject.toml",),
)
report = doctor.DoctorReport(
generated_at="2026-07-11T00:00:00Z",
workspace_root="/tmp",
target_version="0.1.7",
target_tag="v0.1.7",
online=False,
overall_status="blocked",
repos=(repo,),
findings=(),
)
actions = dict(doctor.interactive_actions(report))
self.assertIn("commit-push-dirty", actions)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,71 @@
from __future__ import annotations
import json
from pathlib import Path
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
class ReleaseEntrypointGateTests(unittest.TestCase):
def test_lockstep_release_includes_every_registered_module_and_connector(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
registry = json.loads((META_ROOT / "repositories.json").read_text())
expected = {
item["name"]
for item in registry["repositories"]
if item["category"] not in {"system", "website"}
}
missing = sorted(repo for repo in expected if f'$PARENT/{repo}"' not in script)
self.assertEqual([], missing)
def test_lockstep_release_runs_source_and_full_gates_before_remote_push(self) -> None:
script = (META_ROOT / "tools" / "release" / "push-release-tag.sh").read_text()
manifest_gate = script.index("run_manifest_shape_gate\n\nconfirm_release")
confirm = script.index("\nconfirm_release\n", manifest_gate)
workflow = script[confirm:]
source_gate = workflow.index("run_version_alignment_gate source")
first_commit = workflow.index('run git -C "$repo" commit')
lock_generation = workflow.index("generate_release_lock")
full_gate = workflow.index("run_version_alignment_gate", source_gate + 1)
first_push = workflow.index('run git -C "$repo" push')
self.assertLess(source_gate, first_commit)
self.assertLess(first_commit, lock_generation)
self.assertLess(lock_generation, full_gate)
self.assertLess(full_gate, first_push)
self.assertLess(manifest_gate, confirm)
def test_source_catalog_generator_enforces_explicit_repo_versions(self) -> None:
script = (META_ROOT / "tools" / "release" / "generate-release-catalog.py").read_text()
gate = script.index("selected_repository_version_issues(")
write = script.index("output.write_text(")
self.assertLess(gate, write)
def test_candidate_publication_uses_existing_keyring_as_trust_anchor(self) -> None:
publisher = (META_ROOT / "tools" / "release" / "govoplan_release" / "publisher.py").read_text()
self.assertIn("candidate_catalog_version_issues(candidate_payload)", publisher)
self.assertIn("trusted_keys_from_keyring(\n target_keyring_payload", publisher)
self.assertIn("trusted_keys=publication_trust", publisher)
self.assertNotIn("trusted_keys=candidate_keys", publisher)
self.assertIn("module_directory_payloads(", publisher)
self.assertIn("verify_committed_publication(", publisher)
self.assertNotIn("shutil.copytree(candidate_modules", publisher)
def test_release_integration_honors_independent_module_versions(self) -> None:
script = (META_ROOT / "tools" / "checks" / "check-release-integration.sh").read_text()
self.assertNotIn('expected_tag = os.environ["RELEASE_TAG"]', script)
self.assertIn('dependency_version = tag_match.group(1)', script)
self.assertIn('repo_tag="$(release_tag_for_package "$repo")"', script)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,956 @@
from __future__ import annotations
import json
from pathlib import Path
import os
import subprocess
import sys
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.model import RepositorySpec # noqa: E402
from govoplan_release.git_state import sanitized_git_environment # noqa: E402
from govoplan_release.release_execution import ( # noqa: E402
ExecutorSpec,
ReleaseExecutionAmbiguous,
ReleaseExecutionBlocked,
build_selected_wheels,
execute_repository_step,
executor_spec,
reconciled_repository_receipt,
repository_state_receipt,
require_trusted_release_runtime,
verify_frozen_repository_tag_receipt,
verify_repository_preflight_binding,
_clone_catalog_sources,
_checkout_frozen_source,
_flatpak_proxy_environment,
_run_isolated_wheel_builder,
_trusted_flatpak_builder_proxy,
_trusted_host_bubblewrap,
_verify_exact_publication_delta,
isolated_builder_launcher,
)
from govoplan_release.selective_catalog import ( # noqa: E402
enforce_frozen_selected_source_provenance,
enforce_selected_source_provenance,
)
class ReleaseExecutionTests(unittest.TestCase):
def test_flatpak_proxy_environment_drops_unrelated_values(self) -> None:
with patch.dict(
os.environ,
{
"DBUS_SESSION_BUS_ADDRESS": "unix:path=/attacker/bus",
"XDG_RUNTIME_DIR": "/run/user/1000",
"PYTHONPATH": "/attacker",
"GIT_DIR": "/attacker/repository",
},
clear=True,
):
environment = _flatpak_proxy_environment()
self.assertEqual("/usr/bin:/bin", environment["PATH"])
self.assertEqual(
"unix:path=/run/flatpak/bus",
environment["DBUS_SESSION_BUS_ADDRESS"],
)
self.assertNotIn("XDG_RUNTIME_DIR", environment)
self.assertNotIn("PYTHONPATH", environment)
self.assertNotIn("GIT_DIR", environment)
def test_host_bubblewrap_preflight_requires_root_owned_safe_binary(self) -> None:
with patch(
"govoplan_release.release_execution.subprocess.run",
return_value=SimpleNamespace(
returncode=0,
stdout="81ed|0\n",
),
):
self.assertTrue(_trusted_host_bubblewrap())
with patch(
"govoplan_release.release_execution.subprocess.run",
return_value=SimpleNamespace(
returncode=0,
stdout="81ff|1000\n",
),
):
self.assertFalse(_trusted_host_bubblewrap())
def test_builder_launcher_uses_only_preflighted_flatpak_proxy(self) -> None:
with patch(
"govoplan_release.release_execution._trusted_flatpak_builder_proxy",
return_value=True,
):
self.assertEqual(
("/usr/bin/flatpak-spawn", "--host", "/usr/bin/bwrap"),
isolated_builder_launcher(),
)
def test_flatpak_proxy_requires_recognized_container(self) -> None:
with patch.dict(os.environ, {}, clear=True):
self.assertFalse(_trusted_flatpak_builder_proxy())
def test_flatpak_builder_launch_uses_sanitized_environment(self) -> None:
process = SimpleNamespace(wait=lambda timeout: 0)
command = (
"/usr/bin/flatpak-spawn",
"--host",
"/usr/bin/bwrap",
)
with (
patch.dict(
os.environ,
{
"DBUS_SESSION_BUS_ADDRESS": "unix:path=/attacker/bus",
"PYTHONPATH": "/attacker",
},
clear=True,
),
patch(
"govoplan_release.release_execution.subprocess.Popen",
return_value=process,
) as popen,
):
self.assertEqual(0, _run_isolated_wheel_builder(command))
environment = popen.call_args.kwargs["env"]
self.assertEqual(
"unix:path=/run/flatpak/bus",
environment["DBUS_SESSION_BUS_ADDRESS"],
)
self.assertNotIn("PYTHONPATH", environment)
def test_git_environment_ignores_inherited_redirection_and_commands(self) -> None:
with patch.dict(
"os.environ",
{
"GIT_DIR": "/attacker/repository",
"GIT_CONFIG_GLOBAL": "/attacker/config",
"GIT_SSH_COMMAND": "/attacker/ssh",
"PATH": "/attacker/bin",
"SSH_AUTH_SOCK": "/operator/agent.sock",
},
clear=False,
):
environment = sanitized_git_environment()
self.assertNotIn("GIT_DIR", environment)
self.assertEqual("/dev/null", environment["GIT_CONFIG_GLOBAL"])
self.assertEqual("/usr/bin:/bin", environment["PATH"])
self.assertEqual(
"/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8",
environment["GIT_SSH_COMMAND"],
)
self.assertEqual("/operator/agent.sock", environment["SSH_AUTH_SOCK"])
def test_publication_reconciliation_rejects_unrelated_commit_delta(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
website = Path(temp_dir) / "website"
_git(Path(temp_dir), "init", "-b", "main", str(website))
_git(website, "config", "user.name", "Release Test")
_git(website, "config", "user.email", "release@example.test")
catalog = website / "public/catalogs/v1/channels/stable.json"
keyring = website / "public/catalogs/v1/keyring.json"
modules = website / "public/catalogs/v1/modules"
catalog.parent.mkdir(parents=True)
modules.mkdir(parents=True)
catalog.write_text("{}\n", encoding="utf-8")
keyring.write_text("{}\n", encoding="utf-8")
(website / "README.md").write_text("before\n", encoding="utf-8")
_git(website, "add", ".")
_git(website, "commit", "-m", "Base")
parent = _git_output(website, "rev-parse", "HEAD")
expected = {
"public/catalogs/v1/channels/stable.json": b'{"new": true}\n',
"public/catalogs/v1/keyring.json": b"{}\n",
}
catalog.write_bytes(expected["public/catalogs/v1/channels/stable.json"])
(website / "README.md").write_text("unrelated\n", encoding="utf-8")
_git(website, "add", ".")
_git(website, "commit", "-m", "Catalog plus unrelated change")
commit = _git_output(website, "rev-parse", "HEAD")
with self.assertRaisesRegex(
ReleaseExecutionBlocked,
"outside the exact derived delta",
):
_verify_exact_publication_delta(
web_root=website,
frozen_parent=parent,
commit=commit,
expected_blobs=expected,
module_root=modules,
)
def test_runtime_trust_rejects_replaceable_workspace_boundary(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
meta = root / "govoplan"
workspace = root / "workspace"
_git(root, "init", "-b", "main", str(meta))
(meta / "tools" / "release").mkdir(parents=True)
(meta / "tools" / "checks").mkdir(parents=True)
(meta / "tools" / "release" / "tool.py").write_text(
"# trusted\n", encoding="utf-8"
)
(meta / "tools" / "checks" / "check.py").write_text(
"# trusted\n", encoding="utf-8"
)
registry = meta / "repositories.json"
registry.write_text('{"repositories": []}\n', encoding="utf-8")
workspace.mkdir(mode=0o700)
with (
patch(
"govoplan_release.release_execution.META_ROOT",
meta,
),
patch(
"govoplan_release.release_execution.REPOSITORIES_FILE",
registry,
),
patch(
"govoplan_release.release_execution.publication_runtime_trust_issues",
return_value=(),
),
):
require_trusted_release_runtime(workspace_root=workspace)
workspace.chmod(0o777)
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "operator-owned"
):
require_trusted_release_runtime(workspace_root=workspace)
def test_durable_binding_rejects_group_writable_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("release\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
(repository / ".git" / "config").chmod(0o664)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(_repository_spec(remote),),
):
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "group/world writable"
):
repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
def test_durable_binding_rejects_nested_writable_git_authority(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("release\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
(repository / ".git" / "refs" / "heads").chmod(0o777)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(_repository_spec(remote),),
):
with self.assertRaisesRegex(
ReleaseExecutionBlocked,
"Git metadata directory",
):
repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
def test_creation_binding_rejects_head_and_push_remote_drift(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("one\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "-u", "origin", "main")
spec = _repository_spec(remote)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(spec,),
):
frozen = repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
step = {
"repo": "govoplan-files",
"source_binding": frozen,
}
(repository / "README.md").write_text("two\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Drift")
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "state drifted"
):
verify_repository_preflight_binding(
plan_step=step,
version="1.2.3",
workspace_root=workspace,
)
_git(
repository,
"remote",
"set-url",
"--add",
"--push",
"origin",
str(workspace / "other.git"),
)
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "registered release remote"
):
repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
def test_reconciliation_proves_annotated_local_and_remote_tag(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("release\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "-u", "origin", "main")
spec = _repository_spec(remote)
plan_step = {
"repo": "govoplan-files",
"source_binding": {},
}
tag_spec = ExecutorSpec("tag", "TAG", "tag_created", True)
push_spec = ExecutorSpec("push", "PUBLISH", "tag_published", True)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(spec,),
):
before = repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
_git(repository, "tag", "-a", "v1.2.3", "-m", "Release")
tagged = reconciled_repository_receipt(
spec=tag_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=before,
)
self.assertIsNotNone(tagged["tag_object"])
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "Remote annotated tag"
):
reconciled_repository_receipt(
spec=push_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=tagged,
)
_git(
repository,
"push",
"--atomic",
"origin",
"HEAD:refs/heads/main",
"refs/tags/v1.2.3",
)
published = reconciled_repository_receipt(
spec=push_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=tagged,
)
self.assertEqual(tagged, published)
(repository / "README.md").write_text(
"uncommitted drift\n", encoding="utf-8"
)
self.assertEqual(
tagged,
verify_frozen_repository_tag_receipt(
receipt=tagged,
workspace_root=workspace,
),
)
(repository / "README.md").write_text(
"remote branch drift\n", encoding="utf-8"
)
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Remote branch drift")
drift_commit = _git_output(repository, "rev-parse", "HEAD")
_git(
repository,
"push",
"origin",
f"{drift_commit}:refs/heads/drift-source",
)
_git(remote, "update-ref", "refs/heads/main", drift_commit)
_git(repository, "reset", "--hard", str(tagged["head"]))
with self.assertRaisesRegex(
ReleaseExecutionBlocked,
"Remote branch",
):
reconciled_repository_receipt(
spec=push_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=tagged,
)
def test_post_tag_receipt_failure_is_ambiguous(self) -> None:
spec = ExecutorSpec("tag", "TAG", "tag_created", True)
plan_step = {
"id": "govoplan-files:tag",
"repo": "govoplan-files",
"source_binding": {"kind": "repository_state"},
}
with (
patch(
"govoplan_release.release_execution.require_trusted_release_runtime"
),
patch(
"govoplan_release.release_execution.verify_repository_step_precondition"
),
patch(
"govoplan_release.release_execution.tag_repositories",
return_value={
"status": "tagged",
"repositories": [{"status": "tagged"}],
},
),
patch(
"govoplan_release.release_execution.repository_state_receipt",
side_effect=ReleaseExecutionBlocked("post-effect probe failed"),
),
):
with self.assertRaisesRegex(
ReleaseExecutionAmbiguous, "post-effect repository receipt"
):
execute_repository_step(
spec=spec,
plan_step=plan_step,
repo_versions={"govoplan-files": "1.2.3"},
workspace_root=Path("/workspace"),
remote="origin",
expected_receipt={"kind": "repository_state"},
)
def test_commit_step_has_no_durable_executor(self) -> None:
self.assertIsNone(
executor_spec(
{
"id": "govoplan-files:commit",
"status": "planned",
"mutating": True,
"repo": "govoplan-files",
"source_binding": {
"kind": "repository_state",
},
}
)
)
def test_frozen_checkout_uses_receipt_commit_not_live_worktree(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
repository.mkdir()
_git(repository, "init", "-b", "main")
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "version.txt").write_text("frozen\n", encoding="utf-8")
_git(repository, "add", "version.txt")
_git(repository, "commit", "-m", "Frozen")
frozen_commit = _git_output(repository, "rev-parse", "HEAD")
_git(repository, "tag", "-a", "v1.2.3", "-m", "Release")
tag_object = _git_output(repository, "rev-parse", "refs/tags/v1.2.3")
(repository / "version.txt").write_text("live drift\n", encoding="utf-8")
source_root = workspace / "isolated"
source_root.mkdir()
checkout = _checkout_frozen_source(
repo="govoplan-files",
source_remote=repository,
commit=frozen_commit,
tag="v1.2.3",
tag_object=tag_object,
source_root=source_root,
)
self.assertEqual(
"frozen\n", (checkout / "version.txt").read_text(encoding="utf-8")
)
def test_frozen_provenance_accepts_only_exact_clean_detached_tag(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.3"\n',
encoding="utf-8",
)
_git(repository, "add", "pyproject.toml")
_git(repository, "commit", "-m", "Release")
commit = _git_output(repository, "rev-parse", "HEAD")
_git(repository, "tag", "-a", "v1.2.3", "-m", "Release")
tag_object = _git_output(
repository, "rev-parse", "refs/tags/v1.2.3"
)
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "origin", "main", "refs/tags/v1.2.3")
_git(repository, "checkout", "--detach", commit)
spec = _repository_spec(remote)
with (
patch(
"govoplan_release.source_provenance.load_repository_specs",
return_value=(spec,),
),
patch(
"govoplan_release.selective_catalog.load_repository_specs",
return_value=(spec,),
),
):
with self.assertRaisesRegex(ValueError, "named branch"):
enforce_selected_source_provenance(
repo_versions={"govoplan-files": "1.2.3"},
workspace=workspace,
)
observed = enforce_frozen_selected_source_provenance(
repo_versions={"govoplan-files": "1.2.3"},
expected_provenance={
"govoplan-files": {
"commit_sha": commit,
"tag_object_sha": tag_object,
}
},
workspace=workspace,
)
self.assertEqual(commit, observed["govoplan-files"]["commit_sha"])
self.assertEqual(
tag_object, observed["govoplan-files"]["tag_object_sha"]
)
def test_wheel_staging_skips_selected_tag_only_repository(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
python_repo = workspace / "govoplan-files"
tag_only_repo = workspace / "govoplan-meta-notes"
python_repo.mkdir()
tag_only_repo.mkdir()
(python_repo / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.3"\n',
encoding="utf-8",
)
_git(workspace, "init", "-b", "main", str(python_repo))
_git(python_repo, "config", "user.name", "Release Test")
_git(python_repo, "config", "user.email", "release@example.test")
_git(python_repo, "add", "pyproject.toml")
_git(python_repo, "commit", "-m", "Source")
specs = (
RepositorySpec(
name="govoplan-files",
category="module",
subtype="infrastructure",
remote="git@example.test/files.git",
path="govoplan-files",
),
RepositorySpec(
name="govoplan-meta-notes",
category="system",
subtype="metadata",
remote="git@example.test/notes.git",
path="govoplan-meta-notes",
),
)
candidate = workspace / "candidate"
def build(command: tuple[str, ...]) -> int:
wheel_dir = Path(command[command.index("--bind") + 1])
second_bind = command.index("--bind", command.index("--bind") + 1)
source_dir = Path(command[second_bind + 1])
(source_dir / "build-mutated.txt").write_text(
"isolated copy",
encoding="utf-8",
)
(
wheel_dir / "govoplan_files-1.2.3-py3-none-any.whl"
).write_bytes(b"wheel")
return 0
with (
patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=specs,
),
patch(
"govoplan_release.release_execution.isolated_builder_launcher",
return_value=("/usr/bin/bwrap",),
),
patch(
"govoplan_release.release_execution._run_isolated_wheel_builder",
side_effect=build,
) as runner,
patch(
"govoplan_release.release_execution.inspect_python_wheel",
return_value=SimpleNamespace(
package_name="govoplan-files",
package_version="1.2.3",
),
),
):
artifacts = build_selected_wheels(
repo_versions={
"govoplan-files": "1.2.3",
"govoplan-meta-notes": "1.0.0",
},
workspace_root=workspace,
candidate_output=candidate,
source_receipts={
"govoplan-files": {
"head": "a" * 40,
"target_tag": "v1.2.3",
"tag_object": "c" * 40,
},
"govoplan-meta-notes": {
"head": "b" * 40,
"target_tag": "v1.0.0",
"tag_object": "d" * 40,
},
},
frozen_sources={
"govoplan-files": python_repo,
"govoplan-meta-notes": tag_only_repo,
},
)
self.assertFalse((python_repo / "build-mutated.txt").exists())
self.assertEqual({"govoplan-files"}, set(artifacts))
self.assertEqual(1, runner.call_count)
command = runner.call_args.args[0]
self.assertNotIn(str(python_repo), command)
self.assertTrue(
any(
value.startswith(str(candidate / "artifacts"))
and value.endswith("/source")
for value in command
)
)
self.assertIn("--unshare-all", command)
self.assertIn("PIP_NO_INDEX", command)
self.assertNotIn(str(Path.home()), command)
def test_catalog_synthesis_clones_selected_and_unchanged_base_sources(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
files_repo, files_remote = _tagged_repository(
workspace,
repo="govoplan-files",
bare="files.git",
version="1.2.3",
)
core_repo, core_remote = _tagged_repository(
workspace,
repo="govoplan-core",
bare="core.git",
version="1.0.0",
)
specs = (
_repository_spec(files_remote),
RepositorySpec(
name="govoplan-core",
category="core",
subtype="core",
remote=str(core_remote),
path="govoplan-core",
),
)
destination = workspace / "synthesis"
destination.mkdir()
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=specs,
):
receipt = repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
cloned = _clone_catalog_sources(
source_versions={
"govoplan-core": "1.0.0",
"govoplan-files": "1.2.3",
},
repo_versions={"govoplan-files": "1.2.3"},
source_receipts={"govoplan-files": receipt},
workspace_root=workspace,
destination=destination,
)
self.assertEqual(
_git_output(files_repo, "rev-parse", "refs/tags/v1.2.3^{commit}"),
_git_output(cloned["govoplan-files"], "rev-parse", "HEAD"),
)
self.assertEqual(
_git_output(core_repo, "rev-parse", "refs/tags/v1.0.0^{commit}"),
_git_output(cloned["govoplan-core"], "rev-parse", "HEAD"),
)
self.assertEqual(
"HEAD",
_git_output(cloned["govoplan-core"], "rev-parse", "--abbrev-ref", "HEAD"),
)
def test_catalog_synthesis_adds_exact_core_bundle_dependency(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
idm_repo, idm_remote = _tagged_repository(
workspace,
repo="govoplan-idm",
bare="idm.git",
version="0.1.8",
)
idm_commit = _git_output(idm_repo, "rev-parse", "v0.1.8^{commit}")
(idm_repo / "pyproject.toml").write_text(
'[project]\nname = "govoplan-idm"\nversion = "0.1.9"\n',
encoding="utf-8",
)
_git(idm_repo, "add", "pyproject.toml")
_git(idm_repo, "commit", "-m", "Next release")
_git(idm_repo, "tag", "-a", "v0.1.9", "-m", "Next release")
_git(idm_repo, "push", "origin", "main", "refs/tags/v0.1.9")
catalog_commit = _git_output(idm_repo, "rev-parse", "v0.1.9^{commit}")
core_repo = workspace / "govoplan-core"
core_remote = workspace / "core.git"
_git(workspace, "init", "--bare", str(core_remote))
_git(workspace, "init", "-b", "main", str(core_repo))
_git(core_repo, "config", "user.name", "Release Test")
_git(core_repo, "config", "user.email", "release@example.test")
(core_repo / "webui").mkdir()
(core_repo / "pyproject.toml").write_text(
'[project]\nname = "govoplan-core"\nversion = "1.0.0"\n',
encoding="utf-8",
)
reference = (
"git+ssh://git@example.test/add-ideas/"
"govoplan-idm.git#v0.1.8"
)
(core_repo / "webui/package.release.json").write_text(
json.dumps(
{
"version": "1.0.0",
"dependencies": {"@govoplan/idm-webui": reference},
}
),
encoding="utf-8",
)
(core_repo / "webui/package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {
"version": "1.0.0",
"dependencies": {
"@govoplan/idm-webui": reference,
},
},
"node_modules/@govoplan/idm-webui": {
"version": "0.1.8",
"resolved": f"{reference.rsplit('#', 1)[0]}#{idm_commit}",
},
}
}
),
encoding="utf-8",
)
_git(core_repo, "add", ".")
_git(core_repo, "commit", "-m", "Release")
_git(core_repo, "tag", "-a", "v1.0.0", "-m", "Release")
_git(core_repo, "remote", "add", "origin", str(core_remote))
_git(core_repo, "push", "-u", "origin", "main", "refs/tags/v1.0.0")
specs = (
RepositorySpec(
name="govoplan-core",
category="system",
subtype="kernel",
remote=str(core_remote),
path="govoplan-core",
),
RepositorySpec(
name="govoplan-idm",
category="module",
subtype="platform",
remote=str(idm_remote),
path="govoplan-idm",
),
)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=specs,
):
synthesis = workspace / "synthesis"
synthesis.mkdir()
cloned = _clone_catalog_sources(
source_versions={"govoplan-core": "1.0.0"},
repo_versions={},
source_receipts={},
workspace_root=workspace,
destination=synthesis,
)
newer_synthesis = workspace / "newer-synthesis"
newer_synthesis.mkdir()
cloned_with_newer_catalog_source = _clone_catalog_sources(
source_versions={
"govoplan-core": "1.0.0",
"govoplan-idm": "0.1.9",
},
repo_versions={},
source_receipts={},
workspace_root=workspace,
destination=newer_synthesis,
)
self.assertEqual(
{"govoplan-core", "govoplan-idm"},
set(cloned),
)
self.assertEqual(
idm_commit,
_git_output(cloned["govoplan-idm"], "rev-parse", "HEAD"),
)
self.assertEqual(
catalog_commit,
_git_output(
cloned_with_newer_catalog_source["govoplan-idm"],
"rev-parse",
"HEAD",
),
)
self.assertEqual(
idm_commit,
_git_output(
cloned_with_newer_catalog_source["govoplan-idm"],
"rev-parse",
"v0.1.8^{commit}",
),
)
def _repository_spec(remote: Path) -> RepositorySpec:
return RepositorySpec(
name="govoplan-files",
category="module",
subtype="infrastructure",
remote=str(remote),
path="govoplan-files",
)
def _tagged_repository(
workspace: Path, *, repo: str, bare: str, version: str
) -> tuple[Path, Path]:
repository = workspace / repo
remote = workspace / bare
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "pyproject.toml").write_text(
f'[project]\nname = "{repo}"\nversion = "{version}"\n',
encoding="utf-8",
)
_git(repository, "add", "pyproject.toml")
_git(repository, "commit", "-m", "Release")
_git(repository, "tag", "-a", f"v{version}", "-m", "Release")
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "-u", "origin", "main", f"refs/tags/v{version}")
return repository, remote
def _git(cwd: Path, *arguments: str) -> None:
subprocess.run( # noqa: S603
("git", *arguments),
cwd=cwd,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
def _git_output(cwd: Path, *arguments: str) -> str:
return subprocess.run( # noqa: S603
("git", *arguments),
cwd=cwd,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
).stdout.strip()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,72 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.http_fetch import ( # noqa: E402
DEFAULT_MAX_RESPONSE_BYTES,
fetch_http,
)
class ReleaseHttpFetchTests(unittest.TestCase):
def test_content_length_over_default_limit_blocks_before_read(self) -> None:
response = FakeResponse(
body=b"ignored",
headers={"Content-Length": str(DEFAULT_MAX_RESPONSE_BYTES + 1)},
)
with mock.patch(
"govoplan_release.http_fetch.urllib.request.urlopen",
return_value=response,
):
with self.assertRaisesRegex(ValueError, "exceeds the configured size"):
fetch_http("https://example.invalid/catalog.json", timeout=1)
self.assertEqual([], response.read_sizes)
def test_streamed_body_is_capped_when_length_is_absent(self) -> None:
response = FakeResponse(body=b"x" * 9, headers={})
with mock.patch(
"govoplan_release.http_fetch.urllib.request.urlopen",
return_value=response,
):
with self.assertRaisesRegex(ValueError, "exceeds the configured size"):
fetch_http(
"https://example.invalid/catalog.json",
timeout=1,
max_response_bytes=8,
)
self.assertEqual([9], response.read_sizes)
class FakeResponse:
def __init__(self, *, body: bytes, headers: dict[str, str]) -> None:
self.status = 200
self.headers = headers
self.body = body
self.read_sizes: list[int] = []
def __enter__(self) -> FakeResponse:
return self
def __exit__(self, *args: object) -> None:
return None
def read(self, size: int) -> bytes:
self.read_sizes.append(size)
return self.body[:size]
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,104 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from tools.checks.release_integration import (
SOURCE_COUPLED_CORE_TESTS,
InstalledModuleArtifact,
artifact_contract_issues,
filter_test_suite,
release_package_names,
)
def _named_test(test_id: str) -> unittest.TestCase:
class NamedTest(unittest.TestCase):
def id(self) -> str:
return test_id
def runTest(self) -> None:
return None
return NamedTest()
class ReleaseIntegrationTests(unittest.TestCase):
def test_release_requirement_parser_selects_only_immutable_module_refs(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
requirements = Path(temp_dir) / "requirements-release.txt"
requirements.write_text(
"\n".join(
(
"../govoplan-core[server]",
"govoplan-idm @ git+ssh://git@example.test/add-ideas/govoplan-idm.git@v0.1.8",
"govoplan-campaign @ git+ssh://git@example.test/add-ideas/govoplan-campaign.git@v0.1.8",
"govoplan-idm @ git+ssh://git@example.test/add-ideas/govoplan-idm.git@v0.1.8",
"other-package==1.0",
)
),
encoding="utf-8",
)
self.assertEqual(
release_package_names(requirements),
("govoplan-idm", "govoplan-campaign"),
)
def test_artifact_contracts_compare_distribution_entry_point_and_discovery_versions(self) -> None:
artifacts = (
InstalledModuleArtifact("govoplan-access", "0.1.8", "access", "0.1.8"),
InstalledModuleArtifact("govoplan-idm", "0.1.8", "idm", "0.1.7"),
InstalledModuleArtifact("govoplan-shadow-idm", "0.1.8", "idm", "0.1.8"),
)
issues = artifact_contract_issues(
artifacts,
release_packages=("govoplan-access", "govoplan-idm", "govoplan-shadow-idm", "govoplan-campaign"),
available_manifest_versions={"access": "0.1.8", "idm": "0.1.6"},
)
self.assertEqual(len(issues), 5)
self.assertTrue(any("metadata version '0.1.8' does not match" in issue for issue in issues))
self.assertTrue(any("does not publish a govoplan.modules entry point" in issue for issue in issues))
self.assertTrue(any("multiple release packages" in issue for issue in issues))
def test_release_suite_excludes_only_named_source_coupled_tests(self) -> None:
retained_id = "tests.test_core_events.CoreEventTests.test_event_delivery"
nested = unittest.TestSuite(
(
unittest.TestSuite(_named_test(test_id) for test_id in SOURCE_COUPLED_CORE_TESTS),
unittest.TestSuite((_named_test(retained_id),)),
)
)
filtered, removed = filter_test_suite(nested, tuple(SOURCE_COUPLED_CORE_TESTS))
self.assertEqual(set(removed), set(SOURCE_COUPLED_CORE_TESTS))
self.assertEqual([test.id() for test in filtered], [retained_id])
self.assertEqual(len(SOURCE_COUPLED_CORE_TESTS), 4)
def test_release_entrypoint_uses_artifact_checks_and_filtered_core_suite(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
script = (meta_root / "tools" / "checks" / "check-release-integration.sh").read_text(encoding="utf-8")
artifact_check = script.index('"$META_ROOT/tools/checks/release_integration.py" artifacts')
core_tests = script.index('"$META_ROOT/tools/checks/release_integration.py" core-tests')
module_tests = script.index('run_step "Run cloned module backend tests"')
self.assertLess(artifact_check, core_tests)
self.assertLess(core_tests, module_tests)
self.assertNotIn('"$PYTHON" -m unittest \\\n tests.test_module_system', script)
def test_module_matrix_uses_artifact_aware_core_suite(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
script = (meta_root / "tools" / "checks" / "check-module-matrix.sh").read_text(encoding="utf-8")
self.assertIn('"$META_ROOT/tools/checks/release_integration.py" core-tests', script)
self.assertIn('--core-root "$ROOT"', script)
self.assertNotIn('"$PYTHON" -m unittest tests.test_module_system', script)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,190 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "release" / "release-migration-audit.py"
def load_audit_module():
spec = importlib.util.spec_from_file_location("release_migration_audit", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class ReleaseMigrationAuditTests(unittest.TestCase):
def test_typed_alembic_variables_are_parsed(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
path = Path(directory) / "1234_example.py"
path.write_text(
"""
from __future__ import annotations
from typing import Sequence, Union
revision: str = "1234"
down_revision: Union[str, None] = "base"
depends_on: Union[str, None] = "core"
branch_labels: Union[str, Sequence[str], None] = None
""",
encoding="utf-8",
)
migration = audit.parse_migration_file("govoplan-core", path)
self.assertIsNotNone(migration)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ("base",))
self.assertEqual(migration.depends_on, ("core",))
self.assertEqual(migration.branch_labels, ())
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-core", Path("base.py"), "base", (), (), ()),
audit.Migration("govoplan-core", Path("head.py"), "head", ("base",), (), ()),
]
report = audit.build_report(
migrations,
baseline={
"version": 1,
"releases": [
{
"release": "0.1.0",
"heads": [{"owner": "govoplan-core", "revision": "head"}],
}
],
},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
)
self.assertEqual(report["current_heads"], ["head"])
self.assertEqual(report["graph_errors"], [])
self.assertEqual(report["strict_errors"], [])
def test_owner_heads_are_accepted_for_subset_installs(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-core", Path("base.py"), "base", (), (), ()),
audit.Migration("govoplan-core", Path("core_head.py"), "core-head", ("base",), (), ()),
]
report = audit.build_report(
migrations,
baseline={
"version": 1,
"releases": [
{
"release": "0.1.0",
"heads": [{"owner": "govoplan-files", "revision": "files-head"}],
"owner_heads": [{"owner": "govoplan-core", "revisions": ["core-head"]}],
}
],
},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
)
self.assertEqual(report["current_heads"], ["core-head"])
self.assertEqual(report["strict_errors"], [])
def test_records_current_release_baseline(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-core", Path("base.py"), "base", (), (), ()),
audit.Migration("govoplan-core", Path("core_head.py"), "core-head", ("base",), (), ()),
audit.Migration("govoplan-files", Path("files_head.py"), "files-head", ("core-head",), (), ()),
]
report = audit.build_report(
migrations,
baseline={"version": 1, "releases": []},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
)
baseline = audit.record_release_baseline(
{"version": 1, "releases": []},
report,
release="0.2.0",
replace=False,
)
release = baseline["releases"][0]
self.assertEqual(release["release"], "0.2.0")
self.assertEqual(release["squash_policy"], "reviewed-manual")
self.assertEqual(release["heads"], [{"owner": "govoplan-files", "revision": "files-head"}])
self.assertIn({"owner": "govoplan-core", "revisions": ["core-head"]}, release["owner_heads"])
def test_missing_down_revision_is_a_graph_error(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-core", Path("head.py"), "head", ("missing",), (), ()),
]
report = audit.build_report(
migrations,
baseline={"version": 1, "releases": []},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
)
self.assertIn("references missing down_revision", report["graph_errors"][0])
def test_depends_on_participates_in_graph_validation_and_heads(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-core", Path("core.py"), "core", (), (), ()),
audit.Migration("govoplan-files", Path("files.py"), "files", (), ("core",), ()),
]
report = audit.build_report(
migrations,
baseline={"version": 1, "releases": []},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
)
self.assertEqual(report["graph_errors"], [])
self.assertEqual(report["current_heads"], ["files"])
def test_missing_depends_on_is_a_graph_error(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-files", Path("files.py"), "files", (), ("missing",), ()),
]
report = audit.build_report(
migrations,
baseline={"version": 1, "releases": []},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
)
self.assertIn("references missing depends_on", report["graph_errors"][0])
def test_dev_track_does_not_emit_release_baseline_warnings(self) -> None:
audit = load_audit_module()
migrations = [
audit.Migration("govoplan-core", Path("base.py"), "base", (), (), ()),
audit.Migration("govoplan-core", Path("head.py"), "head", ("base",), (), ()),
]
report = audit.build_report(
migrations,
baseline={"version": 1, "releases": []},
baseline_file=Path("docs/migration-release-baselines.json"),
workspace_root=Path("/workspace"),
track="dev",
)
self.assertEqual(report["track"], "dev")
self.assertEqual(report["strict_errors"], [])
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,82 @@
from __future__ import annotations
from pathlib import Path
import sys
import unittest
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.module_directory import ( # noqa: E402
module_directory_payloads,
safe_path_part,
)
class ReleaseModuleDirectoryTests(unittest.TestCase):
def test_signed_catalog_timestamp_makes_derived_files_reproducible(self) -> None:
catalog = {
"generated_at": "2026-07-22T12:00:00Z",
"sequence": 7,
"modules": [],
}
first = module_directory_payloads(
catalog_payload=catalog,
keyring_payload={},
channel="stable",
)
second = module_directory_payloads(
catalog_payload=catalog,
keyring_payload={},
channel="stable",
)
self.assertEqual(first, second)
self.assertEqual(
"2026-07-22T12:00:00Z",
first[-1][1]["generated_at"],
)
def test_dot_segments_and_noncanonical_ids_never_become_paths(self) -> None:
for value in (".", "..", "../escape", "/absolute", "module/child"):
with self.subTest(value=value), self.assertRaises(ValueError):
safe_path_part(value)
for module_id in ("..", "../escape", "UPPER", "bad.id"):
with self.subTest(module_id=module_id), mock.patch(
"govoplan_release.module_directory.module_rows",
return_value=[
{
"module_id": module_id,
"version": "1.2.3",
}
],
):
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={},
keyring_payload={},
channel="stable",
)
def test_noncanonical_version_and_channel_fail_before_paths(self) -> None:
with mock.patch(
"govoplan_release.module_directory.module_rows",
return_value=[{"module_id": "demo", "version": "../1.2.3"}],
):
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={}, keyring_payload={}, channel="stable"
)
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={}, keyring_payload={}, channel="../stable"
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,203 @@
from __future__ import annotations
import sys
import tempfile
import unittest
from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.model import ( # noqa: E402
CatalogSnapshot,
DashboardSummary,
ReleaseDashboard,
RepositorySnapshot,
RepositorySpec,
VersionSnapshot,
)
from govoplan_release.selective_planner import build_selective_release_plan # noqa: E402
class ReleasePlanGuidanceTests(unittest.TestCase):
def test_unprepared_target_has_structured_remediation_and_recommendation(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.3"\n',
encoding="utf-8",
)
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.3"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertEqual("blocked", plan.status)
self.assertFalse(plan.source_preflight_ready)
finding = next(
item
for item in plan.gate_findings
if item.code == "repository_version_alignment"
)
self.assertEqual("govoplan-files", finding.repo)
self.assertEqual("pyproject.toml", finding.source)
self.assertEqual("1.2.4", finding.expected)
self.assertEqual("1.2.3", finding.actual)
self.assertIn("Regenerate lockfiles", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
self.assertEqual("govoplan-files", plan.recommended_action.repo)
def test_aligned_target_recommends_non_mutating_tag_preview(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.4"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertTrue(plan.source_preflight_ready)
self.assertEqual("preview_source_release", plan.recommended_action.id)
self.assertIn("Preview Tag + Publish", plan.recommended_action.remediation)
def test_malformed_version_metadata_is_a_structured_blocker(self) -> None:
cases = (
("package.json", "{not-json\n", "JSONDecodeError"),
("pyproject.toml", "[project\n", "TOMLDecodeError"),
)
for relative_path, malformed, error_name in cases:
with self.subTest(relative_path=relative_path):
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
repo_path.mkdir()
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
(repo_path / relative_path).write_text(malformed, encoding="utf-8")
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.4"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertEqual("blocked", plan.status)
finding = next(
item
for item in plan.gate_findings
if item.code == "repository_version_metadata_unreadable"
)
self.assertEqual("govoplan-files", finding.repo)
self.assertIn(error_name, finding.message)
self.assertIn("Repair the malformed TOML or JSON", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
def test_malformed_webui_metadata_is_a_structured_blocker(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repo_path = workspace / "govoplan-files"
(repo_path / "webui").mkdir(parents=True)
(repo_path / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.4"\n',
encoding="utf-8",
)
(repo_path / "webui" / "package.json").write_text(
"{not-json\n", encoding="utf-8"
)
plan = build_selective_release_plan(
dashboard(workspace=workspace, version="1.2.4"),
selected_repos=("govoplan-files",),
repo_versions={"govoplan-files": "1.2.4"},
)
self.assertEqual("blocked", plan.status)
codes = {item.code for item in plan.gate_findings}
self.assertIn("repository_version_metadata_unreadable", codes)
finding = next(
item
for item in plan.gate_findings
if item.code == "release_webui_metadata_unreadable"
)
self.assertIn("JSONDecodeError", finding.message)
self.assertIn("Repair malformed JSON", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
def test_webui_renders_recommendation_and_remediation(self) -> None:
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
self.assertIn("plan.gate_findings", webui)
self.assertIn("plan.recommended_action", webui)
self.assertIn("recommended next", webui)
self.assertIn('plan.status === "blocked" ? "block"', webui)
self.assertIn('pill("recommended next", recommendationKind)', webui)
self.assertIn("<strong>Remediation:</strong>", webui)
def dashboard(*, workspace: Path, version: str) -> ReleaseDashboard:
repo = RepositorySnapshot(
spec=RepositorySpec(
name="govoplan-files",
category="module",
subtype="infrastructure",
remote="git@example.test:add-ideas/govoplan-files.git",
path="govoplan-files",
),
absolute_path=str(workspace / "govoplan-files"),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject=version),
local_target_tag_exists=False,
)
return ReleaseDashboard(
generated_at="2026-07-22T00:00:00Z",
meta_root=str(workspace / "govoplan"),
workspace_root=str(workspace),
target_version=None,
target_tag=None,
online=False,
include_migrations=False,
summary=DashboardSummary(
repository_count=1,
missing_count=0,
dirty_count=0,
ahead_count=0,
behind_count=0,
no_head_count=0,
error_count=0,
safe_directory_count=0,
local_target_tag_missing_count=0,
status="ready",
),
repositories=(repo,),
catalog=CatalogSnapshot(
channel="stable",
website_path="",
catalog_path="",
catalog_exists=False,
keyring_path="",
keyring_exists=False,
),
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,440 @@
from __future__ import annotations
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.repository_tag import tag_repositories # noqa: E402
from server.app import create_app # noqa: E402
class ReleaseRepositoryTagTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.workspace = self.root / "workspace"
self.workspace.mkdir()
self.repo, self.remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-core",
version="0.1.10",
)
self.manifest_repo, self.manifest_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-access",
version="0.1.10",
)
add_scoped_workflow_manifest(self.manifest_repo)
def tearDown(self) -> None:
self.temporary.cleanup()
def test_core_preview_is_non_mutating_and_publish_is_idempotent(self) -> None:
preview = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=False,
push=True,
)
self.assertEqual(preview["status"], "planned")
row = preview["repositories"][0]
self.assertEqual(row["status"], "planned")
self.assertIn("git tag -a v0.1.10", row["create_command"])
self.assertIn("git push --atomic origin", row["publish_command"])
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.remote, "refs/tags/v0.1.10"))
published = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
head = git_text(self.repo, "rev-parse", "HEAD")
self.assertEqual(published["status"], "published")
self.assertEqual(git_text(self.repo, "cat-file", "-t", "refs/tags/v0.1.10"), "tag")
self.assertEqual(git_text(self.repo, "rev-parse", "refs/tags/v0.1.10^{commit}"), head)
self.assertEqual(git_text(self.remote, "rev-parse", "refs/tags/v0.1.10^{commit}"), head)
repeated = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual(repeated["status"], "published")
self.assertIn("already published", repeated["repositories"][0]["detail"])
self.assertEqual(git_text(self.remote, "rev-parse", "refs/tags/v0.1.10^{commit}"), head)
def test_core_alignment_and_clean_worktree_are_release_gates(self) -> None:
mismatch = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.11"},
workspace_root=self.workspace,
)
self.assertEqual(mismatch["status"], "blocked")
self.assertIn("version alignment gate failed", mismatch["repositories"][0]["detail"])
(self.repo / "uncommitted.txt").write_text("not reviewed\n", encoding="utf-8")
dirty = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
)
self.assertEqual(dirty["status"], "blocked")
self.assertIn("worktree is not clean", dirty["repositories"][0]["detail"])
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
def test_user_workflow_scope_contract_blocks_source_tagging(self) -> None:
replace_with_unscoped_workflow_manifest(self.manifest_repo)
result = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
)
self.assertEqual(result["status"], "blocked")
self.assertIn("scope-conditioned alternative", result["repositories"][0]["detail"])
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
def test_empty_manifest_workspace_blocks_source_tagging(self) -> None:
workspace = self.root / "empty-manifest-workspace"
remote_root = self.root / "empty-manifest-remotes"
workspace.mkdir()
remote_root.mkdir()
core, _ = create_release_repo(
root=remote_root,
workspace=workspace,
name="govoplan-core",
version="0.1.10",
)
result = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=workspace,
)
self.assertEqual(result["status"], "blocked")
self.assertIn("No module manifests found", result["repositories"][0]["detail"])
self.assertFalse(ref_exists(core, "refs/tags/v0.1.10"))
def test_batch_preflight_blocks_every_mutation_when_a_later_repo_is_dirty(self) -> None:
access = self.manifest_repo
access_remote = self.manifest_remote
(access / "uncommitted.txt").write_text("not reviewed\n", encoding="utf-8")
result = tag_repositories(
repos=("govoplan-core", "govoplan-access"),
repo_versions={"govoplan-core": "0.1.10", "govoplan-access": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual(result["status"], "blocked")
self.assertIn("no selected repository was mutated", result["detail"])
self.assertEqual(result["repositories"][0]["status"], "skipped")
self.assertEqual(result["repositories"][1]["status"], "blocked")
for repository in (self.repo, self.remote, access, access_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_batch_preflight_blocks_selected_webui_version_missing_from_core_bundle(self) -> None:
campaign, campaign_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-campaign",
version="0.1.10",
)
campaign_webui = campaign / "webui"
campaign_webui.mkdir()
(campaign_webui / "package.json").write_text(
'{"name":"@govoplan/campaign-webui","version":"0.1.10"}\n',
encoding="utf-8",
)
git(campaign, "add", "webui/package.json")
git(campaign, "commit", "-m", "Add Campaign WebUI package")
git(campaign, "tag", "-a", "v0.1.9", "-m", "Prior Campaign release")
git(campaign, "push", "origin", "main", "refs/tags/v0.1.9")
campaign_head = git_text(campaign, "rev-parse", "HEAD")
core_webui = self.repo / "webui"
core_webui.mkdir()
old_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v0.1.9"
(core_webui / "package.release.json").write_text(
json.dumps(
{
"name": "@govoplan/core-webui",
"version": "0.1.10",
"dependencies": {"@govoplan/campaign-webui": old_ref},
}
)
+ "\n",
encoding="utf-8",
)
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {
"name": "@govoplan/core-webui",
"version": "0.1.10",
"dependencies": {"@govoplan/campaign-webui": old_ref},
},
"node_modules/@govoplan/campaign-webui": {
"version": "0.1.9",
"resolved": f"git+ssh://git@example.test/acme/govoplan-campaign.git#{campaign_head}",
},
}
}
)
+ "\n",
encoding="utf-8",
)
git(self.repo, "add", "webui/package.release.json", "webui/package-lock.release.json")
git(self.repo, "commit", "-m", "Add release WebUI composition")
git(self.repo, "push", "origin", "main")
result = tag_repositories(
repos=("govoplan-core", "govoplan-campaign"),
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual("blocked", result["status"])
self.assertIn("no selected repository was mutated", result["detail"])
self.assertEqual("skipped", result["repositories"][0]["status"])
self.assertEqual("blocked", result["repositories"][1]["status"])
self.assertIn("release WebUI composition gate failed", result["repositories"][1]["detail"])
for repository in (self.repo, self.remote, campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_api_keeps_legacy_source_release_preview_only(self) -> None:
with TestClient(
create_app(workspace_root=self.workspace, token="token")
) as client:
headers = {"X-Release-Console-Token": "token"}
rejected = client.post(
"/api/repositories/tag",
headers=headers,
json={
"repos": ["govoplan-core"],
"repo_versions": {"govoplan-core": "0.1.10"},
"apply": True,
"push": True,
"confirm": "TAG",
},
)
preview = client.post(
"/api/repositories/tag",
headers=headers,
json={
"repos": ["govoplan-core"],
"repo_versions": {"govoplan-core": "0.1.10"},
"apply": False,
"push": True,
},
)
legacy_push = client.post(
"/api/repositories/push",
headers=headers,
json={
"repos": ["govoplan-core"],
"apply": True,
"confirm": "PUSH",
},
)
legacy_sync = client.post(
"/api/repositories/sync",
headers=headers,
json={
"repos": ["govoplan-core"],
"apply": True,
"confirm": "SYNC",
},
)
ui = client.get("/")
self.assertEqual(rejected.status_code, 409)
self.assertIn("durable release run", rejected.json()["detail"])
self.assertEqual(preview.status_code, 200)
self.assertEqual(409, legacy_push.status_code)
self.assertEqual(409, legacy_sync.status_code)
self.assertIn("durable", legacy_push.json()["detail"])
self.assertIn("durable", legacy_sync.json()["detail"])
self.assertEqual(preview.json()["repositories"][0]["repo"], "govoplan-core")
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
self.assertIn('id="publishReleaseTags"', ui.text)
self.assertIn(
'id="publishReleaseTags" data-release-disabled="true" disabled',
ui.text,
)
self.assertIn('postJson("/api/repositories/tag"', ui.text)
self.assertIn(
'id="syncRepos" disabled data-release-disabled="true"', ui.text
)
self.assertIn(
'id="pushRepos" disabled data-release-disabled="true"', ui.text
)
self.assertIn("Signed Website Catalog", ui.text)
self.assertIn("Apply + Website Tag", ui.text)
def git(cwd: Path, *args: str) -> None:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
def create_release_repo(*, root: Path, workspace: Path, name: str, version: str) -> tuple[Path, Path]:
remote = root / f"{name}.git"
repo = workspace / name
git(root, "init", "--bare", str(remote))
git(workspace, "init", "-b", "main", str(repo))
git(repo, "config", "user.name", "GovOPlaN Release Test")
git(repo, "config", "user.email", "release-test@example.invalid")
(repo / "pyproject.toml").write_text(
f'[project]\nname = "{name}"\nversion = "{version}"\n',
encoding="utf-8",
)
git(repo, "add", "pyproject.toml")
git(repo, "commit", "-m", f"{name} {version}")
git(repo, "remote", "add", "origin", str(remote))
git(repo, "push", "-u", "origin", "main")
return repo, remote
def add_scoped_workflow_manifest(repo: Path) -> None:
package = repo / "src" / "govoplan_access"
backend = package / "backend"
backend.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8")
(backend / "__init__.py").write_text("", encoding="utf-8")
(backend / "manifest.py").write_text(
"""from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition
def get_manifest():
return ModuleManifest(
id="access",
name="Access",
version="0.1.10",
permissions=(
PermissionDefinition(
scope="access:item:read",
module_id="access",
resource="item",
action="read",
label="Read items",
description="Read example items.",
category="Example",
level="tenant",
),
),
documentation=(
DocumentationTopic(
id="access.workflow.scoped",
title="Scoped task",
summary="This is a valid release fixture.",
documentation_types=("user",),
conditions=(DocumentationCondition(required_scopes=("access:item:read",)),),
metadata={"kind": "workflow"},
),
),
)
""",
encoding="utf-8",
)
with (repo / "pyproject.toml").open("a", encoding="utf-8") as handle:
handle.write(
'\n[project.entry-points."govoplan.modules"]\n'
'access = "govoplan_access.backend.manifest:get_manifest"\n'
)
git(repo, "add", "pyproject.toml", "src")
git(repo, "commit", "-m", "Add scoped workflow manifest")
git(repo, "push", "origin", "main")
def replace_with_unscoped_workflow_manifest(repo: Path) -> None:
manifest = repo / "src" / "govoplan_access" / "backend" / "manifest.py"
manifest.write_text(
"""from govoplan_core.core.modules import DocumentationTopic, ModuleManifest
def get_manifest():
return ModuleManifest(
id="access",
name="Access",
version="0.1.10",
documentation=(
DocumentationTopic(
id="access.workflow.unscoped",
title="Unscoped task",
summary="This must block release.",
documentation_types=("user",),
metadata={"kind": "workflow"},
),
),
)
""",
encoding="utf-8",
)
git(repo, "add", str(manifest.relative_to(repo)))
git(repo, "commit", "-m", "Replace scoped workflow with unscoped workflow")
git(repo, "push", "origin", "main")
def git_text(cwd: Path, *args: str) -> str:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
return result.stdout.strip()
def ref_exists(cwd: Path, ref: str) -> bool:
return subprocess.run(
("git", "rev-parse", "--verify", ref),
cwd=cwd,
check=False,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
).returncode == 0
if __name__ == "__main__":
unittest.main()

File diff suppressed because it is too large Load Diff

126
tests/test_release_sbom.py Normal file
View File

@@ -0,0 +1,126 @@
from __future__ import annotations
import base64
from datetime import UTC, datetime
import hashlib
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.sbom import build_release_sbom, render_sbom, sbom_sha256, validate_sbom # noqa: E402
class ReleaseSbomTests(unittest.TestCase):
@staticmethod
def _build_at(timestamp: datetime) -> dict:
return build_release_sbom(
product_version="1.2.3",
pip_inspect={
"version": "1",
"installed": [{"metadata": {"name": "FastAPI", "version": "0.100.0"}}],
},
npm_lock={"packages": {"": {"name": "govoplan", "version": "1.2.3"}}},
timestamp=timestamp,
)
def test_builds_valid_deduplicated_python_and_npm_components(self) -> None:
integrity = base64.b64encode(b"example digest").decode("ascii")
payload = build_release_sbom(
product_version="1.2.3",
pip_inspect={
"version": "1",
"installed": [
{"metadata": {"name": "FastAPI", "version": "0.100.0"}},
{"metadata": {"name": "FastAPI", "version": "0.100.0"}},
],
},
npm_lock={
"packages": {
"": {"name": "govoplan", "version": "1.2.3"},
"node_modules/@scope/example": {
"name": "@scope/example",
"version": "4.5.6",
"integrity": f"sha512-{integrity}",
},
}
},
)
validate_sbom(payload)
self.assertEqual(2, len(payload["components"]))
npm = next(item for item in payload["components"] if item["name"] == "@scope/example")
self.assertEqual("SHA-512", npm["hashes"][0]["alg"])
self.assertIn("pkg:npm/%40scope/example@4.5.6", npm["purl"])
self.assertEqual(
hashlib.sha256(render_sbom(payload).encode("utf-8")).hexdigest(),
sbom_sha256(payload),
)
def test_rejects_empty_component_inventory(self) -> None:
payload = build_release_sbom(
product_version="1.2.3",
pip_inspect={"version": "1", "installed": []},
npm_lock={"packages": {"": {"version": "1.2.3"}}},
)
with self.assertRaisesRegex(ValueError, "no dependency components"):
validate_sbom(payload)
def test_fixed_timestamp_produces_identical_bytes_and_digest(self) -> None:
timestamp = datetime(2026, 7, 21, 8, 30, tzinfo=UTC)
first = self._build_at(timestamp)
second = self._build_at(timestamp)
self.assertEqual(render_sbom(first), render_sbom(second))
self.assertEqual(sbom_sha256(first), sbom_sha256(second))
def test_timestamp_changes_serial_and_digest(self) -> None:
first = self._build_at(datetime(2026, 7, 21, 8, 30, tzinfo=UTC))
second = self._build_at(datetime(2026, 7, 21, 8, 31, tzinfo=UTC))
self.assertNotEqual(first["serialNumber"], second["serialNumber"])
self.assertNotEqual(sbom_sha256(first), sbom_sha256(second))
def test_local_and_credential_bearing_distribution_urls_are_omitted(self) -> None:
payload = build_release_sbom(
product_version="1.2.3",
pip_inspect={
"version": "1",
"installed": [
{
"metadata": {"name": "LocalPackage", "version": "1.0.0"},
"direct_url": {"url": "file:///private/build/govoplan-core"},
},
{
"metadata": {"name": "PrivatePackage", "version": "1.0.0"},
"direct_url": {"url": "https://user:secret@example.test/private.whl"},
},
{
"metadata": {"name": "PublicPackage", "version": "1.0.0"},
"direct_url": {"url": "https://example.test/public.whl"},
},
],
},
npm_lock={"packages": {"": {"name": "govoplan", "version": "1.2.3"}}},
timestamp=datetime(2026, 7, 21, 8, 30, tzinfo=UTC),
)
by_name = {component["name"]: component for component in payload["components"]}
self.assertNotIn("externalReferences", by_name["LocalPackage"])
self.assertNotIn("externalReferences", by_name["PrivatePackage"])
self.assertEqual(
[{"type": "distribution", "url": "https://example.test/public.whl"}],
by_name["PublicPackage"]["externalReferences"],
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,373 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from fastapi.testclient import TestClient
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.publisher import publish_catalog_candidate # noqa: E402
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
build_selective_catalog_candidate,
enforce_selected_source_provenance,
parse_signing_key,
public_key_base64,
signature,
)
from govoplan_release.source_provenance import ( # noqa: E402
catalog_source_selection,
read_only_https_remote,
source_tag_provenance_issues,
tagged_source_version_issues,
)
from server.app import create_app # noqa: E402
class ReleaseSourceProvenanceTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.workspace = self.root / "workspace"
self.workspace.mkdir()
def tearDown(self) -> None:
self.temporary.cleanup()
def test_selected_source_requires_local_and_remote_annotated_tag_at_clean_head(self) -> None:
repo, remote = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
missing = source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
)
self.assertIn("missing from the local source repository", messages(missing))
self.assertIn("missing from remote 'origin'", messages(missing))
git(repo, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(repo, "push", "origin", "refs/tags/v1.2.3")
self.assertEqual(
(),
source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
),
)
git(remote, "config", "user.name", "Different Release Actor")
git(remote, "config", "user.email", "different@example.invalid")
git(remote, "tag", "-f", "-a", "v1.2.3", git_text(repo, "rev-parse", "HEAD"), "-m", "Different annotation")
inconsistent = source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
)
self.assertIn("local and remote annotated tag objects differ", messages(inconsistent))
(repo / "dirty.txt").write_text("not reviewed\n", encoding="utf-8")
dirty = source_tag_provenance_issues(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
require_head_repos=("govoplan-core",),
)
self.assertIn("selected source worktree is not clean", messages(dirty))
self.assertEqual(git_text(remote, "rev-parse", "refs/tags/v1.2.3^{commit}"), git_text(repo, "rev-parse", "HEAD"))
def test_selected_source_gate_is_fail_closed_before_signing(self) -> None:
make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
with self.assertRaisesRegex(ValueError, "Source tag provenance gate failed") as error:
enforce_selected_source_provenance(
repo_versions={"govoplan-core": "1.2.3"},
workspace=self.workspace,
)
self.assertIn("v1.2.3", str(error.exception))
self.assertIn("missing", str(error.exception))
def test_historical_tag_uses_installable_and_manifest_version_metadata(self) -> None:
repo, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
package = repo / "src" / "govoplan_core"
package.mkdir(parents=True)
(package / "__init__.py").write_text('__version__ = "1.2.2"\n', encoding="utf-8")
git(repo, "add", "src/govoplan_core/__init__.py")
git(repo, "commit", "-m", "Keep legacy runtime version string")
git(repo, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
issues = tagged_source_version_issues(
path=repo,
repo="govoplan-core",
tag="v1.2.3",
expected_version="1.2.3",
)
self.assertEqual((), issues)
def test_signed_candidate_records_selected_commit_and_tag_object(self) -> None:
core, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
git(core, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(core, "push", "origin", "refs/tags/v1.2.3")
private_key = Ed25519PrivateKey.generate()
keyring = {
"keys": [
{
"key_id": "test-key",
"status": "active",
"public_key": public_key_base64(private_key),
}
]
}
base_keyring = self.root / "base-keyring.json"
base_keyring.write_text(json.dumps(keyring), encoding="utf-8")
base_catalog = self.root / "base.json"
base_payload = {
"channel": "stable",
"sequence": 1,
"core_release": {
"version": "1.2.2",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.2",
},
"modules": [],
"release": {
"version": "1.2.2",
"tag": "v1.2.2",
"keyring_sha256": canonical_hash(keyring),
},
}
base_payload["signatures"] = [
signature(base_payload, key_id="test-key", private_key=private_key)
]
base_catalog.write_text(json.dumps(base_payload), encoding="utf-8")
key_path = self.root / "release.pem"
key_path.write_bytes(
private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
key_path.chmod(0o600)
output = self.root / "candidate"
with patch(
"govoplan_release.selective_catalog.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = build_selective_catalog_candidate(
repo_versions={"govoplan-core": "1.2.3"},
workspace_root=self.workspace,
output_dir=output,
base_catalog=base_catalog,
base_keyring=base_keyring,
signing_keys=(f"test-key={key_path}",),
check_public=False,
)
payload = json.loads(Path(result.catalog_path).read_text(encoding="utf-8"))
selected = payload["release"]["selected_units"][0]
self.assertEqual(git_text(core, "rev-parse", "refs/tags/v1.2.3^{commit}"), selected["commit_sha"])
self.assertEqual(git_text(core, "rev-parse", "refs/tags/v1.2.3"), selected["tag_object_sha"])
self.assertEqual("test-key", payload["signatures"][0]["key_id"])
def test_catalog_signing_key_must_be_operator_private(self) -> None:
private_key = Ed25519PrivateKey.generate()
key_path = self.root / "release.pem"
key_path.write_bytes(
private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
key_path.chmod(0o644)
with self.assertRaisesRegex(ValueError, "inaccessible to other users"):
parse_signing_key(f"test-key={key_path}")
key_path.chmod(0o600)
key_id, _parsed = parse_signing_key(f"test-key={key_path}")
self.assertEqual("test-key", key_id)
def test_catalog_publication_checks_every_preserved_source_ref(self) -> None:
core, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
git(core, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(core, "push", "origin", "refs/tags/v1.2.3")
make_repo(self.workspace, self.root, "govoplan-access", "1.1.0")
candidate = self.root / "candidate"
(candidate / "channels").mkdir(parents=True)
payload = catalog_payload(
commit_sha="f" * 40,
tag_object_sha=git_text(core, "rev-parse", "refs/tags/v1.2.3"),
)
(candidate / "channels" / "stable.json").write_text(json.dumps(payload), encoding="utf-8")
keyring = {"keys": [{"key_id": "release-key", "status": "active", "public_key": "trusted"}]}
(candidate / "keyring.json").write_text(json.dumps(keyring), encoding="utf-8")
web_root = self.root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(keyring), encoding="utf-8")
with patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
):
result = publish_catalog_candidate(
candidate_dir=candidate,
workspace_root=self.workspace,
web_root=web_root,
)
detail = " ".join(result.notes)
self.assertEqual("blocked", result.status)
self.assertIn("govoplan-access v1.1.0", detail)
self.assertIn("missing from the local source repository", detail)
self.assertIn("missing from remote 'origin'", detail)
self.assertIn("no longer matches signed commit_sha", detail)
self.assertNotIn("govoplan-core v1.2.3: annotated tag is missing", detail)
def test_catalog_selection_includes_core_and_all_modules(self) -> None:
selection = catalog_source_selection(catalog_payload())
self.assertEqual(
{"govoplan-core": "1.2.3", "govoplan-access": "1.1.0"},
selection.all_versions,
)
self.assertEqual({"govoplan-core": "1.2.3"}, selection.selected_versions)
self.assertEqual((), selection.issues)
def test_console_renders_publication_provenance_notes(self) -> None:
ui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
self.assertIn("Array.isArray(result.notes)", ui)
self.assertIn("Validation and provenance details", ui)
def test_console_disables_unclaimed_legacy_candidate_signing(self) -> None:
with TestClient(
create_app(workspace_root=self.workspace, token="token")
) as client:
response = client.post(
"/api/catalog-candidates",
headers={"X-Release-Console-Token": "token"},
json={
"repo_versions": {"govoplan-core": "1.2.3"},
"signing_keys": ["test=/unused/key.pem"],
},
)
self.assertEqual(409, response.status_code)
self.assertIn("durable release run", response.json()["detail"])
def test_read_only_ref_checks_can_reuse_the_same_gitea_https_endpoint(self) -> None:
self.assertEqual(
"https://git.example.test/acme/govoplan-core.git",
read_only_https_remote("git@git.example.test:acme/govoplan-core.git"),
)
self.assertEqual(
"https://git.example.test/acme/govoplan-core.git",
read_only_https_remote("ssh://git@git.example.test/acme/govoplan-core.git"),
)
self.assertIsNone(read_only_https_remote("ssh://git@git.example.test:2222/acme/govoplan-core.git"))
self.assertIsNone(read_only_https_remote("/srv/git/govoplan-core.git"))
def catalog_payload(
*,
commit_sha: str = "0" * 40,
tag_object_sha: str = "1" * 40,
) -> dict[str, object]:
return {
"channel": "stable",
"core_release": {
"version": "1.2.3",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.3",
},
"modules": [
{
"module_id": "access",
"version": "1.1.0",
"python_package": "govoplan-access",
"python_ref": "govoplan-access @ git+ssh://git@example.test/acme/govoplan-access.git@v1.1.0",
}
],
"release": {
"version": "1.2.3",
"tag": "v1.2.3",
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": commit_sha,
"tag_object_sha": tag_object_sha,
},
],
},
}
def make_repo(workspace: Path, root: Path, name: str, version: str) -> tuple[Path, Path]:
remote = root / f"{name}.git"
repo = workspace / name
git(root, "init", "--bare", str(remote))
git(workspace, "init", "-b", "main", str(repo))
git(repo, "config", "user.name", "GovOPlaN Release Test")
git(repo, "config", "user.email", "release-test@example.invalid")
(repo / "pyproject.toml").write_text(
f'[project]\nname = "{name}"\nversion = "{version}"\n',
encoding="utf-8",
)
git(repo, "add", "pyproject.toml")
git(repo, "commit", "-m", f"{name} {version}")
git(repo, "remote", "add", "origin", str(remote))
git(repo, "push", "-u", "origin", "main")
return repo, remote
def messages(issues: object) -> str:
return " ".join(issue.message for issue in issues) # type: ignore[attr-defined]
def git(cwd: Path, *args: str) -> None:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
def git_text(cwd: Path, *args: str) -> str:
result = subprocess.run(
("git", *args),
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
if result.returncode != 0:
raise AssertionError(f"git {' '.join(args)} failed: {result.stderr or result.stdout}")
return result.stdout.strip()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,364 @@
from __future__ import annotations
import importlib.util
import json
from pathlib import Path
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "repo" / "bootstrap-repositories.py"
def load_bootstrap_module():
spec = importlib.util.spec_from_file_location("bootstrap_repositories", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class RepositoryBootstrapTests(unittest.TestCase):
def test_public_https_transport_rewrites_registered_gitea_remotes(self) -> None:
bootstrap = load_bootstrap_module()
self.assertEqual(
"https://git.add-ideas.de/add-ideas/govoplan-core.git",
bootstrap.clone_remote(
"git@git.add-ideas.de:add-ideas/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
self.assertEqual(
"https://git.add-ideas.de/add-ideas/govoplan-core.git",
bootstrap.clone_remote(
"ssh://git@git.add-ideas.de/add-ideas/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
self.assertEqual(
"https://git.add-ideas.de/add-ideas/govoplan-core.git",
bootstrap.clone_remote(
"https://git.add-ideas.de/add-ideas/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
def test_registered_transport_preserves_the_manifest_remote(self) -> None:
bootstrap = load_bootstrap_module()
remote = "git@example.test:private/repository.git"
self.assertEqual(
remote,
bootstrap.clone_remote(
remote,
transport=bootstrap.REGISTERED_TRANSPORT,
),
)
def test_public_https_transport_fails_closed_for_other_hosts(self) -> None:
bootstrap = load_bootstrap_module()
unsafe_remotes = (
"git@example.test:add-ideas/govoplan-core.git",
"https://token@git.add-ideas.de/add-ideas/govoplan-core.git",
"https://git.add-ideas.de:443/add-ideas/govoplan-core.git",
"https://git.add-ideas.de/add-ideas/../govoplan-core.git",
"https://git.add-ideas.de/add-ideas/govoplan-core.git?ref=main",
"ssh://root@git.add-ideas.de/add-ideas/govoplan-core.git",
)
for remote in unsafe_remotes:
with self.subTest(remote=remote), self.assertRaises(ValueError):
bootstrap.clone_remote(
remote,
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
)
def test_main_clones_missing_repositories_over_public_https(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(0, status)
runner.assert_called_once_with(
[
"git",
"-c",
"credential.helper=",
"clone",
"--",
"https://git.add-ideas.de/add-ideas/govoplan-core.git",
str(parent / "govoplan-core"),
],
check=True,
env=runner.call_args.kwargs["env"],
)
environment = runner.call_args.kwargs["env"]
self.assertEqual("/bin/false", environment["GIT_ASKPASS"])
self.assertEqual("0", environment["GIT_TERMINAL_PROMPT"])
def test_main_validates_every_remote_before_cloning(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/govoplan-core.git"
),
},
{
"name": "unsafe",
"path": "unsafe",
"remote": "git@example.test:private/unsafe.git",
},
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
self.assertRaises(ValueError),
):
bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
runner.assert_not_called()
def test_main_preserves_an_explicit_private_repository_transport(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "website",
"path": "website",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/addideas-govoplan-website.git"
),
"bootstrap_transport": (
bootstrap.REGISTERED_TRANSPORT
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(0, status)
command = runner.call_args.args[0]
self.assertEqual(
"git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git",
command[-2],
)
def test_main_limits_bootstrap_to_selected_repositories(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": name,
"path": name,
"remote": (
"git@git.add-ideas.de:add-ideas/"
f"{name}.git"
),
}
for name in ("govoplan-core", "govoplan-poll")
],
}
),
encoding="utf-8",
)
for repository_filter in (
["--repo", "govoplan-core"],
["--exclude-repo", "govoplan-poll"],
):
with (
self.subTest(repository_filter=repository_filter),
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
*repository_filter,
]
)
self.assertEqual(0, status)
runner.assert_called_once()
self.assertEqual(
"https://git.add-ideas.de/add-ideas/govoplan-core.git",
runner.call_args.args[0][-2],
)
def test_main_rejects_unknown_or_conflicting_repository_filters(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
with self.assertRaisesRegex(ValueError, "unknown registered"):
bootstrap.main(["--repo", "govoplan-missing"])
with self.assertRaisesRegex(ValueError, "selected and excluded"):
bootstrap.main(
[
"--repo",
"govoplan-core",
"--exclude-repo",
"govoplan-core",
]
)
runner.assert_not_called()
def test_check_reports_missing_without_cloning(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--check",
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(1, status)
runner.assert_not_called()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,248 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RESOLVER = (
META_ROOT / "tools" / "checks" / "security-audit" / "resolve_workspace_mount.py"
)
def load_resolver_module():
spec = importlib.util.spec_from_file_location(
"security_audit_mount_resolver",
RESOLVER,
)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {RESOLVER}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class SecurityAuditMountResolverTests(unittest.TestCase):
def setUp(self) -> None:
self.resolver = load_resolver_module()
self.root = "/workspace/add-ideas/govoplan/govoplan"
self.workspace = "/workspace/add-ideas/govoplan"
def test_resolves_only_the_named_workspace_volume_for_both_scopes(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
{
"Type": "volume",
"Name": "actions-workspace",
"Source": "/var/lib/docker/volumes/actions-workspace/_data",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "volume",
"Name": "actions-environment",
"Source": "/var/lib/docker/volumes/actions-environment/_data",
"Destination": "/var/run/act",
"RW": True,
},
]
for scope in ("current", "govoplan"):
with self.subTest(scope=scope):
self.assertEqual(
(f"type=volume,source=actions-workspace,target={self.workspace}"),
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope=scope,
reports_dir="audit-reports",
),
)
def test_resolves_a_workspace_bind_without_other_mounts(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/srv/gitea/actions/task-123",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
]
self.assertEqual(
(f"type=bind,source=/srv/gitea/actions/task-123,target={self.workspace}"),
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
),
)
def test_nested_repository_mount_is_valid_only_for_current_scope(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "all-repositories",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "volume",
"Name": "current-repository",
"Destination": self.root,
"RW": True,
},
]
self.assertIn(
"source=current-repository",
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="current",
reports_dir="audit-reports",
),
)
with self.assertRaisesRegex(
self.resolver.MountResolutionError,
"nested job-container mounts",
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
def test_rejects_unsafe_or_unusable_mount_layouts(self) -> None:
cases = {
"missing": [],
"path-boundary": [
{
"Type": "volume",
"Name": "wrong-workspace",
"Destination": "/workspace/add-ideas/govoplan-other",
"RW": True,
}
],
"read-only": [
{
"Type": "volume",
"Name": "actions-workspace",
"Destination": self.workspace,
"RW": False,
}
],
"ambiguous": [
{
"Type": "volume",
"Name": name,
"Destination": self.workspace,
"RW": True,
}
for name in ("workspace-one", "workspace-two")
],
"scope-too-narrow": [
{
"Type": "volume",
"Name": "repository-only",
"Destination": self.root,
"RW": True,
}
],
}
for name, mounts in cases.items():
with (
self.subTest(name=name),
self.assertRaises(self.resolver.MountResolutionError),
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
def test_rejects_reports_outside_the_selected_workspace_mount(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "actions-workspace",
"Destination": self.workspace,
"RW": True,
}
]
with self.assertRaisesRegex(
self.resolver.MountResolutionError,
"reports path .* outside",
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="current",
reports_dir="/tmp/audit-reports",
)
def test_rejects_broad_or_sensitive_workspace_bind_sources(self) -> None:
for source in (
"/",
"/home",
"/var/run/docker.sock",
"/srv/../etc/shadow",
):
with (
self.subTest(source=source),
self.assertRaisesRegex(
self.resolver.MountResolutionError,
"too broad or sensitive",
),
):
self.resolver.resolve_workspace_mount(
[
{
"Type": "bind",
"Source": source,
"Destination": self.workspace,
"RW": True,
}
],
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
with self.assertRaises(self.resolver.MountResolutionError):
self.resolver.resolve_workspace_mount(
[
{
"Type": "bind",
"Source": "//var/lib/workspace",
"Destination": self.workspace,
"RW": True,
}
],
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,516 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import textwrap
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
AUDIT_SCRIPT = META_ROOT / "tools" / "checks" / "check-security-audit.sh"
CONTAINER_RUNNER = META_ROOT / "tools" / "checks" / "security-audit" / "run.sh"
class SecurityAuditWrapperTests(unittest.TestCase):
def setUp(self) -> None:
self._temporary_directory = tempfile.TemporaryDirectory()
self.temporary_root = Path(self._temporary_directory.name)
self.checkout = self.temporary_root / "checkout"
checks = self.checkout / "tools" / "checks"
checks.mkdir(parents=True)
shutil.copy2(AUDIT_SCRIPT, checks / AUDIT_SCRIPT.name)
(self.checkout / "README.md").write_text("audit fixture\n", encoding="utf-8")
subprocess.run(["git", "init", "-q", str(self.checkout)], check=True)
subprocess.run(
[
"git",
"-C",
str(self.checkout),
"config",
"user.email",
"audit@example.invalid",
],
check=True,
)
subprocess.run(
["git", "-C", str(self.checkout), "config", "user.name", "Audit Test"],
check=True,
)
subprocess.run(["git", "-C", str(self.checkout), "add", "."], check=True)
subprocess.run(
["git", "-C", str(self.checkout), "commit", "-qm", "audit fixture"],
check=True,
)
self.stub_bin = self.temporary_root / "bin"
self.stub_bin.mkdir()
self._write_stub(
"semgrep",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'semgrep stub 1.0'
exit 0
fi
if [[ " $* " != *" --error "* ]]; then
echo 'missing --error' >&2
exit 3
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output' ]]; then
output="$2"
shift 2
else
shift
fi
done
case "${STUB_SEMGREP_REPORT:-valid}" in
valid)
printf '%s\n' '{"version":"2.1.0","runs":[]}' > "$output"
;;
malformed)
printf '%s\n' '{' > "$output"
;;
invalid-sarif)
printf '%s\n' '{}' > "$output"
;;
missing)
;;
esac
if [[ -n "${STUB_MUTATE_PATH:-}" ]]; then
printf '%s\n' 'changed during audit' >> "$STUB_MUTATE_PATH"
fi
exit "${STUB_SEMGREP_EXIT:-0}"
""",
)
self._write_stub(
"gitleaks",
r"""
if [[ "${1:-}" == '--version' || "${1:-}" == 'version' ]]; then
echo 'gitleaks stub 1.0'
exit 0
fi
if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--report-path' ]]; then
output="$2"
shift 2
else
shift
fi
done
mkdir -p "$(dirname "$output")"
printf '%s\n' '[]' > "$output"
exit "${STUB_GITLEAKS_EXIT:-0}"
""",
)
for tool in ("bandit", "ruff"):
self._write_stub(
tool,
f"""
if [[ " $* " == *" --version "* ]]; then
echo '{tool} stub 1.0'
fi
exit 0
""",
)
def tearDown(self) -> None:
self._temporary_directory.cleanup()
def _write_stub(self, name: str, body: str) -> None:
path = self.stub_bin / name
path.write_text(
"#!/usr/bin/env bash\nset -euo pipefail\n" + textwrap.dedent(body).lstrip(),
encoding="utf-8",
)
path.chmod(0o755)
def _install_full_mode_stubs(self) -> None:
self._write_stub(
"trivy",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'trivy stub 1.0'
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output' ]]; then
output="$2"
shift 2
else
shift
fi
done
printf '%s\n' '{"version":"2.1.0","runs":[]}' > "$output"
""",
)
self._write_stub(
"osv-scanner",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'osv-scanner stub 1.0'
exit 0
fi
if [[ " $* " == *" scan --help "* ]]; then
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output-file' ]]; then
output="$2"
shift 2
else
shift
fi
done
printf '{"results": %s}\n' "${STUB_OSV_RESULTS:-null}" > "$output"
""",
)
self._write_stub(
"jscpd",
r"""
if [[ " $* " == *" --version "* ]]; then
echo 'jscpd stub 1.0'
exit 0
fi
output=''
while [[ $# -gt 0 ]]; do
if [[ "$1" == '--output' ]]; then
output="$2"
shift 2
else
shift
fi
done
mkdir -p "$output"
printf '%s\n' '{"duplicates":[],"statistics":{}}' > "$output/jscpd-report.json"
""",
)
for tool in ("pip-audit", "npm", "radon", "xenon"):
self._write_stub(
tool,
f"""
if [[ " $* " == *" --version "* ]]; then
echo '{tool} stub 1.0'
exit 0
fi
exit 0
""",
)
def _run(
self, *arguments: str, mode: str = "quick", **environment: str
) -> tuple[subprocess.CompletedProcess[str], Path]:
reports = self.temporary_root / "reports"
env = os.environ.copy()
env.update(
{
"PATH": f"{self.stub_bin}:{env['PATH']}",
"SECURITY_AUDIT_REQUIRE_TOOLS": "1",
"SECURITY_AUDIT_TOOLBOX_FINGERPRINT": 'stub-"safe',
**environment,
}
)
result = subprocess.run(
[
"bash",
str(self.checkout / "tools" / "checks" / AUDIT_SCRIPT.name),
"--mode",
mode,
"--scope",
"current",
"--reports-dir",
str(reports),
*arguments,
],
cwd=self.checkout,
env=env,
check=False,
capture_output=True,
text=True,
)
return result, reports
@staticmethod
def _manifest(reports: Path) -> dict[str, object]:
return json.loads((reports / "manifest.json").read_text(encoding="utf-8"))
def test_report_only_findings_are_nonfatal_and_strict_findings_fail(self) -> None:
result, reports = self._run("--report-only", STUB_SEMGREP_EXIT="1")
self.assertEqual(0, result.returncode, result.stderr)
manifest = self._manifest(reports)
self.assertEqual(0, manifest["overall_status"])
self.assertEqual(1, manifest["finding_status"])
self.assertEqual(0, manifest["execution_error_status"])
self.assertIn(
"Semgrep SAST\t1\tfindings", (reports / "step-status.tsv").read_text()
)
strict_result, strict_reports = self._run("--strict", STUB_SEMGREP_EXIT="1")
self.assertEqual(1, strict_result.returncode)
self.assertEqual(1, self._manifest(strict_reports)["overall_status"])
def test_scanner_execution_error_is_fatal_in_report_only_mode(self) -> None:
result, reports = self._run("--report-only", STUB_SEMGREP_EXIT="2")
self.assertEqual(1, result.returncode)
manifest = self._manifest(reports)
self.assertEqual(1, manifest["overall_status"])
self.assertEqual(1, manifest["execution_error_status"])
self.assertIn(
"Semgrep SAST\t2\texecution-error",
(reports / "step-status.tsv").read_text(),
)
def test_malformed_or_missing_machine_report_is_fatal(self) -> None:
for report_kind in ("malformed", "invalid-sarif", "missing"):
with self.subTest(report_kind=report_kind):
result, reports = self._run(
"--report-only",
STUB_SEMGREP_REPORT=report_kind,
)
self.assertEqual(1, result.returncode)
manifest = self._manifest(reports)
self.assertEqual(1, manifest["execution_error_status"])
if report_kind == "missing":
self.assertIn("semgrep.sarif", manifest["missing_reports"])
self.assertIn(
"Validate machine-readable audit reports\t2\texecution-error",
(reports / "step-status.tsv").read_text(),
)
def test_osv_null_results_are_valid_but_wrong_shapes_fail(self) -> None:
self._install_full_mode_stubs()
result, reports = self._run("--report-only", mode="full")
self.assertEqual(0, result.returncode, result.stderr)
osv_report = json.loads(
(reports / "osv-scanner-checkout.json").read_text(encoding="utf-8")
)
self.assertIsNone(osv_report["results"])
invalid_result, invalid_reports = self._run(
"--report-only",
mode="full",
STUB_OSV_RESULTS="{}",
)
self.assertEqual(1, invalid_result.returncode)
self.assertIn(
"invalid OSV-Scanner report",
invalid_result.stderr,
)
self.assertEqual(1, self._manifest(invalid_reports)["execution_error_status"])
def test_workspace_mutation_invalidates_the_audit(self) -> None:
result, reports = self._run(
"--report-only",
STUB_MUTATE_PATH=str(self.checkout / "README.md"),
)
self.assertEqual(1, result.returncode)
manifest = self._manifest(reports)
self.assertFalse(manifest["workspace_unchanged"])
self.assertEqual(1, manifest["overall_status"])
self.assertNotEqual(
(reports / "workspace-state-start.tsv").read_text(),
(reports / "workspace-state-end.tsv").read_text(),
)
def test_manifest_and_checksums_cover_only_current_run_reports(self) -> None:
reports = self.temporary_root / "reports"
reports.mkdir()
(reports / "stale-malformed.json").write_text("{", encoding="utf-8")
result, reports = self._run("--report-only")
self.assertEqual(0, result.returncode, result.stderr)
manifest = self._manifest(reports)
self.assertEqual('stub-"safe', manifest["toolbox_fingerprint"])
self.assertNotIn("stale-malformed.json", manifest["reports"])
self.assertEqual([], manifest["missing_reports"])
checksums = (reports / "report-sha256.txt").read_text(encoding="utf-8")
self.assertNotIn("stale-malformed.json", checksums)
verification = subprocess.run(
["sha256sum", "--check", "report-sha256.txt"],
cwd=reports,
check=False,
capture_output=True,
text=True,
)
self.assertEqual(0, verification.returncode, verification.stderr)
checksummed_paths = {
line.split(" ", 1)[1] for line in checksums.splitlines() if " " in line
}
self.assertEqual(set(manifest["reports"]), checksummed_paths)
class SecurityAuditContainerRunnerTests(unittest.TestCase):
def setUp(self) -> None:
self._temporary_directory = tempfile.TemporaryDirectory(
prefix="govoplan-audit-runner-"
)
root = Path(self._temporary_directory.name)
self.stub_bin = root / "bin"
self.stub_bin.mkdir()
self.docker_log = root / "docker.jsonl"
docker_stub = self.stub_bin / "docker"
docker_stub.write_text(
textwrap.dedent(
"""\
#!/usr/bin/env python3
import json
import os
from pathlib import Path
import sys
arguments = sys.argv[1:]
with Path(os.environ["DOCKER_STUB_LOG"]).open(
"a", encoding="utf-8"
) as handle:
handle.write(json.dumps(arguments) + "\\n")
if arguments and arguments[0] == "version":
print("26.1.0")
if arguments[:2] == ["container", "inspect"]:
print(os.environ["DOCKER_STUB_MOUNTS"])
"""
),
encoding="utf-8",
)
docker_stub.chmod(0o755)
self.environment = os.environ.copy()
self.environment.update(
{
"DOCKER_STUB_LOG": str(self.docker_log),
"DOCKER_STUB_MOUNTS": "[]",
"PATH": f"{self.stub_bin}:{self.environment['PATH']}",
}
)
def tearDown(self) -> None:
self._temporary_directory.cleanup()
def _run(
self,
*,
scope: str,
actions_mounts: object | None = None,
) -> tuple[subprocess.CompletedProcess[str], list[list[str]]]:
self.docker_log.write_text("", encoding="utf-8")
environment = self.environment.copy()
if actions_mounts is None:
environment.pop("GITEA_ACTIONS", None)
else:
environment["GITEA_ACTIONS"] = "true"
environment["DOCKER_STUB_MOUNTS"] = json.dumps(actions_mounts)
result = subprocess.run(
[
"bash",
str(CONTAINER_RUNNER),
"--mode",
"quick",
"--scope",
scope,
],
cwd=META_ROOT,
env=environment,
check=False,
capture_output=True,
text=True,
)
commands = [
json.loads(line)
for line in self.docker_log.read_text(encoding="utf-8").splitlines()
]
return result, commands
def test_gitea_job_shares_only_its_workspace_mount(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
{
"Type": "volume",
"Name": "govoplan-actions-workspace",
"Destination": str(META_ROOT.parent),
"RW": True,
},
{
"Type": "volume",
"Name": "govoplan-actions-environment",
"Destination": "/var/run/act",
"RW": True,
},
]
container_id = subprocess.run(
["hostname"],
check=True,
capture_output=True,
text=True,
).stdout.strip()
for scope in ("current", "govoplan"):
with self.subTest(scope=scope):
result, commands = self._run(scope=scope, actions_mounts=mounts)
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(
command for command in commands if command[0] == "run"
)
inspect_command = next(
command
for command in commands
if command[:2] == ["container", "inspect"]
)
self.assertEqual(container_id, inspect_command[-1])
mount_index = run_command.index("--mount")
self.assertEqual(
(
"type=volume,source=govoplan-actions-workspace,"
f"target={META_ROOT.parent}"
),
run_command[mount_index + 1],
)
self.assertNotIn("--volumes-from", run_command)
self.assertNotIn("-v", run_command)
self.assertNotIn("/var/run/docker.sock", " ".join(run_command))
self.assertNotIn("/var/run/act", " ".join(run_command))
repository_roots = [
value
for value in run_command
if value.startswith("GOVOPLAN_REPOS_ROOT=")
]
expected_roots = (
[f"GOVOPLAN_REPOS_ROOT={META_ROOT.parent}"]
if scope == "govoplan"
else []
)
self.assertEqual(expected_roots, repository_roots)
def test_non_actions_runner_keeps_the_scoped_bind_mount(self) -> None:
result, commands = self._run(scope="govoplan")
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(command for command in commands if command[0] == "run")
bind_index = run_command.index("-v")
self.assertEqual(
f"{META_ROOT.parent}:/workspace",
run_command[bind_index + 1],
)
self.assertNotIn("--mount", run_command)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,467 @@
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.version_alignment import ( # noqa: E402
candidate_catalog_version_issues,
release_composition_issues,
selected_release_webui_bundle_issues,
repository_version_issues,
selected_repository_version_issues,
)
from govoplan_release.git_state import sanitized_git_environment # noqa: E402
from govoplan_release.model import ( # noqa: E402
CatalogSnapshot,
DashboardSummary,
ReleaseDashboard,
RepositorySnapshot,
RepositorySpec,
VersionSnapshot,
)
from govoplan_release.selective_planner import build_selective_release_plan, build_unit # noqa: E402
from govoplan_release.selective_catalog import enforce_selected_version_alignment # noqa: E402
class VersionAlignmentTests(unittest.TestCase):
def test_candidate_catalog_refs_and_selected_units_must_match_versions(self) -> None:
payload = {
"core_release": {
"version": "1.2.3",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.3",
},
"modules": [
{
"module_id": "files",
"version": "1.2.3",
"python_ref": "govoplan-files @ git+ssh://git@example.test/acme/govoplan-files.git@v1.2.4",
"webui_package": "@govoplan/files-webui",
"webui_ref": "git+ssh://git@example.test/acme/govoplan-files.git#v1.2.3",
}
],
"release": {
"selected_units": [
{"repo": "govoplan-files", "version": "1.2.5", "tag": "v1.2.6"},
]
},
}
issues = candidate_catalog_version_issues(payload)
messages = {issue.message for issue in issues}
self.assertIn("Python ref tag must match the catalog entry version", messages)
self.assertIn("selected-unit tag must match its version", messages)
self.assertIn("selected-unit version must match the catalog entry", messages)
def test_catalog_candidate_gate_rejects_selected_repo_drift(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
repo = workspace / "govoplan-files"
(repo / "webui").mkdir(parents=True)
(repo / "pyproject.toml").write_text('[project]\nname="govoplan-files"\nversion="1.2.3"\n')
(repo / "webui" / "package.json").write_text('{"version":"1.2.4"}\n')
with self.assertRaisesRegex(ValueError, "Version alignment gate failed"):
enforce_selected_version_alignment(
repo_versions={"govoplan-files": "1.2.5"},
workspace=workspace,
)
def test_catalog_candidate_gate_rejects_requested_version_that_differs_from_source(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
repo = workspace / "govoplan-files"
(repo / "webui").mkdir(parents=True)
(repo / "pyproject.toml").write_text('[project]\nname="govoplan-files"\nversion="1.2.3"\n')
(repo / "webui" / "package.json").write_text('{"version":"1.2.3"}\n')
with self.assertRaisesRegex(ValueError, "source version must match the requested release version"):
enforce_selected_version_alignment(
repo_versions={"govoplan-files": "1.2.4"},
workspace=workspace,
)
def test_selected_webui_release_must_match_core_bundle_input_and_lock(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
core_webui = workspace / "govoplan-core" / "webui"
campaign_webui = workspace / "govoplan-campaign" / "webui"
core_webui.mkdir(parents=True)
campaign_webui.mkdir(parents=True)
dependency = "@govoplan/campaign-webui"
old_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v1.2.2"
(campaign_webui / "package.json").write_text(
json.dumps({"name": dependency, "version": "1.2.3"})
)
(core_webui / "package.release.json").write_text(
json.dumps({"dependencies": {dependency: old_ref}})
)
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"dependencies": {dependency: old_ref}},
f"node_modules/{dependency}": {"version": "1.2.2"},
}
}
)
)
issues = selected_release_webui_bundle_issues(
repo_versions={"govoplan-core": "2.0.0", "govoplan-campaign": "1.2.3"},
workspace=workspace,
)
with self.assertRaisesRegex(ValueError, "Core release WebUI dependency"):
enforce_selected_version_alignment(
repo_versions={"govoplan-campaign": "1.2.3"},
workspace=workspace,
)
self.assertEqual(2, len(issues))
self.assertTrue(all(issue.repo == "govoplan-campaign" for issue in issues))
self.assertEqual(
{
"Core release WebUI dependency must match the selected module version",
"Core release lock must resolve the selected module version",
},
{issue.message for issue in issues},
)
def test_selected_webui_gate_ignores_unselected_module_pins(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
core_webui = workspace / "govoplan-core" / "webui"
campaign_webui = workspace / "govoplan-campaign" / "webui"
core_webui.mkdir(parents=True)
campaign_webui.mkdir(parents=True)
campaign_package = "@govoplan/campaign-webui"
files_package = "@govoplan/files-webui"
campaign_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v1.2.3"
files_ref = "git+ssh://git@example.test/acme/govoplan-files.git#v0.9.0"
(campaign_webui / "package.json").write_text(
json.dumps({"name": campaign_package, "version": "1.2.3"})
)
dependencies = {campaign_package: campaign_ref, files_package: files_ref}
(core_webui / "package.release.json").write_text(json.dumps({"dependencies": dependencies}))
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"dependencies": dependencies},
f"node_modules/{campaign_package}": {"version": "1.2.3"},
f"node_modules/{files_package}": {"version": "0.9.0"},
}
}
)
)
issues = selected_release_webui_bundle_issues(
repo_versions={"govoplan-campaign": "1.2.3"},
workspace=workspace,
)
self.assertEqual((), issues)
def test_selective_plan_blocks_mismatched_selected_webui_bundle(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
core_webui = workspace / "govoplan-core" / "webui"
campaign_webui = workspace / "govoplan-campaign" / "webui"
core_webui.mkdir(parents=True)
campaign_webui.mkdir(parents=True)
package_name = "@govoplan/campaign-webui"
old_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v1.2.2"
(campaign_webui / "package.json").write_text(
json.dumps({"name": package_name, "version": "1.2.3"})
)
(core_webui / "package.release.json").write_text(
json.dumps({"dependencies": {package_name: old_ref}})
)
(core_webui / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"dependencies": {package_name: old_ref}},
f"node_modules/{package_name}": {"version": "1.2.2"},
}
}
)
)
campaign = RepositorySnapshot(
spec=RepositorySpec(
name="govoplan-campaign",
category="module",
subtype="domain",
remote="git@example.test:acme/govoplan-campaign.git",
path="govoplan-campaign",
),
absolute_path=str(workspace / "govoplan-campaign"),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject="1.2.3", webui_package="1.2.3"),
)
dashboard = ReleaseDashboard(
generated_at="2026-01-01T00:00:00Z",
meta_root=str(workspace / "govoplan"),
workspace_root=str(workspace),
target_version=None,
target_tag=None,
online=False,
include_migrations=False,
summary=DashboardSummary(
repository_count=1,
missing_count=0,
dirty_count=0,
ahead_count=0,
behind_count=0,
no_head_count=0,
error_count=0,
safe_directory_count=0,
local_target_tag_missing_count=0,
status="ready",
),
repositories=(campaign,),
catalog=CatalogSnapshot(
channel="stable",
website_path="",
catalog_path="",
catalog_exists=False,
keyring_path="",
keyring_exists=False,
),
)
plan = build_selective_release_plan(
dashboard,
selected_repos=("govoplan-campaign",),
repo_versions={"govoplan-campaign": "1.2.3"},
)
self.assertEqual("blocked", plan.status)
self.assertEqual("blocked", plan.units[0].status)
self.assertTrue(any("Core release WebUI dependency" in item for item in plan.units[0].blockers))
finding = next(
item
for item in plan.gate_findings
if item.code == "release_webui_composition_alignment"
)
self.assertIn("package.release.json", finding.source)
self.assertIn("regenerate webui/package-lock.release.json", finding.remediation)
self.assertEqual("resolve_release_gate", plan.recommended_action.id)
def test_selected_repository_gate_reports_missing_version_metadata(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
(workspace / "govoplan-files").mkdir()
issues = selected_repository_version_issues(
repo_versions={"govoplan-files": "1.2.3"},
workspace=workspace,
)
self.assertEqual(1, len(issues))
self.assertEqual("repository has no version metadata", issues[0].message)
def test_selective_release_unit_is_blocked_by_version_drift(self) -> None:
repo = RepositorySnapshot(
spec=RepositorySpec(
name="govoplan-example",
category="module",
subtype="domain",
remote="git@example.test:acme/govoplan-example.git",
path="govoplan-example",
),
absolute_path="/tmp/govoplan-example",
exists=True,
is_git=True,
has_head=True,
versions=VersionSnapshot(pyproject="1.2.3", webui_package="1.2.4", manifests=("1.2.3",)),
)
unit = build_unit(repo, target_version="1.2.5", contracts=None)
self.assertEqual("blocked", unit.status)
self.assertIn("version metadata is not aligned", unit.blockers[0])
def test_repository_versions_and_lockfiles_must_match(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp) / "govoplan-example"
(root / "src" / "govoplan_example" / "backend").mkdir(parents=True)
(root / "webui").mkdir()
(root / "pyproject.toml").write_text('[project]\nname = "govoplan-example"\nversion = "1.2.3"\n')
(root / "package.json").write_text('{"name":"@govoplan/example","version":"1.2.3"}\n')
(root / "webui" / "package.json").write_text('{"name":"@govoplan/example-webui","version":"1.2.4"}\n')
(root / "webui" / "package-lock.json").write_text(
json.dumps({"packages": {"": {"version": "1.2.2"}}})
)
(root / "src" / "govoplan_example" / "backend" / "manifest.py").write_text(
'manifest = ModuleManifest(\n version="1.2.3",\n)\n'
)
issues = repository_version_issues(root)
self.assertEqual(2, len(issues))
self.assertEqual({"webui/package.json", "webui/package-lock.json"}, {issue.source for issue in issues})
def test_public_runtime_version_must_match_package_metadata(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp) / "govoplan-example"
package = root / "src" / "govoplan_example"
package.mkdir(parents=True)
(root / "pyproject.toml").write_text(
'[project]\nname = "govoplan-example"\nversion = "1.2.3"\n'
)
(package / "__init__.py").write_text('__version__ = "1.2.2"\n')
issues = repository_version_issues(root)
self.assertEqual(1, len(issues))
self.assertEqual("package __init__.py 1", issues[0].source)
self.assertEqual("1.2.3", issues[0].expected)
self.assertEqual("1.2.2", issues[0].actual)
def test_release_backend_and_frontend_refs_must_match(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
meta = Path(tmp) / "govoplan"
core = Path(tmp) / "govoplan-core"
meta.mkdir()
(core / "webui").mkdir(parents=True)
(meta / "requirements-release.txt").write_text(
"govoplan-example @ git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3\n"
)
(core / "webui" / "package.release.json").write_text(
json.dumps(
{
"dependencies": {
"@govoplan/example-webui": (
"git+ssh://git@example.test/acme/govoplan-example.git#v1.2.4"
)
}
}
)
)
issues = release_composition_issues(meta, core_root=core)
self.assertEqual(1, len(issues))
self.assertEqual("1.2.3", issues[0].expected)
self.assertEqual("1.2.4", issues[0].actual)
def test_release_refs_must_point_at_installable_versioned_artifacts(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
module = workspace / "govoplan-example"
meta.mkdir()
(core / "webui").mkdir(parents=True)
module.mkdir()
(meta / "requirements-release.txt").write_text(
"govoplan-example @ git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3\n"
)
(core / "webui" / "package.release.json").write_text(
json.dumps(
{
"dependencies": {
"@govoplan/example-webui": (
"git+ssh://git@example.test/acme/govoplan-example.git#v1.2.3"
)
}
}
)
)
subprocess.run(["git", "init", "-q", str(module)], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.email", "test@example.test"], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.name", "Test"], check=True)
(module / "README.md").write_text("uninstallable release\n")
subprocess.run(["git", "-C", str(module), "add", "README.md"], check=True)
subprocess.run(["git", "-C", str(module), "commit", "-qm", "release"], check=True)
subprocess.run(["git", "-C", str(module), "tag", "v1.2.3"], check=True)
issues = release_composition_issues(meta, core_root=core)
self.assertEqual(2, len(issues))
self.assertEqual(
{"v1.2.3:pyproject.toml", "v1.2.3:webui/package.json"},
{issue.source for issue in issues},
)
self.assertTrue(all("must contain" in issue.message for issue in issues))
def test_annotated_release_tags_work_with_sanitized_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
module = workspace / "govoplan-example"
meta.mkdir()
(core / "webui").mkdir(parents=True)
(module / "webui").mkdir(parents=True)
backend_ref = "git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3"
webui_ref = "git+ssh://git@example.test/acme/govoplan-example.git#v1.2.3"
(meta / "requirements-release.txt").write_text(f"govoplan-example @ {backend_ref}\n")
(module / "pyproject.toml").write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
(module / "webui" / "package.json").write_text(
'{"name":"@govoplan/example-webui","version":"1.2.3"}\n'
)
subprocess.run(["git", "init", "-q", str(module)], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.email", "test@example.test"], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.name", "Test"], check=True)
subprocess.run(["git", "-C", str(module), "add", "pyproject.toml", "webui/package.json"], check=True)
subprocess.run(["git", "-C", str(module), "commit", "-qm", "release"], check=True)
subprocess.run(
["git", "-C", str(module), "tag", "-a", "v1.2.3", "-m", "Release v1.2.3"],
check=True,
)
tagged_commit = subprocess.run(
["git", "-C", str(module), "rev-parse", "refs/tags/v1.2.3^{commit}"],
check=True,
text=True,
stdout=subprocess.PIPE,
).stdout.strip()
dependencies = {"@govoplan/example-webui": webui_ref}
(core / "pyproject.toml").write_text('[project]\nname="govoplan-core"\nversion="2.0.0"\n')
(core / "webui" / "package.release.json").write_text(
json.dumps({"version": "2.0.0", "dependencies": dependencies})
)
(core / "webui" / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"version": "2.0.0", "dependencies": dependencies},
"node_modules/@govoplan/example-webui": {
"version": "1.2.3",
"resolved": f"git+ssh://git@example.test/acme/govoplan-example.git#{tagged_commit}",
},
}
}
)
)
git_environment = sanitized_git_environment()
git_environment["GIT_TEST_ASSUME_DIFFERENT_OWNER"] = "1"
with patch(
"govoplan_release.version_alignment.sanitized_git_environment",
return_value=git_environment,
):
composition_issues = release_composition_issues(meta, core_root=core)
core_issues = repository_version_issues(core)
self.assertEqual((), composition_issues)
self.assertEqual((), core_issues)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,451 @@
#!/usr/bin/env python3
"""Re-run a capability-fit assessment against a trusted release catalog."""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import re
import sys
META_ROOT = Path(__file__).resolve().parents[2]
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
for tools_root in (ASSESSMENT_TOOLS_ROOT, RELEASE_TOOLS_ROOT):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from govoplan_assessment import ( # noqa: E402
collect_installed_composition,
render_review,
review_capability_fit,
)
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
from govoplan_assessment.evidence import validate_payload # noqa: E402
from govoplan_release.catalog import DEFAULT_PUBLIC_BASE_URL, fetch_json # noqa: E402
TRUSTED_KEYRING_FILE_ENV = "GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE"
MAX_ASSESSMENT_INPUT_BYTES = 16 * 1024 * 1024
MAX_EVIDENCE_INPUT_BYTES = 4 * 1024 * 1024
MAX_REPORT_OUTPUT_BYTES = 16 * 1024 * 1024
OPAQUE_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,159}$")
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Validate a fit assessment and identify conclusions needing review after release drift."
)
parser.add_argument(
"--assessment",
type=Path,
default=META_ROOT / "docs" / "capability-fit-current.json",
)
parser.add_argument(
"--schema",
type=Path,
default=META_ROOT / "docs" / "capability-fit.schema.json",
)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument(
"--public",
action="store_true",
help="Use the fixed public GovOPlaN stable catalog and keyring URLs.",
)
source.add_argument(
"--catalog", type=Path, help="Read a catalog from a local file."
)
parser.add_argument(
"--keyring",
type=Path,
help="Published or candidate keyring whose hash is pinned by --catalog.",
)
parser.add_argument(
"--trusted-keyring",
type=Path,
help=(
"Independently pinned local keyring used as the catalog signature trust "
f"root; defaults to ${TRUSTED_KEYRING_FILE_ENV}."
),
)
parser.add_argument(
"--workspace-root",
type=Path,
default=META_ROOT.parent,
help="Workspace containing repository checkouts for local tag provenance.",
)
parser.add_argument(
"--skip-tag-provenance",
action="store_true",
help="Compare signed metadata without checking local annotated tags.",
)
installed = parser.add_mutually_exclusive_group()
installed.add_argument(
"--installed-evidence",
type=Path,
help="Validate an existing bounded installed-composition evidence document.",
)
installed.add_argument(
"--collect-installed-evidence",
type=Path,
help=(
"Collect the current interpreter's GovOPlaN distributions, write the "
"bounded evidence document, and include it in this review. Loading "
"module entry points executes installed GovOPlaN manifest factories."
),
)
parser.add_argument(
"--installed-evidence-schema",
type=Path,
default=META_ROOT / "docs" / "installed-composition-evidence.schema.json",
)
parser.add_argument(
"--installer-receipt",
type=Path,
help="Signed installer receipt binding installed payloads to this catalog.",
)
parser.add_argument(
"--installer-receipt-schema",
type=Path,
default=META_ROOT / "docs" / "installer-receipt.schema.json",
)
parser.add_argument(
"--installer-authority-keyring",
type=Path,
help=(
"Independently provisioned, role-scoped trust root for installer "
"receipt signatures."
),
)
parser.add_argument(
"--installer-authority-keyring-schema",
type=Path,
default=META_ROOT
/ "docs"
/ "installer-receipt-authority-keyring.schema.json",
)
parser.add_argument(
"--boundary-evidence",
type=Path,
help=(
"Externally issued target/provider/production proof bound to the "
"installed-evidence digest."
),
)
parser.add_argument(
"--boundary-evidence-schema",
type=Path,
default=META_ROOT / "docs" / "capability-fit-boundary-evidence.schema.json",
)
parser.add_argument(
"--boundary-authority-keyring",
type=Path,
help=(
"Independently provisioned keyring authorizing proof signers for "
"specific target/provider/production scopes."
),
)
parser.add_argument(
"--boundary-authority-keyring-schema",
type=Path,
default=META_ROOT
/ "docs"
/ "capability-fit-proof-authority-keyring.schema.json",
)
parser.add_argument(
"--verification-time",
help="Optional ISO-8601 time for reproducible boundary-proof verification.",
)
parser.add_argument(
"--expected-external-provider-subject",
help=(
"Opaque provider-environment ID that external-provider proof must match; "
"never supply a URL, credential, or endpoint identifier."
),
)
parser.add_argument(
"--json", action="store_true", help="Print the machine-readable review report."
)
parser.add_argument(
"--output",
type=Path,
help="Also write the machine-readable review report to this path.",
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
if args.catalog is not None and args.keyring is None:
raise SystemExit("--keyring is required with --catalog")
if args.public and args.keyring is not None:
raise SystemExit("--keyring cannot be combined with --public")
if (args.boundary_evidence is None) != (args.boundary_authority_keyring is None):
raise SystemExit(
"--boundary-evidence and --boundary-authority-keyring are required together"
)
if (args.installer_receipt is None) != (
args.installer_authority_keyring is None
):
raise SystemExit(
"--installer-receipt and --installer-authority-keyring are required together"
)
if args.installer_receipt is not None and (
args.installed_evidence is None and args.collect_installed_evidence is None
):
raise SystemExit(
"--installer-receipt requires --installed-evidence or --collect-installed-evidence"
)
if args.boundary_evidence is not None and (
args.installed_evidence is None and args.collect_installed_evidence is None
):
raise SystemExit(
"--boundary-evidence requires --installed-evidence or --collect-installed-evidence"
)
if (
args.expected_external_provider_subject is not None
and OPAQUE_ID_PATTERN.fullmatch(args.expected_external_provider_subject) is None
):
raise SystemExit(
"--expected-external-provider-subject must be a bounded opaque ID"
)
if (
args.collect_installed_evidence is not None
and args.output is not None
and args.collect_installed_evidence.resolve() == args.output.resolve()
):
raise SystemExit("Evidence and review output paths must differ")
configured_trusted_keyring = os.getenv(TRUSTED_KEYRING_FILE_ENV, "").strip()
trusted_keyring_path = args.trusted_keyring or (
Path(configured_trusted_keyring) if configured_trusted_keyring else None
)
if trusted_keyring_path is None:
raise SystemExit(
f"--trusted-keyring or ${TRUSTED_KEYRING_FILE_ENV} is required"
)
assessment = read_object(
args.assessment,
label="assessment",
max_bytes=MAX_ASSESSMENT_INPUT_BYTES,
)
schema = read_object(
args.schema,
label="schema",
max_bytes=MAX_ASSESSMENT_INPUT_BYTES,
)
if args.public:
catalog = fetch_public_json(
f"{DEFAULT_PUBLIC_BASE_URL}/catalogs/v1/channels/stable.json",
label="catalog",
)
published_keyring = fetch_public_json(
f"{DEFAULT_PUBLIC_BASE_URL}/catalogs/v1/keyring.json",
label="published keyring",
)
else:
catalog = read_object(
args.catalog, label="catalog", max_bytes=MAX_ASSESSMENT_INPUT_BYTES
)
published_keyring = read_object(
args.keyring,
label="published keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
trusted_keyring = read_object(
trusted_keyring_path,
label="trusted keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installed_evidence = None
installed_evidence_schema = None
if (
args.installed_evidence is not None
or args.collect_installed_evidence is not None
):
installed_evidence_schema = read_object(
args.installed_evidence_schema,
label="installed evidence schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
if args.installed_evidence is not None:
installed_evidence = read_object(
args.installed_evidence,
label="installed evidence",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
else:
installed_evidence = collect_installed_composition(assessment=assessment)
collection_schema_errors = validate_payload(
payload=installed_evidence,
schema=installed_evidence_schema,
)
if collection_schema_errors:
raise SystemExit(
"Collected installed evidence did not satisfy its bounded schema"
)
write_object(
args.collect_installed_evidence,
installed_evidence,
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
label="installed evidence",
)
boundary_evidence = None
boundary_evidence_schema = None
boundary_authority_keyring = None
boundary_authority_keyring_schema = None
if args.boundary_evidence is not None:
boundary_evidence = read_object(
args.boundary_evidence,
label="boundary evidence",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
boundary_evidence_schema = read_object(
args.boundary_evidence_schema,
label="boundary evidence schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
boundary_authority_keyring = read_object(
args.boundary_authority_keyring,
label="boundary authority keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
boundary_authority_keyring_schema = read_object(
args.boundary_authority_keyring_schema,
label="boundary authority keyring schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_receipt = None
installer_receipt_schema = None
installer_authority_keyring = None
installer_authority_keyring_schema = None
if args.installer_receipt is not None:
installer_receipt = read_object(
args.installer_receipt,
label="installer receipt",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_receipt_schema = read_object(
args.installer_receipt_schema,
label="installer receipt schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_authority_keyring = read_object(
args.installer_authority_keyring,
label="installer authority keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_authority_keyring_schema = read_object(
args.installer_authority_keyring_schema,
label="installer authority keyring schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
verification_time = parse_datetime(args.verification_time)
report = review_capability_fit(
assessment=assessment,
schema=schema,
catalog=catalog,
published_keyring=published_keyring,
trusted_keyring=trusted_keyring,
workspace_root=None
if args.skip_tag_provenance
else args.workspace_root.resolve(),
installed_evidence=installed_evidence,
installed_evidence_schema=installed_evidence_schema,
installer_receipt=installer_receipt,
installer_receipt_schema=installer_receipt_schema,
installer_authority_keyring=installer_authority_keyring,
installer_authority_keyring_schema=installer_authority_keyring_schema,
boundary_evidence=boundary_evidence,
boundary_evidence_schema=boundary_evidence_schema,
boundary_authority_keyring=boundary_authority_keyring,
boundary_authority_keyring_schema=boundary_authority_keyring_schema,
verification_time=verification_time,
installed_evidence_mode=(
"direct_local"
if args.collect_installed_evidence is not None
else "imported_unsigned"
),
expected_external_provider_subject=args.expected_external_provider_subject,
)
encoded = json.dumps(report, indent=2, sort_keys=True) + "\n"
if args.output is not None:
write_object(
args.output,
report,
max_bytes=MAX_REPORT_OUTPUT_BYTES,
label="review report",
)
sys.stdout.write(encoded if args.json else render_review(report))
return {"current": 0, "blocked": 1, "review_required": 2}[report["status"]]
def read_object(path: Path | None, *, label: str, max_bytes: int) -> dict[str, object]:
if path is None:
raise SystemExit(f"Missing {label} path")
try:
if path.stat().st_size > max_bytes:
raise SystemExit(
f"{label.capitalize()} exceeds the {max_bytes}-byte input limit"
)
with path.open("rb") as handle:
encoded = handle.read(max_bytes + 1)
if len(encoded) > max_bytes:
raise SystemExit(
f"{label.capitalize()} exceeds the {max_bytes}-byte input limit"
)
payload = json.loads(encoded.decode("utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise SystemExit(f"Could not read {label} {path}: {exc}") from exc
if not isinstance(payload, dict):
raise SystemExit(f"{label.capitalize()} must be a JSON object: {path}")
return payload
def write_object(
path: Path | None,
payload: dict[str, object],
*,
max_bytes: int,
label: str,
) -> None:
if path is None:
raise SystemExit(f"Missing {label} output path")
try:
atomic_write_json(path, payload, max_bytes=max_bytes)
except AtomicJsonWriteError as exc:
raise SystemExit(f"Could not securely write {label}") from exc
def parse_datetime(value: str | None):
if value is None:
return None
from datetime import datetime
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise SystemExit("--verification-time must be an ISO-8601 timestamp") from exc
if parsed.tzinfo is None:
raise SystemExit("--verification-time must include a timezone")
return parsed
def fetch_public_json(url: str, *, label: str) -> dict[str, object]:
result = fetch_json(url)
if result.get("ok") is not True or not isinstance(result.get("payload"), dict):
raise SystemExit(f"Could not fetch fixed public {label} endpoint.")
return result["payload"]
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,10 @@
"""Repeatable GovOPlaN product-assessment tooling."""
from .capability_fit import review_capability_fit, render_review
from .evidence import collect_installed_composition
__all__ = (
"collect_installed_composition",
"render_review",
"review_capability_fit",
)

View File

@@ -0,0 +1,230 @@
"""Bounded, private, atomic JSON output for assessment evidence."""
from __future__ import annotations
import json
import os
from pathlib import Path
import stat
import tempfile
from typing import Any
class AtomicJsonWriteError(RuntimeError):
"""Raised when JSON output cannot be written with the required guarantees."""
def atomic_write_json(
path: str | os.PathLike[str],
payload: Any,
*,
max_bytes: int,
) -> None:
"""Write bounded JSON through a private same-directory temporary file.
A successful return means the file contents and the containing directory
entry have both been flushed. If flushing the directory fails after the
atomic replacement, the replacement may be visible but its crash
durability is unknown, so the function reports failure. Every parent must
already exist as a real directory; symbolic-link components are rejected.
"""
if isinstance(max_bytes, bool) or not isinstance(max_bytes, int) or max_bytes <= 0:
raise ValueError("max_bytes must be a positive integer")
encoded = _encode_bounded_json(payload, max_bytes=max_bytes)
target = Path(os.path.abspath(os.fspath(path)))
parent = target.parent
descriptor = -1
verification_descriptor = -1
temporary_path: Path | None = None
temporary_identity: tuple[int, int] | None = None
try:
_assert_real_parent(parent)
_assert_replaceable_target(target)
try:
descriptor, temporary_name = tempfile.mkstemp(
dir=parent,
prefix=".govoplan-json-",
suffix=".tmp",
)
temporary_path = Path(temporary_name)
os.fchmod(descriptor, 0o600)
metadata = os.fstat(descriptor)
if (
not stat.S_ISREG(metadata.st_mode)
or stat.S_IMODE(metadata.st_mode) != 0o600
):
raise AtomicJsonWriteError(
"Atomic JSON temporary file could not be secured."
)
temporary_identity = (metadata.st_dev, metadata.st_ino)
verification_descriptor = os.dup(descriptor)
handle = os.fdopen(descriptor, "wb")
descriptor = -1
with handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
# Recheck after creating and flushing the temporary file so an
# unsafe target discovered before replacement is never followed.
_assert_real_parent(parent)
_assert_replaceable_target(target)
os.replace(temporary_path, target)
temporary_path = None
_assert_installed_target(
target,
expected_identity=temporary_identity,
recovery_descriptor=verification_descriptor,
)
_fsync_directory(parent)
finally:
try:
if descriptor >= 0:
os.close(descriptor)
finally:
try:
if verification_descriptor >= 0:
os.close(verification_descriptor)
finally:
if temporary_path is not None:
try:
temporary_path.unlink()
except FileNotFoundError:
pass
except AtomicJsonWriteError:
raise
except OSError as exc:
raise AtomicJsonWriteError(
"JSON output could not be written atomically."
) from exc
def _encode_bounded_json(payload: Any, *, max_bytes: int) -> bytes:
encoder = json.JSONEncoder(
allow_nan=False,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
chunks: list[bytes] = []
encoded_size = 1 # The document always ends with one newline.
for chunk in encoder.iterencode(payload):
encoded_chunk = chunk.encode("utf-8")
encoded_size += len(encoded_chunk)
if encoded_size > max_bytes:
raise AtomicJsonWriteError("JSON output exceeds its size limit.")
chunks.append(encoded_chunk)
return b"".join(chunks) + b"\n"
def _assert_replaceable_target(path: Path) -> None:
try:
metadata = path.lstat()
except FileNotFoundError:
return
except OSError as exc:
raise AtomicJsonWriteError(
"JSON output target could not be inspected safely."
) from exc
if stat.S_ISLNK(metadata.st_mode):
raise AtomicJsonWriteError("JSON output target must not be a symbolic link.")
if not stat.S_ISREG(metadata.st_mode):
raise AtomicJsonWriteError("JSON output target must be a regular file.")
def _assert_real_parent(path: Path) -> None:
current = Path(path.anchor)
for part in path.parts[1:]:
current /= part
try:
metadata = current.lstat()
except FileNotFoundError as exc:
raise AtomicJsonWriteError(
"JSON output parent directory must already exist."
) from exc
except OSError as exc:
raise AtomicJsonWriteError(
"JSON output parent directory could not be inspected safely."
) from exc
if stat.S_ISLNK(metadata.st_mode):
raise AtomicJsonWriteError(
"JSON output parent path must not contain symbolic links."
)
if not stat.S_ISDIR(metadata.st_mode):
raise AtomicJsonWriteError(
"JSON output parent path must contain only directories."
)
def _assert_installed_target(
path: Path,
*,
expected_identity: tuple[int, int] | None,
recovery_descriptor: int,
) -> None:
try:
metadata = path.lstat()
except OSError as exc:
raise AtomicJsonWriteError(
"Atomic JSON replacement could not be verified."
) from exc
observed_identity = (metadata.st_dev, metadata.st_ino)
if (
expected_identity is None
or observed_identity != expected_identity
or not stat.S_ISREG(metadata.st_mode)
):
raise AtomicJsonWriteError(
"Atomic JSON replacement did not preserve the secured regular file."
)
if stat.S_IMODE(metadata.st_mode) == 0o600:
return
try:
recovery_metadata = os.fstat(recovery_descriptor)
if (
not stat.S_ISREG(recovery_metadata.st_mode)
or (recovery_metadata.st_dev, recovery_metadata.st_ino) != expected_identity
):
raise AtomicJsonWriteError(
"Atomic JSON replacement recovery descriptor is invalid."
)
os.fchmod(recovery_descriptor, 0o600)
os.fsync(recovery_descriptor)
recovered_metadata = os.fstat(recovery_descriptor)
installed_metadata = path.lstat()
except OSError as exc:
raise AtomicJsonWriteError(
"Atomic JSON replacement permissions could not be restored."
) from exc
if (
stat.S_IMODE(recovered_metadata.st_mode) != 0o600
or (installed_metadata.st_dev, installed_metadata.st_ino) != expected_identity
or not stat.S_ISREG(installed_metadata.st_mode)
or stat.S_IMODE(installed_metadata.st_mode) != 0o600
):
raise AtomicJsonWriteError(
"Atomic JSON replacement permissions could not be restored."
)
_fsync_directory(path.parent)
raise AtomicJsonWriteError(
"Atomic JSON replacement permissions changed and were restored."
)
def _fsync_directory(path: Path) -> None:
flags = os.O_RDONLY
if hasattr(os, "O_DIRECTORY"):
flags |= os.O_DIRECTORY
if hasattr(os, "O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
descriptor = os.open(path, flags)
try:
if not stat.S_ISDIR(os.fstat(descriptor).st_mode):
raise AtomicJsonWriteError("JSON output parent path must be a directory.")
os.fsync(descriptor)
finally:
os.close(descriptor)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,204 @@
"""Issuance of role-scoped receipts from same-process installed observations."""
from __future__ import annotations
import base64
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, Mapping
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from .capability_fit import canonical_hash
from .evidence import (
MAX_LOCAL_OBSERVATION_AGE,
OPAQUE_ID_PATTERN,
_catalog_artifact_identities,
_record_integrity_semantics_valid,
canonical_bytes,
canonical_sha256,
normalize_package_name,
)
from govoplan_release.artifact_identity import inspect_python_wheel
def issue_installer_receipt(
*,
assessment: Mapping[str, Any],
catalog: Mapping[str, Any],
installed_evidence: dict[str, Any],
receipt_id: str,
key_id: str,
private_key: Ed25519PrivateKey,
consumed_artifacts: Mapping[str, Path | str] | None = None,
issued_at: datetime | None = None,
same_process_observation: bool = False,
) -> dict[str, Any]:
"""Sign a receipt only for a fresh observation collected by this process.
File-based evidence is intentionally not an admitted issuance input. The CLI
collector calls this function directly with its in-memory observation.
"""
if same_process_observation is not True:
raise ValueError("installer receipts require a same-process observation")
if OPAQUE_ID_PATTERN.fullmatch(receipt_id) is None:
raise ValueError("receipt ID must be a bounded opaque ID")
if OPAQUE_ID_PATTERN.fullmatch(key_id) is None:
raise ValueError("installer signing key ID must be a bounded opaque ID")
if not isinstance(consumed_artifacts, Mapping) or not consumed_artifacts:
raise ValueError("installer receipt issuance requires exact consumed wheels")
release = catalog.get("release")
raw_artifacts = release.get("artifacts") if isinstance(release, Mapping) else None
catalog_artifacts, artifact_findings = _catalog_artifact_identities(raw_artifacts)
if artifact_findings:
raise ValueError("signed catalog artifact manifest is invalid")
evidence_rows = installed_evidence.get("artifacts")
if not isinstance(evidence_rows, list) or not evidence_rows:
raise ValueError("installed evidence has no artifact observations")
expected_packages = _expected_packages(assessment=assessment, catalog=catalog)
receipt_rows: list[dict[str, Any]] = []
seen: set[str] = set()
for artifact in evidence_rows:
if not isinstance(artifact, dict):
raise ValueError("installed evidence contains a malformed artifact")
package_name = normalize_package_name(str(artifact.get("package_name") or ""))
package_version = str(artifact.get("package_version") or "")
if package_name in seen:
raise ValueError("installed evidence contains duplicate packages")
seen.add(package_name)
catalog_artifact = catalog_artifacts.get(package_name)
record = artifact.get("record_integrity")
installed_payload = (
record.get("installed_payload_identity")
if isinstance(record, Mapping)
else None
)
if (
catalog_artifact is None
or catalog_artifact.get("package_version") != package_version
or not isinstance(record, Mapping)
or record.get("status") != "verified"
or not _record_integrity_semantics_valid(record)
or not isinstance(installed_payload, Mapping)
):
raise ValueError(
"installed payload is not a verified match for a catalog artifact"
)
artifact_path = consumed_artifacts.get(package_name)
if artifact_path is None:
raise ValueError("every installed package requires its exact consumed wheel")
consumed = inspect_python_wheel(artifact_path)
if (
consumed.package_name != package_name
or consumed.package_version != package_version
or consumed.archive_sha256 != catalog_artifact.get("archive_sha256")
or consumed.catalog_payload().get("installed_payload")
!= catalog_artifact.get("installed_payload")
or record.get("artifact_payload_identity")
!= catalog_artifact.get("installed_payload")
):
raise ValueError(
"consumed wheel or observed wheel payload differs from the signed catalog identity"
)
receipt_rows.append(
{
"package_name": package_name,
"package_version": package_version,
"catalog_archive_sha256": catalog_artifact["archive_sha256"],
"installed_payload": dict(installed_payload),
}
)
if seen != expected_packages:
raise ValueError(
"installed evidence must contain exactly the enabled assessed catalog packages"
)
if set(consumed_artifacts) != expected_packages:
raise ValueError("consumed wheel set must exactly match enabled packages")
receipt_rows.sort(key=lambda item: (item["package_name"], item["package_version"]))
observed_at = issued_at or datetime.now(tz=UTC)
if observed_at.tzinfo is None:
raise ValueError("installer receipt issuance time must include a timezone")
collected_at = _datetime(installed_evidence.get("collected_at"))
if (
collected_at is None
or observed_at < collected_at
or observed_at - collected_at > MAX_LOCAL_OBSERVATION_AGE
):
raise ValueError(
"installer receipt issuance must follow live collection within five minutes"
)
assessment_release = assessment.get("release")
assessment_release_ref = (
assessment_release.get("ref")
if isinstance(assessment_release, Mapping)
else None
)
receipt: dict[str, Any] = {
"$schema": "./installer-receipt.schema.json",
"schema_version": "0.1.0",
"evidence_kind": "govoplan.installer-receipt",
"receipt_id": receipt_id,
"assessment_id": assessment.get("assessment_id"),
"assessment_release": assessment_release_ref,
"installed_evidence_sha256": canonical_sha256(installed_evidence),
"catalog": {
"channel": catalog.get("channel"),
"sequence": catalog.get("sequence"),
"sha256": canonical_hash(catalog),
},
"issued_at": observed_at.astimezone(UTC).isoformat().replace("+00:00", "Z"),
"artifacts": receipt_rows,
}
receipt["signatures"] = [
{
"algorithm": "ed25519",
"key_id": key_id,
"value": base64.b64encode(
private_key.sign(canonical_bytes(receipt))
).decode("ascii"),
}
]
return receipt
def _expected_packages(
*, assessment: Mapping[str, Any], catalog: Mapping[str, Any]
) -> set[str]:
entries: dict[str, Mapping[str, Any]] = {}
core = catalog.get("core_release")
if isinstance(core, Mapping):
entries["core"] = core
modules = catalog.get("modules")
if isinstance(modules, list):
for entry in modules:
if isinstance(entry, Mapping) and isinstance(entry.get("module_id"), str):
entries[str(entry["module_id"])] = entry
expected: set[str] = set()
composition = assessment.get("composition")
if not isinstance(composition, list):
raise ValueError("assessment composition is unavailable")
for component in composition:
if not isinstance(component, Mapping) or component.get("enabled") is not True:
continue
entry = entries.get(str(component.get("module_id") or ""))
package = entry.get("python_package") if isinstance(entry, Mapping) else None
if not isinstance(package, str):
raise ValueError("enabled assessment component has no catalog package")
normalized = normalize_package_name(package)
if normalized in expected:
raise ValueError("enabled assessment packages are ambiguous")
expected.add(normalized)
if not expected:
raise ValueError("assessment has no enabled catalog packages")
return expected
def _datetime(value: object) -> datetime | None:
if not isinstance(value, str):
return None
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return None
return parsed.astimezone(UTC) if parsed.tzinfo is not None else None

View File

@@ -0,0 +1,170 @@
#!/usr/bin/env python3
"""Collect live installed payloads and issue an independently signed receipt."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
META_ROOT = Path(__file__).resolve().parents[2]
for tools_root in (META_ROOT / "tools" / "assessments", META_ROOT / "tools" / "release"):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from cryptography.hazmat.primitives import serialization # noqa: E402
from cryptography.hazmat.primitives.asymmetric.ed25519 import ( # noqa: E402
Ed25519PrivateKey,
)
from govoplan_assessment import collect_installed_composition # noqa: E402
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
from govoplan_assessment.capability_fit import ( # noqa: E402
canonical_hash,
review_capability_fit,
)
from govoplan_assessment.evidence import validate_payload # noqa: E402
from govoplan_assessment.installer_receipt import ( # noqa: E402
issue_installer_receipt,
)
MAX_INPUT_BYTES = 16 * 1024 * 1024
MAX_OUTPUT_BYTES = 4 * 1024 * 1024
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--assessment", type=Path, required=True)
parser.add_argument("--assessment-schema", type=Path, default=META_ROOT / "docs" / "capability-fit.schema.json")
parser.add_argument("--catalog", type=Path, required=True)
parser.add_argument("--keyring", type=Path, required=True)
parser.add_argument("--trusted-keyring", type=Path, required=True)
parser.add_argument("--receipt-id", required=True)
parser.add_argument("--signing-key", required=True, metavar="KEY_ID=/PATH/PRIVATE.pem")
parser.add_argument(
"--python-artifact",
action="append",
default=[],
required=True,
metavar="PACKAGE=/PATH/TO/CONSUMED.whl",
help="Exact wheel consumed by this installer; repeat for every package.",
)
parser.add_argument("--evidence-output", type=Path, required=True)
parser.add_argument("--receipt-output", type=Path, required=True)
parser.add_argument("--installed-evidence-schema", type=Path, default=META_ROOT / "docs" / "installed-composition-evidence.schema.json")
parser.add_argument("--receipt-schema", type=Path, default=META_ROOT / "docs" / "installer-receipt.schema.json")
args = parser.parse_args(argv)
if args.evidence_output.resolve() == args.receipt_output.resolve():
parser.error("evidence and receipt output paths must differ")
assessment = read_object(args.assessment, label="assessment")
assessment_schema = read_object(args.assessment_schema, label="assessment schema")
catalog = read_object(args.catalog, label="catalog")
keyring = read_object(args.keyring, label="published keyring")
trusted_keyring = read_object(args.trusted_keyring, label="trusted keyring")
evidence_schema = read_object(args.installed_evidence_schema, label="installed evidence schema")
receipt_schema = read_object(args.receipt_schema, label="installer receipt schema")
release = catalog.get("release")
if not isinstance(release, dict) or release.get("keyring_sha256") != canonical_hash(keyring):
parser.error("catalog does not pin the supplied published keyring")
evidence = collect_installed_composition(assessment=assessment)
if validate_payload(payload=evidence, schema=evidence_schema):
parser.error("live installed observation does not satisfy its bounded schema")
report = review_capability_fit(
assessment=assessment,
schema=assessment_schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=trusted_keyring,
workspace_root=None,
installed_evidence=evidence,
installed_evidence_schema=evidence_schema,
installed_evidence_mode="direct_local",
)
required_scopes = (
"catalog_signature_and_keyring",
"catalog_signature_and_trusted_keyring",
"published_keyring_hash",
"release_metadata",
"installed_artifacts",
"installed_record_integrity",
)
if any(report["proof_scope"].get(scope, {}).get("valid") is not True for scope in required_scopes):
parser.error("trusted catalog and live installed-composition checks must pass before receipt issuance")
key_id, private_key = read_signing_key(args.signing_key)
consumed_artifacts = parse_package_paths(tuple(args.python_artifact))
receipt = issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id=args.receipt_id,
key_id=key_id,
private_key=private_key,
consumed_artifacts=consumed_artifacts,
same_process_observation=True,
)
if validate_payload(payload=receipt, schema=receipt_schema):
parser.error("issued installer receipt does not satisfy its bounded schema")
write_object(args.evidence_output, evidence, label="installed evidence")
write_object(args.receipt_output, receipt, label="installer receipt")
print(json.dumps({"receipt_id": args.receipt_id, "evidence_output": str(args.evidence_output), "receipt_output": str(args.receipt_output)}, sort_keys=True))
return 0
def read_object(path: Path, *, label: str) -> dict[str, object]:
try:
if path.stat().st_size > MAX_INPUT_BYTES:
raise ValueError("input exceeds size limit")
encoded = path.read_bytes()
payload = json.loads(encoded.decode("utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError, ValueError) as exc:
raise SystemExit(f"Could not read {label}") from exc
if not isinstance(payload, dict):
raise SystemExit(f"{label.capitalize()} must be a JSON object")
return payload
def read_signing_key(value: str) -> tuple[str, Ed25519PrivateKey]:
key_id, separator, path_text = value.partition("=")
if not separator or not key_id or not path_text:
raise SystemExit("--signing-key must use KEY_ID=/path/to/private.pem")
try:
key = serialization.load_pem_private_key(Path(path_text).expanduser().read_bytes(), password=None)
except (OSError, ValueError) as exc:
raise SystemExit("Could not read installer signing key") from exc
if not isinstance(key, Ed25519PrivateKey):
raise SystemExit("Installer signing key must be Ed25519")
return key_id, key
def parse_package_paths(values: tuple[str, ...]) -> dict[str, Path]:
result: dict[str, Path] = {}
for value in values:
package, separator, path_text = value.partition("=")
package = package.strip()
path_text = path_text.strip()
if not separator or not package or not path_text or package in result:
raise SystemExit(
"--python-artifact must uniquely use PACKAGE=/path/to/consumed.whl"
)
result[package] = Path(path_text).expanduser()
return result
def write_object(path: Path, payload: dict[str, object], *, label: str) -> None:
try:
atomic_write_json(path, payload, max_bytes=MAX_OUTPUT_BYTES)
except AtomicJsonWriteError as exc:
raise SystemExit(f"Could not securely write {label}") from exc
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""Static cross-repository module contract check."""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(META_ROOT / "tools" / "release"))
from govoplan_release.contracts import ( # noqa: E402
collect_contracts,
format_version_range,
interface_impact_map,
provided_interface_map,
validate_contracts,
)
from govoplan_release.model import RepositorySnapshot, to_jsonable # noqa: E402
from govoplan_release.workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root # noqa: E402
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--workspace-root",
type=Path,
default=None,
help="Workspace root containing the repositories. Defaults to repositories.json default_parent.",
)
parser.add_argument("--include-website", action="store_true", help="Also scan website repositories.")
parser.add_argument("--json", action="store_true", help="Print machine-readable JSON.")
parser.add_argument("--no-impact", action="store_true", help="Omit human-readable provider impact output.")
parser.add_argument("--strict-warnings", action="store_true", help="Fail when warnings are present.")
args = parser.parse_args()
workspace_root = resolve_workspace_root(args.workspace_root)
repositories = repository_snapshots(workspace_root=workspace_root, include_website=args.include_website)
contracts = collect_contracts(repositories)
issues = validate_contracts(contracts)
providers = provided_interface_map(contracts)
consumers = interface_impact_map(contracts)
payload = {
"workspace_root": str(workspace_root),
"repository_count": len(repositories),
"contract_count": len(contracts),
"provider_count": sum(len(contract.provides_interfaces) for contract in contracts),
"requirement_count": sum(len(contract.requires_interfaces) for contract in contracts),
"providers": provider_payload(providers, consumers),
"issues": to_jsonable(issues),
}
blockers = [issue for issue in issues if issue.severity == "blocker"]
warnings = [issue for issue in issues if issue.severity == "warning"]
exit_code = 1 if blockers or (args.strict_warnings and warnings) else 0
if args.json:
print(json.dumps(payload, indent=2, sort_keys=True))
return exit_code
print("Static module contract scan")
print(f"Workspace: {workspace_root}")
print(
f"Contracts: {payload['contract_count']} modules, "
f"{payload['provider_count']} providers, {payload['requirement_count']} requirements"
)
if not args.no_impact:
print()
print("Provider impact:")
if providers:
for interface_name, provider_items in providers.items():
provider_text = ", ".join(
f"{contract.module_id}@{provider.version} ({contract.repo})"
for contract, provider in provider_items
)
consumer_text = ", ".join(
f"{contract.module_id} ({format_version_range(version_min=req.version_min, version_max_exclusive=req.version_max_exclusive)})"
for contract, req in consumers.get(interface_name, ())
)
if not consumer_text:
consumer_text = "no consumers"
print(f"- {interface_name}: {provider_text}; consumers: {consumer_text}")
else:
print("- no provided interfaces found")
if issues:
print()
print("Contract issues:")
for issue in issues:
location = issue.repo or "workspace"
if issue.module_id:
location += f"/{issue.module_id}"
print(f"- [{issue.severity}] {issue.code}: {location}: {issue.message}")
else:
print()
print("Contract check passed.")
return exit_code
def repository_snapshots(*, workspace_root: Path, include_website: bool) -> tuple[RepositorySnapshot, ...]:
snapshots: list[RepositorySnapshot] = []
for spec in load_repository_specs(include_website=include_website):
path = resolve_repo_path(spec, workspace_root)
snapshots.append(
RepositorySnapshot(
spec=spec,
absolute_path=str(path),
exists=path.exists(),
is_git=(path / ".git").exists(),
)
)
return tuple(snapshots)
def provider_payload(providers, consumers):
payload: dict[str, dict[str, list[dict[str, str | bool | None]]]] = {}
for interface_name, provider_items in providers.items():
payload[interface_name] = {
"providers": [
{
"repo": contract.repo,
"module_id": contract.module_id,
"version": provider.version,
"manifest_path": contract.manifest_path,
}
for contract, provider in provider_items
],
"consumers": [
{
"repo": contract.repo,
"module_id": contract.module_id,
"version_min": requirement.version_min,
"version_max_exclusive": requirement.version_max_exclusive,
"optional": requirement.optional,
"manifest_path": contract.manifest_path,
}
for contract, requirement in consumers.get(interface_name, ())
],
}
return payload
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,12 @@
#!/usr/bin/env bash
set -euo pipefail
META_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
VENV_ROOT="${GOVOPLAN_VENV_ROOT:-$META_ROOT/.venv}"
PYTHON="${PYTHON:-$VENV_ROOT/bin/python}"
if [[ ! -x "$PYTHON" ]]; then
PYTHON=python3
fi
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" "$@"

View File

@@ -33,59 +33,14 @@ PY
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
from __future__ import annotations from __future__ import annotations
import importlib.metadata
import pathlib
import sys
prefix = pathlib.Path(sys.prefix)
legacy_patterns = (
"__editable__.govoplan_module_multimailer-*.pth",
"__editable___govoplan_module_multimailer_*_finder.py",
"govoplan_module_multimailer-*.dist-info",
)
problems: list[pathlib.Path] = []
for site_packages in (prefix / "lib").glob("python*/site-packages"):
for pattern in legacy_patterns:
problems.extend(site_packages.glob(pattern))
for dist in importlib.metadata.distributions():
if dist.metadata.get("Name", "").lower() == "govoplan-module-multimailer":
problems.append(pathlib.Path(str(dist.locate_file(""))))
if problems:
print("Dependency hygiene failed: stale legacy multimailer install metadata found.", file=sys.stderr)
for path in sorted(set(problems)):
print(f" {path}", file=sys.stderr)
print("Remove the stale files or recreate the venv. This package pins obsolete dependencies.", file=sys.stderr)
raise SystemExit(1)
print("No stale legacy multimailer package metadata found.")
PY
"$PYTHON" - <<'PY'
from __future__ import annotations
import pathlib import pathlib
import sys import sys
root = pathlib.Path.cwd() root = pathlib.Path.cwd()
scan_roots = [ scan_roots = [root / "tests"] + [
root / "src", repo / "src"
root / "tests", for repo in sorted(root.parent.glob("govoplan*"))
root.parent / "govoplan-access" / "src", if (repo / "src").is_dir()
root.parent / "govoplan-admin" / "src",
root.parent / "govoplan-audit" / "src",
root.parent / "govoplan-calendar" / "src",
root.parent / "govoplan-campaign" / "src",
root.parent / "govoplan-dashboard" / "src",
root.parent / "govoplan-files" / "src",
root.parent / "govoplan-identity" / "src",
root.parent / "govoplan-mail" / "src",
root.parent / "govoplan-ops" / "src",
root.parent / "govoplan-organizations" / "src",
root.parent / "govoplan-policy" / "src",
root.parent / "govoplan-tenancy" / "src",
] ]
deprecated_tokens = { deprecated_tokens = {
"HTTP_422_UNPROCESSABLE_ENTITY": "Use HTTP_422_UNPROCESSABLE_CONTENT; the numeric status remains 422.", "HTTP_422_UNPROCESSABLE_ENTITY": "Use HTTP_422_UNPROCESSABLE_CONTENT; the numeric status remains 422.",

View File

@@ -27,26 +27,25 @@ unset npm_config_tmp NPM_CONFIG_TMP
cd "$ROOT" cd "$ROOT"
GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh" GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh"
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py"
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
import ast import ast
import pathlib import pathlib
import sys import sys
repos_root = pathlib.Path("/mnt/DATA/git")
roots = [ roots = [
pathlib.Path("/mnt/DATA/git/govoplan-core/src"), repo / "src"
pathlib.Path("/mnt/DATA/git/govoplan-access/src"), for repo in sorted(repos_root.glob("govoplan*"))
pathlib.Path("/mnt/DATA/git/govoplan-admin/src"), if (repo / "src").is_dir()
pathlib.Path("/mnt/DATA/git/govoplan-tenancy/src"),
pathlib.Path("/mnt/DATA/git/govoplan-organizations/src"),
pathlib.Path("/mnt/DATA/git/govoplan-identity/src"),
pathlib.Path("/mnt/DATA/git/govoplan-policy/src"),
pathlib.Path("/mnt/DATA/git/govoplan-audit/src"),
pathlib.Path("/mnt/DATA/git/govoplan-mail/src"),
pathlib.Path("/mnt/DATA/git/govoplan-files/src"),
pathlib.Path("/mnt/DATA/git/govoplan-campaign/src"),
pathlib.Path("/mnt/DATA/git/govoplan-mail/tests"),
] ]
roots.extend(
repo / "tests"
for repo in sorted(repos_root.glob("govoplan*"))
if (repo / "tests").is_dir()
)
errors = [] errors = []
count = 0 count = 0
@@ -67,7 +66,7 @@ if errors:
print(f"AST syntax check passed for {count} Python files") print(f"AST syntax check passed for {count} Python files")
PY PY
"$PYTHON" -c 'import govoplan_core.db.bootstrap; import govoplan_access.backend.admin.service; import govoplan_files.backend.router; import govoplan_mail.backend.sending.imap; print("targeted backend imports passed")' "$PYTHON" -c 'import govoplan_core.db.bootstrap; import govoplan_access.backend.admin.service; import govoplan_addresses.backend.manifest; import govoplan_files.backend.router; import govoplan_mail.backend.sending.imap; print("targeted backend imports passed")'
"$META_ROOT/tools/checks/check_dependency_boundaries.py" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
"$PYTHON" -m unittest tests.test_module_system "$PYTHON" -m unittest tests.test_module_system
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests

View File

@@ -0,0 +1,124 @@
#!/usr/bin/env python3
"""Load and validate every available GovOPlaN module manifest from source."""
from __future__ import annotations
import argparse
import importlib
import json
import re
import sys
import tomllib
from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[2]
MODULE_NAME_PATTERN = re.compile(
r"[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*"
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--workspace-root",
type=Path,
default=None,
help="Directory containing the GovOPlaN repositories. Defaults to repositories.json default_parent.",
)
args = parser.parse_args()
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
workspace_root = (args.workspace_root or Path(catalog["default_parent"])).resolve()
repositories = tuple(catalog["repositories"])
source_roots: list[Path] = []
manifest_sources: list[tuple[str, Path, Path]] = []
for repository in repositories:
repository_root = workspace_root / repository["path"]
source_root = repository_root / "src"
if not source_root.is_dir():
continue
source_roots.append(source_root)
manifest_sources.extend(
(repository["name"], source_root, manifest_path)
for manifest_path in sorted(source_root.glob("*/backend/manifest.py"))
)
if not manifest_sources:
print(f"No module manifests found below {workspace_root}.", file=sys.stderr)
return 1
core_source = workspace_root / "govoplan-core" / "src"
ordered_source_roots = [core_source, *(root for root in source_roots if root != core_source)]
sys.path[:0] = [str(root) for root in ordered_source_roots]
from govoplan_core.core.modules import ModuleManifest # noqa: PLC0415
from govoplan_core.core.registry import PlatformRegistry, RegistryError # noqa: PLC0415
manifests: list[ModuleManifest] = []
errors: list[str] = []
for repository_name, source_root, manifest_path in manifest_sources:
module_name = ".".join(manifest_path.relative_to(source_root).with_suffix("").parts)
if MODULE_NAME_PATTERN.fullmatch(module_name) is None:
errors.append(
f"{repository_name}: manifest path does not map to a safe Python module name: {module_name!r}"
)
continue
try:
# Module names are derived from repository-owned manifest paths and
# constrained to canonical Python identifiers immediately above.
loaded_module = importlib.import_module(module_name) # nosemgrep: python.lang.security.audit.non-literal-import.non-literal-import
get_manifest = getattr(loaded_module, "get_manifest")
manifest = get_manifest()
except Exception as exc: # pragma: no cover - rendered as a check failure
errors.append(f"{repository_name}: could not load {module_name}: {exc}")
continue
if not isinstance(manifest, ModuleManifest):
errors.append(f"{repository_name}: {module_name}.get_manifest() did not return ModuleManifest")
continue
entry_points = _module_entry_points(manifest_path.parents[3] / "pyproject.toml")
expected_target = f"{module_name}:get_manifest"
actual_target = entry_points.get(manifest.id)
if actual_target != expected_target:
declared = ", ".join(f"{name}={target}" for name, target in sorted(entry_points.items())) or "none"
errors.append(
f"{repository_name}: module {manifest.id!r} must declare entry point "
f"{manifest.id}={expected_target}; found {declared}"
)
continue
manifests.append(manifest)
if errors:
print("\n".join(errors), file=sys.stderr)
return 1
registry = PlatformRegistry()
try:
for manifest in manifests:
registry.register(manifest)
snapshot = registry.validate()
except RegistryError as exc:
print(f"Manifest registry validation failed: {exc}", file=sys.stderr)
return 1
print(
f"Manifest registry check passed: {len(snapshot.manifests)} manifests "
f"from {len(manifest_sources)} source files."
)
return 0
def _module_entry_points(pyproject_path: Path) -> dict[str, str]:
if not pyproject_path.is_file():
return {}
pyproject = tomllib.loads(pyproject_path.read_text(encoding="utf-8"))
values = pyproject.get("project", {}).get("entry-points", {}).get("govoplan.modules", {})
return {str(name): str(target) for name, target in values.items()}
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -18,7 +18,9 @@ if [[ ! -x "$NPM" ]]; then
fi fi
cd "$ROOT" cd "$ROOT"
"$PYTHON" -m unittest tests.test_module_system tests.test_access_contracts "$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" core-tests \
--core-root "$ROOT"
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py" "$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
cd "$ROOT/webui" cd "$ROOT/webui"

View File

@@ -32,9 +32,8 @@ with (Path.cwd() / "pyproject.toml").open("rb") as handle:
print(tomllib.load(handle)["project"]["version"]) print(tomllib.load(handle)["project"]["version"])
PY PY
)" )"
RELEASE_TAG="${GOVOPLAN_RELEASE_TAG:-v$RELEASE_VERSION}"
export RELEASE_TAG RELEASE_VERSION export RELEASE_VERSION
export META_ROOT export META_ROOT
export GOVOPLAN_CORE_SOURCE_ROOT="$ROOT" export GOVOPLAN_CORE_SOURCE_ROOT="$ROOT"
export GOVOPLAN_MIGRATION_TRACK=release export GOVOPLAN_MIGRATION_TRACK=release
@@ -91,7 +90,33 @@ install_cloned_webui_dependencies() {
) )
} }
release_tag_for_package() {
local package_name="$1"
"$PYTHON" - "$META_ROOT/requirements-release.txt" "$package_name" <<'PY'
from __future__ import annotations
from pathlib import Path
import re
import sys
requirements_path = Path(sys.argv[1])
package_name = sys.argv[2]
pattern = re.compile(
rf"^{re.escape(package_name)}\s+@\s+git\+ssh://.+?\.git@(v[0-9]+\.[0-9]+\.[0-9]+)$"
)
for line in requirements_path.read_text(encoding="utf-8").splitlines():
match = pattern.match(line.strip())
if match:
print(match.group(1))
raise SystemExit(0)
raise SystemExit(f"No immutable release reference found for {package_name}")
PY
}
run_step "Validate release refs and installed package metadata" run_step "Validate release refs and installed package metadata"
"$PYTHON" "$META_ROOT/tools/checks/check-version-alignment.py" --release-composition
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
from __future__ import annotations from __future__ import annotations
@@ -107,7 +132,6 @@ root = Path.cwd()
release_requirements = Path(os.environ["META_ROOT"]) / "requirements-release.txt" release_requirements = Path(os.environ["META_ROOT"]) / "requirements-release.txt"
release_webui = root / "webui" / "package.release.json" release_webui = root / "webui" / "package.release.json"
expected_version = os.environ["RELEASE_VERSION"] expected_version = os.environ["RELEASE_VERSION"]
expected_tag = os.environ["RELEASE_TAG"]
errors: list[str] = [] errors: list[str] = []
python_requirements: list[tuple[str, str, str]] = [] python_requirements: list[tuple[str, str, str]] = []
@@ -120,8 +144,6 @@ if not python_requirements:
errors.append("No GovOPlaN git+ssh release requirements found.") errors.append("No GovOPlaN git+ssh release requirements found.")
for package_name, repo_url, tag in python_requirements: for package_name, repo_url, tag in python_requirements:
if tag != expected_tag:
errors.append(f"{package_name} release requirement uses {tag!r}, expected {expected_tag!r}.")
version = metadata.version(package_name) version = metadata.version(package_name)
if version != tag.removeprefix("v"): if version != tag.removeprefix("v"):
errors.append(f"{package_name} installed version {version!r} does not match {tag!r}.") errors.append(f"{package_name} installed version {version!r} does not match {tag!r}.")
@@ -145,15 +167,18 @@ for package_name, spec in sorted(webui_dependencies.items()):
if not package_name.startswith("@govoplan/"): if not package_name.startswith("@govoplan/"):
continue continue
tag = spec.rsplit("#", 1)[-1] if "#" in spec else "" tag = spec.rsplit("#", 1)[-1] if "#" in spec else ""
if tag != expected_tag: tag_match = re.fullmatch(r"v([0-9]+\.[0-9]+\.[0-9]+)", tag)
errors.append(f"{package_name} release dependency uses {tag!r}, expected {expected_tag}.") if tag_match is None:
errors.append(f"{package_name} release dependency has invalid immutable tag {tag!r}.")
continue
dependency_version = tag_match.group(1)
package_json = root / "webui" / "node_modules" / package_name / "package.json" package_json = root / "webui" / "node_modules" / package_name / "package.json"
if not package_json.exists(): if not package_json.exists():
errors.append(f"{package_name} is missing from release node_modules.") errors.append(f"{package_name} is missing from release node_modules.")
continue continue
installed = json.loads(package_json.read_text(encoding="utf-8")) installed = json.loads(package_json.read_text(encoding="utf-8"))
if installed.get("version") != expected_version: if installed.get("version") != dependency_version:
errors.append(f"{package_name} installed version {installed.get('version')!r}, expected {expected_version!r}.") errors.append(f"{package_name} installed version {installed.get('version')!r}, expected {dependency_version!r} from {tag!r}.")
shared_peers = ("lucide-react", "react", "react-dom", "react-router-dom") shared_peers = ("lucide-react", "react", "react-dom", "react-router-dom")
root_peers = release_package.get("peerDependencies", {}) root_peers = release_package.get("peerDependencies", {})
@@ -172,40 +197,14 @@ print(f"Release refs and metadata passed for {len(python_requirements)} Python p
PY PY
run_step "Validate installed module manifests and registry" run_step "Validate installed module manifests and registry"
"$PYTHON" - <<'PY' "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \
from __future__ import annotations --requirements "$META_ROOT/requirements-release.txt"
import sys run_step "Generate release dependency provenance"
from govoplan_core.server.registry import available_module_manifests, build_platform_registry "$PYTHON" "$META_ROOT/tools/release/generate-release-sbom.py" \
--python "$PYTHON" \
expected = { --core-root "$ROOT" \
"access", --output "$WORK_ROOT/govoplan-sbom.cdx.json"
"admin",
"audit",
"calendar",
"campaigns",
"dashboard",
"docs",
"files",
"identity",
"idm",
"mail",
"ops",
"organizations",
"policy",
"tenancy",
}
manifests = available_module_manifests()
missing = sorted(expected - set(manifests))
if missing:
print(f"Missing installed module manifests: {', '.join(missing)}", file=sys.stderr)
raise SystemExit(1)
registry = build_platform_registry(tuple(sorted(expected)))
snapshot = registry.validate()
print("Registry modules:", ", ".join(manifest.id for manifest in snapshot.manifests))
PY
run_step "Run SQLite migration smoke for release modules" run_step "Run SQLite migration smoke for release modules"
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
@@ -244,16 +243,13 @@ PY
run_step "Clone release module test sources" run_step "Clone release module test sources"
for repo in govoplan-mail govoplan-calendar govoplan-campaign; do for repo in govoplan-mail govoplan-calendar govoplan-campaign; do
git clone --depth 1 --branch "$RELEASE_TAG" "git@git.add-ideas.de:add-ideas/${repo}.git" "$WORK_ROOT/$repo" repo_tag="$(release_tag_for_package "$repo")"
git clone --depth 1 --branch "$repo_tag" "git@git.add-ideas.de:add-ideas/${repo}.git" "$WORK_ROOT/$repo"
done done
run_step "Run core backend release tests" run_step "Run core backend release tests"
"$PYTHON" -m unittest \ "$PYTHON" "$META_ROOT/tools/checks/release_integration.py" core-tests \
tests.test_module_system \ --core-root "$ROOT"
tests.test_access_contracts \
tests.test_identity_organization_contracts \
tests.test_core_events \
tests.test_policy_contracts
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py" "$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
run_step "Run cloned module backend tests" run_step "Run cloned module backend tests"

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -uo pipefail set -uo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
MODE="${SECURITY_AUDIT_MODE:-ci}" MODE="${SECURITY_AUDIT_MODE:-ci}"
SCOPE="${SECURITY_AUDIT_SCOPE:-current}" SCOPE="${SECURITY_AUDIT_SCOPE:-current}"
REPORTS_DIR="${SECURITY_AUDIT_REPORTS_DIR:-$ROOT/audit-reports}" REPORTS_DIR="${SECURITY_AUDIT_REPORTS_DIR:-$ROOT/audit-reports}"
@@ -77,6 +77,15 @@ if [[ "$REPORTS_DIR" != /* ]]; then
REPORTS_DIR="$ROOT/$REPORTS_DIR" REPORTS_DIR="$ROOT/$REPORTS_DIR"
fi fi
mkdir -p "$REPORTS_DIR" mkdir -p "$REPORTS_DIR"
REPORTS_DIR="$(cd "$REPORTS_DIR" && pwd -P)"
for flag_name in FAIL_ON_FINDINGS REQUIRE_TOOLS; do
flag_value="${!flag_name}"
if [[ "$flag_value" != "0" && "$flag_value" != "1" ]]; then
echo "$flag_name must be 0 or 1, got: $flag_value" >&2
exit 2
fi
done
declare -a REPOS=() declare -a REPOS=()
if [[ "$SCOPE" == "govoplan" ]]; then if [[ "$SCOPE" == "govoplan" ]]; then
@@ -95,14 +104,30 @@ if [[ "${#REPOS[@]}" -eq 0 ]]; then
exit 1 exit 1
fi fi
echo "Security audit mode: $MODE"
echo "Security audit scope: $SCOPE"
echo "Security audit reports: $REPORTS_DIR"
echo "Security audit repositories: ${#REPOS[@]}"
if [[ "${#REPOS[@]}" -le 12 ]]; then
printf ' %s\n' "${REPOS[@]}"
fi
declare -a PY_ROOTS=() declare -a PY_ROOTS=()
declare -a PY_PROD_ROOTS=()
declare -a PY_TEST_ROOTS=()
for repo in "${REPOS[@]}"; do for repo in "${REPOS[@]}"; do
[[ -d "$repo/src" ]] && PY_ROOTS+=("$repo/src") if [[ -d "$repo/src" ]]; then
[[ -d "$repo/tests" ]] && PY_ROOTS+=("$repo/tests") PY_ROOTS+=("$repo/src")
PY_PROD_ROOTS+=("$repo/src")
fi
if [[ -d "$repo/tests" ]]; then
PY_ROOTS+=("$repo/tests")
PY_TEST_ROOTS+=("$repo/tests")
fi
done done
safe_name() { safe_name() {
basename "$1" | tr -c 'A-Za-z0-9_.-' '_' printf '%s' "$(basename "$1")" | tr -c 'A-Za-z0-9_.-' '_'
} }
has_tool() { has_tool() {
@@ -112,77 +137,386 @@ has_tool() {
overall_status=0 overall_status=0
finding_status=0 finding_status=0
missing_status=0 missing_status=0
execution_status=0
audit_started_at="$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
audit_completed_at=""
workspace_unchanged="unknown"
audit_toolbox_fingerprint="${SECURITY_AUDIT_TOOLBOX_FINGERPRINT:-direct-host}"
declare -A REPORT_FILES=()
declare -A MACHINE_REPORTS=()
register_report() {
local path="$1"
local report_kind="${2:-text}"
case "$path" in
"$REPORTS_DIR"/*) ;;
*)
echo "Refusing to register an audit report outside $REPORTS_DIR: $path" >&2
return 2
;;
esac
REPORT_FILES["$path"]=1
if [[ "$report_kind" == "machine" ]]; then
MACHINE_REPORTS["$path"]=1
fi
}
prepare_machine_report() {
local path="$1"
register_report "$path" machine || return 2
if [[ -e "$path" || -L "$path" ]]; then
rm -f -- "$path" || return 2
fi
}
run_step() { run_step() {
local name="$1" local name="$1"
shift local exit_contract="$2"
shift 2
echo echo
echo "==> $name" echo "==> $name"
"$@" "$@"
local status=$? local status=$?
if [[ "$status" -ne 0 ]]; then local outcome="passed"
echo "Audit step reported findings or failed: $name (exit $status)" >&2 if [[ "$status" -eq 0 ]]; then
:
elif [[ "$exit_contract" == "findings-exit-one" && "$status" -eq 1 ]]; then
outcome="findings"
echo "Audit step reported findings: $name (exit $status)" >&2
finding_status=1 finding_status=1
if [[ "$FAIL_ON_FINDINGS" == "1" ]]; then if [[ "$FAIL_ON_FINDINGS" == "1" ]]; then
overall_status=1 overall_status=1
fi fi
else
outcome="execution-error"
echo "Audit step failed to execute successfully: $name (exit $status)" >&2
execution_status=1
overall_status=1
fi fi
printf '%s\t%s\t%s\n' "$name" "$status" "$outcome" >> "$REPORTS_DIR/step-status.tsv"
} }
skip_or_fail_missing() { skip_or_fail_missing() {
local tool="$1" local tool="$1"
echo "Skipping $tool: command not found. Use tools/checks/security-audit/run.sh for the containerized toolbox." >&2 echo "Skipping $tool: command not found. Use tools/checks/security-audit/run.sh for the containerized toolbox." >&2
missing_status=1
if [[ "$REQUIRE_TOOLS" == "1" ]]; then if [[ "$REQUIRE_TOOLS" == "1" ]]; then
missing_status=1
overall_status=1 overall_status=1
fi fi
printf '%s\t127\tmissing\n' "$tool" >> "$REPORTS_DIR/step-status.tsv"
} }
write_manifest() { write_manifest() {
{ local -a present_reports=()
printf '{\n' local -a expected_reports=()
printf ' "mode": "%s",\n' "$MODE" local path
printf ' "scope": "%s",\n' "$SCOPE" while IFS= read -r path; do
printf ' "fail_on_findings": "%s",\n' "$FAIL_ON_FINDINGS" expected_reports+=("${path#"$REPORTS_DIR"/}")
printf ' "repositories": [\n' if [[ -f "$path" ]]; then
local index=0 present_reports+=("${path#"$REPORTS_DIR"/}")
for repo in "${REPOS[@]}"; do fi
[[ "$index" -gt 0 ]] && printf ',\n' done < <(printf '%s\n' "${!REPORT_FILES[@]}" | sort)
printf ' "%s"' "$repo"
index=$((index + 1)) python3 - \
done "$REPORTS_DIR/manifest.json" \
printf '\n ]\n' "$MODE" \
printf '}\n' "$SCOPE" \
} > "$REPORTS_DIR/manifest.json" "$FAIL_ON_FINDINGS" \
"$audit_started_at" \
"$audit_completed_at" \
"$workspace_unchanged" \
"$overall_status" \
"$finding_status" \
"$execution_status" \
"$missing_status" \
"$audit_toolbox_fingerprint" \
"${#REPOS[@]}" \
"${REPOS[@]}" \
"${#present_reports[@]}" \
"${present_reports[@]}" \
"${expected_reports[@]}" <<'PY'
from __future__ import annotations
import json
from pathlib import Path
import sys
(
manifest_path,
mode,
scope,
fail_on_findings,
started_at,
completed_at,
workspace_unchanged,
overall_status,
finding_status,
execution_status,
missing_status,
toolbox_fingerprint,
repository_count,
*remaining,
) = sys.argv[1:]
repo_count = int(repository_count)
repositories = remaining[:repo_count]
remaining = remaining[repo_count:]
report_count = int(remaining[0])
reports = remaining[1 : report_count + 1]
expected_reports = remaining[report_count + 1 :]
payload = {
"mode": mode,
"scope": scope,
"fail_on_findings": fail_on_findings == "1",
"started_at_utc": started_at,
"completed_at_utc": completed_at,
"toolbox_fingerprint": toolbox_fingerprint,
"workspace_unchanged": workspace_unchanged == "true",
"overall_status": int(overall_status),
"finding_status": int(finding_status),
"execution_error_status": int(execution_status),
"missing_tool_status": int(missing_status),
"tool_versions": "tool-versions.txt",
"workspace_state_start": "workspace-state-start.tsv",
"workspace_state_end": "workspace-state-end.tsv",
"report_checksums": "report-sha256.txt",
"step_status": "step-status.tsv",
"repositories": repositories,
"reports": reports,
"expected_reports": expected_reports,
"missing_reports": sorted(set(expected_reports) - set(reports)),
}
Path(manifest_path).write_text(
json.dumps(payload, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
PY
}
accumulate_exit_status() {
local -n aggregate="$1"
local child_status="$2"
if [[ "$child_status" -gt 1 ]]; then
aggregate=2
elif [[ "$child_status" -eq 1 && "$aggregate" -eq 0 ]]; then
aggregate=1
fi
}
repository_fingerprint() {
local repo="$1"
(
set -e
cd "$repo" || exit 1
git rev-parse HEAD
git status --porcelain=v1 -z
git diff --binary --no-ext-diff HEAD --
while IFS= read -r -d '' path; do
printf '%s\0' "$path"
if [[ -L "$path" ]]; then
printf 'symlink\0'
readlink -- "$path"
else
sha256sum -- "$path"
fi
done < <(git ls-files --others --exclude-standard -z)
) | sha256sum | cut -d' ' -f1
}
write_workspace_state() {
local destination="$1"
local head
local fingerprint
: > "$destination"
for repo in "${REPOS[@]}"; do
if ! head="$(git -C "$repo" rev-parse HEAD)"; then
return 2
fi
if ! fingerprint="$(repository_fingerprint "$repo")"; then
return 2
fi
printf '%s\t%s\t%s\n' \
"$repo" \
"$head" \
"$fingerprint" \
>> "$destination"
done
}
write_tool_versions() {
local destination="$REPORTS_DIR/tool-versions.txt"
local status=0
local version
: > "$destination"
local tool
for tool in semgrep bandit ruff gitleaks trivy pip-audit npm osv-scanner jscpd radon xenon; do
has_tool "$tool" || continue
if [[ "$tool" == "gitleaks" ]]; then
version="$("$tool" version 2>&1)"
else
version="$("$tool" --version 2>&1)"
fi
local version_status=$?
version="${version%%$'\n'*}"
if [[ "$version_status" -ne 0 || -z "$version" ]]; then
printf '%s\t%s\n' "$tool" "version-command-failed:$version_status" >> "$destination"
status=2
else
printf '%s\t%s\n' "$tool" "$version" >> "$destination"
fi
done
return "$status"
}
write_report_checksums() {
local path
local relative_path
: > "$REPORTS_DIR/report-sha256.txt"
while IFS= read -r path; do
[[ -f "$path" ]] || continue
relative_path="${path#"$REPORTS_DIR"/}"
(cd "$REPORTS_DIR" && sha256sum -- "$relative_path") \
>> "$REPORTS_DIR/report-sha256.txt" || return 2
done < <(printf '%s\n' "${!REPORT_FILES[@]}" | sort)
}
validate_machine_reports() {
python3 - "$REPORTS_DIR" "${!MACHINE_REPORTS[@]}" <<'PY'
from __future__ import annotations
import json
from pathlib import Path
import sys
root = Path(sys.argv[1]).resolve()
errors: list[str] = []
validated = 0
for raw_path in sorted(set(sys.argv[2:])):
path = Path(raw_path)
try:
relative_path = path.resolve().relative_to(root)
except (OSError, ValueError) as exc:
errors.append(f"{path}: invalid report path: {exc}")
continue
if not path.is_file():
errors.append(f"{relative_path}: expected report was not produced")
continue
if path.stat().st_size == 0:
errors.append(f"{relative_path}: report is empty")
continue
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
errors.append(f"{relative_path}: invalid JSON: {exc}")
continue
validated += 1
if path.suffix == ".sarif" and (
not isinstance(payload, dict)
or not isinstance(payload.get("version"), str)
or not isinstance(payload.get("runs"), list)
):
errors.append(f"{relative_path}: invalid SARIF document")
continue
report_name = path.name
if report_name.startswith("bandit") and (
not isinstance(payload, dict)
or not isinstance(payload.get("errors"), list)
or not isinstance(payload.get("metrics"), dict)
or not isinstance(payload.get("results"), list)
):
errors.append(f"{relative_path}: invalid Bandit report")
elif report_name.startswith("ruff-security") and not isinstance(payload, list):
errors.append(f"{relative_path}: invalid Ruff report")
elif report_name.startswith("gitleaks-") and not isinstance(payload, list):
errors.append(f"{relative_path}: invalid Gitleaks report")
elif report_name.startswith("pip-audit-") and (
not isinstance(payload, dict)
or not isinstance(payload.get("dependencies"), list)
):
errors.append(f"{relative_path}: invalid pip-audit report")
elif report_name.startswith("npm-audit-") and (
not isinstance(payload, dict)
or payload.get("error")
or not isinstance(payload.get("metadata"), dict)
or not isinstance(payload.get("vulnerabilities"), dict)
):
errors.append(f"{relative_path}: npm audit returned an invalid or error report")
elif report_name.startswith("osv-scanner-") and (
not isinstance(payload, dict)
or "results" not in payload
or (payload["results"] is not None and not isinstance(payload["results"], list))
):
errors.append(f"{relative_path}: invalid OSV-Scanner report")
elif report_name == "jscpd-report.json" and (
not isinstance(payload, dict)
or not isinstance(payload.get("duplicates"), list)
or not isinstance(payload.get("statistics"), dict)
):
errors.append(f"{relative_path}: invalid jscpd report")
if errors:
print("\n".join(errors), file=sys.stderr)
raise SystemExit(2)
print(f"Validated {validated} JSON/SARIF reports.")
PY
} }
run_semgrep() { run_semgrep() {
local report="$REPORTS_DIR/semgrep.sarif" local report="$REPORTS_DIR/semgrep.sarif"
prepare_machine_report "$report" || return 2
local config_args=(--config "$ROOT/tools/checks/security-audit/semgrep-govoplan.yml") local config_args=(--config "$ROOT/tools/checks/security-audit/semgrep-govoplan.yml")
if [[ "$MODE" != "quick" ]]; then if [[ "$MODE" != "quick" ]]; then
config_args+=(--config p/default --config p/owasp-top-ten) config_args+=(--config p/default --config p/owasp-top-ten)
fi fi
semgrep scan \ semgrep scan \
--metrics=off \ --metrics=off \
--error \
--no-git-ignore \
--sarif \ --sarif \
--output "$report" \ --output "$report" \
--exclude .git \
--exclude node_modules \ --exclude node_modules \
--exclude .venv \ --exclude .venv \
--exclude dist \ --exclude dist \
--exclude build \ --exclude build \
--exclude runtime \ --exclude runtime \
--exclude audit-reports \
--exclude '**/.*-test-build/**' \
"${config_args[@]}" \ "${config_args[@]}" \
"${REPOS[@]}" "${REPOS[@]}"
} }
run_bandit() { run_bandit() {
[[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0 local status=0
bandit -r "${PY_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit.json" if [[ "${#PY_PROD_ROOTS[@]}" -gt 0 ]]; then
prepare_machine_report "$REPORTS_DIR/bandit.json" || return 2
bandit -r "${PY_PROD_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit.json"
accumulate_exit_status status "$?"
fi
if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then
prepare_machine_report "$REPORTS_DIR/bandit-tests.json" || return 2
bandit -r "${PY_TEST_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit-tests.json"
local test_status=$?
[[ "$test_status" -le 1 ]] || status=2
fi
return "$status"
} }
run_ruff_security() { run_ruff_security() {
[[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0 local status=0
ruff check --select S --output-format json "${PY_ROOTS[@]}" > "$REPORTS_DIR/ruff-security.json" if [[ "${#PY_PROD_ROOTS[@]}" -gt 0 ]]; then
prepare_machine_report "$REPORTS_DIR/ruff-security.json" || return 2
ruff check --select S --output-format json "${PY_PROD_ROOTS[@]}" > "$REPORTS_DIR/ruff-security.json"
accumulate_exit_status status "$?"
fi
if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then
prepare_machine_report "$REPORTS_DIR/ruff-security-tests.json" || return 2
ruff check --select S --output-format json "${PY_TEST_ROOTS[@]}" > "$REPORTS_DIR/ruff-security-tests.json"
local test_status=$?
[[ "$test_status" -le 1 ]] || status=2
fi
return "$status"
} }
run_gitleaks() { run_gitleaks() {
@@ -191,19 +525,34 @@ run_gitleaks() {
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
if gitleaks git --help >/dev/null 2>&1; then if gitleaks git --help >/dev/null 2>&1; then
prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2
prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2
gitleaks git \ gitleaks git \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
--report-path "$REPORTS_DIR/gitleaks-$name.json" \ --report-path "$REPORTS_DIR/gitleaks-history-$name.json" \
--no-banner \ --no-banner \
"$repo" || status=1 "$repo"
accumulate_exit_status status "$?"
# History scanning does not include new or modified working-tree files.
# Scan the directory as well so pre-commit audits cover the exact code
# under review, while retaining the history scan above.
gitleaks dir \
--config "$ROOT/.gitleaks.toml" \
--report-format json \
--report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \
--no-banner \
"$repo"
accumulate_exit_status status "$?"
else else
prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2
gitleaks detect \ gitleaks detect \
--source "$repo" \ --source "$repo" \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
--report-path "$REPORTS_DIR/gitleaks-$name.json" \ --report-path "$REPORTS_DIR/gitleaks-$name.json" \
--no-banner || status=1 --no-banner
accumulate_exit_status status "$?"
fi fi
done done
return "$status" return "$status"
@@ -214,6 +563,7 @@ run_trivy() {
for repo in "${REPOS[@]}"; do for repo in "${REPOS[@]}"; do
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
prepare_machine_report "$REPORTS_DIR/trivy-$name.sarif" || return 2
trivy fs \ trivy fs \
--scanners vuln,secret,misconfig \ --scanners vuln,secret,misconfig \
--skip-dirs node_modules \ --skip-dirs node_modules \
@@ -221,8 +571,10 @@ run_trivy() {
--skip-dirs dist \ --skip-dirs dist \
--skip-dirs build \ --skip-dirs build \
--format sarif \ --format sarif \
--exit-code 1 \
--output "$REPORTS_DIR/trivy-$name.sarif" \ --output "$REPORTS_DIR/trivy-$name.sarif" \
"$repo" || status=1 "$repo"
accumulate_exit_status status "$?"
done done
return "$status" return "$status"
} }
@@ -233,7 +585,11 @@ run_pip_audit_manifests() {
[[ -f "$repo/requirements.txt" ]] || continue [[ -f "$repo/requirements.txt" ]] || continue
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
pip-audit -r "$repo/requirements.txt" --progress-spinner off --format json --output "$REPORTS_DIR/pip-audit-$name.json" || status=1 prepare_machine_report "$REPORTS_DIR/pip-audit-$name.json" || return 2
# Requirements may contain project-relative entries such as `.[server]`.
# Resolve them from the owning repository instead of the meta-repository.
(cd "$repo" && pip-audit -r requirements.txt --progress-spinner off --format json --output "$REPORTS_DIR/pip-audit-$name.json")
accumulate_exit_status status "$?"
done done
return "$status" return "$status"
} }
@@ -244,7 +600,9 @@ run_npm_audit_manifests() {
[[ -f "$repo/webui/package-lock.json" ]] || continue [[ -f "$repo/webui/package-lock.json" ]] || continue
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
(cd "$repo/webui" && npm audit --omit=dev --json > "$REPORTS_DIR/npm-audit-$name.json") || status=1 prepare_machine_report "$REPORTS_DIR/npm-audit-$name.json" || return 2
(cd "$repo/webui" && npm audit --omit=dev --json > "$REPORTS_DIR/npm-audit-$name.json")
accumulate_exit_status status "$?"
done done
return "$status" return "$status"
} }
@@ -255,75 +613,122 @@ run_osv_scanner() {
local name local name
name="$(safe_name "$repo")" name="$(safe_name "$repo")"
if osv-scanner scan --help >/dev/null 2>&1; then if osv-scanner scan --help >/dev/null 2>&1; then
osv-scanner scan -r --format json --output "$REPORTS_DIR/osv-scanner-$name.json" "$repo" || status=1 prepare_machine_report "$REPORTS_DIR/osv-scanner-$name.json" || return 2
osv-scanner scan -r \
--allow-no-lockfiles \
--format json \
--output-file "$REPORTS_DIR/osv-scanner-$name.json" \
"$repo"
accumulate_exit_status status "$?"
else else
osv-scanner -r --format json --output "$REPORTS_DIR/osv-scanner-$name.json" "$repo" || status=1 prepare_machine_report "$REPORTS_DIR/osv-scanner-$name.json" || return 2
osv-scanner -r --format json --output "$REPORTS_DIR/osv-scanner-$name.json" "$repo"
accumulate_exit_status status "$?"
fi fi
done done
return "$status" return "$status"
} }
run_jscpd() { run_jscpd() {
prepare_machine_report "$REPORTS_DIR/jscpd/jscpd-report.json" || return 2
local ignored_paths
ignored_paths=(
"**/.git/**"
"**/node_modules/**"
"**/.venv/**"
"**/dist/**"
"**/build/**"
"**/audit-reports/**"
"**/package-lock.json"
"**/package-lock.release.json"
"**/package.json"
"package.json"
"**/generatedTranslations.ts"
"**/docs/gitea-labels.json"
"**/*.md"
"**/*.md:*"
"*.md"
"*.md:*"
"**/*.svg"
"*.svg"
".gitea/workflows/*.yml"
"**/.gitea/workflows/*.yml"
"**/backend/schema/*.schema.json"
)
local ignored_path_pattern
ignored_path_pattern="$(printf '%s,' "${ignored_paths[@]}")"
ignored_path_pattern="${ignored_path_pattern%,}"
jscpd \ jscpd \
--silent \ --silent \
--min-tokens 80 \ --min-tokens 80 \
--reporters json \ --reporters json \
--output "$REPORTS_DIR/jscpd" \ --output "$REPORTS_DIR/jscpd" \
--ignore "**/.git/**,**/node_modules/**,**/.venv/**,**/dist/**,**/build/**,**/audit-reports/**,**/package-lock.json,**/package-lock.release.json,**/generatedTranslations.ts,**/docs/gitea-labels.json" \ --ignore "$ignored_path_pattern" \
"${REPOS[@]}" "${REPOS[@]}"
} }
run_radon() { run_radon() {
[[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0 [[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0
register_report "$REPORTS_DIR/radon-cc.txt" || return 2
radon cc -s -a "${PY_ROOTS[@]}" > "$REPORTS_DIR/radon-cc.txt" radon cc -s -a "${PY_ROOTS[@]}" > "$REPORTS_DIR/radon-cc.txt"
} }
run_xenon() { run_xenon() {
[[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0 [[ "${#PY_ROOTS[@]}" -gt 0 ]] || return 0
register_report "$REPORTS_DIR/xenon.txt" || return 2
xenon --max-absolute C --max-modules B --max-average A "${PY_ROOTS[@]}" > "$REPORTS_DIR/xenon.txt" 2>&1 xenon --max-absolute C --max-modules B --max-average A "${PY_ROOTS[@]}" > "$REPORTS_DIR/xenon.txt" 2>&1
} }
write_manifest register_report "$REPORTS_DIR/step-status.tsv"
register_report "$REPORTS_DIR/tool-versions.txt"
register_report "$REPORTS_DIR/workspace-state-start.tsv"
register_report "$REPORTS_DIR/workspace-state-end.tsv"
: > "$REPORTS_DIR/step-status.tsv"
if ! write_workspace_state "$REPORTS_DIR/workspace-state-start.tsv"; then
echo "Could not capture the repository state before the audit." >&2
exit 1
fi
run_step "Record audit tool versions" execution-only write_tool_versions
if has_tool semgrep; then if has_tool semgrep; then
run_step "Semgrep SAST" run_semgrep run_step "Semgrep SAST" findings-exit-one run_semgrep
else else
skip_or_fail_missing semgrep skip_or_fail_missing semgrep
fi fi
if has_tool bandit; then if has_tool bandit; then
run_step "Bandit Python security scan" run_bandit run_step "Bandit Python security scan" findings-exit-one run_bandit
else else
skip_or_fail_missing bandit skip_or_fail_missing bandit
fi fi
if has_tool ruff; then if has_tool ruff; then
run_step "Ruff flake8-bandit security rules" run_ruff_security run_step "Ruff flake8-bandit security rules" findings-exit-one run_ruff_security
else else
skip_or_fail_missing ruff skip_or_fail_missing ruff
fi fi
if has_tool gitleaks; then if has_tool gitleaks; then
run_step "Gitleaks secret scan" run_gitleaks run_step "Gitleaks secret scan" findings-exit-one run_gitleaks
else else
skip_or_fail_missing gitleaks skip_or_fail_missing gitleaks
fi fi
if [[ "$MODE" != "quick" ]]; then if [[ "$MODE" != "quick" ]]; then
if has_tool trivy; then if has_tool trivy; then
run_step "Trivy filesystem vulnerability/secret/misconfig scan" run_trivy run_step "Trivy filesystem vulnerability/secret/misconfig scan" findings-exit-one run_trivy
else else
skip_or_fail_missing trivy skip_or_fail_missing trivy
fi fi
if has_tool pip-audit; then if has_tool pip-audit; then
run_step "pip-audit requirements scan" run_pip_audit_manifests run_step "pip-audit requirements scan" findings-exit-one run_pip_audit_manifests
else else
skip_or_fail_missing pip-audit skip_or_fail_missing pip-audit
fi fi
if has_tool npm; then if has_tool npm; then
run_step "npm audit lockfile scan" run_npm_audit_manifests run_step "npm audit lockfile scan" findings-exit-one run_npm_audit_manifests
else else
skip_or_fail_missing npm skip_or_fail_missing npm
fi fi
@@ -331,37 +736,62 @@ fi
if [[ "$MODE" == "full" ]]; then if [[ "$MODE" == "full" ]]; then
if has_tool osv-scanner; then if has_tool osv-scanner; then
run_step "OSV-Scanner recursive dependency scan" run_osv_scanner run_step "OSV-Scanner recursive dependency scan" findings-exit-one run_osv_scanner
else else
skip_or_fail_missing osv-scanner skip_or_fail_missing osv-scanner
fi fi
if has_tool jscpd; then if has_tool jscpd; then
run_step "jscpd duplicate code scan" run_jscpd run_step "jscpd duplicate code scan" execution-only run_jscpd
else else
skip_or_fail_missing jscpd skip_or_fail_missing jscpd
fi fi
if has_tool radon; then if has_tool radon; then
run_step "Radon complexity report" run_radon run_step "Radon complexity report" execution-only run_radon
else else
skip_or_fail_missing radon skip_or_fail_missing radon
fi fi
if has_tool xenon; then if has_tool xenon; then
run_step "Xenon complexity threshold scan" run_xenon run_step "Xenon complexity threshold scan" findings-exit-one run_xenon
else else
skip_or_fail_missing xenon skip_or_fail_missing xenon
fi fi
fi fi
run_step "Validate machine-readable audit reports" execution-only validate_machine_reports
echo echo
echo "Security audit reports written to: $REPORTS_DIR" echo "Security audit reports written to: $REPORTS_DIR"
if [[ "$finding_status" -ne 0 && "$FAIL_ON_FINDINGS" != "1" ]]; then if [[ "$finding_status" -ne 0 && "$FAIL_ON_FINDINGS" != "1" ]]; then
echo "Findings were reported, but this run is report-only. Re-run with --strict to fail on findings." echo "Findings were reported, but this run is report-only. Re-run with --strict to fail on findings."
fi fi
if [[ "$missing_status" -ne 0 ]]; then if [[ "$missing_status" -ne 0 ]]; then
echo "One or more required tools were missing." >&2 echo "One or more audit tools were missing." >&2
fi
if ! write_workspace_state "$REPORTS_DIR/workspace-state-end.tsv"; then
workspace_unchanged=false
execution_status=1
overall_status=1
echo "Could not capture the repository state after the audit." >&2
elif cmp -s "$REPORTS_DIR/workspace-state-start.tsv" "$REPORTS_DIR/workspace-state-end.tsv"; then
workspace_unchanged=true
else
workspace_unchanged=false
overall_status=1
echo "Repository state changed during the audit; discard this mixed snapshot and rerun." >&2
fi
audit_completed_at="$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
if ! write_report_checksums; then
execution_status=1
overall_status=1
echo "Could not generate report checksums." >&2
fi
if ! write_manifest; then
echo "Could not write the audit manifest." >&2
exit 1
fi fi
exit "$overall_status" exit "$overall_status"

View File

@@ -0,0 +1,130 @@
#!/usr/bin/env python3
"""Fail when backend, manifest, frontend, lock, or release-ref versions drift."""
from __future__ import annotations
import argparse
from dataclasses import asdict
import json
from pathlib import Path
import sys
META_ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(META_ROOT / "tools" / "release"))
from govoplan_release.version_alignment import ( # noqa: E402
release_composition_issues,
repository_version_issues,
selected_repository_version_issues,
)
from govoplan_release.workspace import ( # noqa: E402
load_repository_specs,
resolve_repo_path,
resolve_workspace_root,
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--workspace-root", type=Path, default=None)
parser.add_argument("--repo", action="append", default=[], help="Repository name to check; may be repeated.")
parser.add_argument(
"--repo-version",
action="append",
default=[],
metavar="REPO=VERSION",
help="Require a repository's aligned source metadata to equal VERSION; may be repeated.",
)
parser.add_argument(
"--release-composition",
action="store_true",
help="Also compare tagged backend and WebUI references in the release composition.",
)
parser.add_argument(
"--source-metadata-only",
action="store_true",
help="Skip lockfile checks for the pre-commit source gate; never use for publication.",
)
parser.add_argument("--json", action="store_true", help="Print machine-readable output.")
args = parser.parse_args()
workspace_root = resolve_workspace_root(args.workspace_root)
expected_versions = _parse_repo_versions(parser, args.repo_version)
selected = {*args.repo, *expected_versions}
known = {spec.name for spec in load_repository_specs()}
unknown = sorted(selected - known)
if unknown:
parser.error(f"unknown repository name(s): {', '.join(unknown)}")
checked: list[str] = []
issues = list(
selected_repository_version_issues(
repo_versions=expected_versions,
workspace=workspace_root,
include_lockfiles=not args.source_metadata_only,
)
)
for spec in load_repository_specs():
if selected and spec.name not in selected:
continue
repo_path = resolve_repo_path(spec, workspace_root)
if not repo_path.exists():
continue
checked.append(spec.name)
if spec.name in expected_versions:
continue
issues.extend(
repository_version_issues(
repo_path,
include_lockfiles=not args.source_metadata_only,
)
)
if args.release_composition:
issues.extend(
release_composition_issues(
META_ROOT,
core_root=workspace_root / "govoplan-core",
)
)
payload = {
"workspace_root": str(workspace_root),
"repositories_checked": checked,
"release_composition_checked": args.release_composition,
"issues": [asdict(issue) for issue in issues],
}
if args.json:
print(json.dumps(payload, indent=2, sort_keys=True))
elif issues:
print("Version alignment failed:", file=sys.stderr)
for issue in issues:
print(
f"- {issue.repo}: {issue.source}: {issue.actual!r}; expected {issue.expected!r} "
f"({issue.message})",
file=sys.stderr,
)
else:
suffix = " including release composition" if args.release_composition else ""
print(f"Version alignment passed for {len(checked)} repositories{suffix}.")
return 1 if issues else 0
def _parse_repo_versions(parser: argparse.ArgumentParser, values: list[str]) -> dict[str, str]:
result: dict[str, str] = {}
for value in values:
if "=" not in value:
parser.error(f"--repo-version must use REPO=VERSION: {value}")
repo, version = (item.strip() for item in value.split("=", 1))
version = version.removeprefix("v")
if not repo or not version:
parser.error(f"--repo-version must use REPO=VERSION: {value}")
if repo in result and result[repo] != version:
parser.error(f"conflicting versions requested for {repo}")
result[repo] = version
return result
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -11,48 +11,89 @@ from pathlib import Path
META_ROOT = Path(__file__).resolve().parents[2] META_ROOT = Path(__file__).resolve().parents[2]
CORE_ROOT = Path(os.environ.get("GOVOPLAN_CORE_ROOT", META_ROOT.parent / "govoplan-core")).resolve() CORE_ROOT = Path(os.environ.get("GOVOPLAN_CORE_ROOT", META_ROOT.parent / "govoplan-core")).resolve()
REPOS_ROOT = Path(os.environ.get("GOVOPLAN_REPOS_ROOT", CORE_ROOT.parent)).resolve() REPOS_ROOT = Path(os.environ.get("GOVOPLAN_REPOS_ROOT", CORE_ROOT.parent)).resolve()
REPOS = {
"core": CORE_ROOT / "src" / "govoplan_core",
"access": REPOS_ROOT / "govoplan-access" / "src" / "govoplan_access", def _discover_python_repos(
"admin": REPOS_ROOT / "govoplan-admin" / "src" / "govoplan_admin", repos_root: Path = REPOS_ROOT,
"tenancy": REPOS_ROOT / "govoplan-tenancy" / "src" / "govoplan_tenancy", ) -> tuple[dict[str, Path], dict[str, str], tuple[str, ...]]:
"organizations": REPOS_ROOT / "govoplan-organizations" / "src" / "govoplan_organizations", repos: dict[str, Path] = {}
"identity": REPOS_ROOT / "govoplan-identity" / "src" / "govoplan_identity", prefixes: dict[str, str] = {}
"policy": REPOS_ROOT / "govoplan-policy" / "src" / "govoplan_policy", errors: list[str] = []
"audit": REPOS_ROOT / "govoplan-audit" / "src" / "govoplan_audit", for repo_root in sorted(repos_root.glob("govoplan*")):
"dashboard": REPOS_ROOT / "govoplan-dashboard" / "src" / "govoplan_dashboard", source_root = repo_root / "src"
"files": REPOS_ROOT / "govoplan-files" / "src" / "govoplan_files", if not source_root.is_dir():
"mail": REPOS_ROOT / "govoplan-mail" / "src" / "govoplan_mail", continue
"campaign": REPOS_ROOT / "govoplan-campaign" / "src" / "govoplan_campaign", packages = sorted(
} path
PREFIXES = { for path in source_root.glob("govoplan_*")
"core": "govoplan_core", if path.is_dir() and (path / "__init__.py").is_file()
"access": "govoplan_access", )
"admin": "govoplan_admin", if len(packages) != 1:
"tenancy": "govoplan_tenancy", discovered = ", ".join(path.name for path in packages) or "none"
"organizations": "govoplan_organizations", errors.append(
"identity": "govoplan_identity", f"{repo_root.name}: expected exactly one importable govoplan_* package "
"policy": "govoplan_policy", f"under {source_root}, found {len(packages)} ({discovered})"
"audit": "govoplan_audit", )
"dashboard": "govoplan_dashboard", continue
"files": "govoplan_files", owner = repo_root.name
"mail": "govoplan_mail", prefix = packages[0].name
"campaign": "govoplan_campaign", duplicate_owner = next(
} (candidate_owner for candidate_owner, candidate_prefix in prefixes.items() if candidate_prefix == prefix),
FEATURE_OWNERS = ("files", "mail", "campaign") None,
PLATFORM_OWNERS = ("admin", "tenancy", "organizations", "identity", "policy", "audit", "dashboard") )
WEBUI_REPOS = { if duplicate_owner is not None:
"core": CORE_ROOT / "webui", errors.append(
"files": REPOS_ROOT / "govoplan-files" / "webui", f"{owner}: import package {prefix!r} is already owned by {duplicate_owner}; "
"mail": REPOS_ROOT / "govoplan-mail" / "webui", "Python package ownership must be unique"
"campaign": REPOS_ROOT / "govoplan-campaign" / "webui", )
} continue
WEBUI_PACKAGES = { repos[owner] = packages[0]
"core": "@govoplan/core-webui", prefixes[owner] = prefix
"files": "@govoplan/files-webui", return repos, prefixes, tuple(errors)
"mail": "@govoplan/mail-webui",
"campaign": "@govoplan/campaign-webui",
} def _discover_webui_repos(
repos_root: Path = REPOS_ROOT,
) -> tuple[dict[str, Path], dict[str, str], tuple[str, ...]]:
repos: dict[str, Path] = {}
packages: dict[str, str] = {}
errors: list[str] = []
for repo_root in sorted(repos_root.glob("govoplan*")):
webui_root = repo_root / "webui"
if not webui_root.is_dir():
continue
package_path = webui_root / "package.json"
if not package_path.is_file():
errors.append(f"{repo_root.name}: WebUI directory has no package.json: {package_path}")
continue
try:
payload = json.loads(package_path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
errors.append(f"{repo_root.name}: invalid {package_path}: {exc}")
continue
package_name = payload.get("name") if isinstance(payload, dict) else None
if not isinstance(package_name, str) or not package_name:
errors.append(f"{repo_root.name}: {package_path} must declare a non-empty package name")
continue
owner = repo_root.name
duplicate_owner = next(
(candidate_owner for candidate_owner, candidate_package in packages.items() if candidate_package == package_name),
None,
)
if duplicate_owner is not None:
errors.append(
f"{owner}: WebUI package {package_name!r} is already owned by {duplicate_owner}; "
"WebUI package ownership must be unique"
)
continue
repos[owner] = webui_root
packages[owner] = package_name
return repos, packages, tuple(errors)
REPOS, PREFIXES, PYTHON_DISCOVERY_ERRORS = _discover_python_repos()
WEBUI_REPOS, WEBUI_PACKAGES, WEBUI_DISCOVERY_ERRORS = _discover_webui_repos()
CORE_OWNER = CORE_ROOT.name
WEBUI_IMPORT_RE = re.compile( WEBUI_IMPORT_RE = re.compile(
r"(?:from\s+|import\s*\(\s*|require\s*\(\s*|export\s+[^;]*?\s+from\s+)" r"(?:from\s+|import\s*\(\s*|require\s*\(\s*|export\s+[^;]*?\s+from\s+)"
r"[\"'](?P<package>@govoplan/[a-z0-9_-]+-webui)(?:/[A-Za-z0-9_./-]+)?[\"']" r"[\"'](?P<package>@govoplan/[a-z0-9_-]+-webui)(?:/[A-Za-z0-9_./-]+)?[\"']"
@@ -123,18 +164,7 @@ def _violations() -> list[Violation]:
imported_owner = _import_owner(imported) imported_owner = _import_owner(imported)
if imported_owner is None or imported_owner == owner: if imported_owner is None or imported_owner == owner:
continue continue
if owner == "core" and imported_owner in FEATURE_OWNERS: if imported_owner != CORE_OWNER and not _allowed(owner, path, imported_owner):
if not _allowed(owner, path, imported_owner):
violations.append(Violation("python", owner, path, lineno, imported, imported_owner))
elif owner == "access" and imported_owner in FEATURE_OWNERS:
violations.append(Violation("python", owner, path, lineno, imported, imported_owner))
elif owner in PLATFORM_OWNERS and imported_owner == "access":
if not _allowed(owner, path, imported_owner):
violations.append(Violation("python", owner, path, lineno, imported, imported_owner))
elif owner in FEATURE_OWNERS and imported_owner in FEATURE_OWNERS:
if not _allowed(owner, path, imported_owner):
violations.append(Violation("python", owner, path, lineno, imported, imported_owner))
elif owner in FEATURE_OWNERS and imported_owner == "access":
violations.append(Violation("python", owner, path, lineno, imported, imported_owner)) violations.append(Violation("python", owner, path, lineno, imported, imported_owner))
violations.extend(_webui_violations()) violations.extend(_webui_violations())
return violations return violations
@@ -187,7 +217,12 @@ def _webui_package_json_violations(owner: str, root: Path) -> list[Violation]:
imported_owner = _webui_package_owner(str(package_name)) imported_owner = _webui_package_owner(str(package_name))
if imported_owner is None or imported_owner == owner: if imported_owner is None or imported_owner == owner:
continue continue
if owner in FEATURE_OWNERS and imported_owner in FEATURE_OWNERS: # The core WebUI package is also the development composition host;
# package declarations make independently built module frontends
# available to Vite without authorizing source-level imports.
if owner == CORE_OWNER:
continue
if imported_owner != CORE_OWNER:
violations.append(Violation("webui-package", owner, package_path, 1, str(package_name), imported_owner)) violations.append(Violation("webui-package", owner, package_path, 1, str(package_name), imported_owner))
return violations return violations
@@ -202,9 +237,7 @@ def _webui_source_violations(owner: str, root: Path) -> list[Violation]:
if imported_owner is None or imported_owner == owner: if imported_owner is None or imported_owner == owner:
continue continue
lineno = text.count("\n", 0, match.start()) + 1 lineno = text.count("\n", 0, match.start()) + 1
if owner == "core" and imported_owner in FEATURE_OWNERS: if imported_owner != CORE_OWNER:
violations.append(Violation("webui-source", owner, path, lineno, package_name, imported_owner))
elif owner in FEATURE_OWNERS and imported_owner in FEATURE_OWNERS:
violations.append(Violation("webui-source", owner, path, lineno, package_name, imported_owner)) violations.append(Violation("webui-source", owner, path, lineno, package_name, imported_owner))
return violations return violations
@@ -220,9 +253,21 @@ def _webui_violations() -> list[Violation]:
def main() -> int: def main() -> int:
discovery_errors = (*PYTHON_DISCOVERY_ERRORS, *WEBUI_DISCOVERY_ERRORS)
if discovery_errors:
print("Dependency boundary discovery errors:")
for error in discovery_errors:
print(f"- {error}")
print("\nThe boundary scan is incomplete; fix package discovery before accepting this gate.")
return 1
violations = _violations() violations = _violations()
if not violations: if not violations:
print(f"Dependency boundary check passed ({len(ALLOWLIST)} transitional exceptions tracked)") print(
"Dependency boundary check passed "
f"({len(REPOS)} Python repos, {len(WEBUI_REPOS)} WebUI repos, "
f"{len(ALLOWLIST)} transitional exceptions tracked)"
)
return 0 return 0
print("Dependency boundary violations:") print("Dependency boundary violations:")

View File

@@ -1,6 +1,9 @@
#!/usr/bin/env python #!/usr/bin/env python
from __future__ import annotations from __future__ import annotations
# Application imports intentionally follow the test environment and source-path setup below.
# ruff: noqa: E402
import argparse import argparse
import json import json
import os import os
@@ -31,7 +34,6 @@ from govoplan_core.core.maintenance import MaintenanceMode, save_maintenance_mod
from govoplan_core.core.module_installer import ( from govoplan_core.core.module_installer import (
cancel_module_installer_request, cancel_module_installer_request,
claim_next_module_installer_request, claim_next_module_installer_request,
module_installer_daemon_status,
module_installer_lock_status, module_installer_lock_status,
queue_module_installer_request, queue_module_installer_request,
read_module_installer_run, read_module_installer_run,
@@ -44,8 +46,6 @@ from govoplan_core.core.module_installer import (
) )
from govoplan_core.core.module_management import ( from govoplan_core.core.module_management import (
ModuleInstallPlan, ModuleInstallPlan,
ModuleInstallPlanItem,
saved_desired_enabled_modules,
saved_module_install_plan, saved_module_install_plan,
save_desired_enabled_modules, save_desired_enabled_modules,
save_module_install_plan, save_module_install_plan,

View File

@@ -47,6 +47,14 @@ def main() -> int:
) )
parser.add_argument("--skip-migrate", action="store_true", help="Skip govoplan_core.commands.init_db.") parser.add_argument("--skip-migrate", action="store_true", help="Skip govoplan_core.commands.init_db.")
parser.add_argument("--skip-smoke", action="store_true", help="Skip govoplan_core.devserver --smoke.") parser.add_argument("--skip-smoke", action="store_true", help="Skip govoplan_core.devserver --smoke.")
parser.add_argument(
"--skip-retirement-atomicity",
action="store_true",
help=(
"Skip the isolated destructive-retirement atomicity proof. "
"Do not use this option for a release gate."
),
)
parser.add_argument("--python", default=sys.executable, help="Python executable to use. Defaults to the current interpreter.") parser.add_argument("--python", default=sys.executable, help="Python executable to use. Defaults to the current interpreter.")
parser.add_argument("--app-env", default="staging", help="APP_ENV value to pass to subprocesses. Defaults to staging.") parser.add_argument("--app-env", default="staging", help="APP_ENV value to pass to subprocesses. Defaults to staging.")
args = parser.parse_args() args = parser.parse_args()
@@ -59,8 +67,21 @@ def main() -> int:
module_sets = _parse_module_sets(args.module_set) module_sets = _parse_module_sets(args.module_set)
ok = True ok = True
if not args.skip_retirement_atomicity:
print("\n== destructive retirement atomicity ==", flush=True)
retirement_env = _check_env(
database_url=database_url,
modules="tenancy,access,admin,audit,files",
app_env=args.app_env,
)
retirement_env["GOVOPLAN_POSTGRES_DATABASE_URL"] = database_url
retirement_env["GOVOPLAN_REQUIRE_POSTGRES_RETIREMENT_PROOF"] = "1"
ok = _run(
[args.python, "-m", "unittest", "-v", "tests.test_postgres_retirement_atomicity"],
env=retirement_env,
) and ok
for name, modules in module_sets: for name, modules in module_sets:
print(f"\n== {name}: {modules} ==") print(f"\n== {name}: {modules} ==", flush=True)
if args.reset_schema: if args.reset_schema:
_reset_public_schema(database_url) _reset_public_schema(database_url)
env = _check_env(database_url=database_url, modules=modules, app_env=args.app_env) env = _check_env(database_url=database_url, modules=modules, app_env=args.app_env)
@@ -100,7 +121,7 @@ def _check_env(*, database_url: str, modules: str, app_env: str) -> dict[str, st
def _run(command: list[str], *, env: dict[str, str]) -> bool: def _run(command: list[str], *, env: dict[str, str]) -> bool:
print("+ " + " ".join(command)) print("+ " + " ".join(command), flush=True)
result = subprocess.run(command, cwd=ROOT, env=env, check=False) result = subprocess.run(command, cwd=ROOT, env=env, check=False)
if result.returncode != 0: if result.returncode != 0:
print(f"Command failed with exit code {result.returncode}: {' '.join(command)}", file=sys.stderr) print(f"Command failed with exit code {result.returncode}: {' '.join(command)}", file=sys.stderr)

View File

@@ -0,0 +1,380 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import importlib.metadata as metadata
import os
import re
import sys
import tempfile
import unittest
from collections.abc import Iterator, Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
CORE_RELEASE_TEST_MODULES = (
"tests.test_module_system",
"tests.test_access_contracts",
"tests.test_identity_organization_contracts",
"tests.test_core_events",
"tests.test_policy_contracts",
)
# These tests intentionally compare the current source workspace with sibling
# source repositories. The release gate instead installs independently tagged
# wheels and verifies the equivalent contracts below from artifact metadata and
# public module capability factories.
SOURCE_COUPLED_CORE_TESTS: Mapping[str, str] = {
"tests.test_module_system.ModuleSystemTests.test_discovers_installed_core_and_product_module_manifests": (
"asserts the current Campaign source manifest rather than the independently tagged artifact"
),
"tests.test_module_system.ModuleSystemTests.test_module_manifest_versions_match_source_project_versions": (
"requires sibling source pyproject.toml files that wheels do not contain"
),
"tests.test_module_system.ModuleSystemTests.test_release_catalog_generator_reads_manifest_interface_contracts": (
"requires the sibling meta/source workspace and is retained by the normal source gate"
),
"tests.test_identity_organization_contracts.IdentityOrganizationContractTests."
"test_sql_identity_and_organization_directories_return_dtos": (
"directly constructs the current IDM source implementation instead of using its public capability factory"
),
}
_RELEASE_REQUIREMENT_PATTERN = re.compile(
r"^(govoplan-[a-z0-9-]+)\s+@\s+git\+ssh://.+?\.git@v[0-9]+\.[0-9]+\.[0-9]+$"
)
@dataclass(frozen=True, slots=True)
class InstalledModuleArtifact:
package_name: str
package_version: str
manifest_id: str
manifest_version: str
def release_package_names(path: Path) -> tuple[str, ...]:
names: list[str] = []
for line in path.read_text(encoding="utf-8").splitlines():
match = _RELEASE_REQUIREMENT_PATTERN.match(line.strip())
if match:
names.append(match.group(1))
return tuple(dict.fromkeys(names))
def artifact_contract_issues(
artifacts: Sequence[InstalledModuleArtifact],
*,
release_packages: Sequence[str],
available_manifest_versions: Mapping[str, str],
) -> tuple[str, ...]:
issues: list[str] = []
artifacts_by_package: dict[str, list[InstalledModuleArtifact]] = {}
artifacts_by_module: dict[str, list[InstalledModuleArtifact]] = {}
for artifact in artifacts:
artifacts_by_package.setdefault(artifact.package_name, []).append(artifact)
artifacts_by_module.setdefault(artifact.manifest_id, []).append(artifact)
if artifact.package_version != artifact.manifest_version:
issues.append(
f"{artifact.package_name} metadata version {artifact.package_version!r} does not match "
f"manifest {artifact.manifest_id!r} version {artifact.manifest_version!r}"
)
available_version = available_manifest_versions.get(artifact.manifest_id)
if available_version is None:
issues.append(f"Installed manifest {artifact.manifest_id!r} is not discoverable through Core")
elif available_version != artifact.manifest_version:
issues.append(
f"Discovered manifest {artifact.manifest_id!r} version {available_version!r} does not match "
f"its {artifact.package_name} entry point version {artifact.manifest_version!r}"
)
for package_name in release_packages:
if package_name not in artifacts_by_package:
issues.append(f"{package_name} does not publish a govoplan.modules entry point")
for module_id, providers in artifacts_by_module.items():
if len(providers) > 1:
packages = ", ".join(sorted(item.package_name for item in providers))
issues.append(f"Module id {module_id!r} is published by multiple release packages: {packages}")
return tuple(issues)
def iter_test_cases(suite: unittest.TestSuite) -> Iterator[unittest.TestCase]:
for test in suite:
if isinstance(test, unittest.TestSuite):
yield from iter_test_cases(test)
else:
yield test
def filter_test_suite(
suite: unittest.TestSuite,
excluded_ids: Sequence[str],
) -> tuple[unittest.TestSuite, tuple[str, ...]]:
excluded = set(excluded_ids)
kept: list[unittest.TestCase] = []
removed: list[str] = []
for test in iter_test_cases(suite):
test_id = test.id()
if test_id in excluded:
removed.append(test_id)
else:
kept.append(test)
return unittest.TestSuite(kept), tuple(removed)
def _collect_installed_artifacts(
package_names: Sequence[str],
) -> tuple[tuple[InstalledModuleArtifact, ...], tuple[str, ...]]:
artifacts: list[InstalledModuleArtifact] = []
issues: list[str] = []
for package_name in package_names:
try:
distribution = metadata.distribution(package_name)
except metadata.PackageNotFoundError:
issues.append(f"Release package is not installed: {package_name}")
continue
entry_points = tuple(
entry_point
for entry_point in distribution.entry_points
if entry_point.group == "govoplan.modules"
)
if not entry_points:
continue
for entry_point in entry_points:
try:
factory = entry_point.load()
manifest = factory()
manifest_id = str(manifest.id)
manifest_version = str(manifest.version)
except Exception as exc:
issues.append(f"Could not load {package_name} module entry point {entry_point.name!r}: {exc}")
continue
artifacts.append(
InstalledModuleArtifact(
package_name=package_name,
package_version=distribution.version,
manifest_id=manifest_id,
manifest_version=manifest_version,
)
)
return tuple(artifacts), tuple(issues)
def _require_equal(actual: object, expected: object, message: str) -> None:
if actual != expected:
raise RuntimeError(f"{message}: expected {expected!r}, got {actual!r}")
def _run_directory_capability_smoke(registry: object) -> None:
from govoplan_core.core.identity import CAPABILITY_IDENTITY_DIRECTORY, IdentityDirectory
from govoplan_core.core.idm import CAPABILITY_IDM_DIRECTORY, IdmDirectory
from govoplan_core.core.modules import ModuleContext
from govoplan_core.core.organizations import CAPABILITY_ORGANIZATION_DIRECTORY, OrganizationDirectory
from govoplan_core.db.base import Base
from govoplan_core.db.session import configure_database, reset_database, set_database
from govoplan_core.tenancy.scope import create_scope_tables
from govoplan_identity.backend.db.models import Identity, IdentityAccountLink
from govoplan_idm.backend.db.models import IdmOrganizationFunctionAssignment
from govoplan_organizations.backend.db.models import (
OrganizationFunction,
OrganizationFunctionType,
OrganizationUnit,
OrganizationUnitType,
)
from govoplan_tenancy.backend.db.models import Tenant
registry.configure_capability_context(ModuleContext(registry=registry, settings=object()))
identity_directory = registry.require_capability(CAPABILITY_IDENTITY_DIRECTORY)
organization_directory = registry.require_capability(CAPABILITY_ORGANIZATION_DIRECTORY)
idm_directory = registry.require_capability(CAPABILITY_IDM_DIRECTORY)
if not isinstance(identity_directory, IdentityDirectory):
raise RuntimeError("Installed Identity capability does not satisfy IdentityDirectory")
if not isinstance(organization_directory, OrganizationDirectory):
raise RuntimeError("Installed Organizations capability does not satisfy OrganizationDirectory")
if not isinstance(idm_directory, IdmDirectory):
raise RuntimeError("Installed IDM capability does not satisfy IdmDirectory")
with tempfile.TemporaryDirectory(prefix="govoplan-release-directory-") as temp_dir:
database = configure_database(
f"sqlite:///{Path(temp_dir) / 'directory.sqlite3'}",
dispose_previous=True,
)
set_database(database)
try:
create_scope_tables(database.engine)
Base.metadata.create_all(bind=database.engine)
with database.session() as session:
tenant = Tenant(
id="release-tenant",
slug="release-tenant",
name="Release Tenant",
settings={},
)
identity = Identity(
id="release-identity",
display_name="Release Person",
source="local",
)
link = IdentityAccountLink(
identity_id=identity.id,
account_id="release-account",
is_primary=True,
)
unit_type = OrganizationUnitType(
id="release-unit-type",
tenant_id=tenant.id,
slug="office",
name="Office",
)
unit = OrganizationUnit(
id="release-unit",
tenant_id=tenant.id,
unit_type_id=unit_type.id,
slug="registry",
name="Registry",
)
function_type = OrganizationFunctionType(
id="release-function-type",
tenant_id=tenant.id,
slug="clerk",
name="Clerk",
organization_unit_type_id=unit_type.id,
)
function = OrganizationFunction(
id="release-function",
tenant_id=tenant.id,
function_type_id=function_type.id,
organization_unit_id=unit.id,
slug="registry-clerk",
name="Registry Clerk",
)
assignment = IdmOrganizationFunctionAssignment(
id="release-assignment",
tenant_id=tenant.id,
identity_id=identity.id,
function_id=function.id,
organization_unit_id=unit.id,
applies_to_subunits=True,
)
session.add_all((tenant, identity, link, unit_type, unit, function_type, function, assignment))
session.commit()
identity_ref = identity_directory.identity_for_account("release-account")
_require_equal(
identity_ref.id if identity_ref is not None else None,
"release-identity",
"Identity capability account projection",
)
function_ref = organization_directory.get_function("release-function")
_require_equal(
function_ref.name if function_ref is not None else None,
"Registry Clerk",
"Organizations capability function projection",
)
identity_assignments = idm_directory.organization_function_assignments_for_identity(
"release-identity",
tenant_id="release-tenant",
)
account_assignments = idm_directory.organization_function_assignments_for_account(
"release-account",
tenant_id="release-tenant",
)
_require_equal(
tuple(item.id for item in identity_assignments),
("release-assignment",),
"IDM identity assignment projection",
)
_require_equal(
tuple(item.id for item in account_assignments),
("release-assignment",),
"IDM account assignment projection",
)
finally:
reset_database(dispose=True)
def run_installed_artifact_checks(requirements_path: Path) -> int:
from govoplan_core.server.registry import available_module_manifests, build_platform_registry
package_names = release_package_names(requirements_path)
if not package_names:
print(f"No tagged GovOPlaN packages found in {requirements_path}", file=sys.stderr)
return 1
artifacts, collection_issues = _collect_installed_artifacts(package_names)
available = available_module_manifests()
contract_issues = artifact_contract_issues(
artifacts,
release_packages=package_names,
available_manifest_versions={module_id: str(manifest.version) for module_id, manifest in available.items()},
)
issues = (*collection_issues, *contract_issues)
if issues:
print("\n".join(issues), file=sys.stderr)
return 1
module_ids = tuple(dict.fromkeys(artifact.manifest_id for artifact in artifacts))
registry = build_platform_registry(module_ids)
snapshot = registry.validate()
discovered_ids = {manifest.id for manifest in snapshot.manifests}
if discovered_ids != set(module_ids):
print(
"Release registry module mismatch: "
f"expected {', '.join(sorted(module_ids))}; got {', '.join(sorted(discovered_ids))}",
file=sys.stderr,
)
return 1
_run_directory_capability_smoke(registry)
versions = ", ".join(
f"{artifact.manifest_id}={artifact.manifest_version}"
for artifact in sorted(artifacts, key=lambda item: item.manifest_id)
)
print(f"Installed release artifact contracts passed: {versions}")
return 0
def run_core_release_tests(core_root: Path) -> int:
resolved_root = core_root.resolve()
os.chdir(resolved_root)
sys.path.insert(0, str(resolved_root))
loader = unittest.defaultTestLoader
loaded = loader.loadTestsFromNames(CORE_RELEASE_TEST_MODULES)
suite, removed = filter_test_suite(loaded, tuple(SOURCE_COUPLED_CORE_TESTS))
missing_exclusions = set(SOURCE_COUPLED_CORE_TESTS) - set(removed)
if missing_exclusions:
print(
"Release source-test exclusions no longer match the Core suite: "
+ ", ".join(sorted(missing_exclusions)),
file=sys.stderr,
)
return 1
for test_id in removed:
print(f"Artifact contract replaces source-coupled test: {test_id} ({SOURCE_COUPLED_CORE_TESTS[test_id]})")
result = unittest.TextTestRunner(verbosity=1).run(suite)
return 0 if result.wasSuccessful() else 1
def _parser() -> argparse.ArgumentParser:
meta_root = Path(__file__).resolve().parents[2]
parser = argparse.ArgumentParser(description="Artifact-aware GovOPlaN release integration checks")
subparsers = parser.add_subparsers(dest="command", required=True)
artifact_parser = subparsers.add_parser("artifacts", help="Validate installed module artifacts and capabilities")
artifact_parser.add_argument(
"--requirements",
type=Path,
default=meta_root / "requirements-release.txt",
)
core_parser = subparsers.add_parser("core-tests", help="Run Core tests that are valid for tagged module artifacts")
core_parser.add_argument("--core-root", type=Path, required=True)
return parser
def main(argv: Sequence[str] | None = None) -> int:
args = _parser().parse_args(argv)
if args.command == "artifacts":
return run_installed_artifact_checks(args.requirements)
return run_core_release_tests(args.core_root)
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -31,17 +31,24 @@ RUN apt-get update \
COPY --from=go-tools /go/bin/gitleaks /usr/local/bin/gitleaks COPY --from=go-tools /go/bin/gitleaks /usr/local/bin/gitleaks
COPY --from=go-tools /go/bin/osv-scanner /usr/local/bin/osv-scanner COPY --from=go-tools /go/bin/osv-scanner /usr/local/bin/osv-scanner
COPY requirements-audit.txt /tmp/requirements-audit.txt COPY requirements-audit.txt /tmp/requirements-audit.txt
COPY tools/checks/security-audit/semgrep-govoplan.yml /tmp/semgrep-govoplan.yml
COPY tools/checks/check-version-alignment.py /tmp/semgrep-smoke.py
RUN python -m pip install --upgrade pip \ RUN python -m pip install --upgrade pip \
&& python -m pip install --no-cache-dir -r /tmp/requirements-audit.txt \ && python -m pip install --no-cache-dir -r /tmp/requirements-audit.txt \
&& python -m pip install --no-cache-dir "semgrep>=1.140,<2" \
&& python -m pip install --no-cache-dir --upgrade --no-deps "click==8.3.3" \
&& python -m pip install --no-cache-dir --upgrade --no-deps "mcp==1.28.1" \
&& npm install -g jscpd \ && npm install -g jscpd \
&& semgrep --version \ && semgrep --version \
&& semgrep scan --metrics=off --config /tmp/semgrep-govoplan.yml /tmp/semgrep-smoke.py \
&& bandit --version \ && bandit --version \
&& ruff --version \ && ruff --version \
&& gitleaks version \ && gitleaks version \
&& osv-scanner --version \ && osv-scanner --version \
&& trivy --version \ && trivy --version \
&& jscpd --version && jscpd --version \
&& pip-audit --progress-spinner off
RUN mkdir -p /workspace \ RUN mkdir -p /workspace \
&& chmod 0777 /workspace && chmod 0777 /workspace

View File

@@ -0,0 +1,202 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import posixpath
from pathlib import Path
import re
import sys
from typing import Any
VOLUME_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*")
UNSAFE_BIND_SOURCES = {
Path("/"),
Path("/bin"),
Path("/boot"),
Path("/dev"),
Path("/etc"),
Path("/home"),
Path("/lib"),
Path("/lib64"),
Path("/mnt"),
Path("/opt"),
Path("/proc"),
Path("/root"),
Path("/run"),
Path("/sbin"),
Path("/srv"),
Path("/sys"),
Path("/tmp"),
Path("/usr"),
Path("/var"),
Path("/var/run"),
}
SENSITIVE_BIND_ROOTS = {
Path("/dev"),
Path("/etc"),
Path("/proc"),
Path("/root"),
Path("/run"),
Path("/sys"),
Path("/var/run"),
}
class MountResolutionError(ValueError):
pass
def normalized_absolute_path(
value: str,
*,
field: str,
resolve_symlinks: bool = True,
) -> Path:
if not value.startswith("/") or value.startswith("//"):
raise MountResolutionError(f"{field} must be an absolute path")
path = Path(posixpath.normpath(value))
return path.resolve(strict=False) if resolve_symlinks else path
def contains_path(container: Path, path: Path) -> bool:
return path == container or container in path.parents
def mount_option_value(value: str, *, field: str) -> str:
if not value or any(character in value for character in (",", "\n", "\r")):
raise MountResolutionError(f"{field} cannot be represented safely")
return value
def resolve_workspace_mount(
mounts: Any,
*,
root: str,
scope: str,
reports_dir: str,
) -> str:
if scope not in {"current", "govoplan"}:
raise MountResolutionError(f"unsupported audit scope: {scope}")
if not isinstance(mounts, list):
raise MountResolutionError("Docker mount metadata must be a list")
root_path = normalized_absolute_path(root, field="audit root")
scan_root = root_path if scope == "current" else root_path.parent
reports_path = Path(reports_dir)
if not reports_path.is_absolute():
reports_path = root_path / reports_path
reports_path = reports_path.resolve(strict=False)
parsed_mounts: list[tuple[Path, dict[str, Any]]] = []
for mount in mounts:
if not isinstance(mount, dict) or not isinstance(mount.get("Destination"), str):
continue
destination = normalized_absolute_path(
mount["Destination"],
field="mount destination",
)
parsed_mounts.append((destination, mount))
candidates = [
(destination, mount)
for destination, mount in parsed_mounts
if destination != Path("/") and contains_path(destination, scan_root)
]
if not candidates:
raise MountResolutionError(
f"no job-container mount covers audit scope root {scan_root}"
)
longest_depth = max(len(destination.parts) for destination, _ in candidates)
selected = [
candidate
for candidate in candidates
if len(candidate[0].parts) == longest_depth
]
if len(selected) != 1:
raise MountResolutionError("job-container workspace mount is ambiguous")
destination, mount = selected[0]
nested_mounts = [
nested_destination
for nested_destination, _ in parsed_mounts
if nested_destination != destination
and contains_path(scan_root, nested_destination)
]
if nested_mounts:
raise MountResolutionError(
"nested job-container mounts inside the audit scope cannot be "
"reproduced safely"
)
if mount.get("RW") is not True:
raise MountResolutionError("job-container workspace mount is not writable")
if not contains_path(destination, reports_path):
raise MountResolutionError(
f"audit reports path {reports_path} is outside the workspace mount"
)
destination_value = mount_option_value(
str(destination),
field="mount destination",
)
mount_type = mount.get("Type")
if mount_type == "volume":
name = mount.get("Name")
if not isinstance(name, str) or VOLUME_NAME.fullmatch(name) is None:
raise MountResolutionError("workspace volume name is missing or malformed")
source_value = name
elif mount_type == "bind":
source = mount.get("Source")
if not isinstance(source, str):
raise MountResolutionError("workspace bind source is missing")
source_path = normalized_absolute_path(
source,
field="mount source",
resolve_symlinks=False,
)
if source_path in UNSAFE_BIND_SOURCES or any(
contains_path(sensitive_root, source_path)
for sensitive_root in SENSITIVE_BIND_ROOTS
):
raise MountResolutionError(
"workspace bind source is too broad or sensitive"
)
source_value = mount_option_value(str(source_path), field="mount source")
else:
raise MountResolutionError(f"unsupported workspace mount type: {mount_type!r}")
return f"type={mount_type},source={source_value},target={destination_value}"
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Resolve one safe Gitea Actions workspace mount for an audit container."
)
parser.add_argument("--root", required=True)
parser.add_argument("--scope", choices=("current", "govoplan"), required=True)
parser.add_argument("--reports-dir", required=True)
return parser.parse_args()
def main() -> int:
args = parse_args()
try:
mounts = json.load(sys.stdin)
print(
resolve_workspace_mount(
mounts,
root=args.root,
scope=args.scope,
reports_dir=args.reports_dir,
)
)
except (json.JSONDecodeError, MountResolutionError) as exc:
print(f"workspace mount error: {exc}", file=sys.stderr)
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main())

85
tools/checks/security-audit/run.sh Normal file → Executable file
View File

@@ -71,6 +71,32 @@ REBUILD="${REBUILD:-0}"
UPDATE="${UPDATE:-0}" UPDATE="${UPDATE:-0}"
BUILD_ONLY="${BUILD_ONLY:-0}" BUILD_ONLY="${BUILD_ONLY:-0}"
declare -a DOCKER_CLI=()
DOCKER_LOCATION=""
if command -v docker >/dev/null 2>&1; then
DOCKER_CLI=(docker)
DOCKER_LOCATION="current environment"
elif command -v flatpak-spawn >/dev/null 2>&1 \
&& flatpak-spawn --host sh -lc 'command -v docker >/dev/null 2>&1'; then
DOCKER_CLI=(flatpak-spawn --host docker)
DOCKER_LOCATION="Flatpak host"
else
echo "Docker is not available in this environment or through a Flatpak host bridge." >&2
echo "Install Docker, expose its CLI/socket, or run this script from the host." >&2
exit 1
fi
if ! DOCKER_VERSION_OUTPUT="$("${DOCKER_CLI[@]}" version --format '{{.Server.Version}}' 2>&1)"; then
echo "Docker was found on the $DOCKER_LOCATION, but its daemon is not accessible:" >&2
printf ' %s\n' "$DOCKER_VERSION_OUTPUT" >&2
if [[ "$DOCKER_LOCATION" == "Flatpak host" && "$DOCKER_VERSION_OUTPUT" == *"permission denied"* ]]; then
echo "Grant the host account access to the Docker socket, then fully log out and back in." >&2
echo "On a conventional rootful Docker installation: sudo usermod -aG docker \"$(id -un)\"" >&2
echo "Membership in the docker group is root-equivalent; use rootless Docker instead where that is preferred." >&2
fi
exit 1
fi
IMAGE_NAME="${IMAGE##*/}" IMAGE_NAME="${IMAGE##*/}"
if [[ "$IMAGE_NAME" == *:* ]]; then if [[ "$IMAGE_NAME" == *:* ]]; then
IMAGE_REPOSITORY="${IMAGE%:*}" IMAGE_REPOSITORY="${IMAGE%:*}"
@@ -84,6 +110,8 @@ IMAGE_FINGERPRINT="$(
{ {
sha256sum "$ROOT/tools/checks/security-audit/Dockerfile" sha256sum "$ROOT/tools/checks/security-audit/Dockerfile"
sha256sum "$ROOT/requirements-audit.txt" sha256sum "$ROOT/requirements-audit.txt"
sha256sum "$ROOT/tools/checks/security-audit/semgrep-govoplan.yml"
sha256sum "$ROOT/tools/checks/check-version-alignment.py"
} | sha256sum | cut -c1-16 } | sha256sum | cut -c1-16
)" )"
FINGERPRINT_IMAGE="${SECURITY_AUDIT_FINGERPRINT_IMAGE:-$IMAGE_REPOSITORY:$IMAGE_FINGERPRINT}" FINGERPRINT_IMAGE="${SECURITY_AUDIT_FINGERPRINT_IMAGE:-$IMAGE_REPOSITORY:$IMAGE_FINGERPRINT}"
@@ -93,35 +121,75 @@ build_image() {
if [[ "$UPDATE" == "1" ]]; then if [[ "$UPDATE" == "1" ]]; then
build_args=(--pull --no-cache "${build_args[@]}") build_args=(--pull --no-cache "${build_args[@]}")
fi fi
docker build "${build_args[@]}" "$ROOT" "${DOCKER_CLI[@]}" build "${build_args[@]}" "$ROOT"
} }
if [[ "$REBUILD" == "1" ]] || ! docker image inspect "$FINGERPRINT_IMAGE" >/dev/null 2>&1; then if [[ "$REBUILD" == "1" ]] || ! "${DOCKER_CLI[@]}" image inspect "$FINGERPRINT_IMAGE" >/dev/null 2>&1; then
echo "Building security audit toolbox image: $FINGERPRINT_IMAGE" echo "Building security audit toolbox image: $FINGERPRINT_IMAGE"
build_image build_image
else else
echo "Reusing security audit toolbox image: $FINGERPRINT_IMAGE" echo "Reusing security audit toolbox image: $FINGERPRINT_IMAGE"
if ! docker image inspect "$IMAGE_ALIAS" >/dev/null 2>&1; then if ! "${DOCKER_CLI[@]}" image inspect "$IMAGE_ALIAS" >/dev/null 2>&1; then
docker tag "$FINGERPRINT_IMAGE" "$IMAGE_ALIAS" "${DOCKER_CLI[@]}" tag "$FINGERPRINT_IMAGE" "$IMAGE_ALIAS"
fi fi
fi fi
if [[ "$BUILD_ONLY" == "1" ]]; then if [[ "$BUILD_ONLY" == "1" ]]; then
docker image inspect "$FINGERPRINT_IMAGE" --format 'Built audit toolbox image {{.RepoTags}} {{.Id}}' "${DOCKER_CLI[@]}" image inspect "$FINGERPRINT_IMAGE" --format 'Built audit toolbox image {{.RepoTags}} {{.Id}}'
exit 0 exit 0
fi fi
if [[ "$SCOPE" == "govoplan" ]]; then declare -a WORKSPACE_MOUNT=()
if [[ "${GITEA_ACTIONS:-}" == "true" ]]; then
ACTIONS_CONTAINER_ID="$(hostname)"
if ! ACTIONS_MOUNTS_JSON="$(
"${DOCKER_CLI[@]}" container inspect \
--format '{{json .Mounts}}' \
"$ACTIONS_CONTAINER_ID"
)"; then
echo "Gitea Actions is using a host Docker daemon, but the current job container could not be resolved." >&2
echo "Cannot safely share the checked-out workspace with the audit container." >&2
exit 1
fi
if command -v python3 >/dev/null 2>&1; then
MOUNT_PYTHON=python3
elif command -v python >/dev/null 2>&1; then
MOUNT_PYTHON=python
else
echo "Python is required to validate the Gitea Actions workspace mount." >&2
exit 1
fi
if ! ACTIONS_WORKSPACE_MOUNT="$(
printf '%s' "$ACTIONS_MOUNTS_JSON" \
| "$MOUNT_PYTHON" "$ROOT/tools/checks/security-audit/resolve_workspace_mount.py" \
--root "$ROOT" \
--scope "$SCOPE" \
--reports-dir "$REPORTS_DIR"
)"; then
echo "Cannot safely share the Gitea Actions workspace with the audit container." >&2
exit 1
fi
WORKSPACE_MOUNT=(--mount "$ACTIONS_WORKSPACE_MOUNT")
MOUNT_ROOT="$(dirname "$ROOT")"
WORKDIR="$ROOT"
if [[ "$SCOPE" == "govoplan" ]]; then
EXTRA_ENV=(-e "GOVOPLAN_REPOS_ROOT=$MOUNT_ROOT")
else
EXTRA_ENV=()
fi
elif [[ "$SCOPE" == "govoplan" ]]; then
MOUNT_ROOT="$(dirname "$ROOT")" MOUNT_ROOT="$(dirname "$ROOT")"
WORKDIR="/workspace/$(basename "$ROOT")" WORKDIR="/workspace/$(basename "$ROOT")"
WORKSPACE_MOUNT=(-v "$MOUNT_ROOT:/workspace")
EXTRA_ENV=(-e "GOVOPLAN_REPOS_ROOT=/workspace") EXTRA_ENV=(-e "GOVOPLAN_REPOS_ROOT=/workspace")
else else
MOUNT_ROOT="$ROOT" MOUNT_ROOT="$ROOT"
WORKDIR="/workspace" WORKDIR="/workspace"
WORKSPACE_MOUNT=(-v "$MOUNT_ROOT:/workspace")
EXTRA_ENV=() EXTRA_ENV=()
fi fi
docker run --rm \ "${DOCKER_CLI[@]}" run --rm \
--user "$(id -u):$(id -g)" \ --user "$(id -u):$(id -g)" \
-e HOME=/tmp/security-audit-home \ -e HOME=/tmp/security-audit-home \
-e SECURITY_AUDIT_SCOPE="$SCOPE" \ -e SECURITY_AUDIT_SCOPE="$SCOPE" \
@@ -129,8 +197,9 @@ docker run --rm \
-e SECURITY_AUDIT_REPORTS_DIR="$REPORTS_DIR" \ -e SECURITY_AUDIT_REPORTS_DIR="$REPORTS_DIR" \
-e SECURITY_AUDIT_FAIL_ON_FINDINGS="${SECURITY_AUDIT_FAIL_ON_FINDINGS:-0}" \ -e SECURITY_AUDIT_FAIL_ON_FINDINGS="${SECURITY_AUDIT_FAIL_ON_FINDINGS:-0}" \
-e SECURITY_AUDIT_REQUIRE_TOOLS="${SECURITY_AUDIT_REQUIRE_TOOLS:-0}" \ -e SECURITY_AUDIT_REQUIRE_TOOLS="${SECURITY_AUDIT_REQUIRE_TOOLS:-0}" \
-e SECURITY_AUDIT_TOOLBOX_FINGERPRINT="$IMAGE_FINGERPRINT" \
"${EXTRA_ENV[@]}" \ "${EXTRA_ENV[@]}" \
-v "$MOUNT_ROOT:/workspace" \ "${WORKSPACE_MOUNT[@]}" \
-w "$WORKDIR" \ -w "$WORKDIR" \
"$FINGERPRINT_IMAGE" \ "$FINGERPRINT_IMAGE" \
bash tools/checks/check-security-audit.sh --mode "$MODE" --scope "$SCOPE" --reports-dir "$REPORTS_DIR" "${SCRIPT_ARGS[@]}" bash tools/checks/check-security-audit.sh --mode "$MODE" --scope "$SCOPE" --reports-dir "$REPORTS_DIR" "${SCRIPT_ARGS[@]}"

View File

@@ -0,0 +1,9 @@
#!/usr/bin/env python3
"""Compatibility wrapper for the release catalog generator."""
from __future__ import annotations
from pathlib import Path
import runpy
runpy.run_path(str(Path(__file__).resolve().parent / "release" / "generate-release-catalog.py"), run_name="__main__")

View File

@@ -7,7 +7,6 @@ import argparse
import base64 import base64
import dataclasses import dataclasses
import hashlib import hashlib
import os
import pathlib import pathlib
import re import re
import shutil import shutil

View File

@@ -4,14 +4,13 @@
from __future__ import annotations from __future__ import annotations
import dataclasses import dataclasses
import http.client
import json import json
import os import os
import pathlib import pathlib
import re import re
import subprocess import subprocess
import urllib.error
import urllib.parse import urllib.parse
import urllib.request
from typing import Any from typing import Any
@@ -31,7 +30,7 @@ class RepoTarget:
@property @property
def api_base(self) -> str: def api_base(self) -> str:
return f"{self.base_url.rstrip('/')}/api/v1" return f"{validated_http_base_url(self.base_url)}/api/v1"
def repo_root(start: pathlib.Path) -> pathlib.Path: def repo_root(start: pathlib.Path) -> pathlib.Path:
@@ -80,10 +79,33 @@ def quote_path(value: str) -> str:
return urllib.parse.quote(value, safe="") return urllib.parse.quote(value, safe="")
def validated_http_base_url(value: str) -> str:
parsed = urllib.parse.urlparse(value.rstrip("/"))
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise GiteaError("Gitea URL must use http:// or https:// with a hostname.")
if parsed.username or parsed.password:
raise GiteaError("Gitea URL must not include embedded credentials.")
return urllib.parse.urlunparse((parsed.scheme, parsed.netloc, parsed.path.rstrip("/"), "", "", ""))
class GiteaClient: class GiteaClient:
def __init__(self, target: RepoTarget, token: str | None) -> None: def __init__(self, target: RepoTarget, token: str | None, *, timeout: float = 30.0) -> None:
self.target = target self.target = target
self.token = token self.token = token
self.timeout = timeout
self._api_url = urllib.parse.urlparse(target.api_base)
self._connection: http.client.HTTPConnection | None = None
def __enter__(self) -> GiteaClient:
return self
def __exit__(self, *_exc_info: object) -> None:
self.close()
def close(self) -> None:
if self._connection is not None:
self._connection.close()
self._connection = None
def request_json( def request_json(
self, self,
@@ -93,14 +115,13 @@ class GiteaClient:
body: dict[str, Any] | None = None, body: dict[str, Any] | None = None,
query: dict[str, Any] | None = None, query: dict[str, Any] | None = None,
) -> Any: ) -> Any:
url = f"{self.target.api_base}{path}" request_target = self._request_target(path, query)
if query:
url = f"{url}?{urllib.parse.urlencode(query, doseq=True)}"
data = None data = None
headers = { headers = {
"Accept": "application/json", "Accept": "application/json",
"User-Agent": "codex-gitea-maintenance", "User-Agent": "codex-gitea-maintenance",
"Connection": "keep-alive",
} }
if self.token: if self.token:
headers["Authorization"] = f"token {self.token}" headers["Authorization"] = f"token {self.token}"
@@ -108,18 +129,71 @@ class GiteaClient:
data = json.dumps(body).encode("utf-8") data = json.dumps(body).encode("utf-8")
headers["Content-Type"] = "application/json" headers["Content-Type"] = "application/json"
request = urllib.request.Request(url, data=data, headers=headers, method=method) method = method.upper()
safe_to_retry = method in {"GET", "HEAD", "OPTIONS"}
last_error: BaseException | None = None
for attempt in range(2 if safe_to_retry else 1):
try:
status, _reason, payload = self._request_once(method, request_target, data, headers)
break
except (OSError, http.client.HTTPException) as exc:
self.close()
last_error = exc
if attempt == 0 and safe_to_retry:
continue
raise GiteaError(f"{method} {self._display_url(request_target)} failed: {exc}") from exc
else:
raise GiteaError(f"{method} {self._display_url(request_target)} failed: {last_error}")
if status >= 400:
message = payload.decode("utf-8", errors="replace")
raise GiteaError(f"{method} {self._display_url(request_target)} failed with HTTP {status}: {message}")
if not payload:
return None
return json.loads(payload.decode("utf-8"))
def _connection_for_request(self) -> http.client.HTTPConnection:
if self._connection is not None:
return self._connection
host = self._api_url.hostname
if not host:
raise GiteaError("Gitea API URL is missing a hostname.")
if self._api_url.scheme == "https":
self._connection = http.client.HTTPSConnection(host, self._api_url.port, timeout=self.timeout) # nosemgrep: python.lang.security.audit.httpsconnection-detected.httpsconnection-detected
elif self._api_url.scheme == "http":
self._connection = http.client.HTTPConnection(host, self._api_url.port, timeout=self.timeout)
else:
raise GiteaError("Gitea API URL must use http:// or https://.")
return self._connection
def _request_once(
self,
method: str,
request_target: str,
data: bytes | None,
headers: dict[str, str],
) -> tuple[int, str, bytes]:
connection = self._connection_for_request()
connection.request(method, request_target, body=data, headers=headers)
response = connection.getresponse()
try: try:
with urllib.request.urlopen(request, timeout=30) as response: payload = response.read()
payload = response.read().decode("utf-8") return response.status, response.reason, payload
if not payload: finally:
return None if response.will_close:
return json.loads(payload) self.close()
except urllib.error.HTTPError as exc:
message = exc.read().decode("utf-8", errors="replace") def _request_target(self, path: str, query: dict[str, Any] | None) -> str:
raise GiteaError(f"{method} {url} failed with HTTP {exc.code}: {message}") from exc base_path = self._api_url.path.rstrip("/")
except urllib.error.URLError as exc: suffix = path if path.startswith("/") else f"/{path}"
raise GiteaError(f"{method} {url} failed: {exc.reason}") from exc request_target = f"{base_path}{suffix}"
if query:
request_target = f"{request_target}?{urllib.parse.urlencode(query, doseq=True)}"
return request_target
def _display_url(self, request_target: str) -> str:
netloc = self._api_url.netloc
return urllib.parse.urlunparse((self._api_url.scheme, netloc, request_target, "", "", ""))
def paginate( def paginate(
self, self,

View File

@@ -15,11 +15,12 @@ BACKEND_HOST="${GOVOPLAN_BACKEND_HOST:-127.0.0.1}"
BACKEND_PORT="${GOVOPLAN_BACKEND_PORT:-8000}" BACKEND_PORT="${GOVOPLAN_BACKEND_PORT:-8000}"
FRONTEND_HOST="${GOVOPLAN_FRONTEND_HOST:-127.0.0.1}" FRONTEND_HOST="${GOVOPLAN_FRONTEND_HOST:-127.0.0.1}"
FRONTEND_PORT="${GOVOPLAN_FRONTEND_PORT:-5173}" FRONTEND_PORT="${GOVOPLAN_FRONTEND_PORT:-5173}"
OPEN_BROWSER="${OPEN_BROWSER:-1}" OPEN_BROWSER="${GOVOPLAN_OPEN_BROWSER:-${OPEN_BROWSER:-0}}"
FRONTEND_FORCE_RELOAD="${GOVOPLAN_FRONTEND_FORCE_RELOAD:-1}" FRONTEND_FORCE_RELOAD="${GOVOPLAN_FRONTEND_FORCE_RELOAD:-1}"
FRONTEND_CLEAR_VITE_CACHE="${GOVOPLAN_FRONTEND_CLEAR_VITE_CACHE:-1}" FRONTEND_CLEAR_VITE_CACHE="${GOVOPLAN_FRONTEND_CLEAR_VITE_CACHE:-1}"
FRONTEND_USE_POLLING="${GOVOPLAN_FRONTEND_USE_POLLING:-1}" FRONTEND_USE_POLLING="${GOVOPLAN_FRONTEND_USE_POLLING:-1}"
FRONTEND_POLLING_INTERVAL="${GOVOPLAN_FRONTEND_POLLING_INTERVAL:-500}" FRONTEND_POLLING_INTERVAL="${GOVOPLAN_FRONTEND_POLLING_INTERVAL:-500}"
AUTO_SYNC_PYTHON="${GOVOPLAN_AUTO_SYNC_PYTHON:-1}"
DEV_DATABASE_BACKEND="${GOVOPLAN_DEV_DATABASE_BACKEND:-postgres}" DEV_DATABASE_BACKEND="${GOVOPLAN_DEV_DATABASE_BACKEND:-postgres}"
LOG_DIR="${GOVOPLAN_DEV_LOG_DIR:-$META_ROOT/runtime/dev-launcher}" LOG_DIR="${GOVOPLAN_DEV_LOG_DIR:-$META_ROOT/runtime/dev-launcher}"
@@ -137,6 +138,14 @@ trap cleanup EXIT INT TERM
[ -f "$WEBUI_ROOT/package.json" ] || fail "WebUI package.json not found at $WEBUI_ROOT/package.json" [ -f "$WEBUI_ROOT/package.json" ] || fail "WebUI package.json not found at $WEBUI_ROOT/package.json"
[ -d "$WEBUI_ROOT/node_modules" ] || fail "WebUI node_modules missing. Run: cd $WEBUI_ROOT && PATH=$NODE_BIN:\$PATH $NPM install" [ -d "$WEBUI_ROOT/node_modules" ] || fail "WebUI node_modules missing. Run: cd $WEBUI_ROOT && PATH=$NODE_BIN:\$PATH $NPM install"
if [ "$AUTO_SYNC_PYTHON" = "1" ]; then
"$PYTHON" "$META_ROOT/tools/repo/sync-python-environment.py" \
--requirements "$META_ROOT/requirements-dev.txt" \
--python "$PYTHON" \
--stamp "$VENV_ROOT/.govoplan-sync-requirements-dev.json" \
|| fail "Python environment synchronization failed"
fi
( (
cd "$META_ROOT" cd "$META_ROOT"
"$PYTHON" - <<'PY' "$PYTHON" - <<'PY'
@@ -210,6 +219,7 @@ Frontend reload:
Vite cache cleared: $FRONTEND_CLEAR_VITE_CACHE Vite cache cleared: $FRONTEND_CLEAR_VITE_CACHE
Vite --force: $FRONTEND_FORCE_RELOAD Vite --force: $FRONTEND_FORCE_RELOAD
Watch polling: $FRONTEND_USE_POLLING (${FRONTEND_POLLING_INTERVAL}ms) Watch polling: $FRONTEND_USE_POLLING (${FRONTEND_POLLING_INTERVAL}ms)
Open browser: $OPEN_BROWSER
Press Ctrl+C to stop both processes. Press Ctrl+C to stop both processes.
EOF EOF

View File

@@ -27,12 +27,14 @@ STOP_DEPENDENCIES_ON_EXIT="${GOVOPLAN_STOP_PROFILE_DEPENDENCIES_ON_EXIT:-0}"
LOG_DIR="$META_ROOT/runtime/production-like/logs" LOG_DIR="$META_ROOT/runtime/production-like/logs"
BACKEND_LOG="$LOG_DIR/backend.log" BACKEND_LOG="$LOG_DIR/backend.log"
WORKER_LOG="$LOG_DIR/worker.log" WORKER_LOG="$LOG_DIR/worker.log"
BEAT_LOG="$LOG_DIR/beat.log"
FRONTEND_LOG="$LOG_DIR/frontend.log" FRONTEND_LOG="$LOG_DIR/frontend.log"
BACKEND_URL="http://$BACKEND_HOST:$BACKEND_PORT" BACKEND_URL="http://$BACKEND_HOST:$BACKEND_PORT"
FRONTEND_URL="http://$FRONTEND_HOST:$FRONTEND_PORT" FRONTEND_URL="http://$FRONTEND_HOST:$FRONTEND_PORT"
backend_pid="" backend_pid=""
worker_pid="" worker_pid=""
beat_pid=""
frontend_pid="" frontend_pid=""
fail() { fail() {
@@ -49,12 +51,12 @@ set +a
export APP_ENV="${APP_ENV:-staging}" export APP_ENV="${APP_ENV:-staging}"
export GOVOPLAN_INSTALL_PROFILE="${GOVOPLAN_INSTALL_PROFILE:-production-like}" export GOVOPLAN_INSTALL_PROFILE="${GOVOPLAN_INSTALL_PROFILE:-production-like}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops}" export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,notifications,docs,ops}"
export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}" export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}" export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}" export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}"
export CELERY_ENABLED="${CELERY_ENABLED:-true}" export CELERY_ENABLED="${CELERY_ENABLED:-true}"
export CELERY_QUEUES="${CELERY_QUEUES:-send_email,append_sent,default}" export CELERY_QUEUES="${CELERY_QUEUES:-send_email,append_sent,notifications,calendar,default}"
export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}" export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}"
export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}" export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}"
export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}" export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}"
@@ -146,6 +148,9 @@ cleanup() {
if [ -n "${worker_pid:-}" ] && kill -0 "$worker_pid" 2>/dev/null; then if [ -n "${worker_pid:-}" ] && kill -0 "$worker_pid" 2>/dev/null; then
kill "$worker_pid" 2>/dev/null || true kill "$worker_pid" 2>/dev/null || true
fi fi
if [ -n "${beat_pid:-}" ] && kill -0 "$beat_pid" 2>/dev/null; then
kill "$beat_pid" 2>/dev/null || true
fi
if [ "$STOP_DEPENDENCIES_ON_EXIT" = "1" ]; then if [ "$STOP_DEPENDENCIES_ON_EXIT" = "1" ]; then
compose down >/dev/null 2>&1 || true compose down >/dev/null 2>&1 || true
fi fi
@@ -160,6 +165,7 @@ trap cleanup EXIT INT TERM
mkdir -p "$LOG_DIR" "$FILE_STORAGE_LOCAL_ROOT" mkdir -p "$LOG_DIR" "$FILE_STORAGE_LOCAL_ROOT"
: > "$BACKEND_LOG" : > "$BACKEND_LOG"
: > "$WORKER_LOG" : > "$WORKER_LOG"
: > "$BEAT_LOG"
: > "$FRONTEND_LOG" : > "$FRONTEND_LOG"
port_is_free "$BACKEND_HOST" "$BACKEND_PORT" || fail "$BACKEND_URL is already in use" port_is_free "$BACKEND_HOST" "$BACKEND_PORT" || fail "$BACKEND_URL is already in use"
@@ -200,6 +206,14 @@ printf 'Starting Celery worker for queues %s\n' "$CELERY_QUEUES"
) >"$WORKER_LOG" 2>&1 & ) >"$WORKER_LOG" 2>&1 &
worker_pid="$!" worker_pid="$!"
printf 'Starting Celery beat for durable recovery schedules\n'
(
cd "$ROOT"
"$PYTHON" -m celery -A govoplan_core.celery_app:celery beat \
--loglevel INFO
) >"$BEAT_LOG" 2>&1 &
beat_pid="$!"
printf 'Starting GovOPlaN WebUI at %s\n' "$FRONTEND_URL" printf 'Starting GovOPlaN WebUI at %s\n' "$FRONTEND_URL"
( (
cd "$WEBUI_ROOT" cd "$WEBUI_ROOT"
@@ -235,10 +249,11 @@ Files: $FILE_STORAGE_LOCAL_ROOT
Logs: Logs:
Backend: $BACKEND_LOG Backend: $BACKEND_LOG
Worker: $WORKER_LOG Worker: $WORKER_LOG
Beat: $BEAT_LOG
WebUI: $FRONTEND_LOG WebUI: $FRONTEND_LOG
Docker dependencies remain running after Ctrl+C unless GOVOPLAN_STOP_PROFILE_DEPENDENCIES_ON_EXIT=1. Docker dependencies remain running after Ctrl+C unless GOVOPLAN_STOP_PROFILE_DEPENDENCIES_ON_EXIT=1.
Press Ctrl+C to stop API, worker, and WebUI. Press Ctrl+C to stop API, worker, beat, and WebUI.
EOF EOF
wait wait

View File

@@ -19,7 +19,7 @@ usage() {
Usage: tools/launch/production-like-dev.sh <command> [--yes] Usage: tools/launch/production-like-dev.sh <command> [--yes]
Commands: Commands:
start Start the production-like API, worker, WebUI, PostgreSQL, and Redis profile. start Start the production-like API, worker, scheduler, WebUI, PostgreSQL, and Redis profile.
stop Stop PostgreSQL and Redis profile containers. stop Stop PostgreSQL and Redis profile containers.
status Show profile container status and validate the effective environment. status Show profile container status and validate the effective environment.
seed Run migrations and seed development data against the profile database. seed Run migrations and seed development data against the profile database.
@@ -27,7 +27,7 @@ Commands:
validate-config Validate the profile environment only. validate-config Validate the profile environment only.
The start command delegates to tools/launch/launch-production-like-dev.sh. Stop/reset The start command delegates to tools/launch/launch-production-like-dev.sh. Stop/reset
operate on Docker dependencies; API/WebUI/worker processes started by the operate on Docker dependencies; API/WebUI/worker/scheduler processes started by the
launcher are stopped with Ctrl+C in that launcher terminal. launcher are stopped with Ctrl+C in that launcher terminal.
EOF EOF
} }
@@ -66,7 +66,7 @@ export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-po
export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}" export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}" export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}"
export CELERY_ENABLED="${CELERY_ENABLED:-true}" export CELERY_ENABLED="${CELERY_ENABLED:-true}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,docs,ops}" export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,notifications,docs,ops}"
export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}" export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}"
export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}" export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}"
export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}" export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}"

View File

@@ -19,9 +19,11 @@ from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[2] META_ROOT = Path(__file__).resolve().parents[2]
CORE_ROOT = Path(os.environ.get("GOVOPLAN_CORE_ROOT", META_ROOT.parent / "govoplan-core")).resolve() CORE_ROOT = Path(os.environ.get("GOVOPLAN_CORE_ROOT", META_ROOT.parent / "govoplan-core")).resolve()
sys.path.insert(0, str(CORE_ROOT / "src")) sys.path.insert(0, str(CORE_ROOT / "src"))
sys.path.insert(0, str(META_ROOT / "tools" / "release"))
from govoplan_core.core.modules import ModuleManifest # noqa: E402 from govoplan_core.core.modules import ModuleManifest # noqa: E402
from govoplan_core.server.registry import available_module_manifests # noqa: E402 from govoplan_core.server.registry import available_module_manifests # noqa: E402
from govoplan_release.version_alignment import selected_repository_version_issues # noqa: E402
GITEA_BASE = "git+ssh://git@git.add-ideas.de/add-ideas" GITEA_BASE = "git+ssh://git@git.add-ideas.de/add-ideas"
@@ -110,6 +112,15 @@ CATALOG_MODULES = (
tags=("official", "platform-module"), tags=("official", "platform-module"),
webui_package="@govoplan/dashboard-webui", webui_package="@govoplan/dashboard-webui",
), ),
CatalogModule(
module_id="addresses",
repo="govoplan-addresses",
python_package="govoplan-addresses",
name="Addresses",
description="Reusable address directories, recipient sources, consent metadata, and address quality workflows.",
tags=("official", "business-module"),
webui_package="@govoplan/addresses-webui",
),
CatalogModule( CatalogModule(
module_id="files", module_id="files",
repo="govoplan-files", repo="govoplan-files",
@@ -187,6 +198,19 @@ def main() -> int:
args = parser.parse_args() args = parser.parse_args()
version = args.version.removeprefix("v") version = args.version.removeprefix("v")
version_issues = selected_repository_version_issues(
repo_versions={
"govoplan-core": version,
**{module.repo: version for module in CATALOG_MODULES},
},
workspace=CORE_ROOT.parent,
)
if version_issues:
details = "; ".join(
f"{issue.repo}: {issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
for issue in version_issues
)
parser.error(f"version alignment gate failed: {details}")
tag = f"v{version}" tag = f"v{version}"
generated_at = datetime.now(tz=UTC) generated_at = datetime.now(tz=UTC)
sequence = args.sequence if args.sequence is not None else int(generated_at.strftime("%Y%m%d%H%M")) sequence = args.sequence if args.sequence is not None else int(generated_at.strftime("%Y%m%d%H%M"))

View File

@@ -17,6 +17,10 @@ the regenerated release lockfile.
Options: Options:
--npm <path> npm executable to use. --npm <path> npm executable to use.
--core-root <path> govoplan-core checkout. Defaults to ../govoplan-core. --core-root <path> govoplan-core checkout. Defaults to ../govoplan-core.
--local-git-repo <path>
Resolve that repository's release tag from its local Git
object store while preserving the published Git URL in
the lockfile. May be repeated.
-h, --help Show this help. -h, --help Show this help.
USAGE USAGE
} }
@@ -32,6 +36,7 @@ CORE_ROOT="$(cd "$CORE_ROOT" && pwd)"
WEBUI="$CORE_ROOT/webui" WEBUI="$CORE_ROOT/webui"
NPM_BIN="${NPM:-}" NPM_BIN="${NPM:-}"
NODE_BIN="${NODE:-}" NODE_BIN="${NODE:-}"
LOCAL_GIT_REPOS=()
if [[ -z "$NPM_BIN" ]]; then if [[ -z "$NPM_BIN" ]]; then
if [[ -x "/home/zemion/.nvm/versions/node/v22.22.3/bin/npm" ]]; then if [[ -x "/home/zemion/.nvm/versions/node/v22.22.3/bin/npm" ]]; then
@@ -61,6 +66,11 @@ while [[ $# -gt 0 ]]; do
WEBUI="$CORE_ROOT/webui" WEBUI="$CORE_ROOT/webui"
shift 2 shift 2
;; ;;
--local-git-repo)
[[ $# -ge 2 ]] || fail "missing value for $1"
LOCAL_GIT_REPOS+=("$(cd "$2" && pwd)")
shift 2
;;
-h|--help) -h|--help)
usage usage
exit 0 exit 0
@@ -90,9 +100,27 @@ fi
echo "Generating release lockfile from $WEBUI/package.release.json" echo "Generating release lockfile from $WEBUI/package.release.json"
echo "Temporary workspace: $TMP_DIR" echo "Temporary workspace: $TMP_DIR"
GIT_ENV=(env)
git_config_count=0
for local_repo in "${LOCAL_GIT_REPOS[@]}"; do
[[ -d "$local_repo/.git" ]] || fail "local Git release source is not a repository: $local_repo"
remote_url="$(git -C "$local_repo" remote get-url origin)"
remote_url="${remote_url#git+}"
if [[ "$remote_url" != *://* && "$remote_url" =~ ^([^@]+@)?([^:]+):(.+)$ ]]; then
remote_url="ssh://${BASH_REMATCH[1]}${BASH_REMATCH[2]}/${BASH_REMATCH[3]}"
fi
GIT_ENV+=(
"GIT_CONFIG_KEY_${git_config_count}=url.file://$local_repo/.insteadOf"
"GIT_CONFIG_VALUE_${git_config_count}=$remote_url"
)
git_config_count=$((git_config_count + 1))
done
GIT_ENV+=("GIT_CONFIG_COUNT=$git_config_count")
( (
cd "$TMP_DIR" cd "$TMP_DIR"
PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" install --package-lock-only --ignore-scripts "${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" install --package-lock-only --ignore-scripts
mapfile -t GIT_PACKAGES < <( mapfile -t GIT_PACKAGES < <(
PATH="$(dirname "$NODE_BIN"):$PATH" "$NODE_BIN" <<'NODE' PATH="$(dirname "$NODE_BIN"):$PATH" "$NODE_BIN" <<'NODE'
const fs = require("fs"); const fs = require("fs");
@@ -108,7 +136,7 @@ NODE
) )
if [[ "${#GIT_PACKAGES[@]}" -gt 0 ]]; then if [[ "${#GIT_PACKAGES[@]}" -gt 0 ]]; then
echo "Refreshing git package lock entries: ${GIT_PACKAGES[*]}" echo "Refreshing git package lock entries: ${GIT_PACKAGES[*]}"
PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}" "${GIT_ENV[@]}" PATH="$(dirname "$NPM_BIN"):$PATH" "$NPM_BIN" update --package-lock-only --ignore-scripts "${GIT_PACKAGES[@]}"
fi fi
) )

View File

@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""Generate a CycloneDX SBOM from the resolved GovOPlaN release environment."""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import subprocess
import sys
import tomllib
META_ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(META_ROOT / "tools" / "release"))
from govoplan_release.sbom import ( # noqa: E402
build_release_sbom,
parse_sbom_timestamp,
render_sbom,
sbom_sha256,
timestamp_from_source_date_epoch,
validate_sbom,
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--python", default=sys.executable, help="Python executable for pip inspect.")
parser.add_argument("--core-root", type=Path, default=META_ROOT.parent / "govoplan-core")
parser.add_argument(
"--output",
type=Path,
default=META_ROOT / "runtime" / "release-artifacts" / "govoplan-sbom.cdx.json",
)
parser.add_argument("--pip-inspect", type=Path, help="Use an existing pip inspect report instead of executing pip.")
parser.add_argument(
"--timestamp",
help="ISO-8601 build timestamp. Defaults to SOURCE_DATE_EPOCH, then the current time.",
)
args = parser.parse_args()
core_root = args.core_root.expanduser().resolve()
product_version = _project_version(core_root / "pyproject.toml")
npm_lock_path = core_root / "webui" / "package-lock.release.json"
if not npm_lock_path.exists():
parser.error(f"release npm lockfile does not exist: {npm_lock_path}")
pip_inspect = _read_json(args.pip_inspect) if args.pip_inspect else _run_pip_inspect(args.python)
npm_lock = _read_json(npm_lock_path)
try:
timestamp = _resolved_timestamp(args.timestamp)
payload = build_release_sbom(
product_version=product_version,
pip_inspect=pip_inspect,
npm_lock=npm_lock,
timestamp=timestamp,
)
except ValueError as exc:
parser.error(str(exc))
validate_sbom(payload)
output = args.output.expanduser().resolve()
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(render_sbom(payload), encoding="utf-8")
print(f"Wrote {len(payload['components'])} components to {output}")
print(f"SHA-256: {sbom_sha256(payload)}")
return 0
def _run_pip_inspect(python: str) -> dict:
result = subprocess.run(
[python, "-m", "pip", "inspect", "--local"],
check=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
timeout=120,
)
if result.returncode != 0:
raise SystemExit(f"pip inspect failed: {result.stderr.strip()}")
return json.loads(result.stdout)
def _read_json(path: Path) -> dict:
payload = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(payload, dict):
raise ValueError(f"expected a JSON object in {path}")
return payload
def _project_version(path: Path) -> str:
with path.open("rb") as handle:
payload = tomllib.load(handle)
project = payload.get("project")
if not isinstance(project, dict) or not isinstance(project.get("version"), str):
raise ValueError(f"project.version is missing from {path}")
return project["version"]
def _resolved_timestamp(explicit: str | None):
if explicit is not None:
return parse_sbom_timestamp(explicit)
source_date_epoch = os.environ.get("SOURCE_DATE_EPOCH")
if source_date_epoch is not None:
return timestamp_from_source_date_epoch(source_date_epoch)
return None
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,72 @@
"""Shared release tooling for the local GovOPlaN release console."""
from __future__ import annotations
from .model import ReleaseDashboard, ReleasePlan, RepositorySnapshot, RepositorySpec
__all__ = [
"ReleaseDashboard",
"ReleasePlan",
"RepositorySnapshot",
"RepositorySpec",
"build_dashboard",
"build_release_intelligence",
"build_release_plan",
"build_selective_catalog_candidate",
"build_selective_release_plan",
"publish_catalog_candidate",
"prepare_repositories",
"push_repositories",
"sync_repositories",
"tag_repositories",
]
def __getattr__(name: str):
value = _load_lazy_export(name)
globals()[name] = value
return value
def _load_lazy_export(name: str):
if name == "build_dashboard":
from .dashboard import build_dashboard
return build_dashboard
if name == "build_release_intelligence":
from .release_intelligence import build_release_intelligence
return build_release_intelligence
if name == "build_release_plan":
from .planner import build_release_plan
return build_release_plan
if name == "build_selective_catalog_candidate":
from .selective_catalog import build_selective_catalog_candidate
return build_selective_catalog_candidate
if name == "build_selective_release_plan":
from .selective_planner import build_selective_release_plan
return build_selective_release_plan
if name == "publish_catalog_candidate":
from .publisher import publish_catalog_candidate
return publish_catalog_candidate
if name == "prepare_repositories":
from .repository_prepare import prepare_repositories
return prepare_repositories
if name == "push_repositories":
from .repository_push import push_repositories
return push_repositories
if name == "sync_repositories":
from .repository_sync import sync_repositories
return sync_repositories
if name == "tag_repositories":
from .repository_tag import tag_repositories
return tag_repositories
raise AttributeError(name)

View File

@@ -0,0 +1,397 @@
"""Independent identities for immutable Python wheel release artifacts."""
from __future__ import annotations
import base64
from dataclasses import dataclass
from email.parser import BytesParser
from email.policy import compat32
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import stat
import zipfile
WHEEL_PAYLOAD_ALGORITHM = "govoplan-wheel-declared-payload-v1"
INSTALLED_PAYLOAD_ALGORITHM = "govoplan-installed-record-payload-v1"
MAX_WHEEL_BYTES = 512 * 1024 * 1024
MAX_WHEEL_ENTRIES = 10_000
MAX_WHEEL_UNCOMPRESSED_BYTES = 1024 * 1024 * 1024
MAX_WHEEL_ENTRY_BYTES = 64 * 1024 * 1024
MAX_METADATA_BYTES = 1024 * 1024
_PACKAGE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
_VERSION = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._+!-]{0,127}$")
_PYTHON_REF = re.compile(
r"/(?P<repo>govoplan-[a-z0-9-]+)[.]git@v(?P<version>[^\s;]+)$"
)
class ArtifactIdentityError(ValueError):
"""A built artifact cannot provide a bounded immutable identity."""
@dataclass(frozen=True, slots=True)
class PythonWheelIdentity:
package_name: str
package_version: str
archive_sha256: str
archive_size: int
payload_sha256: str
payload_file_count: int
requires_installer_receipt: bool
def catalog_payload(self) -> dict[str, object]:
return {
"artifact_kind": "python-wheel",
"package_name": self.package_name,
"package_version": self.package_version,
"archive_sha256": self.archive_sha256,
"archive_size": self.archive_size,
"installed_payload": {
"algorithm": WHEEL_PAYLOAD_ALGORITHM,
"sha256": self.payload_sha256,
"file_count": self.payload_file_count,
},
"requires_installer_receipt": self.requires_installer_receipt,
}
def inspect_python_wheel(path: Path | str) -> PythonWheelIdentity:
"""Hash a regular built wheel and derive its install-stable payload identity."""
wheel = Path(path).expanduser()
flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)
try:
descriptor = os.open(wheel, flags)
except OSError as exc:
raise ArtifactIdentityError("built artifact cannot be opened safely") from exc
try:
archive_sha256, archive_size, initial = _hash_open_artifact(descriptor)
os.lseek(descriptor, 0, os.SEEK_SET)
try:
with os.fdopen(os.dup(descriptor), "rb") as artifact_handle:
with zipfile.ZipFile(artifact_handle) as archive:
(
package_name,
package_version,
payload_rows,
requires_receipt,
) = _inspect_wheel_archive(archive)
except (OSError, zipfile.BadZipFile, RuntimeError) as exc:
raise ArtifactIdentityError(
"built Python artifact is not a readable wheel"
) from exc
final = os.fstat(descriptor)
if _stat_fingerprint(final) != _stat_fingerprint(initial):
raise ArtifactIdentityError("built artifact changed while it was inspected")
finally:
os.close(descriptor)
return PythonWheelIdentity(
package_name=package_name,
package_version=package_version,
archive_sha256=archive_sha256,
archive_size=archive_size,
payload_sha256=payload_identity_sha256(
algorithm=WHEEL_PAYLOAD_ALGORITHM, rows=payload_rows
),
payload_file_count=len(payload_rows),
requires_installer_receipt=requires_receipt,
)
def _inspect_wheel_archive(
archive: zipfile.ZipFile,
) -> tuple[str, str, list[dict[str, object]], bool]:
infos = archive.infolist()
if not infos or len(infos) > MAX_WHEEL_ENTRIES:
raise ArtifactIdentityError("wheel entry count is empty or exceeds its limit")
_validate_members(infos)
metadata_members = [
item
for item in infos
if not item.is_dir()
and PurePosixPath(item.filename).name == "METADATA"
and PurePosixPath(item.filename).parent.name.endswith(".dist-info")
]
if len(metadata_members) != 1:
raise ArtifactIdentityError(
"wheel must contain exactly one dist-info METADATA file"
)
metadata_member = metadata_members[0]
dist_info = PurePosixPath(metadata_member.filename).parent
metadata_bytes = _read_member(
archive, metadata_member, max_bytes=MAX_METADATA_BYTES
)
parsed = BytesParser(policy=compat32).parsebytes(metadata_bytes)
package_name = normalize_package_name(str(parsed.get("Name") or ""))
package_version = str(parsed.get("Version") or "").strip()
if _PACKAGE.fullmatch(package_name) is None:
raise ArtifactIdentityError("wheel METADATA has an invalid package name")
if _VERSION.fullmatch(package_version) is None:
raise ArtifactIdentityError("wheel METADATA has an invalid package version")
payload_rows: list[dict[str, object]] = []
requires_receipt = False
for info in infos:
if info.is_dir():
continue
normalized, transformed = _wheel_payload_path(
PurePosixPath(info.filename), dist_info=dist_info
)
requires_receipt = requires_receipt or transformed
if normalized is None:
continue
encoded = _read_member(archive, info, max_bytes=MAX_WHEEL_ENTRY_BYTES)
payload_rows.append(
{
"path": normalized,
"sha256": hashlib.sha256(encoded).hexdigest(),
"size": len(encoded),
}
)
payload_rows.sort(key=lambda item: str(item["path"]))
if not payload_rows:
raise ArtifactIdentityError("wheel has no immutable payload entries")
if len({str(item["path"]) for item in payload_rows}) != len(payload_rows):
raise ArtifactIdentityError(
"wheel payload paths are ambiguous after installation mapping"
)
entry_points = f"{dist_info.as_posix()}/entry_points.txt"
if any(item.filename == entry_points for item in infos):
entry_point_info = next(item for item in infos if item.filename == entry_points)
entry_point_text = _read_member(
archive, entry_point_info, max_bytes=MAX_METADATA_BYTES
).decode("utf-8", errors="replace")
if re.search(r"(?m)^\s*\[(?:console|gui)_scripts\]\s*$", entry_point_text):
requires_receipt = True
return package_name, package_version, payload_rows, requires_receipt
def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
"""Report selected Python releases without a matching built-wheel identity."""
if not isinstance(payload, dict):
return ("candidate catalog must be a JSON object",)
release = payload.get("release")
selected_units = release.get("selected_units") if isinstance(release, dict) else None
artifacts = release.get("artifacts") if isinstance(release, dict) else None
if not isinstance(selected_units, list) or not selected_units:
return ("release.selected_units is missing or empty",)
entries: list[object] = [payload.get("core_release")]
modules = payload.get("modules")
if isinstance(modules, list):
entries.extend(modules)
package_by_repo: dict[str, tuple[str, str]] = {}
for entry in entries:
if not isinstance(entry, dict):
continue
python_ref = entry.get("python_ref")
match = _PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
package_name = entry.get("python_package")
version = entry.get("version")
if (
match is not None
and isinstance(package_name, str)
and _PACKAGE.fullmatch(package_name) is not None
and isinstance(version, str)
):
package_by_repo[match.group("repo")] = (package_name, version)
artifacts_by_package: dict[str, dict[str, object]] = {}
malformed_artifacts = False
if not isinstance(artifacts, list):
artifacts = []
for artifact in artifacts:
if not isinstance(artifact, dict):
malformed_artifacts = True
continue
package_name = artifact.get("package_name")
if not isinstance(package_name, str) or package_name in artifacts_by_package:
malformed_artifacts = True
continue
if not _catalog_artifact_shape_valid(artifact):
malformed_artifacts = True
continue
artifacts_by_package[package_name] = artifact
issues: list[str] = []
if malformed_artifacts:
issues.append("release.artifacts contains malformed or duplicate identities")
seen_repos: set[str] = set()
for unit in selected_units:
if not isinstance(unit, dict):
issues.append("release.selected_units contains a malformed entry")
continue
repo = unit.get("repo")
if not isinstance(repo, str) or repo in seen_repos:
issues.append("release.selected_units contains a duplicate or invalid repository")
continue
seen_repos.add(repo)
expected = package_by_repo.get(repo)
if expected is None:
continue # Selected non-Python repositories have no wheel identity.
package_name, version = expected
artifact = artifacts_by_package.get(package_name)
if artifact is None:
issues.append(
f"selected Python repository {repo} has no built artifact identity for {package_name} {version}"
)
elif artifact.get("package_version") != version:
issues.append(
f"selected Python repository {repo} artifact identity has version {artifact.get('package_version')!r}, expected {version!r}"
)
return tuple(issues)
def _catalog_artifact_shape_valid(value: dict[str, object]) -> bool:
if set(value) != {
"artifact_kind",
"package_name",
"package_version",
"archive_sha256",
"archive_size",
"installed_payload",
"requires_installer_receipt",
}:
return False
package_name = value.get("package_name")
version = value.get("package_version")
archive_sha256 = value.get("archive_sha256")
archive_size = value.get("archive_size")
installed_payload = value.get("installed_payload")
return (
value.get("artifact_kind") == "python-wheel"
and isinstance(package_name, str)
and _PACKAGE.fullmatch(package_name) is not None
and isinstance(version, str)
and _VERSION.fullmatch(version) is not None
and isinstance(archive_sha256, str)
and re.fullmatch(r"[0-9a-f]{64}", archive_sha256) is not None
and isinstance(archive_size, int)
and not isinstance(archive_size, bool)
and 0 < archive_size <= MAX_WHEEL_BYTES
and isinstance(value.get("requires_installer_receipt"), bool)
and isinstance(installed_payload, dict)
and set(installed_payload) == {"algorithm", "sha256", "file_count"}
and installed_payload.get("algorithm") == WHEEL_PAYLOAD_ALGORITHM
and isinstance(installed_payload.get("sha256"), str)
and re.fullmatch(r"[0-9a-f]{64}", str(installed_payload["sha256"]))
is not None
and isinstance(installed_payload.get("file_count"), int)
and not isinstance(installed_payload.get("file_count"), bool)
and 0 < int(installed_payload["file_count"]) <= MAX_WHEEL_ENTRIES
)
def payload_identity_sha256(
*, algorithm: str, rows: list[dict[str, object]]
) -> str:
payload = {"algorithm": algorithm, "files": rows}
encoded = json.dumps(
payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def record_hash_hex(value: str) -> str | None:
try:
decoded = base64.urlsafe_b64decode(value + "=" * (-len(value) % 4))
except (ValueError, TypeError):
return None
return decoded.hex() if len(decoded) == hashlib.sha256().digest_size else None
def normalize_package_name(value: str) -> str:
return re.sub(r"[-_.]+", "-", value.strip().lower())
def _hash_open_artifact(descriptor: int) -> tuple[str, int, os.stat_result]:
digest = hashlib.sha256()
total = 0
initial = os.fstat(descriptor)
if not stat.S_ISREG(initial.st_mode) or initial.st_size > MAX_WHEEL_BYTES:
raise ArtifactIdentityError("built artifact must be a bounded regular file")
while total < initial.st_size:
chunk = os.read(descriptor, min(1024 * 1024, initial.st_size - total))
if not chunk:
break
total += len(chunk)
digest.update(chunk)
final = os.fstat(descriptor)
if total != initial.st_size or _stat_fingerprint(final) != _stat_fingerprint(initial):
raise ArtifactIdentityError("built artifact changed while it was hashed")
return digest.hexdigest(), total, initial
def _stat_fingerprint(value: os.stat_result) -> tuple[int, int, int, int, int]:
return (
value.st_dev,
value.st_ino,
value.st_size,
value.st_mtime_ns,
value.st_ctime_ns,
)
def _validate_members(infos: list[zipfile.ZipInfo]) -> None:
names: set[str] = set()
total = 0
for info in infos:
path = PurePosixPath(info.filename.replace("\\", "/"))
if (
not info.filename
or path.is_absolute()
or ".." in path.parts
or any(ord(character) < 32 for character in info.filename)
or info.flag_bits & 0x1
):
raise ArtifactIdentityError("wheel contains an unsafe member")
normalized = path.as_posix()
if normalized in names:
raise ArtifactIdentityError("wheel contains duplicate members")
names.add(normalized)
if _zip_member_is_symlink(info):
raise ArtifactIdentityError("wheel contains a symbolic-link member")
total += info.file_size
if info.file_size > MAX_WHEEL_ENTRY_BYTES or total > MAX_WHEEL_UNCOMPRESSED_BYTES:
raise ArtifactIdentityError("wheel uncompressed payload exceeds its limit")
def _zip_member_is_symlink(info: zipfile.ZipInfo) -> bool:
return stat.S_ISLNK((info.external_attr >> 16) & 0o177777)
def _read_member(
archive: zipfile.ZipFile, info: zipfile.ZipInfo, *, max_bytes: int
) -> bytes:
if info.file_size > max_bytes:
raise ArtifactIdentityError("wheel member exceeds its size limit")
with archive.open(info, "r") as handle:
encoded = handle.read(max_bytes + 1)
if len(encoded) > max_bytes or len(encoded) != info.file_size:
raise ArtifactIdentityError("wheel member size is inconsistent")
return encoded
def _wheel_payload_path(
path: PurePosixPath, *, dist_info: PurePosixPath
) -> tuple[str | None, bool]:
if path.parent == dist_info and path.name in {"RECORD", "RECORD.jws", "RECORD.p7s"}:
return None, False
parts = path.parts
data_prefix = dist_info.name.removesuffix(".dist-info") + ".data"
if parts and parts[0] == data_prefix:
if len(parts) < 3:
raise ArtifactIdentityError("wheel data member has no installation target")
scheme = parts[1]
remainder = PurePosixPath(*parts[2:]).as_posix()
if scheme in {"purelib", "platlib"}:
return remainder, False
if scheme == "data":
return f"@data/{remainder}", False
if scheme in {"scripts", "headers"}:
return None, True
raise ArtifactIdentityError("wheel uses an unsupported installation scheme")
return path.as_posix(), False

View File

@@ -0,0 +1,254 @@
"""Opaque, workspace-scoped handles for generated catalog candidates."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import hmac
import json
import os
from pathlib import Path
import re
import stat
from typing import Iterable
from .catalog import canonical_hash
MAX_CATALOG_BYTES = 16 * 1024 * 1024
_CANDIDATE_ID = re.compile(r"^candidate-[0-9a-f]{32}$")
_CHANNEL = re.compile(r"^[a-z][a-z0-9_-]{0,63}$")
class CandidateArtifactError(ValueError):
"""A candidate handle or the artifact behind it is unsafe or inconsistent."""
@dataclass(frozen=True, slots=True)
class CandidateArtifactReceipt:
candidate_id: str
catalog_sha256: str
def issue_candidate_id(*seed_parts: str) -> str:
"""Derive an opaque, deterministic candidate basename before output exists."""
if not seed_parts or any(
not isinstance(part, str) or not part or len(part) > 512
for part in seed_parts
):
raise CandidateArtifactError("candidate ID seeds must be non-empty bounded strings")
encoded = json.dumps(
list(seed_parts), sort_keys=False, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return f"candidate-{hashlib.sha256(encoded).hexdigest()[:32]}"
def validate_release_channel(value: str) -> str:
"""Return one bounded channel basename or reject path-capable input."""
if not isinstance(value, str) or _CHANNEL.fullmatch(value) is None:
raise CandidateArtifactError(
"release channel must be a bounded lowercase identifier"
)
return value
def candidate_output_path(root: Path | str, candidate_id: str) -> Path:
"""Resolve a not-yet-created candidate path below a trusted configured root."""
checked_id = _checked_candidate_id(candidate_id)
root_path = Path(root).expanduser()
ensure_private_candidate_root(root_path, create=True)
candidate = root_path / checked_id
try:
mode = candidate.lstat().st_mode
except FileNotFoundError:
return candidate
except OSError as exc:
raise CandidateArtifactError("candidate output path cannot be inspected") from exc
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise CandidateArtifactError("candidate output path must be a real directory")
_require_private_owner_mode(candidate, directory=True, label="candidate output path")
return candidate
def ensure_private_candidate_root(
root: Path | str, *, create: bool = False
) -> Path:
"""Admit only an operator-owned candidate root inaccessible to other users."""
root_path = Path(root).expanduser()
_reject_symlink_components(root_path, allow_missing=create)
if create:
try:
root_path.mkdir(mode=0o700, parents=True, exist_ok=True)
except OSError as exc:
raise CandidateArtifactError("candidate root cannot be created") from exc
_reject_symlink_components(root_path, allow_missing=False)
_require_private_owner_mode(root_path, directory=True, label="candidate root")
return root_path
def harden_private_candidate_tree(path: Path | str) -> Path:
"""Seal a newly generated, operator-owned candidate tree before a receipt."""
root = Path(path).expanduser()
_reject_symlink_components(root, allow_missing=False)
_require_current_owner(root, label="candidate directory")
try:
os.chmod(root, 0o700, follow_symlinks=False)
for current, directory_names, file_names in os.walk(root, followlinks=False):
current_path = Path(current)
_require_current_owner(current_path, label="candidate directory")
os.chmod(current_path, 0o700, follow_symlinks=False)
for name in (*directory_names, *file_names):
child = current_path / name
mode = child.lstat().st_mode
if stat.S_ISLNK(mode):
raise CandidateArtifactError("candidate tree contains a symlink")
_require_current_owner(child, label="candidate tree member")
if stat.S_ISDIR(mode):
os.chmod(child, 0o700, follow_symlinks=False)
elif stat.S_ISREG(mode):
os.chmod(child, 0o600, follow_symlinks=False)
else:
raise CandidateArtifactError(
"candidate tree contains a non-file member"
)
except OSError as exc:
raise CandidateArtifactError("candidate tree cannot be sealed") from exc
return root
def issue_candidate_receipt(
*, root: Path | str, candidate_id: str, channel: str
) -> CandidateArtifactReceipt:
"""Hash the signed catalog at a generated candidate handle."""
candidate = _existing_candidate_path(root, candidate_id)
payload = _read_signed_catalog(candidate, channel=channel)
return CandidateArtifactReceipt(
candidate_id=candidate_id,
catalog_sha256=canonical_hash(payload),
)
def verify_candidate_receipt(
*,
root: Path | str,
candidate_id: str,
catalog_sha256: str,
channel: str,
) -> Path:
"""Re-resolve and re-hash a persisted receipt before candidate consumption."""
if re.fullmatch(r"[0-9a-f]{64}", catalog_sha256) is None:
raise CandidateArtifactError("candidate catalog digest is malformed")
candidate = _existing_candidate_path(root, candidate_id)
payload = _read_signed_catalog(candidate, channel=channel)
if not hmac.compare_digest(canonical_hash(payload), catalog_sha256):
raise CandidateArtifactError("candidate catalog no longer matches its receipt")
return candidate
def _checked_candidate_id(candidate_id: str) -> str:
if not isinstance(candidate_id, str) or _CANDIDATE_ID.fullmatch(candidate_id) is None:
raise CandidateArtifactError("candidate ID is not an issued opaque basename")
return candidate_id
def _existing_candidate_path(root: Path | str, candidate_id: str) -> Path:
candidate = candidate_output_path(root, candidate_id)
try:
mode = candidate.lstat().st_mode
except OSError as exc:
raise CandidateArtifactError("candidate directory is unavailable") from exc
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise CandidateArtifactError("candidate directory must be a real directory")
_reject_symlink_components(candidate, allow_missing=False)
return candidate
def _read_signed_catalog(candidate: Path, *, channel: str) -> dict[str, object]:
channel = validate_release_channel(channel)
channels = candidate / "channels"
catalog_path = channels / f"{channel}.json"
_require_real_directory(channels, label="candidate channels directory")
_require_regular_file(catalog_path, label="candidate catalog")
try:
with catalog_path.open("rb") as handle:
encoded = handle.read(MAX_CATALOG_BYTES + 1)
except OSError as exc:
raise CandidateArtifactError("candidate catalog cannot be read") from exc
if len(encoded) > MAX_CATALOG_BYTES:
raise CandidateArtifactError("candidate catalog exceeds its size limit")
try:
payload = json.loads(encoded.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise CandidateArtifactError("candidate catalog is not valid JSON") from exc
if not isinstance(payload, dict):
raise CandidateArtifactError("candidate catalog must be a JSON object")
if payload.get("channel") != channel:
raise CandidateArtifactError(
"candidate catalog channel does not match its requested handle"
)
signatures = payload.get("signatures")
if not isinstance(signatures, list) or not signatures:
raise CandidateArtifactError("candidate catalog has no signature envelope")
return payload
def _reject_symlink_components(path: Path, *, allow_missing: bool) -> None:
absolute = path.absolute()
parts: Iterable[Path] = reversed((absolute, *absolute.parents))
for component in parts:
try:
mode = component.lstat().st_mode
except FileNotFoundError:
if allow_missing:
continue
raise CandidateArtifactError("candidate path has a missing component")
except OSError as exc:
raise CandidateArtifactError("candidate path cannot be inspected") from exc
if stat.S_ISLNK(mode):
raise CandidateArtifactError("candidate path must not traverse symlinks")
def _require_real_directory(path: Path, *, label: str) -> None:
try:
mode = path.lstat().st_mode
except OSError as exc:
raise CandidateArtifactError(f"{label} is unavailable") from exc
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise CandidateArtifactError(f"{label} must be a real directory")
_require_private_owner_mode(path, directory=True, label=label)
def _require_regular_file(path: Path, *, label: str) -> None:
try:
mode = path.lstat().st_mode
except OSError as exc:
raise CandidateArtifactError(f"{label} is unavailable") from exc
if stat.S_ISLNK(mode) or not stat.S_ISREG(mode):
raise CandidateArtifactError(f"{label} must be a regular file")
_require_private_owner_mode(path, directory=False, label=label)
def _require_current_owner(path: Path, *, label: str) -> os.stat_result:
try:
observed = path.lstat()
except OSError as exc:
raise CandidateArtifactError(f"{label} cannot be inspected") from exc
if observed.st_uid != os.geteuid():
raise CandidateArtifactError(f"{label} is not owned by the current operator")
return observed
def _require_private_owner_mode(
path: Path, *, directory: bool, label: str
) -> None:
observed = _require_current_owner(path, label=label)
expected_type = stat.S_ISDIR if directory else stat.S_ISREG
if not expected_type(observed.st_mode) or stat.S_IMODE(observed.st_mode) & 0o077:
raise CandidateArtifactError(f"{label} is accessible to another user")

View File

@@ -0,0 +1,199 @@
"""Release catalog inspection."""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
from urllib.error import HTTPError, URLError
from .http_fetch import DEFAULT_MAX_RESPONSE_BYTES, fetch_http
from .model import CatalogSnapshot
from .workspace import website_root
DEFAULT_PUBLIC_BASE_URL = "https://govoplan.add-ideas.de"
MAX_PUBLIC_JSON_RESPONSE_BYTES = DEFAULT_MAX_RESPONSE_BYTES
def collect_catalog_snapshot(
*,
workspace_root: Path,
channel: str = "stable",
public_base_url: str = DEFAULT_PUBLIC_BASE_URL,
check_public: bool = False,
) -> CatalogSnapshot:
web_root = website_root(workspace_root)
catalog_path = (
web_root / "public" / "catalogs" / "v1" / "channels" / f"{channel}.json"
)
keyring_path = web_root / "public" / "catalogs" / "v1" / "keyring.json"
public_catalog_url = (
f"{public_base_url.rstrip('/')}/catalogs/v1/channels/{channel}.json"
)
public_keyring_url = f"{public_base_url.rstrip('/')}/catalogs/v1/keyring.json"
catalog_version: str | None = None
core_release_version: str | None = None
module_count = 0
signature_count = 0
generated_at: str | None = None
expires_at: str | None = None
catalog_error: str | None = None
local_catalog_hash: str | None = None
if catalog_path.exists():
try:
payload = json.loads(catalog_path.read_text(encoding="utf-8"))
local_catalog_hash = canonical_hash(payload)
parsed = parse_catalog_payload(payload)
catalog_version = parsed["catalog_version"]
core_release_version = parsed["core_release_version"]
module_count = parsed["module_count"] or 0
signature_count = parsed["signature_count"] or 0
generated_at = parsed["generated_at"]
expires_at = parsed["expires_at"]
except Exception as exc: # noqa: BLE001 - shown in local operator UI.
catalog_error = str(exc)
key_count = 0
keyring_error: str | None = None
local_keyring_hash: str | None = None
if keyring_path.exists():
try:
payload = json.loads(keyring_path.read_text(encoding="utf-8"))
local_keyring_hash = canonical_hash(payload)
keys = payload.get("keys") if isinstance(payload, dict) else None
key_count = len(keys) if isinstance(keys, list) else 0
except Exception as exc: # noqa: BLE001 - shown in local operator UI.
keyring_error = str(exc)
public_catalog_exists: bool | None = None
public_keyring_exists: bool | None = None
public_core_release_version: str | None = None
public_module_count: int | None = None
public_signature_count: int | None = None
public_key_count: int | None = None
public_catalog_hash: str | None = None
public_keyring_hash: str | None = None
public_catalog_error: str | None = None
public_keyring_error: str | None = None
if check_public:
public_catalog = fetch_json(public_catalog_url)
if public_catalog["ok"]:
public_catalog_exists = True
payload = public_catalog["payload"]
public_catalog_hash = canonical_hash(payload)
parsed = parse_catalog_payload(payload)
public_core_release_version = parsed["core_release_version"]
public_module_count = parsed["module_count"]
public_signature_count = parsed["signature_count"]
else:
public_catalog_exists = False
public_catalog_error = str(public_catalog["error"])
public_keyring = fetch_json(public_keyring_url)
if public_keyring["ok"]:
public_keyring_exists = True
payload = public_keyring["payload"]
public_keyring_hash = canonical_hash(payload)
keys = payload.get("keys") if isinstance(payload, dict) else None
public_key_count = len(keys) if isinstance(keys, list) else 0
else:
public_keyring_exists = False
public_keyring_error = str(public_keyring["error"])
return CatalogSnapshot(
channel=channel,
website_path=str(web_root),
catalog_path=str(catalog_path),
catalog_exists=catalog_path.exists(),
keyring_path=str(keyring_path),
keyring_exists=keyring_path.exists(),
catalog_version=catalog_version,
core_release_version=core_release_version,
module_count=module_count,
signature_count=signature_count,
key_count=key_count,
generated_at=generated_at,
expires_at=expires_at,
catalog_error=catalog_error,
keyring_error=keyring_error,
local_catalog_hash=local_catalog_hash,
local_keyring_hash=local_keyring_hash,
public_catalog_url=public_catalog_url,
public_keyring_url=public_keyring_url,
public_checked=check_public,
public_catalog_exists=public_catalog_exists,
public_keyring_exists=public_keyring_exists,
public_core_release_version=public_core_release_version,
public_module_count=public_module_count,
public_signature_count=public_signature_count,
public_key_count=public_key_count,
public_catalog_hash=public_catalog_hash,
public_keyring_hash=public_keyring_hash,
public_catalog_error=public_catalog_error,
public_keyring_error=public_keyring_error,
catalog_matches_public=compare_hashes(local_catalog_hash, public_catalog_hash)
if check_public
else None,
keyring_matches_public=compare_hashes(local_keyring_hash, public_keyring_hash)
if check_public
else None,
)
def parse_catalog_payload(payload: object) -> dict[str, str | int | None]:
if not isinstance(payload, dict):
return {
"catalog_version": None,
"core_release_version": None,
"module_count": 0,
"signature_count": 0,
"generated_at": None,
"expires_at": None,
}
core_release = payload.get("core_release")
modules = payload.get("modules")
signatures = payload.get("signatures")
return {
"catalog_version": string_or_none(payload.get("catalog_version")),
"core_release_version": string_or_none(core_release.get("version"))
if isinstance(core_release, dict)
else None,
"module_count": len(modules) if isinstance(modules, list) else 0,
"signature_count": len(signatures) if isinstance(signatures, list) else 0,
"generated_at": string_or_none(payload.get("generated_at")),
"expires_at": string_or_none(payload.get("expires_at")),
}
def fetch_json(url: str) -> dict[str, object]:
try:
response = fetch_http(
url,
timeout=10,
label="Public release catalog URL",
headers={"User-Agent": "GovOPlaN release-console"},
max_response_bytes=MAX_PUBLIC_JSON_RESPONSE_BYTES,
)
return {"ok": True, "payload": json.loads(response.text())}
except (HTTPError, URLError, TimeoutError, ValueError, json.JSONDecodeError) as exc:
return {"ok": False, "error": str(exc)}
def canonical_hash(payload: object) -> str:
canonical = json.dumps(
payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return hashlib.sha256(canonical).hexdigest()
def compare_hashes(local_hash: str | None, public_hash: str | None) -> bool | None:
if local_hash is None or public_hash is None:
return None
return local_hash == public_hash
def string_or_none(value: object) -> str | None:
return value if isinstance(value, str) else None

View File

@@ -0,0 +1,378 @@
"""Synthesize initial catalog entries from a repository's package and manifest."""
from __future__ import annotations
import argparse
from contextlib import contextmanager
import importlib
import json
import os
from pathlib import Path
import subprocess
import sys
import tomllib
from types import ModuleType
from typing import Iterator
from govoplan_core.core.modules import ModuleManifest
from govoplan_core.core.versioning import version_satisfies_range
from .workspace import load_repository_specs, resolve_repo_path
_INSPECTION_CHILD = "GOVOPLAN_CATALOG_ENTRY_INSPECTION_CHILD"
_INSPECTION_MARKER = "GOVOPLAN_CATALOG_ENTRY_JSON="
def synthesize_repository_catalog_entries(
*,
repo: str,
version: str,
workspace: Path,
repository_base: str,
) -> tuple[dict[str, object], ...]:
"""Build install entries from tagged, preflighted local source metadata.
The caller owns source-tag and worktree provenance checks. This function
accepts no hand-maintained module catalog registry: distribution metadata
identifies the runtime entry point and the runtime ``ModuleManifest`` is
the authoritative dependency/interface/frontend description.
"""
if os.getenv(_INSPECTION_CHILD) == "1":
return synthesize_repository_catalog_entries_in_process(
repo=repo,
version=version,
workspace=workspace,
repository_base=repository_base,
)
command = (
sys.executable,
"-m",
"govoplan_release.catalog_entry_synthesis",
"--inspect-repo",
repo,
"--version",
version,
"--workspace",
str(workspace),
"--repository-base",
repository_base,
)
environment = os.environ.copy()
environment[_INSPECTION_CHILD] = "1"
release_root = str(Path(__file__).resolve().parents[1])
environment["PYTHONPATH"] = os.pathsep.join(
item for item in (release_root, environment.get("PYTHONPATH", "")) if item
)
try:
result = subprocess.run(
command,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=30,
env=environment,
)
except subprocess.TimeoutExpired as exc:
raise ValueError(f"Cannot synthesize {repo}: isolated manifest inspection timed out.") from exc
if result.returncode != 0:
detail = result.stderr.strip() or result.stdout.strip() or "unknown inspection error"
raise ValueError(f"Cannot synthesize {repo}: isolated manifest inspection failed: {detail}")
payload_line = next(
(line.removeprefix(_INSPECTION_MARKER) for line in reversed(result.stdout.splitlines()) if line.startswith(_INSPECTION_MARKER)),
None,
)
if payload_line is None:
raise ValueError(f"Cannot synthesize {repo}: isolated manifest inspection returned no metadata.")
payload = json.loads(payload_line)
if not isinstance(payload, list) or not all(isinstance(item, dict) for item in payload):
raise ValueError(f"Cannot synthesize {repo}: isolated manifest inspection returned invalid metadata.")
return tuple(payload)
def synthesize_repository_catalog_entries_in_process(
*,
repo: str,
version: str,
workspace: Path,
repository_base: str,
) -> tuple[dict[str, object], ...]:
specs = {item.name: item for item in load_repository_specs(include_website=False)}
spec = specs.get(repo)
if spec is None:
raise ValueError(f"Cannot synthesize {repo}: repository is not registered.")
root = resolve_repo_path(spec, workspace)
project = read_project_metadata(root / "pyproject.toml")
package = required_text(project, "name", source=f"{repo}/pyproject.toml")
project_version = required_text(project, "version", source=f"{repo}/pyproject.toml").removeprefix("v")
expected_version = version.removeprefix("v")
if project_version != expected_version:
raise ValueError(
f"Cannot synthesize {repo}: project version {project_version!r} does not match selected version {expected_version!r}."
)
description = optional_text(project.get("description"))
entry_points = module_entry_points(project, repo=repo)
entries: list[dict[str, object]] = []
for declared_module_id, target in sorted(entry_points.items()):
manifest = load_manifest(root=root, target=target, repo=repo)
if manifest.id != declared_module_id:
raise ValueError(
f"Cannot synthesize {repo}: entry point {declared_module_id!r} returns manifest {manifest.id!r}."
)
if manifest.version.removeprefix("v") != expected_version:
raise ValueError(
f"Cannot synthesize {repo}/{manifest.id}: manifest version {manifest.version!r} does not match {expected_version!r}."
)
entry = manifest_catalog_entry(
manifest=manifest,
repo=repo,
package=package,
version=expected_version,
description=description,
root=root,
repository_base=repository_base.rstrip("/"),
)
entries.append(entry)
return tuple(entries)
def manifest_catalog_entry(
*,
manifest: ModuleManifest,
repo: str,
package: str,
version: str,
description: str | None,
root: Path,
repository_base: str,
) -> dict[str, object]:
tag = f"v{version}"
entry: dict[str, object] = {
"module_id": manifest.id,
"name": manifest.name,
"version": version,
"action": "install",
"python_package": package,
"python_ref": f"{package} @ {repository_base}/{repo}.git@{tag}",
"license_features": [f"module.{manifest.id}"],
"tags": ["official"],
}
if description:
entry["description"] = description
if manifest.dependencies:
entry["dependencies"] = list(manifest.dependencies)
if manifest.optional_dependencies:
entry["optional_dependencies"] = list(manifest.optional_dependencies)
if manifest.provides_interfaces:
entry["provides_interfaces"] = [
{"name": item.name, "version": item.version}
for item in manifest.provides_interfaces
]
if manifest.requires_interfaces:
entry["requires_interfaces"] = [
{
"name": item.name,
**({"version_min": item.version_min} if item.version_min else {}),
**({"version_max_exclusive": item.version_max_exclusive} if item.version_max_exclusive else {}),
"optional": item.optional,
}
for item in manifest.requires_interfaces
]
if manifest.frontend is not None and manifest.frontend.package_name:
validate_webui_package(root=root, package=manifest.frontend.package_name, version=version, repo=repo)
entry["webui_package"] = manifest.frontend.package_name
entry["webui_ref"] = f"{repository_base}/{repo}.git#{tag}"
if manifest.migration_spec is not None:
migration = manifest.migration_spec
entry["migration_safety"] = "requires_review"
entry["migration_notes"] = (
"Module owns database migrations; review release notes and migration output before activation."
)
if migration.migration_after:
entry["migration_after"] = list(migration.migration_after)
if migration.migration_before:
entry["migration_before"] = list(migration.migration_before)
if migration.migration_tasks:
entry["migration_tasks"] = [
{
"task_id": task.task_id,
"phase": task.phase,
"summary": task.summary,
"task_version": task.task_version,
"safety": task.safety,
"idempotent": task.idempotent,
**({"timeout_seconds": task.timeout_seconds} if task.timeout_seconds is not None else {}),
}
for task in migration.migration_tasks
]
return entry
def validate_initial_entry_closure(
*,
catalog_modules: list[object],
initial_module_ids: set[str],
) -> None:
entries = [item for item in catalog_modules if isinstance(item, dict)]
by_id = {str(item.get("module_id") or ""): item for item in entries}
providers: dict[str, list[str]] = {}
for entry in entries:
for raw in entry.get("provides_interfaces") or ():
if isinstance(raw, dict) and isinstance(raw.get("name"), str) and isinstance(raw.get("version"), str):
providers.setdefault(raw["name"], []).append(raw["version"])
issues: list[str] = []
for module_id in sorted(initial_module_ids):
entry = by_id[module_id]
for dependency in entry.get("dependencies") or ():
if isinstance(dependency, str) and dependency not in by_id:
issues.append(f"{module_id} requires catalog module {dependency!r}")
for raw in entry.get("requires_interfaces") or ():
if not isinstance(raw, dict) or raw.get("optional") is True:
continue
name = raw.get("name")
if not isinstance(name, str):
continue
available = providers.get(name, [])
if not any(
version_satisfies_range(
provider_version,
version_min=raw.get("version_min") if isinstance(raw.get("version_min"), str) else None,
version_max_exclusive=raw.get("version_max_exclusive")
if isinstance(raw.get("version_max_exclusive"), str)
else None,
)
for provider_version in available
):
issues.append(f"{module_id} requires unavailable compatible interface {name!r}")
if issues:
raise ValueError("Initial catalog entry dependency closure failed: " + "; ".join(issues))
def read_project_metadata(path: Path) -> dict[str, object]:
if not path.exists():
raise ValueError(f"Cannot synthesize catalog entry: missing {path}.")
with path.open("rb") as handle:
payload = tomllib.load(handle)
project = payload.get("project")
if not isinstance(project, dict):
raise ValueError(f"Cannot synthesize catalog entry: {path} has no [project] table.")
return project
def module_entry_points(project: dict[str, object], *, repo: str) -> dict[str, str]:
groups = project.get("entry-points")
raw = groups.get("govoplan.modules") if isinstance(groups, dict) else None
if not isinstance(raw, dict) or not raw:
raise ValueError(f"Cannot synthesize {repo}: pyproject has no govoplan.modules entry point.")
result = {
str(module_id).strip(): str(target).strip()
for module_id, target in raw.items()
if str(module_id).strip() and str(target).strip()
}
if len(result) != len(raw):
raise ValueError(f"Cannot synthesize {repo}: govoplan.modules contains an empty id or target.")
return result
def load_manifest(*, root: Path, target: str, repo: str) -> ModuleManifest:
module_name, separator, attribute = target.partition(":")
if not separator or not module_name or not attribute or "[" in attribute:
raise ValueError(f"Cannot synthesize {repo}: unsupported module entry point {target!r}.")
src = (root / "src").resolve()
expected_module = (src / Path(*module_name.split("."))).with_suffix(".py")
expected_package = src / Path(*module_name.split(".")) / "__init__.py"
expected = expected_module if expected_module.exists() else expected_package
if not expected.exists():
raise ValueError(f"Cannot synthesize {repo}: entry point source {expected_module} is missing.")
top_level = module_name.split(".", 1)[0]
with isolated_source_import(src=src, top_level=top_level):
try:
module = importlib.import_module(module_name)
except Exception as exc: # noqa: BLE001 - exact load failure is release evidence.
raise ValueError(f"Cannot synthesize {repo}: loading {target!r} failed: {exc}") from exc
origin = Path(str(getattr(module, "__file__", ""))).resolve()
if origin != expected.resolve():
raise ValueError(f"Cannot synthesize {repo}: entry point resolved outside selected source ({origin}).")
factory = getattr(module, attribute, None)
if not callable(factory):
raise ValueError(f"Cannot synthesize {repo}: entry point target {target!r} is not callable.")
manifest = factory()
if not isinstance(manifest, ModuleManifest):
raise ValueError(f"Cannot synthesize {repo}: entry point {target!r} did not return ModuleManifest.")
return manifest
@contextmanager
def isolated_source_import(*, src: Path, top_level: str) -> Iterator[None]:
saved: dict[str, ModuleType] = {
name: module
for name, module in tuple(sys.modules.items())
if name == top_level or name.startswith(f"{top_level}.")
}
for name in saved:
sys.modules.pop(name, None)
sys.path.insert(0, str(src))
importlib.invalidate_caches()
try:
yield
finally:
sys.path.remove(str(src))
for name in tuple(sys.modules):
if name == top_level or name.startswith(f"{top_level}."):
sys.modules.pop(name, None)
sys.modules.update(saved)
importlib.invalidate_caches()
def validate_webui_package(*, root: Path, package: str, version: str, repo: str) -> None:
declarations: list[tuple[str, str]] = []
for path in (root / "package.json", root / "webui" / "package.json"):
if not path.exists():
continue
payload = json.loads(path.read_text(encoding="utf-8"))
if isinstance(payload, dict) and isinstance(payload.get("name"), str) and isinstance(payload.get("version"), str):
declarations.append((payload["name"], payload["version"]))
if (package, version) not in declarations:
raise ValueError(
f"Cannot synthesize {repo}: frontend manifest declares {package}@{version}, but package metadata does not."
)
def required_text(payload: dict[str, object], key: str, *, source: str) -> str:
value = optional_text(payload.get(key))
if value is None:
raise ValueError(f"Cannot synthesize catalog entry: {source} has no {key!r}.")
return value
def optional_text(value: object) -> str | None:
return value.strip() if isinstance(value, str) and value.strip() else None
def main() -> int:
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument("--inspect-repo", required=True)
parser.add_argument("--version", required=True)
parser.add_argument("--workspace", type=Path, required=True)
parser.add_argument("--repository-base", required=True)
args = parser.parse_args()
try:
entries = synthesize_repository_catalog_entries(
repo=args.inspect_repo,
version=args.version,
workspace=args.workspace.resolve(),
repository_base=args.repository_base,
)
except ValueError as exc:
print(str(exc), file=sys.stderr)
return 1
print(_INSPECTION_MARKER + json.dumps(entries, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())

Some files were not shown because too many files have changed in this diff Show More