Compare commits

...

43 Commits

Author SHA1 Message Date
1aea3e7c4f fix: repair dev modules and scoped release git trust
Some checks are pending
Dependency Audit / dependency-audit (push) Waiting to run
Security Audit / security-audit (push) Waiting to run
2026-07-29 20:54:58 +02:00
3f9567af18 Gate Search on PostgreSQL in module CI
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 20:37:08 +02:00
de16f11ce8 test(datasources): include immutable payload schema
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 20:13:49 +02:00
9d6cdff4b8 CSS changes for release console
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 19:35:49 +02:00
aa4050c0ca feat: expose focused backend reload mode
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 18:52:55 +02:00
d3cdbd8c7a feat: include risk compliance in dev composition
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 18:46:53 +02:00
7115c4711d chore: register search wiki projects and tickets
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 15:50:35 +02:00
a3beca6fc5 chore: harden audit checks and issue taxonomy
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-29 14:16:27 +02:00
11d45bce25 Register governed Views module
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 21:04:55 +02:00
7b6135b89b Document governed Views architecture
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 19:05:37 +02:00
5b79e7d377 test: verify pinned dataflow publication composition
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 13:48:11 +02:00
6afb8fea76 fix: exclude generated artifacts from wiki sync
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 12:52:46 +02:00
163b35c0af feat: integrate datasource and workflow modules
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 12:45:05 +02:00
603e07cec5 Integrate Connectors and Dataflow validation
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 11:14:38 +02:00
97dfd333c6 Register Dataflow module and work taxonomy
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-28 01:34:06 +02:00
ba88c574b9 chore: move repositories to GovOPlaN organization
Some checks failed
Dependency Audit / dependency-audit (push) Has been cancelled
Security Audit / security-audit (push) Has been cancelled
2026-07-27 15:46:51 +02:00
8d292184d4 ci: execute tagged module pytest suites
All checks were successful
Security Audit / security-audit (push) Successful in 4m3s
Security Audit Toolbox Update / toolbox-update (push) Successful in 4s
Dependency Audit / dependency-audit (push) Successful in 1m51s
2026-07-23 01:18:47 +02:00
52bd3527cd ci: install release test harness dependencies
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m37s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 01:15:07 +02:00
ff49dabf8f ci: retain installed artifact checks in matrix
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m42s
Security Audit / security-audit (push) Successful in 4m7s
2026-07-23 00:57:24 +02:00
cd15aa514e ci: run artifact-aware module matrix tests
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m36s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 00:53:16 +02:00
a042baa1d3 fix(release): verify tags in mounted workspaces 2026-07-23 00:53:16 +02:00
e80de00f29 chore(release): align Mail with Campaign contract
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m37s
Security Audit / security-audit (push) Has been cancelled
2026-07-23 00:47:46 +02:00
c69acf0dee ci: use Gitea-compatible artifact upload
All checks were successful
Dependency Audit / dependency-audit (push) Successful in 1m39s
Security Audit / security-audit (push) Successful in 4m11s
2026-07-23 00:36:59 +02:00
8b93bbc6b6 ci: mount audit workspace safely
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m43s
Security Audit / security-audit (push) Failing after 4m10s
2026-07-23 00:30:28 +02:00
3fc17701df ci: fix action repository access
Some checks failed
Dependency Audit / dependency-audit (push) Successful in 1m45s
Security Audit / security-audit (push) Failing after 3m33s
2026-07-23 00:10:29 +02:00
9788bdde0c ci: harden repository bootstrap transport 2026-07-23 00:09:43 +02:00
a10a01c903 docs: surface system action status
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 23:45:42 +02:00
e005e54333 Seal catalog publication Git metadata
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:31:58 +02:00
76e4baa76e Define the catalog publication identity
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:23:01 +02:00
b6e9a9bd81 Pin the website publication toolchain
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 15s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:20:09 +02:00
dc46bda224 Validate complete Core release bundle sources
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 14s
Security Audit / security-audit (push) Failing after 13s
2026-07-22 22:10:35 +02:00
b00d4e55ee Harden isolated release builds in Flatpak
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 16s
Security Audit / security-audit (push) Failing after 15s
2026-07-22 21:48:48 +02:00
76f19dc602 Lock down the local release console 2026-07-22 21:37:33 +02:00
5381e37a9e Execute receipt-bound durable release runs 2026-07-22 21:37:18 +02:00
7ecf1f17b0 Harden release source execution 2026-07-22 21:37:14 +02:00
349a099e5a Harden catalog publication transaction 2026-07-22 21:20:08 +02:00
fdee766993 Authenticate selective catalog base pairs 2026-07-22 20:59:00 +02:00
47b9c05bde chore(release): align Campaign release dependency 2026-07-22 20:44:43 +02:00
4e42911477 Document artifact origin and installer receipts 2026-07-22 20:42:35 +02:00
9c0650b2ed Bind installed releases to signed artifact receipts 2026-07-22 20:42:05 +02:00
4dc0c8d013 Harden release artifact publication 2026-07-22 20:41:42 +02:00
35c346a1fa Add private release candidate handles 2026-07-22 20:41:36 +02:00
4edbc11fe5 feat(release): bound durable run retention 2026-07-22 20:16:19 +02:00
95 changed files with 15370 additions and 871 deletions

View File

@@ -2,17 +2,24 @@
# Copy to a deployment-local .env or secret store. Do not commit populated secrets.
APP_ENV=production
# Live graph changes are useful in development. Production should apply saved
# module state through a coordinated restart of all API and worker processes.
GOVOPLAN_MODULE_LIVE_APPLY_ENABLED=
GOVOPLAN_INSTALL_PROFILE=self-hosted
MASTER_KEY_B64=<generate-with-govoplan-config-env-template-generate-secrets>
DATABASE_URL=postgresql+psycopg://govoplan:change-me@127.0.0.1:5432/govoplan
GOVOPLAN_DATABASE_URL_PGTOOLS=postgresql://govoplan:change-me@127.0.0.1:5432/govoplan
GOVOPLAN_DB_POOL_SIZE=5
GOVOPLAN_DB_MAX_OVERFLOW=10
GOVOPLAN_DB_POOL_TIMEOUT_SECONDS=30
GOVOPLAN_DB_POOL_RECYCLE_SECONDS=1800
ENABLED_MODULES=tenancy,organizations,identity,access,admin,dashboard,policy,audit,files,mail,campaigns,calendar,docs,ops
ENABLED_MODULES=tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,files,mail,campaigns,calendar,poll,scheduling,connectors,datasources,dataflow,workflow,views,search,risk_compliance,postbox,notifications,docs,ops
CELERY_ENABLED=true
REDIS_URL=redis://127.0.0.1:6379/0
CELERY_QUEUES=send_email,append_sent,notifications,calendar,default
CELERY_QUEUES=send_email,append_sent,notifications,calendar,dataflow,events,default
CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=false
@@ -24,10 +31,14 @@ GOVOPLAN_HTTP_MAX_REQUEST_BODY_BYTES=536870912
GOVOPLAN_HTTP_HSTS_SECONDS=31536000
AUTH_LOGIN_THROTTLE_ENABLED=true
AUTH_ACTIVITY_TOUCH_INTERVAL_SECONDS=300
AUTH_LOGIN_THROTTLE_IDENTITY_LIMIT=10
AUTH_LOGIN_THROTTLE_CLIENT_LIMIT=100
AUTH_LOGIN_THROTTLE_WINDOW_SECONDS=900
AUTH_LOGIN_THROTTLE_REDIS_RETRY_SECONDS=30
# Production startup fails without Redis unless this explicit single-process
# risk acknowledgement is enabled.
GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE=false
CORS_ORIGINS=https://govoplan.example.org
GOVOPLAN_TRUSTED_HOSTS=govoplan.example.org

View File

@@ -21,23 +21,13 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "22"
- name: Configure SSH for release dependencies
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
- name: Use anonymous HTTPS for public GovOPlaN repositories
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
- name: Bootstrap GovOPlaN repositories
working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent ..
run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend dev audit dependencies
working-directory: govoplan
run: |

View File

@@ -7,6 +7,18 @@ on:
jobs:
module-matrix:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_DB: govoplan_test
POSTGRES_USER: govoplan
POSTGRES_PASSWORD: govoplan_test
options: >-
--health-cmd "pg_isready -U govoplan -d govoplan_test"
--health-interval 5s
--health-timeout 5s
--health-retries 20
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
@@ -17,23 +29,13 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "22"
- name: Configure SSH for release dependencies
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
- name: Use anonymous HTTPS for public GovOPlaN repositories
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
- name: Bootstrap GovOPlaN repositories
working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent ..
run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend release dependencies
working-directory: govoplan
run: |
@@ -43,6 +45,21 @@ jobs:
- name: Install WebUI release dependencies with test scripts
working-directory: govoplan
run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui
- name: Validate Search against PostgreSQL
working-directory: govoplan
env:
GOVOPLAN_SEARCH_POSTGRES_URL: postgresql+psycopg://govoplan:govoplan_test@postgres:5432/govoplan_test
run: |
GOVOPLAN_CORE_ROOT="$PWD/../govoplan-core" .venv/bin/python tools/checks/postgres-integration-check.py \
--database-url "$GOVOPLAN_SEARCH_POSTGRES_URL" \
--module-set search=tenancy,access,search \
--reset-schema \
--skip-retirement-atomicity
PYTHONPATH="$PWD/../govoplan-search/src:$PWD/../govoplan-core/src" \
.venv/bin/python -m unittest discover \
-s ../govoplan-search/tests \
-p test_postgres_search.py \
-v
- name: Run module matrix and contract tests
working-directory: govoplan
run: GOVOPLAN_CORE_ROOT="$PWD/../govoplan-core" PYTHON="$PWD/.venv/bin/python" bash tools/checks/check-module-matrix.sh

View File

@@ -16,29 +16,19 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "22"
- name: Configure SSH for release dependencies
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
- name: Use anonymous HTTPS for public GovOPlaN repositories
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
if [ -z "${GOVOPLAN_RELEASE_SSH_KEY_B64:-}" ]; then
echo "GOVOPLAN_RELEASE_SSH_KEY_B64 secret is required for git+ssh release dependencies."
exit 1
fi
printf '%s' "$GOVOPLAN_RELEASE_SSH_KEY_B64" | base64 -d > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo 'git.add-ideas.de ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDe48IOof2fJS1dTbJtLWQnWnr+JorZXKIFdOAM9ct8G' > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
- name: Bootstrap GovOPlaN repositories
working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent ..
run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https --exclude-repo addideas-govoplan-website
- name: Install backend release integration dependencies
working-directory: govoplan
run: |
python -m venv .venv
.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python .venv/bin/python --upgrade-pip
.venv/bin/python -m pip install '../govoplan-core[dev]'
.venv/bin/python -m pip install -r requirements-release-tests.txt '../govoplan-core[dev]'
- name: Install WebUI release dependencies
working-directory: govoplan
run: bash tools/release/install-webui-release-dependencies.sh ../govoplan-core/webui

View File

@@ -1,7 +1,6 @@
name: Security Audit
on:
pull_request:
push:
branches:
- main
@@ -21,6 +20,10 @@ jobs:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
path: govoplan
- name: Use anonymous HTTPS for public GovOPlaN repositories
run: |
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "git@git.add-ideas.de:GovOPlaN/govoplan"
git config --global --add url."https://git.add-ideas.de/GovOPlaN/govoplan".insteadOf "ssh://git@git.add-ideas.de/GovOPlaN/govoplan"
- name: Configure SSH for repository bootstrap
env:
GOVOPLAN_RELEASE_SSH_KEY_B64: ${{ secrets.GOVOPLAN_RELEASE_SSH_KEY_B64 }}
@@ -37,13 +40,13 @@ jobs:
chmod 600 ~/.ssh/known_hosts
- name: Bootstrap GovOPlaN repositories
working-directory: govoplan
run: python tools/repo/bootstrap-repositories.py --parent ..
- name: Run security audit
run: python tools/repo/bootstrap-repositories.py --parent .. --transport public-https
- name: Run whole-system security audit
working-directory: govoplan
run: tools/checks/security-audit/run.sh --mode "$SECURITY_AUDIT_MODE" --scope "$SECURITY_AUDIT_SCOPE" --reports-dir audit-reports
- name: Upload audit reports
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: security-audit-reports
path: govoplan/audit-reports

View File

@@ -4,6 +4,11 @@
**Repository type:** system (meta).
<!-- govoplan-repository-type:end -->
[![Module Matrix](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/module-matrix.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=module-matrix.yml&actor=0&status=0)
[![Release Integration](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/release-integration.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=release-integration.yml&actor=0&status=0)
[![Dependency Audit](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/dependency-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=dependency-audit.yml&actor=0&status=0)
[![Security Audit](https://git.add-ideas.de/GovOPlaN/govoplan/actions/workflows/security-audit.yml/badge.svg?branch=main)](https://git.add-ideas.de/GovOPlaN/govoplan/actions?workflow=security-audit.yml&actor=0&status=0)
This is the GovOPlaN meta repository. It is the operator entry point for
whole-product development, release orchestration, repository bootstrap, and
system-level Docker composition.
@@ -36,6 +41,16 @@ Open the WebUI in a browser after launch only when explicitly requested:
GOVOPLAN_OPEN_BROWSER=1 ./tools/launch/launch-dev.sh
```
Limit backend reload triggers during focused module work without changing the
enabled module graph:
```sh
GOVOPLAN_BACKEND_RELOAD_MODULES=calendar,campaign ./tools/launch/launch-dev.sh
```
Set `GOVOPLAN_BACKEND_RELOAD_MODULES=none` to watch only core/config sources.
Leaving it unset keeps the broad default and watches all enabled modules.
Start the shared development PostgreSQL service:
```sh
@@ -146,6 +161,9 @@ Meta ownership and module install/contract boundaries are documented in
`docs/META_REPO_SCAN.md` and `docs/MODULE_CONTRACTS_AND_INSTALLS.md`.
Frontend layout principles for module pages are documented in
`docs/FRONTEND_LAYOUT_PRINCIPLES.md`.
The provider-neutral datasource boundary and reusable Dataflow/Workflow graph
contract are documented in
`docs/DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md`.
The cross-product destination, stakeholder visions, configuration archetypes,
connected outcome stories, and capability horizons are documented in
the [Connected Governance Platform Roadmap](docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md).

View File

@@ -57,7 +57,7 @@ profile intentionally runs only PostgreSQL and Redis in containers; API,
WebUI, worker, and scheduler processes still run from editable source trees. A
target deployment must supply TLS termination, process supervision, secret
injection, monitoring, backup storage, and recovery procedures. The open
[Core backup/restore issue #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29)
[Core backup/restore issue #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29)
is a production gate. Identity federation, external audit export, and a tested
disaster-recovery plan are also not complete.
@@ -134,7 +134,7 @@ persistence, and durable shared or object storage. Run one scheduler only when a
selected module needs scheduled recovery. Multiple scheduler replicas require
leader election or an external lock, which is not established by this report.
This topology is a recommendation from the [Ops scalability profiles](https://git.add-ideas.de/add-ideas/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md),
This topology is a recommendation from the [Ops scalability profiles](https://git.add-ideas.de/GovOPlaN/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md),
not a currently shipped production Compose/Kubernetes/systemd package.
## Functional capability matrix
@@ -154,7 +154,7 @@ not a currently shipped production Compose/Kubernetes/systemd package.
| Configured-system documentation and Ops status pages | `verified` | Docs/Ops manifests contribute protected routes and WebUI packages; Ops code checks database, Redis, workers, storage and deployment-security settings. | This does not replace external monitoring or a target runbook. |
| Calendar/CalDAV integration | `partial` | Calendar `v0.1.8` supplies the catalogued storage/sync foundation. The durable external-write outbox, worker recovery, reconciliation, and retention work is committed, tested, and pushed on Calendar `main` after that tag. | The post-tag outbox work is remote-integrated source, not local-only WIP, but it is not in the signed stable package baseline and has not passed a target CalDAV drill. Bulk synchronized-calendar migration semantics remain separate work. |
| External LDAP/AD, OIDC/SAML or SCIM identity integration | `scaffold` | IDM owns normalized assignment APIs and documents connector boundaries. | Provider connectors, login callback flow and target directory reconciliation are not an implemented end-to-end capability. Use local accounts for this pilot. |
| Export-control/embargo-list screening | `planned` | Product-level [GovOPlaN #12](https://git.add-ideas.de/add-ideas/govoplan/issues/12) defines the consumer-independent user story. | No screening provider, list provenance, matching policy, review flow or legal evidence exists in this composition. |
| Export-control/embargo-list screening | `planned` | Product-level [GovOPlaN #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12) defines the consumer-independent user story. | No screening provider, list provenance, matching policy, review flow or legal evidence exists in this composition. |
| Workflow-driven journeys and views | `planned` | Workflow contracts/concepts exist outside this assessment. | Explicitly postponed. Do not include Workflow in pilot or production claims from this report. |
## Infrastructure matrix
@@ -177,13 +177,13 @@ not a currently shipped production Compose/Kubernetes/systemd package.
| Health/readiness | `verified` | `/health`, protected `/health/details`, and Ops checks for DB, Redis, workers, storage, maintenance and cookie/CORS posture. | Add external probes and distinguish liveness from dependency readiness for the chosen orchestrator. |
| Metrics, logs and alerting | `partial` | Correlation IDs, slow-request/query metrics in logs, worker inspection and operator status are implemented. | No bundled metrics exporter, log collector, dashboards, queue-depth alerts, pager route or SLO is verified. |
| Audit | `partial` | Local audit tables and retry outbox are verified. | Retention, tamper-evident export, privileged access review and SIEM integration are unproved. |
| Backup and restore | `partial` | Operator guide and installer hooks describe `pg_dump`/`pg_restore`; SQLite and simulated installer rollback drills exist. | [Core #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29) remains open. No target PostgreSQL + files + secrets restore drill or measured RTO/RPO exists. |
| Backup and restore | `partial` | Operator guide and installer hooks describe `pg_dump`/`pg_restore`; SQLite and simulated installer rollback drills exist. | [Core #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29) remains open. No target PostgreSQL + files + secrets restore drill or measured RTO/RPO exists. |
| Disaster recovery | `not_assessed` | The Ops guide asks for RPO/RTO and restore drills. | No agreed RPO/RTO, off-site copy, failover topology, dependency recovery order, communications plan or exercise evidence was supplied. |
The scalability and sizing documentation delivered the documentation portions
of [Core #217](https://git.add-ideas.de/add-ideas/govoplan-core/issues/217) and
[Core #219](https://git.add-ideas.de/add-ideas/govoplan-core/issues/219).
[Core #28](https://git.add-ideas.de/add-ideas/govoplan-core/issues/28) records the
of [Core #217](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/217) and
[Core #219](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/219).
[Core #28](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/28) records the
operator-documentation slice. These closed tickets are evidence of documented
models, not evidence that an organization's production environment has passed
them.
@@ -389,7 +389,7 @@ that observation to the assessment and signed catalog:
--output /var/tmp/govoplan-fit-review.json
```
The collector follows the strict version `0.3.0`
The collector follows the strict version `0.4.0`
[`installed-composition-evidence.schema.json`](installed-composition-evidence.schema.json)
contract. It enumerates all installed distributions whose normalized name starts
with `govoplan-`, compares the enabled assessed package and module-manifest
@@ -419,6 +419,9 @@ For each distribution, the collector also:
package-index/unknown origins and malformed metadata;
- compares a non-editable VCS commit with the assessment commit and, when one is
present, the signed catalog `selected_units` commit;
- derives two content identities from bytes it actually verifies: an
install-stable identity for immutable wheel-declared files and a full
installed-RECORD identity, while retaining only SHA-256 digests and counts;
- records only package/module identifiers, versions, bounded counters, hashes
and stable error codes. It never writes direct URLs, paths, hostnames,
usernames, exception text or file contents.
@@ -436,25 +439,31 @@ remain mutable even when their small editable-install `RECORD` is valid. Archive
or index artifacts without a hash anchored by the assessed release remain
unanchored. A matching declaration is reported under
`installed_source_provenance` as local consistency only, with
`release_origin_bound: false`. The separate `installed_release_origin` scope
remains unchecked until an independently anchored artifact digest or signed
installer receipt binds each installed payload to the signed catalog. The tool
does not accept a digest merely asserted inside the installed evidence document.
`release_origin_bound: false` until the separate origin proof succeeds. A signed
catalog can contain `release.artifacts` identities computed directly from built,
bounded wheel files. Wheels with installer-transformed scripts or headers are
marked `requires_installer_receipt`; catalog metadata alone cannot attest those
post-install bytes. A role-scoped installer receipt instead binds the exact
installed-evidence digest, full installed payload identities, catalog archive
digests and canonical signed-catalog digest. The tool never accepts an installed
hash merely because the installed evidence asserted it.
Only evidence produced in-process by `--collect-installed-evidence` is a local
observation, and it must be no more than five minutes old (with at most 30
seconds of future clock skew) at the selected verification time. Use
`--installed-evidence PATH` to compare evidence collected in a separate
environment. Imported JSON is unsigned and therefore always carries a blocker:
it can identify differences but cannot establish checked or valid installed
proof. A future signed collector-attestation format is required to cross that
environment. Imported JSON without a valid independent installer receipt
carries a blocker: it can identify differences but cannot establish checked or
valid installed proof. A receipt signed by a separately provisioned installer
authority authenticates that exact imported evidence across the process
boundary. The proof scope records observation mode, collection/evaluation time,
freshness, and whether evaluation used current time or an explicit historical
override. Evidence input and output are bounded to 4 MiB. Collection loads
the `govoplan.modules` entry-point factories in order to read module IDs and
manifest versions; that executes installed GovOPlaN manifest code. Run it only
inside the installation being assessed and with the same isolation expected for
other installed-artifact acceptance checks. This collector does not observe
receipt authentication, freshness, and whether evaluation used current time or
an explicit historical override. Evidence input and output are bounded to 4
MiB. Collection loads the `govoplan.modules` entry-point factories in order to
read module IDs and manifest versions; that executes installed GovOPlaN manifest
code. Run it only inside the installation being assessed and with the same
isolation expected for other installed-artifact acceptance checks. This
collector does not observe
which modules a running service activated, migrations, configuration, health,
or reference-journey behavior. Runtime activation therefore remains an explicit
unchecked boundary.
@@ -495,14 +504,50 @@ the tool's deterministic canonicalization.
```
With `--installed-evidence`, this command performs comparison and proof-binding
diagnostics only: the imported installed document is unsigned, so neither it nor
the boundary claim becomes accepted proof. Direct in-process collection removes
that unsigned-import gap, but still establishes only local metadata consistency.
No boundary claim is admitted by the current format because it has no separately
anchored artifact digest or signed installer receipt. A future evidence contract
must bind such an independently trusted release-artifact receipt to the installed
evidence digest; a signed collector-attestation format should also make that
exchange durable across processes.
diagnostics. Without an installer receipt, the imported document remains
unsigned, so neither it nor the boundary claim becomes accepted proof. Direct
in-process collection can match catalog-anchored artifact identities for wheels
without installer-transformed files. The installer-receipt flow below
authenticates a durable cross-process observation and is required for transformed
script/header payloads.
Issue a receipt only in the installation process performing the live collection.
The command refuses evidence supplied from a file and first validates the
assessment, independently trusted signed catalog, composition and RECORD bytes:
```bash
./.venv/bin/python tools/assessments/installer-receipt.py \
--assessment /srv/govoplan/assessment.json \
--catalog /srv/govoplan/catalogs/stable.json \
--keyring /srv/govoplan/catalogs/keyring.json \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--receipt-id install:production:20260722 \
--signing-key installer-production=/run/secrets/installer-ed25519.pem \
--python-artifact govoplan-core=/srv/govoplan/staged/govoplan_core-0.1.13-py3-none-any.whl \
--python-artifact govoplan-campaign=/srv/govoplan/staged/govoplan_campaign-0.1.10-py3-none-any.whl \
--evidence-output /srv/govoplan/evidence/installed.json \
--receipt-output /srv/govoplan/evidence/installer-receipt.json
```
Repeat `--python-artifact PACKAGE=/exact/consumed.whl` for every enabled
GovOPlaN distribution. The issuer reopens each exact wheel, verifies its archive
and payload identities against the signed catalog, and refuses a different
build with the same name and version. Receipt issuance must follow collection
within five minutes, and live verification must still fall inside that bounded
window; retain the pair for a reproducible historical review at its explicit
verification time.
Verify that durable pair with its separately managed trust root:
```bash
./.venv/bin/python tools/assessments/capability-fit.py \
--catalog /srv/govoplan/catalogs/stable.json \
--keyring /srv/govoplan/catalogs/keyring.json \
--trusted-keyring /srv/govoplan/trust/catalog-keyring.json \
--installed-evidence /srv/govoplan/evidence/installed.json \
--installer-receipt /srv/govoplan/evidence/installer-receipt.json \
--installer-authority-keyring /srv/govoplan/trust/installer-authorities.json
```
Target-environment and production-approval claims must use the assessment's
`deployment_profile.id` as `subject_id`. External-provider claims require the
@@ -510,11 +555,16 @@ operator to supply a bounded opaque expected subject with
`--expected-external-provider-subject`; without it, such a claim remains
unchecked and blocks. Expected and observed IDs are retained in proof scope.
The authority keyring is a governance trust root. Do not download or generate it
from the proof bundle being checked. The checker rejects proof-authority public
keys reused by either the published or independently trusted catalog keyring,
and rejects the same proof public-key material assigned to multiple authority
IDs. A malformed key, an invalid or empty validity interval, or ambiguous key
Both authority keyrings are governance trust roots. Installer receipt keys use
the strict
[`installer-receipt-authority-keyring.schema.json`](installer-receipt-authority-keyring.schema.json)
contract and may attest only `installed_release_origin`; their public material
must not be reused by catalog or boundary-proof authorities. Do not download or
generate them from the proof bundle being checked. The checker rejects
proof-authority public keys reused by either the published or independently
trusted catalog keyring, and rejects the same proof public-key material assigned
to multiple authority IDs. A malformed key, an invalid or empty validity
interval, or ambiguous key
identity invalidates the supplied authority set. Authority `not_after` is
exclusive. A target operator or approver must validate the referenced
drill/result artifacts before signing. The rerun verifies the
@@ -534,13 +584,13 @@ source tests or signed release metadata.
- [Production-like profile](../dev/production-like/README.md) and
[Compose dependencies](../dev/production-like/docker-compose.yml)
- [Module contracts and install boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
- [Core deployment operator guide](https://git.add-ideas.de/add-ideas/govoplan-core/src/branch/main/docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [Ops scalability profiles](https://git.add-ideas.de/add-ideas/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md)
- [Core deployment operator guide](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/branch/main/docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [Ops scalability profiles](https://git.add-ideas.de/GovOPlaN/govoplan-ops/src/branch/main/docs/SCALABILITY_PROFILES.md)
- Actual module manifests in the pinned repositories and the static contract
checker in this meta repository
- Core module-system/API smoke/auth/install-config tests, plus focused Campaign,
Files, Mail, Audit and Addresses tests
- [Campaign delivery runbook](https://git.add-ideas.de/add-ideas/govoplan-campaign/src/branch/main/docs/CAMPAIGN_DELIVERY_RUNBOOK.md)
- [Campaign delivery runbook](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/src/branch/main/docs/CAMPAIGN_DELIVERY_RUNBOOK.md)
- [Live signed stable catalog](https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json)
and [published keyring](https://govoplan.add-ideas.de/catalogs/v1/keyring.json),
verified against a separately provisioned local trust keyring

View File

@@ -10,7 +10,7 @@ baseline to that destination.
It is a durable direction, not a release promise or a substitute for issue
tracking. Live work state belongs in Gitea issues. The
[Core master roadmap](https://git.add-ideas.de/add-ideas/govoplan-core/src/branch/main/docs/GOVOPLAN_MASTER_ROADMAP.md)
[Core master roadmap](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/branch/main/docs/GOVOPLAN_MASTER_ROADMAP.md)
remains the technical module and wave sequence; this document supplies the
cross-product vision that sequence serves.
@@ -448,7 +448,7 @@ without creating disconnected ticket systems.
- As a manager, I can distinguish demand, backlog, SLA risk, recurring cause,
and verified resolution.
**Composition.** Issue reporting, helpdesk, cases, tasks, facilities, assets,
**Composition.** Tickets, helpdesk profiles, cases, tasks, facilities, assets,
inspections, booking, resources, calendar, files, notifications, connectors,
reporting, policy, and audit.
@@ -530,7 +530,7 @@ access, no hidden cross-module coupling, correction rather than fictional undo,
and a durable action/effect trail.
**Roadmap role.** The canonical product story is
[meta issue #12](https://git.add-ideas.de/add-ideas/govoplan/issues/12). It is a
[meta issue #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12). It is a
future cross-product configuration and contract program, not Campaign work and
not a claim that screening is implemented today.
@@ -1279,8 +1279,9 @@ them; they do not block the product vision today.
- Which exact HIS/CampusOnline interfaces, student-statistics fields and
official keys, accepted calculation, freeze/correction policy, privacy
profile, and drill-down level should define the first analytical data product?
- After the first source-to-report proof, do repeated source/dataflow contracts
justify separate `govoplan-datasources` and `govoplan-dataflow` modules?
- Which database, REST, directory, and managed-file providers should follow the
implemented separation of `govoplan-connectors`, `govoplan-datasources`, and
`govoplan-dataflow`, and which quality/promotion policy should prove it first?
- Which collaborative editor should be the first target, and should its first
accepted experience emphasize concurrent editing, controlled check-out, or
both?
@@ -1381,7 +1382,7 @@ integration. Conversely, 30 repositories had no open issue; for many
later-wave modules this meant no implementation program had been opened, not
that the capability was complete.
[Poll #2](https://git.add-ideas.de/add-ideas/govoplan-poll/issues/2) was a clear
[Poll #2](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/2) was a clear
tracker-drift example: its configurable transition engine, agreed transition
matrix/history, idempotent keyed retries, re-decision audit, archive/unarchive,
and preservation behavior were implemented and pushed while the issue still
@@ -1389,27 +1390,27 @@ reported `needs-info`.
Issue anchors that informed the bridge from the baseline into this roadmap:
- [Meta #10](https://git.add-ideas.de/add-ideas/govoplan/issues/10) for the
- [Meta #10](https://git.add-ideas.de/GovOPlaN/govoplan/issues/10) for the
capability/infrastructure assessment and its target proof;
- [Meta #11](https://git.add-ideas.de/add-ideas/govoplan/issues/11) for the
- [Meta #11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) for the
universal interface and focused-view direction;
- [Core #225](https://git.add-ideas.de/add-ideas/govoplan-core/issues/225) for
- [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225) for
guided, safe configuration;
- [Core #29](https://git.add-ideas.de/add-ideas/govoplan-core/issues/29) for the
- [Core #29](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29) for the
backup/restore production gate;
- [Core #263](https://git.add-ideas.de/add-ideas/govoplan-core/issues/263) and
[Campaign #63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63),
[#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62),
[#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65), and
[#69](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/69) for the
- [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263) and
[Campaign #63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63),
[#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62),
[#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65), and
[#69](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/69) for the
reference interface/delivery vocabulary and behavior;
- [Poll #1](https://git.add-ideas.de/add-ideas/govoplan-poll/issues/1) for the
- [Poll #1](https://git.add-ideas.de/GovOPlaN/govoplan-poll/issues/1) for the
database-enforced respondent invariant exposed by Scheduling;
- [Connectors #6](https://git.add-ideas.de/add-ideas/govoplan-connectors/issues/6)
- [Connectors #6](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/6)
for the governed connector configuration/simulation foundation;
- [Meta #9](https://git.add-ideas.de/add-ideas/govoplan/issues/9) for the first
- [Meta #9](https://git.add-ideas.de/GovOPlaN/govoplan/issues/9) for the first
permit-to-payment reference process; and
- [Meta #12](https://git.add-ideas.de/add-ideas/govoplan/issues/12) for the
- [Meta #12](https://git.add-ideas.de/GovOPlaN/govoplan/issues/12) for the
deliberately deferred, consumer-independent export-control story.
Live Gitea issue state remains canonical. These dated facts explain the roadmap

View File

@@ -0,0 +1,82 @@
# Datasource And Definition Graph Architecture
## Two-Layer Data Boundary
GovOPlaN separates governed data identity from external acquisition:
| Layer | Owner | Responsibilities |
| --- | --- | --- |
| Datasource layer | `govoplan-datasources` | Catalogue, tenant visibility, live/cached/static mode, staging, immutable materializations, frozen states, schema, fingerprints, provenance, and bounded reads |
| Connector layer | `govoplan-connectors` and protocol/provider modules | External protocols, endpoints, connection profiles, credentials, discovery, provider health, source-side filtering, and query pushdown |
Connectors publish versioned datasource origins. Datasources registers those
origins and presents one stable capability to Dataflow, Workflow, Reporting,
Risk Compliance, and other consumers. Consumers must not import connector
implementations or retain credentials.
The initial provider path is:
1. Connectors imports a bounded JSON/CSV snapshot and exposes it as an origin.
2. Datasources registers it as live or cached, or accepts a direct static upload
through staging.
3. Cached refreshes and static promotions append immutable materializations.
4. Any datasource may expose a frozen state for reproducible execution evidence.
5. Dataflow stores an opaque datasource reference, state policy, and expected
fingerprint.
6. A pinned Dataflow run may publish a complete bounded result as a new
immutable materialization through an idempotent Datasources capability.
Database, REST/HTTP, LDAP/directory, managed file, watched-directory, feed, and
stream providers fit behind the same origin contract. Provider-specific
configuration remains in Connectors.
## Shared Definition Graph
Core owns domain-neutral graph primitives:
- nodes, typed ports, edges, and configuration field descriptors;
- node libraries and category labels;
- graph size, connectivity, cycle, and node-count constraints;
- shared backend validation and frontend connection checks.
Domain modules own their semantics:
- Dataflow provides load, combine, filter, transform, and output nodes. Its
graph is acyclic and has one output.
- Workflow provides trigger, activity, review, decision, wait, module-action,
Dataflow, and outcome nodes. It permits governed loops and has exactly one
trigger plus one or more outcomes.
This division permits a shared editor shell without making Workflow a special
kind of Dataflow or leaking either module into Core.
## Current Implementation
- Core graph and datasource contracts are versioned at `0.1.0`.
- Workflow exposes a reusable graph editor, node-library discovery, validation,
tenant-isolated definitions, immutable revisions, and activation pinning.
- Datasources exposes catalogue, origins, staging, promotion, preview,
materialization history, refresh, freeze, retirement, and producer
publication APIs.
- Datasources WebUI exposes all current lifecycle views.
- Connectors adapts existing tabular snapshots to datasource origins.
- Dataflow consumes only Datasources catalogue/lifecycle capabilities and can
request current, live, or latest-frozen state.
- Dataflow exposes a pinned run-lifecycle capability and a Run/Publish surface.
Its first synchronous runner records lineage and terminal state, publishes
only complete bounded results, and retains output datasource/materialization
references.
- The focused composition check proves Connector origin -> Datasource ->
pinned Dataflow run -> frozen published materialization, including replay.
## Next Slices
1. Add persisted Workflow instances, resumable transitions, human activities,
retry policy, and event subscriptions against pinned definition revisions.
2. Add SQL database and governed REST origin providers with credential-envelope
references and bounded pushdown.
3. Add managed-file and directory origins.
4. Add datasource quality rules, schema compatibility policy, retention, and
promotion approvals.
5. Add asynchronous Dataflow workers and durable artifact-backed outputs for
runs that exceed the synchronous row/time/byte limits.

View File

@@ -8,11 +8,11 @@ The same pattern is reusable outside GovOPlaN for any project where Codex works
The repository contains Gitea issue templates in `.gitea/ISSUE_TEMPLATE`, a pull request template in `.gitea/PULL_REQUEST_TEMPLATE.md`, and the label taxonomy in `docs/gitea-labels.json`.
The scripts infer this repository from `origin` (`git@git.add-ideas.de:add-ideas/govoplan.git`). Override inference when needed:
The scripts infer this repository from `origin` (`git@git.add-ideas.de:GovOPlaN/govoplan.git`). Override inference when needed:
```bash
export GITEA_URL=https://git.add-ideas.de
export GITEA_OWNER=add-ideas
export GITEA_OWNER=GovOPlaN
export GITEA_REPO=govoplan
export GITEA_TOKEN=...
```
@@ -23,7 +23,7 @@ The API scripts also read `GITEA_*` values from the target repository's `.env` f
GITEA_TOKEN=...
# Optional if origin inference is not enough:
GITEA_URL=https://git.add-ideas.de
GITEA_OWNER=add-ideas
GITEA_OWNER=GovOPlaN
GITEA_REPO=govoplan
```

View File

@@ -444,25 +444,25 @@ reason to infer that a pattern is satisfied.
and does not duplicate a central component.
- Behavioral/accessibility evidence is linked from the rollout matrix and issue.
- Configured-system help can reach the applicable pattern or reference topic
when [Docs #15](https://git.add-ideas.de/add-ideas/govoplan-docs/issues/15)
when [Docs #15](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/15)
supplies that experience.
## First Pilot: Campaign
[Campaign #74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74)
[Campaign #74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74)
is the first full-domain audit and migration. It should prove patterns before
generic extraction:
- [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) and
[#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73): stable,
- [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) and
[#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73): stable,
accessible preview and attachment-detail overlays
- [#63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63): review
- [#63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63): review
stages, outcomes, blockers, and intervention vocabulary
- [#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62): explicit
- [#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62): explicit
synchronous/asynchronous send mode and durable delivery progress
- [#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65): one
- [#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65): one
coherent report filtering and count-affordance model
- [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35): guided
- [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35): guided
first-campaign entry
These slices do not depend on the Workflow runtime. Campaign's current

View File

@@ -59,16 +59,16 @@ The access guard column reports only the route-level declaration in
| Route | Owner / render evidence | Route-level access evidence | Primary task | Target archetype | Status / priority |
| --- | --- | --- | --- | --- | --- |
| `/admin` | `govoplan-access` `AdminPage` | Any core `adminReadScopes` | Administer system and tenant concerns assembled from module sections | Administration/configuration | Contributed; unreviewed; P1 under [Core #225](https://git.add-ideas.de/add-ideas/govoplan-core/issues/225) |
| `/admin` | `govoplan-access` `AdminPage` | Any core `adminReadScopes` | Administer system and tenant concerns assembled from module sections | Administration/configuration | Contributed; unreviewed; P1 under [Core #225](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/225) |
| `/address-book` | `govoplan-addresses` `AddressBookPage` | `addresses:contact:read` | Browse and manage contacts, address books, and lists | Directory/list-detail | Contributed; unreviewed; P2 after Campaign |
| `/calendar` | `govoplan-calendar` `CalendarPage` | `calendar:event:read` | Browse calendars/events and act on calendar data | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/campaigns` | `govoplan-campaign` `CampaignListPage` | `campaigns:campaign:read` | Find, compare, create, and open campaigns | List-detail entry | Pilot; P1 [Campaign #74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74) |
| `/campaigns` | `govoplan-campaign` `CampaignListPage` | `campaigns:campaign:read` | Find, compare, create, and open campaigns | List-detail entry | Pilot; P1 [Campaign #74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74) |
| `/campaigns/:campaignId/*` | `govoplan-campaign` `CampaignResourceRoute` and `CampaignWorkspace` | `campaigns:campaign:read`, plus resource probe | Configure, review, send, and inspect one campaign/version | List-detail workspace containing edit, review, monitoring, and evidence surfaces | Pilot; P1 Campaign #74 |
| `/operator` | `govoplan-campaign` `OperatorQueuePage` | `campaigns:campaign:read` and any of queue, control, retry, or reconcile | Monitor and intervene in campaign jobs through authority-specific controls | Monitoring/work queue | Pilot; durable queue controls delivered in [Campaign #78](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/78); #74 audit remains |
| `/reports` | `govoplan-campaign` `AggregateReportsPage` | `campaigns:report:read` | Compare privacy-protected cross-campaign outcome totals without recipient detail, diagnostics, export, or drill-down | Aggregate reporting | Pilot; aggregate-reader surface delivered in [Campaign #80](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/80); #74 audit remains |
| `/operator` | `govoplan-campaign` `OperatorQueuePage` | `campaigns:campaign:read` and any of queue, control, retry, or reconcile | Monitor and intervene in campaign jobs through authority-specific controls | Monitoring/work queue | Pilot; durable queue controls delivered in [Campaign #78](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/78); #74 audit remains |
| `/reports` | `govoplan-campaign` `AggregateReportsPage` | `campaigns:report:read` | Compare privacy-protected cross-campaign outcome totals without recipient detail, diagnostics, export, or drill-down | Aggregate reporting | Pilot; aggregate-reader surface delivered in [Campaign #80](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/80); #74 audit remains |
| `/templates` | `govoplan-campaign` `TemplatesPage` | No route guard declared in `module.ts` | Browse/manage campaign templates | Directory/list-detail | Pilot audit; permission intent must be verified; P2 |
| `/dashboard` | `govoplan-dashboard` `DashboardPage` | No route-specific scope | Assemble module-provided actionable widgets | Dashboard | Contributed; unreviewed; P2 |
| `/docs` | `govoplan-docs` `DocsPage` | Docs read or system/tenant settings read scopes | Read configured, available, and evidence-aware documentation | Documentation directory/reference | Contributed; unreviewed; P1 [Docs #15](https://git.add-ideas.de/add-ideas/govoplan-docs/issues/15) after initial pattern content |
| `/docs` | `govoplan-docs` `DocsPage` | Docs read or system/tenant settings read scopes | Read configured, available, and evidence-aware documentation | Documentation directory/reference | Contributed; unreviewed; P1 [Docs #15](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/15) after initial pattern content |
| `/files` | `govoplan-files` `FilesPage` | `files:file:read` | Browse folders/files and perform managed-file work | Directory/explorer | Contributed; metadata gap; unreviewed; P2 after Campaign |
| `/idm` | `govoplan-idm` `IdmPage` | Any IDM assignment/write or organization function-assign scope | Inspect and govern identity/function assignments | List-detail/configuration | Contributed; unreviewed; P2 |
| `/mail` | `govoplan-mail` `MailboxPage` | `mail:mailbox:read` | Browse mailboxes and messages | Directory/list-detail | Contributed; metadata gap; unreviewed; P2 after Campaign |
@@ -138,7 +138,7 @@ above; they are not independent routes.
Campaign is detailed first because it exercises almost every archetype. The
recipient-data editor is now consolidated into the `recipients` section on
remote `main`; [Campaign #67](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/67)
remote `main`; [Campaign #67](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/67)
records the accepted and verified integration boundary.
Campaign already consumes core primitives including `ModuleSubnav`, `Card`,
@@ -150,25 +150,25 @@ prove that the composition or states satisfy the pattern.
| Surface / code evidence | Primary task | Target pattern | Material consequence/state | Known issue / rollout |
| --- | --- | --- | --- | --- |
| Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | Audit in [#74](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/74); guided entry [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35) |
| Campaign list (`CampaignListPage`) | Find, compare, create, open | List-detail entry | Campaign lifecycle/status and creation | Audit in [#74](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/74); guided entry [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) |
| Overview (`CampaignOverviewPage`) | Understand/edit campaign identity, version, access, lifecycle | Object overview plus adaptive edit | Lock/archive/delete/access changes need real consequence and reversibility wording | #74 remaining audit |
| Fields (`CampaignFieldsPage`) | Define recipient/template field schema | Structured editor | Schema changes can invalidate recipient/template data | #74 audit |
| Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74; attachment-detail [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) |
| Attachments/files (`AttachmentsDataPage`, `AttachmentRulesOverlay`) | Select sources and attachment/ZIP rules | Directory chooser plus adaptive rule editor | Missing or mismatched files affect built messages | #74; attachment-detail [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) |
| Recipients (`RecipientDataPage`) | Select/import/map/edit recipients, address fields and per-recipient values/files | Import/mapping plus list-detail editor | Personal data, validation, bulk activation, file links | Consolidated editor delivered in #67; #74 remaining audit and guided entry #35 |
| Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74; stable overlay [#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73) |
| Template (`TemplateDataPage`, placeholder/expression dialogs) | Author subject/body and preview substitutions | Adaptive editor plus stable preview | Generated communication content and unresolved expressions | #74; stable overlay [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) |
| Mail settings (`MailSettingsPage` settings view) | Select/configure campaign mail transport | Adaptive configuration | Credentials, SMTP/IMAP destinations, test outcomes | #74; align with Core #225 mail pattern |
| Campaign settings (`GlobalSettingsPage` settings view) | Configure campaign behavior | Adaptive configuration | Can alter validation/build/send behavior | #74 audit |
| Mail policy (`MailSettingsPage` policy view) | Inspect/override effective mail policy | Effective policy/provenance editor | Inheritance and locks affect allowed delivery | #74; Core #225 policy pattern |
| Campaign policy (`GlobalSettingsPage` policy view) | Inspect/override campaign policy | Effective policy/provenance editor | Inheritance, actor authority, and blocked edits | #74; Core #225 policy pattern |
| Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, bounded synchronous execution, persisted queue mode, partial effects, retries, evidence | Bounded synchronous and explicit/persisted queued modes delivered in [#62](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/62) and [#79](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/79); [#63](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/63) wording and #74 audit remain |
| Message and attachment detail overlays | Inspect one built/mock message and its attachment links | Stable detail/review dialog | Personal data, exact outbound content, reviewed state | Delivered and verified in [#59](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/59) and [#73](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/73) |
| Campaign report (`CampaignReportPage`) | Filter and inspect delivery outcomes | Reporting/list-detail | Partial, failed, explicitly excluded/skipped, SMTP/IMAP outcomes and retries | Server-owned filtering and counts delivered in [#65](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/65) with the full-result DataGrid contract from [Core #263](https://git.add-ideas.de/add-ideas/govoplan-core/issues/263); excluded semantics in [#66](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/66) |
| Review/send (`ReviewSendPage`) | Validate, build, mock-test, confirm/send, inspect results | Guided review/decision plus durable progress | External communication, bounded synchronous execution, persisted queue mode, partial effects, retries, evidence | Bounded synchronous and explicit/persisted queued modes delivered in [#62](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/62) and [#79](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/79); [#63](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/63) wording and #74 audit remain |
| Message and attachment detail overlays | Inspect one built/mock message and its attachment links | Stable detail/review dialog | Personal data, exact outbound content, reviewed state | Delivered and verified in [#59](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/59) and [#73](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/73) |
| Campaign report (`CampaignReportPage`) | Filter and inspect delivery outcomes | Reporting/list-detail | Partial, failed, explicitly excluded/skipped, SMTP/IMAP outcomes and retries | Server-owned filtering and counts delivered in [#65](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/65) with the full-result DataGrid contract from [Core #263](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/263); excluded semantics in [#66](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/66) |
| Audit (`CampaignAuditPage`) | Inspect campaign evidence/history | Provenance timeline/report | Actor/action/effect trace | #74 audit |
| JSON (`CampaignJsonView`) | Inspect expert representation | Advanced diagnostics/reference | Raw data may contain personal/configuration values; not a primary editor | #74 privacy/redaction audit |
| Create wizard (`CreateWizard`) | Seed a campaign through basics, sender, fields, recipients, template, attachments, review, send | Guided setup | Current steps mix creation and later consequential delivery; completion semantics need audit | Guided first campaign [#35](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/35) |
| Create wizard (`CreateWizard`) | Seed a campaign through basics, sender, fields, recipients, template, attachments, review, send | Guided setup | Current steps mix creation and later consequential delivery; completion semantics need audit | Guided first campaign [#35](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/35) |
| Review/send wizard routes | Alternate guided review/send shells | Guided review | Tracked routes exist; implementation relationship to `ReviewSendPage` must be established, not guessed | #74 inventory decision |
| Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Campaign/version/job identity, historical active-version discovery, fixed action positions, authority-aware disabled states, exact non-overlapping queue counts, server-paged jobs, bounded refresh, retry/queue/reconcile per version, campaign-wide pause/resume/cancel, and leave/return progress | Durable operator controls delivered in [#78](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/78); #74 wording/accessibility audit remains |
| Aggregate reports (`AggregateReportsPage`) | Compare cross-campaign delivery outcomes | Privacy-preserving aggregate reporting | Tenant/campaign ACL, deployment/tenant small-cell policy, complementary and overlapping-cell suppression, explicit denominator, and no recipient detail/diagnostics/export/drill-down | Separate aggregate-reader surface delivered in [#80](https://git.add-ideas.de/add-ideas/govoplan-campaign/issues/80); not parity with the permission-gated per-campaign detail report |
| Operator queue (`OperatorQueuePage`) | Monitor jobs and intervene | Monitoring/work queue | Campaign/version/job identity, historical active-version discovery, fixed action positions, authority-aware disabled states, exact non-overlapping queue counts, server-paged jobs, bounded refresh, retry/queue/reconcile per version, campaign-wide pause/resume/cancel, and leave/return progress | Durable operator controls delivered in [#78](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/78); #74 wording/accessibility audit remains |
| Aggregate reports (`AggregateReportsPage`) | Compare cross-campaign delivery outcomes | Privacy-preserving aggregate reporting | Tenant/campaign ACL, deployment/tenant small-cell policy, complementary and overlapping-cell suppression, explicit denominator, and no recipient detail/diagnostics/export/drill-down | Separate aggregate-reader surface delivered in [#80](https://git.add-ideas.de/GovOPlaN/govoplan-campaign/issues/80); not parity with the permission-gated per-campaign detail report |
| Templates route (`TemplatesPage`) | Browse template records | Directory/list-detail | Template availability and later generated outputs | #74 audit; verify missing route guard intent |
The five review stages currently named in code are `Validate and inspect`,
@@ -186,7 +186,7 @@ The following local repositories contain a backend manifest but no
`govoplan-contracts`, `govoplan-dist-lists`, `govoplan-evaluation`,
`govoplan-facilities`, `govoplan-forms-runtime`, `govoplan-grants`,
`govoplan-helpdesk`, `govoplan-identity`, `govoplan-inspections`,
`govoplan-issue-reporting`, `govoplan-learning`, `govoplan-permits`,
`govoplan-tickets`, `govoplan-learning`, `govoplan-permits`,
`govoplan-poll`, `govoplan-procurement`, `govoplan-records`,
`govoplan-resources`, `govoplan-rest`, `govoplan-risk-compliance`,
`govoplan-soap`, `govoplan-tenancy`, and `govoplan-transparency`.
@@ -199,7 +199,7 @@ backend-only modules may remain intentionally headless.
| Order | Scope | Current evidence | Target | Owner / issue | Verification gate | Status |
| --- | --- | --- | --- | --- | --- | --- |
| 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/add-ideas/govoplan/issues/11) | Docs links/diff checks; issue/wiki sync after integration | Initial slice in this document |
| 0 | Product grammar and route inventory | Doctrine, ledger, layout rules, module contract, current route sources | One reconciled pattern language and evidence inventory | Meta [#11](https://git.add-ideas.de/GovOPlaN/govoplan/issues/11) | Docs links/diff checks; issue/wiki sync after integration | Initial slice in this document |
| 1 | Campaign baseline integration | Recipient-editor WIP and tracker state have been reconciled with remote `main` | Integrated, testable baseline before migration claims | Campaign #67 and tracker cleanup | Backend and focused WebUI suites; issue evidence | Complete 2026-07-22 |
| 2 | Campaign previews/details | Stable shared dialog with bounded scrolling and fixed responsive preview workspace | Stable header/body/footer, accessible long-content detail | Campaign #59 and #73 | Review-preview and overlay structure tests | Complete 2026-07-22 |
| 3 | Campaign review/interventions | Five domain-owned stages with unresolved intervention language | Clear stages, outcomes, blockers, next actor/action, reviewed evidence | Campaign #63 | State matrix behavior/accessibility tests and agreed vocabulary | P1 needs product wording decision |

View File

@@ -114,7 +114,10 @@ For release validation:
```sh
./.venv/bin/python tools/repo/sync-python-environment.py --requirements requirements-release.txt --python ./.venv/bin/python
./.venv/bin/python -m pip install -r requirements-release-tests.txt
```
That install is necessary because the release environment intentionally resolves
tagged package refs, not local editable source trees.
tagged package refs, not local editable source trees. The second requirements
file contains only the harness needed to execute tests from those immutable
source tags; it is not part of the deployable release dependency set.

View File

@@ -445,8 +445,9 @@ that first needs them:
- exact HIS/CampusOnline source endpoints, student-statistics keys and accepted
calculation, freeze/correction policy, privacy profile, and permitted
drill-down level;
- whether repeated data-source and dataflow contracts justify separate
`govoplan-datasources` and `govoplan-dataflow` modules after the Stage 3 proof;
- datasource provider selection and quality/promotion policy; the architecture
now separates `govoplan-datasources` lifecycle from `govoplan-connectors`
acquisition and `govoplan-dataflow` transformation;
- first collaborative editor/provider and whether the first UX is concurrent
editing, controlled check-out, or both; and
- first Records/archive target and approval/signature assurance level.

View File

@@ -5,8 +5,9 @@ GovOPlaN releases. It belongs to the `govoplan` meta repository because it works
across all local checkouts, release scripts, module manifests, migration audits,
catalog files, Git state, and signing keys.
The current implementation has a read-only dashboard plus guarded local
candidate/publish actions:
The current implementation has a read-only dashboard, preview-only legacy
controls, and a bounded durable executor for release steps whose inputs and
effects can be verified safely:
- inspect repositories from `repositories.json`
- show dirty, ahead, behind, missing, no-HEAD, and tag state
@@ -19,10 +20,12 @@ candidate/publish actions:
- select the repositories that should advance through checkboxes
- generate dry-run selective release plans for independently versioned packages
- freeze a selective plan as a durable, resumable local release run
- create annotated source release tags for selected clean, version-aligned
repositories and publish each branch/tag pair atomically
- generate signed catalog candidates for the selected release units
- preview/apply/push reviewed catalog candidates behind explicit confirmations
- durably preflight a creation-time-bound repository, create its annotated tag,
and publish its branch/tag pair atomically
- build selected Python wheels and generate a private, signed, receipt-bound
catalog candidate
- publish that exact candidate through a verified website commit and immutable
tag after explicit confirmation
Start it from the meta repository:
@@ -30,8 +33,32 @@ Start it from the meta repository:
./.venv/bin/python tools/release/release-console.py
```
The server binds to `127.0.0.1` by default and prints a URL containing a local
API token. Open that URL in a browser on the same machine.
The launcher accepts only a numeric loopback address, binds to `127.0.0.1` by
default, always creates a fresh API token, and prints that token only in the URL
fragment. Open that URL in a browser on the same machine. A deliberately
tokenless embedded app is read-only: non-GET `/api/` requests fail with `403`.
Non-loopback operation needs a separately deployed authenticated TLS boundary;
the local launcher will not expose the mutation API that way.
Run durable release mutation only against an operator-private workspace.
Repository roots, every path ancestor, critical and nested Git metadata, and
tracked worktree files must be owned by the console process UID (or root where
appropriate) and must not be group/world writable. Symlink/special Git
metadata, object alternates, and grafts are rejected. The console pins
`/usr/bin/git`, `/usr/bin/ssh`, a fixed system `PATH`, disabled hooks, isolated
Git configuration, and no replace objects. Shared or `nfsnobody`-owned
checkouts remain usable for read-only planning, but every durable executor
fails closed there; clone the registered origins into a private workspace
before releasing.
The runtime itself is part of the authority boundary. Durable run creation
verifies the meta checkout, release/check tooling, repository registry, Python
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
executables. It then binds a clean meta-repository HEAD and named branch that
exactly match the registered `origin`. Every execution and reconciliation
rechecks that receipt. Permission checks cannot prove that code was safe before
it entered a writable tree, so release from a fresh operator-private clone or a
separately verified installed console artifact.
The web UI starts with a repository table. Each repository can be checked
independently and assigned its own target version. Repositories without version
@@ -57,6 +84,12 @@ into a versioned local run record. The server rebuilds the selective plan and
requires the plan to resolve exactly the requested repositories and target
versions; the browser cannot submit or replace the plan snapshot. The input and
plan are then immutable and covered by a canonical SHA-256 integrity digest.
Every executable repository step also carries its creation-time full HEAD,
branch, worktree state, target tag, and a SHA-256 over both the fetch and push
URLs of `origin`. Both URLs must exactly equal the remote registered in
`repositories.json`; a changed HEAD, branch, worktree, remote, or push URL
requires a new run rather than silently retargeting the frozen compatibility
decision.
The complete record also has a checksum so a valid-looking manual edit to its
mutable state fails closed. File permissions remain the authority boundary;
these digests detect accidental or manual corruption, not an attacker who can
@@ -66,9 +99,10 @@ channel, or gate input requires a new run.
Creation requires a caller-generated `request_id`. Its SHA-256 fingerprint is
the private, workspace-scoped durable mapping to exactly one run; the raw
identifier is never persisted. Repeating the same identifier with the same
immutable inputs returns that run without rebuilding the plan, even if the
live dashboard has since drifted. Reusing it with different inputs fails
closed. The browser keeps an uncertain create identifier in session storage,
immutable inputs returns that run without rebuilding the plan while it remains
inside the bounded local retention window, even if the live dashboard has
since drifted. Reusing it with different inputs fails closed. The browser keeps
an uncertain create identifier in session storage,
replays it after reload, and selects the known run returned by the server. A
successful create remains shown as saved if only the subsequent list refresh
fails.
@@ -86,20 +120,30 @@ modes, malformed schemas, unknown fields, invalid state combinations,
oversized files, and digest mismatches fail closed. A bad record is neither
rewritten nor automatically quarantined.
The store retains at most 512 workspace-scoped run records created through the
console. On creation at that limit it removes only the oldest fully completed,
integrity-verified record. Running, planned, attention, blocked, foreign, and
unreadable records are never deleted implicitly; if no completed record is
available, creation fails closed with a retention remediation. Unavailable
records still consume the bound and remain visible in cursor-paginated lists
so corruption cannot be hidden by normal turnover.
The full resolved-workspace SHA-256 is part of both the private storage
namespace and immutable input snapshot. Every list, read, and state transition
checks it. An alternate workspace therefore cannot list or resume another
workspace's runs. This remains true for an embedding/test `run_state_root`
override: the server always appends
`workspace-<full-sha256>/release-runs/` rather than treating the override as a
shared record directory. A corrupt record from one workspace therefore cannot
leak even its identifier or an integrity error into another workspace.
shared record directory. Durable candidates use its private sibling
`release-candidates/` directory, never a checkout-local runtime path. A corrupt
record from one workspace therefore cannot leak even its identifier or an
integrity error into another workspace.
Each frozen plan step has an explicit `pending`, `running`, `succeeded`,
`failed`, or `interrupted` state. Plan order remains a prerequisite: a later
step is unavailable until earlier steps have succeeded. Exact attempt and
resume/retry/reconciliation request identifiers are fingerprinted so delayed
repetitions remain idempotent for the run's lifetime. These fingerprints are
repetitions remain idempotent for the retained run's lifetime. These fingerprints are
never evicted: the store fails closed before accepting more than 2,048 commands
or 2,048 attempts and asks the operator to create a fresh run. The display
record keeps at most 256 server-generated state events. Events contain only an
@@ -131,6 +175,27 @@ effect, and `unresolved` keeps the run blocked. Each outcome emits a bounded,
code-only state event. A known failed attempt can likewise be prepared for
retry. The UI keeps unavailable controls visible and disabled.
Supported executors durably claim the step before invoking an effect. Exact
attempt replays return the recorded outcome and never invoke the executor a
second time. Successful repository preflight, tag, and push steps persist a
bounded repository-state receipt. Tag reconciliation independently requires an
annotated local tag at the frozen HEAD; push reconciliation additionally
requires both the remote annotated tag object and remote branch to match.
Catalog generation persists
only its server-issued opaque candidate ID and canonical catalog SHA-256, then
re-resolves and re-hashes that private candidate before publication.
The safe baseline is intentionally narrower than the complete dry-run plan.
Dirty worktrees and version changes still show commit/version steps, but those
steps have no durable executor because the frozen run does not bind the exact
proposed file content. A run selecting Core together with one or more modules
begins with a disabled dependency-ordering barrier: local module tags, Core
release-lock regeneration/commit, Core tagging, alignment verification, and
pushes need an explicit DAG executor before that composition can mutate.
Operators must prepare and review those changes outside the console and create
a new run from the resulting clean HEAD. The console never skips these steps or
claims an end-to-end release succeeded.
The browser likewise retains the request identifier for an uncertain
resume/retry/reconciliation response and replays it after reload. A successful
replay selects the returned run state. Transport and server failures retain the
@@ -142,26 +207,40 @@ other `/api/` route:
- `POST /api/release-runs` requires `request_id`, then idempotently rebuilds and
freezes a selective plan only when that creation is not already known.
- `GET /api/release-runs` lists bounded summaries ordered by verified
`updated_at`, then `created_at` and run identifier. Unreadable entries are
appended deterministically when room remains instead of displacing newer
verified runs.
- `GET /api/release-runs` lists bounded summaries ordered by immutable
`created_at` and run identifier. `next_cursor` advances a stable descending
traversal even while older runs are updated, without offset duplicates or
skips. Unreadable entries
remain a deterministic final section and are therefore reachable through
pagination instead of displacing newer verified runs.
- `GET /api/release-runs/{run_id}` reads and verifies one exact record.
- `POST /api/release-runs/{run_id}/resume` records explicit recovery.
- `POST /api/release-runs/{run_id}/steps/{step_id}/retry` prepares a failed or
read-only interrupted step for another attempt.
- `POST /api/release-runs/{run_id}/steps/{step_id}/reconcile` records a
confirmed observed outcome for an interrupted mutating step.
- `POST /api/release-runs/{run_id}/steps/{step_id}/execute` claims and invokes
only the narrow executor declared by the immutable plan step. Durable release
execution accepts only the registered `origin`, never a caller-selected
remote.
- `POST /api/release-runs/{run_id}/steps/{step_id}/preview` provides the
non-mutating preview for receipt-bound catalog publication.
Run-storage errors are confined to the Durable Run State card; dashboard and
release-preview collection continue and show the bounded storage remediation.
This foundation tracks state only. It deliberately does not run a plan step or
infer success from a button click. Existing preview and executor controls stay
separate and continue to require `COMMIT`, `TAG`, `PUBLISH`, `APPLY`, or `PUSH`
at their existing narrow boundaries. Wiring confirmed executors to claim and
finish run steps, plus package/install verification, remains follow-up work;
until then the console must not present a run as execution evidence.
The run record is execution evidence only for a supported step whose durable
claim and bounded result receipt were persisted. The console never infers
success from a button click or process exit alone. An executor exception or a
lost result write leaves the attempt interrupted and non-retriable until
explicit recovery. Catalog publication persists the candidate and keyring
hashes, exact website commit, annotated tag object and peeled commit, branch,
tag name, and registered-origin digest. A successful reconciliation revalidates
the candidate against the trust anchor in the frozen website parent, requires
the exact deterministic catalog/keyring/module blobs and full commit delta, a
sole frozen parent, and matching local and remote branch/tag identities. If any
part cannot be proved, the run remains interrupted and may only be recorded as
`unresolved`.
Dashboard collection is also fail closed. Unreadable Core version metadata or a
malformed module contract is returned as a bounded `collection_errors` entry
@@ -188,11 +267,12 @@ Plain repository pushes are separate from catalog publication. `Preview Push`
shows the selected repository push commands. `Push Selected` requires `PUSH` in
the repository push confirmation field.
The source release panel closes the former gap between a release plan and a
catalog candidate. `Preview Tag + Publish` is non-mutating. `Create Tags`
requires `TAG`; `Publish Tags` requires `PUBLISH` and atomically pushes the
selected branch and annotated tag. The gate requires an aligned target version,
a clean named branch with a HEAD, and a checkout that is not behind. Existing
The source release panel retains `Preview Tag + Publish` as a non-mutating
inspection. Its legacy `Create Tags` and `Publish Tags` controls stay visible
but disabled; the corresponding mutation endpoint rejects apply requests.
Creation and atomic branch/tag publication use the `TAG` and `PUBLISH`
confirmations on the durable run steps. The gate requires an aligned target
version, a clean named branch with a HEAD, and a checkout that is not behind. Existing
local or remote tags must resolve to the selected HEAD and are never moved.
The local and remote annotated tag objects must also be identical, not merely
point at the same commit. Before any source tag is created, the console loads
@@ -200,13 +280,17 @@ the cross-repository module registry. This release gate also rejects every
user-facing workflow documentation topic that has no scope condition, or has
an alternative condition without `required_scopes` or `any_scopes`.
The catalog workflow panel can also operate on the same selected rows:
The catalog workflow panel can also operate on the same selected rows for
generation and preview. Its legacy apply/push controls stay disabled; durable
publication consumes only the candidate receipt recorded by that run:
- `Generate` creates a signed candidate below `runtime/release-candidates/`.
- `Generate` creates a signed candidate in the operator's private XDG state
directory and records only an opaque candidate ID plus the canonical catalog
SHA-256 in durable run state.
- `Preview` validates a candidate and shows what would be copied into the
website repository.
- `Apply + Website Tag` requires `APPLY` in the confirmation field.
- `Push Website Release` requires `PUSH` in the confirmation field.
- `Apply + Website Tag` remains visible but disabled outside a durable run.
- `Push Website Release` remains visible but disabled outside a durable run.
Source release tags belong to Core or module repositories. Website catalog
publication creates a separate catalog tag in the website repository; the UI
@@ -231,6 +315,53 @@ cannot be applied or published while any referenced source tag is absent or
inconsistent; the preview and API response list each repository and missing or
invalid ref so the operator can repair the exact source releases first.
Every selected Python release must also supply its exact built wheel. Durable
generation first verifies that the receipt-bound annotated tag is the same
object locally and on the registered origin. It clones an isolated checkout at
the receipt's exact commit and builds from that checkout, never from the mutable
live worktree. Every authenticated base-catalog source is likewise cloned from
its registered origin at an annotated release tag; only selected repositories
contribute synthesized fields. The base catalog and keyring are read as one
authenticated, exact private snapshot before synthesis.
Python wheels are built by a required Bubblewrap worker with no network,
private temporary/home directories, a read-only exact source mount, no host
home or file keys, cleared environment, fixed system tools, and CPU/address
space/process/file-size limits. If a trusted Bubblewrap launcher is unavailable,
generation fails closed. The worker must return one bounded regular wheel; the
console copies it through no-follow descriptors into a fresh private file,
`fsync`s it, then validates its package identity. Generation computes the
archive SHA-256 and an install-stable
payload identity from one bounded, regular-file descriptor and signs those
values into `release.artifacts`. Updating a Python version without a matching
wheel removes the stale identity. A source-only preview can still explain the
gap, but apply, commit, tag, and push fail closed until every selected Python
unit has a matching built-artifact identity. Repositories selected only for a
meta/source tag do not need a Python artifact.
Candidate directories and files are operator-owned `0700`/`0600` state. Before
any later release step consumes one, the executor re-resolves the opaque handle
below the configured root and re-hashes the signed catalog against its persisted
receipt. Shared roots, symlinks, channel path fragments, altered candidates, and
unowned files are rejected.
The current same-host worker is a containment baseline, not the final hostile
build-service boundary. `RLIMIT_FSIZE` is per file, and the worker does not yet
have a size-limited filesystem/cgroup quota, a fresh kernel keyring, or a
seccomp profile denying keyctl/request-key/ptrace/mount operations. A malicious
backend could therefore exhaust scratch disk/inodes or target same-UID kernel
facilities. Closing that denial-of-service/isolation gap requires a dedicated
quota/cgroup worker with a fresh keyring and seccomp policy.
The server-owned wheel builds remain under the private candidate's `artifacts/`
directory. This slice signs their identities but does **not** upload the wheel or
add a download URL to the public catalog; the existing Git `python_ref` is source
provenance and rebuilding it is not an equivalent artifact. Keep the private
candidate until the exact wheels have been transferred through an approved
deployment channel, verified against `archive_sha256`, consumed by the installer,
and covered by its signed receipt. Public artifact transport and receipt-aware
candidate cleanup remain separate release-lifecycle work.
Read-only provenance checks derive a same-host HTTPS Git URL from conventional
SSH remotes when possible, with a non-interactive check of the configured remote
as fallback. Source tag creation and atomic publication always use the explicit
@@ -238,7 +369,10 @@ configured Git remote.
The default signing key is
`$HOME/.config/govoplan/release-keys/release-key-1.pem` when no signing key is
entered in the UI.
entered in the UI. Signing-key files must be regular, owned by the operator, and
inaccessible to group/other users. The browser sends a configured key path only
on the initial execution request; it never persists signing material in session
storage, and request-ID recovery replays no key path.
Generate a selective release plan from the terminal:
@@ -259,18 +393,28 @@ Build a signed selective catalog candidate:
```sh
KEY_DIR="$HOME/.config/govoplan/release-keys"
ARTIFACT_DIR="$(mktemp -d)"
../govoplan-files/.venv/bin/python -m pip wheel \
--no-deps \
--no-build-isolation \
--wheel-dir "$ARTIFACT_DIR" \
../govoplan-files
./.venv/bin/python tools/release/release-catalog.py selective \
--repo-version govoplan-files=0.1.9 \
--python-artifact \
"govoplan-files=$ARTIFACT_DIR/govoplan_files-0.1.9-py3-none-any.whl" \
--channel stable \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem"
```
This writes candidate catalog/keyring files below `runtime/release-candidates/`,
generates the browsable module directory below `modules/`, validates the signed
candidate with the module installer validator, and reports whether the candidate
catalog/keyring match the currently published channel. It does not publish to
the website repository.
This writes candidate catalog/keyring files below
`$XDG_STATE_HOME/govoplan/release-candidates/` (or
`~/.local/state/govoplan/release-candidates/`), generates the browsable module
directory below `modules/`, validates the signed candidate with the module
installer validator, and reports whether the candidate catalog/keyring match the
currently published channel. It does not publish to the website repository.
Regenerate the browsable module directory from an existing catalog/keyring:
@@ -285,8 +429,10 @@ Regenerate the browsable module directory from an existing catalog/keyring:
Preview publication of a reviewed candidate:
```sh
CANDIDATE_ROOT="${XDG_STATE_HOME:-$HOME/.local/state}/govoplan/release-candidates"
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir runtime/release-candidates/stable-YYYYMMDD-HHMMSS \
--candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable
```
@@ -294,7 +440,7 @@ Apply the reviewed candidate into the website repository without pushing:
```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir runtime/release-candidates/stable-YYYYMMDD-HHMMSS \
--candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable \
--apply \
--commit \
@@ -305,7 +451,7 @@ Push is a separate explicit flag:
```sh
./.venv/bin/python tools/release/release-catalog.py publish-candidate \
--candidate-dir runtime/release-candidates/stable-YYYYMMDD-HHMMSS \
--candidate-dir "$CANDIDATE_ROOT/stable-YYYYMMDD-HHMMSS" \
--channel stable \
--apply \
--commit \
@@ -313,6 +459,13 @@ Push is a separate explicit flag:
--push
```
Publication validates the same in-memory catalog object that it writes; it does
not copy a path that can change after validation. On commit, the console reads
the catalog, keyring, and complete module directory back from the immutable Git
tree and requires byte-for-byte equality with those validated objects. Tags and
remote branch updates then reference that exact commit SHA rather than the
mutable worktree `HEAD`.
Published channels are expected below the public catalog base URL:
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`

View File

@@ -6,81 +6,86 @@ Generated from `repositories.json`. Use that JSON file as the machine-readable s
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `govoplan` | `meta` | `../govoplan` | [govoplan](https://git.add-ideas.de/add-ideas/govoplan) |
| `govoplan-core` | `kernel` | `../govoplan-core` | [govoplan-core](https://git.add-ideas.de/add-ideas/govoplan-core) |
| `govoplan` | `meta` | `../govoplan` | [govoplan](https://git.add-ideas.de/GovOPlaN/govoplan) |
| `govoplan-core` | `kernel` | `../govoplan-core` | [govoplan-core](https://git.add-ideas.de/GovOPlaN/govoplan-core) |
## Module
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `govoplan-access` | `platform` | `../govoplan-access` | [govoplan-access](https://git.add-ideas.de/add-ideas/govoplan-access) |
| `govoplan-addresses` | `domain` | `../govoplan-addresses` | [govoplan-addresses](https://git.add-ideas.de/add-ideas/govoplan-addresses) |
| `govoplan-admin` | `platform` | `../govoplan-admin` | [govoplan-admin](https://git.add-ideas.de/add-ideas/govoplan-admin) |
| `govoplan-appointments` | `domain` | `../govoplan-appointments` | [govoplan-appointments](https://git.add-ideas.de/add-ideas/govoplan-appointments) |
| `govoplan-approvals` | `domain` | `../govoplan-approvals` | [govoplan-approvals](https://git.add-ideas.de/add-ideas/govoplan-approvals) |
| `govoplan-assets` | `domain` | `../govoplan-assets` | [govoplan-assets](https://git.add-ideas.de/add-ideas/govoplan-assets) |
| `govoplan-audit` | `platform` | `../govoplan-audit` | [govoplan-audit](https://git.add-ideas.de/add-ideas/govoplan-audit) |
| `govoplan-booking` | `domain` | `../govoplan-booking` | [govoplan-booking](https://git.add-ideas.de/add-ideas/govoplan-booking) |
| `govoplan-calendar` | `domain` | `../govoplan-calendar` | [govoplan-calendar](https://git.add-ideas.de/add-ideas/govoplan-calendar) |
| `govoplan-campaign` | `domain` | `../govoplan-campaign` | [govoplan-campaign](https://git.add-ideas.de/add-ideas/govoplan-campaign) |
| `govoplan-cases` | `domain` | `../govoplan-cases` | [govoplan-cases](https://git.add-ideas.de/add-ideas/govoplan-cases) |
| `govoplan-certificates` | `domain` | `../govoplan-certificates` | [govoplan-certificates](https://git.add-ideas.de/add-ideas/govoplan-certificates) |
| `govoplan-committee` | `domain` | `../govoplan-committee` | [govoplan-committee](https://git.add-ideas.de/add-ideas/govoplan-committee) |
| `govoplan-consultation` | `domain` | `../govoplan-consultation` | [govoplan-consultation](https://git.add-ideas.de/add-ideas/govoplan-consultation) |
| `govoplan-contracts` | `domain` | `../govoplan-contracts` | [govoplan-contracts](https://git.add-ideas.de/add-ideas/govoplan-contracts) |
| `govoplan-dashboard` | `platform` | `../govoplan-dashboard` | [govoplan-dashboard](https://git.add-ideas.de/add-ideas/govoplan-dashboard) |
| `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/add-ideas/govoplan-dms) |
| `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/add-ideas/govoplan-dist-lists) |
| `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/add-ideas/govoplan-docs) |
| `govoplan-erp` | `domain` | `../govoplan-erp` | [govoplan-erp](https://git.add-ideas.de/add-ideas/govoplan-erp) |
| `govoplan-evaluation` | `domain` | `../govoplan-evaluation` | [govoplan-evaluation](https://git.add-ideas.de/add-ideas/govoplan-evaluation) |
| `govoplan-facilities` | `domain` | `../govoplan-facilities` | [govoplan-facilities](https://git.add-ideas.de/add-ideas/govoplan-facilities) |
| `govoplan-files` | `domain` | `../govoplan-files` | [govoplan-files](https://git.add-ideas.de/add-ideas/govoplan-files) |
| `govoplan-forms` | `domain` | `../govoplan-forms` | [govoplan-forms](https://git.add-ideas.de/add-ideas/govoplan-forms) |
| `govoplan-forms-runtime` | `platform` | `../govoplan-forms-runtime` | [govoplan-forms-runtime](https://git.add-ideas.de/add-ideas/govoplan-forms-runtime) |
| `govoplan-grants` | `domain` | `../govoplan-grants` | [govoplan-grants](https://git.add-ideas.de/add-ideas/govoplan-grants) |
| `govoplan-helpdesk` | `domain` | `../govoplan-helpdesk` | [govoplan-helpdesk](https://git.add-ideas.de/add-ideas/govoplan-helpdesk) |
| `govoplan-identity` | `platform` | `../govoplan-identity` | [govoplan-identity](https://git.add-ideas.de/add-ideas/govoplan-identity) |
| `govoplan-identity-trust` | `platform` | `../govoplan-identity-trust` | [govoplan-identity-trust](https://git.add-ideas.de/add-ideas/govoplan-identity-trust) |
| `govoplan-idm` | `platform` | `../govoplan-idm` | [govoplan-idm](https://git.add-ideas.de/add-ideas/govoplan-idm) |
| `govoplan-inspections` | `domain` | `../govoplan-inspections` | [govoplan-inspections](https://git.add-ideas.de/add-ideas/govoplan-inspections) |
| `govoplan-issue-reporting` | `domain` | `../govoplan-issue-reporting` | [govoplan-issue-reporting](https://git.add-ideas.de/add-ideas/govoplan-issue-reporting) |
| `govoplan-learning` | `domain` | `../govoplan-learning` | [govoplan-learning](https://git.add-ideas.de/add-ideas/govoplan-learning) |
| `govoplan-ledger` | `domain` | `../govoplan-ledger` | [govoplan-ledger](https://git.add-ideas.de/add-ideas/govoplan-ledger) |
| `govoplan-mail` | `domain` | `../govoplan-mail` | [govoplan-mail](https://git.add-ideas.de/add-ideas/govoplan-mail) |
| `govoplan-notifications` | `platform` | `../govoplan-notifications` | [govoplan-notifications](https://git.add-ideas.de/add-ideas/govoplan-notifications) |
| `govoplan-ops` | `platform` | `../govoplan-ops` | [govoplan-ops](https://git.add-ideas.de/add-ideas/govoplan-ops) |
| `govoplan-organizations` | `platform` | `../govoplan-organizations` | [govoplan-organizations](https://git.add-ideas.de/add-ideas/govoplan-organizations) |
| `govoplan-payments` | `domain` | `../govoplan-payments` | [govoplan-payments](https://git.add-ideas.de/add-ideas/govoplan-payments) |
| `govoplan-permits` | `domain` | `../govoplan-permits` | [govoplan-permits](https://git.add-ideas.de/add-ideas/govoplan-permits) |
| `govoplan-policy` | `platform` | `../govoplan-policy` | [govoplan-policy](https://git.add-ideas.de/add-ideas/govoplan-policy) |
| `govoplan-poll` | `domain` | `../govoplan-poll` | [govoplan-poll](https://git.add-ideas.de/add-ideas/govoplan-poll) |
| `govoplan-portal` | `domain` | `../govoplan-portal` | [govoplan-portal](https://git.add-ideas.de/add-ideas/govoplan-portal) |
| `govoplan-postbox` | `domain` | `../govoplan-postbox` | [govoplan-postbox](https://git.add-ideas.de/add-ideas/govoplan-postbox) |
| `govoplan-procurement` | `domain` | `../govoplan-procurement` | [govoplan-procurement](https://git.add-ideas.de/add-ideas/govoplan-procurement) |
| `govoplan-records` | `domain` | `../govoplan-records` | [govoplan-records](https://git.add-ideas.de/add-ideas/govoplan-records) |
| `govoplan-reporting` | `domain` | `../govoplan-reporting` | [govoplan-reporting](https://git.add-ideas.de/add-ideas/govoplan-reporting) |
| `govoplan-resources` | `domain` | `../govoplan-resources` | [govoplan-resources](https://git.add-ideas.de/add-ideas/govoplan-resources) |
| `govoplan-risk-compliance` | `domain` | `../govoplan-risk-compliance` | [govoplan-risk-compliance](https://git.add-ideas.de/add-ideas/govoplan-risk-compliance) |
| `govoplan-scheduling` | `domain` | `../govoplan-scheduling` | [govoplan-scheduling](https://git.add-ideas.de/add-ideas/govoplan-scheduling) |
| `govoplan-search` | `platform` | `../govoplan-search` | [govoplan-search](https://git.add-ideas.de/add-ideas/govoplan-search) |
| `govoplan-tasks` | `domain` | `../govoplan-tasks` | [govoplan-tasks](https://git.add-ideas.de/add-ideas/govoplan-tasks) |
| `govoplan-templates` | `domain` | `../govoplan-templates` | [govoplan-templates](https://git.add-ideas.de/add-ideas/govoplan-templates) |
| `govoplan-tenancy` | `platform` | `../govoplan-tenancy` | [govoplan-tenancy](https://git.add-ideas.de/add-ideas/govoplan-tenancy) |
| `govoplan-transparency` | `domain` | `../govoplan-transparency` | [govoplan-transparency](https://git.add-ideas.de/add-ideas/govoplan-transparency) |
| `govoplan-workflow` | `platform` | `../govoplan-workflow` | [govoplan-workflow](https://git.add-ideas.de/add-ideas/govoplan-workflow) |
| `govoplan-access` | `platform` | `../govoplan-access` | [govoplan-access](https://git.add-ideas.de/GovOPlaN/govoplan-access) |
| `govoplan-addresses` | `domain` | `../govoplan-addresses` | [govoplan-addresses](https://git.add-ideas.de/GovOPlaN/govoplan-addresses) |
| `govoplan-admin` | `platform` | `../govoplan-admin` | [govoplan-admin](https://git.add-ideas.de/GovOPlaN/govoplan-admin) |
| `govoplan-appointments` | `domain` | `../govoplan-appointments` | [govoplan-appointments](https://git.add-ideas.de/GovOPlaN/govoplan-appointments) |
| `govoplan-approvals` | `domain` | `../govoplan-approvals` | [govoplan-approvals](https://git.add-ideas.de/GovOPlaN/govoplan-approvals) |
| `govoplan-assets` | `domain` | `../govoplan-assets` | [govoplan-assets](https://git.add-ideas.de/GovOPlaN/govoplan-assets) |
| `govoplan-audit` | `platform` | `../govoplan-audit` | [govoplan-audit](https://git.add-ideas.de/GovOPlaN/govoplan-audit) |
| `govoplan-booking` | `domain` | `../govoplan-booking` | [govoplan-booking](https://git.add-ideas.de/GovOPlaN/govoplan-booking) |
| `govoplan-calendar` | `domain` | `../govoplan-calendar` | [govoplan-calendar](https://git.add-ideas.de/GovOPlaN/govoplan-calendar) |
| `govoplan-campaign` | `domain` | `../govoplan-campaign` | [govoplan-campaign](https://git.add-ideas.de/GovOPlaN/govoplan-campaign) |
| `govoplan-cases` | `domain` | `../govoplan-cases` | [govoplan-cases](https://git.add-ideas.de/GovOPlaN/govoplan-cases) |
| `govoplan-certificates` | `domain` | `../govoplan-certificates` | [govoplan-certificates](https://git.add-ideas.de/GovOPlaN/govoplan-certificates) |
| `govoplan-committee` | `domain` | `../govoplan-committee` | [govoplan-committee](https://git.add-ideas.de/GovOPlaN/govoplan-committee) |
| `govoplan-consultation` | `domain` | `../govoplan-consultation` | [govoplan-consultation](https://git.add-ideas.de/GovOPlaN/govoplan-consultation) |
| `govoplan-contracts` | `domain` | `../govoplan-contracts` | [govoplan-contracts](https://git.add-ideas.de/GovOPlaN/govoplan-contracts) |
| `govoplan-dashboard` | `platform` | `../govoplan-dashboard` | [govoplan-dashboard](https://git.add-ideas.de/GovOPlaN/govoplan-dashboard) |
| `govoplan-dataflow` | `platform` | `../govoplan-dataflow` | [govoplan-dataflow](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow) |
| `govoplan-datasources` | `platform` | `../govoplan-datasources` | [govoplan-datasources](https://git.add-ideas.de/GovOPlaN/govoplan-datasources) |
| `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/GovOPlaN/govoplan-dms) |
| `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists) |
| `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/GovOPlaN/govoplan-docs) |
| `govoplan-erp` | `domain` | `../govoplan-erp` | [govoplan-erp](https://git.add-ideas.de/GovOPlaN/govoplan-erp) |
| `govoplan-evaluation` | `domain` | `../govoplan-evaluation` | [govoplan-evaluation](https://git.add-ideas.de/GovOPlaN/govoplan-evaluation) |
| `govoplan-facilities` | `domain` | `../govoplan-facilities` | [govoplan-facilities](https://git.add-ideas.de/GovOPlaN/govoplan-facilities) |
| `govoplan-files` | `domain` | `../govoplan-files` | [govoplan-files](https://git.add-ideas.de/GovOPlaN/govoplan-files) |
| `govoplan-forms` | `domain` | `../govoplan-forms` | [govoplan-forms](https://git.add-ideas.de/GovOPlaN/govoplan-forms) |
| `govoplan-forms-runtime` | `platform` | `../govoplan-forms-runtime` | [govoplan-forms-runtime](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime) |
| `govoplan-grants` | `domain` | `../govoplan-grants` | [govoplan-grants](https://git.add-ideas.de/GovOPlaN/govoplan-grants) |
| `govoplan-helpdesk` | `domain` | `../govoplan-helpdesk` | [govoplan-helpdesk](https://git.add-ideas.de/GovOPlaN/govoplan-helpdesk) |
| `govoplan-identity` | `platform` | `../govoplan-identity` | [govoplan-identity](https://git.add-ideas.de/GovOPlaN/govoplan-identity) |
| `govoplan-identity-trust` | `platform` | `../govoplan-identity-trust` | [govoplan-identity-trust](https://git.add-ideas.de/GovOPlaN/govoplan-identity-trust) |
| `govoplan-idm` | `platform` | `../govoplan-idm` | [govoplan-idm](https://git.add-ideas.de/GovOPlaN/govoplan-idm) |
| `govoplan-inspections` | `domain` | `../govoplan-inspections` | [govoplan-inspections](https://git.add-ideas.de/GovOPlaN/govoplan-inspections) |
| `govoplan-learning` | `domain` | `../govoplan-learning` | [govoplan-learning](https://git.add-ideas.de/GovOPlaN/govoplan-learning) |
| `govoplan-ledger` | `domain` | `../govoplan-ledger` | [govoplan-ledger](https://git.add-ideas.de/GovOPlaN/govoplan-ledger) |
| `govoplan-mail` | `domain` | `../govoplan-mail` | [govoplan-mail](https://git.add-ideas.de/GovOPlaN/govoplan-mail) |
| `govoplan-notifications` | `platform` | `../govoplan-notifications` | [govoplan-notifications](https://git.add-ideas.de/GovOPlaN/govoplan-notifications) |
| `govoplan-ops` | `platform` | `../govoplan-ops` | [govoplan-ops](https://git.add-ideas.de/GovOPlaN/govoplan-ops) |
| `govoplan-organizations` | `platform` | `../govoplan-organizations` | [govoplan-organizations](https://git.add-ideas.de/GovOPlaN/govoplan-organizations) |
| `govoplan-payments` | `domain` | `../govoplan-payments` | [govoplan-payments](https://git.add-ideas.de/GovOPlaN/govoplan-payments) |
| `govoplan-permits` | `domain` | `../govoplan-permits` | [govoplan-permits](https://git.add-ideas.de/GovOPlaN/govoplan-permits) |
| `govoplan-policy` | `platform` | `../govoplan-policy` | [govoplan-policy](https://git.add-ideas.de/GovOPlaN/govoplan-policy) |
| `govoplan-poll` | `domain` | `../govoplan-poll` | [govoplan-poll](https://git.add-ideas.de/GovOPlaN/govoplan-poll) |
| `govoplan-portal` | `domain` | `../govoplan-portal` | [govoplan-portal](https://git.add-ideas.de/GovOPlaN/govoplan-portal) |
| `govoplan-postbox` | `domain` | `../govoplan-postbox` | [govoplan-postbox](https://git.add-ideas.de/GovOPlaN/govoplan-postbox) |
| `govoplan-procurement` | `domain` | `../govoplan-procurement` | [govoplan-procurement](https://git.add-ideas.de/GovOPlaN/govoplan-procurement) |
| `govoplan-projects` | `domain` | `../govoplan-projects` | [govoplan-projects](https://git.add-ideas.de/GovOPlaN/govoplan-projects) |
| `govoplan-records` | `domain` | `../govoplan-records` | [govoplan-records](https://git.add-ideas.de/GovOPlaN/govoplan-records) |
| `govoplan-reporting` | `domain` | `../govoplan-reporting` | [govoplan-reporting](https://git.add-ideas.de/GovOPlaN/govoplan-reporting) |
| `govoplan-resources` | `domain` | `../govoplan-resources` | [govoplan-resources](https://git.add-ideas.de/GovOPlaN/govoplan-resources) |
| `govoplan-risk-compliance` | `domain` | `../govoplan-risk-compliance` | [govoplan-risk-compliance](https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance) |
| `govoplan-scheduling` | `domain` | `../govoplan-scheduling` | [govoplan-scheduling](https://git.add-ideas.de/GovOPlaN/govoplan-scheduling) |
| `govoplan-search` | `platform` | `../govoplan-search` | [govoplan-search](https://git.add-ideas.de/GovOPlaN/govoplan-search) |
| `govoplan-tasks` | `domain` | `../govoplan-tasks` | [govoplan-tasks](https://git.add-ideas.de/GovOPlaN/govoplan-tasks) |
| `govoplan-templates` | `domain` | `../govoplan-templates` | [govoplan-templates](https://git.add-ideas.de/GovOPlaN/govoplan-templates) |
| `govoplan-tenancy` | `platform` | `../govoplan-tenancy` | [govoplan-tenancy](https://git.add-ideas.de/GovOPlaN/govoplan-tenancy) |
| `govoplan-tickets` | `domain` | `../govoplan-tickets` | [govoplan-tickets](https://git.add-ideas.de/GovOPlaN/govoplan-tickets) |
| `govoplan-transparency` | `domain` | `../govoplan-transparency` | [govoplan-transparency](https://git.add-ideas.de/GovOPlaN/govoplan-transparency) |
| `govoplan-views` | `platform` | `../govoplan-views` | [govoplan-views](https://git.add-ideas.de/GovOPlaN/govoplan-views) |
| `govoplan-wiki` | `domain` | `../govoplan-wiki` | [govoplan-wiki](https://git.add-ideas.de/GovOPlaN/govoplan-wiki) |
| `govoplan-workflow` | `platform` | `../govoplan-workflow` | [govoplan-workflow](https://git.add-ideas.de/GovOPlaN/govoplan-workflow) |
## Connector
| Repository | Subtype | Local path | Gitea |
| --- | --- | --- | --- |
| `govoplan-connectors` | `connector-hub` | `../govoplan-connectors` | [govoplan-connectors](https://git.add-ideas.de/add-ideas/govoplan-connectors) |
| `govoplan-fit-connect` | `standard` | `../govoplan-fit-connect` | [govoplan-fit-connect](https://git.add-ideas.de/add-ideas/govoplan-fit-connect) |
| `govoplan-rest` | `protocol` | `../govoplan-rest` | [govoplan-rest](https://git.add-ideas.de/add-ideas/govoplan-rest) |
| `govoplan-soap` | `protocol` | `../govoplan-soap` | [govoplan-soap](https://git.add-ideas.de/add-ideas/govoplan-soap) |
| `govoplan-xoev` | `standard` | `../govoplan-xoev` | [govoplan-xoev](https://git.add-ideas.de/add-ideas/govoplan-xoev) |
| `govoplan-xrechnung` | `standard` | `../govoplan-xrechnung` | [govoplan-xrechnung](https://git.add-ideas.de/add-ideas/govoplan-xrechnung) |
| `govoplan-xta-osci` | `standard` | `../govoplan-xta-osci` | [govoplan-xta-osci](https://git.add-ideas.de/add-ideas/govoplan-xta-osci) |
| `govoplan-connectors` | `connector-hub` | `../govoplan-connectors` | [govoplan-connectors](https://git.add-ideas.de/GovOPlaN/govoplan-connectors) |
| `govoplan-fit-connect` | `standard` | `../govoplan-fit-connect` | [govoplan-fit-connect](https://git.add-ideas.de/GovOPlaN/govoplan-fit-connect) |
| `govoplan-rest` | `protocol` | `../govoplan-rest` | [govoplan-rest](https://git.add-ideas.de/GovOPlaN/govoplan-rest) |
| `govoplan-soap` | `protocol` | `../govoplan-soap` | [govoplan-soap](https://git.add-ideas.de/GovOPlaN/govoplan-soap) |
| `govoplan-xoev` | `standard` | `../govoplan-xoev` | [govoplan-xoev](https://git.add-ideas.de/GovOPlaN/govoplan-xoev) |
| `govoplan-xrechnung` | `standard` | `../govoplan-xrechnung` | [govoplan-xrechnung](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung) |
| `govoplan-xta-osci` | `standard` | `../govoplan-xta-osci` | [govoplan-xta-osci](https://git.add-ideas.de/GovOPlaN/govoplan-xta-osci) |
## Website

View File

@@ -125,7 +125,7 @@ tools/checks/security-audit/run.sh --mode ci --scope current --strict
## Audit Burndown Workflow
Treat Gitea issues as the active audit state. A full GovOPlaN audit should
produce one tracker issue in `add-ideas/govoplan` and child issues in the
produce one tracker issue in `GovOPlaN/govoplan` and child issues in the
repository that owns each fix.
Use the tracker issue for:
@@ -162,7 +162,14 @@ clusters that cross module ownership or make behavior harder to change safely.
The regular `Security Audit` workflow reuses the fingerprinted toolbox image
when the Docker daemon is persistent, which is the normal case for the
self-hosted Gitea runner using the host Docker socket. The separate
self-hosted Gitea runner using the host Docker socket. Trusted push, schedule,
and manual runs scan all registered repositories; authenticated SSH is used
only for the private website repository. The wrapper inspects the Actions job
mount table and forwards only the narrowest writable mount covering the audit
scope; it never inherits the job's Docker socket or unrelated runner mounts.
Pull-request audit runs stay disabled while the audit runner exposes its host
Docker socket: PR-controlled audit code must run on a disposable or rootless
runner without host-socket access. The separate
`Security Audit Toolbox Update` workflow runs weekly with
`SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the
allowed tool version ranges into a refreshed local image.

View File

@@ -19,7 +19,7 @@ This is a product-level story owned by the GovOPlaN platform rather than by an
individual domain module. It joins installation, module lifecycle, operations,
configuration packages, and release provenance into one administrator journey.
The canonical backlog item is
[GovOPlaN #13](https://git.add-ideas.de/add-ideas/govoplan/issues/13).
[GovOPlaN #13](https://git.add-ideas.de/GovOPlaN/govoplan/issues/13).
## Terms

137
docs/VIEWS_ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,137 @@
# GovOPlaN Views Architecture
## Purpose
GovOPlaN Views are governed presentation projections for a task,
responsibility, or workflow step. A View can reduce the visible modules,
navigation entries, routes, page sections, and commands to the interface
needed for the current job.
Views are optional. If `govoplan-views` is not installed or enabled, the normal
permission-derived interface remains unchanged.
## Security Boundary
A View is not an authorization mechanism.
- Access, tenant isolation, resource guards, and backend permission checks
remain authoritative.
- A View may hide an interface surface that the actor is otherwise allowed to
use.
- A View can never expose a route, action, tenant, or resource that normal
authorization denies.
- An authorized deep link outside the current View should offer an explicit
temporary escape or View switch. It must not be presented as a permission
denial.
This boundary lets Views improve focus without creating a second, weaker RBAC
system.
## Ownership
Core owns the versioned, module-neutral surface contract and WebUI runtime
hooks. Modules declare stable surfaces and use shared hooks to respect the
effective projection. Modules do not import `govoplan-views`.
`govoplan-views` owns:
- draft and immutable published View revisions
- system, tenant, group, and user assignments
- default, mandatory, and user-selectable Views
- active per-user View state
- effective projection resolution and provenance
- the View editor, preview, validation, and stale-surface diagnostics
Policy optionally owns inherited ceilings and explainable decisions. Workflow
optionally references a pinned View revision for an instance or step and may
narrow it further.
## Surface Contract
Modules announce only useful, semantic surfaces:
- module
- navigation item
- route or workspace
- section or panel
- command or action
Each descriptor has a stable namespaced id, parent id, kind, label, default
visibility, ordering, and dependency metadata where needed. Surface ids are
public module contracts, not CSS selectors, component paths, or arbitrary DOM
fragments.
The first release supports visible or hidden. Read-only states, layout
replacement, visual emphasis, and arbitrary styling are separate concerns and
are deferred.
## Effective Resolution
The effective interface is the intersection of:
1. installed and enabled modules
2. actor permissions and resource access
3. administrator and Policy ceilings
4. an assigned or user-selected View
5. an optional workflow instance or step overlay
Lower scopes and workflow overlays may narrow inherited visibility but cannot
broaden it. Every inherited, locked, hidden, unavailable, or stale choice
should carry provenance that the editor and runtime can explain.
Published View revisions are immutable. Active workflow instances pin the
revision they use. Unknown or retired surface ids produce diagnostics rather
than breaking startup. If no valid effective View can be resolved, the system
uses the last valid projection or the normal authorized interface and reports
the configuration problem to administrators.
## Workflow Behavior
A workflow definition may reference a View for the whole instance or a
particular step. Starting, resuming, or advancing the workflow activates the
appropriate projection. Users can intentionally leave focused mode and return
from an open-work widget or notification without losing workflow state.
Module handoffs carry the workflow and View context through Core contracts.
Workflow does not import the target module or the Views implementation.
## Delivery Order
1. Define the Core surface registry and runtime hooks.
2. Initialize `govoplan-views` and persist versioned definitions.
3. Add assignment, selection, resolution, provenance, and the editor.
4. Add Policy inheritance and administrator ceilings.
5. Add Workflow instance and step activation.
6. Adopt semantic section/action descriptors module by module.
## Implementation Status
Implemented in the initial Views slice:
- Core contract version `1`, stable module/navigation/route identifiers, custom
section/action descriptors, manifest validation, and platform API metadata
- shell navigation, route-boundary, settings, administration, dashboard-widget,
embedded-capability, and organization-action filtering
- `govoplan-views` definitions, immutable revisions, system/tenant/group/user
assignments, user selection, provenance, and stale-surface recovery
- a system and tenant administration editor with unsaved-change protection,
publish/archive controls, assignment management, and server-enforced lockout
prevention
- surface declarations for every currently installed module that contributes a
WebUI, including finer-grained shared administration and settings surfaces
Still intentionally separate:
- Policy-owned inherited ceilings and policy decision provenance
- workflow-instance and workflow-step activation of pinned View revisions
- read-only and layout-replacement projections beyond the version `1`
visible/hidden contract
## Gitea Work Packages
- `govoplan#17`: task-focused Views user story
- `govoplan#16`: initialize and implement `govoplan-views`
- `govoplan-core#271`: versioned surface and runtime contracts
- `govoplan-policy#9`: inheritance, ceilings, and provenance
- `govoplan-workflow#7`: workflow instance and step activation
- `govoplan-workflow#3`: focused workflow mode user story

View File

@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit-boundary-evidence.schema.json",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/capability-fit-boundary-evidence.schema.json",
"title": "GovOPlaN externally issued capability-fit boundary evidence",
"type": "object",
"additionalProperties": false,

View File

@@ -507,7 +507,7 @@
{
"kind": "issue",
"scope": "documented_model",
"locator": "https://git.add-ideas.de/add-ideas/govoplan/issues/12"
"locator": "https://git.add-ideas.de/GovOPlaN/govoplan/issues/12"
}
],
"conditions": [],
@@ -793,7 +793,7 @@
{
"kind": "issue",
"scope": "documented_model",
"locator": "https://git.add-ideas.de/add-ideas/govoplan-core/issues/29"
"locator": "https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/29"
}
],
"conditions": [],

View File

@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit-proof-authority-keyring.schema.json",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/capability-fit-proof-authority-keyring.schema.json",
"title": "GovOPlaN capability-fit proof authority keyring",
"type": "object",
"additionalProperties": false,

View File

@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/capability-fit.schema.json",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/capability-fit.schema.json",
"title": "GovOPlaN capability and infrastructure fit assessment",
"type": "object",
"additionalProperties": false,

View File

@@ -149,12 +149,36 @@
"description": "GovOPlaN core runner, shared primitives, shell, or extension points.",
"exclusive": false
},
{
"name": "module/dashboard",
"color": "1d76db",
"description": "GovOPlaN Dashboard module behavior or integration.",
"exclusive": false
},
{
"name": "module/dataflow",
"color": "1d76db",
"description": "GovOPlaN Dataflow module behavior or integration.",
"exclusive": false
},
{
"name": "module/datasources",
"color": "006b75",
"description": "GovOPlaN governed datasource contracts, catalogs, and integrations.",
"exclusive": false
},
{
"name": "module/dms",
"color": "c5def5",
"description": "GovOPlaN Dms module behavior or integration.",
"exclusive": false
},
{
"name": "module/docs",
"color": "c5def5",
"description": "GovOPlaN Docs module behavior or integration.",
"exclusive": false
},
{
"name": "module/dist-lists",
"color": "0e8a16",
@@ -191,6 +215,12 @@
"description": "GovOPlaN Forms module behavior or integration.",
"exclusive": false
},
{
"name": "module/helpdesk",
"color": "c2e0c6",
"description": "GovOPlaN Helpdesk module behavior or integration.",
"exclusive": false
},
{
"name": "module/identity-trust",
"color": "d4c5f9",
@@ -275,12 +305,24 @@
"description": "GovOPlaN Postbox module behavior or integration.",
"exclusive": false
},
{
"name": "module/projects",
"color": "5319e7",
"description": "GovOPlaN Projects module behavior or integration.",
"exclusive": false
},
{
"name": "module/reporting",
"color": "c2e0c6",
"description": "GovOPlaN Reporting module behavior or integration.",
"exclusive": false
},
{
"name": "module/risk-compliance",
"color": "b60205",
"description": "GovOPlaN Risk Compliance module behavior or integration.",
"exclusive": false
},
{
"name": "module/search",
"color": "bfdadc",
@@ -311,6 +353,24 @@
"description": "GovOPlaN Tenancy module behavior or integration.",
"exclusive": false
},
{
"name": "module/tickets",
"color": "0e8a16",
"description": "GovOPlaN Tickets module behavior or integration.",
"exclusive": false
},
{
"name": "module/views",
"color": "c5def5",
"description": "GovOPlaN governed task views, interface projections, and workflow view integration.",
"exclusive": false
},
{
"name": "module/wiki",
"color": "006b75",
"description": "GovOPlaN Wiki module behavior or integration.",
"exclusive": false
},
{
"name": "module/workflow",
"color": "f9d0c4",

View File

@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/add-ideas/govoplan/src/branch/main/docs/installed-composition-evidence.schema.json",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/installed-composition-evidence.schema.json",
"title": "GovOPlaN installed composition evidence",
"type": "object",
"additionalProperties": false,
@@ -20,7 +20,7 @@
"format": "uri-reference"
},
"schema_version": {
"const": "0.3.0"
"const": "0.4.0"
},
"evidence_kind": {
"const": "govoplan.installed-composition"
@@ -182,7 +182,9 @@
"generated_unhashed_file_count",
"unverifiable_file_count",
"missing_file_count",
"mismatched_file_count"
"mismatched_file_count",
"artifact_payload_identity",
"installed_payload_identity"
],
"properties": {
"status": {
@@ -211,6 +213,18 @@
},
"mismatched_file_count": {
"$ref": "#/$defs/count"
},
"artifact_payload_identity": {
"oneOf": [
{ "$ref": "#/$defs/artifact_payload_identity" },
{ "type": "null" }
]
},
"installed_payload_identity": {
"oneOf": [
{ "$ref": "#/$defs/installed_payload_identity" },
{ "type": "null" }
]
}
},
"allOf": [
@@ -222,7 +236,9 @@
"permitted_unhashed_file_count": { "minimum": 1 },
"unverifiable_file_count": { "const": 0 },
"missing_file_count": { "const": 0 },
"mismatched_file_count": { "const": 0 }
"mismatched_file_count": { "const": 0 },
"artifact_payload_identity": { "$ref": "#/$defs/artifact_payload_identity" },
"installed_payload_identity": { "$ref": "#/$defs/installed_payload_identity" }
}
}
},
@@ -278,6 +294,34 @@
}
}
},
"artifact_payload_identity": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-wheel-declared-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
},
"installed_payload_identity": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-installed-record-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
},
"count": {
"type": "integer",
"minimum": 0,

View File

@@ -0,0 +1,53 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/installer-receipt-authority-keyring.schema.json",
"title": "GovOPlaN installer receipt authority keyring",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "purpose", "keys"],
"properties": {
"$schema": { "type": "string", "format": "uri-reference" },
"schema_version": { "const": "0.1.0" },
"purpose": { "const": "govoplan.installer-receipt-authorities" },
"keys": {
"type": "array",
"minItems": 1,
"maxItems": 64,
"items": { "$ref": "#/$defs/key" }
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"key": {
"type": "object",
"additionalProperties": false,
"required": ["key_id", "status", "public_key", "allowed_scopes"],
"properties": {
"key_id": { "$ref": "#/$defs/opaque_id" },
"status": {
"enum": ["active", "next", "revoked", "disabled", "retired"]
},
"public_key": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
},
"allowed_scopes": {
"type": "array",
"minItems": 1,
"maxItems": 1,
"uniqueItems": true,
"items": { "const": "installed_release_origin" }
},
"not_before": { "type": "string", "format": "date-time" },
"not_after": { "type": "string", "format": "date-time" }
}
}
}
}

View File

@@ -0,0 +1,122 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/installer-receipt.schema.json",
"title": "GovOPlaN signed installer receipt",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"evidence_kind",
"receipt_id",
"assessment_id",
"assessment_release",
"installed_evidence_sha256",
"catalog",
"issued_at",
"artifacts",
"signatures"
],
"properties": {
"$schema": { "type": "string", "format": "uri-reference" },
"schema_version": { "const": "0.1.0" },
"evidence_kind": { "const": "govoplan.installer-receipt" },
"receipt_id": { "$ref": "#/$defs/opaque_id" },
"assessment_id": { "$ref": "#/$defs/opaque_id" },
"assessment_release": { "$ref": "#/$defs/opaque_id" },
"installed_evidence_sha256": { "$ref": "#/$defs/sha256" },
"catalog": {
"type": "object",
"additionalProperties": false,
"required": ["channel", "sequence", "sha256"],
"properties": {
"channel": {
"type": "string",
"pattern": "^[a-z][a-z0-9_-]{0,63}$"
},
"sequence": { "type": "integer", "minimum": 1 },
"sha256": { "$ref": "#/$defs/sha256" }
}
},
"issued_at": { "type": "string", "format": "date-time" },
"artifacts": {
"type": "array",
"minItems": 1,
"maxItems": 256,
"items": { "$ref": "#/$defs/artifact" }
},
"signatures": {
"type": "array",
"minItems": 1,
"maxItems": 16,
"items": { "$ref": "#/$defs/signature" }
}
},
"$defs": {
"opaque_id": {
"type": "string",
"minLength": 1,
"maxLength": 160,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"package_name": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
},
"version": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._+!-]*$"
},
"artifact": {
"type": "object",
"additionalProperties": false,
"required": [
"package_name",
"package_version",
"catalog_archive_sha256",
"installed_payload"
],
"properties": {
"package_name": { "$ref": "#/$defs/package_name" },
"package_version": { "$ref": "#/$defs/version" },
"catalog_archive_sha256": { "$ref": "#/$defs/sha256" },
"installed_payload": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "sha256", "file_count"],
"properties": {
"algorithm": { "const": "govoplan-installed-record-payload-v1" },
"sha256": { "$ref": "#/$defs/sha256" },
"file_count": {
"type": "integer",
"minimum": 1,
"maximum": 10000
}
}
}
}
},
"signature": {
"type": "object",
"additionalProperties": false,
"required": ["algorithm", "key_id", "value"],
"properties": {
"algorithm": { "const": "ed25519" },
"key_id": { "$ref": "#/$defs/opaque_id" },
"value": {
"type": "string",
"minLength": 1,
"maxLength": 256,
"contentEncoding": "base64"
}
}
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}

View File

@@ -1,73 +1,78 @@
{
"version": 1,
"organization": "add-ideas",
"organization": "GovOPlaN",
"default_parent": "/mnt/DATA/git",
"repositories": [
{"name": "govoplan", "category": "system", "subtype": "meta", "remote": "git@git.add-ideas.de:add-ideas/govoplan.git", "path": "govoplan"},
{"name": "govoplan-core", "category": "system", "subtype": "kernel", "remote": "git@git.add-ideas.de:add-ideas/govoplan-core.git", "path": "govoplan-core"},
{"name": "govoplan-access", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-access.git", "path": "govoplan-access"},
{"name": "govoplan-addresses", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-addresses.git", "path": "govoplan-addresses"},
{"name": "govoplan-admin", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-admin.git", "path": "govoplan-admin"},
{"name": "govoplan-appointments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-appointments.git", "path": "govoplan-appointments"},
{"name": "govoplan-approvals", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-approvals.git", "path": "govoplan-approvals"},
{"name": "govoplan-assets", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-assets.git", "path": "govoplan-assets"},
{"name": "govoplan-audit", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-audit.git", "path": "govoplan-audit"},
{"name": "govoplan-booking", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-booking.git", "path": "govoplan-booking"},
{"name": "govoplan-calendar", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-calendar.git", "path": "govoplan-calendar"},
{"name": "govoplan-campaign", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-campaign.git", "path": "govoplan-campaign"},
{"name": "govoplan-cases", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-cases.git", "path": "govoplan-cases"},
{"name": "govoplan-certificates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-certificates.git", "path": "govoplan-certificates"},
{"name": "govoplan-committee", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-committee.git", "path": "govoplan-committee"},
{"name": "govoplan-connectors", "category": "connector", "subtype": "connector-hub", "remote": "git@git.add-ideas.de:add-ideas/govoplan-connectors.git", "path": "govoplan-connectors"},
{"name": "govoplan-consultation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-consultation.git", "path": "govoplan-consultation"},
{"name": "govoplan-contracts", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-contracts.git", "path": "govoplan-contracts"},
{"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dashboard.git", "path": "govoplan-dashboard"},
{"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dms.git", "path": "govoplan-dms"},
{"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-dist-lists.git", "path": "govoplan-dist-lists"},
{"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-docs.git", "path": "govoplan-docs"},
{"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-erp.git", "path": "govoplan-erp"},
{"name": "govoplan-evaluation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-evaluation.git", "path": "govoplan-evaluation"},
{"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-facilities.git", "path": "govoplan-facilities"},
{"name": "govoplan-files", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-files.git", "path": "govoplan-files"},
{"name": "govoplan-fit-connect", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-fit-connect.git", "path": "govoplan-fit-connect"},
{"name": "govoplan-forms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-forms.git", "path": "govoplan-forms"},
{"name": "govoplan-forms-runtime", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-forms-runtime.git", "path": "govoplan-forms-runtime"},
{"name": "govoplan-grants", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-grants.git", "path": "govoplan-grants"},
{"name": "govoplan-helpdesk", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-helpdesk.git", "path": "govoplan-helpdesk"},
{"name": "govoplan-identity", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-identity.git", "path": "govoplan-identity"},
{"name": "govoplan-identity-trust", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-identity-trust.git", "path": "govoplan-identity-trust"},
{"name": "govoplan-idm", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-idm.git", "path": "govoplan-idm"},
{"name": "govoplan-inspections", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-inspections.git", "path": "govoplan-inspections"},
{"name": "govoplan-issue-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-issue-reporting.git", "path": "govoplan-issue-reporting"},
{"name": "govoplan-learning", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-learning.git", "path": "govoplan-learning"},
{"name": "govoplan-ledger", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-ledger.git", "path": "govoplan-ledger"},
{"name": "govoplan-mail", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-mail.git", "path": "govoplan-mail"},
{"name": "govoplan-notifications", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-notifications.git", "path": "govoplan-notifications"},
{"name": "govoplan-ops", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-ops.git", "path": "govoplan-ops"},
{"name": "govoplan-organizations", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-organizations.git", "path": "govoplan-organizations"},
{"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-payments.git", "path": "govoplan-payments"},
{"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-permits.git", "path": "govoplan-permits"},
{"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-policy.git", "path": "govoplan-policy"},
{"name": "govoplan-poll", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-poll.git", "path": "govoplan-poll"},
{"name": "govoplan-portal", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-portal.git", "path": "govoplan-portal"},
{"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-postbox.git", "path": "govoplan-postbox"},
{"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-procurement.git", "path": "govoplan-procurement"},
{"name": "govoplan-records", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-records.git", "path": "govoplan-records"},
{"name": "govoplan-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-reporting.git", "path": "govoplan-reporting"},
{"name": "govoplan-resources", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-resources.git", "path": "govoplan-resources"},
{"name": "govoplan-rest", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:add-ideas/govoplan-rest.git", "path": "govoplan-rest"},
{"name": "govoplan-risk-compliance", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-risk-compliance.git", "path": "govoplan-risk-compliance"},
{"name": "govoplan-scheduling", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-scheduling.git", "path": "govoplan-scheduling"},
{"name": "govoplan-search", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-search.git", "path": "govoplan-search"},
{"name": "govoplan-soap", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:add-ideas/govoplan-soap.git", "path": "govoplan-soap"},
{"name": "govoplan-tasks", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-tasks.git", "path": "govoplan-tasks"},
{"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-templates.git", "path": "govoplan-templates"},
{"name": "govoplan-tenancy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-tenancy.git", "path": "govoplan-tenancy"},
{"name": "govoplan-transparency", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:add-ideas/govoplan-transparency.git", "path": "govoplan-transparency"},
{"name": "addideas-govoplan-website", "category": "website", "subtype": "public-site", "remote": "git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git", "path": "addideas-govoplan-website"},
{"name": "govoplan-workflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:add-ideas/govoplan-workflow.git", "path": "govoplan-workflow"},
{"name": "govoplan-xoev", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xoev.git", "path": "govoplan-xoev"},
{"name": "govoplan-xrechnung", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xrechnung.git", "path": "govoplan-xrechnung"},
{"name": "govoplan-xta-osci", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:add-ideas/govoplan-xta-osci.git", "path": "govoplan-xta-osci"}
{"name": "govoplan", "category": "system", "subtype": "meta", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan.git", "path": "govoplan"},
{"name": "govoplan-core", "category": "system", "subtype": "kernel", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-core.git", "path": "govoplan-core"},
{"name": "govoplan-access", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-access.git", "path": "govoplan-access"},
{"name": "govoplan-addresses", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-addresses.git", "path": "govoplan-addresses"},
{"name": "govoplan-admin", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-admin.git", "path": "govoplan-admin"},
{"name": "govoplan-appointments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-appointments.git", "path": "govoplan-appointments"},
{"name": "govoplan-approvals", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-approvals.git", "path": "govoplan-approvals"},
{"name": "govoplan-assets", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-assets.git", "path": "govoplan-assets"},
{"name": "govoplan-audit", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-audit.git", "path": "govoplan-audit"},
{"name": "govoplan-booking", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-booking.git", "path": "govoplan-booking"},
{"name": "govoplan-calendar", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-calendar.git", "path": "govoplan-calendar"},
{"name": "govoplan-campaign", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-campaign.git", "path": "govoplan-campaign"},
{"name": "govoplan-cases", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-cases.git", "path": "govoplan-cases"},
{"name": "govoplan-certificates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-certificates.git", "path": "govoplan-certificates"},
{"name": "govoplan-committee", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-committee.git", "path": "govoplan-committee"},
{"name": "govoplan-connectors", "category": "connector", "subtype": "connector-hub", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-connectors.git", "path": "govoplan-connectors"},
{"name": "govoplan-consultation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-consultation.git", "path": "govoplan-consultation"},
{"name": "govoplan-contracts", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-contracts.git", "path": "govoplan-contracts"},
{"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dashboard.git", "path": "govoplan-dashboard"},
{"name": "govoplan-dataflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dataflow.git", "path": "govoplan-dataflow"},
{"name": "govoplan-datasources", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-datasources.git", "path": "govoplan-datasources"},
{"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dms.git", "path": "govoplan-dms"},
{"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dist-lists.git", "path": "govoplan-dist-lists"},
{"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-docs.git", "path": "govoplan-docs"},
{"name": "govoplan-erp", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-erp.git", "path": "govoplan-erp"},
{"name": "govoplan-evaluation", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-evaluation.git", "path": "govoplan-evaluation"},
{"name": "govoplan-facilities", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-facilities.git", "path": "govoplan-facilities"},
{"name": "govoplan-files", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-files.git", "path": "govoplan-files"},
{"name": "govoplan-fit-connect", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-fit-connect.git", "path": "govoplan-fit-connect"},
{"name": "govoplan-forms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-forms.git", "path": "govoplan-forms"},
{"name": "govoplan-forms-runtime", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-forms-runtime.git", "path": "govoplan-forms-runtime"},
{"name": "govoplan-grants", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-grants.git", "path": "govoplan-grants"},
{"name": "govoplan-helpdesk", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-helpdesk.git", "path": "govoplan-helpdesk"},
{"name": "govoplan-identity", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-identity.git", "path": "govoplan-identity"},
{"name": "govoplan-identity-trust", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-identity-trust.git", "path": "govoplan-identity-trust"},
{"name": "govoplan-idm", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-idm.git", "path": "govoplan-idm"},
{"name": "govoplan-inspections", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-inspections.git", "path": "govoplan-inspections"},
{"name": "govoplan-learning", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-learning.git", "path": "govoplan-learning"},
{"name": "govoplan-ledger", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-ledger.git", "path": "govoplan-ledger"},
{"name": "govoplan-mail", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-mail.git", "path": "govoplan-mail"},
{"name": "govoplan-notifications", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-notifications.git", "path": "govoplan-notifications"},
{"name": "govoplan-ops", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-ops.git", "path": "govoplan-ops"},
{"name": "govoplan-organizations", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-organizations.git", "path": "govoplan-organizations"},
{"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-payments.git", "path": "govoplan-payments"},
{"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-permits.git", "path": "govoplan-permits"},
{"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-policy.git", "path": "govoplan-policy"},
{"name": "govoplan-poll", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-poll.git", "path": "govoplan-poll"},
{"name": "govoplan-portal", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-portal.git", "path": "govoplan-portal"},
{"name": "govoplan-postbox", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-postbox.git", "path": "govoplan-postbox"},
{"name": "govoplan-procurement", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-procurement.git", "path": "govoplan-procurement"},
{"name": "govoplan-projects", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-projects.git", "path": "govoplan-projects"},
{"name": "govoplan-records", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-records.git", "path": "govoplan-records"},
{"name": "govoplan-reporting", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-reporting.git", "path": "govoplan-reporting"},
{"name": "govoplan-resources", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-resources.git", "path": "govoplan-resources"},
{"name": "govoplan-rest", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-rest.git", "path": "govoplan-rest"},
{"name": "govoplan-risk-compliance", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-risk-compliance.git", "path": "govoplan-risk-compliance"},
{"name": "govoplan-scheduling", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-scheduling.git", "path": "govoplan-scheduling"},
{"name": "govoplan-search", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-search.git", "path": "govoplan-search"},
{"name": "govoplan-soap", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-soap.git", "path": "govoplan-soap"},
{"name": "govoplan-tasks", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tasks.git", "path": "govoplan-tasks"},
{"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-templates.git", "path": "govoplan-templates"},
{"name": "govoplan-tenancy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tenancy.git", "path": "govoplan-tenancy"},
{"name": "govoplan-tickets", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tickets.git", "path": "govoplan-tickets"},
{"name": "govoplan-transparency", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-transparency.git", "path": "govoplan-transparency"},
{"name": "govoplan-views", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-views.git", "path": "govoplan-views"},
{"name": "govoplan-wiki", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-wiki.git", "path": "govoplan-wiki"},
{"name": "addideas-govoplan-website", "category": "website", "subtype": "public-site", "remote": "git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git", "path": "addideas-govoplan-website", "bootstrap_transport": "registered"},
{"name": "govoplan-workflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-workflow.git", "path": "govoplan-workflow"},
{"name": "govoplan-xoev", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-xoev.git", "path": "govoplan-xoev"},
{"name": "govoplan-xrechnung", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-xrechnung.git", "path": "govoplan-xrechnung"},
{"name": "govoplan-xta-osci", "category": "connector", "subtype": "standard", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-xta-osci.git", "path": "govoplan-xta-osci"}
]
}

View File

@@ -19,6 +19,14 @@
-e ../govoplan-mail
-e ../govoplan-campaign
-e ../govoplan-calendar
-e ../govoplan-connectors
-e ../govoplan-datasources
-e ../govoplan-dataflow
-e ../govoplan-workflow
-e ../govoplan-views
-e ../govoplan-search
-e ../govoplan-risk-compliance
-e ../govoplan-postbox
-e ../govoplan-poll
-e ../govoplan-scheduling
-e ../govoplan-notifications
@@ -32,5 +40,7 @@ idna>=3.15
jsonschema>=4,<5
pip>=26.1.2
pip-audit>=2.9,<3
pytest>=9.0.3,<10
pygments>=2.20,<3
python-multipart>=0.0.31
ruff>=0.14,<1

View File

@@ -0,0 +1,5 @@
# Test-harness dependencies used against immutable release source tags.
# Keep these separate from requirements-release.txt so they are not part of the
# deployable product dependency set.
pytest>=9.0.3,<10
pygments>=2.20,<3

View File

@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published.
../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-dashboard.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.8
govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.8
govoplan-campaign @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-campaign.git@v0.1.10
govoplan-calendar @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-calendar.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-ops.git@v0.1.8
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.8
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.8
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.8
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.8
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.8
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.8
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.8
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.8
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.8
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.8
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.10
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.11
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.8
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.8
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.8

View File

@@ -37,6 +37,10 @@ from govoplan_assessment.evidence import ( # noqa: E402
review_installed_composition,
validate_payload,
)
from govoplan_assessment.installer_receipt import ( # noqa: E402
issue_installer_receipt,
)
from govoplan_release.artifact_identity import inspect_python_wheel # noqa: E402
from tests.test_capability_fit_review import signed_catalog # noqa: E402
@@ -89,6 +93,202 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
report["proof_scope"]["installed_artifacts"]["evidence_sha256"],
)
def test_signed_catalog_artifact_identities_bind_installed_release_origin(
self,
) -> None:
evidence = matching_installed_evidence(self.assessment)
catalog, keyring = signed_catalog(
self.assessment,
release_artifacts=catalog_artifacts_for(evidence),
)
report = self.review(
catalog=catalog,
keyring=keyring,
installed_evidence=evidence,
)
origin = report["proof_scope"]["installed_release_origin"]
self.assertEqual("current", report["status"])
self.assertTrue(origin["checked"])
self.assertTrue(origin["valid"])
self.assertTrue(origin["release_origin_bound"])
self.assertEqual(len(evidence["artifacts"]), origin["anchored_artifact_digest_count"])
self.assertEqual(0, origin["signed_installer_receipt_count"])
def test_role_scoped_installer_receipt_admits_transformed_artifacts_and_boundary(
self,
) -> None:
evidence = matching_installed_evidence(self.assessment)
artifact_rows = catalog_artifacts_for(evidence, requires_receipt=True)
catalog, keyring = signed_catalog(
self.assessment,
release_artifacts=artifact_rows,
)
receipt, installer_keys = signed_installer_receipt(
assessment=self.assessment,
installed=evidence,
catalog=catalog,
catalog_artifacts=artifact_rows,
)
proof, proof_keys = signed_boundary_evidence(
assessment=self.assessment,
installed=evidence,
claims=[boundary_claim("target_environment", "passed")],
allowed_scopes=["target_environment"],
)
report = self.review(
catalog=catalog,
keyring=keyring,
installed_evidence=evidence,
installer_receipt=receipt,
installer_authority_keyring=installer_keys,
boundary_evidence=proof,
authority_keyring=proof_keys,
installed_evidence_mode="imported_unsigned",
)
origin = report["proof_scope"]["installed_release_origin"]
self.assertTrue(origin["valid"])
self.assertEqual(0, origin["anchored_artifact_digest_count"])
self.assertEqual(len(evidence["artifacts"]), origin["signed_installer_receipt_count"])
self.assertTrue(report["proof_scope"]["target_environment"]["checked"])
self.assertTrue(report["proof_scope"]["target_environment"]["valid"])
self.assertTrue(
report["proof_scope"]["installed_evidence_observation"][
"receipt_authenticated"
]
)
self.assertNotIn(
"installed_evidence_unsigned_import",
{item["code"] for item in report["findings"]},
)
def test_installer_receipt_cannot_reuse_release_key_or_cover_forged_payload(
self,
) -> None:
evidence = matching_installed_evidence(self.assessment)
artifact_rows = catalog_artifacts_for(evidence, requires_receipt=True)
catalog, keyring = signed_catalog(
self.assessment,
release_artifacts=artifact_rows,
)
receipt, installer_keys = signed_installer_receipt(
assessment=self.assessment,
installed=evidence,
catalog=catalog,
catalog_artifacts=artifact_rows,
)
forged = deepcopy(evidence)
forged["artifacts"][0]["record_integrity"]["installed_payload_identity"][
"sha256"
] = "f" * 64
reused = deepcopy(installer_keys)
reused["keys"][0]["public_key"] = keyring["keys"][0]["public_key"]
forged_report = self.review(
catalog=catalog,
keyring=keyring,
installed_evidence=forged,
installer_receipt=receipt,
installer_authority_keyring=installer_keys,
)
reused_report = self.review(
catalog=catalog,
keyring=keyring,
installed_evidence=evidence,
installer_receipt=receipt,
installer_authority_keyring=reused,
)
self.assertIn(
"installer_receipt_evidence_mismatch",
{item["code"] for item in forged_report["findings"]},
)
self.assertIn(
"installer_authority_untrusted",
{item["code"] for item in reused_report["findings"]},
)
self.assertFalse(
forged_report["proof_scope"]["installed_release_origin"]["valid"]
)
self.assertFalse(
reused_report["proof_scope"]["installed_release_origin"]["valid"]
)
def test_receipt_issuer_refuses_file_like_or_unverified_observations(self) -> None:
evidence = matching_installed_evidence(self.assessment)
artifacts = catalog_artifacts_for(evidence, requires_receipt=True)
catalog, _ = signed_catalog(
self.assessment,
release_artifacts=artifacts,
)
private_key = Ed25519PrivateKey.generate()
with self.assertRaisesRegex(ValueError, "same-process"):
issue_installer_receipt(
assessment=self.assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=private_key,
)
evidence["artifacts"][0]["record_integrity"]["status"] = "partial"
evidence["artifacts"][0]["record_integrity"]["unverifiable_file_count"] = 1
with self.assertRaisesRegex(ValueError, "not a verified match"):
issue_installer_receipt(
assessment=self.assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=private_key,
same_process_observation=True,
consumed_artifacts={
str(item["package_name"]): Path("unused.whl")
for item in evidence["artifacts"]
},
)
def test_installer_receipt_and_observation_must_be_current_together(self) -> None:
evidence = matching_installed_evidence(self.assessment)
artifact_rows = catalog_artifacts_for(evidence, requires_receipt=True)
catalog, keyring = signed_catalog(
self.assessment,
release_artifacts=artifact_rows,
)
receipt, installer_keys = signed_installer_receipt(
assessment=self.assessment,
installed=evidence,
catalog=catalog,
catalog_artifacts=artifact_rows,
)
report = self.review(
catalog=catalog,
keyring=keyring,
installed_evidence=evidence,
installer_receipt=receipt,
installer_authority_keyring=installer_keys,
installed_evidence_mode="imported_unsigned",
verification_time=datetime(2026, 7, 23, 12, 5, 1, tzinfo=UTC),
)
self.assertIn(
"installer_receipt_time_invalid",
{item["code"] for item in report["findings"]},
)
self.assertFalse(
report["proof_scope"]["installed_release_origin"]["valid"]
)
self.assertFalse(
report["proof_scope"]["installed_evidence_observation"][
"receipt_authenticated"
]
)
def test_catalog_trust_blocker_invalidates_dependent_evidence_scopes(
self,
) -> None:
@@ -523,6 +723,7 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
text=True,
)
wheel = next(wheels.glob("govoplan_core-*.whl"))
built_identity = inspect_python_wheel(wheel)
subprocess.run(
(
str(build_python),
@@ -568,6 +769,15 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
self.assertGreater(record["permitted_unhashed_file_count"], 0)
self.assertGreater(record["generated_unhashed_file_count"], 0)
self.assertEqual(0, record["unverifiable_file_count"])
self.assertEqual(
built_identity.payload_sha256,
record["artifact_payload_identity"]["sha256"],
)
self.assertEqual(
built_identity.payload_file_count,
record["artifact_payload_identity"]["file_count"],
)
self.assertTrue(built_identity.requires_installer_receipt)
self.assertEqual(
(), validate_payload(payload=evidence, schema=self.installed_schema)
)
@@ -1162,6 +1372,8 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
catalog: dict[str, object],
keyring: dict[str, object],
installed_evidence: dict[str, object] | None = None,
installer_receipt: dict[str, object] | None = None,
installer_authority_keyring: dict[str, object] | None = None,
boundary_evidence: dict[str, object] | None = None,
authority_keyring: dict[str, object] | None = None,
installed_evidence_mode: str = "direct_local",
@@ -1179,6 +1391,16 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
installed_evidence_schema=(installed_schema or self.installed_schema)
if installed_evidence is not None
else None,
installer_receipt=installer_receipt,
installer_receipt_schema=load_json("docs/installer-receipt.schema.json")
if installer_receipt is not None
else None,
installer_authority_keyring=installer_authority_keyring,
installer_authority_keyring_schema=load_json(
"docs/installer-receipt-authority-keyring.schema.json"
)
if installer_receipt is not None
else None,
boundary_evidence=boundary_evidence,
boundary_evidence_schema=self.boundary_schema
if boundary_evidence is not None
@@ -1196,7 +1418,7 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
def matching_installed_evidence(assessment: dict[str, object]) -> dict[str, object]:
return {
"$schema": "./installed-composition-evidence.schema.json",
"schema_version": "0.3.0",
"schema_version": "0.4.0",
"evidence_kind": "govoplan.installed-composition",
"assessment_id": assessment["assessment_id"],
"assessment_release": assessment["release"]["ref"],
@@ -1238,6 +1460,20 @@ def artifact(
"unverifiable_file_count": 0,
"missing_file_count": 0,
"mismatched_file_count": 0,
"artifact_payload_identity": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": hashlib.sha256(
f"artifact:{package_name}:{version}".encode()
).hexdigest(),
"file_count": 1,
},
"installed_payload_identity": {
"algorithm": "govoplan-installed-record-payload-v1",
"sha256": hashlib.sha256(
f"installed:{package_name}:{version}".encode()
).hexdigest(),
"file_count": 1,
},
},
}
@@ -1323,6 +1559,107 @@ def signed_boundary_evidence(
return proof, keyring
def catalog_artifacts_for(
installed: dict[str, object], *, requires_receipt: bool = False
) -> list[dict[str, object]]:
rows: list[dict[str, object]] = []
for item in installed["artifacts"]:
record = item["record_integrity"]
rows.append(
{
"artifact_kind": "python-wheel",
"package_name": item["package_name"],
"package_version": item["package_version"],
"archive_sha256": hashlib.sha256(
f"wheel:{item['package_name']}:{item['package_version']}".encode()
).hexdigest(),
"archive_size": 1024,
"installed_payload": deepcopy(record["artifact_payload_identity"]),
"requires_installer_receipt": requires_receipt,
}
)
return rows
def signed_installer_receipt(
*,
assessment: dict[str, object],
installed: dict[str, object],
catalog: dict[str, object],
catalog_artifacts: list[dict[str, object]],
) -> tuple[dict[str, object], dict[str, object]]:
private_key = Ed25519PrivateKey.generate()
public_key = base64.b64encode(
private_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
).decode("ascii")
catalog_by_package = {
str(item["package_name"]): item for item in catalog_artifacts
}
receipt: dict[str, object] = {
"$schema": "./installer-receipt.schema.json",
"schema_version": "0.1.0",
"evidence_kind": "govoplan.installer-receipt",
"receipt_id": "install:test",
"assessment_id": assessment["assessment_id"],
"assessment_release": assessment["release"]["ref"],
"installed_evidence_sha256": canonical_sha256(installed),
"catalog": {
"channel": catalog["channel"],
"sequence": catalog["sequence"],
"sha256": hashlib.sha256(
json.dumps(
catalog,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
).hexdigest(),
},
"issued_at": "2026-07-23T11:59:00Z",
"artifacts": [
{
"package_name": item["package_name"],
"package_version": item["package_version"],
"catalog_archive_sha256": catalog_by_package[
str(item["package_name"])
]["archive_sha256"],
"installed_payload": deepcopy(
item["record_integrity"]["installed_payload_identity"]
),
}
for item in installed["artifacts"]
],
}
receipt["signatures"] = [
{
"algorithm": "ed25519",
"key_id": "installer:test",
"value": base64.b64encode(
private_key.sign(canonical_bytes(receipt))
).decode("ascii"),
}
]
keyring = {
"$schema": "./installer-receipt-authority-keyring.schema.json",
"schema_version": "0.1.0",
"purpose": "govoplan.installer-receipt-authorities",
"keys": [
{
"key_id": "installer:test",
"status": "active",
"public_key": public_key,
"allowed_scopes": ["installed_release_origin"],
"not_before": "2026-07-01T00:00:00Z",
"not_after": "2026-08-01T00:00:00Z",
}
],
}
return receipt, keyring
def create_distribution(
root: Path,
) -> tuple[metadata.PathDistribution, Path]:

View File

@@ -606,6 +606,7 @@ def signed_catalog(
sequence: int = 202607220843,
selected_units: list[dict[str, object]] | None = None,
include_selected_units: bool = True,
release_artifacts: list[dict[str, object]] | None = None,
) -> tuple[dict[str, object], dict[str, object]]:
versions = versions or {}
private_key = Ed25519PrivateKey.generate()
@@ -661,6 +662,8 @@ def signed_catalog(
release: dict[str, object] = {"keyring_sha256": canonical_hash(keyring)}
if include_selected_units:
release["selected_units"] = selected_units
if release_artifacts is not None:
release["artifacts"] = release_artifacts
catalog: dict[str, object] = {
"catalog_version": "1",
"channel": "stable",

View File

@@ -0,0 +1,49 @@
from __future__ import annotations
import importlib.util
import pathlib
import sys
import tempfile
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
SCRIPT = ROOT / "tools" / "gitea" / "gitea-sync-wiki.py"
TOOLS = SCRIPT.parent
if str(TOOLS) not in sys.path:
sys.path.insert(0, str(TOOLS))
SPEC = importlib.util.spec_from_file_location("gitea_sync_wiki", SCRIPT)
assert SPEC and SPEC.loader
wiki_sync = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = wiki_sync
SPEC.loader.exec_module(wiki_sync)
class WikiSourceDiscoveryTests(unittest.TestCase):
def test_generated_and_incidental_govoplan_files_are_not_docs(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary)
paths = {
"readme": root / "README.md",
"architecture": root / "docs" / "DATASOURCE_ARCHITECTURE.md",
"plan": root / "workflow-plan.md",
"audit": root / "audit-reports" / "full" / "manifest.json",
"runtime": root / "runtime" / "release" / "manifest.json",
"incidental": root / "tools" / "semgrep" / "govoplan.yml",
"manifest": root / "manifest.json",
}
for path in paths.values():
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("content\n", encoding="utf-8")
self.assertTrue(wiki_sync.is_repo_doc(root, paths["readme"]))
self.assertTrue(wiki_sync.is_repo_doc(root, paths["architecture"]))
self.assertTrue(wiki_sync.is_repo_doc(root, paths["plan"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["audit"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["runtime"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["incidental"]))
self.assertFalse(wiki_sync.is_repo_doc(root, paths["manifest"]))
if __name__ == "__main__":
unittest.main()

View File

@@ -77,6 +77,76 @@ class PythonEnvironmentSyncTests(unittest.TestCase):
self.assertIn(str(root / "govoplan-core"), command)
self.assertIn(str(root / "govoplan-access"), command)
def test_missing_editable_distribution_is_not_hidden_by_current_stamp(self) -> None:
sync = load_sync_module()
with tempfile.TemporaryDirectory(prefix="govoplan-python-sync-") as directory:
root = Path(directory)
project_root = root / "govoplan-probe-missing"
project_root.mkdir()
(project_root / "pyproject.toml").write_text(
"\n".join(
(
"[project]",
'name = "govoplan-probe-definitely-not-installed"',
'version = "0.1.0"',
"",
'[project.entry-points."govoplan.modules"]',
'probe_missing = "govoplan_probe.backend.manifest:get_manifest"',
"",
)
),
encoding="utf-8",
)
requirements = root / "requirements-dev.txt"
requirements.write_text("-e ./govoplan-probe-missing\n", encoding="utf-8")
entries = sync.local_requirement_entries(requirements)
validation = sync.validate_local_installations(sys.executable, entries)
plan = sync.build_environment_repair_plan(
python=sys.executable,
stale_requirements=validation.stale_requirements,
)
self.assertFalse(validation.current)
self.assertEqual(validation.stale_requirements, entries)
self.assertIn("distribution is not installed", validation.issues[0])
self.assertEqual(plan.mode, "Selective Python environment repair")
self.assertEqual(plan.commands[0][-2:], ("-e", str(project_root)))
def test_declared_module_entry_points_are_part_of_environment_validation(self) -> None:
sync = load_sync_module()
with tempfile.TemporaryDirectory(prefix="govoplan-python-sync-") as directory:
root = Path(directory)
project_root = root / "govoplan-probe"
project_root.mkdir()
(project_root / "pyproject.toml").write_text(
"\n".join(
(
"[project]",
'name = "govoplan-probe"',
'version = "0.1.0"',
"",
'[project.entry-points."govoplan.modules"]',
'probe = "govoplan_probe.backend.manifest:get_manifest"',
"",
)
),
encoding="utf-8",
)
requirements = root / "requirements-dev.txt"
requirements.write_text("-e ./govoplan-probe\n", encoding="utf-8")
expectations = sync.local_installation_expectations(
sync.local_requirement_entries(requirements)
)
self.assertEqual(len(expectations), 1)
self.assertEqual(expectations[0].distribution, "govoplan-probe")
self.assertEqual(
expectations[0].entry_points,
(("govoplan.modules", "probe", "govoplan_probe.backend.manifest:get_manifest"),),
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,277 @@
from __future__ import annotations
import csv
from datetime import UTC, datetime
import io
from pathlib import Path
import sys
import tempfile
import unittest
from unittest import mock
import zipfile
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.artifact_identity import ( # noqa: E402
ArtifactIdentityError,
inspect_python_wheel,
selected_artifact_identity_issues,
)
from govoplan_release.selective_catalog import ( # noqa: E402
apply_python_artifact_identities,
)
ASSESSMENT_TOOLS_ROOT = META_ROOT / "tools" / "assessments"
if str(ASSESSMENT_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(ASSESSMENT_TOOLS_ROOT))
from govoplan_assessment.installer_receipt import issue_installer_receipt # noqa: E402
class ReleaseArtifactIdentityTests(unittest.TestCase):
def test_built_wheel_bytes_become_catalog_identity(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
wheel = self._wheel(Path(temp_dir), console_script=True)
payload = self._catalog_payload()
changes = apply_python_artifact_identities(
payload,
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={"govoplan-demo": wheel},
)
identity = payload["release"]["artifacts"][0]
self.assertEqual("govoplan-demo", identity["package_name"])
self.assertEqual("1.2.3", identity["package_version"])
self.assertRegex(identity["archive_sha256"], r"^[0-9a-f]{64}$")
self.assertTrue(identity["requires_installer_receipt"])
self.assertEqual("release.artifact", changes[0].field)
self.assertEqual((), selected_artifact_identity_issues(payload))
def test_version_update_without_wheel_removes_stale_identity_and_blocks_publish(self) -> None:
payload = self._catalog_payload()
payload["release"]["artifacts"] = [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-demo",
"package_version": "1.2.2",
"archive_sha256": "a" * 64,
"archive_size": 10,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "b" * 64,
"file_count": 1,
},
"requires_installer_receipt": False,
}
]
apply_python_artifact_identities(
payload,
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={},
)
self.assertNotIn("artifacts", payload["release"])
self.assertIn(
"no built artifact identity",
" ".join(selected_artifact_identity_issues(payload)),
)
def test_tag_only_selection_needs_no_python_artifact_mapping(self) -> None:
payload = self._catalog_payload()
payload["release"]["selected_units"].append(
{"repo": "govoplan", "version": "1.2.3"}
)
apply_python_artifact_identities(
payload,
repo_versions={"govoplan": "1.2.3"},
python_artifacts={},
)
self.assertIn("selected_units", payload["release"])
def test_unsafe_or_mismatched_wheel_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
unsafe = root / "govoplan_demo-1.2.3-py3-none-any.whl"
with zipfile.ZipFile(unsafe, "w") as archive:
archive.writestr("../escape", b"bad")
archive.writestr(
"govoplan_demo-1.2.3.dist-info/METADATA",
"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: 1.2.3\n",
)
wrong = self._wheel(root, version="9.9.9")
with self.assertRaisesRegex(ArtifactIdentityError, "unsafe"):
inspect_python_wheel(unsafe)
with self.assertRaisesRegex(ValueError, "expected govoplan-demo 1.2.3"):
apply_python_artifact_identities(
self._catalog_payload(),
repo_versions={"govoplan-demo": "1.2.3"},
python_artifacts={"govoplan-demo": wrong},
)
def test_receipt_issuer_hashes_exact_consumed_wheel(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
original = self._wheel(root / "original", value=b"VALUE = 1\n")
different = self._wheel(root / "different", value=b"VALUE = 2\n")
identity = inspect_python_wheel(original)
catalog = self._catalog_payload()
catalog["channel"] = "stable"
catalog["sequence"] = 1
catalog["release"]["artifacts"] = [identity.catalog_payload()]
assessment = {
"assessment_id": "assessment:test",
"release": {"ref": "stable-catalog-1"},
"composition": [{"module_id": "demo", "enabled": True}],
}
installed_payload = {
"algorithm": "govoplan-installed-record-payload-v1",
"sha256": "c" * 64,
"file_count": identity.payload_file_count,
}
evidence = {
"assessment_id": "assessment:test",
"assessment_release": "stable-catalog-1",
"collected_at": "2026-07-22T12:00:00Z",
"artifacts": [
{
"package_name": "govoplan-demo",
"package_version": "1.2.3",
"record_integrity": {
"status": "verified",
"hashed_file_count": identity.payload_file_count,
"permitted_unhashed_file_count": 1,
"generated_unhashed_file_count": 0,
"unverifiable_file_count": 0,
"missing_file_count": 0,
"mismatched_file_count": 0,
"artifact_payload_identity": catalog["release"]["artifacts"][0]["installed_payload"],
"installed_payload_identity": installed_payload,
},
}
],
}
key = Ed25519PrivateKey.generate()
receipt = issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": original},
issued_at=datetime(2026, 7, 22, 12, 1, tzinfo=UTC),
same_process_observation=True,
)
with self.assertRaisesRegex(ValueError, "differs from the signed"):
issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": different},
issued_at=datetime(2026, 7, 22, 12, 1, tzinfo=UTC),
same_process_observation=True,
)
with self.assertRaisesRegex(ValueError, "within five minutes"):
issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id="install:test",
key_id="installer:test",
private_key=key,
consumed_artifacts={"govoplan-demo": original},
issued_at=datetime(2026, 7, 22, 12, 6, tzinfo=UTC),
same_process_observation=True,
)
self.assertEqual(identity.archive_sha256, receipt["artifacts"][0]["catalog_archive_sha256"])
def test_path_replacement_during_one_descriptor_inspection_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
target = self._wheel(root / "target", value=b"VALUE = 1\n")
replacement = self._wheel(root / "replacement", value=b"VALUE = 2\n")
real_zip = zipfile.ZipFile
def swap_path(file_or_path, *args, **kwargs):
target.unlink()
replacement.rename(target)
return real_zip(file_or_path, *args, **kwargs)
with (
mock.patch(
"govoplan_release.artifact_identity.zipfile.ZipFile",
side_effect=swap_path,
),
self.assertRaisesRegex(ArtifactIdentityError, "changed"),
):
inspect_python_wheel(target)
@staticmethod
def _catalog_payload() -> dict[str, object]:
return {
"core_release": {},
"modules": [
{
"module_id": "demo",
"version": "1.2.3",
"python_package": "govoplan-demo",
"python_ref": "govoplan-demo @ git+ssh://git@example.test/acme/govoplan-demo.git@v1.2.3",
}
],
"release": {
"selected_units": [
{"repo": "govoplan-demo", "version": "1.2.3"}
]
},
}
@staticmethod
def _wheel(
root: Path,
*,
version: str = "1.2.3",
console_script: bool = False,
value: bytes = b"VALUE = 1\n",
) -> Path:
root.mkdir(parents=True, exist_ok=True)
wheel = root / f"govoplan_demo-{version}-py3-none-any.whl"
dist_info = f"govoplan_demo-{version}.dist-info"
files: dict[str, bytes] = {
"govoplan_demo.py": value,
f"{dist_info}/METADATA": (
f"Metadata-Version: 2.1\nName: govoplan-demo\nVersion: {version}\n"
).encode(),
f"{dist_info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
}
if console_script:
files[f"{dist_info}/entry_points.txt"] = (
b"[console_scripts]\ngovoplan-demo = govoplan_demo:main\n"
)
record_buffer = io.StringIO()
writer = csv.writer(record_buffer, lineterminator="\n")
for name, encoded in files.items():
writer.writerow((name, "", len(encoded)))
writer.writerow((f"{dist_info}/RECORD", "", ""))
files[f"{dist_info}/RECORD"] = record_buffer.getvalue().encode()
with zipfile.ZipFile(wheel, "w", compression=zipfile.ZIP_DEFLATED) as archive:
for name, encoded in files.items():
archive.writestr(name, encoded)
return wheel
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,184 @@
from __future__ import annotations
import json
from datetime import UTC, datetime
from pathlib import Path
import sys
import tempfile
import unittest
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
candidate_keyring_from_authenticated_base,
load_authenticated_catalog_base,
public_key_base64,
signature,
)
class ReleaseBaseCatalogTrustTests(unittest.TestCase):
def test_exact_signed_base_pair_is_loaded_once_and_carried_in_memory(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
authenticated = load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={"release-key": public_key_base64(private_key)},
)
keyring.unlink()
self.assertEqual("release-key", authenticated.keyring["keys"][0]["key_id"])
self.assertEqual(
canonical_hash(authenticated.keyring),
authenticated.catalog["release"]["keyring_sha256"],
)
def test_deliberate_old_to_new_signer_rotation_is_carried_forward(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
new_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
configured = {
"release-key": public_key_base64(private_key),
"release-key-next": public_key_base64(new_key),
}
authenticated = load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys=configured,
)
candidate = candidate_keyring_from_authenticated_base(
authenticated.keyring,
signer_public_keys=configured,
generated_at=datetime(2026, 7, 22, 12, tzinfo=UTC),
)
self.assertEqual(
{"release-key", "release-key-next"},
{item["key_id"] for item in candidate["keys"]},
)
def test_injected_extra_key_without_catalog_authorization_is_rejected(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
extra_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
payload = json.loads(keyring.read_text(encoding="utf-8"))
payload["keys"].append(
{
"key_id": "injected-key",
"status": "active",
"public_key": public_key_base64(extra_key),
}
)
keyring.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
def test_unpinned_or_symlinked_base_keyring_fails_closed(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
private_key = Ed25519PrivateKey.generate()
catalog, keyring = self._write_pair(root, private_key=private_key)
payload = json.loads(keyring.read_text(encoding="utf-8"))
payload["generated_at"] = "changed"
keyring.write_text(json.dumps(payload), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
keyring.unlink()
outside = root / "outside.json"
outside.write_text("{}", encoding="utf-8")
keyring.symlink_to(outside)
with self.assertRaisesRegex(ValueError, "opened safely"):
load_authenticated_catalog_base(
base_catalog=catalog,
base_keyring=keyring,
web_root=root,
channel="stable",
public_base_url="https://invalid.example",
signer_public_keys={
"release-key": public_key_base64(private_key)
},
)
@staticmethod
def _write_pair(
root: Path, *, private_key: Ed25519PrivateKey
) -> tuple[Path, Path]:
keys = [
{
"key_id": "release-key",
"status": "active",
"public_key": public_key_base64(private_key),
}
]
keyring_payload = {
"keyring_version": "1",
"purpose": "govoplan module package catalog signatures",
"keys": keys,
}
catalog_payload = {
"channel": "stable",
"sequence": 1,
"core_release": {"version": "1.0.0"},
"modules": [],
"release": {"keyring_sha256": canonical_hash(keyring_payload)},
}
catalog_payload["signatures"] = [
signature(
catalog_payload,
key_id="release-key",
private_key=private_key,
)
]
catalog = root / "stable.json"
keyring = root / "keyring.json"
catalog.write_text(json.dumps(catalog_payload), encoding="utf-8")
keyring.write_text(json.dumps(keyring_payload), encoding="utf-8")
return catalog, keyring
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,153 @@
from __future__ import annotations
import json
from pathlib import Path
import sys
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.candidate_artifact import ( # noqa: E402
CandidateArtifactError,
candidate_output_path,
issue_candidate_id,
issue_candidate_receipt,
verify_candidate_receipt,
validate_release_channel,
)
class CandidateArtifactTests(unittest.TestCase):
def test_channel_is_one_canonical_basename(self) -> None:
self.assertEqual("stable_1", validate_release_channel("stable_1"))
for value in ("../stable", "/stable", ".", "..", "Stable", "stable/name"):
with self.subTest(value=value), self.assertRaises(CandidateArtifactError):
validate_release_channel(value)
def test_handle_is_deterministic_and_can_precede_output(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "not-created" / "release-candidates"
candidate_id = issue_candidate_id("rr-123", "attempt-456")
first = candidate_output_path(root, candidate_id)
second = candidate_output_path(root, candidate_id)
self.assertEqual(first, second)
self.assertEqual(candidate_id, first.name)
self.assertRegex(candidate_id, r"^candidate-[0-9a-f]{32}$")
def test_receipt_rejects_catalog_drift(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
catalog = self._write_candidate(root / candidate_id)
receipt = issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
resolved = verify_candidate_receipt(
root=root,
candidate_id=receipt.candidate_id,
catalog_sha256=receipt.catalog_sha256,
channel="stable",
)
catalog.write_text(
json.dumps({"channel": "stable", "sequence": 2, "signatures": [{}]}),
encoding="utf-8",
)
self.assertEqual(root / candidate_id, resolved)
with self.assertRaisesRegex(CandidateArtifactError, "no longer matches"):
verify_candidate_receipt(
root=root,
candidate_id=receipt.candidate_id,
catalog_sha256=receipt.catalog_sha256,
channel="stable",
)
def test_unissued_ids_traversal_and_symlinks_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
root.mkdir(mode=0o700)
candidate_id = issue_candidate_id("run", "attempt")
outside = Path(temp_dir) / "outside"
outside.mkdir()
(root / candidate_id).symlink_to(outside, target_is_directory=True)
with self.assertRaises(CandidateArtifactError):
candidate_output_path(root, "../candidate-" + "0" * 32)
with self.assertRaisesRegex(CandidateArtifactError, "real directory"):
candidate_output_path(root, candidate_id)
def test_catalog_and_channel_components_must_not_be_symlinks(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
candidate = root / candidate_id
target = Path(temp_dir) / "target"
target.mkdir()
(target / "stable.json").write_text(
json.dumps({"signatures": [{}]}), encoding="utf-8"
)
root.mkdir(mode=0o700)
candidate.mkdir(mode=0o700)
(candidate / "channels").symlink_to(target, target_is_directory=True)
with self.assertRaisesRegex(CandidateArtifactError, "real directory"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
def test_shared_candidate_roots_and_catalog_files_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
root.mkdir(mode=0o755)
candidate_id = issue_candidate_id("run", "attempt")
with self.assertRaisesRegex(CandidateArtifactError, "another user"):
candidate_output_path(root, candidate_id)
root.chmod(0o700)
catalog = self._write_candidate(root / candidate_id)
catalog.chmod(0o640)
with self.assertRaisesRegex(CandidateArtifactError, "another user"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
def test_receipt_rejects_catalog_with_inconsistent_channel(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "release-candidates"
candidate_id = issue_candidate_id("run", "attempt")
catalog = self._write_candidate(root / candidate_id)
catalog.write_text(
json.dumps({"channel": "next", "signatures": [{}]}),
encoding="utf-8",
)
with self.assertRaisesRegex(CandidateArtifactError, "does not match"):
issue_candidate_receipt(
root=root, candidate_id=candidate_id, channel="stable"
)
@staticmethod
def _write_candidate(candidate: Path) -> Path:
candidate.parent.mkdir(mode=0o700, exist_ok=True)
candidate.mkdir(mode=0o700)
channels = candidate / "channels"
channels.mkdir(mode=0o700)
catalog = channels / "stable.json"
catalog.write_text(
json.dumps({"channel": "stable", "sequence": 1, "signatures": [{}]}),
encoding="utf-8",
)
catalog.chmod(0o600)
return catalog
if __name__ == "__main__":
unittest.main()

View File

@@ -24,7 +24,7 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
"module_id": "access",
"version": "0.1.11",
"python_package": "govoplan-access",
"python_ref": "govoplan-access @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-access.git@v0.1.11",
"python_ref": "govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.11",
}
],
}
@@ -37,7 +37,7 @@ class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
"govoplan-scheduling": "0.1.11",
},
repo_contracts={},
repository_base="git+ssh://git@git.add-ideas.de/add-ideas",
repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN",
workspace=META_ROOT.parent,
)

View File

@@ -1,7 +1,10 @@
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
@@ -13,10 +16,691 @@ RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.publisher import publish_catalog_candidate # noqa: E402
from govoplan_release.publisher import ( # noqa: E402
FrozenPublicationRemote,
bind_publication_remote,
commit_publication_tree,
publication_mutation_trust_issues,
publish_catalog_candidate,
remote_publication_identity,
run_checked,
verify_committed_publication,
verify_remote_branch_head,
verify_remote_publication,
_git_bytes,
_sanitized_git_environment,
_seal_git_metadata_file,
_trusted_npm_command,
)
class ReleaseCatalogPublicationTests(unittest.TestCase):
def test_publication_git_writes_ignore_permissive_operator_umask(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
repository = Path(temp_dir) / "website"
repository.mkdir()
self._git(repository, "init", "--quiet")
payload_number = 0
while True:
payload = f"private publication object {payload_number}\n".encode()
header = f"blob {len(payload)}\0".encode()
expected_object = hashlib.sha1(
header + payload,
usedforsecurity=False,
).hexdigest()
object_parent = repository / ".git" / "objects" / expected_object[:2]
if not object_parent.exists():
break
payload_number += 1
previous_umask = os.umask(0o002)
try:
object_id = (
_git_bytes(
repository,
"hash-object",
"-w",
"--stdin",
input_bytes=payload,
)
.decode("ascii")
.strip()
)
finally:
os.umask(previous_umask)
object_path = object_parent / expected_object[2:]
self.assertEqual(expected_object, object_id)
self.assertEqual(os.geteuid(), object_parent.stat().st_uid)
self.assertEqual(0o700, object_parent.stat().st_mode & 0o777)
self.assertEqual(os.geteuid(), object_path.stat().st_uid)
self.assertEqual(0o400, object_path.stat().st_mode & 0o777)
def test_publication_git_metadata_is_sealed_after_git_writes(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
metadata = Path(temp_dir) / "index"
metadata.write_bytes(b"index")
metadata.chmod(0o664)
_seal_git_metadata_file(metadata, label="test index")
self.assertEqual(0o600, metadata.stat().st_mode & 0o777)
def test_publication_git_identity_is_fixed_and_non_personal(self) -> None:
with patch.dict(
os.environ,
{
"GIT_AUTHOR_NAME": "Attacker",
"GIT_AUTHOR_EMAIL": "attacker@example.test",
"GIT_COMMITTER_NAME": "Attacker",
"GIT_COMMITTER_EMAIL": "attacker@example.test",
},
clear=False,
):
environment = _sanitized_git_environment()
self.assertEqual(
"GovOPlaN Release Automation",
environment["GIT_AUTHOR_NAME"],
)
self.assertEqual(
"release@govoplan.invalid",
environment["GIT_AUTHOR_EMAIL"],
)
self.assertEqual(
environment["GIT_AUTHOR_NAME"],
environment["GIT_COMMITTER_NAME"],
)
self.assertEqual(
environment["GIT_AUTHOR_EMAIL"],
environment["GIT_COMMITTER_EMAIL"],
)
def test_build_command_uses_its_pinned_node_directory(self) -> None:
completed = subprocess.CompletedProcess(
["/trusted/node/bin/npm"],
0,
stdout=b"",
stderr=b"",
)
with patch(
"govoplan_release.publisher.subprocess.run",
return_value=completed,
) as runner:
run_checked(
["/trusted/node/bin/npm", "run", "build"],
cwd=Path("/trusted/website"),
)
environment = runner.call_args.kwargs["env"]
self.assertEqual(
"/trusted/node/bin:/usr/bin:/bin",
environment["PATH"],
)
def test_npm_command_requires_trusted_npm_and_sibling_node(self) -> None:
with (
patch(
"govoplan_release.publisher.shutil.which",
return_value="/trusted/node/bin/npm",
),
patch(
"govoplan_release.publisher._trusted_runtime_executable_issue",
side_effect=(None, None),
) as trust_check,
):
self.assertEqual(
"/trusted/node/bin/npm",
_trusted_npm_command("npm"),
)
self.assertEqual(Path("/trusted/node/bin/npm"), trust_check.call_args_list[0].args[0])
self.assertEqual(Path("/trusted/node/bin/node"), trust_check.call_args_list[1].args[0])
def test_npm_command_rejects_caller_relative_path(self) -> None:
with (
patch("govoplan_release.publisher.shutil.which") as which,
self.assertRaisesRegex(RuntimeError, "absolute path or the bare name"),
):
_trusted_npm_command("./npm")
which.assert_not_called()
def test_committed_publication_must_match_validated_blobs_exactly(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
web_root = Path(tmp) / "website"
module_root = web_root / "public" / "catalogs" / "v1" / "modules"
module_root.mkdir(parents=True)
catalog = (
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
)
keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
catalog.parent.mkdir(parents=True)
expected = {
catalog.relative_to(web_root).as_posix(): b'{"catalog":true}\n',
keyring.relative_to(web_root).as_posix(): b'{"keys":[]}\n',
(module_root / "index.json")
.relative_to(web_root)
.as_posix(): b'{"modules":[]}\n',
}
for relative, encoded in expected.items():
target = web_root / relative
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(encoded)
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "publication")
commit_sha = self._git(web_root, "rev-parse", "HEAD").strip()
verify_committed_publication(
web_root=web_root,
commit_sha=commit_sha,
expected_blobs=expected,
module_root=module_root,
)
changed = dict(expected)
changed[catalog.relative_to(web_root).as_posix()] = b'{"catalog":false}\n'
with self.assertRaisesRegex(RuntimeError, "differs"):
verify_committed_publication(
web_root=web_root,
commit_sha=commit_sha,
expected_blobs=changed,
module_root=module_root,
)
catalog_path = catalog.relative_to(web_root).as_posix()
self._git(web_root, "update-index", "--chmod=+x", catalog_path)
self._git(web_root, "commit", "--quiet", "-m", "unsafe mode")
executable_commit = self._git(web_root, "rev-parse", "HEAD").strip()
with self.assertRaisesRegex(RuntimeError, "regular blob"):
verify_committed_publication(
web_root=web_root,
commit_sha=executable_commit,
expected_blobs=expected,
module_root=module_root,
)
def test_private_index_commit_has_one_parent_and_only_computed_delta(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
web_root = root / "website"
module_root = web_root / "public" / "catalogs" / "v1" / "modules"
channel = (
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
)
keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
old_module = module_root / "obsolete.json"
unrelated = web_root / "unrelated.txt"
for path, encoded in (
(channel, b'{"old":true}\n'),
(keyring, b'{"keys":[]}\n'),
(old_module, b'{"obsolete":true}\n'),
(unrelated, b"keep\n"),
):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(encoded)
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
frozen_head = self._git(web_root, "rev-parse", "HEAD").strip()
branch = self._git(web_root, "branch", "--show-current").strip()
malicious = web_root / "not-in-publication.txt"
malicious.write_text("staged but excluded\n", encoding="utf-8")
self._git(web_root, "add", malicious.name)
marker = root / "reference-hook-ran"
hook = web_root / ".git" / "hooks" / "reference-transaction"
hook.write_text(f"#!/bin/sh\ntouch {marker}\n", encoding="utf-8")
hook.chmod(0o755)
expected = {
channel.relative_to(web_root).as_posix(): b'{"new":true}\n',
keyring.relative_to(web_root).as_posix(): b'{"keys":["release"]}\n',
(module_root / "index.json")
.relative_to(web_root)
.as_posix(): b'{"modules":[]}\n',
}
redirected = root / "attacker-index"
with patch.dict(
os.environ,
{
"GIT_DIR": str(root / "attacker-git-dir"),
"GIT_INDEX_FILE": str(redirected),
"GIT_OBJECT_DIRECTORY": str(root / "attacker-objects"),
"GIT_CONFIG_GLOBAL": str(root / "attacker-config"),
},
):
commit_sha = commit_publication_tree(
web_root=web_root,
frozen_head=frozen_head,
branch=branch,
expected_blobs=expected,
module_root=module_root,
message="Exact publication",
)
parents = self._git(
web_root, "rev-list", "--parents", "-n", "1", commit_sha
).split()
changed = set(
filter(
None,
self._git(
web_root,
"diff-tree",
"--no-commit-id",
"--name-only",
"-r",
frozen_head,
commit_sha,
).splitlines(),
)
)
hook_ran = marker.exists()
inherited_index_used = redirected.exists()
commit_paths = self._git(
web_root, "ls-tree", "-r", "--name-only", commit_sha
)
self.assertEqual([commit_sha, frozen_head], parents)
self.assertEqual(
{
"public/catalogs/v1/channels/stable.json",
"public/catalogs/v1/keyring.json",
"public/catalogs/v1/modules/index.json",
"public/catalogs/v1/modules/obsolete.json",
},
changed,
)
self.assertFalse(hook_ran)
self.assertFalse(inherited_index_used)
self.assertNotIn("not-in-publication.txt", commit_paths)
def test_remote_binding_and_annotated_publication_identity_are_exact(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
remote_root = root / "website.git"
remote_root.mkdir()
self._git(remote_root, "init", "--bare", "--quiet")
web_root = root / "website"
web_root.mkdir()
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "branch", "-M", "main")
web_root.joinpath("base.txt").write_text("base\n", encoding="utf-8")
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
self._git(web_root, "remote", "add", "origin", str(remote_root))
self._git(web_root, "push", "--quiet", "-u", "origin", "main")
base_commit = self._git(web_root, "rev-parse", "HEAD").strip()
frozen = bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
self.assertIsInstance(frozen, FrozenPublicationRemote)
verify_remote_branch_head(
web_root=web_root,
remote_url=frozen.url,
branch="main",
expected_commit=base_commit,
)
web_root.joinpath("catalog.json").write_text("{}\n", encoding="utf-8")
self._git(web_root, "add", "catalog.json")
self._git(web_root, "commit", "--quiet", "-m", "publication")
commit_sha = self._git(web_root, "rev-parse", "HEAD").strip()
tag_name = "catalog-test"
self._git(web_root, "tag", "-a", tag_name, "-m", "publication")
tag_object = self._git(
web_root, "rev-parse", f"refs/tags/{tag_name}"
).strip()
self._git(
web_root,
"push",
"--quiet",
"--atomic",
"origin",
f"{commit_sha}:refs/heads/main",
f"{tag_object}:refs/tags/{tag_name}",
)
identity = remote_publication_identity(
web_root=web_root,
remote_url=frozen.url,
branch="main",
tag_name=tag_name,
)
verified = verify_remote_publication(
web_root=web_root,
remote_url=frozen.url,
branch="main",
tag_name=tag_name,
expected_commit=commit_sha,
expected_tag_object=tag_object,
)
self._git(
web_root,
"remote",
"set-url",
"--add",
"--push",
"origin",
str(root / "other.git"),
)
with self.assertRaisesRegex(RuntimeError, "do not match"):
bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
self.assertEqual(identity, verified)
self.assertEqual(commit_sha, identity["publication_commit_sha"])
self.assertEqual(tag_object, identity["publication_tag_object_sha"])
self.assertEqual(commit_sha, identity["publication_tag_commit_sha"])
def test_mutation_rejects_writable_website_and_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
web_root = root / "website"
web_root.mkdir()
self._git(web_root, "init", "--quiet")
candidate = self._candidate(root, key="trusted-key")
target_root = web_root / "public" / "catalogs" / "v1"
target_catalog = target_root / "channels" / "stable.json"
target_keyring = target_root / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text("{}\n", encoding="utf-8")
web_root.chmod(0o777)
root_issues = publication_mutation_trust_issues(
web_root=web_root,
candidate_catalog=candidate / "channels" / "stable.json",
candidate_keyring=candidate / "keyring.json",
target_catalog=target_catalog,
target_keyring=target_keyring,
target_modules=target_root / "modules",
)
web_root.chmod(0o755)
config = web_root / ".git" / "config"
config.chmod(0o666)
config_issues = publication_mutation_trust_issues(
web_root=web_root,
candidate_catalog=candidate / "channels" / "stable.json",
candidate_keyring=candidate / "keyring.json",
target_catalog=target_catalog,
target_keyring=target_keyring,
target_modules=target_root / "modules",
)
self.assertIn("website root is writable by another user", root_issues)
self.assertIn("website Git config is writable by another user", config_issues)
def test_push_returns_only_independently_verified_publication_receipt(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog["sequence"] = 7
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
],
"artifacts": [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-core",
"package_version": "1.2.3",
"archive_sha256": "c" * 64,
"archive_size": 100,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "d" * 64,
"file_count": 1,
},
"requires_installer_receipt": True,
}
],
}
catalog["signatures"] = [{}]
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
remote_root = root / "website.git"
remote_root.mkdir()
self._git(remote_root, "init", "--bare", "--quiet")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "branch", "-M", "main")
self._git(web_root, "remote", "add", "origin", str(remote_root))
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
self._git(web_root, "push", "--quiet", "-u", "origin", "main")
frozen_head = self._git(web_root, "rev-parse", "HEAD").strip()
frozen_remote = bind_publication_remote(
web_root=web_root,
remote="origin",
registered_remote=str(remote_root),
)
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.publication_runtime_trust_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.registered_website_remote",
return_value=str(remote_root),
),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
push=True,
branch="main",
tag_name="catalog-test",
expected_website_head=frozen_head,
expected_website_branch="main",
expected_remote_sha256=frozen_remote.sha256,
)
remote_identity = remote_publication_identity(
web_root=web_root,
remote_url=str(remote_root),
branch="main",
tag_name="catalog-test",
)
self.assertEqual("published", result.status)
self.assertEqual("origin", result.remote)
self.assertEqual(
remote_identity["publication_commit_sha"], result.publication_commit_sha
)
self.assertEqual(
remote_identity["publication_tag_object_sha"],
result.publication_tag_object_sha,
)
self.assertEqual(
remote_identity["publication_tag_commit_sha"],
result.publication_tag_commit_sha,
)
def test_apply_writes_validated_objects_even_if_candidate_path_changes(
self,
) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog.update({"channel": "stable", "sequence": 1})
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
],
"artifacts": [
{
"artifact_kind": "python-wheel",
"package_name": "govoplan-core",
"package_version": "1.2.3",
"archive_sha256": "c" * 64,
"archive_size": 100,
"installed_payload": {
"algorithm": "govoplan-wheel-declared-payload-v1",
"sha256": "d" * 64,
"file_count": 1,
},
"requires_installer_receipt": True,
}
],
}
catalog["signatures"] = [{}]
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
registered_remote = "ssh://example.test/release/website.git"
self._git(web_root, "init", "--quiet")
self._git(web_root, "config", "user.email", "test@example.test")
self._git(web_root, "config", "user.name", "Test")
self._git(web_root, "remote", "add", "origin", registered_remote)
self._git(web_root, "add", ".")
self._git(web_root, "commit", "--quiet", "-m", "base")
def validate_and_swap(*args, **kwargs):
del args, kwargs
catalog_path.write_text(
json.dumps(
{
"channel": "stable",
"sequence": 999,
"malicious": True,
"signatures": [{}],
}
),
encoding="utf-8",
)
return {"valid": True, "warnings": [], "error": None}
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
side_effect=validate_and_swap,
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch("govoplan_release.publisher.website_dirty", return_value=False),
patch(
"govoplan_release.publisher.publication_runtime_trust_issues",
return_value=(),
),
patch(
"govoplan_release.publisher.registered_website_remote",
return_value=registered_remote,
),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
allow_dirty_website=True,
)
published = json.loads(
(
web_root / "public" / "catalogs" / "v1" / "channels" / "stable.json"
).read_text(encoding="utf-8")
)
self.assertEqual("applied", result.status)
self.assertEqual(1, published["sequence"])
self.assertNotIn("malicious", published)
def test_apply_blocks_selected_python_release_without_built_identity(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
candidate = self._candidate(root, key="trusted-key")
catalog_path = candidate / "channels" / "stable.json"
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
catalog["core_release"]["python_package"] = "govoplan-core"
catalog["release"] = {
"selected_units": [
{
"repo": "govoplan-core",
"version": "1.2.3",
"tag": "v1.2.3",
"commit_sha": "a" * 40,
"tag_object_sha": "b" * 40,
}
]
}
catalog_path.write_text(json.dumps(catalog), encoding="utf-8")
web_root = root / "website"
target_keyring = web_root / "public" / "catalogs" / "v1" / "keyring.json"
target_keyring.parent.mkdir(parents=True)
target_keyring.write_text(json.dumps(self._keyring("trusted-key")))
(web_root / ".git").mkdir()
with (
patch(
"govoplan_release.publisher.validate_module_package_catalog",
return_value={"valid": True, "warnings": [], "error": None},
),
patch(
"govoplan_release.publisher.source_tag_provenance_issues",
return_value=(),
),
patch("govoplan_release.publisher.website_dirty", return_value=False),
):
result = publish_catalog_candidate(
candidate_dir=candidate,
web_root=web_root,
workspace_root=root,
apply=True,
)
self.assertEqual("blocked", result.status)
self.assertIn(
"selected Python repository govoplan-core has no built artifact identity",
" ".join(result.notes),
)
def test_publication_requires_an_existing_trust_anchor(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
@@ -79,7 +763,9 @@ class ReleaseCatalogPublicationTests(unittest.TestCase):
}
)
)
candidate.joinpath("keyring.json").write_text(json.dumps(ReleaseCatalogPublicationTests._keyring(key)))
candidate.joinpath("keyring.json").write_text(
json.dumps(ReleaseCatalogPublicationTests._keyring(key))
)
return candidate
@staticmethod
@@ -94,6 +780,17 @@ class ReleaseCatalogPublicationTests(unittest.TestCase):
]
}
@staticmethod
def _git(root: Path, *args: str) -> str:
return subprocess.run(
["git", *args],
cwd=root,
check=True,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
).stdout
if __name__ == "__main__":
unittest.main()

View File

@@ -1,5 +1,6 @@
from __future__ import annotations
import subprocess
import sys
import unittest
from pathlib import Path
@@ -36,6 +37,30 @@ class ReleaseConsoleSecurityTests(unittest.TestCase):
self.assertIn("window.location.hash.slice(1)", webui)
self.assertIn("history.replaceState", webui)
def test_tokenless_embedding_is_read_only(self) -> None:
with TestClient(create_app(workspace_root=META_ROOT)) as client:
read_response = client.get("/api/health")
write_response = client.post("/api/selective-plan", json={})
self.assertEqual(200, read_response.status_code)
self.assertEqual(403, write_response.status_code)
self.assertIn("read-only", write_response.json()["detail"])
def test_launcher_rejects_non_loopback_and_tokenless_modes(self) -> None:
launcher = RELEASE_ROOT / "release-console.py"
for arguments in (("--host", "0.0.0.0"), ("--no-token",)):
with self.subTest(arguments=arguments):
result = subprocess.run(
(sys.executable, str(launcher), *arguments),
cwd=META_ROOT,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=10,
)
self.assertEqual(2, result.returncode)
if __name__ == "__main__":
unittest.main()

View File

@@ -55,7 +55,8 @@ class ReleaseEntrypointGateTests(unittest.TestCase):
self.assertIn("trusted_keys_from_keyring(\n target_keyring_payload", publisher)
self.assertIn("trusted_keys=publication_trust", publisher)
self.assertNotIn("trusted_keys=candidate_keys", publisher)
self.assertIn("write_module_directory(", publisher)
self.assertIn("module_directory_payloads(", publisher)
self.assertIn("verify_committed_publication(", publisher)
self.assertNotIn("shutil.copytree(candidate_modules", publisher)
def test_release_integration_honors_independent_module_versions(self) -> None:

View File

@@ -0,0 +1,956 @@
from __future__ import annotations
import json
from pathlib import Path
import os
import subprocess
import sys
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.model import RepositorySpec # noqa: E402
from govoplan_release.git_state import sanitized_git_environment # noqa: E402
from govoplan_release.release_execution import ( # noqa: E402
ExecutorSpec,
ReleaseExecutionAmbiguous,
ReleaseExecutionBlocked,
build_selected_wheels,
execute_repository_step,
executor_spec,
reconciled_repository_receipt,
repository_state_receipt,
require_trusted_release_runtime,
verify_frozen_repository_tag_receipt,
verify_repository_preflight_binding,
_clone_catalog_sources,
_checkout_frozen_source,
_flatpak_proxy_environment,
_run_isolated_wheel_builder,
_trusted_flatpak_builder_proxy,
_trusted_host_bubblewrap,
_verify_exact_publication_delta,
isolated_builder_launcher,
)
from govoplan_release.selective_catalog import ( # noqa: E402
enforce_frozen_selected_source_provenance,
enforce_selected_source_provenance,
)
class ReleaseExecutionTests(unittest.TestCase):
def test_flatpak_proxy_environment_drops_unrelated_values(self) -> None:
with patch.dict(
os.environ,
{
"DBUS_SESSION_BUS_ADDRESS": "unix:path=/attacker/bus",
"XDG_RUNTIME_DIR": "/run/user/1000",
"PYTHONPATH": "/attacker",
"GIT_DIR": "/attacker/repository",
},
clear=True,
):
environment = _flatpak_proxy_environment()
self.assertEqual("/usr/bin:/bin", environment["PATH"])
self.assertEqual(
"unix:path=/run/flatpak/bus",
environment["DBUS_SESSION_BUS_ADDRESS"],
)
self.assertNotIn("XDG_RUNTIME_DIR", environment)
self.assertNotIn("PYTHONPATH", environment)
self.assertNotIn("GIT_DIR", environment)
def test_host_bubblewrap_preflight_requires_root_owned_safe_binary(self) -> None:
with patch(
"govoplan_release.release_execution.subprocess.run",
return_value=SimpleNamespace(
returncode=0,
stdout="81ed|0\n",
),
):
self.assertTrue(_trusted_host_bubblewrap())
with patch(
"govoplan_release.release_execution.subprocess.run",
return_value=SimpleNamespace(
returncode=0,
stdout="81ff|1000\n",
),
):
self.assertFalse(_trusted_host_bubblewrap())
def test_builder_launcher_uses_only_preflighted_flatpak_proxy(self) -> None:
with patch(
"govoplan_release.release_execution._trusted_flatpak_builder_proxy",
return_value=True,
):
self.assertEqual(
("/usr/bin/flatpak-spawn", "--host", "/usr/bin/bwrap"),
isolated_builder_launcher(),
)
def test_flatpak_proxy_requires_recognized_container(self) -> None:
with patch.dict(os.environ, {}, clear=True):
self.assertFalse(_trusted_flatpak_builder_proxy())
def test_flatpak_builder_launch_uses_sanitized_environment(self) -> None:
process = SimpleNamespace(wait=lambda timeout: 0)
command = (
"/usr/bin/flatpak-spawn",
"--host",
"/usr/bin/bwrap",
)
with (
patch.dict(
os.environ,
{
"DBUS_SESSION_BUS_ADDRESS": "unix:path=/attacker/bus",
"PYTHONPATH": "/attacker",
},
clear=True,
),
patch(
"govoplan_release.release_execution.subprocess.Popen",
return_value=process,
) as popen,
):
self.assertEqual(0, _run_isolated_wheel_builder(command))
environment = popen.call_args.kwargs["env"]
self.assertEqual(
"unix:path=/run/flatpak/bus",
environment["DBUS_SESSION_BUS_ADDRESS"],
)
self.assertNotIn("PYTHONPATH", environment)
def test_git_environment_ignores_inherited_redirection_and_commands(self) -> None:
with patch.dict(
"os.environ",
{
"GIT_DIR": "/attacker/repository",
"GIT_CONFIG_GLOBAL": "/attacker/config",
"GIT_SSH_COMMAND": "/attacker/ssh",
"PATH": "/attacker/bin",
"SSH_AUTH_SOCK": "/operator/agent.sock",
},
clear=False,
):
environment = sanitized_git_environment()
self.assertNotIn("GIT_DIR", environment)
self.assertEqual("/dev/null", environment["GIT_CONFIG_GLOBAL"])
self.assertEqual("/usr/bin:/bin", environment["PATH"])
self.assertEqual(
"/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8",
environment["GIT_SSH_COMMAND"],
)
self.assertEqual("/operator/agent.sock", environment["SSH_AUTH_SOCK"])
def test_publication_reconciliation_rejects_unrelated_commit_delta(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
website = Path(temp_dir) / "website"
_git(Path(temp_dir), "init", "-b", "main", str(website))
_git(website, "config", "user.name", "Release Test")
_git(website, "config", "user.email", "release@example.test")
catalog = website / "public/catalogs/v1/channels/stable.json"
keyring = website / "public/catalogs/v1/keyring.json"
modules = website / "public/catalogs/v1/modules"
catalog.parent.mkdir(parents=True)
modules.mkdir(parents=True)
catalog.write_text("{}\n", encoding="utf-8")
keyring.write_text("{}\n", encoding="utf-8")
(website / "README.md").write_text("before\n", encoding="utf-8")
_git(website, "add", ".")
_git(website, "commit", "-m", "Base")
parent = _git_output(website, "rev-parse", "HEAD")
expected = {
"public/catalogs/v1/channels/stable.json": b'{"new": true}\n',
"public/catalogs/v1/keyring.json": b"{}\n",
}
catalog.write_bytes(expected["public/catalogs/v1/channels/stable.json"])
(website / "README.md").write_text("unrelated\n", encoding="utf-8")
_git(website, "add", ".")
_git(website, "commit", "-m", "Catalog plus unrelated change")
commit = _git_output(website, "rev-parse", "HEAD")
with self.assertRaisesRegex(
ReleaseExecutionBlocked,
"outside the exact derived delta",
):
_verify_exact_publication_delta(
web_root=website,
frozen_parent=parent,
commit=commit,
expected_blobs=expected,
module_root=modules,
)
def test_runtime_trust_rejects_replaceable_workspace_boundary(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
meta = root / "govoplan"
workspace = root / "workspace"
_git(root, "init", "-b", "main", str(meta))
(meta / "tools" / "release").mkdir(parents=True)
(meta / "tools" / "checks").mkdir(parents=True)
(meta / "tools" / "release" / "tool.py").write_text(
"# trusted\n", encoding="utf-8"
)
(meta / "tools" / "checks" / "check.py").write_text(
"# trusted\n", encoding="utf-8"
)
registry = meta / "repositories.json"
registry.write_text('{"repositories": []}\n', encoding="utf-8")
workspace.mkdir(mode=0o700)
with (
patch(
"govoplan_release.release_execution.META_ROOT",
meta,
),
patch(
"govoplan_release.release_execution.REPOSITORIES_FILE",
registry,
),
patch(
"govoplan_release.release_execution.publication_runtime_trust_issues",
return_value=(),
),
):
require_trusted_release_runtime(workspace_root=workspace)
workspace.chmod(0o777)
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "operator-owned"
):
require_trusted_release_runtime(workspace_root=workspace)
def test_durable_binding_rejects_group_writable_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("release\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
(repository / ".git" / "config").chmod(0o664)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(_repository_spec(remote),),
):
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "group/world writable"
):
repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
def test_durable_binding_rejects_nested_writable_git_authority(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("release\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
(repository / ".git" / "refs" / "heads").chmod(0o777)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(_repository_spec(remote),),
):
with self.assertRaisesRegex(
ReleaseExecutionBlocked,
"Git metadata directory",
):
repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
def test_creation_binding_rejects_head_and_push_remote_drift(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("one\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "-u", "origin", "main")
spec = _repository_spec(remote)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(spec,),
):
frozen = repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
step = {
"repo": "govoplan-files",
"source_binding": frozen,
}
(repository / "README.md").write_text("two\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Drift")
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "state drifted"
):
verify_repository_preflight_binding(
plan_step=step,
version="1.2.3",
workspace_root=workspace,
)
_git(
repository,
"remote",
"set-url",
"--add",
"--push",
"origin",
str(workspace / "other.git"),
)
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "registered release remote"
):
repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
def test_reconciliation_proves_annotated_local_and_remote_tag(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "README.md").write_text("release\n", encoding="utf-8")
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Initial")
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "-u", "origin", "main")
spec = _repository_spec(remote)
plan_step = {
"repo": "govoplan-files",
"source_binding": {},
}
tag_spec = ExecutorSpec("tag", "TAG", "tag_created", True)
push_spec = ExecutorSpec("push", "PUBLISH", "tag_published", True)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=(spec,),
):
before = repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
_git(repository, "tag", "-a", "v1.2.3", "-m", "Release")
tagged = reconciled_repository_receipt(
spec=tag_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=before,
)
self.assertIsNotNone(tagged["tag_object"])
with self.assertRaisesRegex(
ReleaseExecutionBlocked, "Remote annotated tag"
):
reconciled_repository_receipt(
spec=push_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=tagged,
)
_git(
repository,
"push",
"--atomic",
"origin",
"HEAD:refs/heads/main",
"refs/tags/v1.2.3",
)
published = reconciled_repository_receipt(
spec=push_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=tagged,
)
self.assertEqual(tagged, published)
(repository / "README.md").write_text(
"uncommitted drift\n", encoding="utf-8"
)
self.assertEqual(
tagged,
verify_frozen_repository_tag_receipt(
receipt=tagged,
workspace_root=workspace,
),
)
(repository / "README.md").write_text(
"remote branch drift\n", encoding="utf-8"
)
_git(repository, "add", "README.md")
_git(repository, "commit", "-m", "Remote branch drift")
drift_commit = _git_output(repository, "rev-parse", "HEAD")
_git(
repository,
"push",
"origin",
f"{drift_commit}:refs/heads/drift-source",
)
_git(remote, "update-ref", "refs/heads/main", drift_commit)
_git(repository, "reset", "--hard", str(tagged["head"]))
with self.assertRaisesRegex(
ReleaseExecutionBlocked,
"Remote branch",
):
reconciled_repository_receipt(
spec=push_spec,
plan_step=plan_step,
version="1.2.3",
workspace_root=workspace,
expected_receipt=tagged,
)
def test_post_tag_receipt_failure_is_ambiguous(self) -> None:
spec = ExecutorSpec("tag", "TAG", "tag_created", True)
plan_step = {
"id": "govoplan-files:tag",
"repo": "govoplan-files",
"source_binding": {"kind": "repository_state"},
}
with (
patch(
"govoplan_release.release_execution.require_trusted_release_runtime"
),
patch(
"govoplan_release.release_execution.verify_repository_step_precondition"
),
patch(
"govoplan_release.release_execution.tag_repositories",
return_value={
"status": "tagged",
"repositories": [{"status": "tagged"}],
},
),
patch(
"govoplan_release.release_execution.repository_state_receipt",
side_effect=ReleaseExecutionBlocked("post-effect probe failed"),
),
):
with self.assertRaisesRegex(
ReleaseExecutionAmbiguous, "post-effect repository receipt"
):
execute_repository_step(
spec=spec,
plan_step=plan_step,
repo_versions={"govoplan-files": "1.2.3"},
workspace_root=Path("/workspace"),
remote="origin",
expected_receipt={"kind": "repository_state"},
)
def test_commit_step_has_no_durable_executor(self) -> None:
self.assertIsNone(
executor_spec(
{
"id": "govoplan-files:commit",
"status": "planned",
"mutating": True,
"repo": "govoplan-files",
"source_binding": {
"kind": "repository_state",
},
}
)
)
def test_frozen_checkout_uses_receipt_commit_not_live_worktree(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
repository.mkdir()
_git(repository, "init", "-b", "main")
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "version.txt").write_text("frozen\n", encoding="utf-8")
_git(repository, "add", "version.txt")
_git(repository, "commit", "-m", "Frozen")
frozen_commit = _git_output(repository, "rev-parse", "HEAD")
_git(repository, "tag", "-a", "v1.2.3", "-m", "Release")
tag_object = _git_output(repository, "rev-parse", "refs/tags/v1.2.3")
(repository / "version.txt").write_text("live drift\n", encoding="utf-8")
source_root = workspace / "isolated"
source_root.mkdir()
checkout = _checkout_frozen_source(
repo="govoplan-files",
source_remote=repository,
commit=frozen_commit,
tag="v1.2.3",
tag_object=tag_object,
source_root=source_root,
)
self.assertEqual(
"frozen\n", (checkout / "version.txt").read_text(encoding="utf-8")
)
def test_frozen_provenance_accepts_only_exact_clean_detached_tag(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
repository = workspace / "govoplan-files"
remote = workspace / "files.git"
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.3"\n',
encoding="utf-8",
)
_git(repository, "add", "pyproject.toml")
_git(repository, "commit", "-m", "Release")
commit = _git_output(repository, "rev-parse", "HEAD")
_git(repository, "tag", "-a", "v1.2.3", "-m", "Release")
tag_object = _git_output(
repository, "rev-parse", "refs/tags/v1.2.3"
)
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "origin", "main", "refs/tags/v1.2.3")
_git(repository, "checkout", "--detach", commit)
spec = _repository_spec(remote)
with (
patch(
"govoplan_release.source_provenance.load_repository_specs",
return_value=(spec,),
),
patch(
"govoplan_release.selective_catalog.load_repository_specs",
return_value=(spec,),
),
):
with self.assertRaisesRegex(ValueError, "named branch"):
enforce_selected_source_provenance(
repo_versions={"govoplan-files": "1.2.3"},
workspace=workspace,
)
observed = enforce_frozen_selected_source_provenance(
repo_versions={"govoplan-files": "1.2.3"},
expected_provenance={
"govoplan-files": {
"commit_sha": commit,
"tag_object_sha": tag_object,
}
},
workspace=workspace,
)
self.assertEqual(commit, observed["govoplan-files"]["commit_sha"])
self.assertEqual(
tag_object, observed["govoplan-files"]["tag_object_sha"]
)
def test_wheel_staging_skips_selected_tag_only_repository(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
python_repo = workspace / "govoplan-files"
tag_only_repo = workspace / "govoplan-meta-notes"
python_repo.mkdir()
tag_only_repo.mkdir()
(python_repo / "pyproject.toml").write_text(
'[project]\nname = "govoplan-files"\nversion = "1.2.3"\n',
encoding="utf-8",
)
_git(workspace, "init", "-b", "main", str(python_repo))
_git(python_repo, "config", "user.name", "Release Test")
_git(python_repo, "config", "user.email", "release@example.test")
_git(python_repo, "add", "pyproject.toml")
_git(python_repo, "commit", "-m", "Source")
specs = (
RepositorySpec(
name="govoplan-files",
category="module",
subtype="infrastructure",
remote="git@example.test/files.git",
path="govoplan-files",
),
RepositorySpec(
name="govoplan-meta-notes",
category="system",
subtype="metadata",
remote="git@example.test/notes.git",
path="govoplan-meta-notes",
),
)
candidate = workspace / "candidate"
def build(command: tuple[str, ...]) -> int:
wheel_dir = Path(command[command.index("--bind") + 1])
second_bind = command.index("--bind", command.index("--bind") + 1)
source_dir = Path(command[second_bind + 1])
(source_dir / "build-mutated.txt").write_text(
"isolated copy",
encoding="utf-8",
)
(
wheel_dir / "govoplan_files-1.2.3-py3-none-any.whl"
).write_bytes(b"wheel")
return 0
with (
patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=specs,
),
patch(
"govoplan_release.release_execution.isolated_builder_launcher",
return_value=("/usr/bin/bwrap",),
),
patch(
"govoplan_release.release_execution._run_isolated_wheel_builder",
side_effect=build,
) as runner,
patch(
"govoplan_release.release_execution.inspect_python_wheel",
return_value=SimpleNamespace(
package_name="govoplan-files",
package_version="1.2.3",
),
),
):
artifacts = build_selected_wheels(
repo_versions={
"govoplan-files": "1.2.3",
"govoplan-meta-notes": "1.0.0",
},
workspace_root=workspace,
candidate_output=candidate,
source_receipts={
"govoplan-files": {
"head": "a" * 40,
"target_tag": "v1.2.3",
"tag_object": "c" * 40,
},
"govoplan-meta-notes": {
"head": "b" * 40,
"target_tag": "v1.0.0",
"tag_object": "d" * 40,
},
},
frozen_sources={
"govoplan-files": python_repo,
"govoplan-meta-notes": tag_only_repo,
},
)
self.assertFalse((python_repo / "build-mutated.txt").exists())
self.assertEqual({"govoplan-files"}, set(artifacts))
self.assertEqual(1, runner.call_count)
command = runner.call_args.args[0]
self.assertNotIn(str(python_repo), command)
self.assertTrue(
any(
value.startswith(str(candidate / "artifacts"))
and value.endswith("/source")
for value in command
)
)
self.assertIn("--unshare-all", command)
self.assertIn("PIP_NO_INDEX", command)
self.assertNotIn(str(Path.home()), command)
def test_catalog_synthesis_clones_selected_and_unchanged_base_sources(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
files_repo, files_remote = _tagged_repository(
workspace,
repo="govoplan-files",
bare="files.git",
version="1.2.3",
)
core_repo, core_remote = _tagged_repository(
workspace,
repo="govoplan-core",
bare="core.git",
version="1.0.0",
)
specs = (
_repository_spec(files_remote),
RepositorySpec(
name="govoplan-core",
category="core",
subtype="core",
remote=str(core_remote),
path="govoplan-core",
),
)
destination = workspace / "synthesis"
destination.mkdir()
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=specs,
):
receipt = repository_state_receipt(
repo="govoplan-files",
target_tag="v1.2.3",
workspace_root=workspace,
)
cloned = _clone_catalog_sources(
source_versions={
"govoplan-core": "1.0.0",
"govoplan-files": "1.2.3",
},
repo_versions={"govoplan-files": "1.2.3"},
source_receipts={"govoplan-files": receipt},
workspace_root=workspace,
destination=destination,
)
self.assertEqual(
_git_output(files_repo, "rev-parse", "refs/tags/v1.2.3^{commit}"),
_git_output(cloned["govoplan-files"], "rev-parse", "HEAD"),
)
self.assertEqual(
_git_output(core_repo, "rev-parse", "refs/tags/v1.0.0^{commit}"),
_git_output(cloned["govoplan-core"], "rev-parse", "HEAD"),
)
self.assertEqual(
"HEAD",
_git_output(cloned["govoplan-core"], "rev-parse", "--abbrev-ref", "HEAD"),
)
def test_catalog_synthesis_adds_exact_core_bundle_dependency(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
idm_repo, idm_remote = _tagged_repository(
workspace,
repo="govoplan-idm",
bare="idm.git",
version="0.1.8",
)
idm_commit = _git_output(idm_repo, "rev-parse", "v0.1.8^{commit}")
(idm_repo / "pyproject.toml").write_text(
'[project]\nname = "govoplan-idm"\nversion = "0.1.9"\n',
encoding="utf-8",
)
_git(idm_repo, "add", "pyproject.toml")
_git(idm_repo, "commit", "-m", "Next release")
_git(idm_repo, "tag", "-a", "v0.1.9", "-m", "Next release")
_git(idm_repo, "push", "origin", "main", "refs/tags/v0.1.9")
catalog_commit = _git_output(idm_repo, "rev-parse", "v0.1.9^{commit}")
core_repo = workspace / "govoplan-core"
core_remote = workspace / "core.git"
_git(workspace, "init", "--bare", str(core_remote))
_git(workspace, "init", "-b", "main", str(core_repo))
_git(core_repo, "config", "user.name", "Release Test")
_git(core_repo, "config", "user.email", "release@example.test")
(core_repo / "webui").mkdir()
(core_repo / "pyproject.toml").write_text(
'[project]\nname = "govoplan-core"\nversion = "1.0.0"\n',
encoding="utf-8",
)
reference = (
"git+ssh://git@example.test/add-ideas/"
"govoplan-idm.git#v0.1.8"
)
(core_repo / "webui/package.release.json").write_text(
json.dumps(
{
"version": "1.0.0",
"dependencies": {"@govoplan/idm-webui": reference},
}
),
encoding="utf-8",
)
(core_repo / "webui/package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {
"version": "1.0.0",
"dependencies": {
"@govoplan/idm-webui": reference,
},
},
"node_modules/@govoplan/idm-webui": {
"version": "0.1.8",
"resolved": f"{reference.rsplit('#', 1)[0]}#{idm_commit}",
},
}
}
),
encoding="utf-8",
)
_git(core_repo, "add", ".")
_git(core_repo, "commit", "-m", "Release")
_git(core_repo, "tag", "-a", "v1.0.0", "-m", "Release")
_git(core_repo, "remote", "add", "origin", str(core_remote))
_git(core_repo, "push", "-u", "origin", "main", "refs/tags/v1.0.0")
specs = (
RepositorySpec(
name="govoplan-core",
category="system",
subtype="kernel",
remote=str(core_remote),
path="govoplan-core",
),
RepositorySpec(
name="govoplan-idm",
category="module",
subtype="platform",
remote=str(idm_remote),
path="govoplan-idm",
),
)
with patch(
"govoplan_release.release_execution.load_repository_specs",
return_value=specs,
):
synthesis = workspace / "synthesis"
synthesis.mkdir()
cloned = _clone_catalog_sources(
source_versions={"govoplan-core": "1.0.0"},
repo_versions={},
source_receipts={},
workspace_root=workspace,
destination=synthesis,
)
newer_synthesis = workspace / "newer-synthesis"
newer_synthesis.mkdir()
cloned_with_newer_catalog_source = _clone_catalog_sources(
source_versions={
"govoplan-core": "1.0.0",
"govoplan-idm": "0.1.9",
},
repo_versions={},
source_receipts={},
workspace_root=workspace,
destination=newer_synthesis,
)
self.assertEqual(
{"govoplan-core", "govoplan-idm"},
set(cloned),
)
self.assertEqual(
idm_commit,
_git_output(cloned["govoplan-idm"], "rev-parse", "HEAD"),
)
self.assertEqual(
catalog_commit,
_git_output(
cloned_with_newer_catalog_source["govoplan-idm"],
"rev-parse",
"HEAD",
),
)
self.assertEqual(
idm_commit,
_git_output(
cloned_with_newer_catalog_source["govoplan-idm"],
"rev-parse",
"v0.1.8^{commit}",
),
)
def _repository_spec(remote: Path) -> RepositorySpec:
return RepositorySpec(
name="govoplan-files",
category="module",
subtype="infrastructure",
remote=str(remote),
path="govoplan-files",
)
def _tagged_repository(
workspace: Path, *, repo: str, bare: str, version: str
) -> tuple[Path, Path]:
repository = workspace / repo
remote = workspace / bare
_git(workspace, "init", "--bare", str(remote))
_git(workspace, "init", "-b", "main", str(repository))
_git(repository, "config", "user.name", "Release Test")
_git(repository, "config", "user.email", "release@example.test")
(repository / "pyproject.toml").write_text(
f'[project]\nname = "{repo}"\nversion = "{version}"\n',
encoding="utf-8",
)
_git(repository, "add", "pyproject.toml")
_git(repository, "commit", "-m", "Release")
_git(repository, "tag", "-a", f"v{version}", "-m", "Release")
_git(repository, "remote", "add", "origin", str(remote))
_git(repository, "push", "-u", "origin", "main", f"refs/tags/v{version}")
return repository, remote
def _git(cwd: Path, *arguments: str) -> None:
subprocess.run( # noqa: S603
("git", *arguments),
cwd=cwd,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
def _git_output(cwd: Path, *arguments: str) -> str:
return subprocess.run( # noqa: S603
("git", *arguments),
cwd=cwd,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
).stdout.strip()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,39 @@
from __future__ import annotations
from pathlib import Path
import subprocess
import sys
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release import git_state # noqa: E402
class ReleaseGitStateTests(unittest.TestCase):
def test_git_trusts_only_the_resolved_repository_for_each_command(self) -> None:
repository = Path("/workspace/../workspace/govoplan-core")
completed = subprocess.CompletedProcess([], 0, "", "")
with patch.object(git_state.subprocess, "run", return_value=completed) as run:
result = git_state.git(repository, "status", "--porcelain")
self.assertEqual(result.returncode, 0)
command = run.call_args.args[0]
self.assertIn(f"safe.directory={repository.resolve()}", command)
self.assertNotIn("safe.directory=*", command)
self.assertEqual(run.call_args.kwargs["cwd"], repository)
self.assertEqual(
run.call_args.kwargs["env"]["GIT_CONFIG_GLOBAL"],
"/dev/null",
)
if __name__ == "__main__":
unittest.main()

View File

@@ -1,8 +1,10 @@
from __future__ import annotations
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from tools.checks.release_integration import (
SOURCE_COUPLED_CORE_TESTS,
@@ -10,6 +12,7 @@ from tools.checks.release_integration import (
artifact_contract_issues,
filter_test_suite,
release_package_names,
run_module_release_tests,
)
@@ -32,9 +35,9 @@ class ReleaseIntegrationTests(unittest.TestCase):
"\n".join(
(
"../govoplan-core[server]",
"govoplan-idm @ git+ssh://git@example.test/add-ideas/govoplan-idm.git@v0.1.8",
"govoplan-campaign @ git+ssh://git@example.test/add-ideas/govoplan-campaign.git@v0.1.8",
"govoplan-idm @ git+ssh://git@example.test/add-ideas/govoplan-idm.git@v0.1.8",
"govoplan-idm @ git+ssh://git@example.test/GovOPlaN/govoplan-idm.git@v0.1.8",
"govoplan-campaign @ git+ssh://git@example.test/GovOPlaN/govoplan-campaign.git@v0.1.8",
"govoplan-idm @ git+ssh://git@example.test/GovOPlaN/govoplan-idm.git@v0.1.8",
"other-package==1.0",
)
),
@@ -89,8 +92,67 @@ class ReleaseIntegrationTests(unittest.TestCase):
self.assertLess(artifact_check, core_tests)
self.assertLess(core_tests, module_tests)
self.assertNotIn("unittest discover", script)
self.assertNotIn('"$PYTHON" -m unittest \\\n tests.test_module_system', script)
def test_tagged_module_runner_executes_pytest_functions_from_module_root(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-release-module-tests-") as temp_dir:
module_root = Path(temp_dir)
tests_root = module_root / "tests"
tests_root.mkdir()
marker = module_root / "pytest-function-ran"
(tests_root / "test_probe.py").write_text(
"""from pathlib import Path
import os
def test_pytest_style_function():
expected_root = Path(os.environ[\"GOVOPLAN_TEST_MODULE_ROOT\"])
assert Path.cwd() == expected_root
Path(os.environ[\"GOVOPLAN_TEST_MARKER\"]).write_text(\"ran\", encoding=\"utf-8\")
""",
encoding="utf-8",
)
with patch.dict(
os.environ,
{
"GOVOPLAN_TEST_MODULE_ROOT": str(module_root),
"GOVOPLAN_TEST_MARKER": str(marker),
},
):
status = run_module_release_tests(module_root)
self.assertEqual(status, 0)
self.assertEqual(marker.read_text(encoding="utf-8"), "ran")
def test_tagged_module_runner_skips_missing_test_tree(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-release-module-tests-") as temp_dir:
self.assertEqual(run_module_release_tests(Path(temp_dir)), 0)
def test_module_matrix_uses_artifact_aware_core_suite(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
script = (meta_root / "tools" / "checks" / "check-module-matrix.sh").read_text(encoding="utf-8")
artifact_check = script.index('"$META_ROOT/tools/checks/release_integration.py" artifacts')
core_tests = script.index('"$META_ROOT/tools/checks/release_integration.py" core-tests')
self.assertLess(artifact_check, core_tests)
self.assertIn('--requirements "$META_ROOT/requirements-release.txt"', script)
self.assertIn('--core-root "$ROOT"', script)
self.assertNotIn('"$PYTHON" -m unittest tests.test_module_system', script)
def test_release_workflow_installs_tagged_source_test_harness(self) -> None:
meta_root = Path(__file__).resolve().parents[1]
workflow = (meta_root / ".gitea" / "workflows" / "release-integration.yml").read_text(encoding="utf-8")
requirements = (meta_root / "requirements-release-tests.txt").read_text(encoding="utf-8")
install = workflow.index("pip install -r requirements-release-tests.txt")
checks = workflow.index("bash tools/checks/check-release-integration.sh")
self.assertLess(install, checks)
self.assertIn("pytest>=9.0.3,<10", requirements)
self.assertNotIn("requirements-release-tests.txt", (meta_root / "requirements-release.txt").read_text(encoding="utf-8"))
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,82 @@
from __future__ import annotations
from pathlib import Path
import sys
import unittest
from unittest import mock
META_ROOT = Path(__file__).resolve().parents[1]
RELEASE_TOOLS_ROOT = META_ROOT / "tools" / "release"
if str(RELEASE_TOOLS_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_TOOLS_ROOT))
from govoplan_release.module_directory import ( # noqa: E402
module_directory_payloads,
safe_path_part,
)
class ReleaseModuleDirectoryTests(unittest.TestCase):
def test_signed_catalog_timestamp_makes_derived_files_reproducible(self) -> None:
catalog = {
"generated_at": "2026-07-22T12:00:00Z",
"sequence": 7,
"modules": [],
}
first = module_directory_payloads(
catalog_payload=catalog,
keyring_payload={},
channel="stable",
)
second = module_directory_payloads(
catalog_payload=catalog,
keyring_payload={},
channel="stable",
)
self.assertEqual(first, second)
self.assertEqual(
"2026-07-22T12:00:00Z",
first[-1][1]["generated_at"],
)
def test_dot_segments_and_noncanonical_ids_never_become_paths(self) -> None:
for value in (".", "..", "../escape", "/absolute", "module/child"):
with self.subTest(value=value), self.assertRaises(ValueError):
safe_path_part(value)
for module_id in ("..", "../escape", "UPPER", "bad.id"):
with self.subTest(module_id=module_id), mock.patch(
"govoplan_release.module_directory.module_rows",
return_value=[
{
"module_id": module_id,
"version": "1.2.3",
}
],
):
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={},
keyring_payload={},
channel="stable",
)
def test_noncanonical_version_and_channel_fail_before_paths(self) -> None:
with mock.patch(
"govoplan_release.module_directory.module_rows",
return_value=[{"module_id": "demo", "version": "../1.2.3"}],
):
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={}, keyring_payload={}, channel="stable"
)
with self.assertRaises(ValueError):
module_directory_payloads(
catalog_payload={}, keyring_payload={}, channel="../stable"
)
if __name__ == "__main__":
unittest.main()

View File

@@ -156,7 +156,7 @@ def dashboard(*, workspace: Path, version: str) -> ReleaseDashboard:
name="govoplan-files",
category="module",
subtype="infrastructure",
remote="git@example.test:add-ideas/govoplan-files.git",
remote="git@example.test:GovOPlaN/govoplan-files.git",
path="govoplan-files",
),
absolute_path=str(workspace / "govoplan-files"),

View File

@@ -230,10 +230,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
for repository in (self.repo, self.remote, campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_api_requires_explicit_confirmation_and_ui_exposes_source_release(self) -> None:
with TestClient(create_app(workspace_root=self.workspace)) as client:
def test_api_keeps_legacy_source_release_preview_only(self) -> None:
with TestClient(
create_app(workspace_root=self.workspace, token="token")
) as client:
headers = {"X-Release-Console-Token": "token"}
rejected = client.post(
"/api/repositories/tag",
headers=headers,
json={
"repos": ["govoplan-core"],
"repo_versions": {"govoplan-core": "0.1.10"},
@@ -244,6 +248,7 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
)
preview = client.post(
"/api/repositories/tag",
headers=headers,
json={
"repos": ["govoplan-core"],
"repo_versions": {"govoplan-core": "0.1.10"},
@@ -251,14 +256,47 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
"push": True,
},
)
legacy_push = client.post(
"/api/repositories/push",
headers=headers,
json={
"repos": ["govoplan-core"],
"apply": True,
"confirm": "PUSH",
},
)
legacy_sync = client.post(
"/api/repositories/sync",
headers=headers,
json={
"repos": ["govoplan-core"],
"apply": True,
"confirm": "SYNC",
},
)
ui = client.get("/")
self.assertEqual(rejected.status_code, 400)
self.assertEqual(rejected.status_code, 409)
self.assertIn("durable release run", rejected.json()["detail"])
self.assertEqual(preview.status_code, 200)
self.assertEqual(409, legacy_push.status_code)
self.assertEqual(409, legacy_sync.status_code)
self.assertIn("durable", legacy_push.json()["detail"])
self.assertIn("durable", legacy_sync.json()["detail"])
self.assertEqual(preview.json()["repositories"][0]["repo"], "govoplan-core")
self.assertFalse(ref_exists(self.repo, "refs/tags/v0.1.10"))
self.assertIn('id="publishReleaseTags"', ui.text)
self.assertIn(
'id="publishReleaseTags" data-release-disabled="true" disabled',
ui.text,
)
self.assertIn('postJson("/api/repositories/tag"', ui.text)
self.assertIn(
'id="syncRepos" disabled data-release-disabled="true"', ui.text
)
self.assertIn(
'id="pushRepos" disabled data-release-disabled="true"', ui.text
)
self.assertIn("Signed Website Catalog", ui.text)
self.assertIn("Apply + Website Tag", ui.text)

View File

@@ -32,6 +32,12 @@ from govoplan_release.release_run import ( # noqa: E402
ReleaseRunStore,
_record_digest,
)
from govoplan_release.candidate_artifact import ( # noqa: E402
CandidateArtifactReceipt,
candidate_output_path,
harden_private_candidate_tree,
issue_candidate_receipt,
)
from server.app import ( # noqa: E402
create_app,
default_release_run_root,
@@ -51,8 +57,24 @@ class ReleaseRunStoreTests(unittest.TestCase):
self.root,
expected_workspace_fingerprint=WORKSPACE_FINGERPRINT,
)
self.runtime_trust = patch(
"server.app.require_trusted_release_runtime"
)
self.runtime_binding = patch(
"server.app.verify_release_runtime_binding"
)
self.source_bindings = patch(
"server.app.bind_plan_source_states",
side_effect=bind_test_plan_source_states,
)
self.runtime_trust.start()
self.runtime_binding.start()
self.source_bindings.start()
def tearDown(self) -> None:
self.source_bindings.stop()
self.runtime_binding.stop()
self.runtime_trust.stop()
self.temporary.cleanup()
def test_create_persists_private_immutable_plan_and_ordered_steps(self) -> None:
@@ -68,7 +90,7 @@ class ReleaseRunStoreTests(unittest.TestCase):
record_path = self.root / f"{run['run_id']}.json"
self.assertEqual("govoplan.release-run", reopened["schema"])
self.assertEqual(3, reopened["schema_version"])
self.assertEqual(4, reopened["schema_version"])
self.assertEqual(
{"govoplan-core": "1.2.3", "govoplan-files": "1.2.4"},
reopened["immutable"]["input"]["repo_versions"],
@@ -82,6 +104,119 @@ class ReleaseRunStoreTests(unittest.TestCase):
self.assertNotIn("processed_requests", reopened["state"])
self.assertNotIn("attempt_fingerprint", reopened["state"]["steps"][0])
def test_catalog_receipt_is_bounded_persisted_and_idempotent(self) -> None:
run_id = create_run(
self.store,
plan_snapshot=mutating_first_plan(),
request_id="receipt-create-request-0001",
)["run_id"]
attempt_id = "receipt-attempt-request-0001"
self.store.start_step(run_id, "core:tag", attempt_id=attempt_id)
receipt = {
"kind": "catalog_candidate",
"candidate_id": "candidate-" + "a" * 32,
"catalog_sha256": "b" * 64,
}
finished = self.store.finish_step(
run_id,
"core:tag",
attempt_id=attempt_id,
succeeded=True,
result_code="catalog_candidate_ready",
result_receipt=receipt,
)
replayed = self.store.finish_step(
run_id,
"core:tag",
attempt_id=attempt_id,
succeeded=True,
result_code="catalog_candidate_ready",
result_receipt=receipt,
)
self.assertEqual(receipt, finished["state"]["steps"][0]["result_receipt"])
self.assertEqual(finished, replayed)
with self.assertRaisesRegex(ReleaseRunConflict, "different recorded outcome"):
self.store.finish_step(
run_id,
"core:tag",
attempt_id=attempt_id,
succeeded=True,
result_code="catalog_candidate_ready",
result_receipt=receipt | {"catalog_sha256": "c" * 64},
)
def test_catalog_publication_receipt_binds_candidate_origin_commit_and_tag(
self,
) -> None:
run_id = create_run(
self.store,
plan_snapshot=mutating_first_plan(),
request_id="publication-receipt-create-0001",
)["run_id"]
attempt_id = "publication-receipt-attempt-0001"
self.store.start_step(run_id, "core:tag", attempt_id=attempt_id)
receipt = {
"kind": "catalog_publication",
"candidate_id": "candidate-" + "a" * 32,
"catalog_sha256": "b" * 64,
"keyring_sha256": "c" * 64,
"publication_commit_sha": "d" * 40,
"publication_tag_object_sha": "e" * 40,
"publication_tag_commit_sha": "d" * 40,
"branch": "main",
"tag_name": "catalog-stable-1",
"remote": "origin",
"remote_sha256": "f" * 64,
}
finished = self.store.finish_step(
run_id,
"core:tag",
attempt_id=attempt_id,
succeeded=True,
result_code="catalog_published",
result_receipt=receipt,
)
self.assertEqual(receipt, finished["state"]["steps"][0]["result_receipt"])
with self.assertRaisesRegex(
ReleaseRunConflict, "publication receipt identity"
):
self.store.finish_step(
run_id,
"core:tag",
attempt_id=attempt_id,
succeeded=True,
result_code="catalog_published",
result_receipt=receipt
| {"publication_tag_commit_sha": "0" * 40},
)
def test_schema_three_record_is_verified_then_migrated_privately(self) -> None:
run_id = create_run(
self.store,
request_id="legacy-schema-create-request-0001",
)["run_id"]
path = self.root / f"{run_id}.json"
payload = json.loads(path.read_text(encoding="utf-8"))
payload["schema_version"] = 3
for step in payload["state"]["steps"]:
step.pop("result_receipt")
payload["record_digest"] = _record_digest(payload)
path.write_text(json.dumps(payload), encoding="utf-8")
path.chmod(0o600)
migrated = self.store.get(run_id)
persisted = json.loads(path.read_text(encoding="utf-8"))
self.assertEqual(4, migrated["schema_version"])
self.assertEqual(4, persisted["schema_version"])
self.assertTrue(
all(step["result_receipt"] is None for step in persisted["state"]["steps"])
)
def test_zero_step_plan_is_rejected(self) -> None:
plan = release_plan()
plan["dry_run_steps"] = []
@@ -293,7 +428,7 @@ class ReleaseRunStoreTests(unittest.TestCase):
self.assertEqual(0, untouched["state"]["steps"][0]["attempt_count"])
accepted_plan = mutating_first_plan()
accepted_plan["notes"] = ["x" * (MAX_RECORD_BYTES - 3_000)]
accepted_plan["notes"] = ["x" * (MAX_RECORD_BYTES - 4_300)]
accepted_run_id = create_run(
self.store,
request_id="byte-capacity-accepted-create-0001",
@@ -334,7 +469,7 @@ class ReleaseRunStoreTests(unittest.TestCase):
) -> None:
plan = mutating_first_plan()
plan["dry_run_steps"][0]["mutating"] = False # type: ignore[index]
plan["notes"] = ["x" * (MAX_RECORD_BYTES - 3_000)]
plan["notes"] = ["x" * (MAX_RECORD_BYTES - 5_500)]
run_id = create_run(
self.store,
request_id="byte-capacity-read-only-create-0001",
@@ -747,7 +882,7 @@ class ReleaseRunStoreTests(unittest.TestCase):
listed = self.store.list(limit=1)
self.assertEqual([local_run["run_id"]], [item["run_id"] for item in listed])
def test_list_orders_verified_runs_by_updated_timestamp_before_unavailable(
def test_list_orders_verified_runs_by_creation_timestamp_before_unavailable(
self,
) -> None:
with patch(
@@ -786,11 +921,139 @@ class ReleaseRunStoreTests(unittest.TestCase):
listed = self.store.list(limit=3)
self.assertEqual(
[older["run_id"], newer["run_id"], corrupt["run_id"]],
[newer["run_id"], older["run_id"], corrupt["run_id"]],
[item["run_id"] for item in listed],
)
self.assertEqual("unavailable", listed[-1]["status"])
self.assertEqual(older["run_id"], self.store.list(limit=1)[0]["run_id"])
self.assertEqual(newer["run_id"], self.store.list(limit=1)[0]["run_id"])
def test_cursor_pagination_is_stable_and_keeps_unavailable_records_visible(
self,
) -> None:
created: list[dict[str, object]] = []
for index in range(3):
with patch(
"govoplan_release.release_run._timestamp",
return_value=f"2026-07-22T0{index}:00:00Z",
):
created.append(
create_run(
self.store,
request_id=f"cursor-page-create-request-{index:04d}",
)
)
corrupt_path = self.root / f"{created[0]['run_id']}.json"
corrupt_path.write_text("{}", encoding="utf-8")
corrupt_path.chmod(0o600)
first = self.store.list_page(limit=1)
second = self.store.list_page(limit=1, cursor=first["next_cursor"])
third = self.store.list_page(limit=1, cursor=second["next_cursor"])
self.assertEqual(created[2]["run_id"], first["runs"][0]["run_id"])
self.assertEqual(created[1]["run_id"], second["runs"][0]["run_id"])
self.assertEqual("unavailable", third["runs"][0]["status"])
self.assertIsNone(third["next_cursor"])
with self.assertRaisesRegex(ReleaseRunConflict, "cursor is invalid"):
self.store.list_page(limit=1, cursor=f"{first['next_cursor']}tampered")
def test_cursor_does_not_skip_or_repeat_when_an_older_run_is_updated(
self,
) -> None:
created: list[dict[str, object]] = []
for index in range(3):
with patch(
"govoplan_release.release_run._timestamp",
return_value=f"2026-07-22T0{index}:00:00Z",
):
created.append(
create_run(
self.store,
request_id=f"cursor-update-create-request-{index:04d}",
)
)
first = self.store.list_page(limit=1)
with patch(
"govoplan_release.release_run._timestamp",
return_value="2026-07-22T12:00:00Z",
):
self.store.start_step(
created[0]["run_id"],
"core:preflight",
attempt_id="cursor-update-attempt-request-0001",
)
second = self.store.list_page(limit=1, cursor=first["next_cursor"])
third = self.store.list_page(limit=1, cursor=second["next_cursor"])
self.assertEqual(
[item["run_id"] for item in reversed(created)],
[
first["runs"][0]["run_id"],
second["runs"][0]["run_id"],
third["runs"][0]["run_id"],
],
)
def test_retention_prunes_only_oldest_completed_records_and_fails_closed(
self,
) -> None:
with patch("govoplan_release.release_run.MAX_RUN_RECORDS", 3):
completed: list[dict[str, object]] = []
for index in range(2):
with patch(
"govoplan_release.release_run._timestamp",
return_value=f"2026-07-22T0{index}:00:00Z",
):
run = create_run(
self.store,
request_id=f"retention-completed-create-{index:04d}",
plan_snapshot=mutating_first_plan(),
)
attempt_id = f"retention-completed-attempt-{index:04d}"
self.store.start_step(
run["run_id"], "core:tag", attempt_id=attempt_id
)
self.store.finish_step(
run["run_id"],
"core:tag",
attempt_id=attempt_id,
succeeded=True,
result_code="tag_created",
)
completed.append(run)
active = create_run(
self.store,
request_id="retention-active-create-0001",
)
replacement = create_run(
self.store,
request_id="retention-replacement-create-0001",
)
self.assertFalse(
(self.root / f"{completed[0]['run_id']}.json").exists()
)
self.assertTrue((self.root / f"{completed[1]['run_id']}.json").exists())
self.assertTrue((self.root / f"{active['run_id']}.json").exists())
self.assertTrue((self.root / f"{replacement['run_id']}.json").exists())
corrupt_path = self.root / f"{completed[1]['run_id']}.json"
corrupt_path.write_text("{}", encoding="utf-8")
corrupt_path.chmod(0o600)
with self.assertRaisesRegex(ReleaseRunConflict, "retention limit"):
create_run(
self.store,
request_id="retention-refused-create-0001",
)
self.assertEqual(3, len(list(self.root.glob("*.json"))))
self.assertIn(
completed[1]["run_id"],
{
item["run_id"]
for item in self.store.list_page(limit=3)["runs"]
if item["status"] == "unavailable"
},
)
def test_list_enumeration_oserror_is_normalized(self) -> None:
self.store.list()
@@ -872,6 +1135,399 @@ class ReleaseRunStoreTests(unittest.TestCase):
class ReleaseRunApiTests(unittest.TestCase):
def setUp(self) -> None:
self.runtime_trust = patch("server.app.require_trusted_release_runtime")
self.runtime_binding = patch("server.app.verify_release_runtime_binding")
self.source_bindings = patch(
"server.app.bind_plan_source_states",
side_effect=bind_test_plan_source_states,
)
self.runtime_trust.start()
self.runtime_binding.start()
self.source_bindings.start()
def tearDown(self) -> None:
self.source_bindings.stop()
self.runtime_binding.stop()
self.runtime_trust.stop()
def test_api_exposes_and_validates_cursor_pagination(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
app = create_app(
workspace_root=Path(temp_dir),
run_state_root=Path(temp_dir) / "runs",
token="token",
)
headers = {"X-Release-Console-Token": "token"}
with (
patch("server.app.build_dashboard", return_value=object()),
patch(
"server.app.build_selective_release_plan",
return_value=release_plan(),
),
TestClient(app) as client,
):
for index in range(2):
response = client.post(
"/api/release-runs",
headers=headers,
json=api_run_input(
request_id=f"api-page-create-request-{index:04d}"
),
)
self.assertEqual(200, response.status_code)
first = client.get(
"/api/release-runs?limit=1", headers=headers
).json()
second = client.get(
"/api/release-runs",
headers=headers,
params={"limit": 1, "cursor": first["next_cursor"]},
).json()
invalid = client.get(
"/api/release-runs?cursor=not-a-cursor", headers=headers
)
self.assertEqual(1, len(first["runs"]))
self.assertEqual(1, len(second["runs"]))
self.assertNotEqual(
first["runs"][0]["run_id"], second["runs"][0]["run_id"]
)
self.assertIsNone(second["next_cursor"])
self.assertEqual(409, invalid.status_code)
def test_executor_claims_before_effect_and_replays_lost_success_response(
self,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
app = create_app(
workspace_root=Path(temp_dir),
run_state_root=Path(temp_dir) / "runs",
candidate_root=Path(temp_dir) / "candidates",
token="token",
)
headers = {"X-Release-Console-Token": "token"}
with (
patch("server.app.build_dashboard", return_value=object()),
patch(
"server.app.build_selective_release_plan",
return_value=release_plan(),
),
patch(
"server.app.bind_plan_source_states",
side_effect=bind_test_plan_source_states,
),
patch(
"server.app.verify_repository_preflight_binding",
return_value=repository_receipt(),
),
patch(
"server.app.execute_repository_step",
return_value=({"status": "inspected"}, repository_receipt()),
) as executor,
TestClient(app) as client,
):
created = client.post(
"/api/release-runs",
headers=headers,
json=api_run_input(request_id="api-executor-create-0001"),
).json()
run_id = created["run_id"]
self.assertTrue(created["state"]["steps"][0]["executor"]["available"])
self.assertEqual(
"preflight", created["state"]["steps"][0]["executor"]["kind"]
)
payload = {"request_id": "api-executor-attempt-0001"}
executed = client.post(
f"/api/release-runs/{run_id}/steps/core:preflight/execute",
headers=headers,
json=payload,
)
replayed = client.post(
f"/api/release-runs/{run_id}/steps/core:preflight/execute",
headers=headers,
json=payload,
)
self.assertEqual(200, executed.status_code)
self.assertEqual("succeeded", executed.json()["state"]["steps"][0]["state"])
self.assertEqual("replayed", replayed.json()["execution_result"]["status"])
executor.assert_called_once()
def test_lost_finish_write_interrupts_without_reexecuting_effect(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
app = create_app(
workspace_root=Path(temp_dir),
run_state_root=Path(temp_dir) / "runs",
token="token",
)
headers = {"X-Release-Console-Token": "token"}
with (
patch("server.app.build_dashboard", return_value=object()),
patch(
"server.app.build_selective_release_plan",
return_value=release_plan(),
),
patch(
"server.app.bind_plan_source_states",
side_effect=bind_test_plan_source_states,
),
patch(
"server.app.verify_repository_preflight_binding",
return_value=repository_receipt(),
),
patch(
"server.app.execute_repository_step",
return_value=({"status": "inspected"}, repository_receipt()),
) as executor,
TestClient(app) as client,
):
run_id = client.post(
"/api/release-runs",
headers=headers,
json=api_run_input(request_id="api-lost-finish-create-0001"),
).json()["run_id"]
with patch.object(
app.state.release_runs,
"finish_step",
side_effect=ReleaseRunCorrupt("simulated durable write failure"),
):
first = client.post(
f"/api/release-runs/{run_id}/steps/core:preflight/execute",
headers=headers,
json={"request_id": "api-lost-finish-attempt-0001"},
)
replay = client.post(
f"/api/release-runs/{run_id}/steps/core:preflight/execute",
headers=headers,
json={"request_id": "api-lost-finish-attempt-0001"},
)
loaded = client.get(
f"/api/release-runs/{run_id}", headers=headers
).json()
self.assertEqual(409, first.status_code)
self.assertEqual(409, replay.status_code)
self.assertEqual("interrupted", loaded["state"]["steps"][0]["state"])
executor.assert_called_once()
def test_executor_exception_is_interrupted_and_never_retried_implicitly(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
app = create_app(
workspace_root=Path(temp_dir),
run_state_root=Path(temp_dir) / "runs",
token="token",
)
headers = {"X-Release-Console-Token": "token"}
with (
patch("server.app.build_dashboard", return_value=object()),
patch(
"server.app.build_selective_release_plan",
return_value=mutating_first_plan(),
),
patch(
"server.app.bind_plan_source_states",
side_effect=bind_test_plan_source_states,
),
patch("server.app.verify_repository_step_precondition"),
patch(
"server.app.execute_repository_step",
side_effect=RuntimeError("connection lost after push"),
) as executor,
TestClient(app) as client,
):
run_id = client.post(
"/api/release-runs",
headers=headers,
json=api_run_input(request_id="api-ambiguous-create-0001"),
).json()["run_id"]
payload = {
"request_id": "api-ambiguous-attempt-0001",
"confirm": "TAG",
}
first = client.post(
f"/api/release-runs/{run_id}/steps/core:tag/execute",
headers=headers,
json=payload,
)
second = client.post(
f"/api/release-runs/{run_id}/steps/core:tag/execute",
headers=headers,
json=payload,
)
loaded = client.get(
f"/api/release-runs/{run_id}", headers=headers
).json()
self.assertEqual(409, first.status_code)
self.assertEqual(409, second.status_code)
self.assertEqual("interrupted", loaded["state"]["steps"][0]["state"])
self.assertEqual(
"executor_outcome_unknown",
loaded["state"]["steps"][0]["result_code"],
)
executor.assert_called_once()
def test_catalog_execution_persists_and_consumes_only_issued_receipt(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
candidate_root = Path(temp_dir) / "candidates"
app = create_app(
workspace_root=Path(temp_dir),
run_state_root=Path(temp_dir) / "runs",
candidate_root=candidate_root,
token="token",
)
headers = {"X-Release-Console-Token": "token"}
def generated(**kwargs: object) -> tuple[dict[str, object], CandidateArtifactReceipt]:
candidate_id = str(kwargs["candidate_id"])
candidate = candidate_output_path(candidate_root, candidate_id)
channels = candidate / "channels"
channels.mkdir(parents=True)
(channels / "stable.json").write_text(
json.dumps({"channel": "stable", "signatures": [{}]}),
encoding="utf-8",
)
harden_private_candidate_tree(candidate)
return (
{"status": "ready"},
issue_candidate_receipt(
root=candidate_root,
candidate_id=candidate_id,
channel="stable",
),
)
with (
patch("server.app.build_dashboard", return_value=object()),
patch(
"server.app.build_selective_release_plan",
return_value=catalog_release_plan(),
),
patch(
"server.app.bind_plan_source_states",
side_effect=bind_test_plan_source_states,
),
patch(
"server.app.verify_catalog_publication_precondition",
return_value=repository_receipt(
repo="addideas-govoplan-website", target_tag=""
),
),
patch("server.app.default_signing_keys", return_value=("key=/private",)),
patch(
"server.app.generate_catalog_candidate", side_effect=generated
) as generator,
patch(
"server.app.publish_received_candidate",
side_effect=published_candidate_result,
) as publisher,
TestClient(app) as client,
):
run_id = client.post(
"/api/release-runs",
headers=headers,
json=api_run_input(request_id="api-candidate-create-0001"),
).json()["run_id"]
generated_response = client.post(
f"/api/release-runs/{run_id}/steps/catalog:selective-generator/execute",
headers=headers,
json={
"request_id": "api-candidate-generate-0001",
"confirm": "GENERATE",
},
)
with patch("server.app.default_signing_keys", return_value=()):
replayed_generation = client.post(
f"/api/release-runs/{run_id}/steps/catalog:selective-generator/execute",
headers=headers,
json={
"request_id": "api-candidate-generate-0001",
"confirm": "GENERATE",
},
)
published = client.post(
f"/api/release-runs/{run_id}/steps/catalog:validate-sign-publish/execute",
headers=headers,
json={
"request_id": "api-candidate-publish-0001",
"confirm": "PUSH",
},
)
receipt = generated_response.json()["state"]["steps"][0]["result_receipt"]
self.assertRegex(receipt["candidate_id"], r"^candidate-[0-9a-f]{32}$")
self.assertEqual(64, len(receipt["catalog_sha256"]))
self.assertEqual(200, replayed_generation.status_code)
self.assertEqual(
"replayed",
replayed_generation.json()["execution_result"]["status"],
)
generator.assert_called_once()
self.assertEqual(200, published.status_code)
self.assertEqual("completed", published.json()["state"]["status"])
publisher.assert_called_once_with(
candidate_path=candidate_root / receipt["candidate_id"],
candidate_receipt=receipt,
channel="stable",
workspace_root=Path(temp_dir),
remote="origin",
expected_website_receipt=repository_receipt(
repo="addideas-govoplan-website", target_tag=""
),
)
def test_legacy_catalog_endpoint_is_preview_only(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
app = create_app(workspace_root=Path(temp_dir), token="token")
with TestClient(app) as client:
response = client.post(
"/api/catalog-candidates/publish",
headers={"X-Release-Console-Token": "token"},
json={
"candidate_dir": "/tmp/untrusted",
"apply": True,
"push": True,
"confirm": "PUSH",
},
)
self.assertEqual(409, response.status_code)
self.assertIn("durable release run", response.json()["detail"])
def test_release_channels_and_durable_remote_are_strictly_validated(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
app = create_app(workspace_root=Path(temp_dir), token="token")
headers = {"X-Release-Console-Token": "token"}
with (
patch("server.app.build_dashboard") as dashboard,
TestClient(app) as client,
):
query = client.get(
"/api/dashboard", headers=headers, params={"channel": "../stable"}
)
created = client.post(
"/api/release-runs",
headers=headers,
json=api_run_input(
request_id="api-invalid-channel-create-0001",
channel="/absolute",
),
)
execute = client.post(
"/api/release-runs/not-a-run/steps/core:tag/execute",
headers=headers,
json={
"request_id": "api-invalid-remote-attempt-0001",
"confirm": "TAG",
"remote": "upstream",
},
)
self.assertEqual(422, query.status_code)
self.assertEqual(422, created.status_code)
self.assertEqual(422, execute.status_code)
dashboard.assert_not_called()
def test_token_guarded_create_list_read_resume_and_retry(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir) / "runs"
@@ -1069,6 +1725,10 @@ class ReleaseRunApiTests(unittest.TestCase):
"server.app.build_selective_release_plan",
return_value=mutating_first_plan(),
),
patch(
"server.app.reconciled_repository_receipt",
return_value=repository_receipt(),
),
TestClient(app) as client,
):
created = client.post(
@@ -1236,12 +1896,16 @@ class ReleaseRunApiTests(unittest.TestCase):
)
self.assertIn("Durable Run State", webui)
self.assertIn("State tracking only.", webui)
self.assertIn("Durable execution.", webui)
self.assertIn("Create Run from Selection", webui)
self.assertIn("Prepare Retry", webui)
self.assertIn("step.retry_available", webui)
self.assertIn("Record Reconciliation", webui)
self.assertIn("effect_succeeded", webui)
self.assertNotIn(
'step.id === "catalog:validate-sign-publish" ? "disabled"',
webui,
)
self.assertIn("renderReleaseRunStoreUnavailable", webui)
self.assertIn('const runsRequest = api("/api/release-runs")', webui)
self.assertIn("pendingReleaseRunPayload", webui)
@@ -1249,13 +1913,30 @@ class ReleaseRunApiTests(unittest.TestCase):
self.assertIn("recoverPendingRunCommands", webui)
self.assertIn("inFlightRunCommandId", webui)
self.assertIn("Run saved; list refresh unavailable", webui)
self.assertIn("This foundation tracks state only.", runbook)
self.assertIn("The run record is execution evidence only", runbook)
self.assertIn("isolated checkout", runbook)
self.assertIn("effect_absent", runbook)
self.assertIn("$XDG_STATE_HOME", runbook)
self.assertIn("same-directory", runbook)
self.assertIn("caller-generated `request_id`", runbook)
self.assertIn("reserves the two command-ledger", runbook)
def test_entire_release_console_script_is_valid_javascript(self) -> None:
node = shutil.which("node")
if node is None:
self.skipTest("Node.js is required for the Release Console UI check")
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
match = re.search(r"<script>(?P<script>.*?)</script>", webui, re.DOTALL)
self.assertIsNotNone(match, "release console script element is missing")
result = subprocess.run( # noqa: S603
[node, "--check", "-"],
input=match.group("script") if match is not None else "",
check=False,
capture_output=True,
text=True,
)
self.assertEqual(0, result.returncode, result.stderr)
def test_ui_reuses_uncertain_ids_and_separates_saved_run_list_failure(
self,
) -> None:
@@ -1357,6 +2038,16 @@ async function api(_path) {
const reconcileRequest = pendingRunCommandRequest("reconcile:rr-known:core:tag:effect_succeeded", "reconcile-request");
assert(reconcileRequest.path.endsWith("/steps/core%3Atag/reconcile"), "reconciliation recovery path was not executable");
assert(reconcileRequest.body.confirm === "RECONCILE" && reconcileRequest.body.outcome === "effect_succeeded", "reconciliation recovery body changed");
const executeKey = "execute:rr-known:catalog:selective-generator:GENERATE";
const executeId = inFlightRunCommandId(executeKey, "execute", { signing_keys: ["release-key=/private/key"] });
assert(!values.get(pendingRunCommandsKey).includes("signing_keys"), "signing key path was persisted in session storage");
const executeRequest = pendingRunCommandRequest(executeKey, executeId);
assert(!Object.hasOwn(executeRequest.body, "signing_keys"), "execute replay reconstructed signing material");
values.set(pendingRunCommandsKey, JSON.stringify({ [commandKey]: commandOne, [executeKey]: { request_id: executeId, body: { signing_keys: ["legacy-secret"] } } }));
const migratedCommands = readInFlightRunCommands();
assert(migratedCommands[executeKey] === executeId, "legacy pending execute request ID was lost");
assert(!values.get(pendingRunCommandsKey).includes("legacy-secret"), "legacy persisted signing material was not purged");
clearInFlightRunCommand(executeKey, executeId);
failCreate = true;
const uncertainCommand = await recoverPendingRunCommands();
@@ -1455,6 +2146,64 @@ def run_input(
}
def repository_receipt(
*, repo: str = "govoplan-core", target_tag: str = "v1.2.3"
) -> dict[str, object]:
return {
"kind": "repository_state",
"repo": repo,
"head": "a" * 40,
"branch": "main",
"remote": "origin",
"remote_sha256": "b" * 64,
"worktree_clean": True,
"target_tag": target_tag,
"tag_object": None,
}
def published_candidate_result(
**kwargs: object,
) -> tuple[dict[str, object], dict[str, object]]:
candidate = kwargs["candidate_receipt"]
website = kwargs["expected_website_receipt"]
if not isinstance(candidate, dict) or not isinstance(website, dict):
raise AssertionError("publication test receipts are unavailable")
receipt = {
"kind": "catalog_publication",
"candidate_id": candidate["candidate_id"],
"catalog_sha256": candidate["catalog_sha256"],
"keyring_sha256": "c" * 64,
"publication_commit_sha": "d" * 40,
"publication_tag_object_sha": "e" * 40,
"publication_tag_commit_sha": "d" * 40,
"branch": website["branch"],
"tag_name": "catalog-stable-7",
"remote": "origin",
"remote_sha256": website["remote_sha256"],
}
return {"status": "published"}, receipt
def bind_test_plan_source_states(**kwargs: object) -> dict[str, object]:
plan = kwargs["plan"]
if not isinstance(plan, dict):
raise AssertionError("test plan is not an object")
steps = plan.get("dry_run_steps")
if not isinstance(steps, list):
raise AssertionError("test plan has no steps")
for step in steps:
if not isinstance(step, dict):
continue
if isinstance(step.get("repo"), str):
step["source_binding"] = repository_receipt()
elif step.get("id") == "catalog:validate-sign-publish":
step["source_binding"] = repository_receipt(
repo="addideas-govoplan-website", target_tag=""
)
return plan
def release_plan() -> dict[str, object]:
return {
"generated_at": "2026-07-22T00:00:00Z",
@@ -1505,5 +2254,32 @@ def mutating_first_plan() -> dict[str, object]:
return plan
def catalog_release_plan() -> dict[str, object]:
plan = release_plan()
plan["dry_run_steps"] = [
{
"id": "catalog:selective-generator",
"title": "Generate candidate",
"detail": "Build artifacts and sign a candidate.",
"command": "release-catalog selective",
"cwd": "/workspace/govoplan",
"mutating": True,
"repo": None,
"status": "planned",
},
{
"id": "catalog:validate-sign-publish",
"title": "Publish candidate",
"detail": "Publish only the receipt-bound candidate.",
"command": "release-catalog publish-candidate",
"cwd": "/workspace/govoplan",
"mutating": True,
"repo": None,
"status": "planned",
},
]
return plan
if __name__ == "__main__":
unittest.main()

View File

@@ -19,9 +19,13 @@ if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.publisher import publish_catalog_candidate # noqa: E402
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
build_selective_catalog_candidate,
enforce_selected_source_provenance,
parse_signing_key,
public_key_base64,
signature,
)
from govoplan_release.source_provenance import ( # noqa: E402
catalog_source_selection,
@@ -117,23 +121,37 @@ class ReleaseSourceProvenanceTests(unittest.TestCase):
core, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
git(core, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
git(core, "push", "origin", "refs/tags/v1.2.3")
base_catalog = self.root / "base.json"
base_catalog.write_text(
json.dumps(
{
"channel": "stable",
"sequence": 1,
"core_release": {
"version": "1.2.2",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.2",
},
"modules": [],
"release": {"version": "1.2.2", "tag": "v1.2.2"},
}
),
encoding="utf-8",
)
private_key = Ed25519PrivateKey.generate()
keyring = {
"keys": [
{
"key_id": "test-key",
"status": "active",
"public_key": public_key_base64(private_key),
}
]
}
base_keyring = self.root / "base-keyring.json"
base_keyring.write_text(json.dumps(keyring), encoding="utf-8")
base_catalog = self.root / "base.json"
base_payload = {
"channel": "stable",
"sequence": 1,
"core_release": {
"version": "1.2.2",
"python_ref": "govoplan-core @ git+ssh://git@example.test/acme/govoplan-core.git@v1.2.2",
},
"modules": [],
"release": {
"version": "1.2.2",
"tag": "v1.2.2",
"keyring_sha256": canonical_hash(keyring),
},
}
base_payload["signatures"] = [
signature(base_payload, key_id="test-key", private_key=private_key)
]
base_catalog.write_text(json.dumps(base_payload), encoding="utf-8")
key_path = self.root / "release.pem"
key_path.write_bytes(
private_key.private_bytes(
@@ -142,6 +160,7 @@ class ReleaseSourceProvenanceTests(unittest.TestCase):
encryption_algorithm=serialization.NoEncryption(),
)
)
key_path.chmod(0o600)
output = self.root / "candidate"
with patch(
@@ -153,6 +172,7 @@ class ReleaseSourceProvenanceTests(unittest.TestCase):
workspace_root=self.workspace,
output_dir=output,
base_catalog=base_catalog,
base_keyring=base_keyring,
signing_keys=(f"test-key={key_path}",),
check_public=False,
)
@@ -163,6 +183,25 @@ class ReleaseSourceProvenanceTests(unittest.TestCase):
self.assertEqual(git_text(core, "rev-parse", "refs/tags/v1.2.3"), selected["tag_object_sha"])
self.assertEqual("test-key", payload["signatures"][0]["key_id"])
def test_catalog_signing_key_must_be_operator_private(self) -> None:
private_key = Ed25519PrivateKey.generate()
key_path = self.root / "release.pem"
key_path.write_bytes(
private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
key_path.chmod(0o644)
with self.assertRaisesRegex(ValueError, "inaccessible to other users"):
parse_signing_key(f"test-key={key_path}")
key_path.chmod(0o600)
key_id, _parsed = parse_signing_key(f"test-key={key_path}")
self.assertEqual("test-key", key_id)
def test_catalog_publication_checks_every_preserved_source_ref(self) -> None:
core, _ = make_repo(self.workspace, self.root, "govoplan-core", "1.2.3")
git(core, "tag", "-a", "v1.2.3", "-m", "Core 1.2.3")
@@ -217,22 +256,21 @@ class ReleaseSourceProvenanceTests(unittest.TestCase):
self.assertIn("Array.isArray(result.notes)", ui)
self.assertIn("Validation and provenance details", ui)
def test_console_returns_actionable_conflict_for_candidate_provenance_gate(self) -> None:
with patch(
"server.app.build_selective_catalog_candidate",
side_effect=ValueError("Complete catalog source provenance gate failed: govoplan-core v1.2.3: missing"),
):
with TestClient(create_app(workspace_root=self.workspace)) as client:
response = client.post(
"/api/catalog-candidates",
json={
"repo_versions": {"govoplan-core": "1.2.3"},
"signing_keys": ["test=/unused/key.pem"],
},
)
def test_console_disables_unclaimed_legacy_candidate_signing(self) -> None:
with TestClient(
create_app(workspace_root=self.workspace, token="token")
) as client:
response = client.post(
"/api/catalog-candidates",
headers={"X-Release-Console-Token": "token"},
json={
"repo_versions": {"govoplan-core": "1.2.3"},
"signing_keys": ["test=/unused/key.pem"],
},
)
self.assertEqual(409, response.status_code)
self.assertIn("govoplan-core v1.2.3: missing", response.json()["detail"])
self.assertIn("durable release run", response.json()["detail"])
def test_read_only_ref_checks_can_reuse_the_same_gitea_https_endpoint(self) -> None:
self.assertEqual(

View File

@@ -0,0 +1,364 @@
from __future__ import annotations
import importlib.util
import json
from pathlib import Path
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = META_ROOT / "tools" / "repo" / "bootstrap-repositories.py"
def load_bootstrap_module():
spec = importlib.util.spec_from_file_location("bootstrap_repositories", SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {SCRIPT}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class RepositoryBootstrapTests(unittest.TestCase):
def test_public_https_transport_rewrites_registered_gitea_remotes(self) -> None:
bootstrap = load_bootstrap_module()
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
bootstrap.clone_remote(
"git@git.add-ideas.de:GovOPlaN/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
bootstrap.clone_remote(
"ssh://git@git.add-ideas.de/GovOPlaN/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
bootstrap.clone_remote(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
),
)
def test_registered_transport_preserves_the_manifest_remote(self) -> None:
bootstrap = load_bootstrap_module()
remote = "git@example.test:private/repository.git"
self.assertEqual(
remote,
bootstrap.clone_remote(
remote,
transport=bootstrap.REGISTERED_TRANSPORT,
),
)
def test_public_https_transport_fails_closed_for_other_hosts(self) -> None:
bootstrap = load_bootstrap_module()
unsafe_remotes = (
"git@example.test:GovOPlaN/govoplan-core.git",
"https://token@git.add-ideas.de/GovOPlaN/govoplan-core.git",
"https://git.add-ideas.de:443/GovOPlaN/govoplan-core.git",
"https://git.add-ideas.de/add-ideas/../govoplan-core.git",
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git?ref=main",
"ssh://root@git.add-ideas.de/GovOPlaN/govoplan-core.git",
)
for remote in unsafe_remotes:
with self.subTest(remote=remote), self.assertRaises(ValueError):
bootstrap.clone_remote(
remote,
transport=bootstrap.PUBLIC_HTTPS_TRANSPORT,
)
def test_main_clones_missing_repositories_over_public_https(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(0, status)
runner.assert_called_once_with(
[
"git",
"-c",
"credential.helper=",
"clone",
"--",
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
str(parent / "govoplan-core"),
],
check=True,
env=runner.call_args.kwargs["env"],
)
environment = runner.call_args.kwargs["env"]
self.assertEqual("/bin/false", environment["GIT_ASKPASS"])
self.assertEqual("0", environment["GIT_TERMINAL_PROMPT"])
def test_main_validates_every_remote_before_cloning(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
},
{
"name": "unsafe",
"path": "unsafe",
"remote": "git@example.test:private/unsafe.git",
},
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
self.assertRaises(ValueError),
):
bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
runner.assert_not_called()
def test_main_preserves_an_explicit_private_repository_transport(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "website",
"path": "website",
"remote": (
"git@git.add-ideas.de:"
"add-ideas/addideas-govoplan-website.git"
),
"bootstrap_transport": (
bootstrap.REGISTERED_TRANSPORT
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(0, status)
command = runner.call_args.args[0]
self.assertEqual(
"git@git.add-ideas.de:add-ideas/addideas-govoplan-website.git",
command[-2],
)
def test_main_limits_bootstrap_to_selected_repositories(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": name,
"path": name,
"remote": (
"git@git.add-ideas.de:GovOPlaN/"
f"{name}.git"
),
}
for name in ("govoplan-core", "govoplan-poll")
],
}
),
encoding="utf-8",
)
for repository_filter in (
["--repo", "govoplan-core"],
["--exclude-repo", "govoplan-poll"],
):
with (
self.subTest(repository_filter=repository_filter),
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
*repository_filter,
]
)
self.assertEqual(0, status)
runner.assert_called_once()
self.assertEqual(
"https://git.add-ideas.de/GovOPlaN/govoplan-core.git",
runner.call_args.args[0][-2],
)
def test_main_rejects_unknown_or_conflicting_repository_filters(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
with self.assertRaisesRegex(ValueError, "unknown registered"):
bootstrap.main(["--repo", "govoplan-missing"])
with self.assertRaisesRegex(ValueError, "selected and excluded"):
bootstrap.main(
[
"--repo",
"govoplan-core",
"--exclude-repo",
"govoplan-core",
]
)
runner.assert_not_called()
def test_check_reports_missing_without_cloning(self) -> None:
bootstrap = load_bootstrap_module()
with tempfile.TemporaryDirectory(prefix="govoplan-bootstrap-") as directory:
root = Path(directory)
parent = root / "checkouts"
parent.mkdir()
root.joinpath("repositories.json").write_text(
json.dumps(
{
"default_parent": str(parent),
"repositories": [
{
"name": "govoplan-core",
"path": "govoplan-core",
"remote": (
"git@git.add-ideas.de:"
"GovOPlaN/govoplan-core.git"
),
}
],
}
),
encoding="utf-8",
)
with (
patch.object(bootstrap, "ROOT", root),
patch.object(bootstrap.subprocess, "run") as runner,
):
status = bootstrap.main(
[
"--check",
"--parent",
str(parent),
"--transport",
bootstrap.PUBLIC_HTTPS_TRANSPORT,
]
)
self.assertEqual(1, status)
runner.assert_not_called()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,248 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sys
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
RESOLVER = (
META_ROOT / "tools" / "checks" / "security-audit" / "resolve_workspace_mount.py"
)
def load_resolver_module():
spec = importlib.util.spec_from_file_location(
"security_audit_mount_resolver",
RESOLVER,
)
if spec is None or spec.loader is None:
raise RuntimeError(f"Could not load {RESOLVER}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
class SecurityAuditMountResolverTests(unittest.TestCase):
def setUp(self) -> None:
self.resolver = load_resolver_module()
self.root = "/workspace/GovOPlaN/govoplan/govoplan"
self.workspace = "/workspace/GovOPlaN/govoplan"
def test_resolves_only_the_named_workspace_volume_for_both_scopes(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
{
"Type": "volume",
"Name": "actions-workspace",
"Source": "/var/lib/docker/volumes/actions-workspace/_data",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "volume",
"Name": "actions-environment",
"Source": "/var/lib/docker/volumes/actions-environment/_data",
"Destination": "/var/run/act",
"RW": True,
},
]
for scope in ("current", "govoplan"):
with self.subTest(scope=scope):
self.assertEqual(
(f"type=volume,source=actions-workspace,target={self.workspace}"),
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope=scope,
reports_dir="audit-reports",
),
)
def test_resolves_a_workspace_bind_without_other_mounts(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/srv/gitea/actions/task-123",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
]
self.assertEqual(
(f"type=bind,source=/srv/gitea/actions/task-123,target={self.workspace}"),
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
),
)
def test_nested_repository_mount_is_valid_only_for_current_scope(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "all-repositories",
"Destination": self.workspace,
"RW": True,
},
{
"Type": "volume",
"Name": "current-repository",
"Destination": self.root,
"RW": True,
},
]
self.assertIn(
"source=current-repository",
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="current",
reports_dir="audit-reports",
),
)
with self.assertRaisesRegex(
self.resolver.MountResolutionError,
"nested job-container mounts",
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
def test_rejects_unsafe_or_unusable_mount_layouts(self) -> None:
cases = {
"missing": [],
"path-boundary": [
{
"Type": "volume",
"Name": "wrong-workspace",
"Destination": "/workspace/GovOPlaN/govoplan-other",
"RW": True,
}
],
"read-only": [
{
"Type": "volume",
"Name": "actions-workspace",
"Destination": self.workspace,
"RW": False,
}
],
"ambiguous": [
{
"Type": "volume",
"Name": name,
"Destination": self.workspace,
"RW": True,
}
for name in ("workspace-one", "workspace-two")
],
"scope-too-narrow": [
{
"Type": "volume",
"Name": "repository-only",
"Destination": self.root,
"RW": True,
}
],
}
for name, mounts in cases.items():
with (
self.subTest(name=name),
self.assertRaises(self.resolver.MountResolutionError),
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
def test_rejects_reports_outside_the_selected_workspace_mount(self) -> None:
mounts = [
{
"Type": "volume",
"Name": "actions-workspace",
"Destination": self.workspace,
"RW": True,
}
]
with self.assertRaisesRegex(
self.resolver.MountResolutionError,
"reports path .* outside",
):
self.resolver.resolve_workspace_mount(
mounts,
root=self.root,
scope="current",
reports_dir="/tmp/audit-reports",
)
def test_rejects_broad_or_sensitive_workspace_bind_sources(self) -> None:
for source in (
"/",
"/home",
"/var/run/docker.sock",
"/srv/../etc/shadow",
):
with (
self.subTest(source=source),
self.assertRaisesRegex(
self.resolver.MountResolutionError,
"too broad or sensitive",
),
):
self.resolver.resolve_workspace_mount(
[
{
"Type": "bind",
"Source": source,
"Destination": self.workspace,
"RW": True,
}
],
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
with self.assertRaises(self.resolver.MountResolutionError):
self.resolver.resolve_workspace_mount(
[
{
"Type": "bind",
"Source": "//var/lib/workspace",
"Destination": self.workspace,
"RW": True,
}
],
root=self.root,
scope="govoplan",
reports_dir="audit-reports",
)
if __name__ == "__main__":
unittest.main()

View File

@@ -12,6 +12,7 @@ import unittest
META_ROOT = Path(__file__).resolve().parents[1]
AUDIT_SCRIPT = META_ROOT / "tools" / "checks" / "check-security-audit.sh"
CONTAINER_RUNNER = META_ROOT / "tools" / "checks" / "security-audit" / "run.sh"
class SecurityAuditWrapperTests(unittest.TestCase):
@@ -354,5 +355,162 @@ class SecurityAuditWrapperTests(unittest.TestCase):
self.assertEqual(set(manifest["reports"]), checksummed_paths)
class SecurityAuditContainerRunnerTests(unittest.TestCase):
def setUp(self) -> None:
self._temporary_directory = tempfile.TemporaryDirectory(
prefix="govoplan-audit-runner-"
)
root = Path(self._temporary_directory.name)
self.stub_bin = root / "bin"
self.stub_bin.mkdir()
self.docker_log = root / "docker.jsonl"
docker_stub = self.stub_bin / "docker"
docker_stub.write_text(
textwrap.dedent(
"""\
#!/usr/bin/env python3
import json
import os
from pathlib import Path
import sys
arguments = sys.argv[1:]
with Path(os.environ["DOCKER_STUB_LOG"]).open(
"a", encoding="utf-8"
) as handle:
handle.write(json.dumps(arguments) + "\\n")
if arguments and arguments[0] == "version":
print("26.1.0")
if arguments[:2] == ["container", "inspect"]:
print(os.environ["DOCKER_STUB_MOUNTS"])
"""
),
encoding="utf-8",
)
docker_stub.chmod(0o755)
self.environment = os.environ.copy()
self.environment.update(
{
"DOCKER_STUB_LOG": str(self.docker_log),
"DOCKER_STUB_MOUNTS": "[]",
"PATH": f"{self.stub_bin}:{self.environment['PATH']}",
}
)
def tearDown(self) -> None:
self._temporary_directory.cleanup()
def _run(
self,
*,
scope: str,
actions_mounts: object | None = None,
) -> tuple[subprocess.CompletedProcess[str], list[list[str]]]:
self.docker_log.write_text("", encoding="utf-8")
environment = self.environment.copy()
if actions_mounts is None:
environment.pop("GITEA_ACTIONS", None)
else:
environment["GITEA_ACTIONS"] = "true"
environment["DOCKER_STUB_MOUNTS"] = json.dumps(actions_mounts)
result = subprocess.run(
[
"bash",
str(CONTAINER_RUNNER),
"--mode",
"quick",
"--scope",
scope,
],
cwd=META_ROOT,
env=environment,
check=False,
capture_output=True,
text=True,
)
commands = [
json.loads(line)
for line in self.docker_log.read_text(encoding="utf-8").splitlines()
]
return result, commands
def test_gitea_job_shares_only_its_workspace_mount(self) -> None:
mounts = [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"RW": True,
},
{
"Type": "volume",
"Name": "govoplan-actions-workspace",
"Destination": str(META_ROOT.parent),
"RW": True,
},
{
"Type": "volume",
"Name": "govoplan-actions-environment",
"Destination": "/var/run/act",
"RW": True,
},
]
container_id = subprocess.run(
["hostname"],
check=True,
capture_output=True,
text=True,
).stdout.strip()
for scope in ("current", "govoplan"):
with self.subTest(scope=scope):
result, commands = self._run(scope=scope, actions_mounts=mounts)
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(
command for command in commands if command[0] == "run"
)
inspect_command = next(
command
for command in commands
if command[:2] == ["container", "inspect"]
)
self.assertEqual(container_id, inspect_command[-1])
mount_index = run_command.index("--mount")
self.assertEqual(
(
"type=volume,source=govoplan-actions-workspace,"
f"target={META_ROOT.parent}"
),
run_command[mount_index + 1],
)
self.assertNotIn("--volumes-from", run_command)
self.assertNotIn("-v", run_command)
self.assertNotIn("/var/run/docker.sock", " ".join(run_command))
self.assertNotIn("/var/run/act", " ".join(run_command))
repository_roots = [
value
for value in run_command
if value.startswith("GOVOPLAN_REPOS_ROOT=")
]
expected_roots = (
[f"GOVOPLAN_REPOS_ROOT={META_ROOT.parent}"]
if scope == "govoplan"
else []
)
self.assertEqual(expected_roots, repository_roots)
def test_non_actions_runner_keeps_the_scoped_bind_mount(self) -> None:
result, commands = self._run(scope="govoplan")
self.assertEqual(0, result.returncode, result.stderr)
run_command = next(command for command in commands if command[0] == "run")
bind_index = run_command.index("-v")
self.assertEqual(
f"{META_ROOT.parent}:/workspace",
run_command[bind_index + 1],
)
self.assertNotIn("--mount", run_command)
if __name__ == "__main__":
unittest.main()

View File

@@ -6,6 +6,7 @@ import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
META_ROOT = Path(__file__).resolve().parents[1]
@@ -20,6 +21,7 @@ from govoplan_release.version_alignment import ( # noqa: E402
repository_version_issues,
selected_repository_version_issues,
)
from govoplan_release.git_state import sanitized_git_environment # noqa: E402
from govoplan_release.model import ( # noqa: E402
CatalogSnapshot,
DashboardSummary,
@@ -398,6 +400,68 @@ class VersionAlignmentTests(unittest.TestCase):
)
self.assertTrue(all("must contain" in issue.message for issue in issues))
def test_annotated_release_tags_work_with_sanitized_git_configuration(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
workspace = Path(tmp)
meta = workspace / "govoplan"
core = workspace / "govoplan-core"
module = workspace / "govoplan-example"
meta.mkdir()
(core / "webui").mkdir(parents=True)
(module / "webui").mkdir(parents=True)
backend_ref = "git+ssh://git@example.test/acme/govoplan-example.git@v1.2.3"
webui_ref = "git+ssh://git@example.test/acme/govoplan-example.git#v1.2.3"
(meta / "requirements-release.txt").write_text(f"govoplan-example @ {backend_ref}\n")
(module / "pyproject.toml").write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
(module / "webui" / "package.json").write_text(
'{"name":"@govoplan/example-webui","version":"1.2.3"}\n'
)
subprocess.run(["git", "init", "-q", str(module)], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.email", "test@example.test"], check=True)
subprocess.run(["git", "-C", str(module), "config", "user.name", "Test"], check=True)
subprocess.run(["git", "-C", str(module), "add", "pyproject.toml", "webui/package.json"], check=True)
subprocess.run(["git", "-C", str(module), "commit", "-qm", "release"], check=True)
subprocess.run(
["git", "-C", str(module), "tag", "-a", "v1.2.3", "-m", "Release v1.2.3"],
check=True,
)
tagged_commit = subprocess.run(
["git", "-C", str(module), "rev-parse", "refs/tags/v1.2.3^{commit}"],
check=True,
text=True,
stdout=subprocess.PIPE,
).stdout.strip()
dependencies = {"@govoplan/example-webui": webui_ref}
(core / "pyproject.toml").write_text('[project]\nname="govoplan-core"\nversion="2.0.0"\n')
(core / "webui" / "package.release.json").write_text(
json.dumps({"version": "2.0.0", "dependencies": dependencies})
)
(core / "webui" / "package-lock.release.json").write_text(
json.dumps(
{
"packages": {
"": {"version": "2.0.0", "dependencies": dependencies},
"node_modules/@govoplan/example-webui": {
"version": "1.2.3",
"resolved": f"git+ssh://git@example.test/acme/govoplan-example.git#{tagged_commit}",
},
}
}
)
)
git_environment = sanitized_git_environment()
git_environment["GIT_TEST_ASSUME_DIFFERENT_OWNER"] = "1"
with patch(
"govoplan_release.version_alignment.sanitized_git_environment",
return_value=git_environment,
):
composition_issues = release_composition_issues(meta, core_root=core)
core_issues = repository_version_issues(core)
self.assertEqual((), composition_issues)
self.assertEqual((), core_issues)
if __name__ == "__main__":
unittest.main()

View File

@@ -105,6 +105,31 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
type=Path,
default=META_ROOT / "docs" / "installed-composition-evidence.schema.json",
)
parser.add_argument(
"--installer-receipt",
type=Path,
help="Signed installer receipt binding installed payloads to this catalog.",
)
parser.add_argument(
"--installer-receipt-schema",
type=Path,
default=META_ROOT / "docs" / "installer-receipt.schema.json",
)
parser.add_argument(
"--installer-authority-keyring",
type=Path,
help=(
"Independently provisioned, role-scoped trust root for installer "
"receipt signatures."
),
)
parser.add_argument(
"--installer-authority-keyring-schema",
type=Path,
default=META_ROOT
/ "docs"
/ "installer-receipt-authority-keyring.schema.json",
)
parser.add_argument(
"--boundary-evidence",
type=Path,
@@ -165,6 +190,18 @@ def main(argv: list[str] | None = None) -> int:
raise SystemExit(
"--boundary-evidence and --boundary-authority-keyring are required together"
)
if (args.installer_receipt is None) != (
args.installer_authority_keyring is None
):
raise SystemExit(
"--installer-receipt and --installer-authority-keyring are required together"
)
if args.installer_receipt is not None and (
args.installed_evidence is None and args.collect_installed_evidence is None
):
raise SystemExit(
"--installer-receipt requires --installed-evidence or --collect-installed-evidence"
)
if args.boundary_evidence is not None and (
args.installed_evidence is None and args.collect_installed_evidence is None
):
@@ -286,6 +323,32 @@ def main(argv: list[str] | None = None) -> int:
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_receipt = None
installer_receipt_schema = None
installer_authority_keyring = None
installer_authority_keyring_schema = None
if args.installer_receipt is not None:
installer_receipt = read_object(
args.installer_receipt,
label="installer receipt",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_receipt_schema = read_object(
args.installer_receipt_schema,
label="installer receipt schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_authority_keyring = read_object(
args.installer_authority_keyring,
label="installer authority keyring",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
installer_authority_keyring_schema = read_object(
args.installer_authority_keyring_schema,
label="installer authority keyring schema",
max_bytes=MAX_EVIDENCE_INPUT_BYTES,
)
verification_time = parse_datetime(args.verification_time)
report = review_capability_fit(
assessment=assessment,
@@ -298,6 +361,10 @@ def main(argv: list[str] | None = None) -> int:
else args.workspace_root.resolve(),
installed_evidence=installed_evidence,
installed_evidence_schema=installed_evidence_schema,
installer_receipt=installer_receipt,
installer_receipt_schema=installer_receipt_schema,
installer_authority_keyring=installer_authority_keyring,
installer_authority_keyring_schema=installer_authority_keyring_schema,
boundary_evidence=boundary_evidence,
boundary_evidence_schema=boundary_evidence_schema,
boundary_authority_keyring=boundary_authority_keyring,

View File

@@ -59,6 +59,10 @@ def review_capability_fit(
workspace_root: Path | None = None,
installed_evidence: dict[str, Any] | None = None,
installed_evidence_schema: dict[str, Any] | None = None,
installer_receipt: dict[str, Any] | None = None,
installer_receipt_schema: dict[str, Any] | None = None,
installer_authority_keyring: dict[str, Any] | None = None,
installer_authority_keyring_schema: dict[str, Any] | None = None,
boundary_evidence: dict[str, Any] | None = None,
boundary_evidence_schema: dict[str, Any] | None = None,
boundary_authority_keyring: dict[str, Any] | None = None,
@@ -377,6 +381,24 @@ def review_capability_fit(
verification_time=evidence_verification_time,
verification_mode=evidence_verification_mode,
catalog_trusted=catalog_dependency_trusted,
catalog_artifacts=(
catalog.get("release", {}).get("artifacts")
if isinstance(catalog.get("release"), dict)
and "artifacts" in catalog.get("release", {})
else None
),
catalog_sha256=canonical_hash(catalog),
catalog_channel=_text(catalog.get("channel")),
catalog_sequence=_integer(catalog.get("sequence")),
installer_receipt=installer_receipt,
installer_receipt_schema=installer_receipt_schema,
installer_authority_keyring=installer_authority_keyring,
installer_authority_keyring_schema=installer_authority_keyring_schema,
forbidden_installer_public_keys=public_key_material_from_keyrings(
published_keyring,
trusted_keyring,
boundary_authority_keyring or {},
),
)
findings.extend(
Finding(item.severity, item.code, item.message, item.assessment_ids)
@@ -398,6 +420,7 @@ def review_capability_fit(
forbidden_authority_public_keys=public_key_material_from_keyrings(
published_keyring,
trusted_keyring,
installer_authority_keyring or {},
),
)
findings.extend(
@@ -939,7 +962,7 @@ def build_report(
**boundary_scope,
}
return {
"report_version": "0.4.0",
"report_version": "0.5.0",
"status": status,
"assessment_id": assessment.get("assessment_id"),
"assessment_release": assessment.get("release", {}).get("ref")
@@ -977,7 +1000,11 @@ def render_review(report: dict[str, Any]) -> str:
(
"Scope: schema, signed release metadata, optional local tag provenance, "
+ (
"and locally observed installed-composition consistency evidence; release origin remains a separate proof boundary."
(
"and locally observed installed-composition evidence independently bound to signed release origin."
if release_origin_checked
else "and locally observed installed-composition consistency evidence; release origin remains a separate proof boundary."
)
if installed_checked
else (
"and supplied installed-composition evidence that did not establish local proof."

View File

@@ -25,6 +25,12 @@ from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from jsonschema import Draft202012Validator, FormatChecker
from jsonschema.exceptions import SchemaError
from govoplan_release.artifact_identity import (
INSTALLED_PAYLOAD_ALGORITHM,
WHEEL_PAYLOAD_ALGORITHM,
payload_identity_sha256,
)
MAX_GOVOPLAN_DISTRIBUTIONS = 256
MAX_DISTRIBUTION_FILES = 10_000
@@ -243,7 +249,7 @@ def collect_installed_composition(
sorted_issues.sort(key=lambda item: (item["package_name"], item["code"]))
return {
"$schema": "./installed-composition-evidence.schema.json",
"schema_version": "0.3.0",
"schema_version": "0.4.0",
"evidence_kind": "govoplan.installed-composition",
"assessment_id": _safe_opaque_id(
assessment.get("assessment_id"), fallback="invalid-assessment"
@@ -277,6 +283,15 @@ def review_installed_composition(
verification_time: datetime | None = None,
verification_mode: str = "current",
catalog_trusted: bool = False,
catalog_artifacts: object = None,
catalog_sha256: str | None = None,
catalog_channel: str | None = None,
catalog_sequence: int | None = None,
installer_receipt: dict[str, Any] | None = None,
installer_receipt_schema: dict[str, Any] | None = None,
installer_authority_keyring: dict[str, Any] | None = None,
installer_authority_keyring_schema: dict[str, Any] | None = None,
forbidden_installer_public_keys: frozenset[bytes] = frozenset(),
) -> InstalledEvidenceReview:
if evidence is None:
return InstalledEvidenceReview(
@@ -403,14 +418,6 @@ def review_installed_composition(
observation_trusted = True
elif effective_observation_mode == "imported_unsigned":
freshness = "unsigned_import"
findings.append(
EvidenceFinding(
"blocker",
"installed_evidence_unsigned_import",
"Imported unsigned installed evidence may be compared but cannot establish installed proof.",
("assessment.installed_composition",),
)
)
if effective_verification_mode == "invalid":
observation_trusted = False
if catalog_trusted is not True:
@@ -528,7 +535,7 @@ def review_installed_composition(
provenance_declared_match = 0
provenance_mutable = 0
provenance_unanchored = 0
artifact_valid = binding_valid and not duplicate_packages
payload_consistent = binding_valid and not duplicate_packages
for package_name, (component, catalog_entry) in sorted(expected_by_package.items()):
module_id = str(component["module_id"])
@@ -537,7 +544,7 @@ def review_installed_composition(
record_expected += 1
provenance_expected += 1
if package_name in duplicate_packages:
artifact_valid = False
payload_consistent = False
_record_change(
findings=findings,
changes=changes,
@@ -550,7 +557,7 @@ def review_installed_composition(
)
continue
if artifact is None:
artifact_valid = False
payload_consistent = False
_record_change(
findings=findings,
changes=changes,
@@ -567,7 +574,7 @@ def review_installed_composition(
package_version = str(artifact.get("package_version") or "")
catalog_version = str(catalog_entry.get("version") or "")
if package_version != expected_version or package_version != catalog_version:
artifact_valid = False
payload_consistent = False
_record_change(
findings=findings,
changes=changes,
@@ -606,7 +613,7 @@ def review_installed_composition(
and str(expected_manifest.get("manifest_version") or "") != expected_version
)
if manifest_mismatch:
artifact_valid = False
payload_consistent = False
_record_change(
findings=findings,
changes=changes,
@@ -619,7 +626,7 @@ def review_installed_composition(
)
unexpected_modules = sorted(set(module_rows) - {module_id})
if duplicate_module_ids or unexpected_modules:
artifact_valid = False
payload_consistent = False
_record_change(
findings=findings,
changes=changes,
@@ -647,7 +654,7 @@ def review_installed_composition(
if record_status == "verified" and record_semantics_valid:
record_verified += 1
else:
artifact_valid = False
payload_consistent = False
_record_change(
findings=findings,
changes=changes,
@@ -676,7 +683,6 @@ def review_installed_composition(
str(provenance.get("basis") or "") if isinstance(provenance, dict) else ""
)
if provenance_basis != "local-pep610-metadata":
artifact_valid = False
provenance_unanchored += 1
_record_change(
findings=findings,
@@ -703,7 +709,6 @@ def review_installed_composition(
if commit_matches and selected_matches:
provenance_declared_match += 1
else:
artifact_valid = False
_record_change(
findings=findings,
changes=changes,
@@ -716,7 +721,6 @@ def review_installed_composition(
)
elif provenance_kind in {"editable-vcs", "editable-local", "local-directory"}:
provenance_mutable += 1
artifact_valid = False
_record_change(
findings=findings,
changes=changes,
@@ -729,7 +733,6 @@ def review_installed_composition(
)
else:
provenance_unanchored += 1
artifact_valid = False
_record_change(
findings=findings,
changes=changes,
@@ -743,7 +746,7 @@ def review_installed_composition(
expected_packages = set(expected_by_package)
for package_name in sorted(set(artifacts_by_package) - expected_packages):
artifact_valid = False
payload_consistent = False
matching_module_id = next(
(
module_id
@@ -788,7 +791,7 @@ def review_installed_composition(
continue
package_name = str(issue.get("package_name") or "")
code = str(issue.get("code") or "")
artifact_valid = False
payload_consistent = False
findings.append(
EvidenceFinding(
"review",
@@ -801,18 +804,68 @@ def review_installed_composition(
exact_set = (
set(artifacts_by_package) == expected_packages and not duplicate_packages
)
installed_checked = observation_trusted
origin_scope, origin_findings = _review_installed_release_origin(
expected_by_package=expected_by_package,
installed_artifacts=artifacts_by_package,
installed_evidence=evidence,
installed_evidence_sha256=digest,
installed_composition_valid=(
binding_valid
and exact_set
and payload_consistent
and record_verified == record_expected
and record_expected > 0
),
observation_trusted=observation_trusted,
catalog_artifacts=catalog_artifacts,
catalog_sha256=catalog_sha256,
catalog_channel=catalog_channel,
catalog_sequence=catalog_sequence,
catalog_trusted=catalog_trusted,
receipt=installer_receipt,
receipt_schema=installer_receipt_schema,
authority_keyring=installer_authority_keyring,
authority_keyring_schema=installer_authority_keyring_schema,
verification_time=now,
forbidden_public_keys=forbidden_installer_public_keys,
)
findings.extend(origin_findings)
receipt_authenticated = (
origin_scope.get("valid") is True
and origin_scope.get("signed_installer_receipt_count") == record_expected
and record_expected > 0
)
effective_observation_trusted = observation_trusted or receipt_authenticated
if (
effective_observation_mode == "imported_unsigned"
and not receipt_authenticated
):
findings.append(
EvidenceFinding(
"blocker",
"installed_evidence_unsigned_import",
"Imported installed evidence requires a valid independent installer receipt before it can establish installed proof.",
("assessment.installed_composition",),
)
)
installed_checked = effective_observation_trusted
record_checked = (
observation_trusted and binding_valid and exact_set and record_expected > 0
effective_observation_trusted
and binding_valid
and exact_set
and record_expected > 0
)
provenance_checked = (
observation_trusted and binding_valid and exact_set and provenance_expected > 0
effective_observation_trusted
and binding_valid
and exact_set
and provenance_expected > 0
)
proof_scope = {
"installed_artifacts": {
"checked": installed_checked,
"valid": artifact_valid if installed_checked else None,
"comparison_valid": artifact_valid,
"valid": payload_consistent if installed_checked else None,
"comparison_valid": payload_consistent,
"evidence_sha256": digest,
"expected_distribution_count": len(expected_packages),
"observed_distribution_count": len(artifact_rows),
@@ -836,16 +889,7 @@ def review_installed_composition(
"expected_distribution_count": provenance_expected,
"release_origin_bound": False,
},
"installed_release_origin": {
"checked": False,
"valid": None,
"release_origin_bound": False,
"anchored_artifact_digest_count": 0,
"expected_distribution_count": provenance_expected,
"required_evidence": (
"An independently anchored artifact digest or signed installer receipt that binds each installed payload to the signed release catalog.",
),
},
"installed_release_origin": origin_scope,
"runtime_activation": {
"checked": False,
"valid": None,
@@ -861,6 +905,7 @@ def review_installed_composition(
"evaluated_at": _iso_datetime(now),
"verification_mode": effective_verification_mode,
"catalog_trusted": catalog_trusted is True,
"receipt_authenticated": receipt_authenticated,
"freshness_window_seconds": int(MAX_LOCAL_OBSERVATION_AGE.total_seconds()),
},
}
@@ -874,6 +919,489 @@ def review_installed_composition(
)
def _review_installed_release_origin(
*,
expected_by_package: Mapping[str, tuple[dict[str, Any], Mapping[str, Any]]],
installed_artifacts: Mapping[str, dict[str, Any]],
installed_evidence: dict[str, Any],
installed_evidence_sha256: str,
installed_composition_valid: bool,
observation_trusted: bool,
catalog_artifacts: object,
catalog_sha256: str | None,
catalog_channel: str | None,
catalog_sequence: int | None,
catalog_trusted: bool,
receipt: dict[str, Any] | None,
receipt_schema: dict[str, Any] | None,
authority_keyring: dict[str, Any] | None,
authority_keyring_schema: dict[str, Any] | None,
verification_time: datetime,
forbidden_public_keys: frozenset[bytes],
) -> tuple[dict[str, Any], tuple[EvidenceFinding, ...]]:
findings: list[EvidenceFinding] = []
expected_count = len(expected_by_package)
scope: dict[str, Any] = {
"checked": False,
"valid": None,
"release_origin_bound": False,
"anchored_artifact_digest_count": 0,
"signed_installer_receipt_count": 0,
"expected_distribution_count": expected_count,
"catalog_sha256": catalog_sha256,
"installer_receipt_sha256": None,
"required_evidence": (
"A signed catalog identity computed from each built immutable artifact, or a role-scoped installer receipt signed by an independently provisioned authority.",
),
}
manifest_supplied = catalog_artifacts is not None
receipt_supplied = receipt is not None
if not manifest_supplied and not receipt_supplied:
return scope, ()
scope["checked"] = True
scope["valid"] = False
if not installed_composition_valid or catalog_trusted is not True:
findings.append(
EvidenceFinding(
"blocker",
"installed_origin_dependency_untrusted",
"Installed release-origin proof requires a trusted catalog and valid fresh installed-composition evidence.",
("assessment.release", "assessment.installed_composition"),
)
)
return scope, tuple(findings)
catalog_by_package, catalog_findings = _catalog_artifact_identities(
catalog_artifacts
)
findings.extend(catalog_findings)
if catalog_findings:
return scope, tuple(findings)
direct_bound: set[str] = set()
receipt_required: set[str] = set()
for package_name, (component, catalog_entry) in sorted(expected_by_package.items()):
expected_version = str(catalog_entry.get("version") or "")
identity = catalog_by_package.get(package_name)
if identity is None or identity.get("package_version") != expected_version:
findings.append(
EvidenceFinding(
"review",
"installed_origin_catalog_artifact_missing",
f"The signed catalog has no built-artifact identity for {package_name!r} at the selected version.",
(f"composition.{component['module_id']}",),
)
)
continue
if identity.get("requires_installer_receipt") is True:
receipt_required.add(package_name)
continue
if not observation_trusted:
continue
installed = installed_artifacts.get(package_name)
record = installed.get("record_integrity") if isinstance(installed, dict) else None
observed = (
record.get("artifact_payload_identity") if isinstance(record, dict) else None
)
expected = identity.get("installed_payload")
if observed == expected:
direct_bound.add(package_name)
else:
findings.append(
EvidenceFinding(
"blocker",
"installed_origin_artifact_payload_mismatch",
f"Installed payload for {package_name!r} does not match the identity independently computed from the signed catalog artifact.",
(f"composition.{component['module_id']}",),
)
)
receipt_bound: set[str] = set()
if receipt_supplied:
receipt_bound, receipt_findings, receipt_digest = _verify_installer_receipt(
expected_by_package=expected_by_package,
installed_artifacts=installed_artifacts,
installed_evidence=installed_evidence,
installed_evidence_sha256=installed_evidence_sha256,
catalog_by_package=catalog_by_package,
catalog_sha256=catalog_sha256,
catalog_channel=catalog_channel,
catalog_sequence=catalog_sequence,
receipt=receipt,
receipt_schema=receipt_schema,
authority_keyring=authority_keyring,
authority_keyring_schema=authority_keyring_schema,
verification_time=verification_time,
forbidden_public_keys=forbidden_public_keys,
)
findings.extend(receipt_findings)
scope["installer_receipt_sha256"] = receipt_digest
elif receipt_required:
findings.append(
EvidenceFinding(
"review",
"installed_origin_receipt_required",
"At least one selected wheel produces installer-transformed files, so its signed installer receipt is required for complete origin proof.",
("assessment.installed_composition",),
)
)
bound = direct_bound | receipt_bound
scope["anchored_artifact_digest_count"] = len(direct_bound)
scope["signed_installer_receipt_count"] = len(receipt_bound)
valid = len(bound) == expected_count and not any(
finding.severity == "blocker" for finding in findings
)
scope["valid"] = valid
scope["release_origin_bound"] = valid
return scope, tuple(findings)
def _catalog_artifact_identities(
value: object,
) -> tuple[dict[str, dict[str, Any]], tuple[EvidenceFinding, ...]]:
if value is None:
return {}, ()
if not isinstance(value, list) or len(value) > MAX_GOVOPLAN_DISTRIBUTIONS:
return {}, (
EvidenceFinding(
"blocker",
"catalog_artifact_manifest_invalid",
"Signed catalog release artifacts must be a bounded list.",
("assessment.release",),
),
)
result: dict[str, dict[str, Any]] = {}
invalid = False
for item in value:
if not isinstance(item, dict) or set(item) != {
"artifact_kind",
"package_name",
"package_version",
"archive_sha256",
"archive_size",
"installed_payload",
"requires_installer_receipt",
}:
invalid = True
continue
package_name = item.get("package_name")
version = item.get("package_version")
archive_size = item.get("archive_size")
payload = item.get("installed_payload")
if (
item.get("artifact_kind") != "python-wheel"
or not isinstance(package_name, str)
or PACKAGE_PATTERN.fullmatch(package_name) is None
or not isinstance(version, str)
or VERSION_PATTERN.fullmatch(version) is None
or not isinstance(item.get("archive_sha256"), str)
or SHA256_PATTERN.fullmatch(str(item["archive_sha256"])) is None
or not isinstance(archive_size, int)
or isinstance(archive_size, bool)
or not 0 < archive_size <= MAX_HASHED_DISTRIBUTION_BYTES
or not isinstance(item.get("requires_installer_receipt"), bool)
or not _payload_identity_semantics_valid(
payload,
algorithm=WHEEL_PAYLOAD_ALGORITHM,
maximum_files=MAX_DISTRIBUTION_FILES,
)
or package_name in result
):
invalid = True
continue
result[package_name] = item
if invalid:
return {}, (
EvidenceFinding(
"blocker",
"catalog_artifact_manifest_invalid",
"Signed catalog release artifacts contain malformed or duplicate package identities.",
("assessment.release",),
),
)
return result, ()
def _verify_installer_receipt(
*,
expected_by_package: Mapping[str, tuple[dict[str, Any], Mapping[str, Any]]],
installed_artifacts: Mapping[str, dict[str, Any]],
installed_evidence: dict[str, Any],
installed_evidence_sha256: str,
catalog_by_package: Mapping[str, dict[str, Any]],
catalog_sha256: str | None,
catalog_channel: str | None,
catalog_sequence: int | None,
receipt: dict[str, Any],
receipt_schema: dict[str, Any] | None,
authority_keyring: dict[str, Any] | None,
authority_keyring_schema: dict[str, Any] | None,
verification_time: datetime,
forbidden_public_keys: frozenset[bytes],
) -> tuple[set[str], tuple[EvidenceFinding, ...], str]:
receipt_digest = canonical_sha256(receipt)
findings: list[EvidenceFinding] = []
if receipt_schema is None or authority_keyring_schema is None:
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_schema_missing",
"Installer receipt proof requires both bounded validation schemas.",
("assessment.installed_composition",),
)
)
return set(), tuple(findings), receipt_digest
receipt_errors = validate_payload(payload=receipt, schema=receipt_schema)
keyring_errors = (
validate_payload(payload=authority_keyring, schema=authority_keyring_schema)
if isinstance(authority_keyring, dict)
else ("$: an independently provisioned installer authority keyring is required",)
)
findings.extend(
EvidenceFinding(
"blocker",
"installer_receipt_schema",
message,
("assessment.installed_composition",),
)
for message in receipt_errors
)
findings.extend(
EvidenceFinding(
"blocker",
"installer_authority_keyring_schema",
message,
("assessment.installed_composition",),
)
for message in keyring_errors
)
if receipt_errors or keyring_errors or not isinstance(authority_keyring, dict):
return set(), tuple(findings), receipt_digest
expected_release = installed_evidence.get("assessment_release")
expected_assessment = installed_evidence.get("assessment_id")
catalog_binding = receipt.get("catalog")
binding_valid = True
if (
receipt.get("assessment_id") != expected_assessment
or receipt.get("assessment_release") != expected_release
or receipt.get("installed_evidence_sha256") != installed_evidence_sha256
):
binding_valid = False
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_evidence_mismatch",
"Installer receipt is not bound to the supplied assessment and installed-evidence digest.",
("assessment.installed_composition",),
)
)
if not isinstance(catalog_binding, dict) or (
catalog_binding.get("sha256") != catalog_sha256
or catalog_binding.get("channel") != catalog_channel
or catalog_binding.get("sequence") != catalog_sequence
):
binding_valid = False
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_catalog_mismatch",
"Installer receipt is not bound to the supplied signed catalog identity.",
("assessment.release", "assessment.installed_composition"),
)
)
issued_at = _datetime(receipt.get("issued_at"))
collected_at = _datetime(installed_evidence.get("collected_at"))
if (
issued_at is None
or collected_at is None
or issued_at < collected_at
or issued_at - collected_at > MAX_LOCAL_OBSERVATION_AGE
or issued_at > verification_time + MAX_LOCAL_OBSERVATION_FUTURE_SKEW
or verification_time - issued_at > MAX_LOCAL_OBSERVATION_AGE
):
binding_valid = False
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_time_invalid",
"Installer receipt issuance must follow the observation within five minutes and remain fresh at the selected verification time.",
("assessment.installed_composition",),
)
)
receipt_rows = [
item for item in receipt.get("artifacts", []) if isinstance(item, dict)
]
rows_by_package: dict[str, dict[str, Any]] = {}
duplicate_packages: set[str] = set()
for row in receipt_rows:
package_name = str(row.get("package_name") or "")
if package_name in rows_by_package:
duplicate_packages.add(package_name)
else:
rows_by_package[package_name] = row
expected_packages = set(expected_by_package)
if duplicate_packages or set(rows_by_package) != expected_packages:
binding_valid = False
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_artifact_set_mismatch",
"Installer receipt must bind exactly one row for every expected installed distribution.",
("assessment.installed_composition",),
)
)
bound: set[str] = set()
for package_name, (_, catalog_entry) in sorted(expected_by_package.items()):
row = rows_by_package.get(package_name)
installed = installed_artifacts.get(package_name)
record = installed.get("record_integrity") if isinstance(installed, dict) else None
observed_payload = (
record.get("installed_payload_identity") if isinstance(record, dict) else None
)
catalog_identity = catalog_by_package.get(package_name)
if (
row is None
or catalog_identity is None
or row.get("package_version") != catalog_entry.get("version")
or row.get("catalog_archive_sha256")
!= catalog_identity.get("archive_sha256")
or row.get("installed_payload") != observed_payload
):
binding_valid = False
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_artifact_mismatch",
f"Installer receipt does not bind the observed {package_name!r} payload to its signed catalog artifact.",
("assessment.installed_composition",),
)
)
else:
bound.add(package_name)
keys, key_findings = _installer_receipt_keys(
authority_keyring=authority_keyring,
issued_at=issued_at,
forbidden_public_keys=forbidden_public_keys,
)
findings.extend(key_findings)
signature_payload = dict(receipt)
signature_payload.pop("signatures", None)
signature_bytes = canonical_bytes(signature_payload)
signature_valid = False
seen_ids: set[str] = set()
duplicate_ids = False
for signature in receipt.get("signatures", []):
if not isinstance(signature, dict):
continue
key_id = str(signature.get("key_id") or "")
if key_id in seen_ids:
duplicate_ids = True
continue
seen_ids.add(key_id)
public_key = keys.get(key_id)
if public_key is None or signature.get("algorithm") != "ed25519":
continue
try:
encoded = base64.b64decode(str(signature.get("value") or ""), validate=True)
Ed25519PublicKey.from_public_bytes(public_key).verify(
encoded, signature_bytes
)
signature_valid = True
except (ValueError, InvalidSignature):
continue
if duplicate_ids or not signature_valid:
findings.append(
EvidenceFinding(
"blocker",
"installer_receipt_signature_untrusted",
"Installer receipt has no unique valid signature from an authorized independent installer key.",
("assessment.installed_composition",),
)
)
if not binding_valid or not signature_valid or key_findings:
return set(), tuple(findings), receipt_digest
return bound, tuple(findings), receipt_digest
def _installer_receipt_keys(
*,
authority_keyring: dict[str, Any],
issued_at: datetime | None,
forbidden_public_keys: frozenset[bytes],
) -> tuple[dict[str, bytes], tuple[EvidenceFinding, ...]]:
findings: list[EvidenceFinding] = []
result: dict[str, bytes] = {}
raw_keys = authority_keyring.get("keys")
if not isinstance(raw_keys, list) or issued_at is None:
return {}, (
EvidenceFinding(
"blocker",
"installer_authority_untrusted",
"Installer receipt authority keyring or issuance time is invalid.",
("assessment.installed_composition",),
),
)
seen_ids: set[str] = set()
seen_material: set[bytes] = set()
malformed = False
for item in raw_keys:
if not isinstance(item, dict):
malformed = True
continue
key_id = str(item.get("key_id") or "")
try:
public_key = base64.b64decode(str(item.get("public_key") or ""), validate=True)
Ed25519PublicKey.from_public_bytes(public_key)
except ValueError:
malformed = True
continue
not_before = _datetime(item.get("not_before"))
not_after = _datetime(item.get("not_after"))
interval_valid = not (
not_before is not None
and not_after is not None
and not_before >= not_after
)
if (
OPAQUE_ID_PATTERN.fullmatch(key_id) is None
or key_id in seen_ids
or public_key in seen_material
or public_key in forbidden_public_keys
or item.get("allowed_scopes") != ["installed_release_origin"]
or not interval_valid
):
malformed = True
continue
seen_ids.add(key_id)
seen_material.add(public_key)
status = item.get("status")
active_at_issue = (
(not_before is None or issued_at >= not_before)
and (not_after is None or issued_at < not_after)
)
historically_retired = status == "retired" and not_after is not None
if active_at_issue and (
status in {"active", "next"} or historically_retired
):
result[key_id] = public_key
if malformed or not result:
findings.append(
EvidenceFinding(
"blocker",
"installer_authority_untrusted",
"Installer authority keyring is malformed, reuses another trust-domain key, or has no role-scoped key valid at receipt issuance.",
("assessment.installed_composition",),
)
)
return {}, tuple(findings)
return result, ()
def review_boundary_evidence(
*,
assessment: dict[str, Any],
@@ -1606,13 +2134,17 @@ def _record_integrity(
"missing_file_count": 0,
"mismatched_file_count": 0,
}
identities = {
"artifact_payload_identity": None,
"installed_payload_identity": None,
}
try:
distribution_root = Path(distribution.locate_file("")).resolve()
installation_root = Path(sys.prefix).resolve()
except Exception:
return {"status": "unavailable", **counts}
return {"status": "unavailable", **counts, **identities}
if not _trusted_distribution_root(distribution_root):
return {"status": "unavailable", **counts}
return {"status": "unavailable", **counts, **identities}
entries, malformed_rows, record_status = _read_record_entries(
distribution=distribution,
distribution_root=distribution_root,
@@ -1620,13 +2152,13 @@ def _record_integrity(
)
if record_status == "limit-exceeded":
budget.remaining_files = 0
return {"status": "limit-exceeded", **counts}
return {"status": "limit-exceeded", **counts, **identities}
if record_status != "ok":
counts["unverifiable_file_count"] = malformed_rows
return {"status": "unavailable", **counts}
return {"status": "unavailable", **counts, **identities}
record_file_count = len(entries) + malformed_rows
if record_file_count == 0:
return {"status": "unavailable", **counts}
return {"status": "unavailable", **counts, **identities}
budget.remaining_files -= record_file_count
counts["unverifiable_file_count"] = malformed_rows
declared_scripts = _declared_distribution_scripts(distribution)
@@ -1666,6 +2198,8 @@ def _record_integrity(
}
total_hashed_bytes = 0
limit_exceeded = False
artifact_payload_rows: list[dict[str, object]] = []
installed_payload_rows: list[dict[str, object]] = []
for entry, resolved_path in resolved_entries:
if resolved_path is None:
counts["unverifiable_file_count"] += 1
@@ -1731,6 +2265,31 @@ def _record_integrity(
continue
if hmac.compare_digest(actual, expected):
counts["hashed_file_count"] += 1
installed_path = _installed_payload_path(
resolved_path,
distribution_root=distribution_root,
installation_root=installation_root,
declared_scripts=declared_scripts,
)
if installed_path is None:
counts["unverifiable_file_count"] += 1
continue
row = {
"path": installed_path,
"sha256": actual.hex(),
"size": file_stat.st_size,
}
installed_payload_rows.append(row)
artifact_path = _artifact_payload_path(
resolved_path,
distribution_root=distribution_root,
installation_root=installation_root,
declared_scripts=declared_scripts,
)
if artifact_path is not None and not _installer_generated_metadata(
resolved_path, distribution_root=distribution_root
):
artifact_payload_rows.append({**row, "path": artifact_path})
else:
counts["mismatched_file_count"] += 1
if limit_exceeded:
@@ -1743,7 +2302,85 @@ def _record_integrity(
status = "partial"
else:
status = "verified"
return {"status": status, **counts}
if status == "verified":
installed_identity = _payload_identity(
algorithm=INSTALLED_PAYLOAD_ALGORITHM,
rows=installed_payload_rows,
)
artifact_identity = _payload_identity(
algorithm=WHEEL_PAYLOAD_ALGORITHM,
rows=artifact_payload_rows,
)
if installed_identity is None or artifact_identity is None:
status = "partial"
counts["unverifiable_file_count"] += 1
else:
identities["installed_payload_identity"] = installed_identity
identities["artifact_payload_identity"] = artifact_identity
return {"status": status, **counts, **identities}
def _payload_identity(
*, algorithm: str, rows: list[dict[str, object]]
) -> dict[str, object] | None:
ordered = sorted(rows, key=lambda item: str(item["path"]))
paths = [str(item["path"]) for item in ordered]
if not ordered or len(set(paths)) != len(paths):
return None
return {
"algorithm": algorithm,
"sha256": payload_identity_sha256(algorithm=algorithm, rows=ordered),
"file_count": len(ordered),
}
def _installed_payload_path(
path: Path,
*,
distribution_root: Path,
installation_root: Path,
declared_scripts: frozenset[str],
) -> str | None:
if _is_relative_to(path, distribution_root):
return path.relative_to(distribution_root).as_posix()
for script_root in (
(installation_root / "bin").resolve(),
(installation_root / "Scripts").resolve(),
):
if path.parent == script_root and path.name in declared_scripts:
return f"@scripts/{path.name}"
if _is_relative_to(path, installation_root):
return f"@data/{path.relative_to(installation_root).as_posix()}"
return None
def _artifact_payload_path(
path: Path,
*,
distribution_root: Path,
installation_root: Path,
declared_scripts: frozenset[str],
) -> str | None:
installed = _installed_payload_path(
path,
distribution_root=distribution_root,
installation_root=installation_root,
declared_scripts=declared_scripts,
)
return None if installed is None or installed.startswith("@scripts/") else installed
def _installer_generated_metadata(
path: Path, *, distribution_root: Path
) -> bool:
if not _is_relative_to(path, distribution_root):
return False
relative = path.relative_to(distribution_root)
return (
len(relative.parts) >= 2
and relative.parent.name.endswith(".dist-info")
and relative.name in {"direct_url.json", "INSTALLER", "REQUESTED"}
)
def _resolve_record_path(
@@ -1867,12 +2504,25 @@ def _record_integrity_semantics_valid(record: Mapping[str, Any]) -> bool:
missing = counts["missing_file_count"]
mismatched = counts["mismatched_file_count"]
if status == "verified":
artifact_identity = record.get("artifact_payload_identity")
installed_identity = record.get("installed_payload_identity")
return (
hashed > 0
and counts["permitted_unhashed_file_count"] > 0
and unverifiable == 0
and missing == 0
and mismatched == 0
and _payload_identity_semantics_valid(
artifact_identity,
algorithm=WHEEL_PAYLOAD_ALGORITHM,
maximum_files=hashed,
)
and _payload_identity_semantics_valid(
installed_identity,
algorithm=INSTALLED_PAYLOAD_ALGORITHM,
maximum_files=hashed,
exact_files=hashed,
)
)
if status == "partial":
return hashed > 0 and unverifiable > 0 and missing == 0 and mismatched == 0
@@ -1885,6 +2535,31 @@ def _record_integrity_semantics_valid(record: Mapping[str, Any]) -> bool:
return False
def _payload_identity_semantics_valid(
value: object,
*,
algorithm: str,
maximum_files: int,
exact_files: int | None = None,
) -> bool:
if not isinstance(value, Mapping) or set(value) != {
"algorithm",
"sha256",
"file_count",
}:
return False
file_count = value.get("file_count")
return (
value.get("algorithm") == algorithm
and isinstance(value.get("sha256"), str)
and SHA256_PATTERN.fullmatch(str(value["sha256"])) is not None
and isinstance(file_count, int)
and not isinstance(file_count, bool)
and 0 < file_count <= maximum_files
and (exact_files is None or file_count == exact_files)
)
def _permitted_unhashed_record_path(
path: Path,
*,

View File

@@ -0,0 +1,204 @@
"""Issuance of role-scoped receipts from same-process installed observations."""
from __future__ import annotations
import base64
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, Mapping
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from .capability_fit import canonical_hash
from .evidence import (
MAX_LOCAL_OBSERVATION_AGE,
OPAQUE_ID_PATTERN,
_catalog_artifact_identities,
_record_integrity_semantics_valid,
canonical_bytes,
canonical_sha256,
normalize_package_name,
)
from govoplan_release.artifact_identity import inspect_python_wheel
def issue_installer_receipt(
*,
assessment: Mapping[str, Any],
catalog: Mapping[str, Any],
installed_evidence: dict[str, Any],
receipt_id: str,
key_id: str,
private_key: Ed25519PrivateKey,
consumed_artifacts: Mapping[str, Path | str] | None = None,
issued_at: datetime | None = None,
same_process_observation: bool = False,
) -> dict[str, Any]:
"""Sign a receipt only for a fresh observation collected by this process.
File-based evidence is intentionally not an admitted issuance input. The CLI
collector calls this function directly with its in-memory observation.
"""
if same_process_observation is not True:
raise ValueError("installer receipts require a same-process observation")
if OPAQUE_ID_PATTERN.fullmatch(receipt_id) is None:
raise ValueError("receipt ID must be a bounded opaque ID")
if OPAQUE_ID_PATTERN.fullmatch(key_id) is None:
raise ValueError("installer signing key ID must be a bounded opaque ID")
if not isinstance(consumed_artifacts, Mapping) or not consumed_artifacts:
raise ValueError("installer receipt issuance requires exact consumed wheels")
release = catalog.get("release")
raw_artifacts = release.get("artifacts") if isinstance(release, Mapping) else None
catalog_artifacts, artifact_findings = _catalog_artifact_identities(raw_artifacts)
if artifact_findings:
raise ValueError("signed catalog artifact manifest is invalid")
evidence_rows = installed_evidence.get("artifacts")
if not isinstance(evidence_rows, list) or not evidence_rows:
raise ValueError("installed evidence has no artifact observations")
expected_packages = _expected_packages(assessment=assessment, catalog=catalog)
receipt_rows: list[dict[str, Any]] = []
seen: set[str] = set()
for artifact in evidence_rows:
if not isinstance(artifact, dict):
raise ValueError("installed evidence contains a malformed artifact")
package_name = normalize_package_name(str(artifact.get("package_name") or ""))
package_version = str(artifact.get("package_version") or "")
if package_name in seen:
raise ValueError("installed evidence contains duplicate packages")
seen.add(package_name)
catalog_artifact = catalog_artifacts.get(package_name)
record = artifact.get("record_integrity")
installed_payload = (
record.get("installed_payload_identity")
if isinstance(record, Mapping)
else None
)
if (
catalog_artifact is None
or catalog_artifact.get("package_version") != package_version
or not isinstance(record, Mapping)
or record.get("status") != "verified"
or not _record_integrity_semantics_valid(record)
or not isinstance(installed_payload, Mapping)
):
raise ValueError(
"installed payload is not a verified match for a catalog artifact"
)
artifact_path = consumed_artifacts.get(package_name)
if artifact_path is None:
raise ValueError("every installed package requires its exact consumed wheel")
consumed = inspect_python_wheel(artifact_path)
if (
consumed.package_name != package_name
or consumed.package_version != package_version
or consumed.archive_sha256 != catalog_artifact.get("archive_sha256")
or consumed.catalog_payload().get("installed_payload")
!= catalog_artifact.get("installed_payload")
or record.get("artifact_payload_identity")
!= catalog_artifact.get("installed_payload")
):
raise ValueError(
"consumed wheel or observed wheel payload differs from the signed catalog identity"
)
receipt_rows.append(
{
"package_name": package_name,
"package_version": package_version,
"catalog_archive_sha256": catalog_artifact["archive_sha256"],
"installed_payload": dict(installed_payload),
}
)
if seen != expected_packages:
raise ValueError(
"installed evidence must contain exactly the enabled assessed catalog packages"
)
if set(consumed_artifacts) != expected_packages:
raise ValueError("consumed wheel set must exactly match enabled packages")
receipt_rows.sort(key=lambda item: (item["package_name"], item["package_version"]))
observed_at = issued_at or datetime.now(tz=UTC)
if observed_at.tzinfo is None:
raise ValueError("installer receipt issuance time must include a timezone")
collected_at = _datetime(installed_evidence.get("collected_at"))
if (
collected_at is None
or observed_at < collected_at
or observed_at - collected_at > MAX_LOCAL_OBSERVATION_AGE
):
raise ValueError(
"installer receipt issuance must follow live collection within five minutes"
)
assessment_release = assessment.get("release")
assessment_release_ref = (
assessment_release.get("ref")
if isinstance(assessment_release, Mapping)
else None
)
receipt: dict[str, Any] = {
"$schema": "./installer-receipt.schema.json",
"schema_version": "0.1.0",
"evidence_kind": "govoplan.installer-receipt",
"receipt_id": receipt_id,
"assessment_id": assessment.get("assessment_id"),
"assessment_release": assessment_release_ref,
"installed_evidence_sha256": canonical_sha256(installed_evidence),
"catalog": {
"channel": catalog.get("channel"),
"sequence": catalog.get("sequence"),
"sha256": canonical_hash(catalog),
},
"issued_at": observed_at.astimezone(UTC).isoformat().replace("+00:00", "Z"),
"artifacts": receipt_rows,
}
receipt["signatures"] = [
{
"algorithm": "ed25519",
"key_id": key_id,
"value": base64.b64encode(
private_key.sign(canonical_bytes(receipt))
).decode("ascii"),
}
]
return receipt
def _expected_packages(
*, assessment: Mapping[str, Any], catalog: Mapping[str, Any]
) -> set[str]:
entries: dict[str, Mapping[str, Any]] = {}
core = catalog.get("core_release")
if isinstance(core, Mapping):
entries["core"] = core
modules = catalog.get("modules")
if isinstance(modules, list):
for entry in modules:
if isinstance(entry, Mapping) and isinstance(entry.get("module_id"), str):
entries[str(entry["module_id"])] = entry
expected: set[str] = set()
composition = assessment.get("composition")
if not isinstance(composition, list):
raise ValueError("assessment composition is unavailable")
for component in composition:
if not isinstance(component, Mapping) or component.get("enabled") is not True:
continue
entry = entries.get(str(component.get("module_id") or ""))
package = entry.get("python_package") if isinstance(entry, Mapping) else None
if not isinstance(package, str):
raise ValueError("enabled assessment component has no catalog package")
normalized = normalize_package_name(package)
if normalized in expected:
raise ValueError("enabled assessment packages are ambiguous")
expected.add(normalized)
if not expected:
raise ValueError("assessment has no enabled catalog packages")
return expected
def _datetime(value: object) -> datetime | None:
if not isinstance(value, str):
return None
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return None
return parsed.astimezone(UTC) if parsed.tzinfo is not None else None

View File

@@ -0,0 +1,170 @@
#!/usr/bin/env python3
"""Collect live installed payloads and issue an independently signed receipt."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
META_ROOT = Path(__file__).resolve().parents[2]
for tools_root in (META_ROOT / "tools" / "assessments", META_ROOT / "tools" / "release"):
if str(tools_root) not in sys.path:
sys.path.insert(0, str(tools_root))
from cryptography.hazmat.primitives import serialization # noqa: E402
from cryptography.hazmat.primitives.asymmetric.ed25519 import ( # noqa: E402
Ed25519PrivateKey,
)
from govoplan_assessment import collect_installed_composition # noqa: E402
from govoplan_assessment.atomic_io import ( # noqa: E402
AtomicJsonWriteError,
atomic_write_json,
)
from govoplan_assessment.capability_fit import ( # noqa: E402
canonical_hash,
review_capability_fit,
)
from govoplan_assessment.evidence import validate_payload # noqa: E402
from govoplan_assessment.installer_receipt import ( # noqa: E402
issue_installer_receipt,
)
MAX_INPUT_BYTES = 16 * 1024 * 1024
MAX_OUTPUT_BYTES = 4 * 1024 * 1024
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--assessment", type=Path, required=True)
parser.add_argument("--assessment-schema", type=Path, default=META_ROOT / "docs" / "capability-fit.schema.json")
parser.add_argument("--catalog", type=Path, required=True)
parser.add_argument("--keyring", type=Path, required=True)
parser.add_argument("--trusted-keyring", type=Path, required=True)
parser.add_argument("--receipt-id", required=True)
parser.add_argument("--signing-key", required=True, metavar="KEY_ID=/PATH/PRIVATE.pem")
parser.add_argument(
"--python-artifact",
action="append",
default=[],
required=True,
metavar="PACKAGE=/PATH/TO/CONSUMED.whl",
help="Exact wheel consumed by this installer; repeat for every package.",
)
parser.add_argument("--evidence-output", type=Path, required=True)
parser.add_argument("--receipt-output", type=Path, required=True)
parser.add_argument("--installed-evidence-schema", type=Path, default=META_ROOT / "docs" / "installed-composition-evidence.schema.json")
parser.add_argument("--receipt-schema", type=Path, default=META_ROOT / "docs" / "installer-receipt.schema.json")
args = parser.parse_args(argv)
if args.evidence_output.resolve() == args.receipt_output.resolve():
parser.error("evidence and receipt output paths must differ")
assessment = read_object(args.assessment, label="assessment")
assessment_schema = read_object(args.assessment_schema, label="assessment schema")
catalog = read_object(args.catalog, label="catalog")
keyring = read_object(args.keyring, label="published keyring")
trusted_keyring = read_object(args.trusted_keyring, label="trusted keyring")
evidence_schema = read_object(args.installed_evidence_schema, label="installed evidence schema")
receipt_schema = read_object(args.receipt_schema, label="installer receipt schema")
release = catalog.get("release")
if not isinstance(release, dict) or release.get("keyring_sha256") != canonical_hash(keyring):
parser.error("catalog does not pin the supplied published keyring")
evidence = collect_installed_composition(assessment=assessment)
if validate_payload(payload=evidence, schema=evidence_schema):
parser.error("live installed observation does not satisfy its bounded schema")
report = review_capability_fit(
assessment=assessment,
schema=assessment_schema,
catalog=catalog,
published_keyring=keyring,
trusted_keyring=trusted_keyring,
workspace_root=None,
installed_evidence=evidence,
installed_evidence_schema=evidence_schema,
installed_evidence_mode="direct_local",
)
required_scopes = (
"catalog_signature_and_keyring",
"catalog_signature_and_trusted_keyring",
"published_keyring_hash",
"release_metadata",
"installed_artifacts",
"installed_record_integrity",
)
if any(report["proof_scope"].get(scope, {}).get("valid") is not True for scope in required_scopes):
parser.error("trusted catalog and live installed-composition checks must pass before receipt issuance")
key_id, private_key = read_signing_key(args.signing_key)
consumed_artifacts = parse_package_paths(tuple(args.python_artifact))
receipt = issue_installer_receipt(
assessment=assessment,
catalog=catalog,
installed_evidence=evidence,
receipt_id=args.receipt_id,
key_id=key_id,
private_key=private_key,
consumed_artifacts=consumed_artifacts,
same_process_observation=True,
)
if validate_payload(payload=receipt, schema=receipt_schema):
parser.error("issued installer receipt does not satisfy its bounded schema")
write_object(args.evidence_output, evidence, label="installed evidence")
write_object(args.receipt_output, receipt, label="installer receipt")
print(json.dumps({"receipt_id": args.receipt_id, "evidence_output": str(args.evidence_output), "receipt_output": str(args.receipt_output)}, sort_keys=True))
return 0
def read_object(path: Path, *, label: str) -> dict[str, object]:
try:
if path.stat().st_size > MAX_INPUT_BYTES:
raise ValueError("input exceeds size limit")
encoded = path.read_bytes()
payload = json.loads(encoded.decode("utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError, ValueError) as exc:
raise SystemExit(f"Could not read {label}") from exc
if not isinstance(payload, dict):
raise SystemExit(f"{label.capitalize()} must be a JSON object")
return payload
def read_signing_key(value: str) -> tuple[str, Ed25519PrivateKey]:
key_id, separator, path_text = value.partition("=")
if not separator or not key_id or not path_text:
raise SystemExit("--signing-key must use KEY_ID=/path/to/private.pem")
try:
key = serialization.load_pem_private_key(Path(path_text).expanduser().read_bytes(), password=None)
except (OSError, ValueError) as exc:
raise SystemExit("Could not read installer signing key") from exc
if not isinstance(key, Ed25519PrivateKey):
raise SystemExit("Installer signing key must be Ed25519")
return key_id, key
def parse_package_paths(values: tuple[str, ...]) -> dict[str, Path]:
result: dict[str, Path] = {}
for value in values:
package, separator, path_text = value.partition("=")
package = package.strip()
path_text = path_text.strip()
if not separator or not package or not path_text or package in result:
raise SystemExit(
"--python-artifact must uniquely use PACKAGE=/path/to/consumed.whl"
)
result[package] = Path(path_text).expanduser()
return result
def write_object(path: Path, payload: dict[str, object], *, label: str) -> None:
try:
atomic_write_json(path, payload, max_bytes=MAX_OUTPUT_BYTES)
except AtomicJsonWriteError as exc:
raise SystemExit(f"Could not securely write {label}") from exc
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,264 @@
#!/usr/bin/env python3
"""Exercise connector -> datasource -> dataflow publication capabilities."""
from __future__ import annotations
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_connectors.backend.db.models import ConnectorTabularSource
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.dataflows import (
DataflowPublicationTarget,
DataflowRunRequest,
dataflow_run_lifecycle,
)
from govoplan_core.core.datasources import (
DatasourceReadRequest,
datasource_catalogue,
datasource_lifecycle,
datasource_publication,
)
from govoplan_core.core.modules import ModuleContext
from govoplan_core.core.tabular_sources import (
TabularSnapshotInput,
tabular_snapshot_writer,
)
from govoplan_core.db.base import Base
from govoplan_core.server.registry import build_platform_registry
from govoplan_dataflow.backend.schemas import (
GraphEdge,
GraphNode,
GraphPosition,
PipelineGraph,
PipelineCreateRequest,
PipelinePreviewRequest,
)
from govoplan_dataflow.backend.db.models import (
DataflowPipeline,
DataflowPipelineRevision,
DataflowRun,
)
from govoplan_dataflow.backend.service import create_pipeline, preview_pipeline
from govoplan_datasources.backend.db.models import (
DatasourceMaterializationRecord,
DatasourcePayloadRecord,
DatasourcePayloadRowRecord,
DatasourcePublicationRecord,
DatasourceRecord,
DatasourceStageRecord,
)
def main() -> int:
registry = build_platform_registry(
("connectors", "datasources", "dataflow", "workflow")
)
registry.configure_capability_context(
ModuleContext(registry=registry, settings=object())
)
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(
engine,
tables=[
ConnectorTabularSource.__table__,
DatasourceRecord.__table__,
DatasourcePayloadRecord.__table__,
DatasourcePayloadRowRecord.__table__,
DatasourceMaterializationRecord.__table__,
DatasourceStageRecord.__table__,
DatasourcePublicationRecord.__table__,
DataflowPipeline.__table__,
DataflowPipelineRevision.__table__,
DataflowRun.__table__,
],
)
session_factory = sessionmaker(bind=engine)
with session_factory() as session:
principal = _principal()
writer = tabular_snapshot_writer(registry)
lifecycle = datasource_lifecycle(registry)
catalogue = datasource_catalogue(registry)
publisher = datasource_publication(registry)
runner = dataflow_run_lifecycle(registry)
if (
writer is None
or lifecycle is None
or catalogue is None
or publisher is None
or runner is None
):
raise RuntimeError("Datasource composition capabilities are incomplete.")
origin = writer.create_snapshot(
session,
principal,
snapshot=TabularSnapshotInput(
name="Monthly cases",
source_name="connector_monthly_cases",
rows=(
{"id": 1, "amount": 5},
{"id": 2, "amount": 15},
),
),
)
datasource = lifecycle.register_origin(
session,
principal,
origin_ref=origin.ref,
name="Monthly cases cache",
source_name="monthly_cases",
mode="cached",
)
result = preview_pipeline(
session,
tenant_id="tenant-1",
actor_id="account-1",
payload=PipelinePreviewRequest(
graph=_graph(
datasource_ref=datasource.ref,
fingerprint=datasource.fingerprint,
),
row_limit=100,
),
principal=principal,
registry=registry,
)
expected_rows = [
{"id": 1, "amount": 5},
{"id": 2, "amount": 15},
]
if result.status != "succeeded":
raise RuntimeError(f"Dataflow preview failed: {result.diagnostics}")
if result.rows != expected_rows:
raise RuntimeError(f"Unexpected Dataflow rows: {result.rows!r}")
if result.source_fingerprints[0]["source_ref"] != datasource.ref:
raise RuntimeError("Dataflow lineage did not retain the datasource reference.")
pipeline = create_pipeline(
session,
tenant_id="tenant-1",
actor_id="account-1",
payload=PipelineCreateRequest(
name="Monthly case output",
status="active",
graph=_graph(
datasource_ref=datasource.ref,
fingerprint=datasource.fingerprint,
),
editor_mode="graph",
),
)
run_request = DataflowRunRequest(
pipeline_ref=f"pipeline:{pipeline.id}",
revision=1,
idempotency_key="composition-run-1",
publication=DataflowPublicationTarget(
name="Monthly case result",
source_name="monthly_case_result",
freeze=True,
frozen_label="Composition evidence",
),
)
published = runner.start_run(
session,
principal,
request=run_request,
)
replayed = runner.start_run(
session,
principal,
request=run_request,
)
if published.status != "succeeded":
raise RuntimeError(f"Dataflow publication failed: {published.error}")
if replayed.ref != published.ref or not replayed.replayed:
raise RuntimeError("Dataflow run idempotency did not replay the prior run.")
if (
not published.output_datasource_ref
or not published.output_materialization_ref
):
raise RuntimeError("Dataflow publication did not retain output references.")
output = catalogue.read_datasource(
session,
principal,
request=DatasourceReadRequest(
datasource_ref=published.output_datasource_ref,
),
)
if list(output.rows) != expected_rows:
raise RuntimeError(
f"Unexpected published Dataflow rows: {list(output.rows)!r}"
)
if (
output.materialization is None
or output.materialization.ref != published.output_materialization_ref
or output.materialization.frozen_at is None
):
raise RuntimeError(
"Published Datasource materialization is not pinned and frozen."
)
engine.dispose()
print(
"Connector -> Datasources -> pinned Dataflow publication composition passed."
)
return 0
def _principal() -> ApiPrincipal:
return ApiPrincipal(
principal=PrincipalRef(
account_id="account-1",
membership_id="membership-1",
tenant_id="tenant-1",
scopes=frozenset(
{
"connectors:source:read",
"connectors:source:write",
"datasources:catalogue:read",
"datasources:source:write",
"datasources:stage:write",
"dataflow:pipeline:run",
}
),
),
account=object(),
user=object(),
)
def _graph(*, datasource_ref: str, fingerprint: str) -> PipelineGraph:
return PipelineGraph(
nodes=[
GraphNode(
id="source",
type="source.reference",
label="Cases",
position=GraphPosition(x=0, y=0),
config={
"source_ref": datasource_ref,
"source_name": "monthly_cases",
"expected_fingerprint": fingerprint,
"consistency": "current",
},
),
GraphNode(
id="output",
type="output",
label="Output",
position=GraphPosition(x=200, y=0),
config={},
),
],
edges=[
GraphEdge(
id="source-output",
source="source",
target="output",
)
],
)
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -69,6 +69,14 @@ PY
"$PYTHON" -c 'import govoplan_core.db.bootstrap; import govoplan_access.backend.admin.service; import govoplan_addresses.backend.manifest; import govoplan_files.backend.router; import govoplan_mail.backend.sending.imap; print("targeted backend imports passed")'
"$META_ROOT/tools/checks/check_dependency_boundaries.py"
"$PYTHON" -m unittest tests.test_module_system
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-workflow/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
@@ -77,6 +85,21 @@ cd "$ROOT/webui"
"$NPM" run test:module-capabilities
"$NPM" run test:module-permutations
cd /mnt/DATA/git/govoplan-dataflow/webui
"$NPM" run test:structure
cd /mnt/DATA/git/govoplan-datasources/webui
"$NPM" run typecheck
cd /mnt/DATA/git/govoplan-workflow/webui
"$NPM" run typecheck
cd /mnt/DATA/git/govoplan-dashboard/webui
"$NPM" run test:dashboard-layout
cd /mnt/DATA/git/govoplan-postbox/webui
"$NPM" run test:ui-structure
cd /mnt/DATA/git/govoplan-mail/webui
"$NPM" run test:mail-ui

View File

@@ -19,7 +19,10 @@ fi
cd "$ROOT"
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
"$PYTHON" -m unittest tests.test_module_system tests.test_access_contracts
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" artifacts \
--requirements "$META_ROOT/requirements-release.txt"
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" core-tests \
--core-root "$ROOT"
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
cd "$ROOT/webui"

View File

@@ -60,24 +60,6 @@ retry() {
done
}
run_discovered_tests() {
local suite_dir="$1"
local status
if ! find "$suite_dir" -type f -name 'test*.py' -print -quit | grep -q .; then
echo "No unittest test files found under $suite_dir; skipping."
return 0
fi
set +e
"$PYTHON" -m unittest discover -s "$suite_dir"
status=$?
set -e
if [[ "$status" == 5 ]]; then
echo "No unittest tests discovered under $suite_dir; skipping."
return 0
fi
return "$status"
}
install_cloned_webui_dependencies() {
local webui_dir="$1"
if [[ ! -f "$webui_dir/package.json" ]]; then
@@ -244,7 +226,7 @@ PY
run_step "Clone release module test sources"
for repo in govoplan-mail govoplan-calendar govoplan-campaign; do
repo_tag="$(release_tag_for_package "$repo")"
git clone --depth 1 --branch "$repo_tag" "git@git.add-ideas.de:add-ideas/${repo}.git" "$WORK_ROOT/$repo"
git clone --depth 1 --branch "$repo_tag" "git@git.add-ideas.de:GovOPlaN/${repo}.git" "$WORK_ROOT/$repo"
done
run_step "Run core backend release tests"
@@ -253,9 +235,10 @@ run_step "Run core backend release tests"
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
run_step "Run cloned module backend tests"
run_discovered_tests "$WORK_ROOT/govoplan-mail/tests"
run_discovered_tests "$WORK_ROOT/govoplan-calendar/tests"
run_discovered_tests "$WORK_ROOT/govoplan-campaign/tests"
for repo in govoplan-mail govoplan-calendar govoplan-campaign; do
"$PYTHON" "$META_ROOT/tools/checks/release_integration.py" module-tests \
--module-root "$WORK_ROOT/$repo"
done
run_step "Run core WebUI release tests and build"
cd "$ROOT/webui"

View File

@@ -496,7 +496,12 @@ run_bandit() {
fi
if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then
prepare_machine_report "$REPORTS_DIR/bandit-tests.json" || return 2
bandit -r "${PY_TEST_ROOTS[@]}" -f json -o "$REPORTS_DIR/bandit-tests.json"
# Tests intentionally use assertions and synthetic credentials. Keep the
# separate test scan useful by excluding only those test-specific signals.
bandit -r "${PY_TEST_ROOTS[@]}" \
--skip B101,B105,B106,B107 \
-f json \
-o "$REPORTS_DIR/bandit-tests.json"
local test_status=$?
[[ "$test_status" -le 1 ]] || status=2
fi
@@ -512,7 +517,11 @@ run_ruff_security() {
fi
if [[ "${#PY_TEST_ROOTS[@]}" -gt 0 ]]; then
prepare_machine_report "$REPORTS_DIR/ruff-security-tests.json" || return 2
ruff check --select S --output-format json "${PY_TEST_ROOTS[@]}" > "$REPORTS_DIR/ruff-security-tests.json"
ruff check \
--select S \
--ignore S101,S105,S106,S107 \
--output-format json \
"${PY_TEST_ROOTS[@]}" > "$REPORTS_DIR/ruff-security-tests.json"
local test_status=$?
[[ "$test_status" -le 1 ]] || status=2
fi
@@ -582,14 +591,32 @@ run_trivy() {
run_pip_audit_manifests() {
local status=0
for repo in "${REPOS[@]}"; do
[[ -f "$repo/requirements.txt" ]] || continue
local name
name="$(safe_name "$repo")"
prepare_machine_report "$REPORTS_DIR/pip-audit-$name.json" || return 2
# Requirements may contain project-relative entries such as `.[server]`.
# Resolve them from the owning repository instead of the meta-repository.
(cd "$repo" && pip-audit -r requirements.txt --progress-spinner off --format json --output "$REPORTS_DIR/pip-audit-$name.json")
accumulate_exit_status status "$?"
local repo_name
repo_name="$(safe_name "$repo")"
while IFS= read -r -d '' requirements_file; do
local manifest_name report_path
manifest_name="$(safe_name "$requirements_file")"
report_path="$REPORTS_DIR/pip-audit-$repo_name-$manifest_name.json"
prepare_machine_report "$report_path" || return 2
# Requirements may contain project-relative entries such as `.[server]`.
# Resolve them from the owning repository instead of the meta-repository.
(
cd "$repo" &&
pip-audit \
-r "$(basename "$requirements_file")" \
--progress-spinner off \
--format json \
--output "$report_path"
)
accumulate_exit_status status "$?"
done < <(
find "$repo" \
-maxdepth 1 \
-type f \
-name 'requirements*.txt' \
-print0 |
sort -z
)
done
return "$status"
}
@@ -597,12 +624,38 @@ run_pip_audit_manifests() {
run_npm_audit_manifests() {
local status=0
for repo in "${REPOS[@]}"; do
[[ -f "$repo/webui/package-lock.json" ]] || continue
local name
name="$(safe_name "$repo")"
prepare_machine_report "$REPORTS_DIR/npm-audit-$name.json" || return 2
(cd "$repo/webui" && npm audit --omit=dev --json > "$REPORTS_DIR/npm-audit-$name.json")
accumulate_exit_status status "$?"
local repo_name
repo_name="$(safe_name "$repo")"
while IFS= read -r -d '' lock_file; do
local lock_dir lock_name runtime_report all_report
lock_dir="$(dirname "$lock_file")"
lock_name="$(
printf '%s' "${lock_file#"$repo"/}" |
tr -c 'A-Za-z0-9_.-' '_'
)"
runtime_report="$REPORTS_DIR/npm-audit-runtime-$repo_name-$lock_name.json"
all_report="$REPORTS_DIR/npm-audit-all-$repo_name-$lock_name.json"
prepare_machine_report "$runtime_report" || return 2
(
cd "$lock_dir" &&
npm audit --omit=dev --json > "$runtime_report"
)
accumulate_exit_status status "$?"
prepare_machine_report "$all_report" || return 2
(
cd "$lock_dir" &&
npm audit --json > "$all_report"
)
accumulate_exit_status status "$?"
done < <(
find "$repo" \
-maxdepth 3 \
-type f \
-name package-lock.json \
-not -path '*/node_modules/*' \
-print0 |
sort -z
)
done
return "$status"
}

View File

@@ -5,6 +5,7 @@ import argparse
import importlib.metadata as metadata
import os
import re
import subprocess
import sys
import tempfile
import unittest
@@ -354,6 +355,24 @@ def run_core_release_tests(core_root: Path) -> int:
return 0 if result.wasSuccessful() else 1
def run_module_release_tests(module_root: Path) -> int:
resolved_root = module_root.resolve()
tests_root = resolved_root / "tests"
if not tests_root.is_dir() or not any(tests_root.rglob("test*.py")):
print(f"No test files found under {tests_root}; skipping.")
return 0
result = subprocess.run(
(sys.executable, "-m", "pytest", "-q", "tests"),
cwd=resolved_root,
check=False,
)
if result.returncode == 5:
print(f"No tests collected under {tests_root}; skipping.")
return 0
return result.returncode
def _parser() -> argparse.ArgumentParser:
meta_root = Path(__file__).resolve().parents[2]
parser = argparse.ArgumentParser(description="Artifact-aware GovOPlaN release integration checks")
@@ -366,6 +385,8 @@ def _parser() -> argparse.ArgumentParser:
)
core_parser = subparsers.add_parser("core-tests", help="Run Core tests that are valid for tagged module artifacts")
core_parser.add_argument("--core-root", type=Path, required=True)
module_parser = subparsers.add_parser("module-tests", help="Run tests from one tagged module source checkout")
module_parser.add_argument("--module-root", type=Path, required=True)
return parser
@@ -373,7 +394,9 @@ def main(argv: Sequence[str] | None = None) -> int:
args = _parser().parse_args(argv)
if args.command == "artifacts":
return run_installed_artifact_checks(args.requirements)
return run_core_release_tests(args.core_root)
if args.command == "core-tests":
return run_core_release_tests(args.core_root)
return run_module_release_tests(args.module_root)
if __name__ == "__main__":

View File

@@ -0,0 +1,202 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import posixpath
from pathlib import Path
import re
import sys
from typing import Any
VOLUME_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*")
UNSAFE_BIND_SOURCES = {
Path("/"),
Path("/bin"),
Path("/boot"),
Path("/dev"),
Path("/etc"),
Path("/home"),
Path("/lib"),
Path("/lib64"),
Path("/mnt"),
Path("/opt"),
Path("/proc"),
Path("/root"),
Path("/run"),
Path("/sbin"),
Path("/srv"),
Path("/sys"),
Path("/tmp"),
Path("/usr"),
Path("/var"),
Path("/var/run"),
}
SENSITIVE_BIND_ROOTS = {
Path("/dev"),
Path("/etc"),
Path("/proc"),
Path("/root"),
Path("/run"),
Path("/sys"),
Path("/var/run"),
}
class MountResolutionError(ValueError):
pass
def normalized_absolute_path(
value: str,
*,
field: str,
resolve_symlinks: bool = True,
) -> Path:
if not value.startswith("/") or value.startswith("//"):
raise MountResolutionError(f"{field} must be an absolute path")
path = Path(posixpath.normpath(value))
return path.resolve(strict=False) if resolve_symlinks else path
def contains_path(container: Path, path: Path) -> bool:
return path == container or container in path.parents
def mount_option_value(value: str, *, field: str) -> str:
if not value or any(character in value for character in (",", "\n", "\r")):
raise MountResolutionError(f"{field} cannot be represented safely")
return value
def resolve_workspace_mount(
mounts: Any,
*,
root: str,
scope: str,
reports_dir: str,
) -> str:
if scope not in {"current", "govoplan"}:
raise MountResolutionError(f"unsupported audit scope: {scope}")
if not isinstance(mounts, list):
raise MountResolutionError("Docker mount metadata must be a list")
root_path = normalized_absolute_path(root, field="audit root")
scan_root = root_path if scope == "current" else root_path.parent
reports_path = Path(reports_dir)
if not reports_path.is_absolute():
reports_path = root_path / reports_path
reports_path = reports_path.resolve(strict=False)
parsed_mounts: list[tuple[Path, dict[str, Any]]] = []
for mount in mounts:
if not isinstance(mount, dict) or not isinstance(mount.get("Destination"), str):
continue
destination = normalized_absolute_path(
mount["Destination"],
field="mount destination",
)
parsed_mounts.append((destination, mount))
candidates = [
(destination, mount)
for destination, mount in parsed_mounts
if destination != Path("/") and contains_path(destination, scan_root)
]
if not candidates:
raise MountResolutionError(
f"no job-container mount covers audit scope root {scan_root}"
)
longest_depth = max(len(destination.parts) for destination, _ in candidates)
selected = [
candidate
for candidate in candidates
if len(candidate[0].parts) == longest_depth
]
if len(selected) != 1:
raise MountResolutionError("job-container workspace mount is ambiguous")
destination, mount = selected[0]
nested_mounts = [
nested_destination
for nested_destination, _ in parsed_mounts
if nested_destination != destination
and contains_path(scan_root, nested_destination)
]
if nested_mounts:
raise MountResolutionError(
"nested job-container mounts inside the audit scope cannot be "
"reproduced safely"
)
if mount.get("RW") is not True:
raise MountResolutionError("job-container workspace mount is not writable")
if not contains_path(destination, reports_path):
raise MountResolutionError(
f"audit reports path {reports_path} is outside the workspace mount"
)
destination_value = mount_option_value(
str(destination),
field="mount destination",
)
mount_type = mount.get("Type")
if mount_type == "volume":
name = mount.get("Name")
if not isinstance(name, str) or VOLUME_NAME.fullmatch(name) is None:
raise MountResolutionError("workspace volume name is missing or malformed")
source_value = name
elif mount_type == "bind":
source = mount.get("Source")
if not isinstance(source, str):
raise MountResolutionError("workspace bind source is missing")
source_path = normalized_absolute_path(
source,
field="mount source",
resolve_symlinks=False,
)
if source_path in UNSAFE_BIND_SOURCES or any(
contains_path(sensitive_root, source_path)
for sensitive_root in SENSITIVE_BIND_ROOTS
):
raise MountResolutionError(
"workspace bind source is too broad or sensitive"
)
source_value = mount_option_value(str(source_path), field="mount source")
else:
raise MountResolutionError(f"unsupported workspace mount type: {mount_type!r}")
return f"type={mount_type},source={source_value},target={destination_value}"
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Resolve one safe Gitea Actions workspace mount for an audit container."
)
parser.add_argument("--root", required=True)
parser.add_argument("--scope", choices=("current", "govoplan"), required=True)
parser.add_argument("--reports-dir", required=True)
return parser.parse_args()
def main() -> int:
args = parse_args()
try:
mounts = json.load(sys.stdin)
print(
resolve_workspace_mount(
mounts,
root=args.root,
scope=args.scope,
reports_dir=args.reports_dir,
)
)
except (json.JSONDecodeError, MountResolutionError) as exc:
print(f"workspace mount error: {exc}", file=sys.stderr)
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main())

44
tools/checks/security-audit/run.sh Normal file → Executable file
View File

@@ -139,13 +139,53 @@ if [[ "$BUILD_ONLY" == "1" ]]; then
exit 0
fi
if [[ "$SCOPE" == "govoplan" ]]; then
declare -a WORKSPACE_MOUNT=()
if [[ "${GITEA_ACTIONS:-}" == "true" ]]; then
ACTIONS_CONTAINER_ID="$(hostname)"
if ! ACTIONS_MOUNTS_JSON="$(
"${DOCKER_CLI[@]}" container inspect \
--format '{{json .Mounts}}' \
"$ACTIONS_CONTAINER_ID"
)"; then
echo "Gitea Actions is using a host Docker daemon, but the current job container could not be resolved." >&2
echo "Cannot safely share the checked-out workspace with the audit container." >&2
exit 1
fi
if command -v python3 >/dev/null 2>&1; then
MOUNT_PYTHON=python3
elif command -v python >/dev/null 2>&1; then
MOUNT_PYTHON=python
else
echo "Python is required to validate the Gitea Actions workspace mount." >&2
exit 1
fi
if ! ACTIONS_WORKSPACE_MOUNT="$(
printf '%s' "$ACTIONS_MOUNTS_JSON" \
| "$MOUNT_PYTHON" "$ROOT/tools/checks/security-audit/resolve_workspace_mount.py" \
--root "$ROOT" \
--scope "$SCOPE" \
--reports-dir "$REPORTS_DIR"
)"; then
echo "Cannot safely share the Gitea Actions workspace with the audit container." >&2
exit 1
fi
WORKSPACE_MOUNT=(--mount "$ACTIONS_WORKSPACE_MOUNT")
MOUNT_ROOT="$(dirname "$ROOT")"
WORKDIR="$ROOT"
if [[ "$SCOPE" == "govoplan" ]]; then
EXTRA_ENV=(-e "GOVOPLAN_REPOS_ROOT=$MOUNT_ROOT")
else
EXTRA_ENV=()
fi
elif [[ "$SCOPE" == "govoplan" ]]; then
MOUNT_ROOT="$(dirname "$ROOT")"
WORKDIR="/workspace/$(basename "$ROOT")"
WORKSPACE_MOUNT=(-v "$MOUNT_ROOT:/workspace")
EXTRA_ENV=(-e "GOVOPLAN_REPOS_ROOT=/workspace")
else
MOUNT_ROOT="$ROOT"
WORKDIR="/workspace"
WORKSPACE_MOUNT=(-v "$MOUNT_ROOT:/workspace")
EXTRA_ENV=()
fi
@@ -159,7 +199,7 @@ fi
-e SECURITY_AUDIT_REQUIRE_TOOLS="${SECURITY_AUDIT_REQUIRE_TOOLS:-0}" \
-e SECURITY_AUDIT_TOOLBOX_FINGERPRINT="$IMAGE_FINGERPRINT" \
"${EXTRA_ENV[@]}" \
-v "$MOUNT_ROOT:/workspace" \
"${WORKSPACE_MOUNT[@]}" \
-w "$WORKDIR" \
"$FINGERPRINT_IMAGE" \
bash tools/checks/check-security-audit.sh --mode "$MODE" --scope "$SCOPE" --reports-dir "$REPORTS_DIR" "${SCRIPT_ARGS[@]}"

View File

@@ -27,12 +27,17 @@ EXCLUDED_PARTS = {
".git",
".gitea",
".venv",
".mypy_cache",
".pytest_cache",
".ruff_cache",
"node_modules",
"__pycache__",
"audit-reports",
"dist",
"build",
".cache",
".module-test-build",
"runtime",
}
EXCLUDED_NAMES = {
"package-lock.json",
@@ -40,6 +45,14 @@ EXCLUDED_NAMES = {
"yarn.lock",
"uv.lock",
}
REPO_DOC_NAME_TOKENS = {
"architecture",
"backlog",
"plan",
"roadmap",
"todo",
"workflow",
}
@dataclasses.dataclass(frozen=True)
@@ -216,7 +229,8 @@ def is_repo_doc(repo_root_path: pathlib.Path, path: pathlib.Path) -> bool:
return False
if rel.parts[0] in {"docs", "doc", "codex"} and path.suffix.lower() in {".md", ".txt", ".csv"}:
return True
if re.search(r"(backlog|todo|roadmap|plan|architecture|workflow|manifest)", path.name, re.IGNORECASE):
name_tokens = set(re.split(r"[^a-z0-9]+", path.stem.lower()))
if name_tokens & REPO_DOC_NAME_TOKENS:
return True
return False

View File

@@ -22,6 +22,7 @@ FRONTEND_USE_POLLING="${GOVOPLAN_FRONTEND_USE_POLLING:-1}"
FRONTEND_POLLING_INTERVAL="${GOVOPLAN_FRONTEND_POLLING_INTERVAL:-500}"
AUTO_SYNC_PYTHON="${GOVOPLAN_AUTO_SYNC_PYTHON:-1}"
DEV_DATABASE_BACKEND="${GOVOPLAN_DEV_DATABASE_BACKEND:-postgres}"
BACKEND_RELOAD_MODULES="${GOVOPLAN_BACKEND_RELOAD_MODULES:-}"
LOG_DIR="${GOVOPLAN_DEV_LOG_DIR:-$META_ROOT/runtime/dev-launcher}"
BACKEND_LOG="$LOG_DIR/backend.log"
@@ -167,7 +168,16 @@ port_is_free "$BACKEND_HOST" "$BACKEND_PORT" || fail "$BACKEND_URL is already in
port_is_free "$FRONTEND_HOST" "$FRONTEND_PORT" || fail "$FRONTEND_URL is already in use"
printf 'Starting GovOPlaN backend at %s\n' "$BACKEND_URL"
run_grouped "$ROOT" "$BACKEND_LOG" "$PYTHON" -m govoplan_core.devserver --host "$BACKEND_HOST" --port "$BACKEND_PORT"
backend_args=(-m govoplan_core.devserver --host "$BACKEND_HOST" --port "$BACKEND_PORT")
if [ "$BACKEND_RELOAD_MODULES" = "none" ]; then
backend_args+=(--reload-core-only)
elif [ -n "$BACKEND_RELOAD_MODULES" ]; then
IFS=',' read -r -a reload_modules <<< "$BACKEND_RELOAD_MODULES"
for reload_module in "${reload_modules[@]}"; do
[ -n "$reload_module" ] && backend_args+=(--reload-module "$reload_module")
done
fi
run_grouped "$ROOT" "$BACKEND_LOG" "$PYTHON" "${backend_args[@]}"
backend_pid="$run_grouped_pid"
printf 'Waiting for %s/health\n' "$BACKEND_URL"

View File

@@ -51,7 +51,7 @@ set +a
export APP_ENV="${APP_ENV:-staging}"
export GOVOPLAN_INSTALL_PROFILE="${GOVOPLAN_INSTALL_PROFILE:-production-like}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,notifications,docs,ops}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,connectors,datasources,dataflow,workflow,views,search,risk_compliance,notifications,docs,ops}"
export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-postgresql+psycopg://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}"

View File

@@ -66,7 +66,7 @@ export DATABASE_URL="${DATABASE_URL:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL:-po
export GOVOPLAN_DATABASE_URL_PGTOOLS="${GOVOPLAN_DATABASE_URL_PGTOOLS:-${GOVOPLAN_PRODUCTION_LIKE_DATABASE_URL_PGTOOLS:-postgresql://govoplan:govoplan-dev@127.0.0.1:55433/govoplan}}"
export REDIS_URL="${REDIS_URL:-${GOVOPLAN_PRODUCTION_LIKE_REDIS_URL:-redis://127.0.0.1:56379/0}}"
export CELERY_ENABLED="${CELERY_ENABLED:-true}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,notifications,docs,ops}"
export ENABLED_MODULES="${ENABLED_MODULES:-tenancy,organizations,identity,access,admin,dashboard,policy,audit,campaigns,files,mail,calendar,poll,scheduling,connectors,datasources,dataflow,workflow,views,search,risk_compliance,notifications,docs,ops}"
export FILE_STORAGE_BACKEND="${FILE_STORAGE_BACKEND:-local}"
export FILE_STORAGE_LOCAL_ROOT="${FILE_STORAGE_LOCAL_ROOT:-$META_ROOT/runtime/production-like/files}"
export DEV_AUTO_MIGRATE_ENABLED="${DEV_AUTO_MIGRATE_ENABLED:-false}"

View File

@@ -26,7 +26,7 @@ from govoplan_core.server.registry import available_module_manifests # noqa: E4
from govoplan_release.version_alignment import selected_repository_version_issues # noqa: E402
GITEA_BASE = "git+ssh://git@git.add-ideas.de/add-ideas"
GITEA_BASE = "git+ssh://git@git.add-ideas.de/GovOPlaN"
@dataclass(frozen=True, slots=True)

View File

@@ -0,0 +1,397 @@
"""Independent identities for immutable Python wheel release artifacts."""
from __future__ import annotations
import base64
from dataclasses import dataclass
from email.parser import BytesParser
from email.policy import compat32
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import stat
import zipfile
WHEEL_PAYLOAD_ALGORITHM = "govoplan-wheel-declared-payload-v1"
INSTALLED_PAYLOAD_ALGORITHM = "govoplan-installed-record-payload-v1"
MAX_WHEEL_BYTES = 512 * 1024 * 1024
MAX_WHEEL_ENTRIES = 10_000
MAX_WHEEL_UNCOMPRESSED_BYTES = 1024 * 1024 * 1024
MAX_WHEEL_ENTRY_BYTES = 64 * 1024 * 1024
MAX_METADATA_BYTES = 1024 * 1024
_PACKAGE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
_VERSION = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._+!-]{0,127}$")
_PYTHON_REF = re.compile(
r"/(?P<repo>govoplan-[a-z0-9-]+)[.]git@v(?P<version>[^\s;]+)$"
)
class ArtifactIdentityError(ValueError):
"""A built artifact cannot provide a bounded immutable identity."""
@dataclass(frozen=True, slots=True)
class PythonWheelIdentity:
package_name: str
package_version: str
archive_sha256: str
archive_size: int
payload_sha256: str
payload_file_count: int
requires_installer_receipt: bool
def catalog_payload(self) -> dict[str, object]:
return {
"artifact_kind": "python-wheel",
"package_name": self.package_name,
"package_version": self.package_version,
"archive_sha256": self.archive_sha256,
"archive_size": self.archive_size,
"installed_payload": {
"algorithm": WHEEL_PAYLOAD_ALGORITHM,
"sha256": self.payload_sha256,
"file_count": self.payload_file_count,
},
"requires_installer_receipt": self.requires_installer_receipt,
}
def inspect_python_wheel(path: Path | str) -> PythonWheelIdentity:
"""Hash a regular built wheel and derive its install-stable payload identity."""
wheel = Path(path).expanduser()
flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)
try:
descriptor = os.open(wheel, flags)
except OSError as exc:
raise ArtifactIdentityError("built artifact cannot be opened safely") from exc
try:
archive_sha256, archive_size, initial = _hash_open_artifact(descriptor)
os.lseek(descriptor, 0, os.SEEK_SET)
try:
with os.fdopen(os.dup(descriptor), "rb") as artifact_handle:
with zipfile.ZipFile(artifact_handle) as archive:
(
package_name,
package_version,
payload_rows,
requires_receipt,
) = _inspect_wheel_archive(archive)
except (OSError, zipfile.BadZipFile, RuntimeError) as exc:
raise ArtifactIdentityError(
"built Python artifact is not a readable wheel"
) from exc
final = os.fstat(descriptor)
if _stat_fingerprint(final) != _stat_fingerprint(initial):
raise ArtifactIdentityError("built artifact changed while it was inspected")
finally:
os.close(descriptor)
return PythonWheelIdentity(
package_name=package_name,
package_version=package_version,
archive_sha256=archive_sha256,
archive_size=archive_size,
payload_sha256=payload_identity_sha256(
algorithm=WHEEL_PAYLOAD_ALGORITHM, rows=payload_rows
),
payload_file_count=len(payload_rows),
requires_installer_receipt=requires_receipt,
)
def _inspect_wheel_archive(
archive: zipfile.ZipFile,
) -> tuple[str, str, list[dict[str, object]], bool]:
infos = archive.infolist()
if not infos or len(infos) > MAX_WHEEL_ENTRIES:
raise ArtifactIdentityError("wheel entry count is empty or exceeds its limit")
_validate_members(infos)
metadata_members = [
item
for item in infos
if not item.is_dir()
and PurePosixPath(item.filename).name == "METADATA"
and PurePosixPath(item.filename).parent.name.endswith(".dist-info")
]
if len(metadata_members) != 1:
raise ArtifactIdentityError(
"wheel must contain exactly one dist-info METADATA file"
)
metadata_member = metadata_members[0]
dist_info = PurePosixPath(metadata_member.filename).parent
metadata_bytes = _read_member(
archive, metadata_member, max_bytes=MAX_METADATA_BYTES
)
parsed = BytesParser(policy=compat32).parsebytes(metadata_bytes)
package_name = normalize_package_name(str(parsed.get("Name") or ""))
package_version = str(parsed.get("Version") or "").strip()
if _PACKAGE.fullmatch(package_name) is None:
raise ArtifactIdentityError("wheel METADATA has an invalid package name")
if _VERSION.fullmatch(package_version) is None:
raise ArtifactIdentityError("wheel METADATA has an invalid package version")
payload_rows: list[dict[str, object]] = []
requires_receipt = False
for info in infos:
if info.is_dir():
continue
normalized, transformed = _wheel_payload_path(
PurePosixPath(info.filename), dist_info=dist_info
)
requires_receipt = requires_receipt or transformed
if normalized is None:
continue
encoded = _read_member(archive, info, max_bytes=MAX_WHEEL_ENTRY_BYTES)
payload_rows.append(
{
"path": normalized,
"sha256": hashlib.sha256(encoded).hexdigest(),
"size": len(encoded),
}
)
payload_rows.sort(key=lambda item: str(item["path"]))
if not payload_rows:
raise ArtifactIdentityError("wheel has no immutable payload entries")
if len({str(item["path"]) for item in payload_rows}) != len(payload_rows):
raise ArtifactIdentityError(
"wheel payload paths are ambiguous after installation mapping"
)
entry_points = f"{dist_info.as_posix()}/entry_points.txt"
if any(item.filename == entry_points for item in infos):
entry_point_info = next(item for item in infos if item.filename == entry_points)
entry_point_text = _read_member(
archive, entry_point_info, max_bytes=MAX_METADATA_BYTES
).decode("utf-8", errors="replace")
if re.search(r"(?m)^\s*\[(?:console|gui)_scripts\]\s*$", entry_point_text):
requires_receipt = True
return package_name, package_version, payload_rows, requires_receipt
def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
"""Report selected Python releases without a matching built-wheel identity."""
if not isinstance(payload, dict):
return ("candidate catalog must be a JSON object",)
release = payload.get("release")
selected_units = release.get("selected_units") if isinstance(release, dict) else None
artifacts = release.get("artifacts") if isinstance(release, dict) else None
if not isinstance(selected_units, list) or not selected_units:
return ("release.selected_units is missing or empty",)
entries: list[object] = [payload.get("core_release")]
modules = payload.get("modules")
if isinstance(modules, list):
entries.extend(modules)
package_by_repo: dict[str, tuple[str, str]] = {}
for entry in entries:
if not isinstance(entry, dict):
continue
python_ref = entry.get("python_ref")
match = _PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
package_name = entry.get("python_package")
version = entry.get("version")
if (
match is not None
and isinstance(package_name, str)
and _PACKAGE.fullmatch(package_name) is not None
and isinstance(version, str)
):
package_by_repo[match.group("repo")] = (package_name, version)
artifacts_by_package: dict[str, dict[str, object]] = {}
malformed_artifacts = False
if not isinstance(artifacts, list):
artifacts = []
for artifact in artifacts:
if not isinstance(artifact, dict):
malformed_artifacts = True
continue
package_name = artifact.get("package_name")
if not isinstance(package_name, str) or package_name in artifacts_by_package:
malformed_artifacts = True
continue
if not _catalog_artifact_shape_valid(artifact):
malformed_artifacts = True
continue
artifacts_by_package[package_name] = artifact
issues: list[str] = []
if malformed_artifacts:
issues.append("release.artifacts contains malformed or duplicate identities")
seen_repos: set[str] = set()
for unit in selected_units:
if not isinstance(unit, dict):
issues.append("release.selected_units contains a malformed entry")
continue
repo = unit.get("repo")
if not isinstance(repo, str) or repo in seen_repos:
issues.append("release.selected_units contains a duplicate or invalid repository")
continue
seen_repos.add(repo)
expected = package_by_repo.get(repo)
if expected is None:
continue # Selected non-Python repositories have no wheel identity.
package_name, version = expected
artifact = artifacts_by_package.get(package_name)
if artifact is None:
issues.append(
f"selected Python repository {repo} has no built artifact identity for {package_name} {version}"
)
elif artifact.get("package_version") != version:
issues.append(
f"selected Python repository {repo} artifact identity has version {artifact.get('package_version')!r}, expected {version!r}"
)
return tuple(issues)
def _catalog_artifact_shape_valid(value: dict[str, object]) -> bool:
if set(value) != {
"artifact_kind",
"package_name",
"package_version",
"archive_sha256",
"archive_size",
"installed_payload",
"requires_installer_receipt",
}:
return False
package_name = value.get("package_name")
version = value.get("package_version")
archive_sha256 = value.get("archive_sha256")
archive_size = value.get("archive_size")
installed_payload = value.get("installed_payload")
return (
value.get("artifact_kind") == "python-wheel"
and isinstance(package_name, str)
and _PACKAGE.fullmatch(package_name) is not None
and isinstance(version, str)
and _VERSION.fullmatch(version) is not None
and isinstance(archive_sha256, str)
and re.fullmatch(r"[0-9a-f]{64}", archive_sha256) is not None
and isinstance(archive_size, int)
and not isinstance(archive_size, bool)
and 0 < archive_size <= MAX_WHEEL_BYTES
and isinstance(value.get("requires_installer_receipt"), bool)
and isinstance(installed_payload, dict)
and set(installed_payload) == {"algorithm", "sha256", "file_count"}
and installed_payload.get("algorithm") == WHEEL_PAYLOAD_ALGORITHM
and isinstance(installed_payload.get("sha256"), str)
and re.fullmatch(r"[0-9a-f]{64}", str(installed_payload["sha256"]))
is not None
and isinstance(installed_payload.get("file_count"), int)
and not isinstance(installed_payload.get("file_count"), bool)
and 0 < int(installed_payload["file_count"]) <= MAX_WHEEL_ENTRIES
)
def payload_identity_sha256(
*, algorithm: str, rows: list[dict[str, object]]
) -> str:
payload = {"algorithm": algorithm, "files": rows}
encoded = json.dumps(
payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def record_hash_hex(value: str) -> str | None:
try:
decoded = base64.urlsafe_b64decode(value + "=" * (-len(value) % 4))
except (ValueError, TypeError):
return None
return decoded.hex() if len(decoded) == hashlib.sha256().digest_size else None
def normalize_package_name(value: str) -> str:
return re.sub(r"[-_.]+", "-", value.strip().lower())
def _hash_open_artifact(descriptor: int) -> tuple[str, int, os.stat_result]:
digest = hashlib.sha256()
total = 0
initial = os.fstat(descriptor)
if not stat.S_ISREG(initial.st_mode) or initial.st_size > MAX_WHEEL_BYTES:
raise ArtifactIdentityError("built artifact must be a bounded regular file")
while total < initial.st_size:
chunk = os.read(descriptor, min(1024 * 1024, initial.st_size - total))
if not chunk:
break
total += len(chunk)
digest.update(chunk)
final = os.fstat(descriptor)
if total != initial.st_size or _stat_fingerprint(final) != _stat_fingerprint(initial):
raise ArtifactIdentityError("built artifact changed while it was hashed")
return digest.hexdigest(), total, initial
def _stat_fingerprint(value: os.stat_result) -> tuple[int, int, int, int, int]:
return (
value.st_dev,
value.st_ino,
value.st_size,
value.st_mtime_ns,
value.st_ctime_ns,
)
def _validate_members(infos: list[zipfile.ZipInfo]) -> None:
names: set[str] = set()
total = 0
for info in infos:
path = PurePosixPath(info.filename.replace("\\", "/"))
if (
not info.filename
or path.is_absolute()
or ".." in path.parts
or any(ord(character) < 32 for character in info.filename)
or info.flag_bits & 0x1
):
raise ArtifactIdentityError("wheel contains an unsafe member")
normalized = path.as_posix()
if normalized in names:
raise ArtifactIdentityError("wheel contains duplicate members")
names.add(normalized)
if _zip_member_is_symlink(info):
raise ArtifactIdentityError("wheel contains a symbolic-link member")
total += info.file_size
if info.file_size > MAX_WHEEL_ENTRY_BYTES or total > MAX_WHEEL_UNCOMPRESSED_BYTES:
raise ArtifactIdentityError("wheel uncompressed payload exceeds its limit")
def _zip_member_is_symlink(info: zipfile.ZipInfo) -> bool:
return stat.S_ISLNK((info.external_attr >> 16) & 0o177777)
def _read_member(
archive: zipfile.ZipFile, info: zipfile.ZipInfo, *, max_bytes: int
) -> bytes:
if info.file_size > max_bytes:
raise ArtifactIdentityError("wheel member exceeds its size limit")
with archive.open(info, "r") as handle:
encoded = handle.read(max_bytes + 1)
if len(encoded) > max_bytes or len(encoded) != info.file_size:
raise ArtifactIdentityError("wheel member size is inconsistent")
return encoded
def _wheel_payload_path(
path: PurePosixPath, *, dist_info: PurePosixPath
) -> tuple[str | None, bool]:
if path.parent == dist_info and path.name in {"RECORD", "RECORD.jws", "RECORD.p7s"}:
return None, False
parts = path.parts
data_prefix = dist_info.name.removesuffix(".dist-info") + ".data"
if parts and parts[0] == data_prefix:
if len(parts) < 3:
raise ArtifactIdentityError("wheel data member has no installation target")
scheme = parts[1]
remainder = PurePosixPath(*parts[2:]).as_posix()
if scheme in {"purelib", "platlib"}:
return remainder, False
if scheme == "data":
return f"@data/{remainder}", False
if scheme in {"scripts", "headers"}:
return None, True
raise ArtifactIdentityError("wheel uses an unsupported installation scheme")
return path.as_posix(), False

View File

@@ -0,0 +1,254 @@
"""Opaque, workspace-scoped handles for generated catalog candidates."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import hmac
import json
import os
from pathlib import Path
import re
import stat
from typing import Iterable
from .catalog import canonical_hash
MAX_CATALOG_BYTES = 16 * 1024 * 1024
_CANDIDATE_ID = re.compile(r"^candidate-[0-9a-f]{32}$")
_CHANNEL = re.compile(r"^[a-z][a-z0-9_-]{0,63}$")
class CandidateArtifactError(ValueError):
"""A candidate handle or the artifact behind it is unsafe or inconsistent."""
@dataclass(frozen=True, slots=True)
class CandidateArtifactReceipt:
candidate_id: str
catalog_sha256: str
def issue_candidate_id(*seed_parts: str) -> str:
"""Derive an opaque, deterministic candidate basename before output exists."""
if not seed_parts or any(
not isinstance(part, str) or not part or len(part) > 512
for part in seed_parts
):
raise CandidateArtifactError("candidate ID seeds must be non-empty bounded strings")
encoded = json.dumps(
list(seed_parts), sort_keys=False, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return f"candidate-{hashlib.sha256(encoded).hexdigest()[:32]}"
def validate_release_channel(value: str) -> str:
"""Return one bounded channel basename or reject path-capable input."""
if not isinstance(value, str) or _CHANNEL.fullmatch(value) is None:
raise CandidateArtifactError(
"release channel must be a bounded lowercase identifier"
)
return value
def candidate_output_path(root: Path | str, candidate_id: str) -> Path:
"""Resolve a not-yet-created candidate path below a trusted configured root."""
checked_id = _checked_candidate_id(candidate_id)
root_path = Path(root).expanduser()
ensure_private_candidate_root(root_path, create=True)
candidate = root_path / checked_id
try:
mode = candidate.lstat().st_mode
except FileNotFoundError:
return candidate
except OSError as exc:
raise CandidateArtifactError("candidate output path cannot be inspected") from exc
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise CandidateArtifactError("candidate output path must be a real directory")
_require_private_owner_mode(candidate, directory=True, label="candidate output path")
return candidate
def ensure_private_candidate_root(
root: Path | str, *, create: bool = False
) -> Path:
"""Admit only an operator-owned candidate root inaccessible to other users."""
root_path = Path(root).expanduser()
_reject_symlink_components(root_path, allow_missing=create)
if create:
try:
root_path.mkdir(mode=0o700, parents=True, exist_ok=True)
except OSError as exc:
raise CandidateArtifactError("candidate root cannot be created") from exc
_reject_symlink_components(root_path, allow_missing=False)
_require_private_owner_mode(root_path, directory=True, label="candidate root")
return root_path
def harden_private_candidate_tree(path: Path | str) -> Path:
"""Seal a newly generated, operator-owned candidate tree before a receipt."""
root = Path(path).expanduser()
_reject_symlink_components(root, allow_missing=False)
_require_current_owner(root, label="candidate directory")
try:
os.chmod(root, 0o700, follow_symlinks=False)
for current, directory_names, file_names in os.walk(root, followlinks=False):
current_path = Path(current)
_require_current_owner(current_path, label="candidate directory")
os.chmod(current_path, 0o700, follow_symlinks=False)
for name in (*directory_names, *file_names):
child = current_path / name
mode = child.lstat().st_mode
if stat.S_ISLNK(mode):
raise CandidateArtifactError("candidate tree contains a symlink")
_require_current_owner(child, label="candidate tree member")
if stat.S_ISDIR(mode):
os.chmod(child, 0o700, follow_symlinks=False)
elif stat.S_ISREG(mode):
os.chmod(child, 0o600, follow_symlinks=False)
else:
raise CandidateArtifactError(
"candidate tree contains a non-file member"
)
except OSError as exc:
raise CandidateArtifactError("candidate tree cannot be sealed") from exc
return root
def issue_candidate_receipt(
*, root: Path | str, candidate_id: str, channel: str
) -> CandidateArtifactReceipt:
"""Hash the signed catalog at a generated candidate handle."""
candidate = _existing_candidate_path(root, candidate_id)
payload = _read_signed_catalog(candidate, channel=channel)
return CandidateArtifactReceipt(
candidate_id=candidate_id,
catalog_sha256=canonical_hash(payload),
)
def verify_candidate_receipt(
*,
root: Path | str,
candidate_id: str,
catalog_sha256: str,
channel: str,
) -> Path:
"""Re-resolve and re-hash a persisted receipt before candidate consumption."""
if re.fullmatch(r"[0-9a-f]{64}", catalog_sha256) is None:
raise CandidateArtifactError("candidate catalog digest is malformed")
candidate = _existing_candidate_path(root, candidate_id)
payload = _read_signed_catalog(candidate, channel=channel)
if not hmac.compare_digest(canonical_hash(payload), catalog_sha256):
raise CandidateArtifactError("candidate catalog no longer matches its receipt")
return candidate
def _checked_candidate_id(candidate_id: str) -> str:
if not isinstance(candidate_id, str) or _CANDIDATE_ID.fullmatch(candidate_id) is None:
raise CandidateArtifactError("candidate ID is not an issued opaque basename")
return candidate_id
def _existing_candidate_path(root: Path | str, candidate_id: str) -> Path:
candidate = candidate_output_path(root, candidate_id)
try:
mode = candidate.lstat().st_mode
except OSError as exc:
raise CandidateArtifactError("candidate directory is unavailable") from exc
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise CandidateArtifactError("candidate directory must be a real directory")
_reject_symlink_components(candidate, allow_missing=False)
return candidate
def _read_signed_catalog(candidate: Path, *, channel: str) -> dict[str, object]:
channel = validate_release_channel(channel)
channels = candidate / "channels"
catalog_path = channels / f"{channel}.json"
_require_real_directory(channels, label="candidate channels directory")
_require_regular_file(catalog_path, label="candidate catalog")
try:
with catalog_path.open("rb") as handle:
encoded = handle.read(MAX_CATALOG_BYTES + 1)
except OSError as exc:
raise CandidateArtifactError("candidate catalog cannot be read") from exc
if len(encoded) > MAX_CATALOG_BYTES:
raise CandidateArtifactError("candidate catalog exceeds its size limit")
try:
payload = json.loads(encoded.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise CandidateArtifactError("candidate catalog is not valid JSON") from exc
if not isinstance(payload, dict):
raise CandidateArtifactError("candidate catalog must be a JSON object")
if payload.get("channel") != channel:
raise CandidateArtifactError(
"candidate catalog channel does not match its requested handle"
)
signatures = payload.get("signatures")
if not isinstance(signatures, list) or not signatures:
raise CandidateArtifactError("candidate catalog has no signature envelope")
return payload
def _reject_symlink_components(path: Path, *, allow_missing: bool) -> None:
absolute = path.absolute()
parts: Iterable[Path] = reversed((absolute, *absolute.parents))
for component in parts:
try:
mode = component.lstat().st_mode
except FileNotFoundError:
if allow_missing:
continue
raise CandidateArtifactError("candidate path has a missing component")
except OSError as exc:
raise CandidateArtifactError("candidate path cannot be inspected") from exc
if stat.S_ISLNK(mode):
raise CandidateArtifactError("candidate path must not traverse symlinks")
def _require_real_directory(path: Path, *, label: str) -> None:
try:
mode = path.lstat().st_mode
except OSError as exc:
raise CandidateArtifactError(f"{label} is unavailable") from exc
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise CandidateArtifactError(f"{label} must be a real directory")
_require_private_owner_mode(path, directory=True, label=label)
def _require_regular_file(path: Path, *, label: str) -> None:
try:
mode = path.lstat().st_mode
except OSError as exc:
raise CandidateArtifactError(f"{label} is unavailable") from exc
if stat.S_ISLNK(mode) or not stat.S_ISREG(mode):
raise CandidateArtifactError(f"{label} must be a regular file")
_require_private_owner_mode(path, directory=False, label=label)
def _require_current_owner(path: Path, *, label: str) -> os.stat_result:
try:
observed = path.lstat()
except OSError as exc:
raise CandidateArtifactError(f"{label} cannot be inspected") from exc
if observed.st_uid != os.geteuid():
raise CandidateArtifactError(f"{label} is not owned by the current operator")
return observed
def _require_private_owner_mode(
path: Path, *, directory: bool, label: str
) -> None:
observed = _require_current_owner(path, label=label)
expected_type = stat.S_ISDIR if directory else stat.S_ISREG
if not expected_type(observed.st_mode) or stat.S_IMODE(observed.st_mode) & 0o077:
raise CandidateArtifactError(f"{label} is accessible to another user")

View File

@@ -3,6 +3,7 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import re
import subprocess
@@ -163,18 +164,66 @@ def git_success(path: Path, *args: str, timeout: int = 10) -> bool:
def git(path: Path, *args: str, timeout: int = 10) -> subprocess.CompletedProcess[str]:
command = scoped_git_command(path, *args)
try:
return subprocess.run(
["git", *args],
command,
cwd=path,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout,
env=sanitized_git_environment(),
)
except subprocess.TimeoutExpired as exc:
return subprocess.CompletedProcess(["git", *args], 124, exc.stdout or "", exc.stderr or "git command timed out")
return subprocess.CompletedProcess(command, 124, exc.stdout or "", exc.stderr or "git command timed out")
def scoped_git_command(path: Path, *args: str) -> tuple[str, ...]:
"""Trust exactly one resolved catalog checkout for one Git invocation."""
return (
"/usr/bin/git",
"-c",
"core.hooksPath=/dev/null",
"-c",
f"safe.directory={path.resolve()}",
*args,
)
def sanitized_git_environment(
source: dict[str, str] | None = None,
) -> dict[str, str]:
"""Keep only deliberate process/auth inputs and neutralize Git redirection."""
environment = os.environ if source is None else source
result = {
key: environment[key]
for key in (
"HOME",
"LANG",
"LC_ALL",
"LC_CTYPE",
"SSH_AUTH_SOCK",
)
if environment.get(key)
}
result.update(
{
"GIT_CONFIG_GLOBAL": os.devnull,
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_CONFIG_COUNT": "0",
"GIT_NO_REPLACE_OBJECTS": "1",
"GIT_OPTIONAL_LOCKS": "0",
"GIT_PAGER": "cat",
"GIT_SSH_COMMAND": "/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8",
"GIT_TERMINAL_PROMPT": "0",
"PATH": "/usr/bin:/bin",
}
)
return result
def git_error(result: subprocess.CompletedProcess[str]) -> str:

View File

@@ -334,6 +334,7 @@ class CatalogPublishResult:
target_keyring_path: str
branch: str | None
tag_name: str | None
remote: str
validation_valid: bool
validation_error: str | None = None
validation_warnings: tuple[str, ...] = ()
@@ -343,6 +344,9 @@ class CatalogPublishResult:
target_keyring_hash_before: str | None = None
catalog_changed: bool = False
keyring_changed: bool = False
publication_commit_sha: str | None = None
publication_tag_object_sha: str | None = None
publication_tag_commit_sha: str | None = None
steps: tuple[CatalogPublishStep, ...] = ()
notes: tuple[str, ...] = ()

View File

@@ -9,6 +9,7 @@ import re
from typing import Any
from .catalog import DEFAULT_PUBLIC_BASE_URL, canonical_hash
from .candidate_artifact import validate_release_channel
from .release_intelligence import compatibility_rows, module_rows, signature_rows
@@ -42,7 +43,16 @@ def module_directory_payloads(
channel: str,
public_base_url: str = DEFAULT_PUBLIC_BASE_URL,
) -> tuple[tuple[Path, dict[str, Any]], ...]:
generated_at = json_datetime(datetime.now(tz=UTC))
channel = validate_release_channel(channel)
# Publication artifacts must be reproducible from the signed catalog so an
# interrupted push can later be reconciled against the immutable commit.
# Older/ad-hoc callers without a catalog timestamp retain the old fallback.
catalog_generated_at = catalog_payload.get("generated_at")
generated_at = (
catalog_generated_at
if isinstance(catalog_generated_at, str) and catalog_generated_at
else json_datetime(datetime.now(tz=UTC))
)
modules = module_rows(catalog_payload)
compatibility = compatibility_rows(modules)
signatures = signature_rows(catalog_payload, keyring_payload)
@@ -59,8 +69,8 @@ def module_directory_payloads(
if not module_id:
continue
version = str(module.get("version") or "0.0.0")
module_slug = safe_path_part(module_id)
version_slug = safe_path_part(version)
module_slug = safe_module_id(module_id)
version_slug = safe_version(version)
module_base = f"{base_url}/catalogs/v1/modules/{module_slug}"
manifest_url = f"{module_base}/{version_slug}/manifest.json"
module_index_url = f"{module_base}/index.json"
@@ -139,7 +149,25 @@ def module_directory_payloads(
def safe_path_part(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "_", value.strip()) or "_"
if (
not isinstance(value, str)
or value in {".", ".."}
or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.+!-]{0,127}", value) is None
):
raise ValueError("module-directory path part is not canonical")
return value
def safe_module_id(value: str) -> str:
if re.fullmatch(r"[a-z][a-z0-9_-]{0,63}", value) is None:
raise ValueError("module ID is not canonical")
return safe_path_part(value)
def safe_version(value: str) -> str:
if re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+!-]{0,127}", value) is None:
raise ValueError("module version is not canonical")
return safe_path_part(value)
def json_datetime(value: datetime) -> str:

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,14 +1,16 @@
"""Durable state for guided, explicitly executed release runs.
"""Durable state and bounded receipts for explicitly executed release runs.
The run store records an immutable selective-plan snapshot and a small mutable
state machine. It does not execute release commands. Existing release
executors remain responsible for their narrow confirmations and will be wired
to this state in a later slice.
The store freezes a selective plan, claims each supported attempt before its
executor is called, and records only bounded outcomes. Narrow executors remain
separate from this persistence module.
"""
from __future__ import annotations
import base64
import binascii
from copy import deepcopy
from dataclasses import dataclass
from datetime import UTC, datetime
import hashlib
import json
@@ -22,21 +24,25 @@ from typing import Any
from filelock import FileLock, Timeout as FileLockTimeout
from .candidate_artifact import CandidateArtifactError, validate_release_channel
SCHEMA = "govoplan.release-run"
SCHEMA_VERSION = 3
SCHEMA_VERSION = 4
LEGACY_SCHEMA_VERSIONS = {3}
MAX_RECORD_BYTES = 2 * 1024 * 1024
MAX_PLAN_STEPS = 512
MAX_REPOSITORY_VERSIONS = 128
MAX_EVENTS = 256
MAX_COMMANDS = 2_048
MAX_LIST_LIMIT = 100
MAX_RUN_RECORDS = 512
MAX_CURSOR_BYTES = 1_024
RUN_ID_PATTERN = re.compile(
r"^(?:rr-[0-9]{8}T[0-9]{6}Z-[0-9a-f]{12}|rr-request-[0-9a-f]{64})$"
)
STEP_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@-]{0,159}$")
REQUEST_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/-]{7,127}$")
CHANNEL_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$")
REPOSITORY_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$")
WORKSPACE_FINGERPRINT_PATTERN = re.compile(r"^[0-9a-f]{64}$")
VERSION_PATTERN = re.compile(
@@ -61,6 +67,13 @@ RECONCILIATION_CONFIRMATION = "RECONCILE"
START_RECOVERY_RESERVE = 2
PROJECTION_TIMESTAMP = "9999-12-31T23:59:59Z"
PROJECTION_RESULT_CODE = "r" * 64
RECEIPT_KINDS = {
"catalog_candidate",
"catalog_publication",
"repository_state",
}
RECEIPT_ID_PATTERN = re.compile(r"^candidate-[0-9a-f]{32}$")
GIT_OBJECT_PATTERN = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$")
class ReleaseRunError(RuntimeError):
@@ -83,6 +96,17 @@ class ReleaseRunCorrupt(ReleaseRunError):
"""Raised when a persisted record fails its integrity or schema checks."""
@dataclass(frozen=True, slots=True)
class ReleaseStepClaim:
"""Private executor claim result; attempt identifiers remain fingerprinted."""
run: dict[str, Any]
claimed: bool
outcome: str
result_code: str | None
result_receipt: dict[str, Any] | None
class ReleaseRunStore:
"""Atomic, bounded local JSON store for release-run state."""
@@ -131,6 +155,7 @@ class ReleaseRunStore:
"started_at": None,
"finished_at": None,
"result_code": None,
"result_receipt": None,
}
for item in immutable_plan["dry_run_steps"]
]
@@ -165,7 +190,16 @@ class ReleaseRunStore:
input_snapshot=immutable_input,
)
return _view(existing)
retirement = self._retirement_candidate()
self._write_new(record)
if retirement is not None:
try:
self._unlink_retained_record(retirement)
except ReleaseRunError:
# Do not leave the store beyond its hard record bound when
# retirement fails before it can make room.
self._unlink_new_record(self._path(run_id))
raise
return _view(record)
def find_created_run(
@@ -199,59 +233,78 @@ class ReleaseRunStore:
return _view(record)
def list(self, *, limit: int = 20) -> list[dict[str, Any]]:
"""Return the first page for backwards-compatible local callers."""
return self.list_page(limit=limit)["runs"]
def list_page(
self, *, limit: int = 20, cursor: str | None = None
) -> dict[str, Any]:
"""Return a deterministic descending page, including corrupt entries."""
limit = min(max(int(limit), 1), MAX_LIST_LIMIT)
cursor_key = _decode_list_cursor(
cursor,
expected_workspace_fingerprint=self.expected_workspace_fingerprint,
)
with self._locked():
self._ensure_root()
try:
paths = [
path
for path in self.root.iterdir()
if path.name.endswith(".json")
and RUN_ID_PATTERN.fullmatch(path.stem)
]
except OSError as exc:
raise ReleaseRunCorrupt(
"Release run storage could not be enumerated safely."
) from exc
summaries: list[dict[str, Any]] = []
unavailable: list[dict[str, Any]] = []
for path in paths:
entries: list[tuple[tuple[int, str, str, str], dict[str, Any]]] = []
for path in self._record_paths():
try:
record = self._read(path.stem)
except ReleaseRunWorkspaceMismatch:
continue
except (ReleaseRunCorrupt, ReleaseRunNotFound):
unavailable.append(
{
"run_id": path.stem,
"status": "unavailable",
"error": "Run record is unavailable or failed integrity validation.",
}
entries.append(
(
(0, "", "", path.stem),
{
"run_id": path.stem,
"status": "unavailable",
"error": "Run record is unavailable or failed integrity validation.",
},
)
)
continue
view = _view(record)
immutable_input = view["immutable"]["input"]
summaries.append(
{
"run_id": view["run_id"],
"created_at": view["created_at"],
"updated_at": view["updated_at"],
"status": view["state"]["status"],
"channel": immutable_input["channel"],
"repo_versions": immutable_input["repo_versions"],
"recommended_next": view["recommended_next"],
}
entries.append(
(
(
1,
view["created_at"],
view["created_at"],
view["run_id"],
),
{
"run_id": view["run_id"],
"created_at": view["created_at"],
"updated_at": view["updated_at"],
"status": view["state"]["status"],
"channel": immutable_input["channel"],
"repo_versions": immutable_input["repo_versions"],
"recommended_next": view["recommended_next"],
},
)
)
summaries.sort(
key=lambda item: (
item["updated_at"],
item["created_at"],
item["run_id"],
),
reverse=True,
entries.sort(key=lambda item: item[0], reverse=True)
if cursor_key is not None:
entries = [entry for entry in entries if entry[0] < cursor_key]
page = entries[: limit + 1]
has_more = len(page) > limit
visible = page[:limit]
next_cursor = (
_encode_list_cursor(
visible[-1][0],
workspace_fingerprint=self.expected_workspace_fingerprint,
)
unavailable.sort(key=lambda item: item["run_id"], reverse=True)
return (summaries + unavailable)[:limit]
if has_more and visible
else None
)
return {
"runs": [summary for _key, summary in visible],
"next_cursor": next_cursor,
}
def resume(self, run_id: str, *, request_id: str) -> dict[str, Any]:
request_fingerprint = _request_fingerprint(request_id)
@@ -332,6 +385,7 @@ class ReleaseRunStore:
"started_at": None,
"finished_at": None,
"result_code": None,
"result_receipt": None,
}
)
_remember_request(
@@ -359,6 +413,7 @@ class ReleaseRunStore:
request_id: str,
outcome: str,
confirmation: str,
result_receipt: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Record an operator-observed outcome for an interrupted mutation."""
@@ -369,12 +424,22 @@ class ReleaseRunStore:
raise ReleaseRunConflict(
f"Type {RECONCILIATION_CONFIRMATION} to record reconciliation."
)
receipt = _validated_result_receipt(result_receipt)
if receipt is not None and outcome != "effect_succeeded":
raise ReleaseRunConflict(
"Only a reconciled successful effect can carry a result receipt."
)
request_fingerprint = _request_fingerprint(request_id)
with self._locked():
record = self._read(run_id)
if _request_seen(
record, request_fingerprint, "reconcile", step_id, outcome
):
step, _plan_step = _step_pair(record, step_id)
if step.get("result_receipt") != receipt:
raise ReleaseRunConflict(
"The reconciliation request already has a different result receipt."
)
return _view(record)
step, plan_step = _step_pair(record, step_id)
if step["state"] != "interrupted" or not bool(
@@ -406,6 +471,7 @@ class ReleaseRunStore:
"started_at": None,
"finished_at": None,
"result_code": None,
"result_receipt": None,
}
)
elif outcome == "effect_succeeded":
@@ -414,6 +480,7 @@ class ReleaseRunStore:
"state": "succeeded",
"finished_at": _timestamp(),
"result_code": "reconciled_effect_succeeded",
"result_receipt": receipt,
}
)
_remember_request(
@@ -440,10 +507,20 @@ class ReleaseRunStore:
*,
attempt_id: str,
) -> dict[str, Any]:
"""Claim one step for a future confirmed executor.
"""Claim one step while preserving the original store API."""
This is deliberately not an HTTP endpoint in the foundation slice.
"""
return self.claim_step(
run_id, step_id, attempt_id=attempt_id
).run
def claim_step(
self,
run_id: str,
step_id: str,
*,
attempt_id: str,
) -> ReleaseStepClaim:
"""Durably claim a step and distinguish a delayed attempt replay."""
_validate_step_id(step_id)
attempt_fingerprint = _request_fingerprint(attempt_id)
@@ -455,7 +532,14 @@ class ReleaseRunStore:
raise ReleaseRunConflict(
"The attempt identifier was already used for another release step."
)
return _view(record)
step, _plan_step = _step_pair(record, step_id)
return ReleaseStepClaim(
run=_view(record),
claimed=False,
outcome=existing_attempt["outcome"],
result_code=existing_attempt["result_code"],
result_receipt=deepcopy(step.get("result_receipt")),
)
step, plan_step = _step_pair(record, step_id)
available, reason = _step_available(record, step_id)
if not available:
@@ -470,6 +554,7 @@ class ReleaseRunStore:
"started_at": _timestamp(),
"finished_at": None,
"result_code": None,
"result_receipt": None,
}
)
record["state"]["processed_attempts"].append(
@@ -487,7 +572,13 @@ class ReleaseRunStore:
mutating=bool(plan_step.get("mutating")),
)
self._persist_update(record)
return _view(record)
return ReleaseStepClaim(
run=_view(record),
claimed=True,
outcome="running",
result_code=None,
result_receipt=None,
)
def finish_step(
self,
@@ -497,12 +588,16 @@ class ReleaseRunStore:
attempt_id: str,
succeeded: bool,
result_code: str,
result_receipt: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Record a future executor outcome without persisting its output."""
"""Record a bounded executor outcome without persisting process output."""
_validate_step_id(step_id)
attempt_fingerprint = _request_fingerprint(attempt_id)
result_code = _safe_code(result_code)
receipt = _validated_result_receipt(result_receipt)
if receipt is not None and not succeeded:
raise ReleaseRunConflict("A failed release step cannot carry a result receipt.")
target_state = "succeeded" if succeeded else "failed"
with self._locked():
record = self._read(run_id)
@@ -516,6 +611,7 @@ class ReleaseRunStore:
if (
attempt["outcome"] == target_state
and attempt["result_code"] == result_code
and step.get("result_receipt") == receipt
):
return _view(record)
raise ReleaseRunConflict(
@@ -533,6 +629,7 @@ class ReleaseRunStore:
"state": target_state,
"finished_at": _timestamp(),
"result_code": result_code,
"result_receipt": receipt,
}
)
attempt["outcome"] = target_state
@@ -546,6 +643,76 @@ class ReleaseRunStore:
self._persist_update(record)
return _view(record)
def interrupt_step(
self,
run_id: str,
step_id: str,
*,
attempt_id: str,
result_code: str = "executor_outcome_unknown",
) -> dict[str, Any]:
"""Freeze an executor exception as an ambiguous, reconcilable attempt."""
_validate_step_id(step_id)
attempt_fingerprint = _request_fingerprint(attempt_id)
result_code = _safe_code(result_code)
if result_code != "executor_outcome_unknown":
raise ReleaseRunConflict("Executor interruption result code is invalid.")
with self._locked():
record = self._read(run_id)
step, _plan_step = _step_pair(record, step_id)
attempt = _find_attempt(record, attempt_fingerprint)
if attempt is None or attempt["step_id"] != step_id:
raise ReleaseRunConflict(
"The release attempt identifier was not claimed for this step."
)
if attempt["outcome"] == "interrupted":
if attempt["result_code"] == result_code:
return _view(record)
raise ReleaseRunConflict(
"The release attempt already has a different interruption outcome."
)
if (
attempt["outcome"] != "running"
or step["state"] != "running"
or step["attempt_fingerprint"] != attempt_fingerprint
):
raise ReleaseRunConflict(
"The release step is not running with this attempt identifier."
)
step.update(
{
"state": "interrupted",
"finished_at": _timestamp(),
"result_code": result_code,
"result_receipt": None,
}
)
attempt["outcome"] = "interrupted"
attempt["result_code"] = result_code
_append_event(
record,
event_type="step_interrupted",
step_id=step_id,
result_code=result_code,
)
self._persist_update(record)
return _view(record)
def current_attempt_fingerprint(self, run_id: str, step_id: str) -> str:
"""Return the opaque current fingerprint for server-side artifact recovery."""
_validate_step_id(step_id)
with self._locked():
record = self._read(run_id)
step, _plan_step = _step_pair(record, step_id)
fingerprint = step.get("attempt_fingerprint")
if not isinstance(fingerprint, str):
raise ReleaseRunConflict(
"Release step has no current attempt to reconcile."
)
return fingerprint
def _persist_update(self, record: dict[str, Any]) -> None:
self._assert_workspace(record)
record["updated_at"] = _timestamp()
@@ -606,6 +773,9 @@ class ReleaseRunStore:
) from exc
_validate_record(payload, expected_run_id=run_id)
self._assert_workspace(payload)
if payload["schema_version"] in LEGACY_SCHEMA_VERSIONS:
payload = _upgrade_record(payload)
self._atomic_write(path, payload)
return payload
def _read_optional(self, run_id: str) -> dict[str, Any] | None:
@@ -614,6 +784,88 @@ class ReleaseRunStore:
except ReleaseRunNotFound:
return None
def _record_paths(self) -> list[Path]:
self._ensure_root()
try:
return [
path
for path in self.root.iterdir()
if path.name.endswith(".json")
and RUN_ID_PATTERN.fullmatch(path.stem)
]
except OSError as exc:
raise ReleaseRunCorrupt(
"Release run storage could not be enumerated safely."
) from exc
def _retirement_candidate(self) -> Path | None:
"""Select one verified terminal record without deleting it yet."""
paths = self._record_paths()
if len(paths) < MAX_RUN_RECORDS:
return None
if len(paths) > MAX_RUN_RECORDS:
raise ReleaseRunConflict(
"Release run storage exceeds its retention limit; inspect and "
"repair the private store before creating another run."
)
completed: list[tuple[tuple[str, str, str], Path]] = []
for path in paths:
try:
record = self._read(path.stem)
except (ReleaseRunCorrupt, ReleaseRunNotFound, ReleaseRunWorkspaceMismatch):
# Unavailable evidence is deliberately never deleted implicitly.
continue
if record["state"]["status"] == "completed":
completed.append(
(
(
record["updated_at"],
record["created_at"],
record["run_id"],
),
path,
)
)
completed.sort(key=lambda item: item[0])
if not completed:
raise ReleaseRunConflict(
"Release run retention limit is reached; complete or explicitly "
"remove retained runs before creating another run."
)
return completed[0][1]
def _unlink_retained_record(self, path: Path) -> None:
try:
metadata = path.lstat()
if (
not stat.S_ISREG(metadata.st_mode)
or metadata.st_nlink != 1
or (hasattr(os, "geteuid") and metadata.st_uid != os.geteuid())
or metadata.st_mode & 0o077
):
raise ReleaseRunCorrupt(
"Completed release run could not be retired safely."
)
path.unlink()
_fsync_directory(self.root)
except ReleaseRunError:
raise
except OSError as exc:
raise ReleaseRunCorrupt(
"Completed release run could not be retired safely."
) from exc
def _unlink_new_record(self, path: Path) -> None:
try:
path.unlink()
_fsync_directory(self.root)
except OSError as exc:
raise ReleaseRunCorrupt(
"Release run creation could not be rolled back safely after a "
"retention failure. Inspect the private store before continuing."
) from exc
def _assert_workspace(self, record: dict[str, Any]) -> None:
if (
record["immutable"]["input"]["workspace_fingerprint"]
@@ -774,7 +1026,9 @@ def _validated_input(value: dict[str, Any]) -> dict[str, Any]:
raise ReleaseRunConflict("Release run input fields are invalid.")
channel = value.get("channel")
repo_versions = value.get("repo_versions")
if not isinstance(channel, str) or not CHANNEL_PATTERN.fullmatch(channel):
try:
channel = validate_release_channel(channel)
except CandidateArtifactError:
raise ReleaseRunConflict("Release channel is invalid.")
if (
not isinstance(repo_versions, dict)
@@ -810,7 +1064,7 @@ def _validated_plan(value: dict[str, Any]) -> dict[str, Any]:
if not isinstance(value, dict):
raise ReleaseRunConflict("Release plan snapshot must be an object.")
copied = deepcopy(value)
if set(copied) != {
base_fields = {
"generated_at",
"target_channel",
"status",
@@ -821,15 +1075,32 @@ def _validated_plan(value: dict[str, Any]) -> dict[str, Any]:
"gate_findings",
"recommended_action",
"source_preflight_ready",
}:
}
if set(copied) not in (base_fields, base_fields | {"runtime_binding"}):
raise ReleaseRunConflict("Release plan snapshot fields are invalid.")
if "runtime_binding" in copied:
runtime_binding = copied["runtime_binding"]
validated_runtime = _validated_result_receipt(runtime_binding)
if (
validated_runtime != runtime_binding
or runtime_binding.get("kind") != "repository_state"
or runtime_binding.get("repo") != "govoplan"
or runtime_binding.get("target_tag") != ""
or runtime_binding.get("tag_object") is not None
or runtime_binding.get("worktree_clean") is not True
):
raise ReleaseRunConflict(
"Release plan runtime binding is invalid."
)
if not _is_timestamp(copied.get("generated_at")):
raise ReleaseRunConflict("Release plan timestamp is invalid.")
if copied.get("status") not in PLAN_STATUSES:
raise ReleaseRunConflict("Release plan status is invalid.")
if not isinstance(
copied.get("target_channel"), str
) or not CHANNEL_PATTERN.fullmatch(copied["target_channel"]):
try:
target_channel = validate_release_channel(copied.get("target_channel"))
except CandidateArtifactError:
raise ReleaseRunConflict("Release plan channel is invalid.")
if target_channel != copied["target_channel"]:
raise ReleaseRunConflict("Release plan channel is invalid.")
if not isinstance(copied.get("units"), list):
raise ReleaseRunConflict("Release plan units are invalid.")
@@ -855,7 +1126,7 @@ def _validated_plan(value: dict[str, Any]) -> dict[str, Any]:
raise ReleaseRunConflict("Release plan step collection is invalid.")
seen: set[str] = set()
for step in steps:
if not isinstance(step, dict) or set(step) != {
if not isinstance(step, dict) or set(step) not in ({
"id",
"title",
"detail",
@@ -864,7 +1135,17 @@ def _validated_plan(value: dict[str, Any]) -> dict[str, Any]:
"mutating",
"repo",
"status",
}:
}, {
"id",
"title",
"detail",
"command",
"cwd",
"mutating",
"repo",
"status",
"source_binding",
}):
raise ReleaseRunConflict("Release plan step is invalid.")
step_id = step.get("id")
_validate_step_id(step_id)
@@ -883,6 +1164,21 @@ def _validated_plan(value: dict[str, Any]) -> dict[str, Any]:
not isinstance(step[field], str) or len(step[field]) > 8_192
):
raise ReleaseRunConflict("Release plan step metadata is invalid.")
if "source_binding" in step:
binding = step["source_binding"]
if binding is not None:
validated_binding = _validated_result_receipt(binding)
if (
validated_binding != binding
or binding.get("kind") != "repository_state"
or (
step.get("repo") is not None
and binding.get("repo") != step.get("repo")
)
):
raise ReleaseRunConflict(
"Release plan source binding is invalid."
)
encoded = _canonical_json(copied)
if len(encoded) > MAX_RECORD_BYTES:
raise ReleaseRunConflict("Release plan snapshot exceeds its size limit.")
@@ -908,9 +1204,11 @@ def _validate_record(value: Any, *, expected_run_id: str | None = None) -> None:
if (
value.get("schema") != SCHEMA
or type(value.get("schema_version")) is not int
or value["schema_version"] != SCHEMA_VERSION
or value["schema_version"]
not in {SCHEMA_VERSION, *LEGACY_SCHEMA_VERSIONS}
):
raise ValueError
schema_version = value["schema_version"]
run_id = value.get("run_id")
if not isinstance(run_id, str) or not RUN_ID_PATTERN.fullmatch(run_id):
raise ValueError
@@ -970,18 +1268,20 @@ def _validate_record(value: Any, *, expected_run_id: str | None = None) -> None:
if not isinstance(steps, list) or len(steps) != len(plan_steps):
raise ValueError
for step, plan_step in zip(steps, plan_steps, strict=True):
expected_step_fields = {
"id",
"state",
"attempt_count",
"attempt_fingerprint",
"started_at",
"finished_at",
"result_code",
}
if schema_version >= 4:
expected_step_fields.add("result_receipt")
if (
not isinstance(step, dict)
or set(step)
!= {
"id",
"state",
"attempt_count",
"attempt_fingerprint",
"started_at",
"finished_at",
"result_code",
}
or set(step) != expected_step_fields
or step.get("id") != plan_step["id"]
):
raise ValueError
@@ -1000,6 +1300,9 @@ def _validate_record(value: Any, *, expected_run_id: str | None = None) -> None:
result_code = step.get("result_code")
if result_code is not None:
_safe_code(result_code)
receipt = _validated_result_receipt(step.get("result_receipt"))
if receipt != step.get("result_receipt"):
raise ValueError
_validate_step_state_fields(step)
for timestamp in (step.get("started_at"), step.get("finished_at")):
if timestamp is not None and not (
@@ -1107,7 +1410,8 @@ def _validate_record(value: Any, *, expected_run_id: str | None = None) -> None:
_safe_code(attempt_result)
if (
attempt["outcome"] == "interrupted"
and attempt_result != "process_interrupted"
and attempt_result
not in {"process_interrupted", "executor_outcome_unknown"}
):
raise ValueError
if len({attempt["fingerprint"] for attempt in attempts}) != len(attempts):
@@ -1436,6 +1740,7 @@ def _project_finish(
"state": target_state,
"finished_at": PROJECTION_TIMESTAMP,
"result_code": PROJECTION_RESULT_CODE,
"result_receipt": _projection_receipt() if succeeded else None,
}
)
attempt["outcome"] = target_state
@@ -1451,6 +1756,22 @@ def _project_finish(
return projected
def _projection_receipt() -> dict[str, Any]:
return {
"kind": "catalog_publication",
"candidate_id": "candidate-" + "c" * 32,
"catalog_sha256": "c" * 64,
"keyring_sha256": "c" * 64,
"publication_commit_sha": "c" * 64,
"publication_tag_object_sha": "c" * 64,
"publication_tag_commit_sha": "c" * 64,
"branch": "b" * 255,
"tag_name": "t" * 160,
"remote": "origin",
"remote_sha256": "c" * 64,
}
def _project_resume(record: dict[str, Any]) -> dict[str, Any]:
projected = deepcopy(record)
running_steps = [
@@ -1501,6 +1822,7 @@ def _project_retry(record: dict[str, Any], step_id: str) -> dict[str, Any]:
"started_at": None,
"finished_at": None,
"result_code": None,
"result_receipt": None,
}
)
_remember_request(
@@ -1536,6 +1858,7 @@ def _project_reconcile(
"started_at": None,
"finished_at": None,
"result_code": None,
"result_receipt": None,
}
)
elif outcome == "effect_succeeded":
@@ -1544,6 +1867,7 @@ def _project_reconcile(
"state": "succeeded",
"finished_at": PROJECTION_TIMESTAMP,
"result_code": "reconciled_effect_succeeded",
"result_receipt": _projection_receipt(),
}
)
_remember_request(
@@ -1736,6 +2060,63 @@ def _creation_run_id(request_fingerprint: str) -> str:
return f"rr-request-{request_fingerprint}"
def _encode_list_cursor(
key: tuple[int, str, str, str], *, workspace_fingerprint: str
) -> str:
payload = {
"v": 1,
"workspace": workspace_fingerprint,
"key": list(key),
}
encoded = base64.urlsafe_b64encode(_canonical_json(payload)).decode("ascii")
return encoded.rstrip("=")
def _decode_list_cursor(
cursor: str | None, *, expected_workspace_fingerprint: str
) -> tuple[int, str, str, str] | None:
if cursor is None or cursor == "":
return None
if not isinstance(cursor, str) or len(cursor) > MAX_CURSOR_BYTES:
raise ReleaseRunConflict("Release run list cursor is invalid.")
try:
padding = "=" * (-len(cursor) % 4)
encoded = base64.b64decode(
cursor + padding,
altchars=b"-_",
validate=True,
)
payload = json.loads(encoded.decode("utf-8"))
if not isinstance(payload, dict):
raise ValueError
key = payload.get("key")
if (
set(payload) != {"v", "workspace", "key"}
or payload.get("v") != 1
or payload.get("workspace") != expected_workspace_fingerprint
or not isinstance(key, list)
or len(key) != 4
or type(key[0]) is not int
or key[0] not in {0, 1}
or not all(isinstance(item, str) for item in key[1:])
or not RUN_ID_PATTERN.fullmatch(key[3])
):
raise ValueError
if key[0] == 1:
if not _is_timestamp(key[1]) or not _is_timestamp(key[2]):
raise ValueError
elif key[1] or key[2]:
raise ValueError
except (
binascii.Error,
UnicodeDecodeError,
ValueError,
json.JSONDecodeError,
) as exc:
raise ReleaseRunConflict("Release run list cursor is invalid.") from exc
return key[0], key[1], key[2], key[3]
def _validate_create_replay(
record: dict[str, Any],
*,
@@ -1773,6 +2154,118 @@ def _safe_code(value: str) -> str:
return value
def _validated_result_receipt(
value: dict[str, Any] | None,
) -> dict[str, Any] | None:
if value is None:
return None
if not isinstance(value, dict) or value.get("kind") not in RECEIPT_KINDS:
raise ReleaseRunConflict("Release result receipt is invalid.")
if value["kind"] == "catalog_candidate":
if set(value) != {"kind", "candidate_id", "catalog_sha256"}:
raise ReleaseRunConflict("Release result receipt is invalid.")
candidate_id = value.get("candidate_id")
digest = value.get("catalog_sha256")
if (
not isinstance(candidate_id, str)
or not RECEIPT_ID_PATTERN.fullmatch(candidate_id)
or not _is_sha256(digest)
):
raise ReleaseRunConflict("Release result receipt identity is invalid.")
return dict(value)
if value["kind"] == "catalog_publication":
if set(value) != {
"kind",
"candidate_id",
"catalog_sha256",
"keyring_sha256",
"publication_commit_sha",
"publication_tag_object_sha",
"publication_tag_commit_sha",
"branch",
"tag_name",
"remote",
"remote_sha256",
}:
raise ReleaseRunConflict("Catalog publication receipt is invalid.")
candidate_id = value.get("candidate_id")
commit = value.get("publication_commit_sha")
tagged_commit = value.get("publication_tag_commit_sha")
if (
not isinstance(candidate_id, str)
or not RECEIPT_ID_PATTERN.fullmatch(candidate_id)
or not _is_sha256(value.get("catalog_sha256"))
or not _is_sha256(value.get("keyring_sha256"))
or not isinstance(commit, str)
or not GIT_OBJECT_PATTERN.fullmatch(commit)
or not isinstance(value.get("publication_tag_object_sha"), str)
or not GIT_OBJECT_PATTERN.fullmatch(
value["publication_tag_object_sha"]
)
or not isinstance(tagged_commit, str)
or not GIT_OBJECT_PATTERN.fullmatch(tagged_commit)
or tagged_commit != commit
or not isinstance(value.get("branch"), str)
or not value["branch"]
or len(value["branch"]) > 255
or not isinstance(value.get("tag_name"), str)
or not value["tag_name"]
or len(value["tag_name"]) > 160
or value.get("remote") != "origin"
or not _is_sha256(value.get("remote_sha256"))
):
raise ReleaseRunConflict(
"Catalog publication receipt identity is invalid."
)
return dict(value)
if set(value) != {
"kind",
"repo",
"head",
"branch",
"remote",
"remote_sha256",
"worktree_clean",
"target_tag",
"tag_object",
}:
raise ReleaseRunConflict("Repository result receipt is invalid.")
if (
not isinstance(value.get("repo"), str)
or not REPOSITORY_PATTERN.fullmatch(value["repo"])
or not isinstance(value.get("head"), str)
or not GIT_OBJECT_PATTERN.fullmatch(value["head"])
or not isinstance(value.get("branch"), str)
or not value["branch"]
or len(value["branch"]) > 255
or value.get("remote") != "origin"
or not _is_sha256(value.get("remote_sha256"))
or type(value.get("worktree_clean")) is not bool
or not isinstance(value.get("target_tag"), str)
or len(value["target_tag"]) > 160
or (
value.get("tag_object") is not None
and (
not isinstance(value["tag_object"], str)
or not GIT_OBJECT_PATTERN.fullmatch(value["tag_object"])
)
)
):
raise ReleaseRunConflict("Repository result receipt identity is invalid.")
return dict(value)
def _upgrade_record(record: dict[str, Any]) -> dict[str, Any]:
upgraded = deepcopy(record)
if upgraded.get("schema_version") == 3:
for step in upgraded["state"]["steps"]:
step["result_receipt"] = None
upgraded["schema_version"] = SCHEMA_VERSION
upgraded["record_digest"] = _record_digest(upgraded)
_validate_record(upgraded)
return upgraded
def _is_sha256(value: Any) -> bool:
return isinstance(value, str) and bool(re.fullmatch(r"[0-9a-f]{64}", value))
@@ -1796,6 +2289,7 @@ def _validate_step_state_fields(step: dict[str, Any]) -> None:
started_at = step["started_at"]
finished_at = step["finished_at"]
result_code = step["result_code"]
result_receipt = step.get("result_receipt")
if started_at is not None and not _is_timestamp(started_at):
raise ValueError
if finished_at is not None and not _is_timestamp(finished_at):
@@ -1805,19 +2299,34 @@ def _validate_step_state_fields(step: dict[str, Any]) -> None:
if state == "pending":
if any(
value is not None
for value in (fingerprint, started_at, finished_at, result_code)
for value in (
fingerprint,
started_at,
finished_at,
result_code,
result_receipt,
)
):
raise ValueError
return
if attempt_count < 1 or fingerprint is None or started_at is None:
raise ValueError
if state == "running":
if finished_at is not None or result_code is not None:
if (
finished_at is not None
or result_code is not None
or result_receipt is not None
):
raise ValueError
return
if finished_at is None or result_code is None:
raise ValueError
if state == "interrupted" and result_code != "process_interrupted":
if state != "succeeded" and result_receipt is not None:
raise ValueError
if state == "interrupted" and result_code not in {
"process_interrupted",
"executor_outcome_unknown",
}:
raise ValueError
@@ -1843,7 +2352,8 @@ def _validate_event_fields(event: dict[str, Any]) -> None:
raise ValueError
if (
event_type == "step_interrupted"
and event["result_code"] != "process_interrupted"
and event["result_code"]
not in {"process_interrupted", "executor_outcome_unknown"}
):
raise ValueError

View File

@@ -5,7 +5,7 @@ from __future__ import annotations
from pathlib import Path
import subprocess
from .git_state import collect_repository_snapshot
from .git_state import collect_repository_snapshot, scoped_git_command
from .repository_push import command_text, compact_output
from .workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root
@@ -131,7 +131,12 @@ def resolve_message(*, repo: str, version: str | None, message: str | None) -> s
def run(command: tuple[str, ...], *, cwd: Path) -> subprocess.CompletedProcess[str]:
effective_command = (
scoped_git_command(cwd, *command[1:])
if command and Path(command[0]).name == "git"
else command
)
try:
return subprocess.run(command, cwd=cwd, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=120)
return subprocess.run(effective_command, cwd=cwd, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=120)
except subprocess.TimeoutExpired as exc:
return subprocess.CompletedProcess(command, 124, exc.stdout or "", exc.stderr or "git command timed out")
return subprocess.CompletedProcess(effective_command, 124, exc.stdout or "", exc.stderr or "git command timed out")

View File

@@ -6,7 +6,7 @@ from pathlib import Path
import shlex
import subprocess
from .git_state import collect_repository_snapshot
from .git_state import collect_repository_snapshot, scoped_git_command
from .workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root
@@ -138,10 +138,15 @@ def command_text(command: tuple[str, ...]) -> str:
def run(command: tuple[str, ...], *, cwd: Path) -> subprocess.CompletedProcess[str]:
effective_command = (
scoped_git_command(cwd, *command[1:])
if command and Path(command[0]).name == "git"
else command
)
try:
return subprocess.run(command, cwd=cwd, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=120)
return subprocess.run(effective_command, cwd=cwd, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=120)
except subprocess.TimeoutExpired as exc:
return subprocess.CompletedProcess(command, 124, exc.stdout or "", exc.stderr or "git push timed out")
return subprocess.CompletedProcess(effective_command, 124, exc.stdout or "", exc.stderr or "git push timed out")
def compact_output(value: str) -> str:

View File

@@ -5,7 +5,7 @@ from __future__ import annotations
from pathlib import Path
import subprocess
from .git_state import collect_repository_snapshot
from .git_state import collect_repository_snapshot, scoped_git_command
from .repository_push import command_text, compact_output
from .workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root
@@ -93,7 +93,12 @@ def sync_repositories(
def run(command: tuple[str, ...], *, cwd: Path) -> subprocess.CompletedProcess[str]:
effective_command = (
scoped_git_command(cwd, *command[1:])
if command and Path(command[0]).name == "git"
else command
)
try:
return subprocess.run(command, cwd=cwd, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=120)
return subprocess.run(effective_command, cwd=cwd, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=120)
except subprocess.TimeoutExpired as exc:
return subprocess.CompletedProcess(command, 124, exc.stdout or "", exc.stderr or "git fetch timed out")
return subprocess.CompletedProcess(effective_command, 124, exc.stdout or "", exc.stderr or "git fetch timed out")

View File

@@ -9,7 +9,12 @@ import re
import subprocess
import sys
from .git_state import collect_repository_snapshot, git_text
from .git_state import (
collect_repository_snapshot,
git_text,
sanitized_git_environment,
scoped_git_command,
)
from .model import RepositorySnapshot
from .repository_push import command_text, compact_output
from .version_alignment import repository_version_issues, selected_release_webui_bundle_issues
@@ -397,7 +402,16 @@ def manifest_shape_gate_issue(workspace: Path) -> str | None:
"--workspace-root",
str(workspace),
)
result = run(command, cwd=META_ROOT, env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"})
result = run(
command,
cwd=META_ROOT,
env={
key: value
for key, value in os.environ.items()
if not key.startswith("GIT_")
}
| {"PYTHONDONTWRITEBYTECODE": "1"},
)
if result.returncode == 0:
return None
detail = compact_output(result.stderr) or compact_output(result.stdout)
@@ -463,16 +477,21 @@ def run(
timeout: int = 120,
env: dict[str, str] | None = None,
) -> subprocess.CompletedProcess[str]:
effective_command = command
effective_environment = env
if command and Path(command[0]).name == "git":
effective_command = scoped_git_command(cwd, *command[1:])
effective_environment = sanitized_git_environment(env)
try:
return subprocess.run(
command,
effective_command,
cwd=cwd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout,
env=env,
env=effective_environment,
)
except subprocess.TimeoutExpired as exc:
return subprocess.CompletedProcess(command, 124, exc.stdout or "", exc.stderr or "Git command timed out")

View File

@@ -3,27 +3,41 @@
from __future__ import annotations
import base64
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
import json
import os
from pathlib import Path
import re
import stat
import tempfile
from typing import Any
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from cryptography.hazmat.primitives.asymmetric.ed25519 import (
Ed25519PrivateKey,
Ed25519PublicKey,
)
from govoplan_core.core.module_package_catalog import validate_module_package_catalog
from .catalog import DEFAULT_PUBLIC_BASE_URL, canonical_hash, fetch_json
from .artifact_identity import inspect_python_wheel
from .candidate_artifact import (
harden_private_candidate_tree,
validate_release_channel,
)
from .catalog_entry_synthesis import (
synthesize_repository_catalog_entries,
validate_initial_entry_closure,
)
from .contracts import collect_contracts
from .git_state import collect_repository_snapshot
from .git_state import collect_repository_snapshot, git_text
from .model import CatalogEntryChange, SelectiveCatalogCandidate
from .module_directory import write_module_directory
from .source_provenance import (
SourceTagProvenanceIssue,
catalog_source_selection,
selected_source_provenance,
source_tag_provenance_issues,
@@ -32,9 +46,24 @@ from .version_alignment import (
selected_release_webui_bundle_issues,
selected_repository_version_issues,
)
from .workspace import load_repository_specs, resolve_workspace_root, website_root
from .workspace import (
load_repository_specs,
resolve_repo_path,
resolve_workspace_root,
website_root,
)
GITEA_BASE = "git+ssh://git@git.add-ideas.de/add-ideas"
GITEA_BASE = "git+ssh://git@git.add-ideas.de/GovOPlaN"
MAX_BASE_JSON_BYTES = 16 * 1024 * 1024
_SIGNING_KEY_ID = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,159}$")
@dataclass(frozen=True, slots=True)
class AuthenticatedCatalogBase:
catalog_source: Path | str
keyring_source: Path | str
catalog: dict[str, Any]
keyring: dict[str, Any]
def build_selective_catalog_candidate(
@@ -44,6 +73,7 @@ def build_selective_catalog_candidate(
workspace_root: Path | str | None = None,
output_dir: Path | str | None = None,
base_catalog: Path | str | None = None,
base_keyring: Path | str | None = None,
signing_keys: tuple[str, ...] = (),
public_base_url: str = DEFAULT_PUBLIC_BASE_URL,
repository_base: str = GITEA_BASE,
@@ -52,29 +82,47 @@ def build_selective_catalog_candidate(
sequence: int | None = None,
check_public: bool = True,
write_summary: bool = True,
python_artifacts: dict[str, Path | str] | None = None,
frozen_source_provenance: dict[str, dict[str, str]] | None = None,
) -> SelectiveCatalogCandidate:
channel = validate_release_channel(channel)
if not repo_versions:
raise ValueError("At least one repo version must be provided.")
parsed_keys = tuple(parse_signing_key(value) for value in signing_keys)
if not parsed_keys:
raise ValueError("At least one signing key is required.")
signer_public_keys = configured_signer_public_keys(parsed_keys)
workspace = resolve_workspace_root(workspace_root)
enforce_selected_version_alignment(repo_versions=repo_versions, workspace=workspace)
enforce_selected_source_provenance(
repo_versions=repo_versions,
workspace=workspace,
remote=source_remote,
)
selected_provenance = selected_source_provenance(
repo_versions=repo_versions,
workspace=workspace,
)
if frozen_source_provenance is None:
enforce_selected_source_provenance(
repo_versions=repo_versions,
workspace=workspace,
remote=source_remote,
)
selected_provenance = selected_source_provenance(
repo_versions=repo_versions,
workspace=workspace,
)
else:
selected_provenance = enforce_frozen_selected_source_provenance(
repo_versions=repo_versions,
expected_provenance=frozen_source_provenance,
workspace=workspace,
remote=source_remote,
)
web_root = website_root(workspace)
source_catalog = resolve_base_catalog(base_catalog=base_catalog, web_root=web_root, channel=channel)
payload = read_catalog(source_catalog)
if not isinstance(payload, dict):
raise ValueError("Selective catalog updates require object-style catalogs.")
authenticated_base = load_authenticated_catalog_base(
base_catalog=base_catalog,
base_keyring=base_keyring,
web_root=web_root,
channel=channel,
public_base_url=public_base_url,
signer_public_keys=signer_public_keys,
)
source_catalog = authenticated_base.catalog_source
payload = authenticated_base.catalog
generated_at = datetime.now(tz=UTC)
resolved_sequence = sequence or next_sequence(payload, generated_at=generated_at)
@@ -99,7 +147,10 @@ def build_selective_catalog_candidate(
]
candidate["release"] = release
repo_contracts = contracts_by_repo(workspace)
repo_contracts = contracts_by_repo(
workspace,
selected_repositories=set(repo_versions),
)
changes = apply_repo_updates(
candidate,
repo_versions=repo_versions,
@@ -107,19 +158,27 @@ def build_selective_catalog_candidate(
repository_base=repository_base.rstrip("/"),
workspace=workspace,
)
changes.extend(
apply_python_artifact_identities(
candidate,
repo_versions=repo_versions,
python_artifacts=python_artifacts or {},
)
)
enforce_complete_catalog_source_provenance(
payload=candidate,
workspace=workspace,
remote=source_remote,
require_selected_heads=frozen_source_provenance is None,
)
output_root = resolve_output_dir(output_dir=output_dir, channel=channel, generated_at=generated_at)
catalog_path = output_root / "channels" / f"{channel}.json"
keyring_path = output_root / "keyring.json"
summary_path = output_root / "summary.json" if write_summary else None
keyring_payload = keyring_payload_for_candidate(
existing_keyring=web_root / "public" / "catalogs" / "v1" / "keyring.json",
signing_keys=parsed_keys,
keyring_payload = candidate_keyring_from_authenticated_base(
authenticated_base.keyring,
signer_public_keys=signer_public_keys,
generated_at=generated_at,
)
release["keyring_sha256"] = canonical_hash(keyring_payload)
@@ -128,10 +187,15 @@ def build_selective_catalog_candidate(
candidate.pop("signatures", None)
candidate["signatures"] = [signature(candidate, key_id=key_id, private_key=private_key) for key_id, private_key in parsed_keys]
catalog_path.parent.mkdir(parents=True, exist_ok=True)
output_root.mkdir(mode=0o700, parents=True, exist_ok=True)
harden_private_candidate_tree(output_root)
catalog_path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
os.chmod(catalog_path.parent, 0o700, follow_symlinks=False)
catalog_path.write_text(json.dumps(candidate, indent=2, sort_keys=True) + "\n", encoding="utf-8")
os.chmod(catalog_path, 0o600, follow_symlinks=False)
keyring_path.write_text(json.dumps(keyring_payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
os.chmod(keyring_path, 0o600, follow_symlinks=False)
module_directory_files = write_module_directory(
catalog_payload=candidate,
keyring_payload=keyring_payload,
@@ -196,6 +260,7 @@ def build_selective_catalog_candidate(
)
if summary_path is not None:
summary_path.write_text(json.dumps(dataclass_payload(result), indent=2, sort_keys=True) + "\n", encoding="utf-8")
harden_private_candidate_tree(output_root)
return result
@@ -237,11 +302,107 @@ def enforce_selected_source_provenance(
)
def enforce_frozen_selected_source_provenance(
*,
repo_versions: dict[str, str],
expected_provenance: dict[str, dict[str, str]],
workspace: Path,
remote: str = "origin",
) -> dict[str, dict[str, str]]:
"""Verify detached, clean source checkouts against durable object receipts.
Normal interactive candidate generation still requires a named, current
source branch. The durable executor instead operates on isolated clones of
exact annotated origin tags, so it supplies the commit and tag object that
must be present at each detached HEAD.
"""
if set(expected_provenance) != set(repo_versions):
raise ValueError(
"Frozen source provenance must cover exactly the selected repositories."
)
expected_commits: dict[str, str] = {}
expected_tag_objects: dict[str, str] = {}
for repo in sorted(repo_versions):
identity = expected_provenance.get(repo)
commit = identity.get("commit_sha") if isinstance(identity, dict) else None
tag_object = (
identity.get("tag_object_sha") if isinstance(identity, dict) else None
)
if (
not isinstance(identity, dict)
or set(identity) != {"commit_sha", "tag_object_sha"}
or not isinstance(commit, str)
or re.fullmatch(r"[0-9a-f]{40}|[0-9a-f]{64}", commit) is None
or not isinstance(tag_object, str)
or re.fullmatch(r"[0-9a-f]{40}|[0-9a-f]{64}", tag_object) is None
):
raise ValueError(
f"Frozen source provenance is malformed for {repo}."
)
expected_commits[repo] = commit
expected_tag_objects[repo] = tag_object
failures = list(
source_tag_provenance_issues(
repo_versions=repo_versions,
workspace=workspace,
remote=remote,
expected_commits=expected_commits,
expected_tag_objects=expected_tag_objects,
)
)
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
for repo, version in sorted(repo_versions.items()):
spec = specs.get(repo)
if spec is None:
continue
path = resolve_repo_path(spec, workspace)
snapshot = collect_repository_snapshot(
spec,
workspace_root=workspace,
target_tag=f"v{version.removeprefix('v')}",
online=False,
)
if snapshot.dirty_entries:
failures.append(
SourceTagProvenanceIssue(
repo,
f"v{version.removeprefix('v')}",
"frozen source worktree is not clean",
)
)
head = git_text(path, "rev-parse", "--verify", "HEAD")
if head != expected_commits[repo]:
failures.append(
SourceTagProvenanceIssue(
repo,
f"v{version.removeprefix('v')}",
"frozen source HEAD does not match its durable commit receipt",
)
)
if failures:
raise ValueError(
"Frozen source provenance gate failed: "
+ "; ".join(issue.describe() for issue in failures)
)
observed = selected_source_provenance(
repo_versions=repo_versions,
workspace=workspace,
)
if observed != expected_provenance:
raise ValueError(
"Frozen source object identities changed during candidate synthesis."
)
return observed
def enforce_complete_catalog_source_provenance(
*,
payload: object,
workspace: Path,
remote: str = "origin",
require_selected_heads: bool = True,
) -> None:
selection = catalog_source_selection(payload)
failures = [*selection.issues]
@@ -250,7 +411,9 @@ def enforce_complete_catalog_source_provenance(
repo_versions=selection.all_versions,
workspace=workspace,
remote=remote,
require_head_repos=selection.selected_versions,
require_head_repos=(
selection.selected_versions if require_selected_heads else ()
),
expected_commits=selection.selected_commits,
expected_tag_objects=selection.selected_tag_objects,
)
@@ -262,23 +425,170 @@ def enforce_complete_catalog_source_provenance(
)
def resolve_base_catalog(*, base_catalog: Path | str | None, web_root: Path, channel: str) -> Path | str:
if base_catalog is not None:
value = str(base_catalog)
return value if value.startswith(("http://", "https://")) else Path(value).expanduser()
local = web_root / "public" / "catalogs" / "v1" / "channels" / f"{channel}.json"
if local.exists():
return local
return f"{DEFAULT_PUBLIC_BASE_URL}/catalogs/v1/channels/{channel}.json"
def load_authenticated_catalog_base(
*,
base_catalog: Path | str | None,
base_keyring: Path | str | None,
web_root: Path,
channel: str,
public_base_url: str,
signer_public_keys: dict[str, str],
) -> AuthenticatedCatalogBase:
"""Read one base pair once and authenticate it before any synthesis."""
catalog_source, keyring_source = resolve_base_sources(
base_catalog=base_catalog,
base_keyring=base_keyring,
web_root=web_root,
channel=channel,
public_base_url=public_base_url,
)
catalog = read_bounded_json_source(catalog_source, label="base catalog")
keyring = read_bounded_json_source(keyring_source, label="base keyring")
if not isinstance(catalog, dict) or not isinstance(keyring, dict):
raise ValueError("Selective catalog base and keyring must be JSON objects.")
base_trusted_keys = authenticate_base_keyring(keyring)
release = catalog.get("release")
pinned_keyring = (
release.get("keyring_sha256") if isinstance(release, dict) else None
)
if pinned_keyring != canonical_hash(keyring):
raise ValueError(
"Signed base catalog does not pin the exact supplied base keyring."
)
authenticate_base_catalog_signatures(
catalog,
base_trusted_keys=base_trusted_keys,
configured_signers=signer_public_keys,
)
validation = validate_catalog_object(
catalog,
approved_channel=channel,
signer_public_keys={
key_id: public_key
for key_id, public_key in signer_public_keys.items()
if base_trusted_keys.get(key_id) == public_key
},
)
if validation.get("valid") is not True:
raise ValueError(
"Authenticated base catalog failed catalog validation: "
+ str(validation.get("error") or "unknown validation error")
)
return AuthenticatedCatalogBase(
catalog_source=catalog_source,
keyring_source=keyring_source,
catalog=catalog,
keyring=keyring,
)
def read_catalog(source: Path | str) -> object:
def resolve_base_sources(
*,
base_catalog: Path | str | None,
base_keyring: Path | str | None,
web_root: Path,
channel: str,
public_base_url: str,
) -> tuple[Path | str, Path | str]:
if (base_catalog is None) != (base_keyring is None):
raise ValueError("--base-catalog and --base-keyring must be supplied together.")
if base_catalog is not None and base_keyring is not None:
return _json_source(base_catalog), _json_source(base_keyring)
local_root = web_root / "public" / "catalogs" / "v1"
local_catalog = local_root / "channels" / f"{channel}.json"
local_keyring = local_root / "keyring.json"
if local_catalog.exists() and local_keyring.exists():
return local_catalog, local_keyring
public_root = public_base_url.rstrip("/")
return (
f"{public_root}/catalogs/v1/channels/{channel}.json",
f"{public_root}/catalogs/v1/keyring.json",
)
def _json_source(value: Path | str) -> Path | str:
text = str(value)
return text if text.startswith(("http://", "https://")) else Path(text).expanduser()
def read_bounded_json_source(source: Path | str, *, label: str) -> object:
if isinstance(source, str) and source.startswith(("http://", "https://")):
payload = fetch_json(source)
if not payload["ok"]:
raise ValueError(f"Could not fetch catalog {source}: {payload['error']}")
raise ValueError(f"Could not fetch {label}: {payload['error']}")
return payload["payload"]
return json.loads(Path(source).read_text(encoding="utf-8"))
encoded = _read_regular_file_without_symlinks(Path(source), label=label)
try:
return json.loads(encoded.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ValueError(f"{label.capitalize()} is not valid UTF-8 JSON.") from exc
def _read_regular_file_without_symlinks(
path: Path, *, label: str, require_private_owner: bool = False
) -> bytes:
absolute = path.absolute()
parts = absolute.parts[1:]
if not parts:
raise ValueError(f"{label.capitalize()} must name a regular file.")
directory_flags = (
os.O_RDONLY
| getattr(os, "O_DIRECTORY", 0)
| getattr(os, "O_NOFOLLOW", 0)
)
descriptor = os.open(absolute.anchor or "/", directory_flags)
try:
for part in parts[:-1]:
child = os.open(part, directory_flags, dir_fd=descriptor)
os.close(descriptor)
descriptor = child
file_descriptor = os.open(
parts[-1],
os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0),
dir_fd=descriptor,
)
except OSError as exc:
os.close(descriptor)
raise ValueError(f"{label.capitalize()} cannot be opened safely.") from exc
os.close(descriptor)
try:
initial = os.fstat(file_descriptor)
if not stat.S_ISREG(initial.st_mode) or initial.st_size > MAX_BASE_JSON_BYTES:
raise ValueError(
f"{label.capitalize()} must be a bounded regular file."
)
if require_private_owner and (
initial.st_uid != os.geteuid() or stat.S_IMODE(initial.st_mode) & 0o077
):
raise ValueError(
f"{label.capitalize()} must be owned by the current operator "
"and inaccessible to other users."
)
chunks: list[bytes] = []
remaining = initial.st_size
while remaining:
chunk = os.read(file_descriptor, min(1024 * 1024, remaining))
if not chunk:
break
chunks.append(chunk)
remaining -= len(chunk)
final = os.fstat(file_descriptor)
if remaining or _stat_fingerprint(initial) != _stat_fingerprint(final):
raise ValueError(f"{label.capitalize()} changed while it was read.")
return b"".join(chunks)
finally:
os.close(file_descriptor)
def _stat_fingerprint(value: os.stat_result) -> tuple[int, int, int, int, int]:
return (
value.st_dev,
value.st_ino,
value.st_size,
value.st_mtime_ns,
value.st_ctime_ns,
)
def next_sequence(payload: dict[str, Any], *, generated_at: datetime) -> int:
@@ -290,8 +600,14 @@ def next_sequence(payload: dict[str, Any], *, generated_at: datetime) -> int:
return max(current + 1, timestamp_sequence)
def contracts_by_repo(workspace: Path) -> dict[str, Any]:
specs = load_repository_specs(include_website=False)
def contracts_by_repo(
workspace: Path, *, selected_repositories: set[str] | None = None
) -> dict[str, Any]:
specs = tuple(
spec
for spec in load_repository_specs(include_website=False)
if selected_repositories is None or spec.name in selected_repositories
)
snapshots = tuple(
collect_repository_snapshot(spec, workspace_root=workspace, target_tag=None, online=False)
for spec in specs
@@ -412,6 +728,95 @@ def module_entry_repo(entry: dict[str, Any]) -> str | None:
return str(package).split("[", 1)[0] if isinstance(package, str) and package.startswith("govoplan-") else None
def apply_python_artifact_identities(
payload: dict[str, Any],
*,
repo_versions: dict[str, str],
python_artifacts: dict[str, Path | str],
) -> list[CatalogEntryChange]:
"""Replace selected release identities with hashes computed from built wheels.
A version update without a corresponding built artifact deliberately removes
any stale identity for that package. Callers can still prepare a source-only
candidate, but it cannot later establish catalog-anchored installed origin.
"""
unknown = sorted(set(python_artifacts) - set(repo_versions))
if unknown:
raise ValueError(
"Built Python artifacts were supplied for unselected repositories: "
+ ", ".join(unknown)
)
package_by_repo: dict[str, str] = {}
core = payload.get("core_release")
if isinstance(core, dict):
package = core.get("python_package")
if isinstance(package, str):
package_by_repo["govoplan-core"] = package
modules = payload.get("modules")
if isinstance(modules, list):
for entry in modules:
if not isinstance(entry, dict):
continue
repo = module_entry_repo(entry)
package = entry.get("python_package")
if repo in repo_versions and isinstance(package, str):
package_by_repo[repo] = package
unmapped_artifacts = sorted(set(python_artifacts) - set(package_by_repo))
if unmapped_artifacts:
raise ValueError(
"Built artifacts have no catalog Python package mapping: "
+ ", ".join(unmapped_artifacts)
)
release = payload.get("release")
if not isinstance(release, dict):
raise ValueError("Catalog payload has no release object.")
existing = release.get("artifacts", [])
if not isinstance(existing, list) or any(not isinstance(item, dict) for item in existing):
raise ValueError("Catalog release.artifacts must be a list of objects.")
selected_packages = {
package_by_repo[repo] for repo in repo_versions if repo in package_by_repo
}
retained = [
item
for item in existing
if str(item.get("package_name") or "") not in selected_packages
]
changes: list[CatalogEntryChange] = []
for repo, artifact_path in sorted(python_artifacts.items()):
identity = inspect_python_wheel(artifact_path)
expected_package = package_by_repo[repo]
expected_version = repo_versions[repo].removeprefix("v")
if identity.package_name != expected_package or identity.package_version != expected_version:
raise ValueError(
f"Built artifact for {repo} identifies {identity.package_name} "
f"{identity.package_version}, expected {expected_package} {expected_version}."
)
retained.append(identity.catalog_payload())
changes.append(
CatalogEntryChange(
repo=repo,
module_id=None,
field="release.artifact",
before=None,
after=identity.archive_sha256,
)
)
retained.sort(
key=lambda item: (
str(item.get("package_name") or ""),
str(item.get("package_version") or ""),
str(item.get("archive_sha256") or ""),
)
)
if retained:
release["artifacts"] = retained
else:
release.pop("artifacts", None)
return changes
def update_field(target: dict[str, Any], *, repo: str, module_id: str | None, field: str, value: str) -> list[CatalogEntryChange]:
before = target.get(field)
before_text = before if isinstance(before, str) else None
@@ -441,11 +846,21 @@ def parse_signing_key(value: str) -> tuple[str, Ed25519PrivateKey]:
key_id, separator, path_text = value.partition("=")
if not separator or not key_id.strip() or not path_text.strip():
raise ValueError("--catalog-signing-key must use KEY_ID=/path/to/private.pem")
key_id = key_id.strip()
if _SIGNING_KEY_ID.fullmatch(key_id) is None:
raise ValueError("Catalog signing key ID is not a bounded opaque ID.")
path = Path(path_text).expanduser()
private_key = serialization.load_pem_private_key(path.read_bytes(), password=None)
private_key = serialization.load_pem_private_key(
_read_regular_file_without_symlinks(
path,
label="catalog signing key",
require_private_owner=True,
),
password=None,
)
if not isinstance(private_key, Ed25519PrivateKey):
raise ValueError(f"Catalog signing key must be an Ed25519 private key: {path}")
return key_id.strip(), private_key
return key_id, private_key
def signature(payload: dict[str, Any], *, key_id: str, private_key: Ed25519PrivateKey) -> dict[str, str]:
@@ -459,38 +874,193 @@ def signature(payload: dict[str, Any], *, key_id: str, private_key: Ed25519Priva
}
def keyring_payload_for_candidate(
*,
existing_keyring: Path,
def configured_signer_public_keys(
signing_keys: tuple[tuple[str, Ed25519PrivateKey], ...],
) -> dict[str, str]:
result: dict[str, str] = {}
seen_material: set[str] = set()
for key_id, private_key in signing_keys:
public_key = public_key_base64(private_key)
if key_id in result or public_key in seen_material:
raise ValueError("Catalog signing keys contain duplicate identity or material.")
result[key_id] = public_key
seen_material.add(public_key)
return result
def authenticate_base_keyring(payload: dict[str, Any]) -> dict[str, str]:
keys = payload.get("keys")
if not isinstance(keys, list) or not keys or len(keys) > 64:
raise ValueError("Base keyring has no bounded signer set.")
observed_ids: set[str] = set()
seen_material: set[bytes] = set()
trusted: dict[str, str] = {}
for item in keys:
if not isinstance(item, dict):
raise ValueError("Base keyring contains a malformed key.")
key_id = item.get("key_id")
public_text = item.get("public_key")
status = item.get("status", "active")
if (
not isinstance(key_id, str)
or _SIGNING_KEY_ID.fullmatch(key_id) is None
or not isinstance(public_text, str)
or status
not in {"active", "next", "retired", "revoked", "disabled"}
or key_id in observed_ids
):
raise ValueError("Base keyring contains a malformed or duplicate key.")
try:
public_bytes = base64.b64decode(public_text, validate=True)
Ed25519PublicKey.from_public_bytes(public_bytes)
except ValueError as exc:
raise ValueError("Base keyring contains invalid Ed25519 material.") from exc
if public_bytes in seen_material:
raise ValueError("Base keyring reuses signer public material.")
seen_material.add(public_bytes)
observed_ids.add(key_id)
if status in {"active", "next"}:
trusted[key_id] = base64.b64encode(public_bytes).decode("ascii")
if not trusted:
raise ValueError("Base keyring has no active trusted catalog signer.")
return trusted
def authenticate_base_catalog_signatures(
payload: dict[str, Any],
*,
base_trusted_keys: dict[str, str],
configured_signers: dict[str, str],
) -> None:
if payload.get("signature") is not None:
raise ValueError("Base catalog must use one unambiguous signatures envelope.")
signatures = payload.get("signatures")
if not isinstance(signatures, list) or not signatures or len(signatures) > 16:
raise ValueError("Base catalog has no bounded signatures envelope.")
signed_payload = dict(payload)
signed_payload.pop("signatures", None)
encoded = canonical_bytes(signed_payload)
seen: set[str] = set()
for item in signatures:
if not isinstance(item, dict) or item.get("algorithm") != "ed25519":
raise ValueError("Base catalog contains a malformed signature.")
key_id = item.get("key_id")
value = item.get("value")
if (
not isinstance(key_id, str)
or key_id in seen
or key_id not in base_trusted_keys
or not isinstance(value, str)
):
raise ValueError("Base catalog contains an untrusted or duplicate signature.")
try:
public_key = Ed25519PublicKey.from_public_bytes(
base64.b64decode(base_trusted_keys[key_id], validate=True)
)
public_key.verify(base64.b64decode(value, validate=True), encoded)
except (ValueError, InvalidSignature) as exc:
raise ValueError("Base catalog signature verification failed.") from exc
seen.add(key_id)
if not any(
key_id in seen
and configured_signers.get(key_id) == base_trusted_keys.get(key_id)
for key_id in configured_signers
):
raise ValueError(
"Base catalog needs a valid signature from a configured signer "
"already trusted by its pinned keyring."
)
def candidate_keyring_from_authenticated_base(
payload: dict[str, Any],
*,
signer_public_keys: dict[str, str],
generated_at: datetime,
) -> dict[str, Any]:
expected = {key_id: public_key_base64(private_key) for key_id, private_key in signing_keys}
if existing_keyring.exists():
payload = json.loads(existing_keyring.read_text(encoding="utf-8"))
keys = payload.get("keys") if isinstance(payload, dict) else None
if isinstance(keys, list):
existing = {
str(item.get("key_id")): str(item.get("public_key") or item.get("public_key_base64"))
"""Carry authenticated keys forward and add only configured signer material."""
candidate = json.loads(json.dumps(payload))
keys = candidate.get("keys")
if not isinstance(keys, list):
raise ValueError("Authenticated base keyring lost its keys list.")
existing: dict[str, str] = {}
existing_material: set[str] = set()
for item in keys:
if not isinstance(item, dict):
raise ValueError("Authenticated base keyring contains a malformed key.")
key_id = str(item.get("key_id") or "")
public_key = str(item.get("public_key") or "")
existing[key_id] = public_key
existing_material.add(public_key)
changed = False
for key_id, public_key in sorted(signer_public_keys.items()):
if key_id in existing:
if existing[key_id] != public_key:
raise ValueError(
"Configured signer key ID conflicts with authenticated base material."
)
matching = next(
item
for item in keys
if isinstance(item, dict) and item.get("key_id")
}
if all(existing.get(key_id) == public_key for key_id, public_key in expected.items()):
return payload
return {
"keyring_version": "1",
"purpose": "govoplan module package catalog signatures",
"generated_at": json_datetime(generated_at),
"keys": [
if isinstance(item, dict) and item.get("key_id") == key_id
)
if matching.get("status", "active") not in {"active", "next"}:
raise ValueError("Configured signer is disabled in the base keyring.")
continue
if public_key in existing_material:
raise ValueError(
"Configured signer material already has another authenticated key ID."
)
keys.append(
{
"key_id": key_id,
"status": "active",
"public_key": public_key_base64(private_key),
"not_before": generated_at.date().isoformat() + "T00:00:00Z",
"public_key": public_key,
"not_before": generated_at.astimezone(UTC)
.replace(microsecond=0)
.isoformat()
.replace("+00:00", "Z"),
}
for key_id, private_key in signing_keys
],
}
)
existing_material.add(public_key)
changed = True
if changed:
candidate["generated_at"] = json_datetime(generated_at)
return candidate
def validate_catalog_object(
payload: dict[str, Any],
*,
approved_channel: str,
signer_public_keys: dict[str, str],
) -> dict[str, object]:
descriptor, temporary_name = tempfile.mkstemp(
prefix="govoplan-base-catalog-", suffix=".json"
)
temporary = Path(temporary_name)
try:
os.fchmod(descriptor, 0o600)
encoded = (json.dumps(payload, sort_keys=True) + "\n").encode("utf-8")
if len(encoded) > MAX_BASE_JSON_BYTES:
raise ValueError("Base catalog exceeds its size limit.")
with os.fdopen(descriptor, "wb", closefd=True) as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
return validate_module_package_catalog(
temporary,
require_trusted=True,
approved_channels=(approved_channel,),
trusted_keys=signer_public_keys,
)
finally:
try:
os.close(descriptor)
except OSError:
pass
temporary.unlink(missing_ok=True)
def trusted_keys_from_keyring(payload: dict[str, Any]) -> dict[str, str]:
@@ -533,7 +1103,13 @@ def resolve_output_dir(*, output_dir: Path | str | None, channel: str, generated
if output_dir is not None:
return Path(output_dir).expanduser()
stamp = generated_at.strftime("%Y%m%d-%H%M%S")
return Path.cwd() / "runtime" / "release-candidates" / f"{channel}-{stamp}"
configured_state = os.getenv("XDG_STATE_HOME", "").strip()
state_root = (
Path(configured_state).expanduser()
if configured_state
else Path.home() / ".local" / "state"
)
return state_root / "govoplan" / "release-candidates" / f"{channel}-{stamp}"
def json_datetime(value: datetime) -> str:

View File

@@ -501,6 +501,24 @@ def dry_run_steps(
) -> tuple[ReleasePlanStep, ...]:
steps: list[ReleasePlanStep] = []
snapshots = {repo.spec.name: repo for repo in dashboard.repositories}
selected_names = {unit.repo for unit in units}
if "govoplan-core" in selected_names and len(selected_names) > 1:
steps.append(
ReleasePlanStep(
id="release:cross-repository-ordering",
title="Resolve module/Core release ordering",
detail=(
"A combined Core/module release needs a dependency-aware DAG: "
"local module tags, Core release-lock regeneration and commit, "
"Core tag, full alignment, then atomic pushes. The current linear "
"executor must not publish a module before that lock is committed."
),
command=None,
cwd=dashboard.meta_root,
mutating=True,
status="needs-executor",
)
)
for unit in units:
steps.append(
ReleasePlanStep(
@@ -618,6 +636,7 @@ def dry_run_steps(
f'--candidate-dir "$CANDIDATE_DIR" --channel {shlex.quote(channel)}'
),
cwd=dashboard.meta_root,
mutating=True,
status="planned",
)
)

View File

@@ -12,7 +12,12 @@ import tomllib
from typing import Iterable
from urllib.parse import urlsplit
from .git_state import collect_repository_snapshot, git_text
from .git_state import (
collect_repository_snapshot,
git_text,
sanitized_git_environment,
scoped_git_command,
)
from .repository_tag import RemoteTagResult, ref_commit, remote_tag_commit
from .version_alignment import repository_version_issues
from .workspace import load_repository_specs, resolve_repo_path
@@ -413,18 +418,20 @@ def _git_file(path: Path, tag: str, name: str) -> str | None:
def _git(path: Path, *args: str) -> subprocess.CompletedProcess[str]:
command = scoped_git_command(path, *args)
try:
return subprocess.run(
("git", *args),
command,
cwd=path,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=30,
env=sanitized_git_environment(),
)
except subprocess.TimeoutExpired as exc:
return subprocess.CompletedProcess(("git", *args), 124, exc.stdout or "", exc.stderr or "Git command timed out")
return subprocess.CompletedProcess(command, 124, exc.stdout or "", exc.stderr or "Git command timed out")
def _deduplicate(issues: list[SourceTagProvenanceIssue]) -> list[SourceTagProvenanceIssue]:

View File

@@ -9,7 +9,7 @@ import re
import subprocess
import tomllib
from .git_state import collect_versions
from .git_state import collect_versions, sanitized_git_environment
from .workspace import load_repository_specs, resolve_repo_path
@@ -681,30 +681,56 @@ def _git_lock_resolution_issues(
def _git_tag_commit(repo_path: Path, tag: str) -> str | None:
repo_path = repo_path.resolve()
if not (repo_path / ".git").exists():
return None
result = subprocess.run(
["git", "-C", str(repo_path), "rev-list", "-n", "1", tag],
[
"/usr/bin/git",
"-c",
"core.hooksPath=/dev/null",
"-c",
f"safe.directory={repo_path}",
"-C",
str(repo_path),
"rev-parse",
"--verify",
f"refs/tags/{tag}^{{commit}}",
],
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
timeout=10,
env=sanitized_git_environment(),
)
value = result.stdout.strip()
return value if result.returncode == 0 and value else None
def _git_tag_file(repo_path: Path, tag: str, relative_path: str) -> str | None:
repo_path = repo_path.resolve()
if not (repo_path / ".git").exists():
return None
result = subprocess.run(
["git", "-C", str(repo_path), "show", f"{tag}:{relative_path}"],
[
"/usr/bin/git",
"-c",
"core.hooksPath=/dev/null",
"-c",
f"safe.directory={repo_path}",
"-C",
str(repo_path),
"cat-file",
"blob",
f"refs/tags/{tag}^{{commit}}:{relative_path}",
],
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
timeout=10,
env=sanitized_git_environment(),
)
return result.stdout if result.returncode == 0 else None

View File

@@ -85,8 +85,11 @@ PACKAGE_MODULE_REPOS=(
"$PARENT/govoplan-certificates"
"$PARENT/govoplan-committee"
"$PARENT/govoplan-consultation"
"$PARENT/govoplan-connectors"
"$PARENT/govoplan-contracts"
"$PARENT/govoplan-dashboard"
"$PARENT/govoplan-dataflow"
"$PARENT/govoplan-datasources"
"$PARENT/govoplan-docs"
"$PARENT/govoplan-facilities"
"$PARENT/govoplan-files"
@@ -96,7 +99,7 @@ PACKAGE_MODULE_REPOS=(
"$PARENT/govoplan-identity"
"$PARENT/govoplan-idm"
"$PARENT/govoplan-inspections"
"$PARENT/govoplan-issue-reporting"
"$PARENT/govoplan-tickets"
"$PARENT/govoplan-learning"
"$PARENT/govoplan-mail"
"$PARENT/govoplan-ops"
@@ -111,12 +114,14 @@ PACKAGE_MODULE_REPOS=(
"$PARENT/govoplan-tenancy"
"$PARENT/govoplan-transparency"
"$PARENT/govoplan-evaluation"
"$PARENT/govoplan-views"
"$PARENT/govoplan-workflow"
)
TAG_ONLY_MODULE_REPOS=(
"$PARENT/govoplan-addresses"
"$PARENT/govoplan-appointments"
"$PARENT/govoplan-cases"
"$PARENT/govoplan-connectors"
"$PARENT/govoplan-projects"
"$PARENT/govoplan-dms"
"$PARENT/govoplan-dist-lists"
"$PARENT/govoplan-erp"
@@ -132,10 +137,10 @@ TAG_ONLY_MODULE_REPOS=(
"$PARENT/govoplan-rest"
"$PARENT/govoplan-scheduling"
"$PARENT/govoplan-search"
"$PARENT/govoplan-wiki"
"$PARENT/govoplan-soap"
"$PARENT/govoplan-tasks"
"$PARENT/govoplan-templates"
"$PARENT/govoplan-workflow"
"$PARENT/govoplan-xoev"
"$PARENT/govoplan-xrechnung"
"$PARENT/govoplan-xta-osci"

View File

@@ -9,7 +9,10 @@ from pathlib import Path
from govoplan_release.module_directory import write_module_directory
from govoplan_release.model import to_jsonable
from govoplan_release.publisher import publish_catalog_candidate
from govoplan_release.publisher import (
publication_runtime_trust_issues,
publish_catalog_candidate,
)
from govoplan_release.selective_catalog import build_selective_catalog_candidate
from govoplan_release.workspace import DEFAULT_WORKSPACE_ROOT
@@ -18,55 +21,137 @@ def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
subparsers = parser.add_subparsers(dest="command", required=True)
selective = subparsers.add_parser("selective", help="Build a signed selective channel catalog candidate.")
selective.add_argument("--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT)
selective = subparsers.add_parser(
"selective", help="Build a signed selective channel catalog candidate."
)
selective.add_argument(
"--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT
)
selective.add_argument("--channel", default="stable")
selective.add_argument("--repo-version", action="append", default=[], metavar="REPO=VERSION", required=True)
selective.add_argument("--catalog-signing-key", action="append", default=[], metavar="KEY_ID=PRIVATE_KEY", required=True)
selective.add_argument("--base-catalog", help="Catalog path or URL to use as the source. Defaults to local channel, then public channel.")
selective.add_argument("--output-dir", type=Path, help="Candidate output directory. Defaults to runtime/release-candidates/<channel>-<timestamp>.")
selective.add_argument(
"--repo-version",
action="append",
default=[],
metavar="REPO=VERSION",
required=True,
)
selective.add_argument(
"--catalog-signing-key",
action="append",
default=[],
metavar="KEY_ID=PRIVATE_KEY",
required=True,
)
selective.add_argument(
"--python-artifact",
action="append",
default=[],
metavar="REPO=/PATH/TO/WHEEL",
help=(
"Built immutable wheel whose archive and install-stable payload identity "
"will be computed into the signed catalog; may be repeated."
),
)
selective.add_argument(
"--base-catalog",
help=(
"Authenticated catalog path or URL to use as the source; must be "
"paired with --base-keyring. Defaults to the local pair, then public pair."
),
)
selective.add_argument(
"--base-keyring",
help=(
"Exact keyring path or URL pinned by --base-catalog and matching "
"the configured catalog signer set."
),
)
selective.add_argument(
"--output-dir",
type=Path,
help=(
"Private candidate output directory. Defaults below "
"$XDG_STATE_HOME/govoplan/release-candidates (or ~/.local/state)."
),
)
selective.add_argument("--public-base-url", default="https://govoplan.add-ideas.de")
selective.add_argument("--repository-base", default="git+ssh://git@git.add-ideas.de/add-ideas")
selective.add_argument("--source-remote", default="origin", help="Configured Git remote containing immutable source tags.")
selective.add_argument(
"--repository-base", default="git+ssh://git@git.add-ideas.de/GovOPlaN"
)
selective.add_argument(
"--source-remote",
default="origin",
help="Configured Git remote containing immutable source tags.",
)
selective.add_argument("--expires-days", type=int, default=90)
selective.add_argument("--sequence", type=int)
selective.add_argument("--skip-public-check", action="store_true")
selective.add_argument("--json", action="store_true", help="Print machine-readable summary JSON.")
selective.add_argument(
"--json", action="store_true", help="Print machine-readable summary JSON."
)
publish = subparsers.add_parser("publish-candidate", help="Publish a reviewed catalog candidate into the website repo.")
publish = subparsers.add_parser(
"publish-candidate",
help="Publish a reviewed catalog candidate into the website repo.",
)
publish.add_argument("--candidate-dir", type=Path, required=True)
publish.add_argument("--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT)
publish.add_argument("--web-root", type=Path)
publish.add_argument("--channel", default="stable")
publish.add_argument("--apply", action="store_true", help="Copy candidate files into the website repo. Without this, only preview.")
publish.add_argument(
"--apply",
action="store_true",
help="Copy candidate files into the website repo. Without this, only preview.",
)
publish.add_argument("--build-web", action="store_true")
publish.add_argument("--commit", action="store_true")
publish.add_argument("--tag", action="store_true", help="Create a website catalog publication tag. Implies --commit.")
publish.add_argument("--push", action="store_true", help="Push the website branch and tag. Implies --commit.")
publish.add_argument(
"--tag",
action="store_true",
help="Create a website catalog publication tag. Implies --commit.",
)
publish.add_argument(
"--push",
action="store_true",
help="Push the website branch and tag. Implies --commit.",
)
publish.add_argument("--remote", default="origin")
publish.add_argument("--source-remote", default="origin", help="Configured Git remote containing catalog source tags.")
publish.add_argument(
"--source-remote",
default="origin",
help="Configured Git remote containing catalog source tags.",
)
publish.add_argument("--branch")
publish.add_argument("--tag-name")
publish.add_argument("--npm", default="npm")
publish.add_argument("--allow-dirty-website", action="store_true")
publish.add_argument("--json", action="store_true", help="Print machine-readable summary JSON.")
publish.add_argument(
"--json", action="store_true", help="Print machine-readable summary JSON."
)
directory = subparsers.add_parser("module-directory", help="Generate browsable module-directory files from a catalog and keyring.")
directory = subparsers.add_parser(
"module-directory",
help="Generate browsable module-directory files from a catalog and keyring.",
)
directory.add_argument("--catalog", type=Path, required=True)
directory.add_argument("--keyring", type=Path, required=True)
directory.add_argument("--output-dir", type=Path, required=True)
directory.add_argument("--channel", default="stable")
directory.add_argument("--public-base-url", default="https://govoplan.add-ideas.de")
directory.add_argument("--json", action="store_true", help="Print machine-readable summary JSON.")
directory.add_argument(
"--json", action="store_true", help="Print machine-readable summary JSON."
)
args = parser.parse_args()
if args.command == "selective":
require_release_runtime_trust()
result = build_selective_catalog_candidate(
repo_versions=parse_repo_versions(tuple(args.repo_version)),
channel=args.channel,
workspace_root=args.workspace_root,
output_dir=args.output_dir,
base_catalog=args.base_catalog,
base_keyring=args.base_keyring,
signing_keys=tuple(args.catalog_signing_key),
public_base_url=args.public_base_url,
repository_base=args.repository_base,
@@ -74,6 +159,7 @@ def main() -> int:
expires_days=args.expires_days,
sequence=args.sequence,
check_public=not args.skip_public_check,
python_artifacts=parse_repo_paths(tuple(args.python_artifact)),
)
payload = to_jsonable(result)
if args.json:
@@ -82,6 +168,8 @@ def main() -> int:
print_text_summary(payload)
return 0 if result.status == "ready" else 1
if args.command == "publish-candidate":
if args.apply:
require_release_runtime_trust()
result = publish_catalog_candidate(
candidate_dir=args.candidate_dir,
workspace_root=args.workspace_root,
@@ -106,6 +194,7 @@ def main() -> int:
print_publish_summary(payload)
return 0 if result.status in {"ready", "applied", "published"} else 1
if args.command == "module-directory":
require_release_runtime_trust()
catalog_payload = json.loads(args.catalog.read_text(encoding="utf-8"))
keyring_payload = json.loads(args.keyring.read_text(encoding="utf-8"))
files = write_module_directory(
@@ -115,7 +204,11 @@ def main() -> int:
channel=args.channel,
public_base_url=args.public_base_url,
)
payload = {"status": "generated", "output_dir": str(args.output_dir), "files": [str(path) for path in files]}
payload = {
"status": "generated",
"output_dir": str(args.output_dir),
"files": [str(path) for path in files],
}
if args.json:
print(json.dumps(payload, indent=2, sort_keys=True))
else:
@@ -130,6 +223,12 @@ def main() -> int:
return 2
def require_release_runtime_trust() -> None:
issues = publication_runtime_trust_issues()
if issues:
raise SystemExit("Release tooling trust gate failed: " + "; ".join(issues))
def parse_repo_versions(values: tuple[str, ...]) -> dict[str, str]:
result: dict[str, str] = {}
for value in values:
@@ -144,6 +243,22 @@ def parse_repo_versions(values: tuple[str, ...]) -> dict[str, str]:
return result
def parse_repo_paths(values: tuple[str, ...]) -> dict[str, Path]:
result: dict[str, Path] = {}
for value in values:
if "=" not in value:
raise SystemExit(f"--python-artifact must use REPO=/path/to/wheel: {value}")
repo, path_text = value.split("=", 1)
repo = repo.strip()
path_text = path_text.strip()
if not repo or not path_text or repo in result:
raise SystemExit(
f"--python-artifact must uniquely use REPO=/path/to/wheel: {value}"
)
result[repo] = Path(path_text).expanduser()
return result
def print_text_summary(payload: dict[str, object]) -> None:
print("Selective catalog candidate")
for key in (

View File

@@ -4,6 +4,7 @@
from __future__ import annotations
import argparse
import ipaddress
from pathlib import Path
import secrets
import sys
@@ -11,12 +12,31 @@ import sys
from govoplan_release.workspace import DEFAULT_WORKSPACE_ROOT
def loopback_host(value: str) -> str:
try:
address = ipaddress.ip_address(value)
except ValueError as exc:
raise argparse.ArgumentTypeError(
"release console host must be a numeric loopback address"
) from exc
if not address.is_loopback:
raise argparse.ArgumentTypeError(
"release console refuses non-loopback hosts without an authenticated "
"TLS deployment boundary"
)
return value
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--host", default="127.0.0.1", help="Bind host. Defaults to 127.0.0.1.")
parser.add_argument(
"--host",
type=loopback_host,
default="127.0.0.1",
help="Numeric loopback bind address. Defaults to 127.0.0.1.",
)
parser.add_argument("--port", type=int, default=8765, help="Bind port. Defaults to 8765.")
parser.add_argument("--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT)
parser.add_argument("--no-token", action="store_true", help="Disable the local API token guard.")
args = parser.parse_args()
try:
@@ -28,13 +48,13 @@ def main() -> int:
from server.app import create_app
workspace_root = args.workspace_root.expanduser().resolve()
token = None if args.no_token else secrets.token_urlsafe(24)
token = secrets.token_urlsafe(24)
app = create_app(workspace_root=workspace_root, token=token)
url = f"http://{args.host}:{args.port}/"
if token:
# URL fragments are never sent in HTTP requests. The WebUI transfers
# this one-time bootstrap token to sessionStorage and clears the hash.
url = f"{url}#token={token}"
display_host = f"[{args.host}]" if ":" in args.host else args.host
url = f"http://{display_host}:{args.port}/"
# URL fragments are never sent in HTTP requests. The WebUI transfers
# this one-time bootstrap token to sessionStorage and clears the hash.
url = f"{url}#token={token}"
print("GovOPlaN release console")
print(f" workspace: {workspace_root}")
print(f" url: {url}")

View File

@@ -9,13 +9,12 @@ from typing import Literal
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import HTMLResponse, JSONResponse
from pydantic import BaseModel, Field
from pydantic import BaseModel, Field, field_validator
from govoplan_release import (
build_dashboard,
build_release_intelligence,
build_release_plan,
build_selective_catalog_candidate,
build_selective_release_plan,
publish_catalog_candidate,
prepare_repositories,
@@ -24,13 +23,37 @@ from govoplan_release import (
tag_repositories,
)
from govoplan_release.model import to_jsonable
from govoplan_release.candidate_artifact import (
CandidateArtifactError,
issue_candidate_id,
validate_release_channel,
verify_candidate_receipt,
)
from govoplan_release.release_execution import (
DURABLE_REMOTE,
ReleaseExecutionAmbiguous,
ReleaseExecutionBlocked,
bind_plan_source_states,
execute_repository_step,
executor_spec,
generate_catalog_candidate,
publish_received_candidate,
require_trusted_release_runtime,
reconciled_catalog_publication_receipt,
reconciled_repository_receipt,
validated_candidate_receipt,
verify_catalog_publication_precondition,
verify_repository_preflight_binding,
verify_repository_step_precondition,
verify_release_runtime_binding,
)
from govoplan_release.release_run import (
ReleaseRunConflict,
ReleaseRunCorrupt,
ReleaseRunNotFound,
ReleaseRunStore,
)
from govoplan_release.workspace import DEFAULT_WORKSPACE_ROOT, META_ROOT
from govoplan_release.workspace import DEFAULT_WORKSPACE_ROOT, META_ROOT, website_root
class CatalogCandidateRequest(BaseModel):
@@ -41,13 +64,19 @@ class CatalogCandidateRequest(BaseModel):
signing_keys: list[str] = Field(default_factory=list)
output_dir: str | None = None
base_catalog: str | None = None
base_keyring: str | None = None
expires_days: int = 90
sequence: int | None = None
public_base_url: str = "https://govoplan.add-ideas.de"
repository_base: str = "git+ssh://git@git.add-ideas.de/add-ideas"
repository_base: str = "git+ssh://git@git.add-ideas.de/GovOPlaN"
source_remote: str = "origin"
check_public: bool = True
@field_validator("channel")
@classmethod
def validate_channel(cls, value: str) -> str:
return validate_release_channel(value)
class PublishCandidateRequest(BaseModel):
candidate_dir: str
@@ -66,17 +95,22 @@ class PublishCandidateRequest(BaseModel):
allow_dirty_website: bool = False
confirm: str = ""
@field_validator("channel")
@classmethod
def validate_channel(cls, value: str) -> str:
return validate_release_channel(value)
class RepositoryPushRequest(BaseModel):
repos: list[str] = Field(default_factory=list)
remote: str = "origin"
remote: Literal["origin"] = "origin"
apply: bool = False # noqa: A003 - API field mirrors CLI.
confirm: str = ""
class RepositorySyncRequest(BaseModel):
repos: list[str] = Field(default_factory=list)
remote: str = "origin"
remote: Literal["origin"] = "origin"
apply: bool = False # noqa: A003 - API field mirrors CLI.
confirm: str = ""
@@ -108,6 +142,11 @@ class ReleaseRunCreateRequest(BaseModel):
public_catalog: bool = True
include_migrations: bool = False
@field_validator("channel")
@classmethod
def validate_channel(cls, value: str) -> str:
return validate_release_channel(value)
class ReleaseRunCommandRequest(BaseModel):
request_id: str = Field(min_length=8, max_length=128)
@@ -119,11 +158,23 @@ class ReleaseRunReconcileRequest(BaseModel):
confirm: str = Field(default="", max_length=32)
class ReleaseRunExecuteRequest(BaseModel):
request_id: str = Field(min_length=8, max_length=128)
confirm: str = Field(default="", max_length=32)
remote: Literal["origin"] = "origin"
signing_keys: list[str] = Field(default_factory=list, max_length=8)
class ReleaseRunPreviewRequest(BaseModel):
remote: Literal["origin"] = "origin"
def create_app(
*,
workspace_root: Path = DEFAULT_WORKSPACE_ROOT,
token: str | None = None,
run_state_root: Path | None = None,
candidate_root: Path | None = None,
) -> FastAPI:
app = FastAPI(title="GovOPlaN Release Console", version="0.1.0")
app.state.workspace_root = workspace_root
@@ -140,13 +191,31 @@ def create_app(
release_run_root,
expected_workspace_fingerprint=app.state.workspace_fingerprint,
)
app.state.release_candidate_root = (
Path(os.path.abspath(os.fspath(candidate_root.expanduser())))
if candidate_root is not None
else release_run_root.parent / "release-candidates"
)
@app.middleware("http")
async def require_token(request: Request, call_next): # type: ignore[no-untyped-def]
if token and request.url.path.startswith("/api/"):
provided = request.headers.get("x-release-console-token")
if provided != token:
return JSONResponse({"detail": "release console token required"}, status_code=401)
if request.url.path.startswith("/api/"):
if token:
provided = request.headers.get("x-release-console-token")
if provided != token:
return JSONResponse(
{"detail": "release console token required"},
status_code=401,
)
elif request.method not in {"GET", "HEAD", "OPTIONS"}:
return JSONResponse(
{
"detail": (
"release console is read-only without an API token"
)
},
status_code=403,
)
return await call_next(request)
@app.get("/", response_class=HTMLResponse)
@@ -174,6 +243,7 @@ def create_app(
include_website: bool = False,
channel: str = "stable",
) -> dict[str, object]:
channel = http_release_channel(channel)
snapshot = build_dashboard(
workspace_root=app.state.workspace_root,
target_version=target_version,
@@ -195,6 +265,7 @@ def create_app(
include_migrations: bool = False,
channel: str = "stable",
) -> dict[str, object]:
channel = http_release_channel(channel)
snapshot = build_dashboard(
workspace_root=app.state.workspace_root,
target_version=target_version,
@@ -212,6 +283,7 @@ def create_app(
channel: str = "stable",
public_catalog: bool = True,
) -> dict[str, object]:
channel = http_release_channel(channel)
intelligence = build_release_intelligence(
workspace_root=app.state.workspace_root,
channel=channel,
@@ -230,6 +302,7 @@ def create_app(
include_migrations: bool = False,
channel: str = "stable",
) -> dict[str, object]:
channel = http_release_channel(channel)
snapshot = build_dashboard(
workspace_root=app.state.workspace_root,
target_version=target_version,
@@ -249,9 +322,11 @@ def create_app(
return to_jsonable(release_plan)
@app.get("/api/release-runs")
def release_runs(limit: int = 20) -> dict[str, object]:
def release_runs(
limit: int = 20, cursor: str | None = None
) -> dict[str, object]:
try:
return {"runs": app.state.release_runs.list(limit=limit)}
return app.state.release_runs.list_page(limit=limit, cursor=cursor)
except (ReleaseRunConflict, ReleaseRunCorrupt) as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
@@ -278,7 +353,13 @@ def create_app(
except (ReleaseRunConflict, ReleaseRunCorrupt) as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
if existing is not None:
return existing
return release_run_view(existing)
try:
require_trusted_release_runtime(
workspace_root=app.state.workspace_root
)
except ReleaseExecutionBlocked as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
snapshot = build_dashboard(
workspace_root=app.state.workspace_root,
online=request.online,
@@ -317,18 +398,29 @@ def create_app(
),
)
try:
return app.state.release_runs.create(
request_id=request.request_id,
input_snapshot=input_snapshot,
plan_snapshot=plan_payload,
plan_payload = bind_plan_source_states(
plan=plan_payload,
repo_versions=dict(request.repo_versions),
workspace_root=app.state.workspace_root,
)
except (ReleaseRunConflict, ReleaseRunCorrupt) as exc:
return release_run_view(
app.state.release_runs.create(
request_id=request.request_id,
input_snapshot=input_snapshot,
plan_snapshot=plan_payload,
)
)
except (
ReleaseExecutionBlocked,
ReleaseRunConflict,
ReleaseRunCorrupt,
) as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
@app.get("/api/release-runs/{run_id}")
def release_run(run_id: str) -> dict[str, object]:
try:
return app.state.release_runs.get(run_id)
return release_run_view(app.state.release_runs.get(run_id))
except ReleaseRunNotFound as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except (ReleaseRunConflict, ReleaseRunCorrupt) as exc:
@@ -339,7 +431,11 @@ def create_app(
run_id: str, request: ReleaseRunCommandRequest
) -> dict[str, object]:
try:
return app.state.release_runs.resume(run_id, request_id=request.request_id)
return release_run_view(
app.state.release_runs.resume(
run_id, request_id=request.request_id
)
)
except ReleaseRunNotFound as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except (ReleaseRunConflict, ReleaseRunCorrupt) as exc:
@@ -350,10 +446,12 @@ def create_app(
run_id: str, step_id: str, request: ReleaseRunCommandRequest
) -> dict[str, object]:
try:
return app.state.release_runs.retry_step(
run_id,
step_id,
request_id=request.request_id,
return release_run_view(
app.state.release_runs.retry_step(
run_id,
step_id,
request_id=request.request_id,
)
)
except ReleaseRunNotFound as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@@ -365,53 +463,442 @@ def create_app(
run_id: str, step_id: str, request: ReleaseRunReconcileRequest
) -> dict[str, object]:
try:
return app.state.release_runs.reconcile_step(
run_id,
step_id,
request_id=request.request_id,
outcome=request.outcome,
confirmation=request.confirm,
receipt = None
run = app.state.release_runs.get(run_id)
state_step = release_run_state_step(run, step_id)
immutable = run.get("immutable")
frozen_plan = (
immutable.get("plan") if isinstance(immutable, dict) else None
)
verify_release_runtime_binding(
expected=(
frozen_plan.get("runtime_binding")
if isinstance(frozen_plan, dict)
else None
),
workspace_root=app.state.workspace_root,
)
if (
request.outcome == "effect_succeeded"
and state_step.get("state") == "succeeded"
and state_step.get("result_code") == "reconciled_effect_succeeded"
):
receipt = state_step.get("result_receipt")
if (
request.outcome == "effect_succeeded"
and step_id == "catalog:selective-generator"
and state_step.get("state") == "interrupted"
):
attempt_fingerprint = (
app.state.release_runs.current_attempt_fingerprint(
run_id, step_id
)
)
candidate_id = issue_candidate_id(
run_id, step_id, attempt_fingerprint
)
candidate_receipt = validated_candidate_receipt(
candidate_root=app.state.release_candidate_root,
candidate_id=candidate_id,
channel=app.state.release_runs.get(run_id)["immutable"]["input"][
"channel"
],
)
receipt = {
"kind": "catalog_candidate",
"candidate_id": candidate_receipt.candidate_id,
"catalog_sha256": candidate_receipt.catalog_sha256,
}
elif (
request.outcome == "effect_succeeded"
and step_id == "catalog:validate-sign-publish"
and state_step.get("state") == "interrupted"
):
plan_step = release_run_plan_step(run, step_id)
candidate_receipt = release_run_candidate_receipt(run)
receipt = reconciled_catalog_publication_receipt(
candidate_path=verified_run_candidate(
run,
candidate_root=app.state.release_candidate_root,
),
candidate_receipt=candidate_receipt,
channel=run["immutable"]["input"]["channel"],
workspace_root=app.state.workspace_root,
remote=DURABLE_REMOTE,
expected_website_receipt=plan_step.get("source_binding"),
)
elif request.outcome == "effect_succeeded" and state_step.get(
"state"
) == "interrupted":
plan_step = release_run_plan_step(run, step_id)
spec = executor_spec(plan_step)
if spec is not None and spec.kind in {"tag", "push"}:
repo = str(plan_step["repo"])
receipt = reconciled_repository_receipt(
spec=spec,
plan_step=plan_step,
version=run["immutable"]["input"]["repo_versions"][repo],
workspace_root=app.state.workspace_root,
expected_receipt=preceding_repository_receipt(
run, step_id=step_id, repo=repo
),
)
return release_run_view(
app.state.release_runs.reconcile_step(
run_id,
step_id,
request_id=request.request_id,
outcome=request.outcome,
confirmation=request.confirm,
result_receipt=receipt,
)
)
except ReleaseRunNotFound as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except (ReleaseRunConflict, ReleaseRunCorrupt) as exc:
except (
CandidateArtifactError,
ReleaseExecutionBlocked,
ReleaseRunConflict,
ReleaseRunCorrupt,
) as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
@app.post("/api/release-runs/{run_id}/steps/{step_id}/execute")
def execute_release_run_step(
run_id: str, step_id: str, request: ReleaseRunExecuteRequest
) -> dict[str, object]:
claim_acquired = False
try:
run = app.state.release_runs.get(run_id)
plan_step = release_run_plan_step(run, step_id)
immutable = run.get("immutable")
frozen_plan = (
immutable.get("plan") if isinstance(immutable, dict) else None
)
verify_release_runtime_binding(
expected=(
frozen_plan.get("runtime_binding")
if isinstance(frozen_plan, dict)
else None
),
workspace_root=app.state.workspace_root,
)
spec = executor_spec(plan_step)
if spec is None:
raise ReleaseRunConflict(
"This frozen plan step has no bounded durable executor."
)
if request.confirm != spec.confirmation:
required = spec.confirmation or "an empty confirmation"
raise ReleaseRunConflict(
f"Release step requires exactly {required}."
)
expected_repository_receipt = None
claim = app.state.release_runs.claim_step(
run_id,
step_id,
attempt_id=request.request_id,
)
if not claim.claimed:
if claim.outcome == "succeeded":
replay = release_run_view(claim.run)
replay["execution_result"] = {
"status": "replayed",
"result_code": claim.result_code,
"result_receipt": claim.result_receipt,
}
return replay
if claim.outcome == "running":
raise ReleaseRunConflict(
"This exact executor attempt is still running or its response "
"is uncertain; resume and reconcile it before retrying."
)
raise ReleaseRunConflict(
"This exact executor attempt already ended; use the run's explicit "
"retry or reconciliation action."
)
claim_acquired = True
signing_keys = tuple(request.signing_keys or default_signing_keys())
if spec.kind == "catalog_generate" and not signing_keys:
raise ReleaseExecutionBlocked(
"Durable catalog generation requires a configured signing key."
)
candidate_path = None
candidate_receipt = None
website_receipt = None
if spec.kind == "catalog_publish":
candidate_path = verified_run_candidate(
run,
candidate_root=app.state.release_candidate_root,
)
candidate_receipt = release_run_candidate_receipt(run)
website_receipt = verify_catalog_publication_precondition(
plan_step=plan_step,
workspace_root=app.state.workspace_root,
)
if spec.kind in {"preflight", "tag", "push"}:
repo = str(plan_step["repo"])
version = run["immutable"]["input"]["repo_versions"][repo]
expected_repository_receipt = preceding_repository_receipt(
run, step_id=step_id, repo=repo
)
if spec.kind == "preflight":
verify_repository_preflight_binding(
plan_step=plan_step,
version=version,
workspace_root=app.state.workspace_root,
)
else:
verify_repository_step_precondition(
spec=spec,
plan_step=plan_step,
version=version,
workspace_root=app.state.workspace_root,
expected_receipt=expected_repository_receipt,
)
receipt = None
if spec.kind in {"preflight", "tag", "push"}:
result, receipt = execute_repository_step(
spec=spec,
plan_step=plan_step,
repo_versions=run["immutable"]["input"]["repo_versions"],
workspace_root=app.state.workspace_root,
remote=DURABLE_REMOTE,
expected_receipt=expected_repository_receipt,
)
elif spec.kind == "catalog_generate":
attempt_fingerprint = (
app.state.release_runs.current_attempt_fingerprint(
run_id, step_id
)
)
candidate_id = issue_candidate_id(
run_id, step_id, attempt_fingerprint
)
base_catalog, base_keyring = release_catalog_base_paths(
app.state.workspace_root,
channel=run["immutable"]["input"]["channel"],
)
result, candidate_receipt = generate_catalog_candidate(
candidate_root=app.state.release_candidate_root,
candidate_id=candidate_id,
channel=run["immutable"]["input"]["channel"],
repo_versions=run["immutable"]["input"]["repo_versions"],
workspace_root=app.state.workspace_root,
signing_keys=signing_keys,
remote=DURABLE_REMOTE,
check_public=run["immutable"]["input"]["public_catalog"],
source_receipts={
repo: receipt
for repo in run["immutable"]["input"]["repo_versions"]
if isinstance(
(
receipt := preceding_repository_receipt(
run, step_id=step_id, repo=repo
)
),
dict,
)
},
base_catalog=base_catalog,
base_keyring=base_keyring,
)
receipt = {
"kind": "catalog_candidate",
"candidate_id": candidate_receipt.candidate_id,
"catalog_sha256": candidate_receipt.catalog_sha256,
}
elif (
spec.kind == "catalog_publish"
and candidate_path is not None
and candidate_receipt is not None
and website_receipt is not None
):
candidate_path = verified_run_candidate(
run,
candidate_root=app.state.release_candidate_root,
)
result, receipt = publish_received_candidate(
candidate_path=candidate_path,
candidate_receipt=candidate_receipt,
channel=run["immutable"]["input"]["channel"],
workspace_root=app.state.workspace_root,
remote=DURABLE_REMOTE,
expected_website_receipt=website_receipt,
)
try:
verified_run_candidate(
run,
candidate_root=app.state.release_candidate_root,
)
except CandidateArtifactError as exc:
raise ReleaseExecutionAmbiguous(
"Published candidate changed while its effect was in flight; "
"reconcile the immutable website commit and remote tag."
) from exc
else:
raise ReleaseRunConflict("Release executor mapping is incomplete.")
finished = app.state.release_runs.finish_step(
run_id,
step_id,
attempt_id=request.request_id,
succeeded=True,
result_code=spec.result_code,
result_receipt=receipt,
)
response = release_run_view(finished)
response["execution_result"] = {
"status": "succeeded",
"result_code": spec.result_code,
"result_receipt": receipt,
"output": result,
}
return response
except ReleaseExecutionBlocked as exc:
if not claim_acquired:
raise HTTPException(status_code=409, detail=str(exc)[:4000]) from exc
failed = app.state.release_runs.finish_step(
run_id,
step_id,
attempt_id=request.request_id,
succeeded=False,
result_code="executor_blocked",
)
response = release_run_view(failed)
response["execution_result"] = {
"status": "failed",
"result_code": "executor_blocked",
"detail": str(exc)[:4000],
}
return response
except ReleaseExecutionAmbiguous as exc:
app.state.release_runs.interrupt_step(
run_id,
step_id,
attempt_id=request.request_id,
)
raise HTTPException(status_code=409, detail=str(exc)[:4000]) from exc
except ReleaseRunNotFound as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except CandidateArtifactError as exc:
if claim_acquired:
failed = app.state.release_runs.finish_step(
run_id,
step_id,
attempt_id=request.request_id,
succeeded=False,
result_code="candidate_invalid",
)
response = release_run_view(failed)
response["execution_result"] = {
"status": "failed",
"result_code": "candidate_invalid",
"detail": str(exc)[:4000],
}
return response
raise HTTPException(status_code=409, detail=str(exc)) from exc
except ReleaseRunCorrupt as exc:
if claim_acquired:
try:
app.state.release_runs.interrupt_step(
run_id,
step_id,
attempt_id=request.request_id,
)
except (ReleaseRunConflict, ReleaseRunCorrupt, ReleaseRunNotFound):
pass
raise HTTPException(
status_code=409,
detail=(
"The executor outcome could not be persisted safely; resume "
"and reconcile this attempt before retrying."
),
) from exc
raise HTTPException(status_code=409, detail=str(exc)) from exc
except ReleaseRunConflict as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except Exception as exc: # noqa: BLE001 - never retry an uncertain effect.
try:
app.state.release_runs.interrupt_step(
run_id,
step_id,
attempt_id=request.request_id,
)
except (ReleaseRunConflict, ReleaseRunCorrupt, ReleaseRunNotFound):
pass
raise HTTPException(
status_code=409,
detail=(
f"Executor raised {type(exc).__name__}; its effect is unknown. "
"Resume and reconcile before retry."
),
) from exc
@app.post("/api/release-runs/{run_id}/steps/{step_id}/preview")
def preview_release_run_step(
run_id: str, step_id: str, request: ReleaseRunPreviewRequest
) -> dict[str, object]:
try:
run = app.state.release_runs.get(run_id)
plan_step = release_run_plan_step(run, step_id)
spec = executor_spec(plan_step)
if spec is None or spec.kind != "catalog_publish":
raise ReleaseRunConflict(
"Only the receipt-bound catalog publication step has this preview."
)
candidate = verified_run_candidate(
run,
candidate_root=app.state.release_candidate_root,
)
return to_jsonable(
publish_catalog_candidate(
candidate_dir=candidate,
workspace_root=app.state.workspace_root,
channel=run["immutable"]["input"]["channel"],
remote=DURABLE_REMOTE,
source_remote=DURABLE_REMOTE,
apply=False,
)
)
except ReleaseRunNotFound as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except (
CandidateArtifactError,
ReleaseRunConflict,
ReleaseRunCorrupt,
) as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
@app.get("/api/catalog-candidates")
def catalog_candidates() -> dict[str, object]:
return {"candidates": list_catalog_candidates()}
return {
"candidates": list_catalog_candidates(app.state.release_candidate_root)
}
@app.post("/api/catalog-candidates")
def catalog_candidate(request: CatalogCandidateRequest) -> dict[str, object]:
repo_versions = dict(request.repo_versions)
if not repo_versions and request.repos and request.target_version:
repo_versions = {repo: request.target_version for repo in request.repos}
if not repo_versions:
raise HTTPException(status_code=400, detail="repo_versions or repos with target_version is required")
signing_keys = tuple(request.signing_keys or default_signing_keys())
if not signing_keys:
raise HTTPException(status_code=400, detail="No signing key supplied and default release key was not found")
try:
candidate = build_selective_catalog_candidate(
repo_versions=repo_versions,
channel=request.channel,
workspace_root=app.state.workspace_root,
output_dir=request.output_dir,
base_catalog=request.base_catalog,
signing_keys=signing_keys,
public_base_url=request.public_base_url,
repository_base=request.repository_base,
source_remote=request.source_remote,
expires_days=request.expires_days,
sequence=request.sequence,
check_public=request.check_public,
)
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return to_jsonable(candidate)
del request
raise HTTPException(
status_code=409,
detail=(
"Signed catalog generation is available only through a durable "
"release run with a trusted runtime and receipt-bound source tags."
),
)
@app.post("/api/catalog-candidates/publish")
def publish_candidate(request: PublishCandidateRequest) -> dict[str, object]:
if request.apply or request.commit or request.tag or request.push or request.build_web:
raise HTTPException(
status_code=409,
detail=(
"Catalog mutation is available only through a durable release run "
"with a server-issued candidate receipt."
),
)
if request.push and request.confirm != "PUSH":
raise HTTPException(status_code=400, detail="Push requires confirm=PUSH")
if (request.apply or request.commit or request.tag or request.build_web) and not request.push and request.confirm != "APPLY":
@@ -440,8 +927,14 @@ def create_app(
repos = tuple(dict.fromkeys(repo.strip() for repo in request.repos if repo.strip()))
if not repos:
raise HTTPException(status_code=400, detail="At least one repository must be selected")
if request.apply and request.confirm != "PUSH":
raise HTTPException(status_code=400, detail="Repository push requires confirm=PUSH")
if request.apply:
raise HTTPException(
status_code=409,
detail=(
"Repository push mutation is disabled outside a durable "
"receipt-bound maintenance run."
),
)
result = push_repositories(repos=repos, workspace_root=app.state.workspace_root, remote=request.remote, apply=request.apply)
return to_jsonable(result)
@@ -450,8 +943,14 @@ def create_app(
repos = tuple(dict.fromkeys(repo.strip() for repo in request.repos if repo.strip()))
if not repos:
raise HTTPException(status_code=400, detail="At least one repository must be selected")
if request.apply and request.confirm != "SYNC":
raise HTTPException(status_code=400, detail="Repository sync requires confirm=SYNC")
if request.apply:
raise HTTPException(
status_code=409,
detail=(
"Repository sync mutation is disabled outside a durable "
"receipt-bound maintenance run."
),
)
result = sync_repositories(repos=repos, workspace_root=app.state.workspace_root, remote=request.remote, apply=request.apply)
return to_jsonable(result)
@@ -460,6 +959,15 @@ def create_app(
repos = tuple(dict.fromkeys(repo.strip() for repo in request.repos if repo.strip()))
if not repos:
raise HTTPException(status_code=400, detail="At least one repository must be selected")
if request.apply:
raise HTTPException(
status_code=409,
detail=(
"Release preparation mutation is disabled outside a durable "
"release executor; use preview and commit reviewed changes "
"outside this console for now."
),
)
if request.apply and request.confirm != "COMMIT":
raise HTTPException(status_code=400, detail="Repository prepare requires confirm=COMMIT")
result = prepare_repositories(
@@ -476,6 +984,14 @@ def create_app(
repos = tuple(dict.fromkeys(repo.strip() for repo in request.repos if repo.strip()))
if not repos:
raise HTTPException(status_code=400, detail="At least one repository must be selected")
if request.apply:
raise HTTPException(
status_code=409,
detail=(
"Release tag mutation is available only through a durable "
"release run with creation-time repository bindings."
),
)
if request.apply and request.push and request.confirm != "PUBLISH":
raise HTTPException(status_code=400, detail="Release tag publication requires confirm=PUBLISH")
if request.apply and not request.push and request.confirm != "TAG":
@@ -494,6 +1010,164 @@ def create_app(
return app
def release_run_view(run: dict[str, object]) -> dict[str, object]:
state = run.get("state")
immutable = run.get("immutable")
plan = immutable.get("plan") if isinstance(immutable, dict) else None
steps = state.get("steps", []) if isinstance(state, dict) else []
plan_steps = plan.get("dry_run_steps", []) if isinstance(plan, dict) else []
if isinstance(steps, list) and isinstance(plan_steps, list):
plan_by_id = {
step.get("id"): step for step in plan_steps if isinstance(step, dict)
}
for step in steps:
if not isinstance(step, dict):
continue
plan_step = plan_by_id.get(step.get("id"))
spec = executor_spec(plan_step) if isinstance(plan_step, dict) else None
step["executor"] = {
"available": spec is not None,
"kind": spec.kind if spec is not None else None,
"confirmation": spec.confirmation if spec is not None else None,
}
return run
def release_run_plan_step(
run: dict[str, object], step_id: str
) -> dict[str, object]:
immutable = run.get("immutable")
plan = immutable.get("plan") if isinstance(immutable, dict) else None
steps = plan.get("dry_run_steps") if isinstance(plan, dict) else None
if not isinstance(steps, list):
raise ReleaseRunCorrupt("Release run plan steps are unavailable.")
for step in steps:
if isinstance(step, dict) and step.get("id") == step_id:
return step
raise ReleaseRunNotFound("Release run step was not found.")
def release_run_state_step(
run: dict[str, object], step_id: str
) -> dict[str, object]:
state = run.get("state")
steps = state.get("steps") if isinstance(state, dict) else None
if not isinstance(steps, list):
raise ReleaseRunCorrupt("Release run state steps are unavailable.")
for step in steps:
if isinstance(step, dict) and step.get("id") == step_id:
return step
raise ReleaseRunNotFound("Release run step was not found.")
def preceding_repository_receipt(
run: dict[str, object], *, step_id: str, repo: str
) -> dict[str, object] | None:
immutable = run.get("immutable")
plan = immutable.get("plan") if isinstance(immutable, dict) else None
plan_steps = plan.get("dry_run_steps") if isinstance(plan, dict) else None
state = run.get("state")
state_steps = state.get("steps") if isinstance(state, dict) else None
if not isinstance(plan_steps, list) or not isinstance(state_steps, list):
raise ReleaseRunCorrupt("Release run repository receipt chain is unavailable.")
pairs = list(zip(plan_steps, state_steps, strict=True))
current_index = next(
(
index
for index, (planned, _mutable) in enumerate(pairs)
if isinstance(planned, dict) and planned.get("id") == step_id
),
None,
)
if current_index is None:
raise ReleaseRunNotFound("Release run step was not found.")
for planned, mutable in reversed(pairs[:current_index]):
if not isinstance(planned, dict) or planned.get("repo") != repo:
continue
if not isinstance(mutable, dict) or mutable.get("state") != "succeeded":
return None
receipt = mutable.get("result_receipt")
return (
receipt
if isinstance(receipt, dict)
and receipt.get("kind") == "repository_state"
and receipt.get("repo") == repo
else None
)
return None
def verified_run_candidate(
run: dict[str, object], *, candidate_root: Path
) -> Path:
receipt, channel = release_run_candidate_receipt(run, include_channel=True)
return verify_candidate_receipt(
root=candidate_root,
candidate_id=str(receipt.get("candidate_id") or ""),
catalog_sha256=str(receipt.get("catalog_sha256") or ""),
channel=channel,
)
def release_run_candidate_receipt(
run: dict[str, object], *, include_channel: bool = False
) -> dict[str, object] | tuple[dict[str, object], str]:
state = run.get("state")
steps = state.get("steps") if isinstance(state, dict) else None
input_snapshot = (
run.get("immutable", {}).get("input", {})
if isinstance(run.get("immutable"), dict)
else {}
)
channel = input_snapshot.get("channel") if isinstance(input_snapshot, dict) else None
if not isinstance(steps, list) or not isinstance(channel, str):
raise ReleaseRunCorrupt("Release run candidate dependency is unavailable.")
generator = next(
(
step
for step in steps
if isinstance(step, dict)
and step.get("id") == "catalog:selective-generator"
),
None,
)
receipt = generator.get("result_receipt") if isinstance(generator, dict) else None
if (
not isinstance(generator, dict)
or generator.get("state") != "succeeded"
or not isinstance(receipt, dict)
or receipt.get("kind") != "catalog_candidate"
):
raise ReleaseRunConflict(
"Catalog publication requires the successful generator's durable receipt."
)
copied = dict(receipt)
return (copied, channel) if include_channel else copied
def default_release_candidate_root(workspace_root: Path) -> Path:
return default_release_run_root(workspace_root).parent / "release-candidates"
def release_catalog_base_paths(
workspace_root: Path, *, channel: str
) -> tuple[Path, Path]:
"""Resolve durable generation trust material only from the website checkout."""
catalog_root = website_root(workspace_root) / "public" / "catalogs" / "v1"
return (
catalog_root / "channels" / f"{validate_release_channel(channel)}.json",
catalog_root / "keyring.json",
)
def http_release_channel(value: str) -> str:
try:
return validate_release_channel(value)
except CandidateArtifactError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
def default_release_run_root(workspace_root: Path) -> Path:
configured_state_home = os.environ.get("XDG_STATE_HOME", "").strip()
state_home = Path(configured_state_home).expanduser()
@@ -550,8 +1224,8 @@ def default_signing_keys() -> tuple[str, ...]:
return (f"release-key-1={key_path}",)
def list_catalog_candidates() -> list[dict[str, object]]:
root = META_ROOT / "runtime" / "release-candidates"
def list_catalog_candidates(root: Path | None = None) -> list[dict[str, object]]:
root = root or (META_ROOT / "runtime" / "release-candidates")
if not root.exists():
return []
candidates: list[dict[str, object]] = []

View File

@@ -39,6 +39,7 @@
padding: 18px 28px;
position: sticky;
top: 0;
z-index: 10;
}
header h1 {
@@ -281,7 +282,7 @@
table {
width: 100%;
border-collapse: collapse;
min-width: 1220px;
min-width: 960px;
}
th,
@@ -608,7 +609,7 @@
</div>
<div class="units-toolbar">
<button id="buildSelectedPlan">Build Plan</button>
<button id="generateSelectedCandidate">Generate Candidate</button>
<button id="generateSelectedCandidate" disabled data-release-disabled="true" title="Use the receipt-bound generator in Durable Run State.">Generate Candidate</button>
<button class="secondary" id="selectUnpushedUnits" title="Select repositories with committed branch changes to push.">Select Unpushed</button>
<button class="secondary" id="selectChangedUnits" title="Select repositories with dirty/untracked paths, missing HEAD, or unpushed commits.">Select Dirty/Changed</button>
<button class="secondary" id="selectUnreleasedUnits">Select Unreleased</button>
@@ -642,7 +643,7 @@
<small id="runStatus">no run selected</small>
</div>
<div class="details">
<p class="hint"><strong>State tracking only.</strong> A run freezes the selected inputs and plan, guides recovery, and records bounded state events. It does not execute or confirm tags, signing, publication, or catalog changes.</p>
<p class="hint"><strong>Durable execution.</strong> A run freezes the selected inputs and plan, claims each supported executor before its effect, and keeps explicit confirmation and reconciliation boundaries. Unsupported dependency-ordering or version-metadata steps stay visible and disabled.</p>
<div class="form-grid">
<div>
<label for="releaseRun">Saved run</label>
@@ -650,6 +651,9 @@
<option value="">No saved run selected</option>
</select>
</div>
<div class="button-row">
<button class="secondary" id="loadOlderReleaseRuns" disabled>Load older</button>
</div>
</div>
<div class="button-row">
<button id="createReleaseRun">Create Run from Selection</button>
@@ -675,7 +679,7 @@
<small id="pushStatus"></small>
</div>
<div class="details">
<p class="hint">Sync selected fetches remote refs and tags only. It does not merge, rebase, commit, tag, or push.</p>
<p class="hint">Repository sync and push are preview-only until they have a receipt-bound durable maintenance executor.</p>
<div class="form-grid">
<div>
<label for="pushRemote">Remote</label>
@@ -688,9 +692,9 @@
</div>
<div class="button-row">
<button class="secondary" id="previewRepoSync">Preview Sync</button>
<button id="syncRepos">Sync Selected</button>
<button id="syncRepos" disabled data-release-disabled="true" title="Durable maintenance executor required">Sync Selected</button>
<button class="secondary" id="previewRepoPush">Preview Push</button>
<button class="danger" id="pushRepos">Push Selected</button>
<button class="danger" id="pushRepos" disabled data-release-disabled="true" title="Durable maintenance executor required">Push Selected</button>
</div>
<div class="actions" id="pushOutput"></div>
</div>
@@ -702,7 +706,7 @@
<small id="prepareStatus"></small>
</div>
<div class="details">
<p class="hint">Prepare commits selected dirty worktrees only. It does not tag or push.</p>
<p class="hint">Preview preparation only. Commit mutation is disabled outside a content-bound durable executor; commit and review dirty or version-changing work outside this console, then build a new run.</p>
<div class="form-grid">
<div>
<label for="prepareMessage">Commit message</label>
@@ -710,12 +714,12 @@
</div>
<div>
<label for="prepareConfirmText">Confirm</label>
<input id="prepareConfirmText" type="text" placeholder="COMMIT" />
<input id="prepareConfirmText" type="text" placeholder="durable commit executor pending" disabled />
</div>
</div>
<div class="button-row">
<button class="secondary" id="previewPrepare">Preview Prepare</button>
<button class="danger" id="commitPrepare">Commit Selected</button>
<button class="danger" id="commitPrepare" data-release-disabled="true" disabled title="No content-bound durable commit executor is available yet.">Commit Selected</button>
</div>
<div class="actions" id="prepareOutput"></div>
</div>
@@ -727,7 +731,7 @@
<small><span id="tagStatus">idle</span> · plan <span id="planStatus">idle</span></small>
</div>
<div class="details">
<p class="hint">Creates an annotated source tag at the selected clean HEAD. Publish pushes the branch and tag atomically; existing release tags are never moved.</p>
<p class="hint">Preview is available here. Tag creation and publication use the matching durable run steps, bound to the frozen HEAD, branch, clean worktree, and registered origin.</p>
<div class="form-grid">
<div>
<label for="tagMessage">Tag message</label>
@@ -735,13 +739,13 @@
</div>
<div>
<label for="tagConfirmText">Confirm</label>
<input id="tagConfirmText" type="text" placeholder="TAG or PUBLISH" />
<input id="tagConfirmText" type="text" placeholder="use durable run controls" disabled />
</div>
</div>
<div class="button-row">
<button class="secondary" id="previewReleaseTags">Preview Tag + Publish</button>
<button id="createReleaseTags">Create Tags</button>
<button class="danger" id="publishReleaseTags">Publish Tags</button>
<button id="createReleaseTags" data-release-disabled="true" disabled title="Use the receipt-bound tag step in Durable Run State.">Create Tags</button>
<button class="danger" id="publishReleaseTags" data-release-disabled="true" disabled title="Use the receipt-bound push step in Durable Run State.">Publish Tags</button>
</div>
<div class="actions" id="tagOutput"></div>
</div>
@@ -761,18 +765,18 @@
</div>
<div>
<label for="candidateDir">Candidate dir</label>
<input id="candidateDir" type="text" placeholder="runtime/release-candidates/..." />
<input id="candidateDir" type="text" placeholder="private state/release-candidates/candidate-..." />
</div>
<div>
<label for="confirmText">Confirm</label>
<input id="confirmText" type="text" placeholder="APPLY or PUSH" />
<input id="confirmText" type="text" placeholder="use durable run controls" disabled />
</div>
</div>
<div class="button-row">
<button id="generateCandidate">Generate</button>
<button id="generateCandidate" disabled data-release-disabled="true" title="Use the receipt-bound generator in Durable Run State.">Generate</button>
<button class="secondary" id="previewPublish">Preview</button>
<button class="secondary" id="applyPublish">Apply + Website Tag</button>
<button class="danger" id="pushPublish">Push Website Release</button>
<button class="secondary" id="applyPublish" data-release-disabled="true" disabled title="Use the receipt-bound publication step in Durable Run State.">Apply + Website Tag</button>
<button class="danger" id="pushPublish" data-release-disabled="true" disabled title="Use the receipt-bound publication step in Durable Run State.">Push Website Release</button>
</div>
<div class="actions" id="workflowOutput"></div>
</div>
@@ -884,6 +888,7 @@
commitPrepare: document.getElementById("commitPrepare"),
buildSelectedPlan: document.getElementById("buildSelectedPlan"),
releaseRun: document.getElementById("releaseRun"),
loadOlderReleaseRuns: document.getElementById("loadOlderReleaseRuns"),
runStatus: document.getElementById("runStatus"),
runOutput: document.getElementById("runOutput"),
createReleaseRun: document.getElementById("createReleaseRun"),
@@ -907,6 +912,7 @@
manualRepo: "",
currentRun: null,
runs: [],
runNextCursor: null,
runStoreAvailable: true,
};
const pendingReleaseRunKey = "govoplanReleaseConsolePendingRunCreate";
@@ -928,6 +934,7 @@
elements.createReleaseRun.addEventListener("click", createReleaseRun);
elements.resumeReleaseRun.addEventListener("click", resumeReleaseRun);
elements.releaseRun.addEventListener("change", () => loadReleaseRun(elements.releaseRun.value));
elements.loadOlderReleaseRuns.addEventListener("click", loadOlderReleaseRuns);
elements.previewReleaseTags.addEventListener("click", () => releaseSelectedTags({ apply: false, push: true }));
elements.createReleaseTags.addEventListener("click", () => releaseSelectedTags({ apply: true, push: false }));
elements.publishReleaseTags.addEventListener("click", () => releaseSelectedTags({ apply: true, push: true }));
@@ -943,9 +950,11 @@
});
function api(path) {
const query = new URLSearchParams();
const separator = path.indexOf("?");
const basePath = separator === -1 ? path : path.slice(0, separator);
const query = new URLSearchParams(separator === -1 ? "" : path.slice(separator + 1));
if (elements.channel.value.trim()) query.set("channel", elements.channel.value.trim());
if (path === "/api/selective-plan") {
if (basePath === "/api/selective-plan") {
const selected = selectedRepoVersions();
const repoNames = Object.keys(selected);
if (repoNames.length) {
@@ -957,9 +966,9 @@
else query.set("public_catalog", "false");
if (elements.online.checked) query.set("remote_tags", "true");
if (elements.migrations.checked) query.set("include_migrations", "true");
if (elements.website.checked && path === "/api/dashboard") query.set("include_website", "true");
if (elements.website.checked && basePath === "/api/dashboard") query.set("include_website", "true");
const suffix = query.toString() ? `?${query}` : "";
return fetch(`${path}${suffix}`, {
return fetch(`${basePath}${suffix}`, {
headers: token ? { "X-Release-Console-Token": token } : {},
}).then((response) => {
if (!response.ok) {
@@ -1005,7 +1014,7 @@
if (runsResult.error) {
renderReleaseRunStoreUnavailable(runsResult.error);
} else {
renderReleaseRunList(runsResult.data?.runs || []);
renderReleaseRunList(runsResult.data?.runs || [], { nextCursor: runsResult.data?.next_cursor || null });
const pendingCreateResult = await recoverPendingReleaseRun();
const pendingCommandResult = await recoverPendingRunCommands();
const recoveryAttempted = pendingCreateResult !== null || pendingCommandResult !== null;
@@ -1039,18 +1048,23 @@
renderCatalog(data.catalog, data.target_version);
}
function renderReleaseRunList(runs) {
function renderReleaseRunList(runs, renderOptions = {}) {
releaseState.runStoreAvailable = true;
releaseState.runs = Array.isArray(runs) ? runs : [];
const incoming = Array.isArray(runs) ? runs : [];
releaseState.runs = renderOptions.append
? [...new Map([...releaseState.runs, ...incoming].map((run) => [run.run_id, run])).values()]
: incoming;
releaseState.runNextCursor = renderOptions.nextCursor || null;
elements.releaseRun.disabled = false;
elements.createReleaseRun.disabled = false;
elements.loadOlderReleaseRuns.disabled = !releaseState.runNextCursor;
const selectedId = releaseState.currentRun?.run_id || "";
const options = releaseState.runs.map((run) => {
const runOptions = releaseState.runs.map((run) => {
const repositories = run.repo_versions ? Object.keys(run.repo_versions).length : 0;
const label = `${run.run_id} · ${run.status || "unknown"} · ${repositories} repos`;
return `<option value="${escapeAttr(run.run_id)}" ${run.run_id === selectedId ? "selected" : ""}>${escapeHtml(label)}</option>`;
}).join("");
elements.releaseRun.innerHTML = `<option value="">No saved run selected</option>${options}`;
elements.releaseRun.innerHTML = `<option value="">No saved run selected</option>${runOptions}`;
if (selectedId && releaseState.runs.some((run) => run.run_id === selectedId)) {
elements.releaseRun.value = selectedId;
}
@@ -1060,9 +1074,11 @@
releaseState.runStoreAvailable = false;
releaseState.runs = [];
releaseState.currentRun = null;
releaseState.runNextCursor = null;
elements.releaseRun.innerHTML = `<option value="">Run storage unavailable</option>`;
elements.releaseRun.disabled = true;
elements.createReleaseRun.disabled = true;
elements.loadOlderReleaseRuns.disabled = true;
elements.resumeReleaseRun.disabled = true;
elements.runStatus.textContent = "unavailable";
elements.runOutput.innerHTML = `<div class="action"><h3>${pill("unavailable", "block")} Durable run storage cannot be read</h3><p>${escapeHtml(error.message)}</p><p class="action-meta">Dashboard and release previews remain available. Correct the private state-directory problem, then refresh.</p></div>`;
@@ -1157,13 +1173,31 @@
async function refreshReleaseRunListAfterCreate(run) {
try {
const runs = await api("/api/release-runs");
renderReleaseRunList(runs.runs || []);
renderReleaseRunList(runs.runs || [], { nextCursor: runs.next_cursor || null });
elements.releaseRun.value = run.run_id;
} catch (error) {
elements.runOutput.insertAdjacentHTML("afterbegin", `<div class="action"><h3>${pill("saved", "warn")} Run saved; list refresh unavailable</h3><p>${escapeHtml(error.message)}</p><p class="action-meta">The persisted run remains selected and can be reloaded after refresh.</p></div>`);
}
}
async function loadOlderReleaseRuns() {
const cursor = releaseState.runNextCursor;
if (!cursor) return;
setBusy([elements.loadOlderReleaseRuns], true);
try {
const page = await api(`/api/release-runs?cursor=${encodeURIComponent(cursor)}`);
renderReleaseRunList(page.runs || [], {
append: true,
nextCursor: page.next_cursor || null,
});
} catch (error) {
elements.runOutput.insertAdjacentHTML("afterbegin", `<div class="action"><h3>${pill("unavailable", "warn")} Older runs could not be loaded</h3><p>${escapeHtml(error.message)}</p></div>`);
} finally {
setBusy([elements.loadOlderReleaseRuns], false);
elements.loadOlderReleaseRuns.disabled = !releaseState.runNextCursor;
}
}
async function loadReleaseRun(runId, options = {}) {
if (!runId) {
releaseState.currentRun = null;
@@ -1219,8 +1253,15 @@
const stepHtml = steps.map((step) => {
const kind = step.state === "succeeded" ? "ok" : ["failed", "interrupted"].includes(step.state) ? "block" : step.available ? "ok" : "warn";
const retryReason = step.retry_available ? "Prepare this known failed or read-only interrupted step for an explicit retry." : (step.disabled_reason || "Retry is unavailable for this state.");
const executor = step.executor || {};
const confirmation = executor.confirmation || "";
const executeReason = executor.available ? (step.disabled_reason || "Execute this frozen plan step.") : "No bounded executor is available for this frozen step.";
const executorHtml = executor.available ? `<div class="form-grid" data-run-execute-step="${escapeAttr(step.id)}" data-run-execute-confirmation="${escapeAttr(confirmation)}" data-run-execute-available="${step.available ? "true" : "false"}">
${confirmation ? `<div><label>Confirmation</label><input type="text" data-run-execute-confirm placeholder="Type ${escapeAttr(confirmation)}" autocomplete="off" /></div>` : ""}
<div class="button-row">${executor.kind === "catalog_publish" ? `<button class="secondary" data-run-preview-submit ${step.available ? "" : "disabled"}>Preview receipt-bound candidate</button>` : ""}<button data-run-execute-submit ${step.available && !confirmation ? "" : "disabled"} title="${escapeAttr(executeReason)}">Execute Step</button></div>
</div>` : `<p class="action-meta">No durable executor: ${escapeHtml(step.status || "unsupported")}</p>`;
const reconciliationHtml = step.reconciliation_required ? `<div class="form-grid" data-run-reconcile-step="${escapeAttr(step.id)}">
<div><label>Observed external effect</label><select data-run-reconcile-outcome><option value="">Choose verified outcome</option><option value="effect_absent">Effect is absent — retry is safe</option><option value="effect_succeeded">Effect succeeded — advance the run</option><option value="unresolved">Still unresolved — keep blocked</option></select></div>
<div><label>Observed external effect</label><select data-run-reconcile-outcome><option value="">Choose verified outcome</option><option value="effect_absent">Effect is absent — retry is safe</option><option value="effect_succeeded">Effect succeeded — advance the run after server verification</option><option value="unresolved">Still unresolved — keep blocked</option></select></div>
<div><label>Confirmation</label><input type="text" data-run-reconcile-confirm placeholder="Type RECONCILE" autocomplete="off" /></div>
<div class="button-row"><button class="secondary" data-run-reconcile-submit disabled title="Verify local and remote state independently, choose the observed outcome, and type RECONCILE.">Record Reconciliation</button></div>
</div>` : "";
@@ -1228,6 +1269,8 @@
<div class="stage-title"><h3>${escapeHtml(`${step.order}. ${step.title || step.id}`)}</h3>${pill(step.state, kind)}</div>
<p>${escapeHtml(step.detail || "")}</p>
${step.result_code ? `<p class="action-meta">Result code: ${escapeHtml(step.result_code)}</p>` : ""}
${releaseReceiptHtml(step.result_receipt)}
${executorHtml}
<div class="button-row"><button class="secondary" data-run-retry-step="${escapeAttr(step.id)}" ${step.retry_available ? "" : "disabled"} title="${escapeAttr(retryReason)}">Prepare Retry</button></div>
${reconciliationHtml}
</div>`;
@@ -1235,7 +1278,7 @@
const eventHtml = events.length ? `<div class="action"><h3>Recent bounded state events</h3>${events.map((event) => `<p class="action-meta">${escapeHtml(`${event.at} · ${event.type}${event.step_id ? ` · ${event.step_id}` : ""}${event.result_code ? ` · ${event.result_code}` : ""}`)}</p>`).join("")}</div>` : "";
elements.runStatus.textContent = state.status || "unknown";
elements.resumeReleaseRun.disabled = state.status !== "running";
elements.runOutput.innerHTML = `<div class="action"><h3>${pill(state.status || "unknown", state.status === "blocked" ? "block" : state.status === "completed" ? "ok" : "warn")} ${escapeHtml(run.run_id)}</h3><p>Frozen plan ${escapeHtml((run.immutable?.digest || "").slice(0, 16))}… · updated ${escapeHtml(run.updated_at || "-")}</p><p class="action-meta">Executor controls remain separate and keep their existing confirmations.</p></div>${recommendationHtml}${stepHtml}${eventHtml}`;
elements.runOutput.innerHTML = `<div class="action"><h3>${pill(state.status || "unknown", state.status === "blocked" ? "block" : state.status === "completed" ? "ok" : "warn")} ${escapeHtml(run.run_id)}</h3><p>Frozen plan ${escapeHtml((run.immutable?.digest || "").slice(0, 16))}… · updated ${escapeHtml(run.updated_at || "-")}</p><p class="action-meta">Every supported mutation is claimed durably before the existing narrow executor is called.</p></div>${recommendationHtml}${stepHtml}${eventHtml}`;
for (const button of elements.runOutput.querySelectorAll("[data-run-retry-step]")) {
button.addEventListener("click", () => retryReleaseRunStep(button.dataset.runRetryStep));
}
@@ -1250,6 +1293,78 @@
confirmation.addEventListener("input", refresh);
submit.addEventListener("click", () => reconcileReleaseRunStep(controls.dataset.runReconcileStep, outcome.value, confirmation.value.trim(), submit));
}
for (const controls of elements.runOutput.querySelectorAll("[data-run-execute-step]")) {
const required = controls.dataset.runExecuteConfirmation || "";
const available = controls.dataset.runExecuteAvailable === "true";
const input = controls.querySelector("[data-run-execute-confirm]");
const submit = controls.querySelector("[data-run-execute-submit]");
const preview = controls.querySelector("[data-run-preview-submit]");
if (input) input.addEventListener("input", () => {
submit.disabled = !available || input.value.trim() !== required;
});
submit.addEventListener("click", () => executeReleaseRunStep(controls.dataset.runExecuteStep, required, submit));
if (preview) preview.addEventListener("click", () => previewReleaseRunStep(controls.dataset.runExecuteStep, preview));
}
}
async function executeReleaseRunStep(stepId, confirmation, button) {
const run = releaseState.currentRun;
if (!run || !stepId) return;
const commandKey = `execute:${run.run_id}:${stepId}:${confirmation || "NONE"}`;
const commandBody = {
confirm: confirmation,
remote: "origin",
signing_keys: elements.signingKey.value.trim() ? [elements.signingKey.value.trim()] : [],
};
const commandRequestId = inFlightRunCommandId(commandKey, "execute");
setBusy([button], true);
try {
const executed = await postJson(`/api/release-runs/${encodeURIComponent(run.run_id)}/steps/${encodeURIComponent(stepId)}/execute`, {
request_id: commandRequestId,
...commandBody,
});
clearInFlightRunCommand(commandKey, commandRequestId);
releaseState.currentRun = executed;
renderReleaseRun(executed);
const result = executed.execution_result || {};
elements.runOutput.insertAdjacentHTML("afterbegin", `<div class="action"><h3>${pill(result.status || "recorded", result.status === "failed" ? "block" : "ok")} Executor result</h3><p>${escapeHtml(result.detail || result.result_code || "The bounded result was recorded.")}</p></div>`);
} catch (error) {
if (deterministicRunCommandError(error)) clearInFlightRunCommand(commandKey, commandRequestId);
elements.runOutput.insertAdjacentHTML("afterbegin", `<div class="action"><h3>${pill("reconcile", "block")} Executor did not establish a safe result</h3><p>${escapeHtml(error.message)}</p></div>`);
} finally {
button.disabled = false;
}
}
async function previewReleaseRunStep(stepId, button) {
const run = releaseState.currentRun;
if (!run || !stepId) return;
setBusy([button], true);
try {
const result = await postJson(`/api/release-runs/${encodeURIComponent(run.run_id)}/steps/${encodeURIComponent(stepId)}/preview`, {
remote: "origin",
});
elements.runOutput.insertAdjacentHTML("afterbegin", `<div class="action"><h3>${pill(result.status || "preview", result.status === "blocked" ? "block" : "ok")} Receipt-bound publication preview</h3><p>${escapeHtml((result.notes || []).join("; ") || "Preview completed without changing run state.")}</p></div>`);
} catch (error) {
elements.runOutput.insertAdjacentHTML("afterbegin", `<div class="action"><h3>${pill("blocked", "block")} Preview failed</h3><p>${escapeHtml(error.message)}</p></div>`);
} finally {
setBusy([button], false);
}
}
function releaseReceiptHtml(receipt) {
if (!receipt || typeof receipt !== "object") return "";
if (receipt.kind === "catalog_candidate") {
return `<p class="action-meta">Candidate ${escapeHtml(receipt.candidate_id || "-")} · ${escapeHtml((receipt.catalog_sha256 || "").slice(0, 16))}…</p>`;
}
if (receipt.kind === "catalog_publication") {
return `<p class="action-meta">Published ${escapeHtml(receipt.candidate_id || "-")} · commit ${escapeHtml((receipt.publication_commit_sha || "").slice(0, 12))} · tag ${escapeHtml(receipt.tag_name || "-")} (${escapeHtml((receipt.publication_tag_object_sha || "").slice(0, 12))}) · ${escapeHtml(receipt.remote || "origin")}/${escapeHtml(receipt.branch || "-")}</p>`;
}
if (receipt.kind === "repository_state") {
const tag = receipt.tag_object ? ` · tag ${(receipt.tag_object || "").slice(0, 12)}` : "";
return `<p class="action-meta">Bound ${escapeHtml(receipt.repo || "-")} · HEAD ${escapeHtml((receipt.head || "").slice(0, 12))}${escapeHtml(tag)} · ${receipt.worktree_clean ? "clean" : "dirty"}</p>`;
}
return "";
}
async function resumeReleaseRun() {
@@ -1313,7 +1428,17 @@
function readInFlightRunCommands() {
try {
const commands = JSON.parse(sessionStorage.getItem(pendingRunCommandsKey) || "{}");
return commands && typeof commands === "object" && !Array.isArray(commands) ? commands : {};
if (!commands || typeof commands !== "object" || Array.isArray(commands)) return {};
const normalized = {};
for (const [commandKey, savedCommand] of Object.entries(commands)) {
const requestIdToReplay = typeof savedCommand === "string" ? savedCommand : savedCommand?.request_id;
if (typeof requestIdToReplay === "string") normalized[commandKey] = requestIdToReplay;
}
if (JSON.stringify(normalized) !== JSON.stringify(commands)) {
if (Object.keys(normalized).length) sessionStorage.setItem(pendingRunCommandsKey, JSON.stringify(normalized));
else sessionStorage.removeItem(pendingRunCommandsKey);
}
return normalized;
} catch (_error) {
sessionStorage.removeItem(pendingRunCommandsKey);
return {};
@@ -1354,6 +1479,19 @@
body: { request_id: requestIdToReplay, outcome, confirm: "RECONCILE" },
};
}
if (commandKey.startsWith("execute:")) {
const remainder = commandKey.slice("execute:".length);
const runSeparator = remainder.indexOf(":");
const confirmationSeparator = remainder.lastIndexOf(":");
const runId = remainder.slice(0, runSeparator);
const stepId = remainder.slice(runSeparator + 1, confirmationSeparator);
const confirmation = remainder.slice(confirmationSeparator + 1);
if (runSeparator < 1 || confirmationSeparator <= runSeparator + 1 || !stepId) return null;
return {
path: `/api/release-runs/${encodeURIComponent(runId)}/steps/${encodeURIComponent(stepId)}/execute`,
body: { request_id: requestIdToReplay, confirm: confirmation === "NONE" ? "" : confirmation },
};
}
return null;
}
@@ -1365,7 +1503,12 @@
let recoveredRun = null;
let pendingError = null;
let rejectedError = null;
for (const [commandKey, requestIdToReplay] of entries) {
for (const [commandKey, savedCommand] of entries) {
const requestIdToReplay = typeof savedCommand === "string" ? savedCommand : savedCommand?.request_id;
if (typeof requestIdToReplay !== "string") {
delete commands[commandKey];
continue;
}
const command = pendingRunCommandRequest(commandKey, requestIdToReplay);
if (!command) {
clearInFlightRunCommand(commandKey, requestIdToReplay);
@@ -1405,6 +1548,7 @@
function inFlightRunCommandId(commandKey, prefix) {
const commands = readInFlightRunCommands();
if (typeof commands[commandKey] === "string") return commands[commandKey];
if (typeof commands[commandKey]?.request_id === "string") return commands[commandKey].request_id;
const identifier = requestId(prefix);
commands[commandKey] = identifier;
sessionStorage.setItem(pendingRunCommandsKey, JSON.stringify(commands));
@@ -1413,7 +1557,8 @@
function clearInFlightRunCommand(commandKey, requestIdToClear) {
const commands = readInFlightRunCommands();
if (commands[commandKey] !== requestIdToClear) return;
const storedRequestId = typeof commands[commandKey] === "string" ? commands[commandKey] : commands[commandKey]?.request_id;
if (storedRequestId !== requestIdToClear) return;
delete commands[commandKey];
if (Object.keys(commands).length) sessionStorage.setItem(pendingRunCommandsKey, JSON.stringify(commands));
else sessionStorage.removeItem(pendingRunCommandsKey);
@@ -1460,14 +1605,14 @@
tagStatus,
releaseBlockers ? "block" : selected ? "ok" : "warn",
selected
? `${selected} selected; use Preview Tag + Publish, then Create Tags or Publish Tags after reviewing the source-release preflight.`
? `${selected} selected; preview here, then create a durable run and execute its enabled tag/push steps.`
: "Select repositories, set target versions, then preview the source release tags."
),
workflowStage(
"4. Sign and publish website catalog",
signedStatus,
hasCandidate ? "ok" : "warn",
hasCandidate ? `Candidate: ${elements.candidateDir.value.trim()}` : "After every selected source tag is remotely published, generate a signed candidate, preview it, then apply/tag/push the website publication."
hasCandidate ? `Preview candidate: ${elements.candidateDir.value.trim()}` : "After every selected source tag is remotely published, use a durable run to generate and publish its receipt-bound candidate."
),
workflowStage(
"5. Maintain install catalog",
@@ -2315,7 +2460,7 @@
function setBusy(buttons, busy) {
for (const button of buttons) {
button.disabled = busy;
button.disabled = busy || button.dataset.releaseDisabled === "true";
}
}

View File

@@ -3,35 +3,138 @@ from __future__ import annotations
import argparse
import json
import os
import re
import subprocess
import sys
from pathlib import Path
from urllib.parse import urlsplit
ROOT = Path(__file__).resolve().parents[2]
REGISTERED_TRANSPORT = "registered"
PUBLIC_HTTPS_TRANSPORT = "public-https"
GITEA_SSH_PREFIX = "git@git.add-ideas.de:"
GITEA_HTTPS_PREFIX = "https://git.add-ideas.de/"
GITEA_REPOSITORY_PATH = re.compile(
r"(?:GovOPlaN|add-ideas)/[a-z0-9][a-z0-9-]*[.]git"
)
def parse_args() -> argparse.Namespace:
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Clone GovOPlaN repositories listed in repositories.json.")
parser.add_argument("--check", action="store_true", help="Only report missing repositories.")
parser.add_argument("--parent", type=Path, help="Override checkout parent directory.")
return parser.parse_args()
parser.add_argument(
"--repo",
action="append",
default=[],
metavar="NAME",
help="Clone only this registered repository; repeat for more than one.",
)
parser.add_argument(
"--exclude-repo",
action="append",
default=[],
metavar="NAME",
help="Exclude this registered repository; repeat for more than one.",
)
parser.add_argument(
"--transport",
choices=(REGISTERED_TRANSPORT, PUBLIC_HTTPS_TRANSPORT),
default=REGISTERED_TRANSPORT,
help=(
"Clone registered remotes unchanged, or convert public "
"git.add-ideas.de SSH remotes to anonymous HTTPS."
),
)
return parser.parse_args(argv)
def main() -> int:
args = parse_args()
manifest = json.loads((ROOT / "repositories.json").read_text())
def clone_remote(remote: str, *, transport: str) -> str:
if transport == REGISTERED_TRANSPORT:
return remote
if transport != PUBLIC_HTTPS_TRANSPORT:
raise ValueError(f"unsupported repository transport: {transport}")
if remote.startswith(GITEA_SSH_PREFIX):
repository_path = remote.removeprefix(GITEA_SSH_PREFIX)
else:
parsed = urlsplit(remote)
if (
parsed.scheme not in {"https", "ssh"}
or parsed.hostname != "git.add-ideas.de"
or parsed.port is not None
or parsed.password is not None
or parsed.query
or parsed.fragment
):
raise ValueError(
"public HTTPS bootstrap only accepts registered "
"git.add-ideas.de remotes"
)
if parsed.scheme == "https" and parsed.username is not None:
raise ValueError("public HTTPS bootstrap does not accept credentials")
if parsed.scheme == "ssh" and parsed.username != "git":
raise ValueError("registered Gitea SSH remotes must use the git account")
repository_path = parsed.path.removeprefix("/")
if GITEA_REPOSITORY_PATH.fullmatch(repository_path) is None:
raise ValueError("registered Gitea repository path is malformed")
return GITEA_HTTPS_PREFIX + repository_path
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
manifest = json.loads((ROOT / "repositories.json").read_text(encoding="utf-8"))
parent = args.parent or Path(manifest["default_parent"])
missing: list[dict[str, str]] = []
missing: list[tuple[dict[str, str], Path, str]] = []
repositories = manifest["repositories"]
names = [entry["name"] for entry in repositories]
if len(names) != len(set(names)):
raise ValueError("repository registry contains duplicate names")
requested = set(args.repo)
excluded = set(args.exclude_repo)
unknown = (requested | excluded) - set(names)
if unknown:
raise ValueError(f"unknown registered repositories: {', '.join(sorted(unknown))}")
overlap = requested & excluded
if overlap:
raise ValueError(
f"repositories cannot be selected and excluded: {', '.join(sorted(overlap))}"
)
for entry in manifest["repositories"]:
for entry in repositories:
if (requested and entry["name"] not in requested) or entry["name"] in excluded:
continue
repo = parent / entry["path"]
if repo.exists():
continue
missing.append(entry)
transport = entry.get("bootstrap_transport", args.transport)
remote = clone_remote(entry["remote"], transport=transport)
missing.append((entry, repo, remote))
environment = os.environ.copy()
environment.update(
{
"GIT_ASKPASS": "/bin/false",
"GIT_TERMINAL_PROMPT": "0",
}
)
for entry, repo, remote in missing:
print(f"missing: {entry['name']} -> {repo}")
if not args.check:
subprocess.run(["git", "clone", entry["remote"], str(repo)], check=True)
subprocess.run(
[
"git",
"-c",
"credential.helper=",
"clone",
"--",
remote,
str(repo),
],
check=True,
env=environment,
)
if args.check:
return 1 if missing else 0

View File

@@ -12,6 +12,7 @@ from pathlib import Path
import shlex
import subprocess
import sys
import tomllib
from typing import Any
META_ROOT = Path(__file__).resolve().parents[2]
@@ -49,6 +50,23 @@ class InstallPlan:
warnings: tuple[str, ...] = ()
@dataclass(frozen=True, slots=True)
class LocalInstallationExpectation:
requirement: RequirementEntry
distribution: str
entry_points: tuple[tuple[str, str, str], ...]
@dataclass(frozen=True, slots=True)
class EnvironmentValidation:
stale_requirements: tuple[RequirementEntry, ...] = ()
issues: tuple[str, ...] = ()
@property
def current(self) -> bool:
return not self.stale_requirements
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
@@ -85,8 +103,9 @@ def main() -> int:
fingerprint = build_fingerprint(requirements=requirements, python=python, local_requirements=local_requirements)
previous = load_stamp(stamp)
current = previous.get("digest") == fingerprint["digest"]
environment = validate_local_installations(python, local_requirements)
if current and not args.force:
if current and environment.current and not args.force:
payload = stamp_payload(
requirements=requirements,
python=python,
@@ -107,20 +126,31 @@ def main() -> int:
print(f"Python environment is current for {requirements.name}.")
return 0
print(f"Python environment metadata is stale for {requirements.name}.")
if current and not environment.current:
print(f"Python environment metadata is current for {requirements.name}, but installed packages need repair.")
else:
print(f"Python environment metadata is stale for {requirements.name}.")
print(f"Tracked inputs: {len(fingerprint['inputs'])}")
for issue in environment.issues:
print(f"warning: {issue}", file=sys.stderr)
if args.check:
return 1
plan = build_install_plan(
previous=previous,
fingerprint=fingerprint,
requirements=requirements,
python=python,
local_requirements=local_requirements,
force=args.force,
)
if current and environment.stale_requirements and not args.force:
plan = build_environment_repair_plan(
python=python,
stale_requirements=environment.stale_requirements,
)
else:
plan = build_install_plan(
previous=previous,
fingerprint=fingerprint,
requirements=requirements,
python=python,
local_requirements=local_requirements,
force=args.force,
)
print(f"{plan.mode}: {plan.reason}")
for warning in plan.warnings:
@@ -141,6 +171,13 @@ def main() -> int:
for command in commands:
subprocess.run(command, check=True)
repaired_environment = validate_local_installations(python, local_requirements)
if not repaired_environment.current:
for issue in repaired_environment.issues:
print(f"error: {issue}", file=sys.stderr)
print("Python environment synchronization did not install all local package metadata.", file=sys.stderr)
return 2
write_stamp(
stamp,
stamp_payload(
@@ -297,6 +334,127 @@ def build_install_plan(
)
def build_environment_repair_plan(
*,
python: str,
stale_requirements: tuple[RequirementEntry, ...],
) -> InstallPlan:
command = (
python,
"-m",
"pip",
"install",
*(
argument
for requirement in stale_requirements
for argument in requirement.install_args
),
)
return InstallPlan(
"Selective Python environment repair",
f"reinstalling {len(stale_requirements)} missing or stale local distribution(s).",
(command,),
)
def validate_local_installations(
python: str,
local_requirements: tuple[RequirementEntry, ...],
) -> EnvironmentValidation:
expectations = local_installation_expectations(local_requirements)
if not expectations:
return EnvironmentValidation()
probe_input = [
{
"key": expectation.requirement.key,
"distribution": expectation.distribution,
"entry_points": [
{"group": group, "name": name, "value": value}
for group, name, value in expectation.entry_points
],
}
for expectation in expectations
]
try:
result = subprocess.run(
(python, "-c", _LOCAL_INSTALLATION_PROBE),
check=False,
text=True,
input=json.dumps(probe_input),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=20,
)
except (OSError, subprocess.TimeoutExpired) as exc:
return EnvironmentValidation(
stale_requirements=tuple(item.requirement for item in expectations),
issues=(f"could not inspect installed local distributions ({type(exc).__name__})",),
)
if result.returncode != 0:
detail = result.stderr.strip() or result.stdout.strip() or "installation probe failed"
return EnvironmentValidation(
stale_requirements=tuple(item.requirement for item in expectations),
issues=(f"could not inspect installed local distributions: {detail}",),
)
try:
payload = json.loads(result.stdout)
except json.JSONDecodeError:
return EnvironmentValidation(
stale_requirements=tuple(item.requirement for item in expectations),
issues=("installed local distribution probe returned invalid JSON",),
)
raw_issues = payload.get("issues") if isinstance(payload, dict) else None
issues_by_key = raw_issues if isinstance(raw_issues, dict) else {}
stale = tuple(
expectation.requirement
for expectation in expectations
if expectation.requirement.key in issues_by_key
)
issues = tuple(
f"{expectation.distribution}: {issues_by_key[expectation.requirement.key]}"
for expectation in expectations
if expectation.requirement.key in issues_by_key
)
return EnvironmentValidation(stale_requirements=stale, issues=issues)
def local_installation_expectations(
local_requirements: tuple[RequirementEntry, ...],
) -> tuple[LocalInstallationExpectation, ...]:
expectations: list[LocalInstallationExpectation] = []
for requirement in local_requirements:
if requirement.pyproject is None:
continue
with Path(requirement.pyproject).open("rb") as handle:
payload = tomllib.load(handle)
project = payload.get("project")
if not isinstance(project, dict):
continue
distribution = project.get("name")
if not isinstance(distribution, str) or not distribution.strip():
continue
declared_entry_points: list[tuple[str, str, str]] = []
entry_point_groups = project.get("entry-points")
if isinstance(entry_point_groups, dict):
for group, entries in entry_point_groups.items():
if not isinstance(group, str) or not isinstance(entries, dict):
continue
for name, value in entries.items():
if isinstance(name, str) and isinstance(value, str):
declared_entry_points.append((group, name, value))
expectations.append(
LocalInstallationExpectation(
requirement=requirement,
distribution=distribution,
entry_points=tuple(sorted(declared_entry_points)),
)
)
return tuple(expectations)
def requirement_delta_plan(previous_entries: object, current_entries: tuple[RequirementEntry, ...]) -> InstallPlan | None:
parsed_previous = previous_requirement_entries(previous_entries)
if parsed_previous is None:
@@ -481,5 +639,35 @@ def format_command(command: tuple[str, ...]) -> str:
return " ".join(shlex.quote(item) for item in command)
_LOCAL_INSTALLATION_PROBE = """
import importlib.metadata
import json
import sys
expectations = json.load(sys.stdin)
issues = {}
for expectation in expectations:
key = expectation["key"]
distribution_name = expectation["distribution"]
try:
distribution = importlib.metadata.distribution(distribution_name)
except importlib.metadata.PackageNotFoundError:
issues[key] = "distribution is not installed"
continue
installed = {
(entry_point.group, entry_point.name): entry_point.value
for entry_point in distribution.entry_points
}
mismatches = []
for entry_point in expectation["entry_points"]:
identity = (entry_point["group"], entry_point["name"])
if installed.get(identity) != entry_point["value"]:
mismatches.append(f"{identity[0]}:{identity[1]}")
if mismatches:
issues[key] = "entry-point metadata is missing or stale: " + ", ".join(mismatches)
print(json.dumps({"issues": issues}, sort_keys=True))
"""
if __name__ == "__main__":
raise SystemExit(main())