25 Commits
Author SHA1 Message Date
zemion 14b19fbead chore(release): coordinate integrity and performance source updates
Dependency Audit / dependency-audit (push) Successful in 2m9s
Deployment Installer / deployment-installer (push) Successful in 8s
Security Audit / security-audit (push) Successful in 14m36s
Developer Meta-package Release / publish-package (push) Successful in 10s
Release v0.1.46. Coordinated integrity review: GovOPlaN/govoplan-core#298.
2026-09-08 12:53:09 +02:00
zemion 845dcbafdb chore(release): coordinate integrity and performance source updates
Release v0.1.46. Coordinated integrity review: GovOPlaN/govoplan-core#298.
2026-09-08 12:36:36 +02:00
zemion 58d320d9b3 Harden source release preparation and record verified security follow-up
Dependency Audit / dependency-audit (push) Successful in 1m51s
Deployment Installer / deployment-installer (push) Successful in 8s
Security Audit / security-audit (push) Successful in 12m32s
2026-09-08 08:04:12 +02:00
zemion 9554657bb5 fix(release): preserve failed installer retry status
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 7s
Security Audit / security-audit (push) Successful in 11m21s
Cover all retry call sites with isolated regressions in focused checks and installer CI. Add EN/DE operating guidance and record the unreleased Xrechnung and installer audit follow-ups without changing immutable release artifacts.

Refs #54
2026-09-08 05:36:10 +02:00
zemion 88b685ff5e test(release): follow relocated operations runbook
Developer Meta-package Release / publish-package (push) Successful in 9s
Dependency Audit / dependency-audit (push) Successful in 1m41s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 14m12s
2026-09-08 03:02:31 +02:00
zemion be57a1823a fix(release): support validated per-run SSH address family 2026-09-08 02:50:59 +02:00
zemion 6bcb75f577 feat(release): build verified full-registry catalog candidates 2026-09-08 02:26:19 +02:00
zemion 32fe4b7238 chore(release): document hardening and verify complete source package composition 2026-09-08 02:06:35 +02:00
zemion 6a8f53b87d fix(release): prepare aligned 0.1.45 composition and guarded candidate sequencing 2026-09-08 01:52:08 +02:00
zemion 1cec4ee1d8 chore: compose governed Cases access
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m23s
2026-08-24 20:26:44 +02:00
zemion 29d03aa2ca chore: compose purpose-bound Records access
Dependency Audit / dependency-audit (push) Successful in 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m45s
2026-08-24 19:28:37 +02:00
zemion 6fb928d6cf release: publish stable product destinations
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m26s
Developer Meta-package Release / publish-package (push) Successful in 9s
2026-08-24 18:30:01 +02:00
zemion 6edaaadf37 release: align governed tenant erasure
Dependency Audit / dependency-audit (push) Successful in 1m44s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m4s
Developer Meta-package Release / publish-package (push) Successful in 10s
2026-08-24 16:33:16 +02:00
zemion 0b171fbdd4 feat: gate infrastructure changes on provider inventory
Security Audit / security-audit (push) Failing after 12m3s
Developer Meta-package Release / publish-package (push) Successful in 9s
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 6s
2026-08-24 15:18:38 +02:00
zemion ed6790c057 Record resident permit browser evidence
Dependency Audit / dependency-audit (push) Successful in 1m41s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m6s
Developer Meta-package Release / publish-package (push) Successful in 9s
2026-08-24 14:03:57 +02:00
zemion 3766e26377 feat: publish stable product surface composition
Dependency Audit / dependency-audit (push) Successful in 1m43s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m29s
Developer Meta-package Release / publish-package (push) Successful in 10s
2026-08-24 13:41:00 +02:00
zemion 6c2b36af0f feat(inventory): enforce high-risk contextual help
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 12m18s
Developer Meta-package Release / publish-package (push) Successful in 10s
Dependency Audit / dependency-audit (push) Successful in 1m40s
2026-08-24 11:40:21 +02:00
zemion 3f75ca8e48 chore: compose German documentation releases
Dependency Audit / dependency-audit (push) Successful in 1m37s
Deployment Installer / deployment-installer (push) Successful in 5s
Security Audit / security-audit (push) Failing after 11m47s
2026-08-24 01:47:06 +02:00
zemion a886a9b3de chore(release): compose complete German coverage
Deployment Installer / deployment-installer (push) Successful in 6s
Dependency Audit / dependency-audit (push) Successful in 1m48s
Security Audit / security-audit (push) Failing after 12m0s
2026-08-23 21:31:07 +02:00
zemion fe83290d56 chore(release): compose expanded German coverage
Dependency Audit / dependency-audit (push) Successful in 1m48s
Deployment Installer / deployment-installer (push) Successful in 7s
Security Audit / security-audit (push) Failing after 12m5s
2026-08-23 20:51:28 +02:00
zemion c50f699399 chore(release): compose German reference coverage
Dependency Audit / dependency-audit (push) Successful in 1m45s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 11m55s
2026-08-23 19:57:05 +02:00
zemion 59b45a0829 chore(release): compose autonomous integration contracts
Dependency Audit / dependency-audit (push) Successful in 1m48s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 11m57s
2026-08-23 18:14:11 +02:00
zemion 861abcc573 chore(release): compose integration foundations
Dependency Audit / dependency-audit (push) Successful in 1m46s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 11m53s
2026-08-23 11:21:04 +02:00
zemion 5e995fed88 chore(release): compose German documentation batch
Dependency Audit / dependency-audit (push) Successful in 1m37s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 11m4s
2026-08-23 02:14:42 +02:00
zemion 41ca242004 chore(release): compose Reporting and Search 0.1.19
Dependency Audit / dependency-audit (push) Successful in 1m41s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Failing after 11m51s
2026-08-22 21:14:24 +02:00
78 changed files with 8497 additions and 345 deletions
@@ -23,6 +23,9 @@ jobs:
- name: Test declarative deployment bundle - name: Test declarative deployment bundle
working-directory: govoplan working-directory: govoplan
run: python -m unittest -v tests.test_deployment_installer run: python -m unittest -v tests.test_deployment_installer
- name: Test WebUI installer retry failures
working-directory: govoplan
run: python -m unittest -v tests.test_webui_release_dependency_retries
- name: Build single-file deployer artifact - name: Build single-file deployer artifact
working-directory: govoplan working-directory: govoplan
run: | run: |
@@ -72,6 +72,37 @@ Each WebUI module should be able to announce:
The contract references surfaces. It does not permit Core or a product package The contract references surfaces. It does not permit Core or a product package
to import their implementation. to import their implementation.
The versioned `product_surfaces` slice is implemented in Core. It
binds a stable product identity and entry path to one or more owner routes,
View surfaces, presentations, capabilities, search sources, help contexts and
documentation topics. It also carries standard unavailable/degraded
explanations and migration aliases. Mail and Postbox contribute the first
shared identity, `communication.messages`: `/messages` and the migration alias
`/inbox` select the first currently authorized, View-visible owner while the
underlying `/mail` and `/postbox` deep links, custody and permissions remain
unchanged. Tasks, Calendar and Files contribute the corresponding single-owner
identities:
| Product identity | Stable destination | Compatible owner route |
| --- | --- | --- |
| Work | `/work` | `/tasks` |
| Calendar | `/agenda` | `/calendar` |
| Messages | `/messages` (`/inbox` alias) | `/mail`, `/postbox` |
| Files | `/documents` | `/files` |
Core replaces those owner entries in the ordinary rail with the stable product
destinations. A collapsed **All available tools** catalogue retains every
authorized technical owner route independently of View focus; unauthorized
entries are never disclosed. The original deep links remain valid, and all
contributing owner paths keep the corresponding product entry active. Alias
resolution emits a bounded client telemetry event before the redirect.
Core's `ProductAvailabilityState` is the shared presentation primitive for
authorization, Policy, configuration, disabled, missing-capability, offline and
provider-degraded states. Product language is primary; exact module,
capability, provider and correlation provenance is available only in an
expandable technical section.
## Navigation Model ## Navigation Model
The default shell should prioritize: The default shell should prioritize:
@@ -88,10 +119,11 @@ People and Responsibility. They are configurable system/tenant defaults and
Views projections, not hard-coded repository groups. Empty areas disappear; Views projections, not hard-coded repository groups. Empty areas disappear;
single-destination areas may link directly; familiar tools may remain pinned. single-destination areas may link directly; familiar tools may remain pinned.
The complete permission-derived module rail remains available as **All The complete permission-derived module rail is available as the collapsed
available tools**. Its ability to scroll is useful and is not itself the **All available tools** escape. It is deliberately independent of the active
product defect. The defect is requiring people to infer a task or outcome from View while still enforcing authorization. Its ability to scroll is useful and
repository topology. is not itself the product defect. The defect is requiring people to infer a
task or outcome from repository topology.
Task-local Work, Calendar, Messages and Files tools may be contributed to the Task-local Work, Calendar, Messages and Files tools may be contributed to the
optional `govoplan-quick-access` rail. Messages composes Mail, Postbox and optional `govoplan-quick-access` rail. Messages composes Mail, Postbox and
@@ -108,6 +140,12 @@ sections, commands, widgets, and fields. A view must not grant a permission or
change data semantics. Policy can force, allow, or prohibit a surface at system, change data semantics. Policy can force, allow, or prohibit a surface at system,
tenant, group, or user scope. tenant, group, or user scope.
Core browser conformance exercises the German Anwohnerparkausweis reference
context with Work, Calendar, Messages and Files entries, verifies that package
owner labels are absent from the primary rail, expands the technical catalogue,
and runs WCAG 2 A/AA checks over the result. Unit permutations cover two-owner,
one-owner, unauthorized-owner and focused-View compositions.
## Error And Provenance Language ## Error And Provenance Language
Normal errors answer: Normal errors answer:
@@ -132,9 +170,9 @@ first rail slice.
### Slice 1: inventory and aliases ### Slice 1: inventory and aliases
- classify every route, navigation item, widget, setting, search object, and - continue classifying every route, navigation item, widget, setting, search object, and
help context by product area and object type; help context by product area and object type;
- add product aliases without removing existing deep links; - extend the implemented product-surface aliases without removing existing deep links;
- flag raw module IDs in ordinary-user labels and errors. - flag raw module IDs in ordinary-user labels and errors.
### Slice 2: work-first shell ### Slice 2: work-first shell
@@ -39,16 +39,24 @@ The first production-shaped slice is implemented:
order and optional labels. Scoped Views therefore configure product order and optional labels. Scoped Views therefore configure product
presentation for system, tenant, group, user and Workflow contexts; presentation for system, tenant, group, user and Workflow contexts;
- the expanded left rail groups classified destinations while retaining - the expanded left rail groups classified destinations while retaining
Dashboard and every authorized unclassified destination under More tools. Dashboard and every authorized unclassified destination under More tools;
- Core promotes Work (`/work`), Calendar (`/agenda`), Messages (`/messages`)
and Files (`/documents`) into stable primary destinations and collapses the
compatible owner routes under **All available tools**;
- **All available tools** is permission-derived but independent of the active
View, providing a deliberate escape without granting access or discarding
the original `/tasks`, `/calendar`, `/mail`, `/postbox` and `/files` links.
The baseline classification is now manifest-declared for every ordinary The baseline classification and the four initial stable destinations are now
user-facing module and enforced by the workspace manifest check. A separately manifest-declared. The area classification covers every ordinary user-facing
module and is enforced by the workspace manifest check. A separately
versioned launch-context contract carries bounded active-object, acting, versioned launch-context contract carries bounded active-object, acting,
temporal, View and return references into full-page Quick Access fallbacks; temporal, View and return references into full-page Quick Access fallbacks;
Cases publishes the first active-object reference. The remaining rollout is to Cases publishes the first active-object reference. The remaining rollout is to
add useful bounded tools and active-object publishers only where a maintained add useful bounded tools and active-object publishers only where a maintained
journey benefits, and to extend browser evidence to a pinned reference journey benefits. The pinned German Anwohnerparkausweis browser composition
composition. Authorized global and technical routes remain visible through verifies stable product labels, technical escape, keyboard access and WCAG
conformance. Authorized global and technical routes remain visible through
their dedicated shell entry or **All available tools**. their dedicated shell entry or **All available tools**.
## Quick Access Boundary ## Quick Access Boundary
@@ -166,9 +174,10 @@ areas, and users may personalize them within Policy ceilings. An empty area is
omitted. An area with one destination may open it directly. A multi-destination omitted. An area with one destination may open it directly. A multi-destination
area provides a useful work/recent/action surface rather than another menu. area provides a useful work/recent/action surface rather than another menu.
Familiar product nouns such as Calendar, Mail or Files may remain directly Familiar product nouns such as Calendar or Files remain direct product
pinned. The objective is not to hide every module name; it is to prevent destinations. The objective is not to hide every implementation name from
repository topology from determining a person's workflow. administrators; it is to prevent repository topology from determining a
person's workflow.
The initial module classification is deliberately outcome-oriented: The initial module classification is deliberately outcome-oriented:
@@ -5,9 +5,9 @@ A migration-owning module must register and document its canonical DSAR provider
Every other active module requires a reviewed explanation of why it owns no Every other active module requires a reviewed explanation of why it owns no
persistent subject-data store. Adding a migration invalidates that explanation. persistent subject-data store. Adding a migration invalidates that explanation.
- Active modules: 68 - Active modules: 72
- Registered and documented DSAR providers: 48 - Registered and documented DSAR providers: 49
- Reviewed no-store rationales: 20 - Reviewed no-store rationales: 23
- Unexplained coverage gaps: 0 - Unexplained coverage gaps: 0
| Module | Repository | Persistence | Coverage | Rationale | | Module | Repository | Persistence | Coverage | Rationale |
@@ -32,11 +32,14 @@ persistent subject-data store. Adding a migration invalidates that explanation.
| `datasources` | `govoplan-datasources` | Migration-owned | Provider | Provider `privacy.dsar.datasources` is registered and documented. | | `datasources` | `govoplan-datasources` | Migration-owned | Provider | Provider `privacy.dsar.datasources` is registered and documented. |
| `decisions` | `govoplan-decisions` | Migration-owned | Provider | Provider `privacy.dsar.decisions` is registered and documented. | | `decisions` | `govoplan-decisions` | Migration-owned | Provider | Provider `privacy.dsar.decisions` is registered and documented. |
| `dist_lists` | `govoplan-dist-lists` | Migration-owned | Provider | Provider `privacy.dsar.dist_lists` is registered and documented. | | `dist_lists` | `govoplan-dist-lists` | Migration-owned | Provider | Provider `privacy.dsar.dist_lists` is registered and documented. |
| `dms` | `govoplan-dms` | No module migration | Reviewed no-store rationale | Stateless integration-preview module: DMS retains no document, person, credential, or provider-response store; Files and Records remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, or diagnostic. |
| `docs` | `govoplan-docs` | Migration-owned | Provider | Provider `privacy.dsar.docs` is registered and documented. | | `docs` | `govoplan-docs` | Migration-owned | Provider | Provider `privacy.dsar.docs` is registered and documented. |
| `encryption` | `govoplan-encryption` | Migration-owned | Provider | Provider `privacy.dsar.encryption` is registered and documented. | | `encryption` | `govoplan-encryption` | Migration-owned | Provider | Provider `privacy.dsar.encryption` is registered and documented. |
| `erp` | `govoplan-erp` | No module migration | Reviewed no-store rationale | Stateless integration-contract module: ERP retains no invoice, payable, plan, booking observation, provider response, or credential store; Procurement, Payments, Ledger, Files, and Audit remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, reconciliation decision, or diagnostic. |
| `evaluation` | `govoplan-evaluation` | No module migration | Reviewed no-store rationale | Contract-only module: evaluation runs, responses, and scores are not persisted; reassess before adding a migration-owned store. | | `evaluation` | `govoplan-evaluation` | No module migration | Reviewed no-store rationale | Contract-only module: evaluation runs, responses, and scores are not persisted; reassess before adding a migration-owned store. |
| `facilities` | `govoplan-facilities` | No module migration | Reviewed no-store rationale | Contract-only module: facility and maintenance persistence are not implemented; reassess before adding a migration-owned store. | | `facilities` | `govoplan-facilities` | No module migration | Reviewed no-store rationale | Contract-only module: facility and maintenance persistence are not implemented; reassess before adding a migration-owned store. |
| `files` | `govoplan-files` | Migration-owned | Provider | Provider `privacy.dsar.files` is registered and documented. | | `files` | `govoplan-files` | Migration-owned | Provider | Provider `privacy.dsar.files` is registered and documented. |
| `fit_connect` | `govoplan-fit-connect` | No module migration | Reviewed no-store rationale | Stateless transport-contract module: FIT-Connect retains no submission, attachment, receipt, acknowledgement plan, key, provider response, or diagnostic store; the owning Service, Forms, Cases, Files, and Audit workflows remain responsible for subject data. Reassess before persisting any ingress or event-log evidence. |
| `forms` | `govoplan-forms` | Migration-owned | Provider | Provider `privacy.dsar.forms` is registered and documented. | | `forms` | `govoplan-forms` | Migration-owned | Provider | Provider `privacy.dsar.forms` is registered and documented. |
| `forms_runtime` | `govoplan-forms-runtime` | Migration-owned | Provider | Provider `privacy.dsar.forms_runtime` is registered and documented. | | `forms_runtime` | `govoplan-forms-runtime` | Migration-owned | Provider | Provider `privacy.dsar.forms_runtime` is registered and documented. |
| `grants` | `govoplan-grants` | No module migration | Reviewed no-store rationale | Contract-only module: grant applications, awards, and monitoring are not persisted; reassess before adding a migration-owned store. | | `grants` | `govoplan-grants` | No module migration | Reviewed no-store rationale | Contract-only module: grant applications, awards, and monitoring are not persisted; reassess before adding a migration-owned store. |
@@ -72,7 +75,7 @@ persistent subject-data store. Adding a migration invalidates that explanation.
| `soap` | `govoplan-soap` | No module migration | Reviewed no-store rationale | Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store. | | `soap` | `govoplan-soap` | No module migration | Reviewed no-store rationale | Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store. |
| `tasks` | `govoplan-tasks` | Migration-owned | Provider | Provider `privacy.dsar.tasks` is registered and documented. | | `tasks` | `govoplan-tasks` | Migration-owned | Provider | Provider `privacy.dsar.tasks` is registered and documented. |
| `templates` | `govoplan-templates` | Migration-owned | Provider | Provider `privacy.dsar.templates` is registered and documented. | | `templates` | `govoplan-templates` | Migration-owned | Provider | Provider `privacy.dsar.templates` is registered and documented. |
| `tenancy` | `govoplan-tenancy` | No module migration | Reviewed no-store rationale | Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data. | | `tenancy` | `govoplan-tenancy` | Migration-owned | Provider | Provider `privacy.dsar.tenancy` is registered and documented. |
| `tickets` | `govoplan-tickets` | Migration-owned | Provider | Provider `privacy.dsar.tickets` is registered and documented. | | `tickets` | `govoplan-tickets` | Migration-owned | Provider | Provider `privacy.dsar.tickets` is registered and documented. |
| `transparency` | `govoplan-transparency` | No module migration | Reviewed no-store rationale | Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store. | | `transparency` | `govoplan-transparency` | No module migration | Reviewed no-store rationale | Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store. |
| `views` | `govoplan-views` | Migration-owned | Provider | Provider `privacy.dsar.views` is registered and documented. | | `views` | `govoplan-views` | Migration-owned | Provider | Provider `privacy.dsar.views` is registered and documented. |
@@ -80,6 +83,7 @@ persistent subject-data store. Adding a migration invalidates that explanation.
| `wiki` | `govoplan-wiki` | Migration-owned | Provider | Provider `privacy.dsar.wiki` is registered and documented. | | `wiki` | `govoplan-wiki` | Migration-owned | Provider | Provider `privacy.dsar.wiki` is registered and documented. |
| `workflow` | `govoplan-workflow` | No module migration | Reviewed no-store rationale | Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage. | | `workflow` | `govoplan-workflow` | No module migration | Reviewed no-store rationale | Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage. |
| `workflow_engine` | `govoplan-workflow-engine` | Migration-owned | Provider | Provider `privacy.dsar.workflow_engine` is registered and documented. | | `workflow_engine` | `govoplan-workflow-engine` | Migration-owned | Provider | Provider `privacy.dsar.workflow_engine` is registered and documented. |
| `xrechnung` | `govoplan-xrechnung` | No module migration | Reviewed no-store rationale | Stateless validation-contract module: XRechnung persists no invoice, report, diagnostic, or handoff; the invoking Files, Procurement, or Payments workflow remains the subject-data owner. Reassess before adding a validation store. |
Provider search, export minimization, retention, and erasure behavior remains Provider search, export minimization, retention, and erasure behavior remains
documented and tested by each owning module. This matrix verifies adoption and documented and tested by each owning module. This matrix verifies adoption and
+138
View File
@@ -0,0 +1,138 @@
# Full registry candidates / Vollständige Registry-Kandidaten
## Operator workflow (EN)
The canonical `release-catalog.py full-registry` command takes `--package-set`,
`--package-lock`, `--wheelhouse`, `--webui-packages`, `--output-dir`, and a
configured `--catalog-signing-key`. Generate the package set with
`generate-release-package-set.py --profile full` and download its exact artifacts
with `resolve-package-artifacts.py`; do not substitute locally rebuilt wheels.
The candidate compares the package set with the exact developer meta-package
pins, checks archive bytes and package metadata against the lock, and synthesizes
every entry from its immutable tagged manifest. Native package publication and
its CI authority remain trusted: this verifies the published artifact identity,
not independent reproducible-build equivalence to source.
Pass `--selected-repository` once for each newly released repository, including
Core when it changes. These selected units must have clean, version-aligned
named branches whose HEAD equals the annotated local and remote release tag.
Other full-profile packages retain their exact older annotated tags; a later
workflow-only commit on `main` does not relabel those package contents or force
a version bump. Every source fetch/push endpoint must match the registered
origin, and all entries bind their source commit and annotated tag object.
Git replacement objects, caller Git configuration, and executable-path
redirection cannot substitute another tagged manifest tree.
The fixed existing website catalog and keyring are authenticated before signing.
An older catalog without a signed keyring hash can be migrated only through this
complete rebuild, only when its signature verifies and its entire keyring
exactly matches the configured known signers. No old entries or artifact hashes
are reused. The new catalog signs the exact unchanged website keyring hash;
key rotation remains a separate reviewed operation. Selective candidates still
reject unpinned base keyrings. New candidate directories are private and
exclusive: a retry must choose a new directory, not overwrite a reviewed one.
Run these commands on the trusted host, with an operator-private source workspace
and artifact directory. `RELEASE_PYTHON` must select its private environment and
`RELEASE_NPM` an absolute npm executable with a trusted sibling Node 22 binary.
In Flatpak, execute host commands through `flatpak-spawn --host`; sandbox and
host UID mappings are not interchangeable. Do not weaken trust gates or change
system-wide permissions.
```sh
# These paths identify previously prepared private operator resources.
RELEASE_WORKSPACE=/path/to/private/workspace
RELEASE_PYTHON="$RELEASE_WORKSPACE/govoplan/.host-venv/bin/python"
RELEASE_NPM=/path/to/private/node22/bin/npm
ARTIFACT_ROOT=/path/to/private/artifacts
RELEASE_CANDIDATE=/path/to/private/new-candidate
RELEASE_VERSION=0.1.45
RELEASE_TOOLS="$RELEASE_WORKSPACE/govoplan/tools/release"
umask 077
"$RELEASE_PYTHON" "$RELEASE_TOOLS/generate-release-package-set.py" \
--version "$RELEASE_VERSION" --profile full \
--workspace "$RELEASE_WORKSPACE" --output "$ARTIFACT_ROOT/packages.json"
PATH="$(dirname "$RELEASE_NPM"):/usr/bin:/bin" \
"$RELEASE_PYTHON" "$RELEASE_TOOLS/resolve-package-artifacts.py" \
--package-set "$ARTIFACT_ROOT/packages.json" \
--wheelhouse "$ARTIFACT_ROOT/wheels" \
--webui-packages "$ARTIFACT_ROOT/webui" \
--lock-output "$ARTIFACT_ROOT/artifacts.lock.json" \
--python "$RELEASE_PYTHON" --npm "$RELEASE_NPM"
# Repeat --selected-repository for EVERY newly released unit, not just Core.
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" full-registry \
--workspace-root "$RELEASE_WORKSPACE" \
--package-set "$ARTIFACT_ROOT/packages.json" \
--package-lock "$ARTIFACT_ROOT/artifacts.lock.json" \
--wheelhouse "$ARTIFACT_ROOT/wheels" --webui-packages "$ARTIFACT_ROOT/webui" \
--output-dir "$RELEASE_CANDIDATE" --selected-repository govoplan-core \
--catalog-signing-key known-key=/path/to/private/known-key.pem --json
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" publish-candidate \
--workspace-root "$RELEASE_WORKSPACE" --candidate-dir "$RELEASE_CANDIDATE" \
--channel stable --npm "$RELEASE_NPM" --build-web \
--commit --tag --push --tag-name "catalog-v$RELEASE_VERSION" --json
```
The last command is a strict non-mutating preview because `--apply` is absent.
Review its output, then repeat it with `--apply` to publish. The website's locked
build dependencies must already be installed before `--build-web`. The publisher
sanitizes the build and Git environments and pushes the verified immutable
website commit/tag. Source tag publication and registry package availability
must be complete before candidate generation.
Source tags, registry packages, and a signed module catalog do not imply that a
new runtime distribution exists. While runtime images are held, leave the Meta
Gitea runtime Release held too: a normal source-only Release can replace Gitea's
`releases/latest` discovery result despite having no deployment assets. The
deployer still requires an explicit signed manifest, digest, and trusted
keyring; it does not deploy a tag or module catalog directly.
## Betriebsablauf (DE)
`release-catalog.py full-registry` übernimmt den vollständigen Paketbestand,
die Registry-Sperrdatei, das Wheel-Verzeichnis, die WebUI-Archive und den
konfigurierten Signaturschlüssel. Zuerst mit
`generate-release-package-set.py --profile full` die exakten Meta-Paketversionen
ermitteln und mit `resolve-package-artifacts.py` die veröffentlichten Artefakte
herunterladen. Lokal neu gebaute Wheels sind kein Ersatz. Der Kandidat prüft
Paketidentitäten, Dateigrößen und Hashes und erzeugt alle Einträge aus den
unveränderlichen getaggten Manifesten. Die Registry und ihre veröffentlichende
CI bleiben eine Vertrauensgrundlage; dies ist kein unabhängiger Nachweis eines
reproduzierbaren Builds aus dem Quellcode.
Jedes neu veröffentlichte Repository wird mit `--selected-repository`
angegeben. Nur diese Auswahl muss mit dem sauberen, versionsgleichen HEAD eines
benannten Branches und dem annotierten lokalen und entfernten Tag übereinstimmen.
Unveränderte Pakete behalten ihren ursprünglichen Tag, auch wenn auf `main`
bereits eine spätere Workflow-Korrektur liegt. Alle Quelladressen müssen dem
registrierten Ursprung entsprechen; Commit und annotiertes Tag-Objekt werden
für jeden Eintrag gebunden. Git-Ersetzungsobjekte oder fremde Git-Konfiguration
können dabei keinen anderen Manifestbaum unterschieben.
Vor dem Signieren werden der bestehende Website-Katalog und sein Schlüsselbund
geprüft. Ein alter Katalog ohne signierten Schlüsselbund-Hash darf ausschließlich
durch diesen vollständigen Neuaufbau migriert werden: Seine Signatur muss gültig
sein und der gesamte Schlüsselbund exakt den konfigurierten bekannten Signierern
entsprechen. Alte Einträge oder Artefakt-Hashes werden nicht übernommen. Der neue
Katalog bindet den unveränderten Schlüsselbund-Hash; ein Schlüsselwechsel bleibt
ein eigener geprüfter Vorgang. Selektive Kandidaten verlangen weiterhin einen
bereits gebundenen Schlüsselbund. Kandidaten werden nur in neuen privaten
Verzeichnissen erzeugt und niemals überschrieben.
Das obige Befehlsbeispiel wird auf dem vertrauenswürdigen Host ausgeführt. Dafür
die private Python-Umgebung und einen absoluten `--npm`-Pfad zu Node 22 verwenden;
unter Flatpak die Host-Werkzeuge über `flatpak-spawn --host` aufrufen. Die
gesperrten Website-Build-Abhängigkeiten vorher installieren. Keine
Vertrauensprüfung umgehen und keine globalen Rechte ändern. Die Artefaktordner
müssen privat und bei der Auflösung leer sein. Vor der Kandidatenerzeugung
müssen Quell-Tags und Registry-Pakete vollständig veröffentlicht sein.
Die Veröffentlichung zunächst mit `publish-candidate --commit --tag --push
--build-web` ohne `--apply` prüfen und erst nach Prüfung mit `--apply` ausführen.
Solange Laufzeit-Images zurückgestellt sind, bleibt auch das Meta-Gitea-Runtime-
Release zurückgestellt: Ein reines Quellcode-Release könnte sonst als neuestes
Release erscheinen. Eine Installation benötigt weiterhin ein signiertes
Laufzeitmanifest, dessen Digest und einen explizit vertrauenswürdigen Schlüsselbund.
@@ -100,6 +100,7 @@ The private installation directory contains:
| `plan.json` | Latest desired-state diff and readiness findings | | `plan.json` | Latest desired-state diff and readiness findings |
| `receipt.json` | Last successfully applied immutable identities | | `receipt.json` | Last successfully applied immutable identities |
| `infrastructure-capabilities.json` | Deterministic non-secret capability states, endpoint metadata, secret references, consumers, and resumable post-install tasks | | `infrastructure-capabilities.json` | Deterministic non-secret capability states, endpoint metadata, secret references, consumers, and resumable post-install tasks |
| `infrastructure-dependency-inventory.json` | Owner-only, short-lived Ops evidence of actual module-owned configuration and data that depend on infrastructure capabilities |
| `distribution-manifest.json` | Canonical signed runtime/image selection adopted by the installer | | `distribution-manifest.json` | Canonical signed runtime/image selection adopted by the installer |
| `distribution-keyring.json` | Explicitly installed public trust anchor for runtime releases | | `distribution-keyring.json` | Explicitly installed public trust anchor for runtime releases |
| `backup-evidence.json` | Signed provider-neutral coordinated backup and isolated-restore receipt | | `backup-evidence.json` | Signed provider-neutral coordinated backup and isolated-restore receipt |
@@ -123,6 +124,16 @@ environment or initiating an implicit object migration. Invalid receipts fail
closed, while a deployment without a mounted receipt continues to run but closed, while a deployment without a mounted receipt continues to run but
cannot apply receipt-bound configuration fragments. cannot apply receipt-bound configuration fragments.
Enabled modules may also register a Core infrastructure-dependency provider.
The authorized Ops endpoint aggregates those providers without importing their
tables. Mail reports persisted SMTP endpoints, credential-binding counts and
legacy profiles; Files reports its runtime storage binding plus persisted blob
counts and byte totals grouped by backend. Ops reports the active PostgreSQL,
Redis coordination, ingress, and load-balancing runtime bindings. Provider output contains stable
references, bounded numeric metrics and required migration actions, never
credentials, endpoint secrets, tenant identifiers or file keys. A provider
failure makes the entire inventory incomplete.
Build the same dependency-free tool as one downloadable artifact: Build the same dependency-free tool as one downloadable artifact:
```sh ```sh
@@ -350,6 +361,15 @@ and WebUI API proxy traffic across API replicas. The WebUI and API services do
not publish host ports. HAProxy has no Docker socket and discovers only the not publish host ports. HAProxy has no Docker socket and discovers only the
bounded replica slots rendered into `load-balancer.cfg`. bounded replica slots rendered into `load-balancer.cfg`.
New installation specifications default to HAProxy `3.2.23-alpine`, pinned to
registry index `sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
This patch remains on HAProxy 3.2 LTS and Alpine 3.24.1. Loading an existing
specification preserves its explicit image; it does not perform an upgrade.
The [runtime remediation evidence](../security/RUNTIME_IMAGE_REMEDIATION_2026-09-08.md)
records both architecture scans and the pending binary/configuration, runtime,
inventory and final-image checks. The source default is not a release approval:
runtime publication remains held in Meta #52.
Replica counts are desired state: Replica counts are desired state:
```sh ```sh
@@ -424,10 +444,16 @@ infrastructure capability projections.
- Adding a managed component creates its service and persistent volume. - Adding a managed component creates its service and persistent volume.
- Removing a component removes its service container on apply. - Removing a component removes its service container on apply.
- Replacing or removing a capability adds a review action that names the prior - Reconfiguring, replacing or removing a capability adds a review action that
and desired state/source plus declared module consumers. This does not claim names the prior and desired state/source, declared consumers, actual
that the deployer can inspect module-owned database configuration; the provider-reported dependency records and each required migration action.
operator must review that inventory before apply. - The deployer blocks that change when provider inventory is missing,
incomplete, more than five minutes old, from another installation, timestamped
in the future, or does not cover every impacted capability. It never treats
installer-declared consumers as proof that persisted module state is absent.
- The inventory reports impact; it does not migrate or delete module-owned
configuration or data. Complete the reported preparation and collect again
immediately before apply.
- Volumes are retained by default; deleting data requires a separate, - Volumes are retained by default; deleting data requires a separate,
deliberately destructive workflow. deliberately destructive workflow.
- Existing generated credentials are retained unless an explicit future rotate - Existing generated credentials are retained unless an explicit future rotate
@@ -455,6 +481,29 @@ dedicated ConfigMap and read-only file mount. Ops validates the bounded schema
before displaying configured, externally supplied, available-unconfigured, or before displaying configured, externally supplied, available-unconfigured, or
unavailable states and any pending post-install tasks. unavailable states and any pending post-install tasks.
Collect current dependency evidence with an API key whose principal has one of
the Ops read scopes:
```sh
export GOVOPLAN_OPS_API_KEY='<short-lived operator API key>'
python3 govoplan-deploy.pyz collect-infrastructure-inventory \
--directory /srv/govoplan/example
python3 govoplan-deploy.pyz doctor \
--directory /srv/govoplan/example
python3 govoplan-deploy.pyz apply \
--directory /srv/govoplan/example
unset GOVOPLAN_OPS_API_KEY
```
The command defaults to
`<public-url>/api/v1/ops/infrastructure/dependencies`; `--ops-url` may select an
explicit HTTPS endpoint (plain HTTP is accepted only on loopback). `apply`
refreshes the inventory automatically when `GOVOPLAN_OPS_API_KEY` is present.
Otherwise an already collected, current inventory may be used. The API key is
sent only as `X-API-Key`, is never written to the bundle, and the inventory file
is owner-readable only. Because it contains operational references and counts,
handle it as private evidence even though it contains no secret material.
Every apply operation is journalled before image pulls or runtime mutation. A Every apply operation is journalled before image pulls or runtime mutation. A
failure before migration may restore a verified previous bundle. Once migration failure before migration may restore a verified previous bundle. Once migration
starts, recovery is forward-only unless an independently verified database starts, recovery is forward-only unless an independently verified database
@@ -0,0 +1,91 @@
# Integrity and performance source release — September 2026
Coordinated tracking: [Core #298](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/298).
This source publication advances only the affected packages; it is not a signed
catalog publication, runtime-image release, or remote production rollout.
## Package set
| Package | Version |
| --- | --- |
| Core / developer meta-package | 0.1.46 |
| Addresses | 0.1.23 |
| Calendar | 0.1.24 |
| Campaign | 0.1.29 |
| Cases | 0.1.25 |
| Committee | 0.1.22 |
| Connectors | 0.1.27 |
| Dataflow | 0.1.25 |
| Datasources | 0.1.26 |
| Files | 0.1.27 |
| Forms Runtime | 0.1.22 |
| IDM | 0.1.26 |
| Mail | 0.1.28 |
| Reporting | 0.1.22 |
| Tickets | 0.1.23 |
Consumers of new Core helpers require Core 0.1.46 or later. Dataflow and
Datasources also require the matching Core WebUI contract. The release manifests,
immutable Git lock, and developer package describe this coordinated composition.
Unchanged packages retain their independent versions.
## Database and data integrity
The reviewed additive heads are Connectors `d2a4c6e8f0b1`, Datasources
`e2b8d4a0f6c3`, Files `a2b3c4d5e701`, and Mail `b5d6e7f8091a`.
Back up the target deployment and rehearse its normal upgrade before rollout.
Never downgrade away retained CSV originals without a separate recovery plan.
Files preserves historical duplicate copies; Mail leaves legacy maildrop identity
unset rather than guessing an account. A schema upgrade does not authorize POP3
retrieval/reconciliation, provider deletion, or message resending.
Development startup may automatically apply pending migrations after a watched
source change. Therefore, inspect actual Alembic heads and schema before assuming
a live development database is still at its pre-change state. A backup taken
after such an upgrade is a current-state recovery copy, not a pre-upgrade backup.
On 8 September, the local PostgreSQL development database already contained all
four heads. Schema and constraint inspection passed; recomputing the Files
identity backfill checked 7,385 rows with zero mismatches. A private current-state
database backup was created. Per-instance backup paths and row contents are not
published in this repository. The implementation's initial receipt claiming no
live migration occurred was incorrect: watched development-server restarts had
applied the migration files automatically. Restoring the backup into an isolated
PostgreSQL 16 cluster and running the normal upgrade preserved all 281 public
tables, 142,287 rows, and migration heads exactly. Both Datasources PostgreSQL
concurrency regressions passed. The test-only cluster was then stopped.
Release preparation additionally corrected Alembic's ConfigParser handling of
percent-escaped connection URLs, preserving the exact database URL. The static
migration auditor now recognizes the existing reviewed development-wrapper
aliases and peer filenames without executing migration code; historical
migrations are unchanged.
## Verification and boundaries
The implementation passed the required focused workspace gate, including 63
frontend build configurations and 230 browser conformance tests. Owning-module
regressions cover exact import/rollback evidence, authorization-before-pagination,
bounded recurrence and response processing, collision-safe attachment naming,
and stale asynchronous UI completion. English/German feature documentation stays
in each owning module; Core documents shared integrity contracts.
Mock-provider tests and local work-count measurements do not establish production
throughput or provider race behavior. Real S3/SMB/Seafile and POP3 acceptance,
representative load testing, and deployment authentication checks remain separate
operational validation. Source tags trigger package workflows; a pushed source
tag alone does not prove a registry artifact or signed catalog is published.
## Deutsch
Dieses koordinierte Quellrelease veröffentlicht nur die betroffenen Pakete.
Produktions-Images, signierter Modulkatalog und entfernte Produktivinstanzen
werden dadurch nicht ausgerollt. Die vier Migrationen bewahren bestehende Daten;
historische POP3-Zuordnungen werden nicht geraten. POP3 ist ein eigener
Import-Arbeitsablauf innerhalb von **Mail**, kein separat installierbares Modul.
Der Entwicklungsserver kann Migrationen beim automatischen Neustart nach einer
Quelländerung bereits anwenden. Tatsächliche Schema-Stände prüfen; eine danach
erstellte Sicherung enthält den aktuellen Stand und ist keine Sicherung vor dem
Upgrade. Externe Transportaktionen, erneutes Versenden und Löschungen werden
durch die Migration oder Quellveröffentlichung nicht ausgelöst.
+204 -2
View File
@@ -86,6 +86,16 @@ contract. The coordinated release synchronizes `peerDependencies` and
tag, then synchronizes each lockfile root from the final package metadata. A tag, then synchronizes each lockfile root from the final package metadata. A
distinct root package remains independent. distinct root package remains independent.
Every module referenced by Core's Git-based `package.release.json` must expose
its WebUI identity at the repository root, including matching peer requirements
and `webui/`-prefixed entry exports (also CSS subpaths). npm resolves Git
dependencies from the repository root, while the native-package workflow packs
`webui/`; success in one path does not verify the other. Run
`python tools/checks/check-webui-package-facades.py` after changing either
manifest or the release composition. The focused gate also runs this check.
Adding or correcting a facade in an already published repository requires a
new patch tag; never repair an existing immutable tag in place.
It builds one wheel and, where applicable, one npm tarball. The workflow records It builds one wheel and, where applicable, one npm tarball. The workflow records
the source tag, source commit, filename, size, and SHA-256 in the source tag, source commit, filename, size, and SHA-256 in
`package-artifacts.json` before publishing. Gitea rejects a second upload of the `package-artifacts.json` before publishing. Gitea rejects a second upload of the
@@ -179,8 +189,13 @@ than invoking `pip`, `npm`, or Git on the target host.
## Public module directory ## Public module directory
`tools/release/publish-release-catalog.sh` resolves the selected package set and For an operator-reviewed full publication, use
registry lock before it creates a catalog. Catalog entries are synthesized from `tools/release/release-catalog.py full-registry` followed by the same tool's
`publish-candidate` command. Resolve the package set and registry lock first;
the older direct-write shell wrapper is not the strict candidate publication
path. See [Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
for the private host runtime, exact artifact checks, and legacy keyring transition.
Catalog entries are synthesized from
the exact tagged module manifests, never from a hand-maintained module list or the exact tagged module manifests, never from a hand-maintained module list or
the current workspace. Each entry binds its Python wheel and optional WebUI the current workspace. Each entry binds its Python wheel and optional WebUI
tarball to the registry URL, filename, size, SHA-256, package identity, source tarball to the registry URL, filename, size, SHA-256, package identity, source
@@ -248,11 +263,198 @@ python tools/release/generate-developer-meta-package.py
python tools/release/generate-developer-meta-package.py --check python tools/release/generate-developer-meta-package.py --check
``` ```
The direct generator is a development synchronization tool, not a receipt-gated
release executor. For release preparation, use the guarded out-of-run stage below.
`push-release-tag.sh` performs this synchronization before release commits and `push-release-tag.sh` performs this synchronization before release commits and
tags. The meta-package is for editable/developer setup and composition tests. It tags. The meta-package is for editable/developer setup and composition tests. It
does not enable modules, apply migrations, provision services, or establish does not enable modules, apply migrations, provision services, or establish
backup and recovery evidence. backup and recovery evidence.
### Shared source-tag contract and Meta composition
The shared version collector names Meta's real
`packages/govoplan-meta/pyproject.toml` separately from root `pyproject.toml`.
Only the registered `govoplan` system/meta repository with nested project name
`govoplan` receives this contract. Missing, unknown, or misidentified metadata
does not become a versionless exception. Version alignment compares the complete
nested file with the canonical operator-tool generator output: its version must
match Core, and dependencies and `full` composition must match the reviewed
requirements and workspace package versions. Validation never executes a
generator from a selected checkout. The shared trusted manifest checker can
load reviewed application manifests; these checks are not a code sandbox.
Meta's complete generated file is recognized by shared version-mutation discovery,
but the generic durable version executor deliberately cannot write it. A durable
run freezes the release console's own Meta checkout as trusted runtime code;
changing it in place would invalidate that run. The planner therefore places Meta
after Core and exposes only non-executable support preparation/publication steps,
not misleading automatic Meta version, commit, tag, or push actions. A missing or
different Core target produces an actionable preparation prerequisite.
Prepare Core and the intended module inputs first, commit their reviewed state,
then stop active durable runs for the target workspace. Use trusted operator tools
against a separate registered, private source checkout, never the running operator
Meta directory. Preview outside any selected source checkout, for example:
```sh
python tools/release/prepare-developer-meta-package.py \
--workspace /private/release-workspace --target-version X.Y.Z \
> /private/operator/meta-preview.json
python tools/release/prepare-developer-meta-package.py \
--workspace /private/release-workspace --target-version X.Y.Z \
--receipt /private/operator/meta-preview.json --apply --confirm-out-of-run
```
The explicit confirmation attests that no durable run is active for that target
workspace; the helper does not discover or stop other processes. Preview/apply
requires registered clean main sources, matching origins and live-main ancestry,
Core already aligned at the target, the exact nested identity, and no existing or
unverifiable target Meta tag. Frozen receipts cover source HEADs/filesystem
identities, release requirements, every discovered registered full-composition
pyproject, the trusted generator snapshot, and the resulting full-file hash.
Inputs are limited to 128 selected files, 2 MiB per file and 16 MiB aggregate.
The canonical generator renders copied bounded data in a temporary directory;
no generator from the selected checkout executes. Changed receipts block before
the file effect. Apply writes only `packages/govoplan-meta/pyproject.toml`, then
rechecks the other sources and exact output. A write or post-check failure that
may have changed the file reports `needs-reconciliation` and leaves that bounded
delta for explicit review; it never retries, rolls back, commits or publishes.
Review the complete generated composition and manually commit the resulting file.
Complete matching Core publication before guarded Meta source tagging/publication,
then start a fresh durable run from reviewed, clean, published operator tooling.
Hot self-updating durable Meta release execution remains explicitly unsupported;
this out-of-run preparation is the existing developer-meta support contract.
For every `tag_repositories` batch, strict checks apply to every selected
repository before any tag creation, fetch, or push: registered checkout and
origin/push URL, a clean `main` tracking `origin/main`, live remote-main ancestry,
exact frozen HEADs, and immutable annotated local/remote tag objects. Git metadata
must remain inside the operator's trusted workspace. Missing local knowledge of
live remote main is a blocker; fetch and review it separately. Unknown repositories
and non-registered remote aliases fail closed. If selected, Meta runs last.
For Meta only, the matching annotated Core release must exist before its effect; Core may be
an earlier selected repository, or an already tagged dependency. Publication
requires that Core's exact tag and main commit are already remote.
Non-Meta selections do not acquire Meta's composition or Core-tag prerequisite.
Local module-candidate tags still work before Core's final release lock or tag.
The existing Core WebUI bundle gate still applies to module publication and
batches selecting Core: relevant Core release-package and release-lock inputs
must be operator-owned regular files, at most 16 MiB each, and their identities
and content hashes are frozen before preflight and rechecked before every effect.
When Core is unselected, this does not require its checkout to be clean or tagged;
reviewed pending composition inputs retain their previous meaning. Backend-only
selections never read irrelevant Core WebUI files.
Before even read-only Git commands, source ancestry must be owned by root or the
current operator and must not be group/world writable. A sticky shared ancestor
such as `/tmp` is permitted only above an owned, protected child; the workspace
and checkouts receive no writable-directory exception. The current operator must
own source inputs and actual Git/worktree/common metadata, which must be regular
files/directories, non-symlinked, and non-writable by other users. Metadata walks
are bounded to 500,000 entries and 128 levels, and tracked inputs to 100,000 paths
and 16 MiB of listing text. Read-only Git targets, object alternates/grafts and
hidden/sparse/unmerged index entries are blocked. Frozen receipts include actual
checkout/Git directory paths, devices, inodes, owners and modes, so replacing Git
metadata with the same HEAD is still detected. All selected version/composition
inputs must be tracked, including root and WebUI package/lock metadata, discovered
module manifests and package initializers, and Meta's nested package and release
requirements; ignored working files cannot supply declarations absent from a tag.
No chmod, ownership repair or
global Git trust change is performed. A shared writable workspace must first be
recreated or reviewed in the operator's protected release area by an explicitly
authorized preparation workflow.
Preview is read-only. Local-tag mode creates only pinned annotated tags (or
retrieves an identical published annotation); it does not publish main or tags.
Publish mode atomically pushes the frozen main commit and annotation object,
without force, retagging, fallback, or automatic retry. The complete source
receipt is rechecked before every effect and afterward; remote main and the
exact annotated tag must both match, not merely the Git exit status. Changes
after preflight stop the remaining batch. Atomicity is per repository, not
across repositories: earlier successful publications and a newly created local
tag can remain after a later failure. Inspect reported receipts and obtain a new
review before retrying; do not move immutable tags.
Whole-batch revalidation deliberately repeats source and live-remote checks around
each repository effect; the number of checks can grow quadratically with batch
size. Plan release time accordingly rather than bypassing trust checks. The
shared internal preflight is read-only and exposes no legacy mutation path.
The fixture suite covers Meta and non-Meta preview/local-tag/
publication using temporary local bare remotes, including stale compositions,
unsafe origins, divergent branches, damaged tag identity, changed receipts and
false publication success. This is local tooling evidence, not a real release
publication or production permission check.
Deutsch: Die gemeinsamen Helfer erkennen ausschließlich das registrierte
Meta-Repository mit dem echten Paket `packages/govoplan-meta/pyproject.toml`
(Projektname `govoplan`). Version und vollständige Zusammensetzung müssen dem
kanonischen Generator, Core und den geprüften Anforderungen entsprechen; der
Generator stammt niemals aus dem ausgewählten Checkout. Der gemeinsame
Manifestprüfer kann geprüften Anwendungscode laden und ist keine Sandbox.
Die gemeinsame Änderungsplanung erkennt die vollständig generierte Paketdatei,
aber der dauerhafte Versionsausführer darf Meta nicht selbst verändern: sein
eingefrorener Lauf bindet den Meta-Checkout als vertrauenswürdigen Programmstand.
Meta erscheint deshalb nach Core ausschließlich mit nicht automatisch ausführbaren
Vorbereitungs-/Veröffentlichungsschritten. Zuerst Core und Modulquellen vorbereiten
und geprüft committen; bei abweichender Core-Zielversion nennt der Plan diese
Voraussetzung ausdrücklich. Aktive dauerhafte Läufe des Ziel-Workspaces beenden.
Mit `prepare-developer-meta-package.py` zunächst eine Vorschau außerhalb der
Quell-Checkouts speichern, dann deren JSON über `--receipt` zusammen mit `--apply`
und `--confirm-out-of-run` bestätigen. Das Ziel muss ein separater registrierter
privater Checkout sein, niemals das laufende Operator-Meta. Die Bestätigung ist
eine Betreibererklärung; der Helfer sucht oder beendet keine fremden Prozesse.
Quell-HEADs, Pfadidentitäten, Anforderungen, alle registrierten vollständigen
Paket-Eingaben, der vertrauenswürdige Generator und der vollständige Ausgabehash
werden eingefroren. Es gelten höchstens 128 Quelldateien, 2 MiB je Datei und
16 MiB insgesamt. Core muss bereits vollständig zur Zielversion passen; vorhandene
oder nicht verifizierbare Meta-Zieltags sperren die Vorbereitung. Geänderte
Nachweise stoppen vor dem Schreiben. Ausschließlich die verschachtelte Paketdatei
wird vollständig generiert und danach geprüft; ein Fehler nach dem Schreiben
meldet `needs-reconciliation` und erfordert die manuelle Prüfung dieser begrenzten
Änderung, ohne automatisches Zurücksetzen. Kein automatischer
Commit, Push oder Wiederholungsversuch findet statt. Zusammensetzung prüfen,
manuell committen, Core zuerst veröffentlichen, dann die geschützte Meta-Tag-Route
verwenden und einen neuen dauerhaften Lauf starten. Eine Selbstaktualisierung
des aktiven dauerhaften Meta-Laufs bleibt ausdrücklich nicht unterstützt.
Für jeden Tag-Stapel, auch ohne Meta, gelten
Vertrauens-, Origin-, saubere Main- und Live-Abstammungsprüfungen für die gesamte
Auswahl vor jeder Änderung. Unbekannte Repositories und nicht registrierte
Remote-Aliase sind gesperrt. Nur bei ausgewähltem Meta gelten zusätzlich dessen
Zusammensetzungsprüfung und der passende annotierte Core-Tag als Voraussetzung;
Meta folgt zuletzt. Für Metas Veröffentlichung müssen Core-Tag und Main-Commit
bereits auf dem Remote vorliegen. Lokale Modul-Kandidatentags bleiben vor Cores
abschließendem Release-Lock und Tag möglich. Die vorhandene Core-WebUI-Bundleprüfung
bleibt bei Modulveröffentlichung und Core-Auswahl erhalten. Relevante Core-Paket-
und Lockdateien müssen eigene reguläre Dateien mit höchstens je 16 MiB sein;
Identität und Inhaltshash werden eingefroren und vor jeder Aktion erneut geprüft.
Nicht ausgewähltes Core benötigt dafür weder einen sauberen Checkout noch einen
Tag. Reine Backend-Auswahlen lesen keine irrelevanten Core-WebUI-Dateien.
Vor Git-Aufrufen werden Eigentümer, Schreibrechte, sichere
Pfadabstammung und echte Git-/Worktree-Metadaten geprüft; veränderbare gemeinsame
Verzeichnisse, fremde Eigentümer, Alternates, Grafts und versteckte Indexeinträge
sind gesperrt. Ein Sticky-Bit-Vorfahr wie `/tmp` ist nur oberhalb eines eigenen
geschützten Unterverzeichnisses zulässig. Es erfolgen weder Rechtereparaturen
noch globale Git-Vertrauensänderungen. Ausgewählte Versions- und Zusammensetzungs-
dateien müssen versioniert sein: Paket-/Lockdateien, Modulmanifeste und
Paketinitialisierer sowie Metas verschachteltes Paket und Release-Anforderungen.
Ignorierte Arbeitsdateien dürfen keine vom Tag abweichenden Angaben liefern.
Die Vorschau schreibt nichts, lokale Tags veröffentlichen nichts,
und die Veröffentlichung überträgt Main und den exakten annotierten Tag atomar
je Repository. Unmittelbar vor und nach den Aktionen werden die eingefrorenen
Quellnachweise erneut geprüft, einschließlich entferntem Main und Tag-Objekt.
Bei Änderungen oder Fehlern stoppt der Rest des Stapels ohne automatischen
Wiederholungsversuch. Frühere Veröffentlichungen und neu erzeugte lokale Tags
können bestehen bleiben: vor einem neuen Versuch Nachweise prüfen und erneut
freigeben, niemals unveränderliche Tags verschieben. Die vollständigen Quell- und
Live-Remote-Prüfungen werden um jede Aktion wiederholt; bei großen Stapeln kann
deren Anzahl quadratisch wachsen. Diese konservativen Prüfkosten gehören zur
Release-Planung. Der interne Vorprüfer ist ausschließlich lesend und besitzt
keinen alten Änderungspfad. Tests für Auswahlen mit und ohne Meta verwenden
nur temporäre lokale Remotes und ersetzen keine echte Veröffentlichungsprüfung.
If the tag-triggered developer meta-package job fails before publication, rerun If the tag-triggered developer meta-package job fails before publication, rerun
`publish-developer-meta-package.yml` with the existing protected version. The `publish-developer-meta-package.yml` with the existing protected version. The
manual path validates that tag against `main`, checks out its exact commit, and manual path validates that tag against `main`, checks out its exact commit, and
+65 -2
View File
@@ -58,6 +58,45 @@ checkouts remain usable for read-only planning, but every durable executor
fails closed there; clone the registered origins into a private workspace fails closed there; clone the registered origins into a private workspace
before releasing. before releasing.
For a host with a confirmed IPv6 connection timeout, set
`GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` only for the release-tool invocation.
When unset, the original SSH command is preserved, including the trusted
operator's per-host `AddressFamily` configuration (normally `any`). Explicit
values accepted by the shared source/tag Git helper are exactly `any`, `inet`
(IPv4 only), and `inet6` (IPv6 only). Empty, misspelled, whitespace-padded, or
injected values fail before Git starts. The selector only adds the corresponding
fixed SSH `AddressFamily` option: it does not change DNS, host-key verification,
the registered remote, authentication, `BatchMode=yes`, or `ConnectTimeout=8`.
Arbitrary `GIT_SSH_COMMAND` overrides remain ignored. For example, start a
single local console invocation with:
```sh
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet \
./.venv/bin/python tools/release/release-console.py
```
The same process-scoped setting applies to canonical source/tag readbacks and
registry-candidate source verification. Under Flatpak, pass it explicitly to
the host invocation with `flatpak-spawn --host /usr/bin/env
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet ...`. It is not a global SSH setting
and does not affect the website publisher's separate transport sanitizer, npm,
or HTTP downloads. An IPv4-only setting cannot reach IPv6-only hosts; omit it
or use `any` when the diagnosed restriction no longer applies.
Deutsch: Bei einem bestätigten IPv6-Verbindungs-Timeout kann für genau einen
Release-Werkzeugaufruf `GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` gesetzt werden.
Ohne diese Variable bleibt der bisherige SSH-Befehl einschließlich der
vertrauenswürdigen Host-Konfiguration unverändert (normalerweise `any`).
Explizit zulässig sind ausschließlich `any`, `inet` (nur IPv4) und `inet6`
(nur IPv6). Andere oder leere Werte werden vor dem Git-Aufruf abgewiesen.
DNS, Hostschlüsselprüfung, registrierte Quelladresse, Authentifizierung und
Zeitlimit bleiben unverändert; frei vorgegebene SSH-Befehle bleiben gesperrt.
Unter Flatpak die Variable ausdrücklich an den Host-Aufruf übergeben. Die
Auswahl gilt für den gemeinsamen Git-Helfer der Quell-/Tag-Prüfungen, nicht
für den separaten Website-Publisher, npm oder HTTP-Downloads. Sie ändert keine
globale Konfiguration. Nach Behebung des Netzwerkproblems die Variable
weglassen oder auf `any` setzen; IPv4-only erreicht keine IPv6-only-Ziele.
The runtime itself is part of the authority boundary. Durable run creation The runtime itself is part of the authority boundary. Durable run creation
verifies the meta checkout, release/check tooling, repository registry, Python verifies the meta checkout, release/check tooling, repository registry, Python
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
@@ -221,6 +260,9 @@ Repository capabilities are frozen into each plan unit (`python-package`,
`core-release-bundle`, and the universal `git-source`) and determine which `core-release-bundle`, and the universal `git-source`) and determine which
steps appear. Internally aligned version changes are rendered deterministically steps appear. Internally aligned version changes are rendered deterministically
from recognized TOML, JSON, lockfile, manifest, and package declarations. from recognized TOML, JSON, lockfile, manifest, and package declarations.
The manifest may use a literal version or a top-level literal `MODULE_VERSION`;
the latter is updated without rewriting independently versioned interfaces.
Computed or missing version declarations fail before any metadata is written.
Pre-existing dirty worktrees remain visible but have no commit executor; the Pre-existing dirty worktrees remain visible but have no commit executor; the
console never absorbs unrelated operator changes. console never absorbs unrelated operator changes.
@@ -232,6 +274,17 @@ runs a receipt-bound alignment gate before exposing any atomic branch/tag push.
A failed step stops later steps while preserving prior receipts for explicit A failed step stops later steps while preserving prior receipts for explicit
retry or reconciliation. retry or reconciliation.
Local module candidate creation deliberately does not require those candidates
to be resolved already in Core's release lock: their annotated tags are inputs
to the next lock-generation step. The internal tag helper applies this ordering
only when no Core repository is selected and remote publication is disabled.
Module version/lock consistency, manifest validity, clean/non-behind worktrees,
and local/remote tag immutability checks still apply. Core candidate tagging
continues to validate its own complete bundle, and every remote-publication
preview and execution requires the selected modules to match Core's release
input and resolved lock. A local candidate is therefore not publication
approval; a stale Core lock blocks publication without changing remote refs.
The browser likewise retains the request identifier for an uncertain The browser likewise retains the request identifier for an uncertain
resume/retry/reconciliation response and replays it after reload. A successful resume/retry/reconciliation response and replays it after reload. A successful
replay selects the returned run state. Transport and server failures retain the replay selects the returned run state. Transport and server failures retain the
@@ -503,6 +556,15 @@ tree and requires byte-for-byte equality with those validated objects. Tags and
remote branch updates then reference that exact commit SHA rather than the remote branch updates then reference that exact commit SHA rather than the
mutable worktree `HEAD`. mutable worktree `HEAD`.
For a full registry-backed release, first build a fresh private candidate using
`release-catalog.py full-registry`. Pass `--selected-repository` for newly
released HEAD-bound units, not every unchanged package in the full profile.
The command independently checks all full-profile registry bytes and annotated
tag provenance, then feeds this same strict `publish-candidate` transaction.
It does not create Gitea runtime Releases or dispatch image builds. See
[Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
for the complete EN/DE workflow and the narrowly scoped legacy keyring transition.
Published channels are expected below the public catalog base URL: Published channels are expected below the public catalog base URL:
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json` - `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
@@ -515,8 +577,9 @@ updated catalog, and keep the published keyring healthy.
When a selected module exposes a WebUI package, its requested version must also When a selected module exposes a WebUI package, its requested version must also
match Core's `webui/package.release.json` input and the resolved match Core's `webui/package.release.json` input and the resolved
`package-lock.release.json` entry. The source-tag preflight, selective plan, and `package-lock.release.json` entry. The source-publication preflight, selective
catalog-candidate writer all enforce this composition boundary. Pins for modules plan, and catalog-candidate writer all enforce this composition boundary;
module-only local candidate tags use the staged order described above. Pins for modules
that are not part of the selective release remain unchanged. that are not part of the selective release remain unchanged.
Release integration also enforces repository and composition version alignment Release integration also enforces repository and composition version alignment
@@ -0,0 +1,91 @@
# WebUI release dependency installer retries
## English
This operational note covers
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
and the exit-status repair tracked in
[Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54).
It applies to release administrators using the legacy runtime WebUI installer;
there are no new application settings, permissions, or end-user workflows.
Each retried npm install or Git clone has at most three attempts. The installer
waits 10 seconds after the first failure and 20 seconds after the second, and
continues immediately after success. If all attempts fail, it exits with the
last command's nonzero status. Its `set -e` execution stops before subsequent
installation stages; callers using `set -e` also stop before subsequent work.
Previously, the retry helper could report success after three failures because
it captured the status of a completed `if` statement instead of the command.
On exhaustion, inspect the npm or Git error and correct the reported cause
before rerunning the installation. The temporary dependency workspace is
removed on exit. Earlier changes to `package.json`, removal of `package-lock.json`,
cache cleaning, and completed dependency installations are not rolled back;
prepare a fresh disposable release workspace when a clean retry is required.
The repair preserves the existing retry count, backoff, cache behavior, and
peer-resolution flags. It does not lift the runtime publication hold tracked in
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
Review the historical `--legacy-peer-deps` workaround separately before lifting
that hold. Strict disposable Git-release and signed catalog verification do not
use this installer; strict release verification must not bypass peer checks.
See [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md) for release context.
Run the isolated regression suite from the meta repository:
```sh
python3 -m unittest -v tests.test_webui_release_dependency_retries
```
The suite executes the actual Bash installer and a caller using `set -e`, with
local npm, Git, Node, and sleep stubs. It covers success on attempts one, two, and
three, final failure status, backoff, and termination at each retry call site.
It performs no network access, real waiting, or changes to the real npm cache.
It checks shell control flow, not package resolution or runtime publication.
## Deutsch
Dieser Betriebshinweis beschreibt
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
und die unter [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54)
erfasste Korrektur des Rückgabestatus. Er richtet sich an Release-Administratoren,
die den bisherigen WebUI-Installer für Laufzeit-Releases verwenden. Neue
Anwendungseinstellungen, Berechtigungen oder Endanwenderabläufe entstehen nicht.
Jede wiederholte npm-Installation und jeder Git-Klon erhält höchstens drei
Versuche. Nach dem ersten Fehlschlag wartet der Installer 10 Sekunden, nach dem
zweiten 20 Sekunden; nach einem Erfolg fährt er sofort fort. Scheitern alle
Versuche, endet er mit dem letzten von null verschiedenen Rückgabestatus.
Durch `set -e` werden nachfolgende Installationsschritte nicht ausgeführt;
auch aufrufende Skripte mit `set -e` brechen vor ihren nächsten Schritten ab.
Bisher konnte die Hilfsfunktion nach drei Fehlschlägen Erfolg melden, weil sie
den Status der abgeschlossenen `if`-Anweisung statt des Befehls übernahm.
Prüfen Sie nach dem Abbruch die npm- oder Git-Fehlermeldung und beheben Sie deren
Ursache vor einem erneuten Installationslauf. Das temporäre Verzeichnis für
Abhängigkeiten wird beim Beenden entfernt. Vorherige Änderungen an `package.json`,
das Entfernen von `package-lock.json`, die Cache-Bereinigung und abgeschlossene
Installationen werden nicht zurückgerollt. Bereiten Sie bei Bedarf einen neuen
temporären Release-Arbeitsbereich für einen sauberen Wiederholungslauf vor.
Die Korrektur erhält Anzahl und Wartezeiten der Versuche, Cache-Verhalten und
Optionen zur Peer-Auflösung. Die Sperre für Laufzeitveröffentlichungen aus
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52) bleibt bestehen.
Der bisherige Einsatz von `--legacy-peer-deps` muss vor ihrer Aufhebung gesondert
geprüft werden. Die strenge Git-Release-Prüfung in einem temporären Arbeitsbereich
und die Prüfung signierter Kataloge verwenden diesen Installer nicht; die strenge
Release-Prüfung darf Peer-Prüfungen nicht umgehen. Weitere Zusammenhänge erläutert
[Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md).
Führen Sie die isolierten Regressionstests im Meta-Repository aus:
```sh
python3 -m unittest -v tests.test_webui_release_dependency_retries
```
Die Tests führen den tatsächlichen Bash-Installer und ein aufrufendes Skript mit
`set -e` aus. Lokale Testprogramme ersetzen npm, Git, Node und sleep. Geprüft werden
Erfolge im ersten, zweiten und dritten Versuch, der letzte Fehlerstatus,
Warteintervalle und der Abbruch an jeder Aufrufstelle. Es gibt keine
Netzwerkzugriffe, echten Wartezeiten oder Änderungen am tatsächlichen npm-Cache.
Die Tests prüfen den Shell-Ablauf, nicht die Paketauflösung oder Veröffentlichung.
+80
View File
@@ -0,0 +1,80 @@
# GovOPlaN 0.1.45 — usability, reliability and security hardening
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
The exact independently versioned composition is recorded in
`packages/govoplan-meta/pyproject.toml`; unchanged modules retain their versions.
This source release does not by itself establish a deployed or independently
approved production environment. Package, signed catalog and runtime publication
results are recorded separately in the coordination issue.
## Runtime publication hold
The [runtime image audit](../security/RUNTIME_IMAGE_AUDIT_2026-09-08.md) completed
eleven registry-only amd64 scans, but found unresolved vulnerabilities and
inventory gaps. Runtime publication remains held separately from this source
release. Patch-only image updates are insufficient; maintained minor-line
changes, narrowly evidenced finding decisions, arm64/final-layer scans and
deployment checks remain necessary. No audited candidate was automatically
adopted and no image was executed during those scans.
The remaining gates are tracked in
[GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
## Included changes
- Shared page/action placement, reusable navigation grouping/editing, table and
dialog sizing, field alignment, multi-select filters and predictable tree
selection. Files, Mail, Search, Notifications and domain pages use the same
contracts, with browser regression coverage.
- Campaign draft saving and independent Mail/ZIP-policy repair, persistent and
bulk message review, clearer delivery eligibility, bounded configurable
synchronous delivery, guarded workerless recovery, lightweight SMTP/IMAP
progress, reused IMAP connections and recipient-complete reporting.
- Files archive staging/reuse, unpacking previously uploaded archives, numeric
progress and bounded traversal. Optional native archive acceleration retains
the same validation rules; portable fallbacks remain available.
- Mail credential references and IMAP folder-name decoding; help topics can be
found by area and tags without expanding every occurrence of the same topic.
- Authentication provenance/scope and browser-cache hardening, patched rich-text
dependencies, spreadsheet/archive/template/Dataflow resource limits, batched
Docs/Notifications queries and safe Reporting bind names. See the
[security/performance review](../security/SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
for measurements, test evidence and remaining limitations.
- A deterministic governance-journey clock fixture, fresh-process Campaign
import coverage, and a new Cases patch aligning its root npm facade with its
Python/WebUI package. Historical published tags are not rewritten.
- Git-root WebUI package facades are aligned with their owning packages, with
a cross-composition parity check. Tasks is included in default module
discovery; it remains subject to enabled modules and normal permissions.
## Upgrade and verification
Back up the database and file storage before upgrading. Apply the complete
selected migration graph before starting the new API/workers. This release
includes additive repair migrations `c58a2d7e9f10` (Core ownership history) and
`d8f1b4e7a0c3` (Access external-function mappings), plus Campaign delivery-state
migrations. Existing business evidence is retained; a schema downgrade is not
a substitute for a reviewed backup/restore plan. Restart API and worker
processes together after upgrading their matching packages.
Updated UI consumers require Core 0.1.45 where they use its new shared contracts.
Tenant keys that previously relied on unintended system permissions/wildcards
must be corrected; the release does not preserve that unsafe behavior. Extremely
sparse spreadsheets, oversized generated output and excessive archive paths
can now fail early with a diagnostic.
For archive staging across multiple hosts, provide shared POSIX storage with
working locks or sticky routing. Background delivery still needs configured
workers; increasing the synchronous limit does not create a worker or guarantee
delivery after a process failure. An unknown SMTP outcome must be reconciled,
not automatically resent.
After deployment, manually verify login/logout and least-privilege API keys,
Campaign Settings and independent Mail/ZIP saves, archive upload/unpack,
recipient-complete reports, and SMTP/IMAP progress with an explicitly approved
test mailbox. No release verification sends real campaign mail automatically.
Hard process isolation, forced-password-change/recovery enforcement, bounded
Xrechnung subprocess output and large-history pagination remain separate open
issues. This release is not a claim that all security or performance debt is
resolved. Production-image scans and multi-host evidence must refer to the
actual signed runtime being deployed.
@@ -0,0 +1,125 @@
# Runtime image candidate audit — 8 September 2026
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
Canonical remediation: [GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
This follow-up to the [source security/performance review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
records registry-only scans of nine proposed runtime dependencies and two
same-minor patch candidates. **Runtime publication is held:** patch-only updates
do not resolve the baseline. Source/package publication is a separate outcome.
No images were executed, rebuilt, selected for CI, or published by this audit.
## Method and reproducible evidence
Official Trivy **0.74.0** was installed only in a private local task directory,
without sudo or Docker access. Its Linux-64bit release archive matched both the
official checksums file and GitHub release asset metadata:
- Archive SHA256: `2ae6fe3ee734b7fdf11335663e18c75ea12dccc76062f09f164a3b0f8be4371a`.
- Checksums-file SHA256: `bc701c3c3ee8b9acbea2c23257e41381e3854888f51281616a6ba5dc96963821`.
- Vulnerability database schema 2, updated `2026-09-07T19:06:01.154199452Z`,
downloaded from `mirror.gcr.io/aquasec/trivy-db:2`.
- Scan flags: `--image-src remote --platform linux/amd64 --scanners vuln
--format json --no-progress --timeout 8m --max-image-size 2GB --exit-code 0`.
Findings were counted from validated JSON; exit zero did not mean clean.
- Existing Docker credentials were not read; no private keys or secrets were
used. Checksums over official HTTPS metadata were verified, not independent
Sigstore signatures. See the [official release](https://github.com/aquasecurity/trivy/releases/tag/v0.74.0)
and [registry-only scan documentation](https://trivy.dev/docs/latest/target/container_image/).
Raw evidence is retained locally, not committed:
`/home/zemion/.cache/govoplan-trivy-remote.yKZgjDOg/scan/`.
It contains eleven `reports/*-amd64.json` reports/logs, scanner scripts,
`patch-candidate-inspection.json`, exact successor registry indices, and
`evidence-checksums.json`. Summary SHA256 values:
- `summary.json`: `f2785a731d637452ab9c0b1f5399772c0f8828a63ca83d5fa7496abdad1c757a`.
- `patch-summary.json`: `b3fc6273fcdad98864040ccdf3477ecf379afd46e9f94444b1f2910f48c1d85b`.
All eleven executions succeeded without timeout/rate-limit failure. Initial
summary fields distinguish `scan_execution_complete: true` from
`coverage_complete: false`: Garage has no detectable package inventory.
Checksums preserve evidence identity, not indefinite storage availability.
## Exact requested pins and results
All references below use `docker.io/`. Counts are package-vulnerability records,
not distinct CVEs or confirmed exploitable application defects. A vulnerability
can appear against several installed packages. Unfixed/unknown records remain.
| Image tag | Exact index SHA256 | Critical / High / Medium / Low / Unknown | Fixable C/H |
| --- | --- | --- | ---: |
| `library/python:3.12-slim-bookworm` | `782412e85d0f0984994c290652577d4018aff08145c85b262bb63dc0c7522254` | 5 / 55 / 102 / 103 / 5 | 0 |
| `library/postgres:16-alpine` | `cf78e76683b9ca8c5733cbbdce6c9262b45b6767934dd0a95e671f9a0fc20685` | 1 / 30 / 28 / 14 / 1 | 31 |
| `library/redis:7-alpine` | `ff02b58f971e7d7d156a1267e283fcbbeee91773b6aa36c49dac28ecfe28eadf` | 0 / 0 / 0 / 0 / 0 | 0 |
| `nginxinc/nginx-unprivileged:1.29-alpine` | `0c79d56aee561a1d81c63f00eee5fb5fe29279560cdc55e91425133104c7fbe6` | 0 / 33 / 74 / 37 / 20 | 33 |
| `library/haproxy:3.2.21-alpine` | `66e25cc9a8332635f4e897f7f4b1e5622c25f09f0ee23cddc6ce9bdb3a24772a` | 0 / 2 / 6 / 12 / 0 | 2 |
| `library/caddy:2.10.2-alpine` | `4c6e91c6ed0e2fa03efd5b44747b625fec79bc9cd06ac5235a779726618e530d` | 7 / 75 / 67 / 37 / 4 | 82 |
| `dxflrs/garage:v2.3.0` | `866bd13ed2038ba7e7190e840482bc27234c4afaf77be8cfa439ae088c1e4690` | **Unknown: no inventory** | — |
| `greenmail/standalone:2.1.9` | `3ac5a83dd6727cf95e4d50e18907fb8ee7bbf5f67e8534714dee2fb1b5b2e1d4` | 0 / 0 / 116 / 35 / 0 | 0 |
| `tonistiigi/binfmt:qemu-v10.2.3-68` | `400a4873b838d1b89194d982c45e5fb3cda4593fbfd7e08a02e76b03b21166f0` | 0 / 9 / 2 / 1 / 1 | 9 |
## Patch-only options and limits
Complete publisher tag listings were inspected for nginx 1.29, Caddy 2.10,
HAProxy 3.2, GreenMail 2.1 and binfmt qemu10.2. Two newer candidates were
scanned; their registry index bytes matched both registry and publisher digests,
and contained amd64 and arm64 manifests:
- `library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`:
same Alpine 3.24.1, 24 detected OS packages, zero reported findings. This is
a useful candidate, not a completed compatibility test or application audit.
- `greenmail/standalone:2.1.13@sha256:3df66b7edd01c8a301343ca5e3601d8674760d4708655573560c24745e624fb2`:
upstream changes Ubuntu 22.04 to Debian 13.6; **3 C / 80 H / 98 M / 85 L /
5 unknown**, 30 fixable C/H records. Not selected as a no-base-change update.
- nginx's newest matching Alpine patch is already 1.29.8 at the scanned pin;
Caddy 2.10 remains 2.10.2; binfmt qemu10.2 remains 10.2.3-68. No newer matching
publisher images were found. The current [official Caddy image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy)
uses 2.11.4; switching minor lines requires new scans and compatibility checks.
Priority remediation: Caddy's own seven HIGH records require fixes through
2.11.4, with additional bundled Go/library fixes that must be re-scanned;
nginx's packages include curl/libcurl fixes through 8.22.0-r0, OpenSSL 3.5.8-r0,
c-ares 1.34.8-r0, expat 2.8.1-r0 and libuuid 2.41.6-r1. PostgreSQL's OS records
require OpenSSL 3.5.8-r0 and libuuid 2.42.3-r1; its CRITICAL plus 21 HIGH Go
records concern the **gosu helper**, not PostgreSQL server code. binfmt's nine
HIGH records concern its Go 1.26.4 build, with fixes through 1.26.6. Package
presence does not establish vulnerable-symbol reachability. No unscanned tag
is claimed to meet every fix requirement.
## Python triage and coverage caveats
Python image metadata identifies CPython 3.12.14, but Trivy inventories only
Debian packages and pip, **not CPython/stdlib**. All 60 C/H records concern
Debian packages: 21 CVEs, 50 `affected` records, 9 `fix_deferred`, 1
`will_not_fix`, without a recorded fixed Bookworm version. Five util-linux CVEs
repeat across eight binary packages. These remain installed; they are not all
removed build dependencies. Pip 25.0.1 separately has five MEDIUM/one LOW
records, with fixes through 26.2.0; it is install tooling, and the API image uses
an offline `--no-index` wheelhouse rather than an arbitrary package index.
Narrow triage examples, **not blanket exemptions**:
- Debian states [CVE-2023-45853](https://security-tracker.debian.org/tracker/CVE-2023-45853)
does not affect the built Bookworm zlib binaries because vulnerable minizip
code is not included. Other bundled minizip implementations are separate.
- [CVE-2026-8376](https://security-tracker.debian.org/tracker/CVE-2026-8376)
explicitly requires 32-bit Perl; this scan targets amd64.
- [CVE-2025-7458](https://security-tracker.debian.org/tracker/CVE-2025-7458)
requires crafted arbitrary SQLite SQL; the managed runtime uses PostgreSQL,
but alternate SQLite use must be reviewed.
- Perl's regex and Archive::Tar records need exact binary/module applicability
checks; vendor-deferred status alone is not a finding dismissal.
Only amd64 was scanned. arm64, newly built GovOPlaN API/Web layers and optional
dependency combinations remain unverified. Garage has no inventory; Redis,
HAProxy and PostgreSQL source-built executables, CPython and QEMU static
binaries need supplemental SBOM/source coverage. Zero detected OS findings is
not zero application vulnerabilities. Trivy also lacks Alpine 3.24 EOL metadata
and nginx CVE-2026-80256 detail; unknowns are retained. There were no runtime,
exploitability, secret, misconfiguration, malware or signature-policy checks.
Before lifting the runtime hold: approve and test maintained image-line changes
where necessary, fix or narrowly disposition findings with evidence, close
inventory gaps, scan both architectures and final runtime layers, then run
deployment/ingress smoke checks. Do not silently change base OS, use unpinned
`latest`, rebuild third-party images, or accept all HIGH/CRITICAL findings.
@@ -0,0 +1,139 @@
# Runtime image remediation follow-up — 8 September 2026
Canonical tracking: [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
and [website #9](https://git.add-ideas.de/add-ideas/addideas-govoplan-website/issues/9).
This addendum supplements the [original audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md);
it does not replace that historical baseline or lift either publication or
deployment gate. The immutable 0.1.45 release and `catalog-v0.1.45` are unchanged.
## Source change and candidate decisions
New installer specifications now use
`haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
This is a patch update from 3.2.21 within the supported
[3.2 LTS branch](https://www.haproxy.org/), keeping Alpine 3.24.1.
The [publisher's exact build source](https://github.com/docker-library/haproxy/blob/7a5c202cde713867a737033dca56e7a211a8b8df/3.2/alpine/Dockerfile)
and scanned image configuration retain the non-root `haproxy` user,
`/usr/local/etc/haproxy/haproxy.cfg`, entrypoint and graceful-stop signal.
The [upstream changelog](https://www.haproxy.org/download/3.2/src/CHANGELOG)
includes HTTP parsing, TLS and memory-safety fixes in 3.2.22/3.2.23.
Existing specifications retain their explicit image, including an older pin;
this source change does not update a running installation.
| Candidate | OS | C / H / M / L / Unknown, per architecture | Disposition |
| --- | --- | --- | --- |
| HAProxy `3.2.23-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Installer source default updated; binary/runtime checks pending |
| nginx-unprivileged `1.30.4-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Candidate only; compatibility and website OS upgrade review pending |
| Caddy `2.11.4-alpine` | Alpine 3.23.5 | 1 / 38 / 41 / 12 / 23 | Not selected; all 39 C/H records have recorded fixes |
| Node `24-alpine` (24.20.0) | Alpine 3.24.1 | 0 / 6 / 11 / 12 / 0 | Not selected; major change and six fixable HIGH records |
Each row was scanned separately for **linux/amd64 and linux/arm64**, with the
same counts on both. Counts are package-vulnerability records, not distinct
CVEs or proven exploits. The [machine-readable evidence](runtime-image-candidates-2026-09-08.json)
contains exact index, platform-manifest, config and report digests, inventory
counts, scanner bounds and decisions. It is audit data, not an accepted release
manifest or an installer input.
nginx's candidate reference is
`docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce`.
It inventories 70 Alpine packages including nginx/NJS, uses UID 101 and exposes
8080. The [upstream stable release](https://nginx.org/en/download.html) and
[security advisories](https://nginx.org/en/security_advisories.html) include the
1.30.4 fixes. The publisher retains its
[unprivileged port and temporary-path contract](https://github.com/nginx/docker-nginx-unprivileged).
For the website, this changes nginx 1.27.5 to 1.30.4, NJS 0.8.10 to 1.0.1 and
Alpine 3.21.3 to 3.24.1. These changes are explicit review items; the website
Dockerfile has not been changed. The GovOPlaN Web image still requires an
explicit verified `NGINX_IMAGE` build argument.
Caddy's candidate reference is
`docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648`.
Although this is the current
[official image line](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy),
its inventory still includes Go 1.26.3, `x/crypto` 0.52.0, `x/net` 0.55.0,
`x/text` 0.37.0 and gRPC 1.81.0. Recorded fixes include Go 1.26.6,
`x/crypto` 0.55.0, `x/net` 0.56.0, `x/text` 0.39.0 and gRPC 1.83.1; Alpine
findings also remain in c-ares, curl/libcurl and OpenSSL. The CRITICAL
`CVE-2026-56854` concerns `x/crypto/ssh` source-address enforcement. A module
record alone does not establish that this binary exposes that SSH path; exact
binary symbol/reachability analysis is still required for a disposition.
The [official Node image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/node)
still maps Node 22 Alpine to 22.23.2 and the previously scanned digest. Node 24's
candidate is
`docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf`.
Its HIGH records remain in two OpenSSL packages and npm dependencies
`brace-expansion`, `ip-address` and `tar`; fixing the earlier critical tar
record alone is insufficient. The website builder stays on Node 22 pending
a reviewed build-tool remedy and a final builder scan.
## Method, verification and retained evidence
The existing Trivy 0.74.0 executable was rehashed against the previously verified
archive member: `d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791`.
The same schema-2 vulnerability database was used, updated
`2026-09-07T19:06:01.154199452Z`. No tool installation or database refresh occurred.
Index bytes matched both the registry digest header and Docker Hub publisher
metadata; both platform-manifest byte hashes matched the index. All eight
registry-only scans completed successfully with validated JSON, `--list-all-pkgs`,
`--scanners vuln`, an eight-minute/2GB image bound, an empty Docker configuration
and no inherited credentials. Exit zero means execution succeeded. Private
temporary paths and in-memory artifact cache isolated this follow-up from the
earlier scanner's artifact cache; its vulnerability database was read only.
Raw reports, logs, manifests, publisher metadata and the scanner script are in
`/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh/`:
- `summary.json` SHA-256: `0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56`.
- `frozen-images.json` SHA-256: `d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467`.
- `scan_successors.py` SHA-256: `f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07`.
Local retention is not permanent artifact hosting; preserve this evidence with
the eventual reviewed release. The JSON evidence records compressed registry
layer sizes; these are not expanded filesystem limits or final GovOPlaN sizes.
Installer regression checks cover the new generated image pin, legacy
specification fallback, preserved explicit images, generated topology and
configuration: `python -I -m unittest discover -s tests -p
test_deployment_installer.py` ran 45 tests successfully with one skip because
Core was not importable in that isolated test environment. The skipped Core
startup-configuration integration was subsequently rerun in the shared development
environment with Core available: all 45 installer tests passed with no skips,
including generated-environment startup validation. This is configuration
validation, not execution of the candidate image.
Both repositories passed `git diff --check`; the audit JSON and all eight
report hashes were checked against the retained evidence.
**Docker, Podman and HAProxy executables are unavailable on
this host**, so no image or HAProxy configuration was executed and no daemon was
installed. Publisher metadata and installer tests support the scoped source
patch; they do not establish binary or deployed compatibility.
## Gates that remain open
- Validate `haproxy -c` on generated local, existing-proxy and managed-ingress
configurations using the exact pinned image and target architectures. Run
bounded isolated checks without live mounts, secrets, privilege or external
network access. Then verify DNS discovery, readiness, forwarded headers,
replica routing and graceful termination in the intended runtime.
- Test the nginx candidate with both the website configuration and GovOPlaN
WebUI entrypoint/proxy configuration, including UID 101, writable temporary
paths, health paths, cache headers and static catalog bytes. Approve the
website nginx/NJS/Alpine version changes before changing its Dockerfile.
- Resolve Caddy, Node build-tool and all unchanged baseline dependencies with
updated publisher images or narrow reviewed applicability evidence. No
severity-wide exceptions or custom third-party rebuilds were introduced.
- Close the original source-built/static inventory gaps. HAProxy's 24-package
OS inventory still omits the source-built HAProxy executable. Node's npm
inventory still omits the Node executable/stdlib. Garage, CPython, Redis,
PostgreSQL and QEMU gaps are unchanged. Alpine 3.24 EOL metadata is still
missing from this scanner; zero findings is not complete coverage.
- Scan **final built** API/Web/website layers and the selected managed
dependencies on both architectures, then perform migration, worker,
readiness and ingress smoke checks. Record failure and unknown states.
Secrets, misconfiguration and image signature policy need separate checks.
- Obtain the website deployment host/operator and rebuild/restart authority,
preserving the exact immutable catalog/keyring/module-directory bytes and
verifying fresh public responses after an authorized rollout.
No images were built, executed, published or deployed; no running service,
release tag, signed manifest, CI image input or live infrastructure was changed.
@@ -0,0 +1,219 @@
# Security and performance follow-up — 8 September 2026
This follows the [original review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
and its post-release issue reconciliation. It describes new source work after
the frozen 0.1.45 release; it does not change published tags, packages, signed
catalogs or deployed images. Gitea remains the canonical state log.
## Implemented source slices
- [Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
a shared disposable-process runner enforces wall/CPU/address-space/input/output
limits, bounded stderr, process-group cleanup and non-queuing per-process
admission. A private binary codec bounds decoding before allocating a full
object graph and preserves explicitly supported data types without pickle.
Read the owning Core `docs/BOUNDED_PROCESS_CONTRACT.md` before adding callers.
- Connectors XLSX parsing, Templates rendering, Files ZIP/TAR inspection and
extraction, and Dataflow reference previews/development execution now use
that boundary. Existing authorization, sessions, provider credentials,
idempotency and persistence remain in the parent. No unprotected inline
fallback is used. Each module contributes static EN/DE user/admin limits and
operational consequences through its manifest.
- Files snapshots authorized sources inside shared admission, validates private
staged members, and acknowledges each persisted member before decoding the
next. Numeric progress remains available. The acknowledgement is event-driven,
not a fixed sleep per member. Reads allocate by validated actual file size,
not by the configured ceiling. Failures reap children, clear private staging
and retain the existing transaction/blob cleanup and explicit retry behavior.
- Dataflow's normal reference preview formerly bypassed the backend wrapper;
it now enters the worker too. Nested source configurations cannot collide
merely because subflows reuse node IDs. Combined reference-source data is
checked before creating further columnar copies, while individual providers
retain their own authorized-read bounds. Staging/production still require
DuckDB; this change does not replace that separate backend.
- [Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
current-password change, session/CSRF rotation, cross-tenant session and human
API-key revocation, and optional administrator-assisted recovery. Recovery
codes are hashed, single-use, expire after 15 minutes, require a current local
System owner and explicit identity verification, and recheck current account,
membership, tenant and issuer authority at redemption. A password change also
invalidates outstanding codes issued by that account for other people. Audit
evidence and validation/error responses do not contain passwords or codes.
External-provider and service-account rules remain separate.
- The Access UI provides first-login/required change, self-service change,
policy-aware sign-in help, public code redemption and eligible owner issuance.
Core consumes an optional lazy auth-action capability rather than importing
Access internals. The required-action gate fails closed if its UI is missing.
- [Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
full-history lists batch pinned revisions, while new summary and bounded
step/event endpoints preserve authorization and explicit pagination. Existing
full-history responses are not silently truncated. Exact inbox total semantics
are retained and their counting cost is documented.
- [Meta #55](https://git.add-ideas.de/GovOPlaN/govoplan/issues/55):
shared version/planning helpers recognize the existing nested developer
package, not invented root metadata. All tag batches enforce trusted private
source ownership, registered origins and clean main/upstream state. Meta
batches additionally require exact composition and matching Core evidence.
Whole-batch preflight,
frozen source receipts, annotated immutable tags, object-pinned atomic
publication and post-effect remote checks are covered with temporary local
repositories. Hidden Git index flags, unsafe ancestry, alternates and changed
Git-directory identities are rejected. Selected version/composition metadata
must be tracked, so ignored files cannot describe bytes absent from a tag.
Applicable unselected Core WebUI inputs have bounded, frozen read receipts;
backend-only releases do not read them. The existing local module-candidate
exception remains intact. The weaker legacy mutation path was removed.
Canonical whole-package preview and receipt-bound apply now cover Meta's
version preparation too. Core must already match the target. Preparation
requires a separate trusted checkout, explicit out-of-run confirmation and
unchanged source/tooling receipts; it cannot rewrite the running operator.
Plans place Meta after Core and explain the manual preparation/publication
steps instead of promising a durable self-update. Ambiguous partial writes
require reconciliation, without automatic rollback or retry.
- [Runtime-image follow-up](RUNTIME_IMAGE_REMEDIATION_2026-09-08.md): eight new
registry-only scans cover four exact candidates on amd64 and arm64. New
installer specifications select the patched same-line HAProxy digest;
existing specifications retain their explicit image. Other candidates and
unresolved inventory/deployment gates remain visible, not blanket-approved.
## Verification record
Targeted checks include actual child execution, catastrophic regex CPU,
aggregate memory exhaustion, TAR extension metadata, noisy output, malformed
transport/staging data, Unicode allocation limits, cancellation/callback
failures, descendant cleanup, rollback and explicit retries. Local mixed-owner
composition tests completed nine real children, rejected six overlapping
requests as busy, observed at most one unreaped child and recovered all slots.
This is local admission evidence, not a target deployment load certification.
Workflow fixtures serialize 40 different pinned revisions with five SQL reads;
summary lists use one query for 40 ordinary rows. Exact inbox totals for
40/400/4,000 candidates used one query, with measured local costs approximately
0.011/0.057/0.492 seconds. These are fixture measurements, not production SLOs.
The broader Core API smoke suite exposed three stale campaign assertions.
All three failures were reproduced against the unchanged private frozen 0.1.45
sources. Updated fixtures verify recipient-summary projection, detailed payload
separation and explicit fenced recovery of a confirmed stopped runtime; observing
SENDING alone must not make a claim recoverable. All 76 smoke tests then passed.
No production Campaign behavior was changed to satisfy these tests.
The final release-tool suite passed 279 tests and 68 subtests, including
temporary local remotes and adversarial source/tag/receipt changes. Rechecking
the whole batch before effects is deliberately conservative: its repeated
filesystem/Git/remote work grows quadratically with batch size. It is not a
new unattended publication path or permission to execute unreviewed source.
Strict interface inventory now reports no unclassified endpoints and exact
contextual help for all 133 high-risk controls. Seventeen password browser cases
include actual F1 help from the restricted screen, empty workspace scopes,
EN/DE layouts, Unicode boundaries and no credential values in help URLs.
The initial production bundle remains within the unchanged limits (512,036
raw bytes and 162,415 gzip bytes; 1,713 gzip bytes below its ceiling), with
46 optional descriptors and no eager optional-module imports.
The focused checker now includes the new Core process, mixed-owner admission,
Access password, Templates and Files worker tests, the repaired campaign smoke
cases, and browser-side auth/password transport contracts. The full focused run
passed, including 63 production module/build permutations and all 230 browser
cases. Its two opt-in Datasources PostgreSQL cases were skipped in that run
and subsequently passed against the isolated real database described below.
The final Meta preparation gate was added after that full run and verified
with the owning release-tool suite and the focused release-gate command.
Manifest validation passed for all 72 modules. The full focused log is
`/mnt/DATA/tmp/govoplan-security-followup-20260908-focused.log`.
The first follow-up quick audit captured an unchanged 79-repository snapshot
in `/mnt/DATA/tmp/govoplan-security-followup-quick-20260908-7s8Sgh/`.
All four required scanners completed, with zero missing/execution reports;
all 168 report checksums and 163 machine-readable reports were validated.
Gitleaks found no secrets in all 79 histories and 79 worktrees. Local Semgrep
rules reported zero findings. Production Bandit reported 65 low and four medium
warnings, and production Ruff retained 54 warnings. The two added Bandit
warnings identify the new Core subprocess import and invocation: trusted
server-owned arguments, no shell, and the documented resource/process boundary
were reviewed; warnings remain visible. This is report-only evidence, not a
warning-free audit or a penetration test. A final snapshot follows the
cross-module declaration/contextual-help corrections and release-tool checks.
That final audit completed on 8 September, 05:59:4606:02:18 UTC, in
`/mnt/DATA/tmp/govoplan-security-final-quick-20260908-vAwQIh/`. All 79 start/end
source fingerprints were identical; all four scanners completed, all 168
registered report checksums matched, and all 163 JSON/SARIF reports parsed.
There were no missing reports or scanner execution errors. Semgrep and both
Gitleaks scopes again reported zero findings. Production counts were unchanged
from the first follow-up: Bandit 65 low/four medium and Ruff 54. Test-only
counts were Bandit 140 low/34 medium and Ruff 136. A separate frozen scan of
all ten changed Meta release/deployment Python files reported seven low Bandit
and four Ruff S603 warnings, with no execution errors. Its four argv-only
subprocess sites were reviewed; the preparation additions introduced no new
warnings. No findings were hidden or severity-wide exceptions added.
The audit manifest SHA-256 is
`a997b786239cd11443cb665d5f9041a968cc38f9d49171e68bb868bf2bd73310`;
its report-checksum list SHA-256 is
`dc590ca5b0a4e445019a05536d410226088d67b40d61cd7657bdef4a4eae56d8`.
The audit includes the eight committed feature/website source changes and the
final uncommitted Meta source. Only this evidence document was updated after
the source freeze ended; the final Meta commit and remote publication are
recorded in the linked Gitea issues, not inferred from local audit completion.
Fresh dependency audits are retained in
`/mnt/DATA/tmp/govoplan-dependency-final-20260908-d24LEK/`: all four full npm
lockfile audits (Core WebUI, Mail root/WebUI and website) report zero known
vulnerabilities. Installed Python auditing covers 137 distributions with zero
known vulnerabilities; 51 local GovOPlaN distributions lack PyPI advisory
coverage. Core's 46 linked packages are likewise not claimed covered by public
registry advisories. All 12 dependency-file hashes and the installed inventory
were unchanged. No packages were installed or automatically fixed.
Managed PostgreSQL 16.15 fixtures used private Unix sockets, synthetic roles
and databases, no TCP listener, per-case schemas and bounded SQL/lock waits.
Both previously skipped Datasources races passed. Twenty-one existing Access
password HTTP tests and four additional races passed on PostgreSQL: single-use
redemption, stale-session/password replacement, competing issuance, and issuer
password revocation during redemption. Four release/development migration checks
also passed for Access and Workflow, including credential preservation and
idempotent indexes. The four races are now owning opt-in Access regressions;
see `govoplan-access/docs/PASSWORD_RECOVERY_POSTGRES_TESTS.md`. These local
database checks do not certify a deployment, fleet load or external recovery
handover. With both explicit PostgreSQL test URLs enabled, the full Access suite
passed 122 tests and 18 subtests, and the full Datasources suite passed 57 tests,
without skips. Access retained 12 existing SQLite datetime-adapter warnings in
its separate SQLite migration cases. Both temporary PostgreSQL fixtures were
stopped and independently verified: no server process, private socket,
generated schema or synthetic cluster remains. Scripts, logs and shutdown
receipts are retained under `/home/zemion/.cache/govoplan-pg-security-20260908.RAUitg/`
and `/home/zemion/.cache/govoplan-pg-promoted-20260908.JZcIKL/`.
## Adoption and remaining gates
1. `AUTH_LOCAL_PASSWORD_RECOVERY_ENABLED` remains **false** by default. The
existing flag is still advisory until an operator explicitly adopts and
enables the complete recovery policy. Confirm who verifies identity and how
the one-time code is handed over; automated email recovery is not enabled.
Test first-login, lost-password, code expiry and administrator availability
in the target environment before enforcement.
2. Access migration `e9a2c5f8b1d4` adds recovery evidence; Workflow migration
`9e6b3f8a2c7d` adds summary-pagination indexes. Use normal backed-up upgrade
procedures and account for index-build cost. No manual live migration or
server restart was performed during this work. The user's existing devserver
has automatic reload, so live schema state must not be assumed unchanged.
3. Release preparation must assign new source/package versions and require a
Core version containing the new worker/auth contracts in the affected module
metadata, including matching WebUI assets. The old immutable release must
not be relabelled or treated as containing these APIs.
4. Resource limits are not an arbitrary-code, filesystem or network sandbox.
Admission is per API/worker process, not fleet-wide. Validate Linux/cgroup
memory, disk quotas, process counts, cancellation and legitimate large-file
workloads on the intended runtime before increasing concurrency. Core #297
retains this target-evidence follow-up.
5. Meta #52 and website #9 retain runtime-image/publication/deployment holds.
Docker/Podman/HAProxy executables are unavailable here. Final built images,
binary/source inventories, ingress behavior, migration/readiness/worker
smoke checks and the website's target/operator authority remain outstanding.
Zero findings in a detected package inventory is not full image coverage.
No real messages, IMAP appends, password resets, provider operations or deployment
actions were used as test fixtures. Development tests use temporary databases,
private temporary files, mock transports and managed test-browser servers.
@@ -0,0 +1,207 @@
# Security and performance review — 8 September 2026
Coordinated status: [Core #296](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/296).
This records a workspace-wide automated scan, targeted manual boundary review,
and a verified implementation pass. It is not a penetration test, an exhaustive
line-by-line review, or a security certification. The audit was completed on
local, unpublished changes, preserving existing worktree changes. Subsequent
release preparation/publication is tracked in
[GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51) and the
[0.1.45 release notes](../releases/0.1.45.md).
## Implemented findings
| Area | Finding and change | Evidence / ownership |
| --- | --- | --- |
| Authentication — high | Preserve service-account provenance and current scope ceilings instead of recalculating them as ordinary membership permissions. Tenant API keys cannot retain canonical system permissions or unsafe wildcard grants. | Previously failing isolated regressions; [Access #21](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/21). |
| Authentication — medium | Warm-cache API keys must follow the same explicit-header credential rules as cold authentication. A session cookie cannot turn an API key into a session credential. | Regression covering source-dependent authentication. |
| Browser authority/cache — medium | Clear reusable data on auth changes and write settlement; fence late 200/304 writes and obsolete 401 side effects. Honor server no-store/no-cache and explicit fresh-read requests. Interactive login/logout remove retained automation keys that could shadow cookie-session identity. | 23 real-client regressions. Unchanged settings keep their object identity, preventing profile-fetch loops. Core `docs/API_CLIENT_CACHE_CONTRACT.md`; owning Access EN/DE session/field documentation. |
| Spreadsheet resource exhaustion | Validate actual XLSX coordinates before openpyxl traversal; ignore misleading declared dimensions; count blank row gaps toward the existing limits. | [Connectors #18](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/18), 13 tests and 2 subtests. |
| Template resource exhaustion | Enforce the existing 5 MiB output budget during substitution and item construction, including UTF-8, HTML escaping and separators. | [Templates #7](https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/7), full 20 tests; independent 3,000-case valid-output comparison. |
| Archive resource exhaustion | Inspect regular TAR member limits before traversing payloads. Limit archive paths to 4,096 UTF-8 bytes / 128 components and count derived directories against entry limits. | [Files #46](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/46), 55 archive and 15 documentation tests. Extension-header decoding still needs stronger isolation. |
| Dataflow resource exhaustion | Reject LPAD/RPAD target lengths above the existing 1,000,000-byte preview budget before fill evaluation/allocation. Preserve final serialized-byte checks. | [Dataflow #22](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/22), full 104 tests and 39 subtests; 7 new guard tests independently rerun. |
| Docs performance / defense in depth | Batch revision reads per request, avoid loading pending draft bodies for readers, and validate tenant/entry/publication consistency while retaining owner/audience checks. | [Docs #22](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/22), full 39 tests. |
| Notifications performance / defense in depth | Batch delivery-attempt loading while preserving recipient checks and rejecting inconsistent attempt references, including already-loaded relationships. | [Notifications #6](https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/6), full 23 tests. |
| Session-list performance | Apply active/expiry predicates and the existing 100-row cap in SQL, before loading session history. | Query-shape regression in Access. |
| Reporting correctness | Use structural bind-name suffixes for recursive calculated measures, preserving valid dotted/hyphenated public keys and parameter uniqueness. | [Reporting #10](https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/10), full 29 tests. |
| Audit hygiene | Redact Gitleaks logs and machine reports on current, history and legacy scanner paths. | 13 audit-wrapper tests enforce the flag. |
All changed module workflows/limits have owning EN/DE DocumentationTopic updates.
Independent review found no concrete regression in the backend changes.
## Measured performance changes
These are SQL-query counts in isolated 40-item fixtures, not production latency
or throughput claims. Authorization is still evaluated for each request.
| Projection | Before | After |
| --- | ---: | ---: |
| Docs reader entries | 41 SELECTs | 2 SELECTs |
| Docs editor entries | 81 SELECTs | 2 SELECTs |
| Notification list with attempts | 41 SELECTs | 2 SELECTs |
The Docs 401-entry batching regression uses 3 SELECTs. Resource guards reject
oversized work before the formerly expensive allocation/traversal. This does
not make every legitimate upload or campaign faster. Honoring no-cache can
increase server validation requests; ETags still avoid retransmitting unchanged
bodies. That authorization/freshness trade-off is deliberate.
The original audit snapshot measured 517,380 initial JavaScript bytes and
164,119 gzip bytes. Release preparation's pure-defaults split reduces this to
516,730 initial bytes and 163,908 gzip bytes. Restoring the missing Tasks
descriptor then measures 516,987 initial / 163,976 gzip bytes with all 46 module
descriptors lazy, within the unchanged 524,288 / 164,128 caps. The gzip margin is still small; future
startup work should reduce eager dependencies, not raise the cap automatically.
The full 209-case browser suite passed before the split, followed by 13 focused
browser checks after it. Radon recorded 238 rank-D-or-higher entries; complexity
is a review-priority signal, not a performance measurement.
## Dependency remediation
Core's full npm audit went from 30 affected package entries to zero. Most initial
entries were transitive effects of the same Tiptap advisory, not 30 independent
application exploits. The website went from two affected entries to zero; both
Mail lockfiles also report zero.
- Tiptap packages are aligned at 3.31.3, with direct minimum ranges raised to
3.30.4 in both development and release manifests, with a parity regression.
Added an actual installed-library prototype-attribute regression for
the [maintainer's security advisory](https://github.com/ueberdosis/tiptap/security/advisories/GHSA-cp6q-959q-f8rh).
- Core now resolves xmldom 0.9.12, browserslist 4.28.9 and nanoid 3.3.18.
The website's affected browserslist/nanoid dependencies are patched too.
- Development/audit requirements now require pip >=26.2; the local development
environment uses 26.2.1. The installed audit originally flagged
[CVE-2026-13346](https://github.com/advisories/GHSA-qwm4-qh6w-59xr), requiring an
attacker-controlled package index. This is an installation-tool vulnerability,
not evidence of an exposed application endpoint.
The final installed Python audit enumerated 188 distributions: 137 were
auditable with zero known vulnerabilities, and 51 local distributions were not
available in PyPI. Those skips are covered by source review, not by a claim of
dependency-advisory coverage. Production images and every optional dependency
combination were not independently resolved or scanned.
## Scan coverage and limitations
Evidence directory:
`/mnt/DATA/tmp/govoplan-security-performance-20260908-gsk8jn/`.
The final `final-quick/manifest.json` captures 79 repositories, tool versions,
start/end repository fingerprints, report checksums, 168 report artifacts and
163 validated JSON/SARIF reports. It records an unchanged workspace, complete
coverage for its four required scanners, no execution errors and no missing
reports. It ran in report-only mode: exit zero does **not** mean zero warnings.
- Final production Bandit: 447,008 Python lines; 67 warnings (63 low, 4 medium),
no high findings. Ruff security rules: 54 warnings. SQL-construction warnings
were reviewed against identifier/operator validation and bound values in
DuckDB/Reporting; no injection fix was warranted there. XML import warnings
were checked: feed/BPMN input parsing uses defusedxml; stdlib imports support
types/output construction. Operator-owned fenced-run argv is not a public
arbitrary-command endpoint. Xrechnung output buffering remains a follow-up.
Assertions and error-swallowing markers remain review/maintenance warnings,
not proof that all such code is harmless.
- Final local Semgrep rules: no findings. The broader OWASP-rule pass applied
272 rules to 4,169 tracked targets. Its seven warnings recommended weakening
owner-only 0700 permissions; they were rejected as false positives. One
Calendar rule timeout was rerun with a 60-second budget: zero findings/errors.
Bash and conformance TypeScript checks passed despite two scanner-specific
parser limitations. Ignored/dependency/generated paths are not a complete
line-by-line source audit.
- Gitleaks: 79 Git histories plus 79 worktrees, 158 redacted reports, zero
detected secrets. This does not establish that deployed credentials are safe
or that formerly exposed credentials have been rotated.
- Tool versions included Semgrep 1.176.1, Bandit 1.9.4, Ruff 0.15.21 and
Gitleaks 8.30.1. The downloaded Gitleaks binary archive matched the official
release SHA-256 before execution.
- The containerized full-toolbox path could not access Docker's daemon. Its
full-mode Trivy/misconfiguration and additional OSV scans were **not** run.
A subsequent [registry-only runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md)
successfully scanned nine pinned candidates and two same-minor successors
for amd64 without Docker. It found unresolved vulnerabilities and inventory
gaps; runtime publication is held. This does not complete full-toolbox,
arm64, final-runtime-image or deployment coverage.
No live application probes, database changes, file operations, mail sends,
IMAP appends, imports, notification delivery, deployments, commits or pushes
were performed. Browser tests used isolated mocked fixtures. Package installs,
builds and temporary audit-tool installation were local development operations.
## Verification and remaining work
- 209/209 browser conformance tests pass; production Core/website builds,
conformance TypeScript, 24 Core client/dependency regressions, 4 real-client
Files reload checks, and 72/72 manifest checks pass.
- Access's full 91-test suite passed before the final documentation-only update;
the final documentation suite passed all 4 tests. Other module counts appear
above. The new authentication/resource tests include demonstrated pre-fix
failures rather than only structural assertions.
- The original focused workspace run stopped at the institutional
governance/Portal fixture (`tests/test_institutional_governance_journey.py:223`,
`IndexError`). Release preparation fixes its mixed clocks using the existing
temporal context, retaining validity-boundary exclusions; 7 journey tests and
ambient-year checks pass. Tracked in
[Meta #50](https://git.add-ideas.de/GovOPlaN/govoplan/issues/50).
- Campaign's apparent host-path issue was ruled out by existing tracked
API/build/snapshot guards and 11 passing tests under normal initialization.
Release preparation fixes the standalone import cycle through a deferred
resolver import without changing validation rules. Fresh-process coverage,
all 11 path tests and Campaign's full 611-test suite pass.
Next coordinated work:
1. [Hard resource isolation — Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
regex CPU, aggregate allocation, TAR extension metadata, bounded workers and
cancellation, followed by production-like concurrent load tests.
2. [Forced password change/recovery — Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
the current flag is advisory only. Do not enable enforcement without a usable
local-password/recovery flow and external-provider rules.
3. [Bound subprocess output — Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2):
enforce the existing 2 MiB limit while draining stdout/stderr, not afterwards.
4. [Workflow revision batching/history projection — Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
batch evidence lookups; separately define explicit history pagination and
authorized-total semantics. Docs/notification history volumes also remain.
5. Resolve the [runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md) findings
tracked in [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
and coverage gaps before lifting its publication hold; complete deployment
audits, review exposed development credentials and worker quotas, and
benchmark realistic tenant sizes/concurrency. The sanctions
transport's fixed HTTPS/redirect allowlist is not a demonstrated arbitrary-URL
issue, but migration to Core's pinned egress transport remains desirable.
Operational compatibility: tenant keys relying on accidental system/wildcard
permissions must be corrected rather than weakening the guard. Extreme sparse
spreadsheets, overly deep/long archive paths and oversized padding intermediates
can now fail early with diagnostics. No stored documents or configurations were
deleted or silently migrated.
## Post-release follow-up — 2026-09-08
The findings and scanner counts above describe the original audit snapshot.
The following source fixes are subsequent to the frozen `0.1.45` composition;
they do not change its immutable tags or published package bytes.
- [Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2)
now enforces the existing shared 2 MiB stdout/stderr limit during execution
and kills/reaps the direct validator on overflow, timeout or cancellation.
Report reads are bounded to 16 MiB plus one probe byte before interpretation.
The 30-test module suite passes; noisy-child and report-read regressions were
also demonstrated to fail against the previous source. Owning EN/DE static
documentation is updated. POSIX pipe capture is required; disk quotas,
descendant isolation and process-level CPU/memory limits remain separate work.
- [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54) now preserves
the last command's failure status after exhausted installer retries. Twelve
isolated stage/scenario combinations cover every retry call site, success,
backoff and caller termination under `set -e`. The test is included in the
focused checks and installer CI. See the bilingual
[installer retry note](../operations/WEBUI_RELEASE_DEPENDENCY_RETRIES.md).
These are unreleased follow-up source changes, not a new runtime release or
deployment. The runtime-image hold under Meta #52 remains in force; the
historical peer-dependency workaround still needs its separate review.
Further implementation and adoption gates are tracked in the
[security follow-up](SECURITY_FOLLOWUP_2026-09-08.md), including disposable
parsing/execution workers, opt-in password recovery, workflow read projections
and the newer runtime-image evidence. The original scanner counts above remain
historical and are not silently replaced by later test results.
@@ -0,0 +1,305 @@
{
"schema_version": 1,
"purpose": "Audit evidence only; not a release manifest or active installer configuration.",
"observed_at": "2026-09-08T03:56:47.666808+00:00",
"runtime_publication_held": true,
"website_deployment_held": true,
"scan_execution_complete": true,
"coverage_complete": false,
"scanner": {
"name": "Trivy",
"version": "0.74.0",
"binary_sha256": "d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791",
"database_metadata": {
"Version": 2,
"NextUpdate": "2026-09-08T19:06:01.154199291Z",
"UpdatedAt": "2026-09-07T19:06:01.154199452Z",
"DownloadedAt": "2026-09-07T23:30:53.198039224Z"
},
"source": "remote",
"scanners": [
"vuln"
],
"list_all_packages": true,
"images_executed": false,
"existing_docker_credentials_used": false,
"timeout": "8m",
"maximum_image_size": "2GB"
},
"evidence": {
"private_directory": "/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh",
"summary_sha256": "0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56",
"frozen_images_sha256": "d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467",
"scanner_script_sha256": "f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07"
},
"candidates": [
{
"name": "haproxy",
"image": "docker.io/library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
"disposition": "source_default_updated_binary_runtime_verification_pending",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:0666a2c2f41d341084ed2da85392b48cdcd766adfa28231f31305724ed5c6ea5",
"config_digest": "sha256:9621d75e50a8f26d3738ae3cdbf15e98c6aa5cd48e6baca0699492de502156f5",
"compressed_layer_bytes": 20516844,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 24
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "40cfc3db74e29f09723f38698660ccdd50ac935c8d6e1e75c6e0555b3e9361fb",
"log_sha256": "46881f695780f89c037d5b0b4dc0ded9ea4e459077c7658d80b786efc5754084"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:cd20b9dc6b4713956a2a043997001a1948167d345e7c8d5bd5ff2e667166651f",
"config_digest": "sha256:19796bff8905d4a46c9463c576203b20cac8984d41b7b01ea9cbff55e8422c34",
"compressed_layer_bytes": 20970772,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 24
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "0e1326c58abf358d592fa55c94333228ce1094edf4e489fcc4ece6e32d3320f6",
"log_sha256": "397c2fbf1044cf50733d68b4b9cc4eb68211d96f1f302d5e9f8af0d11fb0de1c"
}
]
},
{
"name": "nginx-stable",
"image": "docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce",
"disposition": "candidate_pending_compatibility",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:b8c179cd3c2ae222a873dd59fbae240fadc03836cae5198afc9e9c19919c3880",
"config_digest": "sha256:8b5953dae38d27a76bca22373bb920fd6ce8d9d7da21578d2926e678002de8a0",
"compressed_layer_bytes": 25526590,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 70
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "3ad164dae3cdf891b12e41451b8a8e65a5e1688824a7c01d848e1274363e6bf9",
"log_sha256": "0f99ff3d9b4396de48655bf8299df30c14ba0c579f480d37baa7d2f6a4c11f1d"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:b6742a0cbd749add25346658991c3da06a8e38796949df120fed78db8c512576",
"config_digest": "sha256:4d8b10f5d2ff99e7aa5f161930d8a4693a86a26f288ec8c0d4cd47f2ef5af179",
"compressed_layer_bytes": 25892370,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 70
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 0,
"MEDIUM": 0,
"LOW": 0,
"UNKNOWN": 0
},
"fixable_high_critical": 0,
"unique_cves": 0,
"report_sha256": "506fdeb97525ed8e4d9ae538c42bab7aa2217f662a7135f0f12d16d20410c7a6",
"log_sha256": "c41ef9ea091d00f18c7a097404672591bee85e7477ae17d8f5ca771d8e42b2bb"
}
]
},
{
"name": "caddy",
"image": "docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648",
"disposition": "not_selected_remaining_fixable_findings",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:98eb57d882ccd5213d1688764db10c1ca2c58a1ca3a6717a3411ad798f7a423a",
"config_digest": "sha256:af555904a0961945f16bb323a501457b13a4f7e9bde969b145b97da80b38ecbe",
"compressed_layer_bytes": 23907283,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.23.5"
},
"inventory": [
{
"type": "alpine",
"packages": 32
},
{
"type": "gobinary",
"packages": 146
}
],
"counts": {
"CRITICAL": 1,
"HIGH": 38,
"MEDIUM": 41,
"LOW": 12,
"UNKNOWN": 23
},
"fixable_high_critical": 39,
"unique_cves": 68,
"report_sha256": "e483352a1d5b9b97950dcf92e4dfcf4600f5b09306af3d0640b10ca229a07779",
"log_sha256": "9cd599d6dd8c101b421fdeb57036cec1f69f815d765a8bf1f850ec60061036f4"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:1172d4213087d3fc30bafc7ff2c2896180eb0c41ff7f75f315568fb36cabdcba",
"config_digest": "sha256:6b08c1b9858ca9a7d99c1da13c3695081e0e604c6cf214ca26a7ce0e2c4fd9b4",
"compressed_layer_bytes": 22722712,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.23.5"
},
"inventory": [
{
"type": "alpine",
"packages": 32
},
{
"type": "gobinary",
"packages": 146
}
],
"counts": {
"CRITICAL": 1,
"HIGH": 38,
"MEDIUM": 41,
"LOW": 12,
"UNKNOWN": 23
},
"fixable_high_critical": 39,
"unique_cves": 68,
"report_sha256": "20e04d820e3b27d57575ea177e523e23109fd83344cdf57e184a4d2fad27e803",
"log_sha256": "a1d9c37aa7948c137db64040d95e5930c5859a8acd71ac5bc41ff168e2b270c5"
}
]
},
{
"name": "node-lts",
"image": "docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf",
"disposition": "not_selected_remaining_fixable_findings",
"platforms": [
{
"platform": "linux/amd64",
"manifest_digest": "sha256:4caaaf42195bcd6f6f3559a413b20cb8f8ad089e231ee874cf7701643966689f",
"config_digest": "sha256:ee289c69ed1ac50a5a042112ea97f132800e2dd53e832da27784f00e45b3289c",
"compressed_layer_bytes": 58486244,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 18
},
{
"type": "node-pkg",
"packages": 146
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 6,
"MEDIUM": 11,
"LOW": 12,
"UNKNOWN": 0
},
"fixable_high_critical": 6,
"unique_cves": 19,
"report_sha256": "c927d995dde700c92027f6328dc6c273f0f1445cd8154301480757cb962e02b2",
"log_sha256": "c7dc1900cbe39c9f91e228b2770bcbd394ec2914d2b3879478295fc7f8f77ab3"
},
{
"platform": "linux/arm64",
"manifest_digest": "sha256:d3724e44ee368606d753e0027eb8d2a94fc1f275e5d9e4620178a12edb655f5f",
"config_digest": "sha256:722cc1507731edf58a4c0bc3e29553c44ce5774d0849242c1b237abc79926a0a",
"compressed_layer_bytes": 58935654,
"scan_exit_code": 0,
"os": {
"Family": "alpine",
"Name": "3.24.1"
},
"inventory": [
{
"type": "alpine",
"packages": 18
},
{
"type": "node-pkg",
"packages": 146
}
],
"counts": {
"CRITICAL": 0,
"HIGH": 6,
"MEDIUM": 11,
"LOW": 12,
"UNKNOWN": 0
},
"fixable_high_critical": 6,
"unique_cves": 19,
"report_sha256": "e5f7799761f23cefc36929381b4186eeb3afb46a2a559c789d12a7eea5dc30d0",
"log_sha256": "fd24671f0da4d3b20dc8bcc2718531e6f6e19155d49bed15958965e21dd10813"
}
]
}
]
}
+68 -64
View File
@@ -4,85 +4,89 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "govoplan" name = "govoplan"
version = "0.1.37" version = "0.1.46"
description = "Developer convenience package for a versioned GovOPlaN composition" description = "Developer convenience package for a versioned GovOPlaN composition"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" } license = { text = "AGPL-3.0-or-later" }
dependencies = [ dependencies = [
"govoplan-core[server]==0.1.37", "govoplan-core[server]==0.1.46",
"govoplan-tenancy==0.1.18", "govoplan-tenancy==0.1.22",
"govoplan-organizations==0.1.18", "govoplan-organizations==0.1.21",
"govoplan-identity==0.1.18", "govoplan-identity==0.1.21",
"govoplan-idm==0.1.20", "govoplan-idm==0.1.26",
"govoplan-access==0.1.20", "govoplan-access==0.1.25",
"govoplan-admin==0.1.19", "govoplan-admin==0.1.23",
"govoplan-policy==0.1.20", "govoplan-policy==0.1.23",
"govoplan-audit==0.1.19", "govoplan-audit==0.1.20",
"govoplan-dashboard==0.1.18", "govoplan-dashboard==0.1.20",
"govoplan-files==0.1.20", "govoplan-files==0.1.27",
"govoplan-mail==0.1.22", "govoplan-mail==0.1.28",
"govoplan-campaign==0.1.24", "govoplan-campaign==0.1.29",
"govoplan-calendar==0.1.18", "govoplan-calendar==0.1.24",
"govoplan-docs==0.1.21", "govoplan-docs==0.1.23",
"govoplan-ops==0.1.19", "govoplan-ops==0.1.22",
] ]
[project.optional-dependencies] [project.optional-dependencies]
full = [ full = [
"govoplan-addresses==0.1.18", "govoplan-addresses==0.1.23",
"govoplan-approvals==0.1.18", "govoplan-approvals==0.1.21",
"govoplan-assets==0.1.19", "govoplan-assets==0.1.20",
"govoplan-booking==0.1.19", "govoplan-booking==0.1.20",
"govoplan-cases==0.1.20", "govoplan-cases==0.1.25",
"govoplan-certificates==0.1.19", "govoplan-certificates==0.1.20",
"govoplan-committee==0.1.18", "govoplan-committee==0.1.22",
"govoplan-connectors==0.1.22", "govoplan-connectors==0.1.27",
"govoplan-consultation==0.1.19", "govoplan-consultation==0.1.20",
"govoplan-contracts==0.1.19", "govoplan-contracts==0.1.20",
"govoplan-dataflow==0.1.20", "govoplan-dataflow==0.1.25",
"govoplan-datasources==0.1.21", "govoplan-datasources==0.1.26",
"govoplan-decisions==0.1.19", "govoplan-decisions==0.1.19",
"govoplan-dist-lists==0.1.18", "govoplan-dist-lists==0.1.21",
"govoplan-encryption==0.1.18", "govoplan-dms==0.1.20",
"govoplan-evaluation==0.1.19", "govoplan-encryption==0.1.20",
"govoplan-facilities==0.1.19", "govoplan-erp==0.1.20",
"govoplan-forms==0.1.20", "govoplan-evaluation==0.1.20",
"govoplan-forms-runtime==0.1.18", "govoplan-facilities==0.1.20",
"govoplan-grants==0.1.19", "govoplan-fit-connect==0.1.20",
"govoplan-helpdesk==0.1.20", "govoplan-forms==0.1.23",
"govoplan-identity-trust==0.1.18", "govoplan-forms-runtime==0.1.22",
"govoplan-inspections==0.1.19", "govoplan-grants==0.1.20",
"govoplan-learning==0.1.19", "govoplan-helpdesk==0.1.21",
"govoplan-identity-trust==0.1.21",
"govoplan-inspections==0.1.20",
"govoplan-learning==0.1.20",
"govoplan-mandates==0.1.19", "govoplan-mandates==0.1.19",
"govoplan-notifications==0.1.18", "govoplan-notifications==0.1.20",
"govoplan-parties==0.1.19", "govoplan-parties==0.1.19",
"govoplan-payments==0.1.20", "govoplan-payments==0.1.22",
"govoplan-permits==0.1.19", "govoplan-permits==0.1.20",
"govoplan-poll==0.1.19", "govoplan-poll==0.1.20",
"govoplan-portal==0.1.19", "govoplan-portal==0.1.22",
"govoplan-postbox==0.1.19", "govoplan-postbox==0.1.23",
"govoplan-procurement==0.1.19", "govoplan-procurement==0.1.20",
"govoplan-projects==0.1.18", "govoplan-projects==0.1.20",
"govoplan-quick-access==0.1.19", "govoplan-quick-access==0.1.21",
"govoplan-records==0.1.20", "govoplan-records==0.1.24",
"govoplan-reporting==0.1.18", "govoplan-reporting==0.1.22",
"govoplan-resources==0.1.19", "govoplan-resources==0.1.20",
"govoplan-rest==0.1.19", "govoplan-rest==0.1.19",
"govoplan-risk-compliance==0.1.18", "govoplan-risk-compliance==0.1.21",
"govoplan-scheduling==0.1.18", "govoplan-scheduling==0.1.22",
"govoplan-search==0.1.18", "govoplan-search==0.1.20",
"govoplan-services==0.1.19", "govoplan-services==0.1.19",
"govoplan-soap==0.1.19", "govoplan-soap==0.1.19",
"govoplan-tasks==0.1.20", "govoplan-tasks==0.1.23",
"govoplan-templates==0.1.18", "govoplan-templates==0.1.22",
"govoplan-tickets==0.1.20", "govoplan-tickets==0.1.23",
"govoplan-transparency==0.1.19", "govoplan-transparency==0.1.20",
"govoplan-views==0.1.19", "govoplan-views==0.1.22",
"govoplan-voting==0.1.18", "govoplan-voting==0.1.21",
"govoplan-wiki==0.1.20", "govoplan-wiki==0.1.22",
"govoplan-workflow==0.1.21", "govoplan-workflow==0.1.23",
"govoplan-workflow-engine==0.1.19", "govoplan-workflow-engine==0.1.21",
"govoplan-xrechnung==0.1.21",
] ]
[project.urls] [project.urls]
@@ -91,6 +91,12 @@ Form launch, persisted submission provenance, idempotent replay, resumable
assisted intake with enforced read-back evidence, and a durable Workflow handoff assisted intake with enforced read-back evidence, and a durable Workflow handoff
that remains visible through Tasks after the database session is reopened and that remains visible through Tasks after the database session is reopened and
disappears only after the Workflow Engine records completion. disappears only after the Workflow Engine records completion.
Core's production-component browser conformance suite additionally executes the
German self-service and assisted Anwohnerparkausweis paths at desktop and mobile
widths. It proves native keyboard order, accessible names and landmarks, WCAG
2.1 A/AA automation, responsive geometry, first-draft persistence, and mixed
per-field person/document/system provenance. Physical screen-reader spot checks
remain target-environment release evidence.
Module-level Records source tests prove exact Form submission, Case revision, Module-level Records source tests prove exact Form submission, Case revision,
and Decision revision filing. Target-environment browser accessibility, and Decision revision filing. Target-environment browser accessibility,
production identity and delivery, a named archive profile, and recovery evidence production identity and delivery, a named archive profile, and recovery evidence
+1 -1
View File
@@ -2,7 +2,7 @@ bandit>=1.8,<2
click>=8.3.3 click>=8.3.3
filelock>=3.20.3 filelock>=3.20.3
idna>=3.15 idna>=3.15
pip>=26.1.2 pip>=26.2
pip-audit>=2.9,<3 pip-audit>=2.9,<3
python-multipart>=0.0.31 python-multipart>=0.0.31
radon>=6,<7 radon>=6,<7
+1 -1
View File
@@ -56,7 +56,7 @@ httpx2>=2.5,<3
filelock>=3.20.3 filelock>=3.20.3
idna>=3.15 idna>=3.15
jsonschema>=4,<5 jsonschema>=4,<5
pip>=26.1.2 pip>=26.2
pip-audit>=2.9,<3 pip-audit>=2.9,<3
pytest>=9.0.3,<10 pytest>=9.0.3,<10
pygments>=2.20,<3 pygments>=2.20,<3
+15 -15
View File
@@ -1,18 +1,18 @@
# Whole-product release install from immutable, independently versioned module tags. # Whole-product release install from immutable, independently versioned module tags.
# Only add a module after its referenced tag has been published. # Only add a module after its referenced tag has been published.
../govoplan-core[server] ../govoplan-core[server]
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.18 govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.18 govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.18 govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.20 govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.26
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.20 govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.19 govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.20 govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.19 govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.18 govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.20 govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.27
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.22 govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.28
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.24 govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.29
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.18 govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.24
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.21 govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.19 govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
+4 -2
View File
@@ -85,13 +85,15 @@
"The configured applicant email issues a short-lived, hash-only status link through Notifications and exposes only the bounded status timeline.", "The configured applicant email issues a short-lived, hash-only status link through Notifications and exposes only the bounded status timeline.",
"An idempotent replay returns the same persisted submission.", "An idempotent replay returns the same persisted submission.",
"The human review handoff survives a database-session restart and remains visible in Tasks until completion.", "The human review handoff survives a database-session restart and remains visible in Tasks until completion.",
"The production self-service and assisted WebUI paths preserve keyboard order, accessible names, WCAG 2.1 A/AA automation, and responsive geometry at desktop and mobile widths.",
"The assisted operator can assign independent source, confidence, and governed declaring-party, document, or system references to every populated field before immutable read-back.",
"The formal decision retains party, mandate, legal-basis, evidence, delivery, review, and exact revision references.", "The formal decision retains party, mandate, legal-basis, evidence, delivery, review, and exact revision references.",
"The Case-bound payment handoff creates a replay-safe obligation and accepts a full manual receipt only with exact amount, currency, transaction reference, and immutable evidence.", "The Case-bound payment handoff creates a replay-safe obligation and accepts a full manual receipt only with exact amount, currency, transaction reference, and immutable evidence.",
"Forms Runtime, Cases, and Decisions can expose exact snapshots for explicit eAkte filing." "Forms Runtime, Cases, and Decisions can expose exact snapshots for explicit eAkte filing."
], ],
"manual_or_target": [ "manual_or_target": [
"Complete the digital journey with keyboard and screen reader at desktop and mobile widths.", "Perform physical screen-reader spot checks for the digital journey at desktop and mobile widths.",
"Complete the assisted operator journey with keyboard and screen reader at desktop and mobile widths.", "Perform physical screen-reader spot checks for the assisted operator journey at desktop and mobile widths.",
"Open, resend, expire, and revoke the applicant status link with keyboard and screen reader at desktop and mobile widths.", "Open, resend, expire, and revoke the applicant status link with keyboard and screen reader at desktop and mobile widths.",
"Verify the configured Postbox or external delivery provider, including unknown outcome and reconciliation.", "Verify the configured Postbox or external delivery provider, including unknown outcome and reconciliation.",
"Restore the pinned composition and reconstruct the exact form, case, decision, delivery evidence, and eAkte chronology.", "Restore the pinned composition and reconstruct the exact form, case, decision, delivery evidence, and eAkte chronology.",
+15
View File
@@ -0,0 +1,15 @@
import assert from "node:assert/strict";
import { findTypeOnlyJsxImports } from "../tools/checks/check-jsx-value-imports.mjs";
const findings = (source) => findTypeOnlyJsxImports([{ path: "/fixture.tsx", source }]).map((item) => item.component);
assert.deepEqual(findings('import type { FormGrid, AuthInfo } from "@govoplan/core-webui"; const page = <Dialog><FormGrid /></Dialog>;'), ["FormGrid"]);
assert.deepEqual(findings('import { type FormGrid as Layout } from "ui"; const page = <Layout>Content</Layout>;'), ["Layout"]);
assert.deepEqual(findings('import type Layout from "ui"; const page = <Layout />;'), ["Layout"]);
assert.deepEqual(findings('import type * as ui from "ui"; const page = <ui.Layout />;'), ["ui.Layout"]);
assert.deepEqual(findings('import type { FormGrid } from "ui"; const page = <div title={<FormGrid />} />;'), ["FormGrid"]);
assert.deepEqual(findings('import { FormGrid, type AuthInfo } from "ui"; const page = <FormGrid />;'), []);
assert.deepEqual(findings('import type { FormGrid } from "ui"; function Page({ FormGrid }: Props) { return <FormGrid />; }'), []);
assert.deepEqual(findings('import type * as ui from "ui"; function Page(ui: RuntimeControls) { return <ui.Layout />; }'), []);
assert.deepEqual(findings('import type { Layout } from "ui"; const page: Layout = {};'), []);
assert.deepEqual(findings('import type { input } from "ui"; const page = <input />;'), []);
console.log("JSX runtime-import AST regression tests passed (10 cases).");
+190
View File
@@ -1,8 +1,10 @@
from __future__ import annotations from __future__ import annotations
from contextlib import redirect_stderr, redirect_stdout from contextlib import redirect_stderr, redirect_stdout
from datetime import UTC, datetime, timedelta
import io import io
import json import json
import os
from pathlib import Path from pathlib import Path
import stat import stat
import subprocess import subprocess
@@ -36,6 +38,7 @@ import govoplan_deploy.cli as deployment_cli # noqa: E402
from govoplan_deploy.capabilities import ( # noqa: E402 from govoplan_deploy.capabilities import ( # noqa: E402
capability_change_impacts, capability_change_impacts,
infrastructure_capability_document, infrastructure_capability_document,
infrastructure_dependency_inventory_from_mapping,
) )
from govoplan_deploy.cluster_evidence import ( # noqa: E402 from govoplan_deploy.cluster_evidence import ( # noqa: E402
collect_kubernetes_evidence, collect_kubernetes_evidence,
@@ -87,6 +90,41 @@ def _kubernetes_test_deployment(component: str, replicas: int) -> dict:
} }
def _dependency_inventory(
installation_id: str,
*,
generated_at: datetime | None = None,
) -> dict:
return {
"schema_version": 1,
"installation_id": installation_id,
"generated_at": (generated_at or datetime.now(UTC)).isoformat(),
"complete": True,
"inspected_capability_ids": ["coordination.redis", "mail.smtp"],
"providers": [
{
"module_id": "mail",
"state": "complete",
"capability_ids": ["mail.smtp"],
"dependency_count": 1,
}
],
"dependencies": [
{
"capability_id": "mail.smtp",
"module_id": "mail",
"dependency_type": "smtp_endpoint",
"dependency_ref": "endpoint:17",
"state": "active",
"scope": "system",
"summary": "Persisted SMTP endpoint has one credential binding.",
"metrics": {"credential_binding_count": 1},
"required_action": "Rebind or migrate this SMTP endpoint.",
}
],
}
class DeploymentInstallerTests(unittest.TestCase): class DeploymentInstallerTests(unittest.TestCase):
def test_kubernetes_evidence_requires_two_node_spread_and_safe_runtime( def test_kubernetes_evidence_requires_two_node_spread_and_safe_runtime(
self, self,
@@ -742,6 +780,11 @@ class DeploymentInstallerTests(unittest.TestCase):
self.assertIn("redis", compose["services"]) self.assertIn("redis", compose["services"])
self.assertIn("worker", compose["services"]) self.assertIn("worker", compose["services"])
self.assertIn("load-balancer", compose["services"]) self.assertIn("load-balancer", compose["services"])
self.assertEqual(
"haproxy:3.2.23-alpine@sha256:"
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
compose["services"]["load-balancer"]["image"],
)
self.assertNotIn("test-mail", compose["services"]) self.assertNotIn("test-mail", compose["services"])
self.assertEqual( self.assertEqual(
["127.0.0.1:8080:8080"], ["127.0.0.1:8080:8080"],
@@ -1034,6 +1077,28 @@ class DeploymentInstallerTests(unittest.TestCase):
self.assertNotIn("old-secret", impacts["database.postgresql"].detail) self.assertNotIn("old-secret", impacts["database.postgresql"].detail)
self.assertNotIn("new-secret", impacts["database.postgresql"].detail) self.assertNotIn("new-secret", impacts["database.postgresql"].detail)
def test_capability_impact_includes_provider_dependency_evidence(self) -> None:
previous_spec = default_spec(mail_mode="test-mail", module_set="full")
desired_spec = default_spec(mail_mode="disabled", module_set="full")
inventory = infrastructure_dependency_inventory_from_mapping(
_dependency_inventory(previous_spec.installation_id)
)
impacts = {
item.capability_id: item
for item in capability_change_impacts(
infrastructure_capability_document(previous_spec, {}),
infrastructure_capability_document(desired_spec, {}),
dependency_inventory=inventory,
)
}
mail = impacts["mail.smtp"]
self.assertTrue(mail.inventory_inspected)
self.assertEqual("endpoint:17", mail.actual_dependencies[0].dependency_ref)
self.assertIn("mail:endpoint:17", mail.detail)
self.assertIn("Rebind or migrate", mail.required_action)
def test_replica_counts_drive_compose_and_load_balancer_discovery(self) -> None: def test_replica_counts_drive_compose_and_load_balancer_discovery(self) -> None:
spec = default_spec( spec = default_spec(
storage_mode="garage", storage_mode="garage",
@@ -1085,8 +1150,26 @@ class DeploymentInstallerTests(unittest.TestCase):
self.assertEqual(1, parsed.replicas.web) self.assertEqual(1, parsed.replicas.web)
self.assertEqual(1, parsed.replicas.worker) self.assertEqual(1, parsed.replicas.worker)
self.assertEqual("managed", parsed.components.load_balancer.mode) self.assertEqual("managed", parsed.components.load_balancer.mode)
self.assertEqual(
default_spec().components.load_balancer.image,
parsed.components.load_balancer.image,
)
self.assertEqual("local", parsed.ingress.mode) self.assertEqual("local", parsed.ingress.mode)
def test_load_balancer_patch_does_not_rewrite_an_existing_image(self) -> None:
for image in (
"haproxy:3.2.21-alpine",
"registry.example.test/haproxy@sha256:" + "a" * 64,
):
with self.subTest(image=image):
saved = default_spec(load_balancer_image=image).to_dict()
restored = parse_spec(json.loads(json.dumps(saved)))
compose = render_compose(restored)
self.assertEqual(image, restored.components.load_balancer.image)
self.assertEqual(image, compose["services"]["load-balancer"]["image"])
def test_compose_contains_no_secret_values(self) -> None: def test_compose_contains_no_secret_values(self) -> None:
spec = default_spec() spec = default_spec()
values = initial_secrets(spec) values = initial_secrets(spec)
@@ -1221,6 +1304,65 @@ class DeploymentInstallerTests(unittest.TestCase):
for check in second_plan.checks for check in second_plan.checks
) )
) )
self.assertTrue(second_plan.blocked)
self.assertTrue(
any(
check.id == "capability.dependency_inventory.missing"
and check.level == "error"
for check in second_plan.checks
)
)
atomic_write(
paths.dependency_inventory,
canonical_json(_dependency_inventory(second_spec.installation_id)),
mode=0o600,
)
evidenced_plan = build_plan(
second_spec,
paths,
include_host_checks=False,
)
self.assertFalse(
any(
check.level == "error"
and check.id.startswith("capability.dependency_inventory.")
for check in evidenced_plan.checks
)
)
self.assertEqual(
"endpoint:17",
{
item.capability_id: item
for item in evidenced_plan.capability_impacts
}["mail.smtp"].actual_dependencies[0].dependency_ref,
)
self.assertTrue(
any(
check.id == "capability.dependency_inventory.current"
and check.level == "ok"
for check in evidenced_plan.checks
)
)
stale = _dependency_inventory(
second_spec.installation_id,
generated_at=datetime.now(UTC) - timedelta(minutes=6),
)
atomic_write(
paths.dependency_inventory,
canonical_json(stale),
mode=0o600,
)
stale_plan = build_plan(second_spec, paths, include_host_checks=False)
self.assertTrue(stale_plan.blocked)
self.assertTrue(
any(
check.id == "capability.dependency_inventory.stale"
for check in stale_plan.checks
)
)
def test_secret_change_is_planned_without_exposing_secret_values(self) -> None: def test_secret_change_is_planned_without_exposing_secret_values(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory: with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
@@ -1330,6 +1472,54 @@ class DeploymentInstallerTests(unittest.TestCase):
)[0], )[0],
) )
def test_cli_collects_bounded_private_dependency_inventory(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
root = Path(directory) / "installation"
self.assertEqual(
0,
run_cli(
[
"init",
"--non-interactive",
"--directory",
str(root),
]
)[0],
)
payload = _dependency_inventory("govoplan-local")
response = MagicMock()
response.__enter__.return_value = response
response.geturl.return_value = "https://ops.example.test/inventory"
response.read.return_value = json.dumps(payload).encode("utf-8")
fetch = MagicMock(return_value=response)
with (
patch.dict(os.environ, {"TEST_OPS_KEY": "secret-api-key"}),
patch.object(deployment_cli, "urlopen", fetch),
):
result, stdout, stderr = run_cli(
[
"collect-infrastructure-inventory",
"--directory",
str(root),
"--ops-url",
"https://ops.example.test/inventory",
"--api-key-env",
"TEST_OPS_KEY",
]
)
self.assertEqual(0, result, stderr)
self.assertIn("1 record(s)", stdout)
evidence_path = root / "infrastructure-dependency-inventory.json"
self.assertEqual(0o600, stat.S_IMODE(evidence_path.stat().st_mode))
self.assertNotIn(
"secret-api-key",
evidence_path.read_text(encoding="utf-8"),
)
request = fetch.call_args.args[0]
self.assertEqual("secret-api-key", request.get_header("X-api-key"))
def test_cli_requires_external_url_when_switching_from_managed(self) -> None: def test_cli_requires_external_url_when_switching_from_managed(self) -> None:
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory: with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
root = Path(directory) / "installation" root = Path(directory) / "installation"
+466
View File
@@ -0,0 +1,466 @@
from __future__ import annotations
import base64
from contextlib import ExitStack
import csv
from copy import deepcopy
import hashlib
import io
import json
import os
from pathlib import Path
import sys
import tarfile
import tempfile
import unittest
from unittest.mock import patch
from urllib.parse import quote
import zipfile
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import full_catalog # noqa: E402
from govoplan_release.artifact_identity import selected_artifact_identity_issues # noqa: E402
from govoplan_release.catalog import canonical_hash # noqa: E402
from govoplan_release.model import RepositorySpec # noqa: E402
from govoplan_release.registry_reference import registry_entry_source # noqa: E402
from govoplan_release.selective_catalog import ( # noqa: E402
load_authenticated_catalog_base, public_key_base64, signature,
)
from govoplan_release.source_provenance import ( # noqa: E402
SourceTagProvenanceIssue, catalog_source_selection,
registered_source_origin_issues,
)
from govoplan_release.version_alignment import candidate_catalog_version_issues # noqa: E402
class FullRegistryCatalogTests(unittest.TestCase):
def setUp(self) -> None:
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.web = self.root / "addideas-govoplan-website"
self.wheels = self.root / "wheels"
self.npm = self.root / "npm"
self.wheels.mkdir(mode=0o700)
self.npm.mkdir(mode=0o700)
self.key = Ed25519PrivateKey.generate()
self.keypath = self.root / "key.pem"
self.keypath.write_bytes(self.key.private_bytes(
serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
serialization.NoEncryption(),
))
self.keypath.chmod(0o600)
self.keyring = {
"keyring_version": "1", "keys": [{
"key_id": "known-key", "public_key": public_key_base64(self.key),
"status": "active",
}],
}
self.base = {
"catalog_version": "1", "channel": "stable", "sequence": 1,
"core_release": {"version": "1.0.0"}, "modules": [],
"release": {},
}
self.write_base()
self.package_set = {
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
"registries": {
"python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
"npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/",
},
"python": [self.package("govoplan-core"), self.package("govoplan-demo")],
"webui": [self.package("govoplan-core", webui=True)],
}
self.seal(self.package_set, "package_set_sha256")
self.lock = {
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
"registries": self.package_set["registries"],
"package_set_sha256": self.package_set["package_set_sha256"],
"python": [], "webui": [],
}
for row in self.package_set["python"]:
path = self.wheel(row["name"])
url = full_catalog._tool("resolve-package-artifacts")["_python_artifact_url"](
self.package_set["registries"]["python"], package=row, filename=path.name,
)
self.lock["python"].append(self.artifact(row, path, url))
npm_package = self.package_set["webui"][0]
npm_path = self.npm / "govoplan-core-webui-1.2.3.tgz"
self.tarball(npm_path, "@govoplan/core-webui", "1.2.3")
url = self.package_set["registries"]["npm"] + quote(npm_package["name"], safe="") + "/-/1.2.3/core-webui-1.2.3.tgz"
row = self.artifact(npm_package, npm_path, url)
row["integrity"] = "sha512-" + base64.b64encode(hashlib.sha512(npm_path.read_bytes()).digest()).decode()
self.lock["webui"].append(row)
self.seal(self.lock, "lock_sha256")
self.set_path = self.root / "package-set.json"
self.lock_path = self.root / "package-lock.json"
self.write_inputs()
self.output = self.root / "candidate"
@staticmethod
def package(repo: str, *, webui: bool = False) -> dict:
row = {
"name": "@govoplan/core-webui" if webui else repo, "version": "1.2.3",
"repository": repo, "tag": "v1.2.3",
"commit": ("a" if repo == "govoplan-core" else "b") * 40,
}
if not webui:
row["extras"] = ["server"] if repo == "govoplan-core" else []
return row
@staticmethod
def artifact(package: dict, path: Path, url: str) -> dict:
encoded = path.read_bytes()
return {**package, "filename": path.name, "url": url,
"sha256": hashlib.sha256(encoded).hexdigest(), "size": len(encoded)}
@staticmethod
def seal(payload: dict, field: str) -> None:
payload.pop(field, None)
payload[field] = hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
def write_inputs(self) -> None:
self.set_path.write_text(json.dumps(self.package_set))
self.lock_path.write_text(json.dumps(self.lock))
def write_base(self) -> None:
self.base.pop("signatures", None)
self.base["signatures"] = [signature(self.base, key_id="known-key", private_key=self.key)]
folder = self.web / "public/catalogs/v1"
(folder / "channels").mkdir(parents=True, exist_ok=True)
(folder / "channels/stable.json").write_text(json.dumps(self.base))
(folder / "keyring.json").write_text(json.dumps(self.keyring))
def wheel(self, package: str) -> Path:
stem = package.replace("-", "_")
info = f"{stem}-1.2.3.dist-info"
files = {
f"{stem}/__init__.py": b"VALUE = 1\n",
f"{info}/METADATA": f"Metadata-Version: 2.1\nName: {package}\nVersion: 1.2.3\n".encode(),
f"{info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
}
record = io.StringIO()
writer = csv.writer(record, lineterminator="\n")
for name, value in files.items():
writer.writerow((name, "", len(value)))
writer.writerow((f"{info}/RECORD", "", ""))
files[f"{info}/RECORD"] = record.getvalue().encode()
path = self.wheels / f"{stem}-1.2.3-py3-none-any.whl"
with zipfile.ZipFile(path, "w") as archive:
for name, value in files.items():
archive.writestr(name, value)
return path
@staticmethod
def tarball(path: Path, name: str, version: str, *, duplicate: bool = False) -> None:
encoded = json.dumps({"name": name, "version": version}).encode()
with tarfile.open(path, "w:gz") as archive:
for _ in range(2 if duplicate else 1):
member = tarfile.TarInfo("package/package.json")
member.size = len(encoded)
archive.addfile(member, io.BytesIO(encoded))
def build(self, *, provenance_errors=(), origin_errors=()) -> dict:
registry_generator = full_catalog._tool("generate-release-catalog")
tools = {name: dict(full_catalog._tool(name)) for name in (
"generate-release-catalog", "generate-release-package-set", "resolve-package-artifacts",
)}
tools["generate-release-package-set"]["generate_package_set"] = lambda **kwargs: self.package_set
self.provenance = {
row["repository"]: {"commit_sha": row["commit"], "tag_object_sha": str(index + 1) * 40}
for index, row in enumerate(self.package_set["python"])
}
entry = {
"module_id": "demo", "name": "Demo", "version": "1.2.3",
"python_package": "govoplan-demo",
"python_ref": "govoplan-demo @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-demo.git@v1.2.3",
}
with ExitStack() as stack:
stack.enter_context(patch.object(full_catalog, "_tool", side_effect=lambda name: tools[name]))
stack.enter_context(patch.dict(registry_generator["_catalog_payload"].__globals__, {
"synthesize_repository_catalog_entries": lambda **kwargs: (dict(entry),),
}))
stack.enter_context(patch.object(full_catalog, "enforce_selected_version_alignment"))
stack.enter_context(patch.object(full_catalog, "registered_source_origin_issues", return_value=origin_errors))
self.provenance_check = stack.enter_context(patch.object(full_catalog, "source_tag_provenance_issues", return_value=provenance_errors))
stack.enter_context(patch.object(full_catalog, "selected_source_provenance", return_value=self.provenance))
return full_catalog.build_full_registry_candidate(
package_set_path=self.set_path, package_lock_path=self.lock_path,
wheelhouse=self.wheels, webui_packages=self.npm, output_dir=self.output,
selected_repositories=("govoplan-core",),
signing_keys=(f"known-key={self.keypath}",), workspace_root=self.root,
)
def candidate(self) -> dict:
return json.loads((self.output / "channels/stable.json").read_text())
def test_full_candidate_uses_registry_bytes_and_preserves_unchanged_tag_provenance(self) -> None:
result = self.build()
candidate = self.candidate()
self.assertEqual("ready", result["status"])
self.assertEqual(2, result["package_count"])
self.assertEqual(1, result["selected_count"])
self.assertEqual(self.keyring, json.loads((self.output / "keyring.json").read_text()))
self.assertEqual(canonical_hash(self.keyring), candidate["release"]["keyring_sha256"])
self.assertEqual(2, len(candidate["release"]["artifacts"]))
self.assertIn("/pypi/files/", candidate["core_release"]["python_ref"])
self.assertEqual((), candidate_catalog_version_issues(candidate))
self.assertEqual((), selected_artifact_identity_issues(candidate))
sources = catalog_source_selection(candidate)
self.assertEqual((), sources.issues)
self.assertEqual({"govoplan-core": "1.2.3"}, sources.selected_versions)
self.assertEqual({"govoplan-core": "1.2.3", "govoplan-demo": "1.2.3"}, sources.all_versions)
self.assertEqual("b" * 40, sources.selected_commits["govoplan-demo"])
self.assertEqual("2" * 40, sources.selected_tag_objects["govoplan-demo"])
self.assertEqual(2, self.provenance_check.call_count)
for call in self.provenance_check.call_args_list:
self.assertEqual({"govoplan-core"}, call.kwargs["require_head_repos"])
for path in [self.output, *self.output.rglob("*")]:
self.assertEqual(0o700 if path.is_dir() else 0o600, path.stat().st_mode & 0o777)
def test_legacy_base_is_authenticated_but_remains_rejected_by_selective(self) -> None:
self.build()
with self.assertRaisesRegex(ValueError, "does not pin"):
load_authenticated_catalog_base(
base_catalog=None, base_keyring=None, web_root=self.web,
channel="stable", public_base_url="https://unused.example",
signer_public_keys={"known-key": public_key_base64(self.key)},
)
def test_injected_key_or_mismatched_pinned_keyring_is_rejected(self) -> None:
for mutation in ("extra-key", "bad-hash"):
with self.subTest(mutation=mutation):
if mutation == "extra-key":
self.keyring["keys"].append({"key_id": "injected", "status": "active", "public_key": public_key_base64(Ed25519PrivateKey.generate())})
else:
self.keyring["keys"] = self.keyring["keys"][:1]
self.base["release"]["keyring_sha256"] = "f" * 64
self.write_base()
with self.assertRaises(ValueError):
self.build()
self.assertFalse(self.output.exists())
def test_tampered_base_signature_is_rejected(self) -> None:
path = self.web / "public/catalogs/v1/channels/stable.json"
payload = json.loads(path.read_text())
payload["sequence"] = 999
path.write_text(json.dumps(payload))
with self.assertRaisesRegex(ValueError, "signature verification"):
self.build()
def test_wrong_registry_bytes_and_reused_candidate_fail_closed(self) -> None:
self.build()
original = (self.output / "channels/stable.json").read_bytes()
with self.assertRaisesRegex(ValueError, "must not already exist"):
self.build()
self.assertEqual(original, (self.output / "channels/stable.json").read_bytes())
self.output = self.root / "candidate-2"
wheel = self.wheels / self.lock["python"][0]["filename"]
with wheel.open("ab") as stream:
stream.write(b"tampered")
with self.assertRaisesRegex(ValueError, "bytes differ"):
self.build()
self.assertFalse(self.output.exists())
def test_webui_identity_and_url_changes_are_rejected(self) -> None:
row = self.lock["webui"][0]
original = row["url"]
for url in (
original + "?alternate=true", original.replace("/-/1.2.3/", "/-/9.9.9/"),
original.replace("/npm/", "/npm/../other/"),
original.replace("/npm/", "/npm/%2e%2e/other/"),
original.replace("/npm/", "/npm/%252e%252e/other/"),
):
with self.subTest(url=url):
row["url"] = url
self.seal(self.lock, "lock_sha256")
self.write_inputs()
with self.assertRaisesRegex(ValueError, "URL differs"):
self.build()
def test_duplicate_missing_and_symlinked_artifacts_are_rejected(self) -> None:
row = self.lock["webui"][0]
self.lock["webui"].append(dict(row))
with self.assertRaisesRegex(ValueError, "duplicate/missing"):
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
self.lock["webui"].pop()
path = self.npm / row["filename"]
moved = self.root / "moved.tgz"
path.rename(moved)
path.symlink_to(moved)
with self.assertRaises(OSError):
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
def test_archive_metadata_is_bounded_and_not_ambiguous(self) -> None:
path = self.npm / "duplicate.tgz"
self.tarball(path, "@govoplan/core-webui", "1.2.3", duplicate=True)
with self.assertRaisesRegex(ValueError, "duplicate"):
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
def test_origin_or_tag_provenance_failure_prevents_output(self) -> None:
issue = SourceTagProvenanceIssue("govoplan-core", "v1.2.3", "wrong immutable identity")
for kwargs in ({"origin_errors": (issue,)}, {"provenance_errors": (issue,)}):
with self.subTest(kwargs=kwargs), self.assertRaisesRegex(ValueError, "gate failed"):
self.build(**kwargs)
self.assertFalse(self.output.exists())
def test_registered_source_origin_requires_exact_fetch_and_push_targets(self) -> None:
repo = self.root / "govoplan-core"
repo.mkdir()
spec = RepositorySpec("govoplan-core", "system", "kernel", "git@example.test:trusted/core.git", "govoplan-core")
with patch("govoplan_release.source_provenance.load_repository_specs", return_value=(spec,)):
for targets in ((spec.remote, spec.remote), ("git@evil.test:core.git", spec.remote), (spec.remote, "git@evil.test:core.git")):
with self.subTest(targets=targets), patch("govoplan_release.source_provenance.git_text", side_effect=targets):
issues = registered_source_origin_issues(repo_versions={"govoplan-core": "1.2.3"}, workspace=self.root, remote="origin")
self.assertEqual(targets != (spec.remote, spec.remote), bool(issues))
def test_registry_metadata_cannot_cross_wire_webui_or_archive_identity(self) -> None:
self.build()
candidate = self.candidate()
entry = candidate["core_release"]
entry["webui_package"] = "@govoplan/files-webui"
with self.assertRaisesRegex(ValueError, "another source repository"):
registry_entry_source(entry)
candidate = self.candidate()
candidate["release"]["artifacts"][0]["archive_sha256"] = "f" * 64
self.assertIn("matching inspected wheel", " ".join(selected_artifact_identity_issues(candidate)))
def test_registry_version_and_selected_source_identity_must_agree(self) -> None:
self.build()
for field in ("commit_sha", "tag_object_sha"):
candidate = self.candidate()
candidate["release"]["selected_units"][0][field] = "f" * 40
self.assertIn("differs", " ".join(issue.message for issue in catalog_source_selection(candidate).issues))
candidate = self.candidate()
candidate["modules"][0]["artifact_integrity"]["python"]["git_ref"] = "v9.9.9"
self.assertTrue(candidate_catalog_version_issues(candidate))
def test_repeated_module_projections_must_bind_identical_python_and_webui_bytes(self) -> None:
self.build()
for kind in ("python", "webui"):
for reversed_order in (False, True):
with self.subTest(kind=kind, reversed_order=reversed_order):
candidate = self.candidate()
entry = deepcopy(candidate["core_release"])
entry["module_id"] = "another-core-projection"
artifact = entry["artifact_integrity"][kind]
artifact["sha256"] = "f" * 64
if kind == "python":
entry["python_ref"] = artifact["ref"] = entry["python_ref"].split("#sha256=", 1)[0] + "#sha256=" + "f" * 64
if reversed_order:
original = candidate["core_release"]
candidate["core_release"] = entry
entry = original
candidate["modules"].append(entry)
self.assertIn(f"conflicting {kind}", " ".join(selected_artifact_identity_issues(candidate)))
self.assertTrue(candidate_catalog_version_issues(candidate))
candidate = self.candidate()
repeated = deepcopy(candidate["modules"][0])
repeated["module_id"] = "second-demo-projection"
candidate["modules"].append(repeated)
self.assertEqual((), selected_artifact_identity_issues(candidate))
self.assertEqual((), candidate_catalog_version_issues(candidate))
def test_metadata_inspection_uses_the_opened_archive_not_a_replaced_path(self) -> None:
path = self.npm / "original.tgz"
replacement = self.npm / "replacement.tgz"
saved = self.npm / "saved.tgz"
self.tarball(path, "@govoplan/incorrect-webui", "1.2.3")
self.tarball(replacement, "@govoplan/core-webui", "1.2.3")
real_open = tarfile.open
def replace_path(*args, **kwargs):
self.assertIn("fileobj", kwargs)
path.rename(saved)
replacement.rename(path)
return real_open(*args, **kwargs)
with patch("govoplan_release.full_catalog.tarfile.open", side_effect=replace_path):
with self.assertRaisesRegex(ValueError, "metadata differs"):
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
def test_registry_url_provenance_rejects_package_version_and_traversal_mismatch(self) -> None:
self.build()
for old, new in (
("/govoplan-core/1.2.3/", "/govoplan-files/1.2.3/"),
("/govoplan-core/1.2.3/", "/govoplan-core/9.9.9/"),
("/pypi/files/", "/pypi/files/%2e%2e/"),
("/pypi/files/", "/pypi/files/%252e%252e/"),
):
with self.subTest(new=new):
entry = self.candidate()["core_release"]
artifact = entry["artifact_integrity"]["python"]
artifact["url"] = artifact["url"].replace(old, new)
artifact["ref"] = entry["python_ref"] = entry["python_ref"].replace(old, new)
with self.assertRaises(ValueError):
registry_entry_source(entry)
def test_package_set_must_match_fixed_meta_pins_not_just_its_own_hash(self) -> None:
payload = deepcopy(self.package_set)
payload["python"][1]["version"] = "9.9.9"
self.seal(payload, "package_set_sha256")
self.set_path.write_text(json.dumps(payload))
with self.assertRaisesRegex(ValueError, "exact Meta full pins"):
self.build()
self.assertFalse(self.output.exists())
def test_package_set_git_reads_ignore_caller_redirection(self) -> None:
tool = full_catalog._tool("generate-release-package-set")
with patch.dict(os.environ, {"GIT_DIR": "/outside", "GIT_CONFIG_GLOBAL": "/outside/config", "PATH": "/outside/bin"}):
with patch("subprocess.check_output", return_value="a" * 40 + "\n") as execute:
self.assertEqual("a" * 40, tool["_git"](self.root, "rev-parse", "HEAD"))
self.assertEqual("/usr/bin/git", execute.call_args.args[0][0])
self.assertNotIn("GIT_DIR", execute.call_args.kwargs["env"])
self.assertEqual(os.devnull, execute.call_args.kwargs["env"]["GIT_CONFIG_GLOBAL"])
self.assertEqual("/usr/bin:/bin", execute.call_args.kwargs["env"]["PATH"])
def test_unchanged_real_annotated_ancestor_is_valid_but_selecting_it_requires_head(self) -> None:
from tests.test_release_source_provenance import git, git_text, make_repo
from govoplan_release.source_provenance import source_tag_provenance_issues
workspace = self.root / "source-workspace"
workspace.mkdir()
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
git(repo, "push", "origin", "refs/tags/v1.2.3")
tagged_commit = git_text(repo, "rev-parse", "HEAD")
(repo / "workflow-only.txt").write_text("post-tag workflow repair\n")
git(repo, "add", "workflow-only.txt")
git(repo, "commit", "-m", "repair workflow without replacing package")
git(repo, "push", "origin", "main")
common = {
"repo_versions": {"govoplan-access": "1.2.3"}, "workspace": workspace,
"expected_commits": {"govoplan-access": tagged_commit},
"expected_tag_objects": {"govoplan-access": git_text(repo, "rev-parse", "refs/tags/v1.2.3")},
}
self.assertEqual((), source_tag_provenance_issues(**common))
issues = source_tag_provenance_issues(**common, require_head_repos=("govoplan-access",))
self.assertIn("not selected HEAD", " ".join(issue.message for issue in issues))
def test_tagged_manifest_synthesis_ignores_local_git_replacement_objects(self) -> None:
from tests.test_release_source_provenance import git, git_text, make_repo
from govoplan_release.catalog_entry_synthesis import materialized_source_tree
workspace = self.root / "materialization-workspace"
workspace.mkdir()
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
original = (repo / "pyproject.toml").read_text()
tagged_commit = git_text(repo, "rev-parse", "HEAD")
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
(repo / "pyproject.toml").write_text(original.replace("1.2.3", "9.9.9"))
git(repo, "add", "pyproject.toml")
git(repo, "commit", "-m", "unreviewed replacement tree")
git(repo, "replace", tagged_commit, git_text(repo, "rev-parse", "HEAD"))
with patch.dict(os.environ, {"GIT_DIR": str(self.root / "outside"), "PATH": "/outside/bin"}):
with materialized_source_tree(repo, source_ref="v1.2.3") as source:
self.assertEqual(original, (source / "pyproject.toml").read_text())
if __name__ == "__main__":
unittest.main()
+45 -2
View File
@@ -25,6 +25,11 @@ from govoplan_core.core.institutional import (
TemporalRevision, TemporalRevision,
service_launch_capability, service_launch_capability,
) )
from govoplan_core.core.temporal import (
TemporalDataContext,
bind_temporal_data_context,
reset_temporal_data_context,
)
from govoplan_cases.backend.party_context import CasePartyContext from govoplan_cases.backend.party_context import CasePartyContext
from govoplan_cases.backend.db.models import ( from govoplan_cases.backend.db.models import (
CaseAccessGrant, CaseAccessGrant,
@@ -134,6 +139,13 @@ class _Registry:
class InstitutionalGovernanceJourneyTests(unittest.TestCase): class InstitutionalGovernanceJourneyTests(unittest.TestCase):
def setUp(self) -> None:
# Portal's effective_at does not replace the SQL provider's request-local
# read clock. Keep both on the journey date, without bypassing validity
# filtering or extending the fixture's finite publication interval.
token = bind_temporal_data_context(TemporalDataContext(evaluated_at=NOW))
self.addCleanup(reset_temporal_data_context, token)
def test_service_to_formal_outcome_retains_governed_context(self) -> None: def test_service_to_formal_outcome_retains_governed_context(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:") engine = create_engine("sqlite+pysqlite:///:memory:")
for table in ( for table in (
@@ -220,13 +232,44 @@ class InstitutionalGovernanceJourneyTests(unittest.TestCase):
service_launch_capability("case"): object(), service_launch_capability("case"): object(),
} }
) )
entry = PortalServiceDirectory(service_registry).list_entries( directory = PortalServiceDirectory(service_registry)
for outside_interval in (
service.temporal.valid_from - timedelta(microseconds=1),
service.temporal.valid_to,
):
with self.subTest(outside_interval=outside_interval):
token = bind_temporal_data_context(
TemporalDataContext(evaluated_at=outside_interval)
)
try:
# Keep Portal inside the valid interval: the real SQL
# provider must still exclude a service outside its own
# temporal read context, before Portal can project it.
self.assertEqual(
(),
directory.list_entries(
session, session,
principal, principal,
tenant_id="tenant-1", tenant_id="tenant-1",
effective_at=NOW, effective_at=NOW,
audiences=("resident",), audiences=("resident",),
)[0] ),
)
finally:
reset_temporal_data_context(token)
entries = directory.list_entries(
session,
principal,
tenant_id="tenant-1",
effective_at=NOW,
audiences=("resident",),
)
self.assertEqual(
(service.reference,),
tuple(entry.definition.reference for entry in entries),
)
entry = entries[0]
self.assertTrue(entry.available) self.assertTrue(entry.available)
intake = CaseServiceIntake().plan( intake = CaseServiceIntake().plan(
entry.definition, entry.definition,
+4 -1
View File
@@ -287,7 +287,10 @@ class InstitutionalServiceJourneyTests(unittest.TestCase):
self.assertEqual("de-DE", JOURNEY["locale"]) self.assertEqual("de-DE", JOURNEY["locale"])
self.assertEqual("email_link", JOURNEY["status_access"]["mode"]) self.assertEqual("email_link", JOURNEY["status_access"]["mode"])
self.assertEqual("manual", JOURNEY["payment"]["mode"]) self.assertEqual("manual", JOURNEY["payment"]["mode"])
self.assertEqual(8, len(JOURNEY["acceptance"]["automated"])) automated = JOURNEY["acceptance"]["automated"]
self.assertEqual(10, len(automated))
self.assertTrue(any("desktop and mobile" in item for item in automated))
self.assertTrue(any("independent source" in item for item in automated))
self.assertEqual(6, len(JOURNEY["acceptance"]["manual_or_target"])) self.assertEqual(6, len(JOURNEY["acceptance"]["manual_or_target"]))
def test_portal_launches_exact_form_revision_and_persists_submission(self) -> None: def test_portal_launches_exact_form_revision_and_persists_submission(self) -> None:
+240
View File
@@ -0,0 +1,240 @@
"""Local mixed-owner admission/recovery evidence, not production capacity certification.
All inputs are synthetic and held in memory. The spawn observer temporarily
holds the admitted parent's handshake so the other owners encounter the same
occupied slot deterministically. Children perform real XLSX, template and
Dataflow work; there is no mocked process execution, database, or live service.
The non-queuing gate promises retryable rejection, not scheduler fairness.
"""
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor
from io import BytesIO
import json
import os
import threading
import time
from types import SimpleNamespace
import unittest
from unittest.mock import patch
try:
from openpyxl import Workbook
from govoplan_connectors.backend.tabular_adapters import (
parse_managed_tabular_content,
)
from govoplan_core.core.templates import TemplateRenderRequest
from govoplan_core.security import bounded_process
from govoplan_core.security.bounded_process import ProcessBudgetError
from govoplan_core.settings import settings
from govoplan_dataflow.backend.backends import execute_typed_graph
from govoplan_dataflow.backend.schemas import (
GraphEdge,
GraphNode,
GraphPosition,
PipelineGraph,
)
from govoplan_templates.backend.rendering import _render_payload
except ImportError as exc:
raise unittest.SkipTest(
"Mixed-owner isolation requires the optional module test environment."
) from exc
class IsolatedWorkCompositionTests(unittest.TestCase):
def setUp(self) -> None:
workbook = Workbook()
workbook.active.append(["name"])
workbook.active.append(["Ada"])
stream = BytesIO()
workbook.save(stream)
workbook.close()
self.workbook = stream.getvalue()
self.graph = PipelineGraph(
nodes=[
GraphNode(
id="source",
type="source.inline",
label="Source",
position=GraphPosition(x=0, y=0),
config={"source_name": "records", "rows": [{"name": "Ada"}]},
),
GraphNode(
id="output",
type="output",
label="Output",
position=GraphPosition(x=100, y=0),
config={},
),
],
edges=[GraphEdge(id="edge", source="source", target="output")],
)
def xlsx(self):
rows, sheet = parse_managed_tabular_content(
self.workbook,
filename="synthetic.xlsx",
content_type=None,
delimiter=",",
sheet_name=None,
)
self.assertEqual(rows, ({"name": "Ada"},))
self.assertEqual(sheet, "Sheet")
return "xlsx"
def templates(self):
payload, content_type, _pages = _render_payload(
SimpleNamespace(name="Synthetic template"),
SimpleNamespace(
content_text="Hello {{item.name}}",
content_html=None,
template_type="letter",
layout={},
output_profiles=[],
),
request=TemplateRenderRequest(
template_id="synthetic", output_format="text"
),
items=({"name": "Ada"},),
)
self.assertEqual(payload, b"Hello Ada")
self.assertEqual(content_type, "text/plain; charset=utf-8")
return "templates"
def dataflow(self):
result = execute_typed_graph(self.graph, backend="reference")
self.assertEqual(result.rows, [{"name": "Ada"}])
return "dataflow"
def test_one_shared_slot_rejects_other_owners_and_all_retries_recover(self):
owners = {
"xlsx": self.xlsx,
"templates": self.templates,
"dataflow": self.dataflow,
}
processes = []
modules = []
hold_next = False
entered = threading.Event()
release = threading.Event()
observer_lock = threading.Lock()
maximum_unreaped = 0
observer_timeouts = 0
original_popen = bounded_process.subprocess.Popen
def observe_spawn(*args, **kwargs):
nonlocal hold_next, maximum_unreaped, observer_timeouts
process = original_popen(*args, **kwargs)
with observer_lock:
processes.append(process)
modules.append(args[0][5])
maximum_unreaped = max(
maximum_unreaped, sum(item.returncode is None for item in processes)
)
should_hold = hold_next
hold_next = False
if should_hold:
entered.set()
if not release.wait(8):
# Return control so the real runner's normal timeout and
# process-group cleanup still own this child on test error.
observer_timeouts += 1
return process
def rejected(operation):
try:
operation()
except Exception as exc:
cause = exc
while cause is not None and not isinstance(cause, ProcessBudgetError):
cause = cause.__cause__
self.assertIsInstance(cause, ProcessBudgetError)
self.assertEqual(cause.code, "busy")
return "busy"
self.fail(
"A different module admitted work while the shared slot was occupied."
)
started = time.monotonic()
busy_count = 0
try:
with (
patch.object(settings, "isolated_process_concurrency", 1),
patch.object(bounded_process.subprocess, "Popen", observe_spawn),
ThreadPoolExecutor(max_workers=3) as executor,
):
for owner, operation in owners.items():
with self.subTest(admitted_owner=owner):
entered.clear()
release.clear()
hold_next = True
holder = executor.submit(operation)
try:
self.assertTrue(
entered.wait(5),
"The admitted operation never spawned its real child.",
)
children_before = len(processes)
others = [
work for label, work in owners.items() if label != owner
]
denied = [
executor.submit(rejected, work) for work in others
]
self.assertEqual(
[future.result(timeout=5) for future in denied],
["busy", "busy"],
)
busy_count += len(denied)
self.assertEqual(len(processes), children_before)
finally:
release.set()
self.assertEqual(holder.result(timeout=15), owner)
self.assertEqual(bounded_process._active, 0)
# Every rejected owner is retried through its real API.
# Each must complete after the previous holder releases.
for other in others:
other()
self.assertEqual(bounded_process._active, 0)
finally:
release.set()
for process in processes:
self.assertIsNotNone(process.returncode, "Worker was not reaped.")
self.assertTrue(
all(
stream.closed
for stream in (process.stdin, process.stdout, process.stderr)
)
)
with self.assertRaises(ChildProcessError):
os.waitpid(process.pid, os.WNOHANG)
self.assertEqual(maximum_unreaped, 1)
self.assertEqual(observer_timeouts, 0)
self.assertEqual(len(processes), 9)
self.assertEqual(busy_count, 6)
self.assertEqual(
set(modules),
{
"govoplan_connectors.backend.tabular_adapters",
"govoplan_templates.backend.rendering",
"govoplan_dataflow.backend.backends.reference",
},
)
print(
json.dumps(
{
"local_composition": {
"successful_children": len(processes),
"busy_rejections": busy_count,
"maximum_unreaped_children": maximum_unreaped,
"all_children_reaped": True,
"seconds": round(time.monotonic() - started, 3),
}
}
)
)
if __name__ == "__main__":
unittest.main()
+18 -3
View File
@@ -62,22 +62,37 @@ class PackageRegistryReleaseTests(unittest.TestCase):
encoding="utf-8" encoding="utf-8"
) )
)["project"]["version"] )["project"]["version"]
meta_package = ROOT / "packages/govoplan-meta/pyproject.toml"
meta_project = tomllib.loads(
meta_package.read_text(encoding="utf-8")
)["project"]
expected_tasks_pin = next(
requirement
for requirement in (
*meta_project["dependencies"],
*meta_project["optional-dependencies"]["full"],
)
if requirement.startswith("govoplan-tasks==")
)
selected = PACKAGE_SET.parse_meta_package( selected = PACKAGE_SET.parse_meta_package(
ROOT / "packages/govoplan-meta/pyproject.toml", meta_package,
core_version=core_version, core_version=core_version,
) )
by_name = {item["name"]: item for item in selected} by_name = {item["name"]: item for item in selected}
self.assertIn("govoplan-core", by_name) self.assertIn("govoplan-core", by_name)
self.assertIn("govoplan-records", by_name) self.assertIn("govoplan-records", by_name)
self.assertEqual("0.1.20", by_name["govoplan-tasks"]["version"]) self.assertEqual(
expected_tasks_pin,
f"govoplan-tasks=={by_name['govoplan-tasks']['version']}",
)
payload = PACKAGE_SET.generate_package_set( payload = PACKAGE_SET.generate_package_set(
core_version=core_version, core_version=core_version,
requirements=ROOT / "requirements-release.txt", requirements=ROOT / "requirements-release.txt",
workspace=ROOT.parent, workspace=ROOT.parent,
profile="full", profile="full",
meta_package=ROOT / "packages/govoplan-meta/pyproject.toml", meta_package=meta_package,
) )
self.assertEqual("full", payload["profile"]) self.assertEqual("full", payload["profile"])
self.assertEqual(len(selected), len(payload["python"])) self.assertEqual(len(selected), len(payload["python"]))
+9 -4
View File
@@ -22,11 +22,16 @@ class PackageSetDispatchTests(unittest.TestCase):
def test_meta_package_resolves_to_exact_tagged_repository_targets(self) -> None: def test_meta_package_resolves_to_exact_tagged_repository_targets(self) -> None:
targets = MODULE.package_targets() targets = MODULE.package_targets()
self.assertEqual(66, len(targets)) self.assertEqual(73, len(targets))
self.assertEqual(66, len({target.distribution for target in targets})) self.assertEqual(73, len({target.distribution for target in targets}))
by_name = {target.distribution: target for target in targets} by_name = {target.distribution: target for target in targets}
self.assertEqual("v0.1.14", by_name["govoplan-core"].tag) self.assertEqual("v0.1.38", by_name["govoplan-core"].tag)
self.assertEqual("v0.1.8", by_name["govoplan-access"].tag) self.assertEqual("v0.1.22", by_name["govoplan-access"].tag)
self.assertEqual("v0.1.20", by_name["govoplan-dms"].tag)
self.assertEqual("v0.1.20", by_name["govoplan-erp"].tag)
self.assertEqual("v0.1.20", by_name["govoplan-fit-connect"].tag)
self.assertEqual("v0.1.23", by_name["govoplan-idm"].tag)
self.assertEqual("v0.1.21", by_name["govoplan-xrechnung"].tag)
self.assertTrue(by_name["govoplan-core"].tag_exists) self.assertTrue(by_name["govoplan-core"].tag_exists)
self.assertTrue(by_name["govoplan-access"].has_webui) self.assertTrue(by_name["govoplan-access"].has_webui)
self.assertEqual( self.assertEqual(
@@ -96,6 +96,18 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
with self.assertRaisesRegex(ValueError, "tracking_issue"): with self.assertRaisesRegex(ValueError, "tracking_issue"):
inventory._load_endpoint_declarations(path) inventory._load_endpoint_declarations(path)
def test_bounded_workflow_read_apis_have_explicit_headless_declarations(self) -> None:
declarations = inventory._load_endpoint_declarations(inventory.DEFAULT_ENDPOINT_DECLARATIONS)
for path in (
"/workflow/instances/summaries", "/workflow/instances/{}/summary",
"/workflow/instances/{}/steps", "/workflow/instances/{}/events",
):
with self.subTest(path=path):
entry = declarations[("govoplan-workflow-engine", "GET", path)]
self.assertEqual("intentionally_headless", entry["category"])
self.assertIn("current-authorized", entry["rationale"])
self.assertIn("workflow.instance-history", entry["rationale"])
def test_inventory_reports_unclassified_and_stale_endpoint_declarations( def test_inventory_reports_unclassified_and_stale_endpoint_declarations(
self, self,
) -> None: ) -> None:
@@ -169,6 +181,69 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
), ),
) )
def test_high_risk_help_baseline_is_validated(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "help-baseline.json"
path.write_text(
json.dumps(
{
"schema_version": 1,
"maximum_missing_exact_help": 3,
}
),
encoding="utf-8",
)
self.assertEqual(
3,
inventory._load_high_risk_help_baseline(path)[
"maximum_missing_exact_help"
],
)
path.write_text(
json.dumps(
{
"schema_version": 1,
"maximum_missing_exact_help": -1,
}
),
encoding="utf-8",
)
with self.assertRaisesRegex(ValueError, "non-negative integer"):
inventory._load_high_risk_help_baseline(path)
def test_declaration_strict_mode_rejects_high_risk_help_regression(
self,
) -> None:
result = {
"translation_health": {"missing_catalog_entries": []},
"api": {
"unclassified_endpoints": [],
"stale_endpoint_declarations": [],
},
"declaration_health": {},
"help_health": {
"invalid_risk_annotations": [],
"unresolved_exact_high_risk_help": [],
"high_risk_help_without_german": [],
"missing_exact_high_risk_help": [{"id": "example.delete"}],
"baseline_maximum_missing": 0,
"baseline_regression": True,
},
}
self.assertEqual(
[
"1 high-risk controls lack exact F1 help; baseline permits at most 0"
],
inventory._strict_failures(
result,
check_translations=False,
check_endpoints=False,
check_declarations=True,
),
)
def test_fastapi_route_scanner_includes_router_prefix(self) -> None: def test_fastapi_route_scanner_includes_router_prefix(self) -> None:
tree = ast.parse( tree = ast.parse(
""" """
+75
View File
@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path from pathlib import Path
import shlex
import subprocess import subprocess
import sys import sys
import unittest import unittest
@@ -17,6 +18,80 @@ from govoplan_release import git_state # noqa: E402
class ReleaseGitStateTests(unittest.TestCase): class ReleaseGitStateTests(unittest.TestCase):
def test_unset_ssh_address_family_preserves_original_command_and_operator_config(self) -> None:
environment = git_state.sanitized_git_environment({})
self.assertEqual(
[
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
],
shlex.split(environment["GIT_SSH_COMMAND"]),
)
self.assertNotIn("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY", environment)
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
def test_ssh_address_family_accepts_only_fixed_choices_and_survives_resanitizing(self) -> None:
for family in ("any", "inet", "inet6"):
with self.subTest(family=family):
environment = git_state.sanitized_git_environment({
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": family,
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
"GIT_SSH": "/attacker/ssh",
"PATH": "/attacker/bin",
})
self.assertEqual(
[
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
"-o", f"AddressFamily={family}",
],
shlex.split(environment["GIT_SSH_COMMAND"]),
)
self.assertNotIn("GIT_SSH", environment)
self.assertEqual("/usr/bin:/bin", environment["PATH"])
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
def test_invalid_ssh_address_family_is_rejected_before_git_runs(self) -> None:
for invalid in (
"", "INET", "ipv4", " inet", "inet ", "inet\n",
"inet; touch /not-executed", "inet -o StrictHostKeyChecking=no",
"$(not-executed)",
):
with (
self.subTest(value=invalid),
patch.dict("os.environ", {"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": invalid}),
patch.object(git_state.subprocess, "run") as run,
):
with self.assertRaisesRegex(
ValueError, "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6",
):
git_state.git(Path("/workspace/govoplan-core"), "status", "--porcelain")
run.assert_not_called()
def test_source_provenance_readback_keeps_family_but_discards_ssh_command_override(self) -> None:
from govoplan_release.source_provenance import inspect_remote_tag
completed = subprocess.CompletedProcess(
[], 0, f"{'a' * 40}\trefs/tags/v1.2.3\n{'b' * 40}\trefs/tags/v1.2.3^{{}}\n", "",
)
with (
patch.dict("os.environ", {
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": "inet",
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
}),
patch("govoplan_release.repository_tag.subprocess.run", return_value=completed) as run,
):
result = inspect_remote_tag(
path=Path("/workspace/govoplan-core"), remote="origin",
remote_url="git@git.add-ideas.de:GovOPlaN/govoplan-core.git", tag="v1.2.3",
)
self.assertEqual("b" * 40, result.commit)
self.assertEqual(
"/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8 -o AddressFamily=inet",
run.call_args.kwargs["env"]["GIT_SSH_COMMAND"],
)
def test_manifest_version_does_not_confuse_interface_versions(self) -> None: def test_manifest_version_does_not_confuse_interface_versions(self) -> None:
from tempfile import TemporaryDirectory from tempfile import TemporaryDirectory
+330
View File
@@ -0,0 +1,330 @@
from __future__ import annotations
from dataclasses import replace
from contextlib import redirect_stdout
import io
import json
from pathlib import Path
import runpy
import shutil
import sys
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import meta_preparation # noqa: E402
from govoplan_release.git_state import collect_repository_snapshot # noqa: E402
from govoplan_release.meta_preparation import ( # noqa: E402
MetaPreparationError,
prepare_developer_meta_package,
)
from govoplan_release.model import RepositorySpec # noqa: E402
from govoplan_release.selective_planner import build_selective_release_plan # noqa: E402
from govoplan_release.version_metadata import ( # noqa: E402
VersionMetadataError,
apply_version_metadata_mutations,
version_metadata_mutations,
)
import test_release_meta_source_tag as meta_fixture # noqa: E402
from test_release_plan_guidance import dashboard # noqa: E402
from test_release_repository_tag import create_release_repo, git, git_text # noqa: E402
class MetaPreparationTests(unittest.TestCase):
synchronize = meta_fixture.MetaSourceTagTests.synchronize
def setUp(self):
meta_fixture.MetaSourceTagTests.setUp(self)
self.operator = self.root / "operator"
self.generator = (
self.operator / "tools/release/generate-developer-meta-package.py"
)
self.generator.parent.mkdir(parents=True)
shutil.copyfile(
ROOT / "tools/release/generate-developer-meta-package.py", self.generator
)
self.enterContext(patch.object(meta_preparation, "META_ROOT", self.operator))
def prepare_core(self):
apply_version_metadata_mutations(self.core, target_version="0.1.11")
git(self.core, "add", ".")
git(self.core, "commit", "-m", "Prepared synthetic Core target")
def preview(self, **kwargs):
return prepare_developer_meta_package(
repo_path=self.meta, target_version="0.1.11", **kwargs
)
def apply(self, preview):
return self.preview(
apply=True, expected_receipt=preview["receipt"], confirm_out_of_run=True
)
def test_full_canonical_preview_apply_and_shared_mutation_discovery(self):
extra, remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-workflow-engine",
version="0.2.3",
)
self.specs.append(
{
"name": extra.name,
"path": extra.name,
"category": "module",
"subtype": "",
"remote": str(remote),
}
)
self.registry.write_text(json.dumps({"repositories": self.specs}))
self.prepare_core()
before = self.package.read_bytes()
preview = self.preview()
self.assertEqual("planned", preview["status"])
self.assertEqual(before, self.package.read_bytes())
mutations = version_metadata_mutations(self.meta, target_version="0.1.11")
self.assertEqual([meta_preparation.PACKAGE], [item.path for item in mutations])
self.assertIn(b"govoplan-workflow-engine==0.2.3", mutations[0].after)
self.assertIn(b"govoplan-core==0.1.11", mutations[0].after)
result = self.apply(preview)
self.assertEqual("prepared", result["status"])
self.assertEqual(mutations[0].after, self.package.read_bytes())
self.assertEqual(
self.render(workspace=self.workspace, requirements=self.requirements),
self.package.read_text(),
)
self.assertEqual(
f"M {meta_preparation.PACKAGE}",
git_text(self.meta, "status", "--porcelain"),
)
self.assertFalse(git_text(self.meta, "tag", "--list"))
def test_core_target_must_already_be_prepared(self):
before = self.package.read_bytes()
with self.assertRaisesRegex(MetaPreparationError, "Prepare and commit Core"):
self.preview()
self.assertEqual(before, self.package.read_bytes())
def test_changed_requirements_receipt_blocks_before_any_output(self):
self.prepare_core()
preview = self.preview()
before = self.package.read_bytes()
self.requirements.write_text(
self.requirements.read_text() + "# reviewed different inputs\n"
)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Changed synthetic requirements")
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
self.apply(preview)
self.assertEqual(before, self.package.read_bytes())
def test_source_change_immediately_before_effect_is_rechecked(self):
self.prepare_core()
preview = self.preview()
before = self.package.read_bytes()
original = meta_preparation.preview_meta_mutation
def changed(**kwargs):
result = original(**kwargs)
self.requirements.write_text(
self.requirements.read_text() + "# concurrent change\n"
)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Concurrent synthetic change")
return result
with patch.object(
meta_preparation, "preview_meta_mutation", side_effect=changed
):
with self.assertRaisesRegex(MetaPreparationError, "changed before"):
self.apply(preview)
self.assertEqual(before, self.package.read_bytes())
def test_core_full_package_and_operator_generator_are_receipt_bound(self):
self.prepare_core()
for path, repository in (
(self.core / "pyproject.toml", self.core),
(self.access / "pyproject.toml", self.access),
(self.generator, None),
):
with self.subTest(input=path.name, repo=str(repository)):
preview = self.preview()
before = self.package.read_bytes()
path.write_text(path.read_text() + "\n# changed frozen input\n")
if repository is not None:
git(repository, "add", ".")
git(
repository,
"commit",
"-m",
"Changed synthetic composition input",
)
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
self.apply(preview)
self.assertEqual(before, self.package.read_bytes())
def test_post_write_source_change_is_reported_without_retry_or_rollback(self):
from govoplan_release import version_metadata
self.prepare_core()
preview = self.preview()
original = version_metadata._atomic_write
def changed(path, payload):
original(path, payload)
self.requirements.write_text(
self.requirements.read_text() + "# concurrent after write\n"
)
with patch.object(
version_metadata, "_atomic_write", side_effect=changed
) as writer:
with self.assertRaisesRegex(
meta_preparation.MetaPreparationAmbiguous, "write/post-check failed"
):
self.apply(preview)
self.assertEqual(1, writer.call_count)
self.assertIn('version = "0.1.11"', self.package.read_text())
self.assertIn("# concurrent after write", self.requirements.read_text())
def test_write_failure_after_replace_requires_reconciliation(self):
from govoplan_release import version_metadata
self.prepare_core()
preview = self.preview()
original = version_metadata._atomic_write
def partial(path, payload):
original(path, payload)
raise OSError("Synthetic directory fsync failure after replacement")
with patch.object(version_metadata, "_atomic_write", side_effect=partial) as writer:
with self.assertRaisesRegex(meta_preparation.MetaPreparationAmbiguous, "may have been written"):
self.apply(preview)
self.assertEqual(1, writer.call_count)
self.assertIn('version = "0.1.11"', self.package.read_text())
def test_cli_requires_reviewed_receipt_and_explicit_out_of_run_confirmation(self):
self.prepare_core()
main = runpy.run_path(
str(ROOT / "tools/release/prepare-developer-meta-package.py")
)["main"]
arguments = [
"prepare-developer-meta-package.py",
"--workspace",
str(self.workspace),
"--target-version",
"0.1.11",
]
output = io.StringIO()
with patch.object(sys, "argv", arguments), redirect_stdout(output):
self.assertEqual(0, main())
preview = self.root / "meta-preview.json"
preview.write_text(output.getvalue())
with (
patch.object(sys, "argv", [*arguments, "--apply"]),
redirect_stdout(io.StringIO()),
):
self.assertEqual(1, main())
with (
patch.object(
sys,
"argv",
[
*arguments,
"--apply",
"--receipt",
str(preview),
"--confirm-out-of-run",
],
),
redirect_stdout(io.StringIO()),
):
self.assertEqual(0, main())
def test_unknown_full_input_and_unsafe_operator_tooling_fail_closed(self):
self.prepare_core()
unknown = self.workspace / "govoplan-unknown/pyproject.toml"
unknown.parent.mkdir()
unknown.write_text('[project]\nname="govoplan-unknown"\nversion="1.0.0"\n')
with self.assertRaisesRegex(MetaPreparationError, "unregistered"):
self.preview()
unknown.unlink()
self.generator.chmod(0o666)
with self.assertRaisesRegex(MetaPreparationError, "owned, bounded regular"):
self.preview()
def test_wrong_nested_identity_and_existing_immutable_tag_fail_closed(self):
self.prepare_core()
original = self.package.read_text()
self.package.write_text(
original.replace('name = "govoplan"', 'name = "not-govoplan"')
)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Wrong synthetic package identity")
with self.assertRaisesRegex(MetaPreparationError, "identity"):
self.preview()
self.package.write_text(original)
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Restore synthetic package identity")
git(self.meta, "tag", "-a", "v0.1.11", "-m", "Immutable target")
with self.assertRaisesRegex(MetaPreparationError, "target Meta tag"):
self.preview()
def test_no_generic_durable_self_mutation_or_running_tooling_target(self):
self.prepare_core()
preview = self.preview()
with self.assertRaisesRegex(VersionMetadataError, "outside durable runs"):
apply_version_metadata_mutations(self.meta, target_version="0.1.11")
with self.assertRaisesRegex(MetaPreparationError, "confirm"):
self.preview(apply=True, expected_receipt=preview["receipt"])
with patch.object(meta_preparation, "META_ROOT", self.meta):
with self.assertRaisesRegex(MetaPreparationError, "running operator"):
self.preview()
def test_next_version_plan_is_actionable_core_first_without_meta_executor(self):
snapshots = tuple(
collect_repository_snapshot(
RepositorySpec(**spec),
workspace_root=self.workspace,
target_tag="v0.1.11",
)
for spec in self.specs[:2]
)
source = replace(
dashboard(workspace=self.workspace, version=self.version),
repositories=snapshots,
)
plan = build_selective_release_plan(
source,
selected_repos=("govoplan", "govoplan-core"),
target_version="0.1.11",
)
self.assertEqual(
["govoplan-core", "govoplan"], [unit.repo for unit in plan.units]
)
findings = [
finding for finding in plan.gate_findings if finding.repo == "govoplan"
]
self.assertEqual(
["developer_meta_core_preparation_required"],
[finding.code for finding in findings],
)
self.assertIn("prepare-developer-meta-package.py", findings[0].remediation)
meta_steps = [step for step in plan.dry_run_steps if step.repo == "govoplan"]
self.assertEqual(
["govoplan:prepare-support", "govoplan:publish-support"],
[step.id for step in meta_steps],
)
self.assertTrue(all(step.status == "needs-executor" for step in meta_steps))
self.prepare_core()
prepared = build_selective_release_plan(
source, selected_repos=("govoplan",), target_version="0.1.11"
)
self.assertEqual("developer_meta_out_of_run", prepared.gate_findings[0].code)
if __name__ == "__main__":
unittest.main()
+569
View File
@@ -0,0 +1,569 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import runpy
import shutil
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import source_tag_batch as meta_source_tag, workspace # noqa: E402
from govoplan_release.git_state import collect_versions # noqa: E402
from govoplan_release.model import RepositorySnapshot, RepositorySpec, VersionSnapshot # noqa: E402
from govoplan_release.repository_tag import tag_repositories # noqa: E402
from govoplan_release.selective_planner import build_unit # noqa: E402
from govoplan_release.version_alignment import repository_version_issues # noqa: E402
from test_release_repository_tag import ( # noqa: E402
add_scoped_workflow_manifest,
create_release_repo,
git,
git_text,
ref_exists,
)
class MetaSourceTagTests(unittest.TestCase):
def setUp(self):
self.temporary = self.enterContext(
tempfile.TemporaryDirectory(prefix="meta-release-tests-")
)
self.root = Path(self.temporary)
self.workspace = self.root / "workspace"
self.workspace.mkdir()
self.version = "0.1.10"
self.core, self.core_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-core",
version=self.version,
)
self.access, self.access_remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-access",
version=self.version,
)
add_scoped_workflow_manifest(self.access)
self.meta = self.workspace / "govoplan"
self.meta_remote = self.root / "govoplan.git"
git(self.root, "init", "--bare", str(self.meta_remote))
git(self.workspace, "init", "-b", "main", str(self.meta))
git(self.meta, "config", "user.name", "Meta Release Fixture")
git(self.meta, "config", "user.email", "release@example.invalid")
self.package = self.meta / "packages/govoplan-meta/pyproject.toml"
self.package.parent.mkdir(parents=True)
self.requirements = self.meta / "requirements-release.txt"
self.requirements.write_text(
"../govoplan-core\ngovoplan-access @ git+ssh://git@example.invalid/GovOPlaN/govoplan-access.git@v0.1.10\n"
)
self.render = runpy.run_path(
str(ROOT / "tools/release/generate-developer-meta-package.py")
)["render"]
self.synchronize()
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Nested developer package")
git(self.meta, "remote", "add", "origin", str(self.meta_remote))
git(self.meta, "push", "-u", "origin", "main")
self.specs = [
{
"name": name,
"category": "system" if subtype else "module",
"subtype": subtype,
"path": name,
"remote": str(remote),
}
for name, subtype, remote in (
("govoplan", "meta", self.meta_remote),
("govoplan-core", "kernel", self.core_remote),
("govoplan-access", "", self.access_remote),
)
]
self.registry = self.root / "repositories.json"
self.registry.write_text(json.dumps({"repositories": self.specs}))
self.enterContext(patch.object(workspace, "REPOSITORIES_FILE", self.registry))
def synchronize(self):
self.package.write_text(
self.render(workspace=self.workspace, requirements=self.requirements)
)
def commit_meta(self):
git(self.meta, "add", ".")
git(self.meta, "commit", "-m", "Changed synthetic metadata")
def tag(
self,
*,
repos=("govoplan", "govoplan-core"),
apply=False,
push=False,
**overrides,
):
return tag_repositories(
repos=repos,
repo_versions={repo: self.version for repo in repos},
workspace_root=self.workspace,
apply=apply,
push=push,
**overrides,
)
def assert_no_tags(self):
for repo in (
self.meta,
self.meta_remote,
self.core,
self.core_remote,
self.access,
self.access_remote,
):
self.assertFalse(ref_exists(repo, "refs/tags/v0.1.10"), str(repo))
def test_explicit_nested_version_collection_and_alignment_without_root_package(
self,
):
versions = collect_versions(self.meta)
self.assertIsNone(versions.pyproject)
self.assertEqual(self.version, versions.developer_meta)
self.assertEqual(self.version, versions.primary)
self.assertFalse((self.meta / "pyproject.toml").exists())
self.assertEqual(
(), repository_version_issues(self.meta, expected_version=self.version)
)
mismatch = repository_version_issues(self.meta, expected_version="0.1.11")
self.assertTrue(
any(
issue.source == "packages/govoplan-meta/pyproject.toml"
for issue in mismatch
)
)
def test_planner_and_console_display_the_explicit_nested_version(self):
snapshot = RepositorySnapshot(
spec=RepositorySpec(**self.specs[0]),
absolute_path=str(self.meta),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(developer_meta=self.version),
)
unit = build_unit(snapshot, target_version=None, contracts=None)
self.assertEqual(self.version, unit.current_version)
self.assertEqual(self.version, unit.target_version)
html = (ROOT / "tools/release/webui/index.html").read_text()
self.assertIn(
"if (versions.developer_meta) return versions.developer_meta;", html
)
drift = RepositorySnapshot(
spec=snapshot.spec,
absolute_path=str(self.meta),
exists=True,
is_git=True,
has_head=True,
branch="main",
versions=VersionSnapshot(pyproject="0.1.9", developer_meta=self.version),
)
self.assertTrue(
any(
"version metadata is not aligned" in item
for item in build_unit(
drift, target_version=self.version, contracts=None
).blockers
)
)
def test_unknown_nested_package_and_missing_or_wrong_meta_identity_fail_closed(
self,
):
unknown = self.workspace / "unknown"
nested = unknown / "packages/govoplan-meta/pyproject.toml"
nested.parent.mkdir(parents=True)
nested.write_text(self.package.read_text())
self.assertIsNone(collect_versions(unknown).primary)
self.assertIn(
"no version metadata",
repository_version_issues(unknown, expected_version=self.version)[
0
].message,
)
for value in ("", '[project]\nname="not-govoplan"\nversion="0.1.10"\n'):
with self.subTest(value=value):
self.package.write_text(value)
self.assertTrue(
repository_version_issues(self.meta, expected_version=self.version)
)
def test_preview_local_tag_and_publish_share_complete_nested_contract(self):
preview = self.tag(push=True)
self.assertEqual("planned", preview["status"], preview)
self.assertEqual(
["govoplan-core", "govoplan"],
[row["repo"] for row in preview["repositories"]],
)
self.assertEqual("registered-meta-batch-v1", preview["source_contract"])
self.assert_no_tags()
local = self.tag(apply=True)
self.assertEqual("tagged", local["status"], local)
for repo in (self.core, self.meta):
self.assertEqual(
"tag", git_text(repo, "cat-file", "-t", "refs/tags/v0.1.10")
)
self.assertFalse(ref_exists(self.meta_remote, "refs/tags/v0.1.10"))
published = self.tag(apply=True, push=True)
self.assertEqual("published", published["status"], published)
for repo, remote in (
(self.core, self.core_remote),
(self.meta, self.meta_remote),
):
self.assertEqual(
git_text(repo, "rev-parse", "HEAD"),
git_text(remote, "rev-parse", "refs/heads/main"),
)
self.assertEqual(
git_text(repo, "rev-parse", "refs/tags/v0.1.10"),
git_text(remote, "rev-parse", "refs/tags/v0.1.10"),
)
again = self.tag(apply=True, push=True)
self.assertEqual("published", again["status"], again)
def test_stale_composition_blocks_whole_batch_before_local_tag_or_push(self):
self.package.write_text(
self.package.read_text().replace(
"govoplan-access==0.1.10", "govoplan-access==0.1.9"
)
)
self.commit_meta()
for apply, push in ((False, False), (True, False), (True, True)):
result = self.tag(
repos=("govoplan-core", "govoplan-access", "govoplan"),
apply=apply,
push=push,
)
self.assertEqual("blocked", result["status"], result)
self.assert_no_tags()
def test_core_outside_batch_requires_matching_existing_and_published_tag(self):
self.assertEqual("blocked", self.tag(repos=("govoplan",))["status"])
git(self.core, "tag", "-a", "v0.1.10", "-m", "Core release")
self.assertEqual("planned", self.tag(repos=("govoplan",))["status"])
self.assertEqual("blocked", self.tag(repos=("govoplan",), push=True)["status"])
git(self.core, "push", "origin", "refs/tags/v0.1.10")
self.assertEqual("planned", self.tag(repos=("govoplan",), push=True)["status"])
def test_changed_core_version_and_explicit_selected_version_mismatch_block(self):
mismatch = tag_repositories(
repos=("govoplan", "govoplan-core"),
repo_versions={"govoplan": self.version, "govoplan-core": "0.1.11"},
workspace_root=self.workspace,
apply=True,
push=True,
)
self.assertEqual("blocked", mismatch["status"], mismatch)
(self.core / "pyproject.toml").write_text(
'[project]\nname="govoplan-core"\nversion="0.1.11"\n'
)
git(self.core, "add", ".")
git(self.core, "commit", "-m", "Core new version")
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assert_no_tags()
def test_unsafe_origin_on_any_selected_repo_blocks_every_effect(self):
for repo in (self.meta, self.core, self.access):
with self.subTest(repo=repo.name):
git(
repo,
"config",
"remote.origin.pushurl",
str(self.root / "unregistered.git"),
)
result = self.tag(
repos=("govoplan-core", "govoplan-access", "govoplan"),
apply=True,
push=True,
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("registered origin", result["repositories"][0]["detail"])
self.assert_no_tags()
git(repo, "config", "--unset", "remote.origin.pushurl")
def test_world_writable_nonsticky_parent_blocks_without_changing_permissions(self):
original = self.root.stat().st_mode & 0o7777
self.root.chmod(0o777)
try:
for apply in (False, True):
result = self.tag(apply=apply, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn(
"group/world writable", result["repositories"][0]["detail"]
)
self.assertEqual(0o777, self.root.stat().st_mode & 0o7777)
self.assert_no_tags()
finally:
self.root.chmod(original)
def test_wrong_metadata_owner_blocks_before_remote_lookup(self):
config = self.meta / ".git/config"
original = Path.lstat
def wrong_owner(path, *args, **kwargs):
observed = original(path, *args, **kwargs)
if path == config:
fields = list(observed)
fields[4] = os.geteuid() + 1
return os.stat_result(fields)
return observed
with (
patch.object(Path, "lstat", new=wrong_owner),
patch.object(
meta_source_tag,
"registered_source_origin_issues",
side_effect=AssertionError("must validate ownership before Git"),
),
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("current operator", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_hidden_index_flags_cannot_disguise_modified_release_metadata(self):
for flag, undo in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
for repo, relative in (
(self.meta, "packages/govoplan-meta/pyproject.toml"),
(self.core, "pyproject.toml"),
):
with self.subTest(flag=flag, repo=repo.name):
target = repo / relative
original = target.read_text()
git(repo, "update-index", flag, relative)
target.write_text(
original
+ "\n# Hidden working-tree input differs from frozen HEAD\n"
)
try:
self.assertEqual("", git_text(repo, "status", "--porcelain"))
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn(
"index entries", result["repositories"][0]["detail"]
)
self.assert_no_tags()
finally:
target.write_text(original)
git(repo, "update-index", undo, relative)
def test_read_only_git_target_is_not_repaired_or_tagged(self):
metadata = self.meta / ".git"
original = metadata.stat().st_mode & 0o7777
metadata.chmod(0o500)
try:
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertEqual(0o500, metadata.stat().st_mode & 0o7777)
self.assert_no_tags()
finally:
metadata.chmod(original)
def test_git_object_alternates_are_rejected_before_remote_lookup(self):
(self.meta / ".git/objects/info/alternates").write_text(
str(self.root / "outside-objects") + "\n"
)
with patch.object(
meta_source_tag,
"registered_source_origin_issues",
side_effect=AssertionError("must reject alternates before Git"),
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("alternates", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_owned_worktree_metadata_inside_private_workspace_is_supported(self):
main_checkout = self.workspace / "meta-main-storage"
self.meta.rename(main_checkout)
git(main_checkout, "worktree", "add", "--force", str(self.meta), "main")
self.assertTrue((self.meta / ".git").is_file())
result = self.tag(apply=True)
self.assertEqual("tagged", result["status"], result)
filesystem = result["source_receipts"]["govoplan"]["filesystem"]
self.assertEqual(
str(main_checkout / ".git"), filesystem["git_common_directory"][0]
)
def test_git_directory_replacement_with_same_head_changes_frozen_receipt(self):
preview = meta_source_tag._preview_repositories
def swapped_git_directory(**kwargs):
result = preview(**kwargs)
original = self.meta / ".git"
backup = self.root / "original-meta-git"
original.rename(backup)
shutil.copytree(backup, original)
return result
with patch.object(
meta_source_tag,
"_preview_repositories",
side_effect=swapped_git_directory,
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_non_main_and_divergent_live_main_fail_even_with_stale_tracking(self):
git(self.meta, "switch", "-c", "feature")
self.assertEqual("blocked", self.tag(apply=True)["status"])
git(self.meta, "switch", "main")
clone = self.root / "other-writer"
git(self.root, "clone", "--branch", "main", str(self.meta_remote), str(clone))
git(clone, "config", "user.name", "Other synthetic writer")
git(clone, "config", "user.email", "other@example.invalid")
(clone / "other.txt").write_text("remote divergence\n")
git(clone, "add", ".")
git(clone, "commit", "-m", "Remote main advanced")
git(clone, "push", "origin", "main")
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("live origin/main", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_symlink_checkout_is_not_a_registered_source(self):
original = self.workspace / "moved-meta"
self.meta.rename(original)
self.meta.symlink_to(original, target_is_directory=True)
result = self.tag(apply=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("symlink", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_lightweight_and_conflicting_annotated_tags_block(self):
git(self.meta, "tag", "v0.1.10")
self.assertEqual("blocked", self.tag(apply=True)["status"])
git(self.meta, "tag", "-d", "v0.1.10")
git(self.meta, "tag", "-a", "v0.1.10", "-m", "First annotation")
git(self.meta, "push", "origin", "refs/tags/v0.1.10")
git(self.meta, "tag", "-d", "v0.1.10")
git(self.meta, "tag", "-a", "v0.1.10", "-m", "Different annotation")
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
def test_meta_source_receipt_changed_after_preflight_blocks_before_first_effect(
self,
):
preview = meta_source_tag._preview_repositories
def changed_after_preflight(**kwargs):
result = preview(**kwargs)
(self.meta / "new-review.txt").write_text("changed after preflight\n")
self.commit_meta()
return result
with patch.object(
meta_source_tag,
"_preview_repositories",
side_effect=changed_after_preflight,
):
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_fabricated_push_success_without_remote_receipt_fails_and_stops_batch(self):
original = meta_source_tag.run
def run(command, **kwargs):
if command[:3] == ("git", "push", "--atomic"):
return subprocess.CompletedProcess(command, 0, "", "")
return original(command, **kwargs)
with patch.object(meta_source_tag, "run", side_effect=run):
result = self.tag(apply=True, push=True)
self.assertEqual("partial", result["status"], result)
self.assertEqual("failed", result["repositories"][0]["status"])
self.assertEqual("skipped", result["repositories"][1]["status"])
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
def test_remote_tag_without_expected_main_receipt_is_not_success(self):
(self.core / "reviewed-change.txt").write_text("release source changes\n")
git(self.core, "add", ".")
git(self.core, "commit", "-m", "Advance reviewed Core source")
original = meta_source_tag.run
pushes = []
def tag_only(command, **kwargs):
if command[:3] == ("git", "push", "--atomic"):
pushes.append(command)
# Simulate a defective transport that claims atomic success,
# while publishing only the exact expected annotation object.
return original(("git", "push", "origin", command[-1]), **kwargs)
return original(command, **kwargs)
with patch.object(meta_source_tag, "run", side_effect=tag_only):
result = self.tag(apply=True, push=True)
self.assertEqual("partial", result["status"], result)
self.assertEqual(1, len(pushes))
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assertEqual(
git_text(self.core, "rev-parse", "refs/tags/v0.1.10"),
git_text(self.core_remote, "rev-parse", "refs/tags/v0.1.10"),
)
self.assertNotEqual(
git_text(self.core, "rev-parse", "HEAD"),
git_text(self.core_remote, "rev-parse", "refs/heads/main"),
)
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
def test_meta_receipt_is_rechecked_after_an_earlier_successful_publication(self):
original = meta_source_tag.run
def changed_after_core(command, **kwargs):
result = original(command, **kwargs)
if (
command[:3] == ("git", "push", "--atomic")
and kwargs["cwd"] == self.core
):
(self.meta / "changed-review.txt").write_text(
"new Meta source after Core publication\n"
)
self.commit_meta()
return result
with patch.object(meta_source_tag, "run", side_effect=changed_after_core):
result = self.tag(apply=True, push=True)
self.assertEqual("partial", result["status"], result)
self.assertTrue(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
self.assertEqual("skipped", result["repositories"][1]["status"])
def test_unknown_selected_repository_cannot_use_meta_support_exception(self):
result = self.tag(repos=("govoplan", "unknown"), apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("not registered", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_selected_checkout_generator_is_never_executed(self):
malicious = self.meta / "tools/release/generate-developer-meta-package.py"
malicious.parent.mkdir(parents=True)
malicious.write_text(
'raise RuntimeError("selected checkout must not execute")\n'
)
self.commit_meta()
self.assertEqual("planned", self.tag()["status"])
if __name__ == "__main__":
unittest.main()
+146
View File
@@ -65,6 +65,8 @@ branch_labels: Union[str, Sequence[str], None] = None
) )
wrapper = development / release.name wrapper = development / release.name
wrapper.write_text( wrapper.write_text(
"from importlib import import_module\n"
'_migration = import_module("govoplan_core.backend.migrations.versions.1234_example")\n'
"revision = _migration.revision\n" "revision = _migration.revision\n"
"down_revision = _migration.down_revision\n" "down_revision = _migration.down_revision\n"
"depends_on = _migration.depends_on\n" "depends_on = _migration.depends_on\n"
@@ -79,6 +81,150 @@ branch_labels: Union[str, Sequence[str], None] = None
self.assertEqual(("base",), migration.down_revisions) self.assertEqual(("base",), migration.down_revisions)
self.assertEqual(("core",), migration.depends_on) self.assertEqual(("core",), migration.depends_on)
def test_literal_wrapper_alias_and_different_filename_are_resolved_without_execution(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_v019_example.py").write_text(
'revision = "1234"\ndown_revision = "base"\ndepends_on = "core"\nbranch_labels = None\n'
'raise AssertionError("Migration implementation must not execute")\n',
encoding="utf-8",
)
wrapper = root / "dev_versions/1234_example.py"
for alias in ("_migration", "edit_revision", "message_actions"):
with self.subTest(alias=alias):
wrapper.write_text(
"from importlib import import_module as load_migration\n"
f'{alias} = load_migration("govoplan_campaign.backend.migrations.versions." "1234_v019_example")\n'
f"revision = {alias}.revision\ndown_revision = {alias}.down_revision\n"
f"depends_on = {alias}.depends_on\nbranch_labels = {alias}.branch_labels\n"
'raise AssertionError("Wrapper must not execute")\n',
encoding="utf-8",
)
migration = audit.parse_migration_file("govoplan-campaign", wrapper)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ("base",))
self.assertEqual(migration.depends_on, ("core",))
def test_core_literal_sibling_file_wrapper_is_resolved_without_execution(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_example.py").write_text(
'revision = "1234"\ndown_revision = None\ndepends_on = None\nbranch_labels = None\n'
'raise AssertionError("Migration implementation must not execute")\n',
encoding="utf-8",
)
wrapper = root / "dev_versions/1234_example.py"
wrapper.write_text(
"from importlib.util import module_from_spec, spec_from_file_location\n"
"from pathlib import Path\n"
'_path = Path(__file__).resolve().parents[1] / "versions" / "1234_example.py"\n'
'_spec = spec_from_file_location("synthetic_migration", _path)\n'
"_module = module_from_spec(_spec)\n_spec.loader.exec_module(_module)\n"
"revision = _module.revision\ndown_revision = _module.down_revision\n"
"depends_on = _module.depends_on\nbranch_labels = _module.branch_labels\n",
encoding="utf-8",
)
migration = audit.parse_migration_file("govoplan-core", wrapper)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ())
def test_wrapper_rejects_dynamic_foreign_missing_or_rebound_targets(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_example.py").write_text('revision = "1234"\n', encoding="utf-8")
wrapper = root / "dev_versions/1234_example.py"
valid = '_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
definitions = (
'_migration = import_module(module_name)\n',
'_migration = import_module("govoplan_mail.backend.migrations.versions.1234_example")\n',
'_migration = import_module("govoplan_campaign.backend.migrations.versions...other.1234_example")\n',
'_migration = import_module("govoplan_campaign.backend.migrations.versions.missing")\n',
valid + "_migration = another_module\n",
"import_module = another_loader\n" + valid,
"def import_module(value):\n return another_module\n" + valid,
"import another_loader as import_module\n" + valid,
valid + "class _migration:\n revision = 'another'\n",
"if condition:\n _migration = another_module\n" + valid,
valid + "del _migration\n",
)
for definition in definitions:
with self.subTest(definition=definition):
wrapper.write_text(
"from importlib import import_module\n" + definition + "revision = _migration.revision\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
audit.parse_migration_file("govoplan-campaign", wrapper)
def test_wrapper_rejects_symlink_and_mixed_release_metadata(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/1234_example.py").write_text('revision = "1234"\ndown_revision = None\n', encoding="utf-8")
(root / "versions/5678_example.py").write_text('revision = "5678"\ndown_revision = None\n', encoding="utf-8")
(root / "versions/linked.py").symlink_to(root / "versions/1234_example.py")
wrapper = root / "dev_versions/1234_example.py"
definitions = (
'_migration = import_module("govoplan_campaign.backend.migrations.versions.linked")\nrevision = _migration.revision\n',
'_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
'_other = import_module("govoplan_campaign.backend.migrations.versions.5678_example")\n'
'revision = _migration.revision\ndown_revision = _other.down_revision\n',
)
for definition in definitions:
with self.subTest(definition=definition):
wrapper.write_text("from importlib import import_module\n" + definition, encoding="utf-8")
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
audit.parse_migration_file("govoplan-campaign", wrapper)
def test_unresolved_revision_expression_is_not_silently_omitted(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
path = Path(directory) / "1234_example.py"
path.write_text("revision = calculate_revision()\n", encoding="utf-8")
with self.assertRaisesRegex(ValueError, "Unsupported or ambiguous migration metadata"):
audit.parse_migration_file("govoplan-core", path)
def test_explicit_metadata_reexport_is_resolved_without_execution(self) -> None:
audit = load_audit_module()
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
root = Path(directory)
(root / "versions").mkdir()
(root / "dev_versions").mkdir()
(root / "versions/a234_example.py").write_text(
'revision = "1234"\ndown_revision = "base"\ndepends_on = None\nbranch_labels = None\n'
'raise AssertionError("Migration implementation must not execute")\n',
encoding="utf-8",
)
wrapper = root / "dev_versions/1234_example.py"
source = "govoplan_organizations.backend.migrations.versions.a234_example"
wrapper.write_text(f"from {source} import revision, down_revision, depends_on, branch_labels\n", encoding="utf-8")
migration = audit.parse_migration_file("govoplan-organizations", wrapper)
self.assertEqual(migration.revision, "1234")
self.assertEqual(migration.down_revisions, ("base",))
for declaration in (
f"from {source} import *\n",
f"from {source} import revision as down_revision\n",
f"from {source} import revision\nrevision = 'different'\n",
f"from {source} import revision\nimport another as revision\n",
f"from {source} import revision\ndef revision():\n pass\n",
f"from {source.replace('govoplan_organizations', 'govoplan_mail')} import revision\n",
):
with self.subTest(declaration=declaration):
wrapper.write_text(declaration, encoding="utf-8")
with self.assertRaises(ValueError):
audit.parse_migration_file("govoplan-organizations", wrapper)
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None: def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
audit = load_audit_module() audit = load_audit_module()
migrations = [ migrations = [
+239 -1
View File
@@ -1,10 +1,12 @@
from __future__ import annotations from __future__ import annotations
import json import json
import runpy
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import unittest import unittest
from unittest.mock import patch
from pathlib import Path from pathlib import Path
from fastapi.testclient import TestClient from fastapi.testclient import TestClient
@@ -16,6 +18,7 @@ if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT)) sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.repository_tag import tag_repositories # noqa: E402 from govoplan_release.repository_tag import tag_repositories # noqa: E402
from govoplan_release import workspace as release_workspace # noqa: E402
from server.app import create_app # noqa: E402 from server.app import create_app # noqa: E402
@@ -38,6 +41,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
version="0.1.10", version="0.1.10",
) )
add_scoped_workflow_manifest(self.manifest_repo) add_scoped_workflow_manifest(self.manifest_repo)
# The operator's test catalog explicitly registers known synthetic
# endpoints; production trust checks are not patched or bypassed.
self.registry = self.root / "registered-test-repositories.json"
self.registered = json.loads((META_ROOT / "repositories.json").read_text())
for spec in self.registered["repositories"]:
spec["remote"] = str(self.root / f"{spec['name']}.git")
self.registry.write_text(json.dumps(self.registered))
self.enterContext(patch.object(release_workspace, "REPOSITORIES_FILE", self.registry))
def tearDown(self) -> None: def tearDown(self) -> None:
self.temporary.cleanup() self.temporary.cleanup()
@@ -127,6 +138,10 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
name="govoplan-core", name="govoplan-core",
version="0.1.10", version="0.1.10",
) )
for spec in self.registered["repositories"]:
if spec["name"] == "govoplan-core":
spec["remote"] = str(remote_root / "govoplan-core.git")
self.registry.write_text(json.dumps(self.registered))
result = tag_repositories( result = tag_repositories(
repos=("govoplan-core",), repos=("govoplan-core",),
@@ -214,12 +229,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
git(self.repo, "commit", "-m", "Add release WebUI composition") git(self.repo, "commit", "-m", "Add release WebUI composition")
git(self.repo, "push", "origin", "main") git(self.repo, "push", "origin", "main")
for push in (False, True):
with self.subTest(push=push):
result = tag_repositories( result = tag_repositories(
repos=("govoplan-core", "govoplan-campaign"), repos=("govoplan-core", "govoplan-campaign"),
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"}, repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
workspace_root=self.workspace, workspace_root=self.workspace,
apply=True, apply=True,
push=True, push=push,
) )
self.assertEqual("blocked", result["status"]) self.assertEqual("blocked", result["status"])
@@ -300,6 +317,183 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
self.assertIn("Signed Website Catalog", ui.text) self.assertIn("Signed Website Catalog", ui.text)
self.assertIn("Apply + Website Tag", ui.text) self.assertIn("Apply + Website Tag", ui.text)
def test_local_module_candidate_precedes_core_lock_but_publication_does_not(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
remotes = (self.remote, self.manifest_remote, campaign_remote)
remote_refs = {path: git_text(path, "show-ref") for path in remotes}
arguments = {
"repos": ("govoplan-campaign",),
"repo_versions": {"govoplan-campaign": "0.1.10"},
"workspace_root": self.workspace,
}
preview = tag_repositories(**arguments, apply=False, push=False)
self.assertEqual("planned", preview["status"], preview)
self.assertFalse(ref_exists(campaign, "refs/tags/v0.1.10"))
candidate = tag_repositories(**arguments, apply=True, push=False)
self.assertEqual("tagged", candidate["status"], candidate)
self.assertEqual("tag", git_text(campaign, "cat-file", "-t", "v0.1.10"))
tag_object = git_text(campaign, "rev-parse", "v0.1.10")
head = git_text(campaign, "rev-parse", "HEAD")
self.assertEqual(head, git_text(campaign, "rev-parse", "v0.1.10^{commit}"))
for apply in (False, True):
with self.subTest(publish_apply=apply):
blocked = tag_repositories(**arguments, apply=apply, push=True)
self.assertEqual("blocked", blocked["status"], blocked)
self.assertIn(
"release WebUI composition gate failed",
blocked["repositories"][0]["detail"],
)
self.assertEqual(tag_object, git_text(campaign, "rev-parse", "v0.1.10"))
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
lock_path = self.repo / "webui" / "package-lock.release.json"
lock = json.loads(lock_path.read_text(encoding="utf-8"))
locked_campaign = lock["packages"]["node_modules/@govoplan/campaign-webui"]
locked_campaign["version"] = "0.1.10"
locked_campaign["resolved"] = f"git+ssh://git@example.test/acme/govoplan-campaign.git#{head}"
lock_path.write_text(json.dumps(lock) + "\n", encoding="utf-8")
git(self.repo, "add", "webui/package-lock.release.json")
git(self.repo, "commit", "-m", "Resolve reviewed local Campaign candidate")
core_candidate = tag_repositories(
repos=("govoplan-core",),
repo_versions={"govoplan-core": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("tagged", core_candidate["status"], core_candidate)
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
published = tag_repositories(**arguments, apply=True, push=True)
self.assertEqual("published", published["status"], published)
self.assertEqual(tag_object, git_text(campaign_remote, "rev-parse", "v0.1.10"))
self.assertEqual(head, git_text(campaign_remote, "rev-parse", "refs/heads/main"))
self.assertEqual(remote_refs[self.remote], git_text(self.remote, "show-ref"))
def test_local_core_candidate_still_requires_resolved_module_tags(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
for selected in (("govoplan-core",), ("govoplan-campaign", "govoplan-core")):
with self.subTest(selected=selected):
result = tag_repositories(
repos=selected,
repo_versions={repo: "0.1.10" for repo in selected},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("blocked", result["status"], result)
core_row = next(row for row in result["repositories"] if row["repo"] == "govoplan-core")
self.assertIn("version alignment gate failed", core_row["detail"])
self.assertIn("@govoplan/campaign-webui:resolved", core_row["detail"])
self.assertIn("expected 'local tag v0.1.10'", core_row["detail"])
for repository in (self.repo, self.remote, campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_local_module_candidate_preserves_version_and_worktree_gates(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
arguments = {
"repos": ("govoplan-campaign",),
"workspace_root": self.workspace,
"apply": True,
"push": False,
}
mismatch = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.11"})
self.assertEqual("blocked", mismatch["status"], mismatch)
self.assertIn("version alignment gate failed", mismatch["repositories"][0]["detail"])
(campaign / "unreviewed.txt").write_text("operator work\n", encoding="utf-8")
dirty = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.10"})
self.assertEqual("blocked", dirty["status"], dirty)
self.assertIn("worktree is not clean", dirty["repositories"][0]["detail"])
for repository in (campaign, campaign_remote):
for tag in ("v0.1.10", "v0.1.11"):
self.assertFalse(ref_exists(repository, f"refs/tags/{tag}"))
def test_local_module_candidate_preserves_manifest_gate(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
replace_with_unscoped_workflow_manifest(self.manifest_repo)
result = tag_repositories(
repos=("govoplan-campaign",),
repo_versions={"govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("scope-conditioned alternative", result["repositories"][0]["detail"])
for repository in (campaign, campaign_remote):
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
def test_local_module_candidate_preserves_remote_tag_immutability(self) -> None:
campaign, campaign_remote = self._staged_campaign_bundle()
git(campaign, "tag", "-a", "v0.1.10", "-m", "Existing immutable tag", "v0.1.9^{commit}")
git(campaign, "push", "origin", "refs/tags/v0.1.10")
remote_refs = git_text(campaign_remote, "show-ref")
for local_exists in (True, False):
with self.subTest(local_exists=local_exists):
if not local_exists:
git(campaign, "tag", "-d", "v0.1.10")
result = tag_repositories(
repos=("govoplan-campaign",),
repo_versions={"govoplan-campaign": "0.1.10"},
workspace_root=self.workspace,
apply=True,
push=False,
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("immutable tag", result["repositories"][0]["detail"])
self.assertIn("not HEAD", result["repositories"][0]["detail"])
self.assertEqual(remote_refs, git_text(campaign_remote, "show-ref"))
self.assertEqual(local_exists, ref_exists(campaign, "refs/tags/v0.1.10"))
def _staged_campaign_bundle(self) -> tuple[Path, Path]:
campaign, remote = create_release_repo(
root=self.root,
workspace=self.workspace,
name="govoplan-campaign",
version="0.1.9",
)
campaign_webui = campaign / "webui"
campaign_webui.mkdir()
package_path = campaign_webui / "package.json"
package_path.write_text(
'{"name":"@govoplan/campaign-webui","version":"0.1.9"}\n', encoding="utf-8"
)
git(campaign, "add", "webui/package.json")
git(campaign, "commit", "-m", "Prior Campaign WebUI package")
git(campaign, "tag", "-a", "v0.1.9", "-m", "Prior Campaign release")
git(campaign, "push", "origin", "main", "refs/tags/v0.1.9")
prior_commit = git_text(campaign, "rev-parse", "HEAD")
for path in (campaign / "pyproject.toml", package_path):
path.write_text(path.read_text(encoding="utf-8").replace("0.1.9", "0.1.10"), encoding="utf-8")
git(campaign, "add", "pyproject.toml", "webui/package.json")
git(campaign, "commit", "-m", "Reviewed Campaign candidate")
core_webui = self.repo / "webui"
core_webui.mkdir()
dependency_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v0.1.10"
package = {
"name": "@govoplan/core-webui",
"version": "0.1.10",
"dependencies": {"@govoplan/campaign-webui": dependency_ref},
}
(core_webui / "package.release.json").write_text(json.dumps(package) + "\n", encoding="utf-8")
(core_webui / "package-lock.release.json").write_text(
json.dumps({"packages": {
"": package,
"node_modules/@govoplan/campaign-webui": {
"version": "0.1.9",
"resolved": f"git+ssh://git@example.test/acme/govoplan-campaign.git#{prior_commit}",
},
}}) + "\n",
encoding="utf-8",
)
git(self.repo, "add", "webui/package.release.json", "webui/package-lock.release.json")
git(self.repo, "commit", "-m", "Stage Core input before candidate lock resolution")
return campaign, remote
def git(cwd: Path, *args: str) -> None: def git(cwd: Path, *args: str) -> None:
result = subprocess.run( result = subprocess.run(
@@ -338,9 +532,50 @@ def add_scoped_workflow_manifest(repo: Path) -> None:
backend.mkdir(parents=True) backend.mkdir(parents=True)
(package / "__init__.py").write_text("", encoding="utf-8") (package / "__init__.py").write_text("", encoding="utf-8")
(backend / "__init__.py").write_text("", encoding="utf-8") (backend / "__init__.py").write_text("", encoding="utf-8")
# This small workspace still has to satisfy the real presentation contract.
# Keep that prerequisite shared by both valid and intentionally unscoped
# documentation fixtures, so each test reaches its intended release gate.
canonical_areas = runpy.run_path(
str(META_ROOT / "tools" / "checks" / "check-manifest-shapes.py")
)["CANONICAL_PRODUCT_AREAS"]
(backend / "release_fixture.py").write_text(
"""from govoplan_core.core.modules import FrontendModule, ProductAreaContribution
from govoplan_core.core.views import ViewSurface
def fixture_frontend():
return FrontendModule(
module_id="access",
view_surfaces=(
ViewSurface(
id="access.section.release-fixture",
module_id="access",
kind="section",
label="Release fixture",
),
),
product_areas=tuple(
ProductAreaContribution(
id=area_id,
module_id="access",
label=label,
icon=icon,
description=description,
order=order,
surface_ids=("access.section.release-fixture",),
)
for area_id, (label, icon, description, order) in CANONICAL_AREAS.items()
),
)
CANONICAL_AREAS = """ + repr(canonical_areas) + "\n",
encoding="utf-8",
)
(backend / "manifest.py").write_text( (backend / "manifest.py").write_text(
"""from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition """from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition
from govoplan_core.core.provider_governance import declared_module_architecture from govoplan_core.core.provider_governance import declared_module_architecture
from .release_fixture import fixture_frontend
def get_manifest(): def get_manifest():
@@ -348,6 +583,7 @@ def get_manifest():
id="access", id="access",
name="Access", name="Access",
version="0.1.10", version="0.1.10",
frontend=fixture_frontend(),
permissions=( permissions=(
PermissionDefinition( PermissionDefinition(
scope="access:item:read", scope="access:item:read",
@@ -403,6 +639,7 @@ def replace_with_unscoped_workflow_manifest(repo: Path) -> None:
manifest.write_text( manifest.write_text(
"""from govoplan_core.core.modules import DocumentationTopic, ModuleManifest """from govoplan_core.core.modules import DocumentationTopic, ModuleManifest
from govoplan_core.core.provider_governance import declared_module_architecture from govoplan_core.core.provider_governance import declared_module_architecture
from .release_fixture import fixture_frontend
def get_manifest(): def get_manifest():
@@ -410,6 +647,7 @@ def get_manifest():
id="access", id="access",
name="Access", name="Access",
version="0.1.10", version="0.1.10",
frontend=fixture_frontend(),
documentation=( documentation=(
DocumentationTopic( DocumentationTopic(
id="access.workflow.unscoped", id="access.workflow.unscoped",
+1 -1
View File
@@ -1891,7 +1891,7 @@ class ReleaseRunApiTests(unittest.TestCase):
def test_ui_and_runbook_state_tracking_boundary_are_explicit(self) -> None: def test_ui_and_runbook_state_tracking_boundary_are_explicit(self) -> None:
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8") webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
runbook = (META_ROOT / "docs" / "RELEASE_CONSOLE.md").read_text( runbook = (META_ROOT / "docs" / "operations" / "RELEASE_CONSOLE.md").read_text(
encoding="utf-8" encoding="utf-8"
) )
+318
View File
@@ -0,0 +1,318 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import subprocess
import sys
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tools/release"))
from govoplan_release import source_tag_batch # noqa: E402
from govoplan_release.repository_tag import tag_repositories # noqa: E402
import test_release_meta_source_tag as meta_fixture # noqa: E402
import test_release_repository_tag as release_fixture # noqa: E402
from test_release_repository_tag import git, git_text, ref_exists # noqa: E402
class RegisteredSourceTagBatchTests(unittest.TestCase):
setUp = meta_fixture.MetaSourceTagTests.setUp
synchronize = meta_fixture.MetaSourceTagTests.synchronize
assert_no_tags = meta_fixture.MetaSourceTagTests.assert_no_tags
def tag(self, *, repos=("govoplan-access",), apply=False, push=False):
return tag_repositories(
repos=repos,
repo_versions={repo: self.version for repo in repos},
workspace_root=self.workspace,
apply=apply,
push=push,
)
def test_python_only_module_publishes_without_meta_or_core_checkout(self):
self.meta.rename(self.root / "unused-meta")
self.core.rename(self.root / "unused-core")
preview = self.tag(push=True)
self.assertEqual("planned", preview["status"], preview)
self.assertEqual("registered-source-batch-v1", preview["source_contract"])
self.assertEqual({}, preview["bundle_input_receipts"])
self.assertFalse(ref_exists(self.access, "refs/tags/v0.1.10"))
published = self.tag(apply=True, push=True)
self.assertEqual("published", published["status"], published)
self.assertEqual(["govoplan-access"], list(published["source_receipts"]))
self.assertEqual(
git_text(self.access, "rev-parse", "HEAD"),
git_text(self.access_remote, "rev-parse", "refs/heads/main"),
)
def test_core_only_batch_has_no_meta_composition_requirement(self):
self.meta.rename(self.root / "unused-meta")
result = self.tag(repos=("govoplan-core",), apply=True, push=True)
self.assertEqual("published", result["status"], result)
self.assertEqual(["govoplan-core"], list(result["source_receipts"]))
def test_backend_only_publication_never_reads_irrelevant_unsafe_core_json(self):
webui = self.core / "webui"
webui.mkdir()
(webui / "package.release.json").write_text("invalid unselected Core JSON")
(webui / "package-lock.release.json").symlink_to(
self.root / "not-a-core-release-lock"
)
from govoplan_release import version_alignment
original = version_alignment._json_object
def read(path):
if path.is_relative_to(webui):
raise AssertionError(
"backend-only publication must not read unrelated Core JSON"
)
return original(path)
with patch.object(version_alignment, "_json_object", side_effect=read):
result = self.tag(apply=True, push=True)
self.assertEqual("published", result["status"], result)
self.assertEqual({}, result["bundle_input_receipts"])
def test_source_origin_parent_and_read_only_target_guards_apply_without_meta(self):
for problem in ("origin", "parent", "read_only"):
with self.subTest(problem=problem):
parent_mode = self.root.stat().st_mode & 0o7777
git_directory = self.access / ".git"
git_mode = git_directory.stat().st_mode & 0o7777
if problem == "origin":
git(
self.access,
"config",
"remote.origin.pushurl",
str(self.root / "unknown.git"),
)
elif problem == "parent":
self.root.chmod(0o777)
else:
git_directory.chmod(0o500)
try:
for apply in (False, True):
result = self.tag(apply=apply, push=True)
self.assertEqual("blocked", result["status"], result)
self.assert_no_tags()
finally:
self.root.chmod(parent_mode)
git_directory.chmod(git_mode)
if problem == "origin":
git(self.access, "config", "--unset", "remote.origin.pushurl")
def test_wrong_owner_symlink_and_hidden_index_are_rejected_without_meta(self):
config = self.access / ".git/config"
original_stat = Path.lstat
def wrong_owner(path, *args, **kwargs):
observed = original_stat(path, *args, **kwargs)
if path == config:
fields = list(observed)
fields[4] = os.geteuid() + 1
return os.stat_result(fields)
return observed
with patch.object(Path, "lstat", new=wrong_owner):
self.assertEqual("blocked", self.tag(apply=True)["status"])
moved = self.root / "moved-access"
self.access.rename(moved)
self.access.symlink_to(moved, target_is_directory=True)
try:
self.assertEqual("blocked", self.tag(apply=True)["status"])
finally:
self.access.unlink()
moved.rename(self.access)
for flag, undo in (
("--assume-unchanged", "--no-assume-unchanged"),
("--skip-worktree", "--no-skip-worktree"),
):
with self.subTest(flag=flag):
git(self.access, "update-index", flag, "pyproject.toml")
try:
self.assertEqual("blocked", self.tag(apply=True)["status"])
self.assert_no_tags()
finally:
git(self.access, "update-index", undo, "pyproject.toml")
def test_live_remote_divergence_blocks_without_cached_tracking_update(self):
clone = self.root / "other-access-writer"
git(self.root, "clone", "--branch", "main", str(self.access_remote), str(clone))
git(clone, "config", "user.name", "Synthetic writer")
git(clone, "config", "user.email", "writer@example.invalid")
(clone / "advance.txt").write_text("new remote main\n")
git(clone, "add", ".")
git(clone, "commit", "-m", "Advance remote without updating original tracking")
git(clone, "push", "origin", "main")
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("live origin/main", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_ignored_selected_version_metadata_cannot_supply_an_untagged_artifact(self):
project = self.access / "pyproject.toml"
original = project.read_text()
git(self.access, "rm", "pyproject.toml")
(self.access / ".gitignore").write_text("/pyproject.toml\n")
git(self.access, "add", ".gitignore")
git(self.access, "commit", "-m", "Ignored metadata absent from frozen tree")
project.write_text(original)
self.assertEqual("", git_text(self.access, "status", "--porcelain"))
result = self.tag(apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn("must be tracked", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_lightweight_and_different_annotation_objects_remain_immutable(self):
git(self.access, "tag", "v0.1.10")
self.assertEqual("blocked", self.tag(apply=True)["status"])
git(self.access, "tag", "-d", "v0.1.10")
git(self.access, "tag", "-a", "v0.1.10", "-m", "Original immutable annotation")
git(self.access, "push", "origin", "refs/tags/v0.1.10")
original = git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
git(self.access, "tag", "-d", "v0.1.10")
git(
self.access,
"tag",
"-a",
"v0.1.10",
"-m",
"Conflicting immutable annotation",
)
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
self.assertEqual(
original, git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
)
def test_changed_source_after_preflight_stops_before_first_batch_effect(self):
original = source_tag_batch._preview_repositories
def changed(**kwargs):
result = original(**kwargs)
(self.access / "changed-source.txt").write_text(
"new source after preflight\n"
)
git(self.access, "add", ".")
git(self.access, "commit", "-m", "Source changed")
return result
with patch.object(
source_tag_batch, "_preview_repositories", side_effect=changed
):
result = self.tag(
repos=("govoplan-core", "govoplan-access"), apply=True, push=True
)
self.assertEqual("blocked", result["status"], result)
self.assertIn("receipt changed", result["repositories"][0]["detail"])
self.assert_no_tags()
def test_false_push_success_is_not_a_receipt_and_never_retries(self):
original = source_tag_batch.run
pushes = []
def run(command, **kwargs):
if command[:3] == ("git", "push", "--atomic"):
pushes.append(command)
return subprocess.CompletedProcess(command, 0, "", "")
return original(command, **kwargs)
with patch.object(source_tag_batch, "run", side_effect=run):
result = self.tag(
repos=("govoplan-access", "govoplan-core"), apply=True, push=True
)
self.assertEqual("partial", result["status"], result)
self.assertEqual(1, len(pushes))
self.assertEqual("skipped", result["repositories"][1]["status"])
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(self.access_remote, "refs/tags/v0.1.10"))
def _ready_bundle(self):
self.repo = self.core # Existing fixture's Core name.
campaign, remote = (
release_fixture.ReleaseRepositoryTagTests._staged_campaign_bundle(self)
)
self.specs.append(
{
"name": "govoplan-campaign",
"category": "module",
"subtype": "domain",
"path": "govoplan-campaign",
"remote": str(remote),
}
)
self.registry.write_text(json.dumps({"repositories": self.specs}))
self.assertEqual(
"tagged", self.tag(repos=("govoplan-campaign",), apply=True)["status"]
)
lock_path = self.core / "webui/package-lock.release.json"
payload = json.loads(lock_path.read_text())
package = payload["packages"]["node_modules/@govoplan/campaign-webui"]
package["version"] = self.version
package["resolved"] = (
"git+ssh://git@example.test/acme/govoplan-campaign.git#"
+ git_text(campaign, "rev-parse", "HEAD")
)
lock_path.write_text(json.dumps(payload))
# Core is a reviewed input only here: do not require an unrelated tag
# or silently impose a new clean-Core prerequisite for module release.
return campaign, remote, lock_path
def test_module_publication_freezes_core_inputs_without_requiring_core_tag(self):
_campaign, _remote, _lock = self._ready_bundle()
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
self.assertEqual("published", result["status"], result)
self.assertEqual(["govoplan-campaign"], list(result["source_receipts"]))
self.assertEqual(
{"webui/package.release.json", "webui/package-lock.release.json"},
set(result["bundle_input_receipts"]),
)
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
def test_changed_or_group_writable_core_bundle_inputs_block_module_publication(
self,
):
campaign, remote, lock = self._ready_bundle()
original = source_tag_batch._preview_repositories
def changed(**kwargs):
result = original(**kwargs)
lock.write_text(lock.read_text() + "\n")
return result
with patch.object(
source_tag_batch, "_preview_repositories", side_effect=changed
):
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
self.assertEqual("blocked", result["status"], result)
self.assertIn(
"bundle input receipt changed", result["repositories"][0]["detail"]
)
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
lock.chmod(0o666)
try:
self.assertEqual(
"blocked",
self.tag(repos=("govoplan-campaign",), apply=True, push=True)["status"],
)
finally:
lock.chmod(0o644)
self.assertTrue(ref_exists(campaign, "refs/tags/v0.1.10"))
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
def test_read_only_shared_preflight_has_no_apply_or_mutating_legacy_entry(self):
import inspect
from govoplan_release import repository_tag
self.assertNotIn(
"apply", inspect.signature(repository_tag._preview_repositories).parameters
)
self.assertFalse(hasattr(repository_tag, "_tag_repositories_legacy"))
if __name__ == "__main__":
unittest.main()
+35
View File
@@ -13,12 +13,47 @@ if str(RELEASE_ROOT) not in sys.path:
sys.path.insert(0, str(RELEASE_ROOT)) sys.path.insert(0, str(RELEASE_ROOT))
from govoplan_release.version_metadata import ( # noqa: E402 from govoplan_release.version_metadata import ( # noqa: E402
VersionMetadataError,
apply_version_metadata_mutations, apply_version_metadata_mutations,
version_metadata_mutations, version_metadata_mutations,
) )
class ReleaseVersionMetadataTests(unittest.TestCase): class ReleaseVersionMetadataTests(unittest.TestCase):
def test_updates_shared_module_version_without_rewriting_independent_interfaces(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
backend = root / "src" / "govoplan_example" / "backend"
backend.mkdir(parents=True)
manifest = backend / "manifest.py"
manifest.write_text(
'MODULE_VERSION: str = "1.2.3"\n'
'manifest = ModuleManifest(id="example", version=MODULE_VERSION,\n'
' provides_interfaces=(ModuleInterfaceProvider(name="api", version="2.0"),))\n',
encoding="utf-8",
)
changed = apply_version_metadata_mutations(root, target_version="1.2.4")
self.assertEqual(("src/govoplan_example/backend/manifest.py",), changed)
self.assertIn('MODULE_VERSION: str = "1.2.4"', manifest.read_text())
self.assertIn('version="2.0"', manifest.read_text())
self.assertEqual((), version_metadata_mutations(root, target_version="1.2.4"))
def test_dynamic_module_version_fails_before_any_metadata_is_written(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
backend = root / "src" / "govoplan_example" / "backend"
backend.mkdir(parents=True)
project = root / "pyproject.toml"
project.write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
before = project.read_bytes()
(backend / "manifest.py").write_text(
'MODULE_VERSION = compute_version()\n'
'manifest = ModuleManifest(id="example", version=MODULE_VERSION)\n',
)
with self.assertRaisesRegex(VersionMetadataError, "no literal MODULE_VERSION"):
apply_version_metadata_mutations(root, target_version="1.2.4")
self.assertEqual(before, project.read_bytes())
def test_updates_recognized_metadata_without_changing_interface_versions( def test_updates_recognized_metadata_without_changing_interface_versions(
self, self,
) -> None: ) -> None:
+4
View File
@@ -98,6 +98,10 @@ class SecurityAuditWrapperTests(unittest.TestCase):
if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then
exit 0 exit 0
fi fi
if [[ " $* " != *" --redact=100 "* ]]; then
echo 'secret scans must redact reports and logs' >&2
exit 3
fi
output='' output=''
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
if [[ "$1" == '--report-path' ]]; then if [[ "$1" == '--report-path' ]]; then
+78
View File
@@ -0,0 +1,78 @@
from __future__ import annotations
import json
from pathlib import Path
import runpy
import tempfile
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
CHECK = runpy.run_path(str(META_ROOT / "tools/checks/check-webui-package-facades.py"))
class WebuiPackageFacadeTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.repository = Path(self.temporary.name) / "govoplan-example"
source = self.repository / "webui/src"
source.mkdir(parents=True)
(source / "index.ts").write_text("export {};\n", encoding="utf-8")
(source / "styles.css").write_text(":root {}\n", encoding="utf-8")
self.webui = {
"name": "@govoplan/example-webui", "version": "0.1.2", "type": "module",
"main": "src/index.ts",
"exports": {".": {"import": "./src/index.ts"}, "./styles.css": "./src/styles.css"},
"peerDependencies": {"@govoplan/core-webui": "^0.1.45"},
"peerDependenciesMeta": {"@govoplan/core-webui": {"optional": True}},
}
self.root = {**self.webui, **{
field: CHECK["prefixed_entries"](self.webui[field])
for field in CHECK["ENTRY_FIELDS"] if field in self.webui
}}
self.write_manifests()
def write_manifests(self) -> None:
(self.repository / "webui/package.json").write_text(json.dumps(self.webui), encoding="utf-8")
(self.repository / "package.json").write_text(json.dumps(self.root), encoding="utf-8")
def issues(self) -> list[str]:
return CHECK["facade_issues"](self.repository, package_name="@govoplan/example-webui")
def test_matching_conditional_and_css_entries_are_accepted(self) -> None:
self.assertEqual([], self.issues())
def test_missing_root_or_generic_package_is_rejected(self) -> None:
(self.repository / "package.json").unlink()
self.assertIn("cannot read", " ".join(self.issues()))
self.root = {"name": "@govoplan/example", "version": "0.1.2"}
self.write_manifests()
self.assertIn("root name differs", " ".join(self.issues()))
self.assertIn("no WebUI entry point", " ".join(self.issues()))
def test_peer_drift_or_missing_entry_is_rejected(self) -> None:
self.root["peerDependencies"] = {"@govoplan/core-webui": "^9.0.0"}
self.write_manifests()
self.assertIn("peerDependencies differs", " ".join(self.issues()))
(self.repository / "webui/src/styles.css").unlink()
self.assertIn("missing exports entry", " ".join(self.issues()))
def test_entry_cannot_escape_webui_even_if_it_exists(self) -> None:
(self.repository / "outside.ts").write_text("export {};\n", encoding="utf-8")
self.root["main"] = "webui/../outside.ts"
self.write_manifests()
self.assertIn("escapes webui/", " ".join(self.issues()))
def test_release_composition_checks_only_declared_module_sources(self) -> None:
core = self.repository.parent / "govoplan-core/webui"
core.mkdir(parents=True)
(core / "package.release.json").write_text(json.dumps({"dependencies": {
"react": "19.2.7",
"@govoplan/example-webui": "git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-example.git#v0.1.2",
}}), encoding="utf-8")
self.assertEqual((1, []), CHECK["check_composition"](self.repository.parent))
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,154 @@
from __future__ import annotations
import os
from pathlib import Path
import subprocess
import tempfile
import textwrap
import unittest
META_ROOT = Path(__file__).resolve().parents[1]
INSTALLER = META_ROOT / "tools" / "release" / "install-webui-release-dependencies.sh"
STAGES = ("base", "clone", "modules")
class WebUIReleaseDependencyRetryTests(unittest.TestCase):
def _run_installer(
self, stage: str, statuses: tuple[int, ...]
) -> tuple[subprocess.CompletedProcess[str], list[str]]:
with tempfile.TemporaryDirectory(prefix="govoplan-installer-retry-test-") as directory:
root = Path(directory)
stub_bin = root / "bin"
stub_bin.mkdir()
core_root = root / "core"
webui = core_root / "web ui"
webui.mkdir(parents=True)
work_root = root / "work"
work_root.mkdir()
log = root / "commands.log"
stub = "#!/usr/bin/env bash\nset -euo pipefail\n" + textwrap.dedent(
r"""
case "${0##*/}" in
node)
# Supply the shell's dependency list without requiring Node.
printf '%s\t%s\n' '@govoplan/example-webui' \
'git+https://example.invalid/module.git#v1.0.0' > "$GOVOPLAN_DEPS"
printf 'node\n' >> "$RETRY_TEST_LOG"
exit 0
;;
sleep)
printf 'sleep %s\n' "$*" >> "$RETRY_TEST_LOG"
exit 0
;;
npm)
case "${1:-}" in
cache)
printf 'cache\n' >> "$RETRY_TEST_LOG"
exit 0
;;
install)
stage=base
for argument in "$@"; do
if [[ "$argument" == --no-save ]]; then
stage=modules
fi
done
;;
*) exit 98 ;;
esac
;;
git)
[[ "${1:-}" == clone ]] || exit 98
stage=clone
;;
*) exit 98 ;;
esac
status=0
if [[ "$stage" == "$RETRY_TEST_STAGE" ]]; then
attempt=0
counter="$RETRY_TEST_ROOT/$stage.count"
if [[ -f "$counter" ]]; then
read -r attempt < "$counter"
fi
read -r -a statuses <<< "$RETRY_TEST_STATUSES"
status="${statuses[$attempt]:-99}"
printf '%s\n' "$((attempt + 1))" > "$counter"
fi
printf '%s %s\n' "$stage" "$status" >> "$RETRY_TEST_LOG"
exit "$status"
"""
)
for name in ("node", "npm", "git", "sleep"):
executable = stub_bin / name
executable.write_text(stub, encoding="utf-8")
executable.chmod(0o755)
env = os.environ.copy()
env.update(
{
"PATH": f"{stub_bin}:{os.defpath}",
"TMPDIR": str(work_root),
"GOVOPLAN_CORE_ROOT": str(core_root),
"GOVOPLAN_WEBUI_PACKAGE_LOCK": "",
"GOVOPLAN_WEBUI_PACKAGE_DIR": "",
"RETRY_TEST_ROOT": str(root),
"RETRY_TEST_LOG": str(log),
"RETRY_TEST_STAGE": stage,
"RETRY_TEST_STATUSES": " ".join(map(str, statuses)),
}
)
result = subprocess.run(
[
"bash",
"-c",
'set -euo pipefail; bash "$1" "$2"; '
'printf "caller-continued\\n" >> "$RETRY_TEST_LOG"',
"retry-test-caller",
str(INSTALLER),
str(webui),
],
cwd=root,
env=env,
text=True,
capture_output=True,
timeout=10,
check=False,
)
self.assertEqual(list(work_root.iterdir()), [], result.stderr)
return result, log.read_text(encoding="utf-8").splitlines()
def _assert_attempts(self, statuses: tuple[int, ...]) -> None:
for retried_stage in STAGES:
with self.subTest(stage=retried_stage, statuses=statuses):
result, commands = self._run_installer(retried_stage, statuses)
expected = ["node", "cache"]
for stage in STAGES:
attempts = statuses if stage == retried_stage else (0,)
for index, status in enumerate(attempts):
expected.append(f"{stage} {status}")
if status and index < 2:
expected.append(f"sleep {(index + 1) * 10}")
if attempts[-1]:
break
if statuses[-1] == 0:
expected.append("caller-continued")
self.assertEqual(result.returncode, statuses[-1], result.stderr)
self.assertEqual(commands, expected, result.stderr)
def test_success_on_first_attempt(self) -> None:
self._assert_attempts((0,))
def test_success_on_second_attempt(self) -> None:
self._assert_attempts((17, 0))
def test_success_on_third_attempt(self) -> None:
self._assert_attempts((17, 23, 0))
def test_exhaustion_preserves_final_status_and_stops_callers(self) -> None:
self._assert_attempts((17, 23, 47))
if __name__ == "__main__":
unittest.main()
+58 -2
View File
@@ -39,13 +39,20 @@ GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact "$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py" PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py"
"$NODE/node" "$META_ROOT/tests/test-jsx-value-imports.mjs"
"$NODE/node" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
"$NODE/node" "$META_ROOT/../govoplan-files/webui/scripts/test-archive-client.mjs"
cd "$META_ROOT" cd "$META_ROOT"
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints "$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
"$PYTHON" tools/repo/sync-module-package-workflows.py --check "$PYTHON" tools/repo/sync-module-package-workflows.py --check
"$PYTHON" tools/release/generate-developer-meta-package.py --check "$PYTHON" tools/release/generate-developer-meta-package.py --check
"$PYTHON" tools/checks/check-webui-package-facades.py
"$PYTHON" -m unittest tests.test_webui_package_facades
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release "$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
"$PYTHON" -m unittest tests.test_deployment_installer "$PYTHON" -m unittest tests.test_deployment_installer tests.test_webui_release_dependency_retries
"$PYTHON" -m pytest -q tests/test_release_meta_source_tag.py tests/test_release_source_tag_batch.py tests/test_release_meta_preparation.py
"$PYTHON" -m unittest tests.test_isolated_work_composition
"$PYTHON" -m unittest tests.test_capability_fit_evidence "$PYTHON" -m unittest tests.test_capability_fit_evidence
"$PYTHON" -m unittest tests.test_capability_fit_generation tests.test_capability_fit_review "$PYTHON" -m unittest tests.test_capability_fit_generation tests.test_capability_fit_review
"$PYTHON" tools/assessments/generate-capability-fit-report.py --check "$PYTHON" tools/assessments/generate-capability-fit-report.py --check
@@ -96,6 +103,22 @@ PY
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-primitives.py" "$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-primitives.py"
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-foundations.py" "$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-foundations.py"
"$PYTHON" -m unittest tests.test_module_system "$PYTHON" -m unittest tests.test_module_system
"$PYTHON" -m unittest tests.test_bounded_process
"$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api
"$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves
"$PYTHON" -m pytest -q \
/mnt/DATA/git/govoplan-files/tests/test_managed_archives.py \
/mnt/DATA/git/govoplan-files/tests/test_archive_work.py \
/mnt/DATA/git/govoplan-files/tests/test_archive_staging.py \
/mnt/DATA/git/govoplan-files/tests/test_upload_response_batching.py \
/mnt/DATA/git/govoplan-files/tests/test_archive_performance.py
"$PYTHON" -m pytest -q \
/mnt/DATA/git/govoplan-files/tests/test_archive_workers.py \
/mnt/DATA/git/govoplan-files/tests/test_archive_inspection_bounds.py \
/mnt/DATA/git/govoplan-files/tests/test_archives.py
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests/test_external_function_mapping_migration.py
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-templates/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
@@ -116,19 +139,45 @@ PY
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests "$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py "$PYTHON" -m pytest -q \
/mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py \
/mnt/DATA/git/govoplan-campaign/tests/test_editor_state_security.py \
/mnt/DATA/git/govoplan-campaign/tests/test_mail_profile_boundary.py \
/mnt/DATA/git/govoplan-campaign/tests/test_independent_configuration_repairs.py \
/mnt/DATA/git/govoplan-campaign/tests/test_incremental_review_persistence.py \
/mnt/DATA/git/govoplan-campaign/tests/test_reviewed_build_mock.py \
/mnt/DATA/git/govoplan-campaign/tests/test_delivery_policy_settings.py \
/mnt/DATA/git/govoplan-campaign/tests/test_synchronous_delivery_policy.py \
/mnt/DATA/git/govoplan-campaign/tests/test_workerless_recovery.py \
/mnt/DATA/git/govoplan-campaign/tests/test_imap_batch_integration.py \
/mnt/DATA/git/govoplan-campaign/tests/test_testbed_claim_recovery.py \
/mnt/DATA/git/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py \
/mnt/DATA/git/govoplan-campaign/tests/test_archive_encryption_governance.py \
/mnt/DATA/git/govoplan-policy/tests/test_campaign_archive_encryption.py \
/mnt/DATA/git/govoplan-policy/tests/test_archive_encryption_api.py
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py" "$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py" "$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-mail/tests/test_campaign_protocol_authorization.py /mnt/DATA/git/govoplan-mail/tests/test_campaign_imap_batch.py
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests "$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count "$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
"$PYTHON" -m unittest \
tests.test_api_smoke.ApiSmokeTests.test_managed_attachment_patterns_preview_build_and_mock_send \
tests.test_api_smoke.ApiSmokeTests.test_reports_and_job_review_are_scoped_to_the_selected_version \
tests.test_api_smoke.ApiSmokeTests.test_worker_loss_becomes_unknown_and_requires_reconciliation_before_retry
cd "$ROOT/webui" cd "$ROOT/webui"
"$NPM" run test:api-client-cache
"$NPM" run test:auth-action-state
"$NPM" run test:dependency-security
"$NPM" run test:layout-primitives "$NPM" run test:layout-primitives
"$NPM" run test:mail-components "$NPM" run test:mail-components
"$NPM" run test:module-capabilities "$NPM" run test:module-capabilities
"$NPM" run test:module-permutations "$NPM" run test:module-permutations
"$NPM" run test:conformance "$NPM" run test:conformance
cd /mnt/DATA/git/govoplan-access/webui
"$NPM" run test:passwords
"$WEBUI_BIN/tsc" -p /mnt/DATA/git/govoplan-payments/webui/tsconfig.json "$WEBUI_BIN/tsc" -p /mnt/DATA/git/govoplan-payments/webui/tsconfig.json
cd /mnt/DATA/git/govoplan-payments/webui cd /mnt/DATA/git/govoplan-payments/webui
@@ -155,12 +204,19 @@ cd /mnt/DATA/git/govoplan-postbox/webui
cd /mnt/DATA/git/govoplan-mail/webui cd /mnt/DATA/git/govoplan-mail/webui
"$NPM" run test:mail-ui "$NPM" run test:mail-ui
cd /mnt/DATA/git/govoplan-files/webui
"$NPM" run test:managed-archive
cd /mnt/DATA/git/govoplan-campaign/webui cd /mnt/DATA/git/govoplan-campaign/webui
"$NPM" run test:policy-ui "$NPM" run test:policy-ui
"$NPM" run test:template-preview "$NPM" run test:template-preview
"$NPM" run test:review-workflow
"$NPM" run test:accessibility-contract "$NPM" run test:accessibility-contract
"$NPM" run test:campaign-collaboration "$NPM" run test:campaign-collaboration
"$NPM" run test:campaign-work "$NPM" run test:campaign-work
cd /mnt/DATA/git/govoplan-policy/webui
"$NPM" run test:archive-encryption
cd /mnt/DATA/git/govoplan-wiki/webui cd /mnt/DATA/git/govoplan-wiki/webui
"$NPM" run test:interface-pattern "$NPM" run test:interface-pattern
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env node
/** Reject erased type-only imports used as runtime JSX component tags. */
import { readFileSync, readdirSync, existsSync } from "node:fs";
import { createRequire } from "node:module";
import { resolve, relative } from "node:path";
import { fileURLToPath } from "node:url";
const workspaceRoot = resolve(import.meta.dirname, "../../..");
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
const ts = require("typescript");
function isTypeOnlyImport(declaration) {
if (ts.isImportSpecifier(declaration)) return declaration.isTypeOnly || declaration.parent.parent.isTypeOnly;
if (ts.isNamespaceImport(declaration)) return declaration.parent.isTypeOnly;
return (ts.isImportClause(declaration) || ts.isImportEqualsDeclaration(declaration)) && declaration.isTypeOnly;
}
/** Resolve lexical bindings, including shadowing; do not typecheck unrelated
* optional dependencies or report ordinary application diagnostics.
*/
export function findTypeOnlyJsxImports(sources) {
const files = new Map(sources.map(({ path, source }) => [resolve(path),
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
const host = ts.createCompilerHost(options);
host.getSourceFile = (path) => files.get(resolve(path));
const program = ts.createProgram([...files.keys()], options, host);
const checker = program.getTypeChecker();
const findings = [];
for (const [path, source] of files) {
function visit(node) {
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
let root = node.tagName;
// Lower-case direct tags are intrinsic HTML, not runtime bindings.
if (!(ts.isIdentifier(root) && /^[a-z]/.test(root.text))) {
while (ts.isPropertyAccessExpression(root)) root = root.expression;
const declarations = checker.getSymbolAtLocation(root)?.declarations ?? [];
if (declarations.some(isTypeOnlyImport)) {
const position = source.getLineAndCharacterOfPosition(node.tagName.getStart(source));
findings.push({ path, line: position.line + 1, column: position.character + 1, component: node.tagName.getText(source) });
}
}
}
ts.forEachChild(node, visit);
}
visit(source);
}
return findings;
}
function sourceFiles(directory) {
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
const path = resolve(directory, entry.name);
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
});
}
export function checkWorkspace(root = workspaceRoot) {
const modules = readdirSync(root, { withFileTypes: true })
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
const paths = modules.flatMap(sourceFiles);
const findings = findTypeOnlyJsxImports(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
for (const finding of findings) {
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: JSX component ${finding.component} is imported type-only and will be erased at runtime.`);
}
if (!findings.length) console.log(`JSX runtime-import contract passed: ${paths.length} TSX files across ${modules.length} WebUI modules.`);
return findings.length ? 1 : 0;
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
process.exitCode = checkWorkspace();
}
+3
View File
@@ -677,6 +677,7 @@ run_gitleaks() {
prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2 prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2
prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2 prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2
gitleaks git \ gitleaks git \
--redact=100 \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
--report-path "$REPORTS_DIR/gitleaks-history-$name.json" \ --report-path "$REPORTS_DIR/gitleaks-history-$name.json" \
@@ -687,6 +688,7 @@ run_gitleaks() {
# Scan the directory as well so pre-commit audits cover the exact code # Scan the directory as well so pre-commit audits cover the exact code
# under review, while retaining the history scan above. # under review, while retaining the history scan above.
gitleaks dir \ gitleaks dir \
--redact=100 \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
--report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \ --report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \
@@ -696,6 +698,7 @@ run_gitleaks() {
else else
prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2 prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2
gitleaks detect \ gitleaks detect \
--redact=100 \
--source "$repo" \ --source "$repo" \
--config "$ROOT/.gitleaks.toml" \ --config "$ROOT/.gitleaks.toml" \
--report-format json \ --report-format json \
@@ -86,6 +86,12 @@ CENTRAL_COMPONENTS = {
"CountBadge": pathlib.Path( "CountBadge": pathlib.Path(
"govoplan-core/webui/src/components/CountBadge.tsx" "govoplan-core/webui/src/components/CountBadge.tsx"
), ),
"MultiSelectFilter": pathlib.Path(
"govoplan-core/webui/src/components/MultiSelectFilter.tsx"
),
"ListSelectionFilter": pathlib.Path(
"govoplan-core/webui/src/components/ListSelectionFilter.tsx"
),
"SelectionList": pathlib.Path( "SelectionList": pathlib.Path(
"govoplan-core/webui/src/components/SelectionList.tsx" "govoplan-core/webui/src/components/SelectionList.tsx"
), ),
@@ -190,7 +196,21 @@ REQUIRED_CONSUMERS = {
"CountBadge": ( "CountBadge": (
pathlib.Path("govoplan-core/webui/src/layout/Titlebar.tsx"), pathlib.Path("govoplan-core/webui/src/layout/Titlebar.tsx"),
pathlib.Path("govoplan-mail/webui/src/features/mail/MailboxPage.tsx"), pathlib.Path("govoplan-mail/webui/src/features/mail/MailboxPage.tsx"),
),
# Search's old count badge was part of a retired module-local filter menu.
# Both surfaces must now compose the owning facet adapter and Core dropdown.
"SearchFilters": (
pathlib.Path("govoplan-search/webui/src/features/search/SearchPage.tsx"), pathlib.Path("govoplan-search/webui/src/features/search/SearchPage.tsx"),
pathlib.Path("govoplan-search/webui/src/components/GlobalSearch.tsx"),
),
"MultiSelectFilter": (
pathlib.Path("govoplan-search/webui/src/components/SearchFilters.tsx"),
pathlib.Path("govoplan-notifications/webui/src/features/notifications/NotificationCenterPage.tsx"),
pathlib.Path("govoplan-docs/webui/src/features/docs/DocsPage.tsx"),
),
"ListSelectionFilter": (
pathlib.Path("govoplan-core/webui/src/components/MultiSelectFilter.tsx"),
pathlib.Path("govoplan-core/webui/src/components/table/DataGrid.tsx"),
), ),
"SelectionListItemContent": ( "SelectionListItemContent": (
pathlib.Path("govoplan-approvals/webui/src/features/approvals/ApprovalsPage.tsx"), pathlib.Path("govoplan-approvals/webui/src/features/approvals/ApprovalsPage.tsx"),
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env python3
"""Check that release Git dependencies expose their owning WebUI package."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
META_ROOT = Path(__file__).resolve().parents[2]
GIT_REPOSITORY = re.compile(r"/(govoplan-[a-z0-9-]+)\.git#v[^/]+$")
PARITY_FIELDS = (
"name", "version", "type", "dependencies", "optionalDependencies",
"peerDependencies", "peerDependenciesMeta",
)
ENTRY_FIELDS = ("main", "module", "types", "exports")
def prefixed_entries(value: object) -> object:
if isinstance(value, str):
return "./webui/" + value[2:] if value.startswith("./") else "webui/" + value
if isinstance(value, dict):
return {key: prefixed_entries(item) for key, item in value.items()}
if isinstance(value, list):
return [prefixed_entries(item) for item in value]
return value
def entry_paths(value: object) -> list[str]:
if isinstance(value, str):
return [value]
if isinstance(value, dict):
return [path for item in value.values() for path in entry_paths(item)]
if isinstance(value, list):
return [path for item in value for path in entry_paths(item)]
return []
def facade_issues(repository: Path, *, package_name: str) -> list[str]:
issues: list[str] = []
try:
root = json.loads((repository / "package.json").read_text(encoding="utf-8"))
webui = json.loads((repository / "webui/package.json").read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
return [f"{repository.name}: cannot read package facades: {exc}"]
if not isinstance(root, dict) or not isinstance(webui, dict):
return [f"{repository.name}: package manifests must be JSON objects"]
if webui.get("name") != package_name:
issues.append(f"{repository.name}: WebUI name does not match {package_name}")
for field in PARITY_FIELDS:
if root.get(field) != webui.get(field):
issues.append(f"{repository.name}: root {field} differs from owning WebUI package")
for field in ENTRY_FIELDS:
expected = prefixed_entries(webui.get(field))
if root.get(field) != expected:
issues.append(f"{repository.name}: root {field} must target the corresponding webui/ entry")
for entry in entry_paths(root.get(field)):
path = repository / entry
if not path.resolve().is_relative_to((repository / "webui").resolve()):
issues.append(f"{repository.name}: {field} entry escapes webui/: {entry}")
elif "*" not in entry and not path.is_file():
issues.append(f"{repository.name}: missing {field} entry: {entry}")
if not root.get("exports") and not root.get("main"):
issues.append(f"{repository.name}: root package has no WebUI entry point")
return issues
def check_composition(workspace: Path, *, core_root: Path | None = None) -> tuple[int, list[str]]:
core = core_root or workspace / "govoplan-core"
release = json.loads((core / "webui/package.release.json").read_text(encoding="utf-8"))
checked = 0
issues: list[str] = []
for name, reference in release.get("dependencies", {}).items():
if not name.startswith("@govoplan/"):
continue
match = GIT_REPOSITORY.search(reference) if isinstance(reference, str) else None
if match is None:
issues.append(f"{name}: release dependency is not a versioned GovOPlaN Git source")
continue
checked += 1
issues.extend(facade_issues(workspace / match.group(1), package_name=name))
return checked, issues
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--workspace-root", type=Path, default=META_ROOT.parent)
parser.add_argument("--core-root", type=Path)
args = parser.parse_args()
checked, issues = check_composition(args.workspace_root, core_root=args.core_root)
if issues:
print("\n".join(issues))
return 1
print(f"Release WebUI package facade checks passed for {checked} Git dependencies")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+5 -2
View File
@@ -4,8 +4,11 @@
"certificates": "Contract-only module: certificate issuance and revocation persistence are not implemented; reassess before adding a migration-owned store.", "certificates": "Contract-only module: certificate issuance and revocation persistence are not implemented; reassess before adding a migration-owned store.",
"consultation": "Contract-only module: consultation submissions and evaluation persistence are not implemented; reassess before adding a migration-owned store.", "consultation": "Contract-only module: consultation submissions and evaluation persistence are not implemented; reassess before adding a migration-owned store.",
"contracts": "Contract-only module: contract, amendment, and obligation persistence are not implemented; reassess before adding a migration-owned store.", "contracts": "Contract-only module: contract, amendment, and obligation persistence are not implemented; reassess before adding a migration-owned store.",
"dms": "Stateless integration-preview module: DMS retains no document, person, credential, or provider-response store; Files and Records remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, or diagnostic.",
"erp": "Stateless integration-contract module: ERP retains no invoice, payable, plan, booking observation, provider response, or credential store; Procurement, Payments, Ledger, Files, and Audit remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, reconciliation decision, or diagnostic.",
"evaluation": "Contract-only module: evaluation runs, responses, and scores are not persisted; reassess before adding a migration-owned store.", "evaluation": "Contract-only module: evaluation runs, responses, and scores are not persisted; reassess before adding a migration-owned store.",
"facilities": "Contract-only module: facility and maintenance persistence are not implemented; reassess before adding a migration-owned store.", "facilities": "Contract-only module: facility and maintenance persistence are not implemented; reassess before adding a migration-owned store.",
"fit_connect": "Stateless transport-contract module: FIT-Connect retains no submission, attachment, receipt, acknowledgement plan, key, provider response, or diagnostic store; the owning Service, Forms, Cases, Files, and Audit workflows remain responsible for subject data. Reassess before persisting any ingress or event-log evidence.",
"grants": "Contract-only module: grant applications, awards, and monitoring are not persisted; reassess before adding a migration-owned store.", "grants": "Contract-only module: grant applications, awards, and monitoring are not persisted; reassess before adding a migration-owned store.",
"inspections": "Contract-only module: inspections, findings, and measures are not persisted; reassess before adding a migration-owned store.", "inspections": "Contract-only module: inspections, findings, and measures are not persisted; reassess before adding a migration-owned store.",
"learning": "Contract-only module: learning offers, enrollment, and completion are not persisted; reassess before adding a migration-owned store.", "learning": "Contract-only module: learning offers, enrollment, and completion are not persisted; reassess before adding a migration-owned store.",
@@ -16,7 +19,7 @@
"resources": "Contract-only module: resource catalog and allocation persistence are not implemented; reassess before adding a migration-owned store.", "resources": "Contract-only module: resource catalog and allocation persistence are not implemented; reassess before adding a migration-owned store.",
"rest": "Transport-only module: REST binds explicitly published functions and owns no domain or subject-data store.", "rest": "Transport-only module: REST binds explicitly published functions and owns no domain or subject-data store.",
"soap": "Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store.", "soap": "Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store.",
"tenancy": "Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data.",
"transparency": "Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store.", "transparency": "Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store.",
"workflow": "Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage." "workflow": "Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage.",
"xrechnung": "Stateless validation-contract module: XRechnung persists no invoice, report, diagnostic, or handoff; the invoking Files, Procurement, or Payments workflow remains the subject-data owner. Reassess before adding a validation store."
} }
@@ -27,6 +27,7 @@ EXISTING_PROXY_FILENAME = "existing-proxy.json"
PLAN_FILENAME = "plan.json" PLAN_FILENAME = "plan.json"
RECEIPT_FILENAME = "receipt.json" RECEIPT_FILENAME = "receipt.json"
CAPABILITIES_FILENAME = "infrastructure-capabilities.json" CAPABILITIES_FILENAME = "infrastructure-capabilities.json"
DEPENDENCY_INVENTORY_FILENAME = "infrastructure-dependency-inventory.json"
MANIFEST_FILENAME = "distribution-manifest.json" MANIFEST_FILENAME = "distribution-manifest.json"
KEYRING_FILENAME = "distribution-keyring.json" KEYRING_FILENAME = "distribution-keyring.json"
BACKUP_EVIDENCE_FILENAME = "backup-evidence.json" BACKUP_EVIDENCE_FILENAME = "backup-evidence.json"
@@ -116,6 +117,7 @@ class BundlePaths:
plan: Path plan: Path
receipt: Path receipt: Path
capabilities: Path capabilities: Path
dependency_inventory: Path
manifest: Path manifest: Path
keyring: Path keyring: Path
backup_evidence: Path backup_evidence: Path
@@ -141,6 +143,7 @@ def bundle_paths(root: Path) -> BundlePaths:
plan=resolved / PLAN_FILENAME, plan=resolved / PLAN_FILENAME,
receipt=resolved / RECEIPT_FILENAME, receipt=resolved / RECEIPT_FILENAME,
capabilities=resolved / CAPABILITIES_FILENAME, capabilities=resolved / CAPABILITIES_FILENAME,
dependency_inventory=resolved / DEPENDENCY_INVENTORY_FILENAME,
manifest=resolved / MANIFEST_FILENAME, manifest=resolved / MANIFEST_FILENAME,
keyring=resolved / KEYRING_FILENAME, keyring=resolved / KEYRING_FILENAME,
backup_evidence=resolved / BACKUP_EVIDENCE_FILENAME, backup_evidence=resolved / BACKUP_EVIDENCE_FILENAME,
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass
from datetime import UTC, datetime
from typing import Mapping from typing import Mapping
from urllib.parse import urlsplit from urllib.parse import urlsplit
@@ -18,6 +19,10 @@ CAPABILITY_STATES = frozenset(
"unavailable", "unavailable",
} }
) )
DEPENDENCY_INVENTORY_SCHEMA_VERSION = 1
DEPENDENCY_STATES = frozenset(
{"active", "inactive", "data_present", "pending_work", "runtime_binding"}
)
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
@@ -50,13 +55,161 @@ class CapabilityChangeImpact:
dependent_modules: tuple[str, ...] dependent_modules: tuple[str, ...]
detail: str detail: str
required_action: str required_action: str
actual_dependencies: tuple["CapabilityDependency", ...] = ()
inventory_inspected: bool = False
def to_dict(self) -> dict[str, object]: def to_dict(self) -> dict[str, object]:
value = asdict(self) value = asdict(self)
value["dependent_modules"] = list(self.dependent_modules) value["dependent_modules"] = list(self.dependent_modules)
value["actual_dependencies"] = [
item.to_dict() for item in self.actual_dependencies
]
return value return value
@dataclass(frozen=True, slots=True)
class CapabilityDependency:
capability_id: str
module_id: str
dependency_type: str
dependency_ref: str
state: str
scope: str
summary: str
metrics: Mapping[str, int]
required_action: str
def to_dict(self) -> dict[str, object]:
return {
"capability_id": self.capability_id,
"module_id": self.module_id,
"dependency_type": self.dependency_type,
"dependency_ref": self.dependency_ref,
"state": self.state,
"scope": self.scope,
"summary": self.summary,
"metrics": dict(sorted(self.metrics.items())),
"required_action": self.required_action,
}
@dataclass(frozen=True, slots=True)
class InfrastructureDependencyInventory:
installation_id: str
generated_at: datetime
complete: bool
inspected_capability_ids: tuple[str, ...]
provider_count: int
dependencies: tuple[CapabilityDependency, ...]
def dependencies_for(
self,
capability_id: str,
) -> tuple[CapabilityDependency, ...]:
return tuple(
item for item in self.dependencies if item.capability_id == capability_id
)
def infrastructure_dependency_inventory_from_mapping(
value: object,
) -> InfrastructureDependencyInventory:
if (
not isinstance(value, Mapping)
or value.get("schema_version") != DEPENDENCY_INVENTORY_SCHEMA_VERSION
):
raise ValueError("Infrastructure dependency inventory schema is unsupported.")
installation_id = _inventory_text(value, "installation_id", maximum=100)
generated_at_text = _inventory_text(value, "generated_at", maximum=100)
try:
generated_at = datetime.fromisoformat(generated_at_text.replace("Z", "+00:00"))
except ValueError as exc:
raise ValueError(
"Infrastructure dependency inventory timestamp is invalid."
) from exc
if generated_at.tzinfo is None:
raise ValueError("Infrastructure dependency inventory timestamp needs a timezone.")
generated_at = generated_at.astimezone(UTC)
complete = value.get("complete")
if type(complete) is not bool:
raise ValueError("Infrastructure dependency inventory completion state is invalid.")
inspected = _inventory_string_list(
value.get("inspected_capability_ids"),
maximum_items=100,
maximum_length=120,
)
if len(inspected) != len(set(inspected)):
raise ValueError("Infrastructure dependency inventory repeats a capability id.")
providers = value.get("providers")
if not isinstance(providers, list) or len(providers) > 100:
raise ValueError("Infrastructure dependency provider reports are invalid.")
provider_states: list[str] = []
provider_declarations: dict[str, tuple[str, ...]] = {}
provider_counts: dict[str, int] = {}
for provider in providers:
if not isinstance(provider, Mapping):
raise ValueError("Infrastructure dependency provider report is invalid.")
module_id = _inventory_text(provider, "module_id", maximum=120)
if module_id in provider_declarations:
raise ValueError("Infrastructure dependency provider is repeated.")
state = _inventory_text(provider, "state", maximum=40)
if state not in {"complete", "error"}:
raise ValueError("Infrastructure dependency provider state is invalid.")
provider_states.append(state)
count = provider.get("dependency_count")
if type(count) is not int or count < 0:
raise ValueError("Infrastructure dependency provider count is invalid.")
capability_ids = _inventory_string_list(
provider.get("capability_ids"),
maximum_items=30,
maximum_length=120,
)
if len(capability_ids) != len(set(capability_ids)):
raise ValueError("Infrastructure dependency provider capability is repeated.")
provider_declarations[module_id] = capability_ids
provider_counts[module_id] = count
if complete and any(state != "complete" for state in provider_states):
raise ValueError("Complete dependency inventory contains a failed provider.")
raw_dependencies = value.get("dependencies")
if not isinstance(raw_dependencies, list) or len(raw_dependencies) > 10_000:
raise ValueError("Infrastructure dependency records are invalid.")
dependencies = tuple(_inventory_dependency(item) for item in raw_dependencies)
if any(
capability_id not in inspected
for capability_ids in provider_declarations.values()
for capability_id in capability_ids
):
raise ValueError(
"Infrastructure dependency provider was not covered by the inspection."
)
if any(item.capability_id not in inspected for item in dependencies):
raise ValueError("Dependency record was not covered by the inventory inspection.")
identities = {
(item.capability_id, item.module_id, item.dependency_type, item.dependency_ref)
for item in dependencies
}
if len(identities) != len(dependencies):
raise ValueError("Infrastructure dependency inventory repeats a record.")
observed_counts = {module_id: 0 for module_id in provider_counts}
for dependency in dependencies:
declarations = provider_declarations.get(dependency.module_id)
if declarations is None or dependency.capability_id not in declarations:
raise ValueError(
"Infrastructure dependency is outside its provider declaration."
)
observed_counts[dependency.module_id] += 1
if observed_counts != provider_counts:
raise ValueError("Infrastructure dependency provider count does not match records.")
return InfrastructureDependencyInventory(
installation_id=installation_id,
generated_at=generated_at,
complete=complete,
inspected_capability_ids=inspected,
provider_count=len(providers),
dependencies=dependencies,
)
def infrastructure_capability_document( def infrastructure_capability_document(
spec: InstallationSpec, spec: InstallationSpec,
environment: Mapping[str, str], environment: Mapping[str, str],
@@ -89,6 +242,8 @@ def infrastructure_capability_document(
def capability_change_impacts( def capability_change_impacts(
previous_document: object, previous_document: object,
desired_document: Mapping[str, object], desired_document: Mapping[str, object],
*,
dependency_inventory: InfrastructureDependencyInventory | None = None,
) -> tuple[CapabilityChangeImpact, ...]: ) -> tuple[CapabilityChangeImpact, ...]:
previous = _capability_map(previous_document) previous = _capability_map(previous_document)
desired = _capability_map(desired_document) desired = _capability_map(desired_document)
@@ -141,6 +296,43 @@ def capability_change_impacts(
previous_secret_refs, previous_secret_refs,
desired_secret_refs, desired_secret_refs,
) )
actual_dependencies = (
dependency_inventory.dependencies_for(capability_id)
if dependency_inventory is not None
else ()
)
inventory_inspected = bool(
dependency_inventory is not None
and capability_id in dependency_inventory.inspected_capability_ids
)
if inventory_inspected and actual_dependencies:
references = ", ".join(
f"{item.module_id}:{item.dependency_ref}"
for item in actual_dependencies
)
inventory_detail = (
f" Provider inventory reports {len(actual_dependencies)} persisted "
f"dependency record(s): {references}."
)
elif inventory_inspected:
inventory_detail = (
" Provider inventory reports no persisted module-owned dependencies."
)
else:
inventory_detail = " Provider inventory did not inspect this capability."
dependency_actions = tuple(
dict.fromkeys(
item.required_action
for item in actual_dependencies
if item.required_action.strip()
)
)
required_action = (
"Review module-owned configuration and data migration or recovery "
"evidence before apply."
)
if dependency_actions:
required_action = f"{required_action} {' '.join(dependency_actions)}"
impacts.append( impacts.append(
CapabilityChangeImpact( CapabilityChangeImpact(
capability_id=capability_id, capability_id=capability_id,
@@ -153,11 +345,11 @@ def capability_change_impacts(
detail=( detail=(
f"{capability_id} changes from {previous_state}/{previous_source} " f"{capability_id} changes from {previous_state}/{previous_source} "
f"to {desired_state}/{desired_source}{binding_change}; " f"to {desired_state}/{desired_source}{binding_change}; "
f"declared consumers: {dependent_label}." f"declared consumers: {dependent_label}.{inventory_detail}"
),
required_action=(
"Review module-owned configuration and data migration or recovery evidence before apply."
), ),
required_action=required_action,
actual_dependencies=actual_dependencies,
inventory_inspected=inventory_inspected,
) )
) )
return tuple(impacts) return tuple(impacts)
@@ -487,3 +679,73 @@ def _binding_change_label(
if previous_secret_refs != desired_secret_refs: if previous_secret_refs != desired_secret_refs:
changes.append("secret-reference binding") changes.append("secret-reference binding")
return f" with changed {' and '.join(changes)}" if changes else "" return f" with changed {' and '.join(changes)}" if changes else ""
def _inventory_text(
value: Mapping[str, object],
key: str,
*,
maximum: int,
) -> str:
raw = value.get(key)
if not isinstance(raw, str):
raise ValueError(f"Infrastructure dependency inventory {key} is invalid.")
result = raw.strip()
if not result or len(result) > maximum or any(ord(char) < 32 for char in result):
raise ValueError(f"Infrastructure dependency inventory {key} is invalid.")
return result
def _inventory_string_list(
value: object,
*,
maximum_items: int,
maximum_length: int,
) -> tuple[str, ...]:
if not isinstance(value, list) or len(value) > maximum_items:
raise ValueError("Infrastructure dependency inventory list is invalid.")
items: list[str] = []
for raw in value:
if not isinstance(raw, str):
raise ValueError("Infrastructure dependency inventory list is invalid.")
item = raw.strip()
if (
not item
or len(item) > maximum_length
or any(ord(char) < 32 for char in item)
):
raise ValueError("Infrastructure dependency inventory list is invalid.")
items.append(item)
return tuple(items)
def _inventory_dependency(value: object) -> CapabilityDependency:
if not isinstance(value, Mapping):
raise ValueError("Infrastructure dependency record is invalid.")
state = _inventory_text(value, "state", maximum=40)
if state not in DEPENDENCY_STATES:
raise ValueError("Infrastructure dependency state is invalid.")
raw_metrics = value.get("metrics")
if not isinstance(raw_metrics, Mapping) or len(raw_metrics) > 20:
raise ValueError("Infrastructure dependency metrics are invalid.")
metrics: dict[str, int] = {}
for raw_key, raw_count in raw_metrics.items():
if not isinstance(raw_key, str):
raise ValueError("Infrastructure dependency metric name is invalid.")
key = raw_key.strip()
if not key or len(key) > 80 or any(ord(char) < 32 for char in key):
raise ValueError("Infrastructure dependency metric name is invalid.")
if type(raw_count) is not int or raw_count < 0:
raise ValueError("Infrastructure dependency metric value is invalid.")
metrics[key] = raw_count
return CapabilityDependency(
capability_id=_inventory_text(value, "capability_id", maximum=120),
module_id=_inventory_text(value, "module_id", maximum=120),
dependency_type=_inventory_text(value, "dependency_type", maximum=120),
dependency_ref=_inventory_text(value, "dependency_ref", maximum=240),
state=state,
scope=_inventory_text(value, "scope", maximum=120),
summary=_inventory_text(value, "summary", maximum=1000),
metrics=metrics,
required_action=_inventory_text(value, "required_action", maximum=1000),
)
+129 -2
View File
@@ -18,7 +18,8 @@ import sys
import time import time
from typing import Iterator, Mapping, Sequence from typing import Iterator, Mapping, Sequence
from urllib.error import URLError from urllib.error import URLError
from urllib.request import urlopen from urllib.parse import urlsplit
from urllib.request import Request, urlopen
from .backup_evidence import ( from .backup_evidence import (
DEFAULT_MAX_BACKUP_AGE_SECONDS, DEFAULT_MAX_BACKUP_AGE_SECONDS,
@@ -28,6 +29,7 @@ from .backup_evidence import (
) )
from .bundle import ( from .bundle import (
BACKUP_RUNTIME_ENV_KEYS, BACKUP_RUNTIME_ENV_KEYS,
BundlePaths,
atomic_write, atomic_write,
bundle_paths, bundle_paths,
canonical_json, canonical_json,
@@ -45,7 +47,11 @@ from .bundle import (
service_names, service_names,
write_env, write_env,
) )
from .capabilities import infrastructure_capability_document from .capabilities import (
InfrastructureDependencyInventory,
infrastructure_capability_document,
infrastructure_dependency_inventory_from_mapping,
)
from .cluster_evidence import collect_kubernetes_evidence from .cluster_evidence import collect_kubernetes_evidence
from .distribution import ( from .distribution import (
MAX_KEYRING_BYTES, MAX_KEYRING_BYTES,
@@ -81,6 +87,7 @@ from .kubernetes import (
write_secret_creation_hint, write_secret_creation_hint,
) )
from .planning import ( from .planning import (
MAX_DEPENDENCY_INVENTORY_BYTES,
DeploymentPlan, DeploymentPlan,
build_plan, build_plan,
release_change_requires_backup, release_change_requires_backup,
@@ -152,6 +159,39 @@ def build_parser() -> argparse.ArgumentParser:
default=120.0, default=120.0,
help="Maximum time to wait for the public health endpoint.", help="Maximum time to wait for the public health endpoint.",
) )
apply_parser.add_argument(
"--ops-url",
help=(
"Dependency inventory URL; defaults to "
"<public-url>/api/v1/ops/infrastructure/dependencies."
),
)
apply_parser.add_argument(
"--api-key-env",
default="GOVOPLAN_OPS_API_KEY",
help=(
"Environment variable containing an API key authorized to read "
"Ops dependency inventory."
),
)
collect_inventory = subparsers.add_parser(
"collect-infrastructure-inventory",
help="Collect current module-owned capability dependencies from Ops.",
)
_directory_argument(collect_inventory)
collect_inventory.add_argument(
"--ops-url",
help=(
"Dependency inventory URL; defaults to "
"<public-url>/api/v1/ops/infrastructure/dependencies."
),
)
collect_inventory.add_argument(
"--api-key-env",
default="GOVOPLAN_OPS_API_KEY",
help="Environment variable containing an authorized Ops API key.",
)
status = subparsers.add_parser( status = subparsers.add_parser(
"status", help="Show desired state and current Compose process state." "status", help="Show desired state and current Compose process state."
@@ -425,6 +465,8 @@ def main(argv: Sequence[str] | None = None) -> int:
return _render_or_doctor(args) return _render_or_doctor(args)
if args.command == "apply": if args.command == "apply":
return _apply(args) return _apply(args)
if args.command == "collect-infrastructure-inventory":
return _collect_infrastructure_inventory(args)
if args.command == "status": if args.command == "status":
return _status(args) return _status(args)
if args.command == "verify-release": if args.command == "verify-release":
@@ -586,6 +628,25 @@ def _apply(args: argparse.Namespace) -> int:
) )
secrets = reconcile_runtime_environment(spec, read_env(paths.env)) secrets = reconcile_runtime_environment(spec, read_env(paths.env))
secrets = _write_bundle(spec, paths, secrets) secrets = _write_bundle(spec, paths, secrets)
preliminary_plan = build_plan(spec, paths, include_host_checks=False)
api_key_env = str(
getattr(args, "api_key_env", "GOVOPLAN_OPS_API_KEY")
).strip()
api_key = os.environ.get(api_key_env, "").strip()
if preliminary_plan.capability_impacts and api_key:
try:
_collect_dependency_inventory(
spec,
paths,
ops_url=getattr(args, "ops_url", None),
api_key=api_key,
)
print("Refreshed infrastructure dependency inventory from Ops.")
except (OSError, ValueError, json.JSONDecodeError) as exc:
print(
f"warning: could not refresh dependency inventory: {exc}",
file=sys.stderr,
)
plan = build_plan(spec, paths, include_host_checks=True) plan = build_plan(spec, paths, include_host_checks=True)
_write_plan(paths.plan, plan) _write_plan(paths.plan, plan)
effective_errors = [ effective_errors = [
@@ -613,6 +674,8 @@ def _apply(args: argparse.Namespace) -> int:
if effective_errors: if effective_errors:
_print_plan(plan) _print_plan(plan)
raise ValueError("deployment plan is blocked; resolve doctor errors first") raise ValueError("deployment plan is blocked; resolve doctor errors first")
if plan.capability_impacts:
_print_plan(plan)
docker = shutil.which("docker") docker = shutil.which("docker")
if docker is None: if docker is None:
raise ValueError("Docker CLI is required for apply") raise ValueError("Docker CLI is required for apply")
@@ -761,6 +824,70 @@ def _apply(args: argparse.Namespace) -> int:
return 0 return 0
def _collect_infrastructure_inventory(args: argparse.Namespace) -> int:
paths = bundle_paths(args.directory)
spec = load_spec(paths.spec)
api_key_env = str(args.api_key_env).strip()
api_key = os.environ.get(api_key_env, "").strip()
if not api_key:
raise ValueError(f"{api_key_env} must contain an authorized Ops API key")
inventory = _collect_dependency_inventory(
spec,
paths,
ops_url=args.ops_url,
api_key=api_key,
)
state = "complete" if inventory.complete else "incomplete"
print(
f"Collected {state} provider dependency inventory with "
f"{len(inventory.dependencies)} record(s) at {paths.dependency_inventory}."
)
return 0 if inventory.complete else 1
def _collect_dependency_inventory(
spec: InstallationSpec,
paths: BundlePaths,
*,
ops_url: str | None,
api_key: str,
) -> InfrastructureDependencyInventory:
url = str(ops_url or "").strip() or (
spec.public_url.rstrip("/")
+ "/api/v1/ops/infrastructure/dependencies"
)
_validate_ops_inventory_url(url)
request = Request(
url,
headers={"Accept": "application/json", "X-API-Key": api_key},
)
with urlopen(request, timeout=15) as response: # noqa: S310
_validate_ops_inventory_url(response.geturl())
encoded = response.read(MAX_DEPENDENCY_INVENTORY_BYTES + 1)
if len(encoded) > MAX_DEPENDENCY_INVENTORY_BYTES:
raise ValueError("Ops dependency inventory exceeds its size limit")
value = json.loads(encoded)
inventory = infrastructure_dependency_inventory_from_mapping(value)
if inventory.installation_id != spec.installation_id:
raise ValueError(
"Ops dependency inventory belongs to a different installation"
)
ensure_private_directory(paths.root)
atomic_write(paths.dependency_inventory, canonical_json(value), mode=0o600)
return inventory
def _validate_ops_inventory_url(url: str) -> None:
parsed = urlsplit(url)
if not parsed.hostname or parsed.username or parsed.password or parsed.fragment:
raise ValueError("Ops dependency inventory URL is invalid")
loopback = parsed.hostname in {"localhost", "127.0.0.1", "::1"}
if parsed.scheme != "https" and not (parsed.scheme == "http" and loopback):
raise ValueError(
"Ops dependency inventory URL requires HTTPS except on loopback"
)
def _status(args: argparse.Namespace) -> int: def _status(args: argparse.Namespace) -> int:
paths = bundle_paths(args.directory) paths = bundle_paths(args.directory)
spec = load_spec(paths.spec) spec = load_spec(paths.spec)
+4 -1
View File
@@ -13,7 +13,10 @@ from urllib.parse import urlsplit
SCHEMA_VERSION = 1 SCHEMA_VERSION = 1
DEFAULT_GARAGE_IMAGE = "dxflrs/garage:v2.3.0" DEFAULT_GARAGE_IMAGE = "dxflrs/garage:v2.3.0"
DEFAULT_LOAD_BALANCER_IMAGE = "haproxy:3.2.21-alpine" DEFAULT_LOAD_BALANCER_IMAGE = (
"haproxy:3.2.23-alpine@sha256:"
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e"
)
DEFAULT_INGRESS_IMAGE = "caddy:2.10.2-alpine" DEFAULT_INGRESS_IMAGE = "caddy:2.10.2-alpine"
INSTALLATION_ID_PATTERN = re.compile(r"^[a-z][a-z0-9-]{1,47}$") INSTALLATION_ID_PATTERN = re.compile(r"^[a-z][a-z0-9-]{1,47}$")
ENV_NAME_PATTERN = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$") ENV_NAME_PATTERN = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$")
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass
from datetime import UTC, datetime
import hashlib import hashlib
import json import json
import os import os
@@ -37,8 +38,10 @@ from .bundle import (
) )
from .capabilities import ( from .capabilities import (
CapabilityChangeImpact, CapabilityChangeImpact,
InfrastructureDependencyInventory,
capability_change_impacts, capability_change_impacts,
infrastructure_capability_document, infrastructure_capability_document,
infrastructure_dependency_inventory_from_mapping,
) )
from .distribution import ( from .distribution import (
MAX_KEYRING_BYTES, MAX_KEYRING_BYTES,
@@ -110,6 +113,9 @@ class DeploymentPlan:
CommandRunner = Callable[[Sequence[str], Path], subprocess.CompletedProcess[str]] CommandRunner = Callable[[Sequence[str], Path], subprocess.CompletedProcess[str]]
MAX_DEPENDENCY_INVENTORY_BYTES = 2 * 1024 * 1024
DEPENDENCY_INVENTORY_MAX_AGE_SECONDS = 300
DEPENDENCY_INVENTORY_MAX_FUTURE_SECONDS = 60
def build_plan( def build_plan(
@@ -135,9 +141,13 @@ def build_plan(
spec, spec,
read_env(paths.env), read_env(paths.env),
) )
dependency_inventory, dependency_inventory_error = (
_read_dependency_inventory(paths.dependency_inventory)
)
capability_impacts = capability_change_impacts( capability_impacts = capability_change_impacts(
previous.get("infrastructure_capabilities"), previous.get("infrastructure_capabilities"),
infrastructure_capabilities, infrastructure_capabilities,
dependency_inventory=dependency_inventory,
) )
actions: list[PlanAction] = [] actions: list[PlanAction] = []
@@ -215,6 +225,14 @@ def build_plan(
) )
for impact in capability_impacts for impact in capability_impacts
) )
checks.extend(
_dependency_inventory_checks(
spec,
capability_impacts,
dependency_inventory,
dependency_inventory_error,
)
)
if include_host_checks: if include_host_checks:
checks.extend(host_checks(spec, paths, command_runner=command_runner)) checks.extend(host_checks(spec, paths, command_runner=command_runner))
return DeploymentPlan( return DeploymentPlan(
@@ -1187,6 +1205,106 @@ def _read_receipt(path: Path) -> Mapping[str, object]:
return value if isinstance(value, dict) else {} return value if isinstance(value, dict) else {}
def _read_dependency_inventory(
path: Path,
) -> tuple[InfrastructureDependencyInventory | None, str]:
if not path.exists():
return None, "missing"
try:
value = load_bounded_json(
path,
maximum_bytes=MAX_DEPENDENCY_INVENTORY_BYTES,
)
return infrastructure_dependency_inventory_from_mapping(value), ""
except (DistributionError, ValueError) as exc:
return None, str(exc)
def _dependency_inventory_checks(
spec: InstallationSpec,
impacts: tuple[CapabilityChangeImpact, ...],
inventory: InfrastructureDependencyInventory | None,
inventory_error: str,
) -> tuple[Check, ...]:
if not impacts:
return ()
collect_action = (
"Run govoplan-deploy collect-infrastructure-inventory with an Ops API "
"key, then review the capability impacts before apply."
)
if inventory is None:
if inventory_error == "missing":
message = "Current provider dependency inventory is missing."
check_id = "capability.dependency_inventory.missing"
else:
message = f"Provider dependency inventory is invalid: {inventory_error}"
check_id = "capability.dependency_inventory.invalid"
return (Check(check_id, "error", message, collect_action),)
if inventory.installation_id != spec.installation_id:
return (
Check(
"capability.dependency_inventory.installation",
"error",
"Provider dependency inventory belongs to a different installation.",
collect_action,
),
)
if not inventory.complete:
return (
Check(
"capability.dependency_inventory.incomplete",
"error",
"Provider dependency inventory is incomplete because at least one provider failed.",
"Resolve the provider failure and collect the inventory again.",
),
)
age_seconds = (datetime.now(UTC) - inventory.generated_at).total_seconds()
if age_seconds < -DEPENDENCY_INVENTORY_MAX_FUTURE_SECONDS:
return (
Check(
"capability.dependency_inventory.future",
"error",
"Provider dependency inventory timestamp is in the future.",
"Correct host clock skew and collect the inventory again.",
),
)
if age_seconds > DEPENDENCY_INVENTORY_MAX_AGE_SECONDS:
return (
Check(
"capability.dependency_inventory.stale",
"error",
"Provider dependency inventory is older than five minutes.",
collect_action,
),
)
impacted_ids = {item.capability_id for item in impacts}
missing_ids = sorted(impacted_ids - set(inventory.inspected_capability_ids))
if missing_ids:
return (
Check(
"capability.dependency_inventory.coverage",
"error",
"Provider dependency inventory did not inspect impacted capabilities: "
+ ", ".join(missing_ids)
+ ".",
collect_action,
),
)
matching_dependencies = sum(
len(inventory.dependencies_for(capability_id))
for capability_id in impacted_ids
)
return (
Check(
"capability.dependency_inventory.current",
"ok",
"Current provider inventory inspected every impacted capability and "
f"reported {matching_dependencies} persisted dependency record(s) from "
f"{inventory.provider_count} provider(s).",
),
)
def _memory_bytes() -> int | None: def _memory_bytes() -> int | None:
try: try:
for line in Path("/proc/meminfo").read_text(encoding="utf-8").splitlines(): for line in Path("/proc/meminfo").read_text(encoding="utf-8").splitlines():
@@ -34,6 +34,7 @@ _BUNDLE_FILES = (
"backup-verification.json", "backup-verification.json",
"receipt.json", "receipt.json",
"infrastructure-capabilities.json", "infrastructure-capabilities.json",
"infrastructure-dependency-inventory.json",
) )
@@ -1379,6 +1379,13 @@
"rationale": "Operational worker, scheduler, reconciliation, or health endpoint; it is not a direct user surface.", "rationale": "Operational worker, scheduler, reconciliation, or health endpoint; it is not a direct user surface.",
"repository": "govoplan-notifications" "repository": "govoplan-notifications"
}, },
{
"category": "intentionally_headless",
"method": "GET",
"path": "/ops/infrastructure/dependencies",
"rationale": "Authorized host-deployer preflight consumes this provider inventory directly; it is private operational evidence rather than a product page.",
"repository": "govoplan-ops"
},
{ {
"category": "worker_internal", "category": "worker_internal",
"method": "GET", "method": "GET",
@@ -1713,6 +1720,13 @@
"rationale": "The module WebUI constructs this endpoint through a mounted router prefix, generic action, or provider path.", "rationale": "The module WebUI constructs this endpoint through a mounted router prefix, generic action, or provider path.",
"repository": "govoplan-projects" "repository": "govoplan-projects"
}, },
{
"category": "ui_reachable",
"method": "POST",
"path": "/records/{}/access-grants/{}/revoke",
"rationale": "The restricted-record access dialog revokes a grant through the shared dynamically constructed record mutation path.",
"repository": "govoplan-records"
},
{ {
"category": "ui_reachable", "category": "ui_reachable",
"method": "POST", "method": "POST",
@@ -2000,6 +2014,20 @@
"rationale": "Published integration, interoperability, public-participant, or health endpoint.", "rationale": "Published integration, interoperability, public-participant, or health endpoint.",
"repository": "govoplan-soap" "repository": "govoplan-soap"
}, },
{
"category": "intentionally_headless",
"method": "GET",
"path": "/admin/tenant-erasure-policy",
"rationale": "Tenant-erasure policy is a consequential operator API with recent-authentication and dedicated-permission gates.",
"repository": "govoplan-tenancy"
},
{
"category": "intentionally_headless",
"method": "PATCH",
"path": "/admin/tenant-erasure-policy",
"rationale": "Tenant-erasure policy is a consequential operator API with recent-authentication and dedicated-permission gates.",
"repository": "govoplan-tenancy"
},
{ {
"category": "intentionally_headless", "category": "intentionally_headless",
"method": "GET", "method": "GET",
@@ -2007,6 +2035,48 @@
"rationale": "Tenant deletion preflight is an administrative safety API consumed before a destructive workflow.", "rationale": "Tenant deletion preflight is an administrative safety API consumed before a destructive workflow.",
"repository": "govoplan-tenancy" "repository": "govoplan-tenancy"
}, },
{
"category": "intentionally_headless",
"method": "POST",
"path": "/admin/tenants/{}/erasure-operations",
"rationale": "Tenant erasure is an audited, provider-driven operator workflow whose API exposes the complete review and recovery evidence.",
"repository": "govoplan-tenancy"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/admin/tenants/{}/erasure-operations/{}",
"rationale": "Tenant erasure is an audited, provider-driven operator workflow whose API exposes the complete review and recovery evidence.",
"repository": "govoplan-tenancy"
},
{
"category": "intentionally_headless",
"method": "POST",
"path": "/admin/tenants/{}/erasure-operations/{}/approve",
"rationale": "Tenant-erasure approval requires typed confirmation, recent authentication, and a distinct authorized account.",
"repository": "govoplan-tenancy"
},
{
"category": "intentionally_headless",
"method": "POST",
"path": "/admin/tenants/{}/erasure-operations/{}/cancel",
"rationale": "Tenant-erasure cancellation is a recovery control available only before destructive work starts.",
"repository": "govoplan-tenancy"
},
{
"category": "intentionally_headless",
"method": "POST",
"path": "/admin/tenants/{}/erasure-operations/{}/execute",
"rationale": "Tenant-erasure execution is a consequential operator API with provider checkpoints and fail-closed reconciliation.",
"repository": "govoplan-tenancy"
},
{
"category": "intentionally_headless",
"method": "POST",
"path": "/admin/tenants/{}/erasure-operations/{}/reconcile",
"rationale": "Tenant-erasure reconciliation resumes idempotent provider steps after pending or outcome-unknown effects.",
"repository": "govoplan-tenancy"
},
{ {
"category": "compatibility", "category": "compatibility",
"method": "POST", "method": "POST",
@@ -2063,6 +2133,34 @@
"rationale": "This capability-first module intentionally exposes a headless API for other modules and integrations.", "rationale": "This capability-first module intentionally exposes a headless API for other modules and integrations.",
"repository": "govoplan-workflow-engine" "repository": "govoplan-workflow-engine"
}, },
{
"category": "intentionally_headless",
"method": "GET",
"path": "/workflow/instances/summaries",
"rationale": "Workflow Engine publishes bounded, current-authorized summary discovery for module and API consumers; the existing Workflow UI retains its compatible full-history contract. See owning topic workflow.instance-history.",
"repository": "govoplan-workflow-engine"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/workflow/instances/{}/events",
"rationale": "Workflow Engine publishes current-authorized, sequence-bounded event history pages with explicit total and continuation semantics for module and API consumers. See owning topic workflow.instance-history.",
"repository": "govoplan-workflow-engine"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/workflow/instances/{}/steps",
"rationale": "Workflow Engine publishes current-authorized, sequence-bounded step history pages with explicit total and continuation semantics for module and API consumers. See owning topic workflow.instance-history.",
"repository": "govoplan-workflow-engine"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/workflow/instances/{}/summary",
"rationale": "Workflow Engine publishes a current-authorized, history-free instance summary for module and API consumers; the existing Workflow UI retains its compatible full-history detail contract. See owning topic workflow.instance-history.",
"repository": "govoplan-workflow-engine"
},
{ {
"category": "ui_reachable", "category": "ui_reachable",
"method": "POST", "method": "POST",
@@ -2189,6 +2287,27 @@
"path": "/tasks/{}", "path": "/tasks/{}",
"rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.", "rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.",
"repository": "govoplan-tasks" "repository": "govoplan-tasks"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/connectors/tabular-sources/{}/original-csv",
"rationale": "Authorized connector clients explicitly export retained original CSV after tenant, lifecycle, read-scope and source-integrity checks; ordinary catalogue responses never include source text.",
"repository": "govoplan-connectors"
},
{
"category": "intentionally_headless",
"method": "GET",
"path": "/datasources/{}/materializations/{}/original-csv",
"rationale": "Administrators explicitly export retained original CSV through an audited API; current and historical field, row and access policies must permit the entire original and source-integrity checks must pass.",
"repository": "govoplan-datasources"
},
{
"category": "intentionally_headless",
"method": "POST",
"path": "/mail/profiles/{}/pop3/imports/{}/bind-maildrop",
"rationale": "Authorized mail operators explicitly reconcile a legacy POP3 import to the current maildrop using confirmed binding, a current transport revision token and an exact retained/downloaded-byte match; the audited API never guesses historical account identity.",
"repository": "govoplan-mail"
} }
], ],
"schema_version": 1 "schema_version": 1
+125 -4
View File
@@ -63,6 +63,28 @@ const helpAttributes = new Set([
"helperText", "helperText",
"helpText" "helpText"
]); ]);
const exactHelpAttributes = new Set([
"data-help-context-id",
"helpContextId"
]);
const helpRiskAttributes = new Set([
"data-help-risk",
"helpRisk"
]);
const reviewedHelpRiskAttributes = new Set([
"data-help-risk-reviewed",
"helpRiskReviewed"
]);
const supportedHelpRisks = new Set([
"authority",
"credential",
"disclosure",
"encryption",
"external-effect",
"irreversible",
"policy",
"retention"
]);
const actionComponentPattern = /(?:Action|Button|Link)$/; const actionComponentPattern = /(?:Action|Button|Link)$/;
const contributionTypes = new Map([ const contributionTypes = new Map([
["AdminSectionsUiCapability", "admin_section"], ["AdminSectionsUiCapability", "admin_section"],
@@ -200,20 +222,43 @@ function inspectSource(repository, sourceRoot, sourcePath) {
const parentAttributes = parentFormField const parentAttributes = parentFormField
? jsxAttributes(parentFormField) ? jsxAttributes(parentFormField)
: new Map(); : new Map();
const scopedAncestorAttributes = nearestScopedHelpAttributes(node);
const label = const label =
attributes.get("label") ?? attributes.get("label") ??
attributes.get("aria-label") ?? attributes.get("aria-label") ??
parentAttributes.get("label") ?? parentAttributes.get("label") ??
null; null;
const help = firstAttribute(attributes, helpAttributes) ?? const help = firstAttribute(attributes, helpAttributes) ??
firstAttribute(parentAttributes, helpAttributes); firstAttribute(parentAttributes, helpAttributes) ??
firstAttribute(scopedAncestorAttributes, helpAttributes);
const hasHelp = hasAnyAttribute(attributes, helpAttributes) || const hasHelp = hasAnyAttribute(attributes, helpAttributes) ||
hasAnyAttribute(parentAttributes, helpAttributes); hasAnyAttribute(parentAttributes, helpAttributes) ||
hasAnyAttribute(scopedAncestorAttributes, helpAttributes);
const hasExactHelp = hasAnyAttribute(attributes, exactHelpAttributes) ||
hasAnyAttribute(parentAttributes, exactHelpAttributes) ||
hasAnyAttribute(scopedAncestorAttributes, exactHelpAttributes);
const helpContextId = firstAttribute(attributes, exactHelpAttributes) ??
firstAttribute(parentAttributes, exactHelpAttributes) ??
firstAttribute(scopedAncestorAttributes, exactHelpAttributes);
const explicitId = firstAttribute( const explicitId = firstAttribute(
attributes, attributes,
new Set(["interfaceId", "data-interface-id", "id", "name", "field"]) new Set(["interfaceId", "data-interface-id", "id", "name", "field"])
); );
const context = nearestNamedContext(node); const context = nearestNamedContext(node);
const risk = helpRiskFor({
component,
context,
file: relativeFile,
label,
explicitId,
name: attributes.get("name") ?? attributes.get("id") ?? attributes.get("field") ?? null,
explicitRisk: firstAttribute(attributes, helpRiskAttributes) ??
firstAttribute(parentAttributes, helpRiskAttributes) ??
firstAttribute(scopedAncestorAttributes, helpRiskAttributes)
});
const riskReviewed = firstAttribute(attributes, reviewedHelpRiskAttributes) ??
firstAttribute(parentAttributes, reviewedHelpRiskAttributes) ??
firstAttribute(scopedAncestorAttributes, reviewedHelpRiskAttributes);
const stableId = sourceIdentity( const stableId = sourceIdentity(
"field", "field",
node, node,
@@ -237,7 +282,14 @@ function inspectSource(repository, sourceRoot, sourcePath) {
help: help ?? null, help: help ?? null,
helpId: hasHelp ? `${stableId}.help` : null, helpId: hasHelp ? `${stableId}.help` : null,
helpDynamic: hasHelp && help === null, helpDynamic: hasHelp && help === null,
helpCandidate: !hasHelp helpCandidate: !hasHelp,
helpExact: hasExactHelp,
helpContextId,
helpContextDynamic: hasExactHelp && helpContextId === null,
helpRisk: risk.value,
helpRiskSource: risk.source,
helpRiskReviewed: riskReviewed,
highRiskHelpMissing: risk.value !== null && !hasExactHelp && riskReviewed !== "standard"
}); });
} }
@@ -261,6 +313,19 @@ function inspectSource(repository, sourceRoot, sourcePath) {
new Set(["interfaceId", "data-interface-id", "id", "name"]) new Set(["interfaceId", "data-interface-id", "id", "name"])
); );
const context = nearestNamedContext(node); const context = nearestNamedContext(node);
const hasHelp = hasAnyAttribute(attributes, helpAttributes);
const hasExactHelp = hasAnyAttribute(attributes, exactHelpAttributes);
const helpContextId = firstAttribute(attributes, exactHelpAttributes);
const risk = helpRiskFor({
component,
context,
file: relativeFile,
label,
explicitId,
name: attributes.get("name") ?? attributes.get("id") ?? null,
explicitRisk: firstAttribute(attributes, helpRiskAttributes)
});
const riskReviewed = firstAttribute(attributes, reviewedHelpRiskAttributes);
result.actions.push({ result.actions.push({
...locate(node), ...locate(node),
id: sourceIdentity( id: sourceIdentity(
@@ -273,7 +338,15 @@ function inspectSource(repository, sourceRoot, sourcePath) {
idSource: explicitId === null ? "source_anchor" : "explicit", idSource: explicitId === null ? "source_anchor" : "explicit",
context, context,
component, component,
label label,
helpExact: hasExactHelp,
helpContextId,
helpContextDynamic: hasExactHelp && helpContextId === null,
helpDynamic: hasHelp && firstAttribute(attributes, helpAttributes) === null,
helpRisk: risk.value,
helpRiskSource: risk.source,
helpRiskReviewed: riskReviewed,
highRiskHelpMissing: risk.value !== null && !hasExactHelp && riskReviewed !== "standard"
}); });
} }
@@ -354,6 +427,26 @@ function inspectSource(repository, sourceRoot, sourcePath) {
return null; return null;
} }
function nearestScopedHelpAttributes(node) {
let current = node.parent;
while (current) {
if (ts.isJsxElement(current)) {
const attributes = jsxAttributes(current.openingElement);
if (attributes.get("data-help-scope") === "field") return attributes;
}
if (
ts.isFunctionDeclaration(current) ||
ts.isMethodDeclaration(current) ||
ts.isArrowFunction(current) ||
ts.isFunctionExpression(current)
) {
return new Map();
}
current = current.parent;
}
return new Map();
}
function jsxAttributes(node) { function jsxAttributes(node) {
const mapped = new Map(); const mapped = new Map();
for (const attribute of node.attributes.properties) { for (const attribute of node.attributes.properties) {
@@ -579,6 +672,34 @@ function hasAnyAttribute(attributes, names) {
return false; return false;
} }
function helpRiskFor({ component, context, file, label, explicitId, name, explicitRisk }) {
if (typeof explicitRisk === "string") {
return supportedHelpRisks.has(explicitRisk)
? { value: explicitRisk, source: "explicit" }
: { value: null, source: "invalid_explicit" };
}
const value = [component, context, file, label, explicitId, name]
.filter((item) => typeof item === "string")
.join(" ")
.toLowerCase()
.replace(/^i18n:/g, "")
.replace(/[._-]+/g, " ");
const patterns = [
["irreversible", /\b(delete|destroy|erase|purge|dispose|disposition|revoke|withdraw|shred)\b/],
["credential", /\b(credential|password|secret|token|api key|private key)\b/],
["retention", /\b(retention|legal hold|archive lifecycle)\b/],
["encryption", /\b(encrypt|encryption|decrypt|decryption|signing key|signature key)\b/],
["disclosure", /\b(disclose|disclosure|publish|share externally|public export)\b/],
["external-effect", /\b(send|deliver|transfer|refund|payment execution|webhook execution)\b/],
["authority", /\b(grant permission|role assignment|approve|reject|formal decision|mandate)\b/],
["policy", /\b(policy apply|policy override|enforcement mode)\b/]
];
for (const [risk, pattern] of patterns) {
if (pattern.test(value)) return { value: risk, source: "inferred" };
}
return { value: null, source: null };
}
function slug(value) { function slug(value) {
const normalized = value const normalized = value
.toLowerCase() .toLowerCase()
@@ -0,0 +1,5 @@
{
"schema_version": 1,
"maximum_missing_exact_help": 0,
"rationale": "The source-derived high-risk queue for Core issue #284 is fully resolved. Strict declarations reject any new high-risk control without an exact, manifest-declared, German-complete F1 context."
}
+186 -2
View File
@@ -31,6 +31,9 @@ ENDPOINT_SURFACE_CATEGORIES = {
DEFAULT_ENDPOINT_DECLARATIONS = ( DEFAULT_ENDPOINT_DECLARATIONS = (
META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json" META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json"
) )
DEFAULT_HIGH_RISK_HELP_BASELINE = (
META_ROOT / "tools" / "inventory" / "high-risk-help-baseline.json"
)
REQUIRED_LOCALES = ("de", "en") REQUIRED_LOCALES = ("de", "en")
REFERENCE_LOCALE = "de" REFERENCE_LOCALE = "de"
@@ -75,6 +78,12 @@ def main() -> int:
default=DEFAULT_ENDPOINT_DECLARATIONS, default=DEFAULT_ENDPOINT_DECLARATIONS,
help="Versioned endpoint-surface declaration registry.", help="Versioned endpoint-surface declaration registry.",
) )
parser.add_argument(
"--high-risk-help-baseline",
type=Path,
default=DEFAULT_HIGH_RISK_HELP_BASELINE,
help="Versioned upper bound for high-risk controls without exact F1 help.",
)
args = parser.parse_args() args = parser.parse_args()
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8")) catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
@@ -85,11 +94,15 @@ def main() -> int:
endpoint_declarations = _load_endpoint_declarations( endpoint_declarations = _load_endpoint_declarations(
args.endpoint_declarations.resolve() args.endpoint_declarations.resolve()
) )
high_risk_help_baseline = _load_high_risk_help_baseline(
args.high_risk_help_baseline.resolve()
)
inventory = _assemble_inventory( inventory = _assemble_inventory(
webui=webui, webui=webui,
backend_endpoints=backend_endpoints, backend_endpoints=backend_endpoints,
manifests=manifests, manifests=manifests,
endpoint_declarations=endpoint_declarations, endpoint_declarations=endpoint_declarations,
high_risk_help_baseline=high_risk_help_baseline,
runtime_snapshot=( runtime_snapshot=(
_load_runtime_snapshot(args.runtime_snapshot.resolve()) _load_runtime_snapshot(args.runtime_snapshot.resolve())
if args.runtime_snapshot is not None if args.runtime_snapshot is not None
@@ -164,6 +177,28 @@ def _strict_failures(
f"{len(declaration_health['stale_runtime_routes'])} runtime route " f"{len(declaration_health['stale_runtime_routes'])} runtime route "
"declarations have no WebUI implementation" "declarations have no WebUI implementation"
) )
help_health = inventory.get("help_health", {})
if check_declarations and help_health.get("invalid_risk_annotations"):
failures.append(
f"{len(help_health['invalid_risk_annotations'])} controls use an "
"unsupported contextual-help risk class"
)
if check_declarations and help_health.get("baseline_regression"):
failures.append(
f"{len(help_health['missing_exact_high_risk_help'])} high-risk "
"controls lack exact F1 help; baseline permits at most "
f"{help_health['baseline_maximum_missing']}"
)
if check_declarations and help_health.get("unresolved_exact_high_risk_help"):
failures.append(
f"{len(help_health['unresolved_exact_high_risk_help'])} high-risk "
"controls reference no manifest DocumentationTopic help context"
)
if check_declarations and help_health.get("high_risk_help_without_german"):
failures.append(
f"{len(help_health['high_risk_help_without_german'])} high-risk "
"controls resolve to documentation without complete German content"
)
runtime_comparison = inventory.get("runtime_comparison") runtime_comparison = inventory.get("runtime_comparison")
if ( if (
check_declarations check_declarations
@@ -355,6 +390,29 @@ def _extract_manifests(
} }
for permission in manifest.permissions for permission in manifest.permissions
], ],
"documentation": [
{
"id": topic.id,
"help_contexts": sorted(
{
str(context)
for context in topic.metadata.get(
"help_contexts", ()
)
if isinstance(context, str) and context.strip()
}
),
"german_complete": (
isinstance(topic.translations.get("de"), dict)
and all(
isinstance(topic.translations["de"].get(field), str)
and topic.translations["de"][field].strip()
for field in ("title", "summary", "body")
)
),
}
for topic in manifest.documentation
],
"architecture": ( "architecture": (
manifest.architecture.to_dict() manifest.architecture.to_dict()
if manifest.architecture is not None if manifest.architecture is not None
@@ -400,6 +458,7 @@ def _assemble_inventory(
backend_endpoints: list[dict[str, Any]], backend_endpoints: list[dict[str, Any]],
manifests: list[dict[str, Any]], manifests: list[dict[str, Any]],
endpoint_declarations: dict[tuple[str, str, str], dict[str, Any]], endpoint_declarations: dict[tuple[str, str, str], dict[str, Any]],
high_risk_help_baseline: dict[str, Any] | None = None,
runtime_snapshot: dict[str, Any] | None = None, runtime_snapshot: dict[str, Any] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
frontend_refs = webui["frontendApiReferences"] frontend_refs = webui["frontendApiReferences"]
@@ -471,8 +530,47 @@ def _assemble_inventory(
if any(key not in catalog_keys.get(locale, set()) for locale in expected_locales) if any(key not in catalog_keys.get(locale, set()) for locale in expected_locales)
] ]
fields = webui["fields"] fields = webui["fields"]
actions = webui.get("actions", [])
help_candidates = [field for field in fields if field["helpCandidate"]] help_candidates = [field for field in fields if field["helpCandidate"]]
dynamic_help = [field for field in fields if field.get("helpDynamic")] dynamic_help = [field for field in fields if field.get("helpDynamic")]
controls = [*fields, *actions]
high_risk_controls = [item for item in controls if item.get("helpRisk")]
missing_exact_high_risk_help = [
item for item in controls if item.get("highRiskHelpMissing")
]
invalid_risk_annotations = [
item
for item in controls
if item.get("helpRiskSource") == "invalid_explicit"
]
documentation_contexts = {
context: {
"module_id": manifest["id"],
"topic_id": topic["id"],
"german_complete": topic["german_complete"],
}
for manifest in manifests
for topic in manifest.get("documentation", [])
for context in topic.get("help_contexts", [])
}
unresolved_exact_high_risk_help = [
item
for item in high_risk_controls
if item.get("helpExact")
and not item.get("helpContextDynamic")
and item.get("helpContextId") not in documentation_contexts
]
high_risk_help_without_german = [
item
for item in high_risk_controls
if item.get("helpContextId") in documentation_contexts
and not documentation_contexts[item["helpContextId"]]["german_complete"]
]
baseline_maximum_missing = (
high_risk_help_baseline["maximum_missing_exact_help"]
if high_risk_help_baseline is not None
else None
)
governance_adoption = Counter( governance_adoption = Counter(
dimension["adoption"] dimension["adoption"]
for manifest in manifests for manifest in manifests
@@ -499,10 +597,34 @@ def _assemble_inventory(
"modules": manifests, "modules": manifests,
"interface_declarations": source_declarations, "interface_declarations": source_declarations,
"declaration_health": declaration_health, "declaration_health": declaration_health,
"help_health": {
"supported_risk_classes": sorted(
{
str(item["helpRisk"])
for item in high_risk_controls
if item.get("helpRisk")
}
),
"high_risk_controls": high_risk_controls,
"missing_exact_high_risk_help": missing_exact_high_risk_help,
"invalid_risk_annotations": invalid_risk_annotations,
"unresolved_exact_high_risk_help": unresolved_exact_high_risk_help,
"high_risk_help_without_german": high_risk_help_without_german,
"dynamic_owner_context_controls": [
item
for item in high_risk_controls
if item.get("helpContextDynamic")
],
"baseline_maximum_missing": baseline_maximum_missing,
"baseline_regression": (
baseline_maximum_missing is not None
and len(missing_exact_high_risk_help) > baseline_maximum_missing
),
},
"runtime_comparison": runtime_comparison, "runtime_comparison": runtime_comparison,
"ui": { "ui": {
"fields": fields, "fields": fields,
"actions": webui.get("actions", []), "actions": actions,
"labels": webui["labels"], "labels": webui["labels"],
"visible_text": webui["visibleText"], "visible_text": webui["visibleText"],
"routes": webui["routes"], "routes": webui["routes"],
@@ -554,7 +676,19 @@ def _assemble_inventory(
"ui_fields_with_resolvable_f1_context": len(fields), "ui_fields_with_resolvable_f1_context": len(fields),
"help_review_candidates": len(help_candidates), "help_review_candidates": len(help_candidates),
"dynamic_help_references": len(dynamic_help), "dynamic_help_references": len(dynamic_help),
"ui_actions": len(webui.get("actions", [])), "ui_actions": len(actions),
"high_risk_controls": len(high_risk_controls),
"high_risk_controls_with_exact_help": (
len(high_risk_controls) - len(missing_exact_high_risk_help)
),
"high_risk_controls_missing_exact_help": len(
missing_exact_high_risk_help
),
"invalid_help_risk_annotations": len(invalid_risk_annotations),
"unresolved_exact_high_risk_help": len(
unresolved_exact_high_risk_help
),
"high_risk_help_without_german": len(high_risk_help_without_german),
"interface_declarations": len(source_declarations), "interface_declarations": len(source_declarations),
"duplicate_interface_ids": len(declaration_health["duplicate_ids"]), "duplicate_interface_ids": len(declaration_health["duplicate_ids"]),
"undeclared_source_surfaces": len( "undeclared_source_surfaces": len(
@@ -885,6 +1019,10 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
for item in inventory["api"]["unreferenced_by_static_webui_scan"] for item in inventory["api"]["unreferenced_by_static_webui_scan"]
) )
classification_counts = inventory["api"]["classification_counts"] classification_counts = inventory["api"]["classification_counts"]
high_risk_by_repository = Counter(
item["repository"]
for item in inventory["help_health"]["missing_exact_high_risk_help"]
)
lines = [ lines = [
"# GovOPlaN Platform Interface Inventory", "# GovOPlaN Platform Interface Inventory",
"", "",
@@ -900,6 +1038,12 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
f"- Fields with a resolvable F1 context: {summary['ui_fields_with_resolvable_f1_context']}", f"- Fields with a resolvable F1 context: {summary['ui_fields_with_resolvable_f1_context']}",
f"- Fields with dynamic help references: {summary['dynamic_help_references']}", f"- Fields with dynamic help references: {summary['dynamic_help_references']}",
f"- Help review candidates: {summary['help_review_candidates']}", f"- Help review candidates: {summary['help_review_candidates']}",
f"- High-risk controls: {summary['high_risk_controls']}",
f"- High-risk controls with exact F1 help: {summary['high_risk_controls_with_exact_help']}",
f"- High-risk controls missing exact F1 help: {summary['high_risk_controls_missing_exact_help']}",
f"- Invalid help-risk annotations: {summary['invalid_help_risk_annotations']}",
f"- High-risk exact contexts missing a manifest topic: {summary['unresolved_exact_high_risk_help']}",
f"- High-risk contexts without complete German topic content: {summary['high_risk_help_without_german']}",
f"- Stable interface declarations: {summary['interface_declarations']}", f"- Stable interface declarations: {summary['interface_declarations']}",
f"- Duplicate interface IDs: {summary['duplicate_interface_ids']}", f"- Duplicate interface IDs: {summary['duplicate_interface_ids']}",
f"- WebUI surfaces missing runtime declarations: {summary['undeclared_source_surfaces']}", f"- WebUI surfaces missing runtime declarations: {summary['undeclared_source_surfaces']}",
@@ -930,6 +1074,23 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
f"| `{repository}` | {count} |" f"| `{repository}` | {count} |"
for repository, count in sorted(help_by_repository.items()) for repository, count in sorted(help_by_repository.items())
) )
lines.extend(
[
"",
"## High-risk Contextual-help Debt",
"",
"Inferred or explicitly classified high-risk controls require an exact",
"F1 context. `data-help-risk-reviewed=\"standard\"` records a reviewed",
"false positive. The versioned baseline makes this queue non-regressing.",
"",
"| Repository | Missing exact contexts |",
"| --- | ---: |",
]
)
lines.extend(
f"| `{repository}` | {count} |"
for repository, count in sorted(high_risk_by_repository.items())
)
lines.extend( lines.extend(
[ [
"", "",
@@ -1005,6 +1166,29 @@ def endpoint_key(endpoint: dict[str, Any]) -> tuple[str, str, str]:
) )
def _load_high_risk_help_baseline(path: Path) -> dict[str, Any]:
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise ValueError(
f"High-risk contextual-help baseline does not exist: {path}"
) from exc
except json.JSONDecodeError as exc:
raise ValueError(
f"High-risk contextual-help baseline is invalid JSON: {exc}"
) from exc
if not isinstance(payload, dict) or payload.get("schema_version") != 1:
raise ValueError(
"High-risk contextual-help baseline must use schema_version 1."
)
maximum = payload.get("maximum_missing_exact_help")
if not isinstance(maximum, int) or isinstance(maximum, bool) or maximum < 0:
raise ValueError(
"High-risk contextual-help baseline maximum must be a non-negative integer."
)
return payload
def _load_endpoint_declarations( def _load_endpoint_declarations(
path: Path, path: Path,
) -> dict[tuple[str, str, str], dict[str, Any]]: ) -> dict[tuple[str, str, str], dict[str, Any]]:
@@ -9,6 +9,7 @@ import json
from pathlib import Path from pathlib import Path
import re import re
import subprocess import subprocess
import sys
import tomllib import tomllib
@@ -216,10 +217,17 @@ def _extras(value: str | None) -> list[str]:
def _git(repository: Path, *arguments: str) -> str: def _git(repository: Path, *arguments: str) -> str:
release_root = str(Path(__file__).resolve().parent)
if release_root not in sys.path:
sys.path.insert(0, release_root)
from govoplan_release.git_state import sanitized_git_environment, scoped_git_command
return subprocess.check_output( return subprocess.check_output(
["git", "-C", str(repository), *arguments], scoped_git_command(repository, "-C", str(repository), *arguments),
text=True, text=True,
stderr=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
env=sanitized_git_environment(),
timeout=30,
).strip() ).strip()
@@ -14,6 +14,8 @@ import re
import stat import stat
import zipfile import zipfile
from .registry_reference import registry_artifact_conflicts, registry_entry_source
WHEEL_PAYLOAD_ALGORITHM = "govoplan-wheel-declared-payload-v1" WHEEL_PAYLOAD_ALGORITHM = "govoplan-wheel-declared-payload-v1"
INSTALLED_PAYLOAD_ALGORITHM = "govoplan-installed-record-payload-v1" INSTALLED_PAYLOAD_ALGORITHM = "govoplan-installed-record-payload-v1"
@@ -185,10 +187,23 @@ def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
modules = payload.get("modules") modules = payload.get("modules")
if isinstance(modules, list): if isinstance(modules, list):
entries.extend(modules) entries.extend(modules)
conflicts = registry_artifact_conflicts(entries)
if conflicts:
return conflicts
package_by_repo: dict[str, tuple[str, str]] = {} package_by_repo: dict[str, tuple[str, str]] = {}
registry_artifacts: dict[str, dict[str, object]] = {}
for entry in entries: for entry in entries:
if not isinstance(entry, dict): if not isinstance(entry, dict):
continue continue
try:
registry_source = registry_entry_source(entry)
except ValueError as exc:
return (str(exc),)
if registry_source is not None:
package = str(entry["python_package"])
package_by_repo[registry_source.repository] = (package, registry_source.version)
registry_artifacts[package] = entry["artifact_integrity"]["python"]
continue
python_ref = entry.get("python_ref") python_ref = entry.get("python_ref")
match = _PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None match = _PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
package_name = entry.get("python_package") package_name = entry.get("python_package")
@@ -219,6 +234,14 @@ def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
issues: list[str] = [] issues: list[str] = []
if malformed_artifacts: if malformed_artifacts:
issues.append("release.artifacts contains malformed or duplicate identities") issues.append("release.artifacts contains malformed or duplicate identities")
for package, registry_artifact in registry_artifacts.items():
artifact = artifacts_by_package.get(package)
if artifact is None or (
artifact.get("archive_sha256") != registry_artifact.get("sha256")
or artifact.get("archive_size") != registry_artifact.get("size")
or artifact.get("package_version") != registry_artifact["registry_identity"].rsplit("@", 1)[-1]
):
issues.append(f"registry artifact {package} has no matching inspected wheel byte identity")
seen_repos: set[str] = set() seen_repos: set[str] = set()
for unit in selected_units: for unit in selected_units:
if not isinstance(unit, dict): if not isinstance(unit, dict):
@@ -19,6 +19,7 @@ from typing import Iterator
from govoplan_core.core.modules import ModuleManifest from govoplan_core.core.modules import ModuleManifest
from govoplan_core.core.versioning import version_satisfies_range from govoplan_core.core.versioning import version_satisfies_range
from .git_state import sanitized_git_environment, scoped_git_command
from .workspace import load_repository_specs, resolve_repo_path from .workspace import load_repository_specs, resolve_repo_path
@@ -255,20 +256,19 @@ def materialized_source_tree(root: Path, *, source_ref: str | None) -> Iterator[
source_root = temporary / "source" source_root = temporary / "source"
source_root.mkdir() source_root.mkdir()
result = subprocess.run( result = subprocess.run(
[ scoped_git_command(
"git", root, "-C", str(root),
"-C",
str(root),
"archive", "archive",
"--format=tar", "--format=tar",
f"--output={archive_path}", f"--output={archive_path}",
source_ref, source_ref,
], ),
check=False, check=False,
stdout=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, text=True,
timeout=30, timeout=30,
env=sanitized_git_environment(),
) )
if result.returncode != 0: if result.returncode != 0:
detail = result.stderr.strip() or "Git archive failed" detail = result.stderr.strip() or "Git archive failed"
@@ -0,0 +1,389 @@
"""Build a private full-profile candidate from exact verified registry bytes."""
from __future__ import annotations
import base64
from datetime import UTC, datetime, timedelta
from functools import lru_cache
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import runpy
import stat
import tarfile
from typing import Any
from urllib.parse import quote
from .artifact_identity import inspect_python_wheel, selected_artifact_identity_issues
from .candidate_artifact import (
ensure_private_candidate_root, harden_private_candidate_tree,
validate_release_channel,
)
from .catalog import canonical_hash
from .module_directory import write_module_directory
from .selective_catalog import (
authenticate_base_catalog_signatures, authenticate_base_keyring,
configured_signer_public_keys, enforce_selected_version_alignment,
next_sequence, parse_signing_key, read_bounded_json_source, signature,
validate_catalog_object,
)
from .source_provenance import (
registered_source_origin_issues, selected_source_provenance,
source_tag_provenance_issues,
)
from .version_alignment import candidate_catalog_version_issues
from .workspace import META_ROOT, resolve_workspace_root, website_root
MAX_ARTIFACT_BYTES = 512 * 1024 * 1024
@lru_cache(maxsize=3)
def _tool(name: str) -> dict[str, Any]:
# These are fixed, operator-controlled release modules, not caller paths.
if name not in {
"generate-release-package-set", "generate-release-catalog",
"resolve-package-artifacts",
}:
raise ValueError("unknown registry release tool")
return runpy.run_path(str(META_ROOT / "tools" / "release" / f"{name}.py"))
def authenticate_full_rebuild_base(
*, web_root: Path, channel: str, signer_public_keys: dict[str, str],
) -> tuple[dict[str, Any], dict[str, Any]]:
"""Authenticate the fixed website pair without reusing legacy entry data.
Only this full rebuild may migrate a legacy catalog without a keyring hash.
In that case *every* active website key must exactly equal a configured
signer; injecting an additional website key cannot extend trust. Selective
candidates retain their stricter existing hash-pinned-base requirement.
"""
root = web_root / "public" / "catalogs" / "v1"
catalog = read_bounded_json_source(root / "channels" / f"{channel}.json", label="published base catalog")
keyring = read_bounded_json_source(root / "keyring.json", label="published website keyring")
if not isinstance(catalog, dict) or not isinstance(keyring, dict):
raise ValueError("published website catalog/keyring must be objects")
trusted_keys = authenticate_base_keyring(keyring)
release = catalog.get("release")
pinned = release.get("keyring_sha256") if isinstance(release, dict) else None
if pinned is None:
if trusted_keys != signer_public_keys or len(keyring["keys"]) != len(trusted_keys):
raise ValueError("legacy full rebuild requires exactly the configured known website signers")
elif pinned != canonical_hash(keyring):
raise ValueError("published base catalog does not pin its exact website keyring")
if any(trusted_keys.get(key) != value for key, value in signer_public_keys.items()):
raise ValueError("full rebuild cannot introduce or replace a website signer")
authenticate_base_catalog_signatures(
catalog, base_trusted_keys=trusted_keys, configured_signers=signer_public_keys,
)
validation = validate_catalog_object(
catalog, approved_channel=channel, signer_public_keys=signer_public_keys,
)
if validation.get("valid") is not True:
raise ValueError(f"published base catalog failed validation: {validation.get('error')}")
return catalog, keyring
def build_full_registry_candidate(
*, package_set_path: Path, package_lock_path: Path,
wheelhouse: Path, webui_packages: Path, output_dir: Path,
selected_repositories: tuple[str, ...], signing_keys: tuple[str, ...],
workspace_root: Path | str | None = None, channel: str = "stable",
source_remote: str = "origin", public_base_url: str = "https://govoplan.add-ideas.de",
expires_days: int = 90, sequence: int | None = None,
) -> dict[str, object]:
channel = validate_release_channel(channel)
if not isinstance(expires_days, int) or isinstance(expires_days, bool) or not 1 <= expires_days <= 365:
raise ValueError("catalog expiry must be between 1 and 365 days")
workspace = resolve_workspace_root(workspace_root)
ensure_private_candidate_root(workspace)
output = output_dir.expanduser().absolute()
ensure_private_candidate_root(output.parent, create=True)
if output.exists() or output.is_symlink():
raise ValueError("full candidate output must not already exist")
parsed_keys = tuple(parse_signing_key(value) for value in signing_keys)
if not parsed_keys:
raise ValueError("full candidate needs a configured signing key")
signer_keys = configured_signer_public_keys(parsed_keys)
base, keyring = authenticate_full_rebuild_base(
web_root=website_root(workspace), channel=channel, signer_public_keys=signer_keys,
)
package_set = _hashed_json(package_set_path, "package_set_sha256")
lock = _hashed_json(package_lock_path, "lock_sha256")
generator = _tool("generate-release-catalog")
version = package_set.get("release_version")
if package_set.get("profile") != "full" or not isinstance(version, str):
raise ValueError("full candidate requires the complete full-profile package set")
expected = _tool("generate-release-package-set")["generate_package_set"](
core_version=version, requirements=META_ROOT / "requirements-release.txt",
workspace=workspace, profile="full",
meta_package=META_ROOT / "packages/govoplan-meta/pyproject.toml",
)
if package_set != expected:
raise ValueError("package set differs from exact Meta full pins or immutable tag metadata")
generator["_validate_release_inputs"](package_set, lock, core_version=version)
if lock.get("registries") != package_set.get("registries"):
raise ValueError("artifact lock uses different package registries")
versions = {row["repository"]: row["version"] for row in package_set["python"]}
origin_failures = registered_source_origin_issues(
repo_versions=versions, workspace=workspace, remote=source_remote,
)
if origin_failures:
raise ValueError("Registered source origin gate failed: " + "; ".join(item.describe() for item in origin_failures))
selected = set(selected_repositories)
if not selected or len(selected) != len(selected_repositories) or not selected <= versions.keys():
raise ValueError("selected repositories must be unique members of the full package set")
selected_versions = {repo: versions[repo] for repo in sorted(selected)}
enforce_selected_version_alignment(repo_versions=selected_versions, workspace=workspace)
failures = source_tag_provenance_issues(
repo_versions=versions, workspace=workspace, remote=source_remote,
require_head_repos=selected,
)
if failures:
raise ValueError("Full source provenance gate failed: " + "; ".join(item.describe() for item in failures))
provenance = selected_source_provenance(repo_versions=versions, workspace=workspace)
wheel_identities = verify_registry_artifacts(
package_set=package_set, lock=lock, wheelhouse=wheelhouse,
webui_packages=webui_packages,
)
generated_at = datetime.now(tz=UTC)
resolved_sequence = sequence if sequence is not None else next_sequence(base, generated_at=generated_at)
if isinstance(resolved_sequence, bool) or not isinstance(resolved_sequence, int) or resolved_sequence <= int(base.get("sequence") or 0):
raise ValueError("full candidate sequence must advance the authenticated published channel")
# Fresh tagged manifests, including unchanged tagged ancestors; no legacy
# entry, registry hash, source URL or dependency contract is carried over.
candidate = generator["_catalog_payload"](
package_set=package_set, package_lock=lock, channel=channel,
sequence=resolved_sequence, generated_at=generated_at,
expires_at=generated_at + timedelta(days=expires_days), workspace=workspace,
public_base_url=public_base_url.rstrip("/"),
)
for entry in [candidate["core_release"], *candidate["modules"]]:
repo = entry["python_package"]
entry["source"] = {
"repository": repo, "tag": f"v{versions[repo]}",
"commit": provenance[repo]["commit_sha"],
"tag_object_sha": provenance[repo]["tag_object_sha"],
"repository_url": f"https://git.add-ideas.de/GovOPlaN/{repo}",
"revision_url": f"https://git.add-ideas.de/GovOPlaN/{repo}/commit/{provenance[repo]['commit_sha']}",
}
candidate["release"].update({
"selected_units": [
{"repo": repo, "version": versions[repo], "tag": f"v{versions[repo]}", **provenance[repo]}
for repo in sorted(selected)
],
"keyring_sha256": canonical_hash(keyring),
"artifacts": wheel_identities,
"base_catalog_sha256": canonical_hash(base),
})
# Recheck the exact objects used by synthesis, including unchanged source
# ancestors, before signing. No late tag movement can change this candidate.
origin_failures = registered_source_origin_issues(
repo_versions=versions, workspace=workspace, remote=source_remote,
)
if origin_failures:
raise ValueError("Registered source origin changed during synthesis: " + "; ".join(item.describe() for item in origin_failures))
failures = source_tag_provenance_issues(
repo_versions=versions, workspace=workspace, remote=source_remote,
require_head_repos=selected,
expected_commits={repo: row["commit_sha"] for repo, row in provenance.items()},
expected_tag_objects={repo: row["tag_object_sha"] for repo, row in provenance.items()},
)
if failures:
raise ValueError("Full source provenance changed during synthesis: " + "; ".join(item.describe() for item in failures))
failures = candidate_catalog_version_issues(candidate)
identity_failures = selected_artifact_identity_issues(candidate)
if failures or identity_failures:
raise ValueError("full candidate identity validation failed: " + "; ".join(
[item.message for item in failures] + list(identity_failures)
))
candidate["signatures"] = [signature(candidate, key_id=key, private_key=value) for key, value in parsed_keys]
validation = validate_catalog_object(candidate, approved_channel=channel, signer_public_keys=signer_keys)
if validation.get("valid") is not True:
raise ValueError(f"signed full candidate failed validation: {validation.get('error')}")
# Exclusive output creation preserves earlier reviewed candidates.
output.mkdir(mode=0o700)
(output / "channels").mkdir(mode=0o700)
catalog_path = output / "channels" / f"{channel}.json"
_write_private_json(catalog_path, candidate)
_write_private_json(output / "keyring.json", keyring)
write_module_directory(
catalog_payload=candidate, keyring_payload=keyring, output_root=output,
channel=channel, public_base_url=public_base_url,
)
result = {
"status": "ready", "candidate_dir": str(output), "catalog_path": str(catalog_path),
"channel": channel, "sequence": resolved_sequence,
"package_count": len(package_set["python"]), "webui_count": len(package_set["webui"]),
"selected_count": len(selected), "candidate_catalog_hash": canonical_hash(candidate),
"candidate_keyring_hash": canonical_hash(keyring), "validation_valid": True,
}
_write_private_json(output / "summary.json", result)
harden_private_candidate_tree(output)
return result
def _hashed_json(path: Path, field: str) -> dict[str, Any]:
payload = read_bounded_json_source(path, label=field)
if not isinstance(payload, dict):
raise ValueError(f"{field} input must be an object")
unsigned = dict(payload)
expected = unsigned.pop(field, None)
# Registry tools use their established ASCII-escaped canonical form.
encoded = json.dumps(unsigned, sort_keys=True, separators=(",", ":")).encode()
if expected != hashlib.sha256(encoded).hexdigest():
raise ValueError(f"{field} does not match its contents")
return payload
def verify_registry_artifacts(
*, package_set: dict[str, Any], lock: dict[str, Any],
wheelhouse: Path, webui_packages: Path,
) -> list[dict[str, object]]:
"""Compare exact registry bytes, metadata and source bindings without installs."""
identities = []
for group, root, suffix in (("python", wheelhouse, ".whl"), ("webui", webui_packages, ".tgz")):
ensure_private_candidate_root(root)
expected = {row["name"]: row for row in package_set[group]}
rows = lock[group]
if not isinstance(rows, list) or len(rows) != len(expected):
raise ValueError(f"artifact lock has duplicate/missing {group} rows")
filenames: set[str] = set()
seen: set[str] = set()
for row in rows:
selected = expected.get(row.get("name")) if isinstance(row, dict) else None
if selected is None or row["name"] in seen:
raise ValueError(f"artifact lock has unexpected/duplicate {group} identities")
seen.add(row["name"])
for key in ("name", "version", "repository", "tag", "commit"):
if row.get(key) != selected[key]:
raise ValueError("artifact lock differs from selected source identity")
filename = row.get("filename")
if not isinstance(filename, str) or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+-]{0,254}", filename) is None or not filename.endswith(suffix) or filename in filenames:
raise ValueError("artifact filename is invalid or duplicated")
filenames.add(filename)
path = root / filename
sha256, sha512, size = _hash_artifact(path)
if (sha256, size) != (row.get("sha256"), row.get("size")):
raise ValueError(f"registry artifact bytes differ from locked identity: {filename}")
if group == "python":
expected_url = _tool("resolve-package-artifacts")["_python_artifact_url"](
package_set["registries"]["python"], package=selected, filename=filename,
)
if row.get("url") != expected_url:
raise ValueError("Python artifact URL differs from the selected registry")
identity = inspect_python_wheel(path)
if (identity.package_name, identity.package_version, identity.archive_sha256, identity.archive_size) != (row["name"], row["version"], sha256, size):
raise ValueError("wheel metadata or bytes differ from the registry lock")
identities.append(identity.catalog_payload())
else:
expected_url = (
package_set["registries"]["npm"].rstrip("/") + "/"
+ quote(row["name"], safe="") + "/-/"
+ quote(row["version"], safe="") + "/"
+ quote(row["name"].split("/", 1)[1] + "-" + row["version"] + ".tgz", safe="")
)
if row.get("url") != expected_url:
raise ValueError("WebUI artifact URL differs from the selected registry")
if row.get("integrity") != "sha512-" + base64.b64encode(sha512).decode("ascii"):
raise ValueError("WebUI registry integrity differs from downloaded bytes")
_inspect_npm_metadata(
path, name=row["name"], version=row["version"],
expected_identity=(sha256, sha512, size),
)
actual = set()
for index, path in enumerate(root.iterdir()):
if index >= 1000:
raise ValueError("registry artifact directory exceeds its inspection bound")
actual.add(path.name)
if actual != filenames:
raise ValueError(f"registry directory contains unexpected or missing {group} files")
return sorted(identities, key=lambda row: str(row["package_name"]))
def _hash_artifact(path: Path) -> tuple[str, bytes, int]:
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
try:
initial = os.fstat(descriptor)
if not stat.S_ISREG(initial.st_mode) or not 0 < initial.st_size <= MAX_ARTIFACT_BYTES:
raise ValueError("registry artifact must be a bounded regular file")
digest, sri, total = hashlib.sha256(), hashlib.sha512(), 0
while chunk := os.read(descriptor, 1024 * 1024):
total += len(chunk)
if total > MAX_ARTIFACT_BYTES:
raise ValueError("registry artifact exceeds its byte bound")
digest.update(chunk)
sri.update(chunk)
final = os.fstat(descriptor)
if (initial.st_ino, initial.st_size, initial.st_mtime_ns, initial.st_ctime_ns) != (final.st_ino, total, final.st_mtime_ns, final.st_ctime_ns):
raise ValueError("registry artifact changed while being inspected")
return digest.hexdigest(), sri.digest(), total
finally:
os.close(descriptor)
def _inspect_npm_metadata(
path: Path, *, name: str, version: str,
expected_identity: tuple[str, bytes, int] | None = None,
) -> None:
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
try:
initial = os.fstat(descriptor)
if not stat.S_ISREG(initial.st_mode) or not 0 < initial.st_size <= MAX_ARTIFACT_BYTES:
raise ValueError("WebUI archive must be a bounded regular file")
digest, sri, total = hashlib.sha256(), hashlib.sha512(), 0
while chunk := os.read(descriptor, 1024 * 1024):
total += len(chunk)
if total > MAX_ARTIFACT_BYTES:
raise ValueError("WebUI archive exceeds its byte bound")
digest.update(chunk)
sri.update(chunk)
if expected_identity is not None and (digest.hexdigest(), sri.digest(), total) != expected_identity:
raise ValueError("WebUI archive changed before metadata inspection")
os.lseek(descriptor, 0, os.SEEK_SET)
with os.fdopen(os.dup(descriptor), "rb") as stream:
_inspect_npm_stream(stream, name=name, version=version)
final = os.fstat(descriptor)
if (initial.st_ino, initial.st_size, initial.st_mtime_ns, initial.st_ctime_ns) != (final.st_ino, total, final.st_mtime_ns, final.st_ctime_ns):
raise ValueError("WebUI archive changed during metadata inspection")
finally:
os.close(descriptor)
def _inspect_npm_stream(stream: Any, *, name: str, version: str) -> None:
found = False
total = 0
with tarfile.open(fileobj=stream, mode="r|gz") as archive:
for index, member in enumerate(archive):
total += member.size
parts = PurePosixPath(member.name).parts
if index >= 10000 or total > 1024 * 1024 * 1024 or member.size > 64 * 1024 * 1024:
raise ValueError("WebUI archive exceeds its inspection bound")
if not parts or parts[0] != "package" or ".." in parts or not (member.isfile() or member.isdir()):
raise ValueError("WebUI archive contains an unsafe member")
if member.name == "package/package.json":
if found or not member.isfile() or member.size > 1024 * 1024:
raise ValueError("WebUI archive has duplicate or oversized metadata")
stream = archive.extractfile(member)
if stream is None:
raise ValueError("WebUI archive metadata cannot be read")
metadata = json.loads(stream.read(1024 * 1024 + 1))
if not isinstance(metadata, dict) or (metadata.get("name"), metadata.get("version")) != (name, version):
raise ValueError("WebUI metadata differs from the registry lock")
found = True
if not found:
raise ValueError("WebUI archive metadata is missing")
def _write_private_json(path: Path, payload: object) -> None:
encoded = (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
if len(encoded) > 16 * 1024 * 1024:
raise ValueError("candidate JSON exceeds its byte bound")
descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0), 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
+42 -1
View File
@@ -11,7 +11,7 @@ import tomllib
from .contracts import parse_manifest_contract from .contracts import parse_manifest_contract
from .model import RepositorySnapshot, RepositorySpec, VersionSnapshot from .model import RepositorySnapshot, RepositorySpec, VersionSnapshot
from .workspace import resolve_repo_path from .workspace import load_repository_specs, resolve_repo_path
def collect_repository_snapshot( def collect_repository_snapshot(
@@ -98,6 +98,7 @@ def collect_repository_snapshot(
def collect_versions(path: Path) -> VersionSnapshot: def collect_versions(path: Path) -> VersionSnapshot:
return VersionSnapshot( return VersionSnapshot(
pyproject=read_pyproject_version(path), pyproject=read_pyproject_version(path),
developer_meta=read_developer_meta_version(path),
package=read_json_version(path / "package.json"), package=read_json_version(path / "package.json"),
webui_package=read_json_version(path / "webui" / "package.json"), webui_package=read_json_version(path / "webui" / "package.json"),
manifests=read_manifest_versions(path), manifests=read_manifest_versions(path),
@@ -105,6 +106,35 @@ def collect_versions(path: Path) -> VersionSnapshot:
) )
def registered_developer_meta_path(path: Path) -> Path | None:
"""Recognize only the catalog's explicit Meta support-repository identity.
This is metadata discovery, not authorization to access a remote or mutate
a checkout. Tagging applies its separate registered source trust contract.
"""
for spec in load_repository_specs(include_website=False):
if (
spec.name == "govoplan"
and spec.category == "system"
and spec.subtype == "meta"
and path.absolute() == resolve_repo_path(spec, path.parent).absolute()
):
return path / "packages" / "govoplan-meta" / "pyproject.toml"
return None
def read_developer_meta_version(path: Path) -> str | None:
package = registered_developer_meta_path(path)
if package is None or not package.is_file():
return None
with package.open("rb") as handle:
project = tomllib.load(handle).get("project")
if isinstance(project, dict) and project.get("name") == "govoplan":
version = project.get("version")
return version if isinstance(version, str) else None
return None
def read_pyproject_version(path: Path) -> str | None: def read_pyproject_version(path: Path) -> str | None:
pyproject = path / "pyproject.toml" pyproject = path / "pyproject.toml"
if not pyproject.exists(): if not pyproject.exists():
@@ -197,6 +227,13 @@ def sanitized_git_environment(
"""Keep only deliberate process/auth inputs and neutralize Git redirection.""" """Keep only deliberate process/auth inputs and neutralize Git redirection."""
environment = os.environ if source is None else source environment = os.environ if source is None else source
address_family = environment.get("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY")
if "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY" in environment and address_family not in (
"any", "inet", "inet6",
):
raise ValueError(
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6"
)
result = { result = {
key: environment[key] key: environment[key]
for key in ( for key in (
@@ -221,6 +258,10 @@ def sanitized_git_environment(
"PATH": "/usr/bin:/bin", "PATH": "/usr/bin:/bin",
} }
) )
if address_family is not None:
result["GIT_SSH_COMMAND"] += f" -o AddressFamily={address_family}"
# Preserve only an explicit, validated choice across re-sanitization.
result["GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY"] = address_family
return result return result
@@ -0,0 +1,380 @@
"""Receipt-bound, out-of-run preparation of the real developer meta-package.
This deliberately does not commit, tag, publish, or update a running release
console. The complete generated file is reviewed in a separate source checkout.
"""
from __future__ import annotations
import copy
import hashlib
import os
from pathlib import Path
import runpy
import stat
import tempfile
import tomllib
from .git_state import (
collect_repository_snapshot,
git,
git_text,
registered_developer_meta_path,
)
from .repository_tag import normalize_version, remote_tag_commit, run
from .source_provenance import registered_source_origin_issues
from .source_tag_batch import (
_OBJECT,
_owned_path,
_source_filesystem,
_tracked_worktree,
_trusted_ancestry,
)
from .workspace import META_ROOT, load_repository_specs, resolve_repo_path
PACKAGE = "packages/govoplan-meta/pyproject.toml"
MAX_INPUT_FILES = 128
MAX_INPUT_BYTES = 2 * 1024 * 1024
MAX_TOTAL_BYTES = 16 * 1024 * 1024
GENERATOR = ".operator/generate-developer-meta-package.py"
class MetaPreparationError(ValueError):
"""Preparation is blocked, or an applied file needs explicit reconciliation."""
class MetaPreparationAmbiguous(MetaPreparationError):
"""The file effect may have happened and requires explicit reconciliation."""
def preparation_command(*, workspace: Path, target_version: str) -> str:
import shlex
return " ".join(
shlex.quote(value)
for value in (
"python",
str(META_ROOT / "tools/release/prepare-developer-meta-package.py"),
"--workspace",
str(workspace),
"--target-version",
target_version,
)
)
def _read_input(path: Path) -> tuple[bytes, dict]:
def identity(value):
return (
value.st_dev,
value.st_ino,
value.st_uid,
value.st_gid,
value.st_mode,
value.st_size,
value.st_mtime_ns,
value.st_ctime_ns,
)
before = path.lstat()
if (
not stat.S_ISREG(before.st_mode)
or before.st_uid != os.geteuid()
or before.st_mode & 0o022
or not 0 < before.st_size <= MAX_INPUT_BYTES
):
raise MetaPreparationError(
"Preparation inputs must be owned, bounded regular files."
)
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
with os.fdopen(descriptor, "rb") as source:
opened = os.fstat(source.fileno())
if identity(opened) != identity(before):
raise MetaPreparationError("Preparation input changed before reading.")
payload = source.read(before.st_size + 1)
if (
identity(os.fstat(source.fileno())) != identity(opened)
or len(payload) != before.st_size
):
raise MetaPreparationError("Preparation input changed while reading.")
return payload, {
"sha256": hashlib.sha256(payload).hexdigest(),
"identity": [
before.st_dev,
before.st_ino,
before.st_uid,
before.st_gid,
stat.S_IMODE(before.st_mode),
before.st_size,
],
}
def _snapshot(*, repo_path: Path, target_version: str, output_dirty: bool = False):
workspace = repo_path.parent
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
if registered_developer_meta_path(repo_path) != repo_path / PACKAGE:
raise MetaPreparationError(
"Only the registered Meta nested-package identity can be prepared."
)
if repo_path.resolve() == META_ROOT.resolve():
raise MetaPreparationError(
"Prepare a separate registered source checkout, never the running operator tooling."
)
paths = [repo_path / PACKAGE, repo_path / "requirements-release.txt"]
for path in workspace.glob("govoplan-*/pyproject.toml"):
paths.append(path)
if len(paths) > MAX_INPUT_FILES:
raise MetaPreparationError(
"Developer composition exceeds its input-file bound."
)
if workspace / "govoplan-core/pyproject.toml" not in paths:
raise MetaPreparationError(
"Prepare and commit matching Core metadata before Meta preparation."
)
repositories = {"govoplan": repo_path}
for path in paths[2:]:
name = path.parent.name
if (
name not in specs
or resolve_repo_path(specs[name], workspace) != path.parent
):
raise MetaPreparationError(
"Developer composition contains an unregistered package checkout."
)
repositories[name] = path.parent
filesystems = {}
for name, path in repositories.items():
filesystems[name] = _source_filesystem(path=path, workspace=workspace)
_tracked_worktree(path)
issues = registered_source_origin_issues(
repo_versions={name: target_version for name in repositories},
workspace=workspace,
remote="origin",
)
if issues:
raise MetaPreparationError(
"Preparation source origins do not match the registered repositories."
)
sources = {}
for name, path in sorted(repositories.items()):
snapshot = collect_repository_snapshot(
specs[name],
workspace_root=workspace,
target_tag=None,
online=False,
)
dirty_allowed = (
output_dirty
and name == "govoplan"
and snapshot.dirty_entries == (f" M {PACKAGE}",)
)
if (
snapshot.errors
or not snapshot.has_head
or snapshot.branch != "main"
or snapshot.upstream != "origin/main"
or snapshot.behind
or (snapshot.dirty and not dirty_allowed)
):
raise MetaPreparationError(
"Preparation requires reviewed clean main sources tracking origin/main."
)
head = git_text(path, "rev-parse", "--verify", "HEAD")
live = run(
("git", "ls-remote", "--exit-code", "--heads", "origin", "refs/heads/main"),
cwd=path,
)
lines = live.stdout.strip().splitlines()
if live.returncode or len(lines) != 1:
raise MetaPreparationError("Could not verify live preparation source main.")
remote_main, separator, reference = lines[0].partition("\t")
if (
not _OBJECT.fullmatch(head)
or not _OBJECT.fullmatch(remote_main)
or not separator
or reference != "refs/heads/main"
or git(path, "merge-base", "--is-ancestor", remote_main, head).returncode
):
raise MetaPreparationError(
"Preparation source main diverged; fetch and review separately."
)
sources[name] = {
"head": head,
"remote_main": remote_main,
"filesystem": filesystems[name],
}
tag = f"v{target_version}"
published = remote_tag_commit(repo_path, remote="origin", tag=tag)
if (
published.error
or published.tag_object
or git_text(repo_path, "rev-parse", "--verify", f"refs/tags/{tag}")
):
raise MetaPreparationError(
"An existing or unverifiable target Meta tag blocks source preparation."
)
inputs, payloads = {}, {}
total = 0
for path in sorted(paths):
payload, identity = _read_input(path)
total += len(payload)
if total > MAX_TOTAL_BYTES:
raise MetaPreparationError(
"Developer composition exceeds its aggregate input bound."
)
relative = path.relative_to(workspace).as_posix()
inputs[relative], payloads[relative] = identity, payload
generator_path = META_ROOT / "tools/release/generate-developer-meta-package.py"
_trusted_ancestry(generator_path.parent)
_owned_path(META_ROOT, directory=True)
generator, generator_identity = _read_input(generator_path)
if total + len(generator) > MAX_TOTAL_BYTES:
raise MetaPreparationError(
"Developer composition exceeds its aggregate input bound."
)
payloads[GENERATOR] = generator
current = tomllib.loads(payloads[f"govoplan/{PACKAGE}"].decode("utf-8")).get(
"project", {}
)
core = tomllib.loads(payloads["govoplan-core/pyproject.toml"].decode("utf-8")).get(
"project", {}
)
if (
not isinstance(current, dict)
or current.get("name") != "govoplan"
or not isinstance(current.get("version"), str)
):
raise MetaPreparationError(
"Nested developer-package identity and version must be exact."
)
if (
not isinstance(core, dict)
or core.get("name") != "govoplan-core"
or core.get("version") != target_version
):
raise MetaPreparationError(
"Prepare and commit Core at the requested target version before Meta preparation."
)
from .version_alignment import repository_version_issues
if repository_version_issues(
workspace / "govoplan-core", expected_version=target_version
):
raise MetaPreparationError(
"Prepare aligned Core version metadata before Meta preparation."
)
receipt = {
"kind": "developer_meta_preparation_v1",
"workspace": str(workspace),
"target_version": target_version,
"sources": sources,
"inputs": inputs,
"operator_generator": generator_identity,
}
return receipt, payloads
def _render(payloads: dict[str, bytes]) -> bytes:
# The trusted operator generator sees only the frozen bounded data snapshot.
with tempfile.TemporaryDirectory(prefix="govoplan-meta-render-") as temporary:
workspace = Path(temporary)
for relative, payload in payloads.items():
path = workspace / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(payload)
render = runpy.run_path(str(workspace / GENERATOR))["render"]
result = render(
workspace=workspace,
requirements=workspace / "govoplan/requirements-release.txt",
).encode("utf-8")
if len(result) > MAX_INPUT_BYTES:
raise MetaPreparationError(
"Generated developer package exceeds its output bound."
)
return result
def preview_meta_mutation(*, repo_path: Path, target_version: str):
version = normalize_version(target_version)
if not version:
raise MetaPreparationError("A valid target release version is required.")
receipt, payloads = _snapshot(repo_path=repo_path, target_version=version)
after = _render(payloads)
observed, _ = _snapshot(repo_path=repo_path, target_version=version)
if observed != receipt:
raise MetaPreparationError(
"Developer preparation inputs changed during preview."
)
receipt["output_sha256"] = hashlib.sha256(after).hexdigest()
return receipt, payloads[f"govoplan/{PACKAGE}"], after
def prepare_developer_meta_package(
*,
repo_path: Path,
target_version: str,
apply: bool = False,
expected_receipt=None,
confirm_out_of_run: bool = False,
) -> dict:
"""Preview or explicitly apply one full generated file; never commit/publish."""
try:
receipt, before, after = preview_meta_mutation(
repo_path=repo_path, target_version=target_version
)
result = {
"status": "planned" if before != after else "noop",
"path": PACKAGE,
"receipt": receipt,
"after_sha256": hashlib.sha256(after).hexdigest(),
"changed": before != after,
}
if not apply:
return result
if not confirm_out_of_run:
raise MetaPreparationError(
"Explicitly confirm that no durable run is active for this source workspace."
)
if receipt != expected_receipt:
raise MetaPreparationError(
"Preparation inputs changed since the reviewed preview; create a fresh preview."
)
if before == after:
return result
from .version_metadata import _atomic_write
source_receipt = {
key: value for key, value in receipt.items() if key != "output_sha256"
}
observed, _ = _snapshot(
repo_path=repo_path, target_version=receipt["target_version"]
)
if observed != source_receipt:
raise MetaPreparationError(
"Preparation inputs changed before the file effect; create a fresh preview."
)
try:
_atomic_write(repo_path / PACKAGE, after)
observed, payloads = _snapshot(
repo_path=repo_path,
target_version=receipt["target_version"],
output_dirty=True,
)
comparison = copy.deepcopy(observed)
output = f"govoplan/{PACKAGE}"
comparison["inputs"][output] = receipt["inputs"][output]
if comparison != source_receipt or payloads[output] != after:
raise MetaPreparationError("Preparation inputs changed after writing.")
except Exception as exc:
raise MetaPreparationAmbiguous(
"Generated file may have been written but its write/post-check failed; "
"review the delta and reconcile manually."
) from exc
return {**result, "status": "prepared", "after_receipt": observed}
except MetaPreparationError:
raise
except (OSError, UnicodeError, ValueError, KeyError, TypeError) as exc:
raise MetaPreparationError(
f"Developer preparation failed closed ({type(exc).__name__})."
) from exc
+2
View File
@@ -23,6 +23,7 @@ class RepositorySpec:
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
class VersionSnapshot: class VersionSnapshot:
pyproject: str | None = None pyproject: str | None = None
developer_meta: str | None = None
package: str | None = None package: str | None = None
webui_package: str | None = None webui_package: str | None = None
manifests: tuple[str, ...] = () manifests: tuple[str, ...] = ()
@@ -32,6 +33,7 @@ class VersionSnapshot:
def primary(self) -> str | None: def primary(self) -> str | None:
return ( return (
self.pyproject self.pyproject
or self.developer_meta
or self.package or self.package
or self.webui_package or self.webui_package
or (self.manifests[0] if self.manifests else None) or (self.manifests[0] if self.manifests else None)
+20 -1
View File
@@ -27,7 +27,10 @@ from .candidate_artifact import validate_release_channel
from .model import CatalogPublishResult, CatalogPublishStep from .model import CatalogPublishResult, CatalogPublishStep
from .module_directory import module_directory_payloads from .module_directory import module_directory_payloads
from .selective_catalog import read_bounded_json_source, trusted_keys_from_keyring from .selective_catalog import read_bounded_json_source, trusted_keys_from_keyring
from .source_provenance import catalog_source_selection, source_tag_provenance_issues from .source_provenance import (
catalog_source_selection, registered_source_origin_issues,
source_tag_provenance_issues,
)
from .version_alignment import candidate_catalog_version_issues from .version_alignment import candidate_catalog_version_issues
from .workspace import ( from .workspace import (
DEFAULT_WORKSPACE_ROOT, DEFAULT_WORKSPACE_ROOT,
@@ -176,6 +179,22 @@ def publish_catalog_candidate(
for issue in version_issues for issue in version_issues
) )
source_selection = catalog_source_selection(candidate_payload) source_selection = catalog_source_selection(candidate_payload)
entries = [candidate_payload.get("core_release")]
if isinstance(candidate_payload.get("modules"), list):
entries.extend(candidate_payload["modules"])
if any(
isinstance(entry, dict)
and isinstance(entry.get("python_ref"), str)
and " @ https://" in entry["python_ref"]
for entry in entries
):
blockers.extend(
f"registered source origin: {issue.describe()}"
for issue in registered_source_origin_issues(
repo_versions=source_selection.all_versions,
workspace=workspace, remote=source_remote,
)
)
blockers.extend( blockers.extend(
f"source provenance: {issue.describe()}" f"source provenance: {issue.describe()}"
for issue in source_selection.issues for issue in source_selection.issues
@@ -0,0 +1,178 @@
"""Strict source identities for immutable, registry-backed catalog entries."""
from __future__ import annotations
import base64
from dataclasses import dataclass
import json
import re
from urllib.parse import unquote, urlsplit
_SHA256 = re.compile(r"[0-9a-f]{64}\Z")
_COMMIT = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?\Z")
_REPO = re.compile(r"govoplan-[a-z0-9-]+\Z")
_VERSION = re.compile(r"\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?\Z")
_PYTHON = re.compile(
r"(?P<name>govoplan-[a-z0-9-]+)(?:\[[a-z0-9_,.-]+\])? @ "
r"(?P<url>https://\S+)#sha256=(?P<digest>[0-9a-f]{64})\Z"
)
@dataclass(frozen=True, slots=True)
class RegistrySource:
repository: str
version: str
commit: str
tag_object: str
def registry_artifact_conflicts(entries: list[object]) -> tuple[str, ...]:
"""Allow repeated module projections only when package artifacts agree."""
observed: dict[tuple[str, str], str] = {}
issues = []
for entry in entries:
if not isinstance(entry, dict):
continue
try:
source = registry_entry_source(entry)
except ValueError as exc:
issues.append(str(exc))
continue
if source is None:
continue
for kind, identity in entry["artifact_integrity"].items():
if kind not in {"python", "webui"}:
continue
key = (source.repository, kind)
encoded = json.dumps(identity, sort_keys=True, separators=(",", ":"))
if observed.setdefault(key, encoded) != encoded:
issues.append(f"conflicting {kind} registry artifacts for {source.repository}")
return tuple(issues)
def registry_entry_source(entry: dict[str, object]) -> RegistrySource | None:
"""Admit registry refs only with a complete matching artifact/source binding.
Git-backed catalogs keep their existing validation path. An HTTPS Python
requirement cannot masquerade as a source-only/non-Python entry when its
registry provenance is missing or inconsistent.
"""
ref = entry.get("python_ref")
if not isinstance(ref, str) or " @ https://" not in ref:
return None
match = _PYTHON.fullmatch(ref)
source = entry.get("source")
version = entry.get("version")
package = entry.get("python_package")
integrity = entry.get("artifact_integrity")
if (
match is None
or not isinstance(source, dict)
or not isinstance(integrity, dict)
or not isinstance(version, str)
or _VERSION.fullmatch(version) is None
or package != match.group("name")
):
raise ValueError("registry entry has no complete package/source identity")
repo = source.get("repository")
commit = source.get("commit")
tag_object = source.get("tag_object_sha")
if (
not isinstance(repo, str)
or _REPO.fullmatch(repo) is None
or repo != package
or source.get("tag") != f"v{version}"
or not isinstance(commit, str)
or _COMMIT.fullmatch(commit) is None
or not isinstance(tag_object, str)
or _COMMIT.fullmatch(tag_object) is None
):
raise ValueError("registry entry has invalid immutable tag provenance")
python = _artifact(
integrity.get("python"), ref=ref, package=package, version=version,
commit=commit,
)
if python["url"] != match.group("url") or python["sha256"] != match.group("digest"):
raise ValueError("registry Python ref differs from its artifact identity")
webui_package = entry.get("webui_package")
webui_ref = entry.get("webui_ref")
if bool(webui_package) != bool(webui_ref):
raise ValueError("registry WebUI package and ref must be declared together")
if webui_package:
if not isinstance(webui_package, str) or re.fullmatch(
r"@govoplan/[a-z0-9-]+-webui", webui_package
) is None or not isinstance(webui_ref, str):
raise ValueError("registry WebUI package identity is malformed")
if webui_package != f"@govoplan/{repo.removeprefix('govoplan-')}-webui":
raise ValueError("registry WebUI package belongs to another source repository")
webui = _artifact(
integrity.get("webui"), ref=webui_ref, package=webui_package,
version=version, commit=commit,
)
if webui_ref != webui["url"]:
raise ValueError("registry WebUI ref differs from its artifact identity")
sri = webui.get("integrity")
try:
valid_sri = isinstance(sri, str) and sri.startswith("sha512-") and len(
base64.b64decode(sri[7:], validate=True)
) == 64
except ValueError:
valid_sri = False
if not valid_sri:
raise ValueError("registry WebUI artifact needs a SHA-512 integrity identity")
elif "webui" in integrity:
raise ValueError("registry entry carries an unexpected WebUI artifact")
return RegistrySource(repo, version, commit, tag_object)
def _artifact(
value: object, *, ref: str, package: str, version: str, commit: str,
) -> dict[str, object]:
if not isinstance(value, dict):
raise ValueError("registry entry is missing artifact integrity")
url = value.get("url")
filename = value.get("filename")
digest = value.get("sha256")
size = value.get("size")
parsed = urlsplit(url) if isinstance(url, str) else None
url_filename = unquote(parsed.path.rsplit("/", 1)[-1]) if parsed else ""
decoded_parts = unquote(parsed.path).split("/") if parsed else []
expected_filenames = {url_filename}
expected_path = [package, version, url_filename]
if package.startswith("@govoplan/"):
# npm pack includes the scope in its local filename; the registry's
# immutable download URL uses the unscoped package basename.
expected_filenames = {
f"govoplan-{package.split('/', 1)[1]}-{version}.tgz",
} if url_filename == f"{package.split('/', 1)[1]}-{version}.tgz" else set()
expected_path = [*package.split("/"), "-", version, url_filename]
if (
parsed is None
or parsed.scheme != "https"
or not parsed.netloc
or parsed.username is not None
or parsed.password is not None
or parsed.fragment
or parsed.query
or any(part in {".", ".."} for part in unquote(parsed.path).split("/"))
or "%" in unquote(parsed.path)
or "\\" in unquote(parsed.path)
or any(ord(character) < 32 for character in url)
or decoded_parts[-len(expected_path):] != expected_path
or not isinstance(filename, str)
or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+-]{0,254}", filename) is None
or filename not in expected_filenames
or not isinstance(digest, str)
or _SHA256.fullmatch(digest) is None
or not isinstance(size, int)
or isinstance(size, bool)
or not 0 < size <= 512 * 1024 * 1024
or value.get("ref") != ref
or value.get("registry_identity") != f"{package}@{version}"
or value.get("git_ref") != f"v{version}"
or value.get("source_commit") != commit
):
raise ValueError("registry artifact ref, version, bytes, or source binding is inconsistent")
return value
+34 -156
View File
@@ -34,11 +34,31 @@ def tag_repositories(
apply: bool = False, # noqa: A002 - mirrors API field. apply: bool = False, # noqa: A002 - mirrors API field.
push: bool = False, push: bool = False,
) -> dict[str, object]: ) -> dict[str, object]:
"""Create annotated tags and optionally publish branch and tag atomically. from .source_tag_batch import tag_source_batch
return tag_source_batch(
repos=repos, repo_versions=repo_versions, workspace_root=workspace_root,
remote=remote, message=message, apply=apply, push=push,
)
def _preview_repositories(
*,
repos: tuple[str, ...],
repo_versions: dict[str, str],
workspace_root: Path | str | None = None,
remote: str = "origin",
message: str | None = None,
push: bool = False,
) -> dict[str, object]:
"""Read-only shared manifest/version/composition/tag preflight.
A release tag is only created for a clean, aligned, non-behind worktree. A release tag is only created for a clean, aligned, non-behind worktree.
Both local and remote tags are resolved to commits before mutation so an Both local and remote tags are resolved to commits before mutation so an
existing immutable tag can never be moved by this operation. existing immutable tag can never be moved by this operation.
Module-only local candidate tags precede Core's release-lock regeneration;
their cross-Core composition gate applies before publication, not creation.
Core candidate tags still require a complete aligned release bundle.
""" """
workspace = resolve_workspace_root(workspace_root) workspace = resolve_workspace_root(workspace_root)
@@ -47,6 +67,12 @@ def tag_repositories(
selected = tuple(dict.fromkeys(repos)) selected = tuple(dict.fromkeys(repos))
results: list[dict[str, object]] = [] results: list[dict[str, object]] = []
bundle_issues_by_repo: dict[str, list[str]] = {} bundle_issues_by_repo: dict[str, list[str]] = {}
# Core's final lock is generated from reviewed local module tags. Requiring
# that lock before those tags exist makes the documented sequence circular.
# This is only a local module staging exception: Core-selected batches and
# every publication still run the cross-repository gate, and each selected
# repository's own version/lock checks below are always enforced.
if push or "govoplan-core" in selected:
for issue in selected_release_webui_bundle_issues( for issue in selected_release_webui_bundle_issues(
repo_versions={repo: repo_versions.get(repo, "") for repo in selected}, repo_versions={repo: repo_versions.get(repo, "") for repo in selected},
workspace=workspace, workspace=workspace,
@@ -55,42 +81,7 @@ def tag_repositories(
f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})" f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
) )
if apply: if selected:
preflight = tag_repositories(
repos=selected,
repo_versions=repo_versions,
workspace_root=workspace,
remote=remote,
message=message,
apply=False,
push=push,
)
preflight_rows = preflight.get("repositories")
if isinstance(preflight_rows, list) and any(
isinstance(item, dict) and item.get("status") in {"blocked", "failed"}
for item in preflight_rows
):
blocked_rows = []
for item in preflight_rows:
if not isinstance(item, dict) or item.get("status") in {"blocked", "failed"}:
blocked_rows.append(item)
continue
blocked_rows.append(
{
**item,
"status": "skipped",
"detail": "preflight passed, but no release tag was changed because another selected repository is blocked",
}
)
return {
"status": "blocked",
"apply": True,
"push": push,
"remote": remote,
"detail": "batch preflight failed; no selected repository was mutated",
"repositories": blocked_rows,
}
elif selected:
manifest_gate_issue = manifest_shape_gate_issue(workspace) manifest_gate_issue = manifest_shape_gate_issue(workspace)
if manifest_gate_issue: if manifest_gate_issue:
return { return {
@@ -246,130 +237,17 @@ def tag_repositories(
"remote_tag_object": remote_result.tag_object, "remote_tag_object": remote_result.tag_object,
} }
) )
if not apply:
detail = preview_detail(tag=tag, local_commit=local_commit, remote_commit=remote_result.commit, push=push) detail = preview_detail(tag=tag, local_commit=local_commit, remote_commit=remote_result.commit, push=push)
status = "noop" if remote_result.commit or (local_commit and not push) else "planned" row_status = "noop" if remote_result.commit or (local_commit and not push) else "planned"
results.append({**row, "status": status, "detail": detail}) results.append({**row, "status": row_status, "detail": detail})
continue
if remote_result.commit:
if not local_commit:
fetch_result = run(("git", "fetch", remote, f"refs/tags/{tag}:refs/tags/{tag}"), cwd=path)
if fetch_result.returncode != 0:
results.append(
{
**row,
"status": "failed",
"detail": f"remote tag {tag} exists at HEAD but could not be fetched locally",
"returncode": fetch_result.returncode,
"stdout": compact_output(fetch_result.stdout),
"stderr": compact_output(fetch_result.stderr),
}
)
continue
results.append(
{
**row,
"status": "published",
"detail": f"immutable tag {tag} is already published at HEAD",
"after_local_tag_commit": head_commit,
"after_remote_tag_commit": head_commit,
}
)
continue
created = False
if not local_commit:
create_result = run(create_command, cwd=path)
if create_result.returncode != 0:
results.append(
{
**row,
"status": "failed",
"detail": f"could not create annotated tag {tag}",
"returncode": create_result.returncode,
"stdout": compact_output(create_result.stdout),
"stderr": compact_output(create_result.stderr),
}
)
continue
created = True
if not push:
results.append(
{
**row,
"status": "tagged" if created else "noop",
"detail": f"created annotated tag {tag} at HEAD" if created else f"annotated tag {tag} already exists at HEAD",
"after_local_tag_commit": head_commit,
}
)
continue
publish_result = run(publish_command, cwd=path)
if publish_result.returncode != 0:
results.append(
{
**row,
"status": "failed",
"detail": f"created local tag {tag}, but atomic branch and tag publication failed" if created else f"atomic branch and tag publication failed for {tag}",
"returncode": publish_result.returncode,
"after_local_tag_commit": head_commit,
"stdout": compact_output(publish_result.stdout),
"stderr": compact_output(publish_result.stderr),
}
)
continue
after_local_object = git_text(path, "rev-parse", "--verify", f"refs/tags/{tag}")
after_remote = remote_tag_commit(path, remote=remote, tag=tag)
if (
after_remote.error
or not after_remote.annotated
or after_remote.commit != head_commit
or after_remote.tag_object != after_local_object
):
verification_detail = after_remote.error or "remote tag did not resolve to the published annotated tag object at HEAD"
results.append(
{
**row,
"status": "failed",
"detail": f"Git push returned success, but the remote release-tag postcondition failed: {verification_detail}",
"returncode": publish_result.returncode,
"after_local_tag_commit": head_commit,
"after_local_tag_object": after_local_object,
"after_remote_tag_commit": after_remote.commit,
"after_remote_tag_object": after_remote.tag_object,
"stdout": compact_output(publish_result.stdout),
"stderr": compact_output(publish_result.stderr),
}
)
continue
results.append(
{
**row,
"status": "published",
"detail": f"published branch {snapshot.branch} and immutable tag {tag} atomically to {remote}",
"returncode": publish_result.returncode,
"after_local_tag_commit": head_commit,
"after_remote_tag_commit": head_commit,
"after_local_tag_object": after_local_object,
"after_remote_tag_object": after_remote.tag_object,
"stdout": compact_output(publish_result.stdout),
"stderr": compact_output(publish_result.stderr),
}
)
if any(item["status"] in {"blocked", "failed"} for item in results): if any(item["status"] in {"blocked", "failed"} for item in results):
status = "blocked" if not apply else "partial" result_status = "blocked"
elif any(item["status"] == "published" for item in results):
status = "published"
elif any(item["status"] == "tagged" for item in results):
status = "tagged"
elif any(item["status"] == "planned" for item in results): elif any(item["status"] == "planned" for item in results):
status = "planned" result_status = "planned"
else: else:
status = "noop" result_status = "noop"
return {"status": status, "apply": apply, "push": push, "remote": remote, "repositories": results} return {"status": result_status, "apply": False, "push": push, "remote": remote, "repositories": results}
def normalize_version(value: str | None) -> str: def normalize_version(value: str | None) -> str:
@@ -8,6 +8,7 @@ from pathlib import Path
import shlex import shlex
from .contracts import validate_contracts from .contracts import validate_contracts
from .git_state import registered_developer_meta_path, read_pyproject_version
from .model import ( from .model import (
CompatibilityIssue, CompatibilityIssue,
InterfaceProviderSnapshot, InterfaceProviderSnapshot,
@@ -111,6 +112,39 @@ def apply_repository_version_gate(
version_update_supported_by_repo: dict[str, bool] = {} version_update_supported_by_repo: dict[str, bool] = {}
deferred_core_lock_repos: set[str] = set() deferred_core_lock_repos: set[str] = set()
for unit in units: for unit in units:
if registered_developer_meta_path(workspace / unit.repo) is not None:
from .meta_preparation import preparation_command
try:
core_version = read_pyproject_version(workspace / "govoplan-core")
except (OSError, ValueError, TypeError):
core_version = None
core_ready = core_version == unit.target_version
issues_by_repo.setdefault(unit.repo, []).append(
ReleaseGateFinding(
code="developer_meta_out_of_run" if core_ready else "developer_meta_core_preparation_required",
severity="blocker",
message=(
"Meta is an out-of-run support release, not a self-updating durable executor."
if core_ready else
"Prepare and commit Core at the requested target before regenerating Meta."
),
remediation=(
"Complete Core and module preparation first. Stop active durable runs for this workspace. "
"In a separate trusted source checkout, preview "
+ preparation_command(workspace=workspace, target_version=unit.target_version)
+ ". Review its receipt; apply with --receipt <preview.json> --apply --confirm-out-of-run. "
"Review and commit the whole generated package, publish the matching Core release first, "
"then use guarded Meta source tagging/publication and create a fresh durable run."
),
repo=unit.repo, source="developer meta-package preparation",
expected=unit.target_version, actual=core_version or "missing Core version",
)
)
version_update_supported_by_repo[unit.repo] = False
# Its complete canonical composition remains a publication gate;
# this plan must not claim a generic in-run mutation/commit path.
continue
version_update_supported = unit.current_version == unit.target_version version_update_supported = unit.current_version == unit.target_version
if unit.current_version and unit.current_version != unit.target_version: if unit.current_version and unit.current_version != unit.target_version:
try: try:
@@ -437,6 +471,7 @@ def build_unit(
value value
for value in ( for value in (
repo.versions.pyproject, repo.versions.pyproject,
repo.versions.developer_meta,
repo.versions.package, repo.versions.package,
repo.versions.webui_package, repo.versions.webui_package,
*repo.versions.manifests, *repo.versions.manifests,
@@ -572,7 +607,7 @@ def repository_capabilities(
def dependency_ordered_units( def dependency_ordered_units(
units: tuple[ReleasePlanUnit, ...], units: tuple[ReleasePlanUnit, ...],
) -> tuple[ReleasePlanUnit, ...]: ) -> tuple[ReleasePlanUnit, ...]:
"""Order module providers before consumers while keeping Core last.""" """Order modules before Core, followed by the out-of-run Meta support unit."""
by_repo = {unit.repo: unit for unit in units} by_repo = {unit.repo: unit for unit in units}
providers: dict[str, set[str]] = {} providers: dict[str, set[str]] = {}
@@ -592,8 +627,10 @@ def dependency_ordered_units(
) )
if "govoplan-core" in dependencies: if "govoplan-core" in dependencies:
dependencies["govoplan-core"].update( dependencies["govoplan-core"].update(
repo for repo in by_repo if repo != "govoplan-core" repo for repo in by_repo if repo not in {"govoplan-core", "govoplan"}
) )
if "govoplan" in dependencies:
dependencies["govoplan"].update(repo for repo in by_repo if repo != "govoplan")
ordered: list[ReleasePlanUnit] = [] ordered: list[ReleasePlanUnit] = []
remaining = set(by_repo) remaining = set(by_repo)
@@ -690,6 +727,8 @@ def dry_run_steps(
*, units: tuple[ReleasePlanUnit, ...], dashboard: ReleaseDashboard, channel: str *, units: tuple[ReleasePlanUnit, ...], dashboard: ReleaseDashboard, channel: str
) -> tuple[ReleasePlanStep, ...]: ) -> tuple[ReleasePlanStep, ...]:
steps: list[ReleasePlanStep] = [] steps: list[ReleasePlanStep] = []
meta_units = tuple(unit for unit in units if unit.repo == "govoplan")
units = tuple(unit for unit in units if unit.repo != "govoplan")
snapshots = {repo.spec.name: repo for repo in dashboard.repositories} snapshots = {repo.spec.name: repo for repo in dashboard.repositories}
core_unit = next((unit for unit in units if unit.repo == "govoplan-core"), None) core_unit = next((unit for unit in units if unit.repo == "govoplan-core"), None)
non_core_units = tuple(unit for unit in units if unit.repo != "govoplan-core") non_core_units = tuple(unit for unit in units if unit.repo != "govoplan-core")
@@ -991,6 +1030,33 @@ def dry_run_steps(
status="planned", status="planned",
) )
) )
for unit in meta_units:
from .meta_preparation import preparation_command
steps.extend((
ReleasePlanStep(
id="govoplan:prepare-support",
title="Prepare the complete developer meta-package outside this run",
detail=(
"First prepare and commit Core at the target and review module/requirements inputs. "
"Stop active runs, preview and explicitly apply the frozen composition in a separate "
"source checkout; review and commit manually. No durable self-update is supported."
),
command=preparation_command(workspace=Path(dashboard.workspace_root), target_version=unit.target_version),
cwd=dashboard.meta_root, repo=unit.repo, status="needs-executor",
),
ReleasePlanStep(
id="govoplan:publish-support",
title="Publish the prepared Meta support source after Core",
detail=(
"After the matching Core annotated tag and exact main are published, use the shared "
"guarded Meta tag preview/local-tag/publish route. Commit/push reviewed preparation "
"and create a fresh durable run; do not update the current runtime binding."
),
cwd=dashboard.meta_root, repo=unit.repo, status="needs-executor",
mutating=True,
),
))
return tuple(steps) return tuple(steps)
@@ -19,10 +19,33 @@ from .git_state import (
scoped_git_command, scoped_git_command,
) )
from .repository_tag import RemoteTagResult, ref_commit, remote_tag_commit from .repository_tag import RemoteTagResult, ref_commit, remote_tag_commit
from .registry_reference import registry_entry_source
from .version_alignment import repository_version_issues from .version_alignment import repository_version_issues
from .workspace import load_repository_specs, resolve_repo_path from .workspace import load_repository_specs, resolve_repo_path
def registered_source_origin_issues(
*, repo_versions: dict[str, str], workspace: Path, remote: str,
) -> tuple[SourceTagProvenanceIssue, ...]:
"""Bind registry candidate attestations to registered source endpoints."""
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
issues = []
for repo, version in sorted(repo_versions.items()):
spec = specs.get(repo)
if spec is None:
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source repository is not registered"))
continue
path = resolve_repo_path(spec, workspace)
if path.absolute() != path.resolve() or not path.resolve().is_relative_to(workspace.resolve()):
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source checkout leaves the private workspace or traverses a symlink"))
continue
fetch = git_text(path, "remote", "get-url", "--all", remote).splitlines()
push = git_text(path, "remote", "get-url", "--push", "--all", remote).splitlines()
if fetch != [spec.remote] or push != [spec.remote]:
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source remote does not exactly match the registered origin"))
return tuple(issues)
_CATALOG_PYTHON_REF = re.compile( _CATALOG_PYTHON_REF = re.compile(
r"/(?P<repo>govoplan-[a-z0-9-]+)\.git@v(?P<version>[^\s;]+)$" r"/(?P<repo>govoplan-[a-z0-9-]+)\.git@v(?P<version>[^\s;]+)$"
) )
@@ -61,6 +84,8 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
) )
versions: dict[str, str] = {} versions: dict[str, str] = {}
registry_commits: dict[str, str] = {}
registry_tag_objects: dict[str, str] = {}
issues: list[SourceTagProvenanceIssue] = [] issues: list[SourceTagProvenanceIssue] = []
entries: list[tuple[str, object]] = [("core_release", payload.get("core_release"))] entries: list[tuple[str, object]] = [("core_release", payload.get("core_release"))]
modules = payload.get("modules") modules = payload.get("modules")
@@ -70,6 +95,21 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
for source, raw_entry in entries: for source, raw_entry in entries:
if not isinstance(raw_entry, dict): if not isinstance(raw_entry, dict):
continue continue
try:
registry_source = registry_entry_source(raw_entry)
except ValueError as exc:
issues.append(SourceTagProvenanceIssue(source, "", str(exc)))
continue
if registry_source is not None:
repo, version = registry_source.repository, registry_source.version
previous = versions.setdefault(repo, version)
previous_commit = registry_commits.setdefault(repo, registry_source.commit)
previous_object = registry_tag_objects.setdefault(repo, registry_source.tag_object)
if (previous, previous_commit, previous_object) != (
version, registry_source.commit, registry_source.tag_object,
):
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "registry entries have conflicting immutable source identities"))
continue
python_ref = raw_entry.get("python_ref") python_ref = raw_entry.get("python_ref")
match = _CATALOG_PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None match = _CATALOG_PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
if match is None: if match is None:
@@ -89,8 +129,8 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
release = payload.get("release") release = payload.get("release")
selected_units = release.get("selected_units") if isinstance(release, dict) else None selected_units = release.get("selected_units") if isinstance(release, dict) else None
selected: dict[str, str] = {} selected: dict[str, str] = {}
selected_commits: dict[str, str] = {} selected_commits: dict[str, str] = dict(registry_commits)
selected_tag_objects: dict[str, str] = {} selected_tag_objects: dict[str, str] = dict(registry_tag_objects)
if not isinstance(selected_units, list) or not selected_units: if not isinstance(selected_units, list) or not selected_units:
issues.append( issues.append(
SourceTagProvenanceIssue( SourceTagProvenanceIssue(
@@ -106,16 +146,22 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
repo = unit.get("repo") repo = unit.get("repo")
version = unit.get("version") version = unit.get("version")
if isinstance(repo, str) and isinstance(version, str): if isinstance(repo, str) and isinstance(version, str):
if repo in selected:
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "duplicate selected repository"))
selected[repo] = version.removeprefix("v") selected[repo] = version.removeprefix("v")
commit = unit.get("commit_sha") commit = unit.get("commit_sha")
tag_object = unit.get("tag_object_sha") tag_object = unit.get("tag_object_sha")
if isinstance(commit, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", commit): if isinstance(commit, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", commit):
if repo in registry_commits and registry_commits[repo] != commit.lower():
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "selected commit differs from registry artifact source"))
selected_commits[repo] = commit.lower() selected_commits[repo] = commit.lower()
else: else:
issues.append( issues.append(
SourceTagProvenanceIssue(repo, f"v{version.removeprefix('v')}", "selected unit has no valid commit_sha provenance") SourceTagProvenanceIssue(repo, f"v{version.removeprefix('v')}", "selected unit has no valid commit_sha provenance")
) )
if isinstance(tag_object, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", tag_object): if isinstance(tag_object, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", tag_object):
if repo in registry_tag_objects and registry_tag_objects[repo] != tag_object.lower():
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "selected tag object differs from registry artifact source"))
selected_tag_objects[repo] = tag_object.lower() selected_tag_objects[repo] = tag_object.lower()
else: else:
issues.append( issues.append(
@@ -0,0 +1,792 @@
"""Strict registered-source release contract for every source-tag batch.
Meta is not a root Python package. Its nested developer package is released only
after a whole-batch source preflight and the matching immutable Core release.
No selected checkout supplies the developer-package generator or release
validation tooling. The trusted shared checker may load reviewed application
manifests; this is not an untrusted-code sandbox.
"""
from __future__ import annotations
from pathlib import Path
import hashlib
import os
import re
import stat
from .git_state import collect_repository_snapshot, git, git_text
from .repository_tag import (
_preview_repositories,
basic_blocker,
normalize_version,
ref_commit,
remote_tag_commit,
run,
)
from .source_provenance import registered_source_origin_issues
from .version_alignment import (
repository_version_issues,
selected_release_webui_bundle_issues,
selected_webui_repository_names,
)
from .workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root
_OBJECT = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?\Z")
class SourceReceiptError(ValueError):
pass
def _owned_path(path, *, directory):
observed = path.lstat()
expected = stat.S_ISDIR if directory else stat.S_ISREG
if stat.S_ISLNK(observed.st_mode) or not expected(observed.st_mode):
raise SourceReceiptError(
"source authority must use real paths, without symlinks or special files"
)
if observed.st_uid != os.geteuid():
raise SourceReceiptError(
"source authority is not owned by the current operator"
)
if observed.st_mode & 0o022:
raise SourceReceiptError("source authority is group/world writable")
return observed
def _trusted_ancestry(path):
# Same ownership/mode policy as the publisher's trust-path guard. A sticky
# shared ancestor such as /tmp may contain an owned, non-writable child;
# the workspace/repository themselves are never given that exception.
for ancestor in (path, *path.parents):
observed = ancestor.lstat()
if stat.S_ISLNK(observed.st_mode) or not stat.S_ISDIR(observed.st_mode):
raise SourceReceiptError(
"source ancestry must contain real directories, not symlinks"
)
if observed.st_uid not in {0, os.geteuid()}:
raise SourceReceiptError("source ancestry has an untrusted owner")
if observed.st_mode & 0o022 and not observed.st_mode & stat.S_ISVTX:
raise SourceReceiptError("source ancestry is group/world writable")
def _git_pointer(path):
_owned_path(path, directory=False)
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
with os.fdopen(descriptor, "rb") as source:
observed = os.fstat(source.fileno())
if not stat.S_ISREG(observed.st_mode) or not 0 < observed.st_size <= 4096:
raise SourceReceiptError("Git metadata pointer is invalid or oversized")
value = source.read(observed.st_size + 1)
if len(value) != observed.st_size:
raise SourceReceiptError("Git metadata pointer changed during validation")
return value.decode("utf-8").strip()
def _git_tree(root):
pending = [(root, 0)]
count = 0
while pending:
directory, depth = pending.pop()
_owned_path(directory, directory=True)
if depth > 128:
raise SourceReceiptError(
"Git metadata exceeds its trust-validation depth limit"
)
with os.scandir(directory) as entries:
for entry in entries:
count += 1
if count > 500_000:
raise SourceReceiptError(
"Git metadata exceeds its trust-validation entry limit"
)
candidate = Path(entry.path)
observed = candidate.lstat()
if stat.S_ISDIR(observed.st_mode):
pending.append((candidate, depth + 1))
else:
_owned_path(candidate, directory=False)
def _filesystem_identity(path, observed):
return [
str(path),
observed.st_dev,
observed.st_ino,
observed.st_uid,
observed.st_gid,
stat.S_IMODE(observed.st_mode),
]
def _source_filesystem(*, path, workspace):
"""Validate source/Git ownership before invoking even read-only Git."""
if path.absolute() != path.resolve() or not path.resolve().is_relative_to(
workspace.resolve()
):
raise SourceReceiptError(
"source checkout leaves the private workspace or traverses a symlink"
)
_trusted_ancestry(path)
workspace_info = _owned_path(workspace, directory=True)
repo_info = _owned_path(path, directory=True)
marker = path / ".git"
marker_info = marker.lstat()
if stat.S_ISDIR(marker_info.st_mode):
git_dir = marker
else:
value = _git_pointer(marker)
if not value.startswith("gitdir: "):
raise SourceReceiptError("Git worktree pointer is invalid")
git_dir = Path(os.path.abspath(path / value.removeprefix("gitdir: ")))
if git_dir.absolute() != git_dir.resolve() or not git_dir.resolve().is_relative_to(
workspace.resolve()
):
raise SourceReceiptError(
"Git checkout metadata must stay inside the trusted workspace"
)
_trusted_ancestry(git_dir)
_owned_path(git_dir, directory=True)
common_dir = git_dir
common_pointer = git_dir / "commondir"
if common_pointer.exists() or common_pointer.is_symlink():
common_dir = Path(os.path.abspath(git_dir / _git_pointer(common_pointer)))
identities = {
"workspace": _filesystem_identity(workspace, workspace_info),
"checkout": _filesystem_identity(path, repo_info),
}
scanned = set()
for label, directory in (
("git_directory", git_dir),
("git_common_directory", common_dir),
):
if (
directory.absolute() != directory.resolve()
or not directory.resolve().is_relative_to(workspace.resolve())
):
raise SourceReceiptError(
"Git checkout metadata must stay inside the trusted workspace"
)
_trusted_ancestry(directory)
observed = _owned_path(directory, directory=True)
if observed.st_mode & 0o700 != 0o700:
raise SourceReceiptError(
"Git metadata target must be readable and writable by its operator"
)
identities[label] = _filesystem_identity(directory, observed)
if directory not in scanned:
_git_tree(directory)
scanned.add(directory)
identities["git_marker"] = _filesystem_identity(
marker, _owned_path(marker, directory=stat.S_ISDIR(marker_info.st_mode))
)
for candidate in (
common_dir / "objects/info/alternates",
common_dir / "objects/info/http-alternates",
common_dir / "info/grafts",
):
if candidate.exists() or candidate.is_symlink():
raise SourceReceiptError(
"Git object alternates and grafts are not permitted"
)
return identities
def _tracked_worktree(path):
tracked = git(path, "ls-files", "-v", "-z", timeout=30)
if tracked.returncode or len(tracked.stdout) > 16 * 1024 * 1024:
raise SourceReceiptError(
"tracked release inputs exceed their trust-validation limit"
)
checked = {path}
tracked_paths = set()
names = tracked.stdout.split("\0")
if len(names) > 100_001:
raise SourceReceiptError(
"tracked release inputs exceed their trust-validation count limit"
)
for entry in filter(None, names):
# git status deliberately hides assume-unchanged and skip-worktree
# paths. Never validate mutable working metadata and then tag different
# committed bytes because an index flag suppressed the dirty evidence.
if not entry.startswith("H "):
raise SourceReceiptError(
"hidden, sparse or unmerged tracked index entries are not permitted"
)
name = entry[2:]
tracked_paths.add(name)
relative = Path(name)
if relative.is_absolute() or ".." in relative.parts:
raise SourceReceiptError("tracked release input has an unsafe path")
candidate = path / relative
for parent in candidate.parents:
if parent == path:
break
if parent not in checked:
_owned_path(parent, directory=True)
checked.add(parent)
_owned_path(candidate, directory=False)
# Version/composition checks inspect existing files, including ignored
# paths. Their declarations must come from the selected committed source,
# not ignored working bytes absent from the tag's tree.
metadata = [
path / name
for name in (
"pyproject.toml",
"package.json",
"package-lock.json",
"webui/package.json",
"webui/package.release.json",
"webui/package-lock.json",
"webui/package-lock.release.json",
)
]
if path.name == "govoplan":
metadata.extend(
path / name
for name in (
"packages/govoplan-meta/pyproject.toml",
"requirements-release.txt",
)
)
metadata.extend((path / "src").glob("**/backend/manifest.py"))
metadata.extend((path / "src").glob("*/__init__.py"))
for candidate in metadata:
if (candidate.exists() or candidate.is_symlink()) and candidate.relative_to(
path
).as_posix() not in tracked_paths:
raise SourceReceiptError(
"selected release version/composition metadata must be tracked in the frozen source"
)
def _receipt(*, spec, workspace, version, filesystem):
path = resolve_repo_path(spec, workspace)
_tracked_worktree(path)
snapshot = collect_repository_snapshot(
spec, workspace_root=workspace, target_tag=f"v{version}", online=False
)
blocker = basic_blocker(snapshot=snapshot, version=version)
if blocker:
raise SourceReceiptError(blocker)
if (
not snapshot.exists
or not snapshot.is_git
or not snapshot.has_head
or snapshot.errors
or snapshot.safe_directory_required
or snapshot.dirty
or snapshot.branch != "main"
or snapshot.upstream != "origin/main"
or snapshot.behind
):
raise SourceReceiptError(
"requires a clean registered main checkout tracking origin/main, not behind"
)
common = git_text(path, "rev-parse", "--path-format=absolute", "--git-common-dir")
if (
not common
or Path(common).absolute() != Path(common).resolve()
or not Path(common).resolve().is_relative_to(workspace.resolve())
or git_text(path, "rev-parse", "--show-toplevel") != str(path.resolve())
):
raise SourceReceiptError(
"Git checkout metadata must stay inside the trusted workspace"
)
head = git_text(path, "rev-parse", "--verify", "HEAD")
if not _OBJECT.fullmatch(head):
raise SourceReceiptError("source HEAD is not an exact commit")
live = run(
("git", "ls-remote", "--exit-code", "--heads", "origin", "refs/heads/main"),
cwd=path,
)
lines = live.stdout.strip().splitlines()
if live.returncode or len(lines) != 1:
raise SourceReceiptError("could not verify live origin/main")
remote_main, separator, ref = lines[0].partition("\t")
if not separator or ref != "refs/heads/main" or not _OBJECT.fullmatch(remote_main):
raise SourceReceiptError("live origin/main returned an invalid source receipt")
if git(path, "merge-base", "--is-ancestor", remote_main, head).returncode != 0:
raise SourceReceiptError(
"live origin/main is unavailable locally or diverges; fetch and review before retrying"
)
tag = f"v{version}"
local_object = git_text(path, "rev-parse", "--verify", f"refs/tags/{tag}") or None
if local_object:
if git_text(path, "cat-file", "-t", f"refs/tags/{tag}") != "tag":
raise SourceReceiptError("local immutable tag must be annotated")
if ref_commit(path, f"refs/tags/{tag}") != head:
raise SourceReceiptError(
"local immutable tag points to another commit, not HEAD"
)
remote_tag = remote_tag_commit(path, remote="origin", tag=tag)
if remote_tag.error:
raise SourceReceiptError("could not verify the remote release tag")
if remote_tag.tag_object:
if not remote_tag.annotated:
raise SourceReceiptError("remote immutable tag must be annotated")
if remote_tag.commit != head:
raise SourceReceiptError(
"remote immutable tag points to another commit, not HEAD"
)
if local_object and remote_tag.tag_object != local_object:
raise SourceReceiptError(
"local and remote immutable tag annotation objects differ"
)
return {
"head": head,
"branch": "main",
"upstream": "origin/main",
"origin": spec.remote,
"remote_main": remote_main,
"tag": tag,
"local_tag_object": local_object,
"remote_tag_object": remote_tag.tag_object,
"filesystem": filesystem,
}
def _collect_receipts(*, versions, specs, workspace):
filesystems = {}
for repo in versions:
if repo not in specs:
raise SourceReceiptError(f"{repo}: source repository is not registered")
filesystems[repo] = _source_filesystem(
path=resolve_repo_path(specs[repo], workspace), workspace=workspace
)
issues = registered_source_origin_issues(
repo_versions=versions, workspace=workspace, remote="origin"
)
if issues:
raise SourceReceiptError(
"; ".join(f"{issue.repo}: {issue.message}" for issue in issues)
)
receipts = {}
for repo, version in versions.items():
if repo not in specs:
raise SourceReceiptError(f"{repo}: source repository is not registered")
try:
receipts[repo] = _receipt(
spec=specs[repo],
workspace=workspace,
version=version,
filesystem=filesystems[repo],
)
except SourceReceiptError as exc:
raise SourceReceiptError(f"{repo}: {exc}") from exc
return receipts
def _bundle_input_receipt(*, selected, workspace, push):
"""Freeze only Core files used by the already-applicable WebUI gate.
These are read-only composition inputs, not a new Core-tag/version or
clean-Core prerequisite. Local module candidates intentionally need none.
"""
if not push and "govoplan-core" not in selected:
return {}
if not selected_webui_repository_names(
repo_versions=dict.fromkeys(selected, ""), workspace=workspace
):
return {}
result = {}
core = workspace / "govoplan-core"
for relative in ("webui/package.release.json", "webui/package-lock.release.json"):
path = core / relative
if not path.exists() and not path.is_symlink():
result[relative] = None # The unchanged shared gate explains missing input.
continue
_trusted_ancestry(path.parent)
_owned_path(core, directory=True)
_owned_path(path.parent, directory=True)
observed = _owned_path(path, directory=False)
if not 0 < observed.st_size <= 16 * 1024 * 1024:
raise SourceReceiptError(
"Core release-bundle input exceeds its 16 MiB limit"
)
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
with os.fdopen(descriptor, "rb") as source:
before = os.fstat(source.fileno())
if (
before.st_dev,
before.st_ino,
before.st_size,
before.st_mtime_ns,
before.st_ctime_ns,
) != (
observed.st_dev,
observed.st_ino,
observed.st_size,
observed.st_mtime_ns,
observed.st_ctime_ns,
):
raise SourceReceiptError(
"Core release-bundle input changed during inspection"
)
content = source.read(before.st_size + 1)
after = os.fstat(source.fileno())
if len(content) != before.st_size or (
before.st_dev,
before.st_ino,
before.st_size,
before.st_mtime_ns,
before.st_ctime_ns,
) != (
after.st_dev,
after.st_ino,
after.st_size,
after.st_mtime_ns,
after.st_ctime_ns,
):
raise SourceReceiptError(
"Core release-bundle input changed during inspection"
)
result[relative] = {
"file": _filesystem_identity(path, observed),
"sha256": hashlib.sha256(content).hexdigest(),
}
return result
def _frozen_receipts(
*, expected, versions, specs, workspace, selected, push, bundle_inputs
):
actual = _collect_receipts(versions=versions, specs=specs, workspace=workspace)
for repo in expected:
if actual[repo] != expected[repo]:
raise SourceReceiptError(
f"{repo}: source receipt changed after whole-batch preflight"
)
for repo in versions:
issues = repository_version_issues(
resolve_repo_path(specs[repo], workspace), expected_version=versions[repo]
)
if issues:
raise SourceReceiptError(
f"{repo}: version/composition changed after whole-batch preflight"
)
if (
_bundle_input_receipt(selected=selected, workspace=workspace, push=push)
!= bundle_inputs
):
raise SourceReceiptError(
"Core release-bundle input receipt changed after whole-batch preflight"
)
if push or "govoplan-core" in selected:
if selected_release_webui_bundle_issues(
repo_versions={repo: versions[repo] for repo in selected},
workspace=workspace,
):
raise SourceReceiptError(
"release WebUI composition changed after whole-batch preflight"
)
def _require_core(receipts, *, push):
core = receipts["govoplan-core"]
if not core["local_tag_object"] or (
push and (not core["remote_tag_object"] or core["remote_main"] != core["head"])
):
raise SourceReceiptError(
"Meta requires the matching annotated Core tag locally and, for publication, remotely"
)
def _blocked(*, selected, apply, push, detail, rows=()): # noqa: A002
known = {row["repo"]: row for row in rows}
identified = next(
(repo for repo in selected if detail.startswith(repo + ":")), None
)
return {
"status": "blocked",
"apply": apply,
"push": push,
"remote": "origin",
"detail": "whole-batch source preflight failed; no selected repository was mutated",
"repositories": [
{
**known.get(repo, {"repo": repo}),
"status": "blocked"
if (
known.get(repo, {}).get("status") == "blocked"
or (not rows and (identified is None or repo == identified))
)
else "skipped",
"detail": known[repo]["detail"]
if known.get(repo, {}).get("status") == "blocked"
else detail,
}
for repo in selected
],
}
def tag_source_batch(
*, repos, repo_versions, workspace_root, remote, message, apply, push
): # noqa: A002
workspace = resolve_workspace_root(workspace_root)
has_meta = "govoplan" in repos
selected = tuple(dict.fromkeys(repos))
if has_meta:
selected = tuple(repo for repo in selected if repo != "govoplan") + (
"govoplan",
)
if not selected:
return {
"status": "noop",
"apply": apply,
"push": push,
"remote": remote.strip() or "origin",
"repositories": [],
}
versions = {repo: normalize_version(repo_versions.get(repo)) for repo in selected}
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
meta_version = versions.get("govoplan")
try:
if remote.strip() not in {"", "origin"}:
raise SourceReceiptError(
"Source-tag batches require the registered origin remote"
)
if any(not version for version in versions.values()):
raise SourceReceiptError(
"every selected repository requires an explicit valid version"
)
if has_meta and versions.get("govoplan-core", meta_version) != meta_version:
raise SourceReceiptError(
"Meta developer-package version must match the selected Core release"
)
# Only Meta requires a frozen version-matched Core source/tag dependency.
if has_meta:
versions.setdefault("govoplan-core", meta_version)
receipts = _collect_receipts(
versions=versions, specs=specs, workspace=workspace
)
if has_meta and "govoplan-core" not in selected:
_require_core(receipts, push=push)
core_issues = (
repository_version_issues(
resolve_repo_path(specs["govoplan-core"], workspace),
expected_version=meta_version,
)
if has_meta
else ()
)
if core_issues:
raise SourceReceiptError(
"Core source metadata must match the selected Meta version"
)
bundle_inputs = _bundle_input_receipt(
selected=selected, workspace=workspace, push=push
)
except (SourceReceiptError, OSError, ValueError) as exc:
return _blocked(selected=selected, apply=apply, push=push, detail=str(exc))
# Preserve the shared complete manifest, package/lock and immutable tag
# preflight. This invocation is always read-only; strict effects stay below.
try:
preview = _preview_repositories(
repos=selected,
repo_versions=repo_versions,
workspace_root=workspace,
remote="origin",
message=message,
push=push,
)
except (OSError, ValueError) as exc:
return _blocked(
selected=selected,
apply=apply,
push=push,
detail=f"shared release preflight could not validate its inputs ({type(exc).__name__})",
)
rows = preview["repositories"]
if preview["status"] in {"blocked", "partial"}:
if not apply:
return preview
return _blocked(
selected=selected,
apply=True,
push=push,
detail="shared release preflight failed",
rows=rows,
)
if not apply:
rows = [
{
**row,
"status": "planned",
"detail": "existing annotated release tag requires atomic main publication",
}
if push
and receipts[row["repo"]]["remote_main"] != receipts[row["repo"]]["head"]
else row
for row in rows
]
preview = {**preview, "repositories": rows}
if any(row["status"] == "planned" for row in rows):
preview["status"] = "planned"
return {
**preview,
"source_receipts": receipts,
"source_contract": "registered-meta-batch-v1"
if has_meta
else "registered-source-batch-v1",
"bundle_input_receipts": bundle_inputs,
}
results = []
effected = False
for row in rows:
repo = row["repo"]
receipt = receipts[repo]
path = resolve_repo_path(specs[repo], workspace)
tag = receipt["tag"]
head = receipt["head"]
try:
# Recheck the entire frozen batch, including Meta and Core, before
# every effect. Earlier successful effects update only their exact
# anticipated tag/branch receipt fields below.
_frozen_receipts(
expected=receipts,
versions=versions,
specs=specs,
workspace=workspace,
selected=selected,
push=push,
bundle_inputs=bundle_inputs,
)
if repo == "govoplan":
_require_core(receipts, push=push)
local_object = receipt["local_tag_object"]
created = False
if not local_object:
if receipt["remote_tag_object"]:
command = (
"git",
"fetch",
"--no-tags",
"origin",
f"refs/tags/{tag}:refs/tags/{tag}",
)
else:
command = ("git", "tag", "-a", tag, head, "-m", row["message"])
created = True
effected = True
if run(command, cwd=path).returncode:
raise SourceReceiptError(
"annotated local release tag could not be created or retrieved"
)
local_object = git_text(
path, "rev-parse", "--verify", f"refs/tags/{tag}"
)
if (
not local_object
or ref_commit(path, f"refs/tags/{tag}") != head
or git_text(path, "cat-file", "-t", f"refs/tags/{tag}") != "tag"
or (
receipt["remote_tag_object"]
and local_object != receipt["remote_tag_object"]
)
):
raise SourceReceiptError("local release tag postcondition failed")
receipt["local_tag_object"] = local_object
_frozen_receipts(
expected=receipts,
versions=versions,
specs=specs,
workspace=workspace,
selected=selected,
push=push,
bundle_inputs=bundle_inputs,
)
if push and (
receipt["remote_tag_object"] != local_object
or receipt["remote_main"] != head
):
# Pin both effects to verified objects, not mutable HEAD/tag
# names. No force, retagging, fallback or non-atomic retry.
command = (
"git",
"push",
"--atomic",
"origin",
f"{head}:refs/heads/main",
f"{local_object}:refs/tags/{tag}",
)
effected = True
if run(command, cwd=path).returncode:
raise SourceReceiptError(
"atomic main and annotated tag publication failed; inspect receipts before retrying"
)
receipt["remote_main"] = head
receipt["remote_tag_object"] = local_object
# Verify remote main AND exact annotated object, as well as local
# source state. A successful Git exit alone is never a receipt.
_frozen_receipts(
expected=receipts,
versions=versions,
specs=specs,
workspace=workspace,
selected=selected,
push=push,
bundle_inputs=bundle_inputs,
)
results.append(
{
**row,
"status": "published" if push else "tagged" if created else "noop",
"detail": "verified strict registered-source release"
if not receipt["remote_tag_object"] or created
else "verified strict registered-source release; immutable annotation already published or present",
"after_local_tag_commit": head,
"after_local_tag_object": local_object,
"after_remote_tag_commit": head
if receipt["remote_tag_object"]
else None,
"after_remote_tag_object": receipt["remote_tag_object"],
"after_remote_main_commit": receipt["remote_main"],
}
)
except (SourceReceiptError, OSError, ValueError) as exc:
results.append(
{
**row,
"status": "failed" if effected else "blocked",
"detail": str(exc),
}
)
results.extend(
{
**later,
"status": "skipped",
"detail": "earlier strict source effect or receipt failed",
}
for later in rows[len(results) :]
)
return {
"status": "partial" if effected else "blocked",
"apply": True,
"push": push,
"remote": "origin",
"repositories": results,
}
status = (
"published"
if push
else "tagged"
if any(row["status"] == "tagged" for row in results)
else "noop"
)
return {
"status": status,
"apply": True,
"push": push,
"remote": "origin",
"repositories": results,
"source_receipts": receipts,
"source_contract": "registered-meta-batch-v1"
if has_meta
else "registered-source-batch-v1",
"bundle_input_receipts": bundle_inputs,
}
@@ -6,11 +6,13 @@ from dataclasses import dataclass
import json import json
from pathlib import Path from pathlib import Path
import re import re
import runpy
import subprocess import subprocess
import tomllib import tomllib
from .git_state import collect_versions, sanitized_git_environment from .git_state import collect_versions, registered_developer_meta_path, sanitized_git_environment
from .workspace import load_repository_specs, resolve_repo_path from .registry_reference import registry_artifact_conflicts, registry_entry_source
from .workspace import META_ROOT, load_repository_specs, resolve_repo_path
_PYTHON_RELEASE_REF = re.compile( _PYTHON_RELEASE_REF = re.compile(
@@ -42,6 +44,7 @@ def repository_version_issues(
versions = collect_versions(repo_path) versions = collect_versions(repo_path)
declared = { declared = {
"pyproject.toml": versions.pyproject, "pyproject.toml": versions.pyproject,
"packages/govoplan-meta/pyproject.toml": versions.developer_meta,
"package.json": versions.package, "package.json": versions.package,
"webui/package.json": versions.webui_package, "webui/package.json": versions.webui_package,
"webui/package.release.json": _json_version(repo_path / "webui" / "package.release.json") "webui/package.release.json": _json_version(repo_path / "webui" / "package.release.json")
@@ -82,6 +85,8 @@ def repository_version_issues(
for source, version in declared.items() for source, version in declared.items()
if version is not None and version != canonical_version if version is not None and version != canonical_version
] ]
if registered_developer_meta_path(repo_path) is not None:
issues.extend(developer_meta_composition_issues(repo_path))
if expected_version is not None and canonical_version.removeprefix("v") != expected_version.removeprefix("v"): if expected_version is not None and canonical_version.removeprefix("v") != expected_version.removeprefix("v"):
issues.append( issues.append(
@@ -119,6 +124,34 @@ def repository_version_issues(
return tuple(issues) return tuple(issues)
def developer_meta_composition_issues(repo_path: Path) -> tuple[VersionAlignmentIssue, ...]:
"""Compare the real nested package with the trusted generator's exact output.
Never execute a generator from a selected checkout. Only the installed
operator tooling provides code; selected TOML/requirements are data inputs.
"""
package_path = registered_developer_meta_path(repo_path)
if package_path is None:
return (VersionAlignmentIssue(repo_path.name, "repository", "registered Meta support repository", "", "nested developer-package identity is not registered"),)
source = "packages/govoplan-meta/pyproject.toml"
try:
current = package_path.read_text(encoding="utf-8")
project = tomllib.loads(current).get("project")
if not isinstance(project, dict) or project.get("name") != "govoplan":
return (VersionAlignmentIssue("govoplan", source + ":project.name", "govoplan", str(project.get("name") if isinstance(project, dict) else ""), "developer meta-package identity must be exact"),)
# META_ROOT belongs to the running operator tools, not repo_path.
generator = runpy.run_path(str(META_ROOT / "tools/release/generate-developer-meta-package.py"))
expected = generator["render"](
workspace=repo_path.parent,
requirements=repo_path / "requirements-release.txt",
)
except (OSError, UnicodeError, KeyError, ValueError, TypeError) as exc:
return (VersionAlignmentIssue("govoplan", source, "readable exact developer composition and Core version", type(exc).__name__, "developer meta-package composition could not be validated"),)
if current != expected:
return (VersionAlignmentIssue("govoplan", source, "trusted generator output matching Core and release requirements", "stale composition", "developer meta-package must exactly match the generator --check contract"),)
return ()
def selected_repository_version_issues( def selected_repository_version_issues(
*, *,
repo_versions: dict[str, str], repo_versions: dict[str, str],
@@ -164,6 +197,11 @@ def selected_repository_version_issues(
return tuple(issues) return tuple(issues)
def selected_webui_repository_names(*, repo_versions: dict[str, str], workspace: Path) -> tuple[str, ...]:
"""Identify the exact selections for which Core's WebUI inputs are relevant."""
return tuple(repo for repo in sorted(repo_versions) if repo != "govoplan-core" and (workspace / repo / "webui/package.json").exists())
def selected_release_webui_bundle_issues( def selected_release_webui_bundle_issues(
*, *,
repo_versions: dict[str, str], repo_versions: dict[str, str],
@@ -176,6 +214,8 @@ def selected_release_webui_bundle_issues(
immutable release package input and lockfile will actually install. immutable release package input and lockfile will actually install.
""" """
if not selected_webui_repository_names(repo_versions=repo_versions, workspace=workspace):
return ()
core_webui = workspace / "govoplan-core" / "webui" core_webui = workspace / "govoplan-core" / "webui"
release_package_path = core_webui / "package.release.json" release_package_path = core_webui / "package.release.json"
release_lock_path = core_webui / "package-lock.release.json" release_lock_path = core_webui / "package-lock.release.json"
@@ -407,6 +447,11 @@ def candidate_catalog_version_issues(payload: object) -> tuple[VersionAlignmentI
) )
release = payload.get("release") release = payload.get("release")
registry_entries = [core_release, *(modules if isinstance(modules, list) else [])]
issues.extend(
_catalog_shape_issue("artifact_integrity", issue)
for issue in registry_artifact_conflicts(registry_entries)
)
if isinstance(release, dict): if isinstance(release, dict):
issues.extend(_catalog_release_issues(release, represented=represented)) issues.extend(_catalog_release_issues(release, represented=represented))
return tuple(issues) return tuple(issues)
@@ -419,6 +464,15 @@ def _catalog_entry_issues(
represented: dict[str, str], represented: dict[str, str],
) -> list[VersionAlignmentIssue]: ) -> list[VersionAlignmentIssue]:
issues: list[VersionAlignmentIssue] = [] issues: list[VersionAlignmentIssue] = []
try:
registry_source = registry_entry_source(entry)
except ValueError as exc:
return [_catalog_shape_issue(source, str(exc))]
if registry_source is not None:
previous = represented.setdefault(registry_source.repository, registry_source.version)
if previous != registry_source.version:
issues.append(_catalog_shape_issue(source, "repository appears with conflicting catalog versions"))
return issues
version = entry.get("version") version = entry.get("version")
normalized_version = version.removeprefix("v") if isinstance(version, str) and version else None normalized_version = version.removeprefix("v") if isinstance(version, str) and version else None
if normalized_version is None: if normalized_version is None:
@@ -34,6 +34,19 @@ def version_metadata_mutations(
) -> tuple[VersionFileMutation, ...]: ) -> tuple[VersionFileMutation, ...]:
"""Render all recognized repository version files without writing them.""" """Render all recognized repository version files without writing them."""
from .git_state import registered_developer_meta_path
if registered_developer_meta_path(repo_path) is not None:
from .meta_preparation import MetaPreparationError, PACKAGE, preview_meta_mutation
try:
_receipt, before, after = preview_meta_mutation(
repo_path=repo_path, target_version=target_version,
)
except (MetaPreparationError, OSError, ValueError, KeyError, TypeError) as exc:
raise VersionMetadataError(str(exc)) from exc
return (VersionFileMutation(PACKAGE, before, after),) if before != after else ()
version = target_version.removeprefix("v") version = target_version.removeprefix("v")
candidates: list[tuple[Path, str]] = [] candidates: list[tuple[Path, str]] = []
if (repo_path / "pyproject.toml").is_file(): if (repo_path / "pyproject.toml").is_file():
@@ -115,6 +128,14 @@ def apply_version_metadata_mutations(
) -> tuple[str, ...]: ) -> tuple[str, ...]:
"""Apply one deterministic version update, rolling back on write failure.""" """Apply one deterministic version update, rolling back on write failure."""
from .git_state import registered_developer_meta_path
if registered_developer_meta_path(repo_path) is not None:
raise VersionMetadataError(
"Meta is prepared outside durable runs with prepare-developer-meta-package.py; "
"review its complete generated composition, commit, and create a fresh run."
)
mutations = version_metadata_mutations( mutations = version_metadata_mutations(
repo_path, repo_path,
target_version=target_version, target_version=target_version,
@@ -269,6 +290,7 @@ def _render_python_version(
except SyntaxError as exc: except SyntaxError as exc:
raise VersionMetadataError(f"Python metadata is malformed: {path.name}") from exc raise VersionMetadataError(f"Python metadata is malformed: {path.name}") from exc
values: list[ast.Constant] = [] values: list[ast.Constant] = []
module_version_references = 0
for node in ast.walk(tree): for node in ast.walk(tree):
if not isinstance(node, ast.Call) or _call_name(node.func) != target_name: if not isinstance(node, ast.Call) or _call_name(node.func) != target_name:
continue continue
@@ -279,6 +301,28 @@ def _render_python_version(
and isinstance(keyword.value.value, str) and isinstance(keyword.value.value, str)
): ):
values.append(keyword.value) values.append(keyword.value)
elif (
keyword.arg == keyword_name
and isinstance(keyword.value, ast.Name)
and keyword.value.id == "MODULE_VERSION"
):
module_version_references += 1
if module_version_references:
if module_version_references != 1 or values:
raise VersionMetadataError(
f"Python metadata has multiple {target_name}.{keyword_name} values: {path.name}"
)
rendered, found = _render_python_assignment(
payload,
path=path,
assignment_name="MODULE_VERSION",
version=version,
)
if not found:
raise VersionMetadataError(
f"Python metadata has no literal MODULE_VERSION declaration: {path.name}"
)
return rendered, True
if not values: if not values:
return payload, False return payload, False
if len(values) != 1: if len(values) != 1:
@@ -19,8 +19,9 @@ retry() {
for attempt in 1 2 3; do for attempt in 1 2 3; do
if "$@"; then if "$@"; then
return 0 return 0
fi else
status=$? status=$?
fi
if [[ "$attempt" == 3 ]]; then if [[ "$attempt" == 3 ]]; then
return "$status" return "$status"
fi fi
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""Preview or explicitly prepare Meta outside a durable release run."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from govoplan_release.meta_preparation import (
MetaPreparationAmbiguous,
MetaPreparationError,
_read_input,
prepare_developer_meta_package,
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--workspace", type=Path, required=True)
parser.add_argument("--target-version", required=True)
parser.add_argument("--apply", action="store_true")
parser.add_argument("--receipt", type=Path)
parser.add_argument("--confirm-out-of-run", action="store_true")
args = parser.parse_args()
try:
expected = None
if args.apply:
if args.receipt is None:
raise MetaPreparationError(
"Apply requires the reviewed preview JSON via --receipt."
)
payload, _ = _read_input(args.receipt)
expected = json.loads(payload)["receipt"]
result = prepare_developer_meta_package(
repo_path=args.workspace.absolute() / "govoplan",
target_version=args.target_version,
apply=args.apply,
expected_receipt=expected,
confirm_out_of_run=args.confirm_out_of_run,
)
except (MetaPreparationError, OSError, ValueError, KeyError, TypeError) as exc:
status = "needs-reconciliation" if isinstance(exc, MetaPreparationAmbiguous) else "blocked"
print(json.dumps({"status": status, "detail": str(exc)}))
return 1
print(json.dumps(result, indent=2))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+31
View File
@@ -21,6 +21,22 @@ def main() -> int:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
subparsers = parser.add_subparsers(dest="command", required=True) subparsers = parser.add_subparsers(dest="command", required=True)
full = subparsers.add_parser("full-registry", help="Build a strict full-profile candidate from verified registry artifacts.")
full.add_argument("--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT)
full.add_argument("--package-set", type=Path, required=True)
full.add_argument("--package-lock", type=Path, required=True)
full.add_argument("--wheelhouse", type=Path, required=True)
full.add_argument("--webui-packages", type=Path, required=True)
full.add_argument("--output-dir", type=Path, required=True)
full.add_argument("--selected-repository", action="append", required=True)
full.add_argument("--catalog-signing-key", action="append", required=True)
full.add_argument("--channel", default="stable")
full.add_argument("--source-remote", default="origin")
full.add_argument("--public-base-url", default="https://govoplan.add-ideas.de")
full.add_argument("--expires-days", type=int, default=90)
full.add_argument("--sequence", type=int)
full.add_argument("--json", action="store_true")
selective = subparsers.add_parser( selective = subparsers.add_parser(
"selective", help="Build a signed selective channel catalog candidate." "selective", help="Build a signed selective channel catalog candidate."
) )
@@ -143,6 +159,21 @@ def main() -> int:
) )
args = parser.parse_args() args = parser.parse_args()
if args.command == "full-registry":
require_release_runtime_trust()
from govoplan_release.full_catalog import build_full_registry_candidate
result = build_full_registry_candidate(
package_set_path=args.package_set, package_lock_path=args.package_lock,
wheelhouse=args.wheelhouse, webui_packages=args.webui_packages,
output_dir=args.output_dir, selected_repositories=tuple(args.selected_repository),
signing_keys=tuple(args.catalog_signing_key), workspace_root=args.workspace_root,
channel=args.channel, source_remote=args.source_remote,
public_base_url=args.public_base_url, expires_days=args.expires_days,
sequence=args.sequence,
)
print(json.dumps(result, indent=2, sort_keys=True))
return 0
if args.command == "selective": if args.command == "selective":
require_release_runtime_trust() require_release_runtime_trust()
result = build_selective_catalog_candidate( result = build_selective_catalog_candidate(
+190 -15
View File
@@ -8,7 +8,7 @@ from datetime import datetime, timezone
import json import json
import os import os
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any, NoReturn
META_ROOT = Path(__file__).resolve().parents[2] META_ROOT = Path(__file__).resolve().parents[2]
@@ -168,12 +168,13 @@ def owner_for_versions_dir(versions_dir: Path) -> str:
def parse_migration_file(owner: str, path: Path) -> Migration | None: def parse_migration_file(owner: str, path: Path) -> Migration | None:
if path.stat().st_size > 2 * 1024 * 1024:
raise ValueError(f"{path}: migration source exceeds the 2 MiB audit bound")
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
values: dict[str, Any] = {} values = _imported_release_metadata(owner, path, tree)
wrapped: Migration | None = None wrapped = _wrapped_release_metadata(owner, path, tree)
release_peer = path.parent.parent / "versions" / path.name if values and wrapped:
if path.parent.name == "dev_versions" and release_peer.is_file(): raise ValueError(f"{path}: mixed migration metadata wrapper styles are ambiguous")
wrapped = parse_migration_file(owner, release_peer)
for statement in tree.body: for statement in tree.body:
if isinstance(statement, ast.Assign): if isinstance(statement, ast.Assign):
for target in statement.targets: for target in statement.targets:
@@ -196,6 +197,8 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
) )
revision = values.get("revision") revision = values.get("revision")
if not isinstance(revision, str): if not isinstance(revision, str):
if "revision" in values:
raise ValueError(f"{path}: migration revision must be an explicit string")
return None return None
return Migration( return Migration(
owner=owner, owner=owner,
@@ -207,28 +210,200 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
) )
def _ast_binding_count(tree: ast.Module, name: str) -> int:
count = 0
for node in ast.walk(tree):
if isinstance(node, ast.Name) and node.id == name and isinstance(node.ctx, (ast.Store, ast.Del)):
count += 1
elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) and node.name == name:
count += 1
elif isinstance(node, (ast.Import, ast.ImportFrom)):
count += sum(
(alias.asname or (alias.name.split(".")[0] if isinstance(node, ast.Import) else alias.name)) == name
for alias in node.names
)
return count
def _release_wrapper_peer(owner: str, path: Path, filename: str) -> Migration:
versions = path.parent.parent / "versions"
peer = versions / filename
if (
path.parent.name != "dev_versions"
or Path(filename).name != filename or not filename.endswith(".py")
or versions.is_symlink() or peer.is_symlink() or not peer.is_file()
or peer.resolve().parent != versions.resolve()
or peer.stat().st_size > 2 * 1024 * 1024
):
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
migration = parse_migration_file(owner, peer)
if migration is None:
raise ValueError(f"{path}: release wrapper target has no migration metadata")
return migration
def _imported_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Any]:
"""Recognize explicit metadata re-exports, never star/dynamic imports."""
names = {"revision", "down_revision", "depends_on", "branch_labels"}
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
values: dict[str, Any] = {}
targets: set[str] = set()
for statement in tree.body:
if not isinstance(statement, ast.ImportFrom):
continue
selected = [alias for alias in statement.names if alias.name in names or (alias.asname or alias.name) in names]
if not selected:
if (statement.module or "").startswith(prefix) and any(alias.name == "*" for alias in statement.names):
raise ValueError(f"{path}: unsupported wildcard migration metadata")
continue
stem = (statement.module or "").removeprefix(prefix)
if (
statement.level or not (statement.module or "").startswith(prefix)
or not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem)
):
raise ValueError(f"{path}: unsupported or ambiguous imported migration metadata")
targets.add(stem)
if len(targets) != 1:
raise ValueError(f"{path}: ambiguous imported migration metadata sources")
migration = _release_wrapper_peer(owner, path, stem + ".py")
projection = {
"revision": migration.revision, "down_revision": migration.down_revisions,
"depends_on": migration.depends_on, "branch_labels": migration.branch_labels,
}
for alias in selected:
name = alias.asname or alias.name
if name != alias.name or name in values or _ast_binding_count(tree, name) != 1:
raise ValueError(f"{path}: ambiguous imported migration metadata assignment")
values[name] = projection[name]
return values
def _wrapped_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Migration]:
"""Resolve only known literal sibling wrappers, without importing any code."""
metadata_names = {"revision", "down_revision", "depends_on", "branch_labels"}
aliases: set[str] = set()
bindings: dict[str, list[ast.expr]] = {}
imported: dict[str, list[str]] = {}
for statement in tree.body:
if isinstance(statement, ast.ImportFrom) and not statement.level:
for alias in statement.names:
imported.setdefault(alias.asname or alias.name, []).append(f"{statement.module}.{alias.name}")
targets: list[ast.expr] = []
value: ast.expr | None = None
if isinstance(statement, ast.Assign):
targets, value = statement.targets, statement.value
elif isinstance(statement, ast.AnnAssign) and statement.value is not None:
targets, value = [statement.target], statement.value
for target in targets:
if isinstance(target, ast.Name) and value is not None:
bindings.setdefault(target.id, []).append(value)
if target.id in metadata_names and isinstance(value, ast.Attribute) and isinstance(value.value, ast.Name):
aliases.add(value.value.id)
if not aliases:
return {}
if path.parent.name != "dev_versions":
raise ValueError(f"{path}: non-literal release migration metadata is unsupported")
def reject() -> NoReturn:
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
def binding_count(name: str) -> int:
return _ast_binding_count(tree, name)
def assigned(name: str) -> ast.expr:
values = bindings.get(name, ())
if len(values) != 1 or binding_count(name) != 1 or name in imported:
reject()
return values[0]
def imported_as(node: ast.expr, qualified: str) -> bool:
return (
isinstance(node, ast.Name)
and imported.get(node.id) == [qualified]
and binding_count(node.id) == 1
)
def call(node: ast.expr, qualified: str, arguments: int) -> bool:
return isinstance(node, ast.Call) and imported_as(node.func, qualified) and len(node.args) == arguments and not node.keywords
def file_wrapper_target(node: ast.expr) -> str:
if binding_count("__file__"):
reject()
if isinstance(node, ast.Name):
node = assigned(node.id)
if not (
isinstance(node, ast.BinOp) and isinstance(node.op, ast.Div)
and isinstance(node.right, ast.Constant) and isinstance(node.right.value, str)
and isinstance(node.left, ast.BinOp) and isinstance(node.left.op, ast.Div)
and isinstance(node.left.right, ast.Constant) and node.left.right.value == "versions"
):
reject()
root = node.left.left
for name in imported:
if imported_as(ast.Name(id=name), "pathlib.Path"):
expected = ast.parse(f"{name}(__file__).resolve().parents[1]", mode="eval").body
if ast.dump(root) == ast.dump(expected):
return node.right.value
reject()
result: dict[str, Migration] = {}
resolved: set[Path] = set()
for alias in sorted(aliases):
value = assigned(alias)
if call(value, "importlib.import_module", 1):
argument = value.args[0]
if not isinstance(argument, ast.Constant) or not isinstance(argument.value, str):
reject()
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
if not argument.value.startswith(prefix):
reject()
stem = argument.value.removeprefix(prefix)
if not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem):
reject()
filename = stem + ".py"
elif call(value, "importlib.util.module_from_spec", 1):
argument = value.args[0]
if not isinstance(argument, ast.Name):
reject()
specification = assigned(argument.id)
if not call(specification, "importlib.util.spec_from_file_location", 2):
reject()
if not isinstance(specification.args[0], ast.Constant) or not isinstance(specification.args[0].value, str):
reject()
filename = file_wrapper_target(specification.args[1])
else:
reject()
migration = _release_wrapper_peer(owner, path, filename)
peer = migration.path
resolved.add(peer.resolve())
if len(resolved) > 1:
reject()
result[alias] = migration
return result
def _migration_assignment_value( def _migration_assignment_value(
name: str, name: str,
value: ast.expr, value: ast.expr,
*, *,
wrapped: Migration | None, wrapped: dict[str, Migration],
) -> Any: ) -> Any:
try: try:
return ast.literal_eval(value) return ast.literal_eval(value)
except (ValueError, TypeError): except (ValueError, TypeError):
if ( if (
wrapped is None not isinstance(value, ast.Attribute)
or not isinstance(value, ast.Attribute)
or not isinstance(value.value, ast.Name) or not isinstance(value.value, ast.Name)
or value.value.id != "_migration" or value.value.id not in wrapped
or value.attr != name or value.attr != name
): ):
return None raise ValueError(f"Unsupported or ambiguous migration metadata: {name}")
migration = wrapped[value.value.id]
return { return {
"revision": wrapped.revision, "revision": migration.revision,
"down_revision": wrapped.down_revisions, "down_revision": migration.down_revisions,
"depends_on": wrapped.depends_on, "depends_on": migration.depends_on,
"branch_labels": wrapped.branch_labels, "branch_labels": migration.branch_labels,
}[name] }[name]
+1
View File
@@ -2141,6 +2141,7 @@
function primaryVersion(repo) { function primaryVersion(repo) {
const versions = repo.versions || {}; const versions = repo.versions || {};
if (versions.pyproject) return versions.pyproject; if (versions.pyproject) return versions.pyproject;
if (versions.developer_meta) return versions.developer_meta;
if (versions.package) return versions.package; if (versions.package) return versions.package;
if (versions.webui_package) return versions.webui_package; if (versions.webui_package) return versions.webui_package;
if (Array.isArray(versions.manifests) && versions.manifests.length) return versions.manifests[0]; if (Array.isArray(versions.manifests) && versions.manifests.length) return versions.manifests[0];