Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
14b19fbead | ||
|
|
845dcbafdb | ||
|
|
58d320d9b3 | ||
|
|
9554657bb5 | ||
|
|
88b685ff5e | ||
|
|
be57a1823a | ||
|
|
6bcb75f577 | ||
|
|
32fe4b7238 | ||
|
|
6a8f53b87d | ||
|
|
1cec4ee1d8 | ||
|
|
29d03aa2ca | ||
|
|
6fb928d6cf | ||
|
|
6edaaadf37 | ||
|
|
0b171fbdd4 | ||
|
|
ed6790c057 | ||
|
|
3766e26377 | ||
|
|
6c2b36af0f | ||
|
|
3f75ca8e48 | ||
|
|
a886a9b3de | ||
|
|
fe83290d56 | ||
|
|
c50f699399 | ||
|
|
59b45a0829 | ||
|
|
861abcc573 | ||
|
|
5e995fed88 | ||
|
|
41ca242004 | ||
|
|
85caa8d337 |
@@ -23,6 +23,9 @@ jobs:
|
||||
- name: Test declarative deployment bundle
|
||||
working-directory: govoplan
|
||||
run: python -m unittest -v tests.test_deployment_installer
|
||||
- name: Test WebUI installer retry failures
|
||||
working-directory: govoplan
|
||||
run: python -m unittest -v tests.test_webui_release_dependency_retries
|
||||
- name: Build single-file deployer artifact
|
||||
working-directory: govoplan
|
||||
run: |
|
||||
|
||||
@@ -72,6 +72,37 @@ Each WebUI module should be able to announce:
|
||||
The contract references surfaces. It does not permit Core or a product package
|
||||
to import their implementation.
|
||||
|
||||
The versioned `product_surfaces` slice is implemented in Core. It
|
||||
binds a stable product identity and entry path to one or more owner routes,
|
||||
View surfaces, presentations, capabilities, search sources, help contexts and
|
||||
documentation topics. It also carries standard unavailable/degraded
|
||||
explanations and migration aliases. Mail and Postbox contribute the first
|
||||
shared identity, `communication.messages`: `/messages` and the migration alias
|
||||
`/inbox` select the first currently authorized, View-visible owner while the
|
||||
underlying `/mail` and `/postbox` deep links, custody and permissions remain
|
||||
unchanged. Tasks, Calendar and Files contribute the corresponding single-owner
|
||||
identities:
|
||||
|
||||
| Product identity | Stable destination | Compatible owner route |
|
||||
| --- | --- | --- |
|
||||
| Work | `/work` | `/tasks` |
|
||||
| Calendar | `/agenda` | `/calendar` |
|
||||
| Messages | `/messages` (`/inbox` alias) | `/mail`, `/postbox` |
|
||||
| Files | `/documents` | `/files` |
|
||||
|
||||
Core replaces those owner entries in the ordinary rail with the stable product
|
||||
destinations. A collapsed **All available tools** catalogue retains every
|
||||
authorized technical owner route independently of View focus; unauthorized
|
||||
entries are never disclosed. The original deep links remain valid, and all
|
||||
contributing owner paths keep the corresponding product entry active. Alias
|
||||
resolution emits a bounded client telemetry event before the redirect.
|
||||
|
||||
Core's `ProductAvailabilityState` is the shared presentation primitive for
|
||||
authorization, Policy, configuration, disabled, missing-capability, offline and
|
||||
provider-degraded states. Product language is primary; exact module,
|
||||
capability, provider and correlation provenance is available only in an
|
||||
expandable technical section.
|
||||
|
||||
## Navigation Model
|
||||
|
||||
The default shell should prioritize:
|
||||
@@ -88,10 +119,11 @@ People and Responsibility. They are configurable system/tenant defaults and
|
||||
Views projections, not hard-coded repository groups. Empty areas disappear;
|
||||
single-destination areas may link directly; familiar tools may remain pinned.
|
||||
|
||||
The complete permission-derived module rail remains available as **All
|
||||
available tools**. Its ability to scroll is useful and is not itself the
|
||||
product defect. The defect is requiring people to infer a task or outcome from
|
||||
repository topology.
|
||||
The complete permission-derived module rail is available as the collapsed
|
||||
**All available tools** escape. It is deliberately independent of the active
|
||||
View while still enforcing authorization. Its ability to scroll is useful and
|
||||
is not itself the product defect. The defect is requiring people to infer a
|
||||
task or outcome from repository topology.
|
||||
|
||||
Task-local Work, Calendar, Messages and Files tools may be contributed to the
|
||||
optional `govoplan-quick-access` rail. Messages composes Mail, Postbox and
|
||||
@@ -108,6 +140,12 @@ sections, commands, widgets, and fields. A view must not grant a permission or
|
||||
change data semantics. Policy can force, allow, or prohibit a surface at system,
|
||||
tenant, group, or user scope.
|
||||
|
||||
Core browser conformance exercises the German Anwohnerparkausweis reference
|
||||
context with Work, Calendar, Messages and Files entries, verifies that package
|
||||
owner labels are absent from the primary rail, expands the technical catalogue,
|
||||
and runs WCAG 2 A/AA checks over the result. Unit permutations cover two-owner,
|
||||
one-owner, unauthorized-owner and focused-View compositions.
|
||||
|
||||
## Error And Provenance Language
|
||||
|
||||
Normal errors answer:
|
||||
@@ -132,9 +170,9 @@ first rail slice.
|
||||
|
||||
### Slice 1: inventory and aliases
|
||||
|
||||
- classify every route, navigation item, widget, setting, search object, and
|
||||
- continue classifying every route, navigation item, widget, setting, search object, and
|
||||
help context by product area and object type;
|
||||
- add product aliases without removing existing deep links;
|
||||
- extend the implemented product-surface aliases without removing existing deep links;
|
||||
- flag raw module IDs in ordinary-user labels and errors.
|
||||
|
||||
### Slice 2: work-first shell
|
||||
|
||||
@@ -39,16 +39,24 @@ The first production-shaped slice is implemented:
|
||||
order and optional labels. Scoped Views therefore configure product
|
||||
presentation for system, tenant, group, user and Workflow contexts;
|
||||
- the expanded left rail groups classified destinations while retaining
|
||||
Dashboard and every authorized unclassified destination under More tools.
|
||||
Dashboard and every authorized unclassified destination under More tools;
|
||||
- Core promotes Work (`/work`), Calendar (`/agenda`), Messages (`/messages`)
|
||||
and Files (`/documents`) into stable primary destinations and collapses the
|
||||
compatible owner routes under **All available tools**;
|
||||
- **All available tools** is permission-derived but independent of the active
|
||||
View, providing a deliberate escape without granting access or discarding
|
||||
the original `/tasks`, `/calendar`, `/mail`, `/postbox` and `/files` links.
|
||||
|
||||
The baseline classification is now manifest-declared for every ordinary
|
||||
user-facing module and enforced by the workspace manifest check. A separately
|
||||
The baseline classification and the four initial stable destinations are now
|
||||
manifest-declared. The area classification covers every ordinary user-facing
|
||||
module and is enforced by the workspace manifest check. A separately
|
||||
versioned launch-context contract carries bounded active-object, acting,
|
||||
temporal, View and return references into full-page Quick Access fallbacks;
|
||||
Cases publishes the first active-object reference. The remaining rollout is to
|
||||
add useful bounded tools and active-object publishers only where a maintained
|
||||
journey benefits, and to extend browser evidence to a pinned reference
|
||||
composition. Authorized global and technical routes remain visible through
|
||||
journey benefits. The pinned German Anwohnerparkausweis browser composition
|
||||
verifies stable product labels, technical escape, keyboard access and WCAG
|
||||
conformance. Authorized global and technical routes remain visible through
|
||||
their dedicated shell entry or **All available tools**.
|
||||
|
||||
## Quick Access Boundary
|
||||
@@ -166,9 +174,10 @@ areas, and users may personalize them within Policy ceilings. An empty area is
|
||||
omitted. An area with one destination may open it directly. A multi-destination
|
||||
area provides a useful work/recent/action surface rather than another menu.
|
||||
|
||||
Familiar product nouns such as Calendar, Mail or Files may remain directly
|
||||
pinned. The objective is not to hide every module name; it is to prevent
|
||||
repository topology from determining a person's workflow.
|
||||
Familiar product nouns such as Calendar or Files remain direct product
|
||||
destinations. The objective is not to hide every implementation name from
|
||||
administrators; it is to prevent repository topology from determining a
|
||||
person's workflow.
|
||||
|
||||
The initial module classification is deliberately outcome-oriented:
|
||||
|
||||
|
||||
@@ -5,9 +5,9 @@ A migration-owning module must register and document its canonical DSAR provider
|
||||
Every other active module requires a reviewed explanation of why it owns no
|
||||
persistent subject-data store. Adding a migration invalidates that explanation.
|
||||
|
||||
- Active modules: 68
|
||||
- Registered and documented DSAR providers: 48
|
||||
- Reviewed no-store rationales: 20
|
||||
- Active modules: 72
|
||||
- Registered and documented DSAR providers: 49
|
||||
- Reviewed no-store rationales: 23
|
||||
- Unexplained coverage gaps: 0
|
||||
|
||||
| Module | Repository | Persistence | Coverage | Rationale |
|
||||
@@ -32,11 +32,14 @@ persistent subject-data store. Adding a migration invalidates that explanation.
|
||||
| `datasources` | `govoplan-datasources` | Migration-owned | Provider | Provider `privacy.dsar.datasources` is registered and documented. |
|
||||
| `decisions` | `govoplan-decisions` | Migration-owned | Provider | Provider `privacy.dsar.decisions` is registered and documented. |
|
||||
| `dist_lists` | `govoplan-dist-lists` | Migration-owned | Provider | Provider `privacy.dsar.dist_lists` is registered and documented. |
|
||||
| `dms` | `govoplan-dms` | No module migration | Reviewed no-store rationale | Stateless integration-preview module: DMS retains no document, person, credential, or provider-response store; Files and Records remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, or diagnostic. |
|
||||
| `docs` | `govoplan-docs` | Migration-owned | Provider | Provider `privacy.dsar.docs` is registered and documented. |
|
||||
| `encryption` | `govoplan-encryption` | Migration-owned | Provider | Provider `privacy.dsar.encryption` is registered and documented. |
|
||||
| `erp` | `govoplan-erp` | No module migration | Reviewed no-store rationale | Stateless integration-contract module: ERP retains no invoice, payable, plan, booking observation, provider response, or credential store; Procurement, Payments, Ledger, Files, and Audit remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, reconciliation decision, or diagnostic. |
|
||||
| `evaluation` | `govoplan-evaluation` | No module migration | Reviewed no-store rationale | Contract-only module: evaluation runs, responses, and scores are not persisted; reassess before adding a migration-owned store. |
|
||||
| `facilities` | `govoplan-facilities` | No module migration | Reviewed no-store rationale | Contract-only module: facility and maintenance persistence are not implemented; reassess before adding a migration-owned store. |
|
||||
| `files` | `govoplan-files` | Migration-owned | Provider | Provider `privacy.dsar.files` is registered and documented. |
|
||||
| `fit_connect` | `govoplan-fit-connect` | No module migration | Reviewed no-store rationale | Stateless transport-contract module: FIT-Connect retains no submission, attachment, receipt, acknowledgement plan, key, provider response, or diagnostic store; the owning Service, Forms, Cases, Files, and Audit workflows remain responsible for subject data. Reassess before persisting any ingress or event-log evidence. |
|
||||
| `forms` | `govoplan-forms` | Migration-owned | Provider | Provider `privacy.dsar.forms` is registered and documented. |
|
||||
| `forms_runtime` | `govoplan-forms-runtime` | Migration-owned | Provider | Provider `privacy.dsar.forms_runtime` is registered and documented. |
|
||||
| `grants` | `govoplan-grants` | No module migration | Reviewed no-store rationale | Contract-only module: grant applications, awards, and monitoring are not persisted; reassess before adding a migration-owned store. |
|
||||
@@ -72,7 +75,7 @@ persistent subject-data store. Adding a migration invalidates that explanation.
|
||||
| `soap` | `govoplan-soap` | No module migration | Reviewed no-store rationale | Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store. |
|
||||
| `tasks` | `govoplan-tasks` | Migration-owned | Provider | Provider `privacy.dsar.tasks` is registered and documented. |
|
||||
| `templates` | `govoplan-templates` | Migration-owned | Provider | Provider `privacy.dsar.templates` is registered and documented. |
|
||||
| `tenancy` | `govoplan-tenancy` | No module migration | Reviewed no-store rationale | Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data. |
|
||||
| `tenancy` | `govoplan-tenancy` | Migration-owned | Provider | Provider `privacy.dsar.tenancy` is registered and documented. |
|
||||
| `tickets` | `govoplan-tickets` | Migration-owned | Provider | Provider `privacy.dsar.tickets` is registered and documented. |
|
||||
| `transparency` | `govoplan-transparency` | No module migration | Reviewed no-store rationale | Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store. |
|
||||
| `views` | `govoplan-views` | Migration-owned | Provider | Provider `privacy.dsar.views` is registered and documented. |
|
||||
@@ -80,6 +83,7 @@ persistent subject-data store. Adding a migration invalidates that explanation.
|
||||
| `wiki` | `govoplan-wiki` | Migration-owned | Provider | Provider `privacy.dsar.wiki` is registered and documented. |
|
||||
| `workflow` | `govoplan-workflow` | No module migration | Reviewed no-store rationale | Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage. |
|
||||
| `workflow_engine` | `govoplan-workflow-engine` | Migration-owned | Provider | Provider `privacy.dsar.workflow_engine` is registered and documented. |
|
||||
| `xrechnung` | `govoplan-xrechnung` | No module migration | Reviewed no-store rationale | Stateless validation-contract module: XRechnung persists no invoice, report, diagnostic, or handoff; the invoking Files, Procurement, or Payments workflow remains the subject-data owner. Reassess before adding a validation store. |
|
||||
|
||||
Provider search, export minimization, retention, and erasure behavior remains
|
||||
documented and tested by each owning module. This matrix verifies adoption and
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
# Full registry candidates / Vollständige Registry-Kandidaten
|
||||
|
||||
## Operator workflow (EN)
|
||||
|
||||
The canonical `release-catalog.py full-registry` command takes `--package-set`,
|
||||
`--package-lock`, `--wheelhouse`, `--webui-packages`, `--output-dir`, and a
|
||||
configured `--catalog-signing-key`. Generate the package set with
|
||||
`generate-release-package-set.py --profile full` and download its exact artifacts
|
||||
with `resolve-package-artifacts.py`; do not substitute locally rebuilt wheels.
|
||||
The candidate compares the package set with the exact developer meta-package
|
||||
pins, checks archive bytes and package metadata against the lock, and synthesizes
|
||||
every entry from its immutable tagged manifest. Native package publication and
|
||||
its CI authority remain trusted: this verifies the published artifact identity,
|
||||
not independent reproducible-build equivalence to source.
|
||||
|
||||
Pass `--selected-repository` once for each newly released repository, including
|
||||
Core when it changes. These selected units must have clean, version-aligned
|
||||
named branches whose HEAD equals the annotated local and remote release tag.
|
||||
Other full-profile packages retain their exact older annotated tags; a later
|
||||
workflow-only commit on `main` does not relabel those package contents or force
|
||||
a version bump. Every source fetch/push endpoint must match the registered
|
||||
origin, and all entries bind their source commit and annotated tag object.
|
||||
Git replacement objects, caller Git configuration, and executable-path
|
||||
redirection cannot substitute another tagged manifest tree.
|
||||
|
||||
The fixed existing website catalog and keyring are authenticated before signing.
|
||||
An older catalog without a signed keyring hash can be migrated only through this
|
||||
complete rebuild, only when its signature verifies and its entire keyring
|
||||
exactly matches the configured known signers. No old entries or artifact hashes
|
||||
are reused. The new catalog signs the exact unchanged website keyring hash;
|
||||
key rotation remains a separate reviewed operation. Selective candidates still
|
||||
reject unpinned base keyrings. New candidate directories are private and
|
||||
exclusive: a retry must choose a new directory, not overwrite a reviewed one.
|
||||
|
||||
Run these commands on the trusted host, with an operator-private source workspace
|
||||
and artifact directory. `RELEASE_PYTHON` must select its private environment and
|
||||
`RELEASE_NPM` an absolute npm executable with a trusted sibling Node 22 binary.
|
||||
In Flatpak, execute host commands through `flatpak-spawn --host`; sandbox and
|
||||
host UID mappings are not interchangeable. Do not weaken trust gates or change
|
||||
system-wide permissions.
|
||||
|
||||
```sh
|
||||
# These paths identify previously prepared private operator resources.
|
||||
RELEASE_WORKSPACE=/path/to/private/workspace
|
||||
RELEASE_PYTHON="$RELEASE_WORKSPACE/govoplan/.host-venv/bin/python"
|
||||
RELEASE_NPM=/path/to/private/node22/bin/npm
|
||||
ARTIFACT_ROOT=/path/to/private/artifacts
|
||||
RELEASE_CANDIDATE=/path/to/private/new-candidate
|
||||
RELEASE_VERSION=0.1.45
|
||||
RELEASE_TOOLS="$RELEASE_WORKSPACE/govoplan/tools/release"
|
||||
|
||||
umask 077
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/generate-release-package-set.py" \
|
||||
--version "$RELEASE_VERSION" --profile full \
|
||||
--workspace "$RELEASE_WORKSPACE" --output "$ARTIFACT_ROOT/packages.json"
|
||||
PATH="$(dirname "$RELEASE_NPM"):/usr/bin:/bin" \
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/resolve-package-artifacts.py" \
|
||||
--package-set "$ARTIFACT_ROOT/packages.json" \
|
||||
--wheelhouse "$ARTIFACT_ROOT/wheels" \
|
||||
--webui-packages "$ARTIFACT_ROOT/webui" \
|
||||
--lock-output "$ARTIFACT_ROOT/artifacts.lock.json" \
|
||||
--python "$RELEASE_PYTHON" --npm "$RELEASE_NPM"
|
||||
|
||||
# Repeat --selected-repository for EVERY newly released unit, not just Core.
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" full-registry \
|
||||
--workspace-root "$RELEASE_WORKSPACE" \
|
||||
--package-set "$ARTIFACT_ROOT/packages.json" \
|
||||
--package-lock "$ARTIFACT_ROOT/artifacts.lock.json" \
|
||||
--wheelhouse "$ARTIFACT_ROOT/wheels" --webui-packages "$ARTIFACT_ROOT/webui" \
|
||||
--output-dir "$RELEASE_CANDIDATE" --selected-repository govoplan-core \
|
||||
--catalog-signing-key known-key=/path/to/private/known-key.pem --json
|
||||
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" publish-candidate \
|
||||
--workspace-root "$RELEASE_WORKSPACE" --candidate-dir "$RELEASE_CANDIDATE" \
|
||||
--channel stable --npm "$RELEASE_NPM" --build-web \
|
||||
--commit --tag --push --tag-name "catalog-v$RELEASE_VERSION" --json
|
||||
```
|
||||
|
||||
The last command is a strict non-mutating preview because `--apply` is absent.
|
||||
Review its output, then repeat it with `--apply` to publish. The website's locked
|
||||
build dependencies must already be installed before `--build-web`. The publisher
|
||||
sanitizes the build and Git environments and pushes the verified immutable
|
||||
website commit/tag. Source tag publication and registry package availability
|
||||
must be complete before candidate generation.
|
||||
|
||||
Source tags, registry packages, and a signed module catalog do not imply that a
|
||||
new runtime distribution exists. While runtime images are held, leave the Meta
|
||||
Gitea runtime Release held too: a normal source-only Release can replace Gitea's
|
||||
`releases/latest` discovery result despite having no deployment assets. The
|
||||
deployer still requires an explicit signed manifest, digest, and trusted
|
||||
keyring; it does not deploy a tag or module catalog directly.
|
||||
|
||||
## Betriebsablauf (DE)
|
||||
|
||||
`release-catalog.py full-registry` übernimmt den vollständigen Paketbestand,
|
||||
die Registry-Sperrdatei, das Wheel-Verzeichnis, die WebUI-Archive und den
|
||||
konfigurierten Signaturschlüssel. Zuerst mit
|
||||
`generate-release-package-set.py --profile full` die exakten Meta-Paketversionen
|
||||
ermitteln und mit `resolve-package-artifacts.py` die veröffentlichten Artefakte
|
||||
herunterladen. Lokal neu gebaute Wheels sind kein Ersatz. Der Kandidat prüft
|
||||
Paketidentitäten, Dateigrößen und Hashes und erzeugt alle Einträge aus den
|
||||
unveränderlichen getaggten Manifesten. Die Registry und ihre veröffentlichende
|
||||
CI bleiben eine Vertrauensgrundlage; dies ist kein unabhängiger Nachweis eines
|
||||
reproduzierbaren Builds aus dem Quellcode.
|
||||
|
||||
Jedes neu veröffentlichte Repository wird mit `--selected-repository`
|
||||
angegeben. Nur diese Auswahl muss mit dem sauberen, versionsgleichen HEAD eines
|
||||
benannten Branches und dem annotierten lokalen und entfernten Tag übereinstimmen.
|
||||
Unveränderte Pakete behalten ihren ursprünglichen Tag, auch wenn auf `main`
|
||||
bereits eine spätere Workflow-Korrektur liegt. Alle Quelladressen müssen dem
|
||||
registrierten Ursprung entsprechen; Commit und annotiertes Tag-Objekt werden
|
||||
für jeden Eintrag gebunden. Git-Ersetzungsobjekte oder fremde Git-Konfiguration
|
||||
können dabei keinen anderen Manifestbaum unterschieben.
|
||||
|
||||
Vor dem Signieren werden der bestehende Website-Katalog und sein Schlüsselbund
|
||||
geprüft. Ein alter Katalog ohne signierten Schlüsselbund-Hash darf ausschließlich
|
||||
durch diesen vollständigen Neuaufbau migriert werden: Seine Signatur muss gültig
|
||||
sein und der gesamte Schlüsselbund exakt den konfigurierten bekannten Signierern
|
||||
entsprechen. Alte Einträge oder Artefakt-Hashes werden nicht übernommen. Der neue
|
||||
Katalog bindet den unveränderten Schlüsselbund-Hash; ein Schlüsselwechsel bleibt
|
||||
ein eigener geprüfter Vorgang. Selektive Kandidaten verlangen weiterhin einen
|
||||
bereits gebundenen Schlüsselbund. Kandidaten werden nur in neuen privaten
|
||||
Verzeichnissen erzeugt und niemals überschrieben.
|
||||
|
||||
Das obige Befehlsbeispiel wird auf dem vertrauenswürdigen Host ausgeführt. Dafür
|
||||
die private Python-Umgebung und einen absoluten `--npm`-Pfad zu Node 22 verwenden;
|
||||
unter Flatpak die Host-Werkzeuge über `flatpak-spawn --host` aufrufen. Die
|
||||
gesperrten Website-Build-Abhängigkeiten vorher installieren. Keine
|
||||
Vertrauensprüfung umgehen und keine globalen Rechte ändern. Die Artefaktordner
|
||||
müssen privat und bei der Auflösung leer sein. Vor der Kandidatenerzeugung
|
||||
müssen Quell-Tags und Registry-Pakete vollständig veröffentlicht sein.
|
||||
|
||||
Die Veröffentlichung zunächst mit `publish-candidate --commit --tag --push
|
||||
--build-web` ohne `--apply` prüfen und erst nach Prüfung mit `--apply` ausführen.
|
||||
Solange Laufzeit-Images zurückgestellt sind, bleibt auch das Meta-Gitea-Runtime-
|
||||
Release zurückgestellt: Ein reines Quellcode-Release könnte sonst als neuestes
|
||||
Release erscheinen. Eine Installation benötigt weiterhin ein signiertes
|
||||
Laufzeitmanifest, dessen Digest und einen explizit vertrauenswürdigen Schlüsselbund.
|
||||
@@ -100,6 +100,7 @@ The private installation directory contains:
|
||||
| `plan.json` | Latest desired-state diff and readiness findings |
|
||||
| `receipt.json` | Last successfully applied immutable identities |
|
||||
| `infrastructure-capabilities.json` | Deterministic non-secret capability states, endpoint metadata, secret references, consumers, and resumable post-install tasks |
|
||||
| `infrastructure-dependency-inventory.json` | Owner-only, short-lived Ops evidence of actual module-owned configuration and data that depend on infrastructure capabilities |
|
||||
| `distribution-manifest.json` | Canonical signed runtime/image selection adopted by the installer |
|
||||
| `distribution-keyring.json` | Explicitly installed public trust anchor for runtime releases |
|
||||
| `backup-evidence.json` | Signed provider-neutral coordinated backup and isolated-restore receipt |
|
||||
@@ -123,6 +124,16 @@ environment or initiating an implicit object migration. Invalid receipts fail
|
||||
closed, while a deployment without a mounted receipt continues to run but
|
||||
cannot apply receipt-bound configuration fragments.
|
||||
|
||||
Enabled modules may also register a Core infrastructure-dependency provider.
|
||||
The authorized Ops endpoint aggregates those providers without importing their
|
||||
tables. Mail reports persisted SMTP endpoints, credential-binding counts and
|
||||
legacy profiles; Files reports its runtime storage binding plus persisted blob
|
||||
counts and byte totals grouped by backend. Ops reports the active PostgreSQL,
|
||||
Redis coordination, ingress, and load-balancing runtime bindings. Provider output contains stable
|
||||
references, bounded numeric metrics and required migration actions, never
|
||||
credentials, endpoint secrets, tenant identifiers or file keys. A provider
|
||||
failure makes the entire inventory incomplete.
|
||||
|
||||
Build the same dependency-free tool as one downloadable artifact:
|
||||
|
||||
```sh
|
||||
@@ -350,6 +361,15 @@ and WebUI API proxy traffic across API replicas. The WebUI and API services do
|
||||
not publish host ports. HAProxy has no Docker socket and discovers only the
|
||||
bounded replica slots rendered into `load-balancer.cfg`.
|
||||
|
||||
New installation specifications default to HAProxy `3.2.23-alpine`, pinned to
|
||||
registry index `sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
|
||||
This patch remains on HAProxy 3.2 LTS and Alpine 3.24.1. Loading an existing
|
||||
specification preserves its explicit image; it does not perform an upgrade.
|
||||
The [runtime remediation evidence](../security/RUNTIME_IMAGE_REMEDIATION_2026-09-08.md)
|
||||
records both architecture scans and the pending binary/configuration, runtime,
|
||||
inventory and final-image checks. The source default is not a release approval:
|
||||
runtime publication remains held in Meta #52.
|
||||
|
||||
Replica counts are desired state:
|
||||
|
||||
```sh
|
||||
@@ -424,10 +444,16 @@ infrastructure capability projections.
|
||||
|
||||
- Adding a managed component creates its service and persistent volume.
|
||||
- Removing a component removes its service container on apply.
|
||||
- Replacing or removing a capability adds a review action that names the prior
|
||||
and desired state/source plus declared module consumers. This does not claim
|
||||
that the deployer can inspect module-owned database configuration; the
|
||||
operator must review that inventory before apply.
|
||||
- Reconfiguring, replacing or removing a capability adds a review action that
|
||||
names the prior and desired state/source, declared consumers, actual
|
||||
provider-reported dependency records and each required migration action.
|
||||
- The deployer blocks that change when provider inventory is missing,
|
||||
incomplete, more than five minutes old, from another installation, timestamped
|
||||
in the future, or does not cover every impacted capability. It never treats
|
||||
installer-declared consumers as proof that persisted module state is absent.
|
||||
- The inventory reports impact; it does not migrate or delete module-owned
|
||||
configuration or data. Complete the reported preparation and collect again
|
||||
immediately before apply.
|
||||
- Volumes are retained by default; deleting data requires a separate,
|
||||
deliberately destructive workflow.
|
||||
- Existing generated credentials are retained unless an explicit future rotate
|
||||
@@ -455,6 +481,29 @@ dedicated ConfigMap and read-only file mount. Ops validates the bounded schema
|
||||
before displaying configured, externally supplied, available-unconfigured, or
|
||||
unavailable states and any pending post-install tasks.
|
||||
|
||||
Collect current dependency evidence with an API key whose principal has one of
|
||||
the Ops read scopes:
|
||||
|
||||
```sh
|
||||
export GOVOPLAN_OPS_API_KEY='<short-lived operator API key>'
|
||||
python3 govoplan-deploy.pyz collect-infrastructure-inventory \
|
||||
--directory /srv/govoplan/example
|
||||
python3 govoplan-deploy.pyz doctor \
|
||||
--directory /srv/govoplan/example
|
||||
python3 govoplan-deploy.pyz apply \
|
||||
--directory /srv/govoplan/example
|
||||
unset GOVOPLAN_OPS_API_KEY
|
||||
```
|
||||
|
||||
The command defaults to
|
||||
`<public-url>/api/v1/ops/infrastructure/dependencies`; `--ops-url` may select an
|
||||
explicit HTTPS endpoint (plain HTTP is accepted only on loopback). `apply`
|
||||
refreshes the inventory automatically when `GOVOPLAN_OPS_API_KEY` is present.
|
||||
Otherwise an already collected, current inventory may be used. The API key is
|
||||
sent only as `X-API-Key`, is never written to the bundle, and the inventory file
|
||||
is owner-readable only. Because it contains operational references and counts,
|
||||
handle it as private evidence even though it contains no secret material.
|
||||
|
||||
Every apply operation is journalled before image pulls or runtime mutation. A
|
||||
failure before migration may restore a verified previous bundle. Once migration
|
||||
starts, recovery is forward-only unless an independently verified database
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# Integrity and performance source release — September 2026
|
||||
|
||||
Coordinated tracking: [Core #298](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/298).
|
||||
This source publication advances only the affected packages; it is not a signed
|
||||
catalog publication, runtime-image release, or remote production rollout.
|
||||
|
||||
## Package set
|
||||
|
||||
| Package | Version |
|
||||
| --- | --- |
|
||||
| Core / developer meta-package | 0.1.46 |
|
||||
| Addresses | 0.1.23 |
|
||||
| Calendar | 0.1.24 |
|
||||
| Campaign | 0.1.29 |
|
||||
| Cases | 0.1.25 |
|
||||
| Committee | 0.1.22 |
|
||||
| Connectors | 0.1.27 |
|
||||
| Dataflow | 0.1.25 |
|
||||
| Datasources | 0.1.26 |
|
||||
| Files | 0.1.27 |
|
||||
| Forms Runtime | 0.1.22 |
|
||||
| IDM | 0.1.26 |
|
||||
| Mail | 0.1.28 |
|
||||
| Reporting | 0.1.22 |
|
||||
| Tickets | 0.1.23 |
|
||||
|
||||
Consumers of new Core helpers require Core 0.1.46 or later. Dataflow and
|
||||
Datasources also require the matching Core WebUI contract. The release manifests,
|
||||
immutable Git lock, and developer package describe this coordinated composition.
|
||||
Unchanged packages retain their independent versions.
|
||||
|
||||
## Database and data integrity
|
||||
|
||||
The reviewed additive heads are Connectors `d2a4c6e8f0b1`, Datasources
|
||||
`e2b8d4a0f6c3`, Files `a2b3c4d5e701`, and Mail `b5d6e7f8091a`.
|
||||
Back up the target deployment and rehearse its normal upgrade before rollout.
|
||||
Never downgrade away retained CSV originals without a separate recovery plan.
|
||||
Files preserves historical duplicate copies; Mail leaves legacy maildrop identity
|
||||
unset rather than guessing an account. A schema upgrade does not authorize POP3
|
||||
retrieval/reconciliation, provider deletion, or message resending.
|
||||
|
||||
Development startup may automatically apply pending migrations after a watched
|
||||
source change. Therefore, inspect actual Alembic heads and schema before assuming
|
||||
a live development database is still at its pre-change state. A backup taken
|
||||
after such an upgrade is a current-state recovery copy, not a pre-upgrade backup.
|
||||
|
||||
On 8 September, the local PostgreSQL development database already contained all
|
||||
four heads. Schema and constraint inspection passed; recomputing the Files
|
||||
identity backfill checked 7,385 rows with zero mismatches. A private current-state
|
||||
database backup was created. Per-instance backup paths and row contents are not
|
||||
published in this repository. The implementation's initial receipt claiming no
|
||||
live migration occurred was incorrect: watched development-server restarts had
|
||||
applied the migration files automatically. Restoring the backup into an isolated
|
||||
PostgreSQL 16 cluster and running the normal upgrade preserved all 281 public
|
||||
tables, 142,287 rows, and migration heads exactly. Both Datasources PostgreSQL
|
||||
concurrency regressions passed. The test-only cluster was then stopped.
|
||||
|
||||
Release preparation additionally corrected Alembic's ConfigParser handling of
|
||||
percent-escaped connection URLs, preserving the exact database URL. The static
|
||||
migration auditor now recognizes the existing reviewed development-wrapper
|
||||
aliases and peer filenames without executing migration code; historical
|
||||
migrations are unchanged.
|
||||
|
||||
## Verification and boundaries
|
||||
|
||||
The implementation passed the required focused workspace gate, including 63
|
||||
frontend build configurations and 230 browser conformance tests. Owning-module
|
||||
regressions cover exact import/rollback evidence, authorization-before-pagination,
|
||||
bounded recurrence and response processing, collision-safe attachment naming,
|
||||
and stale asynchronous UI completion. English/German feature documentation stays
|
||||
in each owning module; Core documents shared integrity contracts.
|
||||
|
||||
Mock-provider tests and local work-count measurements do not establish production
|
||||
throughput or provider race behavior. Real S3/SMB/Seafile and POP3 acceptance,
|
||||
representative load testing, and deployment authentication checks remain separate
|
||||
operational validation. Source tags trigger package workflows; a pushed source
|
||||
tag alone does not prove a registry artifact or signed catalog is published.
|
||||
|
||||
## Deutsch
|
||||
|
||||
Dieses koordinierte Quellrelease veröffentlicht nur die betroffenen Pakete.
|
||||
Produktions-Images, signierter Modulkatalog und entfernte Produktivinstanzen
|
||||
werden dadurch nicht ausgerollt. Die vier Migrationen bewahren bestehende Daten;
|
||||
historische POP3-Zuordnungen werden nicht geraten. POP3 ist ein eigener
|
||||
Import-Arbeitsablauf innerhalb von **Mail**, kein separat installierbares Modul.
|
||||
|
||||
Der Entwicklungsserver kann Migrationen beim automatischen Neustart nach einer
|
||||
Quelländerung bereits anwenden. Tatsächliche Schema-Stände prüfen; eine danach
|
||||
erstellte Sicherung enthält den aktuellen Stand und ist keine Sicherung vor dem
|
||||
Upgrade. Externe Transportaktionen, erneutes Versenden und Löschungen werden
|
||||
durch die Migration oder Quellveröffentlichung nicht ausgelöst.
|
||||
@@ -86,6 +86,16 @@ contract. The coordinated release synchronizes `peerDependencies` and
|
||||
tag, then synchronizes each lockfile root from the final package metadata. A
|
||||
distinct root package remains independent.
|
||||
|
||||
Every module referenced by Core's Git-based `package.release.json` must expose
|
||||
its WebUI identity at the repository root, including matching peer requirements
|
||||
and `webui/`-prefixed entry exports (also CSS subpaths). npm resolves Git
|
||||
dependencies from the repository root, while the native-package workflow packs
|
||||
`webui/`; success in one path does not verify the other. Run
|
||||
`python tools/checks/check-webui-package-facades.py` after changing either
|
||||
manifest or the release composition. The focused gate also runs this check.
|
||||
Adding or correcting a facade in an already published repository requires a
|
||||
new patch tag; never repair an existing immutable tag in place.
|
||||
|
||||
It builds one wheel and, where applicable, one npm tarball. The workflow records
|
||||
the source tag, source commit, filename, size, and SHA-256 in
|
||||
`package-artifacts.json` before publishing. Gitea rejects a second upload of the
|
||||
@@ -179,8 +189,13 @@ than invoking `pip`, `npm`, or Git on the target host.
|
||||
|
||||
## Public module directory
|
||||
|
||||
`tools/release/publish-release-catalog.sh` resolves the selected package set and
|
||||
registry lock before it creates a catalog. Catalog entries are synthesized from
|
||||
For an operator-reviewed full publication, use
|
||||
`tools/release/release-catalog.py full-registry` followed by the same tool's
|
||||
`publish-candidate` command. Resolve the package set and registry lock first;
|
||||
the older direct-write shell wrapper is not the strict candidate publication
|
||||
path. See [Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
|
||||
for the private host runtime, exact artifact checks, and legacy keyring transition.
|
||||
Catalog entries are synthesized from
|
||||
the exact tagged module manifests, never from a hand-maintained module list or
|
||||
the current workspace. Each entry binds its Python wheel and optional WebUI
|
||||
tarball to the registry URL, filename, size, SHA-256, package identity, source
|
||||
@@ -248,11 +263,198 @@ python tools/release/generate-developer-meta-package.py
|
||||
python tools/release/generate-developer-meta-package.py --check
|
||||
```
|
||||
|
||||
The direct generator is a development synchronization tool, not a receipt-gated
|
||||
release executor. For release preparation, use the guarded out-of-run stage below.
|
||||
|
||||
`push-release-tag.sh` performs this synchronization before release commits and
|
||||
tags. The meta-package is for editable/developer setup and composition tests. It
|
||||
does not enable modules, apply migrations, provision services, or establish
|
||||
backup and recovery evidence.
|
||||
|
||||
### Shared source-tag contract and Meta composition
|
||||
|
||||
The shared version collector names Meta's real
|
||||
`packages/govoplan-meta/pyproject.toml` separately from root `pyproject.toml`.
|
||||
Only the registered `govoplan` system/meta repository with nested project name
|
||||
`govoplan` receives this contract. Missing, unknown, or misidentified metadata
|
||||
does not become a versionless exception. Version alignment compares the complete
|
||||
nested file with the canonical operator-tool generator output: its version must
|
||||
match Core, and dependencies and `full` composition must match the reviewed
|
||||
requirements and workspace package versions. Validation never executes a
|
||||
generator from a selected checkout. The shared trusted manifest checker can
|
||||
load reviewed application manifests; these checks are not a code sandbox.
|
||||
|
||||
Meta's complete generated file is recognized by shared version-mutation discovery,
|
||||
but the generic durable version executor deliberately cannot write it. A durable
|
||||
run freezes the release console's own Meta checkout as trusted runtime code;
|
||||
changing it in place would invalidate that run. The planner therefore places Meta
|
||||
after Core and exposes only non-executable support preparation/publication steps,
|
||||
not misleading automatic Meta version, commit, tag, or push actions. A missing or
|
||||
different Core target produces an actionable preparation prerequisite.
|
||||
|
||||
Prepare Core and the intended module inputs first, commit their reviewed state,
|
||||
then stop active durable runs for the target workspace. Use trusted operator tools
|
||||
against a separate registered, private source checkout, never the running operator
|
||||
Meta directory. Preview outside any selected source checkout, for example:
|
||||
|
||||
```sh
|
||||
python tools/release/prepare-developer-meta-package.py \
|
||||
--workspace /private/release-workspace --target-version X.Y.Z \
|
||||
> /private/operator/meta-preview.json
|
||||
python tools/release/prepare-developer-meta-package.py \
|
||||
--workspace /private/release-workspace --target-version X.Y.Z \
|
||||
--receipt /private/operator/meta-preview.json --apply --confirm-out-of-run
|
||||
```
|
||||
|
||||
The explicit confirmation attests that no durable run is active for that target
|
||||
workspace; the helper does not discover or stop other processes. Preview/apply
|
||||
requires registered clean main sources, matching origins and live-main ancestry,
|
||||
Core already aligned at the target, the exact nested identity, and no existing or
|
||||
unverifiable target Meta tag. Frozen receipts cover source HEADs/filesystem
|
||||
identities, release requirements, every discovered registered full-composition
|
||||
pyproject, the trusted generator snapshot, and the resulting full-file hash.
|
||||
Inputs are limited to 128 selected files, 2 MiB per file and 16 MiB aggregate.
|
||||
The canonical generator renders copied bounded data in a temporary directory;
|
||||
no generator from the selected checkout executes. Changed receipts block before
|
||||
the file effect. Apply writes only `packages/govoplan-meta/pyproject.toml`, then
|
||||
rechecks the other sources and exact output. A write or post-check failure that
|
||||
may have changed the file reports `needs-reconciliation` and leaves that bounded
|
||||
delta for explicit review; it never retries, rolls back, commits or publishes.
|
||||
|
||||
Review the complete generated composition and manually commit the resulting file.
|
||||
Complete matching Core publication before guarded Meta source tagging/publication,
|
||||
then start a fresh durable run from reviewed, clean, published operator tooling.
|
||||
Hot self-updating durable Meta release execution remains explicitly unsupported;
|
||||
this out-of-run preparation is the existing developer-meta support contract.
|
||||
|
||||
For every `tag_repositories` batch, strict checks apply to every selected
|
||||
repository before any tag creation, fetch, or push: registered checkout and
|
||||
origin/push URL, a clean `main` tracking `origin/main`, live remote-main ancestry,
|
||||
exact frozen HEADs, and immutable annotated local/remote tag objects. Git metadata
|
||||
must remain inside the operator's trusted workspace. Missing local knowledge of
|
||||
live remote main is a blocker; fetch and review it separately. Unknown repositories
|
||||
and non-registered remote aliases fail closed. If selected, Meta runs last.
|
||||
For Meta only, the matching annotated Core release must exist before its effect; Core may be
|
||||
an earlier selected repository, or an already tagged dependency. Publication
|
||||
requires that Core's exact tag and main commit are already remote.
|
||||
|
||||
Non-Meta selections do not acquire Meta's composition or Core-tag prerequisite.
|
||||
Local module-candidate tags still work before Core's final release lock or tag.
|
||||
The existing Core WebUI bundle gate still applies to module publication and
|
||||
batches selecting Core: relevant Core release-package and release-lock inputs
|
||||
must be operator-owned regular files, at most 16 MiB each, and their identities
|
||||
and content hashes are frozen before preflight and rechecked before every effect.
|
||||
When Core is unselected, this does not require its checkout to be clean or tagged;
|
||||
reviewed pending composition inputs retain their previous meaning. Backend-only
|
||||
selections never read irrelevant Core WebUI files.
|
||||
|
||||
Before even read-only Git commands, source ancestry must be owned by root or the
|
||||
current operator and must not be group/world writable. A sticky shared ancestor
|
||||
such as `/tmp` is permitted only above an owned, protected child; the workspace
|
||||
and checkouts receive no writable-directory exception. The current operator must
|
||||
own source inputs and actual Git/worktree/common metadata, which must be regular
|
||||
files/directories, non-symlinked, and non-writable by other users. Metadata walks
|
||||
are bounded to 500,000 entries and 128 levels, and tracked inputs to 100,000 paths
|
||||
and 16 MiB of listing text. Read-only Git targets, object alternates/grafts and
|
||||
hidden/sparse/unmerged index entries are blocked. Frozen receipts include actual
|
||||
checkout/Git directory paths, devices, inodes, owners and modes, so replacing Git
|
||||
metadata with the same HEAD is still detected. All selected version/composition
|
||||
inputs must be tracked, including root and WebUI package/lock metadata, discovered
|
||||
module manifests and package initializers, and Meta's nested package and release
|
||||
requirements; ignored working files cannot supply declarations absent from a tag.
|
||||
No chmod, ownership repair or
|
||||
global Git trust change is performed. A shared writable workspace must first be
|
||||
recreated or reviewed in the operator's protected release area by an explicitly
|
||||
authorized preparation workflow.
|
||||
|
||||
Preview is read-only. Local-tag mode creates only pinned annotated tags (or
|
||||
retrieves an identical published annotation); it does not publish main or tags.
|
||||
Publish mode atomically pushes the frozen main commit and annotation object,
|
||||
without force, retagging, fallback, or automatic retry. The complete source
|
||||
receipt is rechecked before every effect and afterward; remote main and the
|
||||
exact annotated tag must both match, not merely the Git exit status. Changes
|
||||
after preflight stop the remaining batch. Atomicity is per repository, not
|
||||
across repositories: earlier successful publications and a newly created local
|
||||
tag can remain after a later failure. Inspect reported receipts and obtain a new
|
||||
review before retrying; do not move immutable tags.
|
||||
|
||||
Whole-batch revalidation deliberately repeats source and live-remote checks around
|
||||
each repository effect; the number of checks can grow quadratically with batch
|
||||
size. Plan release time accordingly rather than bypassing trust checks. The
|
||||
shared internal preflight is read-only and exposes no legacy mutation path.
|
||||
The fixture suite covers Meta and non-Meta preview/local-tag/
|
||||
publication using temporary local bare remotes, including stale compositions,
|
||||
unsafe origins, divergent branches, damaged tag identity, changed receipts and
|
||||
false publication success. This is local tooling evidence, not a real release
|
||||
publication or production permission check.
|
||||
|
||||
Deutsch: Die gemeinsamen Helfer erkennen ausschließlich das registrierte
|
||||
Meta-Repository mit dem echten Paket `packages/govoplan-meta/pyproject.toml`
|
||||
(Projektname `govoplan`). Version und vollständige Zusammensetzung müssen dem
|
||||
kanonischen Generator, Core und den geprüften Anforderungen entsprechen; der
|
||||
Generator stammt niemals aus dem ausgewählten Checkout. Der gemeinsame
|
||||
Manifestprüfer kann geprüften Anwendungscode laden und ist keine Sandbox.
|
||||
Die gemeinsame Änderungsplanung erkennt die vollständig generierte Paketdatei,
|
||||
aber der dauerhafte Versionsausführer darf Meta nicht selbst verändern: sein
|
||||
eingefrorener Lauf bindet den Meta-Checkout als vertrauenswürdigen Programmstand.
|
||||
Meta erscheint deshalb nach Core ausschließlich mit nicht automatisch ausführbaren
|
||||
Vorbereitungs-/Veröffentlichungsschritten. Zuerst Core und Modulquellen vorbereiten
|
||||
und geprüft committen; bei abweichender Core-Zielversion nennt der Plan diese
|
||||
Voraussetzung ausdrücklich. Aktive dauerhafte Läufe des Ziel-Workspaces beenden.
|
||||
Mit `prepare-developer-meta-package.py` zunächst eine Vorschau außerhalb der
|
||||
Quell-Checkouts speichern, dann deren JSON über `--receipt` zusammen mit `--apply`
|
||||
und `--confirm-out-of-run` bestätigen. Das Ziel muss ein separater registrierter
|
||||
privater Checkout sein, niemals das laufende Operator-Meta. Die Bestätigung ist
|
||||
eine Betreibererklärung; der Helfer sucht oder beendet keine fremden Prozesse.
|
||||
Quell-HEADs, Pfadidentitäten, Anforderungen, alle registrierten vollständigen
|
||||
Paket-Eingaben, der vertrauenswürdige Generator und der vollständige Ausgabehash
|
||||
werden eingefroren. Es gelten höchstens 128 Quelldateien, 2 MiB je Datei und
|
||||
16 MiB insgesamt. Core muss bereits vollständig zur Zielversion passen; vorhandene
|
||||
oder nicht verifizierbare Meta-Zieltags sperren die Vorbereitung. Geänderte
|
||||
Nachweise stoppen vor dem Schreiben. Ausschließlich die verschachtelte Paketdatei
|
||||
wird vollständig generiert und danach geprüft; ein Fehler nach dem Schreiben
|
||||
meldet `needs-reconciliation` und erfordert die manuelle Prüfung dieser begrenzten
|
||||
Änderung, ohne automatisches Zurücksetzen. Kein automatischer
|
||||
Commit, Push oder Wiederholungsversuch findet statt. Zusammensetzung prüfen,
|
||||
manuell committen, Core zuerst veröffentlichen, dann die geschützte Meta-Tag-Route
|
||||
verwenden und einen neuen dauerhaften Lauf starten. Eine Selbstaktualisierung
|
||||
des aktiven dauerhaften Meta-Laufs bleibt ausdrücklich nicht unterstützt.
|
||||
Für jeden Tag-Stapel, auch ohne Meta, gelten
|
||||
Vertrauens-, Origin-, saubere Main- und Live-Abstammungsprüfungen für die gesamte
|
||||
Auswahl vor jeder Änderung. Unbekannte Repositories und nicht registrierte
|
||||
Remote-Aliase sind gesperrt. Nur bei ausgewähltem Meta gelten zusätzlich dessen
|
||||
Zusammensetzungsprüfung und der passende annotierte Core-Tag als Voraussetzung;
|
||||
Meta folgt zuletzt. Für Metas Veröffentlichung müssen Core-Tag und Main-Commit
|
||||
bereits auf dem Remote vorliegen. Lokale Modul-Kandidatentags bleiben vor Cores
|
||||
abschließendem Release-Lock und Tag möglich. Die vorhandene Core-WebUI-Bundleprüfung
|
||||
bleibt bei Modulveröffentlichung und Core-Auswahl erhalten. Relevante Core-Paket-
|
||||
und Lockdateien müssen eigene reguläre Dateien mit höchstens je 16 MiB sein;
|
||||
Identität und Inhaltshash werden eingefroren und vor jeder Aktion erneut geprüft.
|
||||
Nicht ausgewähltes Core benötigt dafür weder einen sauberen Checkout noch einen
|
||||
Tag. Reine Backend-Auswahlen lesen keine irrelevanten Core-WebUI-Dateien.
|
||||
Vor Git-Aufrufen werden Eigentümer, Schreibrechte, sichere
|
||||
Pfadabstammung und echte Git-/Worktree-Metadaten geprüft; veränderbare gemeinsame
|
||||
Verzeichnisse, fremde Eigentümer, Alternates, Grafts und versteckte Indexeinträge
|
||||
sind gesperrt. Ein Sticky-Bit-Vorfahr wie `/tmp` ist nur oberhalb eines eigenen
|
||||
geschützten Unterverzeichnisses zulässig. Es erfolgen weder Rechtereparaturen
|
||||
noch globale Git-Vertrauensänderungen. Ausgewählte Versions- und Zusammensetzungs-
|
||||
dateien müssen versioniert sein: Paket-/Lockdateien, Modulmanifeste und
|
||||
Paketinitialisierer sowie Metas verschachteltes Paket und Release-Anforderungen.
|
||||
Ignorierte Arbeitsdateien dürfen keine vom Tag abweichenden Angaben liefern.
|
||||
Die Vorschau schreibt nichts, lokale Tags veröffentlichen nichts,
|
||||
und die Veröffentlichung überträgt Main und den exakten annotierten Tag atomar
|
||||
je Repository. Unmittelbar vor und nach den Aktionen werden die eingefrorenen
|
||||
Quellnachweise erneut geprüft, einschließlich entferntem Main und Tag-Objekt.
|
||||
Bei Änderungen oder Fehlern stoppt der Rest des Stapels ohne automatischen
|
||||
Wiederholungsversuch. Frühere Veröffentlichungen und neu erzeugte lokale Tags
|
||||
können bestehen bleiben: vor einem neuen Versuch Nachweise prüfen und erneut
|
||||
freigeben, niemals unveränderliche Tags verschieben. Die vollständigen Quell- und
|
||||
Live-Remote-Prüfungen werden um jede Aktion wiederholt; bei großen Stapeln kann
|
||||
deren Anzahl quadratisch wachsen. Diese konservativen Prüfkosten gehören zur
|
||||
Release-Planung. Der interne Vorprüfer ist ausschließlich lesend und besitzt
|
||||
keinen alten Änderungspfad. Tests für Auswahlen mit und ohne Meta verwenden
|
||||
nur temporäre lokale Remotes und ersetzen keine echte Veröffentlichungsprüfung.
|
||||
|
||||
If the tag-triggered developer meta-package job fails before publication, rerun
|
||||
`publish-developer-meta-package.yml` with the existing protected version. The
|
||||
manual path validates that tag against `main`, checks out its exact commit, and
|
||||
|
||||
@@ -58,6 +58,45 @@ checkouts remain usable for read-only planning, but every durable executor
|
||||
fails closed there; clone the registered origins into a private workspace
|
||||
before releasing.
|
||||
|
||||
For a host with a confirmed IPv6 connection timeout, set
|
||||
`GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` only for the release-tool invocation.
|
||||
When unset, the original SSH command is preserved, including the trusted
|
||||
operator's per-host `AddressFamily` configuration (normally `any`). Explicit
|
||||
values accepted by the shared source/tag Git helper are exactly `any`, `inet`
|
||||
(IPv4 only), and `inet6` (IPv6 only). Empty, misspelled, whitespace-padded, or
|
||||
injected values fail before Git starts. The selector only adds the corresponding
|
||||
fixed SSH `AddressFamily` option: it does not change DNS, host-key verification,
|
||||
the registered remote, authentication, `BatchMode=yes`, or `ConnectTimeout=8`.
|
||||
Arbitrary `GIT_SSH_COMMAND` overrides remain ignored. For example, start a
|
||||
single local console invocation with:
|
||||
|
||||
```sh
|
||||
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet \
|
||||
./.venv/bin/python tools/release/release-console.py
|
||||
```
|
||||
|
||||
The same process-scoped setting applies to canonical source/tag readbacks and
|
||||
registry-candidate source verification. Under Flatpak, pass it explicitly to
|
||||
the host invocation with `flatpak-spawn --host /usr/bin/env
|
||||
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet ...`. It is not a global SSH setting
|
||||
and does not affect the website publisher's separate transport sanitizer, npm,
|
||||
or HTTP downloads. An IPv4-only setting cannot reach IPv6-only hosts; omit it
|
||||
or use `any` when the diagnosed restriction no longer applies.
|
||||
|
||||
Deutsch: Bei einem bestätigten IPv6-Verbindungs-Timeout kann für genau einen
|
||||
Release-Werkzeugaufruf `GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` gesetzt werden.
|
||||
Ohne diese Variable bleibt der bisherige SSH-Befehl einschließlich der
|
||||
vertrauenswürdigen Host-Konfiguration unverändert (normalerweise `any`).
|
||||
Explizit zulässig sind ausschließlich `any`, `inet` (nur IPv4) und `inet6`
|
||||
(nur IPv6). Andere oder leere Werte werden vor dem Git-Aufruf abgewiesen.
|
||||
DNS, Hostschlüsselprüfung, registrierte Quelladresse, Authentifizierung und
|
||||
Zeitlimit bleiben unverändert; frei vorgegebene SSH-Befehle bleiben gesperrt.
|
||||
Unter Flatpak die Variable ausdrücklich an den Host-Aufruf übergeben. Die
|
||||
Auswahl gilt für den gemeinsamen Git-Helfer der Quell-/Tag-Prüfungen, nicht
|
||||
für den separaten Website-Publisher, npm oder HTTP-Downloads. Sie ändert keine
|
||||
globale Konfiguration. Nach Behebung des Netzwerkproblems die Variable
|
||||
weglassen oder auf `any` setzen; IPv4-only erreicht keine IPv6-only-Ziele.
|
||||
|
||||
The runtime itself is part of the authority boundary. Durable run creation
|
||||
verifies the meta checkout, release/check tooling, repository registry, Python
|
||||
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
|
||||
@@ -221,6 +260,9 @@ Repository capabilities are frozen into each plan unit (`python-package`,
|
||||
`core-release-bundle`, and the universal `git-source`) and determine which
|
||||
steps appear. Internally aligned version changes are rendered deterministically
|
||||
from recognized TOML, JSON, lockfile, manifest, and package declarations.
|
||||
The manifest may use a literal version or a top-level literal `MODULE_VERSION`;
|
||||
the latter is updated without rewriting independently versioned interfaces.
|
||||
Computed or missing version declarations fail before any metadata is written.
|
||||
Pre-existing dirty worktrees remain visible but have no commit executor; the
|
||||
console never absorbs unrelated operator changes.
|
||||
|
||||
@@ -232,6 +274,17 @@ runs a receipt-bound alignment gate before exposing any atomic branch/tag push.
|
||||
A failed step stops later steps while preserving prior receipts for explicit
|
||||
retry or reconciliation.
|
||||
|
||||
Local module candidate creation deliberately does not require those candidates
|
||||
to be resolved already in Core's release lock: their annotated tags are inputs
|
||||
to the next lock-generation step. The internal tag helper applies this ordering
|
||||
only when no Core repository is selected and remote publication is disabled.
|
||||
Module version/lock consistency, manifest validity, clean/non-behind worktrees,
|
||||
and local/remote tag immutability checks still apply. Core candidate tagging
|
||||
continues to validate its own complete bundle, and every remote-publication
|
||||
preview and execution requires the selected modules to match Core's release
|
||||
input and resolved lock. A local candidate is therefore not publication
|
||||
approval; a stale Core lock blocks publication without changing remote refs.
|
||||
|
||||
The browser likewise retains the request identifier for an uncertain
|
||||
resume/retry/reconciliation response and replays it after reload. A successful
|
||||
replay selects the returned run state. Transport and server failures retain the
|
||||
@@ -503,6 +556,15 @@ tree and requires byte-for-byte equality with those validated objects. Tags and
|
||||
remote branch updates then reference that exact commit SHA rather than the
|
||||
mutable worktree `HEAD`.
|
||||
|
||||
For a full registry-backed release, first build a fresh private candidate using
|
||||
`release-catalog.py full-registry`. Pass `--selected-repository` for newly
|
||||
released HEAD-bound units, not every unchanged package in the full profile.
|
||||
The command independently checks all full-profile registry bytes and annotated
|
||||
tag provenance, then feeds this same strict `publish-candidate` transaction.
|
||||
It does not create Gitea runtime Releases or dispatch image builds. See
|
||||
[Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
|
||||
for the complete EN/DE workflow and the narrowly scoped legacy keyring transition.
|
||||
|
||||
Published channels are expected below the public catalog base URL:
|
||||
|
||||
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
|
||||
@@ -515,8 +577,9 @@ updated catalog, and keep the published keyring healthy.
|
||||
|
||||
When a selected module exposes a WebUI package, its requested version must also
|
||||
match Core's `webui/package.release.json` input and the resolved
|
||||
`package-lock.release.json` entry. The source-tag preflight, selective plan, and
|
||||
catalog-candidate writer all enforce this composition boundary. Pins for modules
|
||||
`package-lock.release.json` entry. The source-publication preflight, selective
|
||||
plan, and catalog-candidate writer all enforce this composition boundary;
|
||||
module-only local candidate tags use the staged order described above. Pins for modules
|
||||
that are not part of the selective release remain unchanged.
|
||||
|
||||
Release integration also enforces repository and composition version alignment
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# WebUI release dependency installer retries
|
||||
|
||||
## English
|
||||
|
||||
This operational note covers
|
||||
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
|
||||
and the exit-status repair tracked in
|
||||
[Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54).
|
||||
It applies to release administrators using the legacy runtime WebUI installer;
|
||||
there are no new application settings, permissions, or end-user workflows.
|
||||
|
||||
Each retried npm install or Git clone has at most three attempts. The installer
|
||||
waits 10 seconds after the first failure and 20 seconds after the second, and
|
||||
continues immediately after success. If all attempts fail, it exits with the
|
||||
last command's nonzero status. Its `set -e` execution stops before subsequent
|
||||
installation stages; callers using `set -e` also stop before subsequent work.
|
||||
Previously, the retry helper could report success after three failures because
|
||||
it captured the status of a completed `if` statement instead of the command.
|
||||
|
||||
On exhaustion, inspect the npm or Git error and correct the reported cause
|
||||
before rerunning the installation. The temporary dependency workspace is
|
||||
removed on exit. Earlier changes to `package.json`, removal of `package-lock.json`,
|
||||
cache cleaning, and completed dependency installations are not rolled back;
|
||||
prepare a fresh disposable release workspace when a clean retry is required.
|
||||
|
||||
The repair preserves the existing retry count, backoff, cache behavior, and
|
||||
peer-resolution flags. It does not lift the runtime publication hold tracked in
|
||||
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
|
||||
Review the historical `--legacy-peer-deps` workaround separately before lifting
|
||||
that hold. Strict disposable Git-release and signed catalog verification do not
|
||||
use this installer; strict release verification must not bypass peer checks.
|
||||
See [Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md) for release context.
|
||||
|
||||
Run the isolated regression suite from the meta repository:
|
||||
|
||||
```sh
|
||||
python3 -m unittest -v tests.test_webui_release_dependency_retries
|
||||
```
|
||||
|
||||
The suite executes the actual Bash installer and a caller using `set -e`, with
|
||||
local npm, Git, Node, and sleep stubs. It covers success on attempts one, two, and
|
||||
three, final failure status, backoff, and termination at each retry call site.
|
||||
It performs no network access, real waiting, or changes to the real npm cache.
|
||||
It checks shell control flow, not package resolution or runtime publication.
|
||||
|
||||
## Deutsch
|
||||
|
||||
Dieser Betriebshinweis beschreibt
|
||||
[`install-webui-release-dependencies.sh`](../../tools/release/install-webui-release-dependencies.sh)
|
||||
und die unter [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54)
|
||||
erfasste Korrektur des Rückgabestatus. Er richtet sich an Release-Administratoren,
|
||||
die den bisherigen WebUI-Installer für Laufzeit-Releases verwenden. Neue
|
||||
Anwendungseinstellungen, Berechtigungen oder Endanwenderabläufe entstehen nicht.
|
||||
|
||||
Jede wiederholte npm-Installation und jeder Git-Klon erhält höchstens drei
|
||||
Versuche. Nach dem ersten Fehlschlag wartet der Installer 10 Sekunden, nach dem
|
||||
zweiten 20 Sekunden; nach einem Erfolg fährt er sofort fort. Scheitern alle
|
||||
Versuche, endet er mit dem letzten von null verschiedenen Rückgabestatus.
|
||||
Durch `set -e` werden nachfolgende Installationsschritte nicht ausgeführt;
|
||||
auch aufrufende Skripte mit `set -e` brechen vor ihren nächsten Schritten ab.
|
||||
Bisher konnte die Hilfsfunktion nach drei Fehlschlägen Erfolg melden, weil sie
|
||||
den Status der abgeschlossenen `if`-Anweisung statt des Befehls übernahm.
|
||||
|
||||
Prüfen Sie nach dem Abbruch die npm- oder Git-Fehlermeldung und beheben Sie deren
|
||||
Ursache vor einem erneuten Installationslauf. Das temporäre Verzeichnis für
|
||||
Abhängigkeiten wird beim Beenden entfernt. Vorherige Änderungen an `package.json`,
|
||||
das Entfernen von `package-lock.json`, die Cache-Bereinigung und abgeschlossene
|
||||
Installationen werden nicht zurückgerollt. Bereiten Sie bei Bedarf einen neuen
|
||||
temporären Release-Arbeitsbereich für einen sauberen Wiederholungslauf vor.
|
||||
|
||||
Die Korrektur erhält Anzahl und Wartezeiten der Versuche, Cache-Verhalten und
|
||||
Optionen zur Peer-Auflösung. Die Sperre für Laufzeitveröffentlichungen aus
|
||||
[Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52) bleibt bestehen.
|
||||
Der bisherige Einsatz von `--legacy-peer-deps` muss vor ihrer Aufhebung gesondert
|
||||
geprüft werden. Die strenge Git-Release-Prüfung in einem temporären Arbeitsbereich
|
||||
und die Prüfung signierter Kataloge verwenden diesen Installer nicht; die strenge
|
||||
Release-Prüfung darf Peer-Prüfungen nicht umgehen. Weitere Zusammenhänge erläutert
|
||||
[Package Registry Releases](PACKAGE_REGISTRY_RELEASES.md).
|
||||
|
||||
Führen Sie die isolierten Regressionstests im Meta-Repository aus:
|
||||
|
||||
```sh
|
||||
python3 -m unittest -v tests.test_webui_release_dependency_retries
|
||||
```
|
||||
|
||||
Die Tests führen den tatsächlichen Bash-Installer und ein aufrufendes Skript mit
|
||||
`set -e` aus. Lokale Testprogramme ersetzen npm, Git, Node und sleep. Geprüft werden
|
||||
Erfolge im ersten, zweiten und dritten Versuch, der letzte Fehlerstatus,
|
||||
Warteintervalle und der Abbruch an jeder Aufrufstelle. Es gibt keine
|
||||
Netzwerkzugriffe, echten Wartezeiten oder Änderungen am tatsächlichen npm-Cache.
|
||||
Die Tests prüfen den Shell-Ablauf, nicht die Paketauflösung oder Veröffentlichung.
|
||||
@@ -0,0 +1,80 @@
|
||||
# GovOPlaN 0.1.45 — usability, reliability and security hardening
|
||||
|
||||
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
|
||||
The exact independently versioned composition is recorded in
|
||||
`packages/govoplan-meta/pyproject.toml`; unchanged modules retain their versions.
|
||||
This source release does not by itself establish a deployed or independently
|
||||
approved production environment. Package, signed catalog and runtime publication
|
||||
results are recorded separately in the coordination issue.
|
||||
|
||||
## Runtime publication hold
|
||||
|
||||
The [runtime image audit](../security/RUNTIME_IMAGE_AUDIT_2026-09-08.md) completed
|
||||
eleven registry-only amd64 scans, but found unresolved vulnerabilities and
|
||||
inventory gaps. Runtime publication remains held separately from this source
|
||||
release. Patch-only image updates are insufficient; maintained minor-line
|
||||
changes, narrowly evidenced finding decisions, arm64/final-layer scans and
|
||||
deployment checks remain necessary. No audited candidate was automatically
|
||||
adopted and no image was executed during those scans.
|
||||
The remaining gates are tracked in
|
||||
[GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
|
||||
|
||||
## Included changes
|
||||
|
||||
- Shared page/action placement, reusable navigation grouping/editing, table and
|
||||
dialog sizing, field alignment, multi-select filters and predictable tree
|
||||
selection. Files, Mail, Search, Notifications and domain pages use the same
|
||||
contracts, with browser regression coverage.
|
||||
- Campaign draft saving and independent Mail/ZIP-policy repair, persistent and
|
||||
bulk message review, clearer delivery eligibility, bounded configurable
|
||||
synchronous delivery, guarded workerless recovery, lightweight SMTP/IMAP
|
||||
progress, reused IMAP connections and recipient-complete reporting.
|
||||
- Files archive staging/reuse, unpacking previously uploaded archives, numeric
|
||||
progress and bounded traversal. Optional native archive acceleration retains
|
||||
the same validation rules; portable fallbacks remain available.
|
||||
- Mail credential references and IMAP folder-name decoding; help topics can be
|
||||
found by area and tags without expanding every occurrence of the same topic.
|
||||
- Authentication provenance/scope and browser-cache hardening, patched rich-text
|
||||
dependencies, spreadsheet/archive/template/Dataflow resource limits, batched
|
||||
Docs/Notifications queries and safe Reporting bind names. See the
|
||||
[security/performance review](../security/SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
|
||||
for measurements, test evidence and remaining limitations.
|
||||
- A deterministic governance-journey clock fixture, fresh-process Campaign
|
||||
import coverage, and a new Cases patch aligning its root npm facade with its
|
||||
Python/WebUI package. Historical published tags are not rewritten.
|
||||
- Git-root WebUI package facades are aligned with their owning packages, with
|
||||
a cross-composition parity check. Tasks is included in default module
|
||||
discovery; it remains subject to enabled modules and normal permissions.
|
||||
|
||||
## Upgrade and verification
|
||||
|
||||
Back up the database and file storage before upgrading. Apply the complete
|
||||
selected migration graph before starting the new API/workers. This release
|
||||
includes additive repair migrations `c58a2d7e9f10` (Core ownership history) and
|
||||
`d8f1b4e7a0c3` (Access external-function mappings), plus Campaign delivery-state
|
||||
migrations. Existing business evidence is retained; a schema downgrade is not
|
||||
a substitute for a reviewed backup/restore plan. Restart API and worker
|
||||
processes together after upgrading their matching packages.
|
||||
|
||||
Updated UI consumers require Core 0.1.45 where they use its new shared contracts.
|
||||
Tenant keys that previously relied on unintended system permissions/wildcards
|
||||
must be corrected; the release does not preserve that unsafe behavior. Extremely
|
||||
sparse spreadsheets, oversized generated output and excessive archive paths
|
||||
can now fail early with a diagnostic.
|
||||
|
||||
For archive staging across multiple hosts, provide shared POSIX storage with
|
||||
working locks or sticky routing. Background delivery still needs configured
|
||||
workers; increasing the synchronous limit does not create a worker or guarantee
|
||||
delivery after a process failure. An unknown SMTP outcome must be reconciled,
|
||||
not automatically resent.
|
||||
|
||||
After deployment, manually verify login/logout and least-privilege API keys,
|
||||
Campaign Settings and independent Mail/ZIP saves, archive upload/unpack,
|
||||
recipient-complete reports, and SMTP/IMAP progress with an explicitly approved
|
||||
test mailbox. No release verification sends real campaign mail automatically.
|
||||
|
||||
Hard process isolation, forced-password-change/recovery enforcement, bounded
|
||||
Xrechnung subprocess output and large-history pagination remain separate open
|
||||
issues. This release is not a claim that all security or performance debt is
|
||||
resolved. Production-image scans and multi-host evidence must refer to the
|
||||
actual signed runtime being deployed.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Runtime image candidate audit — 8 September 2026
|
||||
|
||||
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
|
||||
Canonical remediation: [GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
|
||||
This follow-up to the [source security/performance review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
|
||||
records registry-only scans of nine proposed runtime dependencies and two
|
||||
same-minor patch candidates. **Runtime publication is held:** patch-only updates
|
||||
do not resolve the baseline. Source/package publication is a separate outcome.
|
||||
No images were executed, rebuilt, selected for CI, or published by this audit.
|
||||
|
||||
## Method and reproducible evidence
|
||||
|
||||
Official Trivy **0.74.0** was installed only in a private local task directory,
|
||||
without sudo or Docker access. Its Linux-64bit release archive matched both the
|
||||
official checksums file and GitHub release asset metadata:
|
||||
|
||||
- Archive SHA256: `2ae6fe3ee734b7fdf11335663e18c75ea12dccc76062f09f164a3b0f8be4371a`.
|
||||
- Checksums-file SHA256: `bc701c3c3ee8b9acbea2c23257e41381e3854888f51281616a6ba5dc96963821`.
|
||||
- Vulnerability database schema 2, updated `2026-09-07T19:06:01.154199452Z`,
|
||||
downloaded from `mirror.gcr.io/aquasec/trivy-db:2`.
|
||||
- Scan flags: `--image-src remote --platform linux/amd64 --scanners vuln
|
||||
--format json --no-progress --timeout 8m --max-image-size 2GB --exit-code 0`.
|
||||
Findings were counted from validated JSON; exit zero did not mean clean.
|
||||
- Existing Docker credentials were not read; no private keys or secrets were
|
||||
used. Checksums over official HTTPS metadata were verified, not independent
|
||||
Sigstore signatures. See the [official release](https://github.com/aquasecurity/trivy/releases/tag/v0.74.0)
|
||||
and [registry-only scan documentation](https://trivy.dev/docs/latest/target/container_image/).
|
||||
|
||||
Raw evidence is retained locally, not committed:
|
||||
`/home/zemion/.cache/govoplan-trivy-remote.yKZgjDOg/scan/`.
|
||||
It contains eleven `reports/*-amd64.json` reports/logs, scanner scripts,
|
||||
`patch-candidate-inspection.json`, exact successor registry indices, and
|
||||
`evidence-checksums.json`. Summary SHA256 values:
|
||||
|
||||
- `summary.json`: `f2785a731d637452ab9c0b1f5399772c0f8828a63ca83d5fa7496abdad1c757a`.
|
||||
- `patch-summary.json`: `b3fc6273fcdad98864040ccdf3477ecf379afd46e9f94444b1f2910f48c1d85b`.
|
||||
|
||||
All eleven executions succeeded without timeout/rate-limit failure. Initial
|
||||
summary fields distinguish `scan_execution_complete: true` from
|
||||
`coverage_complete: false`: Garage has no detectable package inventory.
|
||||
Checksums preserve evidence identity, not indefinite storage availability.
|
||||
|
||||
## Exact requested pins and results
|
||||
|
||||
All references below use `docker.io/`. Counts are package-vulnerability records,
|
||||
not distinct CVEs or confirmed exploitable application defects. A vulnerability
|
||||
can appear against several installed packages. Unfixed/unknown records remain.
|
||||
|
||||
| Image tag | Exact index SHA256 | Critical / High / Medium / Low / Unknown | Fixable C/H |
|
||||
| --- | --- | --- | ---: |
|
||||
| `library/python:3.12-slim-bookworm` | `782412e85d0f0984994c290652577d4018aff08145c85b262bb63dc0c7522254` | 5 / 55 / 102 / 103 / 5 | 0 |
|
||||
| `library/postgres:16-alpine` | `cf78e76683b9ca8c5733cbbdce6c9262b45b6767934dd0a95e671f9a0fc20685` | 1 / 30 / 28 / 14 / 1 | 31 |
|
||||
| `library/redis:7-alpine` | `ff02b58f971e7d7d156a1267e283fcbbeee91773b6aa36c49dac28ecfe28eadf` | 0 / 0 / 0 / 0 / 0 | 0 |
|
||||
| `nginxinc/nginx-unprivileged:1.29-alpine` | `0c79d56aee561a1d81c63f00eee5fb5fe29279560cdc55e91425133104c7fbe6` | 0 / 33 / 74 / 37 / 20 | 33 |
|
||||
| `library/haproxy:3.2.21-alpine` | `66e25cc9a8332635f4e897f7f4b1e5622c25f09f0ee23cddc6ce9bdb3a24772a` | 0 / 2 / 6 / 12 / 0 | 2 |
|
||||
| `library/caddy:2.10.2-alpine` | `4c6e91c6ed0e2fa03efd5b44747b625fec79bc9cd06ac5235a779726618e530d` | 7 / 75 / 67 / 37 / 4 | 82 |
|
||||
| `dxflrs/garage:v2.3.0` | `866bd13ed2038ba7e7190e840482bc27234c4afaf77be8cfa439ae088c1e4690` | **Unknown: no inventory** | — |
|
||||
| `greenmail/standalone:2.1.9` | `3ac5a83dd6727cf95e4d50e18907fb8ee7bbf5f67e8534714dee2fb1b5b2e1d4` | 0 / 0 / 116 / 35 / 0 | 0 |
|
||||
| `tonistiigi/binfmt:qemu-v10.2.3-68` | `400a4873b838d1b89194d982c45e5fb3cda4593fbfd7e08a02e76b03b21166f0` | 0 / 9 / 2 / 1 / 1 | 9 |
|
||||
|
||||
## Patch-only options and limits
|
||||
|
||||
Complete publisher tag listings were inspected for nginx 1.29, Caddy 2.10,
|
||||
HAProxy 3.2, GreenMail 2.1 and binfmt qemu10.2. Two newer candidates were
|
||||
scanned; their registry index bytes matched both registry and publisher digests,
|
||||
and contained amd64 and arm64 manifests:
|
||||
|
||||
- `library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`:
|
||||
same Alpine 3.24.1, 24 detected OS packages, zero reported findings. This is
|
||||
a useful candidate, not a completed compatibility test or application audit.
|
||||
- `greenmail/standalone:2.1.13@sha256:3df66b7edd01c8a301343ca5e3601d8674760d4708655573560c24745e624fb2`:
|
||||
upstream changes Ubuntu 22.04 to Debian 13.6; **3 C / 80 H / 98 M / 85 L /
|
||||
5 unknown**, 30 fixable C/H records. Not selected as a no-base-change update.
|
||||
- nginx's newest matching Alpine patch is already 1.29.8 at the scanned pin;
|
||||
Caddy 2.10 remains 2.10.2; binfmt qemu10.2 remains 10.2.3-68. No newer matching
|
||||
publisher images were found. The current [official Caddy image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy)
|
||||
uses 2.11.4; switching minor lines requires new scans and compatibility checks.
|
||||
|
||||
Priority remediation: Caddy's own seven HIGH records require fixes through
|
||||
2.11.4, with additional bundled Go/library fixes that must be re-scanned;
|
||||
nginx's packages include curl/libcurl fixes through 8.22.0-r0, OpenSSL 3.5.8-r0,
|
||||
c-ares 1.34.8-r0, expat 2.8.1-r0 and libuuid 2.41.6-r1. PostgreSQL's OS records
|
||||
require OpenSSL 3.5.8-r0 and libuuid 2.42.3-r1; its CRITICAL plus 21 HIGH Go
|
||||
records concern the **gosu helper**, not PostgreSQL server code. binfmt's nine
|
||||
HIGH records concern its Go 1.26.4 build, with fixes through 1.26.6. Package
|
||||
presence does not establish vulnerable-symbol reachability. No unscanned tag
|
||||
is claimed to meet every fix requirement.
|
||||
|
||||
## Python triage and coverage caveats
|
||||
|
||||
Python image metadata identifies CPython 3.12.14, but Trivy inventories only
|
||||
Debian packages and pip, **not CPython/stdlib**. All 60 C/H records concern
|
||||
Debian packages: 21 CVEs, 50 `affected` records, 9 `fix_deferred`, 1
|
||||
`will_not_fix`, without a recorded fixed Bookworm version. Five util-linux CVEs
|
||||
repeat across eight binary packages. These remain installed; they are not all
|
||||
removed build dependencies. Pip 25.0.1 separately has five MEDIUM/one LOW
|
||||
records, with fixes through 26.2.0; it is install tooling, and the API image uses
|
||||
an offline `--no-index` wheelhouse rather than an arbitrary package index.
|
||||
|
||||
Narrow triage examples, **not blanket exemptions**:
|
||||
|
||||
- Debian states [CVE-2023-45853](https://security-tracker.debian.org/tracker/CVE-2023-45853)
|
||||
does not affect the built Bookworm zlib binaries because vulnerable minizip
|
||||
code is not included. Other bundled minizip implementations are separate.
|
||||
- [CVE-2026-8376](https://security-tracker.debian.org/tracker/CVE-2026-8376)
|
||||
explicitly requires 32-bit Perl; this scan targets amd64.
|
||||
- [CVE-2025-7458](https://security-tracker.debian.org/tracker/CVE-2025-7458)
|
||||
requires crafted arbitrary SQLite SQL; the managed runtime uses PostgreSQL,
|
||||
but alternate SQLite use must be reviewed.
|
||||
- Perl's regex and Archive::Tar records need exact binary/module applicability
|
||||
checks; vendor-deferred status alone is not a finding dismissal.
|
||||
|
||||
Only amd64 was scanned. arm64, newly built GovOPlaN API/Web layers and optional
|
||||
dependency combinations remain unverified. Garage has no inventory; Redis,
|
||||
HAProxy and PostgreSQL source-built executables, CPython and QEMU static
|
||||
binaries need supplemental SBOM/source coverage. Zero detected OS findings is
|
||||
not zero application vulnerabilities. Trivy also lacks Alpine 3.24 EOL metadata
|
||||
and nginx CVE-2026-80256 detail; unknowns are retained. There were no runtime,
|
||||
exploitability, secret, misconfiguration, malware or signature-policy checks.
|
||||
|
||||
Before lifting the runtime hold: approve and test maintained image-line changes
|
||||
where necessary, fix or narrowly disposition findings with evidence, close
|
||||
inventory gaps, scan both architectures and final runtime layers, then run
|
||||
deployment/ingress smoke checks. Do not silently change base OS, use unpinned
|
||||
`latest`, rebuild third-party images, or accept all HIGH/CRITICAL findings.
|
||||
@@ -0,0 +1,139 @@
|
||||
# Runtime image remediation follow-up — 8 September 2026
|
||||
|
||||
Canonical tracking: [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
|
||||
and [website #9](https://git.add-ideas.de/add-ideas/addideas-govoplan-website/issues/9).
|
||||
This addendum supplements the [original audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md);
|
||||
it does not replace that historical baseline or lift either publication or
|
||||
deployment gate. The immutable 0.1.45 release and `catalog-v0.1.45` are unchanged.
|
||||
|
||||
## Source change and candidate decisions
|
||||
|
||||
New installer specifications now use
|
||||
`haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`.
|
||||
This is a patch update from 3.2.21 within the supported
|
||||
[3.2 LTS branch](https://www.haproxy.org/), keeping Alpine 3.24.1.
|
||||
The [publisher's exact build source](https://github.com/docker-library/haproxy/blob/7a5c202cde713867a737033dca56e7a211a8b8df/3.2/alpine/Dockerfile)
|
||||
and scanned image configuration retain the non-root `haproxy` user,
|
||||
`/usr/local/etc/haproxy/haproxy.cfg`, entrypoint and graceful-stop signal.
|
||||
The [upstream changelog](https://www.haproxy.org/download/3.2/src/CHANGELOG)
|
||||
includes HTTP parsing, TLS and memory-safety fixes in 3.2.22/3.2.23.
|
||||
Existing specifications retain their explicit image, including an older pin;
|
||||
this source change does not update a running installation.
|
||||
|
||||
| Candidate | OS | C / H / M / L / Unknown, per architecture | Disposition |
|
||||
| --- | --- | --- | --- |
|
||||
| HAProxy `3.2.23-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Installer source default updated; binary/runtime checks pending |
|
||||
| nginx-unprivileged `1.30.4-alpine` | Alpine 3.24.1 | 0 / 0 / 0 / 0 / 0 | Candidate only; compatibility and website OS upgrade review pending |
|
||||
| Caddy `2.11.4-alpine` | Alpine 3.23.5 | 1 / 38 / 41 / 12 / 23 | Not selected; all 39 C/H records have recorded fixes |
|
||||
| Node `24-alpine` (24.20.0) | Alpine 3.24.1 | 0 / 6 / 11 / 12 / 0 | Not selected; major change and six fixable HIGH records |
|
||||
|
||||
Each row was scanned separately for **linux/amd64 and linux/arm64**, with the
|
||||
same counts on both. Counts are package-vulnerability records, not distinct
|
||||
CVEs or proven exploits. The [machine-readable evidence](runtime-image-candidates-2026-09-08.json)
|
||||
contains exact index, platform-manifest, config and report digests, inventory
|
||||
counts, scanner bounds and decisions. It is audit data, not an accepted release
|
||||
manifest or an installer input.
|
||||
|
||||
nginx's candidate reference is
|
||||
`docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce`.
|
||||
It inventories 70 Alpine packages including nginx/NJS, uses UID 101 and exposes
|
||||
8080. The [upstream stable release](https://nginx.org/en/download.html) and
|
||||
[security advisories](https://nginx.org/en/security_advisories.html) include the
|
||||
1.30.4 fixes. The publisher retains its
|
||||
[unprivileged port and temporary-path contract](https://github.com/nginx/docker-nginx-unprivileged).
|
||||
For the website, this changes nginx 1.27.5 to 1.30.4, NJS 0.8.10 to 1.0.1 and
|
||||
Alpine 3.21.3 to 3.24.1. These changes are explicit review items; the website
|
||||
Dockerfile has not been changed. The GovOPlaN Web image still requires an
|
||||
explicit verified `NGINX_IMAGE` build argument.
|
||||
|
||||
Caddy's candidate reference is
|
||||
`docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648`.
|
||||
Although this is the current
|
||||
[official image line](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy),
|
||||
its inventory still includes Go 1.26.3, `x/crypto` 0.52.0, `x/net` 0.55.0,
|
||||
`x/text` 0.37.0 and gRPC 1.81.0. Recorded fixes include Go 1.26.6,
|
||||
`x/crypto` 0.55.0, `x/net` 0.56.0, `x/text` 0.39.0 and gRPC 1.83.1; Alpine
|
||||
findings also remain in c-ares, curl/libcurl and OpenSSL. The CRITICAL
|
||||
`CVE-2026-56854` concerns `x/crypto/ssh` source-address enforcement. A module
|
||||
record alone does not establish that this binary exposes that SSH path; exact
|
||||
binary symbol/reachability analysis is still required for a disposition.
|
||||
|
||||
The [official Node image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/node)
|
||||
still maps Node 22 Alpine to 22.23.2 and the previously scanned digest. Node 24's
|
||||
candidate is
|
||||
`docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf`.
|
||||
Its HIGH records remain in two OpenSSL packages and npm dependencies
|
||||
`brace-expansion`, `ip-address` and `tar`; fixing the earlier critical tar
|
||||
record alone is insufficient. The website builder stays on Node 22 pending
|
||||
a reviewed build-tool remedy and a final builder scan.
|
||||
|
||||
## Method, verification and retained evidence
|
||||
|
||||
The existing Trivy 0.74.0 executable was rehashed against the previously verified
|
||||
archive member: `d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791`.
|
||||
The same schema-2 vulnerability database was used, updated
|
||||
`2026-09-07T19:06:01.154199452Z`. No tool installation or database refresh occurred.
|
||||
Index bytes matched both the registry digest header and Docker Hub publisher
|
||||
metadata; both platform-manifest byte hashes matched the index. All eight
|
||||
registry-only scans completed successfully with validated JSON, `--list-all-pkgs`,
|
||||
`--scanners vuln`, an eight-minute/2GB image bound, an empty Docker configuration
|
||||
and no inherited credentials. Exit zero means execution succeeded. Private
|
||||
temporary paths and in-memory artifact cache isolated this follow-up from the
|
||||
earlier scanner's artifact cache; its vulnerability database was read only.
|
||||
|
||||
Raw reports, logs, manifests, publisher metadata and the scanner script are in
|
||||
`/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh/`:
|
||||
|
||||
- `summary.json` SHA-256: `0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56`.
|
||||
- `frozen-images.json` SHA-256: `d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467`.
|
||||
- `scan_successors.py` SHA-256: `f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07`.
|
||||
|
||||
Local retention is not permanent artifact hosting; preserve this evidence with
|
||||
the eventual reviewed release. The JSON evidence records compressed registry
|
||||
layer sizes; these are not expanded filesystem limits or final GovOPlaN sizes.
|
||||
|
||||
Installer regression checks cover the new generated image pin, legacy
|
||||
specification fallback, preserved explicit images, generated topology and
|
||||
configuration: `python -I -m unittest discover -s tests -p
|
||||
test_deployment_installer.py` ran 45 tests successfully with one skip because
|
||||
Core was not importable in that isolated test environment. The skipped Core
|
||||
startup-configuration integration was subsequently rerun in the shared development
|
||||
environment with Core available: all 45 installer tests passed with no skips,
|
||||
including generated-environment startup validation. This is configuration
|
||||
validation, not execution of the candidate image.
|
||||
Both repositories passed `git diff --check`; the audit JSON and all eight
|
||||
report hashes were checked against the retained evidence.
|
||||
**Docker, Podman and HAProxy executables are unavailable on
|
||||
this host**, so no image or HAProxy configuration was executed and no daemon was
|
||||
installed. Publisher metadata and installer tests support the scoped source
|
||||
patch; they do not establish binary or deployed compatibility.
|
||||
|
||||
## Gates that remain open
|
||||
|
||||
- Validate `haproxy -c` on generated local, existing-proxy and managed-ingress
|
||||
configurations using the exact pinned image and target architectures. Run
|
||||
bounded isolated checks without live mounts, secrets, privilege or external
|
||||
network access. Then verify DNS discovery, readiness, forwarded headers,
|
||||
replica routing and graceful termination in the intended runtime.
|
||||
- Test the nginx candidate with both the website configuration and GovOPlaN
|
||||
WebUI entrypoint/proxy configuration, including UID 101, writable temporary
|
||||
paths, health paths, cache headers and static catalog bytes. Approve the
|
||||
website nginx/NJS/Alpine version changes before changing its Dockerfile.
|
||||
- Resolve Caddy, Node build-tool and all unchanged baseline dependencies with
|
||||
updated publisher images or narrow reviewed applicability evidence. No
|
||||
severity-wide exceptions or custom third-party rebuilds were introduced.
|
||||
- Close the original source-built/static inventory gaps. HAProxy's 24-package
|
||||
OS inventory still omits the source-built HAProxy executable. Node's npm
|
||||
inventory still omits the Node executable/stdlib. Garage, CPython, Redis,
|
||||
PostgreSQL and QEMU gaps are unchanged. Alpine 3.24 EOL metadata is still
|
||||
missing from this scanner; zero findings is not complete coverage.
|
||||
- Scan **final built** API/Web/website layers and the selected managed
|
||||
dependencies on both architectures, then perform migration, worker,
|
||||
readiness and ingress smoke checks. Record failure and unknown states.
|
||||
Secrets, misconfiguration and image signature policy need separate checks.
|
||||
- Obtain the website deployment host/operator and rebuild/restart authority,
|
||||
preserving the exact immutable catalog/keyring/module-directory bytes and
|
||||
verifying fresh public responses after an authorized rollout.
|
||||
|
||||
No images were built, executed, published or deployed; no running service,
|
||||
release tag, signed manifest, CI image input or live infrastructure was changed.
|
||||
@@ -0,0 +1,219 @@
|
||||
# Security and performance follow-up — 8 September 2026
|
||||
|
||||
This follows the [original review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
|
||||
and its post-release issue reconciliation. It describes new source work after
|
||||
the frozen 0.1.45 release; it does not change published tags, packages, signed
|
||||
catalogs or deployed images. Gitea remains the canonical state log.
|
||||
|
||||
## Implemented source slices
|
||||
|
||||
- [Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
|
||||
a shared disposable-process runner enforces wall/CPU/address-space/input/output
|
||||
limits, bounded stderr, process-group cleanup and non-queuing per-process
|
||||
admission. A private binary codec bounds decoding before allocating a full
|
||||
object graph and preserves explicitly supported data types without pickle.
|
||||
Read the owning Core `docs/BOUNDED_PROCESS_CONTRACT.md` before adding callers.
|
||||
- Connectors XLSX parsing, Templates rendering, Files ZIP/TAR inspection and
|
||||
extraction, and Dataflow reference previews/development execution now use
|
||||
that boundary. Existing authorization, sessions, provider credentials,
|
||||
idempotency and persistence remain in the parent. No unprotected inline
|
||||
fallback is used. Each module contributes static EN/DE user/admin limits and
|
||||
operational consequences through its manifest.
|
||||
- Files snapshots authorized sources inside shared admission, validates private
|
||||
staged members, and acknowledges each persisted member before decoding the
|
||||
next. Numeric progress remains available. The acknowledgement is event-driven,
|
||||
not a fixed sleep per member. Reads allocate by validated actual file size,
|
||||
not by the configured ceiling. Failures reap children, clear private staging
|
||||
and retain the existing transaction/blob cleanup and explicit retry behavior.
|
||||
- Dataflow's normal reference preview formerly bypassed the backend wrapper;
|
||||
it now enters the worker too. Nested source configurations cannot collide
|
||||
merely because subflows reuse node IDs. Combined reference-source data is
|
||||
checked before creating further columnar copies, while individual providers
|
||||
retain their own authorized-read bounds. Staging/production still require
|
||||
DuckDB; this change does not replace that separate backend.
|
||||
- [Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
|
||||
current-password change, session/CSRF rotation, cross-tenant session and human
|
||||
API-key revocation, and optional administrator-assisted recovery. Recovery
|
||||
codes are hashed, single-use, expire after 15 minutes, require a current local
|
||||
System owner and explicit identity verification, and recheck current account,
|
||||
membership, tenant and issuer authority at redemption. A password change also
|
||||
invalidates outstanding codes issued by that account for other people. Audit
|
||||
evidence and validation/error responses do not contain passwords or codes.
|
||||
External-provider and service-account rules remain separate.
|
||||
- The Access UI provides first-login/required change, self-service change,
|
||||
policy-aware sign-in help, public code redemption and eligible owner issuance.
|
||||
Core consumes an optional lazy auth-action capability rather than importing
|
||||
Access internals. The required-action gate fails closed if its UI is missing.
|
||||
- [Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
|
||||
full-history lists batch pinned revisions, while new summary and bounded
|
||||
step/event endpoints preserve authorization and explicit pagination. Existing
|
||||
full-history responses are not silently truncated. Exact inbox total semantics
|
||||
are retained and their counting cost is documented.
|
||||
- [Meta #55](https://git.add-ideas.de/GovOPlaN/govoplan/issues/55):
|
||||
shared version/planning helpers recognize the existing nested developer
|
||||
package, not invented root metadata. All tag batches enforce trusted private
|
||||
source ownership, registered origins and clean main/upstream state. Meta
|
||||
batches additionally require exact composition and matching Core evidence.
|
||||
Whole-batch preflight,
|
||||
frozen source receipts, annotated immutable tags, object-pinned atomic
|
||||
publication and post-effect remote checks are covered with temporary local
|
||||
repositories. Hidden Git index flags, unsafe ancestry, alternates and changed
|
||||
Git-directory identities are rejected. Selected version/composition metadata
|
||||
must be tracked, so ignored files cannot describe bytes absent from a tag.
|
||||
Applicable unselected Core WebUI inputs have bounded, frozen read receipts;
|
||||
backend-only releases do not read them. The existing local module-candidate
|
||||
exception remains intact. The weaker legacy mutation path was removed.
|
||||
Canonical whole-package preview and receipt-bound apply now cover Meta's
|
||||
version preparation too. Core must already match the target. Preparation
|
||||
requires a separate trusted checkout, explicit out-of-run confirmation and
|
||||
unchanged source/tooling receipts; it cannot rewrite the running operator.
|
||||
Plans place Meta after Core and explain the manual preparation/publication
|
||||
steps instead of promising a durable self-update. Ambiguous partial writes
|
||||
require reconciliation, without automatic rollback or retry.
|
||||
- [Runtime-image follow-up](RUNTIME_IMAGE_REMEDIATION_2026-09-08.md): eight new
|
||||
registry-only scans cover four exact candidates on amd64 and arm64. New
|
||||
installer specifications select the patched same-line HAProxy digest;
|
||||
existing specifications retain their explicit image. Other candidates and
|
||||
unresolved inventory/deployment gates remain visible, not blanket-approved.
|
||||
|
||||
## Verification record
|
||||
|
||||
Targeted checks include actual child execution, catastrophic regex CPU,
|
||||
aggregate memory exhaustion, TAR extension metadata, noisy output, malformed
|
||||
transport/staging data, Unicode allocation limits, cancellation/callback
|
||||
failures, descendant cleanup, rollback and explicit retries. Local mixed-owner
|
||||
composition tests completed nine real children, rejected six overlapping
|
||||
requests as busy, observed at most one unreaped child and recovered all slots.
|
||||
This is local admission evidence, not a target deployment load certification.
|
||||
|
||||
Workflow fixtures serialize 40 different pinned revisions with five SQL reads;
|
||||
summary lists use one query for 40 ordinary rows. Exact inbox totals for
|
||||
40/400/4,000 candidates used one query, with measured local costs approximately
|
||||
0.011/0.057/0.492 seconds. These are fixture measurements, not production SLOs.
|
||||
|
||||
The broader Core API smoke suite exposed three stale campaign assertions.
|
||||
All three failures were reproduced against the unchanged private frozen 0.1.45
|
||||
sources. Updated fixtures verify recipient-summary projection, detailed payload
|
||||
separation and explicit fenced recovery of a confirmed stopped runtime; observing
|
||||
SENDING alone must not make a claim recoverable. All 76 smoke tests then passed.
|
||||
No production Campaign behavior was changed to satisfy these tests.
|
||||
|
||||
The final release-tool suite passed 279 tests and 68 subtests, including
|
||||
temporary local remotes and adversarial source/tag/receipt changes. Rechecking
|
||||
the whole batch before effects is deliberately conservative: its repeated
|
||||
filesystem/Git/remote work grows quadratically with batch size. It is not a
|
||||
new unattended publication path or permission to execute unreviewed source.
|
||||
|
||||
Strict interface inventory now reports no unclassified endpoints and exact
|
||||
contextual help for all 133 high-risk controls. Seventeen password browser cases
|
||||
include actual F1 help from the restricted screen, empty workspace scopes,
|
||||
EN/DE layouts, Unicode boundaries and no credential values in help URLs.
|
||||
The initial production bundle remains within the unchanged limits (512,036
|
||||
raw bytes and 162,415 gzip bytes; 1,713 gzip bytes below its ceiling), with
|
||||
46 optional descriptors and no eager optional-module imports.
|
||||
|
||||
The focused checker now includes the new Core process, mixed-owner admission,
|
||||
Access password, Templates and Files worker tests, the repaired campaign smoke
|
||||
cases, and browser-side auth/password transport contracts. The full focused run
|
||||
passed, including 63 production module/build permutations and all 230 browser
|
||||
cases. Its two opt-in Datasources PostgreSQL cases were skipped in that run
|
||||
and subsequently passed against the isolated real database described below.
|
||||
The final Meta preparation gate was added after that full run and verified
|
||||
with the owning release-tool suite and the focused release-gate command.
|
||||
Manifest validation passed for all 72 modules. The full focused log is
|
||||
`/mnt/DATA/tmp/govoplan-security-followup-20260908-focused.log`.
|
||||
|
||||
The first follow-up quick audit captured an unchanged 79-repository snapshot
|
||||
in `/mnt/DATA/tmp/govoplan-security-followup-quick-20260908-7s8Sgh/`.
|
||||
All four required scanners completed, with zero missing/execution reports;
|
||||
all 168 report checksums and 163 machine-readable reports were validated.
|
||||
Gitleaks found no secrets in all 79 histories and 79 worktrees. Local Semgrep
|
||||
rules reported zero findings. Production Bandit reported 65 low and four medium
|
||||
warnings, and production Ruff retained 54 warnings. The two added Bandit
|
||||
warnings identify the new Core subprocess import and invocation: trusted
|
||||
server-owned arguments, no shell, and the documented resource/process boundary
|
||||
were reviewed; warnings remain visible. This is report-only evidence, not a
|
||||
warning-free audit or a penetration test. A final snapshot follows the
|
||||
cross-module declaration/contextual-help corrections and release-tool checks.
|
||||
|
||||
That final audit completed on 8 September, 05:59:46–06:02:18 UTC, in
|
||||
`/mnt/DATA/tmp/govoplan-security-final-quick-20260908-vAwQIh/`. All 79 start/end
|
||||
source fingerprints were identical; all four scanners completed, all 168
|
||||
registered report checksums matched, and all 163 JSON/SARIF reports parsed.
|
||||
There were no missing reports or scanner execution errors. Semgrep and both
|
||||
Gitleaks scopes again reported zero findings. Production counts were unchanged
|
||||
from the first follow-up: Bandit 65 low/four medium and Ruff 54. Test-only
|
||||
counts were Bandit 140 low/34 medium and Ruff 136. A separate frozen scan of
|
||||
all ten changed Meta release/deployment Python files reported seven low Bandit
|
||||
and four Ruff S603 warnings, with no execution errors. Its four argv-only
|
||||
subprocess sites were reviewed; the preparation additions introduced no new
|
||||
warnings. No findings were hidden or severity-wide exceptions added.
|
||||
The audit manifest SHA-256 is
|
||||
`a997b786239cd11443cb665d5f9041a968cc38f9d49171e68bb868bf2bd73310`;
|
||||
its report-checksum list SHA-256 is
|
||||
`dc590ca5b0a4e445019a05536d410226088d67b40d61cd7657bdef4a4eae56d8`.
|
||||
|
||||
The audit includes the eight committed feature/website source changes and the
|
||||
final uncommitted Meta source. Only this evidence document was updated after
|
||||
the source freeze ended; the final Meta commit and remote publication are
|
||||
recorded in the linked Gitea issues, not inferred from local audit completion.
|
||||
|
||||
Fresh dependency audits are retained in
|
||||
`/mnt/DATA/tmp/govoplan-dependency-final-20260908-d24LEK/`: all four full npm
|
||||
lockfile audits (Core WebUI, Mail root/WebUI and website) report zero known
|
||||
vulnerabilities. Installed Python auditing covers 137 distributions with zero
|
||||
known vulnerabilities; 51 local GovOPlaN distributions lack PyPI advisory
|
||||
coverage. Core's 46 linked packages are likewise not claimed covered by public
|
||||
registry advisories. All 12 dependency-file hashes and the installed inventory
|
||||
were unchanged. No packages were installed or automatically fixed.
|
||||
|
||||
Managed PostgreSQL 16.15 fixtures used private Unix sockets, synthetic roles
|
||||
and databases, no TCP listener, per-case schemas and bounded SQL/lock waits.
|
||||
Both previously skipped Datasources races passed. Twenty-one existing Access
|
||||
password HTTP tests and four additional races passed on PostgreSQL: single-use
|
||||
redemption, stale-session/password replacement, competing issuance, and issuer
|
||||
password revocation during redemption. Four release/development migration checks
|
||||
also passed for Access and Workflow, including credential preservation and
|
||||
idempotent indexes. The four races are now owning opt-in Access regressions;
|
||||
see `govoplan-access/docs/PASSWORD_RECOVERY_POSTGRES_TESTS.md`. These local
|
||||
database checks do not certify a deployment, fleet load or external recovery
|
||||
handover. With both explicit PostgreSQL test URLs enabled, the full Access suite
|
||||
passed 122 tests and 18 subtests, and the full Datasources suite passed 57 tests,
|
||||
without skips. Access retained 12 existing SQLite datetime-adapter warnings in
|
||||
its separate SQLite migration cases. Both temporary PostgreSQL fixtures were
|
||||
stopped and independently verified: no server process, private socket,
|
||||
generated schema or synthetic cluster remains. Scripts, logs and shutdown
|
||||
receipts are retained under `/home/zemion/.cache/govoplan-pg-security-20260908.RAUitg/`
|
||||
and `/home/zemion/.cache/govoplan-pg-promoted-20260908.JZcIKL/`.
|
||||
|
||||
## Adoption and remaining gates
|
||||
|
||||
1. `AUTH_LOCAL_PASSWORD_RECOVERY_ENABLED` remains **false** by default. The
|
||||
existing flag is still advisory until an operator explicitly adopts and
|
||||
enables the complete recovery policy. Confirm who verifies identity and how
|
||||
the one-time code is handed over; automated email recovery is not enabled.
|
||||
Test first-login, lost-password, code expiry and administrator availability
|
||||
in the target environment before enforcement.
|
||||
2. Access migration `e9a2c5f8b1d4` adds recovery evidence; Workflow migration
|
||||
`9e6b3f8a2c7d` adds summary-pagination indexes. Use normal backed-up upgrade
|
||||
procedures and account for index-build cost. No manual live migration or
|
||||
server restart was performed during this work. The user's existing devserver
|
||||
has automatic reload, so live schema state must not be assumed unchanged.
|
||||
3. Release preparation must assign new source/package versions and require a
|
||||
Core version containing the new worker/auth contracts in the affected module
|
||||
metadata, including matching WebUI assets. The old immutable release must
|
||||
not be relabelled or treated as containing these APIs.
|
||||
4. Resource limits are not an arbitrary-code, filesystem or network sandbox.
|
||||
Admission is per API/worker process, not fleet-wide. Validate Linux/cgroup
|
||||
memory, disk quotas, process counts, cancellation and legitimate large-file
|
||||
workloads on the intended runtime before increasing concurrency. Core #297
|
||||
retains this target-evidence follow-up.
|
||||
5. Meta #52 and website #9 retain runtime-image/publication/deployment holds.
|
||||
Docker/Podman/HAProxy executables are unavailable here. Final built images,
|
||||
binary/source inventories, ingress behavior, migration/readiness/worker
|
||||
smoke checks and the website's target/operator authority remain outstanding.
|
||||
Zero findings in a detected package inventory is not full image coverage.
|
||||
|
||||
No real messages, IMAP appends, password resets, provider operations or deployment
|
||||
actions were used as test fixtures. Development tests use temporary databases,
|
||||
private temporary files, mock transports and managed test-browser servers.
|
||||
@@ -0,0 +1,207 @@
|
||||
# Security and performance review — 8 September 2026
|
||||
|
||||
Coordinated status: [Core #296](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/296).
|
||||
This records a workspace-wide automated scan, targeted manual boundary review,
|
||||
and a verified implementation pass. It is not a penetration test, an exhaustive
|
||||
line-by-line review, or a security certification. The audit was completed on
|
||||
local, unpublished changes, preserving existing worktree changes. Subsequent
|
||||
release preparation/publication is tracked in
|
||||
[GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51) and the
|
||||
[0.1.45 release notes](../releases/0.1.45.md).
|
||||
|
||||
## Implemented findings
|
||||
|
||||
| Area | Finding and change | Evidence / ownership |
|
||||
| --- | --- | --- |
|
||||
| Authentication — high | Preserve service-account provenance and current scope ceilings instead of recalculating them as ordinary membership permissions. Tenant API keys cannot retain canonical system permissions or unsafe wildcard grants. | Previously failing isolated regressions; [Access #21](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/21). |
|
||||
| Authentication — medium | Warm-cache API keys must follow the same explicit-header credential rules as cold authentication. A session cookie cannot turn an API key into a session credential. | Regression covering source-dependent authentication. |
|
||||
| Browser authority/cache — medium | Clear reusable data on auth changes and write settlement; fence late 200/304 writes and obsolete 401 side effects. Honor server no-store/no-cache and explicit fresh-read requests. Interactive login/logout remove retained automation keys that could shadow cookie-session identity. | 23 real-client regressions. Unchanged settings keep their object identity, preventing profile-fetch loops. Core `docs/API_CLIENT_CACHE_CONTRACT.md`; owning Access EN/DE session/field documentation. |
|
||||
| Spreadsheet resource exhaustion | Validate actual XLSX coordinates before openpyxl traversal; ignore misleading declared dimensions; count blank row gaps toward the existing limits. | [Connectors #18](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/18), 13 tests and 2 subtests. |
|
||||
| Template resource exhaustion | Enforce the existing 5 MiB output budget during substitution and item construction, including UTF-8, HTML escaping and separators. | [Templates #7](https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/7), full 20 tests; independent 3,000-case valid-output comparison. |
|
||||
| Archive resource exhaustion | Inspect regular TAR member limits before traversing payloads. Limit archive paths to 4,096 UTF-8 bytes / 128 components and count derived directories against entry limits. | [Files #46](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/46), 55 archive and 15 documentation tests. Extension-header decoding still needs stronger isolation. |
|
||||
| Dataflow resource exhaustion | Reject LPAD/RPAD target lengths above the existing 1,000,000-byte preview budget before fill evaluation/allocation. Preserve final serialized-byte checks. | [Dataflow #22](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/22), full 104 tests and 39 subtests; 7 new guard tests independently rerun. |
|
||||
| Docs performance / defense in depth | Batch revision reads per request, avoid loading pending draft bodies for readers, and validate tenant/entry/publication consistency while retaining owner/audience checks. | [Docs #22](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/22), full 39 tests. |
|
||||
| Notifications performance / defense in depth | Batch delivery-attempt loading while preserving recipient checks and rejecting inconsistent attempt references, including already-loaded relationships. | [Notifications #6](https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/6), full 23 tests. |
|
||||
| Session-list performance | Apply active/expiry predicates and the existing 100-row cap in SQL, before loading session history. | Query-shape regression in Access. |
|
||||
| Reporting correctness | Use structural bind-name suffixes for recursive calculated measures, preserving valid dotted/hyphenated public keys and parameter uniqueness. | [Reporting #10](https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/10), full 29 tests. |
|
||||
| Audit hygiene | Redact Gitleaks logs and machine reports on current, history and legacy scanner paths. | 13 audit-wrapper tests enforce the flag. |
|
||||
|
||||
All changed module workflows/limits have owning EN/DE DocumentationTopic updates.
|
||||
Independent review found no concrete regression in the backend changes.
|
||||
|
||||
## Measured performance changes
|
||||
|
||||
These are SQL-query counts in isolated 40-item fixtures, not production latency
|
||||
or throughput claims. Authorization is still evaluated for each request.
|
||||
|
||||
| Projection | Before | After |
|
||||
| --- | ---: | ---: |
|
||||
| Docs reader entries | 41 SELECTs | 2 SELECTs |
|
||||
| Docs editor entries | 81 SELECTs | 2 SELECTs |
|
||||
| Notification list with attempts | 41 SELECTs | 2 SELECTs |
|
||||
|
||||
The Docs 401-entry batching regression uses 3 SELECTs. Resource guards reject
|
||||
oversized work before the formerly expensive allocation/traversal. This does
|
||||
not make every legitimate upload or campaign faster. Honoring no-cache can
|
||||
increase server validation requests; ETags still avoid retransmitting unchanged
|
||||
bodies. That authorization/freshness trade-off is deliberate.
|
||||
|
||||
The original audit snapshot measured 517,380 initial JavaScript bytes and
|
||||
164,119 gzip bytes. Release preparation's pure-defaults split reduces this to
|
||||
516,730 initial bytes and 163,908 gzip bytes. Restoring the missing Tasks
|
||||
descriptor then measures 516,987 initial / 163,976 gzip bytes with all 46 module
|
||||
descriptors lazy, within the unchanged 524,288 / 164,128 caps. The gzip margin is still small; future
|
||||
startup work should reduce eager dependencies, not raise the cap automatically.
|
||||
The full 209-case browser suite passed before the split, followed by 13 focused
|
||||
browser checks after it. Radon recorded 238 rank-D-or-higher entries; complexity
|
||||
is a review-priority signal, not a performance measurement.
|
||||
|
||||
## Dependency remediation
|
||||
|
||||
Core's full npm audit went from 30 affected package entries to zero. Most initial
|
||||
entries were transitive effects of the same Tiptap advisory, not 30 independent
|
||||
application exploits. The website went from two affected entries to zero; both
|
||||
Mail lockfiles also report zero.
|
||||
|
||||
- Tiptap packages are aligned at 3.31.3, with direct minimum ranges raised to
|
||||
3.30.4 in both development and release manifests, with a parity regression.
|
||||
Added an actual installed-library prototype-attribute regression for
|
||||
the [maintainer's security advisory](https://github.com/ueberdosis/tiptap/security/advisories/GHSA-cp6q-959q-f8rh).
|
||||
- Core now resolves xmldom 0.9.12, browserslist 4.28.9 and nanoid 3.3.18.
|
||||
The website's affected browserslist/nanoid dependencies are patched too.
|
||||
- Development/audit requirements now require pip >=26.2; the local development
|
||||
environment uses 26.2.1. The installed audit originally flagged
|
||||
[CVE-2026-13346](https://github.com/advisories/GHSA-qwm4-qh6w-59xr), requiring an
|
||||
attacker-controlled package index. This is an installation-tool vulnerability,
|
||||
not evidence of an exposed application endpoint.
|
||||
|
||||
The final installed Python audit enumerated 188 distributions: 137 were
|
||||
auditable with zero known vulnerabilities, and 51 local distributions were not
|
||||
available in PyPI. Those skips are covered by source review, not by a claim of
|
||||
dependency-advisory coverage. Production images and every optional dependency
|
||||
combination were not independently resolved or scanned.
|
||||
|
||||
## Scan coverage and limitations
|
||||
|
||||
Evidence directory:
|
||||
`/mnt/DATA/tmp/govoplan-security-performance-20260908-gsk8jn/`.
|
||||
|
||||
The final `final-quick/manifest.json` captures 79 repositories, tool versions,
|
||||
start/end repository fingerprints, report checksums, 168 report artifacts and
|
||||
163 validated JSON/SARIF reports. It records an unchanged workspace, complete
|
||||
coverage for its four required scanners, no execution errors and no missing
|
||||
reports. It ran in report-only mode: exit zero does **not** mean zero warnings.
|
||||
|
||||
- Final production Bandit: 447,008 Python lines; 67 warnings (63 low, 4 medium),
|
||||
no high findings. Ruff security rules: 54 warnings. SQL-construction warnings
|
||||
were reviewed against identifier/operator validation and bound values in
|
||||
DuckDB/Reporting; no injection fix was warranted there. XML import warnings
|
||||
were checked: feed/BPMN input parsing uses defusedxml; stdlib imports support
|
||||
types/output construction. Operator-owned fenced-run argv is not a public
|
||||
arbitrary-command endpoint. Xrechnung output buffering remains a follow-up.
|
||||
Assertions and error-swallowing markers remain review/maintenance warnings,
|
||||
not proof that all such code is harmless.
|
||||
- Final local Semgrep rules: no findings. The broader OWASP-rule pass applied
|
||||
272 rules to 4,169 tracked targets. Its seven warnings recommended weakening
|
||||
owner-only 0700 permissions; they were rejected as false positives. One
|
||||
Calendar rule timeout was rerun with a 60-second budget: zero findings/errors.
|
||||
Bash and conformance TypeScript checks passed despite two scanner-specific
|
||||
parser limitations. Ignored/dependency/generated paths are not a complete
|
||||
line-by-line source audit.
|
||||
- Gitleaks: 79 Git histories plus 79 worktrees, 158 redacted reports, zero
|
||||
detected secrets. This does not establish that deployed credentials are safe
|
||||
or that formerly exposed credentials have been rotated.
|
||||
- Tool versions included Semgrep 1.176.1, Bandit 1.9.4, Ruff 0.15.21 and
|
||||
Gitleaks 8.30.1. The downloaded Gitleaks binary archive matched the official
|
||||
release SHA-256 before execution.
|
||||
- The containerized full-toolbox path could not access Docker's daemon. Its
|
||||
full-mode Trivy/misconfiguration and additional OSV scans were **not** run.
|
||||
A subsequent [registry-only runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md)
|
||||
successfully scanned nine pinned candidates and two same-minor successors
|
||||
for amd64 without Docker. It found unresolved vulnerabilities and inventory
|
||||
gaps; runtime publication is held. This does not complete full-toolbox,
|
||||
arm64, final-runtime-image or deployment coverage.
|
||||
|
||||
No live application probes, database changes, file operations, mail sends,
|
||||
IMAP appends, imports, notification delivery, deployments, commits or pushes
|
||||
were performed. Browser tests used isolated mocked fixtures. Package installs,
|
||||
builds and temporary audit-tool installation were local development operations.
|
||||
|
||||
## Verification and remaining work
|
||||
|
||||
- 209/209 browser conformance tests pass; production Core/website builds,
|
||||
conformance TypeScript, 24 Core client/dependency regressions, 4 real-client
|
||||
Files reload checks, and 72/72 manifest checks pass.
|
||||
- Access's full 91-test suite passed before the final documentation-only update;
|
||||
the final documentation suite passed all 4 tests. Other module counts appear
|
||||
above. The new authentication/resource tests include demonstrated pre-fix
|
||||
failures rather than only structural assertions.
|
||||
- The original focused workspace run stopped at the institutional
|
||||
governance/Portal fixture (`tests/test_institutional_governance_journey.py:223`,
|
||||
`IndexError`). Release preparation fixes its mixed clocks using the existing
|
||||
temporal context, retaining validity-boundary exclusions; 7 journey tests and
|
||||
ambient-year checks pass. Tracked in
|
||||
[Meta #50](https://git.add-ideas.de/GovOPlaN/govoplan/issues/50).
|
||||
- Campaign's apparent host-path issue was ruled out by existing tracked
|
||||
API/build/snapshot guards and 11 passing tests under normal initialization.
|
||||
Release preparation fixes the standalone import cycle through a deferred
|
||||
resolver import without changing validation rules. Fresh-process coverage,
|
||||
all 11 path tests and Campaign's full 611-test suite pass.
|
||||
|
||||
Next coordinated work:
|
||||
|
||||
1. [Hard resource isolation — Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
|
||||
regex CPU, aggregate allocation, TAR extension metadata, bounded workers and
|
||||
cancellation, followed by production-like concurrent load tests.
|
||||
2. [Forced password change/recovery — Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
|
||||
the current flag is advisory only. Do not enable enforcement without a usable
|
||||
local-password/recovery flow and external-provider rules.
|
||||
3. [Bound subprocess output — Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2):
|
||||
enforce the existing 2 MiB limit while draining stdout/stderr, not afterwards.
|
||||
4. [Workflow revision batching/history projection — Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
|
||||
batch evidence lookups; separately define explicit history pagination and
|
||||
authorized-total semantics. Docs/notification history volumes also remain.
|
||||
5. Resolve the [runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md) findings
|
||||
tracked in [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
|
||||
and coverage gaps before lifting its publication hold; complete deployment
|
||||
audits, review exposed development credentials and worker quotas, and
|
||||
benchmark realistic tenant sizes/concurrency. The sanctions
|
||||
transport's fixed HTTPS/redirect allowlist is not a demonstrated arbitrary-URL
|
||||
issue, but migration to Core's pinned egress transport remains desirable.
|
||||
|
||||
Operational compatibility: tenant keys relying on accidental system/wildcard
|
||||
permissions must be corrected rather than weakening the guard. Extreme sparse
|
||||
spreadsheets, overly deep/long archive paths and oversized padding intermediates
|
||||
can now fail early with diagnostics. No stored documents or configurations were
|
||||
deleted or silently migrated.
|
||||
|
||||
## Post-release follow-up — 2026-09-08
|
||||
|
||||
The findings and scanner counts above describe the original audit snapshot.
|
||||
The following source fixes are subsequent to the frozen `0.1.45` composition;
|
||||
they do not change its immutable tags or published package bytes.
|
||||
|
||||
- [Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2)
|
||||
now enforces the existing shared 2 MiB stdout/stderr limit during execution
|
||||
and kills/reaps the direct validator on overflow, timeout or cancellation.
|
||||
Report reads are bounded to 16 MiB plus one probe byte before interpretation.
|
||||
The 30-test module suite passes; noisy-child and report-read regressions were
|
||||
also demonstrated to fail against the previous source. Owning EN/DE static
|
||||
documentation is updated. POSIX pipe capture is required; disk quotas,
|
||||
descendant isolation and process-level CPU/memory limits remain separate work.
|
||||
- [Meta #54](https://git.add-ideas.de/GovOPlaN/govoplan/issues/54) now preserves
|
||||
the last command's failure status after exhausted installer retries. Twelve
|
||||
isolated stage/scenario combinations cover every retry call site, success,
|
||||
backoff and caller termination under `set -e`. The test is included in the
|
||||
focused checks and installer CI. See the bilingual
|
||||
[installer retry note](../operations/WEBUI_RELEASE_DEPENDENCY_RETRIES.md).
|
||||
|
||||
These are unreleased follow-up source changes, not a new runtime release or
|
||||
deployment. The runtime-image hold under Meta #52 remains in force; the
|
||||
historical peer-dependency workaround still needs its separate review.
|
||||
|
||||
Further implementation and adoption gates are tracked in the
|
||||
[security follow-up](SECURITY_FOLLOWUP_2026-09-08.md), including disposable
|
||||
parsing/execution workers, opt-in password recovery, workflow read projections
|
||||
and the newer runtime-image evidence. The original scanner counts above remain
|
||||
historical and are not silently replaced by later test results.
|
||||
@@ -0,0 +1,305 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"purpose": "Audit evidence only; not a release manifest or active installer configuration.",
|
||||
"observed_at": "2026-09-08T03:56:47.666808+00:00",
|
||||
"runtime_publication_held": true,
|
||||
"website_deployment_held": true,
|
||||
"scan_execution_complete": true,
|
||||
"coverage_complete": false,
|
||||
"scanner": {
|
||||
"name": "Trivy",
|
||||
"version": "0.74.0",
|
||||
"binary_sha256": "d89bcc6510a267f11b773398cbf1be5520ce39f9e8b6633178c4487f05b7d791",
|
||||
"database_metadata": {
|
||||
"Version": 2,
|
||||
"NextUpdate": "2026-09-08T19:06:01.154199291Z",
|
||||
"UpdatedAt": "2026-09-07T19:06:01.154199452Z",
|
||||
"DownloadedAt": "2026-09-07T23:30:53.198039224Z"
|
||||
},
|
||||
"source": "remote",
|
||||
"scanners": [
|
||||
"vuln"
|
||||
],
|
||||
"list_all_packages": true,
|
||||
"images_executed": false,
|
||||
"existing_docker_credentials_used": false,
|
||||
"timeout": "8m",
|
||||
"maximum_image_size": "2GB"
|
||||
},
|
||||
"evidence": {
|
||||
"private_directory": "/home/zemion/.cache/govoplan-runtime-remediation.qfDSWHJh",
|
||||
"summary_sha256": "0d44390408ab35270e4430516f77bf11aa7877334eff2ef19e11e9a863fe5c56",
|
||||
"frozen_images_sha256": "d0cb156f4a88998531ec55ab950067a3f1350ded648f07650c463af101dad467",
|
||||
"scanner_script_sha256": "f64c69e06efc2ad7b5a657a25aa73f9ff586e3685737d525456bcecbe3ab5f07"
|
||||
},
|
||||
"candidates": [
|
||||
{
|
||||
"name": "haproxy",
|
||||
"image": "docker.io/library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
|
||||
"disposition": "source_default_updated_binary_runtime_verification_pending",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:0666a2c2f41d341084ed2da85392b48cdcd766adfa28231f31305724ed5c6ea5",
|
||||
"config_digest": "sha256:9621d75e50a8f26d3738ae3cdbf15e98c6aa5cd48e6baca0699492de502156f5",
|
||||
"compressed_layer_bytes": 20516844,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 24
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "40cfc3db74e29f09723f38698660ccdd50ac935c8d6e1e75c6e0555b3e9361fb",
|
||||
"log_sha256": "46881f695780f89c037d5b0b4dc0ded9ea4e459077c7658d80b786efc5754084"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:cd20b9dc6b4713956a2a043997001a1948167d345e7c8d5bd5ff2e667166651f",
|
||||
"config_digest": "sha256:19796bff8905d4a46c9463c576203b20cac8984d41b7b01ea9cbff55e8422c34",
|
||||
"compressed_layer_bytes": 20970772,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 24
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "0e1326c58abf358d592fa55c94333228ce1094edf4e489fcc4ece6e32d3320f6",
|
||||
"log_sha256": "397c2fbf1044cf50733d68b4b9cc4eb68211d96f1f302d5e9f8af0d11fb0de1c"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "nginx-stable",
|
||||
"image": "docker.io/nginxinc/nginx-unprivileged:1.30.4-alpine@sha256:442753882674b49ae2c1de83ed67896131c0777f56df5005e356e62bc3f7e7ce",
|
||||
"disposition": "candidate_pending_compatibility",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:b8c179cd3c2ae222a873dd59fbae240fadc03836cae5198afc9e9c19919c3880",
|
||||
"config_digest": "sha256:8b5953dae38d27a76bca22373bb920fd6ce8d9d7da21578d2926e678002de8a0",
|
||||
"compressed_layer_bytes": 25526590,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 70
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "3ad164dae3cdf891b12e41451b8a8e65a5e1688824a7c01d848e1274363e6bf9",
|
||||
"log_sha256": "0f99ff3d9b4396de48655bf8299df30c14ba0c579f480d37baa7d2f6a4c11f1d"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:b6742a0cbd749add25346658991c3da06a8e38796949df120fed78db8c512576",
|
||||
"config_digest": "sha256:4d8b10f5d2ff99e7aa5f161930d8a4693a86a26f288ec8c0d4cd47f2ef5af179",
|
||||
"compressed_layer_bytes": 25892370,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 70
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 0,
|
||||
"MEDIUM": 0,
|
||||
"LOW": 0,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 0,
|
||||
"unique_cves": 0,
|
||||
"report_sha256": "506fdeb97525ed8e4d9ae538c42bab7aa2217f662a7135f0f12d16d20410c7a6",
|
||||
"log_sha256": "c41ef9ea091d00f18c7a097404672591bee85e7477ae17d8f5ca771d8e42b2bb"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "caddy",
|
||||
"image": "docker.io/library/caddy:2.11.4-alpine@sha256:5f5c8640aae01df9654968d946d8f1a56c497f1dd5c5cda4cf95ab7c14d58648",
|
||||
"disposition": "not_selected_remaining_fixable_findings",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:98eb57d882ccd5213d1688764db10c1ca2c58a1ca3a6717a3411ad798f7a423a",
|
||||
"config_digest": "sha256:af555904a0961945f16bb323a501457b13a4f7e9bde969b145b97da80b38ecbe",
|
||||
"compressed_layer_bytes": 23907283,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.23.5"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 32
|
||||
},
|
||||
{
|
||||
"type": "gobinary",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 1,
|
||||
"HIGH": 38,
|
||||
"MEDIUM": 41,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 23
|
||||
},
|
||||
"fixable_high_critical": 39,
|
||||
"unique_cves": 68,
|
||||
"report_sha256": "e483352a1d5b9b97950dcf92e4dfcf4600f5b09306af3d0640b10ca229a07779",
|
||||
"log_sha256": "9cd599d6dd8c101b421fdeb57036cec1f69f815d765a8bf1f850ec60061036f4"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:1172d4213087d3fc30bafc7ff2c2896180eb0c41ff7f75f315568fb36cabdcba",
|
||||
"config_digest": "sha256:6b08c1b9858ca9a7d99c1da13c3695081e0e604c6cf214ca26a7ce0e2c4fd9b4",
|
||||
"compressed_layer_bytes": 22722712,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.23.5"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 32
|
||||
},
|
||||
{
|
||||
"type": "gobinary",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 1,
|
||||
"HIGH": 38,
|
||||
"MEDIUM": 41,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 23
|
||||
},
|
||||
"fixable_high_critical": 39,
|
||||
"unique_cves": 68,
|
||||
"report_sha256": "20e04d820e3b27d57575ea177e523e23109fd83344cdf57e184a4d2fad27e803",
|
||||
"log_sha256": "a1d9c37aa7948c137db64040d95e5930c5859a8acd71ac5bc41ff168e2b270c5"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "node-lts",
|
||||
"image": "docker.io/library/node:24-alpine@sha256:e67514e5d0f6c46656005e1b693b2ec9d52e80b641307de684d4a015ba7a4eaf",
|
||||
"disposition": "not_selected_remaining_fixable_findings",
|
||||
"platforms": [
|
||||
{
|
||||
"platform": "linux/amd64",
|
||||
"manifest_digest": "sha256:4caaaf42195bcd6f6f3559a413b20cb8f8ad089e231ee874cf7701643966689f",
|
||||
"config_digest": "sha256:ee289c69ed1ac50a5a042112ea97f132800e2dd53e832da27784f00e45b3289c",
|
||||
"compressed_layer_bytes": 58486244,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 18
|
||||
},
|
||||
{
|
||||
"type": "node-pkg",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 6,
|
||||
"MEDIUM": 11,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 6,
|
||||
"unique_cves": 19,
|
||||
"report_sha256": "c927d995dde700c92027f6328dc6c273f0f1445cd8154301480757cb962e02b2",
|
||||
"log_sha256": "c7dc1900cbe39c9f91e228b2770bcbd394ec2914d2b3879478295fc7f8f77ab3"
|
||||
},
|
||||
{
|
||||
"platform": "linux/arm64",
|
||||
"manifest_digest": "sha256:d3724e44ee368606d753e0027eb8d2a94fc1f275e5d9e4620178a12edb655f5f",
|
||||
"config_digest": "sha256:722cc1507731edf58a4c0bc3e29553c44ce5774d0849242c1b237abc79926a0a",
|
||||
"compressed_layer_bytes": 58935654,
|
||||
"scan_exit_code": 0,
|
||||
"os": {
|
||||
"Family": "alpine",
|
||||
"Name": "3.24.1"
|
||||
},
|
||||
"inventory": [
|
||||
{
|
||||
"type": "alpine",
|
||||
"packages": 18
|
||||
},
|
||||
{
|
||||
"type": "node-pkg",
|
||||
"packages": 146
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"CRITICAL": 0,
|
||||
"HIGH": 6,
|
||||
"MEDIUM": 11,
|
||||
"LOW": 12,
|
||||
"UNKNOWN": 0
|
||||
},
|
||||
"fixable_high_critical": 6,
|
||||
"unique_cves": 19,
|
||||
"report_sha256": "e5f7799761f23cefc36929381b4186eeb3afb46a2a559c789d12a7eea5dc30d0",
|
||||
"log_sha256": "fd24671f0da4d3b20dc8bcc2718531e6f6e19155d49bed15958965e21dd10813"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -4,85 +4,89 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "govoplan"
|
||||
version = "0.1.36"
|
||||
version = "0.1.46"
|
||||
description = "Developer convenience package for a versioned GovOPlaN composition"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = { text = "AGPL-3.0-or-later" }
|
||||
dependencies = [
|
||||
"govoplan-core[server]==0.1.36",
|
||||
"govoplan-tenancy==0.1.18",
|
||||
"govoplan-organizations==0.1.18",
|
||||
"govoplan-identity==0.1.18",
|
||||
"govoplan-idm==0.1.20",
|
||||
"govoplan-access==0.1.20",
|
||||
"govoplan-admin==0.1.19",
|
||||
"govoplan-policy==0.1.20",
|
||||
"govoplan-audit==0.1.19",
|
||||
"govoplan-dashboard==0.1.18",
|
||||
"govoplan-files==0.1.20",
|
||||
"govoplan-mail==0.1.22",
|
||||
"govoplan-campaign==0.1.24",
|
||||
"govoplan-calendar==0.1.18",
|
||||
"govoplan-docs==0.1.20",
|
||||
"govoplan-ops==0.1.19",
|
||||
"govoplan-core[server]==0.1.46",
|
||||
"govoplan-tenancy==0.1.22",
|
||||
"govoplan-organizations==0.1.21",
|
||||
"govoplan-identity==0.1.21",
|
||||
"govoplan-idm==0.1.26",
|
||||
"govoplan-access==0.1.25",
|
||||
"govoplan-admin==0.1.23",
|
||||
"govoplan-policy==0.1.23",
|
||||
"govoplan-audit==0.1.20",
|
||||
"govoplan-dashboard==0.1.20",
|
||||
"govoplan-files==0.1.27",
|
||||
"govoplan-mail==0.1.28",
|
||||
"govoplan-campaign==0.1.29",
|
||||
"govoplan-calendar==0.1.24",
|
||||
"govoplan-docs==0.1.23",
|
||||
"govoplan-ops==0.1.22",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
full = [
|
||||
"govoplan-addresses==0.1.18",
|
||||
"govoplan-approvals==0.1.18",
|
||||
"govoplan-assets==0.1.19",
|
||||
"govoplan-booking==0.1.19",
|
||||
"govoplan-cases==0.1.20",
|
||||
"govoplan-certificates==0.1.19",
|
||||
"govoplan-committee==0.1.18",
|
||||
"govoplan-connectors==0.1.22",
|
||||
"govoplan-consultation==0.1.19",
|
||||
"govoplan-contracts==0.1.19",
|
||||
"govoplan-dataflow==0.1.20",
|
||||
"govoplan-datasources==0.1.21",
|
||||
"govoplan-decisions==0.1.18",
|
||||
"govoplan-dist-lists==0.1.18",
|
||||
"govoplan-encryption==0.1.18",
|
||||
"govoplan-evaluation==0.1.19",
|
||||
"govoplan-facilities==0.1.19",
|
||||
"govoplan-forms==0.1.20",
|
||||
"govoplan-forms-runtime==0.1.18",
|
||||
"govoplan-grants==0.1.19",
|
||||
"govoplan-helpdesk==0.1.20",
|
||||
"govoplan-identity-trust==0.1.18",
|
||||
"govoplan-inspections==0.1.19",
|
||||
"govoplan-learning==0.1.19",
|
||||
"govoplan-mandates==0.1.18",
|
||||
"govoplan-notifications==0.1.18",
|
||||
"govoplan-parties==0.1.18",
|
||||
"govoplan-payments==0.1.20",
|
||||
"govoplan-permits==0.1.19",
|
||||
"govoplan-poll==0.1.19",
|
||||
"govoplan-portal==0.1.19",
|
||||
"govoplan-postbox==0.1.19",
|
||||
"govoplan-procurement==0.1.19",
|
||||
"govoplan-projects==0.1.18",
|
||||
"govoplan-quick-access==0.1.19",
|
||||
"govoplan-records==0.1.20",
|
||||
"govoplan-reporting==0.1.18",
|
||||
"govoplan-resources==0.1.19",
|
||||
"govoplan-addresses==0.1.23",
|
||||
"govoplan-approvals==0.1.21",
|
||||
"govoplan-assets==0.1.20",
|
||||
"govoplan-booking==0.1.20",
|
||||
"govoplan-cases==0.1.25",
|
||||
"govoplan-certificates==0.1.20",
|
||||
"govoplan-committee==0.1.22",
|
||||
"govoplan-connectors==0.1.27",
|
||||
"govoplan-consultation==0.1.20",
|
||||
"govoplan-contracts==0.1.20",
|
||||
"govoplan-dataflow==0.1.25",
|
||||
"govoplan-datasources==0.1.26",
|
||||
"govoplan-decisions==0.1.19",
|
||||
"govoplan-dist-lists==0.1.21",
|
||||
"govoplan-dms==0.1.20",
|
||||
"govoplan-encryption==0.1.20",
|
||||
"govoplan-erp==0.1.20",
|
||||
"govoplan-evaluation==0.1.20",
|
||||
"govoplan-facilities==0.1.20",
|
||||
"govoplan-fit-connect==0.1.20",
|
||||
"govoplan-forms==0.1.23",
|
||||
"govoplan-forms-runtime==0.1.22",
|
||||
"govoplan-grants==0.1.20",
|
||||
"govoplan-helpdesk==0.1.21",
|
||||
"govoplan-identity-trust==0.1.21",
|
||||
"govoplan-inspections==0.1.20",
|
||||
"govoplan-learning==0.1.20",
|
||||
"govoplan-mandates==0.1.19",
|
||||
"govoplan-notifications==0.1.20",
|
||||
"govoplan-parties==0.1.19",
|
||||
"govoplan-payments==0.1.22",
|
||||
"govoplan-permits==0.1.20",
|
||||
"govoplan-poll==0.1.20",
|
||||
"govoplan-portal==0.1.22",
|
||||
"govoplan-postbox==0.1.23",
|
||||
"govoplan-procurement==0.1.20",
|
||||
"govoplan-projects==0.1.20",
|
||||
"govoplan-quick-access==0.1.21",
|
||||
"govoplan-records==0.1.24",
|
||||
"govoplan-reporting==0.1.22",
|
||||
"govoplan-resources==0.1.20",
|
||||
"govoplan-rest==0.1.19",
|
||||
"govoplan-risk-compliance==0.1.18",
|
||||
"govoplan-scheduling==0.1.18",
|
||||
"govoplan-search==0.1.18",
|
||||
"govoplan-services==0.1.18",
|
||||
"govoplan-risk-compliance==0.1.21",
|
||||
"govoplan-scheduling==0.1.22",
|
||||
"govoplan-search==0.1.20",
|
||||
"govoplan-services==0.1.19",
|
||||
"govoplan-soap==0.1.19",
|
||||
"govoplan-tasks==0.1.20",
|
||||
"govoplan-templates==0.1.18",
|
||||
"govoplan-tickets==0.1.20",
|
||||
"govoplan-transparency==0.1.19",
|
||||
"govoplan-views==0.1.19",
|
||||
"govoplan-voting==0.1.18",
|
||||
"govoplan-wiki==0.1.20",
|
||||
"govoplan-workflow==0.1.21",
|
||||
"govoplan-workflow-engine==0.1.19",
|
||||
"govoplan-tasks==0.1.23",
|
||||
"govoplan-templates==0.1.22",
|
||||
"govoplan-tickets==0.1.23",
|
||||
"govoplan-transparency==0.1.20",
|
||||
"govoplan-views==0.1.22",
|
||||
"govoplan-voting==0.1.21",
|
||||
"govoplan-wiki==0.1.22",
|
||||
"govoplan-workflow==0.1.23",
|
||||
"govoplan-workflow-engine==0.1.21",
|
||||
"govoplan-xrechnung==0.1.21",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
@@ -91,6 +91,12 @@ Form launch, persisted submission provenance, idempotent replay, resumable
|
||||
assisted intake with enforced read-back evidence, and a durable Workflow handoff
|
||||
that remains visible through Tasks after the database session is reopened and
|
||||
disappears only after the Workflow Engine records completion.
|
||||
Core's production-component browser conformance suite additionally executes the
|
||||
German self-service and assisted Anwohnerparkausweis paths at desktop and mobile
|
||||
widths. It proves native keyboard order, accessible names and landmarks, WCAG
|
||||
2.1 A/AA automation, responsive geometry, first-draft persistence, and mixed
|
||||
per-field person/document/system provenance. Physical screen-reader spot checks
|
||||
remain target-environment release evidence.
|
||||
Module-level Records source tests prove exact Form submission, Case revision,
|
||||
and Decision revision filing. Target-environment browser accessibility,
|
||||
production identity and delivery, a named archive profile, and recovery evidence
|
||||
|
||||
@@ -2,7 +2,7 @@ bandit>=1.8,<2
|
||||
click>=8.3.3
|
||||
filelock>=3.20.3
|
||||
idna>=3.15
|
||||
pip>=26.1.2
|
||||
pip>=26.2
|
||||
pip-audit>=2.9,<3
|
||||
python-multipart>=0.0.31
|
||||
radon>=6,<7
|
||||
|
||||
@@ -56,7 +56,7 @@ httpx2>=2.5,<3
|
||||
filelock>=3.20.3
|
||||
idna>=3.15
|
||||
jsonschema>=4,<5
|
||||
pip>=26.1.2
|
||||
pip>=26.2
|
||||
pip-audit>=2.9,<3
|
||||
pytest>=9.0.3,<10
|
||||
pygments>=2.20,<3
|
||||
|
||||
+15
-15
@@ -1,18 +1,18 @@
|
||||
# Whole-product release install from immutable, independently versioned module tags.
|
||||
# Only add a module after its referenced tag has been published.
|
||||
../govoplan-core[server]
|
||||
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.18
|
||||
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.18
|
||||
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.18
|
||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.20
|
||||
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.20
|
||||
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.19
|
||||
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.20
|
||||
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.19
|
||||
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.18
|
||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.20
|
||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.22
|
||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.24
|
||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.18
|
||||
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.20
|
||||
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.19
|
||||
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
|
||||
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
|
||||
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
|
||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.26
|
||||
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
|
||||
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
|
||||
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
|
||||
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
|
||||
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
|
||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.27
|
||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.28
|
||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.29
|
||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.24
|
||||
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
|
||||
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
|
||||
|
||||
+4
-2
@@ -85,13 +85,15 @@
|
||||
"The configured applicant email issues a short-lived, hash-only status link through Notifications and exposes only the bounded status timeline.",
|
||||
"An idempotent replay returns the same persisted submission.",
|
||||
"The human review handoff survives a database-session restart and remains visible in Tasks until completion.",
|
||||
"The production self-service and assisted WebUI paths preserve keyboard order, accessible names, WCAG 2.1 A/AA automation, and responsive geometry at desktop and mobile widths.",
|
||||
"The assisted operator can assign independent source, confidence, and governed declaring-party, document, or system references to every populated field before immutable read-back.",
|
||||
"The formal decision retains party, mandate, legal-basis, evidence, delivery, review, and exact revision references.",
|
||||
"The Case-bound payment handoff creates a replay-safe obligation and accepts a full manual receipt only with exact amount, currency, transaction reference, and immutable evidence.",
|
||||
"Forms Runtime, Cases, and Decisions can expose exact snapshots for explicit eAkte filing."
|
||||
],
|
||||
"manual_or_target": [
|
||||
"Complete the digital journey with keyboard and screen reader at desktop and mobile widths.",
|
||||
"Complete the assisted operator journey with keyboard and screen reader at desktop and mobile widths.",
|
||||
"Perform physical screen-reader spot checks for the digital journey at desktop and mobile widths.",
|
||||
"Perform physical screen-reader spot checks for the assisted operator journey at desktop and mobile widths.",
|
||||
"Open, resend, expire, and revoke the applicant status link with keyboard and screen reader at desktop and mobile widths.",
|
||||
"Verify the configured Postbox or external delivery provider, including unknown outcome and reconciliation.",
|
||||
"Restore the pinned composition and reconstruct the exact form, case, decision, delivery evidence, and eAkte chronology.",
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { findTypeOnlyJsxImports } from "../tools/checks/check-jsx-value-imports.mjs";
|
||||
|
||||
const findings = (source) => findTypeOnlyJsxImports([{ path: "/fixture.tsx", source }]).map((item) => item.component);
|
||||
assert.deepEqual(findings('import type { FormGrid, AuthInfo } from "@govoplan/core-webui"; const page = <Dialog><FormGrid /></Dialog>;'), ["FormGrid"]);
|
||||
assert.deepEqual(findings('import { type FormGrid as Layout } from "ui"; const page = <Layout>Content</Layout>;'), ["Layout"]);
|
||||
assert.deepEqual(findings('import type Layout from "ui"; const page = <Layout />;'), ["Layout"]);
|
||||
assert.deepEqual(findings('import type * as ui from "ui"; const page = <ui.Layout />;'), ["ui.Layout"]);
|
||||
assert.deepEqual(findings('import type { FormGrid } from "ui"; const page = <div title={<FormGrid />} />;'), ["FormGrid"]);
|
||||
assert.deepEqual(findings('import { FormGrid, type AuthInfo } from "ui"; const page = <FormGrid />;'), []);
|
||||
assert.deepEqual(findings('import type { FormGrid } from "ui"; function Page({ FormGrid }: Props) { return <FormGrid />; }'), []);
|
||||
assert.deepEqual(findings('import type * as ui from "ui"; function Page(ui: RuntimeControls) { return <ui.Layout />; }'), []);
|
||||
assert.deepEqual(findings('import type { Layout } from "ui"; const page: Layout = {};'), []);
|
||||
assert.deepEqual(findings('import type { input } from "ui"; const page = <input />;'), []);
|
||||
console.log("JSX runtime-import AST regression tests passed (10 cases).");
|
||||
@@ -1,8 +1,10 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import redirect_stderr, redirect_stdout
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import subprocess
|
||||
@@ -36,6 +38,7 @@ import govoplan_deploy.cli as deployment_cli # noqa: E402
|
||||
from govoplan_deploy.capabilities import ( # noqa: E402
|
||||
capability_change_impacts,
|
||||
infrastructure_capability_document,
|
||||
infrastructure_dependency_inventory_from_mapping,
|
||||
)
|
||||
from govoplan_deploy.cluster_evidence import ( # noqa: E402
|
||||
collect_kubernetes_evidence,
|
||||
@@ -87,6 +90,41 @@ def _kubernetes_test_deployment(component: str, replicas: int) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _dependency_inventory(
|
||||
installation_id: str,
|
||||
*,
|
||||
generated_at: datetime | None = None,
|
||||
) -> dict:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"installation_id": installation_id,
|
||||
"generated_at": (generated_at or datetime.now(UTC)).isoformat(),
|
||||
"complete": True,
|
||||
"inspected_capability_ids": ["coordination.redis", "mail.smtp"],
|
||||
"providers": [
|
||||
{
|
||||
"module_id": "mail",
|
||||
"state": "complete",
|
||||
"capability_ids": ["mail.smtp"],
|
||||
"dependency_count": 1,
|
||||
}
|
||||
],
|
||||
"dependencies": [
|
||||
{
|
||||
"capability_id": "mail.smtp",
|
||||
"module_id": "mail",
|
||||
"dependency_type": "smtp_endpoint",
|
||||
"dependency_ref": "endpoint:17",
|
||||
"state": "active",
|
||||
"scope": "system",
|
||||
"summary": "Persisted SMTP endpoint has one credential binding.",
|
||||
"metrics": {"credential_binding_count": 1},
|
||||
"required_action": "Rebind or migrate this SMTP endpoint.",
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class DeploymentInstallerTests(unittest.TestCase):
|
||||
def test_kubernetes_evidence_requires_two_node_spread_and_safe_runtime(
|
||||
self,
|
||||
@@ -742,6 +780,11 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertIn("redis", compose["services"])
|
||||
self.assertIn("worker", compose["services"])
|
||||
self.assertIn("load-balancer", compose["services"])
|
||||
self.assertEqual(
|
||||
"haproxy:3.2.23-alpine@sha256:"
|
||||
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e",
|
||||
compose["services"]["load-balancer"]["image"],
|
||||
)
|
||||
self.assertNotIn("test-mail", compose["services"])
|
||||
self.assertEqual(
|
||||
["127.0.0.1:8080:8080"],
|
||||
@@ -1034,6 +1077,28 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertNotIn("old-secret", impacts["database.postgresql"].detail)
|
||||
self.assertNotIn("new-secret", impacts["database.postgresql"].detail)
|
||||
|
||||
def test_capability_impact_includes_provider_dependency_evidence(self) -> None:
|
||||
previous_spec = default_spec(mail_mode="test-mail", module_set="full")
|
||||
desired_spec = default_spec(mail_mode="disabled", module_set="full")
|
||||
inventory = infrastructure_dependency_inventory_from_mapping(
|
||||
_dependency_inventory(previous_spec.installation_id)
|
||||
)
|
||||
|
||||
impacts = {
|
||||
item.capability_id: item
|
||||
for item in capability_change_impacts(
|
||||
infrastructure_capability_document(previous_spec, {}),
|
||||
infrastructure_capability_document(desired_spec, {}),
|
||||
dependency_inventory=inventory,
|
||||
)
|
||||
}
|
||||
|
||||
mail = impacts["mail.smtp"]
|
||||
self.assertTrue(mail.inventory_inspected)
|
||||
self.assertEqual("endpoint:17", mail.actual_dependencies[0].dependency_ref)
|
||||
self.assertIn("mail:endpoint:17", mail.detail)
|
||||
self.assertIn("Rebind or migrate", mail.required_action)
|
||||
|
||||
def test_replica_counts_drive_compose_and_load_balancer_discovery(self) -> None:
|
||||
spec = default_spec(
|
||||
storage_mode="garage",
|
||||
@@ -1085,8 +1150,26 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertEqual(1, parsed.replicas.web)
|
||||
self.assertEqual(1, parsed.replicas.worker)
|
||||
self.assertEqual("managed", parsed.components.load_balancer.mode)
|
||||
self.assertEqual(
|
||||
default_spec().components.load_balancer.image,
|
||||
parsed.components.load_balancer.image,
|
||||
)
|
||||
self.assertEqual("local", parsed.ingress.mode)
|
||||
|
||||
def test_load_balancer_patch_does_not_rewrite_an_existing_image(self) -> None:
|
||||
for image in (
|
||||
"haproxy:3.2.21-alpine",
|
||||
"registry.example.test/haproxy@sha256:" + "a" * 64,
|
||||
):
|
||||
with self.subTest(image=image):
|
||||
saved = default_spec(load_balancer_image=image).to_dict()
|
||||
|
||||
restored = parse_spec(json.loads(json.dumps(saved)))
|
||||
compose = render_compose(restored)
|
||||
|
||||
self.assertEqual(image, restored.components.load_balancer.image)
|
||||
self.assertEqual(image, compose["services"]["load-balancer"]["image"])
|
||||
|
||||
def test_compose_contains_no_secret_values(self) -> None:
|
||||
spec = default_spec()
|
||||
values = initial_secrets(spec)
|
||||
@@ -1221,6 +1304,65 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
for check in second_plan.checks
|
||||
)
|
||||
)
|
||||
self.assertTrue(second_plan.blocked)
|
||||
self.assertTrue(
|
||||
any(
|
||||
check.id == "capability.dependency_inventory.missing"
|
||||
and check.level == "error"
|
||||
for check in second_plan.checks
|
||||
)
|
||||
)
|
||||
|
||||
atomic_write(
|
||||
paths.dependency_inventory,
|
||||
canonical_json(_dependency_inventory(second_spec.installation_id)),
|
||||
mode=0o600,
|
||||
)
|
||||
evidenced_plan = build_plan(
|
||||
second_spec,
|
||||
paths,
|
||||
include_host_checks=False,
|
||||
)
|
||||
|
||||
self.assertFalse(
|
||||
any(
|
||||
check.level == "error"
|
||||
and check.id.startswith("capability.dependency_inventory.")
|
||||
for check in evidenced_plan.checks
|
||||
)
|
||||
)
|
||||
self.assertEqual(
|
||||
"endpoint:17",
|
||||
{
|
||||
item.capability_id: item
|
||||
for item in evidenced_plan.capability_impacts
|
||||
}["mail.smtp"].actual_dependencies[0].dependency_ref,
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
check.id == "capability.dependency_inventory.current"
|
||||
and check.level == "ok"
|
||||
for check in evidenced_plan.checks
|
||||
)
|
||||
)
|
||||
|
||||
stale = _dependency_inventory(
|
||||
second_spec.installation_id,
|
||||
generated_at=datetime.now(UTC) - timedelta(minutes=6),
|
||||
)
|
||||
atomic_write(
|
||||
paths.dependency_inventory,
|
||||
canonical_json(stale),
|
||||
mode=0o600,
|
||||
)
|
||||
stale_plan = build_plan(second_spec, paths, include_host_checks=False)
|
||||
self.assertTrue(stale_plan.blocked)
|
||||
self.assertTrue(
|
||||
any(
|
||||
check.id == "capability.dependency_inventory.stale"
|
||||
for check in stale_plan.checks
|
||||
)
|
||||
)
|
||||
|
||||
def test_secret_change_is_planned_without_exposing_secret_values(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
|
||||
@@ -1330,6 +1472,54 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
)[0],
|
||||
)
|
||||
|
||||
def test_cli_collects_bounded_private_dependency_inventory(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
|
||||
root = Path(directory) / "installation"
|
||||
self.assertEqual(
|
||||
0,
|
||||
run_cli(
|
||||
[
|
||||
"init",
|
||||
"--non-interactive",
|
||||
"--directory",
|
||||
str(root),
|
||||
]
|
||||
)[0],
|
||||
)
|
||||
payload = _dependency_inventory("govoplan-local")
|
||||
response = MagicMock()
|
||||
response.__enter__.return_value = response
|
||||
response.geturl.return_value = "https://ops.example.test/inventory"
|
||||
response.read.return_value = json.dumps(payload).encode("utf-8")
|
||||
fetch = MagicMock(return_value=response)
|
||||
|
||||
with (
|
||||
patch.dict(os.environ, {"TEST_OPS_KEY": "secret-api-key"}),
|
||||
patch.object(deployment_cli, "urlopen", fetch),
|
||||
):
|
||||
result, stdout, stderr = run_cli(
|
||||
[
|
||||
"collect-infrastructure-inventory",
|
||||
"--directory",
|
||||
str(root),
|
||||
"--ops-url",
|
||||
"https://ops.example.test/inventory",
|
||||
"--api-key-env",
|
||||
"TEST_OPS_KEY",
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(0, result, stderr)
|
||||
self.assertIn("1 record(s)", stdout)
|
||||
evidence_path = root / "infrastructure-dependency-inventory.json"
|
||||
self.assertEqual(0o600, stat.S_IMODE(evidence_path.stat().st_mode))
|
||||
self.assertNotIn(
|
||||
"secret-api-key",
|
||||
evidence_path.read_text(encoding="utf-8"),
|
||||
)
|
||||
request = fetch.call_args.args[0]
|
||||
self.assertEqual("secret-api-key", request.get_header("X-api-key"))
|
||||
|
||||
def test_cli_requires_external_url_when_switching_from_managed(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
|
||||
root = Path(directory) / "installation"
|
||||
|
||||
@@ -0,0 +1,466 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from contextlib import ExitStack
|
||||
import csv
|
||||
from copy import deepcopy
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
from urllib.parse import quote
|
||||
import zipfile
|
||||
|
||||
from cryptography.hazmat.primitives import serialization
|
||||
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import full_catalog # noqa: E402
|
||||
from govoplan_release.artifact_identity import selected_artifact_identity_issues # noqa: E402
|
||||
from govoplan_release.catalog import canonical_hash # noqa: E402
|
||||
from govoplan_release.model import RepositorySpec # noqa: E402
|
||||
from govoplan_release.registry_reference import registry_entry_source # noqa: E402
|
||||
from govoplan_release.selective_catalog import ( # noqa: E402
|
||||
load_authenticated_catalog_base, public_key_base64, signature,
|
||||
)
|
||||
from govoplan_release.source_provenance import ( # noqa: E402
|
||||
SourceTagProvenanceIssue, catalog_source_selection,
|
||||
registered_source_origin_issues,
|
||||
)
|
||||
from govoplan_release.version_alignment import candidate_catalog_version_issues # noqa: E402
|
||||
|
||||
|
||||
class FullRegistryCatalogTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.web = self.root / "addideas-govoplan-website"
|
||||
self.wheels = self.root / "wheels"
|
||||
self.npm = self.root / "npm"
|
||||
self.wheels.mkdir(mode=0o700)
|
||||
self.npm.mkdir(mode=0o700)
|
||||
self.key = Ed25519PrivateKey.generate()
|
||||
self.keypath = self.root / "key.pem"
|
||||
self.keypath.write_bytes(self.key.private_bytes(
|
||||
serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
|
||||
serialization.NoEncryption(),
|
||||
))
|
||||
self.keypath.chmod(0o600)
|
||||
self.keyring = {
|
||||
"keyring_version": "1", "keys": [{
|
||||
"key_id": "known-key", "public_key": public_key_base64(self.key),
|
||||
"status": "active",
|
||||
}],
|
||||
}
|
||||
self.base = {
|
||||
"catalog_version": "1", "channel": "stable", "sequence": 1,
|
||||
"core_release": {"version": "1.0.0"}, "modules": [],
|
||||
"release": {},
|
||||
}
|
||||
self.write_base()
|
||||
self.package_set = {
|
||||
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
|
||||
"registries": {
|
||||
"python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
|
||||
"npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/",
|
||||
},
|
||||
"python": [self.package("govoplan-core"), self.package("govoplan-demo")],
|
||||
"webui": [self.package("govoplan-core", webui=True)],
|
||||
}
|
||||
self.seal(self.package_set, "package_set_sha256")
|
||||
self.lock = {
|
||||
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
|
||||
"registries": self.package_set["registries"],
|
||||
"package_set_sha256": self.package_set["package_set_sha256"],
|
||||
"python": [], "webui": [],
|
||||
}
|
||||
for row in self.package_set["python"]:
|
||||
path = self.wheel(row["name"])
|
||||
url = full_catalog._tool("resolve-package-artifacts")["_python_artifact_url"](
|
||||
self.package_set["registries"]["python"], package=row, filename=path.name,
|
||||
)
|
||||
self.lock["python"].append(self.artifact(row, path, url))
|
||||
npm_package = self.package_set["webui"][0]
|
||||
npm_path = self.npm / "govoplan-core-webui-1.2.3.tgz"
|
||||
self.tarball(npm_path, "@govoplan/core-webui", "1.2.3")
|
||||
url = self.package_set["registries"]["npm"] + quote(npm_package["name"], safe="") + "/-/1.2.3/core-webui-1.2.3.tgz"
|
||||
row = self.artifact(npm_package, npm_path, url)
|
||||
row["integrity"] = "sha512-" + base64.b64encode(hashlib.sha512(npm_path.read_bytes()).digest()).decode()
|
||||
self.lock["webui"].append(row)
|
||||
self.seal(self.lock, "lock_sha256")
|
||||
self.set_path = self.root / "package-set.json"
|
||||
self.lock_path = self.root / "package-lock.json"
|
||||
self.write_inputs()
|
||||
self.output = self.root / "candidate"
|
||||
|
||||
@staticmethod
|
||||
def package(repo: str, *, webui: bool = False) -> dict:
|
||||
row = {
|
||||
"name": "@govoplan/core-webui" if webui else repo, "version": "1.2.3",
|
||||
"repository": repo, "tag": "v1.2.3",
|
||||
"commit": ("a" if repo == "govoplan-core" else "b") * 40,
|
||||
}
|
||||
if not webui:
|
||||
row["extras"] = ["server"] if repo == "govoplan-core" else []
|
||||
return row
|
||||
|
||||
@staticmethod
|
||||
def artifact(package: dict, path: Path, url: str) -> dict:
|
||||
encoded = path.read_bytes()
|
||||
return {**package, "filename": path.name, "url": url,
|
||||
"sha256": hashlib.sha256(encoded).hexdigest(), "size": len(encoded)}
|
||||
|
||||
@staticmethod
|
||||
def seal(payload: dict, field: str) -> None:
|
||||
payload.pop(field, None)
|
||||
payload[field] = hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
|
||||
|
||||
def write_inputs(self) -> None:
|
||||
self.set_path.write_text(json.dumps(self.package_set))
|
||||
self.lock_path.write_text(json.dumps(self.lock))
|
||||
|
||||
def write_base(self) -> None:
|
||||
self.base.pop("signatures", None)
|
||||
self.base["signatures"] = [signature(self.base, key_id="known-key", private_key=self.key)]
|
||||
folder = self.web / "public/catalogs/v1"
|
||||
(folder / "channels").mkdir(parents=True, exist_ok=True)
|
||||
(folder / "channels/stable.json").write_text(json.dumps(self.base))
|
||||
(folder / "keyring.json").write_text(json.dumps(self.keyring))
|
||||
|
||||
def wheel(self, package: str) -> Path:
|
||||
stem = package.replace("-", "_")
|
||||
info = f"{stem}-1.2.3.dist-info"
|
||||
files = {
|
||||
f"{stem}/__init__.py": b"VALUE = 1\n",
|
||||
f"{info}/METADATA": f"Metadata-Version: 2.1\nName: {package}\nVersion: 1.2.3\n".encode(),
|
||||
f"{info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
|
||||
}
|
||||
record = io.StringIO()
|
||||
writer = csv.writer(record, lineterminator="\n")
|
||||
for name, value in files.items():
|
||||
writer.writerow((name, "", len(value)))
|
||||
writer.writerow((f"{info}/RECORD", "", ""))
|
||||
files[f"{info}/RECORD"] = record.getvalue().encode()
|
||||
path = self.wheels / f"{stem}-1.2.3-py3-none-any.whl"
|
||||
with zipfile.ZipFile(path, "w") as archive:
|
||||
for name, value in files.items():
|
||||
archive.writestr(name, value)
|
||||
return path
|
||||
|
||||
@staticmethod
|
||||
def tarball(path: Path, name: str, version: str, *, duplicate: bool = False) -> None:
|
||||
encoded = json.dumps({"name": name, "version": version}).encode()
|
||||
with tarfile.open(path, "w:gz") as archive:
|
||||
for _ in range(2 if duplicate else 1):
|
||||
member = tarfile.TarInfo("package/package.json")
|
||||
member.size = len(encoded)
|
||||
archive.addfile(member, io.BytesIO(encoded))
|
||||
|
||||
def build(self, *, provenance_errors=(), origin_errors=()) -> dict:
|
||||
registry_generator = full_catalog._tool("generate-release-catalog")
|
||||
tools = {name: dict(full_catalog._tool(name)) for name in (
|
||||
"generate-release-catalog", "generate-release-package-set", "resolve-package-artifacts",
|
||||
)}
|
||||
tools["generate-release-package-set"]["generate_package_set"] = lambda **kwargs: self.package_set
|
||||
self.provenance = {
|
||||
row["repository"]: {"commit_sha": row["commit"], "tag_object_sha": str(index + 1) * 40}
|
||||
for index, row in enumerate(self.package_set["python"])
|
||||
}
|
||||
entry = {
|
||||
"module_id": "demo", "name": "Demo", "version": "1.2.3",
|
||||
"python_package": "govoplan-demo",
|
||||
"python_ref": "govoplan-demo @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-demo.git@v1.2.3",
|
||||
}
|
||||
with ExitStack() as stack:
|
||||
stack.enter_context(patch.object(full_catalog, "_tool", side_effect=lambda name: tools[name]))
|
||||
stack.enter_context(patch.dict(registry_generator["_catalog_payload"].__globals__, {
|
||||
"synthesize_repository_catalog_entries": lambda **kwargs: (dict(entry),),
|
||||
}))
|
||||
stack.enter_context(patch.object(full_catalog, "enforce_selected_version_alignment"))
|
||||
stack.enter_context(patch.object(full_catalog, "registered_source_origin_issues", return_value=origin_errors))
|
||||
self.provenance_check = stack.enter_context(patch.object(full_catalog, "source_tag_provenance_issues", return_value=provenance_errors))
|
||||
stack.enter_context(patch.object(full_catalog, "selected_source_provenance", return_value=self.provenance))
|
||||
return full_catalog.build_full_registry_candidate(
|
||||
package_set_path=self.set_path, package_lock_path=self.lock_path,
|
||||
wheelhouse=self.wheels, webui_packages=self.npm, output_dir=self.output,
|
||||
selected_repositories=("govoplan-core",),
|
||||
signing_keys=(f"known-key={self.keypath}",), workspace_root=self.root,
|
||||
)
|
||||
|
||||
def candidate(self) -> dict:
|
||||
return json.loads((self.output / "channels/stable.json").read_text())
|
||||
|
||||
def test_full_candidate_uses_registry_bytes_and_preserves_unchanged_tag_provenance(self) -> None:
|
||||
result = self.build()
|
||||
candidate = self.candidate()
|
||||
self.assertEqual("ready", result["status"])
|
||||
self.assertEqual(2, result["package_count"])
|
||||
self.assertEqual(1, result["selected_count"])
|
||||
self.assertEqual(self.keyring, json.loads((self.output / "keyring.json").read_text()))
|
||||
self.assertEqual(canonical_hash(self.keyring), candidate["release"]["keyring_sha256"])
|
||||
self.assertEqual(2, len(candidate["release"]["artifacts"]))
|
||||
self.assertIn("/pypi/files/", candidate["core_release"]["python_ref"])
|
||||
self.assertEqual((), candidate_catalog_version_issues(candidate))
|
||||
self.assertEqual((), selected_artifact_identity_issues(candidate))
|
||||
sources = catalog_source_selection(candidate)
|
||||
self.assertEqual((), sources.issues)
|
||||
self.assertEqual({"govoplan-core": "1.2.3"}, sources.selected_versions)
|
||||
self.assertEqual({"govoplan-core": "1.2.3", "govoplan-demo": "1.2.3"}, sources.all_versions)
|
||||
self.assertEqual("b" * 40, sources.selected_commits["govoplan-demo"])
|
||||
self.assertEqual("2" * 40, sources.selected_tag_objects["govoplan-demo"])
|
||||
self.assertEqual(2, self.provenance_check.call_count)
|
||||
for call in self.provenance_check.call_args_list:
|
||||
self.assertEqual({"govoplan-core"}, call.kwargs["require_head_repos"])
|
||||
for path in [self.output, *self.output.rglob("*")]:
|
||||
self.assertEqual(0o700 if path.is_dir() else 0o600, path.stat().st_mode & 0o777)
|
||||
|
||||
def test_legacy_base_is_authenticated_but_remains_rejected_by_selective(self) -> None:
|
||||
self.build()
|
||||
with self.assertRaisesRegex(ValueError, "does not pin"):
|
||||
load_authenticated_catalog_base(
|
||||
base_catalog=None, base_keyring=None, web_root=self.web,
|
||||
channel="stable", public_base_url="https://unused.example",
|
||||
signer_public_keys={"known-key": public_key_base64(self.key)},
|
||||
)
|
||||
|
||||
def test_injected_key_or_mismatched_pinned_keyring_is_rejected(self) -> None:
|
||||
for mutation in ("extra-key", "bad-hash"):
|
||||
with self.subTest(mutation=mutation):
|
||||
if mutation == "extra-key":
|
||||
self.keyring["keys"].append({"key_id": "injected", "status": "active", "public_key": public_key_base64(Ed25519PrivateKey.generate())})
|
||||
else:
|
||||
self.keyring["keys"] = self.keyring["keys"][:1]
|
||||
self.base["release"]["keyring_sha256"] = "f" * 64
|
||||
self.write_base()
|
||||
with self.assertRaises(ValueError):
|
||||
self.build()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_tampered_base_signature_is_rejected(self) -> None:
|
||||
path = self.web / "public/catalogs/v1/channels/stable.json"
|
||||
payload = json.loads(path.read_text())
|
||||
payload["sequence"] = 999
|
||||
path.write_text(json.dumps(payload))
|
||||
with self.assertRaisesRegex(ValueError, "signature verification"):
|
||||
self.build()
|
||||
|
||||
def test_wrong_registry_bytes_and_reused_candidate_fail_closed(self) -> None:
|
||||
self.build()
|
||||
original = (self.output / "channels/stable.json").read_bytes()
|
||||
with self.assertRaisesRegex(ValueError, "must not already exist"):
|
||||
self.build()
|
||||
self.assertEqual(original, (self.output / "channels/stable.json").read_bytes())
|
||||
self.output = self.root / "candidate-2"
|
||||
wheel = self.wheels / self.lock["python"][0]["filename"]
|
||||
with wheel.open("ab") as stream:
|
||||
stream.write(b"tampered")
|
||||
with self.assertRaisesRegex(ValueError, "bytes differ"):
|
||||
self.build()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_webui_identity_and_url_changes_are_rejected(self) -> None:
|
||||
row = self.lock["webui"][0]
|
||||
original = row["url"]
|
||||
for url in (
|
||||
original + "?alternate=true", original.replace("/-/1.2.3/", "/-/9.9.9/"),
|
||||
original.replace("/npm/", "/npm/../other/"),
|
||||
original.replace("/npm/", "/npm/%2e%2e/other/"),
|
||||
original.replace("/npm/", "/npm/%252e%252e/other/"),
|
||||
):
|
||||
with self.subTest(url=url):
|
||||
row["url"] = url
|
||||
self.seal(self.lock, "lock_sha256")
|
||||
self.write_inputs()
|
||||
with self.assertRaisesRegex(ValueError, "URL differs"):
|
||||
self.build()
|
||||
|
||||
def test_duplicate_missing_and_symlinked_artifacts_are_rejected(self) -> None:
|
||||
row = self.lock["webui"][0]
|
||||
self.lock["webui"].append(dict(row))
|
||||
with self.assertRaisesRegex(ValueError, "duplicate/missing"):
|
||||
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
|
||||
self.lock["webui"].pop()
|
||||
path = self.npm / row["filename"]
|
||||
moved = self.root / "moved.tgz"
|
||||
path.rename(moved)
|
||||
path.symlink_to(moved)
|
||||
with self.assertRaises(OSError):
|
||||
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
|
||||
|
||||
def test_archive_metadata_is_bounded_and_not_ambiguous(self) -> None:
|
||||
path = self.npm / "duplicate.tgz"
|
||||
self.tarball(path, "@govoplan/core-webui", "1.2.3", duplicate=True)
|
||||
with self.assertRaisesRegex(ValueError, "duplicate"):
|
||||
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
|
||||
|
||||
def test_origin_or_tag_provenance_failure_prevents_output(self) -> None:
|
||||
issue = SourceTagProvenanceIssue("govoplan-core", "v1.2.3", "wrong immutable identity")
|
||||
for kwargs in ({"origin_errors": (issue,)}, {"provenance_errors": (issue,)}):
|
||||
with self.subTest(kwargs=kwargs), self.assertRaisesRegex(ValueError, "gate failed"):
|
||||
self.build(**kwargs)
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_registered_source_origin_requires_exact_fetch_and_push_targets(self) -> None:
|
||||
repo = self.root / "govoplan-core"
|
||||
repo.mkdir()
|
||||
spec = RepositorySpec("govoplan-core", "system", "kernel", "git@example.test:trusted/core.git", "govoplan-core")
|
||||
with patch("govoplan_release.source_provenance.load_repository_specs", return_value=(spec,)):
|
||||
for targets in ((spec.remote, spec.remote), ("git@evil.test:core.git", spec.remote), (spec.remote, "git@evil.test:core.git")):
|
||||
with self.subTest(targets=targets), patch("govoplan_release.source_provenance.git_text", side_effect=targets):
|
||||
issues = registered_source_origin_issues(repo_versions={"govoplan-core": "1.2.3"}, workspace=self.root, remote="origin")
|
||||
self.assertEqual(targets != (spec.remote, spec.remote), bool(issues))
|
||||
|
||||
def test_registry_metadata_cannot_cross_wire_webui_or_archive_identity(self) -> None:
|
||||
self.build()
|
||||
candidate = self.candidate()
|
||||
entry = candidate["core_release"]
|
||||
entry["webui_package"] = "@govoplan/files-webui"
|
||||
with self.assertRaisesRegex(ValueError, "another source repository"):
|
||||
registry_entry_source(entry)
|
||||
candidate = self.candidate()
|
||||
candidate["release"]["artifacts"][0]["archive_sha256"] = "f" * 64
|
||||
self.assertIn("matching inspected wheel", " ".join(selected_artifact_identity_issues(candidate)))
|
||||
|
||||
def test_registry_version_and_selected_source_identity_must_agree(self) -> None:
|
||||
self.build()
|
||||
for field in ("commit_sha", "tag_object_sha"):
|
||||
candidate = self.candidate()
|
||||
candidate["release"]["selected_units"][0][field] = "f" * 40
|
||||
self.assertIn("differs", " ".join(issue.message for issue in catalog_source_selection(candidate).issues))
|
||||
candidate = self.candidate()
|
||||
candidate["modules"][0]["artifact_integrity"]["python"]["git_ref"] = "v9.9.9"
|
||||
self.assertTrue(candidate_catalog_version_issues(candidate))
|
||||
|
||||
def test_repeated_module_projections_must_bind_identical_python_and_webui_bytes(self) -> None:
|
||||
self.build()
|
||||
for kind in ("python", "webui"):
|
||||
for reversed_order in (False, True):
|
||||
with self.subTest(kind=kind, reversed_order=reversed_order):
|
||||
candidate = self.candidate()
|
||||
entry = deepcopy(candidate["core_release"])
|
||||
entry["module_id"] = "another-core-projection"
|
||||
artifact = entry["artifact_integrity"][kind]
|
||||
artifact["sha256"] = "f" * 64
|
||||
if kind == "python":
|
||||
entry["python_ref"] = artifact["ref"] = entry["python_ref"].split("#sha256=", 1)[0] + "#sha256=" + "f" * 64
|
||||
if reversed_order:
|
||||
original = candidate["core_release"]
|
||||
candidate["core_release"] = entry
|
||||
entry = original
|
||||
candidate["modules"].append(entry)
|
||||
self.assertIn(f"conflicting {kind}", " ".join(selected_artifact_identity_issues(candidate)))
|
||||
self.assertTrue(candidate_catalog_version_issues(candidate))
|
||||
candidate = self.candidate()
|
||||
repeated = deepcopy(candidate["modules"][0])
|
||||
repeated["module_id"] = "second-demo-projection"
|
||||
candidate["modules"].append(repeated)
|
||||
self.assertEqual((), selected_artifact_identity_issues(candidate))
|
||||
self.assertEqual((), candidate_catalog_version_issues(candidate))
|
||||
|
||||
def test_metadata_inspection_uses_the_opened_archive_not_a_replaced_path(self) -> None:
|
||||
path = self.npm / "original.tgz"
|
||||
replacement = self.npm / "replacement.tgz"
|
||||
saved = self.npm / "saved.tgz"
|
||||
self.tarball(path, "@govoplan/incorrect-webui", "1.2.3")
|
||||
self.tarball(replacement, "@govoplan/core-webui", "1.2.3")
|
||||
real_open = tarfile.open
|
||||
|
||||
def replace_path(*args, **kwargs):
|
||||
self.assertIn("fileobj", kwargs)
|
||||
path.rename(saved)
|
||||
replacement.rename(path)
|
||||
return real_open(*args, **kwargs)
|
||||
|
||||
with patch("govoplan_release.full_catalog.tarfile.open", side_effect=replace_path):
|
||||
with self.assertRaisesRegex(ValueError, "metadata differs"):
|
||||
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
|
||||
|
||||
def test_registry_url_provenance_rejects_package_version_and_traversal_mismatch(self) -> None:
|
||||
self.build()
|
||||
for old, new in (
|
||||
("/govoplan-core/1.2.3/", "/govoplan-files/1.2.3/"),
|
||||
("/govoplan-core/1.2.3/", "/govoplan-core/9.9.9/"),
|
||||
("/pypi/files/", "/pypi/files/%2e%2e/"),
|
||||
("/pypi/files/", "/pypi/files/%252e%252e/"),
|
||||
):
|
||||
with self.subTest(new=new):
|
||||
entry = self.candidate()["core_release"]
|
||||
artifact = entry["artifact_integrity"]["python"]
|
||||
artifact["url"] = artifact["url"].replace(old, new)
|
||||
artifact["ref"] = entry["python_ref"] = entry["python_ref"].replace(old, new)
|
||||
with self.assertRaises(ValueError):
|
||||
registry_entry_source(entry)
|
||||
|
||||
def test_package_set_must_match_fixed_meta_pins_not_just_its_own_hash(self) -> None:
|
||||
payload = deepcopy(self.package_set)
|
||||
payload["python"][1]["version"] = "9.9.9"
|
||||
self.seal(payload, "package_set_sha256")
|
||||
self.set_path.write_text(json.dumps(payload))
|
||||
with self.assertRaisesRegex(ValueError, "exact Meta full pins"):
|
||||
self.build()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_package_set_git_reads_ignore_caller_redirection(self) -> None:
|
||||
tool = full_catalog._tool("generate-release-package-set")
|
||||
with patch.dict(os.environ, {"GIT_DIR": "/outside", "GIT_CONFIG_GLOBAL": "/outside/config", "PATH": "/outside/bin"}):
|
||||
with patch("subprocess.check_output", return_value="a" * 40 + "\n") as execute:
|
||||
self.assertEqual("a" * 40, tool["_git"](self.root, "rev-parse", "HEAD"))
|
||||
self.assertEqual("/usr/bin/git", execute.call_args.args[0][0])
|
||||
self.assertNotIn("GIT_DIR", execute.call_args.kwargs["env"])
|
||||
self.assertEqual(os.devnull, execute.call_args.kwargs["env"]["GIT_CONFIG_GLOBAL"])
|
||||
self.assertEqual("/usr/bin:/bin", execute.call_args.kwargs["env"]["PATH"])
|
||||
|
||||
def test_unchanged_real_annotated_ancestor_is_valid_but_selecting_it_requires_head(self) -> None:
|
||||
from tests.test_release_source_provenance import git, git_text, make_repo
|
||||
from govoplan_release.source_provenance import source_tag_provenance_issues
|
||||
|
||||
workspace = self.root / "source-workspace"
|
||||
workspace.mkdir()
|
||||
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
|
||||
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
|
||||
git(repo, "push", "origin", "refs/tags/v1.2.3")
|
||||
tagged_commit = git_text(repo, "rev-parse", "HEAD")
|
||||
(repo / "workflow-only.txt").write_text("post-tag workflow repair\n")
|
||||
git(repo, "add", "workflow-only.txt")
|
||||
git(repo, "commit", "-m", "repair workflow without replacing package")
|
||||
git(repo, "push", "origin", "main")
|
||||
common = {
|
||||
"repo_versions": {"govoplan-access": "1.2.3"}, "workspace": workspace,
|
||||
"expected_commits": {"govoplan-access": tagged_commit},
|
||||
"expected_tag_objects": {"govoplan-access": git_text(repo, "rev-parse", "refs/tags/v1.2.3")},
|
||||
}
|
||||
self.assertEqual((), source_tag_provenance_issues(**common))
|
||||
issues = source_tag_provenance_issues(**common, require_head_repos=("govoplan-access",))
|
||||
self.assertIn("not selected HEAD", " ".join(issue.message for issue in issues))
|
||||
|
||||
def test_tagged_manifest_synthesis_ignores_local_git_replacement_objects(self) -> None:
|
||||
from tests.test_release_source_provenance import git, git_text, make_repo
|
||||
from govoplan_release.catalog_entry_synthesis import materialized_source_tree
|
||||
|
||||
workspace = self.root / "materialization-workspace"
|
||||
workspace.mkdir()
|
||||
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
|
||||
original = (repo / "pyproject.toml").read_text()
|
||||
tagged_commit = git_text(repo, "rev-parse", "HEAD")
|
||||
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
|
||||
(repo / "pyproject.toml").write_text(original.replace("1.2.3", "9.9.9"))
|
||||
git(repo, "add", "pyproject.toml")
|
||||
git(repo, "commit", "-m", "unreviewed replacement tree")
|
||||
git(repo, "replace", tagged_commit, git_text(repo, "rev-parse", "HEAD"))
|
||||
with patch.dict(os.environ, {"GIT_DIR": str(self.root / "outside"), "PATH": "/outside/bin"}):
|
||||
with materialized_source_tree(repo, source_ref="v1.2.3") as source:
|
||||
self.assertEqual(original, (source / "pyproject.toml").read_text())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -25,6 +25,11 @@ from govoplan_core.core.institutional import (
|
||||
TemporalRevision,
|
||||
service_launch_capability,
|
||||
)
|
||||
from govoplan_core.core.temporal import (
|
||||
TemporalDataContext,
|
||||
bind_temporal_data_context,
|
||||
reset_temporal_data_context,
|
||||
)
|
||||
from govoplan_cases.backend.party_context import CasePartyContext
|
||||
from govoplan_cases.backend.db.models import (
|
||||
CaseAccessGrant,
|
||||
@@ -134,6 +139,13 @@ class _Registry:
|
||||
|
||||
|
||||
class InstitutionalGovernanceJourneyTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
# Portal's effective_at does not replace the SQL provider's request-local
|
||||
# read clock. Keep both on the journey date, without bypassing validity
|
||||
# filtering or extending the fixture's finite publication interval.
|
||||
token = bind_temporal_data_context(TemporalDataContext(evaluated_at=NOW))
|
||||
self.addCleanup(reset_temporal_data_context, token)
|
||||
|
||||
def test_service_to_formal_outcome_retains_governed_context(self) -> None:
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
for table in (
|
||||
@@ -220,13 +232,44 @@ class InstitutionalGovernanceJourneyTests(unittest.TestCase):
|
||||
service_launch_capability("case"): object(),
|
||||
}
|
||||
)
|
||||
entry = PortalServiceDirectory(service_registry).list_entries(
|
||||
directory = PortalServiceDirectory(service_registry)
|
||||
for outside_interval in (
|
||||
service.temporal.valid_from - timedelta(microseconds=1),
|
||||
service.temporal.valid_to,
|
||||
):
|
||||
with self.subTest(outside_interval=outside_interval):
|
||||
token = bind_temporal_data_context(
|
||||
TemporalDataContext(evaluated_at=outside_interval)
|
||||
)
|
||||
try:
|
||||
# Keep Portal inside the valid interval: the real SQL
|
||||
# provider must still exclude a service outside its own
|
||||
# temporal read context, before Portal can project it.
|
||||
self.assertEqual(
|
||||
(),
|
||||
directory.list_entries(
|
||||
session,
|
||||
principal,
|
||||
tenant_id="tenant-1",
|
||||
effective_at=NOW,
|
||||
audiences=("resident",),
|
||||
),
|
||||
)
|
||||
finally:
|
||||
reset_temporal_data_context(token)
|
||||
|
||||
entries = directory.list_entries(
|
||||
session,
|
||||
principal,
|
||||
tenant_id="tenant-1",
|
||||
effective_at=NOW,
|
||||
audiences=("resident",),
|
||||
)[0]
|
||||
)
|
||||
self.assertEqual(
|
||||
(service.reference,),
|
||||
tuple(entry.definition.reference for entry in entries),
|
||||
)
|
||||
entry = entries[0]
|
||||
self.assertTrue(entry.available)
|
||||
intake = CaseServiceIntake().plan(
|
||||
entry.definition,
|
||||
|
||||
@@ -287,7 +287,10 @@ class InstitutionalServiceJourneyTests(unittest.TestCase):
|
||||
self.assertEqual("de-DE", JOURNEY["locale"])
|
||||
self.assertEqual("email_link", JOURNEY["status_access"]["mode"])
|
||||
self.assertEqual("manual", JOURNEY["payment"]["mode"])
|
||||
self.assertEqual(8, len(JOURNEY["acceptance"]["automated"]))
|
||||
automated = JOURNEY["acceptance"]["automated"]
|
||||
self.assertEqual(10, len(automated))
|
||||
self.assertTrue(any("desktop and mobile" in item for item in automated))
|
||||
self.assertTrue(any("independent source" in item for item in automated))
|
||||
self.assertEqual(6, len(JOURNEY["acceptance"]["manual_or_target"]))
|
||||
|
||||
def test_portal_launches_exact_form_revision_and_persists_submission(self) -> None:
|
||||
|
||||
@@ -0,0 +1,240 @@
|
||||
"""Local mixed-owner admission/recovery evidence, not production capacity certification.
|
||||
|
||||
All inputs are synthetic and held in memory. The spawn observer temporarily
|
||||
holds the admitted parent's handshake so the other owners encounter the same
|
||||
occupied slot deterministically. Children perform real XLSX, template and
|
||||
Dataflow work; there is no mocked process execution, database, or live service.
|
||||
The non-queuing gate promises retryable rejection, not scheduler fairness.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from io import BytesIO
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from openpyxl import Workbook
|
||||
from govoplan_connectors.backend.tabular_adapters import (
|
||||
parse_managed_tabular_content,
|
||||
)
|
||||
from govoplan_core.core.templates import TemplateRenderRequest
|
||||
from govoplan_core.security import bounded_process
|
||||
from govoplan_core.security.bounded_process import ProcessBudgetError
|
||||
from govoplan_core.settings import settings
|
||||
from govoplan_dataflow.backend.backends import execute_typed_graph
|
||||
from govoplan_dataflow.backend.schemas import (
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
GraphPosition,
|
||||
PipelineGraph,
|
||||
)
|
||||
from govoplan_templates.backend.rendering import _render_payload
|
||||
except ImportError as exc:
|
||||
raise unittest.SkipTest(
|
||||
"Mixed-owner isolation requires the optional module test environment."
|
||||
) from exc
|
||||
|
||||
|
||||
class IsolatedWorkCompositionTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
workbook = Workbook()
|
||||
workbook.active.append(["name"])
|
||||
workbook.active.append(["Ada"])
|
||||
stream = BytesIO()
|
||||
workbook.save(stream)
|
||||
workbook.close()
|
||||
self.workbook = stream.getvalue()
|
||||
self.graph = PipelineGraph(
|
||||
nodes=[
|
||||
GraphNode(
|
||||
id="source",
|
||||
type="source.inline",
|
||||
label="Source",
|
||||
position=GraphPosition(x=0, y=0),
|
||||
config={"source_name": "records", "rows": [{"name": "Ada"}]},
|
||||
),
|
||||
GraphNode(
|
||||
id="output",
|
||||
type="output",
|
||||
label="Output",
|
||||
position=GraphPosition(x=100, y=0),
|
||||
config={},
|
||||
),
|
||||
],
|
||||
edges=[GraphEdge(id="edge", source="source", target="output")],
|
||||
)
|
||||
|
||||
def xlsx(self):
|
||||
rows, sheet = parse_managed_tabular_content(
|
||||
self.workbook,
|
||||
filename="synthetic.xlsx",
|
||||
content_type=None,
|
||||
delimiter=",",
|
||||
sheet_name=None,
|
||||
)
|
||||
self.assertEqual(rows, ({"name": "Ada"},))
|
||||
self.assertEqual(sheet, "Sheet")
|
||||
return "xlsx"
|
||||
|
||||
def templates(self):
|
||||
payload, content_type, _pages = _render_payload(
|
||||
SimpleNamespace(name="Synthetic template"),
|
||||
SimpleNamespace(
|
||||
content_text="Hello {{item.name}}",
|
||||
content_html=None,
|
||||
template_type="letter",
|
||||
layout={},
|
||||
output_profiles=[],
|
||||
),
|
||||
request=TemplateRenderRequest(
|
||||
template_id="synthetic", output_format="text"
|
||||
),
|
||||
items=({"name": "Ada"},),
|
||||
)
|
||||
self.assertEqual(payload, b"Hello Ada")
|
||||
self.assertEqual(content_type, "text/plain; charset=utf-8")
|
||||
return "templates"
|
||||
|
||||
def dataflow(self):
|
||||
result = execute_typed_graph(self.graph, backend="reference")
|
||||
self.assertEqual(result.rows, [{"name": "Ada"}])
|
||||
return "dataflow"
|
||||
|
||||
def test_one_shared_slot_rejects_other_owners_and_all_retries_recover(self):
|
||||
owners = {
|
||||
"xlsx": self.xlsx,
|
||||
"templates": self.templates,
|
||||
"dataflow": self.dataflow,
|
||||
}
|
||||
processes = []
|
||||
modules = []
|
||||
hold_next = False
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
observer_lock = threading.Lock()
|
||||
maximum_unreaped = 0
|
||||
observer_timeouts = 0
|
||||
original_popen = bounded_process.subprocess.Popen
|
||||
|
||||
def observe_spawn(*args, **kwargs):
|
||||
nonlocal hold_next, maximum_unreaped, observer_timeouts
|
||||
process = original_popen(*args, **kwargs)
|
||||
with observer_lock:
|
||||
processes.append(process)
|
||||
modules.append(args[0][5])
|
||||
maximum_unreaped = max(
|
||||
maximum_unreaped, sum(item.returncode is None for item in processes)
|
||||
)
|
||||
should_hold = hold_next
|
||||
hold_next = False
|
||||
if should_hold:
|
||||
entered.set()
|
||||
if not release.wait(8):
|
||||
# Return control so the real runner's normal timeout and
|
||||
# process-group cleanup still own this child on test error.
|
||||
observer_timeouts += 1
|
||||
return process
|
||||
|
||||
def rejected(operation):
|
||||
try:
|
||||
operation()
|
||||
except Exception as exc:
|
||||
cause = exc
|
||||
while cause is not None and not isinstance(cause, ProcessBudgetError):
|
||||
cause = cause.__cause__
|
||||
self.assertIsInstance(cause, ProcessBudgetError)
|
||||
self.assertEqual(cause.code, "busy")
|
||||
return "busy"
|
||||
self.fail(
|
||||
"A different module admitted work while the shared slot was occupied."
|
||||
)
|
||||
|
||||
started = time.monotonic()
|
||||
busy_count = 0
|
||||
try:
|
||||
with (
|
||||
patch.object(settings, "isolated_process_concurrency", 1),
|
||||
patch.object(bounded_process.subprocess, "Popen", observe_spawn),
|
||||
ThreadPoolExecutor(max_workers=3) as executor,
|
||||
):
|
||||
for owner, operation in owners.items():
|
||||
with self.subTest(admitted_owner=owner):
|
||||
entered.clear()
|
||||
release.clear()
|
||||
hold_next = True
|
||||
holder = executor.submit(operation)
|
||||
try:
|
||||
self.assertTrue(
|
||||
entered.wait(5),
|
||||
"The admitted operation never spawned its real child.",
|
||||
)
|
||||
children_before = len(processes)
|
||||
others = [
|
||||
work for label, work in owners.items() if label != owner
|
||||
]
|
||||
denied = [
|
||||
executor.submit(rejected, work) for work in others
|
||||
]
|
||||
self.assertEqual(
|
||||
[future.result(timeout=5) for future in denied],
|
||||
["busy", "busy"],
|
||||
)
|
||||
busy_count += len(denied)
|
||||
self.assertEqual(len(processes), children_before)
|
||||
finally:
|
||||
release.set()
|
||||
self.assertEqual(holder.result(timeout=15), owner)
|
||||
self.assertEqual(bounded_process._active, 0)
|
||||
# Every rejected owner is retried through its real API.
|
||||
# Each must complete after the previous holder releases.
|
||||
for other in others:
|
||||
other()
|
||||
self.assertEqual(bounded_process._active, 0)
|
||||
finally:
|
||||
release.set()
|
||||
for process in processes:
|
||||
self.assertIsNotNone(process.returncode, "Worker was not reaped.")
|
||||
self.assertTrue(
|
||||
all(
|
||||
stream.closed
|
||||
for stream in (process.stdin, process.stdout, process.stderr)
|
||||
)
|
||||
)
|
||||
with self.assertRaises(ChildProcessError):
|
||||
os.waitpid(process.pid, os.WNOHANG)
|
||||
self.assertEqual(maximum_unreaped, 1)
|
||||
self.assertEqual(observer_timeouts, 0)
|
||||
self.assertEqual(len(processes), 9)
|
||||
self.assertEqual(busy_count, 6)
|
||||
self.assertEqual(
|
||||
set(modules),
|
||||
{
|
||||
"govoplan_connectors.backend.tabular_adapters",
|
||||
"govoplan_templates.backend.rendering",
|
||||
"govoplan_dataflow.backend.backends.reference",
|
||||
},
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"local_composition": {
|
||||
"successful_children": len(processes),
|
||||
"busy_rejections": busy_count,
|
||||
"maximum_unreaped_children": maximum_unreaped,
|
||||
"all_children_reaped": True,
|
||||
"seconds": round(time.monotonic() - started, 3),
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -62,22 +62,37 @@ class PackageRegistryReleaseTests(unittest.TestCase):
|
||||
encoding="utf-8"
|
||||
)
|
||||
)["project"]["version"]
|
||||
meta_package = ROOT / "packages/govoplan-meta/pyproject.toml"
|
||||
meta_project = tomllib.loads(
|
||||
meta_package.read_text(encoding="utf-8")
|
||||
)["project"]
|
||||
expected_tasks_pin = next(
|
||||
requirement
|
||||
for requirement in (
|
||||
*meta_project["dependencies"],
|
||||
*meta_project["optional-dependencies"]["full"],
|
||||
)
|
||||
if requirement.startswith("govoplan-tasks==")
|
||||
)
|
||||
selected = PACKAGE_SET.parse_meta_package(
|
||||
ROOT / "packages/govoplan-meta/pyproject.toml",
|
||||
meta_package,
|
||||
core_version=core_version,
|
||||
)
|
||||
|
||||
by_name = {item["name"]: item for item in selected}
|
||||
self.assertIn("govoplan-core", by_name)
|
||||
self.assertIn("govoplan-records", by_name)
|
||||
self.assertEqual("0.1.20", by_name["govoplan-tasks"]["version"])
|
||||
self.assertEqual(
|
||||
expected_tasks_pin,
|
||||
f"govoplan-tasks=={by_name['govoplan-tasks']['version']}",
|
||||
)
|
||||
|
||||
payload = PACKAGE_SET.generate_package_set(
|
||||
core_version=core_version,
|
||||
requirements=ROOT / "requirements-release.txt",
|
||||
workspace=ROOT.parent,
|
||||
profile="full",
|
||||
meta_package=ROOT / "packages/govoplan-meta/pyproject.toml",
|
||||
meta_package=meta_package,
|
||||
)
|
||||
self.assertEqual("full", payload["profile"])
|
||||
self.assertEqual(len(selected), len(payload["python"]))
|
||||
|
||||
@@ -22,11 +22,16 @@ class PackageSetDispatchTests(unittest.TestCase):
|
||||
def test_meta_package_resolves_to_exact_tagged_repository_targets(self) -> None:
|
||||
targets = MODULE.package_targets()
|
||||
|
||||
self.assertEqual(66, len(targets))
|
||||
self.assertEqual(66, len({target.distribution for target in targets}))
|
||||
self.assertEqual(73, len(targets))
|
||||
self.assertEqual(73, len({target.distribution for target in targets}))
|
||||
by_name = {target.distribution: target for target in targets}
|
||||
self.assertEqual("v0.1.14", by_name["govoplan-core"].tag)
|
||||
self.assertEqual("v0.1.8", by_name["govoplan-access"].tag)
|
||||
self.assertEqual("v0.1.38", by_name["govoplan-core"].tag)
|
||||
self.assertEqual("v0.1.22", by_name["govoplan-access"].tag)
|
||||
self.assertEqual("v0.1.20", by_name["govoplan-dms"].tag)
|
||||
self.assertEqual("v0.1.20", by_name["govoplan-erp"].tag)
|
||||
self.assertEqual("v0.1.20", by_name["govoplan-fit-connect"].tag)
|
||||
self.assertEqual("v0.1.23", by_name["govoplan-idm"].tag)
|
||||
self.assertEqual("v0.1.21", by_name["govoplan-xrechnung"].tag)
|
||||
self.assertTrue(by_name["govoplan-core"].tag_exists)
|
||||
self.assertTrue(by_name["govoplan-access"].has_webui)
|
||||
self.assertEqual(
|
||||
|
||||
@@ -96,6 +96,18 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
|
||||
with self.assertRaisesRegex(ValueError, "tracking_issue"):
|
||||
inventory._load_endpoint_declarations(path)
|
||||
|
||||
def test_bounded_workflow_read_apis_have_explicit_headless_declarations(self) -> None:
|
||||
declarations = inventory._load_endpoint_declarations(inventory.DEFAULT_ENDPOINT_DECLARATIONS)
|
||||
for path in (
|
||||
"/workflow/instances/summaries", "/workflow/instances/{}/summary",
|
||||
"/workflow/instances/{}/steps", "/workflow/instances/{}/events",
|
||||
):
|
||||
with self.subTest(path=path):
|
||||
entry = declarations[("govoplan-workflow-engine", "GET", path)]
|
||||
self.assertEqual("intentionally_headless", entry["category"])
|
||||
self.assertIn("current-authorized", entry["rationale"])
|
||||
self.assertIn("workflow.instance-history", entry["rationale"])
|
||||
|
||||
def test_inventory_reports_unclassified_and_stale_endpoint_declarations(
|
||||
self,
|
||||
) -> None:
|
||||
@@ -169,6 +181,69 @@ class PlatformInterfaceInventoryTests(unittest.TestCase):
|
||||
),
|
||||
)
|
||||
|
||||
def test_high_risk_help_baseline_is_validated(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "help-baseline.json"
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"maximum_missing_exact_help": 3,
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
3,
|
||||
inventory._load_high_risk_help_baseline(path)[
|
||||
"maximum_missing_exact_help"
|
||||
],
|
||||
)
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"maximum_missing_exact_help": -1,
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "non-negative integer"):
|
||||
inventory._load_high_risk_help_baseline(path)
|
||||
|
||||
def test_declaration_strict_mode_rejects_high_risk_help_regression(
|
||||
self,
|
||||
) -> None:
|
||||
result = {
|
||||
"translation_health": {"missing_catalog_entries": []},
|
||||
"api": {
|
||||
"unclassified_endpoints": [],
|
||||
"stale_endpoint_declarations": [],
|
||||
},
|
||||
"declaration_health": {},
|
||||
"help_health": {
|
||||
"invalid_risk_annotations": [],
|
||||
"unresolved_exact_high_risk_help": [],
|
||||
"high_risk_help_without_german": [],
|
||||
"missing_exact_high_risk_help": [{"id": "example.delete"}],
|
||||
"baseline_maximum_missing": 0,
|
||||
"baseline_regression": True,
|
||||
},
|
||||
}
|
||||
|
||||
self.assertEqual(
|
||||
[
|
||||
"1 high-risk controls lack exact F1 help; baseline permits at most 0"
|
||||
],
|
||||
inventory._strict_failures(
|
||||
result,
|
||||
check_translations=False,
|
||||
check_endpoints=False,
|
||||
check_declarations=True,
|
||||
),
|
||||
)
|
||||
|
||||
def test_fastapi_route_scanner_includes_router_prefix(self) -> None:
|
||||
tree = ast.parse(
|
||||
"""
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
@@ -17,6 +18,80 @@ from govoplan_release import git_state # noqa: E402
|
||||
|
||||
|
||||
class ReleaseGitStateTests(unittest.TestCase):
|
||||
def test_unset_ssh_address_family_preserves_original_command_and_operator_config(self) -> None:
|
||||
environment = git_state.sanitized_git_environment({})
|
||||
|
||||
self.assertEqual(
|
||||
[
|
||||
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
|
||||
],
|
||||
shlex.split(environment["GIT_SSH_COMMAND"]),
|
||||
)
|
||||
self.assertNotIn("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY", environment)
|
||||
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
|
||||
|
||||
def test_ssh_address_family_accepts_only_fixed_choices_and_survives_resanitizing(self) -> None:
|
||||
for family in ("any", "inet", "inet6"):
|
||||
with self.subTest(family=family):
|
||||
environment = git_state.sanitized_git_environment({
|
||||
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": family,
|
||||
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
|
||||
"GIT_SSH": "/attacker/ssh",
|
||||
"PATH": "/attacker/bin",
|
||||
})
|
||||
|
||||
self.assertEqual(
|
||||
[
|
||||
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
|
||||
"-o", f"AddressFamily={family}",
|
||||
],
|
||||
shlex.split(environment["GIT_SSH_COMMAND"]),
|
||||
)
|
||||
self.assertNotIn("GIT_SSH", environment)
|
||||
self.assertEqual("/usr/bin:/bin", environment["PATH"])
|
||||
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
|
||||
|
||||
def test_invalid_ssh_address_family_is_rejected_before_git_runs(self) -> None:
|
||||
for invalid in (
|
||||
"", "INET", "ipv4", " inet", "inet ", "inet\n",
|
||||
"inet; touch /not-executed", "inet -o StrictHostKeyChecking=no",
|
||||
"$(not-executed)",
|
||||
):
|
||||
with (
|
||||
self.subTest(value=invalid),
|
||||
patch.dict("os.environ", {"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": invalid}),
|
||||
patch.object(git_state.subprocess, "run") as run,
|
||||
):
|
||||
with self.assertRaisesRegex(
|
||||
ValueError, "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6",
|
||||
):
|
||||
git_state.git(Path("/workspace/govoplan-core"), "status", "--porcelain")
|
||||
run.assert_not_called()
|
||||
|
||||
def test_source_provenance_readback_keeps_family_but_discards_ssh_command_override(self) -> None:
|
||||
from govoplan_release.source_provenance import inspect_remote_tag
|
||||
|
||||
completed = subprocess.CompletedProcess(
|
||||
[], 0, f"{'a' * 40}\trefs/tags/v1.2.3\n{'b' * 40}\trefs/tags/v1.2.3^{{}}\n", "",
|
||||
)
|
||||
with (
|
||||
patch.dict("os.environ", {
|
||||
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": "inet",
|
||||
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
|
||||
}),
|
||||
patch("govoplan_release.repository_tag.subprocess.run", return_value=completed) as run,
|
||||
):
|
||||
result = inspect_remote_tag(
|
||||
path=Path("/workspace/govoplan-core"), remote="origin",
|
||||
remote_url="git@git.add-ideas.de:GovOPlaN/govoplan-core.git", tag="v1.2.3",
|
||||
)
|
||||
|
||||
self.assertEqual("b" * 40, result.commit)
|
||||
self.assertEqual(
|
||||
"/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8 -o AddressFamily=inet",
|
||||
run.call_args.kwargs["env"]["GIT_SSH_COMMAND"],
|
||||
)
|
||||
|
||||
def test_manifest_version_does_not_confuse_interface_versions(self) -> None:
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
from contextlib import redirect_stdout
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import shutil
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import meta_preparation # noqa: E402
|
||||
from govoplan_release.git_state import collect_repository_snapshot # noqa: E402
|
||||
from govoplan_release.meta_preparation import ( # noqa: E402
|
||||
MetaPreparationError,
|
||||
prepare_developer_meta_package,
|
||||
)
|
||||
from govoplan_release.model import RepositorySpec # noqa: E402
|
||||
from govoplan_release.selective_planner import build_selective_release_plan # noqa: E402
|
||||
from govoplan_release.version_metadata import ( # noqa: E402
|
||||
VersionMetadataError,
|
||||
apply_version_metadata_mutations,
|
||||
version_metadata_mutations,
|
||||
)
|
||||
import test_release_meta_source_tag as meta_fixture # noqa: E402
|
||||
from test_release_plan_guidance import dashboard # noqa: E402
|
||||
from test_release_repository_tag import create_release_repo, git, git_text # noqa: E402
|
||||
|
||||
|
||||
class MetaPreparationTests(unittest.TestCase):
|
||||
synchronize = meta_fixture.MetaSourceTagTests.synchronize
|
||||
|
||||
def setUp(self):
|
||||
meta_fixture.MetaSourceTagTests.setUp(self)
|
||||
self.operator = self.root / "operator"
|
||||
self.generator = (
|
||||
self.operator / "tools/release/generate-developer-meta-package.py"
|
||||
)
|
||||
self.generator.parent.mkdir(parents=True)
|
||||
shutil.copyfile(
|
||||
ROOT / "tools/release/generate-developer-meta-package.py", self.generator
|
||||
)
|
||||
self.enterContext(patch.object(meta_preparation, "META_ROOT", self.operator))
|
||||
|
||||
def prepare_core(self):
|
||||
apply_version_metadata_mutations(self.core, target_version="0.1.11")
|
||||
git(self.core, "add", ".")
|
||||
git(self.core, "commit", "-m", "Prepared synthetic Core target")
|
||||
|
||||
def preview(self, **kwargs):
|
||||
return prepare_developer_meta_package(
|
||||
repo_path=self.meta, target_version="0.1.11", **kwargs
|
||||
)
|
||||
|
||||
def apply(self, preview):
|
||||
return self.preview(
|
||||
apply=True, expected_receipt=preview["receipt"], confirm_out_of_run=True
|
||||
)
|
||||
|
||||
def test_full_canonical_preview_apply_and_shared_mutation_discovery(self):
|
||||
extra, remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-workflow-engine",
|
||||
version="0.2.3",
|
||||
)
|
||||
self.specs.append(
|
||||
{
|
||||
"name": extra.name,
|
||||
"path": extra.name,
|
||||
"category": "module",
|
||||
"subtype": "",
|
||||
"remote": str(remote),
|
||||
}
|
||||
)
|
||||
self.registry.write_text(json.dumps({"repositories": self.specs}))
|
||||
self.prepare_core()
|
||||
before = self.package.read_bytes()
|
||||
preview = self.preview()
|
||||
self.assertEqual("planned", preview["status"])
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
mutations = version_metadata_mutations(self.meta, target_version="0.1.11")
|
||||
self.assertEqual([meta_preparation.PACKAGE], [item.path for item in mutations])
|
||||
self.assertIn(b"govoplan-workflow-engine==0.2.3", mutations[0].after)
|
||||
self.assertIn(b"govoplan-core==0.1.11", mutations[0].after)
|
||||
result = self.apply(preview)
|
||||
self.assertEqual("prepared", result["status"])
|
||||
self.assertEqual(mutations[0].after, self.package.read_bytes())
|
||||
self.assertEqual(
|
||||
self.render(workspace=self.workspace, requirements=self.requirements),
|
||||
self.package.read_text(),
|
||||
)
|
||||
self.assertEqual(
|
||||
f"M {meta_preparation.PACKAGE}",
|
||||
git_text(self.meta, "status", "--porcelain"),
|
||||
)
|
||||
self.assertFalse(git_text(self.meta, "tag", "--list"))
|
||||
|
||||
def test_core_target_must_already_be_prepared(self):
|
||||
before = self.package.read_bytes()
|
||||
with self.assertRaisesRegex(MetaPreparationError, "Prepare and commit Core"):
|
||||
self.preview()
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_changed_requirements_receipt_blocks_before_any_output(self):
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
before = self.package.read_bytes()
|
||||
self.requirements.write_text(
|
||||
self.requirements.read_text() + "# reviewed different inputs\n"
|
||||
)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Changed synthetic requirements")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_source_change_immediately_before_effect_is_rechecked(self):
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
before = self.package.read_bytes()
|
||||
original = meta_preparation.preview_meta_mutation
|
||||
|
||||
def changed(**kwargs):
|
||||
result = original(**kwargs)
|
||||
self.requirements.write_text(
|
||||
self.requirements.read_text() + "# concurrent change\n"
|
||||
)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Concurrent synthetic change")
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
meta_preparation, "preview_meta_mutation", side_effect=changed
|
||||
):
|
||||
with self.assertRaisesRegex(MetaPreparationError, "changed before"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_core_full_package_and_operator_generator_are_receipt_bound(self):
|
||||
self.prepare_core()
|
||||
for path, repository in (
|
||||
(self.core / "pyproject.toml", self.core),
|
||||
(self.access / "pyproject.toml", self.access),
|
||||
(self.generator, None),
|
||||
):
|
||||
with self.subTest(input=path.name, repo=str(repository)):
|
||||
preview = self.preview()
|
||||
before = self.package.read_bytes()
|
||||
path.write_text(path.read_text() + "\n# changed frozen input\n")
|
||||
if repository is not None:
|
||||
git(repository, "add", ".")
|
||||
git(
|
||||
repository,
|
||||
"commit",
|
||||
"-m",
|
||||
"Changed synthetic composition input",
|
||||
)
|
||||
with self.assertRaisesRegex(MetaPreparationError, "changed since"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(before, self.package.read_bytes())
|
||||
|
||||
def test_post_write_source_change_is_reported_without_retry_or_rollback(self):
|
||||
from govoplan_release import version_metadata
|
||||
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
original = version_metadata._atomic_write
|
||||
|
||||
def changed(path, payload):
|
||||
original(path, payload)
|
||||
self.requirements.write_text(
|
||||
self.requirements.read_text() + "# concurrent after write\n"
|
||||
)
|
||||
|
||||
with patch.object(
|
||||
version_metadata, "_atomic_write", side_effect=changed
|
||||
) as writer:
|
||||
with self.assertRaisesRegex(
|
||||
meta_preparation.MetaPreparationAmbiguous, "write/post-check failed"
|
||||
):
|
||||
self.apply(preview)
|
||||
self.assertEqual(1, writer.call_count)
|
||||
self.assertIn('version = "0.1.11"', self.package.read_text())
|
||||
self.assertIn("# concurrent after write", self.requirements.read_text())
|
||||
|
||||
def test_write_failure_after_replace_requires_reconciliation(self):
|
||||
from govoplan_release import version_metadata
|
||||
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
original = version_metadata._atomic_write
|
||||
|
||||
def partial(path, payload):
|
||||
original(path, payload)
|
||||
raise OSError("Synthetic directory fsync failure after replacement")
|
||||
|
||||
with patch.object(version_metadata, "_atomic_write", side_effect=partial) as writer:
|
||||
with self.assertRaisesRegex(meta_preparation.MetaPreparationAmbiguous, "may have been written"):
|
||||
self.apply(preview)
|
||||
self.assertEqual(1, writer.call_count)
|
||||
self.assertIn('version = "0.1.11"', self.package.read_text())
|
||||
|
||||
def test_cli_requires_reviewed_receipt_and_explicit_out_of_run_confirmation(self):
|
||||
self.prepare_core()
|
||||
main = runpy.run_path(
|
||||
str(ROOT / "tools/release/prepare-developer-meta-package.py")
|
||||
)["main"]
|
||||
arguments = [
|
||||
"prepare-developer-meta-package.py",
|
||||
"--workspace",
|
||||
str(self.workspace),
|
||||
"--target-version",
|
||||
"0.1.11",
|
||||
]
|
||||
output = io.StringIO()
|
||||
with patch.object(sys, "argv", arguments), redirect_stdout(output):
|
||||
self.assertEqual(0, main())
|
||||
preview = self.root / "meta-preview.json"
|
||||
preview.write_text(output.getvalue())
|
||||
with (
|
||||
patch.object(sys, "argv", [*arguments, "--apply"]),
|
||||
redirect_stdout(io.StringIO()),
|
||||
):
|
||||
self.assertEqual(1, main())
|
||||
with (
|
||||
patch.object(
|
||||
sys,
|
||||
"argv",
|
||||
[
|
||||
*arguments,
|
||||
"--apply",
|
||||
"--receipt",
|
||||
str(preview),
|
||||
"--confirm-out-of-run",
|
||||
],
|
||||
),
|
||||
redirect_stdout(io.StringIO()),
|
||||
):
|
||||
self.assertEqual(0, main())
|
||||
|
||||
def test_unknown_full_input_and_unsafe_operator_tooling_fail_closed(self):
|
||||
self.prepare_core()
|
||||
unknown = self.workspace / "govoplan-unknown/pyproject.toml"
|
||||
unknown.parent.mkdir()
|
||||
unknown.write_text('[project]\nname="govoplan-unknown"\nversion="1.0.0"\n')
|
||||
with self.assertRaisesRegex(MetaPreparationError, "unregistered"):
|
||||
self.preview()
|
||||
unknown.unlink()
|
||||
self.generator.chmod(0o666)
|
||||
with self.assertRaisesRegex(MetaPreparationError, "owned, bounded regular"):
|
||||
self.preview()
|
||||
|
||||
def test_wrong_nested_identity_and_existing_immutable_tag_fail_closed(self):
|
||||
self.prepare_core()
|
||||
original = self.package.read_text()
|
||||
self.package.write_text(
|
||||
original.replace('name = "govoplan"', 'name = "not-govoplan"')
|
||||
)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Wrong synthetic package identity")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "identity"):
|
||||
self.preview()
|
||||
self.package.write_text(original)
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Restore synthetic package identity")
|
||||
git(self.meta, "tag", "-a", "v0.1.11", "-m", "Immutable target")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "target Meta tag"):
|
||||
self.preview()
|
||||
|
||||
def test_no_generic_durable_self_mutation_or_running_tooling_target(self):
|
||||
self.prepare_core()
|
||||
preview = self.preview()
|
||||
with self.assertRaisesRegex(VersionMetadataError, "outside durable runs"):
|
||||
apply_version_metadata_mutations(self.meta, target_version="0.1.11")
|
||||
with self.assertRaisesRegex(MetaPreparationError, "confirm"):
|
||||
self.preview(apply=True, expected_receipt=preview["receipt"])
|
||||
with patch.object(meta_preparation, "META_ROOT", self.meta):
|
||||
with self.assertRaisesRegex(MetaPreparationError, "running operator"):
|
||||
self.preview()
|
||||
|
||||
def test_next_version_plan_is_actionable_core_first_without_meta_executor(self):
|
||||
snapshots = tuple(
|
||||
collect_repository_snapshot(
|
||||
RepositorySpec(**spec),
|
||||
workspace_root=self.workspace,
|
||||
target_tag="v0.1.11",
|
||||
)
|
||||
for spec in self.specs[:2]
|
||||
)
|
||||
source = replace(
|
||||
dashboard(workspace=self.workspace, version=self.version),
|
||||
repositories=snapshots,
|
||||
)
|
||||
plan = build_selective_release_plan(
|
||||
source,
|
||||
selected_repos=("govoplan", "govoplan-core"),
|
||||
target_version="0.1.11",
|
||||
)
|
||||
self.assertEqual(
|
||||
["govoplan-core", "govoplan"], [unit.repo for unit in plan.units]
|
||||
)
|
||||
findings = [
|
||||
finding for finding in plan.gate_findings if finding.repo == "govoplan"
|
||||
]
|
||||
self.assertEqual(
|
||||
["developer_meta_core_preparation_required"],
|
||||
[finding.code for finding in findings],
|
||||
)
|
||||
self.assertIn("prepare-developer-meta-package.py", findings[0].remediation)
|
||||
meta_steps = [step for step in plan.dry_run_steps if step.repo == "govoplan"]
|
||||
self.assertEqual(
|
||||
["govoplan:prepare-support", "govoplan:publish-support"],
|
||||
[step.id for step in meta_steps],
|
||||
)
|
||||
self.assertTrue(all(step.status == "needs-executor" for step in meta_steps))
|
||||
self.prepare_core()
|
||||
prepared = build_selective_release_plan(
|
||||
source, selected_repos=("govoplan",), target_version="0.1.11"
|
||||
)
|
||||
self.assertEqual("developer_meta_out_of_run", prepared.gate_findings[0].code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,569 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import source_tag_batch as meta_source_tag, workspace # noqa: E402
|
||||
from govoplan_release.git_state import collect_versions # noqa: E402
|
||||
from govoplan_release.model import RepositorySnapshot, RepositorySpec, VersionSnapshot # noqa: E402
|
||||
from govoplan_release.repository_tag import tag_repositories # noqa: E402
|
||||
from govoplan_release.selective_planner import build_unit # noqa: E402
|
||||
from govoplan_release.version_alignment import repository_version_issues # noqa: E402
|
||||
from test_release_repository_tag import ( # noqa: E402
|
||||
add_scoped_workflow_manifest,
|
||||
create_release_repo,
|
||||
git,
|
||||
git_text,
|
||||
ref_exists,
|
||||
)
|
||||
|
||||
|
||||
class MetaSourceTagTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temporary = self.enterContext(
|
||||
tempfile.TemporaryDirectory(prefix="meta-release-tests-")
|
||||
)
|
||||
self.root = Path(self.temporary)
|
||||
self.workspace = self.root / "workspace"
|
||||
self.workspace.mkdir()
|
||||
self.version = "0.1.10"
|
||||
self.core, self.core_remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-core",
|
||||
version=self.version,
|
||||
)
|
||||
self.access, self.access_remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-access",
|
||||
version=self.version,
|
||||
)
|
||||
add_scoped_workflow_manifest(self.access)
|
||||
self.meta = self.workspace / "govoplan"
|
||||
self.meta_remote = self.root / "govoplan.git"
|
||||
git(self.root, "init", "--bare", str(self.meta_remote))
|
||||
git(self.workspace, "init", "-b", "main", str(self.meta))
|
||||
git(self.meta, "config", "user.name", "Meta Release Fixture")
|
||||
git(self.meta, "config", "user.email", "release@example.invalid")
|
||||
self.package = self.meta / "packages/govoplan-meta/pyproject.toml"
|
||||
self.package.parent.mkdir(parents=True)
|
||||
self.requirements = self.meta / "requirements-release.txt"
|
||||
self.requirements.write_text(
|
||||
"../govoplan-core\ngovoplan-access @ git+ssh://git@example.invalid/GovOPlaN/govoplan-access.git@v0.1.10\n"
|
||||
)
|
||||
self.render = runpy.run_path(
|
||||
str(ROOT / "tools/release/generate-developer-meta-package.py")
|
||||
)["render"]
|
||||
self.synchronize()
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Nested developer package")
|
||||
git(self.meta, "remote", "add", "origin", str(self.meta_remote))
|
||||
git(self.meta, "push", "-u", "origin", "main")
|
||||
self.specs = [
|
||||
{
|
||||
"name": name,
|
||||
"category": "system" if subtype else "module",
|
||||
"subtype": subtype,
|
||||
"path": name,
|
||||
"remote": str(remote),
|
||||
}
|
||||
for name, subtype, remote in (
|
||||
("govoplan", "meta", self.meta_remote),
|
||||
("govoplan-core", "kernel", self.core_remote),
|
||||
("govoplan-access", "", self.access_remote),
|
||||
)
|
||||
]
|
||||
self.registry = self.root / "repositories.json"
|
||||
self.registry.write_text(json.dumps({"repositories": self.specs}))
|
||||
self.enterContext(patch.object(workspace, "REPOSITORIES_FILE", self.registry))
|
||||
|
||||
def synchronize(self):
|
||||
self.package.write_text(
|
||||
self.render(workspace=self.workspace, requirements=self.requirements)
|
||||
)
|
||||
|
||||
def commit_meta(self):
|
||||
git(self.meta, "add", ".")
|
||||
git(self.meta, "commit", "-m", "Changed synthetic metadata")
|
||||
|
||||
def tag(
|
||||
self,
|
||||
*,
|
||||
repos=("govoplan", "govoplan-core"),
|
||||
apply=False,
|
||||
push=False,
|
||||
**overrides,
|
||||
):
|
||||
return tag_repositories(
|
||||
repos=repos,
|
||||
repo_versions={repo: self.version for repo in repos},
|
||||
workspace_root=self.workspace,
|
||||
apply=apply,
|
||||
push=push,
|
||||
**overrides,
|
||||
)
|
||||
|
||||
def assert_no_tags(self):
|
||||
for repo in (
|
||||
self.meta,
|
||||
self.meta_remote,
|
||||
self.core,
|
||||
self.core_remote,
|
||||
self.access,
|
||||
self.access_remote,
|
||||
):
|
||||
self.assertFalse(ref_exists(repo, "refs/tags/v0.1.10"), str(repo))
|
||||
|
||||
def test_explicit_nested_version_collection_and_alignment_without_root_package(
|
||||
self,
|
||||
):
|
||||
versions = collect_versions(self.meta)
|
||||
self.assertIsNone(versions.pyproject)
|
||||
self.assertEqual(self.version, versions.developer_meta)
|
||||
self.assertEqual(self.version, versions.primary)
|
||||
self.assertFalse((self.meta / "pyproject.toml").exists())
|
||||
self.assertEqual(
|
||||
(), repository_version_issues(self.meta, expected_version=self.version)
|
||||
)
|
||||
mismatch = repository_version_issues(self.meta, expected_version="0.1.11")
|
||||
self.assertTrue(
|
||||
any(
|
||||
issue.source == "packages/govoplan-meta/pyproject.toml"
|
||||
for issue in mismatch
|
||||
)
|
||||
)
|
||||
|
||||
def test_planner_and_console_display_the_explicit_nested_version(self):
|
||||
snapshot = RepositorySnapshot(
|
||||
spec=RepositorySpec(**self.specs[0]),
|
||||
absolute_path=str(self.meta),
|
||||
exists=True,
|
||||
is_git=True,
|
||||
has_head=True,
|
||||
branch="main",
|
||||
versions=VersionSnapshot(developer_meta=self.version),
|
||||
)
|
||||
unit = build_unit(snapshot, target_version=None, contracts=None)
|
||||
self.assertEqual(self.version, unit.current_version)
|
||||
self.assertEqual(self.version, unit.target_version)
|
||||
html = (ROOT / "tools/release/webui/index.html").read_text()
|
||||
self.assertIn(
|
||||
"if (versions.developer_meta) return versions.developer_meta;", html
|
||||
)
|
||||
drift = RepositorySnapshot(
|
||||
spec=snapshot.spec,
|
||||
absolute_path=str(self.meta),
|
||||
exists=True,
|
||||
is_git=True,
|
||||
has_head=True,
|
||||
branch="main",
|
||||
versions=VersionSnapshot(pyproject="0.1.9", developer_meta=self.version),
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
"version metadata is not aligned" in item
|
||||
for item in build_unit(
|
||||
drift, target_version=self.version, contracts=None
|
||||
).blockers
|
||||
)
|
||||
)
|
||||
|
||||
def test_unknown_nested_package_and_missing_or_wrong_meta_identity_fail_closed(
|
||||
self,
|
||||
):
|
||||
unknown = self.workspace / "unknown"
|
||||
nested = unknown / "packages/govoplan-meta/pyproject.toml"
|
||||
nested.parent.mkdir(parents=True)
|
||||
nested.write_text(self.package.read_text())
|
||||
self.assertIsNone(collect_versions(unknown).primary)
|
||||
self.assertIn(
|
||||
"no version metadata",
|
||||
repository_version_issues(unknown, expected_version=self.version)[
|
||||
0
|
||||
].message,
|
||||
)
|
||||
for value in ("", '[project]\nname="not-govoplan"\nversion="0.1.10"\n'):
|
||||
with self.subTest(value=value):
|
||||
self.package.write_text(value)
|
||||
self.assertTrue(
|
||||
repository_version_issues(self.meta, expected_version=self.version)
|
||||
)
|
||||
|
||||
def test_preview_local_tag_and_publish_share_complete_nested_contract(self):
|
||||
preview = self.tag(push=True)
|
||||
self.assertEqual("planned", preview["status"], preview)
|
||||
self.assertEqual(
|
||||
["govoplan-core", "govoplan"],
|
||||
[row["repo"] for row in preview["repositories"]],
|
||||
)
|
||||
self.assertEqual("registered-meta-batch-v1", preview["source_contract"])
|
||||
self.assert_no_tags()
|
||||
local = self.tag(apply=True)
|
||||
self.assertEqual("tagged", local["status"], local)
|
||||
for repo in (self.core, self.meta):
|
||||
self.assertEqual(
|
||||
"tag", git_text(repo, "cat-file", "-t", "refs/tags/v0.1.10")
|
||||
)
|
||||
self.assertFalse(ref_exists(self.meta_remote, "refs/tags/v0.1.10"))
|
||||
published = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", published["status"], published)
|
||||
for repo, remote in (
|
||||
(self.core, self.core_remote),
|
||||
(self.meta, self.meta_remote),
|
||||
):
|
||||
self.assertEqual(
|
||||
git_text(repo, "rev-parse", "HEAD"),
|
||||
git_text(remote, "rev-parse", "refs/heads/main"),
|
||||
)
|
||||
self.assertEqual(
|
||||
git_text(repo, "rev-parse", "refs/tags/v0.1.10"),
|
||||
git_text(remote, "rev-parse", "refs/tags/v0.1.10"),
|
||||
)
|
||||
again = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", again["status"], again)
|
||||
|
||||
def test_stale_composition_blocks_whole_batch_before_local_tag_or_push(self):
|
||||
self.package.write_text(
|
||||
self.package.read_text().replace(
|
||||
"govoplan-access==0.1.10", "govoplan-access==0.1.9"
|
||||
)
|
||||
)
|
||||
self.commit_meta()
|
||||
for apply, push in ((False, False), (True, False), (True, True)):
|
||||
result = self.tag(
|
||||
repos=("govoplan-core", "govoplan-access", "govoplan"),
|
||||
apply=apply,
|
||||
push=push,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_core_outside_batch_requires_matching_existing_and_published_tag(self):
|
||||
self.assertEqual("blocked", self.tag(repos=("govoplan",))["status"])
|
||||
git(self.core, "tag", "-a", "v0.1.10", "-m", "Core release")
|
||||
self.assertEqual("planned", self.tag(repos=("govoplan",))["status"])
|
||||
self.assertEqual("blocked", self.tag(repos=("govoplan",), push=True)["status"])
|
||||
git(self.core, "push", "origin", "refs/tags/v0.1.10")
|
||||
self.assertEqual("planned", self.tag(repos=("govoplan",), push=True)["status"])
|
||||
|
||||
def test_changed_core_version_and_explicit_selected_version_mismatch_block(self):
|
||||
mismatch = tag_repositories(
|
||||
repos=("govoplan", "govoplan-core"),
|
||||
repo_versions={"govoplan": self.version, "govoplan-core": "0.1.11"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=True,
|
||||
)
|
||||
self.assertEqual("blocked", mismatch["status"], mismatch)
|
||||
(self.core / "pyproject.toml").write_text(
|
||||
'[project]\nname="govoplan-core"\nversion="0.1.11"\n'
|
||||
)
|
||||
git(self.core, "add", ".")
|
||||
git(self.core, "commit", "-m", "Core new version")
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_unsafe_origin_on_any_selected_repo_blocks_every_effect(self):
|
||||
for repo in (self.meta, self.core, self.access):
|
||||
with self.subTest(repo=repo.name):
|
||||
git(
|
||||
repo,
|
||||
"config",
|
||||
"remote.origin.pushurl",
|
||||
str(self.root / "unregistered.git"),
|
||||
)
|
||||
result = self.tag(
|
||||
repos=("govoplan-core", "govoplan-access", "govoplan"),
|
||||
apply=True,
|
||||
push=True,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("registered origin", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
git(repo, "config", "--unset", "remote.origin.pushurl")
|
||||
|
||||
def test_world_writable_nonsticky_parent_blocks_without_changing_permissions(self):
|
||||
original = self.root.stat().st_mode & 0o7777
|
||||
self.root.chmod(0o777)
|
||||
try:
|
||||
for apply in (False, True):
|
||||
result = self.tag(apply=apply, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn(
|
||||
"group/world writable", result["repositories"][0]["detail"]
|
||||
)
|
||||
self.assertEqual(0o777, self.root.stat().st_mode & 0o7777)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
self.root.chmod(original)
|
||||
|
||||
def test_wrong_metadata_owner_blocks_before_remote_lookup(self):
|
||||
config = self.meta / ".git/config"
|
||||
original = Path.lstat
|
||||
|
||||
def wrong_owner(path, *args, **kwargs):
|
||||
observed = original(path, *args, **kwargs)
|
||||
if path == config:
|
||||
fields = list(observed)
|
||||
fields[4] = os.geteuid() + 1
|
||||
return os.stat_result(fields)
|
||||
return observed
|
||||
|
||||
with (
|
||||
patch.object(Path, "lstat", new=wrong_owner),
|
||||
patch.object(
|
||||
meta_source_tag,
|
||||
"registered_source_origin_issues",
|
||||
side_effect=AssertionError("must validate ownership before Git"),
|
||||
),
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("current operator", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_hidden_index_flags_cannot_disguise_modified_release_metadata(self):
|
||||
for flag, undo in (
|
||||
("--assume-unchanged", "--no-assume-unchanged"),
|
||||
("--skip-worktree", "--no-skip-worktree"),
|
||||
):
|
||||
for repo, relative in (
|
||||
(self.meta, "packages/govoplan-meta/pyproject.toml"),
|
||||
(self.core, "pyproject.toml"),
|
||||
):
|
||||
with self.subTest(flag=flag, repo=repo.name):
|
||||
target = repo / relative
|
||||
original = target.read_text()
|
||||
git(repo, "update-index", flag, relative)
|
||||
target.write_text(
|
||||
original
|
||||
+ "\n# Hidden working-tree input differs from frozen HEAD\n"
|
||||
)
|
||||
try:
|
||||
self.assertEqual("", git_text(repo, "status", "--porcelain"))
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn(
|
||||
"index entries", result["repositories"][0]["detail"]
|
||||
)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
target.write_text(original)
|
||||
git(repo, "update-index", undo, relative)
|
||||
|
||||
def test_read_only_git_target_is_not_repaired_or_tagged(self):
|
||||
metadata = self.meta / ".git"
|
||||
original = metadata.stat().st_mode & 0o7777
|
||||
metadata.chmod(0o500)
|
||||
try:
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertEqual(0o500, metadata.stat().st_mode & 0o7777)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
metadata.chmod(original)
|
||||
|
||||
def test_git_object_alternates_are_rejected_before_remote_lookup(self):
|
||||
(self.meta / ".git/objects/info/alternates").write_text(
|
||||
str(self.root / "outside-objects") + "\n"
|
||||
)
|
||||
with patch.object(
|
||||
meta_source_tag,
|
||||
"registered_source_origin_issues",
|
||||
side_effect=AssertionError("must reject alternates before Git"),
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("alternates", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_owned_worktree_metadata_inside_private_workspace_is_supported(self):
|
||||
main_checkout = self.workspace / "meta-main-storage"
|
||||
self.meta.rename(main_checkout)
|
||||
git(main_checkout, "worktree", "add", "--force", str(self.meta), "main")
|
||||
self.assertTrue((self.meta / ".git").is_file())
|
||||
result = self.tag(apply=True)
|
||||
self.assertEqual("tagged", result["status"], result)
|
||||
filesystem = result["source_receipts"]["govoplan"]["filesystem"]
|
||||
self.assertEqual(
|
||||
str(main_checkout / ".git"), filesystem["git_common_directory"][0]
|
||||
)
|
||||
|
||||
def test_git_directory_replacement_with_same_head_changes_frozen_receipt(self):
|
||||
preview = meta_source_tag._preview_repositories
|
||||
|
||||
def swapped_git_directory(**kwargs):
|
||||
result = preview(**kwargs)
|
||||
original = self.meta / ".git"
|
||||
backup = self.root / "original-meta-git"
|
||||
original.rename(backup)
|
||||
shutil.copytree(backup, original)
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
meta_source_tag,
|
||||
"_preview_repositories",
|
||||
side_effect=swapped_git_directory,
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_non_main_and_divergent_live_main_fail_even_with_stale_tracking(self):
|
||||
git(self.meta, "switch", "-c", "feature")
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
git(self.meta, "switch", "main")
|
||||
clone = self.root / "other-writer"
|
||||
git(self.root, "clone", "--branch", "main", str(self.meta_remote), str(clone))
|
||||
git(clone, "config", "user.name", "Other synthetic writer")
|
||||
git(clone, "config", "user.email", "other@example.invalid")
|
||||
(clone / "other.txt").write_text("remote divergence\n")
|
||||
git(clone, "add", ".")
|
||||
git(clone, "commit", "-m", "Remote main advanced")
|
||||
git(clone, "push", "origin", "main")
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("live origin/main", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_symlink_checkout_is_not_a_registered_source(self):
|
||||
original = self.workspace / "moved-meta"
|
||||
self.meta.rename(original)
|
||||
self.meta.symlink_to(original, target_is_directory=True)
|
||||
result = self.tag(apply=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("symlink", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_lightweight_and_conflicting_annotated_tags_block(self):
|
||||
git(self.meta, "tag", "v0.1.10")
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
git(self.meta, "tag", "-d", "v0.1.10")
|
||||
git(self.meta, "tag", "-a", "v0.1.10", "-m", "First annotation")
|
||||
git(self.meta, "push", "origin", "refs/tags/v0.1.10")
|
||||
git(self.meta, "tag", "-d", "v0.1.10")
|
||||
git(self.meta, "tag", "-a", "v0.1.10", "-m", "Different annotation")
|
||||
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_meta_source_receipt_changed_after_preflight_blocks_before_first_effect(
|
||||
self,
|
||||
):
|
||||
preview = meta_source_tag._preview_repositories
|
||||
|
||||
def changed_after_preflight(**kwargs):
|
||||
result = preview(**kwargs)
|
||||
(self.meta / "new-review.txt").write_text("changed after preflight\n")
|
||||
self.commit_meta()
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
meta_source_tag,
|
||||
"_preview_repositories",
|
||||
side_effect=changed_after_preflight,
|
||||
):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_fabricated_push_success_without_remote_receipt_fails_and_stops_batch(self):
|
||||
original = meta_source_tag.run
|
||||
|
||||
def run(command, **kwargs):
|
||||
if command[:3] == ("git", "push", "--atomic"):
|
||||
return subprocess.CompletedProcess(command, 0, "", "")
|
||||
return original(command, **kwargs)
|
||||
|
||||
with patch.object(meta_source_tag, "run", side_effect=run):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertEqual("failed", result["repositories"][0]["status"])
|
||||
self.assertEqual("skipped", result["repositories"][1]["status"])
|
||||
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_remote_tag_without_expected_main_receipt_is_not_success(self):
|
||||
(self.core / "reviewed-change.txt").write_text("release source changes\n")
|
||||
git(self.core, "add", ".")
|
||||
git(self.core, "commit", "-m", "Advance reviewed Core source")
|
||||
original = meta_source_tag.run
|
||||
pushes = []
|
||||
|
||||
def tag_only(command, **kwargs):
|
||||
if command[:3] == ("git", "push", "--atomic"):
|
||||
pushes.append(command)
|
||||
# Simulate a defective transport that claims atomic success,
|
||||
# while publishing only the exact expected annotation object.
|
||||
return original(("git", "push", "origin", command[-1]), **kwargs)
|
||||
return original(command, **kwargs)
|
||||
|
||||
with patch.object(meta_source_tag, "run", side_effect=tag_only):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertEqual(1, len(pushes))
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assertEqual(
|
||||
git_text(self.core, "rev-parse", "refs/tags/v0.1.10"),
|
||||
git_text(self.core_remote, "rev-parse", "refs/tags/v0.1.10"),
|
||||
)
|
||||
self.assertNotEqual(
|
||||
git_text(self.core, "rev-parse", "HEAD"),
|
||||
git_text(self.core_remote, "rev-parse", "refs/heads/main"),
|
||||
)
|
||||
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_meta_receipt_is_rechecked_after_an_earlier_successful_publication(self):
|
||||
original = meta_source_tag.run
|
||||
|
||||
def changed_after_core(command, **kwargs):
|
||||
result = original(command, **kwargs)
|
||||
if (
|
||||
command[:3] == ("git", "push", "--atomic")
|
||||
and kwargs["cwd"] == self.core
|
||||
):
|
||||
(self.meta / "changed-review.txt").write_text(
|
||||
"new Meta source after Core publication\n"
|
||||
)
|
||||
self.commit_meta()
|
||||
return result
|
||||
|
||||
with patch.object(meta_source_tag, "run", side_effect=changed_after_core):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertTrue(ref_exists(self.core_remote, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(self.meta, "refs/tags/v0.1.10"))
|
||||
self.assertEqual("skipped", result["repositories"][1]["status"])
|
||||
|
||||
def test_unknown_selected_repository_cannot_use_meta_support_exception(self):
|
||||
result = self.tag(repos=("govoplan", "unknown"), apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("not registered", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_selected_checkout_generator_is_never_executed(self):
|
||||
malicious = self.meta / "tools/release/generate-developer-meta-package.py"
|
||||
malicious.parent.mkdir(parents=True)
|
||||
malicious.write_text(
|
||||
'raise RuntimeError("selected checkout must not execute")\n'
|
||||
)
|
||||
self.commit_meta()
|
||||
self.assertEqual("planned", self.tag()["status"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -65,6 +65,8 @@ branch_labels: Union[str, Sequence[str], None] = None
|
||||
)
|
||||
wrapper = development / release.name
|
||||
wrapper.write_text(
|
||||
"from importlib import import_module\n"
|
||||
'_migration = import_module("govoplan_core.backend.migrations.versions.1234_example")\n'
|
||||
"revision = _migration.revision\n"
|
||||
"down_revision = _migration.down_revision\n"
|
||||
"depends_on = _migration.depends_on\n"
|
||||
@@ -79,6 +81,150 @@ branch_labels: Union[str, Sequence[str], None] = None
|
||||
self.assertEqual(("base",), migration.down_revisions)
|
||||
self.assertEqual(("core",), migration.depends_on)
|
||||
|
||||
def test_literal_wrapper_alias_and_different_filename_are_resolved_without_execution(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_v019_example.py").write_text(
|
||||
'revision = "1234"\ndown_revision = "base"\ndepends_on = "core"\nbranch_labels = None\n'
|
||||
'raise AssertionError("Migration implementation must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
for alias in ("_migration", "edit_revision", "message_actions"):
|
||||
with self.subTest(alias=alias):
|
||||
wrapper.write_text(
|
||||
"from importlib import import_module as load_migration\n"
|
||||
f'{alias} = load_migration("govoplan_campaign.backend.migrations.versions." "1234_v019_example")\n'
|
||||
f"revision = {alias}.revision\ndown_revision = {alias}.down_revision\n"
|
||||
f"depends_on = {alias}.depends_on\nbranch_labels = {alias}.branch_labels\n"
|
||||
'raise AssertionError("Wrapper must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
migration = audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||
self.assertEqual(migration.revision, "1234")
|
||||
self.assertEqual(migration.down_revisions, ("base",))
|
||||
self.assertEqual(migration.depends_on, ("core",))
|
||||
|
||||
def test_core_literal_sibling_file_wrapper_is_resolved_without_execution(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_example.py").write_text(
|
||||
'revision = "1234"\ndown_revision = None\ndepends_on = None\nbranch_labels = None\n'
|
||||
'raise AssertionError("Migration implementation must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
wrapper.write_text(
|
||||
"from importlib.util import module_from_spec, spec_from_file_location\n"
|
||||
"from pathlib import Path\n"
|
||||
'_path = Path(__file__).resolve().parents[1] / "versions" / "1234_example.py"\n'
|
||||
'_spec = spec_from_file_location("synthetic_migration", _path)\n'
|
||||
"_module = module_from_spec(_spec)\n_spec.loader.exec_module(_module)\n"
|
||||
"revision = _module.revision\ndown_revision = _module.down_revision\n"
|
||||
"depends_on = _module.depends_on\nbranch_labels = _module.branch_labels\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
migration = audit.parse_migration_file("govoplan-core", wrapper)
|
||||
self.assertEqual(migration.revision, "1234")
|
||||
self.assertEqual(migration.down_revisions, ())
|
||||
|
||||
def test_wrapper_rejects_dynamic_foreign_missing_or_rebound_targets(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_example.py").write_text('revision = "1234"\n', encoding="utf-8")
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
valid = '_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
|
||||
definitions = (
|
||||
'_migration = import_module(module_name)\n',
|
||||
'_migration = import_module("govoplan_mail.backend.migrations.versions.1234_example")\n',
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions...other.1234_example")\n',
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions.missing")\n',
|
||||
valid + "_migration = another_module\n",
|
||||
"import_module = another_loader\n" + valid,
|
||||
"def import_module(value):\n return another_module\n" + valid,
|
||||
"import another_loader as import_module\n" + valid,
|
||||
valid + "class _migration:\n revision = 'another'\n",
|
||||
"if condition:\n _migration = another_module\n" + valid,
|
||||
valid + "del _migration\n",
|
||||
)
|
||||
for definition in definitions:
|
||||
with self.subTest(definition=definition):
|
||||
wrapper.write_text(
|
||||
"from importlib import import_module\n" + definition + "revision = _migration.revision\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
|
||||
audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||
|
||||
def test_wrapper_rejects_symlink_and_mixed_release_metadata(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/1234_example.py").write_text('revision = "1234"\ndown_revision = None\n', encoding="utf-8")
|
||||
(root / "versions/5678_example.py").write_text('revision = "5678"\ndown_revision = None\n', encoding="utf-8")
|
||||
(root / "versions/linked.py").symlink_to(root / "versions/1234_example.py")
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
definitions = (
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions.linked")\nrevision = _migration.revision\n',
|
||||
'_migration = import_module("govoplan_campaign.backend.migrations.versions.1234_example")\n'
|
||||
'_other = import_module("govoplan_campaign.backend.migrations.versions.5678_example")\n'
|
||||
'revision = _migration.revision\ndown_revision = _other.down_revision\n',
|
||||
)
|
||||
for definition in definitions:
|
||||
with self.subTest(definition=definition):
|
||||
wrapper.write_text("from importlib import import_module\n" + definition, encoding="utf-8")
|
||||
with self.assertRaisesRegex(ValueError, "unsupported or ambiguous"):
|
||||
audit.parse_migration_file("govoplan-campaign", wrapper)
|
||||
|
||||
def test_unresolved_revision_expression_is_not_silently_omitted(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
path = Path(directory) / "1234_example.py"
|
||||
path.write_text("revision = calculate_revision()\n", encoding="utf-8")
|
||||
with self.assertRaisesRegex(ValueError, "Unsupported or ambiguous migration metadata"):
|
||||
audit.parse_migration_file("govoplan-core", path)
|
||||
|
||||
def test_explicit_metadata_reexport_is_resolved_without_execution(self) -> None:
|
||||
audit = load_audit_module()
|
||||
with tempfile.TemporaryDirectory(prefix="migration-audit-test-") as directory:
|
||||
root = Path(directory)
|
||||
(root / "versions").mkdir()
|
||||
(root / "dev_versions").mkdir()
|
||||
(root / "versions/a234_example.py").write_text(
|
||||
'revision = "1234"\ndown_revision = "base"\ndepends_on = None\nbranch_labels = None\n'
|
||||
'raise AssertionError("Migration implementation must not execute")\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
wrapper = root / "dev_versions/1234_example.py"
|
||||
source = "govoplan_organizations.backend.migrations.versions.a234_example"
|
||||
wrapper.write_text(f"from {source} import revision, down_revision, depends_on, branch_labels\n", encoding="utf-8")
|
||||
migration = audit.parse_migration_file("govoplan-organizations", wrapper)
|
||||
self.assertEqual(migration.revision, "1234")
|
||||
self.assertEqual(migration.down_revisions, ("base",))
|
||||
for declaration in (
|
||||
f"from {source} import *\n",
|
||||
f"from {source} import revision as down_revision\n",
|
||||
f"from {source} import revision\nrevision = 'different'\n",
|
||||
f"from {source} import revision\nimport another as revision\n",
|
||||
f"from {source} import revision\ndef revision():\n pass\n",
|
||||
f"from {source.replace('govoplan_organizations', 'govoplan_mail')} import revision\n",
|
||||
):
|
||||
with self.subTest(declaration=declaration):
|
||||
wrapper.write_text(declaration, encoding="utf-8")
|
||||
with self.assertRaises(ValueError):
|
||||
audit.parse_migration_file("govoplan-organizations", wrapper)
|
||||
|
||||
def test_release_baseline_matches_current_heads_in_strict_report(self) -> None:
|
||||
audit = load_audit_module()
|
||||
migrations = [
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import runpy
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
@@ -16,6 +18,7 @@ if str(RELEASE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(RELEASE_ROOT))
|
||||
|
||||
from govoplan_release.repository_tag import tag_repositories # noqa: E402
|
||||
from govoplan_release import workspace as release_workspace # noqa: E402
|
||||
from server.app import create_app # noqa: E402
|
||||
|
||||
|
||||
@@ -38,6 +41,14 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
version="0.1.10",
|
||||
)
|
||||
add_scoped_workflow_manifest(self.manifest_repo)
|
||||
# The operator's test catalog explicitly registers known synthetic
|
||||
# endpoints; production trust checks are not patched or bypassed.
|
||||
self.registry = self.root / "registered-test-repositories.json"
|
||||
self.registered = json.loads((META_ROOT / "repositories.json").read_text())
|
||||
for spec in self.registered["repositories"]:
|
||||
spec["remote"] = str(self.root / f"{spec['name']}.git")
|
||||
self.registry.write_text(json.dumps(self.registered))
|
||||
self.enterContext(patch.object(release_workspace, "REPOSITORIES_FILE", self.registry))
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.temporary.cleanup()
|
||||
@@ -127,6 +138,10 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
name="govoplan-core",
|
||||
version="0.1.10",
|
||||
)
|
||||
for spec in self.registered["repositories"]:
|
||||
if spec["name"] == "govoplan-core":
|
||||
spec["remote"] = str(remote_root / "govoplan-core.git")
|
||||
self.registry.write_text(json.dumps(self.registered))
|
||||
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-core",),
|
||||
@@ -214,21 +229,23 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
git(self.repo, "commit", "-m", "Add release WebUI composition")
|
||||
git(self.repo, "push", "origin", "main")
|
||||
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-core", "govoplan-campaign"),
|
||||
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=True,
|
||||
)
|
||||
for push in (False, True):
|
||||
with self.subTest(push=push):
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-core", "govoplan-campaign"),
|
||||
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=push,
|
||||
)
|
||||
|
||||
self.assertEqual("blocked", result["status"])
|
||||
self.assertIn("no selected repository was mutated", result["detail"])
|
||||
self.assertEqual("skipped", result["repositories"][0]["status"])
|
||||
self.assertEqual("blocked", result["repositories"][1]["status"])
|
||||
self.assertIn("release WebUI composition gate failed", result["repositories"][1]["detail"])
|
||||
for repository in (self.repo, self.remote, campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
self.assertEqual("blocked", result["status"])
|
||||
self.assertIn("no selected repository was mutated", result["detail"])
|
||||
self.assertEqual("skipped", result["repositories"][0]["status"])
|
||||
self.assertEqual("blocked", result["repositories"][1]["status"])
|
||||
self.assertIn("release WebUI composition gate failed", result["repositories"][1]["detail"])
|
||||
for repository in (self.repo, self.remote, campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_api_keeps_legacy_source_release_preview_only(self) -> None:
|
||||
with TestClient(
|
||||
@@ -300,6 +317,183 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
self.assertIn("Signed Website Catalog", ui.text)
|
||||
self.assertIn("Apply + Website Tag", ui.text)
|
||||
|
||||
def test_local_module_candidate_precedes_core_lock_but_publication_does_not(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
remotes = (self.remote, self.manifest_remote, campaign_remote)
|
||||
remote_refs = {path: git_text(path, "show-ref") for path in remotes}
|
||||
arguments = {
|
||||
"repos": ("govoplan-campaign",),
|
||||
"repo_versions": {"govoplan-campaign": "0.1.10"},
|
||||
"workspace_root": self.workspace,
|
||||
}
|
||||
|
||||
preview = tag_repositories(**arguments, apply=False, push=False)
|
||||
self.assertEqual("planned", preview["status"], preview)
|
||||
self.assertFalse(ref_exists(campaign, "refs/tags/v0.1.10"))
|
||||
candidate = tag_repositories(**arguments, apply=True, push=False)
|
||||
self.assertEqual("tagged", candidate["status"], candidate)
|
||||
self.assertEqual("tag", git_text(campaign, "cat-file", "-t", "v0.1.10"))
|
||||
tag_object = git_text(campaign, "rev-parse", "v0.1.10")
|
||||
head = git_text(campaign, "rev-parse", "HEAD")
|
||||
self.assertEqual(head, git_text(campaign, "rev-parse", "v0.1.10^{commit}"))
|
||||
|
||||
for apply in (False, True):
|
||||
with self.subTest(publish_apply=apply):
|
||||
blocked = tag_repositories(**arguments, apply=apply, push=True)
|
||||
self.assertEqual("blocked", blocked["status"], blocked)
|
||||
self.assertIn(
|
||||
"release WebUI composition gate failed",
|
||||
blocked["repositories"][0]["detail"],
|
||||
)
|
||||
self.assertEqual(tag_object, git_text(campaign, "rev-parse", "v0.1.10"))
|
||||
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
|
||||
|
||||
lock_path = self.repo / "webui" / "package-lock.release.json"
|
||||
lock = json.loads(lock_path.read_text(encoding="utf-8"))
|
||||
locked_campaign = lock["packages"]["node_modules/@govoplan/campaign-webui"]
|
||||
locked_campaign["version"] = "0.1.10"
|
||||
locked_campaign["resolved"] = f"git+ssh://git@example.test/acme/govoplan-campaign.git#{head}"
|
||||
lock_path.write_text(json.dumps(lock) + "\n", encoding="utf-8")
|
||||
git(self.repo, "add", "webui/package-lock.release.json")
|
||||
git(self.repo, "commit", "-m", "Resolve reviewed local Campaign candidate")
|
||||
|
||||
core_candidate = tag_repositories(
|
||||
repos=("govoplan-core",),
|
||||
repo_versions={"govoplan-core": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("tagged", core_candidate["status"], core_candidate)
|
||||
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
|
||||
published = tag_repositories(**arguments, apply=True, push=True)
|
||||
self.assertEqual("published", published["status"], published)
|
||||
self.assertEqual(tag_object, git_text(campaign_remote, "rev-parse", "v0.1.10"))
|
||||
self.assertEqual(head, git_text(campaign_remote, "rev-parse", "refs/heads/main"))
|
||||
self.assertEqual(remote_refs[self.remote], git_text(self.remote, "show-ref"))
|
||||
|
||||
def test_local_core_candidate_still_requires_resolved_module_tags(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
for selected in (("govoplan-core",), ("govoplan-campaign", "govoplan-core")):
|
||||
with self.subTest(selected=selected):
|
||||
result = tag_repositories(
|
||||
repos=selected,
|
||||
repo_versions={repo: "0.1.10" for repo in selected},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
core_row = next(row for row in result["repositories"] if row["repo"] == "govoplan-core")
|
||||
self.assertIn("version alignment gate failed", core_row["detail"])
|
||||
self.assertIn("@govoplan/campaign-webui:resolved", core_row["detail"])
|
||||
self.assertIn("expected 'local tag v0.1.10'", core_row["detail"])
|
||||
for repository in (self.repo, self.remote, campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_local_module_candidate_preserves_version_and_worktree_gates(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
arguments = {
|
||||
"repos": ("govoplan-campaign",),
|
||||
"workspace_root": self.workspace,
|
||||
"apply": True,
|
||||
"push": False,
|
||||
}
|
||||
mismatch = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.11"})
|
||||
self.assertEqual("blocked", mismatch["status"], mismatch)
|
||||
self.assertIn("version alignment gate failed", mismatch["repositories"][0]["detail"])
|
||||
|
||||
(campaign / "unreviewed.txt").write_text("operator work\n", encoding="utf-8")
|
||||
dirty = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.10"})
|
||||
self.assertEqual("blocked", dirty["status"], dirty)
|
||||
self.assertIn("worktree is not clean", dirty["repositories"][0]["detail"])
|
||||
for repository in (campaign, campaign_remote):
|
||||
for tag in ("v0.1.10", "v0.1.11"):
|
||||
self.assertFalse(ref_exists(repository, f"refs/tags/{tag}"))
|
||||
|
||||
def test_local_module_candidate_preserves_manifest_gate(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
replace_with_unscoped_workflow_manifest(self.manifest_repo)
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-campaign",),
|
||||
repo_versions={"govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("scope-conditioned alternative", result["repositories"][0]["detail"])
|
||||
for repository in (campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_local_module_candidate_preserves_remote_tag_immutability(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
git(campaign, "tag", "-a", "v0.1.10", "-m", "Existing immutable tag", "v0.1.9^{commit}")
|
||||
git(campaign, "push", "origin", "refs/tags/v0.1.10")
|
||||
remote_refs = git_text(campaign_remote, "show-ref")
|
||||
for local_exists in (True, False):
|
||||
with self.subTest(local_exists=local_exists):
|
||||
if not local_exists:
|
||||
git(campaign, "tag", "-d", "v0.1.10")
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-campaign",),
|
||||
repo_versions={"govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("immutable tag", result["repositories"][0]["detail"])
|
||||
self.assertIn("not HEAD", result["repositories"][0]["detail"])
|
||||
self.assertEqual(remote_refs, git_text(campaign_remote, "show-ref"))
|
||||
self.assertEqual(local_exists, ref_exists(campaign, "refs/tags/v0.1.10"))
|
||||
|
||||
def _staged_campaign_bundle(self) -> tuple[Path, Path]:
|
||||
campaign, remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-campaign",
|
||||
version="0.1.9",
|
||||
)
|
||||
campaign_webui = campaign / "webui"
|
||||
campaign_webui.mkdir()
|
||||
package_path = campaign_webui / "package.json"
|
||||
package_path.write_text(
|
||||
'{"name":"@govoplan/campaign-webui","version":"0.1.9"}\n', encoding="utf-8"
|
||||
)
|
||||
git(campaign, "add", "webui/package.json")
|
||||
git(campaign, "commit", "-m", "Prior Campaign WebUI package")
|
||||
git(campaign, "tag", "-a", "v0.1.9", "-m", "Prior Campaign release")
|
||||
git(campaign, "push", "origin", "main", "refs/tags/v0.1.9")
|
||||
prior_commit = git_text(campaign, "rev-parse", "HEAD")
|
||||
for path in (campaign / "pyproject.toml", package_path):
|
||||
path.write_text(path.read_text(encoding="utf-8").replace("0.1.9", "0.1.10"), encoding="utf-8")
|
||||
git(campaign, "add", "pyproject.toml", "webui/package.json")
|
||||
git(campaign, "commit", "-m", "Reviewed Campaign candidate")
|
||||
|
||||
core_webui = self.repo / "webui"
|
||||
core_webui.mkdir()
|
||||
dependency_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v0.1.10"
|
||||
package = {
|
||||
"name": "@govoplan/core-webui",
|
||||
"version": "0.1.10",
|
||||
"dependencies": {"@govoplan/campaign-webui": dependency_ref},
|
||||
}
|
||||
(core_webui / "package.release.json").write_text(json.dumps(package) + "\n", encoding="utf-8")
|
||||
(core_webui / "package-lock.release.json").write_text(
|
||||
json.dumps({"packages": {
|
||||
"": package,
|
||||
"node_modules/@govoplan/campaign-webui": {
|
||||
"version": "0.1.9",
|
||||
"resolved": f"git+ssh://git@example.test/acme/govoplan-campaign.git#{prior_commit}",
|
||||
},
|
||||
}}) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
git(self.repo, "add", "webui/package.release.json", "webui/package-lock.release.json")
|
||||
git(self.repo, "commit", "-m", "Stage Core input before candidate lock resolution")
|
||||
return campaign, remote
|
||||
|
||||
|
||||
def git(cwd: Path, *args: str) -> None:
|
||||
result = subprocess.run(
|
||||
@@ -338,9 +532,50 @@ def add_scoped_workflow_manifest(repo: Path) -> None:
|
||||
backend.mkdir(parents=True)
|
||||
(package / "__init__.py").write_text("", encoding="utf-8")
|
||||
(backend / "__init__.py").write_text("", encoding="utf-8")
|
||||
# This small workspace still has to satisfy the real presentation contract.
|
||||
# Keep that prerequisite shared by both valid and intentionally unscoped
|
||||
# documentation fixtures, so each test reaches its intended release gate.
|
||||
canonical_areas = runpy.run_path(
|
||||
str(META_ROOT / "tools" / "checks" / "check-manifest-shapes.py")
|
||||
)["CANONICAL_PRODUCT_AREAS"]
|
||||
(backend / "release_fixture.py").write_text(
|
||||
"""from govoplan_core.core.modules import FrontendModule, ProductAreaContribution
|
||||
from govoplan_core.core.views import ViewSurface
|
||||
|
||||
|
||||
def fixture_frontend():
|
||||
return FrontendModule(
|
||||
module_id="access",
|
||||
view_surfaces=(
|
||||
ViewSurface(
|
||||
id="access.section.release-fixture",
|
||||
module_id="access",
|
||||
kind="section",
|
||||
label="Release fixture",
|
||||
),
|
||||
),
|
||||
product_areas=tuple(
|
||||
ProductAreaContribution(
|
||||
id=area_id,
|
||||
module_id="access",
|
||||
label=label,
|
||||
icon=icon,
|
||||
description=description,
|
||||
order=order,
|
||||
surface_ids=("access.section.release-fixture",),
|
||||
)
|
||||
for area_id, (label, icon, description, order) in CANONICAL_AREAS.items()
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
CANONICAL_AREAS = """ + repr(canonical_areas) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(backend / "manifest.py").write_text(
|
||||
"""from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition
|
||||
from govoplan_core.core.provider_governance import declared_module_architecture
|
||||
from .release_fixture import fixture_frontend
|
||||
|
||||
|
||||
def get_manifest():
|
||||
@@ -348,6 +583,7 @@ def get_manifest():
|
||||
id="access",
|
||||
name="Access",
|
||||
version="0.1.10",
|
||||
frontend=fixture_frontend(),
|
||||
permissions=(
|
||||
PermissionDefinition(
|
||||
scope="access:item:read",
|
||||
@@ -403,6 +639,7 @@ def replace_with_unscoped_workflow_manifest(repo: Path) -> None:
|
||||
manifest.write_text(
|
||||
"""from govoplan_core.core.modules import DocumentationTopic, ModuleManifest
|
||||
from govoplan_core.core.provider_governance import declared_module_architecture
|
||||
from .release_fixture import fixture_frontend
|
||||
|
||||
|
||||
def get_manifest():
|
||||
@@ -410,6 +647,7 @@ def get_manifest():
|
||||
id="access",
|
||||
name="Access",
|
||||
version="0.1.10",
|
||||
frontend=fixture_frontend(),
|
||||
documentation=(
|
||||
DocumentationTopic(
|
||||
id="access.workflow.unscoped",
|
||||
|
||||
@@ -1891,7 +1891,7 @@ class ReleaseRunApiTests(unittest.TestCase):
|
||||
|
||||
def test_ui_and_runbook_state_tracking_boundary_are_explicit(self) -> None:
|
||||
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
|
||||
runbook = (META_ROOT / "docs" / "RELEASE_CONSOLE.md").read_text(
|
||||
runbook = (META_ROOT / "docs" / "operations" / "RELEASE_CONSOLE.md").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import source_tag_batch # noqa: E402
|
||||
from govoplan_release.repository_tag import tag_repositories # noqa: E402
|
||||
import test_release_meta_source_tag as meta_fixture # noqa: E402
|
||||
import test_release_repository_tag as release_fixture # noqa: E402
|
||||
from test_release_repository_tag import git, git_text, ref_exists # noqa: E402
|
||||
|
||||
|
||||
class RegisteredSourceTagBatchTests(unittest.TestCase):
|
||||
setUp = meta_fixture.MetaSourceTagTests.setUp
|
||||
synchronize = meta_fixture.MetaSourceTagTests.synchronize
|
||||
assert_no_tags = meta_fixture.MetaSourceTagTests.assert_no_tags
|
||||
|
||||
def tag(self, *, repos=("govoplan-access",), apply=False, push=False):
|
||||
return tag_repositories(
|
||||
repos=repos,
|
||||
repo_versions={repo: self.version for repo in repos},
|
||||
workspace_root=self.workspace,
|
||||
apply=apply,
|
||||
push=push,
|
||||
)
|
||||
|
||||
def test_python_only_module_publishes_without_meta_or_core_checkout(self):
|
||||
self.meta.rename(self.root / "unused-meta")
|
||||
self.core.rename(self.root / "unused-core")
|
||||
preview = self.tag(push=True)
|
||||
self.assertEqual("planned", preview["status"], preview)
|
||||
self.assertEqual("registered-source-batch-v1", preview["source_contract"])
|
||||
self.assertEqual({}, preview["bundle_input_receipts"])
|
||||
self.assertFalse(ref_exists(self.access, "refs/tags/v0.1.10"))
|
||||
published = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", published["status"], published)
|
||||
self.assertEqual(["govoplan-access"], list(published["source_receipts"]))
|
||||
self.assertEqual(
|
||||
git_text(self.access, "rev-parse", "HEAD"),
|
||||
git_text(self.access_remote, "rev-parse", "refs/heads/main"),
|
||||
)
|
||||
|
||||
def test_core_only_batch_has_no_meta_composition_requirement(self):
|
||||
self.meta.rename(self.root / "unused-meta")
|
||||
result = self.tag(repos=("govoplan-core",), apply=True, push=True)
|
||||
self.assertEqual("published", result["status"], result)
|
||||
self.assertEqual(["govoplan-core"], list(result["source_receipts"]))
|
||||
|
||||
def test_backend_only_publication_never_reads_irrelevant_unsafe_core_json(self):
|
||||
webui = self.core / "webui"
|
||||
webui.mkdir()
|
||||
(webui / "package.release.json").write_text("invalid unselected Core JSON")
|
||||
(webui / "package-lock.release.json").symlink_to(
|
||||
self.root / "not-a-core-release-lock"
|
||||
)
|
||||
from govoplan_release import version_alignment
|
||||
|
||||
original = version_alignment._json_object
|
||||
|
||||
def read(path):
|
||||
if path.is_relative_to(webui):
|
||||
raise AssertionError(
|
||||
"backend-only publication must not read unrelated Core JSON"
|
||||
)
|
||||
return original(path)
|
||||
|
||||
with patch.object(version_alignment, "_json_object", side_effect=read):
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("published", result["status"], result)
|
||||
self.assertEqual({}, result["bundle_input_receipts"])
|
||||
|
||||
def test_source_origin_parent_and_read_only_target_guards_apply_without_meta(self):
|
||||
for problem in ("origin", "parent", "read_only"):
|
||||
with self.subTest(problem=problem):
|
||||
parent_mode = self.root.stat().st_mode & 0o7777
|
||||
git_directory = self.access / ".git"
|
||||
git_mode = git_directory.stat().st_mode & 0o7777
|
||||
if problem == "origin":
|
||||
git(
|
||||
self.access,
|
||||
"config",
|
||||
"remote.origin.pushurl",
|
||||
str(self.root / "unknown.git"),
|
||||
)
|
||||
elif problem == "parent":
|
||||
self.root.chmod(0o777)
|
||||
else:
|
||||
git_directory.chmod(0o500)
|
||||
try:
|
||||
for apply in (False, True):
|
||||
result = self.tag(apply=apply, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
self.root.chmod(parent_mode)
|
||||
git_directory.chmod(git_mode)
|
||||
if problem == "origin":
|
||||
git(self.access, "config", "--unset", "remote.origin.pushurl")
|
||||
|
||||
def test_wrong_owner_symlink_and_hidden_index_are_rejected_without_meta(self):
|
||||
config = self.access / ".git/config"
|
||||
original_stat = Path.lstat
|
||||
|
||||
def wrong_owner(path, *args, **kwargs):
|
||||
observed = original_stat(path, *args, **kwargs)
|
||||
if path == config:
|
||||
fields = list(observed)
|
||||
fields[4] = os.geteuid() + 1
|
||||
return os.stat_result(fields)
|
||||
return observed
|
||||
|
||||
with patch.object(Path, "lstat", new=wrong_owner):
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
moved = self.root / "moved-access"
|
||||
self.access.rename(moved)
|
||||
self.access.symlink_to(moved, target_is_directory=True)
|
||||
try:
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
finally:
|
||||
self.access.unlink()
|
||||
moved.rename(self.access)
|
||||
for flag, undo in (
|
||||
("--assume-unchanged", "--no-assume-unchanged"),
|
||||
("--skip-worktree", "--no-skip-worktree"),
|
||||
):
|
||||
with self.subTest(flag=flag):
|
||||
git(self.access, "update-index", flag, "pyproject.toml")
|
||||
try:
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
self.assert_no_tags()
|
||||
finally:
|
||||
git(self.access, "update-index", undo, "pyproject.toml")
|
||||
|
||||
def test_live_remote_divergence_blocks_without_cached_tracking_update(self):
|
||||
clone = self.root / "other-access-writer"
|
||||
git(self.root, "clone", "--branch", "main", str(self.access_remote), str(clone))
|
||||
git(clone, "config", "user.name", "Synthetic writer")
|
||||
git(clone, "config", "user.email", "writer@example.invalid")
|
||||
(clone / "advance.txt").write_text("new remote main\n")
|
||||
git(clone, "add", ".")
|
||||
git(clone, "commit", "-m", "Advance remote without updating original tracking")
|
||||
git(clone, "push", "origin", "main")
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("live origin/main", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_ignored_selected_version_metadata_cannot_supply_an_untagged_artifact(self):
|
||||
project = self.access / "pyproject.toml"
|
||||
original = project.read_text()
|
||||
git(self.access, "rm", "pyproject.toml")
|
||||
(self.access / ".gitignore").write_text("/pyproject.toml\n")
|
||||
git(self.access, "add", ".gitignore")
|
||||
git(self.access, "commit", "-m", "Ignored metadata absent from frozen tree")
|
||||
project.write_text(original)
|
||||
self.assertEqual("", git_text(self.access, "status", "--porcelain"))
|
||||
result = self.tag(apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("must be tracked", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_lightweight_and_different_annotation_objects_remain_immutable(self):
|
||||
git(self.access, "tag", "v0.1.10")
|
||||
self.assertEqual("blocked", self.tag(apply=True)["status"])
|
||||
git(self.access, "tag", "-d", "v0.1.10")
|
||||
git(self.access, "tag", "-a", "v0.1.10", "-m", "Original immutable annotation")
|
||||
git(self.access, "push", "origin", "refs/tags/v0.1.10")
|
||||
original = git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
|
||||
git(self.access, "tag", "-d", "v0.1.10")
|
||||
git(
|
||||
self.access,
|
||||
"tag",
|
||||
"-a",
|
||||
"v0.1.10",
|
||||
"-m",
|
||||
"Conflicting immutable annotation",
|
||||
)
|
||||
self.assertEqual("blocked", self.tag(apply=True, push=True)["status"])
|
||||
self.assertEqual(
|
||||
original, git_text(self.access_remote, "rev-parse", "refs/tags/v0.1.10")
|
||||
)
|
||||
|
||||
def test_changed_source_after_preflight_stops_before_first_batch_effect(self):
|
||||
original = source_tag_batch._preview_repositories
|
||||
|
||||
def changed(**kwargs):
|
||||
result = original(**kwargs)
|
||||
(self.access / "changed-source.txt").write_text(
|
||||
"new source after preflight\n"
|
||||
)
|
||||
git(self.access, "add", ".")
|
||||
git(self.access, "commit", "-m", "Source changed")
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
source_tag_batch, "_preview_repositories", side_effect=changed
|
||||
):
|
||||
result = self.tag(
|
||||
repos=("govoplan-core", "govoplan-access"), apply=True, push=True
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("receipt changed", result["repositories"][0]["detail"])
|
||||
self.assert_no_tags()
|
||||
|
||||
def test_false_push_success_is_not_a_receipt_and_never_retries(self):
|
||||
original = source_tag_batch.run
|
||||
pushes = []
|
||||
|
||||
def run(command, **kwargs):
|
||||
if command[:3] == ("git", "push", "--atomic"):
|
||||
pushes.append(command)
|
||||
return subprocess.CompletedProcess(command, 0, "", "")
|
||||
return original(command, **kwargs)
|
||||
|
||||
with patch.object(source_tag_batch, "run", side_effect=run):
|
||||
result = self.tag(
|
||||
repos=("govoplan-access", "govoplan-core"), apply=True, push=True
|
||||
)
|
||||
self.assertEqual("partial", result["status"], result)
|
||||
self.assertEqual(1, len(pushes))
|
||||
self.assertEqual("skipped", result["repositories"][1]["status"])
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(self.access_remote, "refs/tags/v0.1.10"))
|
||||
|
||||
def _ready_bundle(self):
|
||||
self.repo = self.core # Existing fixture's Core name.
|
||||
campaign, remote = (
|
||||
release_fixture.ReleaseRepositoryTagTests._staged_campaign_bundle(self)
|
||||
)
|
||||
self.specs.append(
|
||||
{
|
||||
"name": "govoplan-campaign",
|
||||
"category": "module",
|
||||
"subtype": "domain",
|
||||
"path": "govoplan-campaign",
|
||||
"remote": str(remote),
|
||||
}
|
||||
)
|
||||
self.registry.write_text(json.dumps({"repositories": self.specs}))
|
||||
self.assertEqual(
|
||||
"tagged", self.tag(repos=("govoplan-campaign",), apply=True)["status"]
|
||||
)
|
||||
lock_path = self.core / "webui/package-lock.release.json"
|
||||
payload = json.loads(lock_path.read_text())
|
||||
package = payload["packages"]["node_modules/@govoplan/campaign-webui"]
|
||||
package["version"] = self.version
|
||||
package["resolved"] = (
|
||||
"git+ssh://git@example.test/acme/govoplan-campaign.git#"
|
||||
+ git_text(campaign, "rev-parse", "HEAD")
|
||||
)
|
||||
lock_path.write_text(json.dumps(payload))
|
||||
# Core is a reviewed input only here: do not require an unrelated tag
|
||||
# or silently impose a new clean-Core prerequisite for module release.
|
||||
return campaign, remote, lock_path
|
||||
|
||||
def test_module_publication_freezes_core_inputs_without_requiring_core_tag(self):
|
||||
_campaign, _remote, _lock = self._ready_bundle()
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
|
||||
self.assertEqual("published", result["status"], result)
|
||||
self.assertEqual(["govoplan-campaign"], list(result["source_receipts"]))
|
||||
self.assertEqual(
|
||||
{"webui/package.release.json", "webui/package-lock.release.json"},
|
||||
set(result["bundle_input_receipts"]),
|
||||
)
|
||||
self.assertFalse(ref_exists(self.core, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_changed_or_group_writable_core_bundle_inputs_block_module_publication(
|
||||
self,
|
||||
):
|
||||
campaign, remote, lock = self._ready_bundle()
|
||||
original = source_tag_batch._preview_repositories
|
||||
|
||||
def changed(**kwargs):
|
||||
result = original(**kwargs)
|
||||
lock.write_text(lock.read_text() + "\n")
|
||||
return result
|
||||
|
||||
with patch.object(
|
||||
source_tag_batch, "_preview_repositories", side_effect=changed
|
||||
):
|
||||
result = self.tag(repos=("govoplan-campaign",), apply=True, push=True)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn(
|
||||
"bundle input receipt changed", result["repositories"][0]["detail"]
|
||||
)
|
||||
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
|
||||
lock.chmod(0o666)
|
||||
try:
|
||||
self.assertEqual(
|
||||
"blocked",
|
||||
self.tag(repos=("govoplan-campaign",), apply=True, push=True)["status"],
|
||||
)
|
||||
finally:
|
||||
lock.chmod(0o644)
|
||||
self.assertTrue(ref_exists(campaign, "refs/tags/v0.1.10"))
|
||||
self.assertFalse(ref_exists(remote, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_read_only_shared_preflight_has_no_apply_or_mutating_legacy_entry(self):
|
||||
import inspect
|
||||
from govoplan_release import repository_tag
|
||||
|
||||
self.assertNotIn(
|
||||
"apply", inspect.signature(repository_tag._preview_repositories).parameters
|
||||
)
|
||||
self.assertFalse(hasattr(repository_tag, "_tag_repositories_legacy"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -13,12 +13,47 @@ if str(RELEASE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(RELEASE_ROOT))
|
||||
|
||||
from govoplan_release.version_metadata import ( # noqa: E402
|
||||
VersionMetadataError,
|
||||
apply_version_metadata_mutations,
|
||||
version_metadata_mutations,
|
||||
)
|
||||
|
||||
|
||||
class ReleaseVersionMetadataTests(unittest.TestCase):
|
||||
def test_updates_shared_module_version_without_rewriting_independent_interfaces(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
root = Path(temp_dir)
|
||||
backend = root / "src" / "govoplan_example" / "backend"
|
||||
backend.mkdir(parents=True)
|
||||
manifest = backend / "manifest.py"
|
||||
manifest.write_text(
|
||||
'MODULE_VERSION: str = "1.2.3"\n'
|
||||
'manifest = ModuleManifest(id="example", version=MODULE_VERSION,\n'
|
||||
' provides_interfaces=(ModuleInterfaceProvider(name="api", version="2.0"),))\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
changed = apply_version_metadata_mutations(root, target_version="1.2.4")
|
||||
self.assertEqual(("src/govoplan_example/backend/manifest.py",), changed)
|
||||
self.assertIn('MODULE_VERSION: str = "1.2.4"', manifest.read_text())
|
||||
self.assertIn('version="2.0"', manifest.read_text())
|
||||
self.assertEqual((), version_metadata_mutations(root, target_version="1.2.4"))
|
||||
|
||||
def test_dynamic_module_version_fails_before_any_metadata_is_written(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
root = Path(temp_dir)
|
||||
backend = root / "src" / "govoplan_example" / "backend"
|
||||
backend.mkdir(parents=True)
|
||||
project = root / "pyproject.toml"
|
||||
project.write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
|
||||
before = project.read_bytes()
|
||||
(backend / "manifest.py").write_text(
|
||||
'MODULE_VERSION = compute_version()\n'
|
||||
'manifest = ModuleManifest(id="example", version=MODULE_VERSION)\n',
|
||||
)
|
||||
with self.assertRaisesRegex(VersionMetadataError, "no literal MODULE_VERSION"):
|
||||
apply_version_metadata_mutations(root, target_version="1.2.4")
|
||||
self.assertEqual(before, project.read_bytes())
|
||||
|
||||
def test_updates_recognized_metadata_without_changing_interface_versions(
|
||||
self,
|
||||
) -> None:
|
||||
|
||||
@@ -98,6 +98,10 @@ class SecurityAuditWrapperTests(unittest.TestCase):
|
||||
if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then
|
||||
exit 0
|
||||
fi
|
||||
if [[ " $* " != *" --redact=100 "* ]]; then
|
||||
echo 'secret scans must redact reports and logs' >&2
|
||||
exit 3
|
||||
fi
|
||||
output=''
|
||||
while [[ $# -gt 0 ]]; do
|
||||
if [[ "$1" == '--report-path' ]]; then
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECK = runpy.run_path(str(META_ROOT / "tools/checks/check-webui-package-facades.py"))
|
||||
|
||||
|
||||
class WebuiPackageFacadeTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.repository = Path(self.temporary.name) / "govoplan-example"
|
||||
source = self.repository / "webui/src"
|
||||
source.mkdir(parents=True)
|
||||
(source / "index.ts").write_text("export {};\n", encoding="utf-8")
|
||||
(source / "styles.css").write_text(":root {}\n", encoding="utf-8")
|
||||
self.webui = {
|
||||
"name": "@govoplan/example-webui", "version": "0.1.2", "type": "module",
|
||||
"main": "src/index.ts",
|
||||
"exports": {".": {"import": "./src/index.ts"}, "./styles.css": "./src/styles.css"},
|
||||
"peerDependencies": {"@govoplan/core-webui": "^0.1.45"},
|
||||
"peerDependenciesMeta": {"@govoplan/core-webui": {"optional": True}},
|
||||
}
|
||||
self.root = {**self.webui, **{
|
||||
field: CHECK["prefixed_entries"](self.webui[field])
|
||||
for field in CHECK["ENTRY_FIELDS"] if field in self.webui
|
||||
}}
|
||||
self.write_manifests()
|
||||
|
||||
def write_manifests(self) -> None:
|
||||
(self.repository / "webui/package.json").write_text(json.dumps(self.webui), encoding="utf-8")
|
||||
(self.repository / "package.json").write_text(json.dumps(self.root), encoding="utf-8")
|
||||
|
||||
def issues(self) -> list[str]:
|
||||
return CHECK["facade_issues"](self.repository, package_name="@govoplan/example-webui")
|
||||
|
||||
def test_matching_conditional_and_css_entries_are_accepted(self) -> None:
|
||||
self.assertEqual([], self.issues())
|
||||
|
||||
def test_missing_root_or_generic_package_is_rejected(self) -> None:
|
||||
(self.repository / "package.json").unlink()
|
||||
self.assertIn("cannot read", " ".join(self.issues()))
|
||||
self.root = {"name": "@govoplan/example", "version": "0.1.2"}
|
||||
self.write_manifests()
|
||||
self.assertIn("root name differs", " ".join(self.issues()))
|
||||
self.assertIn("no WebUI entry point", " ".join(self.issues()))
|
||||
|
||||
def test_peer_drift_or_missing_entry_is_rejected(self) -> None:
|
||||
self.root["peerDependencies"] = {"@govoplan/core-webui": "^9.0.0"}
|
||||
self.write_manifests()
|
||||
self.assertIn("peerDependencies differs", " ".join(self.issues()))
|
||||
(self.repository / "webui/src/styles.css").unlink()
|
||||
self.assertIn("missing exports entry", " ".join(self.issues()))
|
||||
|
||||
def test_entry_cannot_escape_webui_even_if_it_exists(self) -> None:
|
||||
(self.repository / "outside.ts").write_text("export {};\n", encoding="utf-8")
|
||||
self.root["main"] = "webui/../outside.ts"
|
||||
self.write_manifests()
|
||||
self.assertIn("escapes webui/", " ".join(self.issues()))
|
||||
|
||||
def test_release_composition_checks_only_declared_module_sources(self) -> None:
|
||||
core = self.repository.parent / "govoplan-core/webui"
|
||||
core.mkdir(parents=True)
|
||||
(core / "package.release.json").write_text(json.dumps({"dependencies": {
|
||||
"react": "19.2.7",
|
||||
"@govoplan/example-webui": "git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-example.git#v0.1.2",
|
||||
}}), encoding="utf-8")
|
||||
self.assertEqual((1, []), CHECK["check_composition"](self.repository.parent))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,154 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import textwrap
|
||||
import unittest
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
INSTALLER = META_ROOT / "tools" / "release" / "install-webui-release-dependencies.sh"
|
||||
STAGES = ("base", "clone", "modules")
|
||||
|
||||
|
||||
class WebUIReleaseDependencyRetryTests(unittest.TestCase):
|
||||
def _run_installer(
|
||||
self, stage: str, statuses: tuple[int, ...]
|
||||
) -> tuple[subprocess.CompletedProcess[str], list[str]]:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-installer-retry-test-") as directory:
|
||||
root = Path(directory)
|
||||
stub_bin = root / "bin"
|
||||
stub_bin.mkdir()
|
||||
core_root = root / "core"
|
||||
webui = core_root / "web ui"
|
||||
webui.mkdir(parents=True)
|
||||
work_root = root / "work"
|
||||
work_root.mkdir()
|
||||
log = root / "commands.log"
|
||||
stub = "#!/usr/bin/env bash\nset -euo pipefail\n" + textwrap.dedent(
|
||||
r"""
|
||||
case "${0##*/}" in
|
||||
node)
|
||||
# Supply the shell's dependency list without requiring Node.
|
||||
printf '%s\t%s\n' '@govoplan/example-webui' \
|
||||
'git+https://example.invalid/module.git#v1.0.0' > "$GOVOPLAN_DEPS"
|
||||
printf 'node\n' >> "$RETRY_TEST_LOG"
|
||||
exit 0
|
||||
;;
|
||||
sleep)
|
||||
printf 'sleep %s\n' "$*" >> "$RETRY_TEST_LOG"
|
||||
exit 0
|
||||
;;
|
||||
npm)
|
||||
case "${1:-}" in
|
||||
cache)
|
||||
printf 'cache\n' >> "$RETRY_TEST_LOG"
|
||||
exit 0
|
||||
;;
|
||||
install)
|
||||
stage=base
|
||||
for argument in "$@"; do
|
||||
if [[ "$argument" == --no-save ]]; then
|
||||
stage=modules
|
||||
fi
|
||||
done
|
||||
;;
|
||||
*) exit 98 ;;
|
||||
esac
|
||||
;;
|
||||
git)
|
||||
[[ "${1:-}" == clone ]] || exit 98
|
||||
stage=clone
|
||||
;;
|
||||
*) exit 98 ;;
|
||||
esac
|
||||
|
||||
status=0
|
||||
if [[ "$stage" == "$RETRY_TEST_STAGE" ]]; then
|
||||
attempt=0
|
||||
counter="$RETRY_TEST_ROOT/$stage.count"
|
||||
if [[ -f "$counter" ]]; then
|
||||
read -r attempt < "$counter"
|
||||
fi
|
||||
read -r -a statuses <<< "$RETRY_TEST_STATUSES"
|
||||
status="${statuses[$attempt]:-99}"
|
||||
printf '%s\n' "$((attempt + 1))" > "$counter"
|
||||
fi
|
||||
printf '%s %s\n' "$stage" "$status" >> "$RETRY_TEST_LOG"
|
||||
exit "$status"
|
||||
"""
|
||||
)
|
||||
for name in ("node", "npm", "git", "sleep"):
|
||||
executable = stub_bin / name
|
||||
executable.write_text(stub, encoding="utf-8")
|
||||
executable.chmod(0o755)
|
||||
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
{
|
||||
"PATH": f"{stub_bin}:{os.defpath}",
|
||||
"TMPDIR": str(work_root),
|
||||
"GOVOPLAN_CORE_ROOT": str(core_root),
|
||||
"GOVOPLAN_WEBUI_PACKAGE_LOCK": "",
|
||||
"GOVOPLAN_WEBUI_PACKAGE_DIR": "",
|
||||
"RETRY_TEST_ROOT": str(root),
|
||||
"RETRY_TEST_LOG": str(log),
|
||||
"RETRY_TEST_STAGE": stage,
|
||||
"RETRY_TEST_STATUSES": " ".join(map(str, statuses)),
|
||||
}
|
||||
)
|
||||
result = subprocess.run(
|
||||
[
|
||||
"bash",
|
||||
"-c",
|
||||
'set -euo pipefail; bash "$1" "$2"; '
|
||||
'printf "caller-continued\\n" >> "$RETRY_TEST_LOG"',
|
||||
"retry-test-caller",
|
||||
str(INSTALLER),
|
||||
str(webui),
|
||||
],
|
||||
cwd=root,
|
||||
env=env,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=10,
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(list(work_root.iterdir()), [], result.stderr)
|
||||
return result, log.read_text(encoding="utf-8").splitlines()
|
||||
|
||||
def _assert_attempts(self, statuses: tuple[int, ...]) -> None:
|
||||
for retried_stage in STAGES:
|
||||
with self.subTest(stage=retried_stage, statuses=statuses):
|
||||
result, commands = self._run_installer(retried_stage, statuses)
|
||||
expected = ["node", "cache"]
|
||||
for stage in STAGES:
|
||||
attempts = statuses if stage == retried_stage else (0,)
|
||||
for index, status in enumerate(attempts):
|
||||
expected.append(f"{stage} {status}")
|
||||
if status and index < 2:
|
||||
expected.append(f"sleep {(index + 1) * 10}")
|
||||
if attempts[-1]:
|
||||
break
|
||||
if statuses[-1] == 0:
|
||||
expected.append("caller-continued")
|
||||
self.assertEqual(result.returncode, statuses[-1], result.stderr)
|
||||
self.assertEqual(commands, expected, result.stderr)
|
||||
|
||||
def test_success_on_first_attempt(self) -> None:
|
||||
self._assert_attempts((0,))
|
||||
|
||||
def test_success_on_second_attempt(self) -> None:
|
||||
self._assert_attempts((17, 0))
|
||||
|
||||
def test_success_on_third_attempt(self) -> None:
|
||||
self._assert_attempts((17, 23, 0))
|
||||
|
||||
def test_exhaustion_preserves_final_status_and_stops_callers(self) -> None:
|
||||
self._assert_attempts((17, 23, 47))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -39,13 +39,20 @@ GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py"
|
||||
"$NODE/node" "$META_ROOT/tests/test-jsx-value-imports.mjs"
|
||||
"$NODE/node" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
|
||||
"$NODE/node" "$META_ROOT/../govoplan-files/webui/scripts/test-archive-client.mjs"
|
||||
|
||||
cd "$META_ROOT"
|
||||
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
|
||||
"$PYTHON" tools/repo/sync-module-package-workflows.py --check
|
||||
"$PYTHON" tools/release/generate-developer-meta-package.py --check
|
||||
"$PYTHON" tools/checks/check-webui-package-facades.py
|
||||
"$PYTHON" -m unittest tests.test_webui_package_facades
|
||||
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
|
||||
"$PYTHON" -m unittest tests.test_deployment_installer
|
||||
"$PYTHON" -m unittest tests.test_deployment_installer tests.test_webui_release_dependency_retries
|
||||
"$PYTHON" -m pytest -q tests/test_release_meta_source_tag.py tests/test_release_source_tag_batch.py tests/test_release_meta_preparation.py
|
||||
"$PYTHON" -m unittest tests.test_isolated_work_composition
|
||||
"$PYTHON" -m unittest tests.test_capability_fit_evidence
|
||||
"$PYTHON" -m unittest tests.test_capability_fit_generation tests.test_capability_fit_review
|
||||
"$PYTHON" tools/assessments/generate-capability-fit-report.py --check
|
||||
@@ -96,6 +103,22 @@ PY
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-primitives.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-foundations.py"
|
||||
"$PYTHON" -m unittest tests.test_module_system
|
||||
"$PYTHON" -m unittest tests.test_bounded_process
|
||||
"$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api
|
||||
"$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_managed_archives.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_work.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_staging.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_upload_response_batching.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_performance.py
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_workers.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_inspection_bounds.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archives.py
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests/test_external_function_mapping_migration.py
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-templates/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
|
||||
@@ -116,19 +139,45 @@ PY
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_editor_state_security.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_mail_profile_boundary.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_independent_configuration_repairs.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_incremental_review_persistence.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_reviewed_build_mock.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_delivery_policy_settings.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_synchronous_delivery_policy.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_workerless_recovery.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_imap_batch_integration.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_testbed_claim_recovery.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_archive_encryption_governance.py \
|
||||
/mnt/DATA/git/govoplan-policy/tests/test_campaign_archive_encryption.py \
|
||||
/mnt/DATA/git/govoplan-policy/tests/test_archive_encryption_api.py
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-mail/tests/test_campaign_protocol_authorization.py /mnt/DATA/git/govoplan-mail/tests/test_campaign_imap_batch.py
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
|
||||
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
|
||||
"$PYTHON" -m unittest \
|
||||
tests.test_api_smoke.ApiSmokeTests.test_managed_attachment_patterns_preview_build_and_mock_send \
|
||||
tests.test_api_smoke.ApiSmokeTests.test_reports_and_job_review_are_scoped_to_the_selected_version \
|
||||
tests.test_api_smoke.ApiSmokeTests.test_worker_loss_becomes_unknown_and_requires_reconciliation_before_retry
|
||||
|
||||
cd "$ROOT/webui"
|
||||
"$NPM" run test:api-client-cache
|
||||
"$NPM" run test:auth-action-state
|
||||
"$NPM" run test:dependency-security
|
||||
"$NPM" run test:layout-primitives
|
||||
"$NPM" run test:mail-components
|
||||
"$NPM" run test:module-capabilities
|
||||
"$NPM" run test:module-permutations
|
||||
"$NPM" run test:conformance
|
||||
|
||||
cd /mnt/DATA/git/govoplan-access/webui
|
||||
"$NPM" run test:passwords
|
||||
|
||||
"$WEBUI_BIN/tsc" -p /mnt/DATA/git/govoplan-payments/webui/tsconfig.json
|
||||
|
||||
cd /mnt/DATA/git/govoplan-payments/webui
|
||||
@@ -155,12 +204,19 @@ cd /mnt/DATA/git/govoplan-postbox/webui
|
||||
cd /mnt/DATA/git/govoplan-mail/webui
|
||||
"$NPM" run test:mail-ui
|
||||
|
||||
cd /mnt/DATA/git/govoplan-files/webui
|
||||
"$NPM" run test:managed-archive
|
||||
|
||||
cd /mnt/DATA/git/govoplan-campaign/webui
|
||||
"$NPM" run test:policy-ui
|
||||
"$NPM" run test:template-preview
|
||||
"$NPM" run test:review-workflow
|
||||
"$NPM" run test:accessibility-contract
|
||||
"$NPM" run test:campaign-collaboration
|
||||
"$NPM" run test:campaign-work
|
||||
|
||||
cd /mnt/DATA/git/govoplan-policy/webui
|
||||
"$NPM" run test:archive-encryption
|
||||
|
||||
cd /mnt/DATA/git/govoplan-wiki/webui
|
||||
"$NPM" run test:interface-pattern
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env node
|
||||
/** Reject erased type-only imports used as runtime JSX component tags. */
|
||||
import { readFileSync, readdirSync, existsSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { resolve, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const workspaceRoot = resolve(import.meta.dirname, "../../..");
|
||||
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
|
||||
const ts = require("typescript");
|
||||
|
||||
function isTypeOnlyImport(declaration) {
|
||||
if (ts.isImportSpecifier(declaration)) return declaration.isTypeOnly || declaration.parent.parent.isTypeOnly;
|
||||
if (ts.isNamespaceImport(declaration)) return declaration.parent.isTypeOnly;
|
||||
return (ts.isImportClause(declaration) || ts.isImportEqualsDeclaration(declaration)) && declaration.isTypeOnly;
|
||||
}
|
||||
|
||||
/** Resolve lexical bindings, including shadowing; do not typecheck unrelated
|
||||
* optional dependencies or report ordinary application diagnostics.
|
||||
*/
|
||||
export function findTypeOnlyJsxImports(sources) {
|
||||
const files = new Map(sources.map(({ path, source }) => [resolve(path),
|
||||
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
|
||||
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
|
||||
const host = ts.createCompilerHost(options);
|
||||
host.getSourceFile = (path) => files.get(resolve(path));
|
||||
const program = ts.createProgram([...files.keys()], options, host);
|
||||
const checker = program.getTypeChecker();
|
||||
const findings = [];
|
||||
for (const [path, source] of files) {
|
||||
function visit(node) {
|
||||
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
|
||||
let root = node.tagName;
|
||||
// Lower-case direct tags are intrinsic HTML, not runtime bindings.
|
||||
if (!(ts.isIdentifier(root) && /^[a-z]/.test(root.text))) {
|
||||
while (ts.isPropertyAccessExpression(root)) root = root.expression;
|
||||
const declarations = checker.getSymbolAtLocation(root)?.declarations ?? [];
|
||||
if (declarations.some(isTypeOnlyImport)) {
|
||||
const position = source.getLineAndCharacterOfPosition(node.tagName.getStart(source));
|
||||
findings.push({ path, line: position.line + 1, column: position.character + 1, component: node.tagName.getText(source) });
|
||||
}
|
||||
}
|
||||
}
|
||||
ts.forEachChild(node, visit);
|
||||
}
|
||||
visit(source);
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
function sourceFiles(directory) {
|
||||
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
|
||||
const path = resolve(directory, entry.name);
|
||||
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
|
||||
});
|
||||
}
|
||||
|
||||
export function checkWorkspace(root = workspaceRoot) {
|
||||
const modules = readdirSync(root, { withFileTypes: true })
|
||||
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
|
||||
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
|
||||
const paths = modules.flatMap(sourceFiles);
|
||||
const findings = findTypeOnlyJsxImports(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
|
||||
for (const finding of findings) {
|
||||
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: JSX component ${finding.component} is imported type-only and will be erased at runtime.`);
|
||||
}
|
||||
if (!findings.length) console.log(`JSX runtime-import contract passed: ${paths.length} TSX files across ${modules.length} WebUI modules.`);
|
||||
return findings.length ? 1 : 0;
|
||||
}
|
||||
|
||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
process.exitCode = checkWorkspace();
|
||||
}
|
||||
@@ -677,6 +677,7 @@ run_gitleaks() {
|
||||
prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2
|
||||
prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2
|
||||
gitleaks git \
|
||||
--redact=100 \
|
||||
--config "$ROOT/.gitleaks.toml" \
|
||||
--report-format json \
|
||||
--report-path "$REPORTS_DIR/gitleaks-history-$name.json" \
|
||||
@@ -687,6 +688,7 @@ run_gitleaks() {
|
||||
# Scan the directory as well so pre-commit audits cover the exact code
|
||||
# under review, while retaining the history scan above.
|
||||
gitleaks dir \
|
||||
--redact=100 \
|
||||
--config "$ROOT/.gitleaks.toml" \
|
||||
--report-format json \
|
||||
--report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \
|
||||
@@ -696,6 +698,7 @@ run_gitleaks() {
|
||||
else
|
||||
prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2
|
||||
gitleaks detect \
|
||||
--redact=100 \
|
||||
--source "$repo" \
|
||||
--config "$ROOT/.gitleaks.toml" \
|
||||
--report-format json \
|
||||
|
||||
@@ -86,6 +86,12 @@ CENTRAL_COMPONENTS = {
|
||||
"CountBadge": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/CountBadge.tsx"
|
||||
),
|
||||
"MultiSelectFilter": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/MultiSelectFilter.tsx"
|
||||
),
|
||||
"ListSelectionFilter": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/ListSelectionFilter.tsx"
|
||||
),
|
||||
"SelectionList": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/SelectionList.tsx"
|
||||
),
|
||||
@@ -190,7 +196,21 @@ REQUIRED_CONSUMERS = {
|
||||
"CountBadge": (
|
||||
pathlib.Path("govoplan-core/webui/src/layout/Titlebar.tsx"),
|
||||
pathlib.Path("govoplan-mail/webui/src/features/mail/MailboxPage.tsx"),
|
||||
),
|
||||
# Search's old count badge was part of a retired module-local filter menu.
|
||||
# Both surfaces must now compose the owning facet adapter and Core dropdown.
|
||||
"SearchFilters": (
|
||||
pathlib.Path("govoplan-search/webui/src/features/search/SearchPage.tsx"),
|
||||
pathlib.Path("govoplan-search/webui/src/components/GlobalSearch.tsx"),
|
||||
),
|
||||
"MultiSelectFilter": (
|
||||
pathlib.Path("govoplan-search/webui/src/components/SearchFilters.tsx"),
|
||||
pathlib.Path("govoplan-notifications/webui/src/features/notifications/NotificationCenterPage.tsx"),
|
||||
pathlib.Path("govoplan-docs/webui/src/features/docs/DocsPage.tsx"),
|
||||
),
|
||||
"ListSelectionFilter": (
|
||||
pathlib.Path("govoplan-core/webui/src/components/MultiSelectFilter.tsx"),
|
||||
pathlib.Path("govoplan-core/webui/src/components/table/DataGrid.tsx"),
|
||||
),
|
||||
"SelectionListItemContent": (
|
||||
pathlib.Path("govoplan-approvals/webui/src/features/approvals/ApprovalsPage.tsx"),
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check that release Git dependencies expose their owning WebUI package."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[2]
|
||||
GIT_REPOSITORY = re.compile(r"/(govoplan-[a-z0-9-]+)\.git#v[^/]+$")
|
||||
PARITY_FIELDS = (
|
||||
"name", "version", "type", "dependencies", "optionalDependencies",
|
||||
"peerDependencies", "peerDependenciesMeta",
|
||||
)
|
||||
ENTRY_FIELDS = ("main", "module", "types", "exports")
|
||||
|
||||
|
||||
def prefixed_entries(value: object) -> object:
|
||||
if isinstance(value, str):
|
||||
return "./webui/" + value[2:] if value.startswith("./") else "webui/" + value
|
||||
if isinstance(value, dict):
|
||||
return {key: prefixed_entries(item) for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [prefixed_entries(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def entry_paths(value: object) -> list[str]:
|
||||
if isinstance(value, str):
|
||||
return [value]
|
||||
if isinstance(value, dict):
|
||||
return [path for item in value.values() for path in entry_paths(item)]
|
||||
if isinstance(value, list):
|
||||
return [path for item in value for path in entry_paths(item)]
|
||||
return []
|
||||
|
||||
|
||||
def facade_issues(repository: Path, *, package_name: str) -> list[str]:
|
||||
issues: list[str] = []
|
||||
try:
|
||||
root = json.loads((repository / "package.json").read_text(encoding="utf-8"))
|
||||
webui = json.loads((repository / "webui/package.json").read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError) as exc:
|
||||
return [f"{repository.name}: cannot read package facades: {exc}"]
|
||||
if not isinstance(root, dict) or not isinstance(webui, dict):
|
||||
return [f"{repository.name}: package manifests must be JSON objects"]
|
||||
if webui.get("name") != package_name:
|
||||
issues.append(f"{repository.name}: WebUI name does not match {package_name}")
|
||||
for field in PARITY_FIELDS:
|
||||
if root.get(field) != webui.get(field):
|
||||
issues.append(f"{repository.name}: root {field} differs from owning WebUI package")
|
||||
for field in ENTRY_FIELDS:
|
||||
expected = prefixed_entries(webui.get(field))
|
||||
if root.get(field) != expected:
|
||||
issues.append(f"{repository.name}: root {field} must target the corresponding webui/ entry")
|
||||
for entry in entry_paths(root.get(field)):
|
||||
path = repository / entry
|
||||
if not path.resolve().is_relative_to((repository / "webui").resolve()):
|
||||
issues.append(f"{repository.name}: {field} entry escapes webui/: {entry}")
|
||||
elif "*" not in entry and not path.is_file():
|
||||
issues.append(f"{repository.name}: missing {field} entry: {entry}")
|
||||
if not root.get("exports") and not root.get("main"):
|
||||
issues.append(f"{repository.name}: root package has no WebUI entry point")
|
||||
return issues
|
||||
|
||||
|
||||
def check_composition(workspace: Path, *, core_root: Path | None = None) -> tuple[int, list[str]]:
|
||||
core = core_root or workspace / "govoplan-core"
|
||||
release = json.loads((core / "webui/package.release.json").read_text(encoding="utf-8"))
|
||||
checked = 0
|
||||
issues: list[str] = []
|
||||
for name, reference in release.get("dependencies", {}).items():
|
||||
if not name.startswith("@govoplan/"):
|
||||
continue
|
||||
match = GIT_REPOSITORY.search(reference) if isinstance(reference, str) else None
|
||||
if match is None:
|
||||
issues.append(f"{name}: release dependency is not a versioned GovOPlaN Git source")
|
||||
continue
|
||||
checked += 1
|
||||
issues.extend(facade_issues(workspace / match.group(1), package_name=name))
|
||||
return checked, issues
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--workspace-root", type=Path, default=META_ROOT.parent)
|
||||
parser.add_argument("--core-root", type=Path)
|
||||
args = parser.parse_args()
|
||||
checked, issues = check_composition(args.workspace_root, core_root=args.core_root)
|
||||
if issues:
|
||||
print("\n".join(issues))
|
||||
return 1
|
||||
print(f"Release WebUI package facade checks passed for {checked} Git dependencies")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -4,8 +4,11 @@
|
||||
"certificates": "Contract-only module: certificate issuance and revocation persistence are not implemented; reassess before adding a migration-owned store.",
|
||||
"consultation": "Contract-only module: consultation submissions and evaluation persistence are not implemented; reassess before adding a migration-owned store.",
|
||||
"contracts": "Contract-only module: contract, amendment, and obligation persistence are not implemented; reassess before adding a migration-owned store.",
|
||||
"dms": "Stateless integration-preview module: DMS retains no document, person, credential, or provider-response store; Files and Records remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, or diagnostic.",
|
||||
"erp": "Stateless integration-contract module: ERP retains no invoice, payable, plan, booking observation, provider response, or credential store; Procurement, Payments, Ledger, Files, and Audit remain the subject-data owners. Reassess before persisting a target binding, plan, receipt, reconciliation decision, or diagnostic.",
|
||||
"evaluation": "Contract-only module: evaluation runs, responses, and scores are not persisted; reassess before adding a migration-owned store.",
|
||||
"facilities": "Contract-only module: facility and maintenance persistence are not implemented; reassess before adding a migration-owned store.",
|
||||
"fit_connect": "Stateless transport-contract module: FIT-Connect retains no submission, attachment, receipt, acknowledgement plan, key, provider response, or diagnostic store; the owning Service, Forms, Cases, Files, and Audit workflows remain responsible for subject data. Reassess before persisting any ingress or event-log evidence.",
|
||||
"grants": "Contract-only module: grant applications, awards, and monitoring are not persisted; reassess before adding a migration-owned store.",
|
||||
"inspections": "Contract-only module: inspections, findings, and measures are not persisted; reassess before adding a migration-owned store.",
|
||||
"learning": "Contract-only module: learning offers, enrollment, and completion are not persisted; reassess before adding a migration-owned store.",
|
||||
@@ -16,7 +19,7 @@
|
||||
"resources": "Contract-only module: resource catalog and allocation persistence are not implemented; reassess before adding a migration-owned store.",
|
||||
"rest": "Transport-only module: REST binds explicitly published functions and owns no domain or subject-data store.",
|
||||
"soap": "Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store.",
|
||||
"tenancy": "Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data.",
|
||||
"transparency": "Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store.",
|
||||
"workflow": "Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage."
|
||||
"workflow": "Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage.",
|
||||
"xrechnung": "Stateless validation-contract module: XRechnung persists no invoice, report, diagnostic, or handoff; the invoking Files, Procurement, or Payments workflow remains the subject-data owner. Reassess before adding a validation store."
|
||||
}
|
||||
|
||||
@@ -27,6 +27,7 @@ EXISTING_PROXY_FILENAME = "existing-proxy.json"
|
||||
PLAN_FILENAME = "plan.json"
|
||||
RECEIPT_FILENAME = "receipt.json"
|
||||
CAPABILITIES_FILENAME = "infrastructure-capabilities.json"
|
||||
DEPENDENCY_INVENTORY_FILENAME = "infrastructure-dependency-inventory.json"
|
||||
MANIFEST_FILENAME = "distribution-manifest.json"
|
||||
KEYRING_FILENAME = "distribution-keyring.json"
|
||||
BACKUP_EVIDENCE_FILENAME = "backup-evidence.json"
|
||||
@@ -116,6 +117,7 @@ class BundlePaths:
|
||||
plan: Path
|
||||
receipt: Path
|
||||
capabilities: Path
|
||||
dependency_inventory: Path
|
||||
manifest: Path
|
||||
keyring: Path
|
||||
backup_evidence: Path
|
||||
@@ -141,6 +143,7 @@ def bundle_paths(root: Path) -> BundlePaths:
|
||||
plan=resolved / PLAN_FILENAME,
|
||||
receipt=resolved / RECEIPT_FILENAME,
|
||||
capabilities=resolved / CAPABILITIES_FILENAME,
|
||||
dependency_inventory=resolved / DEPENDENCY_INVENTORY_FILENAME,
|
||||
manifest=resolved / MANIFEST_FILENAME,
|
||||
keyring=resolved / KEYRING_FILENAME,
|
||||
backup_evidence=resolved / BACKUP_EVIDENCE_FILENAME,
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
from datetime import UTC, datetime
|
||||
from typing import Mapping
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
@@ -18,6 +19,10 @@ CAPABILITY_STATES = frozenset(
|
||||
"unavailable",
|
||||
}
|
||||
)
|
||||
DEPENDENCY_INVENTORY_SCHEMA_VERSION = 1
|
||||
DEPENDENCY_STATES = frozenset(
|
||||
{"active", "inactive", "data_present", "pending_work", "runtime_binding"}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
@@ -50,13 +55,161 @@ class CapabilityChangeImpact:
|
||||
dependent_modules: tuple[str, ...]
|
||||
detail: str
|
||||
required_action: str
|
||||
actual_dependencies: tuple["CapabilityDependency", ...] = ()
|
||||
inventory_inspected: bool = False
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
value = asdict(self)
|
||||
value["dependent_modules"] = list(self.dependent_modules)
|
||||
value["actual_dependencies"] = [
|
||||
item.to_dict() for item in self.actual_dependencies
|
||||
]
|
||||
return value
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CapabilityDependency:
|
||||
capability_id: str
|
||||
module_id: str
|
||||
dependency_type: str
|
||||
dependency_ref: str
|
||||
state: str
|
||||
scope: str
|
||||
summary: str
|
||||
metrics: Mapping[str, int]
|
||||
required_action: str
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"capability_id": self.capability_id,
|
||||
"module_id": self.module_id,
|
||||
"dependency_type": self.dependency_type,
|
||||
"dependency_ref": self.dependency_ref,
|
||||
"state": self.state,
|
||||
"scope": self.scope,
|
||||
"summary": self.summary,
|
||||
"metrics": dict(sorted(self.metrics.items())),
|
||||
"required_action": self.required_action,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class InfrastructureDependencyInventory:
|
||||
installation_id: str
|
||||
generated_at: datetime
|
||||
complete: bool
|
||||
inspected_capability_ids: tuple[str, ...]
|
||||
provider_count: int
|
||||
dependencies: tuple[CapabilityDependency, ...]
|
||||
|
||||
def dependencies_for(
|
||||
self,
|
||||
capability_id: str,
|
||||
) -> tuple[CapabilityDependency, ...]:
|
||||
return tuple(
|
||||
item for item in self.dependencies if item.capability_id == capability_id
|
||||
)
|
||||
|
||||
|
||||
def infrastructure_dependency_inventory_from_mapping(
|
||||
value: object,
|
||||
) -> InfrastructureDependencyInventory:
|
||||
if (
|
||||
not isinstance(value, Mapping)
|
||||
or value.get("schema_version") != DEPENDENCY_INVENTORY_SCHEMA_VERSION
|
||||
):
|
||||
raise ValueError("Infrastructure dependency inventory schema is unsupported.")
|
||||
installation_id = _inventory_text(value, "installation_id", maximum=100)
|
||||
generated_at_text = _inventory_text(value, "generated_at", maximum=100)
|
||||
try:
|
||||
generated_at = datetime.fromisoformat(generated_at_text.replace("Z", "+00:00"))
|
||||
except ValueError as exc:
|
||||
raise ValueError(
|
||||
"Infrastructure dependency inventory timestamp is invalid."
|
||||
) from exc
|
||||
if generated_at.tzinfo is None:
|
||||
raise ValueError("Infrastructure dependency inventory timestamp needs a timezone.")
|
||||
generated_at = generated_at.astimezone(UTC)
|
||||
complete = value.get("complete")
|
||||
if type(complete) is not bool:
|
||||
raise ValueError("Infrastructure dependency inventory completion state is invalid.")
|
||||
inspected = _inventory_string_list(
|
||||
value.get("inspected_capability_ids"),
|
||||
maximum_items=100,
|
||||
maximum_length=120,
|
||||
)
|
||||
if len(inspected) != len(set(inspected)):
|
||||
raise ValueError("Infrastructure dependency inventory repeats a capability id.")
|
||||
providers = value.get("providers")
|
||||
if not isinstance(providers, list) or len(providers) > 100:
|
||||
raise ValueError("Infrastructure dependency provider reports are invalid.")
|
||||
provider_states: list[str] = []
|
||||
provider_declarations: dict[str, tuple[str, ...]] = {}
|
||||
provider_counts: dict[str, int] = {}
|
||||
for provider in providers:
|
||||
if not isinstance(provider, Mapping):
|
||||
raise ValueError("Infrastructure dependency provider report is invalid.")
|
||||
module_id = _inventory_text(provider, "module_id", maximum=120)
|
||||
if module_id in provider_declarations:
|
||||
raise ValueError("Infrastructure dependency provider is repeated.")
|
||||
state = _inventory_text(provider, "state", maximum=40)
|
||||
if state not in {"complete", "error"}:
|
||||
raise ValueError("Infrastructure dependency provider state is invalid.")
|
||||
provider_states.append(state)
|
||||
count = provider.get("dependency_count")
|
||||
if type(count) is not int or count < 0:
|
||||
raise ValueError("Infrastructure dependency provider count is invalid.")
|
||||
capability_ids = _inventory_string_list(
|
||||
provider.get("capability_ids"),
|
||||
maximum_items=30,
|
||||
maximum_length=120,
|
||||
)
|
||||
if len(capability_ids) != len(set(capability_ids)):
|
||||
raise ValueError("Infrastructure dependency provider capability is repeated.")
|
||||
provider_declarations[module_id] = capability_ids
|
||||
provider_counts[module_id] = count
|
||||
if complete and any(state != "complete" for state in provider_states):
|
||||
raise ValueError("Complete dependency inventory contains a failed provider.")
|
||||
raw_dependencies = value.get("dependencies")
|
||||
if not isinstance(raw_dependencies, list) or len(raw_dependencies) > 10_000:
|
||||
raise ValueError("Infrastructure dependency records are invalid.")
|
||||
dependencies = tuple(_inventory_dependency(item) for item in raw_dependencies)
|
||||
if any(
|
||||
capability_id not in inspected
|
||||
for capability_ids in provider_declarations.values()
|
||||
for capability_id in capability_ids
|
||||
):
|
||||
raise ValueError(
|
||||
"Infrastructure dependency provider was not covered by the inspection."
|
||||
)
|
||||
if any(item.capability_id not in inspected for item in dependencies):
|
||||
raise ValueError("Dependency record was not covered by the inventory inspection.")
|
||||
identities = {
|
||||
(item.capability_id, item.module_id, item.dependency_type, item.dependency_ref)
|
||||
for item in dependencies
|
||||
}
|
||||
if len(identities) != len(dependencies):
|
||||
raise ValueError("Infrastructure dependency inventory repeats a record.")
|
||||
observed_counts = {module_id: 0 for module_id in provider_counts}
|
||||
for dependency in dependencies:
|
||||
declarations = provider_declarations.get(dependency.module_id)
|
||||
if declarations is None or dependency.capability_id not in declarations:
|
||||
raise ValueError(
|
||||
"Infrastructure dependency is outside its provider declaration."
|
||||
)
|
||||
observed_counts[dependency.module_id] += 1
|
||||
if observed_counts != provider_counts:
|
||||
raise ValueError("Infrastructure dependency provider count does not match records.")
|
||||
return InfrastructureDependencyInventory(
|
||||
installation_id=installation_id,
|
||||
generated_at=generated_at,
|
||||
complete=complete,
|
||||
inspected_capability_ids=inspected,
|
||||
provider_count=len(providers),
|
||||
dependencies=dependencies,
|
||||
)
|
||||
|
||||
|
||||
def infrastructure_capability_document(
|
||||
spec: InstallationSpec,
|
||||
environment: Mapping[str, str],
|
||||
@@ -89,6 +242,8 @@ def infrastructure_capability_document(
|
||||
def capability_change_impacts(
|
||||
previous_document: object,
|
||||
desired_document: Mapping[str, object],
|
||||
*,
|
||||
dependency_inventory: InfrastructureDependencyInventory | None = None,
|
||||
) -> tuple[CapabilityChangeImpact, ...]:
|
||||
previous = _capability_map(previous_document)
|
||||
desired = _capability_map(desired_document)
|
||||
@@ -141,6 +296,43 @@ def capability_change_impacts(
|
||||
previous_secret_refs,
|
||||
desired_secret_refs,
|
||||
)
|
||||
actual_dependencies = (
|
||||
dependency_inventory.dependencies_for(capability_id)
|
||||
if dependency_inventory is not None
|
||||
else ()
|
||||
)
|
||||
inventory_inspected = bool(
|
||||
dependency_inventory is not None
|
||||
and capability_id in dependency_inventory.inspected_capability_ids
|
||||
)
|
||||
if inventory_inspected and actual_dependencies:
|
||||
references = ", ".join(
|
||||
f"{item.module_id}:{item.dependency_ref}"
|
||||
for item in actual_dependencies
|
||||
)
|
||||
inventory_detail = (
|
||||
f" Provider inventory reports {len(actual_dependencies)} persisted "
|
||||
f"dependency record(s): {references}."
|
||||
)
|
||||
elif inventory_inspected:
|
||||
inventory_detail = (
|
||||
" Provider inventory reports no persisted module-owned dependencies."
|
||||
)
|
||||
else:
|
||||
inventory_detail = " Provider inventory did not inspect this capability."
|
||||
dependency_actions = tuple(
|
||||
dict.fromkeys(
|
||||
item.required_action
|
||||
for item in actual_dependencies
|
||||
if item.required_action.strip()
|
||||
)
|
||||
)
|
||||
required_action = (
|
||||
"Review module-owned configuration and data migration or recovery "
|
||||
"evidence before apply."
|
||||
)
|
||||
if dependency_actions:
|
||||
required_action = f"{required_action} {' '.join(dependency_actions)}"
|
||||
impacts.append(
|
||||
CapabilityChangeImpact(
|
||||
capability_id=capability_id,
|
||||
@@ -153,11 +345,11 @@ def capability_change_impacts(
|
||||
detail=(
|
||||
f"{capability_id} changes from {previous_state}/{previous_source} "
|
||||
f"to {desired_state}/{desired_source}{binding_change}; "
|
||||
f"declared consumers: {dependent_label}."
|
||||
),
|
||||
required_action=(
|
||||
"Review module-owned configuration and data migration or recovery evidence before apply."
|
||||
f"declared consumers: {dependent_label}.{inventory_detail}"
|
||||
),
|
||||
required_action=required_action,
|
||||
actual_dependencies=actual_dependencies,
|
||||
inventory_inspected=inventory_inspected,
|
||||
)
|
||||
)
|
||||
return tuple(impacts)
|
||||
@@ -487,3 +679,73 @@ def _binding_change_label(
|
||||
if previous_secret_refs != desired_secret_refs:
|
||||
changes.append("secret-reference binding")
|
||||
return f" with changed {' and '.join(changes)}" if changes else ""
|
||||
|
||||
|
||||
def _inventory_text(
|
||||
value: Mapping[str, object],
|
||||
key: str,
|
||||
*,
|
||||
maximum: int,
|
||||
) -> str:
|
||||
raw = value.get(key)
|
||||
if not isinstance(raw, str):
|
||||
raise ValueError(f"Infrastructure dependency inventory {key} is invalid.")
|
||||
result = raw.strip()
|
||||
if not result or len(result) > maximum or any(ord(char) < 32 for char in result):
|
||||
raise ValueError(f"Infrastructure dependency inventory {key} is invalid.")
|
||||
return result
|
||||
|
||||
|
||||
def _inventory_string_list(
|
||||
value: object,
|
||||
*,
|
||||
maximum_items: int,
|
||||
maximum_length: int,
|
||||
) -> tuple[str, ...]:
|
||||
if not isinstance(value, list) or len(value) > maximum_items:
|
||||
raise ValueError("Infrastructure dependency inventory list is invalid.")
|
||||
items: list[str] = []
|
||||
for raw in value:
|
||||
if not isinstance(raw, str):
|
||||
raise ValueError("Infrastructure dependency inventory list is invalid.")
|
||||
item = raw.strip()
|
||||
if (
|
||||
not item
|
||||
or len(item) > maximum_length
|
||||
or any(ord(char) < 32 for char in item)
|
||||
):
|
||||
raise ValueError("Infrastructure dependency inventory list is invalid.")
|
||||
items.append(item)
|
||||
return tuple(items)
|
||||
|
||||
|
||||
def _inventory_dependency(value: object) -> CapabilityDependency:
|
||||
if not isinstance(value, Mapping):
|
||||
raise ValueError("Infrastructure dependency record is invalid.")
|
||||
state = _inventory_text(value, "state", maximum=40)
|
||||
if state not in DEPENDENCY_STATES:
|
||||
raise ValueError("Infrastructure dependency state is invalid.")
|
||||
raw_metrics = value.get("metrics")
|
||||
if not isinstance(raw_metrics, Mapping) or len(raw_metrics) > 20:
|
||||
raise ValueError("Infrastructure dependency metrics are invalid.")
|
||||
metrics: dict[str, int] = {}
|
||||
for raw_key, raw_count in raw_metrics.items():
|
||||
if not isinstance(raw_key, str):
|
||||
raise ValueError("Infrastructure dependency metric name is invalid.")
|
||||
key = raw_key.strip()
|
||||
if not key or len(key) > 80 or any(ord(char) < 32 for char in key):
|
||||
raise ValueError("Infrastructure dependency metric name is invalid.")
|
||||
if type(raw_count) is not int or raw_count < 0:
|
||||
raise ValueError("Infrastructure dependency metric value is invalid.")
|
||||
metrics[key] = raw_count
|
||||
return CapabilityDependency(
|
||||
capability_id=_inventory_text(value, "capability_id", maximum=120),
|
||||
module_id=_inventory_text(value, "module_id", maximum=120),
|
||||
dependency_type=_inventory_text(value, "dependency_type", maximum=120),
|
||||
dependency_ref=_inventory_text(value, "dependency_ref", maximum=240),
|
||||
state=state,
|
||||
scope=_inventory_text(value, "scope", maximum=120),
|
||||
summary=_inventory_text(value, "summary", maximum=1000),
|
||||
metrics=metrics,
|
||||
required_action=_inventory_text(value, "required_action", maximum=1000),
|
||||
)
|
||||
|
||||
@@ -18,7 +18,8 @@ import sys
|
||||
import time
|
||||
from typing import Iterator, Mapping, Sequence
|
||||
from urllib.error import URLError
|
||||
from urllib.request import urlopen
|
||||
from urllib.parse import urlsplit
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
from .backup_evidence import (
|
||||
DEFAULT_MAX_BACKUP_AGE_SECONDS,
|
||||
@@ -28,6 +29,7 @@ from .backup_evidence import (
|
||||
)
|
||||
from .bundle import (
|
||||
BACKUP_RUNTIME_ENV_KEYS,
|
||||
BundlePaths,
|
||||
atomic_write,
|
||||
bundle_paths,
|
||||
canonical_json,
|
||||
@@ -45,7 +47,11 @@ from .bundle import (
|
||||
service_names,
|
||||
write_env,
|
||||
)
|
||||
from .capabilities import infrastructure_capability_document
|
||||
from .capabilities import (
|
||||
InfrastructureDependencyInventory,
|
||||
infrastructure_capability_document,
|
||||
infrastructure_dependency_inventory_from_mapping,
|
||||
)
|
||||
from .cluster_evidence import collect_kubernetes_evidence
|
||||
from .distribution import (
|
||||
MAX_KEYRING_BYTES,
|
||||
@@ -81,6 +87,7 @@ from .kubernetes import (
|
||||
write_secret_creation_hint,
|
||||
)
|
||||
from .planning import (
|
||||
MAX_DEPENDENCY_INVENTORY_BYTES,
|
||||
DeploymentPlan,
|
||||
build_plan,
|
||||
release_change_requires_backup,
|
||||
@@ -152,6 +159,39 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
default=120.0,
|
||||
help="Maximum time to wait for the public health endpoint.",
|
||||
)
|
||||
apply_parser.add_argument(
|
||||
"--ops-url",
|
||||
help=(
|
||||
"Dependency inventory URL; defaults to "
|
||||
"<public-url>/api/v1/ops/infrastructure/dependencies."
|
||||
),
|
||||
)
|
||||
apply_parser.add_argument(
|
||||
"--api-key-env",
|
||||
default="GOVOPLAN_OPS_API_KEY",
|
||||
help=(
|
||||
"Environment variable containing an API key authorized to read "
|
||||
"Ops dependency inventory."
|
||||
),
|
||||
)
|
||||
|
||||
collect_inventory = subparsers.add_parser(
|
||||
"collect-infrastructure-inventory",
|
||||
help="Collect current module-owned capability dependencies from Ops.",
|
||||
)
|
||||
_directory_argument(collect_inventory)
|
||||
collect_inventory.add_argument(
|
||||
"--ops-url",
|
||||
help=(
|
||||
"Dependency inventory URL; defaults to "
|
||||
"<public-url>/api/v1/ops/infrastructure/dependencies."
|
||||
),
|
||||
)
|
||||
collect_inventory.add_argument(
|
||||
"--api-key-env",
|
||||
default="GOVOPLAN_OPS_API_KEY",
|
||||
help="Environment variable containing an authorized Ops API key.",
|
||||
)
|
||||
|
||||
status = subparsers.add_parser(
|
||||
"status", help="Show desired state and current Compose process state."
|
||||
@@ -425,6 +465,8 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
return _render_or_doctor(args)
|
||||
if args.command == "apply":
|
||||
return _apply(args)
|
||||
if args.command == "collect-infrastructure-inventory":
|
||||
return _collect_infrastructure_inventory(args)
|
||||
if args.command == "status":
|
||||
return _status(args)
|
||||
if args.command == "verify-release":
|
||||
@@ -586,6 +628,25 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
)
|
||||
secrets = reconcile_runtime_environment(spec, read_env(paths.env))
|
||||
secrets = _write_bundle(spec, paths, secrets)
|
||||
preliminary_plan = build_plan(spec, paths, include_host_checks=False)
|
||||
api_key_env = str(
|
||||
getattr(args, "api_key_env", "GOVOPLAN_OPS_API_KEY")
|
||||
).strip()
|
||||
api_key = os.environ.get(api_key_env, "").strip()
|
||||
if preliminary_plan.capability_impacts and api_key:
|
||||
try:
|
||||
_collect_dependency_inventory(
|
||||
spec,
|
||||
paths,
|
||||
ops_url=getattr(args, "ops_url", None),
|
||||
api_key=api_key,
|
||||
)
|
||||
print("Refreshed infrastructure dependency inventory from Ops.")
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
print(
|
||||
f"warning: could not refresh dependency inventory: {exc}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
plan = build_plan(spec, paths, include_host_checks=True)
|
||||
_write_plan(paths.plan, plan)
|
||||
effective_errors = [
|
||||
@@ -613,6 +674,8 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
if effective_errors:
|
||||
_print_plan(plan)
|
||||
raise ValueError("deployment plan is blocked; resolve doctor errors first")
|
||||
if plan.capability_impacts:
|
||||
_print_plan(plan)
|
||||
docker = shutil.which("docker")
|
||||
if docker is None:
|
||||
raise ValueError("Docker CLI is required for apply")
|
||||
@@ -761,6 +824,70 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _collect_infrastructure_inventory(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
spec = load_spec(paths.spec)
|
||||
api_key_env = str(args.api_key_env).strip()
|
||||
api_key = os.environ.get(api_key_env, "").strip()
|
||||
if not api_key:
|
||||
raise ValueError(f"{api_key_env} must contain an authorized Ops API key")
|
||||
inventory = _collect_dependency_inventory(
|
||||
spec,
|
||||
paths,
|
||||
ops_url=args.ops_url,
|
||||
api_key=api_key,
|
||||
)
|
||||
state = "complete" if inventory.complete else "incomplete"
|
||||
print(
|
||||
f"Collected {state} provider dependency inventory with "
|
||||
f"{len(inventory.dependencies)} record(s) at {paths.dependency_inventory}."
|
||||
)
|
||||
return 0 if inventory.complete else 1
|
||||
|
||||
|
||||
def _collect_dependency_inventory(
|
||||
spec: InstallationSpec,
|
||||
paths: BundlePaths,
|
||||
*,
|
||||
ops_url: str | None,
|
||||
api_key: str,
|
||||
) -> InfrastructureDependencyInventory:
|
||||
url = str(ops_url or "").strip() or (
|
||||
spec.public_url.rstrip("/")
|
||||
+ "/api/v1/ops/infrastructure/dependencies"
|
||||
)
|
||||
_validate_ops_inventory_url(url)
|
||||
request = Request(
|
||||
url,
|
||||
headers={"Accept": "application/json", "X-API-Key": api_key},
|
||||
)
|
||||
with urlopen(request, timeout=15) as response: # noqa: S310
|
||||
_validate_ops_inventory_url(response.geturl())
|
||||
encoded = response.read(MAX_DEPENDENCY_INVENTORY_BYTES + 1)
|
||||
if len(encoded) > MAX_DEPENDENCY_INVENTORY_BYTES:
|
||||
raise ValueError("Ops dependency inventory exceeds its size limit")
|
||||
value = json.loads(encoded)
|
||||
inventory = infrastructure_dependency_inventory_from_mapping(value)
|
||||
if inventory.installation_id != spec.installation_id:
|
||||
raise ValueError(
|
||||
"Ops dependency inventory belongs to a different installation"
|
||||
)
|
||||
ensure_private_directory(paths.root)
|
||||
atomic_write(paths.dependency_inventory, canonical_json(value), mode=0o600)
|
||||
return inventory
|
||||
|
||||
|
||||
def _validate_ops_inventory_url(url: str) -> None:
|
||||
parsed = urlsplit(url)
|
||||
if not parsed.hostname or parsed.username or parsed.password or parsed.fragment:
|
||||
raise ValueError("Ops dependency inventory URL is invalid")
|
||||
loopback = parsed.hostname in {"localhost", "127.0.0.1", "::1"}
|
||||
if parsed.scheme != "https" and not (parsed.scheme == "http" and loopback):
|
||||
raise ValueError(
|
||||
"Ops dependency inventory URL requires HTTPS except on loopback"
|
||||
)
|
||||
|
||||
|
||||
def _status(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
spec = load_spec(paths.spec)
|
||||
|
||||
@@ -13,7 +13,10 @@ from urllib.parse import urlsplit
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
DEFAULT_GARAGE_IMAGE = "dxflrs/garage:v2.3.0"
|
||||
DEFAULT_LOAD_BALANCER_IMAGE = "haproxy:3.2.21-alpine"
|
||||
DEFAULT_LOAD_BALANCER_IMAGE = (
|
||||
"haproxy:3.2.23-alpine@sha256:"
|
||||
"6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e"
|
||||
)
|
||||
DEFAULT_INGRESS_IMAGE = "caddy:2.10.2-alpine"
|
||||
INSTALLATION_ID_PATTERN = re.compile(r"^[a-z][a-z0-9-]{1,47}$")
|
||||
ENV_NAME_PATTERN = re.compile(r"^[A-Z][A-Z0-9_]{1,63}$")
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
from datetime import UTC, datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -37,8 +38,10 @@ from .bundle import (
|
||||
)
|
||||
from .capabilities import (
|
||||
CapabilityChangeImpact,
|
||||
InfrastructureDependencyInventory,
|
||||
capability_change_impacts,
|
||||
infrastructure_capability_document,
|
||||
infrastructure_dependency_inventory_from_mapping,
|
||||
)
|
||||
from .distribution import (
|
||||
MAX_KEYRING_BYTES,
|
||||
@@ -110,6 +113,9 @@ class DeploymentPlan:
|
||||
|
||||
|
||||
CommandRunner = Callable[[Sequence[str], Path], subprocess.CompletedProcess[str]]
|
||||
MAX_DEPENDENCY_INVENTORY_BYTES = 2 * 1024 * 1024
|
||||
DEPENDENCY_INVENTORY_MAX_AGE_SECONDS = 300
|
||||
DEPENDENCY_INVENTORY_MAX_FUTURE_SECONDS = 60
|
||||
|
||||
|
||||
def build_plan(
|
||||
@@ -135,9 +141,13 @@ def build_plan(
|
||||
spec,
|
||||
read_env(paths.env),
|
||||
)
|
||||
dependency_inventory, dependency_inventory_error = (
|
||||
_read_dependency_inventory(paths.dependency_inventory)
|
||||
)
|
||||
capability_impacts = capability_change_impacts(
|
||||
previous.get("infrastructure_capabilities"),
|
||||
infrastructure_capabilities,
|
||||
dependency_inventory=dependency_inventory,
|
||||
)
|
||||
|
||||
actions: list[PlanAction] = []
|
||||
@@ -215,6 +225,14 @@ def build_plan(
|
||||
)
|
||||
for impact in capability_impacts
|
||||
)
|
||||
checks.extend(
|
||||
_dependency_inventory_checks(
|
||||
spec,
|
||||
capability_impacts,
|
||||
dependency_inventory,
|
||||
dependency_inventory_error,
|
||||
)
|
||||
)
|
||||
if include_host_checks:
|
||||
checks.extend(host_checks(spec, paths, command_runner=command_runner))
|
||||
return DeploymentPlan(
|
||||
@@ -1187,6 +1205,106 @@ def _read_receipt(path: Path) -> Mapping[str, object]:
|
||||
return value if isinstance(value, dict) else {}
|
||||
|
||||
|
||||
def _read_dependency_inventory(
|
||||
path: Path,
|
||||
) -> tuple[InfrastructureDependencyInventory | None, str]:
|
||||
if not path.exists():
|
||||
return None, "missing"
|
||||
try:
|
||||
value = load_bounded_json(
|
||||
path,
|
||||
maximum_bytes=MAX_DEPENDENCY_INVENTORY_BYTES,
|
||||
)
|
||||
return infrastructure_dependency_inventory_from_mapping(value), ""
|
||||
except (DistributionError, ValueError) as exc:
|
||||
return None, str(exc)
|
||||
|
||||
|
||||
def _dependency_inventory_checks(
|
||||
spec: InstallationSpec,
|
||||
impacts: tuple[CapabilityChangeImpact, ...],
|
||||
inventory: InfrastructureDependencyInventory | None,
|
||||
inventory_error: str,
|
||||
) -> tuple[Check, ...]:
|
||||
if not impacts:
|
||||
return ()
|
||||
collect_action = (
|
||||
"Run govoplan-deploy collect-infrastructure-inventory with an Ops API "
|
||||
"key, then review the capability impacts before apply."
|
||||
)
|
||||
if inventory is None:
|
||||
if inventory_error == "missing":
|
||||
message = "Current provider dependency inventory is missing."
|
||||
check_id = "capability.dependency_inventory.missing"
|
||||
else:
|
||||
message = f"Provider dependency inventory is invalid: {inventory_error}"
|
||||
check_id = "capability.dependency_inventory.invalid"
|
||||
return (Check(check_id, "error", message, collect_action),)
|
||||
if inventory.installation_id != spec.installation_id:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.installation",
|
||||
"error",
|
||||
"Provider dependency inventory belongs to a different installation.",
|
||||
collect_action,
|
||||
),
|
||||
)
|
||||
if not inventory.complete:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.incomplete",
|
||||
"error",
|
||||
"Provider dependency inventory is incomplete because at least one provider failed.",
|
||||
"Resolve the provider failure and collect the inventory again.",
|
||||
),
|
||||
)
|
||||
age_seconds = (datetime.now(UTC) - inventory.generated_at).total_seconds()
|
||||
if age_seconds < -DEPENDENCY_INVENTORY_MAX_FUTURE_SECONDS:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.future",
|
||||
"error",
|
||||
"Provider dependency inventory timestamp is in the future.",
|
||||
"Correct host clock skew and collect the inventory again.",
|
||||
),
|
||||
)
|
||||
if age_seconds > DEPENDENCY_INVENTORY_MAX_AGE_SECONDS:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.stale",
|
||||
"error",
|
||||
"Provider dependency inventory is older than five minutes.",
|
||||
collect_action,
|
||||
),
|
||||
)
|
||||
impacted_ids = {item.capability_id for item in impacts}
|
||||
missing_ids = sorted(impacted_ids - set(inventory.inspected_capability_ids))
|
||||
if missing_ids:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.coverage",
|
||||
"error",
|
||||
"Provider dependency inventory did not inspect impacted capabilities: "
|
||||
+ ", ".join(missing_ids)
|
||||
+ ".",
|
||||
collect_action,
|
||||
),
|
||||
)
|
||||
matching_dependencies = sum(
|
||||
len(inventory.dependencies_for(capability_id))
|
||||
for capability_id in impacted_ids
|
||||
)
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.current",
|
||||
"ok",
|
||||
"Current provider inventory inspected every impacted capability and "
|
||||
f"reported {matching_dependencies} persisted dependency record(s) from "
|
||||
f"{inventory.provider_count} provider(s).",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _memory_bytes() -> int | None:
|
||||
try:
|
||||
for line in Path("/proc/meminfo").read_text(encoding="utf-8").splitlines():
|
||||
|
||||
@@ -34,6 +34,7 @@ _BUNDLE_FILES = (
|
||||
"backup-verification.json",
|
||||
"receipt.json",
|
||||
"infrastructure-capabilities.json",
|
||||
"infrastructure-dependency-inventory.json",
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1379,6 +1379,13 @@
|
||||
"rationale": "Operational worker, scheduler, reconciliation, or health endpoint; it is not a direct user surface.",
|
||||
"repository": "govoplan-notifications"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/ops/infrastructure/dependencies",
|
||||
"rationale": "Authorized host-deployer preflight consumes this provider inventory directly; it is private operational evidence rather than a product page.",
|
||||
"repository": "govoplan-ops"
|
||||
},
|
||||
{
|
||||
"category": "worker_internal",
|
||||
"method": "GET",
|
||||
@@ -1713,6 +1720,13 @@
|
||||
"rationale": "The module WebUI constructs this endpoint through a mounted router prefix, generic action, or provider path.",
|
||||
"repository": "govoplan-projects"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
"path": "/records/{}/access-grants/{}/revoke",
|
||||
"rationale": "The restricted-record access dialog revokes a grant through the shared dynamically constructed record mutation path.",
|
||||
"repository": "govoplan-records"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
@@ -2000,6 +2014,20 @@
|
||||
"rationale": "Published integration, interoperability, public-participant, or health endpoint.",
|
||||
"repository": "govoplan-soap"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/admin/tenant-erasure-policy",
|
||||
"rationale": "Tenant-erasure policy is a consequential operator API with recent-authentication and dedicated-permission gates.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "PATCH",
|
||||
"path": "/admin/tenant-erasure-policy",
|
||||
"rationale": "Tenant-erasure policy is a consequential operator API with recent-authentication and dedicated-permission gates.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
@@ -2007,6 +2035,48 @@
|
||||
"rationale": "Tenant deletion preflight is an administrative safety API consumed before a destructive workflow.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations",
|
||||
"rationale": "Tenant erasure is an audited, provider-driven operator workflow whose API exposes the complete review and recovery evidence.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}",
|
||||
"rationale": "Tenant erasure is an audited, provider-driven operator workflow whose API exposes the complete review and recovery evidence.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/approve",
|
||||
"rationale": "Tenant-erasure approval requires typed confirmation, recent authentication, and a distinct authorized account.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/cancel",
|
||||
"rationale": "Tenant-erasure cancellation is a recovery control available only before destructive work starts.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/execute",
|
||||
"rationale": "Tenant-erasure execution is a consequential operator API with provider checkpoints and fail-closed reconciliation.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/reconcile",
|
||||
"rationale": "Tenant-erasure reconciliation resumes idempotent provider steps after pending or outcome-unknown effects.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "compatibility",
|
||||
"method": "POST",
|
||||
@@ -2063,6 +2133,34 @@
|
||||
"rationale": "This capability-first module intentionally exposes a headless API for other modules and integrations.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/summaries",
|
||||
"rationale": "Workflow Engine publishes bounded, current-authorized summary discovery for module and API consumers; the existing Workflow UI retains its compatible full-history contract. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/{}/events",
|
||||
"rationale": "Workflow Engine publishes current-authorized, sequence-bounded event history pages with explicit total and continuation semantics for module and API consumers. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/{}/steps",
|
||||
"rationale": "Workflow Engine publishes current-authorized, sequence-bounded step history pages with explicit total and continuation semantics for module and API consumers. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/workflow/instances/{}/summary",
|
||||
"rationale": "Workflow Engine publishes a current-authorized, history-free instance summary for module and API consumers; the existing Workflow UI retains its compatible full-history detail contract. See owning topic workflow.instance-history.",
|
||||
"repository": "govoplan-workflow-engine"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
@@ -2189,6 +2287,27 @@
|
||||
"path": "/tasks/{}",
|
||||
"rationale": "Task command clients retrieve one explicit task and its strong revision token; the Work UI already receives the same projection through the aggregated list.",
|
||||
"repository": "govoplan-tasks"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/connectors/tabular-sources/{}/original-csv",
|
||||
"rationale": "Authorized connector clients explicitly export retained original CSV after tenant, lifecycle, read-scope and source-integrity checks; ordinary catalogue responses never include source text.",
|
||||
"repository": "govoplan-connectors"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/datasources/{}/materializations/{}/original-csv",
|
||||
"rationale": "Administrators explicitly export retained original CSV through an audited API; current and historical field, row and access policies must permit the entire original and source-integrity checks must pass.",
|
||||
"repository": "govoplan-datasources"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/mail/profiles/{}/pop3/imports/{}/bind-maildrop",
|
||||
"rationale": "Authorized mail operators explicitly reconcile a legacy POP3 import to the current maildrop using confirmed binding, a current transport revision token and an exact retained/downloaded-byte match; the audited API never guesses historical account identity.",
|
||||
"repository": "govoplan-mail"
|
||||
}
|
||||
],
|
||||
"schema_version": 1
|
||||
|
||||
@@ -63,6 +63,28 @@ const helpAttributes = new Set([
|
||||
"helperText",
|
||||
"helpText"
|
||||
]);
|
||||
const exactHelpAttributes = new Set([
|
||||
"data-help-context-id",
|
||||
"helpContextId"
|
||||
]);
|
||||
const helpRiskAttributes = new Set([
|
||||
"data-help-risk",
|
||||
"helpRisk"
|
||||
]);
|
||||
const reviewedHelpRiskAttributes = new Set([
|
||||
"data-help-risk-reviewed",
|
||||
"helpRiskReviewed"
|
||||
]);
|
||||
const supportedHelpRisks = new Set([
|
||||
"authority",
|
||||
"credential",
|
||||
"disclosure",
|
||||
"encryption",
|
||||
"external-effect",
|
||||
"irreversible",
|
||||
"policy",
|
||||
"retention"
|
||||
]);
|
||||
const actionComponentPattern = /(?:Action|Button|Link)$/;
|
||||
const contributionTypes = new Map([
|
||||
["AdminSectionsUiCapability", "admin_section"],
|
||||
@@ -200,20 +222,43 @@ function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
const parentAttributes = parentFormField
|
||||
? jsxAttributes(parentFormField)
|
||||
: new Map();
|
||||
const scopedAncestorAttributes = nearestScopedHelpAttributes(node);
|
||||
const label =
|
||||
attributes.get("label") ??
|
||||
attributes.get("aria-label") ??
|
||||
parentAttributes.get("label") ??
|
||||
null;
|
||||
const help = firstAttribute(attributes, helpAttributes) ??
|
||||
firstAttribute(parentAttributes, helpAttributes);
|
||||
firstAttribute(parentAttributes, helpAttributes) ??
|
||||
firstAttribute(scopedAncestorAttributes, helpAttributes);
|
||||
const hasHelp = hasAnyAttribute(attributes, helpAttributes) ||
|
||||
hasAnyAttribute(parentAttributes, helpAttributes);
|
||||
hasAnyAttribute(parentAttributes, helpAttributes) ||
|
||||
hasAnyAttribute(scopedAncestorAttributes, helpAttributes);
|
||||
const hasExactHelp = hasAnyAttribute(attributes, exactHelpAttributes) ||
|
||||
hasAnyAttribute(parentAttributes, exactHelpAttributes) ||
|
||||
hasAnyAttribute(scopedAncestorAttributes, exactHelpAttributes);
|
||||
const helpContextId = firstAttribute(attributes, exactHelpAttributes) ??
|
||||
firstAttribute(parentAttributes, exactHelpAttributes) ??
|
||||
firstAttribute(scopedAncestorAttributes, exactHelpAttributes);
|
||||
const explicitId = firstAttribute(
|
||||
attributes,
|
||||
new Set(["interfaceId", "data-interface-id", "id", "name", "field"])
|
||||
);
|
||||
const context = nearestNamedContext(node);
|
||||
const risk = helpRiskFor({
|
||||
component,
|
||||
context,
|
||||
file: relativeFile,
|
||||
label,
|
||||
explicitId,
|
||||
name: attributes.get("name") ?? attributes.get("id") ?? attributes.get("field") ?? null,
|
||||
explicitRisk: firstAttribute(attributes, helpRiskAttributes) ??
|
||||
firstAttribute(parentAttributes, helpRiskAttributes) ??
|
||||
firstAttribute(scopedAncestorAttributes, helpRiskAttributes)
|
||||
});
|
||||
const riskReviewed = firstAttribute(attributes, reviewedHelpRiskAttributes) ??
|
||||
firstAttribute(parentAttributes, reviewedHelpRiskAttributes) ??
|
||||
firstAttribute(scopedAncestorAttributes, reviewedHelpRiskAttributes);
|
||||
const stableId = sourceIdentity(
|
||||
"field",
|
||||
node,
|
||||
@@ -237,7 +282,14 @@ function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
help: help ?? null,
|
||||
helpId: hasHelp ? `${stableId}.help` : null,
|
||||
helpDynamic: hasHelp && help === null,
|
||||
helpCandidate: !hasHelp
|
||||
helpCandidate: !hasHelp,
|
||||
helpExact: hasExactHelp,
|
||||
helpContextId,
|
||||
helpContextDynamic: hasExactHelp && helpContextId === null,
|
||||
helpRisk: risk.value,
|
||||
helpRiskSource: risk.source,
|
||||
helpRiskReviewed: riskReviewed,
|
||||
highRiskHelpMissing: risk.value !== null && !hasExactHelp && riskReviewed !== "standard"
|
||||
});
|
||||
|
||||
}
|
||||
@@ -261,6 +313,19 @@ function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
new Set(["interfaceId", "data-interface-id", "id", "name"])
|
||||
);
|
||||
const context = nearestNamedContext(node);
|
||||
const hasHelp = hasAnyAttribute(attributes, helpAttributes);
|
||||
const hasExactHelp = hasAnyAttribute(attributes, exactHelpAttributes);
|
||||
const helpContextId = firstAttribute(attributes, exactHelpAttributes);
|
||||
const risk = helpRiskFor({
|
||||
component,
|
||||
context,
|
||||
file: relativeFile,
|
||||
label,
|
||||
explicitId,
|
||||
name: attributes.get("name") ?? attributes.get("id") ?? null,
|
||||
explicitRisk: firstAttribute(attributes, helpRiskAttributes)
|
||||
});
|
||||
const riskReviewed = firstAttribute(attributes, reviewedHelpRiskAttributes);
|
||||
result.actions.push({
|
||||
...locate(node),
|
||||
id: sourceIdentity(
|
||||
@@ -273,7 +338,15 @@ function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
idSource: explicitId === null ? "source_anchor" : "explicit",
|
||||
context,
|
||||
component,
|
||||
label
|
||||
label,
|
||||
helpExact: hasExactHelp,
|
||||
helpContextId,
|
||||
helpContextDynamic: hasExactHelp && helpContextId === null,
|
||||
helpDynamic: hasHelp && firstAttribute(attributes, helpAttributes) === null,
|
||||
helpRisk: risk.value,
|
||||
helpRiskSource: risk.source,
|
||||
helpRiskReviewed: riskReviewed,
|
||||
highRiskHelpMissing: risk.value !== null && !hasExactHelp && riskReviewed !== "standard"
|
||||
});
|
||||
}
|
||||
|
||||
@@ -354,6 +427,26 @@ function inspectSource(repository, sourceRoot, sourcePath) {
|
||||
return null;
|
||||
}
|
||||
|
||||
function nearestScopedHelpAttributes(node) {
|
||||
let current = node.parent;
|
||||
while (current) {
|
||||
if (ts.isJsxElement(current)) {
|
||||
const attributes = jsxAttributes(current.openingElement);
|
||||
if (attributes.get("data-help-scope") === "field") return attributes;
|
||||
}
|
||||
if (
|
||||
ts.isFunctionDeclaration(current) ||
|
||||
ts.isMethodDeclaration(current) ||
|
||||
ts.isArrowFunction(current) ||
|
||||
ts.isFunctionExpression(current)
|
||||
) {
|
||||
return new Map();
|
||||
}
|
||||
current = current.parent;
|
||||
}
|
||||
return new Map();
|
||||
}
|
||||
|
||||
function jsxAttributes(node) {
|
||||
const mapped = new Map();
|
||||
for (const attribute of node.attributes.properties) {
|
||||
@@ -579,6 +672,34 @@ function hasAnyAttribute(attributes, names) {
|
||||
return false;
|
||||
}
|
||||
|
||||
function helpRiskFor({ component, context, file, label, explicitId, name, explicitRisk }) {
|
||||
if (typeof explicitRisk === "string") {
|
||||
return supportedHelpRisks.has(explicitRisk)
|
||||
? { value: explicitRisk, source: "explicit" }
|
||||
: { value: null, source: "invalid_explicit" };
|
||||
}
|
||||
const value = [component, context, file, label, explicitId, name]
|
||||
.filter((item) => typeof item === "string")
|
||||
.join(" ")
|
||||
.toLowerCase()
|
||||
.replace(/^i18n:/g, "")
|
||||
.replace(/[._-]+/g, " ");
|
||||
const patterns = [
|
||||
["irreversible", /\b(delete|destroy|erase|purge|dispose|disposition|revoke|withdraw|shred)\b/],
|
||||
["credential", /\b(credential|password|secret|token|api key|private key)\b/],
|
||||
["retention", /\b(retention|legal hold|archive lifecycle)\b/],
|
||||
["encryption", /\b(encrypt|encryption|decrypt|decryption|signing key|signature key)\b/],
|
||||
["disclosure", /\b(disclose|disclosure|publish|share externally|public export)\b/],
|
||||
["external-effect", /\b(send|deliver|transfer|refund|payment execution|webhook execution)\b/],
|
||||
["authority", /\b(grant permission|role assignment|approve|reject|formal decision|mandate)\b/],
|
||||
["policy", /\b(policy apply|policy override|enforcement mode)\b/]
|
||||
];
|
||||
for (const [risk, pattern] of patterns) {
|
||||
if (pattern.test(value)) return { value: risk, source: "inferred" };
|
||||
}
|
||||
return { value: null, source: null };
|
||||
}
|
||||
|
||||
function slug(value) {
|
||||
const normalized = value
|
||||
.toLowerCase()
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"maximum_missing_exact_help": 0,
|
||||
"rationale": "The source-derived high-risk queue for Core issue #284 is fully resolved. Strict declarations reject any new high-risk control without an exact, manifest-declared, German-complete F1 context."
|
||||
}
|
||||
@@ -31,6 +31,9 @@ ENDPOINT_SURFACE_CATEGORIES = {
|
||||
DEFAULT_ENDPOINT_DECLARATIONS = (
|
||||
META_ROOT / "tools" / "inventory" / "endpoint-surface-declarations.json"
|
||||
)
|
||||
DEFAULT_HIGH_RISK_HELP_BASELINE = (
|
||||
META_ROOT / "tools" / "inventory" / "high-risk-help-baseline.json"
|
||||
)
|
||||
REQUIRED_LOCALES = ("de", "en")
|
||||
REFERENCE_LOCALE = "de"
|
||||
|
||||
@@ -75,6 +78,12 @@ def main() -> int:
|
||||
default=DEFAULT_ENDPOINT_DECLARATIONS,
|
||||
help="Versioned endpoint-surface declaration registry.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--high-risk-help-baseline",
|
||||
type=Path,
|
||||
default=DEFAULT_HIGH_RISK_HELP_BASELINE,
|
||||
help="Versioned upper bound for high-risk controls without exact F1 help.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
|
||||
@@ -85,11 +94,15 @@ def main() -> int:
|
||||
endpoint_declarations = _load_endpoint_declarations(
|
||||
args.endpoint_declarations.resolve()
|
||||
)
|
||||
high_risk_help_baseline = _load_high_risk_help_baseline(
|
||||
args.high_risk_help_baseline.resolve()
|
||||
)
|
||||
inventory = _assemble_inventory(
|
||||
webui=webui,
|
||||
backend_endpoints=backend_endpoints,
|
||||
manifests=manifests,
|
||||
endpoint_declarations=endpoint_declarations,
|
||||
high_risk_help_baseline=high_risk_help_baseline,
|
||||
runtime_snapshot=(
|
||||
_load_runtime_snapshot(args.runtime_snapshot.resolve())
|
||||
if args.runtime_snapshot is not None
|
||||
@@ -164,6 +177,28 @@ def _strict_failures(
|
||||
f"{len(declaration_health['stale_runtime_routes'])} runtime route "
|
||||
"declarations have no WebUI implementation"
|
||||
)
|
||||
help_health = inventory.get("help_health", {})
|
||||
if check_declarations and help_health.get("invalid_risk_annotations"):
|
||||
failures.append(
|
||||
f"{len(help_health['invalid_risk_annotations'])} controls use an "
|
||||
"unsupported contextual-help risk class"
|
||||
)
|
||||
if check_declarations and help_health.get("baseline_regression"):
|
||||
failures.append(
|
||||
f"{len(help_health['missing_exact_high_risk_help'])} high-risk "
|
||||
"controls lack exact F1 help; baseline permits at most "
|
||||
f"{help_health['baseline_maximum_missing']}"
|
||||
)
|
||||
if check_declarations and help_health.get("unresolved_exact_high_risk_help"):
|
||||
failures.append(
|
||||
f"{len(help_health['unresolved_exact_high_risk_help'])} high-risk "
|
||||
"controls reference no manifest DocumentationTopic help context"
|
||||
)
|
||||
if check_declarations and help_health.get("high_risk_help_without_german"):
|
||||
failures.append(
|
||||
f"{len(help_health['high_risk_help_without_german'])} high-risk "
|
||||
"controls resolve to documentation without complete German content"
|
||||
)
|
||||
runtime_comparison = inventory.get("runtime_comparison")
|
||||
if (
|
||||
check_declarations
|
||||
@@ -355,6 +390,29 @@ def _extract_manifests(
|
||||
}
|
||||
for permission in manifest.permissions
|
||||
],
|
||||
"documentation": [
|
||||
{
|
||||
"id": topic.id,
|
||||
"help_contexts": sorted(
|
||||
{
|
||||
str(context)
|
||||
for context in topic.metadata.get(
|
||||
"help_contexts", ()
|
||||
)
|
||||
if isinstance(context, str) and context.strip()
|
||||
}
|
||||
),
|
||||
"german_complete": (
|
||||
isinstance(topic.translations.get("de"), dict)
|
||||
and all(
|
||||
isinstance(topic.translations["de"].get(field), str)
|
||||
and topic.translations["de"][field].strip()
|
||||
for field in ("title", "summary", "body")
|
||||
)
|
||||
),
|
||||
}
|
||||
for topic in manifest.documentation
|
||||
],
|
||||
"architecture": (
|
||||
manifest.architecture.to_dict()
|
||||
if manifest.architecture is not None
|
||||
@@ -400,6 +458,7 @@ def _assemble_inventory(
|
||||
backend_endpoints: list[dict[str, Any]],
|
||||
manifests: list[dict[str, Any]],
|
||||
endpoint_declarations: dict[tuple[str, str, str], dict[str, Any]],
|
||||
high_risk_help_baseline: dict[str, Any] | None = None,
|
||||
runtime_snapshot: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
frontend_refs = webui["frontendApiReferences"]
|
||||
@@ -471,8 +530,47 @@ def _assemble_inventory(
|
||||
if any(key not in catalog_keys.get(locale, set()) for locale in expected_locales)
|
||||
]
|
||||
fields = webui["fields"]
|
||||
actions = webui.get("actions", [])
|
||||
help_candidates = [field for field in fields if field["helpCandidate"]]
|
||||
dynamic_help = [field for field in fields if field.get("helpDynamic")]
|
||||
controls = [*fields, *actions]
|
||||
high_risk_controls = [item for item in controls if item.get("helpRisk")]
|
||||
missing_exact_high_risk_help = [
|
||||
item for item in controls if item.get("highRiskHelpMissing")
|
||||
]
|
||||
invalid_risk_annotations = [
|
||||
item
|
||||
for item in controls
|
||||
if item.get("helpRiskSource") == "invalid_explicit"
|
||||
]
|
||||
documentation_contexts = {
|
||||
context: {
|
||||
"module_id": manifest["id"],
|
||||
"topic_id": topic["id"],
|
||||
"german_complete": topic["german_complete"],
|
||||
}
|
||||
for manifest in manifests
|
||||
for topic in manifest.get("documentation", [])
|
||||
for context in topic.get("help_contexts", [])
|
||||
}
|
||||
unresolved_exact_high_risk_help = [
|
||||
item
|
||||
for item in high_risk_controls
|
||||
if item.get("helpExact")
|
||||
and not item.get("helpContextDynamic")
|
||||
and item.get("helpContextId") not in documentation_contexts
|
||||
]
|
||||
high_risk_help_without_german = [
|
||||
item
|
||||
for item in high_risk_controls
|
||||
if item.get("helpContextId") in documentation_contexts
|
||||
and not documentation_contexts[item["helpContextId"]]["german_complete"]
|
||||
]
|
||||
baseline_maximum_missing = (
|
||||
high_risk_help_baseline["maximum_missing_exact_help"]
|
||||
if high_risk_help_baseline is not None
|
||||
else None
|
||||
)
|
||||
governance_adoption = Counter(
|
||||
dimension["adoption"]
|
||||
for manifest in manifests
|
||||
@@ -499,10 +597,34 @@ def _assemble_inventory(
|
||||
"modules": manifests,
|
||||
"interface_declarations": source_declarations,
|
||||
"declaration_health": declaration_health,
|
||||
"help_health": {
|
||||
"supported_risk_classes": sorted(
|
||||
{
|
||||
str(item["helpRisk"])
|
||||
for item in high_risk_controls
|
||||
if item.get("helpRisk")
|
||||
}
|
||||
),
|
||||
"high_risk_controls": high_risk_controls,
|
||||
"missing_exact_high_risk_help": missing_exact_high_risk_help,
|
||||
"invalid_risk_annotations": invalid_risk_annotations,
|
||||
"unresolved_exact_high_risk_help": unresolved_exact_high_risk_help,
|
||||
"high_risk_help_without_german": high_risk_help_without_german,
|
||||
"dynamic_owner_context_controls": [
|
||||
item
|
||||
for item in high_risk_controls
|
||||
if item.get("helpContextDynamic")
|
||||
],
|
||||
"baseline_maximum_missing": baseline_maximum_missing,
|
||||
"baseline_regression": (
|
||||
baseline_maximum_missing is not None
|
||||
and len(missing_exact_high_risk_help) > baseline_maximum_missing
|
||||
),
|
||||
},
|
||||
"runtime_comparison": runtime_comparison,
|
||||
"ui": {
|
||||
"fields": fields,
|
||||
"actions": webui.get("actions", []),
|
||||
"actions": actions,
|
||||
"labels": webui["labels"],
|
||||
"visible_text": webui["visibleText"],
|
||||
"routes": webui["routes"],
|
||||
@@ -554,7 +676,19 @@ def _assemble_inventory(
|
||||
"ui_fields_with_resolvable_f1_context": len(fields),
|
||||
"help_review_candidates": len(help_candidates),
|
||||
"dynamic_help_references": len(dynamic_help),
|
||||
"ui_actions": len(webui.get("actions", [])),
|
||||
"ui_actions": len(actions),
|
||||
"high_risk_controls": len(high_risk_controls),
|
||||
"high_risk_controls_with_exact_help": (
|
||||
len(high_risk_controls) - len(missing_exact_high_risk_help)
|
||||
),
|
||||
"high_risk_controls_missing_exact_help": len(
|
||||
missing_exact_high_risk_help
|
||||
),
|
||||
"invalid_help_risk_annotations": len(invalid_risk_annotations),
|
||||
"unresolved_exact_high_risk_help": len(
|
||||
unresolved_exact_high_risk_help
|
||||
),
|
||||
"high_risk_help_without_german": len(high_risk_help_without_german),
|
||||
"interface_declarations": len(source_declarations),
|
||||
"duplicate_interface_ids": len(declaration_health["duplicate_ids"]),
|
||||
"undeclared_source_surfaces": len(
|
||||
@@ -885,6 +1019,10 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
|
||||
for item in inventory["api"]["unreferenced_by_static_webui_scan"]
|
||||
)
|
||||
classification_counts = inventory["api"]["classification_counts"]
|
||||
high_risk_by_repository = Counter(
|
||||
item["repository"]
|
||||
for item in inventory["help_health"]["missing_exact_high_risk_help"]
|
||||
)
|
||||
lines = [
|
||||
"# GovOPlaN Platform Interface Inventory",
|
||||
"",
|
||||
@@ -900,6 +1038,12 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
|
||||
f"- Fields with a resolvable F1 context: {summary['ui_fields_with_resolvable_f1_context']}",
|
||||
f"- Fields with dynamic help references: {summary['dynamic_help_references']}",
|
||||
f"- Help review candidates: {summary['help_review_candidates']}",
|
||||
f"- High-risk controls: {summary['high_risk_controls']}",
|
||||
f"- High-risk controls with exact F1 help: {summary['high_risk_controls_with_exact_help']}",
|
||||
f"- High-risk controls missing exact F1 help: {summary['high_risk_controls_missing_exact_help']}",
|
||||
f"- Invalid help-risk annotations: {summary['invalid_help_risk_annotations']}",
|
||||
f"- High-risk exact contexts missing a manifest topic: {summary['unresolved_exact_high_risk_help']}",
|
||||
f"- High-risk contexts without complete German topic content: {summary['high_risk_help_without_german']}",
|
||||
f"- Stable interface declarations: {summary['interface_declarations']}",
|
||||
f"- Duplicate interface IDs: {summary['duplicate_interface_ids']}",
|
||||
f"- WebUI surfaces missing runtime declarations: {summary['undeclared_source_surfaces']}",
|
||||
@@ -930,6 +1074,23 @@ def _render_markdown(inventory: dict[str, Any]) -> str:
|
||||
f"| `{repository}` | {count} |"
|
||||
for repository, count in sorted(help_by_repository.items())
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"## High-risk Contextual-help Debt",
|
||||
"",
|
||||
"Inferred or explicitly classified high-risk controls require an exact",
|
||||
"F1 context. `data-help-risk-reviewed=\"standard\"` records a reviewed",
|
||||
"false positive. The versioned baseline makes this queue non-regressing.",
|
||||
"",
|
||||
"| Repository | Missing exact contexts |",
|
||||
"| --- | ---: |",
|
||||
]
|
||||
)
|
||||
lines.extend(
|
||||
f"| `{repository}` | {count} |"
|
||||
for repository, count in sorted(high_risk_by_repository.items())
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
@@ -1005,6 +1166,29 @@ def endpoint_key(endpoint: dict[str, Any]) -> tuple[str, str, str]:
|
||||
)
|
||||
|
||||
|
||||
def _load_high_risk_help_baseline(path: Path) -> dict[str, Any]:
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except FileNotFoundError as exc:
|
||||
raise ValueError(
|
||||
f"High-risk contextual-help baseline does not exist: {path}"
|
||||
) from exc
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(
|
||||
f"High-risk contextual-help baseline is invalid JSON: {exc}"
|
||||
) from exc
|
||||
if not isinstance(payload, dict) or payload.get("schema_version") != 1:
|
||||
raise ValueError(
|
||||
"High-risk contextual-help baseline must use schema_version 1."
|
||||
)
|
||||
maximum = payload.get("maximum_missing_exact_help")
|
||||
if not isinstance(maximum, int) or isinstance(maximum, bool) or maximum < 0:
|
||||
raise ValueError(
|
||||
"High-risk contextual-help baseline maximum must be a non-negative integer."
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
def _load_endpoint_declarations(
|
||||
path: Path,
|
||||
) -> dict[tuple[str, str, str], dict[str, Any]]:
|
||||
|
||||
@@ -9,6 +9,7 @@ import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tomllib
|
||||
|
||||
|
||||
@@ -216,10 +217,17 @@ def _extras(value: str | None) -> list[str]:
|
||||
|
||||
|
||||
def _git(repository: Path, *arguments: str) -> str:
|
||||
release_root = str(Path(__file__).resolve().parent)
|
||||
if release_root not in sys.path:
|
||||
sys.path.insert(0, release_root)
|
||||
from govoplan_release.git_state import sanitized_git_environment, scoped_git_command
|
||||
|
||||
return subprocess.check_output(
|
||||
["git", "-C", str(repository), *arguments],
|
||||
scoped_git_command(repository, "-C", str(repository), *arguments),
|
||||
text=True,
|
||||
stderr=subprocess.DEVNULL,
|
||||
env=sanitized_git_environment(),
|
||||
timeout=30,
|
||||
).strip()
|
||||
|
||||
|
||||
|
||||
@@ -14,6 +14,8 @@ import re
|
||||
import stat
|
||||
import zipfile
|
||||
|
||||
from .registry_reference import registry_artifact_conflicts, registry_entry_source
|
||||
|
||||
|
||||
WHEEL_PAYLOAD_ALGORITHM = "govoplan-wheel-declared-payload-v1"
|
||||
INSTALLED_PAYLOAD_ALGORITHM = "govoplan-installed-record-payload-v1"
|
||||
@@ -185,10 +187,23 @@ def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
|
||||
modules = payload.get("modules")
|
||||
if isinstance(modules, list):
|
||||
entries.extend(modules)
|
||||
conflicts = registry_artifact_conflicts(entries)
|
||||
if conflicts:
|
||||
return conflicts
|
||||
package_by_repo: dict[str, tuple[str, str]] = {}
|
||||
registry_artifacts: dict[str, dict[str, object]] = {}
|
||||
for entry in entries:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
try:
|
||||
registry_source = registry_entry_source(entry)
|
||||
except ValueError as exc:
|
||||
return (str(exc),)
|
||||
if registry_source is not None:
|
||||
package = str(entry["python_package"])
|
||||
package_by_repo[registry_source.repository] = (package, registry_source.version)
|
||||
registry_artifacts[package] = entry["artifact_integrity"]["python"]
|
||||
continue
|
||||
python_ref = entry.get("python_ref")
|
||||
match = _PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
|
||||
package_name = entry.get("python_package")
|
||||
@@ -219,6 +234,14 @@ def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
|
||||
issues: list[str] = []
|
||||
if malformed_artifacts:
|
||||
issues.append("release.artifacts contains malformed or duplicate identities")
|
||||
for package, registry_artifact in registry_artifacts.items():
|
||||
artifact = artifacts_by_package.get(package)
|
||||
if artifact is None or (
|
||||
artifact.get("archive_sha256") != registry_artifact.get("sha256")
|
||||
or artifact.get("archive_size") != registry_artifact.get("size")
|
||||
or artifact.get("package_version") != registry_artifact["registry_identity"].rsplit("@", 1)[-1]
|
||||
):
|
||||
issues.append(f"registry artifact {package} has no matching inspected wheel byte identity")
|
||||
seen_repos: set[str] = set()
|
||||
for unit in selected_units:
|
||||
if not isinstance(unit, dict):
|
||||
|
||||
@@ -19,6 +19,7 @@ from typing import Iterator
|
||||
from govoplan_core.core.modules import ModuleManifest
|
||||
from govoplan_core.core.versioning import version_satisfies_range
|
||||
|
||||
from .git_state import sanitized_git_environment, scoped_git_command
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
@@ -255,20 +256,19 @@ def materialized_source_tree(root: Path, *, source_ref: str | None) -> Iterator[
|
||||
source_root = temporary / "source"
|
||||
source_root.mkdir()
|
||||
result = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(root),
|
||||
scoped_git_command(
|
||||
root, "-C", str(root),
|
||||
"archive",
|
||||
"--format=tar",
|
||||
f"--output={archive_path}",
|
||||
source_ref,
|
||||
],
|
||||
),
|
||||
check=False,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
timeout=30,
|
||||
env=sanitized_git_environment(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
detail = result.stderr.strip() or "Git archive failed"
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
"""Build a private full-profile candidate from exact verified registry bytes."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from functools import lru_cache
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path, PurePosixPath
|
||||
import re
|
||||
import runpy
|
||||
import stat
|
||||
import tarfile
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
from .artifact_identity import inspect_python_wheel, selected_artifact_identity_issues
|
||||
from .candidate_artifact import (
|
||||
ensure_private_candidate_root, harden_private_candidate_tree,
|
||||
validate_release_channel,
|
||||
)
|
||||
from .catalog import canonical_hash
|
||||
from .module_directory import write_module_directory
|
||||
from .selective_catalog import (
|
||||
authenticate_base_catalog_signatures, authenticate_base_keyring,
|
||||
configured_signer_public_keys, enforce_selected_version_alignment,
|
||||
next_sequence, parse_signing_key, read_bounded_json_source, signature,
|
||||
validate_catalog_object,
|
||||
)
|
||||
from .source_provenance import (
|
||||
registered_source_origin_issues, selected_source_provenance,
|
||||
source_tag_provenance_issues,
|
||||
)
|
||||
from .version_alignment import candidate_catalog_version_issues
|
||||
from .workspace import META_ROOT, resolve_workspace_root, website_root
|
||||
|
||||
|
||||
MAX_ARTIFACT_BYTES = 512 * 1024 * 1024
|
||||
|
||||
|
||||
@lru_cache(maxsize=3)
|
||||
def _tool(name: str) -> dict[str, Any]:
|
||||
# These are fixed, operator-controlled release modules, not caller paths.
|
||||
if name not in {
|
||||
"generate-release-package-set", "generate-release-catalog",
|
||||
"resolve-package-artifacts",
|
||||
}:
|
||||
raise ValueError("unknown registry release tool")
|
||||
return runpy.run_path(str(META_ROOT / "tools" / "release" / f"{name}.py"))
|
||||
|
||||
|
||||
def authenticate_full_rebuild_base(
|
||||
*, web_root: Path, channel: str, signer_public_keys: dict[str, str],
|
||||
) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
"""Authenticate the fixed website pair without reusing legacy entry data.
|
||||
|
||||
Only this full rebuild may migrate a legacy catalog without a keyring hash.
|
||||
In that case *every* active website key must exactly equal a configured
|
||||
signer; injecting an additional website key cannot extend trust. Selective
|
||||
candidates retain their stricter existing hash-pinned-base requirement.
|
||||
"""
|
||||
|
||||
root = web_root / "public" / "catalogs" / "v1"
|
||||
catalog = read_bounded_json_source(root / "channels" / f"{channel}.json", label="published base catalog")
|
||||
keyring = read_bounded_json_source(root / "keyring.json", label="published website keyring")
|
||||
if not isinstance(catalog, dict) or not isinstance(keyring, dict):
|
||||
raise ValueError("published website catalog/keyring must be objects")
|
||||
trusted_keys = authenticate_base_keyring(keyring)
|
||||
release = catalog.get("release")
|
||||
pinned = release.get("keyring_sha256") if isinstance(release, dict) else None
|
||||
if pinned is None:
|
||||
if trusted_keys != signer_public_keys or len(keyring["keys"]) != len(trusted_keys):
|
||||
raise ValueError("legacy full rebuild requires exactly the configured known website signers")
|
||||
elif pinned != canonical_hash(keyring):
|
||||
raise ValueError("published base catalog does not pin its exact website keyring")
|
||||
if any(trusted_keys.get(key) != value for key, value in signer_public_keys.items()):
|
||||
raise ValueError("full rebuild cannot introduce or replace a website signer")
|
||||
authenticate_base_catalog_signatures(
|
||||
catalog, base_trusted_keys=trusted_keys, configured_signers=signer_public_keys,
|
||||
)
|
||||
validation = validate_catalog_object(
|
||||
catalog, approved_channel=channel, signer_public_keys=signer_public_keys,
|
||||
)
|
||||
if validation.get("valid") is not True:
|
||||
raise ValueError(f"published base catalog failed validation: {validation.get('error')}")
|
||||
return catalog, keyring
|
||||
|
||||
|
||||
def build_full_registry_candidate(
|
||||
*, package_set_path: Path, package_lock_path: Path,
|
||||
wheelhouse: Path, webui_packages: Path, output_dir: Path,
|
||||
selected_repositories: tuple[str, ...], signing_keys: tuple[str, ...],
|
||||
workspace_root: Path | str | None = None, channel: str = "stable",
|
||||
source_remote: str = "origin", public_base_url: str = "https://govoplan.add-ideas.de",
|
||||
expires_days: int = 90, sequence: int | None = None,
|
||||
) -> dict[str, object]:
|
||||
channel = validate_release_channel(channel)
|
||||
if not isinstance(expires_days, int) or isinstance(expires_days, bool) or not 1 <= expires_days <= 365:
|
||||
raise ValueError("catalog expiry must be between 1 and 365 days")
|
||||
workspace = resolve_workspace_root(workspace_root)
|
||||
ensure_private_candidate_root(workspace)
|
||||
output = output_dir.expanduser().absolute()
|
||||
ensure_private_candidate_root(output.parent, create=True)
|
||||
if output.exists() or output.is_symlink():
|
||||
raise ValueError("full candidate output must not already exist")
|
||||
parsed_keys = tuple(parse_signing_key(value) for value in signing_keys)
|
||||
if not parsed_keys:
|
||||
raise ValueError("full candidate needs a configured signing key")
|
||||
signer_keys = configured_signer_public_keys(parsed_keys)
|
||||
base, keyring = authenticate_full_rebuild_base(
|
||||
web_root=website_root(workspace), channel=channel, signer_public_keys=signer_keys,
|
||||
)
|
||||
package_set = _hashed_json(package_set_path, "package_set_sha256")
|
||||
lock = _hashed_json(package_lock_path, "lock_sha256")
|
||||
generator = _tool("generate-release-catalog")
|
||||
version = package_set.get("release_version")
|
||||
if package_set.get("profile") != "full" or not isinstance(version, str):
|
||||
raise ValueError("full candidate requires the complete full-profile package set")
|
||||
expected = _tool("generate-release-package-set")["generate_package_set"](
|
||||
core_version=version, requirements=META_ROOT / "requirements-release.txt",
|
||||
workspace=workspace, profile="full",
|
||||
meta_package=META_ROOT / "packages/govoplan-meta/pyproject.toml",
|
||||
)
|
||||
if package_set != expected:
|
||||
raise ValueError("package set differs from exact Meta full pins or immutable tag metadata")
|
||||
generator["_validate_release_inputs"](package_set, lock, core_version=version)
|
||||
if lock.get("registries") != package_set.get("registries"):
|
||||
raise ValueError("artifact lock uses different package registries")
|
||||
versions = {row["repository"]: row["version"] for row in package_set["python"]}
|
||||
origin_failures = registered_source_origin_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
)
|
||||
if origin_failures:
|
||||
raise ValueError("Registered source origin gate failed: " + "; ".join(item.describe() for item in origin_failures))
|
||||
selected = set(selected_repositories)
|
||||
if not selected or len(selected) != len(selected_repositories) or not selected <= versions.keys():
|
||||
raise ValueError("selected repositories must be unique members of the full package set")
|
||||
selected_versions = {repo: versions[repo] for repo in sorted(selected)}
|
||||
enforce_selected_version_alignment(repo_versions=selected_versions, workspace=workspace)
|
||||
failures = source_tag_provenance_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
require_head_repos=selected,
|
||||
)
|
||||
if failures:
|
||||
raise ValueError("Full source provenance gate failed: " + "; ".join(item.describe() for item in failures))
|
||||
provenance = selected_source_provenance(repo_versions=versions, workspace=workspace)
|
||||
wheel_identities = verify_registry_artifacts(
|
||||
package_set=package_set, lock=lock, wheelhouse=wheelhouse,
|
||||
webui_packages=webui_packages,
|
||||
)
|
||||
generated_at = datetime.now(tz=UTC)
|
||||
resolved_sequence = sequence if sequence is not None else next_sequence(base, generated_at=generated_at)
|
||||
if isinstance(resolved_sequence, bool) or not isinstance(resolved_sequence, int) or resolved_sequence <= int(base.get("sequence") or 0):
|
||||
raise ValueError("full candidate sequence must advance the authenticated published channel")
|
||||
# Fresh tagged manifests, including unchanged tagged ancestors; no legacy
|
||||
# entry, registry hash, source URL or dependency contract is carried over.
|
||||
candidate = generator["_catalog_payload"](
|
||||
package_set=package_set, package_lock=lock, channel=channel,
|
||||
sequence=resolved_sequence, generated_at=generated_at,
|
||||
expires_at=generated_at + timedelta(days=expires_days), workspace=workspace,
|
||||
public_base_url=public_base_url.rstrip("/"),
|
||||
)
|
||||
for entry in [candidate["core_release"], *candidate["modules"]]:
|
||||
repo = entry["python_package"]
|
||||
entry["source"] = {
|
||||
"repository": repo, "tag": f"v{versions[repo]}",
|
||||
"commit": provenance[repo]["commit_sha"],
|
||||
"tag_object_sha": provenance[repo]["tag_object_sha"],
|
||||
"repository_url": f"https://git.add-ideas.de/GovOPlaN/{repo}",
|
||||
"revision_url": f"https://git.add-ideas.de/GovOPlaN/{repo}/commit/{provenance[repo]['commit_sha']}",
|
||||
}
|
||||
candidate["release"].update({
|
||||
"selected_units": [
|
||||
{"repo": repo, "version": versions[repo], "tag": f"v{versions[repo]}", **provenance[repo]}
|
||||
for repo in sorted(selected)
|
||||
],
|
||||
"keyring_sha256": canonical_hash(keyring),
|
||||
"artifacts": wheel_identities,
|
||||
"base_catalog_sha256": canonical_hash(base),
|
||||
})
|
||||
# Recheck the exact objects used by synthesis, including unchanged source
|
||||
# ancestors, before signing. No late tag movement can change this candidate.
|
||||
origin_failures = registered_source_origin_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
)
|
||||
if origin_failures:
|
||||
raise ValueError("Registered source origin changed during synthesis: " + "; ".join(item.describe() for item in origin_failures))
|
||||
failures = source_tag_provenance_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
require_head_repos=selected,
|
||||
expected_commits={repo: row["commit_sha"] for repo, row in provenance.items()},
|
||||
expected_tag_objects={repo: row["tag_object_sha"] for repo, row in provenance.items()},
|
||||
)
|
||||
if failures:
|
||||
raise ValueError("Full source provenance changed during synthesis: " + "; ".join(item.describe() for item in failures))
|
||||
failures = candidate_catalog_version_issues(candidate)
|
||||
identity_failures = selected_artifact_identity_issues(candidate)
|
||||
if failures or identity_failures:
|
||||
raise ValueError("full candidate identity validation failed: " + "; ".join(
|
||||
[item.message for item in failures] + list(identity_failures)
|
||||
))
|
||||
candidate["signatures"] = [signature(candidate, key_id=key, private_key=value) for key, value in parsed_keys]
|
||||
validation = validate_catalog_object(candidate, approved_channel=channel, signer_public_keys=signer_keys)
|
||||
if validation.get("valid") is not True:
|
||||
raise ValueError(f"signed full candidate failed validation: {validation.get('error')}")
|
||||
# Exclusive output creation preserves earlier reviewed candidates.
|
||||
output.mkdir(mode=0o700)
|
||||
(output / "channels").mkdir(mode=0o700)
|
||||
catalog_path = output / "channels" / f"{channel}.json"
|
||||
_write_private_json(catalog_path, candidate)
|
||||
_write_private_json(output / "keyring.json", keyring)
|
||||
write_module_directory(
|
||||
catalog_payload=candidate, keyring_payload=keyring, output_root=output,
|
||||
channel=channel, public_base_url=public_base_url,
|
||||
)
|
||||
result = {
|
||||
"status": "ready", "candidate_dir": str(output), "catalog_path": str(catalog_path),
|
||||
"channel": channel, "sequence": resolved_sequence,
|
||||
"package_count": len(package_set["python"]), "webui_count": len(package_set["webui"]),
|
||||
"selected_count": len(selected), "candidate_catalog_hash": canonical_hash(candidate),
|
||||
"candidate_keyring_hash": canonical_hash(keyring), "validation_valid": True,
|
||||
}
|
||||
_write_private_json(output / "summary.json", result)
|
||||
harden_private_candidate_tree(output)
|
||||
return result
|
||||
|
||||
|
||||
def _hashed_json(path: Path, field: str) -> dict[str, Any]:
|
||||
payload = read_bounded_json_source(path, label=field)
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError(f"{field} input must be an object")
|
||||
unsigned = dict(payload)
|
||||
expected = unsigned.pop(field, None)
|
||||
# Registry tools use their established ASCII-escaped canonical form.
|
||||
encoded = json.dumps(unsigned, sort_keys=True, separators=(",", ":")).encode()
|
||||
if expected != hashlib.sha256(encoded).hexdigest():
|
||||
raise ValueError(f"{field} does not match its contents")
|
||||
return payload
|
||||
|
||||
|
||||
def verify_registry_artifacts(
|
||||
*, package_set: dict[str, Any], lock: dict[str, Any],
|
||||
wheelhouse: Path, webui_packages: Path,
|
||||
) -> list[dict[str, object]]:
|
||||
"""Compare exact registry bytes, metadata and source bindings without installs."""
|
||||
|
||||
identities = []
|
||||
for group, root, suffix in (("python", wheelhouse, ".whl"), ("webui", webui_packages, ".tgz")):
|
||||
ensure_private_candidate_root(root)
|
||||
expected = {row["name"]: row for row in package_set[group]}
|
||||
rows = lock[group]
|
||||
if not isinstance(rows, list) or len(rows) != len(expected):
|
||||
raise ValueError(f"artifact lock has duplicate/missing {group} rows")
|
||||
filenames: set[str] = set()
|
||||
seen: set[str] = set()
|
||||
for row in rows:
|
||||
selected = expected.get(row.get("name")) if isinstance(row, dict) else None
|
||||
if selected is None or row["name"] in seen:
|
||||
raise ValueError(f"artifact lock has unexpected/duplicate {group} identities")
|
||||
seen.add(row["name"])
|
||||
for key in ("name", "version", "repository", "tag", "commit"):
|
||||
if row.get(key) != selected[key]:
|
||||
raise ValueError("artifact lock differs from selected source identity")
|
||||
filename = row.get("filename")
|
||||
if not isinstance(filename, str) or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+-]{0,254}", filename) is None or not filename.endswith(suffix) or filename in filenames:
|
||||
raise ValueError("artifact filename is invalid or duplicated")
|
||||
filenames.add(filename)
|
||||
path = root / filename
|
||||
sha256, sha512, size = _hash_artifact(path)
|
||||
if (sha256, size) != (row.get("sha256"), row.get("size")):
|
||||
raise ValueError(f"registry artifact bytes differ from locked identity: {filename}")
|
||||
if group == "python":
|
||||
expected_url = _tool("resolve-package-artifacts")["_python_artifact_url"](
|
||||
package_set["registries"]["python"], package=selected, filename=filename,
|
||||
)
|
||||
if row.get("url") != expected_url:
|
||||
raise ValueError("Python artifact URL differs from the selected registry")
|
||||
identity = inspect_python_wheel(path)
|
||||
if (identity.package_name, identity.package_version, identity.archive_sha256, identity.archive_size) != (row["name"], row["version"], sha256, size):
|
||||
raise ValueError("wheel metadata or bytes differ from the registry lock")
|
||||
identities.append(identity.catalog_payload())
|
||||
else:
|
||||
expected_url = (
|
||||
package_set["registries"]["npm"].rstrip("/") + "/"
|
||||
+ quote(row["name"], safe="") + "/-/"
|
||||
+ quote(row["version"], safe="") + "/"
|
||||
+ quote(row["name"].split("/", 1)[1] + "-" + row["version"] + ".tgz", safe="")
|
||||
)
|
||||
if row.get("url") != expected_url:
|
||||
raise ValueError("WebUI artifact URL differs from the selected registry")
|
||||
if row.get("integrity") != "sha512-" + base64.b64encode(sha512).decode("ascii"):
|
||||
raise ValueError("WebUI registry integrity differs from downloaded bytes")
|
||||
_inspect_npm_metadata(
|
||||
path, name=row["name"], version=row["version"],
|
||||
expected_identity=(sha256, sha512, size),
|
||||
)
|
||||
actual = set()
|
||||
for index, path in enumerate(root.iterdir()):
|
||||
if index >= 1000:
|
||||
raise ValueError("registry artifact directory exceeds its inspection bound")
|
||||
actual.add(path.name)
|
||||
if actual != filenames:
|
||||
raise ValueError(f"registry directory contains unexpected or missing {group} files")
|
||||
return sorted(identities, key=lambda row: str(row["package_name"]))
|
||||
|
||||
|
||||
def _hash_artifact(path: Path) -> tuple[str, bytes, int]:
|
||||
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
try:
|
||||
initial = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(initial.st_mode) or not 0 < initial.st_size <= MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("registry artifact must be a bounded regular file")
|
||||
digest, sri, total = hashlib.sha256(), hashlib.sha512(), 0
|
||||
while chunk := os.read(descriptor, 1024 * 1024):
|
||||
total += len(chunk)
|
||||
if total > MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("registry artifact exceeds its byte bound")
|
||||
digest.update(chunk)
|
||||
sri.update(chunk)
|
||||
final = os.fstat(descriptor)
|
||||
if (initial.st_ino, initial.st_size, initial.st_mtime_ns, initial.st_ctime_ns) != (final.st_ino, total, final.st_mtime_ns, final.st_ctime_ns):
|
||||
raise ValueError("registry artifact changed while being inspected")
|
||||
return digest.hexdigest(), sri.digest(), total
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _inspect_npm_metadata(
|
||||
path: Path, *, name: str, version: str,
|
||||
expected_identity: tuple[str, bytes, int] | None = None,
|
||||
) -> None:
|
||||
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
try:
|
||||
initial = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(initial.st_mode) or not 0 < initial.st_size <= MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("WebUI archive must be a bounded regular file")
|
||||
digest, sri, total = hashlib.sha256(), hashlib.sha512(), 0
|
||||
while chunk := os.read(descriptor, 1024 * 1024):
|
||||
total += len(chunk)
|
||||
if total > MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("WebUI archive exceeds its byte bound")
|
||||
digest.update(chunk)
|
||||
sri.update(chunk)
|
||||
if expected_identity is not None and (digest.hexdigest(), sri.digest(), total) != expected_identity:
|
||||
raise ValueError("WebUI archive changed before metadata inspection")
|
||||
os.lseek(descriptor, 0, os.SEEK_SET)
|
||||
with os.fdopen(os.dup(descriptor), "rb") as stream:
|
||||
_inspect_npm_stream(stream, name=name, version=version)
|
||||
final = os.fstat(descriptor)
|
||||
if (initial.st_ino, initial.st_size, initial.st_mtime_ns, initial.st_ctime_ns) != (final.st_ino, total, final.st_mtime_ns, final.st_ctime_ns):
|
||||
raise ValueError("WebUI archive changed during metadata inspection")
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _inspect_npm_stream(stream: Any, *, name: str, version: str) -> None:
|
||||
found = False
|
||||
total = 0
|
||||
with tarfile.open(fileobj=stream, mode="r|gz") as archive:
|
||||
for index, member in enumerate(archive):
|
||||
total += member.size
|
||||
parts = PurePosixPath(member.name).parts
|
||||
if index >= 10000 or total > 1024 * 1024 * 1024 or member.size > 64 * 1024 * 1024:
|
||||
raise ValueError("WebUI archive exceeds its inspection bound")
|
||||
if not parts or parts[0] != "package" or ".." in parts or not (member.isfile() or member.isdir()):
|
||||
raise ValueError("WebUI archive contains an unsafe member")
|
||||
if member.name == "package/package.json":
|
||||
if found or not member.isfile() or member.size > 1024 * 1024:
|
||||
raise ValueError("WebUI archive has duplicate or oversized metadata")
|
||||
stream = archive.extractfile(member)
|
||||
if stream is None:
|
||||
raise ValueError("WebUI archive metadata cannot be read")
|
||||
metadata = json.loads(stream.read(1024 * 1024 + 1))
|
||||
if not isinstance(metadata, dict) or (metadata.get("name"), metadata.get("version")) != (name, version):
|
||||
raise ValueError("WebUI metadata differs from the registry lock")
|
||||
found = True
|
||||
if not found:
|
||||
raise ValueError("WebUI archive metadata is missing")
|
||||
|
||||
|
||||
def _write_private_json(path: Path, payload: object) -> None:
|
||||
encoded = (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
|
||||
if len(encoded) > 16 * 1024 * 1024:
|
||||
raise ValueError("candidate JSON exceeds its byte bound")
|
||||
descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0), 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
@@ -11,7 +11,7 @@ import tomllib
|
||||
|
||||
from .contracts import parse_manifest_contract
|
||||
from .model import RepositorySnapshot, RepositorySpec, VersionSnapshot
|
||||
from .workspace import resolve_repo_path
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
def collect_repository_snapshot(
|
||||
@@ -98,6 +98,7 @@ def collect_repository_snapshot(
|
||||
def collect_versions(path: Path) -> VersionSnapshot:
|
||||
return VersionSnapshot(
|
||||
pyproject=read_pyproject_version(path),
|
||||
developer_meta=read_developer_meta_version(path),
|
||||
package=read_json_version(path / "package.json"),
|
||||
webui_package=read_json_version(path / "webui" / "package.json"),
|
||||
manifests=read_manifest_versions(path),
|
||||
@@ -105,6 +106,35 @@ def collect_versions(path: Path) -> VersionSnapshot:
|
||||
)
|
||||
|
||||
|
||||
def registered_developer_meta_path(path: Path) -> Path | None:
|
||||
"""Recognize only the catalog's explicit Meta support-repository identity.
|
||||
|
||||
This is metadata discovery, not authorization to access a remote or mutate
|
||||
a checkout. Tagging applies its separate registered source trust contract.
|
||||
"""
|
||||
for spec in load_repository_specs(include_website=False):
|
||||
if (
|
||||
spec.name == "govoplan"
|
||||
and spec.category == "system"
|
||||
and spec.subtype == "meta"
|
||||
and path.absolute() == resolve_repo_path(spec, path.parent).absolute()
|
||||
):
|
||||
return path / "packages" / "govoplan-meta" / "pyproject.toml"
|
||||
return None
|
||||
|
||||
|
||||
def read_developer_meta_version(path: Path) -> str | None:
|
||||
package = registered_developer_meta_path(path)
|
||||
if package is None or not package.is_file():
|
||||
return None
|
||||
with package.open("rb") as handle:
|
||||
project = tomllib.load(handle).get("project")
|
||||
if isinstance(project, dict) and project.get("name") == "govoplan":
|
||||
version = project.get("version")
|
||||
return version if isinstance(version, str) else None
|
||||
return None
|
||||
|
||||
|
||||
def read_pyproject_version(path: Path) -> str | None:
|
||||
pyproject = path / "pyproject.toml"
|
||||
if not pyproject.exists():
|
||||
@@ -197,6 +227,13 @@ def sanitized_git_environment(
|
||||
"""Keep only deliberate process/auth inputs and neutralize Git redirection."""
|
||||
|
||||
environment = os.environ if source is None else source
|
||||
address_family = environment.get("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY")
|
||||
if "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY" in environment and address_family not in (
|
||||
"any", "inet", "inet6",
|
||||
):
|
||||
raise ValueError(
|
||||
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6"
|
||||
)
|
||||
result = {
|
||||
key: environment[key]
|
||||
for key in (
|
||||
@@ -221,6 +258,10 @@ def sanitized_git_environment(
|
||||
"PATH": "/usr/bin:/bin",
|
||||
}
|
||||
)
|
||||
if address_family is not None:
|
||||
result["GIT_SSH_COMMAND"] += f" -o AddressFamily={address_family}"
|
||||
# Preserve only an explicit, validated choice across re-sanitization.
|
||||
result["GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY"] = address_family
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,380 @@
|
||||
"""Receipt-bound, out-of-run preparation of the real developer meta-package.
|
||||
|
||||
This deliberately does not commit, tag, publish, or update a running release
|
||||
console. The complete generated file is reviewed in a separate source checkout.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import stat
|
||||
import tempfile
|
||||
import tomllib
|
||||
|
||||
from .git_state import (
|
||||
collect_repository_snapshot,
|
||||
git,
|
||||
git_text,
|
||||
registered_developer_meta_path,
|
||||
)
|
||||
from .repository_tag import normalize_version, remote_tag_commit, run
|
||||
from .source_provenance import registered_source_origin_issues
|
||||
from .source_tag_batch import (
|
||||
_OBJECT,
|
||||
_owned_path,
|
||||
_source_filesystem,
|
||||
_tracked_worktree,
|
||||
_trusted_ancestry,
|
||||
)
|
||||
from .workspace import META_ROOT, load_repository_specs, resolve_repo_path
|
||||
|
||||
PACKAGE = "packages/govoplan-meta/pyproject.toml"
|
||||
MAX_INPUT_FILES = 128
|
||||
MAX_INPUT_BYTES = 2 * 1024 * 1024
|
||||
MAX_TOTAL_BYTES = 16 * 1024 * 1024
|
||||
GENERATOR = ".operator/generate-developer-meta-package.py"
|
||||
|
||||
|
||||
class MetaPreparationError(ValueError):
|
||||
"""Preparation is blocked, or an applied file needs explicit reconciliation."""
|
||||
|
||||
|
||||
class MetaPreparationAmbiguous(MetaPreparationError):
|
||||
"""The file effect may have happened and requires explicit reconciliation."""
|
||||
|
||||
|
||||
def preparation_command(*, workspace: Path, target_version: str) -> str:
|
||||
import shlex
|
||||
|
||||
return " ".join(
|
||||
shlex.quote(value)
|
||||
for value in (
|
||||
"python",
|
||||
str(META_ROOT / "tools/release/prepare-developer-meta-package.py"),
|
||||
"--workspace",
|
||||
str(workspace),
|
||||
"--target-version",
|
||||
target_version,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _read_input(path: Path) -> tuple[bytes, dict]:
|
||||
def identity(value):
|
||||
return (
|
||||
value.st_dev,
|
||||
value.st_ino,
|
||||
value.st_uid,
|
||||
value.st_gid,
|
||||
value.st_mode,
|
||||
value.st_size,
|
||||
value.st_mtime_ns,
|
||||
value.st_ctime_ns,
|
||||
)
|
||||
|
||||
before = path.lstat()
|
||||
if (
|
||||
not stat.S_ISREG(before.st_mode)
|
||||
or before.st_uid != os.geteuid()
|
||||
or before.st_mode & 0o022
|
||||
or not 0 < before.st_size <= MAX_INPUT_BYTES
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Preparation inputs must be owned, bounded regular files."
|
||||
)
|
||||
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
opened = os.fstat(source.fileno())
|
||||
if identity(opened) != identity(before):
|
||||
raise MetaPreparationError("Preparation input changed before reading.")
|
||||
payload = source.read(before.st_size + 1)
|
||||
if (
|
||||
identity(os.fstat(source.fileno())) != identity(opened)
|
||||
or len(payload) != before.st_size
|
||||
):
|
||||
raise MetaPreparationError("Preparation input changed while reading.")
|
||||
return payload, {
|
||||
"sha256": hashlib.sha256(payload).hexdigest(),
|
||||
"identity": [
|
||||
before.st_dev,
|
||||
before.st_ino,
|
||||
before.st_uid,
|
||||
before.st_gid,
|
||||
stat.S_IMODE(before.st_mode),
|
||||
before.st_size,
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _snapshot(*, repo_path: Path, target_version: str, output_dirty: bool = False):
|
||||
workspace = repo_path.parent
|
||||
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
|
||||
if registered_developer_meta_path(repo_path) != repo_path / PACKAGE:
|
||||
raise MetaPreparationError(
|
||||
"Only the registered Meta nested-package identity can be prepared."
|
||||
)
|
||||
if repo_path.resolve() == META_ROOT.resolve():
|
||||
raise MetaPreparationError(
|
||||
"Prepare a separate registered source checkout, never the running operator tooling."
|
||||
)
|
||||
paths = [repo_path / PACKAGE, repo_path / "requirements-release.txt"]
|
||||
for path in workspace.glob("govoplan-*/pyproject.toml"):
|
||||
paths.append(path)
|
||||
if len(paths) > MAX_INPUT_FILES:
|
||||
raise MetaPreparationError(
|
||||
"Developer composition exceeds its input-file bound."
|
||||
)
|
||||
if workspace / "govoplan-core/pyproject.toml" not in paths:
|
||||
raise MetaPreparationError(
|
||||
"Prepare and commit matching Core metadata before Meta preparation."
|
||||
)
|
||||
repositories = {"govoplan": repo_path}
|
||||
for path in paths[2:]:
|
||||
name = path.parent.name
|
||||
if (
|
||||
name not in specs
|
||||
or resolve_repo_path(specs[name], workspace) != path.parent
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Developer composition contains an unregistered package checkout."
|
||||
)
|
||||
repositories[name] = path.parent
|
||||
filesystems = {}
|
||||
for name, path in repositories.items():
|
||||
filesystems[name] = _source_filesystem(path=path, workspace=workspace)
|
||||
_tracked_worktree(path)
|
||||
issues = registered_source_origin_issues(
|
||||
repo_versions={name: target_version for name in repositories},
|
||||
workspace=workspace,
|
||||
remote="origin",
|
||||
)
|
||||
if issues:
|
||||
raise MetaPreparationError(
|
||||
"Preparation source origins do not match the registered repositories."
|
||||
)
|
||||
sources = {}
|
||||
for name, path in sorted(repositories.items()):
|
||||
snapshot = collect_repository_snapshot(
|
||||
specs[name],
|
||||
workspace_root=workspace,
|
||||
target_tag=None,
|
||||
online=False,
|
||||
)
|
||||
dirty_allowed = (
|
||||
output_dirty
|
||||
and name == "govoplan"
|
||||
and snapshot.dirty_entries == (f" M {PACKAGE}",)
|
||||
)
|
||||
if (
|
||||
snapshot.errors
|
||||
or not snapshot.has_head
|
||||
or snapshot.branch != "main"
|
||||
or snapshot.upstream != "origin/main"
|
||||
or snapshot.behind
|
||||
or (snapshot.dirty and not dirty_allowed)
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Preparation requires reviewed clean main sources tracking origin/main."
|
||||
)
|
||||
head = git_text(path, "rev-parse", "--verify", "HEAD")
|
||||
live = run(
|
||||
("git", "ls-remote", "--exit-code", "--heads", "origin", "refs/heads/main"),
|
||||
cwd=path,
|
||||
)
|
||||
lines = live.stdout.strip().splitlines()
|
||||
if live.returncode or len(lines) != 1:
|
||||
raise MetaPreparationError("Could not verify live preparation source main.")
|
||||
remote_main, separator, reference = lines[0].partition("\t")
|
||||
if (
|
||||
not _OBJECT.fullmatch(head)
|
||||
or not _OBJECT.fullmatch(remote_main)
|
||||
or not separator
|
||||
or reference != "refs/heads/main"
|
||||
or git(path, "merge-base", "--is-ancestor", remote_main, head).returncode
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Preparation source main diverged; fetch and review separately."
|
||||
)
|
||||
sources[name] = {
|
||||
"head": head,
|
||||
"remote_main": remote_main,
|
||||
"filesystem": filesystems[name],
|
||||
}
|
||||
tag = f"v{target_version}"
|
||||
published = remote_tag_commit(repo_path, remote="origin", tag=tag)
|
||||
if (
|
||||
published.error
|
||||
or published.tag_object
|
||||
or git_text(repo_path, "rev-parse", "--verify", f"refs/tags/{tag}")
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"An existing or unverifiable target Meta tag blocks source preparation."
|
||||
)
|
||||
inputs, payloads = {}, {}
|
||||
total = 0
|
||||
for path in sorted(paths):
|
||||
payload, identity = _read_input(path)
|
||||
total += len(payload)
|
||||
if total > MAX_TOTAL_BYTES:
|
||||
raise MetaPreparationError(
|
||||
"Developer composition exceeds its aggregate input bound."
|
||||
)
|
||||
relative = path.relative_to(workspace).as_posix()
|
||||
inputs[relative], payloads[relative] = identity, payload
|
||||
generator_path = META_ROOT / "tools/release/generate-developer-meta-package.py"
|
||||
_trusted_ancestry(generator_path.parent)
|
||||
_owned_path(META_ROOT, directory=True)
|
||||
generator, generator_identity = _read_input(generator_path)
|
||||
if total + len(generator) > MAX_TOTAL_BYTES:
|
||||
raise MetaPreparationError(
|
||||
"Developer composition exceeds its aggregate input bound."
|
||||
)
|
||||
payloads[GENERATOR] = generator
|
||||
current = tomllib.loads(payloads[f"govoplan/{PACKAGE}"].decode("utf-8")).get(
|
||||
"project", {}
|
||||
)
|
||||
core = tomllib.loads(payloads["govoplan-core/pyproject.toml"].decode("utf-8")).get(
|
||||
"project", {}
|
||||
)
|
||||
if (
|
||||
not isinstance(current, dict)
|
||||
or current.get("name") != "govoplan"
|
||||
or not isinstance(current.get("version"), str)
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Nested developer-package identity and version must be exact."
|
||||
)
|
||||
if (
|
||||
not isinstance(core, dict)
|
||||
or core.get("name") != "govoplan-core"
|
||||
or core.get("version") != target_version
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Prepare and commit Core at the requested target version before Meta preparation."
|
||||
)
|
||||
from .version_alignment import repository_version_issues
|
||||
|
||||
if repository_version_issues(
|
||||
workspace / "govoplan-core", expected_version=target_version
|
||||
):
|
||||
raise MetaPreparationError(
|
||||
"Prepare aligned Core version metadata before Meta preparation."
|
||||
)
|
||||
receipt = {
|
||||
"kind": "developer_meta_preparation_v1",
|
||||
"workspace": str(workspace),
|
||||
"target_version": target_version,
|
||||
"sources": sources,
|
||||
"inputs": inputs,
|
||||
"operator_generator": generator_identity,
|
||||
}
|
||||
return receipt, payloads
|
||||
|
||||
|
||||
def _render(payloads: dict[str, bytes]) -> bytes:
|
||||
# The trusted operator generator sees only the frozen bounded data snapshot.
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-meta-render-") as temporary:
|
||||
workspace = Path(temporary)
|
||||
for relative, payload in payloads.items():
|
||||
path = workspace / relative
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(payload)
|
||||
render = runpy.run_path(str(workspace / GENERATOR))["render"]
|
||||
result = render(
|
||||
workspace=workspace,
|
||||
requirements=workspace / "govoplan/requirements-release.txt",
|
||||
).encode("utf-8")
|
||||
if len(result) > MAX_INPUT_BYTES:
|
||||
raise MetaPreparationError(
|
||||
"Generated developer package exceeds its output bound."
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def preview_meta_mutation(*, repo_path: Path, target_version: str):
|
||||
version = normalize_version(target_version)
|
||||
if not version:
|
||||
raise MetaPreparationError("A valid target release version is required.")
|
||||
receipt, payloads = _snapshot(repo_path=repo_path, target_version=version)
|
||||
after = _render(payloads)
|
||||
observed, _ = _snapshot(repo_path=repo_path, target_version=version)
|
||||
if observed != receipt:
|
||||
raise MetaPreparationError(
|
||||
"Developer preparation inputs changed during preview."
|
||||
)
|
||||
receipt["output_sha256"] = hashlib.sha256(after).hexdigest()
|
||||
return receipt, payloads[f"govoplan/{PACKAGE}"], after
|
||||
|
||||
|
||||
def prepare_developer_meta_package(
|
||||
*,
|
||||
repo_path: Path,
|
||||
target_version: str,
|
||||
apply: bool = False,
|
||||
expected_receipt=None,
|
||||
confirm_out_of_run: bool = False,
|
||||
) -> dict:
|
||||
"""Preview or explicitly apply one full generated file; never commit/publish."""
|
||||
try:
|
||||
receipt, before, after = preview_meta_mutation(
|
||||
repo_path=repo_path, target_version=target_version
|
||||
)
|
||||
result = {
|
||||
"status": "planned" if before != after else "noop",
|
||||
"path": PACKAGE,
|
||||
"receipt": receipt,
|
||||
"after_sha256": hashlib.sha256(after).hexdigest(),
|
||||
"changed": before != after,
|
||||
}
|
||||
if not apply:
|
||||
return result
|
||||
if not confirm_out_of_run:
|
||||
raise MetaPreparationError(
|
||||
"Explicitly confirm that no durable run is active for this source workspace."
|
||||
)
|
||||
if receipt != expected_receipt:
|
||||
raise MetaPreparationError(
|
||||
"Preparation inputs changed since the reviewed preview; create a fresh preview."
|
||||
)
|
||||
if before == after:
|
||||
return result
|
||||
from .version_metadata import _atomic_write
|
||||
|
||||
source_receipt = {
|
||||
key: value for key, value in receipt.items() if key != "output_sha256"
|
||||
}
|
||||
observed, _ = _snapshot(
|
||||
repo_path=repo_path, target_version=receipt["target_version"]
|
||||
)
|
||||
if observed != source_receipt:
|
||||
raise MetaPreparationError(
|
||||
"Preparation inputs changed before the file effect; create a fresh preview."
|
||||
)
|
||||
try:
|
||||
_atomic_write(repo_path / PACKAGE, after)
|
||||
observed, payloads = _snapshot(
|
||||
repo_path=repo_path,
|
||||
target_version=receipt["target_version"],
|
||||
output_dirty=True,
|
||||
)
|
||||
comparison = copy.deepcopy(observed)
|
||||
output = f"govoplan/{PACKAGE}"
|
||||
comparison["inputs"][output] = receipt["inputs"][output]
|
||||
if comparison != source_receipt or payloads[output] != after:
|
||||
raise MetaPreparationError("Preparation inputs changed after writing.")
|
||||
except Exception as exc:
|
||||
raise MetaPreparationAmbiguous(
|
||||
"Generated file may have been written but its write/post-check failed; "
|
||||
"review the delta and reconcile manually."
|
||||
) from exc
|
||||
return {**result, "status": "prepared", "after_receipt": observed}
|
||||
except MetaPreparationError:
|
||||
raise
|
||||
except (OSError, UnicodeError, ValueError, KeyError, TypeError) as exc:
|
||||
raise MetaPreparationError(
|
||||
f"Developer preparation failed closed ({type(exc).__name__})."
|
||||
) from exc
|
||||
@@ -23,6 +23,7 @@ class RepositorySpec:
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class VersionSnapshot:
|
||||
pyproject: str | None = None
|
||||
developer_meta: str | None = None
|
||||
package: str | None = None
|
||||
webui_package: str | None = None
|
||||
manifests: tuple[str, ...] = ()
|
||||
@@ -32,6 +33,7 @@ class VersionSnapshot:
|
||||
def primary(self) -> str | None:
|
||||
return (
|
||||
self.pyproject
|
||||
or self.developer_meta
|
||||
or self.package
|
||||
or self.webui_package
|
||||
or (self.manifests[0] if self.manifests else None)
|
||||
|
||||
@@ -27,7 +27,10 @@ from .candidate_artifact import validate_release_channel
|
||||
from .model import CatalogPublishResult, CatalogPublishStep
|
||||
from .module_directory import module_directory_payloads
|
||||
from .selective_catalog import read_bounded_json_source, trusted_keys_from_keyring
|
||||
from .source_provenance import catalog_source_selection, source_tag_provenance_issues
|
||||
from .source_provenance import (
|
||||
catalog_source_selection, registered_source_origin_issues,
|
||||
source_tag_provenance_issues,
|
||||
)
|
||||
from .version_alignment import candidate_catalog_version_issues
|
||||
from .workspace import (
|
||||
DEFAULT_WORKSPACE_ROOT,
|
||||
@@ -176,6 +179,22 @@ def publish_catalog_candidate(
|
||||
for issue in version_issues
|
||||
)
|
||||
source_selection = catalog_source_selection(candidate_payload)
|
||||
entries = [candidate_payload.get("core_release")]
|
||||
if isinstance(candidate_payload.get("modules"), list):
|
||||
entries.extend(candidate_payload["modules"])
|
||||
if any(
|
||||
isinstance(entry, dict)
|
||||
and isinstance(entry.get("python_ref"), str)
|
||||
and " @ https://" in entry["python_ref"]
|
||||
for entry in entries
|
||||
):
|
||||
blockers.extend(
|
||||
f"registered source origin: {issue.describe()}"
|
||||
for issue in registered_source_origin_issues(
|
||||
repo_versions=source_selection.all_versions,
|
||||
workspace=workspace, remote=source_remote,
|
||||
)
|
||||
)
|
||||
blockers.extend(
|
||||
f"source provenance: {issue.describe()}"
|
||||
for issue in source_selection.issues
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
"""Strict source identities for immutable, registry-backed catalog entries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
import re
|
||||
from urllib.parse import unquote, urlsplit
|
||||
|
||||
|
||||
_SHA256 = re.compile(r"[0-9a-f]{64}\Z")
|
||||
_COMMIT = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?\Z")
|
||||
_REPO = re.compile(r"govoplan-[a-z0-9-]+\Z")
|
||||
_VERSION = re.compile(r"\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?\Z")
|
||||
_PYTHON = re.compile(
|
||||
r"(?P<name>govoplan-[a-z0-9-]+)(?:\[[a-z0-9_,.-]+\])? @ "
|
||||
r"(?P<url>https://\S+)#sha256=(?P<digest>[0-9a-f]{64})\Z"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegistrySource:
|
||||
repository: str
|
||||
version: str
|
||||
commit: str
|
||||
tag_object: str
|
||||
|
||||
|
||||
def registry_artifact_conflicts(entries: list[object]) -> tuple[str, ...]:
|
||||
"""Allow repeated module projections only when package artifacts agree."""
|
||||
observed: dict[tuple[str, str], str] = {}
|
||||
issues = []
|
||||
for entry in entries:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
try:
|
||||
source = registry_entry_source(entry)
|
||||
except ValueError as exc:
|
||||
issues.append(str(exc))
|
||||
continue
|
||||
if source is None:
|
||||
continue
|
||||
for kind, identity in entry["artifact_integrity"].items():
|
||||
if kind not in {"python", "webui"}:
|
||||
continue
|
||||
key = (source.repository, kind)
|
||||
encoded = json.dumps(identity, sort_keys=True, separators=(",", ":"))
|
||||
if observed.setdefault(key, encoded) != encoded:
|
||||
issues.append(f"conflicting {kind} registry artifacts for {source.repository}")
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
def registry_entry_source(entry: dict[str, object]) -> RegistrySource | None:
|
||||
"""Admit registry refs only with a complete matching artifact/source binding.
|
||||
|
||||
Git-backed catalogs keep their existing validation path. An HTTPS Python
|
||||
requirement cannot masquerade as a source-only/non-Python entry when its
|
||||
registry provenance is missing or inconsistent.
|
||||
"""
|
||||
|
||||
ref = entry.get("python_ref")
|
||||
if not isinstance(ref, str) or " @ https://" not in ref:
|
||||
return None
|
||||
match = _PYTHON.fullmatch(ref)
|
||||
source = entry.get("source")
|
||||
version = entry.get("version")
|
||||
package = entry.get("python_package")
|
||||
integrity = entry.get("artifact_integrity")
|
||||
if (
|
||||
match is None
|
||||
or not isinstance(source, dict)
|
||||
or not isinstance(integrity, dict)
|
||||
or not isinstance(version, str)
|
||||
or _VERSION.fullmatch(version) is None
|
||||
or package != match.group("name")
|
||||
):
|
||||
raise ValueError("registry entry has no complete package/source identity")
|
||||
repo = source.get("repository")
|
||||
commit = source.get("commit")
|
||||
tag_object = source.get("tag_object_sha")
|
||||
if (
|
||||
not isinstance(repo, str)
|
||||
or _REPO.fullmatch(repo) is None
|
||||
or repo != package
|
||||
or source.get("tag") != f"v{version}"
|
||||
or not isinstance(commit, str)
|
||||
or _COMMIT.fullmatch(commit) is None
|
||||
or not isinstance(tag_object, str)
|
||||
or _COMMIT.fullmatch(tag_object) is None
|
||||
):
|
||||
raise ValueError("registry entry has invalid immutable tag provenance")
|
||||
python = _artifact(
|
||||
integrity.get("python"), ref=ref, package=package, version=version,
|
||||
commit=commit,
|
||||
)
|
||||
if python["url"] != match.group("url") or python["sha256"] != match.group("digest"):
|
||||
raise ValueError("registry Python ref differs from its artifact identity")
|
||||
webui_package = entry.get("webui_package")
|
||||
webui_ref = entry.get("webui_ref")
|
||||
if bool(webui_package) != bool(webui_ref):
|
||||
raise ValueError("registry WebUI package and ref must be declared together")
|
||||
if webui_package:
|
||||
if not isinstance(webui_package, str) or re.fullmatch(
|
||||
r"@govoplan/[a-z0-9-]+-webui", webui_package
|
||||
) is None or not isinstance(webui_ref, str):
|
||||
raise ValueError("registry WebUI package identity is malformed")
|
||||
if webui_package != f"@govoplan/{repo.removeprefix('govoplan-')}-webui":
|
||||
raise ValueError("registry WebUI package belongs to another source repository")
|
||||
webui = _artifact(
|
||||
integrity.get("webui"), ref=webui_ref, package=webui_package,
|
||||
version=version, commit=commit,
|
||||
)
|
||||
if webui_ref != webui["url"]:
|
||||
raise ValueError("registry WebUI ref differs from its artifact identity")
|
||||
sri = webui.get("integrity")
|
||||
try:
|
||||
valid_sri = isinstance(sri, str) and sri.startswith("sha512-") and len(
|
||||
base64.b64decode(sri[7:], validate=True)
|
||||
) == 64
|
||||
except ValueError:
|
||||
valid_sri = False
|
||||
if not valid_sri:
|
||||
raise ValueError("registry WebUI artifact needs a SHA-512 integrity identity")
|
||||
elif "webui" in integrity:
|
||||
raise ValueError("registry entry carries an unexpected WebUI artifact")
|
||||
return RegistrySource(repo, version, commit, tag_object)
|
||||
|
||||
|
||||
def _artifact(
|
||||
value: object, *, ref: str, package: str, version: str, commit: str,
|
||||
) -> dict[str, object]:
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("registry entry is missing artifact integrity")
|
||||
url = value.get("url")
|
||||
filename = value.get("filename")
|
||||
digest = value.get("sha256")
|
||||
size = value.get("size")
|
||||
parsed = urlsplit(url) if isinstance(url, str) else None
|
||||
url_filename = unquote(parsed.path.rsplit("/", 1)[-1]) if parsed else ""
|
||||
decoded_parts = unquote(parsed.path).split("/") if parsed else []
|
||||
expected_filenames = {url_filename}
|
||||
expected_path = [package, version, url_filename]
|
||||
if package.startswith("@govoplan/"):
|
||||
# npm pack includes the scope in its local filename; the registry's
|
||||
# immutable download URL uses the unscoped package basename.
|
||||
expected_filenames = {
|
||||
f"govoplan-{package.split('/', 1)[1]}-{version}.tgz",
|
||||
} if url_filename == f"{package.split('/', 1)[1]}-{version}.tgz" else set()
|
||||
expected_path = [*package.split("/"), "-", version, url_filename]
|
||||
if (
|
||||
parsed is None
|
||||
or parsed.scheme != "https"
|
||||
or not parsed.netloc
|
||||
or parsed.username is not None
|
||||
or parsed.password is not None
|
||||
or parsed.fragment
|
||||
or parsed.query
|
||||
or any(part in {".", ".."} for part in unquote(parsed.path).split("/"))
|
||||
or "%" in unquote(parsed.path)
|
||||
or "\\" in unquote(parsed.path)
|
||||
or any(ord(character) < 32 for character in url)
|
||||
or decoded_parts[-len(expected_path):] != expected_path
|
||||
or not isinstance(filename, str)
|
||||
or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+-]{0,254}", filename) is None
|
||||
or filename not in expected_filenames
|
||||
or not isinstance(digest, str)
|
||||
or _SHA256.fullmatch(digest) is None
|
||||
or not isinstance(size, int)
|
||||
or isinstance(size, bool)
|
||||
or not 0 < size <= 512 * 1024 * 1024
|
||||
or value.get("ref") != ref
|
||||
or value.get("registry_identity") != f"{package}@{version}"
|
||||
or value.get("git_ref") != f"v{version}"
|
||||
or value.get("source_commit") != commit
|
||||
):
|
||||
raise ValueError("registry artifact ref, version, bytes, or source binding is inconsistent")
|
||||
return value
|
||||
@@ -34,11 +34,31 @@ def tag_repositories(
|
||||
apply: bool = False, # noqa: A002 - mirrors API field.
|
||||
push: bool = False,
|
||||
) -> dict[str, object]:
|
||||
"""Create annotated tags and optionally publish branch and tag atomically.
|
||||
from .source_tag_batch import tag_source_batch
|
||||
|
||||
return tag_source_batch(
|
||||
repos=repos, repo_versions=repo_versions, workspace_root=workspace_root,
|
||||
remote=remote, message=message, apply=apply, push=push,
|
||||
)
|
||||
|
||||
|
||||
def _preview_repositories(
|
||||
*,
|
||||
repos: tuple[str, ...],
|
||||
repo_versions: dict[str, str],
|
||||
workspace_root: Path | str | None = None,
|
||||
remote: str = "origin",
|
||||
message: str | None = None,
|
||||
push: bool = False,
|
||||
) -> dict[str, object]:
|
||||
"""Read-only shared manifest/version/composition/tag preflight.
|
||||
|
||||
A release tag is only created for a clean, aligned, non-behind worktree.
|
||||
Both local and remote tags are resolved to commits before mutation so an
|
||||
existing immutable tag can never be moved by this operation.
|
||||
Module-only local candidate tags precede Core's release-lock regeneration;
|
||||
their cross-Core composition gate applies before publication, not creation.
|
||||
Core candidate tags still require a complete aligned release bundle.
|
||||
"""
|
||||
|
||||
workspace = resolve_workspace_root(workspace_root)
|
||||
@@ -47,50 +67,21 @@ def tag_repositories(
|
||||
selected = tuple(dict.fromkeys(repos))
|
||||
results: list[dict[str, object]] = []
|
||||
bundle_issues_by_repo: dict[str, list[str]] = {}
|
||||
for issue in selected_release_webui_bundle_issues(
|
||||
repo_versions={repo: repo_versions.get(repo, "") for repo in selected},
|
||||
workspace=workspace,
|
||||
):
|
||||
bundle_issues_by_repo.setdefault(issue.repo, []).append(
|
||||
f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
|
||||
)
|
||||
|
||||
if apply:
|
||||
preflight = tag_repositories(
|
||||
repos=selected,
|
||||
repo_versions=repo_versions,
|
||||
workspace_root=workspace,
|
||||
remote=remote,
|
||||
message=message,
|
||||
apply=False,
|
||||
push=push,
|
||||
)
|
||||
preflight_rows = preflight.get("repositories")
|
||||
if isinstance(preflight_rows, list) and any(
|
||||
isinstance(item, dict) and item.get("status") in {"blocked", "failed"}
|
||||
for item in preflight_rows
|
||||
# Core's final lock is generated from reviewed local module tags. Requiring
|
||||
# that lock before those tags exist makes the documented sequence circular.
|
||||
# This is only a local module staging exception: Core-selected batches and
|
||||
# every publication still run the cross-repository gate, and each selected
|
||||
# repository's own version/lock checks below are always enforced.
|
||||
if push or "govoplan-core" in selected:
|
||||
for issue in selected_release_webui_bundle_issues(
|
||||
repo_versions={repo: repo_versions.get(repo, "") for repo in selected},
|
||||
workspace=workspace,
|
||||
):
|
||||
blocked_rows = []
|
||||
for item in preflight_rows:
|
||||
if not isinstance(item, dict) or item.get("status") in {"blocked", "failed"}:
|
||||
blocked_rows.append(item)
|
||||
continue
|
||||
blocked_rows.append(
|
||||
{
|
||||
**item,
|
||||
"status": "skipped",
|
||||
"detail": "preflight passed, but no release tag was changed because another selected repository is blocked",
|
||||
}
|
||||
)
|
||||
return {
|
||||
"status": "blocked",
|
||||
"apply": True,
|
||||
"push": push,
|
||||
"remote": remote,
|
||||
"detail": "batch preflight failed; no selected repository was mutated",
|
||||
"repositories": blocked_rows,
|
||||
}
|
||||
elif selected:
|
||||
bundle_issues_by_repo.setdefault(issue.repo, []).append(
|
||||
f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
|
||||
)
|
||||
|
||||
if selected:
|
||||
manifest_gate_issue = manifest_shape_gate_issue(workspace)
|
||||
if manifest_gate_issue:
|
||||
return {
|
||||
@@ -246,130 +237,17 @@ def tag_repositories(
|
||||
"remote_tag_object": remote_result.tag_object,
|
||||
}
|
||||
)
|
||||
if not apply:
|
||||
detail = preview_detail(tag=tag, local_commit=local_commit, remote_commit=remote_result.commit, push=push)
|
||||
status = "noop" if remote_result.commit or (local_commit and not push) else "planned"
|
||||
results.append({**row, "status": status, "detail": detail})
|
||||
continue
|
||||
|
||||
if remote_result.commit:
|
||||
if not local_commit:
|
||||
fetch_result = run(("git", "fetch", remote, f"refs/tags/{tag}:refs/tags/{tag}"), cwd=path)
|
||||
if fetch_result.returncode != 0:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"remote tag {tag} exists at HEAD but could not be fetched locally",
|
||||
"returncode": fetch_result.returncode,
|
||||
"stdout": compact_output(fetch_result.stdout),
|
||||
"stderr": compact_output(fetch_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "published",
|
||||
"detail": f"immutable tag {tag} is already published at HEAD",
|
||||
"after_local_tag_commit": head_commit,
|
||||
"after_remote_tag_commit": head_commit,
|
||||
}
|
||||
)
|
||||
continue
|
||||
|
||||
created = False
|
||||
if not local_commit:
|
||||
create_result = run(create_command, cwd=path)
|
||||
if create_result.returncode != 0:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"could not create annotated tag {tag}",
|
||||
"returncode": create_result.returncode,
|
||||
"stdout": compact_output(create_result.stdout),
|
||||
"stderr": compact_output(create_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
created = True
|
||||
|
||||
if not push:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "tagged" if created else "noop",
|
||||
"detail": f"created annotated tag {tag} at HEAD" if created else f"annotated tag {tag} already exists at HEAD",
|
||||
"after_local_tag_commit": head_commit,
|
||||
}
|
||||
)
|
||||
continue
|
||||
|
||||
publish_result = run(publish_command, cwd=path)
|
||||
if publish_result.returncode != 0:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"created local tag {tag}, but atomic branch and tag publication failed" if created else f"atomic branch and tag publication failed for {tag}",
|
||||
"returncode": publish_result.returncode,
|
||||
"after_local_tag_commit": head_commit,
|
||||
"stdout": compact_output(publish_result.stdout),
|
||||
"stderr": compact_output(publish_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
after_local_object = git_text(path, "rev-parse", "--verify", f"refs/tags/{tag}")
|
||||
after_remote = remote_tag_commit(path, remote=remote, tag=tag)
|
||||
if (
|
||||
after_remote.error
|
||||
or not after_remote.annotated
|
||||
or after_remote.commit != head_commit
|
||||
or after_remote.tag_object != after_local_object
|
||||
):
|
||||
verification_detail = after_remote.error or "remote tag did not resolve to the published annotated tag object at HEAD"
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed",
|
||||
"detail": f"Git push returned success, but the remote release-tag postcondition failed: {verification_detail}",
|
||||
"returncode": publish_result.returncode,
|
||||
"after_local_tag_commit": head_commit,
|
||||
"after_local_tag_object": after_local_object,
|
||||
"after_remote_tag_commit": after_remote.commit,
|
||||
"after_remote_tag_object": after_remote.tag_object,
|
||||
"stdout": compact_output(publish_result.stdout),
|
||||
"stderr": compact_output(publish_result.stderr),
|
||||
}
|
||||
)
|
||||
continue
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "published",
|
||||
"detail": f"published branch {snapshot.branch} and immutable tag {tag} atomically to {remote}",
|
||||
"returncode": publish_result.returncode,
|
||||
"after_local_tag_commit": head_commit,
|
||||
"after_remote_tag_commit": head_commit,
|
||||
"after_local_tag_object": after_local_object,
|
||||
"after_remote_tag_object": after_remote.tag_object,
|
||||
"stdout": compact_output(publish_result.stdout),
|
||||
"stderr": compact_output(publish_result.stderr),
|
||||
}
|
||||
)
|
||||
detail = preview_detail(tag=tag, local_commit=local_commit, remote_commit=remote_result.commit, push=push)
|
||||
row_status = "noop" if remote_result.commit or (local_commit and not push) else "planned"
|
||||
results.append({**row, "status": row_status, "detail": detail})
|
||||
|
||||
if any(item["status"] in {"blocked", "failed"} for item in results):
|
||||
status = "blocked" if not apply else "partial"
|
||||
elif any(item["status"] == "published" for item in results):
|
||||
status = "published"
|
||||
elif any(item["status"] == "tagged" for item in results):
|
||||
status = "tagged"
|
||||
result_status = "blocked"
|
||||
elif any(item["status"] == "planned" for item in results):
|
||||
status = "planned"
|
||||
result_status = "planned"
|
||||
else:
|
||||
status = "noop"
|
||||
return {"status": status, "apply": apply, "push": push, "remote": remote, "repositories": results}
|
||||
result_status = "noop"
|
||||
return {"status": result_status, "apply": False, "push": push, "remote": remote, "repositories": results}
|
||||
|
||||
|
||||
def normalize_version(value: str | None) -> str:
|
||||
|
||||
@@ -8,6 +8,7 @@ from pathlib import Path
|
||||
import shlex
|
||||
|
||||
from .contracts import validate_contracts
|
||||
from .git_state import registered_developer_meta_path, read_pyproject_version
|
||||
from .model import (
|
||||
CompatibilityIssue,
|
||||
InterfaceProviderSnapshot,
|
||||
@@ -111,6 +112,39 @@ def apply_repository_version_gate(
|
||||
version_update_supported_by_repo: dict[str, bool] = {}
|
||||
deferred_core_lock_repos: set[str] = set()
|
||||
for unit in units:
|
||||
if registered_developer_meta_path(workspace / unit.repo) is not None:
|
||||
from .meta_preparation import preparation_command
|
||||
|
||||
try:
|
||||
core_version = read_pyproject_version(workspace / "govoplan-core")
|
||||
except (OSError, ValueError, TypeError):
|
||||
core_version = None
|
||||
core_ready = core_version == unit.target_version
|
||||
issues_by_repo.setdefault(unit.repo, []).append(
|
||||
ReleaseGateFinding(
|
||||
code="developer_meta_out_of_run" if core_ready else "developer_meta_core_preparation_required",
|
||||
severity="blocker",
|
||||
message=(
|
||||
"Meta is an out-of-run support release, not a self-updating durable executor."
|
||||
if core_ready else
|
||||
"Prepare and commit Core at the requested target before regenerating Meta."
|
||||
),
|
||||
remediation=(
|
||||
"Complete Core and module preparation first. Stop active durable runs for this workspace. "
|
||||
"In a separate trusted source checkout, preview "
|
||||
+ preparation_command(workspace=workspace, target_version=unit.target_version)
|
||||
+ ". Review its receipt; apply with --receipt <preview.json> --apply --confirm-out-of-run. "
|
||||
"Review and commit the whole generated package, publish the matching Core release first, "
|
||||
"then use guarded Meta source tagging/publication and create a fresh durable run."
|
||||
),
|
||||
repo=unit.repo, source="developer meta-package preparation",
|
||||
expected=unit.target_version, actual=core_version or "missing Core version",
|
||||
)
|
||||
)
|
||||
version_update_supported_by_repo[unit.repo] = False
|
||||
# Its complete canonical composition remains a publication gate;
|
||||
# this plan must not claim a generic in-run mutation/commit path.
|
||||
continue
|
||||
version_update_supported = unit.current_version == unit.target_version
|
||||
if unit.current_version and unit.current_version != unit.target_version:
|
||||
try:
|
||||
@@ -437,6 +471,7 @@ def build_unit(
|
||||
value
|
||||
for value in (
|
||||
repo.versions.pyproject,
|
||||
repo.versions.developer_meta,
|
||||
repo.versions.package,
|
||||
repo.versions.webui_package,
|
||||
*repo.versions.manifests,
|
||||
@@ -572,7 +607,7 @@ def repository_capabilities(
|
||||
def dependency_ordered_units(
|
||||
units: tuple[ReleasePlanUnit, ...],
|
||||
) -> tuple[ReleasePlanUnit, ...]:
|
||||
"""Order module providers before consumers while keeping Core last."""
|
||||
"""Order modules before Core, followed by the out-of-run Meta support unit."""
|
||||
|
||||
by_repo = {unit.repo: unit for unit in units}
|
||||
providers: dict[str, set[str]] = {}
|
||||
@@ -592,8 +627,10 @@ def dependency_ordered_units(
|
||||
)
|
||||
if "govoplan-core" in dependencies:
|
||||
dependencies["govoplan-core"].update(
|
||||
repo for repo in by_repo if repo != "govoplan-core"
|
||||
repo for repo in by_repo if repo not in {"govoplan-core", "govoplan"}
|
||||
)
|
||||
if "govoplan" in dependencies:
|
||||
dependencies["govoplan"].update(repo for repo in by_repo if repo != "govoplan")
|
||||
|
||||
ordered: list[ReleasePlanUnit] = []
|
||||
remaining = set(by_repo)
|
||||
@@ -690,6 +727,8 @@ def dry_run_steps(
|
||||
*, units: tuple[ReleasePlanUnit, ...], dashboard: ReleaseDashboard, channel: str
|
||||
) -> tuple[ReleasePlanStep, ...]:
|
||||
steps: list[ReleasePlanStep] = []
|
||||
meta_units = tuple(unit for unit in units if unit.repo == "govoplan")
|
||||
units = tuple(unit for unit in units if unit.repo != "govoplan")
|
||||
snapshots = {repo.spec.name: repo for repo in dashboard.repositories}
|
||||
core_unit = next((unit for unit in units if unit.repo == "govoplan-core"), None)
|
||||
non_core_units = tuple(unit for unit in units if unit.repo != "govoplan-core")
|
||||
@@ -991,6 +1030,33 @@ def dry_run_steps(
|
||||
status="planned",
|
||||
)
|
||||
)
|
||||
for unit in meta_units:
|
||||
from .meta_preparation import preparation_command
|
||||
|
||||
steps.extend((
|
||||
ReleasePlanStep(
|
||||
id="govoplan:prepare-support",
|
||||
title="Prepare the complete developer meta-package outside this run",
|
||||
detail=(
|
||||
"First prepare and commit Core at the target and review module/requirements inputs. "
|
||||
"Stop active runs, preview and explicitly apply the frozen composition in a separate "
|
||||
"source checkout; review and commit manually. No durable self-update is supported."
|
||||
),
|
||||
command=preparation_command(workspace=Path(dashboard.workspace_root), target_version=unit.target_version),
|
||||
cwd=dashboard.meta_root, repo=unit.repo, status="needs-executor",
|
||||
),
|
||||
ReleasePlanStep(
|
||||
id="govoplan:publish-support",
|
||||
title="Publish the prepared Meta support source after Core",
|
||||
detail=(
|
||||
"After the matching Core annotated tag and exact main are published, use the shared "
|
||||
"guarded Meta tag preview/local-tag/publish route. Commit/push reviewed preparation "
|
||||
"and create a fresh durable run; do not update the current runtime binding."
|
||||
),
|
||||
cwd=dashboard.meta_root, repo=unit.repo, status="needs-executor",
|
||||
mutating=True,
|
||||
),
|
||||
))
|
||||
return tuple(steps)
|
||||
|
||||
|
||||
|
||||
@@ -19,10 +19,33 @@ from .git_state import (
|
||||
scoped_git_command,
|
||||
)
|
||||
from .repository_tag import RemoteTagResult, ref_commit, remote_tag_commit
|
||||
from .registry_reference import registry_entry_source
|
||||
from .version_alignment import repository_version_issues
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
def registered_source_origin_issues(
|
||||
*, repo_versions: dict[str, str], workspace: Path, remote: str,
|
||||
) -> tuple[SourceTagProvenanceIssue, ...]:
|
||||
"""Bind registry candidate attestations to registered source endpoints."""
|
||||
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
|
||||
issues = []
|
||||
for repo, version in sorted(repo_versions.items()):
|
||||
spec = specs.get(repo)
|
||||
if spec is None:
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source repository is not registered"))
|
||||
continue
|
||||
path = resolve_repo_path(spec, workspace)
|
||||
if path.absolute() != path.resolve() or not path.resolve().is_relative_to(workspace.resolve()):
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source checkout leaves the private workspace or traverses a symlink"))
|
||||
continue
|
||||
fetch = git_text(path, "remote", "get-url", "--all", remote).splitlines()
|
||||
push = git_text(path, "remote", "get-url", "--push", "--all", remote).splitlines()
|
||||
if fetch != [spec.remote] or push != [spec.remote]:
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source remote does not exactly match the registered origin"))
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
_CATALOG_PYTHON_REF = re.compile(
|
||||
r"/(?P<repo>govoplan-[a-z0-9-]+)\.git@v(?P<version>[^\s;]+)$"
|
||||
)
|
||||
@@ -61,6 +84,8 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
)
|
||||
|
||||
versions: dict[str, str] = {}
|
||||
registry_commits: dict[str, str] = {}
|
||||
registry_tag_objects: dict[str, str] = {}
|
||||
issues: list[SourceTagProvenanceIssue] = []
|
||||
entries: list[tuple[str, object]] = [("core_release", payload.get("core_release"))]
|
||||
modules = payload.get("modules")
|
||||
@@ -70,6 +95,21 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
for source, raw_entry in entries:
|
||||
if not isinstance(raw_entry, dict):
|
||||
continue
|
||||
try:
|
||||
registry_source = registry_entry_source(raw_entry)
|
||||
except ValueError as exc:
|
||||
issues.append(SourceTagProvenanceIssue(source, "", str(exc)))
|
||||
continue
|
||||
if registry_source is not None:
|
||||
repo, version = registry_source.repository, registry_source.version
|
||||
previous = versions.setdefault(repo, version)
|
||||
previous_commit = registry_commits.setdefault(repo, registry_source.commit)
|
||||
previous_object = registry_tag_objects.setdefault(repo, registry_source.tag_object)
|
||||
if (previous, previous_commit, previous_object) != (
|
||||
version, registry_source.commit, registry_source.tag_object,
|
||||
):
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "registry entries have conflicting immutable source identities"))
|
||||
continue
|
||||
python_ref = raw_entry.get("python_ref")
|
||||
match = _CATALOG_PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
|
||||
if match is None:
|
||||
@@ -89,8 +129,8 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
release = payload.get("release")
|
||||
selected_units = release.get("selected_units") if isinstance(release, dict) else None
|
||||
selected: dict[str, str] = {}
|
||||
selected_commits: dict[str, str] = {}
|
||||
selected_tag_objects: dict[str, str] = {}
|
||||
selected_commits: dict[str, str] = dict(registry_commits)
|
||||
selected_tag_objects: dict[str, str] = dict(registry_tag_objects)
|
||||
if not isinstance(selected_units, list) or not selected_units:
|
||||
issues.append(
|
||||
SourceTagProvenanceIssue(
|
||||
@@ -106,16 +146,22 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
repo = unit.get("repo")
|
||||
version = unit.get("version")
|
||||
if isinstance(repo, str) and isinstance(version, str):
|
||||
if repo in selected:
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "duplicate selected repository"))
|
||||
selected[repo] = version.removeprefix("v")
|
||||
commit = unit.get("commit_sha")
|
||||
tag_object = unit.get("tag_object_sha")
|
||||
if isinstance(commit, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", commit):
|
||||
if repo in registry_commits and registry_commits[repo] != commit.lower():
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "selected commit differs from registry artifact source"))
|
||||
selected_commits[repo] = commit.lower()
|
||||
else:
|
||||
issues.append(
|
||||
SourceTagProvenanceIssue(repo, f"v{version.removeprefix('v')}", "selected unit has no valid commit_sha provenance")
|
||||
)
|
||||
if isinstance(tag_object, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", tag_object):
|
||||
if repo in registry_tag_objects and registry_tag_objects[repo] != tag_object.lower():
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "selected tag object differs from registry artifact source"))
|
||||
selected_tag_objects[repo] = tag_object.lower()
|
||||
else:
|
||||
issues.append(
|
||||
|
||||
@@ -0,0 +1,792 @@
|
||||
"""Strict registered-source release contract for every source-tag batch.
|
||||
|
||||
Meta is not a root Python package. Its nested developer package is released only
|
||||
after a whole-batch source preflight and the matching immutable Core release.
|
||||
No selected checkout supplies the developer-package generator or release
|
||||
validation tooling. The trusted shared checker may load reviewed application
|
||||
manifests; this is not an untrusted-code sandbox.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import hashlib
|
||||
import os
|
||||
import re
|
||||
import stat
|
||||
|
||||
from .git_state import collect_repository_snapshot, git, git_text
|
||||
from .repository_tag import (
|
||||
_preview_repositories,
|
||||
basic_blocker,
|
||||
normalize_version,
|
||||
ref_commit,
|
||||
remote_tag_commit,
|
||||
run,
|
||||
)
|
||||
from .source_provenance import registered_source_origin_issues
|
||||
from .version_alignment import (
|
||||
repository_version_issues,
|
||||
selected_release_webui_bundle_issues,
|
||||
selected_webui_repository_names,
|
||||
)
|
||||
from .workspace import load_repository_specs, resolve_repo_path, resolve_workspace_root
|
||||
|
||||
_OBJECT = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?\Z")
|
||||
|
||||
|
||||
class SourceReceiptError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def _owned_path(path, *, directory):
|
||||
observed = path.lstat()
|
||||
expected = stat.S_ISDIR if directory else stat.S_ISREG
|
||||
if stat.S_ISLNK(observed.st_mode) or not expected(observed.st_mode):
|
||||
raise SourceReceiptError(
|
||||
"source authority must use real paths, without symlinks or special files"
|
||||
)
|
||||
if observed.st_uid != os.geteuid():
|
||||
raise SourceReceiptError(
|
||||
"source authority is not owned by the current operator"
|
||||
)
|
||||
if observed.st_mode & 0o022:
|
||||
raise SourceReceiptError("source authority is group/world writable")
|
||||
return observed
|
||||
|
||||
|
||||
def _trusted_ancestry(path):
|
||||
# Same ownership/mode policy as the publisher's trust-path guard. A sticky
|
||||
# shared ancestor such as /tmp may contain an owned, non-writable child;
|
||||
# the workspace/repository themselves are never given that exception.
|
||||
for ancestor in (path, *path.parents):
|
||||
observed = ancestor.lstat()
|
||||
if stat.S_ISLNK(observed.st_mode) or not stat.S_ISDIR(observed.st_mode):
|
||||
raise SourceReceiptError(
|
||||
"source ancestry must contain real directories, not symlinks"
|
||||
)
|
||||
if observed.st_uid not in {0, os.geteuid()}:
|
||||
raise SourceReceiptError("source ancestry has an untrusted owner")
|
||||
if observed.st_mode & 0o022 and not observed.st_mode & stat.S_ISVTX:
|
||||
raise SourceReceiptError("source ancestry is group/world writable")
|
||||
|
||||
|
||||
def _git_pointer(path):
|
||||
_owned_path(path, directory=False)
|
||||
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
observed = os.fstat(source.fileno())
|
||||
if not stat.S_ISREG(observed.st_mode) or not 0 < observed.st_size <= 4096:
|
||||
raise SourceReceiptError("Git metadata pointer is invalid or oversized")
|
||||
value = source.read(observed.st_size + 1)
|
||||
if len(value) != observed.st_size:
|
||||
raise SourceReceiptError("Git metadata pointer changed during validation")
|
||||
return value.decode("utf-8").strip()
|
||||
|
||||
|
||||
def _git_tree(root):
|
||||
pending = [(root, 0)]
|
||||
count = 0
|
||||
while pending:
|
||||
directory, depth = pending.pop()
|
||||
_owned_path(directory, directory=True)
|
||||
if depth > 128:
|
||||
raise SourceReceiptError(
|
||||
"Git metadata exceeds its trust-validation depth limit"
|
||||
)
|
||||
with os.scandir(directory) as entries:
|
||||
for entry in entries:
|
||||
count += 1
|
||||
if count > 500_000:
|
||||
raise SourceReceiptError(
|
||||
"Git metadata exceeds its trust-validation entry limit"
|
||||
)
|
||||
candidate = Path(entry.path)
|
||||
observed = candidate.lstat()
|
||||
if stat.S_ISDIR(observed.st_mode):
|
||||
pending.append((candidate, depth + 1))
|
||||
else:
|
||||
_owned_path(candidate, directory=False)
|
||||
|
||||
|
||||
def _filesystem_identity(path, observed):
|
||||
return [
|
||||
str(path),
|
||||
observed.st_dev,
|
||||
observed.st_ino,
|
||||
observed.st_uid,
|
||||
observed.st_gid,
|
||||
stat.S_IMODE(observed.st_mode),
|
||||
]
|
||||
|
||||
|
||||
def _source_filesystem(*, path, workspace):
|
||||
"""Validate source/Git ownership before invoking even read-only Git."""
|
||||
if path.absolute() != path.resolve() or not path.resolve().is_relative_to(
|
||||
workspace.resolve()
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"source checkout leaves the private workspace or traverses a symlink"
|
||||
)
|
||||
_trusted_ancestry(path)
|
||||
workspace_info = _owned_path(workspace, directory=True)
|
||||
repo_info = _owned_path(path, directory=True)
|
||||
marker = path / ".git"
|
||||
marker_info = marker.lstat()
|
||||
if stat.S_ISDIR(marker_info.st_mode):
|
||||
git_dir = marker
|
||||
else:
|
||||
value = _git_pointer(marker)
|
||||
if not value.startswith("gitdir: "):
|
||||
raise SourceReceiptError("Git worktree pointer is invalid")
|
||||
git_dir = Path(os.path.abspath(path / value.removeprefix("gitdir: ")))
|
||||
if git_dir.absolute() != git_dir.resolve() or not git_dir.resolve().is_relative_to(
|
||||
workspace.resolve()
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Git checkout metadata must stay inside the trusted workspace"
|
||||
)
|
||||
_trusted_ancestry(git_dir)
|
||||
_owned_path(git_dir, directory=True)
|
||||
common_dir = git_dir
|
||||
common_pointer = git_dir / "commondir"
|
||||
if common_pointer.exists() or common_pointer.is_symlink():
|
||||
common_dir = Path(os.path.abspath(git_dir / _git_pointer(common_pointer)))
|
||||
identities = {
|
||||
"workspace": _filesystem_identity(workspace, workspace_info),
|
||||
"checkout": _filesystem_identity(path, repo_info),
|
||||
}
|
||||
scanned = set()
|
||||
for label, directory in (
|
||||
("git_directory", git_dir),
|
||||
("git_common_directory", common_dir),
|
||||
):
|
||||
if (
|
||||
directory.absolute() != directory.resolve()
|
||||
or not directory.resolve().is_relative_to(workspace.resolve())
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Git checkout metadata must stay inside the trusted workspace"
|
||||
)
|
||||
_trusted_ancestry(directory)
|
||||
observed = _owned_path(directory, directory=True)
|
||||
if observed.st_mode & 0o700 != 0o700:
|
||||
raise SourceReceiptError(
|
||||
"Git metadata target must be readable and writable by its operator"
|
||||
)
|
||||
identities[label] = _filesystem_identity(directory, observed)
|
||||
if directory not in scanned:
|
||||
_git_tree(directory)
|
||||
scanned.add(directory)
|
||||
identities["git_marker"] = _filesystem_identity(
|
||||
marker, _owned_path(marker, directory=stat.S_ISDIR(marker_info.st_mode))
|
||||
)
|
||||
for candidate in (
|
||||
common_dir / "objects/info/alternates",
|
||||
common_dir / "objects/info/http-alternates",
|
||||
common_dir / "info/grafts",
|
||||
):
|
||||
if candidate.exists() or candidate.is_symlink():
|
||||
raise SourceReceiptError(
|
||||
"Git object alternates and grafts are not permitted"
|
||||
)
|
||||
return identities
|
||||
|
||||
|
||||
def _tracked_worktree(path):
|
||||
tracked = git(path, "ls-files", "-v", "-z", timeout=30)
|
||||
if tracked.returncode or len(tracked.stdout) > 16 * 1024 * 1024:
|
||||
raise SourceReceiptError(
|
||||
"tracked release inputs exceed their trust-validation limit"
|
||||
)
|
||||
checked = {path}
|
||||
tracked_paths = set()
|
||||
names = tracked.stdout.split("\0")
|
||||
if len(names) > 100_001:
|
||||
raise SourceReceiptError(
|
||||
"tracked release inputs exceed their trust-validation count limit"
|
||||
)
|
||||
for entry in filter(None, names):
|
||||
# git status deliberately hides assume-unchanged and skip-worktree
|
||||
# paths. Never validate mutable working metadata and then tag different
|
||||
# committed bytes because an index flag suppressed the dirty evidence.
|
||||
if not entry.startswith("H "):
|
||||
raise SourceReceiptError(
|
||||
"hidden, sparse or unmerged tracked index entries are not permitted"
|
||||
)
|
||||
name = entry[2:]
|
||||
tracked_paths.add(name)
|
||||
relative = Path(name)
|
||||
if relative.is_absolute() or ".." in relative.parts:
|
||||
raise SourceReceiptError("tracked release input has an unsafe path")
|
||||
candidate = path / relative
|
||||
for parent in candidate.parents:
|
||||
if parent == path:
|
||||
break
|
||||
if parent not in checked:
|
||||
_owned_path(parent, directory=True)
|
||||
checked.add(parent)
|
||||
_owned_path(candidate, directory=False)
|
||||
# Version/composition checks inspect existing files, including ignored
|
||||
# paths. Their declarations must come from the selected committed source,
|
||||
# not ignored working bytes absent from the tag's tree.
|
||||
metadata = [
|
||||
path / name
|
||||
for name in (
|
||||
"pyproject.toml",
|
||||
"package.json",
|
||||
"package-lock.json",
|
||||
"webui/package.json",
|
||||
"webui/package.release.json",
|
||||
"webui/package-lock.json",
|
||||
"webui/package-lock.release.json",
|
||||
)
|
||||
]
|
||||
if path.name == "govoplan":
|
||||
metadata.extend(
|
||||
path / name
|
||||
for name in (
|
||||
"packages/govoplan-meta/pyproject.toml",
|
||||
"requirements-release.txt",
|
||||
)
|
||||
)
|
||||
metadata.extend((path / "src").glob("**/backend/manifest.py"))
|
||||
metadata.extend((path / "src").glob("*/__init__.py"))
|
||||
for candidate in metadata:
|
||||
if (candidate.exists() or candidate.is_symlink()) and candidate.relative_to(
|
||||
path
|
||||
).as_posix() not in tracked_paths:
|
||||
raise SourceReceiptError(
|
||||
"selected release version/composition metadata must be tracked in the frozen source"
|
||||
)
|
||||
|
||||
|
||||
def _receipt(*, spec, workspace, version, filesystem):
|
||||
path = resolve_repo_path(spec, workspace)
|
||||
_tracked_worktree(path)
|
||||
snapshot = collect_repository_snapshot(
|
||||
spec, workspace_root=workspace, target_tag=f"v{version}", online=False
|
||||
)
|
||||
blocker = basic_blocker(snapshot=snapshot, version=version)
|
||||
if blocker:
|
||||
raise SourceReceiptError(blocker)
|
||||
if (
|
||||
not snapshot.exists
|
||||
or not snapshot.is_git
|
||||
or not snapshot.has_head
|
||||
or snapshot.errors
|
||||
or snapshot.safe_directory_required
|
||||
or snapshot.dirty
|
||||
or snapshot.branch != "main"
|
||||
or snapshot.upstream != "origin/main"
|
||||
or snapshot.behind
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"requires a clean registered main checkout tracking origin/main, not behind"
|
||||
)
|
||||
common = git_text(path, "rev-parse", "--path-format=absolute", "--git-common-dir")
|
||||
if (
|
||||
not common
|
||||
or Path(common).absolute() != Path(common).resolve()
|
||||
or not Path(common).resolve().is_relative_to(workspace.resolve())
|
||||
or git_text(path, "rev-parse", "--show-toplevel") != str(path.resolve())
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Git checkout metadata must stay inside the trusted workspace"
|
||||
)
|
||||
head = git_text(path, "rev-parse", "--verify", "HEAD")
|
||||
if not _OBJECT.fullmatch(head):
|
||||
raise SourceReceiptError("source HEAD is not an exact commit")
|
||||
live = run(
|
||||
("git", "ls-remote", "--exit-code", "--heads", "origin", "refs/heads/main"),
|
||||
cwd=path,
|
||||
)
|
||||
lines = live.stdout.strip().splitlines()
|
||||
if live.returncode or len(lines) != 1:
|
||||
raise SourceReceiptError("could not verify live origin/main")
|
||||
remote_main, separator, ref = lines[0].partition("\t")
|
||||
if not separator or ref != "refs/heads/main" or not _OBJECT.fullmatch(remote_main):
|
||||
raise SourceReceiptError("live origin/main returned an invalid source receipt")
|
||||
if git(path, "merge-base", "--is-ancestor", remote_main, head).returncode != 0:
|
||||
raise SourceReceiptError(
|
||||
"live origin/main is unavailable locally or diverges; fetch and review before retrying"
|
||||
)
|
||||
tag = f"v{version}"
|
||||
local_object = git_text(path, "rev-parse", "--verify", f"refs/tags/{tag}") or None
|
||||
if local_object:
|
||||
if git_text(path, "cat-file", "-t", f"refs/tags/{tag}") != "tag":
|
||||
raise SourceReceiptError("local immutable tag must be annotated")
|
||||
if ref_commit(path, f"refs/tags/{tag}") != head:
|
||||
raise SourceReceiptError(
|
||||
"local immutable tag points to another commit, not HEAD"
|
||||
)
|
||||
remote_tag = remote_tag_commit(path, remote="origin", tag=tag)
|
||||
if remote_tag.error:
|
||||
raise SourceReceiptError("could not verify the remote release tag")
|
||||
if remote_tag.tag_object:
|
||||
if not remote_tag.annotated:
|
||||
raise SourceReceiptError("remote immutable tag must be annotated")
|
||||
if remote_tag.commit != head:
|
||||
raise SourceReceiptError(
|
||||
"remote immutable tag points to another commit, not HEAD"
|
||||
)
|
||||
if local_object and remote_tag.tag_object != local_object:
|
||||
raise SourceReceiptError(
|
||||
"local and remote immutable tag annotation objects differ"
|
||||
)
|
||||
return {
|
||||
"head": head,
|
||||
"branch": "main",
|
||||
"upstream": "origin/main",
|
||||
"origin": spec.remote,
|
||||
"remote_main": remote_main,
|
||||
"tag": tag,
|
||||
"local_tag_object": local_object,
|
||||
"remote_tag_object": remote_tag.tag_object,
|
||||
"filesystem": filesystem,
|
||||
}
|
||||
|
||||
|
||||
def _collect_receipts(*, versions, specs, workspace):
|
||||
filesystems = {}
|
||||
for repo in versions:
|
||||
if repo not in specs:
|
||||
raise SourceReceiptError(f"{repo}: source repository is not registered")
|
||||
filesystems[repo] = _source_filesystem(
|
||||
path=resolve_repo_path(specs[repo], workspace), workspace=workspace
|
||||
)
|
||||
issues = registered_source_origin_issues(
|
||||
repo_versions=versions, workspace=workspace, remote="origin"
|
||||
)
|
||||
if issues:
|
||||
raise SourceReceiptError(
|
||||
"; ".join(f"{issue.repo}: {issue.message}" for issue in issues)
|
||||
)
|
||||
receipts = {}
|
||||
for repo, version in versions.items():
|
||||
if repo not in specs:
|
||||
raise SourceReceiptError(f"{repo}: source repository is not registered")
|
||||
try:
|
||||
receipts[repo] = _receipt(
|
||||
spec=specs[repo],
|
||||
workspace=workspace,
|
||||
version=version,
|
||||
filesystem=filesystems[repo],
|
||||
)
|
||||
except SourceReceiptError as exc:
|
||||
raise SourceReceiptError(f"{repo}: {exc}") from exc
|
||||
return receipts
|
||||
|
||||
|
||||
def _bundle_input_receipt(*, selected, workspace, push):
|
||||
"""Freeze only Core files used by the already-applicable WebUI gate.
|
||||
|
||||
These are read-only composition inputs, not a new Core-tag/version or
|
||||
clean-Core prerequisite. Local module candidates intentionally need none.
|
||||
"""
|
||||
if not push and "govoplan-core" not in selected:
|
||||
return {}
|
||||
if not selected_webui_repository_names(
|
||||
repo_versions=dict.fromkeys(selected, ""), workspace=workspace
|
||||
):
|
||||
return {}
|
||||
result = {}
|
||||
core = workspace / "govoplan-core"
|
||||
for relative in ("webui/package.release.json", "webui/package-lock.release.json"):
|
||||
path = core / relative
|
||||
if not path.exists() and not path.is_symlink():
|
||||
result[relative] = None # The unchanged shared gate explains missing input.
|
||||
continue
|
||||
_trusted_ancestry(path.parent)
|
||||
_owned_path(core, directory=True)
|
||||
_owned_path(path.parent, directory=True)
|
||||
observed = _owned_path(path, directory=False)
|
||||
if not 0 < observed.st_size <= 16 * 1024 * 1024:
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input exceeds its 16 MiB limit"
|
||||
)
|
||||
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
with os.fdopen(descriptor, "rb") as source:
|
||||
before = os.fstat(source.fileno())
|
||||
if (
|
||||
before.st_dev,
|
||||
before.st_ino,
|
||||
before.st_size,
|
||||
before.st_mtime_ns,
|
||||
before.st_ctime_ns,
|
||||
) != (
|
||||
observed.st_dev,
|
||||
observed.st_ino,
|
||||
observed.st_size,
|
||||
observed.st_mtime_ns,
|
||||
observed.st_ctime_ns,
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input changed during inspection"
|
||||
)
|
||||
content = source.read(before.st_size + 1)
|
||||
after = os.fstat(source.fileno())
|
||||
if len(content) != before.st_size or (
|
||||
before.st_dev,
|
||||
before.st_ino,
|
||||
before.st_size,
|
||||
before.st_mtime_ns,
|
||||
before.st_ctime_ns,
|
||||
) != (
|
||||
after.st_dev,
|
||||
after.st_ino,
|
||||
after.st_size,
|
||||
after.st_mtime_ns,
|
||||
after.st_ctime_ns,
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input changed during inspection"
|
||||
)
|
||||
result[relative] = {
|
||||
"file": _filesystem_identity(path, observed),
|
||||
"sha256": hashlib.sha256(content).hexdigest(),
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
def _frozen_receipts(
|
||||
*, expected, versions, specs, workspace, selected, push, bundle_inputs
|
||||
):
|
||||
actual = _collect_receipts(versions=versions, specs=specs, workspace=workspace)
|
||||
for repo in expected:
|
||||
if actual[repo] != expected[repo]:
|
||||
raise SourceReceiptError(
|
||||
f"{repo}: source receipt changed after whole-batch preflight"
|
||||
)
|
||||
for repo in versions:
|
||||
issues = repository_version_issues(
|
||||
resolve_repo_path(specs[repo], workspace), expected_version=versions[repo]
|
||||
)
|
||||
if issues:
|
||||
raise SourceReceiptError(
|
||||
f"{repo}: version/composition changed after whole-batch preflight"
|
||||
)
|
||||
if (
|
||||
_bundle_input_receipt(selected=selected, workspace=workspace, push=push)
|
||||
!= bundle_inputs
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Core release-bundle input receipt changed after whole-batch preflight"
|
||||
)
|
||||
if push or "govoplan-core" in selected:
|
||||
if selected_release_webui_bundle_issues(
|
||||
repo_versions={repo: versions[repo] for repo in selected},
|
||||
workspace=workspace,
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"release WebUI composition changed after whole-batch preflight"
|
||||
)
|
||||
|
||||
|
||||
def _require_core(receipts, *, push):
|
||||
core = receipts["govoplan-core"]
|
||||
if not core["local_tag_object"] or (
|
||||
push and (not core["remote_tag_object"] or core["remote_main"] != core["head"])
|
||||
):
|
||||
raise SourceReceiptError(
|
||||
"Meta requires the matching annotated Core tag locally and, for publication, remotely"
|
||||
)
|
||||
|
||||
|
||||
def _blocked(*, selected, apply, push, detail, rows=()): # noqa: A002
|
||||
known = {row["repo"]: row for row in rows}
|
||||
identified = next(
|
||||
(repo for repo in selected if detail.startswith(repo + ":")), None
|
||||
)
|
||||
return {
|
||||
"status": "blocked",
|
||||
"apply": apply,
|
||||
"push": push,
|
||||
"remote": "origin",
|
||||
"detail": "whole-batch source preflight failed; no selected repository was mutated",
|
||||
"repositories": [
|
||||
{
|
||||
**known.get(repo, {"repo": repo}),
|
||||
"status": "blocked"
|
||||
if (
|
||||
known.get(repo, {}).get("status") == "blocked"
|
||||
or (not rows and (identified is None or repo == identified))
|
||||
)
|
||||
else "skipped",
|
||||
"detail": known[repo]["detail"]
|
||||
if known.get(repo, {}).get("status") == "blocked"
|
||||
else detail,
|
||||
}
|
||||
for repo in selected
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def tag_source_batch(
|
||||
*, repos, repo_versions, workspace_root, remote, message, apply, push
|
||||
): # noqa: A002
|
||||
workspace = resolve_workspace_root(workspace_root)
|
||||
has_meta = "govoplan" in repos
|
||||
selected = tuple(dict.fromkeys(repos))
|
||||
if has_meta:
|
||||
selected = tuple(repo for repo in selected if repo != "govoplan") + (
|
||||
"govoplan",
|
||||
)
|
||||
if not selected:
|
||||
return {
|
||||
"status": "noop",
|
||||
"apply": apply,
|
||||
"push": push,
|
||||
"remote": remote.strip() or "origin",
|
||||
"repositories": [],
|
||||
}
|
||||
versions = {repo: normalize_version(repo_versions.get(repo)) for repo in selected}
|
||||
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
|
||||
meta_version = versions.get("govoplan")
|
||||
try:
|
||||
if remote.strip() not in {"", "origin"}:
|
||||
raise SourceReceiptError(
|
||||
"Source-tag batches require the registered origin remote"
|
||||
)
|
||||
if any(not version for version in versions.values()):
|
||||
raise SourceReceiptError(
|
||||
"every selected repository requires an explicit valid version"
|
||||
)
|
||||
if has_meta and versions.get("govoplan-core", meta_version) != meta_version:
|
||||
raise SourceReceiptError(
|
||||
"Meta developer-package version must match the selected Core release"
|
||||
)
|
||||
# Only Meta requires a frozen version-matched Core source/tag dependency.
|
||||
if has_meta:
|
||||
versions.setdefault("govoplan-core", meta_version)
|
||||
receipts = _collect_receipts(
|
||||
versions=versions, specs=specs, workspace=workspace
|
||||
)
|
||||
if has_meta and "govoplan-core" not in selected:
|
||||
_require_core(receipts, push=push)
|
||||
core_issues = (
|
||||
repository_version_issues(
|
||||
resolve_repo_path(specs["govoplan-core"], workspace),
|
||||
expected_version=meta_version,
|
||||
)
|
||||
if has_meta
|
||||
else ()
|
||||
)
|
||||
if core_issues:
|
||||
raise SourceReceiptError(
|
||||
"Core source metadata must match the selected Meta version"
|
||||
)
|
||||
bundle_inputs = _bundle_input_receipt(
|
||||
selected=selected, workspace=workspace, push=push
|
||||
)
|
||||
except (SourceReceiptError, OSError, ValueError) as exc:
|
||||
return _blocked(selected=selected, apply=apply, push=push, detail=str(exc))
|
||||
|
||||
# Preserve the shared complete manifest, package/lock and immutable tag
|
||||
# preflight. This invocation is always read-only; strict effects stay below.
|
||||
try:
|
||||
preview = _preview_repositories(
|
||||
repos=selected,
|
||||
repo_versions=repo_versions,
|
||||
workspace_root=workspace,
|
||||
remote="origin",
|
||||
message=message,
|
||||
push=push,
|
||||
)
|
||||
except (OSError, ValueError) as exc:
|
||||
return _blocked(
|
||||
selected=selected,
|
||||
apply=apply,
|
||||
push=push,
|
||||
detail=f"shared release preflight could not validate its inputs ({type(exc).__name__})",
|
||||
)
|
||||
rows = preview["repositories"]
|
||||
if preview["status"] in {"blocked", "partial"}:
|
||||
if not apply:
|
||||
return preview
|
||||
return _blocked(
|
||||
selected=selected,
|
||||
apply=True,
|
||||
push=push,
|
||||
detail="shared release preflight failed",
|
||||
rows=rows,
|
||||
)
|
||||
if not apply:
|
||||
rows = [
|
||||
{
|
||||
**row,
|
||||
"status": "planned",
|
||||
"detail": "existing annotated release tag requires atomic main publication",
|
||||
}
|
||||
if push
|
||||
and receipts[row["repo"]]["remote_main"] != receipts[row["repo"]]["head"]
|
||||
else row
|
||||
for row in rows
|
||||
]
|
||||
preview = {**preview, "repositories": rows}
|
||||
if any(row["status"] == "planned" for row in rows):
|
||||
preview["status"] = "planned"
|
||||
return {
|
||||
**preview,
|
||||
"source_receipts": receipts,
|
||||
"source_contract": "registered-meta-batch-v1"
|
||||
if has_meta
|
||||
else "registered-source-batch-v1",
|
||||
"bundle_input_receipts": bundle_inputs,
|
||||
}
|
||||
|
||||
results = []
|
||||
effected = False
|
||||
for row in rows:
|
||||
repo = row["repo"]
|
||||
receipt = receipts[repo]
|
||||
path = resolve_repo_path(specs[repo], workspace)
|
||||
tag = receipt["tag"]
|
||||
head = receipt["head"]
|
||||
try:
|
||||
# Recheck the entire frozen batch, including Meta and Core, before
|
||||
# every effect. Earlier successful effects update only their exact
|
||||
# anticipated tag/branch receipt fields below.
|
||||
_frozen_receipts(
|
||||
expected=receipts,
|
||||
versions=versions,
|
||||
specs=specs,
|
||||
workspace=workspace,
|
||||
selected=selected,
|
||||
push=push,
|
||||
bundle_inputs=bundle_inputs,
|
||||
)
|
||||
if repo == "govoplan":
|
||||
_require_core(receipts, push=push)
|
||||
local_object = receipt["local_tag_object"]
|
||||
created = False
|
||||
if not local_object:
|
||||
if receipt["remote_tag_object"]:
|
||||
command = (
|
||||
"git",
|
||||
"fetch",
|
||||
"--no-tags",
|
||||
"origin",
|
||||
f"refs/tags/{tag}:refs/tags/{tag}",
|
||||
)
|
||||
else:
|
||||
command = ("git", "tag", "-a", tag, head, "-m", row["message"])
|
||||
created = True
|
||||
effected = True
|
||||
if run(command, cwd=path).returncode:
|
||||
raise SourceReceiptError(
|
||||
"annotated local release tag could not be created or retrieved"
|
||||
)
|
||||
local_object = git_text(
|
||||
path, "rev-parse", "--verify", f"refs/tags/{tag}"
|
||||
)
|
||||
if (
|
||||
not local_object
|
||||
or ref_commit(path, f"refs/tags/{tag}") != head
|
||||
or git_text(path, "cat-file", "-t", f"refs/tags/{tag}") != "tag"
|
||||
or (
|
||||
receipt["remote_tag_object"]
|
||||
and local_object != receipt["remote_tag_object"]
|
||||
)
|
||||
):
|
||||
raise SourceReceiptError("local release tag postcondition failed")
|
||||
receipt["local_tag_object"] = local_object
|
||||
_frozen_receipts(
|
||||
expected=receipts,
|
||||
versions=versions,
|
||||
specs=specs,
|
||||
workspace=workspace,
|
||||
selected=selected,
|
||||
push=push,
|
||||
bundle_inputs=bundle_inputs,
|
||||
)
|
||||
if push and (
|
||||
receipt["remote_tag_object"] != local_object
|
||||
or receipt["remote_main"] != head
|
||||
):
|
||||
# Pin both effects to verified objects, not mutable HEAD/tag
|
||||
# names. No force, retagging, fallback or non-atomic retry.
|
||||
command = (
|
||||
"git",
|
||||
"push",
|
||||
"--atomic",
|
||||
"origin",
|
||||
f"{head}:refs/heads/main",
|
||||
f"{local_object}:refs/tags/{tag}",
|
||||
)
|
||||
effected = True
|
||||
if run(command, cwd=path).returncode:
|
||||
raise SourceReceiptError(
|
||||
"atomic main and annotated tag publication failed; inspect receipts before retrying"
|
||||
)
|
||||
receipt["remote_main"] = head
|
||||
receipt["remote_tag_object"] = local_object
|
||||
# Verify remote main AND exact annotated object, as well as local
|
||||
# source state. A successful Git exit alone is never a receipt.
|
||||
_frozen_receipts(
|
||||
expected=receipts,
|
||||
versions=versions,
|
||||
specs=specs,
|
||||
workspace=workspace,
|
||||
selected=selected,
|
||||
push=push,
|
||||
bundle_inputs=bundle_inputs,
|
||||
)
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "published" if push else "tagged" if created else "noop",
|
||||
"detail": "verified strict registered-source release"
|
||||
if not receipt["remote_tag_object"] or created
|
||||
else "verified strict registered-source release; immutable annotation already published or present",
|
||||
"after_local_tag_commit": head,
|
||||
"after_local_tag_object": local_object,
|
||||
"after_remote_tag_commit": head
|
||||
if receipt["remote_tag_object"]
|
||||
else None,
|
||||
"after_remote_tag_object": receipt["remote_tag_object"],
|
||||
"after_remote_main_commit": receipt["remote_main"],
|
||||
}
|
||||
)
|
||||
except (SourceReceiptError, OSError, ValueError) as exc:
|
||||
results.append(
|
||||
{
|
||||
**row,
|
||||
"status": "failed" if effected else "blocked",
|
||||
"detail": str(exc),
|
||||
}
|
||||
)
|
||||
results.extend(
|
||||
{
|
||||
**later,
|
||||
"status": "skipped",
|
||||
"detail": "earlier strict source effect or receipt failed",
|
||||
}
|
||||
for later in rows[len(results) :]
|
||||
)
|
||||
return {
|
||||
"status": "partial" if effected else "blocked",
|
||||
"apply": True,
|
||||
"push": push,
|
||||
"remote": "origin",
|
||||
"repositories": results,
|
||||
}
|
||||
status = (
|
||||
"published"
|
||||
if push
|
||||
else "tagged"
|
||||
if any(row["status"] == "tagged" for row in results)
|
||||
else "noop"
|
||||
)
|
||||
return {
|
||||
"status": status,
|
||||
"apply": True,
|
||||
"push": push,
|
||||
"remote": "origin",
|
||||
"repositories": results,
|
||||
"source_receipts": receipts,
|
||||
"source_contract": "registered-meta-batch-v1"
|
||||
if has_meta
|
||||
else "registered-source-batch-v1",
|
||||
"bundle_input_receipts": bundle_inputs,
|
||||
}
|
||||
@@ -6,11 +6,13 @@ from dataclasses import dataclass
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import runpy
|
||||
import subprocess
|
||||
import tomllib
|
||||
|
||||
from .git_state import collect_versions, sanitized_git_environment
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
from .git_state import collect_versions, registered_developer_meta_path, sanitized_git_environment
|
||||
from .registry_reference import registry_artifact_conflicts, registry_entry_source
|
||||
from .workspace import META_ROOT, load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
_PYTHON_RELEASE_REF = re.compile(
|
||||
@@ -42,6 +44,7 @@ def repository_version_issues(
|
||||
versions = collect_versions(repo_path)
|
||||
declared = {
|
||||
"pyproject.toml": versions.pyproject,
|
||||
"packages/govoplan-meta/pyproject.toml": versions.developer_meta,
|
||||
"package.json": versions.package,
|
||||
"webui/package.json": versions.webui_package,
|
||||
"webui/package.release.json": _json_version(repo_path / "webui" / "package.release.json")
|
||||
@@ -82,6 +85,8 @@ def repository_version_issues(
|
||||
for source, version in declared.items()
|
||||
if version is not None and version != canonical_version
|
||||
]
|
||||
if registered_developer_meta_path(repo_path) is not None:
|
||||
issues.extend(developer_meta_composition_issues(repo_path))
|
||||
|
||||
if expected_version is not None and canonical_version.removeprefix("v") != expected_version.removeprefix("v"):
|
||||
issues.append(
|
||||
@@ -119,6 +124,34 @@ def repository_version_issues(
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
def developer_meta_composition_issues(repo_path: Path) -> tuple[VersionAlignmentIssue, ...]:
|
||||
"""Compare the real nested package with the trusted generator's exact output.
|
||||
|
||||
Never execute a generator from a selected checkout. Only the installed
|
||||
operator tooling provides code; selected TOML/requirements are data inputs.
|
||||
"""
|
||||
package_path = registered_developer_meta_path(repo_path)
|
||||
if package_path is None:
|
||||
return (VersionAlignmentIssue(repo_path.name, "repository", "registered Meta support repository", "", "nested developer-package identity is not registered"),)
|
||||
source = "packages/govoplan-meta/pyproject.toml"
|
||||
try:
|
||||
current = package_path.read_text(encoding="utf-8")
|
||||
project = tomllib.loads(current).get("project")
|
||||
if not isinstance(project, dict) or project.get("name") != "govoplan":
|
||||
return (VersionAlignmentIssue("govoplan", source + ":project.name", "govoplan", str(project.get("name") if isinstance(project, dict) else ""), "developer meta-package identity must be exact"),)
|
||||
# META_ROOT belongs to the running operator tools, not repo_path.
|
||||
generator = runpy.run_path(str(META_ROOT / "tools/release/generate-developer-meta-package.py"))
|
||||
expected = generator["render"](
|
||||
workspace=repo_path.parent,
|
||||
requirements=repo_path / "requirements-release.txt",
|
||||
)
|
||||
except (OSError, UnicodeError, KeyError, ValueError, TypeError) as exc:
|
||||
return (VersionAlignmentIssue("govoplan", source, "readable exact developer composition and Core version", type(exc).__name__, "developer meta-package composition could not be validated"),)
|
||||
if current != expected:
|
||||
return (VersionAlignmentIssue("govoplan", source, "trusted generator output matching Core and release requirements", "stale composition", "developer meta-package must exactly match the generator --check contract"),)
|
||||
return ()
|
||||
|
||||
|
||||
def selected_repository_version_issues(
|
||||
*,
|
||||
repo_versions: dict[str, str],
|
||||
@@ -164,6 +197,11 @@ def selected_repository_version_issues(
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
def selected_webui_repository_names(*, repo_versions: dict[str, str], workspace: Path) -> tuple[str, ...]:
|
||||
"""Identify the exact selections for which Core's WebUI inputs are relevant."""
|
||||
return tuple(repo for repo in sorted(repo_versions) if repo != "govoplan-core" and (workspace / repo / "webui/package.json").exists())
|
||||
|
||||
|
||||
def selected_release_webui_bundle_issues(
|
||||
*,
|
||||
repo_versions: dict[str, str],
|
||||
@@ -176,6 +214,8 @@ def selected_release_webui_bundle_issues(
|
||||
immutable release package input and lockfile will actually install.
|
||||
"""
|
||||
|
||||
if not selected_webui_repository_names(repo_versions=repo_versions, workspace=workspace):
|
||||
return ()
|
||||
core_webui = workspace / "govoplan-core" / "webui"
|
||||
release_package_path = core_webui / "package.release.json"
|
||||
release_lock_path = core_webui / "package-lock.release.json"
|
||||
@@ -407,6 +447,11 @@ def candidate_catalog_version_issues(payload: object) -> tuple[VersionAlignmentI
|
||||
)
|
||||
|
||||
release = payload.get("release")
|
||||
registry_entries = [core_release, *(modules if isinstance(modules, list) else [])]
|
||||
issues.extend(
|
||||
_catalog_shape_issue("artifact_integrity", issue)
|
||||
for issue in registry_artifact_conflicts(registry_entries)
|
||||
)
|
||||
if isinstance(release, dict):
|
||||
issues.extend(_catalog_release_issues(release, represented=represented))
|
||||
return tuple(issues)
|
||||
@@ -419,6 +464,15 @@ def _catalog_entry_issues(
|
||||
represented: dict[str, str],
|
||||
) -> list[VersionAlignmentIssue]:
|
||||
issues: list[VersionAlignmentIssue] = []
|
||||
try:
|
||||
registry_source = registry_entry_source(entry)
|
||||
except ValueError as exc:
|
||||
return [_catalog_shape_issue(source, str(exc))]
|
||||
if registry_source is not None:
|
||||
previous = represented.setdefault(registry_source.repository, registry_source.version)
|
||||
if previous != registry_source.version:
|
||||
issues.append(_catalog_shape_issue(source, "repository appears with conflicting catalog versions"))
|
||||
return issues
|
||||
version = entry.get("version")
|
||||
normalized_version = version.removeprefix("v") if isinstance(version, str) and version else None
|
||||
if normalized_version is None:
|
||||
|
||||
@@ -34,6 +34,19 @@ def version_metadata_mutations(
|
||||
) -> tuple[VersionFileMutation, ...]:
|
||||
"""Render all recognized repository version files without writing them."""
|
||||
|
||||
from .git_state import registered_developer_meta_path
|
||||
|
||||
if registered_developer_meta_path(repo_path) is not None:
|
||||
from .meta_preparation import MetaPreparationError, PACKAGE, preview_meta_mutation
|
||||
|
||||
try:
|
||||
_receipt, before, after = preview_meta_mutation(
|
||||
repo_path=repo_path, target_version=target_version,
|
||||
)
|
||||
except (MetaPreparationError, OSError, ValueError, KeyError, TypeError) as exc:
|
||||
raise VersionMetadataError(str(exc)) from exc
|
||||
return (VersionFileMutation(PACKAGE, before, after),) if before != after else ()
|
||||
|
||||
version = target_version.removeprefix("v")
|
||||
candidates: list[tuple[Path, str]] = []
|
||||
if (repo_path / "pyproject.toml").is_file():
|
||||
@@ -115,6 +128,14 @@ def apply_version_metadata_mutations(
|
||||
) -> tuple[str, ...]:
|
||||
"""Apply one deterministic version update, rolling back on write failure."""
|
||||
|
||||
from .git_state import registered_developer_meta_path
|
||||
|
||||
if registered_developer_meta_path(repo_path) is not None:
|
||||
raise VersionMetadataError(
|
||||
"Meta is prepared outside durable runs with prepare-developer-meta-package.py; "
|
||||
"review its complete generated composition, commit, and create a fresh run."
|
||||
)
|
||||
|
||||
mutations = version_metadata_mutations(
|
||||
repo_path,
|
||||
target_version=target_version,
|
||||
@@ -269,6 +290,7 @@ def _render_python_version(
|
||||
except SyntaxError as exc:
|
||||
raise VersionMetadataError(f"Python metadata is malformed: {path.name}") from exc
|
||||
values: list[ast.Constant] = []
|
||||
module_version_references = 0
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Call) or _call_name(node.func) != target_name:
|
||||
continue
|
||||
@@ -279,6 +301,28 @@ def _render_python_version(
|
||||
and isinstance(keyword.value.value, str)
|
||||
):
|
||||
values.append(keyword.value)
|
||||
elif (
|
||||
keyword.arg == keyword_name
|
||||
and isinstance(keyword.value, ast.Name)
|
||||
and keyword.value.id == "MODULE_VERSION"
|
||||
):
|
||||
module_version_references += 1
|
||||
if module_version_references:
|
||||
if module_version_references != 1 or values:
|
||||
raise VersionMetadataError(
|
||||
f"Python metadata has multiple {target_name}.{keyword_name} values: {path.name}"
|
||||
)
|
||||
rendered, found = _render_python_assignment(
|
||||
payload,
|
||||
path=path,
|
||||
assignment_name="MODULE_VERSION",
|
||||
version=version,
|
||||
)
|
||||
if not found:
|
||||
raise VersionMetadataError(
|
||||
f"Python metadata has no literal MODULE_VERSION declaration: {path.name}"
|
||||
)
|
||||
return rendered, True
|
||||
if not values:
|
||||
return payload, False
|
||||
if len(values) != 1:
|
||||
|
||||
@@ -19,8 +19,9 @@ retry() {
|
||||
for attempt in 1 2 3; do
|
||||
if "$@"; then
|
||||
return 0
|
||||
else
|
||||
status=$?
|
||||
fi
|
||||
status=$?
|
||||
if [[ "$attempt" == 3 ]]; then
|
||||
return "$status"
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Preview or explicitly prepare Meta outside a durable release run."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from govoplan_release.meta_preparation import (
|
||||
MetaPreparationAmbiguous,
|
||||
MetaPreparationError,
|
||||
_read_input,
|
||||
prepare_developer_meta_package,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--workspace", type=Path, required=True)
|
||||
parser.add_argument("--target-version", required=True)
|
||||
parser.add_argument("--apply", action="store_true")
|
||||
parser.add_argument("--receipt", type=Path)
|
||||
parser.add_argument("--confirm-out-of-run", action="store_true")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
expected = None
|
||||
if args.apply:
|
||||
if args.receipt is None:
|
||||
raise MetaPreparationError(
|
||||
"Apply requires the reviewed preview JSON via --receipt."
|
||||
)
|
||||
payload, _ = _read_input(args.receipt)
|
||||
expected = json.loads(payload)["receipt"]
|
||||
result = prepare_developer_meta_package(
|
||||
repo_path=args.workspace.absolute() / "govoplan",
|
||||
target_version=args.target_version,
|
||||
apply=args.apply,
|
||||
expected_receipt=expected,
|
||||
confirm_out_of_run=args.confirm_out_of_run,
|
||||
)
|
||||
except (MetaPreparationError, OSError, ValueError, KeyError, TypeError) as exc:
|
||||
status = "needs-reconciliation" if isinstance(exc, MetaPreparationAmbiguous) else "blocked"
|
||||
print(json.dumps({"status": status, "detail": str(exc)}))
|
||||
return 1
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -21,6 +21,22 @@ def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
full = subparsers.add_parser("full-registry", help="Build a strict full-profile candidate from verified registry artifacts.")
|
||||
full.add_argument("--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT)
|
||||
full.add_argument("--package-set", type=Path, required=True)
|
||||
full.add_argument("--package-lock", type=Path, required=True)
|
||||
full.add_argument("--wheelhouse", type=Path, required=True)
|
||||
full.add_argument("--webui-packages", type=Path, required=True)
|
||||
full.add_argument("--output-dir", type=Path, required=True)
|
||||
full.add_argument("--selected-repository", action="append", required=True)
|
||||
full.add_argument("--catalog-signing-key", action="append", required=True)
|
||||
full.add_argument("--channel", default="stable")
|
||||
full.add_argument("--source-remote", default="origin")
|
||||
full.add_argument("--public-base-url", default="https://govoplan.add-ideas.de")
|
||||
full.add_argument("--expires-days", type=int, default=90)
|
||||
full.add_argument("--sequence", type=int)
|
||||
full.add_argument("--json", action="store_true")
|
||||
|
||||
selective = subparsers.add_parser(
|
||||
"selective", help="Build a signed selective channel catalog candidate."
|
||||
)
|
||||
@@ -143,6 +159,21 @@ def main() -> int:
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
if args.command == "full-registry":
|
||||
require_release_runtime_trust()
|
||||
from govoplan_release.full_catalog import build_full_registry_candidate
|
||||
|
||||
result = build_full_registry_candidate(
|
||||
package_set_path=args.package_set, package_lock_path=args.package_lock,
|
||||
wheelhouse=args.wheelhouse, webui_packages=args.webui_packages,
|
||||
output_dir=args.output_dir, selected_repositories=tuple(args.selected_repository),
|
||||
signing_keys=tuple(args.catalog_signing_key), workspace_root=args.workspace_root,
|
||||
channel=args.channel, source_remote=args.source_remote,
|
||||
public_base_url=args.public_base_url, expires_days=args.expires_days,
|
||||
sequence=args.sequence,
|
||||
)
|
||||
print(json.dumps(result, indent=2, sort_keys=True))
|
||||
return 0
|
||||
if args.command == "selective":
|
||||
require_release_runtime_trust()
|
||||
result = build_selective_catalog_candidate(
|
||||
|
||||
@@ -8,7 +8,7 @@ from datetime import datetime, timezone
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import Any, NoReturn
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[2]
|
||||
@@ -168,12 +168,13 @@ def owner_for_versions_dir(versions_dir: Path) -> str:
|
||||
|
||||
|
||||
def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||
if path.stat().st_size > 2 * 1024 * 1024:
|
||||
raise ValueError(f"{path}: migration source exceeds the 2 MiB audit bound")
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
values: dict[str, Any] = {}
|
||||
wrapped: Migration | None = None
|
||||
release_peer = path.parent.parent / "versions" / path.name
|
||||
if path.parent.name == "dev_versions" and release_peer.is_file():
|
||||
wrapped = parse_migration_file(owner, release_peer)
|
||||
values = _imported_release_metadata(owner, path, tree)
|
||||
wrapped = _wrapped_release_metadata(owner, path, tree)
|
||||
if values and wrapped:
|
||||
raise ValueError(f"{path}: mixed migration metadata wrapper styles are ambiguous")
|
||||
for statement in tree.body:
|
||||
if isinstance(statement, ast.Assign):
|
||||
for target in statement.targets:
|
||||
@@ -196,6 +197,8 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||
)
|
||||
revision = values.get("revision")
|
||||
if not isinstance(revision, str):
|
||||
if "revision" in values:
|
||||
raise ValueError(f"{path}: migration revision must be an explicit string")
|
||||
return None
|
||||
return Migration(
|
||||
owner=owner,
|
||||
@@ -207,28 +210,200 @@ def parse_migration_file(owner: str, path: Path) -> Migration | None:
|
||||
)
|
||||
|
||||
|
||||
def _ast_binding_count(tree: ast.Module, name: str) -> int:
|
||||
count = 0
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Name) and node.id == name and isinstance(node.ctx, (ast.Store, ast.Del)):
|
||||
count += 1
|
||||
elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) and node.name == name:
|
||||
count += 1
|
||||
elif isinstance(node, (ast.Import, ast.ImportFrom)):
|
||||
count += sum(
|
||||
(alias.asname or (alias.name.split(".")[0] if isinstance(node, ast.Import) else alias.name)) == name
|
||||
for alias in node.names
|
||||
)
|
||||
return count
|
||||
|
||||
|
||||
def _release_wrapper_peer(owner: str, path: Path, filename: str) -> Migration:
|
||||
versions = path.parent.parent / "versions"
|
||||
peer = versions / filename
|
||||
if (
|
||||
path.parent.name != "dev_versions"
|
||||
or Path(filename).name != filename or not filename.endswith(".py")
|
||||
or versions.is_symlink() or peer.is_symlink() or not peer.is_file()
|
||||
or peer.resolve().parent != versions.resolve()
|
||||
or peer.stat().st_size > 2 * 1024 * 1024
|
||||
):
|
||||
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
|
||||
migration = parse_migration_file(owner, peer)
|
||||
if migration is None:
|
||||
raise ValueError(f"{path}: release wrapper target has no migration metadata")
|
||||
return migration
|
||||
|
||||
|
||||
def _imported_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Any]:
|
||||
"""Recognize explicit metadata re-exports, never star/dynamic imports."""
|
||||
names = {"revision", "down_revision", "depends_on", "branch_labels"}
|
||||
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
|
||||
values: dict[str, Any] = {}
|
||||
targets: set[str] = set()
|
||||
for statement in tree.body:
|
||||
if not isinstance(statement, ast.ImportFrom):
|
||||
continue
|
||||
selected = [alias for alias in statement.names if alias.name in names or (alias.asname or alias.name) in names]
|
||||
if not selected:
|
||||
if (statement.module or "").startswith(prefix) and any(alias.name == "*" for alias in statement.names):
|
||||
raise ValueError(f"{path}: unsupported wildcard migration metadata")
|
||||
continue
|
||||
stem = (statement.module or "").removeprefix(prefix)
|
||||
if (
|
||||
statement.level or not (statement.module or "").startswith(prefix)
|
||||
or not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem)
|
||||
):
|
||||
raise ValueError(f"{path}: unsupported or ambiguous imported migration metadata")
|
||||
targets.add(stem)
|
||||
if len(targets) != 1:
|
||||
raise ValueError(f"{path}: ambiguous imported migration metadata sources")
|
||||
migration = _release_wrapper_peer(owner, path, stem + ".py")
|
||||
projection = {
|
||||
"revision": migration.revision, "down_revision": migration.down_revisions,
|
||||
"depends_on": migration.depends_on, "branch_labels": migration.branch_labels,
|
||||
}
|
||||
for alias in selected:
|
||||
name = alias.asname or alias.name
|
||||
if name != alias.name or name in values or _ast_binding_count(tree, name) != 1:
|
||||
raise ValueError(f"{path}: ambiguous imported migration metadata assignment")
|
||||
values[name] = projection[name]
|
||||
return values
|
||||
|
||||
|
||||
def _wrapped_release_metadata(owner: str, path: Path, tree: ast.Module) -> dict[str, Migration]:
|
||||
"""Resolve only known literal sibling wrappers, without importing any code."""
|
||||
metadata_names = {"revision", "down_revision", "depends_on", "branch_labels"}
|
||||
aliases: set[str] = set()
|
||||
bindings: dict[str, list[ast.expr]] = {}
|
||||
imported: dict[str, list[str]] = {}
|
||||
for statement in tree.body:
|
||||
if isinstance(statement, ast.ImportFrom) and not statement.level:
|
||||
for alias in statement.names:
|
||||
imported.setdefault(alias.asname or alias.name, []).append(f"{statement.module}.{alias.name}")
|
||||
targets: list[ast.expr] = []
|
||||
value: ast.expr | None = None
|
||||
if isinstance(statement, ast.Assign):
|
||||
targets, value = statement.targets, statement.value
|
||||
elif isinstance(statement, ast.AnnAssign) and statement.value is not None:
|
||||
targets, value = [statement.target], statement.value
|
||||
for target in targets:
|
||||
if isinstance(target, ast.Name) and value is not None:
|
||||
bindings.setdefault(target.id, []).append(value)
|
||||
if target.id in metadata_names and isinstance(value, ast.Attribute) and isinstance(value.value, ast.Name):
|
||||
aliases.add(value.value.id)
|
||||
if not aliases:
|
||||
return {}
|
||||
if path.parent.name != "dev_versions":
|
||||
raise ValueError(f"{path}: non-literal release migration metadata is unsupported")
|
||||
|
||||
def reject() -> NoReturn:
|
||||
raise ValueError(f"{path}: unsupported or ambiguous development migration wrapper")
|
||||
|
||||
def binding_count(name: str) -> int:
|
||||
return _ast_binding_count(tree, name)
|
||||
|
||||
def assigned(name: str) -> ast.expr:
|
||||
values = bindings.get(name, ())
|
||||
if len(values) != 1 or binding_count(name) != 1 or name in imported:
|
||||
reject()
|
||||
return values[0]
|
||||
|
||||
def imported_as(node: ast.expr, qualified: str) -> bool:
|
||||
return (
|
||||
isinstance(node, ast.Name)
|
||||
and imported.get(node.id) == [qualified]
|
||||
and binding_count(node.id) == 1
|
||||
)
|
||||
|
||||
def call(node: ast.expr, qualified: str, arguments: int) -> bool:
|
||||
return isinstance(node, ast.Call) and imported_as(node.func, qualified) and len(node.args) == arguments and not node.keywords
|
||||
|
||||
def file_wrapper_target(node: ast.expr) -> str:
|
||||
if binding_count("__file__"):
|
||||
reject()
|
||||
if isinstance(node, ast.Name):
|
||||
node = assigned(node.id)
|
||||
if not (
|
||||
isinstance(node, ast.BinOp) and isinstance(node.op, ast.Div)
|
||||
and isinstance(node.right, ast.Constant) and isinstance(node.right.value, str)
|
||||
and isinstance(node.left, ast.BinOp) and isinstance(node.left.op, ast.Div)
|
||||
and isinstance(node.left.right, ast.Constant) and node.left.right.value == "versions"
|
||||
):
|
||||
reject()
|
||||
root = node.left.left
|
||||
for name in imported:
|
||||
if imported_as(ast.Name(id=name), "pathlib.Path"):
|
||||
expected = ast.parse(f"{name}(__file__).resolve().parents[1]", mode="eval").body
|
||||
if ast.dump(root) == ast.dump(expected):
|
||||
return node.right.value
|
||||
reject()
|
||||
|
||||
result: dict[str, Migration] = {}
|
||||
resolved: set[Path] = set()
|
||||
for alias in sorted(aliases):
|
||||
value = assigned(alias)
|
||||
if call(value, "importlib.import_module", 1):
|
||||
argument = value.args[0]
|
||||
if not isinstance(argument, ast.Constant) or not isinstance(argument.value, str):
|
||||
reject()
|
||||
prefix = f"{owner.replace('-', '_')}.backend.migrations.versions."
|
||||
if not argument.value.startswith(prefix):
|
||||
reject()
|
||||
stem = argument.value.removeprefix(prefix)
|
||||
if not stem or any(character not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for character in stem):
|
||||
reject()
|
||||
filename = stem + ".py"
|
||||
elif call(value, "importlib.util.module_from_spec", 1):
|
||||
argument = value.args[0]
|
||||
if not isinstance(argument, ast.Name):
|
||||
reject()
|
||||
specification = assigned(argument.id)
|
||||
if not call(specification, "importlib.util.spec_from_file_location", 2):
|
||||
reject()
|
||||
if not isinstance(specification.args[0], ast.Constant) or not isinstance(specification.args[0].value, str):
|
||||
reject()
|
||||
filename = file_wrapper_target(specification.args[1])
|
||||
else:
|
||||
reject()
|
||||
migration = _release_wrapper_peer(owner, path, filename)
|
||||
peer = migration.path
|
||||
resolved.add(peer.resolve())
|
||||
if len(resolved) > 1:
|
||||
reject()
|
||||
result[alias] = migration
|
||||
return result
|
||||
|
||||
|
||||
def _migration_assignment_value(
|
||||
name: str,
|
||||
value: ast.expr,
|
||||
*,
|
||||
wrapped: Migration | None,
|
||||
wrapped: dict[str, Migration],
|
||||
) -> Any:
|
||||
try:
|
||||
return ast.literal_eval(value)
|
||||
except (ValueError, TypeError):
|
||||
if (
|
||||
wrapped is None
|
||||
or not isinstance(value, ast.Attribute)
|
||||
not isinstance(value, ast.Attribute)
|
||||
or not isinstance(value.value, ast.Name)
|
||||
or value.value.id != "_migration"
|
||||
or value.value.id not in wrapped
|
||||
or value.attr != name
|
||||
):
|
||||
return None
|
||||
raise ValueError(f"Unsupported or ambiguous migration metadata: {name}")
|
||||
migration = wrapped[value.value.id]
|
||||
return {
|
||||
"revision": wrapped.revision,
|
||||
"down_revision": wrapped.down_revisions,
|
||||
"depends_on": wrapped.depends_on,
|
||||
"branch_labels": wrapped.branch_labels,
|
||||
"revision": migration.revision,
|
||||
"down_revision": migration.down_revisions,
|
||||
"depends_on": migration.depends_on,
|
||||
"branch_labels": migration.branch_labels,
|
||||
}[name]
|
||||
|
||||
|
||||
|
||||
@@ -2141,6 +2141,7 @@
|
||||
function primaryVersion(repo) {
|
||||
const versions = repo.versions || {};
|
||||
if (versions.pyproject) return versions.pyproject;
|
||||
if (versions.developer_meta) return versions.developer_meta;
|
||||
if (versions.package) return versions.package;
|
||||
if (versions.webui_package) return versions.webui_package;
|
||||
if (Array.isArray(versions.manifests) && versions.manifests.length) return versions.manifests[0];
|
||||
|
||||
Reference in New Issue
Block a user