Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
88b685ff5e | ||
|
|
be57a1823a | ||
|
|
6bcb75f577 | ||
|
|
32fe4b7238 | ||
|
|
6a8f53b87d | ||
|
|
1cec4ee1d8 | ||
|
|
29d03aa2ca | ||
|
|
6fb928d6cf | ||
|
|
6edaaadf37 | ||
|
|
0b171fbdd4 |
@@ -72,7 +72,7 @@ Each WebUI module should be able to announce:
|
||||
The contract references surfaces. It does not permit Core or a product package
|
||||
to import their implementation.
|
||||
|
||||
The first versioned `product_surfaces` slice is now implemented in Core. It
|
||||
The versioned `product_surfaces` slice is implemented in Core. It
|
||||
binds a stable product identity and entry path to one or more owner routes,
|
||||
View surfaces, presentations, capabilities, search sources, help contexts and
|
||||
documentation topics. It also carries standard unavailable/degraded
|
||||
@@ -80,8 +80,22 @@ explanations and migration aliases. Mail and Postbox contribute the first
|
||||
shared identity, `communication.messages`: `/messages` and the migration alias
|
||||
`/inbox` select the first currently authorized, View-visible owner while the
|
||||
underlying `/mail` and `/postbox` deep links, custody and permissions remain
|
||||
unchanged. Alias resolution emits a bounded client telemetry event before the
|
||||
redirect.
|
||||
unchanged. Tasks, Calendar and Files contribute the corresponding single-owner
|
||||
identities:
|
||||
|
||||
| Product identity | Stable destination | Compatible owner route |
|
||||
| --- | --- | --- |
|
||||
| Work | `/work` | `/tasks` |
|
||||
| Calendar | `/agenda` | `/calendar` |
|
||||
| Messages | `/messages` (`/inbox` alias) | `/mail`, `/postbox` |
|
||||
| Files | `/documents` | `/files` |
|
||||
|
||||
Core replaces those owner entries in the ordinary rail with the stable product
|
||||
destinations. A collapsed **All available tools** catalogue retains every
|
||||
authorized technical owner route independently of View focus; unauthorized
|
||||
entries are never disclosed. The original deep links remain valid, and all
|
||||
contributing owner paths keep the corresponding product entry active. Alias
|
||||
resolution emits a bounded client telemetry event before the redirect.
|
||||
|
||||
Core's `ProductAvailabilityState` is the shared presentation primitive for
|
||||
authorization, Policy, configuration, disabled, missing-capability, offline and
|
||||
@@ -105,10 +119,11 @@ People and Responsibility. They are configurable system/tenant defaults and
|
||||
Views projections, not hard-coded repository groups. Empty areas disappear;
|
||||
single-destination areas may link directly; familiar tools may remain pinned.
|
||||
|
||||
The complete permission-derived module rail remains available as **All
|
||||
available tools**. Its ability to scroll is useful and is not itself the
|
||||
product defect. The defect is requiring people to infer a task or outcome from
|
||||
repository topology.
|
||||
The complete permission-derived module rail is available as the collapsed
|
||||
**All available tools** escape. It is deliberately independent of the active
|
||||
View while still enforcing authorization. Its ability to scroll is useful and
|
||||
is not itself the product defect. The defect is requiring people to infer a
|
||||
task or outcome from repository topology.
|
||||
|
||||
Task-local Work, Calendar, Messages and Files tools may be contributed to the
|
||||
optional `govoplan-quick-access` rail. Messages composes Mail, Postbox and
|
||||
@@ -125,6 +140,12 @@ sections, commands, widgets, and fields. A view must not grant a permission or
|
||||
change data semantics. Policy can force, allow, or prohibit a surface at system,
|
||||
tenant, group, or user scope.
|
||||
|
||||
Core browser conformance exercises the German Anwohnerparkausweis reference
|
||||
context with Work, Calendar, Messages and Files entries, verifies that package
|
||||
owner labels are absent from the primary rail, expands the technical catalogue,
|
||||
and runs WCAG 2 A/AA checks over the result. Unit permutations cover two-owner,
|
||||
one-owner, unauthorized-owner and focused-View compositions.
|
||||
|
||||
## Error And Provenance Language
|
||||
|
||||
Normal errors answer:
|
||||
|
||||
@@ -39,16 +39,24 @@ The first production-shaped slice is implemented:
|
||||
order and optional labels. Scoped Views therefore configure product
|
||||
presentation for system, tenant, group, user and Workflow contexts;
|
||||
- the expanded left rail groups classified destinations while retaining
|
||||
Dashboard and every authorized unclassified destination under More tools.
|
||||
Dashboard and every authorized unclassified destination under More tools;
|
||||
- Core promotes Work (`/work`), Calendar (`/agenda`), Messages (`/messages`)
|
||||
and Files (`/documents`) into stable primary destinations and collapses the
|
||||
compatible owner routes under **All available tools**;
|
||||
- **All available tools** is permission-derived but independent of the active
|
||||
View, providing a deliberate escape without granting access or discarding
|
||||
the original `/tasks`, `/calendar`, `/mail`, `/postbox` and `/files` links.
|
||||
|
||||
The baseline classification is now manifest-declared for every ordinary
|
||||
user-facing module and enforced by the workspace manifest check. A separately
|
||||
The baseline classification and the four initial stable destinations are now
|
||||
manifest-declared. The area classification covers every ordinary user-facing
|
||||
module and is enforced by the workspace manifest check. A separately
|
||||
versioned launch-context contract carries bounded active-object, acting,
|
||||
temporal, View and return references into full-page Quick Access fallbacks;
|
||||
Cases publishes the first active-object reference. The remaining rollout is to
|
||||
add useful bounded tools and active-object publishers only where a maintained
|
||||
journey benefits, and to extend browser evidence to a pinned reference
|
||||
composition. Authorized global and technical routes remain visible through
|
||||
journey benefits. The pinned German Anwohnerparkausweis browser composition
|
||||
verifies stable product labels, technical escape, keyboard access and WCAG
|
||||
conformance. Authorized global and technical routes remain visible through
|
||||
their dedicated shell entry or **All available tools**.
|
||||
|
||||
## Quick Access Boundary
|
||||
@@ -166,9 +174,10 @@ areas, and users may personalize them within Policy ceilings. An empty area is
|
||||
omitted. An area with one destination may open it directly. A multi-destination
|
||||
area provides a useful work/recent/action surface rather than another menu.
|
||||
|
||||
Familiar product nouns such as Calendar, Mail or Files may remain directly
|
||||
pinned. The objective is not to hide every module name; it is to prevent
|
||||
repository topology from determining a person's workflow.
|
||||
Familiar product nouns such as Calendar or Files remain direct product
|
||||
destinations. The objective is not to hide every implementation name from
|
||||
administrators; it is to prevent repository topology from determining a
|
||||
person's workflow.
|
||||
|
||||
The initial module classification is deliberately outcome-oriented:
|
||||
|
||||
|
||||
@@ -6,8 +6,8 @@ Every other active module requires a reviewed explanation of why it owns no
|
||||
persistent subject-data store. Adding a migration invalidates that explanation.
|
||||
|
||||
- Active modules: 72
|
||||
- Registered and documented DSAR providers: 48
|
||||
- Reviewed no-store rationales: 24
|
||||
- Registered and documented DSAR providers: 49
|
||||
- Reviewed no-store rationales: 23
|
||||
- Unexplained coverage gaps: 0
|
||||
|
||||
| Module | Repository | Persistence | Coverage | Rationale |
|
||||
@@ -75,7 +75,7 @@ persistent subject-data store. Adding a migration invalidates that explanation.
|
||||
| `soap` | `govoplan-soap` | No module migration | Reviewed no-store rationale | Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store. |
|
||||
| `tasks` | `govoplan-tasks` | Migration-owned | Provider | Provider `privacy.dsar.tasks` is registered and documented. |
|
||||
| `templates` | `govoplan-templates` | Migration-owned | Provider | Provider `privacy.dsar.templates` is registered and documented. |
|
||||
| `tenancy` | `govoplan-tenancy` | No module migration | Reviewed no-store rationale | Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data. |
|
||||
| `tenancy` | `govoplan-tenancy` | Migration-owned | Provider | Provider `privacy.dsar.tenancy` is registered and documented. |
|
||||
| `tickets` | `govoplan-tickets` | Migration-owned | Provider | Provider `privacy.dsar.tickets` is registered and documented. |
|
||||
| `transparency` | `govoplan-transparency` | No module migration | Reviewed no-store rationale | Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store. |
|
||||
| `views` | `govoplan-views` | Migration-owned | Provider | Provider `privacy.dsar.views` is registered and documented. |
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
# Full registry candidates / Vollständige Registry-Kandidaten
|
||||
|
||||
## Operator workflow (EN)
|
||||
|
||||
The canonical `release-catalog.py full-registry` command takes `--package-set`,
|
||||
`--package-lock`, `--wheelhouse`, `--webui-packages`, `--output-dir`, and a
|
||||
configured `--catalog-signing-key`. Generate the package set with
|
||||
`generate-release-package-set.py --profile full` and download its exact artifacts
|
||||
with `resolve-package-artifacts.py`; do not substitute locally rebuilt wheels.
|
||||
The candidate compares the package set with the exact developer meta-package
|
||||
pins, checks archive bytes and package metadata against the lock, and synthesizes
|
||||
every entry from its immutable tagged manifest. Native package publication and
|
||||
its CI authority remain trusted: this verifies the published artifact identity,
|
||||
not independent reproducible-build equivalence to source.
|
||||
|
||||
Pass `--selected-repository` once for each newly released repository, including
|
||||
Core when it changes. These selected units must have clean, version-aligned
|
||||
named branches whose HEAD equals the annotated local and remote release tag.
|
||||
Other full-profile packages retain their exact older annotated tags; a later
|
||||
workflow-only commit on `main` does not relabel those package contents or force
|
||||
a version bump. Every source fetch/push endpoint must match the registered
|
||||
origin, and all entries bind their source commit and annotated tag object.
|
||||
Git replacement objects, caller Git configuration, and executable-path
|
||||
redirection cannot substitute another tagged manifest tree.
|
||||
|
||||
The fixed existing website catalog and keyring are authenticated before signing.
|
||||
An older catalog without a signed keyring hash can be migrated only through this
|
||||
complete rebuild, only when its signature verifies and its entire keyring
|
||||
exactly matches the configured known signers. No old entries or artifact hashes
|
||||
are reused. The new catalog signs the exact unchanged website keyring hash;
|
||||
key rotation remains a separate reviewed operation. Selective candidates still
|
||||
reject unpinned base keyrings. New candidate directories are private and
|
||||
exclusive: a retry must choose a new directory, not overwrite a reviewed one.
|
||||
|
||||
Run these commands on the trusted host, with an operator-private source workspace
|
||||
and artifact directory. `RELEASE_PYTHON` must select its private environment and
|
||||
`RELEASE_NPM` an absolute npm executable with a trusted sibling Node 22 binary.
|
||||
In Flatpak, execute host commands through `flatpak-spawn --host`; sandbox and
|
||||
host UID mappings are not interchangeable. Do not weaken trust gates or change
|
||||
system-wide permissions.
|
||||
|
||||
```sh
|
||||
# These paths identify previously prepared private operator resources.
|
||||
RELEASE_WORKSPACE=/path/to/private/workspace
|
||||
RELEASE_PYTHON="$RELEASE_WORKSPACE/govoplan/.host-venv/bin/python"
|
||||
RELEASE_NPM=/path/to/private/node22/bin/npm
|
||||
ARTIFACT_ROOT=/path/to/private/artifacts
|
||||
RELEASE_CANDIDATE=/path/to/private/new-candidate
|
||||
RELEASE_VERSION=0.1.45
|
||||
RELEASE_TOOLS="$RELEASE_WORKSPACE/govoplan/tools/release"
|
||||
|
||||
umask 077
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/generate-release-package-set.py" \
|
||||
--version "$RELEASE_VERSION" --profile full \
|
||||
--workspace "$RELEASE_WORKSPACE" --output "$ARTIFACT_ROOT/packages.json"
|
||||
PATH="$(dirname "$RELEASE_NPM"):/usr/bin:/bin" \
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/resolve-package-artifacts.py" \
|
||||
--package-set "$ARTIFACT_ROOT/packages.json" \
|
||||
--wheelhouse "$ARTIFACT_ROOT/wheels" \
|
||||
--webui-packages "$ARTIFACT_ROOT/webui" \
|
||||
--lock-output "$ARTIFACT_ROOT/artifacts.lock.json" \
|
||||
--python "$RELEASE_PYTHON" --npm "$RELEASE_NPM"
|
||||
|
||||
# Repeat --selected-repository for EVERY newly released unit, not just Core.
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" full-registry \
|
||||
--workspace-root "$RELEASE_WORKSPACE" \
|
||||
--package-set "$ARTIFACT_ROOT/packages.json" \
|
||||
--package-lock "$ARTIFACT_ROOT/artifacts.lock.json" \
|
||||
--wheelhouse "$ARTIFACT_ROOT/wheels" --webui-packages "$ARTIFACT_ROOT/webui" \
|
||||
--output-dir "$RELEASE_CANDIDATE" --selected-repository govoplan-core \
|
||||
--catalog-signing-key known-key=/path/to/private/known-key.pem --json
|
||||
|
||||
"$RELEASE_PYTHON" "$RELEASE_TOOLS/release-catalog.py" publish-candidate \
|
||||
--workspace-root "$RELEASE_WORKSPACE" --candidate-dir "$RELEASE_CANDIDATE" \
|
||||
--channel stable --npm "$RELEASE_NPM" --build-web \
|
||||
--commit --tag --push --tag-name "catalog-v$RELEASE_VERSION" --json
|
||||
```
|
||||
|
||||
The last command is a strict non-mutating preview because `--apply` is absent.
|
||||
Review its output, then repeat it with `--apply` to publish. The website's locked
|
||||
build dependencies must already be installed before `--build-web`. The publisher
|
||||
sanitizes the build and Git environments and pushes the verified immutable
|
||||
website commit/tag. Source tag publication and registry package availability
|
||||
must be complete before candidate generation.
|
||||
|
||||
Source tags, registry packages, and a signed module catalog do not imply that a
|
||||
new runtime distribution exists. While runtime images are held, leave the Meta
|
||||
Gitea runtime Release held too: a normal source-only Release can replace Gitea's
|
||||
`releases/latest` discovery result despite having no deployment assets. The
|
||||
deployer still requires an explicit signed manifest, digest, and trusted
|
||||
keyring; it does not deploy a tag or module catalog directly.
|
||||
|
||||
## Betriebsablauf (DE)
|
||||
|
||||
`release-catalog.py full-registry` übernimmt den vollständigen Paketbestand,
|
||||
die Registry-Sperrdatei, das Wheel-Verzeichnis, die WebUI-Archive und den
|
||||
konfigurierten Signaturschlüssel. Zuerst mit
|
||||
`generate-release-package-set.py --profile full` die exakten Meta-Paketversionen
|
||||
ermitteln und mit `resolve-package-artifacts.py` die veröffentlichten Artefakte
|
||||
herunterladen. Lokal neu gebaute Wheels sind kein Ersatz. Der Kandidat prüft
|
||||
Paketidentitäten, Dateigrößen und Hashes und erzeugt alle Einträge aus den
|
||||
unveränderlichen getaggten Manifesten. Die Registry und ihre veröffentlichende
|
||||
CI bleiben eine Vertrauensgrundlage; dies ist kein unabhängiger Nachweis eines
|
||||
reproduzierbaren Builds aus dem Quellcode.
|
||||
|
||||
Jedes neu veröffentlichte Repository wird mit `--selected-repository`
|
||||
angegeben. Nur diese Auswahl muss mit dem sauberen, versionsgleichen HEAD eines
|
||||
benannten Branches und dem annotierten lokalen und entfernten Tag übereinstimmen.
|
||||
Unveränderte Pakete behalten ihren ursprünglichen Tag, auch wenn auf `main`
|
||||
bereits eine spätere Workflow-Korrektur liegt. Alle Quelladressen müssen dem
|
||||
registrierten Ursprung entsprechen; Commit und annotiertes Tag-Objekt werden
|
||||
für jeden Eintrag gebunden. Git-Ersetzungsobjekte oder fremde Git-Konfiguration
|
||||
können dabei keinen anderen Manifestbaum unterschieben.
|
||||
|
||||
Vor dem Signieren werden der bestehende Website-Katalog und sein Schlüsselbund
|
||||
geprüft. Ein alter Katalog ohne signierten Schlüsselbund-Hash darf ausschließlich
|
||||
durch diesen vollständigen Neuaufbau migriert werden: Seine Signatur muss gültig
|
||||
sein und der gesamte Schlüsselbund exakt den konfigurierten bekannten Signierern
|
||||
entsprechen. Alte Einträge oder Artefakt-Hashes werden nicht übernommen. Der neue
|
||||
Katalog bindet den unveränderten Schlüsselbund-Hash; ein Schlüsselwechsel bleibt
|
||||
ein eigener geprüfter Vorgang. Selektive Kandidaten verlangen weiterhin einen
|
||||
bereits gebundenen Schlüsselbund. Kandidaten werden nur in neuen privaten
|
||||
Verzeichnissen erzeugt und niemals überschrieben.
|
||||
|
||||
Das obige Befehlsbeispiel wird auf dem vertrauenswürdigen Host ausgeführt. Dafür
|
||||
die private Python-Umgebung und einen absoluten `--npm`-Pfad zu Node 22 verwenden;
|
||||
unter Flatpak die Host-Werkzeuge über `flatpak-spawn --host` aufrufen. Die
|
||||
gesperrten Website-Build-Abhängigkeiten vorher installieren. Keine
|
||||
Vertrauensprüfung umgehen und keine globalen Rechte ändern. Die Artefaktordner
|
||||
müssen privat und bei der Auflösung leer sein. Vor der Kandidatenerzeugung
|
||||
müssen Quell-Tags und Registry-Pakete vollständig veröffentlicht sein.
|
||||
|
||||
Die Veröffentlichung zunächst mit `publish-candidate --commit --tag --push
|
||||
--build-web` ohne `--apply` prüfen und erst nach Prüfung mit `--apply` ausführen.
|
||||
Solange Laufzeit-Images zurückgestellt sind, bleibt auch das Meta-Gitea-Runtime-
|
||||
Release zurückgestellt: Ein reines Quellcode-Release könnte sonst als neuestes
|
||||
Release erscheinen. Eine Installation benötigt weiterhin ein signiertes
|
||||
Laufzeitmanifest, dessen Digest und einen explizit vertrauenswürdigen Schlüsselbund.
|
||||
@@ -100,6 +100,7 @@ The private installation directory contains:
|
||||
| `plan.json` | Latest desired-state diff and readiness findings |
|
||||
| `receipt.json` | Last successfully applied immutable identities |
|
||||
| `infrastructure-capabilities.json` | Deterministic non-secret capability states, endpoint metadata, secret references, consumers, and resumable post-install tasks |
|
||||
| `infrastructure-dependency-inventory.json` | Owner-only, short-lived Ops evidence of actual module-owned configuration and data that depend on infrastructure capabilities |
|
||||
| `distribution-manifest.json` | Canonical signed runtime/image selection adopted by the installer |
|
||||
| `distribution-keyring.json` | Explicitly installed public trust anchor for runtime releases |
|
||||
| `backup-evidence.json` | Signed provider-neutral coordinated backup and isolated-restore receipt |
|
||||
@@ -123,6 +124,16 @@ environment or initiating an implicit object migration. Invalid receipts fail
|
||||
closed, while a deployment without a mounted receipt continues to run but
|
||||
cannot apply receipt-bound configuration fragments.
|
||||
|
||||
Enabled modules may also register a Core infrastructure-dependency provider.
|
||||
The authorized Ops endpoint aggregates those providers without importing their
|
||||
tables. Mail reports persisted SMTP endpoints, credential-binding counts and
|
||||
legacy profiles; Files reports its runtime storage binding plus persisted blob
|
||||
counts and byte totals grouped by backend. Ops reports the active PostgreSQL,
|
||||
Redis coordination, ingress, and load-balancing runtime bindings. Provider output contains stable
|
||||
references, bounded numeric metrics and required migration actions, never
|
||||
credentials, endpoint secrets, tenant identifiers or file keys. A provider
|
||||
failure makes the entire inventory incomplete.
|
||||
|
||||
Build the same dependency-free tool as one downloadable artifact:
|
||||
|
||||
```sh
|
||||
@@ -424,10 +435,16 @@ infrastructure capability projections.
|
||||
|
||||
- Adding a managed component creates its service and persistent volume.
|
||||
- Removing a component removes its service container on apply.
|
||||
- Replacing or removing a capability adds a review action that names the prior
|
||||
and desired state/source plus declared module consumers. This does not claim
|
||||
that the deployer can inspect module-owned database configuration; the
|
||||
operator must review that inventory before apply.
|
||||
- Reconfiguring, replacing or removing a capability adds a review action that
|
||||
names the prior and desired state/source, declared consumers, actual
|
||||
provider-reported dependency records and each required migration action.
|
||||
- The deployer blocks that change when provider inventory is missing,
|
||||
incomplete, more than five minutes old, from another installation, timestamped
|
||||
in the future, or does not cover every impacted capability. It never treats
|
||||
installer-declared consumers as proof that persisted module state is absent.
|
||||
- The inventory reports impact; it does not migrate or delete module-owned
|
||||
configuration or data. Complete the reported preparation and collect again
|
||||
immediately before apply.
|
||||
- Volumes are retained by default; deleting data requires a separate,
|
||||
deliberately destructive workflow.
|
||||
- Existing generated credentials are retained unless an explicit future rotate
|
||||
@@ -455,6 +472,29 @@ dedicated ConfigMap and read-only file mount. Ops validates the bounded schema
|
||||
before displaying configured, externally supplied, available-unconfigured, or
|
||||
unavailable states and any pending post-install tasks.
|
||||
|
||||
Collect current dependency evidence with an API key whose principal has one of
|
||||
the Ops read scopes:
|
||||
|
||||
```sh
|
||||
export GOVOPLAN_OPS_API_KEY='<short-lived operator API key>'
|
||||
python3 govoplan-deploy.pyz collect-infrastructure-inventory \
|
||||
--directory /srv/govoplan/example
|
||||
python3 govoplan-deploy.pyz doctor \
|
||||
--directory /srv/govoplan/example
|
||||
python3 govoplan-deploy.pyz apply \
|
||||
--directory /srv/govoplan/example
|
||||
unset GOVOPLAN_OPS_API_KEY
|
||||
```
|
||||
|
||||
The command defaults to
|
||||
`<public-url>/api/v1/ops/infrastructure/dependencies`; `--ops-url` may select an
|
||||
explicit HTTPS endpoint (plain HTTP is accepted only on loopback). `apply`
|
||||
refreshes the inventory automatically when `GOVOPLAN_OPS_API_KEY` is present.
|
||||
Otherwise an already collected, current inventory may be used. The API key is
|
||||
sent only as `X-API-Key`, is never written to the bundle, and the inventory file
|
||||
is owner-readable only. Because it contains operational references and counts,
|
||||
handle it as private evidence even though it contains no secret material.
|
||||
|
||||
Every apply operation is journalled before image pulls or runtime mutation. A
|
||||
failure before migration may restore a verified previous bundle. Once migration
|
||||
starts, recovery is forward-only unless an independently verified database
|
||||
|
||||
@@ -86,6 +86,16 @@ contract. The coordinated release synchronizes `peerDependencies` and
|
||||
tag, then synchronizes each lockfile root from the final package metadata. A
|
||||
distinct root package remains independent.
|
||||
|
||||
Every module referenced by Core's Git-based `package.release.json` must expose
|
||||
its WebUI identity at the repository root, including matching peer requirements
|
||||
and `webui/`-prefixed entry exports (also CSS subpaths). npm resolves Git
|
||||
dependencies from the repository root, while the native-package workflow packs
|
||||
`webui/`; success in one path does not verify the other. Run
|
||||
`python tools/checks/check-webui-package-facades.py` after changing either
|
||||
manifest or the release composition. The focused gate also runs this check.
|
||||
Adding or correcting a facade in an already published repository requires a
|
||||
new patch tag; never repair an existing immutable tag in place.
|
||||
|
||||
It builds one wheel and, where applicable, one npm tarball. The workflow records
|
||||
the source tag, source commit, filename, size, and SHA-256 in
|
||||
`package-artifacts.json` before publishing. Gitea rejects a second upload of the
|
||||
@@ -179,8 +189,13 @@ than invoking `pip`, `npm`, or Git on the target host.
|
||||
|
||||
## Public module directory
|
||||
|
||||
`tools/release/publish-release-catalog.sh` resolves the selected package set and
|
||||
registry lock before it creates a catalog. Catalog entries are synthesized from
|
||||
For an operator-reviewed full publication, use
|
||||
`tools/release/release-catalog.py full-registry` followed by the same tool's
|
||||
`publish-candidate` command. Resolve the package set and registry lock first;
|
||||
the older direct-write shell wrapper is not the strict candidate publication
|
||||
path. See [Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
|
||||
for the private host runtime, exact artifact checks, and legacy keyring transition.
|
||||
Catalog entries are synthesized from
|
||||
the exact tagged module manifests, never from a hand-maintained module list or
|
||||
the current workspace. Each entry binds its Python wheel and optional WebUI
|
||||
tarball to the registry URL, filename, size, SHA-256, package identity, source
|
||||
|
||||
@@ -58,6 +58,45 @@ checkouts remain usable for read-only planning, but every durable executor
|
||||
fails closed there; clone the registered origins into a private workspace
|
||||
before releasing.
|
||||
|
||||
For a host with a confirmed IPv6 connection timeout, set
|
||||
`GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` only for the release-tool invocation.
|
||||
When unset, the original SSH command is preserved, including the trusted
|
||||
operator's per-host `AddressFamily` configuration (normally `any`). Explicit
|
||||
values accepted by the shared source/tag Git helper are exactly `any`, `inet`
|
||||
(IPv4 only), and `inet6` (IPv6 only). Empty, misspelled, whitespace-padded, or
|
||||
injected values fail before Git starts. The selector only adds the corresponding
|
||||
fixed SSH `AddressFamily` option: it does not change DNS, host-key verification,
|
||||
the registered remote, authentication, `BatchMode=yes`, or `ConnectTimeout=8`.
|
||||
Arbitrary `GIT_SSH_COMMAND` overrides remain ignored. For example, start a
|
||||
single local console invocation with:
|
||||
|
||||
```sh
|
||||
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet \
|
||||
./.venv/bin/python tools/release/release-console.py
|
||||
```
|
||||
|
||||
The same process-scoped setting applies to canonical source/tag readbacks and
|
||||
registry-candidate source verification. Under Flatpak, pass it explicitly to
|
||||
the host invocation with `flatpak-spawn --host /usr/bin/env
|
||||
GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet ...`. It is not a global SSH setting
|
||||
and does not affect the website publisher's separate transport sanitizer, npm,
|
||||
or HTTP downloads. An IPv4-only setting cannot reach IPv6-only hosts; omit it
|
||||
or use `any` when the diagnosed restriction no longer applies.
|
||||
|
||||
Deutsch: Bei einem bestätigten IPv6-Verbindungs-Timeout kann für genau einen
|
||||
Release-Werkzeugaufruf `GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY=inet` gesetzt werden.
|
||||
Ohne diese Variable bleibt der bisherige SSH-Befehl einschließlich der
|
||||
vertrauenswürdigen Host-Konfiguration unverändert (normalerweise `any`).
|
||||
Explizit zulässig sind ausschließlich `any`, `inet` (nur IPv4) und `inet6`
|
||||
(nur IPv6). Andere oder leere Werte werden vor dem Git-Aufruf abgewiesen.
|
||||
DNS, Hostschlüsselprüfung, registrierte Quelladresse, Authentifizierung und
|
||||
Zeitlimit bleiben unverändert; frei vorgegebene SSH-Befehle bleiben gesperrt.
|
||||
Unter Flatpak die Variable ausdrücklich an den Host-Aufruf übergeben. Die
|
||||
Auswahl gilt für den gemeinsamen Git-Helfer der Quell-/Tag-Prüfungen, nicht
|
||||
für den separaten Website-Publisher, npm oder HTTP-Downloads. Sie ändert keine
|
||||
globale Konfiguration. Nach Behebung des Netzwerkproblems die Variable
|
||||
weglassen oder auf `any` setzen; IPv4-only erreicht keine IPv6-only-Ziele.
|
||||
|
||||
The runtime itself is part of the authority boundary. Durable run creation
|
||||
verifies the meta checkout, release/check tooling, repository registry, Python
|
||||
environment, loaded `govoplan_core` and cryptography packages, and Git/SSH
|
||||
@@ -221,6 +260,9 @@ Repository capabilities are frozen into each plan unit (`python-package`,
|
||||
`core-release-bundle`, and the universal `git-source`) and determine which
|
||||
steps appear. Internally aligned version changes are rendered deterministically
|
||||
from recognized TOML, JSON, lockfile, manifest, and package declarations.
|
||||
The manifest may use a literal version or a top-level literal `MODULE_VERSION`;
|
||||
the latter is updated without rewriting independently versioned interfaces.
|
||||
Computed or missing version declarations fail before any metadata is written.
|
||||
Pre-existing dirty worktrees remain visible but have no commit executor; the
|
||||
console never absorbs unrelated operator changes.
|
||||
|
||||
@@ -232,6 +274,17 @@ runs a receipt-bound alignment gate before exposing any atomic branch/tag push.
|
||||
A failed step stops later steps while preserving prior receipts for explicit
|
||||
retry or reconciliation.
|
||||
|
||||
Local module candidate creation deliberately does not require those candidates
|
||||
to be resolved already in Core's release lock: their annotated tags are inputs
|
||||
to the next lock-generation step. The internal tag helper applies this ordering
|
||||
only when no Core repository is selected and remote publication is disabled.
|
||||
Module version/lock consistency, manifest validity, clean/non-behind worktrees,
|
||||
and local/remote tag immutability checks still apply. Core candidate tagging
|
||||
continues to validate its own complete bundle, and every remote-publication
|
||||
preview and execution requires the selected modules to match Core's release
|
||||
input and resolved lock. A local candidate is therefore not publication
|
||||
approval; a stale Core lock blocks publication without changing remote refs.
|
||||
|
||||
The browser likewise retains the request identifier for an uncertain
|
||||
resume/retry/reconciliation response and replays it after reload. A successful
|
||||
replay selects the returned run state. Transport and server failures retain the
|
||||
@@ -503,6 +556,15 @@ tree and requires byte-for-byte equality with those validated objects. Tags and
|
||||
remote branch updates then reference that exact commit SHA rather than the
|
||||
mutable worktree `HEAD`.
|
||||
|
||||
For a full registry-backed release, first build a fresh private candidate using
|
||||
`release-catalog.py full-registry`. Pass `--selected-repository` for newly
|
||||
released HEAD-bound units, not every unchanged package in the full profile.
|
||||
The command independently checks all full-profile registry bytes and annotated
|
||||
tag provenance, then feeds this same strict `publish-candidate` transaction.
|
||||
It does not create Gitea runtime Releases or dispatch image builds. See
|
||||
[Full registry candidates / Vollständige Registry-Kandidaten](FULL_REGISTRY_CANDIDATES.md)
|
||||
for the complete EN/DE workflow and the narrowly scoped legacy keyring transition.
|
||||
|
||||
Published channels are expected below the public catalog base URL:
|
||||
|
||||
- `https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`
|
||||
@@ -515,8 +577,9 @@ updated catalog, and keep the published keyring healthy.
|
||||
|
||||
When a selected module exposes a WebUI package, its requested version must also
|
||||
match Core's `webui/package.release.json` input and the resolved
|
||||
`package-lock.release.json` entry. The source-tag preflight, selective plan, and
|
||||
catalog-candidate writer all enforce this composition boundary. Pins for modules
|
||||
`package-lock.release.json` entry. The source-publication preflight, selective
|
||||
plan, and catalog-candidate writer all enforce this composition boundary;
|
||||
module-only local candidate tags use the staged order described above. Pins for modules
|
||||
that are not part of the selective release remain unchanged.
|
||||
|
||||
Release integration also enforces repository and composition version alignment
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
# GovOPlaN 0.1.45 — usability, reliability and security hardening
|
||||
|
||||
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
|
||||
The exact independently versioned composition is recorded in
|
||||
`packages/govoplan-meta/pyproject.toml`; unchanged modules retain their versions.
|
||||
This source release does not by itself establish a deployed or independently
|
||||
approved production environment. Package, signed catalog and runtime publication
|
||||
results are recorded separately in the coordination issue.
|
||||
|
||||
## Runtime publication hold
|
||||
|
||||
The [runtime image audit](../security/RUNTIME_IMAGE_AUDIT_2026-09-08.md) completed
|
||||
eleven registry-only amd64 scans, but found unresolved vulnerabilities and
|
||||
inventory gaps. Runtime publication remains held separately from this source
|
||||
release. Patch-only image updates are insufficient; maintained minor-line
|
||||
changes, narrowly evidenced finding decisions, arm64/final-layer scans and
|
||||
deployment checks remain necessary. No audited candidate was automatically
|
||||
adopted and no image was executed during those scans.
|
||||
The remaining gates are tracked in
|
||||
[GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
|
||||
|
||||
## Included changes
|
||||
|
||||
- Shared page/action placement, reusable navigation grouping/editing, table and
|
||||
dialog sizing, field alignment, multi-select filters and predictable tree
|
||||
selection. Files, Mail, Search, Notifications and domain pages use the same
|
||||
contracts, with browser regression coverage.
|
||||
- Campaign draft saving and independent Mail/ZIP-policy repair, persistent and
|
||||
bulk message review, clearer delivery eligibility, bounded configurable
|
||||
synchronous delivery, guarded workerless recovery, lightweight SMTP/IMAP
|
||||
progress, reused IMAP connections and recipient-complete reporting.
|
||||
- Files archive staging/reuse, unpacking previously uploaded archives, numeric
|
||||
progress and bounded traversal. Optional native archive acceleration retains
|
||||
the same validation rules; portable fallbacks remain available.
|
||||
- Mail credential references and IMAP folder-name decoding; help topics can be
|
||||
found by area and tags without expanding every occurrence of the same topic.
|
||||
- Authentication provenance/scope and browser-cache hardening, patched rich-text
|
||||
dependencies, spreadsheet/archive/template/Dataflow resource limits, batched
|
||||
Docs/Notifications queries and safe Reporting bind names. See the
|
||||
[security/performance review](../security/SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
|
||||
for measurements, test evidence and remaining limitations.
|
||||
- A deterministic governance-journey clock fixture, fresh-process Campaign
|
||||
import coverage, and a new Cases patch aligning its root npm facade with its
|
||||
Python/WebUI package. Historical published tags are not rewritten.
|
||||
- Git-root WebUI package facades are aligned with their owning packages, with
|
||||
a cross-composition parity check. Tasks is included in default module
|
||||
discovery; it remains subject to enabled modules and normal permissions.
|
||||
|
||||
## Upgrade and verification
|
||||
|
||||
Back up the database and file storage before upgrading. Apply the complete
|
||||
selected migration graph before starting the new API/workers. This release
|
||||
includes additive repair migrations `c58a2d7e9f10` (Core ownership history) and
|
||||
`d8f1b4e7a0c3` (Access external-function mappings), plus Campaign delivery-state
|
||||
migrations. Existing business evidence is retained; a schema downgrade is not
|
||||
a substitute for a reviewed backup/restore plan. Restart API and worker
|
||||
processes together after upgrading their matching packages.
|
||||
|
||||
Updated UI consumers require Core 0.1.45 where they use its new shared contracts.
|
||||
Tenant keys that previously relied on unintended system permissions/wildcards
|
||||
must be corrected; the release does not preserve that unsafe behavior. Extremely
|
||||
sparse spreadsheets, oversized generated output and excessive archive paths
|
||||
can now fail early with a diagnostic.
|
||||
|
||||
For archive staging across multiple hosts, provide shared POSIX storage with
|
||||
working locks or sticky routing. Background delivery still needs configured
|
||||
workers; increasing the synchronous limit does not create a worker or guarantee
|
||||
delivery after a process failure. An unknown SMTP outcome must be reconciled,
|
||||
not automatically resent.
|
||||
|
||||
After deployment, manually verify login/logout and least-privilege API keys,
|
||||
Campaign Settings and independent Mail/ZIP saves, archive upload/unpack,
|
||||
recipient-complete reports, and SMTP/IMAP progress with an explicitly approved
|
||||
test mailbox. No release verification sends real campaign mail automatically.
|
||||
|
||||
Hard process isolation, forced-password-change/recovery enforcement, bounded
|
||||
Xrechnung subprocess output and large-history pagination remain separate open
|
||||
issues. This release is not a claim that all security or performance debt is
|
||||
resolved. Production-image scans and multi-host evidence must refer to the
|
||||
actual signed runtime being deployed.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Runtime image candidate audit — 8 September 2026
|
||||
|
||||
Release coordination: [GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51).
|
||||
Canonical remediation: [GovOPlaN #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52).
|
||||
This follow-up to the [source security/performance review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
|
||||
records registry-only scans of nine proposed runtime dependencies and two
|
||||
same-minor patch candidates. **Runtime publication is held:** patch-only updates
|
||||
do not resolve the baseline. Source/package publication is a separate outcome.
|
||||
No images were executed, rebuilt, selected for CI, or published by this audit.
|
||||
|
||||
## Method and reproducible evidence
|
||||
|
||||
Official Trivy **0.74.0** was installed only in a private local task directory,
|
||||
without sudo or Docker access. Its Linux-64bit release archive matched both the
|
||||
official checksums file and GitHub release asset metadata:
|
||||
|
||||
- Archive SHA256: `2ae6fe3ee734b7fdf11335663e18c75ea12dccc76062f09f164a3b0f8be4371a`.
|
||||
- Checksums-file SHA256: `bc701c3c3ee8b9acbea2c23257e41381e3854888f51281616a6ba5dc96963821`.
|
||||
- Vulnerability database schema 2, updated `2026-09-07T19:06:01.154199452Z`,
|
||||
downloaded from `mirror.gcr.io/aquasec/trivy-db:2`.
|
||||
- Scan flags: `--image-src remote --platform linux/amd64 --scanners vuln
|
||||
--format json --no-progress --timeout 8m --max-image-size 2GB --exit-code 0`.
|
||||
Findings were counted from validated JSON; exit zero did not mean clean.
|
||||
- Existing Docker credentials were not read; no private keys or secrets were
|
||||
used. Checksums over official HTTPS metadata were verified, not independent
|
||||
Sigstore signatures. See the [official release](https://github.com/aquasecurity/trivy/releases/tag/v0.74.0)
|
||||
and [registry-only scan documentation](https://trivy.dev/docs/latest/target/container_image/).
|
||||
|
||||
Raw evidence is retained locally, not committed:
|
||||
`/home/zemion/.cache/govoplan-trivy-remote.yKZgjDOg/scan/`.
|
||||
It contains eleven `reports/*-amd64.json` reports/logs, scanner scripts,
|
||||
`patch-candidate-inspection.json`, exact successor registry indices, and
|
||||
`evidence-checksums.json`. Summary SHA256 values:
|
||||
|
||||
- `summary.json`: `f2785a731d637452ab9c0b1f5399772c0f8828a63ca83d5fa7496abdad1c757a`.
|
||||
- `patch-summary.json`: `b3fc6273fcdad98864040ccdf3477ecf379afd46e9f94444b1f2910f48c1d85b`.
|
||||
|
||||
All eleven executions succeeded without timeout/rate-limit failure. Initial
|
||||
summary fields distinguish `scan_execution_complete: true` from
|
||||
`coverage_complete: false`: Garage has no detectable package inventory.
|
||||
Checksums preserve evidence identity, not indefinite storage availability.
|
||||
|
||||
## Exact requested pins and results
|
||||
|
||||
All references below use `docker.io/`. Counts are package-vulnerability records,
|
||||
not distinct CVEs or confirmed exploitable application defects. A vulnerability
|
||||
can appear against several installed packages. Unfixed/unknown records remain.
|
||||
|
||||
| Image tag | Exact index SHA256 | Critical / High / Medium / Low / Unknown | Fixable C/H |
|
||||
| --- | --- | --- | ---: |
|
||||
| `library/python:3.12-slim-bookworm` | `782412e85d0f0984994c290652577d4018aff08145c85b262bb63dc0c7522254` | 5 / 55 / 102 / 103 / 5 | 0 |
|
||||
| `library/postgres:16-alpine` | `cf78e76683b9ca8c5733cbbdce6c9262b45b6767934dd0a95e671f9a0fc20685` | 1 / 30 / 28 / 14 / 1 | 31 |
|
||||
| `library/redis:7-alpine` | `ff02b58f971e7d7d156a1267e283fcbbeee91773b6aa36c49dac28ecfe28eadf` | 0 / 0 / 0 / 0 / 0 | 0 |
|
||||
| `nginxinc/nginx-unprivileged:1.29-alpine` | `0c79d56aee561a1d81c63f00eee5fb5fe29279560cdc55e91425133104c7fbe6` | 0 / 33 / 74 / 37 / 20 | 33 |
|
||||
| `library/haproxy:3.2.21-alpine` | `66e25cc9a8332635f4e897f7f4b1e5622c25f09f0ee23cddc6ce9bdb3a24772a` | 0 / 2 / 6 / 12 / 0 | 2 |
|
||||
| `library/caddy:2.10.2-alpine` | `4c6e91c6ed0e2fa03efd5b44747b625fec79bc9cd06ac5235a779726618e530d` | 7 / 75 / 67 / 37 / 4 | 82 |
|
||||
| `dxflrs/garage:v2.3.0` | `866bd13ed2038ba7e7190e840482bc27234c4afaf77be8cfa439ae088c1e4690` | **Unknown: no inventory** | — |
|
||||
| `greenmail/standalone:2.1.9` | `3ac5a83dd6727cf95e4d50e18907fb8ee7bbf5f67e8534714dee2fb1b5b2e1d4` | 0 / 0 / 116 / 35 / 0 | 0 |
|
||||
| `tonistiigi/binfmt:qemu-v10.2.3-68` | `400a4873b838d1b89194d982c45e5fb3cda4593fbfd7e08a02e76b03b21166f0` | 0 / 9 / 2 / 1 / 1 | 9 |
|
||||
|
||||
## Patch-only options and limits
|
||||
|
||||
Complete publisher tag listings were inspected for nginx 1.29, Caddy 2.10,
|
||||
HAProxy 3.2, GreenMail 2.1 and binfmt qemu10.2. Two newer candidates were
|
||||
scanned; their registry index bytes matched both registry and publisher digests,
|
||||
and contained amd64 and arm64 manifests:
|
||||
|
||||
- `library/haproxy:3.2.23-alpine@sha256:6343ce34a132a5dceaa24767d739df2bd519f8f7c1079ae39e4821334e8eb42e`:
|
||||
same Alpine 3.24.1, 24 detected OS packages, zero reported findings. This is
|
||||
a useful candidate, not a completed compatibility test or application audit.
|
||||
- `greenmail/standalone:2.1.13@sha256:3df66b7edd01c8a301343ca5e3601d8674760d4708655573560c24745e624fb2`:
|
||||
upstream changes Ubuntu 22.04 to Debian 13.6; **3 C / 80 H / 98 M / 85 L /
|
||||
5 unknown**, 30 fixable C/H records. Not selected as a no-base-change update.
|
||||
- nginx's newest matching Alpine patch is already 1.29.8 at the scanned pin;
|
||||
Caddy 2.10 remains 2.10.2; binfmt qemu10.2 remains 10.2.3-68. No newer matching
|
||||
publisher images were found. The current [official Caddy image catalogue](https://raw.githubusercontent.com/docker-library/official-images/master/library/caddy)
|
||||
uses 2.11.4; switching minor lines requires new scans and compatibility checks.
|
||||
|
||||
Priority remediation: Caddy's own seven HIGH records require fixes through
|
||||
2.11.4, with additional bundled Go/library fixes that must be re-scanned;
|
||||
nginx's packages include curl/libcurl fixes through 8.22.0-r0, OpenSSL 3.5.8-r0,
|
||||
c-ares 1.34.8-r0, expat 2.8.1-r0 and libuuid 2.41.6-r1. PostgreSQL's OS records
|
||||
require OpenSSL 3.5.8-r0 and libuuid 2.42.3-r1; its CRITICAL plus 21 HIGH Go
|
||||
records concern the **gosu helper**, not PostgreSQL server code. binfmt's nine
|
||||
HIGH records concern its Go 1.26.4 build, with fixes through 1.26.6. Package
|
||||
presence does not establish vulnerable-symbol reachability. No unscanned tag
|
||||
is claimed to meet every fix requirement.
|
||||
|
||||
## Python triage and coverage caveats
|
||||
|
||||
Python image metadata identifies CPython 3.12.14, but Trivy inventories only
|
||||
Debian packages and pip, **not CPython/stdlib**. All 60 C/H records concern
|
||||
Debian packages: 21 CVEs, 50 `affected` records, 9 `fix_deferred`, 1
|
||||
`will_not_fix`, without a recorded fixed Bookworm version. Five util-linux CVEs
|
||||
repeat across eight binary packages. These remain installed; they are not all
|
||||
removed build dependencies. Pip 25.0.1 separately has five MEDIUM/one LOW
|
||||
records, with fixes through 26.2.0; it is install tooling, and the API image uses
|
||||
an offline `--no-index` wheelhouse rather than an arbitrary package index.
|
||||
|
||||
Narrow triage examples, **not blanket exemptions**:
|
||||
|
||||
- Debian states [CVE-2023-45853](https://security-tracker.debian.org/tracker/CVE-2023-45853)
|
||||
does not affect the built Bookworm zlib binaries because vulnerable minizip
|
||||
code is not included. Other bundled minizip implementations are separate.
|
||||
- [CVE-2026-8376](https://security-tracker.debian.org/tracker/CVE-2026-8376)
|
||||
explicitly requires 32-bit Perl; this scan targets amd64.
|
||||
- [CVE-2025-7458](https://security-tracker.debian.org/tracker/CVE-2025-7458)
|
||||
requires crafted arbitrary SQLite SQL; the managed runtime uses PostgreSQL,
|
||||
but alternate SQLite use must be reviewed.
|
||||
- Perl's regex and Archive::Tar records need exact binary/module applicability
|
||||
checks; vendor-deferred status alone is not a finding dismissal.
|
||||
|
||||
Only amd64 was scanned. arm64, newly built GovOPlaN API/Web layers and optional
|
||||
dependency combinations remain unverified. Garage has no inventory; Redis,
|
||||
HAProxy and PostgreSQL source-built executables, CPython and QEMU static
|
||||
binaries need supplemental SBOM/source coverage. Zero detected OS findings is
|
||||
not zero application vulnerabilities. Trivy also lacks Alpine 3.24 EOL metadata
|
||||
and nginx CVE-2026-80256 detail; unknowns are retained. There were no runtime,
|
||||
exploitability, secret, misconfiguration, malware or signature-policy checks.
|
||||
|
||||
Before lifting the runtime hold: approve and test maintained image-line changes
|
||||
where necessary, fix or narrowly disposition findings with evidence, close
|
||||
inventory gaps, scan both architectures and final runtime layers, then run
|
||||
deployment/ingress smoke checks. Do not silently change base OS, use unpinned
|
||||
`latest`, rebuild third-party images, or accept all HIGH/CRITICAL findings.
|
||||
@@ -0,0 +1,176 @@
|
||||
# Security and performance review — 8 September 2026
|
||||
|
||||
Coordinated status: [Core #296](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/296).
|
||||
This records a workspace-wide automated scan, targeted manual boundary review,
|
||||
and a verified implementation pass. It is not a penetration test, an exhaustive
|
||||
line-by-line review, or a security certification. The audit was completed on
|
||||
local, unpublished changes, preserving existing worktree changes. Subsequent
|
||||
release preparation/publication is tracked in
|
||||
[GovOPlaN #51](https://git.add-ideas.de/GovOPlaN/govoplan/issues/51) and the
|
||||
[0.1.45 release notes](../releases/0.1.45.md).
|
||||
|
||||
## Implemented findings
|
||||
|
||||
| Area | Finding and change | Evidence / ownership |
|
||||
| --- | --- | --- |
|
||||
| Authentication — high | Preserve service-account provenance and current scope ceilings instead of recalculating them as ordinary membership permissions. Tenant API keys cannot retain canonical system permissions or unsafe wildcard grants. | Previously failing isolated regressions; [Access #21](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/21). |
|
||||
| Authentication — medium | Warm-cache API keys must follow the same explicit-header credential rules as cold authentication. A session cookie cannot turn an API key into a session credential. | Regression covering source-dependent authentication. |
|
||||
| Browser authority/cache — medium | Clear reusable data on auth changes and write settlement; fence late 200/304 writes and obsolete 401 side effects. Honor server no-store/no-cache and explicit fresh-read requests. Interactive login/logout remove retained automation keys that could shadow cookie-session identity. | 23 real-client regressions. Unchanged settings keep their object identity, preventing profile-fetch loops. Core `docs/API_CLIENT_CACHE_CONTRACT.md`; owning Access EN/DE session/field documentation. |
|
||||
| Spreadsheet resource exhaustion | Validate actual XLSX coordinates before openpyxl traversal; ignore misleading declared dimensions; count blank row gaps toward the existing limits. | [Connectors #18](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/18), 13 tests and 2 subtests. |
|
||||
| Template resource exhaustion | Enforce the existing 5 MiB output budget during substitution and item construction, including UTF-8, HTML escaping and separators. | [Templates #7](https://git.add-ideas.de/GovOPlaN/govoplan-templates/issues/7), full 20 tests; independent 3,000-case valid-output comparison. |
|
||||
| Archive resource exhaustion | Inspect regular TAR member limits before traversing payloads. Limit archive paths to 4,096 UTF-8 bytes / 128 components and count derived directories against entry limits. | [Files #46](https://git.add-ideas.de/GovOPlaN/govoplan-files/issues/46), 55 archive and 15 documentation tests. Extension-header decoding still needs stronger isolation. |
|
||||
| Dataflow resource exhaustion | Reject LPAD/RPAD target lengths above the existing 1,000,000-byte preview budget before fill evaluation/allocation. Preserve final serialized-byte checks. | [Dataflow #22](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/22), full 104 tests and 39 subtests; 7 new guard tests independently rerun. |
|
||||
| Docs performance / defense in depth | Batch revision reads per request, avoid loading pending draft bodies for readers, and validate tenant/entry/publication consistency while retaining owner/audience checks. | [Docs #22](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/22), full 39 tests. |
|
||||
| Notifications performance / defense in depth | Batch delivery-attempt loading while preserving recipient checks and rejecting inconsistent attempt references, including already-loaded relationships. | [Notifications #6](https://git.add-ideas.de/GovOPlaN/govoplan-notifications/issues/6), full 23 tests. |
|
||||
| Session-list performance | Apply active/expiry predicates and the existing 100-row cap in SQL, before loading session history. | Query-shape regression in Access. |
|
||||
| Reporting correctness | Use structural bind-name suffixes for recursive calculated measures, preserving valid dotted/hyphenated public keys and parameter uniqueness. | [Reporting #10](https://git.add-ideas.de/GovOPlaN/govoplan-reporting/issues/10), full 29 tests. |
|
||||
| Audit hygiene | Redact Gitleaks logs and machine reports on current, history and legacy scanner paths. | 13 audit-wrapper tests enforce the flag. |
|
||||
|
||||
All changed module workflows/limits have owning EN/DE DocumentationTopic updates.
|
||||
Independent review found no concrete regression in the backend changes.
|
||||
|
||||
## Measured performance changes
|
||||
|
||||
These are SQL-query counts in isolated 40-item fixtures, not production latency
|
||||
or throughput claims. Authorization is still evaluated for each request.
|
||||
|
||||
| Projection | Before | After |
|
||||
| --- | ---: | ---: |
|
||||
| Docs reader entries | 41 SELECTs | 2 SELECTs |
|
||||
| Docs editor entries | 81 SELECTs | 2 SELECTs |
|
||||
| Notification list with attempts | 41 SELECTs | 2 SELECTs |
|
||||
|
||||
The Docs 401-entry batching regression uses 3 SELECTs. Resource guards reject
|
||||
oversized work before the formerly expensive allocation/traversal. This does
|
||||
not make every legitimate upload or campaign faster. Honoring no-cache can
|
||||
increase server validation requests; ETags still avoid retransmitting unchanged
|
||||
bodies. That authorization/freshness trade-off is deliberate.
|
||||
|
||||
The original audit snapshot measured 517,380 initial JavaScript bytes and
|
||||
164,119 gzip bytes. Release preparation's pure-defaults split reduces this to
|
||||
516,730 initial bytes and 163,908 gzip bytes. Restoring the missing Tasks
|
||||
descriptor then measures 516,987 initial / 163,976 gzip bytes with all 46 module
|
||||
descriptors lazy, within the unchanged 524,288 / 164,128 caps. The gzip margin is still small; future
|
||||
startup work should reduce eager dependencies, not raise the cap automatically.
|
||||
The full 209-case browser suite passed before the split, followed by 13 focused
|
||||
browser checks after it. Radon recorded 238 rank-D-or-higher entries; complexity
|
||||
is a review-priority signal, not a performance measurement.
|
||||
|
||||
## Dependency remediation
|
||||
|
||||
Core's full npm audit went from 30 affected package entries to zero. Most initial
|
||||
entries were transitive effects of the same Tiptap advisory, not 30 independent
|
||||
application exploits. The website went from two affected entries to zero; both
|
||||
Mail lockfiles also report zero.
|
||||
|
||||
- Tiptap packages are aligned at 3.31.3, with direct minimum ranges raised to
|
||||
3.30.4 in both development and release manifests, with a parity regression.
|
||||
Added an actual installed-library prototype-attribute regression for
|
||||
the [maintainer's security advisory](https://github.com/ueberdosis/tiptap/security/advisories/GHSA-cp6q-959q-f8rh).
|
||||
- Core now resolves xmldom 0.9.12, browserslist 4.28.9 and nanoid 3.3.18.
|
||||
The website's affected browserslist/nanoid dependencies are patched too.
|
||||
- Development/audit requirements now require pip >=26.2; the local development
|
||||
environment uses 26.2.1. The installed audit originally flagged
|
||||
[CVE-2026-13346](https://github.com/advisories/GHSA-qwm4-qh6w-59xr), requiring an
|
||||
attacker-controlled package index. This is an installation-tool vulnerability,
|
||||
not evidence of an exposed application endpoint.
|
||||
|
||||
The final installed Python audit enumerated 188 distributions: 137 were
|
||||
auditable with zero known vulnerabilities, and 51 local distributions were not
|
||||
available in PyPI. Those skips are covered by source review, not by a claim of
|
||||
dependency-advisory coverage. Production images and every optional dependency
|
||||
combination were not independently resolved or scanned.
|
||||
|
||||
## Scan coverage and limitations
|
||||
|
||||
Evidence directory:
|
||||
`/mnt/DATA/tmp/govoplan-security-performance-20260908-gsk8jn/`.
|
||||
|
||||
The final `final-quick/manifest.json` captures 79 repositories, tool versions,
|
||||
start/end repository fingerprints, report checksums, 168 report artifacts and
|
||||
163 validated JSON/SARIF reports. It records an unchanged workspace, complete
|
||||
coverage for its four required scanners, no execution errors and no missing
|
||||
reports. It ran in report-only mode: exit zero does **not** mean zero warnings.
|
||||
|
||||
- Final production Bandit: 447,008 Python lines; 67 warnings (63 low, 4 medium),
|
||||
no high findings. Ruff security rules: 54 warnings. SQL-construction warnings
|
||||
were reviewed against identifier/operator validation and bound values in
|
||||
DuckDB/Reporting; no injection fix was warranted there. XML import warnings
|
||||
were checked: feed/BPMN input parsing uses defusedxml; stdlib imports support
|
||||
types/output construction. Operator-owned fenced-run argv is not a public
|
||||
arbitrary-command endpoint. Xrechnung output buffering remains a follow-up.
|
||||
Assertions and error-swallowing markers remain review/maintenance warnings,
|
||||
not proof that all such code is harmless.
|
||||
- Final local Semgrep rules: no findings. The broader OWASP-rule pass applied
|
||||
272 rules to 4,169 tracked targets. Its seven warnings recommended weakening
|
||||
owner-only 0700 permissions; they were rejected as false positives. One
|
||||
Calendar rule timeout was rerun with a 60-second budget: zero findings/errors.
|
||||
Bash and conformance TypeScript checks passed despite two scanner-specific
|
||||
parser limitations. Ignored/dependency/generated paths are not a complete
|
||||
line-by-line source audit.
|
||||
- Gitleaks: 79 Git histories plus 79 worktrees, 158 redacted reports, zero
|
||||
detected secrets. This does not establish that deployed credentials are safe
|
||||
or that formerly exposed credentials have been rotated.
|
||||
- Tool versions included Semgrep 1.176.1, Bandit 1.9.4, Ruff 0.15.21 and
|
||||
Gitleaks 8.30.1. The downloaded Gitleaks binary archive matched the official
|
||||
release SHA-256 before execution.
|
||||
- The containerized full-toolbox path could not access Docker's daemon. Its
|
||||
full-mode Trivy/misconfiguration and additional OSV scans were **not** run.
|
||||
A subsequent [registry-only runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md)
|
||||
successfully scanned nine pinned candidates and two same-minor successors
|
||||
for amd64 without Docker. It found unresolved vulnerabilities and inventory
|
||||
gaps; runtime publication is held. This does not complete full-toolbox,
|
||||
arm64, final-runtime-image or deployment coverage.
|
||||
|
||||
No live application probes, database changes, file operations, mail sends,
|
||||
IMAP appends, imports, notification delivery, deployments, commits or pushes
|
||||
were performed. Browser tests used isolated mocked fixtures. Package installs,
|
||||
builds and temporary audit-tool installation were local development operations.
|
||||
|
||||
## Verification and remaining work
|
||||
|
||||
- 209/209 browser conformance tests pass; production Core/website builds,
|
||||
conformance TypeScript, 24 Core client/dependency regressions, 4 real-client
|
||||
Files reload checks, and 72/72 manifest checks pass.
|
||||
- Access's full 91-test suite passed before the final documentation-only update;
|
||||
the final documentation suite passed all 4 tests. Other module counts appear
|
||||
above. The new authentication/resource tests include demonstrated pre-fix
|
||||
failures rather than only structural assertions.
|
||||
- The original focused workspace run stopped at the institutional
|
||||
governance/Portal fixture (`tests/test_institutional_governance_journey.py:223`,
|
||||
`IndexError`). Release preparation fixes its mixed clocks using the existing
|
||||
temporal context, retaining validity-boundary exclusions; 7 journey tests and
|
||||
ambient-year checks pass. Tracked in
|
||||
[Meta #50](https://git.add-ideas.de/GovOPlaN/govoplan/issues/50).
|
||||
- Campaign's apparent host-path issue was ruled out by existing tracked
|
||||
API/build/snapshot guards and 11 passing tests under normal initialization.
|
||||
Release preparation fixes the standalone import cycle through a deferred
|
||||
resolver import without changing validation rules. Fresh-process coverage,
|
||||
all 11 path tests and Campaign's full 611-test suite pass.
|
||||
|
||||
Next coordinated work:
|
||||
|
||||
1. [Hard resource isolation — Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
|
||||
regex CPU, aggregate allocation, TAR extension metadata, bounded workers and
|
||||
cancellation, followed by production-like concurrent load tests.
|
||||
2. [Forced password change/recovery — Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
|
||||
the current flag is advisory only. Do not enable enforcement without a usable
|
||||
local-password/recovery flow and external-provider rules.
|
||||
3. [Bound subprocess output — Xrechnung #2](https://git.add-ideas.de/GovOPlaN/govoplan-xrechnung/issues/2):
|
||||
enforce the existing 2 MiB limit while draining stdout/stderr, not afterwards.
|
||||
4. [Workflow revision batching/history projection — Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
|
||||
batch evidence lookups; separately define explicit history pagination and
|
||||
authorized-total semantics. Docs/notification history volumes also remain.
|
||||
5. Resolve the [runtime image audit](RUNTIME_IMAGE_AUDIT_2026-09-08.md) findings
|
||||
tracked in [Meta #52](https://git.add-ideas.de/GovOPlaN/govoplan/issues/52)
|
||||
and coverage gaps before lifting its publication hold; complete deployment
|
||||
audits, review exposed development credentials and worker quotas, and
|
||||
benchmark realistic tenant sizes/concurrency. The sanctions
|
||||
transport's fixed HTTPS/redirect allowlist is not a demonstrated arbitrary-URL
|
||||
issue, but migration to Core's pinned egress transport remains desirable.
|
||||
|
||||
Operational compatibility: tenant keys relying on accidental system/wildcard
|
||||
permissions must be corrected rather than weakening the guard. Extreme sparse
|
||||
spreadsheets, overly deep/long archive paths and oversized padding intermediates
|
||||
can now fail early with diagnostics. No stored documents or configurations were
|
||||
deleted or silently migrated.
|
||||
@@ -4,87 +4,87 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "govoplan"
|
||||
version = "0.1.41"
|
||||
version = "0.1.45"
|
||||
description = "Developer convenience package for a versioned GovOPlaN composition"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = { text = "AGPL-3.0-or-later" }
|
||||
dependencies = [
|
||||
"govoplan-core[server]==0.1.41",
|
||||
"govoplan-tenancy==0.1.20",
|
||||
"govoplan-organizations==0.1.20",
|
||||
"govoplan-identity==0.1.20",
|
||||
"govoplan-idm==0.1.24",
|
||||
"govoplan-access==0.1.23",
|
||||
"govoplan-admin==0.1.22",
|
||||
"govoplan-policy==0.1.22",
|
||||
"govoplan-core[server]==0.1.45",
|
||||
"govoplan-tenancy==0.1.22",
|
||||
"govoplan-organizations==0.1.21",
|
||||
"govoplan-identity==0.1.21",
|
||||
"govoplan-idm==0.1.25",
|
||||
"govoplan-access==0.1.25",
|
||||
"govoplan-admin==0.1.23",
|
||||
"govoplan-policy==0.1.23",
|
||||
"govoplan-audit==0.1.20",
|
||||
"govoplan-dashboard==0.1.20",
|
||||
"govoplan-files==0.1.23",
|
||||
"govoplan-mail==0.1.25",
|
||||
"govoplan-campaign==0.1.27",
|
||||
"govoplan-calendar==0.1.22",
|
||||
"govoplan-docs==0.1.22",
|
||||
"govoplan-ops==0.1.20",
|
||||
"govoplan-files==0.1.26",
|
||||
"govoplan-mail==0.1.27",
|
||||
"govoplan-campaign==0.1.28",
|
||||
"govoplan-calendar==0.1.23",
|
||||
"govoplan-docs==0.1.23",
|
||||
"govoplan-ops==0.1.22",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
full = [
|
||||
"govoplan-addresses==0.1.21",
|
||||
"govoplan-approvals==0.1.20",
|
||||
"govoplan-addresses==0.1.22",
|
||||
"govoplan-approvals==0.1.21",
|
||||
"govoplan-assets==0.1.20",
|
||||
"govoplan-booking==0.1.20",
|
||||
"govoplan-cases==0.1.22",
|
||||
"govoplan-cases==0.1.24",
|
||||
"govoplan-certificates==0.1.20",
|
||||
"govoplan-committee==0.1.20",
|
||||
"govoplan-connectors==0.1.25",
|
||||
"govoplan-committee==0.1.21",
|
||||
"govoplan-connectors==0.1.26",
|
||||
"govoplan-consultation==0.1.20",
|
||||
"govoplan-contracts==0.1.20",
|
||||
"govoplan-dataflow==0.1.23",
|
||||
"govoplan-datasources==0.1.24",
|
||||
"govoplan-dataflow==0.1.24",
|
||||
"govoplan-datasources==0.1.25",
|
||||
"govoplan-decisions==0.1.19",
|
||||
"govoplan-dist-lists==0.1.20",
|
||||
"govoplan-dist-lists==0.1.21",
|
||||
"govoplan-dms==0.1.20",
|
||||
"govoplan-encryption==0.1.19",
|
||||
"govoplan-encryption==0.1.20",
|
||||
"govoplan-erp==0.1.20",
|
||||
"govoplan-evaluation==0.1.20",
|
||||
"govoplan-facilities==0.1.20",
|
||||
"govoplan-fit-connect==0.1.20",
|
||||
"govoplan-forms==0.1.22",
|
||||
"govoplan-forms-runtime==0.1.20",
|
||||
"govoplan-forms==0.1.23",
|
||||
"govoplan-forms-runtime==0.1.21",
|
||||
"govoplan-grants==0.1.20",
|
||||
"govoplan-helpdesk==0.1.21",
|
||||
"govoplan-identity-trust==0.1.20",
|
||||
"govoplan-identity-trust==0.1.21",
|
||||
"govoplan-inspections==0.1.20",
|
||||
"govoplan-learning==0.1.20",
|
||||
"govoplan-mandates==0.1.19",
|
||||
"govoplan-notifications==0.1.19",
|
||||
"govoplan-notifications==0.1.20",
|
||||
"govoplan-parties==0.1.19",
|
||||
"govoplan-payments==0.1.21",
|
||||
"govoplan-payments==0.1.22",
|
||||
"govoplan-permits==0.1.20",
|
||||
"govoplan-poll==0.1.20",
|
||||
"govoplan-portal==0.1.21",
|
||||
"govoplan-postbox==0.1.22",
|
||||
"govoplan-portal==0.1.22",
|
||||
"govoplan-postbox==0.1.23",
|
||||
"govoplan-procurement==0.1.20",
|
||||
"govoplan-projects==0.1.19",
|
||||
"govoplan-quick-access==0.1.20",
|
||||
"govoplan-records==0.1.22",
|
||||
"govoplan-reporting==0.1.20",
|
||||
"govoplan-projects==0.1.20",
|
||||
"govoplan-quick-access==0.1.21",
|
||||
"govoplan-records==0.1.24",
|
||||
"govoplan-reporting==0.1.21",
|
||||
"govoplan-resources==0.1.20",
|
||||
"govoplan-rest==0.1.19",
|
||||
"govoplan-risk-compliance==0.1.20",
|
||||
"govoplan-scheduling==0.1.21",
|
||||
"govoplan-search==0.1.19",
|
||||
"govoplan-risk-compliance==0.1.21",
|
||||
"govoplan-scheduling==0.1.22",
|
||||
"govoplan-search==0.1.20",
|
||||
"govoplan-services==0.1.19",
|
||||
"govoplan-soap==0.1.19",
|
||||
"govoplan-tasks==0.1.21",
|
||||
"govoplan-templates==0.1.21",
|
||||
"govoplan-tasks==0.1.23",
|
||||
"govoplan-templates==0.1.22",
|
||||
"govoplan-tickets==0.1.22",
|
||||
"govoplan-transparency==0.1.20",
|
||||
"govoplan-views==0.1.21",
|
||||
"govoplan-voting==0.1.20",
|
||||
"govoplan-views==0.1.22",
|
||||
"govoplan-voting==0.1.21",
|
||||
"govoplan-wiki==0.1.22",
|
||||
"govoplan-workflow==0.1.22",
|
||||
"govoplan-workflow==0.1.23",
|
||||
"govoplan-workflow-engine==0.1.21",
|
||||
"govoplan-xrechnung==0.1.21",
|
||||
]
|
||||
|
||||
@@ -2,7 +2,7 @@ bandit>=1.8,<2
|
||||
click>=8.3.3
|
||||
filelock>=3.20.3
|
||||
idna>=3.15
|
||||
pip>=26.1.2
|
||||
pip>=26.2
|
||||
pip-audit>=2.9,<3
|
||||
python-multipart>=0.0.31
|
||||
radon>=6,<7
|
||||
|
||||
@@ -56,7 +56,7 @@ httpx2>=2.5,<3
|
||||
filelock>=3.20.3
|
||||
idna>=3.15
|
||||
jsonschema>=4,<5
|
||||
pip>=26.1.2
|
||||
pip>=26.2
|
||||
pip-audit>=2.9,<3
|
||||
pytest>=9.0.3,<10
|
||||
pygments>=2.20,<3
|
||||
|
||||
+13
-13
@@ -1,18 +1,18 @@
|
||||
# Whole-product release install from immutable, independently versioned module tags.
|
||||
# Only add a module after its referenced tag has been published.
|
||||
../govoplan-core[server]
|
||||
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.20
|
||||
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.20
|
||||
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.20
|
||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.24
|
||||
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.23
|
||||
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.22
|
||||
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.22
|
||||
govoplan-tenancy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-tenancy.git@v0.1.22
|
||||
govoplan-organizations @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-organizations.git@v0.1.21
|
||||
govoplan-identity @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-identity.git@v0.1.21
|
||||
govoplan-idm @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-idm.git@v0.1.25
|
||||
govoplan-access @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-access.git@v0.1.25
|
||||
govoplan-admin @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-admin.git@v0.1.23
|
||||
govoplan-policy @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-policy.git@v0.1.23
|
||||
govoplan-audit @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-audit.git@v0.1.20
|
||||
govoplan-dashboard @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-dashboard.git@v0.1.20
|
||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.23
|
||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.25
|
||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.27
|
||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.22
|
||||
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.22
|
||||
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.20
|
||||
govoplan-files @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-files.git@v0.1.26
|
||||
govoplan-mail @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-mail.git@v0.1.27
|
||||
govoplan-campaign @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-campaign.git@v0.1.28
|
||||
govoplan-calendar @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-calendar.git@v0.1.23
|
||||
govoplan-docs @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-docs.git@v0.1.23
|
||||
govoplan-ops @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-ops.git@v0.1.22
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { findTypeOnlyJsxImports } from "../tools/checks/check-jsx-value-imports.mjs";
|
||||
|
||||
const findings = (source) => findTypeOnlyJsxImports([{ path: "/fixture.tsx", source }]).map((item) => item.component);
|
||||
assert.deepEqual(findings('import type { FormGrid, AuthInfo } from "@govoplan/core-webui"; const page = <Dialog><FormGrid /></Dialog>;'), ["FormGrid"]);
|
||||
assert.deepEqual(findings('import { type FormGrid as Layout } from "ui"; const page = <Layout>Content</Layout>;'), ["Layout"]);
|
||||
assert.deepEqual(findings('import type Layout from "ui"; const page = <Layout />;'), ["Layout"]);
|
||||
assert.deepEqual(findings('import type * as ui from "ui"; const page = <ui.Layout />;'), ["ui.Layout"]);
|
||||
assert.deepEqual(findings('import type { FormGrid } from "ui"; const page = <div title={<FormGrid />} />;'), ["FormGrid"]);
|
||||
assert.deepEqual(findings('import { FormGrid, type AuthInfo } from "ui"; const page = <FormGrid />;'), []);
|
||||
assert.deepEqual(findings('import type { FormGrid } from "ui"; function Page({ FormGrid }: Props) { return <FormGrid />; }'), []);
|
||||
assert.deepEqual(findings('import type * as ui from "ui"; function Page(ui: RuntimeControls) { return <ui.Layout />; }'), []);
|
||||
assert.deepEqual(findings('import type { Layout } from "ui"; const page: Layout = {};'), []);
|
||||
assert.deepEqual(findings('import type { input } from "ui"; const page = <input />;'), []);
|
||||
console.log("JSX runtime-import AST regression tests passed (10 cases).");
|
||||
@@ -1,8 +1,10 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import redirect_stderr, redirect_stdout
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import subprocess
|
||||
@@ -36,6 +38,7 @@ import govoplan_deploy.cli as deployment_cli # noqa: E402
|
||||
from govoplan_deploy.capabilities import ( # noqa: E402
|
||||
capability_change_impacts,
|
||||
infrastructure_capability_document,
|
||||
infrastructure_dependency_inventory_from_mapping,
|
||||
)
|
||||
from govoplan_deploy.cluster_evidence import ( # noqa: E402
|
||||
collect_kubernetes_evidence,
|
||||
@@ -87,6 +90,41 @@ def _kubernetes_test_deployment(component: str, replicas: int) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _dependency_inventory(
|
||||
installation_id: str,
|
||||
*,
|
||||
generated_at: datetime | None = None,
|
||||
) -> dict:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"installation_id": installation_id,
|
||||
"generated_at": (generated_at or datetime.now(UTC)).isoformat(),
|
||||
"complete": True,
|
||||
"inspected_capability_ids": ["coordination.redis", "mail.smtp"],
|
||||
"providers": [
|
||||
{
|
||||
"module_id": "mail",
|
||||
"state": "complete",
|
||||
"capability_ids": ["mail.smtp"],
|
||||
"dependency_count": 1,
|
||||
}
|
||||
],
|
||||
"dependencies": [
|
||||
{
|
||||
"capability_id": "mail.smtp",
|
||||
"module_id": "mail",
|
||||
"dependency_type": "smtp_endpoint",
|
||||
"dependency_ref": "endpoint:17",
|
||||
"state": "active",
|
||||
"scope": "system",
|
||||
"summary": "Persisted SMTP endpoint has one credential binding.",
|
||||
"metrics": {"credential_binding_count": 1},
|
||||
"required_action": "Rebind or migrate this SMTP endpoint.",
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class DeploymentInstallerTests(unittest.TestCase):
|
||||
def test_kubernetes_evidence_requires_two_node_spread_and_safe_runtime(
|
||||
self,
|
||||
@@ -1034,6 +1072,28 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertNotIn("old-secret", impacts["database.postgresql"].detail)
|
||||
self.assertNotIn("new-secret", impacts["database.postgresql"].detail)
|
||||
|
||||
def test_capability_impact_includes_provider_dependency_evidence(self) -> None:
|
||||
previous_spec = default_spec(mail_mode="test-mail", module_set="full")
|
||||
desired_spec = default_spec(mail_mode="disabled", module_set="full")
|
||||
inventory = infrastructure_dependency_inventory_from_mapping(
|
||||
_dependency_inventory(previous_spec.installation_id)
|
||||
)
|
||||
|
||||
impacts = {
|
||||
item.capability_id: item
|
||||
for item in capability_change_impacts(
|
||||
infrastructure_capability_document(previous_spec, {}),
|
||||
infrastructure_capability_document(desired_spec, {}),
|
||||
dependency_inventory=inventory,
|
||||
)
|
||||
}
|
||||
|
||||
mail = impacts["mail.smtp"]
|
||||
self.assertTrue(mail.inventory_inspected)
|
||||
self.assertEqual("endpoint:17", mail.actual_dependencies[0].dependency_ref)
|
||||
self.assertIn("mail:endpoint:17", mail.detail)
|
||||
self.assertIn("Rebind or migrate", mail.required_action)
|
||||
|
||||
def test_replica_counts_drive_compose_and_load_balancer_discovery(self) -> None:
|
||||
spec = default_spec(
|
||||
storage_mode="garage",
|
||||
@@ -1221,6 +1281,65 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
for check in second_plan.checks
|
||||
)
|
||||
)
|
||||
self.assertTrue(second_plan.blocked)
|
||||
self.assertTrue(
|
||||
any(
|
||||
check.id == "capability.dependency_inventory.missing"
|
||||
and check.level == "error"
|
||||
for check in second_plan.checks
|
||||
)
|
||||
)
|
||||
|
||||
atomic_write(
|
||||
paths.dependency_inventory,
|
||||
canonical_json(_dependency_inventory(second_spec.installation_id)),
|
||||
mode=0o600,
|
||||
)
|
||||
evidenced_plan = build_plan(
|
||||
second_spec,
|
||||
paths,
|
||||
include_host_checks=False,
|
||||
)
|
||||
|
||||
self.assertFalse(
|
||||
any(
|
||||
check.level == "error"
|
||||
and check.id.startswith("capability.dependency_inventory.")
|
||||
for check in evidenced_plan.checks
|
||||
)
|
||||
)
|
||||
self.assertEqual(
|
||||
"endpoint:17",
|
||||
{
|
||||
item.capability_id: item
|
||||
for item in evidenced_plan.capability_impacts
|
||||
}["mail.smtp"].actual_dependencies[0].dependency_ref,
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
check.id == "capability.dependency_inventory.current"
|
||||
and check.level == "ok"
|
||||
for check in evidenced_plan.checks
|
||||
)
|
||||
)
|
||||
|
||||
stale = _dependency_inventory(
|
||||
second_spec.installation_id,
|
||||
generated_at=datetime.now(UTC) - timedelta(minutes=6),
|
||||
)
|
||||
atomic_write(
|
||||
paths.dependency_inventory,
|
||||
canonical_json(stale),
|
||||
mode=0o600,
|
||||
)
|
||||
stale_plan = build_plan(second_spec, paths, include_host_checks=False)
|
||||
self.assertTrue(stale_plan.blocked)
|
||||
self.assertTrue(
|
||||
any(
|
||||
check.id == "capability.dependency_inventory.stale"
|
||||
for check in stale_plan.checks
|
||||
)
|
||||
)
|
||||
|
||||
def test_secret_change_is_planned_without_exposing_secret_values(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
|
||||
@@ -1330,6 +1449,54 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
)[0],
|
||||
)
|
||||
|
||||
def test_cli_collects_bounded_private_dependency_inventory(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
|
||||
root = Path(directory) / "installation"
|
||||
self.assertEqual(
|
||||
0,
|
||||
run_cli(
|
||||
[
|
||||
"init",
|
||||
"--non-interactive",
|
||||
"--directory",
|
||||
str(root),
|
||||
]
|
||||
)[0],
|
||||
)
|
||||
payload = _dependency_inventory("govoplan-local")
|
||||
response = MagicMock()
|
||||
response.__enter__.return_value = response
|
||||
response.geturl.return_value = "https://ops.example.test/inventory"
|
||||
response.read.return_value = json.dumps(payload).encode("utf-8")
|
||||
fetch = MagicMock(return_value=response)
|
||||
|
||||
with (
|
||||
patch.dict(os.environ, {"TEST_OPS_KEY": "secret-api-key"}),
|
||||
patch.object(deployment_cli, "urlopen", fetch),
|
||||
):
|
||||
result, stdout, stderr = run_cli(
|
||||
[
|
||||
"collect-infrastructure-inventory",
|
||||
"--directory",
|
||||
str(root),
|
||||
"--ops-url",
|
||||
"https://ops.example.test/inventory",
|
||||
"--api-key-env",
|
||||
"TEST_OPS_KEY",
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(0, result, stderr)
|
||||
self.assertIn("1 record(s)", stdout)
|
||||
evidence_path = root / "infrastructure-dependency-inventory.json"
|
||||
self.assertEqual(0o600, stat.S_IMODE(evidence_path.stat().st_mode))
|
||||
self.assertNotIn(
|
||||
"secret-api-key",
|
||||
evidence_path.read_text(encoding="utf-8"),
|
||||
)
|
||||
request = fetch.call_args.args[0]
|
||||
self.assertEqual("secret-api-key", request.get_header("X-api-key"))
|
||||
|
||||
def test_cli_requires_external_url_when_switching_from_managed(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="govoplan-deploy-test-") as directory:
|
||||
root = Path(directory) / "installation"
|
||||
|
||||
@@ -0,0 +1,466 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from contextlib import ExitStack
|
||||
import csv
|
||||
from copy import deepcopy
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
from urllib.parse import quote
|
||||
import zipfile
|
||||
|
||||
from cryptography.hazmat.primitives import serialization
|
||||
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools/release"))
|
||||
|
||||
from govoplan_release import full_catalog # noqa: E402
|
||||
from govoplan_release.artifact_identity import selected_artifact_identity_issues # noqa: E402
|
||||
from govoplan_release.catalog import canonical_hash # noqa: E402
|
||||
from govoplan_release.model import RepositorySpec # noqa: E402
|
||||
from govoplan_release.registry_reference import registry_entry_source # noqa: E402
|
||||
from govoplan_release.selective_catalog import ( # noqa: E402
|
||||
load_authenticated_catalog_base, public_key_base64, signature,
|
||||
)
|
||||
from govoplan_release.source_provenance import ( # noqa: E402
|
||||
SourceTagProvenanceIssue, catalog_source_selection,
|
||||
registered_source_origin_issues,
|
||||
)
|
||||
from govoplan_release.version_alignment import candidate_catalog_version_issues # noqa: E402
|
||||
|
||||
|
||||
class FullRegistryCatalogTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.web = self.root / "addideas-govoplan-website"
|
||||
self.wheels = self.root / "wheels"
|
||||
self.npm = self.root / "npm"
|
||||
self.wheels.mkdir(mode=0o700)
|
||||
self.npm.mkdir(mode=0o700)
|
||||
self.key = Ed25519PrivateKey.generate()
|
||||
self.keypath = self.root / "key.pem"
|
||||
self.keypath.write_bytes(self.key.private_bytes(
|
||||
serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
|
||||
serialization.NoEncryption(),
|
||||
))
|
||||
self.keypath.chmod(0o600)
|
||||
self.keyring = {
|
||||
"keyring_version": "1", "keys": [{
|
||||
"key_id": "known-key", "public_key": public_key_base64(self.key),
|
||||
"status": "active",
|
||||
}],
|
||||
}
|
||||
self.base = {
|
||||
"catalog_version": "1", "channel": "stable", "sequence": 1,
|
||||
"core_release": {"version": "1.0.0"}, "modules": [],
|
||||
"release": {},
|
||||
}
|
||||
self.write_base()
|
||||
self.package_set = {
|
||||
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
|
||||
"registries": {
|
||||
"python": "https://git.add-ideas.de/api/packages/GovOPlaN/pypi/simple",
|
||||
"npm": "https://git.add-ideas.de/api/packages/GovOPlaN/npm/",
|
||||
},
|
||||
"python": [self.package("govoplan-core"), self.package("govoplan-demo")],
|
||||
"webui": [self.package("govoplan-core", webui=True)],
|
||||
}
|
||||
self.seal(self.package_set, "package_set_sha256")
|
||||
self.lock = {
|
||||
"schema_version": "1", "release_version": "1.2.3", "profile": "full",
|
||||
"registries": self.package_set["registries"],
|
||||
"package_set_sha256": self.package_set["package_set_sha256"],
|
||||
"python": [], "webui": [],
|
||||
}
|
||||
for row in self.package_set["python"]:
|
||||
path = self.wheel(row["name"])
|
||||
url = full_catalog._tool("resolve-package-artifacts")["_python_artifact_url"](
|
||||
self.package_set["registries"]["python"], package=row, filename=path.name,
|
||||
)
|
||||
self.lock["python"].append(self.artifact(row, path, url))
|
||||
npm_package = self.package_set["webui"][0]
|
||||
npm_path = self.npm / "govoplan-core-webui-1.2.3.tgz"
|
||||
self.tarball(npm_path, "@govoplan/core-webui", "1.2.3")
|
||||
url = self.package_set["registries"]["npm"] + quote(npm_package["name"], safe="") + "/-/1.2.3/core-webui-1.2.3.tgz"
|
||||
row = self.artifact(npm_package, npm_path, url)
|
||||
row["integrity"] = "sha512-" + base64.b64encode(hashlib.sha512(npm_path.read_bytes()).digest()).decode()
|
||||
self.lock["webui"].append(row)
|
||||
self.seal(self.lock, "lock_sha256")
|
||||
self.set_path = self.root / "package-set.json"
|
||||
self.lock_path = self.root / "package-lock.json"
|
||||
self.write_inputs()
|
||||
self.output = self.root / "candidate"
|
||||
|
||||
@staticmethod
|
||||
def package(repo: str, *, webui: bool = False) -> dict:
|
||||
row = {
|
||||
"name": "@govoplan/core-webui" if webui else repo, "version": "1.2.3",
|
||||
"repository": repo, "tag": "v1.2.3",
|
||||
"commit": ("a" if repo == "govoplan-core" else "b") * 40,
|
||||
}
|
||||
if not webui:
|
||||
row["extras"] = ["server"] if repo == "govoplan-core" else []
|
||||
return row
|
||||
|
||||
@staticmethod
|
||||
def artifact(package: dict, path: Path, url: str) -> dict:
|
||||
encoded = path.read_bytes()
|
||||
return {**package, "filename": path.name, "url": url,
|
||||
"sha256": hashlib.sha256(encoded).hexdigest(), "size": len(encoded)}
|
||||
|
||||
@staticmethod
|
||||
def seal(payload: dict, field: str) -> None:
|
||||
payload.pop(field, None)
|
||||
payload[field] = hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
|
||||
|
||||
def write_inputs(self) -> None:
|
||||
self.set_path.write_text(json.dumps(self.package_set))
|
||||
self.lock_path.write_text(json.dumps(self.lock))
|
||||
|
||||
def write_base(self) -> None:
|
||||
self.base.pop("signatures", None)
|
||||
self.base["signatures"] = [signature(self.base, key_id="known-key", private_key=self.key)]
|
||||
folder = self.web / "public/catalogs/v1"
|
||||
(folder / "channels").mkdir(parents=True, exist_ok=True)
|
||||
(folder / "channels/stable.json").write_text(json.dumps(self.base))
|
||||
(folder / "keyring.json").write_text(json.dumps(self.keyring))
|
||||
|
||||
def wheel(self, package: str) -> Path:
|
||||
stem = package.replace("-", "_")
|
||||
info = f"{stem}-1.2.3.dist-info"
|
||||
files = {
|
||||
f"{stem}/__init__.py": b"VALUE = 1\n",
|
||||
f"{info}/METADATA": f"Metadata-Version: 2.1\nName: {package}\nVersion: 1.2.3\n".encode(),
|
||||
f"{info}/WHEEL": b"Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
|
||||
}
|
||||
record = io.StringIO()
|
||||
writer = csv.writer(record, lineterminator="\n")
|
||||
for name, value in files.items():
|
||||
writer.writerow((name, "", len(value)))
|
||||
writer.writerow((f"{info}/RECORD", "", ""))
|
||||
files[f"{info}/RECORD"] = record.getvalue().encode()
|
||||
path = self.wheels / f"{stem}-1.2.3-py3-none-any.whl"
|
||||
with zipfile.ZipFile(path, "w") as archive:
|
||||
for name, value in files.items():
|
||||
archive.writestr(name, value)
|
||||
return path
|
||||
|
||||
@staticmethod
|
||||
def tarball(path: Path, name: str, version: str, *, duplicate: bool = False) -> None:
|
||||
encoded = json.dumps({"name": name, "version": version}).encode()
|
||||
with tarfile.open(path, "w:gz") as archive:
|
||||
for _ in range(2 if duplicate else 1):
|
||||
member = tarfile.TarInfo("package/package.json")
|
||||
member.size = len(encoded)
|
||||
archive.addfile(member, io.BytesIO(encoded))
|
||||
|
||||
def build(self, *, provenance_errors=(), origin_errors=()) -> dict:
|
||||
registry_generator = full_catalog._tool("generate-release-catalog")
|
||||
tools = {name: dict(full_catalog._tool(name)) for name in (
|
||||
"generate-release-catalog", "generate-release-package-set", "resolve-package-artifacts",
|
||||
)}
|
||||
tools["generate-release-package-set"]["generate_package_set"] = lambda **kwargs: self.package_set
|
||||
self.provenance = {
|
||||
row["repository"]: {"commit_sha": row["commit"], "tag_object_sha": str(index + 1) * 40}
|
||||
for index, row in enumerate(self.package_set["python"])
|
||||
}
|
||||
entry = {
|
||||
"module_id": "demo", "name": "Demo", "version": "1.2.3",
|
||||
"python_package": "govoplan-demo",
|
||||
"python_ref": "govoplan-demo @ git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-demo.git@v1.2.3",
|
||||
}
|
||||
with ExitStack() as stack:
|
||||
stack.enter_context(patch.object(full_catalog, "_tool", side_effect=lambda name: tools[name]))
|
||||
stack.enter_context(patch.dict(registry_generator["_catalog_payload"].__globals__, {
|
||||
"synthesize_repository_catalog_entries": lambda **kwargs: (dict(entry),),
|
||||
}))
|
||||
stack.enter_context(patch.object(full_catalog, "enforce_selected_version_alignment"))
|
||||
stack.enter_context(patch.object(full_catalog, "registered_source_origin_issues", return_value=origin_errors))
|
||||
self.provenance_check = stack.enter_context(patch.object(full_catalog, "source_tag_provenance_issues", return_value=provenance_errors))
|
||||
stack.enter_context(patch.object(full_catalog, "selected_source_provenance", return_value=self.provenance))
|
||||
return full_catalog.build_full_registry_candidate(
|
||||
package_set_path=self.set_path, package_lock_path=self.lock_path,
|
||||
wheelhouse=self.wheels, webui_packages=self.npm, output_dir=self.output,
|
||||
selected_repositories=("govoplan-core",),
|
||||
signing_keys=(f"known-key={self.keypath}",), workspace_root=self.root,
|
||||
)
|
||||
|
||||
def candidate(self) -> dict:
|
||||
return json.loads((self.output / "channels/stable.json").read_text())
|
||||
|
||||
def test_full_candidate_uses_registry_bytes_and_preserves_unchanged_tag_provenance(self) -> None:
|
||||
result = self.build()
|
||||
candidate = self.candidate()
|
||||
self.assertEqual("ready", result["status"])
|
||||
self.assertEqual(2, result["package_count"])
|
||||
self.assertEqual(1, result["selected_count"])
|
||||
self.assertEqual(self.keyring, json.loads((self.output / "keyring.json").read_text()))
|
||||
self.assertEqual(canonical_hash(self.keyring), candidate["release"]["keyring_sha256"])
|
||||
self.assertEqual(2, len(candidate["release"]["artifacts"]))
|
||||
self.assertIn("/pypi/files/", candidate["core_release"]["python_ref"])
|
||||
self.assertEqual((), candidate_catalog_version_issues(candidate))
|
||||
self.assertEqual((), selected_artifact_identity_issues(candidate))
|
||||
sources = catalog_source_selection(candidate)
|
||||
self.assertEqual((), sources.issues)
|
||||
self.assertEqual({"govoplan-core": "1.2.3"}, sources.selected_versions)
|
||||
self.assertEqual({"govoplan-core": "1.2.3", "govoplan-demo": "1.2.3"}, sources.all_versions)
|
||||
self.assertEqual("b" * 40, sources.selected_commits["govoplan-demo"])
|
||||
self.assertEqual("2" * 40, sources.selected_tag_objects["govoplan-demo"])
|
||||
self.assertEqual(2, self.provenance_check.call_count)
|
||||
for call in self.provenance_check.call_args_list:
|
||||
self.assertEqual({"govoplan-core"}, call.kwargs["require_head_repos"])
|
||||
for path in [self.output, *self.output.rglob("*")]:
|
||||
self.assertEqual(0o700 if path.is_dir() else 0o600, path.stat().st_mode & 0o777)
|
||||
|
||||
def test_legacy_base_is_authenticated_but_remains_rejected_by_selective(self) -> None:
|
||||
self.build()
|
||||
with self.assertRaisesRegex(ValueError, "does not pin"):
|
||||
load_authenticated_catalog_base(
|
||||
base_catalog=None, base_keyring=None, web_root=self.web,
|
||||
channel="stable", public_base_url="https://unused.example",
|
||||
signer_public_keys={"known-key": public_key_base64(self.key)},
|
||||
)
|
||||
|
||||
def test_injected_key_or_mismatched_pinned_keyring_is_rejected(self) -> None:
|
||||
for mutation in ("extra-key", "bad-hash"):
|
||||
with self.subTest(mutation=mutation):
|
||||
if mutation == "extra-key":
|
||||
self.keyring["keys"].append({"key_id": "injected", "status": "active", "public_key": public_key_base64(Ed25519PrivateKey.generate())})
|
||||
else:
|
||||
self.keyring["keys"] = self.keyring["keys"][:1]
|
||||
self.base["release"]["keyring_sha256"] = "f" * 64
|
||||
self.write_base()
|
||||
with self.assertRaises(ValueError):
|
||||
self.build()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_tampered_base_signature_is_rejected(self) -> None:
|
||||
path = self.web / "public/catalogs/v1/channels/stable.json"
|
||||
payload = json.loads(path.read_text())
|
||||
payload["sequence"] = 999
|
||||
path.write_text(json.dumps(payload))
|
||||
with self.assertRaisesRegex(ValueError, "signature verification"):
|
||||
self.build()
|
||||
|
||||
def test_wrong_registry_bytes_and_reused_candidate_fail_closed(self) -> None:
|
||||
self.build()
|
||||
original = (self.output / "channels/stable.json").read_bytes()
|
||||
with self.assertRaisesRegex(ValueError, "must not already exist"):
|
||||
self.build()
|
||||
self.assertEqual(original, (self.output / "channels/stable.json").read_bytes())
|
||||
self.output = self.root / "candidate-2"
|
||||
wheel = self.wheels / self.lock["python"][0]["filename"]
|
||||
with wheel.open("ab") as stream:
|
||||
stream.write(b"tampered")
|
||||
with self.assertRaisesRegex(ValueError, "bytes differ"):
|
||||
self.build()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_webui_identity_and_url_changes_are_rejected(self) -> None:
|
||||
row = self.lock["webui"][0]
|
||||
original = row["url"]
|
||||
for url in (
|
||||
original + "?alternate=true", original.replace("/-/1.2.3/", "/-/9.9.9/"),
|
||||
original.replace("/npm/", "/npm/../other/"),
|
||||
original.replace("/npm/", "/npm/%2e%2e/other/"),
|
||||
original.replace("/npm/", "/npm/%252e%252e/other/"),
|
||||
):
|
||||
with self.subTest(url=url):
|
||||
row["url"] = url
|
||||
self.seal(self.lock, "lock_sha256")
|
||||
self.write_inputs()
|
||||
with self.assertRaisesRegex(ValueError, "URL differs"):
|
||||
self.build()
|
||||
|
||||
def test_duplicate_missing_and_symlinked_artifacts_are_rejected(self) -> None:
|
||||
row = self.lock["webui"][0]
|
||||
self.lock["webui"].append(dict(row))
|
||||
with self.assertRaisesRegex(ValueError, "duplicate/missing"):
|
||||
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
|
||||
self.lock["webui"].pop()
|
||||
path = self.npm / row["filename"]
|
||||
moved = self.root / "moved.tgz"
|
||||
path.rename(moved)
|
||||
path.symlink_to(moved)
|
||||
with self.assertRaises(OSError):
|
||||
full_catalog.verify_registry_artifacts(package_set=self.package_set, lock=self.lock, wheelhouse=self.wheels, webui_packages=self.npm)
|
||||
|
||||
def test_archive_metadata_is_bounded_and_not_ambiguous(self) -> None:
|
||||
path = self.npm / "duplicate.tgz"
|
||||
self.tarball(path, "@govoplan/core-webui", "1.2.3", duplicate=True)
|
||||
with self.assertRaisesRegex(ValueError, "duplicate"):
|
||||
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
|
||||
|
||||
def test_origin_or_tag_provenance_failure_prevents_output(self) -> None:
|
||||
issue = SourceTagProvenanceIssue("govoplan-core", "v1.2.3", "wrong immutable identity")
|
||||
for kwargs in ({"origin_errors": (issue,)}, {"provenance_errors": (issue,)}):
|
||||
with self.subTest(kwargs=kwargs), self.assertRaisesRegex(ValueError, "gate failed"):
|
||||
self.build(**kwargs)
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_registered_source_origin_requires_exact_fetch_and_push_targets(self) -> None:
|
||||
repo = self.root / "govoplan-core"
|
||||
repo.mkdir()
|
||||
spec = RepositorySpec("govoplan-core", "system", "kernel", "git@example.test:trusted/core.git", "govoplan-core")
|
||||
with patch("govoplan_release.source_provenance.load_repository_specs", return_value=(spec,)):
|
||||
for targets in ((spec.remote, spec.remote), ("git@evil.test:core.git", spec.remote), (spec.remote, "git@evil.test:core.git")):
|
||||
with self.subTest(targets=targets), patch("govoplan_release.source_provenance.git_text", side_effect=targets):
|
||||
issues = registered_source_origin_issues(repo_versions={"govoplan-core": "1.2.3"}, workspace=self.root, remote="origin")
|
||||
self.assertEqual(targets != (spec.remote, spec.remote), bool(issues))
|
||||
|
||||
def test_registry_metadata_cannot_cross_wire_webui_or_archive_identity(self) -> None:
|
||||
self.build()
|
||||
candidate = self.candidate()
|
||||
entry = candidate["core_release"]
|
||||
entry["webui_package"] = "@govoplan/files-webui"
|
||||
with self.assertRaisesRegex(ValueError, "another source repository"):
|
||||
registry_entry_source(entry)
|
||||
candidate = self.candidate()
|
||||
candidate["release"]["artifacts"][0]["archive_sha256"] = "f" * 64
|
||||
self.assertIn("matching inspected wheel", " ".join(selected_artifact_identity_issues(candidate)))
|
||||
|
||||
def test_registry_version_and_selected_source_identity_must_agree(self) -> None:
|
||||
self.build()
|
||||
for field in ("commit_sha", "tag_object_sha"):
|
||||
candidate = self.candidate()
|
||||
candidate["release"]["selected_units"][0][field] = "f" * 40
|
||||
self.assertIn("differs", " ".join(issue.message for issue in catalog_source_selection(candidate).issues))
|
||||
candidate = self.candidate()
|
||||
candidate["modules"][0]["artifact_integrity"]["python"]["git_ref"] = "v9.9.9"
|
||||
self.assertTrue(candidate_catalog_version_issues(candidate))
|
||||
|
||||
def test_repeated_module_projections_must_bind_identical_python_and_webui_bytes(self) -> None:
|
||||
self.build()
|
||||
for kind in ("python", "webui"):
|
||||
for reversed_order in (False, True):
|
||||
with self.subTest(kind=kind, reversed_order=reversed_order):
|
||||
candidate = self.candidate()
|
||||
entry = deepcopy(candidate["core_release"])
|
||||
entry["module_id"] = "another-core-projection"
|
||||
artifact = entry["artifact_integrity"][kind]
|
||||
artifact["sha256"] = "f" * 64
|
||||
if kind == "python":
|
||||
entry["python_ref"] = artifact["ref"] = entry["python_ref"].split("#sha256=", 1)[0] + "#sha256=" + "f" * 64
|
||||
if reversed_order:
|
||||
original = candidate["core_release"]
|
||||
candidate["core_release"] = entry
|
||||
entry = original
|
||||
candidate["modules"].append(entry)
|
||||
self.assertIn(f"conflicting {kind}", " ".join(selected_artifact_identity_issues(candidate)))
|
||||
self.assertTrue(candidate_catalog_version_issues(candidate))
|
||||
candidate = self.candidate()
|
||||
repeated = deepcopy(candidate["modules"][0])
|
||||
repeated["module_id"] = "second-demo-projection"
|
||||
candidate["modules"].append(repeated)
|
||||
self.assertEqual((), selected_artifact_identity_issues(candidate))
|
||||
self.assertEqual((), candidate_catalog_version_issues(candidate))
|
||||
|
||||
def test_metadata_inspection_uses_the_opened_archive_not_a_replaced_path(self) -> None:
|
||||
path = self.npm / "original.tgz"
|
||||
replacement = self.npm / "replacement.tgz"
|
||||
saved = self.npm / "saved.tgz"
|
||||
self.tarball(path, "@govoplan/incorrect-webui", "1.2.3")
|
||||
self.tarball(replacement, "@govoplan/core-webui", "1.2.3")
|
||||
real_open = tarfile.open
|
||||
|
||||
def replace_path(*args, **kwargs):
|
||||
self.assertIn("fileobj", kwargs)
|
||||
path.rename(saved)
|
||||
replacement.rename(path)
|
||||
return real_open(*args, **kwargs)
|
||||
|
||||
with patch("govoplan_release.full_catalog.tarfile.open", side_effect=replace_path):
|
||||
with self.assertRaisesRegex(ValueError, "metadata differs"):
|
||||
full_catalog._inspect_npm_metadata(path, name="@govoplan/core-webui", version="1.2.3")
|
||||
|
||||
def test_registry_url_provenance_rejects_package_version_and_traversal_mismatch(self) -> None:
|
||||
self.build()
|
||||
for old, new in (
|
||||
("/govoplan-core/1.2.3/", "/govoplan-files/1.2.3/"),
|
||||
("/govoplan-core/1.2.3/", "/govoplan-core/9.9.9/"),
|
||||
("/pypi/files/", "/pypi/files/%2e%2e/"),
|
||||
("/pypi/files/", "/pypi/files/%252e%252e/"),
|
||||
):
|
||||
with self.subTest(new=new):
|
||||
entry = self.candidate()["core_release"]
|
||||
artifact = entry["artifact_integrity"]["python"]
|
||||
artifact["url"] = artifact["url"].replace(old, new)
|
||||
artifact["ref"] = entry["python_ref"] = entry["python_ref"].replace(old, new)
|
||||
with self.assertRaises(ValueError):
|
||||
registry_entry_source(entry)
|
||||
|
||||
def test_package_set_must_match_fixed_meta_pins_not_just_its_own_hash(self) -> None:
|
||||
payload = deepcopy(self.package_set)
|
||||
payload["python"][1]["version"] = "9.9.9"
|
||||
self.seal(payload, "package_set_sha256")
|
||||
self.set_path.write_text(json.dumps(payload))
|
||||
with self.assertRaisesRegex(ValueError, "exact Meta full pins"):
|
||||
self.build()
|
||||
self.assertFalse(self.output.exists())
|
||||
|
||||
def test_package_set_git_reads_ignore_caller_redirection(self) -> None:
|
||||
tool = full_catalog._tool("generate-release-package-set")
|
||||
with patch.dict(os.environ, {"GIT_DIR": "/outside", "GIT_CONFIG_GLOBAL": "/outside/config", "PATH": "/outside/bin"}):
|
||||
with patch("subprocess.check_output", return_value="a" * 40 + "\n") as execute:
|
||||
self.assertEqual("a" * 40, tool["_git"](self.root, "rev-parse", "HEAD"))
|
||||
self.assertEqual("/usr/bin/git", execute.call_args.args[0][0])
|
||||
self.assertNotIn("GIT_DIR", execute.call_args.kwargs["env"])
|
||||
self.assertEqual(os.devnull, execute.call_args.kwargs["env"]["GIT_CONFIG_GLOBAL"])
|
||||
self.assertEqual("/usr/bin:/bin", execute.call_args.kwargs["env"]["PATH"])
|
||||
|
||||
def test_unchanged_real_annotated_ancestor_is_valid_but_selecting_it_requires_head(self) -> None:
|
||||
from tests.test_release_source_provenance import git, git_text, make_repo
|
||||
from govoplan_release.source_provenance import source_tag_provenance_issues
|
||||
|
||||
workspace = self.root / "source-workspace"
|
||||
workspace.mkdir()
|
||||
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
|
||||
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
|
||||
git(repo, "push", "origin", "refs/tags/v1.2.3")
|
||||
tagged_commit = git_text(repo, "rev-parse", "HEAD")
|
||||
(repo / "workflow-only.txt").write_text("post-tag workflow repair\n")
|
||||
git(repo, "add", "workflow-only.txt")
|
||||
git(repo, "commit", "-m", "repair workflow without replacing package")
|
||||
git(repo, "push", "origin", "main")
|
||||
common = {
|
||||
"repo_versions": {"govoplan-access": "1.2.3"}, "workspace": workspace,
|
||||
"expected_commits": {"govoplan-access": tagged_commit},
|
||||
"expected_tag_objects": {"govoplan-access": git_text(repo, "rev-parse", "refs/tags/v1.2.3")},
|
||||
}
|
||||
self.assertEqual((), source_tag_provenance_issues(**common))
|
||||
issues = source_tag_provenance_issues(**common, require_head_repos=("govoplan-access",))
|
||||
self.assertIn("not selected HEAD", " ".join(issue.message for issue in issues))
|
||||
|
||||
def test_tagged_manifest_synthesis_ignores_local_git_replacement_objects(self) -> None:
|
||||
from tests.test_release_source_provenance import git, git_text, make_repo
|
||||
from govoplan_release.catalog_entry_synthesis import materialized_source_tree
|
||||
|
||||
workspace = self.root / "materialization-workspace"
|
||||
workspace.mkdir()
|
||||
repo, _remote = make_repo(workspace, self.root, "govoplan-access", "1.2.3")
|
||||
original = (repo / "pyproject.toml").read_text()
|
||||
tagged_commit = git_text(repo, "rev-parse", "HEAD")
|
||||
git(repo, "tag", "-a", "v1.2.3", "-m", "reviewed immutable package")
|
||||
(repo / "pyproject.toml").write_text(original.replace("1.2.3", "9.9.9"))
|
||||
git(repo, "add", "pyproject.toml")
|
||||
git(repo, "commit", "-m", "unreviewed replacement tree")
|
||||
git(repo, "replace", tagged_commit, git_text(repo, "rev-parse", "HEAD"))
|
||||
with patch.dict(os.environ, {"GIT_DIR": str(self.root / "outside"), "PATH": "/outside/bin"}):
|
||||
with materialized_source_tree(repo, source_ref="v1.2.3") as source:
|
||||
self.assertEqual(original, (source / "pyproject.toml").read_text())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -25,6 +25,11 @@ from govoplan_core.core.institutional import (
|
||||
TemporalRevision,
|
||||
service_launch_capability,
|
||||
)
|
||||
from govoplan_core.core.temporal import (
|
||||
TemporalDataContext,
|
||||
bind_temporal_data_context,
|
||||
reset_temporal_data_context,
|
||||
)
|
||||
from govoplan_cases.backend.party_context import CasePartyContext
|
||||
from govoplan_cases.backend.db.models import (
|
||||
CaseAccessGrant,
|
||||
@@ -134,6 +139,13 @@ class _Registry:
|
||||
|
||||
|
||||
class InstitutionalGovernanceJourneyTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
# Portal's effective_at does not replace the SQL provider's request-local
|
||||
# read clock. Keep both on the journey date, without bypassing validity
|
||||
# filtering or extending the fixture's finite publication interval.
|
||||
token = bind_temporal_data_context(TemporalDataContext(evaluated_at=NOW))
|
||||
self.addCleanup(reset_temporal_data_context, token)
|
||||
|
||||
def test_service_to_formal_outcome_retains_governed_context(self) -> None:
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
for table in (
|
||||
@@ -220,13 +232,44 @@ class InstitutionalGovernanceJourneyTests(unittest.TestCase):
|
||||
service_launch_capability("case"): object(),
|
||||
}
|
||||
)
|
||||
entry = PortalServiceDirectory(service_registry).list_entries(
|
||||
directory = PortalServiceDirectory(service_registry)
|
||||
for outside_interval in (
|
||||
service.temporal.valid_from - timedelta(microseconds=1),
|
||||
service.temporal.valid_to,
|
||||
):
|
||||
with self.subTest(outside_interval=outside_interval):
|
||||
token = bind_temporal_data_context(
|
||||
TemporalDataContext(evaluated_at=outside_interval)
|
||||
)
|
||||
try:
|
||||
# Keep Portal inside the valid interval: the real SQL
|
||||
# provider must still exclude a service outside its own
|
||||
# temporal read context, before Portal can project it.
|
||||
self.assertEqual(
|
||||
(),
|
||||
directory.list_entries(
|
||||
session,
|
||||
principal,
|
||||
tenant_id="tenant-1",
|
||||
effective_at=NOW,
|
||||
audiences=("resident",),
|
||||
),
|
||||
)
|
||||
finally:
|
||||
reset_temporal_data_context(token)
|
||||
|
||||
entries = directory.list_entries(
|
||||
session,
|
||||
principal,
|
||||
tenant_id="tenant-1",
|
||||
effective_at=NOW,
|
||||
audiences=("resident",),
|
||||
)[0]
|
||||
)
|
||||
self.assertEqual(
|
||||
(service.reference,),
|
||||
tuple(entry.definition.reference for entry in entries),
|
||||
)
|
||||
entry = entries[0]
|
||||
self.assertTrue(entry.available)
|
||||
intake = CaseServiceIntake().plan(
|
||||
entry.definition,
|
||||
|
||||
@@ -62,22 +62,37 @@ class PackageRegistryReleaseTests(unittest.TestCase):
|
||||
encoding="utf-8"
|
||||
)
|
||||
)["project"]["version"]
|
||||
meta_package = ROOT / "packages/govoplan-meta/pyproject.toml"
|
||||
meta_project = tomllib.loads(
|
||||
meta_package.read_text(encoding="utf-8")
|
||||
)["project"]
|
||||
expected_tasks_pin = next(
|
||||
requirement
|
||||
for requirement in (
|
||||
*meta_project["dependencies"],
|
||||
*meta_project["optional-dependencies"]["full"],
|
||||
)
|
||||
if requirement.startswith("govoplan-tasks==")
|
||||
)
|
||||
selected = PACKAGE_SET.parse_meta_package(
|
||||
ROOT / "packages/govoplan-meta/pyproject.toml",
|
||||
meta_package,
|
||||
core_version=core_version,
|
||||
)
|
||||
|
||||
by_name = {item["name"]: item for item in selected}
|
||||
self.assertIn("govoplan-core", by_name)
|
||||
self.assertIn("govoplan-records", by_name)
|
||||
self.assertEqual("0.1.21", by_name["govoplan-tasks"]["version"])
|
||||
self.assertEqual(
|
||||
expected_tasks_pin,
|
||||
f"govoplan-tasks=={by_name['govoplan-tasks']['version']}",
|
||||
)
|
||||
|
||||
payload = PACKAGE_SET.generate_package_set(
|
||||
core_version=core_version,
|
||||
requirements=ROOT / "requirements-release.txt",
|
||||
workspace=ROOT.parent,
|
||||
profile="full",
|
||||
meta_package=ROOT / "packages/govoplan-meta/pyproject.toml",
|
||||
meta_package=meta_package,
|
||||
)
|
||||
self.assertEqual("full", payload["profile"])
|
||||
self.assertEqual(len(selected), len(payload["python"]))
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
@@ -17,6 +18,80 @@ from govoplan_release import git_state # noqa: E402
|
||||
|
||||
|
||||
class ReleaseGitStateTests(unittest.TestCase):
|
||||
def test_unset_ssh_address_family_preserves_original_command_and_operator_config(self) -> None:
|
||||
environment = git_state.sanitized_git_environment({})
|
||||
|
||||
self.assertEqual(
|
||||
[
|
||||
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
|
||||
],
|
||||
shlex.split(environment["GIT_SSH_COMMAND"]),
|
||||
)
|
||||
self.assertNotIn("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY", environment)
|
||||
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
|
||||
|
||||
def test_ssh_address_family_accepts_only_fixed_choices_and_survives_resanitizing(self) -> None:
|
||||
for family in ("any", "inet", "inet6"):
|
||||
with self.subTest(family=family):
|
||||
environment = git_state.sanitized_git_environment({
|
||||
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": family,
|
||||
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
|
||||
"GIT_SSH": "/attacker/ssh",
|
||||
"PATH": "/attacker/bin",
|
||||
})
|
||||
|
||||
self.assertEqual(
|
||||
[
|
||||
"/usr/bin/ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8",
|
||||
"-o", f"AddressFamily={family}",
|
||||
],
|
||||
shlex.split(environment["GIT_SSH_COMMAND"]),
|
||||
)
|
||||
self.assertNotIn("GIT_SSH", environment)
|
||||
self.assertEqual("/usr/bin:/bin", environment["PATH"])
|
||||
self.assertEqual(environment, git_state.sanitized_git_environment(environment))
|
||||
|
||||
def test_invalid_ssh_address_family_is_rejected_before_git_runs(self) -> None:
|
||||
for invalid in (
|
||||
"", "INET", "ipv4", " inet", "inet ", "inet\n",
|
||||
"inet; touch /not-executed", "inet -o StrictHostKeyChecking=no",
|
||||
"$(not-executed)",
|
||||
):
|
||||
with (
|
||||
self.subTest(value=invalid),
|
||||
patch.dict("os.environ", {"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": invalid}),
|
||||
patch.object(git_state.subprocess, "run") as run,
|
||||
):
|
||||
with self.assertRaisesRegex(
|
||||
ValueError, "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6",
|
||||
):
|
||||
git_state.git(Path("/workspace/govoplan-core"), "status", "--porcelain")
|
||||
run.assert_not_called()
|
||||
|
||||
def test_source_provenance_readback_keeps_family_but_discards_ssh_command_override(self) -> None:
|
||||
from govoplan_release.source_provenance import inspect_remote_tag
|
||||
|
||||
completed = subprocess.CompletedProcess(
|
||||
[], 0, f"{'a' * 40}\trefs/tags/v1.2.3\n{'b' * 40}\trefs/tags/v1.2.3^{{}}\n", "",
|
||||
)
|
||||
with (
|
||||
patch.dict("os.environ", {
|
||||
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY": "inet",
|
||||
"GIT_SSH_COMMAND": "/attacker/ssh -o StrictHostKeyChecking=no",
|
||||
}),
|
||||
patch("govoplan_release.repository_tag.subprocess.run", return_value=completed) as run,
|
||||
):
|
||||
result = inspect_remote_tag(
|
||||
path=Path("/workspace/govoplan-core"), remote="origin",
|
||||
remote_url="git@git.add-ideas.de:GovOPlaN/govoplan-core.git", tag="v1.2.3",
|
||||
)
|
||||
|
||||
self.assertEqual("b" * 40, result.commit)
|
||||
self.assertEqual(
|
||||
"/usr/bin/ssh -o BatchMode=yes -o ConnectTimeout=8 -o AddressFamily=inet",
|
||||
run.call_args.kwargs["env"]["GIT_SSH_COMMAND"],
|
||||
)
|
||||
|
||||
def test_manifest_version_does_not_confuse_interface_versions(self) -> None:
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import runpy
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
@@ -214,21 +215,23 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
git(self.repo, "commit", "-m", "Add release WebUI composition")
|
||||
git(self.repo, "push", "origin", "main")
|
||||
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-core", "govoplan-campaign"),
|
||||
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=True,
|
||||
)
|
||||
for push in (False, True):
|
||||
with self.subTest(push=push):
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-core", "govoplan-campaign"),
|
||||
repo_versions={"govoplan-core": "0.1.10", "govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=push,
|
||||
)
|
||||
|
||||
self.assertEqual("blocked", result["status"])
|
||||
self.assertIn("no selected repository was mutated", result["detail"])
|
||||
self.assertEqual("skipped", result["repositories"][0]["status"])
|
||||
self.assertEqual("blocked", result["repositories"][1]["status"])
|
||||
self.assertIn("release WebUI composition gate failed", result["repositories"][1]["detail"])
|
||||
for repository in (self.repo, self.remote, campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
self.assertEqual("blocked", result["status"])
|
||||
self.assertIn("no selected repository was mutated", result["detail"])
|
||||
self.assertEqual("skipped", result["repositories"][0]["status"])
|
||||
self.assertEqual("blocked", result["repositories"][1]["status"])
|
||||
self.assertIn("release WebUI composition gate failed", result["repositories"][1]["detail"])
|
||||
for repository in (self.repo, self.remote, campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_api_keeps_legacy_source_release_preview_only(self) -> None:
|
||||
with TestClient(
|
||||
@@ -300,6 +303,183 @@ class ReleaseRepositoryTagTests(unittest.TestCase):
|
||||
self.assertIn("Signed Website Catalog", ui.text)
|
||||
self.assertIn("Apply + Website Tag", ui.text)
|
||||
|
||||
def test_local_module_candidate_precedes_core_lock_but_publication_does_not(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
remotes = (self.remote, self.manifest_remote, campaign_remote)
|
||||
remote_refs = {path: git_text(path, "show-ref") for path in remotes}
|
||||
arguments = {
|
||||
"repos": ("govoplan-campaign",),
|
||||
"repo_versions": {"govoplan-campaign": "0.1.10"},
|
||||
"workspace_root": self.workspace,
|
||||
}
|
||||
|
||||
preview = tag_repositories(**arguments, apply=False, push=False)
|
||||
self.assertEqual("planned", preview["status"], preview)
|
||||
self.assertFalse(ref_exists(campaign, "refs/tags/v0.1.10"))
|
||||
candidate = tag_repositories(**arguments, apply=True, push=False)
|
||||
self.assertEqual("tagged", candidate["status"], candidate)
|
||||
self.assertEqual("tag", git_text(campaign, "cat-file", "-t", "v0.1.10"))
|
||||
tag_object = git_text(campaign, "rev-parse", "v0.1.10")
|
||||
head = git_text(campaign, "rev-parse", "HEAD")
|
||||
self.assertEqual(head, git_text(campaign, "rev-parse", "v0.1.10^{commit}"))
|
||||
|
||||
for apply in (False, True):
|
||||
with self.subTest(publish_apply=apply):
|
||||
blocked = tag_repositories(**arguments, apply=apply, push=True)
|
||||
self.assertEqual("blocked", blocked["status"], blocked)
|
||||
self.assertIn(
|
||||
"release WebUI composition gate failed",
|
||||
blocked["repositories"][0]["detail"],
|
||||
)
|
||||
self.assertEqual(tag_object, git_text(campaign, "rev-parse", "v0.1.10"))
|
||||
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
|
||||
|
||||
lock_path = self.repo / "webui" / "package-lock.release.json"
|
||||
lock = json.loads(lock_path.read_text(encoding="utf-8"))
|
||||
locked_campaign = lock["packages"]["node_modules/@govoplan/campaign-webui"]
|
||||
locked_campaign["version"] = "0.1.10"
|
||||
locked_campaign["resolved"] = f"git+ssh://git@example.test/acme/govoplan-campaign.git#{head}"
|
||||
lock_path.write_text(json.dumps(lock) + "\n", encoding="utf-8")
|
||||
git(self.repo, "add", "webui/package-lock.release.json")
|
||||
git(self.repo, "commit", "-m", "Resolve reviewed local Campaign candidate")
|
||||
|
||||
core_candidate = tag_repositories(
|
||||
repos=("govoplan-core",),
|
||||
repo_versions={"govoplan-core": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("tagged", core_candidate["status"], core_candidate)
|
||||
self.assertEqual(remote_refs, {path: git_text(path, "show-ref") for path in remotes})
|
||||
published = tag_repositories(**arguments, apply=True, push=True)
|
||||
self.assertEqual("published", published["status"], published)
|
||||
self.assertEqual(tag_object, git_text(campaign_remote, "rev-parse", "v0.1.10"))
|
||||
self.assertEqual(head, git_text(campaign_remote, "rev-parse", "refs/heads/main"))
|
||||
self.assertEqual(remote_refs[self.remote], git_text(self.remote, "show-ref"))
|
||||
|
||||
def test_local_core_candidate_still_requires_resolved_module_tags(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
for selected in (("govoplan-core",), ("govoplan-campaign", "govoplan-core")):
|
||||
with self.subTest(selected=selected):
|
||||
result = tag_repositories(
|
||||
repos=selected,
|
||||
repo_versions={repo: "0.1.10" for repo in selected},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
core_row = next(row for row in result["repositories"] if row["repo"] == "govoplan-core")
|
||||
self.assertIn("version alignment gate failed", core_row["detail"])
|
||||
self.assertIn("@govoplan/campaign-webui:resolved", core_row["detail"])
|
||||
self.assertIn("expected 'local tag v0.1.10'", core_row["detail"])
|
||||
for repository in (self.repo, self.remote, campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_local_module_candidate_preserves_version_and_worktree_gates(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
arguments = {
|
||||
"repos": ("govoplan-campaign",),
|
||||
"workspace_root": self.workspace,
|
||||
"apply": True,
|
||||
"push": False,
|
||||
}
|
||||
mismatch = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.11"})
|
||||
self.assertEqual("blocked", mismatch["status"], mismatch)
|
||||
self.assertIn("version alignment gate failed", mismatch["repositories"][0]["detail"])
|
||||
|
||||
(campaign / "unreviewed.txt").write_text("operator work\n", encoding="utf-8")
|
||||
dirty = tag_repositories(**arguments, repo_versions={"govoplan-campaign": "0.1.10"})
|
||||
self.assertEqual("blocked", dirty["status"], dirty)
|
||||
self.assertIn("worktree is not clean", dirty["repositories"][0]["detail"])
|
||||
for repository in (campaign, campaign_remote):
|
||||
for tag in ("v0.1.10", "v0.1.11"):
|
||||
self.assertFalse(ref_exists(repository, f"refs/tags/{tag}"))
|
||||
|
||||
def test_local_module_candidate_preserves_manifest_gate(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
replace_with_unscoped_workflow_manifest(self.manifest_repo)
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-campaign",),
|
||||
repo_versions={"govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("scope-conditioned alternative", result["repositories"][0]["detail"])
|
||||
for repository in (campaign, campaign_remote):
|
||||
self.assertFalse(ref_exists(repository, "refs/tags/v0.1.10"))
|
||||
|
||||
def test_local_module_candidate_preserves_remote_tag_immutability(self) -> None:
|
||||
campaign, campaign_remote = self._staged_campaign_bundle()
|
||||
git(campaign, "tag", "-a", "v0.1.10", "-m", "Existing immutable tag", "v0.1.9^{commit}")
|
||||
git(campaign, "push", "origin", "refs/tags/v0.1.10")
|
||||
remote_refs = git_text(campaign_remote, "show-ref")
|
||||
for local_exists in (True, False):
|
||||
with self.subTest(local_exists=local_exists):
|
||||
if not local_exists:
|
||||
git(campaign, "tag", "-d", "v0.1.10")
|
||||
result = tag_repositories(
|
||||
repos=("govoplan-campaign",),
|
||||
repo_versions={"govoplan-campaign": "0.1.10"},
|
||||
workspace_root=self.workspace,
|
||||
apply=True,
|
||||
push=False,
|
||||
)
|
||||
self.assertEqual("blocked", result["status"], result)
|
||||
self.assertIn("immutable tag", result["repositories"][0]["detail"])
|
||||
self.assertIn("not HEAD", result["repositories"][0]["detail"])
|
||||
self.assertEqual(remote_refs, git_text(campaign_remote, "show-ref"))
|
||||
self.assertEqual(local_exists, ref_exists(campaign, "refs/tags/v0.1.10"))
|
||||
|
||||
def _staged_campaign_bundle(self) -> tuple[Path, Path]:
|
||||
campaign, remote = create_release_repo(
|
||||
root=self.root,
|
||||
workspace=self.workspace,
|
||||
name="govoplan-campaign",
|
||||
version="0.1.9",
|
||||
)
|
||||
campaign_webui = campaign / "webui"
|
||||
campaign_webui.mkdir()
|
||||
package_path = campaign_webui / "package.json"
|
||||
package_path.write_text(
|
||||
'{"name":"@govoplan/campaign-webui","version":"0.1.9"}\n', encoding="utf-8"
|
||||
)
|
||||
git(campaign, "add", "webui/package.json")
|
||||
git(campaign, "commit", "-m", "Prior Campaign WebUI package")
|
||||
git(campaign, "tag", "-a", "v0.1.9", "-m", "Prior Campaign release")
|
||||
git(campaign, "push", "origin", "main", "refs/tags/v0.1.9")
|
||||
prior_commit = git_text(campaign, "rev-parse", "HEAD")
|
||||
for path in (campaign / "pyproject.toml", package_path):
|
||||
path.write_text(path.read_text(encoding="utf-8").replace("0.1.9", "0.1.10"), encoding="utf-8")
|
||||
git(campaign, "add", "pyproject.toml", "webui/package.json")
|
||||
git(campaign, "commit", "-m", "Reviewed Campaign candidate")
|
||||
|
||||
core_webui = self.repo / "webui"
|
||||
core_webui.mkdir()
|
||||
dependency_ref = "git+ssh://git@example.test/acme/govoplan-campaign.git#v0.1.10"
|
||||
package = {
|
||||
"name": "@govoplan/core-webui",
|
||||
"version": "0.1.10",
|
||||
"dependencies": {"@govoplan/campaign-webui": dependency_ref},
|
||||
}
|
||||
(core_webui / "package.release.json").write_text(json.dumps(package) + "\n", encoding="utf-8")
|
||||
(core_webui / "package-lock.release.json").write_text(
|
||||
json.dumps({"packages": {
|
||||
"": package,
|
||||
"node_modules/@govoplan/campaign-webui": {
|
||||
"version": "0.1.9",
|
||||
"resolved": f"git+ssh://git@example.test/acme/govoplan-campaign.git#{prior_commit}",
|
||||
},
|
||||
}}) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
git(self.repo, "add", "webui/package.release.json", "webui/package-lock.release.json")
|
||||
git(self.repo, "commit", "-m", "Stage Core input before candidate lock resolution")
|
||||
return campaign, remote
|
||||
|
||||
|
||||
def git(cwd: Path, *args: str) -> None:
|
||||
result = subprocess.run(
|
||||
@@ -338,9 +518,50 @@ def add_scoped_workflow_manifest(repo: Path) -> None:
|
||||
backend.mkdir(parents=True)
|
||||
(package / "__init__.py").write_text("", encoding="utf-8")
|
||||
(backend / "__init__.py").write_text("", encoding="utf-8")
|
||||
# This small workspace still has to satisfy the real presentation contract.
|
||||
# Keep that prerequisite shared by both valid and intentionally unscoped
|
||||
# documentation fixtures, so each test reaches its intended release gate.
|
||||
canonical_areas = runpy.run_path(
|
||||
str(META_ROOT / "tools" / "checks" / "check-manifest-shapes.py")
|
||||
)["CANONICAL_PRODUCT_AREAS"]
|
||||
(backend / "release_fixture.py").write_text(
|
||||
"""from govoplan_core.core.modules import FrontendModule, ProductAreaContribution
|
||||
from govoplan_core.core.views import ViewSurface
|
||||
|
||||
|
||||
def fixture_frontend():
|
||||
return FrontendModule(
|
||||
module_id="access",
|
||||
view_surfaces=(
|
||||
ViewSurface(
|
||||
id="access.section.release-fixture",
|
||||
module_id="access",
|
||||
kind="section",
|
||||
label="Release fixture",
|
||||
),
|
||||
),
|
||||
product_areas=tuple(
|
||||
ProductAreaContribution(
|
||||
id=area_id,
|
||||
module_id="access",
|
||||
label=label,
|
||||
icon=icon,
|
||||
description=description,
|
||||
order=order,
|
||||
surface_ids=("access.section.release-fixture",),
|
||||
)
|
||||
for area_id, (label, icon, description, order) in CANONICAL_AREAS.items()
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
CANONICAL_AREAS = """ + repr(canonical_areas) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(backend / "manifest.py").write_text(
|
||||
"""from govoplan_core.core.modules import DocumentationCondition, DocumentationTopic, ModuleManifest, PermissionDefinition
|
||||
from govoplan_core.core.provider_governance import declared_module_architecture
|
||||
from .release_fixture import fixture_frontend
|
||||
|
||||
|
||||
def get_manifest():
|
||||
@@ -348,6 +569,7 @@ def get_manifest():
|
||||
id="access",
|
||||
name="Access",
|
||||
version="0.1.10",
|
||||
frontend=fixture_frontend(),
|
||||
permissions=(
|
||||
PermissionDefinition(
|
||||
scope="access:item:read",
|
||||
@@ -403,6 +625,7 @@ def replace_with_unscoped_workflow_manifest(repo: Path) -> None:
|
||||
manifest.write_text(
|
||||
"""from govoplan_core.core.modules import DocumentationTopic, ModuleManifest
|
||||
from govoplan_core.core.provider_governance import declared_module_architecture
|
||||
from .release_fixture import fixture_frontend
|
||||
|
||||
|
||||
def get_manifest():
|
||||
@@ -410,6 +633,7 @@ def get_manifest():
|
||||
id="access",
|
||||
name="Access",
|
||||
version="0.1.10",
|
||||
frontend=fixture_frontend(),
|
||||
documentation=(
|
||||
DocumentationTopic(
|
||||
id="access.workflow.unscoped",
|
||||
|
||||
@@ -1891,7 +1891,7 @@ class ReleaseRunApiTests(unittest.TestCase):
|
||||
|
||||
def test_ui_and_runbook_state_tracking_boundary_are_explicit(self) -> None:
|
||||
webui = (RELEASE_ROOT / "webui" / "index.html").read_text(encoding="utf-8")
|
||||
runbook = (META_ROOT / "docs" / "RELEASE_CONSOLE.md").read_text(
|
||||
runbook = (META_ROOT / "docs" / "operations" / "RELEASE_CONSOLE.md").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
|
||||
|
||||
@@ -13,12 +13,47 @@ if str(RELEASE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(RELEASE_ROOT))
|
||||
|
||||
from govoplan_release.version_metadata import ( # noqa: E402
|
||||
VersionMetadataError,
|
||||
apply_version_metadata_mutations,
|
||||
version_metadata_mutations,
|
||||
)
|
||||
|
||||
|
||||
class ReleaseVersionMetadataTests(unittest.TestCase):
|
||||
def test_updates_shared_module_version_without_rewriting_independent_interfaces(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
root = Path(temp_dir)
|
||||
backend = root / "src" / "govoplan_example" / "backend"
|
||||
backend.mkdir(parents=True)
|
||||
manifest = backend / "manifest.py"
|
||||
manifest.write_text(
|
||||
'MODULE_VERSION: str = "1.2.3"\n'
|
||||
'manifest = ModuleManifest(id="example", version=MODULE_VERSION,\n'
|
||||
' provides_interfaces=(ModuleInterfaceProvider(name="api", version="2.0"),))\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
changed = apply_version_metadata_mutations(root, target_version="1.2.4")
|
||||
self.assertEqual(("src/govoplan_example/backend/manifest.py",), changed)
|
||||
self.assertIn('MODULE_VERSION: str = "1.2.4"', manifest.read_text())
|
||||
self.assertIn('version="2.0"', manifest.read_text())
|
||||
self.assertEqual((), version_metadata_mutations(root, target_version="1.2.4"))
|
||||
|
||||
def test_dynamic_module_version_fails_before_any_metadata_is_written(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
root = Path(temp_dir)
|
||||
backend = root / "src" / "govoplan_example" / "backend"
|
||||
backend.mkdir(parents=True)
|
||||
project = root / "pyproject.toml"
|
||||
project.write_text('[project]\nname="govoplan-example"\nversion="1.2.3"\n')
|
||||
before = project.read_bytes()
|
||||
(backend / "manifest.py").write_text(
|
||||
'MODULE_VERSION = compute_version()\n'
|
||||
'manifest = ModuleManifest(id="example", version=MODULE_VERSION)\n',
|
||||
)
|
||||
with self.assertRaisesRegex(VersionMetadataError, "no literal MODULE_VERSION"):
|
||||
apply_version_metadata_mutations(root, target_version="1.2.4")
|
||||
self.assertEqual(before, project.read_bytes())
|
||||
|
||||
def test_updates_recognized_metadata_without_changing_interface_versions(
|
||||
self,
|
||||
) -> None:
|
||||
|
||||
@@ -98,6 +98,10 @@ class SecurityAuditWrapperTests(unittest.TestCase):
|
||||
if [[ "${1:-}" == 'git' && "${2:-}" == '--help' ]]; then
|
||||
exit 0
|
||||
fi
|
||||
if [[ " $* " != *" --redact=100 "* ]]; then
|
||||
echo 'secret scans must redact reports and logs' >&2
|
||||
exit 3
|
||||
fi
|
||||
output=''
|
||||
while [[ $# -gt 0 ]]; do
|
||||
if [[ "$1" == '--report-path' ]]; then
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECK = runpy.run_path(str(META_ROOT / "tools/checks/check-webui-package-facades.py"))
|
||||
|
||||
|
||||
class WebuiPackageFacadeTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.repository = Path(self.temporary.name) / "govoplan-example"
|
||||
source = self.repository / "webui/src"
|
||||
source.mkdir(parents=True)
|
||||
(source / "index.ts").write_text("export {};\n", encoding="utf-8")
|
||||
(source / "styles.css").write_text(":root {}\n", encoding="utf-8")
|
||||
self.webui = {
|
||||
"name": "@govoplan/example-webui", "version": "0.1.2", "type": "module",
|
||||
"main": "src/index.ts",
|
||||
"exports": {".": {"import": "./src/index.ts"}, "./styles.css": "./src/styles.css"},
|
||||
"peerDependencies": {"@govoplan/core-webui": "^0.1.45"},
|
||||
"peerDependenciesMeta": {"@govoplan/core-webui": {"optional": True}},
|
||||
}
|
||||
self.root = {**self.webui, **{
|
||||
field: CHECK["prefixed_entries"](self.webui[field])
|
||||
for field in CHECK["ENTRY_FIELDS"] if field in self.webui
|
||||
}}
|
||||
self.write_manifests()
|
||||
|
||||
def write_manifests(self) -> None:
|
||||
(self.repository / "webui/package.json").write_text(json.dumps(self.webui), encoding="utf-8")
|
||||
(self.repository / "package.json").write_text(json.dumps(self.root), encoding="utf-8")
|
||||
|
||||
def issues(self) -> list[str]:
|
||||
return CHECK["facade_issues"](self.repository, package_name="@govoplan/example-webui")
|
||||
|
||||
def test_matching_conditional_and_css_entries_are_accepted(self) -> None:
|
||||
self.assertEqual([], self.issues())
|
||||
|
||||
def test_missing_root_or_generic_package_is_rejected(self) -> None:
|
||||
(self.repository / "package.json").unlink()
|
||||
self.assertIn("cannot read", " ".join(self.issues()))
|
||||
self.root = {"name": "@govoplan/example", "version": "0.1.2"}
|
||||
self.write_manifests()
|
||||
self.assertIn("root name differs", " ".join(self.issues()))
|
||||
self.assertIn("no WebUI entry point", " ".join(self.issues()))
|
||||
|
||||
def test_peer_drift_or_missing_entry_is_rejected(self) -> None:
|
||||
self.root["peerDependencies"] = {"@govoplan/core-webui": "^9.0.0"}
|
||||
self.write_manifests()
|
||||
self.assertIn("peerDependencies differs", " ".join(self.issues()))
|
||||
(self.repository / "webui/src/styles.css").unlink()
|
||||
self.assertIn("missing exports entry", " ".join(self.issues()))
|
||||
|
||||
def test_entry_cannot_escape_webui_even_if_it_exists(self) -> None:
|
||||
(self.repository / "outside.ts").write_text("export {};\n", encoding="utf-8")
|
||||
self.root["main"] = "webui/../outside.ts"
|
||||
self.write_manifests()
|
||||
self.assertIn("escapes webui/", " ".join(self.issues()))
|
||||
|
||||
def test_release_composition_checks_only_declared_module_sources(self) -> None:
|
||||
core = self.repository.parent / "govoplan-core/webui"
|
||||
core.mkdir(parents=True)
|
||||
(core / "package.release.json").write_text(json.dumps({"dependencies": {
|
||||
"react": "19.2.7",
|
||||
"@govoplan/example-webui": "git+ssh://git@git.add-ideas.de/GovOPlaN/govoplan-example.git#v0.1.2",
|
||||
}}), encoding="utf-8")
|
||||
self.assertEqual((1, []), CHECK["check_composition"](self.repository.parent))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -39,11 +39,16 @@ GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-dsar-coverage.py"
|
||||
"$NODE/node" "$META_ROOT/tests/test-jsx-value-imports.mjs"
|
||||
"$NODE/node" "$META_ROOT/tools/checks/check-jsx-value-imports.mjs"
|
||||
"$NODE/node" "$META_ROOT/../govoplan-files/webui/scripts/test-archive-client.mjs"
|
||||
|
||||
cd "$META_ROOT"
|
||||
"$PYTHON" tools/inventory/platform-interface-inventory.py --strict-declarations --strict-endpoints
|
||||
"$PYTHON" tools/repo/sync-module-package-workflows.py --check
|
||||
"$PYTHON" tools/release/generate-developer-meta-package.py --check
|
||||
"$PYTHON" tools/checks/check-webui-package-facades.py
|
||||
"$PYTHON" -m unittest tests.test_webui_package_facades
|
||||
"$PYTHON" -m unittest tests.test_module_package_workflows tests.test_package_registry_release
|
||||
"$PYTHON" -m unittest tests.test_deployment_installer
|
||||
"$PYTHON" -m unittest tests.test_capability_fit_evidence
|
||||
@@ -96,6 +101,15 @@ PY
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-primitives.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-shared-webui-foundations.py"
|
||||
"$PYTHON" -m unittest tests.test_module_system
|
||||
"$PYTHON" -m unittest tests.test_ownership_history_migration tests.test_ownership tests.test_ownership_api
|
||||
"$PYTHON" -m unittest tests.test_navigation_preferences tests.test_api_smoke.ApiSmokeTests.test_navigation_separator_layout_survives_system_tenant_and_personal_saves
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_managed_archives.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_work.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_staging.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_upload_response_batching.py \
|
||||
/mnt/DATA/git/govoplan-files/tests/test_archive_performance.py
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-access/tests/test_external_function_mapping_migration.py
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-connectors/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-datasources/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dataflow/tests
|
||||
@@ -116,13 +130,31 @@ PY
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-identity-trust/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-wiki/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
|
||||
"$PYTHON" -m pytest -q \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_editor_state_security.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_mail_profile_boundary.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_independent_configuration_repairs.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_incremental_review_persistence.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_reviewed_build_mock.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_delivery_policy_settings.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_synchronous_delivery_policy.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_workerless_recovery.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_imap_batch_integration.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_testbed_claim_recovery.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_campaign_optimistic_concurrency.py \
|
||||
/mnt/DATA/git/govoplan-campaign/tests/test_archive_encryption_governance.py \
|
||||
/mnt/DATA/git/govoplan-policy/tests/test_campaign_archive_encryption.py \
|
||||
/mnt/DATA/git/govoplan-policy/tests/test_archive_encryption_api.py
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-mail/tests/test_campaign_protocol_authorization.py /mnt/DATA/git/govoplan-mail/tests/test_campaign_imap_batch.py
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
|
||||
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
|
||||
|
||||
cd "$ROOT/webui"
|
||||
"$NPM" run test:api-client-cache
|
||||
"$NPM" run test:dependency-security
|
||||
"$NPM" run test:layout-primitives
|
||||
"$NPM" run test:mail-components
|
||||
"$NPM" run test:module-capabilities
|
||||
@@ -155,12 +187,19 @@ cd /mnt/DATA/git/govoplan-postbox/webui
|
||||
cd /mnt/DATA/git/govoplan-mail/webui
|
||||
"$NPM" run test:mail-ui
|
||||
|
||||
cd /mnt/DATA/git/govoplan-files/webui
|
||||
"$NPM" run test:managed-archive
|
||||
|
||||
cd /mnt/DATA/git/govoplan-campaign/webui
|
||||
"$NPM" run test:policy-ui
|
||||
"$NPM" run test:template-preview
|
||||
"$NPM" run test:review-workflow
|
||||
"$NPM" run test:accessibility-contract
|
||||
"$NPM" run test:campaign-collaboration
|
||||
"$NPM" run test:campaign-work
|
||||
|
||||
cd /mnt/DATA/git/govoplan-policy/webui
|
||||
"$NPM" run test:archive-encryption
|
||||
|
||||
cd /mnt/DATA/git/govoplan-wiki/webui
|
||||
"$NPM" run test:interface-pattern
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env node
|
||||
/** Reject erased type-only imports used as runtime JSX component tags. */
|
||||
import { readFileSync, readdirSync, existsSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { resolve, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const workspaceRoot = resolve(import.meta.dirname, "../../..");
|
||||
const require = createRequire(resolve(workspaceRoot, "govoplan-core/webui/package.json"));
|
||||
const ts = require("typescript");
|
||||
|
||||
function isTypeOnlyImport(declaration) {
|
||||
if (ts.isImportSpecifier(declaration)) return declaration.isTypeOnly || declaration.parent.parent.isTypeOnly;
|
||||
if (ts.isNamespaceImport(declaration)) return declaration.parent.isTypeOnly;
|
||||
return (ts.isImportClause(declaration) || ts.isImportEqualsDeclaration(declaration)) && declaration.isTypeOnly;
|
||||
}
|
||||
|
||||
/** Resolve lexical bindings, including shadowing; do not typecheck unrelated
|
||||
* optional dependencies or report ordinary application diagnostics.
|
||||
*/
|
||||
export function findTypeOnlyJsxImports(sources) {
|
||||
const files = new Map(sources.map(({ path, source }) => [resolve(path),
|
||||
ts.createSourceFile(resolve(path), source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX)]));
|
||||
const options = { noEmit: true, noResolve: true, noLib: true, types: [], jsx: ts.JsxEmit.Preserve };
|
||||
const host = ts.createCompilerHost(options);
|
||||
host.getSourceFile = (path) => files.get(resolve(path));
|
||||
const program = ts.createProgram([...files.keys()], options, host);
|
||||
const checker = program.getTypeChecker();
|
||||
const findings = [];
|
||||
for (const [path, source] of files) {
|
||||
function visit(node) {
|
||||
if (ts.isJsxOpeningElement(node) || ts.isJsxSelfClosingElement(node)) {
|
||||
let root = node.tagName;
|
||||
// Lower-case direct tags are intrinsic HTML, not runtime bindings.
|
||||
if (!(ts.isIdentifier(root) && /^[a-z]/.test(root.text))) {
|
||||
while (ts.isPropertyAccessExpression(root)) root = root.expression;
|
||||
const declarations = checker.getSymbolAtLocation(root)?.declarations ?? [];
|
||||
if (declarations.some(isTypeOnlyImport)) {
|
||||
const position = source.getLineAndCharacterOfPosition(node.tagName.getStart(source));
|
||||
findings.push({ path, line: position.line + 1, column: position.character + 1, component: node.tagName.getText(source) });
|
||||
}
|
||||
}
|
||||
}
|
||||
ts.forEachChild(node, visit);
|
||||
}
|
||||
visit(source);
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
function sourceFiles(directory) {
|
||||
return readdirSync(directory, { withFileTypes: true }).flatMap((entry) => {
|
||||
const path = resolve(directory, entry.name);
|
||||
return entry.isDirectory() ? sourceFiles(path) : entry.name.endsWith(".tsx") ? [path] : [];
|
||||
});
|
||||
}
|
||||
|
||||
export function checkWorkspace(root = workspaceRoot) {
|
||||
const modules = readdirSync(root, { withFileTypes: true })
|
||||
.filter((entry) => entry.isDirectory() && entry.name.startsWith("govoplan"))
|
||||
.map((entry) => resolve(root, entry.name, "webui/src")).filter(existsSync);
|
||||
const paths = modules.flatMap(sourceFiles);
|
||||
const findings = findTypeOnlyJsxImports(paths.map((path) => ({ path, source: readFileSync(path, "utf8") })));
|
||||
for (const finding of findings) {
|
||||
console.error(`${relative(root, finding.path)}:${finding.line}:${finding.column}: JSX component ${finding.component} is imported type-only and will be erased at runtime.`);
|
||||
}
|
||||
if (!findings.length) console.log(`JSX runtime-import contract passed: ${paths.length} TSX files across ${modules.length} WebUI modules.`);
|
||||
return findings.length ? 1 : 0;
|
||||
}
|
||||
|
||||
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||
process.exitCode = checkWorkspace();
|
||||
}
|
||||
@@ -677,6 +677,7 @@ run_gitleaks() {
|
||||
prepare_machine_report "$REPORTS_DIR/gitleaks-history-$name.json" || return 2
|
||||
prepare_machine_report "$REPORTS_DIR/gitleaks-worktree-$name.json" || return 2
|
||||
gitleaks git \
|
||||
--redact=100 \
|
||||
--config "$ROOT/.gitleaks.toml" \
|
||||
--report-format json \
|
||||
--report-path "$REPORTS_DIR/gitleaks-history-$name.json" \
|
||||
@@ -687,6 +688,7 @@ run_gitleaks() {
|
||||
# Scan the directory as well so pre-commit audits cover the exact code
|
||||
# under review, while retaining the history scan above.
|
||||
gitleaks dir \
|
||||
--redact=100 \
|
||||
--config "$ROOT/.gitleaks.toml" \
|
||||
--report-format json \
|
||||
--report-path "$REPORTS_DIR/gitleaks-worktree-$name.json" \
|
||||
@@ -696,6 +698,7 @@ run_gitleaks() {
|
||||
else
|
||||
prepare_machine_report "$REPORTS_DIR/gitleaks-$name.json" || return 2
|
||||
gitleaks detect \
|
||||
--redact=100 \
|
||||
--source "$repo" \
|
||||
--config "$ROOT/.gitleaks.toml" \
|
||||
--report-format json \
|
||||
|
||||
@@ -86,6 +86,12 @@ CENTRAL_COMPONENTS = {
|
||||
"CountBadge": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/CountBadge.tsx"
|
||||
),
|
||||
"MultiSelectFilter": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/MultiSelectFilter.tsx"
|
||||
),
|
||||
"ListSelectionFilter": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/ListSelectionFilter.tsx"
|
||||
),
|
||||
"SelectionList": pathlib.Path(
|
||||
"govoplan-core/webui/src/components/SelectionList.tsx"
|
||||
),
|
||||
@@ -190,7 +196,21 @@ REQUIRED_CONSUMERS = {
|
||||
"CountBadge": (
|
||||
pathlib.Path("govoplan-core/webui/src/layout/Titlebar.tsx"),
|
||||
pathlib.Path("govoplan-mail/webui/src/features/mail/MailboxPage.tsx"),
|
||||
),
|
||||
# Search's old count badge was part of a retired module-local filter menu.
|
||||
# Both surfaces must now compose the owning facet adapter and Core dropdown.
|
||||
"SearchFilters": (
|
||||
pathlib.Path("govoplan-search/webui/src/features/search/SearchPage.tsx"),
|
||||
pathlib.Path("govoplan-search/webui/src/components/GlobalSearch.tsx"),
|
||||
),
|
||||
"MultiSelectFilter": (
|
||||
pathlib.Path("govoplan-search/webui/src/components/SearchFilters.tsx"),
|
||||
pathlib.Path("govoplan-notifications/webui/src/features/notifications/NotificationCenterPage.tsx"),
|
||||
pathlib.Path("govoplan-docs/webui/src/features/docs/DocsPage.tsx"),
|
||||
),
|
||||
"ListSelectionFilter": (
|
||||
pathlib.Path("govoplan-core/webui/src/components/MultiSelectFilter.tsx"),
|
||||
pathlib.Path("govoplan-core/webui/src/components/table/DataGrid.tsx"),
|
||||
),
|
||||
"SelectionListItemContent": (
|
||||
pathlib.Path("govoplan-approvals/webui/src/features/approvals/ApprovalsPage.tsx"),
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check that release Git dependencies expose their owning WebUI package."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[2]
|
||||
GIT_REPOSITORY = re.compile(r"/(govoplan-[a-z0-9-]+)\.git#v[^/]+$")
|
||||
PARITY_FIELDS = (
|
||||
"name", "version", "type", "dependencies", "optionalDependencies",
|
||||
"peerDependencies", "peerDependenciesMeta",
|
||||
)
|
||||
ENTRY_FIELDS = ("main", "module", "types", "exports")
|
||||
|
||||
|
||||
def prefixed_entries(value: object) -> object:
|
||||
if isinstance(value, str):
|
||||
return "./webui/" + value[2:] if value.startswith("./") else "webui/" + value
|
||||
if isinstance(value, dict):
|
||||
return {key: prefixed_entries(item) for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [prefixed_entries(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def entry_paths(value: object) -> list[str]:
|
||||
if isinstance(value, str):
|
||||
return [value]
|
||||
if isinstance(value, dict):
|
||||
return [path for item in value.values() for path in entry_paths(item)]
|
||||
if isinstance(value, list):
|
||||
return [path for item in value for path in entry_paths(item)]
|
||||
return []
|
||||
|
||||
|
||||
def facade_issues(repository: Path, *, package_name: str) -> list[str]:
|
||||
issues: list[str] = []
|
||||
try:
|
||||
root = json.loads((repository / "package.json").read_text(encoding="utf-8"))
|
||||
webui = json.loads((repository / "webui/package.json").read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError) as exc:
|
||||
return [f"{repository.name}: cannot read package facades: {exc}"]
|
||||
if not isinstance(root, dict) or not isinstance(webui, dict):
|
||||
return [f"{repository.name}: package manifests must be JSON objects"]
|
||||
if webui.get("name") != package_name:
|
||||
issues.append(f"{repository.name}: WebUI name does not match {package_name}")
|
||||
for field in PARITY_FIELDS:
|
||||
if root.get(field) != webui.get(field):
|
||||
issues.append(f"{repository.name}: root {field} differs from owning WebUI package")
|
||||
for field in ENTRY_FIELDS:
|
||||
expected = prefixed_entries(webui.get(field))
|
||||
if root.get(field) != expected:
|
||||
issues.append(f"{repository.name}: root {field} must target the corresponding webui/ entry")
|
||||
for entry in entry_paths(root.get(field)):
|
||||
path = repository / entry
|
||||
if not path.resolve().is_relative_to((repository / "webui").resolve()):
|
||||
issues.append(f"{repository.name}: {field} entry escapes webui/: {entry}")
|
||||
elif "*" not in entry and not path.is_file():
|
||||
issues.append(f"{repository.name}: missing {field} entry: {entry}")
|
||||
if not root.get("exports") and not root.get("main"):
|
||||
issues.append(f"{repository.name}: root package has no WebUI entry point")
|
||||
return issues
|
||||
|
||||
|
||||
def check_composition(workspace: Path, *, core_root: Path | None = None) -> tuple[int, list[str]]:
|
||||
core = core_root or workspace / "govoplan-core"
|
||||
release = json.loads((core / "webui/package.release.json").read_text(encoding="utf-8"))
|
||||
checked = 0
|
||||
issues: list[str] = []
|
||||
for name, reference in release.get("dependencies", {}).items():
|
||||
if not name.startswith("@govoplan/"):
|
||||
continue
|
||||
match = GIT_REPOSITORY.search(reference) if isinstance(reference, str) else None
|
||||
if match is None:
|
||||
issues.append(f"{name}: release dependency is not a versioned GovOPlaN Git source")
|
||||
continue
|
||||
checked += 1
|
||||
issues.extend(facade_issues(workspace / match.group(1), package_name=name))
|
||||
return checked, issues
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--workspace-root", type=Path, default=META_ROOT.parent)
|
||||
parser.add_argument("--core-root", type=Path)
|
||||
args = parser.parse_args()
|
||||
checked, issues = check_composition(args.workspace_root, core_root=args.core_root)
|
||||
if issues:
|
||||
print("\n".join(issues))
|
||||
return 1
|
||||
print(f"Release WebUI package facade checks passed for {checked} Git dependencies")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -19,7 +19,6 @@
|
||||
"resources": "Contract-only module: resource catalog and allocation persistence are not implemented; reassess before adding a migration-owned store.",
|
||||
"rest": "Transport-only module: REST binds explicitly published functions and owns no domain or subject-data store.",
|
||||
"soap": "Transport-only module: SOAP binds explicitly published operations and owns no domain or subject-data store.",
|
||||
"tenancy": "Orchestration module: tenant lifecycle and settings use Core-owned storage; Access covers account and membership subject data.",
|
||||
"transparency": "Contract-only module: requests, disclosure reviews, and publications are not persisted; reassess before adding a migration-owned store.",
|
||||
"workflow": "Presentation-only module: Workflow edits and projects Workflow Engine state; Workflow Engine owns persistence and DSAR coverage.",
|
||||
"xrechnung": "Stateless validation-contract module: XRechnung persists no invoice, report, diagnostic, or handoff; the invoking Files, Procurement, or Payments workflow remains the subject-data owner. Reassess before adding a validation store."
|
||||
|
||||
@@ -27,6 +27,7 @@ EXISTING_PROXY_FILENAME = "existing-proxy.json"
|
||||
PLAN_FILENAME = "plan.json"
|
||||
RECEIPT_FILENAME = "receipt.json"
|
||||
CAPABILITIES_FILENAME = "infrastructure-capabilities.json"
|
||||
DEPENDENCY_INVENTORY_FILENAME = "infrastructure-dependency-inventory.json"
|
||||
MANIFEST_FILENAME = "distribution-manifest.json"
|
||||
KEYRING_FILENAME = "distribution-keyring.json"
|
||||
BACKUP_EVIDENCE_FILENAME = "backup-evidence.json"
|
||||
@@ -116,6 +117,7 @@ class BundlePaths:
|
||||
plan: Path
|
||||
receipt: Path
|
||||
capabilities: Path
|
||||
dependency_inventory: Path
|
||||
manifest: Path
|
||||
keyring: Path
|
||||
backup_evidence: Path
|
||||
@@ -141,6 +143,7 @@ def bundle_paths(root: Path) -> BundlePaths:
|
||||
plan=resolved / PLAN_FILENAME,
|
||||
receipt=resolved / RECEIPT_FILENAME,
|
||||
capabilities=resolved / CAPABILITIES_FILENAME,
|
||||
dependency_inventory=resolved / DEPENDENCY_INVENTORY_FILENAME,
|
||||
manifest=resolved / MANIFEST_FILENAME,
|
||||
keyring=resolved / KEYRING_FILENAME,
|
||||
backup_evidence=resolved / BACKUP_EVIDENCE_FILENAME,
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
from datetime import UTC, datetime
|
||||
from typing import Mapping
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
@@ -18,6 +19,10 @@ CAPABILITY_STATES = frozenset(
|
||||
"unavailable",
|
||||
}
|
||||
)
|
||||
DEPENDENCY_INVENTORY_SCHEMA_VERSION = 1
|
||||
DEPENDENCY_STATES = frozenset(
|
||||
{"active", "inactive", "data_present", "pending_work", "runtime_binding"}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
@@ -50,13 +55,161 @@ class CapabilityChangeImpact:
|
||||
dependent_modules: tuple[str, ...]
|
||||
detail: str
|
||||
required_action: str
|
||||
actual_dependencies: tuple["CapabilityDependency", ...] = ()
|
||||
inventory_inspected: bool = False
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
value = asdict(self)
|
||||
value["dependent_modules"] = list(self.dependent_modules)
|
||||
value["actual_dependencies"] = [
|
||||
item.to_dict() for item in self.actual_dependencies
|
||||
]
|
||||
return value
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CapabilityDependency:
|
||||
capability_id: str
|
||||
module_id: str
|
||||
dependency_type: str
|
||||
dependency_ref: str
|
||||
state: str
|
||||
scope: str
|
||||
summary: str
|
||||
metrics: Mapping[str, int]
|
||||
required_action: str
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"capability_id": self.capability_id,
|
||||
"module_id": self.module_id,
|
||||
"dependency_type": self.dependency_type,
|
||||
"dependency_ref": self.dependency_ref,
|
||||
"state": self.state,
|
||||
"scope": self.scope,
|
||||
"summary": self.summary,
|
||||
"metrics": dict(sorted(self.metrics.items())),
|
||||
"required_action": self.required_action,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class InfrastructureDependencyInventory:
|
||||
installation_id: str
|
||||
generated_at: datetime
|
||||
complete: bool
|
||||
inspected_capability_ids: tuple[str, ...]
|
||||
provider_count: int
|
||||
dependencies: tuple[CapabilityDependency, ...]
|
||||
|
||||
def dependencies_for(
|
||||
self,
|
||||
capability_id: str,
|
||||
) -> tuple[CapabilityDependency, ...]:
|
||||
return tuple(
|
||||
item for item in self.dependencies if item.capability_id == capability_id
|
||||
)
|
||||
|
||||
|
||||
def infrastructure_dependency_inventory_from_mapping(
|
||||
value: object,
|
||||
) -> InfrastructureDependencyInventory:
|
||||
if (
|
||||
not isinstance(value, Mapping)
|
||||
or value.get("schema_version") != DEPENDENCY_INVENTORY_SCHEMA_VERSION
|
||||
):
|
||||
raise ValueError("Infrastructure dependency inventory schema is unsupported.")
|
||||
installation_id = _inventory_text(value, "installation_id", maximum=100)
|
||||
generated_at_text = _inventory_text(value, "generated_at", maximum=100)
|
||||
try:
|
||||
generated_at = datetime.fromisoformat(generated_at_text.replace("Z", "+00:00"))
|
||||
except ValueError as exc:
|
||||
raise ValueError(
|
||||
"Infrastructure dependency inventory timestamp is invalid."
|
||||
) from exc
|
||||
if generated_at.tzinfo is None:
|
||||
raise ValueError("Infrastructure dependency inventory timestamp needs a timezone.")
|
||||
generated_at = generated_at.astimezone(UTC)
|
||||
complete = value.get("complete")
|
||||
if type(complete) is not bool:
|
||||
raise ValueError("Infrastructure dependency inventory completion state is invalid.")
|
||||
inspected = _inventory_string_list(
|
||||
value.get("inspected_capability_ids"),
|
||||
maximum_items=100,
|
||||
maximum_length=120,
|
||||
)
|
||||
if len(inspected) != len(set(inspected)):
|
||||
raise ValueError("Infrastructure dependency inventory repeats a capability id.")
|
||||
providers = value.get("providers")
|
||||
if not isinstance(providers, list) or len(providers) > 100:
|
||||
raise ValueError("Infrastructure dependency provider reports are invalid.")
|
||||
provider_states: list[str] = []
|
||||
provider_declarations: dict[str, tuple[str, ...]] = {}
|
||||
provider_counts: dict[str, int] = {}
|
||||
for provider in providers:
|
||||
if not isinstance(provider, Mapping):
|
||||
raise ValueError("Infrastructure dependency provider report is invalid.")
|
||||
module_id = _inventory_text(provider, "module_id", maximum=120)
|
||||
if module_id in provider_declarations:
|
||||
raise ValueError("Infrastructure dependency provider is repeated.")
|
||||
state = _inventory_text(provider, "state", maximum=40)
|
||||
if state not in {"complete", "error"}:
|
||||
raise ValueError("Infrastructure dependency provider state is invalid.")
|
||||
provider_states.append(state)
|
||||
count = provider.get("dependency_count")
|
||||
if type(count) is not int or count < 0:
|
||||
raise ValueError("Infrastructure dependency provider count is invalid.")
|
||||
capability_ids = _inventory_string_list(
|
||||
provider.get("capability_ids"),
|
||||
maximum_items=30,
|
||||
maximum_length=120,
|
||||
)
|
||||
if len(capability_ids) != len(set(capability_ids)):
|
||||
raise ValueError("Infrastructure dependency provider capability is repeated.")
|
||||
provider_declarations[module_id] = capability_ids
|
||||
provider_counts[module_id] = count
|
||||
if complete and any(state != "complete" for state in provider_states):
|
||||
raise ValueError("Complete dependency inventory contains a failed provider.")
|
||||
raw_dependencies = value.get("dependencies")
|
||||
if not isinstance(raw_dependencies, list) or len(raw_dependencies) > 10_000:
|
||||
raise ValueError("Infrastructure dependency records are invalid.")
|
||||
dependencies = tuple(_inventory_dependency(item) for item in raw_dependencies)
|
||||
if any(
|
||||
capability_id not in inspected
|
||||
for capability_ids in provider_declarations.values()
|
||||
for capability_id in capability_ids
|
||||
):
|
||||
raise ValueError(
|
||||
"Infrastructure dependency provider was not covered by the inspection."
|
||||
)
|
||||
if any(item.capability_id not in inspected for item in dependencies):
|
||||
raise ValueError("Dependency record was not covered by the inventory inspection.")
|
||||
identities = {
|
||||
(item.capability_id, item.module_id, item.dependency_type, item.dependency_ref)
|
||||
for item in dependencies
|
||||
}
|
||||
if len(identities) != len(dependencies):
|
||||
raise ValueError("Infrastructure dependency inventory repeats a record.")
|
||||
observed_counts = {module_id: 0 for module_id in provider_counts}
|
||||
for dependency in dependencies:
|
||||
declarations = provider_declarations.get(dependency.module_id)
|
||||
if declarations is None or dependency.capability_id not in declarations:
|
||||
raise ValueError(
|
||||
"Infrastructure dependency is outside its provider declaration."
|
||||
)
|
||||
observed_counts[dependency.module_id] += 1
|
||||
if observed_counts != provider_counts:
|
||||
raise ValueError("Infrastructure dependency provider count does not match records.")
|
||||
return InfrastructureDependencyInventory(
|
||||
installation_id=installation_id,
|
||||
generated_at=generated_at,
|
||||
complete=complete,
|
||||
inspected_capability_ids=inspected,
|
||||
provider_count=len(providers),
|
||||
dependencies=dependencies,
|
||||
)
|
||||
|
||||
|
||||
def infrastructure_capability_document(
|
||||
spec: InstallationSpec,
|
||||
environment: Mapping[str, str],
|
||||
@@ -89,6 +242,8 @@ def infrastructure_capability_document(
|
||||
def capability_change_impacts(
|
||||
previous_document: object,
|
||||
desired_document: Mapping[str, object],
|
||||
*,
|
||||
dependency_inventory: InfrastructureDependencyInventory | None = None,
|
||||
) -> tuple[CapabilityChangeImpact, ...]:
|
||||
previous = _capability_map(previous_document)
|
||||
desired = _capability_map(desired_document)
|
||||
@@ -141,6 +296,43 @@ def capability_change_impacts(
|
||||
previous_secret_refs,
|
||||
desired_secret_refs,
|
||||
)
|
||||
actual_dependencies = (
|
||||
dependency_inventory.dependencies_for(capability_id)
|
||||
if dependency_inventory is not None
|
||||
else ()
|
||||
)
|
||||
inventory_inspected = bool(
|
||||
dependency_inventory is not None
|
||||
and capability_id in dependency_inventory.inspected_capability_ids
|
||||
)
|
||||
if inventory_inspected and actual_dependencies:
|
||||
references = ", ".join(
|
||||
f"{item.module_id}:{item.dependency_ref}"
|
||||
for item in actual_dependencies
|
||||
)
|
||||
inventory_detail = (
|
||||
f" Provider inventory reports {len(actual_dependencies)} persisted "
|
||||
f"dependency record(s): {references}."
|
||||
)
|
||||
elif inventory_inspected:
|
||||
inventory_detail = (
|
||||
" Provider inventory reports no persisted module-owned dependencies."
|
||||
)
|
||||
else:
|
||||
inventory_detail = " Provider inventory did not inspect this capability."
|
||||
dependency_actions = tuple(
|
||||
dict.fromkeys(
|
||||
item.required_action
|
||||
for item in actual_dependencies
|
||||
if item.required_action.strip()
|
||||
)
|
||||
)
|
||||
required_action = (
|
||||
"Review module-owned configuration and data migration or recovery "
|
||||
"evidence before apply."
|
||||
)
|
||||
if dependency_actions:
|
||||
required_action = f"{required_action} {' '.join(dependency_actions)}"
|
||||
impacts.append(
|
||||
CapabilityChangeImpact(
|
||||
capability_id=capability_id,
|
||||
@@ -153,11 +345,11 @@ def capability_change_impacts(
|
||||
detail=(
|
||||
f"{capability_id} changes from {previous_state}/{previous_source} "
|
||||
f"to {desired_state}/{desired_source}{binding_change}; "
|
||||
f"declared consumers: {dependent_label}."
|
||||
),
|
||||
required_action=(
|
||||
"Review module-owned configuration and data migration or recovery evidence before apply."
|
||||
f"declared consumers: {dependent_label}.{inventory_detail}"
|
||||
),
|
||||
required_action=required_action,
|
||||
actual_dependencies=actual_dependencies,
|
||||
inventory_inspected=inventory_inspected,
|
||||
)
|
||||
)
|
||||
return tuple(impacts)
|
||||
@@ -487,3 +679,73 @@ def _binding_change_label(
|
||||
if previous_secret_refs != desired_secret_refs:
|
||||
changes.append("secret-reference binding")
|
||||
return f" with changed {' and '.join(changes)}" if changes else ""
|
||||
|
||||
|
||||
def _inventory_text(
|
||||
value: Mapping[str, object],
|
||||
key: str,
|
||||
*,
|
||||
maximum: int,
|
||||
) -> str:
|
||||
raw = value.get(key)
|
||||
if not isinstance(raw, str):
|
||||
raise ValueError(f"Infrastructure dependency inventory {key} is invalid.")
|
||||
result = raw.strip()
|
||||
if not result or len(result) > maximum or any(ord(char) < 32 for char in result):
|
||||
raise ValueError(f"Infrastructure dependency inventory {key} is invalid.")
|
||||
return result
|
||||
|
||||
|
||||
def _inventory_string_list(
|
||||
value: object,
|
||||
*,
|
||||
maximum_items: int,
|
||||
maximum_length: int,
|
||||
) -> tuple[str, ...]:
|
||||
if not isinstance(value, list) or len(value) > maximum_items:
|
||||
raise ValueError("Infrastructure dependency inventory list is invalid.")
|
||||
items: list[str] = []
|
||||
for raw in value:
|
||||
if not isinstance(raw, str):
|
||||
raise ValueError("Infrastructure dependency inventory list is invalid.")
|
||||
item = raw.strip()
|
||||
if (
|
||||
not item
|
||||
or len(item) > maximum_length
|
||||
or any(ord(char) < 32 for char in item)
|
||||
):
|
||||
raise ValueError("Infrastructure dependency inventory list is invalid.")
|
||||
items.append(item)
|
||||
return tuple(items)
|
||||
|
||||
|
||||
def _inventory_dependency(value: object) -> CapabilityDependency:
|
||||
if not isinstance(value, Mapping):
|
||||
raise ValueError("Infrastructure dependency record is invalid.")
|
||||
state = _inventory_text(value, "state", maximum=40)
|
||||
if state not in DEPENDENCY_STATES:
|
||||
raise ValueError("Infrastructure dependency state is invalid.")
|
||||
raw_metrics = value.get("metrics")
|
||||
if not isinstance(raw_metrics, Mapping) or len(raw_metrics) > 20:
|
||||
raise ValueError("Infrastructure dependency metrics are invalid.")
|
||||
metrics: dict[str, int] = {}
|
||||
for raw_key, raw_count in raw_metrics.items():
|
||||
if not isinstance(raw_key, str):
|
||||
raise ValueError("Infrastructure dependency metric name is invalid.")
|
||||
key = raw_key.strip()
|
||||
if not key or len(key) > 80 or any(ord(char) < 32 for char in key):
|
||||
raise ValueError("Infrastructure dependency metric name is invalid.")
|
||||
if type(raw_count) is not int or raw_count < 0:
|
||||
raise ValueError("Infrastructure dependency metric value is invalid.")
|
||||
metrics[key] = raw_count
|
||||
return CapabilityDependency(
|
||||
capability_id=_inventory_text(value, "capability_id", maximum=120),
|
||||
module_id=_inventory_text(value, "module_id", maximum=120),
|
||||
dependency_type=_inventory_text(value, "dependency_type", maximum=120),
|
||||
dependency_ref=_inventory_text(value, "dependency_ref", maximum=240),
|
||||
state=state,
|
||||
scope=_inventory_text(value, "scope", maximum=120),
|
||||
summary=_inventory_text(value, "summary", maximum=1000),
|
||||
metrics=metrics,
|
||||
required_action=_inventory_text(value, "required_action", maximum=1000),
|
||||
)
|
||||
|
||||
@@ -18,7 +18,8 @@ import sys
|
||||
import time
|
||||
from typing import Iterator, Mapping, Sequence
|
||||
from urllib.error import URLError
|
||||
from urllib.request import urlopen
|
||||
from urllib.parse import urlsplit
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
from .backup_evidence import (
|
||||
DEFAULT_MAX_BACKUP_AGE_SECONDS,
|
||||
@@ -28,6 +29,7 @@ from .backup_evidence import (
|
||||
)
|
||||
from .bundle import (
|
||||
BACKUP_RUNTIME_ENV_KEYS,
|
||||
BundlePaths,
|
||||
atomic_write,
|
||||
bundle_paths,
|
||||
canonical_json,
|
||||
@@ -45,7 +47,11 @@ from .bundle import (
|
||||
service_names,
|
||||
write_env,
|
||||
)
|
||||
from .capabilities import infrastructure_capability_document
|
||||
from .capabilities import (
|
||||
InfrastructureDependencyInventory,
|
||||
infrastructure_capability_document,
|
||||
infrastructure_dependency_inventory_from_mapping,
|
||||
)
|
||||
from .cluster_evidence import collect_kubernetes_evidence
|
||||
from .distribution import (
|
||||
MAX_KEYRING_BYTES,
|
||||
@@ -81,6 +87,7 @@ from .kubernetes import (
|
||||
write_secret_creation_hint,
|
||||
)
|
||||
from .planning import (
|
||||
MAX_DEPENDENCY_INVENTORY_BYTES,
|
||||
DeploymentPlan,
|
||||
build_plan,
|
||||
release_change_requires_backup,
|
||||
@@ -152,6 +159,39 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
default=120.0,
|
||||
help="Maximum time to wait for the public health endpoint.",
|
||||
)
|
||||
apply_parser.add_argument(
|
||||
"--ops-url",
|
||||
help=(
|
||||
"Dependency inventory URL; defaults to "
|
||||
"<public-url>/api/v1/ops/infrastructure/dependencies."
|
||||
),
|
||||
)
|
||||
apply_parser.add_argument(
|
||||
"--api-key-env",
|
||||
default="GOVOPLAN_OPS_API_KEY",
|
||||
help=(
|
||||
"Environment variable containing an API key authorized to read "
|
||||
"Ops dependency inventory."
|
||||
),
|
||||
)
|
||||
|
||||
collect_inventory = subparsers.add_parser(
|
||||
"collect-infrastructure-inventory",
|
||||
help="Collect current module-owned capability dependencies from Ops.",
|
||||
)
|
||||
_directory_argument(collect_inventory)
|
||||
collect_inventory.add_argument(
|
||||
"--ops-url",
|
||||
help=(
|
||||
"Dependency inventory URL; defaults to "
|
||||
"<public-url>/api/v1/ops/infrastructure/dependencies."
|
||||
),
|
||||
)
|
||||
collect_inventory.add_argument(
|
||||
"--api-key-env",
|
||||
default="GOVOPLAN_OPS_API_KEY",
|
||||
help="Environment variable containing an authorized Ops API key.",
|
||||
)
|
||||
|
||||
status = subparsers.add_parser(
|
||||
"status", help="Show desired state and current Compose process state."
|
||||
@@ -425,6 +465,8 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
return _render_or_doctor(args)
|
||||
if args.command == "apply":
|
||||
return _apply(args)
|
||||
if args.command == "collect-infrastructure-inventory":
|
||||
return _collect_infrastructure_inventory(args)
|
||||
if args.command == "status":
|
||||
return _status(args)
|
||||
if args.command == "verify-release":
|
||||
@@ -586,6 +628,25 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
)
|
||||
secrets = reconcile_runtime_environment(spec, read_env(paths.env))
|
||||
secrets = _write_bundle(spec, paths, secrets)
|
||||
preliminary_plan = build_plan(spec, paths, include_host_checks=False)
|
||||
api_key_env = str(
|
||||
getattr(args, "api_key_env", "GOVOPLAN_OPS_API_KEY")
|
||||
).strip()
|
||||
api_key = os.environ.get(api_key_env, "").strip()
|
||||
if preliminary_plan.capability_impacts and api_key:
|
||||
try:
|
||||
_collect_dependency_inventory(
|
||||
spec,
|
||||
paths,
|
||||
ops_url=getattr(args, "ops_url", None),
|
||||
api_key=api_key,
|
||||
)
|
||||
print("Refreshed infrastructure dependency inventory from Ops.")
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
print(
|
||||
f"warning: could not refresh dependency inventory: {exc}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
plan = build_plan(spec, paths, include_host_checks=True)
|
||||
_write_plan(paths.plan, plan)
|
||||
effective_errors = [
|
||||
@@ -613,6 +674,8 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
if effective_errors:
|
||||
_print_plan(plan)
|
||||
raise ValueError("deployment plan is blocked; resolve doctor errors first")
|
||||
if plan.capability_impacts:
|
||||
_print_plan(plan)
|
||||
docker = shutil.which("docker")
|
||||
if docker is None:
|
||||
raise ValueError("Docker CLI is required for apply")
|
||||
@@ -761,6 +824,70 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _collect_infrastructure_inventory(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
spec = load_spec(paths.spec)
|
||||
api_key_env = str(args.api_key_env).strip()
|
||||
api_key = os.environ.get(api_key_env, "").strip()
|
||||
if not api_key:
|
||||
raise ValueError(f"{api_key_env} must contain an authorized Ops API key")
|
||||
inventory = _collect_dependency_inventory(
|
||||
spec,
|
||||
paths,
|
||||
ops_url=args.ops_url,
|
||||
api_key=api_key,
|
||||
)
|
||||
state = "complete" if inventory.complete else "incomplete"
|
||||
print(
|
||||
f"Collected {state} provider dependency inventory with "
|
||||
f"{len(inventory.dependencies)} record(s) at {paths.dependency_inventory}."
|
||||
)
|
||||
return 0 if inventory.complete else 1
|
||||
|
||||
|
||||
def _collect_dependency_inventory(
|
||||
spec: InstallationSpec,
|
||||
paths: BundlePaths,
|
||||
*,
|
||||
ops_url: str | None,
|
||||
api_key: str,
|
||||
) -> InfrastructureDependencyInventory:
|
||||
url = str(ops_url or "").strip() or (
|
||||
spec.public_url.rstrip("/")
|
||||
+ "/api/v1/ops/infrastructure/dependencies"
|
||||
)
|
||||
_validate_ops_inventory_url(url)
|
||||
request = Request(
|
||||
url,
|
||||
headers={"Accept": "application/json", "X-API-Key": api_key},
|
||||
)
|
||||
with urlopen(request, timeout=15) as response: # noqa: S310
|
||||
_validate_ops_inventory_url(response.geturl())
|
||||
encoded = response.read(MAX_DEPENDENCY_INVENTORY_BYTES + 1)
|
||||
if len(encoded) > MAX_DEPENDENCY_INVENTORY_BYTES:
|
||||
raise ValueError("Ops dependency inventory exceeds its size limit")
|
||||
value = json.loads(encoded)
|
||||
inventory = infrastructure_dependency_inventory_from_mapping(value)
|
||||
if inventory.installation_id != spec.installation_id:
|
||||
raise ValueError(
|
||||
"Ops dependency inventory belongs to a different installation"
|
||||
)
|
||||
ensure_private_directory(paths.root)
|
||||
atomic_write(paths.dependency_inventory, canonical_json(value), mode=0o600)
|
||||
return inventory
|
||||
|
||||
|
||||
def _validate_ops_inventory_url(url: str) -> None:
|
||||
parsed = urlsplit(url)
|
||||
if not parsed.hostname or parsed.username or parsed.password or parsed.fragment:
|
||||
raise ValueError("Ops dependency inventory URL is invalid")
|
||||
loopback = parsed.hostname in {"localhost", "127.0.0.1", "::1"}
|
||||
if parsed.scheme != "https" and not (parsed.scheme == "http" and loopback):
|
||||
raise ValueError(
|
||||
"Ops dependency inventory URL requires HTTPS except on loopback"
|
||||
)
|
||||
|
||||
|
||||
def _status(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
spec = load_spec(paths.spec)
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass
|
||||
from datetime import UTC, datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -37,8 +38,10 @@ from .bundle import (
|
||||
)
|
||||
from .capabilities import (
|
||||
CapabilityChangeImpact,
|
||||
InfrastructureDependencyInventory,
|
||||
capability_change_impacts,
|
||||
infrastructure_capability_document,
|
||||
infrastructure_dependency_inventory_from_mapping,
|
||||
)
|
||||
from .distribution import (
|
||||
MAX_KEYRING_BYTES,
|
||||
@@ -110,6 +113,9 @@ class DeploymentPlan:
|
||||
|
||||
|
||||
CommandRunner = Callable[[Sequence[str], Path], subprocess.CompletedProcess[str]]
|
||||
MAX_DEPENDENCY_INVENTORY_BYTES = 2 * 1024 * 1024
|
||||
DEPENDENCY_INVENTORY_MAX_AGE_SECONDS = 300
|
||||
DEPENDENCY_INVENTORY_MAX_FUTURE_SECONDS = 60
|
||||
|
||||
|
||||
def build_plan(
|
||||
@@ -135,9 +141,13 @@ def build_plan(
|
||||
spec,
|
||||
read_env(paths.env),
|
||||
)
|
||||
dependency_inventory, dependency_inventory_error = (
|
||||
_read_dependency_inventory(paths.dependency_inventory)
|
||||
)
|
||||
capability_impacts = capability_change_impacts(
|
||||
previous.get("infrastructure_capabilities"),
|
||||
infrastructure_capabilities,
|
||||
dependency_inventory=dependency_inventory,
|
||||
)
|
||||
|
||||
actions: list[PlanAction] = []
|
||||
@@ -215,6 +225,14 @@ def build_plan(
|
||||
)
|
||||
for impact in capability_impacts
|
||||
)
|
||||
checks.extend(
|
||||
_dependency_inventory_checks(
|
||||
spec,
|
||||
capability_impacts,
|
||||
dependency_inventory,
|
||||
dependency_inventory_error,
|
||||
)
|
||||
)
|
||||
if include_host_checks:
|
||||
checks.extend(host_checks(spec, paths, command_runner=command_runner))
|
||||
return DeploymentPlan(
|
||||
@@ -1187,6 +1205,106 @@ def _read_receipt(path: Path) -> Mapping[str, object]:
|
||||
return value if isinstance(value, dict) else {}
|
||||
|
||||
|
||||
def _read_dependency_inventory(
|
||||
path: Path,
|
||||
) -> tuple[InfrastructureDependencyInventory | None, str]:
|
||||
if not path.exists():
|
||||
return None, "missing"
|
||||
try:
|
||||
value = load_bounded_json(
|
||||
path,
|
||||
maximum_bytes=MAX_DEPENDENCY_INVENTORY_BYTES,
|
||||
)
|
||||
return infrastructure_dependency_inventory_from_mapping(value), ""
|
||||
except (DistributionError, ValueError) as exc:
|
||||
return None, str(exc)
|
||||
|
||||
|
||||
def _dependency_inventory_checks(
|
||||
spec: InstallationSpec,
|
||||
impacts: tuple[CapabilityChangeImpact, ...],
|
||||
inventory: InfrastructureDependencyInventory | None,
|
||||
inventory_error: str,
|
||||
) -> tuple[Check, ...]:
|
||||
if not impacts:
|
||||
return ()
|
||||
collect_action = (
|
||||
"Run govoplan-deploy collect-infrastructure-inventory with an Ops API "
|
||||
"key, then review the capability impacts before apply."
|
||||
)
|
||||
if inventory is None:
|
||||
if inventory_error == "missing":
|
||||
message = "Current provider dependency inventory is missing."
|
||||
check_id = "capability.dependency_inventory.missing"
|
||||
else:
|
||||
message = f"Provider dependency inventory is invalid: {inventory_error}"
|
||||
check_id = "capability.dependency_inventory.invalid"
|
||||
return (Check(check_id, "error", message, collect_action),)
|
||||
if inventory.installation_id != spec.installation_id:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.installation",
|
||||
"error",
|
||||
"Provider dependency inventory belongs to a different installation.",
|
||||
collect_action,
|
||||
),
|
||||
)
|
||||
if not inventory.complete:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.incomplete",
|
||||
"error",
|
||||
"Provider dependency inventory is incomplete because at least one provider failed.",
|
||||
"Resolve the provider failure and collect the inventory again.",
|
||||
),
|
||||
)
|
||||
age_seconds = (datetime.now(UTC) - inventory.generated_at).total_seconds()
|
||||
if age_seconds < -DEPENDENCY_INVENTORY_MAX_FUTURE_SECONDS:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.future",
|
||||
"error",
|
||||
"Provider dependency inventory timestamp is in the future.",
|
||||
"Correct host clock skew and collect the inventory again.",
|
||||
),
|
||||
)
|
||||
if age_seconds > DEPENDENCY_INVENTORY_MAX_AGE_SECONDS:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.stale",
|
||||
"error",
|
||||
"Provider dependency inventory is older than five minutes.",
|
||||
collect_action,
|
||||
),
|
||||
)
|
||||
impacted_ids = {item.capability_id for item in impacts}
|
||||
missing_ids = sorted(impacted_ids - set(inventory.inspected_capability_ids))
|
||||
if missing_ids:
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.coverage",
|
||||
"error",
|
||||
"Provider dependency inventory did not inspect impacted capabilities: "
|
||||
+ ", ".join(missing_ids)
|
||||
+ ".",
|
||||
collect_action,
|
||||
),
|
||||
)
|
||||
matching_dependencies = sum(
|
||||
len(inventory.dependencies_for(capability_id))
|
||||
for capability_id in impacted_ids
|
||||
)
|
||||
return (
|
||||
Check(
|
||||
"capability.dependency_inventory.current",
|
||||
"ok",
|
||||
"Current provider inventory inspected every impacted capability and "
|
||||
f"reported {matching_dependencies} persisted dependency record(s) from "
|
||||
f"{inventory.provider_count} provider(s).",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _memory_bytes() -> int | None:
|
||||
try:
|
||||
for line in Path("/proc/meminfo").read_text(encoding="utf-8").splitlines():
|
||||
|
||||
@@ -34,6 +34,7 @@ _BUNDLE_FILES = (
|
||||
"backup-verification.json",
|
||||
"receipt.json",
|
||||
"infrastructure-capabilities.json",
|
||||
"infrastructure-dependency-inventory.json",
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1379,6 +1379,13 @@
|
||||
"rationale": "Operational worker, scheduler, reconciliation, or health endpoint; it is not a direct user surface.",
|
||||
"repository": "govoplan-notifications"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/ops/infrastructure/dependencies",
|
||||
"rationale": "Authorized host-deployer preflight consumes this provider inventory directly; it is private operational evidence rather than a product page.",
|
||||
"repository": "govoplan-ops"
|
||||
},
|
||||
{
|
||||
"category": "worker_internal",
|
||||
"method": "GET",
|
||||
@@ -1713,6 +1720,13 @@
|
||||
"rationale": "The module WebUI constructs this endpoint through a mounted router prefix, generic action, or provider path.",
|
||||
"repository": "govoplan-projects"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
"path": "/records/{}/access-grants/{}/revoke",
|
||||
"rationale": "The restricted-record access dialog revokes a grant through the shared dynamically constructed record mutation path.",
|
||||
"repository": "govoplan-records"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
@@ -2000,6 +2014,20 @@
|
||||
"rationale": "Published integration, interoperability, public-participant, or health endpoint.",
|
||||
"repository": "govoplan-soap"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/admin/tenant-erasure-policy",
|
||||
"rationale": "Tenant-erasure policy is a consequential operator API with recent-authentication and dedicated-permission gates.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "PATCH",
|
||||
"path": "/admin/tenant-erasure-policy",
|
||||
"rationale": "Tenant-erasure policy is a consequential operator API with recent-authentication and dedicated-permission gates.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
@@ -2007,6 +2035,48 @@
|
||||
"rationale": "Tenant deletion preflight is an administrative safety API consumed before a destructive workflow.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations",
|
||||
"rationale": "Tenant erasure is an audited, provider-driven operator workflow whose API exposes the complete review and recovery evidence.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "GET",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}",
|
||||
"rationale": "Tenant erasure is an audited, provider-driven operator workflow whose API exposes the complete review and recovery evidence.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/approve",
|
||||
"rationale": "Tenant-erasure approval requires typed confirmation, recent authentication, and a distinct authorized account.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/cancel",
|
||||
"rationale": "Tenant-erasure cancellation is a recovery control available only before destructive work starts.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/execute",
|
||||
"rationale": "Tenant-erasure execution is a consequential operator API with provider checkpoints and fail-closed reconciliation.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "intentionally_headless",
|
||||
"method": "POST",
|
||||
"path": "/admin/tenants/{}/erasure-operations/{}/reconcile",
|
||||
"rationale": "Tenant-erasure reconciliation resumes idempotent provider steps after pending or outcome-unknown effects.",
|
||||
"repository": "govoplan-tenancy"
|
||||
},
|
||||
{
|
||||
"category": "compatibility",
|
||||
"method": "POST",
|
||||
|
||||
@@ -9,6 +9,7 @@ import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tomllib
|
||||
|
||||
|
||||
@@ -216,10 +217,17 @@ def _extras(value: str | None) -> list[str]:
|
||||
|
||||
|
||||
def _git(repository: Path, *arguments: str) -> str:
|
||||
release_root = str(Path(__file__).resolve().parent)
|
||||
if release_root not in sys.path:
|
||||
sys.path.insert(0, release_root)
|
||||
from govoplan_release.git_state import sanitized_git_environment, scoped_git_command
|
||||
|
||||
return subprocess.check_output(
|
||||
["git", "-C", str(repository), *arguments],
|
||||
scoped_git_command(repository, "-C", str(repository), *arguments),
|
||||
text=True,
|
||||
stderr=subprocess.DEVNULL,
|
||||
env=sanitized_git_environment(),
|
||||
timeout=30,
|
||||
).strip()
|
||||
|
||||
|
||||
|
||||
@@ -14,6 +14,8 @@ import re
|
||||
import stat
|
||||
import zipfile
|
||||
|
||||
from .registry_reference import registry_artifact_conflicts, registry_entry_source
|
||||
|
||||
|
||||
WHEEL_PAYLOAD_ALGORITHM = "govoplan-wheel-declared-payload-v1"
|
||||
INSTALLED_PAYLOAD_ALGORITHM = "govoplan-installed-record-payload-v1"
|
||||
@@ -185,10 +187,23 @@ def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
|
||||
modules = payload.get("modules")
|
||||
if isinstance(modules, list):
|
||||
entries.extend(modules)
|
||||
conflicts = registry_artifact_conflicts(entries)
|
||||
if conflicts:
|
||||
return conflicts
|
||||
package_by_repo: dict[str, tuple[str, str]] = {}
|
||||
registry_artifacts: dict[str, dict[str, object]] = {}
|
||||
for entry in entries:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
try:
|
||||
registry_source = registry_entry_source(entry)
|
||||
except ValueError as exc:
|
||||
return (str(exc),)
|
||||
if registry_source is not None:
|
||||
package = str(entry["python_package"])
|
||||
package_by_repo[registry_source.repository] = (package, registry_source.version)
|
||||
registry_artifacts[package] = entry["artifact_integrity"]["python"]
|
||||
continue
|
||||
python_ref = entry.get("python_ref")
|
||||
match = _PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
|
||||
package_name = entry.get("python_package")
|
||||
@@ -219,6 +234,14 @@ def selected_artifact_identity_issues(payload: object) -> tuple[str, ...]:
|
||||
issues: list[str] = []
|
||||
if malformed_artifacts:
|
||||
issues.append("release.artifacts contains malformed or duplicate identities")
|
||||
for package, registry_artifact in registry_artifacts.items():
|
||||
artifact = artifacts_by_package.get(package)
|
||||
if artifact is None or (
|
||||
artifact.get("archive_sha256") != registry_artifact.get("sha256")
|
||||
or artifact.get("archive_size") != registry_artifact.get("size")
|
||||
or artifact.get("package_version") != registry_artifact["registry_identity"].rsplit("@", 1)[-1]
|
||||
):
|
||||
issues.append(f"registry artifact {package} has no matching inspected wheel byte identity")
|
||||
seen_repos: set[str] = set()
|
||||
for unit in selected_units:
|
||||
if not isinstance(unit, dict):
|
||||
|
||||
@@ -19,6 +19,7 @@ from typing import Iterator
|
||||
from govoplan_core.core.modules import ModuleManifest
|
||||
from govoplan_core.core.versioning import version_satisfies_range
|
||||
|
||||
from .git_state import sanitized_git_environment, scoped_git_command
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
@@ -255,20 +256,19 @@ def materialized_source_tree(root: Path, *, source_ref: str | None) -> Iterator[
|
||||
source_root = temporary / "source"
|
||||
source_root.mkdir()
|
||||
result = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(root),
|
||||
scoped_git_command(
|
||||
root, "-C", str(root),
|
||||
"archive",
|
||||
"--format=tar",
|
||||
f"--output={archive_path}",
|
||||
source_ref,
|
||||
],
|
||||
),
|
||||
check=False,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
timeout=30,
|
||||
env=sanitized_git_environment(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
detail = result.stderr.strip() or "Git archive failed"
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
"""Build a private full-profile candidate from exact verified registry bytes."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from functools import lru_cache
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path, PurePosixPath
|
||||
import re
|
||||
import runpy
|
||||
import stat
|
||||
import tarfile
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
from .artifact_identity import inspect_python_wheel, selected_artifact_identity_issues
|
||||
from .candidate_artifact import (
|
||||
ensure_private_candidate_root, harden_private_candidate_tree,
|
||||
validate_release_channel,
|
||||
)
|
||||
from .catalog import canonical_hash
|
||||
from .module_directory import write_module_directory
|
||||
from .selective_catalog import (
|
||||
authenticate_base_catalog_signatures, authenticate_base_keyring,
|
||||
configured_signer_public_keys, enforce_selected_version_alignment,
|
||||
next_sequence, parse_signing_key, read_bounded_json_source, signature,
|
||||
validate_catalog_object,
|
||||
)
|
||||
from .source_provenance import (
|
||||
registered_source_origin_issues, selected_source_provenance,
|
||||
source_tag_provenance_issues,
|
||||
)
|
||||
from .version_alignment import candidate_catalog_version_issues
|
||||
from .workspace import META_ROOT, resolve_workspace_root, website_root
|
||||
|
||||
|
||||
MAX_ARTIFACT_BYTES = 512 * 1024 * 1024
|
||||
|
||||
|
||||
@lru_cache(maxsize=3)
|
||||
def _tool(name: str) -> dict[str, Any]:
|
||||
# These are fixed, operator-controlled release modules, not caller paths.
|
||||
if name not in {
|
||||
"generate-release-package-set", "generate-release-catalog",
|
||||
"resolve-package-artifacts",
|
||||
}:
|
||||
raise ValueError("unknown registry release tool")
|
||||
return runpy.run_path(str(META_ROOT / "tools" / "release" / f"{name}.py"))
|
||||
|
||||
|
||||
def authenticate_full_rebuild_base(
|
||||
*, web_root: Path, channel: str, signer_public_keys: dict[str, str],
|
||||
) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
"""Authenticate the fixed website pair without reusing legacy entry data.
|
||||
|
||||
Only this full rebuild may migrate a legacy catalog without a keyring hash.
|
||||
In that case *every* active website key must exactly equal a configured
|
||||
signer; injecting an additional website key cannot extend trust. Selective
|
||||
candidates retain their stricter existing hash-pinned-base requirement.
|
||||
"""
|
||||
|
||||
root = web_root / "public" / "catalogs" / "v1"
|
||||
catalog = read_bounded_json_source(root / "channels" / f"{channel}.json", label="published base catalog")
|
||||
keyring = read_bounded_json_source(root / "keyring.json", label="published website keyring")
|
||||
if not isinstance(catalog, dict) or not isinstance(keyring, dict):
|
||||
raise ValueError("published website catalog/keyring must be objects")
|
||||
trusted_keys = authenticate_base_keyring(keyring)
|
||||
release = catalog.get("release")
|
||||
pinned = release.get("keyring_sha256") if isinstance(release, dict) else None
|
||||
if pinned is None:
|
||||
if trusted_keys != signer_public_keys or len(keyring["keys"]) != len(trusted_keys):
|
||||
raise ValueError("legacy full rebuild requires exactly the configured known website signers")
|
||||
elif pinned != canonical_hash(keyring):
|
||||
raise ValueError("published base catalog does not pin its exact website keyring")
|
||||
if any(trusted_keys.get(key) != value for key, value in signer_public_keys.items()):
|
||||
raise ValueError("full rebuild cannot introduce or replace a website signer")
|
||||
authenticate_base_catalog_signatures(
|
||||
catalog, base_trusted_keys=trusted_keys, configured_signers=signer_public_keys,
|
||||
)
|
||||
validation = validate_catalog_object(
|
||||
catalog, approved_channel=channel, signer_public_keys=signer_public_keys,
|
||||
)
|
||||
if validation.get("valid") is not True:
|
||||
raise ValueError(f"published base catalog failed validation: {validation.get('error')}")
|
||||
return catalog, keyring
|
||||
|
||||
|
||||
def build_full_registry_candidate(
|
||||
*, package_set_path: Path, package_lock_path: Path,
|
||||
wheelhouse: Path, webui_packages: Path, output_dir: Path,
|
||||
selected_repositories: tuple[str, ...], signing_keys: tuple[str, ...],
|
||||
workspace_root: Path | str | None = None, channel: str = "stable",
|
||||
source_remote: str = "origin", public_base_url: str = "https://govoplan.add-ideas.de",
|
||||
expires_days: int = 90, sequence: int | None = None,
|
||||
) -> dict[str, object]:
|
||||
channel = validate_release_channel(channel)
|
||||
if not isinstance(expires_days, int) or isinstance(expires_days, bool) or not 1 <= expires_days <= 365:
|
||||
raise ValueError("catalog expiry must be between 1 and 365 days")
|
||||
workspace = resolve_workspace_root(workspace_root)
|
||||
ensure_private_candidate_root(workspace)
|
||||
output = output_dir.expanduser().absolute()
|
||||
ensure_private_candidate_root(output.parent, create=True)
|
||||
if output.exists() or output.is_symlink():
|
||||
raise ValueError("full candidate output must not already exist")
|
||||
parsed_keys = tuple(parse_signing_key(value) for value in signing_keys)
|
||||
if not parsed_keys:
|
||||
raise ValueError("full candidate needs a configured signing key")
|
||||
signer_keys = configured_signer_public_keys(parsed_keys)
|
||||
base, keyring = authenticate_full_rebuild_base(
|
||||
web_root=website_root(workspace), channel=channel, signer_public_keys=signer_keys,
|
||||
)
|
||||
package_set = _hashed_json(package_set_path, "package_set_sha256")
|
||||
lock = _hashed_json(package_lock_path, "lock_sha256")
|
||||
generator = _tool("generate-release-catalog")
|
||||
version = package_set.get("release_version")
|
||||
if package_set.get("profile") != "full" or not isinstance(version, str):
|
||||
raise ValueError("full candidate requires the complete full-profile package set")
|
||||
expected = _tool("generate-release-package-set")["generate_package_set"](
|
||||
core_version=version, requirements=META_ROOT / "requirements-release.txt",
|
||||
workspace=workspace, profile="full",
|
||||
meta_package=META_ROOT / "packages/govoplan-meta/pyproject.toml",
|
||||
)
|
||||
if package_set != expected:
|
||||
raise ValueError("package set differs from exact Meta full pins or immutable tag metadata")
|
||||
generator["_validate_release_inputs"](package_set, lock, core_version=version)
|
||||
if lock.get("registries") != package_set.get("registries"):
|
||||
raise ValueError("artifact lock uses different package registries")
|
||||
versions = {row["repository"]: row["version"] for row in package_set["python"]}
|
||||
origin_failures = registered_source_origin_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
)
|
||||
if origin_failures:
|
||||
raise ValueError("Registered source origin gate failed: " + "; ".join(item.describe() for item in origin_failures))
|
||||
selected = set(selected_repositories)
|
||||
if not selected or len(selected) != len(selected_repositories) or not selected <= versions.keys():
|
||||
raise ValueError("selected repositories must be unique members of the full package set")
|
||||
selected_versions = {repo: versions[repo] for repo in sorted(selected)}
|
||||
enforce_selected_version_alignment(repo_versions=selected_versions, workspace=workspace)
|
||||
failures = source_tag_provenance_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
require_head_repos=selected,
|
||||
)
|
||||
if failures:
|
||||
raise ValueError("Full source provenance gate failed: " + "; ".join(item.describe() for item in failures))
|
||||
provenance = selected_source_provenance(repo_versions=versions, workspace=workspace)
|
||||
wheel_identities = verify_registry_artifacts(
|
||||
package_set=package_set, lock=lock, wheelhouse=wheelhouse,
|
||||
webui_packages=webui_packages,
|
||||
)
|
||||
generated_at = datetime.now(tz=UTC)
|
||||
resolved_sequence = sequence if sequence is not None else next_sequence(base, generated_at=generated_at)
|
||||
if isinstance(resolved_sequence, bool) or not isinstance(resolved_sequence, int) or resolved_sequence <= int(base.get("sequence") or 0):
|
||||
raise ValueError("full candidate sequence must advance the authenticated published channel")
|
||||
# Fresh tagged manifests, including unchanged tagged ancestors; no legacy
|
||||
# entry, registry hash, source URL or dependency contract is carried over.
|
||||
candidate = generator["_catalog_payload"](
|
||||
package_set=package_set, package_lock=lock, channel=channel,
|
||||
sequence=resolved_sequence, generated_at=generated_at,
|
||||
expires_at=generated_at + timedelta(days=expires_days), workspace=workspace,
|
||||
public_base_url=public_base_url.rstrip("/"),
|
||||
)
|
||||
for entry in [candidate["core_release"], *candidate["modules"]]:
|
||||
repo = entry["python_package"]
|
||||
entry["source"] = {
|
||||
"repository": repo, "tag": f"v{versions[repo]}",
|
||||
"commit": provenance[repo]["commit_sha"],
|
||||
"tag_object_sha": provenance[repo]["tag_object_sha"],
|
||||
"repository_url": f"https://git.add-ideas.de/GovOPlaN/{repo}",
|
||||
"revision_url": f"https://git.add-ideas.de/GovOPlaN/{repo}/commit/{provenance[repo]['commit_sha']}",
|
||||
}
|
||||
candidate["release"].update({
|
||||
"selected_units": [
|
||||
{"repo": repo, "version": versions[repo], "tag": f"v{versions[repo]}", **provenance[repo]}
|
||||
for repo in sorted(selected)
|
||||
],
|
||||
"keyring_sha256": canonical_hash(keyring),
|
||||
"artifacts": wheel_identities,
|
||||
"base_catalog_sha256": canonical_hash(base),
|
||||
})
|
||||
# Recheck the exact objects used by synthesis, including unchanged source
|
||||
# ancestors, before signing. No late tag movement can change this candidate.
|
||||
origin_failures = registered_source_origin_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
)
|
||||
if origin_failures:
|
||||
raise ValueError("Registered source origin changed during synthesis: " + "; ".join(item.describe() for item in origin_failures))
|
||||
failures = source_tag_provenance_issues(
|
||||
repo_versions=versions, workspace=workspace, remote=source_remote,
|
||||
require_head_repos=selected,
|
||||
expected_commits={repo: row["commit_sha"] for repo, row in provenance.items()},
|
||||
expected_tag_objects={repo: row["tag_object_sha"] for repo, row in provenance.items()},
|
||||
)
|
||||
if failures:
|
||||
raise ValueError("Full source provenance changed during synthesis: " + "; ".join(item.describe() for item in failures))
|
||||
failures = candidate_catalog_version_issues(candidate)
|
||||
identity_failures = selected_artifact_identity_issues(candidate)
|
||||
if failures or identity_failures:
|
||||
raise ValueError("full candidate identity validation failed: " + "; ".join(
|
||||
[item.message for item in failures] + list(identity_failures)
|
||||
))
|
||||
candidate["signatures"] = [signature(candidate, key_id=key, private_key=value) for key, value in parsed_keys]
|
||||
validation = validate_catalog_object(candidate, approved_channel=channel, signer_public_keys=signer_keys)
|
||||
if validation.get("valid") is not True:
|
||||
raise ValueError(f"signed full candidate failed validation: {validation.get('error')}")
|
||||
# Exclusive output creation preserves earlier reviewed candidates.
|
||||
output.mkdir(mode=0o700)
|
||||
(output / "channels").mkdir(mode=0o700)
|
||||
catalog_path = output / "channels" / f"{channel}.json"
|
||||
_write_private_json(catalog_path, candidate)
|
||||
_write_private_json(output / "keyring.json", keyring)
|
||||
write_module_directory(
|
||||
catalog_payload=candidate, keyring_payload=keyring, output_root=output,
|
||||
channel=channel, public_base_url=public_base_url,
|
||||
)
|
||||
result = {
|
||||
"status": "ready", "candidate_dir": str(output), "catalog_path": str(catalog_path),
|
||||
"channel": channel, "sequence": resolved_sequence,
|
||||
"package_count": len(package_set["python"]), "webui_count": len(package_set["webui"]),
|
||||
"selected_count": len(selected), "candidate_catalog_hash": canonical_hash(candidate),
|
||||
"candidate_keyring_hash": canonical_hash(keyring), "validation_valid": True,
|
||||
}
|
||||
_write_private_json(output / "summary.json", result)
|
||||
harden_private_candidate_tree(output)
|
||||
return result
|
||||
|
||||
|
||||
def _hashed_json(path: Path, field: str) -> dict[str, Any]:
|
||||
payload = read_bounded_json_source(path, label=field)
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError(f"{field} input must be an object")
|
||||
unsigned = dict(payload)
|
||||
expected = unsigned.pop(field, None)
|
||||
# Registry tools use their established ASCII-escaped canonical form.
|
||||
encoded = json.dumps(unsigned, sort_keys=True, separators=(",", ":")).encode()
|
||||
if expected != hashlib.sha256(encoded).hexdigest():
|
||||
raise ValueError(f"{field} does not match its contents")
|
||||
return payload
|
||||
|
||||
|
||||
def verify_registry_artifacts(
|
||||
*, package_set: dict[str, Any], lock: dict[str, Any],
|
||||
wheelhouse: Path, webui_packages: Path,
|
||||
) -> list[dict[str, object]]:
|
||||
"""Compare exact registry bytes, metadata and source bindings without installs."""
|
||||
|
||||
identities = []
|
||||
for group, root, suffix in (("python", wheelhouse, ".whl"), ("webui", webui_packages, ".tgz")):
|
||||
ensure_private_candidate_root(root)
|
||||
expected = {row["name"]: row for row in package_set[group]}
|
||||
rows = lock[group]
|
||||
if not isinstance(rows, list) or len(rows) != len(expected):
|
||||
raise ValueError(f"artifact lock has duplicate/missing {group} rows")
|
||||
filenames: set[str] = set()
|
||||
seen: set[str] = set()
|
||||
for row in rows:
|
||||
selected = expected.get(row.get("name")) if isinstance(row, dict) else None
|
||||
if selected is None or row["name"] in seen:
|
||||
raise ValueError(f"artifact lock has unexpected/duplicate {group} identities")
|
||||
seen.add(row["name"])
|
||||
for key in ("name", "version", "repository", "tag", "commit"):
|
||||
if row.get(key) != selected[key]:
|
||||
raise ValueError("artifact lock differs from selected source identity")
|
||||
filename = row.get("filename")
|
||||
if not isinstance(filename, str) or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+-]{0,254}", filename) is None or not filename.endswith(suffix) or filename in filenames:
|
||||
raise ValueError("artifact filename is invalid or duplicated")
|
||||
filenames.add(filename)
|
||||
path = root / filename
|
||||
sha256, sha512, size = _hash_artifact(path)
|
||||
if (sha256, size) != (row.get("sha256"), row.get("size")):
|
||||
raise ValueError(f"registry artifact bytes differ from locked identity: {filename}")
|
||||
if group == "python":
|
||||
expected_url = _tool("resolve-package-artifacts")["_python_artifact_url"](
|
||||
package_set["registries"]["python"], package=selected, filename=filename,
|
||||
)
|
||||
if row.get("url") != expected_url:
|
||||
raise ValueError("Python artifact URL differs from the selected registry")
|
||||
identity = inspect_python_wheel(path)
|
||||
if (identity.package_name, identity.package_version, identity.archive_sha256, identity.archive_size) != (row["name"], row["version"], sha256, size):
|
||||
raise ValueError("wheel metadata or bytes differ from the registry lock")
|
||||
identities.append(identity.catalog_payload())
|
||||
else:
|
||||
expected_url = (
|
||||
package_set["registries"]["npm"].rstrip("/") + "/"
|
||||
+ quote(row["name"], safe="") + "/-/"
|
||||
+ quote(row["version"], safe="") + "/"
|
||||
+ quote(row["name"].split("/", 1)[1] + "-" + row["version"] + ".tgz", safe="")
|
||||
)
|
||||
if row.get("url") != expected_url:
|
||||
raise ValueError("WebUI artifact URL differs from the selected registry")
|
||||
if row.get("integrity") != "sha512-" + base64.b64encode(sha512).decode("ascii"):
|
||||
raise ValueError("WebUI registry integrity differs from downloaded bytes")
|
||||
_inspect_npm_metadata(
|
||||
path, name=row["name"], version=row["version"],
|
||||
expected_identity=(sha256, sha512, size),
|
||||
)
|
||||
actual = set()
|
||||
for index, path in enumerate(root.iterdir()):
|
||||
if index >= 1000:
|
||||
raise ValueError("registry artifact directory exceeds its inspection bound")
|
||||
actual.add(path.name)
|
||||
if actual != filenames:
|
||||
raise ValueError(f"registry directory contains unexpected or missing {group} files")
|
||||
return sorted(identities, key=lambda row: str(row["package_name"]))
|
||||
|
||||
|
||||
def _hash_artifact(path: Path) -> tuple[str, bytes, int]:
|
||||
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
try:
|
||||
initial = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(initial.st_mode) or not 0 < initial.st_size <= MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("registry artifact must be a bounded regular file")
|
||||
digest, sri, total = hashlib.sha256(), hashlib.sha512(), 0
|
||||
while chunk := os.read(descriptor, 1024 * 1024):
|
||||
total += len(chunk)
|
||||
if total > MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("registry artifact exceeds its byte bound")
|
||||
digest.update(chunk)
|
||||
sri.update(chunk)
|
||||
final = os.fstat(descriptor)
|
||||
if (initial.st_ino, initial.st_size, initial.st_mtime_ns, initial.st_ctime_ns) != (final.st_ino, total, final.st_mtime_ns, final.st_ctime_ns):
|
||||
raise ValueError("registry artifact changed while being inspected")
|
||||
return digest.hexdigest(), sri.digest(), total
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _inspect_npm_metadata(
|
||||
path: Path, *, name: str, version: str,
|
||||
expected_identity: tuple[str, bytes, int] | None = None,
|
||||
) -> None:
|
||||
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
|
||||
try:
|
||||
initial = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(initial.st_mode) or not 0 < initial.st_size <= MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("WebUI archive must be a bounded regular file")
|
||||
digest, sri, total = hashlib.sha256(), hashlib.sha512(), 0
|
||||
while chunk := os.read(descriptor, 1024 * 1024):
|
||||
total += len(chunk)
|
||||
if total > MAX_ARTIFACT_BYTES:
|
||||
raise ValueError("WebUI archive exceeds its byte bound")
|
||||
digest.update(chunk)
|
||||
sri.update(chunk)
|
||||
if expected_identity is not None and (digest.hexdigest(), sri.digest(), total) != expected_identity:
|
||||
raise ValueError("WebUI archive changed before metadata inspection")
|
||||
os.lseek(descriptor, 0, os.SEEK_SET)
|
||||
with os.fdopen(os.dup(descriptor), "rb") as stream:
|
||||
_inspect_npm_stream(stream, name=name, version=version)
|
||||
final = os.fstat(descriptor)
|
||||
if (initial.st_ino, initial.st_size, initial.st_mtime_ns, initial.st_ctime_ns) != (final.st_ino, total, final.st_mtime_ns, final.st_ctime_ns):
|
||||
raise ValueError("WebUI archive changed during metadata inspection")
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _inspect_npm_stream(stream: Any, *, name: str, version: str) -> None:
|
||||
found = False
|
||||
total = 0
|
||||
with tarfile.open(fileobj=stream, mode="r|gz") as archive:
|
||||
for index, member in enumerate(archive):
|
||||
total += member.size
|
||||
parts = PurePosixPath(member.name).parts
|
||||
if index >= 10000 or total > 1024 * 1024 * 1024 or member.size > 64 * 1024 * 1024:
|
||||
raise ValueError("WebUI archive exceeds its inspection bound")
|
||||
if not parts or parts[0] != "package" or ".." in parts or not (member.isfile() or member.isdir()):
|
||||
raise ValueError("WebUI archive contains an unsafe member")
|
||||
if member.name == "package/package.json":
|
||||
if found or not member.isfile() or member.size > 1024 * 1024:
|
||||
raise ValueError("WebUI archive has duplicate or oversized metadata")
|
||||
stream = archive.extractfile(member)
|
||||
if stream is None:
|
||||
raise ValueError("WebUI archive metadata cannot be read")
|
||||
metadata = json.loads(stream.read(1024 * 1024 + 1))
|
||||
if not isinstance(metadata, dict) or (metadata.get("name"), metadata.get("version")) != (name, version):
|
||||
raise ValueError("WebUI metadata differs from the registry lock")
|
||||
found = True
|
||||
if not found:
|
||||
raise ValueError("WebUI archive metadata is missing")
|
||||
|
||||
|
||||
def _write_private_json(path: Path, payload: object) -> None:
|
||||
encoded = (json.dumps(payload, indent=2, sort_keys=True) + "\n").encode()
|
||||
if len(encoded) > 16 * 1024 * 1024:
|
||||
raise ValueError("candidate JSON exceeds its byte bound")
|
||||
descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0), 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
@@ -197,6 +197,13 @@ def sanitized_git_environment(
|
||||
"""Keep only deliberate process/auth inputs and neutralize Git redirection."""
|
||||
|
||||
environment = os.environ if source is None else source
|
||||
address_family = environment.get("GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY")
|
||||
if "GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY" in environment and address_family not in (
|
||||
"any", "inet", "inet6",
|
||||
):
|
||||
raise ValueError(
|
||||
"GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY must be any, inet, or inet6"
|
||||
)
|
||||
result = {
|
||||
key: environment[key]
|
||||
for key in (
|
||||
@@ -221,6 +228,10 @@ def sanitized_git_environment(
|
||||
"PATH": "/usr/bin:/bin",
|
||||
}
|
||||
)
|
||||
if address_family is not None:
|
||||
result["GIT_SSH_COMMAND"] += f" -o AddressFamily={address_family}"
|
||||
# Preserve only an explicit, validated choice across re-sanitization.
|
||||
result["GOVOPLAN_RELEASE_SSH_ADDRESS_FAMILY"] = address_family
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -27,7 +27,10 @@ from .candidate_artifact import validate_release_channel
|
||||
from .model import CatalogPublishResult, CatalogPublishStep
|
||||
from .module_directory import module_directory_payloads
|
||||
from .selective_catalog import read_bounded_json_source, trusted_keys_from_keyring
|
||||
from .source_provenance import catalog_source_selection, source_tag_provenance_issues
|
||||
from .source_provenance import (
|
||||
catalog_source_selection, registered_source_origin_issues,
|
||||
source_tag_provenance_issues,
|
||||
)
|
||||
from .version_alignment import candidate_catalog_version_issues
|
||||
from .workspace import (
|
||||
DEFAULT_WORKSPACE_ROOT,
|
||||
@@ -176,6 +179,22 @@ def publish_catalog_candidate(
|
||||
for issue in version_issues
|
||||
)
|
||||
source_selection = catalog_source_selection(candidate_payload)
|
||||
entries = [candidate_payload.get("core_release")]
|
||||
if isinstance(candidate_payload.get("modules"), list):
|
||||
entries.extend(candidate_payload["modules"])
|
||||
if any(
|
||||
isinstance(entry, dict)
|
||||
and isinstance(entry.get("python_ref"), str)
|
||||
and " @ https://" in entry["python_ref"]
|
||||
for entry in entries
|
||||
):
|
||||
blockers.extend(
|
||||
f"registered source origin: {issue.describe()}"
|
||||
for issue in registered_source_origin_issues(
|
||||
repo_versions=source_selection.all_versions,
|
||||
workspace=workspace, remote=source_remote,
|
||||
)
|
||||
)
|
||||
blockers.extend(
|
||||
f"source provenance: {issue.describe()}"
|
||||
for issue in source_selection.issues
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
"""Strict source identities for immutable, registry-backed catalog entries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
import re
|
||||
from urllib.parse import unquote, urlsplit
|
||||
|
||||
|
||||
_SHA256 = re.compile(r"[0-9a-f]{64}\Z")
|
||||
_COMMIT = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?\Z")
|
||||
_REPO = re.compile(r"govoplan-[a-z0-9-]+\Z")
|
||||
_VERSION = re.compile(r"\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?\Z")
|
||||
_PYTHON = re.compile(
|
||||
r"(?P<name>govoplan-[a-z0-9-]+)(?:\[[a-z0-9_,.-]+\])? @ "
|
||||
r"(?P<url>https://\S+)#sha256=(?P<digest>[0-9a-f]{64})\Z"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegistrySource:
|
||||
repository: str
|
||||
version: str
|
||||
commit: str
|
||||
tag_object: str
|
||||
|
||||
|
||||
def registry_artifact_conflicts(entries: list[object]) -> tuple[str, ...]:
|
||||
"""Allow repeated module projections only when package artifacts agree."""
|
||||
observed: dict[tuple[str, str], str] = {}
|
||||
issues = []
|
||||
for entry in entries:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
try:
|
||||
source = registry_entry_source(entry)
|
||||
except ValueError as exc:
|
||||
issues.append(str(exc))
|
||||
continue
|
||||
if source is None:
|
||||
continue
|
||||
for kind, identity in entry["artifact_integrity"].items():
|
||||
if kind not in {"python", "webui"}:
|
||||
continue
|
||||
key = (source.repository, kind)
|
||||
encoded = json.dumps(identity, sort_keys=True, separators=(",", ":"))
|
||||
if observed.setdefault(key, encoded) != encoded:
|
||||
issues.append(f"conflicting {kind} registry artifacts for {source.repository}")
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
def registry_entry_source(entry: dict[str, object]) -> RegistrySource | None:
|
||||
"""Admit registry refs only with a complete matching artifact/source binding.
|
||||
|
||||
Git-backed catalogs keep their existing validation path. An HTTPS Python
|
||||
requirement cannot masquerade as a source-only/non-Python entry when its
|
||||
registry provenance is missing or inconsistent.
|
||||
"""
|
||||
|
||||
ref = entry.get("python_ref")
|
||||
if not isinstance(ref, str) or " @ https://" not in ref:
|
||||
return None
|
||||
match = _PYTHON.fullmatch(ref)
|
||||
source = entry.get("source")
|
||||
version = entry.get("version")
|
||||
package = entry.get("python_package")
|
||||
integrity = entry.get("artifact_integrity")
|
||||
if (
|
||||
match is None
|
||||
or not isinstance(source, dict)
|
||||
or not isinstance(integrity, dict)
|
||||
or not isinstance(version, str)
|
||||
or _VERSION.fullmatch(version) is None
|
||||
or package != match.group("name")
|
||||
):
|
||||
raise ValueError("registry entry has no complete package/source identity")
|
||||
repo = source.get("repository")
|
||||
commit = source.get("commit")
|
||||
tag_object = source.get("tag_object_sha")
|
||||
if (
|
||||
not isinstance(repo, str)
|
||||
or _REPO.fullmatch(repo) is None
|
||||
or repo != package
|
||||
or source.get("tag") != f"v{version}"
|
||||
or not isinstance(commit, str)
|
||||
or _COMMIT.fullmatch(commit) is None
|
||||
or not isinstance(tag_object, str)
|
||||
or _COMMIT.fullmatch(tag_object) is None
|
||||
):
|
||||
raise ValueError("registry entry has invalid immutable tag provenance")
|
||||
python = _artifact(
|
||||
integrity.get("python"), ref=ref, package=package, version=version,
|
||||
commit=commit,
|
||||
)
|
||||
if python["url"] != match.group("url") or python["sha256"] != match.group("digest"):
|
||||
raise ValueError("registry Python ref differs from its artifact identity")
|
||||
webui_package = entry.get("webui_package")
|
||||
webui_ref = entry.get("webui_ref")
|
||||
if bool(webui_package) != bool(webui_ref):
|
||||
raise ValueError("registry WebUI package and ref must be declared together")
|
||||
if webui_package:
|
||||
if not isinstance(webui_package, str) or re.fullmatch(
|
||||
r"@govoplan/[a-z0-9-]+-webui", webui_package
|
||||
) is None or not isinstance(webui_ref, str):
|
||||
raise ValueError("registry WebUI package identity is malformed")
|
||||
if webui_package != f"@govoplan/{repo.removeprefix('govoplan-')}-webui":
|
||||
raise ValueError("registry WebUI package belongs to another source repository")
|
||||
webui = _artifact(
|
||||
integrity.get("webui"), ref=webui_ref, package=webui_package,
|
||||
version=version, commit=commit,
|
||||
)
|
||||
if webui_ref != webui["url"]:
|
||||
raise ValueError("registry WebUI ref differs from its artifact identity")
|
||||
sri = webui.get("integrity")
|
||||
try:
|
||||
valid_sri = isinstance(sri, str) and sri.startswith("sha512-") and len(
|
||||
base64.b64decode(sri[7:], validate=True)
|
||||
) == 64
|
||||
except ValueError:
|
||||
valid_sri = False
|
||||
if not valid_sri:
|
||||
raise ValueError("registry WebUI artifact needs a SHA-512 integrity identity")
|
||||
elif "webui" in integrity:
|
||||
raise ValueError("registry entry carries an unexpected WebUI artifact")
|
||||
return RegistrySource(repo, version, commit, tag_object)
|
||||
|
||||
|
||||
def _artifact(
|
||||
value: object, *, ref: str, package: str, version: str, commit: str,
|
||||
) -> dict[str, object]:
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("registry entry is missing artifact integrity")
|
||||
url = value.get("url")
|
||||
filename = value.get("filename")
|
||||
digest = value.get("sha256")
|
||||
size = value.get("size")
|
||||
parsed = urlsplit(url) if isinstance(url, str) else None
|
||||
url_filename = unquote(parsed.path.rsplit("/", 1)[-1]) if parsed else ""
|
||||
decoded_parts = unquote(parsed.path).split("/") if parsed else []
|
||||
expected_filenames = {url_filename}
|
||||
expected_path = [package, version, url_filename]
|
||||
if package.startswith("@govoplan/"):
|
||||
# npm pack includes the scope in its local filename; the registry's
|
||||
# immutable download URL uses the unscoped package basename.
|
||||
expected_filenames = {
|
||||
f"govoplan-{package.split('/', 1)[1]}-{version}.tgz",
|
||||
} if url_filename == f"{package.split('/', 1)[1]}-{version}.tgz" else set()
|
||||
expected_path = [*package.split("/"), "-", version, url_filename]
|
||||
if (
|
||||
parsed is None
|
||||
or parsed.scheme != "https"
|
||||
or not parsed.netloc
|
||||
or parsed.username is not None
|
||||
or parsed.password is not None
|
||||
or parsed.fragment
|
||||
or parsed.query
|
||||
or any(part in {".", ".."} for part in unquote(parsed.path).split("/"))
|
||||
or "%" in unquote(parsed.path)
|
||||
or "\\" in unquote(parsed.path)
|
||||
or any(ord(character) < 32 for character in url)
|
||||
or decoded_parts[-len(expected_path):] != expected_path
|
||||
or not isinstance(filename, str)
|
||||
or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+-]{0,254}", filename) is None
|
||||
or filename not in expected_filenames
|
||||
or not isinstance(digest, str)
|
||||
or _SHA256.fullmatch(digest) is None
|
||||
or not isinstance(size, int)
|
||||
or isinstance(size, bool)
|
||||
or not 0 < size <= 512 * 1024 * 1024
|
||||
or value.get("ref") != ref
|
||||
or value.get("registry_identity") != f"{package}@{version}"
|
||||
or value.get("git_ref") != f"v{version}"
|
||||
or value.get("source_commit") != commit
|
||||
):
|
||||
raise ValueError("registry artifact ref, version, bytes, or source binding is inconsistent")
|
||||
return value
|
||||
@@ -39,6 +39,9 @@ def tag_repositories(
|
||||
A release tag is only created for a clean, aligned, non-behind worktree.
|
||||
Both local and remote tags are resolved to commits before mutation so an
|
||||
existing immutable tag can never be moved by this operation.
|
||||
Module-only local candidate tags precede Core's release-lock regeneration;
|
||||
their cross-Core composition gate applies before publication, not creation.
|
||||
Core candidate tags still require a complete aligned release bundle.
|
||||
"""
|
||||
|
||||
workspace = resolve_workspace_root(workspace_root)
|
||||
@@ -47,13 +50,19 @@ def tag_repositories(
|
||||
selected = tuple(dict.fromkeys(repos))
|
||||
results: list[dict[str, object]] = []
|
||||
bundle_issues_by_repo: dict[str, list[str]] = {}
|
||||
for issue in selected_release_webui_bundle_issues(
|
||||
repo_versions={repo: repo_versions.get(repo, "") for repo in selected},
|
||||
workspace=workspace,
|
||||
):
|
||||
bundle_issues_by_repo.setdefault(issue.repo, []).append(
|
||||
f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
|
||||
)
|
||||
# Core's final lock is generated from reviewed local module tags. Requiring
|
||||
# that lock before those tags exist makes the documented sequence circular.
|
||||
# This is only a local module staging exception: Core-selected batches and
|
||||
# every publication still run the cross-repository gate, and each selected
|
||||
# repository's own version/lock checks below are always enforced.
|
||||
if push or "govoplan-core" in selected:
|
||||
for issue in selected_release_webui_bundle_issues(
|
||||
repo_versions={repo: repo_versions.get(repo, "") for repo in selected},
|
||||
workspace=workspace,
|
||||
):
|
||||
bundle_issues_by_repo.setdefault(issue.repo, []).append(
|
||||
f"{issue.source}={issue.actual!r}, expected {issue.expected!r} ({issue.message})"
|
||||
)
|
||||
|
||||
if apply:
|
||||
preflight = tag_repositories(
|
||||
|
||||
@@ -19,10 +19,33 @@ from .git_state import (
|
||||
scoped_git_command,
|
||||
)
|
||||
from .repository_tag import RemoteTagResult, ref_commit, remote_tag_commit
|
||||
from .registry_reference import registry_entry_source
|
||||
from .version_alignment import repository_version_issues
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
def registered_source_origin_issues(
|
||||
*, repo_versions: dict[str, str], workspace: Path, remote: str,
|
||||
) -> tuple[SourceTagProvenanceIssue, ...]:
|
||||
"""Bind registry candidate attestations to registered source endpoints."""
|
||||
specs = {spec.name: spec for spec in load_repository_specs(include_website=False)}
|
||||
issues = []
|
||||
for repo, version in sorted(repo_versions.items()):
|
||||
spec = specs.get(repo)
|
||||
if spec is None:
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source repository is not registered"))
|
||||
continue
|
||||
path = resolve_repo_path(spec, workspace)
|
||||
if path.absolute() != path.resolve() or not path.resolve().is_relative_to(workspace.resolve()):
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source checkout leaves the private workspace or traverses a symlink"))
|
||||
continue
|
||||
fetch = git_text(path, "remote", "get-url", "--all", remote).splitlines()
|
||||
push = git_text(path, "remote", "get-url", "--push", "--all", remote).splitlines()
|
||||
if fetch != [spec.remote] or push != [spec.remote]:
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "source remote does not exactly match the registered origin"))
|
||||
return tuple(issues)
|
||||
|
||||
|
||||
_CATALOG_PYTHON_REF = re.compile(
|
||||
r"/(?P<repo>govoplan-[a-z0-9-]+)\.git@v(?P<version>[^\s;]+)$"
|
||||
)
|
||||
@@ -61,6 +84,8 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
)
|
||||
|
||||
versions: dict[str, str] = {}
|
||||
registry_commits: dict[str, str] = {}
|
||||
registry_tag_objects: dict[str, str] = {}
|
||||
issues: list[SourceTagProvenanceIssue] = []
|
||||
entries: list[tuple[str, object]] = [("core_release", payload.get("core_release"))]
|
||||
modules = payload.get("modules")
|
||||
@@ -70,6 +95,21 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
for source, raw_entry in entries:
|
||||
if not isinstance(raw_entry, dict):
|
||||
continue
|
||||
try:
|
||||
registry_source = registry_entry_source(raw_entry)
|
||||
except ValueError as exc:
|
||||
issues.append(SourceTagProvenanceIssue(source, "", str(exc)))
|
||||
continue
|
||||
if registry_source is not None:
|
||||
repo, version = registry_source.repository, registry_source.version
|
||||
previous = versions.setdefault(repo, version)
|
||||
previous_commit = registry_commits.setdefault(repo, registry_source.commit)
|
||||
previous_object = registry_tag_objects.setdefault(repo, registry_source.tag_object)
|
||||
if (previous, previous_commit, previous_object) != (
|
||||
version, registry_source.commit, registry_source.tag_object,
|
||||
):
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "registry entries have conflicting immutable source identities"))
|
||||
continue
|
||||
python_ref = raw_entry.get("python_ref")
|
||||
match = _CATALOG_PYTHON_REF.search(python_ref) if isinstance(python_ref, str) else None
|
||||
if match is None:
|
||||
@@ -89,8 +129,8 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
release = payload.get("release")
|
||||
selected_units = release.get("selected_units") if isinstance(release, dict) else None
|
||||
selected: dict[str, str] = {}
|
||||
selected_commits: dict[str, str] = {}
|
||||
selected_tag_objects: dict[str, str] = {}
|
||||
selected_commits: dict[str, str] = dict(registry_commits)
|
||||
selected_tag_objects: dict[str, str] = dict(registry_tag_objects)
|
||||
if not isinstance(selected_units, list) or not selected_units:
|
||||
issues.append(
|
||||
SourceTagProvenanceIssue(
|
||||
@@ -106,16 +146,22 @@ def catalog_source_selection(payload: object) -> CatalogSourceSelection:
|
||||
repo = unit.get("repo")
|
||||
version = unit.get("version")
|
||||
if isinstance(repo, str) and isinstance(version, str):
|
||||
if repo in selected:
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "duplicate selected repository"))
|
||||
selected[repo] = version.removeprefix("v")
|
||||
commit = unit.get("commit_sha")
|
||||
tag_object = unit.get("tag_object_sha")
|
||||
if isinstance(commit, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", commit):
|
||||
if repo in registry_commits and registry_commits[repo] != commit.lower():
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "selected commit differs from registry artifact source"))
|
||||
selected_commits[repo] = commit.lower()
|
||||
else:
|
||||
issues.append(
|
||||
SourceTagProvenanceIssue(repo, f"v{version.removeprefix('v')}", "selected unit has no valid commit_sha provenance")
|
||||
)
|
||||
if isinstance(tag_object, str) and re.fullmatch(r"[0-9a-fA-F]{40}|[0-9a-fA-F]{64}", tag_object):
|
||||
if repo in registry_tag_objects and registry_tag_objects[repo] != tag_object.lower():
|
||||
issues.append(SourceTagProvenanceIssue(repo, f"v{version}", "selected tag object differs from registry artifact source"))
|
||||
selected_tag_objects[repo] = tag_object.lower()
|
||||
else:
|
||||
issues.append(
|
||||
|
||||
@@ -10,6 +10,7 @@ import subprocess
|
||||
import tomllib
|
||||
|
||||
from .git_state import collect_versions, sanitized_git_environment
|
||||
from .registry_reference import registry_artifact_conflicts, registry_entry_source
|
||||
from .workspace import load_repository_specs, resolve_repo_path
|
||||
|
||||
|
||||
@@ -407,6 +408,11 @@ def candidate_catalog_version_issues(payload: object) -> tuple[VersionAlignmentI
|
||||
)
|
||||
|
||||
release = payload.get("release")
|
||||
registry_entries = [core_release, *(modules if isinstance(modules, list) else [])]
|
||||
issues.extend(
|
||||
_catalog_shape_issue("artifact_integrity", issue)
|
||||
for issue in registry_artifact_conflicts(registry_entries)
|
||||
)
|
||||
if isinstance(release, dict):
|
||||
issues.extend(_catalog_release_issues(release, represented=represented))
|
||||
return tuple(issues)
|
||||
@@ -419,6 +425,15 @@ def _catalog_entry_issues(
|
||||
represented: dict[str, str],
|
||||
) -> list[VersionAlignmentIssue]:
|
||||
issues: list[VersionAlignmentIssue] = []
|
||||
try:
|
||||
registry_source = registry_entry_source(entry)
|
||||
except ValueError as exc:
|
||||
return [_catalog_shape_issue(source, str(exc))]
|
||||
if registry_source is not None:
|
||||
previous = represented.setdefault(registry_source.repository, registry_source.version)
|
||||
if previous != registry_source.version:
|
||||
issues.append(_catalog_shape_issue(source, "repository appears with conflicting catalog versions"))
|
||||
return issues
|
||||
version = entry.get("version")
|
||||
normalized_version = version.removeprefix("v") if isinstance(version, str) and version else None
|
||||
if normalized_version is None:
|
||||
|
||||
@@ -269,6 +269,7 @@ def _render_python_version(
|
||||
except SyntaxError as exc:
|
||||
raise VersionMetadataError(f"Python metadata is malformed: {path.name}") from exc
|
||||
values: list[ast.Constant] = []
|
||||
module_version_references = 0
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Call) or _call_name(node.func) != target_name:
|
||||
continue
|
||||
@@ -279,6 +280,28 @@ def _render_python_version(
|
||||
and isinstance(keyword.value.value, str)
|
||||
):
|
||||
values.append(keyword.value)
|
||||
elif (
|
||||
keyword.arg == keyword_name
|
||||
and isinstance(keyword.value, ast.Name)
|
||||
and keyword.value.id == "MODULE_VERSION"
|
||||
):
|
||||
module_version_references += 1
|
||||
if module_version_references:
|
||||
if module_version_references != 1 or values:
|
||||
raise VersionMetadataError(
|
||||
f"Python metadata has multiple {target_name}.{keyword_name} values: {path.name}"
|
||||
)
|
||||
rendered, found = _render_python_assignment(
|
||||
payload,
|
||||
path=path,
|
||||
assignment_name="MODULE_VERSION",
|
||||
version=version,
|
||||
)
|
||||
if not found:
|
||||
raise VersionMetadataError(
|
||||
f"Python metadata has no literal MODULE_VERSION declaration: {path.name}"
|
||||
)
|
||||
return rendered, True
|
||||
if not values:
|
||||
return payload, False
|
||||
if len(values) != 1:
|
||||
|
||||
@@ -21,6 +21,22 @@ def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
full = subparsers.add_parser("full-registry", help="Build a strict full-profile candidate from verified registry artifacts.")
|
||||
full.add_argument("--workspace-root", type=Path, default=DEFAULT_WORKSPACE_ROOT)
|
||||
full.add_argument("--package-set", type=Path, required=True)
|
||||
full.add_argument("--package-lock", type=Path, required=True)
|
||||
full.add_argument("--wheelhouse", type=Path, required=True)
|
||||
full.add_argument("--webui-packages", type=Path, required=True)
|
||||
full.add_argument("--output-dir", type=Path, required=True)
|
||||
full.add_argument("--selected-repository", action="append", required=True)
|
||||
full.add_argument("--catalog-signing-key", action="append", required=True)
|
||||
full.add_argument("--channel", default="stable")
|
||||
full.add_argument("--source-remote", default="origin")
|
||||
full.add_argument("--public-base-url", default="https://govoplan.add-ideas.de")
|
||||
full.add_argument("--expires-days", type=int, default=90)
|
||||
full.add_argument("--sequence", type=int)
|
||||
full.add_argument("--json", action="store_true")
|
||||
|
||||
selective = subparsers.add_parser(
|
||||
"selective", help="Build a signed selective channel catalog candidate."
|
||||
)
|
||||
@@ -143,6 +159,21 @@ def main() -> int:
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
if args.command == "full-registry":
|
||||
require_release_runtime_trust()
|
||||
from govoplan_release.full_catalog import build_full_registry_candidate
|
||||
|
||||
result = build_full_registry_candidate(
|
||||
package_set_path=args.package_set, package_lock_path=args.package_lock,
|
||||
wheelhouse=args.wheelhouse, webui_packages=args.webui_packages,
|
||||
output_dir=args.output_dir, selected_repositories=tuple(args.selected_repository),
|
||||
signing_keys=tuple(args.catalog_signing_key), workspace_root=args.workspace_root,
|
||||
channel=args.channel, source_remote=args.source_remote,
|
||||
public_base_url=args.public_base_url, expires_days=args.expires_days,
|
||||
sequence=args.sequence,
|
||||
)
|
||||
print(json.dumps(result, indent=2, sort_keys=True))
|
||||
return 0
|
||||
if args.command == "selective":
|
||||
require_release_runtime_trust()
|
||||
result = build_selective_catalog_candidate(
|
||||
|
||||
Reference in New Issue
Block a user