feat: implement institutional governance and recovery architecture
This commit is contained in:
+1
-1
@@ -15,7 +15,7 @@ GOVOPLAN_DB_MAX_OVERFLOW=10
|
||||
GOVOPLAN_DB_POOL_TIMEOUT_SECONDS=30
|
||||
GOVOPLAN_DB_POOL_RECYCLE_SECONDS=1800
|
||||
|
||||
ENABLED_MODULES=tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,files,mail,campaigns,calendar,poll,scheduling,connectors,datasources,dataflow,workflow,views,search,risk_compliance,postbox,notifications,docs,ops
|
||||
ENABLED_MODULES=tenancy,organizations,identity,idm,access,admin,dashboard,policy,audit,files,mail,campaigns,calendar,poll,scheduling,connectors,datasources,dataflow,workflow_engine,workflow,views,search,risk_compliance,postbox,notifications,services,parties,mandates,decisions,portal,cases,committee,docs,ops
|
||||
|
||||
CELERY_ENABLED=true
|
||||
REDIS_URL=redis://127.0.0.1:6379/0
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# GovOPlaN Workspace Codex Guide
|
||||
|
||||
## Scope
|
||||
|
||||
This repository coordinates the GovOPlaN workspace, release catalog, shared
|
||||
checks, development environment, and cross-repository automation. Business and
|
||||
platform behavior remains owned by the corresponding module repository.
|
||||
|
||||
## Documentation Contract
|
||||
|
||||
- Treat documentation as part of every behavior change. Update the owning module's manifest-driven `DocumentationTopic` contributions for each affected user and administrator workflow, setting, permission, limitation, and operational consequence.
|
||||
- Keep feature content in the owning module. The optional `govoplan-docs` module projects module contributions and must not import feature internals.
|
||||
- Every module manifest must provide a static user and administrator baseline, even when `documentation_providers` add configured-state details.
|
||||
- Run `tools/checks/check-manifest-shapes.py` after module behavior or manifest changes. Run `tools/checks/check-focused.sh` for cross-module changes.
|
||||
|
||||
## Working Rules
|
||||
|
||||
- Treat Gitea issues as the canonical backlog and state log.
|
||||
- Preserve optional module boundaries and use Core contracts or capabilities for integrations.
|
||||
- Prefer targeted checks before full workspace scans.
|
||||
- Do not start persistent development servers unless requested.
|
||||
@@ -155,6 +155,9 @@ The current executable slice and remaining production gates are documented in
|
||||
[Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md).
|
||||
Same-host replica balancing and the multi-host promotion boundary are documented
|
||||
in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
|
||||
The recovery state machine, migration rollback boundary, and required restore
|
||||
drills are documented in
|
||||
[Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md).
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -182,6 +185,10 @@ contract are documented in
|
||||
The cross-product destination, stakeholder visions, configuration archetypes,
|
||||
connected outcome stories, and capability horizons are documented in
|
||||
the [Connected Governance Platform Roadmap](docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md).
|
||||
The reconciled institutional semantics, source-authority modes, module layers,
|
||||
candidate Mandates/Services/Parties/Decisions boundaries, and migration
|
||||
sequence are documented in the
|
||||
[Institutional Governance Target Architecture](docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
|
||||
The selected Campaign-to-Postbox-to-data-to-collaboration implementation path,
|
||||
including stage gates and shared documentation expectations, is in the
|
||||
[Reference Journey Program](docs/REFERENCE_JOURNEY_PROGRAM.md).
|
||||
@@ -193,6 +200,10 @@ reconfiguration semantics, and safe Web update boundary are defined in
|
||||
[Installation and Deployment Architecture](docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md).
|
||||
The concrete replica, worker-node, load-balancer, and shared-state topology is
|
||||
defined in [Scaling and Multi-Host Deployment](docs/SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
|
||||
Durable deployment journals, Core recovery evidence, and the distinction
|
||||
between pre-migration configuration restore and post-migration forward recovery
|
||||
are defined in
|
||||
[Recovery and Rollback Guarantees](docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md).
|
||||
The first Campaign-centric capability and infrastructure fit assessment is in
|
||||
`docs/CAPABILITY_AND_INFRASTRUCTURE_FIT.md`. Its rerun tooling can collect and
|
||||
verify a bounded installed composition; target, provider and production claims
|
||||
|
||||
@@ -14,6 +14,14 @@
|
||||
| Machine-readable companion | [`capability-fit-current.json`](capability-fit-current.json) |
|
||||
| Input schema | [`capability-fit.schema.json`](capability-fit.schema.json) |
|
||||
|
||||
**Snapshot notice:** this assessment remains valid only for the pinned
|
||||
2026-07-22 composition above. Workflow Engine, the optional Workflow editor,
|
||||
Datasources, Dataflow, Search, encryption contracts, and other later main-branch
|
||||
work must not be inferred into this evidence record. The current product
|
||||
direction and implemented-state reconciliation are documented separately in
|
||||
the
|
||||
[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
|
||||
|
||||
This is a fit assessment, not a production approval or security certification.
|
||||
It deliberately does not infer implementation from a repository, issue, or
|
||||
manifest existing. A conclusion needs code plus a route/contract, test,
|
||||
@@ -468,10 +476,11 @@ which modules a running service activated, migrations, configuration, health,
|
||||
or reference-journey behavior. Runtime activation therefore remains an explicit
|
||||
unchecked boundary.
|
||||
|
||||
### Target, provider and production proof boundary
|
||||
### Reference-readiness, provider and production proof boundary
|
||||
|
||||
Installed evidence cannot approve a target environment, an external provider,
|
||||
or production use. These scopes use a separate, expiring
|
||||
Installed evidence cannot establish target acceptance, accessibility, privacy,
|
||||
security, operations, recovery, an external provider, or production use. These
|
||||
scopes use a separate, expiring
|
||||
[`capability-fit-boundary-evidence.schema.json`](capability-fit-boundary-evidence.schema.json)
|
||||
bundle. The bundle is bound to the assessment ID, assessment release and exact
|
||||
installed-evidence SHA-256 digest. It contains only opaque subject/control/result
|
||||
@@ -480,8 +489,9 @@ IDs and content hashes, not endpoints, credentials, people or raw result files.
|
||||
Boundary evidence is accepted only when at least one Ed25519 signature validates
|
||||
against a separately provisioned
|
||||
[`capability-fit-proof-authority-keyring.schema.json`](capability-fit-proof-authority-keyring.schema.json).
|
||||
Each authority key explicitly lists the scopes it may attest. Target and provider
|
||||
claims use `passed` or `failed`; production claims use `approved` or `rejected`.
|
||||
Each authority key explicitly lists the scopes it may attest. Target,
|
||||
accessibility, privacy, security, operations, recovery, and provider claims use
|
||||
`passed` or `failed`; production claims use `approved` or `rejected`.
|
||||
One claim per scope, unique control/artifact IDs, `issued_at < expires_at`, current
|
||||
validity and exact digest binding are mandatory. Any schema, binding, time,
|
||||
signature or authority blocker leaves every supplied boundary claim unchecked;
|
||||
@@ -554,6 +564,10 @@ Target-environment and production-approval claims must use the assessment's
|
||||
operator to supply a bounded opaque expected subject with
|
||||
`--expected-external-provider-subject`; without it, such a claim remains
|
||||
unchecked and blocks. Expected and observed IDs are retained in proof scope.
|
||||
Accessibility, privacy, security, operations, and recovery claims use the same
|
||||
deployment subject. The report emits a cumulative `reference_readiness` verdict
|
||||
only when all six required scopes are checked and positive. This verdict remains
|
||||
separate from production approval and from provider-specific acceptance.
|
||||
|
||||
Both authority keyrings are governance trust roots. Installer receipt keys use
|
||||
the strict
|
||||
@@ -575,9 +589,9 @@ cannot relabel it as current. Live admission must omit it and use the actual
|
||||
current time.
|
||||
|
||||
No boundary bundle or production authority has been supplied for this current
|
||||
assessment. Target environment, provider and production proof therefore remain
|
||||
explicitly unchecked rather than inferred from the local GreenMail journey,
|
||||
source tests or signed release metadata.
|
||||
assessment. Reference-readiness, provider, and production proof therefore
|
||||
remain explicitly unchecked rather than inferred from the local GreenMail
|
||||
journey, source tests, or signed release metadata.
|
||||
|
||||
## Evidence used in this slice
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ cross-product vision that sequence serves.
|
||||
|
||||
Read it together with:
|
||||
|
||||
- the [institutional governance target architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md)
|
||||
- the [selected reference-journey program](REFERENCE_JOURNEY_PROGRAM.md)
|
||||
- the [current capability and infrastructure fit assessment](CAPABILITY_AND_INFRASTRUCTURE_FIT.md)
|
||||
- the [interface pattern language](INTERFACE_PATTERN_LANGUAGE.md)
|
||||
@@ -151,7 +152,7 @@ responsibilities, not a new dependency hierarchy.
|
||||
| Participation and channels | Let internal and external actors enter, receive, discuss, schedule, and respond through suitable channels. | Portal, postbox, mail, notifications, calendar, scheduling, appointments, campaign, consultation, poll |
|
||||
| Work coordination | Turn an input into owned, reviewable work and make exceptions visible. | Forms/runtime, cases, tasks, approvals, workflow, booking, resources, domain modules |
|
||||
| Evidence and institutional memory | Preserve what was known, decided, produced, sent, received, retained, corrected, and disclosed. | Files, templates, DMS, records, audit, search, reporting, transparency |
|
||||
| Institutional governance | Establish who may act, in which capacity, for which organization and tenant, under which policy. | Identity, access, IDM, tenancy, organizations, policy, identity trust, risk/compliance |
|
||||
| Institutional governance | Establish who may act, in which capacity, for which organization and tenant, under which mandate, jurisdiction, and policy. | Identity, access, IDM, tenancy, organizations, policy, identity trust, risk/compliance; candidate mandate and decision contracts |
|
||||
| Integration and operations | Connect sources and destinations, operate them safely, and prove their health and recovery. | Connectors, REST/SOAP and public-sector protocols, mail/calendar/file adapters, ERP handoffs, ops, release and configuration packages |
|
||||
|
||||
Across these planes, GovOPlaN should maintain a connected context graph rather
|
||||
@@ -1136,11 +1137,12 @@ boundary, and retain operational, security, and evidence guarantees.
|
||||
| Addresses and directories | Implemented adapters exist but configuration/target proof varies | Directory source, privacy, conflict and lifecycle package | Reusable people/contact source capability |
|
||||
| Docs/admin/ops/dashboard | Useful cross-product surfaces with incomplete rollout | Configured-system inventory, guided config, monitoring/recovery evidence | Explainability and operation of the configured product |
|
||||
| Organizations/identity/IDM/access/postbox | Normalized ownership concepts and uneven runtime slices | Function-bound delivery, reassignment/delegation, vacancy and access-evidence proof | Institutional responsibility and durable communication spine |
|
||||
| Forms/cases/tasks/approvals/search | Concepts and uneven first slices | Extend the proven responsibility path into one manual end-to-end work/evidence journey | Reusable administrative coordination layer |
|
||||
| Templates/reporting/data sources | Boundary concepts or scaffolds | One reproducible data-backed document/report and safe HIS-style launch | Governed document production, reports, dashboards, and analytical consumption |
|
||||
| Analytical data products/dataflow | Selected direction; platform contracts not yet implemented | One bounded university source-to-indicator path with staging, quality, lineage and promotion proof | Transparent institutional BI and cross-process reporting |
|
||||
| Forms/cases/tasks/approvals | Concepts and uneven first slices | Extend the proven responsibility path into one manual end-to-end work/evidence journey, with shared service, party, mandate, and decision semantics | Reusable administrative coordination layer |
|
||||
| Search | PostgreSQL-backed permission-aware provider and global/contextual UI foundation | Complete provider rollout, indexing operations, and target authorization/performance evidence | Optional cross-module discovery with OpenSearch only as an adapter |
|
||||
| Templates/reporting/data sources | Templates and governed Datasources foundations exist; Reporting remains an early semantic-model slice | One reproducible data-backed document/report and safe HIS-style launch | Governed document production, reports, dashboards, and analytical consumption |
|
||||
| Analytical data products/dataflow | Typed graph/runtime foundations, triggers, staging integration, reusable definitions, and golden-flow fixtures exist | One bounded university source-to-indicator path with quality, lineage, promotion, and target-performance proof | Transparent institutional BI and cross-process reporting |
|
||||
| DMS/collaborative editing | Boundary concept and tag-only scaffold | One Files-backed version lifecycle, then one provider-neutral editing session | Collaborative documents, review, approval and records-ready renditions |
|
||||
| Workflow/automation | Concept only; no discoverable Workflow runtime; program postponed | Stable action/effect providers and an explicitly reprioritized bounded journey | Configurable governed process coordination |
|
||||
| Workflow/automation | Headless Workflow Engine and optional editor are implemented foundations with versioned module baselines, BPMN interchange, action/effect execution, and reconciliation | Prove one resumable human/system journey with target recovery and conformance evidence | Configurable governed process coordination without becoming a second domain layer |
|
||||
| Domain modules | Mostly boundary concepts or seeds | Only the modules required by a reference package | Reusable semantics above the shared spine |
|
||||
| Connectors and protocols | Catalogue/strategy plus several module-specific adapters | Profile/runtime, source-of-truth, health and one real landscape | Coexistence with institutional IT |
|
||||
| Records/transparency/risk-compliance/export screening | Planned or early concepts | Evidence contracts and one regulated reference story | Institutional memory, oversight, and assurance |
|
||||
@@ -1216,8 +1218,9 @@ provides all applicable evidence below.
|
||||
## Near-term portfolio order
|
||||
|
||||
This order is now selected. Detailed slices and gates are in the
|
||||
[Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md). Workflow-driven
|
||||
user-story implementation remains paused.
|
||||
[Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md). Workflow Engine and
|
||||
the optional editor may support these stages, but Workflow work enters the
|
||||
portfolio only through an explicit bounded package or reference journey.
|
||||
|
||||
0. **Continuous safe foundation.** Keep the composition green and version
|
||||
aligned; close connector destination pinning, response limits, throttling,
|
||||
@@ -1250,8 +1253,9 @@ user-story implementation remains paused.
|
||||
target acceptance for calendar/scheduling/poll and other shipped foundations
|
||||
without displacing the selected reference path; implement new feature
|
||||
programs only when they are required by a stage or explicitly reprioritized.
|
||||
7. **Resume Workflow only by explicit priority decision.** When resumed, start
|
||||
with stable actions from a demonstrated package; do not turn it into a
|
||||
7. **Use Workflow only through bounded journeys.** The headless engine and
|
||||
optional editor now exist. Extend them through stable module-owned actions,
|
||||
versioned baselines, and demonstrated packages; do not turn Workflow into a
|
||||
second domain layer.
|
||||
|
||||
## Product decisions to make progressively
|
||||
@@ -1263,8 +1267,8 @@ them; they do not block the product vision today.
|
||||
reference release?
|
||||
- Which real identity, groupware, file/DMS, and deployment stack should define
|
||||
the first integration profile?
|
||||
- When should the postponed Workflow program resume, and which single package
|
||||
will constrain its first implementation?
|
||||
- Which demonstrated package should provide the first target-accepted Workflow
|
||||
execution and recovery profile?
|
||||
- Which objects and fields remain authoritative in GovOPlaN versus each target
|
||||
system, and which conflict/failure behavior is acceptable?
|
||||
- Which default participant privacy profiles should ship for scheduling,
|
||||
|
||||
@@ -6,8 +6,8 @@ GovOPlaN separates governed data identity from external acquisition:
|
||||
|
||||
| Layer | Owner | Responsibilities |
|
||||
| --- | --- | --- |
|
||||
| Datasource layer | `govoplan-datasources` | Catalogue, tenant visibility, live/cached/static mode, staging, immutable materializations, frozen states, schema, fingerprints, provenance, and bounded reads |
|
||||
| Connector layer | `govoplan-connectors` and protocol/provider modules | External protocols, endpoints, connection profiles, credentials, discovery, provider health, source-side filtering, and query pushdown |
|
||||
| Datasource layer | `govoplan-datasources` | Governed data/register catalogue, tenant visibility, live/cached/static mode, source authority, staging, immutable materializations, frozen states, schema, quality/freshness policy, institutional provenance, dependencies, and bounded reads |
|
||||
| Connector layer | `govoplan-connectors` and protocol/provider modules | External protocols, endpoints, connection profiles, credentials, discovery, provider maturity/authority support, health, source-side filtering, query pushdown, and effect reconciliation |
|
||||
|
||||
Connectors publish versioned datasource origins. Datasources registers those
|
||||
origins and presents one stable capability to Dataflow, Workflow, Reporting,
|
||||
@@ -43,9 +43,10 @@ Domain modules own their semantics:
|
||||
|
||||
- Dataflow provides load, combine, filter, transform, and output nodes. Its
|
||||
graph is acyclic and has one output.
|
||||
- Workflow provides trigger, activity, review, decision, wait, module-action,
|
||||
Dataflow, and outcome nodes. It permits governed loops and has exactly one
|
||||
trigger plus one or more outcomes.
|
||||
- Workflow Engine provides trigger, activity, review, decision, wait,
|
||||
module-action, Dataflow, and outcome semantics. It permits governed loops and
|
||||
has exactly one trigger plus one or more outcomes. The optional Workflow
|
||||
module supplies the editor over the same native graph/BPMN language.
|
||||
|
||||
This division permits a shared editor shell without making Workflow a special
|
||||
kind of Dataflow or leaking either module into Core.
|
||||
@@ -53,8 +54,10 @@ kind of Dataflow or leaking either module into Core.
|
||||
## Current Implementation
|
||||
|
||||
- Core graph and datasource contracts are versioned at `0.1.0`.
|
||||
- Workflow exposes a reusable graph editor, node-library discovery, validation,
|
||||
tenant-isolated definitions, immutable revisions, and activation pinning.
|
||||
- Workflow Engine owns tenant-isolated definitions, immutable revisions,
|
||||
activation pinning, module-contributed versioned baselines, runtime instances,
|
||||
governed action/effect execution, retries, waits, and reconciliation. The
|
||||
optional Workflow module exposes the reusable native BPMN graph editor.
|
||||
- Datasources exposes catalogue, origins, staging, promotion, preview,
|
||||
materialization history, refresh, freeze, retirement, and producer
|
||||
publication APIs.
|
||||
@@ -62,21 +65,26 @@ kind of Dataflow or leaking either module into Core.
|
||||
- Connectors adapts existing tabular snapshots to datasource origins.
|
||||
- Dataflow consumes only Datasources catalogue/lifecycle capabilities and can
|
||||
request current, live, or latest-frozen state.
|
||||
- Dataflow exposes a pinned run-lifecycle capability and a Run/Publish surface.
|
||||
Its first synchronous runner records lineage and terminal state, publishes
|
||||
only complete bounded results, and retains output datasource/materialization
|
||||
references.
|
||||
- Dataflow exposes typed graph/IR, registry-driven validation/execution/SQL
|
||||
compilation, expressions and reusable subflows, a pinned run-lifecycle
|
||||
capability, production worker boundary, and Run/Publish surface. Runs record
|
||||
lineage and intermediate artifacts and publish only complete bounded results.
|
||||
- The focused composition check proves Connector origin -> Datasource ->
|
||||
pinned Dataflow run -> frozen published materialization, including replay.
|
||||
|
||||
## Next Slices
|
||||
|
||||
1. Add persisted Workflow instances, resumable transitions, human activities,
|
||||
retry policy, and event subscriptions against pinned definition revisions.
|
||||
2. Add SQL database and governed REST origin providers with credential-envelope
|
||||
1. Add the provider declaration and source-authority binding used consistently
|
||||
by Connectors, Datasources, configuration packages, Ops, and Docs.
|
||||
2. Add typed datasource owner/steward, legal/purpose, quality/freshness,
|
||||
classification, correction, service/process, and downstream dependency
|
||||
metadata under
|
||||
[Datasources #6](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/6).
|
||||
3. Add SQL database and governed REST origin providers with credential-envelope
|
||||
references and bounded pushdown.
|
||||
3. Add managed-file and directory origins.
|
||||
4. Add datasource quality rules, schema compatibility policy, retention, and
|
||||
4. Add managed-file and directory origins.
|
||||
5. Complete datasource quality rules, schema compatibility policy, retention, and
|
||||
promotion approvals.
|
||||
5. Add asynchronous Dataflow workers and durable artifact-backed outputs for
|
||||
runs that exceed the synchronous row/time/byte limits.
|
||||
6. Complete scheduled/event/API/chained Dataflow trigger governance, reusable
|
||||
template inheritance, Reporting publication, human reconciliation transforms,
|
||||
and target resource/recovery evidence for large runs.
|
||||
|
||||
@@ -34,9 +34,15 @@ It currently supports:
|
||||
- generated secrets stored in a private `0600` file;
|
||||
- service-specific environment allowlists so infrastructure containers do not
|
||||
receive unrelated application credentials;
|
||||
- plan, render, doctor, status, and apply commands;
|
||||
- plan, render, doctor, status, apply, Kubernetes export, operation history,
|
||||
and bounded recovery commands;
|
||||
- an installation lock, migration-before-start ordering, readiness polling,
|
||||
and an applied-state receipt;
|
||||
- a durable hash-chained deployment journal captured before runtime mutation;
|
||||
- PostgreSQL advisory serialization for Core and module migrations;
|
||||
- runtime initialization that waits for exact configured migration heads
|
||||
without mutating schema;
|
||||
- runtime node registration, heartbeats, drain state, and a fenced scheduler;
|
||||
- idempotent reconfiguration that preserves generated secrets;
|
||||
- a keyed environment fingerprint that detects private binding changes without
|
||||
writing secret values to plans or receipts;
|
||||
@@ -87,6 +93,9 @@ The private installation directory contains:
|
||||
| `load-balancer.cfg` | Non-secret HAProxy WebUI/API discovery configuration |
|
||||
| `plan.json` | Latest desired-state diff and readiness findings |
|
||||
| `receipt.json` | Last successfully applied immutable identities |
|
||||
| `applied-state/` | Checksum-verified snapshot of the last healthy deployment bundle |
|
||||
| `operations/<id>/` | Private hash-chained deployment progress and recovery evidence |
|
||||
| `kubernetes.json` | Optional stateless multi-host Kubernetes export |
|
||||
| `.deployment.lock` | Same-host operation exclusion |
|
||||
|
||||
The specification contract is
|
||||
@@ -201,11 +210,13 @@ presenting a container as a complete mail service.
|
||||
installations. `garage` provisions Garage 2.3 in its supported single-node
|
||||
bootstrap mode, generates a private application key and bucket, and connects
|
||||
the Files S3 backend to the exact installer-owned internal endpoint. The
|
||||
deployment-only trust marker cannot authorize another S3 host; arbitrary
|
||||
external SDK endpoints remain fail-closed until peer pinning is implemented.
|
||||
managed trust marker cannot authorize another S3 host.
|
||||
Garage metadata and object data use separate persistent volumes. `s3` requires
|
||||
an external endpoint, region, access key, secret key, and bucket values.
|
||||
Self-hosted external S3 endpoints must use HTTPS.
|
||||
Self-hosted external S3 endpoints must be clean HTTPS origins. The generated
|
||||
runtime explicitly sets `FILE_STORAGE_S3_ENDPOINT_TRUSTED=true` for that
|
||||
operator-selected endpoint. The trust flag is not accepted for local storage
|
||||
and cannot be combined with installer-managed Garage trust.
|
||||
|
||||
Local storage must be included in backup and restore drills. Horizontal API or
|
||||
worker scale-out requires shared/object storage. The managed Garage profile is
|
||||
@@ -233,9 +244,10 @@ Replica counts are desired state:
|
||||
```
|
||||
|
||||
Workers are queue consumers, so they are scaled through Redis rather than put
|
||||
behind an HTTP load balancer. The migration runner and Celery scheduler remain
|
||||
singletons. Multiple API replicas are rejected when Redis is disabled because
|
||||
distributed throttling and queued work cannot then be shared correctly.
|
||||
behind an HTTP load balancer. Migrations are serialized with a deployment-wide
|
||||
PostgreSQL advisory lock. The Celery scheduler is run under a renewable,
|
||||
fencing-token lease. Multiple API replicas are rejected when Redis is disabled
|
||||
because distributed throttling and queued work cannot then be shared correctly.
|
||||
|
||||
This is same-host scaling. Docker Compose uses a bridge network and does not
|
||||
place containers on another machine. See
|
||||
@@ -258,13 +270,66 @@ applied state. `plan` compares their canonical hashes and service sets.
|
||||
- Managed-to-external transitions require the new endpoint in the same
|
||||
operation.
|
||||
- Migrations run as a one-shot service before API/worker replacement.
|
||||
- API, worker, and scheduler start commands wait for exact configured migration
|
||||
heads; only the migration command is permitted to change schema.
|
||||
- API and worker replicas register their software/module composition and
|
||||
heartbeat in PostgreSQL. Ops can request and cancel a node drain.
|
||||
- The first upgrade from a direct WebUI host port stops that legacy WebUI
|
||||
container immediately before HAProxy claims the same endpoint.
|
||||
- Health must recover before a new receipt is committed.
|
||||
- Health must recover before a new receipt and applied-state snapshot are
|
||||
committed.
|
||||
|
||||
This is sufficient for one-host reconciliation. Production updates additionally
|
||||
need backup/restore gates, maintenance/drain state, database compatibility
|
||||
windows, image signature verification, and rollback/forward-recovery policy.
|
||||
Every apply operation is journalled before image pulls or runtime mutation. A
|
||||
failure before migration may restore a verified previous bundle. Once migration
|
||||
starts, recovery is forward-only unless an independently verified database
|
||||
backup is restored. See
|
||||
[Recovery And Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md).
|
||||
|
||||
Production updates still need an operator-provided database backup/restore
|
||||
gate, database compatibility declaration, image signature verification, and
|
||||
deployment-specific drain policy. The deployment journal proves its own
|
||||
actions; it does not manufacture backup evidence.
|
||||
|
||||
## Stateless Kubernetes Runtime
|
||||
|
||||
`render-kubernetes` exports the application tier for a standard orchestrator.
|
||||
It requires external PostgreSQL, Redis, and S3 and emits no stateful service or
|
||||
secret value:
|
||||
|
||||
```sh
|
||||
python tools/deployment/govoplan-deploy.py render-kubernetes \
|
||||
--directory /srv/govoplan/default \
|
||||
--namespace govoplan \
|
||||
--secret-name govoplan-runtime
|
||||
```
|
||||
|
||||
The output includes a release-specific migration Job, database-head wait init
|
||||
containers, API readiness/liveness probes, rolling Deployments, Services, Pod
|
||||
disruption budgets, a tokenless ServiceAccount, and one fenced scheduler. Apply
|
||||
the named Secret through the cluster's secret manager and review ingress proxy
|
||||
CIDRs before deployment. Detailed rollout and scaling rules live in
|
||||
[Scaling And Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md).
|
||||
|
||||
## Recovery Commands
|
||||
|
||||
List durable deployment operations:
|
||||
|
||||
```sh
|
||||
python tools/deployment/govoplan-deploy.py operations \
|
||||
--directory /srv/govoplan/default
|
||||
```
|
||||
|
||||
Recover a selected failed operation after reviewing its stage evidence:
|
||||
|
||||
```sh
|
||||
python tools/deployment/govoplan-deploy.py recover \
|
||||
--directory /srv/govoplan/default \
|
||||
--operation-id <operation-id>
|
||||
```
|
||||
|
||||
The command reports whether it restored the pre-migration applied bundle,
|
||||
requires forward recovery, or needs manual intervention. Add `--apply` only
|
||||
after that decision has been reviewed.
|
||||
|
||||
## Web Update Boundary
|
||||
|
||||
@@ -317,5 +382,6 @@ The tests cover profile restrictions, secret persistence, external endpoint
|
||||
requirements, managed Garage bootstrap, S3 policy, replica validation, HAProxy
|
||||
discovery configuration, Compose service selection, secret non-disclosure,
|
||||
service-specific environment isolation, private file modes, external endpoint
|
||||
preflight, first-plan generation, apply ordering, and receipt-based
|
||||
idempotency.
|
||||
preflight, first-plan generation, apply ordering, receipt idempotency,
|
||||
hash-chained recovery journals, migration recovery boundaries, and stateless
|
||||
Kubernetes rendering.
|
||||
|
||||
@@ -0,0 +1,556 @@
|
||||
# Institutional Governance Target Architecture
|
||||
|
||||
## Status and sources
|
||||
|
||||
This document is the accepted architectural reconciliation of two product
|
||||
concepts prepared outside the repositories:
|
||||
|
||||
- `govoplan_concept_dev.md`
|
||||
- `software_big_picture.md`
|
||||
|
||||
The source concepts describe GovOPlaN as an operational governance platform for
|
||||
public institutions. This document merges that direction with the implemented
|
||||
platform state as of 2026-08-01. It is the canonical repository version of the
|
||||
direction. Gitea issues remain the source of truth for delivery state.
|
||||
|
||||
Read this together with:
|
||||
|
||||
- [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md)
|
||||
- [Reference Journey Program](REFERENCE_JOURNEY_PROGRAM.md)
|
||||
- [Module Contracts and Install Boundaries](MODULE_CONTRACTS_AND_INSTALLS.md)
|
||||
- [Datasource and Definition Graph Architecture](DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md)
|
||||
- [Capability and Infrastructure Fit](CAPABILITY_AND_INFRASTRUCTURE_FIT.md)
|
||||
- [Core Module Architecture](../../govoplan-core/docs/MODULE_ARCHITECTURE.md)
|
||||
- [Core External References and Integration Maturity](../../govoplan-core/docs/EXTERNAL_REFERENCES_AND_INTEGRATION_MATURITY.md)
|
||||
- [Core Action, Effect, and Automation Layer](../../govoplan-core/docs/ACTION_EFFECT_AUTOMATION_LAYER.md)
|
||||
|
||||
## Decision
|
||||
|
||||
GovOPlaN is a configurable **institutional governance and operations layer** for
|
||||
public institutions. It should model the institution, coordinate its work,
|
||||
connect its specialist systems, and preserve why and under whose authority an
|
||||
action occurred.
|
||||
|
||||
GovOPlaN is not intended to become one universal ERP, DMS, groupware suite,
|
||||
workflow editor, or specialist procedure. It should own the governance concepts
|
||||
that must remain understandable across those systems and support native,
|
||||
external, mirrored, synchronized, overlay, and link-only operation explicitly.
|
||||
|
||||
This changes product emphasis, not the modular architecture:
|
||||
|
||||
1. The current kernel and optional-module model remains.
|
||||
2. Existing domain owners keep their data and behavior.
|
||||
3. Cross-module semantics become explicit, versioned contracts.
|
||||
4. Successful compositions become product and sector packages, not forks or
|
||||
monolithic replacement applications.
|
||||
5. Repository creation follows a proof threshold; a noun in the information
|
||||
model does not automatically require a module.
|
||||
|
||||
## What recent work already supersedes
|
||||
|
||||
The source concepts predate several implemented foundations. These items are
|
||||
accepted as the current baseline and must not be reopened as greenfield work.
|
||||
|
||||
| Concept requirement | Reconciled current state |
|
||||
| --- | --- |
|
||||
| Slim kernel plus installable modules | Implemented through entry-point discovery, `ModuleManifest`, migrations, capabilities, interfaces, WebUI contributions, and permutation checks. |
|
||||
| Versioned cross-module contracts | Implemented through named interface ranges, capability protocols, static workspace graph checks, activation validation, and release checks. |
|
||||
| Separate headless workflow runtime and editor | Implemented as `govoplan-workflow-engine` and optional `govoplan-workflow`. Module-owned workflow baselines are versioned and reconciled without replacing local overrides. |
|
||||
| Provider-neutral external references | Implemented in Core with stable external identity and cumulative integration maturity from discovery through replacement. |
|
||||
| Governed asynchronous effects | Implemented foundations include the action/effect contract, transactional platform event outbox, module outboxes, idempotency, outcome-unknown states, reconciliation, and worker health. Coverage still varies by provider. |
|
||||
| Acting identity, function assignment, mandate, and ownership recovery | Implemented foundations span Identity, Organizations, IDM, Access, Mandates, generic ownership transfer/recovery, and audit provenance. Effective competence now resolves through a tenant-bound Mandate capability. |
|
||||
| Governed data foundations | Connectors, Datasources, Dataflow, Reporting, and Search now exist. Datasources already provides live/cached/static modes, staging, immutable materializations, and publication contracts. |
|
||||
| Task-focused projections and configured documentation | Views, view-surface declarations, configurable dashboards, and manifest-driven user/admin documentation exist. Rollout and content depth remain incremental. |
|
||||
| Encryption as an optional capability | `govoplan-encryption` now defines key-vault, content-protection, recovery, and disable-preflight boundaries. It is not a reason to move domain ownership into Core. |
|
||||
| Search without mandatory OpenSearch | PostgreSQL-backed, permission-aware search and module provider contracts exist; OpenSearch remains an optional adapter. |
|
||||
| Scale-out and recovery architecture | Stateless API/worker, shared database/object storage, event delivery, deployment, and recovery contracts are documented and partly exercised. Production profiles and drills remain active work. |
|
||||
|
||||
The institutional semantics, provider declaration gate, and first product
|
||||
compositions described here are now implemented. Subsequent work is
|
||||
**product depth and stronger maturity evidence**, not another runtime rewrite
|
||||
or an unimplemented architecture boundary.
|
||||
|
||||
## Implementation status (2026-08-01)
|
||||
|
||||
The architecture contract is implemented as a bounded, executable vertical
|
||||
slice. The portfolio declarations and provider governance gates apply to the
|
||||
whole workspace, while the four semantic domains whose repository thresholds
|
||||
were proven now have independent persistent owners:
|
||||
|
||||
| Area | Implemented state | Remaining rollout |
|
||||
| --- | --- | --- |
|
||||
| Module portfolio metadata | Core validates versioned architecture layer/kind, maturity evidence, known limits, ownership boundaries, authority modes, reference packages, target-tested providers, and migration/upgrade/recovery/security/operations documentation. | Complete for all 62 source manifests. Focused and release checks enforce `--require-architecture`; a new module cannot enter the workspace without truthful declaration and repository-local evidence. |
|
||||
| External providers | Core validates provider objects/field groups, operations, integration maturity, source authority, bounded reads, freshness/health, idempotency, conflicts, outcome-unknown handling, evidence, correction, reconciliation, outage, classification, purpose, retention, and secret handling. Addresses/CardDAV, Files remote storage, Mail SMTP/IMAP, Calendar CalDAV/ICS/Graph/EWS, and Connectors tabular/sanctions providers declare the contract and tenant-bounded secret-free runtime state. | Registry validation rejects any declared external provider without a sanitized state provider. Future adapters must cross the same gate before activation. |
|
||||
| Institutional context | Core provides versioned temporal, actor/representation, institution/unit/function/task/mandate/jurisdiction/service/case/party/work-item/workflow/approval/decision/record, legal-basis, evidence, information-governance, external-source, presentation, and geographic references. Events, automation actions, audit records, and the transactional Audit outbox preserve the envelope. | Owning modules must progressively require the relevant subset for consequential operations. |
|
||||
| Semantic provider contracts | Provider-neutral DTOs and protocols cover Mandate resolution, versioned Service definitions, procedure Parties/representation, and formal Decisions. `govoplan-mandates`, `govoplan-services`, `govoplan-parties`, and `govoplan-decisions` now persist immutable revisions behind those contracts with tenant isolation, bounded reads, replay safety, OCC, migrations, uninstall guards, permissions, APIs, capability documentation, and recovery documentation. | The owners are deliberately headless. Procedure-specific UI remains with consuming modules. |
|
||||
| Formal-outcome proof | Committee persists bodies, meetings, agenda items, votes, minutes, lifecycle events, and an optional protected local Decision projection. It resolves effective Mandate authority and records reconstructable formal Decisions through `decisions.registry` when installed. OCC, replay safety, tenant isolation, bounded reads, closure guards, and a full-height body/meeting/agenda/vote/minutes WebUI cover the aggregate. Provider-bound external or secret ballots use dynamic adapter capabilities; Committee validates and retains only aggregate counts, receipt/hash, and evidence, and rejects generic manual closure. | Concrete ballot-provider packages remain integration work because their protocol, custody, credentials, and operator evidence depend on the selected provider. This does not leave a Committee-owned ballot contract unspecified. |
|
||||
| Service-to-case proof | Services owns the persistent exact definitions consumed by Portal discovery and Cases intake. Forms owns immutable form schemas and Forms Runtime owns definition-aware drafts, validation, submission receipts, status/evidence history, and handoff references. Portal exposes a tenant-scoped service-directory WebUI whose audiences derive from trusted principal/function state, re-fetches the exact Service revision, and delegates URL, Case, Form, or Workflow launch to an installed owner. Cases and Forms Runtime retain exact Service and binding provenance, enforce tenant isolation, replay safety and OCC, and expose bounded owner/manager APIs and WebUI. | Anonymous intake, concrete attachment/signature adapters, conditional multi-page authoring, and automatic domain handoff execution are product depth on established contracts. Portal still fails closed and explains any absent launcher or runtime prerequisite. |
|
||||
| Procedure-party proof | Parties persists effective procedure roles, frozen contact snapshots, and representation powers. Existing powers cannot disappear or be silently rewritten; explicit OCC-guarded revocation is required. Cases resolves the provider capability and excludes expired/revoked authority from downstream delivery. | Procedure modules still decide which contextual fields and actions to present. |
|
||||
| Integrated institutional journey | The executable `product.service-to-decision` fixture uses real SQL-backed Services, Cases, Parties, Mandates, Committee, and Decisions providers. It carries one exact Service version through persisted Case intake, representation and frozen delivery authority, effective Mandate resolution, a body/meeting/agendum/vote/minute sequence, a persisted formal Decision, confirmed Postbox effect, Audit/record evidence, remedy/review, and protected reconstruction. A second executable path proves Portal to exact Form revision, persisted submission, and idempotent replay. | This is architecture and composition evidence. Signed, release-bound target accessibility, privacy, security, operator, delivery-provider, and recovery-drill evidence is still required before the package may claim `reference_ready`. |
|
||||
| Governed data catalogue | Datasources stores typed governance metadata, exposes bounded tenant-scoped filters and update APIs/UI, carries governance through staging, and snapshots it into immutable materializations. Reporting now persists immutable dataset, semantic-model, report, quality-plan, saved-view, and schedule revisions; executes typed semantic queries with quality gates, access checks, replay, pivoting, export/import assessment, and provenance; and exposes the governed analytical WebUI. | Rich dependency/impact traversal, additional expression functions, and policy-specific field visibility can grow on the established contracts without moving connector, transformation, or source ownership. |
|
||||
| Portfolio and change governance | Projects now persists tenant-safe, immutable portfolio/project/milestone revisions with OCC, replay, lifecycle rules, restricted memberships, Search ACL indexing, outcomes, benefits, dependencies, capacity assumptions, change impact, and institutional references. Its WebUI exposes the planning catalogue and core planning fields. | Advanced planning structures already accepted by the API can receive deeper specialized editors without creating a second Policy, Reporting, Resources, or Goals owner. |
|
||||
| Product/package governance | Signed configuration packages distinguish reference, product, sector, deployment, and integration classes; preserve parent/evidence provenance; prevent derived packages from loosening constraints; and preflight provider authority, maturity, exact binding, health, freshness, and recovery expectations. Executable product manifests now exist for governed communication and governed data/assurance and are checked in the module matrix. | Both artifacts deliberately remain product-class until target, accessibility, privacy, security, operations, and recovery evidence justifies reference readiness. |
|
||||
| Projection and release | Platform metadata, signed module catalogs, release synthesis, Ops, and role-aware Docs retain and display architecture/provider declarations. Module-owned state providers add bounded configured/active, authority, health, freshness, conflict, recovery, and observation state; ordinary-user Docs omits binding detail. Static checks validate evidence paths, and the WebUI build verifies consuming types. | Runtime-state adoption and broader portfolio presentation follow truthful provider declaration rollout. |
|
||||
|
||||
The implementation deliberately keeps shared reference contracts in Core and
|
||||
domain tables in their owners. It does not claim unsupported release maturity:
|
||||
the four extracted owners and package remain `vertical_slice`/`product` until
|
||||
target evidence supports a stronger claim. Gitea remains authoritative for
|
||||
feature depth beyond this architecture contract.
|
||||
|
||||
## Target capability layers
|
||||
|
||||
The layers describe ownership and dependency direction. They are not navigation
|
||||
groups and do not imply that every installation exposes every module.
|
||||
|
||||
| Layer | Responsibility | Current owners and declared directions |
|
||||
| --- | --- | --- |
|
||||
| 0. Runtime and meta | Composition, release, migrations, shared contracts, operations, deployment | Core, meta repository, Admin, Ops |
|
||||
| 1. Institutional foundation | Institution, tenant, identity, organization, function, authority, access, trust | Tenancy, Identity, Organizations, IDM, Access, Identity Trust, Encryption, Mandates |
|
||||
| 2. Governance and accountability | Policy, audit, risk, control, explainability, configured projection | Policy, Audit, Risk Compliance, Docs, Views, Search, Decisions |
|
||||
| 3. Human work and procedure | Intake, cases, tasks, approvals, process execution and editing | Services, Forms, Forms Runtime, Cases, Parties, Tasks, Approvals, Workflow Engine, Workflow, Tickets |
|
||||
| 4. Communication and participation | Delivery, participation, scheduling, channels, consultation | Portal, Postbox, Notifications, Mail, Campaign, Calendar, Scheduling, Poll, Appointments, Booking, Consultation, Committee, Addresses, Distribution Lists |
|
||||
| 5. Content, records, and evidence | Managed content, templates, records, knowledge, disclosure | Files, Templates, DMS, Records, Wiki, Transparency, Certificates |
|
||||
| 6. Data, reporting, and integration | Source access, staging, transformation, search, analytics, protocols | Connectors, Datasources, Dataflow, Reporting, Dashboard, REST, SOAP, XOE/V, XTA/OSCI, FIT-Connect, XRechnung, ERP adapters |
|
||||
| 7. Domain capabilities | Reusable public-sector subject matter | Projects, Procurement, Contracts, Grants, Resources, Assets, Facilities, Learning, Payments, Ledger, Permits, Inspections, Evaluation, Helpdesk |
|
||||
| 8. Product and sector packages | Versioned compositions, terminology, forms, processes, controls, reports, integration profiles | Signed configuration packages and reference packages; not runtime modules by default |
|
||||
|
||||
## Canonical institutional semantics
|
||||
|
||||
The connected model must keep these concepts distinct even where one UI
|
||||
combines them.
|
||||
|
||||
| Concept | Canonical answer | Owner or direction |
|
||||
| --- | --- | --- |
|
||||
| Institution and tenant | In which governed installation and tenant does work occur? | Tenancy and Organizations |
|
||||
| Organization and unit | Where is responsibility situated? | Organizations |
|
||||
| Function | Which named organizational responsibility can an incumbent hold? | Organizations |
|
||||
| Identity and account | Who is the person or machine, and through which account do they act? | Identity and Access |
|
||||
| Function assignment | Who holds or represents a function, for which interval and source? | IDM |
|
||||
| Role and permission | What application behavior may the acting principal perform? | Access, constrained by Policy |
|
||||
| Mandate and jurisdiction | Why is an institution, unit, or function competent to act on this subject, territory, population, or interval? | Mandates |
|
||||
| Service | What governed promise can an institution offer, to whom, under which prerequisites, evidence, channel, deadline, and responsibility? | Services; Portal presents it |
|
||||
| Case | Which concrete administrative matter is being handled? | Cases |
|
||||
| Party | In what procedural capacity does a person or organization participate, and who may represent or receive for it? | Parties; Identity/Organizations remain the subject owners |
|
||||
| Work item | What must a responsible actor do next? | Tasks and domain modules |
|
||||
| Workflow | How is work coordinated, including waits, human hand-offs, and governed actions? | Workflow Engine; Workflow is the optional editor |
|
||||
| Approval | Has a proposed action passed a configured review or separation-of-duties gate? | Approvals |
|
||||
| Decision | What formal institutional outcome was reached, by which competent authority, on which facts, rules, evidence, reasoning, and review path? | Decisions |
|
||||
| Evidence and record | What proves the input, state, action, effect, correction, and retained institutional memory? | Domain owner, Files/DMS/Records, and Audit |
|
||||
|
||||
### Extracted semantic modules
|
||||
|
||||
Four horizontal concepts passed the repository proof threshold. Their Core
|
||||
DTOs and provider protocols remain neutral; their persistent data, lifecycle,
|
||||
security, APIs, migrations, and recovery behavior now live in independent
|
||||
repositories.
|
||||
|
||||
#### Mandates
|
||||
|
||||
Mandates should own public or internal tasks, jurisdiction, responsibility,
|
||||
decision/signature authority, legal or organizational basis, and effective
|
||||
history. Organizations continues to own structures and functions; IDM owns
|
||||
incumbency; Access owns permissions; Policy owns constraints.
|
||||
|
||||
`govoplan-mandates` answers: *Was this function competent to act for this case
|
||||
at the relevant time, and on what basis?* Its resolver evaluates effective
|
||||
time, task, authority, unit, function, jurisdiction, subject, conflicts, legal
|
||||
basis, and evidence deterministically. Missing or ambiguous authority fails
|
||||
closed.
|
||||
|
||||
#### Services
|
||||
|
||||
Services should own versioned service definitions: audience, prerequisites,
|
||||
legal basis, evidence, fees, deadlines, channels, responsible unit/function,
|
||||
jurisdiction, forms, case/workflow/result bindings, remedies, service levels,
|
||||
and publication status. Portal presents and starts services but should not own
|
||||
their institutional definition.
|
||||
|
||||
`govoplan-services` now owns those exact versioned definitions. Portal is the
|
||||
first presentation consumer and Cases freezes the selected revision into its
|
||||
intake context. Availability is an independent capability so publication does
|
||||
not imply that all runtime prerequisites are satisfied.
|
||||
|
||||
#### Parties
|
||||
|
||||
Parties should own procedure-local roles and relationships: applicant,
|
||||
respondent, beneficiary, representative, joint applicant, delivery recipient,
|
||||
power or authority to represent, and permitted/preferred channels for the
|
||||
matter. Identity answers who the subject is; Organizations answers which
|
||||
institutional unit it is; Addresses owns contact points; Parties answers how
|
||||
the subject participates here.
|
||||
|
||||
`govoplan-parties` owns the shared effective-dated lifecycle. Cases retains a
|
||||
bounded compatibility projection only when the module is absent; that fallback
|
||||
contains no representation lifecycle and cannot silently become a second
|
||||
authority source.
|
||||
|
||||
#### Decisions
|
||||
|
||||
Decisions should own formal outcomes: subject, type, competent authority,
|
||||
facts, evidence, applicable rule versions, reasoning, operative result,
|
||||
conditions, effect, delivery/publication, remedy/review, correction, revocation,
|
||||
and links to observed effects. Approvals own review gates; Poll owns response
|
||||
collection; Committee owns deliberation, meetings, and votes; Workflow owns
|
||||
coordination.
|
||||
|
||||
`govoplan-decisions` owns the persistent lifecycle and protected reconstruction
|
||||
surface. Committee supplies deliberation context and records through the
|
||||
provider capability. Consumers retain exact Decision references without
|
||||
gaining table access.
|
||||
|
||||
## Source authority and integration maturity
|
||||
|
||||
Two independent dimensions must be recorded. They must not be collapsed into a
|
||||
single `sync` flag.
|
||||
|
||||
### Source-authority mode
|
||||
|
||||
| Mode | Meaning |
|
||||
| --- | --- |
|
||||
| `native_authoritative` | GovOPlaN owns the authoritative object and lifecycle. |
|
||||
| `external_authoritative` | The external system owns the object; GovOPlaN reads or acts through it. |
|
||||
| `external_mirror` | The external system is authoritative and GovOPlaN keeps a governed local projection or immutable snapshots. |
|
||||
| `governed_sync` | Both sides may change supported fields under explicit conflict and reconciliation rules. |
|
||||
| `governance_overlay` | GovOPlaN owns policy, responsibility, evidence, or coordination around an externally executed object. |
|
||||
| `linked_reference` | GovOPlaN keeps only a stable link and minimal display/provenance metadata. |
|
||||
|
||||
Authority may be declared per tenant, organization, service, object type,
|
||||
object, field group, or process step. A broad default must not hide a narrower
|
||||
override.
|
||||
|
||||
### Integration maturity
|
||||
|
||||
The implemented maturity ladder remains `discover`, `link`, `search`, `read`,
|
||||
`publish`, `synchronize`, `migrate`, and `replace`. Maturity says what an
|
||||
adapter can do. Source-authority mode says who owns truth in a particular
|
||||
configuration. For example, a connector may support `synchronize`, while a
|
||||
tenant deliberately configures it as `external_mirror`.
|
||||
|
||||
### Provider declaration
|
||||
|
||||
Every provider that reads or causes external effects must declare:
|
||||
|
||||
- owned object and field groups;
|
||||
- supported source-authority modes and integration maturity;
|
||||
- read, write, delete, search, preview, and dry-run operations;
|
||||
- revision/concurrency tokens, freshness, health, and bounded-read limits;
|
||||
- idempotency, retry, timeout, conflict, and outcome-unknown behavior;
|
||||
- evidence, audit, correction, rollback/compensation, and reconciliation paths;
|
||||
- degraded and outage behavior;
|
||||
- classification, purpose, retention, and secret-handling requirements.
|
||||
|
||||
The common provider declaration composes the external-reference, action/effect,
|
||||
connector-lifecycle, capability, operational-check, and documentation
|
||||
contracts. Core, release tooling, Ops, Docs, and configuration-package
|
||||
preflight validate it; Registry refuses to activate a declared external
|
||||
provider without bounded, sanitized runtime state.
|
||||
|
||||
## Cross-cutting contracts
|
||||
|
||||
The following contracts are mandatory for consequential domain objects. They
|
||||
should be shared reference DTOs and provider protocols, not shared domain
|
||||
tables in Core.
|
||||
|
||||
1. **Time and history:** valid-from/to, recorded-at, superseded-at, revision,
|
||||
change reason, and stable identity.
|
||||
2. **Actor and representation:** real account/identity, system or service
|
||||
account, represented account/function/party, delegation or power, and
|
||||
mandate reference.
|
||||
3. **Institutional context:** tenant, institution, organization unit, function,
|
||||
task/mandate, jurisdiction, service, case, and decision references.
|
||||
4. **Legal and policy basis:** typed, versioned references to rules,
|
||||
obligations, policies, exceptions, and the effective decision source.
|
||||
5. **Requested and observed effect:** intent, approval, dispatch, possible
|
||||
execution, confirmation, reconciliation, correction, and terminal evidence.
|
||||
6. **Evidence and provenance:** source, version, checksum, derivation,
|
||||
responsible actor, timestamps, and inspection links.
|
||||
7. **Information governance:** classification, purpose, legal basis, retention,
|
||||
hold, minimization, and disclosure state.
|
||||
8. **External source:** system/profile/object identity, authority mode,
|
||||
maturity, version, freshness, health, and conflict state.
|
||||
9. **Presentation:** language, accessibility, channel, explanation, and
|
||||
configured availability.
|
||||
|
||||
Existing contracts already cover substantial parts of items 1, 2, 5, 6, 8,
|
||||
and 9. New work should extend those contracts instead of creating parallel DTO
|
||||
families.
|
||||
|
||||
## Existing module direction changes
|
||||
|
||||
### Datasources becomes the governed data and register catalogue
|
||||
|
||||
The implemented live/cached/static, staging, immutable materialization, and
|
||||
publication model includes typed governance metadata for owner/steward,
|
||||
authoritative source and authority mode, legal basis and purpose, semantic
|
||||
definition, quality and freshness policy, classification, transfer agreement,
|
||||
correction process, affected services/processes, and dependent flows,
|
||||
reports, controls, and decisions. Connector credentials and protocol behavior
|
||||
remain outside Datasources.
|
||||
|
||||
### Projects grows into portfolio and change governance
|
||||
|
||||
The Projects boundary already includes portfolios and goals. Extend it through
|
||||
versioned objectives/outcomes, dependencies, capacity, benefits, change impact,
|
||||
and links to mandates, services, risks, contracts, resources, and indicators.
|
||||
Do not create a separate Goals module before more than one domain proves an
|
||||
independent goal lifecycle.
|
||||
|
||||
### Reporting becomes evidence-backed institutional measurement
|
||||
|
||||
Every report, measure, and indicator should explain the institutional question
|
||||
or obligation it serves, owner, source/materialization and flow revision,
|
||||
freshness/quality, calculation version, visibility/purpose limits, publication,
|
||||
and decisions or actions that consumed it. Reporting owns presentation and
|
||||
execution; source and transformation owners retain their domains.
|
||||
|
||||
### Risk Compliance becomes the horizontal assurance model
|
||||
|
||||
Sanctions screening remains a complete vertical slice. The broader reusable
|
||||
model is:
|
||||
|
||||
```text
|
||||
Obligation -> governed object -> risk -> control -> evidence -> finding -> measure -> effectiveness review
|
||||
```
|
||||
|
||||
Risk Compliance now persists that effective-dated, immutable-revision assurance
|
||||
graph, exposes bounded tenant-safe traversal/search/editing, and projects each
|
||||
completed sanctions run into it idempotently. Policy
|
||||
owns enforceable rules and decisions; Audit owns immutable event evidence;
|
||||
domain modules own the governed objects and corrective actions.
|
||||
|
||||
### Connectors exposes authority and effect behavior
|
||||
|
||||
Connector direction (`consume`, `publish`, `bidirectional`) remains useful but
|
||||
is not enough. Profiles and bindings need the source-authority mode and
|
||||
provider declaration above. ERP remains an integration family: finance,
|
||||
workforce, procurement, asset, or other domain modules own semantics while
|
||||
connectors own transport and source interaction.
|
||||
|
||||
### Geography starts as a reference contract
|
||||
|
||||
Before adding a `govoplan-geo` module, define a common reference shape for
|
||||
coordinates, geometry, administrative area, address/location, CRS, source,
|
||||
accuracy, validity, and external GIS identity. Create a repository only when
|
||||
GovOPlaN must own spatial datasets, topology, or independent geospatial
|
||||
lifecycles rather than link to an external GIS.
|
||||
|
||||
## Product and sector packages
|
||||
|
||||
A module says what capability can exist. A product package says how capabilities
|
||||
work together for a bounded outcome. A sector package specializes vocabulary,
|
||||
forms, rules, process baselines, controls, reports, and integration profiles
|
||||
without forking the platform.
|
||||
|
||||
The signed configuration-package mechanism distinguishes:
|
||||
|
||||
- **reference package:** tested composition proving a journey and its recovery
|
||||
behavior;
|
||||
- **product package:** reusable operating capability such as governed
|
||||
communication, service-to-decision, procurement/contracts, or governed BI;
|
||||
- **sector package:** institutional specialization such as municipality,
|
||||
university/research, ministry/program, regulator, grants authority, or
|
||||
committee/council;
|
||||
- **deployment profile:** supported infrastructure and operational topology;
|
||||
- **integration profile:** supported set of external systems, authority modes,
|
||||
bindings, and health expectations.
|
||||
|
||||
Packages may require modules and capabilities, but package definitions remain
|
||||
configuration and evidence. They do not gain access to module-owned tables.
|
||||
|
||||
## Module portfolio metadata
|
||||
|
||||
Repository category is not capability maturity. The runtime manifest, release
|
||||
catalog, Docs projection, and meta repository inventory use one
|
||||
machine-readable declaration with at least:
|
||||
|
||||
- architecture layer and module kind;
|
||||
- lifecycle/maturity claim: `concept`, `scaffold`, `vertical_slice`,
|
||||
`reference_ready`, `supported`, or `lts`;
|
||||
- evidence supporting the claim and known limits;
|
||||
- supported source-authority modes;
|
||||
- owned and explicitly non-owned concepts;
|
||||
- provided/required capabilities and interfaces;
|
||||
- reference packages and target-tested providers;
|
||||
- migration, upgrade, recovery, security, and operations documentation.
|
||||
|
||||
Maturity is a release claim and must be checked against evidence. A manifest
|
||||
must not become “supported” merely because a maintainer changes one string.
|
||||
|
||||
Create a repository only when the capability has distinct data ownership,
|
||||
independent installability, technical assets, a security/lifecycle profile, a
|
||||
release reason, more than one consumer or a proven reference process, and tests
|
||||
that justify the boundary. Otherwise use a shared DTO, provider capability,
|
||||
submodule, configuration fragment, package, or profile.
|
||||
|
||||
## Implemented migration sequence
|
||||
|
||||
### 0. Align the portfolio and contracts - complete
|
||||
|
||||
- This reconciliation is canonical in the meta repository and mirrored to the
|
||||
Gitea wiki.
|
||||
- All 62 source manifests carry validated evidence-based architecture metadata.
|
||||
- External-reference, action/effect, operational-health, ownership, policy,
|
||||
audit, and documentation primitives compose into one enforced provider
|
||||
declaration and sanitized runtime-state contract.
|
||||
- Institutional context, legal basis, evidence, presentation, external source,
|
||||
information governance, temporal revision, and geo references are shared
|
||||
Core DTOs rather than shared domain tables.
|
||||
|
||||
### 1. Prove responsibility and formal outcome - complete
|
||||
|
||||
- Mandate and Decision contracts, deterministic resolution, lifecycle
|
||||
transitions, persistence providers, APIs, permissions, migrations, recovery,
|
||||
and tests are implemented.
|
||||
- Committee and the SQL-backed institutional fixture prove effective-time
|
||||
authority, persisted meeting/agendum/vote/minute context, approval context,
|
||||
reasoning, evidence, observed effect, correction/revision rules, protected
|
||||
reconstruction, and review references.
|
||||
- The independent Mandates and Decisions repositories were created only after
|
||||
persistence and reuse passed the repository threshold.
|
||||
|
||||
### 2. Separate service and party semantics - complete
|
||||
|
||||
- Portal remains the presentation surface while Services owns reusable,
|
||||
versioned definitions and explainable availability.
|
||||
- Parties owns procedure roles, contact snapshots, and append-only
|
||||
representation/revocation authority; Cases consumes the common resolver.
|
||||
- `product.service-to-decision` proves both through a portable administrative
|
||||
service composition.
|
||||
- Forms owns immutable, versioned schemas while Forms Runtime owns drafts,
|
||||
server validation, submission receipts, status/evidence history, and exact
|
||||
Service/Form provenance. Portal delegates Form launch through the runtime
|
||||
capability and fails closed when it is unavailable.
|
||||
|
||||
### 3. Complete governed data, portfolio, and assurance - vertical slices complete
|
||||
|
||||
- Datasources carries typed governance through staging and immutable
|
||||
materializations, with bounded catalogue filters and dependency references.
|
||||
- Reporting owns immutable semantic definitions, safe execution, quality gates,
|
||||
provenance, schedules, saved views, pivoting, and export/import assessment
|
||||
without taking source or transformation ownership.
|
||||
- Risk Compliance persists the horizontal obligation/risk/control/evidence/
|
||||
finding/measure graph and projects sanctions runs idempotently.
|
||||
- Projects persists portfolio/outcome/change-governance revisions as a
|
||||
consuming domain without becoming a second policy or reporting engine.
|
||||
|
||||
### 4. Package repeatable public-sector outcomes - complete at product maturity
|
||||
|
||||
- Governed communication, governed data/assurance, and service-to-decision are
|
||||
portable product package manifests with repository-local evidence.
|
||||
- Package preflight enforces module, capability, provider authority, health,
|
||||
freshness, and recovery expectations without cross-module table access.
|
||||
- Sector and `reference_ready` claims remain gated on target-environment,
|
||||
recovery, accessibility, privacy, security, and operator evidence. This is a
|
||||
maturity gate, not missing architecture implementation.
|
||||
|
||||
## What remains after the executable architecture slice
|
||||
|
||||
The remaining work is not another Core or cross-module architecture rewrite.
|
||||
It falls into two explicitly different categories, neither of which can be
|
||||
truthfully completed by adding generic platform code:
|
||||
|
||||
1. **Concrete provider packages:** the Committee ballot adapter contract is
|
||||
complete, but a real secret/electronic ballot provider requires a selected
|
||||
protocol and product decisions for voter eligibility, custody, secrecy,
|
||||
recount, challenge, retention, and operational assurance. Equivalent future
|
||||
adapters must satisfy the declared provider and recovery gates.
|
||||
Provider selection and certification are tracked in
|
||||
[Committee #1](https://git.add-ideas.de/GovOPlaN/govoplan-committee/issues/1).
|
||||
2. **Target-produced maturity evidence:** `reference_ready`, `supported`, and
|
||||
`lts` cannot be generated from source code. An exact release and deployment
|
||||
must produce signed, expiring accessibility, privacy, security, operator,
|
||||
provider, backup/restore, rollback, and recovery-drill evidence. The verifier
|
||||
and schemas are implemented; the actual claims require those real runs.
|
||||
The pinned-release evidence run is tracked in
|
||||
[GovOPlaN #37](https://git.add-ideas.de/GovOPlaN/govoplan/issues/37).
|
||||
|
||||
Forms and Forms Runtime no longer constitute an architecture gap. Remaining
|
||||
depth includes anonymous/public identity profiles, concrete file and signature
|
||||
providers, conditional multi-page/localized authoring, and automatic handoff
|
||||
adapters. Those additions use the implemented immutable definition, runtime,
|
||||
policy, evidence, service-launch, and domain-owner boundaries rather than
|
||||
requiring another platform split. They are tracked as
|
||||
[Forms #3](https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/3) and
|
||||
Forms Runtime [#2](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/2),
|
||||
[#3](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/3), and
|
||||
[#4](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/4).
|
||||
|
||||
Everything else described as architecture in this document now has a
|
||||
repository owner, versioned contract, bounded implementation, migration and
|
||||
recovery boundary where state exists, documentation, and executable evidence.
|
||||
Further work in those modules is product breadth, UX depth, provider adoption,
|
||||
and evidence renewal.
|
||||
|
||||
## Delivery tracking
|
||||
|
||||
The completed cross-repository architecture epic is
|
||||
[GovOPlaN #29](https://git.add-ideas.de/GovOPlaN/govoplan/issues/29).
|
||||
Its implementation work packages and resulting owners are:
|
||||
|
||||
- [Core #279](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/279):
|
||||
validated module architecture and provider authority declarations;
|
||||
- [Core #280](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/280):
|
||||
shared institutional-context and governed reference primitives;
|
||||
- [GovOPlaN #30](https://git.add-ideas.de/GovOPlaN/govoplan/issues/30) and
|
||||
[govoplan-mandates](https://git.add-ideas.de/GovOPlaN/govoplan-mandates):
|
||||
Mandates semantics and persistent resolver;
|
||||
- [GovOPlaN #31](https://git.add-ideas.de/GovOPlaN/govoplan/issues/31) and
|
||||
[govoplan-services](https://git.add-ideas.de/GovOPlaN/govoplan-services):
|
||||
Services semantics, catalogue, and availability;
|
||||
- [GovOPlaN #32](https://git.add-ideas.de/GovOPlaN/govoplan/issues/32) and
|
||||
[govoplan-parties](https://git.add-ideas.de/GovOPlaN/govoplan-parties):
|
||||
Parties and representation semantics and resolver;
|
||||
- [GovOPlaN #33](https://git.add-ideas.de/GovOPlaN/govoplan/issues/33) and
|
||||
[govoplan-decisions](https://git.add-ideas.de/GovOPlaN/govoplan-decisions):
|
||||
formal Decisions semantics and registry;
|
||||
- [Datasources #6](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/6):
|
||||
governed data/register catalogue;
|
||||
- [Risk Compliance #7](https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance/issues/7):
|
||||
horizontal assurance graph;
|
||||
- [GovOPlaN #34](https://git.add-ideas.de/GovOPlaN/govoplan/issues/34):
|
||||
product and sector package classes; and
|
||||
- [Docs #19](https://git.add-ideas.de/GovOPlaN/govoplan-docs/issues/19):
|
||||
configured architecture, maturity, and source-authority explanations;
|
||||
- [Forms #2](https://git.add-ideas.de/GovOPlaN/govoplan-forms/issues/2):
|
||||
immutable reusable definitions and the designer surface; and
|
||||
- [Forms Runtime #1](https://git.add-ideas.de/GovOPlaN/govoplan-forms-runtime/issues/1):
|
||||
definition-aware submissions and Portal service launch.
|
||||
|
||||
Existing Projects #1, Reporting #4, Portal #1, Cases #1, Datasources #1,
|
||||
Risk Compliance #2, GovOPlaN #14, and GovOPlaN #19 carry product-depth and
|
||||
reference-readiness work instead of duplicating the completed architecture
|
||||
contract.
|
||||
|
||||
## Completion evidence
|
||||
|
||||
The architecture direction is established by the following executable and
|
||||
machine-enforced evidence:
|
||||
|
||||
- the `product.service-to-decision` composition and SQL-backed golden fixture
|
||||
retain institutional context from service entry through a persisted case,
|
||||
party, authority/work context, committee deliberation, decision, observed
|
||||
communication effect, minute/record, and review references;
|
||||
- the Portal/Form journey retains the exact published Service and Form
|
||||
revisions through persisted draft state, validates on the server, and returns
|
||||
the same submission on an idempotent launch replay;
|
||||
- the system can answer who acted, for whom, in which function, under which
|
||||
mandate and jurisdiction, using which rule and evidence versions;
|
||||
- every implemented external binding declares authority mode, maturity,
|
||||
operations, health, freshness, conflict, and recovery behavior, and Registry
|
||||
rejects a declaration without sanitized runtime state;
|
||||
- every material report or decision can be reconstructed from governed source
|
||||
and transformation versions;
|
||||
- product/package manifests are portable without cross-module table access or
|
||||
code forks, while future sector packages inherit the same signed-package
|
||||
constraints; and
|
||||
- documentation and Ops explain the configured composition and its limits to
|
||||
users, administrators, operators, and auditors.
|
||||
|
||||
These criteria complete the architecture contract at `vertical_slice` and
|
||||
`product` maturity. They do not waive the separately enforced evidence needed
|
||||
for a module or package to claim `reference_ready`, `supported`, or `lts`.
|
||||
The capability-fit verifier now computes that cumulative readiness gate from
|
||||
independently signed, expiring claims bound to the exact assessed release,
|
||||
installed payload, deployment subject, controls, and artifact hashes. Actual
|
||||
target runs and recovery drills remain operator-produced evidence.
|
||||
@@ -0,0 +1,147 @@
|
||||
# Recovery And Rollback Guarantees
|
||||
|
||||
## Principle
|
||||
|
||||
GovOPlaN must prove recovery claims with durable state recorded before and
|
||||
after side effects. A failed operation is not automatically rolled back merely
|
||||
because the previous application image still exists. Database schema and
|
||||
external effects may make release rollback unsafe.
|
||||
|
||||
Core therefore distinguishes five recovery modes:
|
||||
|
||||
| Mode | Meaning |
|
||||
| --- | --- |
|
||||
| `atomic` | One database transaction either commits or rolls back. No external effect is claimed. |
|
||||
| `compensation` | Durable evidence identifies explicit inverse actions for completed effects. |
|
||||
| `snapshot_restore` | A separately verified backup reference and restore procedure exist. |
|
||||
| `forward_recovery` | Repair or resume the current version; reverting code/configuration is not claimed safe. |
|
||||
| `irreversible` | No automated recovery is claimed and an approval reference is mandatory. |
|
||||
|
||||
An operation plan must include verification steps. Compensation requires named
|
||||
compensation steps, snapshot restore requires a verified backup reference,
|
||||
forward recovery requires repair steps, and irreversible work requires explicit
|
||||
approval.
|
||||
|
||||
## Core Recovery Ledger
|
||||
|
||||
Core stores recovery operations and append-only, hash-chained checkpoints in
|
||||
PostgreSQL. The contract provides:
|
||||
|
||||
- installation/module/resource identity;
|
||||
- an idempotency key bound to a canonical request hash;
|
||||
- recovery mode, preconditions, verification steps, and references;
|
||||
- optional runtime lease holder and fencing token;
|
||||
- explicit planned, prepared, running, recovery-required, recovering,
|
||||
succeeded, recovered, failed, outcome-unknown, and manual-intervention states;
|
||||
- an evidence-chain head and sequence count;
|
||||
- rejection of plaintext secrets in metadata or evidence.
|
||||
|
||||
Preparation cannot succeed without durable precondition evidence. A non-atomic
|
||||
operation cannot hide a partial effect by transitioning directly from running
|
||||
to failed. Success and recovery require explicit verification evidence with at
|
||||
least one check. The ledger verifies its hash chain before evidence is trusted.
|
||||
|
||||
This is a platform contract, not an assertion that every existing module
|
||||
operation has adopted it. Module operations with external or multi-resource
|
||||
effects must be migrated to the ledger before claiming these guarantees.
|
||||
|
||||
## Deployment Journal
|
||||
|
||||
Every `govoplan-deploy apply` begins an operation journal before it pulls images
|
||||
or mutates runtime state. The private installation directory records:
|
||||
|
||||
```text
|
||||
operations/<operation-id>/operation.json
|
||||
operations/<operation-id>/before/
|
||||
applied-state/
|
||||
```
|
||||
|
||||
Each stage is hash-chained. The previous applied bundle is copied with per-file
|
||||
SHA-256 evidence. Applied state is replaced atomically after health verification;
|
||||
an interrupted replacement restores its previous directory.
|
||||
New journals also bind the complete desired deployment plan, snapshot
|
||||
availability, failure summary, recovery mode, and terminal status to the
|
||||
evidence chain. Recovery verifies every snapshot entry and checksum before it
|
||||
changes any live bundle file, then replaces each live file atomically. A crash
|
||||
between file replacements is recoverable by rerunning the same idempotent
|
||||
recovery command under the deployment lock.
|
||||
|
||||
Inspect operations:
|
||||
|
||||
```sh
|
||||
python tools/deployment/govoplan-deploy.py operations \
|
||||
--directory /srv/govoplan/default
|
||||
```
|
||||
|
||||
Recover the latest failed operation, or provide its identifier:
|
||||
|
||||
```sh
|
||||
python tools/deployment/govoplan-deploy.py recover \
|
||||
--directory /srv/govoplan/default \
|
||||
--operation-id 20260801T120000Z-1234abcd
|
||||
```
|
||||
|
||||
Add `--apply` only after reviewing the reported action.
|
||||
|
||||
## Migration Boundary
|
||||
|
||||
Before database migration starts, a failed deployment with a verified prior
|
||||
applied snapshot may restore its prior release/configuration bundle and
|
||||
reconcile that desired state.
|
||||
|
||||
As soon as migration starts, the journal permanently changes to
|
||||
`forward_recovery`. It will not restore old application configuration because
|
||||
old code may not understand the new schema. Recovery then means one of:
|
||||
|
||||
1. fix and re-run the current release;
|
||||
2. deploy a newer compatible repair release;
|
||||
3. restore a separately verified, coordinated database/object/key backup and
|
||||
then deploy the matching release.
|
||||
|
||||
The deployment tool does not create or validate that database backup. A
|
||||
`backup-required` annotation on the Kubernetes migration Job is an operator
|
||||
gate, not backup evidence. Production automation must provide a backup hook or
|
||||
external backup controller whose artifact, timestamp, scope, encryption key,
|
||||
and restore test can be referenced from the recovery record.
|
||||
|
||||
## Scaled Nodes
|
||||
|
||||
Recovery actions must be safe across replicas:
|
||||
|
||||
- drain affected API and worker nodes before incompatible changes;
|
||||
- use the deployment-wide PostgreSQL advisory lock for schema migration;
|
||||
- use distributed leases and fencing tokens for singleton or externally
|
||||
visible effects;
|
||||
- use idempotency keys for retried commands and jobs;
|
||||
- retain shared object keys and database references until deletion succeeds;
|
||||
- classify uncertain external outcomes instead of retrying blindly;
|
||||
- verify the exact software/module composition after replacement.
|
||||
|
||||
Campaign generated-message objects now follow this model: object writes are
|
||||
compensated when a build fails before database commit, workers verify stored
|
||||
size and digest before delivery, and retention keeps the database reference
|
||||
when storage deletion fails. A hard process loss between object creation and
|
||||
database commit can still leave an orphan object; an inventory reconciler is a
|
||||
separate operational slice and must use the build-specific object prefix.
|
||||
|
||||
## Required Drills
|
||||
|
||||
Record evidence for at least these scenarios before production acceptance:
|
||||
|
||||
1. Kill an API replica and verify traffic continues without session loss.
|
||||
2. Drain and replace a worker while work is queued and while one job is active.
|
||||
3. Start two migration jobs and verify only one mutates schema.
|
||||
4. Kill the fenced scheduler and verify one replacement acquires a higher
|
||||
fencing token.
|
||||
5. Fail deployment before migration and restore the prior applied bundle.
|
||||
6. Fail deployment after migration and verify old configuration is not
|
||||
restored.
|
||||
7. Restore PostgreSQL, object storage, and encryption keys to one coordinated
|
||||
recovery point and verify representative object hashes.
|
||||
8. Interrupt object storage during Campaign build and retention and verify
|
||||
compensation/reference-preservation behavior.
|
||||
9. Tamper with a deployment or Core recovery checkpoint and verify chain
|
||||
validation rejects it.
|
||||
|
||||
No runbook, status badge, or green health endpoint substitutes for a dated,
|
||||
repeatable restore drill against the actual deployment topology.
|
||||
@@ -2,9 +2,16 @@
|
||||
|
||||
## Status
|
||||
|
||||
This is the selected product-development sequence as of 2026-07-21. It turns
|
||||
the long-term connected-platform roadmap into five demonstrable journeys while
|
||||
the Workflow program remains deliberately deferred.
|
||||
This is the selected product-development sequence, originally chosen on
|
||||
2026-07-21 and reconciled with the implemented platform on 2026-07-31. It turns
|
||||
the long-term connected-platform roadmap into five demonstrable journeys.
|
||||
Workflow Engine and the optional Workflow editor now exist, but they are used
|
||||
by a stage only when its package explicitly composes and proves them; Workflow
|
||||
is not an automatic dependency of every journey.
|
||||
|
||||
The institutional semantics and source-authority model applied to these stages
|
||||
are defined in the
|
||||
[Institutional Governance Target Architecture](INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
|
||||
|
||||
The stages are ordered, but they are not monolithic releases. Each stage is
|
||||
delivered as small, reviewable, green increments and is complete only when its
|
||||
|
||||
@@ -31,6 +31,7 @@ Generated from `repositories.json`. Use that JSON file as the machine-readable s
|
||||
| `govoplan-dashboard` | `platform` | `../govoplan-dashboard` | [govoplan-dashboard](https://git.add-ideas.de/GovOPlaN/govoplan-dashboard) |
|
||||
| `govoplan-dataflow` | `platform` | `../govoplan-dataflow` | [govoplan-dataflow](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow) |
|
||||
| `govoplan-datasources` | `platform` | `../govoplan-datasources` | [govoplan-datasources](https://git.add-ideas.de/GovOPlaN/govoplan-datasources) |
|
||||
| `govoplan-decisions` | `domain` | `../govoplan-decisions` | [govoplan-decisions](https://git.add-ideas.de/GovOPlaN/govoplan-decisions) |
|
||||
| `govoplan-dms` | `domain` | `../govoplan-dms` | [govoplan-dms](https://git.add-ideas.de/GovOPlaN/govoplan-dms) |
|
||||
| `govoplan-dist-lists` | `domain` | `../govoplan-dist-lists` | [govoplan-dist-lists](https://git.add-ideas.de/GovOPlaN/govoplan-dist-lists) |
|
||||
| `govoplan-docs` | `platform` | `../govoplan-docs` | [govoplan-docs](https://git.add-ideas.de/GovOPlaN/govoplan-docs) |
|
||||
@@ -50,10 +51,12 @@ Generated from `repositories.json`. Use that JSON file as the machine-readable s
|
||||
| `govoplan-learning` | `domain` | `../govoplan-learning` | [govoplan-learning](https://git.add-ideas.de/GovOPlaN/govoplan-learning) |
|
||||
| `govoplan-ledger` | `domain` | `../govoplan-ledger` | [govoplan-ledger](https://git.add-ideas.de/GovOPlaN/govoplan-ledger) |
|
||||
| `govoplan-mail` | `domain` | `../govoplan-mail` | [govoplan-mail](https://git.add-ideas.de/GovOPlaN/govoplan-mail) |
|
||||
| `govoplan-mandates` | `domain` | `../govoplan-mandates` | [govoplan-mandates](https://git.add-ideas.de/GovOPlaN/govoplan-mandates) |
|
||||
| `govoplan-notifications` | `platform` | `../govoplan-notifications` | [govoplan-notifications](https://git.add-ideas.de/GovOPlaN/govoplan-notifications) |
|
||||
| `govoplan-ops` | `platform` | `../govoplan-ops` | [govoplan-ops](https://git.add-ideas.de/GovOPlaN/govoplan-ops) |
|
||||
| `govoplan-organizations` | `platform` | `../govoplan-organizations` | [govoplan-organizations](https://git.add-ideas.de/GovOPlaN/govoplan-organizations) |
|
||||
| `govoplan-payments` | `domain` | `../govoplan-payments` | [govoplan-payments](https://git.add-ideas.de/GovOPlaN/govoplan-payments) |
|
||||
| `govoplan-parties` | `domain` | `../govoplan-parties` | [govoplan-parties](https://git.add-ideas.de/GovOPlaN/govoplan-parties) |
|
||||
| `govoplan-permits` | `domain` | `../govoplan-permits` | [govoplan-permits](https://git.add-ideas.de/GovOPlaN/govoplan-permits) |
|
||||
| `govoplan-policy` | `platform` | `../govoplan-policy` | [govoplan-policy](https://git.add-ideas.de/GovOPlaN/govoplan-policy) |
|
||||
| `govoplan-poll` | `domain` | `../govoplan-poll` | [govoplan-poll](https://git.add-ideas.de/GovOPlaN/govoplan-poll) |
|
||||
@@ -67,6 +70,7 @@ Generated from `repositories.json`. Use that JSON file as the machine-readable s
|
||||
| `govoplan-risk-compliance` | `domain` | `../govoplan-risk-compliance` | [govoplan-risk-compliance](https://git.add-ideas.de/GovOPlaN/govoplan-risk-compliance) |
|
||||
| `govoplan-scheduling` | `domain` | `../govoplan-scheduling` | [govoplan-scheduling](https://git.add-ideas.de/GovOPlaN/govoplan-scheduling) |
|
||||
| `govoplan-search` | `platform` | `../govoplan-search` | [govoplan-search](https://git.add-ideas.de/GovOPlaN/govoplan-search) |
|
||||
| `govoplan-services` | `domain` | `../govoplan-services` | [govoplan-services](https://git.add-ideas.de/GovOPlaN/govoplan-services) |
|
||||
| `govoplan-tasks` | `domain` | `../govoplan-tasks` | [govoplan-tasks](https://git.add-ideas.de/GovOPlaN/govoplan-tasks) |
|
||||
| `govoplan-templates` | `domain` | `../govoplan-templates` | [govoplan-templates](https://git.add-ideas.de/GovOPlaN/govoplan-templates) |
|
||||
| `govoplan-tenancy` | `platform` | `../govoplan-tenancy` | [govoplan-tenancy](https://git.add-ideas.de/GovOPlaN/govoplan-tenancy) |
|
||||
|
||||
@@ -1,156 +1,183 @@
|
||||
# Scaling And Multi-Host Deployment
|
||||
|
||||
## What Is Implemented
|
||||
## Implemented Contract
|
||||
|
||||
The default installer now supports explicit same-host horizontal scaling:
|
||||
GovOPlaN now supports a stateless application tier backed by logically shared
|
||||
state services. The runtime roles are independently replaceable API, WebUI,
|
||||
worker, and scheduler processes. Every replica in one installation must use the
|
||||
same immutable release composition and the same:
|
||||
|
||||
- one HAProxy container accepts the published HTTP endpoint;
|
||||
- one or more WebUI containers serve assets and proxy API requests;
|
||||
- one or more API containers serve normal domain traffic;
|
||||
- one or more Celery workers consume shared Redis queues;
|
||||
- one migration job runs before replacement;
|
||||
- exactly one Celery scheduler runs;
|
||||
- PostgreSQL, Redis, and file storage are shared by every runtime replica.
|
||||
- `GOVOPLAN_INSTALLATION_ID`;
|
||||
- PostgreSQL database;
|
||||
- Redis broker and coordination service;
|
||||
- `MASTER_KEY_B64` and deployment secret references;
|
||||
- enabled-module graph;
|
||||
- S3-compatible object-storage namespace.
|
||||
|
||||
HAProxy discovers Compose replicas through Docker's internal DNS and performs
|
||||
health-checked round-robin WebUI balancing and least-connection API balancing.
|
||||
It does not mount the Docker socket. Replica counts live in
|
||||
`installation.json`, so `configure`, `plan`, `apply`, and the receipt agree on
|
||||
the desired topology.
|
||||
The application tier must not use node-local durable business data in a
|
||||
multi-host deployment. Files owns managed file metadata while Core provides the
|
||||
storage-backend contract. Campaign build artifacts are stored under opaque
|
||||
object keys and workers read those objects from the shared backend. Temporary
|
||||
build and materialization directories may remain node-local because they are
|
||||
discardable.
|
||||
|
||||
This is not multi-host scheduling. Docker Compose's bridge network belongs to
|
||||
one Docker Engine. Adding a second machine requires an orchestrator or
|
||||
deployment manager that can place equivalent role definitions on multiple
|
||||
hosts.
|
||||
Core validates three explicit state profiles:
|
||||
|
||||
## Recommended Topologies
|
||||
| Profile | Supported shape | Storage rule |
|
||||
| --- | --- | --- |
|
||||
| `local` | One API and one worker process for development | Local filesystem permitted. |
|
||||
| `host-shared` | Multiple processes on one Docker host | A shared host volume is permitted; PostgreSQL and Redis are required. |
|
||||
| `shared` | Multiple independent hosts | PostgreSQL, Redis, and S3-compatible object storage are required. |
|
||||
|
||||
### One Host
|
||||
`shared` also requires a stable installation identifier. Module package
|
||||
mutation is blocked in this profile: build and verify a new immutable release,
|
||||
then roll the complete cluster to it.
|
||||
|
||||
Use the generated Compose bundle:
|
||||
## Same-Host Compose
|
||||
|
||||
The generated Compose bundle provides:
|
||||
|
||||
```text
|
||||
client
|
||||
-> external TLS proxy, when required
|
||||
-> managed HAProxy
|
||||
-> WebUI replica(s)
|
||||
-> managed HAProxy API listener
|
||||
-> API replica(s)
|
||||
client -> TLS proxy -> HAProxy -> WebUI replicas -> HAProxy -> API replicas
|
||||
|
||||
API/worker/scheduler
|
||||
-> PostgreSQL
|
||||
-> Redis
|
||||
-> local volume, managed Garage, or external S3
|
||||
API/worker/scheduler -> PostgreSQL
|
||||
-> Redis
|
||||
-> local volume, managed Garage, or external S3
|
||||
```
|
||||
|
||||
This improves concurrency and permits rolling process replacement, but the host
|
||||
and every managed stateful component remain single failure domains. Two API
|
||||
containers on one failed host do not provide host-level availability.
|
||||
HAProxy discovers Compose replicas through Docker DNS and performs health-aware
|
||||
balancing without mounting the Docker socket. This improves concurrency and
|
||||
permits process replacement, but the Docker host and installer-managed stateful
|
||||
services remain single failure domains. Generated Compose therefore declares
|
||||
the `host-shared` state profile even when its shared storage happens to be an
|
||||
external S3 service. Its API backend checks `/health/ready`, so drain or
|
||||
coordination loss removes a replica from rotation. Container, load-balancer,
|
||||
and Kubernetes probes send the configured public host explicitly, keeping
|
||||
readiness compatible with strict trusted-host validation.
|
||||
|
||||
### Multiple Hosts
|
||||
Managed Garage is a convenient single-node S3-compatible service. It is not a
|
||||
multi-host storage cluster. Use an independently operated Garage cluster or
|
||||
another S3-compatible service for the `shared` profile.
|
||||
|
||||
Use Kubernetes, Nomad, Docker Swarm, or another reviewed orchestrator. Do not
|
||||
extend the Compose installer into a proprietary scheduler. The target topology
|
||||
is:
|
||||
## Kubernetes Export
|
||||
|
||||
```text
|
||||
public TLS ingress/load balancer
|
||||
-> WebUI replicas on at least two nodes
|
||||
-> internal API service/load balancer
|
||||
-> API replicas on at least two nodes
|
||||
The deployment compiler exports a stateless Kubernetes runtime when PostgreSQL,
|
||||
Redis, and S3 are all external:
|
||||
|
||||
queue-specific worker pools on worker nodes
|
||||
one fenced scheduler
|
||||
one fenced migration/deployment job
|
||||
|
||||
shared PostgreSQL
|
||||
shared Redis
|
||||
multi-node Garage or external S3
|
||||
shared secret/config provider
|
||||
central logs, metrics, and health alerts
|
||||
```sh
|
||||
python tools/deployment/govoplan-deploy.py render-kubernetes \
|
||||
--directory /srv/govoplan/default \
|
||||
--namespace govoplan \
|
||||
--secret-name govoplan-runtime \
|
||||
--tls-secret-name govoplan-tls \
|
||||
--ingress-class-name nginx \
|
||||
--output /srv/govoplan/default/kubernetes.json
|
||||
```
|
||||
|
||||
All API and worker nodes must receive the same immutable software composition,
|
||||
`MASTER_KEY_B64`, database URL, Redis URL, enabled-module graph, and object
|
||||
storage binding. Node-local file storage is not valid in this topology.
|
||||
The export contains a Namespace, tokenless ServiceAccount, non-secret
|
||||
ConfigMap, API/WebUI/worker/scheduler Deployments, Services, Pod disruption
|
||||
budgets, Ingress, and a release-specific migration Job. It deliberately emits
|
||||
no Secret values, persistent volume, PostgreSQL, Redis, or object-store
|
||||
deployment. Export is rejected unless both release images use immutable
|
||||
`image@sha256:...` references.
|
||||
|
||||
## Adding Capacity
|
||||
Create the named Secret through the cluster's secret-management path. The
|
||||
command prints the exact required key contract. Review the generated
|
||||
`FORWARDED_ALLOW_IPS` value and replace it with the exact ingress-proxy network
|
||||
before production use.
|
||||
|
||||
### API And WebUI
|
||||
The generated containers run as non-root with a read-only root filesystem and
|
||||
an ephemeral `/tmp`. Runtime Deployments wait for the exact configured database
|
||||
migration heads before starting. The API exposes `/health/ready`, which fails
|
||||
while that API node is draining or cannot prove its runtime-coordination
|
||||
heartbeat.
|
||||
|
||||
Increase API replicas for measured request concurrency, CPU saturation, or
|
||||
latency after checking database load. Increase WebUI replicas for static asset
|
||||
and proxy capacity. No sticky session should be required because durable
|
||||
sessions and throttling use shared services, but this must remain covered by
|
||||
multi-replica integration tests.
|
||||
## Runtime Coordination
|
||||
|
||||
Every added API process also adds database connections. Set the application
|
||||
pool size and the total replica ceiling against PostgreSQL's connection budget;
|
||||
adding replicas can otherwise reduce throughput.
|
||||
Each API and worker incarnation registers in PostgreSQL with its role, software
|
||||
version, module-composition hash, queue set, and heartbeat. Ops shows active,
|
||||
draining, stopped, and stale nodes and compares active counts with configured
|
||||
replica expectations.
|
||||
|
||||
### Workers
|
||||
An operator may request or cancel drain from Ops:
|
||||
|
||||
Workers are not placed behind a load balancer. They compete for jobs on shared
|
||||
Redis queues. Add workers by queue and cap each pool according to the external
|
||||
system it calls. SMTP, IMAP, directory, and connector jobs often reach provider
|
||||
rate limits before CPU limits.
|
||||
- API readiness becomes unavailable on the next heartbeat so the load balancer
|
||||
stops assigning new requests.
|
||||
- A worker stops consuming its configured queues and may finish work already
|
||||
claimed by that process.
|
||||
- A stale process incarnation cannot overwrite a replacement incarnation's
|
||||
heartbeat.
|
||||
- A coordination outage removes API readiness and cancels worker consumers;
|
||||
the existing incarnation must heartbeat successfully before either resumes.
|
||||
|
||||
Use distinct pools when load warrants it:
|
||||
Singleton work uses PostgreSQL-backed leases with monotonically increasing
|
||||
fencing tokens. The generated scheduler runs Celery beat through
|
||||
`govoplan_core.commands.fenced_run`; loss of its lease terminates the child and
|
||||
returns a distinct failure code. A fenced business operation must validate the
|
||||
same lease token immediately before committing its effect.
|
||||
|
||||
- mail send and Sent-folder append;
|
||||
- notifications;
|
||||
- calendar and connector synchronization;
|
||||
- dataflow/workflow execution;
|
||||
- reporting and export;
|
||||
- platform events and default work.
|
||||
## Release Ordering
|
||||
|
||||
Before stopping a worker, mark it draining, stop new claims, and let or safely
|
||||
requeue active jobs. Worker registration and drain control remain part of the
|
||||
larger platform scale-out story.
|
||||
Use this order for every multi-replica rollout:
|
||||
|
||||
### Stateful Services
|
||||
1. Verify immutable image identities, module composition, external state
|
||||
reachability, backup evidence, and the generated plan.
|
||||
2. Drain application replicas when the migration compatibility declaration
|
||||
requires it.
|
||||
3. Run the release-specific migration Job exactly once. PostgreSQL advisory
|
||||
locking serializes all Core and module migration tasks across competing
|
||||
deployment jobs.
|
||||
4. Let runtime init containers run `wait_for_database`. They wait for exact
|
||||
configured Alembic heads and never mutate schema.
|
||||
5. Roll API, workers, scheduler, and WebUI using health-aware replacement.
|
||||
6. Verify runtime composition, expected replica counts, queue consumers,
|
||||
object-storage round trips, and recovery status in Ops.
|
||||
|
||||
- PostgreSQL needs backups, restore drills, connection limits, and an HA/failover
|
||||
design appropriate to the service level.
|
||||
- Redis needs persistence and, for high availability, a supported failover
|
||||
topology. Queue loss is not equivalent to a harmless cache loss.
|
||||
- Managed Garage from the Compose installer is single-node. For redundant
|
||||
storage, deploy a multi-node Garage cluster separately and select external
|
||||
`s3`, or use another compatible object store.
|
||||
- The scheduler stays at one replica until distributed leader election or a
|
||||
fenced lease is implemented.
|
||||
- Migrations and module lifecycle mutations remain one-at-a-time operations.
|
||||
Applying the complete generated manifest is fail-closed: runtime pods remain in
|
||||
their init phase until the migration Job reaches the expected heads. A second
|
||||
release may be submitted concurrently, but advisory locking prevents concurrent
|
||||
schema mutation and each release has a distinct migration Job name.
|
||||
|
||||
## Promotion Path
|
||||
## Storage Trust Boundary
|
||||
|
||||
1. Measure concurrent users, request latency, database query time, connection
|
||||
use, queue age, worker saturation, storage latency, and external throttling.
|
||||
2. Move files to managed Garage or external S3 before introducing independent
|
||||
runtime hosts.
|
||||
3. Keep PostgreSQL and Redis external to stateless runtime nodes, or deploy
|
||||
their reviewed HA operators.
|
||||
4. Publish immutable API/WebUI images and one versioned configuration/secret
|
||||
contract.
|
||||
5. Translate the generated role commands, environment allowlists, health
|
||||
checks, and singleton constraints into the chosen orchestrator.
|
||||
6. Put API and WebUI replicas behind health-aware services and a TLS ingress.
|
||||
7. Add queue-specific workers with fixed upper bounds.
|
||||
8. Prove replica loss, rolling replacement, job redelivery, session continuity,
|
||||
migration exclusion, backup restore, and storage-node loss before claiming
|
||||
high availability.
|
||||
Installer-managed Garage uses its exact generated endpoint. An arbitrary
|
||||
external S3 endpoint is accepted only when the deployment explicitly sets
|
||||
`FILE_STORAGE_S3_ENDPOINT_TRUSTED=true`; that endpoint must be a clean HTTPS
|
||||
origin without embedded credentials, query, fragment, or path. This is an
|
||||
operator trust declaration, not a user-controlled connector bypass. Operators
|
||||
remain responsible for DNS, certificate, network-egress, bucket-policy,
|
||||
versioning, and lifecycle controls.
|
||||
|
||||
## Remaining Platform Work
|
||||
## Capacity
|
||||
|
||||
The one-host installer does not yet provide:
|
||||
- Scale API replicas only within the PostgreSQL connection budget.
|
||||
- Scale workers by queue, with upper bounds based on external provider limits.
|
||||
- Keep one fenced scheduler rather than load-balancing schedulers.
|
||||
- Increase WebUI replicas for asset/proxy capacity.
|
||||
- Measure request latency, database query time and locks, active connections,
|
||||
queue age, retry rate, storage latency, and provider throttling before adding
|
||||
replicas.
|
||||
|
||||
- a Kubernetes, Nomad, or Swarm deployment export/profile;
|
||||
- distributed deployment locks and fenced scheduler leadership;
|
||||
- worker registration, composition-skew reporting, and drain controls;
|
||||
- managed PostgreSQL, Redis, or multi-node Garage high availability;
|
||||
Workers compete for Redis-backed work and are not placed behind a load balancer.
|
||||
SMTP, IMAP, directory, connector, workflow, dataflow, and reporting queues often
|
||||
hit external-system limits before host CPU is exhausted.
|
||||
|
||||
## What This Does Not Claim
|
||||
|
||||
The implemented contract provides stateless runtime placement, shared artifact
|
||||
access, node visibility, drain controls, migration serialization, and scheduler
|
||||
fencing. It does not by itself provide:
|
||||
|
||||
- a highly available PostgreSQL, Redis, or object-store deployment;
|
||||
- automatic PostgreSQL backup, point-in-time recovery, or restore verification;
|
||||
- autoscaling policy;
|
||||
- managed TLS certificate issuance and renewal;
|
||||
- measured multi-replica and failover integration evidence.
|
||||
- central logs, metrics, traces, or alert routing;
|
||||
- managed ingress certificates;
|
||||
- automatic reconciliation of every possible module side effect;
|
||||
- a service-level availability guarantee.
|
||||
|
||||
These are separate production capabilities. The local load balancer and
|
||||
replica model provide the contract they should implement, but they do not by
|
||||
themselves make a cluster highly available.
|
||||
Those are deployment and module-adoption requirements. Before claiming high
|
||||
availability, drill replica loss, rolling replacement, session continuity, job
|
||||
redelivery, scheduler failover, migration exclusion, object-store outage, and a
|
||||
coordinated database/object/key restore. Recovery rules and evidence are
|
||||
defined in [Recovery And Rollback Guarantees](RECOVERY_AND_ROLLBACK_GUARANTEES.md).
|
||||
|
||||
@@ -139,6 +139,17 @@ The canonical backlog item is
|
||||
|
||||
## Implementation slices
|
||||
|
||||
Implementation status as of the current source tree:
|
||||
|
||||
- Slice 6 has a working application-tier foundation: state profiles, shared
|
||||
object storage, runtime node registration/heartbeats/drain, fenced scheduler,
|
||||
migration serialization, exact-head startup waiting, Ops visibility, and a
|
||||
Kubernetes export. Production acceptance still requires topology-specific
|
||||
failover and restore drills.
|
||||
- The recovery foundation for slices 4 and 5 is implemented as a Core recovery
|
||||
ledger and deployment operation journal. Automatic database backup and broad
|
||||
adoption by module-owned external effects remain open work.
|
||||
|
||||
1. **Reproducible Core-only distribution.** Publish pinned multi-architecture
|
||||
images, signed distribution manifest, Core-only Compose profile, bootstrap
|
||||
preflight, generated secrets, readiness, and idempotent rerun/repair.
|
||||
@@ -155,9 +166,11 @@ The canonical backlog item is
|
||||
5. **Safe module update.** Add drain/maintenance coordination, backup gate,
|
||||
migration compatibility window, reconnectable progress, health verification,
|
||||
retry/recovery, and update notification.
|
||||
6. **Stateless replica profile.** Remove remaining local-runtime assumptions,
|
||||
expose role-specific commands/images, implement worker registration/drain,
|
||||
and prove multiple API and worker replicas against shared dependencies.
|
||||
6. **Stateless replica profile.** Continue module adoption and operational
|
||||
proof for the implemented role commands, shared-state validation, runtime
|
||||
registration/drain, fenced scheduler, and Kubernetes application-tier
|
||||
export. Prove multiple API and worker replicas against the target shared
|
||||
dependencies.
|
||||
7. **Configuration revision model.** Define provider export/import schemas,
|
||||
canonical serialization, secret references, validation/diff, immutable
|
||||
revision storage, audit, apply, and undo-as-new-revision.
|
||||
|
||||
@@ -50,7 +50,7 @@
|
||||
"claims": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 3,
|
||||
"maxItems": 8,
|
||||
"items": {
|
||||
"$ref": "#/$defs/claim"
|
||||
}
|
||||
@@ -80,6 +80,11 @@
|
||||
"enum": [
|
||||
"target_environment",
|
||||
"external_providers",
|
||||
"accessibility",
|
||||
"privacy",
|
||||
"security",
|
||||
"operations",
|
||||
"recovery",
|
||||
"production_approval"
|
||||
]
|
||||
},
|
||||
|
||||
@@ -51,12 +51,17 @@
|
||||
"allowed_scopes": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 3,
|
||||
"maxItems": 8,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"enum": [
|
||||
"target_environment",
|
||||
"external_providers",
|
||||
"accessibility",
|
||||
"privacy",
|
||||
"security",
|
||||
"operations",
|
||||
"recovery",
|
||||
"production_approval"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -113,6 +113,12 @@
|
||||
"description": "GovOPlaN Appointments module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/approvals",
|
||||
"color": "d93f0b",
|
||||
"description": "GovOPlaN Approvals module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/audit",
|
||||
"color": "0e8a16",
|
||||
@@ -143,6 +149,12 @@
|
||||
"description": "GovOPlaN Connectors module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/committee",
|
||||
"color": "5319e7",
|
||||
"description": "GovOPlaN Committee module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/core",
|
||||
"color": "0052cc",
|
||||
@@ -185,6 +197,12 @@
|
||||
"description": "GovOPlaN Distribution Lists module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/decisions",
|
||||
"color": "d93f0b",
|
||||
"description": "GovOPlaN formal Decisions module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/erp",
|
||||
"color": "fef2c0",
|
||||
@@ -221,6 +239,12 @@
|
||||
"description": "GovOPlaN Forms module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/forms-runtime",
|
||||
"color": "f9d0c4",
|
||||
"description": "GovOPlaN Forms Runtime module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/helpdesk",
|
||||
"color": "c2e0c6",
|
||||
@@ -257,6 +281,12 @@
|
||||
"description": "GovOPlaN mail module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/mandates",
|
||||
"color": "006b75",
|
||||
"description": "GovOPlaN Mandates, jurisdiction, responsibility, and authority behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/notifications",
|
||||
"color": "d876e3",
|
||||
@@ -281,6 +311,12 @@
|
||||
"description": "GovOPlaN Payments module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/parties",
|
||||
"color": "bfd4f2",
|
||||
"description": "GovOPlaN procedure Parties, representation, and delivery-authority behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/permits",
|
||||
"color": "fbca04",
|
||||
@@ -341,6 +377,12 @@
|
||||
"description": "GovOPlaN Scheduling module behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/services",
|
||||
"color": "1d76db",
|
||||
"description": "GovOPlaN versioned institutional Services behavior or integration.",
|
||||
"exclusive": false
|
||||
},
|
||||
{
|
||||
"name": "module/tasks",
|
||||
"color": "c5def5",
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# Governed Communication Product Package
|
||||
|
||||
This package composes Campaign, Mail, Postbox, Files, Templates, Policy, Audit,
|
||||
and Access into one governed-delivery capability. Modules retain their own
|
||||
tables and lifecycle. The package declares installation and configuration
|
||||
expectations; it does not import module internals.
|
||||
|
||||
## Reference journey
|
||||
|
||||
1. Select governed recipient and attachment data.
|
||||
2. Build and review an immutable Campaign version.
|
||||
3. Apply attachment, access, and delivery policy.
|
||||
4. Deliver through Mail and/or function-bound Postbox targets.
|
||||
5. Reconcile outcomes, acknowledgements, correction, and reports against Audit
|
||||
evidence.
|
||||
|
||||
## Reference-readiness gates
|
||||
|
||||
The artifact remains a `product` package. Promotion to `reference` requires:
|
||||
|
||||
- target-environment delivery and reconciliation tests, including an
|
||||
outcome-unknown transport result;
|
||||
- backup/restore and interrupted-dispatch recovery evidence;
|
||||
- security evidence for recipient, attachment, and postbox isolation;
|
||||
- operator runbooks for queue health, retry, reconciliation, and retirement;
|
||||
- keyboard, focus, screen-reader, and responsive workflow evidence;
|
||||
- privacy evidence for minimization, purpose, retention, redaction, and report
|
||||
access; and
|
||||
- version-pinned user and administrator documentation.
|
||||
|
||||
Optional Notifications, Portal, Reporting, and Workflow Engine integrations do
|
||||
not change the package boundary when absent.
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"package_id": "product.governed-communication",
|
||||
"name": "Governed Communication",
|
||||
"version": "0.1.0",
|
||||
"package_class": "product",
|
||||
"description": "Compose evidence-backed campaigns, mail and function-bound postboxes without merging their domain ownership.",
|
||||
"publisher": "GovOPlaN",
|
||||
"category": "communication",
|
||||
"license": "AGPL-3.0-or-later",
|
||||
"required_modules": [
|
||||
{"module_id": "access"},
|
||||
{"module_id": "audit"},
|
||||
{"module_id": "campaign"},
|
||||
{"module_id": "files"},
|
||||
{"module_id": "mail"},
|
||||
{"module_id": "policy"},
|
||||
{"module_id": "postbox"},
|
||||
{"module_id": "templates"}
|
||||
],
|
||||
"optional_modules": [
|
||||
{"module_id": "notifications"},
|
||||
{"module_id": "portal"},
|
||||
{"module_id": "reporting"},
|
||||
{"module_id": "workflow_engine"}
|
||||
],
|
||||
"evidence": [
|
||||
{
|
||||
"kind": "documentation",
|
||||
"reference": "packages/product/governed-communication/README.md",
|
||||
"summary": "Defines the package boundary, journey, and reference-readiness gates."
|
||||
}
|
||||
],
|
||||
"tags": ["campaign", "mail", "postbox", "public-sector"]
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
# Governed Data and Assurance Product Package
|
||||
|
||||
This package composes Datasources, Dataflow, Reporting, Search, Risk
|
||||
Compliance, Policy, Audit, and Access. Connectors may acquire external data,
|
||||
but Datasources owns the governed catalogue and immutable snapshots; Dataflow
|
||||
owns transformation definitions and runs; Reporting owns measures and
|
||||
presentation; Risk Compliance owns controls, findings, and effectiveness
|
||||
review.
|
||||
|
||||
## Reference journey
|
||||
|
||||
1. Register a typed datasource with source authority, purpose, classification,
|
||||
owner, freshness, and correction policy.
|
||||
2. Acquire or upload an immutable source state.
|
||||
3. execute a versioned flow and retain intermediate materializations and
|
||||
provenance;
|
||||
4. publish a report or decision input against exact source and flow revisions;
|
||||
5. link obligation, governed object, risk, control, evidence, finding,
|
||||
corrective measure, and effectiveness review; and
|
||||
6. search current authorized objects while preserving ownership and access
|
||||
rechecks.
|
||||
|
||||
## Reference-readiness gates
|
||||
|
||||
The artifact remains a `product` package. Promotion to `reference` requires:
|
||||
|
||||
- a target-tested monthly-data and sanctions-screening fixture with expected
|
||||
outputs and complete provenance;
|
||||
- database/object-storage backup and restore plus interrupted-run recovery;
|
||||
- security evidence for datasource credentials, staged data, intermediate
|
||||
states, search documents, and assurance references;
|
||||
- operator runbooks for stale sources, failed runs, index rebuilds, and graph
|
||||
reconciliation;
|
||||
- accessibility evidence for the catalogue, graph editors, result inspection,
|
||||
reports, and assurance graph;
|
||||
- privacy evidence for minimization, purpose, retention, field visibility, and
|
||||
aggregate disclosure; and
|
||||
- version-pinned user and administrator documentation.
|
||||
|
||||
Optional Connectors, Files, Notifications, and Workflow Engine integrations
|
||||
must remain capability-based and absence-safe.
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"package_id": "product.governed-data-assurance",
|
||||
"name": "Governed Data and Assurance",
|
||||
"version": "0.1.0",
|
||||
"package_class": "product",
|
||||
"description": "Compose governed sources, transformations, reports, search, controls and assurance evidence through stable contracts.",
|
||||
"publisher": "GovOPlaN",
|
||||
"category": "data-governance",
|
||||
"license": "AGPL-3.0-or-later",
|
||||
"required_modules": [
|
||||
{"module_id": "access"},
|
||||
{"module_id": "audit"},
|
||||
{"module_id": "dataflow"},
|
||||
{"module_id": "datasources"},
|
||||
{"module_id": "policy"},
|
||||
{"module_id": "reporting"},
|
||||
{"module_id": "risk_compliance"},
|
||||
{"module_id": "search"}
|
||||
],
|
||||
"optional_modules": [
|
||||
{"module_id": "connectors"},
|
||||
{"module_id": "files"},
|
||||
{"module_id": "notifications"},
|
||||
{"module_id": "workflow_engine"}
|
||||
],
|
||||
"evidence": [
|
||||
{
|
||||
"kind": "documentation",
|
||||
"reference": "packages/product/governed-data-assurance/README.md",
|
||||
"summary": "Defines the package boundary, provenance chain, and reference-readiness gates."
|
||||
}
|
||||
],
|
||||
"tags": ["datasources", "dataflow", "reporting", "assurance"]
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
# Governed Service To Decision
|
||||
|
||||
This product package composes independently owned institutional semantics into
|
||||
one reconstructable administrative journey:
|
||||
|
||||
```text
|
||||
Service discovery -> Case intake -> Party and representation -> Mandate
|
||||
resolution -> approval/deliberation -> formal Decision -> observed delivery
|
||||
effect -> record and review references
|
||||
```
|
||||
|
||||
An installed Forms and Forms Runtime pair adds an alternative governed entry
|
||||
path before case/workflow handoff:
|
||||
|
||||
```text
|
||||
Service discovery -> exact Form revision -> validated draft/submission
|
||||
-> receipt and handoff evidence -> Case or Workflow owner
|
||||
```
|
||||
|
||||
Services, Cases, Parties, Mandates, Committee, and Decisions retain immutable
|
||||
provider-owned revisions for the parts they own. Portal, Cases, and Committee
|
||||
consume capabilities for cross-module semantics only. The package does not
|
||||
grant cross-module table access and can omit optional presentation, work,
|
||||
deliberation, delivery, or records modules while retaining explicit references
|
||||
to externally performed steps.
|
||||
|
||||
## Security And Recovery
|
||||
|
||||
Every provider is tenant-bound. Missing or conflicting authority fails closed.
|
||||
Protected Decision content has a separate permission. Writes are replay-safe
|
||||
and OCC-guarded. Database restore is the semantic-state recovery unit; file and
|
||||
communication effects remain governed by their owning providers and are linked
|
||||
through requested/observed effect, evidence, and audit references.
|
||||
|
||||
The executable fixture in
|
||||
`tests/test_institutional_governance_journey.py` proves SQL-backed Service,
|
||||
Case, Party, Mandate, Committee meeting/agendum/vote/minute, and Decision state.
|
||||
`tests/test_institutional_service_journey.py` separately proves exact Portal
|
||||
Form launch, persisted submission provenance, and idempotent replay.
|
||||
Target-environment accessibility, security, operator, privacy,
|
||||
delivery-provider, and recovery evidence are still required before this product
|
||||
package may claim `reference_ready` maturity.
|
||||
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"package_id": "product.service-to-decision",
|
||||
"name": "Governed Service To Decision",
|
||||
"version": "0.1.0",
|
||||
"package_class": "product",
|
||||
"description": "Carry one exact institutional context from service discovery and case intake through parties, authority, formal outcome, communication evidence, and review.",
|
||||
"publisher": "GovOPlaN",
|
||||
"category": "institutional-governance",
|
||||
"license": "AGPL-3.0-or-later",
|
||||
"required_modules": [
|
||||
{"module_id": "audit"},
|
||||
{"module_id": "cases"},
|
||||
{"module_id": "decisions"},
|
||||
{"module_id": "mandates"},
|
||||
{"module_id": "parties"},
|
||||
{"module_id": "policy"},
|
||||
{"module_id": "portal"},
|
||||
{"module_id": "services"}
|
||||
],
|
||||
"required_capabilities": [
|
||||
"cases.party_context",
|
||||
"cases.service_intake",
|
||||
"decisions.registry",
|
||||
"mandates.resolver",
|
||||
"parties.resolver",
|
||||
"portal.service_directory",
|
||||
"services.availability",
|
||||
"services.definitions"
|
||||
],
|
||||
"optional_modules": [
|
||||
{"module_id": "approvals"},
|
||||
{"module_id": "committee"},
|
||||
{"module_id": "files"},
|
||||
{"module_id": "forms"},
|
||||
{"module_id": "forms_runtime"},
|
||||
{"module_id": "postbox"},
|
||||
{"module_id": "records"},
|
||||
{"module_id": "tasks"},
|
||||
{"module_id": "workflow_engine"}
|
||||
],
|
||||
"evidence": [
|
||||
{
|
||||
"kind": "documentation",
|
||||
"reference": "packages/product/service-to-decision/README.md",
|
||||
"summary": "Defines the package boundary, authority path, recovery contract, and known operational limits."
|
||||
},
|
||||
{
|
||||
"kind": "target_test",
|
||||
"reference": "tests/test_institutional_governance_journey.py",
|
||||
"summary": "Executes the SQL-backed provider composition from service discovery through persisted formal Decision reconstruction."
|
||||
},
|
||||
{
|
||||
"kind": "target_test",
|
||||
"reference": "tests/test_institutional_service_journey.py",
|
||||
"summary": "Executes Portal delegation from an exact Service revision through an exact immutable Form revision to a replay-safe persisted submission."
|
||||
}
|
||||
],
|
||||
"tags": ["service", "case", "mandate", "party", "decision", "public-sector"]
|
||||
}
|
||||
@@ -24,6 +24,7 @@
|
||||
{"name": "govoplan-dashboard", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dashboard.git", "path": "govoplan-dashboard"},
|
||||
{"name": "govoplan-dataflow", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dataflow.git", "path": "govoplan-dataflow"},
|
||||
{"name": "govoplan-datasources", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-datasources.git", "path": "govoplan-datasources"},
|
||||
{"name": "govoplan-decisions", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-decisions.git", "path": "govoplan-decisions"},
|
||||
{"name": "govoplan-dms", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dms.git", "path": "govoplan-dms"},
|
||||
{"name": "govoplan-dist-lists", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-dist-lists.git", "path": "govoplan-dist-lists"},
|
||||
{"name": "govoplan-docs", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-docs.git", "path": "govoplan-docs"},
|
||||
@@ -44,10 +45,12 @@
|
||||
{"name": "govoplan-learning", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-learning.git", "path": "govoplan-learning"},
|
||||
{"name": "govoplan-ledger", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-ledger.git", "path": "govoplan-ledger"},
|
||||
{"name": "govoplan-mail", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-mail.git", "path": "govoplan-mail"},
|
||||
{"name": "govoplan-mandates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-mandates.git", "path": "govoplan-mandates"},
|
||||
{"name": "govoplan-notifications", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-notifications.git", "path": "govoplan-notifications"},
|
||||
{"name": "govoplan-ops", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-ops.git", "path": "govoplan-ops"},
|
||||
{"name": "govoplan-organizations", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-organizations.git", "path": "govoplan-organizations"},
|
||||
{"name": "govoplan-payments", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-payments.git", "path": "govoplan-payments"},
|
||||
{"name": "govoplan-parties", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-parties.git", "path": "govoplan-parties"},
|
||||
{"name": "govoplan-permits", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-permits.git", "path": "govoplan-permits"},
|
||||
{"name": "govoplan-policy", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-policy.git", "path": "govoplan-policy"},
|
||||
{"name": "govoplan-poll", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-poll.git", "path": "govoplan-poll"},
|
||||
@@ -62,6 +65,7 @@
|
||||
{"name": "govoplan-risk-compliance", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-risk-compliance.git", "path": "govoplan-risk-compliance"},
|
||||
{"name": "govoplan-scheduling", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-scheduling.git", "path": "govoplan-scheduling"},
|
||||
{"name": "govoplan-search", "category": "module", "subtype": "platform", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-search.git", "path": "govoplan-search"},
|
||||
{"name": "govoplan-services", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-services.git", "path": "govoplan-services"},
|
||||
{"name": "govoplan-soap", "category": "connector", "subtype": "protocol", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-soap.git", "path": "govoplan-soap"},
|
||||
{"name": "govoplan-tasks", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-tasks.git", "path": "govoplan-tasks"},
|
||||
{"name": "govoplan-templates", "category": "module", "subtype": "domain", "remote": "git@git.add-ideas.de:GovOPlaN/govoplan-templates.git", "path": "govoplan-templates"},
|
||||
|
||||
@@ -16,9 +16,18 @@
|
||||
-e ../govoplan-addresses
|
||||
-e ../govoplan-dist-lists
|
||||
-e ../govoplan-files
|
||||
-e ../govoplan-forms
|
||||
-e ../govoplan-forms-runtime
|
||||
-e ../govoplan-mail
|
||||
-e ../govoplan-campaign
|
||||
-e ../govoplan-calendar
|
||||
-e ../govoplan-committee
|
||||
-e ../govoplan-cases
|
||||
-e ../govoplan-portal
|
||||
-e ../govoplan-services
|
||||
-e ../govoplan-parties
|
||||
-e ../govoplan-mandates
|
||||
-e ../govoplan-decisions
|
||||
-e ../govoplan-connectors
|
||||
-e ../govoplan-datasources
|
||||
-e ../govoplan-dataflow
|
||||
|
||||
@@ -1007,6 +1007,44 @@ class CapabilityFitEvidenceTests(unittest.TestCase):
|
||||
report["proof_scope"]["target_environment"]["observed_subject_id"],
|
||||
)
|
||||
|
||||
def test_reference_readiness_scopes_require_release_bound_signed_proof(
|
||||
self,
|
||||
) -> None:
|
||||
installed = matching_installed_evidence(self.assessment)
|
||||
catalog, keyring = signed_catalog(
|
||||
self.assessment,
|
||||
release_artifacts=catalog_artifacts_for(installed),
|
||||
)
|
||||
readiness_scopes = (
|
||||
"target_environment",
|
||||
"accessibility",
|
||||
"privacy",
|
||||
"security",
|
||||
"operations",
|
||||
"recovery",
|
||||
)
|
||||
proof, authority = signed_boundary_evidence(
|
||||
assessment=self.assessment,
|
||||
installed=installed,
|
||||
claims=[boundary_claim(scope, "passed") for scope in readiness_scopes],
|
||||
allowed_scopes=list(readiness_scopes),
|
||||
)
|
||||
|
||||
report = self.review(
|
||||
catalog=catalog,
|
||||
keyring=keyring,
|
||||
installed_evidence=installed,
|
||||
boundary_evidence=proof,
|
||||
authority_keyring=authority,
|
||||
)
|
||||
|
||||
self.assertEqual("current", report["status"])
|
||||
for scope in readiness_scopes:
|
||||
with self.subTest(scope=scope):
|
||||
self.assertTrue(report["proof_scope"][scope]["checked"])
|
||||
self.assertTrue(report["proof_scope"][scope]["valid"])
|
||||
self.assertTrue(report["proof_scope"]["reference_readiness"]["valid"])
|
||||
|
||||
def test_external_provider_claim_requires_explicit_matching_subject(self) -> None:
|
||||
catalog, keyring = signed_catalog(self.assessment)
|
||||
installed = matching_installed_evidence(self.assessment)
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
from govoplan_core.core.configuration_packages import (
|
||||
ConfigurationPackageManifest,
|
||||
ConfigurationPreflightContext,
|
||||
configuration_package_claim_issues,
|
||||
dry_run_configuration_package,
|
||||
)
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
PACKAGE_ROOT = ROOT / "packages"
|
||||
|
||||
|
||||
class ConfigurationPackageArtifactTests(unittest.TestCase):
|
||||
def test_product_package_manifests_are_portable_and_evidence_backed(self) -> None:
|
||||
paths = tuple(sorted(PACKAGE_ROOT.glob("*/*/package.json")))
|
||||
self.assertGreaterEqual(len(paths), 2)
|
||||
package_ids: set[str] = set()
|
||||
|
||||
for path in paths:
|
||||
manifest = ConfigurationPackageManifest.from_mapping(
|
||||
json.loads(path.read_text(encoding="utf-8"))
|
||||
)
|
||||
self.assertNotIn(manifest.package_id, package_ids)
|
||||
package_ids.add(manifest.package_id)
|
||||
self.assertEqual((), configuration_package_claim_issues(manifest))
|
||||
for evidence in manifest.evidence:
|
||||
evidence_path = ROOT / evidence.reference
|
||||
self.assertTrue(
|
||||
evidence_path.is_file(),
|
||||
f"Missing evidence {evidence.reference} for {manifest.package_id}",
|
||||
)
|
||||
if evidence.checksum is not None:
|
||||
self.assertEqual(
|
||||
evidence.checksum,
|
||||
"sha256:" + hashlib.sha256(evidence_path.read_bytes()).hexdigest(),
|
||||
f"Stale evidence checksum for {manifest.package_id}: {evidence.reference}",
|
||||
)
|
||||
for requirement in (
|
||||
*manifest.required_modules,
|
||||
*manifest.optional_modules,
|
||||
):
|
||||
repository = ROOT.parent / (
|
||||
"govoplan-" + requirement.module_id.replace("_", "-")
|
||||
)
|
||||
self.assertTrue(
|
||||
repository.is_dir(),
|
||||
f"Missing repository for {requirement.module_id}",
|
||||
)
|
||||
|
||||
result = dry_run_configuration_package(
|
||||
manifest,
|
||||
(),
|
||||
ConfigurationPreflightContext(
|
||||
installed_modules={
|
||||
item.module_id: item.version or "workspace"
|
||||
for item in manifest.required_modules
|
||||
},
|
||||
capabilities=frozenset(manifest.required_capabilities),
|
||||
),
|
||||
)
|
||||
self.assertFalse(
|
||||
any(item.severity == "blocker" for item in result.diagnostics),
|
||||
tuple(item.to_dict() for item in result.diagnostics),
|
||||
)
|
||||
|
||||
self.assertIn("product.governed-communication", package_ids)
|
||||
self.assertIn("product.governed-data-assurance", package_ids)
|
||||
self.assertIn("product.service-to-decision", package_ids)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -31,6 +31,7 @@ from govoplan_deploy.bundle import ( # noqa: E402
|
||||
)
|
||||
from govoplan_deploy.cli import _receipt_uses_direct_web_port, main # noqa: E402
|
||||
import govoplan_deploy.cli as deployment_cli # noqa: E402
|
||||
from govoplan_deploy.kubernetes import render_kubernetes # noqa: E402
|
||||
from govoplan_deploy.model import ( # noqa: E402
|
||||
SpecError,
|
||||
default_spec,
|
||||
@@ -38,6 +39,12 @@ from govoplan_deploy.model import ( # noqa: E402
|
||||
)
|
||||
from govoplan_deploy.planning import _endpoint_check, build_plan # noqa: E402
|
||||
import govoplan_deploy.planning as deployment_planning # noqa: E402
|
||||
from govoplan_deploy.recovery import ( # noqa: E402
|
||||
DeploymentOperationJournal,
|
||||
load_operation,
|
||||
recover_operation,
|
||||
write_applied_state,
|
||||
)
|
||||
|
||||
|
||||
def run_cli(arguments: list[str]) -> tuple[int, str, str]:
|
||||
@@ -49,6 +56,242 @@ def run_cli(arguments: list[str]) -> tuple[int, str, str]:
|
||||
|
||||
|
||||
class DeploymentInstallerTests(unittest.TestCase):
|
||||
def test_pre_migration_failure_restores_checksum_verified_applied_bundle(
|
||||
self,
|
||||
) -> None:
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix="govoplan-deploy-recovery-"
|
||||
) as directory:
|
||||
paths = bundle_paths(Path(directory))
|
||||
paths.root.chmod(0o700)
|
||||
spec = default_spec()
|
||||
previous_environment = initial_secrets(spec)
|
||||
atomic_write(paths.spec, canonical_json(spec.to_dict()), mode=0o600)
|
||||
write_env(paths.env, previous_environment)
|
||||
atomic_write(
|
||||
paths.compose, canonical_json(render_compose(spec)), mode=0o600
|
||||
)
|
||||
atomic_write(
|
||||
paths.receipt, canonical_json({"spec_sha256": "old"}), mode=0o600
|
||||
)
|
||||
write_applied_state(paths)
|
||||
|
||||
atomic_write(paths.spec, canonical_json({"changed": True}), mode=0o600)
|
||||
journal = DeploymentOperationJournal.begin(
|
||||
paths,
|
||||
plan={"installation_id": spec.installation_id},
|
||||
)
|
||||
journal.failed(RuntimeError("pull failed"))
|
||||
|
||||
result = recover_operation(paths, operation_id=journal.operation_id)
|
||||
|
||||
self.assertEqual("configuration_restored", result.action)
|
||||
self.assertEqual(spec.to_dict(), json.loads(paths.spec.read_text()))
|
||||
self.assertEqual(previous_environment, read_env(paths.env))
|
||||
|
||||
def test_post_migration_failure_requires_forward_recovery(self) -> None:
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix="govoplan-deploy-recovery-"
|
||||
) as directory:
|
||||
paths = bundle_paths(Path(directory))
|
||||
paths.root.chmod(0o700)
|
||||
spec = default_spec()
|
||||
atomic_write(paths.spec, canonical_json(spec.to_dict()), mode=0o600)
|
||||
write_env(paths.env, initial_secrets(spec))
|
||||
atomic_write(
|
||||
paths.compose, canonical_json(render_compose(spec)), mode=0o600
|
||||
)
|
||||
write_applied_state(paths)
|
||||
journal = DeploymentOperationJournal.begin(
|
||||
paths,
|
||||
plan={"installation_id": spec.installation_id},
|
||||
)
|
||||
journal.migration_started()
|
||||
journal.failed(RuntimeError("health failed"))
|
||||
|
||||
result = recover_operation(paths, operation_id=journal.operation_id)
|
||||
|
||||
self.assertEqual("forward_recovery", result.action)
|
||||
self.assertIn("not restored", result.detail)
|
||||
|
||||
def test_recovery_refuses_tampered_migration_boundary(self) -> None:
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix="govoplan-deploy-recovery-"
|
||||
) as directory:
|
||||
paths = bundle_paths(Path(directory))
|
||||
paths.root.chmod(0o700)
|
||||
spec = default_spec()
|
||||
journal = DeploymentOperationJournal.begin(
|
||||
paths,
|
||||
plan={"installation_id": spec.installation_id},
|
||||
)
|
||||
journal.migration_started()
|
||||
payload = json.loads(journal.path.joinpath("operation.json").read_text())
|
||||
payload["migration_started"] = False
|
||||
atomic_write(
|
||||
journal.path / "operation.json",
|
||||
canonical_json(payload),
|
||||
mode=0o600,
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(ValueError, "migration boundary"):
|
||||
load_operation(paths, operation_id=journal.operation_id)
|
||||
|
||||
def test_recovery_verifies_complete_snapshot_before_mutating_bundle(self) -> None:
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix="govoplan-deploy-recovery-"
|
||||
) as directory:
|
||||
paths = bundle_paths(Path(directory))
|
||||
paths.root.chmod(0o700)
|
||||
spec = default_spec()
|
||||
atomic_write(paths.spec, canonical_json(spec.to_dict()), mode=0o600)
|
||||
write_env(paths.env, initial_secrets(spec))
|
||||
atomic_write(
|
||||
paths.compose, canonical_json(render_compose(spec)), mode=0o600
|
||||
)
|
||||
write_applied_state(paths)
|
||||
journal = DeploymentOperationJournal.begin(
|
||||
paths,
|
||||
plan={"installation_id": spec.installation_id},
|
||||
)
|
||||
changed_spec = canonical_json({"changed": True})
|
||||
atomic_write(paths.spec, changed_spec, mode=0o600)
|
||||
atomic_write(
|
||||
journal.path / "before" / "compose.json", b"tampered", mode=0o600
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(ValueError, "checksum failed"):
|
||||
recover_operation(paths, operation_id=journal.operation_id)
|
||||
|
||||
self.assertEqual(changed_spec, paths.spec.read_bytes())
|
||||
|
||||
def test_recovery_refuses_tampered_desired_plan(self) -> None:
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix="govoplan-deploy-recovery-"
|
||||
) as directory:
|
||||
paths = bundle_paths(Path(directory))
|
||||
paths.root.chmod(0o700)
|
||||
spec = default_spec()
|
||||
journal = DeploymentOperationJournal.begin(
|
||||
paths,
|
||||
plan={"installation_id": spec.installation_id},
|
||||
)
|
||||
payload = json.loads(journal.path.joinpath("operation.json").read_text())
|
||||
payload["desired"]["installation_id"] = "other-installation"
|
||||
atomic_write(
|
||||
journal.path / "operation.json",
|
||||
canonical_json(payload),
|
||||
mode=0o600,
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(ValueError, "plan"):
|
||||
load_operation(paths, operation_id=journal.operation_id)
|
||||
|
||||
def test_kubernetes_export_is_stateless_and_uses_exact_secret_contract(
|
||||
self,
|
||||
) -> None:
|
||||
spec = default_spec(
|
||||
installation_id="govoplan-cluster",
|
||||
postgres_mode="external",
|
||||
redis_mode="external",
|
||||
storage_mode="s3",
|
||||
api_replicas=3,
|
||||
web_replicas=2,
|
||||
worker_replicas=4,
|
||||
api_image="registry.example.test/govoplan-api@sha256:" + "a" * 64,
|
||||
web_image="registry.example.test/govoplan-web@sha256:" + "b" * 64,
|
||||
)
|
||||
environment = initial_secrets(
|
||||
spec,
|
||||
supplied={
|
||||
"DATABASE_URL": "postgresql+psycopg://user:db-secret@postgres.example.test/govoplan",
|
||||
"GOVOPLAN_DATABASE_URL_PGTOOLS": "postgresql://user:db-secret@postgres.example.test/govoplan",
|
||||
"REDIS_URL": "rediss://:redis-secret@redis.example.test/0",
|
||||
"FILE_STORAGE_S3_ENDPOINT_URL": "https://s3.example.test",
|
||||
"FILE_STORAGE_S3_REGION": "eu-test-1",
|
||||
"FILE_STORAGE_S3_ACCESS_KEY_ID": "object-key",
|
||||
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": "object-secret",
|
||||
"FILE_STORAGE_S3_BUCKET": "govoplan",
|
||||
},
|
||||
)
|
||||
|
||||
manifest = render_kubernetes(spec, environment)
|
||||
rendered = json.dumps(manifest, sort_keys=True)
|
||||
kinds = [item["kind"] for item in manifest["items"]]
|
||||
deployments = {
|
||||
item["metadata"]["name"]: item
|
||||
for item in manifest["items"]
|
||||
if item["kind"] == "Deployment"
|
||||
}
|
||||
|
||||
self.assertNotIn("db-secret", rendered)
|
||||
self.assertNotIn("redis-secret", rendered)
|
||||
self.assertNotIn("object-secret", rendered)
|
||||
self.assertNotIn("PersistentVolumeClaim", kinds)
|
||||
self.assertNotIn("StatefulSet", kinds)
|
||||
self.assertEqual(3, deployments["govoplan-cluster-api"]["spec"]["replicas"])
|
||||
self.assertEqual(4, deployments["govoplan-cluster-worker"]["spec"]["replicas"])
|
||||
scheduler_command = deployments["govoplan-cluster-scheduler"]["spec"][
|
||||
"template"
|
||||
]["spec"]["containers"][0]["command"]
|
||||
self.assertIn("govoplan_core.commands.fenced_run", scheduler_command)
|
||||
api_init_command = deployments["govoplan-cluster-api"]["spec"]["template"][
|
||||
"spec"
|
||||
]["initContainers"][0]["command"]
|
||||
self.assertIn("govoplan_core.commands.wait_for_database", api_init_command)
|
||||
migration = next(item for item in manifest["items"] if item["kind"] == "Job")
|
||||
self.assertRegex(
|
||||
migration["metadata"]["name"],
|
||||
r"^govoplan-cluster-migrate-[0-9a-f]{8}$",
|
||||
)
|
||||
self.assertEqual(
|
||||
"forward-recovery",
|
||||
migration["metadata"]["annotations"]["govoplan.add-ideas.de/recovery-mode"],
|
||||
)
|
||||
config = next(item for item in manifest["items"] if item["kind"] == "ConfigMap")
|
||||
self.assertEqual("shared", config["data"]["GOVOPLAN_STATE_PROFILE"])
|
||||
self.assertEqual("3", config["data"]["GOVOPLAN_EXPECTED_API_REPLICAS"])
|
||||
self.assertEqual(
|
||||
[{"name": "Host", "value": "127.0.0.1"}],
|
||||
deployments["govoplan-cluster-api"]["spec"]["template"]["spec"][
|
||||
"containers"
|
||||
][0]["readinessProbe"]["httpGet"]["httpHeaders"],
|
||||
)
|
||||
|
||||
def test_kubernetes_export_rejects_mutable_release_images(self) -> None:
|
||||
spec = default_spec(
|
||||
installation_id="govoplan-cluster",
|
||||
postgres_mode="external",
|
||||
redis_mode="external",
|
||||
storage_mode="s3",
|
||||
api_image="registry.example.test/govoplan-api:latest",
|
||||
web_image="registry.example.test/govoplan-web:latest",
|
||||
)
|
||||
environment = initial_secrets(
|
||||
spec,
|
||||
supplied={
|
||||
"DATABASE_URL": "postgresql+psycopg://user:secret@postgres.example.test/govoplan",
|
||||
"GOVOPLAN_DATABASE_URL_PGTOOLS": "postgresql://user:secret@postgres.example.test/govoplan",
|
||||
"REDIS_URL": "rediss://:secret@redis.example.test/0",
|
||||
"FILE_STORAGE_S3_ENDPOINT_URL": "https://s3.example.test",
|
||||
"FILE_STORAGE_S3_REGION": "eu-test-1",
|
||||
"FILE_STORAGE_S3_ACCESS_KEY_ID": "object-key",
|
||||
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": "object-secret",
|
||||
"FILE_STORAGE_S3_BUCKET": "govoplan",
|
||||
},
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(ValueError, "digest-pinned"):
|
||||
render_kubernetes(spec, environment)
|
||||
|
||||
def test_kubernetes_export_rejects_installer_managed_state_services(self) -> None:
|
||||
spec = default_spec(
|
||||
installation_id="govoplan-cluster",
|
||||
storage_mode="garage",
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "external shared PostgreSQL"):
|
||||
render_kubernetes(spec, initial_secrets(spec))
|
||||
|
||||
def test_default_bundle_has_base_modules_and_managed_dependencies(self) -> None:
|
||||
spec = default_spec()
|
||||
compose = render_compose(spec)
|
||||
@@ -80,6 +323,7 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
self.assertNotIn("scheduler", compose["services"])
|
||||
self.assertEqual("false", values["CELERY_ENABLED"])
|
||||
self.assertEqual("true", values["GOVOPLAN_ALLOW_PROCESS_LOCAL_LOGIN_THROTTLE"])
|
||||
self.assertEqual("local", values["GOVOPLAN_STATE_PROFILE"])
|
||||
|
||||
def test_self_hosted_rejects_insecure_or_test_only_choices(self) -> None:
|
||||
with self.assertRaisesRegex(SpecError, "must use HTTPS"):
|
||||
@@ -154,7 +398,7 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual("rediss://:secret@redis.example.test/0", values["REDIS_URL"])
|
||||
|
||||
def test_s3_requires_complete_private_configuration_and_https_in_production(
|
||||
def test_s3_requires_complete_private_configuration_and_https_endpoint(
|
||||
self,
|
||||
) -> None:
|
||||
evaluation = default_spec(storage_mode="s3")
|
||||
@@ -168,17 +412,21 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": "secret",
|
||||
"FILE_STORAGE_S3_BUCKET": "govoplan",
|
||||
}
|
||||
self.assertEqual(
|
||||
"s3",
|
||||
initial_secrets(evaluation, supplied=supplied)["FILE_STORAGE_BACKEND"],
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "must use HTTPS"):
|
||||
initial_secrets(evaluation, supplied=supplied)
|
||||
supplied["FILE_STORAGE_S3_ENDPOINT_URL"] = "https://s3.example.test"
|
||||
values = initial_secrets(evaluation, supplied=supplied)
|
||||
self.assertEqual("s3", values["FILE_STORAGE_BACKEND"])
|
||||
self.assertEqual("true", values["FILE_STORAGE_S3_ENDPOINT_TRUSTED"])
|
||||
production = default_spec(
|
||||
profile="self-hosted",
|
||||
public_url="https://govoplan.example.test",
|
||||
storage_mode="s3",
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "must use HTTPS"):
|
||||
initial_secrets(production, supplied=supplied)
|
||||
self.assertEqual(
|
||||
"s3",
|
||||
initial_secrets(production, supplied=supplied)["FILE_STORAGE_BACKEND"],
|
||||
)
|
||||
|
||||
def test_managed_garage_bootstraps_private_s3_storage(self) -> None:
|
||||
spec = default_spec(storage_mode="garage")
|
||||
@@ -187,6 +435,7 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
|
||||
self.assertEqual("s3", values["FILE_STORAGE_BACKEND"])
|
||||
self.assertEqual("true", values["FILE_STORAGE_S3_DEPLOYMENT_MANAGED"])
|
||||
self.assertEqual("false", values["FILE_STORAGE_S3_ENDPOINT_TRUSTED"])
|
||||
self.assertEqual(
|
||||
"http://garage:3900",
|
||||
values["FILE_STORAGE_S3_ENDPOINT_URL"],
|
||||
@@ -244,6 +493,16 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
)
|
||||
self.assertIn("server-template web- 2 web:8080", config)
|
||||
self.assertIn("server-template api- 3 api:8000", config)
|
||||
self.assertIn("option httpchk GET /health/ready", config)
|
||||
self.assertIn("http-check send hdr Host 127.0.0.1", config)
|
||||
self.assertIn(
|
||||
"headers={'Host': '127.0.0.1'}",
|
||||
compose["services"]["api"]["healthcheck"]["test"][-1],
|
||||
)
|
||||
self.assertEqual(
|
||||
"host-shared",
|
||||
initial_secrets(spec)["GOVOPLAN_STATE_PROFILE"],
|
||||
)
|
||||
|
||||
def test_multi_replica_runtime_requires_shared_redis(self) -> None:
|
||||
with self.assertRaisesRegex(SpecError, "multiple API replicas"):
|
||||
@@ -725,11 +984,17 @@ class DeploymentInstallerTests(unittest.TestCase):
|
||||
for index, command in enumerate(commands)
|
||||
if "--remove-orphans" in command
|
||||
)
|
||||
stop_index = next(
|
||||
index
|
||||
for index, command in enumerate(commands)
|
||||
if "stop" in command and "--timeout" in command
|
||||
)
|
||||
self.assertLess(stop_index, migrate_index)
|
||||
self.assertLess(migrate_index, runtime_index)
|
||||
self.assertIn("postgres", commands[0])
|
||||
self.assertIn("redis", commands[0])
|
||||
wait_for_health.assert_called_once_with(
|
||||
"http://127.0.0.1:8080/health",
|
||||
"http://127.0.0.1:8080/health/ready",
|
||||
timeout_seconds=120.0,
|
||||
)
|
||||
receipt = json.loads((root / "receipt.json").read_text(encoding="utf-8"))
|
||||
|
||||
@@ -0,0 +1,591 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, replace
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import unittest
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.institutional import (
|
||||
ActorRepresentationReference,
|
||||
DecisionEffectReference,
|
||||
EvidenceReference,
|
||||
InformationGovernanceReference,
|
||||
InstitutionalReference,
|
||||
LegalBasisReference,
|
||||
MandateDefinition,
|
||||
PartyRepresentation,
|
||||
PartySubjectReference,
|
||||
ProcedureParty,
|
||||
ServiceBinding,
|
||||
ServiceDefinition,
|
||||
TemporalRevision,
|
||||
service_launch_capability,
|
||||
)
|
||||
from govoplan_cases.backend.party_context import CasePartyContext
|
||||
from govoplan_cases.backend.db.models import (
|
||||
CaseAccessGrant,
|
||||
CaseIdentity,
|
||||
CaseRecordRevision,
|
||||
CaseStatusDefinition,
|
||||
CaseTimelineEntry,
|
||||
CaseTypeDefinition,
|
||||
)
|
||||
from govoplan_cases.backend.service import (
|
||||
create_case_from_intake,
|
||||
get_case,
|
||||
upsert_case_status,
|
||||
upsert_case_type,
|
||||
)
|
||||
from govoplan_cases.backend.service_intake import CaseServiceIntake
|
||||
from govoplan_committee.backend.db.models import (
|
||||
CommitteeDecisionProjection,
|
||||
CommitteeWorkspaceEvent,
|
||||
CommitteeWorkspaceRevision,
|
||||
)
|
||||
from govoplan_committee.backend.decision_path import (
|
||||
CommitteeDecisionPath,
|
||||
CommitteeDecisionProposal,
|
||||
)
|
||||
from govoplan_committee.backend.workspace import (
|
||||
CommitteeWorkspaceRecord,
|
||||
SqlCommitteeWorkspace,
|
||||
get_workspace_object,
|
||||
record_workspace_object,
|
||||
)
|
||||
from govoplan_portal.backend.service_directory import PortalServiceDirectory
|
||||
from govoplan_decisions.backend.db.models import FormalDecisionRevision
|
||||
from govoplan_decisions.backend.service import SqlDecisionRegistry
|
||||
from govoplan_mandates.backend.db.models import MandateRevision
|
||||
from govoplan_mandates.backend.service import SqlMandateResolver, record_mandate
|
||||
from govoplan_parties.backend.db.models import ProcedurePartyRevision
|
||||
from govoplan_parties.backend.service import SqlPartyResolver, record_procedure_party
|
||||
from govoplan_services.backend.db.models import ServiceDefinitionRevision
|
||||
from govoplan_services.backend.service import SqlServiceDefinitionProvider, record_service_definition
|
||||
|
||||
|
||||
NOW = datetime(2026, 8, 1, 10, 0, tzinfo=UTC)
|
||||
|
||||
|
||||
def _reference(
|
||||
kind: str,
|
||||
object_id: str,
|
||||
*,
|
||||
owner: str,
|
||||
version: str = "1",
|
||||
) -> InstitutionalReference:
|
||||
return InstitutionalReference(
|
||||
kind=kind, # type: ignore[arg-type]
|
||||
owner_module=owner,
|
||||
object_id=object_id,
|
||||
tenant_id="tenant-1",
|
||||
version=version,
|
||||
valid_at=NOW,
|
||||
)
|
||||
|
||||
|
||||
def _evidence(kind: str, evidence_id: str, owner: str) -> EvidenceReference:
|
||||
return EvidenceReference(
|
||||
kind=kind, # type: ignore[arg-type]
|
||||
owner_module=owner,
|
||||
evidence_id=evidence_id,
|
||||
tenant_id="tenant-1",
|
||||
version="1",
|
||||
checksum=f"sha256:{evidence_id}",
|
||||
captured_at=NOW,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Principal:
|
||||
tenant_id: str = "tenant-1"
|
||||
account_id: str = "account-1"
|
||||
|
||||
|
||||
class _Registry:
|
||||
def __init__(self, capabilities: dict[str, object]) -> None:
|
||||
self.capabilities = capabilities
|
||||
|
||||
def has(self, module_id: str) -> bool:
|
||||
return module_id in {
|
||||
"portal",
|
||||
"cases",
|
||||
"committee",
|
||||
"postbox",
|
||||
"services",
|
||||
"parties",
|
||||
"mandates",
|
||||
"decisions",
|
||||
}
|
||||
|
||||
def has_capability(self, name: str) -> bool:
|
||||
return name in self.capabilities
|
||||
|
||||
def capability(self, name: str) -> object:
|
||||
return self.capabilities[name]
|
||||
|
||||
|
||||
class InstitutionalGovernanceJourneyTests(unittest.TestCase):
|
||||
def test_service_to_formal_outcome_retains_governed_context(self) -> None:
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
for table in (
|
||||
ServiceDefinitionRevision.__table__,
|
||||
CaseStatusDefinition.__table__,
|
||||
CaseTypeDefinition.__table__,
|
||||
CaseIdentity.__table__,
|
||||
CaseRecordRevision.__table__,
|
||||
CaseAccessGrant.__table__,
|
||||
CaseTimelineEntry.__table__,
|
||||
ProcedurePartyRevision.__table__,
|
||||
MandateRevision.__table__,
|
||||
FormalDecisionRevision.__table__,
|
||||
CommitteeWorkspaceRevision.__table__,
|
||||
CommitteeWorkspaceEvent.__table__,
|
||||
CommitteeDecisionProjection.__table__,
|
||||
):
|
||||
table.create(engine)
|
||||
session = Session(engine)
|
||||
principal = _Principal()
|
||||
self.addCleanup(engine.dispose)
|
||||
self.addCleanup(session.close)
|
||||
|
||||
organization = _reference(
|
||||
"organization_unit",
|
||||
"permit-office",
|
||||
owner="organizations",
|
||||
)
|
||||
function = _reference("function", "permit-officer", owner="organizations")
|
||||
jurisdiction = _reference(
|
||||
"jurisdiction",
|
||||
"city-1",
|
||||
owner="organizations",
|
||||
)
|
||||
mandate_ref = _reference(
|
||||
"mandate",
|
||||
"permit-mandate",
|
||||
owner="mandates",
|
||||
version="4",
|
||||
)
|
||||
legal_basis = LegalBasisReference(
|
||||
kind="law",
|
||||
authority="Example legislature",
|
||||
reference="permit-law:3",
|
||||
version="2026-01",
|
||||
effective_from=NOW - timedelta(days=100),
|
||||
)
|
||||
service = ServiceDefinition(
|
||||
reference=_reference(
|
||||
"service",
|
||||
"permit-service",
|
||||
owner="services",
|
||||
version="5",
|
||||
),
|
||||
key="permit.apply",
|
||||
temporal=TemporalRevision(
|
||||
revision="5",
|
||||
valid_from=NOW - timedelta(days=1),
|
||||
valid_to=NOW + timedelta(days=30),
|
||||
recorded_at=NOW - timedelta(days=2),
|
||||
change_reason="Service published.",
|
||||
),
|
||||
title="Apply for a permit",
|
||||
audience=("resident",),
|
||||
legal_bases=(legal_basis,),
|
||||
required_evidence_types=("application", "identity"),
|
||||
channels=("portal", "postbox"),
|
||||
responsible_organization_ref=organization,
|
||||
responsible_function_ref=function,
|
||||
mandate_ref=mandate_ref,
|
||||
jurisdiction_refs=(jurisdiction,),
|
||||
bindings=(
|
||||
ServiceBinding("case", "permit-application"),
|
||||
ServiceBinding("workflow", "workflow:permit-review"),
|
||||
ServiceBinding("result", "decision:permit"),
|
||||
),
|
||||
remedy_refs=("review:administrative-court",),
|
||||
publication_state="published",
|
||||
)
|
||||
record_service_definition(session, principal, definition=service)
|
||||
service_registry = _Registry(
|
||||
{
|
||||
"services.definitions": SqlServiceDefinitionProvider(),
|
||||
service_launch_capability("case"): object(),
|
||||
}
|
||||
)
|
||||
entry = PortalServiceDirectory(service_registry).list_entries(
|
||||
session,
|
||||
principal,
|
||||
tenant_id="tenant-1",
|
||||
effective_at=NOW,
|
||||
audiences=("resident",),
|
||||
)[0]
|
||||
self.assertTrue(entry.available)
|
||||
intake = CaseServiceIntake().plan(
|
||||
entry.definition,
|
||||
case_id="case-1",
|
||||
effective_at=NOW,
|
||||
)
|
||||
application_evidence = _evidence("document", "application-1", "files")
|
||||
upsert_case_status(
|
||||
session,
|
||||
principal,
|
||||
status_key="intake",
|
||||
label="Intake",
|
||||
)
|
||||
upsert_case_status(
|
||||
session,
|
||||
principal,
|
||||
status_key="decided",
|
||||
label="Decided",
|
||||
category="decided",
|
||||
)
|
||||
upsert_case_type(
|
||||
session,
|
||||
principal,
|
||||
type_key="permit-application",
|
||||
label="Permit application",
|
||||
initial_status_key="intake",
|
||||
allowed_status_keys=("intake", "decided"),
|
||||
)
|
||||
case_record = create_case_from_intake(
|
||||
session,
|
||||
principal,
|
||||
plan=intake,
|
||||
case_number="PERMIT-2026-0001",
|
||||
title="Permit application",
|
||||
status_key=None,
|
||||
opened_at=NOW,
|
||||
recorded_at=NOW,
|
||||
change_reason="Portal application received.",
|
||||
idempotency_key="journey-case-create",
|
||||
evidence_refs=(application_evidence,),
|
||||
deadline_at=NOW + timedelta(days=30),
|
||||
)
|
||||
|
||||
applicant = _reference("party", "applicant", owner="parties")
|
||||
representative = _reference("party", "representative", owner="parties")
|
||||
address_evidence = _evidence(
|
||||
"snapshot",
|
||||
"address-snapshot-1",
|
||||
"addresses",
|
||||
)
|
||||
representative_party = ProcedureParty(
|
||||
reference=representative,
|
||||
procedure_ref=case_record.reference,
|
||||
role="representative",
|
||||
subject=PartySubjectReference(
|
||||
kind="identity",
|
||||
provider="identity",
|
||||
subject_id="identity-2",
|
||||
tenant_id="tenant-1",
|
||||
version="2",
|
||||
),
|
||||
temporal=TemporalRevision(
|
||||
revision="1",
|
||||
valid_from=NOW - timedelta(days=1),
|
||||
recorded_at=NOW - timedelta(days=1),
|
||||
change_reason="Representative added to the procedure.",
|
||||
),
|
||||
permitted_channels=("postbox",),
|
||||
preferred_channels=("postbox",),
|
||||
delivery_recipient=True,
|
||||
representations=(
|
||||
PartyRepresentation(
|
||||
representative_party_ref=representative,
|
||||
represented_party_ref=applicant,
|
||||
power_ref="power-1",
|
||||
permitted_actions=("submit", "receive"),
|
||||
temporal=TemporalRevision(
|
||||
revision="1",
|
||||
valid_from=NOW - timedelta(days=1),
|
||||
recorded_at=NOW - timedelta(days=1),
|
||||
change_reason="Representation power recorded.",
|
||||
),
|
||||
evidence=(address_evidence,),
|
||||
),
|
||||
),
|
||||
contact_snapshot_refs=("addresses:snapshot-1",),
|
||||
evidence=(address_evidence,),
|
||||
)
|
||||
record_procedure_party(session, principal, party=representative_party)
|
||||
party_context = CasePartyContext(
|
||||
_Registry({"parties.resolver": SqlPartyResolver()})
|
||||
)
|
||||
parties = party_context.resolve(
|
||||
session,
|
||||
principal,
|
||||
case_ref=case_record.reference,
|
||||
effective_at=NOW,
|
||||
)
|
||||
delivery_target = party_context.delivery_targets(
|
||||
parties,
|
||||
channel="postbox",
|
||||
)[0]
|
||||
|
||||
mandate = MandateDefinition(
|
||||
reference=mandate_ref,
|
||||
temporal=TemporalRevision(
|
||||
revision="4",
|
||||
valid_from=NOW - timedelta(days=30),
|
||||
valid_to=NOW + timedelta(days=30),
|
||||
recorded_at=NOW - timedelta(days=31),
|
||||
change_reason="Permit authority delegated.",
|
||||
),
|
||||
task_types=("committee.formal_decision",),
|
||||
authority_types=("permit",),
|
||||
organization_unit_refs=(organization,),
|
||||
function_refs=(function,),
|
||||
jurisdiction_refs=(jurisdiction,),
|
||||
legal_bases=(legal_basis,),
|
||||
evidence=(_evidence("record", "mandate-record-4", "mandates"),),
|
||||
authority_ceiling="permit:standard",
|
||||
)
|
||||
record_mandate(session, principal, definition=mandate)
|
||||
workspace = SqlCommitteeWorkspace()
|
||||
body = CommitteeWorkspaceRecord(
|
||||
tenant_id="tenant-1",
|
||||
object_kind="body",
|
||||
object_id="permit-board",
|
||||
revision=1,
|
||||
state="active",
|
||||
title="Permit board",
|
||||
recorded_at=NOW,
|
||||
change_reason="Permit board configured.",
|
||||
attributes={
|
||||
"organization_unit_ref": organization.to_dict(),
|
||||
"function_refs": [function.to_dict()],
|
||||
"quorum": {"minimum_count": 1},
|
||||
},
|
||||
)
|
||||
meeting = CommitteeWorkspaceRecord(
|
||||
tenant_id="tenant-1",
|
||||
object_kind="meeting",
|
||||
object_id="meeting-1",
|
||||
revision=1,
|
||||
state="open",
|
||||
title="Permit board meeting",
|
||||
parent_id=body.object_id,
|
||||
recorded_at=NOW,
|
||||
change_reason="Meeting opened.",
|
||||
attributes={
|
||||
"starts_at": NOW.isoformat(),
|
||||
"ends_at": (NOW + timedelta(hours=1)).isoformat(),
|
||||
},
|
||||
context=case_record.context,
|
||||
)
|
||||
agenda = CommitteeWorkspaceRecord(
|
||||
tenant_id="tenant-1",
|
||||
object_kind="agenda_item",
|
||||
object_id="item-1",
|
||||
revision=1,
|
||||
state="deliberating",
|
||||
title="Permit application",
|
||||
parent_id=meeting.object_id,
|
||||
recorded_at=NOW,
|
||||
change_reason="Agenda item entered deliberation.",
|
||||
attributes={
|
||||
"position": 1,
|
||||
"subject_refs": [case_record.reference.to_dict()],
|
||||
},
|
||||
context=case_record.context,
|
||||
evidence=(application_evidence,),
|
||||
)
|
||||
approval_ref = _reference("approval", "approval-1", owner="approvals")
|
||||
vote = CommitteeWorkspaceRecord(
|
||||
tenant_id="tenant-1",
|
||||
object_kind="vote",
|
||||
object_id="vote-1",
|
||||
revision=1,
|
||||
state="closed",
|
||||
title="Vote on permit application",
|
||||
parent_id=agenda.object_id,
|
||||
recorded_at=NOW,
|
||||
change_reason="Vote result accepted.",
|
||||
attributes={
|
||||
"method": "recorded",
|
||||
"choices": ["yes", "no"],
|
||||
"eligible_count": 3,
|
||||
"cast_count": 3,
|
||||
"counts": {"yes": 3, "no": 0},
|
||||
"quorum_met": True,
|
||||
"approval_ref": approval_ref.to_dict(),
|
||||
},
|
||||
context=case_record.context,
|
||||
evidence=(_evidence("record", "vote-result-1", "committee"),),
|
||||
)
|
||||
for record, key in (
|
||||
(body, "journey-body"),
|
||||
(meeting, "journey-meeting"),
|
||||
(agenda, "journey-agenda"),
|
||||
(vote, "journey-vote"),
|
||||
):
|
||||
record_workspace_object(
|
||||
session,
|
||||
principal,
|
||||
record=record,
|
||||
idempotency_key=key,
|
||||
)
|
||||
decision_registry = SqlDecisionRegistry()
|
||||
decision_result = CommitteeDecisionPath(
|
||||
_Registry(
|
||||
{
|
||||
"mandates.resolver": SqlMandateResolver(),
|
||||
"decisions.registry": decision_registry,
|
||||
"committee.workspace": workspace,
|
||||
}
|
||||
)
|
||||
).decide(
|
||||
session,
|
||||
principal,
|
||||
proposal=CommitteeDecisionProposal(
|
||||
tenant_id="tenant-1",
|
||||
decision_id="decision-1",
|
||||
revision="1",
|
||||
effective_at=NOW,
|
||||
meeting_ref="meeting-1",
|
||||
agenda_item_ref="item-1",
|
||||
decision_type="permit",
|
||||
subject_refs=(case_record.reference,),
|
||||
organization_unit_ref=organization,
|
||||
function_ref=function,
|
||||
actor=ActorRepresentationReference(
|
||||
tenant_id="tenant-1",
|
||||
account_id="account-1",
|
||||
identity_id="identity-1",
|
||||
represented_function_ref=function,
|
||||
mandate_ref=mandate_ref,
|
||||
),
|
||||
approval_refs=(
|
||||
approval_ref,
|
||||
),
|
||||
fact_evidence=(application_evidence, address_evidence),
|
||||
legal_bases=(legal_basis,),
|
||||
operative_result="Permit granted.",
|
||||
reasoning="The application satisfies the effective rule.",
|
||||
case_ref=case_record.reference,
|
||||
jurisdiction_refs=(jurisdiction,),
|
||||
party_refs=(applicant, representative),
|
||||
record_refs=(
|
||||
_reference("record", "record-1", owner="audit"),
|
||||
),
|
||||
remedy_refs=service.remedy_refs,
|
||||
review_refs=("review:administrative-court",),
|
||||
information_governance=InformationGovernanceReference(
|
||||
classification="restricted",
|
||||
purposes=("permit-decision", "party-delivery"),
|
||||
legal_basis_refs=("permit-law:3@2026-01",),
|
||||
retention_policy_ref="records:permit",
|
||||
disclosure_state="partly_disclosable",
|
||||
),
|
||||
),
|
||||
observed_effects=(
|
||||
DecisionEffectReference(
|
||||
effect_key="postbox.deliver_decision",
|
||||
state="confirmed",
|
||||
resource_refs=(
|
||||
f"postbox:{delivery_target.party_ref.object_id}",
|
||||
),
|
||||
audit_event_refs=("audit:delivery-1",),
|
||||
evidence_refs=(address_evidence.evidence_id,),
|
||||
),
|
||||
),
|
||||
)
|
||||
decided_agenda = replace(
|
||||
agenda,
|
||||
revision=2,
|
||||
state="decided",
|
||||
recorded_at=NOW + timedelta(minutes=1),
|
||||
change_reason="Formal Decision recorded.",
|
||||
attributes={
|
||||
**dict(agenda.attributes),
|
||||
"decision_ref": decision_result.decision.reference.to_dict(),
|
||||
},
|
||||
)
|
||||
record_workspace_object(
|
||||
session,
|
||||
principal,
|
||||
record=decided_agenda,
|
||||
expected_revision=1,
|
||||
idempotency_key="journey-agenda-decided",
|
||||
)
|
||||
closed_meeting = replace(
|
||||
meeting,
|
||||
revision=2,
|
||||
state="closed",
|
||||
recorded_at=NOW + timedelta(hours=1),
|
||||
change_reason="All agenda items completed.",
|
||||
)
|
||||
record_workspace_object(
|
||||
session,
|
||||
principal,
|
||||
record=closed_meeting,
|
||||
expected_revision=1,
|
||||
idempotency_key="journey-meeting-closed",
|
||||
)
|
||||
minute = CommitteeWorkspaceRecord(
|
||||
tenant_id="tenant-1",
|
||||
object_kind="minute",
|
||||
object_id="minute-1",
|
||||
revision=1,
|
||||
state="accepted",
|
||||
title="Accepted permit board minutes",
|
||||
parent_id=meeting.object_id,
|
||||
recorded_at=NOW + timedelta(hours=2),
|
||||
change_reason="Minutes approved.",
|
||||
attributes={
|
||||
"content_ref": _reference(
|
||||
"record",
|
||||
"meeting-minutes-1",
|
||||
owner="records",
|
||||
).to_dict(),
|
||||
"approval_ref": _reference(
|
||||
"approval",
|
||||
"minutes-approval-1",
|
||||
owner="approvals",
|
||||
).to_dict(),
|
||||
},
|
||||
context=case_record.context,
|
||||
evidence=(_evidence("record", "minutes-proof-1", "records"),),
|
||||
)
|
||||
record_workspace_object(
|
||||
session,
|
||||
principal,
|
||||
record=minute,
|
||||
idempotency_key="journey-minute",
|
||||
)
|
||||
reconstruction = decision_result.reconstruction_payload()["decision"]
|
||||
persisted = decision_registry.get_decision(
|
||||
session,
|
||||
principal,
|
||||
reference=decision_result.decision.reference,
|
||||
)
|
||||
|
||||
self.assertTrue(decision_result.persisted_by_decision_registry)
|
||||
self.assertEqual(decision_result.decision, persisted)
|
||||
self.assertEqual(
|
||||
case_record,
|
||||
get_case(session, principal, case_id="case-1"),
|
||||
)
|
||||
self.assertEqual(
|
||||
"decided",
|
||||
get_workspace_object(
|
||||
session,
|
||||
principal,
|
||||
object_kind="agenda_item",
|
||||
object_id="item-1",
|
||||
).state,
|
||||
)
|
||||
self.assertEqual(service.reference, intake.context.service_ref)
|
||||
self.assertEqual("applicant", delivery_target.represented_party_refs[0].object_id)
|
||||
self.assertEqual("permit-mandate", reconstruction["authority_context"]["mandate_ref"]["object_id"])
|
||||
self.assertEqual("city-1", reconstruction["authority_context"]["jurisdiction_refs"][0]["object_id"])
|
||||
self.assertEqual("confirmed", reconstruction["observed_effects"][0]["state"])
|
||||
self.assertEqual("audit:delivery-1", reconstruction["observed_effects"][0]["audit_event_refs"][0])
|
||||
self.assertEqual("application-1", reconstruction["fact_evidence"][0]["evidence_id"])
|
||||
self.assertEqual("The application satisfies the effective rule.", reconstruction["reasoning"])
|
||||
self.assertEqual("review:administrative-court", reconstruction["review_refs"][0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,271 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import unittest
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_core.core.institutional import (
|
||||
CAPABILITY_FORM_DEFINITIONS,
|
||||
CAPABILITY_SERVICE_DEFINITIONS,
|
||||
FormDefinition,
|
||||
FormFieldDefinition,
|
||||
InstitutionalReference,
|
||||
ServiceBinding,
|
||||
ServiceDefinition,
|
||||
TemporalRevision,
|
||||
service_launch_capability,
|
||||
)
|
||||
from govoplan_cases.backend.service_intake import (
|
||||
CAPABILITY_CASES_SERVICE_INTAKE,
|
||||
CaseServiceIntake,
|
||||
)
|
||||
from govoplan_forms.backend.db.models import FormDefinitionRevision
|
||||
from govoplan_forms.backend.service import (
|
||||
SqlFormDefinitionProvider,
|
||||
record_form_definition,
|
||||
)
|
||||
from govoplan_forms_runtime.backend.db.models import (
|
||||
FormInstanceEvent,
|
||||
FormInstanceIdentity,
|
||||
FormInstanceRevision,
|
||||
)
|
||||
from govoplan_forms_runtime.backend.service import (
|
||||
FormRuntimeService,
|
||||
FormsServiceLauncher,
|
||||
)
|
||||
from govoplan_portal.backend.service_directory import PortalServiceDirectory
|
||||
|
||||
|
||||
NOW = datetime(2026, 8, 1, 10, 0, tzinfo=UTC)
|
||||
|
||||
|
||||
def _service() -> ServiceDefinition:
|
||||
return ServiceDefinition(
|
||||
reference=InstitutionalReference(
|
||||
kind="service",
|
||||
owner_module="portal",
|
||||
object_id="permit",
|
||||
tenant_id="tenant-1",
|
||||
version="5",
|
||||
),
|
||||
key="permit.apply",
|
||||
temporal=TemporalRevision(
|
||||
revision="5",
|
||||
valid_from=NOW - timedelta(days=1),
|
||||
valid_to=NOW + timedelta(days=1),
|
||||
recorded_at=NOW - timedelta(days=2),
|
||||
),
|
||||
title="Apply for a permit",
|
||||
audience=("resident",),
|
||||
required_evidence_types=("application",),
|
||||
bindings=(
|
||||
ServiceBinding("capability", CAPABILITY_CASES_SERVICE_INTAKE),
|
||||
ServiceBinding("case", "permit-application"),
|
||||
ServiceBinding("workflow", "workflow:permit-review"),
|
||||
),
|
||||
publication_state="published",
|
||||
)
|
||||
|
||||
|
||||
class _Provider:
|
||||
def __init__(self, definition: ServiceDefinition) -> None:
|
||||
self.definition = definition
|
||||
|
||||
def get_service_definition(self, session, principal, *, reference, effective_at=None):
|
||||
return self.definition
|
||||
|
||||
def list_service_definitions(self, session, principal, *, tenant_id, query="", limit=100):
|
||||
return (self.definition,)
|
||||
|
||||
|
||||
class _Registry:
|
||||
def __init__(self, definition: ServiceDefinition) -> None:
|
||||
self.capabilities = {
|
||||
CAPABILITY_SERVICE_DEFINITIONS: _Provider(definition),
|
||||
CAPABILITY_CASES_SERVICE_INTAKE: CaseServiceIntake(),
|
||||
service_launch_capability("case"): object(),
|
||||
}
|
||||
|
||||
def has(self, module_id: str) -> bool:
|
||||
return module_id in {"portal", "cases"}
|
||||
|
||||
def has_capability(self, name: str) -> bool:
|
||||
return name in self.capabilities
|
||||
|
||||
def capability(self, name: str) -> object:
|
||||
return self.capabilities[name]
|
||||
|
||||
def require_capability(self, name: str) -> object:
|
||||
return self.capabilities[name]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Principal:
|
||||
tenant_id: str = "tenant-1"
|
||||
account_id: str = "account-1"
|
||||
|
||||
|
||||
class _FormRegistry(_Registry):
|
||||
def __init__(self, definition: ServiceDefinition) -> None:
|
||||
super().__init__(definition)
|
||||
self.capabilities[CAPABILITY_FORM_DEFINITIONS] = (
|
||||
SqlFormDefinitionProvider()
|
||||
)
|
||||
self.capabilities[service_launch_capability("form")] = (
|
||||
FormsServiceLauncher(self)
|
||||
)
|
||||
|
||||
def has(self, module_id: str) -> bool:
|
||||
return module_id in {"portal", "forms", "forms_runtime"}
|
||||
|
||||
|
||||
class InstitutionalServiceJourneyTests(unittest.TestCase):
|
||||
def test_one_service_version_drives_portal_and_case_intake(self) -> None:
|
||||
definition = _service()
|
||||
registry = _Registry(definition)
|
||||
|
||||
entries = PortalServiceDirectory(registry).list_entries(
|
||||
None,
|
||||
None,
|
||||
tenant_id="tenant-1",
|
||||
effective_at=NOW,
|
||||
audiences=("resident",),
|
||||
)
|
||||
plan = registry.capability(CAPABILITY_CASES_SERVICE_INTAKE).plan(
|
||||
entries[0].definition,
|
||||
case_id="case-1",
|
||||
effective_at=NOW,
|
||||
)
|
||||
|
||||
self.assertTrue(entries[0].available)
|
||||
self.assertIs(definition, entries[0].definition)
|
||||
self.assertEqual(definition.reference, plan.service_ref)
|
||||
self.assertEqual("5", plan.context.service_ref.version)
|
||||
self.assertEqual("workflow:permit-review", plan.workflow_refs[0])
|
||||
|
||||
def test_portal_launches_exact_form_revision_and_persists_submission(self) -> None:
|
||||
engine = create_engine("sqlite+pysqlite:///:memory:")
|
||||
for table in (
|
||||
FormDefinitionRevision.__table__,
|
||||
FormInstanceIdentity.__table__,
|
||||
FormInstanceRevision.__table__,
|
||||
FormInstanceEvent.__table__,
|
||||
):
|
||||
table.create(engine)
|
||||
session = Session(engine)
|
||||
principal = _Principal()
|
||||
try:
|
||||
form = record_form_definition(
|
||||
session,
|
||||
principal,
|
||||
definition=FormDefinition(
|
||||
reference=InstitutionalReference(
|
||||
kind="form",
|
||||
owner_module="forms",
|
||||
object_id="permit-application",
|
||||
tenant_id="tenant-1",
|
||||
version="3",
|
||||
),
|
||||
key="permit-application",
|
||||
temporal=TemporalRevision(
|
||||
revision="3",
|
||||
valid_from=NOW - timedelta(days=1),
|
||||
valid_to=NOW + timedelta(days=1),
|
||||
recorded_at=NOW - timedelta(days=2),
|
||||
change_reason="Publish the permit application.",
|
||||
),
|
||||
title="Permit application",
|
||||
fields=(
|
||||
FormFieldDefinition(
|
||||
key="applicant_name",
|
||||
label="Applicant name",
|
||||
required=True,
|
||||
constraints={"min_length": 2},
|
||||
),
|
||||
),
|
||||
publication_state="published",
|
||||
allow_drafts=True,
|
||||
handoff_kinds=("case",),
|
||||
),
|
||||
)
|
||||
binding = ServiceBinding(
|
||||
"form",
|
||||
f"{form.reference.object_id}/{form.reference.version}",
|
||||
)
|
||||
service = ServiceDefinition(
|
||||
reference=InstitutionalReference(
|
||||
kind="service",
|
||||
owner_module="services",
|
||||
object_id="permit",
|
||||
tenant_id="tenant-1",
|
||||
version="6",
|
||||
),
|
||||
key="permit.apply",
|
||||
temporal=TemporalRevision(
|
||||
revision="6",
|
||||
valid_from=NOW - timedelta(days=1),
|
||||
valid_to=NOW + timedelta(days=1),
|
||||
recorded_at=NOW - timedelta(days=2),
|
||||
),
|
||||
title="Apply for a permit",
|
||||
audience=("public",),
|
||||
bindings=(binding,),
|
||||
publication_state="published",
|
||||
)
|
||||
registry = _FormRegistry(service)
|
||||
directory = PortalServiceDirectory(registry)
|
||||
|
||||
launched = directory.launch_service(
|
||||
session,
|
||||
principal,
|
||||
reference=service.reference,
|
||||
requested_at=NOW,
|
||||
idempotency_key="portal-form-launch-1",
|
||||
parameters={"applicant_name": "Ada Lovelace"},
|
||||
)
|
||||
replay = directory.launch_service(
|
||||
session,
|
||||
principal,
|
||||
reference=service.reference,
|
||||
requested_at=NOW,
|
||||
idempotency_key="portal-form-launch-1",
|
||||
parameters={"applicant_name": "Ada Lovelace"},
|
||||
)
|
||||
instance = FormRuntimeService(registry).get_instance(
|
||||
session,
|
||||
principal,
|
||||
instance_id=str(launched.metadata["form_instance_id"]),
|
||||
)
|
||||
|
||||
self.assertEqual("form_submission", launched.target_ref.kind)
|
||||
self.assertEqual(service.reference, launched.service_ref)
|
||||
self.assertEqual("3", launched.metadata["form_definition_revision"])
|
||||
self.assertTrue(replay.replayed)
|
||||
self.assertEqual(launched.target_ref, replay.target_ref)
|
||||
self.assertEqual(
|
||||
(
|
||||
form.reference.owner_module,
|
||||
form.reference.object_id,
|
||||
form.reference.tenant_id,
|
||||
form.reference.version,
|
||||
),
|
||||
(
|
||||
instance.definition_ref.owner_module,
|
||||
instance.definition_ref.object_id,
|
||||
instance.definition_ref.tenant_id,
|
||||
instance.definition_ref.version,
|
||||
),
|
||||
)
|
||||
self.assertEqual(NOW, instance.definition_ref.valid_at)
|
||||
self.assertEqual(service.reference, instance.service_ref)
|
||||
self.assertEqual("Ada Lovelace", instance.values["applicant_name"])
|
||||
finally:
|
||||
session.close()
|
||||
engine.dispose()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -4,17 +4,101 @@ import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from govoplan_core.core.modules import ModuleManifest
|
||||
from govoplan_core.core.provider_governance import (
|
||||
ExternalProviderDeclaration,
|
||||
ModuleArchitectureDeclaration,
|
||||
ModuleMaturityEvidence,
|
||||
ProviderBehaviorDeclaration,
|
||||
ProviderObjectDeclaration,
|
||||
)
|
||||
|
||||
|
||||
META_ROOT = Path(__file__).resolve().parents[1]
|
||||
RELEASE_ROOT = META_ROOT / "tools" / "release"
|
||||
if str(RELEASE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(RELEASE_ROOT))
|
||||
|
||||
from govoplan_release.catalog_entry_synthesis import validate_initial_entry_closure # noqa: E402
|
||||
from govoplan_release.catalog_entry_synthesis import ( # noqa: E402
|
||||
manifest_catalog_entry,
|
||||
validate_initial_entry_closure,
|
||||
)
|
||||
from govoplan_release.selective_catalog import apply_repo_updates # noqa: E402
|
||||
|
||||
|
||||
class ReleaseCatalogEntrySynthesisTests(unittest.TestCase):
|
||||
def test_catalog_entry_preserves_architecture_and_provider_declarations(
|
||||
self,
|
||||
) -> None:
|
||||
provider = ExternalProviderDeclaration(
|
||||
id="example.records",
|
||||
module_id="example",
|
||||
label="Example records",
|
||||
maturity="read",
|
||||
operations=("read",),
|
||||
objects=(
|
||||
ProviderObjectDeclaration(
|
||||
object_type="record",
|
||||
field_groups=("identity", "content"),
|
||||
authority_modes=("external_mirror",),
|
||||
default_authority_mode="external_mirror",
|
||||
),
|
||||
),
|
||||
behavior=ProviderBehaviorDeclaration(
|
||||
freshness="Reports the acquisition timestamp.",
|
||||
health="Reports source and parser health.",
|
||||
max_read_items=100,
|
||||
classifications=("internal",),
|
||||
purposes=("release contract test",),
|
||||
retention="The owning package retention policy applies.",
|
||||
),
|
||||
)
|
||||
architecture = ModuleArchitectureDeclaration(
|
||||
layer="data_reporting_integration",
|
||||
kind="integration",
|
||||
maturity="vertical_slice",
|
||||
evidence=(
|
||||
ModuleMaturityEvidence(
|
||||
kind="test",
|
||||
reference="tests/test_release_catalog_entry_synthesis.py",
|
||||
summary="Proves catalog metadata preservation.",
|
||||
),
|
||||
ModuleMaturityEvidence(
|
||||
kind="documentation",
|
||||
reference="docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md",
|
||||
summary="Defines the provider declaration contract.",
|
||||
),
|
||||
),
|
||||
known_limits=("Test-only provider.",),
|
||||
supported_authority_modes=("external_mirror",),
|
||||
owned_concepts=("example transport",),
|
||||
target_tested_providers=(provider.id,),
|
||||
)
|
||||
|
||||
entry = manifest_catalog_entry(
|
||||
manifest=ModuleManifest(
|
||||
id="example",
|
||||
name="Example",
|
||||
version="1.2.3",
|
||||
architecture=architecture,
|
||||
external_providers=(provider,),
|
||||
),
|
||||
repo="govoplan-example",
|
||||
package="govoplan-example",
|
||||
version="1.2.3",
|
||||
description=None,
|
||||
root=META_ROOT,
|
||||
repository_base="git+ssh://git@git.add-ideas.de/GovOPlaN",
|
||||
)
|
||||
|
||||
self.assertEqual("vertical_slice", entry["architecture"]["maturity"])
|
||||
self.assertEqual(
|
||||
"external_mirror",
|
||||
entry["external_providers"][0]["objects"][0][
|
||||
"default_authority_mode"
|
||||
],
|
||||
)
|
||||
|
||||
def test_selective_update_synthesizes_initial_entries_from_package_manifests(self) -> None:
|
||||
payload: dict[str, object] = {
|
||||
"core_release": {},
|
||||
|
||||
@@ -17,6 +17,34 @@ from govoplan_release import git_state # noqa: E402
|
||||
|
||||
|
||||
class ReleaseGitStateTests(unittest.TestCase):
|
||||
def test_manifest_version_does_not_confuse_interface_versions(self) -> None:
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
with TemporaryDirectory() as directory:
|
||||
root = Path(directory) / "govoplan-example"
|
||||
manifest = root / "src" / "govoplan_example" / "backend" / "manifest.py"
|
||||
manifest.parent.mkdir(parents=True)
|
||||
manifest.write_text(
|
||||
"\n".join(
|
||||
(
|
||||
'MODULE_VERSION = "1.2.3"',
|
||||
"manifest = ModuleManifest(",
|
||||
' id="example",',
|
||||
" version=MODULE_VERSION,",
|
||||
" provides_interfaces=(",
|
||||
' ModuleInterfaceProvider(name="example.items", version="9.8.7"),',
|
||||
" ),",
|
||||
")",
|
||||
"",
|
||||
)
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
versions = git_state.read_manifest_versions(root)
|
||||
|
||||
self.assertEqual(("1.2.3",), versions)
|
||||
|
||||
def test_git_trusts_only_the_resolved_repository_for_each_command(self) -> None:
|
||||
repository = Path("/workspace/../workspace/govoplan-core")
|
||||
completed = subprocess.CompletedProcess([], 0, "", "")
|
||||
|
||||
@@ -28,6 +28,8 @@ from govoplan_release.version_alignment import candidate_catalog_version_issues
|
||||
from .evidence import (
|
||||
BoundaryEvidenceReview,
|
||||
InstalledEvidenceReview,
|
||||
PROOF_REQUIREMENTS,
|
||||
REFERENCE_READINESS_SCOPES,
|
||||
public_key_material_from_keyrings,
|
||||
review_boundary_evidence,
|
||||
review_installed_composition,
|
||||
@@ -929,9 +931,8 @@ def build_report(
|
||||
boundary_review.proof_scope
|
||||
if boundary_review is not None
|
||||
else {
|
||||
"target_environment": {"checked": False, "valid": None},
|
||||
"external_providers": {"checked": False, "valid": None},
|
||||
"production_approval": {"checked": False, "valid": None},
|
||||
scope: {"checked": False, "valid": None}
|
||||
for scope in PROOF_REQUIREMENTS
|
||||
}
|
||||
)
|
||||
proof_scope = {
|
||||
@@ -961,6 +962,24 @@ def build_report(
|
||||
**installed_scope,
|
||||
**boundary_scope,
|
||||
}
|
||||
missing_readiness = tuple(
|
||||
scope
|
||||
for scope in REFERENCE_READINESS_SCOPES
|
||||
if proof_scope.get(scope, {}).get("checked") is not True
|
||||
)
|
||||
failed_readiness = tuple(
|
||||
scope
|
||||
for scope in REFERENCE_READINESS_SCOPES
|
||||
if proof_scope.get(scope, {}).get("checked") is True
|
||||
and proof_scope.get(scope, {}).get("valid") is not True
|
||||
)
|
||||
proof_scope["reference_readiness"] = {
|
||||
"checked": not missing_readiness,
|
||||
"valid": not missing_readiness and not failed_readiness,
|
||||
"required_scopes": list(REFERENCE_READINESS_SCOPES),
|
||||
"missing_scopes": list(missing_readiness),
|
||||
"failed_scopes": list(failed_readiness),
|
||||
}
|
||||
return {
|
||||
"report_version": "0.5.0",
|
||||
"status": status,
|
||||
@@ -1043,9 +1062,8 @@ def render_review(report: dict[str, Any]) -> str:
|
||||
unchecked_boundaries = [
|
||||
label
|
||||
for key, label in (
|
||||
("target_environment", "target-environment"),
|
||||
("external_providers", "external-provider"),
|
||||
("production_approval", "production-approval"),
|
||||
(scope, scope.replace("_", "-"))
|
||||
for scope in PROOF_REQUIREMENTS
|
||||
)
|
||||
if report.get("proof_scope", {}).get(key, {}).get("checked") is not True
|
||||
]
|
||||
@@ -1057,6 +1075,18 @@ def render_review(report: dict[str, Any]) -> str:
|
||||
)
|
||||
if unchecked_boundaries:
|
||||
lines.append("No " + ", ".join(unchecked_boundaries) + " proof was performed.")
|
||||
readiness = report.get("proof_scope", {}).get("reference_readiness", {})
|
||||
if readiness.get("valid") is True:
|
||||
lines.append("Reference-readiness evidence gate: passed.")
|
||||
else:
|
||||
unresolved = [
|
||||
*readiness.get("missing_scopes", []),
|
||||
*readiness.get("failed_scopes", []),
|
||||
]
|
||||
lines.append(
|
||||
"Reference-readiness evidence gate: not established"
|
||||
+ (f" ({', '.join(dict.fromkeys(unresolved))})." if unresolved else ".")
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
|
||||
@@ -57,11 +57,21 @@ SHA256_PATTERN = re.compile(r"^[0-9a-f]{64}$")
|
||||
POSITIVE_RESULTS = {
|
||||
"target_environment": "passed",
|
||||
"external_providers": "passed",
|
||||
"accessibility": "passed",
|
||||
"privacy": "passed",
|
||||
"security": "passed",
|
||||
"operations": "passed",
|
||||
"recovery": "passed",
|
||||
"production_approval": "approved",
|
||||
}
|
||||
NEGATIVE_RESULTS = {
|
||||
"target_environment": "failed",
|
||||
"external_providers": "failed",
|
||||
"accessibility": "failed",
|
||||
"privacy": "failed",
|
||||
"security": "failed",
|
||||
"operations": "failed",
|
||||
"recovery": "failed",
|
||||
"production_approval": "rejected",
|
||||
}
|
||||
PROOF_REQUIREMENTS = {
|
||||
@@ -75,12 +85,45 @@ PROOF_REQUIREMENTS = {
|
||||
"An independently trusted authority key permitted for external_providers proof.",
|
||||
"Opaque control IDs and content hashes; credentials and endpoint identifiers stay outside the report.",
|
||||
),
|
||||
"accessibility": (
|
||||
"Accessibility conformance results for the assessed journeys, viewports, keyboard paths, and assistive-technology profile.",
|
||||
"An independently trusted authority key permitted for accessibility proof.",
|
||||
"Opaque control IDs and content hashes for the sanitized conformance artifacts.",
|
||||
),
|
||||
"privacy": (
|
||||
"Privacy and data-protection controls evaluated against the exact assessed composition and target profile.",
|
||||
"An independently trusted authority key permitted for privacy proof.",
|
||||
"Opaque control IDs and content hashes for the sanitized review artifacts.",
|
||||
),
|
||||
"security": (
|
||||
"Security controls, scans, and bounded verification results for the exact release and target profile.",
|
||||
"An independently trusted authority key permitted for security proof.",
|
||||
"Opaque control IDs and content hashes for sanitized findings and attestations.",
|
||||
),
|
||||
"operations": (
|
||||
"Operator procedures, health checks, monitoring, alerting, and failure-response exercises for the target profile.",
|
||||
"An independently trusted authority key permitted for operations proof.",
|
||||
"Opaque control IDs and content hashes for sanitized run evidence.",
|
||||
),
|
||||
"recovery": (
|
||||
"A completed backup, restore, rollback, and reconciliation drill bound to the exact release and target profile.",
|
||||
"An independently trusted authority key permitted for recovery proof.",
|
||||
"Opaque control IDs and content hashes for drill outcomes and recovery-point/time observations.",
|
||||
),
|
||||
"production_approval": (
|
||||
"An explicit approval bound to this assessment release and installed evidence whose release origin is independently anchored.",
|
||||
"An independently provisioned authority key permitted for production_approval.",
|
||||
"An unexpired signed proof bundle; an assessment or operator cannot approve itself.",
|
||||
),
|
||||
}
|
||||
REFERENCE_READINESS_SCOPES = (
|
||||
"target_environment",
|
||||
"accessibility",
|
||||
"privacy",
|
||||
"security",
|
||||
"operations",
|
||||
"recovery",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
@@ -1427,6 +1470,11 @@ def review_boundary_evidence(
|
||||
expected_subjects = {
|
||||
"target_environment": deployment_subject,
|
||||
"external_providers": provider_subject,
|
||||
"accessibility": deployment_subject,
|
||||
"privacy": deployment_subject,
|
||||
"security": deployment_subject,
|
||||
"operations": deployment_subject,
|
||||
"recovery": deployment_subject,
|
||||
"production_approval": deployment_subject,
|
||||
}
|
||||
scopes = _unchecked_boundary_scope(expected_subjects=expected_subjects)
|
||||
|
||||
@@ -28,10 +28,14 @@ cd "$ROOT"
|
||||
|
||||
GOVOPLAN_CORE_ROOT="$ROOT" PYTHON="$PYTHON" CHECK_TESTCLIENT_DEPRECATIONS=1 bash "$META_ROOT/tools/checks/check-dependency-hygiene.sh"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-contracts.py" --no-impact
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py"
|
||||
PYTHONDONTWRITEBYTECODE=1 "$PYTHON" "$META_ROOT/tools/checks/check-manifest-shapes.py" --require-architecture
|
||||
|
||||
cd "$META_ROOT"
|
||||
"$PYTHON" -m unittest tests.test_deployment_installer
|
||||
"$PYTHON" -m unittest tests.test_capability_fit_evidence
|
||||
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
|
||||
"$PYTHON" -m unittest tests.test_institutional_governance_journey
|
||||
"$PYTHON" -m unittest tests.test_institutional_service_journey
|
||||
cd "$ROOT"
|
||||
|
||||
"$PYTHON" - <<'PY'
|
||||
@@ -81,6 +85,11 @@ PY
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-views/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-dashboard/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-postbox/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-portal/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-forms-runtime/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-cases/tests
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-committee/tests
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
|
||||
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
|
||||
|
||||
@@ -16,6 +16,7 @@ META_ROOT = Path(__file__).resolve().parents[2]
|
||||
MODULE_NAME_PATTERN = re.compile(
|
||||
r"[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*"
|
||||
)
|
||||
REQUIRED_DOCUMENTATION_TYPES = frozenset({"admin", "user"})
|
||||
|
||||
|
||||
def main() -> int:
|
||||
@@ -26,6 +27,14 @@ def main() -> int:
|
||||
default=None,
|
||||
help="Directory containing the GovOPlaN repositories. Defaults to repositories.json default_parent.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--require-architecture",
|
||||
action="store_true",
|
||||
help=(
|
||||
"Reject every module that has not adopted the versioned architecture "
|
||||
"declaration. GovOPlaN release and focused checks enable this gate."
|
||||
),
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
catalog = json.loads((META_ROOT / "repositories.json").read_text(encoding="utf-8"))
|
||||
@@ -90,6 +99,36 @@ def main() -> int:
|
||||
)
|
||||
continue
|
||||
|
||||
documented_types = {
|
||||
documentation_type
|
||||
for topic in manifest.documentation
|
||||
for documentation_type in topic.documentation_types
|
||||
}
|
||||
missing_documentation_types = sorted(REQUIRED_DOCUMENTATION_TYPES - documented_types)
|
||||
if missing_documentation_types:
|
||||
errors.append(
|
||||
f"{repository_name}: module {manifest.id!r} is missing static documentation for "
|
||||
f"{', '.join(missing_documentation_types)}; add manifest DocumentationTopic entries "
|
||||
"even when runtime documentation providers are registered"
|
||||
)
|
||||
continue
|
||||
|
||||
repository_root = manifest_path.parents[3]
|
||||
if manifest.architecture is None:
|
||||
if args.require_architecture:
|
||||
errors.append(
|
||||
f"{repository_name}: module {manifest.id!r} has no architecture declaration"
|
||||
)
|
||||
continue
|
||||
else:
|
||||
errors.extend(
|
||||
_architecture_evidence_errors(
|
||||
repository_name=repository_name,
|
||||
repository_root=repository_root,
|
||||
manifest=manifest,
|
||||
)
|
||||
)
|
||||
|
||||
manifests.append(manifest)
|
||||
|
||||
if errors:
|
||||
@@ -109,6 +148,11 @@ def main() -> int:
|
||||
f"Manifest registry check passed: {len(snapshot.manifests)} manifests "
|
||||
f"from {len(manifest_sources)} source files."
|
||||
)
|
||||
declared = sum(manifest.architecture is not None for manifest in manifests)
|
||||
print(
|
||||
f"Architecture declaration coverage: {declared}/{len(manifests)} modules "
|
||||
f"({(declared / len(manifests) * 100):.1f}%)."
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
@@ -120,5 +164,53 @@ def _module_entry_points(pyproject_path: Path) -> dict[str, str]:
|
||||
return {str(name): str(target) for name, target in values.items()}
|
||||
|
||||
|
||||
def _architecture_evidence_errors(
|
||||
*,
|
||||
repository_name: str,
|
||||
repository_root: Path,
|
||||
manifest: object,
|
||||
) -> list[str]:
|
||||
architecture = getattr(manifest, "architecture", None)
|
||||
if architecture is None:
|
||||
return []
|
||||
errors: list[str] = []
|
||||
references = [
|
||||
(f"{item.kind} evidence", item.reference)
|
||||
for item in architecture.evidence
|
||||
]
|
||||
for category in ("migration", "upgrade", "recovery", "security", "operations"):
|
||||
references.extend(
|
||||
(f"{category} documentation", reference)
|
||||
for reference in getattr(architecture.documentation, category)
|
||||
)
|
||||
for label, reference in references:
|
||||
if not _looks_like_repository_reference(reference):
|
||||
continue
|
||||
candidate = (repository_root / reference).resolve()
|
||||
try:
|
||||
candidate.relative_to(repository_root.resolve())
|
||||
except ValueError:
|
||||
errors.append(
|
||||
f"{repository_name}: {label} escapes the repository: {reference!r}"
|
||||
)
|
||||
continue
|
||||
if not candidate.exists():
|
||||
errors.append(
|
||||
f"{repository_name}: {label} does not exist: {reference!r}"
|
||||
)
|
||||
return errors
|
||||
|
||||
|
||||
def _looks_like_repository_reference(reference: str) -> bool:
|
||||
normalized = reference.strip()
|
||||
if not normalized or "://" in normalized:
|
||||
return False
|
||||
return (
|
||||
"/" in normalized
|
||||
or normalized.startswith("README")
|
||||
or normalized.endswith((".md", ".py", ".json", ".yaml", ".yml"))
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
||||
@@ -25,6 +25,13 @@ cd "$ROOT"
|
||||
--core-root "$ROOT"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check_dependency_boundaries.py"
|
||||
|
||||
cd "$META_ROOT"
|
||||
"$PYTHON" -m unittest tests.test_configuration_package_artifacts
|
||||
PYTHONPATH="$META_ROOT/../govoplan-portal/src:$META_ROOT/../govoplan-forms/src:$META_ROOT/../govoplan-forms-runtime/src:$META_ROOT/../govoplan-cases/src:$ROOT/src${PYTHONPATH:+:$PYTHONPATH}" \
|
||||
"$PYTHON" -m unittest tests.test_institutional_service_journey
|
||||
PYTHONPATH="$META_ROOT/../govoplan-portal/src:$META_ROOT/../govoplan-cases/src:$META_ROOT/../govoplan-committee/src:$META_ROOT/../govoplan-services/src:$META_ROOT/../govoplan-parties/src:$META_ROOT/../govoplan-mandates/src:$META_ROOT/../govoplan-decisions/src:$ROOT/src${PYTHONPATH:+:$PYTHONPATH}" \
|
||||
"$PYTHON" -m unittest tests.test_institutional_governance_journey
|
||||
|
||||
cd "$ROOT/webui"
|
||||
PATH="$ROOT/webui/node_modules/.bin:$NODE_BIN:$PATH" "$NPM" run test:module-capabilities
|
||||
PATH="$ROOT/webui/node_modules/.bin:$NODE_BIN:$PATH" "$NPM" run test:module-permutations
|
||||
|
||||
@@ -28,6 +28,12 @@ LOCK_FILENAME = ".deployment.lock"
|
||||
RUNTIME_ENV_KEYS = (
|
||||
"APP_ENV",
|
||||
"GOVOPLAN_INSTALL_PROFILE",
|
||||
"GOVOPLAN_INSTALLATION_ID",
|
||||
"GOVOPLAN_STATE_PROFILE",
|
||||
"GOVOPLAN_RUNTIME_HEARTBEAT_SECONDS",
|
||||
"GOVOPLAN_RUNTIME_STALE_AFTER_SECONDS",
|
||||
"GOVOPLAN_EXPECTED_API_REPLICAS",
|
||||
"GOVOPLAN_EXPECTED_WORKER_REPLICAS",
|
||||
"MASTER_KEY_B64",
|
||||
"DATABASE_URL",
|
||||
"GOVOPLAN_DATABASE_URL_PGTOOLS",
|
||||
@@ -55,6 +61,7 @@ RUNTIME_ENV_KEYS = (
|
||||
"FILE_STORAGE_S3_SECRET_ACCESS_KEY",
|
||||
"FILE_STORAGE_S3_BUCKET",
|
||||
"FILE_STORAGE_S3_DEPLOYMENT_MANAGED",
|
||||
"FILE_STORAGE_S3_ENDPOINT_TRUSTED",
|
||||
)
|
||||
|
||||
|
||||
@@ -169,6 +176,18 @@ def reconcile_runtime_environment(
|
||||
{
|
||||
"APP_ENV": "production" if spec.profile == "self-hosted" else "staging",
|
||||
"GOVOPLAN_INSTALL_PROFILE": spec.profile,
|
||||
"GOVOPLAN_INSTALLATION_ID": spec.installation_id,
|
||||
# Generated Compose replicas share one Docker host. A deliberately
|
||||
# Redis-free, single-API evaluation remains process-local; the
|
||||
# Kubernetes exporter overrides this to ``shared`` for independent
|
||||
# hosts.
|
||||
"GOVOPLAN_STATE_PROFILE": (
|
||||
"local" if redis.mode == "disabled" else "host-shared"
|
||||
),
|
||||
"GOVOPLAN_RUNTIME_HEARTBEAT_SECONDS": "15",
|
||||
"GOVOPLAN_RUNTIME_STALE_AFTER_SECONDS": "60",
|
||||
"GOVOPLAN_EXPECTED_API_REPLICAS": str(spec.replicas.api),
|
||||
"GOVOPLAN_EXPECTED_WORKER_REPLICAS": str(spec.replicas.worker),
|
||||
"ENABLED_MODULES": ",".join(spec.enabled_modules),
|
||||
"CELERY_ENABLED": "true" if redis.mode != "disabled" else "false",
|
||||
"CELERY_QUEUES": (
|
||||
@@ -198,6 +217,7 @@ def reconcile_runtime_environment(
|
||||
if storage.mode == "local":
|
||||
values["FILE_STORAGE_BACKEND"] = "local"
|
||||
values["FILE_STORAGE_S3_DEPLOYMENT_MANAGED"] = "false"
|
||||
values["FILE_STORAGE_S3_ENDPOINT_TRUSTED"] = "false"
|
||||
values["FILE_STORAGE_LOCAL_ROOT"] = "/var/lib/govoplan/files"
|
||||
elif storage.mode == "garage":
|
||||
access_key = values.setdefault(
|
||||
@@ -221,11 +241,13 @@ def reconcile_runtime_environment(
|
||||
"FILE_STORAGE_S3_SECRET_ACCESS_KEY": secret_key,
|
||||
"FILE_STORAGE_S3_BUCKET": bucket,
|
||||
"FILE_STORAGE_S3_DEPLOYMENT_MANAGED": "true",
|
||||
"FILE_STORAGE_S3_ENDPOINT_TRUSTED": "false",
|
||||
}
|
||||
)
|
||||
else:
|
||||
values["FILE_STORAGE_BACKEND"] = "s3"
|
||||
values["FILE_STORAGE_S3_DEPLOYMENT_MANAGED"] = "false"
|
||||
values["FILE_STORAGE_S3_ENDPOINT_TRUSTED"] = "true"
|
||||
required = (
|
||||
"FILE_STORAGE_S3_ENDPOINT_URL",
|
||||
"FILE_STORAGE_S3_REGION",
|
||||
@@ -243,12 +265,13 @@ def reconcile_runtime_environment(
|
||||
schemes={"http", "https"},
|
||||
label="S3 endpoint URL",
|
||||
)
|
||||
if spec.profile == "self-hosted" and endpoint.scheme != "https":
|
||||
raise ValueError("self-hosted S3 endpoint URL must use HTTPS")
|
||||
if endpoint.scheme != "https":
|
||||
raise ValueError("external S3 endpoint URL must use HTTPS")
|
||||
return dict(sorted(values.items()))
|
||||
|
||||
|
||||
def render_compose(spec: InstallationSpec) -> dict[str, object]:
|
||||
health_host = urlsplit(spec.public_url).hostname or "localhost"
|
||||
runtime_env = {
|
||||
"environment": _environment_references(RUNTIME_ENV_KEYS),
|
||||
}
|
||||
@@ -378,6 +401,10 @@ def render_compose(spec: InstallationSpec) -> dict[str, object]:
|
||||
|
||||
services["migrate"] = {
|
||||
**runtime_env,
|
||||
"environment": {
|
||||
**runtime_env["environment"],
|
||||
"GOVOPLAN_RUNTIME_ROLE": "migration",
|
||||
},
|
||||
"image": spec.release.api_image,
|
||||
"command": ["python", "-m", "govoplan_core.commands.init_db"],
|
||||
"restart": "no",
|
||||
@@ -387,6 +414,10 @@ def render_compose(spec: InstallationSpec) -> dict[str, object]:
|
||||
}
|
||||
services["api"] = {
|
||||
**common_runtime,
|
||||
"environment": {
|
||||
**runtime_env["environment"],
|
||||
"GOVOPLAN_RUNTIME_ROLE": "api",
|
||||
},
|
||||
"image": spec.release.api_image,
|
||||
"scale": spec.replicas.api,
|
||||
"command": [
|
||||
@@ -407,7 +438,10 @@ def render_compose(spec: InstallationSpec) -> dict[str, object]:
|
||||
"-c",
|
||||
(
|
||||
"import urllib.request;"
|
||||
"urllib.request.urlopen('http://127.0.0.1:8000/health',timeout=3)"
|
||||
"request=urllib.request.Request("
|
||||
"'http://127.0.0.1:8000/health/ready',"
|
||||
f"headers={{'Host': {health_host!r}}});"
|
||||
"urllib.request.urlopen(request,timeout=3)"
|
||||
),
|
||||
],
|
||||
"interval": "10s",
|
||||
@@ -454,6 +488,10 @@ def render_compose(spec: InstallationSpec) -> dict[str, object]:
|
||||
if spec.components.redis.mode != "disabled":
|
||||
services["worker"] = {
|
||||
**common_runtime,
|
||||
"environment": {
|
||||
**runtime_env["environment"],
|
||||
"GOVOPLAN_RUNTIME_ROLE": "worker",
|
||||
},
|
||||
"image": spec.release.api_image,
|
||||
"scale": spec.replicas.worker,
|
||||
"command": [
|
||||
@@ -474,8 +512,20 @@ def render_compose(spec: InstallationSpec) -> dict[str, object]:
|
||||
}
|
||||
services["scheduler"] = {
|
||||
**common_runtime,
|
||||
"environment": {
|
||||
**runtime_env["environment"],
|
||||
"GOVOPLAN_RUNTIME_ROLE": "scheduler",
|
||||
},
|
||||
"image": spec.release.api_image,
|
||||
"command": [
|
||||
"python",
|
||||
"-m",
|
||||
"govoplan_core.commands.fenced_run",
|
||||
"--resource",
|
||||
"scheduler:celery-beat",
|
||||
"--wait-seconds",
|
||||
"120",
|
||||
"--",
|
||||
"python",
|
||||
"-m",
|
||||
"celery",
|
||||
@@ -531,6 +581,7 @@ api_bind_addr = "[::]:3903"
|
||||
|
||||
|
||||
def render_load_balancer_config(spec: InstallationSpec) -> str:
|
||||
health_host = urlsplit(spec.public_url).hostname or "localhost"
|
||||
return f"""global
|
||||
log stdout format raw local0
|
||||
maxconn 4096
|
||||
@@ -572,7 +623,8 @@ frontend internal_api
|
||||
|
||||
backend api_replicas
|
||||
balance leastconn
|
||||
option httpchk GET /health
|
||||
option httpchk GET /health/ready
|
||||
http-check send hdr Host {health_host}
|
||||
http-check expect status 200
|
||||
server-template api- {spec.replicas.api} api:8000 check resolvers docker init-addr libc,none
|
||||
"""
|
||||
|
||||
@@ -47,7 +47,17 @@ from .model import (
|
||||
load_spec,
|
||||
parse_spec,
|
||||
)
|
||||
from .kubernetes import (
|
||||
kubernetes_secret_contract,
|
||||
render_kubernetes,
|
||||
write_secret_creation_hint,
|
||||
)
|
||||
from .planning import DeploymentPlan, build_plan
|
||||
from .recovery import (
|
||||
DeploymentOperationJournal,
|
||||
list_operations,
|
||||
recover_operation,
|
||||
)
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
@@ -115,6 +125,43 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
)
|
||||
_directory_argument(status)
|
||||
status.add_argument("--json", action="store_true", help="Print JSON.")
|
||||
|
||||
kubernetes = subparsers.add_parser(
|
||||
"render-kubernetes",
|
||||
help="Export the stateless multi-host runtime for Kubernetes.",
|
||||
)
|
||||
_directory_argument(kubernetes)
|
||||
kubernetes.add_argument("--namespace", default="govoplan")
|
||||
kubernetes.add_argument("--secret-name", default="govoplan-runtime")
|
||||
kubernetes.add_argument("--tls-secret-name", default="govoplan-tls")
|
||||
kubernetes.add_argument("--ingress-class-name")
|
||||
kubernetes.add_argument(
|
||||
"--output",
|
||||
type=Path,
|
||||
help="Output JSON path; defaults to <directory>/kubernetes.json.",
|
||||
)
|
||||
|
||||
operations = subparsers.add_parser(
|
||||
"operations",
|
||||
help="List durable deployment and recovery operations.",
|
||||
)
|
||||
_directory_argument(operations)
|
||||
operations.add_argument("--json", action="store_true", help="Print JSON.")
|
||||
|
||||
recover = subparsers.add_parser(
|
||||
"recover",
|
||||
help="Restore a pre-migration applied bundle or enter forward recovery.",
|
||||
)
|
||||
_directory_argument(recover)
|
||||
recover.add_argument("--operation-id")
|
||||
recover.add_argument(
|
||||
"--apply",
|
||||
action="store_true",
|
||||
help="Apply the restored or forward-recovery desired state immediately.",
|
||||
)
|
||||
recover.add_argument("--allow-unverified-images", action="store_true")
|
||||
recover.add_argument("--skip-pull", action="store_true")
|
||||
recover.add_argument("--health-timeout-seconds", type=float, default=120.0)
|
||||
return parser
|
||||
|
||||
|
||||
@@ -233,6 +280,12 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
return _apply(args)
|
||||
if args.command == "status":
|
||||
return _status(args)
|
||||
if args.command == "render-kubernetes":
|
||||
return _render_kubernetes(args)
|
||||
if args.command == "operations":
|
||||
return _operations(args)
|
||||
if args.command == "recover":
|
||||
return _recover(args)
|
||||
except (SpecError, ValueError, OSError, subprocess.SubprocessError) as exc:
|
||||
print(f"error: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
@@ -393,6 +446,10 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
docker = shutil.which("docker")
|
||||
if docker is None:
|
||||
raise ValueError("Docker CLI is required for apply")
|
||||
journal = DeploymentOperationJournal.begin(
|
||||
paths,
|
||||
plan=plan.to_dict(),
|
||||
)
|
||||
compose = [
|
||||
docker,
|
||||
"compose",
|
||||
@@ -403,62 +460,73 @@ def _apply(args: argparse.Namespace) -> int:
|
||||
"--file",
|
||||
str(paths.compose),
|
||||
]
|
||||
if not args.skip_pull:
|
||||
_run([*compose, "pull"], cwd=paths.root)
|
||||
dependencies = [
|
||||
name
|
||||
for name in ("postgres", "redis", "garage", "test-mail")
|
||||
if name in service_names(spec)
|
||||
]
|
||||
if dependencies:
|
||||
_run([*compose, "up", "--detach", *dependencies], cwd=paths.root)
|
||||
_run([*compose, "run", "--rm", "migrate"], cwd=paths.root)
|
||||
if _receipt_uses_direct_web_port(paths.receipt):
|
||||
_run([*compose, "stop", "web"], cwd=paths.root)
|
||||
runtime_services = [
|
||||
name
|
||||
for name in ("api", "web", "load-balancer", "worker", "scheduler")
|
||||
if name in service_names(spec)
|
||||
]
|
||||
_run(
|
||||
[*compose, "up", "--detach", "--remove-orphans", *runtime_services],
|
||||
cwd=paths.root,
|
||||
)
|
||||
_wait_for_health(
|
||||
f"{spec.public_url}/health",
|
||||
timeout_seconds=args.health_timeout_seconds,
|
||||
)
|
||||
receipt = {
|
||||
"schema_version": 1,
|
||||
"installation_id": spec.installation_id,
|
||||
"applied_at": _now(),
|
||||
"spec_sha256": digest_json(spec.to_dict()),
|
||||
"compose_sha256": digest_json(render_compose(spec)),
|
||||
"environment_fingerprint": environment_fingerprint(secrets),
|
||||
"release": {
|
||||
"channel": spec.release.channel,
|
||||
"version": spec.release.version,
|
||||
"manifest_sha256": spec.release.manifest_sha256,
|
||||
"api_image": spec.release.api_image,
|
||||
"web_image": spec.release.web_image,
|
||||
},
|
||||
"services": list(service_names(spec)),
|
||||
"replicas": {
|
||||
"api": spec.replicas.api,
|
||||
"web": spec.replicas.web,
|
||||
"worker": spec.replicas.worker,
|
||||
},
|
||||
"listen": {
|
||||
"address": spec.listen.address,
|
||||
"port": spec.listen.port,
|
||||
},
|
||||
"management": {
|
||||
"mode": "govoplan-deploy",
|
||||
"agent": "cli",
|
||||
"web_updates": False,
|
||||
},
|
||||
}
|
||||
atomic_write(paths.receipt, canonical_json(receipt), mode=0o600)
|
||||
try:
|
||||
if not args.skip_pull:
|
||||
_run([*compose, "pull"], cwd=paths.root)
|
||||
journal.record("images-pulled", "succeeded")
|
||||
dependencies = [
|
||||
name
|
||||
for name in ("postgres", "redis", "garage", "test-mail")
|
||||
if name in service_names(spec)
|
||||
]
|
||||
if dependencies:
|
||||
_run([*compose, "up", "--detach", *dependencies], cwd=paths.root)
|
||||
journal.record(
|
||||
"state-services-ready",
|
||||
"succeeded",
|
||||
{"services": dependencies},
|
||||
)
|
||||
mutable_runtime_services = [
|
||||
name
|
||||
for name in ("api", "worker", "scheduler")
|
||||
if name in service_names(spec)
|
||||
]
|
||||
if mutable_runtime_services:
|
||||
_run(
|
||||
[
|
||||
*compose,
|
||||
"stop",
|
||||
"--timeout",
|
||||
"120",
|
||||
*mutable_runtime_services,
|
||||
],
|
||||
cwd=paths.root,
|
||||
)
|
||||
journal.record(
|
||||
"runtime-quiesced",
|
||||
"succeeded",
|
||||
{"services": mutable_runtime_services, "timeout_seconds": 120},
|
||||
)
|
||||
journal.migration_started()
|
||||
_run([*compose, "run", "--rm", "migrate"], cwd=paths.root)
|
||||
journal.migration_completed()
|
||||
if _receipt_uses_direct_web_port(paths.receipt):
|
||||
_run([*compose, "stop", "web"], cwd=paths.root)
|
||||
runtime_services = [
|
||||
name
|
||||
for name in ("api", "web", "load-balancer", "worker", "scheduler")
|
||||
if name in service_names(spec)
|
||||
]
|
||||
_run(
|
||||
[*compose, "up", "--detach", "--remove-orphans", *runtime_services],
|
||||
cwd=paths.root,
|
||||
)
|
||||
journal.record(
|
||||
"runtime-reconciled",
|
||||
"succeeded",
|
||||
{"services": runtime_services},
|
||||
)
|
||||
_wait_for_health(
|
||||
f"{spec.public_url}/health/ready",
|
||||
timeout_seconds=args.health_timeout_seconds,
|
||||
)
|
||||
journal.record("health-verified", "succeeded", {"url": spec.public_url})
|
||||
receipt = _deployment_receipt(spec, secrets)
|
||||
atomic_write(paths.receipt, canonical_json(receipt), mode=0o600)
|
||||
journal.succeeded(paths, receipt=receipt)
|
||||
except BaseException as exc:
|
||||
journal.failed(exc)
|
||||
raise
|
||||
print(f"GovOPlaN is ready at {spec.public_url}")
|
||||
return 0
|
||||
|
||||
@@ -515,6 +583,104 @@ def _status(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _render_kubernetes(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
spec = load_spec(paths.spec)
|
||||
environment = reconcile_runtime_environment(spec, read_env(paths.env))
|
||||
manifest = render_kubernetes(
|
||||
spec,
|
||||
environment,
|
||||
namespace=args.namespace,
|
||||
secret_name=args.secret_name,
|
||||
tls_secret_name=args.tls_secret_name,
|
||||
ingress_class_name=args.ingress_class_name,
|
||||
)
|
||||
output = (args.output or (paths.root / "kubernetes.json")).expanduser().resolve()
|
||||
atomic_write(output, canonical_json(manifest), mode=0o600)
|
||||
print(f"Wrote stateless Kubernetes runtime manifest to {output}")
|
||||
print(
|
||||
"Required Secret keys: " + ", ".join(kubernetes_secret_contract())
|
||||
)
|
||||
print(
|
||||
write_secret_creation_hint(
|
||||
paths.env,
|
||||
namespace=args.namespace,
|
||||
secret_name=args.secret_name,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _operations(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
payload = list_operations(paths)
|
||||
if args.json:
|
||||
print(json.dumps(payload, indent=2, sort_keys=True))
|
||||
elif not payload:
|
||||
print("No deployment operations recorded.")
|
||||
else:
|
||||
for operation in payload:
|
||||
print(
|
||||
f"{operation['operation_id']} {operation['status']} "
|
||||
f"{operation['recovery_mode']} stages={operation['stage_count']}"
|
||||
)
|
||||
if operation.get("failure_summary"):
|
||||
print(f" {operation['failure_summary']}")
|
||||
return 0
|
||||
|
||||
|
||||
def _recover(args: argparse.Namespace) -> int:
|
||||
paths = bundle_paths(args.directory)
|
||||
ensure_private_directory(paths.root)
|
||||
with _deployment_lock(paths.lock):
|
||||
result = recover_operation(paths, operation_id=args.operation_id)
|
||||
print(f"Recovery operation {result.operation_id}: {result.action}")
|
||||
print(result.detail)
|
||||
if args.apply:
|
||||
if not result.apply_allowed:
|
||||
raise ValueError("this recovery state cannot be applied automatically")
|
||||
return _apply(args)
|
||||
if result.apply_allowed:
|
||||
print(f"Reconcile with: govoplan-deploy apply --directory {paths.root}")
|
||||
return 0
|
||||
|
||||
|
||||
def _deployment_receipt(
|
||||
spec: InstallationSpec,
|
||||
secrets: Mapping[str, str],
|
||||
) -> dict[str, object]:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"installation_id": spec.installation_id,
|
||||
"applied_at": _now(),
|
||||
"spec_sha256": digest_json(spec.to_dict()),
|
||||
"compose_sha256": digest_json(render_compose(spec)),
|
||||
"environment_fingerprint": environment_fingerprint(secrets),
|
||||
"release": {
|
||||
"channel": spec.release.channel,
|
||||
"version": spec.release.version,
|
||||
"manifest_sha256": spec.release.manifest_sha256,
|
||||
"api_image": spec.release.api_image,
|
||||
"web_image": spec.release.web_image,
|
||||
},
|
||||
"services": list(service_names(spec)),
|
||||
"replicas": {
|
||||
"api": spec.replicas.api,
|
||||
"web": spec.replicas.web,
|
||||
"worker": spec.replicas.worker,
|
||||
},
|
||||
"listen": {
|
||||
"address": spec.listen.address,
|
||||
"port": spec.listen.port,
|
||||
},
|
||||
"management": {
|
||||
"mode": "govoplan-deploy",
|
||||
"agent": "cli",
|
||||
"web_updates": False,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _updated_spec(
|
||||
current: InstallationSpec, args: argparse.Namespace
|
||||
) -> InstallationSpec:
|
||||
|
||||
@@ -0,0 +1,666 @@
|
||||
"""Deterministic Kubernetes export for the stateless multi-host profile."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from hashlib import sha256
|
||||
from pathlib import Path
|
||||
import re
|
||||
from typing import Any, Mapping
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from .model import InstallationSpec, image_is_digest_pinned
|
||||
|
||||
|
||||
_DNS_LABEL = re.compile(r"^[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?$")
|
||||
_SECRET_KEYS = (
|
||||
"MASTER_KEY_B64",
|
||||
"DATABASE_URL",
|
||||
"GOVOPLAN_DATABASE_URL_PGTOOLS",
|
||||
"REDIS_URL",
|
||||
"FILE_STORAGE_S3_ACCESS_KEY_ID",
|
||||
"FILE_STORAGE_S3_SECRET_ACCESS_KEY",
|
||||
)
|
||||
_CONFIG_KEYS = (
|
||||
"APP_ENV",
|
||||
"GOVOPLAN_INSTALL_PROFILE",
|
||||
"ENABLED_MODULES",
|
||||
"CELERY_ENABLED",
|
||||
"CELERY_QUEUES",
|
||||
"CORS_ORIGINS",
|
||||
"GOVOPLAN_TRUSTED_HOSTS",
|
||||
"FORWARDED_ALLOW_IPS",
|
||||
"AUTH_COOKIE_SECURE",
|
||||
"AUTH_COOKIE_SAMESITE",
|
||||
"GOVOPLAN_HTTP_HSTS_SECONDS",
|
||||
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS",
|
||||
"GOVOPLAN_MIGRATION_TRACK",
|
||||
"DEV_AUTO_MIGRATE_ENABLED",
|
||||
"DEV_BOOTSTRAP_ENABLED",
|
||||
"FILE_STORAGE_BACKEND",
|
||||
"FILE_STORAGE_S3_ENDPOINT_URL",
|
||||
"FILE_STORAGE_S3_REGION",
|
||||
"FILE_STORAGE_S3_BUCKET",
|
||||
"FILE_STORAGE_S3_DEPLOYMENT_MANAGED",
|
||||
"FILE_STORAGE_S3_ENDPOINT_TRUSTED",
|
||||
)
|
||||
|
||||
|
||||
def render_kubernetes(
|
||||
spec: InstallationSpec,
|
||||
environment: Mapping[str, str],
|
||||
*,
|
||||
namespace: str = "govoplan",
|
||||
secret_name: str = "govoplan-runtime",
|
||||
tls_secret_name: str = "govoplan-tls",
|
||||
ingress_class_name: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Render runtime roles only; shared state services stay externally managed."""
|
||||
|
||||
_validate_cluster_profile(spec, environment, namespace, secret_name)
|
||||
name = _resource_name(spec.installation_id)
|
||||
public_host = urlsplit(spec.public_url).hostname or "localhost"
|
||||
labels = {"app.kubernetes.io/name": "govoplan", "app.kubernetes.io/instance": name}
|
||||
config_name = f"{name}-runtime"
|
||||
service_account = f"{name}-runtime"
|
||||
config = {
|
||||
key: str(environment[key])
|
||||
for key in _CONFIG_KEYS
|
||||
if environment.get(key) is not None
|
||||
}
|
||||
config.update(
|
||||
{
|
||||
"GOVOPLAN_INSTALLATION_ID": spec.installation_id,
|
||||
"GOVOPLAN_STATE_PROFILE": "shared",
|
||||
"GOVOPLAN_RUNTIME_HEARTBEAT_SECONDS": "15",
|
||||
"GOVOPLAN_RUNTIME_STALE_AFTER_SECONDS": "60",
|
||||
"GOVOPLAN_EXPECTED_API_REPLICAS": str(spec.replicas.api),
|
||||
"GOVOPLAN_EXPECTED_WORKER_REPLICAS": str(spec.replicas.worker),
|
||||
"FILE_STORAGE_BACKEND": "s3",
|
||||
"FILE_STORAGE_S3_DEPLOYMENT_MANAGED": "false",
|
||||
"FILE_STORAGE_S3_ENDPOINT_TRUSTED": "true",
|
||||
# Trust no cross-pod proxy headers by default. Operators can bind
|
||||
# an exact ingress-controller CIDR through their ConfigMap overlay.
|
||||
"FORWARDED_ALLOW_IPS": "127.0.0.1",
|
||||
}
|
||||
)
|
||||
items: list[dict[str, Any]] = [
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Namespace",
|
||||
"metadata": {"name": namespace, "labels": labels},
|
||||
},
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "ServiceAccount",
|
||||
"metadata": {
|
||||
"name": service_account,
|
||||
"namespace": namespace,
|
||||
"labels": labels,
|
||||
},
|
||||
"automountServiceAccountToken": False,
|
||||
},
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "ConfigMap",
|
||||
"metadata": {"name": config_name, "namespace": namespace, "labels": labels},
|
||||
"data": dict(sorted(config.items())),
|
||||
},
|
||||
_deployment(
|
||||
name=f"{name}-api",
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
role="api",
|
||||
replicas=spec.replicas.api,
|
||||
image=spec.release.api_image,
|
||||
command=(
|
||||
"python",
|
||||
"-m",
|
||||
"uvicorn",
|
||||
"govoplan_core.server.app:app",
|
||||
"--host",
|
||||
"0.0.0.0",
|
||||
"--port",
|
||||
"8000",
|
||||
"--proxy-headers",
|
||||
),
|
||||
config_name=config_name,
|
||||
secret_name=secret_name,
|
||||
service_account=service_account,
|
||||
container_port=8000,
|
||||
readiness_path="/health/ready",
|
||||
liveness_path="/health",
|
||||
probe_host=public_host,
|
||||
),
|
||||
_service(
|
||||
name=f"{name}-api",
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
role="api",
|
||||
port=8000,
|
||||
),
|
||||
_deployment(
|
||||
name=f"{name}-web",
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
role="web",
|
||||
replicas=spec.replicas.web,
|
||||
image=spec.release.web_image,
|
||||
command=(),
|
||||
config_name=None,
|
||||
secret_name=None,
|
||||
service_account=service_account,
|
||||
container_port=8080,
|
||||
extra_environment={"GOVOPLAN_API_UPSTREAM": f"http://{name}-api:8000"},
|
||||
),
|
||||
_service(
|
||||
name=f"{name}-web",
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
role="web",
|
||||
port=8080,
|
||||
),
|
||||
_migration_job(
|
||||
name=name,
|
||||
release_key=_release_key(spec),
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
image=spec.release.api_image,
|
||||
config_name=config_name,
|
||||
secret_name=secret_name,
|
||||
service_account=service_account,
|
||||
),
|
||||
]
|
||||
if spec.replicas.worker:
|
||||
items.append(
|
||||
_deployment(
|
||||
name=f"{name}-worker",
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
role="worker",
|
||||
replicas=spec.replicas.worker,
|
||||
image=spec.release.api_image,
|
||||
command=(
|
||||
"python",
|
||||
"-m",
|
||||
"celery",
|
||||
"-A",
|
||||
"govoplan_core.celery_app:celery",
|
||||
"worker",
|
||||
"--queues",
|
||||
str(environment["CELERY_QUEUES"]),
|
||||
"--loglevel",
|
||||
"INFO",
|
||||
),
|
||||
config_name=config_name,
|
||||
secret_name=secret_name,
|
||||
service_account=service_account,
|
||||
)
|
||||
)
|
||||
items.append(
|
||||
_deployment(
|
||||
name=f"{name}-scheduler",
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
role="scheduler",
|
||||
replicas=1,
|
||||
image=spec.release.api_image,
|
||||
command=(
|
||||
"python",
|
||||
"-m",
|
||||
"govoplan_core.commands.fenced_run",
|
||||
"--resource",
|
||||
"scheduler:celery-beat",
|
||||
"--wait-seconds",
|
||||
"120",
|
||||
"--",
|
||||
"python",
|
||||
"-m",
|
||||
"celery",
|
||||
"-A",
|
||||
"govoplan_core.celery_app:celery",
|
||||
"beat",
|
||||
"--loglevel",
|
||||
"INFO",
|
||||
),
|
||||
config_name=config_name,
|
||||
secret_name=secret_name,
|
||||
service_account=service_account,
|
||||
)
|
||||
)
|
||||
if spec.replicas.api > 1:
|
||||
items.append(_pod_disruption_budget(f"{name}-api", namespace, labels, "api"))
|
||||
if spec.replicas.web > 1:
|
||||
items.append(_pod_disruption_budget(f"{name}-web", namespace, labels, "web"))
|
||||
items.append(
|
||||
_ingress(
|
||||
spec,
|
||||
name=name,
|
||||
namespace=namespace,
|
||||
labels=labels,
|
||||
tls_secret_name=tls_secret_name,
|
||||
ingress_class_name=ingress_class_name,
|
||||
)
|
||||
)
|
||||
return {
|
||||
"apiVersion": "v1",
|
||||
"kind": "List",
|
||||
"metadata": {
|
||||
"annotations": {
|
||||
"govoplan.add-ideas.de/profile": "stateless-shared-state",
|
||||
"govoplan.add-ideas.de/secret-contract": ",".join(_SECRET_KEYS),
|
||||
}
|
||||
},
|
||||
"items": items,
|
||||
}
|
||||
|
||||
|
||||
def kubernetes_secret_contract() -> tuple[str, ...]:
|
||||
return _SECRET_KEYS
|
||||
|
||||
|
||||
def _validate_cluster_profile(
|
||||
spec: InstallationSpec,
|
||||
environment: Mapping[str, str],
|
||||
namespace: str,
|
||||
secret_name: str,
|
||||
) -> None:
|
||||
if not _DNS_LABEL.fullmatch(namespace) or not _DNS_LABEL.fullmatch(secret_name):
|
||||
raise ValueError("Kubernetes namespace and secret names must be DNS labels")
|
||||
if spec.installation_id == "govoplan-local":
|
||||
raise ValueError(
|
||||
"Kubernetes export requires a non-default stable installation id"
|
||||
)
|
||||
if spec.components.postgres.mode != "external":
|
||||
raise ValueError("Kubernetes export requires external shared PostgreSQL")
|
||||
if spec.components.redis.mode != "external":
|
||||
raise ValueError("Kubernetes export requires external shared Redis")
|
||||
if spec.components.storage.mode != "s3":
|
||||
raise ValueError("Kubernetes export requires external shared S3 storage")
|
||||
unpinned_images = [
|
||||
label
|
||||
for label, image in (
|
||||
("release.api_image", spec.release.api_image),
|
||||
("release.web_image", spec.release.web_image),
|
||||
)
|
||||
if not image_is_digest_pinned(image)
|
||||
]
|
||||
if unpinned_images:
|
||||
raise ValueError(
|
||||
"Kubernetes export requires immutable digest-pinned release images: "
|
||||
+ ", ".join(unpinned_images)
|
||||
)
|
||||
missing = [
|
||||
key for key in (*_SECRET_KEYS, "CELERY_QUEUES") if not environment.get(key)
|
||||
]
|
||||
if missing:
|
||||
raise ValueError(
|
||||
"Kubernetes runtime contract is missing: " + ", ".join(missing)
|
||||
)
|
||||
|
||||
|
||||
def _deployment(
|
||||
*,
|
||||
name: str,
|
||||
namespace: str,
|
||||
labels: dict[str, str],
|
||||
role: str,
|
||||
replicas: int,
|
||||
image: str,
|
||||
command: tuple[str, ...],
|
||||
config_name: str | None,
|
||||
secret_name: str | None,
|
||||
service_account: str,
|
||||
container_port: int | None = None,
|
||||
readiness_path: str | None = None,
|
||||
liveness_path: str | None = None,
|
||||
probe_host: str | None = None,
|
||||
extra_environment: Mapping[str, str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
role_labels = {**labels, "app.kubernetes.io/component": role}
|
||||
environment: list[dict[str, Any]] = [
|
||||
{"name": "TMPDIR", "value": "/tmp"},
|
||||
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": role},
|
||||
{
|
||||
"name": "GOVOPLAN_NODE_ID",
|
||||
"valueFrom": {"fieldRef": {"fieldPath": "metadata.name"}},
|
||||
},
|
||||
]
|
||||
environment.extend(
|
||||
{"name": key, "value": value}
|
||||
for key, value in sorted((extra_environment or {}).items())
|
||||
)
|
||||
if secret_name:
|
||||
environment.extend(_secret_environment(secret_name))
|
||||
container: dict[str, Any] = {
|
||||
"name": role,
|
||||
"image": image,
|
||||
"imagePullPolicy": "IfNotPresent",
|
||||
"env": environment,
|
||||
"securityContext": {
|
||||
"allowPrivilegeEscalation": False,
|
||||
"capabilities": {"drop": ["ALL"]},
|
||||
"readOnlyRootFilesystem": True,
|
||||
},
|
||||
"resources": {
|
||||
"requests": {"cpu": "100m", "memory": "256Mi"},
|
||||
"limits": {"memory": "1Gi"},
|
||||
},
|
||||
}
|
||||
if command:
|
||||
container["command"] = list(command)
|
||||
if container_port is not None:
|
||||
container["ports"] = [{"name": "http", "containerPort": container_port}]
|
||||
if readiness_path:
|
||||
container["readinessProbe"] = _http_probe(
|
||||
readiness_path,
|
||||
container_port or 8000,
|
||||
host=probe_host,
|
||||
)
|
||||
if liveness_path:
|
||||
container["livenessProbe"] = _http_probe(
|
||||
liveness_path,
|
||||
container_port or 8000,
|
||||
host=probe_host,
|
||||
)
|
||||
pod_spec: dict[str, Any] = {
|
||||
"serviceAccountName": service_account,
|
||||
"automountServiceAccountToken": False,
|
||||
"securityContext": {
|
||||
"runAsNonRoot": True,
|
||||
"seccompProfile": {"type": "RuntimeDefault"},
|
||||
},
|
||||
"containers": [container],
|
||||
"volumes": [{"name": "tmp", "emptyDir": {}}],
|
||||
"topologySpreadConstraints": [
|
||||
{
|
||||
"maxSkew": 1,
|
||||
"topologyKey": "kubernetes.io/hostname",
|
||||
"whenUnsatisfiable": "ScheduleAnyway",
|
||||
"labelSelector": {"matchLabels": role_labels},
|
||||
}
|
||||
],
|
||||
}
|
||||
container["volumeMounts"] = [{"name": "tmp", "mountPath": "/tmp"}]
|
||||
if config_name:
|
||||
pod_spec["containers"][0]["envFrom"] = [{"configMapRef": {"name": config_name}}]
|
||||
if config_name and secret_name:
|
||||
pod_spec["initContainers"] = [
|
||||
{
|
||||
"name": "wait-for-database",
|
||||
"image": image,
|
||||
"imagePullPolicy": "IfNotPresent",
|
||||
"command": [
|
||||
"python",
|
||||
"-m",
|
||||
"govoplan_core.commands.wait_for_database",
|
||||
"--timeout-seconds",
|
||||
"900",
|
||||
],
|
||||
"envFrom": [{"configMapRef": {"name": config_name}}],
|
||||
"env": [
|
||||
{"name": "TMPDIR", "value": "/tmp"},
|
||||
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration-wait"},
|
||||
*_secret_environment(secret_name),
|
||||
],
|
||||
"securityContext": {
|
||||
"allowPrivilegeEscalation": False,
|
||||
"capabilities": {"drop": ["ALL"]},
|
||||
"readOnlyRootFilesystem": True,
|
||||
},
|
||||
"volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}],
|
||||
}
|
||||
]
|
||||
return {
|
||||
"apiVersion": "apps/v1",
|
||||
"kind": "Deployment",
|
||||
"metadata": {"name": name, "namespace": namespace, "labels": role_labels},
|
||||
"spec": {
|
||||
"replicas": replicas,
|
||||
"strategy": {
|
||||
"type": "RollingUpdate",
|
||||
"rollingUpdate": {"maxUnavailable": 0, "maxSurge": 1},
|
||||
},
|
||||
"selector": {"matchLabels": role_labels},
|
||||
"template": {
|
||||
"metadata": {"labels": role_labels},
|
||||
"spec": pod_spec,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _migration_job(
|
||||
*,
|
||||
name: str,
|
||||
release_key: str,
|
||||
namespace: str,
|
||||
labels: dict[str, str],
|
||||
image: str,
|
||||
config_name: str,
|
||||
secret_name: str,
|
||||
service_account: str,
|
||||
) -> dict[str, Any]:
|
||||
job_labels = {**labels, "app.kubernetes.io/component": "migration"}
|
||||
job_name = _name_with_suffix(name, f"migrate-{release_key}")
|
||||
return {
|
||||
"apiVersion": "batch/v1",
|
||||
"kind": "Job",
|
||||
"metadata": {
|
||||
"name": job_name,
|
||||
"namespace": namespace,
|
||||
"labels": job_labels,
|
||||
"annotations": {
|
||||
"govoplan.add-ideas.de/recovery-mode": "forward-recovery",
|
||||
"govoplan.add-ideas.de/backup-required": "true",
|
||||
"argocd.argoproj.io/sync-wave": "-1",
|
||||
},
|
||||
},
|
||||
"spec": {
|
||||
"backoffLimit": 1,
|
||||
"ttlSecondsAfterFinished": 86400,
|
||||
"template": {
|
||||
"metadata": {"labels": job_labels},
|
||||
"spec": {
|
||||
"restartPolicy": "Never",
|
||||
"serviceAccountName": service_account,
|
||||
"automountServiceAccountToken": False,
|
||||
"securityContext": {
|
||||
"runAsNonRoot": True,
|
||||
"seccompProfile": {"type": "RuntimeDefault"},
|
||||
},
|
||||
"containers": [
|
||||
{
|
||||
"name": "migration",
|
||||
"image": image,
|
||||
"command": [
|
||||
"python",
|
||||
"-m",
|
||||
"govoplan_core.commands.init_db",
|
||||
],
|
||||
"envFrom": [{"configMapRef": {"name": config_name}}],
|
||||
"env": [
|
||||
{"name": "TMPDIR", "value": "/tmp"},
|
||||
{"name": "GOVOPLAN_RUNTIME_ROLE", "value": "migration"},
|
||||
{
|
||||
"name": "GOVOPLAN_NODE_ID",
|
||||
"valueFrom": {
|
||||
"fieldRef": {"fieldPath": "metadata.name"}
|
||||
},
|
||||
},
|
||||
*_secret_environment(secret_name),
|
||||
],
|
||||
"securityContext": {
|
||||
"allowPrivilegeEscalation": False,
|
||||
"capabilities": {"drop": ["ALL"]},
|
||||
"readOnlyRootFilesystem": True,
|
||||
},
|
||||
"volumeMounts": [{"name": "tmp", "mountPath": "/tmp"}],
|
||||
}
|
||||
],
|
||||
"volumes": [{"name": "tmp", "emptyDir": {}}],
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _secret_environment(secret_name: str) -> list[dict[str, Any]]:
|
||||
return [
|
||||
{
|
||||
"name": key,
|
||||
"valueFrom": {
|
||||
"secretKeyRef": {"name": secret_name, "key": key, "optional": False}
|
||||
},
|
||||
}
|
||||
for key in _SECRET_KEYS
|
||||
]
|
||||
|
||||
|
||||
def _service(
|
||||
*,
|
||||
name: str,
|
||||
namespace: str,
|
||||
labels: dict[str, str],
|
||||
role: str,
|
||||
port: int,
|
||||
) -> dict[str, Any]:
|
||||
role_labels = {**labels, "app.kubernetes.io/component": role}
|
||||
return {
|
||||
"apiVersion": "v1",
|
||||
"kind": "Service",
|
||||
"metadata": {"name": name, "namespace": namespace, "labels": role_labels},
|
||||
"spec": {
|
||||
"selector": role_labels,
|
||||
"ports": [{"name": "http", "port": port, "targetPort": "http"}],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _pod_disruption_budget(
|
||||
name: str,
|
||||
namespace: str,
|
||||
labels: dict[str, str],
|
||||
role: str,
|
||||
) -> dict[str, Any]:
|
||||
role_labels = {**labels, "app.kubernetes.io/component": role}
|
||||
return {
|
||||
"apiVersion": "policy/v1",
|
||||
"kind": "PodDisruptionBudget",
|
||||
"metadata": {"name": name, "namespace": namespace, "labels": role_labels},
|
||||
"spec": {"minAvailable": 1, "selector": {"matchLabels": role_labels}},
|
||||
}
|
||||
|
||||
|
||||
def _ingress(
|
||||
spec: InstallationSpec,
|
||||
*,
|
||||
name: str,
|
||||
namespace: str,
|
||||
labels: dict[str, str],
|
||||
tls_secret_name: str,
|
||||
ingress_class_name: str | None,
|
||||
) -> dict[str, Any]:
|
||||
public = urlsplit(spec.public_url)
|
||||
ingress_spec: dict[str, Any] = {
|
||||
"rules": [
|
||||
{
|
||||
"host": public.hostname,
|
||||
"http": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "/",
|
||||
"pathType": "Prefix",
|
||||
"backend": {
|
||||
"service": {
|
||||
"name": f"{name}-web",
|
||||
"port": {"name": "http"},
|
||||
}
|
||||
},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
]
|
||||
}
|
||||
if public.scheme == "https":
|
||||
if not tls_secret_name:
|
||||
raise ValueError("HTTPS Kubernetes export requires a TLS secret name")
|
||||
ingress_spec["tls"] = [
|
||||
{"hosts": [public.hostname], "secretName": tls_secret_name}
|
||||
]
|
||||
if ingress_class_name:
|
||||
ingress_spec["ingressClassName"] = ingress_class_name
|
||||
return {
|
||||
"apiVersion": "networking.k8s.io/v1",
|
||||
"kind": "Ingress",
|
||||
"metadata": {"name": name, "namespace": namespace, "labels": labels},
|
||||
"spec": ingress_spec,
|
||||
}
|
||||
|
||||
|
||||
def _http_probe(
|
||||
path: str,
|
||||
port: int,
|
||||
*,
|
||||
host: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
request: dict[str, Any] = {"path": path, "port": port}
|
||||
if host:
|
||||
request["httpHeaders"] = [{"name": "Host", "value": host}]
|
||||
return {
|
||||
"httpGet": request,
|
||||
"initialDelaySeconds": 10,
|
||||
"periodSeconds": 10,
|
||||
"timeoutSeconds": 3,
|
||||
"failureThreshold": 6,
|
||||
}
|
||||
|
||||
|
||||
def _resource_name(installation_id: str) -> str:
|
||||
value = installation_id[:48].rstrip("-")
|
||||
if not _DNS_LABEL.fullmatch(value):
|
||||
raise ValueError("Installation id cannot be represented as a Kubernetes name")
|
||||
return value
|
||||
|
||||
|
||||
def _release_key(spec: InstallationSpec) -> str:
|
||||
manifest_hash = str(spec.release.manifest_sha256 or "").strip().lower()
|
||||
if re.fullmatch(r"[0-9a-f]{8,}", manifest_hash):
|
||||
return manifest_hash[:8]
|
||||
payload = "\0".join(
|
||||
(
|
||||
spec.release.version,
|
||||
spec.release.api_image,
|
||||
spec.release.web_image,
|
||||
)
|
||||
).encode("utf-8")
|
||||
return sha256(payload).hexdigest()[:8]
|
||||
|
||||
|
||||
def _name_with_suffix(name: str, suffix: str) -> str:
|
||||
available = 63 - len(suffix) - 1
|
||||
prefix = name[:available].rstrip("-")
|
||||
candidate = f"{prefix}-{suffix}"
|
||||
if not _DNS_LABEL.fullmatch(candidate):
|
||||
raise ValueError("Kubernetes resource name cannot include the release suffix")
|
||||
return candidate
|
||||
|
||||
|
||||
def write_secret_creation_hint(
|
||||
environment_path: Path,
|
||||
*,
|
||||
namespace: str,
|
||||
secret_name: str,
|
||||
) -> str:
|
||||
keys = " ".join(f"--from-env-file={environment_path}" for _ in (0,))
|
||||
return (
|
||||
f"kubectl -n {namespace} create secret generic {secret_name} {keys} "
|
||||
"--dry-run=client -o yaml | kubectl apply -f -"
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"kubernetes_secret_contract",
|
||||
"render_kubernetes",
|
||||
"write_secret_creation_hint",
|
||||
]
|
||||
@@ -0,0 +1,497 @@
|
||||
"""Durable deployment operation journal and bounded bundle recovery."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from hashlib import sha256
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
from .bundle import BundlePaths, atomic_write, canonical_json
|
||||
|
||||
|
||||
OPERATIONS_DIRECTORY = "operations"
|
||||
APPLIED_STATE_DIRECTORY = "applied-state"
|
||||
OPERATION_FILENAME = "operation.json"
|
||||
_OPERATION_ID = re.compile(r"^[0-9]{8}T[0-9]{6}Z-[0-9a-f]{8}$")
|
||||
_BUNDLE_FILES = (
|
||||
"installation.json",
|
||||
"secrets.env",
|
||||
"compose.json",
|
||||
"garage.toml",
|
||||
"load-balancer.cfg",
|
||||
"receipt.json",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RecoveryResult:
|
||||
operation_id: str
|
||||
action: str
|
||||
detail: str
|
||||
apply_allowed: bool
|
||||
|
||||
|
||||
class DeploymentOperationJournal:
|
||||
def __init__(self, operation_path: Path, payload: dict[str, Any]) -> None:
|
||||
self.path = operation_path
|
||||
self.payload = payload
|
||||
|
||||
@classmethod
|
||||
def begin(
|
||||
cls,
|
||||
paths: BundlePaths,
|
||||
*,
|
||||
plan: dict[str, Any],
|
||||
) -> DeploymentOperationJournal:
|
||||
operations_root = paths.root / OPERATIONS_DIRECTORY
|
||||
_private_directory(operations_root)
|
||||
operation_id = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ-") + uuid4().hex[:8]
|
||||
operation_path = operations_root / operation_id
|
||||
_private_directory(operation_path)
|
||||
snapshot_source = paths.root / APPLIED_STATE_DIRECTORY
|
||||
snapshot_available = snapshot_source.is_dir()
|
||||
if snapshot_available:
|
||||
shutil.copytree(
|
||||
snapshot_source,
|
||||
operation_path / "before",
|
||||
copy_function=_private_copy,
|
||||
)
|
||||
payload: dict[str, Any] = {
|
||||
"schema_version": 2,
|
||||
"operation_id": operation_id,
|
||||
"installation_id": plan.get("installation_id"),
|
||||
"status": "running",
|
||||
"recovery_mode": (
|
||||
"configuration_rollback" if snapshot_available else "forward_recovery"
|
||||
),
|
||||
"previous_applied_snapshot": snapshot_available,
|
||||
"migration_started": False,
|
||||
"migration_completed": False,
|
||||
"started_at": _now(),
|
||||
"updated_at": _now(),
|
||||
"completed_at": None,
|
||||
"failure_summary": None,
|
||||
"desired": plan,
|
||||
"stages": [],
|
||||
"evidence_head_sha256": None,
|
||||
}
|
||||
journal = cls(operation_path, payload)
|
||||
journal.record(
|
||||
"plan-recorded",
|
||||
"succeeded",
|
||||
{
|
||||
"previous_applied_snapshot": snapshot_available,
|
||||
"desired_sha256": sha256(canonical_json(plan)).hexdigest(),
|
||||
},
|
||||
)
|
||||
return journal
|
||||
|
||||
@property
|
||||
def operation_id(self) -> str:
|
||||
return str(self.payload["operation_id"])
|
||||
|
||||
def record(
|
||||
self,
|
||||
stage: str,
|
||||
status: str,
|
||||
evidence: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
stages = list(self.payload.get("stages") or [])
|
||||
item = {
|
||||
"sequence": len(stages) + 1,
|
||||
"stage": stage,
|
||||
"status": status,
|
||||
"at": _now(),
|
||||
"evidence": dict(evidence or {}),
|
||||
"previous_sha256": self.payload.get("evidence_head_sha256"),
|
||||
}
|
||||
item["checkpoint_sha256"] = sha256(canonical_json(item)).hexdigest()
|
||||
stages.append(item)
|
||||
self.payload["stages"] = stages
|
||||
self.payload["evidence_head_sha256"] = item["checkpoint_sha256"]
|
||||
self.payload["updated_at"] = item["at"]
|
||||
self._write()
|
||||
|
||||
def migration_started(self) -> None:
|
||||
self.payload["migration_started"] = True
|
||||
self.payload["recovery_mode"] = "forward_recovery"
|
||||
self.record(
|
||||
"migration-started",
|
||||
"running",
|
||||
{
|
||||
"rollback_boundary": (
|
||||
"Database migration has started; automatic release/configuration rollback is disabled."
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
def migration_completed(self) -> None:
|
||||
self.payload["migration_completed"] = True
|
||||
self.record("migration-completed", "succeeded")
|
||||
|
||||
def succeeded(self, paths: BundlePaths, *, receipt: dict[str, Any]) -> None:
|
||||
write_applied_state(paths)
|
||||
self.payload["status"] = "succeeded"
|
||||
self.payload["completed_at"] = _now()
|
||||
self.record(
|
||||
"deployment-verified",
|
||||
"succeeded",
|
||||
{
|
||||
"receipt_spec_sha256": receipt.get("spec_sha256"),
|
||||
"receipt_compose_sha256": receipt.get("compose_sha256"),
|
||||
},
|
||||
)
|
||||
|
||||
def failed(self, exc: BaseException) -> None:
|
||||
self.payload["status"] = "failed"
|
||||
self.payload["completed_at"] = _now()
|
||||
self.payload["failure_summary"] = f"{type(exc).__name__}: {str(exc)[:1000]}"
|
||||
self.record(
|
||||
"deployment-failed",
|
||||
"failed",
|
||||
{
|
||||
"exception_type": type(exc).__name__,
|
||||
"recovery_mode": self.payload.get("recovery_mode"),
|
||||
"failure_summary_sha256": sha256(
|
||||
str(self.payload["failure_summary"]).encode("utf-8")
|
||||
).hexdigest(),
|
||||
},
|
||||
)
|
||||
|
||||
def _write(self) -> None:
|
||||
atomic_write(
|
||||
self.path / OPERATION_FILENAME,
|
||||
canonical_json(self.payload),
|
||||
mode=0o600,
|
||||
)
|
||||
|
||||
|
||||
def write_applied_state(paths: BundlePaths) -> Path:
|
||||
target = paths.root / APPLIED_STATE_DIRECTORY
|
||||
temporary = paths.root / f".{APPLIED_STATE_DIRECTORY}.{uuid4().hex}.tmp"
|
||||
previous = paths.root / f".{APPLIED_STATE_DIRECTORY}.previous"
|
||||
_private_directory(temporary)
|
||||
manifest: dict[str, Any] = {"schema_version": 1, "captured_at": _now(), "files": {}}
|
||||
try:
|
||||
for filename in _BUNDLE_FILES:
|
||||
source = paths.root / filename
|
||||
if not source.is_file():
|
||||
manifest["files"][filename] = {"present": False}
|
||||
continue
|
||||
destination = temporary / filename
|
||||
_private_copy(source, destination)
|
||||
manifest["files"][filename] = {
|
||||
"present": True,
|
||||
"sha256": sha256(destination.read_bytes()).hexdigest(),
|
||||
}
|
||||
atomic_write(
|
||||
temporary / "manifest.json",
|
||||
canonical_json(manifest),
|
||||
mode=0o600,
|
||||
)
|
||||
if previous.exists() and not target.exists():
|
||||
previous.replace(target)
|
||||
elif previous.exists():
|
||||
shutil.rmtree(previous)
|
||||
if target.exists():
|
||||
target.replace(previous)
|
||||
try:
|
||||
temporary.replace(target)
|
||||
except BaseException:
|
||||
if previous.exists() and not target.exists():
|
||||
previous.replace(target)
|
||||
raise
|
||||
if previous.exists():
|
||||
shutil.rmtree(previous)
|
||||
finally:
|
||||
if temporary.exists():
|
||||
shutil.rmtree(temporary)
|
||||
return target
|
||||
|
||||
|
||||
def list_operations(paths: BundlePaths) -> list[dict[str, Any]]:
|
||||
root = paths.root / OPERATIONS_DIRECTORY
|
||||
if not root.is_dir():
|
||||
return []
|
||||
operations: list[dict[str, Any]] = []
|
||||
for path in sorted(root.iterdir(), reverse=True):
|
||||
if not path.is_dir() or not _OPERATION_ID.fullmatch(path.name):
|
||||
continue
|
||||
payload = _read_operation(path)
|
||||
operations.append(
|
||||
{
|
||||
"operation_id": payload.get("operation_id"),
|
||||
"status": payload.get("status"),
|
||||
"recovery_mode": payload.get("recovery_mode"),
|
||||
"migration_started": bool(payload.get("migration_started")),
|
||||
"started_at": payload.get("started_at"),
|
||||
"completed_at": payload.get("completed_at"),
|
||||
"failure_summary": payload.get("failure_summary"),
|
||||
"stage_count": len(payload.get("stages") or []),
|
||||
}
|
||||
)
|
||||
return operations
|
||||
|
||||
|
||||
def recover_operation(
|
||||
paths: BundlePaths,
|
||||
*,
|
||||
operation_id: str | None = None,
|
||||
) -> RecoveryResult:
|
||||
operation_path, payload = load_operation(paths, operation_id=operation_id)
|
||||
effective_id = str(payload["operation_id"])
|
||||
if bool(payload.get("migration_started")):
|
||||
journal = DeploymentOperationJournal(operation_path, payload)
|
||||
journal.payload["status"] = "forward_recovery_required"
|
||||
journal.payload["recovery_mode"] = "forward_recovery"
|
||||
journal.record(
|
||||
"forward-recovery-required",
|
||||
"blocked",
|
||||
{
|
||||
"reason": "migration_started",
|
||||
"configuration_restored": False,
|
||||
},
|
||||
)
|
||||
return RecoveryResult(
|
||||
operation_id=effective_id,
|
||||
action="forward_recovery",
|
||||
detail=(
|
||||
"Database migration started. The previous release/configuration was not restored; "
|
||||
"repair the current release or restore a separately verified database backup."
|
||||
),
|
||||
apply_allowed=True,
|
||||
)
|
||||
before = operation_path / "before"
|
||||
if not before.is_dir():
|
||||
return RecoveryResult(
|
||||
operation_id=effective_id,
|
||||
action="manual_intervention",
|
||||
detail="No independently captured previous applied state is available.",
|
||||
apply_allowed=False,
|
||||
)
|
||||
manifest = json.loads((before / "manifest.json").read_text(encoding="utf-8"))
|
||||
entries = _validate_snapshot(before, manifest)
|
||||
for filename, evidence in entries:
|
||||
target = paths.root / filename
|
||||
if evidence.get("present"):
|
||||
source = before / filename
|
||||
_private_copy(source, target)
|
||||
else:
|
||||
target.unlink(missing_ok=True)
|
||||
journal = DeploymentOperationJournal(operation_path, payload)
|
||||
journal.payload["status"] = "configuration_restored"
|
||||
journal.payload["recovery_mode"] = "configuration_rollback"
|
||||
journal.record(
|
||||
"configuration-restored",
|
||||
"succeeded",
|
||||
{"snapshot_manifest_verified": True},
|
||||
)
|
||||
return RecoveryResult(
|
||||
operation_id=effective_id,
|
||||
action="configuration_restored",
|
||||
detail=(
|
||||
"The previously applied bundle was restored and checksum-verified. "
|
||||
"Run apply to reconcile containers to that state."
|
||||
),
|
||||
apply_allowed=True,
|
||||
)
|
||||
|
||||
|
||||
def load_operation(
|
||||
paths: BundlePaths,
|
||||
*,
|
||||
operation_id: str | None = None,
|
||||
) -> tuple[Path, dict[str, Any]]:
|
||||
root = paths.root / OPERATIONS_DIRECTORY
|
||||
if operation_id is None:
|
||||
candidates = (
|
||||
[
|
||||
item
|
||||
for item in sorted(root.iterdir(), reverse=True)
|
||||
if item.is_dir() and _OPERATION_ID.fullmatch(item.name)
|
||||
]
|
||||
if root.is_dir()
|
||||
else []
|
||||
)
|
||||
if not candidates:
|
||||
raise ValueError("No deployment operations are recorded")
|
||||
path = candidates[0]
|
||||
else:
|
||||
if not _OPERATION_ID.fullmatch(operation_id):
|
||||
raise ValueError("Invalid deployment operation id")
|
||||
path = root / operation_id
|
||||
return path, _read_operation(path)
|
||||
|
||||
|
||||
def _read_operation(path: Path) -> dict[str, Any]:
|
||||
try:
|
||||
payload = json.loads((path / OPERATION_FILENAME).read_text(encoding="utf-8"))
|
||||
except (FileNotFoundError, json.JSONDecodeError) as exc:
|
||||
raise ValueError(f"Deployment operation is unreadable: {path.name}") from exc
|
||||
if not isinstance(payload, dict) or payload.get("operation_id") != path.name:
|
||||
raise ValueError(f"Deployment operation identity is invalid: {path.name}")
|
||||
verify_operation_chain(payload)
|
||||
return payload
|
||||
|
||||
|
||||
def verify_operation_chain(payload: dict[str, Any]) -> None:
|
||||
stages = payload.get("stages")
|
||||
if not isinstance(stages, list) or not stages:
|
||||
raise ValueError("Deployment operation has no recovery evidence chain")
|
||||
previous: str | None = None
|
||||
observed_migration_started = False
|
||||
observed_migration_completed = False
|
||||
for sequence, raw_stage in enumerate(stages, start=1):
|
||||
if not isinstance(raw_stage, dict):
|
||||
raise ValueError("Deployment recovery evidence contains an invalid stage")
|
||||
stage = dict(raw_stage)
|
||||
checkpoint = str(stage.pop("checkpoint_sha256", ""))
|
||||
if (
|
||||
stage.get("sequence") != sequence
|
||||
or stage.get("previous_sha256") != previous
|
||||
or sha256(canonical_json(stage)).hexdigest() != checkpoint
|
||||
):
|
||||
raise ValueError(
|
||||
f"Deployment recovery evidence chain failed at stage {sequence}"
|
||||
)
|
||||
previous = checkpoint
|
||||
observed_migration_started |= stage.get("stage") == "migration-started"
|
||||
observed_migration_completed |= stage.get("stage") == "migration-completed"
|
||||
if previous != payload.get("evidence_head_sha256"):
|
||||
raise ValueError("Deployment recovery evidence head does not match its chain")
|
||||
if bool(payload.get("migration_started")) != observed_migration_started:
|
||||
raise ValueError(
|
||||
"Deployment migration boundary does not match recovery evidence"
|
||||
)
|
||||
if bool(payload.get("migration_completed")) != observed_migration_completed:
|
||||
raise ValueError(
|
||||
"Deployment migration completion does not match recovery evidence"
|
||||
)
|
||||
if int(payload.get("schema_version") or 0) >= 2:
|
||||
_verify_operation_state(payload, stages)
|
||||
|
||||
|
||||
def _verify_operation_state(
|
||||
payload: dict[str, Any],
|
||||
stages: list[dict[str, Any]],
|
||||
) -> None:
|
||||
plan_evidence = stages[0].get("evidence")
|
||||
if (
|
||||
stages[0].get("stage") != "plan-recorded"
|
||||
or not isinstance(plan_evidence, dict)
|
||||
or plan_evidence.get("desired_sha256")
|
||||
!= sha256(canonical_json(payload.get("desired"))).hexdigest()
|
||||
or bool(plan_evidence.get("previous_applied_snapshot"))
|
||||
!= bool(payload.get("previous_applied_snapshot"))
|
||||
):
|
||||
raise ValueError("Deployment plan does not match recovery evidence")
|
||||
desired = payload.get("desired")
|
||||
if not isinstance(desired, dict) or desired.get("installation_id") != payload.get(
|
||||
"installation_id"
|
||||
):
|
||||
raise ValueError("Deployment installation identity does not match its plan")
|
||||
status = str(payload.get("status") or "")
|
||||
expected_last_stage = {
|
||||
"failed": "deployment-failed",
|
||||
"succeeded": "deployment-verified",
|
||||
"configuration_restored": "configuration-restored",
|
||||
"forward_recovery_required": "forward-recovery-required",
|
||||
}.get(status)
|
||||
if expected_last_stage and stages[-1].get("stage") != expected_last_stage:
|
||||
raise ValueError("Deployment status does not match recovery evidence")
|
||||
if status == "failed":
|
||||
failure_summary = str(payload.get("failure_summary") or "")
|
||||
evidence = stages[-1].get("evidence")
|
||||
if (
|
||||
not failure_summary
|
||||
or not isinstance(evidence, dict)
|
||||
or evidence.get("failure_summary_sha256")
|
||||
!= sha256(failure_summary.encode("utf-8")).hexdigest()
|
||||
):
|
||||
raise ValueError(
|
||||
"Deployment failure summary does not match recovery evidence"
|
||||
)
|
||||
if status in {"failed", "succeeded"} and not payload.get("completed_at"):
|
||||
raise ValueError("Completed deployment operation has no completion timestamp")
|
||||
migration_started = bool(payload.get("migration_started"))
|
||||
if bool(payload.get("migration_completed")) and not migration_started:
|
||||
raise ValueError("Completed migration has no recorded migration boundary")
|
||||
expected_mode = (
|
||||
"forward_recovery"
|
||||
if migration_started or not bool(payload.get("previous_applied_snapshot"))
|
||||
else "configuration_rollback"
|
||||
)
|
||||
if status == "configuration_restored":
|
||||
expected_mode = "configuration_rollback"
|
||||
if payload.get("recovery_mode") != expected_mode:
|
||||
raise ValueError("Deployment recovery mode does not match recovery evidence")
|
||||
|
||||
|
||||
def _validate_snapshot(
|
||||
before: Path,
|
||||
manifest: object,
|
||||
) -> list[tuple[str, dict[str, Any]]]:
|
||||
if not isinstance(manifest, dict) or manifest.get("schema_version") != 1:
|
||||
raise ValueError("Applied-state snapshot manifest is invalid")
|
||||
files = manifest.get("files")
|
||||
if not isinstance(files, dict) or set(files) != set(_BUNDLE_FILES):
|
||||
raise ValueError("Applied-state snapshot manifest is incomplete")
|
||||
entries: list[tuple[str, dict[str, Any]]] = []
|
||||
for filename in _BUNDLE_FILES:
|
||||
evidence = files.get(filename)
|
||||
if not isinstance(evidence, dict) or not isinstance(
|
||||
evidence.get("present"),
|
||||
bool,
|
||||
):
|
||||
raise ValueError(
|
||||
f"Applied-state snapshot evidence is invalid for {filename}"
|
||||
)
|
||||
if evidence["present"]:
|
||||
source = before / filename
|
||||
expected = str(evidence.get("sha256") or "")
|
||||
if (
|
||||
source.is_symlink()
|
||||
or not source.is_file()
|
||||
or not re.fullmatch(r"[0-9a-f]{64}", expected)
|
||||
or sha256(source.read_bytes()).hexdigest() != expected
|
||||
):
|
||||
raise ValueError(
|
||||
f"Applied-state snapshot checksum failed for {filename}"
|
||||
)
|
||||
entries.append((filename, evidence))
|
||||
return entries
|
||||
|
||||
|
||||
def _private_directory(path: Path) -> None:
|
||||
path.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
path.chmod(0o700)
|
||||
|
||||
|
||||
def _private_copy(source: Path, destination: Path) -> Path:
|
||||
source = Path(source)
|
||||
destination = Path(destination)
|
||||
if source.is_symlink() or not source.is_file():
|
||||
raise ValueError(f"Refusing to copy non-regular bundle file: {source.name}")
|
||||
atomic_write(destination, source.read_bytes(), mode=0o600)
|
||||
return destination
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(UTC).isoformat()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DeploymentOperationJournal",
|
||||
"RecoveryResult",
|
||||
"list_operations",
|
||||
"load_operation",
|
||||
"recover_operation",
|
||||
"verify_operation_chain",
|
||||
"write_applied_state",
|
||||
]
|
||||
@@ -330,6 +330,13 @@ def _manifest_catalog_metadata(manifest: ModuleManifest | None) -> dict[str, obj
|
||||
payload["dependencies"] = list(manifest.dependencies)
|
||||
if manifest.optional_dependencies:
|
||||
payload["optional_dependencies"] = list(manifest.optional_dependencies)
|
||||
if manifest.architecture is not None:
|
||||
payload["architecture"] = manifest.architecture.to_dict()
|
||||
if manifest.external_providers:
|
||||
payload["external_providers"] = [
|
||||
declaration.to_dict()
|
||||
for declaration in manifest.external_providers
|
||||
]
|
||||
if manifest.migration_spec is not None:
|
||||
payload["migration_safety"] = "requires_review"
|
||||
payload["migration_notes"] = "Module owns database migrations; review release notes and migration output before activation."
|
||||
|
||||
@@ -165,6 +165,13 @@ def manifest_catalog_entry(
|
||||
entry["dependencies"] = list(manifest.dependencies)
|
||||
if manifest.optional_dependencies:
|
||||
entry["optional_dependencies"] = list(manifest.optional_dependencies)
|
||||
if manifest.architecture is not None:
|
||||
entry["architecture"] = manifest.architecture.to_dict()
|
||||
if manifest.external_providers:
|
||||
entry["external_providers"] = [
|
||||
declaration.to_dict()
|
||||
for declaration in manifest.external_providers
|
||||
]
|
||||
if manifest.provides_interfaces:
|
||||
entry["provides_interfaces"] = [
|
||||
{"name": item.name, "version": item.version}
|
||||
|
||||
@@ -9,6 +9,7 @@ import re
|
||||
import subprocess
|
||||
import tomllib
|
||||
|
||||
from .contracts import parse_manifest_contract
|
||||
from .model import RepositorySnapshot, RepositorySpec, VersionSnapshot
|
||||
from .workspace import resolve_repo_path
|
||||
|
||||
@@ -130,12 +131,9 @@ def read_manifest_versions(path: Path) -> tuple[str, ...]:
|
||||
return ()
|
||||
versions: list[str] = []
|
||||
for manifest in sorted(src.glob("**/backend/manifest.py")):
|
||||
text = manifest.read_text(encoding="utf-8")
|
||||
match = re.search(r'(?m)^\s*version\s*=\s*["\']([^"\']+)["\']', text)
|
||||
if match is None:
|
||||
match = re.search(r'(?m)^MODULE_VERSION\s*=\s*["\']([^"\']+)["\']', text)
|
||||
if match is not None:
|
||||
versions.append(match.group(1))
|
||||
contract = parse_manifest_contract(manifest, repo_name=path.name)
|
||||
if contract is not None and contract.module_version is not None:
|
||||
versions.append(contract.module_version)
|
||||
return tuple(versions)
|
||||
|
||||
|
||||
|
||||
@@ -401,6 +401,7 @@ def manifest_shape_gate_issue(workspace: Path) -> str | None:
|
||||
str(META_ROOT / "tools" / "checks" / "check-manifest-shapes.py"),
|
||||
"--workspace-root",
|
||||
str(workspace),
|
||||
"--require-architecture",
|
||||
)
|
||||
result = run(
|
||||
command,
|
||||
|
||||
@@ -82,6 +82,7 @@ PACKAGE_MODULE_REPOS=(
|
||||
"$PARENT/govoplan-booking"
|
||||
"$PARENT/govoplan-calendar"
|
||||
"$PARENT/govoplan-campaign"
|
||||
"$PARENT/govoplan-cases"
|
||||
"$PARENT/govoplan-certificates"
|
||||
"$PARENT/govoplan-committee"
|
||||
"$PARENT/govoplan-consultation"
|
||||
@@ -90,6 +91,7 @@ PACKAGE_MODULE_REPOS=(
|
||||
"$PARENT/govoplan-dashboard"
|
||||
"$PARENT/govoplan-dataflow"
|
||||
"$PARENT/govoplan-datasources"
|
||||
"$PARENT/govoplan-decisions"
|
||||
"$PARENT/govoplan-docs"
|
||||
"$PARENT/govoplan-encryption"
|
||||
"$PARENT/govoplan-facilities"
|
||||
@@ -103,15 +105,19 @@ PACKAGE_MODULE_REPOS=(
|
||||
"$PARENT/govoplan-tickets"
|
||||
"$PARENT/govoplan-learning"
|
||||
"$PARENT/govoplan-mail"
|
||||
"$PARENT/govoplan-mandates"
|
||||
"$PARENT/govoplan-ops"
|
||||
"$PARENT/govoplan-organizations"
|
||||
"$PARENT/govoplan-parties"
|
||||
"$PARENT/govoplan-permits"
|
||||
"$PARENT/govoplan-policy"
|
||||
"$PARENT/govoplan-poll"
|
||||
"$PARENT/govoplan-portal"
|
||||
"$PARENT/govoplan-procurement"
|
||||
"$PARENT/govoplan-records"
|
||||
"$PARENT/govoplan-resources"
|
||||
"$PARENT/govoplan-risk-compliance"
|
||||
"$PARENT/govoplan-services"
|
||||
"$PARENT/govoplan-tenancy"
|
||||
"$PARENT/govoplan-transparency"
|
||||
"$PARENT/govoplan-evaluation"
|
||||
@@ -122,7 +128,6 @@ PACKAGE_MODULE_REPOS=(
|
||||
TAG_ONLY_MODULE_REPOS=(
|
||||
"$PARENT/govoplan-addresses"
|
||||
"$PARENT/govoplan-appointments"
|
||||
"$PARENT/govoplan-cases"
|
||||
"$PARENT/govoplan-projects"
|
||||
"$PARENT/govoplan-dms"
|
||||
"$PARENT/govoplan-dist-lists"
|
||||
@@ -133,7 +138,6 @@ TAG_ONLY_MODULE_REPOS=(
|
||||
"$PARENT/govoplan-ledger"
|
||||
"$PARENT/govoplan-notifications"
|
||||
"$PARENT/govoplan-payments"
|
||||
"$PARENT/govoplan-portal"
|
||||
"$PARENT/govoplan-postbox"
|
||||
"$PARENT/govoplan-reporting"
|
||||
"$PARENT/govoplan-rest"
|
||||
@@ -504,7 +508,8 @@ run_manifest_shape_gate() {
|
||||
run env PYTHONDONTWRITEBYTECODE=1 \
|
||||
"$PYTHON" \
|
||||
"$META_ROOT/tools/checks/check-manifest-shapes.py" \
|
||||
--workspace-root "$PARENT"
|
||||
--workspace-root "$PARENT" \
|
||||
--require-architecture
|
||||
}
|
||||
|
||||
run_migration_release_audit() {
|
||||
|
||||
Reference in New Issue
Block a user