From 79afb0719df2dcedec1f492530229cb2ecf3bdac Mon Sep 17 00:00:00 2001 From: Albrecht Degering Date: Sat, 1 Aug 2026 14:57:07 +0200 Subject: [PATCH] Sync wiki from project files --- Codex-Project-Index.md | 2 + Repo-docs-CONFIGURATION-PACKAGES.md | 18 ++- Repo-docs-DOCUMENTATION-MAP.md | 4 +- Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md | 158 ++++++++++++++++++++ Repo-docs-MODULE-ARCHITECTURE.md | 83 ++++++++-- Repo-docs-STATE-AND-RECOVERY-CONTRACT.md | 150 +++++++++++++++++++ 6 files changed, 400 insertions(+), 15 deletions(-) create mode 100644 Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md create mode 100644 Repo-docs-STATE-AND-RECOVERY-CONTRACT.md diff --git a/Codex-Project-Index.md b/Codex-Project-Index.md index ba854c7..8572de0 100644 --- a/Codex-Project-Index.md +++ b/Codex-Project-Index.md @@ -23,6 +23,7 @@ This page is generated from repository and product-directory project files. - [Repo-docs-GITEA-ISSUES](Repo-docs-GITEA-ISSUES) - `/mnt/DATA/git/govoplan-core/docs/GITEA_ISSUES.md` - [Repo-docs-GOVERNANCE-MODEL](Repo-docs-GOVERNANCE-MODEL) - `/mnt/DATA/git/govoplan-core/docs/GOVERNANCE_MODEL.md` - [Repo-docs-GOVOPLAN-MASTER-ROADMAP](Repo-docs-GOVOPLAN-MASTER-ROADMAP) - `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MASTER_ROADMAP.md` +- [Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT](Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/INSTITUTIONAL_CONTEXT_CONTRACT.md` - [Repo-docs-INTERFACE-ETHICS-AND-DESIGN-DOCTRINE](Repo-docs-INTERFACE-ETHICS-AND-DESIGN-DOCTRINE) - `/mnt/DATA/git/govoplan-core/docs/INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` - [Repo-docs-MODULE-ARCHITECTURE](Repo-docs-MODULE-ARCHITECTURE) - `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md` - [Repo-docs-POLICY-CONTRACTS](Repo-docs-POLICY-CONTRACTS) - `/mnt/DATA/git/govoplan-core/docs/POLICY_CONTRACTS.md` @@ -32,6 +33,7 @@ This page is generated from repository and product-directory project files. - [Repo-docs-REMOTE-WEBUI-BUNDLES](Repo-docs-REMOTE-WEBUI-BUNDLES) - `/mnt/DATA/git/govoplan-core/docs/REMOTE_WEBUI_BUNDLES.md` - [Repo-docs-SECURITY-AUDIT](Repo-docs-SECURITY-AUDIT) - `/mnt/DATA/git/govoplan-core/docs/SECURITY_AUDIT.md` - [Repo-docs-SELF-HOSTED-INSTALLABILITY](Repo-docs-SELF-HOSTED-INSTALLABILITY) - `/mnt/DATA/git/govoplan-core/docs/SELF_HOSTED_INSTALLABILITY.md` +- [Repo-docs-STATE-AND-RECOVERY-CONTRACT](Repo-docs-STATE-AND-RECOVERY-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/STATE_AND_RECOVERY_CONTRACT.md` - [Repo-docs-THEMING](Repo-docs-THEMING) - `/mnt/DATA/git/govoplan-core/docs/THEMING.md` - [Repo-docs-THROTTLING](Repo-docs-THROTTLING) - `/mnt/DATA/git/govoplan-core/docs/THROTTLING.md` - [Repo-docs-UI-UX-DECISION-LEDGER](Repo-docs-UI-UX-DECISION-LEDGER) - `/mnt/DATA/git/govoplan-core/docs/UI_UX_DECISION_LEDGER.md` diff --git a/Repo-docs-CONFIGURATION-PACKAGES.md b/Repo-docs-CONFIGURATION-PACKAGES.md index 8a2dfa5..0af87b6 100644 --- a/Repo-docs-CONFIGURATION-PACKAGES.md +++ b/Repo-docs-CONFIGURATION-PACKAGES.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/CONFIGURATION_PACKAGES.md`. > Origin: `repository`. @@ -64,14 +64,26 @@ The same signed package mechanism supports several explicitly named classes: | `reference` | Prove a bounded journey, degraded states, recovery, and target-environment acceptance. | | `product` | Provide a reusable operating capability such as governed communication, service-to-decision, procurement/contracts, or governed BI. | | `sector` | Specialize terminology, forms, rules, process baselines, controls, reports, and integrations for an institutional sector without forking modules. | -| `deployment_profile` | Declare a supported infrastructure topology, operational assumptions, health gates, and recovery evidence. | -| `integration_profile` | Declare external systems, provider bindings, source-authority modes, maturity, required health, and reconciliation behavior. | +| `deployment` | Declare a supported infrastructure topology, operational assumptions, health gates, and recovery evidence. | +| `integration` | Declare external systems, provider bindings, source-authority modes, maturity, required health, and reconciliation behavior. | Package class is metadata and validation context, not additional authority. A sector package does not become a module and cannot write another module's tables. Packages may extend other packages only through versioned fragments and must preserve provenance and parent constraints. +The contract enforces class-specific evidence. Reference packages require +target, recovery, security, operations, accessibility, privacy, and +documentation evidence. Deployment and integration packages require their +corresponding target/recovery/operations evidence, while integration packages +also name provider authority and minimum-maturity expectations. Preflight +blocks a missing, incompatible, or unhealthy provider. A derived package may +tighten parent module, capability, and provider requirements but cannot remove +or loosen them. Every non-documentation claim made by reference, deployment, or +integration packages carries a `sha256:` binding. Repository checks +recompute those hashes, while signed package verification protects the declared +manifest during transport. + ## Package Model A configuration package should be a signed, portable manifest plus module-owned diff --git a/Repo-docs-DOCUMENTATION-MAP.md b/Repo-docs-DOCUMENTATION-MAP.md index 8b863b3..0006df0 100644 --- a/Repo-docs-DOCUMENTATION-MAP.md +++ b/Repo-docs-DOCUMENTATION-MAP.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/DOCUMENTATION_MAP.md`. > Origin: `repository`. @@ -23,7 +23,9 @@ operator, and roadmap pages. | Events and audit trace context | `EVENTS_AND_AUDIT.md` | Event dispatch semantics and audit payload conventions. | | Action/effect automation layer | `ACTION_EFFECT_AUTOMATION_LAYER.md` | Action/effect contracts, consequence preview, runner semantics, and module boundary for automation. | | External references and integration maturity | `EXTERNAL_REFERENCES_AND_INTEGRATION_MATURITY.md` | Stable external identity and cumulative connector maturity; configured source authority is defined by the meta target architecture. | +| Institutional context and governed references | `INSTITUTIONAL_CONTEXT_CONTRACT.md` | Shared temporal, actor/representation, institution, mandate, service, party, decision, evidence, legal-basis, information-governance, presentation, and geo DTO/provider contracts. | | Postbox E2EE target architecture | `POSTBOX_E2EE_ARCHITECTURE.md` | Strategic encrypted postbox/mailbox model, key ownership, role mailbox semantics, and retraction limits. | +| Shared state, runtime coordination, and recovery | `STATE_AND_RECOVERY_CONTRACT.md` | State profiles, object storage, node registration/drain, fenced leases, migration ordering, and recovery evidence. | ## Release And Operations diff --git a/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md b/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md new file mode 100644 index 0000000..a6eea2e --- /dev/null +++ b/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md @@ -0,0 +1,158 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/INSTITUTIONAL_CONTEXT_CONTRACT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Institutional Context And Governed References + +GovOPlaN consequential work must retain enough context to answer who acted, +for whom, through which function, under which mandate and jurisdiction, using +which rule and evidence versions, and with which requested and observed effect. +The shared contract lives in `govoplan_core.core.institutional`. + +Core owns reference shapes and provider protocols only. It does not own shared +Mandate, Service, Party, Decision, evidence, or geography tables. Domain modules +own persistence and authorization; optional capabilities resolve the references. + +## Envelope + +`GovernedContextEnvelope` version 1 carries: + +- a tenant and `TemporalRevision` with validity, recording, supersession, and + change reason; +- the real account or service account and represented account, function, + procedure party, assignment, delegation/power, and mandate; +- institution, organization unit, function, task, mandate, jurisdiction, + service, case, party, work item, workflow, approval, decision, and record + references; +- versioned legal bases and evidence references; +- information classification, purposes, retention/holds, minimization, and + disclosure state; +- external-source authority, maturity, freshness, health, and conflict state; +- language, accessibility, channel, explanation, and availability references. + +Every institutional reference includes the owner module, tenant, stable object +identity, optional version/effective instant, and a protected display label. +Cross-tenant references are rejected. Safe serialization omits labels, +inspection URLs, formal reasoning, operative results, and conditions unless a +caller explicitly requests the protected projection. + +## Semantic Providers + +The first provider-neutral capabilities are: + +- `mandates.resolver`: resolve competence for a task/authority type at an + effective instant and return the governing Mandate definition and evidence; +- `services.definitions`: obtain versioned institutional service definitions; +- `parties.resolver`: obtain effective procedure-local parties and powers of + representation without copying Identity or Organizations subjects; and +- `decisions.registry`: record and retrieve formal Decisions under optimistic + revision control. + +The Mandate, Service, Party/representation, and Decision DTOs have strict +mapping round-trips so they can cross capability, event, package, and storage +boundaries without shared ORM models. Their lifecycle states are explicit: +Mandates distinguish draft/active/suspended/replaced/retired, Services retain +publication state, Parties retain effective representation and revocation, and +Decisions retain correction, revocation, and supersession references. + +The DTOs are a repository threshold, not a mandate to create four modules. +Independent persistence, lifecycle, security/operations behavior, release +reason, reuse, and tests are still required before extraction. + +Mandate resolution is deterministic: Core filters candidates by tenant, +effective interval, active state, task and authority type, stable +organization/function identity, jurisdiction coverage, and subject type. A +result is competent only when exactly one matching Mandate remains and it has +no unresolved conflicts. Evidence from matching definitions is deduplicated +and retained in the explanation result. `revise_mandate_definition` applies +optimistic concurrency and the allowed activation, suspension, replacement, +and retirement transitions while leaving the previous revision immutable. + +`revise_formal_decision` provides the equivalent lifecycle primitive for +formal outcomes. Every accepted transition requires a new recorded revision +and change reason, links `supersedes_ref` to the prior version, updates the +authority envelope to the new version, and records explicit correction or +revocation provenance. Terminal and backward transitions fail closed. Each +Decision also records whether responsibility was human, human-reviewed +automation, or an automated service account acting under mandate. Automation +preparation/recommendation references remain inspectable without being +mistaken for the responsible outcome. + +Procedure-party corrections use `revise_procedure_party`: the stable party +identity is retained, a new revision and reason are required, stale writes are +rejected, and revoked/expired/superseded assignments are terminal. +`revoke_party_representation` separately records when a limited power ceased +to authorize actions. This allows consuming procedures to evaluate historical +delivery or representation authority without rewriting Identity, +Organizations, or Addresses records. + +Service templates and package/tenant specializations use +`derive_service_restriction`. The derived definition retains an explicit +parent-version reference, cannot extend the parent's validity, audience, +channels, or publication ceiling, and cannot remove inherited prerequisites, +required evidence, legal bases, or bindings. This is the fail-closed semantic +rule; configuration-package signature and provenance checks remain the package +transport rule. + +`ServiceAvailabilityRequirement` represents module, capability, mandate, +policy, connector, maintenance, audience, and configuration prerequisites with +an explicit unavailable-or-hidden failure mode and explanation reference. The +optional `services.availability` evaluator returns policy-scoped boolean +assessments, reason codes, and evidence. Unknown consequential requirements +fail closed; a reference itself never grants access. + +## Service Launch + +`ServiceLaunchRequest` and `ServiceLaunchResult` define the owner-neutral +boundary between Portal entry and a case, form, or workflow runtime effect. +The request carries the exact published Service definition, exact selected +binding, tenant, acting identity, timezone-aware request time, bounded +parameters, and idempotency key. The result must retain that exact Service and +binding, a same-tenant target reference, optional same-tenant evidence, and +only a relative or credential-free HTTP(S) destination. + +`service_launch_capability(kind)` maps bindings to owner capabilities: + +- `case` -> `cases.service_launcher` +- `form` -> `forms_runtime.service_launcher` +- `workflow` -> `workflow_engine.service_launcher` + +Portal may discover and invoke those capabilities but cannot write owner +tables. The owner must revalidate its definition/binding and current +authorization, produce its normal audit/event state, and make replay after an +ambiguous response safe. If the capability is absent, the service is +explainably unavailable. URL-only entries pass through the same launch-time +availability check and destination validation. A missing form runtime must +therefore fail closed rather than create a submission without definition-aware +validation. + +## Propagation + +`PlatformEvent`, `ActionExecutionRequest`, and `AuditEvent` can carry the +envelope. Audit persistence stores only its safe projection; platform-event +outbox serialization preserves it across asynchronous delivery. A module must +not invent a parallel context dictionary when the shared fields apply. + +## First Proof + +Committee's `committee.decision_path` capability is the first bounded proof. It +requires one effective, conflict-free Mandate covering the organization unit +function, and jurisdiction, an approval reference, fact evidence, versioned legal bases, +operative result, and reasoning. It emits a reconstructable `FormalDecision`, +including requested/observed effects and information governance. If a Decision +registry is installed it persists there; Committee does not take ownership of +the generic Decision lifecycle. + +## Compatibility And Security + +- Contract version changes follow Core compatibility policy. +- Unknown tenant or reference-kind combinations fail closed. +- Datetimes that affect authority must be timezone-aware. +- Protected labels, reasoning, evidence inspection links, and source details + remain subject to the owning module's access policy. +- References do not grant access to their targets. +- Evidence and audit payloads must contain stable references/checksums, not + plaintext secrets. diff --git a/Repo-docs-MODULE-ARCHITECTURE.md b/Repo-docs-MODULE-ARCHITECTURE.md index 0c91e38..3ad593e 100644 --- a/Repo-docs-MODULE-ARCHITECTURE.md +++ b/Repo-docs-MODULE-ARCHITECTURE.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`. > Origin: `repository`. @@ -161,11 +161,10 @@ may extend the kernel by adding explicit contracts, but existing contracts must remain source-compatible through the 0.1.x split line unless a migration shim and deprecation note are provided. -### Architecture Metadata Target +### Architecture Metadata -`ModuleManifest` currently describes executable composition. It does not yet -declare the complete product-portfolio meaning of a module. A backward- -compatible manifest extension should add validated architecture metadata for: +`ModuleManifest.architecture` is the backward-compatible, versioned product- +portfolio declaration for: - module kind and institutional architecture layer; - evidence-backed maturity claim (`concept`, `scaffold`, `vertical_slice`, @@ -176,9 +175,14 @@ compatible manifest extension should add validated architecture metadata for: - migration, upgrade, recovery, security, operations, and documentation evidence references. -Core should validate shape and project it through platform metadata. The meta -repository and release tooling should verify evidence and cross-repository -consistency. Docs and Ops may display the result. A module cannot make itself +Core validates the claim and all provider references during registry startup. +`reference_ready`, `supported`, and `lts` claims require a named reference +package and the cumulative evidence set; target-tested providers additionally +require provider evidence. A supported module with migrations must include +migration evidence. Signed release catalogs retain and revalidate the +declaration. The meta manifest check validates repository evidence paths and +provides an opt-in `--require-architecture` rollout gate. Platform metadata, +Docs, and Ops project the same declaration. A module cannot make itself supported solely by changing its maturity string. Known access-related capability names are defined in @@ -255,14 +259,35 @@ also state whether GovOPlaN is native authoritative, the external system is authoritative, GovOPlaN keeps a mirror, both sides use governed sync, GovOPlaN adds only a governance overlay, or the object is link-only. -A future provider declaration should compose existing contracts rather than -replace them. It will describe owned object/field groups, authority modes, +`ModuleManifest.external_providers` composes existing contracts rather than +replacing them. Each declaration describes owned object/field groups, authority modes, operations, revisions, freshness, health, limits, idempotency, conflicts, outcome-unknown handling, evidence, correction/compensation, reconciliation, outage behavior, classification, purpose, retention, and secret requirements. Core owns the typed declaration and validation. Connectors and domain modules own the actual protocol and domain behavior; configuration packages select the -effective mode; Docs and Ops explain the result. +effective mode and block missing/incompatible/unhealthy providers; Docs and Ops +explain the result. Effect-capable declarations fail validation unless their +retry, concurrency, outcome-unknown, correction, reconciliation, evidence, +audit, timeout, outage, classification, purpose, retention, and secret behavior +is explicit. + +Declarations are release-time capability claims. Configured state is projected +separately through `ModuleManifest.external_provider_state_providers`. A state +provider receives a bounded tenant context and returns one sanitized observation +per configured binding: stable binding reference, effective authority mode, +active/configured state, health, freshness, conflict, recovery readiness, +observation/last-success time, and scalar metrics. Core validates and aggregates +those observations, isolates provider failures, and never accepts URLs, +credentials, or arbitrary nested metadata as runtime-state fields. Docs removes +binding-level detail from ordinary-user projections; Ops may show the full +sanitized operator projection. + +Configuration-package preflight selects the exact requested binding from this +runtime state before evaluating authority, health, freshness, and recovery. A +healthy sibling binding therefore cannot mask an unhealthy required binding. +Providers with multiple configurations must use non-secret, stable references +such as `calendar:sync-source:`. Current named interfaces, generated from the source manifests by the workspace contract checks, are: @@ -677,6 +702,39 @@ Rules: one migration run, and do not switch a database between tracks unless it is a disposable development database. +### Shared State And Runtime Ordering + +Multi-host application roles use the `shared` state profile. In that profile, +PostgreSQL, Redis, a stable installation identifier, and S3-compatible object +storage are mandatory. Module durable artifacts must use Core's object-storage +contract and module-owned opaque key namespaces; node-local paths are limited +to temporary materialization. Same-host replicas may use the `host-shared` +profile and one shared volume. + +Only the migration command mutates schema. PostgreSQL migration runs acquire a +deployment-wide advisory lock before module pre-tasks, Alembic, and post-tasks. +API, worker, and scheduler roles wait for exact configured migration heads and +fail closed instead of applying migrations during startup. + +Runtime roles register identity, software/module composition, queues, heartbeat, +and drain state in PostgreSQL. Singleton work must use a distributed lease and +validate its monotonically increasing fencing token at the consequential +commit. See `STATE_AND_RECOVERY_CONTRACT.md` for the complete contract. + +### Recovery Evidence + +Operations spanning transactions, object storage, queues, or external systems +must choose an explicit Core recovery mode: atomic, compensation, +snapshot-restore, forward-recovery, or irreversible. Plans require verification +steps and mode-specific recovery material. Use idempotency keys, append-only +evidence checkpoints, and a runtime fence where work may race across nodes. + +The recovery ledger is a shared primitive, not automatic coverage. A module may +claim its guarantees only after its operation records preconditions before side +effects, transitions partial/unknown outcomes honestly, and records verified +completion or recovery. Plaintext secrets must never enter recovery metadata or +evidence. + ## Install, Uninstall, And Catalogs Core owns the install plan, signed catalog validation, license entitlement @@ -1303,6 +1361,9 @@ the same restart/health set after restoring package and database snapshots. The installer preflight is intentionally conservative: - maintenance mode must be active; +- the `shared` state profile blocks in-place package mutation; clustered + installations must roll one verified immutable module composition across all + replicas; - installed module manifests must be compatible with the supported manifest contract and current core version; - uninstalling `tenancy`, `access`, or `admin` is blocked; diff --git a/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md b/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md new file mode 100644 index 0000000..bd336b4 --- /dev/null +++ b/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md @@ -0,0 +1,150 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/STATE_AND_RECOVERY_CONTRACT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# State And Recovery Contract + +## State Profiles + +Core accepts three runtime state profiles: + +| Profile | Runtime placement | Durable storage | +| --- | --- | --- | +| `local` | One development process set | Local filesystem is permitted. | +| `host-shared` | Replicas on one host | One host-shared volume plus PostgreSQL and Redis. | +| `shared` | Independent hosts | PostgreSQL, Redis, and S3-compatible object storage. | + +All replicas in one installation use one stable +`GOVOPLAN_INSTALLATION_ID`, one immutable module composition, and identical +database, broker, encryption-key, and object-storage bindings. Core rejects +replicas with the `local` profile and rejects `shared` without PostgreSQL, +Redis, S3, and a non-default installation identifier. + +`FILE_STORAGE_S3_ENDPOINT_TRUSTED=true` is an explicit deployment trust +declaration for a clean HTTPS S3 origin. It does not authorize a +user-controlled connector endpoint and it is separate from installer-managed +Garage's exact endpoint trust. + +## Object Storage + +`govoplan_core.core.object_storage` is the shared backend contract for durable +module artifacts. It provides bounded read/write/list/stat/delete operations +for local and S3-compatible storage. Modules own their object-key namespace and +business metadata; Core does not interpret module files. + +Rules for modules: + +- Store only opaque object keys in business records, never local absolute + paths. +- Use node-local directories only for temporary materialization. +- Verify expected size and digest before consuming consequential artifacts. +- If object creation precedes database commit, compensate successfully created + objects on failure. +- If object deletion fails, retain the database reference and report a retryable + failure rather than claiming deletion. +- Define an orphan-inventory strategy for hard process loss between object + creation and metadata commit. + +The Files module delegates its backend implementation to this Core contract. +Campaign generated EML artifacts use a Campaign-owned object prefix and are +read by workers through the same shared backend. + +## Runtime Nodes And Leases + +API and worker incarnations register in `core_runtime_nodes` with role, +software version, module-composition hash, queues, start time, and heartbeat. +The registration identity includes a process incarnation so a stale process +cannot update a replacement's row. + +Drain is durable operator intent: + +- an API enters not-ready state after observing drain; +- a worker cancels queue consumers after observing drain; +- cancellation returns an eligible draining node to active state; +- clean shutdown marks the matching incarnation stopped. + +Coordination loss also fails closed. An API reports +`coordination_unavailable` from `/health/ready` until a heartbeat succeeds +again. A worker cancels its local queue consumers on any heartbeat or database +failure and only resumes them after its existing incarnation heartbeats +successfully. It never re-registers from the heartbeat path, so a stale worker +cannot reclaim a node identity from its replacement. + +`core_distributed_leases` provides installation/resource uniqueness, expiry, +holder incarnation, and monotonically increasing fencing tokens. Lease expiry +does not make an old process harmless by itself: code performing an effect must +assert its exact fence before commit. `govoplan_core.commands.fenced_run` +renews a lease around a subprocess and terminates the child when the lease is +lost. The deployment profiles use it for the singleton scheduler. + +## Migration Ordering + +Only `govoplan_core.commands.init_db` mutates schema. On PostgreSQL it acquires +a deterministic installation/track advisory lock before pre-migration tasks, +Alembic, and post-migration tasks. The lock is session-scoped and therefore +released if the migration process dies. + +Runtime roles run `govoplan_core.commands.wait_for_database`. It waits until +the database has exactly the configured Core/module Alembic heads and never +upgrades schema. This permits a migration Job and runtime Deployments to be +submitted together while keeping startup fail-closed. + +## Recovery Ledger + +`govoplan_core.core.recovery` provides a durable operation and evidence +contract. Recovery modes are: + +- `atomic`: one database transaction, no external effect; +- `compensation`: explicit inverse actions; +- `snapshot_restore`: separately verified backup reference; +- `forward_recovery`: repair/resume the current version; +- `irreversible`: explicit approval, no automated recovery claim. + +Every plan requires verification steps. Mode-specific evidence is mandatory. +Operations bind an idempotency key to a canonical request hash, may bind a +runtime fencing token, and emit append-only hash-chained checkpoints. Backup +and approval references are part of the hashed plan evidence. Every low-level +state transition and checkpoint append revalidates the operation's recorded +fence while holding the operation row lock. Plaintext secrets are rejected +from metadata and evidence. + +The state machine makes partial and uncertain outcomes visible. A non-atomic +running operation cannot transition directly to ordinary failure, and success +or recovery requires explicit verified checks. Ops projects states requiring +attention, but module behavior gains this guarantee only after it adopts the +ledger around its own side effects. + +## Recovery Boundary + +Application/configuration rollback and database rollback are not equivalent. +Once an incompatible migration starts, old code may be unsafe even if its image +is available. Deployment automation must switch to forward recovery unless a +coordinated and verified database/object/key backup is restored. + +Core does not create production database backups. The deployment owner must +provide backup, retention, encryption, restore verification, and recovery-point +coordination for PostgreSQL, object storage, and encryption keys. The canonical +operator procedure is documented in +`govoplan/docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md`. + +## Verification + +Focused contracts are covered by: + +```sh +/mnt/DATA/git/govoplan/.venv/bin/python -m pytest -q \ + tests/test_object_storage.py \ + tests/test_runtime_coordination.py \ + tests/test_runtime_agents.py \ + tests/test_fenced_run.py \ + tests/test_migration_lock.py \ + tests/test_wait_for_database.py \ + tests/test_recovery_guarantees.py +``` + +Production acceptance additionally requires multi-node failure and coordinated +restore drills against the actual PostgreSQL, Redis, object-store, ingress, and +secret-provider topology.