Fence and reconcile module lifecycle effects

docs/DEPLOYMENT_OPERATOR_GUIDE.md

@@ -431,6 +431,14 @@ SQLite's backup API; non-SQLite databases require
 `--database-backup-command`, `--database-restore-check-command`, and
 `--database-restore-command`.
 
+Every non-dry run also owns the database-fenced
+`core:module-lifecycle:deployment` recovery operation. The run record includes
+its operation id and status. A supervised run reaches durable `succeeded` only
+after restart and health verification. `recovery_required` or `outcome_unknown`
+blocks another lifecycle mutation until the recorded operation is reconciled;
+do not bypass this by deleting `install.lock`. See
+[`MODULE_LIFECYCLE_RECOVERY.md`](MODULE_LIFECYCLE_RECOVERY.md).
+
 Database hook commands receive:
 
 - `GOVOPLAN_INSTALLER_RUN_DIR`

docs/DOCUMENTATION_MAP.md

@@ -19,6 +19,7 @@ operator, and roadmap pages.
 | Institutional context and governed references | `INSTITUTIONAL_CONTEXT_CONTRACT.md` | Shared temporal, actor/representation, institution, mandate, service, party, decision, evidence, legal-basis, information-governance, presentation, and geo DTO/provider contracts. |
 | Postbox E2EE target architecture | `POSTBOX_E2EE_ARCHITECTURE.md` | Strategic encrypted postbox/mailbox model, key ownership, role mailbox semantics, and retraction limits. |
 | Shared state, runtime coordination, and recovery | `STATE_AND_RECOVERY_CONTRACT.md` | State profiles, object storage, node registration/drain, fenced leases, migration ordering, and recovery evidence. |
+| Module lifecycle recovery | `MODULE_LIFECYCLE_RECOVERY.md` | Installer/live-graph recovery modes, deployment fence, evidence, retry blocking, and operator reconciliation. |
 
 ## Release And Operations
 

docs/MODULE_ARCHITECTURE.md

@@ -1346,6 +1346,11 @@ The package install-plan API records operator intent only:
   default; successful uninstalls are removed from saved startup state by default.
   Use `--no-activate-installed-modules` or
   `--keep-uninstalled-modules-in-desired` only for staged rollout workflows.
+- Every non-dry installer and live active-graph mutation acquires the
+  deployment-wide `core:module-lifecycle:deployment` lease and records a Core
+  recovery operation. Unresolved effects block later lifecycle changes. The
+  operation modes and operator reconciliation contract are defined in
+  `MODULE_LIFECYCLE_RECOVERY.md`.
 - `govoplan-module-installer --supervise --migrate --health-url http://127.0.0.1:8000/health --restart-command '<restart govoplan server>'`
   is the preferred disruptive-change path. It applies the plan, optionally runs
   migrations in a fresh Python process after a fresh-process manifest

docs/MODULE_LIFECYCLE_RECOVERY.md

@@ -0,0 +1,60 @@
+# Module Lifecycle Recovery
+
+Package changes and live module-graph changes use Core's durable recovery
+ledger. The local `install.lock` still prevents duplicate work in one runtime
+directory; the database lease `core:module-lifecycle:deployment` is the
+deployment-wide authority across API, installer, worker, and scheduler nodes.
+
+## Declared Boundaries
+
+| Operation | Recovery mode | Completion condition |
+| --- | --- | --- |
+| `module-lifecycle.pre-migration` | compensation | package, WebUI, manifest, and desired-graph evidence match |
+| `module-lifecycle.post-migration` | forward recovery | migration tasks, manifests, desired graph, restart, and health are verified |
+| `module-retirement.destroy-data` | snapshot restore | a hashed, restore-checked backup exists and retirement state is verified |
+| `module-runtime.apply-graph` | compensation | hooks, capability contexts, active graph, and workflow contributions match |
+
+The installer prepares the recovery operation before it captures the database
+snapshot. A full database restore therefore retains the prepared operation and
+its fence instead of erasing the fact that a mutation was attempted. Backup
+artifacts are hashed and sized before any package, migration, or retirement
+effect starts.
+
+Every command boundary records the command source and canonical hashes of the
+redacted command/result records. Credentials, database URLs, command output,
+and package-registry secrets are never copied into recovery evidence.
+
+## Failure And Retry Rules
+
+- A conclusive failure before effects is terminal `failed`.
+- A command or compensatable effect that started but did not complete is
+  `recovery_required`.
+- A lost or unexpected outcome after a migration/external boundary is
+  `outcome_unknown`.
+- A verified package/database rollback becomes `recovered`.
+- A supervised install becomes `succeeded` only after restart and all configured
+  health probes succeed.
+
+An unresolved lifecycle operation blocks every later lifecycle mutation on the
+same deployment fence, even after its execution lease is released. Operators
+must inspect the checkpoint chain and run record, restore or complete the
+declared recovery path, and explicitly reconcile the operation. A new install
+must not be used as an implicit retry.
+
+Live graph changes use the same fence. A non-migrating hook or registry failure
+restores the prior in-process graph and records verified compensation. A failure
+after migrations begin remains unresolved because restoring the process-local
+registry does not reverse database schema effects.
+
+## Operator Evidence
+
+The installer run record contains the recovery operation id, mode, plan hash,
+and current lifecycle status. The Ops recovery view is authoritative for the
+durable state and evidence-chain result. Keep both the run directory and the
+state-service backup evidence until the operation is terminal and the normal
+retention policy permits removal.
+
+Run the module installer rollback drill and recovery-runtime test matrix before
+enabling lifecycle mutation in a new deployment. Shared-state deployments must
+still use immutable release images; the ledger does not make in-place package
+mutation across replicas safe.