docs: organize cross-product documentation
This commit is contained in:
@@ -0,0 +1,89 @@
|
||||
# Recovery Ledger Adoption
|
||||
|
||||
The Core recovery ledger is a platform primitive, not automatic protection for
|
||||
module-owned effects. The canonical, machine-checked inventory is
|
||||
[`recovery-operation-inventory.json`](../recovery-operation-inventory.json).
|
||||
|
||||
## Classification Rules
|
||||
|
||||
- Use `atomic` only when every mutation commits in one database transaction and
|
||||
no external effect occurs.
|
||||
- Use `compensation` when every completed effect has a bounded, verifiable
|
||||
inverse action. A best-effort delete is not proof of compensation.
|
||||
- Use `snapshot_restore` only with fresh, signed backup evidence that covers all
|
||||
affected state services at one recovery point.
|
||||
- Use `forward_recovery` for provider acceptance, queue publication, cursor
|
||||
advancement, and other effects that may be resumable but cannot safely be
|
||||
undone.
|
||||
- Use `irreversible` for approved purge or destruction where no automated
|
||||
recovery is claimed.
|
||||
|
||||
One feature may cross more than one boundary. Module installation is
|
||||
compensatable before schema migration, forward-only after migration starts, and
|
||||
snapshot-restorable for an approved destructive retirement. Mail submission is
|
||||
forward recovery because losing the response after provider acceptance must not
|
||||
cause an automatic resend.
|
||||
|
||||
## Adoption Order
|
||||
|
||||
1. Campaign build is the reference implementation for a database plus object
|
||||
storage operation. Its operation reserves a build-specific object prefix,
|
||||
persists request and precondition evidence before writes, records the final
|
||||
object manifest, and verifies database/object state before success.
|
||||
2. Campaign delivery and Mail provider effects adopt outcome-unknown semantics
|
||||
without weakening their existing provider-specific idempotency records.
|
||||
3. Files applies the same contract to uploads, purge, integrity reconciliation,
|
||||
and writable connector synchronization.
|
||||
4. Connectors, Dataflow, and Workflow Engine consume the contract at their
|
||||
registry/capability boundaries so optional providers remain optional.
|
||||
5. Core module lifecycle uses the ledger in addition to, not instead of, signed
|
||||
deployment and backup evidence.
|
||||
|
||||
Every fenced operation uses a process incarnation and distributed lease. A
|
||||
stale process cannot append a checkpoint or report success. An expired operation
|
||||
is claimed for recovery through an explicit takeover that preserves the prior
|
||||
fence in the checkpoint chain; it is never resumed as a normal retry.
|
||||
|
||||
Connectors read-only sanctions and feed acquisitions are adopted: source
|
||||
revision/cursor and dry-run evidence are recorded before provider I/O, while
|
||||
the immutable snapshot and terminal checkpoint commit atomically. The generic
|
||||
external-mutation contract is conformance-tested but remains `planned` until a
|
||||
production connector actually publishes, updates, or deletes provider state.
|
||||
|
||||
Dataflow runs are adopted. Database-only execution uses one atomic terminal
|
||||
commit for the run projection and recovery checkpoint. Output publication uses
|
||||
forward recovery: source and output digests are checkpointed before dispatch,
|
||||
a conclusive provider result commits with the run projection, and an expired
|
||||
or failed attempt after dispatch becomes `outcome_unknown`. A stale attempt may
|
||||
be retried only when its durable boundary proves dispatch had not started.
|
||||
|
||||
Workflow Engine is adopted at both declared boundaries. Instance workers,
|
||||
trigger deliveries, and timer resumptions use process-bound distributed fences.
|
||||
Every module-action invocation records the pinned definition, input, preview,
|
||||
authority, provider-idempotency, and action-contract hashes before dispatch.
|
||||
Conclusive results commit with the Workflow projection. A lost acknowledgement,
|
||||
invalid result, or unannounced non-atomic effect becomes `outcome_unknown` and
|
||||
cannot be retried until evidence confirms either that the effect occurred or is
|
||||
absent. Linked Dataflow uncertainty blocks the Workflow without duplicating
|
||||
Dataflow's recovery authority.
|
||||
|
||||
Core module lifecycle is adopted at four boundaries. Installer recovery is
|
||||
prepared before snapshots so a full database restore preserves the attempted
|
||||
operation. Pre-migration package changes use compensation, migrated changes use
|
||||
forward recovery, destructive retirement requires a hashed and restore-checked
|
||||
snapshot, and live graph changes restore the prior registry when no migration
|
||||
ran. A deployment-wide database fence serializes these effects; any unresolved
|
||||
predecessor blocks a differently keyed retry until explicit reconciliation.
|
||||
Supervised installs become successful only after restart and health evidence is
|
||||
recorded.
|
||||
|
||||
## Operator Contract
|
||||
|
||||
Ops lists non-terminal and manual-intervention operations. Operators must verify
|
||||
the checkpoint chain before trusting evidence, distinguish `outcome_unknown`
|
||||
from rejection, and use the owning module's documented reconciliation action.
|
||||
No evidence payload may contain credentials or resolved secrets.
|
||||
|
||||
The parent adoption issue remains open until all inventory rows are adopted and
|
||||
the module matrix proves crash, retry, stale-fence, tamper, and optional-module
|
||||
behavior for each consequential path.
|
||||
Reference in New Issue
Block a user