[Debt] Adopt the Core recovery ledger for production Dataflow execution outputs #19

New Issue

2026-08-03T02:11:45+02:00

zemion commented

2026-08-03 02:11:45 +02:00

Parent: GovOPlaN/govoplan#36.

Objective

Bind production runs that publish datasets, files, messages, or connector mutations to durable recovery operations.

Acceptance criteria

Each run and externally visible output has an idempotency key, canonical input/graph hash, explicit recovery mode, and verification plan.
Scheduled and event-driven runs use lease fencing across worker replicas.
Intermediate datasets and checkpoints identify resumable, compensatable, and outcome-unknown boundaries.
Retry, crash, stale-fence, tamper, and optional-adapter conformance tests are present.
Dataflow and Ops surfaces explain unresolved and recoverable runs.

Parent: GovOPlaN/govoplan#36. ## Objective Bind production runs that publish datasets, files, messages, or connector mutations to durable recovery operations. ## Acceptance criteria - [ ] Each run and externally visible output has an idempotency key, canonical input/graph hash, explicit recovery mode, and verification plan. - [ ] Scheduled and event-driven runs use lease fencing across worker replicas. - [ ] Intermediate datasets and checkpoints identify resumable, compensatable, and outcome-unknown boundaries. - [ ] Retry, crash, stale-fence, tamper, and optional-adapter conformance tests are present. - [ ] Dataflow and Ops surfaces explain unresolved and recoverable runs.

zemion added the module/dataflow

area/security codex/ready

priority

p2

labels 2026-08-03 02:11:45 +02:00

zemion added

and removed

labels 2026-08-03 05:44:54 +02:00

zemion commented

2026-08-03 06:12:01 +02:00

Codex State: done

Summary

Adopted Core recovery ledger and distributed fencing for database-only and published-output runs.
Pre-dispatch stale attempts recover safely; post-dispatch failures and expired leases become outcome_unknown and cannot retry blindly.
Dataflow and Ops now expose recovery status, reconciliation guidance, and attention state.

Changed Files

src/govoplan_dataflow/backend/recovery.py
src/govoplan_dataflow/backend/run_worker.py
docs/DURABLE_RUN_RECOVERY.md

Verification

pytest: 74 passed, 22 subtests passed
WebUI typecheck and structure checks passed
Core recovery runtime: 11 passed; manifest registry: 65/65

## Codex State: done ### Summary - Adopted Core recovery ledger and distributed fencing for database-only and published-output runs. - Pre-dispatch stale attempts recover safely; post-dispatch failures and expired leases become outcome_unknown and cannot retry blindly. - Dataflow and Ops now expose recovery status, reconciliation guidance, and attention state. ### Changed Files - `src/govoplan_dataflow/backend/recovery.py` - `src/govoplan_dataflow/backend/run_worker.py` - `docs/DURABLE_RUN_RECOVERY.md` ### Verification - `pytest: 74 passed, 22 subtests passed` - `WebUI typecheck and structure checks passed` - `Core recovery runtime: 11 passed; manifest registry: 65/65`

zemion closed this issue

2026-08-03 06:12:01 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan-dataflow#19