[Debt] Reconcile orphaned Campaign build artifacts in shared object storage #91

New Issue

2026-08-01T01:48:58+02:00

zemion commented

2026-08-01 01:48:58 +02:00

Problem

Generated EML now uses shared object storage so any API or worker node can consume it. Normal build failures compensate objects written before database commit, but a hard process loss between object creation and metadata commit can leave an unreferenced object.

Parent scaling work: GovOPlaN/govoplan#27.

Acceptance criteria

Inventory only the Campaign-owned campaign-artifacts/ namespace with bounded pagination and tenant/build-prefix isolation.
Compare object keys with committed Campaign job references and apply a conservative grace period before an object is eligible.
Default to dry-run and report candidate key, size, age, reason, and totals without exposing keys to ordinary campaign users.
Applied cleanup is idempotent, auditable, and protected by a Core distributed lease/fencing token so competing nodes cannot commit stale cleanup.
Storage/list/delete failures remain visible and retryable; no database reference is cleared after a failed deletion.
Tests cover process-loss orphan simulation, still-referenced objects, young objects, competing scans, partial storage outage, and repeated cleanup.
Operator documentation explains the namespace, grace period, dry-run/apply flow, evidence, and coordinated database/object/key recovery boundary.

Current baseline

Object writes are compensated for handled build failures, workers verify object size/digest before delivery, and retention preserves references when deletion fails. This issue covers only the uncatchable hard-loss window and must not broaden into general Files blob cleanup.

## Problem Generated EML now uses shared object storage so any API or worker node can consume it. Normal build failures compensate objects written before database commit, but a hard process loss between object creation and metadata commit can leave an unreferenced object. Parent scaling work: GovOPlaN/govoplan#27. ## Acceptance criteria - [ ] Inventory only the Campaign-owned `campaign-artifacts/` namespace with bounded pagination and tenant/build-prefix isolation. - [ ] Compare object keys with committed Campaign job references and apply a conservative grace period before an object is eligible. - [ ] Default to dry-run and report candidate key, size, age, reason, and totals without exposing keys to ordinary campaign users. - [ ] Applied cleanup is idempotent, auditable, and protected by a Core distributed lease/fencing token so competing nodes cannot commit stale cleanup. - [ ] Storage/list/delete failures remain visible and retryable; no database reference is cleared after a failed deletion. - [ ] Tests cover process-loss orphan simulation, still-referenced objects, young objects, competing scans, partial storage outage, and repeated cleanup. - [ ] Operator documentation explains the namespace, grace period, dry-run/apply flow, evidence, and coordinated database/object/key recovery boundary. ## Current baseline Object writes are compensated for handled build failures, workers verify object size/digest before delivery, and retention preserves references when deletion fails. This issue covers only the uncatchable hard-loss window and must not broaden into general Files blob cleanup.

zemion added the

type

debt

module/campaign area/db area/security codex/ready

labels 2026-08-01 01:48:58 +02:00

zemion referenced this issue from GovOPlaN/govoplan

2026-08-01 01:49:40 +02:00

[User Story] One-command Core bootstrap, module lifecycle, scale-out, and environment promotion #13

zemion commented

2026-08-01 02:04:17 +02:00

Codex State: ready

Summary

Campaign generated EML artifacts now use shared object storage with stored size/digest verification, build-failure compensation, and reference-preserving retention failures.

Changed Files

src/govoplan_campaign/backend/sending/jobs.py
tests/test_shared_artifact_storage.py

Verification

29 affected Campaign tests passed

Next / Blocked

Implement the bounded orphan inventory reconciler for hard process loss between object upload and metadata commit.

Suggested status label: status/ready

## Codex State: ready ### Summary - Campaign generated EML artifacts now use shared object storage with stored size/digest verification, build-failure compensation, and reference-preserving retention failures. ### Changed Files - `src/govoplan_campaign/backend/sending/jobs.py` - `tests/test_shared_artifact_storage.py` ### Verification - `29 affected Campaign tests passed` ### Next / Blocked - Implement the bounded orphan inventory reconciler for hard process loss between object upload and metadata commit. Suggested status label: `status/ready`

zemion referenced this issue

2026-08-03 02:11:44 +02:00

[Debt] Adopt the Core recovery ledger for Campaign build, delivery, and retention effects #92

zemion commented

2026-08-03 09:24:12 +02:00

Codex State: done

Summary

Implemented and pushed the bounded Campaign artifact orphan reconciler.

Changed Files

Core bb84122 adds UTC modification evidence to local/S3 list and stat results. Campaign d9195a2 adds the operator-only dry-run/apply API, tenant-prefix and cursor bounds, 24-hour minimum grace period, committed EML/print reference checks, active-build protection, Core distributed fencing, idempotent replay, post-delete verification, hashed recovery/audit evidence, and operator documentation.

Verification

Campaign full backend: 364 tests and 16 subtests passed. Consolidated tools/checks/check-focused.sh passed, including dependency/manifest/architecture checks, backend and migration matrices, 46 WebUI permutations, full-product bundle budgets, and focused module UI tests.

## Codex State: done ### Summary - Implemented and pushed the bounded Campaign artifact orphan reconciler. ### Changed Files - `Core bb84122 adds UTC modification evidence to local/S3 list and stat results. Campaign d9195a2 adds the operator-only dry-run/apply API, tenant-prefix and cursor bounds, 24-hour minimum grace period, committed EML/print reference checks, active-build protection, Core distributed fencing, idempotent replay, post-delete verification, hashed recovery/audit evidence, and operator documentation.` ### Verification - `Campaign full backend: 364 tests and 16 subtests passed. Consolidated tools/checks/check-focused.sh passed, including dependency/manifest/architecture checks, backend and migration matrices, 46 WebUI permutations, full-product bundle budgets, and focused module UI tests.`

zemion closed this issue

2026-08-03 09:24:12 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan-campaign#91