[Integrity review] Preserve CSV source evidence and refresh selected details #9

New Issue

2026-09-08T10:24:53+02:00

zemion commented

2026-09-08 10:24:53 +02:00

I6: retain original CSV with durable stages/materializations and existing governed retention, never transient previews. Original export must enforce current and historical visibility; source text must not leak in catalogue DTOs. P7: successful refresh/freeze can leave same-reference selected preview/history stale; invalidate details with generation guards. Preserve immutable history and bounded provider reads.

Tracking: GovOPlaN/govoplan-core#298

User authorized implementation on 2026-09-08. Work is local until explicitly published; keep this issue open until verification and publication state are recorded. Preserve tenant isolation, current authorization, immutable history, audit evidence, deterministic ordering, retry safety and existing API compatibility. Add focused regressions and owning EN/DE documentation. No live mail/provider side effects or historical data rewrites.

I6: retain original CSV with durable stages/materializations and existing governed retention, never transient previews. Original export must enforce current and historical visibility; source text must not leak in catalogue DTOs. P7: successful refresh/freeze can leave same-reference selected preview/history stale; invalidate details with generation guards. Preserve immutable history and bounded provider reads. Tracking: https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/298 User authorized implementation on 2026-09-08. Work is local until explicitly published; keep this issue open until verification and publication state are recorded. Preserve tenant isolation, current authorization, immutable history, audit evidence, deterministic ordering, retry safety and existing API compatibility. Add focused regressions and owning EN/DE documentation. No live mail/provider side effects or historical data rewrites.

zemion added the

labels 2026-09-08 10:24:53 +02:00

zemion referenced this issue from GovOPlaN/govoplan-core

2026-09-08 10:24:53 +02:00

[Integrity review] Implement 2026-09-08 data-integrity and performance findings #298

zemion commented

2026-09-08 11:51:56 +02:00

Local implementation receipt — 2026-09-08

Durable CSV stages/materializations retain bounded exact originals with projection/approval evidence; promotion/freezing verifies integrity. Frozen originals retain current and historical field/row/external policy restrictions, and corrupted originals cannot be resealed as fresh copies. Export is admin-only, audited and no-store; retention disposal removes original content. Nullable migrations preserve historical rows. Same-selection details reload after mutations; stale authority callbacks and dialogs cannot act on the new context.62 backend tests run with 2 existing skips; frontend 10 regressions and typecheck pass. EN/DE topics updated.

Verification: tools/checks/check-focused.sh completed successfully,72/72 manifest architecture checks passed, and changed-source Ruff F plus git diff --check passed across 16 repositories. Tests use isolated databases/mock providers; no live data/provider mutation, commit, push, tag, deployment or release was performed. Changes remain local and unpublished: keep this issue OPEN until publication/rollout state is recorded. Before rollout, back up and test coordinated Mail/Files/Connectors/Datasources migrations on a database copy. PostgreSQL concurrency, real provider races and representative load testing remain operational verification, not a claimed certification.

## Local implementation receipt — 2026-09-08 Durable CSV stages/materializations retain bounded exact originals with projection/approval evidence; promotion/freezing verifies integrity. Frozen originals retain current and historical field/row/external policy restrictions, and corrupted originals cannot be resealed as fresh copies. Export is admin-only, audited and no-store; retention disposal removes original content. Nullable migrations preserve historical rows. Same-selection details reload after mutations; stale authority callbacks and dialogs cannot act on the new context.62 backend tests run with 2 existing skips; frontend 10 regressions and typecheck pass. EN/DE topics updated. Verification: tools/checks/check-focused.sh completed successfully,72/72 manifest architecture checks passed, and changed-source Ruff F plus git diff --check passed across 16 repositories. Tests use isolated databases/mock providers; no live data/provider mutation, commit, push, tag, deployment or release was performed. Changes remain local and unpublished: keep this issue OPEN until publication/rollout state is recorded. Before rollout, back up and test coordinated Mail/Files/Connectors/Datasources migrations on a database copy. PostgreSQL concurrency, real provider races and representative load testing remain operational verification, not a claimed certification.

zemion commented

2026-09-08 12:57:27 +02:00

Source publication and database verification — 2026-09-08

govoplan-datasources: v0.1.26 · 9d067f1baded
govoplan-core: v0.1.46 · 6591aaa3fdaf

Remote main branches and annotated tags were verified against the exact coordinated commits. The implementation-specific tests and EN/DE documentation recorded above remain the acceptance evidence. Cross-module focused checks, manifest shapes, 63 WebUI build configurations and 230 browser tests passed. This records source publication, not a package-registry release, runtime image, website deployment or production rollout.

Database correction to the previous local-only note: the already-running development server had automatically applied Files a2b3c4d5e701, Mail b5d6e7f8091a, Connectors d2a4c6e8f0b1 and Datasources e2b8d4a0f6c3 during reloads. Their heads and expected schema/indexes were verified in the local development PostgreSQL database. A current-state backup was captured and restored into a private, isolated PostgreSQL 16 cluster; this was not a pre-upgrade backup. The supported no-op migration preserved all 281 tables and 142,287 rows by per-row SHA-256 comparison, and retained the same migration heads. Files source identity recomputation matched all 7,385 rows (2 non-null identities, zero mismatches). Both Datasources PostgreSQL concurrency tests, previously skipped without a dedicated database, passed against that isolated cluster. The test cluster was stopped. No real mail delivery, POP3 retrieval/deletion, legacy mailbox reconciliation or external connector import was initiated.

Representative production load/deadline testing and real-provider race/compatibility verification remain separate operational rollout checks; this is not security certification. Operational follow-up: GovOPlaN/govoplan-core#299.

Closing this implementation issue: its code, regression, documentation and source-publication criteria are satisfied. Operational checks above are not claimed complete.

## Source publication and database verification — 2026-09-08 - govoplan-datasources: [v0.1.26](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/src/tag/v0.1.26) · [9d067f1baded](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/commit/9d067f1baded6e65a221fbba0b46fdd953404b51) - govoplan-core: [v0.1.46](https://git.add-ideas.de/GovOPlaN/govoplan-core/src/tag/v0.1.46) · [6591aaa3fdaf](https://git.add-ideas.de/GovOPlaN/govoplan-core/commit/6591aaa3fdaf19d1900ccff326c56a0b0969d9fa) Remote main branches and annotated tags were verified against the exact coordinated commits. The implementation-specific tests and EN/DE documentation recorded above remain the acceptance evidence. Cross-module focused checks, manifest shapes, 63 WebUI build configurations and 230 browser tests passed. This records source publication, not a package-registry release, runtime image, website deployment or production rollout. Database correction to the previous local-only note: the already-running development server had automatically applied Files a2b3c4d5e701, Mail b5d6e7f8091a, Connectors d2a4c6e8f0b1 and Datasources e2b8d4a0f6c3 during reloads. Their heads and expected schema/indexes were verified in the local development PostgreSQL database. A current-state backup was captured and restored into a private, isolated PostgreSQL 16 cluster; this was not a pre-upgrade backup. The supported no-op migration preserved all 281 tables and 142,287 rows by per-row SHA-256 comparison, and retained the same migration heads. Files source identity recomputation matched all 7,385 rows (2 non-null identities, zero mismatches). Both Datasources PostgreSQL concurrency tests, previously skipped without a dedicated database, passed against that isolated cluster. The test cluster was stopped. No real mail delivery, POP3 retrieval/deletion, legacy mailbox reconciliation or external connector import was initiated. Representative production load/deadline testing and real-provider race/compatibility verification remain separate operational rollout checks; this is not security certification. Operational follow-up: https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/299. Closing this implementation issue: its code, regression, documentation and source-publication criteria are satisfied. Operational checks above are not claimed complete.

zemion closed this issue

2026-09-08 12:57:27 +02:00

zemion removed the

status

in-progress

label 2026-09-08 12:57:27 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan-datasources#9