Harden source release preparation and record verified security follow-up

docs/security/SECURITY_FOLLOWUP_2026-09-08.md

@@ -0,0 +1,219 @@
+# Security and performance follow-up — 8 September 2026
+
+This follows the [original review](SECURITY_PERFORMANCE_REVIEW_2026-09-08.md)
+and its post-release issue reconciliation. It describes new source work after
+the frozen 0.1.45 release; it does not change published tags, packages, signed
+catalogs or deployed images. Gitea remains the canonical state log.
+
+## Implemented source slices
+
+- [Core #297](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/297):
+  a shared disposable-process runner enforces wall/CPU/address-space/input/output
+  limits, bounded stderr, process-group cleanup and non-queuing per-process
+  admission. A private binary codec bounds decoding before allocating a full
+  object graph and preserves explicitly supported data types without pickle.
+  Read the owning Core `docs/BOUNDED_PROCESS_CONTRACT.md` before adding callers.
+- Connectors XLSX parsing, Templates rendering, Files ZIP/TAR inspection and
+  extraction, and Dataflow reference previews/development execution now use
+  that boundary. Existing authorization, sessions, provider credentials,
+  idempotency and persistence remain in the parent. No unprotected inline
+  fallback is used. Each module contributes static EN/DE user/admin limits and
+  operational consequences through its manifest.
+- Files snapshots authorized sources inside shared admission, validates private
+  staged members, and acknowledges each persisted member before decoding the
+  next. Numeric progress remains available. The acknowledgement is event-driven,
+  not a fixed sleep per member. Reads allocate by validated actual file size,
+  not by the configured ceiling. Failures reap children, clear private staging
+  and retain the existing transaction/blob cleanup and explicit retry behavior.
+- Dataflow's normal reference preview formerly bypassed the backend wrapper;
+  it now enters the worker too. Nested source configurations cannot collide
+  merely because subflows reuse node IDs. Combined reference-source data is
+  checked before creating further columnar copies, while individual providers
+  retain their own authorized-read bounds. Staging/production still require
+  DuckDB; this change does not replace that separate backend.
+- [Access #22](https://git.add-ideas.de/GovOPlaN/govoplan-access/issues/22):
+  current-password change, session/CSRF rotation, cross-tenant session and human
+  API-key revocation, and optional administrator-assisted recovery. Recovery
+  codes are hashed, single-use, expire after 15 minutes, require a current local
+  System owner and explicit identity verification, and recheck current account,
+  membership, tenant and issuer authority at redemption. A password change also
+  invalidates outstanding codes issued by that account for other people. Audit
+  evidence and validation/error responses do not contain passwords or codes.
+  External-provider and service-account rules remain separate.
+- The Access UI provides first-login/required change, self-service change,
+  policy-aware sign-in help, public code redemption and eligible owner issuance.
+  Core consumes an optional lazy auth-action capability rather than importing
+  Access internals. The required-action gate fails closed if its UI is missing.
+- [Workflow Engine #3](https://git.add-ideas.de/GovOPlaN/govoplan-workflow-engine/issues/3):
+  full-history lists batch pinned revisions, while new summary and bounded
+  step/event endpoints preserve authorization and explicit pagination. Existing
+  full-history responses are not silently truncated. Exact inbox total semantics
+  are retained and their counting cost is documented.
+- [Meta #55](https://git.add-ideas.de/GovOPlaN/govoplan/issues/55):
+  shared version/planning helpers recognize the existing nested developer
+  package, not invented root metadata. All tag batches enforce trusted private
+  source ownership, registered origins and clean main/upstream state. Meta
+  batches additionally require exact composition and matching Core evidence.
+  Whole-batch preflight,
+  frozen source receipts, annotated immutable tags, object-pinned atomic
+  publication and post-effect remote checks are covered with temporary local
+  repositories. Hidden Git index flags, unsafe ancestry, alternates and changed
+  Git-directory identities are rejected. Selected version/composition metadata
+  must be tracked, so ignored files cannot describe bytes absent from a tag.
+  Applicable unselected Core WebUI inputs have bounded, frozen read receipts;
+  backend-only releases do not read them. The existing local module-candidate
+  exception remains intact. The weaker legacy mutation path was removed.
+  Canonical whole-package preview and receipt-bound apply now cover Meta's
+  version preparation too. Core must already match the target. Preparation
+  requires a separate trusted checkout, explicit out-of-run confirmation and
+  unchanged source/tooling receipts; it cannot rewrite the running operator.
+  Plans place Meta after Core and explain the manual preparation/publication
+  steps instead of promising a durable self-update. Ambiguous partial writes
+  require reconciliation, without automatic rollback or retry.
+- [Runtime-image follow-up](RUNTIME_IMAGE_REMEDIATION_2026-09-08.md): eight new
+  registry-only scans cover four exact candidates on amd64 and arm64. New
+  installer specifications select the patched same-line HAProxy digest;
+  existing specifications retain their explicit image. Other candidates and
+  unresolved inventory/deployment gates remain visible, not blanket-approved.
+
+## Verification record
+
+Targeted checks include actual child execution, catastrophic regex CPU,
+aggregate memory exhaustion, TAR extension metadata, noisy output, malformed
+transport/staging data, Unicode allocation limits, cancellation/callback
+failures, descendant cleanup, rollback and explicit retries. Local mixed-owner
+composition tests completed nine real children, rejected six overlapping
+requests as busy, observed at most one unreaped child and recovered all slots.
+This is local admission evidence, not a target deployment load certification.
+
+Workflow fixtures serialize 40 different pinned revisions with five SQL reads;
+summary lists use one query for 40 ordinary rows. Exact inbox totals for
+40/400/4,000 candidates used one query, with measured local costs approximately
+0.011/0.057/0.492 seconds. These are fixture measurements, not production SLOs.
+
+The broader Core API smoke suite exposed three stale campaign assertions.
+All three failures were reproduced against the unchanged private frozen 0.1.45
+sources. Updated fixtures verify recipient-summary projection, detailed payload
+separation and explicit fenced recovery of a confirmed stopped runtime; observing
+SENDING alone must not make a claim recoverable. All 76 smoke tests then passed.
+No production Campaign behavior was changed to satisfy these tests.
+
+The final release-tool suite passed 279 tests and 68 subtests, including
+temporary local remotes and adversarial source/tag/receipt changes. Rechecking
+the whole batch before effects is deliberately conservative: its repeated
+filesystem/Git/remote work grows quadratically with batch size. It is not a
+new unattended publication path or permission to execute unreviewed source.
+
+Strict interface inventory now reports no unclassified endpoints and exact
+contextual help for all 133 high-risk controls. Seventeen password browser cases
+include actual F1 help from the restricted screen, empty workspace scopes,
+EN/DE layouts, Unicode boundaries and no credential values in help URLs.
+The initial production bundle remains within the unchanged limits (512,036
+raw bytes and 162,415 gzip bytes; 1,713 gzip bytes below its ceiling), with
+46 optional descriptors and no eager optional-module imports.
+
+The focused checker now includes the new Core process, mixed-owner admission,
+Access password, Templates and Files worker tests, the repaired campaign smoke
+cases, and browser-side auth/password transport contracts. The full focused run
+passed, including 63 production module/build permutations and all 230 browser
+cases. Its two opt-in Datasources PostgreSQL cases were skipped in that run
+and subsequently passed against the isolated real database described below.
+The final Meta preparation gate was added after that full run and verified
+with the owning release-tool suite and the focused release-gate command.
+Manifest validation passed for all 72 modules. The full focused log is
+`/mnt/DATA/tmp/govoplan-security-followup-20260908-focused.log`.
+
+The first follow-up quick audit captured an unchanged 79-repository snapshot
+in `/mnt/DATA/tmp/govoplan-security-followup-quick-20260908-7s8Sgh/`.
+All four required scanners completed, with zero missing/execution reports;
+all 168 report checksums and 163 machine-readable reports were validated.
+Gitleaks found no secrets in all 79 histories and 79 worktrees. Local Semgrep
+rules reported zero findings. Production Bandit reported 65 low and four medium
+warnings, and production Ruff retained 54 warnings. The two added Bandit
+warnings identify the new Core subprocess import and invocation: trusted
+server-owned arguments, no shell, and the documented resource/process boundary
+were reviewed; warnings remain visible. This is report-only evidence, not a
+warning-free audit or a penetration test. A final snapshot follows the
+cross-module declaration/contextual-help corrections and release-tool checks.
+
+That final audit completed on 8 September, 05:59:46–06:02:18 UTC, in
+`/mnt/DATA/tmp/govoplan-security-final-quick-20260908-vAwQIh/`. All 79 start/end
+source fingerprints were identical; all four scanners completed, all 168
+registered report checksums matched, and all 163 JSON/SARIF reports parsed.
+There were no missing reports or scanner execution errors. Semgrep and both
+Gitleaks scopes again reported zero findings. Production counts were unchanged
+from the first follow-up: Bandit 65 low/four medium and Ruff 54. Test-only
+counts were Bandit 140 low/34 medium and Ruff 136. A separate frozen scan of
+all ten changed Meta release/deployment Python files reported seven low Bandit
+and four Ruff S603 warnings, with no execution errors. Its four argv-only
+subprocess sites were reviewed; the preparation additions introduced no new
+warnings. No findings were hidden or severity-wide exceptions added.
+The audit manifest SHA-256 is
+`a997b786239cd11443cb665d5f9041a968cc38f9d49171e68bb868bf2bd73310`;
+its report-checksum list SHA-256 is
+`dc590ca5b0a4e445019a05536d410226088d67b40d61cd7657bdef4a4eae56d8`.
+
+The audit includes the eight committed feature/website source changes and the
+final uncommitted Meta source. Only this evidence document was updated after
+the source freeze ended; the final Meta commit and remote publication are
+recorded in the linked Gitea issues, not inferred from local audit completion.
+
+Fresh dependency audits are retained in
+`/mnt/DATA/tmp/govoplan-dependency-final-20260908-d24LEK/`: all four full npm
+lockfile audits (Core WebUI, Mail root/WebUI and website) report zero known
+vulnerabilities. Installed Python auditing covers 137 distributions with zero
+known vulnerabilities; 51 local GovOPlaN distributions lack PyPI advisory
+coverage. Core's 46 linked packages are likewise not claimed covered by public
+registry advisories. All 12 dependency-file hashes and the installed inventory
+were unchanged. No packages were installed or automatically fixed.
+
+Managed PostgreSQL 16.15 fixtures used private Unix sockets, synthetic roles
+and databases, no TCP listener, per-case schemas and bounded SQL/lock waits.
+Both previously skipped Datasources races passed. Twenty-one existing Access
+password HTTP tests and four additional races passed on PostgreSQL: single-use
+redemption, stale-session/password replacement, competing issuance, and issuer
+password revocation during redemption. Four release/development migration checks
+also passed for Access and Workflow, including credential preservation and
+idempotent indexes. The four races are now owning opt-in Access regressions;
+see `govoplan-access/docs/PASSWORD_RECOVERY_POSTGRES_TESTS.md`. These local
+database checks do not certify a deployment, fleet load or external recovery
+handover. With both explicit PostgreSQL test URLs enabled, the full Access suite
+passed 122 tests and 18 subtests, and the full Datasources suite passed 57 tests,
+without skips. Access retained 12 existing SQLite datetime-adapter warnings in
+its separate SQLite migration cases. Both temporary PostgreSQL fixtures were
+stopped and independently verified: no server process, private socket,
+generated schema or synthetic cluster remains. Scripts, logs and shutdown
+receipts are retained under `/home/zemion/.cache/govoplan-pg-security-20260908.RAUitg/`
+and `/home/zemion/.cache/govoplan-pg-promoted-20260908.JZcIKL/`.
+
+## Adoption and remaining gates
+
+1. `AUTH_LOCAL_PASSWORD_RECOVERY_ENABLED` remains **false** by default. The
+   existing flag is still advisory until an operator explicitly adopts and
+   enables the complete recovery policy. Confirm who verifies identity and how
+   the one-time code is handed over; automated email recovery is not enabled.
+   Test first-login, lost-password, code expiry and administrator availability
+   in the target environment before enforcement.
+2. Access migration `e9a2c5f8b1d4` adds recovery evidence; Workflow migration
+   `9e6b3f8a2c7d` adds summary-pagination indexes. Use normal backed-up upgrade
+   procedures and account for index-build cost. No manual live migration or
+   server restart was performed during this work. The user's existing devserver
+   has automatic reload, so live schema state must not be assumed unchanged.
+3. Release preparation must assign new source/package versions and require a
+   Core version containing the new worker/auth contracts in the affected module
+   metadata, including matching WebUI assets. The old immutable release must
+   not be relabelled or treated as containing these APIs.
+4. Resource limits are not an arbitrary-code, filesystem or network sandbox.
+   Admission is per API/worker process, not fleet-wide. Validate Linux/cgroup
+   memory, disk quotas, process counts, cancellation and legitimate large-file
+   workloads on the intended runtime before increasing concurrency. Core #297
+   retains this target-evidence follow-up.
+5. Meta #52 and website #9 retain runtime-image/publication/deployment holds.
+   Docker/Podman/HAProxy executables are unavailable here. Final built images,
+   binary/source inventories, ingress behavior, migration/readiness/worker
+   smoke checks and the website's target/operator authority remain outstanding.
+   Zero findings in a detected package inventory is not full image coverage.
+
+No real messages, IMAP appends, password resets, provider operations or deployment
+actions were used as test fixtures. Development tests use temporary databases,
+private temporary files, mock transports and managed test-browser servers.