docs: organize cross-product documentation

docs/operations/SECURITY_AUDIT.md

@@ -0,0 +1,218 @@
+# Security Audit Toolchain
+
+GovOPlaN uses a free/open-source-first audit toolchain that can run locally,
+inside a container, and in Gitea Actions.
+
+## Tools
+
+- Semgrep: multi-language SAST, with GovOPlaN-specific local rules plus
+  explicit public registry rulesets in CI/full runs.
+- Bandit: Python AST security checks.
+- Ruff `S` rules: fast flake8-bandit-compatible Python security linting.
+- Gitleaks: committed-secret scanning.
+- Trivy: filesystem dependency, secret, and misconfiguration scanning.
+- pip-audit and npm audit: package vulnerability scanning from dependency
+  manifests/locks.
+- OSV-Scanner: recursive dependency vulnerability scan in full mode.
+- jscpd: duplicated-code reports in full mode.
+- Radon/Xenon: Python complexity reports and thresholds in full mode.
+
+## Local Usage
+
+Build and run the toolbox from this repository:
+
+```bash
+cd /mnt/DATA/git/govoplan
+tools/checks/security-audit/run.sh --mode ci --scope current
+```
+
+Scan all sibling GovOPlaN repositories under `/mnt/DATA/git`:
+
+```bash
+cd /mnt/DATA/git/govoplan
+tools/checks/security-audit/run.sh --mode full --scope govoplan
+```
+
+When invoked from a Flatpak development environment without a sandbox-local
+Docker CLI, the wrapper automatically uses `flatpak-spawn --host docker`. The
+host account must still be allowed to open the Docker daemon socket. For a
+conventional rootful installation this commonly means membership in the
+`docker` group followed by a complete logout/login; that membership is
+root-equivalent, so rootless Docker is preferable where the deployment policy
+requires a smaller privilege boundary.
+
+Reports are written to `audit-reports/`, which is intentionally ignored by git.
+Each run records tool versions, report checksums, and start/end repository
+revision plus worktree fingerprints. A repository change during scanning makes
+the run fail so a mixed code snapshot cannot be reported as a valid audit.
+Step exit codes are recorded separately from findings: report-only mode may
+accept findings, but scanner execution errors and malformed JSON/SARIF reports
+always fail the run. The manifest lists the expected, present, and missing
+reports for that invocation. Validation and checksums use that explicit set, so
+reusing a report directory cannot make stale output look like part of a new run.
+It also contains `coverage_status` and structured `scanner_coverage` entries.
+Every required scanner is recorded as `no-findings`, `findings`,
+`scanner-failure`, or `skipped`; this makes an incomplete local run visible
+without treating it as a clean audit.
+
+The wrapper tags the toolbox image by a fingerprint of the Dockerfile,
+`requirements-audit.txt`, and the Semgrep smoke-test inputs. If those inputs have
+not changed, subsequent runs reuse the existing local image instead of
+reinstalling all tools. The stable alias is `govoplan/security-audit:local`
+unless `SECURITY_AUDIT_IMAGE` is set.
+
+Semgrep is installed separately in the toolbox image because current Semgrep
+packages pin affected Click and MCP versions. The image upgrades both after
+Semgrep installation, runs an actual local-rule scan, and audits the final
+toolbox Python environment during the image build. The compatibility releases
+are pinned exactly to keep the tested override reproducible. Remove either
+compatibility override only after Semgrep's own dependency bounds include a
+fixed version.
+
+Force a cached rebuild:
+
+```bash
+tools/checks/security-audit/run.sh --mode ci --scope current --rebuild
+```
+
+Refresh from upstream base images and package ranges:
+
+```bash
+tools/checks/security-audit/run.sh --mode ci --scope current --update
+```
+
+Build or refresh the toolbox without running an audit:
+
+```bash
+tools/checks/security-audit/run.sh --mode quick --scope current --build-only
+tools/checks/security-audit/run.sh --mode quick --scope current --update --build-only
+```
+
+## Modes
+
+- `quick`: local Semgrep rules, Bandit, Ruff security rules, Gitleaks.
+- `ci`: quick plus Semgrep public registry rulesets, Trivy, pip-audit, npm audit.
+- `full`: ci plus OSV-Scanner, jscpd, Radon, and Xenon.
+
+The Gitea workflow uses `full` mode so its coverage contract includes every
+scanner above. Missing scanners fail strict runs and all Actions runs, even
+while actual findings remain report-only. Local report-only runs may finish
+with missing tools for diagnostics, but their manifest is marked
+`coverage_status: incomplete`.
+
+Semgrep and Trivy are invoked with finding-sensitive exit codes. Their exit 1
+is therefore a finding under the wrapper contract; higher exit codes, missing
+output, invalid JSON/SARIF, and scanner error payloads are execution failures.
+
+Bandit and Ruff security reports are split by source kind. Production code under
+`src/` is written to `bandit.json` and `ruff-security.json` and controls strict
+mode. Test code under `tests/` is still scanned for visibility, but its findings
+are written separately to `bandit-tests.json` and `ruff-security-tests.json` so
+fixture passwords, assertions, and temporary paths do not hide the production
+baseline.
+
+## Gating
+
+The Gitea workflow currently runs findings in report-only mode:
+
+```bash
+SECURITY_AUDIT_FAIL_ON_FINDINGS=0
+```
+
+This avoids blocking every push while the first baseline is reviewed. After the
+baseline is clean, switch the workflow to:
+
+```bash
+SECURITY_AUDIT_FAIL_ON_FINDINGS=1
+```
+
+or run locally with:
+
+```bash
+tools/checks/security-audit/run.sh --mode full --scope govoplan --strict
+```
+
+## Audit Burndown Workflow
+
+Treat Gitea issues as the active audit state. A full GovOPlaN audit should
+produce one tracker issue in `GovOPlaN/govoplan` and child issues in the
+repository that owns each fix.
+
+Use the tracker issue for:
+
+- report path, timestamp, mode, and scope
+- scanner counts by category
+- clean scanners and resolved findings
+- links to child issues
+- the next audit run target
+
+Use child issues for concrete code or configuration changes. Apply
+`source/security-audit` to every issue created from a report, then add the
+most specific audit label:
+
+- `audit/quick-fix`: narrow direct remediation
+- `audit/structural`: behavior or architecture needs review
+- `audit/complexity`: Radon/Xenon maintainability finding
+- `audit/duplication`: jscpd duplication finding
+- `audit/false-positive`: reviewed narrow false positive or accepted risk
+- `audit/needs-design`: human decision needed before implementation
+
+Keep active implementation status in issues instead of committing generated
+audit reports. `audit-reports/` is ignored; quote the report directory and the
+important scanner counts in the tracker issue.
+
+The jscpd step is intentionally scoped to application and test source. It
+excludes documentation snippets, package manifests, generated translations,
+public SVG assets and catalog output, workflow YAML, declarative backend schema
+JSON, the generated migration baseline, and mirrored development migration
+directories because those reports produce metadata or generated-source
+repetition rather than actionable source duplication. Keep exclusions narrow
+and create child issues for source-code clusters that cross module ownership or
+make behavior harder to change safely.
+
+The 2026-08-02 full-workspace baseline covered 64 repositories and reported
+1.82% duplicated lines before those generated-source exclusions. The reviewed
+high-value clusters were catalog acceptance persistence, release publication
+result assembly, and local WebUI JSON mutation wrappers. Similar Dataflow and
+Workflow graph/governance code remains independently owned until its shared
+contract is stable enough for Core; a raw similarity score is not grounds for a
+module-to-module dependency.
+
+## Image Freshness
+
+The regular `Security Audit` workflow reuses the fingerprinted toolbox image
+when the Docker daemon is persistent, which is the normal case for the
+self-hosted Gitea runner using the host Docker socket. Trusted push, schedule,
+and manual runs scan all registered repositories; authenticated SSH is used
+only for the private website repository. The wrapper inspects the Actions job
+mount table and forwards only the narrowest writable mount covering the audit
+scope; it never inherits the job's Docker socket or unrelated runner mounts.
+Pull-request audit runs stay disabled while the audit runner exposes its host
+Docker socket: PR-controlled audit code must run on a disposable or rootless
+runner without host-socket access. The separate
+`Security Audit Toolbox Update` workflow runs weekly with
+`SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the
+allowed tool version ranges into a refreshed local image.
+
+## Direct Host Usage
+
+The container is the recommended path. For direct host usage, install the Python
+tools first:
+
+```bash
+cd /mnt/DATA/git/govoplan
+python -m venv .venv
+./.venv/bin/python -m pip install -r requirements-audit.txt
+./.venv/bin/python -m pip install 'semgrep>=1.140,<2'
+./.venv/bin/python -m pip install --upgrade --no-deps 'click==8.3.3'
+./.venv/bin/python -m pip install --upgrade --no-deps 'mcp==1.28.1'
+./.venv/bin/semgrep scan --metrics=off --config tools/checks/security-audit/semgrep-govoplan.yml tools/checks/check-version-alignment.py
+./.venv/bin/pip-audit --progress-spinner off
+```
+
+Then install the non-Python tools (`gitleaks`, `trivy`, `osv-scanner`, `jscpd`)
+through the host package manager or vendor instructions and run:
+
+```bash
+tools/checks/check-security-audit.sh --mode quick --scope current
+```