[Security] Make security-audit tool coverage enforceable in CI #18

Closed
opened 2026-07-29 13:25:49 +02:00 by zemion · 1 comment
Owner

Problem

A host-only audit can currently finish while Semgrep, Gitleaks, Trivy, OSV Scanner, or jscpd are unavailable. The report records those skips, but a green/report-only run is not sufficient evidence that every intended scanner executed.

The July 2026 audit also produced overwhelming test-only assertion/fake-secret noise. The wrapper now excludes only B101/B105/B106/B107 and S101/S105/S106/S107 under test roots; production scans remain unchanged.

Scope

  • Make the pinned audit container the canonical CI execution environment.
  • Fail the CI audit when a required scanner is missing or skipped.
  • Keep report-only local mode available, but make incomplete coverage prominent and machine-readable.
  • Pin and periodically update scanner versions and retain/cache the built image.
  • Scan all module requirement and lock files, not only the current repository.
  • Preserve narrow test-root exclusions; do not introduce broad suppressions.

Acceptance criteria

  • CI proves that Semgrep, Bandit, Ruff security rules, Gitleaks, Trivy, OSV Scanner, pip-audit, npm audit, Xenon/Radon, and jscpd ran.
  • Missing tools make strict/CI mode fail.
  • The report manifest distinguishes no findings, findings, scanner failure, and skipped scanner.
  • Wrapper tests cover required-tool and incomplete-report behavior.
## Problem A host-only audit can currently finish while Semgrep, Gitleaks, Trivy, OSV Scanner, or jscpd are unavailable. The report records those skips, but a green/report-only run is not sufficient evidence that every intended scanner executed. The July 2026 audit also produced overwhelming test-only assertion/fake-secret noise. The wrapper now excludes only B101/B105/B106/B107 and S101/S105/S106/S107 under test roots; production scans remain unchanged. ## Scope - Make the pinned audit container the canonical CI execution environment. - Fail the CI audit when a required scanner is missing or skipped. - Keep report-only local mode available, but make incomplete coverage prominent and machine-readable. - Pin and periodically update scanner versions and retain/cache the built image. - Scan all module requirement and lock files, not only the current repository. - Preserve narrow test-root exclusions; do not introduce broad suppressions. ## Acceptance criteria - CI proves that Semgrep, Bandit, Ruff security rules, Gitleaks, Trivy, OSV Scanner, pip-audit, npm audit, Xenon/Radon, and jscpd ran. - Missing tools make strict/CI mode fail. - The report manifest distinguishes no findings, findings, scanner failure, and skipped scanner. - Wrapper tests cover required-tool and incomplete-report behavior. <!-- audit-tool-coverage-20260729 -->
Author
Owner

Codex State: done

Summary

  • Pushed f2e2eb5: CI now runs the full scanner contract across the GovOPlaN repository set.
  • manifest.json records complete/incomplete coverage and no-findings, findings, scanner-failure, or skipped for every required scanner.
  • Strict and Actions runs fail on missing tools; local report-only runs remain available but are explicitly marked incomplete.
  • Requirement and package-lock discovery is recursive with generated and vendor directories pruned; the toolbox image remains fingerprint-cached and weekly refreshed.

Verification

  • /mnt/DATA/git/govoplan/.venv/bin/python -m unittest discover -s tests: 285 passed
  • python -m unittest tests.test_security_audit_wrapper tests.test_security_audit_mount_resolver: 16 passed
  • bash -n tools/checks/check-security-audit.sh tools/checks/security-audit/run.sh
## Codex State: done ### Summary - Pushed f2e2eb5: CI now runs the full scanner contract across the GovOPlaN repository set. - manifest.json records complete/incomplete coverage and no-findings, findings, scanner-failure, or skipped for every required scanner. - Strict and Actions runs fail on missing tools; local report-only runs remain available but are explicitly marked incomplete. - Requirement and package-lock discovery is recursive with generated and vendor directories pruned; the toolbox image remains fingerprint-cached and weekly refreshed. ### Verification - `/mnt/DATA/git/govoplan/.venv/bin/python -m unittest discover -s tests: 285 passed` - `python -m unittest tests.test_security_audit_wrapper tests.test_security_audit_mount_resolver: 16 passed` - `bash -n tools/checks/check-security-audit.sh tools/checks/security-audit/run.sh`
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan#18