199 lines
8.0 KiB
Markdown
199 lines
8.0 KiB
Markdown
# Security Audit Toolchain
|
|
|
|
GovOPlaN uses a free/open-source-first audit toolchain that can run locally,
|
|
inside a container, and in Gitea Actions.
|
|
|
|
## Tools
|
|
|
|
- Semgrep: multi-language SAST, with GovOPlaN-specific local rules plus
|
|
explicit public registry rulesets in CI/full runs.
|
|
- Bandit: Python AST security checks.
|
|
- Ruff `S` rules: fast flake8-bandit-compatible Python security linting.
|
|
- Gitleaks: committed-secret scanning.
|
|
- Trivy: filesystem dependency, secret, and misconfiguration scanning.
|
|
- pip-audit and npm audit: package vulnerability scanning from dependency
|
|
manifests/locks.
|
|
- OSV-Scanner: recursive dependency vulnerability scan in full mode.
|
|
- jscpd: duplicated-code reports in full mode.
|
|
- Radon/Xenon: Python complexity reports and thresholds in full mode.
|
|
|
|
## Local Usage
|
|
|
|
Build and run the toolbox from this repository:
|
|
|
|
```bash
|
|
cd /mnt/DATA/git/govoplan
|
|
tools/checks/security-audit/run.sh --mode ci --scope current
|
|
```
|
|
|
|
Scan all sibling GovOPlaN repositories under `/mnt/DATA/git`:
|
|
|
|
```bash
|
|
cd /mnt/DATA/git/govoplan
|
|
tools/checks/security-audit/run.sh --mode full --scope govoplan
|
|
```
|
|
|
|
When invoked from a Flatpak development environment without a sandbox-local
|
|
Docker CLI, the wrapper automatically uses `flatpak-spawn --host docker`. The
|
|
host account must still be allowed to open the Docker daemon socket. For a
|
|
conventional rootful installation this commonly means membership in the
|
|
`docker` group followed by a complete logout/login; that membership is
|
|
root-equivalent, so rootless Docker is preferable where the deployment policy
|
|
requires a smaller privilege boundary.
|
|
|
|
Reports are written to `audit-reports/`, which is intentionally ignored by git.
|
|
Each run records tool versions, report checksums, and start/end repository
|
|
revision plus worktree fingerprints. A repository change during scanning makes
|
|
the run fail so a mixed code snapshot cannot be reported as a valid audit.
|
|
Step exit codes are recorded separately from findings: report-only mode may
|
|
accept findings, but scanner execution errors and malformed JSON/SARIF reports
|
|
always fail the run. The manifest lists the expected, present, and missing
|
|
reports for that invocation. Validation and checksums use that explicit set, so
|
|
reusing a report directory cannot make stale output look like part of a new run.
|
|
|
|
The wrapper tags the toolbox image by a fingerprint of the Dockerfile,
|
|
`requirements-audit.txt`, and the Semgrep smoke-test inputs. If those inputs have
|
|
not changed, subsequent runs reuse the existing local image instead of
|
|
reinstalling all tools. The stable alias is `govoplan/security-audit:local`
|
|
unless `SECURITY_AUDIT_IMAGE` is set.
|
|
|
|
Semgrep is installed separately in the toolbox image because current Semgrep
|
|
packages pin affected Click and MCP versions. The image upgrades both after
|
|
Semgrep installation, runs an actual local-rule scan, and audits the final
|
|
toolbox Python environment during the image build. The compatibility releases
|
|
are pinned exactly to keep the tested override reproducible. Remove either
|
|
compatibility override only after Semgrep's own dependency bounds include a
|
|
fixed version.
|
|
|
|
Force a cached rebuild:
|
|
|
|
```bash
|
|
tools/checks/security-audit/run.sh --mode ci --scope current --rebuild
|
|
```
|
|
|
|
Refresh from upstream base images and package ranges:
|
|
|
|
```bash
|
|
tools/checks/security-audit/run.sh --mode ci --scope current --update
|
|
```
|
|
|
|
Build or refresh the toolbox without running an audit:
|
|
|
|
```bash
|
|
tools/checks/security-audit/run.sh --mode quick --scope current --build-only
|
|
tools/checks/security-audit/run.sh --mode quick --scope current --update --build-only
|
|
```
|
|
|
|
## Modes
|
|
|
|
- `quick`: local Semgrep rules, Bandit, Ruff security rules, Gitleaks.
|
|
- `ci`: quick plus Semgrep public registry rulesets, Trivy, pip-audit, npm audit.
|
|
- `full`: ci plus OSV-Scanner, jscpd, Radon, and Xenon.
|
|
|
|
Semgrep and Trivy are invoked with finding-sensitive exit codes. Their exit 1
|
|
is therefore a finding under the wrapper contract; higher exit codes, missing
|
|
output, invalid JSON/SARIF, and scanner error payloads are execution failures.
|
|
|
|
Bandit and Ruff security reports are split by source kind. Production code under
|
|
`src/` is written to `bandit.json` and `ruff-security.json` and controls strict
|
|
mode. Test code under `tests/` is still scanned for visibility, but its findings
|
|
are written separately to `bandit-tests.json` and `ruff-security-tests.json` so
|
|
fixture passwords, assertions, and temporary paths do not hide the production
|
|
baseline.
|
|
|
|
## Gating
|
|
|
|
The initial Gitea workflow runs in report-only mode:
|
|
|
|
```bash
|
|
SECURITY_AUDIT_FAIL_ON_FINDINGS=0
|
|
```
|
|
|
|
This avoids blocking every push while the first baseline is reviewed. After the
|
|
baseline is clean, switch the workflow to:
|
|
|
|
```bash
|
|
SECURITY_AUDIT_FAIL_ON_FINDINGS=1
|
|
```
|
|
|
|
or run locally with:
|
|
|
|
```bash
|
|
tools/checks/security-audit/run.sh --mode ci --scope current --strict
|
|
```
|
|
|
|
## Audit Burndown Workflow
|
|
|
|
Treat Gitea issues as the active audit state. A full GovOPlaN audit should
|
|
produce one tracker issue in `GovOPlaN/govoplan` and child issues in the
|
|
repository that owns each fix.
|
|
|
|
Use the tracker issue for:
|
|
|
|
- report path, timestamp, mode, and scope
|
|
- scanner counts by category
|
|
- clean scanners and resolved findings
|
|
- links to child issues
|
|
- the next audit run target
|
|
|
|
Use child issues for concrete code or configuration changes. Apply
|
|
`source/security-audit` to every issue created from a report, then add the
|
|
most specific audit label:
|
|
|
|
- `audit/quick-fix`: narrow direct remediation
|
|
- `audit/structural`: behavior or architecture needs review
|
|
- `audit/complexity`: Radon/Xenon maintainability finding
|
|
- `audit/duplication`: jscpd duplication finding
|
|
- `audit/false-positive`: reviewed narrow false positive or accepted risk
|
|
- `audit/needs-design`: human decision needed before implementation
|
|
|
|
Keep active implementation status in issues instead of committing generated
|
|
audit reports. `audit-reports/` is ignored; quote the report directory and the
|
|
important scanner counts in the tracker issue.
|
|
|
|
The jscpd step is intentionally scoped to application and test source. It
|
|
excludes documentation snippets, package manifests, generated translations,
|
|
public SVG assets, workflow YAML, and declarative backend schema JSON because
|
|
those reports produce metadata or asset repetition rather than actionable source
|
|
duplication. Keep exclusions narrow and create child issues for source-code
|
|
clusters that cross module ownership or make behavior harder to change safely.
|
|
|
|
## Image Freshness
|
|
|
|
The regular `Security Audit` workflow reuses the fingerprinted toolbox image
|
|
when the Docker daemon is persistent, which is the normal case for the
|
|
self-hosted Gitea runner using the host Docker socket. Trusted push, schedule,
|
|
and manual runs scan all registered repositories; authenticated SSH is used
|
|
only for the private website repository. The wrapper inspects the Actions job
|
|
mount table and forwards only the narrowest writable mount covering the audit
|
|
scope; it never inherits the job's Docker socket or unrelated runner mounts.
|
|
Pull-request audit runs stay disabled while the audit runner exposes its host
|
|
Docker socket: PR-controlled audit code must run on a disposable or rootless
|
|
runner without host-socket access. The separate
|
|
`Security Audit Toolbox Update` workflow runs weekly with
|
|
`SECURITY_AUDIT_UPDATE=1`; it pulls current base images and re-resolves the
|
|
allowed tool version ranges into a refreshed local image.
|
|
|
|
## Direct Host Usage
|
|
|
|
The container is the recommended path. For direct host usage, install the Python
|
|
tools first:
|
|
|
|
```bash
|
|
cd /mnt/DATA/git/govoplan
|
|
python -m venv .venv
|
|
./.venv/bin/python -m pip install -r requirements-audit.txt
|
|
./.venv/bin/python -m pip install 'semgrep>=1.140,<2'
|
|
./.venv/bin/python -m pip install --upgrade --no-deps 'click==8.3.3'
|
|
./.venv/bin/python -m pip install --upgrade --no-deps 'mcp==1.28.1'
|
|
./.venv/bin/semgrep scan --metrics=off --config tools/checks/security-audit/semgrep-govoplan.yml tools/checks/check-version-alignment.py
|
|
./.venv/bin/pip-audit --progress-spinner off
|
|
```
|
|
|
|
Then install the non-Python tools (`gitleaks`, `trivy`, `osv-scanner`, `jscpd`)
|
|
through the host package manager or vendor instructions and run:
|
|
|
|
```bash
|
|
tools/checks/check-security-audit.sh --mode quick --scope current
|
|
```
|