Document production target evidence handoff
This commit is contained in:
@@ -0,0 +1,182 @@
|
||||
# Production Target And Independent Evidence Handoff
|
||||
|
||||
This runbook identifies the external inputs needed to finish
|
||||
[GovOPlaN #27](https://git.add-ideas.de/GovOPlaN/govoplan/issues/27) and
|
||||
[GovOPlaN #37](https://git.add-ideas.de/GovOPlaN/govoplan/issues/37). The
|
||||
repository can render, inspect and sign evidence for a target, but it cannot
|
||||
manufacture an independent failure domain or an independent approval authority.
|
||||
|
||||
## GovOPlaN #27: real two-node target
|
||||
|
||||
The bounded acceptance target is two independently schedulable worker nodes.
|
||||
The API and WebUI must each have ready replicas on both nodes, all Deployments
|
||||
must be available, the active module composition and software versions must be
|
||||
consistent, every configured queue must have a worker, and the database
|
||||
connection budget must pass. The validation then deletes one ready API pod and
|
||||
requires replacement without an observed readiness outage.
|
||||
|
||||
Two virtual machines on different physical hosts or availability zones meet the
|
||||
failure-domain intent. Two containers, VMs or Kubernetes nodes on one physical
|
||||
host are useful development targets but do not close #27. A two-worker cluster
|
||||
also does not prove control-plane high availability. For a self-managed
|
||||
production cluster, use three control-plane nodes plus at least two workers; a
|
||||
managed control plane plus two workers is the shorter path.
|
||||
|
||||
### What the target owner must provide
|
||||
|
||||
Provide these through a secure handoff, not an issue, chat message or Git:
|
||||
|
||||
1. A kubeconfig path with access to the target, for example
|
||||
`~/.config/govoplan/targets/<target>.kubeconfig`, mode `0600`.
|
||||
2. A stable installation ID, public HTTPS hostname, namespace, ingress class and
|
||||
TLS-secret or certificate-manager arrangement.
|
||||
3. Two independently schedulable workers and permission to place API and WebUI
|
||||
replicas on both.
|
||||
4. External, logically shared PostgreSQL, Redis and S3 endpoints with trusted
|
||||
CA material and network reachability from every worker. Do not co-locate the
|
||||
only copies of these services on the two workers used for the failure drill.
|
||||
5. The six runtime secret values required by the generated manifest:
|
||||
`MASTER_KEY_B64`, `DATABASE_URL`, `GOVOPLAN_DATABASE_URL_PGTOOLS`,
|
||||
`REDIS_URL`, `FILE_STORAGE_S3_ACCESS_KEY_ID` and
|
||||
`FILE_STORAGE_S3_SECRET_ACCESS_KEY`.
|
||||
6. A short-lived GovOPlaN API key limited to `ops:operations:read`, supplied in
|
||||
`GOVOPLAN_OPS_API_KEY` only for evidence collection.
|
||||
7. An approved drill window and permission to delete one API pod.
|
||||
|
||||
If no Kubernetes target exists, provide hostnames/IP addresses for the machines,
|
||||
an SSH user and key path, the internal/external DNS plan, and the permitted
|
||||
firewall ports. Those inputs are sufficient to provision a k3s target. They are
|
||||
not sufficient to claim control-plane HA unless three control-plane failure
|
||||
domains are present.
|
||||
|
||||
### Separate deployment and evidence authorities
|
||||
|
||||
The deployment identity may create and update the namespace, Secret,
|
||||
ConfigMap, Deployments, Services, Jobs, PodDisruptionBudgets and Ingress. The
|
||||
evidence collector only needs:
|
||||
|
||||
- cluster scope: `get` and `list` for `nodes`;
|
||||
- target namespace: `get` and `list` for `pods` and `deployments`;
|
||||
- target namespace during the approved drill: `delete` for `pods`.
|
||||
|
||||
Use separate kubeconfig contexts or service accounts when the same person does
|
||||
not hold both roles.
|
||||
|
||||
### Render, apply and verify
|
||||
|
||||
Use the signed, digest-pinned installation bundle selected for the target:
|
||||
|
||||
```bash
|
||||
export KUBECONFIG="$HOME/.config/govoplan/targets/<target>.kubeconfig"
|
||||
|
||||
python tools/deployment/govoplan-deploy.py render-kubernetes \
|
||||
--directory /srv/govoplan/<installation-id> \
|
||||
--namespace govoplan \
|
||||
--secret-name govoplan-runtime \
|
||||
--tls-secret-name govoplan-tls \
|
||||
--ingress-class-name nginx \
|
||||
--output /srv/govoplan/<installation-id>/kubernetes.json
|
||||
|
||||
kubectl apply -f /srv/govoplan/<installation-id>/kubernetes.json
|
||||
kubectl -n govoplan wait --for=condition=available deployment --all --timeout=10m
|
||||
|
||||
export GOVOPLAN_OPS_API_KEY="$(cat /run/secrets/govoplan-ops-evidence-key)"
|
||||
python tools/deployment/govoplan-deploy.py verify-kubernetes \
|
||||
--directory /srv/govoplan/<installation-id> \
|
||||
--namespace govoplan \
|
||||
--exercise-api-pod-loss \
|
||||
--output /srv/govoplan/<installation-id>/evidence/kubernetes-multi-host.json
|
||||
unset GOVOPLAN_OPS_API_KEY
|
||||
```
|
||||
|
||||
The verifier emits sanitized JSON and exits nonzero if the topology, runtime,
|
||||
queue, connection-budget or pod-loss checks fail. Preserve the private cluster
|
||||
logs and manifest alongside the sanitized result in the controlled evidence
|
||||
store.
|
||||
|
||||
## GovOPlaN #37: controlled signed target evidence
|
||||
|
||||
Yes, collection, review and signing can run in containers. A container provides
|
||||
repeatability and process isolation; it does not create independent authority.
|
||||
The production approver must control a different private key from the target
|
||||
operator/assessor and must review the evidence before signing the
|
||||
`production_approval` scope.
|
||||
|
||||
Use at least these three key boundaries:
|
||||
|
||||
1. **Installer authority:** signs installed-release-origin receipts only.
|
||||
2. **Target assessment authority:** signs the permitted target, accessibility,
|
||||
privacy, security, operations and recovery scopes.
|
||||
3. **Production approval authority:** independently signs only
|
||||
`production_approval` after reviewing the other evidence.
|
||||
|
||||
Do not reuse release-catalog keys for any of these roles. Keep private Ed25519
|
||||
keys outside Git, Gitea, GovOPlaN application storage and chat. Publish only the
|
||||
public keyrings. The proof issuer already rejects key reuse across release,
|
||||
installer and proof trust domains.
|
||||
|
||||
### Generate independently held keys
|
||||
|
||||
Each authority runs this command in its own `0700` directory. The generator
|
||||
refuses existing output paths and writes both files as `0600`:
|
||||
|
||||
```bash
|
||||
install -d -m 0700 "$HOME/.config/govoplan/authority-keys"
|
||||
|
||||
python tools/assessments/generate-authority-keypair.py \
|
||||
--purpose proof \
|
||||
--key-id authority:target-2026 \
|
||||
--scope target_environment \
|
||||
--scope accessibility \
|
||||
--scope privacy \
|
||||
--scope security \
|
||||
--scope operations \
|
||||
--scope recovery \
|
||||
--private-key "$HOME/.config/govoplan/authority-keys/target-2026.pem" \
|
||||
--keyring "$HOME/.config/govoplan/authority-keys/target-2026-public.json"
|
||||
```
|
||||
|
||||
The independent production approver generates another key with only
|
||||
`--scope production_approval`. An installer authority uses `--purpose installer`
|
||||
and no `--scope`. Merge public key entries into the separately controlled
|
||||
keyrings only after the responsible authorities verify fingerprints out of
|
||||
band.
|
||||
|
||||
### Container boundary
|
||||
|
||||
Use two one-shot jobs or containers:
|
||||
|
||||
- **Collector/assessor:** network access, read-only source and trust mounts,
|
||||
read/write private evidence output, and the narrowly scoped kubeconfig. It
|
||||
must not receive the production-approval private key.
|
||||
- **Production approver:** `--network none`, read-only assessment/evidence/trust
|
||||
mounts, a read-only secret mount containing only the approval key, and a
|
||||
separate output mount. It must not receive deployment credentials.
|
||||
|
||||
Build or select the assessment image by digest and record that digest in the
|
||||
evidence log. A representative runtime shape is:
|
||||
|
||||
```bash
|
||||
docker run --rm --network none --read-only --tmpfs /tmp \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
--mount type=bind,src="$PWD/evidence",dst=/evidence,readonly \
|
||||
--mount type=bind,src="$PWD/trust",dst=/trust,readonly \
|
||||
--mount type=bind,src="$HOME/.config/govoplan/authority-keys",dst=/run/keys,readonly \
|
||||
--mount type=bind,src="$PWD/approved",dst=/output \
|
||||
<assessment-image>@sha256:<digest> \
|
||||
<assessment command>
|
||||
```
|
||||
|
||||
The current evidence commands and required scopes are documented in
|
||||
[`TARGET_MATURITY_EVIDENCE_RUNBOOK.md`](TARGET_MATURITY_EVIDENCE_RUNBOOK.md).
|
||||
The final proof must cover `target_environment`, `accessibility`, `privacy`,
|
||||
`security`, `operations`, `recovery` and independent `production_approval`, and
|
||||
must bind to the verified installed composition and installer receipt.
|
||||
|
||||
## Completion boundary
|
||||
|
||||
#27 can close after the real target produces a passing pod-loss result. #37 can
|
||||
close after an independently approved, schema-valid proof is generated for that
|
||||
same installed composition and the public authority keyrings, proof and private
|
||||
evidence custody references are recorded. Neither issue should close from a
|
||||
single-host simulation or a self-approved signature.
|
||||
Reference in New Issue
Block a user