Add scalable deployment planning and guided releases
Security Audit / security-audit (push) Failing after 4s
Dependency Audit / dependency-audit (push) Failing after 7s
Deployment Installer / deployment-installer (push) Failing after 4s

This commit is contained in:
2026-07-31 02:49:03 +02:00
parent 3864ce28b1
commit 908090dd0f
13 changed files with 1511 additions and 228 deletions
@@ -23,7 +23,12 @@ It currently supports:
- managed, external, or evaluation-only disabled Redis;
- disabled mail, an external relay declaration, or an evaluation-only
GreenMail service;
- durable local file storage or external S3-compatible storage;
- durable local file storage, managed single-node Garage S3, or external
S3-compatible storage;
- an explicit HAProxy service that load-balances configured WebUI and API
replicas without access to the Docker socket;
- declarative API, WebUI, and worker replica counts while keeping migrations
and the scheduler singleton;
- Core, base, or full initial module selections;
- deterministic Compose JSON accepted by Compose v2;
- generated secrets stored in a private `0600` file;
@@ -47,7 +52,11 @@ Create a local evaluation bundle:
--profile evaluation \
--postgres managed \
--redis managed \
--storage garage \
--mail test-mail \
--api-replicas 2 \
--web-replicas 2 \
--worker-replicas 2 \
--module-set base
```
@@ -74,6 +83,8 @@ The private installation directory contains:
| `installation.json` | Versioned, non-secret desired state |
| `secrets.env` | Deployment-local secrets and external service bindings |
| `compose.json` | Deterministic generated Compose definition |
| `garage.toml` | Non-secret managed Garage server configuration |
| `load-balancer.cfg` | Non-secret HAProxy WebUI/API discovery configuration |
| `plan.json` | Latest desired-state diff and readiness findings |
| `receipt.json` | Last successfully applied immutable identities |
| `.deployment.lock` | Same-host operation exclusion |
@@ -126,7 +137,9 @@ a production distribution:
5. **Deployment agent.** Web updates need a separate privileged reconciler with
a typed command allowlist. The API and browser must never receive the Docker
socket or arbitrary shell access.
6. **Ingress and certificates.** A self-hosted profile needs an explicit choice
6. **Ingress and certificates.** The managed HAProxy service provides HTTP
load balancing inside the deployment boundary; it does not issue or renew
certificates. A self-hosted profile still needs an explicit choice
between an existing reverse proxy and a supported managed ingress, including
trusted-proxy boundaries, TLS certificate issuance, renewal, and health
probing through the public route.
@@ -185,11 +198,49 @@ presenting a container as a complete mail service.
### File Storage
`local` uses a durable Compose volume and is appropriate for one-host
installations. `s3` requires endpoint, region, access key, secret key, and
bucket values. Self-hosted S3 endpoints must use HTTPS.
installations. `garage` provisions Garage 2.3 in its supported single-node
bootstrap mode, generates a private application key and bucket, and connects
the Files S3 backend to the exact installer-owned internal endpoint. The
deployment-only trust marker cannot authorize another S3 host; arbitrary
external SDK endpoints remain fail-closed until peer pinning is implemented.
Garage metadata and object data use separate persistent volumes. `s3` requires
an external endpoint, region, access key, secret key, and bucket values.
Self-hosted external S3 endpoints must use HTTPS.
Local storage must be included in backup and restore drills. Horizontal API or
worker scale-out requires shared/object storage.
worker scale-out requires shared/object storage. The managed Garage profile is
persistent but has no data redundancy; availability-sensitive installations
must use a tested multi-node Garage cluster or another external S3 service.
### Load Balancing And Replicas
The generated Compose topology publishes only `load-balancer`. HAProxy uses
Docker DNS service discovery to distribute public traffic across WebUI replicas
and WebUI API proxy traffic across API replicas. The WebUI and API services do
not publish host ports. HAProxy has no Docker socket and discovers only the
bounded replica slots rendered into `load-balancer.cfg`.
Replica counts are desired state:
```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py configure \
--directory /tmp/govoplan-evaluation \
--api-replicas 3 \
--web-replicas 2 \
--worker-replicas 4
./.venv/bin/python tools/deployment/govoplan-deploy.py apply \
--directory /tmp/govoplan-evaluation
```
Workers are queue consumers, so they are scaled through Redis rather than put
behind an HTTP load balancer. The migration runner and Celery scheduler remain
singletons. Multiple API replicas are rejected when Redis is disabled because
distributed throttling and queued work cannot then be shared correctly.
This is same-host scaling. Docker Compose uses a bridge network and does not
place containers on another machine. See
[Scaling And Multi-Host Deployment](SCALING_AND_MULTI_HOST_DEPLOYMENT.md) for
the supported topology and promotion path.
## Reconfiguration Semantics
@@ -207,6 +258,8 @@ applied state. `plan` compares their canonical hashes and service sets.
- Managed-to-external transitions require the new endpoint in the same
operation.
- Migrations run as a one-shot service before API/worker replacement.
- The first upgrade from a direct WebUI host port stops that legacy WebUI
container immediately before HAProxy claims the same endpoint.
- Health must recover before a new receipt is committed.
This is sufficient for one-host reconciliation. Production updates additionally
@@ -261,7 +314,8 @@ Run the focused tests:
```
The tests cover profile restrictions, secret persistence, external endpoint
requirements, S3 policy, Compose service selection, secret non-disclosure,
requirements, managed Garage bootstrap, S3 policy, replica validation, HAProxy
discovery configuration, Compose service selection, secret non-disclosure,
service-specific environment isolation, private file modes, external endpoint
preflight, first-plan generation, apply ordering, and receipt-based
idempotency.