feat(deploy): add declarative installation workflow
This commit is contained in:
267
docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md
Normal file
267
docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# Installation And Deployment Architecture
|
||||
|
||||
## Goal
|
||||
|
||||
A supported GovOPlaN installation starts with one downloaded, verified
|
||||
bootstrap artifact. The administrator answers a bounded set of questions and
|
||||
receives a working base system. Re-running the same tool repairs or
|
||||
reconfigures that installation instead of creating unrelated state.
|
||||
|
||||
The canonical product journey remains
|
||||
[System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
|
||||
This document defines the deployer boundary and the first executable slice.
|
||||
|
||||
## First Executable Slice
|
||||
|
||||
`tools/deployment/govoplan-deploy.py` is a standard-library-only deployment
|
||||
compiler and reconciler. It can be tested without installing GovOPlaN itself.
|
||||
|
||||
It currently supports:
|
||||
|
||||
- evaluation and self-hosted profiles;
|
||||
- managed or external PostgreSQL;
|
||||
- managed, external, or evaluation-only disabled Redis;
|
||||
- disabled mail, an external relay declaration, or an evaluation-only
|
||||
GreenMail service;
|
||||
- durable local file storage or external S3-compatible storage;
|
||||
- Core, base, or full initial module selections;
|
||||
- deterministic Compose JSON accepted by Compose v2;
|
||||
- generated secrets stored in a private `0600` file;
|
||||
- service-specific environment allowlists so infrastructure containers do not
|
||||
receive unrelated application credentials;
|
||||
- plan, render, doctor, status, and apply commands;
|
||||
- an installation lock, migration-before-start ordering, readiness polling,
|
||||
and an applied-state receipt;
|
||||
- idempotent reconfiguration that preserves generated secrets;
|
||||
- a keyed environment fingerprint that detects private binding changes without
|
||||
writing secret values to plans or receipts;
|
||||
- host CPU, memory, disk, entropy, architecture, Docker daemon, Compose,
|
||||
listen-port, and external endpoint preflight checks;
|
||||
- service removal without implicit data-volume deletion.
|
||||
|
||||
Create a local evaluation bundle:
|
||||
|
||||
```sh
|
||||
./.venv/bin/python tools/deployment/govoplan-deploy.py init \
|
||||
--directory /tmp/govoplan-evaluation \
|
||||
--profile evaluation \
|
||||
--postgres managed \
|
||||
--redis managed \
|
||||
--mail test-mail \
|
||||
--module-set base
|
||||
```
|
||||
|
||||
Inspect the generated intent and host requirements:
|
||||
|
||||
```sh
|
||||
./.venv/bin/python tools/deployment/govoplan-deploy.py doctor \
|
||||
--directory /tmp/govoplan-evaluation
|
||||
```
|
||||
|
||||
Change a component without rotating existing generated secrets:
|
||||
|
||||
```sh
|
||||
./.venv/bin/python tools/deployment/govoplan-deploy.py configure \
|
||||
--directory /tmp/govoplan-evaluation \
|
||||
--redis external \
|
||||
--redis-url 'rediss://:password@redis.example.org:6379/0'
|
||||
```
|
||||
|
||||
The private installation directory contains:
|
||||
|
||||
| File | Purpose |
|
||||
| --- | --- |
|
||||
| `installation.json` | Versioned, non-secret desired state |
|
||||
| `secrets.env` | Deployment-local secrets and external service bindings |
|
||||
| `compose.json` | Deterministic generated Compose definition |
|
||||
| `plan.json` | Latest desired-state diff and readiness findings |
|
||||
| `receipt.json` | Last successfully applied immutable identities |
|
||||
| `.deployment.lock` | Same-host operation exclusion |
|
||||
|
||||
The specification contract is
|
||||
[`installation-spec.schema.json`](installation-spec.schema.json).
|
||||
|
||||
Build the same dependency-free tool as one downloadable artifact:
|
||||
|
||||
```sh
|
||||
./.venv/bin/python tools/deployment/build-deployer-zipapp.py \
|
||||
--output /tmp/govoplan-deploy.pyz
|
||||
python /tmp/govoplan-deploy.pyz --help
|
||||
```
|
||||
|
||||
To exercise reconciliation with locally available evaluation images:
|
||||
|
||||
```sh
|
||||
python /tmp/govoplan-deploy.pyz init \
|
||||
--non-interactive \
|
||||
--directory /tmp/govoplan-evaluation \
|
||||
--profile evaluation \
|
||||
--api-image local/govoplan-api:test \
|
||||
--web-image local/govoplan-web:test
|
||||
python /tmp/govoplan-deploy.pyz apply \
|
||||
--directory /tmp/govoplan-evaluation \
|
||||
--allow-unverified-images \
|
||||
--skip-pull
|
||||
```
|
||||
|
||||
Those images must already contain the selected module set. The override exists
|
||||
only to exercise local orchestration before release artifacts exist; it is
|
||||
rejected for `self-hosted`.
|
||||
|
||||
## Current Production Gates
|
||||
|
||||
The tool deliberately reports blockers instead of pretending the source tree is
|
||||
a production distribution:
|
||||
|
||||
1. **OCI release artifacts.** The release pipeline does not yet publish pinned
|
||||
multi-architecture API and WebUI images.
|
||||
2. **Signed distribution manifest.** A channel manifest must bind exact image
|
||||
digests, Compose compatibility, SBOM/provenance references, and revocation
|
||||
state. Recording a URL and checksum is not signature verification.
|
||||
3. **First administrator.** Production needs a one-time, restricted enrollment
|
||||
identity. The development bootstrap must not be enabled in production.
|
||||
4. **Image/module composition.** The selected module set must be proven present
|
||||
in the exact image or installed from verified offline artifacts before it is
|
||||
enabled.
|
||||
5. **Deployment agent.** Web updates need a separate privileged reconciler with
|
||||
a typed command allowlist. The API and browser must never receive the Docker
|
||||
socket or arbitrary shell access.
|
||||
6. **Ingress and certificates.** A self-hosted profile needs an explicit choice
|
||||
between an existing reverse proxy and a supported managed ingress, including
|
||||
trusted-proxy boundaries, TLS certificate issuance, renewal, and health
|
||||
probing through the public route.
|
||||
|
||||
`apply --allow-unverified-images` is therefore restricted to the evaluation
|
||||
profile. It explicitly acknowledges both mutable image identities and
|
||||
unverified image/module composition. It is a local test escape hatch, not a
|
||||
production setting.
|
||||
|
||||
## Component Choices
|
||||
|
||||
### PostgreSQL
|
||||
|
||||
`managed` creates a persistent PostgreSQL container and private generated
|
||||
credentials. `external` requires an explicit `DATABASE_URL`; switching from
|
||||
managed to external cannot reuse the old `postgres` Docker hostname
|
||||
accidentally.
|
||||
|
||||
Interactive entry hides external URLs because they commonly contain
|
||||
credentials. For unattended automation, provide them through a protected
|
||||
operator mechanism and avoid storing secret-bearing flags in shell history.
|
||||
|
||||
Production policy should support external managed databases and local managed
|
||||
PostgreSQL equally at the application boundary. Backup, point-in-time recovery,
|
||||
high availability, and major-version upgrades remain deployment properties.
|
||||
|
||||
### Redis
|
||||
|
||||
`managed` creates an authenticated, append-only Redis container. `external`
|
||||
requires an explicit `REDIS_URL`. `disabled` is evaluation-only and disables
|
||||
workers while recording the single-process login-throttle risk acknowledgement.
|
||||
|
||||
`doctor` performs a bounded TCP connection check for external PostgreSQL,
|
||||
Redis, and S3 endpoints. This verifies DNS, routing, and that the port accepts a
|
||||
connection; it is not an authentication or semantic health check.
|
||||
|
||||
Production base installations include Redis because durable queues, distributed
|
||||
throttling, notifications, scheduled work, and transactional event delivery
|
||||
must survive API restarts.
|
||||
|
||||
### Mail
|
||||
|
||||
The first slice distinguishes:
|
||||
|
||||
- `disabled`;
|
||||
- `external-relay`, which records the infrastructure decision but leaves Mail
|
||||
server/credential creation as a visible post-install task;
|
||||
- `test-mail`, an evaluation-only GreenMail service.
|
||||
|
||||
A bundled production mail server is intentionally not a default. Operating one
|
||||
requires DNS, reverse DNS, TLS, DKIM, SPF, DMARC, reputation, abuse handling,
|
||||
queue monitoring, and upgrade policy. A later profile may support an
|
||||
operator-selected MTA/relay, but it must expose these requirements rather than
|
||||
presenting a container as a complete mail service.
|
||||
|
||||
### File Storage
|
||||
|
||||
`local` uses a durable Compose volume and is appropriate for one-host
|
||||
installations. `s3` requires endpoint, region, access key, secret key, and
|
||||
bucket values. Self-hosted S3 endpoints must use HTTPS.
|
||||
|
||||
Local storage must be included in backup and restore drills. Horizontal API or
|
||||
worker scale-out requires shared/object storage.
|
||||
|
||||
## Reconfiguration Semantics
|
||||
|
||||
`installation.json` is desired state. `receipt.json` is the last successfully
|
||||
applied state. `plan` compares their canonical hashes and service sets.
|
||||
|
||||
- Adding a managed component creates its service and persistent volume.
|
||||
- Removing a component removes its service container on apply.
|
||||
- Volumes are retained by default; deleting data requires a separate,
|
||||
deliberately destructive workflow.
|
||||
- Existing generated credentials are retained unless an explicit future rotate
|
||||
operation is requested.
|
||||
- Private configuration changes are represented by a keyed fingerprint in the
|
||||
plan and receipt; plaintext values are never copied there.
|
||||
- Managed-to-external transitions require the new endpoint in the same
|
||||
operation.
|
||||
- Migrations run as a one-shot service before API/worker replacement.
|
||||
- Health must recover before a new receipt is committed.
|
||||
|
||||
This is sufficient for one-host reconciliation. Production updates additionally
|
||||
need backup/restore gates, maintenance/drain state, database compatibility
|
||||
windows, image signature verification, and rollback/forward-recovery policy.
|
||||
|
||||
## Web Update Boundary
|
||||
|
||||
The intended update path is:
|
||||
|
||||
1. Ops reads the non-secret installation receipt and reports management mode,
|
||||
current release, component health, and update availability.
|
||||
2. An authorized administrator asks Core to create a typed deployment request,
|
||||
for example `reconcile_release` or `rollback_release`.
|
||||
3. Core persists the reviewed immutable plan, actor, expected current receipt,
|
||||
and idempotency key.
|
||||
4. A separately deployed, narrow deployment agent claims the request.
|
||||
5. The agent verifies signatures/digests, acquires a fenced deployment lock,
|
||||
backs up, pulls, migrates, reconciles, probes health, and writes evidence.
|
||||
6. Ops presents durable progress and the resulting receipt.
|
||||
|
||||
The agent owns container-runtime access. It accepts no command strings from the
|
||||
browser and has no domain-data permissions. Installations managed by Kubernetes,
|
||||
systemd, or another external orchestrator expose read-only status and an export
|
||||
of the reviewed update recipe instead of a non-functional update button.
|
||||
|
||||
## Distribution Workflow
|
||||
|
||||
The downloadable entry point should eventually be:
|
||||
|
||||
```sh
|
||||
curl --proto '=https' --tlsv1.2 --fail --location \
|
||||
https://govoplan.add-ideas.de/install/v1/bootstrap.pyz \
|
||||
--output govoplan-bootstrap.pyz
|
||||
python3 govoplan-bootstrap.pyz init
|
||||
```
|
||||
|
||||
The published documentation must include an independent checksum/signature
|
||||
verification command before execution. The zipapp then downloads only a signed
|
||||
distribution manifest, verifies it against an embedded or explicitly installed
|
||||
keyring, and renders the same installation contract implemented here.
|
||||
|
||||
The source-tree script is the test harness for that future zipapp. It is not yet
|
||||
the internet bootstrap artifact.
|
||||
|
||||
## Verification
|
||||
|
||||
Run the focused tests:
|
||||
|
||||
```sh
|
||||
./.venv/bin/python -m unittest -v tests.test_deployment_installer
|
||||
```
|
||||
|
||||
The tests cover profile restrictions, secret persistence, external endpoint
|
||||
requirements, S3 policy, Compose service selection, secret non-disclosure,
|
||||
service-specific environment isolation, private file modes, external endpoint
|
||||
preflight, first-plan generation, apply ordering, and receipt-based
|
||||
idempotency.
|
||||
269
docs/installation-spec.schema.json
Normal file
269
docs/installation-spec.schema.json
Normal file
@@ -0,0 +1,269 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://govoplan.add-ideas.de/schemas/installation-spec-v1.json",
|
||||
"title": "GovOPlaN installation specification",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"schema_version",
|
||||
"installation_id",
|
||||
"profile",
|
||||
"public_url",
|
||||
"listen",
|
||||
"network_subnet",
|
||||
"release",
|
||||
"components",
|
||||
"enabled_modules"
|
||||
],
|
||||
"properties": {
|
||||
"schema_version": {
|
||||
"const": 1
|
||||
},
|
||||
"installation_id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9-]{1,47}$"
|
||||
},
|
||||
"profile": {
|
||||
"enum": [
|
||||
"evaluation",
|
||||
"self-hosted"
|
||||
]
|
||||
},
|
||||
"public_url": {
|
||||
"type": "string",
|
||||
"format": "uri",
|
||||
"pattern": "^https?://"
|
||||
},
|
||||
"listen": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"address",
|
||||
"port"
|
||||
],
|
||||
"properties": {
|
||||
"address": {
|
||||
"type": "string"
|
||||
},
|
||||
"port": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 65535
|
||||
}
|
||||
}
|
||||
},
|
||||
"network_subnet": {
|
||||
"type": "string"
|
||||
},
|
||||
"release": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"channel",
|
||||
"version",
|
||||
"manifest_url",
|
||||
"manifest_sha256",
|
||||
"api_image",
|
||||
"web_image"
|
||||
],
|
||||
"properties": {
|
||||
"channel": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9-]{1,31}$"
|
||||
},
|
||||
"version": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 80
|
||||
},
|
||||
"manifest_url": {
|
||||
"type": "string"
|
||||
},
|
||||
"manifest_sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^$|^[0-9a-f]{64}$"
|
||||
},
|
||||
"api_image": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 300
|
||||
},
|
||||
"web_image": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 300
|
||||
}
|
||||
}
|
||||
},
|
||||
"components": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"postgres",
|
||||
"redis",
|
||||
"mail",
|
||||
"storage"
|
||||
],
|
||||
"properties": {
|
||||
"postgres": {
|
||||
"$ref": "#/$defs/postgres"
|
||||
},
|
||||
"redis": {
|
||||
"$ref": "#/$defs/redis"
|
||||
},
|
||||
"mail": {
|
||||
"$ref": "#/$defs/mail"
|
||||
},
|
||||
"storage": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"mode"
|
||||
],
|
||||
"properties": {
|
||||
"mode": {
|
||||
"enum": [
|
||||
"local",
|
||||
"s3"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"enabled_modules": {
|
||||
"type": "array",
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9_]{1,63}$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"service": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"mode",
|
||||
"image",
|
||||
"url_env"
|
||||
],
|
||||
"properties": {
|
||||
"mode": {
|
||||
"type": "string"
|
||||
},
|
||||
"image": {
|
||||
"type": "string"
|
||||
},
|
||||
"url_env": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"postgres": {
|
||||
"allOf": [
|
||||
{
|
||||
"$ref": "#/$defs/service"
|
||||
},
|
||||
{
|
||||
"properties": {
|
||||
"mode": {
|
||||
"enum": [
|
||||
"managed",
|
||||
"external"
|
||||
]
|
||||
},
|
||||
"url_env": {
|
||||
"const": "DATABASE_URL"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"redis": {
|
||||
"allOf": [
|
||||
{
|
||||
"$ref": "#/$defs/service"
|
||||
},
|
||||
{
|
||||
"properties": {
|
||||
"mode": {
|
||||
"enum": [
|
||||
"managed",
|
||||
"external",
|
||||
"disabled"
|
||||
]
|
||||
},
|
||||
"url_env": {
|
||||
"const": "REDIS_URL"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"mail": {
|
||||
"allOf": [
|
||||
{
|
||||
"$ref": "#/$defs/service"
|
||||
},
|
||||
{
|
||||
"properties": {
|
||||
"mode": {
|
||||
"enum": [
|
||||
"disabled",
|
||||
"external-relay",
|
||||
"test-mail"
|
||||
]
|
||||
},
|
||||
"url_env": {
|
||||
"const": ""
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": {
|
||||
"properties": {
|
||||
"profile": {
|
||||
"const": "self-hosted"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"properties": {
|
||||
"public_url": {
|
||||
"pattern": "^https://"
|
||||
},
|
||||
"components": {
|
||||
"properties": {
|
||||
"redis": {
|
||||
"properties": {
|
||||
"mode": {
|
||||
"enum": [
|
||||
"managed",
|
||||
"external"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"mail": {
|
||||
"properties": {
|
||||
"mode": {
|
||||
"enum": [
|
||||
"disabled",
|
||||
"external-relay"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user