govoplan/docs/INSTALLATION_AND_DEPLOYMENT_ARCHITECTURE.md

# Installation And Deployment Architecture

## Goal

A supported GovOPlaN installation starts with one downloaded, verified
bootstrap artifact. The administrator answers a bounded set of questions and
receives a working base system. Re-running the same tool repairs or
reconfigures that installation instead of creating unrelated state.

The canonical product journey remains
[System Administrator Lifecycle User Story](SYSTEM_ADMINISTRATOR_LIFECYCLE_USER_STORY.md).
This document defines the deployer boundary and the first executable slice.

## First Executable Slice

`tools/deployment/govoplan-deploy.py` is a standard-library-only deployment
compiler and reconciler. It can be tested without installing GovOPlaN itself.

It currently supports:

- evaluation and self-hosted profiles;
- managed or external PostgreSQL;
- managed, external, or evaluation-only disabled Redis;
- disabled mail, an external relay declaration, or an evaluation-only
  GreenMail service;
- durable local file storage or external S3-compatible storage;
- Core, base, or full initial module selections;
- deterministic Compose JSON accepted by Compose v2;
- generated secrets stored in a private `0600` file;
- service-specific environment allowlists so infrastructure containers do not
  receive unrelated application credentials;
- plan, render, doctor, status, and apply commands;
- an installation lock, migration-before-start ordering, readiness polling,
  and an applied-state receipt;
- idempotent reconfiguration that preserves generated secrets;
- a keyed environment fingerprint that detects private binding changes without
  writing secret values to plans or receipts;
- host CPU, memory, disk, entropy, architecture, Docker daemon, Compose,
  listen-port, and external endpoint preflight checks;
- service removal without implicit data-volume deletion.

Create a local evaluation bundle:

```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py init \
  --directory /tmp/govoplan-evaluation \
  --profile evaluation \
  --postgres managed \
  --redis managed \
  --mail test-mail \
  --module-set base
```

Inspect the generated intent and host requirements:

```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py doctor \
  --directory /tmp/govoplan-evaluation
```

Change a component without rotating existing generated secrets:

```sh
./.venv/bin/python tools/deployment/govoplan-deploy.py configure \
  --directory /tmp/govoplan-evaluation \
  --redis external \
  --redis-url 'rediss://:password@redis.example.org:6379/0'
```

The private installation directory contains:

| File | Purpose |
| --- | --- |
| `installation.json` | Versioned, non-secret desired state |
| `secrets.env` | Deployment-local secrets and external service bindings |
| `compose.json` | Deterministic generated Compose definition |
| `plan.json` | Latest desired-state diff and readiness findings |
| `receipt.json` | Last successfully applied immutable identities |
| `.deployment.lock` | Same-host operation exclusion |

The specification contract is
[`installation-spec.schema.json`](installation-spec.schema.json).

Build the same dependency-free tool as one downloadable artifact:

```sh
./.venv/bin/python tools/deployment/build-deployer-zipapp.py \
  --output /tmp/govoplan-deploy.pyz
python /tmp/govoplan-deploy.pyz --help
```

To exercise reconciliation with locally available evaluation images:

```sh
python /tmp/govoplan-deploy.pyz init \
  --non-interactive \
  --directory /tmp/govoplan-evaluation \
  --profile evaluation \
  --api-image local/govoplan-api:test \
  --web-image local/govoplan-web:test
python /tmp/govoplan-deploy.pyz apply \
  --directory /tmp/govoplan-evaluation \
  --allow-unverified-images \
  --skip-pull
```

Those images must already contain the selected module set. The override exists
only to exercise local orchestration before release artifacts exist; it is
rejected for `self-hosted`.

## Current Production Gates

The tool deliberately reports blockers instead of pretending the source tree is
a production distribution:

1. **OCI release artifacts.** The release pipeline does not yet publish pinned
   multi-architecture API and WebUI images.
2. **Signed distribution manifest.** A channel manifest must bind exact image
   digests, Compose compatibility, SBOM/provenance references, and revocation
   state. Recording a URL and checksum is not signature verification.
3. **First administrator.** Production needs a one-time, restricted enrollment
   identity. The development bootstrap must not be enabled in production.
4. **Image/module composition.** The selected module set must be proven present
   in the exact image or installed from verified offline artifacts before it is
   enabled.
5. **Deployment agent.** Web updates need a separate privileged reconciler with
   a typed command allowlist. The API and browser must never receive the Docker
   socket or arbitrary shell access.
6. **Ingress and certificates.** A self-hosted profile needs an explicit choice
   between an existing reverse proxy and a supported managed ingress, including
   trusted-proxy boundaries, TLS certificate issuance, renewal, and health
   probing through the public route.

`apply --allow-unverified-images` is therefore restricted to the evaluation
profile. It explicitly acknowledges both mutable image identities and
unverified image/module composition. It is a local test escape hatch, not a
production setting.

## Component Choices

### PostgreSQL

`managed` creates a persistent PostgreSQL container and private generated
credentials. `external` requires an explicit `DATABASE_URL`; switching from
managed to external cannot reuse the old `postgres` Docker hostname
accidentally.

Interactive entry hides external URLs because they commonly contain
credentials. For unattended automation, provide them through a protected
operator mechanism and avoid storing secret-bearing flags in shell history.

Production policy should support external managed databases and local managed
PostgreSQL equally at the application boundary. Backup, point-in-time recovery,
high availability, and major-version upgrades remain deployment properties.

### Redis

`managed` creates an authenticated, append-only Redis container. `external`
requires an explicit `REDIS_URL`. `disabled` is evaluation-only and disables
workers while recording the single-process login-throttle risk acknowledgement.

`doctor` performs a bounded TCP connection check for external PostgreSQL,
Redis, and S3 endpoints. This verifies DNS, routing, and that the port accepts a
connection; it is not an authentication or semantic health check.

Production base installations include Redis because durable queues, distributed
throttling, notifications, scheduled work, and transactional event delivery
must survive API restarts.

### Mail

The first slice distinguishes:

- `disabled`;
- `external-relay`, which records the infrastructure decision but leaves Mail
  server/credential creation as a visible post-install task;
- `test-mail`, an evaluation-only GreenMail service.

A bundled production mail server is intentionally not a default. Operating one
requires DNS, reverse DNS, TLS, DKIM, SPF, DMARC, reputation, abuse handling,
queue monitoring, and upgrade policy. A later profile may support an
operator-selected MTA/relay, but it must expose these requirements rather than
presenting a container as a complete mail service.

### File Storage

`local` uses a durable Compose volume and is appropriate for one-host
installations. `s3` requires endpoint, region, access key, secret key, and
bucket values. Self-hosted S3 endpoints must use HTTPS.

Local storage must be included in backup and restore drills. Horizontal API or
worker scale-out requires shared/object storage.

## Reconfiguration Semantics

`installation.json` is desired state. `receipt.json` is the last successfully
applied state. `plan` compares their canonical hashes and service sets.

- Adding a managed component creates its service and persistent volume.
- Removing a component removes its service container on apply.
- Volumes are retained by default; deleting data requires a separate,
  deliberately destructive workflow.
- Existing generated credentials are retained unless an explicit future rotate
  operation is requested.
- Private configuration changes are represented by a keyed fingerprint in the
  plan and receipt; plaintext values are never copied there.
- Managed-to-external transitions require the new endpoint in the same
  operation.
- Migrations run as a one-shot service before API/worker replacement.
- Health must recover before a new receipt is committed.

This is sufficient for one-host reconciliation. Production updates additionally
need backup/restore gates, maintenance/drain state, database compatibility
windows, image signature verification, and rollback/forward-recovery policy.

## Web Update Boundary

The intended update path is:

1. Ops reads the non-secret installation receipt and reports management mode,
   current release, component health, and update availability.
2. An authorized administrator asks Core to create a typed deployment request,
   for example `reconcile_release` or `rollback_release`.
3. Core persists the reviewed immutable plan, actor, expected current receipt,
   and idempotency key.
4. A separately deployed, narrow deployment agent claims the request.
5. The agent verifies signatures/digests, acquires a fenced deployment lock,
   backs up, pulls, migrates, reconciles, probes health, and writes evidence.
6. Ops presents durable progress and the resulting receipt.

The agent owns container-runtime access. It accepts no command strings from the
browser and has no domain-data permissions. Installations managed by Kubernetes,
systemd, or another external orchestrator expose read-only status and an export
of the reviewed update recipe instead of a non-functional update button.

## Distribution Workflow

The downloadable entry point should eventually be:

```sh
curl --proto '=https' --tlsv1.2 --fail --location \
  https://govoplan.add-ideas.de/install/v1/bootstrap.pyz \
  --output govoplan-bootstrap.pyz
python3 govoplan-bootstrap.pyz init
```

The published documentation must include an independent checksum/signature
verification command before execution. The zipapp then downloads only a signed
distribution manifest, verifies it against an embedded or explicitly installed
keyring, and renders the same installation contract implemented here.

The source-tree script is the test harness for that future zipapp. It is not yet
the internet bootstrap artifact.

## Verification

Run the focused tests:

```sh
./.venv/bin/python -m unittest -v tests.test_deployment_installer
```

The tests cover profile restrictions, secret persistence, external endpoint
requirements, S3 policy, Compose service selection, secret non-disclosure,
service-specific environment isolation, private file modes, external endpoint
preflight, first-plan generation, apply ordering, and receipt-based
idempotency.