Sync wiki from project files

2026-07-09 11:42:41 +02:00
parent 601be84a59
commit aa14cbfb56
18 changed files with 1916 additions and 91 deletions

@@ -7,10 +7,23 @@ This page is generated from repository and product-directory project files.
- [Product govoplan-split-concept-action-plan](Product govoplan-split-concept-action-plan) - `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/split-concept-action-plan.md` - [Product govoplan-split-concept-action-plan](Product govoplan-split-concept-action-plan) - `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/split-concept-action-plan.md`
- [Repo-README](Repo-README) - `/mnt/DATA/git/govoplan-core/README.md` - [Repo-README](Repo-README) - `/mnt/DATA/git/govoplan-core/README.md`
- [Repo-docs-ACCESS-EXTRACTION-PLAN](Repo-docs-ACCESS-EXTRACTION-PLAN) - `/mnt/DATA/git/govoplan-core/docs/ACCESS_EXTRACTION_PLAN.md` - [Repo-docs-ACCESS-EXTRACTION-PLAN](Repo-docs-ACCESS-EXTRACTION-PLAN) - `/mnt/DATA/git/govoplan-core/docs/ACCESS_EXTRACTION_PLAN.md`
- [Repo-docs-API-CONDITIONAL-DELTA](Repo-docs-API-CONDITIONAL-DELTA) - `/mnt/DATA/git/govoplan-core/docs/API_CONDITIONAL_DELTA.md`
- [Repo-docs-CATALOG-TRUST-AND-LICENSING](Repo-docs-CATALOG-TRUST-AND-LICENSING) - `/mnt/DATA/git/govoplan-core/docs/CATALOG_TRUST_AND_LICENSING.md`
- [Repo-docs-CODEX-WORKFLOW](Repo-docs-CODEX-WORKFLOW) - `/mnt/DATA/git/govoplan-core/docs/CODEX_WORKFLOW.md` - [Repo-docs-CODEX-WORKFLOW](Repo-docs-CODEX-WORKFLOW) - `/mnt/DATA/git/govoplan-core/docs/CODEX_WORKFLOW.md`
- [Repo-docs-CONFIGURATION-PACKAGES](Repo-docs-CONFIGURATION-PACKAGES) - `/mnt/DATA/git/govoplan-core/docs/CONFIGURATION_PACKAGES.md`
- [Repo-docs-DEPLOYMENT-OPERATOR-GUIDE](Repo-docs-DEPLOYMENT-OPERATOR-GUIDE) - `/mnt/DATA/git/govoplan-core/docs/DEPLOYMENT_OPERATOR_GUIDE.md`
- [Repo-docs-EVENTS-AND-AUDIT](Repo-docs-EVENTS-AND-AUDIT) - `/mnt/DATA/git/govoplan-core/docs/EVENTS_AND_AUDIT.md`
- [Repo-docs-GITEA-ISSUES](Repo-docs-GITEA-ISSUES) - `/mnt/DATA/git/govoplan-core/docs/GITEA_ISSUES.md` - [Repo-docs-GITEA-ISSUES](Repo-docs-GITEA-ISSUES) - `/mnt/DATA/git/govoplan-core/docs/GITEA_ISSUES.md`
- [Repo-docs-GOVERNMENT-OPERATIONS-VISION](Repo-docs-GOVERNMENT-OPERATIONS-VISION) - `/mnt/DATA/git/govoplan-core/docs/GOVERNMENT_OPERATIONS_VISION.md`
- [Repo-docs-GOVOPLAN-MASTER-ROADMAP](Repo-docs-GOVOPLAN-MASTER-ROADMAP) - `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MASTER_ROADMAP.md`
- [Repo-docs-GOVOPLAN-MODULE-ROADMAP](Repo-docs-GOVOPLAN-MODULE-ROADMAP) - `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MODULE_ROADMAP.md` - [Repo-docs-GOVOPLAN-MODULE-ROADMAP](Repo-docs-GOVOPLAN-MODULE-ROADMAP) - `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MODULE_ROADMAP.md`
- [Repo-docs-MODULE-ARCHITECTURE](Repo-docs-MODULE-ARCHITECTURE) - `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md` - [Repo-docs-MODULE-ARCHITECTURE](Repo-docs-MODULE-ARCHITECTURE) - `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`
- [Repo-docs-MODULE-BOUNDARY-DECISIONS](Repo-docs-MODULE-BOUNDARY-DECISIONS) - `/mnt/DATA/git/govoplan-core/docs/MODULE_BOUNDARY_DECISIONS.md`
- [Repo-docs-POLICY-CONTRACTS](Repo-docs-POLICY-CONTRACTS) - `/mnt/DATA/git/govoplan-core/docs/POLICY_CONTRACTS.md`
- [Repo-docs-PUBLIC-SECTOR-INTEGRATION-STRATEGY](Repo-docs-PUBLIC-SECTOR-INTEGRATION-STRATEGY) - `/mnt/DATA/git/govoplan-core/docs/PUBLIC_SECTOR_INTEGRATION_STRATEGY.md`
- [Repo-docs-RBAC-MANIFEST](Repo-docs-RBAC-MANIFEST) - `/mnt/DATA/git/govoplan-core/docs/RBAC_MANIFEST.md` - [Repo-docs-RBAC-MANIFEST](Repo-docs-RBAC-MANIFEST) - `/mnt/DATA/git/govoplan-core/docs/RBAC_MANIFEST.md`
- [Repo-docs-RELEASE-CATALOG-WORKFLOW](Repo-docs-RELEASE-CATALOG-WORKFLOW) - `/mnt/DATA/git/govoplan-core/docs/RELEASE_CATALOG_WORKFLOW.md`
- [Repo-docs-RELEASE-DEPENDENCIES](Repo-docs-RELEASE-DEPENDENCIES) - `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md` - [Repo-docs-RELEASE-DEPENDENCIES](Repo-docs-RELEASE-DEPENDENCIES) - `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md`
- [Repo-docs-REMOTE-WEBUI-BUNDLES](Repo-docs-REMOTE-WEBUI-BUNDLES) - `/mnt/DATA/git/govoplan-core/docs/REMOTE_WEBUI_BUNDLES.md`
- [Repo-docs-SCALABILITY-AND-SIZING](Repo-docs-SCALABILITY-AND-SIZING) - `/mnt/DATA/git/govoplan-core/docs/SCALABILITY_AND_SIZING.md`
- [Repo-docs-SYSTEM-GOVERNANCE-MANIFEST](Repo-docs-SYSTEM-GOVERNANCE-MANIFEST) - `/mnt/DATA/git/govoplan-core/docs/SYSTEM_GOVERNANCE_MANIFEST.md` - [Repo-docs-SYSTEM-GOVERNANCE-MANIFEST](Repo-docs-SYSTEM-GOVERNANCE-MANIFEST) - `/mnt/DATA/git/govoplan-core/docs/SYSTEM_GOVERNANCE_MANIFEST.md`

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:94dab3bb9d95f87e5ba96cff --> <!-- codex-wiki-sync:f744da05674d7f1c94f7b85f -->
> Mirrored from `/mnt/DATA/git/govoplan-core/README.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/README.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -29,6 +29,7 @@ Canonical policy documents live in `docs/`:
- [RBAC_MANIFEST.md](docs/RBAC_MANIFEST.md) - [RBAC_MANIFEST.md](docs/RBAC_MANIFEST.md)
- [SYSTEM_GOVERNANCE_MANIFEST.md](docs/SYSTEM_GOVERNANCE_MANIFEST.md) - [SYSTEM_GOVERNANCE_MANIFEST.md](docs/SYSTEM_GOVERNANCE_MANIFEST.md)
- [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md) - [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md)
- [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [CODEX_WORKFLOW.md](docs/CODEX_WORKFLOW.md) - [CODEX_WORKFLOW.md](docs/CODEX_WORKFLOW.md)
Modules may define module-specific permissions and policy behavior, but the platform-level permission model and governance hierarchy belong here. Modules may define module-specific permissions and policy behavior, but the platform-level permission model and governance hierarchy belong here.
@@ -42,7 +43,7 @@ cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m pip install -r requirements-dev.txt ./.venv/bin/python -m pip install -r requirements-dev.txt
``` ```
Run the platform server from core through the module-aware development runner. The default config reads `ENABLED_MODULES` and discovers installed module entry points. Local development defaults to `tenancy,access,admin,policy,audit,campaigns,files,mail`; set `ENABLED_MODULES` explicitly when testing a smaller module permutation. Run the platform server from core through the module-aware development runner. The default config reads `ENABLED_MODULES` and discovers installed module entry points. Local development defaults to `tenancy,access,admin,policy,audit,campaigns,files,mail,calendar,docs,ops`; set `ENABLED_MODULES` explicitly when testing a smaller module permutation.
```bash ```bash
cd /mnt/DATA/git/govoplan-core cd /mnt/DATA/git/govoplan-core
@@ -62,13 +63,24 @@ ENABLED_MODULES=access,campaigns ./.venv/bin/python -m govoplan_core.devserver \
The runner loads the same `GovoplanServerConfig` as `govoplan_core.server.app:app`, builds the platform registry, and passes core plus enabled module source roots to uvicorn as reload directories. After reinstalling the editable package, the same command is also available as `govoplan-devserver`. The runner loads the same `GovoplanServerConfig` as `govoplan_core.server.app:app`, builds the platform registry, and passes core plus enabled module source roots to uvicorn as reload directories. After reinstalling the editable package, the same command is also available as `govoplan-devserver`.
The default development SQLite database lives at `runtime/multimailer-dev.db`, alongside other local runtime state. The default development database is PostgreSQL at `postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev`. Store the password in `~/.pgpass`. To force the old disposable SQLite fallback, run with `GOVOPLAN_DEV_DATABASE_BACKEND=sqlite`; that database lives at `runtime/multimailer-dev.db`.
Local devserver runs do not require Redis. `CELERY_ENABLED` defaults to `false`, so campaign queue actions update database state without publishing Celery tasks. Use the synchronous send flow for local send tests, or set `CELERY_ENABLED=true` only when a Redis broker and worker are running. Local devserver runs do not require Redis. `CELERY_ENABLED` defaults to `false`, so campaign queue actions update database state without publishing Celery tasks. Use the synchronous send flow for local send tests, or set `CELERY_ENABLED=true` only when a Redis broker and worker are running.
If the configured local SQLite database is missing or empty, `govoplan_core.devserver` enables the development bootstrap before loading settings. This creates the schema and the default development login on startup. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead. To run the production-like local profile with PostgreSQL, Redis, a Celery
worker, explicit module configuration, and persistent local file storage:
To verify the effective runtime paths and missing-SQLite bootstrap without starting uvicorn, run the smoke mode: ```bash
cd /mnt/DATA/git/govoplan-core
scripts/launch-production-like-dev.sh
```
See [dev/production-like/README.md](dev/production-like/README.md) for ports,
environment overrides, and cleanup commands.
`govoplan_core.devserver` enables the development bootstrap before loading settings. In dev, startup migrations create or upgrade the schema and the bootstrap creates the default development login if needed. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead.
To verify the effective runtime paths and bootstrap behavior without starting uvicorn, run the smoke mode:
```bash ```bash
cd /mnt/DATA/git/govoplan-core cd /mnt/DATA/git/govoplan-core
@@ -79,6 +91,8 @@ The smoke mode prints the effective config, runtime root, database URL, modules,
`requirements-dev.txt` links local GovOPlaN module checkouts for development. `requirements-release.txt` installs the packaged modules from tagged git refs for release builds. See [RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md). `requirements-dev.txt` links local GovOPlaN module checkouts for development. `requirements-release.txt` installs the packaged modules from tagged git refs for release builds. See [RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md).
For the install/runtime configuration contract and operator deployment flow, see [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md).
## WebUI development ## WebUI development
Install and run from the core WebUI host: Install and run from the core WebUI host:

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:db830bc22c5217ad6fedc13a --> <!-- codex-wiki-sync:e125a26630361b95cb8701d4 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/ACCESS_EXTRACTION_PLAN.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/ACCESS_EXTRACTION_PLAN.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -85,9 +85,9 @@ WebUI contributions without changing the core shell route wiring again.
Feature modules no longer import core auth dependency wrappers or access-owned Feature modules no longer import core auth dependency wrappers or access-owned
ORM models. Backend routers import the access-published FastAPI dependency API ORM models. Backend routers import the access-published FastAPI dependency API
from `govoplan_access.backend.auth.dependencies`; runtime cooperation uses from `govoplan_access.auth`; runtime cooperation uses kernel capabilities such
kernel capabilities such as `access.directory`, `campaigns.access`, as `access.directory`, `campaigns.access`, `campaigns.mailPolicyContext`, and
`campaigns.mailPolicyContext`, and `campaigns.deliveryTasks`. `campaigns.deliveryTasks`.
## Target Ownership ## Target Ownership
@@ -280,9 +280,8 @@ Tasks:
Acceptance criteria: Acceptance criteria:
- Auth behavior works through the access module. - Auth behavior works through the access module.
- Feature modules do not import access internals except the published - Feature modules do not import access internals. FastAPI routers use the
`govoplan_access.backend.auth.dependencies` dependency API used by FastAPI published `govoplan_access.auth` dependency API.
routers.
- Core can explain startup failure clearly if auth-required routes are enabled - Core can explain startup failure clearly if auth-required routes are enabled
without the access capability. without the access capability.
@@ -316,12 +315,14 @@ Remove direct ORM/model imports from files, mail, and campaign.
Tasks: Tasks:
- Replace `Group`, `Tenant`, `User`, and `UserGroupMembership` imports with - [x] Replace `Group`, `Tenant`, `User`, and `UserGroupMembership` imports with
directory/capability lookups. directory/capability lookups.
- Replace direct cross-module cleanup/count queries with registered providers, - [x] Replace direct cross-module cleanup/count queries with registered providers,
events, or module-owned API contracts. events, or module-owned API contracts.
- Replace mail-profile ownership resolution with stable owner references. - [x] Replace mail-profile ownership resolution with stable owner references.
- Replace campaign/file access checks with resource ACL provider contracts. - [x] Replace campaign/file access checks with resource ACL provider contracts.
- [x] Move feature routers from the old backend auth dependency path to the
public `govoplan_access.auth` dependency API.
Acceptance criteria: Acceptance criteria:
@@ -335,11 +336,11 @@ Finalize the split.
Tasks: Tasks:
- Remove obsolete core compatibility aliases. - [x] Remove obsolete core compatibility aliases.
- Keep the dependency-boundary checker allowlist empty. - [x] Keep the dependency-boundary checker allowlist empty.
- Update release dependency docs. - [x] Update release dependency docs.
- Update operator migration notes. - [x] Update operator migration notes.
- Add full module permutation tests including access-present and access-absent - [x] Add full module permutation tests including access-present and access-absent
behavior. behavior.
Acceptance criteria: Acceptance criteria:
@@ -377,7 +378,7 @@ Acceptance criteria:
- [x] Create `govoplan-access` repository workspace and issue workflow scaffold. - [x] Create `govoplan-access` repository workspace and issue workflow scaffold.
- [x] Create `govoplan-access` repository/package skeleton. - [x] Create `govoplan-access` repository/package skeleton.
- [x] Move `govoplan_core/access` seed package into `govoplan-access`. - [x] Move `govoplan_core/access` seed package into `govoplan-access`.
- [x] Add compatibility wrappers in core for old import paths. - [x] Remove compatibility wrappers in core for old import paths.
- [x] Route existing auth dependency wrappers through access principal/evaluator capabilities. - [x] Route existing auth dependency wrappers through access principal/evaluator capabilities.
- [x] Move FastAPI auth dependency wrappers out of core and into - [x] Move FastAPI auth dependency wrappers out of core and into
`govoplan-access`. `govoplan-access`.
@@ -447,7 +448,9 @@ Acceptance criteria:
capability? capability?
- Audit log storage lives in `govoplan-audit`; policy provenance can build on - Audit log storage lives in `govoplan-audit`; policy provenance can build on
that module boundary. that module boundary.
- How long should core keep compatibility route paths for existing clients? - Core no longer keeps compatibility import paths for access/auth/admin service
modules. HTTP route paths remain stable because the access manifest
contributes the same `/api/v1/auth/*` and `/api/v1/admin/*` routes.
## Verification ## Verification

@@ -0,0 +1,71 @@
<!-- codex-wiki-sync:ff49eae099d66fa2cc85476a -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/API_CONDITIONAL_DELTA.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# Conditional GET And Delta Collections
GovOPlaN uses two complementary mechanisms to reduce reload cost.
## Conditional GET
Core applies conditional GET handling centrally for successful JSON `GET`
responses:
- Responses receive a weak `ETag` based on the serialized JSON body.
- Responses are marked `Cache-Control: private, no-cache`.
- Responses vary by `Authorization`, `Cookie`, `X-API-Key`, and
`Accept-Language`.
- Matching `If-None-Match` requests return `304 Not Modified` without a body.
- Responses with `Set-Cookie`, `Content-Disposition`, `Content-Encoding`, a
non-JSON content type, a non-200 status, or `Cache-Control: no-store` are not
converted.
The WebUI `apiFetch` client keeps an in-memory conditional cache for reusable
safe requests. It sends `If-None-Match` after an endpoint has returned an ETag,
returns the cached payload on `304`, and clears the cache generation after
unsafe methods.
This avoids retransmitting unchanged snapshots. It does not identify which row
changed inside a collection.
## Delta Collections
Collection endpoints that can expose row-level changes should use the shared
delta contract instead of inventing module-specific formats.
Backend shape:
```json
{
"items": [],
"deleted": [],
"watermark": "opaque-next-watermark",
"has_more": false,
"full": false
}
```
Fields:
- `items`: changed or current items since the requested watermark.
- `deleted`: deleted item markers with at least `id`, and optionally `revision`
and `deleted_at`.
- `watermark`: opaque value the client sends as `since` on the next request.
- `has_more`: true when the client should request the next page with the
returned watermark.
- `full`: true when the response is a full snapshot rather than an incremental
delta.
Recommended query parameters:
- `since`: opaque previous watermark. If omitted or expired, return a full
snapshot with `full: true`.
- `limit`: maximum number of changed items plus deleted markers.
- `include_deleted`: whether deleted markers should be returned.
Modules should base watermarks on a monotonic revision, audit/event sequence, or
updated/deleted timestamp that is scoped to the same tenant and authorization
rules as the endpoint response.

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:36e407f77dd59c0601fa5063 --> <!-- codex-wiki-sync:b4b607a8906475ba1d120001 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/CATALOG_TRUST_AND_LICENSING.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/CATALOG_TRUST_AND_LICENSING.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -131,6 +131,39 @@ are rejected.
Catalogs should always expire. Long-lived catalogs make rollback and key Catalogs should always expire. Long-lived catalogs make rollback and key
compromise harder to reason about. compromise harder to reason about.
### Sequence-State Recovery
The sequence state file is operational state, not a trust root. Keep it on
persistent storage and include it in normal backups:
```json
{
"channels": {
"stable": {
"last_sequence": 42,
"accepted_at": "2026-07-07T12:00:00Z",
"key_id": "release-key-1",
"source": "https://govoplan.example/catalogs/v1/channels/stable.json"
}
}
}
```
If the file is lost, restore it from backup. If no backup exists, reconstruct
each channel from the highest sequence already accepted in installer run
records, release records, or the currently deployed module package set. Do not
lower `last_sequence` to make an older catalog pass; publish a new higher
sequence catalog when the accepted point is uncertain.
If the file is corrupted, copy it aside for incident review, validate the
current signed catalog with channel and freshness enforcement, then rewrite the
state with the known accepted sequence. Keep
`GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true` and approved-channel
checks enabled during recovery. Temporarily disabling
`GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE` allows revalidating the same
sequence, but older sequences remain rejected once the reconstructed
`last_sequence` is in place.
## Release Channels ## Release Channels
Approved channels are deployment policy: Approved channels are deployment policy:
@@ -169,10 +202,59 @@ License files are JSON objects with:
- `valid_until` - `valid_until`
- `signature` - `signature`
Issue or renew a license from an operator/release shell that has the Ed25519
private key:
```bash
govoplan-module-installer \
--issue-license /srv/govoplan/license.json \
--license-id customer-2026-07 \
--license-subject "Example Municipality" \
--license-feature module.mail \
--license-feature support.standard \
--license-valid-until 2027-07-31T23:59:59Z \
--license-signing-key-id license-issuer-1 \
--license-signing-private-key /srv/govoplan/secrets/license-issuer-1.pem \
--format json
```
Validate an imported license without exposing secrets:
```bash
govoplan-module-installer \
--validate-license /srv/govoplan/license.json \
--license-trusted-key license-issuer-1="<base64 public key>" \
--require-trusted-license \
--license-required-feature module.mail \
--format json
```
The CLI and admin module catalog panel report the license id, subject,
validity window, signing key id, signed/trusted state, available features, and
missing entitlements for the configured package catalog. They do not expose
private signing material.
License enforcement can run in observe-only mode by leaving License enforcement can run in observe-only mode by leaving
`GOVOPLAN_LICENSE_ENFORCEMENT` unset. In that mode, missing or invalid license `GOVOPLAN_LICENSE_ENFORCEMENT` unset. In that mode, missing or invalid license
data is surfaced as a warning but does not block planning. data is surfaced as a warning but does not block planning.
Renewal is an ordinary re-issuance with a new `license_id`, extended
`valid_until`, and the full intended feature set. Import the renewed JSON to
`GOVOPLAN_LICENSE_FILE`, keep the previous file for audit, and validate it
before setting enforcement.
Revocation is handled through the trusted license keyring. Mark a compromised
or invalid issuer key as `revoked` or `disabled`, publish or deploy the updated
keyring, then reissue affected licenses with an active key. Installations that
run with `GOVOPLAN_LICENSE_ENFORCEMENT=true` reject licenses signed only by a
revoked key after the local keyring is updated.
Emergency fallback is deliberately explicit. Operators can temporarily unset
`GOVOPLAN_LICENSE_ENFORCEMENT` to keep package planning observable while a
license or keyring is recovered. Record the change in the operational incident
log, keep catalog signature and channel enforcement enabled, and restore
license enforcement after a trusted renewal validates successfully.
Licensing is intentionally separate from open-source code licensing. The Licensing is intentionally separate from open-source code licensing. The
catalog/license mechanism can govern support channels, official release catalog/license mechanism can govern support channels, official release
eligibility, hosted update access, professional support, or commercial eligibility, hosted update access, professional support, or commercial
@@ -180,12 +262,10 @@ entitlements without changing the source license of the repositories.
## Production Gaps ## Production Gaps
The current implementation validates signed catalog metadata and offline The current implementation validates signed catalog metadata, offline license
license entitlements. Production-grade distribution still needs: entitlements, and local artifact SHA-256 when artifact paths are supplied in an
install plan. Production-grade distribution still needs:
- artifact digest verification for package archives or registry artifacts - remote registry/git artifact resolution before package-manager apply
- SBOM/provenance fields and verification workflow
- hardened catalog publishing pipeline in `govoplan-web` - hardened catalog publishing pipeline in `govoplan-web`
- automated key rotation runbook and emergency revocation procedure - automated key rotation runbook and emergency revocation procedure
- license issuance and renewal tooling
- tests for remote catalog cache fallback and replay-state upgrades

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:5279514d500c139b03703050 --> <!-- codex-wiki-sync:149be2f8b2427b8cda1f3a2b -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/CONFIGURATION_PACKAGES.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/CONFIGURATION_PACKAGES.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -111,6 +111,33 @@ export(selection, context) -> fragment, data_placeholders, warnings
health(import_result, context) -> diagnostics health(import_result, context) -> diagnostics
``` ```
The initial core contract lives in
`govoplan_core.core.configuration_packages`. Modules register providers through
module-specific capability keys and implement the `ConfigurationProvider`
protocol. The generic `configuration.provider` key names the contract and can be
used in package requirements; concrete providers such as `access.configuration`
are resolved by the admin package wizard. The first DTO surface includes package
manifests, module/capability requirements, module-owned fragments, diagnostics,
required operator data, dry-run plan items, apply results, export selections,
and export results.
The initial implementation includes provider-neutral orchestration helpers:
- `dry_run_configuration_package(...)`
- `apply_configuration_package(...)`
- `export_configuration_package(...)`
The first concrete provider is `govoplan_access.backend.configuration_provider`.
It supports access-owned `roles`, `groups`, and `group_role_assignments`
fragments and applies them idempotently.
The admin wizard backend starts with these routes:
- `GET /api/v1/admin/configuration-packages/catalog`
- `POST /api/v1/admin/configuration-packages/dry-run`
- `POST /api/v1/admin/configuration-packages/apply`
- `POST /api/v1/admin/configuration-packages/export`
## Import Flow ## Import Flow
1. Select a package from a trusted catalog or upload a package file. 1. Select a package from a trusted catalog or upload a package file.
@@ -182,6 +209,20 @@ Configuration catalogs should follow the existing module package catalog model:
a file-backed or remotely fetched JSON catalog with Ed25519 signatures, channel a file-backed or remotely fetched JSON catalog with Ed25519 signatures, channel
gating, trusted key ids, and operator-controlled trust policy. gating, trusted key ids, and operator-controlled trust policy.
The initial catalog validator mirrors the module catalog environment model with
configuration-specific names:
```bash
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG=/srv/govoplan/configuration-catalogs/stable.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_URL=https://govoplan.example/configuration-catalogs/stable.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/configuration-catalog-cache/stable.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_APPROVED_CHANNELS=stable,lts
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/configuration-catalog-keyring.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/configuration-catalog-sequences.json
GOVOPLAN_CONFIGURATION_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true
```
Catalog entries should include: Catalog entries should include:
- package id, version, name, description, publisher, tags, and channel - package id, version, name, description, publisher, tags, and channel

@@ -0,0 +1,310 @@
<!-- codex-wiki-sync:acb892c7bff57bb32a283613 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/DEPLOYMENT_OPERATOR_GUIDE.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# GovOPlaN Deployment Operator Guide
This guide defines the current install/runtime configuration contract and the
operator flow for a production-realistic self-hosted deployment. Keep secrets in
the deployment environment or a secret manager; do not commit populated `.env`
files.
## Runtime Configuration Contract
### Required Runtime Identity
| Setting | Required outside dev | Purpose |
| --- | --- | --- |
| `APP_ENV` | yes | Runtime profile. Use `prod`, `staging`, or a deployment-specific value outside local development. |
| `MASTER_KEY_B64` | yes | Fernet key or base64 encoded 32-byte key used for encrypted module secrets. Rotate through an explicit operator plan. |
| `DATABASE_URL` | yes | SQLAlchemy database URL for core and installed modules. SQLite is supported for dev/small installs; PostgreSQL is the preferred production target. |
| `ENABLED_MODULES` | yes | Comma-separated startup module set. Keep `tenancy,access` enabled; keep `admin` enabled for operator UI. |
Generate a local key for a new non-production environment:
```bash
python - <<'PY'
from cryptography.fernet import Fernet
print(Fernet.generate_key().decode())
PY
```
### Database And Migrations
| Setting | Default | Notes |
| --- | --- | --- |
| `DATABASE_URL` | `postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev` | Local development and production-like profiles use PostgreSQL. Use `GOVOPLAN_DEV_DATABASE_BACKEND=sqlite` only for disposable SQLite runs. |
| `DEV_AUTO_MIGRATE_ENABLED` | `true` | Dev convenience only. Production should run migration commands explicitly during deployment. |
| `DEV_BOOTSTRAP_ENABLED` | `false` | Dev bootstrap only. `govoplan_core.devserver` and `scripts/launch-dev.sh` default it to `true`; use controlled first-admin creation outside dev. |
Operator rule: take a database backup before applying migrations or destructive
module retirement. For non-SQLite databases, configure deployment-specific
backup/restore hooks for the module installer.
### PostgreSQL Production Target
PostgreSQL is the primary development and production target. SQLite remains
supported only for tiny disposable profiles and unit-test style smoke runs.
Production/staging deployments should use a managed PostgreSQL database and
explicit migration commands.
Install the server extra so the `psycopg` driver is available:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m pip install -r requirements-release.txt
```
Example runtime database URLs:
```bash
export DATABASE_URL='postgresql+psycopg://govoplan:change-me@db.example.internal:5432/govoplan'
export GOVOPLAN_DATABASE_URL_PGTOOLS='postgresql://govoplan:change-me@db.example.internal:5432/govoplan'
```
Use the SQLAlchemy URL for GovOPlaN. Use the pg-tools URL for `pg_dump`,
`pg_restore`, and `psql`; these tools do not understand the
`postgresql+psycopg://` driver marker.
Bootstrap or upgrade the schema explicitly during deployment:
```bash
export APP_ENV=prod
export ENABLED_MODULES=tenancy,access,admin,policy,audit,campaigns,files,mail,calendar,docs,ops
./.venv/bin/python -m govoplan_core.commands.init_db \
--database-url "$DATABASE_URL"
```
Backup and restore-check before migration-bearing package changes:
```bash
pg_dump --format=custom \
--file "$PWD/runtime/govoplan-$(date +%Y%m%d%H%M%S).dump" \
"$GOVOPLAN_DATABASE_URL_PGTOOLS"
pg_restore --list "$PWD/runtime/govoplan-YYYYMMDDHHMMSS.dump" >/dev/null
```
Restore a checked backup to the target database:
```bash
pg_restore --clean --if-exists \
--dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" \
"$PWD/runtime/govoplan-YYYYMMDDHHMMSS.dump"
```
For local development, create the host database described in
`dev/postgres/README.md`, then run:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m govoplan_core.commands.init_db \
--database-url postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev \
--with-dev-data
./.venv/bin/python -m govoplan_core.devserver --smoke --no-reload
scripts/launch-dev.sh
```
For disposable local validation against a throwaway PostgreSQL instance, use
the bundled PostgreSQL testbed:
```bash
cd /mnt/DATA/git/govoplan-core/dev/postgres
cp .env.example .env
docker compose --env-file .env up -d
cd /mnt/DATA/git/govoplan-core
set -a
. dev/postgres/.env
set +a
./.venv/bin/python scripts/postgres-integration-check.py \
--database-url "$GOVOPLAN_POSTGRES_DATABASE_URL" \
--reset-schema
```
The integration check runs migrations and startup smoke checks across the
standard module permutations. `--reset-schema` is destructive and belongs only
on throwaway databases.
### Broker And Workers
| Setting | Default | Notes |
| --- | --- | --- |
| `REDIS_URL` | `redis://redis:6379/0` | Celery broker/result backend when async workers are enabled. |
| `CELERY_ENABLED` | `false` | Local/dev can send synchronously. Production campaign delivery should run workers and set this to `true`. |
| `CELERY_QUEUES` | `send_email,append_sent,default` | Queue list expected by worker/process manager definitions. |
Worker command:
```bash
python -m celery -A govoplan_core.celery_app:celery worker \
--queues send_email,append_sent,default \
--loglevel INFO
```
### Storage
| Setting | Default | Notes |
| --- | --- | --- |
| `FILE_STORAGE_BACKEND` | `local` | Use `local` for dev/small deployments; use object storage when files must scale independently. |
| `FILE_STORAGE_LOCAL_ROOT` | `runtime/files` | Must live on durable storage and be backed up when `FILE_STORAGE_BACKEND=local`. |
| `FILE_STORAGE_LOCAL_FALLBACK_ROOTS` | empty | Read-only fallback roots for migrated local files. |
| `FILE_STORAGE_S3_ENDPOINT_URL` | empty | Object-store endpoint for the files module. |
| `FILE_STORAGE_S3_REGION` | empty | Object-store region. |
| `FILE_STORAGE_S3_ACCESS_KEY_ID` | empty | Secret; inject through deployment environment. |
| `FILE_STORAGE_S3_SECRET_ACCESS_KEY` | empty | Secret; inject through deployment environment. |
| `FILE_STORAGE_S3_BUCKET` | `files` | Managed-file object bucket. |
Legacy `S3_*` settings remain for older storage paths but new deployments should
prefer `FILE_STORAGE_*`.
### HTTP, Cookies, And Base URLs
| Setting | Default | Notes |
| --- | --- | --- |
| `CORS_ORIGINS` | local dev origins | Set to the exact WebUI origins in staging/production. |
| `AUTH_SESSION_COOKIE_NAME` | `msm_session` | Change only through a controlled rollout because it logs users out. |
| `AUTH_CSRF_COOKIE_NAME` | `msm_csrf` | Must match WebUI/API deployment. |
| `AUTH_COOKIE_SECURE` | `false` | Set `true` behind HTTPS. |
| `AUTH_COOKIE_SAMESITE` | `lax` | Use a stricter value only after testing login and CSRF flows. |
| `AUTH_COOKIE_DOMAIN` | empty | Set only when the API and WebUI intentionally share a parent domain. |
Public URLs are currently supplied by deployment/reverse-proxy configuration and
module settings. Do not hardcode them in core; configuration packages should ask
for portal, WebUI, postbox, and notification URLs when they become relevant.
### Module Catalogs, Licenses, And Trust Roots
| Setting | Purpose |
| --- | --- |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_URL` or `GOVOPLAN_MODULE_PACKAGE_CATALOG` | Module package catalog source. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE` | Preferred production keyring path. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL` | Approved catalog channel, for example `stable`. |
| `GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE` | Trusted license issuer keyring path. |
| `GOVOPLAN_LICENSE_ENFORCEMENT` | Enables license enforcement when set to `true`. |
Trust roots are deployment-managed and should not be editable through the
running WebUI.
### Mail Test Credentials
Dedicated SMTP/IMAP test credentials belong to the mail/campaign test-bed
configuration, not the core runtime contract. Store them in a local ignored
`.env` file for the test bed or in CI secrets. Required values are:
- SMTP host, port, TLS mode, username, password, and envelope/from address.
- IMAP host, port, TLS mode, username, password, and append folder.
- At least one recipient mailbox that is safe for automated send tests.
## First Deployment Flow
1. Create an environment file or secret set with the runtime contract above.
2. Install the tagged core and module packages from `requirements-release.txt`.
3. Build the WebUI from `webui/package.release.json` or deploy a prebuilt
artifact from the same release tag.
4. Run database migrations with the target `DATABASE_URL`.
5. Create the first tenant and system owner through the controlled bootstrap or
one-time admin command for the deployment.
6. Start the API service with `govoplan_core.server.app:app`.
7. Start workers when `CELERY_ENABLED=true`.
8. Start the WebUI/reverse proxy and verify CORS/cookie settings.
9. Open Admin > System > Modules, verify enabled modules, and save desired
module state if it differs from `ENABLED_MODULES`.
10. Run health checks:
```bash
curl -fsS http://127.0.0.1:8000/health
curl -fsS http://127.0.0.1:8000/api/v1/platform/modules
```
Authenticated health details require `system:settings:read`:
```bash
curl -fsS -H "X-API-Key: $GOVOPLAN_HEALTH_API_KEY" \
http://127.0.0.1:8000/health/details
```
## Production-Like Dev Profile
Use this profile to verify deployment behavior without publishing packages or
using real production credentials. The canonical launcher keeps API, worker, and
WebUI code in the editable repositories while Docker provides PostgreSQL and
Redis:
```bash
cd /mnt/DATA/git/govoplan-core
scripts/launch-production-like-dev.sh
```
The launcher uses `dev/production-like/.env` when present, otherwise the checked
in `.env.example`. It runs:
- PostgreSQL on `127.0.0.1:55433`
- Redis on `127.0.0.1:56379`
- explicit `ENABLED_MODULES`
- explicit migrations and `--with-dev-data` bootstrap
- API via the module-aware devserver
- a Celery worker for `send_email,append_sent,default`
- WebUI through the Vite dev server
- durable local files under `runtime/production-like/files`
This profile validates explicit migration execution, config loading, module
discovery, route aggregation, local storage paths, Redis broker connectivity,
worker heartbeats, and health/readiness startup without real production
credentials. It does not replace a managed PostgreSQL/Redis/WebUI/worker
deployment test.
To stop PostgreSQL and Redis when the launcher exits:
```bash
GOVOPLAN_STOP_PROFILE_DEPENDENCIES_ON_EXIT=1 scripts/launch-production-like-dev.sh
```
## Module Install/Uninstall Operations
Use Admin > System > Modules for planning. Use the operator shell for package
changes:
```bash
govoplan-module-installer --format shell
govoplan-module-installer --apply --build-webui
```
For production-like runs, prefer supervised mode with restart and health checks:
```bash
govoplan-module-installer \
--supervise \
--migrate \
--restart-command 'systemctl restart govoplan-api' \
--restart-command 'systemctl restart govoplan-worker' \
--health-url http://127.0.0.1:8000/health \
--database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL_PGTOOLS" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \
--database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" "$GOVOPLAN_DATABASE_BACKUP_PATH"'
```
Run the rollback drill before relying on installer automation in a new
environment:
```bash
./.venv/bin/python scripts/module-installer-rollback-drill.py --format json
```
See `RELEASE_DEPENDENCIES.md` for release package refs, catalog trust,
installer daemon operation, rollback records, and destructive module retirement.
## Operator Checklist
- Runtime secrets are injected outside git.
- `MASTER_KEY_B64` is set and backed up securely.
- Database backup and restore commands are tested.
- File/object storage is durable and backed up.
- `CORS_ORIGINS` and cookie settings match the deployed WebUI origin.
- Redis and workers are running before `CELERY_ENABLED=true`.
- Module catalog and license keyrings are pinned locally.
- Health endpoints are monitored.
- Test SMTP/IMAP credentials are non-production and isolated.
- Module installer rollback drill has passed in the deployment environment.

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:cabd2e17a62a5748808a62fa --> <!-- codex-wiki-sync:ab273ee5f8cfbb6ac4fed453 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/EVENTS_AND_AUDIT.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/EVENTS_AND_AUDIT.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -13,18 +13,49 @@ there is a concrete need for durable asynchronous command orchestration. Events
are facts about completed work and are safe for audit, projections, optional are facts about completed work and are safe for audit, projections, optional
module reactions, and operator diagnostics. module reactions, and operator diagnostics.
## Current Decision ## Production Transport Decision
The first production target is a **database outbox plus in-process immediate
dispatch**:
- Use `govoplan_core.core.events.PlatformEvent` for domain and platform events. - Use `govoplan_core.core.events.PlatformEvent` for domain and platform events.
- Use `EventBus` as the in-process dispatch contract for immediate module - Use `EventBus` as the in-process dispatch contract for same-process module
reactions. reactions that are safe to run inline.
- Keep durable dispatch out of the first API shape. A database outbox can wrap - Persist durable integration/workflow events through a database outbox before
the same `PlatformEvent` envelope later without changing event producers. acknowledging the state change that produced them.
- Drain the outbox through a small dispatcher process. The dispatcher may call
in-process handlers in the same deployment first, but its storage contract is
database-backed.
- Treat Redis/Celery as worker/job infrastructure, not as the authoritative
first event transport. A Celery dispatcher can consume the outbox later.
- Keep the dispatch implementation pluggable behind the `PlatformEvent`
envelope so a future message broker can be added without changing event
producers.
- Keep commands out of the kernel until workflows need retryable, durable, - Keep commands out of the kernel until workflows need retryable, durable,
operator-visible command records. operator-visible command records.
This keeps the first contract small while preserving room for a DB outbox, This keeps the first contract small, PostgreSQL-friendly, auditable, and
Redis/Celery dispatch, or workflow-owned command orchestration later. recoverable after process crashes. It also avoids making Redis a correctness
dependency for deployments that only need synchronous mail/tests or light
background work.
## Dispatch Semantics
Event producers should write their domain state and outbox event in the same
database transaction wherever possible. Handlers must be idempotent because the
outbox dispatcher can retry after a crash or timeout.
Recommended first outbox columns:
- `event_id`, `event_type`, `module_id`
- `correlation_id`, `causation_id`
- `payload`, `occurred_at`
- `available_at`, `attempt_count`, `claimed_at`, `claim_token`
- `processed_at`, `last_error`
Inline `EventBus` handlers are allowed only for non-critical local reactions.
Anything that must survive process failure, restart, package update, or worker
redeployment belongs in the outbox.
## Trace IDs ## Trace IDs
@@ -43,6 +74,12 @@ Audit logging reads the current event context and stores trace IDs in
`details._trace`. Callers can also pass explicit `correlation_id` and `details._trace`. Callers can also pass explicit `correlation_id` and
`causation_id` to `audit_event` or `audit_from_principal`. `causation_id` to `audit_event` or `audit_from_principal`.
Admin and lifecycle code should use the compact operational detail shape
documented in `govoplan-audit/docs/AUDIT_TRACE_CONTEXT.md`. The core
`audit_operation_context` helper preserves `module_id`, `request_id`, `run_id`,
`outcome`, and `_trace` while applying the shared audit redaction pass to
additional detail values.
## Audit MVP Boundary ## Audit MVP Boundary
`govoplan-audit` owns: `govoplan-audit` owns:

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:481fe8055b18243ca72a351b --> <!-- codex-wiki-sync:306aa8e736ffb9b4d13f1eca -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/GOVERNMENT_OPERATIONS_VISION.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/GOVERNMENT_OPERATIONS_VISION.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -97,6 +97,11 @@ The output should be a ranked connector backlog with owners, protocols,
authentication models, data shapes, operational risks, and minimum viable authentication models, data shapes, operational risks, and minimum viable
integration tests. integration tests.
The core classification index lives in
`docs/PUBLIC_SECTOR_INTEGRATION_STRATEGY.md`; the detailed connector catalogue
and executable target inventory live in
`govoplan-connectors/docs/PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md`.
## Recurring Data Extraction And Transformation ## Recurring Data Extraction And Transformation
A concrete recurring use case is monthly extraction and transformation of data A concrete recurring use case is monthly extraction and transformation of data
@@ -182,6 +187,10 @@ Sizing should account for tenants, concurrent users, cases, files, storage
volume, upload/download rates, campaign volume, workflow jobs, transformation volume, upload/download rates, campaign volume, workflow jobs, transformation
runs, reporting load, and retention/audit growth. runs, reporting load, and retention/audit growth.
See `docs/SCALABILITY_AND_SIZING.md` for the first deployment topology,
readiness/degraded-mode model, worker scaling assumptions, PostgreSQL production
path, and sizing matrix.
## Hardware Requirements ## Hardware Requirements
Hardware guidance should be empirical and deployment-shaped. The first sizing Hardware guidance should be empirical and deployment-shaped. The first sizing
@@ -197,6 +206,9 @@ Each profile should document CPU, memory, disk, database, storage, queue/cache,
backup, and monitoring requirements, plus the assumptions behind the numbers. backup, and monitoring requirements, plus the assumptions behind the numbers.
A later calculator can turn operator inputs into recommended profiles. A later calculator can turn operator inputs into recommended profiles.
The initial matrix lives in `docs/SCALABILITY_AND_SIZING.md`; the calculator is
kept as a follow-up once real deployment measurements are available.
## Collaboration ## Collaboration
Collaboration should be integration-first. GovOPlaN should connect with strong Collaboration should be integration-first. GovOPlaN should connect with strong

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:d50d3a857949d55597475417 --> <!-- codex-wiki-sync:2ad17e6a9dae70657f3db355 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MODULE_ROADMAP.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MODULE_ROADMAP.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -7,7 +7,11 @@
--- ---
# GovOPlaN Module And Integration Roadmap # GovOPlaN Module And Integration Roadmap
This page maps current module and integration ideas to existing GovOPlaN repositories or to missing-module decisions. Issues are the active backlog. This document is durable routing context and should be mirrored to the Gitea wiki. This page maps current module and integration ideas to existing GovOPlaN
repositories or to explicit missing-module decisions. Issues are the active
backlog. This document is durable routing context and should be mirrored to the
Gitea wiki. Boundary decisions are recorded in
`docs/MODULE_BOUNDARY_DECISIONS.md`.
## Current Routing ## Current Routing
@@ -18,48 +22,78 @@ This page maps current module and integration ideas to existing GovOPlaN reposit
| Fully UI-managed configuration with safety controls | `govoplan-admin`, `govoplan-core`, `govoplan-policy`, `govoplan-access`, `govoplan-audit` | `add-ideas/govoplan-core#218` | | Fully UI-managed configuration with safety controls | `govoplan-admin`, `govoplan-core`, `govoplan-policy`, `govoplan-access`, `govoplan-audit` | `add-ideas/govoplan-core#218` |
| Access as a module | `govoplan-access` | `add-ideas/govoplan-access#7` | | Access as a module | `govoplan-access` | `add-ideas/govoplan-access#7` |
| OpenProject API / project management connector | `govoplan-connectors` | `add-ideas/govoplan-connectors#1` | | OpenProject API / project management connector | `govoplan-connectors` | `add-ideas/govoplan-connectors#1` |
| Native project-management module decision | `govoplan-core` | `add-ideas/govoplan-core#196` | | Native project-management module decision | connector-first through `govoplan-connectors`; no native project module yet | `add-ideas/govoplan-core#196`, `add-ideas/govoplan-connectors#1` |
| Datasources for databases, CSV, files, APIs | proposed `govoplan-datasources` | `add-ideas/govoplan-core#197` | | Datasources for databases, CSV, files, APIs | no repository yet; start with connectors/files/reporting and create `govoplan-datasources` only after the first package proves shared source-catalog ownership | `add-ideas/govoplan-core#197` |
| Dataflow for pipelines, BI, publication | proposed `govoplan-dataflow` | `add-ideas/govoplan-core#198` | | Dataflow for pipelines, BI, publication | no repository yet; start with workflow/reporting/connectors and create `govoplan-dataflow` only after repeated pipeline/lineage contracts emerge | `add-ideas/govoplan-core#198` |
| Monthly datasource and transformation workflows | proposed `govoplan-datasources`, proposed `govoplan-dataflow`, `govoplan-workflow`, `govoplan-reporting` | `add-ideas/govoplan-core#216` | | Monthly datasource and transformation workflows | first as configuration package across connectors, files, workflow, reporting, and templates | `add-ideas/govoplan-core#216` |
| Templates for letters, emails, forms, reports | `govoplan-templates` | `add-ideas/govoplan-templates#1` | | Templates for letters, emails, forms, reports | `govoplan-templates`, separate from reporting | `add-ideas/govoplan-core#190`, `add-ideas/govoplan-templates#1` |
| Reporting and BI | `govoplan-reporting` | `add-ideas/govoplan-reporting#1` | | Reporting and BI | `govoplan-reporting`, separate from templates | `add-ideas/govoplan-core#190`, `add-ideas/govoplan-reporting#1` |
| File connectors: Nextcloud, Seafile, SMB, NFS | `govoplan-files` | `add-ideas/govoplan-files#15` | | File connectors: Nextcloud, Seafile, SMB, NFS | `govoplan-files` | `add-ideas/govoplan-files#15` |
| Public-sector software integration catalogue | `govoplan-connectors` | `add-ideas/govoplan-connectors#2` | | Public-sector software integration catalogue | `govoplan-connectors` with core strategy index | `add-ideas/govoplan-core#191`, `add-ideas/govoplan-connectors#2` |
| Public-sector integration landscape catalogue | `govoplan-connectors` with core tracking | `add-ideas/govoplan-core#215` | | Public-sector integration landscape catalogue | `govoplan-connectors` with core tracking | `add-ideas/govoplan-core#215` |
| Cases module concept | `govoplan-cases` | `add-ideas/govoplan-core#174` |
| Workflow module concept | `govoplan-workflow` | `add-ideas/govoplan-core#175` |
| Connectors module concept | `govoplan-connectors` | `add-ideas/govoplan-core#176` |
| Adrema-style address and distribution-list management | `govoplan-addresses` | `add-ideas/govoplan-addresses#1` | | Adrema-style address and distribution-list management | `govoplan-addresses` | `add-ideas/govoplan-addresses#1` |
| Consume sources and become a governed source | `govoplan-connectors` plus proposed `govoplan-dataflow` | `add-ideas/govoplan-connectors#3`, `add-ideas/govoplan-core#198` | | Consume sources and become a governed source | `govoplan-connectors` plus proposed `govoplan-dataflow` | `add-ideas/govoplan-connectors#3`, `add-ideas/govoplan-core#198` |
| Terminfindung and meeting scheduling polls | `govoplan-scheduling` | local scaffold; remote creation tracked by `add-ideas/govoplan-core#199` | | Terminfindung and meeting scheduling polls | `govoplan-scheduling`; calendar primitives remain in calendar | `add-ideas/govoplan-core#193`, `add-ideas/govoplan-scheduling#1` |
| Terminplaner and calendar primitives | `govoplan-calendar` | `add-ideas/govoplan-calendar#1` | | Terminplaner and calendar primitives | `govoplan-calendar` | `add-ideas/govoplan-core#193`, `add-ideas/govoplan-calendar#1` |
| Terminbuchung appointment booking | `govoplan-appointments` | `add-ideas/govoplan-appointments#1` | | Terminbuchung appointment booking | `govoplan-appointments` | `add-ideas/govoplan-core#193`, `add-ideas/govoplan-appointments#1` |
| Collaborative documents | `govoplan-dms` | `add-ideas/govoplan-dms#1` | | Collaborative documents | `govoplan-dms` | `add-ideas/govoplan-dms#1` |
| Forms | `govoplan-forms` | `add-ideas/govoplan-forms#1` | | Forms | `govoplan-forms` for definitions and `govoplan-forms-runtime` for submissions/runtime behavior | `add-ideas/govoplan-core#194`, `add-ideas/govoplan-forms#1` |
| RSS consume and emit | `govoplan-connectors` | `add-ideas/govoplan-connectors#4` | | RSS consume and emit | `govoplan-connectors` | `add-ideas/govoplan-connectors#4` |
| LDAP, Active Directory, OpenDesk identity | `govoplan-idm` | `add-ideas/govoplan-idm#1` | | LDAP, Active Directory, OpenDesk identity | `govoplan-idm` | `add-ideas/govoplan-idm#1` |
| OpenDesk stack integration map | `govoplan-connectors` | `add-ideas/govoplan-connectors#5` | | OpenDesk stack integration map | integration profile across IDM/access, mail/calendar, files/DMS, and connectors; not a monolithic module | `add-ideas/govoplan-core#195`, `add-ideas/govoplan-connectors#5` |
| Open-Xchange mail/groupware | `govoplan-mail` | `add-ideas/govoplan-mail#5` | | Open-Xchange mail/groupware | `govoplan-mail` | `add-ideas/govoplan-mail#5` |
| Open-Xchange calendar | `govoplan-calendar` | `add-ideas/govoplan-calendar#2` | | Open-Xchange calendar | `govoplan-calendar` | `add-ideas/govoplan-calendar#2` |
| Scalability profiles and autoscaling readiness | `govoplan-ops`, `govoplan-core` | `add-ideas/govoplan-core#217` | | Scalability profiles and autoscaling readiness | `govoplan-ops`, `govoplan-core` | `add-ideas/govoplan-core#217` |
| Hardware sizing matrix and requirements calculator | `govoplan-ops`, `govoplan-core` | `add-ideas/govoplan-core#219` | | Hardware sizing matrix and requirements calculator | `govoplan-ops`, `govoplan-core` | `add-ideas/govoplan-core#219` |
| Collaboration suite integration strategy | `govoplan-connectors`, `govoplan-dms`, `govoplan-workflow`, `govoplan-tasks`, `govoplan-appointments`, `govoplan-calendar` | `add-ideas/govoplan-core#220` | | Collaboration suite integration strategy | `govoplan-connectors`, `govoplan-dms`, `govoplan-workflow`, `govoplan-tasks`, `govoplan-appointments`, `govoplan-calendar` | `add-ideas/govoplan-core#220` |
| Install/runtime configuration contract | `govoplan-core`, later `govoplan-ops` | `add-ideas/govoplan-core#19` |
| Installer/deployment operator flow | `govoplan-core`, later `govoplan-ops` | `add-ideas/govoplan-core#26` |
| Production-like deployment documentation | `govoplan-core`, later `govoplan-ops` | `add-ideas/govoplan-core#28` |
## Proposed Missing Modules ## Proposed Missing Modules
`govoplan-datasources` should exist only if source catalog ownership becomes broad enough to justify a module separate from connectors and reporting. Candidate responsibilities: ## Boundary Decision Register
`docs/MODULE_BOUNDARY_DECISIONS.md` is the durable decision register for older
roadmap and boundary issues. Current decisions:
- templates and reporting are separate modules
- RSS/source consume-publish starts in connectors; datasources/dataflow are not
repositories yet
- calendar, scheduling, and appointments are three separate modules
- forms definitions and forms runtime are separate responsibilities
- OpenDesk is an integration profile across modules, not a monolithic module
- OpenProject is connector-first; no native projects module yet
- public-sector integration strategy stays in core; executable catalogue work
lives in connectors
## Proposed Missing Modules
`govoplan-datasources` is not created yet. It should exist only if source
catalog ownership becomes broad enough to justify a module separate from
connectors, files, and reporting. Candidate responsibilities:
- source catalogue for SQL databases, CSV/Excel files, uploaded files, APIs, RSS feeds, and governed file locations - source catalogue for SQL databases, CSV/Excel files, uploaded files, APIs, RSS feeds, and governed file locations
- credentials and connection profiles - credentials and connection profiles
- schema discovery and refresh cadence - schema discovery and refresh cadence
- provenance, freshness, permission boundaries, and audit events - provenance, freshness, permission boundaries, and audit events
`govoplan-dataflow` should exist only if pipelines and publication become first-class product behavior. Candidate responsibilities: `govoplan-dataflow` is not created yet. It should exist only if pipelines and
publication become first-class product behavior beyond workflow, connectors,
and reporting. Candidate responsibilities:
- ingestion, transformation, validation, scheduling, and lineage - ingestion, transformation, validation, scheduling, and lineage
- connecting datasources to reports, APIs, RSS, exports, and downstream systems - connecting datasources to reports, APIs, RSS, exports, and downstream systems
- the "consume sources, become source" lifecycle - the "consume sources, become source" lifecycle
- publication-state and audit integration - publication-state and audit integration
`govoplan-projects` is undecided. The default path should be an OpenProject connector first. A native module is justified only if GovOPlaN must own project semantics beyond cases, tasks, workflow, appointments, documents, and reporting. `govoplan-projects` is intentionally not created yet. The decision is
connector-first through OpenProject. A native module is justified only if
GovOPlaN must own project semantics beyond cases, tasks, workflow,
appointments, documents, and reporting.
## Boundary Notes ## Boundary Notes
@@ -74,6 +108,18 @@ This page maps current module and integration ideas to existing GovOPlaN reposit
- `govoplan-reporting` owns report definitions, BI views, scheduled outputs, and export targets. - `govoplan-reporting` owns report definitions, BI views, scheduled outputs, and export targets.
- `govoplan-templates` owns reusable renderable templates, not data selection or persistence. - `govoplan-templates` owns reusable renderable templates, not data selection or persistence.
## Integration Catalogue Routing
Core keeps the strategy index in
`docs/PUBLIC_SECTOR_INTEGRATION_STRATEGY.md`: integration postures, default
ownership, and prioritization rules. `govoplan-connectors` owns the detailed
target inventory and connector entry shape in
`govoplan-connectors/docs/PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md`.
When a target needs executable behavior, create the implementation issue in the
owning module repository and keep only the cross-module routing or architecture
decision in core.
## Release Tooling Note ## Release Tooling Note
`scripts/push-release-tag.sh` covers the released full-product package set: `scripts/push-release-tag.sh` covers the released full-product package set:
@@ -91,4 +137,6 @@ This page maps current module and integration ideas to existing GovOPlaN reposit
It also includes existing roadmap/scaffold module repositories such as addresses, appointments, connectors, DMS, forms, IDM, reporting, scheduling, templates, workflow, XOE/V, and XRechnung as tag-only repositories. Tag-only repositories are committed, tagged, and pushed with the same release tag, but they are not added to `requirements-release.txt` or `webui/package.release.json` until they contain installable package metadata. It also includes existing roadmap/scaffold module repositories such as addresses, appointments, connectors, DMS, forms, IDM, reporting, scheduling, templates, workflow, XOE/V, and XRechnung as tag-only repositories. Tag-only repositories are committed, tagged, and pushed with the same release tag, but they are not added to `requirements-release.txt` or `webui/package.release.json` until they contain installable package metadata.
Proposed modules without repositories, such as `govoplan-datasources`, `govoplan-dataflow`, and possibly `govoplan-projects`, cannot be included until their repositories are created. Proposed modules without repositories, such as `govoplan-datasources`,
`govoplan-dataflow`, and `govoplan-projects`, cannot be included until a later
decision creates their repositories.

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:24b67ceb3231203599e59944 --> <!-- codex-wiki-sync:17655e955d797a78fafc4a99 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -13,6 +13,12 @@ The current package name is still `govoplan-core`, but the architecture target i
The concrete access/auth/RBAC extraction path is tracked in The concrete access/auth/RBAC extraction path is tracked in
[`ACCESS_EXTRACTION_PLAN.md`](ACCESS_EXTRACTION_PLAN.md). [`ACCESS_EXTRACTION_PLAN.md`](ACCESS_EXTRACTION_PLAN.md).
The event and audit trace contract is tracked in
[`EVENTS_AND_AUDIT.md`](EVENTS_AND_AUDIT.md).
Policy decision, source provenance, and explain-response contracts are tracked
in [`POLICY_CONTRACTS.md`](POLICY_CONTRACTS.md).
The experimental remote WebUI bundle loading design is tracked in
[`REMOTE_WEBUI_BUNDLES.md`](REMOTE_WEBUI_BUNDLES.md).
## Layer Model ## Layer Model
@@ -42,14 +48,15 @@ The kernel must not own product semantics such as users, tenants, RBAC decisions
## Current Compatibility Responsibilities ## Current Compatibility Responsibilities
During the staged split, `govoplan-core` still contains compatibility surfaces During the staged split, `govoplan-core` still contains compatibility surfaces
for access, auth, tenancy, RBAC, governance, audit, CSRF/API helpers, and for tenancy settings, governance/policy contracts, audit helpers, CSRF/API
secret helpers. The extracted access implementation now lives in helpers, and secret helpers. The extracted access implementation lives in
`govoplan-access`; live legacy ORM table definitions have been split across `govoplan-access`; live legacy ORM table definitions have been split across
their platform owners while retaining historical table names. The old core their platform owners while retaining historical table names. The old core
model, route, admin-service, and access-security import shims have been route, admin-service, and access-security re-export modules have been removed.
removed; callers must use module-owned imports or kernel capabilities. The Callers must use module-owned imports, the public `govoplan_access.auth` request
remaining compatibility surfaces are temporary until the matching platform dependency API, or kernel capabilities.
modules are fully self-contained: The remaining platform compatibility surfaces are temporary until the matching
platform modules are fully self-contained:
- `govoplan-access` - `govoplan-access`
- `govoplan-tenancy` - `govoplan-tenancy`
@@ -78,6 +85,7 @@ The following contracts are the baseline API that modules can rely on:
- WebUI module contribution contract - WebUI module contribution contract
- navigation metadata contract - navigation metadata contract
- command/event envelope contract - command/event envelope contract
- policy decision and source provenance contract in `govoplan_core.core.policy`
Changes to these contracts must be versioned or accompanied by compatibility shims. Changes to these contracts must be versioned or accompanied by compatibility shims.
@@ -106,10 +114,10 @@ access/tenant ORM models when they need labels, group membership, default
access provisioning, counts, audit actor labels, or tenant metadata. access provisioning, counts, audit actor labels, or tenant metadata.
FastAPI route dependencies for authenticated endpoints are access-owned and FastAPI route dependencies for authenticated endpoints are access-owned and
published from `govoplan_access.backend.auth.dependencies`. Routers may import published from `govoplan_access.auth`. Routers may import that public API for
that dependency module directly until a more generic request-principal adapter `ApiPrincipal`, `get_api_principal`, `has_scope`, `require_scope`, and
exists; they must not import access ORM models or other access implementation `require_any_scope`; they must not import access ORM models or
internals. `govoplan_access.backend.*` implementation internals.
Current live table ownership: Current live table ownership:
@@ -216,6 +224,10 @@ Rules:
- Optional module migrations may create multiple Alembic heads. Verification - Optional module migrations may create multiple Alembic heads. Verification
should compare the database heads to the configured script heads instead of should compare the database heads to the configured script heads instead of
assuming one linear revision when multiple modules are enabled. assuming one linear revision when multiple modules are enabled.
- Treat migrations as release artifacts. Unreleased migrations may be squashed
or rewritten before a stable release; released revision IDs are immutable
once an installation may have recorded them. Each stable release records its
public migration heads in `docs/migration-release-baselines.json`.
## Install, Uninstall, And Catalogs ## Install, Uninstall, And Catalogs
@@ -361,8 +373,7 @@ The repository includes `scripts/check_dependency_boundaries.py`. It enforces th
- access source may not import files/mail/campaign internals - access source may not import files/mail/campaign internals
- feature modules may not import access implementation internals - feature modules may not import access implementation internals
- feature modules may not add new direct imports of sibling feature modules - feature modules may not add new direct imports of sibling feature modules
- FastAPI routers may import the published - FastAPI routers may import the published `govoplan_access.auth` dependency API
`govoplan_access.backend.auth.dependencies` dependency API
- the transitional allowlist is expected to stay empty - the transitional allowlist is expected to stay empty
Any future exception is extraction debt and must be temporary, documented in the Any future exception is extraction debt and must be temporary, documented in the
@@ -572,6 +583,8 @@ directory with:
- `GOVOPLAN_INSTALLER_RUN_DIR` - `GOVOPLAN_INSTALLER_RUN_DIR`
- `GOVOPLAN_DATABASE_URL` - `GOVOPLAN_DATABASE_URL`
- `GOVOPLAN_DATABASE_URL_PGTOOLS` for PostgreSQL URLs converted to the
`postgresql://` form expected by `pg_dump`, `pg_restore`, and `psql`
- `GOVOPLAN_DATABASE_BACKUP_PATH` - `GOVOPLAN_DATABASE_BACKUP_PATH`
- `GOVOPLAN_DATABASE_BACKUP_METADATA` - `GOVOPLAN_DATABASE_BACKUP_METADATA`

@@ -0,0 +1,215 @@
<!-- codex-wiki-sync:57b6aad7bb1abb07552bb601 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_BOUNDARY_DECISIONS.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# Module Boundary Decisions
This document records durable boundary decisions that close older exploratory
core issues. Implementation work should move to the owning module repositories
once a boundary is clear.
## Decision Principles
- Prefer connector-first when an external specialist system is likely to remain
the system of record.
- Create a native module only when GovOPlaN must own domain semantics,
permissions, audit, retention, configuration-package fragments, or workflow
state.
- Keep optional behavior behind core-mediated capabilities, events, DTOs, route
contributions, and UI contribution points.
- Do not create repositories just because a possible product area exists.
## Templates And Reporting
Tracking: `govoplan-core#190`, `govoplan-templates#1`,
`govoplan-reporting#1`.
Decision: templates and reporting are separate modules.
`govoplan-templates` owns:
- reusable renderable templates for letters, permits, emails, forms, reports,
certificates, and notices
- template versioning, merge-field declarations, rendering profiles, output
format choices, and preview contracts
- template package fragments that other modules can reference
`govoplan-reporting` owns:
- report definitions, data selection, dashboards, BI views, scheduled outputs,
and export targets
- report permissions, report execution history, generated report evidence, and
report-specific retention inputs
- downstream export handoff to files, dataflow, connectors, or publication
surfaces
Boundary:
- Templates do not own data selection, aggregation, scheduling, or BI semantics.
- Reporting may call template rendering through a capability when a formatted
report output is needed.
- Campaign, mail, files, workflow, and cases use templates/reporting through
capabilities and DTOs, never direct imports.
## Sources, RSS, Datasources, And Dataflow
Tracking: `govoplan-core#192`, `govoplan-core#197`,
`govoplan-core#198`, `govoplan-connectors#3`,
`govoplan-connectors#4`.
Decision: do not create `govoplan-datasources` or `govoplan-dataflow` until a
first executable use case proves that connector/reporting/workflow ownership is
too narrow.
First slice:
- `govoplan-connectors` owns RSS/Atom consume/emit connector profiles,
connector health, external references, source lifecycle metadata, and
source/publish capability boundaries.
- `govoplan-files` owns file-backed governed locations and uploaded/stored file
evidence.
- `govoplan-reporting` owns report/data views and scheduled outputs.
- `govoplan-workflow` owns process state, approvals, scheduling of process
steps, and human review.
Future `govoplan-datasources` is justified when GovOPlaN needs a broad source
catalogue for SQL databases, CSV/Excel files, APIs, RSS feeds, uploaded files,
and governed file locations with shared ownership, credentials, schema
discovery, refresh cadence, provenance, and permission boundaries.
Future `govoplan-dataflow` is justified when GovOPlaN needs first-class
pipelines for ingestion, transformation, validation, scheduling, lineage,
publication, audit events, reruns, and source-to-source workflows.
Monthly extraction/transformation work should start as a configuration package
and module collaboration across connectors, files, workflow, reporting, and
possibly templates. Create datasources/dataflow repositories only after that
package exposes repeated contracts that do not belong to an existing module.
## Calendar, Scheduling, And Appointments
Tracking: `govoplan-core#193`, `govoplan-calendar#1`,
`govoplan-calendar#2`, `govoplan-scheduling#1`,
`govoplan-appointments#1`.
Decision: use three separate modules.
`govoplan-calendar` owns:
- calendar collections, events, recurrence, availability/free-busy, resources,
iCalendar import/export, CalDAV/Open-Xchange-style calendar adapters, and
calendar WebUI surfaces
`govoplan-scheduling` owns:
- Terminfindung, meeting-time polls, participant availability collection,
candidate-slot ranking, conflict explanations, reminders, and the handoff
from a selected slot to calendar/appointment/workflow modules
`govoplan-appointments` owns:
- Terminbuchung/fixed-slot appointment booking, appointment types, booking
rules, capacity, cancellation/no-show state, public/internal booking flows,
and appointment evidence
Boundary:
- Calendar provides time primitives and external calendar integration.
- Scheduling chooses a suitable time.
- Appointments owns booked appointment workflows and public/internal booking
semantics.
- Mail and notifications deliver invitations/reminders through capabilities.
## Forms And Workflow Handoff
Tracking: `govoplan-core#194`, `govoplan-forms#1`.
Decision: forms are a reusable module boundary, with runtime behavior separated
from workflow semantics.
`govoplan-forms` owns:
- form definitions, schemas, validation rules, field visibility rules,
localization, versioning, admin editing, and reusable form package fragments
`govoplan-forms-runtime` owns, when implemented:
- public/internal submissions, drafts, submitted values, validation evidence,
attachment references, submission receipts, and handoff events
Boundary:
- Forms do not own cases, workflow transitions, tasks, or portal identity.
- Workflow/cases consume form submission events and evidence references.
- Files owns uploaded file storage and file permissions.
- Reporting/dataflow may consume submitted data through governed DTOs or
source lifecycle contracts.
## OpenDesk Integration Profile
Tracking: `govoplan-core#195`, `govoplan-connectors#5`,
`govoplan-idm#1`, `govoplan-mail#5`, `govoplan-calendar#2`,
`govoplan-connectors#1`.
Decision: OpenDesk is an integration profile, not a monolithic module.
Ownership:
- identity: `govoplan-idm` plus `govoplan-access`
- mail/groupware: `govoplan-mail`
- calendar: `govoplan-calendar`
- files/documents: `govoplan-files` and later `govoplan-dms`
- projects/tasks: `govoplan-connectors` OpenProject connector first
- inventory/health/profile diagnostics: `govoplan-connectors`
The OpenDesk profile should describe required connector profiles, shared
identity assumptions, health checks, and optional module combinations. It must
not create direct module-to-module imports.
## Project Management And OpenProject
Tracking: `govoplan-core#196`, `govoplan-connectors#1`.
Decision: connector-first. Do not create a native `govoplan-projects` module
yet.
OpenProject integration belongs in `govoplan-connectors` first:
- profile test
- project and work-package lookup
- external-reference storage
- selected publish/synchronize capabilities for tasks, workflow, or cases
A native project module is justified only if GovOPlaN needs to own project
semantics beyond cases, tasks, workflow, appointments, documents, and reporting,
for example portfolios, project budgets, project-level resource planning, or
governed project records that cannot remain in OpenProject.
## Public-Sector Integration Landscape
Tracking: `govoplan-core#186`, `govoplan-core#215`,
`govoplan-connectors#2`, `govoplan-connectors#3`.
Decision: core owns strategy and routing; connectors owns executable
integration catalogue entries and operator inventory.
Core documents:
- product-level integration strategy
- native-vs-connector decisions
- owning module routing
- roadmap sequencing
`govoplan-connectors` owns:
- connector entry schema
- external system catalogue
- connector profiles and diagnostics
- source consume/publish lifecycle
- external references
When a target needs executable behavior, create the implementation issue in the
owning module repository and keep only cross-module decisions in core.

@@ -0,0 +1,112 @@
<!-- codex-wiki-sync:fd4ca2789a17453678382e38 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/POLICY_CONTRACTS.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# GovOPlaN Policy Contracts
GovOPlaN has several policy families that are moving out of core into owning
modules. The shared kernel contract keeps their decision and provenance shape
consistent while each module still owns its domain rules.
## Current Policy Inventory
| Policy area | Current owner | Runtime surface | Notes |
| --- | --- | --- | --- |
| Privacy retention | `govoplan-policy` routes with compatibility helpers in core | `/api/v1/admin/privacy-retention/policies/{scope}` and `/explain` | System, tenant, user, group, and campaign sources merge into the effective retention policy. Parent locks block lower-level widening. |
| Mail profile policy | `govoplan-mail` | `/api/v1/mail/policies/{scope}` | Uses the same source-step path format for system, tenant, owner, and campaign provenance. |
| RBAC/access policy | `govoplan-access` | access capabilities in `govoplan_core.core.access` | Permission decisions should use access capability contracts. Explain responses should adopt `PolicyDecision` when an API-level explanation is added. |
| Governance defaults | `govoplan-admin` plus `govoplan-access` materializer | admin settings, governance template routes, access materialization capability | System governance can block tenant-local groups, roles, and API keys. |
| Delegation and ownership policy | access/campaign/mail/files modules | capability checks and owner-scoped APIs | Source provenance should use this contract when policies become externally explainable. |
## Policy Decision
The shared DTO lives in `govoplan_core.core.policy.PolicyDecision`.
```json
{
"allowed": false,
"reason": "Parent retention policy locks lower-level changes.",
"source_path": [
{
"scope_type": "system",
"scope_id": null,
"path": "system",
"label": "System",
"applied_fields": ["allow_lower_level_limits"],
"policy": {}
}
],
"requirements": ["raw_campaign_json_retention_days"],
"details": {
"blocked_fields": ["raw_campaign_json_retention_days"]
}
}
```
`allowed` is the effective answer for the checked action. `reason` is a stable,
human-readable summary. `source_path` lists the policy sources that explain the
answer. `requirements` lists machine-readable blockers or prerequisites, and
`details` carries domain-specific structured context.
Every source step should be concrete enough for an operator to understand the
decision without knowing internal merge rules. Use real scope labels such as
`System`, `Tenant`, `Owner user`, `Group`, or a campaign/profile name. Include
the stable `path`, the fields applied by that step, and the local policy
fragment that caused them. This lets UIs render explanations like
`System: Allow > Tenant: Deny without override` without additional lookups.
If a policy family cannot expose the full local fragment for security reasons,
it must still include a redacted structured value that identifies the applied
field and the effective allow/deny or lock state.
## Source Path Format
Policy source paths are stable string identifiers for provenance steps:
- `system`
- `<scope_type>:<url-encoded-scope-id>`
Supported scope types are `system`, `tenant`, `user`, `group`, and `campaign`.
Examples:
- `tenant:4a45b4fe-1d86-43ce-9d10-6022333f4d4b`
- `campaign:campaign%2Fwith%20space`
Use `policy_source_path()` and `parse_policy_source_path()` instead of building
or splitting these strings manually.
## Retention Explain Endpoint
`GET /api/v1/admin/privacy-retention/policies/{scope_type}/explain` returns:
- `scope_type` and optional `scope_id`
- `decision`, using the shared `PolicyDecision` shape
- `effective_policy`
- optional `parent_policy`
- `effective_policy_sources`
- `parent_policy_sources`
- `blocked_fields`
The endpoint is read-only. Enforcement remains in the existing policy write
path. For lower-level scopes, `blocked_fields` is derived from the parent
policy's `allow_lower_level_limits`; clients can use it to disable local
controls before attempting a write.
## Frontend Contract
Policy UIs must:
- render effective source provenance when `effective_policy_sources` is present
- display a field-level path when the source data is shown next to a specific
setting, using concrete source labels and stop at the first non-overridable
deny/lock
- disable local field controls when the parent policy sets that field's
lower-level limit to `false`
- avoid sending locked fields or re-enable attempts in save payloads
- show inherited values separately from local overrides
The core WebUI helper `privacyRetentionParentAllowsField()` centralizes the
field-lock decision used by the retention editor and its lightweight module
tests.

@@ -0,0 +1,283 @@
<!-- codex-wiki-sync:ceba202482f7f3bf41f0a7fc -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/PUBLIC_SECTOR_INTEGRATION_STRATEGY.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# Public-Sector Integration Strategy
GovOPlaN should integrate with the existing public-sector software landscape
before deciding to replace specialist workflows. This document is the core
strategy index. The executable connector catalogue lives in
`govoplan-connectors/docs/PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md`.
## Strategy Labels
Use one or more of these labels for every external system family:
- `integrate`: GovOPlaN talks to the system through a stable API/protocol.
- `link`: GovOPlaN stores external references and opens the external system for
source-of-truth work.
- `import`: GovOPlaN consumes data or files into governed module storage.
- `synchronize`: GovOPlaN keeps selected records aligned both ways or through a
source-of-truth rule.
- `replace selected workflow`: GovOPlaN may own a narrow workflow where the
external product is weak, but does not replace the whole product family.
- `no first-class support`: GovOPlaN only stores manual references unless a
deployment project creates a specific connector.
## Initial Classification
| System family | Examples | Default strategy | Likely owner |
| --- | --- | --- | --- |
| File providers | SMB/CIFS, WebDAV, Nextcloud, Seafile, SFTP, S3 | integrate, import, link | `govoplan-files`, connector inventory in `govoplan-connectors` |
| Project/task management | OpenProject, Jira, Redmine, Microsoft Planner | link, synchronize selected records, replace selected workflow only after proof | `govoplan-connectors`, later `govoplan-tasks` or workflow modules |
| Identity providers | LDAP, Active Directory, OIDC, SAML, OpenDesk IDM | integrate, synchronize | `govoplan-idm`, `govoplan-access` |
| Mail and groupware | IMAP/SMTP, Open-Xchange, Exchange/M365, CalDAV/CardDAV | integrate, link | `govoplan-mail`, `govoplan-calendar`, `govoplan-connectors` |
| DMS/e-file/archive | d.velop/d.3, enaio, ELO, Fabasoft, CMIS, VIS/eAkte | link, import, synchronize selected metadata | `govoplan-dms`, `govoplan-files`, `govoplan-records` |
| ERP/finance/procurement | SAP, MACH, Infoma, DATEV, procurement feeds | export, import, synchronize; do not replace by default | `govoplan-erp`, `govoplan-procurement`, `govoplan-ledger`, `govoplan-payments` |
| Public-sector transport | FIT-Connect, XTA/OSCI, Peppol access points | integrate, publish, receive | dedicated protocol modules plus `govoplan-connectors` inventory |
| Standards registries | XRepository, XOE/V catalogues | link, import metadata/cache | `govoplan-connectors`, `govoplan-xoev` |
| Publication/data exchange | RSS, open-data APIs, API feeds, CSV/Excel drops | consume, publish, transform | proposed `govoplan-datasources`, proposed `govoplan-dataflow`, `govoplan-reporting` |
| Collaboration suites | Matrix, Jitsi, BigBlueButton, Nextcloud Talk, Collabora/OnlyOffice | integrate, link; native behavior only for governed evidence | `govoplan-connectors`, `govoplan-dms`, `govoplan-workflow` |
| Specialist Fachverfahren | register-specific and domain-specific systems | link first; integrate/import when a real project supplies contracts | domain module or deployment-specific connector |
## Landscape Catalogue
This catalogue is intentionally implementation-oriented. Each entry records the
first API/auth/data assumptions needed to turn an inventory entry into a
connector or module issue.
### Citizen And Service Portals
- Strategy: integrate/link first; replace selected intake workflow only when a
GovOPlaN portal package owns the complete journey.
- Protocol/API surface: REST/JSON APIs, form submission webhooks, OIDC/SAML
login, eID interfaces where available, file-upload callbacks, case-status
callbacks.
- Auth model: OIDC/SAML service clients, signed webhook secrets, tenant-scoped
API keys, later eID/trust-provider handoff.
- Data shape: applicant identity reference, application form payload,
attachment references, consent declarations, status events, receipt IDs.
- Deployment assumptions: externally reachable HTTPS, reverse proxy, portal DMZ
separation, strict CSRF/origin settings, large upload path.
- Risks: personal data exposure, duplicate identity mapping, partial
submissions, upload malware, inconsistent portal status models.
- MVP test path: submit a test application with one file, create a form
submission/case/task stub, return a receipt and status reference.
- Owner/priority: `govoplan-portal`, `govoplan-forms-runtime`,
`govoplan-files`, Wave 1.
### DMS, E-File, Records, And Archives
- Strategy: link/import/synchronize selected metadata; do not replace the DMS by
default.
- Protocol/API surface: CMIS, WebDAV, vendor REST APIs, S3/object archive
staging, file-plan export/import, archive handoff APIs.
- Auth model: service accounts, OAuth/OIDC where supported, mTLS for regulated
archives, secret references for vendor tokens.
- Data shape: document ID, version, file-plan/classification code, retention
metadata, owner/case reference, external URL, checksum, lock/legal-hold state.
- Deployment assumptions: usually internal network or VPN, strict storage
quotas, existing retention policies, archive immutability requirements.
- Risks: record duplication, broken legal hold, permission mismatch, version
drift, destructive retention/export mistakes.
- MVP test path: create a connector inventory entry, test read-only metadata
lookup, link one GovOPlaN file/case evidence item to an external document.
- Owner/priority: `govoplan-dms`, `govoplan-records`, `govoplan-files`,
`govoplan-connectors`, Wave 2/5.
### ERP, Finance, Procurement, And Accounting
- Strategy: export/import/synchronize selected records; replacement only by
narrow domain decision.
- Protocol/API surface: vendor REST/SOAP APIs, CSV/XML batch exchange, SFTP,
XRechnung/Peppol, XBestellung/procurement feeds, payment reconciliation files.
- Auth model: service accounts, client certificates, mTLS, SFTP keys, token
references, environment-specific account separation.
- Data shape: debtor/creditor reference, payment request, invoice, order,
budget/cost-center code, booking status, receipt/evidence reference.
- Deployment assumptions: batch windows, finance-system approval workflows,
test tenants often separated from production by vendor process.
- Risks: financial posting errors, double export, tax/legal data retention,
inconsistent master data, irreversible accounting handoff.
- MVP test path: dry-run export of one payment/accounting handoff file with
checksum, validation report, and no remote posting.
- Owner/priority: `govoplan-payments`, `govoplan-ledger`,
`govoplan-xrechnung`, `govoplan-erp`, `govoplan-procurement`, Wave 1/6.
### Identity, IAM, And Directory Services
- Strategy: integrate/synchronize; access remains GovOPlaN's local
authorization boundary.
- Protocol/API surface: LDAP, Active Directory, SCIM, OIDC, SAML, OpenDesk IDM
APIs, group membership sync, account deactivation feeds.
- Auth model: bind accounts, service clients, OIDC/SAML metadata, SCIM tokens,
certificate-backed clients where required.
- Data shape: account, user, group, membership, role claim, tenant/org-unit
mapping, status, external directory ID.
- Deployment assumptions: directory is usually internal; identity provider may
be organization-wide and not GovOPlaN-owned.
- Risks: privilege escalation through group mapping, stale memberships, account
collision, deprovisioning latency, tenant-boundary mistakes.
- MVP test path: read-only directory profile test, map one external group to a
tenant group, show a dry-run membership diff.
- Owner/priority: `govoplan-idm`, `govoplan-access`, Wave 1.
### Groupware, Mail, Calendar, And Collaboration
- Strategy: integrate/link; native behavior only where GovOPlaN needs governed
evidence or process state.
- Protocol/API surface: IMAP/SMTP, CalDAV/CardDAV, Open-Xchange APIs, Microsoft
Graph/EWS, Matrix APIs, Jitsi/BigBlueButton APIs, Collabora/OnlyOffice
integration points.
- Auth model: service accounts, delegated OAuth/OIDC, app passwords, mailbox
credentials, groupware-specific tokens, secret references.
- Data shape: mailbox folder/message references, event/free-busy data, meeting
URL, chat room ID, participant list, document-editing session reference.
- Deployment assumptions: often internal/existing tenant infrastructure; mail
and calendar may be separate from identity even in OpenDesk-style stacks.
- Risks: mail credential exposure, calendar privacy, double invitations, room
booking conflicts, chat/document data escaping retention rules.
- MVP test path: profile test for mailbox/calendar reachability, read-only
folder/free-busy lookup, create a non-production event/message draft.
- Owner/priority: `govoplan-mail`, `govoplan-calendar`,
`govoplan-connectors`, Wave 1/2.
### Payment And Public Cashier Systems
- Strategy: integrate/export/import; keep the payment provider or cashier as
source of settlement truth.
- Protocol/API surface: payment provider APIs, redirect/callback flows,
reconciliation files, SEPA/export formats, cash-register/cashier interfaces.
- Auth model: provider API keys, signed webhooks, client certificates, mTLS,
tenant-specific merchant accounts.
- Data shape: payment intent, amount/currency, payer reference, provider
transaction ID, settlement status, receipt, refund/cancellation reference.
- Deployment assumptions: public callback URLs, strict environment separation,
PCI-sensitive providers, finance reconciliation cadence.
- Risks: duplicate charges, callback replay, amount mismatch, refund workflow
gaps, evidence-retention mistakes.
- MVP test path: sandbox payment intent, signed callback verification, receipt
evidence link, reconciliation dry-run.
- Owner/priority: `govoplan-payments`, `govoplan-ledger`, Wave 1/6.
### Reporting, BI, Open Data, And Publication
- Strategy: consume/publish/transform; native reporting owns GovOPlaN views, not
every external BI product.
- Protocol/API surface: SQL read replicas, CSV/Excel export/import, REST APIs,
RSS/Atom, open-data APIs, SFTP/WebDAV publication targets.
- Auth model: read-only DB users, API tokens, SFTP keys, OAuth clients, public
anonymous publication profiles where appropriate.
- Data shape: dataset metadata, schema/version, report parameters, generated
file references, publication URL, freshness/lineage, validation results.
- Deployment assumptions: publication can be public or internal; generated
datasets need retention and provenance.
- Risks: leaking restricted data, stale publications, schema drift, expensive
queries, untraceable manual transformations.
- MVP test path: publish one report/export as a governed file plus RSS/Atom
entry with checksum, timestamp, and permission check.
- Owner/priority: `govoplan-reporting`, `govoplan-connectors`, possible future
`govoplan-datasources`/`govoplan-dataflow`, Wave 2.
### Public-Sector Protocols And Registries
- Strategy: integrate/publish/receive; protocol modules own protocol semantics.
- Protocol/API surface: FIT-Connect, XTA/OSCI, XRepository, XOE/V, XRechnung,
XBestellung, Peppol, registry-specific Fachverfahren APIs.
- Auth model: certificates, mTLS, service accounts, destination credentials,
protocol-specific trust anchors and key rotation.
- Data shape: transport envelope, payload schema/version, destination IDs,
receipt/acknowledgement, message status, standard-specific metadata.
- Deployment assumptions: regulated trust chains, test/prod endpoint
separation, formal onboarding, strict logging and retention expectations.
- Risks: invalid schemas, failed delivery receipts, certificate expiry, wrong
destination routing, protocol version drift.
- MVP test path: validate a sample payload against a schema, test endpoint
reachability in sandbox, store receipt/evidence reference.
- Owner/priority: `govoplan-fit-connect`, `govoplan-xoev`,
`govoplan-xrechnung`, `govoplan-xta-osci`, `govoplan-connectors`, Wave 1/2.
### File Providers And Shared Storage
- Strategy: integrate/import/link; files module owns GovOPlaN file semantics.
- Protocol/API surface: SMB/CIFS, WebDAV, Nextcloud, Seafile, SFTP, S3,
local/object storage profiles.
- Auth model: service accounts, user credentials, app tokens, OAuth where
supported, secret references, environment variables for deployment-managed
credentials.
- Data shape: file ID/path, provider object ID, checksum, MIME type, size,
version/ETag, owner, permission snapshot, imported file reference.
- Deployment assumptions: internal networks, large files, existing shares,
variable permissions, provider-specific rate limits.
- Risks: permission mismatch, stale imports, overwrites, duplicate files, path
traversal, storage growth.
- MVP test path: profile test, list folder, import one file into governed
storage, keep provider reference and checksum.
- Owner/priority: `govoplan-files`, `govoplan-connectors`, Wave 0/1.
### Project, Task, And Case-Adjacent Systems
- Strategy: connector-first for OpenProject/Jira/Redmine; native module only
when GovOPlaN owns project semantics.
- Protocol/API surface: OpenProject API v3, webhooks, Jira/Redmine REST APIs,
Microsoft Graph for Planner/Project where applicable.
- Auth model: API tokens, OAuth/OIDC apps, webhook secrets, service accounts.
- Data shape: project ID, work package/task ID, status, assignee reference,
external URL, version/lock token, publish/sync trace.
- Deployment assumptions: external project tool remains source of truth for
broad project management; GovOPlaN links selected records.
- Risks: task duplication, bidirectional sync conflicts, permission mismatch,
over-mirroring comments/attachments.
- MVP test path: OpenProject profile test, project/work-package lookup,
external-reference round-trip.
- Owner/priority: `govoplan-connectors`, later `govoplan-tasks`/workflow/cases
consumers, Wave 0/2.
### Specialist Fachverfahren
- Strategy: link first; integrate/import only when a deployment project supplies
concrete contracts and a domain owner.
- Protocol/API surface: vendor APIs, CSV/XML batch imports, SFTP, database
views, message queues, protocol-specific transports.
- Auth model: usually service accounts, VPN, mTLS, SFTP keys, or vendor tokens.
- Data shape: domain-specific record IDs, status, applicant/person references,
file/evidence references, case/status events.
- Deployment assumptions: strongly local/vendor-specific, often no stable test
API, data model differs by jurisdiction.
- Risks: brittle vendor contracts, legal source-of-truth ambiguity, high
customization cost, migration expectations.
- MVP test path: inventory entry and manual external-reference link; require a
project-specific connector issue before automation.
- Owner/priority: domain module or deployment-specific connector, case by case.
## Prioritization Rules
1. Start with connectors that unblock Wave 0 or Wave 1 reference journeys.
2. Prefer open standards and self-hosted/open-source APIs where they are common
in public-sector deployments.
3. Treat inventory-only entries as useful because operators need a map of their
software landscape even before automation exists.
4. Keep connector code in the owning connector/protocol module. Domain modules
consume capabilities, DTOs, external references, and events through core.
5. Every executable connector needs health diagnostics, secret-reference
handling, lifecycle state, audit events, and retirement behavior.
## Connector Catalogue Handoff
`govoplan-connectors` owns the detailed catalogue entry shape:
- connector type key
- category and owner module
- supported directions and trigger modes
- credential and secret handling
- health check and diagnostics payload
- external-reference shape
- required capabilities and optional module combinations
- lifecycle support
Core should only keep strategy, routing, and cross-module architecture notes.
Connector implementation and public-sector target inventory belong in
`govoplan-connectors`.

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:a97b489b61c47841cd5bb849 --> <!-- codex-wiki-sync:6527293c59e25e64f6553a58 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/RELEASE_CATALOG_WORKFLOW.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/RELEASE_CATALOG_WORKFLOW.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -54,8 +54,13 @@ The wrapper validates the catalog with core using the generated public keyring.
## Publish After A New Release ## Publish After A New Release
For normal module/core releases, first tag and push the module/core repos. Then For normal module/core releases, first audit and record migration baselines,
publish the website catalog: then tag and push the module/core repos. Finally publish the website catalog:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python scripts/release-migration-audit.py --strict
```
```bash ```bash
cd /mnt/DATA/git/govoplan-core cd /mnt/DATA/git/govoplan-core
@@ -84,13 +89,16 @@ https://govoplan.add-ideas.de/catalogs/v1/keyring.json
## Integrated Release Script ## Integrated Release Script
`scripts/push-release-tag.sh` can publish the web catalog after module and core `scripts/push-release-tag.sh` can publish the web catalog after module and core
tags have been pushed: tags have been pushed. It runs the migration release audit in non-strict mode
by default; add `--strict-migration-audit` for stable releases after
`docs/migration-release-baselines.json` has been updated:
```bash ```bash
cd /mnt/DATA/git/govoplan-core cd /mnt/DATA/git/govoplan-core
KEY_DIR="$HOME/.config/govoplan/release-keys" KEY_DIR="$HOME/.config/govoplan/release-keys"
scripts/push-release-tag.sh \ scripts/push-release-tag.sh \
--bump subversion \ --bump subversion \
--strict-migration-audit \
--publish-web-catalog \ --publish-web-catalog \
--catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \
--build-web-catalog --build-web-catalog

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:0a79f39c61130e66fb78a184 --> <!-- codex-wiki-sync:61ed52f82af2f6866512f2cf -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md`. > Mirrored from `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -35,17 +35,84 @@ cd /mnt/DATA/git/govoplan-core
`requirements-release.txt` pins the module repositories to the release tag. Update those refs when cutting a release: `requirements-release.txt` pins the module repositories to the release tag. Update those refs when cutting a release:
```text ```text
govoplan-access git@git.add-ideas.de:add-ideas/govoplan-access.git v0.1.4 govoplan-access git@git.add-ideas.de:add-ideas/govoplan-access.git v0.1.6
govoplan-admin git@git.add-ideas.de:add-ideas/govoplan-admin.git v0.1.4 govoplan-admin git@git.add-ideas.de:add-ideas/govoplan-admin.git v0.1.6
govoplan-tenancy git@git.add-ideas.de:add-ideas/govoplan-tenancy.git v0.1.4 govoplan-tenancy git@git.add-ideas.de:add-ideas/govoplan-tenancy.git v0.1.6
govoplan-policy git@git.add-ideas.de:add-ideas/govoplan-policy.git v0.1.4 govoplan-policy git@git.add-ideas.de:add-ideas/govoplan-policy.git v0.1.6
govoplan-audit git@git.add-ideas.de:add-ideas/govoplan-audit.git v0.1.4 govoplan-audit git@git.add-ideas.de:add-ideas/govoplan-audit.git v0.1.6
govoplan-files git@git.add-ideas.de:add-ideas/govoplan-files.git v0.1.4 govoplan-files git@git.add-ideas.de:add-ideas/govoplan-files.git v0.1.6
govoplan-mail git@git.add-ideas.de:add-ideas/govoplan-mail.git v0.1.4 govoplan-mail git@git.add-ideas.de:add-ideas/govoplan-mail.git v0.1.6
govoplan-campaign git@git.add-ideas.de:add-ideas/govoplan-campaign.git v0.1.4 govoplan-campaign git@git.add-ideas.de:add-ideas/govoplan-campaign.git v0.1.6
govoplan-calendar git@git.add-ideas.de:add-ideas/govoplan-calendar.git v0.1.4 govoplan-calendar git@git.add-ideas.de:add-ideas/govoplan-calendar.git v0.1.6
``` ```
### PostgreSQL Release Check
Release candidates should pass a disposable PostgreSQL migration and startup
smoke check before tagging or publishing catalogs. Start the local testbed, then
run the permutation check from the core checkout:
```bash
cd /mnt/DATA/git/govoplan-core/dev/postgres
cp .env.example .env
docker compose --env-file .env up -d
cd /mnt/DATA/git/govoplan-core
set -a
. dev/postgres/.env
set +a
./.venv/bin/python scripts/postgres-integration-check.py \
--database-url "$GOVOPLAN_POSTGRES_DATABASE_URL" \
--reset-schema
```
The script checks migrations and `/health` startup for core-only, files-only,
mail-only, campaign-only, campaign+files, campaign+mail, and full-product
module sets. `--reset-schema` is destructive and must only be used against a
throwaway database.
### Migration Baselines
Development migrations may be small and numerous while a feature is moving.
Before a stable release, unreleased migrations may be rewritten or squashed into
a release-level baseline or release-to-release upgrade migration. After a
release tag has shipped, released migration revision IDs are immutable.
The release policy is:
- unreleased migrations may be folded before release;
- released migrations are never rewritten or deleted;
- each stable release records the public migration head revisions in
`docs/migration-release-baselines.json`;
- fresh installations should apply release-level baselines/upgrades, not
unreleased create-then-rename churn;
- release-to-release schema changes should be folded into one reviewed
migration per migration owner where practical.
Audit the current graph during release preparation:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python scripts/release-migration-audit.py
```
Use strict mode after the baseline file has been updated for the release:
```bash
./.venv/bin/python scripts/release-migration-audit.py --strict
```
`scripts/push-release-tag.sh` runs the audit in non-strict mode by default so
release operators see the current migration heads before tagging. Pass
`--strict-migration-audit` when cutting a release whose migration baselines have
already been recorded, or `--skip-migration-audit` only for emergency/manual
release work.
Before the first stable release, fold the current development chain into the
first public baseline and record that baseline in
`docs/migration-release-baselines.json`. The tracking issue is
`add-ideas/govoplan-core#223`.
## Runtime Module Package Changes ## Runtime Module Package Changes
The admin module manager can hot-enable and hot-disable packages that are The admin module manager can hot-enable and hot-disable packages that are
@@ -91,9 +158,9 @@ govoplan-module-installer \
--daemon \ --daemon \
--migrate \ --migrate \
--build-webui \ --build-webui \
--database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \ --database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL_PGTOOLS" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \ --database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \
--database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL" "$GOVOPLAN_DATABASE_BACKUP_PATH"' \ --database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--health-url http://127.0.0.1:8000/health \ --health-url http://127.0.0.1:8000/health \
--restart-command '<restart govoplan server>' --restart-command '<restart govoplan server>'
``` ```
@@ -123,9 +190,9 @@ hooks:
govoplan-module-installer \ govoplan-module-installer \
--supervise \ --supervise \
--migrate \ --migrate \
--database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \ --database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL_PGTOOLS" > "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \ --database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null' \
--database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL" "$GOVOPLAN_DATABASE_BACKUP_PATH"' \ --database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" "$GOVOPLAN_DATABASE_BACKUP_PATH"' \
--health-url http://127.0.0.1:8000/health \ --health-url http://127.0.0.1:8000/health \
--restart-command '<restart govoplan server>' --restart-command '<restart govoplan server>'
``` ```
@@ -178,6 +245,9 @@ Database hook commands run with these environment variables:
- `GOVOPLAN_INSTALLER_RUN_DIR`: the run snapshot directory - `GOVOPLAN_INSTALLER_RUN_DIR`: the run snapshot directory
- `GOVOPLAN_DATABASE_URL`: the configured database URL - `GOVOPLAN_DATABASE_URL`: the configured database URL
- `GOVOPLAN_DATABASE_URL_PGTOOLS`: the same PostgreSQL URL converted from
`postgresql+driver://` to `postgresql://` for `pg_dump`, `pg_restore`, and
`psql`; set only for PostgreSQL URLs
- `GOVOPLAN_DATABASE_BACKUP_PATH`: a suggested backup artifact path inside the - `GOVOPLAN_DATABASE_BACKUP_PATH`: a suggested backup artifact path inside the
run directory run directory
- `GOVOPLAN_DATABASE_BACKUP_METADATA`: optional JSON metadata path that backup - `GOVOPLAN_DATABASE_BACKUP_METADATA`: optional JSON metadata path that backup
@@ -198,6 +268,88 @@ govoplan-module-installer --cancel-request <request-id> --format json
govoplan-module-installer --retry-request <request-id> --format json govoplan-module-installer --retry-request <request-id> --format json
``` ```
### Rollback Drill
Before using the installer daemon in production, run the rollback drill in a
controlled shell from the core checkout:
```bash
./.venv/bin/python scripts/module-installer-rollback-drill.py --format json
```
The drill uses temporary SQLite databases and simulated package commands. It
does not install or uninstall real packages. It exercises:
- package command failure followed by supervised rollback
- migration failure with a SQLite database snapshot
- restart-command failure
- health timeout after restart
- destructive retirement executor failure with database rollback
- PostgreSQL-style backup, restore-check, and restore hooks
- daemon heartbeat, request queue claim/update, retry/cancel, and stale lock
detection/removal
Keep the drill runtime for inspection with:
```bash
./.venv/bin/python scripts/module-installer-rollback-drill.py \
--keep-runtime \
--runtime-root /srv/govoplan/drills/installer-rollback-$(date +%Y%m%d)
```
Each scenario writes an installer run record below
`<runtime-root>/<scenario>/installer/runs/<run-id>/record.json`. For a successful
rollback drill, check:
- `status` is `applied` only for the original package phase when rollback later
happens in the supervisor
- `rollback_status` is `rolled-back` when package/database snapshots were
restored
- `supervisor.status` is `rolled-back` for supervised package, migration,
restart, and health failures
- `supervisor.failure_reason` names the failing restart or health target
- `snapshot.database_backup` exists for migrated SQLite runs and destructive
retirement runs
- PostgreSQL hook drills write `database.external.backup`,
`restore-check.marker`, and `restore.marker`
The PostgreSQL drill validates the hook interface without connecting to a live
database. In production, replace the documented example hooks with deployment
commands that target a disposable staging database first:
```bash
--database-backup-command 'pg_dump --format=custom "$GOVOPLAN_DATABASE_URL_PGTOOLS" > "$GOVOPLAN_DATABASE_BACKUP_PATH"'
--database-restore-check-command 'pg_restore --list "$GOVOPLAN_DATABASE_BACKUP_PATH" >/dev/null'
--database-restore-command 'pg_restore --clean --if-exists --dbname "$GOVOPLAN_DATABASE_URL_PGTOOLS" "$GOVOPLAN_DATABASE_BACKUP_PATH"'
```
If a daemon request fails, inspect it and retry only after the underlying cause
is corrected:
```bash
govoplan-module-installer --show-request <request-id> --format json
govoplan-module-installer --show-run <run-id> --format json
govoplan-module-installer --retry-request <request-id> --format json
```
Cancel queued duplicate work before retrying:
```bash
govoplan-module-installer --list-requests --format json
govoplan-module-installer --cancel-request <request-id> --format json
```
For stale locks, first check the lock payload and confirm the recorded process
is no longer running on the host that owns the runtime directory:
```bash
govoplan-module-installer --lock-status --format json
```
Only remove `runtime/module-installer/install.lock` after confirming there is
no active installer process and no package manager command still running. Then
rerun preflight before applying or retrying the request.
Package catalogs can be local files or remote static resources, for example Package catalogs can be local files or remote static resources, for example
served by `govoplan-web`. Set `GOVOPLAN_MODULE_PACKAGE_CATALOG` for a local file served by `govoplan-web`. Set `GOVOPLAN_MODULE_PACKAGE_CATALOG` for a local file
or `GOVOPLAN_MODULE_PACKAGE_CATALOG_URL` for a remote catalog matching or `GOVOPLAN_MODULE_PACKAGE_CATALOG_URL` for a remote catalog matching
@@ -253,6 +405,29 @@ GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-seq
GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true
``` ```
Back up the sequence state with other runtime metadata. If it is lost or
corrupted, reconstruct the highest accepted sequence per channel from installer
run records or release records, never by lowering the sequence to admit an
older catalog. `docs/CATALOG_TRUST_AND_LICENSING.md` includes the recovery JSON
shape and reset procedure.
Catalog module entries may include `artifact_integrity.python` and
`artifact_integrity.webui` metadata. Each artifact entry can declare:
- `ref`: the exact Python or WebUI install ref expected in the plan
- `sha256`: the expected SHA-256 digest of the resolved artifact
- `path` or `artifact_path`: a local artifact path that the installer can hash
before applying the plan
- `sbom_url`: the published SBOM for the artifact
- `provenance_url`: the published provenance or attestation for the artifact
- `registry_identity` or `git_ref`: the expected registry or source identity
When a saved install plan includes this metadata, installer preflight records
verification results in the run record under `preflight.artifact_integrity`.
Set `GOVOPLAN_MODULE_INSTALLER_REQUIRE_ARTIFACT_INTEGRITY=true` in production
to block package changes whose Python/WebUI artifacts are missing integrity
metadata or cannot be locally verified.
Trusted keys can also be loaded from Trusted keys can also be loaded from
`GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE`. A URL-backed keyring is `GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE`. A URL-backed keyring is
supported for development and tightly controlled deployments through supported for development and tightly controlled deployments through
@@ -270,6 +445,23 @@ GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE=/srv/govoplan/trust/license-keyring.json
GOVOPLAN_LICENSE_ENFORCEMENT=true GOVOPLAN_LICENSE_ENFORCEMENT=true
``` ```
Operators can validate the imported license and required catalog entitlements:
```bash
govoplan-module-installer \
--validate-license /srv/govoplan/license.json \
--license-trusted-key license-issuer-1="<base64 public key>" \
--require-trusted-license \
--license-required-feature module.mail \
--format json
```
Release or support operators can issue/renew signed offline licenses with
`govoplan-module-installer --issue-license`. Keep the signing private key off
the application server; the admin UI only displays public diagnostics such as
license id, subject, validity window, trusted key id, features, and missing
entitlements.
See `docs/CATALOG_TRUST_AND_LICENSING.md` for key rotation, replay protection, See `docs/CATALOG_TRUST_AND_LICENSING.md` for key rotation, replay protection,
and licensing details. See `docs/RELEASE_CATALOG_WORKFLOW.md` for the concrete and licensing details. See `docs/RELEASE_CATALOG_WORKFLOW.md` for the concrete
release-machine workflow that generates signing keys and publishes signed release-machine workflow that generates signing keys and publishes signed
@@ -281,7 +473,8 @@ Install rows must use tagged package/git refs or registry packages, not local
`file:` or workspace links. The installer daemon can run `npm install` and `file:` or workspace links. The installer daemon can run `npm install` and
`npm run build` for WebUI package changes; that is the supported path. Browser `npm run build` for WebUI package changes; that is the supported path. Browser
remote bundles are still experimental and should be treated as a controlled remote bundles are still experimental and should be treated as a controlled
deployment option, not the normal install/uninstall mechanism. deployment option, not the normal install/uninstall mechanism. The target
design and follow-up slices are in `docs/REMOTE_WEBUI_BUNDLES.md`.
Module manifests can declare core compatibility bounds and uninstall guard Module manifests can declare core compatibility bounds and uninstall guard
providers. Preflight blocks incompatible manifest contracts/core versions, providers. Preflight blocks incompatible manifest contracts/core versions,
@@ -313,7 +506,7 @@ The normal release path is automated by `scripts/push-release-tag.sh`: it bumps
```bash ```bash
cd /mnt/DATA/git/govoplan-core cd /mnt/DATA/git/govoplan-core
scripts/push-release-tag.sh --version 0.1.2 scripts/push-release-tag.sh --version 0.1.6
``` ```
The script also includes GovOPlaN roadmap/scaffold module repositories that do not yet have package metadata. Those repositories are committed, tagged, and pushed with the same release tag, but they are tag-only until they contain `pyproject.toml`, module manifests, or WebUI packages. Tag-only repositories are not listed in `requirements-release.txt` or `webui/package.release.json`. The script also includes GovOPlaN roadmap/scaffold module repositories that do not yet have package metadata. Those repositories are committed, tagged, and pushed with the same release tag, but they are tag-only until they contain `pyproject.toml`, module manifests, or WebUI packages. Tag-only repositories are not listed in `requirements-release.txt` or `webui/package.release.json`.

@@ -0,0 +1,168 @@
<!-- codex-wiki-sync:3c85bfaf51ef10343c31417c -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/REMOTE_WEBUI_BUNDLES.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# Remote WebUI Bundle Loading
GovOPlaN WebUI modules normally ship through the core WebUI package graph:
install a tagged npm/git dependency, run `npm install`, rebuild the shell, and
restart or reload the served assets. Remote WebUI bundles are an experimental
future path for controlled deployments where a backend-enabled module is not in
the local WebUI package graph and the operator wants the shell to load its
frontend without rebuilding core.
This design defines the target guardrails. The current rebuild/reload path
remains the default production path.
## Current Rebuild Path
The supported release path is:
1. Install or remove backend and WebUI package dependencies through the trusted
installer CLI/daemon.
2. Snapshot package state before mutation.
3. Run `npm install` and optionally `npm run build`.
4. Restart or reload the served WebUI assets.
5. Use installer rollback if package apply, restart, or health checks fail.
Strengths:
- package manager and lockfile semantics stay conventional
- CSP can stay strict because all code is served as built assets
- rollback restores package files and lockfiles
- local development uses sibling workspace dependencies naturally
Costs:
- frontend changes require a rebuild/reload
- hot enabling a module with frontend code is not possible in the running shell
- failed rebuilds happen at install time, not at lazy module-load time
## Remote Bundle Path
Remote loading is only for modules that are enabled by the backend and absent
from `virtual:govoplan-installed-modules`. The backend manifest exposes:
- `FrontendModule.asset_manifest`
- `asset_manifest_integrity`
- `asset_manifest_signature`
- `asset_manifest_public_key_id`
- `asset_manifest_contract_version`
The WebUI shell fetches the asset manifest, verifies manifest integrity and/or
signature, validates the manifest contract, fetches the entry bundle, verifies
the entry integrity, imports the bundle, validates the exported
`PlatformWebModule`, and applies backend metadata before registering routes,
navigation, and UI capabilities.
Unsigned and unhashed manifests are skipped. Entries without integrity are
skipped. A module id mismatch is skipped.
## Asset Manifest Contract
Contract version `1`:
```json
{
"contractVersion": "1",
"moduleId": "files",
"entry": "./files-webui.remote.js",
"entryIntegrity": "SHA-256-<base64-or-hex-digest>",
"moduleExport": "default"
}
```
The backend manifest carries the manifest-level trust metadata. The remote
asset manifest carries the concrete entry URL and entry digest.
## Compatibility Checks
Before a remote bundle is accepted:
- backend module must be enabled
- local WebUI module with the same id must be absent
- manifest contract must be supported by the shell
- manifest `moduleId`, exported module `id`, and backend module id must match
- backend metadata remains authoritative for label, version, dependencies,
nav, and runtime UI capability exposure
- future contract versions must declare required shell capabilities so old
shells fail closed
Remote bundles must not broaden backend permissions. They can only contribute
routes/nav/capabilities that the backend metadata and existing permission
checks allow.
## CSP
The current implementation imports verified entry bytes through a blob URL. A
production CSP for this mode must explicitly allow:
- `connect-src` for the approved asset-manifest and bundle origins
- `script-src` for `blob:` only when remote loading is enabled
- no `unsafe-inline` requirement for remote modules
If an installation cannot allow `blob:` scripts, the follow-up implementation
should use signed same-origin module assets with static URLs instead of blob
imports. Remote loading must be disableable by configuration so strict-CSP
deployments can keep the rebuild path only.
## Cache Invalidation
The shell cache key is:
```text
<module-id>:<asset-manifest-url>:<backend-module-version>
```
Release catalogs should publish immutable manifest and entry URLs or change at
least one cache-key component on every release. Emergency rollback can point the
backend manifest at a previous immutable manifest URL or lower the enabled
module version after the backend package rollback.
Browsers may still cache remote responses. Approved asset servers should use:
- long cache lifetimes only for content-addressed immutable assets
- short cache lifetimes or explicit revalidation for channel/latest manifest
aliases
- `Cache-Control: no-store` for emergency override manifests
## Rollback
Remote WebUI rollback is metadata rollback, not package-manager rollback:
- if the backend package install rolls back, the previous backend frontend
metadata returns
- if only remote assets are bad, publish a new manifest URL or revert the
backend/frontend metadata to the last known good manifest
- failed remote loading must degrade by omitting the remote module frontend,
not by breaking the shell
Installer run records should eventually include remote asset manifest URLs,
integrity, signature key ids, and load-test results when a plan enables a
remote frontend.
## Local And Development Behavior
Local development should keep using workspace/file dependencies and Vite. Remote
loading is useful for integration testing release artifacts, not for day-to-day
module UI development.
Development deployments may use unsigned catalogs and local asset servers only
when signature/integrity enforcement is intentionally disabled. The remote
loader itself still requires an integrity hash or a verifiable signature.
## Follow-Up Slices
1. Add a server-side remote-bundle policy flag and surface whether remote
loading is enabled in platform metadata.
2. Add CSP documentation/config generation for strict rebuild-only mode versus
controlled remote-bundle mode.
3. Add an installer/catalog preflight that validates remote WebUI asset
manifests and records verified identity in installer run records.
4. Add Playwright coverage for a signed test remote bundle, failed digest, bad
module id, cache-key refresh, and fallback when the module is unavailable.
5. Add release tooling to emit immutable remote asset manifests with digest,
signature, key id, and rollback metadata.

@@ -0,0 +1,204 @@
<!-- codex-wiki-sync:e1a2e374fb859a18be46b0f5 -->
> Mirrored from `/mnt/DATA/git/govoplan-core/docs/SCALABILITY_AND_SIZING.md`.
> Origin: `repository`.
> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
---
# Scalability And Sizing Profiles
GovOPlaN scales manually first. Autoscaling can be added only where the runtime
surface is already stateless, health-checked, and backed by shared durable
services.
## Deployment Topologies
| Profile | Intended Use | Topology |
| --- | --- | --- |
| Development | Local feature work and module tests | One API process, Vite dev server, SQLite or disposable PostgreSQL, local file storage, synchronous workers. |
| Pilot | Single office, non-critical early usage | One API process, one WebUI build, PostgreSQL, local or object storage, optional Redis, one worker process when queues are enabled. |
| Small Production | One tenant or small agency | Two API processes behind a reverse proxy, PostgreSQL with backups, durable storage, Redis/Celery workers, health monitoring. |
| Medium Deployment | Municipal deployment with multiple departments | Separate WebUI, API, worker, scheduler, PostgreSQL, object storage, Redis, backup host, metrics/log collection. |
| Shared Platform | Multiple tenants or high campaign/workflow volume | Horizontally scaled WebUI/API/workers, managed PostgreSQL, object storage, queue/cache HA, central monitoring, controlled maintenance windows. |
## Stateless And Stateful Components
Stateless and horizontally replicable:
- WebUI static assets
- API workers when `MASTER_KEY_B64`, `DATABASE_URL`, storage, queue, and module
configuration are shared
- Background workers when queues and idempotency keys are used
- Scheduler replicas only when leader election or an external lock exists
Stateful or singleton-sensitive:
- PostgreSQL
- local file storage when not replaced by object storage
- Redis/queue state
- module installer daemon and package mutation operations
- migration execution
- scheduler without distributed locking
- outgoing campaign append/send jobs unless claim tokens are enforced
## Readiness And Degraded Modes
| Component | Ready When | Degraded Mode |
| --- | --- | --- |
| API | Database reachable, migrations current, enabled module registry builds, maintenance mode understood | Read-only/admin-only where routes allow it; otherwise fail closed. |
| WebUI | Static assets match backend module metadata contract | Show unavailable modules/routes with reason; do not invent routes. |
| PostgreSQL | Accepts connections and migration head is current | Block writes and package changes if migration state is unknown. |
| Storage | Configured backend is reachable and writable for write flows | Read-only file views may continue if storage is read-only but reachable. |
| Redis/Celery | Broker reachable and worker queues have heartbeats | Synchronous dev-only workflows may continue; production async send/workflow queues are degraded. |
| Installer daemon | Lock is free or owned by a live daemon; latest status is fresh | Admin UI can plan changes but not execute them. |
| Mail transport | SMTP/IMAP profiles validate for the selected scope | Campaign validation blocks send/append but allows draft editing. |
## Queue And Worker Scaling
Worker pools should be split by queue once load appears:
- `send_email`: SMTP send throughput, rate limits, retries, and outcome
uncertainty.
- `append_sent`: IMAP append latency and mailbox-side throttling.
- `workflow`: process orchestration and case/task state transitions.
- `transform`: datasource extraction, transformation, and export jobs.
- `notifications`: postbox, email notification, calendar, and external
notification fan-out.
- `reporting`: long-running exports, aggregates, and audit/report generation.
Scaling signal examples:
- queue depth and oldest queued job age
- retry rate and permanent failure rate
- worker CPU and memory saturation
- database lock time and query latency
- SMTP/IMAP provider throttling responses
- storage upload/download latency
Autoscaling should have upper bounds per queue. Mail and connector queues often
hit external throttles before CPU is exhausted, so adding workers blindly can
make failures worse.
## Database Path
PostgreSQL is the production database. SQLite remains a local-development and
tiny disposable profile only.
Production migrations should run explicitly before startup or package
activation. Module install/uninstall workflows must use database backup and
restore-check hooks for PostgreSQL before migrations or destructive retirement.
## First Sizing Matrix
These numbers are starting assumptions, not guarantees. Measure and adjust once
real workload metrics exist.
| Profile | CPU | Memory | Database | Storage | Queue/Cache | Backup/Monitoring Assumptions |
| --- | ---: | ---: | --- | --- | --- | --- |
| Development | 2 cores | 4-8 GB | SQLite or local PostgreSQL | local disk | optional | no SLA; manual reset acceptable |
| Pilot | 2-4 cores | 8 GB | PostgreSQL on same host or small managed instance | 100-500 GB durable local/object storage | optional Redis | daily DB backup; basic health checks |
| Small Production | 4-8 cores | 16 GB | dedicated PostgreSQL, 2-4 vCPU, 8-16 GB RAM | 0.5-2 TB object/durable storage | Redis plus 1-2 workers | daily full backup plus WAL/snapshot policy; uptime alerts |
| Medium Deployment | 8-16 API/worker cores total | 32-64 GB total | PostgreSQL 4-8 vCPU, 16-64 GB RAM | 2-10 TB object storage | Redis, separate worker pools | central logs/metrics, tested restore, queue alerts |
| Shared Platform | sized from measured load | 64 GB+ total | managed HA PostgreSQL, read replicas only after profiling | 10 TB+ object storage | HA queue/cache if required | SLOs, restore drills, capacity alerts, maintenance windows |
## Workload Dimensions For A Calculator
A later calculator should ask for:
- tenants and active users
- concurrent sessions and peak request rate
- files per month, average/max file size, and retention window
- cases/tasks/workflows per month
- campaigns per month, recipients per campaign, and send window
- IMAP append and inbound mailbox volume
- datasource/import/export job volume and file sizes
- report/audit query frequency
- retention policy and audit growth
- required recovery point and recovery time objectives
## Profile Selection Worksheet
The first calculator can be rule-based. It should recommend the lowest profile
that satisfies all hard constraints, then show the inputs that pushed the
operator upward.
| Input | Pilot Threshold | Small Production Threshold | Medium Threshold | Shared Platform Threshold |
| --- | ---: | ---: | ---: | ---: |
| Active tenants | 1 | 1-5 | 5-25 | 25+ |
| Concurrent users | up to 10 | up to 50 | up to 250 | measured/contracted |
| Managed files | under 100 GB | 100 GB-2 TB | 2-10 TB | 10 TB+ |
| Campaign recipients/month | under 5,000 | 5,000-100,000 | 100,000-1,000,000 | provider-limited or multi-tenant |
| Workflow/import jobs/day | under 100 | 100-2,000 | 2,000-25,000 | queue-specific scaling |
| Recovery point objective | daily backup | daily plus WAL/snapshots | tested restore, tighter RPO | formal SLO/SLA |
| Process split requirement | optional | API plus worker | API, workers, scheduler split | horizontal replicas |
Hard constraints override the numeric thresholds:
- Multiple API replicas require PostgreSQL and shared storage.
- Cross-node file access requires object storage or a shared durable file
service, not node-local disk.
- Async campaign send, append, imports, exports, or workflows require Redis and
workers outside development.
- Package install/uninstall in production requires maintenance mode, backup,
restore-check hooks, and installer daemon visibility.
- Autoscaling requires idempotent job claims, readiness probes, external
throttling limits, and queue-specific maximum replica counts.
## Calculator Output Contract
The future UI calculator should emit a structured recommendation:
```json
{
"recommended_profile": "small-production",
"minimum_components": [
"PostgreSQL",
"Redis",
"API process",
"worker process",
"durable file storage"
],
"reasons": [
"Campaign recipients/month exceed pilot threshold.",
"Async mail delivery requires worker split."
],
"warnings": [
"Object storage is recommended before adding a second API node."
],
"open_measurements": [
"Peak concurrent users",
"Database backup restore duration"
]
}
```
Until the calculator is implemented, operators should fill the worksheet
manually and compare it with the Ops page's current profile, readiness checks,
worker queues, and sizing assumptions.
## Minimum Production Requirements
A production deployment, even a small one, should have:
- PostgreSQL, explicit migrations, backups, and a tested restore path.
- A stable `MASTER_KEY_B64` stored outside the repository.
- HTTPS, exact CORS origins, secure cookies, and a reverse proxy.
- Durable file storage with backup or object-store lifecycle policy.
- Redis plus at least one worker when any queued module behavior is enabled.
- Health/readiness checks visible in `govoplan-ops`.
- Maintenance-mode access assigned to at least one operator account.
- Module catalog trust roots and license trust roots pinned by deployment
configuration.
## Manual Before Automatic
The first supported production scaling path is manual:
1. Move from SQLite/local storage to PostgreSQL and durable storage.
2. Add workers and Redis for queue-backed operations.
3. Split WebUI/API/worker processes.
4. Add health checks and deployment profile warnings.
5. Add metrics and queue-depth alerts.
6. Scale API and worker replicas with fixed limits.
7. Add autoscaling only after idempotency, readiness, and external throttles are
understood.