add-ideas/govoplan-core
1
0
Fork 0
Code Issues 45 Pull Requests Actions Packages Projects Releases Wiki Activity

Prepare GovOPlaN self-hosted release workflow
Some checks failed
Dependency Audit / dependency-audit (push) Failing after 50s
Details
Module Matrix / module-matrix (push) Failing after 49s
Details

Browse Source
This commit is contained in:
Albrecht Degering
2026-07-10 21:57:22 +02:00
parent 8dd5123aab
commit 94236a7d7e
51 changed files with 3576 additions and 803 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/DEPLOYMENT_OPERATOR_GUIDE.md
View File

@@ -7,6 +7,30 @@ files.
## Runtime Configuration Contract
Self-hosted installability follows the staged approach documented in
`SELF_HOSTED_INSTALLABILITY.md`: generate an explicit env template, validate it,
run production-like rehearsal with Compose-backed dependencies, then use the
installer CLI/daemon for package mutation under maintenance mode.
Generate a deployment-local template:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m govoplan_core.commands.config env-template \
--profile self-hosted \
--generate-secrets \
--output .env.self-hosted
```
Validate the active shell environment before migration or startup:
```bash
set -a
. .env.self-hosted
set +a
./.venv/bin/python -m govoplan_core.commands.config validate --profile self-hosted
```
### Required Runtime Identity
| Setting | Required outside dev | Purpose |
@@ -231,6 +255,16 @@ cd /mnt/DATA/git/govoplan-core
scripts/launch-production-like-dev.sh
```
The helper wrapper provides explicit lifecycle commands:
```bash
scripts/production-like-dev.sh validate-config
scripts/production-like-dev.sh seed
scripts/production-like-dev.sh start
scripts/production-like-dev.sh stop
scripts/production-like-dev.sh reset --yes
```
The launcher uses `dev/production-like/.env` when present, otherwise the checked
in `.env.example`. It runs:

1
docs/DOCUMENTATION_MAP.md
View File

@@ -21,6 +21,7 @@ operator, and roadmap pages.
| Topic | Canonical document | Notes |
| --- | --- | --- |
| Runtime configuration and operator flow | `DEPLOYMENT_OPERATOR_GUIDE.md` | Production/staging configuration, migrations, backups, installer operation, and rollback drill. |
| Self-hosted installability | `SELF_HOSTED_INSTALLABILITY.md` | Packaging decision, generated env templates, config validation, production-like dev stack commands, and boundary gate. |
| Release dependencies and catalogs | `RELEASE_DEPENDENCIES.md` | Release package refs, migration baselines, release lockfiles, catalog trust/licensing, catalog publishing, and release checklist. |
| Dependency vulnerability audits | `DEPENDENCY_AUDITS.md` | Local and CI audit commands plus dated audit result notes. |
| Remote WebUI bundle design | `REMOTE_WEBUI_BUNDLES.md` | Experimental controlled-deployment design; normal releases still use package builds. |

51
docs/MODULE_ARCHITECTURE.md
View File

@@ -80,6 +80,7 @@ The following contracts are the baseline API that modules can rely on:
- `ModuleManifest`
- `ModuleCompatibility`
- named interface contract provider/requirement metadata
- module uninstall guard provider contract
- `MigrationSpec`
- route factory contract
@@ -125,6 +126,50 @@ Feature modules should prefer these capabilities over direct reads of
access/tenant ORM models when they need labels, group membership, default
access provisioning, counts, audit actor labels, or tenant metadata.
### Named Interface Contracts
Capabilities are runtime objects. Named interface contracts are compatibility
metadata. A module uses them when it depends on a versioned cross-module API
shape but should not hard-code a package or repository release line.
Manifest fields:
- `provides_interfaces`: contracts this module provides, each with `name` and
`version`
- `requires_interfaces`: contracts this module needs, each with `name`,
optional `version_min`, optional `version_max_exclusive`, and optional
`optional: true`
Interface names use dot-separated lower-case identifiers such as
`files.spaces` or `mail.delivery`. A requirement range is interpreted as
`>= version_min` and `< version_max_exclusive`; the exclusive upper bound is
intended for SemVer major-version lines. Missing optional interfaces are
allowed, but an installed provider with an incompatible version blocks
activation because the integration would otherwise bind to an unsafe API.
Current named interfaces:
- `files.campaign_attachments`
- `mail.campaign_delivery`
- `campaigns.access`
- `campaigns.delivery_tasks`
- `campaigns.mail_policy_context`
- `campaigns.policy_context`
- `campaigns.retention`
Core validates named interface contracts in three places:
- registry activation rejects missing required interfaces and incompatible
providers
- installer preflight reports the same failures before a module set is
activated
- signed catalog validation normalizes the metadata and warns when catalog
entries cannot satisfy each other's ranges
Module-id dependencies still decide startup ordering and mandatory package
presence. Named interfaces decide whether the versions in the active module
set are compatible.
FastAPI route dependencies for authenticated endpoints are imported from the
core `govoplan_core.auth` facade. Routers may import that public API for
`ApiPrincipal`, `get_api_principal`, `has_scope`, `require_scope`, and
@@ -450,6 +495,8 @@ Modules should provide:
- pinned backend and WebUI package refs for official catalog entries
- compatibility metadata in the module manifest
- named interface contracts in the manifest and catalog entry when the module
provides or consumes cross-module APIs
- lifecycle hooks when a runtime enable/disable action needs module-specific
work
- uninstall guards for persistent data, active workers, schedulers, or external
@@ -621,6 +668,10 @@ The repository includes `scripts/check_dependency_boundaries.py`. It enforces th
- access source may not import files/mail/campaign internals
- feature modules may not import access implementation internals
- feature modules may not add new direct imports of sibling feature modules
- feature WebUI packages may not depend on or import sibling feature WebUI packages
- core WebUI may list module packages as host dependencies, but core WebUI source
may not import feature WebUI internals directly; module loading stays
declarative through the module contribution contract
- FastAPI routers import the core `govoplan_core.auth` dependency facade
- the transitional allowlist is expected to stay empty

12
docs/POLICY_CONTRACTS.md
View File

@@ -8,7 +8,7 @@ consistent while each module still owns its domain rules.
| Policy area | Current owner | Runtime surface | Notes |
| --- | --- | --- | --- |
| Privacy retention | `govoplan-policy` routes with compatibility helpers in core | `/api/v1/admin/privacy-retention/policies/{scope}` and `/explain` | System, tenant, user, group, and campaign sources merge into the effective retention policy. Parent locks block lower-level widening. |
| Privacy retention | `govoplan-policy` implementation and routes, with compatibility helpers in core | `/api/v1/admin/privacy-retention/policies/{scope}` and `/explain`; capability `policy.privacyRetention` | System, tenant, user, group, and campaign sources merge into the effective retention policy. Parent locks block lower-level widening. |
| Mail profile policy | `govoplan-mail` | `/api/v1/mail/policies/{scope}` | Uses the same source-step path format for system, tenant, owner, and campaign provenance. |
| RBAC/access policy | `govoplan-access` | access capabilities in `govoplan_core.core.access` | Permission decisions should use access capability contracts. Explain responses should adopt `PolicyDecision` when an API-level explanation is added. |
| Governance defaults | `govoplan-admin` plus `govoplan-access` materializer | admin settings, governance template routes, access materialization capability | System governance can block tenant-local groups, roles, and API keys. |
@@ -87,6 +87,16 @@ path. For lower-level scopes, `blocked_fields` is derived from the parent
policy's `allow_lower_level_limits`; clients can use it to disable local
controls before attempting a write.
The retention implementation lives in `govoplan-policy`
(`govoplan_policy.backend.retention`). Core keeps
`govoplan_core.privacy.retention` only as a compatibility facade for older
imports. Effective/scoped retention behavior dispatches through the
`policy.privacyRetention` capability; core does not import policy implementation
code as a hidden fallback when the module is disabled or no runtime is active.
New backend code should import policy-owned retention behavior from
`govoplan-policy` or request the capability, not add new implementation logic
to core.
## Frontend Contract
Policy UIs must:

108
docs/RELEASE_DEPENDENCIES.md
View File

@@ -107,6 +107,15 @@ cd /mnt/DATA/git/govoplan-core
scripts/push-release-tag.sh --version 0.1.6
```
`scripts/generate-release-catalog.py` reads installed/discovered
`ModuleManifest` objects while writing catalog entries. When a manifest is
available, the catalog entry uses the manifest version, points package refs at
`v<manifest.version>`, and copies `provides_interfaces` /
`requires_interfaces` from the manifest. If a manifest cannot be discovered,
the entry falls back to the release version passed with `--version` and omits
interface metadata. This keeps the catalog aligned with independently
versioned module packages instead of relying on a hardcoded compatibility table.
The script also includes GovOPlaN roadmap/scaffold module repositories that do
not yet have package metadata. Those repositories are committed, tagged, and
pushed with the same release tag, but they are tag-only until they contain
@@ -196,10 +205,109 @@ Each module entry can declare:
- WebUI package name and pinned install reference
- display metadata and tags
- `license_features`, the feature entitlements required to plan that install
- `provides_interfaces`, named interface contracts exported by this module
- `requires_interfaces`, named interface contracts and version ranges required
by this module
The signature is Ed25519 over canonical JSON with both `signature` and
`signatures` removed. Core accepts the legacy single `signature` field and the
new `signatures` array.
When `APP_ENV` is `prod` or `production`, module package catalog signature
verification is required by default unless
`GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=false` is set explicitly.
### Independent Module Versions And Interface Ranges
Modules do not need to ship on the same version number. Release catalogs should
pin each package to the exact backend/WebUI ref being installed and declare any
cross-module API compatibility through named interfaces.
Provider shape:
```json
"provides_interfaces": [
{ "name": "files.campaign_attachments", "version": "1.4.0" }
]
```
Requirement shape:
```json
"requires_interfaces": [
{
"name": "files.campaign_attachments",
"version_min": "1.0.0",
"version_max_exclusive": "2.0.0"
}
]
```
`version_min` is inclusive. `version_max_exclusive` is exclusive, so the range
above means `>= 1.0.0` and `< 2.0.0`. Use this for SemVer major-version
compatibility lines. Set `"optional": true` only when the module can operate
without that interface being present. If a provider is installed but its
version is outside the declared optional range, activation is still blocked.
Catalog validation normalizes these fields and warns when catalog entries do
not satisfy each other's ranges. Registry activation and installer preflight
perform the blocking checks against the discovered installed manifests before
the desired module set is activated.
The admin module-management UI shows catalog warnings in the package catalog
section and repeats them as warning-level installer preflight issues while a
package install is planned.
Install-plan items carry a `source` field. Manually entered items use
`source: "manual"`; entries planned from the package catalog use
`source: "catalog"`. Catalog-sourced items also carry a `catalog` metadata
object with the validation snapshot used when the item was planned: catalog
source/path, source type, cache path, channel, sequence, generated/validity
timestamps, signature state, trusted key id, and cache state where available.
Catalog provenance changes preflight severity:
- catalog-sourced installs require a configured, valid package catalog before
activation
- invalid, untrusted, expired, not-yet-valid, replayed, or unapproved-channel
catalogs block catalog-sourced installs
- the same catalog validation failures remain warnings for manual install
plans, so operators can still use offline or emergency package refs
- valid-catalog warnings, such as intentionally unsigned local catalogs when
signature enforcement is disabled, remain warnings
- selected catalog entries with unsatisfied non-optional named interface ranges
block activation before the installer runs
### Update Paths
Package updates are target-state operations, not one-module-at-a-time runtime
toggles. The safe unit of planning is a desired module version set plus a
catalog validation snapshot. The installer may install multiple packages into
the environment before activation, then validate the discovered manifests and
activate the resulting set together.
This avoids circular "upgrade A first / upgrade B first" traps: named interface
requirements are solved against the target set, not against each intermediate
package-install moment. If the target set cannot satisfy all non-optional
interfaces and module dependencies at once, the plan is invalid. Operators
should add the necessary module updates to the same plan instead of trying to
force an order.
Live data upgrades need an even stricter rule:
- migrations must be idempotent and ordered by module migration metadata
- destructive schema/data changes need an explicit retirement or cleanup plan,
not an automatic package update side effect
- cross-module data migrations must be compatible with both the old and target
provider interface until activation finishes
- rollback must restore the package set and database state together, or be
documented as forward-only with a tested recovery procedure
- if two modules require mutually incompatible live-data states, the catalog
must publish an intermediate compatibility release rather than a circular
update chain
In practice, circular dependencies are avoided by designing interfaces with
compatibility windows and by publishing bridge releases. A bridge release keeps
the old interface while introducing the new one, allowing dependent modules to
move first; a later release can retire the old interface after every dependent
module has a compatible target version.
Trusted catalog keys are configured locally:

80
docs/SELF_HOSTED_INSTALLABILITY.md Normal file
View File

@@ -0,0 +1,80 @@
# Self-Hosted Installability
GovOPlaN uses a staged self-hosted installability path.
## Packaging Decision
The early packaging approach is a staged combination:
1. Generate an explicit environment template and validate it before startup.
2. Use a Compose-backed production-like development profile for local rehearsal.
3. Use the deployment operator guide as the runbook for migrations, workers,
backups, health checks, and module installer rollback drills.
4. Use the module installer CLI/daemon for package mutation once the runtime is
already installed and under maintenance mode.
This keeps first installation understandable while still preserving the later
goal of install/update/uninstall through signed catalogs and the installer
daemon. The API server must not run package managers from request handlers.
## Config Bootstrap
Generate a self-hosted template:
```bash
cd /mnt/DATA/git/govoplan-core
./.venv/bin/python -m govoplan_core.commands.config env-template \
--profile self-hosted \
--generate-secrets \
--output .env.self-hosted
```
Validate the current shell environment:
```bash
set -a
. .env.self-hosted
set +a
./.venv/bin/python -m govoplan_core.commands.config validate --profile self-hosted
```
The command reports all known blockers at once. Production-like/self-hosted
profiles require explicit `APP_ENV`, `DATABASE_URL`, `MASTER_KEY_B64`,
`ENABLED_MODULES`, and `CORS_ORIGINS`. Production rejects SQLite, development
bootstrap, insecure auth cookies, and unsigned catalog trust roots when a
catalog source is configured.
## Production-Like Dev Stack
Use the local production-like wrapper for repeatable rehearsal:
```bash
cd /mnt/DATA/git/govoplan-core
scripts/production-like-dev.sh validate-config
scripts/production-like-dev.sh seed
scripts/production-like-dev.sh start
```
Stop Docker dependencies:
```bash
scripts/production-like-dev.sh stop
```
Reset all profile data:
```bash
scripts/production-like-dev.sh reset --yes
```
The start command delegates to `scripts/launch-production-like-dev.sh`, which
runs API, worker, and WebUI in the foreground. Stop those processes with
`Ctrl+C` in the launcher terminal.
## Module Boundary Gate
`scripts/check_dependency_boundaries.py` is part of the focused verification
path. It checks backend imports and WebUI package/source imports so modules do
not grow hidden runtime dependencies on each other. Feature modules should
integrate through core capabilities, backend APIs/events, route contributions,
or explicit UI extension points.

20
docs/module-package-catalog.example.json
View File

@@ -15,6 +15,20 @@
"python_ref": "govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.4",
"webui_package": "@govoplan/files-webui",
"webui_ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git#v0.1.4",
"provides_interfaces": [
{
"name": "files.campaign_attachments",
"version": "1.4.0"
}
],
"requires_interfaces": [
{
"name": "campaigns.access",
"version_min": "1.0.0",
"version_max_exclusive": "2.0.0",
"optional": true
}
],
"artifact_integrity": {
"python": {
"ref": "govoplan-files @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-files.git@v0.1.4",
@@ -44,6 +58,12 @@
"python_ref": "govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.4",
"webui_package": "@govoplan/mail-webui",
"webui_ref": "git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git#v0.1.4",
"provides_interfaces": [
{
"name": "mail.campaign_delivery",
"version": "1.4.0"
}
],
"artifact_integrity": {
"python": {
"ref": "govoplan-mail @ git+ssh://git@git.add-ideas.de/add-ideas/govoplan-mail.git@v0.1.4",