From 7f81edccab3117b6a3b8141ea5ddbbe2fe11733d Mon Sep 17 00:00:00 2001 From: Albrecht Degering Date: Wed, 5 Aug 2026 16:49:06 +0200 Subject: [PATCH] Sync wiki from project files --- Codex-Project-Index.md | 11 + Repo-README.-.md | 178 -- Repo-README.md | 4 +- Repo-docs-ACTION-EFFECT-AUTOMATION-LAYER.md | 28 +- Repo-docs-COMPATIBILITY-INVENTORY.md | 3 +- Repo-docs-CONTEXTUAL-HELP-CONTRACT.md | 72 + Repo-docs-DEPENDENCY-AUDITS.-.md | 74 - Repo-docs-DEPLOYMENT-OPERATOR-GUIDE.md | 54 +- Repo-docs-DOCUMENTATION-MAP.-.md | 55 - Repo-docs-DOCUMENTATION-MAP.md | 10 +- Repo-docs-DURABLE-RECOVERY-OPERATIONS.md | 51 + Repo-docs-INFORMATION-GOVERNANCE-ADOPTION.md | 138 ++ Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.-.md | 166 -- Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md | 19 +- Repo-docs-INTERFACE-PATTERN-MIGRATION.md | 35 + Repo-docs-LOCALIZATION-AND-HELP-QUALITY.md | 67 + Repo-docs-MODULE-ARCHITECTURE.-.md | 1514 ----------------- Repo-docs-MODULE-ARCHITECTURE.md | 147 +- Repo-docs-MODULE-LIFECYCLE-RECOVERY.md | 73 + Repo-docs-RELEASE-DEPENDENCIES.-.md | 629 ------- Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT.md | 33 + Repo-docs-STATE-AND-RECOVERY-CONTRACT.md | 14 +- Repo-docs-TABULAR-SOURCE-CONTRACT.md | 28 + Repo-docs-TEMPLATE-CAPABILITY-CONTRACT.md | 35 + Repo-docs-TEMPORAL-DATA-CONTEXT.md | 75 + Repo-docs-UI-UX-DECISION-LEDGER.md | 28 +- Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT.md | 19 + ...cs-audits-2026-07-09-dependency-audit.-.md | 78 - 28 files changed, 912 insertions(+), 2726 deletions(-) delete mode 100644 Repo-README.-.md create mode 100644 Repo-docs-CONTEXTUAL-HELP-CONTRACT.md delete mode 100644 Repo-docs-DEPENDENCY-AUDITS.-.md delete mode 100644 Repo-docs-DOCUMENTATION-MAP.-.md create mode 100644 Repo-docs-DURABLE-RECOVERY-OPERATIONS.md create mode 100644 Repo-docs-INFORMATION-GOVERNANCE-ADOPTION.md delete mode 100644 Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.-.md create mode 100644 Repo-docs-INTERFACE-PATTERN-MIGRATION.md create mode 100644 Repo-docs-LOCALIZATION-AND-HELP-QUALITY.md delete mode 100644 Repo-docs-MODULE-ARCHITECTURE.-.md create mode 100644 Repo-docs-MODULE-LIFECYCLE-RECOVERY.md delete mode 100644 Repo-docs-RELEASE-DEPENDENCIES.-.md create mode 100644 Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT.md create mode 100644 Repo-docs-TABULAR-SOURCE-CONTRACT.md create mode 100644 Repo-docs-TEMPLATE-CAPABILITY-CONTRACT.md create mode 100644 Repo-docs-TEMPORAL-DATA-CONTEXT.md create mode 100644 Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT.md delete mode 100644 Repo-docs-audits-2026-07-09-dependency-audit.-.md diff --git a/Codex-Project-Index.md b/Codex-Project-Index.md index 8572de0..c7bddfe 100644 --- a/Codex-Project-Index.md +++ b/Codex-Project-Index.md @@ -14,28 +14,39 @@ This page is generated from repository and product-directory project files. - [Repo-docs-COMPATIBILITY-INVENTORY](Repo-docs-COMPATIBILITY-INVENTORY) - `/mnt/DATA/git/govoplan-core/docs/COMPATIBILITY_INVENTORY.md` - [Repo-docs-COMPATIBILITY-POLICY](Repo-docs-COMPATIBILITY-POLICY) - `/mnt/DATA/git/govoplan-core/docs/COMPATIBILITY_POLICY.md` - [Repo-docs-CONFIGURATION-PACKAGES](Repo-docs-CONFIGURATION-PACKAGES) - `/mnt/DATA/git/govoplan-core/docs/CONFIGURATION_PACKAGES.md` +- [Repo-docs-CONTEXTUAL-HELP-CONTRACT](Repo-docs-CONTEXTUAL-HELP-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/CONTEXTUAL_HELP_CONTRACT.md` - [Repo-docs-DATAGRID-SIZING-CONTRACT](Repo-docs-DATAGRID-SIZING-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/DATAGRID_SIZING_CONTRACT.md` - [Repo-docs-DEPENDENCY-AUDITS](Repo-docs-DEPENDENCY-AUDITS) - `/mnt/DATA/git/govoplan-core/docs/DEPENDENCY_AUDITS.md` - [Repo-docs-DEPLOYMENT-OPERATOR-GUIDE](Repo-docs-DEPLOYMENT-OPERATOR-GUIDE) - `/mnt/DATA/git/govoplan-core/docs/DEPLOYMENT_OPERATOR_GUIDE.md` - [Repo-docs-DOCUMENTATION-MAP](Repo-docs-DOCUMENTATION-MAP) - `/mnt/DATA/git/govoplan-core/docs/DOCUMENTATION_MAP.md` +- [Repo-docs-DURABLE-RECOVERY-OPERATIONS](Repo-docs-DURABLE-RECOVERY-OPERATIONS) - `/mnt/DATA/git/govoplan-core/docs/DURABLE_RECOVERY_OPERATIONS.md` - [Repo-docs-EVENTS-AND-AUDIT](Repo-docs-EVENTS-AND-AUDIT) - `/mnt/DATA/git/govoplan-core/docs/EVENTS_AND_AUDIT.md` - [Repo-docs-EXTERNAL-REFERENCES-AND-INTEGRATION-MATURITY](Repo-docs-EXTERNAL-REFERENCES-AND-INTEGRATION-MATURITY) - `/mnt/DATA/git/govoplan-core/docs/EXTERNAL_REFERENCES_AND_INTEGRATION_MATURITY.md` - [Repo-docs-GITEA-ISSUES](Repo-docs-GITEA-ISSUES) - `/mnt/DATA/git/govoplan-core/docs/GITEA_ISSUES.md` - [Repo-docs-GOVERNANCE-MODEL](Repo-docs-GOVERNANCE-MODEL) - `/mnt/DATA/git/govoplan-core/docs/GOVERNANCE_MODEL.md` - [Repo-docs-GOVOPLAN-MASTER-ROADMAP](Repo-docs-GOVOPLAN-MASTER-ROADMAP) - `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MASTER_ROADMAP.md` +- [Repo-docs-INFORMATION-GOVERNANCE-ADOPTION](Repo-docs-INFORMATION-GOVERNANCE-ADOPTION) - `/mnt/DATA/git/govoplan-core/docs/INFORMATION_GOVERNANCE_ADOPTION.md` - [Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT](Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/INSTITUTIONAL_CONTEXT_CONTRACT.md` - [Repo-docs-INTERFACE-ETHICS-AND-DESIGN-DOCTRINE](Repo-docs-INTERFACE-ETHICS-AND-DESIGN-DOCTRINE) - `/mnt/DATA/git/govoplan-core/docs/INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` +- [Repo-docs-INTERFACE-PATTERN-MIGRATION](Repo-docs-INTERFACE-PATTERN-MIGRATION) - `/mnt/DATA/git/govoplan-core/docs/INTERFACE_PATTERN_MIGRATION.md` +- [Repo-docs-LOCALIZATION-AND-HELP-QUALITY](Repo-docs-LOCALIZATION-AND-HELP-QUALITY) - `/mnt/DATA/git/govoplan-core/docs/LOCALIZATION_AND_HELP_QUALITY.md` - [Repo-docs-MODULE-ARCHITECTURE](Repo-docs-MODULE-ARCHITECTURE) - `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md` +- [Repo-docs-MODULE-LIFECYCLE-RECOVERY](Repo-docs-MODULE-LIFECYCLE-RECOVERY) - `/mnt/DATA/git/govoplan-core/docs/MODULE_LIFECYCLE_RECOVERY.md` - [Repo-docs-POLICY-CONTRACTS](Repo-docs-POLICY-CONTRACTS) - `/mnt/DATA/git/govoplan-core/docs/POLICY_CONTRACTS.md` - [Repo-docs-POSTBOX-E2EE-ARCHITECTURE](Repo-docs-POSTBOX-E2EE-ARCHITECTURE) - `/mnt/DATA/git/govoplan-core/docs/POSTBOX_E2EE_ARCHITECTURE.md` - [Repo-docs-PUBLIC-SECTOR-INTEGRATION-STRATEGY](Repo-docs-PUBLIC-SECTOR-INTEGRATION-STRATEGY) - `/mnt/DATA/git/govoplan-core/docs/PUBLIC_SECTOR_INTEGRATION_STRATEGY.md` - [Repo-docs-RELEASE-DEPENDENCIES](Repo-docs-RELEASE-DEPENDENCIES) - `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md` - [Repo-docs-REMOTE-WEBUI-BUNDLES](Repo-docs-REMOTE-WEBUI-BUNDLES) - `/mnt/DATA/git/govoplan-core/docs/REMOTE_WEBUI_BUNDLES.md` +- [Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT](Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/SEARCH_EVENT_INDEXING_CONTRACT.md` - [Repo-docs-SECURITY-AUDIT](Repo-docs-SECURITY-AUDIT) - `/mnt/DATA/git/govoplan-core/docs/SECURITY_AUDIT.md` - [Repo-docs-SELF-HOSTED-INSTALLABILITY](Repo-docs-SELF-HOSTED-INSTALLABILITY) - `/mnt/DATA/git/govoplan-core/docs/SELF_HOSTED_INSTALLABILITY.md` - [Repo-docs-STATE-AND-RECOVERY-CONTRACT](Repo-docs-STATE-AND-RECOVERY-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/STATE_AND_RECOVERY_CONTRACT.md` +- [Repo-docs-TABULAR-SOURCE-CONTRACT](Repo-docs-TABULAR-SOURCE-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/TABULAR_SOURCE_CONTRACT.md` +- [Repo-docs-TEMPLATE-CAPABILITY-CONTRACT](Repo-docs-TEMPLATE-CAPABILITY-CONTRACT) - `/mnt/DATA/git/govoplan-core/docs/TEMPLATE_CAPABILITY_CONTRACT.md` +- [Repo-docs-TEMPORAL-DATA-CONTEXT](Repo-docs-TEMPORAL-DATA-CONTEXT) - `/mnt/DATA/git/govoplan-core/docs/TEMPORAL_DATA_CONTEXT.md` - [Repo-docs-THEMING](Repo-docs-THEMING) - `/mnt/DATA/git/govoplan-core/docs/THEMING.md` - [Repo-docs-THROTTLING](Repo-docs-THROTTLING) - `/mnt/DATA/git/govoplan-core/docs/THROTTLING.md` - [Repo-docs-UI-UX-DECISION-LEDGER](Repo-docs-UI-UX-DECISION-LEDGER) - `/mnt/DATA/git/govoplan-core/docs/UI_UX_DECISION_LEDGER.md` - [Repo-docs-WEBUI-BUNDLE-BUDGETS](Repo-docs-WEBUI-BUNDLE-BUDGETS) - `/mnt/DATA/git/govoplan-core/docs/WEBUI_BUNDLE_BUDGETS.md` +- [Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT](Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT) - `/mnt/DATA/git/govoplan-core/docs/WEBUI_MODULE_PACKAGE_LAYOUT.md` - [Repo-docs-audits-2026-07-09-dependency-audit](Repo-docs-audits-2026-07-09-dependency-audit) - `/mnt/DATA/git/govoplan-core/docs/audits/2026-07-09-dependency-audit.md` diff --git a/Repo-README.-.md b/Repo-README.-.md deleted file mode 100644 index cfebc53..0000000 --- a/Repo-README.-.md +++ /dev/null @@ -1,178 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/README.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# govoplan-core - - -**Repository type:** system (kernel). - - -GovOPlaN core is the platform runner and shared foundation. It owns the server entry point, database/session primitives, module discovery, migration orchestration, capability contracts, install/uninstall orchestration, and the shared WebUI shell. Platform and feature behavior is supplied by installed modules. - -## Repository ownership - -Core owns: - -- `govoplan_core.server.app:app`, the FastAPI entry point used by uvicorn -- `GovoplanServerConfig`, module discovery, registry validation, and route aggregation -- SQLAlchemy base/session helpers and module migration registration -- kernel APIs for platform metadata, module lifecycle, health, and development diagnostics -- `@govoplan/core-webui`, including login, CSRF/API helpers, shell layout, generic UI components, IconRail, DataGrid, access boundaries, and module route/nav contracts - -The shared DataGrid sizing and resize invariants are specified in -[`docs/DATAGRID_SIZING_CONTRACT.md`](docs/DATAGRID_SIZING_CONTRACT.md). - -Platform and feature modules own their backend routers, models, migrations, -permissions, frontend packages, nav items, and route contributions. Access, -tenancy, policy, audit, and admin behavior live in their owning platform -modules. Core should not import feature pages directly; it imports module -manifests and renders their route contributions. - -## Governance docs - -Canonical policy documents live in `docs/`: - -- [DOCUMENTATION_MAP.md](docs/DOCUMENTATION_MAP.md) -- [ACCESS_RBAC_MODEL.md](docs/ACCESS_RBAC_MODEL.md) -- [GOVERNANCE_MODEL.md](docs/GOVERNANCE_MODEL.md) -- [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md) -- [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md) -- [CODEX_WORKFLOW.md](docs/CODEX_WORKFLOW.md) - -Modules define module-specific permissions and policy behavior. Shared DTOs and -composition rules live in core only where they are stable kernel contracts. - -## Backend development - -For whole-product development, create the virtualenv from the meta repository: - -```bash -cd /mnt/DATA/git/govoplan -python3 -m venv .venv -./.venv/bin/python -m pip install --upgrade pip -./.venv/bin/python -m pip install -r requirements-dev.txt -``` - -Run the platform server from core through the module-aware development runner. The default config reads `ENABLED_MODULES` and discovers installed module entry points. Local development defaults to the modules listed by `govoplan_core.settings.Settings.enabled_modules`, including Views when its package is installed; set `ENABLED_MODULES` explicitly when testing a smaller module permutation. - -```bash -cd /mnt/DATA/git/govoplan-core -/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver \ - --host 127.0.0.1 \ - --port 8000 -``` - -For example, to test campaign without files or mail: - -```bash -cd /mnt/DATA/git/govoplan-core -ENABLED_MODULES=access,campaigns /mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver \ - --host 127.0.0.1 \ - --port 8000 -``` - -The runner loads the same `GovoplanServerConfig` as `govoplan_core.server.app:app`, builds the platform registry, and passes core plus enabled module source roots to uvicorn as reload directories. After reinstalling the editable package, the same command is also available as `govoplan-devserver`. - -For focused backend work, keep the complete module graph active while watching -only the module being edited. Core/config sources and explicit `--reload-dir` -paths remain watched: - -```bash -/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver \ - --reload-module calendar \ - --reload-module campaign -``` - -Use `--reload-core-only` when no optional module source tree should trigger a -restart. Omitting both options preserves the broad default and watches every -enabled module. Startup, migration, and compatibility checks still run against -the complete enabled graph whenever the backend restarts. - -The default development database is PostgreSQL at `postgresql+psycopg://govoplan_dev@127.0.0.1:5432/govoplan_dev`. Store the password in `~/.pgpass`. To force the disposable SQLite fallback, run with `GOVOPLAN_DEV_DATABASE_BACKEND=sqlite`; that database lives below `runtime/`. - -Local devserver runs do not require Redis. `CELERY_ENABLED` defaults to `false`, so campaign queue actions update database state without publishing Celery tasks. Use the synchronous send flow for local send tests, or set `CELERY_ENABLED=true` only when a Redis broker and worker are running. - -To run the production-like local profile with PostgreSQL, Redis, a Celery -worker, explicit module configuration, and persistent local file storage: - -```bash -cd /mnt/DATA/git/govoplan -tools/launch/launch-production-like-dev.sh -``` - -See `/mnt/DATA/git/govoplan/dev/production-like/README.md` for ports, -environment overrides, and cleanup commands. Core keeps wrapper commands during -the migration, but whole-product profiles are owned by the meta repository. - -## Security audit - -The repository includes a containerized audit toolbox for SAST, secret scanning, -dependency checks, filesystem misconfiguration scans, duplication, and complexity -reports. See [SECURITY_AUDIT.md](docs/SECURITY_AUDIT.md) for the operating model. - -```bash -cd /mnt/DATA/git/govoplan -tools/checks/security-audit/run.sh --mode ci --scope govoplan -tools/checks/security-audit/run.sh --mode full --scope govoplan -``` - -CI runs the `ci` profile in report-only mode and uploads `audit-reports/` as an -artifact. Once the baseline is clean, set `SECURITY_AUDIT_FAIL_ON_FINDINGS=1` -or pass `--strict` locally to turn findings into a failing gate. - -`govoplan_core.devserver` enables the development bootstrap before loading settings. In dev, startup migrations create or upgrade the schema and the bootstrap creates the default development login if needed. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead. - -To verify the effective runtime paths and bootstrap behavior without starting uvicorn, run the smoke mode: - -```bash -cd /mnt/DATA/git/govoplan-core -/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.devserver --smoke --no-reload -``` - -The smoke mode prints the effective config, runtime root, database URL, modules, reload state, and bootstrap decision, then creates the ASGI app and runs startup once. - -The meta repository owns whole-product `requirements-dev.txt`, -`requirements-release.txt`, and the root `.env.example` operator template. Core -keeps package metadata and runtime commands. See -[RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md). - -For the install/runtime configuration contract and operator deployment flow, see [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md). -For self-hosted config bootstrap and validation: - -```bash -cd /mnt/DATA/git/govoplan-core -/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.commands.config env-template --profile self-hosted --generate-secrets -/mnt/DATA/git/govoplan/.venv/bin/python -m govoplan_core.commands.config validate --profile self-hosted -``` - -## WebUI development - -Install and run from the core WebUI host: - -```bash -cd /mnt/DATA/git/govoplan-core/webui -PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm install -PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run dev -``` - -The local host links sibling module WebUI packages through local file dependencies and Vite filesystem allowances. Release builds should use `webui/package.release.json`, which points WebUI module packages at tagged git refs. See [RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md). - -Production builds lazy-load enabled module descriptors and enforce initial and -asynchronous JavaScript budgets. See -[WEBUI_BUNDLE_BUDGETS.md](docs/WEBUI_BUNDLE_BUDGETS.md). - -## Module contract - -Backend modules register through the `govoplan.modules` entry point and return a `ModuleManifest`. A manifest can contribute: - -- permissions and role templates -- API routers -- SQLAlchemy metadata and migration locations -- nav metadata and frontend package metadata -- resource ACL providers and tenant summary/delete-veto providers - -WebUI modules export a `PlatformWebModule` with nav items and route contributions. Core renders those routes with `settings` and `auth` context. Frontend nav icons must be supplied as core-resolved `iconName` strings, not imported icon components. See [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md) for the full module-building contract. diff --git a/Repo-README.md b/Repo-README.md index cfebc53..d78b475 100644 --- a/Repo-README.md +++ b/Repo-README.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/README.md`. > Origin: `repository`. @@ -124,7 +124,7 @@ CI runs the `ci` profile in report-only mode and uploads `audit-reports/` as an artifact. Once the baseline is clean, set `SECURITY_AUDIT_FAIL_ON_FINDINGS=1` or pass `--strict` locally to turn findings into a failing gate. -`govoplan_core.devserver` enables the development bootstrap before loading settings. In dev, startup migrations create or upgrade the schema and the bootstrap creates the default development login if needed. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead. +`govoplan_core.devserver` enables the development bootstrap before loading settings. In dev, startup migrations create or upgrade the schema and the bootstrap creates the default development login if needed. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments use the separate, expiring single-use flow exposed by `python -m govoplan_core.commands.first_admin`; it cannot enable or consume development bootstrap credentials. To verify the effective runtime paths and bootstrap behavior without starting uvicorn, run the smoke mode: diff --git a/Repo-docs-ACTION-EFFECT-AUTOMATION-LAYER.md b/Repo-docs-ACTION-EFFECT-AUTOMATION-LAYER.md index a765405..44411dc 100644 --- a/Repo-docs-ACTION-EFFECT-AUTOMATION-LAYER.md +++ b/Repo-docs-ACTION-EFFECT-AUTOMATION-LAYER.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/ACTION_EFFECT_AUTOMATION_LAYER.md`. > Origin: `repository`. @@ -54,6 +54,9 @@ Recommended fields: irreversible - expected effects - idempotency key strategy +- recovery mode: atomic, compensating, snapshot restore, forward recovery, or + irreversible +- concrete verification steps which prove whether the effect occurred - audit event names - preview provider @@ -96,10 +99,14 @@ The runner should execute an action plan as follows: 4. Run permission and policy checks. 5. Generate a consequence preview. 6. Reserve or verify the idempotency key. -7. Execute the owning module capability. -8. Record observed effects. -9. Emit events and audit records. -10. Mark the command complete, retryable, quarantined, or requiring manual +7. Create a durable recovery operation and acquire its execution fence. +8. Persist dispatch evidence before a non-atomic provider call. +9. Execute the owning module capability. +10. Verify the provider result and every announced effect using the action's + declared recovery checks. +11. Commit the local projection and verified recovery checkpoint together. +12. Emit events and audit records. +13. Mark the command complete, retryable, quarantined, or requiring manual intervention. The runner must never advance workflow state past a required side effect unless @@ -117,7 +124,16 @@ between: 6. reconciled, corrected, or compensated outcome. An API timeout after dispatch is not a failed effect and must not be retried as -a fresh command. The actor context should retain the real identity/account, +an ordinary process failure or a fresh command. The runner records an unknown +outcome, releases its execution authority, and blocks continuation until an +operator or provider reconciliation proves either that the effect occurred or +that it is absent. + +`ActionDefinition.recovery_mode` and `recovery_verification` are part of the +provider contract. The default is conservative forward recovery with explicit +provider-result and effect verification. Atomic mode is valid only when the +provider effect and its local projection share the same database transaction. +The actor context should retain the real identity/account, represented function or party, delegation or power, and mandate/jurisdiction references when applicable. Domain modules remain responsible for deciding which of those references are required for their action. diff --git a/Repo-docs-COMPATIBILITY-INVENTORY.md b/Repo-docs-COMPATIBILITY-INVENTORY.md index 2d9ddd5..6cfabd2 100644 --- a/Repo-docs-COMPATIBILITY-INVENTORY.md +++ b/Repo-docs-COMPATIBILITY-INVENTORY.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/COMPATIBILITY_INVENTORY.md`. > Origin: `repository`. @@ -38,6 +38,7 @@ such as Redis degradation and language fallback are not compatibility paths. | Legacy tenant aliases in API response schemas | Preserves active-tenant response fields used by `0.1.x` clients. | Tagged `0.1.x` API window. | Remove at `0.2` with response-schema migration notes. | | Optional fields in Poll response references | Accepts providers built against the earlier Poll contract. | Tagged `0.1.x` runtime contract window. | Remove or require a new interface version at `0.2`. | | Legacy single-tenant summary providers | Allows modules without the batch provider introduced in `0.1.x`. | Tagged `0.1.x` module contract window. | Remove at `0.2` after manifests advertise the batch provider contract. | +| WebUI `react-router-dom` build alias | Resolves tagged `0.1.x` module source imports to Core's single `react-router` runtime so one composition never loads two router contexts. | Tagged `0.1.x` WebUI source window. | Remove at `0.2` after every supported module tag imports `react-router` directly. | ## Removed Paths diff --git a/Repo-docs-CONTEXTUAL-HELP-CONTRACT.md b/Repo-docs-CONTEXTUAL-HELP-CONTRACT.md new file mode 100644 index 0000000..6fedb72 --- /dev/null +++ b/Repo-docs-CONTEXTUAL-HELP-CONTRACT.md @@ -0,0 +1,72 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/CONTEXTUAL_HELP_CONTRACT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Contextual Help Contract + +GovOPlaN exposes context-sensitive help through `F1` and the titlebar help +control. The shell resolves a stable help identity from the focused control, +its containing surface, and the current route. The Docs module then projects +the best visible user or administrator topic for that identity. + +## Resolution Order + +The WebUI resolves help in this order: + +1. an explicit `helpContextId` or `data-help-context-id` on the focused item +2. the focused shared control's `interfaceId`, `helpTopicId`, and label key +3. a containing dialog, card, administration section, or page surface +4. the current registered route, including dynamic module routes +5. a stable route-derived fallback when no explicit identity is available + +Focused field and action contexts retain the page context as +`fallback_context`. This lets Docs show a field-specific topic when one exists +and otherwise open the owning page or module documentation instead of a generic +help page. + +## Documentation Lookup + +Static `DocumentationTopic` contributions announce exact contexts through +`metadata.help_contexts`. Core publishes that catalogue with the enabled module +manifest, allowing the shell to link directly to an exact topic when possible. +Docs still performs the authoritative audience, permission, configured-state, +and documentation-type filtering. + +Core also maps explicit route, navigation, settings, and View surface IDs to +the module's static user or administrator documentation baseline. This makes a +page association complete by default and gives every derived field/action +context a useful fallback. Exact `metadata.help_contexts` remain the preferred +authoring mechanism for consequential or unfamiliar controls. + +When there is no exact topic, Docs resolves the page fallback and then the first +visible topic owned by the module. If Docs is unavailable, the shell opens the +hosted documentation with the same context parameters. + +## Authoring Controls + +Core shared controls expose stable help metadata. Prefer these props rather +than adding custom `F1` listeners: + +- `interfaceId` identifies a durable UI surface or action. +- `helpContextId` identifies a documentation context when it differs from the + interface identity. +- `helpTopicId` links directly to a module-owned documentation topic. +- translated label keys provide deterministic field identities for ordinary + `FormField`, `ToggleSwitch`, search, date/time, email, button, dialog, and card + controls. + +Module routes, public routes, settings sections, and administration sections +may also declare `helpContextId` and `helpTopicId`. Each module must keep a +static user/admin documentation baseline and should list its important route, +workflow, setting, permission, and limitation identities in +`metadata.help_contexts`. + +## Boundary + +Help identities describe presentation context; they are not authorization +claims. Opening help never bypasses route or documentation permissions. Docs +owns documentation projection, feature modules own their content, and Core owns +focus capture, context resolution, and fallback routing. diff --git a/Repo-docs-DEPENDENCY-AUDITS.-.md b/Repo-docs-DEPENDENCY-AUDITS.-.md deleted file mode 100644 index a879bd9..0000000 --- a/Repo-docs-DEPENDENCY-AUDITS.-.md +++ /dev/null @@ -1,74 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/docs/DEPENDENCY_AUDITS.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# Dependency Audits - -GovOPlaN keeps dependency vulnerability checks reproducible but separate from -the fast local smoke suite, because both Python and npm audits need network -metadata and can fail for newly disclosed advisories without a source change. - -## Local Workflow - -Install the development audit dependency once: - -```bash -cd /mnt/DATA/git/govoplan-core -./.venv/bin/python -m pip install -r requirements-dev.txt -``` - -Run both backend and WebUI production audits: - -```bash -cd /mnt/DATA/git/govoplan-core -bash scripts/check-dependency-audits.sh -``` - -The script runs: - -- `scripts/check-dependency-hygiene.sh` for pip resolver consistency, stale - legacy editable package metadata, deprecated framework constants, and the - Starlette `TestClient` deprecation smoke when test dependencies are present -- `python -m pip_audit --progress-spinner off` -- `npm audit --omit=dev` in `webui` - -For fast local checks without vulnerability metadata lookups, run: - -```bash -cd /mnt/DATA/git/govoplan-core -CHECK_TESTCLIENT_DEPRECATIONS=1 bash scripts/check-dependency-hygiene.sh -``` - -This is also part of `scripts/check-focused.sh`, so resolver drift and -deprecation regressions fail close to the code change that introduced them. - -Override tool paths when testing from a disposable environment: - -```bash -PYTHON=/tmp/govoplan-audit/bin/python \ -NPM=/home/zemion/.nvm/versions/node/v22.22.3/bin/npm \ -bash scripts/check-dependency-audits.sh -``` - -## CI Workflow - -`.gitea/workflows/dependency-audit.yml` installs release dependencies from -tagged package refs, installs `pip-audit`, and runs the same script on pushes, -pull requests, and a weekly schedule. - -The workflow intentionally uses release dependency refs instead of local -`file:` or editable sibling paths. Development lockfiles may keep local module -links, but release audit results should represent the installable product. - -## Recording Results - -When closing or triaging dependency-audit issues, add a short dated note under -`docs/audits/`. Record: - -- the commands that were run -- whether Python and npm passed -- any advisories accepted as temporary risk -- follow-up issue links for required upgrades diff --git a/Repo-docs-DEPLOYMENT-OPERATOR-GUIDE.md b/Repo-docs-DEPLOYMENT-OPERATOR-GUIDE.md index 307b31e..d737f0a 100644 --- a/Repo-docs-DEPLOYMENT-OPERATOR-GUIDE.md +++ b/Repo-docs-DEPLOYMENT-OPERATOR-GUIDE.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/DEPLOYMENT_OPERATOR_GUIDE.md`. > Origin: `repository`. @@ -43,7 +43,7 @@ set +a | Setting | Required outside dev | Purpose | | --- | --- | --- | | `APP_ENV` | yes | Runtime profile. Use `prod`, `staging`, or a deployment-specific value outside local development. | -| `MASTER_KEY_B64` | yes | Fernet key or base64 encoded 32-byte key used for encrypted module secrets. Rotate through an explicit operator plan. | +| `MASTER_KEY_B64` | yes | Fernet key or base64 encoded 32-byte deployment root used for encrypted module secrets and, when enabled, the Encryption module's local server-envelope provider. Rotate only through an explicit provider-aware migration plan. | | `DATABASE_URL` | yes | SQLAlchemy database URL for core and installed modules. SQLite is supported for dev/small installs; PostgreSQL is the preferred production target. | | `ENABLED_MODULES` | yes | Comma-separated startup module set. Keep `tenancy,access` enabled; keep `admin` enabled for operator UI. | @@ -64,6 +64,8 @@ PY | `GOVOPLAN_MIGRATION_TRACK` | `release` | Use the release track for normal runtime and deployments. Use `dev` only for fresh/disposable databases that intentionally replay detailed development migrations. | | `DEV_AUTO_MIGRATE_ENABLED` | `true` | Dev convenience only. Production should run migration commands explicitly during deployment. | | `DEV_BOOTSTRAP_ENABLED` | `false` | Dev bootstrap only. `govoplan_core.devserver` and `govoplan/tools/launch/launch-dev.sh` default it to `true`; use controlled first-admin creation outside dev. | +| `FIRST_ADMIN_ENROLLMENT_TTL_SECONDS` | `1800` | Lifetime of a locally issued production enrollment credential. Allowed range: 60 seconds to 24 hours. | +| `FIRST_ADMIN_ENROLLMENT_FILE` | `/run/govoplan/first-admin-enrollment.json` | Local operator artifact. The command creates it with mode `0600` and never prints the secret. | Operator rule: take a database backup before applying migrations or destructive module retirement. For non-SQLite databases, configure deployment-specific @@ -165,6 +167,7 @@ release evidence. | `REDIS_URL` | `redis://redis:6379/0` | Celery broker/result backend when async workers are enabled. | | `CELERY_ENABLED` | `false` | Local/dev can send synchronously. Production campaign delivery should run workers and set this to `true`. | | `CELERY_QUEUES` | `send_email,append_sent,notifications,mail,calendar,dataflow,workflow,postbox,events,idm,default` | Queue list expected by worker/process manager definitions. Keep this aligned with every enabled module task route; Ops reports missing worker queue consumers. | +| `CELERY_VISIBILITY_TIMEOUT_SECONDS` | `3600` | Maximum time before Redis may redeliver work left unacknowledged by a lost worker. Set this above the longest supported task duration; changing it requires a worker-loss acceptance drill. | | `PLATFORM_EVENT_OUTBOX_MAX_ATTEMPTS` | `8` | Failed durable consumer deliveries are quarantined after this many attempts. | | `PLATFORM_EVENT_OUTBOX_TERMINAL_RETENTION_DAYS` | `90` | Successful event envelopes older than this are removed by the daily retention task. Quarantined evidence is retained. | @@ -184,6 +187,13 @@ crashes, and expired worker leases: python -m celery -A govoplan_core.celery_app:celery beat --loglevel INFO ``` +Before promoting a worker composition, run the repository worker-runtime drill +against the same Redis and Core build. It uses the bounded +`govoplan.worker.acceptance` task and records publish/consume, retry, warm +SIGTERM, and worker-loss redelivery evidence without accessing tenant data. +Production evidence must use the deployed queue configuration and a visibility +timeout that is longer than every supported business task. + ### Storage | Setting | Default | Notes | @@ -299,8 +309,34 @@ configuration, not the core runtime contract. Store them in a local ignored 3. Build the WebUI from `webui/package.release.json` or deploy a prebuilt artifact from the same release tag. 4. Run database migrations with the target `DATABASE_URL`. -5. Create the first tenant and system owner through the controlled bootstrap or - one-time admin command for the deployment. +5. Create the first tenant and system owner through the controlled bootstrap: + +```bash +python -m govoplan_core.commands.first_admin status +python -m govoplan_core.commands.first_admin issue \ + --reason "initial production installation" +``` + + The issue command fails when an active system administrator already exists, + writes the random credential only to `FIRST_ADMIN_ENROLLMENT_FILE`, and does + not print it. Check `GET /api/v1/bootstrap/status`, then submit the account + and initial tenant fields to `POST /api/v1/bootstrap/first-admin` with the + secret in `X-GovOPlaN-Enrollment-Token`. The operation creates the protected + system owner and initial tenant-owner membership in one transaction and + retires the credential. A repeated identical request returns the same result + without creating another owner. + + If the artifact is lost or expires before use, a local operator may rotate + it only while no durable system administrator exists: + +```bash +python -m govoplan_core.commands.first_admin recover \ + --reason "expired installation handoff" +``` + + Issue and recovery write hash-chained Core evidence and an audit event. They + never enable or reuse `DEV_BOOTSTRAP_ENABLED`, `DEV_BOOTSTRAP_PASSWORD`, or + `DEV_BOOTSTRAP_API_KEY`. 6. Start the API service with `govoplan_core.server.app:app`. 7. Start workers when `CELERY_ENABLED=true`. 8. Start the WebUI/reverse proxy and verify CORS/cookie settings. @@ -430,6 +466,14 @@ SQLite's backup API; non-SQLite databases require `--database-backup-command`, `--database-restore-check-command`, and `--database-restore-command`. +Every non-dry run also owns the database-fenced +`core:module-lifecycle:deployment` recovery operation. The run record includes +its operation id and status. A supervised run reaches durable `succeeded` only +after restart and health verification. `recovery_required` or `outcome_unknown` +blocks another lifecycle mutation until the recorded operation is reconciled; +do not bypass this by deleting `install.lock`. See +[`MODULE_LIFECYCLE_RECOVERY.md`](MODULE_LIFECYCLE_RECOVERY.md). + Database hook commands receive: - `GOVOPLAN_INSTALLER_RUN_DIR` @@ -492,7 +536,7 @@ checks, catalog trust, signing, keyring, replay, and license operation. ## Operator Checklist - Runtime secrets are injected outside git. -- `MASTER_KEY_B64` is set and backed up securely. +- `MASTER_KEY_B64` is set and backed up securely; restores of locally encrypted content fail closed without the exact matching key. - Database backup and restore commands are tested. - File/object storage is durable and backed up. - `CORS_ORIGINS` and cookie settings match the deployed WebUI origin. diff --git a/Repo-docs-DOCUMENTATION-MAP.-.md b/Repo-docs-DOCUMENTATION-MAP.-.md deleted file mode 100644 index 12ba4c6..0000000 --- a/Repo-docs-DOCUMENTATION-MAP.-.md +++ /dev/null @@ -1,55 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/docs/DOCUMENTATION_MAP.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# GovOPlaN Documentation Map - -This map defines the source-of-truth documents for the current repository docs. -Use it to avoid duplicating long procedures across architecture, release, -operator, and roadmap pages. - -## Core Platform - -| Topic | Canonical document | Notes | -| --- | --- | --- | -| Module architecture and kernel contracts | `MODULE_ARCHITECTURE.md` | Stable module contracts, API efficiency contracts, durable boundary decisions, lifecycle, and WebUI contribution rules. | -| RBAC and resource access | `ACCESS_RBAC_MODEL.md` | Current permission, role, API-key, and resource-access model. | -| Governance hierarchy | `GOVERNANCE_MODEL.md` | System, tenant, user/group, campaign policy inheritance and admin UI structure. | -| Policy decision DTOs and provenance | `POLICY_CONTRACTS.md` | Shared explain/provenance shape; module-specific policy docs should link here. | -| Events and audit trace context | `EVENTS_AND_AUDIT.md` | Event dispatch semantics and audit payload conventions. | - -## Release And Operations - -| Topic | Canonical document | Notes | -| --- | --- | --- | -| Runtime configuration and operator flow | `DEPLOYMENT_OPERATOR_GUIDE.md` | Production/staging configuration, migrations, backups, installer operation, and rollback drill. | -| Release dependencies and catalogs | `RELEASE_DEPENDENCIES.md` | Release package refs, migration baselines, release lockfiles, catalog trust/licensing, catalog publishing, and release checklist. | -| Dependency vulnerability audits | `DEPENDENCY_AUDITS.md` | Local and CI audit commands plus dated audit result notes. | -| Remote WebUI bundle design | `REMOTE_WEBUI_BUNDLES.md` | Experimental controlled-deployment design; normal releases still use package builds. | - -## Product And Module Planning - -| Topic | Canonical document | Notes | -| --- | --- | --- | -| Product roadmap and module routing | `GOVOPLAN_MASTER_ROADMAP.md` | Product-level sequencing, implementation gates, issue routing, and missing-module decisions. | -| UI/UX decisions | `UI_UX_DECISION_LEDGER.md` | Binding guided-UI decisions, open decisions, impact index, and review checklist. | -| Public-sector integration posture | `PUBLIC_SECTOR_INTEGRATION_STRATEGY.md` | Strategy index; executable target inventory lives in `govoplan-connectors`. | -| Configuration packages | `CONFIGURATION_PACKAGES.md` | Package model, provider contract, import/export flow, and tracking slices. | - -## Workflow Docs - -| Topic | Canonical document | Notes | -| --- | --- | --- | -| Gitea issues and wiki sync | `GITEA_ISSUES.md` | Issue labels, imports, wiki mirroring, and Codex state updates. | -| Codex local workflow | `CODEX_WORKFLOW.md` | Local agent setup and focused verification commands. | - -## Cross-Repo Rule - -Core docs may keep strategy, kernel contracts, and routing decisions. Module -repositories should own executable module behavior, concrete API/UI contracts, -and operator notes for their own module. When content spans both, keep the -durable decision in core and link to the module document for implementation -details. diff --git a/Repo-docs-DOCUMENTATION-MAP.md b/Repo-docs-DOCUMENTATION-MAP.md index 0006df0..e91235f 100644 --- a/Repo-docs-DOCUMENTATION-MAP.md +++ b/Repo-docs-DOCUMENTATION-MAP.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/DOCUMENTATION_MAP.md`. > Origin: `repository`. @@ -24,8 +24,13 @@ operator, and roadmap pages. | Action/effect automation layer | `ACTION_EFFECT_AUTOMATION_LAYER.md` | Action/effect contracts, consequence preview, runner semantics, and module boundary for automation. | | External references and integration maturity | `EXTERNAL_REFERENCES_AND_INTEGRATION_MATURITY.md` | Stable external identity and cumulative connector maturity; configured source authority is defined by the meta target architecture. | | Institutional context and governed references | `INSTITUTIONAL_CONTEXT_CONTRACT.md` | Shared temporal, actor/representation, institution, mandate, service, party, decision, evidence, legal-basis, information-governance, presentation, and geo DTO/provider contracts. | +| Temporal data read context | `TEMPORAL_DATA_CONTEXT.md` | Valid-time and recorded-time titlebar selection, HTTP/cache contract, security boundary, and module-adoption rule. | +| Cross-module information governance adoption | `INFORMATION_GOVERNANCE_ADOPTION.md` | Manifest evidence and enforcement rules for temporal browsing, purpose-aware access, retention, and institutional context. | +| Context-sensitive F1 help | `CONTEXTUAL_HELP_CONTRACT.md` | Focus, route, module-manifest documentation contexts, Docs projection, and hosted fallback. | +| German localization and help quality gate | `LOCALIZATION_AND_HELP_QUALITY.md` | German reference locale, new-installation default, catalog completeness, automatic page associations, and explicit-help review priorities. | | Postbox E2EE target architecture | `POSTBOX_E2EE_ARCHITECTURE.md` | Strategic encrypted postbox/mailbox model, key ownership, role mailbox semantics, and retraction limits. | | Shared state, runtime coordination, and recovery | `STATE_AND_RECOVERY_CONTRACT.md` | State profiles, object storage, node registration/drain, fenced leases, migration ordering, and recovery evidence. | +| Module lifecycle recovery | `MODULE_LIFECYCLE_RECOVERY.md` | Installer/live-graph recovery modes, deployment fence, evidence, retry blocking, and operator reconciliation. | ## Release And Operations @@ -43,8 +48,11 @@ operator, and roadmap pages. | Topic | Canonical document | Notes | | --- | --- | --- | | Product roadmap and module routing | `GOVOPLAN_MASTER_ROADMAP.md` | Product-level sequencing, implementation gates, issue routing, and missing-module decisions. | +| Stable platform ideas | `govoplan/docs/PLATFORM_CORE_IDEAS.md` | Cross-product thesis, canonical distinctions, product experience rule, maturity rule, and decision test. | +| Current cross-product reconciliation | `govoplan/docs/STRATEGY_STATUS.md` | The only current prose status source; generated evidence and Gitea remain authoritative inputs. | | Institutional governance target | `govoplan/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md` | Cross-product semantic layers, source-authority modes, candidate Mandates/Services/Parties/Decisions boundaries, and migration sequence. | | UI/UX decisions | `UI_UX_DECISION_LEDGER.md` | Binding guided-UI decisions, open decisions, impact index, and review checklist. | +| Core interface migration | `INTERFACE_PATTERN_MIGRATION.md` | Core-owned settings, credential, retention, lifecycle, and shared-component evidence for the product pattern language. | | Interface ethics and design doctrine | `INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` | Product-level doctrine for context, decision, consequence, contestability, responsibility, and traceability. | | Public-sector integration posture | `PUBLIC_SECTOR_INTEGRATION_STRATEGY.md` | Strategy index; executable target inventory lives in `govoplan-connectors`. | | Configuration packages | `CONFIGURATION_PACKAGES.md` | Package model, provider contract, import/export flow, and tracking slices. | diff --git a/Repo-docs-DURABLE-RECOVERY-OPERATIONS.md b/Repo-docs-DURABLE-RECOVERY-OPERATIONS.md new file mode 100644 index 0000000..fe6cf21 --- /dev/null +++ b/Repo-docs-DURABLE-RECOVERY-OPERATIONS.md @@ -0,0 +1,51 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/DURABLE_RECOVERY_OPERATIONS.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Durable Recovery Operations + +Modules must use `begin_durable_recovery_operation` for work whose effects can +outlive the caller's SQLAlchemy transaction. The helper commits the canonical +request hash, recovery plan, precondition evidence, running state, and lease +fence before the caller mutates object storage, a queue, a filesystem, or an +external provider. + +Each later checkpoint is written through an independent database session. A +business-transaction rollback therefore cannot erase evidence of an earlier +effect. Successful completion requires concrete verification checks and a valid +hash chain. Compensation likewise records recovery-required, recovering, and +verified-recovered checkpoints rather than reporting an ordinary failure. +A definitive pre-effect or provider rejection records terminal `rejected` +evidence instead of being mislabeled as success, atomic rollback, or recovery +work. + +If a runtime disappears, another runtime may claim the operation only after the +lease expires. The takeover records both fences. A stale compensatable operation +becomes recovery-required; a stale forward-only or irreversible external effect +becomes outcome-unknown; a database-only atomic operation is recorded failed +because its transaction rolled back. Takeover never re-executes the original +request automatically. + +Evidence and metadata may contain opaque references, digests, counts, and +provider result codes. They must never contain credentials or resolved secrets. +Ops is the platform surface for unresolved operation status; owning modules must +provide the reconciliation action and business-level explanation. + +Database-only operations must use the durable handle's atomic terminal methods +when their module rows and final recovery checkpoint belong to one invariant. +Those methods stage the terminal checkpoint and lease release in the caller's +SQLAlchemy transaction, then commit the domain rows and recovery evidence +together. A failed commit rolls both back and leaves the previously durable +`running` record available for stale-fence handling; modules must not commit +their domain state first and close an `atomic` recovery record afterwards. + +An owning module may reconcile an `outcome_unknown` provider effect through the +claimed durable handle's `resolve_unknown` method. External evidence that the +effect occurred records verified success. Evidence that it did not occur moves +the operation through recovery-required and recovering to verified recovered, +so any later attempt must use a new deliberate idempotency key. The method does +not infer provider state and requires the same terminal verification structure +and hash-chain checks as ordinary completion. diff --git a/Repo-docs-INFORMATION-GOVERNANCE-ADOPTION.md b/Repo-docs-INFORMATION-GOVERNANCE-ADOPTION.md new file mode 100644 index 0000000..3ad8310 --- /dev/null +++ b/Repo-docs-INFORMATION-GOVERNANCE-ADOPTION.md @@ -0,0 +1,138 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/INFORMATION_GOVERNANCE_ADOPTION.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Information Governance Adoption + +## Platform Rule + +Temporal browsing, purpose-aware access, retention, and institutional context +are platform-wide information-governance dimensions. Every module receives the +same contract by default. A module may claim `partial` or `enforced` only with +repository-owned object scope, evidence, and limitations; it may claim +`not_applicable` only when the dimension genuinely does not apply. + +Historical business data is always authorized under the current security +state. No module may use a historical permission, membership, role, function +assignment, or policy projection to weaken present-day access. + +Platform-wide adoption is tracked in +[GovOPlaN #40](https://git.add-ideas.de/GovOPlaN/govoplan/issues/40), with +temporal reads detailed in +[GovOPlaN #39](https://git.add-ideas.de/GovOPlaN/govoplan/issues/39). + +## Manifest Declaration + +`ModuleManifest.information_governance` publishes four dimensions: + +- `temporal_browsing`; +- `purpose_aware_access`; +- `retention`; +- `institutional_context`. + +Each dimension declares: + +- adoption: `not_applicable`, `contract_only`, `partial`, or `enforced`; +- object types covered; +- repository-local test/documentation evidence; +- the remaining limitation for `contract_only` or `partial`. + +The default is intentionally `contract_only`. It applies the platform rule +without pretending that existing domain queries and effects already enforce +it. `reference_ready`, `supported`, and `lts` modules cannot retain an +applicable dimension below `enforced`. + +## Read Contract + +For every persistent domain object, the owner classifies the read: + +1. **Current-only:** historical semantics do not exist and the API says so. +2. **Valid-time:** select facts effective now or at the requested instant. +3. **Bitemporal:** additionally select only revisions known by `recorded_at`. +4. **All-validity:** return effective revisions in a bounded history view. + +The Core temporal middleware supplies the request context. Owners apply it in +repositories or query helpers, include it in cache keys, return evaluated +context, and test current/at/all plus recorded-time boundaries. Search, +reporting, exports, selectors, counts, and drill-through must use the same +projection as the owning list/detail API. + +## Purpose-Aware Access Contract + +Permission establishes a technical action ceiling. Purpose-aware access asks +whether this actor, represented capacity, case/work item, legal basis, and +declared use may access this object now. + +- A client-supplied purpose is an assertion, never authority by itself. +- The owner or Policy capability validates the purpose and returns explainable + provenance. +- Sensitive access can require case assignment, mandate, reason capture, + approval, or break-glass evidence. +- Search, selectors, reporting, exports, background jobs, and connectors apply + the same decision. +- Audit records the validated purpose identifier and decision reference, not + unnecessary content. + +## Retention Contract + +Every persistent object declares an owner, retention class or policy reference, +trigger, start instant, hold behavior, review/disposition action, and evidence. +Retention is not a generic timestamp deletion job. + +- Domain owners enumerate and execute their own effects through a typed + retention provider. +- Policy resolves inherited ceilings and simulation. +- Records owns record disposition; Files owns byte/object effects; Audit owns + audit-detail behavior; external providers declare their own effect and + recovery semantics. +- Dry-run, legal hold, exact revision, idempotency, outcome unknown, + reconciliation, correction, and destruction evidence are mandatory for + consequential removal. + +## Institutional Context Contract + +Consequential objects and effects carry the relevant tenant, institution, +organization unit, function, mandate/jurisdiction, service/case/work item, +party/representation, decision, and record references. Context is minimized to +what the operation needs. Organizational membership is not itself permission +or mandate. + +Events, automation intents, audit evidence, records, and external effects retain +the same governed context envelope or an exact reference to it. Consumers must +not reconstruct authority later from mutable current structures. + +## Adoption Order + +1. Inventory every domain list/detail/search/export/effect and classify all + four dimensions. +2. Migrate institutional owners first: Access, IDM, Organizations, Mandates, + Services, Parties, Cases, Approvals, Committee, Decisions, Voting, and + Records. +3. Migrate communication and content: Addresses, Distribution Lists, Campaign, + Postbox, Mail, Calendar, Files, Templates, and Forms Runtime. +4. Migrate data projections: Connectors, Datasources, Dataflow, Reporting, + Search, Risk Compliance, and Dashboard. +5. Migrate workflow/task/background/provider operations and prove that no + asynchronous path drops context. +6. Advance manifest claims only after owner tests and browser/reference-journey + evidence pass. + +The generated platform inventory reports adoption counts and module details. +Gitea tracks individual migrations; the declaration is evidence and a maturity +gate, not a substitute for implementation. + +## Definition Of Enforced + +A dimension is `enforced` only when: + +- all declared object types and public reads/effects use it; +- list/detail/count/search/export/worker behavior is consistent; +- cache and pagination semantics cannot cross contexts; +- absence, invalid values, and inaccessible referenced context fail safely; +- tests cover current, historical, unauthorized, replay, and module-absence + combinations appropriate to the dimension; +- user/admin documentation explains behavior and limitations; +- the manifest cites those tests and docs. diff --git a/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.-.md b/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.-.md deleted file mode 100644 index e458228..0000000 --- a/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.-.md +++ /dev/null @@ -1,166 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/docs/INSTITUTIONAL_CONTEXT_CONTRACT.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# Institutional Context And Governed References - -GovOPlaN consequential work must retain enough context to answer who acted, -for whom, through which function, under which mandate and jurisdiction, using -which rule and evidence versions, and with which requested and observed effect. -The shared contract lives in `govoplan_core.core.institutional`. - -Core owns reference shapes and provider protocols only. It does not own shared -Mandate, Service, Party, Decision, evidence, or geography tables. Domain modules -own persistence and authorization; optional capabilities resolve the references. - -## Envelope - -`GovernedContextEnvelope` version 1 carries: - -- a tenant and `TemporalRevision` with validity, recording, supersession, and - change reason; -- the real account or service account and represented account, function, - procedure party, assignment, delegation/power, and mandate; -- institution, organization unit, function, task, mandate, jurisdiction, - service, case, party, work item, workflow, approval, decision, and record - references; -- versioned legal bases and evidence references; -- information classification, purposes, retention/holds, minimization, and - disclosure state; -- external-source authority, maturity, freshness, health, and conflict state; -- language, accessibility, channel, explanation, and availability references. - -Every institutional reference includes the owner module, tenant, stable object -identity, optional version/effective instant, and a protected display label. -Cross-tenant references are rejected. Safe serialization omits labels, -inspection URLs, formal reasoning, operative results, and conditions unless a -caller explicitly requests the protected projection. - -## Semantic Providers - -The first provider-neutral capabilities are: - -- `mandates.resolver`: resolve competence for a task/authority type at an - effective instant and return the governing Mandate definition and evidence; -- `services.definitions`: obtain versioned institutional service definitions; -- `parties.resolver`: obtain effective procedure-local parties and powers of - representation without copying Identity or Organizations subjects; and -- `decisions.registry`: record and retrieve formal Decisions under optimistic - revision control. - -The Mandate, Service, Party/representation, and Decision DTOs have strict -mapping round-trips so they can cross capability, event, package, and storage -boundaries without shared ORM models. Their lifecycle states are explicit: -Mandates distinguish draft/active/suspended/replaced/retired, Services retain -publication state, Parties retain effective representation and revocation, and -Decisions retain correction, revocation, and supersession references. - -The DTOs are a repository threshold, not a mandate to create four modules. -Independent persistence, lifecycle, security/operations behavior, release -reason, reuse, and tests are still required before extraction. - -Mandate resolution is deterministic: Core filters candidates by tenant, -effective interval, active state, task and authority type, stable -organization/function identity, jurisdiction coverage, and subject type. A -result is competent only when exactly one matching Mandate remains and it has -no unresolved conflicts. Evidence from matching definitions is deduplicated -and retained in the explanation result. `revise_mandate_definition` applies -optimistic concurrency and the allowed activation, suspension, replacement, -and retirement transitions while leaving the previous revision immutable. - -`revise_formal_decision` provides the equivalent lifecycle primitive for -formal outcomes. Every accepted transition requires a new recorded revision -and change reason, links `supersedes_ref` to the prior version, updates the -authority envelope to the new version, and records explicit correction or -revocation provenance. Terminal and backward transitions fail closed. Each -Decision also records whether responsibility was human, human-reviewed -automation, or an automated service account acting under mandate. Automation -preparation/recommendation references remain inspectable without being -mistaken for the responsible outcome. - -Procedure-party corrections use `revise_procedure_party`: the stable party -identity is retained, a new revision and reason are required, stale writes are -rejected, and revoked/expired/superseded assignments are terminal. -`revoke_party_representation` separately records when a limited power ceased -to authorize actions. This allows consuming procedures to evaluate historical -delivery or representation authority without rewriting Identity, -Organizations, or Addresses records. - -Service templates and package/tenant specializations use -`derive_service_restriction`. The derived definition retains an explicit -parent-version reference, cannot extend the parent's validity, audience, -channels, or publication ceiling, and cannot remove inherited prerequisites, -required evidence, legal bases, or bindings. This is the fail-closed semantic -rule; configuration-package signature and provenance checks remain the package -transport rule. - -`ServiceAvailabilityRequirement` represents module, capability, mandate, -policy, connector, maintenance, audience, and configuration prerequisites with -an explicit unavailable-or-hidden failure mode and explanation reference. The -optional `services.availability` evaluator returns policy-scoped boolean -assessments, reason codes, and evidence. Unknown consequential requirements -fail closed; a reference itself never grants access. - -## Service Launch - -`ServiceLaunchRequest` and `ServiceLaunchResult` define the owner-neutral -boundary between Portal entry and a case, form, or workflow runtime effect. -The request carries the exact published Service definition, exact selected -binding, tenant, acting identity, timezone-aware request time, bounded -parameters, and idempotency key. The result must retain that exact Service and -binding, a same-tenant target reference, optional same-tenant evidence, and -only a relative or credential-free HTTP(S) destination. - -`service_launch_capability(kind)` maps bindings to owner capabilities: - -- `case` -> `cases.service_launcher` -- `form` -> `forms_runtime.service_launcher` -- `workflow` -> `workflow_engine.service_launcher` - -`FormDefinition`, `FormFieldDefinition`, and `forms.definitions` provide the -owner-neutral exact-schema boundary used by Forms Runtime. Definitions carry an -exact tenant/revision, field types/options/constraints/defaults, publication, -draft, attachment, signature, policy, and handoff requirements. Forms owns -those immutable definitions; Forms Runtime persists instances and validation -evidence. A form Service binding uses `/` and the launcher -rejects missing, superseded, unpublished, cross-tenant, or invalid definitions. - -Portal may discover and invoke those capabilities but cannot write owner -tables. The owner must revalidate its definition/binding and current -authorization, produce its normal audit/event state, and make replay after an -ambiguous response safe. If the capability is absent, the service is -explainably unavailable. URL-only entries pass through the same launch-time -availability check and destination validation. Forms Runtime now supplies the -definition-aware form launcher when both Forms and Forms Runtime are active; -otherwise Portal continues to fail closed. - -## Propagation - -`PlatformEvent`, `ActionExecutionRequest`, and `AuditEvent` can carry the -envelope. Audit persistence stores only its safe projection; platform-event -outbox serialization preserves it across asynchronous delivery. A module must -not invent a parallel context dictionary when the shared fields apply. - -## First Proof - -Committee's `committee.decision_path` capability is the first bounded proof. It -requires one effective, conflict-free Mandate covering the organization unit -function, and jurisdiction, an approval reference, fact evidence, versioned legal bases, -operative result, and reasoning. It emits a reconstructable `FormalDecision`, -including requested/observed effects and information governance. If a Decision -registry is installed it persists there; Committee does not take ownership of -the generic Decision lifecycle. - -## Compatibility And Security - -- Contract version changes follow Core compatibility policy. -- Unknown tenant or reference-kind combinations fail closed. -- Datetimes that affect authority must be timezone-aware. -- Protected labels, reasoning, evidence inspection links, and source details - remain subject to the owning module's access policy. -- References do not grant access to their targets. -- Evidence and audit payloads must contain stable references/checksums, not - plaintext secrets. diff --git a/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md b/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md index a6eea2e..9e2fa1b 100644 --- a/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md +++ b/Repo-docs-INSTITUTIONAL-CONTEXT-CONTRACT.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/INSTITUTIONAL_CONTEXT_CONTRACT.md`. > Origin: `repository`. @@ -15,6 +15,9 @@ The shared contract lives in `govoplan_core.core.institutional`. Core owns reference shapes and provider protocols only. It does not own shared Mandate, Service, Party, Decision, evidence, or geography tables. Domain modules own persistence and authorization; optional capabilities resolve the references. +Interactive reads use the separate platform temporal-data context documented in +`TEMPORAL_DATA_CONTEXT.md`; it never changes current authorization or supplies +mutation dates. ## Envelope @@ -120,14 +123,22 @@ only a relative or credential-free HTTP(S) destination. - `form` -> `forms_runtime.service_launcher` - `workflow` -> `workflow_engine.service_launcher` +`FormDefinition`, `FormFieldDefinition`, and `forms.definitions` provide the +owner-neutral exact-schema boundary used by Forms Runtime. Definitions carry an +exact tenant/revision, field types/options/constraints/defaults, publication, +draft, attachment, signature, policy, and handoff requirements. Forms owns +those immutable definitions; Forms Runtime persists instances and validation +evidence. A form Service binding uses `/` and the launcher +rejects missing, superseded, unpublished, cross-tenant, or invalid definitions. + Portal may discover and invoke those capabilities but cannot write owner tables. The owner must revalidate its definition/binding and current authorization, produce its normal audit/event state, and make replay after an ambiguous response safe. If the capability is absent, the service is explainably unavailable. URL-only entries pass through the same launch-time -availability check and destination validation. A missing form runtime must -therefore fail closed rather than create a submission without definition-aware -validation. +availability check and destination validation. Forms Runtime now supplies the +definition-aware form launcher when both Forms and Forms Runtime are active; +otherwise Portal continues to fail closed. ## Propagation diff --git a/Repo-docs-INTERFACE-PATTERN-MIGRATION.md b/Repo-docs-INTERFACE-PATTERN-MIGRATION.md new file mode 100644 index 0000000..960f0b2 --- /dev/null +++ b/Repo-docs-INTERFACE-PATTERN-MIGRATION.md @@ -0,0 +1,35 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/INTERFACE_PATTERN_MIGRATION.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Core Interface Pattern Migration + +This document records the Core-owned part of the product-wide interface +pattern-language rollout. The normative product grammar and complete route +inventory live in the `govoplan` meta repository. Core owns reusable behavior; +domain modules own their compositions. + +## Core Surfaces + +| Surface | Pattern | Consequence and provenance contract | Evidence | +| --- | --- | --- | --- | +| User settings | Two-zone settings workspace with typed controls and unsaved-change protection | Save actions distinguish busy, unchanged, and test-in-progress states; contextual help resolves through Docs or the hosted fallback | `SettingsPage.tsx`, `test-core-interface-patterns.mjs` | +| Reusable credentials | Repeated administration with an adaptive create/edit dialog, optional password generator, and destructive confirmation | Secret values are write-only; generated candidates use the browser cryptographic API without a weak fallback and do not replace the field until explicitly confirmed; scope/permission blockers name the required action, responsible actor, and destination; unavailable row actions remain keyboard-explainable | `CredentialEnvelopeManager.tsx`, shared `PasswordField`, `PasswordGeneratorDialog`, `ActionBlockerHint`, `Button`, `TableActionGroup`, and `ConfirmDialog` | +| Retention policy | Effective-policy editor with inherited source paths and typed, narrowing-only controls | Parent locks and missing write authority are explicit; the save action distinguishes locks, missing target, loading, clean draft, and active save | `RetentionPolicyManagement.tsx`, policy logic tests, `test-core-interface-patterns.mjs` | +| Module lifecycle | Guided operator projection over durable installer-queue evidence | Preflight, handoff, progress, stale evidence, recovery, and rollback consequences remain visible | Admin module lifecycle tests and the Core installer-queue contract | +| Shared configuration primitives | Cross-module component contract | Dialog focus, blocker structure, disabled-action focus, route/page/field/action F1 help, unsaved changes, confirmation, loading, alerts, problem lists, and policy provenance are centralized | Core component tests, `CONTEXTUAL_HELP_CONTRACT.md`, and module-permutation build | + +## Boundary + +Files and Mail are the first two external consumers of the layered +server/credential/policy pattern. Their own repositories retain provider +discovery, transport behavior, authorization, and migration evidence. Remaining +module surfaces are tracked by bounded module-owned issues under GovOPlaN #11; +they are not reasons to add sibling-private behavior to Core. + +Raw JSON remains permitted only for diagnostics, expert inspection, +interchange, or conflict evidence. It is not a primary Core configuration +editor. diff --git a/Repo-docs-LOCALIZATION-AND-HELP-QUALITY.md b/Repo-docs-LOCALIZATION-AND-HELP-QUALITY.md new file mode 100644 index 0000000..6cd7b06 --- /dev/null +++ b/Repo-docs-LOCALIZATION-AND-HELP-QUALITY.md @@ -0,0 +1,67 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/LOCALIZATION_AND_HELP_QUALITY.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Localization And Contextual Help Quality + +## Reference Language + +German (`de`) is GovOPlaN's first-class reference target. Every translation key +used by a shipped WebUI must exist in German and English. German completeness is +a release gate; English remains the source-code fallback language so existing +literal labels and external developer APIs do not change semantics. + +New installations and tenants default to German. Existing system, tenant, and +user preferences are preserved. The available-language and policy model can +still select another default or disable a package at the relevant scope. + +Explicit high-risk help content and browser acceptance are tracked in +[Core #284](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/284). + +The platform inventory recognizes both inline locale objects and generated +catalogs declared as `const de` / `const en`. Its strict mode requires both +locales and reports `de` explicitly as the reference locale. + +## Help Resolution + +Every focusable field and action receives a stable derived F1 identity from the +shared shell, even when the component has no dedicated help text. Resolution +falls back from field/action to dialog or page and then to the module's visible +documentation baseline. + +Backend manifests publish explicit topic associations first. Core additionally +associates declared route, navigation, settings, and View surface IDs with the +module's static user or administrator documentation baseline. Feature modules +should still add exact `metadata.help_contexts` entries for consequential, +unfamiliar, policy-controlled, destructive, security-sensitive, or legally +meaningful fields and actions. + +The generated `help_review_candidates` list is therefore a content-depth queue, +not a list of controls on which F1 cannot work. It should prioritize: + +1. effect, deletion, delivery, retention, disclosure, encryption, and recovery; +2. identity, representation, mandate, institutional context, and purpose; +3. valid-time versus recorded-time selection; +4. provider authority, synchronization, conflict, and outcome unknown; +5. fields whose consequences are not evident from their label. + +## Verification + +```bash +cd /mnt/DATA/git/govoplan +/mnt/DATA/git/govoplan/.venv/bin/python \ + tools/inventory/platform-interface-inventory.py \ + --strict --strict-declarations --strict-endpoints +``` + +The check must report: + +- reference locale `de` present and complete; +- no used key missing from `de` or `en`; +- every field has a resolvable F1 context; +- no duplicate stable IDs; +- no undeclared public WebUI surface; +- no stale runtime route or endpoint declaration. diff --git a/Repo-docs-MODULE-ARCHITECTURE.-.md b/Repo-docs-MODULE-ARCHITECTURE.-.md deleted file mode 100644 index 5e4ca91..0000000 --- a/Repo-docs-MODULE-ARCHITECTURE.-.md +++ /dev/null @@ -1,1514 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# GovOPlaN Module Architecture - -GovOPlaN is structured as a platform kernel plus installable modules. The kernel starts and composes the platform. Modules own product behavior and contribute backend routes, database metadata, permissions, WebUI routes, navigation metadata, capabilities, and events. - -The current package name is still `govoplan-core`, but the architecture target is a smaller kernel. Access, tenancy, policy, audit, and admin semantics are platform-module responsibilities. - -Access extraction is complete enough that current ownership is described here, -in [`ACCESS_RBAC_MODEL.md`](ACCESS_RBAC_MODEL.md), and in the -`govoplan-access` repository docs. -The event and audit trace contract is tracked in -[`EVENTS_AND_AUDIT.md`](EVENTS_AND_AUDIT.md). -Policy decision, source provenance, and explain-response contracts are tracked -in [`POLICY_CONTRACTS.md`](POLICY_CONTRACTS.md). -The experimental remote WebUI bundle loading design is tracked in -[`REMOTE_WEBUI_BUNDLES.md`](REMOTE_WEBUI_BUNDLES.md). -The cross-product semantic layers, source-authority modes, and candidate -Mandates, Services, Parties, and Decisions boundaries are canonical in the -meta repository's -[`INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md`](../../govoplan/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md). - -## Layer Model - -| Layer | Purpose | Examples | -| --- | --- | --- | -| Kernel | Bootstraps and composes the platform | app factory, module registry, route aggregation, migration orchestration, capability/event contracts, health metadata | -| Platform modules | Cross-cutting governance capabilities | access, tenancy, policy, audit, admin, ops | -| Service modules | Reusable operational capabilities | files, mail, templates, recipients, notifications | -| Business modules | Public-sector workflows | campaigns, cases, forms, approvals, appointments | -| Connector modules | External system integration | FIT-Connect, XÖV/XTA, DMS/eAkte, ERP, IDM | - -This table is the technical composition model. The product portfolio uses a -more detailed institutional layer model, but it does not change dependency -direction: Core provides contracts and composition; modules own semantics; -packages compose modules. - -## Institutional Semantic Boundaries - -Cross-module references must keep these answers distinct: - -- Organizations owns where structures, units, and functions exist. -- Identity owns who a subject is; Access owns accounts, roles, permissions, - and authorization decisions; IDM owns effective function assignments. -- A Mandates capability will answer why a unit or function is competent for a - task, jurisdiction, subject, or period. It must not become another RBAC - system. -- A Services capability will own versioned institutional service definitions; - Portal presents and starts them. -- A Parties capability will own procedure-local participant roles, - representation, and delivery authority; it must reference rather than copy - Identity, Organizations, and Addresses subjects. -- A Decisions capability will own formal institutional outcomes and their - authority, facts, rules, reasoning, effects, correction, and review. - Approvals owns review gates, Committee owns deliberation/votes, and Workflow - Engine owns coordination. - -Start each missing concept as a versioned DTO/provider contract used by a -bounded journey. A repository is justified only when the concept gains -independent persistence, lifecycle, security/operations behavior, release -reason, and reuse. Core must not store these domain objects. - -## Kernel Responsibilities - -The kernel target owns: - -- the server entry point and platform configuration -- module discovery, manifest validation, registry validation, route aggregation, and platform metadata APIs -- database engine/session lifecycle and module migration orchestration -- module install-plan validation, signed catalog verification, license entitlement checks, and installer request orchestration -- capability registry, command/event contracts, and lifecycle hooks -- shared WebUI shell contracts, generic WebUI components, and module route/nav rendering -- centralized mapping from serializable icon names to renderable frontend icons -- health, OpenAPI aggregation, and runtime diagnostics - -The kernel must not own product semantics such as users, tenants, RBAC decisions, governance policies, audit storage, mail behavior, file behavior, or campaign behavior. Those belong to platform, service, or business modules. - -## Current Compatibility Responsibilities - -During the staged split, `govoplan-core` still contains compatibility surfaces -for tenancy settings, governance/policy contracts, audit helpers, CSRF/API -helpers, and secret helpers. The extracted access implementation lives in -`govoplan-access`; live ORM table definitions have been split across their -platform owners using module-prefixed table names. The old core route, -admin-service, and access-security re-export modules have been removed. -Callers must use module-owned imports, the public `govoplan_core.auth` request -dependency facade, or kernel capabilities. -The remaining platform compatibility surfaces are temporary until the matching -platform modules are fully self-contained: - -- `govoplan-access` -- `govoplan-tenancy` -- `govoplan-policy` -- `govoplan-audit` -- `govoplan-admin` - -New code should avoid deepening these compatibility dependencies. Prefer explicit kernel contracts and module capabilities over direct imports. - -Core must not import module feature pages or module business logic directly. It should interact with modules through manifests, entry points, metadata, capabilities, events, and route contributions. - -The compatibility/deprecation plan for the current split line is: - -- keep documented public compatibility imports until the owning module exposes a - stable replacement and all in-tree callers have migrated -- remove deep implementation re-export modules once callers can use module-owned - public APIs or kernel capabilities -- preserve migration/table compatibility for already-created development and - release databases -- document remaining compatibility surfaces here and in the owning module README -- reject new cross-module imports that bypass manifests, capabilities, events, - or public module APIs - -The retention windows and removal checklist for database bridges, -configuration/export schemas, and runtime/API aliases are defined in -`COMPATIBILITY_POLICY.md`. - -## Stable Kernel Contracts - -The following contracts are the baseline API that modules can rely on: - -- `ModuleManifest` -- `ModuleCompatibility` -- named interface contract provider/requirement metadata -- module uninstall guard provider contract -- `MigrationSpec` -- route factory contract -- capability factory contract -- access DTO/protocol contracts in `govoplan_core.core.access` -- resource ACL provider contract -- bounded reference-option search provider contract -- single-tenant and optional batched tenant summary provider contracts -- tenant delete-veto provider contract -- WebUI module contribution contract -- navigation metadata contract -- command/event envelope contract -- policy decision and source provenance contract in `govoplan_core.core.policy` -- external object reference and integration-maturity contract in - `govoplan_core.core.external_references` -- action/effect preview and execution contract in - `govoplan_core.core.automation` -- workflow definition contribution and runtime-worker contracts - -Changes to these contracts must be versioned or accompanied by compatibility shims. - -Tenant list pages prefer `tenant_summary_batch_providers`. A batch provider -receives the unique tenant IDs on the current page and returns count mappings -keyed by tenant ID. Missing tenant keys mean that the provider has no counts for -that tenant; provider errors remain visible. Modules that expose only the -single-tenant contract remain compatible through a per-tenant fallback. -Destructive tenant lifecycle planning deliberately continues to use the -single-tenant path so it invokes every registered provider for the target -tenant, independent of ordinary list-page projections. - -This list is the Milestone A kernel-contract freeze baseline. New module work -may extend the kernel by adding explicit contracts, but existing contracts must -remain source-compatible through the 0.1.x split line unless a migration shim -and deprecation note are provided. - -### Architecture Metadata - -`ModuleManifest.architecture` is the backward-compatible, versioned product- -portfolio declaration for: - -- module kind and institutional architecture layer; -- evidence-backed maturity claim (`concept`, `scaffold`, `vertical_slice`, - `reference_ready`, `supported`, or `lts`); -- owned and explicitly non-owned concepts; -- supported source-authority modes; -- reference packages, tested providers, and known limits; -- migration, upgrade, recovery, security, operations, and documentation - evidence references. - -Core validates the claim and all provider references during registry startup. -`reference_ready`, `supported`, and `lts` claims require a named reference -package and the cumulative evidence set; target-tested providers additionally -require provider evidence. A supported module with migrations must include -migration evidence. Signed release catalogs retain and revalidate the -declaration. The meta manifest check validates repository evidence paths and -provides an opt-in `--require-architecture` rollout gate. Platform metadata, -Docs, and Ops project the same declaration. A module cannot make itself -supported solely by changing its maturity string. - -Known access-related capability names are defined in -`govoplan_core.core.access`, including: - -- `access.principalResolver` -- `access.directory` -- `access.permissionEvaluator` -- `access.resourceAccess` -- `access.tenantProvisioner` -- `access.administration` -- `access.governanceMaterializer` -- `tenancy.tenantResolver` -- `security.secretProvider` -- `audit.sink` - -`govoplan-access` currently registers `auth.principalResolver`, -`auth.permissionEvaluator`, `access.principalResolver`, -`access.permissionEvaluator`, `access.directory`, `access.tenantProvisioner`, -`access.administration`, and `access.governanceMaterializer`. -`govoplan-tenancy` registers `tenancy.tenantResolver`. The minimal -authenticated platform set is now `access`; tenancy is optional and adds tenant -administration plus tenant resolver behavior when installed. -Feature modules should prefer these capabilities over direct reads of -access/tenant ORM models when they need labels, group membership, default -access provisioning, counts, audit actor labels, or tenant metadata. - -Other stable runtime capabilities currently include: - -- `identity.directory` and `identity.search` -- `organizations.directory` -- `idm.directory` -- `calendar.outbox` and `calendar.scheduling` -- `poll.scheduling` -- `notifications.dispatch` -- `workflow.definitionContributions` and `workflow.runtimeWorker` - -Modules contribute reusable process baselines through -`ModuleManifest.workflow_definitions`. Each contribution pins its origin module -and version, stable key, schema and content hash, native graph/BPMN content, -governance ceilings, execution mode, and required capabilities/interfaces. -`govoplan-workflow-engine` reconciles these declarations idempotently. A module -upgrade appends a baseline revision without replacing the active revision or -mutating a local override; the optional `govoplan-workflow` package supplies -the comparison, derivation, and reset UI. - -### Named Interface Contracts - -Capabilities are runtime objects. Named interface contracts are compatibility -metadata. A module uses them when it depends on a versioned cross-module API -shape but should not hard-code a package or repository release line. - -Manifest fields: - -- `provides_interfaces`: contracts this module provides, each with `name` and - `version` -- `requires_interfaces`: contracts this module needs, each with `name`, - optional `version_min`, optional `version_max_exclusive`, and optional - `optional: true` - -Interface names use dot-separated lower-case identifiers such as -`files.spaces` or `mail.delivery`. A requirement range is interpreted as -`>= version_min` and `< version_max_exclusive`; the exclusive upper bound is -intended for SemVer major-version lines. Missing optional interfaces are -allowed, but an installed provider with an incompatible version blocks -activation because the integration would otherwise bind to an unsafe API. - -### Source Authority And Provider Operations - -Integration maturity and configured authority are independent. The existing -external-reference maturity ladder describes whether an adapter can discover, -link, search, read, publish, synchronize, migrate, or replace. A binding must -also state whether GovOPlaN is native authoritative, the external system is -authoritative, GovOPlaN keeps a mirror, both sides use governed sync, GovOPlaN -adds only a governance overlay, or the object is link-only. - -`ModuleManifest.external_providers` composes existing contracts rather than -replacing them. Each declaration describes owned object/field groups, authority modes, -operations, revisions, freshness, health, limits, idempotency, conflicts, -outcome-unknown handling, evidence, correction/compensation, reconciliation, -outage behavior, classification, purpose, retention, and secret requirements. -Core owns the typed declaration and validation. Connectors and domain modules -own the actual protocol and domain behavior; configuration packages select the -effective mode and block missing/incompatible/unhealthy providers; Docs and Ops -explain the result. Effect-capable declarations fail validation unless their -retry, concurrency, outcome-unknown, correction, reconciliation, evidence, -audit, timeout, outage, classification, purpose, retention, and secret behavior -is explicit. - -Declarations are release-time capability claims. Configured state is projected -separately through `ModuleManifest.external_provider_state_providers`. A state -provider receives a bounded tenant context and returns one sanitized observation -per configured binding: stable binding reference, effective authority mode, -active/configured state, health, freshness, conflict, recovery readiness, -observation/last-success time, and scalar metrics. Core validates and aggregates -those observations, isolates provider failures, and never accepts URLs, -credentials, or arbitrary nested metadata as runtime-state fields. Docs removes -binding-level detail from ordinary-user projections; Ops may show the full -sanitized operator projection. - -Configuration-package preflight selects the exact requested binding from this -runtime state before evaluating authority, health, freshness, and recovery. A -healthy sibling binding therefore cannot mask an unhealthy required binding. -Providers with multiple configurations must use non-secret, stable references -such as `calendar:sync-source:`. - -Current named interfaces, generated from the source manifests by the workspace -contract checks, are: - -- `addresses.contact_writer`, `addresses.lookup`, `addresses.recipient_source` -- `calendar.outbox`, `calendar.scheduling` -- `campaigns.access`, `campaigns.delivery_tasks`, - `campaigns.mail_policy_context`, `campaigns.policy_context`, - `campaigns.retention` -- `dist_lists.expand`, `dist_lists.source`, `dist_lists.writer` -- `evaluation.feedback`, `evaluation.result_aggregation`, `evaluation.scoring` -- `files.access`, `files.campaign_attachments` -- `mail.campaign_delivery` -- `notifications.dispatch` -- `poll.availability_matrix`, `poll.option_selection`, - `poll.response_collection`, `poll.signed_participation`, - `poll.workflow_context` -- `rest.function_publication` -- `scheduling.candidate_slots`, `scheduling.decision_handoff` -- `soap.operation_publication` - -Core validates named interface contracts in three places: - -- registry activation rejects missing required interfaces and incompatible - providers -- installer preflight reports the same failures before a module set is - activated -- signed catalog validation normalizes the metadata and warns when catalog - entries cannot satisfy each other's ranges - -Module-id dependencies still decide startup ordering and mandatory package -presence. Named interfaces decide whether the versions in the active module -set are compatible. - -FastAPI route dependencies for authenticated endpoints are imported from the -core `govoplan_core.auth` facade. Routers may import that public API for -`ApiPrincipal`, `get_api_principal`, `has_scope`, `require_scope`, and -`require_any_scope`; they must not import access ORM models or -`govoplan_access.backend.*` implementation internals. - -Current live table ownership: - -- core scope table: `core_scopes` (used by access/core baseline; managed by - `govoplan-tenancy` behavior when the tenancy module is installed) -- `govoplan-access`: `access_accounts`, `access_users`, `access_groups`, - `access_roles`, `access_system_role_assignments`, - `access_user_group_memberships`, `access_user_role_assignments`, - `access_group_role_assignments`, `access_api_keys`, - `access_auth_sessions` -- `govoplan-admin`: `admin_governance_templates`, - `admin_governance_template_assignments` -- `govoplan-audit`: `audit_log` -- `govoplan-core`: `core_system_settings` - -Current admin route ownership follows the same boundary: access contributes -users, groups, roles, system accounts/roles, auth, sessions, and API-key -administration; tenancy contributes tenant registry/settings routes; admin -contributes system settings, overview, and governance-template routes; audit -contributes audit-log routes. Governance template metadata and assignment -routes live in `govoplan-admin`; materializing those templates into -access-owned groups and roles is performed by the -`access.governanceMaterializer` capability. - -Current admin WebUI ownership mirrors that route split. `govoplan-access` -contributes the `/admin` route shell and admin nav item. Other platform modules -contribute individual admin sections through the `admin.sections` UI capability. -`govoplan-admin` contributes the overview, system settings, and governance -template sections through that capability. Access-owned tenant/user/group/role -sections remain in the access package until their owning platform modules take -them over. - -Cross-module feature contracts live under focused kernel contract modules. For -example, `govoplan_core.core.campaigns` defines -`campaigns.access`, `campaigns.mailPolicyContext`, -`campaigns.policyContext`, `campaigns.deliveryTasks`, and -`campaigns.retention`. The campaign module registers these capabilities so mail -can resolve campaign owner/policy context and delivery tasks, files can validate -campaign file-share access, and core retention can call campaign-owned cleanup -logic without importing campaign ORM models. Keep these contracts small -DTO/protocol surfaces and register concrete behavior from the owning module. - -## API Efficiency Contracts - -GovOPlaN uses conditional GET and delta collections to reduce reload cost -without giving every module a custom synchronization format. - -### Conditional GET - -Core applies conditional GET handling centrally for successful JSON `GET` -responses: - -- Responses receive a weak `ETag` based on the serialized JSON body. -- Responses are marked `Cache-Control: private, no-cache`. -- Responses vary by `Authorization`, `Cookie`, `X-API-Key`, and - `Accept-Language`. -- Matching `If-None-Match` requests return `304 Not Modified` without a body. -- Responses with `Set-Cookie`, `Content-Disposition`, `Content-Encoding`, a - non-JSON content type, a non-200 status, or `Cache-Control: no-store` are not - converted. - -The WebUI `apiFetch` client keeps an in-memory conditional cache for reusable -safe requests. It sends `If-None-Match` after an endpoint has returned an ETag, -returns the cached payload on `304`, and clears the cache generation after -unsafe methods. - -This avoids retransmitting unchanged snapshots. It does not identify which row -changed inside a collection. - -### Mutation Preconditions - -Weak response ETags are cache validators only. Mutable aggregates expose a -separate positive, monotonic revision and an opaque strong ETag generated by -`govoplan_core.core.concurrency.strong_resource_etag`. HTTP mutations send that -strong ETag in `If-Match`; capability and worker calls carry the equivalent -typed `expected_revision`. - -Core's compare-and-set primitive advances the revision in the same transaction -as the domain mutation. A missing HTTP precondition is `428 Precondition -Required`, a stale HTTP precondition is `412 Precondition Failed`, and a -domain/reconciliation conflict is `409 Conflict`. Conflict responses contain -bounded resource and revision metadata rather than the complete current -object. Modules may opt into the conservative three-way merge helper, but must -declare protected workflow, delivery, ownership, lock, evidence, signature, -and cryptographic paths that can never be merged automatically. - -### Delta Collections - -Collection endpoints that can expose row-level changes should use the shared -delta contract instead of inventing module-specific formats. - -Core provides `core_change_sequence` as the shared monotonic change sequence. -Modules record append-only entries in the same database transaction as the -resource write. Watermarks are encoded as `seq:` and should be treated -as opaque by clients. - -Backend shape: - -```json -{ - "items": [], - "deleted": [], - "watermark": "opaque-next-watermark", - "has_more": false, - "full": false -} -``` - -Fields: - -- `items`: changed or current items since the requested watermark. -- `deleted`: deleted item markers with at least `id`, and optionally - `resource_type`, `revision`, and `deleted_at`. -- `watermark`: opaque value the client sends as `since` on the next request. -- `has_more`: true when the client should request the next page with the - returned watermark. -- `full`: true when the response is a full snapshot rather than an incremental - delta. - -Section-level settings endpoints use the same contract but replace `items` -with: - -- `item`: the full settings object when `full: true`. -- `sections`: a map of changed settings sections when `full: false`. -- `changed_sections`: ordered section identifiers the client can merge into its - local settings object. - -Recommended query parameters: - -- `since`: opaque previous watermark. If omitted or expired, return a full - snapshot with `full: true`. -- `limit`: maximum number of changed items plus deleted markers. -- `include_deleted`: whether deleted markers should be returned. -- `cursor`: opaque keyset cursor for table pages where offset shifts would make - row-level merging unsafe. - -Modules should record changes with: - -- `module_id`: the owning module, for example `files`. -- `collection`: the delta collection, for example `files.assets`. -- `resource_type`: stable row kind, for example `file` or `folder`. -- `resource_id`: stable resource identifier. -- `operation`: `created`, `updated`, or `deleted`. -- `tenant_id`: tenant scope when the change is tenant-owned. -- `payload`: small, non-secret routing metadata that helps determine whether a - tombstone belongs to the requested view. - -Sequence retention is explicit. Cleanup jobs must call -`prune_sequence_entries(...)` rather than deleting `core_change_sequence` rows -directly. Pruning records a retention floor per module, collection, and tenant -scope. Endpoints compare incoming watermarks with that floor; a watermark older -than the floor is not safe for incremental replay, so the endpoint must return a -full snapshot with `full: true`. A first-use `seq:0` watermark remains valid -until such a floor exists, even if unrelated collections have advanced the -global sequence. - -### Bounded Reference Selectors - -Cross-module selectors use the module-neutral contract in -`govoplan_core.core.references`; consumers must not load an optional module's -complete directory and filter it in memory. - -- Providers receive a normalized `ReferenceSearchRequest` with `kind`, - `tenant_id`, `query`, `selected_values`, `limit`, optional opaque `cursor`, - and policy context. -- Providers apply visibility and text filtering before materializing rows and - return `ReferenceSearchPage(options, next_cursor, has_more)`. -- A page contains at most the requested bounded search results. Already-selected - references are retained in addition to that bound so historical values remain - readable and removable even when they are inactive, deleted, or outside the - current search page. -- API consumers expose `next_cursor` and `has_more`. The current searchable - selector requests the first bounded page for each query; later load-more UI - can use the same cursor without changing the provider contract. -- `access.reference_options` supplies SQL-backed account, membership, and group - searches. When it is absent, Core degrades to the legacy Access directory or - to principal-only/unavailable references without importing Access. -- The shared WebUI `apiReferenceOptionProvider` resolves selected values in - chunks of at most 200, preventing a large existing selection from turning - into an unbounded request. - -### Cursor/Keyset Pages - -Offset pagination remains supported for compatibility and for first page loads, -but it is not safe as the merge anchor for row-level deltas on page 2 and later. -When a delta-capable table can be paged beyond the first page, the endpoint -should expose keyset cursors: - -- Core provides `encode_keyset_cursor`, `decode_keyset_cursor`, and - `keyset_query_fingerprint` in `govoplan_core.core.pagination`. -- Cursors are opaque to clients and contain the endpoint scope, query - fingerprint, and last-row keyset values. -- The fingerprint must include every query input that changes membership or - order: scope, tenant, page size, sort column, sort direction, and filters. -- Reusing a cursor with different sort or filter parameters must fail with a - client error rather than returning a mismatched slice. -- Responses may still include `page`, `page_size`, `pages`, and `total` for - existing UI components, but `cursor` identifies the current slice and - `next_cursor` is the safe anchor for the next slice. -- The first visit to an arbitrary page can use offset compatibility. The - response should include the start cursor for that page so later reloads and - delta requests use keyset semantics. - -Concrete consumers: - -- `GET /api/v1/files/delta`: without `since`, returns the current files/folders - snapshot for the requested owner/campaign scope. With `since=seq:`, - returns changed files, changed folders, and tombstones for resources that - left the current view. -- `GET /api/v1/campaigns/delta`: returns accessible campaign rows and campaign - tombstones when ownership, sharing, or soft deletion removes a campaign from - the current list. -- `GET /api/v1/campaigns/{campaign_id}/workspace/delta`: returns a workspace - snapshot first, then changed campaign/version metadata and optional summary - refreshes when version, job, issue, or delivery-attempt changes invalidate the - workspace view. -- `GET /api/v1/campaigns/{campaign_id}/jobs/delta`: returns a paginated job - table snapshot first, then stable row deltas for cursor-backed job pages. - The job list also supports offset compatibility for first visits to a page and - returns `cursor`/`next_cursor` for stable reloads. Filtered, created, deleted, - or stale-watermark requests fall back to a full page snapshot when pagination - membership can shift. -- `GET /api/v1/admin/users/delta`, `/groups/delta`, `/roles/delta`, - `/system/roles/delta`, `/system/accounts/delta`, and `/api-keys/delta`: - return access administration row deltas with tombstones where rows leave the - visible view. -- `GET /api/v1/admin/system/settings/delta`: returns section deltas for system - defaults, tenant capability flags, language packages, privacy retention - policy, maintenance mode, and raw settings. -- `GET /api/v1/admin/tenant/settings/delta`: returns tenant-local setting - sections and also reports language-section changes when system language - packages or enabled language codes change. -- `GET /api/v1/admin/configuration-changes/delta`: returns changed - configuration requests and history records. -- `GET /api/v1/admin/audit` and `/api/v1/admin/audit/delta`: return append-only - audit events using the same scope, sort, and filter query parameters. The list - supports offset compatibility plus `cursor`/`next_cursor` keyset paging; the - delta endpoint can replay changes against a cursor-backed slice. -- `GET /api/v1/mail/settings/delta`: returns mail profile row deltas plus the - current scoped mail profile policy when profile-policy dependencies changed. - The WebUI consumes this for system, tenant, user, group, and campaign mail - settings panels. -- `GET /api/v1/files/connectors/settings/delta`: returns file connector - profile, credential, connector-space, and scoped connector-policy deltas. - Credential changes also include referencing profiles because profile rows - display credential-derived state. - -Open retrofit scope: - -- Additional module-specific settings pages should expose section deltas as - their settings APIs stabilize. Remaining likely candidates are future - booking/resource configuration pages and settings pages introduced by new - modules. -- Remaining high-volume tables should adopt the cursor/keyset contract before - enabling arbitrary-page row deltas. Current rollout follow-ups: - `govoplan-files#22` for large file-space server windows, - `govoplan-mail#9` for provider-aware mailbox message cursors, - `govoplan-calendar#7` for event-window deltas, - `govoplan-tenancy#1` for tenant administration row deltas, and - `govoplan-admin#2` for governance/module-operation list deltas. - -## Module Responsibilities - -A module owns one bounded feature area. A module can include both backend and WebUI code in the same repository so feature behavior and frontend integration evolve together. - -A module owns: - -- backend routers and feature services -- SQLAlchemy models for module-owned tables -- module migrations and migration metadata -- module permissions and role templates -- module-specific schemas, policies, and domain rules -- module package metadata and retirement providers used by install/uninstall preflight -- WebUI pages, feature-specific components, API clients, route contributions, and navigation metadata - -A module should not own generic platform UI. If a component is useful outside one module, move it to `@govoplan/core-webui` and parameterize it there before reusing it. - -## Backend Contract - -Backend modules register through the `govoplan.modules` entry point and expose a `ModuleManifest`. - -Example: - -```toml -[project.entry-points."govoplan.modules"] -files = "govoplan_files.backend.manifest:get_manifest" -``` - -The manifest should declare: - -- `id`, `name`, `version` -- `compatibility` when the module needs a minimum/maximum core version or a - newer manifest contract -- required `dependencies` and `optional_dependencies` -- permissions and role templates -- router factory -- migration metadata and script location -- frontend package metadata -- navigation metadata using serializable icon names -- uninstall guard providers for data, migration, worker, or scheduler vetoes - -A tenant-level managed `RoleTemplate` may set `default_authenticated=True` -only when every authenticated tenant member must receive that narrow baseline -while the contributing module is installed. Access derives the explicit grant -from the active manifest set during authorization without mutating the request -transaction. It may materialize a non-assignable role row for administration, -but no per-user assignment is required and role edits cannot remove the -baseline. -This is not a shortcut for feature authorization: keep the template narrow and -continue to enforce each domain action's own permission and resource policy. -System-level, unmanaged, wildcard-bearing, or slug-colliding automatic -templates are rejected by registry validation. - -Backend nav metadata must use icon-name strings, not frontend components: - -```python -NavItem( - path="/files", - label="Files", - icon="folder", - required_any=("files:file:read",), - order=40, -) -``` - -Core validates manifest shape when the platform registry is built. The current -supported manifest contract version is `1`, and frontend asset manifests use -contract version `1`. Registry validation rejects unsupported contract versions, -invalid module ids, duplicate dependency declarations, self-dependencies, -mismatched migration/frontend metadata, invalid frontend package names, and -frontend/nav routes that do not declare usable paths and labels. - -Backend route contributions are also validated before they are mounted. Startup -routers and live module activation fail fast if two routers register the same -HTTP method and path. That keeps OpenAPI output and FastAPI route order from -silently masking a module collision. - -Tenant deletion and cleanup use the registry-owned delete-veto contract. A -module that owns tenant-bound data may declare `delete_veto_providers` on its -manifest for resource types such as `tenant` or `group`. Providers receive -`(session, tenant_id, resource_id)` and should return `DeleteVetoIssue`, an -iterable of `DeleteVetoIssue`, or `None`; older exception-based providers are -still treated as blocking vetoes. Core attributes each issue to the provider -module and adds resource context before the tenancy module exposes the issues -through the deletion plan. `blocker` issues prevent destructive or retire -operations, `warning` issues explain retained data, and `info` issues document -non-blocking lifecycle facts. - -## Database And Migrations - -Core owns the database/session lifecycle. Modules access the database through core session dependencies and register their models/migrations through their manifest. - -Rules: - -- Do not create independent database engines in modules. -- Use core session dependencies, base metadata, and migration orchestration. -- Keep module-owned tables and migrations in the module repository. -- Keep cross-module foreign-key assumptions explicit and conservative. -- Register module metadata in `MigrationSpec` so core can discover it. -- Optional module migrations may create multiple Alembic heads. Verification - should compare database heads to Alembic's resolved `heads` target instead - of assuming one linear revision when multiple modules are enabled. Owner - heads can be dependency parents and therefore may not all appear in - `alembic_version` after a full-graph upgrade. -- Treat migrations as release artifacts. Unreleased migrations may be squashed - or rewritten before a stable release; released revision IDs are immutable - once an installation may have recorded them. Each stable release records its - public migration heads in `docs/migration-release-baselines.json`. -- GovOPlaN keeps two Alembic tracks. The default `release` track loads - `versions` directories with reviewed release baselines and release-to-release - step-up migrations. The explicit `dev` track loads `dev_versions` - directories with the detailed development chain. Do not load both tracks for - one migration run, and do not switch a database between tracks unless it is a - disposable development database. - -### Shared State And Runtime Ordering - -Multi-host application roles use the `shared` state profile. In that profile, -PostgreSQL, Redis, a stable installation identifier, and S3-compatible object -storage are mandatory. Module durable artifacts must use Core's object-storage -contract and module-owned opaque key namespaces; node-local paths are limited -to temporary materialization. Same-host replicas may use the `host-shared` -profile and one shared volume. - -Only the migration command mutates schema. PostgreSQL migration runs acquire a -deployment-wide advisory lock before module pre-tasks, Alembic, and post-tasks. -API, worker, and scheduler roles wait for exact configured migration heads and -fail closed instead of applying migrations during startup. - -Runtime roles register identity, software/module composition, queues, heartbeat, -and drain state in PostgreSQL. Singleton work must use a distributed lease and -validate its monotonically increasing fencing token at the consequential -commit. See `STATE_AND_RECOVERY_CONTRACT.md` for the complete contract. - -### Recovery Evidence - -Operations spanning transactions, object storage, queues, or external systems -must choose an explicit Core recovery mode: atomic, compensation, -snapshot-restore, forward-recovery, or irreversible. Plans require verification -steps and mode-specific recovery material. Use idempotency keys, append-only -evidence checkpoints, and a runtime fence where work may race across nodes. - -The recovery ledger is a shared primitive, not automatic coverage. A module may -claim its guarantees only after its operation records preconditions before side -effects, transitions partial/unknown outcomes honestly, and records verified -completion or recovery. Plaintext secrets must never enter recovery metadata or -evidence. - -## Install, Uninstall, And Catalogs - -Core owns the install plan, signed catalog validation, license entitlement -check, maintenance-mode guard, replay state, and installer request queue. -Package mutation is performed by `govoplan-module-installer` outside the -FastAPI request process. - -Official catalogs can be served as static JSON from `addideas-govoplan-website`, but core -does not trust the website by location alone. A catalog must pass the configured -signature, channel, freshness, and replay rules before a catalog entry can be -planned. Catalog entries may declare `license_features`; core checks those -against the configured offline license before adding the entry to the install -plan. Catalog entries may also declare `migration_safety` as `automatic`, -`requires_review`, `forward_only`, or `destructive`; forward-only and -destructive entries require explicit operator acknowledgement in the install -plan before installer preflight allows activation. Forward-only and destructive -catalog entries must also declare a tested recovery path. Catalog update entries -can define direct-update windows with `current_version_min` and -`current_version_max_exclusive`, mark intermediate `bridge_release` targets, and -explicitly opt into reviewed downgrade or same-version package-refresh plans. -Module migration order can be declared with `migration_after` and -`migration_before` in manifests or release catalogs; installer preflight turns -that metadata, module dependencies, and named interface relationships into an -ordered migration plan. - -Modules that need live-data work outside Alembic schema revisions may declare -`migration_tasks` on `MigrationSpec`. This is deliberately narrower than a -general lifecycle hook system. Each task has a stable `task_id`, one of four -phases (`pre_migration_check`, `pre_migration_prepare`, -`post_migration_backfill`, `post_migration_verify`), a short operator-facing -summary, a task version, safety metadata, and an idempotent executor. Installer -preflight blocks non-idempotent tasks, forward-only/destructive tasks without -operator acknowledgement, and installed manifest tasks that have no executor. -Catalog task metadata is surfaced before activation as pending because the -executor can only be verified after the package is installed. - -Modules should provide: - -- pinned backend and WebUI package refs for official catalog entries -- module dependency metadata for catalog target-state planning -- migration-safety metadata for catalog update planning -- migration task metadata when live-data checks, preparation, backfills, or - verification must run around Alembic -- compatibility metadata in the module manifest -- named interface contracts in the manifest and catalog entry when the module - provides or consumes cross-module APIs -- lifecycle hooks when a runtime enable/disable action needs module-specific - work -- uninstall guards for persistent data, active workers, schedulers, or external - bindings -- retirement providers when destructive uninstall can safely drop or retire - module-owned data - -Uninstall remains non-destructive unless the operator explicitly requests -`destroy_data` and the module provides a retirement provider that supports it. - -## WebUI Contract - -A WebUI module exports a `PlatformWebModule` from its package. The object -contributes local/fallback metadata and route render functions. The package -must ship `src/module.ts` with the default contribution export: Core's Vite -host imports that descriptor directly after the backend reports the module as -enabled. This keeps package-root re-exports from pulling page implementations -into the initial shell. - -Example: - -```ts -const FilesPage = lazy(() => import("./features/files/FilesPage")); - -export const filesModule: PlatformWebModule = { - id: "files", - label: "Files", - version: "1.0.0", - dependencies: ["access"], - navItems: [ - { to: "/files", label: "Files", iconName: "folder", anyOf: ["files:file:read"], order: 40 } - ], - routes: [ - { path: "/files", anyOf: ["files:file:read"], order: 40, render: ({ settings, auth }) => createElement(FilesPage, { settings, auth }) } - ] -}; -``` - -Route pages and substantial panels must use stable lazy imports. Core supplies -the shared loading and retryable error state around route rendering. The -initial static import closure and largest asynchronous chunk are enforced by -the budgets documented in [WEBUI_BUNDLE_BUDGETS.md](WEBUI_BUNDLE_BUDGETS.md). - -WebUI modules receive only the core route context: - -- `settings` -- `auth` - -A module should call its own API client and module-owned backend routes. Shared API helpers should live in core only when they are truly platform-level concerns. - -Modules can also contribute named UI capabilities for explicit extension -points. Capability values must be narrow, typed contracts, not imports from a -sibling feature package. For admin pages, modules contribute: - -```ts -const adminSections: AdminSectionsUiCapability = { - sections: [ - { - id: "system-settings", - label: "General", - group: "SYSTEM", - order: 10, - allOf: ["system:settings:read"], - render: ({ settings, auth }) => createElement(SystemSettingsPanel, { settings, auth }) - } - ] -}; -``` - -The access admin route shell collects all installed `admin.sections` -capabilities with `usePlatformUiCapabilities("admin.sections")`, filters them -by `anyOf`/`allOf`, and renders them without importing the contributing module's -components directly. - -The configurable dashboard follows the same pattern. Core contributes only a -minimal `/dashboard` fallback when no `dashboard` WebUI module is active. The -`govoplan-dashboard` module owns the real `/dashboard` route and collects -widgets exposed through the `dashboard.widgets` capability: - -```ts -const dashboardWidgets: DashboardWidgetsUiCapability = { - widgets: [ - { - id: "ops.health", - title: "Operations health", - moduleId: "ops", - defaultSize: "wide", - anyOf: ["ops:operations:read"], - render: ({ settings, refreshKey }) => createElement(OpsHealthWidget, { settings, refreshKey }) - } - ] -}; -``` - -Dashboard widgets are module contributions, not cross-module imports. A widget -may render components from its own module and core components only. The -dashboard module is responsible for layout, visibility, refresh context, and -future server-side layout persistence. - -## Icon Rules - -Icons are resolved centrally by core. - -Modules must provide icon names with `iconName` in frontend nav contributions and `icon` in backend manifest metadata. Modules must not import Lucide icons for navigation metadata. - -Current core icon names include: - -- `activity` -- `admin` -- `campaign` -- `dashboard` -- `file` -- `files` -- `folder` -- `form` -- `mail` -- `reports` -- `users` - -If a module needs a new navigation icon, add the name-to-component mapping in core first, then use the name in backend and frontend metadata. - -The access module uses the `admin` icon for its `/admin` route. Core only -resolves that icon name; it does not hard-code the admin route in the rail. - -## Shared Component Rules - -Use this rule of thumb: - -- If it is platform-level or likely reusable by more than one module, define it in `@govoplan/core-webui` with parameters. -- If it is feature-specific and only meaningful inside one bounded module, keep it in that module. -- Modules must not import components from another feature module. -- If one module needs a component currently owned by another module, promote a generic version into core and replace the old usage with the core component. - -Examples: - -- `ExplorerTree` is core because files, mailboxes, and future modules can all render hierarchical navigation. -- `MessageDisplayPanel` is core because mail, campaign sending, and later audit/review surfaces can display message-like content. -- `AdminPageLayout`, `AdminIconButton`, and `AdminSelectionList` are core - because access, admin, tenancy, policy, and audit panels share the same admin - shell language. -- `MailProfileManagement` remains in the mail module because it is specific to mail transport policies and profiles. - -## Cross-Module Integration - -A module can declare required module dependencies, optional module -dependencies, required capabilities, and optional capabilities. Required module -dependencies are reserved for unavoidable startup ownership, such as a module -that cannot import or mount without another module package. Most runtime -relationships should be expressed as capabilities instead. - -Auth/principal access is a capability contract, not a reason to hard-depend on -the `govoplan-access` repository. Current routers import `govoplan_core.auth`; -that facade delegates to access today and is the migration point for a future -provider-neutral auth kernel. Feature manifests should require -`auth.principalResolver` and `auth.permissionEvaluator`, while access remains -the default installed provider. - -Tenancy is optional. Existing scoped data still uses `tenant_id` as a scope -identifier, backed by the core-owned `core_scopes` table. Access and -organizations must not import the tenancy package or declare a hard dependency -on it. Tenancy-specific administration and tenant resolver behavior live behind -the tenancy module and its capabilities. - -Optional behavior should be enabled by module presence, capabilities, and -permissions, not by importing another module's WebUI internals. - -Rules: - -- Use core module metadata to check whether another module is installed. -- Use backend APIs/events/service contracts for runtime cooperation. -- If a sibling module needs owner-specific data, expose a narrow DTO/protocol - capability from the owning module instead of importing its ORM models. -- Keep UI integration declarative where possible: nav items, route contributions, context actions, and explicit extension points. -- Avoid direct imports from one feature module into another feature module unless the imported package is a published API contract designed for that purpose. UI components should be promoted to core instead. - -### Dependency Boundary Enforcement - -The meta repository includes `tools/checks/check_dependency_boundaries.py`. It enforces the current baseline: - -- kernel/core source may not add new direct imports of files/mail/campaign internals -- access source may not import files/mail/campaign internals -- feature modules may not import access implementation internals -- feature modules may not add new direct imports of sibling feature modules -- feature WebUI packages may not depend on or import sibling feature WebUI packages -- core WebUI may list module packages as host dependencies, but core WebUI source - may not import feature WebUI internals directly; module loading stays - declarative through the module contribution contract -- FastAPI routers import the core `govoplan_core.auth` dependency facade -- the transitional allowlist is expected to stay empty - -Any future exception is extraction debt and must be temporary, documented in the -script with a reason, and removed when a capability/API/event contract replaces -it. - -## Boundary Decision Register - -These durable decisions close older exploratory core issues. Implementation -work should live in the owning module repositories once a boundary is clear. - -Decision principles: - -- Prefer connector-first when an external specialist system is likely to remain - the system of record. -- Create a native module only when GovOPlaN must own domain semantics, - permissions, audit, retention, configuration-package fragments, or workflow - state. -- Keep optional behavior behind core-mediated capabilities, events, DTOs, route - contributions, and UI contribution points. -- Do not create repositories just because a possible product area exists. - -### Templates And Reporting - -Tracking: `govoplan-core#190`, `govoplan-templates#1`, -`govoplan-reporting#1`. - -Decision: templates and reporting are separate modules. - -`govoplan-templates` owns: - -- reusable renderable templates for letters, permits, emails, forms, reports, - certificates, and notices -- template versioning, merge-field declarations, rendering profiles, output - format choices, and preview contracts -- template package fragments that other modules can reference - -`govoplan-reporting` owns: - -- report definitions, data selection, dashboards, BI views, scheduled outputs, - and export targets -- report permissions, report execution history, generated report evidence, and - report-specific retention inputs -- downstream export handoff to files, dataflow, connectors, or publication - surfaces - -Boundary: - -- Templates do not own data selection, aggregation, scheduling, or BI semantics. -- Reporting may call template rendering through a capability when a formatted - report output is needed. -- Campaign, mail, files, workflow, and cases use templates/reporting through - capabilities and DTOs, never direct imports. - -### Sources, RSS, Datasources, And Dataflow - -Tracking: `govoplan-core#192`, `govoplan-core#197`, -`govoplan-core#198`, `govoplan-connectors#3`, -`govoplan-connectors#4`. - -Decision: do not create `govoplan-datasources` or `govoplan-dataflow` until a -first executable use case proves that connector/reporting/workflow ownership is -too narrow. - -First slice: - -- `govoplan-connectors` owns RSS/Atom consume/emit connector profiles, - connector health, external references, source lifecycle metadata, and - source/publish capability boundaries. -- `govoplan-files` owns file-backed governed locations and uploaded/stored file - evidence. -- `govoplan-reporting` owns report/data views and scheduled outputs. -- `govoplan-workflow-engine` owns process state, approvals, scheduling of process - steps, and human review. - -Future `govoplan-datasources` is justified when GovOPlaN needs a broad source -catalogue for SQL databases, CSV/Excel files, APIs, RSS feeds, uploaded files, -and governed file locations with shared ownership, credentials, schema -discovery, refresh cadence, provenance, and permission boundaries. - -Future `govoplan-dataflow` is justified when GovOPlaN needs first-class -pipelines for ingestion, transformation, validation, scheduling, lineage, -publication, audit events, reruns, and source-to-source workflows. - -Monthly extraction/transformation work should start as a configuration package -and module collaboration across connectors, files, workflow, reporting, and -possibly templates. Create datasources/dataflow repositories only after that -package exposes repeated contracts that do not belong to an existing module. - -### Calendar, Scheduling, And Appointments - -Tracking: `govoplan-core#193`, `govoplan-calendar#1`, -`govoplan-calendar#2`, `govoplan-scheduling#1`, -`govoplan-appointments#1`. - -Decision: use three separate modules. - -`govoplan-calendar` owns: - -- calendar collections, events, recurrence, availability/free-busy, resources, - iCalendar import/export, CalDAV/Open-Xchange-style calendar adapters, and - calendar WebUI surfaces - -`govoplan-scheduling` owns: - -- Terminfindung, meeting-time polls, participant availability collection, - candidate-slot ranking, conflict explanations, reminders, and the handoff - from a selected slot to calendar/appointment/workflow modules - -`govoplan-appointments` owns: - -- Terminbuchung/fixed-slot appointment booking, appointment types, booking - rules, capacity, cancellation/no-show state, public/internal booking flows, - and appointment evidence - -Boundary: - -- Calendar provides time primitives and external calendar integration. -- Scheduling chooses a suitable time. -- Appointments owns booked appointment workflows and public/internal booking - semantics. -- Mail and notifications deliver invitations/reminders through capabilities. - -### Forms And Workflow Handoff - -Tracking: `govoplan-core#194`, `govoplan-forms#1`. - -Decision: forms are a reusable module boundary, with runtime behavior separated -from workflow semantics. - -`govoplan-forms` owns: - -- form definitions, schemas, validation rules, field visibility rules, - localization, versioning, admin editing, and reusable form package fragments - -`govoplan-forms-runtime` owns: - -- public/internal submissions, drafts, submitted values, validation evidence, - attachment references, submission receipts, and handoff events - -Boundary: - -- Forms do not own cases, workflow transitions, tasks, or portal identity. -- Workflow/cases consume form submission events and evidence references. -- Files owns uploaded file storage and file permissions. -- Reporting/dataflow may consume submitted data through governed DTOs or - source lifecycle contracts. - -Implemented contract: - -- Core owns the provider-neutral `FormDefinition`/`FormFieldDefinition` DTOs. -- Forms persists immutable exact definitions and provides `forms.definitions`. -- Forms Runtime resolves that capability, persists revisioned instances and - events, validates draft/final values, and provides - `forms_runtime.service_launcher`. -- Portal delegates exact `/` bindings and never writes either - owner's tables. - -### OpenDesk Integration Profile - -Tracking: `govoplan-core#195`, `govoplan-connectors#5`, -`govoplan-idm#1`, `govoplan-mail#5`, `govoplan-calendar#2`, -`govoplan-connectors#1`. - -Decision: OpenDesk is an integration profile, not a monolithic module. - -Ownership: - -- identity: `govoplan-idm` plus `govoplan-access` -- mail/groupware: `govoplan-mail` -- calendar: `govoplan-calendar` -- files/documents: `govoplan-files` and later `govoplan-dms` -- projects/tasks: `govoplan-connectors` OpenProject connector first -- inventory/health/profile diagnostics: `govoplan-connectors` - -The OpenDesk profile should describe required connector profiles, shared -identity assumptions, health checks, and optional module combinations. It must -not create direct module-to-module imports. - -### Project Management And OpenProject - -Tracking: `govoplan-core#196`, `govoplan-connectors#1`. - -Decision: connector-first. Do not create a native `govoplan-projects` module -yet. - -OpenProject integration belongs in `govoplan-connectors` first: - -- profile test -- project and work-package lookup -- external-reference storage -- selected publish/synchronize capabilities for tasks, workflow, or cases - -A native project module is justified only if GovOPlaN needs to own project -semantics beyond cases, tasks, workflow, appointments, documents, and reporting, -for example portfolios, project budgets, project-level resource planning, or -governed project records that cannot remain in OpenProject. - -### Public-Sector Integration Landscape - -Tracking: `govoplan-core#186`, `govoplan-core#215`, -`govoplan-connectors#2`, `govoplan-connectors#3`. - -Decision: core owns strategy and routing; connectors owns executable -integration catalogue entries and operator inventory. - -Core documents: - -- product-level integration strategy -- native-vs-connector decisions -- owning module routing -- roadmap sequencing - -`govoplan-connectors` owns: - -- connector entry schema -- external system catalogue -- connector profiles and diagnostics -- source consume/publish lifecycle -- external references - -When a target needs executable behavior, create the implementation issue in the -owning module repository and keep only cross-module decisions in core. - -## Module Lifecycle - -Core exposes the installed module catalog through the admin API and WebUI. The -current lifecycle model separates four states: - -- installed: the Python/WebUI package is available to the process -- active: the module is present in the running platform registry -- desired: the module should be active on the next server startup -- planned package change: an operator-reviewed package install/uninstall item - saved in system settings but not executed by the running server - -The admin module manager can change the desired enabled set and apply it to the -running server. It always keeps `tenancy`, `access`, and `admin` enabled when -saving through the admin UI, and it adds required module dependencies before -saving the desired state. On startup, core always keeps the minimum -authenticated platform set `tenancy`/`access` enabled and keeps `admin` enabled -when the operator configuration includes it. Unknown saved module ids are -ignored when the matching package is no longer installed. The core app factory, -devserver, development bootstrap, background worker registry, and migration -metadata plan all read the saved desired state from `system_settings` before -building their module registry. - -Hot enable/disable is a core design principle for every module: - -- Core keeps one mutable active `PlatformRegistry` object and swaps its manifest - set through the module lifecycle manager. Modules must read module presence, - optional integrations, permissions, role templates, capabilities, navigation, - and frontend contributions from that registry instead of caching sibling - module availability. -- Core validates install state and dependency closure before activation. -- Core applies configured module migrations before activation. Deactivation - never drops tables or data. -- Core mounts module routers once and guards them by active module state. A - deactivated module's routes remain mounted internally but return a disabled - module response until the module is active again. -- Module route factories must be side-effect-light and idempotent. They may - configure module runtime references, but they must not start workers, - schedulers, or irreversible external subscriptions. Use lifecycle hooks for - those resources. -- Modules that own persistent data, background jobs, schedulers, external - subscriptions, or irreversible migration state must expose uninstall guard - providers through their manifest. Guards return `blocker`, `warning`, or - `info` results and may inspect live state through the core-owned DB session. - Default package uninstall is non-destructive, so ordinary persistent data - should warn that data will remain dormant. Guards should block only when - removing the package would corrupt other active modules, workers, external - subscriptions, or deployment state. A guard failure is treated as a blocker. -- Modules that can destroy their own data must also expose a migration - retirement provider. Destructive retirement is opt-in per uninstall plan row - through `destroy_data: true`; the installer then snapshots the database, - invokes the module-owned retirement executor while the package is still - installed, and only then removes Python/WebUI packages. Without that flag, - the same provider is used for preflight reporting only and module tables/data - remain dormant. -- Core refreshes the active registry before frontend metadata is returned from - `/api/v1/platform/modules`; the WebUI shell refetches this metadata after - module changes so navigation, routes, and UI capabilities update without a - page reload. -- Modules can provide `on_activate` and `on_deactivate` hooks for worker, - scheduler, cache, or external subscription lifecycle. These hooks must be - idempotent and must not mutate another module directly. -- Package install/uninstall is performed by the trusted operator installer, not - directly inside FastAPI request handlers. The admin UI can save install plans, - show preflight blockers, and activate/deactivate installed packages. - -The package install-plan API records operator intent only: - -- `GET /api/v1/admin/system/modules/install-plan` reads the saved plan, - renders shell commands, and returns installer preflight status. -- `GET /api/v1/admin/system/modules/install-runs` returns recent installer run - summaries and the current installer lock status. -- `GET /api/v1/admin/system/modules/install-runs/{run_id}` returns the raw run - record for diagnosis. -- `GET /api/v1/admin/system/modules/install-requests` returns daemon handoff - requests queued from the admin UI or CLI plus the current daemon heartbeat. -- `POST /api/v1/admin/system/modules/install-requests` queues a supervised - installer request. It requires maintenance mode and maintenance access. The - FastAPI request writes only the request record; it does not run package - commands. -- `POST /api/v1/admin/system/modules/install-requests/{request_id}/cancel` - cancels a queued request. Running requests are not interrupted by the API; - they remain owned by the installer daemon. -- `POST /api/v1/admin/system/modules/install-requests/{request_id}/retry` - queues a new request using the options from a failed or cancelled request. -- `GET /api/v1/admin/system/modules/package-catalog` reads approved package - references from `GOVOPLAN_MODULE_PACKAGE_CATALOG` so operators can add known - module refs to the install plan without typing them manually. The endpoint - also reports catalog validity, channel, signature, trust state, and the - configured path. -- `POST /api/v1/admin/system/modules/install-plan/catalog/{module_id}` saves - a planned install or update row from a validated catalog entry. Installed - modules are planned as updates. Catalog signature and approved-channel policy - are enforced before the row is saved. When the selected catalog row requires - companion dependency or interface-provider updates, the endpoint adds those - rows to the plan automatically. The saved plan row can also carry a - data-safety acknowledgement used by preflight for forward-only or destructive - catalog entries. -- Install-plan preflight returns a structured `target_plan` summary so the - admin UI can show current version, target version, package refs, - migration-safety level, update-window and bridge metadata, recovery metadata, - and acknowledgement state without requiring JSON editing. -- Install-plan preflight also returns a structured `migration_plan` summary with - target enabled modules and ordered module migration steps. When the installer - runs with migration enabled, the database migration command receives that - target module set and ordered module list. -- `POST /api/v1/admin/system/modules/{module_id}/uninstall-plan` saves a - planned non-destructive uninstall row for an installed module after it has - been disabled. The Python distribution name is resolved from the installed - `govoplan.modules` entry point; the WebUI package name comes from the module - manifest. Operators can then edit the saved plan row and set `destroy_data` - when they explicitly want module-owned tables/data retired before package - removal. -- `PUT /api/v1/admin/system/modules/install-plan` saves planned install or - uninstall rows. Install rows must use tagged package or git references, not - local `file:`/workspace paths. Python install rows must also include the - distribution package name so rollback can uninstall newly added packages. -- `DELETE /api/v1/admin/system/modules/install-plan` clears the plan. -- `govoplan-module-install-plan --format shell` or - `python -m govoplan_core.commands.module_install_plan --format shell` renders - the same commands from a server shell. -- `govoplan-module-installer --format shell` runs the same preflight checks from - the server shell. -- `govoplan-module-installer --apply --build-webui` executes the saved plan - after preflight passes, snapshots `pip freeze` and WebUI package files, writes - a run record under the runtime installer directory, and marks planned rows as - applied after success. Successful installs are added to saved startup state by - default; successful uninstalls are removed from saved startup state by default. - Use `--no-activate-installed-modules` or - `--keep-uninstalled-modules-in-desired` only for staged rollout workflows. -- `govoplan-module-installer --supervise --migrate --health-url http://127.0.0.1:8000/health --restart-command ''` - is the preferred disruptive-change path. It applies the plan, optionally runs - migrations in a fresh Python process after a fresh-process manifest - verification, runs the restart command if provided, polls health, and - automatically rolls packages back from the run snapshot if commands, - migrations, restart, or health recovery fail. -- `govoplan-module-installer --daemon` runs the request executor. It polls the - runtime request queue, claims one request at a time, and executes the same - supervised installer flow. `--daemon-once` processes at most one queued - request and exits, which is useful for tests or process-manager one-shot - units. The daemon writes `daemon.status.json` under the installer runtime - directory so the admin UI and CLI can report heartbeat/status. -- `govoplan-module-installer --enqueue-supervised` creates the same request - record from a shell instead of from the admin UI. -- `govoplan-module-installer --daemon-status --format json` reports the daemon - heartbeat. `--cancel-request ` and `--retry-request ` - provide shell equivalents for the admin UI request controls. -- `govoplan-module-installer --validate-package-catalog [path] --format json` - validates a catalog file or the catalog configured through - `GOVOPLAN_MODULE_PACKAGE_CATALOG`. -- `--sign-package-catalog --catalog-signing-key-id --catalog-signing-private-key ` - signs a catalog with Ed25519. `--require-signed-catalog`, - `--approved-catalog-channel `, and - `--catalog-trusted-key =` enforce the approved - release-channel path from an operator shell. -- `govoplan-module-installer --rollback ` restores the saved package - snapshots, restores the captured SQLite or external database snapshot when - present, restores the previous desired module state when the database was not - restored wholesale, and reruns package installation from the previous freeze - file. -- `--database-backup-command ''` and - `--database-restore-command ''` provide non-SQLite backup/restore - hooks for migrated installer runs. The backup hook runs before migrations; - the restore hook runs during rollback and can be overridden on the rollback - command line. -- `govoplan-module-installer --list-runs --format json`, - `--show-run --format json`, and `--lock-status --format json` - expose the same run-history and lock information from the operator shell. -- `--list-requests --format json` and `--show-request --format json` - expose daemon handoff records from the operator shell. - -The supervisor accepts multiple restart commands and health URLs. This is the -process boundary for web, worker, scheduler, and auxiliary service restarts: -each restart command is executed, each health URL is polled, and rollback uses -the same restart/health set after restoring package and database snapshots. - -The installer preflight is intentionally conservative: - -- maintenance mode must be active; -- the `shared` state profile blocks in-place package mutation; clustered - installations must roll one verified immutable module composition across all - replicas; -- installed module manifests must be compatible with the supported manifest - contract and current core version; -- uninstalling `tenancy`, `access`, or `admin` is blocked; -- uninstalling an active module is blocked; -- uninstalling a module still present in desired startup state is blocked; -- uninstalling a module with active/desired dependents is blocked; -- uninstalling a module that owns migrations is non-destructive by default: - schema/data remain dormant and preflight emits a warning; -- modules that declare explicit migration retirement support should also - register a retirement provider. Without a provider, preflight emits a - manual-review warning. Providers may return blockers, warnings, and an - explanatory summary, but core does not drop schema or data on behalf of - modules. -- module-owned uninstall guard providers can veto data/migration/worker unsafe - removals; -- install refs must be exact versions or tagged git refs; -- Python install rows must include the package distribution name for rollback; -- WebUI package changes require a WebUI root and trigger rebuild/reload status. - -The installer supervisor must run outside the FastAPI server process. A server -request handler cannot reliably restart or roll back the process that is -currently executing the request. The admin UI therefore remains an operator -planning and request-submission surface; the trusted daemon/CLI is the executor. - -Automatic rollback covers Python and WebUI package state. The manual/shell -daemon can run `npm install` and `npm run build`; this remains the supported -WebUI package path. Browser-loaded remote module bundles are experimental and -reserved for controlled deployments with integrity/signature policy. - -For `sqlite:///` database URLs, `--migrate` also captures a SQLite backup and -rollback restores it before the supervisor restarts the server. For non-SQLite -database URLs, `--migrate` requires deployment-specific backup and restore -commands. A `--database-restore-check-command` can validate the created backup -artifact before migrations proceed. Hook commands run in the installer run -directory with: - -- `GOVOPLAN_INSTALLER_RUN_DIR` -- `GOVOPLAN_DATABASE_URL` -- `GOVOPLAN_DATABASE_URL_PGTOOLS` for PostgreSQL URLs converted to the - `postgresql://` form expected by `pg_dump`, `pg_restore`, and `psql` -- `GOVOPLAN_DATABASE_BACKUP_PATH` -- `GOVOPLAN_DATABASE_BACKUP_METADATA` - -Module uninstall does not retire data by default. Package removal leaves -module-owned schema/data dormant. Explicit retirement is a later module-owned -operation guarded by retirement providers. - -The running FastAPI server still reports `package_mutation_supported=false` -because dependency-manager operations are not executed inside request handlers. -The trusted mutation boundary is the operator CLI/daemon. This keeps the -interpreter, npm dependency graph, frontend bundle, migrations, and worker -process set under process-supervisor control. - -Frontend module loading primarily uses the build-time package graph generated by -the core WebUI host. Installing or uninstalling a WebUI package therefore still -uses `npm install` plus a WebUI rebuild/reload by default. Core also exposes an -experimental hot remote-bundle path for modules that are enabled by the backend -but absent from the local WebUI package graph: - -- `FrontendModule.asset_manifest` points at a JSON remote asset manifest. -- `asset_manifest_integrity` is an SRI-style hash for that manifest. -- `asset_manifest_signature` and `asset_manifest_public_key_id` allow the shell - to verify the manifest when a trusted browser key is registered on - `window.__GOVOPLAN_REMOTE_MODULE_KEYS__`. -- The remote asset manifest contract version is `1` and contains `moduleId`, - `entry`, `entryIntegrity`, and optional `moduleExport`. -- The browser fetches and verifies the manifest, fetches and verifies the entry - bundle, imports it dynamically, and then applies backend metadata before - adding the module's routes/nav/capabilities. - -Unsigned/unhashed remote bundles are skipped. This keeps remote loading a -controlled deployment option rather than a replacement for release package -builds. - -## Maintenance Mode - -Maintenance mode is the required operating state for package install/uninstall -and other disruptive system maintenance. - -Core stores maintenance mode in `system_settings.settings.maintenance_mode`. -The public platform status endpoint exposes only the flag and message so the -WebUI can show a clear login-screen notice. Login remains reachable during -maintenance so an operator can sign in. - -Authenticated API access is enforced at the access-principal boundary. When -maintenance mode is enabled, authenticated requests require the system scope -`system:maintenance:access`; otherwise the API returns `503 Service -Unavailable` with a maintenance-mode detail payload. The protected -`system_owner` role grants this through `system:*`. A dedicated -`maintenance_operator` role exists for accounts that should be able to access -the system during maintenance without receiving broad write permissions. - -Changing the maintenance-mode flag requires both system settings write access -and `system:maintenance:access`, so an administrator cannot accidentally enable -a mode they cannot use. - -The first implementation is a platform access gate. It does not replace -database backups, process supervision, migration checks, or external load -balancer maintenance pages. - -## Build And Verification - -Backend verification from core: - -```bash -cd /mnt/DATA/git/govoplan-core -/mnt/DATA/git/govoplan/.venv/bin/python -m compileall src/govoplan_core ../govoplan-access/src/govoplan_access ../govoplan-admin/src/govoplan_admin ../govoplan-tenancy/src/govoplan_tenancy ../govoplan-policy/src/govoplan_policy ../govoplan-audit/src/govoplan_audit ../govoplan-dashboard/src/govoplan_dashboard ../govoplan-files/src/govoplan_files ../govoplan-mail/src/govoplan_mail ../govoplan-campaign/src/govoplan_campaign -/mnt/DATA/git/govoplan/tools/checks/check_dependency_boundaries.py -``` - -`govoplan/tools/checks/check-focused.sh` runs npm with an isolated temporary npm user config -so developer-local npm settings do not create release-check warning noise. - -Focused module contract and permutation verification: - -```bash -cd /mnt/DATA/git/govoplan -GOVOPLAN_CORE_ROOT=/mnt/DATA/git/govoplan-core bash tools/checks/check-module-matrix.sh -``` - -Core WebUI host verification: - -```bash -cd /mnt/DATA/git/govoplan-core/webui -PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build -``` - -Clean generated `dist`, `.vite`, and source-tree `__pycache__` artifacts after verification unless they are intentionally part of a release artifact. - -## Release Dependency Rules - -Local development may use editable Python installs and local WebUI `file:` dependencies so sibling module changes reload quickly. Release builds must use tagged git refs or published packages instead. The meta repository provides: - -- `requirements-dev.txt` for local editable backend installs -- `requirements-release.txt` for tagged backend module installs -- `webui/package.release.json` for tagged WebUI module installs - -Module repositories include root-level npm manifests for git installs. When cutting a release, update the Python versions, WebUI versions, release dependency refs, and repository tags together. diff --git a/Repo-docs-MODULE-ARCHITECTURE.md b/Repo-docs-MODULE-ARCHITECTURE.md index 3ad593e..19ce5e4 100644 --- a/Repo-docs-MODULE-ARCHITECTURE.md +++ b/Repo-docs-MODULE-ARCHITECTURE.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`. > Origin: `repository`. @@ -214,12 +214,20 @@ Other stable runtime capabilities currently include: - `identity.directory` and `identity.search` - `organizations.directory` -- `idm.directory` -- `calendar.outbox` and `calendar.scheduling` +- `idm.directory`, `idm.function_assignments`, `idm.relationships`, and + `idm.assignment_lifecycle` +- `calendar.outbox`, `calendar.scheduling`, `calendar.invitations`, and + `calendar.externalProfiles` - `poll.scheduling` - `notifications.dispatch` - `workflow.definitionContributions` and `workflow.runtimeWorker` +The provider-neutral `idm.relationships` contract carries tenant-scoped typed +groups, effective-dated identity relationships, and explicit membership +decisions. It deliberately does not expose IDM persistence models or imply an +Access permission. Consumers can retain source revisions and inclusion or +exclusion provenance while remaining optional-module safe. + Modules contribute reusable process baselines through `ModuleManifest.workflow_definitions`. Each contribution pins its origin module and version, stable key, schema and content hash, native graph/BPMN content, @@ -292,8 +300,10 @@ such as `calendar:sync-source:`. Current named interfaces, generated from the source manifests by the workspace contract checks, are: -- `addresses.contact_writer`, `addresses.lookup`, `addresses.recipient_source` -- `calendar.outbox`, `calendar.scheduling` +- `addresses.contact_point_resolution`, `addresses.contact_writer`, + `addresses.lookup`, `addresses.recipient_source` +- `calendar.external_profiles`, `calendar.invitations`, `calendar.outbox`, + `calendar.scheduling` - `campaigns.access`, `campaigns.delivery_tasks`, `campaigns.mail_policy_context`, `campaigns.policy_context`, `campaigns.retention` @@ -735,6 +745,13 @@ effects, transitions partial/unknown outcomes honestly, and records verified completion or recovery. Plaintext secrets must never enter recovery metadata or evidence. +For a conclusive external result, modules may commit their local success +projection and the verified terminal checkpoint in one database transaction via +`DurableRecoveryOperation.commit_verified_success`. This does not make the +external provider effect atomic. It prevents a local `succeeded` state from +becoming authoritative when the recovery evidence chain is damaged or the +terminal checkpoint cannot commit. + ## Install, Uninstall, And Catalogs Core owns the install plan, signed catalog validation, license entitlement @@ -824,12 +841,34 @@ the shared loading and retryable error state around route rendering. The initial static import closure and largest asynchronous chunk are enforced by the budgets documented in [WEBUI_BUNDLE_BUDGETS.md](WEBUI_BUNDLE_BUDGETS.md). +Every public platform interface has a stable declaration identity. Backend +routes, capabilities, interfaces, search providers/sources, permissions, +frontend routes/navigation, and View surfaces derive that identity from typed +`ModuleManifest` values. Typed WebUI capabilities declare IDs for settings, +admin sections, widgets, search contexts, and extension actions. Shared form +and action controls accept `interfaceId` and `helpTopicId`; use module-namespaced +values when another contract, documentation topic, or automated check must +refer to the control across source changes. The static inventory assigns a +line-independent source anchor when an explicit ID is absent and reports that +fact for later review. + +Core exposes the sanitized runtime declaration set at +`GET /api/v1/platform/interface-catalog`. The endpoint is read-only, requires +`admin:module:read` or `system:settings:read`, and includes only modules +effective in the caller's active tenant context. It never serializes factories, +credentials, executable callbacks, or mutable module state. Registry validation +rejects conflicting declaration IDs before startup. + WebUI modules receive only the core route context: - `settings` - `auth` A module should call its own API client and module-owned backend routes. Shared API helpers should live in core only when they are truly platform-level concerns. +For ordinary JSON mutations, use Core's `apiPostJson` and `apiPatchJson` +helpers. They preserve the shared authentication, CSRF, error, and request +invalidation behavior while leaving endpoint types and feature semantics in the +owning module. Modules can also contribute named UI capabilities for explicit extension points. Capability values must be narrow, typed contracts, not imports from a @@ -1012,6 +1051,16 @@ Decision: templates and reporting are separate modules. and export targets - report permissions, report execution history, generated report evidence, and report-specific retention inputs + +Cross-module reports use Core's versioned +`reporting.report_provider.` contract. Source modules own +authorization, parameters, source revisions, effective scope, result schema, +and privacy transforms; Reporting owns discovery, validation, governed +execution, provenance, export history, and the global `/reports` route. The +optional `policy.reporting_governance` capability can only tighten execution, +retention, export, and re-identification-risk handling. Reporting exposes +`reporting.retention` so the Policy-owned retention run can minimize expired +provider results without importing Reporting models. - downstream export handoff to files, dataflow, connectors, or publication surfaces @@ -1104,7 +1153,7 @@ from workflow semantics. - form definitions, schemas, validation rules, field visibility rules, localization, versioning, admin editing, and reusable form package fragments -`govoplan-forms-runtime` owns, when implemented: +`govoplan-forms-runtime` owns: - public/internal submissions, drafts, submitted values, validation evidence, attachment references, submission receipts, and handoff events @@ -1117,6 +1166,16 @@ Boundary: - Reporting/dataflow may consume submitted data through governed DTOs or source lifecycle contracts. +Implemented contract: + +- Core owns the provider-neutral `FormDefinition`/`FormFieldDefinition` DTOs. +- Forms persists immutable exact definitions and provides `forms.definitions`. +- Forms Runtime resolves that capability, persists revisioned instances and + events, validates draft/final values, and provides + `forms_runtime.service_launcher`. +- Portal delegates exact `/` bindings and never writes either + owner's tables. + ### OpenDesk Integration Profile Tracking: `govoplan-core#195`, `govoplan-connectors#5`, @@ -1205,6 +1264,61 @@ devserver, development bootstrap, background worker registry, and migration metadata plan all read the saved desired state from `system_settings` before building their module registry. +### Tenant entitlement and personal visibility + +Deployment activation remains process-wide: one installed and active registry +is shared by every tenant served by that process. Tenant module selection is a +separate entitlement document in `core_scopes.settings.module_entitlements`: + +- a system policy marks each installed module `unavailable`, `available`, or + `forced` for one tenant; +- the tenant selection may enable or disable only available modules; +- protected platform modules, forced modules, and transitive dependencies stay + effective; +- malformed explicit entitlement fails closed to protected modules, while an + absent document preserves the pre-entitlement behavior for upgraded tenants; +- an optimistic revision prevents concurrent system and tenant administrators + from silently replacing each other's changes. + +The authenticated platform metadata and module route guard intersect global +runtime activation with the active tenant's effective entitlement. Entitlement +does not grant a permission. Access authorization must still allow every API +operation and resource. + +The same boundary applies outside authenticated request handling: + +- capability factories retain their owning module, and tenant-scoped capability + lookup treats a provider that is unavailable to the tenant as absent; +- workers partition scheduled scans by tenant before claiming rows; +- new work is rejected while a module is unavailable, while already accepted + durable work remains in provider-owned storage and is reported as + `operator_action_required` instead of being dropped or executed; +- Workflow, Dataflow, event consumers, reconciliation jobs, and external-effect + outboxes run inside a tenant execution context, so their optional capability + calls inherit the same provider checks; +- public signed-link modules declare a `public_tenant_resolver`; valid token + context is resolved before the route runs and the module entitlement is then + enforced without requiring an authenticated principal. + +Entitlement resolution uses a bounded process-local cache. A local policy +mutation invalidates its tenant entry immediately; changes made by another node +become authoritative after `TENANT_MODULE_ENTITLEMENT_CACHE_TTL_SECONDS` +(five seconds by default). This is a bounded staleness optimization, not an +authorization grant: a cache miss or resolution failure fails closed. + +Users and groups do not own another module-runtime state. Every WebUI module +already contributes a root `.module` View surface, so personal and +group module visibility is expressed through Views. View policy controls who +may select, assign, edit, derive, or workflow-activate those projections; +required View assignments can retain required UI. Thus tenant entitlement owns +operational availability, Views own presentation, and Access owns authority. + +Capability-style modules such as Encryption must keep activation separate from +domain data state. Making Encryption effective only exposes its capability and +administration surfaces. Encrypting, rekeying, decrypting, or migrating data is +an explicit versioned protection-policy operation owned by Encryption and the +module that owns the data. + Hot enable/disable is a core design principle for every module: - Core keeps one mutable active `PlatformRegistry` object and swaps its manifest @@ -1312,6 +1426,11 @@ The package install-plan API records operator intent only: default; successful uninstalls are removed from saved startup state by default. Use `--no-activate-installed-modules` or `--keep-uninstalled-modules-in-desired` only for staged rollout workflows. +- Every non-dry installer and live active-graph mutation acquires the + deployment-wide `core:module-lifecycle:deployment` lease and records a Core + recovery operation. Unresolved effects block later lifecycle changes. The + operation modes and operator reconciliation contract are defined in + `MODULE_LIFECYCLE_RECOVERY.md`. - `govoplan-module-installer --supervise --migrate --health-url http://127.0.0.1:8000/health --restart-command ''` is the preferred disruptive-change path. It applies the plan, optionally runs migrations in a fresh Python process after a fresh-process manifest @@ -1464,6 +1583,22 @@ The first implementation is a platform access gate. It does not replace database backups, process supervision, migration checks, or external load balancer maintenance pages. +## Connector Runtime Contract + +Core defines provider-neutral connector preview and diagnostic primitives in +`govoplan_core.core.connector_runtime`. The contract keeps optional modules +decoupled: Connectors owns transport, endpoint discovery, retries, and protocol +health; the consuming domain module owns mappings, validation, reconciliation, +and mutations of its records. + +Every dry run is bounded and identifies the source revision, source fingerprint, +immutable input hash, effects, and redacted diagnostics. Its summary must match +the returned effect list exactly. An apply token is usable only when the preview +is complete, current, conflict-free, and contains no error diagnostic. Endpoint +URLs never contain credentials; only credential-envelope references cross the +contract. Provider-specific details belong in sanitized provenance rather than +in a shared domain schema. + ## Build And Verification Backend verification from core: diff --git a/Repo-docs-MODULE-LIFECYCLE-RECOVERY.md b/Repo-docs-MODULE-LIFECYCLE-RECOVERY.md new file mode 100644 index 0000000..3621659 --- /dev/null +++ b/Repo-docs-MODULE-LIFECYCLE-RECOVERY.md @@ -0,0 +1,73 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_LIFECYCLE_RECOVERY.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Module Lifecycle Recovery + +## Migration Revision Namespace + +All enabled module migration directories are assembled into one Alembic graph. Revision IDs are therefore global across Core and every module even though each module owns a separate `migrations/versions` directory. Core validates literal revision declarations before constructing the graph and rejects duplicates with both file paths. A module must assign a new globally unique revision ID; reusing another module's ID can otherwise make Alembic treat an unrelated schema change as already applied or report an ancestor/head overlap. + +When correcting a collision that has already reached a database, first verify the schema objects that identify which migration actually ran. Rename the unapplied migration, or transactionally translate the corresponding `alembic_version` row when the applied owner is unambiguous. Never add both colliding IDs as heads or blindly stamp the database. + +Package changes and live module-graph changes use Core's durable recovery +ledger. The local `install.lock` still prevents duplicate work in one runtime +directory; the database lease `core:module-lifecycle:deployment` is the +deployment-wide authority across API, installer, worker, and scheduler nodes. + +## Declared Boundaries + +| Operation | Recovery mode | Completion condition | +| --- | --- | --- | +| `module-lifecycle.pre-migration` | compensation | package, WebUI, manifest, and desired-graph evidence match | +| `module-lifecycle.post-migration` | forward recovery | migration tasks, manifests, desired graph, restart, and health are verified | +| `module-retirement.destroy-data` | snapshot restore | a hashed, restore-checked backup exists and retirement state is verified | +| `module-runtime.apply-graph` | compensation | hooks, capability contexts, active graph, and workflow contributions match | + +The installer prepares the recovery operation before it captures the database +snapshot. A full database restore therefore retains the prepared operation and +its fence instead of erasing the fact that a mutation was attempted. Backup +artifacts are hashed and sized before any package, migration, or retirement +effect starts. + +Every command boundary records the command source and canonical hashes of the +redacted command/result records. Credentials, database URLs, command output, +and package-registry secrets are never copied into recovery evidence. + +## Failure And Retry Rules + +- A conclusive failure before effects is terminal `failed`. +- A command or compensatable effect that started but did not complete is + `recovery_required`. +- A lost or unexpected outcome after a migration/external boundary is + `outcome_unknown`. +- A verified package/database rollback becomes `recovered`. +- A supervised install becomes `succeeded` only after restart and all configured + health probes succeed. + +An unresolved lifecycle operation blocks every later lifecycle mutation on the +same deployment fence, even after its execution lease is released. Operators +must inspect the checkpoint chain and run record, restore or complete the +declared recovery path, and explicitly reconcile the operation. A new install +must not be used as an implicit retry. + +Live graph changes use the same fence. A non-migrating hook or registry failure +restores the prior in-process graph and records verified compensation. A failure +after migrations begin remains unresolved because restoring the process-local +registry does not reverse database schema effects. + +## Operator Evidence + +The installer run record contains the recovery operation id, mode, plan hash, +and current lifecycle status. The Ops recovery view is authoritative for the +durable state and evidence-chain result. Keep both the run directory and the +state-service backup evidence until the operation is terminal and the normal +retention policy permits removal. + +Run the module installer rollback drill and recovery-runtime test matrix before +enabling lifecycle mutation in a new deployment. Shared-state deployments must +still use immutable release images; the ledger does not make in-place package +mutation across replicas safe. diff --git a/Repo-docs-RELEASE-DEPENDENCIES.-.md b/Repo-docs-RELEASE-DEPENDENCIES.-.md deleted file mode 100644 index c0a3aa8..0000000 --- a/Repo-docs-RELEASE-DEPENDENCIES.-.md +++ /dev/null @@ -1,629 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# GovOPlaN Release Dependencies - -This document owns release package composition, signed package catalogs, -license checks, catalog publishing, migration baselines, and the final release -checklist. - -Operator runtime configuration and module install/uninstall execution live in -`DEPLOYMENT_OPERATOR_GUIDE.md`. - -## Backend Packages - -Release installs must not depend on sibling checkout paths. Local development -can keep editable installs and `file:` WebUI links, but release packaging must -resolve modules from tagged git refs or from a package registry. - -Local development: - -```bash -cd /mnt/DATA/git/govoplan-core -./.venv/bin/python -m pip install -r requirements-dev.txt -``` - -Release install from a core checkout plus tagged module repositories: - -```bash -cd /mnt/DATA/git/govoplan-core -./.venv/bin/python -m pip install -r requirements-release.txt -``` - -`.[server]` is resolved relative to the current working directory. If you -create the virtualenv elsewhere, still run the install command from the core -checkout: - -```bash -cd /mnt/DATA/git/govoplan-core -/tmp/govoplan-release-test/bin/python -m pip install -r requirements-release.txt -``` - -`requirements-release.txt` pins the module repositories to the release tag. -Update those refs when cutting a release: - -```text -govoplan-access git@git.add-ideas.de:add-ideas/govoplan-access.git v0.1.6 -govoplan-admin git@git.add-ideas.de:add-ideas/govoplan-admin.git v0.1.6 -govoplan-tenancy git@git.add-ideas.de:add-ideas/govoplan-tenancy.git v0.1.6 -govoplan-policy git@git.add-ideas.de:add-ideas/govoplan-policy.git v0.1.6 -govoplan-audit git@git.add-ideas.de:add-ideas/govoplan-audit.git v0.1.6 -govoplan-files git@git.add-ideas.de:add-ideas/govoplan-files.git v0.1.6 -govoplan-mail git@git.add-ideas.de:add-ideas/govoplan-mail.git v0.1.6 -govoplan-campaign git@git.add-ideas.de:add-ideas/govoplan-campaign.git v0.1.6 -govoplan-calendar git@git.add-ideas.de:add-ideas/govoplan-calendar.git v0.1.6 -``` - -## WebUI Packages - -Local development uses `webui/package.json`, which may point at sibling module -checkouts while active development is happening. - -Release WebUI installs should use `webui/package.release.json`. It points -module dependencies at the same tagged git repositories. After the module tags -referenced there exist, generate the committed release lockfile without -touching the development package files: - -```bash -cd /mnt/DATA/git/govoplan-core -scripts/generate-release-lock.sh -cd webui -PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build -``` - -The module repositories include root-level npm package manifests so git -installs can resolve `@govoplan/access-webui`, `@govoplan/admin-webui`, -`@govoplan/files-webui`, `@govoplan/mail-webui`, -`@govoplan/campaign-webui`, and `@govoplan/calendar-webui` from repository -roots even though their source lives below `webui/src`. - -### Release Lockfile Strategy - -The supported release composition currently is the full GovOPlaN product: core -plus access, admin, tenancy, policy, audit, files, mail, campaign, and -calendar. Keep one committed full-product release lockfile at -`webui/package-lock.release.json`, generated from -`webui/package.release.json` in a clean release workspace. Development -`package-lock.json` may continue to point at local `file:` dependencies. - -Frontend module permutations are regression-tested through -`GOVOPLAN_WEBUI_MODULE_PACKAGES` and temporary build output, not through -committed lockfiles for every possible combination. If a smaller composition -becomes a separately shipped product, add an explicit release manifest and -lockfile pair for that product, for example -`package.release.files-mail.json` and `package-lock.release.files-mail.json`, -generated in a clean release workspace from tagged git dependencies. - -## Release Tag Script - -The normal release path is automated by `scripts/push-release-tag.sh`: it bumps -or accepts the target version, updates Python/WebUI/module manifest versions, -commits/tags/pushes the module repositories first, regenerates -`webui/package-lock.release.json`, and then commits/tags/pushes core. If the -working tree has already been bumped, pass the current version explicitly: - -```bash -cd /mnt/DATA/git/govoplan-core -scripts/push-release-tag.sh --version 0.1.6 -``` - -The script also includes GovOPlaN roadmap/scaffold module repositories that do -not yet have package metadata. Those repositories are committed, tagged, and -pushed with the same release tag, but they are tag-only until they contain -`pyproject.toml`, module manifests, or WebUI packages. Tag-only repositories -are not listed in `requirements-release.txt` or `webui/package.release.json`. - -Current tag-only module repositories: - -- `govoplan-addresses` -- `govoplan-appointments` -- `govoplan-cases` -- `govoplan-connectors` -- `govoplan-dms` -- `govoplan-erp` -- `govoplan-fit-connect` -- `govoplan-forms` -- `govoplan-identity-trust` -- `govoplan-idm` -- `govoplan-ledger` -- `govoplan-notifications` -- `govoplan-ops` -- `govoplan-payments` -- `govoplan-portal` -- `govoplan-reporting` -- `govoplan-scheduling` -- `govoplan-search` -- `govoplan-tasks` -- `govoplan-templates` -- `govoplan-workflow` -- `govoplan-xoev` -- `govoplan-xrechnung` -- `govoplan-xta-osci` - -## Catalog Trust And Licensing - -GovOPlaN module install and uninstall must remain operator-controlled. The -running server may plan and validate package changes, but package mutation is -performed by the separate installer daemon or an operator shell during -maintenance mode. - -`govoplan-web` is the public static distribution surface for official catalog -resources: - -- signed module package catalogs, grouped by release channel -- public catalog keyrings -- public license verification keyrings -- examples and operator-facing download paths - -`govoplan-core` is the verifier and orchestrator: - -- fetches a local or remote module catalog -- verifies catalog signatures against configured trusted keys -- enforces approved release channels -- rejects expired or not-yet-valid catalogs -- records accepted catalog sequence numbers for replay protection -- checks catalog entry license feature requirements before planning installs -- writes installer plans and request records - -Feature and platform modules own their package artifacts, manifests, migration -metadata, retirement providers, and optional lifecycle behavior. - -Core accepts either a local catalog file or a remote URL: - -```bash -GOVOPLAN_MODULE_PACKAGE_CATALOG=/srv/govoplan/catalogs/stable.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.example/catalogs/v1/channels/stable.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/catalog-cache/stable.json -``` - -If both file and URL are set, the URL wins. The cache is used when a remote -fetch fails, so an operator can still inspect the last known catalog. A cached -catalog must still pass signature, freshness, channel, and replay validation. - -An official catalog is a JSON object with: - -- `catalog_version` -- `channel` -- `sequence` -- `generated_at` -- `not_before` when delayed activation is needed -- `expires_at` -- `modules` -- `signatures` - -Each module entry can declare: - -- backend package name and pinned install reference -- WebUI package name and pinned install reference -- display metadata and tags -- `license_features`, the feature entitlements required to plan that install - -The signature is Ed25519 over canonical JSON with both `signature` and -`signatures` removed. Core accepts the legacy single `signature` field and the -new `signatures` array. - -Trusted catalog keys are configured locally: - -```bash -GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/catalog-keyring.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS='{"release-key-1":""}' -``` - -For development or tightly controlled deployments, a keyring can be read from a -URL and cached: - -```bash -GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_URL=https://govoplan.example/catalogs/v1/keyring.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_CACHE=/srv/govoplan/runtime/catalog-cache/keyring.json -``` - -Production installations should pin the trusted keyring locally or ship it -through deployment configuration. Fetching trusted keys from the same public -origin as the catalog is convenient, but that origin must not become the only -trust root. - -## Dependency Audits - -Dependency vulnerability checks are documented in -[`DEPENDENCY_AUDITS.md`](DEPENDENCY_AUDITS.md). The local audit runner is: - -```bash -cd /mnt/DATA/git/govoplan-core -bash scripts/check-dependency-audits.sh -``` - -The Gitea workflow in `.gitea/workflows/dependency-audit.yml` runs the same -check against release dependency refs on pushes, pull requests, and a weekly -schedule. - -Keyring entries support: - -- `key_id` -- `public_key` or `public_key_base64` -- `status`: `active`, `next`, `retired`, `revoked`, or `disabled` -- `not_before` -- `not_after` - -Rotation process: - -1. Add the next public key to the local trusted keyring with status `next`. -2. Publish catalogs signed by both current and next keys. -3. Upgrade installations so the next key is locally trusted. -4. Promote the next key to `active`. -5. Retire the old key only after every supported installation trusts the new - key. -6. Mark a compromised key `revoked` and publish a higher sequence catalog - signed by an uncompromised key. - -Use replay state in production: - -```bash -GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-sequences.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true -``` - -Core records the accepted sequence per channel after a catalog entry is planned -from the admin interface. With strict sequence enforcement, a previously -accepted sequence is rejected; without strict enforcement, only older sequences -are rejected. Catalogs should always expire. - -The sequence state file is operational state, not a trust root. Keep it on -persistent storage and include it in normal backups: - -```json -{ - "channels": { - "stable": { - "last_sequence": 42, - "accepted_at": "2026-07-07T12:00:00Z", - "key_id": "release-key-1", - "source": "https://govoplan.example/catalogs/v1/channels/stable.json" - } - } -} -``` - -If the file is lost, restore it from backup. If no backup exists, reconstruct -each channel from the highest sequence already accepted in installer run -records, release records, or the currently deployed module package set. Do not -lower `last_sequence` to make an older catalog pass; publish a new higher -sequence catalog when the accepted point is uncertain. - -If the file is corrupted, copy it aside for incident review, validate the -current signed catalog with channel and freshness enforcement, then rewrite the -state with the known accepted sequence. Keep -`GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true` and approved-channel -checks enabled during recovery. Temporarily disabling -`GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE` allows revalidating the same -sequence, but older sequences remain rejected once the reconstructed -`last_sequence` is in place. - -Approved channels are deployment policy: - -```bash -GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable,lts -GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true -``` - -The admin UI can display other catalog metadata, but core rejects catalogs from -unapproved channels when validation is configured. - -Catalog entries can require license features: - -```json -"license_features": ["module.mail", "support.standard"] -``` - -Core checks those requirements against an offline license file before allowing -the entry into the install plan. - -```bash -GOVOPLAN_LICENSE_FILE=/srv/govoplan/license.json -GOVOPLAN_LICENSE_ENFORCEMENT=true -GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE=/srv/govoplan/trust/license-keyring.json -``` - -License files are JSON objects with: - -- `license_id` -- `subject` -- `features` -- `valid_from` -- `valid_until` -- `signature` - -Issue or renew a license from an operator/release shell that has the Ed25519 -private key: - -```bash -govoplan-module-installer \ - --issue-license /srv/govoplan/license.json \ - --license-id customer-2026-07 \ - --license-subject "Example Municipality" \ - --license-feature module.mail \ - --license-feature support.standard \ - --license-valid-until 2027-07-31T23:59:59Z \ - --license-signing-key-id license-issuer-1 \ - --license-signing-private-key /srv/govoplan/secrets/license-issuer-1.pem \ - --format json -``` - -Validate an imported license without exposing secrets: - -```bash -govoplan-module-installer \ - --validate-license /srv/govoplan/license.json \ - --license-trusted-key license-issuer-1="" \ - --require-trusted-license \ - --license-required-feature module.mail \ - --format json -``` - -The CLI and admin module catalog panel report the license id, subject, -validity window, signing key id, signed/trusted state, available features, and -missing entitlements for the configured package catalog. They do not expose -private signing material. - -License enforcement can run in observe-only mode by leaving -`GOVOPLAN_LICENSE_ENFORCEMENT` unset. In that mode, missing or invalid license -data is surfaced as a warning but does not block planning. - -Renewal is an ordinary re-issuance with a new `license_id`, extended -`valid_until`, and the full intended feature set. Import the renewed JSON to -`GOVOPLAN_LICENSE_FILE`, keep the previous file for audit, and validate it -before setting enforcement. - -Revocation is handled through the trusted license keyring. Mark a compromised -or invalid issuer key as `revoked` or `disabled`, publish or deploy the updated -keyring, then reissue affected licenses with an active key. Installations that -run with `GOVOPLAN_LICENSE_ENFORCEMENT=true` reject licenses signed only by a -revoked key after the local keyring is updated. - -Emergency fallback is deliberately explicit. Operators can temporarily unset -`GOVOPLAN_LICENSE_ENFORCEMENT` to keep package planning observable while a -license or keyring is recovered. Record the change in the operational incident -log, keep catalog signature and channel enforcement enabled, and restore -license enforcement after a trusted renewal validates successfully. - -Licensing is intentionally separate from open-source code licensing. The -catalog/license mechanism can govern support channels, official release -eligibility, hosted update access, professional support, or commercial -entitlements without changing the source license of the repositories. - -Production-grade distribution still needs remote registry/git artifact -resolution before package-manager apply, a hardened catalog publishing pipeline -in `govoplan-web`, and automated key rotation and emergency revocation drills. - -## Release Catalog Publishing - -GovOPlaN release catalogs are published by `govoplan-web` as static JSON and -verified by `govoplan-core` before installer plans are accepted. Private signing -keys must stay outside all git repositories. Public keyrings are published with -the website. - -Create the first catalog signing key on the release machine: - -```bash -cd /mnt/DATA/git/govoplan-core -KEY_DIR="$HOME/.config/govoplan/release-keys" -mkdir -p "$KEY_DIR" -./.venv/bin/python scripts/generate-catalog-keypair.py \ - --key-id release-key-1 \ - --private-key "$KEY_DIR/release-key-1.pem" \ - --public-key "$KEY_DIR/release-key-1.pub" \ - --keyring "$KEY_DIR/catalog-keyring.json" -``` - -Keep `release-key-1.pem` private. The generated keyring contains only public -material. - -Generate the signed catalog into `govoplan-web`: - -```bash -cd /mnt/DATA/git/govoplan-core -KEY_DIR="$HOME/.config/govoplan/release-keys" -scripts/publish-release-catalog.sh \ - --version \ - --sequence 202607071340 \ - --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ - --build-web -``` - -This writes: - -- `/mnt/DATA/git/govoplan-web/public/catalogs/v1/channels/stable.json` -- `/mnt/DATA/git/govoplan-web/public/catalogs/v1/keyring.json` - -The wrapper validates the catalog with core using the generated public keyring. - -For normal module/core releases, first audit and record migration baselines, -then tag and push the module/core repos. Finally publish the website catalog: - -```bash -cd /mnt/DATA/git/govoplan-core -./.venv/bin/python scripts/release-migration-audit.py --strict -``` - -```bash -cd /mnt/DATA/git/govoplan-core -KEY_DIR="$HOME/.config/govoplan/release-keys" -scripts/publish-release-catalog.sh \ - --version \ - --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ - --build-web \ - --commit \ - --tag \ - --push -``` - -The website tag is `catalog-v`. The public URL is: - -```text -https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json -``` - -The public keyring URL is: - -```text -https://govoplan.add-ideas.de/catalogs/v1/keyring.json -``` - -`scripts/push-release-tag.sh` can publish the web catalog after module and core -tags have been pushed. It runs the migration release audit in automatic mode: -warning-only before the first recorded migration baseline, strict after a -baseline exists. Add `--strict-migration-audit` when you want to force strict -mode explicitly: - -```bash -cd /mnt/DATA/git/govoplan-core -KEY_DIR="$HOME/.config/govoplan/release-keys" -scripts/push-release-tag.sh \ - --bump subversion \ - --strict-migration-audit \ - --publish-web-catalog \ - --catalog-signing-key "release-key-1=$KEY_DIR/release-key-1.pem" \ - --build-web-catalog -``` - -Use `--catalog-signing-key` more than once during a key rotation window. The -catalog will contain multiple signatures and the public keyring will include the -corresponding public keys. - -On a GovOPlaN installation that should consume the official stable catalog: - -```bash -GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_CACHE=/srv/govoplan/runtime/catalog-cache/stable.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_REQUIRE_SIGNATURE=true -GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable -GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/srv/govoplan/trust/catalog-keyring.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_SEQUENCE_STATE=/srv/govoplan/runtime/catalog-sequences.json -GOVOPLAN_MODULE_PACKAGE_CATALOG_ENFORCE_SEQUENCE=true -``` - -For production, copy the public keyring into deployment configuration and pin it -locally. Do not rely on a URL-fetched keyring as the only trust root. - -`stable.json` includes a top-level `core_release` section for operator/update -tooling. Core is intentionally not listed as a normal module entry because it -must not be added to saved enabled-module state. Core upgrades should remain an -operator-supervised package update with restart and health checks. - -Key rotation for published catalogs: - -1. Generate the next private key outside git. -2. Run `publish-release-catalog.sh` with both signing keys. -3. Publish the web catalog/keyring. -4. Roll the new public keyring into installations. -5. Stop signing with the old key after the supported fleet trusts the new key. -6. Mark compromised keys as revoked in the public keyring and publish a higher - sequence catalog signed by a trusted uncompromised key. - -## PostgreSQL Release Check - -Release candidates should pass a disposable PostgreSQL migration and startup -smoke check before tagging or publishing catalogs. Start the local testbed, -then run the permutation check from the core checkout: - -```bash -cd /mnt/DATA/git/govoplan-core/dev/postgres -cp .env.example .env -docker compose --env-file .env up -d - -cd /mnt/DATA/git/govoplan-core -set -a -. dev/postgres/.env -set +a -./.venv/bin/python scripts/postgres-integration-check.py \ - --database-url "$GOVOPLAN_POSTGRES_DATABASE_URL" \ - --reset-schema -``` - -The script checks migrations and `/health` startup for core-only, files-only, -mail-only, campaign-only, campaign+files, campaign+mail, and full-product -module sets. `--reset-schema` is destructive and must only be used against a -throwaway database. - -## Migration Baselines - -Development migrations may be small and numerous while a feature is moving. -Before a stable release, unreleased migrations may be rewritten or squashed into -a release-level baseline or release-to-release upgrade migration. After a -release tag has shipped, released migration revision IDs are immutable. - -The release policy is: - -- unreleased migrations may be folded before release; -- released migrations are never rewritten or deleted; -- each stable release records the public migration head revisions in - `docs/migration-release-baselines.json`; -- fresh installations should apply release-level baselines/upgrades, not - unreleased create-then-rename churn; -- release-to-release schema changes should be folded into one reviewed - migration per migration owner where practical. - -Audit the current graph during release preparation: - -```bash -cd /mnt/DATA/git/govoplan-core -./.venv/bin/python scripts/release-migration-audit.py -``` - -Generate the reviewed/manual squash checklist: - -```bash -./.venv/bin/python scripts/release-migration-audit.py --squash-plan -``` - -After the release migrations have been reviewed and the graph is final, record -the release baseline: - -```bash -./.venv/bin/python scripts/release-migration-audit.py --record-release -``` - -Use strict mode to verify that the current heads are recorded: - -```bash -./.venv/bin/python scripts/release-migration-audit.py --strict -``` - -`scripts/push-release-tag.sh` runs the audit by default in automatic mode: -non-strict while no release baseline exists, strict after the first baseline is -recorded. Pass `--warn-migration-audit` for an explicit non-strict audit, -`--strict-migration-audit` to force strict mode, or `--skip-migration-audit` -only for emergency/manual release work. - -Before the first stable release, fold the current development chain into the -first public baseline and record that baseline in -`docs/migration-release-baselines.json`. The tracking issue is -`add-ideas/govoplan-core#223`. - -## Related Operator Documents - -- `DEPLOYMENT_OPERATOR_GUIDE.md`: runtime environment, explicit migrations, - backup/restore commands, module installer daemon/supervisor operation, and - rollback drills. -- `REMOTE_WEBUI_BUNDLES.md`: experimental browser-loaded module bundles for - controlled deployments; normal releases use package builds. - -## Release Checklist - -- Keep Python package versions, WebUI package versions, and git tags aligned. -- Tag core, access, admin, tenancy, policy, audit, files, mail, campaign, - calendar, and scaffold module repositories together. -- Update `requirements-release.txt` and `webui/package.release.json` when the - release tag changes. -- Generate the committed full-product release lockfile from - `package.release.json` with `scripts/generate-release-lock.sh`. -- Run `scripts/release-migration-audit.py --strict` after recording a release - baseline. -- Run the PostgreSQL release check against a disposable database. -- Publish the signed catalog through the release catalog publishing flow above. -- Add separate release manifest/lockfile pairs only for module compositions - that are shipped as their own products. -- Do not commit local sibling paths into release manifests. diff --git a/Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT.md b/Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT.md new file mode 100644 index 0000000..f2b3224 --- /dev/null +++ b/Repo-docs-SEARCH-EVENT-INDEXING-CONTRACT.md @@ -0,0 +1,33 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/SEARCH_EVENT_INDEXING_CONTRACT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Search event indexing contract + +Core defines, but does not implement, the optional Search indexing boundary. +Feature modules register `SearchSourceProvider` implementations for bounded +backfills and live authorization checks. A provider may additionally implement +`SearchEventSourceProvider` to translate a committed `PlatformEvent` into one +or more authoritative `SearchIndexChange` values. + +When the Search index-writer capability is active, the platform event worker +uses the durable consumer identity `search.indexing.v1`. It accepts only public +and internal events, passes the outbox delivery key to each event-capable +source, and then advances a bounded batch of queued index changes in the same +worker transaction. Stable change IDs make delivery replay idempotent. + +The boundary has three non-negotiable rules: + +- a source may emit changes only for its registered module, provider, resource + type, and event tenant; +- Search validates every upsert document before queueing it and rejects secret + metadata keys; +- an index ACL is only a candidate filter. Resources marked for authorization + recheck are returned only after the owning source explicitly allows the + current principal at query time. + +Search and its worker remain optional. Core-only startup and feature-module +operation do not require the Search package. diff --git a/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md b/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md index bd336b4..00f03d6 100644 --- a/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md +++ b/Repo-docs-STATE-AND-RECOVERY-CONTRACT.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/STATE_AND_RECOVERY_CONTRACT.md`. > Origin: `repository`. @@ -35,6 +35,11 @@ module artifacts. It provides bounded read/write/list/stat/delete operations for local and S3-compatible storage. Modules own their object-key namespace and business metadata; Core does not interpret module files. +`stat` and `list_objects` return object size plus a UTC `modified_at` value when +the backend can prove it. Reconciliation and retention code may use that value +for conservative grace periods, but must treat a missing timestamp as +ineligible for automatic deletion rather than guessing an age. + Rules for modules: - Store only opaque object keys in business records, never local absolute @@ -59,6 +64,13 @@ software version, module-composition hash, queues, start time, and heartbeat. The registration identity includes a process incarnation so a stale process cannot update a replacement's row. +Worker metadata also records the orchestrator pool and declared concurrency. +Every Celery prefork child disposes the SQLAlchemy pool inherited from its +parent and creates a process-local pool before handling work. Deployment +rendering must therefore budget one database pool for the worker parent and +each child. Ops compares active queue ownership, software versions, and the +order-independent module-composition hash with the graph loaded by the API. + Drain is durable operator intent: - an API enters not-ready state after observing drain; diff --git a/Repo-docs-TABULAR-SOURCE-CONTRACT.md b/Repo-docs-TABULAR-SOURCE-CONTRACT.md new file mode 100644 index 0000000..2b1466c --- /dev/null +++ b/Repo-docs-TABULAR-SOURCE-CONTRACT.md @@ -0,0 +1,28 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/TABULAR_SOURCE_CONTRACT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Tabular Source Preview Contract + +Core defines provider-neutral DTOs for optional tabular source providers. A +source declares whether it is live, cached, file-backed, or static; its schema +and immutable fingerprint; structured health; and the exact projection, +pagination, filter, aggregation, and sorting operations that the provider can +push down. Consumers must not infer pushdown support from a provider name. + +Every preview request carries independent row, byte, and elapsed-time budgets. +A provider may tighten these values but must return its effective limits, +returned byte count, elapsed milliseconds, truncation state, and structured +diagnostics. Equivalent fields on the Datasources read request and result +preserve that evidence when a live source is consumed through the catalogue. +A row that cannot fit within the byte budget fails explicitly rather than +leaking a partial value. Timeout, stale fingerprint, unavailable source, and +authorization failures remain distinct provider-neutral errors. + +Connector health and preview diagnostics must contain no credentials, endpoint +userinfo, row values, or unbounded remote error bodies. A Datasource origin +preserves this contract so registration and staging do not erase source mode, +health, pushdown, or preview-limit evidence. diff --git a/Repo-docs-TEMPLATE-CAPABILITY-CONTRACT.md b/Repo-docs-TEMPLATE-CAPABILITY-CONTRACT.md new file mode 100644 index 0000000..ec9ba0e --- /dev/null +++ b/Repo-docs-TEMPLATE-CAPABILITY-CONTRACT.md @@ -0,0 +1,35 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/TEMPLATE_CAPABILITY_CONTRACT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Template And Generated Artifact Capability Contracts + +Core defines provider-neutral contracts for optional template libraries and +generated artifact storage. Core does not render templates or store generated +files itself. + +## Templates + +- `templates.catalog` lists typed, versioned template references and checks a + consumer's available fields, usage, and output format. +- `templates.renderer` accepts a `TemplateRenderRequest` containing pinned + input data and returns immutable render evidence plus an artifact reference. + +The DTOs contain identifiers, hashes, plain mappings, and scalar metadata. They +do not expose Template ORM models or require Campaign, Distribution Lists, +Addresses, Reporting, Forms, or Mail. + +## Generated Artifacts + +`files.artifact_store` accepts a `ManagedArtifactWriteRequest` and returns a +`ManagedArtifactRef`. Producers supply bytes, a safe filename/content type, +idempotency key, and non-secret provenance. Files owns path normalization, +authorization, versions, storage, and download behavior. + +Consumers must discover both contracts through the module registry and degrade +only the unavailable path. A template renderer may return a bounded download +when Files is absent. A caller must not infer successful external delivery from +successful rendering or artifact persistence. diff --git a/Repo-docs-TEMPORAL-DATA-CONTEXT.md b/Repo-docs-TEMPORAL-DATA-CONTEXT.md new file mode 100644 index 0000000..831fdc7 --- /dev/null +++ b/Repo-docs-TEMPORAL-DATA-CONTEXT.md @@ -0,0 +1,75 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/TEMPORAL_DATA_CONTEXT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Temporal Data Context + +GovOPlaN exposes one read context for data validity and system knowledge. The +calendar control in the authenticated titlebar applies that context to +supported list and detail reads for the current account and tenant. + +## Two Independent Axes + +- **Valid time** answers when a fact applied in the represented domain. +- **Recorded time** answers what the system had recorded by a particular + instant. + +The default is data valid now under the latest recorded state. `At time` +selects a valid-time instant. `All` removes the valid-time interval filter but +still uses the selected recorded state. The optional recorded-state cutoff can +be combined with any valid-time mode, which keeps correction history distinct +from changes in real-world validity. + +An interval is half open: `valid_from <= instant < valid_to`. A revision belongs +to a recorded-state snapshot when `recorded_at <= cutoff` and it was not +superseded at or before that cutoff. + +## Security And Mutation Rules + +The temporal data context is a read projection, not an authorization context. +Authentication, permissions, active delegations, tenant boundaries, module +policy, and maintenance controls are always evaluated under current security +state. A historical projection never restores an expired permission. + +The context also does not supply mutation dates. Writes continue to target the +current lifecycle revision and must carry their explicit valid/effective dates, +expected revision, reason, and evidence where the owning contract requires +them. A screen showing historical data must not silently turn a normal edit +into a historical correction. + +## HTTP Contract + +Core accepts these request headers: + +| Header | Meaning | +| --- | --- | +| `X-Govoplan-Validity-Mode` | `current`, `at`, or `all` | +| `X-Govoplan-Valid-At` | Timezone-aware ISO 8601 instant required by `at` | +| `X-Govoplan-Recorded-At` | Optional timezone-aware system-knowledge cutoff | + +Invalid or naive timestamps fail with HTTP 400. Responses expose the resolved +mode and evaluated instant. Conditional JSON responses vary by all three +request headers, and the shared WebUI API client includes them in request +deduplication and conditional-cache keys. + +## Module Adoption + +Revision-owning modules apply +`govoplan_core.db.temporal.apply_temporal_revision_filter` only to read queries +that are meant to follow the platform context. Explicit version references and +explicit resolver `effective_at` arguments take precedence. Current-row +lookups used for optimistic concurrency, authorization, routing, effects, or +other mutations must remain explicit and context-independent. + +The initial bitemporal adoption covers Decisions, Mandates, Parties, and +Services. Their immutable revisions have indexed valid, recorded, and +superseded timestamps. Modules with effective-dated security records or +recorded-only revision histories require separate display-query adoption so +the global selector cannot affect current authorization or execution. + +The WebUI selection is stored in session storage per account and tenant. A +change remounts the active module route so existing page loaders issue a fresh +request. Returning both axes to their defaults removes the stored selection. diff --git a/Repo-docs-UI-UX-DECISION-LEDGER.md b/Repo-docs-UI-UX-DECISION-LEDGER.md index 3dc157e..340a11c 100644 --- a/Repo-docs-UI-UX-DECISION-LEDGER.md +++ b/Repo-docs-UI-UX-DECISION-LEDGER.md @@ -1,4 +1,4 @@ - + > Mirrored from `/mnt/DATA/git/govoplan-core/docs/UI_UX_DECISION_LEDGER.md`. > Origin: `repository`. @@ -57,6 +57,13 @@ contestability, responsibility, and traceability at the point of action. | UX-024 | Explicit `Discard` actions and dirty in-application navigation use the shared `UnsavedChangesProvider` dialog. A page registers save/discard behavior with `useUnsavedDraftGuard`; its Discard button calls `requestDiscard`, and route changes use `useGuardedNavigate` or `requestNavigation`. | Accepted | All create/edit surfaces | | UX-025 | `window.alert` and the global `alert` function are prohibited. A narrowly necessary exception requires product-owner authorization and an entry in the alert exception register before implementation. | Accepted | All WebUI code | | UX-026 | A table defines one stable ordered action set. A row-level unavailable action remains in its normal position and is disabled, preferably with `disabledReason`; structurally irrelevant actions are omitted for the entire table. Empty rows reserve the same slots so their Add action stays in the normal left-most action position. | Accepted | All structured tables | +| UX-027 | The platform icon rail keeps its brand header and utility footer visible. Only the module-navigation region scrolls when installed and permitted modules exceed the available viewport height. | Accepted | Core WebUI shell | +| UX-028 | Maintenance and offline states use their established textual warning banners centered in the titlebar. They must not recolor the shell or add decorative status icons. Global search is a right-side command, so warnings do not replace its trigger or overlay. | Accepted | Core WebUI shell | +| UX-029 | Recoverable page and module errors use the central compact `DismissibleAlert` presentation with an explicit recovery action where one exists. Full-height workspaces must overlay page feedback instead of allowing an alert to become a stretched workspace row. | Accepted | Core and module WebUIs | +| UX-030 | At narrow widths, the titlebar uses separate context and command rows. Context selectors remain horizontally reachable, search retains its compact trigger, and language/help/notification/account commands remain fixed icon controls without overlap. Shared content padding contracts so domain workspaces retain usable width. | Accepted | Core WebUI shell and all module workspaces | +| UX-031 | Public controls and extension contributions use stable, module-namespaced interface identities. Shared controls expose `interfaceId` and `helpTopicId`; generated source anchors are inventory evidence, not a substitute for an explicit ID when documentation, policy, or automation refers to the control. | Accepted | Core and module WebUIs | +| UX-032 | `F1` resolves help from the focused field or action, then its dialog/section/page and registered route. Focused contexts retain the page fallback; Docs applies audience and permission filtering and falls back to visible module documentation. | Accepted | Core shell, Docs, and all module WebUIs | +| UX-033 | Global search is the left-most titlebar command, immediately before language selection. Its icon, `F3`, and `Ctrl`/`Cmd`+`K` all open the same permission-aware search overlay; the titlebar does not reserve a persistent query field. | Accepted | Core shell and Search WebUI | ## Confirmed Implementation Decisions @@ -230,6 +237,10 @@ instead of reproducing their behavior. - `help` content is contextual guidance, not the accessible name. The persisted `show_inline_help_hints` user preference hides only the `InlineHelp` marker by applying `ui-hide-help-hints` at the document root. +- Shared action-bearing components accept an optional disabled reason. In + particular, `MailServerSettingsPanel` forwards protocol-specific test + blockers into the shared focusable disabled-action tooltip; modules provide + the domain-specific required field, permission, or in-progress reason. - A dirty editor registers once with `useUnsavedDraftGuard`. An explicit Discard button calls `useUnsavedChanges().requestDiscard(afterResolve)`; SPA navigation uses `useGuardedNavigate` or `requestNavigation`. Both paths show @@ -279,7 +290,7 @@ UI documentation until a central cross-repository audit is available. | Core scope | Why `FieldLabel` is omitted | Accessible/context label source | | --- | --- | --- | -| `PasswordField`, `ColorPickerField`, `DateField`, `TimeField`, and `DateTimeField` input internals | These are label-neutral composite primitives and are placed inside `FormField`/`FieldLabel` by the consuming form. Rendering another label inside the primitive would duplicate it. | Enclosing label; a direct consumer must pass an accessible name and record that direct composition here. | +| `PasswordField`, `ColorPickerField`, `DateField`, `TimeField`, and `DateTimeField` input internals | These are label-neutral composite primitives and are placed inside `FormField`/`FieldLabel` by the consuming form. Rendering another label inside the primitive would duplicate it. `PasswordField` may opt into the shared cryptographic generator; the candidate dialog is subordinate to the enclosing field and commits only through its explicit Use action. | Enclosing label; a direct consumer must pass an accessible name and record that direct composition here. | | `ToggleSwitch` native checkbox | The shared component already renders its visible text through `FieldLabel`; the native input must not render a second label. | The enclosing native label and derived `aria-label`. | | `FileDropZone` hidden file input | The input is an implementation detail of the labelled keyboard-operable drop target. | Drop target text and `inputLabel`/`aria-label`. | | `AdminSelectionList` and `DataGrid` list-filter checkboxes | Each option is self-explanatory and already enclosed by its visible option label. | Enclosing native option label. | @@ -310,14 +321,14 @@ converted or reviewed. | Surface | Repository | UX State | Next Action | | --- | --- | --- | --- | -| File connector settings | `govoplan-files` | First adaptive modal slice started: connections and credentials now use full-state create/edit forms with conditional fields, advanced panels, and blocker primitives. Wizard shell is retained for later assisted setup. Central policy card still needs a layered editor. | Finish provider discovery/test-in-flow, then convert policy editing. | -| Mail server settings | `govoplan-mail` / `govoplan-core` | Uses the shared server/credential model visually, but create/edit still needs the same adaptive pattern as files. | Migrate to adaptive server/credential/policy dialogs, with optional assisted wizard later. | +| File connector settings | `govoplan-files` | Migrated to the shared adaptive server/credential/policy pattern with provider discovery, typed controls, actionable blockers, consequence-aware removal, and module-owned verification evidence. | Continue only through bounded Files-owned follow-ups. | +| Mail server settings | `govoplan-mail` / `govoplan-core` | Migrated to the same layered profile/server/credential/policy pattern, including focused connection tests, unsaved-state handling, contextual help, and permission/target blockers. | Continue only through bounded Mail-owned follow-ups. | | Connector policy/effective rows | `govoplan-core`, module UIs | Effective-policy direction exists, but many editors still expose broad option sets. | Put effective value first, move overrides into modal, and explain blocked edits. | | Admin module management | `govoplan-admin` | Has preflight concepts, but operational choices are still technical and dense. | Convert install/uninstall/package changes to operator wizards. | | Configuration packages | `govoplan-admin` | Catalog/import work exists, but package editing can still drift toward technical fields. | Add guided import/review/problem-list flow. | -| Retention and privacy | `govoplan-core` | Functional editor exists; consequence language and provenance can be stronger. | Layer advanced retention options and add review for broad changes. | +| Retention and privacy | `govoplan-core` | Typed effective-policy editor exposes source paths, narrowing semantics, platform locks, permission/target blockers, and explicit clean/loading/save states. | Broader governed-change review remains module-owned where a policy change requires approval. | | API keys | `govoplan-access` / admin UI | Security-sensitive creation needs least-privilege guidance. | Add scoped creation wizard with expiry/owner review. | -| User settings | `govoplan-core` | Preferences persistence exists; interface navigation issue was fixed earlier, but the surface still needs UX review. | Keep simple sections, remove double-click traps, and add quiet explanations. | +| User settings | `govoplan-core` | Simple typed sections use unsaved-change guards, quiet result feedback, contextual help, and explicit busy/clean disabled-action reasons. | Keep bounded; new contributed sections must satisfy the checklist. | ## Impact Index @@ -346,6 +357,11 @@ Every new or changed admin/configuration surface should answer: - Does the screen explain disabled actions and failed validation in plain language? - Does it say who can fix a blocker and where? +- Does a module-localized blocker pass its translated row labels through the + shared `ActionBlockerHint` contract instead of reproducing the component? +- Does longer field or blocker guidance use a stable `DocumentationHelpLink` + topic/context reference, with hosted fallback when the optional Docs module + is absent? - Does it reuse existing core patterns for wizard steps, problem lists, modals, help, and review? - Is there a review or preflight step before broad, destructive, or risky diff --git a/Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT.md b/Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT.md new file mode 100644 index 0000000..d2cfe2d --- /dev/null +++ b/Repo-docs-WEBUI-MODULE-PACKAGE-LAYOUT.md @@ -0,0 +1,19 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/WEBUI_MODULE_PACKAGE_LAYOUT.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# WebUI Module Package Layout + +Core discovers a module contribution from `src/module.ts` when `node_modules` +links directly to a module's `webui` package. Tagged release dependencies are +installed from repository-root packages and expose the same contribution at +`webui/src/module.ts`. The Vite registry accepts both layouts and imports the +contribution descriptor directly so route-level lazy loading is preserved. + +A release package is invalid if neither entry exists. The module-permutation CI +matrix builds source-linked and installed release compositions; it must not fall +back to a package root barrel because that would eagerly pull module pages into +the shell bundle. diff --git a/Repo-docs-audits-2026-07-09-dependency-audit.-.md b/Repo-docs-audits-2026-07-09-dependency-audit.-.md deleted file mode 100644 index 916109e..0000000 --- a/Repo-docs-audits-2026-07-09-dependency-audit.-.md +++ /dev/null @@ -1,78 +0,0 @@ - - -> Mirrored from `/mnt/DATA/git/govoplan-core/docs/audits/2026-07-09-dependency-audit.md`. -> Origin: `repository`. -> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. - ---- -# Dependency Audit - 2026-07-09 - -Commands: - -```bash -cd /mnt/DATA/git/govoplan-core -bash scripts/check-dependency-audits.sh -``` - -Status: remediated. - -Initial result: - -- Python audit failed: 24 advisories were reported across `cryptography`, - `pip`, `python-multipart`, `pyzipper`, and `starlette`. -- npm production audit passed: `npm audit --omit=dev` reported 0 - vulnerabilities. - -Python findings: - -| Package | Installed | Advisory count | Minimum reported fix | -| --- | ---: | ---: | --- | -| `cryptography` | `44.0.0` | 5 | `48.0.1` | -| `pip` | `26.0.1` | 3 | `26.1.2` | -| `python-multipart` | `0.0.17` | 7 | `0.0.31` | -| `pyzipper` | `0.3.6` | 1 | `0.4.0` | -| `starlette` | `0.41.3` | 8 | `1.3.1` | - -Private GovOPlaN packages were skipped by `pip-audit` because they are not -published on PyPI. That is expected for local editable development installs. - -The remediation below upgrades those dependencies and records the compatibility -checks run against core, files, and campaign behavior. - -## Remediation - -Remediation applied on 2026-07-09: - -- upgraded core's FastAPI floor to `fastapi>=0.139,<1`, resolving Starlette to - `starlette==1.3.1` -- upgraded core's cryptography floor to `cryptography>=48.0.1,<50`, resolving - to `cryptography==49.0.0` -- declared the files module upload parser dependency as - `python-multipart>=0.0.31,<1`, resolving to `python-multipart==0.0.32` -- upgraded the campaign ZIP dependency to `pyzipper>=0.4,<1`, resolving to - `pyzipper==0.4.0` -- upgraded the local audit environment to `pip==26.1.2` -- removed the obsolete local `govoplan-module-multimailer` editable install - from the audit environment so the audit reflects the split module product - -Post-remediation result: - -- `bash scripts/check-dependency-audits.sh`: passed, no known Python - vulnerabilities found and npm production audit reported 0 vulnerabilities. -- `python -m pip check`: passed. -- `bash scripts/check-module-matrix.sh`: passed. -- `python -m unittest tests.test_api_smoke`: passed. -- campaign encrypted/plain ZIP smoke with `pyzipper==0.4.0`: passed. - -Notes: - -- `pip-audit` still reports private GovOPlaN packages as skipped because they - are not published on PyPI. That is expected for editable local development - installs. -- `httpx2>=2.5,<3` is included in development requirements so Starlette's - `TestClient` uses the non-deprecated backend. `httpx==0.28.1` remains in dev - requirements for tests that mock connector HTTP responses directly. -- Split-module API routers now use Starlette's renamed - `HTTP_422_UNPROCESSABLE_CONTENT` status constant. This preserves the 422 - status code while avoiding the deprecated `HTTP_422_UNPROCESSABLE_ENTITY` - alias.