diff --git a/Codex-Project-Index.md b/Codex-Project-Index.md index 27d834c..9626032 100644 --- a/Codex-Project-Index.md +++ b/Codex-Project-Index.md @@ -4,4 +4,13 @@ This page is generated from repository and product-directory project files. +- [Product govoplan-split-concept-action-plan](Product govoplan-split-concept-action-plan) - `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/split-concept-action-plan.md` +- [Repo-README](Repo-README) - `/mnt/DATA/git/govoplan-core/README.md` +- [Repo-docs-ACCESS-EXTRACTION-PLAN](Repo-docs-ACCESS-EXTRACTION-PLAN) - `/mnt/DATA/git/govoplan-core/docs/ACCESS_EXTRACTION_PLAN.md` +- [Repo-docs-CODEX-WORKFLOW](Repo-docs-CODEX-WORKFLOW) - `/mnt/DATA/git/govoplan-core/docs/CODEX_WORKFLOW.md` +- [Repo-docs-GITEA-ISSUES](Repo-docs-GITEA-ISSUES) - `/mnt/DATA/git/govoplan-core/docs/GITEA_ISSUES.md` +- [Repo-docs-GOVOPLAN-MODULE-ROADMAP](Repo-docs-GOVOPLAN-MODULE-ROADMAP) - `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MODULE_ROADMAP.md` +- [Repo-docs-MODULE-ARCHITECTURE](Repo-docs-MODULE-ARCHITECTURE) - `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md` +- [Repo-docs-RBAC-MANIFEST](Repo-docs-RBAC-MANIFEST) - `/mnt/DATA/git/govoplan-core/docs/RBAC_MANIFEST.md` - [Repo-docs-RELEASE-DEPENDENCIES](Repo-docs-RELEASE-DEPENDENCIES) - `/mnt/DATA/git/govoplan-core/docs/RELEASE_DEPENDENCIES.md` +- [Repo-docs-SYSTEM-GOVERNANCE-MANIFEST](Repo-docs-SYSTEM-GOVERNANCE-MANIFEST) - `/mnt/DATA/git/govoplan-core/docs/SYSTEM_GOVERNANCE_MANIFEST.md` diff --git a/Product govoplan-split-concept-action-plan.md b/Product govoplan-split-concept-action-plan.md new file mode 100644 index 0000000..caa7735 --- /dev/null +++ b/Product govoplan-split-concept-action-plan.md @@ -0,0 +1,453 @@ + + +> Mirrored from `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/split-concept-action-plan.md`. +> Origin: `product:govoplan`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# GovOPlaN Split Concept and Action Plan + +Source: `split.md` + +## Executive Concept + +GovOPlaN should become a modular governance and workflow platform, not a monolithic Fachverfahren and not a thin event bus only. + +The target structure is: + +- **Kernel**: small platform runtime that composes modules. +- **Platform modules**: cross-cutting capabilities such as access, tenancy, policy, audit, admin, operations. +- **Service modules**: reusable capabilities such as files, mail, templates, recipients, notifications. +- **Business modules**: public-sector workflows such as campaigns, cases, forms, approvals, appointments. +- **Connector modules**: integrations with external public-sector systems and standards. + +The core idea: + +```text +govoplan-kernel starts the platform. +Installed modules register routes, models, migrations, permissions, UI routes, capabilities, and events. +Modules cooperate through kernel contracts, not direct imports. +``` + +This keeps every module independently installable while still allowing richer behavior when other modules are present. + +## Target Architecture + +### Kernel + +The kernel owns only platform composition: + +- app factory and server assembly +- module discovery and registry +- manifest validation +- route aggregation +- capability registry +- command/event contracts +- lifecycle hooks +- configuration loading +- database/session lifecycle +- module migration orchestration +- health and platform metadata +- shared WebUI shell contracts +- OpenAPI aggregation + +The kernel must not own product semantics such as users, tenants, RBAC rules, audit policy, mail behavior, file behavior, or campaign behavior. + +### Platform Modules + +These are cross-cutting but still modules: + +- `govoplan-access`: accounts, authentication, sessions, groups, memberships, RBAC, API keys, principal resolution, login adapters. +- `govoplan-tenancy`: tenant registry, tenant lifecycle, tenant settings boundary, tenant switching, tenant deletion/erasure orchestration. +- `govoplan-policy`: permission catalogue, hierarchical policy engine, delegation ceilings, explain decisions, impact simulation. +- `govoplan-audit`: audit sink, audit queries, evidence exports, retention, compliance reports. +- `govoplan-admin`: system/tenant admin UI and administrative workflows over access, tenancy, policy, audit, and operations. +- `govoplan-ops`: monitoring, backup/restore, update/rollback, maintenance windows, system runbooks. + +### Existing Service and Business Modules + +Current modules keep their ownership: + +- `govoplan-files`: managed files, file spaces, sharing, downloads, file metadata, future legal hold/version lock/DMS integration. +- `govoplan-mail`: SMTP/IMAP profiles, credentials, mail policy, sending, append-to-Sent, mailbox views, future inbound mail-to-case. +- `govoplan-campaign`: campaign authoring, validation, review/send control, reports, optional file/mail integration. + +### Future Module Families + +Recommended future modules: + +- Work management: `govoplan-cases`, `govoplan-workflow`, `govoplan-tasks`, `govoplan-forms`, `govoplan-appointments`, `govoplan-templates`. +- Communication/content: `govoplan-notifications`, expanded files/mail/campaign integrations. +- Connectors: `govoplan-connectors`, `govoplan-fit-connect`, `govoplan-xoev`, `govoplan-xta-osci`, `govoplan-dms`, `govoplan-erp`, `govoplan-xrechnung`. +- Citizen/public interaction: `govoplan-portal`, `govoplan-identity-trust`, `govoplan-payments`. +- Insight/compliance: `govoplan-search`, `govoplan-reporting`, `govoplan-data-catalog`, `govoplan-compliance`. + +## Non-Negotiable Design Principles + +- Modules must run when optional sibling modules are physically absent. +- Modules must not import optional module internals directly. +- Optional behavior must go through kernel contracts: capabilities, metadata, registry queries, events, commands, and API boundaries. +- Kernel owns database/session lifecycle and migration orchestration; modules own models and migrations. +- Shared WebUI shell and generic components belong in core/kernel WebUI. +- Module WebUI packages contribute nav/routes/pages through manifests and module contracts. +- Every policy decision should be explainable, not just boolean. +- Audit should subscribe to platform/domain events instead of being embedded in the kernel. +- Production eventing should use an outbox pattern; in-process events are acceptable for local/dev. + +## Staged Refactor Plan + +### Phase 0: Stabilize Current Modular Baseline + +Goal: make the current core/files/mail/campaign split reliable before deeper extraction. + +Action items: + +- [ ] Document current module contract as the compatibility baseline. +- [ ] Define which APIs are stable kernel contracts and which are temporary compatibility paths. +- [ ] Add dependency-boundary checks that forbid direct optional module imports. +- [ ] Keep module-matrix tests for core-only, files-only, mail-only, campaign-only, campaign+files, campaign+mail, full product. +- [ ] Keep generated WebUI/test artifacts ignored. +- [ ] Mark DataGrid work as explicitly deferred. + +Acceptance criteria: + +- Full product still runs. +- Every supported module permutation builds and starts. +- No optional module is required for another module to import. + +### Phase 1: Freeze Kernel Contracts + +Goal: make the kernel API explicit before extracting platform semantics. + +Action items: + +- [ ] Move or clearly mark these as kernel-stable contracts: + - `ModuleManifest` + - `MigrationSpec` + - capability factory contract + - route factory contract + - WebUI module contribution contract + - resource ACL provider contract + - tenant summary/delete-veto provider contract + - lifecycle hook contract + - event and command contracts +- [ ] Add contract tests for manifests, route aggregation, migration registration, and capability discovery. +- [ ] Add a documented deprecation policy for kernel API changes. +- [ ] Add OpenAPI/route collision validation. +- [ ] Add manifest schema/versioning. + +Acceptance criteria: + +- Modules can declare all backend and WebUI contributions without private imports. +- The kernel can reject invalid manifests with actionable errors. + +### Phase 2: Extract Access + +Goal: turn current access/auth/RBAC functionality into a real platform module. + +Target repository/package: + +```text +govoplan-access +``` + +Action items: + +- [ ] Move accounts, authentication, sessions, API keys, groups, memberships, roles, role assignments, and principal resolution into access. +- [ ] Keep compatibility imports under `govoplan_core.access.*` temporarily with deprecation warnings. +- [ ] Move access permissions and default role templates into the access manifest. +- [ ] Move access migrations into the access module migration registration. +- [ ] Move login/session/admin access UI into access/admin module contributions. +- [ ] Define `PrincipalResolver` and related protocols in the kernel. +- [ ] Ensure core-only startup still works enough to show shell/health, but access is required for authenticated product use. + +Acceptance criteria: + +- Kernel imports no access models directly. +- Access can be installed as a module and provides authentication/RBAC. +- Existing login and admin flows still work through module routes/UI contributions. + +### Phase 3: Extract Tenancy + +Goal: separate tenant lifecycle from access identity. + +Target repository/package: + +```text +govoplan-tenancy +``` + +Action items: + +- [ ] Move tenant registry, tenant settings, tenant lifecycle, tenant switching, and tenant deletion orchestration into tenancy. +- [ ] Keep memberships and role assignments in access. +- [ ] Define `TenantResolver` and tenant-context protocols in the kernel. +- [ ] Define tenant lifecycle events: created, suspended, resumed, deletion requested, erasure completed. +- [ ] Add delete-veto orchestration through module providers. +- [ ] Add tenant lifecycle tests with files/mail/campaign installed and absent. + +Acceptance criteria: + +- Tenant data ownership is clear. +- Modules can register tenant delete-veto and cleanup behavior. +- Access and tenancy interact through protocols/capabilities, not circular imports. + +### Phase 4: Extract Policy + +Goal: make policy the common decision engine for access, tenancy, mail, retention, campaign, files, and future modules. + +Target repository/package: + +```text +govoplan-policy +``` + +Action items: + +- [ ] Define a common `PolicyDecision` object with: + - `allowed` + - `reason` + - `requirements` + - `matched_permissions` + - `denied_by` + - `source_path` + - `policy_version` +- [ ] Move hierarchical policy evaluation into policy. +- [ ] Move delegation ceilings and "more restrictive only" validation into policy. +- [ ] Add explain endpoints and UI component contract for policy source paths. +- [ ] Add policy simulation before destructive/limiting changes. +- [ ] Move mail/retention effective policy display onto shared policy components. +- [ ] Add regression tests for non-viable lower-level options being hidden or disabled. + +Acceptance criteria: + +- UI can explain "how we got here" for effective policy without module-specific logic. +- Saving invalid policy states is prevented in the UI, not only rejected by backend. +- Policy changes can be simulated for impact before applying. + +### Phase 5: Extract Audit + +Goal: make audit an event-consuming platform module. + +Target repository/package: + +```text +govoplan-audit +``` + +Action items: + +- [ ] Define typed `PlatformEvent` envelope with actor, tenant, subject, resource, correlation, causation, classification, and payload. +- [ ] Separate command bus from event bus. +- [ ] Introduce outbox table and dispatcher for production-safe event delivery. +- [ ] Move audit log storage/query/export into audit module. +- [ ] Add event producers for access, tenancy, policy, files, mail, and campaign actions. +- [ ] Add evidence bundle export as a later audit capability. + +Acceptance criteria: + +- Kernel does not own audit storage. +- Audit records are produced by module events. +- Events can be correlated across cases/files/mail/campaign/connectors. + +### Phase 6: Split Admin UI + +Goal: avoid one central admin page owning all platform semantics. + +Action items: + +- [ ] Define admin route/nav contribution contract. +- [ ] Move access admin pages to access module. +- [ ] Move tenant admin pages to tenancy module. +- [ ] Move policy admin pages to policy module. +- [ ] Move audit admin pages to audit module. +- [ ] Keep only shell-level admin layout and module aggregation in kernel/core WebUI. + +Acceptance criteria: + +- Installing a platform module contributes its admin pages. +- Removing a module removes its admin pages without broken imports/routes. + +### Phase 7: CI and Dependency Enforcement + +Goal: make modularity enforceable. + +Action items: + +- [ ] Add module permutation backend startup tests. +- [ ] Add WebUI module permutation builds. +- [ ] Add dependency import linting: + - campaign must not directly import files/mail internals except through explicit optional adapters/capabilities. + - files must not directly require campaign. + - mail must not directly require campaign. + - kernel must not import product/business module internals. +- [ ] Add migration registration tests. +- [ ] Add manifest schema tests. +- [ ] Add compatibility import tests until old paths are removed. + +Acceptance criteria: + +- CI fails when optional module boundaries are violated. +- CI proves each intended product composition can run. + +## Product Roadmap + +### Milestone A: Clean Platform Split + +Priority: highest. + +Action items: + +- [ ] Kernel contract freeze. +- [ ] Access extraction. +- [ ] Tenancy extraction. +- [ ] Policy extraction. +- [ ] Audit extraction. +- [ ] Module-matrix CI. +- [ ] Compatibility/deprecation plan. + +Outcome: + +GovOPlaN becomes a stable modular platform rather than a core-heavy product. + +### Milestone B: Daily Administrative Work + +Priority: next after clean split. + +Action items: + +- [ ] Create `govoplan-cases` concept and MVP. +- [ ] Create `govoplan-tasks` concept and MVP. +- [ ] Extract/generalize reusable workflow/review patterns into `govoplan-workflow`. +- [ ] Add `govoplan-forms`. +- [ ] Add `govoplan-templates`. +- [ ] Add `govoplan-recipients` / `govoplan-address-book`. +- [ ] Link existing files/mail/campaign to cases. + +Outcome: + +Municipalities, universities, and public-sector teams can use GovOPlaN for concrete daily work. + +### Milestone C: Public-Sector Integration Platform + +Priority: after work management MVP. + +Action items: + +- [ ] Build `govoplan-connectors` registry and connector runtime. +- [ ] Add FIT-Connect connector. +- [ ] Add XÖV validation/mapping module. +- [ ] Add DMS/eAkte connector. +- [ ] Add ERP/finance connector. +- [ ] Add XRechnung workflow. +- [ ] Add IDM/LDAP/OIDC/SAML/SCIM provisioning. + +Outcome: + +GovOPlaN coordinates existing systems instead of replacing every specialist application. + +### Milestone D: Governance Control Plane + +Priority: after platform and integration foundations. + +Action items: + +- [ ] Cross-module reporting. +- [ ] Permission-aware search. +- [ ] Data catalogue. +- [ ] Compliance workspace. +- [ ] Evidence bundle verifier. +- [ ] Policy impact analysis. +- [ ] Destructive tenant erasure orchestration. +- [ ] Operations governance dashboard. + +Outcome: + +GovOPlaN becomes a governance, audit, and compliance layer over administrative work. + +## Immediate Backlog Items + +### P0: Architecture Safety + +- [ ] Write a concise kernel responsibility statement in `MODULE_ARCHITECTURE.md`. +- [ ] Add "kernel must not own product semantics" as an architecture rule. +- [ ] Add a dependency-boundary test or lint script. +- [ ] Add backend startup tests for supported module permutations. +- [ ] Add manifest schema/version validation. +- [ ] Keep compatibility imports documented before extraction begins. + +### P0: Access Extraction Prep + +- [ ] Inventory all current `govoplan_core.access`, auth, user, account, group, API key, role, and permission imports. +- [ ] Identify which imports belong to future `govoplan-access`. +- [ ] Define `PrincipalResolver` protocol in kernel. +- [ ] Define access module manifest target. +- [ ] Create extraction checklist for models, migrations, routes, services, WebUI pages, and tests. + +### P0: Policy Extraction Prep + +- [ ] Inventory current mail, retention, RBAC, delegation, and governance policy logic. +- [ ] Define shared `PolicyDecision`. +- [ ] Define policy source path format. +- [ ] Define backend explain endpoint shape. +- [ ] Define frontend effective-policy component contract. +- [ ] Add UI tests for blocked/disallowed lower-level policy choices. + +### P0: Audit/Event Prep + +- [ ] Define typed event envelope. +- [ ] Decide command bus versus event bus API shape. +- [ ] Add correlation/causation ID support. +- [ ] Define audit module MVP boundaries. +- [ ] Identify domain events from access, tenancy, policy, files, mail, campaign. + +### P1: Tenancy Extraction Prep + +- [ ] Inventory tenant model and tenant setting usage. +- [ ] Define `TenantResolver` protocol. +- [ ] Separate tenant lifecycle from membership/roles. +- [ ] Define tenant deletion and module cleanup/veto contract. + +### P1: Admin UI Split Prep + +- [ ] Inventory admin pages by owning future module. +- [ ] Define admin route contribution contract. +- [ ] Move generic admin layout into core WebUI. +- [ ] Keep module-specific admin panels in owning modules. + +### P1: Existing Module Hardening + +- [ ] Ensure campaign uses files/mail only through capabilities/metadata/API boundaries. +- [ ] Ensure files does not require campaign. +- [ ] Ensure mail does not require campaign. +- [ ] Keep mailbox, mail policy, mail credential inheritance, and file storage behavior covered by focused tests. + +### P2: Future Product Concepts + +- [ ] Write `govoplan-cases` concept. +- [ ] Write `govoplan-workflow` concept. +- [ ] Write `govoplan-connectors` concept. +- [ ] Write `govoplan-policy` concept. +- [ ] Write `govoplan-audit` concept. +- [ ] Write `govoplan-idm` concept. + +## Key Decisions Needed + +- [ ] Should `govoplan-core` be renamed conceptually to `govoplan-kernel`, while keeping package compatibility for now? +- [ ] Is access required for any useful product startup, or should kernel-only expose only health/shell/config pages? +- [ ] Should tenants be extracted before policy, or after access but before policy? +- [ ] Should admin be one platform module or split across access/tenancy/policy/audit? +- [ ] Which event transport is the first production target: DB outbox only, Redis/Celery, or pluggable dispatch? +- [ ] Which future module comes first after platform cleanup: cases, workflow, or connectors? + +## Recommended Next Step + +Start with a small architecture-hardening milestone: + +1. Update module architecture docs with the kernel/platform module model. +2. Add dependency-boundary checks. +3. Add missing backend module permutation startup tests. +4. Inventory access/auth/RBAC imports. +5. Draft the `govoplan-access` extraction plan. + +Do not start by physically moving access, tenancy, policy, and audit all at once. The safe path is contract freeze first, then one extraction at a time. diff --git a/Repo-README.md b/Repo-README.md new file mode 100644 index 0000000..bebcb7c --- /dev/null +++ b/Repo-README.md @@ -0,0 +1,104 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/README.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# govoplan-core + +GovOPlaN core is the platform runner and shared foundation. It owns the server entry point, database/session primitives, tenant and RBAC infrastructure, governance policy, audit/auth helpers, module discovery, migration registration, and the shared WebUI shell. Feature code is supplied by installed modules. + +## Repository ownership + +Core owns: + +- `govoplan_core.server.app:app`, the FastAPI entry point used by uvicorn +- `GovoplanServerConfig`, module discovery, registry validation, and route aggregation +- SQLAlchemy base/session helpers and module migration registration +- tenant/account/session/RBAC/governance/audit models and services +- core API routes for auth, admin, platform metadata, audit, and system health +- `@govoplan/core-webui`, including login, CSRF/API helpers, shell layout, generic UI components, IconRail, DataGrid, access boundaries, and module route/nav contracts + +Feature modules own their backend routers, models, migrations, permissions, frontend packages, nav items, and route contributions. Core should not import feature pages directly; it imports module manifests and renders their route contributions. + +## Governance docs + +Canonical policy documents live in `docs/`: + +- [RBAC_MANIFEST.md](docs/RBAC_MANIFEST.md) +- [SYSTEM_GOVERNANCE_MANIFEST.md](docs/SYSTEM_GOVERNANCE_MANIFEST.md) +- [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md) +- [CODEX_WORKFLOW.md](docs/CODEX_WORKFLOW.md) + +Modules may define module-specific permissions and policy behavior, but the platform-level permission model and governance hierarchy belong here. + +## Backend development + +Create or activate the core virtual environment, then install core and sibling modules from this repository: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m pip install -r requirements-dev.txt +``` + +Run the platform server from core through the module-aware development runner. The default config reads `ENABLED_MODULES` and discovers installed module entry points. Local development defaults to `tenancy,access,admin,policy,audit,campaigns,files,mail`; set `ENABLED_MODULES` explicitly when testing a smaller module permutation. + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m govoplan_core.devserver \ + --host 127.0.0.1 \ + --port 8000 +``` + +For example, to test campaign without files or mail: + +```bash +cd /mnt/DATA/git/govoplan-core +ENABLED_MODULES=access,campaigns ./.venv/bin/python -m govoplan_core.devserver \ + --host 127.0.0.1 \ + --port 8000 +``` + +The runner loads the same `GovoplanServerConfig` as `govoplan_core.server.app:app`, builds the platform registry, and passes core plus enabled module source roots to uvicorn as reload directories. After reinstalling the editable package, the same command is also available as `govoplan-devserver`. + +The default development SQLite database lives at `runtime/multimailer-dev.db`, alongside other local runtime state. + +Local devserver runs do not require Redis. `CELERY_ENABLED` defaults to `false`, so campaign queue actions update database state without publishing Celery tasks. Use the synchronous send flow for local send tests, or set `CELERY_ENABLED=true` only when a Redis broker and worker are running. + +If the configured local SQLite database is missing or empty, `govoplan_core.devserver` enables the development bootstrap before loading settings. This creates the schema and the default development login on startup. Explicitly setting `DEV_BOOTSTRAP_ENABLED=false` disables this convenience. Production deployments should use migrations and managed database provisioning instead. + +To verify the effective runtime paths and missing-SQLite bootstrap without starting uvicorn, run the smoke mode: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m govoplan_core.devserver --smoke --no-reload +``` + +The smoke mode prints the effective config, runtime root, database URL, modules, reload state, and bootstrap decision, then creates the ASGI app and runs startup once. + +`requirements-dev.txt` links local GovOPlaN module checkouts for development. `requirements-release.txt` installs the packaged modules from tagged git refs for release builds. See [RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md). + +## WebUI development + +Install and run from the core WebUI host: + +```bash +cd /mnt/DATA/git/govoplan-core/webui +PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm install +PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run dev +``` + +The local host links sibling module WebUI packages through local file dependencies and Vite filesystem allowances. Release builds should use `webui/package.release.json`, which points WebUI module packages at tagged git refs. See [RELEASE_DEPENDENCIES.md](docs/RELEASE_DEPENDENCIES.md). + +## Module contract + +Backend modules register through the `govoplan.modules` entry point and return a `ModuleManifest`. A manifest can contribute: + +- permissions and role templates +- API routers +- SQLAlchemy metadata and migration locations +- nav metadata and frontend package metadata +- resource ACL providers and tenant summary/delete-veto providers + +WebUI modules export a `PlatformWebModule` with nav items and route contributions. Core renders those routes with `settings` and `auth` context. Frontend nav icons must be supplied as core-resolved `iconName` strings, not imported icon components. See [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md) for the full module-building contract. diff --git a/Repo-docs-ACCESS-EXTRACTION-PLAN.md b/Repo-docs-ACCESS-EXTRACTION-PLAN.md new file mode 100644 index 0000000..6ecdbe8 --- /dev/null +++ b/Repo-docs-ACCESS-EXTRACTION-PLAN.md @@ -0,0 +1,463 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/ACCESS_EXTRACTION_PLAN.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# GovOPlaN Access Extraction Plan + +> Backlog state migrated to Gitea issues on 2026-07-06. Keep this document as +> durable extraction context and architecture notes. Track active tasks, +> decisions, blockers, and implementation state in `add-ideas/govoplan-core` +> issues with `module/access` and `source/backlog-import`. + +This plan describes how to extract access, authentication, RBAC, and related +administration behavior from the current compatibility core into a dedicated +`govoplan-access` platform module. + +The goal is not to make access optional in every real deployment. The goal is to +make ownership explicit: the kernel composes modules and exposes contracts; +`govoplan-access` owns identity, sessions, API keys, roles, groups, and access +administration. + +## Current Inventory + +### Existing Access Module Seed + +`govoplan-access` contains the extracted module-shaped seed package under +`src/govoplan_access/backend`. `govoplan-core` keeps compatibility import shims +under `src/govoplan_core/access`: + +- `manifest.py` declares `ModuleManifest(id="access")`. +- `db/models.py` defines access-owned model candidates. +- `auth/principals.py`, `auth/roles.py`, and `auth/tokens.py` contain auth + helper concepts. +- `permissions/definitions.py`, `permissions/evaluator.py`, and + `permissions/registry.py` contain permission catalogue/evaluation concepts. +- `tenancy/datastore.py` contains early tenancy access helpers. + +This seed package is now the starting point for the access module, but it is +not yet the full live source of truth for the running product. + +### Live Compatibility Ownership In Core + +The active implementation still uses core-owned models and services: + +- `src/govoplan_core/db/models.py` + - `Account` + - `Tenant` + - `User` + - `Group` + - `Role` + - `SystemSettings` + - `GovernanceTemplate` + - `GovernanceTemplateAssignment` + - `SystemRoleAssignment` + - `UserGroupMembership` + - `UserRoleAssignment` + - `GroupRoleAssignment` + - `ApiKey` + - `AuthSession` + - `AuditLog` +- `src/govoplan_core/auth/dependencies.py` +- `src/govoplan_core/security/api_keys.py` +- `src/govoplan_core/security/permissions.py` +- `src/govoplan_core/security/sessions.py` +- `src/govoplan_core/security/passwords.py` +- `src/govoplan_core/security/secrets.py` +- `src/govoplan_core/security/module_permissions.py` +- `src/govoplan_core/admin/service.py` +- `src/govoplan_core/admin/governance.py` +- `src/govoplan_core/api/v1/auth.py` +- `src/govoplan_core/api/v1/admin.py` +- `src/govoplan_core/api/v1/admin_schemas.py` +- `src/govoplan_core/api/v1/audit.py` +- `src/govoplan_core/db/bootstrap.py` + +Core still hosts the generic login/settings shell and shared WebUI primitives. +The legacy administration page has moved to the `govoplan-access` WebUI package +and is contributed as the `/admin` route by the access module. Individual admin +panels can now be split further into access, admin, tenancy, policy, and audit +WebUI contributions without changing the core shell route wiring again. + +### Current Module Consumers + +Feature modules no longer import core auth dependency wrappers or access-owned +ORM models. Backend routers import the access-published FastAPI dependency API +from `govoplan_access.backend.auth.dependencies`; runtime cooperation uses +kernel capabilities such as `access.directory`, `campaigns.access`, +`campaigns.mailPolicyContext`, and `campaigns.deliveryTasks`. + +## Target Ownership + +### Kernel + +The kernel keeps only platform composition and stable contracts: + +- module discovery and registry validation +- route aggregation +- migration orchestration +- database engine/session lifecycle +- permission catalogue aggregation +- capability registry +- event/command envelopes +- dependency injection hooks +- health and diagnostics +- compatibility facades while modules migrate + +The kernel should not own account, tenant, role, group, session, or API-key +semantics. + +### `govoplan-access` + +`govoplan-access` owns: + +- accounts +- authentication routes +- session lifecycle +- API keys +- users +- groups and memberships +- roles and assignments +- principal resolution +- permission evaluation +- access administration routes +- access administration WebUI contributions +- access-related migrations +- access permissions and role templates + +### Later Platform Modules + +Some current access-adjacent behavior can remain in access initially, but should +have clean seams for later extraction: + +- `govoplan-tenancy`: tenants, tenant lifecycle, tenant switching, tenant + metadata, tenant settings boundaries. +- `govoplan-policy`: hierarchical policy/effective policy resolution and + provenance display contracts. +- `govoplan-audit`: audit log storage, audit routes, retention hooks, evidence + exports. +- `govoplan-admin`: generic administration shell contributions if they are not + owned by access/tenancy/policy/audit directly. + +## Kernel Contracts To Stabilize First + +Before moving live code, define contracts that feature modules can depend on +without importing access ORM models: + +- `PrincipalResolver` + - resolves the current actor from a request/session/API key. + - returns a stable DTO, not an ORM model. +- `AccessDirectory` + - resolves users, groups, memberships, and display labels by stable IDs. + - supports batch lookups for grids and policy screens. +- `TenantResolver` + - resolves current tenant context and tenant metadata by stable ID. +- `PermissionEvaluator` + - evaluates required scopes for a principal and resource. +- `ResourceAccessProvider` + - lets modules register resource ACL behavior without importing each other. +- `SecretProvider` + - encrypts/decrypts named secrets without tying consumers to access models. +- `AuditSink` + - records audit events without importing audit storage models. + +DTOs should be small and serializable: + +- `PrincipalRef` +- `AccountRef` +- `TenantRef` +- `UserRef` +- `GroupRef` +- `RoleRef` + +## Extraction Stages + +### Stage 0: Baseline Safeguards + +Status: started. + +- Document the kernel/platform module model. +- Add dependency-boundary checks. +- Add backend module permutation startup tests. +- Inventory access/auth/RBAC imports. +- Draft this extraction plan. + +Acceptance criteria: + +- `scripts/check_dependency_boundaries.py` passes. +- backend module permutation tests start every supported module combination. +- current direct imports are tracked as explicit transitional debt. + +### Stage 1: Contract Definitions + +Add kernel-owned protocols and DTOs for access-related interaction. + +Tasks: + +- Add protocol definitions under a kernel contract package. +- Add registry/capability names for access services. +- Keep existing core dependency functions as compatibility wrappers. +- Make wrappers resolve through capabilities when `govoplan-access` is present. +- Add tests for missing-capability behavior and clear error messages. + +Acceptance criteria: + +- Feature modules can receive principal/tenant/group/user references without + importing access ORM models. +- Existing routes continue to work through compatibility wrappers. + +### Stage 2: Create `govoplan-access` + +Create the new repository/package and move the access seed package into it. + +Tasks: + +- Create package metadata and entry points for `govoplan-access`. +- Move `govoplan_core/access` implementation into the new package. +- Publish `ModuleManifest(id="access")`. +- Register access permissions, role templates, routers, and migrations. +- Add compatibility imports in core where needed. +- Add release/dev dependency entries in core. + +Acceptance criteria: + +- `govoplan-core + govoplan-access` starts through normal module discovery. +- `govoplan-core` compatibility imports still work during transition. +- access manifest, migrations, and route contributions are discovered by core. + +### Stage 3: Move Live Models And Migrations + +Move active identity/access models out of `govoplan_core.db.models`. + +Current state: the stable table naming strategy is to keep legacy table names +and move SQLAlchemy class definitions under their platform owners. The old +`govoplan_core.db.models` compatibility re-export has been removed; callers +must import module-owned models or use kernel capabilities. The earlier +`access_*` candidate tables are not active metadata. + +Current table ownership: + +- `govoplan-tenancy`: `tenants` +- `govoplan-access`: `accounts`, tenant memberships in `users`, `groups`, + `roles`, role/group assignment tables, `api_keys`, and `auth_sessions` +- `govoplan-admin`: governance templates and assignments +- `govoplan-audit`: audit log +- `govoplan-core`: system settings + +Tasks: + +- [x] Decide the stable table naming strategy. +- [x] Map legacy model names to access-owned model classes. +- [x] Keep database table names where possible to avoid data migration churn. +- [x] Add Alembic migration metadata owned by the platform module owners. +- [x] Remove core model compatibility aliases after callers moved to + module-owned imports. +- [x] Update bootstrap/create-all compatibility paths. + +Acceptance criteria: + +- Existing development databases migrate without data loss. +- New databases initialize with module-owned metadata. +- Core no longer owns or re-exports live account/user/group/role/session/API-key + model definitions. + +### Stage 4: Move Auth Routes And Dependencies + +Move authentication, sessions, API keys, and principal dependencies. + +Tasks: + +- Move `/api/v1/auth/*` implementation to access. +- Move session/API-key services to access. +- Keep core route compatibility only if required by clients. +- Replace feature-module imports of core auth dependencies with the + access-published FastAPI dependency API. +- Add tests for login, session refresh, tenant selection, API-key auth, and + missing access capability failures. + +Acceptance criteria: + +- Auth behavior works through the access module. +- Feature modules do not import access internals except the published + `govoplan_access.backend.auth.dependencies` dependency API used by FastAPI + routers. +- Core can explain startup failure clearly if auth-required routes are enabled + without the access capability. + +### Stage 5: Move Admin And WebUI Contributions + +Move access administration UI/API ownership into modules. + +Tasks: + +- [x] Move users, groups, roles, API keys, and access settings pages into + `govoplan-access`. +- [ ] Move tenant-specific pages to `govoplan-tenancy` when that module exists, or + keep them temporarily in access with clear boundaries. +- [x] Move overview, system settings, and governance-template panels into + `govoplan-admin`. +- [x] Register admin navigation and admin sections through module + contributions. +- [x] Keep core shell, layout, route rendering, and generic components only. + +Acceptance criteria: + +- Core WebUI shell renders admin/access pages from module contributions. +- Core does not import access page components directly. +- Module nav and route metadata remain serializable. +- The access admin shell consumes module-owned `admin.sections` capability + contributions without importing sibling module panels. + +### Stage 6: Decouple Feature Modules + +Remove direct ORM/model imports from files, mail, and campaign. + +Tasks: + +- Replace `Group`, `Tenant`, `User`, and `UserGroupMembership` imports with + directory/capability lookups. +- Replace direct cross-module cleanup/count queries with registered providers, + events, or module-owned API contracts. +- Replace mail-profile ownership resolution with stable owner references. +- Replace campaign/file access checks with resource ACL provider contracts. + +Acceptance criteria: + +- `govoplan-files`, `govoplan-mail`, and `govoplan-campaign` do not import + access implementation modules or sibling feature modules. +- Dependency-boundary allowlist stays empty after each replacement. + +### Stage 7: Remove Compatibility Debt + +Finalize the split. + +Tasks: + +- Remove obsolete core compatibility aliases. +- Keep the dependency-boundary checker allowlist empty. +- Update release dependency docs. +- Update operator migration notes. +- Add full module permutation tests including access-present and access-absent + behavior. + +Acceptance criteria: + +- Core is a composition kernel. +- Access behavior is owned by `govoplan-access`. +- Feature modules communicate through kernel contracts, capabilities, events, + and their own APIs. + +## Immediate Backlog + +- [x] Document kernel/platform module model. +- [x] Add dependency-boundary check. +- [x] Add backend permutation startup smoke checks. +- [x] Inventory access/auth/RBAC import debt. +- [x] Draft access extraction plan. +- [x] Define kernel access DTOs and protocols. +- [x] Add access capability registry names. +- [x] Register live access directory and tenant resolver capability + implementations backed by the current compatibility tables. +- [x] Register an access tenant-provisioner capability so tenancy can seed + access-owned default roles without importing access service internals. +- [x] Replace `govoplan-files` direct user/group/tenant model imports with the + `access.directory` capability for group membership and share-target checks. +- [x] Replace `govoplan-files` direct campaign model imports with the + `campaigns.access` capability for campaign share/existence checks. +- [x] Replace `govoplan-mail` direct user/group/tenant model imports with + mail-owned policy storage plus access-directory validation and legacy-read + fallback. +- [x] Replace `govoplan-mail` direct campaign model imports with the + `campaigns.mailPolicyContext` capability for campaign-scoped mail policy and + owner context. +- [x] Replace `govoplan-campaign` runtime user/group/tenant model imports with + access-directory lookups and string-based ORM relationships. +- [x] Create `govoplan-access` repository workspace and issue workflow scaffold. +- [x] Create `govoplan-access` repository/package skeleton. +- [x] Move `govoplan_core/access` seed package into `govoplan-access`. +- [x] Add compatibility wrappers in core for old import paths. +- [x] Route existing auth dependency wrappers through access principal/evaluator capabilities. +- [x] Move FastAPI auth dependency wrappers out of core and into + `govoplan-access`. +- [x] Move session, API-key, and password helper services into `govoplan-access`. +- [x] Move interactive auth/session routes behind access module manifest. +- [x] Move legacy admin/API-key routes behind access module manifest. +- [x] Move access-owned legacy admin service helpers into `govoplan-access`. +- [x] Move governance-template CRUD helpers and routes into `govoplan-admin`. +- [x] Move governance-template materialization of access-owned groups and roles + behind the `access.governanceMaterializer` capability. +- [x] Replace legacy access-to-files/campaign admin lookups with module + tenant-summary and group-delete veto providers. +- [x] Split legacy admin route contribution across `govoplan-admin`, + `govoplan-tenancy`, `govoplan-policy`, `govoplan-audit`, and + `govoplan-access` route slices. +- [x] Move legacy admin route-handler ownership out of the access compatibility + router into access, admin, tenancy, policy, and audit module routers. +- [x] Move shared system-settings, tenant-governance, slug/error, and + tenant-count helpers out of access internals into core settings/tenancy + helpers used by the split platform route modules. +- [x] Move campaign schema routes into the campaign route contribution. +- [x] Move development mailbox routes into the mail route contribution. +- [x] Replace core Celery direct campaign/mail imports with the + `campaigns.deliveryTasks` capability. +- [x] Replace core retention direct campaign queries with + `campaigns.policyContext` and `campaigns.retention` capabilities. +- [x] Replace core `create_all` feature-model imports with module registry + metadata discovery. +- [x] Remove transitional boundary checker allowlist entries. +- [x] Move live legacy model definitions out of core and into + `govoplan-access` while preserving existing table names. +- [x] Split the transitional access-owned legacy model graph further into + tenancy, audit, admin, access, and core settings ownership. +- [x] Reverse the tenancy/access dependency direction so `govoplan-access` + depends on `govoplan-tenancy`, and the registry inserts tenancy before access. +- [x] Replace tenancy-to-access default role seeding with an access + tenant-provisioner capability. +- [x] Replace tenancy-to-access owner candidate and owner membership handling + with the access tenant-provisioner capability. +- [x] Replace admin overview counts and audit actor lookup/filtering with the + `access.administration` capability. +- [x] Replace feature-module direct user/group/tenant model imports. + - [x] `govoplan-files` + - [x] `govoplan-mail` + - [x] `govoplan-campaign` +- [x] Replace files/mail/campaign direct cross-module SQL lookups with + provider/capability contracts. +- [x] Move access admin WebUI pages to module route contributions. +- [x] Move generic admin-owned WebUI panels into `govoplan-admin` and register + them through the `admin.sections` UI capability. +- [x] Remove transitional boundary checker allowlist entries as each contract + lands. +- [x] Remove old core import shims for access models, access routes, admin + service helpers, and access-owned security services. +- [ ] Move campaign-scoped mail policy ownership fully behind an API/event + workflow if direct synchronous capability calls become too tight for later + deployment boundaries. + +## Open Decisions + +- Is `govoplan-access` required for any authenticated deployment, while + core-only remains a diagnostics/settings shell? +- Tenant models live in `govoplan-tenancy`; `govoplan-access` depends on + tenancy for authenticated platform composition. +- Historical table names remain stable for painless migrations. +- Should secret encryption remain a kernel primitive or become an access-owned + capability? +- Audit log storage lives in `govoplan-audit`; policy provenance can build on + that module boundary. +- How long should core keep compatibility route paths for existing clients? + +## Verification + +Use the following checks while extracting access: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python scripts/check_dependency_boundaries.py +./.venv/bin/python -m unittest tests.test_module_system +``` + +For each removed dependency-boundary exception, add or update a focused test +that proves the replacement contract starts without the old direct import. diff --git a/Repo-docs-CODEX-WORKFLOW.md b/Repo-docs-CODEX-WORKFLOW.md new file mode 100644 index 0000000..d9ba81a --- /dev/null +++ b/Repo-docs-CODEX-WORKFLOW.md @@ -0,0 +1,86 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/CODEX_WORKFLOW.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Codex Workflow + +This project is split across the core runner and sibling module repositories. Codex works best when all active repositories are writable from the start and routine checks use targeted commands. + +## Personal Codex Config + +Put machine-specific access in `~/.codex/config.toml`, not in a tracked project file: + +```toml +model = "gpt-5.5" +model_reasoning_effort = "xhigh" +personality = "pragmatic" + +sandbox_mode = "workspace-write" +approval_policy = "on-request" +approvals_reviewer = "user" + +[sandbox_workspace_write] +writable_roots = [ + "/mnt/DATA/git", +] +network_access = false + +[projects."/mnt/DATA/git/govoplan-core"] +trust_level = "trusted" + +[projects."/mnt/DATA/git/govoplan-access"] +trust_level = "trusted" + +[projects."/mnt/DATA/git/govoplan-mail"] +trust_level = "trusted" + +[projects."/mnt/DATA/git/govoplan-files"] +trust_level = "trusted" + +[projects."/mnt/DATA/git/govoplan-campaign"] +trust_level = "trusted" +``` + +The broad writable root reduces approval churn. The explicit project trust entries allow project-local `AGENTS.md` guidance to load for each repository. + +## Repository Guidance + +Each active repository has an `AGENTS.md` file. These files define ownership, module boundaries, and focused commands for Codex. Keep durable project conventions there instead of repeating them in every prompt. + +Use `~/.codex/config.toml` for personal defaults, auth/runtime settings, writable roots, and trust decisions. Avoid checking in absolute-path writable roots or model preferences unless they are intentionally team-wide. + +Use Gitea issues as the canonical backlog and state log. See `docs/GITEA_ISSUES.md` for label setup, TODO import, and Codex issue update commands. Durable docs should describe stable behavior; changing work state belongs on the issue. + +## Focused Verification + +Use the consolidated script after changes that touch module discovery, optional integrations, shared mail components, mailbox listing, or cross-module WebUI behavior: + +```bash +cd /mnt/DATA/git/govoplan-core +./scripts/check-focused.sh +``` + +For smaller changes, prefer the narrow command named in the relevant `AGENTS.md` file. Examples: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m unittest tests.test_module_system + +cd /mnt/DATA/git/govoplan-mail +/mnt/DATA/git/govoplan-core/.venv/bin/python -m unittest discover -s tests + +cd /mnt/DATA/git/govoplan-core/webui +PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run test:module-permutations +``` + +## Usage Discipline + +- Prefer `rg`, `sed`, and targeted test commands. +- Avoid broad recursive scans and full builds unless the change warrants them. +- Keep generated build/test folders ignored. +- Keep optional module behavior behind core registry/capability/module metadata boundaries. +- Create or update Gitea issues for TODOs, follow-ups, blockers, and feature requests instead of keeping local backlog files. +- Do not start persistent dev servers unless the user asks. diff --git a/Repo-docs-GITEA-ISSUES.md b/Repo-docs-GITEA-ISSUES.md new file mode 100644 index 0000000..4c5e7f3 --- /dev/null +++ b/Repo-docs-GITEA-ISSUES.md @@ -0,0 +1,279 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/GITEA_ISSUES.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Gitea Issues And Wiki Workflow + +Gitea issues are the canonical backlog for GovOPlaN work: bugs, feature requests, tasks, tech debt, TODO migrations, open decisions, and blocked work should live there. Gitea wiki pages are the canonical project reference for durable project context mirrored from repository docs and product-directory notes. + +The same pattern is reusable outside GovOPlaN for any project where Codex works in a local checkout, VSCodium or another editor is used for human inspection, and Gitea is the issue tracker. In that setup, Gitea is the durable coordination layer; Codex and the editor are clients of that state. + +## Initial Setup + +The repository contains Gitea issue templates in `.gitea/ISSUE_TEMPLATE`, a pull request template in `.gitea/PULL_REQUEST_TEMPLATE.md`, and the label taxonomy in `docs/gitea-labels.json`. + +The scripts infer this repository from `origin` (`git@git.add-ideas.de:add-ideas/govoplan-core.git`). Override inference when needed: + +```bash +export GITEA_URL=https://git.add-ideas.de +export GITEA_OWNER=add-ideas +export GITEA_REPO=govoplan-core +export GITEA_TOKEN=... +``` + +The API scripts also read `GITEA_*` values from the target repository's `.env` file. That file is gitignored in this repo, so it is suitable for local tokens: + +```bash +GITEA_TOKEN=... +# Optional if origin inference is not enough: +GITEA_URL=https://git.add-ideas.de +GITEA_OWNER=add-ideas +GITEA_REPO=govoplan-core +``` + +For a shared credentials file outside the target repository, pass `--env-file`: + +```bash +./scripts/gitea-sync-labels.py --env-file /path/to/private/gitea.env --apply +``` + +Create a Gitea token with issue read/write access and label-management permission for the repository. On scoped-token instances, this usually means issue read/write and, if label writes are rejected, repository write permission too. + +Preview and apply labels: + +```bash +cd /mnt/DATA/git/govoplan-core +./scripts/gitea-sync-labels.py +./scripts/gitea-sync-labels.py --apply +``` + +After the `.gitea` files are pushed to the default branch, Gitea will show the issue template chooser. Blank issues are disabled by `.gitea/ISSUE_TEMPLATE/config.yaml`. + +## Multiple Repositories And Workspaces + +The helper scripts are path-based and can run from this core checkout against any repository with a Gitea remote: + +```bash +cd /mnt/DATA/git/govoplan-core +./scripts/gitea-sync-labels.py --root /mnt/DATA/git/govoplan-mail --apply +./scripts/gitea-todo-import.py --root /mnt/DATA/git/govoplan-mail +./scripts/gitea-codex-note.py --root /mnt/DATA/git/govoplan-mail --issue 123 --status progress +``` + +Each target repository is inferred from its own `origin` remote. Use `GITEA_URL`, `GITEA_OWNER`, or `GITEA_REPO` only when a workspace has unusual remotes or the Gitea web URL cannot be inferred from SSH. + +When one core checkout drives another workspace, prefer `--env-file` for shared credentials instead of putting `GITEA_REPO` in the core `.env`; a repo-specific `GITEA_REPO` can accidentally override target inference. + +For non-GovOPlaN projects, provide a project-specific label file and module/project label: + +```bash +./scripts/gitea-sync-labels.py \ + --root /path/to/project \ + --labels-file /path/to/project/docs/gitea-labels.json \ + --apply + +./scripts/gitea-todo-import.py \ + --root /path/to/project \ + --module-label project/example \ + --extra-label area/backend +``` + +If another project does not use `area/*` labels, disable area inference: + +```bash +./scripts/gitea-todo-import.py \ + --root /path/to/project \ + --module-label project/example \ + --no-area-labels +``` + +Install or refresh the shared issue templates in sibling or external repositories: + +```bash +./scripts/gitea-install-workflow.py /mnt/DATA/git/govoplan-mail +./scripts/gitea-install-workflow.py /mnt/DATA/git/govoplan-mail --apply +``` + +The installer rewrites the default template label from `module/core` to the module label inferred from the target repository name. Known mappings cover the packaged GovOPlaN repositories, and any other `govoplan-` checkout maps to `module/`. For another workspace or repository name, pass an explicit label: + +```bash +./scripts/gitea-install-workflow.py /path/to/repo --module-label module/example --apply +``` + +Use `--include-labels-file` if a repository should carry its own copy of `docs/gitea-labels.json`; otherwise keep the shared taxonomy in core and run the sync script from core. + +For a fully portable workflow kit, copy these files into the other project: + +- `scripts/gitea_common.py` +- `scripts/gitea-sync-labels.py` +- `scripts/gitea-todo-import.py` +- `scripts/gitea-codex-note.py` +- `scripts/gitea-install-workflow.py` +- `.gitea/ISSUE_TEMPLATE/*` +- `.gitea/PULL_REQUEST_TEMPLATE.md` +- a project-specific `docs/gitea-labels.json` + +Keep credentials out of the repository. Put `GITEA_TOKEN` in the shell environment, a gitignored `.env`, a local direnv file, or the user-level Codex/VSCodium environment setup. + +## Label Taxonomy + +Use one `type/*` label: + +- `type/bug` +- `type/feature` +- `type/task` +- `type/debt` +- `type/docs` + +Use one `status/*` label while the issue is open: + +- `status/triage`: needs ownership, priority, or acceptance criteria. +- `status/ready`: ready to implement. +- `status/in-progress`: actively being worked. +- `status/blocked`: blocked on an external dependency, credential, or decision. +- `status/needs-info`: blocked on clarification. + +Use one `priority/*` label when prioritization matters: `priority/p0`, `priority/p1`, `priority/p2`, or `priority/p3`. + +Use `module/*` and `area/*` labels to route work. Module labels are not exclusive because cross-module work can exist. Core issues should still preserve ownership boundaries: module-specific implementation belongs in the owning module repository. + +Use `codex/ready` when the issue has enough context for Codex to work from, and `codex/needs-human` when a human decision is required first. + +## Moving TODOs Into Gitea + +Preview inline markers: + +```bash +cd /mnt/DATA/git/govoplan-core +./scripts/gitea-todo-import.py +``` + +Create missing issues after labels are synced: + +```bash +./scripts/gitea-todo-import.py --apply +``` + +The importer scans `TODO`, `FIXME`, `XXX`, and `HACK` markers, skips markers that already reference an issue, applies `source/todo-scan`, and writes a hidden fingerprint into each generated issue body so reruns do not duplicate already imported items. + +When touching code with an imported marker, either remove the marker as part of the fix or replace it with a short reference: + +```python +# TODO(gitea#123): keep only if the local pointer is still useful +``` + +Do not add new untracked TODO comments. Create the Gitea issue first, then reference it inline only when the local pointer materially helps future readers. + +For a broader project import across all local repositories hosted on `git.add-ideas.de`, use the generic backlog importer: + +```bash +./scripts/gitea-import-all-backlogs.py --env-file /home/zemion/.config/gitea/gitea.env +./scripts/gitea-import-all-backlogs.py --env-file /home/zemion/.config/gitea/gitea.env --apply +``` + +It scans repository and product-directory files with backlog-like names, creates the shared generic labels where needed, imports missing open work, and deduplicates reruns by hidden fingerprint and normalized title. + +## Mirroring Project Docs Into Gitea Wikis + +Preview wiki pages for all local repositories hosted on `git.add-ideas.de`, cross-referenced with product directories under `/mnt/DATA/Nextcloud/ADD ideas UG/Products`: + +```bash +cd /mnt/DATA/git/govoplan-core +./scripts/gitea-sync-wiki.py --env-file /home/zemion/.config/gitea/gitea.env +``` + +Apply the wiki mirror: + +```bash +./scripts/gitea-sync-wiki.py --env-file /home/zemion/.config/gitea/gitea.env --apply +``` + +The wiki sync mirrors durable text documents only: root README-style files, docs/codex project docs, and selected product notes such as roadmap, plan, concept, pitch, and whitepaper files. It skips generated folders, dependency/build output, `.gitea` templates, and filenames that look credential-related. + +Each managed wiki page contains a `codex-wiki-sync` marker and a source path. Reruns update only managed pages unless `--overwrite-unmanaged` is passed. Each repository also gets a managed `Codex-Project-Index` page linking the mirrored pages. + +Use the wiki for durable context: + +- project overviews and architecture +- workflows, operating notes, and setup references +- product concepts, plans, pitches, and whitepapers +- historical context that helps interpret issues + +Keep active state in issues: + +- open tasks, TODOs, feature requests, and bugs +- priority, blocking status, and acceptance criteria +- Codex progress updates and implementation notes + +## Codex State Updates + +Codex should read the relevant issue before making changes when issue access is available. During or after work, Codex should add issue comments with the state that would otherwise drift into local notes: + +- scope understood +- files changed +- tests or manual checks run +- blockers or decisions needed +- follow-up issues created + +Preview and post a standardized note: + +```bash +./scripts/gitea-codex-note.py \ + --issue 123 \ + --status progress \ + --summary "Implemented capability metadata fallback." \ + --changed src/govoplan_core/modules/registry.py \ + --test "./.venv/bin/python -m unittest tests.test_module_system" + +./scripts/gitea-codex-note.py \ + --issue 123 \ + --status progress \ + --summary "Implemented capability metadata fallback." \ + --changed src/govoplan_core/modules/registry.py \ + --test "./.venv/bin/python -m unittest tests.test_module_system" \ + --apply +``` + +Use `--close --apply` only when the acceptance criteria are satisfied and verification is recorded. + +## Ownership Rules + +Create the issue in the repository that owns the change: + +- `govoplan-core`: platform runner, DB/session primitives, auth, tenancy, RBAC, governance, module discovery, migrations, shared WebUI shell, and generic WebUI components. +- `govoplan-access`: access, identity, authentication, sessions, API keys, RBAC, groups, users, and access administration. +- `govoplan-mail`: mail-specific backend, frontend, message workflows, and mail integrations. +- `govoplan-files`: files-specific backend, frontend, storage, and file workflows. +- `govoplan-campaign`: campaign-specific backend, frontend, policy, and template behavior. + +For cross-cutting work, create a tracking issue in `govoplan-core` and link module issues from it. Do not use the core issue as a dumping ground for module-specific implementation details. + +## Cleaning Up Mirrored Sources + +After backlog files have been imported into issues and durable context has been mirrored to wiki, old duplicate sources can be removed from git only when they are tracked files and the Gitea issue/wiki state has been verified. Prefer deleting backlog, TODO, roadmap, and one-off planning files that have become duplicate state. + +Do not delete standard repository entry points such as `README`, `LICENSE`, `SECURITY`, or package metadata just because they are mirrored to the wiki. They remain useful for repository browsing, package registries, and developer onboarding. + +Do not delete untracked files or files outside git history as part of automated cleanup unless there is a separate backup or explicit human confirmation for that specific path. + +## Docs Versus Issues + +Keep durable facts in docs: + +- architecture and extension points +- command references +- module boundaries +- operational conventions + +Keep changing state in Gitea: + +- TODOs and follow-ups +- bugs and feature requests +- blocked status +- acceptance criteria +- implementation notes from active work + +If a decision becomes durable architecture, write the durable result into docs and link back to the issue for history. diff --git a/Repo-docs-GOVOPLAN-MODULE-ROADMAP.md b/Repo-docs-GOVOPLAN-MODULE-ROADMAP.md new file mode 100644 index 0000000..fcba663 --- /dev/null +++ b/Repo-docs-GOVOPLAN-MODULE-ROADMAP.md @@ -0,0 +1,86 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/GOVOPLAN_MODULE_ROADMAP.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# GovOPlaN Module And Integration Roadmap + +This page maps current module and integration ideas to existing GovOPlaN repositories or to missing-module decisions. Issues are the active backlog. This document is durable routing context and should be mirrored to the Gitea wiki. + +## Current Routing + +| Idea | Owner | Tracking | +| --- | --- | --- | +| Access as a module | `govoplan-access` | `add-ideas/govoplan-access#7` | +| OpenProject API / project management connector | `govoplan-connectors` | `add-ideas/govoplan-connectors#1` | +| Native project-management module decision | `govoplan-core` | `add-ideas/govoplan-core#196` | +| Datasources for databases, CSV, files, APIs | proposed `govoplan-datasources` | `add-ideas/govoplan-core#197` | +| Dataflow for pipelines, BI, publication | proposed `govoplan-dataflow` | `add-ideas/govoplan-core#198` | +| Templates for letters, emails, forms, reports | `govoplan-templates` | `add-ideas/govoplan-templates#1` | +| Reporting and BI | `govoplan-reporting` | `add-ideas/govoplan-reporting#1` | +| File connectors: Nextcloud, Seafile, SMB, NFS | `govoplan-files` | `add-ideas/govoplan-files#15` | +| Public-sector software integration catalogue | `govoplan-connectors` | `add-ideas/govoplan-connectors#2` | +| Adrema-style address and distribution-list management | `govoplan-addresses` | `add-ideas/govoplan-addresses#1` | +| Consume sources and become a governed source | `govoplan-connectors` plus proposed `govoplan-dataflow` | `add-ideas/govoplan-connectors#3`, `add-ideas/govoplan-core#198` | +| Terminfindung and meeting scheduling polls | `govoplan-scheduling` | local scaffold; remote creation tracked by `add-ideas/govoplan-core#199` | +| Terminplaner and calendar primitives | `govoplan-calendar` | `add-ideas/govoplan-calendar#1` | +| Terminbuchung appointment booking | `govoplan-appointments` | `add-ideas/govoplan-appointments#1` | +| Collaborative documents | `govoplan-dms` | `add-ideas/govoplan-dms#1` | +| Forms | `govoplan-forms` | `add-ideas/govoplan-forms#1` | +| RSS consume and emit | `govoplan-connectors` | `add-ideas/govoplan-connectors#4` | +| LDAP, Active Directory, OpenDesk identity | `govoplan-idm` | `add-ideas/govoplan-idm#1` | +| OpenDesk stack integration map | `govoplan-connectors` | `add-ideas/govoplan-connectors#5` | +| Open-Xchange mail/groupware | `govoplan-mail` | `add-ideas/govoplan-mail#5` | +| Open-Xchange calendar | `govoplan-calendar` | `add-ideas/govoplan-calendar#2` | + +## Proposed Missing Modules + +`govoplan-datasources` should exist only if source catalog ownership becomes broad enough to justify a module separate from connectors and reporting. Candidate responsibilities: + +- source catalogue for SQL databases, CSV/Excel files, uploaded files, APIs, RSS feeds, and governed file locations +- credentials and connection profiles +- schema discovery and refresh cadence +- provenance, freshness, permission boundaries, and audit events + +`govoplan-dataflow` should exist only if pipelines and publication become first-class product behavior. Candidate responsibilities: + +- ingestion, transformation, validation, scheduling, and lineage +- connecting datasources to reports, APIs, RSS, exports, and downstream systems +- the "consume sources, become source" lifecycle +- publication-state and audit integration + +`govoplan-projects` is undecided. The default path should be an OpenProject connector first. A native module is justified only if GovOPlaN must own project semantics beyond cases, tasks, workflow, appointments, documents, and reporting. + +## Boundary Notes + +- `govoplan-connectors` owns protocol and external-system integration strategy. It should not own business semantics once a domain module exists. +- `govoplan-files` owns file storage semantics and file-provider contracts. Remote provider implementations must stay optional. +- `govoplan-dms` owns document lifecycle, collaboration, versions, approvals, locks, retention, and legal hold. +- `govoplan-addresses` owns persons, organizations, postal/email addresses, distribution lists, and recipient import/export. +- `govoplan-calendar` owns events, availability, resources, recurrence, and groupware calendar adapters. +- `govoplan-scheduling` owns meeting scheduling polls, participant availability collection, candidate-slot ranking, and decision handoff. +- `govoplan-appointments` owns public/internal appointment-booking workflows. +- `govoplan-idm` owns directory, provisioning, and identity-provider integration; `govoplan-access` consumes resolved principals and permissions. +- `govoplan-reporting` owns report definitions, BI views, scheduled outputs, and export targets. +- `govoplan-templates` owns reusable renderable templates, not data selection or persistence. + +## Release Tooling Note + +`scripts/push-release-tag.sh` covers the released full-product package set: + +- `govoplan-access` +- `govoplan-admin` +- `govoplan-tenancy` +- `govoplan-policy` +- `govoplan-audit` +- `govoplan-files` +- `govoplan-mail` +- `govoplan-campaign` +- `govoplan-calendar` +- `govoplan-core` + +It also includes existing roadmap/scaffold module repositories such as addresses, appointments, connectors, DMS, forms, IDM, reporting, scheduling, templates, workflow, XOE/V, and XRechnung as tag-only repositories. Tag-only repositories are committed, tagged, and pushed with the same release tag, but they are not added to `requirements-release.txt` or `webui/package.release.json` until they contain installable package metadata. + +Proposed modules without repositories, such as `govoplan-datasources`, `govoplan-dataflow`, and possibly `govoplan-projects`, cannot be included until their repositories are created. diff --git a/Repo-docs-MODULE-ARCHITECTURE.md b/Repo-docs-MODULE-ARCHITECTURE.md new file mode 100644 index 0000000..8e60513 --- /dev/null +++ b/Repo-docs-MODULE-ARCHITECTURE.md @@ -0,0 +1,506 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/MODULE_ARCHITECTURE.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# GovOPlaN Module Architecture + +GovOPlaN is structured as a platform kernel plus installable modules. The kernel starts and composes the platform. Modules own product behavior and contribute backend routes, database metadata, permissions, WebUI routes, navigation metadata, capabilities, and events. + +The current package name is still `govoplan-core`, but the architecture target is a smaller kernel. Access, tenancy, policy, audit, and admin semantics are platform-module responsibilities and should be extracted in stages. + +The concrete access/auth/RBAC extraction path is tracked in +[`ACCESS_EXTRACTION_PLAN.md`](ACCESS_EXTRACTION_PLAN.md). + +## Layer Model + +| Layer | Purpose | Examples | +| --- | --- | --- | +| Kernel | Bootstraps and composes the platform | app factory, module registry, route aggregation, migration orchestration, capability/event contracts, health metadata | +| Platform modules | Cross-cutting governance capabilities | access, tenancy, policy, audit, admin, ops | +| Service modules | Reusable operational capabilities | files, mail, templates, recipients, notifications | +| Business modules | Public-sector workflows | campaigns, cases, forms, approvals, appointments | +| Connector modules | External system integration | FIT-Connect, XÖV/XTA, DMS/eAkte, ERP, IDM | + +## Kernel Responsibilities + +The kernel target owns: + +- the server entry point and platform configuration +- module discovery, manifest validation, registry validation, route aggregation, and platform metadata APIs +- database engine/session lifecycle and module migration orchestration +- capability registry, command/event contracts, and lifecycle hooks +- shared WebUI shell contracts, generic WebUI components, and module route/nav rendering +- centralized mapping from serializable icon names to renderable frontend icons +- health, OpenAPI aggregation, and runtime diagnostics + +The kernel must not own product semantics such as users, tenants, RBAC decisions, governance policies, audit storage, mail behavior, file behavior, or campaign behavior. Those belong to platform, service, or business modules. + +## Current Compatibility Responsibilities + +During the staged split, `govoplan-core` still contains compatibility surfaces +for access, auth, tenancy, RBAC, governance, audit, CSRF/API helpers, and +secret helpers. The extracted access implementation now lives in +`govoplan-access`; live legacy ORM table definitions have been split across +their platform owners while retaining historical table names. The old core +model, route, admin-service, and access-security import shims have been +removed; callers must use module-owned imports or kernel capabilities. The +remaining compatibility surfaces are temporary until the matching platform +modules are fully self-contained: + +- `govoplan-access` +- `govoplan-tenancy` +- `govoplan-policy` +- `govoplan-audit` +- `govoplan-admin` + +New code should avoid deepening these compatibility dependencies. Prefer explicit kernel contracts and module capabilities over direct imports. + +Core must not import module feature pages or module business logic directly. It should interact with modules through manifests, entry points, metadata, capabilities, events, and route contributions. + +## Stable Kernel Contracts + +The following contracts are the baseline API that modules can rely on: + +- `ModuleManifest` +- `ModuleCompatibility` +- module uninstall guard provider contract +- `MigrationSpec` +- route factory contract +- capability factory contract +- access DTO/protocol contracts in `govoplan_core.core.access` +- resource ACL provider contract +- tenant summary provider contract +- tenant delete-veto provider contract +- WebUI module contribution contract +- navigation metadata contract +- command/event envelope contract + +Changes to these contracts must be versioned or accompanied by compatibility shims. + +Known access-related capability names are defined in +`govoplan_core.core.access`, including: + +- `access.principalResolver` +- `access.directory` +- `access.permissionEvaluator` +- `access.resourceAccess` +- `access.tenantProvisioner` +- `access.administration` +- `access.governanceMaterializer` +- `tenancy.tenantResolver` +- `security.secretProvider` +- `audit.sink` + +`govoplan-access` currently registers `access.principalResolver`, +`access.permissionEvaluator`, `access.directory`, `access.tenantProvisioner`, +`access.administration`, and `access.governanceMaterializer`. +`govoplan-tenancy` registers `tenancy.tenantResolver`. The minimal +authenticated platform set is now `tenancy` plus `access`; the registry +inserts `tenancy` before `access` when only feature modules are requested. +Feature modules should prefer these capabilities over direct reads of +access/tenant ORM models when they need labels, group membership, default +access provisioning, counts, audit actor labels, or tenant metadata. + +FastAPI route dependencies for authenticated endpoints are access-owned and +published from `govoplan_access.backend.auth.dependencies`. Routers may import +that dependency module directly until a more generic request-principal adapter +exists; they must not import access ORM models or other access implementation +internals. + +Current live table ownership: + +- `govoplan-tenancy`: `tenants` +- `govoplan-access`: `accounts`, `users`, `groups`, `roles`, + `system_role_assignments`, `user_group_memberships`, + `user_role_assignments`, `group_role_assignments`, `api_keys`, + `auth_sessions` +- `govoplan-admin`: `governance_templates`, + `governance_template_assignments` +- `govoplan-audit`: `audit_log` +- `govoplan-core`: `system_settings` + +Current admin route ownership follows the same boundary: access contributes +users, groups, roles, system accounts/roles, auth, sessions, and API-key +administration; tenancy contributes tenant registry/settings routes; admin +contributes system settings, overview, and governance-template routes; audit +contributes audit-log routes. Governance template metadata and assignment +routes live in `govoplan-admin`; materializing those templates into +access-owned groups and roles is performed by the +`access.governanceMaterializer` capability. + +Current admin WebUI ownership mirrors that route split. `govoplan-access` +contributes the `/admin` route shell and admin nav item. Other platform modules +contribute individual admin sections through the `admin.sections` UI capability. +`govoplan-admin` contributes the overview, system settings, and governance +template sections through that capability. Access-owned tenant/user/group/role +sections remain in the access package until their owning platform modules take +them over. + +Cross-module feature contracts live under focused kernel contract modules. For +example, `govoplan_core.core.campaigns` defines +`campaigns.access`, `campaigns.mailPolicyContext`, +`campaigns.policyContext`, `campaigns.deliveryTasks`, and +`campaigns.retention`. The campaign module registers these capabilities so mail +can resolve campaign owner/policy context and delivery tasks, files can validate +campaign file-share access, and core retention can call campaign-owned cleanup +logic without importing campaign ORM models. Keep these contracts small +DTO/protocol surfaces and register concrete behavior from the owning module. + +## Module Responsibilities + +A module owns one bounded feature area. A module can include both backend and WebUI code in the same repository so feature behavior and frontend integration evolve together. + +A module owns: + +- backend routers and feature services +- SQLAlchemy models for module-owned tables +- module migrations and migration metadata +- module permissions and role templates +- module-specific schemas, policies, and domain rules +- WebUI pages, feature-specific components, API clients, route contributions, and navigation metadata + +A module should not own generic platform UI. If a component is useful outside one module, move it to `@govoplan/core-webui` and parameterize it there before reusing it. + +## Backend Contract + +Backend modules register through the `govoplan.modules` entry point and expose a `ModuleManifest`. + +Example: + +```toml +[project.entry-points."govoplan.modules"] +files = "govoplan_files.backend.manifest:get_manifest" +``` + +The manifest should declare: + +- `id`, `name`, `version` +- `compatibility` when the module needs a minimum/maximum core version or a + newer manifest contract +- required `dependencies` and `optional_dependencies` +- permissions and role templates +- router factory +- migration metadata and script location +- frontend package metadata +- navigation metadata using serializable icon names +- uninstall guard providers for data, migration, worker, or scheduler vetoes + +Backend nav metadata must use icon-name strings, not frontend components: + +```python +NavItem( + path="/files", + label="Files", + icon="folder", + required_any=("files:file:read",), + order=40, +) +``` + +## Database And Migrations + +Core owns the database/session lifecycle. Modules access the database through core session dependencies and register their models/migrations through their manifest. + +Rules: + +- Do not create independent database engines in modules. +- Use core session dependencies, base metadata, and migration orchestration. +- Keep module-owned tables and migrations in the module repository. +- Keep cross-module foreign-key assumptions explicit and conservative. +- Register module metadata in `MigrationSpec` so core can discover it. +- Optional module migrations may create multiple Alembic heads. Verification + should compare the database heads to the configured script heads instead of + assuming one linear revision when multiple modules are enabled. + +## WebUI Contract + +A WebUI module exports a `PlatformWebModule` from its package. The object contributes local/fallback metadata and route render functions. + +Example: + +```ts +export const filesModule: PlatformWebModule = { + id: "files", + label: "Files", + version: "1.0.0", + dependencies: ["access"], + navItems: [ + { to: "/files", label: "Files", iconName: "folder", anyOf: ["files:file:read"], order: 40 } + ], + routes: [ + { path: "/files", anyOf: ["files:file:read"], order: 40, render: ({ settings, auth }) => createElement(FilesPage, { settings, auth }) } + ] +}; +``` + +WebUI modules receive only the core route context: + +- `settings` +- `auth` + +A module should call its own API client and module-owned backend routes. Shared API helpers should live in core only when they are truly platform-level concerns. + +Modules can also contribute named UI capabilities for explicit extension +points. Capability values must be narrow, typed contracts, not imports from a +sibling feature package. For admin pages, modules contribute: + +```ts +const adminSections: AdminSectionsUiCapability = { + sections: [ + { + id: "system-settings", + label: "General", + group: "SYSTEM", + order: 10, + allOf: ["system:settings:read"], + render: ({ settings, auth }) => createElement(SystemSettingsPanel, { settings, auth }) + } + ] +}; +``` + +The access admin route shell collects all installed `admin.sections` +capabilities with `usePlatformUiCapabilities("admin.sections")`, filters them +by `anyOf`/`allOf`, and renders them without importing the contributing module's +components directly. + +## Icon Rules + +Icons are resolved centrally by core. + +Modules must provide icon names with `iconName` in frontend nav contributions and `icon` in backend manifest metadata. Modules must not import Lucide icons for navigation metadata. + +Current core icon names include: + +- `activity` +- `admin` +- `campaign` +- `dashboard` +- `file` +- `files` +- `folder` +- `form` +- `mail` +- `reports` +- `users` + +If a module needs a new navigation icon, add the name-to-component mapping in core first, then use the name in backend and frontend metadata. + +The access module uses the `admin` icon for its `/admin` route. Core only +resolves that icon name; it does not hard-code the admin route in the rail. + +## Shared Component Rules + +Use this rule of thumb: + +- If it is platform-level or likely reusable by more than one module, define it in `@govoplan/core-webui` with parameters. +- If it is feature-specific and only meaningful inside one bounded module, keep it in that module. +- Modules must not import components from another feature module. +- If one module needs a component currently owned by another module, promote a generic version into core and replace the old usage with the core component. + +Examples: + +- `ExplorerTree` is core because files, mailboxes, and future modules can all render hierarchical navigation. +- `MessageDisplayPanel` is core because mail, campaign sending, and later audit/review surfaces can display message-like content. +- `AdminPageLayout`, `AdminIconButton`, and `AdminSelectionList` are core + because access, admin, tenancy, policy, and audit panels share the same admin + shell language. +- `MailProfileManagement` remains in the mail module because it is specific to mail transport policies and profiles. + +## Cross-Module Integration + +A module can declare required dependencies and optional dependencies. Optional behavior should be enabled by module presence and permissions, not by importing another module's WebUI internals. + +Rules: + +- Use core module metadata to check whether another module is installed. +- Use backend APIs/events/service contracts for runtime cooperation. +- If a sibling module needs owner-specific data, expose a narrow DTO/protocol + capability from the owning module instead of importing its ORM models. +- Keep UI integration declarative where possible: nav items, route contributions, context actions, and explicit extension points. +- Avoid direct imports from one feature module into another feature module unless the imported package is a published API contract designed for that purpose. UI components should be promoted to core instead. + +### Dependency Boundary Enforcement + +The repository includes `scripts/check_dependency_boundaries.py`. It enforces the current baseline: + +- kernel/core source may not add new direct imports of files/mail/campaign internals +- access source may not import files/mail/campaign internals +- feature modules may not import access implementation internals +- feature modules may not add new direct imports of sibling feature modules +- FastAPI routers may import the published + `govoplan_access.backend.auth.dependencies` dependency API +- the transitional allowlist is expected to stay empty + +Any future exception is extraction debt and must be temporary, documented in the +script with a reason, and removed when a capability/API/event contract replaces +it. + +## Module Lifecycle + +Core exposes the installed module catalog through the admin API and WebUI. The +current lifecycle model separates four states: + +- installed: the Python/WebUI package is available to the process +- active: the module is present in the running platform registry +- desired: the module should be active on the next server startup +- planned package change: an operator-reviewed package install/uninstall item + saved in system settings but not executed by the running server + +The admin module manager can change the desired enabled set and apply it to the +running server. It always keeps `tenancy`, `access`, and `admin` enabled when +saving through the admin UI, and it adds required module dependencies before +saving the desired state. On startup, core always keeps the minimum +authenticated platform set `tenancy`/`access` enabled and keeps `admin` enabled +when the operator configuration includes it. Unknown saved module ids are +ignored when the matching package is no longer installed. The core app factory, +devserver, development bootstrap, background worker registry, and migration +metadata plan all read the saved desired state from `system_settings` before +building their module registry. + +Hot enable/disable is a core design principle for every module: + +- Core keeps one mutable active `PlatformRegistry` object and swaps its manifest + set through the module lifecycle manager. Modules must read module presence, + optional integrations, permissions, role templates, capabilities, navigation, + and frontend contributions from that registry instead of caching sibling + module availability. +- Core validates install state and dependency closure before activation. +- Core applies configured module migrations before activation. Deactivation + never drops tables or data. +- Core mounts module routers once and guards them by active module state. A + deactivated module's routes remain mounted internally but return a disabled + module response until the module is active again. +- Module route factories must be side-effect-light and idempotent. They may + configure module runtime references, but they must not start workers, + schedulers, or irreversible external subscriptions. Use lifecycle hooks for + those resources. +- Modules that own persistent data, background jobs, schedulers, external + subscriptions, or irreversible migration state must expose uninstall guard + providers through their manifest. Guards return `blocker`, `warning`, or + `info` results and may inspect live state through the core-owned DB session. + A guard failure is treated as a blocker. +- Core refreshes the active registry before frontend metadata is returned from + `/api/v1/platform/modules`; the WebUI shell refetches this metadata after + module changes so navigation, routes, and UI capabilities update without a + page reload. +- Modules can provide `on_activate` and `on_deactivate` hooks for worker, + scheduler, cache, or external subscription lifecycle. These hooks must be + idempotent and must not mutate another module directly. +- Package install/uninstall is performed by the trusted operator installer, not + directly inside FastAPI request handlers. The admin UI can save install plans, + show preflight blockers, and activate/deactivate installed packages. + +The package install-plan API records operator intent only: + +- `GET /api/v1/admin/system/modules/install-plan` reads the saved plan, + renders shell commands, and returns installer preflight status. +- `PUT /api/v1/admin/system/modules/install-plan` saves planned install or + uninstall rows. Install rows must use tagged package or git references, not + local `file:`/workspace paths. +- `DELETE /api/v1/admin/system/modules/install-plan` clears the plan. +- `govoplan-module-install-plan --format shell` or + `python -m govoplan_core.commands.module_install_plan --format shell` renders + the same commands from a server shell. +- `govoplan-module-installer --format shell` runs the same preflight checks from + the server shell. +- `govoplan-module-installer --apply --build-webui` executes the saved plan + after preflight passes, snapshots `pip freeze` and WebUI package files, writes + a run record under the runtime installer directory, and marks planned rows as + applied after success. +- `govoplan-module-installer --supervise --health-url http://127.0.0.1:8000/health --restart-command ''` + is the preferred disruptive-change path. It applies the plan, runs the + restart command if provided, polls health, and automatically rolls packages + back from the run snapshot if commands, restart, or health recovery fail. +- `govoplan-module-installer --rollback ` restores the saved package + snapshots and reruns package installation from the previous freeze file. + +The installer preflight is intentionally conservative: + +- maintenance mode must be active; +- installed module manifests must be compatible with the supported manifest + contract and current core version; +- uninstalling `tenancy`, `access`, or `admin` is blocked; +- uninstalling an active module is blocked; +- uninstalling a module still present in desired startup state is blocked; +- uninstalling a module with active/desired dependents is blocked; +- module-owned uninstall guard providers can veto data/migration/worker unsafe + removals; +- install refs must be exact versions or tagged git refs; +- WebUI package changes require a WebUI root and trigger rebuild/reload status. + +The installer supervisor must run outside the FastAPI server process. A server +request handler cannot reliably restart or roll back the process that is +currently executing the request. The admin UI therefore remains an operator +planning surface; the trusted daemon/CLI is the executor. + +The running FastAPI server still reports `package_mutation_supported=false` +because dependency-manager operations are not executed inside request handlers. +The trusted mutation boundary is the operator CLI. This keeps the interpreter, +npm dependency graph, frontend bundle, migrations, and worker process set under +process-supervisor control. + +Frontend module loading currently uses the build-time package graph generated by +the core WebUI host. Therefore installing or uninstalling a WebUI package +requires `npm install` plus a WebUI rebuild/reload. True remote runtime bundle +loading would need signed asset manifests, version compatibility checks, CSP +rules, cache invalidation, and rollback at the asset-loader layer; it is a +future deployment hardening track, not the default module installer path. + +## Maintenance Mode + +Maintenance mode is the required operating state for package install/uninstall +and other disruptive system maintenance. + +Core stores maintenance mode in `system_settings.settings.maintenance_mode`. +The public platform status endpoint exposes only the flag and message so the +WebUI can show a clear login-screen notice. Login remains reachable during +maintenance so an operator can sign in. + +Authenticated API access is enforced at the access-principal boundary. When +maintenance mode is enabled, authenticated requests require the system scope +`system:maintenance:access`; otherwise the API returns `503 Service +Unavailable` with a maintenance-mode detail payload. The protected +`system_owner` role grants this through `system:*`. A dedicated +`maintenance_operator` role exists for accounts that should be able to access +the system during maintenance without receiving broad write permissions. + +Changing the maintenance-mode flag requires both system settings write access +and `system:maintenance:access`, so an administrator cannot accidentally enable +a mode they cannot use. + +The first implementation is a platform access gate. It does not replace +database backups, process supervision, migration checks, or external load +balancer maintenance pages. + +## Build And Verification + +Backend verification from core: + +```bash +cd /mnt/DATA/git/govoplan-core +./.venv/bin/python -m compileall src/govoplan_core ../govoplan-access/src/govoplan_access ../govoplan-admin/src/govoplan_admin ../govoplan-tenancy/src/govoplan_tenancy ../govoplan-policy/src/govoplan_policy ../govoplan-audit/src/govoplan_audit ../govoplan-files/src/govoplan_files ../govoplan-mail/src/govoplan_mail ../govoplan-campaign/src/govoplan_campaign +./.venv/bin/python scripts/check_dependency_boundaries.py +``` + +Core WebUI host verification: + +```bash +cd /mnt/DATA/git/govoplan-core/webui +PATH=/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run build +``` + +Clean generated `dist`, `.vite`, and source-tree `__pycache__` artifacts after verification unless they are intentionally part of a release artifact. + +## Release Dependency Rules + +Local development may use editable Python installs and local WebUI `file:` dependencies so sibling module changes reload quickly. Release builds must use tagged git refs or published packages instead. Core provides: + +- `requirements-dev.txt` for local editable backend installs +- `requirements-release.txt` for tagged backend module installs +- `webui/package.release.json` for tagged WebUI module installs + +Module repositories include root-level npm manifests for git installs. When cutting a release, update the Python versions, WebUI versions, release dependency refs, and repository tags together. diff --git a/Repo-docs-RBAC-MANIFEST.md b/Repo-docs-RBAC-MANIFEST.md new file mode 100644 index 0000000..037ca6d --- /dev/null +++ b/Repo-docs-RBAC-MANIFEST.md @@ -0,0 +1,298 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/RBAC_MANIFEST.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Multi Seal Mail - Current RBAC and Resource-Access Model + +**Updated:** 2026-06-16 +**Current migration head:** `f5a6b7c8d9e0` + +## Authorization Equation + +An operation is permitted only when every applicable layer allows it: + +```text +effective role/API-key capability +AND resource ownership/share access +AND workflow state +AND active governance/policy constraints +``` + +RBAC answers what an actor may do. ACLs answer which resource the actor may do it to. Workflow state and policy decide whether the operation is currently valid. + +## Identity and Scope + +```text +Account global login identity ++- User membership tenant-local identity + +- direct tenant roles + +- active group memberships + | +- inherited tenant roles + +- tenant-local API keys + +Account ++- direct system-role assignments +``` + +A browser session has one active tenant membership. System privileges do not silently grant tenant data access. API keys remain tenant-local and receive the intersection of their configured scopes and their owner's live tenant scopes on every request. + +## Wildcards + +```text +tenant:* every canonical tenant permission +system:* every canonical system permission +* legacy alias interpreted as tenant:* only +``` + +Tenant wildcards never grant system permissions. + +## Canonical Tenant Permissions - 53 + +### Campaigns + +```text +campaign:read +campaign:create +campaign:update +campaign:copy +campaign:archive +campaign:delete +campaign:share +campaign:validate +campaign:build +campaign:review +campaign:send_test +campaign:queue +campaign:control +campaign:send +campaign:retry +campaign:reconcile +``` + +### Recipients + +```text +recipients:read +recipients:write +recipients:import +recipients:export +``` + +### Files + +```text +files:read +files:download +files:upload +files:organize +files:share +files:delete +files:admin +``` + +### Reports and Audit + +```text +reports:read +reports:export +reports:send +audit:read +``` + +### Mail Servers + +```text +mail_servers:read +mail_servers:use +mail_servers:test +mail_servers:write +mail_servers:manage_credentials +``` + +### Tenant Administration + +```text +admin:users:read +admin:users:create +admin:users:update +admin:users:suspend + +admin:groups:read +admin:groups:write +admin:groups:manage_members + +admin:roles:read +admin:roles:write +admin:roles:assign + +admin:api_keys:read +admin:api_keys:create +admin:api_keys:revoke + +admin:settings:read +admin:settings:write +admin:policies:read +admin:policies:write +``` + +## Canonical System Permissions - 18 + +```text +system:tenants:read +system:tenants:create +system:tenants:update +system:tenants:suspend + +system:accounts:read +system:accounts:create +system:accounts:update +system:accounts:suspend + +system:roles:read +system:roles:write +system:roles:assign + +system:access:read +system:access:assign + +system:audit:read +system:settings:read +system:settings:write +system:governance:read +system:governance:write +``` + +`system:access:*` remains as a compatibility/read and assignment boundary for cross-tenant/system access handling. It is not a separate primary UI area. + +## Default Tenant Roles + +- **Owner:** `tenant:*`. At least one active operational owner must remain. +- **Tenant administrator:** settings, policies, users, groups, roles and API keys plus read access to campaigns/files/reports/audit. Real delivery remains separately delegable. +- **Administrator (legacy):** all tenant permissions for upgraded installations. +- **Access administrator:** membership and assignment management within delegation limits. +- **Campaign manager:** prepare, validate and build campaigns; no review approval or real delivery by default. +- **Reviewer:** inspect and approve prepared campaign messages. +- **Sender:** mock-test, queue, control, send, retry and reconcile prepared campaigns; can use/test approved mail profiles. +- **File manager:** managed file operations without campaign delivery rights. +- **Viewer:** read campaigns, recipients, files and reports. +- **Auditor:** read campaigns, recipient evidence, reports and audit records; export detailed evidence. + +## Default System Roles + +- **System owner:** `system:*`, protected. At least one active account must retain it. +- **System administrator:** all specific system permissions, editable and not protected. +- **System auditor:** read-only system registry/settings/governance/audit role, editable. + +## Delegation Ceiling + +For role definition, assignment and API-key creation: + +```text +requested scopes subset of actor delegateable scopes +``` + +Rules: + +1. Tenant roles may contain tenant scopes only. +2. System roles may contain system scopes only. +3. Definition rights and assignment rights are separate. +4. Group definition and group membership management are separate. +5. API-key scopes are intersected with the owner's current effective scopes on every request. +6. Suspended accounts, users, tenants or groups stop contributing access immediately. +7. Administrative updates are field-sensitive; a user with only status authority cannot change role assignments. + +## Campaign Ownership and ACLs + +A campaign has exactly one owner: + +```text +owner user OR owner group +``` + +Additional active shares may target users or groups with `read` or `write`. + +Resolution: + +- owner user: read and write; +- member of owner group: read and write; +- explicit read share: read; +- explicit write share: read and write; +- `tenant:*`: tenant-wide ACL bypass; +- ordinary campaign permission without ownership/share: no object access. + +ACLs do not add capabilities. A write share still needs the specific permission for update, validation, review, send, report, retry or reconciliation. + +## Sensitive Recipient Boundary + +Recipient-complete campaign JSON, message data and job detail require `recipients:read`. Recipient edits require `recipients:write`; exports require `recipients:export`; import is reserved for the dedicated recipient import/list workflow. + +## Files + +| Permission | Operations | +|---|---| +| `files:read` | list, search, inspect, resolve metadata | +| `files:download` | download file bytes and generated ZIP archives | +| `files:upload` | upload files and ZIP contents | +| `files:organize` | create folders, rename, move, copy and bulk rename | +| `files:share` | create/revoke file shares | +| `files:delete` | delete/hide files and folders subject to retention | +| `files:admin` | tenant-wide administration of user/group file spaces | + +## Mail Servers + +| Permission | Boundary | +|---|---| +| `mail_servers:read` | profile metadata and effective policy visibility | +| `mail_servers:use` | select an approved profile without reading secrets | +| `mail_servers:test` | run server-side connection tests | +| `mail_servers:write` | define/edit profiles in allowed scopes | +| `mail_servers:manage_credentials` | create/replace SMTP/IMAP secrets or campaign-level credentials where policy allows | + +Reusable encrypted profiles now exist. Effective usability is also constrained by hierarchical mail-profile policy, ownership, allowed/forced profile sets, credential inheritance mode, the lower-level override switch for that mode, and allow/deny patterns. + +## Sessions, API Keys and CSRF + +- Browser login creates an HttpOnly session cookie and a separate readable CSRF cookie. +- Unsafe cookie-authenticated requests require matching CSRF cookie/header and stored CSRF hash. +- API keys remain supported for CLI/automation and do not use browser CSRF. +- Login responses still expose a compatibility session token in the response body; the WebUI does not persist it. + +## Legacy Compatibility + +Runtime aliases remain only for names that are no longer canonical, including: + +```text +campaign:write +attachments:read +attachments:write +admin:users +admin:users:write +admin:api_keys:write +admin:settings +system:tenants:write +system:access:write +``` + +Canonical scopes are not widened by runtime alias expansion after migration. + +## Deferred Permission Families + +Add these only with their corresponding implemented features: + +```text +templates:* +address_books:* +recipient_lists:* +connectors:* +dsar:* +system:monitoring:read +system:backups:run +system:backups:restore +system:updates:apply +system:updates:rollback +``` + +A separate `retention:*` family is not currently canonical because retention is managed through system settings and tenant policy scopes. Add it only if retention operation duties need separation from general policy/settings administration. diff --git a/Repo-docs-SYSTEM-GOVERNANCE-MANIFEST.md b/Repo-docs-SYSTEM-GOVERNANCE-MANIFEST.md new file mode 100644 index 0000000..94bc982 --- /dev/null +++ b/Repo-docs-SYSTEM-GOVERNANCE-MANIFEST.md @@ -0,0 +1,186 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan-core/docs/SYSTEM_GOVERNANCE_MANIFEST.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Multi Seal Mail - Current System and Tenant Governance Model + +**Updated:** 2026-06-16 +**Current migration head:** `f5a6b7c8d9e0` + +## Governance Rule + +System policy is authoritative for tenants and all lower levels. Each lower level may only narrow what it inherits: + +```text +system +-> tenant +-> user or group owner +-> campaign +``` + +Lower levels do not widen privileges, allowed profiles, retention durations or credential rights granted by a higher level. + +## Administration Structure + +```text +SYSTEM +- Settings +- Retention +- Mail servers +- Tenants +- Users +- Groups +- System roles +- Tenant roles +- Audit + +TENANT +- Settings boundary +- Users +- Groups +- Roles +- API keys +- Mail servers +- Retention +- Audit + +USER +- User mail +- User retention + +GROUP +- Group mail +- Group retention +``` + +There is no separate System access page. Compatibility access scopes remain in the backend for assignment/read boundaries. + +## Tenant Governance + +System settings define tenant defaults and whether tenants may narrow selected options. Tenant overrides can only restrict: + +- custom groups; +- custom roles; +- tenant API keys. + +The backend enforces that tenant governance cannot widen system-denied privileges. + +## Mail-Profile Governance + +Mail server profiles may exist at these scopes: + +```text +system +tenant +user +group +campaign +``` + +Effective campaign profile availability follows campaign ownership. A campaign owned by a user resolves through system, tenant, that user and campaign policy. A group-owned campaign resolves through system, tenant, that group and campaign policy. + +Policy semantics: + +- higher levels define the maximum available profile set; +- lower levels can further restrict the set; +- forced profiles mean the lower level must choose from the forced set; +- a forced set with one profile effectively enforces that profile; +- campaign-level profile creation is allowed only if the effective policy permits it; +- SMTP/IMAP credentials use one inheritance decision per protocol: lower levels must inherit profile credentials, may inherit profile credentials, or must provide local credentials; +- the lower-level override switch for `smtp_credentials.inherit` and `imap_credentials.inherit` controls whether descendants may change that inheritance decision; +- deny patterns always win over allow patterns; +- empty or `*` allowlist means allow all except denied; +- non-empty allowlist means at least one allow rule must match and no deny rule may match. + +Pattern targets: + +```text +SMTP hostname +IMAP hostname +envelope sender +From header +recipient domains +``` + +Ownership transfer is intentionally deferred as a two-step workflow: original owner initiates, new owner accepts and reselects/repairs the mail profile if their effective policy requires it. + +## Retention Governance + +Retention policy is hierarchical: + +```text +system -> tenant -> user/group -> campaign +``` + +Managed fields: + +- raw campaign JSON retention days; +- generated EML retention days; +- stored report detail retention days; +- mock mailbox retention days; +- audit detail retention days. + +Rules: + +- system may set concrete defaults or unlimited retention; +- system exposes allow-limiting toggles per field; +- tenants, users/groups and campaigns may only shorten inherited retention where the parent allows limiting; +- blank lower-level values inherit; +- mock mailbox retention is currently system-level because mock mailbox records do not yet carry tenant/campaign ownership metadata; +- dry-run/apply retention actions report affected classes before destructive cleanup. + +## Role Definitions and Assignments + +### System roles + +System roles define instance-wide permissions. `system:*` is stored as one wildcard and displayed as granting the full system catalogue. System owner is protected. + +### Tenant roles + +Tenant roles can be system-governed templates or tenant-local definitions, subject to system tenant-governance settings and actor delegation ceilings. Wildcard counts are expanded against the canonical tenant catalogue. + +## Audit Access + +Audit access remains scope-separated: + +```text +system audit -> system:audit:read +tenant audit -> active tenant + audit:read +``` + +Audit pages use server pagination, filtering and bounded grids. + +## Tenant Switching + +Tenant switching preserves the current URL when possible and falls back when a route/resource is not accessible in the new tenant context. + +The tenant selector is hidden for ordinary single-tenant accounts and visible for multi-tenant or system tenant-management contexts. + +## DataGrid Contract in Administration + +Admin lists use bounded container grids: + +- one flexible fill column; +- fixed total table width; +- compact action/status/count columns; +- resizable text/date columns; +- no intrinsic content growth; +- sticky headers where needed; +- server pagination for audit. + +## Still Deferred + +- real SMTP/IMAP test-bed verification and operator runbook; +- recipient import with column mapping; +- Seafile/external connector governance; +- system/tenant/group/user file-space hierarchy and external storage hierarchy; +- session/device revocation UI; +- backup/restore, monitoring and update procedures; +- DSAR workflows and evidence bundle verifier; +- campaign ownership transfer workflow; +- policy impact analysis before delete/disable/unshare/change; +- LDAP/OIDC/SAML provisioning; +- destructive tenant erasure orchestration.