Sync Product govoplan-split-concept-action-plan from project files

Product govoplan-split-concept-action-plan.-.md

@@ -0,0 +1,453 @@
+<!-- codex-wiki-sync:a312d27742fc63ee61d3595a -->
+
+> Mirrored from `/mnt/DATA/Nextcloud/ADD ideas UG/Products/govoplan/split-concept-action-plan.md`.
+> Origin: `product:govoplan`.
+> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context.
+
+---
+# GovOPlaN Split Concept and Action Plan
+
+Source: `split.md`
+
+## Executive Concept
+
+GovOPlaN should become a modular governance and workflow platform, not a monolithic Fachverfahren and not a thin event bus only.
+
+The target structure is:
+
+- **Kernel**: small platform runtime that composes modules.
+- **Platform modules**: cross-cutting capabilities such as access, tenancy, policy, audit, admin, operations.
+- **Service modules**: reusable capabilities such as files, mail, templates, recipients, notifications.
+- **Business modules**: public-sector workflows such as campaigns, cases, forms, approvals, appointments.
+- **Connector modules**: integrations with external public-sector systems and standards.
+
+The core idea:
+
+```text
+govoplan-kernel starts the platform.
+Installed modules register routes, models, migrations, permissions, UI routes, capabilities, and events.
+Modules cooperate through kernel contracts, not direct imports.
+```
+
+This keeps every module independently installable while still allowing richer behavior when other modules are present.
+
+## Target Architecture
+
+### Kernel
+
+The kernel owns only platform composition:
+
+- app factory and server assembly
+- module discovery and registry
+- manifest validation
+- route aggregation
+- capability registry
+- command/event contracts
+- lifecycle hooks
+- configuration loading
+- database/session lifecycle
+- module migration orchestration
+- health and platform metadata
+- shared WebUI shell contracts
+- OpenAPI aggregation
+
+The kernel must not own product semantics such as users, tenants, RBAC rules, audit policy, mail behavior, file behavior, or campaign behavior.
+
+### Platform Modules
+
+These are cross-cutting but still modules:
+
+- `govoplan-access`: accounts, authentication, sessions, groups, memberships, RBAC, API keys, principal resolution, login adapters.
+- `govoplan-tenancy`: tenant registry, tenant lifecycle, tenant settings boundary, tenant switching, tenant deletion/erasure orchestration.
+- `govoplan-policy`: permission catalogue, hierarchical policy engine, delegation ceilings, explain decisions, impact simulation.
+- `govoplan-audit`: audit sink, audit queries, evidence exports, retention, compliance reports.
+- `govoplan-admin`: system/tenant admin UI and administrative workflows over access, tenancy, policy, audit, and operations.
+- `govoplan-ops`: monitoring, backup/restore, update/rollback, maintenance windows, system runbooks.
+
+### Existing Service and Business Modules
+
+Current modules keep their ownership:
+
+- `govoplan-files`: managed files, file spaces, sharing, downloads, file metadata, future legal hold/version lock/DMS integration.
+- `govoplan-mail`: SMTP/IMAP profiles, credentials, mail policy, sending, append-to-Sent, mailbox views, future inbound mail-to-case.
+- `govoplan-campaign`: campaign authoring, validation, review/send control, reports, optional file/mail integration.
+
+### Future Module Families
+
+Recommended future modules:
+
+- Work management: `govoplan-cases`, `govoplan-workflow`, `govoplan-tasks`, `govoplan-forms`, `govoplan-appointments`, `govoplan-templates`.
+- Communication/content: `govoplan-notifications`, expanded files/mail/campaign integrations.
+- Connectors: `govoplan-connectors`, `govoplan-fit-connect`, `govoplan-xoev`, `govoplan-xta-osci`, `govoplan-dms`, `govoplan-erp`, `govoplan-xrechnung`.
+- Citizen/public interaction: `govoplan-portal`, `govoplan-identity-trust`, `govoplan-payments`.
+- Insight/compliance: `govoplan-search`, `govoplan-reporting`, `govoplan-data-catalog`, `govoplan-compliance`.
+
+## Non-Negotiable Design Principles
+
+- Modules must run when optional sibling modules are physically absent.
+- Modules must not import optional module internals directly.
+- Optional behavior must go through kernel contracts: capabilities, metadata, registry queries, events, commands, and API boundaries.
+- Kernel owns database/session lifecycle and migration orchestration; modules own models and migrations.
+- Shared WebUI shell and generic components belong in core/kernel WebUI.
+- Module WebUI packages contribute nav/routes/pages through manifests and module contracts.
+- Every policy decision should be explainable, not just boolean.
+- Audit should subscribe to platform/domain events instead of being embedded in the kernel.
+- Production eventing should use an outbox pattern; in-process events are acceptable for local/dev.
+
+## Staged Refactor Plan
+
+### Phase 0: Stabilize Current Modular Baseline
+
+Goal: make the current core/files/mail/campaign split reliable before deeper extraction.
+
+Action items:
+
+- [ ] Document current module contract as the compatibility baseline.
+- [ ] Define which APIs are stable kernel contracts and which are temporary compatibility paths.
+- [ ] Add dependency-boundary checks that forbid direct optional module imports.
+- [ ] Keep module-matrix tests for core-only, files-only, mail-only, campaign-only, campaign+files, campaign+mail, full product.
+- [ ] Keep generated WebUI/test artifacts ignored.
+- [ ] Mark DataGrid work as explicitly deferred.
+
+Acceptance criteria:
+
+- Full product still runs.
+- Every supported module permutation builds and starts.
+- No optional module is required for another module to import.
+
+### Phase 1: Freeze Kernel Contracts
+
+Goal: make the kernel API explicit before extracting platform semantics.
+
+Action items:
+
+- [ ] Move or clearly mark these as kernel-stable contracts:
+  - `ModuleManifest`
+  - `MigrationSpec`
+  - capability factory contract
+  - route factory contract
+  - WebUI module contribution contract
+  - resource ACL provider contract
+  - tenant summary/delete-veto provider contract
+  - lifecycle hook contract
+  - event and command contracts
+- [ ] Add contract tests for manifests, route aggregation, migration registration, and capability discovery.
+- [ ] Add a documented deprecation policy for kernel API changes.
+- [ ] Add OpenAPI/route collision validation.
+- [ ] Add manifest schema/versioning.
+
+Acceptance criteria:
+
+- Modules can declare all backend and WebUI contributions without private imports.
+- The kernel can reject invalid manifests with actionable errors.
+
+### Phase 2: Extract Access
+
+Goal: turn current access/auth/RBAC functionality into a real platform module.
+
+Target repository/package:
+
+```text
+govoplan-access
+```
+
+Action items:
+
+- [ ] Move accounts, authentication, sessions, API keys, groups, memberships, roles, role assignments, and principal resolution into access.
+- [ ] Keep compatibility imports under `govoplan_core.access.*` temporarily with deprecation warnings.
+- [ ] Move access permissions and default role templates into the access manifest.
+- [ ] Move access migrations into the access module migration registration.
+- [ ] Move login/session/admin access UI into access/admin module contributions.
+- [ ] Define `PrincipalResolver` and related protocols in the kernel.
+- [ ] Ensure core-only startup still works enough to show shell/health, but access is required for authenticated product use.
+
+Acceptance criteria:
+
+- Kernel imports no access models directly.
+- Access can be installed as a module and provides authentication/RBAC.
+- Existing login and admin flows still work through module routes/UI contributions.
+
+### Phase 3: Extract Tenancy
+
+Goal: separate tenant lifecycle from access identity.
+
+Target repository/package:
+
+```text
+govoplan-tenancy
+```
+
+Action items:
+
+- [ ] Move tenant registry, tenant settings, tenant lifecycle, tenant switching, and tenant deletion orchestration into tenancy.
+- [ ] Keep memberships and role assignments in access.
+- [ ] Define `TenantResolver` and tenant-context protocols in the kernel.
+- [ ] Define tenant lifecycle events: created, suspended, resumed, deletion requested, erasure completed.
+- [ ] Add delete-veto orchestration through module providers.
+- [ ] Add tenant lifecycle tests with files/mail/campaign installed and absent.
+
+Acceptance criteria:
+
+- Tenant data ownership is clear.
+- Modules can register tenant delete-veto and cleanup behavior.
+- Access and tenancy interact through protocols/capabilities, not circular imports.
+
+### Phase 4: Extract Policy
+
+Goal: make policy the common decision engine for access, tenancy, mail, retention, campaign, files, and future modules.
+
+Target repository/package:
+
+```text
+govoplan-policy
+```
+
+Action items:
+
+- [ ] Define a common `PolicyDecision` object with:
+  - `allowed`
+  - `reason`
+  - `requirements`
+  - `matched_permissions`
+  - `denied_by`
+  - `source_path`
+  - `policy_version`
+- [ ] Move hierarchical policy evaluation into policy.
+- [ ] Move delegation ceilings and "more restrictive only" validation into policy.
+- [ ] Add explain endpoints and UI component contract for policy source paths.
+- [ ] Add policy simulation before destructive/limiting changes.
+- [ ] Move mail/retention effective policy display onto shared policy components.
+- [ ] Add regression tests for non-viable lower-level options being hidden or disabled.
+
+Acceptance criteria:
+
+- UI can explain "how we got here" for effective policy without module-specific logic.
+- Saving invalid policy states is prevented in the UI, not only rejected by backend.
+- Policy changes can be simulated for impact before applying.
+
+### Phase 5: Extract Audit
+
+Goal: make audit an event-consuming platform module.
+
+Target repository/package:
+
+```text
+govoplan-audit
+```
+
+Action items:
+
+- [ ] Define typed `PlatformEvent` envelope with actor, tenant, subject, resource, correlation, causation, classification, and payload.
+- [ ] Separate command bus from event bus.
+- [ ] Introduce outbox table and dispatcher for production-safe event delivery.
+- [ ] Move audit log storage/query/export into audit module.
+- [ ] Add event producers for access, tenancy, policy, files, mail, and campaign actions.
+- [ ] Add evidence bundle export as a later audit capability.
+
+Acceptance criteria:
+
+- Kernel does not own audit storage.
+- Audit records are produced by module events.
+- Events can be correlated across cases/files/mail/campaign/connectors.
+
+### Phase 6: Split Admin UI
+
+Goal: avoid one central admin page owning all platform semantics.
+
+Action items:
+
+- [ ] Define admin route/nav contribution contract.
+- [ ] Move access admin pages to access module.
+- [ ] Move tenant admin pages to tenancy module.
+- [ ] Move policy admin pages to policy module.
+- [ ] Move audit admin pages to audit module.
+- [ ] Keep only shell-level admin layout and module aggregation in kernel/core WebUI.
+
+Acceptance criteria:
+
+- Installing a platform module contributes its admin pages.
+- Removing a module removes its admin pages without broken imports/routes.
+
+### Phase 7: CI and Dependency Enforcement
+
+Goal: make modularity enforceable.
+
+Action items:
+
+- [ ] Add module permutation backend startup tests.
+- [ ] Add WebUI module permutation builds.
+- [ ] Add dependency import linting:
+  - campaign must not directly import files/mail internals except through explicit optional adapters/capabilities.
+  - files must not directly require campaign.
+  - mail must not directly require campaign.
+  - kernel must not import product/business module internals.
+- [ ] Add migration registration tests.
+- [ ] Add manifest schema tests.
+- [ ] Add compatibility import tests until old paths are removed.
+
+Acceptance criteria:
+
+- CI fails when optional module boundaries are violated.
+- CI proves each intended product composition can run.
+
+## Product Roadmap
+
+### Milestone A: Clean Platform Split
+
+Priority: highest.
+
+Action items:
+
+- [ ] Kernel contract freeze.
+- [ ] Access extraction.
+- [ ] Tenancy extraction.
+- [ ] Policy extraction.
+- [ ] Audit extraction.
+- [ ] Module-matrix CI.
+- [ ] Compatibility/deprecation plan.
+
+Outcome:
+
+GovOPlaN becomes a stable modular platform rather than a core-heavy product.
+
+### Milestone B: Daily Administrative Work
+
+Priority: next after clean split.
+
+Action items:
+
+- [ ] Create `govoplan-cases` concept and MVP.
+- [ ] Create `govoplan-tasks` concept and MVP.
+- [ ] Extract/generalize reusable workflow/review patterns into `govoplan-workflow`.
+- [ ] Add `govoplan-forms`.
+- [ ] Add `govoplan-templates`.
+- [ ] Add `govoplan-recipients` / `govoplan-address-book`.
+- [ ] Link existing files/mail/campaign to cases.
+
+Outcome:
+
+Municipalities, universities, and public-sector teams can use GovOPlaN for concrete daily work.
+
+### Milestone C: Public-Sector Integration Platform
+
+Priority: after work management MVP.
+
+Action items:
+
+- [ ] Build `govoplan-connectors` registry and connector runtime.
+- [ ] Add FIT-Connect connector.
+- [ ] Add XÖV validation/mapping module.
+- [ ] Add DMS/eAkte connector.
+- [ ] Add ERP/finance connector.
+- [ ] Add XRechnung workflow.
+- [ ] Add IDM/LDAP/OIDC/SAML/SCIM provisioning.
+
+Outcome:
+
+GovOPlaN coordinates existing systems instead of replacing every specialist application.
+
+### Milestone D: Governance Control Plane
+
+Priority: after platform and integration foundations.
+
+Action items:
+
+- [ ] Cross-module reporting.
+- [ ] Permission-aware search.
+- [ ] Data catalogue.
+- [ ] Compliance workspace.
+- [ ] Evidence bundle verifier.
+- [ ] Policy impact analysis.
+- [ ] Destructive tenant erasure orchestration.
+- [ ] Operations governance dashboard.
+
+Outcome:
+
+GovOPlaN becomes a governance, audit, and compliance layer over administrative work.
+
+## Immediate Backlog Items
+
+### P0: Architecture Safety
+
+- [ ] Write a concise kernel responsibility statement in `MODULE_ARCHITECTURE.md`.
+- [ ] Add "kernel must not own product semantics" as an architecture rule.
+- [ ] Add a dependency-boundary test or lint script.
+- [ ] Add backend startup tests for supported module permutations.
+- [ ] Add manifest schema/version validation.
+- [ ] Keep compatibility imports documented before extraction begins.
+
+### P0: Access Extraction Prep
+
+- [ ] Inventory all current `govoplan_core.access`, auth, user, account, group, API key, role, and permission imports.
+- [ ] Identify which imports belong to future `govoplan-access`.
+- [ ] Define `PrincipalResolver` protocol in kernel.
+- [ ] Define access module manifest target.
+- [ ] Create extraction checklist for models, migrations, routes, services, WebUI pages, and tests.
+
+### P0: Policy Extraction Prep
+
+- [ ] Inventory current mail, retention, RBAC, delegation, and governance policy logic.
+- [ ] Define shared `PolicyDecision`.
+- [ ] Define policy source path format.
+- [ ] Define backend explain endpoint shape.
+- [ ] Define frontend effective-policy component contract.
+- [ ] Add UI tests for blocked/disallowed lower-level policy choices.
+
+### P0: Audit/Event Prep
+
+- [ ] Define typed event envelope.
+- [ ] Decide command bus versus event bus API shape.
+- [ ] Add correlation/causation ID support.
+- [ ] Define audit module MVP boundaries.
+- [ ] Identify domain events from access, tenancy, policy, files, mail, campaign.
+
+### P1: Tenancy Extraction Prep
+
+- [ ] Inventory tenant model and tenant setting usage.
+- [ ] Define `TenantResolver` protocol.
+- [ ] Separate tenant lifecycle from membership/roles.
+- [ ] Define tenant deletion and module cleanup/veto contract.
+
+### P1: Admin UI Split Prep
+
+- [ ] Inventory admin pages by owning future module.
+- [ ] Define admin route contribution contract.
+- [ ] Move generic admin layout into core WebUI.
+- [ ] Keep module-specific admin panels in owning modules.
+
+### P1: Existing Module Hardening
+
+- [ ] Ensure campaign uses files/mail only through capabilities/metadata/API boundaries.
+- [ ] Ensure files does not require campaign.
+- [ ] Ensure mail does not require campaign.
+- [ ] Keep mailbox, mail policy, mail credential inheritance, and file storage behavior covered by focused tests.
+
+### P2: Future Product Concepts
+
+- [ ] Write `govoplan-cases` concept.
+- [ ] Write `govoplan-workflow` concept.
+- [ ] Write `govoplan-connectors` concept.
+- [ ] Write `govoplan-policy` concept.
+- [ ] Write `govoplan-audit` concept.
+- [ ] Write `govoplan-idm` concept.
+
+## Key Decisions Needed
+
+- [ ] Should `govoplan-core` be renamed conceptually to `govoplan-kernel`, while keeping package compatibility for now?
+- [ ] Is access required for any useful product startup, or should kernel-only expose only health/shell/config pages?
+- [ ] Should tenants be extracted before policy, or after access but before policy?
+- [ ] Should admin be one platform module or split across access/tenancy/policy/audit?
+- [ ] Which event transport is the first production target: DB outbox only, Redis/Celery, or pluggable dispatch?
+- [ ] Which future module comes first after platform cleanup: cases, workflow, or connectors?
+
+## Recommended Next Step
+
+Start with a small architecture-hardening milestone:
+
+1. Update module architecture docs with the kernel/platform module model.
+2. Add dependency-boundary checks.
+3. Add missing backend module permutation startup tests.
+4. Inventory access/auth/RBAC imports.
+5. Draft the `govoplan-access` extraction plan.
+
+Do not start by physically moving access, tenancy, policy, and audit all at once. The safe path is contract freeze first, then one extraction at a time.