diff --git a/Repo-docs-REFERENCE-JOURNEY-PROGRAM.-.md b/Repo-docs-REFERENCE-JOURNEY-PROGRAM.-.md new file mode 100644 index 0000000..ec424fc --- /dev/null +++ b/Repo-docs-REFERENCE-JOURNEY-PROGRAM.-.md @@ -0,0 +1,418 @@ + + +> Mirrored from `/mnt/DATA/git/govoplan/docs/REFERENCE_JOURNEY_PROGRAM.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# GovOPlaN Reference Journey Program + +## Status + +This is the selected product-development sequence as of 2026-07-21. It turns +the long-term connected-platform roadmap into five demonstrable journeys while +the Workflow program remains deliberately deferred. + +The stages are ordered, but they are not monolithic releases. Each stage is +delivered as small, reviewable, green increments and is complete only when its +user journey, failure behavior, documentation, and operator evidence work in a +pinned composition. + +## Why this sequence + +The sequence grows one connected product rather than advancing repositories in +isolation: + +```text +Campaign reference + -> function-bound Postbox delivery + -> data-backed templates and reports + -> governed analytical data products + -> collaborative document lifecycle +``` + +Each stage reuses the preceding contracts: + +- Campaign proves governed recipient selection, files, profiles, preview, + durable delivery, retry, reconciliation, audit, and role-specific UI. +- Postbox generalizes delivery from an email address to an institutional + responsibility that survives personnel changes. +- Templates and Reporting turn governed source data into reproducible outputs + and provide a safe launch boundary for specialist systems such as HIS. +- The BI path scales those source, transformation, quality, lineage, and report + contracts from one document to reusable analytical data products. +- DMS collaboration turns generated or uploaded artifacts into governed living + documents without confusing storage, editing, approval, or records + ownership. + +## Continuous foundation lane + +This lane is not a sixth product stage. It advances only as far as the next +journey needs and supplies contracts shared by all five stages. + +1. **Governed integrations.** Connector profiles declare endpoint, authority, + source-of-truth mode, capabilities, health, limits, credential references, + and lifecycle. Network destinations are resolved and pinned at connection + time; private-network access is deployment-controlled and unpinned private + transports fail closed. Deleting a connector or provider immediately + deletes every secret it owns and records a non-secret audit event in the + same lifecycle action; an unavailable external secret provider blocks the + deletion or retirement rather than leaving an unaudited orphan. +2. **External effects.** Requested action, durable command, observed effect, + unknown outcome, retry, reconciliation, compensation/correction, and manual + intervention remain distinct and auditable. +3. **Institutional identity.** Identity owns subjects, Organizations owns units + and functions, IDM owns identity-to-function assignments and upstream + synchronization, and Access owns permission and acting-context decisions. +4. **Stable composition.** Modules cooperate through versioned Core-mediated + capabilities and typed references. Optional sibling modules are never + imported as implementation dependencies. +5. **Evidence and provenance.** Every imported datum, generated artifact, + delivery, transformation, and later correction can identify its source, + version, actor or system trigger, policy, and relevant execution evidence. +6. **Focused experience.** Central components and the interface pattern + language are mandatory. Views present only what the current task and actor + need while preserving an explainable escape to the configured full system. +7. **Adaptive documentation.** The owning module contributes canonical subject + matter; GovOPlaN Docs composes it for the installed modules, enabled + configuration, role, route, and task. +8. **Release proof.** Version alignment, clean install/upgrade, module + permutations, backup/restore, degraded-provider behavior, security checks, + and target-environment acceptance gate product claims. +9. **Artifact and infrastructure egress.** Tenant connectors use the pinned + connector boundary. Installer artifacts use a separate deployment-owned + trust path: approved origins, bounded pinned download or operator prefetch, + signature/checksum verification, safe extraction, and offline package-manager + execution. Database, broker, cache, and worker channels are constrained by + deployment network policy and authenticated transport rather than treated as + tenant connector profiles. + +## Documentation contract for every reference stage + +Documentation is one connected knowledge model rendered through different +perspectives, not separately maintained manuals that drift. + +Every demonstrated journey provides: + +- **Task guidance:** “I want to …” instructions using visible UI wording. +- **Process view:** actors, inputs, states, decisions, handoffs, exceptions, + completion, correction, and retention. +- **Concept view:** domain language and ownership boundaries. +- **Administrator view:** permissions, policies, profiles, configuration, + module requirements, migrations, and audit effects. +- **Operator view:** health, queues, retries, reconciliation, backup/restore, + incident handling, and target-provider checks. +- **Integration view:** capabilities, DTOs, APIs, events, idempotency, + compatibility, deep-link/launch contracts, and example payloads. +- **Security, privacy, and audit view:** threats, disclosure rules, secrets, + provenance, evidence, retention, and destructive actions. +- **Acceptance view:** runnable examples, expected results, failure injection, + and release gates. + +The Docs module selects and links these views according to installed +capabilities and actor context. Feature repositories remain the source of +truth for their behavior; Docs owns indexing, conditions, safe disclosure, and +presentation. Generated API/contract reference may supplement, but never +replace, the maintained behavioral explanation. + +## Stage 1: Campaign as the demonstration module + +### Product promise + +> As a communications team, we can compose a campaign from governed recipient +> data, managed files, reusable mail configuration, and policy; preview the +> exact effect; authorize delivery; and resolve every recipient to a known or +> explicitly uncertain outcome without exposing transport internals to normal +> readers. + +Campaign is the first reference composition, not an all-purpose mail client or +contact database. + +### Ownership and composition + +- Campaign owns campaign intent, content, campaign-local recipient snapshots, + personalization, execution state, and delivery evidence. +- Mail owns reusable profiles, credentials, protocol adapters, profile policy, + send/append execution, and provider-facing diagnostics. Campaign persists + only a stable Mail profile reference, never SMTP/IMAP settings or secrets. +- Files owns stored objects, provider connectors, file policy, and provenance. + Campaign persists stable attachment references and frozen execution evidence. +- Addresses or another directory may supply recipients, but Campaign freezes + the execution-time recipient and source evidence needed to explain the send. +- Policy, Access, Audit, Docs, Notifications, Organizations, Identity, IDM, and + Postbox contribute only through declared capabilities when installed. + +### Implementation slices + +1. Enforce the Mail-profile-only boundary and reject inline transport material + on authoring, import, update, build, and delivery paths. Legacy records are + preserved but fail closed until explicitly migrated. +2. Finish the central-component UI pass and consistent task-focused author, + reviewer, operator, and reader surfaces. Normal readers receive business + state, not paths, backend keys, worker claims, or raw provider diagnostics. +3. Prove test, single-send, resend, batch, append-to-Sent, retry, + reconciliation, quarantine, and manual intervention semantics against a + target SMTP/IMAP environment. +4. Complete adaptive Campaign, Mail, and Files documentation under the shared + documentation contract, including runnable reference campaigns and failure + drills. +5. Package the composition with sample data, policies, roles, configured views, + preflight, health checks, and pinned compatible versions. + +### Gate + +A clean installation can run the maintained examples through target providers; +no Campaign payload contains transport credentials; worker/provider failures do +not cause blind duplicate delivery; each supported role sees an appropriate +surface and explanation; install, upgrade, restore, and release evidence are +repeatable. + +## Stage 2: Postbox and delivery to institutional responsibility + +### Product promise + +> As a sender, I can deliver a message to a named function in an organizational +> unit without knowing which individual currently performs it. As the current +> function holder or authorized delegate, I can discover, read, and act on that +> message; when assignments change, access follows current responsibility while +> the delivery and access history remains explainable. + +### Ownership and composition + +- Postbox owns addressable in-platform postboxes, messages, bindings, + participants, attachment references, access-sensitive events, and retention + inputs. +- Organizations defines units and functions. Identity defines subjects. IDM + assigns identities to functions and reconciles upstream directories. Access + resolves roles, delegations, acting context, and permissions. +- Mail is an optional external channel bridge. A Postbox is not an SMTP/IMAP + account and is never owned by one login credential. +- Campaign consumes a generic delivery-target capability. It may target an + email recipient or a function-bound Postbox without importing either + module's internals. + +### Implementation slices + +1. Stabilize Postbox manifest, permissions, DTOs, migrations, directory, + binding administration, access decisions, message create/list/read, and + audit events. +2. Define a typed delivery target and receipt contract for a Postbox id or an + organization-unit/function address. Resolution never copies current holders + into permanent mailbox ownership. +3. Complete the Organizations–Identity–IDM–Access function-assignment and + delegation path, including provenance and compatibility migration from + legacy projections. +4. Add Campaign delivery to a function-bound Postbox, file attachment + references, optional external-mail bridge, and notification attention + signals through capabilities. +5. Document reassignment, delegation, vacancy, expired assignment, unavailable + optional module, retention, and future E2EE limitations honestly. + +### Gate + +A campaign message can be delivered once to a function-bound Postbox; an +authorized holder can act in an explicit context; a reassignment changes future +access without moving or rewriting the message; vacancy and ambiguous context +fail visibly; delivery, access, and later corrections remain auditable. + +## Stage 3: Templates, reports, data sources, and deep launches + +### Product promise + +> As a staff member arriving from a specialist system such as HIS, I can open a +> GovOPlaN report or document task with a governed source context already +> selected, review the effective data and template version, generate the +> document, and return or link the immutable result without re-entering data. + +### Ownership and composition + +- Templates owns reusable definitions, versions, merge schemas, localization, + render profiles, preview, and rendering. +- Reporting owns report definitions, parameters, curated data selection, + execution history, dashboards, sharing, scheduling, publication, and export. +- Connectors own protocol-specific access. A governed data-source catalog owns + connection profiles, source contracts, freshness, and health if repeated + use proves that a separate module is warranted. +- Files stores generated bytes. DMS owns a generated artifact once it becomes a + governed living document. Records owns later retention/archive semantics. + +### Safe launch and generation contract + +1. The source program opens a stable GovOPlaN route with an opaque, short-lived, + single-purpose launch reference—not credentials, arbitrary SQL, or trusted + personal data in the URL. +2. GovOPlaN authenticates the actor, authorizes the requested report/template + and source object, resolves the launch context server-side, and shows source, + freshness, purpose, and any blockers. +3. A curated connector/read-model capability returns a versioned input snapshot + that satisfies the template or report schema. +4. Generation records template/report definition version, parameters, input + snapshot or reproducible source references, transformation version, output + checksum, actor, and policy. +5. The result is stored or handed back through an explicit callback/reference + contract. Retries are idempotent; expired or replayed launch references fail + safely. + +### Implementation slices + +1. Implement the first Templates manifest, version/schema DTOs, safe preview, + deterministic render, package fragments, and Files-backed output. +2. Implement the first Reporting manifest, parameter/source DTOs, one + module-owned read model, execution evidence, and downloadable export. +3. Define the shared data-source/profile and launch-context contracts with one + read-only target connector and a mock HIS-style launch producer. +4. Deliver one end-to-end reference document and one analytical report from the + same governed source contract. +5. Add role-specific UI/docs, source freshness and validation states, deep-link + integration examples, and target-system acceptance tests. + +### Gate + +A user can follow a signed or server-side launch reference from a target system +to a prefilled GovOPlaN task, verify the source and freshness, render an output, +and reproduce why that exact artifact was produced. The browser cannot inject +arbitrary queries or transport secrets, and authorization is re-evaluated in +GovOPlaN. + +## Stage 4: Governed BI and reporting + +### Product promise + +> As an institutional analyst, I can combine approved operational sources into +> versioned, quality-checked analytical data products, define transparent +> measures and reports, drill from aggregates where policy permits, and explain +> the source, transformation, effective organizational hierarchy, and reporting +> date behind every result. + +### Lessons taken from SuperX + +SuperX is useful inspiration because it combines many university sources in a +modular data warehouse, loads and transforms them on a schedule, keeps +historical or reporting-date views, supplies prepared subject modules, and +serves different aggregation levels through a common reporting surface. Its +official overview also emphasizes a lower operational-data layer, validation +logs, configurable load routines, organizational hierarchies, and transparent +report calculations. See the +[SuperX project overview](https://www.superx-projekt.de/__index.htm), +[administration and module architecture](https://www.superx-projekt.de/doku/kern_modul/admin/__index.htm), +and [cross-institution indicator module](https://www.superx-projekt.de/doku/kenn_modul/benutzer/__index.htm). + +GovOPlaN should adopt the principles, not clone the implementation. Its distinct +value is connecting analytical results to governed work, responsibility, +evidence, policy, and correction across independently installable modules. + +### Architecture layers + +1. **Source catalog:** governed profiles, owners, purpose, schema/version, + classification, credentials, health, extraction mode, and source-of-truth + declaration. +2. **Ingestion and staging:** immutable run evidence, source snapshots or + watermarks, schema-drift detection, quarantine, replay, and personal-data + minimization. +3. **Transformation and quality:** versioned transformations, dependencies, + tests, validation findings, correction policy, and lineage. Silent repair of + source facts is not acceptable. +4. **Semantic products:** documented dimensions, measures, official key + mappings, organizational hierarchies, time/stichtag semantics, ownership, + access policy, and version. +5. **Consumption:** Reporting definitions, dashboards, drill-down, scheduled + outputs, APIs, open-data handoffs, and template generation. +6. **Operations:** scheduling, backfill, freshness objectives, observability, + cost/resource limits, retention, backup/restore, promotion between systems, + and reproducible release recipes. + +### Implementation slices + +1. Select one bounded, read-only university dataset and one decision-relevant + report; define its legal/policy basis, source owner, official keys, time + semantics, and acceptance calculation. +2. Generalize the Stage 3 source profile into catalog, ingestion-run, staged + dataset, validation, transformation, lineage, and data-product contracts. +3. Implement one scheduled incremental load with quarantine/replay and a fully + transparent transformation into an aggregate product. +4. Serve the product through Reporting with policy-aware aggregate and + drill-down behavior; export reproducible evidence and calculation metadata. +5. Package mappings, transformations, validations, reports, documentation, and + tests so they can be promoted from development to test to production. + +### Gate + +One university data product can be rebuilt from a declared source snapshot or +watermark, passes explicit quality tests, explains every measure and mapping, +enforces disclosure policy, survives schema/freshness failures visibly, and +produces the same accepted report across promoted environments. + +## Stage 5: Collaborative document editing + +### Product promise + +> As an authorized group, we can turn an uploaded or generated artifact into a +> shared document, edit it concurrently or through controlled check-out, +> comment, review, approve, compare, and recover versions, then freeze the +> accepted rendition as evidence or a record without losing who changed what. + +### Ownership and composition + +- Files owns bytes, checksums, storage providers, upload/download, and low-level + file permissions. +- DMS owns document identity, versions, renditions, editing sessions, locks, + comments, reviews, approvals, signatures, external DMS references, and + document-level evidence. +- A collaboration connector owns provider-specific Collabora, OnlyOffice, + Nextcloud/OpenDesk, or another office-suite protocol. DMS owns the session + lifecycle and authorization even when editing occurs externally. +- Templates may create the first version. Cases, Campaign, Postbox, Reporting, + and Workflow may reference documents. Records owns classification, legal + hold, archive handoff, and disposal after the document enters that lifecycle. + +### Implementation slices + +1. Implement DMS document/version/reference DTOs and one Files-backed version + path, including optimistic concurrency, checksums, locks, and audit. +2. Add comments, review requests, approval state, immutable accepted + renditions, comparison, and recovery of prior versions. +3. Define a provider-neutral editing-session capability with short-lived + grants, callback authentication, save idempotency, health, and reconciliation. +4. Integrate one external collaborative editor, including loss of provider, + concurrent save, stale callback, permission revocation, and restore tests. +5. Connect generated reports/templates, Postbox/Campaign sharing, focused + document views, configured docs, and Records handoff. + +### Gate + +Multiple authorized actors can edit a document without silent lost updates; +every accepted version has stable content and provenance; provider callbacks +cannot bypass current authorization; degraded or ambiguous saves are visible +and reconcilable; the approved rendition can enter records without making Files +or the external editor the document-lifecycle owner. + +## Program-wide acceptance and pause rules + +- Do not start a broad generic platform abstraction until two concrete stages + demonstrate the repeated contract. +- Do not make Workflow a dependency of these stages. Manual transitions, + explicit actions, and focused views should become stable providers that a + later transition engine can coordinate. +- Do not treat a remote system as trusted merely because it supplied a deep + link, callback, webhook, file, identity, or data row. +- Do not claim a stage complete from local unit tests. Use pinned composition, + target integration, failure drills, adaptive docs, and operator evidence. +- A later stage may prototype contracts while the preceding gate is being + proven, but it may not redefine an owning module's boundary by convenience. + +## Decisions intentionally deferred + +The sequence itself is selected. These bounded choices remain for the stage +that first needs them: + +- first target SMTP/IMAP, file/directory, identity, and deployment profile; +- first external IDM source and identity conflict/disable policy; +- first trusted Postbox assurance/E2EE profile and external-mail bridge; +- first HIS or other launch producer and callback/reference contract; +- first university dataset, report, reporting date semantics, and permitted + drill-down level; +- whether repeated data-source and dataflow contracts justify separate + `govoplan-datasources` and `govoplan-dataflow` modules after the Stage 3 proof; +- first collaborative editor/provider and whether the first UX is concurrent + editing, controlled check-out, or both; and +- first Records/archive target and approval/signature assurance level.