GovOPlaN/govoplan
2
0
Fork 0
Code Issues 18 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

[Roadmap] Deliver governed dataflows, search, and established-tool integration #19

New Issue
Open
opened 2026-07-29 15:43:23 +02:00 by zemion · 2 comments
zemion commented 2026-07-29 15:43:23 +02:00
Owner
Copy Link
Copy Source

Outcome

Track the agreed implementation sequence that turns the monthly-processing and sanctions-screening user stories into executable capabilities while preserving optional module boundaries.

Ordered work

  1. Document the integration maturity model and canonical external-reference contract.
  2. Turn the monthly workflow and sanctions screening into executable golden-flow fixtures with sample inputs and expected outputs.
  3. Resolve govoplan-dataflow#14 by making validators, executors, and compilers registry-driven.
  4. Add typed expressions, composite/parameterized templates, schema propagation, and quality/reconciliation operators.
  5. Deliver govoplan-search with PostgreSQL first and an optional OpenSearch adapter second.
  6. Build OpenProject as the first full connector, followed by a Znuny/OTRS-compatible service-desk adapter and MediaWiki/BlueSpice.

Third-party analytical execution engines and the Arrow execution adapter in govoplan-dataflow#15 are deliberately outside this sequence for now.

Tracking

  • Monthly flow: govoplan#8
  • Sanctions screening: govoplan#12
  • Dataflow registry debt: govoplan-dataflow#14
  • Dataflow reuse/governance: govoplan-dataflow#12
  • Search epic: govoplan-search#1
  • Integration epic: govoplan-connectors#11

Each child issue remains open until its own acceptance criteria are met; this issue records ordering and cross-repository status.

Architecture reconciliation (2026-07-31)

External references, registry-driven Dataflow internals, typed execution boundaries, PostgreSQL Search, golden flows, and the Workflow Engine/editor split are now implemented foundations. Remaining work should be read from the child issues and from GovOPlaN/govoplan#29, especially provider source-authority declarations, the governed data/register catalogue, target performance/recovery evidence, optional OpenSearch, and production connector acceptance. Do not reopen completed steps as greenfield work.

## Outcome Track the agreed implementation sequence that turns the monthly-processing and sanctions-screening user stories into executable capabilities while preserving optional module boundaries. ## Ordered work 1. Document the integration maturity model and canonical external-reference contract. 2. Turn the monthly workflow and sanctions screening into executable golden-flow fixtures with sample inputs and expected outputs. 3. Resolve `govoplan-dataflow#14` by making validators, executors, and compilers registry-driven. 4. Add typed expressions, composite/parameterized templates, schema propagation, and quality/reconciliation operators. 5. Deliver `govoplan-search` with PostgreSQL first and an optional OpenSearch adapter second. 6. Build OpenProject as the first full connector, followed by a Znuny/OTRS-compatible service-desk adapter and MediaWiki/BlueSpice. Third-party analytical execution engines and the Arrow execution adapter in `govoplan-dataflow#15` are deliberately outside this sequence for now. ## Tracking - Monthly flow: `govoplan#8` - Sanctions screening: `govoplan#12` - Dataflow registry debt: `govoplan-dataflow#14` - Dataflow reuse/governance: `govoplan-dataflow#12` - Search epic: `govoplan-search#1` - Integration epic: `govoplan-connectors#11` Each child issue remains open until its own acceptance criteria are met; this issue records ordering and cross-repository status. ## Architecture reconciliation (2026-07-31) External references, registry-driven Dataflow internals, typed execution boundaries, PostgreSQL Search, golden flows, and the Workflow Engine/editor split are now implemented foundations. Remaining work should be read from the child issues and from [GovOPlaN/govoplan#29](https://git.add-ideas.de/GovOPlaN/govoplan/issues/29), especially provider source-authority declarations, the governed data/register catalogue, target performance/recovery evidence, optional OpenSearch, and production connector acceptance. Do not reopen completed steps as greenfield work. <!-- codex-institutional-governance-data-integration-2026-07-31 -->
zemion commented 2026-07-29 15:53:33 +02:00
Author
Owner
Copy Link
Copy Source

Codex State: progress

Summary

  • Steps 1 and 2 are pushed: canonical integration contracts plus executable monthly and sanctions golden flows.
  • Steps 3 and 4 have first substantial slices: registry dispatch, typed safe expressions, conversion/replacement, quality/reconciliation, schema propagation, and parameterized pinned subflows.
  • Step 5 has a working PostgreSQL/SQLite Search baseline and shell UI; OpenSearch remains #3. Step 6 now has concrete connector work packages.

Next / Blocked

  • Continue Dataflow complexity decomposition (#14), complete Search ingestion lifecycle (#2), then start OpenProject connector #12.

Suggested status label: status/in-progress

## Codex State: progress ### Summary - Steps 1 and 2 are pushed: canonical integration contracts plus executable monthly and sanctions golden flows. - Steps 3 and 4 have first substantial slices: registry dispatch, typed safe expressions, conversion/replacement, quality/reconciliation, schema propagation, and parameterized pinned subflows. - Step 5 has a working PostgreSQL/SQLite Search baseline and shell UI; OpenSearch remains #3. Step 6 now has concrete connector work packages. ### Next / Blocked - Continue Dataflow complexity decomposition (#14), complete Search ingestion lifecycle (#2), then start OpenProject connector #12. Suggested status label: `status/in-progress`
zemion commented 2026-09-08 05:27:54 +02:00
Author
Owner
Copy Link
Copy Source

Status reconciliation — 2026-09-08

The next-step list in comment 15990 is superseded. Dataflow #14 is closed with registry dispatch/decomposition (69509d5); Dataflow #12 is closed with governed reusable/subflow definitions, current authorization, provenance and reviewed rebase (c1111c6, v0.1.20). Search #2 and Search #1 are closed: the PostgreSQL/SQLite baseline (a156e3d, Core 920e3c9) and durable ingestion/operator/query path (1bb3381, Core 7117673) are implemented. The 0.1.45 focused gate also passed the monthly-data and sanctions-review/rescreening fixtures.

Remaining ordered work belongs to Search #3 (optional OpenSearch), Connectors #12 (OpenProject), Connectors #11 (provider integrations), and named target/browser/operational acceptance under #8 and #12. Keep this roadmap open; do not restart completed foundations or treat fixture checks as production-provider approval.

<!-- govoplan-backlog-reconciliation-20260908-v1 --> ## Status reconciliation — 2026-09-08 The next-step list in comment 15990 is superseded. [Dataflow #14](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/14) is closed with registry dispatch/decomposition (`69509d5`); [Dataflow #12](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/12) is closed with governed reusable/subflow definitions, current authorization, provenance and reviewed rebase (`c1111c6`, v0.1.20). [Search #2](https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/2) and [Search #1](https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/1) are closed: the PostgreSQL/SQLite baseline (`a156e3d`, Core `920e3c9`) and durable ingestion/operator/query path (`1bb3381`, Core `7117673`) are implemented. The 0.1.45 focused gate also passed the monthly-data and sanctions-review/rescreening fixtures. Remaining ordered work belongs to [Search #3](https://git.add-ideas.de/GovOPlaN/govoplan-search/issues/3) (optional OpenSearch), [Connectors #12](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/12) (OpenProject), [Connectors #11](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/11) (provider integrations), and named target/browser/operational acceptance under #8 and #12. Keep this roadmap open; do not restart completed foundations or treat fixture checks as production-provider approval.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/api
HTTP API contracts, routers, schemas, or API smoke behavior.
 area/auth
Authentication, sessions, access bootstrap, or login behavior.
 area/db
Database sessions, models, transactions, or persistence primitives.
 area/devex
Local developer workflow, scripts, tests, tooling, or release helpers.
 area/docs
Durable documentation and project guidance.
 area/governance
Governance policy, audit, privacy, retention, or compliance behavior.
 area/marketing
Public website, product messaging, publication copy, or legal page content.
 area/migrations
Alembic migrations, schema bootstrap, or persistence evolution.
 area/module-system
Module discovery, manifests, capabilities, routing, or optional integrations.
 area/rbac
Permissions, roles, delegation, or authorization policy.
 area/release
Versioning, release locks, tags, packaging, or dependency pins.
 area/security
Security posture, static analysis, supply-chain hardening, or vulnerability remediation.
 area/tenancy
Tenant boundaries, provisioning, or tenant-scoped data behavior.
 area/webui
Shared WebUI shell, frontend components, routing, or frontend tests.
 audit/complexity
Complexity finding from Radon, Xenon, or equivalent maintainability scans.
 audit/duplication
Duplicated-code finding from jscpd or equivalent similarity scans.
 audit/false-positive
Audit finding reviewed as a narrow false positive or acceptable risk.
 audit/needs-design
Audit finding that needs an architectural or product decision before implementation.
 audit/quick-fix
Audit finding that appears narrow and directly fixable.
 audit/structural
Audit finding that needs design, refactoring, or behavior review.
 codex/needs-human
Needs an explicit human decision before Codex should implement.
 codex/ready
Suitable for Codex to pick up with the existing issue context.
 module/access
GovOPlaN access, identity, authentication, RBAC, and administration behavior.
 module/addresses
GovOPlaN Addresses module behavior or integration.
 module/admin
GovOPlaN Admin module behavior or integration.
 module/appointments
GovOPlaN Appointments module behavior or integration.
 module/approvals
GovOPlaN Approvals module behavior or integration.
 module/audit
GovOPlaN Audit module behavior or integration.
 module/calendar
GovOPlaN Calendar module behavior or integration.
 module/campaign
GovOPlaN campaign module behavior or integration.
 module/cases
GovOPlaN Cases module behavior or integration.
 module/committee
GovOPlaN Committee module behavior or integration.
 module/connectors
GovOPlaN Connectors module behavior or integration.
 module/core
GovOPlaN core runner, shared primitives, shell, or extension points.
 module/dashboard
GovOPlaN Dashboard module behavior or integration.
 module/dataflow
GovOPlaN Dataflow module behavior or integration.
 module/datasources
GovOPlaN governed datasource contracts, catalogs, and integrations.
 module/decisions
GovOPlaN formal Decisions module behavior or integration.
 module/dist-lists
GovOPlaN Distribution Lists module behavior or integration.
 module/dms
GovOPlaN Dms module behavior or integration.
 module/docs
GovOPlaN Docs module behavior or integration.
 module/encryption
GovOPlaN Encryption key custody, cryptographic policy, and E2EE integration.
 module/erp
GovOPlaN Erp module behavior or integration.
 module/evaluation
GovOPlaN Evaluation module behavior or integration.
 module/files
GovOPlaN files module behavior or integration.
 module/fit-connect
GovOPlaN Fit Connect module behavior or integration.
 module/forms
GovOPlaN Forms module behavior or integration.
 module/forms-runtime
GovOPlaN Forms Runtime module behavior or integration.
 module/helpdesk
GovOPlaN Helpdesk module behavior or integration.
 module/identity
GovOPlaN Identity module behavior or integration.
 module/identity-trust
GovOPlaN Identity Trust module behavior or integration.
 module/idm
GovOPlaN Idm module behavior or integration.
 module/ledger
GovOPlaN Ledger module behavior or integration.
 module/mail
GovOPlaN mail module behavior or integration.
 module/mandates
GovOPlaN Mandates, jurisdiction, responsibility, and authority behavior or integration.
 module/notifications
GovOPlaN Notifications module behavior or integration.
 module/ops
GovOPlaN Ops module behavior or integration.
 module/organizations
GovOPlaN Organizations module behavior or integration.
 module/parties
GovOPlaN procedure Parties, representation, and delivery-authority behavior or integration.
 module/payments
GovOPlaN Payments module behavior or integration.
 module/permits
GovOPlaN Permits module behavior or integration.
 module/policy
GovOPlaN Policy module behavior or integration.
 module/poll
GovOPlaN Poll module behavior or integration.
 module/portal
GovOPlaN Portal module behavior or integration.
 module/postbox
GovOPlaN Postbox module behavior or integration.
 module/projects
GovOPlaN Projects module behavior or integration.
 module/quick-access
GovOPlaN configurable task-local Quick Access behavior and integrations.
 module/records
GovOPlaN Records and eAkte lifecycle behavior or integration.
 module/reporting
GovOPlaN Reporting module behavior or integration.
 module/risk-compliance
GovOPlaN Risk Compliance module behavior or integration.
 module/scheduling
GovOPlaN Scheduling module behavior or integration.
 module/search
GovOPlaN Search module behavior or integration.
 module/services
GovOPlaN versioned institutional Services behavior or integration.
 module/tasks
GovOPlaN Tasks module behavior or integration.
 module/templates
GovOPlaN Templates module behavior or integration.
 module/tenancy
GovOPlaN Tenancy module behavior or integration.
 module/tickets
GovOPlaN Tickets module behavior or integration.
 module/views
GovOPlaN governed task views, interface projections, and workflow view integration.
 module/voting
GovOPlaN Voting module behavior or integration.
 module/wiki
GovOPlaN Wiki module behavior or integration.
 module/workflow
GovOPlaN Workflow module behavior or integration.
 module/workflow-engine
GovOPlaN Workflow Engine runtime, persistence, or integration.
 module/xoev
GovOPlaN Xoev module behavior or integration.
 module/xrechnung
GovOPlaN Xrechnung module behavior or integration.
 module/xta-osci
GovOPlaN Xta Osci module behavior or integration.
priority
p0
Immediate stop-the-line priority.
priority
p1
High priority for the next focused work window.
priority
p2
Normal planned priority.
priority
p3
Low priority or opportunistic cleanup.
source/backlog-import
Imported from markdown backlog, roadmap, plan, or TODO files.
 source/security-audit
Created from a structured security or code-quality audit report.
 source/todo-scan
Imported from inline TODO/FIXME/HACK markers by the Gitea TODO importer.
status
blocked
Cannot progress without a decision, dependency, credential, or external change.
status
in-progress
Currently being worked.
status
needs-info
Needs clarifying input before implementation can proceed safely.
status
ready
Ready for implementation.
status
triage
Needs review, ownership, priority, or acceptance criteria.
type
bug
A reproducible defect, regression, or incorrect behavior.
type
debt
Cleanup, refactoring, risk reduction, or deferred engineering work.
type
docs
Documentation, process, or developer workflow work.
type
feature
New user-visible behavior or platform capability.
type
task
Implementation, maintenance, migration, or operational work.
type
user-story
End-to-end user journey or real-world process story used to steer product slices.
No labels area/governance area/module-system codex/ready module/connectors module/dataflow module/projects module/search module/tickets module/wiki
priority
p1
status
in-progress
type
task
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
govoplan-package-publisher zemion (Albrecht Degering)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: GovOPlaN/govoplan#19