[Epic] Deliver a governed Dataflow manager with graphical and SQL editing #1
Closed
opened 2026-07-28 00:58:08 +02:00 by zemion
·
10 comments
No Branch/Tag Specified
Labels
Clear labels
area/api
area/auth
area/db
area/devex
area/docs
area/governance
area/marketing
area/migrations
area/module-system
area/rbac
area/release
area/security
area/tenancy
area/webui
audit/complexity
audit/duplication
audit/false-positive
audit/needs-design
audit/quick-fix
audit/structural
codex/needs-human
codex/ready
module/access
module/addresses
module/admin
module/appointments
module/approvals
module/audit
module/calendar
module/campaign
module/cases
module/committee
module/connectors
module/core
module/dashboard
module/dataflow
module/datasources
module/decisions
module/dist-lists
module/dms
module/docs
module/encryption
module/erp
module/evaluation
module/files
module/fit-connect
module/forms
module/forms-runtime
module/helpdesk
module/identity
module/identity-trust
module/idm
module/ledger
module/mail
module/mandates
module/notifications
module/ops
module/organizations
module/parties
module/payments
module/permits
module/policy
module/poll
module/portal
module/postbox
module/projects
module/quick-access
module/records
module/reporting
module/risk-compliance
module/scheduling
module/search
module/services
module/tasks
module/templates
module/tenancy
module/tickets
module/views
module/voting
module/wiki
module/workflow
module/workflow-engine
module/xoev
module/xrechnung
module/xta-osci
source/backlog-import
source/security-audit
source/todo-scan
HTTP API contracts, routers, schemas, or API smoke behavior.
Authentication, sessions, access bootstrap, or login behavior.
Database sessions, models, transactions, or persistence primitives.
Local developer workflow, scripts, tests, tooling, or release helpers.
Durable documentation and project guidance.
Governance policy, audit, privacy, retention, or compliance behavior.
Public website, product messaging, publication copy, or legal page content.
Alembic migrations, schema bootstrap, or persistence evolution.
Module discovery, manifests, capabilities, routing, or optional integrations.
Permissions, roles, delegation, or authorization policy.
Versioning, release locks, tags, packaging, or dependency pins.
Security posture, static analysis, supply-chain hardening, or vulnerability remediation.
Tenant boundaries, provisioning, or tenant-scoped data behavior.
Shared WebUI shell, frontend components, routing, or frontend tests.
Complexity finding from Radon, Xenon, or equivalent maintainability scans.
Duplicated-code finding from jscpd or equivalent similarity scans.
Audit finding reviewed as a narrow false positive or acceptable risk.
Audit finding that needs an architectural or product decision before implementation.
Audit finding that appears narrow and directly fixable.
Audit finding that needs design, refactoring, or behavior review.
Needs an explicit human decision before Codex should implement.
Suitable for Codex to pick up with the existing issue context.
GovOPlaN access, identity, authentication, RBAC, and administration behavior.
GovOPlaN Addresses module behavior or integration.
GovOPlaN Admin module behavior or integration.
GovOPlaN Appointments module behavior or integration.
GovOPlaN Approvals module behavior or integration.
GovOPlaN Audit module behavior or integration.
GovOPlaN Calendar module behavior or integration.
GovOPlaN campaign module behavior or integration.
GovOPlaN Cases module behavior or integration.
GovOPlaN Committee module behavior or integration.
GovOPlaN Connectors module behavior or integration.
GovOPlaN core runner, shared primitives, shell, or extension points.
GovOPlaN Dashboard module behavior or integration.
GovOPlaN Dataflow module behavior or integration.
GovOPlaN governed datasource contracts, catalogs, and integrations.
GovOPlaN formal Decisions module behavior or integration.
GovOPlaN Distribution Lists module behavior or integration.
GovOPlaN Dms module behavior or integration.
GovOPlaN Docs module behavior or integration.
GovOPlaN Encryption key custody, cryptographic policy, and E2EE integration.
GovOPlaN Erp module behavior or integration.
GovOPlaN Evaluation module behavior or integration.
GovOPlaN files module behavior or integration.
GovOPlaN Fit Connect module behavior or integration.
GovOPlaN Forms module behavior or integration.
GovOPlaN Forms Runtime module behavior or integration.
GovOPlaN Helpdesk module behavior or integration.
GovOPlaN Identity module behavior or integration.
GovOPlaN Identity Trust module behavior or integration.
GovOPlaN Idm module behavior or integration.
GovOPlaN Ledger module behavior or integration.
GovOPlaN mail module behavior or integration.
GovOPlaN Mandates, jurisdiction, responsibility, and authority behavior or integration.
GovOPlaN Notifications module behavior or integration.
GovOPlaN Ops module behavior or integration.
GovOPlaN Organizations module behavior or integration.
GovOPlaN procedure Parties, representation, and delivery-authority behavior or integration.
GovOPlaN Payments module behavior or integration.
GovOPlaN Permits module behavior or integration.
GovOPlaN Policy module behavior or integration.
GovOPlaN Poll module behavior or integration.
GovOPlaN Portal module behavior or integration.
GovOPlaN Postbox module behavior or integration.
GovOPlaN Projects module behavior or integration.
GovOPlaN configurable task-local Quick Access behavior and integrations.
GovOPlaN Records and eAkte lifecycle behavior or integration.
GovOPlaN Reporting module behavior or integration.
GovOPlaN Risk Compliance module behavior or integration.
GovOPlaN Scheduling module behavior or integration.
GovOPlaN Search module behavior or integration.
GovOPlaN versioned institutional Services behavior or integration.
GovOPlaN Tasks module behavior or integration.
GovOPlaN Templates module behavior or integration.
GovOPlaN Tenancy module behavior or integration.
GovOPlaN Tickets module behavior or integration.
GovOPlaN governed task views, interface projections, and workflow view integration.
GovOPlaN Voting module behavior or integration.
GovOPlaN Wiki module behavior or integration.
GovOPlaN Workflow module behavior or integration.
GovOPlaN Workflow Engine runtime, persistence, or integration.
GovOPlaN Xoev module behavior or integration.
GovOPlaN Xrechnung module behavior or integration.
GovOPlaN Xta Osci module behavior or integration.
priority
p0
Immediate stop-the-line priority.
priority
p1
High priority for the next focused work window.
priority
p2
Normal planned priority.
priority
p3
Low priority or opportunistic cleanup.
Imported from markdown backlog, roadmap, plan, or TODO files.
Created from a structured security or code-quality audit report.
Imported from inline TODO/FIXME/HACK markers by the Gitea TODO importer.
status
blocked
Cannot progress without a decision, dependency, credential, or external change.
status
in-progress
Currently being worked.
status
needs-info
Needs clarifying input before implementation can proceed safely.
status
ready
Ready for implementation.
status
triage
Needs review, ownership, priority, or acceptance criteria.
type
bug
A reproducible defect, regression, or incorrect behavior.
type
debt
Cleanup, refactoring, risk reduction, or deferred engineering work.
type
docs
Documentation, process, or developer workflow work.
type
feature
New user-visible behavior or platform capability.
type
task
Implementation, maintenance, migration, or operational work.
type
user-story
End-to-end user journey or real-world process story used to steer product slices.
Milestone
No items
No Milestone
Projects
Clear projects
No projects
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: GovOPlaN/govoplan-dataflow#1
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Outcome
Deliver a production-shaped Dataflow module that lets power users define a transformation once, inspect it either as a directed graph or constrained SQL, preview its result, and preserve the exact definition and diagnostics used for every run.
This is the enabling capability for the monthly-data user story, governed BI paths, and sanctions-source normalization. It does not own external connections, legal screening decisions, report presentation, or workflow orchestration.
Boundary
First Release Gate
Product References
Codex State: progress
Summary
df468a2.Changed Files
pyproject.tomlsrc/govoplan_dataflow/backendwebui/srctestsVerification
15 Dataflow backend tests passRuff passesstandalone TypeScript typecheck and structure test passcore Dataflow-only and full-product production builds passconsolidated focused check passes across 44 manifests and all module permutationsNext / Blocked
Suggested status label:
status/in-progressCodex State: progress
Summary
df468a2, core registration at c7183fe, and meta repository/development composition at 97dfd33.Verification
All three published repositories are clean relative to origin; unrelated paused credential work remains uncommitted in core.Suggested status label:
status/in-progressCodex State: progress
Summary
govoplan-core@d36bb94), executable snapshot provider (govoplan-connectors@ba5ccea), and expanded editor/runtime (govoplan-dataflow@dee8380).govoplan@603e07c).Verification
python -m unittest discover -s govoplan-connectors/tests(7 passed)python -m unittest discover -s govoplan-dataflow/tests(24 passed)tools/checks/check-contracts.sh(45 contracts passed)tools/checks/check-focused.sh(core, module, API, and WebUI checks passed)Next / Blocked
Codex State: progress
Dataflow now uses Core's shared definition-graph primitives and consumes the
governed Datasources catalogue/lifecycle contract rather than Connectors
directly (
6ca3058). The source library supports current, direct-live, andlatest-frozen reads with expected-fingerprint drift checks.
Verification includes 24 backend tests, WebUI TypeScript/build checks, static
manifest/contract gates, and a durable Connector -> Datasources -> Dataflow
composition check. Browser-level editor E2E coverage and production run/output
publication remain open.
Codex State: progress
Summary
6305ef9.Verification
27 Dataflow tests passed; cross-module composition and all WebUI module permutations passed.Next / Blocked
Suggested status label:
status/in-progressCodex State: progress
Summary
Changed Files
src/govoplan_dataflow/backend/{schemas,executor,service,router}.pywebui/src/features/dataflow/{DataflowPage,DataflowCanvas,NodeInspector,graphInteractions}.tsxVerification
36 Dataflow backend tests passedDataflow WebUI typecheck and structure checks passedGovOPlaN focused suite, all module permutations, and full-product build passedNext / Blocked
521829ais pushed to main; perform normal browser UX validation of the proximity threshold and reconnect handles.Suggested status label:
status/in-progressCodex State: progress
Summary
Changed Files
src/govoplan_dataflow/backend/sql_compiler.pywebui/src/features/dataflow/DataflowPage.tsxwebui/src/features/dataflow/DataflowCanvas.tsxVerification
37 Dataflow backend tests passed; Ruff passed.Dataflow WebUI typecheck and structure tests passed.Full tools/checks/check-focused.sh integration and WebUI module permutation battery passed.Next / Blocked
Suggested status label:
status/in-progressCodex State: progress
Summary
Changed Files
src/govoplan_dataflow/backend/graph.pysrc/govoplan_dataflow/backend/sql_compiler.pywebui/src/features/dataflow/DataflowPage.tsxwebui/src/styles/dataflow.cssVerification
39 Dataflow backend tests and Ruff passed.Dataflow WebUI typecheck and structure tests passed.Core full-product build and full tools/checks/check-focused.sh battery passed.Suggested status label:
status/in-progressConceptual refinement recorded
Two non-duplicate implementation slices now make the engine direction actionable:
The existing node/editor/compiler/template issues remain authoritative for their current scopes.
Verified complete. All first-release acceptance criteria were already checked. Current validation: 70 backend tests passed; WebUI typecheck and Dataflow structure checks passed. Enhancement work remains independently tracked in Dataflow #7, #8, #9, #11, #12, and #16.