GovOPlaN/govoplan
1
0
Fork 0
Code Issues 11 Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Sync wiki from project files

Albrecht Degering
2026-07-28 13:52:16 +02:00
parent caa8974b11
commit 1ff2b5249f
1 changed files with 17 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
Repo-docs-DATASOURCE-AND-DEFINITION-GRAPH-ARCHITECTURE.md

@@ -1,4 +1,4 @@
<!-- codex-wiki-sync:1111d9caae48b86897d97060 --> <!-- codex-wiki-sync:77773a7ff5a08dd2cea6eccb -->
> Mirrored from `/mnt/DATA/git/govoplan/docs/DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md`. > Mirrored from `/mnt/DATA/git/govoplan/docs/DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md`.
> Origin: `repository`. > Origin: `repository`.
@@ -30,6 +30,8 @@ The initial provider path is:
4. Any datasource may expose a frozen state for reproducible execution evidence. 4. Any datasource may expose a frozen state for reproducible execution evidence.
5. Dataflow stores an opaque datasource reference, state policy, and expected 5. Dataflow stores an opaque datasource reference, state policy, and expected
fingerprint. fingerprint.
6. A pinned Dataflow run may publish a complete bounded result as a new
immutable materialization through an idempotent Datasources capability.
Database, REST/HTTP, LDAP/directory, managed file, watched-directory, feed, and Database, REST/HTTP, LDAP/directory, managed file, watched-directory, feed, and
stream providers fit behind the same origin contract. Provider-specific stream providers fit behind the same origin contract. Provider-specific
@@ -58,21 +60,30 @@ kind of Dataflow or leaking either module into Core.
## Current Implementation ## Current Implementation
- Core graph and datasource contracts are versioned at `0.1.0`. - Core graph and datasource contracts are versioned at `0.1.0`.
- Workflow exposes node-library discovery and definition validation APIs. - Workflow exposes a reusable graph editor, node-library discovery, validation,
tenant-isolated definitions, immutable revisions, and activation pinning.
- Datasources exposes catalogue, origins, staging, promotion, preview, - Datasources exposes catalogue, origins, staging, promotion, preview,
materialization history, refresh, freeze, and retirement APIs. materialization history, refresh, freeze, retirement, and producer
publication APIs.
- Datasources WebUI exposes all current lifecycle views. - Datasources WebUI exposes all current lifecycle views.
- Connectors adapts existing tabular snapshots to datasource origins. - Connectors adapts existing tabular snapshots to datasource origins.
- Dataflow consumes only Datasources catalogue/lifecycle capabilities and can - Dataflow consumes only Datasources catalogue/lifecycle capabilities and can
request current, live, or latest-frozen state. request current, live, or latest-frozen state.
- Dataflow exposes a pinned run-lifecycle capability and a Run/Publish surface.
Its first synchronous runner records lineage and terminal state, publishes
only complete bounded results, and retains output datasource/materialization
references.
- The focused composition check proves Connector origin -> Datasource ->
pinned Dataflow run -> frozen published materialization, including replay.
## Next Slices ## Next Slices
1. Persist and version Workflow definitions, then reuse the graph UI shell. 1. Add persisted Workflow instances, resumable transitions, human activities,
retry policy, and event subscriptions against pinned definition revisions.
2. Add SQL database and governed REST origin providers with credential-envelope 2. Add SQL database and governed REST origin providers with credential-envelope
references and bounded pushdown. references and bounded pushdown.
3. Add managed-file and directory origins. 3. Add managed-file and directory origins.
4. Add datasource quality rules, schema compatibility policy, retention, and 4. Add datasource quality rules, schema compatibility policy, retention, and
promotion approvals. promotion approvals.
5. Publish Dataflow outputs as new materializations and expose run lifecycle to 5. Add asynchronous Dataflow workers and durable artifact-backed outputs for
Workflow. runs that exceed the synchronous row/time/byte limits.