GovOPlaN/govoplan
1
0
Fork 0
Code Issues 21 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement institutional governance and recovery architecture
Dependency Audit / dependency-audit (push) Failing after 10s
Details
Deployment Installer / deployment-installer (push) Successful in 6s
Details
Security Audit / security-audit (push) Failing after 9s
Details

Browse Source
This commit is contained in:
zemion
2026-08-01 17:46:53 +02:00
parent 3c658fa32d
commit d78b13f9d3
45 changed files with 4388 additions and 262 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DATASOURCE_AND_DEFINITION_GRAPH_ARCHITECTURE.md
+26 -18
View File
@@ -6,8 +6,8 @@ GovOPlaN separates governed data identity from external acquisition:
| Layer | Owner | Responsibilities |
| --- | --- | --- |
| Datasource layer | `govoplan-datasources` | Catalogue, tenant visibility, live/cached/static mode, staging, immutable materializations, frozen states, schema, fingerprints, provenance, and bounded reads |
| Connector layer | `govoplan-connectors` and protocol/provider modules | External protocols, endpoints, connection profiles, credentials, discovery, provider health, source-side filtering, and query pushdown |
| Datasource layer | `govoplan-datasources` | Governed data/register catalogue, tenant visibility, live/cached/static mode, source authority, staging, immutable materializations, frozen states, schema, quality/freshness policy, institutional provenance, dependencies, and bounded reads |
| Connector layer | `govoplan-connectors` and protocol/provider modules | External protocols, endpoints, connection profiles, credentials, discovery, provider maturity/authority support, health, source-side filtering, query pushdown, and effect reconciliation |
Connectors publish versioned datasource origins. Datasources registers those
origins and presents one stable capability to Dataflow, Workflow, Reporting,
@@ -43,9 +43,10 @@ Domain modules own their semantics:
- Dataflow provides load, combine, filter, transform, and output nodes. Its
graph is acyclic and has one output.
- Workflow provides trigger, activity, review, decision, wait, module-action,
Dataflow, and outcome nodes. It permits governed loops and has exactly one
trigger plus one or more outcomes.
- Workflow Engine provides trigger, activity, review, decision, wait,
module-action, Dataflow, and outcome semantics. It permits governed loops and
has exactly one trigger plus one or more outcomes. The optional Workflow
module supplies the editor over the same native graph/BPMN language.
This division permits a shared editor shell without making Workflow a special
kind of Dataflow or leaking either module into Core.
@@ -53,8 +54,10 @@ kind of Dataflow or leaking either module into Core.
## Current Implementation
- Core graph and datasource contracts are versioned at `0.1.0`.
- Workflow exposes a reusable graph editor, node-library discovery, validation,
tenant-isolated definitions, immutable revisions, and activation pinning.
- Workflow Engine owns tenant-isolated definitions, immutable revisions,
activation pinning, module-contributed versioned baselines, runtime instances,
governed action/effect execution, retries, waits, and reconciliation. The
optional Workflow module exposes the reusable native BPMN graph editor.
- Datasources exposes catalogue, origins, staging, promotion, preview,
materialization history, refresh, freeze, retirement, and producer
publication APIs.
@@ -62,21 +65,26 @@ kind of Dataflow or leaking either module into Core.
- Connectors adapts existing tabular snapshots to datasource origins.
- Dataflow consumes only Datasources catalogue/lifecycle capabilities and can
request current, live, or latest-frozen state.
- Dataflow exposes a pinned run-lifecycle capability and a Run/Publish surface.
Its first synchronous runner records lineage and terminal state, publishes
only complete bounded results, and retains output datasource/materialization
references.
- Dataflow exposes typed graph/IR, registry-driven validation/execution/SQL
compilation, expressions and reusable subflows, a pinned run-lifecycle
capability, production worker boundary, and Run/Publish surface. Runs record
lineage and intermediate artifacts and publish only complete bounded results.
- The focused composition check proves Connector origin -> Datasource ->
pinned Dataflow run -> frozen published materialization, including replay.
## Next Slices
1. Add persisted Workflow instances, resumable transitions, human activities,
retry policy, and event subscriptions against pinned definition revisions.
2. Add SQL database and governed REST origin providers with credential-envelope
1. Add the provider declaration and source-authority binding used consistently
by Connectors, Datasources, configuration packages, Ops, and Docs.
2. Add typed datasource owner/steward, legal/purpose, quality/freshness,
classification, correction, service/process, and downstream dependency
metadata under
[Datasources #6](https://git.add-ideas.de/GovOPlaN/govoplan-datasources/issues/6).
3. Add SQL database and governed REST origin providers with credential-envelope
references and bounded pushdown.
3. Add managed-file and directory origins.
4. Add datasource quality rules, schema compatibility policy, retention, and
4. Add managed-file and directory origins.
5. Complete datasource quality rules, schema compatibility policy, retention, and
promotion approvals.
5. Add asynchronous Dataflow workers and durable artifact-backed outputs for
runs that exceed the synchronous row/time/byte limits.
6. Complete scheduled/event/API/chained Dataflow trigger governance, reusable
template inheritance, Reporting publication, human reconciliation transforms,
and target resource/recovery evidence for large runs.