Expand governed Dataflow editor and node library

This commit is contained in:
2026-07-28 11:14:01 +02:00
parent df468a2bd8
commit dee8380631
22 changed files with 3564 additions and 291 deletions

View File

@@ -17,10 +17,40 @@ the previewed row contents.
- **Risk Compliance:** sanctions matching policy, review, dispositions, and
legal evidence.
The first implementation supports bounded inline tabular sources and the
`filter`, `select`, `aggregate`, `sort`, `limit`, and `output` transforms. The
connector-backed source contract is tracked separately so credentials never
become part of a pipeline definition.
## Node Library
The canonical backend catalogue is exposed to the WebUI and groups executable
nodes by purpose:
| Group | Nodes |
| --- | --- |
| Load | Inline data, connector source |
| Combine | Append rows, join tables |
| Filter | Filter rows, remove duplicates |
| Transform | Select columns, derive column, aggregate, sort, limit |
| Output | Preview output |
Join nodes have explicit left and right ports. Append nodes accept two or more
inputs. Derived columns use a constrained operation catalogue rather than
arbitrary code.
Connector sources are resolved through the versioned Core capability
`connectors.tabular_sources`; Dataflow does not import Connectors or store its
credentials. Source references pin an expected fingerprint so schema/content
drift fails visibly instead of silently changing a run.
## SQL And Preview Safety
The SQL workbench parses one `SELECT` statement into the canonical graph. The
dialect supports projection, aliases, filters, grouping, aggregate functions,
sorting, limits, `DISTINCT`, and one two-source equi-join. It rejects DDL, DML,
subqueries, arbitrary functions, file access, and unchecked pass-through
execution.
Preview reads at most 250 rows per source and enforces time, intermediate-row,
result-byte, graph-node, and response-row bounds. Saved previews record the
pipeline revision, executor version, source fingerprints, node diagnostics,
and output summary, but not source or result rows.
## Development