Expand governed Dataflow editor and node library
This commit is contained in:
38
README.md
38
README.md
@@ -17,10 +17,40 @@ the previewed row contents.
|
||||
- **Risk Compliance:** sanctions matching policy, review, dispositions, and
|
||||
legal evidence.
|
||||
|
||||
The first implementation supports bounded inline tabular sources and the
|
||||
`filter`, `select`, `aggregate`, `sort`, `limit`, and `output` transforms. The
|
||||
connector-backed source contract is tracked separately so credentials never
|
||||
become part of a pipeline definition.
|
||||
## Node Library
|
||||
|
||||
The canonical backend catalogue is exposed to the WebUI and groups executable
|
||||
nodes by purpose:
|
||||
|
||||
| Group | Nodes |
|
||||
| --- | --- |
|
||||
| Load | Inline data, connector source |
|
||||
| Combine | Append rows, join tables |
|
||||
| Filter | Filter rows, remove duplicates |
|
||||
| Transform | Select columns, derive column, aggregate, sort, limit |
|
||||
| Output | Preview output |
|
||||
|
||||
Join nodes have explicit left and right ports. Append nodes accept two or more
|
||||
inputs. Derived columns use a constrained operation catalogue rather than
|
||||
arbitrary code.
|
||||
|
||||
Connector sources are resolved through the versioned Core capability
|
||||
`connectors.tabular_sources`; Dataflow does not import Connectors or store its
|
||||
credentials. Source references pin an expected fingerprint so schema/content
|
||||
drift fails visibly instead of silently changing a run.
|
||||
|
||||
## SQL And Preview Safety
|
||||
|
||||
The SQL workbench parses one `SELECT` statement into the canonical graph. The
|
||||
dialect supports projection, aliases, filters, grouping, aggregate functions,
|
||||
sorting, limits, `DISTINCT`, and one two-source equi-join. It rejects DDL, DML,
|
||||
subqueries, arbitrary functions, file access, and unchecked pass-through
|
||||
execution.
|
||||
|
||||
Preview reads at most 250 rows per source and enforces time, intermediate-row,
|
||||
result-byte, graph-node, and response-row bounds. Saved previews record the
|
||||
pipeline revision, executor version, source fingerprints, node diagnostics,
|
||||
and output summary, but not source or result rows.
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
Reference in New Issue
Block a user