feat: consume governed datasources and shared graphs
This commit is contained in:
17
README.md
17
README.md
@@ -10,8 +10,10 @@ the previewed row contents.
|
||||
|
||||
- **Dataflow:** pipeline graphs, revisions, validation, constrained SQL,
|
||||
previews/runs, diagnostics, and lineage.
|
||||
- **Connectors:** source connections, credentials, discovery, health, and
|
||||
bounded source access.
|
||||
- **Datasources:** governed source identity, staging, materializations, frozen
|
||||
states, and bounded source access.
|
||||
- **Connectors:** external acquisition, credentials, discovery, and provider
|
||||
health.
|
||||
- **Reporting:** governed datasets, analytical views, dashboards, and exports.
|
||||
- **Workflow:** orchestration, resumability, approvals, and module handoffs.
|
||||
- **Risk Compliance:** sanctions matching policy, review, dispositions, and
|
||||
@@ -24,7 +26,7 @@ nodes by purpose:
|
||||
|
||||
| Group | Nodes |
|
||||
| --- | --- |
|
||||
| Load | Inline data, connector source |
|
||||
| Load | Inline data, datasource |
|
||||
| Combine | Append rows, join tables |
|
||||
| Filter | Filter rows, remove duplicates |
|
||||
| Transform | Select columns, derive column, aggregate, sort, limit |
|
||||
@@ -34,10 +36,11 @@ Join nodes have explicit left and right ports. Append nodes accept two or more
|
||||
inputs. Derived columns use a constrained operation catalogue rather than
|
||||
arbitrary code.
|
||||
|
||||
Connector sources are resolved through the versioned Core capability
|
||||
`connectors.tabular_sources`; Dataflow does not import Connectors or store its
|
||||
credentials. Source references pin an expected fingerprint so schema/content
|
||||
drift fails visibly instead of silently changing a run.
|
||||
Governed inputs are resolved through the versioned Core capability
|
||||
`datasources.catalogue`; Dataflow imports neither Datasources nor Connectors and
|
||||
stores only opaque datasource references plus expected fingerprints. A source
|
||||
node can request the current, live, or latest frozen state. Fingerprint drift
|
||||
fails visibly instead of silently changing a run.
|
||||
|
||||
## SQL And Preview Safety
|
||||
|
||||
|
||||
Reference in New Issue
Block a user