42 lines
1.5 KiB
Markdown
42 lines
1.5 KiB
Markdown
# GovOPlaN Dataflow
|
|
|
|
GovOPlaN Dataflow defines and runs governed tabular transformation pipelines.
|
|
Power users can work with the same pipeline as a graphical node graph or as a
|
|
constrained SQL query. Every saved definition change produces an immutable revision, and
|
|
every preview records diagnostics and reproducibility metadata without storing
|
|
the previewed row contents.
|
|
|
|
## Boundary
|
|
|
|
- **Dataflow:** pipeline graphs, revisions, validation, constrained SQL,
|
|
previews/runs, diagnostics, and lineage.
|
|
- **Connectors:** source connections, credentials, discovery, health, and
|
|
bounded source access.
|
|
- **Reporting:** governed datasets, analytical views, dashboards, and exports.
|
|
- **Workflow:** orchestration, resumability, approvals, and module handoffs.
|
|
- **Risk Compliance:** sanctions matching policy, review, dispositions, and
|
|
legal evidence.
|
|
|
|
The first implementation supports bounded inline tabular sources and the
|
|
`filter`, `select`, `aggregate`, `sort`, `limit`, and `output` transforms. The
|
|
connector-backed source contract is tracked separately so credentials never
|
|
become part of a pipeline definition.
|
|
|
|
## Development
|
|
|
|
```bash
|
|
/mnt/DATA/git/govoplan/.venv/bin/python -m pip install -e .
|
|
/mnt/DATA/git/govoplan/.venv/bin/python -m unittest discover -s tests
|
|
```
|
|
|
|
WebUI:
|
|
|
|
```bash
|
|
cd webui
|
|
PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:$PATH npm run typecheck
|
|
npm run test:structure
|
|
```
|
|
|
|
The implementation epic is
|
|
[`govoplan-dataflow#1`](https://git.add-ideas.de/GovOPlaN/govoplan-dataflow/issues/1).
|