5.0 KiB
govoplan-datasources
Repository type: module (platform).
govoplan-datasources is GovOPlaN's provider-neutral data access layer. It owns
the tenant catalogue, lifecycle, staging, immutable materializations, frozen
states, schema and fingerprint metadata, provenance, and bounded reads.
The module deliberately separates two layers:
- Datasources describe governed data that modules can consume consistently.
- Connectors acquire data from external systems and expose provider origins.
Dataflow, Workflow, Reporting, Risk Compliance, and other consumers depend on Datasource contracts rather than connector implementations.
The first executable slice supports tabular static uploads, connector-backed live and cached sources, staging and promotion, refresh, immutable snapshots, explicit frozen states, previews, retirement, and atomic producer publication. Origin discovery retains each provider's source mode, structured health, and declared pushdown support. Live previews preserve the provider's effective row, serialized-byte, and elapsed-time limits and its redacted diagnostics. Producer modules can append a bounded inline tabular result or pin a larger durable artifact through the same idempotent capability. Artifact references declare a backend, locator, SHA-256 checksum, schema, fingerprint, row and byte counts; the installed provider verifies integrity and serves bounded reads. The publication ledger retains the producer run, output materialization, provenance, and replay identity. On PostgreSQL, a transaction-scoped advisory lock serializes each tenant, producer, and idempotency identity before any output side effect, so retries from multiple application or worker nodes resolve to one durable publication.
Tabular staging also evaluates governed quality rules and classifies schema changes before promotion. Blocking stages remain inspectable, and successful promotion preserves the exact policy hash and validation result in immutable materialization provenance. The supported contract is documented in docs/QUALITY_POLICY.md.
The same lifecycle contract can require an attributable, separated approval quorum before a stage or cached refresh becomes current. Versioned retention rules cover transient stages, ordinary materializations, and frozen evidence. Retention remains an explicit preview-and-apply operation: current revisions, legal holds, pending approvals, and publication evidence are blocked, while every decision, promotion, and disposition is preserved in hash-chained local evidence even when the optional Audit, Access, or Policy modules are absent.
The contracts already model database, HTTP/REST, directory, file, feed,
document, binary, directory, and stream sources so providers can be added
without changing consumers. Storage modules contribute artifact backends
through the provider-neutral datasources.artifactBackends capability; the
Datasources module never imports their internals.
See docs/CONCEPT.md for ownership and lifecycle details.
Data-subject requests
Datasources publishes privacy.dsar.datasources for exact catalogue,
governance-reference, materialization, payload, stage, publication, and
lifecycle-evidence
references and for minimized operator attribution. It never exports connector
references, locators, credentials, arbitrary rows, schemas, validation
samples, metadata, provenance bodies, checkpoints, replay material, or hashes.
The module does not guess subject identity by scanning schema-dependent tabular
payloads; the authoritative source module locates and corrects those facts.
Unpromoted stages and unreferenced payloads can be deleted idempotently. Published or referenced state, immutable materializations, lifecycle evidence, governance evidence, holds, and operator attribution require data-steward review. Dataflow and Reporting derivatives must be refreshed after the source correction.
Git-source WebUI package
The repository root exposes @govoplan/datasources-webui for Git-tagged release
dependencies. It mirrors the owning webui/package.json version, public
TypeScript/CSS exports and peer requirements, with entry paths under
webui/src. Consumers provide the shared Core/React peers; the facade runs no
development or install scripts. The source archive contains webui/src, this
README and any repository license file. Run module development checks from webui/; Python
installation remains governed by pyproject.toml.
Das Repository stellt @govoplan/datasources-webui am Wurzelpfad für versionierte
Git-Abhängigkeiten bereit. Version, öffentliche TypeScript-/CSS-Exporte und
Peer-Anforderungen entsprechen webui/package.json; die Einstiegspfade liegen
unter webui/src. Gemeinsame Core-/React-Peers stellt die einbindende Anwendung
bereit. Die Fassade führt keine Entwicklungs- oder Installationsskripte aus.
Entwicklungsprüfungen bleiben in webui/, die Python-Installation weiterhin in
pyproject.toml definiert.