feat: implement governed datasource catalogue metadata
This commit is contained in:
+48
-1
@@ -2,7 +2,11 @@
|
||||
|
||||
## Boundary
|
||||
|
||||
Datasources owns the governed identity and lifecycle of consumable data.
|
||||
Datasources owns the governed identity and lifecycle of consumable data and
|
||||
registers. It is not only a technical connection list: it is the catalogue in
|
||||
which an institution explains what a dataset means, who is responsible for it,
|
||||
why it may be used, how current and trustworthy it is, and which products or
|
||||
decisions depend on it.
|
||||
Connectors owns external protocols, connection profiles, credentials, provider
|
||||
health, discovery, and source-side query pushdown.
|
||||
|
||||
@@ -26,6 +30,45 @@ only retain opaque datasource and materialization references.
|
||||
Materializations are append-only. Changing source data creates a new revision;
|
||||
old revisions remain addressable for reproducibility.
|
||||
|
||||
The mode above describes how data is read. A separate authority declaration
|
||||
states whether GovOPlaN is authoritative, an external source is authoritative,
|
||||
the local state is a mirror, changes are governed in both directions, GovOPlaN
|
||||
adds a governance overlay, or the entry is link-only.
|
||||
|
||||
## Governance Catalogue
|
||||
|
||||
A catalogue entry has typed, queryable governance fields for:
|
||||
|
||||
- owner, steward, and responsible organization/function references;
|
||||
- authoritative source, source-authority mode, and transfer/data-sharing
|
||||
agreement references;
|
||||
- legal or organizational basis, declared purposes, and permitted consumers;
|
||||
- semantic definition, schema owner, official keys, and correction procedure;
|
||||
- classification, privacy constraints, retention, legal hold, and publication
|
||||
rules;
|
||||
- freshness objective, quality policy, validation status, known limitations,
|
||||
and incident state;
|
||||
- affected services/processes and dependent Dataflows, reports, controls,
|
||||
decisions, and published outputs.
|
||||
|
||||
Authority mode, classification, publication state, owner, and responsible
|
||||
organization are directly filterable catalogue columns. Lists and structured
|
||||
quality/freshness rules retain typed API shapes. The governance editor is
|
||||
available to datasource managers; readers see the effective explanation next
|
||||
to the data preview. Each materialization captures the complete governance
|
||||
state at publication time, so later ownership or policy edits do not rewrite
|
||||
the explanation attached to prior evidence.
|
||||
|
||||
Affected-object and dependency references are also normalized into an indexed,
|
||||
tenant-scoped relation table while the original JSON lists remain readable for
|
||||
compatibility. Catalogue clients can filter by exact `affected_ref` and
|
||||
`dependency_ref` values without receiving row access to either the datasource
|
||||
or the referenced object.
|
||||
|
||||
Changing catalogue governance is audited. It does not rewrite source rows,
|
||||
grant row access, or alter connector credentials. External and synchronized
|
||||
authority modes require an explicit authoritative-source reference.
|
||||
|
||||
## Staging
|
||||
|
||||
Staging is the inspection boundary before data becomes generally consumable.
|
||||
@@ -49,6 +92,10 @@ Consumers request:
|
||||
Reads are bounded and tenant-scoped. Schema and expected fingerprints allow a
|
||||
Dataflow or Workflow definition to detect changed inputs before execution.
|
||||
|
||||
Consumers should be able to request the governance explanation and dependency
|
||||
impact separately from row access. Seeing catalogue metadata must not imply
|
||||
permission to read protected data.
|
||||
|
||||
## Next Providers
|
||||
|
||||
Connector providers should cover:
|
||||
|
||||
Reference in New Issue
Block a user