feat: declare governed external provider state

This commit is contained in:
2026-08-01 17:48:25 +02:00
parent 52fe33568c
commit 7be93785a2
6 changed files with 578 additions and 12 deletions
+16
View File
@@ -0,0 +1,16 @@
# GovOPlaN Connectors Codex Guide
## Scope
This repository owns reusable external connection profiles, protocol adapters, governed snapshots, and connector capability contracts.
## Documentation Contract
- Treat documentation as part of every behavior change. Update this module's manifest-driven `DocumentationTopic` contributions for affected user and administrator behavior.
- Keep feature content here; `govoplan-docs` projects it without importing Connectors internals.
- Maintain a static user/admin baseline and run `/mnt/DATA/git/govoplan/tools/checks/check-manifest-shapes.py` after behavior or manifest changes.
## Boundaries
- Domain modules own business semantics; connectors own transport, credentials, synchronization, and diagnostics.
- Keep optional adapters behind capabilities and enforce egress and peer-validation policy.
+13
View File
@@ -11,6 +11,13 @@ results, credential references, and generic integration events. Protocol-heavy
or domain-heavy integrations may live in dedicated modules once their scope is or domain-heavy integrations may live in dedicated modules once their scope is
clear. clear.
Connector capability is not source ownership. Each configured binding also
declares whether GovOPlaN is authoritative, the external system is
authoritative, GovOPlaN keeps a mirror, both sides use governed synchronization,
GovOPlaN supplies only a governance overlay, or the object is link-only. The
same connector may be configured differently by tenant, service, object type,
or field group.
Detailed follow-up documents: Detailed follow-up documents:
- [Public-sector integration catalogue](PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md) - [Public-sector integration catalogue](PUBLIC_SECTOR_INTEGRATION_CATALOGUE.md)
@@ -29,6 +36,8 @@ The module owns:
- connector health status and last-test evidence - connector health status and last-test evidence
- operator-visible integration inventory - operator-visible integration inventory
- cross-module discovery of available external capabilities - cross-module discovery of available external capabilities
- supported integration maturity, source-authority modes, operation limits,
and effect/reconciliation behavior for each connector type
The module does not own: The module does not own:
@@ -61,6 +70,9 @@ The module should integrate through:
- module manifest metadata, route factories, permissions, and migrations - module manifest metadata, route factories, permissions, and migrations
- a connector catalogue API for listing available connector types - a connector catalogue API for listing available connector types
- a connection profile API with secret references, not plaintext secrets - a connection profile API with secret references, not plaintext secrets
- a provider declaration that composes authority mode, maturity, supported
operations, revisions/freshness, health, limits, idempotency, conflicts,
evidence, and reconciliation behavior
- capability declarations such as `connectors.catalog`, - capability declarations such as `connectors.catalog`,
`connectors.profileTester`, and `connectors.health` `connectors.profileTester`, and `connectors.health`
- events such as `connector.profile_created`, `connector.test_succeeded`, - events such as `connector.profile_created`, `connector.test_succeeded`,
@@ -113,6 +125,7 @@ The first implementation should provide:
- WebUI catalogue and profile pages - WebUI catalogue and profile pages
- configuration-package fragment support - configuration-package fragment support
- generic external-reference DTOs - generic external-reference DTOs
- source-authority binding and provider-operation metadata
- health summary provider - health summary provider
## Permissions ## Permissions
+41 -2
View File
@@ -14,6 +14,22 @@ predictable and avoids hidden module imports.
- `bidirectional`: GovOPlaN supports both directions with conflict detection and - `bidirectional`: GovOPlaN supports both directions with conflict detection and
reconciliation rules. reconciliation rules.
Direction describes transport. Every binding also needs a source-authority
mode:
- `native_authoritative`
- `external_authoritative`
- `external_mirror`
- `governed_sync`
- `governance_overlay`
- `linked_reference`
The authority mode and the connector's integration maturity are orthogonal. A
bidirectional connector may be configured as an external mirror, and a native
GovOPlaN object may publish to an external target without transferring
authority. The effective binding must identify its scope and provenance rather
than relying on a profile-wide `sync` boolean.
## Source Data Lifecycle ## Source Data Lifecycle
Connector profiles have operational states, while individual external records Connector profiles have operational states, while individual external records
@@ -103,8 +119,11 @@ this lifecycle when a connector publishes status.
shape. shape.
3. Connector sends the remote request. 3. Connector sends the remote request.
4. Connector stores the remote id, version/ETag, and response diagnostics. 4. Connector stores the remote id, version/ETag, and response diagnostics.
5. Connector emits `connector.record_published` or `connector.publish_failed`. 5. A timeout or lost acknowledgement after dispatch becomes outcome-unknown,
6. Domain module stores only the external-reference DTO and any domain result. not an ordinary failure or permission to duplicate the command.
6. Connector emits a confirmed, retryable, outcome-unknown, reconciled, or
corrected result event.
7. Domain module stores only the external-reference DTO and any domain result.
## Reconciliation ## Reconciliation
@@ -116,6 +135,25 @@ Every connector that writes to an external system needs a reconciliation story:
- retry policy for temporary failures - retry policy for temporary failures
- explicit operator action for destructive overwrite or deletion - explicit operator action for destructive overwrite or deletion
- audit trace from GovOPlaN record to external request and response summary - audit trace from GovOPlaN record to external request and response summary
- explicit requested, approved, dispatched, possibly-executed, confirmed, and
reconciled/corrected effect states
## Provider Declaration
An executable connector type should publish machine-readable metadata for:
- owned object and field groups, plus supported authority modes;
- supported discovery, link, search, read, publish, synchronize, migrate, and
replacement maturity;
- read/write/delete/preview/dry-run operations and bounded response limits;
- revision/concurrency token, freshness, health, timeout, retry, and conflict
semantics;
- idempotency and outcome-unknown handling;
- evidence, rollback/compensation, correction, and reconciliation paths;
- classification, purpose, retention, secret, degraded, and outage behavior.
This declaration composes Core contracts. It does not move protocol behavior
or domain semantics into Core or Connectors.
## Capability Boundary ## Capability Boundary
@@ -158,5 +196,6 @@ Before shipping an executable connector type:
- Add unavailable-optional-module tests for every consuming domain module. - Add unavailable-optional-module tests for every consuming domain module.
- Add profile test and health status fixtures. - Add profile test and health status fixtures.
- Add external-reference DTO tests. - Add external-reference DTO tests.
- Add source-authority and provider-declaration validation tests.
- Add lifecycle transition tests for pause, retry, retirement, and uninstall - Add lifecycle transition tests for pause, retry, retirement, and uninstall
guard behavior. guard behavior.
+183 -10
View File
@@ -20,6 +20,15 @@ from govoplan_core.core.modules import (
PermissionDefinition, PermissionDefinition,
RoleTemplate, RoleTemplate,
) )
from govoplan_core.core.provider_governance import (
ExternalProviderDeclaration,
ExternalProviderStateProviderRegistration,
ModuleArchitectureDeclaration,
ModuleArchitectureDocumentation,
ModuleMaturityEvidence,
ProviderBehaviorDeclaration,
ProviderObjectDeclaration,
)
from govoplan_core.core.tabular_sources import ( from govoplan_core.core.tabular_sources import (
CAPABILITY_CONNECTORS_TABULAR_SNAPSHOT_WRITER, CAPABILITY_CONNECTORS_TABULAR_SNAPSHOT_WRITER,
CAPABILITY_CONNECTORS_TABULAR_SOURCES, CAPABILITY_CONNECTORS_TABULAR_SOURCES,
@@ -48,6 +57,12 @@ from govoplan_connectors.backend.datasource_origins import (
ConnectorDatasourceOriginProvider, ConnectorDatasourceOriginProvider,
) )
from govoplan_connectors.backend.feeds import ConnectorFeedProvider from govoplan_connectors.backend.feeds import ConnectorFeedProvider
from govoplan_connectors.backend.provider_state import (
SANCTIONS_PROVIDER_ID,
TABULAR_PROVIDER_ID,
sanctions_provider_states,
tabular_provider_states,
)
MODULE_ID = "connectors" MODULE_ID = "connectors"
@@ -57,6 +72,142 @@ DATASOURCE_ORIGIN_INTERFACE_VERSION = "0.1.0"
SANCTIONS_SNAPSHOT_INTERFACE_VERSION = "1.0.0" SANCTIONS_SNAPSHOT_INTERFACE_VERSION = "1.0.0"
FEED_INTERFACE_VERSION = "0.1.0" FEED_INTERFACE_VERSION = "0.1.0"
ARCHITECTURE = ModuleArchitectureDeclaration(
layer="data_reporting_integration",
kind="integration",
maturity="vertical_slice",
evidence=(
ModuleMaturityEvidence(
kind="test",
reference="tests/test_tabular_sources.py",
summary="Exercises tenant-safe immutable tabular snapshots and bounded reads.",
),
ModuleMaturityEvidence(
kind="test",
reference="tests/test_sanctions_sources.py",
summary="Exercises source acquisition health, checksums, retries, and immutable evidence.",
),
ModuleMaturityEvidence(
kind="documentation",
reference="docs/CONNECTOR_SOURCE_LIFECYCLE.md",
summary="Defines source lifecycle, authority, evidence, and outage boundaries.",
),
),
known_limits=(
"The executable generic datasource origin is an immutable tabular snapshot; database and arbitrary REST profiles remain future providers.",
"Feed publication renders a governed document but does not yet push it to an external publishing endpoint.",
),
supported_authority_modes=(
"external_authoritative",
"external_mirror",
"linked_reference",
),
owned_concepts=(
"external transport profiles",
"protocol interaction",
"immutable connector snapshots",
"connector acquisition health",
),
non_owned_concepts=(
"datasource catalogue identity and lifecycle",
"domain records and business semantics",
"data transformations",
"screening dispositions",
),
target_tested_providers=(
TABULAR_PROVIDER_ID,
SANCTIONS_PROVIDER_ID,
),
documentation=ModuleArchitectureDocumentation(
migration=("src/govoplan_connectors/backend/migrations/versions",),
upgrade=("docs/CONNECTOR_SOURCE_LIFECYCLE.md",),
recovery=("docs/CONNECTOR_SOURCE_LIFECYCLE.md",),
security=("docs/CONNECTOR_SOURCE_LIFECYCLE.md",),
operations=("docs/CONNECTOR_SOURCE_LIFECYCLE.md",),
),
)
EXTERNAL_PROVIDERS = (
ExternalProviderDeclaration(
id=TABULAR_PROVIDER_ID,
module_id=MODULE_ID,
label="Immutable tabular snapshot provider",
maturity="read",
operations=("discover", "search", "read", "preview", "dry_run"),
objects=(
ProviderObjectDeclaration(
object_type="tabular_source_snapshot",
field_groups=("identity", "schema", "rows", "source_provenance"),
authority_modes=("external_authoritative", "external_mirror"),
default_authority_mode="external_mirror",
),
),
behavior=ProviderBehaviorDeclaration(
revision_tokens="Source fingerprints and immutable snapshot ids are retained.",
concurrency="Reads may require the expected fingerprint; snapshots never mutate in place.",
freshness="Snapshot acquisition time and source timestamp are exposed.",
health="Import validation and source-read failures are explicit.",
max_read_items=1000,
evidence="Rows, schema, fingerprint, source metadata, and acquisition provenance remain linked.",
correction="Import a replacement snapshot; retain the prior snapshot as evidence.",
reconciliation="Compare source and snapshot fingerprints before selecting a new current state.",
outage="Existing snapshots remain available and visibly stale; no live-source claim is made.",
classifications=("internal", "confidential", "restricted"),
purposes=("governed import", "dataflow input", "evidence reconstruction"),
retention="Datasources or the consuming domain supplies retention and hold policy.",
secret_handling="Generic snapshots contain no connector credential; transport credentials stay in credential envelopes.",
),
capability_names=(
CAPABILITY_CONNECTORS_TABULAR_SOURCES,
CAPABILITY_DATASOURCE_ORIGINS,
),
interface_names=(
"connectors.tabular_sources",
"connectors.datasource_origins",
),
documentation_topic_ids=(
"connectors.authority-and-effects",
"connectors.tabular-sources",
),
),
ExternalProviderDeclaration(
id=SANCTIONS_PROVIDER_ID,
module_id=MODULE_ID,
label="Sanctions source snapshot provider",
maturity="read",
operations=("discover", "search", "read", "preview"),
objects=(
ProviderObjectDeclaration(
object_type="sanctions_source_snapshot",
field_groups=("source_identity", "raw_evidence", "entries", "acquisition_health"),
authority_modes=("external_authoritative", "external_mirror"),
default_authority_mode="external_mirror",
),
),
behavior=ProviderBehaviorDeclaration(
revision_tokens="Provider source version, ETag, Last-Modified, and SHA-256 digest are retained when available.",
concurrency="Refreshes use conditional source requests and create immutable snapshots.",
freshness="Latest successful acquisition, source timestamp, and stale health are reported.",
health="Transport, parsing, source-change, and malformed-source states are explicit.",
max_read_items=5000,
evidence="Raw source bytes, checksum, acquisition run, parser result, and normalized entry count are linked.",
correction="A corrected source creates a new immutable snapshot and acquisition run.",
reconciliation="Compare source version and digest, then preserve both prior and corrected evidence.",
outage="The latest accepted snapshot stays usable with stale/unavailable source health.",
classifications=("public", "internal"),
purposes=("sanctions source acquisition", "compliance screening evidence"),
retention="Risk and Records policies determine accepted snapshot retention and legal holds.",
secret_handling="Public sources require no subject data or source credential; configured proxy secrets remain external to snapshots.",
),
capability_names=(CAPABILITY_CONNECTORS_SANCTIONS_SNAPSHOTS,),
interface_names=("connectors.sanctions_snapshots",),
documentation_topic_ids=(
"connectors.authority-and-effects",
"connectors.sanctions-snapshots",
),
),
)
def _permission(scope: str, label: str, description: str) -> PermissionDefinition: def _permission(scope: str, label: str, description: str) -> PermissionDefinition:
module_id, resource, action = scope.split(":", 2) module_id, resource, action = scope.split(":", 2)
@@ -157,17 +308,12 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
), ),
"connector_sanctions_snapshots": ( "connector_sanctions_snapshots": (
session.query(ConnectorSanctionsSnapshot) session.query(ConnectorSanctionsSnapshot)
.filter( .filter(ConnectorSanctionsSnapshot.tenant_id == tenant_id)
ConnectorSanctionsSnapshot.tenant_id == tenant_id
)
.count() .count()
), ),
"connector_sanctions_runs": ( "connector_sanctions_runs": (
session.query(ConnectorSanctionsAcquisitionRun) session.query(ConnectorSanctionsAcquisitionRun)
.filter( .filter(ConnectorSanctionsAcquisitionRun.tenant_id == tenant_id)
ConnectorSanctionsAcquisitionRun.tenant_id
== tenant_id
)
.count() .count()
), ),
} }
@@ -220,12 +366,24 @@ manifest = ModuleManifest(
CAPABILITY_CONNECTORS_TABULAR_SOURCES: _provider, CAPABILITY_CONNECTORS_TABULAR_SOURCES: _provider,
CAPABILITY_CONNECTORS_TABULAR_SNAPSHOT_WRITER: _provider, CAPABILITY_CONNECTORS_TABULAR_SNAPSHOT_WRITER: _provider,
CAPABILITY_DATASOURCE_ORIGINS: _datasource_origin_provider, CAPABILITY_DATASOURCE_ORIGINS: _datasource_origin_provider,
CAPABILITY_CONNECTORS_SANCTIONS_SNAPSHOTS: ( CAPABILITY_CONNECTORS_SANCTIONS_SNAPSHOTS: (_sanctions_snapshot_provider),
_sanctions_snapshot_provider
),
CAPABILITY_CONNECTORS_FEEDS: _feed_provider, CAPABILITY_CONNECTORS_FEEDS: _feed_provider,
}, },
tenant_summary_providers=(_tenant_summary,), tenant_summary_providers=(_tenant_summary,),
architecture=ARCHITECTURE,
external_providers=EXTERNAL_PROVIDERS,
external_provider_state_providers=(
ExternalProviderStateProviderRegistration(
module_id=MODULE_ID,
provider_id=TABULAR_PROVIDER_ID,
provider=tabular_provider_states,
),
ExternalProviderStateProviderRegistration(
module_id=MODULE_ID,
provider_id=SANCTIONS_PROVIDER_ID,
provider=sanctions_provider_states,
),
),
migration_spec=MigrationSpec( migration_spec=MigrationSpec(
module_id=MODULE_ID, module_id=MODULE_ID,
metadata=Base.metadata, metadata=Base.metadata,
@@ -251,6 +409,21 @@ manifest = ModuleManifest(
), ),
), ),
documentation=( documentation=(
DocumentationTopic(
id="connectors.authority-and-effects",
title="Connector authority and effect behavior",
summary="Connector direction, technical maturity, and configured source authority are separate and must remain visible.",
body=(
"A connector can consume, publish, or work bidirectionally and can mature from discovery through replacement. "
"Each binding separately states whether GovOPlaN is authoritative, follows an external authority, keeps a mirror, synchronizes under conflict rules, adds a governance overlay, or retains only a link. "
"Writable providers must explain revisions, limits, idempotency, outcome-unknown handling, evidence, reconciliation, correction, outage behavior, and secret requirements."
),
layer="available",
documentation_types=("admin", "user"),
audience=("operator", "module_admin", "power_user", "product_owner"),
related_modules=("datasources", "dataflow", "ops", "policy", "audit"),
order=39,
),
DocumentationTopic( DocumentationTopic(
id="connectors.tabular-sources", id="connectors.tabular-sources",
title="Governed tabular sources", title="Governed tabular sources",
@@ -0,0 +1,207 @@
from __future__ import annotations
from collections import defaultdict
from datetime import UTC, datetime
from hashlib import sha256
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from govoplan_connectors.backend.db.models import (
ConnectorSanctionsAcquisitionRun,
ConnectorSanctionsSnapshot,
ConnectorTabularSource,
)
from govoplan_core.core.provider_governance import (
ExternalProviderRuntimeState,
ExternalProviderStateContext,
)
TABULAR_PROVIDER_ID = "connectors.tabular_snapshot"
SANCTIONS_PROVIDER_ID = "connectors.sanctions_snapshot"
def tabular_provider_states(
context: ExternalProviderStateContext,
) -> tuple[ExternalProviderRuntimeState, ...]:
session = _session(context)
statement = select(ConnectorTabularSource).where(
ConnectorTabularSource.deleted_at.is_(None)
)
if context.tenant_id is not None:
statement = statement.where(
ConnectorTabularSource.tenant_id == context.tenant_id
)
sources = tuple(
session.scalars(
statement.order_by(
ConnectorTabularSource.tenant_id,
ConnectorTabularSource.id,
).limit(context.max_items + 1)
)
)
observed_at = datetime.now(UTC)
return tuple(_tabular_state(item, observed_at=observed_at) for item in sources)
def sanctions_provider_states(
context: ExternalProviderStateContext,
) -> tuple[ExternalProviderRuntimeState, ...]:
session = _session(context)
statement = select(ConnectorSanctionsAcquisitionRun)
if context.tenant_id is not None:
statement = statement.where(
ConnectorSanctionsAcquisitionRun.tenant_id == context.tenant_id
)
runs = tuple(
session.scalars(
statement.order_by(
ConnectorSanctionsAcquisitionRun.tenant_id,
ConnectorSanctionsAcquisitionRun.provider_id,
ConnectorSanctionsAcquisitionRun.source_id,
ConnectorSanctionsAcquisitionRun.started_at.desc(),
).limit(max(context.max_items * 10, context.max_items + 1))
)
)
latest_by_binding: dict[tuple[str, str, str], ConnectorSanctionsAcquisitionRun] = {}
for run in runs:
key = (run.tenant_id, run.provider_id, run.source_id)
latest_by_binding.setdefault(key, run)
if len(latest_by_binding) >= context.max_items + 1:
break
snapshot_counts = _snapshot_counts(
session,
binding_keys=tuple(latest_by_binding),
)
observed_at = datetime.now(UTC)
return tuple(
_sanctions_state(
run,
observed_at=observed_at,
snapshot_count=snapshot_counts.get(key, 0),
)
for key, run in latest_by_binding.items()
)
def _session(context: ExternalProviderStateContext) -> Session:
if not isinstance(context.session, Session):
raise RuntimeError("Connectors provider state requires a database session.")
return context.session
def _tabular_state(
source: ConnectorTabularSource,
*,
observed_at: datetime,
) -> ExternalProviderRuntimeState:
active = source.status == "active"
return ExternalProviderRuntimeState(
provider_id=TABULAR_PROVIDER_ID,
binding_ref=f"connectors:tabular-source:{source.id}",
authority_mode="external_mirror",
observed_at=observed_at,
configured=True,
active=active,
health="healthy" if active else "inactive",
freshness="not_applicable",
conflict="not_applicable",
recovery="ready" if active else "not_applicable",
last_success_at=_aware(source.updated_at or source.created_at),
detail=(
"Immutable tabular snapshot is available."
if active
else "Immutable tabular snapshot is inactive."
),
metrics={
"row_count": int(source.row_count),
"byte_count": int(source.byte_count),
"schema_version": int(source.schema_version),
},
)
def _snapshot_counts(
session: Session,
*,
binding_keys: tuple[tuple[str, str, str], ...],
) -> dict[tuple[str, str, str], int]:
if not binding_keys:
return {}
tenant_ids = {item[0] for item in binding_keys}
rows = session.execute(
select(
ConnectorSanctionsSnapshot.tenant_id,
ConnectorSanctionsSnapshot.provider_id,
ConnectorSanctionsSnapshot.source_id,
func.count(ConnectorSanctionsSnapshot.id),
)
.where(ConnectorSanctionsSnapshot.tenant_id.in_(tenant_ids))
.group_by(
ConnectorSanctionsSnapshot.tenant_id,
ConnectorSanctionsSnapshot.provider_id,
ConnectorSanctionsSnapshot.source_id,
)
)
return {
(str(tenant_id), str(provider_id), str(source_id)): int(count)
for tenant_id, provider_id, source_id, count in rows
if (str(tenant_id), str(provider_id), str(source_id)) in binding_keys
}
def _sanctions_state(
run: ConnectorSanctionsAcquisitionRun,
*,
observed_at: datetime,
snapshot_count: int,
) -> ExternalProviderRuntimeState:
status = str(run.status)
success = status in {"succeeded", "success", "not_modified"}
running = status in {"running", "pending", "retry"}
has_snapshot = bool(run.snapshot_id) or snapshot_count > 0
health = "healthy" if success else "warning" if running else "error"
binding_digest = sha256(
f"{run.tenant_id}\0{run.provider_id}\0{run.source_id}".encode("utf-8")
).hexdigest()[:24]
return ExternalProviderRuntimeState(
provider_id=SANCTIONS_PROVIDER_ID,
binding_ref=f"connectors:sanctions-source:{binding_digest}",
authority_mode="external_mirror",
observed_at=observed_at,
configured=True,
active=True,
health=health,
freshness="unknown",
conflict="not_applicable",
recovery="ready" if success and has_snapshot else "attention",
last_success_at=_aware(run.finished_at) if success else None,
detail=(
"Latest sanctions acquisition completed."
if success
else "Sanctions acquisition is in progress."
if running
else "Latest sanctions acquisition failed; prior accepted snapshots remain separate evidence."
),
metrics={
"latest_status": status,
"attempt_count": int(run.attempt_count),
"accepted_snapshots": int(snapshot_count),
},
)
def _aware(value: datetime | None) -> datetime | None:
if value is None:
return None
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
__all__ = [
"SANCTIONS_PROVIDER_ID",
"TABULAR_PROVIDER_ID",
"sanctions_provider_states",
"tabular_provider_states",
]
+118
View File
@@ -0,0 +1,118 @@
from __future__ import annotations
from datetime import UTC, datetime
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_connectors.backend.db.models import (
ConnectorSanctionsAcquisitionRun,
ConnectorSanctionsSnapshot,
ConnectorTabularSource,
)
from govoplan_connectors.backend.manifest import manifest
from govoplan_connectors.backend.provider_state import (
SANCTIONS_PROVIDER_ID,
TABULAR_PROVIDER_ID,
sanctions_provider_states,
tabular_provider_states,
)
from govoplan_core.core.provider_governance import ExternalProviderStateContext
from govoplan_core.db.base import Base
class ConnectorsProviderStateTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite+pysqlite:///:memory:", future=True)
Base.metadata.create_all(
self.engine,
tables=(
ConnectorTabularSource.__table__,
ConnectorSanctionsAcquisitionRun.__table__,
ConnectorSanctionsSnapshot.__table__,
),
)
self.session = sessionmaker(bind=self.engine, expire_on_commit=False)()
def tearDown(self) -> None:
self.session.close()
self.engine.dispose()
def test_immutable_tabular_snapshot_reports_ready_state(self) -> None:
self.session.add(
ConnectorTabularSource(
id="tabular-1",
tenant_id="tenant-1",
source_name="monthly",
name="Monthly input",
status="active",
schema_version=1,
schema_=[{"name": "id", "type": "string"}],
rows=[{"id": "1"}],
fingerprint="a" * 64,
row_count=1,
byte_count=10,
)
)
self.session.commit()
state = tabular_provider_states(
ExternalProviderStateContext(session=self.session, tenant_id="tenant-1")
)[0]
self.assertEqual("healthy", state.health)
self.assertEqual("ready", state.recovery)
self.assertEqual("not_applicable", state.freshness)
def test_sanctions_state_hashes_binding_and_manifest_registers_state(self) -> None:
now = datetime.now(UTC)
run = ConnectorSanctionsAcquisitionRun(
id="run-1",
tenant_id="tenant-1",
provider_id="eu",
source_id="secret-source-name",
status="succeeded",
attempt_count=1,
started_at=now,
finished_at=now,
)
snapshot = ConnectorSanctionsSnapshot(
id="snapshot-1",
tenant_id="tenant-1",
provider_id="eu",
publisher="European Union",
jurisdiction="EU",
list_type="sanctions",
source_id="secret-source-name",
source_version="2026-08-01",
acquired_at=now,
source_url="https://source.example.test/list.xml",
content_type="application/xml",
byte_count=8,
sha256="b" * 64,
parser_version="1",
connector_run_id=run.id,
raw_content=b"<list/>",
)
run.snapshot_id = snapshot.id
self.session.add_all((run, snapshot))
self.session.commit()
state = sanctions_provider_states(
ExternalProviderStateContext(session=self.session, tenant_id="tenant-1")
)[0]
self.assertEqual("healthy", state.health)
self.assertEqual("ready", state.recovery)
rendered = str(state.to_dict())
self.assertNotIn("secret-source-name", rendered)
self.assertNotIn("source.example.test", rendered)
self.assertEqual(
{TABULAR_PROVIDER_ID, SANCTIONS_PROVIDER_ID},
{item.provider_id for item in manifest.external_provider_state_providers},
)
if __name__ == "__main__":
unittest.main()