GovOPlaN/govoplan-connectors
2
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: govern connector configurations and simulations

Browse Source
This commit is contained in:
zemion
2026-08-20 12:58:53 +02:00
parent 3964a2e4e7
commit 0f61fe4607
18 changed files with 3309 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+8
View File
@@ -46,6 +46,13 @@ recovery contract requires stable idempotency, provider verification, and
operator reconciliation, but no connector currently claims a production write
or delete path.
The governed connector runtime adds immutable definition revisions,
revision-pinned tenant configurations, protected local overrides, explicit
package-update adoption, bounded dry-runs and simulations, redacted provenance,
idempotency, and configurable ambiguity handling through review, quarantine,
or rejection. Its administration surface is contributed to the shared system
administration workspace. Provider-specific adapters still own live writes.
Development:
```bash
@@ -60,3 +67,4 @@ See:
- [Connector source lifecycle](docs/CONNECTOR_SOURCE_LIFECYCLE.md)
- [OpenProject connector concept](docs/OPENPROJECT_CONNECTOR.md)
- [OpenDesk integration map](docs/OPENDESK_INTEGRATION_MAP.md)
- [Governed connector configuration](docs/GOVERNED_CONNECTOR_CONFIGURATION.md)
docs/GOVERNED_CONNECTOR_CONFIGURATION.md
+39 -1
View File
@@ -25,7 +25,7 @@ logic should remain visible and reviewable.
## Runtime Expectations
Connectors should support:
Connectors supports the generic governed-definition and simulation portion of:
- discovery where possible
- typed configuration through UI-managed controls
@@ -39,6 +39,44 @@ Connectors should support:
Configuration packages may install connector definitions, but local overrides
must be protected from accidental package updates.
## Implemented Runtime Slice
The module now persists tenant-scoped connector definitions as immutable
revisions. A governed definition explicitly validates its provider, protocol,
capabilities, input/output schemas, mapping version and rules, validation,
preview metadata, audit expectations, classification, retention, limits, and
retry policy. Definitions record whether they are locally owned or supplied by
a named package.
Configurations pin a definition revision. They store an endpoint and a secret
reference, never credentials embedded in the URL. Tenant-local override values
are merged into the pinned definition and every overridden leaf is exposed as
a protected path. Installing a later package revision only marks the
configuration as having an update available. Adoption is an explicit,
optimistically locked action that reapplies the protected overrides over the
new package revision.
The generic execution surface supports bounded dry-runs and simulations. Each
run has a caller idempotency key and retains hashes of its inputs and effective
configuration together with definition, configuration, mapping, external
revision, actor, classification, and retention provenance. Samples redact the
definition's protected fields. Ambiguous uniqueness results follow the
configuration's policy and become either:
- `manual_review` with a pending decision;
- `quarantined` until an administrator decides; or
- `rejected` without a review queue entry.
Review decisions require a reason and are audited. The generic runtime stops
at deterministic preview evidence: provider-specific adapters remain
responsible for live external writes and must satisfy the Core connector
recovery contract before claiming write maturity.
The Connector governance administration page follows the shared workspace
archetype. Reload and Save stay in the semantic action bar, dirty navigation is
guarded, package adoption is a separate action, and simulation results and
review decisions remain visibly distinct from configuration editing.
## Relationship To Datasources And Dataflow
Recurring extraction and transformation should start as configuration across
src/govoplan_connectors/backend/db/models.py
+205
View File
@@ -5,6 +5,7 @@ from datetime import datetime
from typing import Any
from sqlalchemy import (
Boolean,
DateTime,
ForeignKey,
Index,
@@ -255,9 +256,213 @@ class ConnectorSanctionsSnapshot(Base, TimestampMixin):
)
class ConnectorDefinition(Base, TimestampMixin):
__tablename__ = "connector_definitions"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"definition_key",
name="uq_connector_definition_tenant_key",
),
Index(
"ix_connector_definitions_tenant_status",
"tenant_id",
"status",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
definition_key: Mapped[str] = mapped_column(String(160), nullable=False)
name: Mapped[str] = mapped_column(String(300), nullable=False)
description: Mapped[str | None] = mapped_column(Text)
status: Mapped[str] = mapped_column(
String(30),
default="active",
nullable=False,
index=True,
)
current_revision: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
source_package: Mapped[str | None] = mapped_column(String(300))
local_definition: Mapped[bool] = mapped_column(
Boolean,
default=False,
nullable=False,
)
class ConnectorDefinitionRevision(Base, TimestampMixin):
__tablename__ = "connector_definition_revisions"
__table_args__ = (
UniqueConstraint(
"definition_id",
"revision",
name="uq_connector_definition_revision",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
definition_id: Mapped[str] = mapped_column(
ForeignKey("connector_definitions.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
revision: Mapped[int] = mapped_column(Integer, nullable=False)
specification: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
definition_hash: Mapped[str] = mapped_column(
String(64),
nullable=False,
index=True,
)
origin: Mapped[str] = mapped_column(String(30), nullable=False)
package_ref: Mapped[str | None] = mapped_column(String(300))
created_by: Mapped[str | None] = mapped_column(String(255), index=True)
class ConnectorConfiguration(Base, TimestampMixin):
__tablename__ = "connector_configurations"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"name",
name="uq_connector_configuration_tenant_name",
),
Index(
"ix_connector_configurations_tenant_status",
"tenant_id",
"status",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
definition_id: Mapped[str] = mapped_column(
ForeignKey("connector_definitions.id", ondelete="RESTRICT"),
nullable=False,
index=True,
)
name: Mapped[str] = mapped_column(String(300), nullable=False)
status: Mapped[str] = mapped_column(
String(30),
default="draft",
nullable=False,
index=True,
)
endpoint_url: Mapped[str | None] = mapped_column(String(1500))
credential_ref: Mapped[str | None] = mapped_column(String(500))
base_definition_revision: Mapped[int] = mapped_column(
Integer,
nullable=False,
)
local_overrides: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
protected_paths: Mapped[list[str]] = mapped_column(
JSON,
default=list,
nullable=False,
)
effective_configuration: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
effective_hash: Mapped[str] = mapped_column(
String(64),
nullable=False,
index=True,
)
resource_revision: Mapped[int] = mapped_column(
Integer,
default=1,
nullable=False,
)
ambiguity_policy: Mapped[str] = mapped_column(
String(30),
default="manual_review",
nullable=False,
)
updated_by: Mapped[str | None] = mapped_column(String(255), index=True)
class ConnectorSimulationRun(Base, TimestampMixin):
__tablename__ = "connector_simulation_runs"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"configuration_id",
"mode",
"idempotency_key",
name="uq_connector_simulation_run_idempotency",
),
Index(
"ix_connector_simulation_runs_review",
"tenant_id",
"review_state",
"created_at",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
configuration_id: Mapped[str] = mapped_column(
ForeignKey("connector_configurations.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
mode: Mapped[str] = mapped_column(String(30), nullable=False, index=True)
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
request_hash: Mapped[str] = mapped_column(String(64), nullable=False)
status: Mapped[str] = mapped_column(String(30), nullable=False, index=True)
review_state: Mapped[str] = mapped_column(
String(30),
default="not_required",
nullable=False,
index=True,
)
definition_revision: Mapped[int] = mapped_column(Integer, nullable=False)
configuration_revision: Mapped[int] = mapped_column(Integer, nullable=False)
configuration_hash: Mapped[str] = mapped_column(String(64), nullable=False)
input_hash: Mapped[str] = mapped_column(String(64), nullable=False)
summary: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
effects: Mapped[list[dict[str, Any]]] = mapped_column(
JSON,
default=list,
nullable=False,
)
diagnostics: Mapped[list[dict[str, Any]]] = mapped_column(
JSON,
default=list,
nullable=False,
)
provenance: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
created_by: Mapped[str | None] = mapped_column(String(255), index=True)
reviewed_by: Mapped[str | None] = mapped_column(String(255), index=True)
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
review_reason: Mapped[str | None] = mapped_column(Text)
__all__ = [
"ConnectorConfiguration",
"ConnectorDefinition",
"ConnectorDefinitionRevision",
"ConnectorSanctionsAcquisitionRun",
"ConnectorSanctionsSnapshot",
"ConnectorSimulationRun",
"ConnectorTabularSource",
"new_uuid",
]
src/govoplan_connectors/backend/governed_runtime.py
+921
View File
@@ -0,0 +1,921 @@
from __future__ import annotations
import copy
from datetime import UTC, datetime
import hashlib
import json
from typing import Any, Mapping, Sequence
from sqlalchemy.orm import Session
from govoplan_core.audit.logging import audit_from_principal
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.connector_runtime import ConnectorContractError, ConnectorEndpoint
from govoplan_connectors.backend.db.models import (
ConnectorConfiguration,
ConnectorDefinition,
ConnectorDefinitionRevision,
ConnectorSimulationRun,
)
from govoplan_connectors.backend.governed_schemas import (
ConnectorConfigurationCreateRequest,
ConnectorConfigurationItem,
ConnectorConfigurationUpdateRequest,
ConnectorDefinitionItem,
ConnectorDefinitionUpsertRequest,
ConnectorReviewRequest,
ConnectorRunItem,
ConnectorRunRequest,
GovernedConnectorSpecification,
)
class GovernedConnectorError(ValueError):
def __init__(self, code: str, message: str) -> None:
super().__init__(message)
self.code = code
def list_definitions(
session: Session,
*,
tenant_id: str,
) -> list[ConnectorDefinitionItem]:
rows = (
session.query(ConnectorDefinition)
.filter(
ConnectorDefinition.tenant_id == tenant_id,
ConnectorDefinition.status == "active",
)
.order_by(ConnectorDefinition.name.asc())
.all()
)
return [
definition_item(
session,
row,
_definition_revision(session, row, row.current_revision),
)
for row in rows
]
def upsert_definition(
session: Session,
principal: ApiPrincipal,
payload: ConnectorDefinitionUpsertRequest,
) -> ConnectorDefinitionItem:
specification = payload.specification.model_dump(mode="json")
definition_hash = _hash(specification)
definition = (
session.query(ConnectorDefinition)
.filter(
ConnectorDefinition.tenant_id == principal.tenant_id,
ConnectorDefinition.definition_key == payload.definition_key,
)
.one_or_none()
)
expected_local = payload.origin == "local"
if definition is None:
definition = ConnectorDefinition(
tenant_id=principal.tenant_id,
definition_key=payload.definition_key,
name=payload.name.strip(),
description=_optional_text(payload.description),
status="active",
current_revision=0,
source_package=payload.package_ref,
local_definition=expected_local,
)
session.add(definition)
session.flush()
elif definition.local_definition != expected_local:
if definition.local_definition:
raise GovernedConnectorError(
"local_definition_protected",
"A package update cannot replace a locally owned connector definition.",
)
raise GovernedConnectorError(
"package_definition_requires_overrides",
"Use configuration overrides instead of converting a package definition into a local definition.",
)
else:
current = _definition_revision(
session,
definition,
definition.current_revision,
)
if current.definition_hash == definition_hash:
return definition_item(session, definition, current)
definition.name = payload.name.strip()
definition.description = _optional_text(payload.description)
definition.source_package = payload.package_ref
definition.current_revision += 1
revision = ConnectorDefinitionRevision(
definition_id=definition.id,
revision=definition.current_revision,
specification=specification,
definition_hash=definition_hash,
origin=payload.origin,
package_ref=payload.package_ref,
created_by=principal.user.id,
)
session.add(revision)
session.flush()
audit_from_principal(
session,
principal,
action="connectors.definition.revision_created",
object_type="connector_definition",
object_id=definition.id,
details={
"definition_key": definition.definition_key,
"revision": revision.revision,
"definition_hash": definition_hash,
"origin": payload.origin,
"package_ref": payload.package_ref,
"provider": payload.specification.provider,
"protocol": payload.specification.protocol,
"capabilities": sorted(payload.specification.capabilities),
},
)
session.commit()
return definition_item(session, definition, revision)
def list_configurations(
session: Session,
*,
tenant_id: str,
) -> list[ConnectorConfigurationItem]:
rows = (
session.query(ConnectorConfiguration)
.filter(ConnectorConfiguration.tenant_id == tenant_id)
.order_by(ConnectorConfiguration.name.asc())
.all()
)
return [configuration_item(session, row) for row in rows]
def create_configuration(
session: Session,
principal: ApiPrincipal,
payload: ConnectorConfigurationCreateRequest,
) -> ConnectorConfigurationItem:
definition = _tenant_definition(
session,
tenant_id=principal.tenant_id,
definition_id=payload.definition_id,
)
_validate_endpoint(payload.endpoint_url, payload.credential_ref)
revision = _definition_revision(session, definition, definition.current_revision)
overrides = copy.deepcopy(payload.local_overrides)
effective = _effective_specification(revision.specification, overrides)
item = ConnectorConfiguration(
tenant_id=principal.tenant_id,
definition_id=definition.id,
name=payload.name.strip(),
status=payload.status,
endpoint_url=_optional_text(payload.endpoint_url),
credential_ref=_optional_text(payload.credential_ref),
base_definition_revision=definition.current_revision,
local_overrides=overrides,
protected_paths=_protected_paths(overrides),
effective_configuration=effective,
effective_hash=_hash(effective),
resource_revision=1,
ambiguity_policy=payload.ambiguity_policy,
updated_by=principal.user.id,
)
session.add(item)
session.flush()
_audit_configuration(
session,
principal,
item,
action="connectors.configuration.created",
)
session.commit()
return configuration_item(session, item)
def update_configuration(
session: Session,
principal: ApiPrincipal,
*,
configuration_id: str,
payload: ConnectorConfigurationUpdateRequest,
) -> ConnectorConfigurationItem:
item = _tenant_configuration(
session,
tenant_id=principal.tenant_id,
configuration_id=configuration_id,
)
if item.resource_revision != payload.expected_revision:
raise GovernedConnectorError(
"configuration_conflict",
"The connector configuration changed; reload it before saving.",
)
supplied = payload.model_fields_set
definition = _tenant_definition(
session,
tenant_id=principal.tenant_id,
definition_id=item.definition_id,
)
if "name" in supplied and payload.name is not None:
item.name = payload.name.strip()
if "endpoint_url" in supplied:
item.endpoint_url = _optional_text(payload.endpoint_url)
if "credential_ref" in supplied:
item.credential_ref = _optional_text(payload.credential_ref)
if "local_overrides" in supplied and payload.local_overrides is not None:
item.local_overrides = copy.deepcopy(payload.local_overrides)
if payload.ambiguity_policy is not None:
item.ambiguity_policy = payload.ambiguity_policy
if payload.status is not None:
item.status = payload.status
if payload.adopt_latest_definition:
item.base_definition_revision = definition.current_revision
_validate_endpoint(item.endpoint_url, item.credential_ref)
base = _definition_revision(
session,
definition,
item.base_definition_revision,
)
effective = _effective_specification(base.specification, item.local_overrides)
item.protected_paths = _protected_paths(item.local_overrides)
item.effective_configuration = effective
item.effective_hash = _hash(effective)
item.resource_revision += 1
item.updated_by = principal.user.id
_audit_configuration(
session,
principal,
item,
action="connectors.configuration.updated",
extra={"adopted_latest_definition": payload.adopt_latest_definition},
)
session.commit()
return configuration_item(session, item)
def execute_run(
session: Session,
principal: ApiPrincipal,
*,
configuration_id: str,
mode: str,
payload: ConnectorRunRequest,
) -> ConnectorRunItem:
if mode not in {"dry_run", "simulation"}:
raise GovernedConnectorError("invalid_mode", "Unsupported connector run mode.")
configuration = _tenant_configuration(
session,
tenant_id=principal.tenant_id,
configuration_id=configuration_id,
)
if configuration.status == "disabled":
raise GovernedConnectorError(
"configuration_disabled",
"Disabled connector configurations cannot be executed.",
)
specification = GovernedConnectorSpecification.model_validate(
configuration.effective_configuration
)
if mode == "dry_run" and not specification.dry_run.supported:
raise GovernedConnectorError(
"dry_run_unsupported",
"This connector definition does not support dry runs.",
)
if mode == "simulation" and not specification.dry_run.simulation_supported:
raise GovernedConnectorError(
"simulation_unsupported",
"This connector definition does not support simulation.",
)
rows = (
list(payload.input_rows)
if payload.input_rows is not None
else list(specification.dry_run.sample_rows)
)
request_payload = {
"mode": mode,
"configuration_id": configuration.id,
"configuration_revision": configuration.resource_revision,
"configuration_hash": configuration.effective_hash,
"external_revision": payload.external_revision,
"input_rows": rows,
}
request_hash = _hash(request_payload)
existing = (
session.query(ConnectorSimulationRun)
.filter(
ConnectorSimulationRun.tenant_id == principal.tenant_id,
ConnectorSimulationRun.configuration_id == configuration.id,
ConnectorSimulationRun.mode == mode,
ConnectorSimulationRun.idempotency_key == payload.idempotency_key,
)
.one_or_none()
)
if existing is not None:
if existing.request_hash != request_hash:
raise GovernedConnectorError(
"idempotency_conflict",
"The idempotency key was already used with different run inputs.",
)
return run_item(existing)
limit = specification.dry_run.max_items
truncated = len(rows) > limit
bounded_rows = rows[:limit]
effects, diagnostics, ambiguous_count = _simulate(
bounded_rows,
specification,
)
if truncated:
diagnostics.append(
{
"severity": "warning",
"code": "connectors.run.truncated",
"message": f"The run was limited to {limit} input rows.",
"stage": "planning",
"retryable": False,
}
)
errors = sum(1 for item in diagnostics if item["severity"] == "error")
if ambiguous_count:
status_value, review_state = {
"manual_review": ("manual_review", "pending"),
"quarantine": ("quarantined", "quarantined"),
"reject": ("rejected", "not_required"),
}[configuration.ambiguity_policy]
elif errors:
status_value, review_state = "invalid", "not_required"
else:
status_value, review_state = "ready", "not_required"
input_hash = _hash(bounded_rows)
summary = {
"total": len(effects),
"creates": sum(1 for item in effects if item["effect"] == "create"),
"updates": 0,
"deletes": 0,
"conflicts": sum(1 for item in effects if item["effect"] == "conflict"),
"unchanged": 0,
"ignored": sum(1 for item in effects if item["effect"] == "ignored"),
"ambiguous": ambiguous_count,
"errors": errors,
"truncated": truncated,
}
now = datetime.now(UTC)
provenance = {
"contract_version": "1.0",
"definition_id": configuration.definition_id,
"definition_revision": configuration.base_definition_revision,
"configuration_id": configuration.id,
"configuration_revision": configuration.resource_revision,
"configuration_hash": configuration.effective_hash,
"mapping_version": specification.mapping.version,
"input_hash": input_hash,
"external_revision": payload.external_revision,
"generated_at": now.isoformat(),
"actor_id": principal.user.id,
"mode": mode,
"provider": specification.provider,
"protocol": specification.protocol,
"privacy_classification": specification.privacy_classification,
"retention_class": specification.retention_class,
}
run = ConnectorSimulationRun(
tenant_id=principal.tenant_id,
configuration_id=configuration.id,
mode=mode,
idempotency_key=payload.idempotency_key,
request_hash=request_hash,
status=status_value,
review_state=review_state,
definition_revision=configuration.base_definition_revision,
configuration_revision=configuration.resource_revision,
configuration_hash=configuration.effective_hash,
input_hash=input_hash,
summary=summary,
effects=effects,
diagnostics=diagnostics,
provenance=provenance,
created_by=principal.user.id,
)
session.add(run)
session.flush()
audit_from_principal(
session,
principal,
action=f"connectors.configuration.{mode}_completed",
object_type="connector_simulation_run",
object_id=run.id,
details={
"configuration_id": configuration.id,
"configuration_revision": configuration.resource_revision,
"configuration_hash": configuration.effective_hash,
"definition_revision": configuration.base_definition_revision,
"input_hash": input_hash,
"status": status_value,
"review_state": review_state,
"summary": summary,
},
)
session.commit()
return run_item(run)
def list_runs(
session: Session,
*,
tenant_id: str,
configuration_id: str | None = None,
review_state: str | None = None,
limit: int = 100,
) -> list[ConnectorRunItem]:
query = session.query(ConnectorSimulationRun).filter(
ConnectorSimulationRun.tenant_id == tenant_id
)
if configuration_id:
query = query.filter(
ConnectorSimulationRun.configuration_id == configuration_id
)
if review_state:
query = query.filter(ConnectorSimulationRun.review_state == review_state)
rows = (
query.order_by(ConnectorSimulationRun.created_at.desc())
.limit(max(1, min(int(limit), 500)))
.all()
)
return [run_item(row) for row in rows]
def review_run(
session: Session,
principal: ApiPrincipal,
*,
run_id: str,
payload: ConnectorReviewRequest,
) -> ConnectorRunItem:
run = (
session.query(ConnectorSimulationRun)
.filter(
ConnectorSimulationRun.id == run_id,
ConnectorSimulationRun.tenant_id == principal.tenant_id,
)
.one_or_none()
)
if run is None:
raise GovernedConnectorError("run_not_found", "Connector run not found.")
if run.review_state not in {"pending", "quarantined"}:
raise GovernedConnectorError(
"run_not_reviewable",
"Only pending or quarantined connector results can be reviewed.",
)
run.review_state = payload.decision
run.status = f"review_{payload.decision}"
run.reviewed_by = principal.user.id
run.reviewed_at = datetime.now(UTC)
run.review_reason = payload.reason.strip()
audit_from_principal(
session,
principal,
action=f"connectors.simulation.{payload.decision}",
object_type="connector_simulation_run",
object_id=run.id,
details={
"configuration_id": run.configuration_id,
"input_hash": run.input_hash,
"configuration_hash": run.configuration_hash,
"decision": payload.decision,
"reason": run.review_reason,
},
)
session.commit()
return run_item(run)
def definition_item(
session: Session,
definition: ConnectorDefinition,
revision: ConnectorDefinitionRevision,
) -> ConnectorDefinitionItem:
del session
return ConnectorDefinitionItem(
id=definition.id,
tenant_id=definition.tenant_id,
definition_key=definition.definition_key,
name=definition.name,
description=definition.description,
status=definition.status,
current_revision=definition.current_revision,
source_package=definition.source_package,
local_definition=definition.local_definition,
revision_id=revision.id,
definition_hash=revision.definition_hash,
origin=revision.origin,
package_ref=revision.package_ref,
specification=GovernedConnectorSpecification.model_validate(
revision.specification
),
created_at=definition.created_at,
updated_at=definition.updated_at,
)
def configuration_item(
session: Session,
item: ConnectorConfiguration,
) -> ConnectorConfigurationItem:
definition = _tenant_definition(
session,
tenant_id=item.tenant_id,
definition_id=item.definition_id,
)
return ConnectorConfigurationItem(
id=item.id,
tenant_id=item.tenant_id,
definition_id=item.definition_id,
definition_key=definition.definition_key,
definition_name=definition.name,
name=item.name,
status=item.status,
endpoint_url=item.endpoint_url,
credential_ref=item.credential_ref,
base_definition_revision=item.base_definition_revision,
latest_definition_revision=definition.current_revision,
update_available=definition.current_revision > item.base_definition_revision,
local_overrides=dict(item.local_overrides or {}),
protected_paths=list(item.protected_paths or []),
effective_configuration=dict(item.effective_configuration or {}),
effective_hash=item.effective_hash,
resource_revision=item.resource_revision,
ambiguity_policy=item.ambiguity_policy,
updated_at=item.updated_at,
)
def run_item(run: ConnectorSimulationRun) -> ConnectorRunItem:
return ConnectorRunItem(
id=run.id,
tenant_id=run.tenant_id,
configuration_id=run.configuration_id,
mode=run.mode, # type: ignore[arg-type]
idempotency_key=run.idempotency_key,
status=run.status,
review_state=run.review_state,
definition_revision=run.definition_revision,
configuration_revision=run.configuration_revision,
configuration_hash=run.configuration_hash,
input_hash=run.input_hash,
summary=dict(run.summary or {}),
effects=list(run.effects or []),
diagnostics=list(run.diagnostics or []),
provenance=dict(run.provenance or {}),
reviewed_by=run.reviewed_by,
reviewed_at=run.reviewed_at,
review_reason=run.review_reason,
created_at=run.created_at,
)
def _simulate(
rows: Sequence[Mapping[str, Any]],
specification: GovernedConnectorSpecification,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]], int]:
mapped_rows: list[dict[str, Any]] = []
row_errors: dict[int, list[dict[str, Any]]] = {}
diagnostics: list[dict[str, Any]] = []
for index, row in enumerate(rows):
mapped: dict[str, Any] = {}
for rule in specification.mapping.rules:
value, found = _path_value(row, rule.source)
if not found:
value = rule.default
if rule.required and value in (None, ""):
diagnostic = _diagnostic(
"error",
"connectors.mapping.required_source_missing",
f"Required source field {rule.source!r} is missing.",
"mapping",
index=index,
field=rule.source,
)
row_errors.setdefault(index, []).append(diagnostic)
diagnostics.append(diagnostic)
_set_path(mapped, rule.target, value)
mapped_rows.append(mapped)
ambiguous_indexes: set[int] = set()
for rule in specification.validation_rules:
if rule.kind == "unique":
seen: dict[str, list[int]] = {}
for index, row in enumerate(mapped_rows):
value, found = _path_value(row, rule.field)
if found and value not in (None, ""):
seen.setdefault(_stable_value(value), []).append(index)
for indexes in seen.values():
if len(indexes) > 1:
ambiguous_indexes.update(indexes)
for index in indexes:
diagnostic = _diagnostic(
rule.severity,
rule.code,
rule.message,
"validation",
index=index,
field=rule.field,
)
row_errors.setdefault(index, []).append(diagnostic)
diagnostics.append(diagnostic)
continue
for index, row in enumerate(mapped_rows):
value, found = _path_value(row, rule.field)
invalid = (
rule.kind == "required" and (not found or value in (None, ""))
) or (
rule.kind == "one_of" and found and value not in rule.values
)
if invalid:
diagnostic = _diagnostic(
rule.severity,
rule.code,
rule.message,
"validation",
index=index,
field=rule.field,
)
row_errors.setdefault(index, []).append(diagnostic)
diagnostics.append(diagnostic)
redacted = set(specification.dry_run.redacted_fields)
effects: list[dict[str, Any]] = []
for index, mapped in enumerate(mapped_rows):
errors = row_errors.get(index, [])
has_error = any(item["severity"] == "error" for item in errors)
effect = "conflict" if has_error or index in ambiguous_indexes else "create"
effects.append(
{
"effect": effect,
"source_object_ref": _source_ref(rows[index], index),
"target_object_ref": None,
"changed_fields": sorted(_leaf_paths(mapped)),
"sample": _redact_fields(mapped, redacted),
"reason_code": (
"ambiguous_external_result"
if index in ambiguous_indexes
else errors[0]["code"] if errors else None
),
"outcome": "preview",
"revision": specification.mapping.version,
}
)
return effects, diagnostics, len(ambiguous_indexes)
def _audit_configuration(
session: Session,
principal: ApiPrincipal,
item: ConnectorConfiguration,
*,
action: str,
extra: Mapping[str, Any] | None = None,
) -> None:
audit_from_principal(
session,
principal,
action=action,
object_type="connector_configuration",
object_id=item.id,
details={
"definition_id": item.definition_id,
"base_definition_revision": item.base_definition_revision,
"resource_revision": item.resource_revision,
"effective_hash": item.effective_hash,
"protected_paths": list(item.protected_paths or []),
"ambiguity_policy": item.ambiguity_policy,
"status": item.status,
"credential_reference_present": bool(item.credential_ref),
**dict(extra or {}),
},
)
def _tenant_definition(
session: Session,
*,
tenant_id: str,
definition_id: str,
) -> ConnectorDefinition:
item = (
session.query(ConnectorDefinition)
.filter(
ConnectorDefinition.id == definition_id,
ConnectorDefinition.tenant_id == tenant_id,
)
.one_or_none()
)
if item is None:
raise GovernedConnectorError(
"definition_not_found",
"Connector definition not found.",
)
return item
def _tenant_configuration(
session: Session,
*,
tenant_id: str,
configuration_id: str,
) -> ConnectorConfiguration:
item = (
session.query(ConnectorConfiguration)
.filter(
ConnectorConfiguration.id == configuration_id,
ConnectorConfiguration.tenant_id == tenant_id,
)
.one_or_none()
)
if item is None:
raise GovernedConnectorError(
"configuration_not_found",
"Connector configuration not found.",
)
return item
def _definition_revision(
session: Session,
definition: ConnectorDefinition,
revision: int,
) -> ConnectorDefinitionRevision:
item = (
session.query(ConnectorDefinitionRevision)
.filter(
ConnectorDefinitionRevision.definition_id == definition.id,
ConnectorDefinitionRevision.revision == revision,
)
.one_or_none()
)
if item is None:
raise GovernedConnectorError(
"definition_revision_not_found",
"Connector definition revision not found.",
)
return item
def _effective_specification(
base: Mapping[str, Any],
overrides: Mapping[str, Any],
) -> dict[str, Any]:
merged = _deep_merge(base, overrides)
return GovernedConnectorSpecification.model_validate(merged).model_dump(mode="json")
def _deep_merge(
base: Mapping[str, Any],
overrides: Mapping[str, Any],
) -> dict[str, Any]:
result = copy.deepcopy(dict(base))
for key, value in overrides.items():
if isinstance(value, Mapping) and isinstance(result.get(key), Mapping):
result[key] = _deep_merge(result[key], value) # type: ignore[arg-type]
else:
result[key] = copy.deepcopy(value)
return result
def _protected_paths(value: Mapping[str, Any], prefix: str = "") -> list[str]:
paths: list[str] = []
for key in sorted(value):
path = f"{prefix}.{key}" if prefix else str(key)
item = value[key]
if isinstance(item, Mapping) and item:
paths.extend(_protected_paths(item, path))
else:
paths.append(path)
return paths
def _leaf_paths(value: Mapping[str, Any], prefix: str = "") -> set[str]:
paths: set[str] = set()
for key, item in value.items():
path = f"{prefix}.{key}" if prefix else str(key)
if isinstance(item, Mapping) and item:
paths.update(_leaf_paths(item, path))
else:
paths.add(path)
return paths
def _path_value(value: Mapping[str, Any], path: str) -> tuple[Any, bool]:
current: Any = value
for part in path.split("."):
if not isinstance(current, Mapping) or part not in current:
return None, False
current = current[part]
return current, True
def _set_path(target: dict[str, Any], path: str, value: Any) -> None:
parts = path.split(".")
current = target
for part in parts[:-1]:
nested = current.get(part)
if not isinstance(nested, dict):
nested = {}
current[part] = nested
current = nested
current[parts[-1]] = value
def _source_ref(row: Mapping[str, Any], index: int) -> str:
for key in ("id", "external_id", "source_id"):
value = row.get(key)
if value not in (None, ""):
return f"sample:{value}"
return f"sample:row:{index + 1}"
def _redact_fields(value: Mapping[str, Any], fields: set[str]) -> dict[str, Any]:
result = copy.deepcopy(dict(value))
for path in fields:
parts = path.split(".")
current: Any = result
for part in parts[:-1]:
if not isinstance(current, dict):
break
current = current.get(part)
else:
if isinstance(current, dict) and parts[-1] in current:
current[parts[-1]] = "<redacted>"
return result
def _diagnostic(
severity: str,
code: str,
message: str,
stage: str,
**details: Any,
) -> dict[str, Any]:
return {
"severity": severity,
"code": code,
"message": message,
"stage": stage,
"retryable": False,
"details": details,
}
def _validate_endpoint(
endpoint_url: str | None,
credential_ref: str | None,
) -> None:
if endpoint_url:
try:
ConnectorEndpoint(
url=endpoint_url,
credential_ref=_optional_text(credential_ref),
)
except ConnectorContractError as exc:
raise GovernedConnectorError("invalid_endpoint", str(exc)) from exc
elif credential_ref:
raise GovernedConnectorError(
"credential_without_endpoint",
"A credential reference requires a configured endpoint.",
)
def _stable_value(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), default=str)
def _hash(value: Any) -> str:
return hashlib.sha256(
json.dumps(value, sort_keys=True, separators=(",", ":"), default=str).encode()
).hexdigest()
def _optional_text(value: object | None) -> str | None:
normalized = str(value or "").strip()
return normalized or None
__all__ = [
"GovernedConnectorError",
"configuration_item",
"create_configuration",
"execute_run",
"list_configurations",
"list_definitions",
"list_runs",
"review_run",
"run_item",
"update_configuration",
"upsert_definition",
]
src/govoplan_connectors/backend/governed_schemas.py
+221
View File
@@ -0,0 +1,221 @@
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
class ConnectorMappingRule(BaseModel):
model_config = ConfigDict(extra="forbid")
source: str = Field(min_length=1, max_length=255)
target: str = Field(min_length=1, max_length=255)
required: bool = False
default: Any = None
class ConnectorMappingDefinition(BaseModel):
model_config = ConfigDict(extra="forbid")
version: str = Field(min_length=1, max_length=100)
rules: list[ConnectorMappingRule] = Field(default_factory=list, max_length=500)
class ConnectorValidationRule(BaseModel):
model_config = ConfigDict(extra="forbid")
kind: Literal["required", "one_of", "unique"]
field: str = Field(min_length=1, max_length=255)
values: list[Any] = Field(default_factory=list, max_length=500)
severity: Literal["warning", "error"] = "error"
code: str = Field(min_length=1, max_length=120)
message: str = Field(min_length=1, max_length=500)
@model_validator(mode="after")
def validate_values(self) -> "ConnectorValidationRule":
if self.kind == "one_of" and not self.values:
raise ValueError("one_of validation requires allowed values")
if self.kind != "one_of" and self.values:
raise ValueError("Only one_of validation accepts values")
return self
class ConnectorDryRunMetadata(BaseModel):
model_config = ConfigDict(extra="forbid")
supported: bool = True
simulation_supported: bool = True
sample_rows: list[dict[str, Any]] = Field(default_factory=list, max_length=500)
max_items: int = Field(default=500, ge=1, le=10_000)
redacted_fields: list[str] = Field(default_factory=list, max_length=100)
class ConnectorAuditMetadata(BaseModel):
model_config = ConfigDict(extra="forbid")
event_prefix: str = Field(min_length=1, max_length=120)
expected_events: list[str] = Field(default_factory=list, max_length=100)
evidence_fields: list[str] = Field(default_factory=list, max_length=100)
class GovernedConnectorSpecification(BaseModel):
model_config = ConfigDict(extra="forbid")
provider: str = Field(min_length=1, max_length=120)
protocol: str = Field(min_length=1, max_length=80)
capabilities: list[str] = Field(min_length=1, max_length=100)
input_schema: dict[str, Any]
output_schema: dict[str, Any]
mapping: ConnectorMappingDefinition
validation_rules: list[ConnectorValidationRule] = Field(
default_factory=list,
max_length=500,
)
dry_run: ConnectorDryRunMetadata
audit: ConnectorAuditMetadata
privacy_classification: Literal[
"public",
"internal",
"confidential",
"restricted",
] = "internal"
retention_class: str = Field(min_length=1, max_length=120)
operational_limits: dict[str, Any] = Field(default_factory=dict)
retry_policy: dict[str, Any] = Field(default_factory=dict)
class ConnectorDefinitionUpsertRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
definition_key: str = Field(pattern=r"^[a-z0-9][a-z0-9._-]{0,159}$")
name: str = Field(min_length=1, max_length=300)
description: str | None = Field(default=None, max_length=4000)
origin: Literal["package", "local"] = "local"
package_ref: str | None = Field(default=None, max_length=300)
specification: GovernedConnectorSpecification
@model_validator(mode="after")
def package_provenance(self) -> "ConnectorDefinitionUpsertRequest":
if self.origin == "package" and not self.package_ref:
raise ValueError("Package definitions require package_ref")
if self.origin == "local" and self.package_ref:
raise ValueError("Local definitions cannot claim package_ref")
return self
class ConnectorDefinitionItem(BaseModel):
id: str
tenant_id: str
definition_key: str
name: str
description: str | None = None
status: str
current_revision: int
source_package: str | None = None
local_definition: bool
revision_id: str
definition_hash: str
origin: str
package_ref: str | None = None
specification: GovernedConnectorSpecification
created_at: datetime
updated_at: datetime
class ConnectorDefinitionListResponse(BaseModel):
items: list[ConnectorDefinitionItem]
class ConnectorConfigurationCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
definition_id: str = Field(min_length=1, max_length=36)
name: str = Field(min_length=1, max_length=300)
endpoint_url: str | None = Field(default=None, max_length=1500)
credential_ref: str | None = Field(default=None, max_length=500)
local_overrides: dict[str, Any] = Field(default_factory=dict)
ambiguity_policy: Literal["manual_review", "quarantine", "reject"] = (
"manual_review"
)
status: Literal["draft", "active", "disabled"] = "draft"
class ConnectorConfigurationUpdateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
expected_revision: int = Field(ge=1)
name: str | None = Field(default=None, min_length=1, max_length=300)
endpoint_url: str | None = Field(default=None, max_length=1500)
credential_ref: str | None = Field(default=None, max_length=500)
local_overrides: dict[str, Any] | None = None
ambiguity_policy: Literal["manual_review", "quarantine", "reject"] | None = None
status: Literal["draft", "active", "disabled"] | None = None
adopt_latest_definition: bool = False
class ConnectorConfigurationItem(BaseModel):
id: str
tenant_id: str
definition_id: str
definition_key: str
definition_name: str
name: str
status: str
endpoint_url: str | None = None
credential_ref: str | None = None
base_definition_revision: int
latest_definition_revision: int
update_available: bool
local_overrides: dict[str, Any]
protected_paths: list[str]
effective_configuration: dict[str, Any]
effective_hash: str
resource_revision: int
ambiguity_policy: str
updated_at: datetime
class ConnectorConfigurationListResponse(BaseModel):
items: list[ConnectorConfigurationItem]
class ConnectorRunRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
idempotency_key: str = Field(min_length=1, max_length=255)
input_rows: list[dict[str, Any]] | None = Field(default=None, max_length=500)
external_revision: str | None = Field(default=None, max_length=255)
class ConnectorRunItem(BaseModel):
id: str
tenant_id: str
configuration_id: str
mode: Literal["dry_run", "simulation"]
idempotency_key: str
status: str
review_state: str
definition_revision: int
configuration_revision: int
configuration_hash: str
input_hash: str
summary: dict[str, Any]
effects: list[dict[str, Any]]
diagnostics: list[dict[str, Any]]
provenance: dict[str, Any]
reviewed_by: str | None = None
reviewed_at: datetime | None = None
review_reason: str | None = None
created_at: datetime
class ConnectorRunListResponse(BaseModel):
items: list[ConnectorRunItem]
class ConnectorReviewRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
decision: Literal["approved", "rejected"]
reason: str = Field(min_length=5, max_length=1000)
src/govoplan_connectors/backend/manifest.py
+92 -1
View File
@@ -14,11 +14,13 @@ from govoplan_core.core.datasources import CAPABILITY_DATASOURCE_ORIGINS
from govoplan_core.core.feeds import CAPABILITY_CONNECTORS_FEEDS
from govoplan_core.core.modules import (
DocumentationTopic,
FrontendModule,
MigrationSpec,
ModuleInterfaceProvider,
ModuleManifest,
PermissionDefinition,
RoleTemplate,
ViewSurface,
)
from govoplan_core.core.provider_governance import (
ExternalProviderDeclaration,
@@ -38,8 +40,12 @@ from govoplan_core.core.sanctions import (
)
from govoplan_core.db.base import Base
from govoplan_connectors.backend.db.models import (
ConnectorConfiguration,
ConnectorDefinition,
ConnectorDefinitionRevision,
ConnectorSanctionsAcquisitionRun,
ConnectorSanctionsSnapshot,
ConnectorSimulationRun,
ConnectorTabularSource,
)
from govoplan_connectors.backend.sanctions_sources import (
@@ -93,6 +99,11 @@ ARCHITECTURE = ModuleArchitectureDeclaration(
reference="tests/test_recovery.py",
summary="Proves atomic snapshot commits, idempotent replay, distributed fences, tamper rejection, and unknown external-effect handling.",
),
ModuleMaturityEvidence(
kind="test",
reference="tests/test_governed_runtime.py",
summary="Exercises immutable definition revisions, protected local overrides, idempotent simulations, and explicit ambiguity review.",
),
ModuleMaturityEvidence(
kind="documentation",
reference="docs/CONNECTOR_SOURCE_LIFECYCLE.md",
@@ -102,6 +113,7 @@ ARCHITECTURE = ModuleArchitectureDeclaration(
known_limits=(
"The executable generic datasource origin is an immutable tabular snapshot; database and arbitrary REST profiles remain future providers.",
"Feed publication renders a governed document but does not yet push it to an external publishing endpoint.",
"The generic governed runtime simulates deterministic mapping and validation; provider-specific live writes remain owned by explicit connector adapters.",
),
supported_authority_modes=(
"external_authoritative",
@@ -254,7 +266,7 @@ PERMISSIONS = (
_permission(
ADMIN_SCOPE,
"Administer connector sources",
"Manage every tenant connector source and future source policies.",
"Manage tenant connector sources, versioned definitions, protected overrides, and review policies.",
),
_permission(
SANCTIONS_READ_SCOPE,
@@ -269,6 +281,18 @@ PERMISSIONS = (
)
ROLE_TEMPLATES = (
RoleTemplate(
slug="connector_administrator",
name="Connector administrator",
description="Govern connector definitions, local configurations, simulations, and manual review.",
permissions=(
READ_SCOPE,
WRITE_SCOPE,
ADMIN_SCOPE,
SANCTIONS_READ_SCOPE,
SANCTIONS_REFRESH_SCOPE,
),
),
RoleTemplate(
slug="connector_source_manager",
name="Connector source manager",
@@ -315,6 +339,21 @@ def _feed_provider(_context) -> ConnectorFeedProvider:
def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
return {
"connector_definitions": (
session.query(ConnectorDefinition)
.filter(ConnectorDefinition.tenant_id == tenant_id)
.count()
),
"connector_configurations": (
session.query(ConnectorConfiguration)
.filter(ConnectorConfiguration.tenant_id == tenant_id)
.count()
),
"connector_simulation_runs": (
session.query(ConnectorSimulationRun)
.filter(ConnectorSimulationRun.tenant_id == tenant_id)
.count()
),
"connector_tabular_sources": (
session.query(ConnectorTabularSource)
.filter(
@@ -383,6 +422,27 @@ manifest = ModuleManifest(
permissions=PERMISSIONS,
role_templates=ROLE_TEMPLATES,
route_factory=_router,
frontend=FrontendModule(
module_id=MODULE_ID,
package_name="@govoplan/connectors-webui",
view_surfaces=(
ViewSurface(
id="connectors.admin.governed-configurations",
module_id=MODULE_ID,
kind="section",
label="Connector governance",
order=45,
),
ViewSurface(
id="connectors.admin.simulation-review",
module_id=MODULE_ID,
kind="section",
label="Connector simulation review",
parent_id="connectors.admin.governed-configurations",
order=20,
),
),
),
capability_factories={
CAPABILITY_CONNECTORS_TABULAR_SOURCES: _provider,
CAPABILITY_CONNECTORS_TABULAR_SNAPSHOT_WRITER: _provider,
@@ -411,6 +471,10 @@ manifest = ModuleManifest(
script_location=str(Path(__file__).with_name("migrations") / "versions"),
retirement_supported=True,
retirement_provider=drop_table_retirement_provider(
ConnectorSimulationRun,
ConnectorConfiguration,
ConnectorDefinitionRevision,
ConnectorDefinition,
ConnectorSanctionsSnapshot,
ConnectorSanctionsAcquisitionRun,
ConnectorTabularSource,
@@ -423,6 +487,10 @@ manifest = ModuleManifest(
),
uninstall_guard_providers=(
persistent_table_uninstall_guard(
ConnectorSimulationRun,
ConnectorConfiguration,
ConnectorDefinitionRevision,
ConnectorDefinition,
ConnectorSanctionsSnapshot,
ConnectorSanctionsAcquisitionRun,
ConnectorTabularSource,
@@ -430,6 +498,29 @@ manifest = ModuleManifest(
),
),
documentation=(
DocumentationTopic(
id="connectors.governed-configuration",
title="Govern connector definitions and simulations",
summary="Version connector schemas and mappings while preserving tenant-local overrides and review evidence.",
body=(
"Connector administrators create package-managed or local definitions that explicitly declare provider, protocol, capabilities, schemas, mapping rules, validation, preview support, audit expectations, privacy, retention, limits, and retry metadata. Every definition change creates an immutable revision. A tenant configuration pins one revision and stores only a credential reference; package updates remain available but do not change the effective configuration until an administrator adopts them. Local override leaf paths are displayed as protected and are reapplied when an update is adopted. Dry-runs and simulations are bounded, redact configured fields, are idempotent by caller key, and retain configuration, mapping, input, and external revision provenance. Ambiguous results follow the configuration policy: manual review, quarantine, or rejection. Pending and quarantined evidence requires an explicit approve or reject decision with a reason. Provider-specific live writes are not implied by a successful generic simulation."
),
layer="configured",
documentation_types=("admin", "user"),
audience=("operator", "module_admin", "integration_admin"),
related_modules=("policy", "audit", "dataflow", "ops"),
order=38,
metadata={
"kind": "guide",
"help_contexts": ["connectors.admin.governed-configurations"],
"prerequisites": [
"A connector definition has been installed or authored.",
"Credential material is stored outside the connector URL and referenced by an approved secret identifier.",
],
"outcome": "The active connector behavior is inspectable, version-pinned, testable, and reviewable before any provider-specific write.",
"verification": "Reload the configuration, inspect protected paths and effective hash, run a simulation with a new idempotency key, and resolve any pending review result.",
},
),
DocumentationTopic(
id="connectors.authority-and-effects",
title="Connector authority and effect behavior",
src/govoplan_connectors/backend/migrations/versions/a8d9e0f1b2c3_governed_connector_runtime.py
+214
View File
@@ -0,0 +1,214 @@
"""governed connector definitions and simulation evidence
Revision ID: a8d9e0f1b2c3
Revises: f7c8d9e0a1b2
Create Date: 2026-08-20 12:30:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "a8d9e0f1b2c3"
down_revision = "f7c8d9e0a1b2"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"connector_definitions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("definition_key", sa.String(length=160), nullable=False),
sa.Column("name", sa.String(length=300), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("status", sa.String(length=30), nullable=False),
sa.Column("current_revision", sa.Integer(), nullable=False),
sa.Column("source_package", sa.String(length=300), nullable=True),
sa.Column("local_definition", sa.Boolean(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_connector_definitions")),
sa.UniqueConstraint(
"tenant_id",
"definition_key",
name="uq_connector_definition_tenant_key",
),
)
op.create_index(
op.f("ix_connector_definitions_tenant_id"),
"connector_definitions",
["tenant_id"],
)
op.create_index(
op.f("ix_connector_definitions_status"),
"connector_definitions",
["status"],
)
op.create_index(
"ix_connector_definitions_tenant_status",
"connector_definitions",
["tenant_id", "status"],
)
op.create_table(
"connector_definition_revisions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("definition_id", sa.String(length=36), nullable=False),
sa.Column("revision", sa.Integer(), nullable=False),
sa.Column("specification", sa.JSON(), nullable=False),
sa.Column("definition_hash", sa.String(length=64), nullable=False),
sa.Column("origin", sa.String(length=30), nullable=False),
sa.Column("package_ref", sa.String(length=300), nullable=True),
sa.Column("created_by", sa.String(length=255), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["definition_id"],
["connector_definitions.id"],
name=op.f(
"fk_connector_definition_revisions_definition_id_connector_definitions"
),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_connector_definition_revisions"),
),
sa.UniqueConstraint(
"definition_id",
"revision",
name="uq_connector_definition_revision",
),
)
op.create_index(
op.f("ix_connector_definition_revisions_definition_id"),
"connector_definition_revisions",
["definition_id"],
)
op.create_index(
op.f("ix_connector_definition_revisions_definition_hash"),
"connector_definition_revisions",
["definition_hash"],
)
op.create_index(
op.f("ix_connector_definition_revisions_created_by"),
"connector_definition_revisions",
["created_by"],
)
op.create_table(
"connector_configurations",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("definition_id", sa.String(length=36), nullable=False),
sa.Column("name", sa.String(length=300), nullable=False),
sa.Column("status", sa.String(length=30), nullable=False),
sa.Column("endpoint_url", sa.String(length=1500), nullable=True),
sa.Column("credential_ref", sa.String(length=500), nullable=True),
sa.Column("base_definition_revision", sa.Integer(), nullable=False),
sa.Column("local_overrides", sa.JSON(), nullable=False),
sa.Column("protected_paths", sa.JSON(), nullable=False),
sa.Column("effective_configuration", sa.JSON(), nullable=False),
sa.Column("effective_hash", sa.String(length=64), nullable=False),
sa.Column("resource_revision", sa.Integer(), nullable=False),
sa.Column("ambiguity_policy", sa.String(length=30), nullable=False),
sa.Column("updated_by", sa.String(length=255), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["definition_id"],
["connector_definitions.id"],
name=op.f(
"fk_connector_configurations_definition_id_connector_definitions"
),
ondelete="RESTRICT",
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_connector_configurations")),
sa.UniqueConstraint(
"tenant_id",
"name",
name="uq_connector_configuration_tenant_name",
),
)
for name, columns in (
("ix_connector_configurations_tenant_id", ["tenant_id"]),
("ix_connector_configurations_definition_id", ["definition_id"]),
("ix_connector_configurations_status", ["status"]),
("ix_connector_configurations_effective_hash", ["effective_hash"]),
("ix_connector_configurations_updated_by", ["updated_by"]),
):
op.create_index(op.f(name), "connector_configurations", columns)
op.create_index(
"ix_connector_configurations_tenant_status",
"connector_configurations",
["tenant_id", "status"],
)
op.create_table(
"connector_simulation_runs",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("configuration_id", sa.String(length=36), nullable=False),
sa.Column("mode", sa.String(length=30), nullable=False),
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
sa.Column("request_hash", sa.String(length=64), nullable=False),
sa.Column("status", sa.String(length=30), nullable=False),
sa.Column("review_state", sa.String(length=30), nullable=False),
sa.Column("definition_revision", sa.Integer(), nullable=False),
sa.Column("configuration_revision", sa.Integer(), nullable=False),
sa.Column("configuration_hash", sa.String(length=64), nullable=False),
sa.Column("input_hash", sa.String(length=64), nullable=False),
sa.Column("summary", sa.JSON(), nullable=False),
sa.Column("effects", sa.JSON(), nullable=False),
sa.Column("diagnostics", sa.JSON(), nullable=False),
sa.Column("provenance", sa.JSON(), nullable=False),
sa.Column("created_by", sa.String(length=255), nullable=True),
sa.Column("reviewed_by", sa.String(length=255), nullable=True),
sa.Column("reviewed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("review_reason", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["configuration_id"],
["connector_configurations.id"],
name=op.f(
"fk_connector_simulation_runs_configuration_id_connector_configurations"
),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_connector_simulation_runs"),
),
sa.UniqueConstraint(
"tenant_id",
"configuration_id",
"mode",
"idempotency_key",
name="uq_connector_simulation_run_idempotency",
),
)
for name, columns in (
("ix_connector_simulation_runs_tenant_id", ["tenant_id"]),
("ix_connector_simulation_runs_configuration_id", ["configuration_id"]),
("ix_connector_simulation_runs_mode", ["mode"]),
("ix_connector_simulation_runs_status", ["status"]),
("ix_connector_simulation_runs_review_state", ["review_state"]),
("ix_connector_simulation_runs_created_by", ["created_by"]),
("ix_connector_simulation_runs_reviewed_by", ["reviewed_by"]),
):
op.create_index(op.f(name), "connector_simulation_runs", columns)
op.create_index(
"ix_connector_simulation_runs_review",
"connector_simulation_runs",
["tenant_id", "review_state", "created_at"],
)
def downgrade() -> None:
op.drop_table("connector_simulation_runs")
op.drop_table("connector_configurations")
op.drop_table("connector_definition_revisions")
op.drop_table("connector_definitions")
src/govoplan_connectors/backend/router.py
+237
View File
@@ -48,6 +48,30 @@ from govoplan_connectors.backend.schemas import (
TabularSourceResponse,
)
from govoplan_connectors.backend.feeds import ConnectorFeedProvider, feed_rows
from govoplan_connectors.backend.governed_runtime import (
GovernedConnectorError,
create_configuration,
execute_run,
list_configurations,
list_definitions,
list_runs,
review_run,
update_configuration,
upsert_definition,
)
from govoplan_connectors.backend.governed_schemas import (
ConnectorConfigurationCreateRequest,
ConnectorConfigurationItem,
ConnectorConfigurationListResponse,
ConnectorConfigurationUpdateRequest,
ConnectorDefinitionItem,
ConnectorDefinitionListResponse,
ConnectorDefinitionUpsertRequest,
ConnectorReviewRequest,
ConnectorRunItem,
ConnectorRunListResponse,
ConnectorRunRequest,
)
from govoplan_connectors.backend.recovery import (
ConnectorRecoveryError,
begin_connector_read_snapshot,
@@ -135,6 +159,26 @@ def _recovery_http_error(exc: ConnectorRecoveryError) -> HTTPException:
)
def _governed_http_error(exc: GovernedConnectorError) -> HTTPException:
if exc.code.endswith("_not_found"):
status_code = status.HTTP_404_NOT_FOUND
elif exc.code in {
"configuration_conflict",
"configuration_disabled",
"idempotency_conflict",
"local_definition_protected",
"package_definition_requires_overrides",
"run_not_reviewable",
}:
status_code = status.HTTP_409_CONFLICT
else:
status_code = status.HTTP_422_UNPROCESSABLE_CONTENT
return HTTPException(
status_code=status_code,
detail={"code": exc.code, "message": str(exc)},
)
@router.post("/feeds/preview", response_model=FeedDocumentResponse)
def api_preview_feed(
payload: FeedAcquireRequest,
@@ -609,6 +653,199 @@ def api_list_sanctions_runs(
)
@router.get(
"/governed/definitions",
response_model=ConnectorDefinitionListResponse,
)
def api_list_governed_definitions(
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorDefinitionListResponse:
_require_any_scope(principal, READ_SCOPE, ADMIN_SCOPE)
try:
items = list_definitions(session, tenant_id=principal.tenant_id)
except GovernedConnectorError as exc:
raise _governed_http_error(exc) from exc
return ConnectorDefinitionListResponse(items=items)
@router.post(
"/governed/definitions",
response_model=ConnectorDefinitionItem,
)
def api_upsert_governed_definition(
payload: ConnectorDefinitionUpsertRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorDefinitionItem:
_require_any_scope(principal, ADMIN_SCOPE)
try:
return upsert_definition(session, principal, payload)
except GovernedConnectorError as exc:
session.rollback()
raise _governed_http_error(exc) from exc
@router.get(
"/governed/configurations",
response_model=ConnectorConfigurationListResponse,
)
def api_list_governed_configurations(
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorConfigurationListResponse:
_require_any_scope(principal, READ_SCOPE, ADMIN_SCOPE)
try:
items = list_configurations(session, tenant_id=principal.tenant_id)
except GovernedConnectorError as exc:
raise _governed_http_error(exc) from exc
return ConnectorConfigurationListResponse(items=items)
@router.post(
"/governed/configurations",
response_model=ConnectorConfigurationItem,
status_code=status.HTTP_201_CREATED,
)
def api_create_governed_configuration(
payload: ConnectorConfigurationCreateRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorConfigurationItem:
_require_any_scope(principal, ADMIN_SCOPE)
try:
return create_configuration(session, principal, payload)
except GovernedConnectorError as exc:
session.rollback()
raise _governed_http_error(exc) from exc
@router.put(
"/governed/configurations/{configuration_id}",
response_model=ConnectorConfigurationItem,
)
def api_update_governed_configuration(
configuration_id: str,
payload: ConnectorConfigurationUpdateRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorConfigurationItem:
_require_any_scope(principal, ADMIN_SCOPE)
try:
return update_configuration(
session,
principal,
configuration_id=configuration_id,
payload=payload,
)
except GovernedConnectorError as exc:
session.rollback()
raise _governed_http_error(exc) from exc
def _api_execute_governed_run(
*,
configuration_id: str,
mode: str,
payload: ConnectorRunRequest,
session: Session,
principal: ApiPrincipal,
) -> ConnectorRunItem:
_require_any_scope(principal, WRITE_SCOPE, ADMIN_SCOPE)
try:
return execute_run(
session,
principal,
configuration_id=configuration_id,
mode=mode,
payload=payload,
)
except GovernedConnectorError as exc:
session.rollback()
raise _governed_http_error(exc) from exc
@router.post(
"/governed/configurations/{configuration_id}/dry-runs",
response_model=ConnectorRunItem,
)
def api_dry_run_governed_configuration(
configuration_id: str,
payload: ConnectorRunRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorRunItem:
return _api_execute_governed_run(
configuration_id=configuration_id,
mode="dry_run",
payload=payload,
session=session,
principal=principal,
)
@router.post(
"/governed/configurations/{configuration_id}/simulations",
response_model=ConnectorRunItem,
)
def api_simulate_governed_configuration(
configuration_id: str,
payload: ConnectorRunRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorRunItem:
return _api_execute_governed_run(
configuration_id=configuration_id,
mode="simulation",
payload=payload,
session=session,
principal=principal,
)
@router.get(
"/governed/runs",
response_model=ConnectorRunListResponse,
)
def api_list_governed_runs(
configuration_id: str | None = Query(default=None),
review_state: str | None = Query(default=None),
limit: int = Query(default=100, ge=1, le=500),
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorRunListResponse:
_require_any_scope(principal, READ_SCOPE, ADMIN_SCOPE)
try:
items = list_runs(
session,
tenant_id=principal.tenant_id,
configuration_id=configuration_id,
review_state=review_state,
limit=limit,
)
except GovernedConnectorError as exc:
raise _governed_http_error(exc) from exc
return ConnectorRunListResponse(items=items)
@router.post(
"/governed/runs/{run_id}/review",
response_model=ConnectorRunItem,
)
def api_review_governed_run(
run_id: str,
payload: ConnectorReviewRequest,
session: Session = Depends(get_session),
principal: ApiPrincipal = Depends(get_api_principal),
) -> ConnectorRunItem:
_require_any_scope(principal, ADMIN_SCOPE)
try:
return review_run(session, principal, run_id=run_id, payload=payload)
except GovernedConnectorError as exc:
session.rollback()
raise _governed_http_error(exc) from exc
def _source_response(source: TabularSource) -> TabularSourceResponse:
return TabularSourceResponse(
ref=source.ref,
tests/test_governed_runtime.py
+339
View File
@@ -0,0 +1,339 @@
from __future__ import annotations
from types import SimpleNamespace
import unittest
from unittest.mock import patch
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.db.base import Base
from govoplan_connectors.backend.db.models import (
ConnectorConfiguration,
ConnectorDefinition,
ConnectorDefinitionRevision,
ConnectorSimulationRun,
)
from govoplan_connectors.backend.governed_runtime import (
GovernedConnectorError,
create_configuration,
execute_run,
list_configurations,
review_run,
update_configuration,
upsert_definition,
)
from govoplan_connectors.backend.governed_schemas import (
ConnectorConfigurationCreateRequest,
ConnectorConfigurationUpdateRequest,
ConnectorDefinitionUpsertRequest,
ConnectorReviewRequest,
ConnectorRunRequest,
)
def principal(tenant_id: str = "tenant-1") -> ApiPrincipal:
return ApiPrincipal(
principal=PrincipalRef(
account_id="account-1",
membership_id="membership-1",
tenant_id=tenant_id,
scopes=frozenset(
{
"connectors:source:read",
"connectors:source:write",
"connectors:source:admin",
}
),
),
account=SimpleNamespace(id="account-1"),
user=SimpleNamespace(id="user-1"),
)
def definition_payload(
*,
mapping_version: str = "1",
package_ref: str = "municipal-addresses@1",
timeout_seconds: int = 10,
) -> ConnectorDefinitionUpsertRequest:
return ConnectorDefinitionUpsertRequest.model_validate(
{
"definition_key": "municipal.addresses",
"name": "Municipal addresses",
"description": "A package-managed reference connector.",
"origin": "package",
"package_ref": package_ref,
"specification": {
"provider": "municipal-directory",
"protocol": "rest",
"capabilities": ["discover", "read", "dry_run"],
"input_schema": {"type": "object"},
"output_schema": {"type": "object"},
"mapping": {
"version": mapping_version,
"rules": [
{
"source": "external_id",
"target": "address.external_id",
"required": True,
},
{
"source": "street",
"target": "address.street",
"required": True,
},
],
},
"validation_rules": [
{
"kind": "unique",
"field": "address.external_id",
"severity": "error",
"code": "addresses.external_id.ambiguous",
"message": "The external identifier is not unique.",
}
],
"dry_run": {
"supported": True,
"simulation_supported": True,
"max_items": 50,
"redacted_fields": ["address.street"],
"sample_rows": [
{"external_id": "A-1", "street": "Sample street"}
],
},
"audit": {
"event_prefix": "connectors.municipal_addresses",
"expected_events": ["simulation.completed"],
"evidence_fields": ["input_hash", "configuration_hash"],
},
"privacy_classification": "confidential",
"retention_class": "connector-preview-30d",
"operational_limits": {"timeout_seconds": timeout_seconds},
"retry_policy": {"max_attempts": 2},
},
}
)
class GovernedConnectorRuntimeTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite:///:memory:")
self.tables = [
ConnectorDefinition.__table__,
ConnectorDefinitionRevision.__table__,
ConnectorConfiguration.__table__,
ConnectorSimulationRun.__table__,
]
Base.metadata.create_all(self.engine, tables=self.tables)
self.Session = sessionmaker(bind=self.engine)
self.session = self.Session()
self.audit = patch(
"govoplan_connectors.backend.governed_runtime.audit_from_principal"
)
self.audit_mock = self.audit.start()
def tearDown(self) -> None:
self.audit.stop()
self.session.close()
Base.metadata.drop_all(self.engine, tables=reversed(self.tables))
self.engine.dispose()
def _configuration(
self,
*,
ambiguity_policy: str = "manual_review",
):
definition = upsert_definition(
self.session,
principal(),
definition_payload(),
)
return create_configuration(
self.session,
principal(),
ConnectorConfigurationCreateRequest(
definition_id=definition.id,
name=f"Address import {ambiguity_policy}",
endpoint_url="https://directory.example.invalid/v1",
credential_ref="vault://connectors/address-reader",
local_overrides={"retry_policy": {"max_attempts": 5}},
ambiguity_policy=ambiguity_policy,
status="active",
),
)
def test_package_update_is_explicit_and_preserves_local_overrides(self) -> None:
configuration = self._configuration()
updated_definition = upsert_definition(
self.session,
principal(),
definition_payload(
mapping_version="2",
package_ref="municipal-addresses@2",
timeout_seconds=20,
),
)
unchanged = next(
item
for item in list_configurations(self.session, tenant_id="tenant-1")
if item.id == configuration.id
)
self.assertEqual(2, updated_definition.current_revision)
self.assertTrue(unchanged.update_available)
self.assertEqual("1", unchanged.effective_configuration["mapping"]["version"])
self.assertEqual(5, unchanged.effective_configuration["retry_policy"]["max_attempts"])
self.assertEqual(["retry_policy.max_attempts"], unchanged.protected_paths)
adopted = update_configuration(
self.session,
principal(),
configuration_id=configuration.id,
payload=ConnectorConfigurationUpdateRequest(
expected_revision=configuration.resource_revision,
adopt_latest_definition=True,
),
)
self.assertFalse(adopted.update_available)
self.assertEqual("2", adopted.effective_configuration["mapping"]["version"])
self.assertEqual(
20,
adopted.effective_configuration["operational_limits"]["timeout_seconds"],
)
self.assertEqual(5, adopted.effective_configuration["retry_policy"]["max_attempts"])
self.assertEqual(["retry_policy.max_attempts"], adopted.protected_paths)
def test_ambiguous_simulation_requires_review_and_is_idempotent(self) -> None:
configuration = self._configuration()
payload = ConnectorRunRequest(
idempotency_key="simulation-1",
external_revision="directory-etag-22",
input_rows=[
{"external_id": "duplicate", "street": "First"},
{"external_id": "duplicate", "street": "Second"},
],
)
created = execute_run(
self.session,
principal(),
configuration_id=configuration.id,
mode="simulation",
payload=payload,
)
replayed = execute_run(
self.session,
principal(),
configuration_id=configuration.id,
mode="simulation",
payload=payload,
)
self.assertEqual(created.id, replayed.id)
self.assertEqual("manual_review", created.status)
self.assertEqual("pending", created.review_state)
self.assertEqual(2, created.summary["ambiguous"])
self.assertEqual("<redacted>", created.effects[0]["sample"]["address"]["street"])
self.assertEqual("directory-etag-22", created.provenance["external_revision"])
reviewed = review_run(
self.session,
principal(),
run_id=created.id,
payload=ConnectorReviewRequest(
decision="approved",
reason="The duplicate rows represent an approved upstream alias.",
),
)
self.assertEqual("approved", reviewed.review_state)
self.assertEqual("review_approved", reviewed.status)
with self.assertRaisesRegex(GovernedConnectorError, "different run inputs"):
execute_run(
self.session,
principal(),
configuration_id=configuration.id,
mode="simulation",
payload=ConnectorRunRequest(
idempotency_key="simulation-1",
input_rows=[{"external_id": "other", "street": "Other"}],
),
)
def test_ambiguity_policy_can_quarantine_or_reject(self) -> None:
for policy, expected_status, expected_review in (
("quarantine", "quarantined", "quarantined"),
("reject", "rejected", "not_required"),
):
configuration = self._configuration(ambiguity_policy=policy)
result = execute_run(
self.session,
principal(),
configuration_id=configuration.id,
mode="dry_run",
payload=ConnectorRunRequest(
idempotency_key=f"{policy}-1",
input_rows=[
{"external_id": "same", "street": "First"},
{"external_id": "same", "street": "Second"},
],
),
)
self.assertEqual(expected_status, result.status)
self.assertEqual(expected_review, result.review_state)
def test_endpoint_credentials_and_stale_saves_are_rejected(self) -> None:
definition = upsert_definition(
self.session,
principal(),
definition_payload(),
)
with self.assertRaisesRegex(GovernedConnectorError, "must not contain credentials"):
create_configuration(
self.session,
principal(),
ConnectorConfigurationCreateRequest(
definition_id=definition.id,
name="Unsafe",
endpoint_url="https://user:secret@example.invalid/v1",
),
)
configuration = self._configuration()
with self.assertRaisesRegex(GovernedConnectorError, "reload it before saving"):
update_configuration(
self.session,
principal(),
configuration_id=configuration.id,
payload=ConnectorConfigurationUpdateRequest(
expected_revision=configuration.resource_revision + 1,
name="Stale",
),
)
def test_definition_ownership_cannot_change_implicitly(self) -> None:
package_definition = upsert_definition(
self.session,
principal(),
definition_payload(),
)
local_payload = definition_payload().model_copy(
update={"origin": "local", "package_ref": None}
)
with self.assertRaisesRegex(
GovernedConnectorError,
"configuration overrides",
):
upsert_definition(self.session, principal(), local_payload)
self.assertFalse(package_definition.local_definition)
if __name__ == "__main__":
unittest.main()
tests/test_manifest.py
+8
View File
@@ -33,6 +33,14 @@ class ConnectorsManifestTests(unittest.TestCase):
manifest.capability_factories,
)
self.assertIsNotNone(manifest.migration_spec)
self.assertEqual(
"@govoplan/connectors-webui",
manifest.frontend.package_name,
)
self.assertIn(
"connectors.governed-configuration",
{topic.id for topic in manifest.documentation},
)
if __name__ == "__main__":
tests/test_migrations.py
+8 -9
View File
@@ -24,20 +24,19 @@ class ConnectorsMigrationTests(unittest.TestCase):
try:
with engine.connect() as connection:
self.assertIn(
"f7c8d9e0a1b2",
"a8d9e0f1b2c3",
set(MigrationContext.configure(connection).get_current_heads()),
)
self.assertIn(
self.assertTrue(
{
"connector_tabular_sources",
inspect(connection).get_table_names(),
)
self.assertIn(
"connector_sanctions_snapshots",
inspect(connection).get_table_names(),
)
self.assertIn(
"connector_sanctions_acquisition_runs",
inspect(connection).get_table_names(),
"connector_definitions",
"connector_definition_revisions",
"connector_configurations",
"connector_simulation_runs",
}.issubset(inspect(connection).get_table_names())
)
finally:
engine.dispose()
webui/package.json
+29
View File
@@ -0,0 +1,29 @@
{
"name": "@govoplan/connectors-webui",
"version": "0.1.18",
"private": true,
"type": "module",
"main": "src/index.ts",
"module": "src/index.ts",
"types": "src/index.ts",
"exports": {
".": {
"types": "./src/index.ts",
"import": "./src/index.ts"
},
"./styles/connectors.css": "./src/styles/connectors.css"
},
"peerDependencies": {
"@govoplan/core-webui": "^0.1.18",
"react": ">=19.2.7 <20",
"react-dom": ">=19.2.7 <20"
},
"peerDependenciesMeta": {
"@govoplan/core-webui": {
"optional": true
}
},
"scripts": {
"test:connector-governance-ui": "node tests/connector-governance-ui-structure.test.mjs"
}
}
webui/src/api/governedConnectors.ts
+204
View File
@@ -0,0 +1,204 @@
import { apiFetch, apiPath, type ApiSettings } from "@govoplan/core-webui";
export type GovernedConnectorSpecification = {
provider: string;
protocol: string;
capabilities: string[];
input_schema: Record<string, unknown>;
output_schema: Record<string, unknown>;
mapping: {
version: string;
rules: Array<{
source: string;
target: string;
required?: boolean;
default?: unknown;
}>;
};
validation_rules: Array<{
kind: "required" | "one_of" | "unique";
field: string;
values?: unknown[];
severity: "warning" | "error";
code: string;
message: string;
}>;
dry_run: {
supported: boolean;
simulation_supported: boolean;
sample_rows: Array<Record<string, unknown>>;
max_items: number;
redacted_fields: string[];
};
audit: {
event_prefix: string;
expected_events: string[];
evidence_fields: string[];
};
privacy_classification: "public" | "internal" | "confidential" | "restricted";
retention_class: string;
operational_limits: Record<string, unknown>;
retry_policy: Record<string, unknown>;
};
export type ConnectorDefinition = {
id: string;
definition_key: string;
name: string;
description?: string | null;
status: string;
current_revision: number;
source_package?: string | null;
local_definition: boolean;
definition_hash: string;
specification: GovernedConnectorSpecification;
updated_at: string;
};
export type ConnectorConfiguration = {
id: string;
definition_id: string;
definition_key: string;
definition_name: string;
name: string;
status: "draft" | "active" | "disabled";
endpoint_url?: string | null;
credential_ref?: string | null;
base_definition_revision: number;
latest_definition_revision: number;
update_available: boolean;
local_overrides: Record<string, unknown>;
protected_paths: string[];
effective_configuration: GovernedConnectorSpecification;
effective_hash: string;
resource_revision: number;
ambiguity_policy: "manual_review" | "quarantine" | "reject";
updated_at: string;
};
export type ConnectorRun = {
id: string;
configuration_id: string;
mode: "dry_run" | "simulation";
idempotency_key: string;
status: string;
review_state: string;
definition_revision: number;
configuration_revision: number;
configuration_hash: string;
input_hash: string;
summary: Record<string, number | boolean>;
effects: Array<Record<string, unknown>>;
diagnostics: Array<Record<string, unknown>>;
provenance: Record<string, unknown>;
reviewed_by?: string | null;
reviewed_at?: string | null;
review_reason?: string | null;
created_at: string;
};
export type ConnectorConfigurationDraft = {
name: string;
status: ConnectorConfiguration["status"];
endpoint_url: string;
credential_ref: string;
local_overrides: string;
ambiguity_policy: ConnectorConfiguration["ambiguity_policy"];
};
const ROOT = "/api/v1/connectors/governed";
export async function listConnectorDefinitions(
settings: ApiSettings
): Promise<ConnectorDefinition[]> {
const response = await apiFetch<{ items: ConnectorDefinition[] }>(
settings,
`${ROOT}/definitions`
);
return response.items;
}
export async function upsertConnectorDefinition(
settings: ApiSettings,
payload: Record<string, unknown>
): Promise<ConnectorDefinition> {
return apiFetch(settings, `${ROOT}/definitions`, {
method: "POST",
body: JSON.stringify(payload)
});
}
export async function listConnectorConfigurations(
settings: ApiSettings
): Promise<ConnectorConfiguration[]> {
const response = await apiFetch<{ items: ConnectorConfiguration[] }>(
settings,
`${ROOT}/configurations`
);
return response.items;
}
export function createConnectorConfiguration(
settings: ApiSettings,
payload: Record<string, unknown>
): Promise<ConnectorConfiguration> {
return apiFetch(settings, `${ROOT}/configurations`, {
method: "POST",
body: JSON.stringify(payload)
});
}
export function updateConnectorConfiguration(
settings: ApiSettings,
configurationId: string,
payload: Record<string, unknown>
): Promise<ConnectorConfiguration> {
return apiFetch(
settings,
`${ROOT}/configurations/${encodeURIComponent(configurationId)}`,
{ method: "PUT", body: JSON.stringify(payload) }
);
}
export async function listConnectorRuns(
settings: ApiSettings,
configurationId?: string
): Promise<ConnectorRun[]> {
const response = await apiFetch<{ items: ConnectorRun[] }>(
settings,
apiPath(`${ROOT}/runs`, {
configuration_id: configurationId || undefined,
limit: 100
})
);
return response.items;
}
export function executeConnectorRun(
settings: ApiSettings,
configurationId: string,
mode: "dry-runs" | "simulations",
payload: Record<string, unknown>
): Promise<ConnectorRun> {
return apiFetch(
settings,
`${ROOT}/configurations/${encodeURIComponent(configurationId)}/${mode}`,
{ method: "POST", body: JSON.stringify(payload) }
);
}
export function reviewConnectorRun(
settings: ApiSettings,
runId: string,
decision: "approved" | "rejected",
reason: string
): Promise<ConnectorRun> {
return apiFetch(
settings,
`${ROOT}/runs/${encodeURIComponent(runId)}/review`,
{
method: "POST",
body: JSON.stringify({ decision, reason })
}
);
}
webui/src/features/ConnectorGovernancePage.tsx
+661
View File
@@ -0,0 +1,661 @@
import { useCallback, useEffect, useMemo, useState } from "react";
import {
AdminPageLayout,
Button,
Card,
DataGrid,
Dialog,
FilterBar,
FormField,
FormGrid,
MetricCard,
MetricGrid,
PageActionBar,
SelectionList,
SelectionListItem,
SelectionListItemContent,
StatePanel,
StatusBadge,
TableActionGroup,
WorkspaceLayout,
formatDateTime,
hasScope,
useUnsavedChanges,
useUnsavedDraftGuard,
type ApiSettings,
type AuthInfo,
type DataGridColumn
} from "@govoplan/core-webui";
import {
createConnectorConfiguration,
executeConnectorRun,
listConnectorConfigurations,
listConnectorDefinitions,
listConnectorRuns,
reviewConnectorRun,
updateConnectorConfiguration,
upsertConnectorDefinition,
type ConnectorConfiguration,
type ConnectorConfigurationDraft,
type ConnectorDefinition,
type ConnectorRun
} from "../api/governedConnectors";
type Props = {
settings: ApiSettings;
auth: AuthInfo;
};
const EMPTY_DRAFT: ConnectorConfigurationDraft = {
name: "",
status: "draft",
endpoint_url: "",
credential_ref: "",
local_overrides: "{}",
ambiguity_policy: "manual_review"
};
const EXAMPLE_DEFINITION = JSON.stringify({
definition_key: "example.reference-data",
name: "Example reference data",
description: "Locally governed example connector",
origin: "local",
specification: {
provider: "example-provider",
protocol: "rest",
capabilities: ["discover", "read", "dry_run"],
input_schema: { type: "object" },
output_schema: { type: "object" },
mapping: {
version: "1",
rules: [{ source: "id", target: "record.id", required: true }]
},
validation_rules: [{
kind: "unique",
field: "record.id",
severity: "error",
code: "record.id.ambiguous",
message: "The record identifier is not unique."
}],
dry_run: {
supported: true,
simulation_supported: true,
sample_rows: [{ id: "sample-1" }],
max_items: 500,
redacted_fields: []
},
audit: {
event_prefix: "connectors.example",
expected_events: ["simulation.completed"],
evidence_fields: ["input_hash", "configuration_hash"]
},
privacy_classification: "internal",
retention_class: "connector-preview-30d",
operational_limits: { timeout_seconds: 30 },
retry_policy: { max_attempts: 2 }
}
}, null, 2);
export default function ConnectorGovernancePage({ settings, auth }: Props) {
const [definitions, setDefinitions] = useState<ConnectorDefinition[]>([]);
const [configurations, setConfigurations] = useState<ConnectorConfiguration[]>([]);
const [runs, setRuns] = useState<ConnectorRun[]>([]);
const [selectedId, setSelectedId] = useState("");
const [draft, setDraft] = useState<ConnectorConfigurationDraft>(EMPTY_DRAFT);
const [savedKey, setSavedKey] = useState("");
const [search, setSearch] = useState("");
const [loading, setLoading] = useState(true);
const [busy, setBusy] = useState(false);
const [error, setError] = useState("");
const [success, setSuccess] = useState("");
const [definitionOpen, setDefinitionOpen] = useState(false);
const [definitionJson, setDefinitionJson] = useState(EXAMPLE_DEFINITION);
const [configurationOpen, setConfigurationOpen] = useState(false);
const [newDefinitionId, setNewDefinitionId] = useState("");
const [newDraft, setNewDraft] = useState<ConnectorConfigurationDraft>(EMPTY_DRAFT);
const [sampleJson, setSampleJson] = useState("[]");
const [externalRevision, setExternalRevision] = useState("");
const [reviewRun, setReviewRun] = useState<ConnectorRun | null>(null);
const [reviewReason, setReviewReason] = useState("");
const { requestDiscard } = useUnsavedChanges();
const selected = configurations.find((item) => item.id === selectedId) ?? null;
const canAdmin = hasScope(auth, "connectors:source:admin");
const canExecute = canAdmin || hasScope(auth, "connectors:source:write");
const dirty = Boolean(selected && draftKey(draft) !== savedKey);
const applyConfiguration = useCallback((item: ConnectorConfiguration | null) => {
const next = item ? draftFromConfiguration(item) : EMPTY_DRAFT;
setDraft(next);
setSavedKey(item ? draftKey(next) : "");
setSampleJson(JSON.stringify(
item?.effective_configuration.dry_run.sample_rows ?? [],
null,
2
));
}, []);
const reload = useCallback(async (preferredId?: string) => {
setLoading(true);
setError("");
try {
const [nextDefinitions, nextConfigurations] = await Promise.all([
listConnectorDefinitions(settings),
listConnectorConfigurations(settings)
]);
const nextId = preferredId && nextConfigurations.some((item) => item.id === preferredId)
? preferredId
: nextConfigurations.some((item) => item.id === selectedId)
? selectedId
: nextConfigurations[0]?.id ?? "";
const nextRuns = await listConnectorRuns(settings, nextId || undefined);
setDefinitions(nextDefinitions);
setConfigurations(nextConfigurations);
setRuns(nextRuns);
setSelectedId(nextId);
applyConfiguration(nextConfigurations.find((item) => item.id === nextId) ?? null);
if (!newDefinitionId && nextDefinitions[0]) setNewDefinitionId(nextDefinitions[0].id);
} catch (caught) {
setError(errorMessage(caught));
} finally {
setLoading(false);
}
}, [applyConfiguration, newDefinitionId, selectedId, settings]);
useEffect(() => {
void reload();
}, [settings.apiBaseUrl, settings.apiKey, settings.accessToken]);
const visibleConfigurations = useMemo(() => {
const needle = search.trim().toLocaleLowerCase();
return configurations.filter((item) => !needle ||
`${item.name} ${item.definition_name} ${item.status}`
.toLocaleLowerCase()
.includes(needle));
}, [configurations, search]);
const save = async (): Promise<boolean> => {
if (!selected || !canAdmin) return false;
setBusy(true);
setError("");
try {
const overrides = parseObject(draft.local_overrides, "Local overrides");
const updated = await updateConnectorConfiguration(settings, selected.id, {
expected_revision: selected.resource_revision,
name: draft.name.trim(),
status: draft.status,
endpoint_url: draft.endpoint_url.trim() || null,
credential_ref: draft.credential_ref.trim() || null,
local_overrides: overrides,
ambiguity_policy: draft.ambiguity_policy
});
setSuccess("Connector configuration saved.");
await reload(updated.id);
return true;
} catch (caught) {
setError(errorMessage(caught));
return false;
} finally {
setBusy(false);
}
};
useUnsavedDraftGuard({
dirty,
onSave: save,
onDiscard: () => applyConfiguration(selected),
title: "Unsaved connector changes",
message: "Save or discard the current connector changes before continuing."
});
const selectConfiguration = (item: ConnectorConfiguration) => {
if (item.id === selectedId) return;
requestDiscard(() => {
setSelectedId(item.id);
applyConfiguration(item);
setRuns([]);
setError("");
setSuccess("");
void listConnectorRuns(settings, item.id).then(setRuns).catch((caught) => {
setError(errorMessage(caught));
});
});
};
const createDefinition = async () => {
setBusy(true);
setError("");
try {
const payload = parseObject(definitionJson, "Definition");
const created = await upsertConnectorDefinition(settings, payload);
setDefinitionOpen(false);
setSuccess("Connector definition revision saved.");
setNewDefinitionId(created.id);
await reload(selectedId);
} catch (caught) {
setError(errorMessage(caught));
} finally {
setBusy(false);
}
};
const createConfiguration = async () => {
if (!newDefinitionId || !newDraft.name.trim()) return;
setBusy(true);
setError("");
try {
const created = await createConnectorConfiguration(settings, {
definition_id: newDefinitionId,
name: newDraft.name.trim(),
status: newDraft.status,
endpoint_url: newDraft.endpoint_url.trim() || null,
credential_ref: newDraft.credential_ref.trim() || null,
local_overrides: parseObject(newDraft.local_overrides, "Local overrides"),
ambiguity_policy: newDraft.ambiguity_policy
});
setConfigurationOpen(false);
setNewDraft(EMPTY_DRAFT);
setSuccess("Connector configuration created.");
await reload(created.id);
} catch (caught) {
setError(errorMessage(caught));
} finally {
setBusy(false);
}
};
const adoptUpdate = async () => {
if (!selected || dirty || !selected.update_available) return;
setBusy(true);
setError("");
try {
const updated = await updateConnectorConfiguration(settings, selected.id, {
expected_revision: selected.resource_revision,
adopt_latest_definition: true
});
setSuccess("Package revision adopted; protected local overrides were reapplied.");
await reload(updated.id);
} catch (caught) {
setError(errorMessage(caught));
} finally {
setBusy(false);
}
};
const run = async (mode: "dry-runs" | "simulations") => {
if (!selected || dirty) return;
setBusy(true);
setError("");
try {
const inputRows = parseRows(sampleJson);
const created = await executeConnectorRun(settings, selected.id, mode, {
idempotency_key: `${mode}-${crypto.randomUUID()}`,
input_rows: inputRows,
external_revision: externalRevision.trim() || null
});
setSuccess(`${mode === "dry-runs" ? "Dry-run" : "Simulation"} completed with status ${created.status}.`);
setRuns(await listConnectorRuns(settings, selected.id));
} catch (caught) {
setError(errorMessage(caught));
} finally {
setBusy(false);
}
};
const decideReview = async (decision: "approved" | "rejected") => {
if (!reviewRun || reviewReason.trim().length < 5) return;
setBusy(true);
setError("");
try {
await reviewConnectorRun(settings, reviewRun.id, decision, reviewReason.trim());
setReviewRun(null);
setReviewReason("");
setSuccess(`Simulation ${decision}.`);
setRuns(await listConnectorRuns(settings, selectedId));
} catch (caught) {
setError(errorMessage(caught));
} finally {
setBusy(false);
}
};
const runColumns = useMemo<DataGridColumn<ConnectorRun>[]>(() => [
{
id: "created",
header: "Run",
width: 190,
sortable: true,
value: (row) => row.created_at,
render: (row) => <>{row.mode}<br /><span className="muted">{formatDateTime(row.created_at)}</span></>
},
{
id: "status",
header: "Status",
width: 150,
sortable: true,
value: (row) => row.status,
render: (row) => <StatusBadge status={row.status} label={row.status.replaceAll("_", " ")} />
},
{
id: "summary",
header: "Effects",
width: "1fr",
minWidth: 220,
render: (row) => `${row.summary.total ?? 0} total · ${row.summary.ambiguous ?? 0} ambiguous · ${row.summary.errors ?? 0} errors`
},
{
id: "revision",
header: "Evidence",
width: 170,
render: (row) => <code title={row.configuration_hash}>r{row.configuration_revision} · {row.input_hash.slice(0, 8)}</code>
},
{
id: "actions",
header: "Actions",
width: 90,
sticky: "end",
align: "right",
render: (row) => <TableActionGroup actions={[{
id: "review",
label: "Review result",
icon: <span>✓</span>,
applicable: ["pending", "quarantined"].includes(row.review_state),
disabled: !canAdmin || busy,
disabledReason: !canAdmin ? "Connector administration permission is required." : undefined,
onClick: () => setReviewRun(row)
}]} />
}
], [busy, canAdmin]);
const actionBar = <PageActionBar
variant="editor"
state={busy ? "saving" : dirty ? "dirty" : "clean"}
refreshable
reloadAction={{ onReload: () => void reload(selectedId), loading }}
primaryActions={<>
<Button onClick={() => setDefinitionOpen(true)} disabled={!canAdmin || busy}>
New definition
</Button>
<Button onClick={() => setConfigurationOpen(true)} disabled={!canAdmin || busy || !definitions.length}>
New configuration
</Button>
{selected?.update_available ? <Button
variant="secondary"
onClick={() => void adoptUpdate()}
disabled={!canAdmin || busy || dirty}
disabledReason={dirty ? "Save or discard local edits before adopting a package update." : undefined}
>
Adopt package revision {selected.latest_definition_revision}
</Button> : null}
</>}
discardAction={{
label: "Discard changes",
disabled: !selected,
onClick: () => applyConfiguration(selected)
}}
saveAction={{
label: "Save",
disabled: !selected || !canAdmin || busy,
disabledReason: !canAdmin ? "Connector administration permission is required." : undefined,
onClick: () => void save()
}}
/>;
return <AdminPageLayout
archetype="workspace"
title="Connector governance"
description="Version schemas and mappings, protect local overrides, and review deterministic simulations before provider-specific writes."
loading={loading && !configurations.length}
error={error}
success={success}
actions={actionBar}
className="connector-governance-page"
helpContextId="connectors.admin.governed-configurations"
>
<MetricGrid columns={4} density="compact" minimum="compact">
<MetricCard label="Definitions" value={definitions.length} />
<MetricCard label="Configurations" value={configurations.length} />
<MetricCard label="Updates available" value={configurations.filter((item) => item.update_available).length} tone="warning" />
<MetricCard label="Awaiting review" value={runs.filter((item) => ["pending", "quarantined"].includes(item.review_state)).length} tone="warning" />
</MetricGrid>
<WorkspaceLayout
variant="split"
primarySize="compact"
surface="contained"
primaryScrollable={false}
contentScrollable={false}
primaryLabel="Connector configurations"
contentLabel="Configuration details"
primary={<div className="connector-governance-list">
<FilterBar surface="panel">
<input
type="search"
value={search}
onChange={(event) => setSearch(event.target.value)}
placeholder="Search configurations"
aria-label="Search connector configurations"
/>
</FilterBar>
<SelectionList variant="navigation" label="Connector configurations">
{visibleConfigurations.map((item) => <SelectionListItem
key={item.id}
selected={item.id === selectedId}
onClick={() => selectConfiguration(item)}
>
<SelectionListItemContent
title={item.name}
description={`${item.definition_name} · revision ${item.base_definition_revision}`}
/>
<StatusBadge status={item.update_available ? "warning" : item.status} />
</SelectionListItem>)}
{!visibleConfigurations.length
? <StatePanel size="compact" description="No matching configurations." />
: null}
</SelectionList>
</div>}
>
{!selected ? <StatePanel
size="fill"
title="Connector configurations"
description="Create or select a configuration to inspect its pinned definition and simulation evidence."
/> : <div className="connector-governance-detail">
<Card title={selected.name}>
<div className="connector-revision-line">
<StatusBadge status={selected.status} />
<span>Definition revision {selected.base_definition_revision}</span>
{selected.update_available
? <StatusBadge status="warning" label={`Revision ${selected.latest_definition_revision} available`} />
: null}
<code title={selected.effective_hash}>{selected.effective_hash.slice(0, 12)}</code>
</div>
<FormGrid columns={2} collapseAt="standard" className="">
<FormField label="Name">
<input value={draft.name} disabled={!canAdmin || busy} onChange={(event) => setDraft({ ...draft, name: event.target.value })} />
</FormField>
<FormField label="Status">
<select value={draft.status} disabled={!canAdmin || busy} onChange={(event) => setDraft({ ...draft, status: event.target.value as ConnectorConfigurationDraft["status"] })}>
<option value="draft">Draft</option>
<option value="active">Active</option>
<option value="disabled">Disabled</option>
</select>
</FormField>
<FormField label="Endpoint URL" hint="Credentials are rejected in URLs.">
<input value={draft.endpoint_url} disabled={!canAdmin || busy} placeholder="https://provider.example/api" onChange={(event) => setDraft({ ...draft, endpoint_url: event.target.value })} />
</FormField>
<FormField label="Credential reference" hint="Reference an approved secret; do not paste a secret.">
<input value={draft.credential_ref} disabled={!canAdmin || busy} placeholder="vault://connectors/provider" onChange={(event) => setDraft({ ...draft, credential_ref: event.target.value })} />
</FormField>
<FormField label="Ambiguous-result policy">
<select value={draft.ambiguity_policy} disabled={!canAdmin || busy} onChange={(event) => setDraft({ ...draft, ambiguity_policy: event.target.value as ConnectorConfigurationDraft["ambiguity_policy"] })}>
<option value="manual_review">Manual review</option>
<option value="quarantine">Quarantine</option>
<option value="reject">Reject</option>
</select>
</FormField>
<FormField label="Protected local overrides" hint="JSON object leaf paths remain protected when package revisions are adopted.">
<textarea rows={8} value={draft.local_overrides} disabled={!canAdmin || busy} onChange={(event) => setDraft({ ...draft, local_overrides: event.target.value })} />
</FormField>
</FormGrid>
<p className="muted">
Protected paths: {selected.protected_paths.length ? selected.protected_paths.join(", ") : "none"}
</p>
</Card>
<Card title="Dry-run and simulation">
<p className="muted">Runs never apply changes. They retain redacted, revision-pinned evidence for review.</p>
<FormGrid columns={2} collapseAt="standard" className="">
<FormField label="Sample input rows" hint="JSON array, bounded by the definition's maximum.">
<textarea rows={9} value={sampleJson} disabled={!canExecute || busy} onChange={(event) => setSampleJson(event.target.value)} />
</FormField>
<FormField label="External revision" hint="Optional provider ETag, cursor, or snapshot revision.">
<input value={externalRevision} disabled={!canExecute || busy} onChange={(event) => setExternalRevision(event.target.value)} />
</FormField>
</FormGrid>
<div className="connector-run-actions">
<Button onClick={() => void run("dry-runs")} disabled={!canExecute || busy || dirty}>Run dry-run</Button>
<Button variant="primary" onClick={() => void run("simulations")} disabled={!canExecute || busy || dirty}>Run simulation</Button>
{dirty ? <span className="muted">Save or discard configuration changes before running.</span> : null}
</div>
</Card>
<Card title="Simulation evidence">
<DataGrid
id="connector-simulation-runs"
rows={runs}
columns={runColumns}
getRowKey={(row) => row.id}
initialFit="container"
emptyText="No dry-runs or simulations have been recorded."
/>
</Card>
<Card title="Effective governed definition">
<pre className="connector-json-preview">{JSON.stringify(selected.effective_configuration, null, 2)}</pre>
</Card>
</div>}
</WorkspaceLayout>
<Dialog
open={definitionOpen}
title="Create or revise connector definition"
onClose={() => !busy && setDefinitionOpen(false)}
closeDisabled={busy}
footer={<>
<Button onClick={() => setDefinitionOpen(false)} disabled={busy}>Cancel</Button>
<Button variant="primary" onClick={() => void createDefinition()} disabled={!canAdmin || busy}>Save definition revision</Button>
</>}
>
<p className="muted">The definition is schema-validated and every changed specification creates an immutable revision. Package definitions must name their package reference.</p>
<FormField label="Governed definition JSON">
<textarea className="connector-definition-editor" rows={24} value={definitionJson} disabled={busy} onChange={(event) => setDefinitionJson(event.target.value)} />
</FormField>
</Dialog>
<Dialog
open={configurationOpen}
title="Create connector configuration"
onClose={() => !busy && setConfigurationOpen(false)}
closeDisabled={busy}
footer={<>
<Button onClick={() => setConfigurationOpen(false)} disabled={busy}>Cancel</Button>
<Button variant="primary" onClick={() => void createConfiguration()} disabled={busy || !newDefinitionId || !newDraft.name.trim()}>Create configuration</Button>
</>}
>
<FormGrid columns={2} collapseAt="standard" className="">
<FormField label="Definition">
<select value={newDefinitionId} disabled={busy} onChange={(event) => setNewDefinitionId(event.target.value)}>
{definitions.map((item) => <option key={item.id} value={item.id}>{item.name} · revision {item.current_revision}</option>)}
</select>
</FormField>
<FormField label="Name">
<input value={newDraft.name} disabled={busy} onChange={(event) => setNewDraft({ ...newDraft, name: event.target.value })} />
</FormField>
<FormField label="Endpoint URL">
<input value={newDraft.endpoint_url} disabled={busy} onChange={(event) => setNewDraft({ ...newDraft, endpoint_url: event.target.value })} />
</FormField>
<FormField label="Credential reference">
<input value={newDraft.credential_ref} disabled={busy} onChange={(event) => setNewDraft({ ...newDraft, credential_ref: event.target.value })} />
</FormField>
<FormField label="Ambiguous-result policy">
<select value={newDraft.ambiguity_policy} disabled={busy} onChange={(event) => setNewDraft({ ...newDraft, ambiguity_policy: event.target.value as ConnectorConfigurationDraft["ambiguity_policy"] })}>
<option value="manual_review">Manual review</option>
<option value="quarantine">Quarantine</option>
<option value="reject">Reject</option>
</select>
</FormField>
<FormField label="Local overrides">
<textarea rows={7} value={newDraft.local_overrides} disabled={busy} onChange={(event) => setNewDraft({ ...newDraft, local_overrides: event.target.value })} />
</FormField>
</FormGrid>
</Dialog>
<Dialog
open={Boolean(reviewRun)}
title="Review ambiguous connector result"
onClose={() => !busy && setReviewRun(null)}
closeDisabled={busy}
footer={<>
<Button onClick={() => setReviewRun(null)} disabled={busy}>Cancel</Button>
<Button variant="danger" onClick={() => void decideReview("rejected")} disabled={busy || reviewReason.trim().length < 5}>Reject</Button>
<Button variant="primary" onClick={() => void decideReview("approved")} disabled={busy || reviewReason.trim().length < 5}>Approve</Button>
</>}
>
<p>Review {reviewRun?.summary.ambiguous ?? 0} ambiguous effects against the retained input and configuration hashes before deciding.</p>
<FormField label="Decision reason">
<textarea rows={4} value={reviewReason} disabled={busy} onChange={(event) => setReviewReason(event.target.value)} />
</FormField>
<pre className="connector-json-preview">{JSON.stringify(reviewRun?.diagnostics ?? [], null, 2)}</pre>
</Dialog>
</AdminPageLayout>;
}
function draftFromConfiguration(item: ConnectorConfiguration): ConnectorConfigurationDraft {
return {
name: item.name,
status: item.status,
endpoint_url: item.endpoint_url ?? "",
credential_ref: item.credential_ref ?? "",
local_overrides: JSON.stringify(item.local_overrides, null, 2),
ambiguity_policy: item.ambiguity_policy
};
}
function draftKey(value: ConnectorConfigurationDraft): string {
return JSON.stringify({
name: value.name.trim(),
status: value.status,
endpoint_url: value.endpoint_url.trim(),
credential_ref: value.credential_ref.trim(),
local_overrides: normalizeJson(value.local_overrides),
ambiguity_policy: value.ambiguity_policy
});
}
function normalizeJson(value: string): unknown {
try {
return JSON.parse(value);
} catch {
return value.trim();
}
}
function parseObject(value: string, label: string): Record<string, unknown> {
const parsed: unknown = JSON.parse(value);
if (!parsed || Array.isArray(parsed) || typeof parsed !== "object") {
throw new Error(`${label} must be a JSON object.`);
}
return parsed as Record<string, unknown>;
}
function parseRows(value: string): Array<Record<string, unknown>> {
const parsed: unknown = JSON.parse(value);
if (!Array.isArray(parsed) || parsed.some((item) => !item || Array.isArray(item) || typeof item !== "object")) {
throw new Error("Sample input must be a JSON array of objects.");
}
return parsed as Array<Record<string, unknown>>;
}
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
webui/src/index.ts
+1
View File
@@ -0,0 +1 @@
export { connectorsModule as default, connectorsModule } from "./module";
webui/src/module.ts
+62
View File
@@ -0,0 +1,62 @@
import { createElement, lazy } from "react";
import type {
AdminSectionsUiCapability,
PlatformWebModule
} from "@govoplan/core-webui";
import "./styles/connectors.css";
const ConnectorGovernancePage = lazy(
() => import("./features/ConnectorGovernancePage")
);
const readScopes = [
"connectors:source:read",
"connectors:source:admin"
];
const adminSections: AdminSectionsUiCapability = {
sections: [
{
id: "connector-governance",
moduleId: "connectors",
kind: "management",
surfaceId: "connectors.admin.governed-configurations",
label: "Connector governance",
group: "SYSTEM",
order: 45,
anyOf: readScopes,
render: ({ settings, auth }) =>
createElement(ConnectorGovernancePage, { settings, auth })
}
]
};
export const connectorsModule: PlatformWebModule = {
id: "connectors",
label: "Connectors",
version: "0.1.18",
dependencies: [],
optionalDependencies: ["access", "audit", "policy", "ops"],
viewSurfaces: [
{
id: "connectors.admin.governed-configurations",
moduleId: "connectors",
kind: "section",
label: "Connector governance",
order: 45
},
{
id: "connectors.admin.simulation-review",
moduleId: "connectors",
kind: "section",
label: "Connector simulation review",
parentId: "connectors.admin.governed-configurations",
order: 20
}
],
uiCapabilities: {
"admin.sections": adminSections
}
};
export default connectorsModule;
webui/src/styles/connectors.css
+32
View File
@@ -0,0 +1,32 @@
.connector-governance-page .connector-governance-list,
.connector-governance-page .connector-governance-detail {
display: grid;
gap: var(--space-4, 1rem);
}
.connector-governance-page .connector-revision-line,
.connector-governance-page .connector-run-actions {
align-items: center;
display: flex;
flex-wrap: wrap;
gap: var(--space-3, 0.75rem);
margin-bottom: var(--space-4, 1rem);
}
.connector-governance-page textarea {
font-family: var(--font-family-mono, ui-monospace, monospace);
resize: vertical;
}
.connector-governance-page .connector-definition-editor,
.connector-governance-page .connector-json-preview {
max-height: 34rem;
overflow: auto;
white-space: pre-wrap;
}
.connector-governance-page .connector-json-preview {
background: var(--surface-subtle);
border-radius: var(--radius-md, 0.5rem);
padding: var(--space-4, 1rem);
}
webui/tests/connector-governance-ui-structure.test.mjs
+25
View File
@@ -0,0 +1,25 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const moduleSource = readFileSync("src/module.ts", "utf8");
const page = readFileSync("src/features/ConnectorGovernancePage.tsx", "utf8");
const api = readFileSync("src/api/governedConnectors.ts", "utf8");
assert.match(moduleSource, /"admin.sections": adminSections/);
assert.match(moduleSource, /connectors\.admin\.governed-configurations/);
assert.match(page, /<AdminPageLayout/);
assert.match(page, /<PageActionBar/);
assert.match(page, /refreshable/);
assert.match(page, /saveAction=/);
assert.match(page, /useUnsavedDraftGuard/);
assert.match(page, /<WorkspaceLayout/);
assert.match(page, /Adopt package revision/);
assert.match(page, /Protected paths/);
assert.match(page, /manual_review/);
assert.match(page, /quarantine/);
assert.match(page, /Run simulation/);
assert.match(page, /Review ambiguous connector result/);
assert.match(api, /idempotency_key/);
assert.match(api, /credential_ref/);
console.log("Connector governance UI structural contract passed.");