Prove reference journey compositions
Dependency Audit / dependency-audit (push) Successful in 1m45s
Deployment Installer / deployment-installer (push) Successful in 6s
Security Audit / security-audit (push) Successful in 11m16s

This commit is contained in:
2026-08-06 12:42:20 +02:00
parent 8bdf7b5f7e
commit 69ba1037bf
9 changed files with 383 additions and 17 deletions
+7 -5
View File
@@ -42,11 +42,13 @@ does neither stays in the backlog until a concrete consumer exists.
The Records vertical now supplies the journey's native file plan, immutable
record and item revisions, chronology, close/reopen, retention calculation,
holds, appraisal, independent disposition approval, recovery-ledger evidence,
and archive-neutral package simulation. The executable Records fixture proves
those native transitions without claiming archival custody. The journey still
needs Forms Runtime/Decisions filing, resumable user handoffs, search
reconstruction in the pinned composition, and one target-tested archive
profile.
and archive-neutral package simulation. Forms Runtime, Cases, and Decisions
expose exact, permission-rechecked source revisions for explicit filing, and
all three contribute metadata-only native Search projections that can be
rebuilt from authoritative state. The executable fixtures prove those native
transitions without claiming archival custody. The journey still needs
resumable user handoffs, pinned-composition reconstruction evidence, and one
target-tested archive profile.
## Why this sequence
+12 -6
View File
@@ -4,7 +4,7 @@
| Field | Value |
| --- | --- |
| Reconciled on | 2026-08-05 |
| Reconciled on | 2026-08-06 |
| Source scope | Local workspace manifests, source inventory, focused journey checks, signed release evidence, and live Gitea issue state |
| Stable direction | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) and [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) |
| Delivery source | Gitea issues |
@@ -67,8 +67,12 @@ prevent a reference-ready claim.
Services, Forms, Forms Runtime, Cases, Parties, Mandates, Approvals, Committee,
Voting, Decisions, Portal, Postbox, and Audit have an executable service-to-
decision fixture. Browser-complete assisted intake, production identity,
records, delivery, and target evidence remain.
decision fixture. Public and invitation intake can retain Files-backed
evidence; Forms submissions, Cases, and formal Decisions can be explicitly
filed as exact eAkte source revisions and reconstructed through permission-
rechecked native Search projections. Browser-complete assisted intake,
resumable work guidance, production identity and delivery, a named archive
profile, and target evidence remain.
### Governed data and assurance
@@ -97,9 +101,11 @@ reconciliation, publication/export, and guided handoff evidence.
2. Complete governed communication and Postbox against a named target.
3. Complete the monthly-data flow and use it as the data foundation for
sanctions screening.
4. Complete one digital and assisted service-to-decision journey with an eAkte.
5. Add native PostgreSQL search coverage for the objects used by those
journeys; keep OpenSearch optional.
4. Complete the browser and resumable-work proof for the digital and assisted
service-to-decision journey with its existing exact eAkte filing contracts.
5. Extend native PostgreSQL search coverage from the implemented service-to-
decision objects to the communication and monthly-data journeys; keep
OpenSearch optional.
6. Prove one external product connector and one GovOPlaN federation exchange.
7. Finish multi-host, restore, provider, accessibility, and independent signed
target evidence before increasing maturity claims.
@@ -12,7 +12,7 @@ review.
1. Register a typed datasource with source authority, purpose, classification,
owner, freshness, and correction policy.
2. Acquire or upload an immutable source state.
3. execute a versioned flow and retain intermediate materializations and
3. Execute a versioned flow and retain intermediate materializations and
provenance;
4. publish a report or decision input against exact source and flow revisions;
5. link obligation, governed object, risk, control, evidence, finding,
@@ -39,3 +39,24 @@ The artifact remains a `product` package. Promotion to `reference` requires:
Optional Connectors, Files, Notifications, and Workflow Engine integrations
must remain capability-based and absence-safe.
## Executable evidence
- `tools/checks/check-datasource-composition.py` composes Connector snapshots,
governed Datasources, queued Dataflow execution, frozen publication,
idempotent replay, and recovery evidence.
- `govoplan-dataflow/fixtures/golden/monthly-reconciliation` pins synthetic
monthly inputs, stable reconciliation hashes, reviewed decisions, expected
output, source fingerprints, and output hashes.
- `tools/checks/check-sanctions-screening-composition.py` composes an immutable
Connector acquisition, idempotent Risk Compliance import and screening,
independent disposition, a cleared gate, changed-source invalidation, and
the rescreening queue through the registered versioned capabilities.
- `govoplan-dataflow/fixtures/golden/sanctions-screening` independently proves
the deterministic normalization and matching graph with exact expected
output.
These checks use synthetic data and run without network access. They prove the
module contracts and durable state transitions; they do not replace the
deployment, security, privacy, accessibility, and operator evidence still
listed above.
@@ -28,7 +28,17 @@
"kind": "documentation",
"reference": "packages/product/governed-data-assurance/README.md",
"summary": "Defines the package boundary, provenance chain, and reference-readiness gates."
},
{
"kind": "target_test",
"reference": "tools/checks/check-datasource-composition.py",
"summary": "Proves governed Connector acquisition, Datasource registration, queued Dataflow execution, frozen publication, idempotency, and recovery evidence."
},
{
"kind": "target_test",
"reference": "tools/checks/check-sanctions-screening-composition.py",
"summary": "Proves immutable sanctions acquisition, import, screening replay, independent review, freshness gates, and rescreening across module capabilities."
}
],
"tags": ["datasources", "dataflow", "reporting", "assurance"]
"tags": ["datasources", "dataflow", "reporting", "sanctions", "assurance"]
}
+16 -4
View File
@@ -24,19 +24,31 @@ grant cross-module table access and can omit optional presentation, work,
deliberation, delivery, or records modules while retaining explicit references
to externally performed steps.
When Records is present, Forms Runtime, Cases, and Decisions expose exact,
digest-bound source snapshots for explicit filing. The source module rechecks
current access, Records chooses the destination and preserves chronology, and
the filed reference never becomes an editable copy. When Search is present,
the same three owners contribute rebuildable metadata-only projections. Form
values, evidence payloads, Decision reasoning, operative results, and
conditions are excluded; every candidate is authorized again before it is
shown.
## Security And Recovery
Every provider is tenant-bound. Missing or conflicting authority fails closed.
Protected Decision content has a separate permission. Writes are replay-safe
and OCC-guarded. Database restore is the semantic-state recovery unit; file and
communication effects remain governed by their owning providers and are linked
through requested/observed effect, evidence, and audit references.
through requested/observed effect, evidence, and audit references. Search is a
derived recovery unit and can be rebuilt from authoritative module state.
The executable fixture in
`tests/test_institutional_governance_journey.py` proves SQL-backed Service,
Case, Party, Mandate, Committee meeting/agendum/vote/minute, and Decision state.
`tests/test_institutional_service_journey.py` separately proves exact Portal
Form launch, persisted submission provenance, and idempotent replay.
Target-environment accessibility, security, operator, privacy,
delivery-provider, and recovery evidence are still required before this product
package may claim `reference_ready` maturity.
Module-level Records source tests prove exact Form submission, Case revision,
and Decision revision filing. Target-environment browser accessibility,
resumable work guidance, production identity and delivery, a named archive
profile, and recovery evidence are still required before this product package
may claim `reference_ready` maturity.
@@ -35,6 +35,7 @@
{"module_id": "forms_runtime"},
{"module_id": "postbox"},
{"module_id": "records"},
{"module_id": "search"},
{"module_id": "tasks"},
{"module_id": "workflow_engine"}
],
+1
View File
@@ -109,6 +109,7 @@ PY
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
@@ -0,0 +1,306 @@
#!/usr/bin/env python3
"""Prove the governed Connectors -> Risk Compliance sanctions journey."""
from __future__ import annotations
from types import SimpleNamespace
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from govoplan_connectors.backend.db.models import (
ConnectorSanctionsAcquisitionRun,
ConnectorSanctionsSnapshot,
)
from govoplan_connectors.backend.sanctions_sources import (
SANCTIONS_READ_SCOPE as CONNECTOR_SANCTIONS_READ_SCOPE,
SANCTIONS_REFRESH_SCOPE,
SYNTHETIC_PROVIDER_ID,
)
from govoplan_core.auth import ApiPrincipal
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.modules import ModuleContext
from govoplan_core.core.recovery import RecoveryCheckpoint, RecoveryOperation
from govoplan_core.core.runtime_coordination import (
DistributedLease,
RuntimeIdentity,
bind_process_runtime_identity,
)
from govoplan_core.core.sanctions import (
SanctionsScreeningFreshnessRequest,
SanctionsScreeningPolicy,
SanctionsScreeningRequest,
SanctionsScreeningSubject,
sanctions_screening_provider,
sanctions_snapshot_provider,
)
from govoplan_core.db.base import Base
from govoplan_core.server.registry import build_platform_registry
from govoplan_risk_compliance.backend.db.models import (
RiskAssuranceEdge,
RiskAssuranceNode,
RiskSanctionsAddress,
RiskSanctionsAlias,
RiskSanctionsDate,
RiskSanctionsEntry,
RiskSanctionsIdentifier,
RiskSanctionsListSnapshot,
RiskScreeningCandidate,
RiskScreeningDisposition,
RiskScreeningException,
RiskScreeningRun,
RiskScreeningSubjectSnapshot,
)
from govoplan_risk_compliance.backend.permissions import (
SANCTIONS_ADMIN_SCOPE,
SANCTIONS_READ_SCOPE,
SANCTIONS_REVIEW_SCOPE,
SANCTIONS_SCREEN_SCOPE,
)
from govoplan_risk_compliance.backend.review import (
DispositionInput,
record_disposition,
)
from govoplan_risk_compliance.backend.sanctions_catalog import (
import_connector_snapshot,
)
from govoplan_risk_compliance.backend.screening import (
get_screening_run,
list_rescreening_requirements,
)
TABLES = (
DistributedLease.__table__,
RecoveryOperation.__table__,
RecoveryCheckpoint.__table__,
ConnectorSanctionsAcquisitionRun.__table__,
ConnectorSanctionsSnapshot.__table__,
RiskAssuranceNode.__table__,
RiskAssuranceEdge.__table__,
RiskSanctionsListSnapshot.__table__,
RiskSanctionsEntry.__table__,
RiskSanctionsAlias.__table__,
RiskSanctionsIdentifier.__table__,
RiskSanctionsDate.__table__,
RiskSanctionsAddress.__table__,
RiskScreeningSubjectSnapshot.__table__,
RiskScreeningRun.__table__,
RiskScreeningCandidate.__table__,
RiskScreeningDisposition.__table__,
RiskScreeningException.__table__,
)
def main() -> int:
registry = build_platform_registry(("connectors", "risk_compliance"))
registry.configure_capability_context(
ModuleContext(registry=registry, settings=object())
)
snapshot_provider = sanctions_snapshot_provider(registry)
screening_provider = sanctions_screening_provider(registry)
refresh_source = getattr(snapshot_provider, "refresh_source", None)
if snapshot_provider is None or not callable(refresh_source):
raise RuntimeError("Connectors sanctions acquisition is unavailable.")
if screening_provider is None:
raise RuntimeError("Risk Compliance sanctions screening is unavailable.")
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(engine, tables=TABLES)
bind_process_runtime_identity(
RuntimeIdentity(
installation_id="sanctions-composition-check",
node_id="sanctions-worker",
incarnation="sanctions-worker-incarnation",
role="worker",
software_version="test",
composition_hash="d" * 64,
)
)
try:
with Session(engine) as session:
operator = _principal("operator-1", operational=True)
reviewer = _principal("reviewer-1", operational=False)
acquired = refresh_source(
session,
operator,
provider_id=SYNTHETIC_PROVIDER_ID,
idempotency_key="synthetic-sanctions-2026-08-01",
)
replay = refresh_source(
session,
operator,
provider_id=SYNTHETIC_PROVIDER_ID,
idempotency_key="synthetic-sanctions-2026-08-01",
)
if acquired.status != "succeeded" or acquired.snapshot is None:
raise RuntimeError(f"Synthetic acquisition failed: {acquired!r}")
if (
replay.run_id != acquired.run_id
or replay.snapshot is None
or replay.snapshot.ref != acquired.snapshot.ref
or replay.snapshot.sha256 != acquired.snapshot.sha256
):
raise RuntimeError("Acquisition idempotency did not replay exact evidence.")
imported, created = import_connector_snapshot(
session,
operator,
registry=registry,
connector_snapshot_ref=acquired.snapshot.ref,
)
imported_replay, replay_created = import_connector_snapshot(
session,
operator,
registry=registry,
connector_snapshot_ref=acquired.snapshot.ref,
)
if not created or replay_created or imported_replay.id != imported.id:
raise RuntimeError("Risk Compliance snapshot import is not idempotent.")
subject = SanctionsScreeningSubject(
subject_type="person",
primary_name="Alex Example",
subject_ref="party:fixture-person-1",
)
policy = SanctionsScreeningPolicy(failure_policy="block")
request = SanctionsScreeningRequest(
list_snapshot_id=imported.id,
idempotency_key="fixture-party-screening-1",
subject=subject,
policy=policy,
)
screened = screening_provider.request_screening(
session,
operator,
request,
)
screened_replay = screening_provider.request_screening(
session,
operator,
request,
)
if not screened.created or screened_replay.created:
raise RuntimeError("Screening request idempotency is not stable.")
if screened.evidence.ref != screened_replay.evidence.ref:
raise RuntimeError("Screening replay returned different evidence.")
if screened.evidence.outcome != "potential" or screened.evidence.candidate_count != 1:
raise RuntimeError(f"Synthetic match was not reviewable: {screened.evidence!r}")
run = get_screening_run(
session,
operator,
run_id=screened.evidence.run_id,
)
candidate, disposition = record_disposition(
session,
reviewer,
candidate_id=run.candidates[0].id,
disposition=DispositionInput(
decision="false_positive",
reason="Independent fixture evidence excludes the screened party.",
evidence_refs=(acquired.snapshot.raw_evidence_ref,),
),
)
if candidate.review_status != "false_positive":
raise RuntimeError("Independent review did not resolve the candidate.")
if disposition.separation_status != "independent":
raise RuntimeError("Reviewer separation evidence was not retained.")
cleared = screening_provider.check_freshness(
session,
operator,
SanctionsScreeningFreshnessRequest(
evidence_ref=screened.evidence.ref,
current_subject=subject,
expected_list_snapshot_id=imported.id,
policy=policy,
),
)
if not cleared.fresh or cleared.gate_decision != "allow":
raise RuntimeError(f"Reviewed evidence did not clear the gate: {cleared!r}")
# Acquisition and review are separate durable commands in production.
session.commit()
session.expire_all()
refreshed = refresh_source(
session,
operator,
provider_id=SYNTHETIC_PROVIDER_ID,
idempotency_key="synthetic-sanctions-2026-08-02",
)
if refreshed.snapshot is None or refreshed.snapshot.ref == acquired.snapshot.ref:
raise RuntimeError("A new acquisition did not create new immutable evidence.")
current, current_created = import_connector_snapshot(
session,
operator,
registry=registry,
connector_snapshot_ref=refreshed.snapshot.ref,
)
if not current_created:
raise RuntimeError("The refreshed list state was not imported separately.")
stale = screening_provider.check_freshness(
session,
operator,
SanctionsScreeningFreshnessRequest(
evidence_ref=screened.evidence.ref,
current_subject=subject,
expected_list_snapshot_id=current.id,
policy=policy,
),
)
if stale.fresh or stale.gate_decision != "block":
raise RuntimeError(f"Changed source evidence did not close the gate: {stale!r}")
if "source_snapshot_changed" not in stale.reasons:
raise RuntimeError("Source change provenance was not reported.")
requirements = list_rescreening_requirements(session, operator)
if screened.evidence.run_id not in {item.run.id for item in requirements}:
raise RuntimeError("The stale screening is absent from the rescreening queue.")
session.commit()
if session.query(RecoveryOperation).count() != 2:
raise RuntimeError("Connector acquisition recovery evidence is incomplete.")
if session.query(RiskScreeningDisposition).count() != 1:
raise RuntimeError("Disposition evidence was duplicated or lost.")
finally:
bind_process_runtime_identity(None)
engine.dispose()
print(
"Connectors -> immutable sanctions snapshot -> Risk Compliance review "
"and rescreening composition passed."
)
return 0
def _principal(account_id: str, *, operational: bool) -> ApiPrincipal:
scopes = {
SANCTIONS_READ_SCOPE,
SANCTIONS_REVIEW_SCOPE,
}
if operational:
scopes.update(
{
CONNECTOR_SANCTIONS_READ_SCOPE,
SANCTIONS_REFRESH_SCOPE,
SANCTIONS_ADMIN_SCOPE,
SANCTIONS_SCREEN_SCOPE,
}
)
return ApiPrincipal(
principal=PrincipalRef(
account_id=account_id,
membership_id=f"membership-{account_id}",
tenant_id="tenant-1",
scopes=frozenset(scopes),
),
account=SimpleNamespace(id=account_id),
user=SimpleNamespace(id=f"membership-{account_id}"),
)
if __name__ == "__main__":
raise SystemExit(main())
@@ -798,6 +798,13 @@
"rationale": "The vault action group rotates the current key with policy, assurance, reason, and revision evidence.",
"repository": "govoplan-encryption"
},
{
"category": "public_integration",
"method": "POST",
"path": "/files/form-evidence/upload",
"rationale": "A short-lived purpose-bound bearer grant lets the public Forms Runtime surface stream one attachment directly to Files without granting general Files access.",
"repository": "govoplan-files"
},
{
"category": "ui_reachable",
"method": "POST",