Prove reference journey compositions
This commit is contained in:
@@ -42,11 +42,13 @@ does neither stays in the backlog until a concrete consumer exists.
|
||||
The Records vertical now supplies the journey's native file plan, immutable
|
||||
record and item revisions, chronology, close/reopen, retention calculation,
|
||||
holds, appraisal, independent disposition approval, recovery-ledger evidence,
|
||||
and archive-neutral package simulation. The executable Records fixture proves
|
||||
those native transitions without claiming archival custody. The journey still
|
||||
needs Forms Runtime/Decisions filing, resumable user handoffs, search
|
||||
reconstruction in the pinned composition, and one target-tested archive
|
||||
profile.
|
||||
and archive-neutral package simulation. Forms Runtime, Cases, and Decisions
|
||||
expose exact, permission-rechecked source revisions for explicit filing, and
|
||||
all three contribute metadata-only native Search projections that can be
|
||||
rebuilt from authoritative state. The executable fixtures prove those native
|
||||
transitions without claiming archival custody. The journey still needs
|
||||
resumable user handoffs, pinned-composition reconstruction evidence, and one
|
||||
target-tested archive profile.
|
||||
|
||||
## Why this sequence
|
||||
|
||||
|
||||
+12
-6
@@ -4,7 +4,7 @@
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Reconciled on | 2026-08-05 |
|
||||
| Reconciled on | 2026-08-06 |
|
||||
| Source scope | Local workspace manifests, source inventory, focused journey checks, signed release evidence, and live Gitea issue state |
|
||||
| Stable direction | [Platform Core Ideas](PLATFORM_CORE_IDEAS.md) and [Connected Governance Platform Roadmap](CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md) |
|
||||
| Delivery source | Gitea issues |
|
||||
@@ -67,8 +67,12 @@ prevent a reference-ready claim.
|
||||
|
||||
Services, Forms, Forms Runtime, Cases, Parties, Mandates, Approvals, Committee,
|
||||
Voting, Decisions, Portal, Postbox, and Audit have an executable service-to-
|
||||
decision fixture. Browser-complete assisted intake, production identity,
|
||||
records, delivery, and target evidence remain.
|
||||
decision fixture. Public and invitation intake can retain Files-backed
|
||||
evidence; Forms submissions, Cases, and formal Decisions can be explicitly
|
||||
filed as exact eAkte source revisions and reconstructed through permission-
|
||||
rechecked native Search projections. Browser-complete assisted intake,
|
||||
resumable work guidance, production identity and delivery, a named archive
|
||||
profile, and target evidence remain.
|
||||
|
||||
### Governed data and assurance
|
||||
|
||||
@@ -97,9 +101,11 @@ reconciliation, publication/export, and guided handoff evidence.
|
||||
2. Complete governed communication and Postbox against a named target.
|
||||
3. Complete the monthly-data flow and use it as the data foundation for
|
||||
sanctions screening.
|
||||
4. Complete one digital and assisted service-to-decision journey with an eAkte.
|
||||
5. Add native PostgreSQL search coverage for the objects used by those
|
||||
journeys; keep OpenSearch optional.
|
||||
4. Complete the browser and resumable-work proof for the digital and assisted
|
||||
service-to-decision journey with its existing exact eAkte filing contracts.
|
||||
5. Extend native PostgreSQL search coverage from the implemented service-to-
|
||||
decision objects to the communication and monthly-data journeys; keep
|
||||
OpenSearch optional.
|
||||
6. Prove one external product connector and one GovOPlaN federation exchange.
|
||||
7. Finish multi-host, restore, provider, accessibility, and independent signed
|
||||
target evidence before increasing maturity claims.
|
||||
|
||||
@@ -12,7 +12,7 @@ review.
|
||||
1. Register a typed datasource with source authority, purpose, classification,
|
||||
owner, freshness, and correction policy.
|
||||
2. Acquire or upload an immutable source state.
|
||||
3. execute a versioned flow and retain intermediate materializations and
|
||||
3. Execute a versioned flow and retain intermediate materializations and
|
||||
provenance;
|
||||
4. publish a report or decision input against exact source and flow revisions;
|
||||
5. link obligation, governed object, risk, control, evidence, finding,
|
||||
@@ -39,3 +39,24 @@ The artifact remains a `product` package. Promotion to `reference` requires:
|
||||
|
||||
Optional Connectors, Files, Notifications, and Workflow Engine integrations
|
||||
must remain capability-based and absence-safe.
|
||||
|
||||
## Executable evidence
|
||||
|
||||
- `tools/checks/check-datasource-composition.py` composes Connector snapshots,
|
||||
governed Datasources, queued Dataflow execution, frozen publication,
|
||||
idempotent replay, and recovery evidence.
|
||||
- `govoplan-dataflow/fixtures/golden/monthly-reconciliation` pins synthetic
|
||||
monthly inputs, stable reconciliation hashes, reviewed decisions, expected
|
||||
output, source fingerprints, and output hashes.
|
||||
- `tools/checks/check-sanctions-screening-composition.py` composes an immutable
|
||||
Connector acquisition, idempotent Risk Compliance import and screening,
|
||||
independent disposition, a cleared gate, changed-source invalidation, and
|
||||
the rescreening queue through the registered versioned capabilities.
|
||||
- `govoplan-dataflow/fixtures/golden/sanctions-screening` independently proves
|
||||
the deterministic normalization and matching graph with exact expected
|
||||
output.
|
||||
|
||||
These checks use synthetic data and run without network access. They prove the
|
||||
module contracts and durable state transitions; they do not replace the
|
||||
deployment, security, privacy, accessibility, and operator evidence still
|
||||
listed above.
|
||||
|
||||
@@ -28,7 +28,17 @@
|
||||
"kind": "documentation",
|
||||
"reference": "packages/product/governed-data-assurance/README.md",
|
||||
"summary": "Defines the package boundary, provenance chain, and reference-readiness gates."
|
||||
},
|
||||
{
|
||||
"kind": "target_test",
|
||||
"reference": "tools/checks/check-datasource-composition.py",
|
||||
"summary": "Proves governed Connector acquisition, Datasource registration, queued Dataflow execution, frozen publication, idempotency, and recovery evidence."
|
||||
},
|
||||
{
|
||||
"kind": "target_test",
|
||||
"reference": "tools/checks/check-sanctions-screening-composition.py",
|
||||
"summary": "Proves immutable sanctions acquisition, import, screening replay, independent review, freshness gates, and rescreening across module capabilities."
|
||||
}
|
||||
],
|
||||
"tags": ["datasources", "dataflow", "reporting", "assurance"]
|
||||
"tags": ["datasources", "dataflow", "reporting", "sanctions", "assurance"]
|
||||
}
|
||||
|
||||
@@ -24,19 +24,31 @@ grant cross-module table access and can omit optional presentation, work,
|
||||
deliberation, delivery, or records modules while retaining explicit references
|
||||
to externally performed steps.
|
||||
|
||||
When Records is present, Forms Runtime, Cases, and Decisions expose exact,
|
||||
digest-bound source snapshots for explicit filing. The source module rechecks
|
||||
current access, Records chooses the destination and preserves chronology, and
|
||||
the filed reference never becomes an editable copy. When Search is present,
|
||||
the same three owners contribute rebuildable metadata-only projections. Form
|
||||
values, evidence payloads, Decision reasoning, operative results, and
|
||||
conditions are excluded; every candidate is authorized again before it is
|
||||
shown.
|
||||
|
||||
## Security And Recovery
|
||||
|
||||
Every provider is tenant-bound. Missing or conflicting authority fails closed.
|
||||
Protected Decision content has a separate permission. Writes are replay-safe
|
||||
and OCC-guarded. Database restore is the semantic-state recovery unit; file and
|
||||
communication effects remain governed by their owning providers and are linked
|
||||
through requested/observed effect, evidence, and audit references.
|
||||
through requested/observed effect, evidence, and audit references. Search is a
|
||||
derived recovery unit and can be rebuilt from authoritative module state.
|
||||
|
||||
The executable fixture in
|
||||
`tests/test_institutional_governance_journey.py` proves SQL-backed Service,
|
||||
Case, Party, Mandate, Committee meeting/agendum/vote/minute, and Decision state.
|
||||
`tests/test_institutional_service_journey.py` separately proves exact Portal
|
||||
Form launch, persisted submission provenance, and idempotent replay.
|
||||
Target-environment accessibility, security, operator, privacy,
|
||||
delivery-provider, and recovery evidence are still required before this product
|
||||
package may claim `reference_ready` maturity.
|
||||
Module-level Records source tests prove exact Form submission, Case revision,
|
||||
and Decision revision filing. Target-environment browser accessibility,
|
||||
resumable work guidance, production identity and delivery, a named archive
|
||||
profile, and recovery evidence are still required before this product package
|
||||
may claim `reference_ready` maturity.
|
||||
|
||||
@@ -35,6 +35,7 @@
|
||||
{"module_id": "forms_runtime"},
|
||||
{"module_id": "postbox"},
|
||||
{"module_id": "records"},
|
||||
{"module_id": "search"},
|
||||
{"module_id": "tasks"},
|
||||
{"module_id": "workflow_engine"}
|
||||
],
|
||||
|
||||
@@ -109,6 +109,7 @@ PY
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-encryption/tests
|
||||
"$PYTHON" -m pytest -q /mnt/DATA/git/govoplan-campaign/tests/test_approval_gate.py
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-datasource-composition.py"
|
||||
"$PYTHON" "$META_ROOT/tools/checks/check-sanctions-screening-composition.py"
|
||||
"$PYTHON" -m unittest discover -s /mnt/DATA/git/govoplan-mail/tests
|
||||
"$PYTHON" -m unittest tests.test_api_smoke.ApiSmokeTests.test_mailbox_message_listing_reports_total_count
|
||||
|
||||
|
||||
@@ -0,0 +1,306 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Prove the governed Connectors -> Risk Compliance sanctions journey."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from govoplan_connectors.backend.db.models import (
|
||||
ConnectorSanctionsAcquisitionRun,
|
||||
ConnectorSanctionsSnapshot,
|
||||
)
|
||||
from govoplan_connectors.backend.sanctions_sources import (
|
||||
SANCTIONS_READ_SCOPE as CONNECTOR_SANCTIONS_READ_SCOPE,
|
||||
SANCTIONS_REFRESH_SCOPE,
|
||||
SYNTHETIC_PROVIDER_ID,
|
||||
)
|
||||
from govoplan_core.auth import ApiPrincipal
|
||||
from govoplan_core.core.access import PrincipalRef
|
||||
from govoplan_core.core.modules import ModuleContext
|
||||
from govoplan_core.core.recovery import RecoveryCheckpoint, RecoveryOperation
|
||||
from govoplan_core.core.runtime_coordination import (
|
||||
DistributedLease,
|
||||
RuntimeIdentity,
|
||||
bind_process_runtime_identity,
|
||||
)
|
||||
from govoplan_core.core.sanctions import (
|
||||
SanctionsScreeningFreshnessRequest,
|
||||
SanctionsScreeningPolicy,
|
||||
SanctionsScreeningRequest,
|
||||
SanctionsScreeningSubject,
|
||||
sanctions_screening_provider,
|
||||
sanctions_snapshot_provider,
|
||||
)
|
||||
from govoplan_core.db.base import Base
|
||||
from govoplan_core.server.registry import build_platform_registry
|
||||
from govoplan_risk_compliance.backend.db.models import (
|
||||
RiskAssuranceEdge,
|
||||
RiskAssuranceNode,
|
||||
RiskSanctionsAddress,
|
||||
RiskSanctionsAlias,
|
||||
RiskSanctionsDate,
|
||||
RiskSanctionsEntry,
|
||||
RiskSanctionsIdentifier,
|
||||
RiskSanctionsListSnapshot,
|
||||
RiskScreeningCandidate,
|
||||
RiskScreeningDisposition,
|
||||
RiskScreeningException,
|
||||
RiskScreeningRun,
|
||||
RiskScreeningSubjectSnapshot,
|
||||
)
|
||||
from govoplan_risk_compliance.backend.permissions import (
|
||||
SANCTIONS_ADMIN_SCOPE,
|
||||
SANCTIONS_READ_SCOPE,
|
||||
SANCTIONS_REVIEW_SCOPE,
|
||||
SANCTIONS_SCREEN_SCOPE,
|
||||
)
|
||||
from govoplan_risk_compliance.backend.review import (
|
||||
DispositionInput,
|
||||
record_disposition,
|
||||
)
|
||||
from govoplan_risk_compliance.backend.sanctions_catalog import (
|
||||
import_connector_snapshot,
|
||||
)
|
||||
from govoplan_risk_compliance.backend.screening import (
|
||||
get_screening_run,
|
||||
list_rescreening_requirements,
|
||||
)
|
||||
|
||||
|
||||
TABLES = (
|
||||
DistributedLease.__table__,
|
||||
RecoveryOperation.__table__,
|
||||
RecoveryCheckpoint.__table__,
|
||||
ConnectorSanctionsAcquisitionRun.__table__,
|
||||
ConnectorSanctionsSnapshot.__table__,
|
||||
RiskAssuranceNode.__table__,
|
||||
RiskAssuranceEdge.__table__,
|
||||
RiskSanctionsListSnapshot.__table__,
|
||||
RiskSanctionsEntry.__table__,
|
||||
RiskSanctionsAlias.__table__,
|
||||
RiskSanctionsIdentifier.__table__,
|
||||
RiskSanctionsDate.__table__,
|
||||
RiskSanctionsAddress.__table__,
|
||||
RiskScreeningSubjectSnapshot.__table__,
|
||||
RiskScreeningRun.__table__,
|
||||
RiskScreeningCandidate.__table__,
|
||||
RiskScreeningDisposition.__table__,
|
||||
RiskScreeningException.__table__,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
registry = build_platform_registry(("connectors", "risk_compliance"))
|
||||
registry.configure_capability_context(
|
||||
ModuleContext(registry=registry, settings=object())
|
||||
)
|
||||
snapshot_provider = sanctions_snapshot_provider(registry)
|
||||
screening_provider = sanctions_screening_provider(registry)
|
||||
refresh_source = getattr(snapshot_provider, "refresh_source", None)
|
||||
if snapshot_provider is None or not callable(refresh_source):
|
||||
raise RuntimeError("Connectors sanctions acquisition is unavailable.")
|
||||
if screening_provider is None:
|
||||
raise RuntimeError("Risk Compliance sanctions screening is unavailable.")
|
||||
|
||||
engine = create_engine("sqlite:///:memory:")
|
||||
Base.metadata.create_all(engine, tables=TABLES)
|
||||
bind_process_runtime_identity(
|
||||
RuntimeIdentity(
|
||||
installation_id="sanctions-composition-check",
|
||||
node_id="sanctions-worker",
|
||||
incarnation="sanctions-worker-incarnation",
|
||||
role="worker",
|
||||
software_version="test",
|
||||
composition_hash="d" * 64,
|
||||
)
|
||||
)
|
||||
try:
|
||||
with Session(engine) as session:
|
||||
operator = _principal("operator-1", operational=True)
|
||||
reviewer = _principal("reviewer-1", operational=False)
|
||||
|
||||
acquired = refresh_source(
|
||||
session,
|
||||
operator,
|
||||
provider_id=SYNTHETIC_PROVIDER_ID,
|
||||
idempotency_key="synthetic-sanctions-2026-08-01",
|
||||
)
|
||||
replay = refresh_source(
|
||||
session,
|
||||
operator,
|
||||
provider_id=SYNTHETIC_PROVIDER_ID,
|
||||
idempotency_key="synthetic-sanctions-2026-08-01",
|
||||
)
|
||||
if acquired.status != "succeeded" or acquired.snapshot is None:
|
||||
raise RuntimeError(f"Synthetic acquisition failed: {acquired!r}")
|
||||
if (
|
||||
replay.run_id != acquired.run_id
|
||||
or replay.snapshot is None
|
||||
or replay.snapshot.ref != acquired.snapshot.ref
|
||||
or replay.snapshot.sha256 != acquired.snapshot.sha256
|
||||
):
|
||||
raise RuntimeError("Acquisition idempotency did not replay exact evidence.")
|
||||
|
||||
imported, created = import_connector_snapshot(
|
||||
session,
|
||||
operator,
|
||||
registry=registry,
|
||||
connector_snapshot_ref=acquired.snapshot.ref,
|
||||
)
|
||||
imported_replay, replay_created = import_connector_snapshot(
|
||||
session,
|
||||
operator,
|
||||
registry=registry,
|
||||
connector_snapshot_ref=acquired.snapshot.ref,
|
||||
)
|
||||
if not created or replay_created or imported_replay.id != imported.id:
|
||||
raise RuntimeError("Risk Compliance snapshot import is not idempotent.")
|
||||
|
||||
subject = SanctionsScreeningSubject(
|
||||
subject_type="person",
|
||||
primary_name="Alex Example",
|
||||
subject_ref="party:fixture-person-1",
|
||||
)
|
||||
policy = SanctionsScreeningPolicy(failure_policy="block")
|
||||
request = SanctionsScreeningRequest(
|
||||
list_snapshot_id=imported.id,
|
||||
idempotency_key="fixture-party-screening-1",
|
||||
subject=subject,
|
||||
policy=policy,
|
||||
)
|
||||
screened = screening_provider.request_screening(
|
||||
session,
|
||||
operator,
|
||||
request,
|
||||
)
|
||||
screened_replay = screening_provider.request_screening(
|
||||
session,
|
||||
operator,
|
||||
request,
|
||||
)
|
||||
if not screened.created or screened_replay.created:
|
||||
raise RuntimeError("Screening request idempotency is not stable.")
|
||||
if screened.evidence.ref != screened_replay.evidence.ref:
|
||||
raise RuntimeError("Screening replay returned different evidence.")
|
||||
if screened.evidence.outcome != "potential" or screened.evidence.candidate_count != 1:
|
||||
raise RuntimeError(f"Synthetic match was not reviewable: {screened.evidence!r}")
|
||||
|
||||
run = get_screening_run(
|
||||
session,
|
||||
operator,
|
||||
run_id=screened.evidence.run_id,
|
||||
)
|
||||
candidate, disposition = record_disposition(
|
||||
session,
|
||||
reviewer,
|
||||
candidate_id=run.candidates[0].id,
|
||||
disposition=DispositionInput(
|
||||
decision="false_positive",
|
||||
reason="Independent fixture evidence excludes the screened party.",
|
||||
evidence_refs=(acquired.snapshot.raw_evidence_ref,),
|
||||
),
|
||||
)
|
||||
if candidate.review_status != "false_positive":
|
||||
raise RuntimeError("Independent review did not resolve the candidate.")
|
||||
if disposition.separation_status != "independent":
|
||||
raise RuntimeError("Reviewer separation evidence was not retained.")
|
||||
|
||||
cleared = screening_provider.check_freshness(
|
||||
session,
|
||||
operator,
|
||||
SanctionsScreeningFreshnessRequest(
|
||||
evidence_ref=screened.evidence.ref,
|
||||
current_subject=subject,
|
||||
expected_list_snapshot_id=imported.id,
|
||||
policy=policy,
|
||||
),
|
||||
)
|
||||
if not cleared.fresh or cleared.gate_decision != "allow":
|
||||
raise RuntimeError(f"Reviewed evidence did not clear the gate: {cleared!r}")
|
||||
|
||||
# Acquisition and review are separate durable commands in production.
|
||||
session.commit()
|
||||
session.expire_all()
|
||||
|
||||
refreshed = refresh_source(
|
||||
session,
|
||||
operator,
|
||||
provider_id=SYNTHETIC_PROVIDER_ID,
|
||||
idempotency_key="synthetic-sanctions-2026-08-02",
|
||||
)
|
||||
if refreshed.snapshot is None or refreshed.snapshot.ref == acquired.snapshot.ref:
|
||||
raise RuntimeError("A new acquisition did not create new immutable evidence.")
|
||||
current, current_created = import_connector_snapshot(
|
||||
session,
|
||||
operator,
|
||||
registry=registry,
|
||||
connector_snapshot_ref=refreshed.snapshot.ref,
|
||||
)
|
||||
if not current_created:
|
||||
raise RuntimeError("The refreshed list state was not imported separately.")
|
||||
|
||||
stale = screening_provider.check_freshness(
|
||||
session,
|
||||
operator,
|
||||
SanctionsScreeningFreshnessRequest(
|
||||
evidence_ref=screened.evidence.ref,
|
||||
current_subject=subject,
|
||||
expected_list_snapshot_id=current.id,
|
||||
policy=policy,
|
||||
),
|
||||
)
|
||||
if stale.fresh or stale.gate_decision != "block":
|
||||
raise RuntimeError(f"Changed source evidence did not close the gate: {stale!r}")
|
||||
if "source_snapshot_changed" not in stale.reasons:
|
||||
raise RuntimeError("Source change provenance was not reported.")
|
||||
requirements = list_rescreening_requirements(session, operator)
|
||||
if screened.evidence.run_id not in {item.run.id for item in requirements}:
|
||||
raise RuntimeError("The stale screening is absent from the rescreening queue.")
|
||||
|
||||
session.commit()
|
||||
if session.query(RecoveryOperation).count() != 2:
|
||||
raise RuntimeError("Connector acquisition recovery evidence is incomplete.")
|
||||
if session.query(RiskScreeningDisposition).count() != 1:
|
||||
raise RuntimeError("Disposition evidence was duplicated or lost.")
|
||||
finally:
|
||||
bind_process_runtime_identity(None)
|
||||
engine.dispose()
|
||||
|
||||
print(
|
||||
"Connectors -> immutable sanctions snapshot -> Risk Compliance review "
|
||||
"and rescreening composition passed."
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _principal(account_id: str, *, operational: bool) -> ApiPrincipal:
|
||||
scopes = {
|
||||
SANCTIONS_READ_SCOPE,
|
||||
SANCTIONS_REVIEW_SCOPE,
|
||||
}
|
||||
if operational:
|
||||
scopes.update(
|
||||
{
|
||||
CONNECTOR_SANCTIONS_READ_SCOPE,
|
||||
SANCTIONS_REFRESH_SCOPE,
|
||||
SANCTIONS_ADMIN_SCOPE,
|
||||
SANCTIONS_SCREEN_SCOPE,
|
||||
}
|
||||
)
|
||||
return ApiPrincipal(
|
||||
principal=PrincipalRef(
|
||||
account_id=account_id,
|
||||
membership_id=f"membership-{account_id}",
|
||||
tenant_id="tenant-1",
|
||||
scopes=frozenset(scopes),
|
||||
),
|
||||
account=SimpleNamespace(id=account_id),
|
||||
user=SimpleNamespace(id=f"membership-{account_id}"),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -798,6 +798,13 @@
|
||||
"rationale": "The vault action group rotates the current key with policy, assurance, reason, and revision evidence.",
|
||||
"repository": "govoplan-encryption"
|
||||
},
|
||||
{
|
||||
"category": "public_integration",
|
||||
"method": "POST",
|
||||
"path": "/files/form-evidence/upload",
|
||||
"rationale": "A short-lived purpose-bound bearer grant lets the public Forms Runtime surface stream one attachment directly to Files without granting general Files access.",
|
||||
"repository": "govoplan-files"
|
||||
},
|
||||
{
|
||||
"category": "ui_reachable",
|
||||
"method": "POST",
|
||||
|
||||
Reference in New Issue
Block a user