Compare commits

...
91 Commits
Author SHA1 Message Date
zemion 6d37aa527f fix(ui): unify heading help, table sizing and navigation contracts
Verified with the coordinated workspace changes by devkit full run
2026-09-08T225814-186389-0000-3e3ed7cd (all seven phases passed).
This shared UI pass does not mark the individual module reviews complete.
2026-09-09 02:03:16 +02:00
zemion 6591aaa3fd fix(core): preserve data integrity and bound shared UI and response work
Module Package Release / publish-packages (push) Successful in 13s
Release v0.1.46. Coordinated integrity review: GovOPlaN/govoplan-core#298.
2026-09-08 12:30:38 +02:00
zemion dc1f244f17 feat(security): isolate bounded work and support required auth actions 2026-09-08 07:47:17 +02:00
zemion a6d056a3df release: lock complete 0.1.45 WebUI composition to tagged sources
Module Package Release / publish-packages (push) Successful in 12s
2026-09-08 03:15:07 +02:00
zemion c3daa4a9aa perf(webui): load optional password generation on demand 2026-09-08 03:15:07 +02:00
zemion c209b3c27d fix(release): include Tasks and complete Git-installable module composition 2026-09-08 02:06:35 +02:00
zemion 32c70a4657 perf(webui): keep settings-only appearance defaults out of startup 2026-09-08 01:41:04 +02:00
zemion b75ca34295 feat: consolidate shared UI and harden browser authority for release 2026-09-08 01:35:05 +02:00
zemion ac40774785 feat(webui): expose stable product destinations
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 18:06:52 +02:00
zemion 9a3008002d feat: define governed tenant erasure contracts
Module Package Release / publish-packages (push) Successful in 12s
2026-08-24 16:00:06 +02:00
zemion 9cb2080938 feat: collect infrastructure dependency inventories
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 15:00:08 +02:00
zemion 08c3e47b6d Exercise accessible resident permit journeys
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 14:03:43 +02:00
zemion 6e518fa6a2 feat: compose stable product surfaces
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 13:21:24 +02:00
zemion f98cf9ced8 feat(help): enforce owner-aware high-risk contexts
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 11:48:23 +02:00
zemion d2e491348d feat(docs): validate owner structured translations
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 01:15:30 +02:00
zemion 562d278f60 feat(core): add structured documentation localization contract
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 20:25:12 +02:00
zemion c6f6faf64f feat(core): add datasource lifecycle governance contracts
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 19:37:44 +02:00
zemion 1c3ee9e8c7 Release Core v0.1.35 configuration package safeguards
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 18:05:33 +02:00
zemion aa91063211 Release Core v0.1.34 with JMAP mail contracts
Module Package Release / publish-packages (push) Successful in 14s
2026-08-22 17:08:54 +02:00
zemion fa2d5d40dd Release Core v0.1.33 with redirect-sensitive headers
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 16:11:02 +02:00
zemion 6ccef162f6 Release Core v0.1.32 with bounded HTTP request bodies
Module Package Release / publish-packages (push) Successful in 14s
2026-08-22 14:32:24 +02:00
zemion 48dac139a5 feat(wiki): compose governed Wiki WebUI
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 13:32:20 +02:00
zemion a090e5af20 feat(tickets): add optional integration contracts and WebUI composition
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 12:06:04 +02:00
zemion 0c1358b862 feat(policy): add delegation and escalation contracts
Module Package Release / publish-packages (push) Failing after 6s
2026-08-22 03:12:30 +02:00
zemion a9035c4c3b feat: add Campaign work orchestration contract
Module Package Release / publish-packages (push) Failing after 6s
2026-08-22 02:15:32 +02:00
zemion 137c7c005f test(core): recognize Campaign work integrations
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 01:06:12 +02:00
zemion 8eeea968f2 chore(release): load Campaign 0.1.22
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 01:01:35 +02:00
zemion 0ca6568005 chore(release): load Campaign 0.1.21
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 00:54:00 +02:00
zemion af90db44c9 chore(webui): load Campaign collaboration release
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 00:18:06 +02:00
zemion 5de46e9c0e chore(webui): load Files folder sync release
Module Package Release / publish-packages (push) Successful in 12s
2026-08-21 22:50:48 +02:00
zemion 1d9b677c1b chore(webui): load IDM relationship administration release
Module Package Release / publish-packages (push) Successful in 12s
2026-08-21 22:06:29 +02:00
zemion 54178ee56c chore(webui): load Access credential help release
Module Package Release / publish-packages (push) Successful in 12s
2026-08-21 21:15:31 +02:00
zemion 10e7597612 feat(core): define datasource visibility policy contract
Module Package Release / publish-packages (push) Failing after 6s
2026-08-21 20:31:51 +02:00
zemion 142ccbc587 feat(core): define managed tabular file contract
Module Package Release / publish-packages (push) Successful in 12s
2026-08-21 19:29:48 +02:00
zemion f75ad48d78 test(core): isolate entitlement navigation fixture 2026-08-21 19:26:00 +02:00
zemion 5a9e8f79f9 test(webui): cover contextual help in the browser 2026-08-21 17:36:06 +02:00
zemion fbea74a74b feat(core): define durable datasource artifact contract 2026-08-21 17:36:05 +02:00
zemion 925dc33696 feat(docs): define semantic subject contract 2026-08-21 14:55:58 +02:00
zemion 4b0737e1cd feat: support autonomous campaign schedule results 2026-08-20 21:10:04 +02:00
zemion 4f4007aff1 feat: define policy impact subject contract 2026-08-20 20:27:17 +02:00
zemion 8e687c4420 feat: define bulk governance projection contract 2026-08-20 19:46:53 +02:00
zemion 604f20eed7 feat: classify audit evidence export scope 2026-08-20 18:00:18 +02:00
zemion 6a2da94e47 feat: load connector governance webui 2026-08-20 12:46:01 +02:00
zemion e121ca900e feat: load identity administration webui 2026-08-20 12:27:37 +02:00
zemion 79629c5a2c feat: define campaign archive encryption policy contract 2026-08-20 12:12:27 +02:00
zemion 026e451aa4 feat: configure scheduling self-enrollment limits 2026-08-20 11:45:07 +02:00
zemion f11c675d11 feat: govern accessible appearance overrides 2026-08-20 10:50:55 +02:00
zemion 0fae09ba3c feat: extend calendar scheduling reconciliation contract 2026-08-20 07:44:16 +02:00
zemion 8f642bd618 feat: govern effective appearance defaults 2026-08-20 07:03:28 +02:00
zemion 6643c8fc1e feat: add validated appearance palettes 2026-08-20 06:26:52 +02:00
zemion fd90b60430 feat: centralize access explanation subject selection 2026-08-20 06:16:43 +02:00
zemion 0aae6f0539 feat(postbox): schedule lifecycle notification reconciliation 2026-08-20 04:58:35 +02:00
zemion be7b79612c feat(postbox): expose governed grouping policy 2026-08-20 04:22:29 +02:00
zemion 557c77670b feat(postbox): expose configurable protection contracts 2026-08-20 03:42:58 +02:00
zemion d277218784 feat(webui): preserve state in shared widget launches 2026-08-20 02:39:41 +02:00
zemion cf16a7b27a feat(core): add layered side rail preferences 2026-08-20 01:47:04 +02:00
zemion 51bf14f376 docs(core): define collaboration connector strategy 2026-08-19 23:15:35 +02:00
zemion 8d9bcfd8b5 test(core): prove Views cannot grant access 2026-08-19 22:55:19 +02:00
zemion 3c7a593f63 feat(templates): carry content field requirements 2026-08-19 21:54:51 +02:00
zemion 9d1352ba30 feat(mail): define standard IMAP folder mappings 2026-08-19 21:27:42 +02:00
zemion 4cf2bfeb3e feat(operations): add runtime work status contract 2026-08-19 20:52:44 +02:00
zemion 94c94fefb4 test(webui): cover View-focused Quick Access catalogues 2026-08-19 20:14:02 +02:00
zemion d600bca374 test: make full discovery optional-state independent 2026-08-19 19:20:05 +02:00
zemion ffaab543d2 feat(webui): govern actions, quick access, and metrics
Refs #264, #285, #289
2026-08-19 18:47:46 +02:00
zemion 41db78c201 Define semantic page archetypes 2026-08-19 14:26:25 +02:00
zemion c042244da8 Define semantic page action layouts 2026-08-19 13:33:51 +02:00
zemion 5a2e99f496 feat: add status and payment capability contracts 2026-08-19 12:33:34 +02:00
zemion 8a925782ab feat: add UI conformance and launch context 2026-08-18 21:32:25 +02:00
zemion 7685a103e8 Centralize shared WebUI structural primitives 2026-08-18 13:17:31 +02:00
zemion ee5c881df9 Centralize metric and description layouts 2026-08-18 11:30:39 +02:00
zemion 6814a41ae4 Add shared WebUI layout primitives 2026-08-18 10:42:55 +02:00
zemion dd7ad4d9c7 feat: add shared workspace layouts 2026-08-18 02:17:13 +02:00
zemion 934db6d44b feat: centralize shared page layouts 2026-08-18 01:03:33 +02:00
zemion d307e29145 feat: add exact credential help contexts 2026-08-17 19:51:51 +02:00
zemion ff88142471 feat: add exact retention help contexts 2026-08-17 17:41:45 +02:00
zemion 887e9beb9e docs: update meta documentation links 2026-08-17 16:52:53 +02:00
zemion e6457b3f6b feat: add governed DSAR workflow 2026-08-07 14:53:07 +02:00
zemion eb0c01c5d2 feat: expose typed deployment capability receipts 2026-08-07 11:15:49 +02:00
zemion 40cc012124 feat(modules): validate catalog permission declarations 2026-08-07 01:59:24 +02:00
zemion 44196f5620 Validate module catalog provenance and availability 2026-08-06 22:42:08 +02:00
zemion 9ceb1b8c22 Add trusted public module catalog installs 2026-08-06 21:12:54 +02:00
zemion 32c234fbdb Add product presentation extension contracts 2026-08-06 19:02:53 +02:00
zemion d65d7a8e5f Add shared work-item provider contracts 2026-08-06 16:06:17 +02:00
zemion b5f5be15f6 Add governed form evidence contract 2026-08-06 12:42:20 +02:00
zemion f5949427cc feat(records): define archive transfer provider contract 2026-08-06 05:36:11 +02:00
zemion 5d1287735e feat(core): add provider-neutral records filing contract 2026-08-06 01:43:08 +02:00
zemion 7ea0cb8655 Fix responsive DataGrid contraction 2026-08-05 22:42:21 +02:00
zemion b553513c9f Release v0.1.18
Module Package Release / publish-packages (push) Successful in 12s
2026-08-05 21:08:17 +02:00
zemion b5a4eb177a Release v0.1.17
Module Package Release / publish-packages (push) Successful in 12s
2026-08-05 20:34:47 +02:00
zemion f1a5be2a93 Release v0.1.16
Module Package Release / publish-packages (push) Successful in 12s
2026-08-05 19:53:46 +02:00
zemion add7a99f6d feat: add temporal context and contextual help 2026-08-05 00:03:31 +02:00
395 changed files with 41465 additions and 2243 deletions
+3
View File
@@ -138,6 +138,7 @@ dist
# Local WebUI test/build scratch directories
.component-test-build/
.component-test-build-*/
.file-drop-test-build/
.module-test-build/
.policy-test-build/
@@ -149,6 +150,8 @@ webui/.module-test-build/
webui/.policy-test-build/
webui/.template-preview-test-build/
webui/.import-test-build/
webui/dist-conformance/
webui/test-results/
# Security audit reports
audit-reports/
+13 -4
View File
@@ -26,18 +26,27 @@ cd /mnt/DATA/git/govoplan-core
For WebUI checks:
```bash
cd /mnt/DATA/git/govoplan
./devkit doctor --repo core
./devkit check --profile ui --repo core --dry-run
./devkit check --profile ui --repo core
```
For an individually selected component batch (one compilation):
```bash
cd /mnt/DATA/git/govoplan-core/webui
PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run test:mail-components
PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run test:module-capabilities
PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versions/node/v22.22.3/bin:$PATH /home/zemion/.nvm/versions/node/v22.22.3/bin/npm run test:module-permutations
npm run test:components -- mail-components page-layout
npm run test:module-capabilities
npm run test:module-permutations
```
Run the consolidated focused check when a change touches module discovery, optional integrations, shared mail components, or mailbox listing:
```bash
cd /mnt/DATA/git/govoplan
tools/checks/check-focused.sh
./devkit check --profile full
```
## Working Rules
+2
View File
@@ -33,6 +33,8 @@ Canonical policy documents live in `docs/`:
- [ACCESS_RBAC_MODEL.md](docs/ACCESS_RBAC_MODEL.md)
- [GOVERNANCE_MODEL.md](docs/GOVERNANCE_MODEL.md)
- [MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md)
- [INTEGRITY_PERFORMANCE_CONTRACT.md](docs/INTEGRITY_PERFORMANCE_CONTRACT.md)
- [TABULAR_SOURCE_CONTRACT.md](docs/TABULAR_SOURCE_CONTRACT.md)
- [DEPLOYMENT_OPERATOR_GUIDE.md](docs/DEPLOYMENT_OPERATOR_GUIDE.md)
- [CODEX_WORKFLOW.md](docs/CODEX_WORKFLOW.md)
@@ -0,0 +1,23 @@
from __future__ import annotations
from importlib.util import module_from_spec, spec_from_file_location
from pathlib import Path
_path = (
Path(__file__).resolve().parents[1]
/ "versions"
/ "a36d8e4f9b12_german_reference_locale.py"
)
_spec = spec_from_file_location("govoplan_german_reference_locale_migration", _path)
if _spec is None or _spec.loader is None:
raise RuntimeError(f"Unable to load migration implementation from {_path}")
_module = module_from_spec(_spec)
_spec.loader.exec_module(_module)
revision = _module.revision
down_revision = _module.down_revision
branch_labels = _module.branch_labels
depends_on = _module.depends_on
upgrade = _module.upgrade
downgrade = _module.downgrade
@@ -0,0 +1,23 @@
from __future__ import annotations
from importlib.util import module_from_spec, spec_from_file_location
from pathlib import Path
_path = (
Path(__file__).resolve().parents[1]
/ "versions"
/ "b47e6f809a13_data_subject_requests.py"
)
_spec = spec_from_file_location("govoplan_data_subject_requests_migration", _path)
if _spec is None or _spec.loader is None:
raise RuntimeError(f"Unable to load migration implementation from {_path}")
_module = module_from_spec(_spec)
_spec.loader.exec_module(_module)
revision = _module.revision
down_revision = _module.down_revision
branch_labels = _module.branch_labels
depends_on = _module.depends_on
upgrade = _module.upgrade
downgrade = _module.downgrade
@@ -0,0 +1,21 @@
"""Development-track wrapper for the ownership history repair."""
from __future__ import annotations
from importlib.util import module_from_spec, spec_from_file_location
from pathlib import Path
_path = Path(__file__).resolve().parents[1] / "versions" / "c58a2d7e9f10_ownership_decision_history.py"
_spec = spec_from_file_location("govoplan_ownership_decision_history_migration", _path)
if _spec is None or _spec.loader is None:
raise RuntimeError(f"Unable to load migration implementation from {_path}")
_module = module_from_spec(_spec)
_spec.loader.exec_module(_module)
revision = _module.revision
down_revision = _module.down_revision
branch_labels = _module.branch_labels
depends_on = _module.depends_on
upgrade = _module.upgrade
downgrade = _module.downgrade
+3 -1
View File
@@ -26,7 +26,9 @@ from govoplan_core.tenancy.scope import scope_registry
config = context.config
database_url = config.attributes.get("database_url") or settings.database_url
config.set_main_option("sqlalchemy.url", database_url)
# Alembic stores options through ConfigParser: escape only its interpolation
# syntax so URL-encoded credentials/socket paths reach SQLAlchemy unchanged.
config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%"))
if config.config_file_name is not None:
# Migrations can run inside the long-lived application process when module
@@ -0,0 +1,44 @@
"""adopt German as the untouched system reference locale
Revision ID: a36d8e4f9b12
Revises: f25c9d3e7a01
Create Date: 2026-08-05 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "a36d8e4f9b12"
down_revision = "f25c9d3e7a01"
branch_labels = None
depends_on = None
def upgrade() -> None:
bind = op.get_bind()
if "core_system_settings" not in set(sa.inspect(bind).get_table_names()):
return
settings = sa.table(
"core_system_settings",
sa.column("id", sa.String),
sa.column("default_locale", sa.String),
sa.column("created_at", sa.DateTime(timezone=True)),
sa.column("updated_at", sa.DateTime(timezone=True)),
)
bind.execute(
settings.update()
.where(settings.c.id == "global")
.where(settings.c.default_locale == "en")
.where(settings.c.created_at == settings.c.updated_at)
.values(default_locale="de", updated_at=sa.func.now())
)
def downgrade() -> None:
# Locale selection is user-visible state. A downgrade must not overwrite a
# German value that may have been selected explicitly after this migration.
pass
@@ -0,0 +1,77 @@
"""add governed data-subject request workflow
Revision ID: b47e6f809a13
Revises: a36d8e4f9b12
Create Date: 2026-08-07 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "b47e6f809a13"
down_revision = "a36d8e4f9b12"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if "core_data_subject_requests" in inspector.get_table_names():
return
op.create_table(
"core_data_subject_requests",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("reference", sa.String(length=120), nullable=False),
sa.Column("request_kind", sa.String(length=30), nullable=False),
sa.Column("status", sa.String(length=30), nullable=False),
sa.Column("subject", sa.JSON(), nullable=False),
sa.Column("purpose", sa.String(length=1000), nullable=False),
sa.Column("legal_basis", sa.String(length=1000), nullable=True),
sa.Column("due_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("requested_by_account_id", sa.String(length=36), nullable=False),
sa.Column("search_result", sa.JSON(), nullable=False),
sa.Column("erasure_plan", sa.JSON(), nullable=False),
sa.Column("execution_result", sa.JSON(), nullable=False),
sa.Column("coverage", sa.JSON(), nullable=False),
sa.Column("evidence_sha256", sa.String(length=64), nullable=True),
sa.Column("resource_revision", sa.Integer(), nullable=False),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("notes", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_core_data_subject_requests")),
)
op.create_index(
op.f("ix_core_data_subject_requests_tenant_id"),
"core_data_subject_requests",
["tenant_id"],
unique=False,
)
op.create_index(
op.f("ix_core_data_subject_requests_status"),
"core_data_subject_requests",
["status"],
unique=False,
)
op.create_index(
op.f("ix_core_data_subject_requests_due_at"),
"core_data_subject_requests",
["due_at"],
unique=False,
)
op.create_index(
"ix_core_data_subject_requests_tenant_status",
"core_data_subject_requests",
["tenant_id", "status"],
unique=False,
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if "core_data_subject_requests" in inspector.get_table_names():
op.drop_table("core_data_subject_requests")
@@ -0,0 +1,37 @@
"""repair decision history on previously upgraded ownership tables
Revision ID: c58a2d7e9f10
Revises: b47e6f809a13
Create Date: 2026-09-07
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c58a2d7e9f10"
down_revision = "b47e6f809a13"
branch_labels = None
depends_on = None
def upgrade() -> None:
# The original ownership migration gained this column after some databases
# had already applied it. create_all/checkfirst cannot upgrade those tables.
# Fresh installations already have it; never replace their audit evidence.
columns = {column["name"] for column in sa.inspect(op.get_bind()).get_columns(
"core_ownership_transfers"
)}
if "decisions" not in columns:
op.add_column(
"core_ownership_transfers",
sa.Column("decisions", sa.JSON(), nullable=False, server_default=sa.text("'[]'")),
)
def downgrade() -> None:
# Older installations and fresh installations at the preceding revision
# differ. Keep the additive column and any subsequently recorded evidence.
pass
+1
View File
@@ -142,6 +142,7 @@ system:tenants:read
system:tenants:create
system:tenants:update
system:tenants:suspend
system:tenants:erase
system:accounts:read
system:accounts:create
+35
View File
@@ -0,0 +1,35 @@
# Shared API client cache and authority boundaries
All optional WebUI modules use the Core API client. Its bounded in-memory caches
are an optimization, never an authorization mechanism. The backend must check
the current principal, tenant, and permissions even for conditional GETs.
- Identical simultaneous safe requests can share one network request. Requests
with caller-owned cancellation are independent.
- Responses allowing reuse have at most a 750 ms recent-response window.
`no-store` and `Vary: *` responses are not retained. `no-cache` and zero-age
responses require a server check; permitted ETags retain conditional GET
support without bypassing authorization. This follows the relevant
[HTTP cache-control semantics](https://www.rfc-editor.org/rfc/rfc9111.html#section-5.2.2).
- Explicit `cache: "no-store"`, `"reload"`, or `"no-cache"` reads bypass older
response data and supersede older requests for that resource. Reload is not a
mutation. Owning read helpers must pass these options through pagination.
- Writes invalidate caches before execution and again on settlement, including
failures whose server outcome may be uncertain. Reads started before or
during the write cannot seed reusable data after it finishes.
- The shell calls `clearApiReadCache()` before explicit auth updates and when
refreshing authoritative session data. API-settings changes, clearing the
token, authentication expiry, and changes to the paired session/CSRF cookie
also invalidate both stored and in-flight reuse. Cookie observation also
covers sign-in/out in another tab; it does not read the HttpOnly session token.
- Interactive sign-in and sign-out clear a previously saved automation key.
Explicit key-based connection settings still select the key's identity;
profile-only updates preserve settings identity to avoid reload loops.
- Expired responses from superseded reads or downloads do not trigger a login
prompt in a newer session.
- Every completion (including 304) must still own its cache slot and generation
before storing anything. An old caller may receive its own result, so feature
components must continue guarding displayed state against obsolete requests.
Regression coverage: `npm run test:api-client-cache` uses the real client and
isolated network fixtures. No live API or account data is involved.
+81
View File
@@ -0,0 +1,81 @@
# Disposable resource-bounded operations
`security.bounded_process.run_bounded_operation` runs a trusted, importable,
module-level `bytes -> bytes` function in a fresh interpreter. Core owns the
process lifecycle, not the business parser. Owners retain authorization,
sessions, provider reads, idempotency and persistence in the parent and pass
only explicit bounded data. Never accept the operation, module or source path
from a client. Use `security.worker_payload` for typed values; it does not use
pickle, arbitrary constructors or JSON object hooks.
The runner requires POSIX process groups, `waitid(WNOWAIT)` and resource limits.
Unsupported controls fail closed; there is no in-process fallback. The child
uses `-I -B`, a fixed minimal environment, `/` as working directory, closed
inherited descriptors and a new process session. Limits are installed before
the owning module is imported. Installed dependencies must support isolated
Python imports; development `PYTHONPATH` alone is insufficient.
`ProcessLimits` specifies wall-clock seconds (including child startup), CPU
seconds, virtual address space, input/output pipe bytes and maximum regular-file
size. Defaults are 10 seconds wall/CPU, 256 MiB address space, 8 MiB input and
output, and no regular-file output. Wall/CPU limits are at most 600 seconds,
memory 64 MiB8 GiB, pipe limits 1 byte256 MiB, and file size 02 GiB. Owners
must document their tighter functional limits; a transport cap does not replace
row, archive expansion, item-count or artifact limits.
Admission is non-queuing. `GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` defaults to 1
(range 116) and applies across these operations **within each API/worker
process**. Multiply capacity and memory budgets by the number of API/worker
processes when sizing an installation. This is not a fleet-wide semaphore,
cgroup quota, filesystem/network sandbox or permission to run arbitrary code.
`RLIMIT_FSIZE` is per file, not a total disk quota. Owners creating staged files
must enforce cumulative quotas and clean up their own private directories.
Owners preparing bounded local snapshots can enter
`bounded_operation_admission()` before preparation and pass its token as
`admission=` to the runner. This reuses shared capacity rather than reserving a
second slot. Tokens belong to their active context, thread and process; expired,
cross-thread and overlapping reuse fail. Preparation exceptions release the
slot without launching a child. Never hold admission while waiting for a user;
parent-side preparation still requires explicit I/O and byte bounds.
The parent concurrently drains stdout/stderr while writing input. Output is
bounded during reading, stderr is discarded and capped at 64 KiB, and raw child
tracebacks are never returned. Every success, exception, timeout, cancellation
and callback failure kills the owned process group before reaping its leader,
including descendants which close their inherited pipes. A module-level child
handler must return bytes; it must not print logs/progress to stdout.
The optional `cancelled` callback runs in the parent at most roughly every
50 ms while waiting. It must be fast and must not return an awaitable. It may
also service a module-owned bounded progress protocol; exceptions terminate
the child and propagate. There is no fabricated progress for killed work.
`ProcessBudgetError.code` distinguishes busy, cancelled, timeout, CPU, memory,
input/output limits, unavailable controls and worker failure. Owners map these
to their existing structured diagnostics and recovery semantics.
The private typed-data codec supports null, booleans, strings, bytes, integers,
floats, Decimal, UUID, date/time/datetime, lists, tuples and string-keyed maps.
It rejects unsupported objects, malformed/trailing bytes, duplicate keys,
excess depth (64) and node counts (1,000,000). Operation DTOs remain owner
contracts and require owner validation. Do not persist this private wire format
or use it as a public API.
Tests use real child processes for catastrophic regex, memory exhaustion,
noisy output, exact limits, cancellation, closed-pipe hangs and descendant
cleanup. These are local process regression tests, not production concurrent
load certification. Operators still need target Linux/cgroup, cancellation,
worker-count, memory and disk-quota evidence before raising concurrency.
## Deutsche Betriebszusammenfassung
Rechenintensive, vertrauenswürdige Moduloperationen laufen in einem frischen
Prozess mit harten Laufzeit-, CPU-, Speicher- und Ausgabegrenzen. Berechtigungen,
Sitzungen, Zugangsdaten und Datenbankänderungen bleiben im Hauptprozess. Fehlende
Betriebssystemkontrollen führen zu einer Diagnose, nicht zu ungeschützter
Ausführung. `GOVOPLAN_ISOLATED_PROCESS_CONCURRENCY` begrenzt die gemeinsame
Zulassung je API-/Worker-Prozess, standardmäßig auf 1. Mehrere Prozesse haben
jeweils eigene Grenzen; systemweite Speicher- und Festplattenquoten müssen
Betreiber zusätzlich konfigurieren und auf der Zielinstallation prüfen. Die
Schnittstelle ist keine Sandbox für beliebigen Code. Modul-Dokumentation nennt
die jeweiligen fachlichen Grenzen, Fortschritts- und Wiederholungsregeln.
+66 -1
View File
@@ -157,9 +157,61 @@ The initial implementation includes provider-neutral orchestration helpers:
- `apply_configuration_package(...)`
- `export_configuration_package(...)`
Portable fragments may bind deployment-specific operator input without placing
that value in the signed reusable definition. A payload value of
`{"$data": "requirement_key"}` references a key declared in the manifest's
`data_requirements`. Preflight fails before invoking the owning provider when a
reference is malformed, undeclared, or unresolved. Once supplied, Core replaces
the reference in memory and passes only the resolved fragment to the provider.
This mechanism is for deployment bindings and wording, not plaintext secrets:
credential-envelope or environment references remain the normal portable
boundary.
The first concrete provider is `govoplan_access.backend.configuration_provider`.
It supports access-owned `roles`, `groups`, and `group_role_assignments`
fragments and applies them idempotently.
fragments and applies them idempotently. Mail and Files also register providers
for deployment configuration: Mail owns receipt-bound SMTP profiles and Files
validates the deployment-owned managed-storage binding.
### Deployment capability receipt
The installer mounts a bounded, non-secret infrastructure receipt at the path
named by `GOVOPLAN_DEPLOYMENT_CAPABILITIES_PATH`. Core validates that document
once for configuration-package context and exposes typed capability and
post-install-task records to providers. Invalid receipts fail closed. Endpoint
metadata is sanitized, and secret fields may cross this boundary only as
`env:VARIABLE_NAME` references.
Feature providers remain responsible for their own semantics:
- Mail can derive host and port from `mail.smtp`, collect missing non-secret
transport fields, and bind an existing credential-envelope id. It never
accepts or exports a username, password, token, or decrypted credential.
- Files compares `files.storage` with the effective runtime backend, endpoint,
trust marker, bucket, and presence of referenced environment secrets. Storage
remains deployment-owned, so the provider reports `skip` when they agree and
blocks drift instead of rewriting process environment or storage credentials.
- A system-scoped Mail profile requires system configuration authority. Tenant
scope is the conservative default.
- Existing Mail configuration is preserved unless a reviewed fragment
explicitly selects `on_conflict: update`. Reapplying an unchanged fragment is
a no-op.
Ops projects the same Core-validated receipt. It must not maintain a second
parser with different validation or secret-handling rules.
Core also defines the inverse, read-only dependency-inventory contract used
before the installer changes one of those infrastructure capabilities. An
enabled module registers
`infrastructure.dependency_inventory.<module_id>` and returns bounded, stable
references to its persisted configuration or data, a lifecycle state, scope,
numeric metrics, and a required operator action. Providers must not return
secrets or use this read to migrate state. The Core collector validates provider
identity and capability coverage, orders records deterministically, and marks
the complete inventory failed when any provider raises or violates the
contract. Ops is the authorized projection boundary; the installer remains the
consumer and must match installation id, freshness, completion and impacted
capability coverage before apply.
The admin wizard backend starts with these routes:
@@ -183,6 +235,14 @@ The admin wizard backend starts with these routes:
10. Store import provenance, package version, supplied non-secret metadata, and
audit events.
Provider applies may commit independently. Core therefore stops at the first
apply or health blocker and reports an explicit rollback state. A blocked
preflight or a no-op needs no recovery; a successful multi-provider mutation
retains the reviewed pre-apply database snapshot as its generic rollback path;
a later-provider failure is reported as a partial apply that requires snapshot
recovery or an explicitly supported module-owned compensation. The generic
wizard never claims atomic cross-module undo.
The wizard should display everything necessary and nothing unnecessary. Generic
sections should cover package trust, dependency plan, required data, conflicts,
review, and result. Module-specific fields should appear only when the selected
@@ -233,6 +293,11 @@ Exported packages should record provenance: source GovOPlaN version, module
versions, exporter identity, timestamp, selected scope, redactions, and
validation status.
The orchestrator emits this provenance independently of provider payloads and
lists secret requirement keys as redacted without serializing their supplied
values. Providers still own the deeper rule that credentials, tokens, and
decrypted envelope contents must never appear in exported fragments.
## Catalogs And Trust
Configuration catalogs should follow the existing module package catalog model:
+84
View File
@@ -0,0 +1,84 @@
# Contextual Help Contract
GovOPlaN exposes context-sensitive help through `F1` and the titlebar help
control. The shell resolves a stable help identity from the focused control,
its containing surface, and the current route. The Docs module then projects
the best visible user or administrator topic for that identity.
## Resolution Order
The WebUI resolves help in this order:
1. an explicit `helpContextId` or `data-help-context-id` on the focused item
2. the focused shared control's `interfaceId`, `helpTopicId`, and label key
3. a containing dialog, card, administration section, or page surface
4. the current registered route, including dynamic module routes
5. a stable route-derived fallback when no explicit identity is available
Focused field and action contexts retain the page context as
`fallback_context`. This lets Docs show a field-specific topic when one exists
and otherwise open the owning page or module documentation instead of a generic
help page.
## Documentation Lookup
Static `DocumentationTopic` contributions announce exact contexts through
`metadata.help_contexts`. Core publishes that catalogue with the enabled module
manifest, allowing the shell to link directly to an exact topic when possible.
Docs still performs the authoritative audience, permission, configured-state,
and documentation-type filtering.
Core also maps explicit route, navigation, settings, and View surface IDs to
the module's static user or administrator documentation baseline. This makes a
page association complete by default and gives every derived field/action
context a useful fallback. Exact `metadata.help_contexts` remain the preferred
authoring mechanism for consequential or unfamiliar controls.
When there is no exact topic, Docs resolves the page fallback and then the first
visible topic owned by the module. If Docs is unavailable, the shell opens the
hosted documentation with the same context parameters.
## Authoring Controls
Core shared controls expose stable help metadata. Prefer these props rather
than adding custom `F1` listeners:
- `interfaceId` identifies a durable UI surface or action.
- `helpContextId` identifies a documentation context when it differs from the
interface identity.
- `helpModuleId` identifies the documentation-owning module when a shared
control is embedded in another module's page.
- `helpTopicId` links directly to a module-owned documentation topic.
- translated label keys provide deterministic field identities for ordinary
`FormField`, `ToggleSwitch`, search, date/time, email, button, dialog, and card
controls.
- `TableActionGroup` action definitions carry the same identities so focused
row actions can resolve consequence-specific help.
- `PageLayout` owns the page help scope and documentation identity for ordinary
headed pages. `WorkspaceLayout` owns the full-canvas workspace scope and its
labelled primary/content panes; pages inside it use `PageLayout` in
`workspace` mode and retain their own route-level help identity.
- `PasswordField` passes its owner context and module through reveal/generate
actions and the shared generator dialog. Credential consumers must supply an
exact owner context; the generic component does not own credential policy.
High-risk controls use one of the source-inventory risk classes (`authority`,
`credential`, `disclosure`, `encryption`, `external-effect`, `irreversible`,
`policy`, or `retention`) and require exact F1 help. The extractor infers
obvious cases conservatively; components may declare `data-help-risk`
explicitly or mark a reviewed ordinary control with
`data-help-risk-reviewed="standard"`. The strict workspace gate rejects new
unresolved high-risk debt.
Module routes, public routes, settings sections, and administration sections
may also declare `helpContextId` and `helpTopicId`. Each module must keep a
static user/admin documentation baseline and should list its important route,
workflow, setting, permission, and limitation identities in
`metadata.help_contexts`.
## Boundary
Help identities describe presentation context; they are not authorization
claims. Opening help never bypasses route or documentation permissions. Docs
owns documentation projection, feature modules own their content, and Core owns
focus capture, context resolution, and fallback routing.
+117 -5
View File
@@ -1,8 +1,31 @@
# DataGrid Sizing Contract
Auto-height grids reserve no empty vertical scrollbar gutter. The table fills
its card to the right edge; an actual constrained vertical scrollbar still
occupies its normal space. `Card bodyLayout="table"` provides an explicit
zero-inset surface, including with loading wrappers and padded notices. The
Organizations/IDM browser fixtures assert row geometry, not just outer shells.
The containing card removes its body padding; tables stay at `width: 100%` and
`max-width: 100%`. Do not cancel padding with negative margins or an expanded
`calc()` width: the table's maximum width correctly clamps that expansion, and
the result is a visible gap. Existing cards with only a DataGrid, admin table
surface, or connection tree inherit the same zero-inset geometry, directly or
through a `LoadingFrame`. Its loading overlay does not count as content and
stays within the table body. Mixed prose/form content keeps the usual insets;
choose `bodyLayout="table"` explicitly when combining a full-width table with
separately padded notices. Ordinary standalone tables retain their border.
Deutsch: Tabellenkarten entfernen den Innenabstand am Karteninhalt. Die Tabelle
bleibt bei 100 Prozent Breite; negative Ränder und verbreiternde `calc()`-Werte
sind nicht erforderlich. Das gilt auch während des Ladens. Karten mit Text
oder Formularen behalten ihren Innenabstand. Für eine randlose Tabelle neben
einem separat gepolsterten Hinweis ist `bodyLayout="table"` ausdrücklich zu
setzen; alleinstehende Tabellen behalten ihren Rahmen.
`DataGrid` turns every declared track into a deterministic pixel layout after
its container has a measurable width. The same contract is used on initial
layout, container resize, persisted-layout restore, and pointer resize.
layout, container resize, persisted-layout restore, and pointer/keyboard resize.
## Column Declarations
@@ -17,6 +40,77 @@ layout, container resize, persisted-layout restore, and pointer resize.
- `maxWidth` bounds direct user growth and free/constrained compensation. In a
cover layout it is a preferred maximum: passive tracks may exceed it when
that is necessary to keep the table flush with its container.
- `preferredMaxWidth` is a presentation-only automatic-fit ceiling. It retains
sensible initial proportions without limiting direct user resizing or
right-side resize compensation. A declared `maxWidth` still wins as the
manual limit. Cover may exceed preferred ceilings after other automatic
sizing targets are exhausted, preserving the no-blank-filler contract.
- Do not give expandable text columns arbitrary `maxWidth` values merely to
control their initial appearance. At that limit, a resize handle can shrink
but cannot grow, regardless of whether the next column is resizable. Prefer
weighted `minmax(..., Nfr)` declarations with `preferredMaxWidth` when needed;
ordinary cover-mode growth may
create horizontal overflow without changing fixed neighbors. Campaign's
Recipient(s), Delivery and configurable recipient-field columns follow this
contract. The existing signature format is unchanged for other columns;
introducing a preferred ceiling invalidates only that grid's previous width
contract, while sort/filter preferences remain intact.
- `columnType: "actions"` marks a custom action/control column. Canonical
`TableActionGroup` content is recognized automatically, even in existing
column declarations. Use `sticky: "end"` for the normal row-action surface.
## Action Visibility and Constrained Containers
Action tracks reserve the width of actual buttons, disabled-action wrappers,
reserved empty-state slots, gaps, and cell padding. A historic `width: 72`
preference therefore cannot clip a four-button action group. Ordinary data text
does not participate in this content measurement; long field values do not
silently widen all tracks. Changes to the rendered action set are remeasured.
`TableActionButton` remains a compact 36 px control, including the Add action
in an empty grid. Never stretch it with a last-column `.btn { width: 100% }`
rule. Its shared maximum width and fixed flex basis protect against broad
consumer button rules, which can otherwise feed stretched widths back into
action-track measurement and consume the data area.
When the full group needs more than half the scroll viewport, its measured
minimum is capped at half the viewport and the group wraps. Explicit hard
minima remain authoritative. Custom action groups should use wrapping-capable
flex layouts and semantic groups, preferably composing `TableActionGroup`.
The grid's physical width matches its pixel tracks, including horizontal
overflow, so right-sticky actions remain inside the correct scroll bounds.
If explicitly wide or persisted sticky tracks would obscure the readable data
area, horizontal stickiness is released until space returns. No columns or
actions are hidden: the labelled scroll region is focusable and supports native
keyboard scrolling. Vertical header stickiness remains available.
## Resizing Controls
Drag a resize handle with a mouse, pen, or touch pointer. Pointer capture keeps
the drag active when it leaves the handle. Escape or pointer cancellation
restores the layout before that drag; releasing the pointer commits it. Losing
window focus ends a drag without leaving the table stuck in resizing mode.
Each handle is a focusable vertical separator exposing its current and allowed
widths. Left/Right changes its width by 10 px; Shift+Left/Right uses 40 px. Enter
or a double-click resets that column's explicit override to the declared sizing
rules. Other columns retain their preferences, so cover/compensation constraints
still apply. These operations only change personal browser layout, never rows.
Deutsch: Spalten lassen sich mit Maus, Stift oder Touch ziehen. Escape verwirft
den laufenden Ziehvorgang. Am fokussierten Trenner ändern Links/Rechts die Breite
um 10 px, mit Umschalt um 40 px. Eingabe oder Doppelklick setzt die persönliche
Breite dieser Spalte zurück. Schmale Aktionenspalten umbrechen ihre Schaltflächen;
breite Tabellen bleiben horizontal scrollbar.
Eine feste Maximalbreite stoppt das Vergrößern unabhängig von Nachbarspalten.
`preferredMaxWidth` begrenzt dagegen nur die automatische Anfangsaufteilung,
nicht persönliche Breiten oder den Ausgleich beim Ziehen. Empfänger,
Zustellung und konfigurierte Empfängerfelder verwenden diese Darstellungsgrenze.
Bei Bedarf verbreitert sich die Tabelle und bleibt scrollbar; persönliche
Breiten ändern keine Kampagnendaten. Am rechten Scrollrand kann eine breite
linke Spalte wieder verkleinert werden, weil rechte Textspalten über ihre
bevorzugte Anfangsbreite hinaus Platz aufnehmen dürfen.
## Layout Modes
@@ -36,16 +130,25 @@ the column remains stopped until the pointer crosses the same boundary again.
## Persistence
Only the pixel layout resulting from an explicit user resize is persisted.
Only the pixel layout resulting from an explicit user resize is persisted,
together with the container width at which the user selected it.
Persisted widths are keyed by a signature containing column IDs, declared
widths and bounds, resize affordances, sticky placement, initial fit, and resize
widths and bounds, sort/filter/resize affordances, column type, sticky placement, initial fit, and resize
behavior. A changed signature discards the old override and recomputes the
declared layout.
Container reconciliation is suspended while a pointer drag is active. On
release, the already-rendered pixel layout becomes the persisted preference.
Reconciliation may grow it to prevent underflow, but never shrinks intentional
user overflow, so there is no drag-end snap.
Reconciliation at that same container width never shrinks intentional user
overflow, so there is no drag-end snap. If the surrounding layout later
contracts, persisted tracks may shrink toward their hard minima. The layout
retains only the amount of horizontal overflow deliberately created by the
user; an exact-cover layout therefore remains exact-cover at narrower widths.
Legacy snapshots from the former hard-pixel persistence contract are discarded
once and recomputed from the declared column layout. The current `v3` signature
also discards old snapshots that predate action and header-control minima;
sort/filter preferences remain intact. Measured action widths are not included
in the signature, so changing rows does not erase user sizing intent.
## Regression Matrix
@@ -56,6 +159,7 @@ user overflow, so there is no drag-end snap.
- hard-minimum horizontal overflow;
- fixed-only cover grids;
- persisted overrides under growth and viewport pressure;
- responsive contraction of persisted layouts without losing deliberate overflow;
- stale layout signatures;
- first and middle-column right-side compensation;
- last-resizable-column overflow, underflow stop, and reverse-pointer boundary;
@@ -65,3 +169,11 @@ user overflow, so there is no drag-end snap.
`webui/tests/data-grid-actions.test.tsx` also verifies the rendered fixed-cover
shape and guards against reintroducing a synthetic buffer cell.
`webui/conformance/tests/data-grid-layout.spec.ts` exercises the real rendered
grid with deliberately undersized action preferences, constrained containers,
horizontal scrolling, changing/empty action sets, keyboard and pointer resizing,
Escape cancellation, remount persistence, responsive contraction and restoration,
free/content mode, and constrained compensation. Run with
`npm run test:conformance -- data-grid-layout.spec.ts`; its isolated test server
is stopped automatically afterwards.
+91
View File
@@ -0,0 +1,91 @@
# Data-Subject Request Contract
This document defines the provider-neutral workflow for access and erasure
requests. It is an operational control and evidence mechanism. It does not
replace legal review, identity verification, retention policy, or the
institution's statutory response process.
## Ownership
Core owns the request aggregate, lifecycle API, optimistic concurrency,
provider discovery, export manifest, execution orchestration, and audit event
names. Modules that store subject-related data own their search, explanation,
retention, and mutation behavior through a `privacy.dsar.<module>` capability.
Core never scans module tables or guesses how a foreign resource may be
erased.
Access owns the first provider. It finds tenant memberships plus safe account,
identity, assignment, API-key, and session metadata. It does not export secret
hashes, session tokens, IP addresses, or browser fingerprints. Tenant-local
membership data can be anonymized and authentication material can be revoked.
Global accounts and identities require manual system-level review because they
may serve more than one tenant.
## Lifecycle
1. A privacy officer records a verified selector, purpose, legal basis, due
date, and internal reference.
2. Search invokes every available tenant capability independently. A provider
failure is isolated and recorded; it cannot turn an incomplete search into
a successful one.
3. The JSON export contains the request, records, provider runs, coverage,
retention reasons, execution evidence, and a SHA-256 manifest digest.
4. An erasure request produces stable provider-owned actions. Immutable
evidence generates an explicit non-executable `retain` decision.
5. Execution accepts only selected executable actions from the current plan.
It requires `If-Match`, the current resource revision, the dedicated erase
permission, and the exact `ERASE <request-id>` confirmation phrase.
6. Provider execution is idempotent. Completed or unchanged effects remain
durable in the request's execution evidence.
The API is rooted at
`/api/v1/admin/privacy/data-subject-requests`. Access exposes the independent
permissions `access:privacy:read`, `access:privacy:manage`,
`access:privacy:export`, and `access:privacy:erase`; the built-in privacy
officer role contains all four.
## Provider Rules
A provider must:
- enforce tenant ownership for every record and action;
- return stable, unique resource and action identities;
- avoid credentials, hashes, tokens, unnecessary telemetry, and unrelated
third-party data;
- distinguish mutable personal data from immutable institutional evidence;
- state a retention reason for immutable evidence;
- propose manual review instead of an automatic action when authority is
ambiguous or a resource spans tenants;
- return exactly one execution result per requested action;
- make execution idempotent and avoid committing the caller's transaction;
- keep all actual mutations inside the owning module.
Each active module without a DSAR provider is listed in coverage. This is a
deliberate fail-visible state, not proof that the module stores personal data.
An institution may call an export complete only after it has reviewed both the
provider runs and that coverage list.
## Retention And Evidence
Erasure and retention are separate decisions. Stable object IDs, authorization
history, function incumbency, formal decisions, delivery evidence, and audit
records may remain necessary for accountability. Providers expose those items
with a concrete reason and Core prevents them from being selected as executable
actions. Policy may further restrict an action, but it must never silently
loosen a provider's retention decision.
All lifecycle mutations and exports produce tenant audit events. The request
stores an evidence digest after every revision. This digest detects accidental
or unauthorized mutation of the aggregate; it is not a digital signature or a
substitute for signed recovery evidence.
## Current Limits
- Access is the first native provider. Other enabled modules appear in the
coverage list until they add a provider or an explicit no-subject-data
declaration is standardized.
- Verification of the requester's identity and statutory deadline escalation
remain institutional workflows outside this API.
- Global account or identity erasure is deliberately manual.
- Exports are JSON. A human-readable signed response package remains a later
Reporting/Templates integration.
+49 -2
View File
@@ -7,6 +7,16 @@ files.
## Runtime Configuration Contract
Worker and queue observability is provider-neutral. Runtime modules register a
bounded `RuntimeWorkStatusProviderRegistration` with Core; the Ops module
projects its sanitized status without importing Celery, Redis, or module job
implementations. Providers must use explicit `null` values for unsupported
queue depth, active/reserved work, failure count, and heartbeat evidence. An
unavailable metric must never be interpreted as zero or as proof of health.
The standard Core adapter reports the configured Celery/Redis runtime and
combines its bounded inspection result with registered worker heartbeat and
stale-threshold evidence.
Self-hosted installability follows the staged approach documented in
`SELF_HOSTED_INSTALLABILITY.md`: generate an explicit env template, validate it,
run production-like rehearsal with Compose-backed dependencies, then use the
@@ -64,6 +74,19 @@ Operator rule: take a database backup before applying migrations or destructive
module retirement. For non-SQLite databases, configure deployment-specific
backup/restore hooks for the module installer.
#### Ownership-history upgrade repair
Core revision `c58a2d7e9f10` repairs existing ownership-transfer tables that
predate the `decisions` column. Such installations can otherwise return HTTP
500 from `/api/v1/ownership/transfers`, including Campaign Settings. Apply the
normal forward migrations after taking a backup; do not stamp a revision or
recreate the table. The repair is available on both migration tracks, adds only
the missing non-null JSON column, and initializes old rows with an empty list.
It preserves owners, approvals, transfer states, revisions, timestamps, and any
existing decision history. Historical decisions are not reconstructed or
invented. Downgrading this repair retains the additive column and its evidence.
Verify that ownership-transfer listing and Campaign Settings load after upgrade.
### PostgreSQL Production Target
PostgreSQL is the primary development and production target. SQLite remains
@@ -278,12 +301,15 @@ through the same trusted address range.
| --- | --- |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_URL` or `GOVOPLAN_MODULE_PACKAGE_CATALOG` | Module package catalog source. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE` | Preferred production keyring path. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL` | Approved catalog channel, for example `stable`. |
| `GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS` | Comma-separated approved catalog channels, for example `stable`. The legacy singular name remains readable during migration. |
| `GOVOPLAN_LICENSE_TRUSTED_KEYS_FILE` | Trusted license issuer keyring path. |
| `GOVOPLAN_LICENSE_ENFORCEMENT` | Enables license enforcement when set to `true`. |
Trust roots are deployment-managed and should not be editable through the
running WebUI.
running WebUI. When no catalog override is configured, the Admin package
directory uses GovOPlaN's public stable catalog and the trust anchor bundled
with the installed Core release. Production operators may still pin a newer or
institution-specific catalog/keyring explicitly with the settings above.
### Mail Test Credentials
@@ -395,6 +421,27 @@ To stop PostgreSQL and Redis when the launcher exits:
GOVOPLAN_STOP_PROFILE_DEPENDENCIES_ON_EXIT=1 tools/launch/launch-production-like-dev.sh
```
## Development WebUI Dependency Caches
The application and browser-conformance harness share installed JavaScript
packages but must not share Vite's optimized-dependency cache. The application
uses `webui/node_modules/.vite/govoplan-app`; the conformance harness uses
`webui/node_modules/.vite/govoplan-conformance`. Keep these explicit sibling
directories when adding development or test configurations. Setting a different
Vite `root` alone does not isolate this cache.
A shared cache can make otherwise healthy Workflow, Dataflow or deferred editors
show “The resource could not be loaded.” The browser then reports an asset such
as `@xyflow_react.js` with HTTP 504 `Outdated Optimize Dep`, while the corresponding
API still returns HTTP 200. This is not a missing workflow permission or a reason
to rerun a pipeline. Preserve unsaved work, let the existing development server
reload the corrected configuration (or restart that WebUI server), then reload
the browser. Do not clear application data, change grants or restart delivery
workers to repair a frontend dependency cache.
Run `npm run test:vite-cache-isolation` in `govoplan-core/webui` to verify the real
resolved Vite configurations without starting servers or overwriting caches.
## Module Install/Uninstall Operations
Use Admin > System > Modules for planning. The running API server validates and
+11 -1
View File
@@ -17,6 +17,14 @@ operator, and roadmap pages.
| Action/effect automation layer | `ACTION_EFFECT_AUTOMATION_LAYER.md` | Action/effect contracts, consequence preview, runner semantics, and module boundary for automation. |
| External references and integration maturity | `EXTERNAL_REFERENCES_AND_INTEGRATION_MATURITY.md` | Stable external identity and cumulative connector maturity; configured source authority is defined by the meta target architecture. |
| Institutional context and governed references | `INSTITUTIONAL_CONTEXT_CONTRACT.md` | Shared temporal, actor/representation, institution, mandate, service, party, decision, evidence, legal-basis, information-governance, presentation, and geo DTO/provider contracts. |
| Provider-neutral record filing | `RECORDS_FILING_CONTRACT.md` | Exact source-revision identity, current source authorization, idempotent filing, capability discovery, and ownership boundary. |
| Ticket routing and Case escalation | `TICKET_INTEGRATION_CONTRACTS.md` | Optional fail-open routing, replay-safe Case handoff, authorization, evidence, and ownership boundaries. |
| Temporal data read context | `TEMPORAL_DATA_CONTEXT.md` | Valid-time and recorded-time titlebar selection, HTTP/cache contract, security boundary, and module-adoption rule. |
| Cross-module information governance adoption | `INFORMATION_GOVERNANCE_ADOPTION.md` | Manifest evidence and enforcement rules for temporal browsing, purpose-aware access, retention, and institutional context. |
| Data-subject access and erasure requests | `DATA_SUBJECT_REQUESTS.md` | Provider-owned search and mutation, explicit coverage, governed export, retained evidence, permissions, and idempotent execution. |
| Context-sensitive F1 help | `CONTEXTUAL_HELP_CONTRACT.md` | Focus, route, module-manifest documentation contexts, Docs projection, and hosted fallback. |
| Semantic documentation subjects | `SEMANTIC_DOCUMENTATION_SUBJECTS.md` | Stable configured-artifact identity, safe provider discovery, revision review, authorization, and lifecycle semantics. |
| German localization and help quality gate | `LOCALIZATION_AND_HELP_QUALITY.md` | German reference locale, new-installation default, catalog completeness, automatic page associations, and explicit-help review priorities. |
| Postbox E2EE target architecture | `POSTBOX_E2EE_ARCHITECTURE.md` | Strategic encrypted postbox/mailbox model, key ownership, role mailbox semantics, and retraction limits. |
| Shared state, runtime coordination, and recovery | `STATE_AND_RECOVERY_CONTRACT.md` | State profiles, object storage, node registration/drain, fenced leases, migration ordering, and recovery evidence. |
| Module lifecycle recovery | `MODULE_LIFECYCLE_RECOVERY.md` | Installer/live-graph recovery modes, deployment fence, evidence, retry blocking, and operator reconciliation. |
@@ -37,7 +45,9 @@ operator, and roadmap pages.
| Topic | Canonical document | Notes |
| --- | --- | --- |
| Product roadmap and module routing | `GOVOPLAN_MASTER_ROADMAP.md` | Product-level sequencing, implementation gates, issue routing, and missing-module decisions. |
| Institutional governance target | `govoplan/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md` | Cross-product semantic layers, source-authority modes, candidate Mandates/Services/Parties/Decisions boundaries, and migration sequence. |
| Stable platform ideas | `govoplan/docs/strategy/PLATFORM_CORE_IDEAS.md` | Cross-product thesis, canonical distinctions, product experience rule, maturity rule, and decision test. |
| Current cross-product reconciliation | `govoplan/docs/strategy/STRATEGY_STATUS.md` | The only current prose status source; generated evidence and Gitea remain authoritative inputs. |
| Institutional governance target | `govoplan/docs/architecture/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md` | Cross-product semantic layers, source-authority modes, candidate Mandates/Services/Parties/Decisions boundaries, and migration sequence. |
| UI/UX decisions | `UI_UX_DECISION_LEDGER.md` | Binding guided-UI decisions, open decisions, impact index, and review checklist. |
| Core interface migration | `INTERFACE_PATTERN_MIGRATION.md` | Core-owned settings, credential, retention, lifecycle, and shared-component evidence for the product pattern language. |
| Interface ethics and design doctrine | `INTERFACE_ETHICS_AND_DESIGN_DOCTRINE.md` | Product-level doctrine for context, decision, consequence, contestability, responsibility, and traceability. |
@@ -40,7 +40,7 @@ must not imply that GovOPlaN holds an authoritative copy.
Integration maturity states what an adapter is capable of doing. It does not
decide which system owns truth for a configured object or field group. A
binding separately selects one of the source-authority modes defined by the
[institutional governance target architecture](../../govoplan/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md):
[institutional governance target architecture](../../govoplan/docs/architecture/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md):
- `native_authoritative`
- `external_authoritative`
+1 -1
View File
@@ -13,4 +13,4 @@ tools/gitea/gitea-sync-wiki.py --help
Canonical documentation:
- `/mnt/DATA/git/govoplan/docs/GITEA_ISSUES.md`
- `/mnt/DATA/git/govoplan/docs/project/GITEA_ISSUES.md`
+2 -1
View File
@@ -221,7 +221,8 @@ Admin lists use bounded container grids:
- recipient import with column mapping;
- session/device revocation UI;
- backup/restore, monitoring, and update procedures;
- DSAR workflows and evidence bundle verifier;
- additional module providers and signed human-readable response packages for
the implemented DSAR workflow described in `DATA_SUBJECT_REQUESTS.md`;
- campaign ownership transfer workflow;
- policy impact analysis before delete/disable/unshare/change;
- LDAP/OIDC/SAML provisioning;
+3 -3
View File
@@ -10,15 +10,15 @@ gates. Issues are the active backlog; this document is durable architecture
planning context and should be mirrored to the Gitea wiki.
The meta repository's
[Connected Governance Platform Roadmap](https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/CONNECTED_GOVERNANCE_PLATFORM_ROADMAP.md)
[GovOPlaN Roadmap](https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/strategy/ROADMAP.md)
describes the corresponding cross-product stakeholder visions, configurable
service and operating configurations, connected outcome stories, and
capability horizons. The selected five-stage delivery sequence and its gates
are in the meta repository's
[Reference Journey Program](https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/REFERENCE_JOURNEY_PROGRAM.md).
[Reference Journey Program](https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/strategy/REFERENCE_JOURNEY_PROGRAM.md).
The semantic target, source-authority modes, and reconciliation with the
implemented platform are in the meta repository's
[Institutional Governance Target Architecture](https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
[Institutional Governance Target Architecture](https://git.add-ideas.de/GovOPlaN/govoplan/src/branch/main/docs/architecture/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
Those product documents are canonical; this Core roadmap remains their
technical sequencing and module-routing companion.
+131
View File
@@ -0,0 +1,131 @@
# Information Governance Adoption
## Platform Rule
Temporal browsing, purpose-aware access, retention, and institutional context
are platform-wide information-governance dimensions. Every module receives the
same contract by default. A module may claim `partial` or `enforced` only with
repository-owned object scope, evidence, and limitations; it may claim
`not_applicable` only when the dimension genuinely does not apply.
Historical business data is always authorized under the current security
state. No module may use a historical permission, membership, role, function
assignment, or policy projection to weaken present-day access.
Platform-wide adoption is tracked in
[GovOPlaN #40](https://git.add-ideas.de/GovOPlaN/govoplan/issues/40), with
temporal reads detailed in
[GovOPlaN #39](https://git.add-ideas.de/GovOPlaN/govoplan/issues/39).
## Manifest Declaration
`ModuleManifest.information_governance` publishes four dimensions:
- `temporal_browsing`;
- `purpose_aware_access`;
- `retention`;
- `institutional_context`.
Each dimension declares:
- adoption: `not_applicable`, `contract_only`, `partial`, or `enforced`;
- object types covered;
- repository-local test/documentation evidence;
- the remaining limitation for `contract_only` or `partial`.
The default is intentionally `contract_only`. It applies the platform rule
without pretending that existing domain queries and effects already enforce
it. `reference_ready`, `supported`, and `lts` modules cannot retain an
applicable dimension below `enforced`.
## Read Contract
For every persistent domain object, the owner classifies the read:
1. **Current-only:** historical semantics do not exist and the API says so.
2. **Valid-time:** select facts effective now or at the requested instant.
3. **Bitemporal:** additionally select only revisions known by `recorded_at`.
4. **All-validity:** return effective revisions in a bounded history view.
The Core temporal middleware supplies the request context. Owners apply it in
repositories or query helpers, include it in cache keys, return evaluated
context, and test current/at/all plus recorded-time boundaries. Search,
reporting, exports, selectors, counts, and drill-through must use the same
projection as the owning list/detail API.
## Purpose-Aware Access Contract
Permission establishes a technical action ceiling. Purpose-aware access asks
whether this actor, represented capacity, case/work item, legal basis, and
declared use may access this object now.
- A client-supplied purpose is an assertion, never authority by itself.
- The owner or Policy capability validates the purpose and returns explainable
provenance.
- Sensitive access can require case assignment, mandate, reason capture,
approval, or break-glass evidence.
- Search, selectors, reporting, exports, background jobs, and connectors apply
the same decision.
- Audit records the validated purpose identifier and decision reference, not
unnecessary content.
## Retention Contract
Every persistent object declares an owner, retention class or policy reference,
trigger, start instant, hold behavior, review/disposition action, and evidence.
Retention is not a generic timestamp deletion job.
- Domain owners enumerate and execute their own effects through a typed
retention provider.
- Policy resolves inherited ceilings and simulation.
- Records owns record disposition; Files owns byte/object effects; Audit owns
audit-detail behavior; external providers declare their own effect and
recovery semantics.
- Dry-run, legal hold, exact revision, idempotency, outcome unknown,
reconciliation, correction, and destruction evidence are mandatory for
consequential removal.
## Institutional Context Contract
Consequential objects and effects carry the relevant tenant, institution,
organization unit, function, mandate/jurisdiction, service/case/work item,
party/representation, decision, and record references. Context is minimized to
what the operation needs. Organizational membership is not itself permission
or mandate.
Events, automation intents, audit evidence, records, and external effects retain
the same governed context envelope or an exact reference to it. Consumers must
not reconstruct authority later from mutable current structures.
## Adoption Order
1. Inventory every domain list/detail/search/export/effect and classify all
four dimensions.
2. Migrate institutional owners first: Access, IDM, Organizations, Mandates,
Services, Parties, Cases, Approvals, Committee, Decisions, Voting, and
Records.
3. Migrate communication and content: Addresses, Distribution Lists, Campaign,
Postbox, Mail, Calendar, Files, Templates, and Forms Runtime.
4. Migrate data projections: Connectors, Datasources, Dataflow, Reporting,
Search, Risk Compliance, and Dashboard.
5. Migrate workflow/task/background/provider operations and prove that no
asynchronous path drops context.
6. Advance manifest claims only after owner tests and browser/reference-journey
evidence pass.
The generated platform inventory reports adoption counts and module details.
Gitea tracks individual migrations; the declaration is evidence and a maturity
gate, not a substitute for implementation.
## Definition Of Enforced
A dimension is `enforced` only when:
- all declared object types and public reads/effects use it;
- list/detail/count/search/export/worker behavior is consistent;
- cache and pagination semantics cannot cross contexts;
- absence, invalid values, and inaccessible referenced context fail safely;
- tests cover current, historical, unauthorized, replay, and module-absence
combinations appropriate to the dimension;
- user/admin documentation explains behavior and limitations;
- the manifest cites those tests and docs.
+3
View File
@@ -8,6 +8,9 @@ The shared contract lives in `govoplan_core.core.institutional`.
Core owns reference shapes and provider protocols only. It does not own shared
Mandate, Service, Party, Decision, evidence, or geography tables. Domain modules
own persistence and authorization; optional capabilities resolve the references.
Interactive reads use the separate platform temporal-data context documented in
`TEMPORAL_DATA_CONTEXT.md`; it never changes current authorization or supplies
mutation dates.
## Envelope
+101
View File
@@ -0,0 +1,101 @@
# Integrity-preserving performance contracts
These are implementation guarantees and regression-test boundaries, not a
security certification or production load-test result. Feature-specific policies
and help remain in the owning modules' English and German documentation topics.
## Refreshes, edits and table rendering
The shell runs at most one module-load refresh per authority generation. Multiple
invalidations coalesce into one trailing refresh; stale results and errors cannot
replace a newer generation. Focus refreshes are throttled to five seconds and
focus/visibility refreshes on hidden pages are suppressed; explicit module
invalidations still trigger a read. Authentication/tenant reset disposes the
previous controller. Authentication and authorization checks are not cached away.
Editable modules must reconcile a save against the submitted draft and accepted
server revision: edits made while a request is pending remain dirty. Completion
must be fenced by security-relevant authority and selection, not merely an auth
object's reference identity. A harmless profile refresh must not discard an
accepted newly created ID and invite a duplicate create. An old account's mutation
continuation must not reload its catalogue into the current account's page.
DataGrid precomputes first-occurrence row indices once for client sorting and
filtering. Duplicate object references, primitive values, `NaN` and sparse arrays
retain `Array.indexOf` behavior. Comparator order, visible pagination, server-side
pagination, sizing and resize rules are unchanged. Do not replace this with a
last-occurrence map or change ordering as a side effect of an optimization.
## Conditional responses
The shared JSON GET middleware performs route handling, including authorization,
before considering `If-None-Match`. It buffers only responses up to 1 MiB for a
body-derived ETag. Known larger responses bypass buffering; an unknown-length
stream crossing that limit replays its exact prefix and streams the remainder.
No data is truncated and no large joined copy is created. The crossing chunk is
already producer-owned: this is not a process-wide or route-output memory limit.
Empty chunks do not accumulate. Large responses may no longer receive a
middleware-generated ETag/304; explicit route ETags remain intact. Small-response
cache semantics and credential/language/context `Vary` fields are retained.
## Shared helpers and concurrency
Central helpers replace exact live-code duplicates only. Actor precedence,
whitespace handling and service-account differences remain explicit owner choices;
historical migration code is not redirected to mutable runtime helpers. Connector
search ACL token projection keeps its first-seen ordering and existing 500-token
cap, stopping work once that cap is reached. Provider schemas are inferred in one
pass without storing a second list of every column value.
The keyed-list three-way merge retains insertion anchors when unrelated fields
change. Concurrent additions use deterministic ordering; contradictory anchors
produce a collection-order conflict instead of silently relocating an item.
No existing endpoint is newly opted into merge behavior by this change.
SQL JSON authorization predicates support the explicitly tested SQLite and
PostgreSQL dialects, retain exact string membership and reject unsupported
dialects. Apply tenant and authorization predicates before counting/pagination;
never page a broader result first and filter away unauthorized records afterward.
SQLite execution and PostgreSQL SQL compilation are not substitutes for a
deployment's PostgreSQL concurrency and representative-data load tests.
## Migration connection URLs
Alembic preserves the configured database URL exactly in online and offline
migration modes, including percent-encoded credentials and PostgreSQL Unix-socket
paths. Escaping applies only at its ConfigParser boundary; operators must not
double-escape `%` in `DATABASE_URL` or alter working connection credentials to
work around interpolation errors. This does not change the target database,
authentication, TLS policy, or migration contents.
## Deutsch: Integrität vor Geschwindigkeit
Der zentrale Modul-Refresh bündelt gleichzeitige Auslöser und verwirft veraltete
Ergebnisse einschließlich Fehlermeldungen. Ein Wechsel von Anmeldung oder Mandant
beendet die bisherige Generation. Fokusaktualisierungen sind auf einen Auslöser
je fünf Sekunden begrenzt. Berechtigungsprüfungen bleiben erhalten.
Speicherantworten dürfen zwischenzeitliche Bearbeitungen nicht überschreiben.
Ein unveränderter Berechtigungskontext mit einem neuen Profilobjekt darf eine
bereits bestätigte neue ID oder Revision nicht verwerfen. Umgekehrt dürfen alte
Anfragen nach einem Kontowechsel keine Daten in den neuen Kontext übernehmen.
Die DataGrid-Optimierung erhält Reihenfolge, Filter-, Seiten- und Größenverhalten.
Die ETag-Middleware puffert höchstens 1 MiB Nutzdaten zuzüglich eines bereits vom
Erzeuger gelieferten Grenz-Chunks. Größere Antworten werden vollständig weitergereicht,
nicht abgeschnitten; automatisch erzeugte ETags können dabei entfallen.
Autorisierung läuft auch bei bedingten Anfragen. Das ist keine allgemeine
Speicherbegrenzung für Routen oder Prozesse.
Gemeinsame Helfer erhalten die bisherigen fachlichen Unterschiede. Listen-Merges
bewahren Einfügepositionen oder melden widersprüchliche Reihenfolgen explizit als
Konflikt. Datenbankseitige Autorisierung erfolgt vor Zählung und Seitenauswahl.
Regressionstests belegen diese Verträge; reale Provider-, PostgreSQL- und Lasttests
in einer repräsentativen Umgebung bleiben Teil der Betriebsfreigabe.
Alembic übernimmt die konfigurierte Datenbank-URL in Online- und Offline-Läufen
unverändert, einschließlich prozentkodierter Zugangsdaten und PostgreSQL-
Unix-Socket-Pfade. Die Maskierung erfolgt ausschließlich an der ConfigParser-
Grenze; `%` in `DATABASE_URL` nicht doppelt maskieren und funktionierende
Zugangsdaten nicht als Umgehung ändern. Zieldatenbank, Anmeldung, TLS-Vorgaben und
Migrationsinhalte bleiben unverändert.
+63 -1
View File
@@ -13,10 +13,60 @@ domain modules own their compositions.
| Reusable credentials | Repeated administration with an adaptive create/edit dialog, optional password generator, and destructive confirmation | Secret values are write-only; generated candidates use the browser cryptographic API without a weak fallback and do not replace the field until explicitly confirmed; scope/permission blockers name the required action, responsible actor, and destination; unavailable row actions remain keyboard-explainable | `CredentialEnvelopeManager.tsx`, shared `PasswordField`, `PasswordGeneratorDialog`, `ActionBlockerHint`, `Button`, `TableActionGroup`, and `ConfirmDialog` |
| Retention policy | Effective-policy editor with inherited source paths and typed, narrowing-only controls | Parent locks and missing write authority are explicit; the save action distinguishes locks, missing target, loading, clean draft, and active save | `RetentionPolicyManagement.tsx`, policy logic tests, `test-core-interface-patterns.mjs` |
| Module lifecycle | Guided operator projection over durable installer-queue evidence | Preflight, handoff, progress, stale evidence, recovery, and rollback consequences remain visible | Admin module lifecycle tests and the Core installer-queue contract |
| Shared configuration primitives | Cross-module component contract | Dialog focus, blocker structure, disabled-action focus, contextual help, unsaved changes, confirmation, loading, alerts, problem lists, and policy provenance are centralized | Core component tests and module-permutation build |
| Shared page frame | Domain-neutral headed page layout used by Core and optional modules | Standalone, workspace, and embedded modes make inset and scroll ownership explicit; sticky heading, rich descriptions, route actions, notices, loading, narrow-layout collapse, and contextual-help identity are centralized; composite administration workspaces may delegate the visible heading to their contributed panel while retaining the same frame; `AdminPageLayout` composes the contract | `PageLayout.tsx`, `page-layout.test.tsx`, Core Settings, Access administration, Docs, Mail bounce processing, Dashboard, Ops, Campaign, and `check-shared-webui-layouts.py` |
| Full-canvas workspace | Navigation/content and list/detail canvases that own pane geometry and scrolling | Navigation and split variants, primary-pane width, pane-owned or contained scrolling, responsive stacking or navigation collapse, pane labels, and contextual-help identity are centralized without encoding domain navigation | `WorkspaceLayout.tsx`, `workspace-layout.test.tsx`, Core Settings, Access administration, Docs, Organizations, Campaign, Templates, Approvals, and `check-shared-webui-layouts.py`; the raw-workspace exception baseline is empty |
| Full-height module frame | Outer module landmark and viewport/container sizing | `WorkspaceFrame` centralizes surface, overflow, box sizing, accessible naming, help identity, and application-viewport height so modules do not copy the `100vh - shell` frame | `WorkspaceFrame.tsx`, `layout-primitives.test.tsx`, Dataflow, Workflow, Datasources, Distribution Lists, Notifications, Tasks, Scheduling, Forms, Portal, Projects, Records, and Reporting |
| Responsive action toolbar | Domain-neutral action and filter grouping for pages, workspaces, editors, and overlays | Density, surface, grouping, flexible space, accessible naming, toolbar help identity, and responsive wrapping are centralized while modules retain action wording, authority, and consequence | `ActionToolbar.tsx`, `layout-primitives.test.tsx`, WYSIWYG, Calendar, Files, Forms, Templates, and the product-wide primitive check |
| Semantic page and pane action bars | Overview, collection, detail, editor, and workspace intent declared independently from frame geometry; full-canvas panes add workspace/collection/detail/editor scope | Core renders guarded Reload in the right-aligned group immediately before Create/primary actions; editor persistence owns clean, dirty, invalid, saving, failed, and conflict feedback plus guarded Discard and far-right Save; destructive actions occupy an explicit named boundary; read-only surfaces do not invent Save | `PageActionBar.tsx`, `WorkspaceActionBar.tsx`, `PAGE_LAYOUT_USAGE_GUIDELINES.md`, component and browser conformance, every headed page and full-canvas workspace, and the discovery-based `check-shared-webui-layouts.py` |
| Catalogue and state composition | Search/filter bars, selectable navigation lists, count badges, and empty/blocked/error panels | Width, surface, wrap, selection geometry, title/description truncation, numeric emphasis, state sizing, tone and action placement are centralized; modules retain query behavior, object state and consequences | `FilterBar.tsx`, `SelectionList.tsx`, `CountBadge.tsx`, `StatePanel.tsx`, `layout-primitives.test.tsx`, and list/detail modules across Cases, Committee, Dataflow, Forms, Notifications, Portal, Postbox, Projects, Records, Reporting, Tasks, Templates, and Workflow |
| Content and form grids | Equal-column content, field, and native-form geometry | Explicit 14 columns, standard gaps, item spans, alignment, and named narrow/workspace/standard/wide collapse points replace generic and module-prefixed copies; unequal domain tracks remain local | `ContentGrid.tsx`, `layout-primitives.test.tsx`, Core dashboard/settings/mail, Calendar dialogs, Forms editor, Datasources, Postbox, Campaign, administration surfaces, and the product-wide primitive check |
| Content sections | Repeated editor/detail section surfaces | Border, surface, compact/default density, stacked flow and block rhythm are centralized without encoding section contents | `ContentSection.tsx`, `layout-primitives.test.tsx`, Datasources, Distribution Lists, Templates, Dataflow, and Workflow |
| Form sections | Reusable heading/description/action/content grouping inside forms | Plain, separated, and panel variants centralize hierarchy and narrow action placement without moving validation, permissions, values, or domain wording into Core | `FormSection.tsx`, `layout-primitives.test.tsx`, Addresses contact editing, and Quick Access preferences |
| Metric groups and drill-downs | Reusable responsive grouping around metric cards with an explicit optional detail affordance | Fixed one-to-five and auto-fit columns, minimum card widths, density, block/inset/zero spacing, and named collapse points replace the product-wide `metric-grid` class and cross-module dashboard overrides; typed link or in-page drill-downs name their destination while summary-only, non-enumerable, derived, or privacy-suppressed values remain inert | `MetricGrid.tsx`, `MetricCard.tsx`, `metric-card.test.tsx`, `layout-primitives.test.tsx`, Core and Dashboard summaries, administration, Campaign, Ops, Files, Search, and dashboard widgets |
| Description lists | Semantic property and fact presentation | Stacked and inline variants, one-to-five list columns, density, term width, wrapping, and responsive collapse replace both `admin-details-grid` and `detail-list`; `DescriptionItem` preserves native `dt`/`dd` anatomy | `DescriptionList.tsx`, `layout-primitives.test.tsx`, Access and Tenancy administration, Audit, Policy, Campaign reports/imports, Docs, Settings, Ops, and Reporting |
| Dialog anatomy | Shared outer dialog plus composable body and footer regions | Size and administration variants, body padding, descriptions, notices, fixed action wrapping, native form flow, and section grouping are centralized; focus trapping and stack lifecycle remain unchanged | `Dialog.tsx`, `DialogAnatomy.tsx`, `dialog-focus.test.tsx`, `layout-primitives.test.tsx`, Addresses, Calendar, Records, Datasources, Distribution Lists, Files, and Templates |
| Definition-editor visuals | Reusable graph palette, canvas chrome, node icon/port geometry, empty overlay and floating activity state | Core owns visual and responsive anatomy while node/edge types, validation, execution, provenance and workflow semantics remain in Dataflow or Workflow | `DefinitionPalette.tsx`, `DefinitionNodeIcon.tsx`, `FloatingStatus.tsx`, shared definition styles, Dataflow and Workflow structure/build checks |
| Shared configuration primitives | Cross-module component contract | Dialog focus, blocker structure, disabled-action focus, route/page/field/action F1 help, unsaved changes, confirmation, loading, alerts, problem lists, and policy provenance are centralized | Core component tests, `CONTEXTUAL_HELP_CONTRACT.md`, and module-permutation build |
| Measured operation feedback | `LoadingFrame` over existing content | Use `indicator="none"` with native measured progress for long-running operations; `progress={null}` means unknown, never a synthetic percentage. Keep dialog content inert and close controls disabled until success or error. Existing consumers retain their loading indicator. | Files archive inspection/extraction, layout primitive tests, managed-archive browser conformance |
## Boundary
Side-rail customization uses the shared `NavigationPreferenceEditor` for system,
tenant, personal, and View layouts. Modules and labelled separators share one
ordered list, with pointer drag-and-drop, keyboard reordering, and explicit
add/remove actions. Consumers retain persistence and dirty-state ownership;
mounting the editor does not create a draft change. See
`NAVIGATION_LAYOUT_CONTRACT.md` for inheritance, locked items, optional-module
preservation, and collapsed-rail grouping.
Action columns use `TableActionGroup` or declare `columnType: "actions"` when
their composition differs. DataGrid owns measured action minima, initial column
allocation, persistent resizing, and local horizontal scrolling; consumers must
not compensate with clipped overflow or copied fixed widths. See
`DATAGRID_SIZING_CONTRACT.md`. Dialog forms use `DialogForm` and `FormGrid` inside
the shared size-bounded dialog. Do not add a content minimum wider than the
panel's padded interior. Genuinely wide content, such as a table, owns its own
local scroller instead of making the entire dialog scroll horizontally.
In `FormGrid` and `FormLayout`, direct `FormField` and `ToggleSwitch` items
align their controls at the row's lower edge. A single control inside a
`GridItem` follows the same rule. Labels may wrap without shifting adjacent
switches up into the label row. Do not add per-module top margins or empty
labels; single-column layouts must not retain a phantom label spacer.
Credential editors resolve public reference labels when opened. A failed
save displays its error inside the dialog and keeps the entered draft for an
explicit retry. While a write is pending, repeated submission, edits, and
dialog dismissal are disabled; no configured secret is read back from storage.
The shared rich-text editor emits content changes only for actual document
edits. Mounting, read-only changes, loading a saved value, and switching between
visual and source inspection must preserve the controlled HTML without marking
the owning page dirty. This is especially important for legacy Campaign HTML:
merely visiting Template must not normalize it or require a save on leaving.
The WYSIWYG lifecycle browser conformance covers both visual and legacy-source
initial content, as well as genuine typing.
Files and Mail are the first two external consumers of the layered
server/credential/policy pattern. Their own repositories retain provider
discovery, transport behavior, authorization, and migration evidence. Remaining
@@ -26,3 +76,15 @@ they are not reasons to add sibling-private behavior to Core.
Raw JSON remains permitted only for diagnostics, expert inspection,
interchange, or conflict evidence. It is not a primary Core configuration
editor.
New headed pages use `PageLayout`; full-canvas modules use `WorkspaceFrame`
and, where applicable, `WorkspaceLayout`, so Core owns the frame and pane
scrolling. Module CSS continues to own unequal domain content layout, never the
shared page, workspace, toolbar, state, list, filter, metric, section, or graph
chrome. Retired copies and module-local component definitions are rejected by
`check-shared-webui-primitives.py`. That check also requires standard dialog
widths to use `Dialog size` and keeps every remaining domain-specific width in
a reviewed, decrease-only exception baseline. The companion layout check now
has zero raw page-frame and zero raw workspace exceptions, discovers semantic
consumers without a hand-maintained route list, requires semantic action bars
on `WorkspaceFrame` routes, and rejects ad-hoc panel-header toolbars.
+151
View File
@@ -0,0 +1,151 @@
# Localization And Contextual Help Quality
## Reference Language
German (`de`) is GovOPlaN's first-class reference target. Every translation key
used by a shipped WebUI must exist in German and English. German completeness is
a release gate; English remains the source-code fallback language so existing
literal labels and external developer APIs do not change semantics.
New installations and tenants default to German. Existing system, tenant, and
user preferences are preserved. The available-language and policy model can
still select another default or disable a package at the relevant scope.
Explicit high-risk help content and browser acceptance are tracked in
[Core #284](https://git.add-ideas.de/GovOPlaN/govoplan-core/issues/284).
The platform inventory recognizes both inline locale objects and generated
catalogs declared as `const de` / `const en`. Its strict mode requires both
locales and reports `de` explicitly as the reference locale.
## Structured Documentation Localization
`DocumentationTopic.translations` continues to own localized title, summary,
and body prose. Topics whose metadata contains rendered prose opt into the
separate `structured_translation_version="1"` contract and provide a complete
same-shape value for each translated metadata key in
`structured_translations`. Version 1 covers workflow prerequisites, steps,
outcome, result and verification; reference fields; limitations, constraints,
consequences and consequence classes; and the other rendered explanation
fields declared by Core.
The registry rejects an unversioned translation, an unsupported contract
version, missing structured keys, changed object keys or list lengths, empty
translated strings, and changed non-text values. Stable field IDs, routes,
permission scopes, and other technical leaves therefore remain structurally
bound to the source metadata. The Docs module overlays only a validated locale
at response time and reports the selected structured locale separately from the
title/body locale. Missing structured translations fall back to source content
and remain visible in public coverage until the owning module adopts the
contract.
## Help Resolution
Every focusable field and action receives a stable derived F1 identity from the
shared shell, even when the component has no dedicated help text. Resolution
falls back from field/action to dialog or page and then to the module's visible
documentation baseline.
Backend manifests publish explicit topic associations first. Core additionally
associates declared route, navigation, settings, and View surface IDs with the
module's static user or administrator documentation baseline. Feature modules
should still add exact `metadata.help_contexts` entries for consequential,
unfamiliar, policy-controlled, destructive, security-sensitive, or legally
meaningful fields and actions.
The shared retention-policy editor exposes explicit contexts for each stored
data category, audit-detail control, lower-level override switch, target
selector, reload, and save action. The Policy module owns the matching German
administrator guidance. Retention execution surfaces use separate contexts for
dry-run, destructive apply, confirmation, and outcome review so F1 opens the
consequence and recovery guidance closest to the focused control.
Shared controls may set `helpModuleId` when their documentation owner differs
from the containing page; the retention editor uses this to resolve Policy help
from both administration and Campaign surfaces.
The shared reusable-credential manager keeps Access as its documentation owner
and publishes exact contexts for credential kind, secret replacement/removal,
module and server restrictions, lower-scope visibility, activation, save, and
irreversible deletion. This ensures F1 explains secret custody and the effect on
dependent connections from system, tenant, group, user, and personal surfaces.
The source inventory treats literal `helpContextId` and
`data-help-context-id` declarations as authored help associations, including a
native control nested in `FormField`. Dynamic context expressions remain
separate evidence and generic derived fallbacks remain in the richer-help
candidate queue.
The same inventory classifies controls whose labels, identities, component
context, or explicit `data-help-risk` indicate authority, credentials,
disclosure, encryption, external effects, irreversible changes, policy, or
retention. These controls require an exact context rather than relying only on
page fallback. Reviewed false positives carry
`data-help-risk-reviewed="standard"`. Invalid risk classes and any increase
above the versioned `tools/inventory/high-risk-help-baseline.json` ceiling fail
strict declaration checks; the ceiling is lowered as the finite queue is
resolved. Password fields and their generator dialog propagate the owning
field's context so shared credential controls never invent a Core-owned topic.
The generated `help_review_candidates` list is therefore a content-depth queue,
not a list of controls on which F1 cannot work. It should prioritize:
1. effect, deletion, delivery, retention, disclosure, encryption, and recovery;
2. identity, representation, mandate, institutional context, and purpose;
3. valid-time versus recorded-time selection;
4. provider authority, synchronization, conflict, and outcome unknown;
5. fields whose consequences are not evident from their label.
The shared browser conformance journey mounts the production Help menu and
resolver. It proves that F1 uses the focused control rather than only the page,
maps an exact retention action to Policy-owned administrator documentation,
retains the page context as fallback for derived actions, exposes an accessible
modal at narrow widths, closes with Escape, and restores focus to the triggering
control. Module journeys should add their own exact high-risk mappings; they do
not need to reimplement the keyboard or dialog mechanics.
The same conformance suite mounts the production Forms Runtime self-service and
assisted Anwohnerparkausweis surfaces with German module translations. Desktop
and mobile runs traverse native controls by keyboard, inspect accessible names
and landmarks, run WCAG 2.1 A/AA automation, verify responsive overflow, and
retain independent per-field assisted provenance. Physical assistive-technology
spot checks remain release evidence rather than being represented as browser
automation.
## Verification
```bash
cd /mnt/DATA/git/govoplan
/mnt/DATA/git/govoplan/.venv/bin/python \
tools/inventory/platform-interface-inventory.py \
--strict --strict-declarations --strict-endpoints
```
The check must report:
- reference locale `de` present and complete;
- no used key missing from `de` or `en`;
- every field has a resolvable F1 context;
- no duplicate stable IDs;
- no undeclared public WebUI surface;
- no stale runtime route or endpoint declaration.
- no invalid high-risk help annotation or regression above the recorded
exact-context debt ceiling.
Browser acceptance is part of the focused workspace gate and can be run alone:
```bash
cd /mnt/DATA/git/govoplan-core/webui
npm run test:conformance
```
For shared component regressions, `npm run test:components -- page-layout
documentation-help layout-primitives` compiles the component test configuration
once and executes the selected suites. With no suite names it runs every shared
component suite. Existing single-suite aliases such as `npm run test:page-layout`
remain supported. Every invocation owns an ignored, uniquely named temporary
output directory and removes it on completion, failure, or handled interruption;
parallel invocations cannot delete each other's compiled tests. A forcibly killed
process may leave its own ignored directory behind. The launcher itself is
checked with `npm run test:component-runner` without compiling the application or
starting a server. Browser conformance remains a separate verification layer;
passing component tests does not constitute a complete module review.
+109 -3
View File
@@ -16,7 +16,7 @@ The experimental remote WebUI bundle loading design is tracked in
The cross-product semantic layers, source-authority modes, and candidate
Mandates, Services, Parties, and Decisions boundaries are canonical in the
meta repository's
[`INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md`](../../govoplan/docs/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
[`INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md`](../../govoplan/docs/architecture/INSTITUTIONAL_GOVERNANCE_TARGET_ARCHITECTURE.md).
## Layer Model
@@ -128,6 +128,8 @@ The following contracts are the baseline API that modules can rely on:
- bounded reference-option search provider contract
- single-tenant and optional batched tenant summary provider contracts
- tenant delete-veto provider contract
- provider-neutral tenant-erasure preview, step, idempotency, and
reconciliation contracts in `govoplan_core.core.tenant_erasure`
- WebUI module contribution contract
- navigation metadata contract
- command/event envelope contract
@@ -149,6 +151,17 @@ Destructive tenant lifecycle planning deliberately continues to use the
single-tenant path so it invokes every registered provider for the target
tenant, independent of ordinary list-page projections.
Governed populated-tenant erasure is separate from ordinary delete vetoes.
Modules contribute `tenancy.erasure_provider.<module_id>` capabilities with a
bounded resource inventory, explicit erase/retain/legal-hold/external/key/
backup dispositions, ordered destructive warnings, idempotent step execution,
and reconciliation. The collector fails closed when a provider is invalid or
fails. A module with nonzero tenant summary counts and no erasure capability is
reported as unsupported and blocks execution; modules with neither contract
are explicitly projected as outside tenant-persistence scope. Provider
evidence contains counts and stable references only and must never contain
secrets or erased subject data.
This list is the Milestone A kernel-contract freeze baseline. New module work
may extend the kernel by adding explicit contracts, but existing contracts must
remain source-compatible through the 0.1.x split line unless a migration shim
@@ -213,8 +226,29 @@ Other stable runtime capabilities currently include:
`calendar.externalProfiles`
- `poll.scheduling`
- `notifications.dispatch`
- `application_status.projection`
- `payments.requests`
- `workflow.definitionContributions` and `workflow.runtimeWorker`
`calendar.scheduling` keeps workflow modules independent of Calendar-owned
models and transport adapters. Consumers may create tentative events, promote
the selected event in place, and release unused events idempotently. The
provider returns bounded external-delivery and outbox references so consumers
can retain retry state without copying Calendar's synchronization internals.
`application_status.projection` lets a presentation module resolve the tenant
and display or request access to an owner-supplied, deliberately bounded
applicant-status view. The provider retains policy, authorization, token, and
record ownership; consumers must not query provider tables or enlarge the
projection.
`payments.requests` carries replay-safe payment obligations and evidence-bound
manual reconciliation across module boundaries. Procedure modules identify the
source Case or Workflow in the command and retain the returned payment ID;
Payments remains authoritative for amount, currency, state, transaction
reference, and reconciliation evidence. Ledger, invoice, and external payment
providers remain separate follow-on contracts.
The provider-neutral `idm.relationships` contract carries tenant-scoped typed
groups, effective-dated identity relationships, and explicit membership
decisions. It deliberately does not expose IDM persistence models or imply an
@@ -305,6 +339,8 @@ contract checks, are:
- `files.access`, `files.campaign_attachments`
- `mail.campaign_delivery`
- `notifications.dispatch`
- `application_status.projection`
- `payments.requests`
- `poll.availability_matrix`, `poll.option_selection`,
`poll.response_collection`, `poll.signed_participation`,
`poll.workflow_context`
@@ -1007,6 +1043,49 @@ Any future exception is extraction debt and must be temporary, documented in the
script with a reason, and removed when a capability/API/event contract replaces
it.
## Product Surface Contributions
`FrontendModule.product_surfaces` is the versioned product-composition contract
for stable identities that may have one or more technical owners. A contribution
declares contract version 1, a product identity, common label/icon/description,
stable entry path, owner route and View surfaces, supported task/reader/admin/
operator presentations, authorization requirements, capabilities, search
sources, help contexts, documentation topics, migration aliases, and standard
unavailable/degraded explanations.
Core validates every reference against the owning manifest. Contributors that
share an identity must agree on its common product metadata and entry path;
entry and alias paths cannot belong to another product identity. The WebUI
composes valid owners by product id, filters them through authorization and the
effective View, and resolves the stable entry or migration alias to the first
available owner route. It emits `govoplan:product-surface-route-resolved` before
the redirect so migration telemetry can observe alias use without making the
technical module part of the ordinary label.
The shell projects every authorized, View-visible owner route with a product
contribution into one stable product navigation item. The product label and
entry path replace package topology in the primary rail; every contributing
owner path still marks that item active. `All available tools` is a collapsed,
permission-derived catalogue built independently of the active View, so a
focused workflow cannot remove the explicit escape. It may reveal an
authorized owner route that a View omitted, but never an unauthorized route.
Navigation visibility preferences do not delete catalogue entries, and the
original owner routes remain compatible deep links.
The initial promoted destinations are `work.items` at `/work`,
`meetings.calendar` at `/agenda`, `communication.messages` at `/messages`
(with `/inbox` as an alias), and `records.files` at `/documents`. Their labels
and availability language are centralized in Core while Tasks, Calendar,
Mail/Postbox, and Files retain route, command, search, help, documentation,
authorization, and data ownership.
Use `ProductAvailabilityState` for unavailable and degraded outcomes. The
ordinary state explains the attempted outcome, consequence, recovery path and
responsible role. Exact module, capability, provider and correlation values may
be supplied as a collapsed technical detail; they are not the primary error.
The state is presentation only and never grants authority or changes provider
health.
## Boundary Decision Register
These durable decisions close older exploratory core issues. Implementation
@@ -1377,8 +1456,10 @@ The package install-plan API records operator intent only:
- `GET /api/v1/admin/system/modules/package-catalog` reads approved package
references from `GOVOPLAN_MODULE_PACKAGE_CATALOG` so operators can add known
module refs to the install plan without typing them manually. The endpoint
also reports catalog validity, channel, signature, trust state, and the
configured path.
also reports catalog validity, channel, signature, trust state, source and
artifact provenance, release availability, configuration requirements, and
per-entry compatibility/blocker state. Withdrawn entries are visible for
diagnosis but cannot be planned.
- `POST /api/v1/admin/system/modules/install-plan/catalog/{module_id}` saves
a planned install or update row from a validated catalog entry. Installed
modules are planned as updates. Catalog signature and approved-channel policy
@@ -1476,6 +1557,10 @@ The installer preflight is intentionally conservative:
- the `shared` state profile blocks in-place package mutation; clustered
installations must roll one verified immutable module composition across all
replicas;
- official runtime images carry the full verified package profile, while the
desired module graph controls activation and tenant/View/Policy contracts
control availability and presentation; package lifecycle must not be reused
as a tenant or user visibility switch;
- installed module manifests must be compatible with the supported manifest
contract and current core version;
- uninstalling `tenancy`, `access`, or `admin` is blocked;
@@ -1550,6 +1635,16 @@ Unsigned/unhashed remote bundles are skipped. This keeps remote loading a
controlled deployment option rather than a replacement for release package
builds.
A failed local WebUI package import receives one automatic retry after 250 ms.
Descriptor validation still fails closed; it is not bypassed by the retry.
If an enabled local module still cannot load, the signed-in shell warns that its
screens and integrations may be unavailable and identifies the module. This is
a loading failure, not an uninstall. Save other drafts before manually reloading
the page; there is no automatic page reload or persistent retry loop. A verified
remote fallback that successfully loads clears that module's warning. Packages
absent from the optional build graph remain absent, and effective View filtering
continues to control which loaded UI capabilities are exposed.
## Maintenance Mode
Maintenance mode is the required operating state for package install/uninstall
@@ -1592,6 +1687,17 @@ URLs never contain credentials; only credential-envelope references cross the
contract. Provider-specific details belong in sanitized provenance rather than
in a shared domain schema.
## Semantic Documentation Subject Contract
Optional modules expose configured artifacts that can be documented through
the module-scoped `documentation.semantic_subjects.<module_id>` capability.
Core supplies stable tenant-scoped references, typed nested anchors, safe
localized descriptors, revision/fingerprint review signals, and explicit
availability states. Providers remain responsible for authorization and do not
expose configuration payloads or credentials. Docs discovers the capability
and owns authored content; it does not import feature internals. See
`SEMANTIC_DOCUMENTATION_SUBJECTS.md` for the contract and adoption rules.
## Build And Verification
Backend verification from core:
+96
View File
@@ -0,0 +1,96 @@
# Shared navigation layout contract
Core owns `NavigationPreferenceEditor`, ordered layout resolution, and rail
rendering. Admin, Tenancy, personal Settings and Views reuse this editor. They
own loading, authorization, Save, Reload and dirty-state guards; the editor
emits a draft only after a real edit. A drag onto the same position, keyboard
pickup/drop without movement, and opening inherited settings do not save or
create an override.
## Standard layout / Standardanordnung
No configuration is required to enable sections. Installed modules contribute
the standard product-area membership, labels, and order: Work; Services and
cases; Records and documents; Communication; Meetings and decisions; Data and
assurance; People and responsibility. The dashboard stays first; unclassified
destinations remain available under More tools. Only nonempty, authorized and
View-visible groups appear. Composed product entries keep their placement
owner's directly declared area. If none exists, the navigation aliases of
other authorized owners preserve area membership. These are live defaults,
not a saved copy that needs replacing whenever optional modules change.
The editor explains whether the current level is inherited, custom grouped, or
explicitly flat. To restore defaults, choose **Use inherited layout** and Save
on the owning page: personal Settings inherits the tenant, tenant settings
inherit the system, and system settings inherit the standard product areas.
An explicitly flat personal, tenant, system, or View configuration is never
silently replaced with standard groups. A View may also explicitly select flat
navigation. Reset the responsible override rather than changing permissions.
Abschnitte sind ohne zusätzliche Konfiguration aktiv. Die Standardanordnung
gliedert verfügbare Module in Arbeit; Leistungen und Vorgänge; Akten und
Dokumente; Kommunikation; Termine und Entscheidungen; Daten und
Qualitätssicherung; Personen und Verantwortung. Das Dashboard steht davor,
nicht zugeordnete Ziele bleiben unter Weitere Werkzeuge erreichbar. Leere oder
nicht zugängliche Gruppen erscheinen nicht. Der gemeinsame Editor zeigt an,
ob eine geerbte, eigene gruppierte oder ausdrücklich ungegliederte Anordnung
vorliegt. **Geerbte Anordnung verwenden** und anschließendes Speichern entfernt
nur die Anpassung dieser Ebene: persönlich → Mandant → System →
Standardanordnung. Bewusst gespeicherte ungegliederte Anordnungen bleiben
erhalten. Diese Darstellung erteilt keine Berechtigungen und ändert keine Daten.
## Stored document and precedence
The version-1 navigation document retains `order`, `hidden` and `locked` and
adds optional `separators`, each containing a stable `separator:`-prefixed ID
and an optional plain-text label of at most 120 characters. Separator IDs and
module navigation IDs occupy the same `order` list. Separators are presentation
metadata and never become routes, modules, permissions or authorized surfaces.
Omitting `separators` or using null preserves inherited grouping. An explicit
empty array removes grouping. Resetting the entire navigation document to null
removes that scope's override. Existing order-only documents remain readable;
the editor materializes group markers into a draft only when edited. Unknown
optional-module order IDs remain stored when currently visible items move, so
uninstalling or temporarily disabling a module does not destroy its preference.
User order and visibility take precedence over tenant and system preferences.
System and tenant visibility locks accumulate; lower scopes cannot hide those
destinations, but may move them. Views may supply a navigation presentation
inside the already authorized and View-filtered destination set. An explicit
personal order/layout or visibility preference takes precedence over that
presentation, not over authorization or the View's surface restrictions.
Views cannot introduce locks. When multiple modules contribute one product
entry, it inherits the earliest effective rail position/section and all
authorized contributors' locks; this does not change operational route
selection. An owner alias in View layout refers to that composed entry.
## Interaction and reuse
Drag the handle to move either a module or separator before/after another row.
The handle also supports Space to pick up, arrow keys to move, Enter to drop,
and Escape to restore the pre-drag draft. Up/down buttons offer the same moves.
Add module restores an available hidden entry; Remove only hides navigation,
never uninstalls a module or deletes records. Add separator inserts a new
optional group label. Remove separator changes grouping only.
Expanded rails display group labels without divider lines. Collapsed rails
replace these labels with horizontal group dividers; empty groups are not
rendered. Both modes use the same resolved order.
The editor receives product-area metadata to show inherited grouping and uses
container-responsive rows rather than a fixed dialog/page width. Its English
and German labels load with the editor, not the initial shell bundle.
Give the ordered list a full-span `GridItem` when a settings page contains
multiple cards; do not squeeze the entire editor into an otherwise half-empty
two-column settings grid. Central spacing tokens provide real row padding and
separation at both wide and narrow sizes, covered by computed-style assertions.
## Verification
Core navigation unit tests and HTTP settings/profile tests cover persistence,
separator inheritance, explicit flat layouts, reset and locks. Module-capability
tests cover View aliases, composed destinations and personal precedence. The
browser conformance suite tests all four editor scopes, pointer/keyboard moves,
no-op cleanliness, unavailable-module preservation, collapsed dividers and a
German narrow read-only layout. Use the same shared component for future
navigation-definition surfaces rather than implementing another sortable list.
+251
View File
@@ -0,0 +1,251 @@
# Page Layout and Action Guidelines
This document defines the binding composition grammar for headed GovOPlaN
pages. Core owns the reusable anatomy; each module owns its domain actions,
wording, authorization, consequences, and data state.
The cross-product [UI design principles](UI_DESIGN_PRINCIPLES.md) define the
display-first editing model, heading-adjacent help, and module-review process.
Normal overview/detail surfaces show readable facts; edit coherent settings in
scoped dialogs. Use an explicit editor mode for a justified broad-editing task,
not merely because a page also offers filters or contains an edit dialog (UI-02).
## Required Page Frame
- Use `PageLayout` for every headed standalone, workspace, or embedded page.
- Declare exactly one semantic `archetype`; do not infer page intent from the
`mode`, which controls geometry and scroll ownership only.
- Use `WorkspaceFrame` for a full-height module surface and
`WorkspaceLayout` only where navigation/content or list/detail panes are
genuinely part of the interaction.
- Put page-wide feedback in `PageLayout` notices. Use `DismissibleAlert` for a
recoverable warning or failure and `StatePanel` when the entire surface is
loading, empty, unavailable, or blocked.
- Do not reproduce shared page padding, heading, toolbar, form-grid, section,
table, dialog, or breakpoint CSS in a module.
## Product Side Rail
Module manifests contribute stable navigation surface identifiers, labels,
paths, icons, and default order. Core owns the side-rail composition and the
shared `NavigationPreferenceEditor`; modules must not fork this editor or
persist their own rail ordering.
Navigation preferences are layered in this order: module defaults, system,
tenant, then user. Each higher layer may reorder or change visibility. System
and tenant administrators may lock an entry visible; a lower layer can still
move that entry, but cannot hide it. Personal preferences cannot create locks.
An unset preference inherits the complete lower layer, while “Use inherited
order” removes the current layer rather than copying its values. Unknown item
identifiers remain harmless so uninstalling, disabling, or later reinstalling
a module does not corrupt the rail.
The platform module response projects module, system, and tenant layer states
alongside the effective user state. Editors must initialize from the layer
immediately below the scope they edit, so a system or tenant administrator's
personal preference is never promoted accidentally. Preference saves refresh
the platform module projection. View policy, permissions, and tenant module
entitlements remain independent final visibility gates; changing rail
preferences never grants access.
## Semantic Page Archetypes
| Archetype | Use when |
| --- | --- |
| `overview` | The page summarizes health, metrics, or several peer areas without owning one primary collection or draft. |
| `collection` | The primary object is a searchable/listable collection and Create, when available, applies to that collection. |
| `detail` | The page primarily presents one record, report, or immutable projection. |
| `editor` | The page owns one explicit draft with Save and Discard behavior. |
| `workspace` | The page coordinates several panes, stages, or task-local operations that cannot honestly be reduced to one record or draft. |
The archetype remains stable for the current interaction. A page may switch
from `overview` to `editor` when the user explicitly enters configuration
mode. It must not call a page an editor merely because a dialog or an inline
filter is editable.
## Page Action Rules
Pass one `PageActionBar` to the `PageLayout` `actions` slot. Full-canvas
workspaces use the same contract through `WorkspaceActionBar`, with an explicit
`workspace`, `collection-pane`, `detail-pane`, or `editor-pane` scope. The
variant makes the surface's intent inspectable and preserves the same keyboard
and visual order across modules. `ActionToolbar` remains the lower-level
component for section-local controls; it is not a substitute for a semantic
page or pane action bar.
| Page kind | Leading group | Trailing group |
| --- | --- | --- |
| Overview | Context | Reload when refreshable, then ordinary primary actions |
| Collection | Collection context such as export | Reload when refreshable, then Create at the far right |
| Detail | Object context | Reload when refreshable, ordinary primary actions, then a separated destructive group |
| Editor | Context | Dirty state, Reload if distinctly safe, ordinary primary actions, separated destructive actions, Discard, then Save at the far right |
| Workspace | Task context | Reload when refreshable, ordinary primary actions, then a separated destructive group |
Documentation is associated with text, not an action-group slot (UI-01). Put
`DocumentationHelpLink` in the heading component's `titleHelp`, use `TextWithHelp`
for existing contextual words, and retain `FormField`/`FieldLabel` documentation
beside field labels. Full-canvas `WorkspaceActionBar` surfaces can supply `title`
and `titleHelp` together on the leading side. Do not duplicate a heading or use
`helpAction` for a detached documentation icon. See the shared principle for
card/dialog accessibility and examples.
Reload and Create belong to the same right-aligned group, in that order. A
collection-wide toolbar stays above its workspace, not inside the left tree or
conditionally inside an editor. Changing selection or opening an editor must
not remove it. Permission-blocked creation remains visible with an explanation.
On narrow screens the trailing group wraps while retaining right alignment and
the same DOM/keyboard order.
An editor's **Cancel/Close** action is not the same as **Reset/Discard changes**.
When it exits an editing mode, set `discardAction.behavior` to `"exit"`: it stays
enabled for a clean draft, while Save remains disabled until there are changes.
The default `"reset"` behavior still requires changes. Both stay blocked during
an in-flight save; the owning page must use the shared discard confirmation before
exiting a dirty draft. Never require a meaningless edit just to leave configuration.
Deutsch: Abbrechen oder Schließen beendet den Bearbeitungsmodus auch ohne
Änderungen (`discardAction.behavior="exit"`). Zurücksetzen setzt dagegen einen
geänderten Entwurf voraus. Beim Abbrechen eines geänderten Entwurfs vor dem
Verwerfen nachfragen; eine laufende Speicherung bleibt geschützt.
Use `Card bodyLayout="table"` for table surfaces, including tables wrapped by
`LoadingFrame`. This removes body padding explicitly, without relying on the
number of children or negative margins. Place any meaningful explanation or
warning in a padded `ContentSection`; do not add a redundant tagline to every
table. Use `ContentGrid` for sibling cards so spacing does not depend on fragments.
The shared stylesheet also recognizes existing table-only card bodies, either
directly or through `LoadingFrame`; loading must not add an inset, expand the
overlay outside the card, or change the table's measured width. Keep the table
at 100% of its container. Do not reintroduce module-local negative margins,
expanded `calc()` widths, or padding overrides to make a table fit. Mixed
content cards retain their normal padding unless they explicitly declare the
table layout. The browser conformance matrix checks all four card edges,
loading/collapse transitions, scrolling, and row actions at desktop and mobile
widths.
Deutsch: Tabellen belegen den Karteninhalt ohne zusätzlichen Innenabstand,
auch während des Ladens. Neue Tabellenkarten verwenden ausdrücklich
`bodyLayout="table"`; ergänzende Hinweise erhalten ihren eigenen Innenabstand.
Negative Ränder oder modulbezogene Breitenkorrekturen sind nicht nötig.
Karten mit gemischtem Text- oder Formularinhalt behalten ihre normalen Abstände.
Use `MultiSelectFilter` for standalone list facets. It and DataGrid share the
same checkbox body and Select all / Deselect all behavior. `null` means no
restriction, `[]` means no matches, and multiple values mean OR within a facet.
Apply remote filters before server pagination/limits and discard stale reads.
Do not replace this with rows of toggles or implement a second checkbox menu.
The dropdown's body portal escapes clipped containers. Inside Core dialogs it
joins the existing dialog stack: Tab/Shift+Tab stay in the filter, Space toggles
the focused checkbox, and Escape closes only the filter and restores its
trigger. Long option labels wrap without widening the popup.
Keep facet definitions, URL serialization and request cancellation in one owning
module adapter when the same search appears on a page and in an overlay. Do not
translate an explicit empty selection into an unrestricted backend query. Keep
legacy API meanings at the adapter boundary; retain scope and unrelated URL
parameters when clearing filters. A query, context, account or tenant change
invalidates both initial and cursor requests, including results still visible
during a debounce interval.
Explorer workspaces keep collection commands in a persistent header. Files
uses Reload, Create folder, then primary Upload; frequent selected-item actions
stay near the list. Group less-common selection and connection operations in
labelled domain dialogs using `Dialog`, `FormSection` and shared action bars,
with an explicit destructive section. Do not move an overloaded toolbar into
another ungrouped row. Mail's read-only workspace has one Reload for its current
profile, folder, index and preview; narrower refreshes belong in Mailbox tools.
Do not invent a New or Save button on a workspace that owns neither workflow.
Reload must not become import, synchronization, delivery or another mutation.
Explicit Reload reads must bypass short-lived client response reuse (for
example, pass `cache: "no-store"` through the owning read API), including each
page of a refreshed listing. Routine navigation may retain normal deduplication.
Conformance must observe a fresh request, not just an enabled Reload button.
Tree icons/disclosure controls expand and collapse; labels select. A module's
`ExplorerTree.onOpen` must not toggle expansion. Use occurrence-specific node
IDs when the same semantic record appears in multiple branches; selection and
ancestor expansion must follow the clicked occurrence, not every copy.
Reload means re-fetch or re-evaluate the current surface. A page declaring
`refreshable` must provide it, and a non-refreshable page must not use Reload as
a synonym for Cancel, Reset, or Discard. Reload never silently destroys a dirty
draft. Create is a collection-wide action and is not duplicated in a
persistent side panel. Save is present only where the page owns an editable
draft; a read-only detail page must not display a disabled or inert Save merely
to fill the slot.
Editor bars always keep Discard and Save visible. Their required `state`
projection is one of `clean`, `dirty`, `invalid`, `saving`, `save-failed`, or
`conflict`, and the central component announces it through a live status label.
Clean states disable Save and reset-style Discard, but keep exit-style Cancel
available; saving disables both persistence actions. Invalid disables Save
while retaining Discard. Failed saves and conflicts keep the draft recoverable
and allow an authorized retry after the module has shown the owning error or
conflict evidence. A module may add a more specific validation, policy, or
permission blocker. The editor must register its draft with
`useUnsavedDraftGuard` (or a shared hook that uses the same registration
contract), so browser unload, route navigation, section changes, Reload, and
the explicit Discard path cannot silently lose work.
Reload is rendered by Core from a descriptor rather than passed as arbitrary
button markup. It can project `current`, `stale`, `reloading`, or
`reload-failed`; `loading` is the shorthand for `reloading`. A failed refresh
must preserve usable loaded data, expose its stale/failure state, and leave
Reload available for recovery. Reload goes through the same unsaved-navigation
guard as route changes.
Destructive page actions use `destructiveActions`; never put a danger action in
`contextActions` or the ordinary primary group. Core renders a persistent
visual and semantic boundary before this group. In an editor it precedes the
Discard/Save pair, keeping Save in the final keyboard and visual position.
`PageActionBar` controls non-editor placement and owns the standard editor
persistence buttons. Other actions continue to use central
`Button`, `IconButton`, or `TableActionGroup` components. When an action is
visible but unavailable because of permission, target, policy, state, or
validation, keep it in its stable slot and supply `disabledReason`. Do not
silently hide a normally applicable action.
## Forms and Dialogs
- Compose forms from `FormLayout`/`FormGrid`, `FormSection`, and `FormField`.
- Use `FieldLabel` through `FormField` for every field that is not genuinely
self-explanatory; record justified omissions in the owning UI ledger.
- Use `Dialog`, `DialogForm`, `DialogSection`, and `DialogActions` for modal
work. A dialog can be domain-specific while its anatomy remains central.
- Use `useUnsavedDraftGuard` for explicit Discard and guarded navigation on an
editable page or dialog.
- Explain irreversible or operationally consequential actions before the
commit button, including reversibility and durable evidence.
## Collections and Details
- Use `FilterBar` for collection query controls and `DataGrid` for tabular
collections. Keep a single ordered `TableActionGroup` action set per table.
- Use `MetricGrid`/`MetricCard` for summary measures, `Card` or
`ContentSection` for logical sections, and `DescriptionList` for labelled
facts.
- Add a `MetricCard.drilldown` only when the displayed measure has a useful,
authorized underlying collection or detail. Name the destination explicitly
(for example, “Review failed deliveries”) and preserve the current scope and
filters in its `href` or action. The card itself remains non-interactive so
the action is visible and keyboard-predictable. Derived, privacy-suppressed,
non-enumerable, or purely informational aggregates remain plain metrics;
when an ordinarily available drill-down is temporarily blocked, keep its
action and provide `disabledReason`.
- Preserve loaded data after a refresh failure and mark it stale; offer Reload
as the recovery action. Distinguish initial loading, empty, unavailable,
permission-blocked, conflict, success, and retry states.
## Review Evidence
Every new or changed page or workspace pane must have structural evidence for
its frame, semantic archetype/scope and slot order, refresh declaration, shared
component usage, stable disabled actions, dirty guard, destructive boundary,
and module-owned help identity. Type checks enforce conditional Reload and
editor persistence props. The product check discovers all consumers, rejects
undeclared archetypes and `ActionToolbar` panel-header copies, and requires
semantic actions for every `WorkspaceFrame` route. Browser conformance confirms
keyboard order, lifecycle changes, accessibility, destructive separation,
narrow wrapping, and screenshot geometry.
+41
View File
@@ -14,6 +14,7 @@ consistent while each module still owns its domain rules.
| Governance defaults | `govoplan-admin` plus `govoplan-access` materializer | admin settings, governance template routes, access materialization capability | System governance can block tenant-local groups, roles, and API keys. |
| Delegation and ownership policy | access/campaign/mail/files modules | capability checks and owner-scoped APIs | Source provenance should use this contract when policies become externally explainable. |
| Definition governance | `govoplan-policy` | capability `policy.definitionGovernance` | Resolves view, edit, run/start, reuse, derive, and automate for system, tenant, group, and user Dataflow/Workflow definitions. |
| Function assignment governance | `govoplan-policy` | capability `policy.functionAssignmentGovernance` | Returns current review steps, delegation depth/validity ceilings, and explicit timed-escalation targets consumed by IDM. |
## Policy Decision
@@ -126,6 +127,43 @@ When the capability is absent, modules must not silently emulate cross-scope
inheritance. Their conservative fallback is limited to local tenant
definitions and disables reuse, derivation, and automation.
## Function Assignment Delegation And Escalation
`FunctionAssignmentGovernanceDecision` is the versioned cross-module contract
for request/grant review. In addition to the required holder, authority, and
recipient steps, it returns `delegation_allowed`,
`maximum_delegation_depth`, `maximum_delegated_validity_days`, and typed
`FunctionAssignmentEscalationRule` entries. Each escalation entry binds one
review step to an exact target function and timeout.
The decision is a current ceiling, not durable authorization. IDM must recheck
the complete assignment-source chain and all recorded decisions before final
application. An elapsed timeout creates explicit state and evidence; it must
never be interpreted as approval or as permission to silently substitute an
approver. Missing providers, malformed rules, invalid chains, or tightened
limits fail closed with an explainable reason.
## Bounded Impact-Subject Providers
Policy impact previews discover optional subject providers through capability
names beginning with `policy.impactSubjects.`. The suffix is the stable
provider ID; for example, Views contributes `policy.impactSubjects.views`.
Providers implement `PolicyImpactSubjectProvider` and receive a
`PolicyImpactPopulationRequest` containing the active tenant, policy family,
an explicit selector, actor scopes, detail-disclosure decision, and a limit of
at most 500. They return `PolicyImpactSubjectBatch` with unique opaque subject
references and an explicit `complete`, `sampled`, `truncated`, or `unavailable`
state. An unavailable batch must explain the gap, and a total may never be
smaller than the returned subject count.
Core does not scan module data or evaluate domain policy. The owning module
selects and permission-filters its candidates; Policy compares the current and
proposed decisions and controls response disclosure. A caller must select one
or more provider populations explicitly. This preserves optional-module
boundaries and prevents a seemingly harmless preview from becoming an
unbounded platform query. Providers must not include credentials, secrets, or
unfiltered cross-tenant labels in subject attributes.
## Frontend Contract
Policy UIs must:
@@ -138,6 +176,9 @@ Policy UIs must:
lower-level limit to `false`
- avoid sending locked fields or re-enable attempts in save payloads
- show inherited values separately from local overrides
- require a current impact preview before enabling a governed high-impact save,
preserve its proposal hash on commit, and explain incomplete population
coverage rather than presenting unavailable providers as zero impact
The core WebUI helper `privacyRetentionParentAllowsField()` centralizes the
field-lock decision used by the retention editor and its lightweight module
+55 -4
View File
@@ -1,9 +1,12 @@
# Postbox End-To-End Encryption Architecture
This document records the strategic encryption target for GovOPlaN postboxes.
It does not require the first postbox implementation to ship full E2EE, but it
defines the architecture so early data models and APIs do not make the stronger
model impossible.
This document records the encryption boundary for GovOPlaN postboxes. Postbox
now implements the server-side contracts for three selectable profiles:
unencrypted content, institution-managed server envelopes, and externally
produced E2EE envelopes. The E2EE contract is operational—the server rejects
plaintext and retains ciphertext, signed manifests, wrapped keys, and digest
evidence—but a reviewed browser/device client and private-key custody provider
remain separately deployed responsibilities.
The core principle is that a postbox can become a trusted administrative
communication channel without requiring the server to see plaintext content.
@@ -35,6 +38,54 @@ Algorithm choices should remain replaceable behind a crypto profile. The first
profile should prefer standard, reviewed primitives such as HPKE for key
wrapping and AEAD encryption for content.
## Product Profiles And Default
The content-protection policy is configurable per exact Postbox or immutable
template revision:
- `server_envelope_v1` is the recommended default. An institution-selected
Encryption vault controls server-readable envelopes and their migration
evidence. It is not end-to-end encryption.
- `external_e2ee_v1` is server-blind. An approved client or producer supplies
the ciphertext reference, signed manifest, wrapped recipient keys, key epoch,
and SHA-256 plaintext digest. GovOPlaN has no private key that can decrypt it.
- `plaintext_v1` stores clear content for institutions that explicitly choose
that boundary.
Operational metadata—including subject, routing, participants,
classifications, timestamps, attachment references, receipts, and retention
state—remains visible under every profile. Administrators therefore choose a
content-protection boundary, not a metadata-anonymity profile.
The standard policy grants new incumbents history since assignment, uses key
rewrapping for ordinary rotation and content re-encryption after compromise,
requires two-person institutional recovery and dual-control hand-over,
emergency, export, and destruction, requires strong external identity, and
limits vacancy escalation to metadata. Deployments may select other policy
values rather than inheriting a decision from GovOPlaN.
## Governed Profile Changes
A profile transition applies to new messages immediately and increments the
Postbox key epoch. Retained history can remain under the previous profile or be
migrated. The transition ledger records source and target profiles/vaults,
authority route, consent and key-holder evidence, quorum, reason, immutable
configuration snapshot, per-message source and target digest, and outcome.
Plaintext and managed-envelope migrations can use the server-side Encryption
capability. Managed decrypt, export, and re-encryption operations also create
Encryption migration records so old envelopes are disposed of through the
governed provider contract. Any transition to or from E2EE pauses each retained
message for an approved client transform. The client must return plaintext or
ciphertext as appropriate, plus evidence and the original content digest;
Postbox verifies digest continuity before changing the stored representation.
Leaving E2EE requires user-consent evidence, while changing managed history
requires institutional key-holder evidence. Dual control can require both.
This transition mechanism cannot revoke plaintext already decrypted, copied,
printed, or exported. Administrators must explicitly acknowledge that residual
disclosure before a transition is accepted.
## Identity And Device Keys
The platform should distinguish:
@@ -141,6 +141,73 @@ connector or module issue.
- Owner/priority: `govoplan-mail`, `govoplan-calendar`,
`govoplan-connectors`, Wave 1/2.
#### Collaboration-suite boundary and hand-offs
Collaboration remains connector-first. The product named below never changes
which GovOPlaN module owns the administrative meaning of the work:
| External family | Initial posture | GovOPlaN semantic owner | Connector-owned boundary |
| --- | --- | --- | --- |
| Collabora Online, OnlyOffice, Nextcloud Office | Link an externally edited document and its editing session; import a governed rendition only when required | DMS owns document/version, lock, review, approval, retention, and collaboration-session evidence; Files owns stored bytes | Discovery, endpoint health, WOPI/vendor session exchange, callbacks, and provider object references |
| Matrix, Mattermost, Rocket.Chat, Nextcloud Talk | Create or link a room/thread for a governed work context; do not mirror all conversation history by default | The initiating Case, Workflow, or Task owns the work-context link and disposition; DMS/Records own retained evidence deliberately captured from it | Room/thread creation, membership synchronization, webhook/event normalization, and stable external links |
| Jitsi and BigBlueButton | Provision or link a conference for an existing appointment/event | Appointments owns booking intent; Calendar owns event, attendee, invitation, and time semantics | Conference provisioning, join/moderator references, provider lifecycle, and bounded attendance/result callbacks |
| OpenProject and comparable project suites | Link first, then publish or synchronize selected work packages | Tasks owns GovOPlaN task state; Workflow owns orchestration; Cases own case state and evidence references | Project/work-package lookup, publish/synchronize transport, webhooks, version tokens, and external URLs |
| Cross-suite activity streams | Consume normalized, bounded events only for an authorized work context | The receiving module decides whether an event changes state or becomes evidence; Audit records the GovOPlaN operation | Provider subscriptions, cursor/checkpoint handling, signature validation, event normalization, and replay protection |
Native collaboration behavior is justified only when GovOPlaN must own the
semantic state, authorization decision, audit evidence, retention/legal-hold
rule, or configuration-package fragment. Endpoint profiles, tokens, health,
protocol clients, provider IDs, retries, and webhook transport remain in
Connectors (or the owning protocol connector). A feature module consumes a
Core capability/DTO and must still start and fail explicitly when that optional
connector is absent; it never imports a provider client.
The minimum hand-off sequences are:
1. **Appointment to conference:** Appointments confirms the booking intent;
Calendar creates or updates the event and invitations; an optional
conference connector provisions the room idempotently and returns an
opaque join reference. Calendar stores that reference with the event, not
the provider credential.
2. **Case or Workflow to collaborative document:** the initiating module asks
DMS for a governed document/session; DMS requests an optional office-suite
connector session and retains version, lock, approval, and callback
evidence. The Case/Workflow keeps only the DMS reference.
3. **Case, Workflow, or Task to chat:** the semantic owner requests a room or
thread with an idempotency key and bounded membership intent. The connector
returns an external reference; capturing messages as evidence requires an
explicit DMS/Records action and policy decision.
4. **Task or Workflow to project suite:** Tasks supplies the task payload and
Workflow supplies correlation; the OpenProject connector publishes or
reconciles the work package and returns versioned external-reference and
retry/conflict evidence. Neither consumer writes connector tables.
Every executable collaboration connector must pass the common connector
contract checks plus a provider-focused minimum proof:
- optional-module startup and partial compositions work without the provider;
- profile health uses secret references and redacts credentials and remote
response bodies;
- tenant/resource authorization is checked before discovery, provisioning,
lookup, synchronization, or evidence capture;
- dry-run/simulation performs no remote mutation and explains unsupported
operations;
- create/publish calls are idempotent, retries preserve the same external
reference, and outcome-unknown or version conflicts remain reconcilable;
- callbacks/webhooks verify authenticity, tenant/profile binding, replay
protection, and bounded payloads;
- disable/retire behavior revokes new use while preserving non-secret audit and
external-reference evidence;
- Collabora/OnlyOffice prove discovery plus one non-production editing-session
round trip; Matrix/Mattermost/Rocket.Chat prove room lookup/create plus one
authenticated bounded event; Jitsi/BigBlueButton prove conference
provision/cancel; OpenProject proves project/work-package lookup, idempotent
publish, and conflict handling.
These are connector acceptance tests, not a claim that those connectors are
already implemented. Their implementation state remains in the owning
connector issues and catalogue.
### Payment And Public Cashier Systems
- Strategy: integrate/export/import; keep the payment provider or cashier as
+94
View File
@@ -0,0 +1,94 @@
# Records Filing Contract
Core exposes a small provider-neutral contract for filing exact source
revisions into an institutional record. Core does not own records semantics,
source-object authorization, or source bytes. `govoplan-records` owns filing
orchestration and chronology; each source module owns resolution of its exact
revision.
## Capability Names
- `records.filing` is supplied by the enabled Records module.
- `records.source.<module>` is supplied by an enabled source module, for
example `records.source.files` or `records.source.cases`.
- `records.archive.<provider>` is supplied by an enabled archive-transfer
adapter. Discovery does not imply conformance or current health.
Callers discover capabilities through the module registry. They must not
import optional source-module internals.
## Exact Source Identity
`RecordSourceLocator` identifies one tenant, source module, resource type,
resource ID, and immutable source revision. A source provider must:
1. reject cross-tenant resolution;
2. require a non-empty purpose;
3. re-evaluate the caller's current module and object authorization;
4. resolve exactly the requested revision, never a mutable "current" alias;
5. return safe display/provenance metadata and a SHA-256 digest when the source
has stable bytes or a canonical snapshot;
6. fail closed when the revision is missing, quarantined, corrupt, or no longer
authorized.
Historical Records browsing never revives historical access rights. The
source's current authorization decision remains authoritative when filing.
## Filing Semantics
`RecordFilingRequest` binds the exact source to a record, purpose, filing
reason, relationship, institutional context, and idempotency key. Records must
persist source identity and resolution evidence together with the filing actor,
represented capacity, valid time, recorded time, and immutable chronology.
An idempotency key may replay only an identical request. A conflicting reuse
must fail. Filing does not transfer ownership of source content and must not
silently copy mutable source state.
## Versioning
The Python DTOs and protocols live in `govoplan_core.core.records`. The
manifest interface `records.filing` starts at `1.0.0`. Incompatible DTO or
behavior changes require a new interface version and release impact analysis;
additional optional metadata remains backward compatible.
## Initial Providers
- Files resolves an exact managed `FileVersion`, verifies current Files access
and blob integrity, and returns its stored content digest.
- Cases resolves an exact immutable case revision after current case access and
returns a digest of the canonical revision snapshot.
Provider-specific selection UI belongs to the source module. The generic
Records dialog remains a diagnostic/manual fallback for exact identifiers.
## Archive Transfer Boundary
`RecordTransferPackage` binds a stable package ID, record revision, provider
profile, canonical manifest, and manifest SHA-256. An archive provider exposes
`RecordArchiveProviderState` before dispatch and accepts only a
`RecordArchiveTransferRequest` for a declared healthy profile. Its receipt must
identify the same package and provider and return one bounded outcome:
`accepted`, `rejected`, or `outcome_unknown`.
An unknown outcome is never retry-safe. Callers must retain the intent and
reconcile it against the provider before another effect. Provider state also
declares authority mode, freshness, limitations, and whether the provider is a
simulation. Credentials, transport configuration, archive-specific package
schemas, and custody semantics remain provider-owned.
Records includes `records.archive.simulation` to prove package and receipt
handling. The simulation is explicitly non-conformant, transfers no custody,
and cannot be used as evidence of an archive handoff. A real provider requires
a selected target/profile, provider-specific recovery declaration, and target
test evidence.
## Form Evidence Boundary
Form attachments use the separate provider-neutral contract in
`govoplan_core.core.form_evidence`. Forms Runtime requests short-lived,
purpose-bound upload grants and re-inspects the exact provider-owned evidence
before final submission. The provider keeps byte storage, quarantine,
classification, and retention ownership; Forms Runtime stores only immutable
evidence references and bounded verification results. This contract is not an
alternative path for Records filing or archive custody.
+41
View File
@@ -95,6 +95,15 @@ package entry. Keep one committed full-product release lockfile at
`webui/package.release.json` in a clean release workspace. Development
`package-lock.json` may continue to point at local `file:` dependencies.
The default WebUI build discovers every declared, installed module package,
including Tasks and its Work page, dashboard widget, and Quick Access tool.
Discovery still respects enabled backend modules and user permissions; adding
a package never grants access. `GOVOPLAN_WEBUI_MODULE_PACKAGES` selects an
explicit smaller build when set, and an explicitly empty value selects
core-only. The prebuild interface check compares the default descriptor list
with the package manifest so explicit permutation tests cannot conceal a
module accidentally omitted from the ordinary build.
Frontend module permutations are regression-tested through
`GOVOPLAN_WEBUI_MODULE_PACKAGES` and temporary build output, not through
committed lockfiles for every possible combination. If a smaller composition
@@ -197,6 +206,13 @@ If both file and URL are set, the URL wins. The cache is used when a remote
fetch fails, so an operator can still inspect the last known catalog. A cached
catalog must still pass signature, freshness, channel, and replay validation.
If neither source is configured, the Admin package directory discovers the
official public stable catalog at
`https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json`. Core verifies
that fallback against the public key pinned in the installed Core package. An
explicit deployment catalog always takes precedence; a configured source that
is unavailable or invalid fails closed instead of silently falling back.
An official catalog is a JSON object with:
- `catalog_version`
@@ -212,6 +228,14 @@ Each module entry can declare:
- backend package name and pinned install reference
- WebUI package name and pinned install reference
- `artifact_integrity` for each package, including the HTTPS registry URL,
filename, byte size, SHA-256, package identity, source tag, and source commit
- `source`, binding the repository and immutable tag/commit identity, with
optional HTTPS repository and revision links
- `availability`, either `available` or `withdrawn`; a withdrawn entry must
carry an operator-readable `availability_reason` and cannot be planned
- `configuration_requirements` and an optional HTTPS `release_notes_url` for
prerequisites and release-specific operator guidance
- display metadata and tags
- `license_features`, the feature entitlements required to plan that install
- `dependencies` and `optional_dependencies`, the module ids expected in the
@@ -239,6 +263,12 @@ Each module entry can declare:
- `requires_interfaces`, named interface contracts and version ranges required
by this module
Core validates these fields before exposing the directory. Admin derives a
read-only catalog state from the installed package set, catalog dependency
closure, named-interface providers, current-version window, availability, and
generic license policy. This is an early operator diagnostic; trusted installer
preflight remains the authoritative mutation gate.
The signature is Ed25519 over canonical JSON with both `signature` and
`signatures` removed. Core accepts the legacy single `signature` field and the
new `signatures` array.
@@ -302,6 +332,12 @@ Catalog provenance changes preflight severity:
plans, so operators can still use offline or emergency package refs
- valid-catalog warnings, such as intentionally unsigned local catalogs when
signature enforcement is disabled, remain warnings
- a saved catalog plan must match the currently validated entry exactly;
altered package refs, artifact identities, channel, sequence, trust state, or
signing-key identity block the run and require replanning
- a trusted remote artifact is downloaded before mutation into a private
SHA-256-addressed installer cache, checked for exact size and digest, and
passed to `pip` or npm only as that verified local file
- selected catalog entries with unsatisfied non-optional named interface ranges
block activation before the installer runs
- selected catalog entries whose target dependencies are neither installed nor
@@ -519,6 +555,11 @@ Catalog entries can require license features:
Core checks those requirements against an offline license file before allowing
the entry into the install plan.
Official open-source GovOPlaN entries do not declare license features. The
license contract remains generic for external catalogs, deployment presets,
configuration/package directories, and support offerings; it gates only an
entry that explicitly asks for a feature.
```bash
GOVOPLAN_LICENSE_FILE=/srv/govoplan/license.json
GOVOPLAN_LICENSE_ENFORCEMENT=true
+3 -1
View File
@@ -12,4 +12,6 @@ tools/checks/security-audit/run.sh --mode full --scope govoplan
Canonical documentation:
- `/mnt/DATA/git/govoplan/docs/SECURITY_AUDIT.md`
- `/mnt/DATA/git/govoplan/docs/operations/SECURITY_AUDIT.md`
Implementation contract: [disposable resource-bounded operations](BOUNDED_PROCESS_CONTRACT.md).
+118
View File
@@ -0,0 +1,118 @@
# Semantic Documentation Subjects
## Purpose And Ownership
The semantic-documentation subject contract lets an optional module expose the
configured artifacts that administrators may document: for example a form, a
form field, a workflow, or a workflow state. It is a discovery and resolution
contract, not a second configuration API.
The module that owns an artifact also owns its subject provider, authorization,
identity, revision, route, and lifecycle semantics. Docs may discover those
providers through Core and attach authored documentation to their stable
references. Docs must not import the feature module, read its tables, or copy
configuration content into a generic index.
This contract is additive to manifest `DocumentationTopic` contributions and
configured-state `documentation_providers`. Every providing module must retain
static user and administrator documentation baselines. The baselines explain
the feature even when the provider is disabled, unavailable, or has no
configured subjects.
## Identity And Versioning
`SemanticDocumentationSubjectReference` identifies a subject with:
- owning module and tenant;
- a module-defined subject kind and stable identifier;
- an optional typed nested anchor, such as `field/registration-number`;
- the revision and canonical fingerprint observed when documentation was
authored or reviewed.
The `stable_key` derives only from identity. A rename or configuration revision
therefore does not detach existing documentation. A nested anchor has its own
identity so a field can be documented independently from its form.
Providers must resolve an old reference as one of:
- `available`: the observed revision/fingerprint is still current;
- `changed`: the same stable subject has changed and may need review;
- `superseded`: another stable reference replaced it;
- `missing`: the subject was removed or is no longer resolvable;
- `temporarily_unavailable`: the provider cannot currently determine state.
Absence is not authorization. A provider returns `None` when the principal may
not learn whether a subject exists. Core also rejects cross-tenant list and
resolution requests before calling a provider.
## Safe Projection
Descriptors contain only bounded, explicit presentation fields: localized
labels and descriptions, breadcrumbs, a local route, audience,
classification, and required scopes. They must not contain credentials,
personal data, arbitrary provider metadata, configuration payloads, or the
authored documentation itself. Routes are application-local and are still
subject to normal route authorization.
The fingerprint is a review signal, not a concurrency token or a content hash
that callers may use to reconstruct configuration. Providers should calculate
it from the smallest canonical JSON projection whose semantic changes require
documentation review. Volatile timestamps and secrets must be excluded.
## Provider Registration
A provider is registered under its exact module-scoped capability name:
```python
from govoplan_core.core.modules import CapabilityDocumentation
from govoplan_core.core.semantic_documentation import (
SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION,
semantic_documentation_subject_capability,
)
capability = semantic_documentation_subject_capability("forms")
manifest = ModuleManifest(
id="forms",
# ...
capability_factories={capability: build_semantic_subject_provider},
capability_documentation={
capability: CapabilityDocumentation(
label="Form semantic subjects",
summary="Lists authorized configured forms and fields for Docs.",
contract_version=SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION,
documentation_types=("admin", "user"),
)
},
documentation=(admin_baseline, user_baseline),
)
```
The capability is `documentation.semantic_subjects.<module_id>`. Registry
validation rejects a mismatched owner, missing capability documentation, a
wrong contract version, or missing static baselines.
`list_semantic_documentation_subjects` performs authorized, paginated discovery
across installed providers. `resolve_semantic_documentation_subject` targets
one owner without loading another feature module. Providers must apply the
current tenant and principal on every call and must not infer visibility from a
previous list result.
## Lifecycle And Integration Rules
- Keep subject and anchor identifiers stable across display-name and route
changes.
- Return `superseded` only with the replacement reference; do not silently
rewrite stored references.
- Return a reason code for missing or temporarily unavailable subjects without
exposing sensitive detail.
- Reauthorize both discovery and resolution. Stored documentation references
confer no access to a live artifact.
- Treat a changed fingerprint as a request for editorial review. It does not
automatically invalidate or publish authored documentation.
- Removing a feature module leaves references resolvable as provider
unavailable. Docs can preserve history without importing the module.
Forms, Workflow, and later modules should implement their subject providers in
their own repositories. Docs owns the authored semantic-documentation records,
review workflow, and projection UI.
+10 -3
View File
@@ -93,10 +93,17 @@ Alembic, and post-migration tasks. The lock is session-scoped and therefore
released if the migration process dies.
Runtime roles run `govoplan_core.commands.wait_for_database`. It waits until
the database has exactly the configured Core/module Alembic heads and never
upgrades schema. This permits a migration Job and runtime Deployments to be
the database has exactly the configured, dependency-resolved Core/module
Alembic heads and never upgrades schema. Cross-module `depends_on` revisions
therefore do not leave runtime roles waiting for a branch marker Alembic has
correctly consumed. This permits a migration Job and runtime Deployments to be
submitted together while keeping startup fail-closed.
Runtime coordination records the installed `govoplan-core` distribution
version for API, worker and scheduler roles. FastAPI/OpenAPI metadata versions
are presentation metadata and must not be used as deployable software identity;
mixing the two would create a false version-skew readiness failure.
## Recovery Ledger
`govoplan_core.core.recovery` provides a durable operation and evidence
@@ -133,7 +140,7 @@ Core does not create production database backups. The deployment owner must
provide backup, retention, encryption, restore verification, and recovery-point
coordination for PostgreSQL, object storage, and encryption keys. The canonical
operator procedure is documented in
`govoplan/docs/RECOVERY_AND_ROLLBACK_GUARANTEES.md`.
`govoplan/docs/operations/RECOVERY_AND_ROLLBACK_GUARANTEES.md`.
## Verification
+69
View File
@@ -19,3 +19,72 @@ Connector health and preview diagnostics must contain no credentials, endpoint
userinfo, row values, or unbounded remote error bodies. A Datasource origin
preserves this contract so registration and staging do not erase source mode,
health, pushdown, or preview-limit evidence.
## Durable CSV imports and original evidence
`TabularCsvSource` optionally accompanies a durable `TabularSnapshotInput` or
`DatasourceStageInput`. It carries the exact submitted Unicode text, delimiter,
explicit value mode and parser profile. It is not part of ordinary catalogue,
preview or stage DTOs. Transient inspection does not retain original content.
The `text` mode preserves cell strings, including whitespace, leading zeroes,
decimal spelling, boolean-looking text and explicit empty cells. It rejects
malformed quoting and rows with missing or extra cells. Header normalization is
unchanged. The API default remains `legacy_typed` for existing integrations;
interactive CSV imports offer text mode by default and an explicit legacy choice.
JSON and existing stored snapshots are not reinterpreted. Core and Datasources
retain distinct versioned legacy parser profiles where their historical coercion
rules differ. Shared schema inference preserves first-seen column order, missing
value nullability and the owning provider's type naming.
Owners verify that the source reparses to exactly the stored projection, including
scalar types: `true`, `1` and `1.0` are not equivalent evidence. Raw input and row
projections each have a 5,000,000-byte limit; row parsing is capped at 10,000 rows.
The original text has its own UTF-8 SHA-256 and byte count, separate from the
existing row fingerprint. Only a small allowlisted source summary enters metadata.
Checksums detect drift; they are not digital signatures or protection against an
attacker who can rewrite the database and all its evidence.
Original exports are explicit owner APIs, tenant scoped, integrity checked and
`no-store`. Datasources additionally requires administrator scope, audits the
export, and denies the whole original when current or historical governance
restricts any row or field. Freezing verifies the prior summary before copying
source evidence and retains prior policy restrictions, including referenced
policy evaluations. At most 32 distinct governance snapshots may accompany one
original; further incompatible history fails explicitly. Payload disposal also
disposes retained original content. Connectors applies its own current read,
tenant and lifecycle checks. See the owning module's documentation for endpoints.
Original UTF-8 text is not proof of pre-decoding file bytes, and does not undo
CSV spreadsheet formula semantics. Exported content is deliberately unmodified;
operators must treat it as untrusted input when opening it in a spreadsheet.
New nullable columns require the owning modules' additive migrations. Historical
rows remain unchanged and report original content unavailable, not reconstructed.
Back up retained originals before any schema downgrade that removes those columns.
## Deutsch: CSV-Datentreue
Dauerhafte CSV-Importe können den unveränderten übermittelten Unicode-Text mit
Trennzeichen, Parserprofil und explizitem Wertemodus aufbewahren. Der Textmodus
erhält Zellwerte einschließlich Leerzeichen, führender Nullen und Dezimalschreibweise.
Fehlerhafte Zeilen werden abgewiesen. Die API bleibt aus Kompatibilitätsgründen bei
der bisherigen Typumwandlung als Standard; im Importdialog ist Text voreingestellt.
Bestehende Daten und JSON-Importe werden nicht neu interpretiert.
Original und Zeilenprojektion werden getrennt begrenzt und geprüft; boolesche
Werte, Ganzzahlen und Gleitkommazahlen sind keine austauschbaren Belege. Der
Originaltext erscheint weder im Katalog noch in Vorschauantworten. Die expliziten
Export-APIs prüfen Mandant, Berechtigungen, Lebenszyklus und gespeicherten Hash.
Datasources verlangt zusätzlich Administrationsrechte, protokolliert Exporte und
berücksichtigt aktuelle sowie historische Zeilen-, Feld- und Zugriffsrichtlinien.
Eingeschränkte Originale werden vollständig gesperrt, nicht teilweise freigegeben.
Eingefrorene Kopien übernehmen diese Einschränkungen; nach 32 unterschiedlichen
Richtlinienständen wird eine weitere Kopie mit neuer Richtlinie explizit abgewiesen.
Die Aufbewahrungsbereinigung entfernt auch das gespeicherte Original.
Die Grenzen betragen jeweils 5.000.000 UTF-8-Bytes für Original und Projektion
sowie 10.000 Zeilen. Prüfsummen sind keine Signaturen. Ein CSV-Original bleibt beim
Export unverändert und kann Tabellenkalkulationsformeln enthalten. Frühere
Dateikodierungen lassen sich daraus nicht rekonstruieren. Additive Migrationen
ändern keine historischen Zeilen; fehlende Originale werden nicht erfunden.
Vor einem Schema-Downgrade sind aufbewahrte Originale zu sichern.
+68
View File
@@ -0,0 +1,68 @@
# Temporal Data Context
GovOPlaN exposes one read context for data validity and system knowledge. The
calendar control in the authenticated titlebar applies that context to
supported list and detail reads for the current account and tenant.
## Two Independent Axes
- **Valid time** answers when a fact applied in the represented domain.
- **Recorded time** answers what the system had recorded by a particular
instant.
The default is data valid now under the latest recorded state. `At time`
selects a valid-time instant. `All` removes the valid-time interval filter but
still uses the selected recorded state. The optional recorded-state cutoff can
be combined with any valid-time mode, which keeps correction history distinct
from changes in real-world validity.
An interval is half open: `valid_from <= instant < valid_to`. A revision belongs
to a recorded-state snapshot when `recorded_at <= cutoff` and it was not
superseded at or before that cutoff.
## Security And Mutation Rules
The temporal data context is a read projection, not an authorization context.
Authentication, permissions, active delegations, tenant boundaries, module
policy, and maintenance controls are always evaluated under current security
state. A historical projection never restores an expired permission.
The context also does not supply mutation dates. Writes continue to target the
current lifecycle revision and must carry their explicit valid/effective dates,
expected revision, reason, and evidence where the owning contract requires
them. A screen showing historical data must not silently turn a normal edit
into a historical correction.
## HTTP Contract
Core accepts these request headers:
| Header | Meaning |
| --- | --- |
| `X-Govoplan-Validity-Mode` | `current`, `at`, or `all` |
| `X-Govoplan-Valid-At` | Timezone-aware ISO 8601 instant required by `at` |
| `X-Govoplan-Recorded-At` | Optional timezone-aware system-knowledge cutoff |
Invalid or naive timestamps fail with HTTP 400. Responses expose the resolved
mode and evaluated instant. Conditional JSON responses vary by all three
request headers, and the shared WebUI API client includes them in request
deduplication and conditional-cache keys.
## Module Adoption
Revision-owning modules apply
`govoplan_core.db.temporal.apply_temporal_revision_filter` only to read queries
that are meant to follow the platform context. Explicit version references and
explicit resolver `effective_at` arguments take precedence. Current-row
lookups used for optimistic concurrency, authorization, routing, effects, or
other mutations must remain explicit and context-independent.
The initial bitemporal adoption covers Decisions, Mandates, Parties, and
Services. Their immutable revisions have indexed valid, recorded, and
superseded timestamps. Modules with effective-dated security records or
recorded-only revision histories require separate display-query adoption so
the global selector cannot affect current authorization or execution.
The WebUI selection is stored in session storage per account and tenant. A
change remounts the active module route so existing page loaders issue a fresh
request. Returning both axes to their defaults removes the stored selection.
+39 -5
View File
@@ -4,6 +4,9 @@ GovOPlaN supports `system`, `light`, and `dark` as persisted user preferences.
`system` follows `prefers-color-scheme` live; it is not resolved permanently at
save time. Core applies the resolved mode through `data-theme` on the document
root and exposes the selected preference through `data-theme-preference`.
Each user may also choose a validated `default`, `civic_blue`, `forest`, or
`plum` accent palette. Core applies it through `data-palette`; every module
inherits the result through semantic tokens without module-specific CSS.
## Ownership
@@ -12,17 +15,48 @@ root and exposes the selected preference through `data-theme-preference`.
- Modules consume semantic tokens such as `--surface`, `--text`, `--line`, and
the status token families. They may define domain aliases whose values resolve
to shared tokens.
- User preference selects the mode. Tenant and system policy may provide a
future default, but must not silently replace an explicit user choice.
- Tenant branding is a separate policy surface and must preserve contrast and
status semantics in both modes.
- Palette defaults form a provenance chain: system, tenant, then an explicit
user choice. Invalid stored values are ignored. Reset means inheritance and
does not copy the current parent value into the child scope.
- A policy lock is separate from the default. A system lock wins over every
child scope; otherwise a tenant lock suppresses a personal override. The
authenticated profile reports the effective palette, source, inherited
palette, and lock state.
- Advanced personal overrides are a separately governed surface. The system
must opt in, a tenant may inherit or block that decision, and palette locks
always suppress overrides. Changing either policy requires
`admin:policies:write` in addition to the owning settings permission.
## Palette safety and scope
The Settings preview shows the chosen or inherited accent in every applicable
light/dark preview before Save. Presets are checked for WCAG AA contrast in the
theme contract. When policy permits, the shared advanced editor can atomically
override accent, surface, and semantic status pairs for both modes. Every
foreground/background pair must meet WCAG AA contrast, and success,
information, warning, and danger colors must remain distinct. Invalid stored
documents fail closed and are not partially applied.
Import and export use the exact versioned JSON schema `schema_version: "1"`.
Both `light` and `dark` must contain every supported token exactly once as a
six-digit hex value. Import changes only the local draft; Save persists the
whole document. Removing overrides returns to palette and policy inheritance.
The system default is disabled so upgrades do not unexpectedly admit arbitrary
branding. Tenant `null` means inherit, `false` blocks, and `true` is accepted
only while the system permits overrides.
Do not introduce fixed foreground/background colors in a module merely to make
one mode look correct. Add or reuse a semantic Core token, then define both
light and dark values. Bitmap content and externally authored HTML are exempt,
but their surrounding controls must still use the shared tokens.
`npm run test:theme-contract` verifies the root behavior and representative
`npm run test:theme-contract` verifies root mode/palette behavior, preset and
custom-override validation/application, and representative
Campaign, Calendar, Files, and Mail token consumption. The check runs before a
production WebUI build.
Runtime validation and token application live in the dependency-free
`webui/src/components/appearanceOverrides.ts`; both the shell and the shared
editor use it. The shell must not import the editor to apply an existing theme:
settings controls load with their route, while valid saved colors apply
synchronously and invalid documents still fail closed before any token is set.
+59
View File
@@ -0,0 +1,59 @@
# Ticket Integration Capability Contracts
Core owns two narrow, optional contracts that let the Tickets module compose
with policy and formal-procedure modules without importing either one. Tickets
remains the authority for operational ticket identity, lifecycle, assignment,
comments, links, and immutable history.
## Capability Names
- `tickets.routing` optionally supplies a `TicketRoutingProvider`.
- `tickets.case_escalation` optionally supplies a
`TicketCaseEscalationProvider`.
Both contracts are version 1 and are defined in
`govoplan_core.core.tickets`. Registry helpers return `None` when a capability
is absent or has the wrong shape, so optional-module absence is normal runtime
state rather than a startup failure.
## Routing
Tickets sends a bounded, tenant-scoped `TicketRoutingRequest` containing the
ticket reference, type, priority, title, receive time, optional queue hint, and
non-secret attributes. The provider returns its identity and may return a queue
reference, timezone-aware service target, human-readable explanation, and
bounded metadata.
The provider is advisory. Tickets snapshots any returned queue and target into
its own record and history. An absent provider, a no-match plan, or an absent
queue must not prevent ticket intake; authorized staff can route manually.
Providers must not persist a second ticket lifecycle.
## Case Escalation
Tickets sends a `TicketCaseEscalationCommand` with stable tenant, ticket, and
display references, the requested Case type, actor-visible handoff note,
timezone-aware occurrence time, and an idempotency key. The provider returns a
stable Case identifier, number, bounded application-relative URL, replay flag,
and bounded metadata.
Providers must:
- recheck tenant and Case-creation authorization;
- reject an absent or inactive requested Case type;
- make identical retries resolve the same Case;
- preserve the Ticket reference in governed Case context; and
- return only an application-relative path, never an untrusted external URL.
Tickets records the result and its own escalation evidence. Cases remains the
authority for the formal procedure; Tickets remains the authority for the
operational request. Creating a Case does not merge or silently close either
lifecycle.
## Failure And Transaction Semantics
Capability calls receive the caller's active persistence session so a concrete
provider can participate in the same unit of work. Authorization and validation
errors fail the requested routing/escalation mutation explicitly. The caller
must still apply its own permission checks, tenant boundary, replay protection,
and immutable evidence rules.
+150
View File
@@ -0,0 +1,150 @@
# Product UI design principles
These rules govern every GovOPlaN module, including administration, tenant and
user settings, public forms, widgets, dialogs and shared shell surfaces. Core
owns the reusable components; modules own their meaning and authorized behavior.
They complement the [page composition contract](PAGE_LAYOUT_USAGE_GUIDELINES.md).
Initial rule revision: **UI-2026-09-08**. Record this identifier in review evidence;
subsequent rule changes need a dated change reference and a UI-09 propagation check.
## UI-01 — Help belongs to visible text
Place a small documentation book immediately after the heading or label whose
meaning it explains. For example, the Dashboard book belongs to **Dashboard**,
not the Reload, Edit or Save group. The same applies to page, pane, card and
dialog titles, field labels, widget summaries and contextual explanations.
Do not put an isolated documentation icon in an action bar or in an otherwise
empty row. A navigation icon is not a contextual documentation link.
Use `titleHelp={<DocumentationHelpLink reference={...} />}` on `PageLayout`,
`PageHeader`, `PageTitle`, `AdminPageLayout`, `Card` or `Dialog`. Full-canvas
workspaces use the visible `title` and `titleHelp` of `WorkspaceActionBar` (or
`PageActionBar`), separate from its action groups. Use `TextWithHelp` around
existing text for smaller contexts; fields use `FormField`/`FieldLabel` and
their documentation reference. Do not duplicate the heading to add help.
Dashboard widgets provide their owning `documentation` reference in
`DashboardWidgetContribution`; Dashboard renders it beside the existing widget
card title. Do not add a second widget heading or a footer-only documentation icon.
Keep the icon outside the heading's accessible name and outside other buttons
or links. It needs an accessible purpose, keyboard focus and a usable hit area.
Long German headings must wrap without colliding with actions. Collapsing a
card or configuring a dashboard must not change which title the book explains.
The existing optional Docs-module/hosted-documentation fallback remains intact;
placement does not change permissions or require the Docs module.
## UI-02 — Display first; edit deliberately
Ordinary overview and detail pages show compact, readable data. Group related
facts with shared description, summary and card components. Offer a clear Edit
action that opens a scoped dialog for a coherent group of settings. Avoid
presenting every possible input, select and save control simultaneously.
The Campaign dashboard is a priority application: show the effective settings
and their state compactly, then edit related settings in focused dialogs. This
is review work to implement, not a claim that all current screens already comply.
An explicit editing mode is a justified exception for genuinely broad editing,
large tables, graphical designers or other tasks poorly served by a modal.
Record why the exception helps the user. Keep reading and editing distinguishable;
enter editing deliberately and preserve shared Save/Cancel, validation, conflict,
permission and unsaved-draft behavior. Filters, search and selection do not need
an edit dialog: they change the view rather than the underlying data.
Cancel/Close must work even without changes. Never save a draft merely because
a dialog opens, selection changes or navigation happens. Commit only the intended
fields; do not let unrelated incomplete settings block an independent edit.
## UI-03 — Predictable actions and consequences
Follow the shared semantic page/pane action contract: context on the left;
Reload on refreshable surfaces followed by Create on the right. Editable drafts
have the standard state, Cancel/Discard and Save controls. Keep positions stable
across selection and loading changes, and explain unavailable applicable actions.
Separate destructive actions visually and semantically. Reload is a read, not
an implicit synchronization, import, send or other mutation.
## UI-04 — Shared geometry and readable collections
Use central page, card, table, form-grid and dialog primitives. Table cards use
`bodyLayout="table"`, spanning the card body without negative margins or expanded
`calc()` widths. Keep row actions reachable and deliberate column resizing
predictable. Avoid redundant taglines, arbitrary gaps, fixed dialog widths and
horizontal scrolling where content can wrap. Use actual pagination and shared
list filters; every result and every validation detail must remain reachable.
## UI-05 — Stable feedback and loading
Preserve useful loaded content on refresh failure; distinguish initial loading,
empty, unavailable, stale, blocked, failed and successful states. Use the shared
scoped loading overlay. Long operations show truthful processed/total and outcome
counts when known, not simulated progress or whole-page background reloads.
Prevent duplicate submissions without trapping the user in a clean editing mode.
## UI-06 — Predictable navigation and selection
Use shared breadcrumbs, side rail and tree contracts. Tree disclosure controls
expand/collapse; labels select. Track the clicked occurrence when one topic or
record appears in several branches. Preserve context, filters and selection
where appropriate, without carrying data across authorization or tenant scopes.
## UI-07 — Accessibility and German parity
Check keyboard order, visible focus, accessible names, dialog focus restoration,
contrast, zoom and narrow layouts. Status must not rely on color alone. Use
plain, task-oriented wording and complete German UI and module-owned help for
the changed workflow. Long labels must not hide actions or force needless scroll.
## UI-08 — Integrity and permissions are not visual shortcuts
Preserve authorization, versioning, audit evidence, immutable historical data and
safe retries. Do not remove a guard to make an action look available. Keep drafts
recoverable after failures and conflicts. Loading, cancellation and dialog
transitions must not silently write, discard or overwrite data. Never verify UI
changes by sending real mail or changing production records without authorization.
## UI-09 — Review incrementally and propagate lessons
The [Meta UI-review epic](https://git.add-ideas.de/GovOPlaN/govoplan/issues/56)
links one review ledger per module. Each starts pending, including modules
touched by a focused shared-component pass. An implemented help-placement rule
does not constitute a full module review.
Inventory the module's surfaces, inspect them with realistic data and permissions,
record findings and decisions, implement bounded fixes, then attach automated
and manual evidence. Distinguish **to do**, **implemented**, **verified**, and
**accepted exception**. Record the applicable principle IDs and actual screen
coverage; do not mark a module complete from a code search alone.
When a review reveals a reusable lesson, update this document and the central
component/check first where possible. Record the rule change in Meta and create
a follow-up or reopen affected already-reviewed modules. The review ledger must
show which rule revision each module has actually verified. Prefer finishing
the active review slice before expanding functionality, while recording unrelated
bugs and decision-dependent work without losing them.
## Kurzfassung auf Deutsch
- **UI-01:** Das Dokumentationsbuch steht direkt rechts neben der zugehörigen
Überschrift oder Beschriftung, nicht zwischen Aktionsschaltflächen.
- **UI-02:** Daten zunächst kompakt anzeigen; zusammengehörige Einstellungen
gezielt in einem Dialog bearbeiten. Ein ausdrücklich aktivierter
Bearbeitungsmodus, etwa für große Tabellen, ist eine begründete Ausnahme.
Abbrechen funktioniert auch ohne Änderungen; Änderungen werden bewusst gespeichert.
- **UI-03UI-08:** Einheitliche Aktionen und Abstände, erreichbare Tabelleninhalte,
verständlicher Fortschritt, vorhersehbare Navigation, Barrierefreiheit und
deutsche Texte dürfen Berechtigungen oder Datenintegrität nicht schwächen.
- **UI-09:** Alle Module werden einzeln geprüft. Neue Gestaltungsregeln werden
auch in bereits geprüften Modulen nachgezogen und mit Nachweisen dokumentiert.
## Automated coverage and its limits
`tools/checks/check-heading-help.mjs` in Meta checks JSX documentation-link
placement throughout the workspace, including aliases and simple local variables.
Core component and browser tests check heading association, accessible names,
focus, collapse/loading transitions and narrow/wide layouts. Existing layout,
action, dialog and DataGrid checks continue to apply. These tests protect shared
contracts; they do not certify every module's usability or replace the individual
screen reviews and their documented exceptions.
+17 -6
View File
@@ -51,10 +51,14 @@ contestability, responsibility, and traceability at the point of action.
| UX-025 | `window.alert` and the global `alert` function are prohibited. A narrowly necessary exception requires product-owner authorization and an entry in the alert exception register before implementation. | Accepted | All WebUI code |
| UX-026 | A table defines one stable ordered action set. A row-level unavailable action remains in its normal position and is disabled, preferably with `disabledReason`; structurally irrelevant actions are omitted for the entire table. Empty rows reserve the same slots so their Add action stays in the normal left-most action position. | Accepted | All structured tables |
| UX-027 | The platform icon rail keeps its brand header and utility footer visible. Only the module-navigation region scrolls when installed and permitted modules exceed the available viewport height. | Accepted | Core WebUI shell |
| UX-028 | Maintenance and offline state change the titlebar surface and repeat a quiet status label behind its controls. They must not replace, cover, or intercept the centered global-search surface; an accessible status control remains in the leading titlebar area. | Accepted | Core WebUI shell |
| UX-028 | Maintenance and offline states use their established textual warning banners centered in the titlebar. They must not recolor the shell or add decorative status icons. Global search is a right-side command, so warnings do not replace its trigger or overlay. | Accepted | Core WebUI shell |
| UX-029 | Recoverable page and module errors use the central compact `DismissibleAlert` presentation with an explicit recovery action where one exists. Full-height workspaces must overlay page feedback instead of allowing an alert to become a stretched workspace row. | Accepted | Core and module WebUIs |
| UX-030 | At narrow widths, the titlebar uses separate context and command rows. Context selectors remain horizontally reachable, search retains its compact trigger, and language/help/notification/account commands remain fixed icon controls without overlap. Shared content padding contracts so domain workspaces retain usable width. | Accepted | Core WebUI shell and all module workspaces |
| UX-031 | Public controls and extension contributions use stable, module-namespaced interface identities. Shared controls expose `interfaceId` and `helpTopicId`; generated source anchors are inventory evidence, not a substitute for an explicit ID when documentation, policy, or automation refers to the control. | Accepted | Core and module WebUIs |
| UX-032 | `F1` resolves help from the focused field or action, then its dialog/section/page and registered route. Focused contexts retain the page fallback; Docs applies audience and permission filtering and falls back to visible module documentation. | Accepted | Core shell, Docs, and all module WebUIs |
| UX-033 | Global search is the left-most titlebar command, immediately before language selection. Its icon, `F3`, and `Ctrl`/`Cmd`+`K` all open the same permission-aware search overlay; the titlebar does not reserve a persistent query field. | Accepted | Core shell and Search WebUI |
| UX-034 | Every headed `PageLayout` declares one of `overview`, `collection`, `detail`, `editor`, or `workspace` independently from its standalone/workspace/embedded geometry. Its actions use the matching semantic `PageActionBar`: refreshable pages provide Reload in the right-aligned trailing group immediately before Create/primary actions; collections keep Create far right; read-only pages do not invent Save. The trailing placement supersedes the earlier leading-Reload rule (2026-09-07, Core #295). | Accepted | Core and all module WebUIs |
| UX-035 | Editor action bars expose clean, dirty, and saving state; always retain Discard immediately before the far-right Save; centrally disable both while clean or saving; and participate in the unsaved-change navigation guard. Danger actions occupy the explicit separated destructive group after ordinary actions and before editor persistence. | Accepted | Core and all module WebUIs |
## Confirmed Implementation Decisions
@@ -161,14 +165,19 @@ Decision: the WebUI shell exposes a small, stable appearance contract based on
shared CSS tokens and persisted user preference selection.
- Core applies `system`, `light`, and `dark` preferences at the document root.
- Core applies validated user accent presets through `data-palette`; palette
values change semantic tokens globally and never require module CSS changes.
- Core owns shared tokens such as `--bg`, `--bar`, `--panel`, `--surface`,
`--line`, `--line-dark`, `--text`, `--text-strong`, `--muted`, semantic
status colors, radii, shadows, and disabled-control colors.
- Modules must style new UI with these tokens and shared controls. Module-local
CSS may tune layout and spacing, but it must not introduce a separate
appearance system.
- Appearance controls live in user settings first. Tenant defaults and policy
enforcement can be added later without changing the token contract.
- Appearance controls live in user settings. A personal palette wins over
unlocked tenant and system defaults; system and tenant locks take precedence.
Advanced personal token overrides additionally require system opt-in and may
be narrowed by tenant policy. Their versioned import/export document is
validated and applied all-or-nothing in both light and dark modes.
- Visual preview in settings is illustrative; it must reflect token families,
not become a second theme implementation.
@@ -227,7 +236,9 @@ instead of reproducing their behavior.
not self-explanatory.
- `help` content is contextual guidance, not the accessible name. The persisted
`show_inline_help_hints` user preference hides only the `InlineHelp` marker by
applying `ui-hide-help-hints` at the document root.
applying `ui-hide-help-hints` at the document root. When shown, the shared
marker is a labelled, keyboard-focusable help control and exposes its tooltip
on focus as well as pointer hover.
- Shared action-bearing components accept an optional disabled reason. In
particular, `MailServerSettingsPanel` forwards protocol-specific test
blockers into the shared focusable disabled-action tooltip; modules provide
@@ -319,7 +330,7 @@ converted or reviewed.
| Configuration packages | `govoplan-admin` | Catalog/import work exists, but package editing can still drift toward technical fields. | Add guided import/review/problem-list flow. |
| Retention and privacy | `govoplan-core` | Typed effective-policy editor exposes source paths, narrowing semantics, platform locks, permission/target blockers, and explicit clean/loading/save states. | Broader governed-change review remains module-owned where a policy change requires approval. |
| API keys | `govoplan-access` / admin UI | Security-sensitive creation needs least-privilege guidance. | Add scoped creation wizard with expiry/owner review. |
| User settings | `govoplan-core` | Simple typed sections use unsaved-change guards, quiet result feedback, contextual help, and explicit busy/clean disabled-action reasons. | Keep bounded; new contributed sections must satisfy the checklist. |
| User settings | `govoplan-core` | Simple typed sections use unsaved-change guards, quiet result feedback, contextual help, explicit busy/clean disabled-action reasons, and an effective appearance source. Palette selection and light/dark preview are shared with system and tenant administration. | Keep bounded; new contributed sections must satisfy the checklist. |
## Impact Index
@@ -335,7 +346,7 @@ converted or reviewed.
| Automation/workflow commands | Hidden side effects would undermine accountability. | Action/effect preview, system-actor display, command record, retry/quarantine/manual states, and audit links. |
| Postbox and encrypted communication | Retraction and access can be misunderstood. | Honest key-fetch/decryption state, expiry limits, recipient/device access provenance, and delivery evidence. |
| API keys | Security-sensitive creation and scope selection. | Scoped creation wizard, least-privilege suggestions, clear expiry/owner explanation. |
| User settings | Needs clarity and persistence across profile/interface/preferences. | Simple settings sections with immediate feedback and no double-click navigation traps. |
| User settings | Needs clarity and persistence across profile/interface/preferences. | Simple settings sections with immediate feedback, explicit inherit/reset semantics, effective-source provenance, and no double-click navigation traps. |
## Review Checklist
+38
View File
@@ -59,6 +59,14 @@ The first budgeted full-product build reported:
## Verification
Development startup explicitly prebundles the Excel reader's browser/universal
entrypoints and the lazy rich-text editor's Tiptap dependencies. These are
Core-installed vendor dependencies, not eager optional-module imports. This
avoids first-time Campaign/Template navigation triggering a second dependency
optimization and page reload. Module descriptors and pages remain lazy, and
production bundle budgets remain unchanged. The Core interface-pattern check
verifies this include list and keeps optional GovOPlaN modules excluded.
```bash
cd /mnt/DATA/git/govoplan-core/webui
npm run build
@@ -69,3 +77,33 @@ npm run test:module-permutations
The build gate also catches accidental eager imports: a page pulled into the
entry closure consumes the initial budget, while an oversized page or module
descriptor consumes the asynchronous chunk budget.
The startup shell imports appearance validation/application from the pure
`appearanceOverrides.ts` runtime. Settings-only color controls, JSON import/export,
and previews remain in `AppearanceOverridesEditor.tsx` behind the existing lazy
Settings route. Importing a runtime helper from a module that also owns editor
components can accidentally pull the entire editor into the startup chunk.
Public helper exports remain compatible; theme application is still synchronous.
The versioned default color document and its deep-clone helper live in
`appearanceOverrideDefaults.ts`, loaded with that editor. Applying saved overrides
does not load editor defaults or construct a draft. The default values and public
helper names are unchanged; the theme regression checks independent draft clones
as well as synchronous validation, application, and reset.
`PasswordField` keeps ordinary input and reveal controls synchronous. Its
optional `PasswordGeneratorDialog` is imported only after an enabled, editable
generator is explicitly opened, not for every sign-in/password field. Loading
and failures use the shared resource boundary; the underlying field remains
usable. Closing or revoking generation while loading cannot apply a candidate.
The secure browser RNG, generation policy, public exports, and explicit
"Use password" confirmation remain unchanged. The isolated browser fixture
does not import the Core barrel, so it can verify that the generator is not
requested before opening it, along with cancel/use and focus restoration.
Deutsch: Normale Passworteingabe und Sichtbarkeitssteuerung bleiben unmittelbar
verfügbar. Der optionale Generator wird erst beim bewussten Öffnen eines
aktivierten, bearbeitbaren Felds geladen; Lade- und Fehlerzustände nutzen die
gemeinsame Ressourcenanzeige. Ohne "Passwort verwenden" wird kein Kandidat
übernommen. Sichere Browser-Zufallszahlen, Richtlinien und öffentliche
Schnittstellen bleiben unverändert. Wird die Generierung während des Ladens
deaktiviert, öffnet eine verspätete Antwort keinen Dialog.
File diff suppressed because it is too large Load Diff
+3 -3
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan-core"
version = "0.1.15"
version = "0.1.46"
description = "Reusable GovOPlaN platform core, access, tenancy, and RBAC components."
readme = "README.md"
requires-python = ">=3.12"
@@ -15,7 +15,7 @@ dependencies = [
"fastapi>=0.139,<1",
"pydantic>=2,<3",
"pydantic-settings>=2,<3",
"cryptography>=48.0.1,<50",
"cryptography>=50.0.0,<51",
"celery>=5,<6",
"redis>=5,<6",
"alembic>=1,<2",
@@ -26,7 +26,7 @@ dependencies = [
where = ["src"]
[tool.setuptools.package-data]
govoplan_core = ["py.typed"]
govoplan_core = ["py.typed", "resources/*.json"]
[tool.setuptools.data-files]
"govoplan_core_runtime" = ["alembic.ini"]
+1 -2
View File
@@ -12,7 +12,7 @@ class SystemSettings(Base, TimestampMixin):
__tablename__ = "core_system_settings"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default="global")
default_locale: Mapped[str] = mapped_column(String(20), default="en", nullable=False)
default_locale: Mapped[str] = mapped_column(String(20), default="de", nullable=False)
allow_tenant_custom_groups: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
allow_tenant_custom_roles: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
allow_tenant_api_keys: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
@@ -20,4 +20,3 @@ class SystemSettings(Base, TimestampMixin):
__all__ = ["SystemSettings"]
+73 -5
View File
@@ -3,7 +3,9 @@ from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
from pydantic import BaseModel, ConfigDict, Field, model_validator
from govoplan_core.core.appearance import normalize_appearance_overrides
class AuditLogItemResponse(BaseModel):
@@ -84,7 +86,7 @@ class TenantInfo(BaseModel):
slug: str
name: str
is_active: bool = True
default_locale: str = "en"
default_locale: str = "de"
enabled_language_codes: list[str] = Field(default_factory=list)
@@ -93,6 +95,53 @@ class TenantMembershipInfo(TenantInfo):
is_active: bool = True
class NavigationSeparatorPayload(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str = Field(pattern=r"^separator:[a-zA-Z0-9_.:-]+$", max_length=255)
label: str = Field(default="", max_length=120, pattern=r"^[^\x00-\x1f]*$")
class NavigationPreferencesPayload(BaseModel):
model_config = ConfigDict(extra="forbid")
contract_version: Literal["1"] = "1"
order: list[str] = Field(default_factory=list, max_length=256)
hidden: list[str] = Field(default_factory=list, max_length=256)
locked: list[str] = Field(default_factory=list, max_length=256)
separators: list[NavigationSeparatorPayload] | None = Field(default=None, max_length=256)
class AppearanceModeOverrides(BaseModel):
model_config = ConfigDict(extra="forbid")
accent: str
accent_foreground: str
surface: str
surface_foreground: str
success: str
success_foreground: str
info: str
info_foreground: str
warning: str
warning_foreground: str
danger: str
danger_foreground: str
class AppearanceOverridesDocument(BaseModel):
model_config = ConfigDict(extra="forbid")
schema_version: Literal["1"] = "1"
light: AppearanceModeOverrides
dark: AppearanceModeOverrides
@model_validator(mode="after")
def validate_accessibility(self) -> "AppearanceOverridesDocument":
normalize_appearance_overrides(self.model_dump(mode="json"))
return self
class UserUiPreferences(BaseModel):
model_config = ConfigDict(extra="ignore")
@@ -101,6 +150,20 @@ class UserUiPreferences(BaseModel):
reduce_motion: bool = False
sticky_section_sidebars: bool = True
theme: Literal["system", "light", "dark"] = "system"
palette: Literal["default", "civic_blue", "forest", "plum"] | None = None
appearance_overrides: AppearanceOverridesDocument | None = None
navigation: NavigationPreferencesPayload | None = None
class EffectiveAppearanceInfo(BaseModel):
palette: Literal["default", "civic_blue", "forest", "plum"] = "default"
source: Literal["user", "tenant", "system", "tenant_lock", "system_lock"] = "system"
locked: bool = False
system_default_palette: Literal["default", "civic_blue", "forest", "plum"] = "default"
tenant_default_palette: Literal["default", "civic_blue", "forest", "plum"] | None = None
inherited_palette: Literal["default", "civic_blue", "forest", "plum"] = "default"
custom_overrides: AppearanceOverridesDocument | None = None
custom_overrides_allowed: bool = False
class UserInfo(BaseModel):
@@ -113,9 +176,12 @@ class UserInfo(BaseModel):
tenant_display_name: str | None = None
is_tenant_admin: bool = False
password_reset_required: bool = False
required_auth_action: Literal["change_password"] | None = None
local_password: bool = False
preferred_language: str | None = None
enabled_language_codes: list[str] = Field(default_factory=list)
ui_preferences: UserUiPreferences = Field(default_factory=UserUiPreferences)
appearance: EffectiveAppearanceInfo = Field(default_factory=EffectiveAppearanceInfo)
class AuthSessionUserInfo(BaseModel):
@@ -126,6 +192,8 @@ class AuthSessionUserInfo(BaseModel):
tenant_display_name: str | None = None
is_tenant_admin: bool = False
password_reset_required: bool = False
required_auth_action: Literal["change_password"] | None = None
local_password: bool = False
class AuthSessionResponse(BaseModel):
@@ -210,7 +278,7 @@ class AuthProfileResponse(BaseModel):
active_tenant: TenantInfo
available_languages: list[LanguageInfo] = Field(default_factory=list)
enabled_language_codes: list[str] = Field(default_factory=list)
default_language: str = "en"
default_language: str = "de"
profile_loaded: bool = True
@@ -239,7 +307,7 @@ class LoginResponse(BaseModel):
principal: PrincipalContextInfo | None = None
available_languages: list[LanguageInfo] = Field(default_factory=list)
enabled_language_codes: list[str] = Field(default_factory=list)
default_language: str = "en"
default_language: str = "de"
profile_loaded: bool = True
roles_loaded: bool = True
groups_loaded: bool = True
@@ -257,7 +325,7 @@ class MeResponse(BaseModel):
principal: PrincipalContextInfo | None = None
available_languages: list[LanguageInfo] = Field(default_factory=list)
enabled_language_codes: list[str] = Field(default_factory=list)
default_language: str = "en"
default_language: str = "de"
profile_loaded: bool = True
roles_loaded: bool = True
groups_loaded: bool = True
+106 -10
View File
@@ -18,7 +18,9 @@ from celery.signals import (
from govoplan_core.core.campaigns import (
CAPABILITY_CAMPAIGNS_DELIVERY_TASKS,
CAPABILITY_CAMPAIGNS_SCHEDULES,
CampaignDeliveryTaskProvider,
CampaignScheduleProvider,
)
from govoplan_core.core.calendar import (
CAPABILITY_CALENDAR_OUTBOX,
@@ -100,6 +102,7 @@ celery.conf.update(
task_routes={
"govoplan.campaigns.send_email": {"queue": "send_email"},
"govoplan.campaigns.append_sent": {"queue": "append_sent"},
"govoplan.campaigns.dispatch_schedules": {"queue": "default"},
"govoplan.notifications.deliver": {"queue": "notifications"},
"govoplan.notifications.deliver_pending": {"queue": "notifications"},
"govoplan.mail.dispatch_outbox": {"queue": "mail"},
@@ -132,6 +135,11 @@ celery.conf.update(
"schedule": 60.0,
"args": (None, 100),
},
"campaign-schedules-every-minute": {
"task": "govoplan.campaigns.dispatch_schedules",
"schedule": 60.0,
"args": (None, 50),
},
"mail-outbox-every-five-seconds": {
"task": "govoplan.mail.dispatch_outbox",
"schedule": 5.0,
@@ -555,6 +563,18 @@ def _campaign_delivery_tasks(
return capability
def _campaign_schedules(
registry: PlatformRegistry | None = None,
) -> CampaignScheduleProvider | None:
registry = registry or _platform_registry()
if not registry.has_capability(CAPABILITY_CAMPAIGNS_SCHEDULES):
return None
capability = registry.require_capability(CAPABILITY_CAMPAIGNS_SCHEDULES)
if not isinstance(capability, CampaignScheduleProvider):
raise RuntimeError("Campaign schedule capability is invalid")
return capability
def _notification_dispatch(
registry: PlatformRegistry | None = None,
) -> NotificationDispatchProvider:
@@ -699,13 +719,70 @@ def _idm_assignment_lifecycle(
return capability
@celery.task(
name="govoplan.campaigns.dispatch_schedules",
bind=True,
max_retries=0,
)
def dispatch_campaign_schedules(
self,
tenant_id: str | None = None,
limit: int = 50,
):
"""Prepare manual drafts or governed autonomous Mail commands for due schedules."""
from govoplan_core.db.session import get_database
with get_database().SessionLocal() as session:
registry = _platform_registry()
defaults = {
"selected": 0,
"prepared": 0,
"autonomous_prepared": 0,
"failed": 0,
"completed": 0,
"coalesced": 0,
"duplicates": 0,
"deferred": 0,
"campaign_ids": [],
"operator_actions": [],
"refreshed": {
"checked": 0,
"accepted": 0,
"uncertain": 0,
"failed": 0,
"skipped": 0,
},
}
if not registry.has_capability(CAPABILITY_CAMPAIGNS_SCHEDULES):
return defaults
result = _run_tenant_worker_batches(
registry,
session,
capability_name=CAPABILITY_CAMPAIGNS_SCHEDULES,
tenant_id=tenant_id,
operation=lambda effective_tenant_id: _campaign_schedules(
registry
).dispatch_due( # type: ignore[union-attr]
session,
tenant_id=effective_tenant_id,
limit=limit,
),
defaults=defaults,
work_state="new",
)
session.commit()
return result
@celery.task(name="govoplan.campaigns.send_email", bind=True, max_retries=0)
def send_email(self, job_id: str):
"""Send one explicitly queued campaign job.
SMTP failures are persisted but are not retried implicitly. A worker-loss
redelivery is safe because the delivery service converts an unfinished
SMTP attempt into ``outcome_unknown`` instead of transmitting again.
redelivery leaves an active delivery claim unchanged instead of transmitting
again. Explicit fenced recovery requires stopped-owner evidence before an
abandoned attempt can become ``outcome_unknown`` for reconciliation.
"""
from govoplan_core.db.session import get_database
@@ -1122,7 +1199,7 @@ def dispatch_postbox_routes(
tenant_id: str | None = None,
limit: int = 50,
):
"""Deliver due Postbox vacancy escalations from durable route rows."""
"""Deliver due routes and reconcile assignment-derived notification facts."""
from govoplan_core.db.session import get_database
@@ -1136,21 +1213,40 @@ def dispatch_postbox_routes(
"cancelled": 0,
"failed": 0,
"route_ids": [],
"lifecycle_scanned": 0,
"lifecycle_changed": 0,
"lifecycle_events": 0,
"lifecycle_notifications": 0,
"lifecycle_notification_failures": 0,
}
if not registry.has_capability(CAPABILITY_POSTBOX_ROUTING):
return defaults
provider = _postbox_routing_provider(registry)
def dispatch_tenant(effective_tenant_id: str) -> Mapping[str, object]:
route_result = provider.dispatch_due_routes( # type: ignore[union-attr]
session,
tenant_id=effective_tenant_id,
limit=limit,
)
lifecycle_result = provider.reconcile_notification_lifecycle( # type: ignore[union-attr]
session,
tenant_id=effective_tenant_id,
limit=limit,
)
return {
**route_result,
**{
f"lifecycle_{key}": value for key, value in lifecycle_result.items()
},
}
result = _run_tenant_worker_batches(
registry,
session,
capability_name=CAPABILITY_POSTBOX_ROUTING,
tenant_id=tenant_id,
operation=lambda effective_tenant_id: _postbox_routing_provider(
registry
).dispatch_due_routes( # type: ignore[union-attr]
session,
tenant_id=effective_tenant_id,
limit=limit,
),
operation=dispatch_tenant,
defaults=defaults,
)
session.commit()
+114
View File
@@ -24,6 +24,10 @@ CAPABILITY_ACCESS_TENANT_PROVISIONER = "access.tenantProvisioner"
CAPABILITY_ACCESS_FIRST_ADMIN_PROVISIONER = "access.firstAdminProvisioner"
CAPABILITY_ACCESS_ADMINISTRATION = "access.administration"
CAPABILITY_ACCESS_GOVERNANCE_MATERIALIZER = "access.governanceMaterializer"
CAPABILITY_ACCESS_GOVERNANCE_PROJECTION_V1 = "access.governanceProjection.v1"
CAPABILITY_POLICY_ACCESS_EXPLANATION_SUBJECTS = (
"policy.access_explanation_subjects"
)
CAPABILITY_TENANCY_TENANT_RESOLVER = "tenancy.tenantResolver"
CAPABILITY_AUDIT_SINK = "audit.sink"
CAPABILITY_AUDIT_RECORDER = "audit.recorder"
@@ -49,6 +53,7 @@ ACCESS_CAPABILITY_NAMES = frozenset(
CAPABILITY_ACCESS_FIRST_ADMIN_PROVISIONER,
CAPABILITY_ACCESS_ADMINISTRATION,
CAPABILITY_ACCESS_GOVERNANCE_MATERIALIZER,
CAPABILITY_ACCESS_GOVERNANCE_PROJECTION_V1,
CAPABILITY_TENANCY_TENANT_RESOLVER,
CAPABILITY_AUDIT_SINK,
CAPABILITY_AUDIT_RECORDER,
@@ -177,6 +182,15 @@ class PrincipalRef:
)
@dataclass(frozen=True, slots=True)
class AccessExplanationSubjectDecision:
allow_other_users: bool
reason: str
source: str
required_scope: str | None = None
provenance: Mapping[str, object] = field(default_factory=dict)
def _optional_str(value: object | None) -> str | None:
return str(value) if value is not None else None
@@ -378,6 +392,82 @@ class GovernanceTemplateMaterialization:
required: bool = False
GovernanceProjectionOperation = Literal["upsert", "remove"]
GovernanceProjectionStatus = Literal[
"created",
"updated",
"unchanged",
"removed",
"absent",
"blocked",
"failed",
]
@dataclass(frozen=True, slots=True)
class GovernanceProjectionCommand:
"""Stable Access-owned input for one governance assignment projection."""
assignment_id: str
operation: GovernanceProjectionOperation
template: GovernanceTemplateMaterialization
provenance: Mapping[str, str] = field(default_factory=dict)
def __post_init__(self) -> None:
if not self.assignment_id or len(self.assignment_id) > 255:
raise ValueError("Governance projection assignment ids must contain at most 255 characters.")
if len(self.provenance) > 20:
raise ValueError("Governance projection provenance supports at most 20 entries.")
for key, value in self.provenance.items():
if not key or len(key) > 100 or len(value) > 500:
raise ValueError("Governance projection provenance entries exceed their bounds.")
@dataclass(frozen=True, slots=True)
class GovernanceProjectionBatch:
"""Versioned, bounded reconciliation request independent of Admin internals."""
operation_id: str
commands: tuple[GovernanceProjectionCommand, ...]
version: Literal["1"] = "1"
dry_run: bool = False
def __post_init__(self) -> None:
if not self.operation_id or len(self.operation_id) > 255:
raise ValueError("Governance projection operation ids must contain at most 255 characters.")
if not self.commands or len(self.commands) > 500:
raise ValueError("Governance projection batches must contain between 1 and 500 commands.")
assignment_ids = [command.assignment_id for command in self.commands]
if len(assignment_ids) != len(set(assignment_ids)):
raise ValueError("Governance projection assignment ids must be unique within a batch.")
@dataclass(frozen=True, slots=True)
class GovernanceProjectionOutcome:
assignment_id: str
template_id: str
tenant_id: str
kind: Literal["group", "role"]
operation: GovernanceProjectionOperation
status: GovernanceProjectionStatus
resource_id: str | None = None
blocker_codes: tuple[str, ...] = ()
message: str | None = None
provenance: Mapping[str, str] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class GovernanceProjectionResult:
operation_id: str
outcomes: tuple[GovernanceProjectionOutcome, ...]
version: Literal["1"] = "1"
dry_run: bool = False
@property
def blocked(self) -> tuple[GovernanceProjectionOutcome, ...]:
return tuple(item for item in self.outcomes if item.status in {"blocked", "failed"})
@dataclass(frozen=True, slots=True)
class AuditEvent:
event_type: str
@@ -564,6 +654,18 @@ class AccessExplanationService(Protocol):
...
@runtime_checkable
class AccessExplanationSubjectPolicy(Protocol):
def decide_subject_selection(
self,
session: object,
principal: PrincipalRef,
*,
tenant_id: str,
) -> AccessExplanationSubjectDecision:
...
@runtime_checkable
class TenantAccessProvisioner(Protocol):
def ensure_default_roles(self, session: object, tenant: object | None = None) -> Mapping[str, object]:
@@ -669,6 +771,18 @@ class AccessGovernanceMaterializer(Protocol):
...
@runtime_checkable
class AccessGovernanceProjectionV1(Protocol):
"""Bulk reconciliation boundary for Admin-owned governance assignments."""
def reconcile(
self,
session: object,
batch: GovernanceProjectionBatch,
) -> GovernanceProjectionResult:
...
@runtime_checkable
class AuditSink(Protocol):
def record(self, event: AuditEvent) -> None:
+223
View File
@@ -0,0 +1,223 @@
from __future__ import annotations
from dataclasses import dataclass
import re
from typing import Any, Literal, Mapping
AppearancePalette = Literal["default", "civic_blue", "forest", "plum"]
AppearanceSource = Literal["user", "tenant", "system", "tenant_lock", "system_lock"]
APPEARANCE_PALETTES: tuple[AppearancePalette, ...] = ("default", "civic_blue", "forest", "plum")
APPEARANCE_SETTINGS_KEY = "appearance"
APPEARANCE_OVERRIDE_SCHEMA_VERSION = "1"
APPEARANCE_OVERRIDE_TOKENS: tuple[str, ...] = (
"accent", "accent_foreground", "surface", "surface_foreground",
"success", "success_foreground", "info", "info_foreground",
"warning", "warning_foreground", "danger", "danger_foreground",
)
_STATUS_TOKENS = ("success", "info", "warning", "danger")
_HEX_COLOR = re.compile(r"^#[0-9a-fA-F]{6}$")
@dataclass(frozen=True, slots=True)
class EffectiveAppearance:
palette: AppearancePalette
source: AppearanceSource
locked: bool
system_default_palette: AppearancePalette
tenant_default_palette: AppearancePalette | None
inherited_palette: AppearancePalette
custom_overrides: dict[str, object] | None = None
custom_overrides_allowed: bool = False
def as_dict(self) -> dict[str, object]:
return {
"palette": self.palette,
"source": self.source,
"locked": self.locked,
"system_default_palette": self.system_default_palette,
"tenant_default_palette": self.tenant_default_palette,
"inherited_palette": self.inherited_palette,
"custom_overrides": self.custom_overrides,
"custom_overrides_allowed": self.custom_overrides_allowed,
}
def normalize_appearance_palette(value: object, *, fallback: AppearancePalette | None = None) -> AppearancePalette | None:
normalized = str(value or "").strip().lower()
return normalized if normalized in APPEARANCE_PALETTES else fallback # type: ignore[return-value]
def appearance_settings(settings: Mapping[str, Any] | None) -> tuple[AppearancePalette | None, bool]:
raw = settings.get(APPEARANCE_SETTINGS_KEY) if isinstance(settings, Mapping) else None
if not isinstance(raw, Mapping):
return None, False
return normalize_appearance_palette(raw.get("default_palette")), raw.get("palette_locked") is True
def appearance_custom_overrides_policy(settings: Mapping[str, Any] | None) -> bool | None:
raw = settings.get(APPEARANCE_SETTINGS_KEY) if isinstance(settings, Mapping) else None
if not isinstance(raw, Mapping) or "allow_custom_overrides" not in raw:
return None
return raw.get("allow_custom_overrides") is True
def update_appearance_custom_overrides_policy(
settings: Mapping[str, Any] | None,
*,
allowed: bool | None,
) -> dict[str, Any]:
updated = dict(settings or {})
appearance = dict(updated.get(APPEARANCE_SETTINGS_KEY) or {}) if isinstance(updated.get(APPEARANCE_SETTINGS_KEY), Mapping) else {}
if allowed is None:
appearance.pop("allow_custom_overrides", None)
else:
appearance["allow_custom_overrides"] = allowed
if appearance:
updated[APPEARANCE_SETTINGS_KEY] = appearance
else:
updated.pop(APPEARANCE_SETTINGS_KEY, None)
return updated
def normalize_appearance_overrides(value: object) -> dict[str, object] | None:
"""Validate and canonicalize the versioned, all-or-nothing color contract."""
if value is None:
return None
if not isinstance(value, Mapping):
raise ValueError("Appearance overrides must be an object.")
if set(value) != {"schema_version", "light", "dark"}:
raise ValueError("Appearance overrides must contain only schema_version, light, and dark.")
if str(value.get("schema_version")) != APPEARANCE_OVERRIDE_SCHEMA_VERSION:
raise ValueError("Unsupported appearance override schema version.")
normalized: dict[str, object] = {"schema_version": APPEARANCE_OVERRIDE_SCHEMA_VERSION}
for mode in ("light", "dark"):
raw_mode = value.get(mode)
if not isinstance(raw_mode, Mapping) or set(raw_mode) != set(APPEARANCE_OVERRIDE_TOKENS):
raise ValueError(f"Appearance override mode {mode} must define every supported token exactly once.")
colors: dict[str, str] = {}
for token in APPEARANCE_OVERRIDE_TOKENS:
color = str(raw_mode.get(token) or "").strip().lower()
if not _HEX_COLOR.fullmatch(color):
raise ValueError(f"Appearance override {mode}.{token} must be a six-digit hexadecimal color.")
colors[token] = color
_validate_mode_accessibility(mode, colors)
normalized[mode] = colors
return normalized
def _validate_mode_accessibility(mode: str, colors: Mapping[str, str]) -> None:
pairs = (
("accent", "accent_foreground"), ("surface", "surface_foreground"),
("success", "success_foreground"), ("info", "info_foreground"),
("warning", "warning_foreground"), ("danger", "danger_foreground"),
)
for background, foreground in pairs:
if _contrast_ratio(colors[background], colors[foreground]) < 4.5:
raise ValueError(f"Appearance override {mode}.{foreground} must have WCAG AA contrast against {mode}.{background}.")
status_colors = [colors[token] for token in _STATUS_TOKENS]
for index, first in enumerate(status_colors):
for second in status_colors[index + 1:]:
if _rgb_distance(first, second) < 12:
raise ValueError(f"Appearance override status colors in {mode} must remain visibly distinct.")
def _relative_luminance(color: str) -> float:
channels = [int(color[index:index + 2], 16) / 255 for index in (1, 3, 5)]
linear = [channel / 12.92 if channel <= 0.04045 else ((channel + 0.055) / 1.055) ** 2.4 for channel in channels]
return 0.2126 * linear[0] + 0.7152 * linear[1] + 0.0722 * linear[2]
def _contrast_ratio(first: str, second: str) -> float:
high, low = sorted((_relative_luminance(first), _relative_luminance(second)), reverse=True)
return (high + 0.05) / (low + 0.05)
def _rgb_distance(first: str, second: str) -> float:
first_channels = [int(first[index:index + 2], 16) for index in (1, 3, 5)]
second_channels = [int(second[index:index + 2], 16) for index in (1, 3, 5)]
return sum((left - right) ** 2 for left, right in zip(first_channels, second_channels, strict=True)) ** 0.5
def update_appearance_settings(
settings: Mapping[str, Any] | None,
*,
default_palette: AppearancePalette | None,
palette_locked: bool,
) -> dict[str, Any]:
updated = dict(settings or {})
appearance = dict(updated.get(APPEARANCE_SETTINGS_KEY) or {}) if isinstance(updated.get(APPEARANCE_SETTINGS_KEY), Mapping) else {}
if default_palette is None:
appearance.pop("default_palette", None)
else:
normalized = normalize_appearance_palette(default_palette)
if normalized is None:
raise ValueError("Unsupported appearance palette.")
appearance["default_palette"] = normalized
if palette_locked:
appearance["palette_locked"] = True
else:
appearance.pop("palette_locked", None)
if appearance:
updated[APPEARANCE_SETTINGS_KEY] = appearance
else:
updated.pop(APPEARANCE_SETTINGS_KEY, None)
return updated
def resolve_effective_appearance(
*,
system_settings: Mapping[str, Any] | None,
tenant_settings: Mapping[str, Any] | None,
user_settings: Mapping[str, Any] | None,
) -> EffectiveAppearance:
system_palette, system_locked = appearance_settings(system_settings)
system_palette = system_palette or "default"
tenant_palette, tenant_locked = appearance_settings(tenant_settings)
inherited_palette = tenant_palette or system_palette
raw_ui = user_settings.get("ui") if isinstance(user_settings, Mapping) else None
user_palette = normalize_appearance_palette(raw_ui.get("palette")) if isinstance(raw_ui, Mapping) else None
system_custom_policy = appearance_custom_overrides_policy(system_settings) is True
tenant_custom_policy = appearance_custom_overrides_policy(tenant_settings)
custom_overrides_allowed = system_custom_policy and tenant_custom_policy is not False and not system_locked and not tenant_locked
try:
custom_overrides = normalize_appearance_overrides(raw_ui.get("appearance_overrides")) if isinstance(raw_ui, Mapping) else None
except ValueError:
custom_overrides = None
if not custom_overrides_allowed:
custom_overrides = None
if system_locked:
return EffectiveAppearance(system_palette, "system_lock", True, system_palette, tenant_palette, system_palette)
if tenant_locked:
return EffectiveAppearance(inherited_palette, "tenant_lock", True, system_palette, tenant_palette, inherited_palette)
return EffectiveAppearance(
user_palette or inherited_palette,
"user" if user_palette else "tenant" if tenant_palette else "system",
False,
system_palette,
tenant_palette,
inherited_palette,
custom_overrides,
custom_overrides_allowed,
)
__all__ = [
"APPEARANCE_PALETTES",
"APPEARANCE_SETTINGS_KEY",
"APPEARANCE_OVERRIDE_SCHEMA_VERSION",
"APPEARANCE_OVERRIDE_TOKENS",
"AppearancePalette",
"AppearanceSource",
"EffectiveAppearance",
"appearance_settings",
"appearance_custom_overrides_policy",
"normalize_appearance_overrides",
"normalize_appearance_palette",
"resolve_effective_appearance",
"update_appearance_settings",
"update_appearance_custom_overrides_policy",
]
@@ -0,0 +1,81 @@
from __future__ import annotations
from collections.abc import Mapping
from datetime import datetime
from typing import Protocol, runtime_checkable
CAPABILITY_APPLICATION_STATUS_PROJECTION = "application_status.projection"
@runtime_checkable
class ApplicationStatusProjectionProvider(Protocol):
"""Bounded applicant-status access without exposing the owning module's data."""
def tenant_id_for_tracking_id(
self,
session: object,
*,
tracking_id: str,
) -> str | None:
...
def public_access_challenge(
self,
session: object,
*,
tracking_id: str,
) -> Mapping[str, object]:
...
def get_authenticated_projection(
self,
session: object,
principal: object,
*,
tracking_id: str,
observed_at: datetime,
) -> Mapping[str, object]:
...
def get_public_projection(
self,
session: object,
*,
tracking_id: str,
token: str | None,
observed_at: datetime,
) -> Mapping[str, object]:
...
def request_email_link(
self,
session: object,
*,
tracking_id: str,
email: str,
requested_at: datetime,
) -> bool:
...
def application_status_projection_provider(
registry: object | None,
) -> ApplicationStatusProjectionProvider | None:
if registry is None or not hasattr(registry, "has_capability"):
return None
if not registry.has_capability(CAPABILITY_APPLICATION_STATUS_PROJECTION):
return None
capability = registry.capability(CAPABILITY_APPLICATION_STATUS_PROJECTION)
return (
capability
if isinstance(capability, ApplicationStatusProjectionProvider)
else None
)
__all__ = [
"ApplicationStatusProjectionProvider",
"CAPABILITY_APPLICATION_STATUS_PROJECTION",
"application_status_projection_provider",
]
+30
View File
@@ -45,6 +45,15 @@ class CalendarEventRef:
outbox_operation_id: str | None = None
@dataclass(frozen=True, slots=True)
class CalendarEventReleaseRef:
event_id: str
accepted: bool = True
already_released: bool = False
external_state: str = "local_released"
outbox_operation_id: str | None = None
@dataclass(frozen=True, slots=True)
class CalendarInvitationAttendeeRequest:
address: str
@@ -156,6 +165,27 @@ class CalendarSchedulingProvider(Protocol):
) -> CalendarEventRef:
...
def promote_event(
self,
session: object,
*,
tenant_id: str,
user_id: str | None,
event_id: str,
request: CalendarEventRequest,
) -> CalendarEventRef:
...
def release_event(
self,
session: object,
*,
tenant_id: str,
user_id: str | None,
event_id: str,
) -> CalendarEventReleaseRef:
...
@runtime_checkable
class CalendarOutboxProvider(Protocol):
+155 -1
View File
@@ -3,14 +3,29 @@ from __future__ import annotations
from collections.abc import Callable, Iterable, Mapping
from dataclasses import dataclass, field
from datetime import datetime
from typing import Protocol, runtime_checkable
from typing import Literal, Protocol, runtime_checkable
CAPABILITY_CAMPAIGNS_MAIL_POLICY_CONTEXT = "campaigns.mailPolicyContext"
CAPABILITY_CAMPAIGNS_ACCESS = "campaigns.access"
CAPABILITY_CAMPAIGNS_POLICY_CONTEXT = "campaigns.policyContext"
CAPABILITY_CAMPAIGNS_DELIVERY_TASKS = "campaigns.deliveryTasks"
CAPABILITY_CAMPAIGNS_SCHEDULES = "campaigns.schedules"
CAPABILITY_CAMPAIGNS_RETENTION = "campaigns.retention"
CAPABILITY_CAMPAIGNS_WORK_ORCHESTRATION = "campaigns.workOrchestration"
CampaignWorkAssigneeKind = Literal[
"account",
"group",
"organization_function",
]
CampaignWorkHandoffStatus = Literal[
"open",
"in_progress",
"completed",
"rejected",
"cancelled",
]
@dataclass(frozen=True, slots=True)
@@ -31,6 +46,88 @@ class CampaignPolicyContext:
settings: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class CampaignWorkHandoffRequest:
"""Typed request used by Workflow to open accountable Campaign work."""
tenant_id: str
idempotency_key: str
purpose: str
assignee_kind: CampaignWorkAssigneeKind
assignee_id: str
campaign_id: str | None = None
create_external_id: str | None = None
create_name: str | None = None
create_description: str | None = None
expected_campaign_revision: int | None = None
due_at: datetime | None = None
mirror_to_tasks: bool = True
correlation_id: str | None = None
workflow_instance_id: str | None = None
workflow_step_id: str | None = None
def __post_init__(self) -> None:
for value, label in (
(self.tenant_id, "Campaign hand-off tenant"),
(self.idempotency_key, "Campaign hand-off idempotency key"),
(self.purpose, "Campaign hand-off purpose"),
(self.assignee_id, "Campaign hand-off assignee"),
):
if not value.strip():
raise ValueError(f"{label} is required")
references_existing = bool(self.campaign_id and self.campaign_id.strip())
creates_new = bool(
self.create_external_id
and self.create_external_id.strip()
and self.create_name
and self.create_name.strip()
)
if references_existing == creates_new:
raise ValueError(
"Campaign hand-offs must either reference one campaign or "
"declare one new campaign."
)
if self.expected_campaign_revision is not None and (
self.expected_campaign_revision < 1
):
raise ValueError("Expected Campaign revisions start at one")
if self.due_at is not None and self.due_at.tzinfo is None:
raise ValueError("Campaign hand-off due dates require a timezone")
@dataclass(frozen=True, slots=True)
class CampaignWorkHandoffRef:
"""Stable, revision-bearing reference returned to the Workflow instance."""
tenant_id: str
campaign_id: str
campaign_version_id: str
campaign_revision: int
assignment_id: str
assignment_revision: int
status: CampaignWorkHandoffStatus
action_url: str
campaign_ref: str
assignment_ref: str
event_type: str = "campaign.work.changed"
replayed: bool = False
optional_capabilities: Mapping[str, bool] = field(default_factory=dict)
provenance: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class CampaignWorkHandoffInspection:
"""Current authorization and revision check before Workflow continuation."""
allowed: bool
status: CampaignWorkHandoffStatus | None = None
assignment_revision: int | None = None
action_url: str | None = None
assignment_ref: str | None = None
reason: str | None = None
provenance: Mapping[str, object] = field(default_factory=dict)
@runtime_checkable
class CampaignMailPolicyContextProvider(Protocol):
def get_campaign_mail_policy_context(
@@ -105,6 +202,21 @@ class CampaignDeliveryTaskProvider(Protocol):
...
@runtime_checkable
class CampaignScheduleProvider(Protocol):
"""Durable boundary for due manual drafts and governed autonomous occurrences."""
def dispatch_due(
self,
session: object,
*,
tenant_id: str | None = None,
now: datetime | None = None,
limit: int = 50,
) -> Mapping[str, object]:
...
@runtime_checkable
class CampaignRetentionProvider(Protocol):
def apply_retention(
@@ -116,3 +228,45 @@ class CampaignRetentionProvider(Protocol):
policy_for_campaign_id: Callable[[str | None], object],
) -> Mapping[str, Mapping[str, int]]:
...
@runtime_checkable
class CampaignWorkOrchestrationProvider(Protocol):
"""Optional Campaign boundary for durable Workflow-owned hand-offs."""
def prepare_handoff(
self,
session: object,
principal: object,
*,
request: CampaignWorkHandoffRequest,
) -> CampaignWorkHandoffRef:
...
def inspect_handoff(
self,
session: object,
principal: object,
*,
tenant_id: str,
assignment_id: str,
expected_revision: int | None = None,
) -> CampaignWorkHandoffInspection:
...
def campaign_work_orchestration_provider(
registry: object | None,
) -> CampaignWorkOrchestrationProvider | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_CAMPAIGNS_WORK_ORCHESTRATION)
):
return None
capability = registry.capability(CAPABILITY_CAMPAIGNS_WORK_ORCHESTRATION)
return (
capability
if isinstance(capability, CampaignWorkOrchestrationProvider)
else None
)
+27 -3
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import copy
import hashlib
import heapq
import json
from dataclasses import dataclass, field
from typing import Any, Callable, Iterable, Mapping, Sequence
@@ -436,10 +437,33 @@ def _merge_list(
result_by_id[identity] = merged.value
order_source = local_order if local_reordered and not current_reordered else current_order
merged_order = [identity for identity in order_source if identity in result_by_id]
for identity in identities:
if identity in result_by_id and identity not in merged_order:
secondary_order = current_order if order_source is local_order else local_order
# Keep the chosen side's order and both sides' insertion anchors. Appending
# missing IDs would silently relocate an insertion during a disjoint edit.
# An incompatible reorder/insertion cycle is an explicit conflict.
edges: dict[str, set[str]] = {identity: set() for identity in result_by_id}
incoming = dict.fromkeys(result_by_id, 0)
for order, insertions_only in ((order_source, False), (secondary_order, True)):
selected = [identity for identity in order if identity in result_by_id]
for left, right in zip(selected, selected[1:]):
if insertions_only and left in base_by_id and right in base_by_id:
continue
if right not in edges[left]:
edges[left].add(right)
incoming[right] += 1
priority = {identity: index for index, identity in enumerate(identities)}
ready = [(priority[identity], identity) for identity, count in incoming.items() if count == 0]
heapq.heapify(ready)
merged_order: list[str] = []
while ready:
_, identity = heapq.heappop(ready)
merged_order.append(identity)
for following in edges[identity]:
incoming[following] -= 1
if incoming[following] == 0:
heapq.heappush(ready, (priority[following], following))
if len(merged_order) != len(result_by_id):
return _conflict(path, "collection_reorder", base, local, current)
return ThreeWayMergeResult(
value=[result_by_id[identity] for identity in merged_order],
conflicts=conflicts,
@@ -371,6 +371,24 @@ def _approval_count(request: dict[str, Any]) -> int:
def _sanitize_value(key: str, value: object) -> object:
if key == "campaign_archive_encryption_policy":
if not isinstance(value, dict):
return "<redacted>"
# These are format/channel names, never passwords. Preserve only the
# exact public enum lists so rollback history remains useful without
# exempting arbitrary password-named fields from secret redaction.
allowed_values = {
"allowed_password_encryption_methods": frozenset({"aes", "zip_standard"}),
"allowed_password_delivery_channels": frozenset({"separate_mail", "sms", "letter", "phone", "in_person"}),
}
return {
name: list(items)
if name in allowed_values
and isinstance(items, list)
and all(isinstance(item, str) and item in allowed_values[name] for item in items)
else "<redacted>"
for name, items in value.items()
}
field = classify_configuration_field(key)
if field is not None and field.secret_handling in {"reference_only", "env_only"}:
return _redact_secrets(value)
@@ -3,6 +3,8 @@ from __future__ import annotations
import base64
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from datetime import UTC, datetime
from importlib.metadata import PackageNotFoundError, version as package_version
from pathlib import Path
import json
import os
@@ -28,6 +30,9 @@ from govoplan_core.core.external_references import (
SourceAuthorityMode,
integration_maturity_rank,
)
from govoplan_core.core.infrastructure_capabilities import (
InfrastructureCapabilityReceipt,
)
from govoplan_core.security.http_fetch import fetch_http_text
@@ -35,6 +40,12 @@ CONFIGURATION_PROVIDER_CAPABILITY = "configuration.provider"
DiagnosticSeverity = Literal["blocker", "warning", "info"]
PlanAction = Literal["create", "update", "bind", "skip", "blocked", "noop"]
ConfigurationRollbackStatus = Literal[
"blocked_before_apply",
"not_required",
"database_restore_required",
"partial_apply_requires_recovery",
]
ConfigurationPackageClass = Literal[
"reference",
"product",
@@ -441,6 +452,9 @@ class ConfigurationPreflightContext:
default_factory=dict
)
dry_run: bool = True
operator_scopes: frozenset[str] = frozenset()
infrastructure_receipt: InfrastructureCapabilityReceipt | None = None
infrastructure_receipt_error: str | None = None
@dataclass(frozen=True, slots=True)
@@ -455,6 +469,21 @@ class ConfigurationApplyResult:
diagnostics: tuple[ConfigurationDiagnostic, ...] = ()
created_refs: Mapping[str, str] = field(default_factory=dict)
updated_refs: Mapping[str, str] = field(default_factory=dict)
rollback: "ConfigurationRollbackState | None" = None
@dataclass(frozen=True, slots=True)
class ConfigurationRollbackState:
status: ConfigurationRollbackStatus
summary: str
recovery_action: str | None = None
def to_dict(self) -> dict[str, object]:
return {
"status": self.status,
"summary": self.summary,
"recovery_action": self.recovery_action,
}
@dataclass(frozen=True, slots=True)
@@ -465,11 +494,40 @@ class ConfigurationExportSelection:
object_refs: tuple[str, ...] = ()
@dataclass(frozen=True, slots=True)
class ConfigurationExportProvenance:
exported_at: str
source_core_version: str
module_versions: Mapping[str, str]
tenant_id: str | None
exporter_id: str | None
scopes: tuple[str, ...] = ()
module_ids: tuple[str, ...] = ()
object_refs: tuple[str, ...] = ()
redacted_secret_keys: tuple[str, ...] = ()
def to_dict(self) -> dict[str, object]:
return {
"exported_at": self.exported_at,
"source_core_version": self.source_core_version,
"module_versions": dict(self.module_versions),
"tenant_id": self.tenant_id,
"exporter_id": self.exporter_id,
"selection": {
"scopes": list(self.scopes),
"module_ids": list(self.module_ids),
"object_refs": list(self.object_refs),
},
"redacted_secret_keys": list(self.redacted_secret_keys),
}
@dataclass(frozen=True, slots=True)
class ConfigurationExportResult:
fragments: tuple[ConfigurationPackageFragment, ...] = ()
data_requirements: tuple[ConfigurationRequiredData, ...] = ()
diagnostics: tuple[ConfigurationDiagnostic, ...] = ()
provenance: ConfigurationExportProvenance | None = None
@runtime_checkable
@@ -502,6 +560,7 @@ def dry_run_configuration_package(
diagnostics: list[ConfigurationDiagnostic] = []
required_data: list[ConfigurationRequiredData] = []
plan: list[ConfigurationPlanItem] = []
declared_data: dict[str, ConfigurationRequiredData] = {}
diagnostics.extend(_module_requirement_diagnostics(manifest, context))
diagnostics.extend(_capability_requirement_diagnostics(manifest, context))
@@ -509,6 +568,7 @@ def dry_run_configuration_package(
for item in manifest.data_requirements:
requirement = ConfigurationRequiredData.from_mapping(item)
required_data.append(requirement)
declared_data[requirement.key] = requirement
if requirement.required and requirement.key not in context.supplied_data:
diagnostics.append(ConfigurationDiagnostic(
severity="blocker",
@@ -519,6 +579,25 @@ def dry_run_configuration_package(
))
for fragment in manifest.fragments:
data_ref_diagnostics = _fragment_data_reference_diagnostics(
fragment,
declared_data=declared_data,
supplied_data=context.supplied_data,
)
if data_ref_diagnostics:
diagnostics.extend(data_ref_diagnostics)
plan.append(ConfigurationPlanItem(
action="blocked",
module_id=fragment.module_id,
fragment_type=fragment.fragment_type,
fragment_id=fragment.fragment_id,
summary="Fragment needs declared deployment data before provider preflight.",
))
continue
resolved_fragment = _resolve_fragment_data_references(
fragment,
context.supplied_data,
)
provider = provider_map.get(fragment.module_id)
if provider is None:
diagnostics.append(ConfigurationDiagnostic(
@@ -544,7 +623,7 @@ def dry_run_configuration_package(
plan.append(ConfigurationPlanItem(action="blocked", module_id=fragment.module_id, fragment_type=fragment.fragment_type, fragment_id=fragment.fragment_id, summary="Fragment type is unsupported."))
continue
try:
result = provider.preflight(fragment, context)
result = provider.preflight(resolved_fragment, context)
except Exception as exc:
diagnostics.append(ConfigurationDiagnostic(
severity="blocker",
@@ -586,29 +665,54 @@ def apply_configuration_package(
apply_context = ConfigurationPreflightContext(
tenant_id=context.tenant_id,
operator_user_id=context.operator_user_id,
operator_scopes=context.operator_scopes,
supplied_data=supplied_data if supplied_data is not None else context.supplied_data,
installed_modules=context.installed_modules,
capabilities=context.capabilities,
external_provider_declarations=context.external_provider_declarations,
external_provider_states=context.external_provider_states,
infrastructure_receipt=context.infrastructure_receipt,
infrastructure_receipt_error=context.infrastructure_receipt_error,
dry_run=False,
)
preflight = dry_run_configuration_package(manifest, providers, apply_context)
blockers = [item for item in preflight.diagnostics if item.severity == "blocker"]
if blockers:
return ConfigurationApplyResult(diagnostics=tuple(blockers))
return ConfigurationApplyResult(
diagnostics=tuple(blockers),
rollback=ConfigurationRollbackState(
status="blocked_before_apply",
summary="No provider changes were attempted because package preflight is blocked.",
),
)
provider_map = _configuration_provider_map(providers)
diagnostics: list[ConfigurationDiagnostic] = list(preflight.diagnostics)
created_refs: dict[str, str] = {}
updated_refs: dict[str, str] = {}
stopped_after_blocker = False
for fragment in manifest.fragments:
provider = provider_map[fragment.module_id]
resolved_fragment = _resolve_fragment_data_references(
fragment,
apply_context.supplied_data,
)
try:
result = provider.apply(fragment, apply_context.supplied_data, apply_context)
result = provider.apply(
resolved_fragment,
apply_context.supplied_data,
apply_context,
)
diagnostics.extend(result.diagnostics)
created_refs.update(result.created_refs)
updated_refs.update(result.updated_refs)
diagnostics.extend(provider.health(result, apply_context))
health_diagnostics = provider.health(result, apply_context)
diagnostics.extend(health_diagnostics)
if any(
item.severity == "blocker"
for item in (*result.diagnostics, *health_diagnostics)
):
stopped_after_blocker = True
break
except Exception as exc:
diagnostics.append(ConfigurationDiagnostic(
severity="blocker",
@@ -618,10 +722,36 @@ def apply_configuration_package(
object_ref=fragment.fragment_id or fragment.fragment_type,
resolution="Stop the import, keep previous configuration, and inspect provider logs.",
))
stopped_after_blocker = True
break
changed = bool(created_refs or updated_refs)
if stopped_after_blocker and changed:
rollback = ConfigurationRollbackState(
status="partial_apply_requires_recovery",
summary="At least one provider committed changes before a later provider blocked the package.",
recovery_action="Restore the reviewed pre-apply database snapshot or use module-owned compensation where explicitly supported.",
)
elif stopped_after_blocker:
rollback = ConfigurationRollbackState(
status="blocked_before_apply",
summary="The first provider blocked before any configuration reference was created or updated.",
)
elif changed:
rollback = ConfigurationRollbackState(
status="database_restore_required",
summary="The package changed provider-owned configuration; generic cross-module compensation is not available.",
recovery_action="Retain the pre-apply database snapshot until verification is complete; restore it if the package must be rolled back.",
)
else:
rollback = ConfigurationRollbackState(
status="not_required",
summary="All package fragments were no-ops, so no rollback action is required.",
)
return ConfigurationApplyResult(
diagnostics=tuple(_dedupe_diagnostics(diagnostics)),
created_refs=created_refs,
updated_refs=updated_refs,
rollback=rollback,
)
@@ -660,10 +790,29 @@ def export_configuration_package(
fragments.extend(result.fragments)
data_requirements.extend(result.data_requirements)
diagnostics.extend(result.diagnostics)
deduped_required_data = tuple(_dedupe_required_data(data_requirements))
provenance = ConfigurationExportProvenance(
exported_at=datetime.now(UTC).isoformat(),
source_core_version=_installed_core_version(),
module_versions={
module_id: context.installed_modules[module_id]
for module_id in sorted(set(module_ids))
if module_id in context.installed_modules
},
tenant_id=selection.tenant_id,
exporter_id=context.operator_user_id,
scopes=selection.scopes,
module_ids=tuple(module_ids),
object_refs=selection.object_refs,
redacted_secret_keys=tuple(
sorted(item.key for item in deduped_required_data if item.secret)
),
)
return ConfigurationExportResult(
fragments=tuple(fragments),
data_requirements=tuple(_dedupe_required_data(data_requirements)),
data_requirements=deduped_required_data,
diagnostics=tuple(_dedupe_diagnostics(diagnostics)),
provenance=provenance,
)
@@ -1274,6 +1423,103 @@ def _dedupe_required_data(items: Sequence[ConfigurationRequiredData]) -> list[Co
return result
def _fragment_data_reference_diagnostics(
fragment: ConfigurationPackageFragment,
*,
declared_data: Mapping[str, ConfigurationRequiredData],
supplied_data: Mapping[str, Any],
) -> list[ConfigurationDiagnostic]:
references: set[str] = set()
invalid = _collect_fragment_data_references(fragment.payload, references)
diagnostics: list[ConfigurationDiagnostic] = []
object_ref = fragment.fragment_id or fragment.fragment_type
if invalid:
diagnostics.append(ConfigurationDiagnostic(
severity="blocker",
code="fragment_data_reference_invalid",
message="Configuration fragment data references must be objects containing only a non-empty $data key.",
module_id=fragment.module_id,
object_ref=object_ref,
resolution="Replace malformed references with {\"$data\": \"declared_requirement_key\"}.",
))
for key in sorted(references - set(declared_data)):
diagnostics.append(ConfigurationDiagnostic(
severity="blocker",
code="fragment_data_reference_undeclared",
message=f"Configuration fragment references undeclared operator data {key!r}.",
module_id=fragment.module_id,
object_ref=key,
resolution="Declare the key in package data_requirements before using it in a fragment.",
))
for key in sorted(references & set(declared_data)):
if key in supplied_data:
continue
diagnostics.append(ConfigurationDiagnostic(
severity="blocker",
code="fragment_data_reference_missing",
message=f"Configuration fragment needs operator data {declared_data[key].label!r} before provider preflight.",
module_id=fragment.module_id,
object_ref=key,
resolution="Provide the value in the generated configuration package form.",
))
return diagnostics
def _collect_fragment_data_references(value: object, references: set[str]) -> bool:
invalid = False
if isinstance(value, Mapping):
if "$data" in value:
key = value.get("$data")
if len(value) != 1 or not isinstance(key, str) or not key.strip():
return True
references.add(key.strip())
return False
for item in value.values():
invalid = _collect_fragment_data_references(item, references) or invalid
elif isinstance(value, Sequence) and not isinstance(value, (str, bytes)):
for item in value:
invalid = _collect_fragment_data_references(item, references) or invalid
return invalid
def _resolve_fragment_data_references(
fragment: ConfigurationPackageFragment,
supplied_data: Mapping[str, Any],
) -> ConfigurationPackageFragment:
payload = _resolve_data_reference_value(fragment.payload, supplied_data)
if not isinstance(payload, Mapping):
raise ValueError("Resolved configuration fragment payload must remain an object.")
return ConfigurationPackageFragment(
module_id=fragment.module_id,
fragment_type=fragment.fragment_type,
fragment_id=fragment.fragment_id,
payload=payload,
)
def _resolve_data_reference_value(value: object, supplied_data: Mapping[str, Any]) -> object:
if isinstance(value, Mapping):
if set(value) == {"$data"}:
key = value.get("$data")
if not isinstance(key, str) or key not in supplied_data:
raise ValueError("Configuration fragment contains an unresolved $data reference.")
return supplied_data[key]
return {
str(key): _resolve_data_reference_value(item, supplied_data)
for key, item in value.items()
}
if isinstance(value, Sequence) and not isinstance(value, (str, bytes)):
return [_resolve_data_reference_value(item, supplied_data) for item in value]
return value
def _installed_core_version() -> str:
try:
return package_version("govoplan-core")
except PackageNotFoundError:
return "workspace"
def _catalog_source(path: Path | str | None) -> Path | str | None:
if path is not None:
return path if isinstance(path, str) and _is_http_url(path) else Path(path).expanduser()
@@ -107,6 +107,20 @@ class _ConfigurationChangeSafetyState:
_CONFIGURATION_FIELD_SAFETY: tuple[ConfigurationFieldSafety, ...] = (
ConfigurationFieldSafety(
key="campaign_delivery_policy.system", label="System Campaign synchronous delivery limit",
owner_module="campaigns", scope="system", storage="system_settings", ui_managed=True,
risk="medium", required_scopes=("system:settings:write",),
audit_event="campaign.delivery_policy_updated", rollback_history_required=True,
notes="A bounded 0500 recipient-job maximum for one interactive Send now request. Explicit deployment ceilings remain authoritative; saving never delivers mail or changes review evidence.",
),
ConfigurationFieldSafety(
key="campaign_delivery_policy.tenant", label="Tenant Campaign synchronous delivery limit",
owner_module="campaigns", scope="tenant", storage="tenant_settings", ui_managed=True,
risk="medium", required_scopes=("admin:policies:write",),
audit_event="campaign.delivery_policy_updated", rollback_history_required=True,
notes="Tenant policy may only narrow the inherited system/deployment recipient-job maximum; clearing an override restores inheritance. Changes retain before/after history.",
),
ConfigurationFieldSafety(
key="module_management.desired_enabled",
label="Enabled modules",
@@ -171,6 +185,21 @@ _CONFIGURATION_FIELD_SAFETY: tuple[ConfigurationFieldSafety, ...] = (
rollback_history_required=True,
notes="Maintenance mode controls platform availability and gates dangerous operations.",
),
ConfigurationFieldSafety(
key="campaign_archive_encryption_policy",
label="Campaign archive encryption policy",
owner_module="policy",
scope="system",
storage="policy_overrides",
ui_managed=True,
risk="high",
required_scopes=("system:settings:write", "admin:policies:write"),
validation_required=True,
policy_explanation_required=True,
audit_event="campaign_archive_encryption_policy.updated",
rollback_history_required=True,
notes="Explicit system ceiling for Campaign archive methods and separate password-delivery channels. Policy validates allowed values and retains before/after history; lower scopes may only narrow. Legacy use additionally requires the dedicated Campaign permission and reasoned weak-encryption acknowledgement, so saving policy alone never enables or sends an archive.",
),
ConfigurationFieldSafety(
key="privacy_retention_policy",
label="Privacy retention policy",
+162 -2
View File
@@ -5,6 +5,7 @@ from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal, Protocol, runtime_checkable
from govoplan_core.core.access import PrincipalRef
from govoplan_core.core.external_references import (
SOURCE_AUTHORITY_MODES,
SourceAuthorityMode,
@@ -13,6 +14,7 @@ from govoplan_core.core.tabular_sources import (
DEFAULT_PREVIEW_BYTES,
DEFAULT_PREVIEW_TIMEOUT_MS,
TabularPreviewDiagnostic,
TabularCsvSource,
TabularPushdown,
TabularSourceHealth,
TabularSourceMode,
@@ -23,6 +25,8 @@ CAPABILITY_DATASOURCE_CATALOGUE = "datasources.catalogue"
CAPABILITY_DATASOURCE_LIFECYCLE = "datasources.lifecycle"
CAPABILITY_DATASOURCE_PUBLICATION = "datasources.publication"
CAPABILITY_DATASOURCE_ORIGINS = "connectors.datasourceOrigins"
CAPABILITY_DATASOURCE_ARTIFACT_BACKENDS = "datasources.artifactBackends"
CAPABILITY_POLICY_DATASOURCE_VISIBILITY = "policy.datasourceVisibility"
DatasourceMode = Literal["live", "cached", "static"]
DatasourceKind = Literal[
@@ -37,6 +41,12 @@ DatasourceKind = Literal[
]
DatasourceShape = Literal["tabular", "document", "binary", "directory", "stream"]
DatasourceConsistency = Literal["current", "live", "frozen"]
DatasourceVisibilityAction = Literal["discover", "read"]
DatasourcePublicationStatus = Literal[
"published",
"published_with_warnings",
"review_required",
]
class DatasourceError(ValueError):
@@ -64,6 +74,7 @@ class DatasourceField:
name: str
data_type: str
nullable: bool = True
classification: str = "internal"
@dataclass(frozen=True, slots=True)
@@ -84,11 +95,15 @@ class DatasourceGovernance:
classification: str = "internal"
privacy_profile_ref: str | None = None
retention_policy_ref: str | None = None
access_policy_ref: str | None = None
visibility_policy: Mapping[str, object] = field(default_factory=dict)
hold_refs: tuple[str, ...] = ()
publication_state: str = "draft"
transfer_agreement_ref: str | None = None
freshness_policy: Mapping[str, object] = field(default_factory=dict)
quality_policy: Mapping[str, object] = field(default_factory=dict)
approval_policy: Mapping[str, object] = field(default_factory=dict)
retention_policy: Mapping[str, object] = field(default_factory=dict)
known_limits: tuple[str, ...] = ()
correction_procedure_ref: str | None = None
affected_refs: tuple[str, ...] = ()
@@ -154,6 +169,12 @@ class DatasourceGovernance:
retention_policy_ref=_optional_governance_text(
source.get("retention_policy_ref")
),
access_policy_ref=_optional_governance_text(
source.get("access_policy_ref")
),
visibility_policy=_governance_mapping(
source.get("visibility_policy")
),
hold_refs=_governance_texts(source.get("hold_refs")),
publication_state=str(source.get("publication_state") or "draft"),
transfer_agreement_ref=_optional_governance_text(
@@ -161,6 +182,8 @@ class DatasourceGovernance:
),
freshness_policy=_governance_mapping(source.get("freshness_policy")),
quality_policy=_governance_mapping(source.get("quality_policy")),
approval_policy=_governance_mapping(source.get("approval_policy")),
retention_policy=_governance_mapping(source.get("retention_policy")),
known_limits=_governance_texts(source.get("known_limits")),
correction_procedure_ref=_optional_governance_text(
source.get("correction_procedure_ref")
@@ -185,11 +208,15 @@ class DatasourceGovernance:
"classification": self.classification,
"privacy_profile_ref": self.privacy_profile_ref,
"retention_policy_ref": self.retention_policy_ref,
"access_policy_ref": self.access_policy_ref,
"visibility_policy": dict(self.visibility_policy),
"hold_refs": list(self.hold_refs),
"publication_state": self.publication_state,
"transfer_agreement_ref": self.transfer_agreement_ref,
"freshness_policy": dict(self.freshness_policy),
"quality_policy": dict(self.quality_policy),
"approval_policy": dict(self.approval_policy),
"retention_policy": dict(self.retention_policy),
"known_limits": list(self.known_limits),
"correction_procedure_ref": self.correction_procedure_ref,
"affected_refs": list(self.affected_refs),
@@ -197,6 +224,39 @@ class DatasourceGovernance:
}
@dataclass(frozen=True, slots=True)
class DatasourceVisibilityPolicyRequest:
tenant_id: str
datasource_ref: str
principal: PrincipalRef
action: DatasourceVisibilityAction
classification: str = "internal"
policy_ref: str | None = None
consistency: DatasourceConsistency = "current"
materialization_ref: str | None = None
@dataclass(frozen=True, slots=True)
class DatasourceVisibilityPolicyDecision:
allowed: bool
reason: str | None = None
policies: tuple[Mapping[str, object], ...] = ()
decision_ref: str | None = None
provenance: Mapping[str, object] = field(default_factory=dict)
@runtime_checkable
class DatasourceVisibilityPolicyProvider(Protocol):
"""Optionally tighten Datasources-owned local visibility policy."""
def decide_datasource_visibility(
self,
session: object,
*,
request: DatasourceVisibilityPolicyRequest,
) -> DatasourceVisibilityPolicyDecision: ...
@dataclass(frozen=True, slots=True)
class DatasourceDescriptor:
ref: str
@@ -236,6 +296,8 @@ class DatasourceMaterialization:
frozen_label: str | None = None
source_timestamp: datetime | None = None
created_at: datetime | None = None
disposed_at: datetime | None = None
disposition: Mapping[str, object] = field(default_factory=dict)
provenance: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
governance: DatasourceGovernance = field(default_factory=DatasourceGovernance)
@@ -256,6 +318,7 @@ class DatasourceStage:
row_count: int | None = None
byte_count: int | None = None
validation: Mapping[str, object] = field(default_factory=dict)
approval: Mapping[str, object] = field(default_factory=dict)
created_at: datetime | None = None
promoted_at: datetime | None = None
promoted_materialization_ref: str | None = None
@@ -307,6 +370,67 @@ class DatasourceStageInput:
provenance: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
governance: DatasourceGovernance | None = None
csv_source: TabularCsvSource | None = None
@dataclass(frozen=True, slots=True)
class DatasourceArtifactReference:
"""Immutable provider-neutral reference to a durable tabular payload.
The producer owns creation of the payload. Datasources pins its locator,
checksum and declared shape without importing the artifact-owning module;
a configured payload backend verifies integrity and provides bounded reads.
"""
backend: str
locator: str
checksum: str
row_count: int
byte_count: int
schema: tuple[DatasourceField, ...]
fingerprint: str
media_type: str = "application/x-ndjson"
checkpoint: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
validation: Mapping[str, object] = field(default_factory=dict)
@runtime_checkable
class DatasourceArtifactBackend(Protocol):
"""Storage-module boundary for immutable artifact-backed tabular data."""
backend: str
def verify(
self,
session: object,
*,
tenant_id: str,
artifact: DatasourceArtifactReference,
) -> None: ...
def read_rows(
self,
session: object,
*,
tenant_id: str,
artifact: DatasourceArtifactReference,
offset: int,
limit: int,
) -> Sequence[Mapping[str, object]]: ...
def delete(
self,
session: object,
*,
tenant_id: str,
artifact: DatasourceArtifactReference,
) -> None: ...
@runtime_checkable
class DatasourceArtifactBackendProvider(Protocol):
def artifact_backends(self) -> Sequence[DatasourceArtifactBackend]: ...
@dataclass(frozen=True, slots=True)
@@ -314,7 +438,8 @@ class DatasourcePublicationRequest:
producer_module: str
producer_run_ref: str
idempotency_key: str
rows: tuple[Mapping[str, object], ...]
rows: tuple[Mapping[str, object], ...] | None = None
artifact: DatasourceArtifactReference | None = None
target_datasource_ref: str | None = None
name: str | None = None
source_name: str | None = None
@@ -331,7 +456,7 @@ class DatasourcePublicationRequest:
@dataclass(frozen=True, slots=True)
class DatasourcePublicationResult:
ref: str
status: str
status: DatasourcePublicationStatus
datasource: DatasourceDescriptor
materialization: DatasourceMaterialization
replayed: bool = False
@@ -564,6 +689,17 @@ def datasource_catalogue(registry: object | None) -> DatasourceCatalogueProvider
return capability if isinstance(capability, DatasourceCatalogueProvider) else None
def datasource_artifact_backend_provider(
registry: object | None,
) -> DatasourceArtifactBackendProvider | None:
capability = _capability(registry, CAPABILITY_DATASOURCE_ARTIFACT_BACKENDS)
return (
capability
if isinstance(capability, DatasourceArtifactBackendProvider)
else None
)
def datasource_lifecycle(registry: object | None) -> DatasourceLifecycleProvider | None:
capability = _capability(registry, CAPABILITY_DATASOURCE_LIFECYCLE)
return capability if isinstance(capability, DatasourceLifecycleProvider) else None
@@ -581,6 +717,13 @@ def datasource_origins(registry: object | None) -> DatasourceOriginProvider | No
return capability if isinstance(capability, DatasourceOriginProvider) else None
def datasource_visibility_policy_provider(
registry: object | None,
) -> DatasourceVisibilityPolicyProvider | None:
capability = _capability(registry, CAPABILITY_POLICY_DATASOURCE_VISIBILITY)
return capability if isinstance(capability, DatasourceVisibilityPolicyProvider) else None
def _capability(registry: object | None, name: str) -> object | None:
if (
registry is None
@@ -613,9 +756,15 @@ def _governance_mapping(value: object) -> Mapping[str, object]:
__all__ = [
"CAPABILITY_DATASOURCE_CATALOGUE",
"CAPABILITY_DATASOURCE_ARTIFACT_BACKENDS",
"CAPABILITY_DATASOURCE_LIFECYCLE",
"CAPABILITY_DATASOURCE_ORIGINS",
"CAPABILITY_DATASOURCE_PUBLICATION",
"CAPABILITY_POLICY_DATASOURCE_VISIBILITY",
"DatasourceAccessError",
"DatasourceArtifactReference",
"DatasourceArtifactBackend",
"DatasourceArtifactBackendProvider",
"DatasourceCatalogueProvider",
"DatasourceConsistency",
"DatasourceDescriptor",
@@ -631,6 +780,10 @@ __all__ = [
"DatasourceOriginProvider",
"DatasourceOriginReadRequest",
"DatasourceOriginReadResult",
"DatasourcePublicationProvider",
"DatasourcePublicationRequest",
"DatasourcePublicationResult",
"DatasourcePublicationStatus",
"DatasourceReadRequest",
"DatasourceReadResult",
"DatasourceShape",
@@ -638,7 +791,14 @@ __all__ = [
"DatasourceStageInput",
"DatasourceUnavailableError",
"DatasourceValidationError",
"DatasourceVisibilityAction",
"DatasourceVisibilityPolicyDecision",
"DatasourceVisibilityPolicyProvider",
"DatasourceVisibilityPolicyRequest",
"datasource_catalogue",
"datasource_artifact_backend_provider",
"datasource_lifecycle",
"datasource_origins",
"datasource_publication",
"datasource_visibility_policy_provider",
]
+197
View File
@@ -0,0 +1,197 @@
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal, Protocol, runtime_checkable
DSAR_CAPABILITY_PREFIX = "privacy.dsar."
DsarRequestKind = Literal["access", "erasure", "access_and_erasure"]
DsarActionKind = Literal[
"delete",
"anonymize",
"revoke",
"detach",
"retain",
"manual_review",
]
@dataclass(frozen=True, slots=True)
class DsarSubjectRef:
account_id: str | None = None
identity_id: str | None = None
membership_id: str | None = None
email: str | None = None
external_references: Mapping[str, str] = field(default_factory=dict)
def has_selector(self) -> bool:
return bool(
self.account_id
or self.identity_id
or self.membership_id
or self.email
or self.external_references
)
def to_dict(self) -> dict[str, object]:
return {
"account_id": self.account_id,
"identity_id": self.identity_id,
"membership_id": self.membership_id,
"email": self.email,
"external_references": dict(self.external_references),
}
@dataclass(frozen=True, slots=True)
class DsarRecordRef:
provider_id: str
module_id: str
resource_type: str
resource_id: str
category: str
title: str
data: Mapping[str, object] = field(default_factory=dict)
observed_at: datetime | None = None
immutable_evidence: bool = False
retention_reason: str | None = None
source_path: str | None = None
def to_dict(self) -> dict[str, object]:
return {
"provider_id": self.provider_id,
"module_id": self.module_id,
"resource_type": self.resource_type,
"resource_id": self.resource_id,
"category": self.category,
"title": self.title,
"data": dict(self.data),
"observed_at": self.observed_at.isoformat() if self.observed_at else None,
"immutable_evidence": self.immutable_evidence,
"retention_reason": self.retention_reason,
"source_path": self.source_path,
}
@dataclass(frozen=True, slots=True)
class DsarErasureActionRef:
action_id: str
provider_id: str
module_id: str
kind: DsarActionKind
resource_type: str
resource_id: str
title: str
rationale: str
executable: bool
irreversible: bool = False
metadata: Mapping[str, object] = field(default_factory=dict)
def to_dict(self) -> dict[str, object]:
return {
"action_id": self.action_id,
"provider_id": self.provider_id,
"module_id": self.module_id,
"kind": self.kind,
"resource_type": self.resource_type,
"resource_id": self.resource_id,
"title": self.title,
"rationale": self.rationale,
"executable": self.executable,
"irreversible": self.irreversible,
"metadata": dict(self.metadata),
}
@dataclass(frozen=True, slots=True)
class DsarExecutionResultRef:
action_id: str
status: Literal["executed", "unchanged", "failed", "blocked"]
summary: str
evidence: Mapping[str, object] = field(default_factory=dict)
def to_dict(self) -> dict[str, object]:
return {
"action_id": self.action_id,
"status": self.status,
"summary": self.summary,
"evidence": dict(self.evidence),
}
@runtime_checkable
class DsarProvider(Protocol):
provider_id: str
module_id: str
def search_subject(
self,
session: object,
*,
tenant_id: str,
subject: DsarSubjectRef,
) -> Sequence[DsarRecordRef]: ...
def plan_erasure(
self,
session: object,
*,
tenant_id: str,
subject: DsarSubjectRef,
records: Sequence[DsarRecordRef],
) -> Sequence[DsarErasureActionRef]: ...
def execute_erasure(
self,
session: object,
*,
tenant_id: str,
subject: DsarSubjectRef,
actions: Sequence[DsarErasureActionRef],
request_id: str,
) -> Sequence[DsarExecutionResultRef]: ...
def dsar_capability_name(module_id: str) -> str:
normalized = module_id.strip().casefold()
if not normalized or not normalized.replace("_", "").isalnum():
raise ValueError("DSAR module id must be an identifier.")
return f"{DSAR_CAPABILITY_PREFIX}{normalized}"
def dsar_provider_names(registry: object | None) -> tuple[str, ...]:
if registry is None or not hasattr(registry, "capability_names"):
return ()
return tuple(
name
for name in registry.capability_names()
if name.startswith(DSAR_CAPABILITY_PREFIX)
)
def dsar_provider(
registry: object,
capability_name: str,
) -> DsarProvider:
provider = registry.require_capability(capability_name)
if not isinstance(provider, DsarProvider):
raise TypeError(f"{capability_name} does not implement DsarProvider")
return provider
__all__ = [
"DSAR_CAPABILITY_PREFIX",
"DsarActionKind",
"DsarErasureActionRef",
"DsarExecutionResultRef",
"DsarProvider",
"DsarRecordRef",
"DsarRequestKind",
"DsarSubjectRef",
"dsar_capability_name",
"dsar_provider",
"dsar_provider_names",
]
+176
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime
from typing import Protocol, runtime_checkable
from govoplan_core.core.access import ResourceAccessExplanationProvider
@@ -9,6 +10,49 @@ from govoplan_core.core.access import ResourceAccessExplanationProvider
CAPABILITY_FILES_ACCESS = "files.access"
CAPABILITY_FILES_ARTIFACT_STORE = "files.artifact_store"
CAPABILITY_FILES_POSTBOX_REFERENCES = "files.postbox_references"
CAPABILITY_FILES_TABULAR_CONTENT = "files.tabular_content"
class ManagedTabularFileError(ValueError):
"""Stable base error for exact-version managed tabular file access."""
class ManagedTabularFileNotFoundError(ManagedTabularFileError):
pass
class ManagedTabularFileAccessError(ManagedTabularFileError):
pass
class ManagedTabularFileUnavailableError(ManagedTabularFileError):
pass
class ManagedTabularFileValidationError(ManagedTabularFileError):
pass
@dataclass(frozen=True, slots=True)
class ManagedTabularFile:
"""Authorized metadata for one immutable managed file version."""
file_asset_id: str
file_version_id: str
filename: str
display_path: str
content_type: str | None
size_bytes: int
sha256: str
updated_at: datetime | None = None
current_version: bool = True
@dataclass(frozen=True, slots=True)
class ManagedTabularFileContent:
file: ManagedTabularFile
payload: bytes
@dataclass(frozen=True, slots=True)
@@ -34,6 +78,30 @@ class ManagedArtifactRef:
provenance: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class PostboxFileReferenceRequest:
reference_type: str
reference_id: str
postbox_id: str
message_id: str
@dataclass(frozen=True, slots=True)
class PostboxFileReferenceRef:
reference_type: str
reference_id: str
available: bool
reason_code: str
file_asset_id: str | None = None
file_version_id: str | None = None
filename: str | None = None
content_type: str | None = None
size_bytes: int | None = None
sha256: str | None = None
download_path: str | None = None
provenance: Mapping[str, object] = field(default_factory=dict)
@runtime_checkable
class FileAccessProvider(ResourceAccessExplanationProvider, Protocol):
"""Resource-level access explanation provider for Files-owned resources."""
@@ -50,3 +118,111 @@ class ManagedArtifactStore(Protocol):
*,
request: ManagedArtifactWriteRequest,
) -> ManagedArtifactRef: ...
@runtime_checkable
class PostboxFileReferenceProvider(Protocol):
"""Resolve Files-owned references after Postbox and Files authorization."""
def resolve_postbox_references(
self,
session: object,
principal: object,
*,
tenant_id: str,
requests: tuple[PostboxFileReferenceRequest, ...],
) -> tuple[PostboxFileReferenceRef, ...]: ...
@runtime_checkable
class ManagedTabularFileProvider(Protocol):
"""List and open authorized CSV/XLSX content without exposing Files internals."""
def list_tabular_files(
self,
session: object,
principal: object,
*,
query: str = "",
limit: int = 100,
) -> tuple[ManagedTabularFile, ...]: ...
def get_tabular_file(
self,
session: object,
principal: object,
*,
file_asset_id: str,
file_version_id: str | None = None,
) -> ManagedTabularFile | None: ...
def read_tabular_file(
self,
session: object,
principal: object,
*,
file_asset_id: str,
file_version_id: str,
max_bytes: int,
) -> ManagedTabularFileContent: ...
def postbox_file_reference_provider(
registry: object | None,
) -> PostboxFileReferenceProvider | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_FILES_POSTBOX_REFERENCES)
):
return None
provider = registry.require_capability(CAPABILITY_FILES_POSTBOX_REFERENCES)
if not isinstance(provider, PostboxFileReferenceProvider):
raise TypeError(
"files.postbox_references provider does not implement "
"PostboxFileReferenceProvider"
)
return provider
def managed_tabular_file_provider(
registry: object | None,
) -> ManagedTabularFileProvider | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_FILES_TABULAR_CONTENT)
):
return None
provider = registry.require_capability(CAPABILITY_FILES_TABULAR_CONTENT)
if not isinstance(provider, ManagedTabularFileProvider):
raise TypeError(
"files.tabular_content provider does not implement "
"ManagedTabularFileProvider"
)
return provider
__all__ = [
"CAPABILITY_FILES_ACCESS",
"CAPABILITY_FILES_ARTIFACT_STORE",
"CAPABILITY_FILES_POSTBOX_REFERENCES",
"CAPABILITY_FILES_TABULAR_CONTENT",
"FileAccessProvider",
"ManagedArtifactRef",
"ManagedArtifactStore",
"ManagedArtifactWriteRequest",
"ManagedTabularFile",
"ManagedTabularFileAccessError",
"ManagedTabularFileContent",
"ManagedTabularFileError",
"ManagedTabularFileNotFoundError",
"ManagedTabularFileProvider",
"ManagedTabularFileUnavailableError",
"ManagedTabularFileValidationError",
"PostboxFileReferenceProvider",
"PostboxFileReferenceRef",
"PostboxFileReferenceRequest",
"managed_tabular_file_provider",
"postbox_file_reference_provider",
]
+1 -1
View File
@@ -356,7 +356,7 @@ def consume_first_admin_credential(
tenant = Tenant(
slug=clean_tenant_slug,
name=clean_tenant_name,
default_locale="en",
default_locale="de",
settings={},
is_active=True,
)
+227
View File
@@ -0,0 +1,227 @@
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal, Protocol, runtime_checkable
from govoplan_core.core.institutional import EvidenceReference, InstitutionalReference
CAPABILITY_FORM_EVIDENCE_PREFIX = "forms_runtime.evidence."
FormEvidenceState = Literal[
"accepted",
"pending",
"rejected",
"expired",
"revoked",
"unavailable",
]
_FORM_EVIDENCE_STATES = {
"accepted",
"pending",
"rejected",
"expired",
"revoked",
"unavailable",
}
class FormEvidenceContractError(ValueError):
"""Stable error for provider-neutral Form evidence operations."""
@dataclass(frozen=True, slots=True)
class FormEvidenceGrantRequest:
"""Request a short-lived, purpose-bound grant from an evidence owner."""
tenant_id: str
instance_id: str
definition_ref: InstitutionalReference
evidence_kind: str
purpose: str
idempotency_key: str
expires_at: datetime
custodian_ref: str | None = None
max_size_bytes: int | None = None
allowed_content_types: tuple[str, ...] = ()
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_require_text(self.tenant_id, "Form evidence tenant")
_require_text(self.instance_id, "Form evidence instance")
_require_text(self.evidence_kind, "Form evidence kind")
_require_text(self.purpose, "Form evidence purpose")
_require_text(self.idempotency_key, "Form evidence idempotency key")
if (
self.definition_ref.kind != "form"
or self.definition_ref.tenant_id != self.tenant_id
or not self.definition_ref.version
):
raise FormEvidenceContractError(
"Form evidence grants require an exact same-tenant Form definition."
)
if self.expires_at.tzinfo is None or self.expires_at.utcoffset() is None:
raise FormEvidenceContractError(
"Form evidence grant expiry must include a timezone."
)
if self.max_size_bytes is not None and self.max_size_bytes <= 0:
raise FormEvidenceContractError(
"Form evidence grant size limits must be positive."
)
if any(not item.strip() for item in self.allowed_content_types):
raise FormEvidenceContractError(
"Form evidence content types cannot be empty."
)
@dataclass(frozen=True, slots=True)
class FormEvidenceGrant:
provider_id: str
grant_id: str
upload_token: str | None
upload_url: str
expires_at: datetime
max_size_bytes: int
allowed_content_types: tuple[str, ...] = ()
replayed: bool = False
def __post_init__(self) -> None:
for value, label in (
(self.provider_id, "Form evidence provider"),
(self.grant_id, "Form evidence grant"),
(self.upload_url, "Form evidence upload URL"),
):
_require_text(value, label)
if self.upload_token is not None:
_require_text(self.upload_token, "Form evidence upload token")
if self.max_size_bytes <= 0:
raise FormEvidenceContractError(
"Form evidence grant size limits must be positive."
)
if self.expires_at.tzinfo is None or self.expires_at.utcoffset() is None:
raise FormEvidenceContractError(
"Form evidence grant expiry must include a timezone."
)
@dataclass(frozen=True, slots=True)
class FormEvidenceInspectionRequest:
tenant_id: str
instance_id: str
definition_ref: InstitutionalReference
evidence: EvidenceReference
purpose: str
final: bool
def __post_init__(self) -> None:
_require_text(self.tenant_id, "Form evidence tenant")
_require_text(self.instance_id, "Form evidence instance")
_require_text(self.purpose, "Form evidence purpose")
if self.definition_ref.tenant_id != self.tenant_id:
raise FormEvidenceContractError(
"Form evidence inspection cannot cross tenants."
)
if self.evidence.tenant_id != self.tenant_id:
raise FormEvidenceContractError(
"Form evidence inspection cannot cross tenants."
)
@dataclass(frozen=True, slots=True)
class FormEvidenceInspection:
provider_id: str
reference: EvidenceReference
state: FormEvidenceState
observed_at: datetime
retryable: bool = False
reason: str | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_require_text(self.provider_id, "Form evidence provider")
if self.state not in _FORM_EVIDENCE_STATES:
raise FormEvidenceContractError(
f"Unsupported Form evidence state: {self.state!r}."
)
if self.observed_at.tzinfo is None or self.observed_at.utcoffset() is None:
raise FormEvidenceContractError(
"Form evidence inspection time must include a timezone."
)
if self.state == "accepted" and self.retryable:
raise FormEvidenceContractError(
"Accepted Form evidence cannot require a retry."
)
@property
def accepted(self) -> bool:
return self.state == "accepted"
@runtime_checkable
class FormEvidenceProvider(Protocol):
provider_id: str
def supported_kinds(self) -> Sequence[str]: ...
def create_upload_grant(
self,
session: object,
principal: object,
*,
request: FormEvidenceGrantRequest,
) -> FormEvidenceGrant: ...
def inspect_evidence(
self,
session: object,
principal: object,
*,
request: FormEvidenceInspectionRequest,
) -> FormEvidenceInspection: ...
def form_evidence_capability(provider_id: str) -> str:
normalized = str(provider_id or "").strip().lower().replace("-", "_")
if not normalized or not normalized.replace("_", "").isalnum():
raise FormEvidenceContractError("Invalid Form evidence provider id.")
return f"{CAPABILITY_FORM_EVIDENCE_PREFIX}{normalized}"
def form_evidence_provider(
registry: object | None,
provider_id: str,
) -> FormEvidenceProvider | None:
capability_name = form_evidence_capability(provider_id)
if registry is None or not hasattr(registry, "has_capability"):
return None
if not registry.has_capability(capability_name):
return None
if hasattr(registry, "require_capability"):
provider = registry.require_capability(capability_name)
elif hasattr(registry, "capability"):
provider = registry.capability(capability_name)
else:
return None
return provider if isinstance(provider, FormEvidenceProvider) else None
def _require_text(value: str, label: str) -> None:
if not isinstance(value, str) or not value.strip():
raise FormEvidenceContractError(f"{label} is required.")
__all__ = [
"CAPABILITY_FORM_EVIDENCE_PREFIX",
"FormEvidenceContractError",
"FormEvidenceGrant",
"FormEvidenceGrantRequest",
"FormEvidenceInspection",
"FormEvidenceInspectionRequest",
"FormEvidenceProvider",
"FormEvidenceState",
"form_evidence_capability",
"form_evidence_provider",
]
@@ -0,0 +1,236 @@
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Literal, Mapping, cast
InformationGovernanceAdoption = Literal[
"not_applicable",
"contract_only",
"partial",
"enforced",
]
INFORMATION_GOVERNANCE_ADOPTION_ORDER: tuple[InformationGovernanceAdoption, ...] = (
"not_applicable",
"contract_only",
"partial",
"enforced",
)
class InformationGovernanceDeclarationError(ValueError):
pass
@dataclass(frozen=True, slots=True)
class InformationGovernanceDimension:
"""Truthful module-level adoption claim for one cross-cutting dimension."""
adoption: InformationGovernanceAdoption = "contract_only"
object_types: tuple[str, ...] = ()
evidence: tuple[str, ...] = ()
limitation: str | None = None
def __post_init__(self) -> None:
if self.adoption not in INFORMATION_GOVERNANCE_ADOPTION_ORDER:
raise InformationGovernanceDeclarationError(
f"Unsupported information-governance adoption: {self.adoption!r}."
)
for field_name in ("object_types", "evidence"):
values = getattr(self, field_name)
if len(values) != len(set(values)) or any(not item.strip() for item in values):
raise InformationGovernanceDeclarationError(
f"Information-governance {field_name.replace('_', ' ')} must "
"contain unique non-empty values."
)
if self.adoption == "enforced" and not self.evidence:
raise InformationGovernanceDeclarationError(
"An enforced information-governance dimension requires evidence."
)
if self.adoption in {"partial", "enforced"} and not self.object_types:
raise InformationGovernanceDeclarationError(
"Partial and enforced information-governance dimensions must "
"name their covered object types."
)
if self.adoption == "not_applicable" and self.object_types:
raise InformationGovernanceDeclarationError(
"A non-applicable information-governance dimension cannot declare object types."
)
if self.adoption in {"contract_only", "partial"} and not str(
self.limitation or ""
).strip():
raise InformationGovernanceDeclarationError(
"Contract-only and partial adoption must state the current limitation."
)
def to_dict(self) -> dict[str, object]:
return {
"adoption": self.adoption,
"object_types": list(self.object_types),
"evidence": list(self.evidence),
"limitation": self.limitation,
}
def _contract_only_dimension() -> InformationGovernanceDimension:
return InformationGovernanceDimension(
adoption="contract_only",
limitation=(
"The platform contract applies, but module-specific adoption evidence "
"has not been declared."
),
)
@dataclass(frozen=True, slots=True)
class ModuleInformationGovernance:
"""Cross-cutting data-use requirements and honest adoption evidence."""
temporal_browsing: InformationGovernanceDimension = field(
default_factory=_contract_only_dimension
)
purpose_aware_access: InformationGovernanceDimension = field(
default_factory=_contract_only_dimension
)
retention: InformationGovernanceDimension = field(
default_factory=_contract_only_dimension
)
institutional_context: InformationGovernanceDimension = field(
default_factory=_contract_only_dimension
)
current_authorization_for_historical_reads: bool = True
contract_version: str = "1"
def __post_init__(self) -> None:
if self.contract_version != "1":
raise InformationGovernanceDeclarationError(
"Unsupported module information-governance contract version."
)
if not self.current_authorization_for_historical_reads:
raise InformationGovernanceDeclarationError(
"Historical reads must always use current authorization."
)
for name, dimension in self.dimensions.items():
if not isinstance(dimension, InformationGovernanceDimension):
raise InformationGovernanceDeclarationError(
f"Information-governance dimension {name!r} has an invalid value."
)
@property
def dimensions(self) -> Mapping[str, InformationGovernanceDimension]:
return {
"temporal_browsing": self.temporal_browsing,
"purpose_aware_access": self.purpose_aware_access,
"retention": self.retention,
"institutional_context": self.institutional_context,
}
def to_dict(self) -> dict[str, object]:
return {
"contract_version": self.contract_version,
"current_authorization_for_historical_reads": (
self.current_authorization_for_historical_reads
),
"dimensions": {
name: dimension.to_dict()
for name, dimension in self.dimensions.items()
},
}
def information_governance_from_mapping(
value: Mapping[str, object],
) -> ModuleInformationGovernance:
raw_dimensions = value.get("dimensions")
if not isinstance(raw_dimensions, Mapping):
raise InformationGovernanceDeclarationError(
"Information-governance dimensions must be an object."
)
def dimension(name: str) -> InformationGovernanceDimension:
raw_dimension = raw_dimensions.get(name)
if not isinstance(raw_dimension, Mapping):
raise InformationGovernanceDeclarationError(
f"Information-governance dimension {name!r} must be an object."
)
def text_tuple(field_name: str) -> tuple[str, ...]:
raw_values = raw_dimension.get(field_name, ())
if not isinstance(raw_values, (list, tuple)):
raise InformationGovernanceDeclarationError(
f"Information-governance {name}.{field_name} must be a list."
)
if any(not isinstance(item, str) for item in raw_values):
raise InformationGovernanceDeclarationError(
f"Information-governance {name}.{field_name} must contain strings."
)
return tuple(raw_values)
raw_limitation = raw_dimension.get("limitation")
raw_adoption = raw_dimension.get("adoption") or "contract_only"
if not isinstance(raw_adoption, str):
raise InformationGovernanceDeclarationError(
f"Information-governance {name}.adoption must be a string."
)
if raw_limitation is not None and not isinstance(raw_limitation, str):
raise InformationGovernanceDeclarationError(
f"Information-governance {name}.limitation must be a string."
)
return InformationGovernanceDimension(
adoption=cast(
InformationGovernanceAdoption,
raw_adoption,
),
object_types=text_tuple("object_types"),
evidence=text_tuple("evidence"),
limitation=raw_limitation,
)
current_authorization = value.get(
"current_authorization_for_historical_reads",
True,
)
if not isinstance(current_authorization, bool):
raise InformationGovernanceDeclarationError(
"current_authorization_for_historical_reads must be boolean."
)
return ModuleInformationGovernance(
contract_version=str(value.get("contract_version") or "1"),
current_authorization_for_historical_reads=current_authorization,
temporal_browsing=dimension("temporal_browsing"),
purpose_aware_access=dimension("purpose_aware_access"),
retention=dimension("retention"),
institutional_context=dimension("institutional_context"),
)
def information_governance_maturity_issues(
declaration: ModuleInformationGovernance,
*,
maturity: str | None,
) -> tuple[str, ...]:
if maturity not in {"reference_ready", "supported", "lts"}:
return ()
incomplete = [
name
for name, dimension in declaration.dimensions.items()
if dimension.adoption not in {"not_applicable", "enforced"}
]
if not incomplete:
return ()
return (
f"Maturity {maturity!r} requires enforced or explicitly non-applicable "
"information governance for: " + ", ".join(incomplete),
)
__all__ = [
"INFORMATION_GOVERNANCE_ADOPTION_ORDER",
"InformationGovernanceAdoption",
"InformationGovernanceDeclarationError",
"InformationGovernanceDimension",
"ModuleInformationGovernance",
"information_governance_from_mapping",
"information_governance_maturity_issues",
]
@@ -0,0 +1,614 @@
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from datetime import UTC, datetime
import json
import os
from pathlib import Path
import re
from typing import Any, Protocol, runtime_checkable
DEPLOYMENT_CAPABILITIES_ENV = "GOVOPLAN_DEPLOYMENT_CAPABILITIES_PATH"
INFRASTRUCTURE_DEPENDENCY_PROVIDER_CAPABILITY_PREFIX = (
"infrastructure.dependency_inventory."
)
MAX_CAPABILITY_DOCUMENT_BYTES = 256 * 1024
CAPABILITY_STATES = frozenset(
{
"configured",
"available_unconfigured",
"externally_supplied",
"unavailable",
}
)
_ENV_REFERENCE_RE = re.compile(r"^env:[A-Za-z_][A-Za-z0-9_]*$")
_DEPENDENCY_STATES = frozenset(
{"active", "inactive", "data_present", "pending_work", "runtime_binding"}
)
class InfrastructureCapabilityReceiptError(ValueError):
pass
@dataclass(frozen=True, slots=True)
class InfrastructureDependency:
"""A non-secret module-owned dependency on deployment infrastructure."""
capability_id: str
module_id: str
dependency_type: str
dependency_ref: str
state: str
scope: str
summary: str
metrics: Mapping[str, int]
required_action: str
def __post_init__(self) -> None:
for field_name, value, maximum in (
("capability_id", self.capability_id, 120),
("module_id", self.module_id, 120),
("dependency_type", self.dependency_type, 120),
("dependency_ref", self.dependency_ref, 240),
("scope", self.scope, 120),
("summary", self.summary, 1000),
("required_action", self.required_action, 1000),
):
if (
not value.strip()
or len(value) > maximum
or any(ord(char) < 32 for char in value)
):
raise ValueError(
f"Infrastructure dependency {field_name} is invalid."
)
if self.state not in _DEPENDENCY_STATES:
raise ValueError(
f"Infrastructure dependency state is unsupported: {self.state!r}."
)
if len(self.metrics) > 20 or any(
not isinstance(key, str)
or not key.strip()
or len(key) > 80
or any(ord(char) < 32 for char in key)
or type(value) is not int
or value < 0
for key, value in self.metrics.items()
):
raise ValueError("Infrastructure dependency metrics are invalid.")
def to_dict(self) -> dict[str, object]:
return {
"capability_id": self.capability_id,
"module_id": self.module_id,
"dependency_type": self.dependency_type,
"dependency_ref": self.dependency_ref,
"state": self.state,
"scope": self.scope,
"summary": self.summary,
"metrics": dict(sorted(self.metrics.items())),
"required_action": self.required_action,
}
@runtime_checkable
class InfrastructureDependencyProvider(Protocol):
module_id: str
capability_ids: tuple[str, ...]
def infrastructure_dependencies(self) -> tuple[InfrastructureDependency, ...]:
...
@dataclass(frozen=True, slots=True)
class InfrastructureDependencyProviderReport:
module_id: str
capability_ids: tuple[str, ...]
state: str
dependency_count: int
error: str | None = None
def to_dict(self) -> dict[str, object]:
return {
"module_id": self.module_id,
"capability_ids": list(self.capability_ids),
"state": self.state,
"dependency_count": self.dependency_count,
"error": self.error,
}
@dataclass(frozen=True, slots=True)
class InfrastructureDependencyInventory:
installation_id: str
generated_at: str
complete: bool
inspected_capability_ids: tuple[str, ...]
providers: tuple[InfrastructureDependencyProviderReport, ...]
dependencies: tuple[InfrastructureDependency, ...]
schema_version: int = 1
def to_dict(self) -> dict[str, object]:
return {
"schema_version": self.schema_version,
"installation_id": self.installation_id,
"generated_at": self.generated_at,
"complete": self.complete,
"inspected_capability_ids": list(self.inspected_capability_ids),
"providers": [item.to_dict() for item in self.providers],
"dependencies": [item.to_dict() for item in self.dependencies],
}
@dataclass(frozen=True, slots=True)
class InfrastructureCapability:
id: str
label: str
state: str
source: str
detail: str
endpoint: Mapping[str, object]
secret_refs: tuple[str, ...]
dependent_modules: tuple[str, ...]
def to_dict(self) -> dict[str, object]:
return {
"id": self.id,
"label": self.label,
"state": self.state,
"source": self.source,
"detail": self.detail,
"endpoint": dict(self.endpoint),
"secret_refs": list(self.secret_refs),
"dependent_modules": list(self.dependent_modules),
}
@dataclass(frozen=True, slots=True)
class InfrastructurePostInstallTask:
id: str
resume_key: str
capability_id: str
state: str
owner_module: str
summary: str
required_inputs: tuple[str, ...]
secret_boundary: str
def to_dict(self) -> dict[str, object]:
return {
"id": self.id,
"resume_key": self.resume_key,
"capability_id": self.capability_id,
"state": self.state,
"owner_module": self.owner_module,
"summary": self.summary,
"required_inputs": list(self.required_inputs),
"secret_boundary": self.secret_boundary,
}
@dataclass(frozen=True, slots=True)
class InfrastructureCapabilityReceipt:
installation_id: str
profile: str
capabilities: tuple[InfrastructureCapability, ...]
post_install_tasks: tuple[InfrastructurePostInstallTask, ...]
schema_version: int = 1
def capability(self, capability_id: str) -> InfrastructureCapability | None:
return next(
(item for item in self.capabilities if item.id == capability_id),
None,
)
def tasks_for(
self,
*,
capability_id: str | None = None,
owner_module: str | None = None,
) -> tuple[InfrastructurePostInstallTask, ...]:
return tuple(
item
for item in self.post_install_tasks
if (capability_id is None or item.capability_id == capability_id)
and (owner_module is None or item.owner_module == owner_module)
)
def to_dict(self) -> dict[str, object]:
return {
"schema_version": self.schema_version,
"installation_id": self.installation_id,
"profile": self.profile,
"capabilities": [item.to_dict() for item in self.capabilities],
"post_install_tasks": [
item.to_dict() for item in self.post_install_tasks
],
}
def load_infrastructure_capability_receipt(
path: Path | str | None = None,
) -> InfrastructureCapabilityReceipt | None:
configured_path = path
if configured_path is None:
raw_path = os.getenv(DEPLOYMENT_CAPABILITIES_ENV, "").strip()
if not raw_path:
return None
configured_path = raw_path
return read_infrastructure_capability_receipt(Path(configured_path))
def read_infrastructure_capability_receipt(
path: Path,
) -> InfrastructureCapabilityReceipt:
if path.is_symlink() or not path.is_file():
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt is not a regular file."
)
try:
expected_size = path.stat().st_size
if expected_size > MAX_CAPABILITY_DOCUMENT_BYTES:
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt exceeds 256 KiB."
)
raw = path.read_bytes()
except OSError as exc:
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt could not be read."
) from exc
if len(raw) != expected_size:
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt changed while being read."
)
try:
payload = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt is not valid UTF-8 JSON."
) from exc
return infrastructure_capability_receipt_from_mapping(payload)
def infrastructure_capability_receipt_from_mapping(
payload: object,
) -> InfrastructureCapabilityReceipt:
if (
not isinstance(payload, Mapping)
or type(payload.get("schema_version")) is not int
or payload.get("schema_version") != 1
):
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt has an unsupported schema."
)
raw_capabilities = payload.get("capabilities")
if not isinstance(raw_capabilities, list) or len(raw_capabilities) > 100:
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt has invalid capabilities."
)
capabilities = tuple(_capability(item) for item in raw_capabilities)
capability_ids = [item.id for item in capabilities]
if len(capability_ids) != len(set(capability_ids)):
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt repeats a capability id."
)
raw_tasks = payload.get("post_install_tasks", [])
if not isinstance(raw_tasks, list) or len(raw_tasks) > 100:
raise InfrastructureCapabilityReceiptError(
"Deployment capability receipt has invalid post-install tasks."
)
tasks = tuple(_task(item) for item in raw_tasks)
known_capability_ids = set(capability_ids)
if any(item.capability_id not in known_capability_ids for item in tasks):
raise InfrastructureCapabilityReceiptError(
"Deployment post-install task references an unknown capability."
)
return InfrastructureCapabilityReceipt(
installation_id=_required_text(payload, "installation_id", maximum=100),
profile=_required_text(payload, "profile", maximum=100),
capabilities=capabilities,
post_install_tasks=tasks,
)
def deployment_capability_status(
path: Path | str | None = None,
) -> dict[str, object]:
try:
receipt = load_infrastructure_capability_receipt(path)
except InfrastructureCapabilityReceiptError as exc:
return _unavailable_status(configured=True, error=str(exc))
if receipt is None:
return _unavailable_status(configured=False, error=None)
return {
"configured": True,
"available": True,
**receipt.to_dict(),
"error": None,
}
def collect_infrastructure_dependency_inventory(
registry: object,
*,
installation_id: str,
observed_at: datetime | None = None,
) -> InfrastructureDependencyInventory:
"""Collect actual module-owned dependencies without importing module internals."""
normalized_installation_id = installation_id.strip()
if not normalized_installation_id or len(normalized_installation_id) > 100:
raise ValueError("Infrastructure dependency installation id is invalid.")
capability_names = getattr(registry, "capability_names", None)
capability = getattr(registry, "capability", None)
if not callable(capability_names) or not callable(capability):
raise ValueError("Infrastructure dependency inventory requires a module registry.")
provider_names = tuple(
name
for name in capability_names()
if isinstance(name, str)
and name.startswith(INFRASTRUCTURE_DEPENDENCY_PROVIDER_CAPABILITY_PREFIX)
)
reports: list[InfrastructureDependencyProviderReport] = []
dependencies: list[InfrastructureDependency] = []
inspected_capability_ids: set[str] = set()
complete = True
for provider_name in sorted(provider_names):
expected_module_id = provider_name.removeprefix(
INFRASTRUCTURE_DEPENDENCY_PROVIDER_CAPABILITY_PREFIX
)
module_id = expected_module_id or "unknown"
declared_ids: tuple[str, ...] = ()
try:
provider = capability(provider_name)
if not isinstance(provider, InfrastructureDependencyProvider):
raise TypeError("provider does not implement the inventory contract")
module_id = provider.module_id.strip()
declared_ids = tuple(
sorted(
{
item.strip()
for item in provider.capability_ids
if isinstance(item, str) and item.strip()
}
)
)
if (
module_id != expected_module_id
or len(module_id) > 120
or any(ord(char) < 32 for char in module_id)
or not declared_ids
or len(declared_ids) > 30
or any(
len(item) > 120 or any(ord(char) < 32 for char in item)
for item in declared_ids
)
):
raise ValueError("provider identity or capability declaration is invalid")
provider_dependencies = tuple(provider.infrastructure_dependencies())
if len(provider_dependencies) > 10_000:
raise ValueError("provider dependency inventory is too large")
seen_refs: set[tuple[str, str, str]] = set()
for item in provider_dependencies:
if not isinstance(item, InfrastructureDependency):
raise TypeError("provider returned an invalid dependency")
if item.module_id != module_id or item.capability_id not in declared_ids:
raise ValueError("provider returned a dependency outside its declaration")
identity = (
item.capability_id,
item.dependency_type,
item.dependency_ref,
)
if identity in seen_refs:
raise ValueError("provider returned a duplicate dependency")
seen_refs.add(identity)
if len(dependencies) + len(provider_dependencies) > 10_000:
raise ValueError("combined dependency inventory is too large")
dependencies.extend(provider_dependencies)
inspected_capability_ids.update(declared_ids)
reports.append(
InfrastructureDependencyProviderReport(
module_id=module_id,
capability_ids=declared_ids,
state="complete",
dependency_count=len(provider_dependencies),
)
)
except Exception as exc:
complete = False
inspected_capability_ids.update(declared_ids)
reports.append(
InfrastructureDependencyProviderReport(
module_id=module_id,
capability_ids=declared_ids,
state="error",
dependency_count=0,
error=(
f"{type(exc).__name__}: provider inventory could not be completed"
),
)
)
timestamp = observed_at or datetime.now(UTC)
if timestamp.tzinfo is None:
timestamp = timestamp.replace(tzinfo=UTC)
return InfrastructureDependencyInventory(
installation_id=normalized_installation_id,
generated_at=timestamp.astimezone(UTC).isoformat(),
complete=complete,
inspected_capability_ids=tuple(sorted(inspected_capability_ids)),
providers=tuple(
sorted(reports, key=lambda item: (item.module_id, item.capability_ids))
),
dependencies=tuple(
sorted(
dependencies,
key=lambda item: (
item.capability_id,
item.module_id,
item.dependency_type,
item.dependency_ref,
),
)
),
)
def _capability(value: object) -> InfrastructureCapability:
if not isinstance(value, Mapping):
raise InfrastructureCapabilityReceiptError(
"Deployment capability entries must be objects."
)
state = _required_text(value, "state", maximum=40)
if state not in CAPABILITY_STATES:
raise InfrastructureCapabilityReceiptError(
f"Deployment capability state is unsupported: {state!r}."
)
normalized_endpoint = _normalized_endpoint(value.get("endpoint", {}))
secret_refs = _string_list(value.get("secret_refs"), maximum_items=30)
if any(not _ENV_REFERENCE_RE.fullmatch(item) for item in secret_refs):
raise InfrastructureCapabilityReceiptError(
"Deployment capability secrets must use environment references."
)
return InfrastructureCapability(
id=_required_text(value, "id", maximum=120),
label=_required_text(value, "label", maximum=200),
state=state,
source=_required_text(value, "source", maximum=120),
detail=_required_text(value, "detail", maximum=1000),
endpoint=normalized_endpoint,
secret_refs=secret_refs,
dependent_modules=_string_list(
value.get("dependent_modules"),
maximum_items=100,
),
)
def _normalized_endpoint(value: object) -> dict[str, object]:
if not isinstance(value, Mapping) or len(value) > 10:
raise InfrastructureCapabilityReceiptError(
"Deployment capability endpoint metadata is invalid."
)
endpoint: dict[str, object] = {}
for key, raw in value.items():
if not isinstance(key, str) or not key or len(key) > 50:
raise InfrastructureCapabilityReceiptError(
"Deployment capability endpoint key is invalid."
)
if any(
marker in key.casefold()
for marker in ("password", "secret", "token", "credential")
):
raise InfrastructureCapabilityReceiptError(
"Deployment capability endpoint metadata contains a secret field."
)
if key.casefold() == "port" and (
type(raw) is not int or not 1 <= raw <= 65535
):
raise InfrastructureCapabilityReceiptError(
"Deployment capability endpoint port is invalid."
)
if isinstance(raw, bool) or raw is None:
endpoint[key] = raw
elif isinstance(raw, int):
endpoint[key] = raw
elif isinstance(raw, str) and len(raw) <= 500:
endpoint[key] = raw
else:
raise InfrastructureCapabilityReceiptError(
"Deployment capability endpoint value is invalid."
)
return endpoint
def _task(value: object) -> InfrastructurePostInstallTask:
if not isinstance(value, Mapping):
raise InfrastructureCapabilityReceiptError(
"Deployment post-install task entries must be objects."
)
return InfrastructurePostInstallTask(
id=_required_text(value, "id", maximum=120),
resume_key=_required_text(value, "resume_key", maximum=240),
capability_id=_required_text(value, "capability_id", maximum=120),
state=_required_text(value, "state", maximum=40),
owner_module=_required_text(value, "owner_module", maximum=120),
summary=_required_text(value, "summary", maximum=1000),
required_inputs=_string_list(
value.get("required_inputs"),
maximum_items=30,
),
secret_boundary=_required_text(
value,
"secret_boundary",
maximum=120,
),
)
def _required_text(
value: Mapping[str, Any],
key: str,
*,
maximum: int,
) -> str:
raw = value.get(key)
text = str(raw).strip() if raw is not None else ""
if not text or len(text) > maximum:
raise InfrastructureCapabilityReceiptError(
f"Deployment capability field {key!r} is invalid."
)
return text
def _string_list(value: object, *, maximum_items: int) -> tuple[str, ...]:
if not isinstance(value, list) or len(value) > maximum_items:
raise InfrastructureCapabilityReceiptError(
"Deployment capability list field is invalid."
)
result: list[str] = []
for item in value:
if not isinstance(item, str) or not item.strip() or len(item) > 500:
raise InfrastructureCapabilityReceiptError(
"Deployment capability list item is invalid."
)
result.append(item.strip())
return tuple(result)
def _unavailable_status(*, configured: bool, error: str | None) -> dict[str, object]:
return {
"configured": configured,
"available": False,
"schema_version": None,
"installation_id": None,
"profile": None,
"capabilities": [],
"post_install_tasks": [],
"error": error,
}
__all__ = [
"CAPABILITY_STATES",
"DEPLOYMENT_CAPABILITIES_ENV",
"INFRASTRUCTURE_DEPENDENCY_PROVIDER_CAPABILITY_PREFIX",
"InfrastructureCapability",
"InfrastructureCapabilityReceipt",
"InfrastructureCapabilityReceiptError",
"InfrastructureDependency",
"InfrastructureDependencyInventory",
"InfrastructureDependencyProvider",
"InfrastructureDependencyProviderReport",
"InfrastructurePostInstallTask",
"collect_infrastructure_dependency_inventory",
"deployment_capability_status",
"infrastructure_capability_receipt_from_mapping",
"load_infrastructure_capability_receipt",
"read_infrastructure_capability_receipt",
]
+11 -4
View File
@@ -698,12 +698,15 @@ def _validate_module_catalog_trust(
"A module catalog source is configured without a trusted keyring file.",
"Pin the published GovOPlaN catalog keyring locally and set GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE.",
)
if not _clean(env.get("GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL")):
if not (
_clean(env.get("GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS"))
or _clean(env.get("GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL"))
):
collector.add(
"error",
"GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL",
"GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS",
"A module catalog source is configured without an approved release channel.",
"Set GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL=stable or another approved deployment channel.",
"Set GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable or another approved deployment channel.",
)
@@ -734,6 +737,8 @@ CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
PLATFORM_EVENT_OUTBOX_MAX_ATTEMPTS=8
PLATFORM_EVENT_OUTBOX_TERMINAL_RETENTION_DAYS=90
SCHEDULING_CANCELLATION_NOTICE_DAYS=30
SCHEDULING_PUBLIC_SELF_ENROLLMENT_ENABLED=true
SCHEDULING_PUBLIC_SELF_ENROLLMENT_MAX_CAPACITY=10000
# Deployment-wide connector egress policy. Enable private networks only when
# this installation intentionally integrates with internal services.
@@ -777,7 +782,7 @@ DEV_MAILBOX_API_ENABLED=false
GOVOPLAN_MODULE_PACKAGE_CATALOG_URL=https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_TRUSTED_KEYS_FILE=/etc/govoplan/catalog-keyring.json
GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL=stable
GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS=stable
"""
@@ -815,6 +820,8 @@ CALENDAR_OUTBOX_TERMINAL_RETENTION_DAYS=90
PLATFORM_EVENT_OUTBOX_MAX_ATTEMPTS=8
PLATFORM_EVENT_OUTBOX_TERMINAL_RETENTION_DAYS=90
SCHEDULING_CANCELLATION_NOTICE_DAYS=30
SCHEDULING_PUBLIC_SELF_ENROLLMENT_ENABLED=true
SCHEDULING_PUBLIC_SELF_ENROLLMENT_MAX_CAPACITY=10000
GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS=true
GOVOPLAN_CONNECTOR_MAX_STRUCTURED_RESPONSE_BYTES=16777216
+52
View File
@@ -9,6 +9,7 @@ from typing import Protocol, runtime_checkable
CAPABILITY_MAIL_DELIVERY_OUTBOX = "mail.delivery_outbox"
CAPABILITY_MAIL_NOTIFICATION_DELIVERY = "mail.notificationDelivery"
CAPABILITY_MAIL_BOUNCE_PROCESSING = "mail.bounce_processing"
CAPABILITY_MAIL_POSTBOX_BRIDGE = "mail.postbox_bridge"
@dataclass(frozen=True, slots=True)
@@ -81,6 +82,28 @@ class MailBounceObservationRef:
evidence: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class MailPostboxBridgeRequest:
tenant_id: str
target: object
profile_id: str
folder: str
uid: str
uidvalidity: str
raw_message: bytes
classification: str = "internal"
metadata: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class MailPostboxBridgeResult:
postbox_id: str
message_id: str
delivery_id: str
duplicate: bool
source_digest: str
@runtime_checkable
class MailBounceProcessingProvider(Protocol):
"""Mail-owned DSN ingestion and durable correlation boundary."""
@@ -116,6 +139,17 @@ class MailBounceProcessingProvider(Protocol):
...
@runtime_checkable
class MailPostboxBridgeProvider(Protocol):
"""Translate one immutable Mail observation into Postbox delivery."""
def bridge_message(
self,
session: object,
request: MailPostboxBridgeRequest,
) -> MailPostboxBridgeResult: ...
def notification_mail_delivery_provider(
registry: object | None,
) -> NotificationMailDeliveryProvider | None:
@@ -150,3 +184,21 @@ def mail_bounce_processing_provider(
"MailBounceProcessingProvider"
)
return provider
def mail_postbox_bridge_provider(
registry: object | None,
) -> MailPostboxBridgeProvider | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_MAIL_POSTBOX_BRIDGE)
):
return None
provider = registry.require_capability(CAPABILITY_MAIL_POSTBOX_BRIDGE)
if not isinstance(provider, MailPostboxBridgeProvider):
raise TypeError(
"mail.postbox_bridge provider does not implement "
"MailPostboxBridgeProvider"
)
return provider
+358 -17
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
from collections import defaultdict
from collections.abc import Iterable, Mapping
from contextlib import AbstractContextManager, closing
from dataclasses import dataclass, field
from dataclasses import dataclass, field, replace
from datetime import UTC, datetime
from importlib import metadata
import hashlib
@@ -18,6 +18,7 @@ import sqlite3
import stat
import subprocess # nosec B404 - installer commands are structured and policy-validated before execution.
import sys
import tempfile
import tomllib
from typing import Any, Literal
import time
@@ -62,6 +63,7 @@ MIGRATION_TASK_PHASES = (
MIGRATION_TASK_MUTATING_PHASES = {"pre_migration_prepare", "post_migration_backfill"}
MIGRATION_TASK_REVIEW_SAFETY = {"requires_review", "forward_only", "destructive"}
MIGRATION_TASK_BLOCKING_SAFETY = {"forward_only", "destructive"}
MAX_PACKAGE_ARTIFACT_BYTES = 512 * 1024 * 1024
@dataclass(frozen=True, slots=True)
@@ -464,14 +466,20 @@ def _package_target_action_preflight_issues(
"Python installs must include the distribution package name so rollback can uninstall newly added packages.",
item.module_id,
))
if item.python_ref and not _looks_pinned_dependency_ref(item.python_ref):
if item.python_ref and not (
_looks_pinned_dependency_ref(item.python_ref)
or _artifact_ref_is_digest_pinned(item, "python", item.python_ref)
):
issues.append(ModuleInstallerIssue(
"blocker",
"unpinned_python_ref",
"Python install refs must be pinned to an exact version or tagged git ref.",
item.module_id,
))
if item.webui_ref and not _looks_pinned_dependency_ref(item.webui_ref):
if item.webui_ref and not (
_looks_pinned_dependency_ref(item.webui_ref)
or _artifact_ref_is_digest_pinned(item, "webui", item.webui_ref)
):
issues.append(ModuleInstallerIssue(
"blocker",
"unpinned_webui_ref",
@@ -481,6 +489,23 @@ def _package_target_action_preflight_issues(
return tuple(issues)
def _artifact_ref_is_digest_pinned(
item: ModuleInstallPlanItem,
kind: str,
package_ref: str,
) -> bool:
metadata = _artifact_metadata(item.artifact_integrity, kind)
if metadata is None:
return False
expected_ref = _artifact_text(metadata, "ref") or _artifact_text(metadata, "expected_ref")
sha256 = _artifact_text(metadata, "sha256")
return bool(
expected_ref == package_ref
and sha256
and re.fullmatch(r"[0-9a-f]{64}", sha256.lower())
)
def _frontend_rebuild_preflight_issues(
*,
frontend_rebuild_required: bool,
@@ -519,6 +544,7 @@ def run_module_install_plan(
) -> ModuleInstallerRunResult:
maintenance_mode = saved_maintenance_mode(session)
effective_runtime_dir = runtime_dir or default_installer_runtime_dir(database_url)
effective_plan = plan
preflight = module_install_preflight(
plan=plan,
available=available,
@@ -532,9 +558,30 @@ def run_module_install_plan(
if not preflight.allowed:
raise ModuleInstallerError("Install preflight is blocked: " + "; ".join(issue.message for issue in preflight.issues if issue.severity == "blocker"))
if not dry_run:
effective_plan = acquire_catalog_package_artifacts(
plan,
runtime_dir=effective_runtime_dir,
)
preflight = module_install_preflight(
plan=effective_plan,
available=available,
current_enabled=current_enabled,
desired_enabled=desired_enabled,
maintenance_mode=maintenance_mode.enabled,
session=session,
webui_root=webui_root,
runtime_dir=effective_runtime_dir,
)
if not preflight.allowed:
raise ModuleInstallerError(
"Install preflight is blocked after artifact acquisition: "
+ "; ".join(issue.message for issue in preflight.issues if issue.severity == "blocker")
)
state = _prepare_module_install_run(
session=session,
plan=plan,
plan=effective_plan,
preflight=preflight,
database_url=database_url,
effective_runtime_dir=effective_runtime_dir,
@@ -556,7 +603,7 @@ def run_module_install_plan(
executed, failed_error = _execute_module_install_run(
session=session,
plan=plan,
plan=effective_plan,
available=available,
effective_runtime_dir=effective_runtime_dir,
state=state,
@@ -566,7 +613,7 @@ def run_module_install_plan(
return _failed_module_install_run_result(
session=session,
state=state,
plan=plan,
plan=effective_plan,
executed=executed,
failed_error=failed_error,
effective_runtime_dir=effective_runtime_dir,
@@ -578,7 +625,7 @@ def run_module_install_plan(
return _applied_module_install_run_result(
session=session,
plan=plan,
plan=effective_plan,
desired_enabled=desired_enabled,
activate_installed_modules=activate_installed_modules,
remove_uninstalled_modules_from_desired=remove_uninstalled_modules_from_desired,
@@ -1568,9 +1615,11 @@ def _structured_item_commands(
webui_changed = False
if item.action in PACKAGE_TARGET_ACTIONS:
if item.python_ref:
commands.append(_structured_command([sys.executable, "-m", "pip", "install", item.python_ref], source="module-plan.python"))
python_source = _verified_artifact_install_ref(item, "python") or item.python_ref
commands.append(_structured_command([sys.executable, "-m", "pip", "install", python_source], source="module-plan.python"))
if item.webui_package and item.webui_ref and webui_root is not None:
commands.append(_structured_command([npm_bin, "pkg", "set", f"dependencies.{item.webui_package}={item.webui_ref}"], cwd=webui_root, source="module-plan.webui"))
webui_source = _verified_artifact_install_ref(item, "webui") or item.webui_ref
commands.append(_structured_command([npm_bin, "pkg", "set", f"dependencies.{item.webui_package}={webui_source}"], cwd=webui_root, source="module-plan.webui"))
webui_changed = True
elif item.action == "uninstall":
if item.python_package:
@@ -1581,6 +1630,14 @@ def _structured_item_commands(
return tuple(commands), webui_changed
def _verified_artifact_install_ref(item: ModuleInstallPlanItem, kind: str) -> str | None:
metadata = _artifact_metadata(item.artifact_integrity, kind)
path = _artifact_path(metadata) if metadata is not None else None
if path is None:
return None
return path.as_uri() if kind == "webui" else str(path)
def _structured_webui_followup_commands(
*,
webui_changed: bool,
@@ -1932,9 +1989,7 @@ def _package_catalog_preflight_issues(
return ()
catalog_items = tuple(item for item in package_items if item.source == "catalog")
try:
from govoplan_core.core.module_package_catalog import validate_module_package_catalog
result = validate_module_package_catalog()
result = _validate_catalog_for_plan(catalog_items)
except Exception as exc:
return _catalog_validation_exception_issues(exc, catalog_items=bool(catalog_items))
issues = list(_catalog_validation_result_issues(result, catalog_items=bool(catalog_items)))
@@ -1942,10 +1997,33 @@ def _package_catalog_preflight_issues(
return tuple(issues)
issues.extend(_catalog_warning_issues(result))
if catalog_items:
issues.extend(_catalog_plan_binding_issues(catalog_items, result))
issues.extend(_selected_catalog_interface_issues(catalog_items, result, available))
return tuple(issues)
def _validate_catalog_for_plan(
catalog_items: tuple[ModuleInstallPlanItem, ...],
) -> dict[str, object]:
from govoplan_core.core.module_package_catalog import (
OFFICIAL_MODULE_PACKAGE_CATALOG_URL,
validate_module_package_catalog,
validate_official_module_package_catalog,
)
configured = validate_module_package_catalog()
if configured.get("configured") or not catalog_items:
return configured
sources = {
str(item.catalog.get("source") or "")
for item in catalog_items
if isinstance(item.catalog, Mapping)
}
if sources == {OFFICIAL_MODULE_PACKAGE_CATALOG_URL}:
return validate_official_module_package_catalog()
return configured
def _catalog_validation_exception_issues(exc: Exception, *, catalog_items: bool) -> tuple[ModuleInstallerIssue, ...]:
severity: IssueSeverity = "blocker" if catalog_items else "warning"
return (ModuleInstallerIssue(
@@ -1985,6 +2063,97 @@ def _catalog_warning_issues(result: Mapping[str, object]) -> tuple[ModuleInstall
return tuple(ModuleInstallerIssue("warning", "catalog_warning", str(warning)) for warning in warnings)
def _catalog_plan_binding_issues(
items: tuple[ModuleInstallPlanItem, ...],
validation: Mapping[str, object],
) -> tuple[ModuleInstallerIssue, ...]:
"""Require every trusted plan row to match its signed catalog entry exactly."""
modules = _catalog_modules_by_id(validation)
issues: list[ModuleInstallerIssue] = []
for item in items:
entry = modules.get(item.module_id)
if entry is None or entry.get("action") not in PACKAGE_TARGET_ACTIONS:
issues.append(ModuleInstallerIssue(
"blocker",
"catalog_plan_entry_missing",
f"The validated catalog no longer contains an install or update entry for {item.module_id!r}.",
item.module_id,
))
continue
mismatches = _catalog_plan_entry_mismatches(item, entry, validation)
if mismatches:
issues.append(ModuleInstallerIssue(
"blocker",
"catalog_plan_binding_mismatch",
(
"The saved package plan differs from its validated signed catalog entry "
f"for: {', '.join(mismatches)}. Remove and add the catalog item again."
),
item.module_id,
))
return tuple(issues)
def _catalog_plan_entry_mismatches(
item: ModuleInstallPlanItem,
entry: Mapping[str, object],
validation: Mapping[str, object],
) -> tuple[str, ...]:
mismatches: list[str] = []
for attribute in ("python_package", "python_ref", "webui_package", "webui_ref"):
if getattr(item, attribute) != _catalog_optional_string(entry, attribute):
mismatches.append(attribute)
if _catalog_integrity_identity(item.artifact_integrity) != _catalog_integrity_identity(
entry.get("artifact_integrity")
):
mismatches.append("artifact_integrity")
catalog = item.catalog if isinstance(item.catalog, Mapping) else {}
expected_snapshot = {
"source": validation.get("source") or validation.get("path"),
"channel": validation.get("channel"),
"sequence": validation.get("sequence"),
"signed": bool(validation.get("signed")),
"trusted": bool(validation.get("trusted")),
"key_id": validation.get("key_id"),
}
for attribute, expected in expected_snapshot.items():
actual = catalog.get(attribute)
if actual != expected:
mismatches.append(f"catalog.{attribute}")
return tuple(mismatches)
def _catalog_integrity_identity(value: object) -> dict[str, dict[str, object]]:
if not isinstance(value, Mapping):
return {}
identity: dict[str, dict[str, object]] = {}
for kind in ("python", "webui"):
raw = value.get(kind)
if not isinstance(raw, Mapping):
continue
identity[kind] = {
field: raw.get(field)
for field in (
"ref",
"url",
"filename",
"sha256",
"size",
"integrity",
"sbom_url",
"provenance_url",
"registry_identity",
"git_ref",
"source_commit",
)
if raw.get(field) is not None
}
return identity
def _module_install_target_plan(
plan: ModuleInstallPlan,
available: Mapping[str, ModuleManifest],
@@ -2817,12 +2986,15 @@ def _topological_cycle_ids(incoming: Mapping[str, set[str]]) -> tuple[str, ...]:
def _catalog_modules_for_target_plan(
planned_items: tuple[ModuleInstallPlanItem, ...],
) -> dict[str, Mapping[str, object]]:
if not any(item.source == "catalog" and item.action in PACKAGE_TARGET_ACTIONS for item in planned_items):
catalog_items = tuple(
item
for item in planned_items
if item.source == "catalog" and item.action in PACKAGE_TARGET_ACTIONS
)
if not catalog_items:
return {}
try:
from govoplan_core.core.module_package_catalog import validate_module_package_catalog
result = validate_module_package_catalog()
result = _validate_catalog_for_plan(catalog_items)
except Exception:
return {}
if result.get("valid") is not True:
@@ -3772,6 +3944,156 @@ def _configured_require_artifact_integrity() -> bool:
return os.getenv("GOVOPLAN_MODULE_INSTALLER_REQUIRE_ARTIFACT_INTEGRITY", "").strip().lower() in {"1", "true", "yes", "on"}
def acquire_catalog_package_artifacts(
plan: ModuleInstallPlan,
*,
runtime_dir: Path,
) -> ModuleInstallPlan:
"""Materialize trusted catalog archives before package mutation."""
items: list[ModuleInstallPlanItem] = []
for item in plan.items:
if item.status != "planned" or item.action not in PACKAGE_TARGET_ACTIONS:
items.append(item)
continue
raw_integrity = item.artifact_integrity
if not isinstance(raw_integrity, Mapping):
items.append(item)
continue
integrity: dict[str, object] = dict(raw_integrity)
changed = False
for kind in ("python", "webui"):
metadata = _artifact_metadata(integrity, kind)
if metadata is None or _artifact_path(metadata) is not None:
continue
if not _catalog_artifact_acquisition_ready(item, metadata):
continue
updated = dict(metadata)
updated["artifact_path"] = str(
_acquire_package_artifact(
metadata,
runtime_dir=runtime_dir,
module_id=item.module_id,
kind=kind,
)
)
integrity[kind] = updated
changed = True
items.append(replace(item, artifact_integrity=integrity) if changed else item)
return replace(plan, items=tuple(items))
def _catalog_artifact_acquisition_ready(
item: ModuleInstallPlanItem,
metadata: Mapping[str, object],
) -> bool:
catalog = item.catalog
if (
item.source != "catalog"
or not isinstance(catalog, Mapping)
or catalog.get("signed") is not True
or catalog.get("trusted") is not True
):
return False
url = _artifact_text(metadata, "url")
filename = _artifact_text(metadata, "filename")
sha256 = _artifact_text(metadata, "sha256")
size = metadata.get("size")
return bool(
url
and url.startswith("https://")
and filename
and Path(filename).name == filename
and sha256
and re.fullmatch(r"[0-9a-f]{64}", sha256.lower())
and isinstance(size, int)
and not isinstance(size, bool)
and 0 < size <= MAX_PACKAGE_ARTIFACT_BYTES
)
def _acquire_package_artifact(
metadata: Mapping[str, object],
*,
runtime_dir: Path,
module_id: str,
kind: str,
) -> Path:
url = validate_http_url(_artifact_text(metadata, "url") or "", label=f"{kind.capitalize()} package URL")
if not url.startswith("https://"):
raise ModuleInstallerError(f"{kind.capitalize()} package URL must use HTTPS.")
filename = _artifact_text(metadata, "filename") or ""
expected_sha256 = (_artifact_text(metadata, "sha256") or "").lower()
expected_size = metadata.get("size")
if (
not filename
or Path(filename).name != filename
or re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._+!-]{0,255}", filename) is None
or re.fullmatch(r"[0-9a-f]{64}", expected_sha256) is None
or not isinstance(expected_size, int)
or isinstance(expected_size, bool)
or not 0 < expected_size <= MAX_PACKAGE_ARTIFACT_BYTES
):
raise ModuleInstallerError(f"Catalog artifact metadata is incomplete for {module_id}/{kind}.")
cache_root = runtime_dir / "artifacts"
_ensure_private_artifact_directory(cache_root)
digest_root = cache_root / expected_sha256
_ensure_private_artifact_directory(digest_root)
target = digest_root / filename
if target.exists() or target.is_symlink():
if target.is_symlink() or not target.is_file():
raise ModuleInstallerError(f"Cached package artifact is not a regular file: {target}")
if target.stat().st_size != expected_size or _sha256_file(target) != expected_sha256:
raise ModuleInstallerError(f"Cached package artifact does not match its catalog identity: {target}")
return target
try:
response = fetch_http(
url,
timeout=float(os.getenv("GOVOPLAN_MODULE_INSTALLER_DOWNLOAD_TIMEOUT_SECONDS", "120")),
label=f"{module_id} {kind} package URL",
max_bytes=min(expected_size + 1, MAX_PACKAGE_ARTIFACT_BYTES),
)
except (OSError, ValueError) as exc:
raise ModuleInstallerError(f"Could not download {module_id} {kind} package: {exc}") from exc
if response.status < 200 or response.status >= 300:
raise ModuleInstallerError(f"Could not download {module_id} {kind} package: HTTP {response.status}.")
if len(response.body) != expected_size or hashlib.sha256(response.body).hexdigest() != expected_sha256:
raise ModuleInstallerError(f"Downloaded {module_id} {kind} package does not match its signed catalog identity.")
temporary_path: Path | None = None
try:
with tempfile.NamedTemporaryFile(
mode="wb",
prefix=f".{filename}.",
suffix=".tmp",
dir=digest_root,
delete=False,
) as handle:
temporary_path = Path(handle.name)
handle.write(response.body)
handle.flush()
os.fsync(handle.fileno())
temporary_path.chmod(0o600)
os.replace(temporary_path, target)
target.chmod(0o600)
except OSError as exc:
if temporary_path is not None:
temporary_path.unlink(missing_ok=True)
raise ModuleInstallerError(f"Could not cache {module_id} {kind} package.") from exc
return target
def _ensure_private_artifact_directory(path: Path) -> None:
if path.is_symlink():
raise ModuleInstallerError(f"Installer artifact cache must not be a symlink: {path}")
path.mkdir(parents=True, mode=0o700, exist_ok=True)
path.chmod(0o700)
if not path.is_dir() or stat.S_IMODE(path.stat().st_mode) != 0o700:
raise ModuleInstallerError(f"Installer artifact cache is not private: {path}")
def _verify_artifact_integrity(
planned_items: tuple[ModuleInstallPlanItem, ...],
*,
@@ -3839,7 +4161,17 @@ def _verify_artifact_metadata(
}
if package_name:
record["package"] = package_name
for key in ("sha256", "sbom_url", "provenance_url", "registry_identity", "git_ref"):
for key in (
"sha256",
"url",
"filename",
"integrity",
"sbom_url",
"provenance_url",
"registry_identity",
"git_ref",
"source_commit",
):
value = _artifact_text(metadata, key)
if value:
record[key] = value
@@ -3860,6 +4192,15 @@ def _verify_artifact_metadata(
item.module_id,
))
return record, tuple(issues)
if artifact_path is None and _catalog_artifact_acquisition_ready(item, metadata):
record["acquisition_pending"] = True
issues.append(ModuleInstallerIssue(
"info",
"artifact_acquisition_pending",
f"{kind.capitalize()} artifact will be downloaded and verified by the installer daemon before package mutation.",
item.module_id,
))
return record, tuple(issues)
if artifact_path is None:
issues.append(ModuleInstallerIssue(
"blocker" if require_verified else "warning",
@@ -6,6 +6,7 @@ from collections import defaultdict
from collections.abc import Mapping
from dataclasses import dataclass
from datetime import UTC, datetime
from importlib.resources import files
from pathlib import Path
import json
import os
@@ -17,6 +18,10 @@ from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
from govoplan_core.core.versioning import format_version_range, version_range_is_valid, version_satisfies_range
from govoplan_core.core.information_governance import (
information_governance_from_mapping,
information_governance_maturity_issues,
)
from govoplan_core.core.provider_governance import (
external_provider_from_mapping,
module_architecture_from_mapping,
@@ -25,6 +30,11 @@ from govoplan_core.core.provider_governance import (
from govoplan_core.security.http_fetch import fetch_http_text, is_http_url
_INTERFACE_NAME_RE = re.compile(r"^[a-z][a-z0-9_]*(?:\.[a-z][a-z0-9_]*)+$")
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
_ARTIFACT_FILENAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._+!-]{0,255}$")
_SOURCE_REPOSITORY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,254}$")
_SOURCE_REF_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/+!-]{0,127}$")
_SOURCE_COMMIT_RE = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$")
CATALOG_MIGRATION_SAFETY = ("automatic", "requires_review", "forward_only", "destructive")
CATALOG_MIGRATION_TASK_PHASES = (
"pre_migration_check",
@@ -32,6 +42,8 @@ CATALOG_MIGRATION_TASK_PHASES = (
"post_migration_backfill",
"post_migration_verify",
)
OFFICIAL_MODULE_PACKAGE_CATALOG_URL = "https://govoplan.add-ideas.de/catalogs/v1/channels/stable.json"
OFFICIAL_MODULE_PACKAGE_CATALOG_CHANNEL = "stable"
@dataclass(frozen=True, slots=True)
@@ -97,6 +109,18 @@ def validate_module_package_catalog(
return _valid_catalog_result(catalog_source, state)
def validate_official_module_package_catalog() -> dict[str, object]:
"""Read the public GovOPlaN directory against Core's pinned trust anchor."""
keyring = files("govoplan_core").joinpath("resources/catalog-keyring.json").read_text(encoding="utf-8")
return validate_module_package_catalog(
OFFICIAL_MODULE_PACKAGE_CATALOG_URL,
require_trusted=True,
approved_channels=(OFFICIAL_MODULE_PACKAGE_CATALOG_CHANNEL,),
trusted_keys=_parse_trusted_keys(keyring),
)
def _catalog_validation_state(
source: Path | str | None,
*,
@@ -319,7 +343,10 @@ def _configured_require_signature() -> bool:
def _configured_approved_channels() -> tuple[str, ...]:
value = os.getenv("GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS", "").strip()
value = (
os.getenv("GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNELS", "").strip()
or os.getenv("GOVOPLAN_MODULE_PACKAGE_CATALOG_APPROVED_CHANNEL", "").strip()
)
if not value:
return ()
return tuple(item.strip() for item in value.split(",") if item.strip())
@@ -628,8 +655,29 @@ def _normalize_catalog_item(value: Any) -> dict[str, object]:
"requires_interfaces": _normalize_catalog_interface_requirements(value.get("requires_interfaces"), module_id=module_id),
"notes": _optional_str(value, "notes"),
"tags": _string_list(value.get("tags")),
"availability": _catalog_availability(value, module_id=module_id),
"availability_reason": _optional_str(value, "availability_reason"),
"configuration_requirements": _string_list(value.get("configuration_requirements")),
"permissions": _normalize_catalog_permissions(
value.get("permissions"),
module_id=module_id,
),
}
if item["availability"] == "withdrawn" and not item["availability_reason"]:
raise ValueError(
f"Withdrawn module package catalog entry {module_id!r} requires availability_reason."
)
release_notes_url = _optional_str(value, "release_notes_url")
if release_notes_url is not None:
item["release_notes_url"] = _catalog_https_url(
release_notes_url,
label=f"Module package catalog release_notes_url for {module_id!r}",
)
source = _normalize_catalog_source(value.get("source"), module_id=module_id)
if source:
item["source"] = source
raw_architecture = value.get("architecture")
architecture_maturity: str | None = None
if raw_architecture is not None:
if not isinstance(raw_architecture, Mapping):
raise ValueError(
@@ -647,6 +695,27 @@ def _normalize_catalog_item(value: Any) -> dict[str, object]:
+ "; ".join(issues)
)
item["architecture"] = architecture.to_dict()
architecture_maturity = architecture.maturity
raw_information_governance = value.get("information_governance")
if raw_information_governance is not None:
if not isinstance(raw_information_governance, Mapping):
raise ValueError(
"Module package catalog information_governance for "
f"{module_id!r} must be an object."
)
information_governance = information_governance_from_mapping(
raw_information_governance
)
governance_issues = information_governance_maturity_issues(
information_governance,
maturity=architecture_maturity,
)
if governance_issues:
raise ValueError(
"Module package catalog information_governance for "
f"{module_id!r} is invalid: " + "; ".join(governance_issues)
)
item["information_governance"] = information_governance.to_dict()
raw_providers = value.get("external_providers")
if raw_providers is not None:
if not isinstance(raw_providers, list):
@@ -710,6 +779,170 @@ def _normalize_catalog_item(value: Any) -> dict[str, object]:
return item
def _catalog_availability(value: Mapping[str, object], *, module_id: str) -> str:
availability = str(value.get("availability") or "available").strip().lower()
if availability not in {"available", "withdrawn"}:
raise ValueError(
f"Unsupported catalog availability for {module_id!r}: {availability!r}."
)
return availability
def _normalize_catalog_permissions(
value: object,
*,
module_id: str,
) -> list[dict[str, object]]:
if value is None:
return []
if not isinstance(value, list):
raise ValueError(
f"Module package catalog permissions for {module_id!r} must be a list."
)
if len(value) > 1000:
raise ValueError(
f"Module package catalog permissions for {module_id!r} exceed 1000 entries."
)
normalized: list[dict[str, object]] = []
seen: set[str] = set()
for raw in value:
if not isinstance(raw, Mapping):
raise ValueError(
f"Module package catalog permission entries for {module_id!r} must be objects."
)
scope = _bounded_catalog_permission_text(
raw,
"scope",
module_id=module_id,
maximum=200,
)
if scope in seen:
raise ValueError(
f"Module package catalog entry {module_id!r} declares permission {scope!r} more than once."
)
seen.add(scope)
level = _bounded_catalog_permission_text(
raw,
"level",
module_id=module_id,
maximum=20,
)
if level not in {"system", "tenant"}:
raise ValueError(
f"Module package catalog permission {module_id!r}/{scope!r} has unsupported level {level!r}."
)
deprecated = raw.get("deprecated", False)
if not isinstance(deprecated, bool):
raise ValueError(
f"Module package catalog permission {module_id!r}/{scope!r} deprecated must be true or false."
)
normalized.append(
{
"scope": scope,
"label": _bounded_catalog_permission_text(
raw,
"label",
module_id=module_id,
maximum=200,
),
"description": _bounded_catalog_permission_text(
raw,
"description",
module_id=module_id,
maximum=1000,
),
"category": _bounded_catalog_permission_text(
raw,
"category",
module_id=module_id,
maximum=120,
),
"level": level,
"resource": _bounded_catalog_permission_text(
raw,
"resource",
module_id=module_id,
maximum=120,
),
"action": _bounded_catalog_permission_text(
raw,
"action",
module_id=module_id,
maximum=120,
),
"deprecated": deprecated,
}
)
return normalized
def _bounded_catalog_permission_text(
value: Mapping[str, object],
key: str,
*,
module_id: str,
maximum: int,
) -> str:
text = _required_str(value, key)
if len(text) > maximum:
raise ValueError(
f"Module package catalog permission {key!r} for {module_id!r} exceeds {maximum} characters."
)
return text
def _normalize_catalog_source(
value: object,
*,
module_id: str,
) -> dict[str, object]:
if value is None:
return {}
if not isinstance(value, Mapping):
raise ValueError(
f"Module package catalog source for {module_id!r} must be an object."
)
repository = _required_str(value, "repository")
tag = _required_str(value, "tag")
commit = _required_str(value, "commit").lower()
if (
_SOURCE_REPOSITORY_RE.fullmatch(repository) is None
or repository.startswith("/")
or repository.endswith("/")
or ".." in repository.split("/")
):
raise ValueError(
f"Module package catalog source repository for {module_id!r} is invalid."
)
if _SOURCE_REF_RE.fullmatch(tag) is None or ".." in tag.split("/"):
raise ValueError(
f"Module package catalog source tag for {module_id!r} is invalid."
)
if _SOURCE_COMMIT_RE.fullmatch(commit) is None:
raise ValueError(
f"Module package catalog source commit for {module_id!r} is invalid."
)
source: dict[str, object] = {
"repository": repository,
"tag": tag,
"commit": commit,
}
for field in ("repository_url", "revision_url"):
url = _optional_str(value, field)
if url is not None:
source[field] = _catalog_https_url(
url,
label=f"Module package catalog source {field} for {module_id!r}",
)
return source
def _catalog_https_url(value: str, *, label: str) -> str:
if not is_http_url(value) or not value.startswith("https://"):
raise ValueError(f"{label} must use HTTPS.")
return value
def _catalog_migration_safety(value: Any, *, module_id: str) -> str:
if value is None:
return "automatic"
@@ -780,14 +1013,14 @@ def _catalog_optional_positive_int(value: dict[str, Any], key: str, *, module_id
return integer
def _required_str(value: dict[str, Any], key: str) -> str:
def _required_str(value: Mapping[str, Any], key: str) -> str:
item = _optional_str(value, key)
if not item:
raise ValueError(f"Module package catalog entry is missing {key!r}.")
return item
def _optional_str(value: dict[str, Any], key: str) -> str | None:
def _optional_str(value: Mapping[str, Any], key: str) -> str | None:
item = value.get(key)
if item is None:
return None
@@ -943,20 +1176,40 @@ def _normalize_artifact_integrity(value: Any) -> dict[str, object]:
continue
if not isinstance(raw, dict):
raise ValueError(f"Module package catalog artifact_integrity.{key} must be an object.")
clean = {
clean: dict[str, object] = {
field: text
for field in (
"ref",
"path",
"artifact_path",
"url",
"filename",
"sha256",
"integrity",
"sbom_url",
"provenance_url",
"registry_identity",
"git_ref",
"source_commit",
)
if (text := _optional_str(raw, field))
}
url = clean.get("url")
if isinstance(url, str) and (not is_http_url(url) or not url.startswith("https://")):
raise ValueError(f"Module package catalog artifact_integrity.{key}.url must use HTTPS.")
filename = clean.get("filename")
if isinstance(filename, str) and _ARTIFACT_FILENAME_RE.fullmatch(filename) is None:
raise ValueError(f"Module package catalog artifact_integrity.{key}.filename is invalid.")
sha256 = clean.get("sha256")
if isinstance(sha256, str) and _SHA256_RE.fullmatch(sha256.lower()) is None:
raise ValueError(f"Module package catalog artifact_integrity.{key}.sha256 is invalid.")
if isinstance(sha256, str):
clean["sha256"] = sha256.lower()
size = raw.get("size")
if size is not None:
if not isinstance(size, int) or isinstance(size, bool) or size <= 0 or size > 512 * 1024 * 1024:
raise ValueError(f"Module package catalog artifact_integrity.{key}.size is invalid.")
clean["size"] = size
if clean:
normalized[key] = clean
return normalized
+360 -19
View File
@@ -1,9 +1,10 @@
from __future__ import annotations
from collections.abc import Callable, Iterable, Mapping, Sequence
from dataclasses import dataclass, field
from dataclasses import dataclass, field, replace
from typing import Any, Literal, Protocol, TYPE_CHECKING
from govoplan_core.core.information_governance import ModuleInformationGovernance
from govoplan_core.core.ownership import OwnershipProviderRegistration
from govoplan_core.core.provider_governance import (
ExternalProviderDeclaration,
@@ -14,16 +15,22 @@ from govoplan_core.core.views import ViewSurface
if TYPE_CHECKING:
from fastapi import APIRouter
from govoplan_core.core.operations import OperationalCheckProviderRegistration
from govoplan_core.core.operations import (
OperationalCheckProviderRegistration,
RuntimeWorkStatusProviderRegistration,
)
from govoplan_core.core.search import (
SearchProviderRegistration,
SearchSourceProviderRegistration,
)
from govoplan_core.core.tasks import WorkItemProviderRegistration
from govoplan_core.core.workflows import WorkflowDefinitionContribution
SUPPORTED_MANIFEST_CONTRACT_VERSION = "1"
SUPPORTED_FRONTEND_ASSET_MANIFEST_CONTRACT_VERSION = "1"
SUPPORTED_PRESENTATION_CONTRACT_VERSION = "1"
SUPPORTED_PRODUCT_SURFACE_CONTRACT_VERSION = "1"
PermissionLevel = Literal["system", "tenant"]
SubjectType = Literal["account", "membership", "group", "service_account", "tenant"]
@@ -33,7 +40,9 @@ MigrationTaskPhase = Literal[
"post_migration_backfill",
"post_migration_verify",
]
MigrationTaskSafety = Literal["automatic", "requires_review", "forward_only", "destructive"]
MigrationTaskSafety = Literal[
"automatic", "requires_review", "forward_only", "destructive"
]
MigrationTaskStatus = Literal["ok", "warning", "blocked", "skipped"]
@@ -74,8 +83,6 @@ class NavItem:
surface_id: str | None = None
@dataclass(frozen=True, slots=True)
class FrontendRoute:
path: str
@@ -95,6 +102,92 @@ class PublicFrontendRoute:
order: int = 100
@dataclass(frozen=True, slots=True)
class ProductAreaContribution:
"""Assign module-owned surfaces to a user-facing product area."""
id: str
module_id: str
label: str
icon: str
surface_ids: tuple[str, ...]
description: str | None = None
order: int = 100
ProductSurfacePresentation = Literal["task", "reader", "admin", "operator"]
ProductAvailabilityReason = Literal[
"authorization",
"policy",
"configuration",
"disabled",
"capability",
"offline",
"provider_degraded",
]
@dataclass(frozen=True, slots=True)
class ProductAvailabilityExplanation:
"""Explain a product outcome without making package topology user-facing."""
reason: ProductAvailabilityReason
title: str
description: str
resolution: str
responsible_role: str | None = None
@dataclass(frozen=True, slots=True)
class ProductSurfaceContribution:
"""Bind an owner route to a stable, cross-module product identity."""
id: str
module_id: str
label: str
icon: str
entry_path: str
route_path: str
surface_ids: tuple[str, ...]
unavailable: ProductAvailabilityExplanation
description: str | None = None
degraded: ProductAvailabilityExplanation | None = None
presentations: tuple[ProductSurfacePresentation, ...] = ("task",)
capability_ids: tuple[str, ...] = ()
search_source_ids: tuple[str, ...] = ()
help_context_ids: tuple[str, ...] = ()
documentation_topic_ids: tuple[str, ...] = ()
required_all: tuple[str, ...] = ()
required_any: tuple[str, ...] = ()
aliases: tuple[str, ...] = ()
order: int = 100
contract_version: str = SUPPORTED_PRODUCT_SURFACE_CONTRACT_VERSION
@dataclass(frozen=True, slots=True)
class QuickAccessTool:
"""Declare a versioned, bounded module-owned Quick Access tool."""
id: str
module_id: str
category_id: str
label: str
surface_id: str
icon: str
description: str | None = None
full_page_path: str | None = None
required_all: tuple[str, ...] = ()
required_any: tuple[str, ...] = ()
order: int = 100
default_enabled: bool = True
modes: tuple[str, ...] = ("browse",)
contract_version: str = "1"
availability: Literal["global", "active_object"] = "global"
accepted_reference_kinds: tuple[str, ...] = ()
returned_reference_kinds: tuple[str, ...] = ()
help_context_id: str | None = None
@dataclass(frozen=True, slots=True)
class FrontendModule:
module_id: str
@@ -109,6 +202,9 @@ class FrontendModule:
nav_items: tuple[NavItem, ...] = ()
settings_routes: tuple[FrontendRoute, ...] = ()
view_surfaces: tuple[ViewSurface, ...] = ()
product_areas: tuple[ProductAreaContribution, ...] = ()
product_surfaces: tuple[ProductSurfaceContribution, ...] = ()
quick_access_tools: tuple[QuickAccessTool, ...] = ()
@dataclass(frozen=True, slots=True)
@@ -146,7 +242,9 @@ class ModuleMigrationTaskResult:
details: Mapping[str, Any] = field(default_factory=dict)
ModuleMigrationTaskExecutor = Callable[[ModuleMigrationTaskContext], ModuleMigrationTaskResult | None]
ModuleMigrationTaskExecutor = Callable[
[ModuleMigrationTaskContext], ModuleMigrationTaskResult | None
]
@dataclass(frozen=True, slots=True)
@@ -203,7 +301,9 @@ class ModuleUninstallGuardResult:
message: str
UninstallGuardProvider = Callable[[object | None, str], Iterable[ModuleUninstallGuardResult]]
UninstallGuardProvider = Callable[
[object | None, str], Iterable[ModuleUninstallGuardResult]
]
@dataclass(frozen=True, slots=True)
@@ -223,7 +323,9 @@ class ModuleContext:
DocumentationLayer = Literal["always", "configured", "available", "evidence"]
DocumentationLinkKind = Literal["runtime", "api", "repository", "wiki", "public"]
DocumentationType = Literal["admin", "user"]
DocumentationConfigurationState = Literal["enabled", "disabled", "inherited", "unavailable"]
DocumentationConfigurationState = Literal[
"enabled", "disabled", "inherited", "unavailable"
]
DocumentationSourceKind = Literal[
"manifest",
"route",
@@ -238,6 +340,30 @@ DocumentationSourceState = Literal["configured", "disabled", "unavailable"]
CapabilityStability = Literal["experimental", "stable", "deprecated"]
DOCUMENTATION_STRUCTURED_TRANSLATION_VERSION = "1"
DOCUMENTATION_LOCALIZABLE_METADATA_KEYS = frozenset(
{
"admin_explanation",
"consequence_classes",
"consequences",
"constraints",
"current_configuration",
"fields",
"limitations",
"operational_consequences",
"outcome",
"prerequisites",
"privacy_notes",
"purpose",
"result",
"steps",
"user_explanation",
"verification",
"when_used",
}
)
@dataclass(frozen=True, slots=True)
class DocumentationLink:
label: str
@@ -273,12 +399,161 @@ class DocumentationTopic:
configuration_keys: tuple[str, ...] = ()
i18n_key: str | None = None
translations: Mapping[str, Mapping[str, str]] = field(default_factory=dict)
structured_translation_version: str | None = None
structured_translations: Mapping[str, Mapping[str, Any]] = field(
default_factory=dict
)
source_module_id: str | None = None
version_min: str | None = None
version_max_exclusive: str | None = None
metadata: Mapping[str, Any] = field(default_factory=dict)
def localize_documentation_topics(
topics: Iterable[DocumentationTopic],
*,
locale: str,
translations: Mapping[str, Mapping[str, str]],
) -> tuple[DocumentationTopic, ...]:
"""Merge owner-supplied text translations without moving feature content."""
localized: list[DocumentationTopic] = []
for topic in topics:
translated = translations.get(topic.id)
if translated is None:
localized.append(topic)
continue
values = {name: dict(value) for name, value in topic.translations.items()}
values[locale] = {**values.get(locale, {}), **translated}
localized.append(replace(topic, translations=values))
return tuple(localized)
def localizable_documentation_metadata_keys(
topic: DocumentationTopic,
) -> tuple[str, ...]:
"""Return structured metadata keys whose values are public prose."""
return tuple(
sorted(DOCUMENTATION_LOCALIZABLE_METADATA_KEYS.intersection(topic.metadata))
)
def localized_documentation_metadata(
topic: DocumentationTopic,
locale: str,
) -> dict[str, Any]:
"""Overlay one validated structured translation onto source metadata."""
localized = dict(topic.metadata)
translation = topic.structured_translations.get(locale)
if translation:
localized.update(translation)
return localized
def documentation_structured_translation_issues(
topic: DocumentationTopic,
) -> tuple[str, ...]:
"""Validate the opt-in, versioned structured-documentation translation."""
version = topic.structured_translation_version
translations = topic.structured_translations
if version is None:
if translations:
return (
"structured_translations require structured_translation_version",
)
return ()
if version != DOCUMENTATION_STRUCTURED_TRANSLATION_VERSION:
return (
"unsupported structured_translation_version "
f"{version!r}; expected {DOCUMENTATION_STRUCTURED_TRANSLATION_VERSION!r}",
)
localizable_keys = set(localizable_documentation_metadata_keys(topic))
issues: list[str] = []
for locale, translation in translations.items():
if not locale.strip():
issues.append("structured translation locale must not be empty")
continue
translated_keys = set(translation)
for key in sorted(translated_keys - localizable_keys):
issues.append(
f"structured translation {locale!r} contains non-localizable or missing metadata key {key!r}"
)
for key in sorted(localizable_keys - translated_keys):
issues.append(
f"structured translation {locale!r} is missing metadata key {key!r}"
)
for key in sorted(localizable_keys & translated_keys):
issues.extend(
_structured_translation_shape_issues(
topic.metadata[key],
translation[key],
path=f"{locale}.{key}",
)
)
return tuple(issues)
def _structured_translation_shape_issues(
source: object,
translated: object,
*,
path: str,
) -> tuple[str, ...]:
if isinstance(source, str):
if not isinstance(translated, str) or not translated.strip():
return (f"structured translation {path} must be a non-empty string",)
return ()
if isinstance(source, Mapping):
if not isinstance(translated, Mapping):
return (f"structured translation {path} must preserve object shape",)
issues: list[str] = []
source_keys = {str(key) for key in source}
translated_keys = {str(key) for key in translated}
if source_keys != translated_keys:
issues.append(
f"structured translation {path} must preserve object keys"
)
return tuple(issues)
for key, value in source.items():
issues.extend(
_structured_translation_shape_issues(
value,
translated[key],
path=f"{path}.{key}",
)
)
return tuple(issues)
if isinstance(source, Sequence) and not isinstance(
source, (str, bytes, bytearray)
):
if not isinstance(translated, Sequence) or isinstance(
translated, (str, bytes, bytearray)
):
return (f"structured translation {path} must preserve list shape",)
if len(source) != len(translated):
return (f"structured translation {path} must preserve list length",)
issues: list[str] = []
for index, (source_item, translated_item) in enumerate(
zip(source, translated, strict=True)
):
issues.extend(
_structured_translation_shape_issues(
source_item,
translated_item,
path=f"{path}[{index}]",
)
)
return tuple(issues)
if translated != source:
return (
f"structured translation {path} must preserve non-text value {source!r}",
)
return ()
def user_workflow_scope_condition_issues(topic: DocumentationTopic) -> tuple[str, ...]:
"""Return fail-closed authoring issues for a user-facing workflow topic.
@@ -288,16 +563,23 @@ def user_workflow_scope_condition_issues(topic: DocumentationTopic) -> tuple[str
"""
raw_kind = topic.metadata.get("kind")
kind = raw_kind.strip().lower().replace("_", "-") if isinstance(raw_kind, str) else ""
kind = (
raw_kind.strip().lower().replace("_", "-") if isinstance(raw_kind, str) else ""
)
if kind != "workflow" or "user" not in topic.documentation_types:
return ()
if not topic.conditions:
return ("user workflow topics must declare at least one scope-conditioned alternative",)
return (
"user workflow topics must declare at least one scope-conditioned alternative",
)
unscoped_alternatives = tuple(
index
for index, condition in enumerate(topic.conditions, start=1)
if not any(scope.strip() for scope in (*condition.required_scopes, *condition.any_scopes))
if not any(
scope.strip()
for scope in (*condition.required_scopes, *condition.any_scopes)
)
)
if not unscoped_alternatives:
return ()
@@ -369,14 +651,11 @@ class CapabilityDocumentation:
class ResourceAclProvider(Protocol):
resource_type: str
def can_read(self, principal: object, resource_id: str) -> bool:
...
def can_read(self, principal: object, resource_id: str) -> bool: ...
def can_write(self, principal: object, resource_id: str) -> bool:
...
def can_write(self, principal: object, resource_id: str) -> bool: ...
def explain(self, principal: object, resource_id: str) -> AccessDecision:
...
def explain(self, principal: object, resource_id: str) -> AccessDecision: ...
TenantSummaryProvider = Callable[[object, str], Mapping[str, int]]
@@ -435,17 +714,29 @@ class ModuleManifest:
ownership_providers: tuple[OwnershipProviderRegistration, ...] = ()
tenant_summary_providers: tuple[TenantSummaryProvider, ...] = ()
tenant_summary_batch_providers: tuple[TenantSummaryBatchProvider, ...] = ()
delete_veto_providers: Mapping[str, Sequence[DeleteVetoProvider]] = field(default_factory=dict)
delete_veto_providers: Mapping[str, Sequence[DeleteVetoProvider]] = field(
default_factory=dict
)
uninstall_guard_providers: tuple[UninstallGuardProvider, ...] = ()
capability_factories: Mapping[str, CapabilityFactory] = field(default_factory=dict)
capability_documentation: Mapping[str, CapabilityDocumentation] = field(default_factory=dict)
capability_documentation: Mapping[str, CapabilityDocumentation] = field(
default_factory=dict
)
search_providers: tuple["SearchProviderRegistration", ...] = ()
search_sources: tuple["SearchSourceProviderRegistration", ...] = ()
work_item_providers: tuple["WorkItemProviderRegistration", ...] = ()
operational_check_providers: tuple[
"OperationalCheckProviderRegistration",
...,
] = ()
runtime_work_status_providers: tuple[
"RuntimeWorkStatusProviderRegistration",
...,
] = ()
architecture: ModuleArchitectureDeclaration | None = None
information_governance: ModuleInformationGovernance = field(
default_factory=ModuleInformationGovernance
)
external_providers: tuple[ExternalProviderDeclaration, ...] = ()
external_provider_state_providers: tuple[
ExternalProviderStateProviderRegistration,
@@ -466,3 +757,53 @@ class ModuleManifest:
# runtime module ID changes.
permission_namespace: str | None = None
workflow_definitions: tuple["WorkflowDefinitionContribution", ...] = ()
def with_documentation_structured_translations(
manifest: ModuleManifest,
*,
locale: str,
translations: Mapping[str, Mapping[str, Any]],
) -> ModuleManifest:
"""Merge module-owned structured documentation translations by topic id.
The helper keeps feature prose in its owning module while giving every
manifest the same fail-closed merge behavior. Unknown topic ids and
incomplete or shape-changing locale maps are rejected immediately.
"""
locale = locale.strip()
if not locale:
raise ValueError("structured documentation locale must not be empty")
topics_by_id = {topic.id: topic for topic in manifest.documentation}
unknown_topic_ids = sorted(set(translations) - set(topics_by_id))
if unknown_topic_ids:
raise ValueError(
"structured documentation translations reference unknown topic ids: "
+ ", ".join(unknown_topic_ids)
)
localized_topics: list[DocumentationTopic] = []
for topic in manifest.documentation:
translation = translations.get(topic.id)
if translation is None:
localized_topics.append(topic)
continue
structured_translations = dict(topic.structured_translations)
structured_translations[locale] = translation
localized_topic = replace(
topic,
structured_translation_version=DOCUMENTATION_STRUCTURED_TRANSLATION_VERSION,
structured_translations=structured_translations,
)
issues = documentation_structured_translation_issues(localized_topic)
if issues:
raise ValueError(
f"invalid {locale!r} structured documentation translation for "
f"{topic.id!r}: {'; '.join(issues)}"
)
localized_topics.append(localized_topic)
return replace(manifest, documentation=tuple(localized_topics))
+229
View File
@@ -0,0 +1,229 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Iterable, Mapping
NAVIGATION_PREFERENCES_KEY = "navigation_preferences"
NAVIGATION_PREFERENCES_CONTRACT_VERSION = "1"
_MAX_ITEMS = 256
@dataclass(frozen=True, slots=True)
class NavigationSeparator:
id: str
label: str = ""
def as_dict(self) -> dict[str, str]:
return {"id": self.id, "label": self.label}
@dataclass(frozen=True, slots=True)
class NavigationPreferences:
order: tuple[str, ...] = ()
hidden: tuple[str, ...] = ()
locked: tuple[str, ...] = ()
# None preserves inherited grouping; an empty tuple explicitly removes it.
separators: tuple[NavigationSeparator, ...] | None = None
def as_dict(self) -> dict[str, object]:
return {
"contract_version": NAVIGATION_PREFERENCES_CONTRACT_VERSION,
"order": list(self.order),
"hidden": list(self.hidden),
"locked": list(self.locked),
**({"separators": [item.as_dict() for item in self.separators]} if self.separators is not None else {}),
}
@dataclass(frozen=True, slots=True)
class EffectiveNavigationItem:
id: str
order: int
visible: bool
locked: bool
order_source: str
visibility_source: str
lock_source: str | None = None
section: NavigationSeparator | None = None
custom_layout: bool = False
layout_source: str = "module"
def as_dict(self) -> dict[str, object]:
return {
"navigation_id": self.id,
"order": self.order,
"navigation_visible": self.visible,
"navigation_locked": self.locked,
"navigation_order_source": self.order_source,
"navigation_visibility_source": self.visibility_source,
"navigation_lock_source": self.lock_source,
"navigation_section": self.section.as_dict() if self.section else None,
"navigation_custom_layout": self.custom_layout,
"navigation_layout_source": self.layout_source,
}
def navigation_preferences_from_settings(
settings: object,
) -> NavigationPreferences | None:
if not isinstance(settings, Mapping):
return None
raw = settings.get(NAVIGATION_PREFERENCES_KEY)
if not isinstance(raw, Mapping):
return None
return navigation_preferences_from_mapping(raw)
def navigation_preferences_from_mapping(
raw: Mapping[str, Any],
) -> NavigationPreferences:
return NavigationPreferences(
order=_ids(raw.get("order")),
hidden=_ids(raw.get("hidden")),
locked=_ids(raw.get("locked")),
separators=_separators(raw.get("separators")),
)
def update_navigation_preferences(
settings: object,
preferences: NavigationPreferences | Mapping[str, Any] | None,
) -> dict[str, Any]:
updated = dict(settings) if isinstance(settings, Mapping) else {}
if preferences is None:
updated.pop(NAVIGATION_PREFERENCES_KEY, None)
else:
raw = preferences.as_dict() if isinstance(preferences, NavigationPreferences) else preferences
updated[NAVIGATION_PREFERENCES_KEY] = navigation_preferences_from_mapping(
raw
).as_dict()
return updated
def resolve_navigation_preferences(
item_ids: Iterable[str],
*,
system: NavigationPreferences | None = None,
tenant: NavigationPreferences | None = None,
user: NavigationPreferences | None = None,
) -> dict[str, EffectiveNavigationItem]:
ordered = list(dict.fromkeys(_clean_id(item_id) for item_id in item_ids))
ordered = [item_id for item_id in ordered if item_id]
available = set(ordered)
order_source = {item_id: "module" for item_id in ordered}
visibility = {item_id: True for item_id in ordered}
visibility_source = {item_id: "module" for item_id in ordered}
locks: dict[str, str] = {}
separators: dict[str, NavigationSeparator] = {}
custom_layout = False
layout_source = "module"
for source, preferences, may_lock in (
("system", system, True),
("tenant", tenant, True),
("user", user, False),
):
if preferences is None:
continue
if preferences.separators is not None:
separators = {item.id: item for item in preferences.separators if item.id not in available}
ordered = [item_id for item_id in ordered if item_id in available or item_id in separators]
ordered.extend(item_id for item_id in separators if item_id not in ordered)
custom_layout = True
layout_source = source
requested_order = list(dict.fromkeys(item_id for item_id in preferences.order if item_id in available or item_id in separators))
if requested_order:
requested = set(requested_order)
ordered = [*requested_order, *(item_id for item_id in ordered if item_id not in requested)]
for item_id in requested_order:
order_source[item_id] = source
requested_hidden = set(preferences.hidden).intersection(available)
for item_id in available:
if item_id in locks:
visibility[item_id] = True
visibility_source[item_id] = locks[item_id]
continue
visibility[item_id] = item_id not in requested_hidden
visibility_source[item_id] = source
if may_lock:
for item_id in preferences.locked:
if item_id not in available:
continue
locks[item_id] = source
visibility[item_id] = True
visibility_source[item_id] = source
result: dict[str, EffectiveNavigationItem] = {}
section: NavigationSeparator | None = None
for index, item_id in enumerate(ordered):
if item_id in separators:
section = separators[item_id]
continue
result[item_id] = EffectiveNavigationItem(
id=item_id,
order=index,
visible=visibility[item_id],
locked=item_id in locks,
order_source=order_source[item_id],
visibility_source=visibility_source[item_id],
lock_source=locks.get(item_id),
section=section,
custom_layout=custom_layout,
layout_source=layout_source,
)
return result
def _separators(value: object) -> tuple[NavigationSeparator, ...] | None:
if not isinstance(value, (list, tuple)):
return None
items: dict[str, NavigationSeparator] = {}
for raw in value[:_MAX_ITEMS]:
if not isinstance(raw, Mapping):
continue
item_id = _clean_id(raw.get("id"))
label = raw.get("label", "")
if not item_id.startswith("separator:") or not isinstance(label, str):
continue
label = label.strip()[:120]
if any(ord(character) < 32 for character in label):
continue
items[item_id] = NavigationSeparator(item_id, label)
return tuple(items.values())
def _ids(value: object) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)):
return ()
cleaned = tuple(
dict.fromkeys(
item_id
for item in value[:_MAX_ITEMS]
if (item_id := _clean_id(item))
)
)
return cleaned
def _clean_id(value: object) -> str:
if not isinstance(value, str):
return ""
clean = value.strip()
if not clean or len(clean) > 255 or any(ord(character) < 32 for character in clean):
return ""
return clean
__all__ = [
"EffectiveNavigationItem",
"NAVIGATION_PREFERENCES_CONTRACT_VERSION",
"NAVIGATION_PREFERENCES_KEY",
"NavigationPreferences",
"NavigationSeparator",
"navigation_preferences_from_mapping",
"navigation_preferences_from_settings",
"resolve_navigation_preferences",
"update_navigation_preferences",
]
+83 -1
View File
@@ -1,7 +1,8 @@
from __future__ import annotations
from collections.abc import Callable, Mapping
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal
@@ -42,3 +43,84 @@ class OperationalCheckProviderRegistration:
provider: OperationalCheckProvider
cache_seconds: int = 60
RuntimeWorkState = Literal[
"disabled",
"unconfigured",
"starting",
"healthy",
"idle",
"busy",
"degraded",
"stale",
"unreachable",
]
@dataclass(frozen=True, slots=True)
class RuntimeWorkStatusContext:
"""Sanitized process evidence supplied to a runtime-work provider."""
profile: str
observed_at: datetime
stale_after_seconds: int
runtime_nodes: Sequence[Mapping[str, object]] = ()
@dataclass(frozen=True, slots=True)
class RuntimeWorkStatus:
"""Bounded worker/queue status with explicit unsupported metrics."""
provider_id: str
label: str
backend: str
enabled: bool
configured: bool
state: RuntimeWorkState
detail: str
observed_at: datetime
active_workers: int | None = None
last_heartbeat_at: datetime | None = None
queue_depths: Mapping[str, int | None] = field(default_factory=dict)
active_work: int | None = None
reserved_work: int | None = None
failures: int | None = None
stale_after_seconds: int | None = None
guidance: str = ""
def as_dict(self) -> dict[str, object]:
return {
"provider_id": self.provider_id,
"label": self.label,
"backend": self.backend,
"enabled": self.enabled,
"configured": self.configured,
"state": self.state,
"detail": self.detail,
"observed_at": self.observed_at.isoformat(),
"active_workers": self.active_workers,
"last_heartbeat_at": (
self.last_heartbeat_at.isoformat()
if self.last_heartbeat_at is not None
else None
),
"queue_depths": dict(self.queue_depths),
"active_work": self.active_work,
"reserved_work": self.reserved_work,
"failures": self.failures,
"stale_after_seconds": self.stale_after_seconds,
"guidance": self.guidance,
}
RuntimeWorkStatusProvider = Callable[[RuntimeWorkStatusContext], RuntimeWorkStatus]
@dataclass(frozen=True, slots=True)
class RuntimeWorkStatusProviderRegistration:
"""Register one optional provider-neutral worker/queue observation."""
module_id: str
provider_id: str
provider: RuntimeWorkStatusProvider
cache_seconds: int = 15
+87
View File
@@ -0,0 +1,87 @@
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime
from typing import Protocol, runtime_checkable
from govoplan_core.core.institutional import EvidenceReference
CAPABILITY_PAYMENT_REQUESTS = "payments.requests"
@dataclass(frozen=True, slots=True)
class PaymentRequestCommand:
tenant_id: str
source_module: str
source_resource_type: str
source_resource_id: str
amount_minor: int
currency: str
subject: str
idempotency_key: str
requested_at: datetime
requested_by_ref: str
due_at: datetime | None = None
context_refs: Mapping[str, str] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class ManualPaymentReconciliationCommand:
tenant_id: str
payment_id: str
amount_minor: int
currency: str
transaction_reference: str
evidence_ref: EvidenceReference
idempotency_key: str
received_at: datetime
recorded_at: datetime
recorded_by_ref: str
metadata: Mapping[str, object] = field(default_factory=dict)
@runtime_checkable
class PaymentRequestProvider(Protocol):
def request_payment(
self,
session: object,
command: PaymentRequestCommand,
) -> Mapping[str, object]:
...
def get_payment(
self,
session: object,
*,
tenant_id: str,
payment_id: str,
) -> Mapping[str, object] | None:
...
def reconcile_manual_payment(
self,
session: object,
command: ManualPaymentReconciliationCommand,
) -> Mapping[str, object]:
...
def payment_request_provider(registry: object | None) -> PaymentRequestProvider | None:
if registry is None or not hasattr(registry, "has_capability"):
return None
if not registry.has_capability(CAPABILITY_PAYMENT_REQUESTS):
return None
capability = registry.capability(CAPABILITY_PAYMENT_REQUESTS)
return capability if isinstance(capability, PaymentRequestProvider) else None
__all__ = [
"CAPABILITY_PAYMENT_REQUESTS",
"ManualPaymentReconciliationCommand",
"PaymentRequestCommand",
"PaymentRequestProvider",
"payment_request_provider",
]
@@ -21,11 +21,14 @@ PlatformInterfaceKind = Literal[
"frontend_route",
"navigation",
"permission",
"product_area",
"product_surface",
"provided_interface",
"public_route",
"search_provider",
"search_source",
"settings_route",
"quick_access_tool",
"view_surface",
]
@@ -217,6 +220,80 @@ def manifest_interface_declarations(
},
)
)
for area in frontend.product_areas:
declarations.append(
PlatformInterfaceDeclaration(
id=f"{manifest.id}.{area.id}",
module_id=manifest.id,
kind="product_area",
label=area.label,
required_all=(),
required_any=(),
metadata={
"area_id": area.id,
"icon": area.icon,
"description": area.description,
"order": area.order,
"surface_ids": list(area.surface_ids),
},
)
)
for surface in frontend.product_surfaces:
declarations.append(
PlatformInterfaceDeclaration(
id=f"{manifest.id}.{surface.id}",
module_id=manifest.id,
kind="product_surface",
label=surface.label,
path=surface.route_path,
required_all=surface.required_all,
required_any=surface.required_any,
metadata={
"contract_version": surface.contract_version,
"product_surface_id": surface.id,
"description": surface.description,
"icon": surface.icon,
"entry_path": surface.entry_path,
"surface_ids": list(surface.surface_ids),
"presentations": list(surface.presentations),
"capability_ids": list(surface.capability_ids),
"search_source_ids": list(surface.search_source_ids),
"help_context_ids": list(surface.help_context_ids),
"documentation_topic_ids": list(
surface.documentation_topic_ids
),
"aliases": list(surface.aliases),
"order": surface.order,
"unavailable_reason": surface.unavailable.reason,
"degraded_reason": (
surface.degraded.reason
if surface.degraded is not None
else None
),
},
)
)
for tool in frontend.quick_access_tools:
declarations.append(
PlatformInterfaceDeclaration(
id=tool.id,
module_id=manifest.id,
kind="quick_access_tool",
label=tool.label,
path=tool.full_page_path,
required_all=tool.required_all,
required_any=tool.required_any,
metadata={
"category_id": tool.category_id,
"surface_id": tool.surface_id,
"icon": tool.icon,
"description": tool.description,
"order": tool.order,
"default_enabled": tool.default_enabled,
"modes": list(tool.modes),
},
)
)
frontend_navigation = {
declaration.id: declaration
+255
View File
@@ -7,6 +7,20 @@ from urllib.parse import quote, unquote
from govoplan_core.core.access import PrincipalRef
PolicyScopeType = Literal["system", "tenant", "user", "group", "campaign"]
PolicyImpactPopulationState = Literal[
"complete",
"sampled",
"truncated",
"unavailable",
]
CampaignArchiveEncryptionMethod = Literal["aes", "zip_standard"]
CampaignArchivePasswordDeliveryChannel = Literal[
"separate_mail",
"sms",
"letter",
"phone",
"in_person",
]
SchedulingParticipantVisibility = Literal["aggregates_only", "names_and_statuses"]
DefinitionScopeType = Literal["system", "tenant", "group", "user"]
DefinitionKind = Literal["flow", "template"]
@@ -27,10 +41,12 @@ ViewGovernanceAction = Literal[
"workflow_activate",
]
FunctionAssignmentChangeKind = Literal["request", "grant"]
FunctionAssignmentReviewStep = Literal["holder", "authority", "recipient"]
FunctionAssignmentGovernanceAction = Literal[
"submit",
"approve_holder",
"approve_authority",
"approve_escalation",
"accept_recipient",
"request_changes",
"respond",
@@ -45,6 +61,8 @@ CAPABILITY_POLICY_SCHEDULING_PARTICIPANT_PRIVACY = "policy.schedulingParticipant
CAPABILITY_POLICY_DEFINITION_GOVERNANCE = "policy.definitionGovernance"
CAPABILITY_POLICY_VIEW_GOVERNANCE = "policy.viewGovernance"
CAPABILITY_POLICY_FUNCTION_ASSIGNMENT_GOVERNANCE = "policy.functionAssignmentGovernance"
CAPABILITY_POLICY_CAMPAIGN_ARCHIVE_ENCRYPTION = "policy.campaignArchiveEncryption"
CAPABILITY_POLICY_IMPACT_SUBJECT_PREFIX = "policy.impactSubjects."
POLICY_SCOPE_TYPES: tuple[PolicyScopeType, ...] = (
"system",
@@ -182,6 +200,213 @@ class PolicyDecision:
}
@dataclass(frozen=True, slots=True)
class PolicyImpactPopulationRequest:
"""One explicit, bounded request to an optional impact-subject provider."""
tenant_id: str
policy_family: str
selector: Mapping[str, Any] = field(default_factory=dict)
limit: int = 200
actor_scopes: tuple[str, ...] = ()
allow_sensitive_details: bool = False
def __post_init__(self) -> None:
if not self.tenant_id.strip():
raise ValueError("Policy impact population requires a tenant ID")
if not self.policy_family.strip() or len(self.policy_family) > 120:
raise ValueError(
"Policy impact population family must contain 1 to 120 characters"
)
if self.limit < 1 or self.limit > 500:
raise ValueError("Policy impact population limit must be between 1 and 500")
@dataclass(frozen=True, slots=True)
class PolicyImpactSubject:
"""Provider-owned reference safe for Policy to compare without domain imports."""
module_id: str
resource_type: str
resource_id: str
action: str
label: str | None = None
scope_type: PolicyScopeType | None = None
scope_id: str | None = None
attributes: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
for label, value, maximum in (
("module ID", self.module_id, 80),
("resource type", self.resource_type, 80),
("resource ID", self.resource_id, 240),
("action", self.action, 120),
):
if not value.strip() or len(value) > maximum:
raise ValueError(
f"Policy impact subject {label} must contain 1 to {maximum} characters"
)
@property
def key(self) -> tuple[str, str, str, str]:
return (
self.module_id,
self.resource_type,
self.resource_id,
self.action,
)
def to_dict(self) -> dict[str, Any]:
return {
"module_id": self.module_id,
"resource_type": self.resource_type,
"resource_id": self.resource_id,
"action": self.action,
"label": self.label,
"scope_type": self.scope_type,
"scope_id": self.scope_id,
"attributes": dict(self.attributes),
}
@dataclass(frozen=True, slots=True)
class PolicyImpactSubjectBatch:
provider_id: str
subjects: tuple[PolicyImpactSubject, ...] = ()
state: PolicyImpactPopulationState = "complete"
total_available: int | None = None
explanation: str | None = None
def __post_init__(self) -> None:
if not self.provider_id.strip() or len(self.provider_id) > 120:
raise ValueError(
"Policy impact provider ID must contain 1 to 120 characters"
)
if len(self.subjects) > 500:
raise ValueError("Policy impact providers may return at most 500 subjects")
if len({subject.key for subject in self.subjects}) != len(self.subjects):
raise ValueError("Policy impact provider returned duplicate subjects")
if self.total_available is not None and self.total_available < len(self.subjects):
raise ValueError(
"Policy impact population total cannot be smaller than its subjects"
)
if self.state == "unavailable" and not self.explanation:
raise ValueError("Unavailable policy impact populations need an explanation")
def to_dict(self, *, include_subjects: bool = True) -> dict[str, Any]:
return {
"provider_id": self.provider_id,
"state": self.state,
"returned": len(self.subjects),
"total_available": self.total_available,
"explanation": self.explanation,
"subjects": (
[subject.to_dict() for subject in self.subjects]
if include_subjects
else []
),
}
@runtime_checkable
class PolicyImpactSubjectProvider(Protocol):
provider_id: str
supported_policy_families: tuple[str, ...]
def collect_policy_impact_subjects(
self,
session: object | None = None,
*,
request: PolicyImpactPopulationRequest,
) -> PolicyImpactSubjectBatch: ...
def policy_impact_subject_provider(
registry: object | None,
provider_id: str,
) -> PolicyImpactSubjectProvider | None:
clean_provider_id = provider_id.strip()
if not clean_provider_id or registry is None:
return None
capability_name = f"{CAPABILITY_POLICY_IMPACT_SUBJECT_PREFIX}{clean_provider_id}"
if (
not hasattr(registry, "has_capability")
or not registry.has_capability(capability_name)
):
return None
capability = registry.capability(capability_name)
if not isinstance(capability, PolicyImpactSubjectProvider):
return None
return capability
@dataclass(frozen=True, slots=True)
class CampaignArchiveEncryptionRequest:
"""Context required to resolve one Campaign archive-encryption ceiling.
The owning module supplies the stable Campaign and owner references. Policy
owns hierarchy evaluation; Campaign owns archive configuration and evidence.
"""
tenant_id: str
campaign_id: str
owner_type: Literal["user", "group"] | None = None
owner_id: str | None = None
@dataclass(frozen=True, slots=True)
class CampaignArchiveEncryptionDecision:
allowed_password_encryption_methods: frozenset[CampaignArchiveEncryptionMethod]
allowed_password_delivery_channels: frozenset[
CampaignArchivePasswordDeliveryChannel
]
policy_hash: str
source_path: tuple[PolicySourceStep, ...] = ()
reason: str | None = None
diagnostics: tuple[Mapping[str, Any], ...] = ()
def to_dict(self) -> dict[str, Any]:
return {
"allowed_password_encryption_methods": sorted(
self.allowed_password_encryption_methods
),
"allowed_password_delivery_channels": sorted(
self.allowed_password_delivery_channels
),
"policy_hash": self.policy_hash,
"source_path": [step.to_dict() for step in self.source_path],
"reason": self.reason,
"diagnostics": [dict(item) for item in self.diagnostics],
}
@runtime_checkable
class CampaignArchiveEncryptionPolicy(Protocol):
def resolve_campaign_archive_encryption(
self,
session: object | None = None,
*,
request: CampaignArchiveEncryptionRequest,
) -> CampaignArchiveEncryptionDecision: ...
def campaign_archive_encryption_policy(
registry: object | None,
) -> CampaignArchiveEncryptionPolicy | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_POLICY_CAMPAIGN_ARCHIVE_ENCRYPTION)
):
return None
capability = registry.capability(CAPABILITY_POLICY_CAMPAIGN_ARCHIVE_ENCRYPTION)
return (
capability
if isinstance(capability, CampaignArchiveEncryptionPolicy)
else None
)
@dataclass(frozen=True, slots=True)
class FunctionAssignmentGovernanceRequest:
tenant_id: str
@@ -196,6 +421,20 @@ class FunctionAssignmentGovernanceRequest:
context: Mapping[str, Any] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class FunctionAssignmentEscalationRule:
step: FunctionAssignmentReviewStep
target_function_id: str
timeout_hours: int
def to_dict(self) -> dict[str, Any]:
return {
"step": self.step,
"target_function_id": self.target_function_id,
"timeout_hours": self.timeout_hours,
}
@dataclass(frozen=True, slots=True)
class FunctionAssignmentGovernanceDecision:
allowed: bool
@@ -208,6 +447,10 @@ class FunctionAssignmentGovernanceDecision:
separation_of_duties: bool = True
quorum: int = 1
maximum_validity_days: int | None = None
delegation_allowed: bool = False
maximum_delegation_depth: int = 0
maximum_delegated_validity_days: int | None = None
escalation_rules: tuple[FunctionAssignmentEscalationRule, ...] = ()
request_expiry_hours: int = 336
source_path: tuple[PolicySourceStep, ...] = ()
requirements: tuple[str, ...] = ()
@@ -225,12 +468,24 @@ class FunctionAssignmentGovernanceDecision:
"separation_of_duties": self.separation_of_duties,
"quorum": self.quorum,
"maximum_validity_days": self.maximum_validity_days,
"delegation_allowed": self.delegation_allowed,
"maximum_delegation_depth": self.maximum_delegation_depth,
"maximum_delegated_validity_days": (
self.maximum_delegated_validity_days
),
"escalation_rules": [rule.to_dict() for rule in self.escalation_rules],
"request_expiry_hours": self.request_expiry_hours,
"source_path": [step.to_dict() for step in self.source_path],
"requirements": list(self.requirements),
"details": dict(self.details),
}
def escalation_rule(
self,
step: FunctionAssignmentReviewStep,
) -> FunctionAssignmentEscalationRule | None:
return next((rule for rule in self.escalation_rules if rule.step == step), None)
@runtime_checkable
class FunctionAssignmentGovernancePolicy(Protocol):
+58
View File
@@ -15,6 +15,7 @@ CAPABILITY_POSTBOX_MESSAGES = "postbox.messages"
CAPABILITY_POSTBOX_DELIVERY = "postbox.delivery"
CAPABILITY_POSTBOX_EVIDENCE = "postbox.evidence"
CAPABILITY_POSTBOX_ROUTING = "postbox.routing"
CAPABILITY_POSTBOX_PORTAL = "postbox.portal_projection"
PostboxAction = Literal[
"discover",
@@ -171,6 +172,11 @@ class PostboxDirectoryEntryRef:
template_revision_id: str | None = None
holder_count: int = 0
vacant: bool = True
encryption_profile: str = "plaintext_v1"
key_epoch: int = 1
encryption_vault_id: str | None = None
protection_policy: Mapping[str, object] = field(default_factory=dict)
grouping_policy: Mapping[str, object] = field(default_factory=dict)
access: PostboxAccessDecisionRef | None = None
resource_revision: int = 1
etag: str | None = None
@@ -259,6 +265,9 @@ class PostboxMessageAuthoringRequest:
idempotency_key: str
subject: str
body_text: str | None = None
ciphertext_ref: str | None = None
signed_manifest_ref: str | None = None
wrapped_keys: tuple[PostboxWrappedKeyRef, ...] = ()
classification: str = "internal"
participants: tuple[PostboxParticipantRef, ...] = ()
attachments: tuple[PostboxAttachmentRef, ...] = ()
@@ -300,6 +309,7 @@ class PostboxDeliveryRequest:
body_text: str | None = None
sender_label: str | None = None
classification: str = "internal"
action_required: bool = False
participants: tuple[PostboxParticipantRef, ...] = ()
attachments: tuple[PostboxAttachmentRef, ...] = ()
expires_at: datetime | None = None
@@ -323,6 +333,14 @@ class PostboxDeliveryResult:
evidence: Mapping[str, object] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class PostboxPortalEntryRef:
postbox: PostboxDirectoryEntryRef
unread_count: int = 0
latest_message_at: datetime | None = None
route_path: str = "/postbox"
class PostboxDeliveryRejected(RuntimeError):
"""A delivery was rejected before the provider accepted any effect."""
@@ -494,6 +512,31 @@ class PostboxRoutingProvider(Protocol):
) -> Mapping[str, object]:
...
def reconcile_notification_lifecycle(
self,
session: object,
*,
tenant_id: str | None = None,
limit: int = 50,
) -> Mapping[str, object]:
"""Reconcile assignment-derived Postbox notification facts."""
...
@runtime_checkable
class PostboxPortalProjectionProvider(Protocol):
"""Project portal-enabled Postboxes without transferring access ownership."""
def list_portal_entries(
self,
session: object,
principal: object,
*,
tenant_id: str,
limit: int = 100,
) -> Sequence[PostboxPortalEntryRef]: ...
def _postbox_provider(
registry: object | None,
@@ -573,3 +616,18 @@ def postbox_routing_provider(
provider_type=PostboxRoutingProvider,
)
return provider if isinstance(provider, PostboxRoutingProvider) else None
def postbox_portal_projection_provider(
registry: object | None,
) -> PostboxPortalProjectionProvider | None:
provider = _postbox_provider(
registry,
capability_name=CAPABILITY_POSTBOX_PORTAL,
provider_type=PostboxPortalProjectionProvider,
)
return (
provider
if isinstance(provider, PostboxPortalProjectionProvider)
else None
)
+37
View File
@@ -0,0 +1,37 @@
"""Pure principal attribution mechanics, not authorization or tenant resolution.
The two existing contracts intentionally differ in precedence and whitespace.
Callers retain their own service-account, scope and resource-access decisions.
"""
def principal_actor_ids(principal: object) -> tuple[str, ...]:
"""Account-first legacy IDs, unique in encounter order; retain nonblank text."""
user = getattr(principal, "user", None)
return tuple(
dict.fromkeys(
str(value)
for value in (
getattr(principal, "account_id", None),
getattr(principal, "identity_id", None),
getattr(principal, "membership_id", None),
getattr(user, "id", None),
)
if str(value or "").strip()
)
)
def principal_user_first_actor(principal: object) -> str | None:
"""First nonblank user/account/identity/membership ID, with trimmed text."""
user = getattr(principal, "user", None)
for value in (
getattr(user, "id", None),
getattr(principal, "account_id", None),
getattr(principal, "identity_id", None),
getattr(principal, "membership_id", None),
):
candidate = str(value or "").strip()
if candidate:
return candidate
return None
+369
View File
@@ -0,0 +1,369 @@
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal, Protocol, runtime_checkable
CAPABILITY_RECORDS_FILING = "records.filing"
CAPABILITY_RECORD_SOURCE_PREFIX = "records.source."
CAPABILITY_RECORD_ARCHIVE_PREFIX = "records.archive."
RecordSourceAuthority = Literal[
"native_authoritative",
"external_authoritative",
"external_mirror",
"governed_sync",
"governance_overlay",
"linked_reference",
]
RecordArchiveOutcome = Literal["accepted", "rejected", "outcome_unknown"]
_RECORD_SOURCE_AUTHORITIES = {
"native_authoritative",
"external_authoritative",
"external_mirror",
"governed_sync",
"governance_overlay",
"linked_reference",
}
_RECORD_ARCHIVE_OUTCOMES = {"accepted", "rejected", "outcome_unknown"}
class RecordContractError(ValueError):
"""Stable error for provider-neutral record filing operations."""
@dataclass(frozen=True, slots=True)
class RecordSourceLocator:
"""Exact source revision requested for filing into a record."""
tenant_id: str
source_module: str
resource_type: str
resource_id: str
source_revision: str
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_require_text_fields(
self,
"tenant_id",
"source_module",
"resource_type",
"resource_id",
"source_revision",
)
if len(self.resource_id) > 500 or len(self.source_revision) > 255:
raise RecordContractError("Record source identity is too long.")
@dataclass(frozen=True, slots=True)
class RecordSourceReference:
"""Provider-resolved immutable source metadata safe to preserve in Records."""
locator: RecordSourceLocator
label: str
authority_mode: RecordSourceAuthority = "linked_reference"
content_sha256: str | None = None
content_type: str | None = None
size_bytes: int | None = None
valid_from: datetime | None = None
valid_to: datetime | None = None
recorded_at: datetime | None = None
launch_url: str | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
if not self.label.strip():
raise RecordContractError("Record source references require a label.")
if len(self.label) > 500:
raise RecordContractError(
"Record source labels are limited to 500 characters."
)
if self.size_bytes is not None and self.size_bytes < 0:
raise RecordContractError("Record source sizes cannot be negative.")
if self.content_sha256 is not None:
digest = self.content_sha256.removeprefix("sha256:")
if len(digest) != 64 or any(
character not in "0123456789abcdefABCDEF" for character in digest
):
raise RecordContractError(
"Record source SHA-256 digests must be hexadecimal."
)
if self.valid_from and self.valid_to and self.valid_to <= self.valid_from:
raise RecordContractError(
"Record source valid_to must be after valid_from."
)
@dataclass(frozen=True, slots=True)
class RecordFilingRequest:
tenant_id: str
record_id: str
source: RecordSourceLocator
purpose: str
filing_reason: str
idempotency_key: str
volume_id: str | None = None
relationship: str = "contains"
institutional_context: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_require_text_fields(
self,
"tenant_id",
"record_id",
"purpose",
"filing_reason",
"idempotency_key",
"relationship",
)
if self.source.tenant_id != self.tenant_id:
raise RecordContractError("Record filing cannot cross tenants.")
if len(self.purpose) > 255 or len(self.filing_reason) > 2_000:
raise RecordContractError("Record filing purpose or reason is too long.")
if len(self.idempotency_key) > 255:
raise RecordContractError(
"Record filing idempotency keys are limited to 255 characters."
)
@dataclass(frozen=True, slots=True)
class RecordFilingResult:
record_id: str
item_id: str
sequence: int
source: RecordSourceReference
filed_at: datetime
replayed: bool = False
@dataclass(frozen=True, slots=True)
class RecordTransferPackage:
"""Exact, digest-bound package prepared by Records for one provider profile."""
tenant_id: str
package_id: str
record_id: str
record_revision: int
profile: str
manifest_sha256: str
manifest: Mapping[str, object]
def __post_init__(self) -> None:
_require_text_fields(
self,
"tenant_id",
"package_id",
"record_id",
"profile",
"manifest_sha256",
)
if self.record_revision < 1:
raise RecordContractError("Record transfer revisions must be positive.")
_require_sha256(self.manifest_sha256, "Record transfer manifest")
@dataclass(frozen=True, slots=True)
class RecordArchiveProviderState:
provider_id: str
label: str
profiles: tuple[str, ...]
authority_modes: tuple[RecordSourceAuthority, ...]
healthy: bool
checked_at: datetime
last_success_at: datetime | None = None
freshness_seconds: int | None = None
limitations: tuple[str, ...] = ()
simulated: bool = False
def __post_init__(self) -> None:
_require_text_fields(self, "provider_id", "label")
if not self.profiles or any(not item.strip() for item in self.profiles):
raise RecordContractError(
"Record archive providers require at least one profile."
)
if not self.authority_modes:
raise RecordContractError(
"Record archive providers require an authority mode."
)
if any(mode not in _RECORD_SOURCE_AUTHORITIES for mode in self.authority_modes):
raise RecordContractError(
"Record archive providers declared an invalid authority mode."
)
if self.freshness_seconds is not None and self.freshness_seconds < 0:
raise RecordContractError(
"Record archive provider freshness cannot be negative."
)
@dataclass(frozen=True, slots=True)
class RecordArchiveTransferRequest:
package: RecordTransferPackage
purpose: str
idempotency_key: str
institutional_context: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_require_text_fields(self, "purpose", "idempotency_key")
if len(self.purpose) > 255 or len(self.idempotency_key) > 255:
raise RecordContractError(
"Record archive purpose or idempotency key is too long."
)
@dataclass(frozen=True, slots=True)
class RecordArchiveReceipt:
provider_id: str
package_id: str
outcome: RecordArchiveOutcome
observed_at: datetime
receipt_sha256: str
external_reference: str | None = None
retry_safe: bool = False
simulated: bool = False
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_require_text_fields(
self,
"provider_id",
"package_id",
"outcome",
"receipt_sha256",
)
_require_sha256(self.receipt_sha256, "Record archive receipt")
if self.outcome not in _RECORD_ARCHIVE_OUTCOMES:
raise RecordContractError("Record archive receipt outcome is invalid.")
if self.outcome == "outcome_unknown" and self.retry_safe:
raise RecordContractError(
"Unknown archive outcomes cannot be declared retry-safe."
)
@runtime_checkable
class RecordSourceProvider(Protocol):
provider_id: str
def resource_types(self) -> Sequence[str]: ...
def resolve(
self,
session: object,
principal: object,
*,
locator: RecordSourceLocator,
purpose: str,
) -> RecordSourceReference:
"""Resolve one currently authorized, exact source revision."""
@runtime_checkable
class RecordFilingService(Protocol):
def file(
self,
session: object,
principal: object,
*,
request: RecordFilingRequest,
) -> RecordFilingResult: ...
@runtime_checkable
class RecordArchiveProvider(Protocol):
provider_id: str
def state(self) -> RecordArchiveProviderState: ...
def dispatch(
self,
session: object,
principal: object,
*,
request: RecordArchiveTransferRequest,
) -> RecordArchiveReceipt:
"""Dispatch one prepared package without retrying an unknown outcome."""
def record_source_capability(source_module: str) -> str:
normalized = _capability_suffix(source_module, "source module")
return f"{CAPABILITY_RECORD_SOURCE_PREFIX}{normalized}"
def record_archive_capability(provider_id: str) -> str:
normalized = _capability_suffix(provider_id, "archive provider")
return f"{CAPABILITY_RECORD_ARCHIVE_PREFIX}{normalized}"
def record_source_capabilities(registry: object | None) -> tuple[str, ...]:
if registry is None or not hasattr(registry, "capability_names"):
return ()
return tuple(
name
for name in registry.capability_names()
if str(name).startswith(CAPABILITY_RECORD_SOURCE_PREFIX)
)
def record_archive_capabilities(registry: object | None) -> tuple[str, ...]:
if registry is None or not hasattr(registry, "capability_names"):
return ()
return tuple(
name
for name in registry.capability_names()
if str(name).startswith(CAPABILITY_RECORD_ARCHIVE_PREFIX)
)
def _require_text_fields(value: object, *field_names: str) -> None:
for field_name in field_names:
if not str(getattr(value, field_name, "") or "").strip():
raise RecordContractError(
f"Record contract field {field_name} is required."
)
def _capability_suffix(value: str, label: str) -> str:
normalized = value.strip().lower()
if not normalized or any(
character not in "abcdefghijklmnopqrstuvwxyz0123456789_-"
for character in normalized
):
raise RecordContractError(f"Record {label} identifiers are invalid.")
return normalized
def _require_sha256(value: str, label: str) -> None:
digest = value.removeprefix("sha256:")
if len(digest) != 64 or any(
character not in "0123456789abcdefABCDEF" for character in digest
):
raise RecordContractError(f"{label} SHA-256 must be hexadecimal.")
__all__ = [
"CAPABILITY_RECORD_ARCHIVE_PREFIX",
"CAPABILITY_RECORDS_FILING",
"CAPABILITY_RECORD_SOURCE_PREFIX",
"RecordArchiveOutcome",
"RecordArchiveProvider",
"RecordArchiveProviderState",
"RecordArchiveReceipt",
"RecordArchiveTransferRequest",
"RecordContractError",
"RecordFilingRequest",
"RecordFilingResult",
"RecordFilingService",
"RecordSourceAuthority",
"RecordSourceLocator",
"RecordSourceProvider",
"RecordSourceReference",
"RecordTransferPackage",
"record_archive_capabilities",
"record_archive_capability",
"record_source_capabilities",
"record_source_capability",
]
File diff suppressed because it is too large Load Diff
+258
View File
@@ -0,0 +1,258 @@
from __future__ import annotations
from collections.abc import Callable, Mapping
from datetime import datetime
from typing import Any
from govoplan_core.core.operations import (
RuntimeWorkState,
RuntimeWorkStatus,
RuntimeWorkStatusContext,
)
from govoplan_core.settings import settings
QueueDepthReader = Callable[[list[str]], Mapping[str, int | None]]
def celery_runtime_work_status(context: RuntimeWorkStatusContext) -> RuntimeWorkStatus:
"""Observe the built-in Celery backend without exposing it to Ops."""
queues = _configured_queues()
backend_configured = bool(str(settings.redis_url or "").strip())
if not settings.celery_enabled:
return _status(
context,
enabled=False,
configured=backend_configured,
state="disabled",
detail="Background workers are intentionally disabled.",
active_workers=0,
queue_depths={queue: None for queue in queues},
active_work=0,
reserved_work=0,
guidance=(
"Synchronous development paths may be used in development. "
"Enable and monitor workers before production queue-backed work."
if context.profile in {"development", "local-dev"}
else "Enable a worker backend before accepting queue-backed work."
),
)
if not backend_configured or not queues:
return _status(
context,
enabled=True,
configured=False,
state="unconfigured",
detail="Background workers are enabled but their backend or queue list is not configured.",
queue_depths={queue: None for queue in queues},
guidance="Configure the broker and an explicit queue list, then start the required worker pools.",
)
try:
from govoplan_core.celery_app import celery
inspector = celery.control.inspect(timeout=0.75)
return collect_celery_runtime_work_status(
context,
inspector=inspector,
queues=queues,
queue_depth_reader=_redis_queue_depths,
)
except Exception: # noqa: BLE001 - status must isolate and sanitize provider failures.
return _status(
context,
enabled=True,
configured=True,
state="unreachable",
detail="The configured worker backend did not return bounded status evidence.",
queue_depths={queue: None for queue in queues},
guidance="Verify broker reachability and worker processes; do not infer health from missing metrics.",
)
def collect_celery_runtime_work_status(
context: RuntimeWorkStatusContext,
*,
inspector: Any,
queues: list[str],
queue_depth_reader: QueueDepthReader,
) -> RuntimeWorkStatus:
replies = inspector.ping() or {}
if not isinstance(replies, Mapping) or not replies:
state = "starting" if _fresh_worker_nodes(context) else "unreachable"
return _status(
context,
enabled=True,
configured=True,
state=state,
detail=(
"Worker processes are starting but have not answered the bounded status probe."
if state == "starting"
else "No configured worker answered the bounded status probe."
),
active_workers=0,
queue_depths={queue: None for queue in queues},
guidance="Wait for startup or verify worker and broker connectivity.",
)
active_queues_by_worker = inspector.active_queues() or {}
active_by_worker = inspector.active() or {}
reserved_by_worker = inspector.reserved() or {}
active_queues = sorted(
{
str(queue.get("name"))
for worker_queues in active_queues_by_worker.values()
if isinstance(worker_queues, list)
for queue in worker_queues
if isinstance(queue, Mapping) and queue.get("name")
}
)
missing_queues = sorted(set(queues) - set(active_queues))
active_work = _task_count(active_by_worker)
reserved_work = _task_count(reserved_by_worker)
try:
measured_depths = dict(queue_depth_reader(queues))
except Exception: # noqa: BLE001 - queue depth remains explicitly unsupported.
measured_depths = {}
queue_depths = {
queue: _bounded_count(measured_depths.get(queue)) for queue in queues
}
known_depth = sum(value for value in queue_depths.values() if value is not None)
worker_nodes = _worker_nodes(context)
stale_nodes = [node for node in worker_nodes if node.get("stale") is True]
latest_heartbeat = _latest_heartbeat(worker_nodes)
if stale_nodes and len(stale_nodes) >= len(worker_nodes) > 0:
state = "stale"
detail = "All registered worker heartbeats are stale."
guidance = "Restore worker heartbeats or replace the stale worker incarnations."
elif missing_queues or stale_nodes:
state = "degraded"
detail = "Worker status is partial: a queue lacks a consumer or a registered worker is stale."
guidance = "Restore the missing queue consumers and investigate stale worker heartbeats."
elif active_work + reserved_work + known_depth > 0:
state = "busy"
detail = "Workers are processing or waiting to process queued work."
guidance = "Monitor queue age and failures; scale only within configured provider limits."
elif all(value is not None for value in queue_depths.values()):
state = "idle"
detail = "Workers are available and all measured queues are empty."
guidance = "No action is required."
else:
state = "healthy"
detail = "Workers answered and cover all configured queues; queue depth is unavailable."
guidance = "Treat queue depth as unavailable, not empty."
return _status(
context,
enabled=True,
configured=True,
state=state,
detail=detail,
active_workers=len(replies),
last_heartbeat_at=latest_heartbeat,
queue_depths=queue_depths,
active_work=active_work,
reserved_work=reserved_work,
guidance=guidance,
)
def _status(
context: RuntimeWorkStatusContext,
*,
enabled: bool,
configured: bool,
state: RuntimeWorkState,
detail: str,
active_workers: int | None = None,
last_heartbeat_at: datetime | None = None,
queue_depths: Mapping[str, int | None] | None = None,
active_work: int | None = None,
reserved_work: int | None = None,
guidance: str,
) -> RuntimeWorkStatus:
return RuntimeWorkStatus(
provider_id="core.celery",
label="Background workers",
backend="Celery",
enabled=enabled,
configured=configured,
state=state,
detail=detail,
observed_at=context.observed_at,
active_workers=active_workers,
last_heartbeat_at=last_heartbeat_at,
queue_depths=queue_depths or {},
active_work=active_work,
reserved_work=reserved_work,
failures=None,
stale_after_seconds=context.stale_after_seconds,
guidance=guidance,
)
def _configured_queues() -> list[str]:
return sorted(
{
item.strip()
for item in str(settings.celery_queues or "").split(",")
if item.strip()
}
)
def _redis_queue_depths(queues: list[str]) -> Mapping[str, int | None]:
from redis import Redis
client = Redis.from_url(
settings.redis_url,
socket_connect_timeout=0.75,
socket_timeout=0.75,
)
pipeline = client.pipeline(transaction=False)
for queue in queues:
pipeline.llen(queue)
values = pipeline.execute()
return {
queue: _bounded_count(value)
for queue, value in zip(queues, values, strict=True)
}
def _task_count(tasks_by_worker: object) -> int:
if not isinstance(tasks_by_worker, Mapping):
return 0
return sum(
len(tasks) for tasks in tasks_by_worker.values() if isinstance(tasks, list)
)
def _bounded_count(value: object) -> int | None:
if isinstance(value, bool) or not isinstance(value, int | float):
return None
return max(0, int(value))
def _worker_nodes(context: RuntimeWorkStatusContext) -> list[Mapping[str, object]]:
return [node for node in context.runtime_nodes if node.get("role") == "worker"]
def _fresh_worker_nodes(context: RuntimeWorkStatusContext) -> list[Mapping[str, object]]:
return [node for node in _worker_nodes(context) if node.get("stale") is not True]
def _latest_heartbeat(nodes: list[Mapping[str, object]]) -> datetime | None:
values: list[datetime] = []
for node in nodes:
raw = node.get("last_heartbeat_at")
if isinstance(raw, datetime):
values.append(raw)
continue
if isinstance(raw, str):
try:
values.append(datetime.fromisoformat(raw.replace("Z", "+00:00")))
except ValueError:
continue
return max(values) if values else None
@@ -0,0 +1,621 @@
from __future__ import annotations
import hashlib
import json
import re
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from typing import Literal, Protocol, runtime_checkable
SEMANTIC_DOCUMENTATION_SUBJECT_CAPABILITY_PREFIX = (
"documentation.semantic_subjects."
)
SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION = "1"
SemanticDocumentationSubjectAvailability = Literal[
"available",
"changed",
"superseded",
"missing",
"temporarily_unavailable",
]
_MODULE_ID_RE = re.compile(r"^[a-z][a-z0-9_]{0,79}$")
_KIND_RE = re.compile(r"^[a-z][a-z0-9_.-]{0,119}$")
_IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:@-]{0,254}$")
_LOCALE_RE = re.compile(r"^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{2,8})*$")
_REASON_CODE_RE = re.compile(r"^[a-z][a-z0-9_]{0,79}$")
_SHA256_RE = re.compile(r"^(?:sha256:)?[0-9a-fA-F]{64}$")
class SemanticDocumentationContractError(ValueError):
"""Raised when a semantic-documentation subject violates the Core contract."""
@dataclass(frozen=True, slots=True)
class SemanticDocumentationSubjectAnchor:
kind: str
id: str
def __post_init__(self) -> None:
_require_match(self.kind, _KIND_RE, "Semantic subject anchor kind")
_require_match(self.id, _IDENTIFIER_RE, "Semantic subject anchor id")
def to_dict(self) -> dict[str, str]:
return {"kind": self.kind, "id": self.id}
@classmethod
def from_mapping(
cls, value: Mapping[str, object]
) -> SemanticDocumentationSubjectAnchor:
_require_keys(value, {"kind", "id"}, "Semantic subject anchor")
return cls(kind=_required_text(value, "kind"), id=_required_text(value, "id"))
@dataclass(frozen=True, slots=True)
class SemanticDocumentationSubjectReference:
module_id: str
tenant_id: str
subject_kind: str
subject_id: str
anchor: SemanticDocumentationSubjectAnchor | None = None
observed_revision: str | None = None
observed_fingerprint: str | None = None
def __post_init__(self) -> None:
_require_match(self.module_id, _MODULE_ID_RE, "Semantic subject module id")
_require_match(self.tenant_id, _IDENTIFIER_RE, "Semantic subject tenant id")
_require_match(self.subject_kind, _KIND_RE, "Semantic subject kind")
_require_match(self.subject_id, _IDENTIFIER_RE, "Semantic subject id")
_optional_text(self.observed_revision, "Semantic subject observed revision", 255)
if self.observed_fingerprint is not None and not _SHA256_RE.fullmatch(
self.observed_fingerprint
):
raise SemanticDocumentationContractError(
"Semantic subject observed fingerprint must be a SHA-256 digest."
)
@property
def stable_key(self) -> str:
identity = {
"anchor": self.anchor.to_dict() if self.anchor else None,
"module_id": self.module_id,
"subject_id": self.subject_id,
"subject_kind": self.subject_kind,
"tenant_id": self.tenant_id,
}
encoded = json.dumps(
identity, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
return f"sha256:{hashlib.sha256(encoded).hexdigest()}"
def to_dict(self) -> dict[str, object]:
return {
"module_id": self.module_id,
"tenant_id": self.tenant_id,
"subject_kind": self.subject_kind,
"subject_id": self.subject_id,
"anchor": self.anchor.to_dict() if self.anchor else None,
"observed_revision": self.observed_revision,
"observed_fingerprint": self.observed_fingerprint,
}
@classmethod
def from_mapping(
cls, value: Mapping[str, object]
) -> SemanticDocumentationSubjectReference:
_require_keys(
value,
{
"module_id",
"tenant_id",
"subject_kind",
"subject_id",
"anchor",
"observed_revision",
"observed_fingerprint",
},
"Semantic subject reference",
)
raw_anchor = value.get("anchor")
if raw_anchor is not None and not isinstance(raw_anchor, Mapping):
raise SemanticDocumentationContractError(
"Semantic subject anchor must be an object."
)
return cls(
module_id=_required_text(value, "module_id"),
tenant_id=_required_text(value, "tenant_id"),
subject_kind=_required_text(value, "subject_kind"),
subject_id=_required_text(value, "subject_id"),
anchor=(
SemanticDocumentationSubjectAnchor.from_mapping(raw_anchor)
if isinstance(raw_anchor, Mapping)
else None
),
observed_revision=_mapping_optional_text(value, "observed_revision"),
observed_fingerprint=_mapping_optional_text(
value, "observed_fingerprint"
),
)
@dataclass(frozen=True, slots=True)
class SemanticDocumentationBreadcrumb:
label: str
subject_kind: str
subject_id: str
anchor: SemanticDocumentationSubjectAnchor | None = None
def __post_init__(self) -> None:
_required_bounded_text(self.label, "Semantic subject breadcrumb label", 300)
_require_match(self.subject_kind, _KIND_RE, "Semantic breadcrumb kind")
_require_match(self.subject_id, _IDENTIFIER_RE, "Semantic breadcrumb id")
def to_dict(self) -> dict[str, object]:
return {
"label": self.label,
"subject_kind": self.subject_kind,
"subject_id": self.subject_id,
"anchor": self.anchor.to_dict() if self.anchor else None,
}
@dataclass(frozen=True, slots=True)
class SemanticDocumentationSubjectDescriptor:
reference: SemanticDocumentationSubjectReference
labels: Mapping[str, str]
descriptions: Mapping[str, str] = field(default_factory=dict)
breadcrumbs: tuple[SemanticDocumentationBreadcrumb, ...] = ()
route: str | None = None
route_anchor: str | None = None
audience: tuple[str, ...] = ()
classification: str = "internal"
required_scopes: tuple[str, ...] = ()
def __post_init__(self) -> None:
if not self.reference.observed_revision:
raise SemanticDocumentationContractError(
"Semantic subject descriptors require a current revision."
)
if not self.reference.observed_fingerprint:
raise SemanticDocumentationContractError(
"Semantic subject descriptors require a current fingerprint."
)
_localized_text(self.labels, "Semantic subject labels", required=True, limit=300)
_localized_text(
self.descriptions,
"Semantic subject descriptions",
required=False,
limit=2_000,
)
if len(self.breadcrumbs) > 32:
raise SemanticDocumentationContractError(
"Semantic subject breadcrumbs are limited to 32 items."
)
if self.route is not None:
_optional_text(self.route, "Semantic subject route", 2_000)
if not self.route.startswith("/") or self.route.startswith("//"):
raise SemanticDocumentationContractError(
"Semantic subject routes must be local absolute paths."
)
if self.route_anchor is not None:
_require_match(
self.route_anchor,
_IDENTIFIER_RE,
"Semantic subject route anchor",
)
_text_tuple(self.audience, "Semantic subject audience", maximum=32)
_required_bounded_text(
self.classification, "Semantic subject classification", 120
)
_text_tuple(
self.required_scopes, "Semantic subject required scopes", maximum=64
)
def to_dict(self) -> dict[str, object]:
return {
"reference": self.reference.to_dict(),
"labels": dict(self.labels),
"descriptions": dict(self.descriptions),
"breadcrumbs": [item.to_dict() for item in self.breadcrumbs],
"route": self.route,
"route_anchor": self.route_anchor,
"audience": list(self.audience),
"classification": self.classification,
"required_scopes": list(self.required_scopes),
}
@dataclass(frozen=True, slots=True)
class SemanticDocumentationSubjectResolution:
requested_reference: SemanticDocumentationSubjectReference
availability: SemanticDocumentationSubjectAvailability
subject: SemanticDocumentationSubjectDescriptor | None = None
superseded_by: SemanticDocumentationSubjectReference | None = None
reason_code: str | None = None
def __post_init__(self) -> None:
if self.reason_code is not None:
_require_match(
self.reason_code, _REASON_CODE_RE, "Semantic resolution reason code"
)
if self.availability in {"available", "changed"}:
if self.subject is None:
raise SemanticDocumentationContractError(
f"Semantic subject {self.availability} resolutions require a descriptor."
)
if (
self.subject.reference.stable_key
!= self.requested_reference.stable_key
):
raise SemanticDocumentationContractError(
"Semantic subject resolution changed the requested identity."
)
changed = _reference_changed(
self.requested_reference, self.subject.reference
)
if self.availability == "available" and changed:
raise SemanticDocumentationContractError(
"Changed semantic subjects must use the changed availability."
)
if self.availability == "changed" and not changed:
raise SemanticDocumentationContractError(
"Changed semantic subject resolutions require a revision or fingerprint change."
)
elif self.subject is not None:
raise SemanticDocumentationContractError(
f"Semantic subject {self.availability} resolutions cannot include a descriptor."
)
if self.availability == "superseded":
if self.superseded_by is None:
raise SemanticDocumentationContractError(
"Superseded semantic subjects require a replacement reference."
)
elif self.superseded_by is not None:
raise SemanticDocumentationContractError(
"Only superseded semantic subjects may declare a replacement."
)
if self.availability in {"missing", "temporarily_unavailable"} and not self.reason_code:
raise SemanticDocumentationContractError(
f"Semantic subject {self.availability} resolutions require a reason code."
)
def to_dict(self) -> dict[str, object]:
return {
"requested_reference": self.requested_reference.to_dict(),
"availability": self.availability,
"subject": self.subject.to_dict() if self.subject else None,
"superseded_by": (
self.superseded_by.to_dict() if self.superseded_by else None
),
"reason_code": self.reason_code,
}
@dataclass(frozen=True, slots=True)
class SemanticDocumentationSubjectQuery:
tenant_id: str
query: str = ""
subject_kinds: tuple[str, ...] = ()
limit: int = 50
cursor: str | None = None
def __post_init__(self) -> None:
_require_match(self.tenant_id, _IDENTIFIER_RE, "Semantic query tenant id")
if not isinstance(self.query, str) or len(self.query) > 300:
raise SemanticDocumentationContractError(
"Semantic subject query must be text of at most 300 characters."
)
if self.query:
_required_bounded_text(self.query, "Semantic subject query", 300)
if not 1 <= self.limit <= 200:
raise SemanticDocumentationContractError(
"Semantic subject query limit must be between 1 and 200."
)
_text_tuple(self.subject_kinds, "Semantic query subject kinds", maximum=100)
for kind in self.subject_kinds:
_require_match(kind, _KIND_RE, "Semantic query subject kind")
_optional_text(self.cursor, "Semantic query cursor", 1_000)
@dataclass(frozen=True, slots=True)
class SemanticDocumentationSubjectPage:
subjects: tuple[SemanticDocumentationSubjectDescriptor, ...] = ()
next_cursor: str | None = None
has_more: bool = False
def __post_init__(self) -> None:
keys = tuple(item.reference.stable_key for item in self.subjects)
if len(keys) != len(set(keys)):
raise SemanticDocumentationContractError(
"Semantic subject pages cannot contain duplicate identities."
)
_optional_text(self.next_cursor, "Semantic subject page cursor", 1_000)
if self.has_more and not self.next_cursor:
raise SemanticDocumentationContractError(
"Semantic subject pages with more results require a cursor."
)
@runtime_checkable
class SemanticDocumentationSubjectProvider(Protocol):
provider_id: str
module_id: str
contract_version: str
def list_subjects(
self,
session: object,
principal: object,
*,
request: SemanticDocumentationSubjectQuery,
) -> SemanticDocumentationSubjectPage: ...
def resolve_subject(
self,
session: object,
principal: object,
*,
reference: SemanticDocumentationSubjectReference,
) -> SemanticDocumentationSubjectResolution | None:
"""Return None when the principal may not know whether a subject exists."""
def semantic_documentation_subject_capability(module_id: str) -> str:
_require_match(module_id, _MODULE_ID_RE, "Semantic subject module id")
return f"{SEMANTIC_DOCUMENTATION_SUBJECT_CAPABILITY_PREFIX}{module_id}"
def semantic_documentation_subject_provider_names(
registry: object | None,
) -> tuple[str, ...]:
if registry is None or not hasattr(registry, "capability_names"):
return ()
return tuple(
str(name)
for name in registry.capability_names()
if str(name).startswith(SEMANTIC_DOCUMENTATION_SUBJECT_CAPABILITY_PREFIX)
)
def semantic_documentation_subject_providers(
registry: object | None,
) -> tuple[tuple[str, SemanticDocumentationSubjectProvider], ...]:
if registry is None or not hasattr(registry, "capability"):
return ()
providers: list[tuple[str, SemanticDocumentationSubjectProvider]] = []
for capability_name in semantic_documentation_subject_provider_names(registry):
module_id = capability_name.removeprefix(
SEMANTIC_DOCUMENTATION_SUBJECT_CAPABILITY_PREFIX
)
provider = registry.capability(capability_name)
if not isinstance(provider, SemanticDocumentationSubjectProvider):
raise TypeError(
f"Invalid semantic-documentation provider capability: {capability_name}"
)
if provider.module_id != module_id:
raise SemanticDocumentationContractError(
f"Semantic provider module {provider.module_id!r} does not match "
f"capability {capability_name!r}."
)
if (
provider.contract_version
!= SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION
):
raise SemanticDocumentationContractError(
f"Unsupported semantic-documentation provider contract: "
f"{provider.contract_version!r}."
)
providers.append((module_id, provider))
return tuple(providers)
def list_semantic_documentation_subjects(
registry: object | None,
session: object,
principal: object,
*,
request: SemanticDocumentationSubjectQuery,
) -> tuple[tuple[str, SemanticDocumentationSubjectPage], ...]:
if _principal_tenant_id(principal) != request.tenant_id:
return ()
pages: list[tuple[str, SemanticDocumentationSubjectPage]] = []
for module_id, provider in semantic_documentation_subject_providers(registry):
page = provider.list_subjects(
session,
principal,
request=request,
)
if any(
subject.reference.module_id != module_id
or subject.reference.tenant_id != request.tenant_id
for subject in page.subjects
):
raise SemanticDocumentationContractError(
f"Semantic provider {module_id!r} returned a foreign subject."
)
pages.append((module_id, page))
return tuple(pages)
def resolve_semantic_documentation_subject(
registry: object | None,
session: object,
principal: object,
*,
reference: SemanticDocumentationSubjectReference,
) -> SemanticDocumentationSubjectResolution | None:
if _principal_tenant_id(principal) != reference.tenant_id:
return None
capability_name = semantic_documentation_subject_capability(reference.module_id)
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(capability_name)
):
return SemanticDocumentationSubjectResolution(
requested_reference=reference,
availability="temporarily_unavailable",
reason_code="provider_unavailable",
)
provider = registry.capability(capability_name)
if not isinstance(provider, SemanticDocumentationSubjectProvider):
raise TypeError(
f"Invalid semantic-documentation provider capability: {capability_name}"
)
if (
provider.module_id != reference.module_id
or provider.contract_version
!= SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION
):
raise SemanticDocumentationContractError(
f"Semantic provider {capability_name!r} does not match the Core contract."
)
result = provider.resolve_subject(
session,
principal,
reference=reference,
)
if result is None:
return None
if result.requested_reference != reference:
raise SemanticDocumentationContractError(
"Semantic provider returned a resolution for another reference."
)
return result
def semantic_documentation_fingerprint(value: object) -> str:
try:
encoded = json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise SemanticDocumentationContractError(
"Semantic fingerprint input must be canonical JSON data."
) from exc
return f"sha256:{hashlib.sha256(encoded).hexdigest()}"
def _reference_changed(
requested: SemanticDocumentationSubjectReference,
current: SemanticDocumentationSubjectReference,
) -> bool:
comparisons = (
(requested.observed_revision, current.observed_revision),
(requested.observed_fingerprint, current.observed_fingerprint),
)
return any(expected is not None and expected != actual for expected, actual in comparisons)
def _principal_tenant_id(principal: object) -> str:
return str(getattr(principal, "tenant_id", "") or "")
def _localized_text(
values: Mapping[str, str],
label: str,
*,
required: bool,
limit: int,
) -> None:
if required and not values:
raise SemanticDocumentationContractError(f"{label} are required.")
if len(values) > 20:
raise SemanticDocumentationContractError(f"{label} are limited to 20 locales.")
for locale, value in values.items():
if not _LOCALE_RE.fullmatch(str(locale)):
raise SemanticDocumentationContractError(
f"{label} contain an invalid locale: {locale!r}."
)
_required_bounded_text(value, f"{label} value", limit)
def _text_tuple(values: Sequence[str], label: str, *, maximum: int) -> None:
if len(values) > maximum:
raise SemanticDocumentationContractError(
f"{label} are limited to {maximum} items."
)
normalized = tuple(str(value).strip() for value in values)
if any(not value or len(value) > 255 for value in normalized):
raise SemanticDocumentationContractError(
f"{label} must contain non-empty bounded text."
)
if len(normalized) != len(set(normalized)):
raise SemanticDocumentationContractError(f"{label} must be unique.")
def _require_match(value: str, pattern: re.Pattern[str], label: str) -> None:
if not isinstance(value, str) or not pattern.fullmatch(value):
raise SemanticDocumentationContractError(f"{label} is invalid.")
def _required_bounded_text(value: str, label: str, limit: int) -> None:
if not isinstance(value, str) or not value.strip() or len(value) > limit:
raise SemanticDocumentationContractError(
f"{label} must be non-empty and at most {limit} characters."
)
if any(ord(character) < 32 and character not in "\n\t" for character in value):
raise SemanticDocumentationContractError(f"{label} contains control characters.")
def _optional_text(value: str | None, label: str, limit: int) -> None:
if value is not None:
_required_bounded_text(value, label, limit)
def _require_keys(
value: Mapping[str, object], allowed: set[str], label: str
) -> None:
unexpected = sorted(str(key) for key in value if str(key) not in allowed)
if unexpected:
raise SemanticDocumentationContractError(
f"{label} contains unsupported fields: {', '.join(unexpected)}."
)
def _required_text(value: Mapping[str, object], key: str) -> str:
result = value.get(key)
if not isinstance(result, str) or not result.strip():
raise SemanticDocumentationContractError(
f"Semantic subject field {key} is required."
)
return result
def _mapping_optional_text(value: Mapping[str, object], key: str) -> str | None:
result = value.get(key)
if result is None:
return None
if not isinstance(result, str):
raise SemanticDocumentationContractError(
f"Semantic subject field {key} must be text."
)
return result
__all__ = [
"SEMANTIC_DOCUMENTATION_SUBJECT_CAPABILITY_PREFIX",
"SEMANTIC_DOCUMENTATION_SUBJECT_CONTRACT_VERSION",
"SemanticDocumentationBreadcrumb",
"SemanticDocumentationContractError",
"SemanticDocumentationSubjectAnchor",
"SemanticDocumentationSubjectAvailability",
"SemanticDocumentationSubjectDescriptor",
"SemanticDocumentationSubjectPage",
"SemanticDocumentationSubjectProvider",
"SemanticDocumentationSubjectQuery",
"SemanticDocumentationSubjectReference",
"SemanticDocumentationSubjectResolution",
"list_semantic_documentation_subjects",
"resolve_semantic_documentation_subject",
"semantic_documentation_fingerprint",
"semantic_documentation_subject_capability",
"semantic_documentation_subject_provider_names",
"semantic_documentation_subject_providers",
]
+156 -7
View File
@@ -1,11 +1,15 @@
from __future__ import annotations
import csv
import hashlib
import io
import json
import math
import re
from collections.abc import Mapping, Sequence
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass, field
from datetime import datetime
from decimal import Decimal
from typing import Literal, Protocol, runtime_checkable
@@ -17,6 +21,77 @@ DEFAULT_PREVIEW_TIMEOUT_MS = 2_000
TabularSourceMode = Literal["live", "cached", "file_backed", "static"]
TabularHealthStatus = Literal["healthy", "warning", "error", "unknown"]
TabularDiagnosticSeverity = Literal["info", "warning", "error"]
CsvValueMode = Literal["legacy_typed", "text"]
@dataclass(frozen=True, slots=True)
class TabularCsvSource:
"""Original upload text, retained only with an explicitly durable import.
This is never catalogue metadata or transient preview retention. Owners
enforce access, size limits, lifecycle and export authorization separately.
"""
text: str
delimiter: str = ","
value_mode: CsvValueMode = "legacy_typed"
parser_profile: str = "core.csv.v1"
def csv_source_payload(source: TabularCsvSource, *, max_bytes: int = 5_000_000) -> dict[str, object]:
encoded = _csv_utf8_bytes(source.text)
if len(encoded) > max_bytes:
raise TabularSourceValidationError(f"Original CSV input is limited to {max_bytes:,} UTF-8 bytes.")
if source.value_mode not in {"text", "legacy_typed"} or len(source.delimiter) != 1:
raise TabularSourceValidationError("Invalid CSV source parsing options.")
return {
"text": source.text,
"delimiter": source.delimiter,
"value_mode": source.value_mode,
"parser_profile": source.parser_profile,
"sha256": hashlib.sha256(encoded).hexdigest(),
"byte_count": len(encoded),
}
def csv_source_summary(payload: Mapping[str, object]) -> dict[str, object]:
"""Allowlist the small, non-content evidence safe for catalogue DTOs."""
result = {key: payload[key] for key in ("delimiter", "value_mode", "parser_profile", "sha256", "byte_count")}
if "governance_history" in payload:
result["governance_sha256"] = hashlib.sha256(json.dumps(payload["governance_history"], sort_keys=True, separators=(",", ":"), allow_nan=False).encode("utf-8")).hexdigest()
return result
def verified_csv_source_text(payload: Mapping[str, object], *, expected_summary: Mapping[str, object] | None = None) -> str:
text = payload.get("text")
if not isinstance(text, str):
raise TabularSourceUnavailableError("Original CSV source text is unavailable.")
try:
encoded = text.encode("utf-8")
except UnicodeError as exc:
raise TabularSourceUnavailableError("Original CSV source encoding is invalid.") from exc
if len(encoded) != payload.get("byte_count") or hashlib.sha256(encoded).hexdigest() != payload.get("sha256"):
raise TabularSourceUnavailableError("Original CSV source integrity verification failed.")
if expected_summary is not None:
try:
actual = json.dumps(csv_source_summary(payload), sort_keys=True, separators=(",", ":"), allow_nan=False)
expected = json.dumps(expected_summary, sort_keys=True, separators=(",", ":"), allow_nan=False)
except (KeyError, TypeError, ValueError) as exc:
raise TabularSourceUnavailableError("Original CSV source evidence is invalid.") from exc
if actual != expected:
raise TabularSourceUnavailableError("Original CSV source no longer matches its recorded evidence.")
return text
def csv_projection_matches(expected: Sequence[Mapping[str, object]], actual: Sequence[Mapping[str, object]]) -> bool:
"""CSV cells are scalar: booleans, integers and floats are not interchangeable."""
return len(expected) == len(actual) and all(
left.keys() == right.keys() and all(
type(value) is type(right[name]) and value == right[name]
for name, value in left.items()
)
for left, right in zip(expected, actual, strict=True)
)
class TabularSourceError(ValueError):
@@ -39,29 +114,44 @@ class TabularSourceUnavailableError(TabularSourceError):
pass
def _csv_utf8_bytes(text: str) -> bytes:
try:
return text.encode("utf-8")
except UnicodeError as exc:
raise TabularSourceValidationError("CSV input must be valid Unicode encodable as UTF-8.") from exc
def parse_tabular_csv(
csv_text: str,
*,
delimiter: str = ",",
max_rows: int = 10_000,
max_bytes: int = 5_000_000,
value_mode: CsvValueMode = "legacy_typed",
) -> tuple[Mapping[str, object], ...]:
"""Parse a bounded CSV document into JSON-compatible tabular rows."""
"""Parse CSV with an explicit lexical-text or backward-compatible typed mode."""
if len(delimiter) != 1:
raise TabularSourceValidationError("CSV delimiter must be one character.")
if value_mode not in {"legacy_typed", "text"}:
raise TabularSourceValidationError("Unsupported CSV value mode.")
if len(_csv_utf8_bytes(csv_text)) > max_bytes:
raise TabularSourceValidationError(f"CSV input is limited to {max_bytes:,} UTF-8 bytes.")
try:
reader = csv.DictReader(io.StringIO(csv_text), delimiter=delimiter)
reader = csv.DictReader(io.StringIO(csv_text), delimiter=delimiter, strict=value_mode == "text")
original_headers, normalized_headers = _csv_headers(reader.fieldnames)
rows: list[dict[str, object]] = []
for row in reader:
_validate_csv_row_shape(row)
if _csv_row_is_empty(row, original_headers):
if value_mode == "text" and (None in row or any(row.get(header) is None for header in original_headers)):
raise TabularSourceValidationError("CSV text rows must have exactly the number of values defined by the header.")
if value_mode == "legacy_typed" and _csv_row_is_empty(row, original_headers):
continue
if len(rows) >= max_rows:
raise TabularSourceValidationError(
f"CSV snapshots are limited to {max_rows:,} rows."
)
rows.append(_csv_row(row, original_headers, normalized_headers))
rows.append(_csv_row(row, original_headers, normalized_headers, value_mode=value_mode))
return tuple(rows)
except csv.Error as exc:
raise TabularSourceValidationError(f"CSV input could not be parsed: {exc}") from exc
@@ -106,9 +196,11 @@ def _csv_row(
row: Mapping[str | None, str | list[str] | None],
original_headers: Sequence[str],
normalized_headers: Sequence[str],
*,
value_mode: CsvValueMode = "legacy_typed",
) -> dict[str, object]:
return {
normalized: _csv_scalar(value if isinstance(value, str) else None)
normalized: (value if value_mode == "text" else _csv_scalar(value if isinstance(value, str) else None))
for original, normalized in zip(
original_headers,
normalized_headers,
@@ -128,9 +220,15 @@ def _csv_scalar(value: str | None) -> object:
if lowered in {"true", "false"}:
return lowered == "true"
if re.fullmatch(r"-?(?:0|[1-9][0-9]*)", text):
try:
return int(text)
except ValueError as exc:
raise TabularSourceValidationError("CSV integer exceeds the conversion limit; use text mode to preserve it.") from exc
if re.fullmatch(r"-?(?:0|[1-9][0-9]*)\.[0-9]+", text):
return float(text)
value = float(text)
if not math.isfinite(value):
raise TabularSourceValidationError("CSV numeric value exceeds the finite number range; use text mode to preserve it.")
return value
return text
@@ -141,6 +239,48 @@ class TabularColumn:
nullable: bool = True
def tabular_type_name(value: object, *, casefold_unknown: bool = False) -> str:
if isinstance(value, bool):
return "boolean"
if isinstance(value, int):
return "integer"
if isinstance(value, (float, Decimal)):
return "number"
if isinstance(value, str):
return "string"
if isinstance(value, list):
return "array"
if isinstance(value, dict):
return "object"
name = type(value).__name__
return name.casefold() if casefold_unknown else name.lower()
def infer_tabular_schema(
rows: Sequence[Mapping[str, object]],
*,
type_name: Callable[[object], str] = tabular_type_name,
) -> tuple[TabularColumn, ...]:
"""Infer first-seen columns in one pass without retaining column values.
The classifier is explicit so legacy providers can preserve their exact
type naming. Missing keys and explicit None both make a column nullable.
"""
states: dict[str, tuple[str | None, int]] = {}
for row in rows:
for name, value in row.items():
kind, concrete = states.get(name, (None, 0))
if value is not None:
value_kind = type_name(value)
kind = value_kind if kind is None else kind if kind == value_kind else "mixed"
concrete += 1
states[name] = (kind, concrete)
return tuple(
TabularColumn(name=name, data_type=kind if kind is not None else "unknown", nullable=concrete != len(rows))
for name, (kind, concrete) in states.items()
)
@dataclass(frozen=True, slots=True)
class TabularPushdown:
projections: bool = False
@@ -221,6 +361,7 @@ class TabularSnapshotInput:
rows: tuple[Mapping[str, object], ...]
description: str | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
csv_source: TabularCsvSource | None = None
@runtime_checkable
@@ -296,6 +437,10 @@ __all__ = [
"CAPABILITY_CONNECTORS_TABULAR_SOURCES",
"DEFAULT_PREVIEW_BYTES",
"DEFAULT_PREVIEW_TIMEOUT_MS",
"CsvValueMode",
"TabularCsvSource",
"tabular_type_name",
"infer_tabular_schema",
"TabularColumn",
"TabularPreviewDiagnostic",
"TabularPushdown",
@@ -313,6 +458,10 @@ __all__ = [
"TabularSourceUnavailableError",
"TabularSourceValidationError",
"parse_tabular_csv",
"csv_source_payload",
"csv_source_summary",
"verified_csv_source_text",
"csv_projection_matches",
"tabular_snapshot_writer",
"tabular_source_provider",
]
+332
View File
@@ -0,0 +1,332 @@
from __future__ import annotations
from collections.abc import Callable, Mapping
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal, Protocol, runtime_checkable
from govoplan_core.core.modules import ModuleContext
WorkItemStatus = Literal[
"open",
"in_progress",
"deferred",
"blocked",
"completed",
"cancelled",
]
WorkItemPriority = Literal["low", "normal", "high", "urgent"]
WorkAssignmentKind = Literal[
"account",
"group",
"role",
"function",
"function_assignment",
"anyone",
]
WORK_ITEM_CONTRACT_VERSION = "1"
CAPABILITY_TASK_COMMANDS = "tasks.commands"
@dataclass(frozen=True, slots=True)
class WorkAssignmentRef:
kind: WorkAssignmentKind
id: str
label: str | None = None
def __post_init__(self) -> None:
if self.kind not in {
"account",
"group",
"role",
"function",
"function_assignment",
"anyone",
}:
raise ValueError(f"Unsupported work-assignment kind: {self.kind!r}.")
if not self.id.strip():
raise ValueError("Work assignments require an id.")
if len(self.id) > 255:
raise ValueError("Work assignment ids are limited to 255 characters.")
if self.label is not None and len(self.label) > 500:
raise ValueError("Work assignment labels are limited to 500 characters.")
if self.kind == "anyone" and self.id != "*":
raise ValueError("Broad work assignments use the canonical '*' id.")
@dataclass(frozen=True, slots=True)
class WorkSourceRef:
module_id: str
resource_type: str
resource_id: str
revision: str | None = None
url: str | None = None
label: str | None = None
def __post_init__(self) -> None:
required = (self.module_id, self.resource_type, self.resource_id)
if any(not value.strip() for value in required):
raise ValueError("Work source references require module, type, and id.")
limits = {
"module_id": 100,
"resource_type": 100,
"resource_id": 255,
"revision": 255,
"url": 1_500,
"label": 500,
}
for field_name, limit in limits.items():
value = getattr(self, field_name)
if value is not None and len(value) > limit:
raise ValueError(
f"Work source {field_name} is limited to {limit} characters."
)
@dataclass(frozen=True, slots=True)
class WorkItem:
id: str
provider_id: str
owner_module: str
tenant_id: str
title: str
status: WorkItemStatus = "open"
priority: WorkItemPriority = "normal"
summary: str | None = None
required_action: str | None = None
action_url: str | None = None
due_at: datetime | None = None
deferred_until: datetime | None = None
assignments: tuple[WorkAssignmentRef, ...] = ()
sources: tuple[WorkSourceRef, ...] = ()
provenance: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
revision: str = "1"
created_at: datetime | None = None
updated_at: datetime | None = None
def __post_init__(self) -> None:
required = {
"id": self.id,
"provider_id": self.provider_id,
"owner_module": self.owner_module,
"tenant_id": self.tenant_id,
"title": self.title,
"revision": self.revision,
}
if any(not value.strip() for value in required.values()):
raise ValueError(
"Work items require stable identity, owner, tenant, and title."
)
limits = {
"id": 255,
"provider_id": 200,
"owner_module": 100,
"tenant_id": 255,
"title": 500,
"summary": 4_000,
"required_action": 500,
"action_url": 1_500,
"revision": 255,
}
for field_name, limit in limits.items():
value = getattr(self, field_name)
if value is not None and len(value) > limit:
raise ValueError(
f"Work item {field_name} is limited to {limit} characters."
)
if len(self.assignments) > 100 or len(self.sources) > 100:
raise ValueError("Work items support at most 100 assignments and sources.")
_validate_action_url(self.action_url)
if self.status not in {
"open",
"in_progress",
"deferred",
"blocked",
"completed",
"cancelled",
}:
raise ValueError(f"Unsupported work-item status: {self.status!r}.")
if self.priority not in {"low", "normal", "high", "urgent"}:
raise ValueError(f"Unsupported work-item priority: {self.priority!r}.")
@dataclass(frozen=True, slots=True)
class WorkItemQuery:
tenant_id: str
statuses: tuple[WorkItemStatus, ...] = (
"open",
"in_progress",
"deferred",
"blocked",
)
priorities: tuple[WorkItemPriority, ...] = ()
provider_ids: tuple[str, ...] = ()
owner_modules: tuple[str, ...] = ()
due_before: datetime | None = None
text: str = ""
limit: int = 100
def __post_init__(self) -> None:
if not self.tenant_id.strip():
raise ValueError("Work-item queries require a tenant.")
if not 1 <= self.limit <= 500:
raise ValueError("Work-item query limits must be between 1 and 500.")
normalized = self.text.strip()
if len(normalized) > 500:
raise ValueError("Work-item query text is limited to 500 characters.")
if len(self.provider_ids) > 50 or len(self.owner_modules) > 50:
raise ValueError("Work-item queries support at most 50 provider filters.")
object.__setattr__(self, "text", normalized)
@dataclass(frozen=True, slots=True)
class WorkItemPage:
items: tuple[WorkItem, ...]
total: int
truncated: bool = False
def __post_init__(self) -> None:
if self.total < len(self.items):
raise ValueError(
"Work-item totals cannot be smaller than the returned page."
)
@runtime_checkable
class WorkItemProvider(Protocol):
def list_items(
self,
session: object,
principal: object,
*,
query: WorkItemQuery,
) -> WorkItemPage:
"""Return only items the current principal may discover and act on."""
@dataclass(frozen=True, slots=True)
class TaskCreateCommand:
tenant_id: str
title: str
idempotency_key: str
summary: str | None = None
priority: WorkItemPriority = "normal"
due_at: datetime | None = None
required_action: str | None = None
action_url: str | None = None
assignments: tuple[WorkAssignmentRef, ...] = ()
sources: tuple[WorkSourceRef, ...] = ()
provenance: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
if not self.tenant_id.strip() or not self.title.strip():
raise ValueError("Task commands require a tenant and title.")
if not self.idempotency_key.strip() or len(self.idempotency_key) > 255:
raise ValueError("Task commands require a bounded idempotency key.")
if not self.assignments:
raise ValueError("Explicit tasks require at least one assignment.")
if self.priority not in {"low", "normal", "high", "urgent"}:
raise ValueError(f"Unsupported task priority: {self.priority!r}.")
limits = {
"title": 500,
"summary": 4_000,
"required_action": 500,
"action_url": 1_500,
}
for field_name, limit in limits.items():
value = getattr(self, field_name)
if value is not None and len(value) > limit:
raise ValueError(
f"Task command {field_name} is limited to {limit} characters."
)
if len(self.assignments) > 100 or len(self.sources) > 100:
raise ValueError(
"Task commands support at most 100 assignments and sources."
)
_validate_action_url(self.action_url)
def _validate_action_url(value: str | None) -> None:
if value is None:
return
candidate = value.strip()
if not candidate:
return
if (
not candidate.startswith("/")
or candidate.startswith("//")
or "\\" in candidate
or any(ord(character) < 32 or ord(character) == 127 for character in candidate)
):
raise ValueError("Work-item action URLs must be application-relative paths.")
@runtime_checkable
class TaskCommandProvider(Protocol):
def create_task(
self,
session: object,
principal: object,
*,
command: TaskCreateCommand,
) -> WorkItem: ...
WorkItemProviderFactory = Callable[[ModuleContext], WorkItemProvider]
@dataclass(frozen=True, slots=True)
class WorkItemProviderRegistration:
id: str
factory: WorkItemProviderFactory
order: int = 100
def create(self, context: ModuleContext) -> WorkItemProvider:
provider = self.factory(context)
if not isinstance(provider, WorkItemProvider):
raise TypeError(
f"Work-item provider {self.id!r} does not implement WorkItemProvider."
)
return provider
@dataclass(frozen=True, slots=True)
class RegisteredWorkItemProvider:
module_id: str
registration: WorkItemProviderRegistration
def task_command_provider(registry: object | None) -> TaskCommandProvider | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not hasattr(registry, "capability")
or not registry.has_capability(CAPABILITY_TASK_COMMANDS)
):
return None
provider = registry.capability(CAPABILITY_TASK_COMMANDS)
return provider if isinstance(provider, TaskCommandProvider) else None
__all__ = [
"CAPABILITY_TASK_COMMANDS",
"RegisteredWorkItemProvider",
"TaskCommandProvider",
"TaskCreateCommand",
"WORK_ITEM_CONTRACT_VERSION",
"WorkAssignmentKind",
"WorkAssignmentRef",
"WorkItem",
"WorkItemPage",
"WorkItemPriority",
"WorkItemProvider",
"WorkItemProviderFactory",
"WorkItemProviderRegistration",
"WorkItemQuery",
"WorkItemStatus",
"WorkSourceRef",
"task_command_provider",
]
+39
View File
@@ -8,6 +8,7 @@ from typing import Literal, Protocol, runtime_checkable
CAPABILITY_TEMPLATE_CATALOG = "templates.catalog"
CAPABILITY_TEMPLATE_RENDERER = "templates.renderer"
CAPABILITY_TEMPLATE_CONTENT_LIBRARY = "templates.content_library"
TemplateType = Literal[
"label",
@@ -17,6 +18,7 @@ TemplateType = Literal[
"form_letter",
"list_layout",
"email",
"content_fragment",
"generic",
]
TemplateOutputFormat = Literal["html", "text"]
@@ -79,6 +81,10 @@ class TemplateRevisionRef:
locale: str
required_fields: tuple[TemplateFieldRequirement, ...]
output_profiles: tuple[TemplateOutputProfile, ...]
content_text: str | None = None
content_html: str | None = None
layout: Mapping[str, object] = field(default_factory=dict)
metadata: Mapping[str, object] = field(default_factory=dict)
published_at: datetime | None = None
provenance: Mapping[str, object] = field(default_factory=dict)
@@ -166,6 +172,23 @@ class TemplateRenderResult:
payload: bytes | None = None
@dataclass(frozen=True, slots=True)
class TemplateContentDraftRequest:
"""Provider-neutral request for a reusable text/HTML content draft."""
name: str
template_type: TemplateType
usages: tuple[str, ...]
content_text: str | None = None
content_html: str | None = None
description: str | None = None
locale: str = "de"
scope_type: Literal["tenant", "group", "user"] = "tenant"
scope_id: str | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
required_fields: tuple[TemplateFieldRequirement, ...] = ()
@runtime_checkable
class TemplateCatalogProvider(Protocol):
def list_templates(
@@ -213,13 +236,29 @@ class TemplateRendererProvider(Protocol):
) -> TemplateRenderResult: ...
@runtime_checkable
class TemplateContentLibraryProvider(Protocol):
"""Create reusable content drafts while Templates retains ownership."""
def create_content_draft(
self,
session: object,
principal: object,
*,
request: TemplateContentDraftRequest,
) -> TemplateRef: ...
__all__ = [
"CAPABILITY_TEMPLATE_CATALOG",
"CAPABILITY_TEMPLATE_CONTENT_LIBRARY",
"CAPABILITY_TEMPLATE_RENDERER",
"TemplateArtifactRef",
"TemplateCatalogProvider",
"TemplateCompatibility",
"TemplateCompatibilityError",
"TemplateContentDraftRequest",
"TemplateContentLibraryProvider",
"TemplateContractError",
"TemplateFieldRequirement",
"TemplateNotFoundError",
+182
View File
@@ -0,0 +1,182 @@
from __future__ import annotations
from contextvars import ContextVar, Token
from dataclasses import dataclass, field
from datetime import UTC, datetime
from typing import Literal
TemporalValidityMode = Literal["current", "at", "all"]
VALIDITY_MODES = frozenset({"current", "at", "all"})
VALIDITY_MODE_HEADER = "X-Govoplan-Validity-Mode"
VALID_AT_HEADER = "X-Govoplan-Valid-At"
RECORDED_AT_HEADER = "X-Govoplan-Recorded-At"
TEMPORAL_EVALUATED_AT_HEADER = "X-Govoplan-Temporal-Evaluated-At"
TEMPORAL_VARY_HEADERS = (
VALIDITY_MODE_HEADER,
VALID_AT_HEADER,
RECORDED_AT_HEADER,
)
class TemporalContextError(ValueError):
pass
@dataclass(frozen=True, slots=True)
class TemporalDataContext:
"""Bitemporal read context.
Valid time answers when a fact applies. Recorded time answers which version
of that fact was known to the system. Authorization remains outside this
context and is always evaluated under the current security state.
"""
validity_mode: TemporalValidityMode = "current"
valid_at: datetime | None = None
recorded_at: datetime | None = None
evaluated_at: datetime = field(default_factory=lambda: datetime.now(UTC))
def __post_init__(self) -> None:
if self.validity_mode not in VALIDITY_MODES:
raise TemporalContextError(
f"Unsupported temporal validity mode: {self.validity_mode!r}."
)
for name in ("valid_at", "recorded_at", "evaluated_at"):
value = getattr(self, name)
if value is not None and value.tzinfo is None:
raise TemporalContextError(f"Temporal {name} must include a timezone.")
if self.validity_mode == "at" and self.valid_at is None:
raise TemporalContextError("Validity mode 'at' requires valid_at.")
if self.validity_mode != "at" and self.valid_at is not None:
raise TemporalContextError(
"valid_at is only permitted when validity mode is 'at'."
)
@property
def validity_instant(self) -> datetime | None:
if self.validity_mode == "all":
return None
if self.validity_mode == "at":
return self.valid_at
return self.evaluated_at
@property
def is_default(self) -> bool:
return self.validity_mode == "current" and self.recorded_at is None
def to_dict(self) -> dict[str, str | None]:
return {
"validity_mode": self.validity_mode,
"valid_at": _datetime_text(self.valid_at),
"recorded_at": _datetime_text(self.recorded_at),
"evaluated_at": _datetime_text(self.evaluated_at),
}
_temporal_context: ContextVar[TemporalDataContext | None] = ContextVar(
"govoplan_temporal_data_context",
default=None,
)
def parse_temporal_data_context(
*,
validity_mode: str | None = None,
valid_at: str | None = None,
recorded_at: str | None = None,
evaluated_at: datetime | None = None,
) -> TemporalDataContext:
clean_mode = (validity_mode or "current").strip().lower()
if clean_mode not in VALIDITY_MODES:
raise TemporalContextError(
"Temporal validity mode must be one of: current, at, all."
)
return TemporalDataContext(
validity_mode=clean_mode, # type: ignore[arg-type]
valid_at=_parse_datetime(valid_at, "valid_at"),
recorded_at=_parse_datetime(recorded_at, "recorded_at"),
evaluated_at=evaluated_at or datetime.now(UTC),
)
def current_temporal_data_context() -> TemporalDataContext:
return _temporal_context.get() or TemporalDataContext()
def bind_temporal_data_context(
context: TemporalDataContext,
) -> Token[TemporalDataContext | None]:
return _temporal_context.set(context)
def reset_temporal_data_context(token: Token[TemporalDataContext | None]) -> None:
_temporal_context.reset(token)
def temporal_revision_matches(
context: TemporalDataContext,
*,
valid_from: datetime | None = None,
valid_to: datetime | None = None,
revision_recorded_at: datetime | None = None,
superseded_at: datetime | None = None,
) -> bool:
cutoff = context.recorded_at
if cutoff is None:
if superseded_at is not None:
return False
else:
if revision_recorded_at is None or revision_recorded_at > cutoff:
return False
if superseded_at is not None and superseded_at <= cutoff:
return False
instant = context.validity_instant
if instant is None:
return True
return (valid_from is None or valid_from <= instant) and (
valid_to is None or valid_to > instant
)
def _parse_datetime(value: str | None, name: str) -> datetime | None:
clean = str(value or "").strip()
if not clean:
return None
if len(clean) > 64:
raise TemporalContextError(f"Temporal {name} is too long.")
normalized = f"{clean[:-1]}+00:00" if clean.endswith(("Z", "z")) else clean
try:
parsed = datetime.fromisoformat(normalized)
except ValueError as exc:
raise TemporalContextError(
f"Temporal {name} must be an ISO 8601 timestamp."
) from exc
if parsed.tzinfo is None:
raise TemporalContextError(f"Temporal {name} must include a timezone.")
return parsed.astimezone(UTC)
def _datetime_text(value: datetime | None) -> str | None:
if value is None:
return None
return value.astimezone(UTC).isoformat().replace("+00:00", "Z")
__all__ = [
"RECORDED_AT_HEADER",
"TEMPORAL_EVALUATED_AT_HEADER",
"TEMPORAL_VARY_HEADERS",
"VALIDITY_MODE_HEADER",
"VALID_AT_HEADER",
"TemporalContextError",
"TemporalDataContext",
"TemporalValidityMode",
"bind_temporal_data_context",
"current_temporal_data_context",
"parse_temporal_data_context",
"reset_temporal_data_context",
"temporal_revision_matches",
]
+467
View File
@@ -0,0 +1,467 @@
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import UTC, datetime
from typing import Literal, Protocol, runtime_checkable
TENANT_ERASURE_PROVIDER_CAPABILITY_PREFIX = "tenancy.erasure_provider."
TenantErasureDisposition = Literal[
"erase",
"retain",
"legal_hold",
"external_cleanup",
"key_destroy",
"backup_expiry",
"unavailable",
]
TenantErasureStepKind = Literal[
"export",
"erase",
"retain",
"external_cleanup",
"key_destroy",
"backup_expiry",
"verify",
]
TenantErasureResultState = Literal[
"completed",
"pending",
"blocked",
"outcome_unknown",
]
_DISPOSITIONS = frozenset(
{
"erase",
"retain",
"legal_hold",
"external_cleanup",
"key_destroy",
"backup_expiry",
"unavailable",
}
)
_STEP_KINDS = frozenset(
{
"export",
"erase",
"retain",
"external_cleanup",
"key_destroy",
"backup_expiry",
"verify",
}
)
_RESULT_STATES = frozenset(
{"completed", "pending", "blocked", "outcome_unknown"}
)
def _text(value: str, label: str, *, maximum: int) -> str:
normalized = value.strip()
if (
not normalized
or len(normalized) > maximum
or any(ord(character) < 32 for character in normalized)
):
raise ValueError(f"Tenant erasure {label} is invalid.")
return normalized
def _texts(
values: tuple[str, ...],
label: str,
*,
maximum_items: int = 100,
maximum_length: int = 500,
) -> tuple[str, ...]:
if len(values) > maximum_items:
raise ValueError(f"Tenant erasure {label} has too many entries.")
normalized = tuple(
_text(value, label, maximum=maximum_length) for value in values
)
if len(normalized) != len(set(normalized)):
raise ValueError(f"Tenant erasure {label} contains duplicates.")
return normalized
def _metrics(values: Mapping[str, int]) -> dict[str, int]:
if len(values) > 30:
raise ValueError("Tenant erasure metrics has too many entries.")
normalized: dict[str, int] = {}
for key, value in values.items():
normalized_key = _text(key, "metric key", maximum=80)
if type(value) is not int or value < 0:
raise ValueError("Tenant erasure metric values must be non-negative integers.")
normalized[normalized_key] = value
return normalized
@dataclass(frozen=True, slots=True)
class TenantErasureResource:
resource_type: str
count: int
disposition: TenantErasureDisposition
summary: str
governance_ref: str | None = None
external: bool = False
def __post_init__(self) -> None:
_text(self.resource_type, "resource type", maximum=120)
_text(self.summary, "resource summary", maximum=1000)
if type(self.count) is not int or self.count < 0:
raise ValueError("Tenant erasure resource count is invalid.")
if self.disposition not in _DISPOSITIONS:
raise ValueError("Tenant erasure resource disposition is invalid.")
if self.governance_ref is not None:
_text(self.governance_ref, "governance reference", maximum=300)
def to_dict(self) -> dict[str, object]:
return {
"resource_type": self.resource_type,
"count": self.count,
"disposition": self.disposition,
"summary": self.summary,
"governance_ref": self.governance_ref,
"external": self.external,
}
@dataclass(frozen=True, slots=True)
class TenantErasureStep:
step_id: str
kind: TenantErasureStepKind
summary: str
destructive: bool
irreversible: bool
requires_reconciliation: bool = False
depends_on: tuple[str, ...] = ()
def __post_init__(self) -> None:
_text(self.step_id, "step id", maximum=160)
_text(self.summary, "step summary", maximum=1000)
if self.kind not in _STEP_KINDS:
raise ValueError("Tenant erasure step kind is invalid.")
_texts(self.depends_on, "step dependencies", maximum_length=160)
if self.step_id in self.depends_on:
raise ValueError("Tenant erasure step cannot depend on itself.")
if self.irreversible and not self.destructive:
raise ValueError("An irreversible tenant erasure step must be destructive.")
def to_dict(self) -> dict[str, object]:
return {
"step_id": self.step_id,
"kind": self.kind,
"summary": self.summary,
"destructive": self.destructive,
"irreversible": self.irreversible,
"requires_reconciliation": self.requires_reconciliation,
"depends_on": list(self.depends_on),
}
@dataclass(frozen=True, slots=True)
class TenantErasurePreview:
module_id: str
complete: bool
resources: tuple[TenantErasureResource, ...] = ()
steps: tuple[TenantErasureStep, ...] = ()
blockers: tuple[str, ...] = ()
warnings: tuple[str, ...] = ()
provider_revision: str = "1"
def __post_init__(self) -> None:
_text(self.module_id, "module id", maximum=120)
_text(self.provider_revision, "provider revision", maximum=120)
_texts(self.blockers, "blockers", maximum_length=1000)
_texts(self.warnings, "warnings", maximum_length=1000)
if len(self.resources) > 500 or len(self.steps) > 500:
raise ValueError("Tenant erasure preview is too large.")
resource_types = [item.resource_type for item in self.resources]
if len(resource_types) != len(set(resource_types)):
raise ValueError("Tenant erasure preview repeats a resource type.")
resources_requiring_action = tuple(
item for item in self.resources if item.count > 0
)
if resources_requiring_action and not self.steps and not self.blockers:
raise ValueError(
"Tenant erasure resources require steps or an explicit blocker."
)
if any(
item.count > 0 and item.disposition == "unavailable"
for item in self.resources
) and not self.blockers:
raise ValueError(
"Unavailable tenant erasure resources require an explicit blocker."
)
if not self.complete and not self.blockers:
raise ValueError(
"An incomplete tenant erasure preview requires an explicit blocker."
)
step_ids = [item.step_id for item in self.steps]
if len(step_ids) != len(set(step_ids)):
raise ValueError("Tenant erasure preview repeats a step id.")
known_step_ids = set(step_ids)
if any(
dependency not in known_step_ids
for step in self.steps
for dependency in step.depends_on
):
raise ValueError("Tenant erasure step references an unknown dependency.")
remaining = {
step.step_id: set(step.depends_on)
for step in self.steps
}
resolved: set[str] = set()
while remaining:
ready = sorted(
step_id
for step_id, dependencies in remaining.items()
if dependencies.issubset(resolved)
)
if not ready:
raise ValueError("Tenant erasure step dependencies contain a cycle.")
resolved.update(ready)
for step_id in ready:
remaining.pop(step_id)
@property
def allowed(self) -> bool:
return self.complete and not self.blockers
def to_dict(self) -> dict[str, object]:
return {
"module_id": self.module_id,
"complete": self.complete,
"allowed": self.allowed,
"provider_revision": self.provider_revision,
"resources": [item.to_dict() for item in self.resources],
"steps": [item.to_dict() for item in self.steps],
"blockers": list(self.blockers),
"warnings": list(self.warnings),
}
@dataclass(frozen=True, slots=True)
class TenantErasureStepResult:
state: TenantErasureResultState
summary: str
receipt_ref: str | None = None
metrics: Mapping[str, int] = field(default_factory=dict)
def __post_init__(self) -> None:
if self.state not in _RESULT_STATES:
raise ValueError("Tenant erasure result state is invalid.")
_text(self.summary, "result summary", maximum=1000)
if self.receipt_ref is not None:
_text(self.receipt_ref, "receipt reference", maximum=500)
_metrics(self.metrics)
def to_dict(self) -> dict[str, object]:
return {
"state": self.state,
"summary": self.summary,
"receipt_ref": self.receipt_ref,
"metrics": dict(sorted(_metrics(self.metrics).items())),
}
@runtime_checkable
class TenantErasureProvider(Protocol):
module_id: str
def preview_tenant_erasure(
self,
session: object,
tenant_id: str,
) -> TenantErasurePreview:
...
def execute_tenant_erasure_step(
self,
session: object,
tenant_id: str,
step_id: str,
idempotency_key: str,
) -> TenantErasureStepResult:
...
def reconcile_tenant_erasure_step(
self,
session: object,
tenant_id: str,
step_id: str,
idempotency_key: str,
) -> TenantErasureStepResult:
...
@dataclass(frozen=True, slots=True)
class TenantErasureInventory:
tenant_id: str
generated_at: datetime
complete: bool
modules: tuple[TenantErasurePreview, ...]
@property
def allowed(self) -> bool:
return self.complete and all(item.allowed for item in self.modules)
def to_dict(self) -> dict[str, object]:
generated_at = self.generated_at
if generated_at.tzinfo is None:
generated_at = generated_at.replace(tzinfo=UTC)
return {
"schema_version": 1,
"tenant_id": self.tenant_id,
"generated_at": generated_at.astimezone(UTC).isoformat(),
"complete": self.complete,
"allowed": self.allowed,
"modules": [item.to_dict() for item in self.modules],
}
def tenant_erasure_providers(registry: object) -> dict[str, TenantErasureProvider]:
capability_names = getattr(registry, "capability_names", None)
capability = getattr(registry, "capability", None)
if not callable(capability_names) or not callable(capability):
raise ValueError("Tenant erasure requires a module registry.")
providers: dict[str, TenantErasureProvider] = {}
for capability_name in sorted(capability_names()):
if not capability_name.startswith(TENANT_ERASURE_PROVIDER_CAPABILITY_PREFIX):
continue
expected_module_id = capability_name.removeprefix(
TENANT_ERASURE_PROVIDER_CAPABILITY_PREFIX
)
provider = capability(capability_name)
if not isinstance(provider, TenantErasureProvider):
raise TypeError(
f"Tenant erasure provider {expected_module_id or 'unknown'} is invalid."
)
module_id = _text(provider.module_id, "provider module id", maximum=120)
if module_id != expected_module_id or module_id in providers:
raise ValueError("Tenant erasure provider identity is invalid.")
providers[module_id] = provider
return providers
def collect_tenant_erasure_inventory(
registry: object,
session: object,
tenant_id: str,
*,
observed_at: datetime | None = None,
) -> TenantErasureInventory:
normalized_tenant_id = _text(tenant_id, "tenant id", maximum=120)
manifests = getattr(registry, "manifests", None)
summary_providers = getattr(registry, "tenant_summary_providers", None)
if not callable(manifests) or not callable(summary_providers):
raise ValueError("Tenant erasure inventory requires a module registry.")
provider_by_module = tenant_erasure_providers(registry)
summary_by_module = dict(summary_providers())
manifest_ids = {
str(manifest.id)
for manifest in manifests()
if getattr(manifest, "id", None)
}
module_ids = manifest_ids | set(summary_by_module) | set(provider_by_module)
previews: list[TenantErasurePreview] = []
complete = True
for module_id in sorted(module_ids):
provider = provider_by_module.get(module_id)
if provider is not None:
try:
preview = provider.preview_tenant_erasure(session, normalized_tenant_id)
if not isinstance(preview, TenantErasurePreview):
raise TypeError("provider returned an invalid preview")
if preview.module_id != module_id:
raise ValueError("provider returned another module's preview")
except Exception as exc:
complete = False
preview = TenantErasurePreview(
module_id=module_id,
complete=False,
blockers=(
f"{type(exc).__name__}: provider preview could not be completed",
),
)
previews.append(preview)
complete = complete and preview.complete
continue
summary_provider = summary_by_module.get(module_id)
if summary_provider is None:
previews.append(
TenantErasurePreview(
module_id=module_id,
complete=True,
warnings=(
"Module declares no tenant-owned summary or erasure provider; no tenant persistence is in scope.",
),
provider_revision="manifest-no-tenant-data",
)
)
continue
try:
raw_counts = summary_provider(session, normalized_tenant_id)
counts = _metrics({str(key): int(value) for key, value in raw_counts.items()})
resources = tuple(
TenantErasureResource(
resource_type=resource_type,
count=count,
disposition="unavailable" if count else "erase",
summary=(
"Tenant-owned data requires a module erasure provider."
if count
else "The module reported no tenant-owned records."
),
)
for resource_type, count in sorted(counts.items())
)
blockers = (
("Tenant-owned data exists but the module has no erasure provider.",)
if any(counts.values())
else ()
)
preview = TenantErasurePreview(
module_id=module_id,
complete=True,
resources=resources,
blockers=blockers,
provider_revision="tenant-summary-fallback",
)
except Exception as exc:
complete = False
preview = TenantErasurePreview(
module_id=module_id,
complete=False,
blockers=(
f"{type(exc).__name__}: tenant summary could not be completed",
),
provider_revision="tenant-summary-fallback",
)
previews.append(preview)
timestamp = observed_at or datetime.now(UTC)
if timestamp.tzinfo is None:
timestamp = timestamp.replace(tzinfo=UTC)
return TenantErasureInventory(
tenant_id=normalized_tenant_id,
generated_at=timestamp,
complete=complete,
modules=tuple(previews),
)
__all__ = [
"TENANT_ERASURE_PROVIDER_CAPABILITY_PREFIX",
"TenantErasureInventory",
"TenantErasurePreview",
"TenantErasureProvider",
"TenantErasureResource",
"TenantErasureStep",
"TenantErasureStepResult",
"collect_tenant_erasure_inventory",
"tenant_erasure_providers",
]
+179
View File
@@ -0,0 +1,179 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import datetime
from typing import Mapping, Protocol, runtime_checkable
TICKET_INTEGRATION_CONTRACT_VERSION = "1"
CAPABILITY_TICKET_ROUTING = "tickets.routing"
CAPABILITY_TICKET_CASE_ESCALATION = "tickets.case_escalation"
@dataclass(frozen=True, slots=True)
class TicketRoutingRequest:
tenant_id: str
ticket_id: str
ticket_type: str
priority: str
title: str
received_at: datetime
queue_hint: str | None = None
attributes: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_required(self.tenant_id, "Ticket routing tenant", 255)
_required(self.ticket_id, "Ticket routing ticket", 255)
_required(self.ticket_type, "Ticket routing type", 80)
_required(self.priority, "Ticket routing priority", 40)
_required(self.title, "Ticket routing title", 500)
_aware(self.received_at, "Ticket routing received_at")
_optional(self.queue_hint, "Ticket routing queue hint", 255)
if len(self.attributes) > 100:
raise ValueError("Ticket routing attributes are limited to 100 entries.")
@dataclass(frozen=True, slots=True)
class TicketRoutingPlan:
provider_id: str
queue_ref: str | None = None
service_target_at: datetime | None = None
explanation: str | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_required(self.provider_id, "Ticket routing provider", 200)
_optional(self.queue_ref, "Ticket routing queue reference", 255)
_optional(self.explanation, "Ticket routing explanation", 4_000)
_aware(self.service_target_at, "Ticket routing service_target_at")
if len(self.metadata) > 100:
raise ValueError("Ticket routing metadata is limited to 100 entries.")
@runtime_checkable
class TicketRoutingProvider(Protocol):
def route_ticket(
self,
session: object,
principal: object,
*,
request: TicketRoutingRequest,
) -> TicketRoutingPlan: ...
@dataclass(frozen=True, slots=True)
class TicketCaseEscalationCommand:
tenant_id: str
ticket_id: str
ticket_number: str
title: str
case_type_key: str
occurred_at: datetime
idempotency_key: str
handoff_note: str | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_required(self.tenant_id, "Ticket escalation tenant", 255)
_required(self.ticket_id, "Ticket escalation ticket", 255)
_required(self.ticket_number, "Ticket escalation number", 255)
_required(self.title, "Ticket escalation title", 500)
_required(self.case_type_key, "Ticket escalation case type", 120)
_required(self.idempotency_key, "Ticket escalation idempotency key", 255)
_optional(self.handoff_note, "Ticket escalation handoff note", 10_000)
_aware(self.occurred_at, "Ticket escalation occurred_at")
if len(self.metadata) > 100:
raise ValueError("Ticket escalation metadata is limited to 100 entries.")
@dataclass(frozen=True, slots=True)
class TicketCaseEscalationResult:
provider_id: str
case_id: str
case_number: str
case_url: str
replayed: bool = False
metadata: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
_required(self.provider_id, "Ticket escalation provider", 200)
_required(self.case_id, "Ticket escalation case", 255)
_required(self.case_number, "Ticket escalation case number", 255)
_relative_url(self.case_url)
if len(self.metadata) > 100:
raise ValueError("Ticket escalation metadata is limited to 100 entries.")
@runtime_checkable
class TicketCaseEscalationProvider(Protocol):
def escalate_ticket(
self,
session: object,
principal: object,
*,
command: TicketCaseEscalationCommand,
) -> TicketCaseEscalationResult: ...
def ticket_routing_provider(registry: object | None) -> TicketRoutingProvider | None:
provider = _capability(registry, CAPABILITY_TICKET_ROUTING)
return provider if isinstance(provider, TicketRoutingProvider) else None
def ticket_case_escalation_provider(
registry: object | None,
) -> TicketCaseEscalationProvider | None:
provider = _capability(registry, CAPABILITY_TICKET_CASE_ESCALATION)
return provider if isinstance(provider, TicketCaseEscalationProvider) else None
def _capability(registry: object | None, name: str) -> object | None:
if (
registry is None
or not hasattr(registry, "has_capability")
or not hasattr(registry, "capability")
or not registry.has_capability(name)
):
return None
return registry.capability(name)
def _required(value: str, label: str, maximum: int) -> None:
if not value.strip() or len(value) > maximum:
raise ValueError(f"{label} must contain 1 to {maximum} characters.")
def _optional(value: str | None, label: str, maximum: int) -> None:
if value is not None and (not value.strip() or len(value) > maximum):
raise ValueError(f"{label} must contain 1 to {maximum} characters when set.")
def _aware(value: datetime | None, label: str) -> None:
if value is not None and (value.tzinfo is None or value.utcoffset() is None):
raise ValueError(f"{label} must include a timezone.")
def _relative_url(value: str) -> None:
if (
not value.startswith("/")
or value.startswith("//")
or "\\" in value
or len(value) > 1_500
or any(ord(character) < 32 or ord(character) == 127 for character in value)
):
raise ValueError("Ticket escalation URLs must be bounded application-relative paths.")
__all__ = [
"CAPABILITY_TICKET_CASE_ESCALATION",
"CAPABILITY_TICKET_ROUTING",
"TICKET_INTEGRATION_CONTRACT_VERSION",
"TicketCaseEscalationCommand",
"TicketCaseEscalationProvider",
"TicketCaseEscalationResult",
"TicketRoutingPlan",
"TicketRoutingProvider",
"TicketRoutingRequest",
"ticket_case_escalation_provider",
"ticket_routing_provider",
]
+5 -2
View File
@@ -2,8 +2,8 @@ from __future__ import annotations
import re
from collections.abc import Iterable
from dataclasses import dataclass
from typing import Literal, Protocol, runtime_checkable
from dataclasses import dataclass, field
from typing import Literal, Mapping, Protocol, runtime_checkable
VIEWS_MODULE_ID = "views"
@@ -17,6 +17,8 @@ ViewSurfaceKind = Literal[
"section",
"action",
"selector",
"product_area",
"quick_access",
]
_SURFACE_ID_RE = re.compile(r"^[a-z][a-z0-9_.-]{2,159}$")
@@ -42,6 +44,7 @@ class EffectiveView:
revision_id: str | None
name: str | None
visible_surface_ids: frozenset[str]
presentation: Mapping[str, object] = field(default_factory=dict)
locked: bool = False
projection_active: bool = False
provenance: tuple[dict[str, object], ...] = ()
+1 -1
View File
@@ -73,7 +73,7 @@ def bootstrap_dev_data(
) -> BootstrapResult:
tenant = session.query(Tenant).filter(Tenant.slug == tenant_slug).one_or_none()
if tenant is None:
tenant = Tenant(slug=tenant_slug, name="Default Tenant", default_locale="en", settings={})
tenant = Tenant(slug=tenant_slug, name="Default Tenant", default_locale="de", settings={})
session.add(tenant)
session.flush()
+147
View File
@@ -0,0 +1,147 @@
"""Exact, bound-parameter JSON permission predicates for SQLite/PostgreSQL.
These primitives only match strings (never coerced numbers/booleans) and
objects inside actual arrays. Owners still define tenant, subject, permission,
purpose and current-state policy. Unsupported dialects fail at compilation.
"""
from __future__ import annotations
import re
from collections.abc import Mapping
from sqlalchemy import Boolean, literal
from sqlalchemy.exc import CompileError
from sqlalchemy.ext.compiler import compiles
from sqlalchemy.sql.functions import FunctionElement
class _ArrayString(FunctionElement):
type = Boolean()
inherit_cache = True
class _ObjectStrings(FunctionElement):
type = Boolean()
inherit_cache = True
class _ArrayObjectStrings(FunctionElement):
type = Boolean()
inherit_cache = True
def json_array_contains_string(column, value: str):
if type(value) is not str:
raise TypeError("JSON string membership requires a string value.")
return _ArrayString(column, literal(value))
def _field_arguments(fields: Mapping[str, str]):
if not isinstance(fields, Mapping) or not 1 <= len(fields) <= 16:
raise ValueError("JSON object matching requires between 1 and 16 string fields.")
arguments = []
for key, value in sorted(fields.items()):
if type(key) is not str or re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]{0,127}", key) is None:
raise ValueError("JSON object field names must be simple identifiers.")
if type(value) is not str:
raise TypeError("JSON object matching requires string values.")
arguments.extend((literal(key), literal(value)))
return arguments
def json_object_matches_strings(column, fields: Mapping[str, str]):
return _ObjectStrings(column, *_field_arguments(fields))
def json_array_contains_object_strings(column, fields: Mapping[str, str]):
return _ArrayObjectStrings(column, *_field_arguments(fields))
@compiles(_ArrayString)
@compiles(_ObjectStrings)
@compiles(_ArrayObjectStrings)
def _unsupported(element, compiler, **kwargs):
raise CompileError("Exact JSON permission predicates support only SQLite and PostgreSQL.")
def _parts(element, compiler, kwargs):
return [compiler.process(item, **kwargs) for item in element.clauses]
def _sqlite_array(value):
return f"CASE WHEN json_type({value}) = 'array' THEN {value} ELSE '[]' END"
def _postgres_array(value):
value = f"CAST({value} AS JSON)"
return f"CASE WHEN json_typeof({value}) = 'array' THEN {value} ELSE '[]'::json END"
def _sqlite_fields(value, fields):
terms = []
for index in range(0, len(fields), 2):
key, expected = fields[index:index + 2]
path = f"('$.' || {key})"
terms.extend((f"json_type({value}, {path}) = 'text'", f"json_extract({value}, {path}) = {expected}"))
return " AND ".join(terms)
def _postgres_fields(value, fields):
terms = []
for index in range(0, len(fields), 2):
key, expected = fields[index:index + 2]
terms.extend((f"json_typeof(({value}) -> {key}) = 'string'", f"(({value}) ->> {key}) = {expected}"))
return " AND ".join(terms)
@compiles(_ArrayString, "sqlite")
def _array_string_sqlite(element, compiler, **kwargs):
value, expected = _parts(element, compiler, kwargs)
return (
f"EXISTS (SELECT 1 FROM json_each({_sqlite_array(value)}) AS gp_json_string "
f"WHERE gp_json_string.type = 'text' AND gp_json_string.value = {expected})"
)
@compiles(_ArrayString, "postgresql")
def _array_string_postgres(element, compiler, **kwargs):
value, expected = _parts(element, compiler, kwargs)
return (
f"EXISTS (SELECT 1 FROM json_array_elements({_postgres_array(value)}) AS gp_json_string(value) "
f"WHERE json_typeof(gp_json_string.value) = 'string' "
f"AND (gp_json_string.value #>> '{{}}') = {expected})"
)
@compiles(_ObjectStrings, "sqlite")
def _object_sqlite(element, compiler, **kwargs):
value, *fields = _parts(element, compiler, kwargs)
value = f"CASE WHEN json_type({value}) = 'object' THEN {value} ELSE '{{}}' END"
return f"({_sqlite_fields(value, fields)})"
@compiles(_ObjectStrings, "postgresql")
def _object_postgres(element, compiler, **kwargs):
value, *fields = _parts(element, compiler, kwargs)
value = f"CAST({value} AS JSON)"
return f"(json_typeof({value}) = 'object' AND {_postgres_fields(value, fields)})"
@compiles(_ArrayObjectStrings, "sqlite")
def _array_object_sqlite(element, compiler, **kwargs):
value, *fields = _parts(element, compiler, kwargs)
item = "CASE WHEN gp_json_object.type = 'object' THEN gp_json_object.value ELSE '{}' END"
return (
f"EXISTS (SELECT 1 FROM json_each({_sqlite_array(value)}) AS gp_json_object "
f"WHERE {_sqlite_fields(item, fields)})"
)
@compiles(_ArrayObjectStrings, "postgresql")
def _array_object_postgres(element, compiler, **kwargs):
value, *fields = _parts(element, compiler, kwargs)
return (
f"EXISTS (SELECT 1 FROM json_array_elements({_postgres_array(value)}) AS gp_json_object(value) "
f"WHERE json_typeof(gp_json_object.value) = 'object' "
f"AND {_postgres_fields('gp_json_object.value', fields)})"
)
+4 -1
View File
@@ -685,7 +685,10 @@ def configured_migration_heads(
manifest_factories=manifest_factories,
migration_track=migration_track,
)
return tuple(sorted(ScriptDirectory.from_config(config).get_heads()))
scripts = ScriptDirectory.from_config(config)
return tuple(
sorted(revision.revision for revision in scripts.get_revisions("heads"))
)
def database_is_at_configured_heads(
+74
View File
@@ -0,0 +1,74 @@
from __future__ import annotations
from typing import Any
from sqlalchemy import or_
from govoplan_core.core.temporal import (
TemporalContextError,
TemporalDataContext,
current_temporal_data_context,
)
def apply_temporal_revision_filter(
query: Any,
model: type[Any],
*,
context: TemporalDataContext | None = None,
valid_from: str | None = "valid_from",
valid_to: str | None = "valid_to",
recorded_at: str | None = "recorded_at",
superseded_at: str | None = "superseded_at",
) -> Any:
"""Apply latest/as-recorded and valid-time clauses to a revision query."""
resolved = context or current_temporal_data_context()
clauses: list[Any] = []
superseded_column = _optional_column(model, superseded_at)
recorded_column = _optional_column(model, recorded_at)
if resolved.recorded_at is None:
if superseded_column is not None:
clauses.append(superseded_column.is_(None))
else:
if recorded_column is None or superseded_column is None:
raise TemporalContextError(
f"{model.__name__} does not expose recorded/superseded revision time."
)
clauses.extend(
(
recorded_column <= resolved.recorded_at,
or_(
superseded_column.is_(None),
superseded_column > resolved.recorded_at,
),
)
)
instant = resolved.validity_instant
if instant is not None:
valid_from_column = _optional_column(model, valid_from)
valid_to_column = _optional_column(model, valid_to)
if valid_from_column is not None:
clauses.append(
or_(valid_from_column.is_(None), valid_from_column <= instant)
)
if valid_to_column is not None:
clauses.append(or_(valid_to_column.is_(None), valid_to_column > instant))
return query.filter(*clauses) if clauses else query
def _optional_column(model: type[Any], name: str | None) -> Any | None:
if name is None:
return None
column = getattr(model, name, None)
if column is None:
raise TemporalContextError(
f"{model.__name__} has no temporal column named {name!r}."
)
return column
__all__ = ["apply_temporal_revision_filter"]
+14 -7
View File
@@ -4,10 +4,12 @@ import re
from typing import Any, Iterable
I18N_SETTINGS_KEY = "i18n"
REFERENCE_LANGUAGE_CODE = "de"
SOURCE_LANGUAGE_CODE = "en"
DEFAULT_LANGUAGE_PACKAGES: tuple[dict[str, str], ...] = (
{"code": "en", "label": "English", "native_label": "English"},
{"code": "de", "label": "German", "native_label": "Deutsch"},
{"code": "en", "label": "English", "native_label": "English"},
)
@@ -66,7 +68,7 @@ def normalize_enabled_language_codes(
available_languages: Iterable[dict[str, Any]],
*,
default_locale: object = None,
fallback_codes: Iterable[str] = ("en", "de"),
fallback_codes: Iterable[str] = (REFERENCE_LANGUAGE_CODE, SOURCE_LANGUAGE_CODE),
) -> list[str]:
available_codes = [normalize_language_code(item.get("code")) for item in available_languages if isinstance(item, dict)]
available = {code for code in available_codes if code}
@@ -88,8 +90,10 @@ def normalize_enabled_language_codes(
if enabled:
return enabled
if "en" in available:
return ["en"]
if REFERENCE_LANGUAGE_CODE in available:
return [REFERENCE_LANGUAGE_CODE]
if SOURCE_LANGUAGE_CODE in available:
return [SOURCE_LANGUAGE_CODE]
return available_codes[:1]
@@ -126,7 +130,7 @@ def preferred_language_code(user_settings: dict[str, Any] | None, allowed_codes:
default = resolve_language_code(default_locale, enabled)
if default:
return default
return enabled[0] if enabled else "en"
return enabled[0] if enabled else REFERENCE_LANGUAGE_CODE
def update_i18n_settings(settings: dict[str, Any] | None, **values: Any) -> dict[str, Any]:
@@ -141,10 +145,13 @@ def system_i18n_payload(settings_item: Any | None) -> dict[str, object]:
settings = getattr(settings_item, "settings", None)
packages = system_language_packages(settings)
available_codes = [item["code"] for item in packages]
default_language = resolve_language_code(getattr(settings_item, "default_locale", None), available_codes) or "en"
default_language = (
resolve_language_code(getattr(settings_item, "default_locale", None), available_codes)
or REFERENCE_LANGUAGE_CODE
)
enabled = system_enabled_language_codes(settings, default_locale=default_language)
if default_language not in enabled:
default_language = enabled[0] if enabled else "en"
default_language = enabled[0] if enabled else REFERENCE_LANGUAGE_CODE
return {
"available_languages": packages,
"enabled_languages": enabled,
+7 -2
View File
@@ -1,3 +1,8 @@
from govoplan_core.mail.config import ImapConfig, SmtpConfig, TransportSecurity
from govoplan_core.mail.config import (
ImapConfig,
ImapFolderMappings,
SmtpConfig,
TransportSecurity,
)
__all__ = ["ImapConfig", "SmtpConfig", "TransportSecurity"]
__all__ = ["ImapConfig", "ImapFolderMappings", "SmtpConfig", "TransportSecurity"]
+43 -3
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
from enum import StrEnum
from typing import Any
from pydantic import BaseModel, ConfigDict, Field, model_validator
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
class StrictModel(BaseModel):
@@ -34,21 +34,61 @@ class SmtpServerConfig(StrictModel):
return self
class ImapFolderMappings(StrictModel):
"""Profile-level names for the standard IMAP mailbox roles."""
inbox: str | None = None
sent: str | None = None
drafts: str | None = None
trash: str | None = None
archive: str | None = None
junk: str | None = None
@field_validator("*", mode="before")
@classmethod
def normalize_folder_name(cls, value: Any) -> Any:
if value is None:
return None
normalized = str(value).strip()
return normalized or None
class ImapServerConfig(StrictModel):
host: str | None = None
port: int | None = Field(default=None, ge=1, le=65535)
security: TransportSecurity = TransportSecurity.TLS
sent_folder: str = "auto"
folder_mappings: ImapFolderMappings | None = None
timeout_seconds: int = Field(default=30, ge=1)
@model_validator(mode="before")
@classmethod
def discard_legacy_enabled(cls, value: Any) -> Any:
if isinstance(value, dict) and "enabled" in value:
if not isinstance(value, dict):
return value
data = dict(value)
data.pop("enabled", None)
mappings_value = data.get("folder_mappings")
mappings = (
mappings_value.model_dump(exclude_none=True)
if isinstance(mappings_value, ImapFolderMappings)
else dict(mappings_value)
if isinstance(mappings_value, dict)
else {}
)
mapped_sent = str(mappings.get("sent") or "").strip()
legacy_sent = str(data.get("sent_folder") or "").strip()
if mapped_sent:
# The typed mapping is canonical when both new and legacy callers
# provide a Sent value. Keep the legacy field synchronized for
# existing Campaign append consumers.
data["sent_folder"] = mapped_sent
elif legacy_sent and legacy_sent != "auto":
data["sent_folder"] = legacy_sent
mappings["sent"] = legacy_sent
if mappings:
data["folder_mappings"] = mappings
return data
return value
@model_validator(mode="after")
def apply_default_port(self) -> "ImapServerConfig":
+753
View File
@@ -0,0 +1,753 @@
from __future__ import annotations
import hashlib
import json
import uuid
from collections import defaultdict
from collections.abc import Iterable, Mapping, Sequence
from datetime import datetime, timezone
from typing import Any
from sqlalchemy import DateTime, Index, Integer, JSON, String, Text
from sqlalchemy.orm import Mapped, Session, mapped_column
from govoplan_core.core.concurrency import RevisionConflictError, strong_resource_etag
from govoplan_core.core.dsar import (
DsarErasureActionRef,
DsarExecutionResultRef,
DsarProvider,
DsarRecordRef,
DsarRequestKind,
DsarSubjectRef,
dsar_provider_names,
)
from govoplan_core.db.base import Base, TimestampMixin
MAX_RECORDS_PER_PROVIDER = 10_000
MAX_PROVIDER_RESULT_BYTES = 10 * 1024 * 1024
class DataSubjectRequest(Base, TimestampMixin):
__tablename__ = "core_data_subject_requests"
__table_args__ = (
Index(
"ix_core_data_subject_requests_tenant_status",
"tenant_id",
"status",
),
)
id: Mapped[str] = mapped_column(
String(36), primary_key=True, default=lambda: str(uuid.uuid4())
)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
reference: Mapped[str] = mapped_column(String(120), nullable=False)
request_kind: Mapped[str] = mapped_column(String(30), nullable=False)
status: Mapped[str] = mapped_column(
String(30), default="draft", nullable=False, index=True
)
subject: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
purpose: Mapped[str] = mapped_column(String(1000), nullable=False)
legal_basis: Mapped[str | None] = mapped_column(String(1000), nullable=True)
due_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, index=True
)
requested_by_account_id: Mapped[str] = mapped_column(String(36), nullable=False)
search_result: Mapped[dict[str, Any]] = mapped_column(
JSON, default=dict, nullable=False
)
erasure_plan: Mapped[dict[str, Any]] = mapped_column(
JSON, default=dict, nullable=False
)
execution_result: Mapped[dict[str, Any]] = mapped_column(
JSON, default=dict, nullable=False
)
coverage: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
evidence_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
resource_revision: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
completed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
notes: Mapped[str | None] = mapped_column(Text, nullable=True)
@property
def strong_etag(self) -> str:
return strong_resource_etag(
"data_subject_request",
self.id,
self.resource_revision,
)
def create_data_subject_request(
session: Session,
*,
tenant_id: str,
reference: str,
request_kind: DsarRequestKind,
subject: DsarSubjectRef,
purpose: str,
legal_basis: str | None,
due_at: datetime | None,
requested_by_account_id: str,
notes: str | None = None,
) -> DataSubjectRequest:
if not subject.has_selector():
raise ValueError("At least one data-subject selector is required.")
row = DataSubjectRequest(
tenant_id=tenant_id,
reference=reference.strip(),
request_kind=request_kind,
subject=subject.to_dict(),
purpose=purpose.strip(),
legal_basis=(legal_basis or "").strip() or None,
due_at=due_at,
requested_by_account_id=requested_by_account_id,
notes=(notes or "").strip() or None,
)
session.add(row)
session.flush()
row.evidence_sha256 = _evidence_digest(row)
return row
def get_data_subject_request(
session: Session,
*,
tenant_id: str,
request_id: str,
for_update: bool = False,
) -> DataSubjectRequest:
query = session.query(DataSubjectRequest).filter(
DataSubjectRequest.id == request_id,
DataSubjectRequest.tenant_id == tenant_id,
)
if for_update:
query = query.with_for_update()
row = query.one_or_none()
if row is None or row.tenant_id != tenant_id:
raise LookupError("Data-subject request not found.")
return row
def list_data_subject_requests(
session: Session,
*,
tenant_id: str,
limit: int = 200,
) -> tuple[DataSubjectRequest, ...]:
return tuple(
session.query(DataSubjectRequest)
.filter(DataSubjectRequest.tenant_id == tenant_id)
.order_by(
DataSubjectRequest.created_at.desc(),
DataSubjectRequest.id.desc(),
)
.limit(max(1, min(limit, 500)))
.all()
)
def search_data_subject_request(
session: Session,
*,
registry: object,
row: DataSubjectRequest,
expected_revision: int,
) -> DataSubjectRequest:
_assert_revision(row, expected_revision)
subject = _subject(row.subject)
records: list[dict[str, object]] = []
provider_runs: list[dict[str, object]] = []
providers, discovery = _providers(
registry,
session,
tenant_id=row.tenant_id,
)
for capability_name, provider in providers:
started_at = _now()
try:
with session.begin_nested():
provider_records = tuple(
provider.search_subject(
session,
tenant_id=row.tenant_id,
subject=subject,
)
)
_validate_records(provider, provider_records)
if len(provider_records) > MAX_RECORDS_PER_PROVIDER:
raise ValueError("DSAR provider result exceeds the record limit.")
encoded = _json_bytes([item.to_dict() for item in provider_records])
if len(encoded) > MAX_PROVIDER_RESULT_BYTES:
raise ValueError("DSAR provider result exceeds the payload limit.")
records.extend(item.to_dict() for item in provider_records)
provider_runs.append(
{
"capability": capability_name,
"provider_id": provider.provider_id,
"module_id": provider.module_id,
"status": "complete",
"record_count": len(provider_records),
"started_at": started_at,
"completed_at": _now(),
}
)
except Exception as exc:
provider_runs.append(
{
"capability": capability_name,
"provider_id": getattr(provider, "provider_id", capability_name),
"module_id": getattr(provider, "module_id", "unknown"),
"status": "failed",
"record_count": 0,
"error": _safe_error(exc),
"started_at": started_at,
"completed_at": _now(),
}
)
failures = sum(1 for item in provider_runs if item["status"] == "failed")
row.search_result = {
"schema": "govoplan.dsars.search.v1",
"searched_at": _now(),
"records": records,
"provider_runs": provider_runs,
"record_count": len(records),
}
row.coverage = discovery
row.erasure_plan = {}
row.execution_result = {}
row.status = "search_partial" if failures else "searched"
row.completed_at = None
_advance(row)
return row
def plan_data_subject_erasure(
session: Session,
*,
registry: object,
row: DataSubjectRequest,
expected_revision: int,
) -> DataSubjectRequest:
_assert_revision(row, expected_revision)
if not row.search_result.get("searched_at"):
raise ValueError("Run the data-subject search before planning erasure.")
if row.request_kind == "access":
raise ValueError("This request does not include erasure.")
subject = _subject(row.subject)
records = tuple(_record(item) for item in row.search_result.get("records", []))
by_provider: dict[str, list[DsarRecordRef]] = defaultdict(list)
for record in records:
by_provider[record.provider_id].append(record)
providers, discovery = _providers(
registry,
session,
tenant_id=row.tenant_id,
)
actions: list[DsarErasureActionRef] = []
provider_runs: list[dict[str, object]] = []
for capability_name, provider in providers:
started_at = _now()
try:
with session.begin_nested():
proposed = tuple(
provider.plan_erasure(
session,
tenant_id=row.tenant_id,
subject=subject,
records=tuple(by_provider.get(provider.provider_id, ())),
)
)
_validate_actions(provider, proposed)
actions.extend(proposed)
provider_runs.append(
{
"capability": capability_name,
"provider_id": provider.provider_id,
"module_id": provider.module_id,
"status": "complete",
"action_count": len(proposed),
"started_at": started_at,
"completed_at": _now(),
}
)
except Exception as exc:
provider_runs.append(
{
"capability": capability_name,
"provider_id": getattr(provider, "provider_id", capability_name),
"module_id": getattr(provider, "module_id", "unknown"),
"status": "failed",
"action_count": 0,
"error": _safe_error(exc),
"started_at": started_at,
"completed_at": _now(),
}
)
existing_resources = {
(item.provider_id, item.resource_type, item.resource_id) for item in actions
}
for record in records:
key = (record.provider_id, record.resource_type, record.resource_id)
if not record.immutable_evidence or key in existing_resources:
continue
digest = hashlib.sha256("\0".join(key).encode("utf-8")).hexdigest()[:24]
actions.append(
DsarErasureActionRef(
action_id=f"retain:{digest}",
provider_id=record.provider_id,
module_id=record.module_id,
kind="retain",
resource_type=record.resource_type,
resource_id=record.resource_id,
title=f"Retain {record.title}",
rationale=(
record.retention_reason
or "Immutable institutional evidence must be retained."
),
executable=False,
)
)
duplicate_ids = _duplicates(item.action_id for item in actions)
if duplicate_ids:
raise ValueError(
f"DSAR providers returned duplicate action ids: {', '.join(duplicate_ids)}"
)
failures = sum(1 for item in provider_runs if item["status"] == "failed")
row.erasure_plan = {
"schema": "govoplan.dsars.erasure-plan.v1",
"planned_at": _now(),
"actions": [item.to_dict() for item in actions],
"provider_runs": provider_runs,
"executable_count": sum(1 for item in actions if item.executable),
"retained_count": sum(1 for item in actions if item.kind == "retain"),
}
row.coverage = {**discovery, "plan_provider_runs": provider_runs}
row.execution_result = {}
row.status = "plan_partial" if failures else "plan_ready"
row.completed_at = None
_advance(row)
return row
def execute_data_subject_erasure(
session: Session,
*,
registry: object,
row: DataSubjectRequest,
expected_revision: int,
action_ids: Sequence[str],
) -> DataSubjectRequest:
_assert_revision(row, expected_revision)
raw_actions = row.erasure_plan.get("actions")
if not isinstance(raw_actions, list):
raise ValueError("Create an erasure plan before execution.")
actions_by_id = {
action.action_id: action for action in (_action(item) for item in raw_actions)
}
selected_ids = tuple(dict.fromkeys(str(item) for item in action_ids if str(item)))
if not selected_ids:
raise ValueError("Select at least one executable erasure action.")
missing = [item for item in selected_ids if item not in actions_by_id]
if missing:
raise ValueError("The erasure plan changed; refresh before executing it.")
selected = tuple(actions_by_id[item] for item in selected_ids)
blocked = [item.action_id for item in selected if not item.executable]
if blocked:
raise ValueError("Retained or review-only actions cannot be executed.")
providers, _discovery = _providers(
registry,
session,
tenant_id=row.tenant_id,
)
providers_by_id = {provider.provider_id: provider for _name, provider in providers}
grouped: dict[str, list[DsarErasureActionRef]] = defaultdict(list)
for action in selected:
grouped[action.provider_id].append(action)
previous = {
str(item.get("action_id")): item
for item in row.execution_result.get("results", [])
if isinstance(item, Mapping)
and item.get("status") in {"executed", "unchanged"}
}
results: list[dict[str, object]] = list(previous.values())
for provider_id, provider_actions in grouped.items():
provider = providers_by_id.get(provider_id)
if provider is None:
results.extend(
DsarExecutionResultRef(
action_id=action.action_id,
status="blocked",
summary="The owning DSAR provider is not currently available.",
).to_dict()
for action in provider_actions
)
continue
pending = tuple(
action for action in provider_actions if action.action_id not in previous
)
if not pending:
continue
try:
with session.begin_nested():
executed = tuple(
provider.execute_erasure(
session,
tenant_id=row.tenant_id,
subject=_subject(row.subject),
actions=pending,
request_id=row.id,
)
)
_validate_execution_results(pending, executed)
results.extend(item.to_dict() for item in executed)
except Exception as exc:
results.extend(
DsarExecutionResultRef(
action_id=action.action_id,
status="failed",
summary=_safe_error(exc),
).to_dict()
for action in pending
)
result_by_id = {str(item["action_id"]): item for item in results}
all_executable = {
action.action_id for action in actions_by_id.values() if action.executable
}
successful = {
action_id
for action_id, item in result_by_id.items()
if item.get("status") in {"executed", "unchanged"}
}
failed = {
action_id
for action_id, item in result_by_id.items()
if item.get("status") in {"failed", "blocked"}
}
row.execution_result = {
"schema": "govoplan.dsars.execution.v1",
"executed_at": _now(),
"results": list(result_by_id.values()),
"successful_count": len(successful),
"failed_count": len(failed),
}
if all_executable.issubset(successful):
row.status = "completed"
row.completed_at = datetime.now(timezone.utc)
elif failed:
row.status = "execution_partial"
row.completed_at = None
else:
row.status = "execution_pending"
row.completed_at = None
_advance(row)
return row
def data_subject_export(row: DataSubjectRequest) -> bytes:
payload: dict[str, object] = {
"schema": "govoplan.dsars.export.v1",
"generated_at": _now(),
"request": data_subject_request_dict(row, include_subject=True),
"search": row.search_result,
"erasure_plan": row.erasure_plan,
"execution": row.execution_result,
"coverage": row.coverage,
"limitations": [
"Only providers listed as complete contributed data.",
"Retained immutable evidence is exported with its stated retention reason.",
"An export does not itself erase or alter source data.",
],
}
digest = hashlib.sha256(_json_bytes(payload)).hexdigest()
payload["manifest_sha256"] = digest
return _json_bytes(payload, pretty=True)
def data_subject_request_dict(
row: DataSubjectRequest,
*,
include_subject: bool,
) -> dict[str, object]:
return {
"id": row.id,
"tenant_id": row.tenant_id,
"reference": row.reference,
"request_kind": row.request_kind,
"status": row.status,
"subject": dict(row.subject) if include_subject else {},
"purpose": row.purpose,
"legal_basis": row.legal_basis,
"due_at": row.due_at.isoformat() if row.due_at else None,
"requested_by_account_id": row.requested_by_account_id,
"record_count": int(row.search_result.get("record_count", 0)),
"executable_action_count": int(
row.erasure_plan.get("executable_count", 0)
),
"coverage": dict(row.coverage),
"evidence_sha256": row.evidence_sha256,
"resource_revision": row.resource_revision,
"etag": row.strong_etag,
"created_at": row.created_at.isoformat(),
"updated_at": row.updated_at.isoformat(),
"completed_at": row.completed_at.isoformat() if row.completed_at else None,
"notes": row.notes,
}
def _providers(
registry: object,
session: Session,
*,
tenant_id: str,
) -> tuple[tuple[tuple[str, DsarProvider], ...], dict[str, object]]:
values: list[tuple[str, DsarProvider]] = []
failures: list[dict[str, str]] = []
active_modules = _effective_module_ids(registry, session, tenant_id=tenant_id)
names = dsar_provider_names(registry)
active_names: list[str] = []
inactive_names: list[str] = []
for name in names:
owner = (
registry.capability_owner(name)
if hasattr(registry, "capability_owner")
else None
)
if owner and owner not in active_modules:
inactive_names.append(name)
continue
active_names.append(name)
try:
if hasattr(registry, "require_tenant_capability"):
candidate = registry.require_tenant_capability(
name,
session,
tenant_id=tenant_id,
work_state="interactive",
)
else:
candidate = registry.require_capability(name)
if not isinstance(candidate, DsarProvider):
raise TypeError("Capability does not implement DsarProvider.")
values.append((name, candidate))
except Exception as exc:
failures.append({"capability": name, "error": _safe_error(exc)})
covered_modules = sorted({provider.module_id for _name, provider in values})
return tuple(values), {
"provider_capabilities": active_names,
"inactive_provider_capabilities": inactive_names,
"covered_modules": covered_modules,
"modules_without_provider": sorted(set(active_modules) - set(covered_modules)),
"provider_discovery_failures": failures,
}
def _effective_module_ids(
registry: object,
session: Session,
*,
tenant_id: str,
) -> list[str]:
resolver_factory = getattr(registry, "tenant_entitlement_resolver", None)
if callable(resolver_factory):
state = resolver_factory().resolve(session, tenant_id)
return sorted(str(item) for item in state.effective_modules)
if hasattr(registry, "manifests"):
return sorted(str(manifest.id) for manifest in registry.manifests())
return []
def _subject(value: Mapping[str, object]) -> DsarSubjectRef:
raw_refs = value.get("external_references")
return DsarSubjectRef(
account_id=_optional_text(value.get("account_id")),
identity_id=_optional_text(value.get("identity_id")),
membership_id=_optional_text(value.get("membership_id")),
email=_optional_text(value.get("email")),
external_references={
str(key): str(item)
for key, item in (raw_refs.items() if isinstance(raw_refs, Mapping) else ())
},
)
def _record(value: object) -> DsarRecordRef:
if not isinstance(value, Mapping):
raise ValueError("Stored DSAR record is invalid.")
observed_at = _parse_datetime(value.get("observed_at"))
data = value.get("data")
return DsarRecordRef(
provider_id=str(value.get("provider_id") or ""),
module_id=str(value.get("module_id") or ""),
resource_type=str(value.get("resource_type") or ""),
resource_id=str(value.get("resource_id") or ""),
category=str(value.get("category") or ""),
title=str(value.get("title") or ""),
data=dict(data) if isinstance(data, Mapping) else {},
observed_at=observed_at,
immutable_evidence=bool(value.get("immutable_evidence")),
retention_reason=_optional_text(value.get("retention_reason")),
source_path=_optional_text(value.get("source_path")),
)
def _action(value: object) -> DsarErasureActionRef:
if not isinstance(value, Mapping):
raise ValueError("Stored DSAR action is invalid.")
metadata = value.get("metadata")
kind = str(value.get("kind") or "manual_review")
if kind not in {
"delete",
"anonymize",
"revoke",
"detach",
"retain",
"manual_review",
}:
raise ValueError("Stored DSAR action kind is invalid.")
return DsarErasureActionRef(
action_id=str(value.get("action_id") or ""),
provider_id=str(value.get("provider_id") or ""),
module_id=str(value.get("module_id") or ""),
kind=kind, # type: ignore[arg-type]
resource_type=str(value.get("resource_type") or ""),
resource_id=str(value.get("resource_id") or ""),
title=str(value.get("title") or ""),
rationale=str(value.get("rationale") or ""),
executable=bool(value.get("executable")),
irreversible=bool(value.get("irreversible")),
metadata=dict(metadata) if isinstance(metadata, Mapping) else {},
)
def _validate_records(
provider: DsarProvider,
records: Sequence[DsarRecordRef],
) -> None:
for record in records:
if record.provider_id != provider.provider_id or record.module_id != provider.module_id:
raise ValueError("DSAR record ownership does not match its provider.")
if not record.resource_type or not record.resource_id or not record.title:
raise ValueError("DSAR records require resource identity and title.")
def _validate_actions(
provider: DsarProvider,
actions: Sequence[DsarErasureActionRef],
) -> None:
for action in actions:
if action.provider_id != provider.provider_id or action.module_id != provider.module_id:
raise ValueError("DSAR action ownership does not match its provider.")
if not action.action_id or not action.resource_type or not action.resource_id:
raise ValueError("DSAR actions require stable identities.")
if action.kind in {"retain", "manual_review"} and action.executable:
raise ValueError("Retain and manual-review actions cannot be executable.")
def _validate_execution_results(
actions: Sequence[DsarErasureActionRef],
results: Sequence[DsarExecutionResultRef],
) -> None:
expected = {action.action_id for action in actions}
returned = {result.action_id for result in results}
if expected != returned or len(returned) != len(results):
raise ValueError("DSAR provider did not return one result per action.")
def _assert_revision(row: DataSubjectRequest, expected_revision: int) -> None:
if row.resource_revision != expected_revision:
raise RevisionConflictError(
resource_type="data_subject_request",
resource_id=row.id,
current_revision=row.resource_revision,
submitted_base_revision=expected_revision,
current_etag=row.strong_etag,
refresh_path=f"/api/v1/admin/privacy/data-subject-requests/{row.id}",
)
def _advance(row: DataSubjectRequest) -> None:
row.resource_revision += 1
row.evidence_sha256 = _evidence_digest(row)
def _evidence_digest(row: DataSubjectRequest) -> str:
return hashlib.sha256(
_json_bytes(
{
"id": row.id,
"tenant_id": row.tenant_id,
"reference": row.reference,
"request_kind": row.request_kind,
"status": row.status,
"subject": row.subject,
"search_result": row.search_result,
"erasure_plan": row.erasure_plan,
"execution_result": row.execution_result,
"coverage": row.coverage,
"resource_revision": row.resource_revision,
}
)
).hexdigest()
def _duplicates(values: Iterable[str]) -> tuple[str, ...]:
seen: set[str] = set()
duplicate: set[str] = set()
for value in values:
if value in seen:
duplicate.add(value)
seen.add(value)
return tuple(sorted(duplicate))
def _json_bytes(value: object, *, pretty: bool = False) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
indent=2 if pretty else None,
separators=None if pretty else (",", ":"),
default=str,
).encode("utf-8")
def _safe_error(exc: Exception) -> str:
text = " ".join(str(exc).split())
return (text or exc.__class__.__name__)[:1000]
def _optional_text(value: object | None) -> str | None:
text = str(value or "").strip()
return text or None
def _parse_datetime(value: object | None) -> datetime | None:
if isinstance(value, datetime):
return value
if isinstance(value, str) and value:
try:
return datetime.fromisoformat(value)
except ValueError:
return None
return None
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
__all__ = [
"DataSubjectRequest",
"create_data_subject_request",
"data_subject_export",
"data_subject_request_dict",
"execute_data_subject_erasure",
"get_data_subject_request",
"list_data_subject_requests",
"plan_data_subject_erasure",
"search_data_subject_request",
]
@@ -0,0 +1,13 @@
{
"generated_at": "2026-07-11T15:18:35.649400Z",
"keyring_version": "1",
"keys": [
{
"key_id": "release-key-1",
"not_before": "2026-07-11T00:00:00Z",
"public_key": "jOXIlZXytoNJCH8tsmrYRklg6ShpjGXRY0uV3jApRiA=",
"status": "active"
}
],
"purpose": "govoplan module package catalog signatures"
}
@@ -0,0 +1,285 @@
"""Resource-bounded, disposable workers for trusted module-owned byte operations.
This is not an arbitrary-code sandbox. Callers supply a server-owned top-level
function, never a client-selected module/path/callable. Sessions, credentials and
authority remain in the parent; only explicit bounded bytes cross the pipe.
"""
from __future__ import annotations
from collections.abc import Callable
from contextlib import contextmanager
from dataclasses import dataclass
import inspect
import math
import os
from pathlib import Path
import selectors
import signal
import subprocess
import sys
import threading
import time
@dataclass(frozen=True, slots=True)
class ProcessLimits:
wall_seconds: float = 10.0
cpu_seconds: int = 10
memory_bytes: int = 256 * 1024 * 1024
input_bytes: int = 8 * 1024 * 1024
output_bytes: int = 8 * 1024 * 1024
file_bytes: int = 0
def __post_init__(self) -> None:
if not math.isfinite(self.wall_seconds) or not 0 < self.wall_seconds <= 600:
raise ValueError("Worker wall time must be finite and within (0, 600] seconds.")
for name, minimum, maximum in (
("cpu_seconds", 1, 600),
("memory_bytes", 64 * 1024 * 1024, 8 * 1024 * 1024 * 1024),
("input_bytes", 1, 256 * 1024 * 1024),
("output_bytes", 1, 256 * 1024 * 1024),
("file_bytes", 0, 2 * 1024 * 1024 * 1024),
):
value = getattr(self, name)
if type(value) is not int or not minimum <= value <= maximum:
raise ValueError(f"Invalid worker {name} limit.")
class ProcessBudgetError(RuntimeError):
def __init__(self, code: str) -> None:
messages = {
"busy": "The isolated-work capacity is busy; retry later.",
"cancelled": "Isolated work was cancelled.",
"timeout": "Isolated work exceeded its wall-clock limit.",
"cpu_limit": "Isolated work exceeded its CPU limit.",
"memory_limit": "Isolated work exceeded its memory limit.",
"input_limit": "Isolated work input exceeded its byte limit.",
"output_limit": "Isolated work output exceeded its byte limit.",
"unavailable": "Required isolated-worker resource controls are unavailable.",
"worker_failed": "Isolated work could not complete safely.",
}
self.code = code
super().__init__(messages[code])
_gate = threading.Lock()
_active = 0
_STDERR_LIMIT = 64 * 1024
def _reset_after_fork() -> None:
global _gate, _active
_gate, _active = threading.Lock(), 0
if hasattr(os, "register_at_fork"):
os.register_at_fork(after_in_child=_reset_after_fork)
def _reserve() -> None:
from govoplan_core.settings import settings
global _active
with _gate:
if _active >= settings.isolated_process_concurrency:
raise ProcessBudgetError("busy")
_active += 1
def _release() -> None:
global _active
with _gate:
_active -= 1
@dataclass(slots=True, eq=False)
class _OperationAdmission:
process_id: int
thread_id: int
active: bool = True
running: bool = False
@contextmanager
def bounded_operation_admission():
"""Reserve shared capacity before bounded parent-side input preparation.
The yielded token may be reused sequentially in this thread only. Never
expose it to clients or hold it while waiting for user input/network work.
Parent preparation still needs its own byte/time/disk bounds.
"""
_reserve()
admission = _OperationAdmission(os.getpid(), threading.get_ident())
try:
yield admission
finally:
admission.active = False
# Fork resets the child's admission counter. An inherited context must
# still expire its token, but cannot release the parent's reservation.
if admission.process_id == os.getpid():
_release()
def _enter_admission(admission: _OperationAdmission) -> None:
with _gate:
if (
not isinstance(admission, _OperationAdmission)
or not admission.active or admission.running
or admission.process_id != os.getpid()
or admission.thread_id != threading.get_ident()
):
raise ValueError("Worker admission must be active, unused and owned by this thread/process.")
admission.running = True
def run_bounded_operation(
operation: Callable[[bytes], bytes],
payload: bytes,
*,
limits: ProcessLimits = ProcessLimits(),
cancelled: Callable[[], bool] | None = None,
admission: _OperationAdmission | None = None,
) -> bytes:
"""Run a trusted, importable module function without inheriting parent state.
Admission is non-queuing and per API/worker process. POSIX process groups,
resource limits and waitid(WNOWAIT) are required; no in-process fallback.
Output and stderr are drained incrementally, including while input is sent.
Every exit path kills the owned process group before reaping its leader.
"""
if type(payload) is not bytes or len(payload) > limits.input_bytes:
raise ProcessBudgetError("input_limit")
if os.name != "posix" or not hasattr(os, "WNOWAIT") or not hasattr(os, "waitid"):
raise ProcessBudgetError("unavailable")
module = inspect.getmodule(operation)
name = getattr(operation, "__name__", "")
if (
module is None or not name.isidentifier()
or getattr(module, name, None) is not operation
or not inspect.isfunction(operation) or not getattr(module, "__file__", None)
):
raise ValueError("Isolated operations must be server-owned module-level functions.")
module_name = module.__name__
if not all(part.isidentifier() for part in module_name.split(".")):
raise ValueError("Invalid isolated operation module.")
# Explicit source identity supports installed modules and editable development
# checkouts without inheriting arbitrary PYTHONPATH or the parent's cwd.
source = Path(module.__file__).resolve()
source_root = source.parents[len(module_name.split(".")) - 1]
if source.name == "__init__.py":
source_root = source_root.parent
_check_cancelled(cancelled)
if admission is None:
with bounded_operation_admission() as reserved:
return run_bounded_operation(
operation, payload, limits=limits, cancelled=cancelled, admission=reserved,
)
_enter_admission(admission)
try:
return _run(module_name, name, source_root, payload, limits, cancelled)
finally:
admission.running = False
def _check_cancelled(cancelled: Callable[[], bool] | None) -> None:
if cancelled is not None and cancelled():
raise ProcessBudgetError("cancelled")
def _run(
module: str, name: str, source_root: Path, payload: bytes,
limits: ProcessLimits, cancelled: Callable[[], bool] | None,
) -> bytes:
command = [
sys.executable, "-I", "-B", "-m", "govoplan_core.security.process_worker",
module, name, str(source_root), str(limits.cpu_seconds),
str(limits.memory_bytes), str(limits.input_bytes), str(limits.output_bytes),
str(limits.file_bytes),
]
deadline = time.monotonic() + limits.wall_seconds
try:
process = subprocess.Popen( # noqa: S603 - fixed interpreter/bootstrap, trusted operation.
command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
bufsize=0, close_fds=True, start_new_session=True, cwd="/",
env={
"LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", "TZ": "UTC",
"OPENBLAS_NUM_THREADS": "1", "OMP_NUM_THREADS": "1",
"MKL_NUM_THREADS": "1", "NUMEXPR_NUM_THREADS": "1",
},
)
except OSError as exc:
raise ProcessBudgetError("unavailable") from exc
output = bytearray()
sent = 0
stderr_bytes = 0
try:
with selectors.DefaultSelector() as selector:
for stream in (process.stdin, process.stdout, process.stderr):
os.set_blocking(stream.fileno(), False)
selector.register(process.stdout, selectors.EVENT_READ, "stdout")
selector.register(process.stderr, selectors.EVENT_READ, "stderr")
if payload:
selector.register(process.stdin, selectors.EVENT_WRITE, "stdin")
else:
process.stdin.close()
while True:
_check_cancelled(cancelled)
remaining = deadline - time.monotonic()
if remaining <= 0:
raise ProcessBudgetError("timeout")
# WNOWAIT keeps the owned group leader's PID reserved until the
# group is killed, including descendants that close their pipes.
exited = os.waitid(os.P_PID, process.pid, os.WEXITED | os.WNOHANG | os.WNOWAIT)
if exited is not None and not selector.get_map():
break
events = selector.select(min(remaining, 0.05))
for key, _event in events:
if key.data == "stdin":
try:
sent += os.write(key.fd, memoryview(payload)[sent:sent + 65536])
except BrokenPipeError:
sent = len(payload)
except BlockingIOError:
continue
if sent == len(payload):
selector.unregister(key.fileobj)
key.fileobj.close()
continue
available = (
limits.output_bytes - len(output)
if key.data == "stdout" else _STDERR_LIMIT - stderr_bytes
)
try:
chunk = os.read(key.fd, min(65536, available + 1))
except BlockingIOError:
continue
if not chunk:
selector.unregister(key.fileobj)
elif len(chunk) > available:
raise ProcessBudgetError("output_limit")
elif key.data == "stdout":
output.extend(chunk)
else:
# Never expose raw exception/log output to a client.
stderr_bytes += len(chunk)
exit_code = os.waitstatus_to_exitcode(
(exited.si_status << 8) if exited.si_code == os.CLD_EXITED else exited.si_status
)
finally:
# Do not communicate(): it would collect unbounded output during cleanup.
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
pass
finally:
for stream in (process.stdin, process.stdout, process.stderr):
stream.close()
process.wait()
if exit_code != 0:
code = {
71: "memory_limit", 72: "input_limit", 73: "output_limit",
74: "unavailable", -signal.SIGXCPU: "cpu_limit",
-signal.SIGXFSZ: "output_limit",
}.get(exit_code, "worker_failed")
raise ProcessBudgetError(code)
return bytes(output)

Some files were not shown because too many files have changed in this diff Show More