govoplan-core/docs/EVENTS_AND_AUDIT.md

Events And Audit

GovOPlaN uses a small kernel event contract first, not a broad command bus. Commands remain module-owned application-service methods or API endpoints until there is a concrete need for durable asynchronous command orchestration. Events are facts about completed work and are safe for audit, projections, optional module reactions, and operator diagnostics.

Production Transport Decision

The production transport is a transactional database outbox plus retrying dispatcher:

• Use govoplan_core.core.events.PlatformEvent for domain and platform events.
• Call emit_platform_event(session, event) to bind event delivery to the domain transaction.
• The optional platform.eventOutbox capability persists events atomically. The Audit module provides the current SQL implementation.
• Without an outbox provider, Core publishes to EventBus only after the outer transaction commits. This preserves reduced installations but is not durable across process failure or multiple workers.
• Use EventBus as the in-process dispatch contract for module reactions invoked by the outbox dispatcher or for non-critical fallback reactions.
• Use the shared audit_event / audit_from_principal helper for audited module actions. The helper persists the audit row and transactionally emits a governed PlatformEvent whose type is the audit action.
• Use record_change for module delta feeds. It persists the change-sequence row and transactionally emits a generic module change event such as mail.profile.updated.
• Drain the outbox with the Celery govoplan.events.dispatch_outbox task on the events queue. The periodic schedule also retries pending rows.
• The dispatcher invokes Dataflow event ingestion when that capability is active, then publishes to the process-local bus.
• Treat Redis/Celery as worker/job infrastructure, not as the authoritative first event transport. PostgreSQL remains authoritative until dispatch is recorded.
• Keep the dispatch implementation pluggable behind the PlatformEvent envelope so a future message broker can be added without changing event producers.
• Keep commands out of the kernel until workflows need retryable, durable, operator-visible command records.

This keeps the first contract small, PostgreSQL-friendly, auditable, and recoverable after process crashes. It also avoids making Redis a correctness dependency for deployments that only need synchronous mail/tests or light background work.

Dispatch Semantics

Event producers should write their domain state and outbox event in the same database transaction wherever possible. Handlers must be idempotent because the outbox dispatcher can retry after a crash or timeout.

The current outbox stores:

• event_id, event_type, module_id
• correlation_id, causation_id
• classification, serialized event payload
• status, attempts, next_attempt_at
• dispatched_at, last_error, timestamps

Handlers must be idempotent: a worker may complete an external effect and fail before marking its outbox row dispatched. Anything that must survive process failure, restart, package update, or worker redeployment requires the outbox provider and dispatcher.

Trace IDs

Every PlatformEvent has:

• event_id: unique ID for that event.
• correlation_id: stable ID for the whole request, workflow, or job.
• causation_id: the event ID or external operation ID that caused this event.
• actor: optional typed actor reference, for example user, API key, system actor, delegated actor, or installer daemon.
• tenant: optional tenant reference when the event is tenant-scoped.
• subject: optional typed subject reference for the person, organization, account, case, campaign, or other entity the event is about.
• resource: optional typed resource reference for the object changed or observed by the event.
• classification: payload sensitivity, currently public, internal, confidential, or restricted.
• payload: JSON-serializable module-owned event details.

The FastAPI app factory creates an event context for every request. It accepts X-Correlation-ID or X-Request-ID when the value is a compact safe trace ID, otherwise it generates a new ID. Responses include X-Correlation-ID.

Audit logging reads the current event context and stores trace IDs in details._trace. Callers can also pass explicit correlation_id and causation_id to audit_event or audit_from_principal. The same trace is copied into the emitted PlatformEvent, so audit rows and event subscribers can be correlated without route-specific glue code.

Admin and lifecycle code should use the compact operational detail shape documented in govoplan-audit/docs/AUDIT_TRACE_CONTEXT.md. The core audit_operation_context helper preserves module_id, request_id, run_id, outcome, and _trace while applying the shared audit redaction pass to additional detail values.

Audit MVP Boundary

govoplan-audit owns:

• the audit_log table and audit API routes
• audit route contribution through its module manifest
• audit retention behavior in cooperation with policy/retention settings
• future audit sink/export capability implementations

govoplan-core owns:

• AuditEvent and AuditSink protocol contracts
• request and event trace context
• the compatibility audit_event helper while routes are still migrating
• retention orchestration that calls module capabilities

Feature modules should record audit facts through the shared helper or a future audit.sink capability. They should not import audit storage internals. Module state changes that only need delta-feed visibility should go through record_change; route-level business actions should still record a semantic audit action.

Initial Domain Event Inventory

Access:

• access.account.created
• access.account.updated
• access.membership.created
• access.membership.updated
• access.group.created
• access.group.updated
• access.role.created
• access.role.updated
• access.role.deleted
• access.api_key.created
• access.api_key.revoked
• access.session.created
• access.session.revoked

Tenancy:

• tenant.created
• tenant.updated
• tenant.suspended
• tenant.resumed
• tenant.deletion_requested
• tenant.erasure_completed

Policy:

• policy.system.updated
• policy.tenant.updated
• policy.user.updated
• policy.group.updated
• policy.campaign.updated
• policy.effective_policy.changed

Files:

• files.file.uploaded
• files.file.renamed
• files.file.deleted
• files.file.frozen
• files.share.created
• files.share.revoked
• files.connector.imported
• files.connector.access_denied

Mail:

• mail.profile.created
• mail.profile.updated
• mail.profile.credentials_rotated
• mail.profile.tested
• mail.message.sent
• mail.message.send_failed
• mail.imap.appended
• mail.imap.append_failed
• mail.mailbox.message_seen

Campaign:

• campaign.created
• campaign.version.created
• campaign.validated
• campaign.built
• campaign.reviewed
• campaign.queued
• campaign.send_started
• campaign.recipient_attempted
• campaign.recipient_delivered
• campaign.recipient_failed
• campaign.paused
• campaign.resumed
• campaign.cancelled
• campaign.report.exported

Event Payload Rules

• Payloads must be JSON-serializable.
• Use stable IDs, not ORM objects.
• Include tenant ID when tenant-scoped.
• Include actor/principal, subject, and resource references only as typed DTOs or primitive IDs.
• Set classification to the highest sensitivity needed by the envelope or payload, not the lowest sensitivity of any individual field.
• Do not include secrets, raw message bodies, full recipient lists, or file content.
• Put large evidence in owning module storage and reference it by ID.