govoplan-core/docs/EVENTS_AND_AUDIT.md

# Events And Audit

GovOPlaN uses a small kernel event contract first, not a broad command bus.
Commands remain module-owned application-service methods or API endpoints until
there is a concrete need for durable asynchronous command orchestration. Events
are facts about completed work and are safe for audit, projections, optional
module reactions, and operator diagnostics.

## Production Transport Decision

The production transport is a **transactional database outbox plus retrying
dispatcher**:

- Use `govoplan_core.core.events.PlatformEvent` for domain and platform events.
- Call `emit_platform_event(session, event)` to bind event delivery to the
  domain transaction.
- The optional `platform.eventOutbox` capability persists events atomically.
  The Audit module provides the current SQL implementation.
- Without an outbox provider, Core publishes to `EventBus` only after the outer
  transaction commits. This preserves reduced installations but is not durable
  across process failure or multiple workers.
- Use `EventBus` as the in-process dispatch contract for module reactions
  invoked by the outbox dispatcher or for non-critical fallback reactions.
- Use the shared `audit_event` / `audit_from_principal` helper for audited
  module actions. The helper persists the audit row and transactionally emits a
  governed `PlatformEvent` whose `type` is the audit action.
- Use `record_change` for module delta feeds. It persists the change-sequence
  row and transactionally emits a generic module change event such as
  `mail.profile.updated`.
- Drain the outbox with the Celery `govoplan.events.dispatch_outbox` task on the
  `events` queue. The periodic schedule also retries pending rows.
- The dispatcher invokes Dataflow event ingestion when that capability is
  active, then publishes to the process-local bus.
- Treat Redis/Celery as worker/job infrastructure, not as the authoritative
  first event transport. PostgreSQL remains authoritative until dispatch is
  recorded.
- Keep the dispatch implementation pluggable behind the `PlatformEvent`
  envelope so a future message broker can be added without changing event
  producers.
- Keep commands out of the kernel until workflows need retryable, durable,
  operator-visible command records.

This keeps the first contract small, PostgreSQL-friendly, auditable, and
recoverable after process crashes. It also avoids making Redis a correctness
dependency for deployments that only need synchronous mail/tests or light
background work.

## Dispatch Semantics

Event producers should write their domain state and outbox event in the same
database transaction wherever possible. Handlers must be idempotent because the
outbox dispatcher can retry after a crash or timeout.

The current outbox stores:

- `event_id`, `event_type`, `module_id`
- `correlation_id`, `causation_id`
- `classification`, serialized event `payload`
- `status`, `attempts`, `next_attempt_at`
- `dispatched_at`, `last_error`, timestamps

Handlers must be idempotent: a worker may complete an external effect and fail
before marking its outbox row dispatched. Anything that must survive process
failure, restart, package update, or worker redeployment requires the outbox
provider and dispatcher.

## Trace IDs

Every `PlatformEvent` has:

- `event_id`: unique ID for that event.
- `correlation_id`: stable ID for the whole request, workflow, or job.
- `causation_id`: the event ID or external operation ID that caused this
  event.
- `actor`: optional typed actor reference, for example user, API key, system
  actor, delegated actor, or installer daemon.
- `tenant`: optional tenant reference when the event is tenant-scoped.
- `subject`: optional typed subject reference for the person, organization,
  account, case, campaign, or other entity the event is about.
- `resource`: optional typed resource reference for the object changed or
  observed by the event.
- `classification`: payload sensitivity, currently `public`, `internal`,
  `confidential`, or `restricted`.
- `payload`: JSON-serializable module-owned event details.

The FastAPI app factory creates an event context for every request. It accepts
`X-Correlation-ID` or `X-Request-ID` when the value is a compact safe trace ID,
otherwise it generates a new ID. Responses include `X-Correlation-ID`.

Audit logging reads the current event context and stores trace IDs in
`details._trace`. Callers can also pass explicit `correlation_id` and
`causation_id` to `audit_event` or `audit_from_principal`. The same trace is
copied into the emitted `PlatformEvent`, so audit rows and event subscribers can
be correlated without route-specific glue code.

Admin and lifecycle code should use the compact operational detail shape
documented in `govoplan-audit/docs/AUDIT_TRACE_CONTEXT.md`. The core
`audit_operation_context` helper preserves `module_id`, `request_id`, `run_id`,
`outcome`, and `_trace` while applying the shared audit redaction pass to
additional detail values.

## Audit MVP Boundary

`govoplan-audit` owns:

- the `audit_log` table and audit API routes
- audit route contribution through its module manifest
- audit retention behavior in cooperation with policy/retention settings
- future audit sink/export capability implementations

`govoplan-core` owns:

- `AuditEvent` and `AuditSink` protocol contracts
- request and event trace context
- the compatibility `audit_event` helper while routes are still migrating
- retention orchestration that calls module capabilities

Feature modules should record audit facts through the shared helper or a future
`audit.sink` capability. They should not import audit storage internals. Module
state changes that only need delta-feed visibility should go through
`record_change`; route-level business actions should still record a semantic
audit action.

## Initial Domain Event Inventory

Access:

- `access.account.created`
- `access.account.updated`
- `access.membership.created`
- `access.membership.updated`
- `access.group.created`
- `access.group.updated`
- `access.role.created`
- `access.role.updated`
- `access.role.deleted`
- `access.api_key.created`
- `access.api_key.revoked`
- `access.session.created`
- `access.session.revoked`

Tenancy:

- `tenant.created`
- `tenant.updated`
- `tenant.suspended`
- `tenant.resumed`
- `tenant.deletion_requested`
- `tenant.erasure_completed`

Policy:

- `policy.system.updated`
- `policy.tenant.updated`
- `policy.user.updated`
- `policy.group.updated`
- `policy.campaign.updated`
- `policy.effective_policy.changed`

Files:

- `files.file.uploaded`
- `files.file.renamed`
- `files.file.deleted`
- `files.file.frozen`
- `files.share.created`
- `files.share.revoked`
- `files.connector.imported`
- `files.connector.access_denied`

Mail:

- `mail.profile.created`
- `mail.profile.updated`
- `mail.profile.credentials_rotated`
- `mail.profile.tested`
- `mail.message.sent`
- `mail.message.send_failed`
- `mail.imap.appended`
- `mail.imap.append_failed`
- `mail.mailbox.message_seen`

Campaign:

- `campaign.created`
- `campaign.version.created`
- `campaign.validated`
- `campaign.built`
- `campaign.reviewed`
- `campaign.queued`
- `campaign.send_started`
- `campaign.recipient_attempted`
- `campaign.recipient_delivered`
- `campaign.recipient_failed`
- `campaign.paused`
- `campaign.resumed`
- `campaign.cancelled`
- `campaign.report.exported`

## Event Payload Rules

- Payloads must be JSON-serializable.
- Use stable IDs, not ORM objects.
- Include tenant ID when tenant-scoped.
- Include actor/principal, subject, and resource references only as typed DTOs
  or primitive IDs.
- Set `classification` to the highest sensitivity needed by the envelope or
  payload, not the lowest sensitivity of any individual field.
- Do not include secrets, raw message bodies, full recipient lists, or file
  content.
- Put large evidence in owning module storage and reference it by ID.