govoplan-addresses/docs/ADDRESS_MODULE_ARCHITECTURE.md

GovOPlaN Addresses Module Architecture

Decision

govoplan-addresses owns reusable contact and recipient-source data. Campaigns, mail, scheduling, portal, postbox, forms, reporting, and other modules consume address data through core-mediated capabilities and APIs, not by importing address-module internals.

The implementation reference for contact data is vCard. CardDAV is the primary address-book sync protocol. LDAP/Active Directory, Exchange/Microsoft 365, Google Contacts, CSV/XLSX, LDIF, and batch vCard import/export are connector targets layered on top of the same local model and sync contracts.

Ownership

govoplan-addresses owns:

• scoped address books
• vCard-compatible contacts and postal/email/phone contact-point data
• vCard import/export and vCard-compatible field mapping
• reusable recipient sources and classical address lists
• contact tags, categories, communication preferences, consent, and legal basis
• deduplication, merge, address quality checks, and suppression lists
• contact provenance, audit history, soft delete, and restore
• external-source bindings, sync state, conflicts, and read-only source markers

It does not own:

• campaign-local recipient snapshots and evidence
• mail transport, mailbox access, or delivery queues
• calendar events or iCalendar event storage
• global identity authentication or authorization decisions
• IDM identities, typed groups, effective-dated relationships, or identity lifecycle state
• organization structures, units, function definitions, or function assignments
• operational distribution lists/Verteiler with mixed address contacts, IDM identities/groups, functions, raw targets, Dataflow rows, and nested lists

Scopes

Address books can live in these scopes:

• user: personal address books and remembered contacts
• group: team/shared address books
• tenant: tenant-wide directories and approved shared lists
• system: platform-wide public/shared directories where policy allows it

The scope determines visibility, default permissions, sync credentials, and whether downstream modules may reuse or mutate entries.

Data Model Principles

The canonical model should preserve enough vCard semantics to round-trip common fields:

• name components and formatted names
• nicknames and display names
• email addresses, phone numbers, postal addresses, URLs, notes, categories
• organization, title, role, department, KIND, and RELATED values needed for vCard round-trip compatibility
• birthday/anniversary where allowed by policy
• photos/avatars where storage and privacy policy allow them
• calendar or scheduling addresses where present
• source IDs, revisions, ETags, sync tokens, and provenance

The model should support both normalized query fields and a preserved original representation for import/export and conflict handling.

The local baseline implements scoped address books, contacts, normalized email/phone/postal-address tables, tags, source kind/reference fields, first-class source payload/revision fields, preserved original contact-point values, and append-only field provenance. Imported vCards preserve raw source payload and revision metadata for audit/debugging. Sync sources, attempt state, tombstones, conflicts, and diagnostics are now first-class backend tables and API resources. Connector-specific diffing, CardDAV discovery, and conflict-resolution UI remain part of the connector milestones.

Capabilities

The first stable capabilities are:

• addresses.recipient_source: return immutable recipient snapshots for campaigns, forms, reporting, and other send/build workflows.
• addresses.lookup: provide read-only lookup and autocomplete for mail, campaign, scheduling, postbox, portal, and case workflows.
• addresses.contact_writer: provide address-book-scoped write target decisions and contact creation for local or otherwise writable sources.
• addresses.contact_point_resolution version 1.x: resolve channel-neutral contact points and freeze immutable recipient evidence.

Capabilities use DTOs and source IDs. Consumers must not receive ORM objects or write address tables directly. Consumers that need historical evidence must store their own immutable snapshot with source ID, source revision, and provenance; they must not treat live address records as historical evidence.

addresses.recipient_source exposes both complete address books and classical address lists. Address-book sources use addresses:address_book:<id>. Address-list sources use addresses:address_list:<id> and include the address-list entry ID in each recipient's provenance. The legacy snapshot DTO remains email-oriented for compatible campaign consumers.

Channel-neutral consumers use addresses.contact_point_resolution, which supports email, postal, internal-mail, and portal targets, including postal-only address-list entries. Requests make effective date, communication purpose, address purpose, fallback behavior, locale, and domestic/international postal formatting explicit. Results retain stable subject/contact/contact-point IDs, source, preference and consent revisions, provenance, and reasons for excluded or unresolved candidates.

Live previews are bounded to 500 rows per page and 20,000 source members per request. Frozen snapshots persist resolved values and exclusions with a deterministic hash; reading a snapshot never resolves the live contact again. Mixed-audience expansion and final cross-provider Policy/channel decisions remain owned by Distribution Lists and Policy. The contract is defined in Core, and Addresses does not import IDM, Organizations, or Distribution Lists implementations.

The writer capability is intentionally address-book specific. It answers whether the current principal may perform an operation such as create_contact, update_contact, or delete_contact against a concrete address book. The decision payload includes:

• allowed
• stable reason
• user-facing message
• required scopes
• source kind
• read-only state
• scope and tenant provenance

Policy modules or connector sync state may later add inputs to this decision, but consumers must continue to call the address capability/API instead of importing policy logic or address services directly. Disabled or read-only UI actions should surface the returned message on hover.

Sync Model

Every synced address book tracks or can track:

• connector type and external account/source
• external address-book ID and display name
• local address-book scope
• sync direction: read-only, one-way import, one-way export, two-way
• sync token, ETag/revision, last successful sync, last attempted sync
• deleted markers/tombstones
• conflict status and resolution decision
• connector diagnostics and rate-limit/backoff state

Sync conflict UX must show the local value, remote value, source, timestamp, and available action. Silent overwrite is not acceptable.

Sync infrastructure is intentionally connector-neutral. CardDAV, LDAP, Exchange/Microsoft 365, Google Contacts, CSV/XLSX/LDIF import profiles, and future connectors must write through addresses_sync_sources and related records instead of inventing connector-specific status tables. Connector jobs may mark a source running, succeeded, failed, or conflict; read-only and one-way-import sources propagate a read-only decision to the owning address book, which in turn blocks normal contact writes through the existing writer capability/API.

The first CardDAV implementation supports discovery, source binding, dry-run preview, inbound vCard sync, outbound create/update/delete for writable sources, sync-token/full-sync fallback, tombstones, diagnostics, and persisted conflicts. Outbound writes use ETag preconditions; stale local state must become a conflict instead of silently overwriting remote data. The first conflict review UI compares stored local and remote field payloads and can apply a stored remote vCard payload or a manual per-field local/remote merge payload. Source disconnect/delete removes the source binding and related sync records while keeping local contacts. Because API-managed CardDAV credentials are encrypted in the source row, the same transaction physically removes their ciphertext and emits non-secret credential-deletion audit evidence. Destructive module retirement audits all remaining owned credential material before table removal. An unowned legacy reference is detached rather than passed to an external secret provider.

LDAP and Active Directory use the same source, plan, diagnostic, tombstone, and provider-health records. Endpoints must use LDAPS or StartTLS and may reference only a visible reusable credential envelope; bind secrets are never copied into source metadata. Root-DSE discovery returns candidate base DNs. A configured source performs bounded paged searches and maps explicit attributes to contact fields. Stable source keys plus modifyTimestamp, uSNChanged, entryCSN, or a deterministic attribute digest make refreshes idempotent. Only a complete successful search can infer deletion. A timeout, bind failure, malformed entry, duplicate key, or configured entry limit retains existing contacts and reports the source as failed/stale instead of creating tombstones.

Static Tabular Imports

CSV and XLSX use versioned, scoped mapping profiles rather than live sync sources. Profiles retain delimiter, encoding, header or worksheet selection, stable source-key mapping, field mappings, locale and tags, row limits, and explicit duplicate, blank-value, and existing-contact policies. Updating a profile creates an immutable next version; prior import runs continue to point at the reviewed version.

Preview decodes at most 10 MB and 10,000 rows, validates every referenced column and source key, and returns an effect or diagnostic for every data row. XLSX parsing is read-only; formulas are rejected and macros/legacy workbook formats are not accepted. The input SHA-256 and deterministic plan hash are stored with full effects. Apply uses exactly that plan, rejects changed target contacts, and is idempotent. Created IDs and pre-update snapshots provide a guarded rollback: rollback proceeds only while each imported contact still matches its recorded post-apply hash. Arbitrary transforms remain Dataflow's responsibility; Files and Datasources are optional origins, not prerequisites for direct upload.

Persisted import runs can be resumed through /address-book?import_run=<id>. The WebUI reloads the bounded run projection, selects its address book and mapping version, and restores statistics, diagnostics, effects, and lifecycle state. Apply and rollback both carry the reviewed plan hash. The normal read projection never includes uploaded bytes or private before-images, and an unknown, expired, hidden, or cross-tenant id is presented as one unavailable state so the deep link cannot enumerate another tenant's imports.

Quality, Deduplication, And Recovery

Quality is evidence about a concrete contact point, separate from communication consent or Policy. Effective decisions use one of valid, invalid, returned, stale, or undeliverable, retain reason/evidence references, and end an overlapping prior decision rather than rewriting history. Recipient capabilities project the current decision into a stable status and reason code; consumers can exclude invalid points or explicitly handle stale points without copying Addresses rules.

Duplicate suggestions are bounded to 500 scanned contacts and 100 returned pairs. Every score is composed from visible exact-match features such as a normalized email, phone, postal address, or name/organization combination. A suggestion does not mutate data.

A merge is an explicit, transactional decision. The caller selects a surviving contact, scalar-field sources, source precedence, and either union or survivor-only contact-point handling. The merge records before/after evidence and hashes, field/contact-point decisions, copied quality/governance evidence, and stable loser-to-winner redirects. Address-list entries are repointed in the same transaction. Undo and split restore the recorded contacts and memberships only when the current evidence still matches the post-merge hash; later edits must be reconciled first. Core change-sequence evidence is always written. Core audit entries are written by HTTP mutation routes without requiring the optional Audit module.

Connector Direction

Implement connectors in this order:

1. vCard import/export and batch import.
2. CardDAV address books.
3. LDAP/Active Directory read-only directories and reusable CSV/XLSX mapping profiles (implemented).
4. Microsoft Graph for Microsoft 365, explicit on-premises Exchange profiles, and Google People.
5. LDIF import and selective/large-batch vCard workflows.

The live connectors use the existing sync-source model. LDAP is read-only; Microsoft Graph and Google start with read-only/import and gate two-way mode on conditional-write and outcome-reconciliation tests. On-premises Exchange first probes and records an explicit supported server/API profile. CSV/XLSX, LDIF, and uploaded vCard are static one-way imports, not writable remote sources.

Connector runtime behavior should reuse shared connector concepts where useful: configured endpoints, credentials, dry-run, diagnostics, rate limits, and audit events. The shared contract work is tracked in govoplan-connectors#8; Addresses remains the owner of contact mapping, provenance, quality, and sync state.

Cross-Module Integration

Campaigns should consume addresses.recipient_source through the platform registry and freeze snapshots into campaign versions. Mail should consume addresses.lookup for autocomplete and addresses.contact_writer for "add contact" workflows. Scheduling should use lookup for attendees and organizers. Portal, postbox, cases, forms, and reporting should link to contact records by stable IDs while keeping their own domain evidence. Cross-module UI must hide write actions when no writable target exists, or show the writer decision message when a disabled action remains visible for context.

Operational distribution lists and reusable dynamic segments belong in govoplan-dist-lists. They may consume address lists as one entry type, but they own mixed recipient expansion for address contacts, IDM identities and typed groups, organization units, functions/effective incumbents, raw targets, Dataflow-backed rows, and nested lists. Workflow owns Umlauf execution state; distribution lists define who is included, not how work circulates.

Organizations owns unit and function definitions. IDM owns effective-dated identity-to-function assignments and typed group relationships. Identity lifecycle status is not a business audience status; a selectable business status is represented by a group, function, or effective-dated relationship. Addresses may link contact points to stable provider references without copying those provider-owned facts.

Deferred Work

The following are valuable but not required for the first functional milestone:

• two-way sync conflict UI
• Microsoft/Google connectors
• richer vCard KIND/RELATED round-trip and provider-reference linking
• advanced consent-policy automation
• contact activity timeline across all modules