govoplan-addresses/docs/ADDRESS_MODULE_ARCHITECTURE.md

# GovOPlaN Addresses Module Architecture

## Decision

`govoplan-addresses` owns reusable contact and recipient-source data. Campaigns,
mail, scheduling, portal, postbox, forms, reporting, and other modules consume
address data through core-mediated capabilities and APIs, not by importing
address-module internals.

The implementation reference for contact data is vCard. CardDAV is the primary
address-book sync protocol. LDAP/Active Directory, Exchange/Microsoft 365,
Google Contacts, CSV/XLSX, LDIF, and batch vCard import/export are connector
targets layered on top of the same local model and sync contracts.

## Ownership

`govoplan-addresses` owns:

- scoped address books
- vCard-compatible contacts and postal/email/phone contact-point data
- vCard import/export and vCard-compatible field mapping
- reusable recipient sources and classical address lists
- contact tags, categories, communication preferences, consent, and legal basis
- deduplication, merge, address quality checks, and suppression lists
- contact provenance, audit history, soft delete, and restore
- external-source bindings, sync state, conflicts, and read-only source markers

It does not own:

- campaign-local recipient snapshots and evidence
- mail transport, mailbox access, or delivery queues
- calendar events or iCalendar event storage
- global identity authentication or authorization decisions
- IDM identities, typed groups, effective-dated relationships, or identity
  lifecycle state
- organization structures, units, function definitions, or function
  assignments
- operational distribution lists/`Verteiler` with mixed address contacts, IDM
  identities/groups, functions, raw targets, Dataflow rows, and nested lists

## Scopes

Address books can live in these scopes:

- `user`: personal address books and remembered contacts
- `group`: team/shared address books
- `tenant`: tenant-wide directories and approved shared lists
- `system`: platform-wide public/shared directories where policy allows it

The scope determines visibility, default permissions, sync credentials, and
whether downstream modules may reuse or mutate entries.

## Data Model Principles

The canonical model should preserve enough vCard semantics to round-trip common
fields:

- name components and formatted names
- nicknames and display names
- email addresses, phone numbers, postal addresses, URLs, notes, categories
- organization, title, role, department, `KIND`, and `RELATED` values needed for
  vCard round-trip compatibility
- birthday/anniversary where allowed by policy
- photos/avatars where storage and privacy policy allow them
- calendar or scheduling addresses where present
- source IDs, revisions, ETags, sync tokens, and provenance

The model should support both normalized query fields and a preserved original
representation for import/export and conflict handling.

The local baseline implements scoped address books, contacts, normalized
email/phone/postal-address tables, tags, source kind/reference fields,
first-class source payload/revision fields, preserved original contact-point
values, and append-only field provenance. Imported
vCards preserve raw source payload and revision metadata for audit/debugging.
Sync sources, attempt state, tombstones, conflicts, and diagnostics are now
first-class backend tables and API resources. Connector-specific diffing,
CardDAV discovery, and conflict-resolution UI remain part of the connector
milestones.

## Capabilities

The first stable capabilities are:

- `addresses.recipient_source`: return immutable recipient snapshots for
  campaigns, forms, reporting, and other send/build workflows.
- `addresses.lookup`: provide read-only lookup and autocomplete for mail,
  campaign, scheduling, postbox, portal, and case workflows.
- `addresses.contact_writer`: provide address-book-scoped write target decisions
  and contact creation for local or otherwise writable sources.
- `addresses.contact_point_resolution` version 1.x: resolve channel-neutral
  contact points and freeze immutable recipient evidence.

Capabilities use DTOs and source IDs. Consumers must not receive ORM objects or
write address tables directly. Consumers that need historical evidence must
store their own immutable snapshot with source ID, source revision, and
provenance; they must not treat live address records as historical evidence.

`addresses.recipient_source` exposes both complete address books and classical
address lists. Address-book sources use `addresses:address_book:<id>`.
Address-list sources use `addresses:address_list:<id>` and include the
address-list entry ID in each recipient's provenance. The legacy snapshot DTO
remains email-oriented for compatible campaign consumers.

Channel-neutral consumers use `addresses.contact_point_resolution`, which
supports email, postal, internal-mail, and portal targets, including postal-only
address-list entries. Requests make effective date, communication purpose,
address purpose, fallback behavior, locale, and domestic/international postal
formatting explicit. Results retain stable subject/contact/contact-point IDs,
source, preference and consent revisions, provenance, and reasons for excluded
or unresolved candidates.

Live previews are bounded to 500 rows per page and 20,000 source members per
request. Frozen snapshots persist resolved values and exclusions with a
deterministic hash; reading a snapshot never resolves the live contact again.
Mixed-audience expansion and final cross-provider Policy/channel decisions
remain owned by Distribution Lists and Policy. The contract is defined in Core,
and Addresses does not import IDM, Organizations, or Distribution Lists
implementations.

The writer capability is intentionally address-book specific. It answers
whether the current principal may perform an operation such as `create_contact`,
`update_contact`, or `delete_contact` against a concrete address book. The
decision payload includes:

- `allowed`
- stable `reason`
- user-facing `message`
- required scopes
- source kind
- read-only state
- scope and tenant provenance

Policy modules or connector sync state may later add inputs to this decision,
but consumers must continue to call the address capability/API instead of
importing policy logic or address services directly. Disabled or read-only UI
actions should surface the returned `message` on hover.

## Sync Model

Every synced address book tracks or can track:

- connector type and external account/source
- external address-book ID and display name
- local address-book scope
- sync direction: read-only, one-way import, one-way export, two-way
- sync token, ETag/revision, last successful sync, last attempted sync
- deleted markers/tombstones
- conflict status and resolution decision
- connector diagnostics and rate-limit/backoff state

Sync conflict UX must show the local value, remote value, source, timestamp, and
available action. Silent overwrite is not acceptable.

Sync infrastructure is intentionally connector-neutral. CardDAV, LDAP,
Exchange/Microsoft 365, Google Contacts, CSV/XLSX/LDIF import profiles, and
future connectors must write through `addresses_sync_sources` and related
records instead of inventing connector-specific status tables. Connector jobs
may mark a source `running`, `succeeded`, `failed`, or `conflict`; read-only and
one-way-import sources propagate a read-only decision to the owning address
book, which in turn blocks normal contact writes through the existing writer
capability/API.

The first CardDAV implementation supports discovery, source binding, dry-run
preview, inbound vCard sync, outbound create/update/delete for writable
sources, sync-token/full-sync fallback, tombstones, diagnostics, and persisted
conflicts. Outbound writes use ETag preconditions; stale local state must become
a conflict instead of silently overwriting remote data. The first conflict
review UI compares stored local and remote field payloads and can apply a
stored remote vCard payload or a manual per-field local/remote merge payload.
Source disconnect/delete removes the source binding and related sync records
while keeping local contacts. Because API-managed CardDAV credentials are
encrypted in the source row, the same transaction physically removes their
ciphertext and emits non-secret credential-deletion audit evidence. Destructive
module retirement audits all remaining owned credential material before table
removal. An unowned legacy reference is detached rather than passed to an
external secret provider.

LDAP and Active Directory use the same source, plan, diagnostic, tombstone, and
provider-health records. Endpoints must use LDAPS or StartTLS and may reference
only a visible reusable credential envelope; bind secrets are never copied into
source metadata. Root-DSE discovery returns candidate base DNs. A configured
source performs bounded paged searches and maps explicit attributes to contact
fields. Stable source keys plus `modifyTimestamp`, `uSNChanged`, `entryCSN`, or
a deterministic attribute digest make refreshes idempotent. Only a complete
successful search can infer deletion. A timeout, bind failure, malformed entry,
duplicate key, or configured entry limit retains existing contacts and reports
the source as failed/stale instead of creating tombstones.

## Static Tabular Imports

CSV and XLSX use versioned, scoped mapping profiles rather than live sync
sources. Profiles retain delimiter, encoding, header or worksheet selection,
stable source-key mapping, field mappings, locale and tags, row limits, and
explicit duplicate, blank-value, and existing-contact policies. Updating a
profile creates an immutable next version; prior import runs continue to point
at the reviewed version.

Preview decodes at most 10 MB and 10,000 rows, validates every referenced
column and source key, and returns an effect or diagnostic for every data row.
XLSX parsing is read-only; formulas are rejected and macros/legacy workbook
formats are not accepted. The input SHA-256 and deterministic plan hash are
stored with full effects. Apply uses exactly that plan, rejects changed target
contacts, and is idempotent. Created IDs and pre-update snapshots provide a
guarded rollback: rollback proceeds only while each imported contact still
matches its recorded post-apply hash. Arbitrary transforms remain Dataflow's
responsibility; Files and Datasources are optional origins, not prerequisites
for direct upload.

Persisted import runs can be resumed through `/address-book?import_run=<id>`.
The WebUI reloads the bounded run projection, selects its address book and
mapping version, and restores statistics, diagnostics, effects, and lifecycle
state. Apply and rollback both carry the reviewed plan hash. The normal read
projection never includes uploaded bytes or private before-images, and an
unknown, expired, hidden, or cross-tenant id is presented as one unavailable
state so the deep link cannot enumerate another tenant's imports.

## Quality, Deduplication, And Recovery

Quality is evidence about a concrete contact point, separate from communication
consent or Policy. Effective decisions use one of `valid`, `invalid`,
`returned`, `stale`, or `undeliverable`, retain reason/evidence references, and
end an overlapping prior decision rather than rewriting history. Recipient
capabilities project the current decision into a stable status and reason code;
consumers can exclude invalid points or explicitly handle stale points without
copying Addresses rules.

Duplicate suggestions are bounded to 500 scanned contacts and 100 returned
pairs. Every score is composed from visible exact-match features such as a
normalized email, phone, postal address, or name/organization combination. A
suggestion does not mutate data.

A merge is an explicit, transactional decision. The caller selects a surviving
contact, scalar-field sources, source precedence, and either union or
survivor-only contact-point handling. The merge records before/after evidence
and hashes, field/contact-point decisions, copied quality/governance evidence,
and stable loser-to-winner redirects. Address-list entries are repointed in the
same transaction. Undo and split restore the recorded contacts and memberships
only when the current evidence still matches the post-merge hash; later edits
must be reconciled first. Core change-sequence evidence is always written. Core
audit entries are written by HTTP mutation routes without requiring the
optional Audit module.

## Connector Direction

Implement connectors in this order:

1. vCard import/export and batch import.
2. CardDAV address books.
3. LDAP/Active Directory read-only directories and reusable CSV/XLSX mapping
   profiles (implemented).
4. [Microsoft Graph for Microsoft 365](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/16),
   [explicit on-premises Exchange profiles](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/17),
   and [Google People](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/18).
5. [LDIF import](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/20)
   and [selective/large-batch vCard workflows](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/21).

The live connectors use the existing sync-source model. LDAP is read-only;
Microsoft Graph and Google start with read-only/import and gate two-way mode on
conditional-write and outcome-reconciliation tests. On-premises Exchange first
probes and records an explicit supported server/API profile. CSV/XLSX, LDIF,
and uploaded vCard are static one-way imports, not writable remote sources.

Connector runtime behavior should reuse shared connector concepts where useful:
configured endpoints, credentials, dry-run, diagnostics, rate limits, and audit
events. The shared contract work is tracked in
[`govoplan-connectors#8`](https://git.add-ideas.de/GovOPlaN/govoplan-connectors/issues/8);
Addresses remains the owner of contact mapping, provenance, quality, and sync
state.

## Cross-Module Integration

Campaigns should consume `addresses.recipient_source` through the platform
registry and freeze snapshots into campaign versions. Mail should consume
`addresses.lookup` for autocomplete and `addresses.contact_writer` for "add
contact" workflows. Scheduling should use lookup for attendees and organizers.
Portal, postbox, cases, forms, and reporting should link to contact records by
stable IDs while keeping their own domain evidence. Cross-module UI must hide
write actions when no writable target exists, or show the writer decision
message when a disabled action remains visible for context.

Operational distribution lists and reusable dynamic segments belong in
`govoplan-dist-lists`. They may consume address lists as one entry type, but
they own mixed recipient expansion for address contacts, IDM identities and
typed groups, organization units, functions/effective incumbents, raw targets,
Dataflow-backed rows, and nested lists. Workflow owns `Umlauf` execution state;
distribution lists define who is included, not how work circulates.

Organizations owns unit and function definitions. IDM owns effective-dated
identity-to-function assignments and typed group relationships. Identity
lifecycle status is not a business audience status; a selectable business
status is represented by a group, function, or effective-dated relationship.
Addresses may link contact points to stable provider references without copying
those provider-owned facts.

## Deferred Work

The following are valuable but not required for the first functional milestone:

- two-way sync conflict UI
- Microsoft/Google connectors
- richer vCard `KIND`/`RELATED` round-trip and provider-reference linking
- advanced consent-policy automation
- contact activity timeline across all modules