Implement address quality and reversible contact merges

This commit is contained in:
2026-08-02 07:03:27 +02:00
parent 2e78b9ae50
commit 19e9096572
15 changed files with 4369 additions and 31 deletions
+28 -2
View File
@@ -70,7 +70,8 @@ representation for import/export and conflict handling.
The local baseline implements scoped address books, contacts, normalized
email/phone/postal-address tables, tags, source kind/reference fields,
first-class source payload/revision fields, and provenance JSON. Imported
first-class source payload/revision fields, preserved original contact-point
values, and append-only field provenance. Imported
vCards preserve raw source payload and revision metadata for audit/debugging.
Sync sources, attempt state, tombstones, conflicts, and diagnostics are now
first-class backend tables and API resources. Connector-specific diffing,
@@ -175,6 +176,32 @@ module retirement audits all remaining owned credential material before table
removal. An unowned legacy reference is detached rather than passed to an
external secret provider.
## Quality, Deduplication, And Recovery
Quality is evidence about a concrete contact point, separate from communication
consent or Policy. Effective decisions use one of `valid`, `invalid`,
`returned`, `stale`, or `undeliverable`, retain reason/evidence references, and
end an overlapping prior decision rather than rewriting history. Recipient
capabilities project the current decision into a stable status and reason code;
consumers can exclude invalid points or explicitly handle stale points without
copying Addresses rules.
Duplicate suggestions are bounded to 500 scanned contacts and 100 returned
pairs. Every score is composed from visible exact-match features such as a
normalized email, phone, postal address, or name/organization combination. A
suggestion does not mutate data.
A merge is an explicit, transactional decision. The caller selects a surviving
contact, scalar-field sources, source precedence, and either union or
survivor-only contact-point handling. The merge records before/after evidence
and hashes, field/contact-point decisions, copied quality/governance evidence,
and stable loser-to-winner redirects. Address-list entries are repointed in the
same transaction. Undo and split restore the recorded contacts and memberships
only when the current evidence still matches the post-merge hash; later edits
must be reconciled first. Core change-sequence evidence is always written. Core
audit entries are written by HTTP mutation routes without requiring the
optional Audit module.
## Connector Direction
Implement connectors in this order:
@@ -218,7 +245,6 @@ those provider-owned facts.
The following are valuable but not required for the first functional milestone:
- automatic deduplication and merge suggestions
- two-way sync conflict UI
- Microsoft/Google connectors
- richer vCard `KIND`/`RELATED` round-trip and provider-reference linking
+16 -10
View File
@@ -219,17 +219,19 @@ Primary issues: `govoplan-addresses#8`, `govoplan-addresses#9`,
Tasks:
- LDAP/Active Directory read-only directory connector
- Exchange/Microsoft 365 contacts connector
- Google Contacts connector
- CSV/XLSX/LDIF import mapping profiles
- classical address-list UI; reusable static/dynamic operational segments move
- [ ] LDAP/Active Directory read-only directory connector
- [ ] Exchange/Microsoft 365 contacts connector
- [ ] Google Contacts connector
- [ ] CSV/XLSX/LDIF import mapping profiles
- [x] classical address-list UI; reusable static/dynamic operational segments move
to `govoplan-dist-lists`
- operational distribution lists move to `govoplan-dist-lists`
- consent, legal-basis, suppression, and communication preferences
- deduplication and merge workflow
- address quality checks and normalization
- richer vCard `KIND`/`RELATED` round-trip and stable links to IDM/Organizations
- [x] operational distribution lists move to `govoplan-dist-lists`
- [x] consent, legal-basis, suppression, and communication preferences
- [x] bounded, explainable deduplication and reversible merge/split workflow
- [x] contact-point quality states, normalization, original-value preservation,
field provenance, and correction dashboard
- [x] stable redirect resolution for merged contact references
- [ ] richer vCard `KIND`/`RELATED` round-trip and stable links to IDM/Organizations
Exit criteria:
@@ -237,6 +239,10 @@ Exit criteria:
- users can understand where data came from and whether they may edit it
- downstream modules can safely use contacts without owning them
Issues #9 and #10 are implemented. Issue #8 tracks the connector portfolio and
is split into independently deliverable connector/import follow-ups rather than
keeping one cross-protocol implementation ticket open.
## First Implementation Recommendation
Start with Milestone 1 and enough of Milestone 2 to define the data model
+57
View File
@@ -0,0 +1,57 @@
# Address Quality And Reversible Merges
## Operator Workflow
Open the shield action for a selected address book to review its quality. The
dialog shows:
- the number of contacts and contact points in the bounded scan
- current invalid, returned, stale, and undeliverable contact points
- explainable duplicate suggestions with their score inputs
- active and recovered merge records
Each contact point also has a `Quality` action in the contact detail. Recording
a new state ends an overlapping current state and retains both entries in
history. Use a stable reason code and an evidence reference when the state came
from delivery, import, or correction evidence.
`valid` makes the point normally usable. `invalid`, `returned`, and
`undeliverable` make it invalid for recipient resolution. `stale` remains a
distinct status so a downstream workflow can warn, request confirmation, or
block according to Policy. A later `valid` decision is a correction; it does
not delete the earlier evidence.
## Duplicate Review
Suggestions do not merge automatically. The score is the bounded sum of named
exact-match features. The operator chooses the surviving contact and whether to
combine unique contact points or retain only the survivor's points. The API can
additionally select the source contact for each scalar field and rank source
kinds.
A successful merge:
- archives each duplicate and redirects its stable contact ID to the survivor
- records scalar and contact-point survivorship decisions
- carries field and contact-point source provenance forward
- copies applicable quality and communication-governance evidence
- repoints address-list entries to the survivor and mapped contact point
- stores deterministic before/after evidence hashes
- emits core change-sequence and audit evidence
The merge history offers `Undo` and `Split`. Both restore the exact recorded
pre-merge contacts and list memberships. Recovery is deliberately rejected when
the contact or membership evidence changed after the merge. Reconcile those
later edits before retrying; the system does not silently discard them.
## Consumer Contract
Consumers resolve live contacts through `addresses.contact_point_resolution` or
`distribution.recipient_channel_facts`. They receive quality status, stable
reason codes, evidence provenance, and the current source revision. Consumers
must not read Addresses tables or recreate quality rules. A workflow requiring
historical proof freezes a contact-point snapshot before delivery.
The duplicate and quality endpoints are bounded. `truncated=true` means the
operator should narrow the source or run a staged API review; it does not mean
that the unreturned contacts were found clean.