Implement address quality and reversible contact merges
This commit is contained in:
@@ -70,7 +70,8 @@ representation for import/export and conflict handling.
|
||||
|
||||
The local baseline implements scoped address books, contacts, normalized
|
||||
email/phone/postal-address tables, tags, source kind/reference fields,
|
||||
first-class source payload/revision fields, and provenance JSON. Imported
|
||||
first-class source payload/revision fields, preserved original contact-point
|
||||
values, and append-only field provenance. Imported
|
||||
vCards preserve raw source payload and revision metadata for audit/debugging.
|
||||
Sync sources, attempt state, tombstones, conflicts, and diagnostics are now
|
||||
first-class backend tables and API resources. Connector-specific diffing,
|
||||
@@ -175,6 +176,32 @@ module retirement audits all remaining owned credential material before table
|
||||
removal. An unowned legacy reference is detached rather than passed to an
|
||||
external secret provider.
|
||||
|
||||
## Quality, Deduplication, And Recovery
|
||||
|
||||
Quality is evidence about a concrete contact point, separate from communication
|
||||
consent or Policy. Effective decisions use one of `valid`, `invalid`,
|
||||
`returned`, `stale`, or `undeliverable`, retain reason/evidence references, and
|
||||
end an overlapping prior decision rather than rewriting history. Recipient
|
||||
capabilities project the current decision into a stable status and reason code;
|
||||
consumers can exclude invalid points or explicitly handle stale points without
|
||||
copying Addresses rules.
|
||||
|
||||
Duplicate suggestions are bounded to 500 scanned contacts and 100 returned
|
||||
pairs. Every score is composed from visible exact-match features such as a
|
||||
normalized email, phone, postal address, or name/organization combination. A
|
||||
suggestion does not mutate data.
|
||||
|
||||
A merge is an explicit, transactional decision. The caller selects a surviving
|
||||
contact, scalar-field sources, source precedence, and either union or
|
||||
survivor-only contact-point handling. The merge records before/after evidence
|
||||
and hashes, field/contact-point decisions, copied quality/governance evidence,
|
||||
and stable loser-to-winner redirects. Address-list entries are repointed in the
|
||||
same transaction. Undo and split restore the recorded contacts and memberships
|
||||
only when the current evidence still matches the post-merge hash; later edits
|
||||
must be reconciled first. Core change-sequence evidence is always written. Core
|
||||
audit entries are written by HTTP mutation routes without requiring the
|
||||
optional Audit module.
|
||||
|
||||
## Connector Direction
|
||||
|
||||
Implement connectors in this order:
|
||||
@@ -218,7 +245,6 @@ those provider-owned facts.
|
||||
|
||||
The following are valuable but not required for the first functional milestone:
|
||||
|
||||
- automatic deduplication and merge suggestions
|
||||
- two-way sync conflict UI
|
||||
- Microsoft/Google connectors
|
||||
- richer vCard `KIND`/`RELATED` round-trip and provider-reference linking
|
||||
|
||||
+16
-10
@@ -219,17 +219,19 @@ Primary issues: `govoplan-addresses#8`, `govoplan-addresses#9`,
|
||||
|
||||
Tasks:
|
||||
|
||||
- LDAP/Active Directory read-only directory connector
|
||||
- Exchange/Microsoft 365 contacts connector
|
||||
- Google Contacts connector
|
||||
- CSV/XLSX/LDIF import mapping profiles
|
||||
- classical address-list UI; reusable static/dynamic operational segments move
|
||||
- [ ] LDAP/Active Directory read-only directory connector
|
||||
- [ ] Exchange/Microsoft 365 contacts connector
|
||||
- [ ] Google Contacts connector
|
||||
- [ ] CSV/XLSX/LDIF import mapping profiles
|
||||
- [x] classical address-list UI; reusable static/dynamic operational segments move
|
||||
to `govoplan-dist-lists`
|
||||
- operational distribution lists move to `govoplan-dist-lists`
|
||||
- consent, legal-basis, suppression, and communication preferences
|
||||
- deduplication and merge workflow
|
||||
- address quality checks and normalization
|
||||
- richer vCard `KIND`/`RELATED` round-trip and stable links to IDM/Organizations
|
||||
- [x] operational distribution lists move to `govoplan-dist-lists`
|
||||
- [x] consent, legal-basis, suppression, and communication preferences
|
||||
- [x] bounded, explainable deduplication and reversible merge/split workflow
|
||||
- [x] contact-point quality states, normalization, original-value preservation,
|
||||
field provenance, and correction dashboard
|
||||
- [x] stable redirect resolution for merged contact references
|
||||
- [ ] richer vCard `KIND`/`RELATED` round-trip and stable links to IDM/Organizations
|
||||
|
||||
Exit criteria:
|
||||
|
||||
@@ -237,6 +239,10 @@ Exit criteria:
|
||||
- users can understand where data came from and whether they may edit it
|
||||
- downstream modules can safely use contacts without owning them
|
||||
|
||||
Issues #9 and #10 are implemented. Issue #8 tracks the connector portfolio and
|
||||
is split into independently deliverable connector/import follow-ups rather than
|
||||
keeping one cross-protocol implementation ticket open.
|
||||
|
||||
## First Implementation Recommendation
|
||||
|
||||
Start with Milestone 1 and enough of Milestone 2 to define the data model
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
# Address Quality And Reversible Merges
|
||||
|
||||
## Operator Workflow
|
||||
|
||||
Open the shield action for a selected address book to review its quality. The
|
||||
dialog shows:
|
||||
|
||||
- the number of contacts and contact points in the bounded scan
|
||||
- current invalid, returned, stale, and undeliverable contact points
|
||||
- explainable duplicate suggestions with their score inputs
|
||||
- active and recovered merge records
|
||||
|
||||
Each contact point also has a `Quality` action in the contact detail. Recording
|
||||
a new state ends an overlapping current state and retains both entries in
|
||||
history. Use a stable reason code and an evidence reference when the state came
|
||||
from delivery, import, or correction evidence.
|
||||
|
||||
`valid` makes the point normally usable. `invalid`, `returned`, and
|
||||
`undeliverable` make it invalid for recipient resolution. `stale` remains a
|
||||
distinct status so a downstream workflow can warn, request confirmation, or
|
||||
block according to Policy. A later `valid` decision is a correction; it does
|
||||
not delete the earlier evidence.
|
||||
|
||||
## Duplicate Review
|
||||
|
||||
Suggestions do not merge automatically. The score is the bounded sum of named
|
||||
exact-match features. The operator chooses the surviving contact and whether to
|
||||
combine unique contact points or retain only the survivor's points. The API can
|
||||
additionally select the source contact for each scalar field and rank source
|
||||
kinds.
|
||||
|
||||
A successful merge:
|
||||
|
||||
- archives each duplicate and redirects its stable contact ID to the survivor
|
||||
- records scalar and contact-point survivorship decisions
|
||||
- carries field and contact-point source provenance forward
|
||||
- copies applicable quality and communication-governance evidence
|
||||
- repoints address-list entries to the survivor and mapped contact point
|
||||
- stores deterministic before/after evidence hashes
|
||||
- emits core change-sequence and audit evidence
|
||||
|
||||
The merge history offers `Undo` and `Split`. Both restore the exact recorded
|
||||
pre-merge contacts and list memberships. Recovery is deliberately rejected when
|
||||
the contact or membership evidence changed after the merge. Reconcile those
|
||||
later edits before retrying; the system does not silently discard them.
|
||||
|
||||
## Consumer Contract
|
||||
|
||||
Consumers resolve live contacts through `addresses.contact_point_resolution` or
|
||||
`distribution.recipient_channel_facts`. They receive quality status, stable
|
||||
reason codes, evidence provenance, and the current source revision. Consumers
|
||||
must not read Addresses tables or recreate quality rules. A workflow requiring
|
||||
historical proof freezes a contact-point snapshot before delivery.
|
||||
|
||||
The duplicate and quality endpoints are bounded. `truncated=true` means the
|
||||
operator should narrow the source or run a staged API review; it does not mean
|
||||
that the unreturned contacts were found clean.
|
||||
Reference in New Issue
Block a user