13 Commits
Author SHA1 Message Date
zemion 958a9959c8 fix(ui): align contextual documentation with headings
Verified with the coordinated workspace changes by devkit full run
2026-09-08T225814-186389-0000-3e3ed7cd (all seven phases passed).
This shared UI pass does not mark the individual module reviews complete.
2026-09-09 02:03:29 +02:00
zemion d4fa024034 fix(addresses): preserve complete import rollback evidence and batch previews
Module Package Release / publish-packages (push) Successful in 14s
Release v0.1.23. Coordinated integrity review: GovOPlaN/govoplan-core#298.
2026-09-08 12:19:36 +02:00
zemion 53490e7be7 Release govoplan-addresses v0.1.22: unify interface contracts and documentation
Module Package Release / publish-packages (push) Successful in 13s
2026-09-08 01:32:21 +02:00
zemion 3f1ff79e87 fix(webui): bind consequential address actions to help
Module Package Release / publish-packages (push) Successful in 13s
2026-08-24 11:36:31 +02:00
zemion 5cdf0ae9ff docs: complete German structured documentation
Module Package Release / publish-packages (push) Successful in 12s
2026-08-24 01:15:30 +02:00
zemion 3458306e04 docs(addresses): complete German reference coverage
Module Package Release / publish-packages (push) Successful in 12s
2026-08-23 20:11:47 +02:00
zemion 8740fb33f8 feat(addresses): add governed DSAR coverage 2026-08-21 01:19:17 +02:00
zemion 42fe262376 feat: add selective vCard batch workflows 2026-08-20 17:24:42 +02:00
zemion 147f34c5c1 feat: add governed LDIF contact imports 2026-08-20 11:19:23 +02:00
zemion ec32ad2c37 feat(imports): resume persisted address runs 2026-08-19 20:39:21 +02:00
zemion 7e5c1cf10c feat: align addresses with shared UI foundations 2026-08-18 21:32:34 +02:00
zemion 63d59a7eaa Adopt shared WebUI layout primitives 2026-08-18 11:30:39 +02:00
zemion e6165b4d26 Adopt shared WebUI layout primitives 2026-08-18 10:42:50 +02:00
35 changed files with 6602 additions and 503 deletions
+2
View File
@@ -142,11 +142,13 @@ dist
.policy-test-build/ .policy-test-build/
.template-preview-test-build/ .template-preview-test-build/
.import-test-build/ .import-test-build/
.import-run-test-build/
webui/.component-test-build/ webui/.component-test-build/
webui/.module-test-build/ webui/.module-test-build/
webui/.policy-test-build/ webui/.policy-test-build/
webui/.template-preview-test-build/ webui/.template-preview-test-build/
webui/.import-test-build/ webui/.import-test-build/
webui/.import-run-test-build/
# ---> Python # ---> Python
# Byte-compiled / optimized / DLL files # Byte-compiled / optimized / DLL files
+19 -2
View File
@@ -20,8 +20,14 @@ tenant summaries, and uninstall guards.
The first UI supports user, group, tenant, and system-scoped address books, The first UI supports user, group, tenant, and system-scoped address books,
multi-value contact methods, soft deletion, restore, read-only lookup/search, multi-value contact methods, soft deletion, restore, read-only lookup/search,
and vCard import/export for common contact fields. Imported vCards preserve and vCard import/export for common contact fields. Multi-file vCard imports now
source payload and revision metadata for later sync/conflict work. create a persisted preview before mutation, expose duplicate suggestions and
per-card create/update/ignore choices, reject stale plans, and make identical
commit retries idempotent. Pending batches can be reloaded or cancelled.
Address-book, address-list, and selected-contact exports explicitly support
vCard 3.0 or 4.0 with deterministic ordering and a recorded content hash.
Imported vCards preserve source payload and revision metadata for later
sync/conflict work, while batch diagnostics expose only bounded metadata.
The backend and WebUI also support classical address lists: reusable groupings The backend and WebUI also support classical address lists: reusable groupings
of contacts or specific contact methods within one address book. Campaigns can of contacts or specific contact methods within one address book. Campaigns can
@@ -98,6 +104,17 @@ The module exposes core-mediated capabilities for:
pickers. pickers.
- `distribution.recipient_channel_facts`: current channel, governance, and - `distribution.recipient_channel_facts`: current channel, governance, and
quality facts for distribution and Policy consumers. quality facts for distribution and Policy consumers.
- `privacy.dsar.addresses`: tenant-bounded, minimized data-subject discovery
across contacts, contact points, list use, governance, provenance,
synchronization evidence, and operator attribution.
The DSAR provider accepts corroborated email/account selectors and namespaced
Addresses references. It does not export connector state, raw import or sync
payloads, opaque metadata, snapshot payloads, or merge before/after payloads.
Reusable contacts are never deleted automatically: shared/synchronized contact
changes require an authorized dependency review through the ordinary Addresses
workflows, while governance, quality, merge, sync, import, and attribution
evidence is retained with an explicit reason.
`addresses.recipient_source` returns: `addresses.recipient_source` returns:
+19 -2
View File
@@ -187,9 +187,9 @@ successful search can infer deletion. A timeout, bind failure, malformed entry,
duplicate key, or configured entry limit retains existing contacts and reports duplicate key, or configured entry limit retains existing contacts and reports
the source as failed/stale instead of creating tombstones. the source as failed/stale instead of creating tombstones.
## Static Tabular Imports ## Static Mapped Imports
CSV and XLSX use versioned, scoped mapping profiles rather than live sync CSV, XLSX, and LDIF use versioned, scoped mapping profiles rather than live sync
sources. Profiles retain delimiter, encoding, header or worksheet selection, sources. Profiles retain delimiter, encoding, header or worksheet selection,
stable source-key mapping, field mappings, locale and tags, row limits, and stable source-key mapping, field mappings, locale and tags, row limits, and
explicit duplicate, blank-value, and existing-contact policies. Updating a explicit duplicate, blank-value, and existing-contact policies. Updating a
@@ -207,6 +207,23 @@ matches its recorded post-apply hash. Arbitrary transforms remain Dataflow's
responsibility; Files and Datasources are optional origins, not prerequisites responsibility; Files and Datasources are optional origins, not prerequisites
for direct upload. for direct upload.
LDIF is unfolded and parsed as a bounded stream of entries. Attribute names are
case-insensitive; UTF-8 and base64-encoded text and repeated values are retained
for mapping, while binary and URL values produce diagnostics and are never
projected or fetched. The default change-record policy rejects change records.
A profile can instead ignore them, or treat `changetype: add` as a static entry;
modify and delete records are never translated into contact mutations. Entry
hashes, input/plan hashes, mapping-version provenance, and the same correction
and guarded rollback lifecycle apply as for CSV and XLSX.
Persisted import runs can be resumed through `/address-book?import_run=<id>`.
The WebUI reloads the bounded run projection, selects its address book and
mapping version, and restores statistics, diagnostics, effects, and lifecycle
state. Apply and rollback both carry the reviewed plan hash. The normal read
projection never includes uploaded bytes or private before-images, and an
unknown, expired, hidden, or cross-tenant id is presented as one unavailable
state so the deep link cannot enumerate another tenant's imports.
## Quality, Deduplication, And Recovery ## Quality, Deduplication, And Recovery
Quality is evidence about a concrete contact point, separate from communication Quality is evidence about a concrete contact point, separate from communication
+1 -1
View File
@@ -224,7 +224,7 @@ Tasks:
- [ ] [On-premises Exchange connector profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/17) - [ ] [On-premises Exchange connector profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/17)
- [ ] [Google People contacts connector](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/18) - [ ] [Google People contacts connector](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/18)
- [ ] [Reusable CSV/XLSX import mapping profiles](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/19) - [ ] [Reusable CSV/XLSX import mapping profiles](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/19)
- [ ] [Bounded LDIF import profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/20) - [x] [Bounded LDIF import profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/20)
- [ ] [Selective and large-batch vCard workflows](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/21) - [ ] [Selective and large-batch vCard workflows](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/21)
- [x] classical address-list UI; reusable static/dynamic operational segments move - [x] classical address-list UI; reusable static/dynamic operational segments move
to `govoplan-dist-lists` to `govoplan-dist-lists`
+5
View File
@@ -26,6 +26,11 @@ imports, synchronization, quality review, and reversible merge operations.
source diagnostics and conflict state as explicitly stated. source diagnostics and conflict state as explicitly stated.
- Imports and synchronization separate preview from apply; incomplete external - Imports and synchronization separate preview from apply; incomplete external
reads never infer deletions. reads never infer deletions.
- vCard batch upload accepts multiple files, persists a non-mutating preview,
and requires an explicit create/update/ignore choice for every reviewed card.
Reload and cancellation preserve the pending plan; only applying the matching
plan hash mutates contacts. Scoped exports name the selected vCard version and
use deterministic ordering.
- Merge and communication-governance operations append auditable evidence and - Merge and communication-governance operations append auditable evidence and
never silently erase prior state. never silently erase prior state.
- Request feedback is rendered as a compact shared alert over the full-height - Request feedback is rendered as a compact shared alert over the full-height
+2 -2
View File
@@ -1,6 +1,6 @@
{ {
"name": "@govoplan/addresses-webui", "name": "@govoplan/addresses-webui",
"version": "0.1.18", "version": "0.1.23",
"private": true, "private": true,
"type": "module", "type": "module",
"main": "webui/src/index.ts", "main": "webui/src/index.ts",
@@ -18,7 +18,7 @@
"README.md" "README.md"
], ],
"peerDependencies": { "peerDependencies": {
"@govoplan/core-webui": "^0.1.18", "@govoplan/core-webui": "^0.1.45",
"lucide-react": "^1.23.0", "lucide-react": "^1.23.0",
"react": ">=19.2.7 <20", "react": ">=19.2.7 <20",
"react-dom": ">=19.2.7 <20", "react-dom": ">=19.2.7 <20",
+2 -2
View File
@@ -4,14 +4,14 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "govoplan-addresses" name = "govoplan-addresses"
version = "0.1.18" version = "0.1.23"
description = "GovOPlaN reusable address and recipient-source module." description = "GovOPlaN reusable address and recipient-source module."
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
authors = [{ name = "GovOPlaN" }] authors = [{ name = "GovOPlaN" }]
dependencies = [ dependencies = [
"defusedxml>=0.7.1", "defusedxml>=0.7.1",
"govoplan-core>=0.1.18", "govoplan-core>=0.1.45",
"ldap3>=2.9.1,<3", "ldap3>=2.9.1,<3",
"openpyxl>=3.1.5,<4", "openpyxl>=3.1.5,<4",
] ]
+3 -3
View File
@@ -587,9 +587,9 @@ class AddressImportRun(Base, TimestampMixin):
nullable=False, nullable=False,
index=True, index=True,
) )
profile_id: Mapped[str] = mapped_column( profile_id: Mapped[str | None] = mapped_column(
ForeignKey("addresses_import_profiles.id", ondelete="RESTRICT"), ForeignKey("addresses_import_profiles.id", ondelete="RESTRICT"),
nullable=False, nullable=True,
index=True, index=True,
) )
source_filename: Mapped[str] = mapped_column(String(500), nullable=False) source_filename: Mapped[str] = mapped_column(String(500), nullable=False)
@@ -607,7 +607,7 @@ class AddressImportRun(Base, TimestampMixin):
rolled_back_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True) rolled_back_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
address_book: Mapped[AddressBook] = relationship() address_book: Mapped[AddressBook] = relationship()
profile: Mapped[AddressImportProfile] = relationship() profile: Mapped[AddressImportProfile | None] = relationship()
__all__ = [ __all__ = [
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,55 @@
"""German translations for public structured documentation metadata."""
from __future__ import annotations
from typing import Any
GERMAN_STRUCTURED_TRANSLATIONS: dict[str, dict[str, Any]] = {'addresses.reference.fields-and-consequences': {'consequence_classes': {'archive': 'Entfernt das '
'Objekt aus '
'der '
'gewöhnlichen '
'Auswahl, '
'während die '
'verwaltete '
'Geschichte '
'und '
'Referenzen '
'beibehalten '
'werden.',
'governance_fact': 'Hinzufügen '
'oder '
'Beenden '
'einer '
'effektiv '
'datierten '
'Kommunikationsentscheidung '
'ohne '
'vorherige '
'Fakten '
'zu '
'löschen.',
'import_or_sync': 'Wendet '
'nur '
'einen '
'überprüften '
'Bounded '
'Plan '
'an und '
'behält '
'die '
'Quellenrevision, '
'Diagnose '
'und '
'Herkunft '
'bei.',
'merge': 'Repoints '
'verwaltet '
'Verweise auf '
'einen '
'Überlebenden '
'und behält '
'reversible '
'Redirect und '
'Provenienz '
'Nachweise.'}}}
@@ -6,7 +6,7 @@ from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator from pydantic import BaseModel, ConfigDict, Field, model_validator
AddressImportFormat = Literal["csv", "xlsx"] AddressImportFormat = Literal["csv", "xlsx", "ldif"]
AddressImportScope = Literal["user", "group", "tenant", "system"] AddressImportScope = Literal["user", "group", "tenant", "system"]
IMPORT_TARGET_FIELDS = frozenset( IMPORT_TARGET_FIELDS = frozenset(
@@ -41,6 +41,7 @@ class AddressImportConfiguration(BaseModel):
duplicate_source_key_policy: Literal["reject", "first", "last"] = "reject" duplicate_source_key_policy: Literal["reject", "first", "last"] = "reject"
existing_contact_policy: Literal["update", "ignore", "reject"] = "update" existing_contact_policy: Literal["update", "ignore", "reject"] = "update"
blank_value_policy: Literal["ignore", "clear", "reject"] = "ignore" blank_value_policy: Literal["ignore", "clear", "reject"] = "ignore"
ldif_change_record_policy: Literal["reject", "ignore", "treat_add_as_entry"] = "reject"
locale: str | None = Field(default=None, max_length=35) locale: str | None = Field(default=None, max_length=35)
default_tags: list[str] = Field(default_factory=list, max_length=100) default_tags: list[str] = Field(default_factory=list, max_length=100)
max_rows: int = Field(default=10_000, ge=1, le=10_000) max_rows: int = Field(default=10_000, ge=1, le=10_000)
@@ -128,7 +129,7 @@ class AddressImportDiagnosticResponse(BaseModel):
class AddressImportRunResponse(BaseModel): class AddressImportRunResponse(BaseModel):
id: str id: str
address_book_id: str address_book_id: str
profile_id: str profile_id: str | None
source_filename: str source_filename: str
source_format: str source_format: str
input_hash: str input_hash: str
@@ -151,6 +152,7 @@ class AddressImportCommitRequest(BaseModel):
class AddressImportRollbackRequest(BaseModel): class AddressImportRollbackRequest(BaseModel):
expected_plan_hash: str = Field(min_length=64, max_length=64)
reason: str = Field(min_length=3, max_length=2000) reason: str = Field(min_length=3, max_length=2000)
+482 -55
View File
@@ -3,19 +3,28 @@ from __future__ import annotations
import base64 import base64
import binascii import binascii
import csv import csv
import copy
import hashlib import hashlib
import json import json
from collections import Counter from collections import Counter
from datetime import UTC, datetime
from io import BytesIO, StringIO from io import BytesIO, StringIO
from typing import Any from typing import Any
from sqlalchemy import and_, false, or_ from sqlalchemy import and_, false, func, or_, select
from sqlalchemy.orm import Session from sqlalchemy.orm import Session, selectinload
from sqlalchemy.orm.attributes import set_committed_value
from govoplan_addresses.backend.db.models import ( from govoplan_addresses.backend.db.models import (
AddressImportProfile, AddressImportProfile,
AddressImportRun, AddressImportRun,
AddressListEntry,
Contact, Contact,
ContactChannelRule,
ContactEmail,
ContactPhone,
ContactPointQualityDecision,
ContactPostalAddress,
) )
from govoplan_addresses.backend.import_schemas import ( from govoplan_addresses.backend.import_schemas import (
AddressImportConfiguration, AddressImportConfiguration,
@@ -24,6 +33,7 @@ from govoplan_addresses.backend.import_schemas import (
AddressImportProfileUpdateRequest, AddressImportProfileUpdateRequest,
AddressImportRollbackRequest, AddressImportRollbackRequest,
) )
from govoplan_addresses.backend.ldif import parse_ldif_rows
from govoplan_addresses.backend.schemas import ( from govoplan_addresses.backend.schemas import (
ContactCreateRequest, ContactCreateRequest,
ContactEmailPayload, ContactEmailPayload,
@@ -33,6 +43,12 @@ from govoplan_addresses.backend.schemas import (
) )
from govoplan_addresses.backend.service import ( from govoplan_addresses.backend.service import (
AddressBookError, AddressBookError,
_contact_change_payload,
_record_address_contact_change,
_require_mutable_book,
_replace_emails,
_replace_phones,
_replace_postal_addresses,
create_contact, create_contact,
delete_contact, delete_contact,
get_visible_address_book, get_visible_address_book,
@@ -46,6 +62,16 @@ from govoplan_core.db.base import utcnow
MAX_IMPORT_BYTES = 10_000_000 MAX_IMPORT_BYTES = 10_000_000
MAX_IMPORT_COLUMNS = 200 MAX_IMPORT_COLUMNS = 200
CONTACT_LOOKUP_BATCH_SIZE = 250
ROLLBACK_SNAPSHOT_VERSION = 2
CONTACT_POINTS = {
"emails": (ContactEmail, ("email",), _replace_emails),
"phones": (ContactPhone, ("phone",), _replace_phones),
"postal_addresses": (
ContactPostalAddress, ("street", "postal_code", "locality", "region", "country"),
_replace_postal_addresses,
),
}
def _account_id(principal: ApiPrincipal) -> str: def _account_id(principal: ApiPrincipal) -> str:
@@ -199,6 +225,7 @@ def preview_address_import(
input_hash=input_hash, input_hash=input_hash,
rows=rows, rows=rows,
config=config, config=config,
source_format=profile.source_format,
) )
diagnostics = [*parse_diagnostics, *map_diagnostics] diagnostics = [*parse_diagnostics, *map_diagnostics]
statistics = dict(Counter(item["action"] for item in plan_data)) statistics = dict(Counter(item["action"] for item in plan_data))
@@ -239,15 +266,19 @@ def get_import_run(
session: Session, session: Session,
principal: ApiPrincipal, principal: ApiPrincipal,
run_id: str, run_id: str,
*,
lock: bool = False,
) -> AddressImportRun: ) -> AddressImportRun:
visible_book_ids = [book.id for book in _visible_import_books(session, principal)] visible_book_ids = [book.id for book in _visible_import_books(session, principal)]
if not visible_book_ids: if not visible_book_ids:
raise AddressBookError("Address import run not found.") raise AddressBookError("Address import run not found.")
run = ( query = (
session.query(AddressImportRun) session.query(AddressImportRun)
.filter(AddressImportRun.id == run_id, AddressImportRun.address_book_id.in_(visible_book_ids)) .filter(AddressImportRun.id == run_id, AddressImportRun.address_book_id.in_(visible_book_ids))
.one_or_none()
) )
if lock:
query = query.populate_existing().with_for_update()
run = query.one_or_none()
if run is None: if run is None:
raise AddressBookError("Address import run not found.") raise AddressBookError("Address import run not found.")
return run return run
@@ -260,13 +291,13 @@ def apply_address_import(
*, *,
expected_plan_hash: str, expected_plan_hash: str,
) -> AddressImportRun: ) -> AddressImportRun:
run = get_import_run(session, principal, run_id) run = get_import_run(session, principal, run_id, lock=True)
if run.plan_hash != expected_plan_hash:
raise AddressBookError("The reviewed import plan changed; create a new preview.")
if run.status == "applied": if run.status == "applied":
return run return run
if run.status != "previewed": if run.status != "previewed":
raise AddressBookError(f"Import run cannot be applied from status {run.status!r}.") raise AddressBookError(f"Import run cannot be applied from status {run.status!r}.")
if run.plan_hash != expected_plan_hash:
raise AddressBookError("The reviewed import plan changed; create a new preview.")
if any(item.get("severity") == "error" for item in run.diagnostics or []): if any(item.get("severity") == "error" for item in run.diagnostics or []):
raise AddressBookError("Import plans with error diagnostics cannot be applied.") raise AddressBookError("Import plans with error diagnostics cannot be applied.")
if any(item.get("action") == "conflict" for item in run.plan_data or []): if any(item.get("action") == "conflict" for item in run.plan_data or []):
@@ -274,12 +305,15 @@ def apply_address_import(
created_ids: list[str] = [] created_ids: list[str] = []
updated: list[dict[str, Any]] = [] updated: list[dict[str, Any]] = []
for item in run.plan_data or []: # SQLAlchemy JSON columns do not track nested mutations. Keep the persisted
# preview untouched until assigning a genuinely changed complete plan.
applied_plan = copy.deepcopy(run.plan_data or [])
for item in applied_plan:
action = item.get("action") action = item.get("action")
if action in {"ignored", "unchanged"}: if action in {"ignored", "unchanged"}:
continue continue
source_ref = str(item["source_ref"]) source_ref = str(item["source_ref"])
existing = _contact_by_source_ref(session, run.address_book_id, source_ref) existing = _contact_by_source_ref(session, run.address_book_id, source_ref, lock=True)
if action == "create": if action == "create":
if existing is not None and existing.deleted_at is None: if existing is not None and existing.deleted_at is None:
raise AddressBookError("A target contact appeared after preview; preview the import again.") raise AddressBookError("A target contact appeared after preview; preview the import again.")
@@ -297,18 +331,27 @@ def apply_address_import(
elif action == "update": elif action == "update":
if existing is None: if existing is None:
raise AddressBookError("An import target disappeared after preview; preview the import again.") raise AddressBookError("An import target disappeared after preview; preview the import again.")
_lock_contact_points(session, existing)
if _contact_hash(existing) != item.get("expected_contact_hash"): if _contact_hash(existing) != item.get("expected_contact_hash"):
raise AddressBookError( raise AddressBookError(
f'Contact "{existing.display_name}" changed after preview; preview the import again.' f'Contact "{existing.display_name}" changed after preview; preview the import again.'
) )
before = _contact_snapshot(existing) _require_mutable_book(existing.address_book)
before = {
"version": ROLLBACK_SNAPSHOT_VERSION,
"contact": copy.deepcopy(_contact_snapshot(existing)),
"deleted_at": _deleted_at_value(existing.deleted_at),
}
update_payload = ContactUpdateRequest.model_validate(item["payload"])
points = _prepare_import_points(session, existing, update_payload)
if existing.deleted_at is not None: if existing.deleted_at is not None:
restore_contact(session, principal, existing.id) restore_contact(session, principal, existing.id)
_apply_import_points(existing, points)
contact = update_contact( contact = update_contact(
session, session,
principal, principal,
existing.id, existing.id,
ContactUpdateRequest.model_validate(item["payload"]), _without_contact_points(update_payload),
) )
_stamp_import_contact(contact, run=run, item=item) _stamp_import_contact(contact, run=run, item=item)
session.flush() session.flush()
@@ -317,7 +360,7 @@ def apply_address_import(
run.status = "applied" run.status = "applied"
run.applied_at = utcnow() run.applied_at = utcnow()
run.plan_data = list(run.plan_data or []) run.plan_data = applied_plan
run.result_evidence = { run.result_evidence = {
"input_hash": run.input_hash, "input_hash": run.input_hash,
"plan_hash": run.plan_hash, "plan_hash": run.plan_hash,
@@ -335,7 +378,9 @@ def rollback_address_import(
run_id: str, run_id: str,
payload: AddressImportRollbackRequest, payload: AddressImportRollbackRequest,
) -> AddressImportRun: ) -> AddressImportRun:
run = get_import_run(session, principal, run_id) run = get_import_run(session, principal, run_id, lock=True)
if run.plan_hash != payload.expected_plan_hash:
raise AddressBookError("The reviewed import plan changed; reload the import run.")
if run.status == "rolled_back": if run.status == "rolled_back":
return run return run
if run.status != "applied": if run.status != "applied":
@@ -343,6 +388,12 @@ def rollback_address_import(
evidence = dict(run.result_evidence or {}) evidence = dict(run.result_evidence or {})
updated = list(evidence.get("updated_contacts") or []) updated = list(evidence.get("updated_contacts") or [])
created_ids = list(evidence.get("created_contact_ids") or []) created_ids = list(evidence.get("created_contact_ids") or [])
# Validate every before-image before touching any contact. Older runs did
# not record deletion state, so automatic recovery cannot infer it safely.
before_images = {
str(item["contact_id"]): _validated_rollback_snapshot(item.get("before"))
for item in updated
}
expected_hashes = { expected_hashes = {
str(item["contact_id"]): str(item["after_hash"]) str(item["contact_id"]): str(item["after_hash"])
@@ -355,12 +406,20 @@ def rollback_address_import(
if item.get("contact_id") in created_ids and item.get("after_hash") if item.get("contact_id") in created_ids and item.get("after_hash")
} }
) )
for contact_id, expected_hash in expected_hashes.items(): if not set(created_ids).issubset(expected_hashes):
contact = get_visible_contact(session, principal, contact_id, include_deleted=True) raise AddressBookError("Import rollback evidence is incomplete; automatic rollback is unsafe.")
for contact_id, expected_hash in sorted(expected_hashes.items()):
contact = get_visible_contact(session, principal, contact_id, include_deleted=True, lock=True)
_lock_contact_points(session, contact)
if contact.address_book_id != run.address_book_id:
raise AddressBookError("An import target moved to another address book; automatic rollback is unsafe.")
_require_mutable_book(contact.address_book)
if _contact_hash(contact) != expected_hash: if _contact_hash(contact) != expected_hash:
raise AddressBookError( raise AddressBookError(
f'Contact "{contact.display_name}" changed after import; automatic rollback is unsafe.' f'Contact "{contact.display_name}" changed after import; automatic rollback is unsafe.'
) )
if contact_id in before_images:
_validate_point_restoration(session, contact, before_images[contact_id][0]["points"])
for contact_id in created_ids: for contact_id in created_ids:
contact = get_visible_contact(session, principal, contact_id, include_deleted=True) contact = get_visible_contact(session, principal, contact_id, include_deleted=True)
@@ -368,22 +427,23 @@ def rollback_address_import(
delete_contact(session, principal, contact.id) delete_contact(session, principal, contact.id)
for item in updated: for item in updated:
contact = get_visible_contact(session, principal, str(item["contact_id"]), include_deleted=True) contact = get_visible_contact(session, principal, str(item["contact_id"]), include_deleted=True)
snapshot = dict(item["before"]) snapshot, previous_deleted_at = before_images[str(item["contact_id"])]
if contact.deleted_at is not None: if contact.deleted_at is not None:
restore_contact(session, principal, contact.id) restore_contact(session, principal, contact.id)
update_contact( previous = _contact_change_payload(contact, prefix="previous_")
session, _restore_contact_points(contact, snapshot["points"])
principal, # A validated stored before-image is not a fresh user edit: do not trim,
contact.id, # normalize or coerce it through the generic update path a second time.
ContactUpdateRequest.model_validate(snapshot["payload"]), for field in ("display_name", "given_name", "family_name", "organization", "role_title", "note", "tags"):
) setattr(contact, field, copy.deepcopy(snapshot["payload"][field]))
contact.source_kind = snapshot.get("source_kind") or "local" for field in ("source_kind", "source_ref", "source_revision", "source_payload_kind", "source_payload_raw", "provenance"):
contact.source_ref = snapshot.get("source_ref") setattr(contact, field, copy.deepcopy(snapshot[field]))
contact.source_revision = snapshot.get("source_revision") contact.metadata_ = copy.deepcopy(snapshot["metadata"])
contact.source_payload_kind = snapshot.get("source_payload_kind") contact.updated_by_account_id = _account_id(principal)
contact.source_payload_raw = snapshot.get("source_payload_raw") _record_address_contact_change(session, principal, contact=contact, operation="updated", previous=previous)
contact.provenance = dict(snapshot.get("provenance") or {}) if previous_deleted_at is not None:
contact.metadata_ = dict(snapshot.get("metadata") or {}) delete_contact(session, principal, contact.id)
contact.deleted_at = previous_deleted_at
run.status = "rolled_back" run.status = "rolled_back"
run.rolled_back_at = utcnow() run.rolled_back_at = utcnow()
@@ -499,13 +559,24 @@ def _parse_rows(
filename: str, filename: str,
source_format: str, source_format: str,
config: AddressImportConfiguration, config: AddressImportConfiguration,
) -> tuple[list[tuple[int, dict[str, str]]], list[dict[str, Any]]]: ) -> tuple[list[tuple[int, dict[str, Any]]], list[dict[str, Any]]]:
if source_format == "csv": if source_format == "csv":
return _parse_csv(raw, config=config) return _parse_csv(raw, config=config)
if source_format == "xlsx": if source_format == "xlsx":
if not filename.casefold().endswith(".xlsx"): if not filename.casefold().endswith(".xlsx"):
raise AddressBookError("XLSX imports require an .xlsx file; macros and legacy workbooks are not accepted.") raise AddressBookError("XLSX imports require an .xlsx file; macros and legacy workbooks are not accepted.")
return _parse_xlsx(raw, config=config) return _parse_xlsx(raw, config=config)
if source_format == "ldif":
if not filename.casefold().endswith((".ldif", ".ldi")):
raise AddressBookError("LDIF imports require an .ldif or .ldi file.")
try:
return parse_ldif_rows(
raw,
max_entries=config.max_rows,
change_record_policy=config.ldif_change_record_policy,
)
except ValueError as exc:
raise AddressBookError(str(exc)) from exc
raise AddressBookError(f"Unsupported address import format: {source_format!r}.") raise AddressBookError(f"Unsupported address import format: {source_format!r}.")
@@ -598,32 +669,81 @@ def _row_dict(headers: list[str], values: list[Any]) -> dict[str, str]:
} }
def _column_key(value: object, *, casefold: bool) -> str:
column = str(value).strip()
return column.casefold() if casefold else column
def _row_value(row: dict[str, Any], column: str, *, casefold: bool) -> object:
if column in row:
return row[column]
if not casefold:
return ""
expected = column.casefold()
for key, value in row.items():
if key.casefold() == expected:
return value
return ""
def _import_values(value: object) -> list[str]:
raw_values = value if isinstance(value, list) else [value]
return [str(item).strip() for item in raw_values if item is not None and str(item).strip()]
def _first_import_value(value: object) -> str:
values = _import_values(value)
return values[0] if values else ""
def _plan_rows( def _plan_rows(
session: Session, session: Session,
*, *,
book_id: str, book_id: str,
profile: AddressImportProfile, profile: AddressImportProfile,
input_hash: str, input_hash: str,
rows: list[tuple[int, dict[str, str]]], rows: list[tuple[int, dict[str, Any]]],
config: AddressImportConfiguration, config: AddressImportConfiguration,
source_format: str,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]: ) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
diagnostics: list[dict[str, Any]] = [] diagnostics: list[dict[str, Any]] = []
plan: list[dict[str, Any]] = [] plan: list[dict[str, Any]] = []
headers = set(rows[0][1]) if rows else set() casefold_columns = source_format == "ldif"
referenced_columns = set(config.field_mappings.values()) headers = {
_column_key(column, casefold=casefold_columns)
for column in (rows[0][1] if rows else set())
if not str(column).startswith("__")
}
referenced_columns = {
_column_key(column, casefold=casefold_columns)
for column in config.field_mappings.values()
}
key_column = config.source_key_column or config.field_mappings["source_key"]
normalized_key_column = _column_key(key_column, casefold=casefold_columns)
if config.source_key_column: if config.source_key_column:
referenced_columns.add(config.source_key_column) referenced_columns.add(_column_key(config.source_key_column, casefold=casefold_columns))
missing_columns = sorted(referenced_columns.difference(headers)) missing_columns = sorted(referenced_columns.difference(headers))
for column in missing_columns: for column in missing_columns:
diagnostics.append(_diagnostic("error", "missing_column", f'Configured column "{column}" is missing.', field=column)) optional_ldif_attribute = casefold_columns and column != normalized_key_column
if missing_columns: diagnostics.append(
_diagnostic(
"warning" if optional_ldif_attribute else "error",
"missing_attribute" if optional_ldif_attribute else "missing_column",
(
f'Configured LDIF attribute "{column}" is absent from this file.'
if optional_ldif_attribute
else f'Configured column "{column}" is missing.'
),
field=column,
)
)
if normalized_key_column in missing_columns or (missing_columns and not casefold_columns):
return [], diagnostics return [], diagnostics
key_column = config.source_key_column or config.field_mappings["source_key"] keyed_rows: list[tuple[int, dict[str, Any], str]] = []
keyed_rows: list[tuple[int, dict[str, str], str]] = []
key_counts: Counter[str] = Counter() key_counts: Counter[str] = Counter()
for row_number, row in rows: for row_number, row in rows:
key = row.get(key_column, "").strip() key = _first_import_value(_row_value(row, key_column, casefold=casefold_columns)).strip()
if not key: if not key:
diagnostics.append(_diagnostic("error", "missing_source_key", "Stable source key is blank.", row_number=row_number, field=key_column)) diagnostics.append(_diagnostic("error", "missing_source_key", "Stable source key is blank.", row_number=row_number, field=key_column))
plan.append(_plan_effect(row_number, "conflict", source_key=None, message="Stable source key is blank.")) plan.append(_plan_effect(row_number, "conflict", source_key=None, message="Stable source key is blank."))
@@ -637,6 +757,12 @@ def _plan_rows(
first_index.setdefault(key, index) first_index.setdefault(key, index)
last_index[key] = index last_index[key] = index
existing_contacts = _contacts_by_source_refs(
session,
book_id,
[f"import:{profile.profile_key}:{key}" for key in key_counts],
)
for index, (row_number, row, key) in enumerate(keyed_rows): for index, (row_number, row, key) in enumerate(keyed_rows):
if key_counts[key] > 1: if key_counts[key] > 1:
if config.duplicate_source_key_policy == "reject": if config.duplicate_source_key_policy == "reject":
@@ -649,11 +775,23 @@ def _plan_rows(
plan.append(_plan_effect(row_number, "ignored", source_key=key, message="Duplicate row ignored by profile policy.")) plan.append(_plan_effect(row_number, "ignored", source_key=key, message="Duplicate row ignored by profile policy."))
continue continue
mapped, row_diagnostics = _mapped_fields(row_number, row, config=config) mapped, row_diagnostics = _mapped_fields(
row_number,
row,
config=config,
casefold_columns=casefold_columns,
)
diagnostics.extend(row_diagnostics) diagnostics.extend(row_diagnostics)
source_ref = f"import:{profile.profile_key}:{key}" source_ref = f"import:{profile.profile_key}:{key}"
existing = _contact_by_source_ref(session, book_id, source_ref) existing = existing_contacts.get(source_ref)
payload = _payload_from_mapped(mapped, profile=profile, input_hash=input_hash, row_number=row_number, source_key=key) payload = _payload_from_mapped(
mapped,
profile=profile,
input_hash=input_hash,
row_number=row_number,
source_key=key,
source_row=row,
)
display_name = payload.get("display_name") or payload.get("email") or key display_name = payload.get("display_name") or payload.get("email") or key
if any(item["severity"] == "error" for item in row_diagnostics): if any(item["severity"] == "error" for item in row_diagnostics):
plan.append(_plan_effect(row_number, "conflict", source_key=key, display_name=display_name, source_ref=source_ref, payload=payload, message="Row validation failed.")) plan.append(_plan_effect(row_number, "conflict", source_key=key, display_name=display_name, source_ref=source_ref, payload=payload, message="Row validation failed."))
@@ -687,23 +825,46 @@ def _plan_rows(
def _mapped_fields( def _mapped_fields(
row_number: int, row_number: int,
row: dict[str, str], row: dict[str, Any],
*, *,
config: AddressImportConfiguration, config: AddressImportConfiguration,
casefold_columns: bool = False,
) -> tuple[dict[str, Any], list[dict[str, Any]]]: ) -> tuple[dict[str, Any], list[dict[str, Any]]]:
mapped: dict[str, Any] = {} mapped: dict[str, Any] = {}
diagnostics: list[dict[str, Any]] = [] diagnostics: list[dict[str, Any]] = []
for target, column in config.field_mappings.items(): for target, column in config.field_mappings.items():
if target == "source_key": if target == "source_key":
continue continue
value = row.get(column, "").strip() values = _import_values(_row_value(row, column, casefold=casefold_columns))
if not value: if not values:
if config.blank_value_policy == "reject": if config.blank_value_policy == "reject":
diagnostics.append(_diagnostic("error", "blank_value", f'Column "{column}" is blank.', row_number=row_number, field=target)) diagnostics.append(_diagnostic("error", "blank_value", f'Column "{column}" is blank.', row_number=row_number, field=target))
elif config.blank_value_policy == "clear": elif config.blank_value_policy == "clear":
mapped[target] = [] if target == "tags" else None mapped[target] = [] if target == "tags" else None
continue continue
mapped[target] = [item.strip() for item in value.split(",") if item.strip()] if target == "tags" else value if target == "tags":
mapped[target] = list(
dict.fromkeys(
item.strip()
for value in values
for item in value.split(",")
if item.strip()
)
)
elif target in {"email", "phone"}:
mapped[target] = values if len(values) > 1 else values[0]
else:
mapped[target] = values[0]
if len(values) > 1:
diagnostics.append(
_diagnostic(
"warning",
"multiple_values_truncated",
f'Attribute "{column}" has multiple values; only the first maps to {target!r}.',
row_number=row_number,
field=target,
)
)
if config.default_tags: if config.default_tags:
mapped["tags"] = list(dict.fromkeys([*(mapped.get("tags") or []), *config.default_tags])) mapped["tags"] = list(dict.fromkeys([*(mapped.get("tags") or []), *config.default_tags]))
if not any(mapped.get(name) for name in ("display_name", "given_name", "family_name", "email", "organization")): if not any(mapped.get(name) for name in ("display_name", "given_name", "family_name", "email", "organization")):
@@ -718,8 +879,22 @@ def _payload_from_mapped(
input_hash: str, input_hash: str,
row_number: int, row_number: int,
source_key: str, source_key: str,
source_row: dict[str, Any] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
display_name = mapped.get("display_name") or " ".join(filter(None, [mapped.get("given_name"), mapped.get("family_name")])) or mapped.get("email") or mapped.get("organization") display_name = (
_first_import_value(mapped.get("display_name"))
or " ".join(
filter(
None,
[
_first_import_value(mapped.get("given_name")),
_first_import_value(mapped.get("family_name")),
],
)
)
or _first_import_value(mapped.get("email"))
or _first_import_value(mapped.get("organization"))
)
payload: dict[str, Any] = { payload: dict[str, Any] = {
key: mapped.get(key) key: mapped.get(key)
for key in ("given_name", "family_name", "organization", "role_title", "note", "tags") for key in ("given_name", "family_name", "organization", "role_title", "note", "tags")
@@ -727,9 +902,17 @@ def _payload_from_mapped(
} }
payload["display_name"] = display_name payload["display_name"] = display_name
if "email" in mapped: if "email" in mapped:
payload["emails"] = [] if mapped["email"] is None else [ContactEmailPayload(email=mapped["email"], is_primary=True).model_dump(mode="json")] email_values = _import_values(mapped["email"])
payload["emails"] = [
ContactEmailPayload(email=value, is_primary=index == 0).model_dump(mode="json")
for index, value in enumerate(email_values)
]
if "phone" in mapped: if "phone" in mapped:
payload["phones"] = [] if mapped["phone"] is None else [ContactPhonePayload(phone=mapped["phone"], is_primary=True).model_dump(mode="json")] phone_values = _import_values(mapped["phone"])
payload["phones"] = [
ContactPhonePayload(phone=value, is_primary=index == 0).model_dump(mode="json")
for index, value in enumerate(phone_values)
]
postal_keys = {"street", "postal_code", "locality", "region", "country"} postal_keys = {"street", "postal_code", "locality", "region", "country"}
if postal_keys.intersection(mapped): if postal_keys.intersection(mapped):
postal = {key: mapped.get(key) for key in postal_keys if key in mapped} postal = {key: mapped.get(key) for key in postal_keys if key in mapped}
@@ -744,6 +927,8 @@ def _payload_from_mapped(
"source_key": source_key, "source_key": source_key,
"locale": profile.configuration.get("locale"), "locale": profile.configuration.get("locale"),
"visibility": mapped.get("visibility"), "visibility": mapped.get("visibility"),
"source_format": profile.source_format,
"source_record_hash": (source_row or {}).get("__ldif_record_hash"),
} }
} }
return ContactCreateRequest.model_validate(payload).model_dump( return ContactCreateRequest.model_validate(payload).model_dump(
@@ -768,7 +953,19 @@ def _changed_fields(contact: Contact, mapped: dict[str, Any]) -> list[str]:
if contact.postal_addresses: if contact.postal_addresses:
postal = contact.postal_addresses[0] postal = contact.postal_addresses[0]
current.update({key: getattr(postal, key) for key in ("street", "postal_code", "locality", "region", "country")}) current.update({key: getattr(postal, key) for key in ("street", "postal_code", "locality", "region", "country")})
return sorted(key for key, value in mapped.items() if key != "visibility" and current.get(key) != value) changed: list[str] = []
for key, value in mapped.items():
if key == "visibility":
continue
if key == "email" and isinstance(value, list):
current_value: Any = [item.email for item in contact.emails]
elif key == "phone" and isinstance(value, list):
current_value = [item.phone for item in contact.phones]
else:
current_value = current.get(key)
if current_value != value:
changed.append(key)
return sorted(changed)
def _plan_effect( def _plan_effect(
@@ -816,13 +1013,79 @@ def _diagnostic(
} }
def _contact_by_source_ref(session: Session, book_id: str, source_ref: str) -> Contact | None: def _contacts_by_source_refs(session: Session, book_id: str, source_refs: list[str]) -> dict[str, Contact]:
return ( contacts: dict[str, Contact] = {}
for offset in range(0, len(source_refs), CONTACT_LOOKUP_BATCH_SIZE):
candidates = (
select(
Contact.id,
func.row_number().over(
partition_by=Contact.source_ref,
order_by=(Contact.created_at.asc(), Contact.id.asc()),
).label("source_position"),
)
.where(
Contact.address_book_id == book_id,
Contact.source_ref.in_(source_refs[offset : offset + CONTACT_LOOKUP_BATCH_SIZE]),
)
.subquery()
)
rows = (
session.query(Contact)
.join(candidates, candidates.c.id == Contact.id)
.filter(candidates.c.source_position == 1)
.options(selectinload(Contact.emails), selectinload(Contact.phones), selectinload(Contact.postal_addresses))
.populate_existing()
.order_by(Contact.created_at.asc(), Contact.id.asc())
.all()
)
for contact in rows:
# Preserve the historical first-match choice for duplicate stored
# source references, independently of batch/database row order.
contacts.setdefault(str(contact.source_ref), contact)
return contacts
def _contact_by_source_ref(session: Session, book_id: str, source_ref: str, *, lock: bool = False) -> Contact | None:
query = (
session.query(Contact) session.query(Contact)
.filter(Contact.address_book_id == book_id, Contact.source_ref == source_ref) .filter(Contact.address_book_id == book_id, Contact.source_ref == source_ref)
.order_by(Contact.created_at.asc(), Contact.id.asc()) .order_by(Contact.created_at.asc(), Contact.id.asc())
.first()
) )
if lock:
query = query.populate_existing().with_for_update()
return query.first()
def _validated_rollback_snapshot(value: object) -> tuple[dict[str, Any], datetime | None]:
if (
not isinstance(value, dict)
or value.get("version") != ROLLBACK_SNAPSHOT_VERSION
or "deleted_at" not in value
or not isinstance(value.get("contact"), dict)
):
raise AddressBookError(
"This import has incomplete legacy rollback evidence; automatic rollback is unsafe. "
"Review and reconcile the affected contacts manually."
)
snapshot = value["contact"]
required_fields = {
"display_name", "given_name", "family_name", "organization", "role_title",
"note", "tags", "emails", "phones", "postal_addresses", "provenance",
}
if (
not isinstance(snapshot.get("payload"), dict)
or not required_fields.issubset(snapshot["payload"])
or not {"source_kind", "source_ref", "source_revision", "source_payload_kind", "source_payload_raw", "provenance", "metadata"}.issubset(snapshot)
):
raise AddressBookError("Import rollback evidence is incomplete; automatic rollback is unsafe.")
try:
deleted_at = datetime.fromisoformat(value["deleted_at"]) if value["deleted_at"] is not None else None
ContactUpdateRequest.model_validate(snapshot["payload"])
_validate_point_snapshot(snapshot.get("points"))
except (TypeError, ValueError) as exc:
raise AddressBookError("Import rollback evidence is invalid; automatic rollback is unsafe.") from exc
return snapshot, deleted_at
def _stamp_import_contact(contact: Contact, *, run: AddressImportRun, item: dict[str, Any]) -> None: def _stamp_import_contact(contact: Contact, *, run: AddressImportRun, item: dict[str, Any]) -> None:
@@ -872,12 +1135,176 @@ def _contact_snapshot(contact: Contact) -> dict[str, Any]:
"source_payload_kind": contact.source_payload_kind, "source_payload_kind": contact.source_payload_kind,
"source_payload_raw": contact.source_payload_raw, "source_payload_raw": contact.source_payload_raw,
"provenance": dict(contact.provenance or {}), "provenance": dict(contact.provenance or {}),
"metadata": dict(contact.metadata_ or {}), "metadata": copy.deepcopy(contact.metadata_),
"points": {
name: [_point_snapshot(point) for point in getattr(contact, name)]
for name in CONTACT_POINTS
},
} }
def _contact_hash(contact: Contact) -> str: def _contact_hash(contact: Contact) -> str:
return _hash_json({**_contact_snapshot(contact), "deleted_at": contact.deleted_at.isoformat() if contact.deleted_at else None}) return _hash_json({**_contact_snapshot(contact), "deleted_at": _deleted_at_value(contact.deleted_at)})
def _point_snapshot(point: ContactEmail | ContactPhone | ContactPostalAddress) -> dict[str, Any]:
# Include every persisted evidence field, including identity, originals,
# normalization, provenance, ordering and timestamps, but never the parent FK.
return {
column.key: (
_deleted_at_value(getattr(point, column.key))
if column.key in {"created_at", "updated_at"}
else copy.deepcopy(getattr(point, column.key))
)
for column in point.__table__.columns if column.key != "contact_id"
}
def _lock_contact_points(session: Session, contact: Contact) -> None:
# The parent is locked by the caller. Point-only evidence updates need their
# own row locks; they need not update the parent row on PostgreSQL.
for name, (model, _identity, _replace) in CONTACT_POINTS.items():
positions = {point.id: index for index, point in enumerate(getattr(contact, name))}
points = session.query(model).filter(model.contact_id == contact.id).order_by(
model.id.asc(),
).populate_existing().with_for_update().all()
# Lock acquisition order is not presentation order. Preserve the
# relationship's existing order for tied indexes; new identities still
# enter the hash and therefore cannot evade the reviewed-state guard.
points.sort(key=lambda point: (point.order_index, positions.get(point.id, len(positions)), point.id))
set_committed_value(contact, name, points)
def _validate_point_snapshot(value: object) -> None:
if not isinstance(value, dict) or set(value) != set(CONTACT_POINTS):
raise ValueError("Missing complete contact-point evidence")
for name, (model, _identity, _replace) in CONTACT_POINTS.items():
points = value[name]
columns = {column.key: column for column in model.__table__.columns if column.key != "contact_id"}
ids: set[str] = set()
if not isinstance(points, list):
raise ValueError("Invalid contact-point collection")
for point in points:
if not isinstance(point, dict) or set(point) != set(columns):
raise ValueError("Incomplete contact-point evidence")
for key, column in columns.items():
field = point[key]
if field is None and column.nullable:
continue
if key in {"created_at", "updated_at"}:
datetime.fromisoformat(field)
elif type(field) is not column.type.python_type:
raise ValueError("Invalid contact-point evidence type")
if not point["id"] or point["id"] in ids:
raise ValueError("Invalid contact-point identity")
ids.add(point["id"])
def _without_contact_points(payload: ContactUpdateRequest) -> ContactUpdateRequest:
return ContactUpdateRequest.model_validate(
payload.model_dump(exclude=set(CONTACT_POINTS), exclude_unset=True)
)
def _require_unreferenced_points(session: Session, contact: Contact, removed: set[str]) -> None:
if not removed:
return
# Do not detach address-list selections or silently retarget point-specific
# consent/quality decisions when an import removes or replaces a value.
referenced = session.query(AddressListEntry.id).filter(
AddressListEntry.contact_id == contact.id,
or_(AddressListEntry.contact_email_id.in_(removed), AddressListEntry.contact_postal_address_id.in_(removed)),
).first()
for model in (ContactChannelRule, ContactPointQualityDecision):
if referenced is not None:
break
referenced = session.query(model.id).filter(
model.contact_id == contact.id, model.contact_point_id.in_(removed),
).first()
if referenced is not None:
raise AddressBookError(
"An affected contact point has address-list or governance references; "
"review and reconcile those references before applying or rolling back this import."
)
def _prepare_import_points(session: Session, contact: Contact, payload: ContactUpdateRequest):
staged = Contact(
source_kind=contact.source_kind, source_ref=contact.source_ref,
source_revision=contact.source_revision,
created_by_account_id=contact.created_by_account_id,
updated_by_account_id=contact.updated_by_account_id,
)
result = {}
removed: set[str] = set()
for name, (_model, identity, replace) in CONTACT_POINTS.items():
if name not in payload.model_fields_set:
continue
replace(staged, getattr(payload, name) or [])
available = list(getattr(contact, name))
matches = []
for candidate in list(getattr(staged, name)):
# Detach the normalization-only parent before a new point enters the
# persistent collection; otherwise save-update cascade can enlist it.
candidate.contact = None
original = next((point for point in available if all(
getattr(point, key) == getattr(candidate, key) for key in identity
)), None)
if original is not None:
available.remove(original)
matches.append((original, candidate))
removed.update(point.id for point in available)
result[name] = matches
_require_unreferenced_points(session, contact, removed)
return result
def _apply_import_points(contact: Contact, prepared) -> None:
for name, matches in prepared.items():
points = []
for original, candidate in matches:
if original is None:
points.append(candidate)
else:
# Same value retains its exact original/normalized evidence and
# identity; only explicitly imported presentation fields change.
for field in ("label", "is_primary", "order_index"):
setattr(original, field, getattr(candidate, field))
points.append(original)
setattr(contact, name, points)
def _validate_point_restoration(session: Session, contact: Contact, points: dict[str, Any]) -> None:
removed: set[str] = set()
for name, (model, _identity, _replace) in CONTACT_POINTS.items():
desired = {point["id"] for point in points[name]}
removed.update(point.id for point in getattr(contact, name) if point.id not in desired)
# A deleted original identity must never be reclaimed from another contact.
if desired and session.query(model.id).filter(
model.id.in_(desired), model.contact_id != contact.id,
).first() is not None:
raise AddressBookError("Contact-point identity changed; automatic rollback is unsafe.")
_require_unreferenced_points(session, contact, removed)
def _restore_contact_points(contact: Contact, snapshots: dict[str, Any]) -> None:
for name, (model, _identity, _replace) in CONTACT_POINTS.items():
existing = {point.id: point for point in getattr(contact, name)}
restored = []
for snapshot in snapshots[name]:
point = existing.get(snapshot["id"])
if point is None:
point = model()
for key, value in snapshot.items():
setattr(point, key, datetime.fromisoformat(value) if key in {"created_at", "updated_at"} else copy.deepcopy(value))
restored.append(point)
setattr(contact, name, restored)
def _deleted_at_value(value: datetime | None) -> str | None:
if value is None:
return None
return (value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)).isoformat()
def _hash_json(value: object) -> str: def _hash_json(value: object) -> str:
+286
View File
@@ -0,0 +1,286 @@
from __future__ import annotations
import base64
import binascii
import hashlib
from collections.abc import Iterable
from io import BytesIO
from typing import Any, Literal
LdifChangeRecordPolicy = Literal["reject", "ignore", "treat_add_as_entry"]
MAX_LDIF_ATTRIBUTES = 500
MAX_LDIF_VALUES_PER_ATTRIBUTE = 100
MAX_LDIF_LOGICAL_LINE_BYTES = 1_000_000
def parse_ldif_rows(
raw: bytes,
*,
max_entries: int,
change_record_policy: LdifChangeRecordPolicy = "reject",
) -> tuple[list[tuple[int, dict[str, Any]]], list[dict[str, Any]]]:
"""Parse bounded LDIF entries without fetching URL or decoding binary values."""
rows: list[tuple[int, dict[str, Any]]] = []
diagnostics: list[dict[str, Any]] = []
record_lines: list[tuple[int, bytes]] = []
def finish_record() -> None:
if not record_lines:
return
row_number = record_lines[0][0]
row, record_diagnostics = _parse_record(record_lines)
diagnostics.extend(record_diagnostics)
record_lines.clear()
if not row:
return
if set(row) == {"version"} and _first(row.get("version")) == "1":
return
change_type = _first(row.get("changetype")).casefold()
if change_type:
if change_record_policy == "ignore":
diagnostics.append(
_diagnostic(
"warning",
"ldif_change_record_ignored",
f"LDIF change record {change_type!r} was ignored by profile policy.",
row_number=row_number,
field="changetype",
)
)
return
if change_record_policy != "treat_add_as_entry" or change_type != "add":
diagnostics.append(
_diagnostic(
"error",
"ldif_change_record_rejected",
f"LDIF change record {change_type!r} is not permitted by the profile policy.",
row_number=row_number,
field="changetype",
)
)
return
diagnostics.append(
_diagnostic(
"info",
"ldif_add_record_imported",
"LDIF add change record is treated as a static contact entry by profile policy.",
row_number=row_number,
field="changetype",
)
)
row.pop("changetype", None)
row["__ldif_record_hash"] = hashlib.sha256(_canonical_record(row)).hexdigest()
rows.append((row_number, row))
if len(rows) > max_entries:
raise ValueError(f"LDIF exceeds the configured {max_entries}-entry limit.")
for line_number, logical_line in _logical_lines(raw):
if not logical_line:
finish_record()
continue
if logical_line.startswith(b"#"):
continue
record_lines.append((line_number, logical_line))
finish_record()
if not rows and not any(item["severity"] == "error" for item in diagnostics):
diagnostics.append(_diagnostic("warning", "ldif_no_entries", "No importable LDIF entries were found."))
return rows, diagnostics
def _logical_lines(raw: bytes) -> Iterable[tuple[int, bytes]]:
current: bytearray | None = None
start_line = 0
for line_number, physical_with_ending in enumerate(BytesIO(raw), start=1):
physical = physical_with_ending.rstrip(b"\r\n")
if physical.startswith(b" "):
if current is None:
yield line_number, b"!invalid-fold-without-preceding-line"
continue
current.extend(physical[1:])
if len(current) > MAX_LDIF_LOGICAL_LINE_BYTES:
raise ValueError(f"LDIF logical line starting at {start_line} exceeds the size limit.")
continue
if current is not None:
yield start_line, bytes(current)
current = bytearray(physical)
start_line = line_number
if len(current) > MAX_LDIF_LOGICAL_LINE_BYTES:
raise ValueError(f"LDIF logical line {line_number} exceeds the size limit.")
if current is not None:
yield start_line, bytes(current)
def _parse_record(
lines: list[tuple[int, bytes]],
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
row: dict[str, list[str]] = {}
diagnostics: list[dict[str, Any]] = []
for line_number, line in lines:
if line == b"!invalid-fold-without-preceding-line":
diagnostics.append(
_diagnostic(
"error",
"ldif_invalid_fold",
"LDIF continuation line has no preceding attribute.",
row_number=line_number,
)
)
continue
if b":" not in line:
diagnostics.append(
_diagnostic(
"error",
"ldif_invalid_line",
"LDIF line is missing the attribute separator.",
row_number=line_number,
)
)
continue
if line == b"-":
# Attribute-operation separators are meaningful only inside change
# records, whose enclosing policy is evaluated after the record.
continue
raw_name, raw_value = line.split(b":", 1)
try:
name_parts = [part.strip().casefold() for part in raw_name.decode("ascii").split(";")]
name = name_parts[0]
except UnicodeDecodeError:
name_parts = []
name = ""
if not name or any(character.isspace() for character in name):
diagnostics.append(
_diagnostic(
"error",
"ldif_invalid_attribute",
"LDIF attribute name is invalid.",
row_number=line_number,
)
)
continue
if "binary" in name_parts[1:]:
diagnostics.append(
_diagnostic(
"warning",
"ldif_binary_value_ignored",
f"Binary LDIF attribute {name!r} was ignored; binary data is never projected into contacts.",
row_number=line_number,
field=name,
)
)
continue
if name not in row and len(row) >= MAX_LDIF_ATTRIBUTES:
diagnostics.append(
_diagnostic(
"error",
"ldif_too_many_attributes",
f"LDIF entry exceeds the {MAX_LDIF_ATTRIBUTES}-attribute limit.",
row_number=line_number,
)
)
continue
value, value_diagnostic = _decode_value(raw_value, attribute=name, line_number=line_number)
if value_diagnostic is not None:
diagnostics.append(value_diagnostic)
if value is None:
continue
values = row.setdefault(name, [])
if len(values) >= MAX_LDIF_VALUES_PER_ATTRIBUTE:
diagnostics.append(
_diagnostic(
"error",
"ldif_too_many_values",
f"LDIF attribute {name!r} exceeds the value limit.",
row_number=line_number,
field=name,
)
)
continue
values.append(value)
return row, diagnostics
def _decode_value(
raw_value: bytes,
*,
attribute: str,
line_number: int,
) -> tuple[str | None, dict[str, Any] | None]:
if raw_value.startswith(b":"):
encoded = raw_value[1:].lstrip(b" ")
try:
decoded = base64.b64decode(encoded, validate=True)
except (binascii.Error, ValueError):
return None, _diagnostic(
"error",
"ldif_invalid_base64",
f"LDIF attribute {attribute!r} contains invalid base64.",
row_number=line_number,
field=attribute,
)
try:
return decoded.decode("utf-8"), None
except UnicodeDecodeError:
return None, _diagnostic(
"warning",
"ldif_binary_value_ignored",
f"Binary LDIF attribute {attribute!r} was ignored; binary data is never projected into contacts.",
row_number=line_number,
field=attribute,
)
if raw_value.startswith(b"<"):
return None, _diagnostic(
"warning",
"ldif_url_value_ignored",
f"External LDIF URL value for {attribute!r} was ignored; imports never fetch referenced content.",
row_number=line_number,
field=attribute,
)
value_bytes = raw_value[1:] if raw_value.startswith(b" ") else raw_value
try:
return value_bytes.decode("utf-8"), None
except UnicodeDecodeError:
return None, _diagnostic(
"error",
"ldif_invalid_utf8",
f"LDIF attribute {attribute!r} is not valid UTF-8.",
row_number=line_number,
field=attribute,
)
def _canonical_record(row: dict[str, Any]) -> bytes:
lines = []
for name in sorted(key for key in row if not key.startswith("__")):
values = row[name] if isinstance(row[name], list) else [row[name]]
lines.extend(f"{name}:{value}" for value in values)
return "\n".join(lines).encode("utf-8")
def _first(value: object) -> str:
if isinstance(value, list):
return str(value[0]) if value else ""
return str(value or "")
def _diagnostic(
severity: str,
code: str,
message: str,
*,
row_number: int | None = None,
field: str | None = None,
) -> dict[str, Any]:
return {
"severity": severity,
"code": code,
"message": message,
"row_number": row_number,
"field": field,
"details": {},
}
__all__ = ["LdifChangeRecordPolicy", "parse_ldif_rows"]
+648 -70
View File
@@ -1,5 +1,8 @@
from __future__ import annotations from __future__ import annotations
from govoplan_core.core.modules import with_documentation_structured_translations
from govoplan_addresses.backend.german_structured_documentation import GERMAN_STRUCTURED_TRANSLATIONS
from dataclasses import replace from dataclasses import replace
from pathlib import Path from pathlib import Path
@@ -11,12 +14,22 @@ from govoplan_addresses.backend.capabilities import (
CAPABILITY_ADDRESSES_RECIPIENT_SOURCE, CAPABILITY_ADDRESSES_RECIPIENT_SOURCE,
) )
from govoplan_addresses.backend.db import models as addresses_models # noqa: F401 - populate address ORM metadata from govoplan_addresses.backend.db import models as addresses_models # noqa: F401 - populate address ORM metadata
from govoplan_core.core.access import CAPABILITY_AUTH_PERMISSION_EVALUATOR, CAPABILITY_AUTH_PRINCIPAL_RESOLVER from govoplan_core.core.access import (
from govoplan_core.core.contact_points import CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION CAPABILITY_AUTH_PERMISSION_EVALUATOR,
from govoplan_core.core.module_guards import drop_table_retirement_provider, persistent_table_uninstall_guard CAPABILITY_AUTH_PRINCIPAL_RESOLVER,
)
from govoplan_core.core.contact_points import (
CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION,
)
from govoplan_core.core.module_guards import (
drop_table_retirement_provider,
persistent_table_uninstall_guard,
)
from govoplan_core.core.people import CAPABILITY_ADDRESSES_PEOPLE_SEARCH from govoplan_core.core.people import CAPABILITY_ADDRESSES_PEOPLE_SEARCH
from govoplan_core.core.distribution_lists import CAPABILITY_RECIPIENT_CHANNEL_FACTS from govoplan_core.core.distribution_lists import CAPABILITY_RECIPIENT_CHANNEL_FACTS
from govoplan_core.core.modules import ( from govoplan_core.core.modules import (
CapabilityDocumentation,
DocumentationCondition,
DocumentationTopic, DocumentationTopic,
FrontendModule, FrontendModule,
FrontendRoute, FrontendRoute,
@@ -26,6 +39,7 @@ from govoplan_core.core.modules import (
ModuleManifest, ModuleManifest,
NavItem, NavItem,
PermissionDefinition, PermissionDefinition,
ProductAreaContribution,
RoleTemplate, RoleTemplate,
) )
from govoplan_core.core.provider_governance import ( from govoplan_core.core.provider_governance import (
@@ -37,6 +51,7 @@ from govoplan_core.core.provider_governance import (
) )
from govoplan_core.core.views import ViewSurface from govoplan_core.core.views import ViewSurface
from govoplan_core.db.base import Base from govoplan_core.db.base import Base
from govoplan_addresses.backend.dsar_provider import ADDRESSES_DSAR_CAPABILITY
from govoplan_addresses.backend.provider_state import ( from govoplan_addresses.backend.provider_state import (
CARDDAV_PROVIDER_ID, CARDDAV_PROVIDER_ID,
LDAP_PROVIDER_ID, LDAP_PROVIDER_ID,
@@ -45,6 +60,13 @@ from govoplan_addresses.backend.provider_state import (
) )
def _addresses_dsar_provider(context: ModuleContext) -> object:
del context
from govoplan_addresses.backend.dsar_provider import AddressesDsarProvider
return AddressesDsarProvider()
_addresses_table_retirement_provider = drop_table_retirement_provider( _addresses_table_retirement_provider = drop_table_retirement_provider(
addresses_models.AddressImportRun, addresses_models.AddressImportRun,
addresses_models.AddressImportProfile, addresses_models.AddressImportProfile,
@@ -76,10 +98,18 @@ def _addresses_retirement_provider(session: object | None, module_id: str):
return plan return plan
def executor(execute_session: object, execute_module_id: str) -> None: def executor(execute_session: object, execute_module_id: str) -> None:
if not hasattr(execute_session, "get_bind") or not hasattr(execute_session, "query"): if not hasattr(execute_session, "get_bind") or not hasattr(
raise RuntimeError("No database session is available for Addresses credential retirement.") execute_session, "query"
if inspect(execute_session.get_bind()).has_table(addresses_models.AddressSyncSource.__tablename__): ):
from govoplan_addresses.backend.service import audit_address_credentials_for_retirement raise RuntimeError(
"No database session is available for Addresses credential retirement."
)
if inspect(execute_session.get_bind()).has_table(
addresses_models.AddressSyncSource.__tablename__
):
from govoplan_addresses.backend.service import (
audit_address_credentials_for_retirement,
)
audit_address_credentials_for_retirement(execute_session) audit_address_credentials_for_retirement(execute_session)
base_executor(execute_session, execute_module_id) base_executor(execute_session, execute_module_id)
@@ -109,21 +139,77 @@ def _permission(scope: str, label: str, description: str) -> PermissionDefinitio
PERMISSIONS = ( PERMISSIONS = (
_permission("addresses:address_book:read", "View address books", "List address books visible to the current principal."), _permission(
_permission("addresses:address_book:write", "Manage address books", "Create and edit local address books."), "addresses:address_book:read",
_permission("addresses:address_book:delete", "Delete address books", "Soft-delete local address books."), "View address books",
_permission("addresses:address_book:admin", "Administer address books", "Manage system-scoped address books and future sync sources."), "List address books visible to the current principal.",
_permission("addresses:address_list:read", "View address lists", "List reusable address lists and their entries."), ),
_permission("addresses:address_list:write", "Manage address lists", "Create and edit reusable address lists."), _permission(
_permission("addresses:address_list:delete", "Delete address lists", "Soft-delete reusable address lists."), "addresses:address_book:write",
_permission("addresses:contact:read", "View contacts", "List and lookup contacts in visible address books."), "Manage address books",
_permission("addresses:contact:write", "Manage contacts", "Create and edit local contacts."), "Create and edit local address books.",
_permission("addresses:contact:delete", "Delete contacts", "Soft-delete local contacts."), ),
_permission("addresses:governance:read", "View communication governance", "Inspect effective-dated consent, suppression, and channel-preference facts."), _permission(
_permission("addresses:governance:write", "Manage communication governance", "Record and end consent, suppression, and channel-preference facts."), "addresses:address_book:delete",
_permission("addresses:sync:read", "View address sync", "Inspect address sync sources, conflicts, tombstones, and diagnostics."), "Delete address books",
_permission("addresses:sync:write", "Manage address sync", "Bind address books to external sources and record sync state."), "Soft-delete local address books.",
_permission("addresses:sync:admin", "Administer address sync", "Administer address sync connectors and future destructive sync operations."), ),
_permission(
"addresses:address_book:admin",
"Administer address books",
"Manage system-scoped address books and future sync sources.",
),
_permission(
"addresses:address_list:read",
"View address lists",
"List reusable address lists and their entries.",
),
_permission(
"addresses:address_list:write",
"Manage address lists",
"Create and edit reusable address lists.",
),
_permission(
"addresses:address_list:delete",
"Delete address lists",
"Soft-delete reusable address lists.",
),
_permission(
"addresses:contact:read",
"View contacts",
"List and lookup contacts in visible address books.",
),
_permission(
"addresses:contact:write", "Manage contacts", "Create and edit local contacts."
),
_permission(
"addresses:contact:delete", "Delete contacts", "Soft-delete local contacts."
),
_permission(
"addresses:governance:read",
"View communication governance",
"Inspect effective-dated consent, suppression, and channel-preference facts.",
),
_permission(
"addresses:governance:write",
"Manage communication governance",
"Record and end consent, suppression, and channel-preference facts.",
),
_permission(
"addresses:sync:read",
"View address sync",
"Inspect address sync sources, conflicts, tombstones, and diagnostics.",
),
_permission(
"addresses:sync:write",
"Manage address sync",
"Bind address books to external sources and record sync state.",
),
_permission(
"addresses:sync:admin",
"Administer address sync",
"Administer address sync connectors and future destructive sync operations.",
),
) )
@@ -152,7 +238,13 @@ ROLE_TEMPLATES = (
slug="address_book_reader", slug="address_book_reader",
name="Address book reader", name="Address book reader",
description="Read visible address books and contacts.", description="Read visible address books and contacts.",
permissions=("addresses:address_book:read", "addresses:address_list:read", "addresses:contact:read", "addresses:governance:read", "addresses:sync:read"), permissions=(
"addresses:address_book:read",
"addresses:address_list:read",
"addresses:contact:read",
"addresses:governance:read",
"addresses:sync:read",
),
), ),
) )
@@ -171,15 +263,42 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
) )
return { return {
"address_books": session.query(AddressBook).filter(AddressBook.tenant_id == tenant_id, AddressBook.deleted_at.is_(None)).count(), "address_books": session.query(AddressBook)
"address_lists": session.query(AddressList).filter(AddressList.tenant_id == tenant_id, AddressList.deleted_at.is_(None)).count(), .filter(AddressBook.tenant_id == tenant_id, AddressBook.deleted_at.is_(None))
"contacts": session.query(Contact).filter(Contact.tenant_id == tenant_id, Contact.deleted_at.is_(None)).count(), .count(),
"active_contact_merges": session.query(ContactMergeRecord).filter(ContactMergeRecord.tenant_id == tenant_id, ContactMergeRecord.status == "active").count(), "address_lists": session.query(AddressList)
"contact_quality_decisions": session.query(ContactPointQualityDecision).filter(ContactPointQualityDecision.tenant_id == tenant_id).count(), .filter(AddressList.tenant_id == tenant_id, AddressList.deleted_at.is_(None))
"contact_point_snapshots": session.query(ContactPointSnapshot).filter(ContactPointSnapshot.tenant_id == tenant_id).count(), .count(),
"sync_sources": session.query(AddressSyncSource).filter(AddressSyncSource.tenant_id == tenant_id, AddressSyncSource.enabled.is_(True)).count(), "contacts": session.query(Contact)
"address_import_profiles": session.query(AddressImportProfile).filter(AddressImportProfile.tenant_id == tenant_id, AddressImportProfile.is_current.is_(True)).count(), .filter(Contact.tenant_id == tenant_id, Contact.deleted_at.is_(None))
"address_import_runs": session.query(AddressImportRun).filter(AddressImportRun.tenant_id == tenant_id).count(), .count(),
"active_contact_merges": session.query(ContactMergeRecord)
.filter(
ContactMergeRecord.tenant_id == tenant_id,
ContactMergeRecord.status == "active",
)
.count(),
"contact_quality_decisions": session.query(ContactPointQualityDecision)
.filter(ContactPointQualityDecision.tenant_id == tenant_id)
.count(),
"contact_point_snapshots": session.query(ContactPointSnapshot)
.filter(ContactPointSnapshot.tenant_id == tenant_id)
.count(),
"sync_sources": session.query(AddressSyncSource)
.filter(
AddressSyncSource.tenant_id == tenant_id,
AddressSyncSource.enabled.is_(True),
)
.count(),
"address_import_profiles": session.query(AddressImportProfile)
.filter(
AddressImportProfile.tenant_id == tenant_id,
AddressImportProfile.is_current.is_(True),
)
.count(),
"address_import_runs": session.query(AddressImportRun)
.filter(AddressImportRun.tenant_id == tenant_id)
.count(),
} }
@@ -199,13 +318,28 @@ CARDDAV_PROVIDER = ExternalProviderDeclaration(
ProviderObjectDeclaration( ProviderObjectDeclaration(
object_type="address_book", object_type="address_book",
field_groups=("identity", "display", "sync_state"), field_groups=("identity", "display", "sync_state"),
authority_modes=("external_authoritative", "external_mirror", "governed_sync"), authority_modes=(
"external_authoritative",
"external_mirror",
"governed_sync",
),
default_authority_mode="external_mirror", default_authority_mode="external_mirror",
), ),
ProviderObjectDeclaration( ProviderObjectDeclaration(
object_type="contact", object_type="contact",
field_groups=("identity", "name", "postal", "email", "phone", "source_metadata"), field_groups=(
authority_modes=("external_authoritative", "external_mirror", "governed_sync"), "identity",
"name",
"postal",
"email",
"phone",
"source_metadata",
),
authority_modes=(
"external_authoritative",
"external_mirror",
"governed_sync",
),
default_authority_mode="governed_sync", default_authority_mode="governed_sync",
), ),
), ),
@@ -251,7 +385,15 @@ LDAP_PROVIDER = ExternalProviderDeclaration(
objects=( objects=(
ProviderObjectDeclaration( ProviderObjectDeclaration(
object_type="contact", object_type="contact",
field_groups=("identity", "name", "organization", "postal", "email", "phone", "source_metadata"), field_groups=(
"identity",
"name",
"organization",
"postal",
"email",
"phone",
"source_metadata",
),
authority_modes=("external_authoritative", "external_mirror"), authority_modes=("external_authoritative", "external_mirror"),
default_authority_mode="external_authoritative", default_authority_mode="external_authoritative",
), ),
@@ -280,7 +422,11 @@ LDAP_PROVIDER = ExternalProviderDeclaration(
reconciliation="Only a complete paged search may infer an absent source object and create a local tombstone.", reconciliation="Only a complete paged search may infer an absent source object and create a local tombstone.",
outage="Existing contacts remain available and visibly stale; an unavailable directory never causes deletes.", outage="Existing contacts remain available and visibly stale; an unavailable directory never causes deletes.",
classifications=("personal", "confidential", "restricted"), classifications=("personal", "confidential", "restricted"),
purposes=("directory projection", "recipient resolution", "identity-linked contact discovery"), purposes=(
"directory projection",
"recipient resolution",
"identity-linked contact discovery",
),
retention="Address, audit, and records policies govern local projections and tombstone evidence.", retention="Address, audit, and records policies govern local projections and tombstone evidence.",
secret_handling="Bind secrets remain in reusable credential envelopes; URLs, previews, and diagnostics contain no credentials.", secret_handling="Bind secrets remain in reusable credential envelopes; URLs, previews, and diagnostics contain no credentials.",
), ),
@@ -292,34 +438,129 @@ LDAP_PROVIDER = ExternalProviderDeclaration(
manifest = ModuleManifest( manifest = ModuleManifest(
id="addresses", id="addresses",
name="Addresses", name="Addresses",
version="0.1.18", version="0.1.23",
required_capabilities=(CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PERMISSION_EVALUATOR), required_capabilities=(
optional_dependencies=("campaigns", "mail", "forms", "reporting", "portal", "postbox", "connectors"), CAPABILITY_AUTH_PRINCIPAL_RESOLVER,
CAPABILITY_AUTH_PERMISSION_EVALUATOR,
),
optional_dependencies=(
"campaigns",
"mail",
"forms",
"reporting",
"portal",
"postbox",
"connectors",
),
provides_interfaces=( provides_interfaces=(
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_LOOKUP, version="0.1.8"), ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_LOOKUP, version="0.1.8"),
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_PEOPLE_SEARCH, version="0.1.0"), ModuleInterfaceProvider(
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_RECIPIENT_SOURCE, version="0.1.9"), name=CAPABILITY_ADDRESSES_PEOPLE_SEARCH, version="0.1.0"
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION, version="1.0.0"), ),
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_CONTACT_WRITER, version="0.1.8"), ModuleInterfaceProvider(
ModuleInterfaceProvider(name=CAPABILITY_RECIPIENT_CHANNEL_FACTS, version="0.1.0"), name=CAPABILITY_ADDRESSES_RECIPIENT_SOURCE, version="0.1.9"
),
ModuleInterfaceProvider(
name=CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION, version="1.0.0"
),
ModuleInterfaceProvider(
name=CAPABILITY_ADDRESSES_CONTACT_WRITER, version="0.1.8"
),
ModuleInterfaceProvider(
name=CAPABILITY_RECIPIENT_CHANNEL_FACTS, version="0.1.0"
),
ModuleInterfaceProvider(name=ADDRESSES_DSAR_CAPABILITY, version="0.1.0"),
), ),
permissions=PERMISSIONS, permissions=PERMISSIONS,
route_factory=_addresses_router, route_factory=_addresses_router,
role_templates=ROLE_TEMPLATES, role_templates=ROLE_TEMPLATES,
tenant_summary_providers=(_tenant_summary,), tenant_summary_providers=(_tenant_summary,),
nav_items=(NavItem(path="/address-book", label="Address Book", icon="book-user", required_any=("addresses:contact:read",), order=80),), nav_items=(
NavItem(
path="/address-book",
label="Address Book",
icon="book-user",
required_any=("addresses:contact:read",),
order=80,
),
),
frontend=FrontendModule( frontend=FrontendModule(
module_id="addresses", module_id="addresses",
package_name="@govoplan/addresses-webui", package_name="@govoplan/addresses-webui",
routes=(FrontendRoute(path="/address-book", component="AddressBookPage", required_any=("addresses:contact:read",), order=80),), routes=(
nav_items=(NavItem(path="/address-book", label="Address Book", icon="book-user", required_any=("addresses:contact:read",), order=80),), FrontendRoute(
path="/address-book",
component="AddressBookPage",
required_any=("addresses:contact:read",),
order=80,
),
),
nav_items=(
NavItem(
path="/address-book",
label="Address Book",
icon="book-user",
required_any=("addresses:contact:read",),
order=80,
),
),
product_areas=(
ProductAreaContribution(
id="people-responsibility",
module_id="addresses",
label="i18n:govoplan-core.product_area.people_responsibility",
icon="users",
description="i18n:govoplan-core.product_area.people_responsibility_description",
surface_ids=(
"addresses.nav.address.book",
"addresses.route.address.book",
),
order=70,
),
),
view_surfaces=( view_surfaces=(
ViewSurface(id="addresses.page", module_id="addresses", kind="route", label="Address Book", order=80), ViewSurface(
ViewSurface(id="addresses.sources", module_id="addresses", kind="section", label="Address sources", order=10), id="addresses.page",
ViewSurface(id="addresses.contacts", module_id="addresses", kind="section", label="Contacts", order=20), module_id="addresses",
ViewSurface(id="addresses.detail", module_id="addresses", kind="section", label="Contact detail", order=30), kind="route",
ViewSurface(id="addresses.governance", module_id="addresses", kind="action", label="Communication governance", order=40), label="Address Book",
ViewSurface(id="addresses.sync", module_id="addresses", kind="action", label="Address synchronization", order=50), order=80,
),
ViewSurface(
id="addresses.sources",
module_id="addresses",
kind="section",
label="Address sources",
order=10,
),
ViewSurface(
id="addresses.contacts",
module_id="addresses",
kind="section",
label="Contacts",
order=20,
),
ViewSurface(
id="addresses.detail",
module_id="addresses",
kind="section",
label="Contact detail",
order=30,
),
ViewSurface(
id="addresses.governance",
module_id="addresses",
kind="action",
label="Communication governance",
order=40,
),
ViewSurface(
id="addresses.sync",
module_id="addresses",
kind="action",
label="Address synchronization",
order=50,
),
), ),
), ),
migration_spec=MigrationSpec( migration_spec=MigrationSpec(
@@ -331,8 +572,13 @@ manifest = ModuleManifest(
retirement_notes="Destructive retirement drops address-owned database tables after the installer captures a database snapshot.", retirement_notes="Destructive retirement drops address-owned database tables after the installer captures a database snapshot.",
), ),
capability_factories={ capability_factories={
CAPABILITY_ADDRESSES_LOOKUP: lambda context: __import__("govoplan_addresses.backend.capabilities", fromlist=["lookup_capability"]).lookup_capability(context), CAPABILITY_ADDRESSES_LOOKUP: lambda context: __import__(
CAPABILITY_ADDRESSES_PEOPLE_SEARCH: lambda context: __import__("govoplan_addresses.backend.capabilities", fromlist=["people_search_capability"]).people_search_capability(context), "govoplan_addresses.backend.capabilities", fromlist=["lookup_capability"]
).lookup_capability(context),
CAPABILITY_ADDRESSES_PEOPLE_SEARCH: lambda context: __import__(
"govoplan_addresses.backend.capabilities",
fromlist=["people_search_capability"],
).people_search_capability(context),
CAPABILITY_ADDRESSES_RECIPIENT_SOURCE: lambda context: __import__( CAPABILITY_ADDRESSES_RECIPIENT_SOURCE: lambda context: __import__(
"govoplan_addresses.backend.capabilities", "govoplan_addresses.backend.capabilities",
fromlist=["recipient_source_capability"], fromlist=["recipient_source_capability"],
@@ -349,6 +595,20 @@ manifest = ModuleManifest(
"govoplan_addresses.backend.capabilities", "govoplan_addresses.backend.capabilities",
fromlist=["contact_point_resolution_capability"], fromlist=["contact_point_resolution_capability"],
).contact_point_resolution_capability(context), ).contact_point_resolution_capability(context),
ADDRESSES_DSAR_CAPABILITY: _addresses_dsar_provider,
},
capability_documentation={
ADDRESSES_DSAR_CAPABILITY: CapabilityDocumentation(
label="Addresses data-subject request provider",
summary=(
"Finds bounded contact, contact-point, address-list, governance, "
"provenance, synchronization, and operator-attribution data without "
"exporting raw source payloads, connector state, or opaque evidence."
),
contract_version="0.1.0",
documentation_types=("admin",),
audience=("privacy_officer", "addresses_admin", "records_manager"),
),
}, },
uninstall_guard_providers=( uninstall_guard_providers=(
persistent_table_uninstall_guard( persistent_table_uninstall_guard(
@@ -375,24 +635,138 @@ manifest = ModuleManifest(
), ),
), ),
documentation=( documentation=(
DocumentationTopic(
id="addresses.privacy.data-subject-requests",
title="Review Addresses data in a data-subject request",
summary=(
"Collect tenant-scoped contact data while preserving shared address, "
"recipient, synchronization, and provenance evidence."
),
body=(
"Addresses searches corroborated email and account selectors plus "
"namespaced contact and contact-point references. A matching contact "
"exports bounded identity, email, telephone, and postal values together "
"with its address-list use and minimized governance, quality, provenance, "
"merge, redirect, and synchronization evidence. Account matches add only "
"minimized operator attribution for governed configuration and evidence. "
"The provider excludes raw imported or synchronized source payloads, "
"connector tokens and revisions that could act as credentials, opaque "
"metadata, snapshot request and resolution payloads, import plans, merge "
"before/after payloads, unrelated contacts, and other tenants. Quality, "
"governance, provenance, merge, redirect, synchronization, import, "
"snapshot, and operator evidence is retained with an explicit reason. "
"Because reusable contacts can be shared, synchronized, merged, or "
"referenced by immutable recipient snapshots, the DSAR provider never "
"deletes them automatically. An authorized operator must review "
"dependencies and use the normal Addresses correction, archive, source, "
"merge, or governance workflow."
),
layer="static",
documentation_types=("admin",),
audience=(
"privacy_officer",
"addresses_admin",
"records_manager",
"operator",
),
related_modules=(
"access",
"audit",
"campaigns",
"dist_lists",
"records",
),
order=29,
translations={
"de": {
"title": "Addresses-Daten in einer Betroffenenanfrage prüfen",
"summary": (
"Mandantenbezogene Kontaktdaten erfassen und dabei gemeinsame Adress-, Empfänger-, Synchronisations- und "
"Herkunftsnachweise bewahren."
),
"body": (
"Addresses durchsucht bestätigte E-Mail- und Kontoselektoren sowie namensraumgebundene Kontakt- und "
"Kontaktpunktverweise. Zu einem passenden Kontakt werden begrenzte Identitäts-, E-Mail-, Telefon- und Postwerte "
"einschließlich seiner Adresslistennutzung und minimierter Nachweise zu Governance, Qualität, Herkunft, Zusammenführung, "
"Weiterleitung und Synchronisation exportiert. Kontotreffer ergänzen nur minimierte Zuordnungen von Betriebspersonen zu "
"gesteuerter Konfiguration und Nachweisen. Ausgeschlossen sind rohe importierte oder synchronisierte Quelldaten, "
"Connector-Token und Revisionen mit Zugangsdatencharakter, undurchsichtige Metadaten, Anfrage- und Auflösungsnutzdaten "
"von Snapshots, Importpläne, Vorher-/Nachher-Daten von Zusammenführungen, unbeteiligte Kontakte und andere Mandanten. "
"Nachweise zu Qualität, Governance, Herkunft, Zusammenführung, Weiterleitung, Synchronisation, Import, Snapshot und "
"Betriebszuordnung werden mit ausdrücklicher Begründung aufbewahrt. Weil wiederverwendbare Kontakte geteilt, "
"synchronisiert, zusammengeführt oder von unveränderlichen Empfänger-Snapshots referenziert sein können, löscht der "
"DSAR-Provider sie niemals automatisch. Eine berechtigte Betriebsperson muss Abhängigkeiten prüfen und den regulären "
"Addresses-Ablauf für Korrektur, Archivierung, Quelle, Zusammenführung oder Governance verwenden."
),
}
},
metadata={
"seed": True,
"help_contexts": [
"addresses.contacts",
"addresses.governance",
"addresses.action.archive",
],
},
),
DocumentationTopic( DocumentationTopic(
id="addresses.boundary", id="addresses.boundary",
title="Reusable address ownership", title="Reusable address ownership",
summary="Reusable person, organization, household, postal, and email recipient sources belong to the addresses module.", summary="Reusable person, organization, household, postal, and email recipient sources belong to the addresses module.",
body=( body=(
"Open the book beside Address books for documentation of the address workspace. "
"Campaigns may keep immutable campaign-local recipient snapshots, but durable address directories, " "Campaigns may keep immutable campaign-local recipient snapshots, but durable address directories, "
"recipient-source definitions, consent metadata, provenance, deduplication, and import/export workflows " "recipient-source definitions, consent metadata, provenance, deduplication, and import/export workflows "
"are owned by govoplan-addresses." "are owned by govoplan-addresses. The Address Book workspace keeps Reload directly "
"before Add address book at the upper right. Import / export, Connections, and Address "
"quality open labelled, scoped tools; Manage applies to the selected book or list and "
"separates archive actions from editing. Export version is chosen in Import / export. "
"Contact creation remains beside the contact list. Folder icons alone expand or collapse "
"the tree; labels select a group, book, or list without changing expansion. A selected "
"group is navigation, not an aggregate contact book. Reload preserves collapsed "
"branches. All permission and read-only reasons, confirmations, import safeguards, "
"and contact-to-list drag and drop still apply."
), ),
layer="configured", layer="configured",
documentation_types=("admin", "user"), documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin"), audience=("tenant_admin", "operator", "module_admin"),
related_modules=("campaigns", "mail", "forms", "reporting", "portal", "postbox"), related_modules=(
"campaigns",
"mail",
"forms",
"reporting",
"portal",
"postbox",
),
order=30, order=30,
translations={
"de": {
"title": "Zuständigkeit für wiederverwendbare Adressen",
"summary": (
"Wiederverwendbare Personen-, Organisations-, Haushalts-, Post- und E-Mail-Empfängerquellen gehören dem "
"Addresses-Modul."
),
"body": (
"Öffnen Sie das Buch neben Adressbücher für die Dokumentation des Adressarbeitsbereichs. "
"Campaigns darf unveränderliche campaignlokale Empfänger-Snapshots halten. Dauerhafte Adressverzeichnisse, "
"Empfängerquellendefinitionen, Einwilligungsmetadaten, Herkunft, Dublettenbereinigung sowie Import- und Exportabläufe "
"gehören jedoch govoplan-addresses. Im Adressbuch steht Neuladen oben rechts unmittelbar vor "
"Adressbuch hinzufügen. Import / Export, Verbindungen und Adressqualität öffnen beschriftete, "
"kontextbezogene Werkzeuge. Verwalten bezieht sich auf das ausgewählte Adressbuch oder die Liste "
"und trennt Archivieren vom Bearbeiten. Die Exportversion wird unter Import / Export gewählt. "
"Kontakte werden weiterhin direkt neben der Kontaktliste angelegt. Nur Ordnersymbole klappen "
"den Baum auf oder zu; Beschriftungen wählen eine Gruppe, ein Adressbuch oder eine Liste aus, "
"ohne die Aufklappstellung zu ändern. Eine ausgewählte Gruppe dient der Navigation und ist "
"kein zusammengefasstes Adressbuch. Neuladen bewahrt zugeklappte Zweige. Berechtigungs- und "
"Schreibschutzgründe, Bestätigungen, Importsicherungen und das Ziehen von Kontakten in Listen gelten unverändert."
),
}
},
metadata={ metadata={
"seed": True, "seed": True,
"help_contexts": [ "help_contexts": [
"addresses.page", "addresses.page",
"addresses.explorer.transfer",
"addresses.sources", "addresses.sources",
"addresses.contacts", "addresses.contacts",
"addresses.detail", "addresses.detail",
@@ -415,6 +789,18 @@ manifest = ModuleManifest(
audience=("tenant_admin", "operator", "module_admin"), audience=("tenant_admin", "operator", "module_admin"),
related_modules=("dist_lists", "campaigns", "policy", "templates"), related_modules=("dist_lists", "campaigns", "policy", "templates"),
order=31, order=31,
translations={
"de": {
"title": "Kontaktpunktauflösung und Snapshots",
"summary": "Zweckbezogene Kanalziele auflösen und unveränderliche Empfängernachweise einfrieren.",
"body": (
"Addresses stellt eine versionierte Kontaktpunktfähigkeit für E-Mail-, Post-, Hauspost- und Portalziele bereit. "
"Aufrufende können Wirksamkeitsdatum, Kommunikationszweck, Adresszweck, Rückfallregel, Spracheinstellung und Postformat "
"angeben. Begrenzte Vorschauen bleiben aktuell; eingefrorene Snapshots bewahren aufgelöste Werte, Ausschlüsse, Quellen- "
"und Governance-Revisionen, Herkunft und einen deterministischen Nachweishash auch nach späteren Kontaktänderungen."
),
}
},
metadata={ metadata={
"seed": True, "seed": True,
"help_contexts": [ "help_contexts": [
@@ -428,20 +814,136 @@ manifest = ModuleManifest(
), ),
DocumentationTopic( DocumentationTopic(
id="addresses.tabular-imports", id="addresses.tabular-imports",
title="CSV and XLSX contact imports", title="CSV, XLSX, and LDIF contact imports",
summary="Preview and apply reusable, versioned contact mappings without silent row loss.", summary="Preview and apply reusable, versioned contact mappings without silent row or entry loss.",
body=( body=(
"CSV and XLSX files can be mapped with scoped, reusable profile versions. Each preview validates headers, " "CSV, XLSX, and LDIF files can be mapped with scoped, reusable profile versions. Each preview validates headers or attributes, "
"encodings, source keys, duplicates, blank values, workbook limits, and contact identity before any mutation. " "encodings, source keys, duplicates, blank values, format limits, and contact identity before any mutation. "
"The reviewed input hash and plan hash are retained with row-level effects and diagnostics. Apply is idempotent, " "The reviewed input hash and plan hash are retained with row-level effects and diagnostics. Apply is idempotent, "
"rejects contacts changed after preview, and records sufficient evidence for a guarded rollback. XLSX formulas, " "rejects contacts changed after preview, and records sufficient evidence for a guarded rollback. Each new update "
"macros, and legacy workbook formats are never executed or imported." "uses version-2 before-images for editable contact values, source metadata, complete contact-point identities, "
"original and normalized values, provenance, order and timestamps, and prior deletion state: rolling back an "
"import that restored a deleted contact archives it again. Older update runs without complete before-images require "
"manual reconciliation; automatic rollback stops before changing any contacts. Created-contact identities and "
"after-hashes are persisted with the applied plan and required for rollback; missing older guards or contacts "
"moved to another book also require reconciliation. Point-evidence edits are included in change guards. "
"Unchanged point values retain their identities and original evidence. Removing or replacing a point referenced "
"by an address list or point-specific consent/quality decision requires explicit reconciliation, including when "
"a new reference would otherwise be detached by rollback. Current book visibility and change "
"guards still apply. Preview source lookups and contact collections are loaded in bounded batches, without changing "
"duplicate policies, reviewed hashes, or apply-time validation. A persisted run "
"can be reopened with its run link after navigation or reload; previewed, applied, rolled-back, expired, and "
"unavailable states remain explicit. Both apply and rollback submit the reviewed plan hash. Missing, expired, "
"hidden, and cross-tenant runs disclose no source payload. XLSX formulas, macros, and legacy workbook formats "
"are never executed or imported. LDIF folded lines, UTF-8 and base64 text, repeated attributes, and comments are parsed; "
"binary and URL values are never projected or fetched. Change records default to rejected diagnostics and may only be ignored "
"or treat add records as static entries through an explicit profile policy."
), ),
layer="configured", layer="configured",
documentation_types=("admin", "user"), documentation_types=("admin", "user"),
audience=("tenant_admin", "module_admin", "power_user"), audience=("tenant_admin", "module_admin", "power_user"),
conditions=(
DocumentationCondition(
required_modules=("addresses",),
any_scopes=("addresses:contact:write", "addresses:sync:write"),
),
),
related_modules=("connectors", "datasources", "dataflow", "files", "audit"), related_modules=("connectors", "datasources", "dataflow", "files", "audit"),
order=33, order=33,
translations={
"de": {
"title": "Kontakte aus CSV, XLSX und LDIF importieren",
"summary": (
"Wiederverwendbare, versionierte Kontaktzuordnungen vorprüfen und anwenden, ohne Zeilen oder Einträge "
"stillschweigend zu verlieren."
),
"body": (
"CSV-, XLSX- und LDIF-Dateien lassen sich mit bereichsgebundenen, wiederverwendbaren Profilversionen zuordnen. Jede "
"Vorschau prüft Überschriften oder Attribute, Kodierung, Quellschlüssel, Dubletten, Leerwerte, Formatgrenzen und "
"Kontaktidentität vor jeder Änderung. Der geprüfte Eingabe- und Planhash wird mit zeilenbezogenen Wirkungen und Diagnosen "
"aufbewahrt. Die Anwendung ist idempotent, verwirft seit der Vorschau geänderte Kontakte und zeichnet ausreichende "
"Nachweise für eine gesicherte Rücknahme auf. Ein gespeicherter Lauf kann nach Navigation oder Neuladen über seinen Link "
"erneut geöffnet werden. Vorher-Bilder der Version 2 enthalten bearbeitbare Kontaktwerte, Quellmetadaten, "
"vollständige Kontaktpunktkennungen, Original- und normalisierte Werte, Herkunft, Reihenfolge, Zeitstempel und den "
"vorherigen Löschzustand: Die Rücknahme archiviert einen durch den Import wiederhergestellten Kontakt erneut. "
"Ältere Änderungsläufe ohne vollständige Vorher-Bilder erfordern einen manuellen Abgleich; die automatische "
"Rücknahme stoppt vor jeder Kontaktänderung. Kennungen neu angelegter Kontakte und Nachher-Hashes werden mit dem "
"angewendeten Plan gespeichert und sind für die Rücknahme erforderlich; fehlende ältere Sicherungen oder in ein "
"anderes Buch verschobene Kontakte erfordern ebenfalls einen Abgleich. Änderungen an Punktnachweisen werden vom "
"Änderungsschutz erfasst. Unveränderte Punktwerte behalten Kennung und Originalnachweise. Das Entfernen oder "
"Ersetzen eines in Adresslisten oder punktspezifischen Einwilligungs-/Qualitätsentscheidungen referenzierten "
"Punkts erfordert einen ausdrücklichen Abgleich; dies gilt auch für neue Referenzen, die eine Rücknahme sonst "
"lösen würde. Aktuelle Adressbuchsichtbarkeit und "
"Änderungsschutz bleiben wirksam. "
"Quellzuordnungen und Kontaktpunkte werden für die Vorschau in begrenzten Stapeln geladen, ohne Dublettenregeln, "
"geprüfte Hashes oder die erneute Prüfung bei Anwendung zu ändern. Ein gespeicherter Lauf kann über seinen Link "
"erneut geöffnet werden; Vorschau-, Anwendungs-, Rücknahme-, Ablauf- und Nichtverfügbarkeitszustände bleiben eindeutig. "
"Anwendung und Rücknahme übermitteln den geprüften Planhash. Fehlende, abgelaufene, verborgene und mandantenfremde Läufe "
"legen keine Quelldaten offen. XLSX-Formeln, Makros und ältere Arbeitsmappenformate werden niemals ausgeführt oder "
"importiert. Gefaltete LDIF-Zeilen, UTF-8- und Base64-Text, wiederholte Attribute und Kommentare werden verarbeitet; "
"Binär- und URL-Werte werden weder projiziert noch abgerufen. Änderungsdatensätze gelten standardmäßig als abgelehnte "
"Diagnose und dürfen nur über eine ausdrückliche Profilrichtlinie ignoriert oder bei Add-Einträgen als statische Daten "
"behandelt werden."
),
}
},
metadata={
"kind": "workflow",
"help_contexts": [
"addresses.action.import",
"addresses.contacts",
"addresses.sources",
],
},
),
DocumentationTopic(
id="addresses.vcard-batches",
title="Selective vCard batch import and export",
summary="Preview multiple vCard files, choose each card's effect, and export deterministic scoped files.",
body=(
"One or more UTF-8 .vcf files are parsed into a persisted, non-mutating preview with bounded diagnostics, "
"duplicate suggestions, an input hash, a parser version, and a deterministic plan hash. Operators choose "
"create, update, or ignore only where the reviewed plan permits it. Apply rejects stale contact targets and "
"is idempotent for the same selection; a different retry is rejected. Pending runs can be reloaded or cancelled "
"without changing contacts. Upload size, file count, card count, line count, and unfolded-line length are bounded. "
"Exports can target a complete address book, one address list, or explicit contacts; vCard 3.0 or 4.0 is selected "
"explicitly and contacts use deterministic display-name and stable-ID ordering. Export and import evidence records "
"hashes and counts, while diagnostics never disclose raw contact payloads. Large previews remain persisted and expose "
"their batch execution mode so a runtime job capability can execute them asynchronously when available."
),
layer="configured",
documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin", "power_user"),
related_modules=("files", "audit", "connectors"),
order=34,
translations={
"de": {
"title": "vCard-Stapel selektiv importieren und exportieren",
"summary": (
"Mehrere vCard-Dateien vorprüfen, die Wirkung jeder Karte wählen und deterministische bereichsgebundene Dateien exportieren."
),
"body": (
"Eine oder mehrere UTF-8-.vcf-Dateien werden in eine gespeicherte, nicht verändernde Vorschau mit begrenzten Diagnosen, "
"Dublettenhinweisen, Eingabehash, Parser-Version und deterministischem Planhash eingelesen. Betriebspersonen wählen "
"Anlegen, Aktualisieren oder Ignorieren nur dort, wo der geprüfte Plan es erlaubt. Die Anwendung verwirft veraltete "
"Kontaktziele und ist für dieselbe Auswahl idempotent; eine abweichende Wiederholung wird abgelehnt. Ausstehende Läufe "
"lassen sich neu laden oder abbrechen, ohne Kontakte zu verändern. Uploadgröße, Datei- und Kartenanzahl, Zeilenanzahl und "
"Länge entfalteter Zeilen sind begrenzt. Exporte können ein vollständiges Adressbuch, eine Adressliste oder ausgewählte "
"Kontakte umfassen; vCard 3.0 oder 4.0 wird ausdrücklich gewählt und Kontakte werden deterministisch nach Anzeigename und "
"stabiler Kennung sortiert. Export- und Importnachweise speichern Hashes und Anzahlen, während Diagnosen niemals rohe "
"Kontaktdaten offenlegen. Große Vorschauen bleiben gespeichert und geben ihren Stapelausführungsmodus an, sodass eine "
"Laufzeit-Jobfähigkeit sie bei Verfügbarkeit asynchron ausführen kann."
),
}
},
metadata={
"seed": True,
"help_contexts": [
"addresses.action.import",
"addresses.contacts",
"addresses.sources",
],
},
), ),
DocumentationTopic( DocumentationTopic(
id="addresses.ldap-directory", id="addresses.ldap-directory",
@@ -458,7 +960,23 @@ manifest = ModuleManifest(
documentation_types=("admin", "user"), documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin"), audience=("tenant_admin", "operator", "module_admin"),
related_modules=("connectors", "idm", "access", "policy", "audit"), related_modules=("connectors", "idm", "access", "policy", "audit"),
order=34, order=35,
translations={
"de": {
"title": "LDAP- und Active-Directory-Adressquellen",
"summary": (
"Maßgebliche Verzeichniskontakte über eine begrenzte, schreibgeschützte Synchronisationsquelle projizieren."
),
"body": (
"LDAP-Quellen verwenden LDAPS oder StartTLS und wiederverwendbare Zugangsdatenhüllen. Die Ermittlung findet verfügbare "
"Basis-DNs; anschließend steuert das Quellprofil einen begrenzten seitenweisen Filter und eine ausdrückliche "
"Attributzuordnung. Die Vorschau verändert niemals Kontakte. Ein vollständiger erfolgreicher Lesevorgang darf lokale "
"Projektionen anlegen, aktualisieren oder als entfernt markieren; abgeschnittene oder fehlgeschlagene Lesevorgänge "
"unterdrücken Löschungen aufgrund von Abwesenheit und markieren die Quelle als veraltet. Stabile Quellschlüssel, "
"Revisionen, normalisierte Felder und Herkunft bleiben mit jedem erhaltenen Kontakt verbunden."
),
}
},
), ),
DocumentationTopic( DocumentationTopic(
id="addresses.quality-and-merge", id="addresses.quality-and-merge",
@@ -478,6 +996,21 @@ manifest = ModuleManifest(
audience=("tenant_admin", "operator", "module_admin"), audience=("tenant_admin", "operator", "module_admin"),
related_modules=("campaigns", "dist_lists", "policy", "audit"), related_modules=("campaigns", "dist_lists", "policy", "audit"),
order=32, order=32,
translations={
"de": {
"title": "Kontaktqualität, Dubletten und umkehrbare Zusammenführungen",
"summary": "Adressqualität und Dublettenvorschläge prüfen, ohne Quellnachweise zu verlieren.",
"body": (
"Addresses bewahrt ursprüngliche und normalisierte Kontaktpunktwerte, zeichnet die Herkunft je Feld auf und überführt "
"ungültige, zurückgesandte, veraltete oder unzustellbare Zustände mit stabilen Grundcodes in die Empfängerauflösung. "
"Dublettenvorschläge sind begrenzt und erklären ihre Übereinstimmungsmerkmale. Eine Betriebsperson kann die zu erhaltenden "
"Werte wählen, Kontaktpunkte zusammenführen und die Zusammenführung später rückgängig machen oder aufteilen, solange der "
"aufgezeichnete Nachweis nach der Zusammenführung noch übereinstimmt. Kontaktweiterleitungen halten gespeicherte Verweise "
"auflösbar, und Mitgliedschaften in Adresslisten werden transaktional repariert. Audit bleibt eine optionale Integration; "
"die Änderungsfolge und Zusammenführungsnachweise von Addresses werden stets aufbewahrt."
),
}
},
), ),
DocumentationTopic( DocumentationTopic(
id="addresses.reference.fields-and-consequences", id="addresses.reference.fields-and-consequences",
@@ -495,9 +1028,37 @@ manifest = ModuleManifest(
layer="configured", layer="configured",
documentation_types=("admin", "user"), documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin", "power_user"), audience=("tenant_admin", "operator", "module_admin", "power_user"),
related_modules=("dist_lists", "connectors", "datasources", "campaigns", "policy", "audit"), related_modules=(
order=35, "dist_lists",
"connectors",
"datasources",
"campaigns",
"policy",
"audit",
),
order=36,
translations={
"de": {
"title": "Adressfelder, Geltungsbereiche und Folgen von Aktionen",
"summary": (
"Geltungsbereich, Quellenhoheit, Kontaktpunkte, Listenmitgliedschaft sowie Folgen von Archivierung, Synchronisation und "
"Zusammenführung."
),
"body": (
"Adressbücher sind einer Person, Gruppe, einem Mandanten oder einem berechtigten Systemkontext zugeordnet. Geerbte und "
"extern maßgebliche Bücher können sichtbar, aber schreibgeschützt bleiben. Kontakte besitzen wiederverwendbare Angaben "
"zu Name, Organisation, elektronischen und telefonischen Kontaktpunkten, Postanschrift, Schlagwörtern, Notizen, Qualität "
"und Herkunft. Adresslisten verweisen auf Kontaktpunkte desselben Buchs und ersetzen keine Distribution Lists. Eine "
"Archivierung entfernt Buch, Liste oder Kontakt aus der gewöhnlichen Auswahl, bewahrt aber gesteuerte Historie und "
"Verweise. CardDAV- und LDAP-Quellen zeigen Richtung, Hoheit, Aktualität, Diagnosen, Konflikte und Verhalten bei "
"veraltetem Zustand. Import und Synchronisation erfordern vor jeder Änderung eine Vorschau. Kontaktzusammenführungen "
"wählen überlebenden Kontakt und Feldherkunft, reparieren Listenverweise transaktional und bewahren Weiterleitungen und "
"Nachweise, sodass eine passende Zusammenführung rückgängig gemacht oder aufgeteilt werden kann."
),
}
},
metadata={ metadata={
"kind": "reference",
"seed": True, "seed": True,
"help_contexts": [ "help_contexts": [
"addresses.field.book-scope", "addresses.field.book-scope",
@@ -537,15 +1098,27 @@ manifest = ModuleManifest(
maturity="vertical_slice", maturity="vertical_slice",
documentation_ref="docs/ADDRESS_MODULE_ARCHITECTURE.md", documentation_ref="docs/ADDRESS_MODULE_ARCHITECTURE.md",
test_ref="tests/test_addresses_service.py", test_ref="tests/test_addresses_service.py",
known_limits=("External address-book synchronization remains a bounded connector slice rather than a supported provider profile.",), known_limits=(
"External address-book synchronization remains a bounded connector slice rather than a supported provider profile.",
),
supported_authority_modes=( supported_authority_modes=(
"native_authoritative", "native_authoritative",
"external_authoritative", "external_authoritative",
"external_mirror", "external_mirror",
"governed_sync", "governed_sync",
), ),
owned_concepts=("contact point", "address book", "contact consent", "recipient source"), owned_concepts=(
non_owned_concepts=("identity", "organization", "campaign recipient snapshot", "procedure party"), "contact point",
"address book",
"contact consent",
"recipient source",
),
non_owned_concepts=(
"identity",
"organization",
"campaign recipient snapshot",
"procedure party",
),
target_tested_providers=(CARDDAV_PROVIDER_ID,), target_tested_providers=(CARDDAV_PROVIDER_ID,),
security_docs=("docs/ADDRESS_MODULE_ARCHITECTURE.md",), security_docs=("docs/ADDRESS_MODULE_ARCHITECTURE.md",),
operations_docs=("README.md",), operations_docs=("README.md",),
@@ -553,5 +1126,10 @@ manifest = ModuleManifest(
) )
manifest = with_documentation_structured_translations(
manifest, locale="de", translations=GERMAN_STRUCTURED_TRANSLATIONS
)
def get_manifest() -> ModuleManifest: def get_manifest() -> ModuleManifest:
return manifest return manifest
@@ -0,0 +1,34 @@
"""Allow profile-free persisted vCard batch runs.
Revision ID: d6e8f9a0b1c2
Revises: c5d7e8f9a0b1
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d6e8f9a0b1c2"
down_revision = "c5d7e8f9a0b1"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("addresses_import_runs") as batch:
batch.alter_column(
"profile_id",
existing_type=sa.String(length=36),
nullable=True,
)
def downgrade() -> None:
with op.batch_alter_table("addresses_import_runs") as batch:
batch.alter_column(
"profile_id",
existing_type=sa.String(length=36),
nullable=False,
)
+351 -78
View File
@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import asdict from dataclasses import asdict
import json
import re import re
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
@@ -56,6 +57,23 @@ from govoplan_addresses.backend.imports import (
rollback_address_import, rollback_address_import,
update_import_profile, update_import_profile,
) )
from govoplan_addresses.backend.vcard_batch_schemas import (
VCardBatchCancelRequest,
VCardBatchCommitRequest,
VCardBatchPreviewRequest,
VCardBatchRunResponse,
VCardExportRequest,
VCardExportResponse,
)
from govoplan_addresses.backend.vcard_batches import (
apply_vcard_batch,
cancel_vcard_batch,
export_vcards,
get_vcard_batch_run,
preview_vcard_batch,
vcard_batch_payload,
vcard_diagnostics_payload,
)
from govoplan_addresses.backend.capabilities import ( from govoplan_addresses.backend.capabilities import (
AddressesContactPointResolutionCapability, AddressesContactPointResolutionCapability,
AddressesContactWriterCapability, AddressesContactWriterCapability,
@@ -238,9 +256,7 @@ def _contact_response(
decision = quality.get((channel, point_id)) or quality.get((channel, None)) decision = quality.get((channel, point_id)) or quality.get((channel, None))
return { return {
"quality_state": decision.state if decision is not None else "valid", "quality_state": decision.state if decision is not None else "valid",
"quality_reason_code": ( "quality_reason_code": (decision.reason_code if decision is not None else None),
decision.reason_code if decision is not None else None
),
} }
return ContactResponse.model_validate( return ContactResponse.model_validate(
@@ -323,9 +339,7 @@ def _contact_point_audit_details(
"postal": [item.id for item in contact.postal_addresses], "postal": [item.id for item in contact.postal_addresses],
} }
return { return {
f"{key_prefix}contact_point_counts": { f"{key_prefix}contact_point_counts": {channel: len(ids) for channel, ids in point_ids.items()},
channel: len(ids) for channel, ids in point_ids.items()
},
f"{key_prefix}contact_point_ids": point_ids, f"{key_prefix}contact_point_ids": point_ids,
} }
@@ -454,7 +468,9 @@ def _sync_source_response(sync_source: AddressSyncSource) -> AddressSyncSourceRe
) )
def _sync_diagnostic_response(diagnostic: AddressSyncDiagnostic) -> AddressSyncDiagnosticResponse: def _sync_diagnostic_response(
diagnostic: AddressSyncDiagnostic,
) -> AddressSyncDiagnosticResponse:
return AddressSyncDiagnosticResponse.model_validate( return AddressSyncDiagnosticResponse.model_validate(
{ {
"id": diagnostic.id, "id": diagnostic.id,
@@ -470,7 +486,9 @@ def _sync_diagnostic_response(diagnostic: AddressSyncDiagnostic) -> AddressSyncD
) )
def _sync_tombstone_response(tombstone: AddressSyncTombstone) -> AddressSyncTombstoneResponse: def _sync_tombstone_response(
tombstone: AddressSyncTombstone,
) -> AddressSyncTombstoneResponse:
return AddressSyncTombstoneResponse.model_validate( return AddressSyncTombstoneResponse.model_validate(
{ {
"id": tombstone.id, "id": tombstone.id,
@@ -490,7 +508,9 @@ def _sync_tombstone_response(tombstone: AddressSyncTombstone) -> AddressSyncTomb
) )
def _sync_conflict_response(conflict: AddressSyncConflict) -> AddressSyncConflictResponse: def _sync_conflict_response(
conflict: AddressSyncConflict,
) -> AddressSyncConflictResponse:
return AddressSyncConflictResponse.model_validate( return AddressSyncConflictResponse.model_validate(
{ {
"id": conflict.id, "id": conflict.id,
@@ -583,7 +603,11 @@ def api_list_address_books(
return AddressBookListResponse(address_books=[_book_response(book, contact_count=counts.get(book.id, 0)) for book in books]) return AddressBookListResponse(address_books=[_book_response(book, contact_count=counts.get(book.id, 0)) for book in books])
@router.post("/address-books", response_model=AddressBookResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-books",
response_model=AddressBookResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_book( def api_create_address_book(
payload: AddressBookCreateRequest, payload: AddressBookCreateRequest,
principal: ApiPrincipal = Depends(get_api_principal), principal: ApiPrincipal = Depends(get_api_principal),
@@ -591,7 +615,12 @@ def api_create_address_book(
): ):
_require_scope(principal, "addresses:address_book:write") _require_scope(principal, "addresses:address_book:write")
try: try:
book = create_address_book(session, principal, payload, allow_system=has_scope(principal, "addresses:address_book:admin")) book = create_address_book(
session,
principal,
payload,
allow_system=has_scope(principal, "addresses:address_book:admin"),
)
session.commit() session.commit()
session.refresh(book) session.refresh(book)
return _book_response(book) return _book_response(book)
@@ -738,10 +767,7 @@ def api_suggest_duplicate_contacts(
right=_contact_response(item.right), right=_contact_response(item.right),
score=item.score, score=item.score,
confidence=item.confidence, confidence=item.confidence,
features=[ features=[ContactDuplicateFeatureResponse(**asdict(feature)) for feature in item.features],
ContactDuplicateFeatureResponse(**asdict(feature))
for feature in item.features
],
) )
for item in scan.suggestions for item in scan.suggestions
], ],
@@ -778,10 +804,7 @@ def api_address_quality_summary(
quality_counts=summary.quality_counts, quality_counts=summary.quality_counts,
duplicate_suggestion_count=summary.duplicate_suggestion_count, duplicate_suggestion_count=summary.duplicate_suggestion_count,
correction_count=summary.correction_count, correction_count=summary.correction_count,
corrections=[ corrections=[AddressQualityCorrectionResponse(**asdict(item)) for item in summary.corrections],
AddressQualityCorrectionResponse(**asdict(item))
for item in summary.corrections
],
truncated=summary.truncated, truncated=summary.truncated,
) )
except AddressBookError as exc: except AddressBookError as exc:
@@ -895,9 +918,7 @@ def api_resolve_contact_redirect(
): ):
_require_scope(principal, "addresses:contact:read") _require_scope(principal, "addresses:contact:read")
try: try:
return ContactRedirectResponse.model_validate( return ContactRedirectResponse.model_validate(asdict(resolve_contact_redirect(session, principal, contact_id)))
asdict(resolve_contact_redirect(session, principal, contact_id))
)
except AddressBookError as exc: except AddressBookError as exc:
raise _error(exc) from exc raise _error(exc) from exc
@@ -1167,16 +1188,23 @@ def api_list_address_lists(
): ):
_require_scope(principal, "addresses:address_list:read") _require_scope(principal, "addresses:address_list:read")
try: try:
address_lists = list_address_lists(session, principal, address_book_id=address_book_id, include_deleted=include_deleted) address_lists = list_address_lists(
counts = address_list_entry_counts(session, [address_list.id for address_list in address_lists]) session,
return AddressListListResponse( principal,
address_lists=[_address_list_response(address_list, entry_count=counts.get(address_list.id, 0)) for address_list in address_lists] address_book_id=address_book_id,
include_deleted=include_deleted,
) )
counts = address_list_entry_counts(session, [address_list.id for address_list in address_lists])
return AddressListListResponse(address_lists=[_address_list_response(address_list, entry_count=counts.get(address_list.id, 0)) for address_list in address_lists])
except AddressBookError as exc: except AddressBookError as exc:
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/address-books/{book_id}/address-lists", response_model=AddressListResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-books/{book_id}/address-lists",
response_model=AddressListResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_list( def api_create_address_list(
book_id: str, book_id: str,
payload: AddressListCreateRequest, payload: AddressListCreateRequest,
@@ -1247,7 +1275,10 @@ def api_restore_address_list(
raise _error(exc) from exc raise _error(exc) from exc
@router.get("/address-lists/{address_list_id}/entries", response_model=AddressListEntryListResponse) @router.get(
"/address-lists/{address_list_id}/entries",
response_model=AddressListEntryListResponse,
)
def api_list_address_list_entries( def api_list_address_list_entries(
address_list_id: str, address_list_id: str,
principal: ApiPrincipal = Depends(get_api_principal), principal: ApiPrincipal = Depends(get_api_principal),
@@ -1261,7 +1292,11 @@ def api_list_address_list_entries(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/address-lists/{address_list_id}/entries", response_model=AddressListEntryResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-lists/{address_list_id}/entries",
response_model=AddressListEntryResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_list_entry( def api_create_address_list_entry(
address_list_id: str, address_list_id: str,
payload: AddressListEntryCreateRequest, payload: AddressListEntryCreateRequest,
@@ -1332,7 +1367,10 @@ def api_list_write_targets(
return AddressBookWriteTargetsResponse(targets=[_write_decision_response(decision) for decision in decisions]) return AddressBookWriteTargetsResponse(targets=[_write_decision_response(decision) for decision in decisions])
@router.get("/address-books/{book_id}/write-decision", response_model=AddressBookWriteDecisionResponse) @router.get(
"/address-books/{book_id}/write-decision",
response_model=AddressBookWriteDecisionResponse,
)
def api_get_address_book_write_decision( def api_get_address_book_write_decision(
book_id: str, book_id: str,
operation: str = Query(default="create_contact"), operation: str = Query(default="create_contact"),
@@ -1378,9 +1416,7 @@ def api_discover_ldap_base_dns(
): ):
_require_scope(principal, "addresses:sync:write") _require_scope(principal, "addresses:sync:write")
try: try:
return AddressLdapDiscoveryResponse( return AddressLdapDiscoveryResponse(base_dns=list(discover_ldap_base_dns(session, principal, payload)))
base_dns=list(discover_ldap_base_dns(session, principal, payload))
)
except (AddressBookError, AddressLdapError) as exc: except (AddressBookError, AddressLdapError) as exc:
raise _error(AddressBookError(str(exc))) from exc raise _error(AddressBookError(str(exc))) from exc
@@ -1438,7 +1474,11 @@ def api_list_address_credentials(
) )
@router.post("/address-books/{book_id}/carddav/sources", response_model=AddressSyncSourceResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-books/{book_id}/carddav/sources",
response_model=AddressSyncSourceResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_carddav_sync_source( def api_create_carddav_sync_source(
book_id: str, book_id: str,
payload: AddressCardDavSourceCreateRequest, payload: AddressCardDavSourceCreateRequest,
@@ -1454,7 +1494,11 @@ def api_create_carddav_sync_source(
action="addresses.sync_source_created", action="addresses.sync_source_created",
object_type="address_sync_source", object_type="address_sync_source",
object_id=sync_source.id, object_id=sync_source.id,
details={"address_book_id": book_id, "connector_type": "carddav", "sync_direction": sync_source.sync_direction}, details={
"address_book_id": book_id,
"connector_type": "carddav",
"sync_direction": sync_source.sync_direction,
},
) )
session.commit() session.commit()
session.refresh(sync_source) session.refresh(sync_source)
@@ -1473,7 +1517,12 @@ def api_list_sync_sources(
): ):
_require_scope(principal, "addresses:sync:read") _require_scope(principal, "addresses:sync:read")
try: try:
sync_sources = list_sync_sources(session, principal, address_book_id=address_book_id, include_disabled=include_disabled) sync_sources = list_sync_sources(
session,
principal,
address_book_id=address_book_id,
include_disabled=include_disabled,
)
return AddressSyncSourceListResponse(sync_sources=[_sync_source_response(sync_source) for sync_source in sync_sources]) return AddressSyncSourceListResponse(sync_sources=[_sync_source_response(sync_source) for sync_source in sync_sources])
except AddressBookError as exc: except AddressBookError as exc:
raise _error(exc) from exc raise _error(exc) from exc
@@ -1575,7 +1624,11 @@ def api_run_sync_source(
raise _error(AddressBookError(message)) from exc raise _error(AddressBookError(message)) from exc
@router.post("/address-books/{book_id}/sync-sources", response_model=AddressSyncSourceResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-books/{book_id}/sync-sources",
response_model=AddressSyncSourceResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_sync_source( def api_create_sync_source(
book_id: str, book_id: str,
payload: AddressSyncSourceCreateRequest, payload: AddressSyncSourceCreateRequest,
@@ -1591,7 +1644,11 @@ def api_create_sync_source(
action="addresses.sync_source_created", action="addresses.sync_source_created",
object_type="address_sync_source", object_type="address_sync_source",
object_id=sync_source.id, object_id=sync_source.id,
details={"address_book_id": book_id, "connector_type": sync_source.connector_type, "sync_direction": sync_source.sync_direction}, details={
"address_book_id": book_id,
"connector_type": sync_source.connector_type,
"sync_direction": sync_source.sync_direction,
},
) )
session.commit() session.commit()
session.refresh(sync_source) session.refresh(sync_source)
@@ -1617,7 +1674,11 @@ def api_update_sync_source(
action="addresses.sync_source_updated", action="addresses.sync_source_updated",
object_type="address_sync_source", object_type="address_sync_source",
object_id=sync_source.id, object_id=sync_source.id,
details={"connector_type": sync_source.connector_type, "sync_direction": sync_source.sync_direction, "enabled": sync_source.enabled}, details={
"connector_type": sync_source.connector_type,
"sync_direction": sync_source.sync_direction,
"enabled": sync_source.enabled,
},
) )
session.commit() session.commit()
session.refresh(sync_source) session.refresh(sync_source)
@@ -1651,7 +1712,10 @@ def api_delete_sync_source(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/attempts/start", response_model=AddressSyncSourceResponse) @router.post(
"/sync-sources/{sync_source_id}/attempts/start",
response_model=AddressSyncSourceResponse,
)
def api_start_sync_attempt( def api_start_sync_attempt(
sync_source_id: str, sync_source_id: str,
principal: ApiPrincipal = Depends(get_api_principal), principal: ApiPrincipal = Depends(get_api_principal),
@@ -1676,7 +1740,10 @@ def api_start_sync_attempt(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/attempts/finish", response_model=AddressSyncSourceResponse) @router.post(
"/sync-sources/{sync_source_id}/attempts/finish",
response_model=AddressSyncSourceResponse,
)
def api_finish_sync_attempt( def api_finish_sync_attempt(
sync_source_id: str, sync_source_id: str,
payload: AddressSyncAttemptFinishRequest, payload: AddressSyncAttemptFinishRequest,
@@ -1692,7 +1759,11 @@ def api_finish_sync_attempt(
action="addresses.sync_finished", action="addresses.sync_finished",
object_type="address_sync_source", object_type="address_sync_source",
object_id=sync_source.id, object_id=sync_source.id,
details={"connector_type": sync_source.connector_type, "status": sync_source.status, "error": sync_source.last_error}, details={
"connector_type": sync_source.connector_type,
"status": sync_source.status,
"error": sync_source.last_error,
},
) )
session.commit() session.commit()
session.refresh(sync_source) session.refresh(sync_source)
@@ -1702,7 +1773,10 @@ def api_finish_sync_attempt(
raise _error(exc) from exc raise _error(exc) from exc
@router.get("/sync-sources/{sync_source_id}/diagnostics", response_model=AddressSyncDiagnosticListResponse) @router.get(
"/sync-sources/{sync_source_id}/diagnostics",
response_model=AddressSyncDiagnosticListResponse,
)
def api_list_sync_diagnostics( def api_list_sync_diagnostics(
sync_source_id: str, sync_source_id: str,
limit: int = Query(default=100, ge=1, le=500), limit: int = Query(default=100, ge=1, le=500),
@@ -1717,7 +1791,11 @@ def api_list_sync_diagnostics(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/diagnostics", response_model=AddressSyncDiagnosticResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/sync-sources/{sync_source_id}/diagnostics",
response_model=AddressSyncDiagnosticResponse,
status_code=status.HTTP_201_CREATED,
)
def api_record_sync_diagnostic( def api_record_sync_diagnostic(
sync_source_id: str, sync_source_id: str,
payload: AddressSyncDiagnosticCreateRequest, payload: AddressSyncDiagnosticCreateRequest,
@@ -1735,7 +1813,10 @@ def api_record_sync_diagnostic(
raise _error(exc) from exc raise _error(exc) from exc
@router.get("/sync-sources/{sync_source_id}/tombstones", response_model=AddressSyncTombstoneListResponse) @router.get(
"/sync-sources/{sync_source_id}/tombstones",
response_model=AddressSyncTombstoneListResponse,
)
def api_list_sync_tombstones( def api_list_sync_tombstones(
sync_source_id: str, sync_source_id: str,
limit: int = Query(default=200, ge=1, le=1000), limit: int = Query(default=200, ge=1, le=1000),
@@ -1750,7 +1831,11 @@ def api_list_sync_tombstones(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/tombstones", response_model=AddressSyncTombstoneResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/sync-sources/{sync_source_id}/tombstones",
response_model=AddressSyncTombstoneResponse,
status_code=status.HTTP_201_CREATED,
)
def api_record_sync_tombstone( def api_record_sync_tombstone(
sync_source_id: str, sync_source_id: str,
payload: AddressSyncTombstoneCreateRequest, payload: AddressSyncTombstoneCreateRequest,
@@ -1768,7 +1853,10 @@ def api_record_sync_tombstone(
raise _error(exc) from exc raise _error(exc) from exc
@router.get("/sync-sources/{sync_source_id}/conflicts", response_model=AddressSyncConflictListResponse) @router.get(
"/sync-sources/{sync_source_id}/conflicts",
response_model=AddressSyncConflictListResponse,
)
def api_list_sync_conflicts( def api_list_sync_conflicts(
sync_source_id: str, sync_source_id: str,
status_filter: str | None = Query(default="open", alias="status"), status_filter: str | None = Query(default="open", alias="status"),
@@ -1784,7 +1872,11 @@ def api_list_sync_conflicts(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/conflicts", response_model=AddressSyncConflictResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/sync-sources/{sync_source_id}/conflicts",
response_model=AddressSyncConflictResponse,
status_code=status.HTTP_201_CREATED,
)
def api_record_sync_conflict( def api_record_sync_conflict(
sync_source_id: str, sync_source_id: str,
payload: AddressSyncConflictCreateRequest, payload: AddressSyncConflictCreateRequest,
@@ -1818,7 +1910,11 @@ def api_resolve_sync_conflict(
action="addresses.sync_conflict_resolved", action="addresses.sync_conflict_resolved",
object_type="address_sync_conflict", object_type="address_sync_conflict",
object_id=conflict.id, object_id=conflict.id,
details={"sync_source_id": conflict.sync_source_id, "status": conflict.status, "resolution": conflict.resolution}, details={
"sync_source_id": conflict.sync_source_id,
"status": conflict.status,
"resolution": conflict.resolution,
},
) )
session.commit() session.commit()
session.refresh(conflict) session.refresh(conflict)
@@ -1828,7 +1924,11 @@ def api_resolve_sync_conflict(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/address-books/{book_id}/contacts", response_model=ContactResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-books/{book_id}/contacts",
response_model=ContactResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_contact( def api_create_contact(
book_id: str, book_id: str,
payload: ContactCreateRequest, payload: ContactCreateRequest,
@@ -1869,11 +1969,7 @@ def api_update_contact(
_require_scope(principal, "addresses:contact:write") _require_scope(principal, "addresses:contact:write")
try: try:
previous_contact = session.get(Contact, contact_id) previous_contact = session.get(Contact, contact_id)
previous_point_details = ( previous_point_details = _contact_point_audit_details(previous_contact, prefix="previous") if previous_contact is not None else {}
_contact_point_audit_details(previous_contact, prefix="previous")
if previous_contact is not None
else {}
)
contact = update_contact(session, principal, contact_id, payload) contact = update_contact(session, principal, contact_id, payload)
audit_from_principal( audit_from_principal(
session, session,
@@ -1897,7 +1993,10 @@ def api_update_contact(
raise _error(exc) from exc raise _error(exc) from exc
@router.get("/contacts/{contact_id}/channel-rules", response_model=ContactChannelRuleListResponse) @router.get(
"/contacts/{contact_id}/channel-rules",
response_model=ContactChannelRuleListResponse,
)
def api_list_contact_channel_rules( def api_list_contact_channel_rules(
contact_id: str, contact_id: str,
principal: ApiPrincipal = Depends(get_api_principal), principal: ApiPrincipal = Depends(get_api_principal),
@@ -1905,12 +2004,7 @@ def api_list_contact_channel_rules(
): ):
_require_scope(principal, "addresses:governance:read") _require_scope(principal, "addresses:governance:read")
try: try:
return ContactChannelRuleListResponse( return ContactChannelRuleListResponse(rules=[_channel_rule_response(rule) for rule in list_contact_channel_rules(session, principal, contact_id)])
rules=[
_channel_rule_response(rule)
for rule in list_contact_channel_rules(session, principal, contact_id)
]
)
except AddressBookError as exc: except AddressBookError as exc:
raise _error(exc) from exc raise _error(exc) from exc
@@ -2044,7 +2138,11 @@ def api_restore_contact(
raise _error(exc) from exc raise _error(exc) from exc
@router.post("/address-books/{book_id}/vcards/import", response_model=VCardImportResponse, status_code=status.HTTP_201_CREATED) @router.post(
"/address-books/{book_id}/vcards/import",
response_model=VCardImportResponse,
status_code=status.HTTP_201_CREATED,
)
def api_import_address_book_vcards( def api_import_address_book_vcards(
book_id: str, book_id: str,
payload: VCardImportRequest, payload: VCardImportRequest,
@@ -2091,6 +2189,136 @@ def api_import_address_book_vcards(
raise _error(AddressBookError(str(exc))) from exc raise _error(AddressBookError(str(exc))) from exc
@router.post(
"/address-books/{book_id}/vcard-batches/preview",
response_model=VCardBatchRunResponse,
status_code=status.HTTP_201_CREATED,
)
def api_preview_vcard_batch(
book_id: str,
payload: VCardBatchPreviewRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:write")
try:
run = preview_vcard_batch(session, principal, book_id, payload)
audit_from_principal(
session,
principal,
action="addresses.vcard_batch_previewed",
object_type="address_import_run",
object_id=run.id,
details={
"address_book_id": book_id,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
)
session.commit()
session.refresh(run)
return VCardBatchRunResponse.model_validate(vcard_batch_payload(run))
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.get("/vcard-batches/{run_id}", response_model=VCardBatchRunResponse)
def api_get_vcard_batch(
run_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
try:
return VCardBatchRunResponse.model_validate(vcard_batch_payload(get_vcard_batch_run(session, principal, run_id)))
except AddressBookError as exc:
raise _error(exc) from exc
@router.post("/vcard-batches/{run_id}/apply", response_model=VCardBatchRunResponse)
def api_apply_vcard_batch(
run_id: str,
payload: VCardBatchCommitRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:write")
try:
run = apply_vcard_batch(session, principal, run_id, payload)
session.flush()
audit_from_principal(
session,
principal,
action="addresses.vcard_batch_applied",
object_type="address_import_run",
object_id=run.id,
details={
"address_book_id": run.address_book_id,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
)
session.commit()
session.refresh(run)
return VCardBatchRunResponse.model_validate(vcard_batch_payload(run))
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.post("/vcard-batches/{run_id}/cancel", response_model=VCardBatchRunResponse)
def api_cancel_vcard_batch(
run_id: str,
payload: VCardBatchCancelRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:write")
try:
run = cancel_vcard_batch(session, principal, run_id, payload)
audit_from_principal(
session,
principal,
action="addresses.vcard_batch_cancelled",
object_type="address_import_run",
object_id=run.id,
details={"address_book_id": run.address_book_id, "reason": payload.reason},
)
session.commit()
session.refresh(run)
return VCardBatchRunResponse.model_validate(vcard_batch_payload(run))
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.get("/vcard-batches/{run_id}/diagnostics")
def api_export_vcard_batch_diagnostics(
run_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
try:
run = get_vcard_batch_run(session, principal, run_id)
content = json.dumps(
vcard_diagnostics_payload(run),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
return Response(
content=content,
media_type="application/json",
headers={"Content-Disposition": f'attachment; filename="vcard-batch-{run.id}.json"'},
)
except AddressBookError as exc:
raise _error(exc) from exc
@router.get("/import-profiles", response_model=AddressImportProfileListResponse) @router.get("/import-profiles", response_model=AddressImportProfileListResponse)
def api_list_address_import_profiles( def api_list_address_import_profiles(
include_history: bool = Query(default=False), include_history: bool = Query(default=False),
@@ -2098,15 +2326,14 @@ def api_list_address_import_profiles(
session: Session = Depends(get_session), session: Session = Depends(get_session),
): ):
_require_scope(principal, "addresses:contact:read") _require_scope(principal, "addresses:contact:read")
return AddressImportProfileListResponse( return AddressImportProfileListResponse(profiles=[AddressImportProfileResponse.model_validate(item) for item in list_import_profiles(session, principal, include_history=include_history)])
profiles=[
AddressImportProfileResponse.model_validate(item)
for item in list_import_profiles(session, principal, include_history=include_history) @router.post(
] "/import-profiles",
response_model=AddressImportProfileResponse,
status_code=status.HTTP_201_CREATED,
) )
@router.post("/import-profiles", response_model=AddressImportProfileResponse, status_code=status.HTTP_201_CREATED)
def api_create_address_import_profile( def api_create_address_import_profile(
payload: AddressImportProfileCreateRequest, payload: AddressImportProfileCreateRequest,
principal: ApiPrincipal = Depends(get_api_principal), principal: ApiPrincipal = Depends(get_api_principal),
@@ -2122,7 +2349,11 @@ def api_create_address_import_profile(
action="addresses.import_profile_created", action="addresses.import_profile_created",
object_type="address_import_profile", object_type="address_import_profile",
object_id=profile.profile_key, object_id=profile.profile_key,
details={"version": profile.version, "source_format": profile.source_format, "scope_type": profile.scope_type}, details={
"version": profile.version,
"source_format": profile.source_format,
"scope_type": profile.scope_type,
},
) )
session.commit() session.commit()
session.refresh(profile) session.refresh(profile)
@@ -2149,7 +2380,10 @@ def api_update_address_import_profile(
action="addresses.import_profile_versioned", action="addresses.import_profile_versioned",
object_type="address_import_profile", object_type="address_import_profile",
object_id=profile.profile_key, object_id=profile.profile_key,
details={"version": profile.version, "source_format": profile.source_format}, details={
"version": profile.version,
"source_format": profile.source_format,
},
) )
session.commit() session.commit()
session.refresh(profile) session.refresh(profile)
@@ -2202,7 +2436,11 @@ def api_preview_address_import(
action="addresses.import_previewed", action="addresses.import_previewed",
object_type="address_import_run", object_type="address_import_run",
object_id=run.id, object_id=run.id,
details={"input_hash": run.input_hash, "plan_hash": run.plan_hash, "statistics": run.statistics}, details={
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
) )
session.commit() session.commit()
session.refresh(run) session.refresh(run)
@@ -2220,9 +2458,7 @@ def api_get_address_import_run(
): ):
_require_scope(principal, "addresses:contact:read") _require_scope(principal, "addresses:contact:read")
try: try:
return AddressImportRunResponse.model_validate( return AddressImportRunResponse.model_validate(import_run_payload(get_import_run(session, principal, run_id)))
import_run_payload(get_import_run(session, principal, run_id))
)
except AddressBookError as exc: except AddressBookError as exc:
raise _error(exc) from exc raise _error(exc) from exc
@@ -2244,7 +2480,11 @@ def api_apply_address_import(
action="addresses.import_applied", action="addresses.import_applied",
object_type="address_import_run", object_type="address_import_run",
object_id=run.id, object_id=run.id,
details={"input_hash": run.input_hash, "plan_hash": run.plan_hash, "statistics": run.statistics}, details={
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
) )
session.commit() session.commit()
session.refresh(run) session.refresh(run)
@@ -2300,6 +2540,39 @@ def api_export_address_book_vcards(
raise _error(exc) from exc raise _error(exc) from exc
@router.post(
"/address-books/{book_id}/vcards/export",
response_model=VCardExportResponse,
)
def api_export_selected_vcards(
book_id: str,
payload: VCardExportRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
try:
result = export_vcards(session, principal, book_id, payload)
audit_from_principal(
session,
principal,
action="addresses.vcards_exported",
object_type="address_book",
object_id=book_id,
details={
"scope": result["scope"],
"version": result["version"],
"contact_count": result["contact_count"],
"content_hash": result["content_hash"],
},
)
session.commit()
return VCardExportResponse.model_validate(result)
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.get("/contacts/{contact_id}/vcard") @router.get("/contacts/{contact_id}/vcard")
def api_export_contact_vcard( def api_export_contact_vcard(
contact_id: str, contact_id: str,
+5 -2
View File
@@ -3278,8 +3278,11 @@ def _filtered_contact_query(
return contact_query return contact_query
def get_visible_contact(session: Session, principal: ApiPrincipal, contact_id: str, *, include_deleted: bool = False) -> Contact: def get_visible_contact(session: Session, principal: ApiPrincipal, contact_id: str, *, include_deleted: bool = False, lock: bool = False) -> Contact:
contact = _visible_contact_query(session, principal, include_deleted=include_deleted, include_deleted_books=include_deleted).filter(Contact.id == contact_id).one_or_none() query = _visible_contact_query(session, principal, include_deleted=include_deleted, include_deleted_books=include_deleted).filter(Contact.id == contact_id)
if lock:
query = query.populate_existing().with_for_update(of=Contact)
contact = query.one_or_none()
if contact is None: if contact is None:
raise AddressBookError("Contact not found.") raise AddressBookError("Contact not found.")
return contact return contact
+82 -35
View File
@@ -16,6 +16,12 @@ class VCardError(ValueError):
pass pass
VCARD_PARSER_VERSION = "govoplan-vcard/2"
MAX_VCARD_CARDS = 10_000
MAX_VCARD_LINES = 200_000
MAX_VCARD_UNFOLDED_LINE_CHARS = 16_384
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
class ParsedVCard: class ParsedVCard:
payload: ContactCreateRequest payload: ContactCreateRequest
@@ -60,12 +66,16 @@ class _VCardDraft:
def _normalize_lines(content: str) -> list[str]: def _normalize_lines(content: str) -> list[str]:
raw_lines = content.replace("\r\n", "\n").replace("\r", "\n").split("\n") raw_lines = content.replace("\r\n", "\n").replace("\r", "\n").split("\n")
if len(raw_lines) > MAX_VCARD_LINES:
raise VCardError(f"vCard input exceeds the {MAX_VCARD_LINES}-line parser limit.")
lines: list[str] = [] lines: list[str] = []
for line in raw_lines: for line in raw_lines:
if line.startswith((" ", "\t")) and lines: if line.startswith((" ", "\t")) and lines:
lines[-1] += line[1:] lines[-1] += line[1:]
elif line: elif line:
lines.append(line) lines.append(line)
if lines and len(lines[-1]) > MAX_VCARD_UNFOLDED_LINE_CHARS:
raise VCardError(f"vCard unfolded lines are limited to {MAX_VCARD_UNFOLDED_LINE_CHARS} characters.")
return lines return lines
@@ -90,27 +100,13 @@ def _split_unescaped(value: str, separator: str) -> list[str]:
def _unescape_text(value: str) -> str: def _unescape_text(value: str) -> str:
return ( return value.replace("\\n", "\n").replace("\\N", "\n").replace("\\,", ",").replace("\\;", ";").replace("\\\\", "\\").strip()
value.replace("\\n", "\n")
.replace("\\N", "\n")
.replace("\\,", ",")
.replace("\\;", ";")
.replace("\\\\", "\\")
.strip()
)
def _escape_text(value: str | None) -> str: def _escape_text(value: str | None) -> str:
if value is None: if value is None:
return "" return ""
return ( return value.replace("\\", "\\\\").replace("\r\n", "\n").replace("\r", "\n").replace("\n", "\\n").replace(";", "\\;").replace(",", "\\,")
value.replace("\\", "\\\\")
.replace("\r\n", "\n")
.replace("\r", "\n")
.replace("\n", "\\n")
.replace(";", "\\;")
.replace(",", "\\,")
)
def _parse_head(head: str) -> tuple[str, dict[str, list[str]]]: def _parse_head(head: str) -> tuple[str, dict[str, list[str]]]:
@@ -154,7 +150,7 @@ def _is_pref(params: dict[str, list[str]]) -> bool:
def _card_blocks(content: str) -> list[list[str]]: def _card_blocks(content: str) -> list[list[str]]:
result = _card_blocks_with_issues(content) result = _card_blocks_with_issues(content, max_cards=MAX_VCARD_CARDS)
if result.issues: if result.issues:
raise VCardError(result.issues[0].message) raise VCardError(result.issues[0].message)
return result.cards return result.cards
@@ -166,8 +162,14 @@ class _CardBlockResult:
issues: list[ParsedVCardIssue] issues: list[ParsedVCardIssue]
def _card_blocks_with_issues(content: str) -> _CardBlockResult: def _card_blocks_with_issues(content: str, *, max_cards: int) -> _CardBlockResult:
try:
lines = _normalize_lines(content) lines = _normalize_lines(content)
except VCardError as exc:
return _CardBlockResult(
cards=[],
issues=[ParsedVCardIssue(index=0, message=str(exc))],
)
blocks: list[list[str]] = [] blocks: list[list[str]] = []
issues: list[ParsedVCardIssue] = [] issues: list[ParsedVCardIssue] = []
current: list[str] | None = None current: list[str] | None = None
@@ -189,6 +191,15 @@ def _card_blocks_with_issues(content: str) -> _CardBlockResult:
issues.append(ParsedVCardIssue(index=card_index, message="vCard END appears before BEGIN.")) issues.append(ParsedVCardIssue(index=card_index, message="vCard END appears before BEGIN."))
continue continue
current.append(line) current.append(line)
if len(blocks) >= max_cards:
issues.append(
ParsedVCardIssue(
index=len(blocks) + 1,
message=f"vCard input exceeds the configured {max_cards}-card limit.",
)
)
current = None
break
blocks.append(current) blocks.append(current)
current = None current = None
elif current is not None: elif current is not None:
@@ -258,7 +269,14 @@ def _apply_card_metadata(
if name == "VERSION": if name == "VERSION":
draft.version = value.strip() draft.version = value.strip()
if draft.version and draft.version not in {"3.0", "4.0"}: if draft.version and draft.version not in {"3.0", "4.0"}:
issues.append(ParsedVCardIssue(index=index, severity="warning", field="VERSION", message=f"vCard version {draft.version} is not fully supported.")) issues.append(
ParsedVCardIssue(
index=index,
severity="warning",
field="VERSION",
message=f"vCard version {draft.version} is not fully supported.",
)
)
return True return True
if name == "UID": if name == "UID":
draft.uid = _unescape_text(value) or draft.uid draft.uid = _unescape_text(value) or draft.uid
@@ -325,15 +343,34 @@ def _append_card_email(
if not email: if not email:
return return
if "@" not in email: if "@" not in email:
issues.append(ParsedVCardIssue(index=index, severity="warning", field="EMAIL", message=f"Skipped invalid email address: {email}")) issues.append(
ParsedVCardIssue(
index=index,
severity="warning",
field="EMAIL",
message=f"Skipped invalid email address: {email}",
)
)
return return
draft.emails.append(ContactEmailPayload(label=_label_from_params(params), email=email, is_primary=_is_pref(params) or not draft.emails)) draft.emails.append(
ContactEmailPayload(
label=_label_from_params(params),
email=email,
is_primary=_is_pref(params) or not draft.emails,
)
)
def _append_card_phone(draft: _VCardDraft, params: dict[str, list[str]], value: str) -> None: def _append_card_phone(draft: _VCardDraft, params: dict[str, list[str]], value: str) -> None:
phone = _unescape_text(value) phone = _unescape_text(value)
if phone: if phone:
draft.phones.append(ContactPhonePayload(label=_label_from_params(params), phone=phone, is_primary=_is_pref(params) or not draft.phones)) draft.phones.append(
ContactPhonePayload(
label=_label_from_params(params),
phone=phone,
is_primary=_is_pref(params) or not draft.phones,
)
)
def _append_card_address(draft: _VCardDraft, params: dict[str, list[str]], value: str) -> None: def _append_card_address(draft: _VCardDraft, params: dict[str, list[str]], value: str) -> None:
@@ -388,8 +425,14 @@ def _draft_contact_payload(draft: _VCardDraft) -> ContactCreateRequest:
return payload return payload
def parse_vcards_with_issues(content: str) -> VCardParseResult: def parse_vcards_with_issues(
blocks = _card_blocks_with_issues(content) content: str,
*,
max_cards: int = MAX_VCARD_CARDS,
) -> VCardParseResult:
if max_cards < 1 or max_cards > MAX_VCARD_CARDS:
raise VCardError(f"max_cards must be between 1 and {MAX_VCARD_CARDS}.")
blocks = _card_blocks_with_issues(content, max_cards=max_cards)
parsed: list[ParsedVCard] = [] parsed: list[ParsedVCard] = []
issues = list(blocks.issues) issues = list(blocks.issues)
skipped = 0 skipped = 0
@@ -411,8 +454,8 @@ def parse_vcards(content: str) -> list[ParsedVCard]:
return result.cards return result.cards
def contact_to_vcard(contact: Contact) -> str: def contact_to_vcard(contact: Contact, *, version: Literal["3.0", "4.0"] = "4.0") -> str:
lines = _contact_identity_lines(contact) lines = _contact_identity_lines(contact, version=version)
lines.extend(_contact_email_lines(contact)) lines.extend(_contact_email_lines(contact))
lines.extend(_contact_phone_lines(contact)) lines.extend(_contact_phone_lines(contact))
lines.extend(_contact_address_lines(contact)) lines.extend(_contact_address_lines(contact))
@@ -422,10 +465,14 @@ def contact_to_vcard(contact: Contact) -> str:
return "\r\n".join(lines) + "\r\n" return "\r\n".join(lines) + "\r\n"
def _contact_identity_lines(contact: Contact) -> list[str]: def _contact_identity_lines(
contact: Contact,
*,
version: Literal["3.0", "4.0"],
) -> list[str]:
lines = [ lines = [
"BEGIN:VCARD", "BEGIN:VCARD",
"VERSION:4.0", f"VERSION:{version}",
f"FN:{_escape_text(contact.display_name)}", f"FN:{_escape_text(contact.display_name)}",
f"N:{_escape_text(contact.family_name)};{_escape_text(contact.given_name)};;;", f"N:{_escape_text(contact.family_name)};{_escape_text(contact.given_name)};;;",
] ]
@@ -460,11 +507,7 @@ def _contact_address_lines(contact: Contact) -> list[str]:
lines: list[str] = [] lines: list[str] = []
for address in contact.postal_addresses: for address in contact.postal_addresses:
label = f";TYPE={_escape_text(address.label)}" if address.label else "" label = f";TYPE={_escape_text(address.label)}" if address.label else ""
lines.append( lines.append(f"ADR{label}:;;{_escape_text(address.street)};{_escape_text(address.locality)};{_escape_text(address.region)};{_escape_text(address.postal_code)};{_escape_text(address.country)}")
"ADR"
f"{label}:;;{_escape_text(address.street)};{_escape_text(address.locality)};"
f"{_escape_text(address.region)};{_escape_text(address.postal_code)};{_escape_text(address.country)}"
)
return lines return lines
@@ -496,5 +539,9 @@ def _contact_vcard_urls(contact: Contact) -> object:
return vcard.get("urls") return vcard.get("urls")
def contacts_to_vcard(contacts: list[Contact]) -> str: def contacts_to_vcard(
return "".join(contact_to_vcard(contact) for contact in contacts) contacts: list[Contact],
*,
version: Literal["3.0", "4.0"] = "4.0",
) -> str:
return "".join(contact_to_vcard(contact, version=version) for contact in contacts)
@@ -0,0 +1,144 @@
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, Field, model_validator
VCardPlanAction = Literal["create", "update", "ignore", "unchanged", "conflict"]
VCardCommitAction = Literal["create", "update", "ignore"]
class VCardBatchFilePayload(BaseModel):
filename: str = Field(min_length=1, max_length=500)
content_base64: str = Field(min_length=1, max_length=14_000_000)
class VCardBatchPreviewRequest(BaseModel):
files: list[VCardBatchFilePayload] = Field(min_length=1, max_length=50)
duplicate_card_policy: Literal["reject", "first", "last"] = "reject"
existing_contact_policy: Literal["update", "ignore", "reject"] = "update"
class VCardDuplicateSuggestion(BaseModel):
contact_id: str
display_name: str
reasons: list[str] = Field(default_factory=list)
class VCardBatchPlanItemResponse(BaseModel):
source_key: str
source_filename: str
card_index: int
action: VCardPlanAction
allowed_actions: list[VCardCommitAction] = Field(default_factory=list)
contact_id: str | None = None
display_name: str | None = None
changed_fields: list[str] = Field(default_factory=list)
duplicate_suggestions: list[VCardDuplicateSuggestion] = Field(default_factory=list)
message: str | None = None
class VCardBatchDiagnosticResponse(BaseModel):
severity: Literal["info", "warning", "error"]
code: str
message: str
source_filename: str | None = None
card_index: int | None = None
field: str | None = None
details: dict[str, Any] = Field(default_factory=dict)
class VCardBatchProgressResponse(BaseModel):
total: int
completed: int
created: int = 0
updated: int = 0
ignored: int = 0
failed: int = 0
class VCardBatchRunResponse(BaseModel):
id: str
address_book_id: str
status: str
input_hash: str
plan_hash: str
parser_version: str
execution_mode: Literal["bounded_sync", "persisted_batch"]
file_count: int
card_count: int
statistics: dict[str, int | str] = Field(default_factory=dict)
diagnostics: list[VCardBatchDiagnosticResponse] = Field(default_factory=list)
plan: list[VCardBatchPlanItemResponse] = Field(default_factory=list)
progress: VCardBatchProgressResponse
can_apply: bool
can_cancel: bool
commit_hash: str | None = None
created_at: datetime
updated_at: datetime
applied_at: datetime | None = None
class VCardBatchSelection(BaseModel):
source_key: str = Field(min_length=1, max_length=1000)
action: VCardCommitAction
class VCardBatchCommitRequest(BaseModel):
expected_plan_hash: str = Field(min_length=64, max_length=64)
selections: list[VCardBatchSelection] = Field(
default_factory=list, max_length=10_000
)
class VCardBatchCancelRequest(BaseModel):
expected_plan_hash: str = Field(min_length=64, max_length=64)
reason: str = Field(min_length=3, max_length=2000)
class VCardExportRequest(BaseModel):
scope: Literal["address_book", "address_list", "contacts"] = "address_book"
address_list_id: str | None = Field(default=None, max_length=36)
contact_ids: list[str] = Field(default_factory=list, max_length=10_000)
version: Literal["3.0", "4.0"] = "4.0"
@model_validator(mode="after")
def validate_scope(self) -> "VCardExportRequest":
if self.scope == "address_list" and not self.address_list_id:
raise ValueError("Address-list export requires address_list_id.")
if self.scope == "contacts" and not self.contact_ids:
raise ValueError(
"Selected-contact export requires at least one contact id."
)
if self.scope != "address_list" and self.address_list_id:
raise ValueError("address_list_id is only valid for address-list export.")
if self.scope != "contacts" and self.contact_ids:
raise ValueError("contact_ids are only valid for selected-contact export.")
if len(set(self.contact_ids)) != len(self.contact_ids):
raise ValueError("Selected contact ids must be unique.")
return self
class VCardExportResponse(BaseModel):
filename: str
media_type: str = "text/vcard"
scope: str
version: str
ordering: str
contact_count: int
content_hash: str
content: str
__all__ = [
"VCardBatchCancelRequest",
"VCardBatchCommitRequest",
"VCardBatchFilePayload",
"VCardBatchPreviewRequest",
"VCardBatchRunResponse",
"VCardBatchSelection",
"VCardExportRequest",
"VCardExportResponse",
]
@@ -0,0 +1,901 @@
from __future__ import annotations
import base64
import binascii
from collections import Counter, defaultdict
import hashlib
import json
import os
from typing import Any
from sqlalchemy.orm import Session, selectinload
from govoplan_addresses.backend.db.models import (
AddressImportRun,
AddressListEntry,
Contact,
)
from govoplan_addresses.backend.schemas import (
ContactCreateRequest,
ContactUpdateRequest,
)
from govoplan_addresses.backend.service import (
AddressBookError,
create_contact,
get_visible_address_book,
get_visible_address_list,
get_visible_contact,
update_contact,
)
from govoplan_addresses.backend.vcard import (
MAX_VCARD_CARDS,
VCARD_PARSER_VERSION,
contacts_to_vcard,
parse_vcards_with_issues,
)
from govoplan_addresses.backend.vcard_batch_schemas import (
VCardBatchCancelRequest,
VCardBatchCommitRequest,
VCardBatchPreviewRequest,
VCardExportRequest,
)
from govoplan_core.auth import ApiPrincipal
from govoplan_core.db.base import utcnow
MAX_VCARD_BATCH_BYTES = 10_000_000
DEFAULT_PERSISTED_BATCH_THRESHOLD = 500
def preview_vcard_batch(
session: Session,
principal: ApiPrincipal,
address_book_id: str,
payload: VCardBatchPreviewRequest,
) -> AddressImportRun:
book = get_visible_address_book(session, principal, address_book_id)
if book.read_only:
raise AddressBookError("Static vCard imports require a writable address book.")
decoded = _decode_files(payload)
input_hash = _hash_json(
{
"files": [
{
"filename": filename,
"sha256": hashlib.sha256(raw).hexdigest(),
"size": len(raw),
}
for filename, raw in decoded
]
}
)
parsed_cards, diagnostics = _parse_files(decoded)
plan = _plan_cards(
session,
book.id,
parsed_cards,
duplicate_card_policy=payload.duplicate_card_policy,
existing_contact_policy=payload.existing_contact_policy,
)
statistics: dict[str, int | str] = dict(
Counter(str(item["action"]) for item in plan)
)
statistics.update(
{
"files": len(decoded),
"cards": len(parsed_cards),
"errors": sum(item["severity"] == "error" for item in diagnostics),
"warnings": sum(item["severity"] == "warning" for item in diagnostics),
"parser_version": VCARD_PARSER_VERSION,
"execution_mode": _execution_mode(len(parsed_cards)),
}
)
plan_hash = _hash_json(
{
"address_book_id": book.id,
"input_hash": input_hash,
"parser_version": VCARD_PARSER_VERSION,
"duplicate_card_policy": payload.duplicate_card_policy,
"existing_contact_policy": payload.existing_contact_policy,
"plan": plan,
}
)
source_filename = decoded[0][0]
if len(decoded) > 1:
source_filename = f"{source_filename} (+{len(decoded) - 1} files)"
run = AddressImportRun(
tenant_id=book.tenant_id,
address_book_id=book.id,
profile_id=None,
source_filename=source_filename[:500],
source_format="vcard",
input_hash=input_hash,
plan_hash=plan_hash,
status="previewed",
row_count=len(parsed_cards),
statistics=statistics,
diagnostics=diagnostics,
plan_data=plan,
result_evidence={
"parser_version": VCARD_PARSER_VERSION,
"file_manifest": [
{
"filename": filename,
"sha256": hashlib.sha256(raw).hexdigest(),
"size": len(raw),
}
for filename, raw in decoded
],
"progress": _progress(len(plan)),
},
created_by_account_id=principal.account_id,
)
session.add(run)
session.flush()
return run
def get_vcard_batch_run(
session: Session,
principal: ApiPrincipal,
run_id: str,
) -> AddressImportRun:
book_ids = [book.id for book in _visible_books(session, principal)]
if not book_ids:
raise AddressBookError("vCard batch run not found.")
item = (
session.query(AddressImportRun)
.filter(
AddressImportRun.id == run_id,
AddressImportRun.address_book_id.in_(book_ids),
AddressImportRun.source_format == "vcard",
)
.one_or_none()
)
if item is None:
raise AddressBookError("vCard batch run not found.")
return item
def apply_vcard_batch(
session: Session,
principal: ApiPrincipal,
run_id: str,
payload: VCardBatchCommitRequest,
) -> AddressImportRun:
run = get_vcard_batch_run(session, principal, run_id)
if run.plan_hash != payload.expected_plan_hash:
raise AddressBookError("The reviewed vCard plan changed; create a new preview.")
selections = _selection_map(payload)
commit_hash = _hash_json(
{
"plan_hash": run.plan_hash,
"selections": [
{"source_key": key, "action": selections[key]}
for key in sorted(selections)
],
}
)
evidence = dict(run.result_evidence or {})
if run.status == "applied":
if evidence.get("commit_hash") != commit_hash:
raise AddressBookError(
"This vCard batch was already applied with a different selection."
)
return run
if run.status != "previewed":
raise AddressBookError(
f"vCard batch cannot be applied from status {run.status!r}."
)
if not selections:
raise AddressBookError(
"Select at least one vCard action before applying the batch."
)
plan_by_key = {str(item["source_key"]): item for item in run.plan_data or []}
unknown = sorted(set(selections).difference(plan_by_key))
if unknown:
raise AddressBookError(
"The selection contains cards that are not part of the reviewed plan."
)
created_ids: list[str] = []
updated_ids: list[str] = []
ignored = 0
for source_key in sorted(plan_by_key):
item = plan_by_key[source_key]
action = selections.get(source_key, "ignore")
allowed = set(item.get("allowed_actions") or [])
if action not in allowed:
raise AddressBookError(
f'Action {action!r} is not allowed for vCard "{item.get("display_name") or source_key}".'
)
if action == "ignore":
ignored += 1
continue
contact = _apply_plan_item(
session,
principal,
run=run,
item=item,
action=action,
)
if action == "create":
created_ids.append(contact.id)
else:
updated_ids.append(contact.id)
run.status = "applied"
run.applied_at = utcnow()
run.result_evidence = {
**evidence,
"commit_hash": commit_hash,
"selection_count": len(selections),
"created_contact_ids": created_ids,
"updated_contact_ids": updated_ids,
"ignored_count": ignored,
"applied_by_account_id": principal.account_id,
"applied_at": run.applied_at.isoformat(),
"progress": {
"total": len(plan_by_key),
"completed": len(plan_by_key),
"created": len(created_ids),
"updated": len(updated_ids),
"ignored": ignored,
"failed": 0,
},
}
run.statistics = {
**dict(run.statistics or {}),
"applied_create": len(created_ids),
"applied_update": len(updated_ids),
"applied_ignore": ignored,
}
return run
def cancel_vcard_batch(
session: Session,
principal: ApiPrincipal,
run_id: str,
payload: VCardBatchCancelRequest,
) -> AddressImportRun:
run = get_vcard_batch_run(session, principal, run_id)
if run.plan_hash != payload.expected_plan_hash:
raise AddressBookError("The reviewed vCard plan changed; reload the batch.")
if run.status == "cancelled":
return run
if run.status != "previewed":
raise AddressBookError("Only a previewed vCard batch can be cancelled.")
run.status = "cancelled"
run.result_evidence = {
**dict(run.result_evidence or {}),
"cancel_reason": payload.reason.strip(),
"cancelled_by_account_id": principal.account_id,
"cancelled_at": utcnow().isoformat(),
}
return run
def vcard_batch_payload(run: AddressImportRun) -> dict[str, Any]:
evidence = dict(run.result_evidence or {})
progress = dict(evidence.get("progress") or _progress(run.row_count))
return {
"id": run.id,
"address_book_id": run.address_book_id,
"status": run.status,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"parser_version": str(evidence.get("parser_version") or VCARD_PARSER_VERSION),
"execution_mode": str(
(run.statistics or {}).get("execution_mode") or "bounded_sync"
),
"file_count": int((run.statistics or {}).get("files") or 0),
"card_count": run.row_count,
"statistics": dict(run.statistics or {}),
"diagnostics": list(run.diagnostics or []),
"plan": [_public_plan_item(item) for item in run.plan_data or []],
"progress": progress,
"can_apply": run.status == "previewed" and bool(run.plan_data),
"can_cancel": run.status == "previewed",
"commit_hash": evidence.get("commit_hash"),
"created_at": run.created_at,
"updated_at": run.updated_at,
"applied_at": run.applied_at,
}
def vcard_diagnostics_payload(run: AddressImportRun) -> dict[str, Any]:
return {
"schema_version": "1.0",
"run_id": run.id,
"status": run.status,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"parser_version": (run.result_evidence or {}).get("parser_version"),
"statistics": dict(run.statistics or {}),
"diagnostics": list(run.diagnostics or []),
"effects": [_public_plan_item(item) for item in run.plan_data or []],
}
def export_vcards(
session: Session,
principal: ApiPrincipal,
address_book_id: str,
payload: VCardExportRequest,
) -> dict[str, Any]:
book = get_visible_address_book(session, principal, address_book_id)
contacts = _export_contacts(session, principal, book.id, payload)
contacts.sort(key=lambda item: (item.display_name.casefold(), item.id))
content = contacts_to_vcard(contacts, version=payload.version)
scope_label = {
"address_book": book.name,
"address_list": "address-list",
"contacts": "selected-contacts",
}[payload.scope]
return {
"filename": f"{_safe_filename(scope_label)}-{payload.version.replace('.', '')}.vcf",
"media_type": "text/vcard",
"scope": payload.scope,
"version": payload.version,
"ordering": "display_name_casefold_then_contact_id",
"contact_count": len(contacts),
"content_hash": hashlib.sha256(content.encode("utf-8")).hexdigest(),
"content": content,
}
def _decode_files(payload: VCardBatchPreviewRequest) -> list[tuple[str, bytes]]:
decoded: list[tuple[str, bytes]] = []
total = 0
for item in payload.files:
filename = item.filename.strip()
if not filename.casefold().endswith(".vcf"):
raise AddressBookError("vCard batch uploads accept only .vcf files.")
try:
raw = base64.b64decode(item.content_base64, validate=True)
except (binascii.Error, ValueError) as exc:
raise AddressBookError(
f'vCard file "{filename}" is not valid base64.'
) from exc
if not raw:
raise AddressBookError(f'vCard file "{filename}" is empty.')
total += len(raw)
if total > MAX_VCARD_BATCH_BYTES:
raise AddressBookError(
f"Combined vCard uploads are limited to {MAX_VCARD_BATCH_BYTES} bytes."
)
decoded.append((filename, raw))
return decoded
def _parse_files(
decoded: list[tuple[str, bytes]],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
cards: list[dict[str, Any]] = []
diagnostics: list[dict[str, Any]] = []
for file_index, (filename, raw) in enumerate(decoded):
try:
content = raw.decode("utf-8-sig")
except UnicodeDecodeError as exc:
raise AddressBookError(
f'vCard file "{filename}" is not valid UTF-8: {exc}.'
) from exc
remaining = MAX_VCARD_CARDS - len(cards)
if remaining < 1:
raise AddressBookError(
f"vCard batches are limited to {MAX_VCARD_CARDS} cards."
)
result = parse_vcards_with_issues(content, max_cards=remaining)
for issue in result.issues:
diagnostics.append(
{
"severity": issue.severity,
"code": "vcard_parse_error"
if issue.severity == "error"
else "vcard_parse_warning",
"message": issue.message,
"source_filename": filename,
"card_index": issue.index or None,
"field": issue.field,
"details": {"line": issue.line} if issue.line is not None else {},
}
)
for card_index, parsed in enumerate(result.cards, start=1):
raw_hash = hashlib.sha256(parsed.raw.encode("utf-8")).hexdigest()
identity = (
f"uid:{parsed.source_ref.strip()}"
if parsed.source_ref and parsed.source_ref.strip()
else f"sha256:{raw_hash}"
)
source_key = hashlib.sha256(
f"{file_index}:{filename}:{card_index}:{raw_hash}".encode("utf-8")
).hexdigest()
cards.append(
{
"source_key": source_key,
"source_identity": identity,
"source_filename": filename,
"card_index": card_index,
"raw": parsed.raw,
"source_ref": parsed.source_ref.strip()
if parsed.source_ref
else None,
"source_revision": parsed.source_revision.strip()
if parsed.source_revision
else None,
"payload": parsed.payload.model_dump(mode="json"),
}
)
return cards, diagnostics
def _plan_cards(
session: Session,
address_book_id: str,
cards: list[dict[str, Any]],
*,
duplicate_card_policy: str,
existing_contact_policy: str,
) -> list[dict[str, Any]]:
contacts = (
session.query(Contact)
.options(
selectinload(Contact.emails),
selectinload(Contact.phones),
selectinload(Contact.postal_addresses),
)
.filter(
Contact.address_book_id == address_book_id, Contact.deleted_at.is_(None)
)
.all()
)
by_source: dict[str, list[Contact]] = defaultdict(list)
by_email: dict[str, list[Contact]] = defaultdict(list)
for contact in contacts:
if contact.source_ref:
by_source[contact.source_ref.strip()].append(contact)
for email in contact.emails:
normalized = (email.normalized_email or email.email).strip().casefold()
if normalized:
by_email[normalized].append(contact)
identity_positions: dict[str, list[int]] = defaultdict(list)
for index, card in enumerate(cards):
identity_positions[str(card["source_identity"])].append(index)
result: list[dict[str, Any]] = []
for index, card in enumerate(cards):
positions = identity_positions[str(card["source_identity"])]
if len(positions) > 1:
chosen = positions[0] if duplicate_card_policy == "first" else positions[-1]
if duplicate_card_policy == "reject":
result.append(
_planned_card(
card,
action="conflict",
allowed=["ignore"],
message="Duplicate UID or identical card appears in this batch.",
)
)
continue
if index != chosen:
result.append(
_planned_card(
card,
action="ignore",
allowed=["ignore"],
message=f"Duplicate card ignored by {duplicate_card_policy} policy.",
)
)
continue
suggestions = _duplicate_candidates(
card, by_source=by_source, by_email=by_email
)
exact_source = [item for item in suggestions if "source_uid" in item["reasons"]]
candidates = exact_source or suggestions
if len(candidates) > 1:
result.append(
_planned_card(
card,
action="conflict",
allowed=["create", "ignore"],
suggestions=suggestions,
message="Multiple existing contacts match this card; create explicitly or ignore it.",
)
)
continue
existing = next(
(
contact
for contact in contacts
if candidates and contact.id == candidates[0]["contact_id"]
),
None,
)
if existing is None:
result.append(
_planned_card(
card,
action="create",
allowed=["create", "ignore"],
suggestions=suggestions,
)
)
continue
changed = _changed_fields(existing, card["payload"])
if not changed:
result.append(
_planned_card(
card,
action="unchanged",
allowed=["ignore"],
contact=existing,
suggestions=suggestions,
message="Existing contact already matches the parsed card.",
)
)
elif existing_contact_policy == "reject":
result.append(
_planned_card(
card,
action="conflict",
allowed=["ignore"],
contact=existing,
suggestions=suggestions,
changed=changed,
message="An existing contact matches and the preview policy rejects updates.",
)
)
elif existing_contact_policy == "ignore":
result.append(
_planned_card(
card,
action="ignore",
allowed=["update", "ignore"],
contact=existing,
suggestions=suggestions,
changed=changed,
message="Existing contact is ignored by preview policy.",
)
)
else:
result.append(
_planned_card(
card,
action="update",
allowed=["update", "ignore"],
contact=existing,
suggestions=suggestions,
changed=changed,
)
)
return result
def _planned_card(
card: dict[str, Any],
*,
action: str,
allowed: list[str],
contact: Contact | None = None,
suggestions: list[dict[str, Any]] | None = None,
changed: list[str] | None = None,
message: str | None = None,
) -> dict[str, Any]:
return {
**card,
"row_number": int(card["card_index"]),
"action": action,
"allowed_actions": allowed,
"contact_id": contact.id if contact is not None else None,
"expected_contact_hash": _contact_hash(contact)
if contact is not None
else None,
"display_name": card["payload"].get("display_name"),
"changed_fields": changed or [],
"duplicate_suggestions": suggestions or [],
"message": message,
}
def _duplicate_candidates(
card: dict[str, Any],
*,
by_source: dict[str, list[Contact]],
by_email: dict[str, list[Contact]],
) -> list[dict[str, Any]]:
reasons: dict[str, set[str]] = defaultdict(set)
contacts: dict[str, Contact] = {}
source_ref = card.get("source_ref")
if source_ref:
for contact in by_source.get(str(source_ref), []):
contacts[contact.id] = contact
reasons[contact.id].add("source_uid")
for item in card["payload"].get("emails") or []:
normalized = str(item.get("email") or "").strip().casefold()
for contact in by_email.get(normalized, []):
contacts[contact.id] = contact
reasons[contact.id].add("email")
return [
{
"contact_id": contact_id,
"display_name": contacts[contact_id].display_name,
"reasons": sorted(reasons[contact_id]),
}
for contact_id in sorted(
contacts, key=lambda item: (contacts[item].display_name.casefold(), item)
)[:5]
]
def _apply_plan_item(
session: Session,
principal: ApiPrincipal,
*,
run: AddressImportRun,
item: dict[str, Any],
action: str,
) -> Contact:
contact_payload = ContactCreateRequest.model_validate(item["payload"])
if action == "create":
if item.get("source_ref"):
appeared = (
session.query(Contact)
.filter(
Contact.address_book_id == run.address_book_id,
Contact.source_ref == item["source_ref"],
Contact.deleted_at.is_(None),
)
.first()
)
if appeared is not None:
raise AddressBookError(
"A matching vCard UID appeared after preview; preview the batch again."
)
contact = create_contact(
session, principal, run.address_book_id, contact_payload
)
else:
contact_id = str(item.get("contact_id") or "")
if not contact_id:
raise AddressBookError(
"The reviewed vCard update has no stable target contact."
)
current = get_visible_contact(session, principal, contact_id)
if _contact_hash(current) != item.get("expected_contact_hash"):
raise AddressBookError(
f'Contact "{current.display_name}" changed after preview; preview the batch again.'
)
contact = update_contact(
session,
principal,
current.id,
ContactUpdateRequest.model_validate(item["payload"]),
)
contact.source_kind = "vcard"
contact.source_ref = (
item.get("source_ref")
or f"vcard-sha256:{str(item['source_identity']).split(':', 1)[-1]}"
)
contact.source_payload_kind = "vcard"
contact.source_payload_raw = item["raw"]
contact.source_revision = item.get("source_revision")
provenance = dict(contact.provenance or {})
provenance["vcard_batch"] = {
"run_id": run.id,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"parser_version": VCARD_PARSER_VERSION,
"source_filename": item["source_filename"],
"card_index": item["card_index"],
}
contact.provenance = provenance
session.flush()
return contact
def _changed_fields(contact: Contact, payload: dict[str, Any]) -> list[str]:
current = _contact_projection(contact)
incoming = _payload_projection(payload)
return sorted(key for key in incoming if current.get(key) != incoming.get(key))
def _contact_hash(contact: Contact) -> str:
return _hash_json(_contact_projection(contact))
def _contact_projection(contact: Contact) -> dict[str, Any]:
return {
"display_name": contact.display_name,
"given_name": contact.given_name,
"family_name": contact.family_name,
"organization": contact.organization,
"role_title": contact.role_title,
"note": contact.note,
"tags": list(contact.tags or []),
"emails": [
{"label": item.label, "email": item.email, "is_primary": item.is_primary}
for item in contact.emails
],
"phones": [
{"label": item.label, "phone": item.phone, "is_primary": item.is_primary}
for item in contact.phones
],
"postal_addresses": [
{
"label": item.label,
"street": item.street,
"postal_code": item.postal_code,
"locality": item.locality,
"region": item.region,
"country": item.country,
"is_primary": item.is_primary,
}
for item in contact.postal_addresses
],
}
def _payload_projection(payload: dict[str, Any]) -> dict[str, Any]:
return {
key: payload.get(key)
for key in (
"display_name",
"given_name",
"family_name",
"organization",
"role_title",
"note",
"tags",
"emails",
"phones",
"postal_addresses",
)
}
def _selection_map(payload: VCardBatchCommitRequest) -> dict[str, str]:
result: dict[str, str] = {}
for selection in payload.selections:
if selection.source_key in result:
raise AddressBookError("Each vCard may be selected only once.")
result[selection.source_key] = selection.action
return result
def _public_plan_item(item: dict[str, Any]) -> dict[str, Any]:
return {
key: item.get(key)
for key in (
"source_key",
"source_filename",
"card_index",
"action",
"allowed_actions",
"contact_id",
"display_name",
"changed_fields",
"duplicate_suggestions",
"message",
)
}
def _export_contacts(
session: Session,
principal: ApiPrincipal,
address_book_id: str,
payload: VCardExportRequest,
) -> list[Contact]:
if payload.scope == "address_book":
return _loaded_contacts(session, address_book_id=address_book_id)
if payload.scope == "contacts":
contacts = [
get_visible_contact(session, principal, contact_id)
for contact_id in payload.contact_ids
]
if any(contact.address_book_id != address_book_id for contact in contacts):
raise AddressBookError(
"Every selected contact must belong to the exported address book."
)
return contacts
address_list = get_visible_address_list(
session, principal, str(payload.address_list_id)
)
if address_list.address_book_id != address_book_id:
raise AddressBookError(
"The selected address list does not belong to the exported address book."
)
contact_ids = [
item.contact_id
for item in (
session.query(AddressListEntry)
.filter(AddressListEntry.address_list_id == address_list.id)
.order_by(AddressListEntry.order_index.asc(), AddressListEntry.id.asc())
.all()
)
]
if not contact_ids:
return []
return _loaded_contacts(
session, address_book_id=address_book_id, contact_ids=set(contact_ids)
)
def _loaded_contacts(
session: Session,
*,
address_book_id: str,
contact_ids: set[str] | None = None,
) -> list[Contact]:
query = (
session.query(Contact)
.options(
selectinload(Contact.emails),
selectinload(Contact.phones),
selectinload(Contact.postal_addresses),
)
.filter(
Contact.address_book_id == address_book_id, Contact.deleted_at.is_(None)
)
)
if contact_ids is not None:
query = query.filter(Contact.id.in_(contact_ids))
return query.all()
def _visible_books(session: Session, principal: ApiPrincipal):
from govoplan_addresses.backend.service import list_address_books
return list_address_books(session, principal)
def _execution_mode(card_count: int) -> str:
raw = os.getenv(
"GOVOPLAN_ADDRESSES_VCARD_JOB_THRESHOLD", str(DEFAULT_PERSISTED_BATCH_THRESHOLD)
)
try:
threshold = max(1, min(MAX_VCARD_CARDS, int(raw)))
except ValueError:
threshold = DEFAULT_PERSISTED_BATCH_THRESHOLD
return "persisted_batch" if card_count >= threshold else "bounded_sync"
def _progress(total: int) -> dict[str, int]:
return {
"total": total,
"completed": 0,
"created": 0,
"updated": 0,
"ignored": 0,
"failed": 0,
}
def _hash_json(value: Any) -> str:
return hashlib.sha256(
json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
).hexdigest()
def _safe_filename(value: str) -> str:
safe = "".join(
character if character.isalnum() or character in {"-", "_"} else "-"
for character in value.strip()
)
return safe.strip("-")[:120] or "contacts"
__all__ = [
"apply_vcard_batch",
"cancel_vcard_batch",
"export_vcards",
"get_vcard_batch_run",
"preview_vcard_batch",
"vcard_batch_payload",
"vcard_diagnostics_payload",
]
+588
View File
@@ -0,0 +1,588 @@
from __future__ import annotations
import unittest
from datetime import datetime, timezone
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_access.backend.db.models import Account, User
from govoplan_addresses.backend.db.models import (
AddressBook,
AddressImportRun,
AddressList,
AddressListEntry,
AddressSyncConflict,
AddressSyncSource,
AddressSyncTombstone,
Contact,
ContactChannelRule,
ContactEmail,
ContactFieldProvenance,
ContactMergeRecord,
ContactPhone,
ContactPointQualityDecision,
ContactPostalAddress,
ContactRedirect,
)
from govoplan_addresses.backend.dsar_provider import (
ADDRESSES_DSAR_CAPABILITY,
AddressesDsarProvider,
)
from govoplan_addresses.backend.manifest import manifest
from govoplan_core.core.dsar import (
DsarErasureActionRef,
DsarProvider,
DsarSubjectRef,
)
from govoplan_core.db.base import Base
from govoplan_core.privacy.dsar_workflow import (
create_data_subject_request,
search_data_subject_request,
)
class _Registry:
def __init__(
self,
provider: AddressesDsarProvider,
*,
addresses_active: bool = True,
) -> None:
self.provider = provider
self.addresses_active = addresses_active
def capability_names(self):
return (ADDRESSES_DSAR_CAPABILITY,)
def capability_owner(self, name):
self._assert_capability(name)
return "addresses"
def tenant_entitlement_resolver(self):
addresses_active = self.addresses_active
class _Resolver:
@staticmethod
def resolve(session, tenant_id):
del session, tenant_id
return type(
"State",
(),
{"effective_modules": (("addresses",) if addresses_active else ())},
)()
return _Resolver()
def require_tenant_capability(self, name, session, **kwargs):
del session, kwargs
self._assert_capability(name)
return self.provider
def manifests(self):
return (type("Manifest", (), {"id": "addresses"})(),)
@staticmethod
def _assert_capability(name: str) -> None:
if name != ADDRESSES_DSAR_CAPABILITY:
raise KeyError(name)
class AddressesDsarProviderTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite:///:memory:", future=True)
Base.metadata.create_all(bind=self.engine)
self.session = sessionmaker(bind=self.engine, future=True)()
now = datetime.now(timezone.utc)
self.account = Account(
id="account-1",
email="subject@example.test",
normalized_email="subject@example.test",
display_name="Subject",
password_hash="password-secret-do-not-export",
)
self.user = User(
id="membership-1",
tenant_id="tenant-1",
account_id=self.account.id,
email=self.account.email,
display_name="Subject",
)
self.book = AddressBook(
id="book-1",
tenant_id="tenant-1",
scope_type="tenant",
scope_id="tenant-1",
name="Residents",
created_by_account_id=self.account.id,
metadata_={"secret": "book-metadata-do-not-export"},
)
self.contact = Contact(
id="contact-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
display_name="Subject Person",
given_name="Subject",
family_name="Person",
organization="Example household",
note="A bounded subject note",
tags=["resident"],
source_kind="carddav",
source_ref="https://source.invalid/private/contact.vcf",
source_payload_kind="vcard",
source_payload_raw="raw-source-payload-do-not-export",
source_revision="revision-7",
provenance={"secret": "contact-provenance-do-not-export"},
created_by_account_id=self.account.id,
metadata_={"secret": "contact-metadata-do-not-export"},
)
self.email = ContactEmail(
id="email-1",
contact_id=self.contact.id,
label="private",
email="Subject@Example.test",
original_email="Subject@Example.test",
normalized_email="subject@example.test",
provenance={"secret": "email-provenance-do-not-export"},
is_primary=True,
)
self.phone = ContactPhone(
id="phone-1",
contact_id=self.contact.id,
label="mobile",
phone="+49 30 123456",
original_phone="030 123456",
normalized_phone="+4930123456",
provenance={"secret": "phone-provenance-do-not-export"},
is_primary=True,
)
self.postal = ContactPostalAddress(
id="postal-1",
contact_id=self.contact.id,
label="home",
street="Example Street 1",
postal_code="10115",
locality="Berlin",
country="DE",
original_value={"secret": "postal-original-do-not-export"},
normalized_value={"secret": "postal-normalized-do-not-export"},
provenance={"secret": "postal-provenance-do-not-export"},
is_primary=True,
)
self.address_list = AddressList(
id="list-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
name="District residents",
created_by_account_id="another-account",
)
self.list_entry = AddressListEntry(
id="entry-1",
address_list_id=self.address_list.id,
contact_id=self.contact.id,
contact_email_id=self.email.id,
target_kind="email",
metadata_={"secret": "list-entry-metadata-do-not-export"},
)
self.channel_rule = ContactChannelRule(
id="rule-1",
tenant_id="tenant-1",
contact_id=self.contact.id,
channel="email",
purpose="resident-notice",
contact_point_id=self.email.id,
decision="allow",
legal_basis="public task",
evidence_ref="records://consent/evidence-1",
reason="Current resident preference",
effective_from=now,
created_by_account_id="another-account",
metadata_={"secret": "rule-metadata-do-not-export"},
)
self.quality = ContactPointQualityDecision(
id="quality-1",
tenant_id="tenant-1",
contact_id=self.contact.id,
channel="email",
contact_point_id=self.email.id,
state="valid",
reason_code="verified",
reason="Verified by operator",
evidence_ref="files://private/evidence",
effective_from=now,
created_by_account_id="another-account",
metadata_={"secret": "quality-metadata-do-not-export"},
)
self.provenance = ContactFieldProvenance(
id="provenance-1",
tenant_id="tenant-1",
contact_id=self.contact.id,
field_path="emails[0].email",
value={"secret": "field-value-do-not-export"},
source_kind="carddav",
source_ref="https://source.invalid/private",
source_revision="revision-7",
precedence=10,
selected=True,
reason_code="source_authority",
explanation="Selected from the authoritative source",
visibility="operator",
created_by_account_id="another-account",
metadata_={"secret": "field-metadata-do-not-export"},
)
self.sync_source = AddressSyncSource(
id="source-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
connector_type="carddav",
display_name="Residents CardDAV",
external_account_ref="private-account-ref-do-not-export",
external_address_book_ref="private-book-ref-do-not-export",
sync_token="sync-token-do-not-export",
etag="private-etag-do-not-export",
remote_revision="private-remote-revision-do-not-export",
last_diagnostic={"secret": "diagnostic-do-not-export"},
created_by_account_id=self.account.id,
metadata_={"secret": "source-metadata-do-not-export"},
)
self.tombstone = AddressSyncTombstone(
id="tombstone-1",
tenant_id="tenant-1",
sync_source_id=self.sync_source.id,
address_book_id=self.book.id,
contact_id=self.contact.id,
remote_uid="private-uid-do-not-export",
resource_href="private-href-do-not-export",
synced_at=now,
metadata_={"secret": "tombstone-metadata-do-not-export"},
)
self.conflict = AddressSyncConflict(
id="conflict-1",
tenant_id="tenant-1",
sync_source_id=self.sync_source.id,
address_book_id=self.book.id,
contact_id=self.contact.id,
remote_uid="private-conflict-uid-do-not-export",
resource_href="private-conflict-href-do-not-export",
field_path="family_name",
local_value={"secret": "local-value-do-not-export"},
remote_value={"secret": "remote-value-do-not-export"},
status="resolved",
resolution="local",
resolved_at=now,
resolved_by_account_id=self.account.id,
metadata_={"secret": "conflict-metadata-do-not-export"},
)
self.import_run = AddressImportRun(
id="import-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
source_filename="contacts.csv",
source_format="csv",
input_hash="a" * 64,
plan_hash="b" * 64,
status="applied",
row_count=1,
statistics={"secret": "statistics-do-not-export"},
diagnostics=[{"secret": "import-diagnostic-do-not-export"}],
plan_data=[{"secret": "import-plan-do-not-export"}],
result_evidence={"secret": "import-result-do-not-export"},
created_by_account_id=self.account.id,
applied_at=now,
)
self.merge = ContactMergeRecord(
id="merge-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
winner_contact_id=self.contact.id,
loser_contact_ids=["old-contact-1"],
status="active",
reason="Duplicate contact",
survivorship={"secret": "survivorship-do-not-export"},
decisions=[{"secret": "merge-decisions-do-not-export"}],
before_payload={"secret": "merge-before-do-not-export"},
after_payload={"secret": "merge-after-do-not-export"},
before_hash="c" * 64,
after_hash="d" * 64,
created_by_account_id="another-account",
provenance={"secret": "merge-provenance-do-not-export"},
)
self.redirect = ContactRedirect(
id="redirect-1",
tenant_id="tenant-1",
source_contact_id="old-contact-1",
target_contact_id=self.contact.id,
merge_record_id=self.merge.id,
)
self.unrelated = Contact(
id="contact-unrelated",
tenant_id="tenant-1",
address_book_id=self.book.id,
display_name="Unrelated Person",
note="unrelated-person-do-not-export",
)
unrelated_email = ContactEmail(
id="email-unrelated",
contact_id=self.unrelated.id,
email="unrelated@example.test",
original_email="unrelated@example.test",
normalized_email="unrelated@example.test",
)
tenant_two_book = AddressBook(
id="book-tenant-2",
tenant_id="tenant-2",
scope_type="tenant",
scope_id="tenant-2",
name="Other tenant",
)
tenant_two_contact = Contact(
id="contact-tenant-2",
tenant_id="tenant-2",
address_book_id=tenant_two_book.id,
display_name="Other Tenant Subject",
note="other-tenant-do-not-export",
)
tenant_two_email = ContactEmail(
id="email-tenant-2",
contact_id=tenant_two_contact.id,
email="subject@example.test",
original_email="subject@example.test",
normalized_email="subject@example.test",
)
self.session.add_all(
[
self.account,
self.user,
self.book,
self.contact,
self.email,
self.phone,
self.postal,
self.address_list,
self.list_entry,
self.channel_rule,
self.quality,
self.provenance,
self.sync_source,
self.tombstone,
self.conflict,
self.import_run,
self.merge,
self.redirect,
self.unrelated,
unrelated_email,
tenant_two_book,
tenant_two_contact,
tenant_two_email,
]
)
self.session.commit()
self.provider = AddressesDsarProvider()
self.subject = DsarSubjectRef(
account_id=self.account.id,
email=self.account.email,
)
def tearDown(self) -> None:
self.session.close()
self.engine.dispose()
def test_manifest_publishes_protocol_conforming_provider(self) -> None:
provided_names = {item.name for item in manifest.provides_interfaces}
self.assertIn(ADDRESSES_DSAR_CAPABILITY, provided_names)
provider = manifest.capability_factories[ADDRESSES_DSAR_CAPABILITY](None)
self.assertIsInstance(provider, DsarProvider)
def test_search_is_tenant_scoped_related_and_minimized(self) -> None:
records = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=self.subject,
)
resource_types = {record.resource_type for record in records}
self.assertTrue(
{
"addresses_contact",
"addresses_contact_email",
"addresses_contact_phone",
"addresses_contact_postal_address",
"addresses_list_membership",
"addresses_channel_rule",
"addresses_quality_decision",
"addresses_field_provenance",
"addresses_merge_record",
"addresses_contact_redirect",
"addresses_sync_tombstone",
"addresses_sync_conflict",
"addresses_address_book_attribution",
"addresses_sync_source_attribution",
"addresses_import_run_attribution",
}.issubset(resource_types)
)
serialized = repr([record.to_dict() for record in records])
excluded_values = (
"password-secret-do-not-export",
"raw-source-payload-do-not-export",
"contact-provenance-do-not-export",
"book-metadata-do-not-export",
"field-value-do-not-export",
"sync-token-do-not-export",
"private-account-ref-do-not-export",
"private-remote-revision-do-not-export",
"local-value-do-not-export",
"remote-value-do-not-export",
"import-plan-do-not-export",
"merge-before-do-not-export",
"merge-after-do-not-export",
"unrelated-person-do-not-export",
"other-tenant-do-not-export",
)
for value in excluded_values:
self.assertNotIn(value, serialized)
def test_conflicting_selectors_and_uncorroborated_reference_fail_closed(
self,
) -> None:
conflict = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=DsarSubjectRef(
email="subject@example.test",
external_references={"addresses.email": "other@example.test"},
),
)
uncorroborated = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=DsarSubjectRef(
email="subject@example.test",
external_references={"addresses.contact": self.unrelated.id},
),
)
self.assertEqual((), conflict)
self.assertEqual((), uncorroborated)
def test_plan_retains_evidence_and_routes_contact_data_to_review(self) -> None:
records = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=self.subject,
)
actions = self.provider.plan_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
records=records,
)
kinds = {action.kind for action in actions}
self.assertEqual({"manual_review", "retain"}, kinds)
self.assertFalse(any(action.executable for action in actions))
retained = [action for action in actions if action.kind == "retain"]
self.assertTrue(retained)
self.assertTrue(all(action.rationale for action in retained))
results = self.provider.execute_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
actions=actions,
request_id="dsar-addresses-1",
)
self.assertEqual({"blocked"}, {result.status for result in results})
self.assertIsNotNone(self.session.get(Contact, self.contact.id))
def test_execution_rejects_foreign_or_forged_executable_actions(self) -> None:
foreign = DsarErasureActionRef(
action_id="mail:delete:contact:contact-1",
provider_id="mail",
module_id="mail",
kind="delete",
resource_type="addresses_contact",
resource_id=self.contact.id,
title="Foreign delete",
rationale="Must be rejected",
executable=True,
)
forged = DsarErasureActionRef(
action_id="addresses:delete:addresses_contact:contact-1",
provider_id="addresses",
module_id="addresses",
kind="delete",
resource_type="addresses_contact",
resource_id=self.contact.id,
title="Forged delete",
rationale="Must be rejected",
executable=True,
)
with self.assertRaises(ValueError):
self.provider.execute_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
actions=(foreign,),
request_id="dsar-addresses-2",
)
with self.assertRaises(ValueError):
self.provider.execute_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
actions=(forged,),
request_id="dsar-addresses-2",
)
def test_core_workflow_discovers_active_and_inactive_provider(self) -> None:
request = create_data_subject_request(
self.session,
tenant_id="tenant-1",
reference="DSAR-ADDRESSES-1",
request_kind="access",
subject=self.subject,
purpose="Respond to an authorized privacy request.",
legal_basis="Article 15 GDPR",
due_at=None,
requested_by_account_id="privacy-officer",
)
self.session.commit()
search_data_subject_request(
self.session,
registry=_Registry(self.provider),
row=request,
expected_revision=1,
)
self.assertEqual("searched", request.status)
self.assertEqual(["addresses"], request.coverage["covered_modules"])
self.assertEqual([], request.coverage["modules_without_provider"])
disabled = create_data_subject_request(
self.session,
tenant_id="tenant-1",
reference="DSAR-ADDRESSES-DISABLED",
request_kind="access",
subject=self.subject,
purpose="Verify disabled-module coverage.",
legal_basis="Article 15 GDPR",
due_at=None,
requested_by_account_id="privacy-officer",
)
search_data_subject_request(
self.session,
registry=_Registry(self.provider, addresses_active=False),
row=disabled,
expected_revision=1,
)
self.assertEqual(0, disabled.search_result["record_count"])
self.assertEqual(
[ADDRESSES_DSAR_CAPABILITY],
disabled.coverage["inactive_provider_capabilities"],
)
if __name__ == "__main__":
unittest.main()
+16 -1
View File
@@ -6,6 +6,18 @@ from govoplan_addresses.backend.manifest import manifest
class AddressesInterfaceDocumentationContractTests(unittest.TestCase): class AddressesInterfaceDocumentationContractTests(unittest.TestCase):
def test_all_static_topics_have_complete_german_content(self) -> None:
for topic in manifest.documentation:
german = (topic.translations or {}).get("de", {})
self.assertEqual(
{"title", "summary", "body"},
set(german),
topic.id,
)
self.assertTrue(
all(str(value).strip() for value in german.values()), topic.id
)
def test_route_and_surfaces_remain_declared(self) -> None: def test_route_and_surfaces_remain_declared(self) -> None:
frontend = manifest.frontend frontend = manifest.frontend
self.assertIsNotNone(frontend) self.assertIsNotNone(frontend)
@@ -29,7 +41,10 @@ class AddressesInterfaceDocumentationContractTests(unittest.TestCase):
reference = topics["addresses.reference.fields-and-consequences"] reference = topics["addresses.reference.fields-and-consequences"]
self.assertIn("addresses.state.read-only", boundary.metadata["help_contexts"]) self.assertIn("addresses.state.read-only", boundary.metadata["help_contexts"])
self.assertIn("addresses.field.communication-purpose", governance.metadata["help_contexts"]) self.assertIn(
"addresses.field.communication-purpose",
governance.metadata["help_contexts"],
)
self.assertIn("addresses.action.sync", reference.metadata["help_contexts"]) self.assertIn("addresses.action.sync", reference.metadata["help_contexts"])
self.assertIn("merge", reference.metadata["consequence_classes"]) self.assertIn("merge", reference.metadata["consequence_classes"])
self.assertIn("governance_fact", reference.metadata["consequence_classes"]) self.assertIn("governance_fact", reference.metadata["consequence_classes"])
+208
View File
@@ -0,0 +1,208 @@
from __future__ import annotations
import base64
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_addresses.backend.db.models import AddressBook, Contact
from govoplan_addresses.backend.import_schemas import (
AddressImportConfiguration,
AddressImportPreviewRequest,
AddressImportProfileCreateRequest,
)
from govoplan_addresses.backend.imports import (
apply_address_import,
create_import_profile,
import_run_payload,
preview_address_import,
)
from govoplan_addresses.backend.ldif import parse_ldif_rows
from govoplan_core.db.base import Base
class Principal:
account_id = "account-1"
group_ids = frozenset()
@property
def tenant_id(self) -> str:
return "tenant-1"
def has(self, scope: str) -> bool:
return scope in {
"addresses:address_book:read",
"addresses:address_book:write",
"addresses:contact:read",
"addresses:contact:write",
}
def encoded(value: bytes | str) -> str:
raw = value.encode() if isinstance(value, str) else value
return base64.b64encode(raw).decode()
class AddressLdifImportTests(unittest.TestCase):
def setUp(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
Base.metadata.create_all(engine)
self.session = sessionmaker(bind=engine, expire_on_commit=False)()
self.principal = Principal()
self.book = AddressBook(
tenant_id="tenant-1",
scope_type="tenant",
scope_id="tenant-1",
name="Imported contacts",
source_kind="local",
read_only=False,
)
self.session.add(self.book)
def create_profile(self, **configuration_overrides):
configuration = AddressImportConfiguration(
field_mappings={
"source_key": "dn",
"display_name": "cn",
"given_name": "givenName",
"family_name": "sn",
"email": "mail",
"phone": "telephoneNumber",
"organization": "o",
},
**configuration_overrides,
)
profile = create_import_profile(
self.session,
self.principal,
AddressImportProfileCreateRequest(
scope_type="tenant",
name="Directory export",
source_format="ldif",
configuration=configuration,
),
)
self.session.flush()
return profile
def test_parser_unfolds_and_decodes_text_without_projecting_binary_or_urls(self) -> None:
rows, diagnostics = parse_ldif_rows(
b"version: 1\n\n"
b"# exported contact\n"
b"dn: uid=ada,ou=people,dc=example,dc=test\n"
b"cn:: QWRhIExvdmVsYWNl\n"
b"sn: Love\n"
b" lace\n"
b"mail: ada@example.test\n"
b"mail: ada.work@example.test\n"
b"jpegPhoto:: /9j/\n"
b"seeAlso:< https://example.test/contact/ada\n",
max_entries=10,
)
self.assertEqual(1, len(rows))
self.assertEqual(["Ada Lovelace"], rows[0][1]["cn"])
self.assertEqual(["Lovelace"], rows[0][1]["sn"])
self.assertEqual(["ada@example.test", "ada.work@example.test"], rows[0][1]["mail"])
self.assertNotIn("jpegphoto", rows[0][1])
self.assertNotIn("seealso", rows[0][1])
self.assertEqual(
{"ldif_binary_value_ignored", "ldif_url_value_ignored"},
{item["code"] for item in diagnostics},
)
def test_preview_apply_and_repeat_preserve_multivalue_provenance(self) -> None:
profile = self.create_profile(default_tags=["ldif"])
source = (
"dn: uid=ada,ou=people,dc=example,dc=test\n"
"cn: Ada Lovelace\n"
"givenName: Ada\n"
"sn: Lovelace\n"
"mail: ada@example.test\n"
"mail: ada.work@example.test\n"
"telephoneNumber: +49 30 123\n"
"o: Analysis Office\n"
)
request = AddressImportPreviewRequest(
profile_id=profile.id,
filename="contacts.ldif",
content_base64=encoded(source),
)
run = preview_address_import(self.session, self.principal, self.book.id, request)
self.assertEqual(1, run.statistics["create"])
self.assertFalse([item for item in run.diagnostics if item["severity"] == "error"])
apply_address_import(self.session, self.principal, run.id, expected_plan_hash=run.plan_hash)
contact = self.session.query(Contact).one()
self.assertEqual(["ada@example.test", "ada.work@example.test"], [item.email for item in contact.emails])
self.assertEqual("ldif", contact.source_kind)
self.assertEqual("ldif", contact.provenance["import"]["source_format"])
self.assertEqual(64, len(contact.provenance["import"]["source_record_hash"]))
self.assertNotIn(source, repr(import_run_payload(run)))
repeated = preview_address_import(self.session, self.principal, self.book.id, request)
self.assertEqual(1, repeated.statistics["unchanged"])
apply_address_import(self.session, self.principal, repeated.id, expected_plan_hash=repeated.plan_hash)
self.assertEqual(1, self.session.query(Contact).count())
def test_change_records_are_rejected_by_default_and_add_requires_explicit_policy(self) -> None:
source = (
"dn: uid=ada,ou=people,dc=example,dc=test\n"
"changetype: add\n"
"cn: Ada Lovelace\n"
"mail: ada@example.test\n"
)
rejected_profile = self.create_profile()
rejected = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=rejected_profile.id,
filename="changes.ldif",
content_base64=encoded(source),
),
)
self.assertEqual(0, rejected.row_count)
self.assertIn("ldif_change_record_rejected", {item["code"] for item in rejected.diagnostics})
self.assertFalse(import_run_payload(rejected)["can_apply"])
allowed_profile = self.create_profile(ldif_change_record_policy="treat_add_as_entry")
allowed = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=allowed_profile.id,
filename="changes.ldif",
content_base64=encoded(source),
),
)
self.assertEqual(1, allowed.statistics["create"])
self.assertIn("ldif_add_record_imported", {item["code"] for item in allowed.diagnostics})
def test_invalid_base64_is_a_correction_diagnostic_and_blocks_apply(self) -> None:
profile = self.create_profile()
run = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=profile.id,
filename="broken.ldif",
content_base64=encoded(
"dn: uid=ada,dc=example,dc=test\n"
"cn:: this-is-not-base64!\n"
"mail: ada@example.test\n"
),
),
)
self.assertIn("ldif_invalid_base64", {item["code"] for item in run.diagnostics})
self.assertFalse(import_run_payload(run)["can_apply"])
with self.assertRaisesRegex(ValueError, "error diagnostics"):
apply_address_import(self.session, self.principal, run.id, expected_plan_hash=run.plan_hash)
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -35,7 +35,7 @@ class AddressesMigrationTests(unittest.TestCase):
try: try:
with engine.connect() as connection: with engine.connect() as connection:
self.assertIn( self.assertIn(
"c5d7e8f9a0b1", "d6e8f9a0b1c2",
set(MigrationContext.configure(connection).get_current_heads()), set(MigrationContext.configure(connection).get_current_heads()),
) )
tables = set(inspect(connection).get_table_names()) tables = set(inspect(connection).get_table_names())
+381 -3
View File
@@ -3,12 +3,13 @@ from __future__ import annotations
import base64 import base64
from io import BytesIO from io import BytesIO
import unittest import unittest
from unittest.mock import patch
from sqlalchemy import create_engine from sqlalchemy import create_engine, event
from sqlalchemy.orm import sessionmaker from sqlalchemy.orm import sessionmaker
from openpyxl import Workbook from openpyxl import Workbook
from govoplan_addresses.backend.db.models import AddressBook, Contact from govoplan_addresses.backend.db.models import AddressBook, AddressList, AddressListEntry, Contact, ContactEmail, ContactPhone, ContactPostalAddress
from govoplan_addresses.backend.import_schemas import ( from govoplan_addresses.backend.import_schemas import (
AddressImportConfiguration, AddressImportConfiguration,
AddressImportPreviewRequest, AddressImportPreviewRequest,
@@ -18,13 +19,17 @@ from govoplan_addresses.backend.import_schemas import (
) )
from govoplan_addresses.backend.imports import ( from govoplan_addresses.backend.imports import (
apply_address_import, apply_address_import,
_contact_hash,
_contact_snapshot,
create_import_profile, create_import_profile,
get_import_run,
import_run_payload, import_run_payload,
preview_address_import, preview_address_import,
rollback_address_import, rollback_address_import,
update_import_profile, update_import_profile,
) )
from govoplan_core.db.base import Base from govoplan_core.db.base import Base
from govoplan_addresses.backend.service import delete_contact
class Principal: class Principal:
@@ -44,6 +49,12 @@ class Principal:
} }
class OtherTenantPrincipal(Principal):
@property
def tenant_id(self) -> str:
return "tenant-2"
def encoded(value: str) -> str: def encoded(value: str) -> str:
return base64.b64encode(value.encode()).decode() return base64.b64encode(value.encode()).decode()
@@ -84,6 +95,315 @@ class AddressTabularImportTests(unittest.TestCase):
) )
self.session.flush() self.session.flush()
def _preview_rows(self, count: int = 1, *, organization: str = "Office"):
return preview_address_import(
self.session, self.principal, self.book.id,
AddressImportPreviewRequest(
profile_id=self.profile.id, filename="fixture.csv",
content_base64=encoded(
"id;first;last;email;organization\n"
+ "".join(f"{index};Given;Family;u{index}@example.test;{organization}\n" for index in range(count))
),
),
)
def _apply(self, run):
apply_address_import(self.session, self.principal, run.id, expected_plan_hash=run.plan_hash)
self.session.commit()
def test_rollback_restores_previously_deleted_state_and_source_fields(self) -> None:
self._apply(self._preview_rows())
contact = self.session.query(Contact).one()
delete_contact(self.session, self.principal, contact.id)
self.session.commit()
prior_deleted_at = contact.deleted_at
prior_source_revision = contact.source_revision
run = self._preview_rows(organization="Changed")
self._apply(run)
self.assertIsNone(contact.deleted_at)
before = run.result_evidence["updated_contacts"][0]["before"]
self.assertEqual(2, before["version"])
self.assertEqual(prior_deleted_at.isoformat(), before["deleted_at"])
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Restore the reviewed previous state."),
)
self.session.commit()
self.assertEqual("rolled_back", run.status)
self.assertEqual(prior_deleted_at, contact.deleted_at)
self.assertEqual("Office", contact.organization)
self.assertEqual(prior_source_revision, contact.source_revision)
def test_rollback_restores_all_point_evidence_and_identities_after_reload(self) -> None:
self._apply(self._preview_rows())
contact = self.session.query(Contact).one()
contact.emails[0].original_email = " U0@EXAMPLE.TEST "
contact.emails[0].provenance = {"nested": {"original": "email evidence"}}
contact.note = " Exact retained note\r\n"
contact.tags = ["Exact", "Exact", " padded "]
contact.metadata_ = None
contact.phones.append(ContactPhone(
phone="+49 123", original_phone=" +49 (123) ", normalized_phone="+49123",
provenance={"original": "phone evidence"}, label="Office", is_primary=True, order_index=4,
))
contact.postal_addresses.append(ContactPostalAddress(
street="Main Street", original_value={"street": " Main Street "},
normalized_value={"street": "main street"}, provenance={"original": "postal evidence"},
is_primary=True, order_index=7,
))
self.session.commit()
before = _contact_snapshot(contact)["points"]
run = self._preview_rows(organization="Changed")
self._apply(run)
self.assertEqual(before["emails"], _contact_snapshot(contact)["points"]["emails"])
self.session.expire_all()
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Restore all original point evidence."),
)
self.session.commit()
self.session.expire_all()
self.assertEqual(before, _contact_snapshot(contact)["points"])
self.assertEqual(" Exact retained note\r\n", contact.note)
self.assertEqual(["Exact", "Exact", " padded "], contact.tags)
self.assertIsNone(contact.metadata_)
def test_post_import_point_provenance_edit_is_guarded(self) -> None:
self._apply(self._preview_rows())
run = self._preview_rows(organization="Changed")
self._apply(run)
contact = self.session.query(Contact).one()
after_hash = _contact_hash(contact)
contact.emails[0].provenance = {"later": "manual evidence"}
self.session.commit()
self.assertNotEqual(after_hash, _contact_hash(contact))
with self.assertRaisesRegex(ValueError, "changed after import"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Do not erase a later point edit."),
)
self.assertEqual({"later": "manual evidence"}, contact.emails[0].provenance)
def test_point_lock_order_does_not_change_tied_collection_order_or_reviewed_hash(self) -> None:
self._apply(self._preview_rows())
contact = self.session.query(Contact).one()
contact.emails.append(ContactEmail(
id="00000000-0000-0000-0000-000000000000", email="extra@example.test",
original_email="extra@example.test", normalized_email="extra@example.test",
label="Extra", is_primary=False, order_index=0,
))
self.session.commit()
self.session.expire_all()
before = _contact_snapshot(contact)["points"]
run = self._preview_rows(organization="Changed")
self._apply(run)
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Preserve tied contact-point ordering."),
)
self.session.commit()
self.session.expire_all()
self.assertEqual(before, _contact_snapshot(contact)["points"])
def _link_email(self, contact):
address_list = AddressList(address_book_id=self.book.id, tenant_id="tenant-1", name="Recipients")
entry = AddressListEntry(address_list=address_list, contact=contact, contact_email=contact.emails[0], target_kind="email")
self.session.add(entry)
self.session.commit()
return entry
def test_unchanged_point_keeps_address_list_identity_through_apply_and_rollback(self) -> None:
self._apply(self._preview_rows())
contact = self.session.query(Contact).one()
entry = self._link_email(contact)
point_id = contact.emails[0].id
run = self._preview_rows(organization="Changed")
self._apply(run)
self.session.expire_all()
self.assertEqual(point_id, entry.contact_email_id)
self.assertEqual(point_id, contact.emails[0].id)
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Keep the explicit recipient reference."),
)
self.session.commit()
self.session.expire_all()
self.assertEqual(point_id, entry.contact_email_id)
self.assertEqual(point_id, contact.emails[0].id)
def test_replacing_a_linked_point_requires_reconciliation_before_contact_mutation(self) -> None:
self._apply(self._preview_rows())
contact = self.session.query(Contact).one()
entry = self._link_email(contact)
run = preview_address_import(
self.session, self.principal, self.book.id,
AddressImportPreviewRequest(profile_id=self.profile.id, filename="fixture.csv", content_base64=encoded(
"id;first;last;email;organization\n0;Given;Family;different@example.test;Changed\n"
)),
)
before = _contact_hash(contact)
with self.assertRaisesRegex(ValueError, "address-list or governance references"):
apply_address_import(self.session, self.principal, run.id, expected_plan_hash=run.plan_hash)
self.assertEqual(before, _contact_hash(contact))
self.assertEqual(contact.emails[0].id, entry.contact_email_id)
self.assertEqual("previewed", run.status)
def test_new_reference_to_imported_point_blocks_destructive_rollback(self) -> None:
self._apply(self._preview_rows())
run = preview_address_import(
self.session, self.principal, self.book.id,
AddressImportPreviewRequest(profile_id=self.profile.id, filename="fixture.csv", content_base64=encoded(
"id;first;last;email;organization\n0;Given;Family;different@example.test;Changed\n"
)),
)
self._apply(run)
contact = self.session.query(Contact).one()
entry = self._link_email(contact)
before = _contact_hash(contact)
with self.assertRaisesRegex(ValueError, "address-list or governance references"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Retain the newly referenced recipient point."),
)
self.assertEqual(before, _contact_hash(contact))
self.assertEqual(contact.emails[0].id, entry.contact_email_id)
self.assertEqual("applied", run.status)
def test_version_one_point_incomplete_evidence_is_not_accepted(self) -> None:
import copy
self._apply(self._preview_rows())
run = self._preview_rows(organization="Changed")
self._apply(run)
evidence = copy.deepcopy(run.result_evidence)
evidence["updated_contacts"][0]["before"]["version"] = 1
run.result_evidence = evidence
self.session.commit()
with self.assertRaisesRegex(ValueError, "incomplete legacy rollback evidence"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Do not infer missing point evidence."),
)
def test_legacy_or_incomplete_before_images_fail_before_any_rollback_mutation(self) -> None:
self._apply(self._preview_rows())
run = self._preview_rows(2, organization="Changed")
self._apply(run)
evidence = dict(run.result_evidence)
updates = [dict(item) for item in evidence["updated_contacts"]]
updates[0]["before"] = updates[0]["before"]["contact"]
evidence["updated_contacts"] = updates
run.result_evidence = evidence
self.session.commit()
with self.assertRaisesRegex(ValueError, "incomplete legacy rollback evidence"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Reject an incomplete previous state."),
)
self.assertEqual("applied", run.status)
self.assertEqual(2, self.session.query(Contact).filter(Contact.deleted_at.is_(None)).count())
self.assertTrue(all(contact.organization == "Changed" for contact in self.session.query(Contact)))
def test_preview_queries_are_batched_and_relationships_are_eager(self) -> None:
queries = []
def capture(conn, cursor, statement, parameters, context, executemany):
if statement.lstrip().upper().startswith("SELECT"):
queries.append(statement)
event.listen(self.session.bind, "before_cursor_execute", capture)
try:
first = self._preview_rows(50)
self.assertEqual(50, first.statistics["create"])
self.assertLessEqual(len(queries), 3)
self._apply(first)
self.session.expunge_all()
queries.clear()
repeat = self._preview_rows(50)
self.assertEqual(50, repeat.statistics["unchanged"])
self.assertLessEqual(len(queries), 6)
finally:
event.remove(self.session.bind, "before_cursor_execute", capture)
def test_preview_batches_preserve_first_source_match_and_book_scope(self) -> None:
self._apply(self._preview_rows(5))
original = self.session.query(Contact).order_by(Contact.created_at, Contact.id).first()
other_book = AddressBook(tenant_id="tenant-1", scope_type="tenant", scope_id="tenant-1", name="Other", source_kind="local", read_only=False)
self.session.add(other_book)
self.session.flush()
self.session.add_all([
Contact(tenant_id="tenant-1", address_book_id=original.address_book_id, display_name="Later duplicate", source_ref=original.source_ref),
Contact(tenant_id="tenant-1", address_book_id=other_book.id, display_name="Other book", source_ref=original.source_ref),
])
self.session.commit()
with patch("govoplan_addresses.backend.imports.CONTACT_LOOKUP_BATCH_SIZE", 2):
repeat = self._preview_rows(5)
self.assertEqual(5, repeat.statistics["unchanged"])
def test_missing_created_after_hash_blocks_rollback(self) -> None:
run = self._preview_rows()
self._apply(run)
run.plan_data = [{key: value for key, value in item.items() if key != "after_hash"} for item in run.plan_data]
self.session.commit()
with self.assertRaisesRegex(ValueError, "evidence is incomplete"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Incomplete evidence must not delete contacts."),
)
self.assertEqual(1, self.session.query(Contact).filter(Contact.deleted_at.is_(None)).count())
def test_applied_effect_hashes_survive_commit_and_reload(self) -> None:
run = self._preview_rows()
self._apply(run)
run_id, plan_hash = run.id, run.plan_hash
self.session.expunge_all()
reloaded = get_import_run(self.session, self.principal, run_id)
self.assertTrue(reloaded.plan_data[0]["contact_id"])
self.assertEqual(64, len(reloaded.plan_data[0]["after_hash"]))
rollback_address_import(
self.session, self.principal, run_id,
AddressImportRollbackRequest(expected_plan_hash=plan_hash, reason="Durable after-images guard rollback."),
)
self.session.commit()
self.assertEqual(0, self.session.query(Contact).filter(Contact.deleted_at.is_(None)).count())
def test_replayed_apply_still_requires_the_reviewed_plan_hash(self) -> None:
run = self._preview_rows()
self._apply(run)
with self.assertRaisesRegex(ValueError, "reviewed import plan changed"):
apply_address_import(self.session, self.principal, run.id, expected_plan_hash="0" * 64)
def test_rollback_rejects_a_moved_target_without_archiving_it(self) -> None:
run = self._preview_rows()
self._apply(run)
other_book = AddressBook(tenant_id="tenant-1", scope_type="tenant", scope_id="tenant-1", name="Other", source_kind="local", read_only=False)
self.session.add(other_book)
self.session.flush()
contact = self.session.query(Contact).one()
contact.address_book_id = other_book.id
self.session.commit()
with self.assertRaisesRegex(ValueError, "moved to another address book"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Moved contacts require manual reconciliation."),
)
self.assertIsNone(contact.deleted_at)
self.assertEqual("applied", run.status)
def test_rollback_retains_post_import_edits(self) -> None:
run = self._preview_rows()
self._apply(run)
contact = self.session.query(Contact).one()
contact.organization = "Later manual edit"
self.session.commit()
with self.assertRaisesRegex(ValueError, "changed after import"):
rollback_address_import(
self.session, self.principal, run.id,
AddressImportRollbackRequest(expected_plan_hash=run.plan_hash, reason="Later changes must remain untouched."),
)
self.assertIsNone(contact.deleted_at)
self.assertEqual("Later manual edit", contact.organization)
def test_preview_apply_repeat_and_guarded_rollback(self) -> None: def test_preview_apply_repeat_and_guarded_rollback(self) -> None:
payload = AddressImportPreviewRequest( payload = AddressImportPreviewRequest(
profile_id=self.profile.id, profile_id=self.profile.id,
@@ -121,11 +441,69 @@ class AddressTabularImportTests(unittest.TestCase):
self.session, self.session,
self.principal, self.principal,
run.id, run.id,
AddressImportRollbackRequest(reason="The operator selected the wrong monthly file."), AddressImportRollbackRequest(
expected_plan_hash=run.plan_hash,
reason="The operator selected the wrong monthly file.",
),
) )
self.assertEqual("rolled_back", rolled_back.status) self.assertEqual("rolled_back", rolled_back.status)
self.assertEqual(0, self.session.query(Contact).filter(Contact.deleted_at.is_(None)).count()) self.assertEqual(0, self.session.query(Contact).filter(Contact.deleted_at.is_(None)).count())
def test_rollback_rejects_a_stale_review_hash(self) -> None:
run = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=self.profile.id,
filename="contacts.csv",
content_base64=encoded(
"id;first;last;email;organization\n"
"1;Ada;Lovelace;ada@example.test;Analysis Office\n"
),
),
)
apply_address_import(
self.session,
self.principal,
run.id,
expected_plan_hash=run.plan_hash,
)
with self.assertRaisesRegex(ValueError, "reviewed import plan changed"):
rollback_address_import(
self.session,
self.principal,
run.id,
AddressImportRollbackRequest(
expected_plan_hash="0" * 64,
reason="The operator selected the wrong monthly file.",
),
)
def test_persisted_run_read_is_tenant_bounded_and_source_safe(self) -> None:
source = "id;first;last;email;organization\n1;Ada;Lovelace;ada@example.test;Analysis Office\n"
run = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=self.profile.id,
filename="contacts.csv",
content_base64=encoded(source),
),
)
self.session.flush()
payload = import_run_payload(get_import_run(self.session, self.principal, run.id))
self.assertEqual("previewed", payload["status"])
self.assertEqual(run.plan_hash, payload["plan_hash"])
self.assertNotIn("plan_data", payload)
self.assertNotIn(source, repr(payload))
with self.assertRaisesRegex(ValueError, "not found"):
get_import_run(self.session, OtherTenantPrincipal(), run.id)
def test_duplicate_keys_and_changed_targets_block_apply(self) -> None: def test_duplicate_keys_and_changed_targets_block_apply(self) -> None:
duplicate = preview_address_import( duplicate = preview_address_import(
self.session, self.session,
+270
View File
@@ -0,0 +1,270 @@
from __future__ import annotations
import base64
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_addresses.backend.db.models import (
AddressBook,
AddressList,
AddressListEntry,
Contact,
)
from govoplan_addresses.backend.schemas import ContactCreateRequest
from govoplan_addresses.backend.service import AddressBookError, create_contact
from govoplan_addresses.backend.vcard import (
MAX_VCARD_UNFOLDED_LINE_CHARS,
parse_vcards_with_issues,
)
from govoplan_addresses.backend.vcard_batch_schemas import (
VCardBatchCancelRequest,
VCardBatchCommitRequest,
VCardBatchFilePayload,
VCardBatchPreviewRequest,
VCardBatchSelection,
VCardExportRequest,
)
from govoplan_addresses.backend.vcard_batches import (
apply_vcard_batch,
cancel_vcard_batch,
export_vcards,
preview_vcard_batch,
vcard_batch_payload,
)
from govoplan_core.db.base import Base
class Principal:
account_id = "account-1"
group_ids = frozenset()
@property
def tenant_id(self) -> str:
return "tenant-1"
def has(self, scope: str) -> bool:
return scope in {
"addresses:address_book:read",
"addresses:address_book:write",
"addresses:contact:read",
"addresses:contact:write",
}
def vcard(uid: str, name: str, email: str) -> str:
return (
"BEGIN:VCARD\r\n"
"VERSION:4.0\r\n"
f"UID:{uid}\r\n"
f"FN:{name}\r\n"
f"EMAIL:{email}\r\n"
"END:VCARD\r\n"
)
def batch_file(filename: str, content: str) -> VCardBatchFilePayload:
return VCardBatchFilePayload(
filename=filename,
content_base64=base64.b64encode(content.encode()).decode(),
)
class VCardBatchTests(unittest.TestCase):
def setUp(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
Base.metadata.create_all(engine)
self.session = sessionmaker(bind=engine, expire_on_commit=False)()
self.principal = Principal()
self.book = AddressBook(
tenant_id="tenant-1",
scope_type="tenant",
scope_id="tenant-1",
name="Batch contacts",
source_kind="local",
read_only=False,
)
self.session.add(self.book)
self.session.flush()
def test_multifile_preview_selective_apply_and_repeat_are_idempotent(self) -> None:
run = preview_vcard_batch(
self.session,
self.principal,
self.book.id,
VCardBatchPreviewRequest(
files=[
batch_file(
"ada.vcf", vcard("ada-1", "Ada Lovelace", "ada@example.test")
),
batch_file(
"grace.vcf",
vcard("grace-1", "Grace Hopper", "grace@example.test"),
),
]
),
)
self.assertEqual(0, self.session.query(Contact).count())
self.assertEqual("previewed", run.status)
self.assertEqual(2, run.row_count)
self.assertEqual("govoplan-vcard/2", run.result_evidence["parser_version"])
self.assertNotIn("ada@example.test", repr(vcard_batch_payload(run)))
selections = [
VCardBatchSelection(
source_key=run.plan_data[0]["source_key"], action="create"
),
VCardBatchSelection(
source_key=run.plan_data[1]["source_key"], action="ignore"
),
]
request = VCardBatchCommitRequest(
expected_plan_hash=run.plan_hash, selections=selections
)
applied = apply_vcard_batch(self.session, self.principal, run.id, request)
repeated = apply_vcard_batch(self.session, self.principal, run.id, request)
self.assertIs(applied, repeated)
self.assertEqual(1, self.session.query(Contact).count())
self.assertEqual("Ada Lovelace", self.session.query(Contact).one().display_name)
self.assertEqual(1, applied.result_evidence["progress"]["ignored"])
with self.assertRaisesRegex(AddressBookError, "different selection"):
apply_vcard_batch(
self.session,
self.principal,
run.id,
VCardBatchCommitRequest(
expected_plan_hash=run.plan_hash,
selections=[
VCardBatchSelection(
source_key=run.plan_data[1]["source_key"], action="create"
)
],
),
)
def test_duplicate_uid_policy_and_cancellation(self) -> None:
run = preview_vcard_batch(
self.session,
self.principal,
self.book.id,
VCardBatchPreviewRequest(
files=[
batch_file(
"duplicates.vcf",
vcard("same", "First", "first@example.test")
+ vcard("same", "Last", "last@example.test"),
)
],
duplicate_card_policy="reject",
),
)
self.assertEqual(
["conflict", "conflict"], [item["action"] for item in run.plan_data]
)
cancelled = cancel_vcard_batch(
self.session,
self.principal,
run.id,
VCardBatchCancelRequest(
expected_plan_hash=run.plan_hash,
reason="Operator rejected duplicate source UIDs.",
),
)
self.assertEqual("cancelled", cancelled.status)
self.assertEqual(0, self.session.query(Contact).count())
last = preview_vcard_batch(
self.session,
self.principal,
self.book.id,
VCardBatchPreviewRequest(
files=[
batch_file(
"duplicates.vcf",
vcard("same", "First", "first@example.test")
+ vcard("same", "Last", "last@example.test"),
)
],
duplicate_card_policy="last",
),
)
self.assertEqual(
["ignore", "create"], [item["action"] for item in last.plan_data]
)
def test_deterministic_scoped_export_supports_vcard_versions(self) -> None:
grace = create_contact(
self.session,
self.principal,
self.book.id,
ContactCreateRequest(display_name="Grace Hopper"),
)
ada = create_contact(
self.session,
self.principal,
self.book.id,
ContactCreateRequest(display_name="Ada Lovelace"),
)
address_list = AddressList(
tenant_id="tenant-1",
address_book_id=self.book.id,
name="Selected",
source_kind="local",
read_only=False,
)
self.session.add(address_list)
self.session.flush()
self.session.add(
AddressListEntry(
address_list_id=address_list.id, contact_id=grace.id, order_index=0
)
)
self.session.flush()
selected = export_vcards(
self.session,
self.principal,
self.book.id,
VCardExportRequest(
scope="contacts", contact_ids=[grace.id, ada.id], version="3.0"
),
)
repeated = export_vcards(
self.session,
self.principal,
self.book.id,
VCardExportRequest(
scope="contacts", contact_ids=[ada.id, grace.id], version="3.0"
),
)
listed = export_vcards(
self.session,
self.principal,
self.book.id,
VCardExportRequest(scope="address_list", address_list_id=address_list.id),
)
self.assertEqual(selected["content_hash"], repeated["content_hash"])
self.assertLess(
selected["content"].index("Ada Lovelace"),
selected["content"].index("Grace Hopper"),
)
self.assertIn("VERSION:3.0", selected["content"])
self.assertEqual(1, listed["contact_count"])
self.assertIn("Grace Hopper", listed["content"])
def test_parser_rejects_pathological_unfolded_lines(self) -> None:
result = parse_vcards_with_issues(
"BEGIN:VCARD\nFN:"
+ ("a" * (MAX_VCARD_UNFOLDED_LINE_CHARS + 1))
+ "\nEND:VCARD"
)
self.assertEqual([], result.cards)
self.assertIn("unfolded lines", result.issues[0].message)
if __name__ == "__main__":
unittest.main()
+4 -3
View File
@@ -1,6 +1,6 @@
{ {
"name": "@govoplan/addresses-webui", "name": "@govoplan/addresses-webui",
"version": "0.1.18", "version": "0.1.23",
"private": true, "private": true,
"type": "module", "type": "module",
"main": "src/index.ts", "main": "src/index.ts",
@@ -14,10 +14,11 @@
"./styles/addresses.css": "./src/styles/addresses.css" "./styles/addresses.css": "./src/styles/addresses.css"
}, },
"scripts": { "scripts": {
"test:ui-structure": "node scripts/test-selection-list-structure.mjs" "test:ui-structure": "node scripts/test-selection-list-structure.mjs",
"test:import-run": "rm -rf .import-run-test-build && mkdir -p .import-run-test-build && printf '{\"type\":\"commonjs\"}\\n' > .import-run-test-build/package.json && ../../govoplan-core/webui/node_modules/.bin/tsc -p tsconfig.import-run-tests.json && node .import-run-test-build/tests/import-run-state.test.js"
}, },
"peerDependencies": { "peerDependencies": {
"@govoplan/core-webui": "^0.1.18", "@govoplan/core-webui": "^0.1.45",
"lucide-react": "^1.23.0", "lucide-react": "^1.23.0",
"react": ">=19.2.7 <20", "react": ">=19.2.7 <20",
"react-dom": ">=19.2.7 <20", "react-dom": ">=19.2.7 <20",
@@ -17,5 +17,14 @@ assert.match(page, /<SegmentedControl<ConflictMergeChoice>[\s\S]*role="group"[\s
assert.doesNotMatch(page, /<button[\s\S]{0,160}(?:address-contact-row|address-sync-result-row)/); assert.doesNotMatch(page, /<button[\s\S]{0,160}(?:address-contact-row|address-sync-result-row)/);
assert.doesNotMatch(styles, /\.address-conflict-choice button/); assert.doesNotMatch(styles, /\.address-conflict-choice button/);
assert.doesNotMatch(styles, /\.address-contact-row:(?:hover|focus-visible)/); assert.doesNotMatch(styles, /\.address-contact-row:(?:hover|focus-visible)/);
assert.match(page, /<PageActionBar[\s\S]*variant="collection"[\s\S]*reloadAction=[\s\S]*createAction=/);
assert.doesNotMatch(page, /renderSelectedBookActions|address-icon-actions/);
assert.match(page, /renderAddressActions\(\)/);
assert.match(page, /<FormSection variant="separated" title="i18n:govoplan-addresses\.explorer\.archive_section"/);
const openTreeNode = page.slice(page.indexOf(" function openTreeNode("), page.indexOf(" function toggleTreeNode("));
assert.doesNotMatch(openTreeNode, /toggleTreeNode\(/, "Labels only select, never expand or collapse.");
assert.match(openTreeNode, /setSelectedTreeGroup\(\{ id: node\.id, label: node\.label \}\)/);
assert.match(page, /selectedTreeGroup\?\.id \?\?/);
assert.match(page, /if \(!previousBranchIds\.has\(id\)\) next\.add\(id\)/, "Reload preserves collapsed branches.");
console.log("Address-book flat selections use central components."); console.log("Address-book flat selections use central components.");
+123 -7
View File
@@ -354,6 +354,60 @@ export type VCardImportResult = {
issues: Array<{ index: number; message: string; severity: "warning" | "error"; field?: string | null; line?: number | null }>; issues: Array<{ index: number; message: string; severity: "warning" | "error"; field?: string | null; line?: number | null }>;
}; };
export type VCardBatchPlanItem = {
source_key: string;
source_filename: string;
card_index: number;
action: "create" | "update" | "ignore" | "unchanged" | "conflict";
allowed_actions: Array<"create" | "update" | "ignore">;
contact_id?: string | null;
display_name?: string | null;
changed_fields: string[];
duplicate_suggestions: Array<{ contact_id: string; display_name: string; reasons: string[] }>;
message?: string | null;
};
export type VCardBatchRun = {
id: string;
address_book_id: string;
status: string;
input_hash: string;
plan_hash: string;
parser_version: string;
execution_mode: "bounded_sync" | "persisted_batch";
file_count: number;
card_count: number;
statistics: Record<string, number | string>;
diagnostics: Array<{
severity: "info" | "warning" | "error";
code: string;
message: string;
source_filename?: string | null;
card_index?: number | null;
field?: string | null;
details: Record<string, unknown>;
}>;
plan: VCardBatchPlanItem[];
progress: { total: number; completed: number; created: number; updated: number; ignored: number; failed: number };
can_apply: boolean;
can_cancel: boolean;
commit_hash?: string | null;
created_at: string;
updated_at: string;
applied_at?: string | null;
};
export type VCardExportResult = {
filename: string;
media_type: string;
scope: "address_book" | "address_list" | "contacts";
version: "3.0" | "4.0";
ordering: string;
contact_count: number;
content_hash: string;
content: string;
};
export type AddressSyncSource = { export type AddressSyncSource = {
id: string; id: string;
tenant_id?: string | null; tenant_id?: string | null;
@@ -494,6 +548,7 @@ export type AddressImportConfiguration = {
duplicate_source_key_policy: "reject" | "first" | "last"; duplicate_source_key_policy: "reject" | "first" | "last";
existing_contact_policy: "update" | "ignore" | "reject"; existing_contact_policy: "update" | "ignore" | "reject";
blank_value_policy: "ignore" | "clear" | "reject"; blank_value_policy: "ignore" | "clear" | "reject";
ldif_change_record_policy: "reject" | "ignore" | "treat_add_as_entry";
locale?: string | null; locale?: string | null;
default_tags: string[]; default_tags: string[];
max_rows: number; max_rows: number;
@@ -508,7 +563,7 @@ export type AddressImportProfile = {
scope_id?: string | null; scope_id?: string | null;
name: string; name: string;
description?: string | null; description?: string | null;
source_format: "csv" | "xlsx"; source_format: "csv" | "xlsx" | "ldif";
configuration: AddressImportConfiguration; configuration: AddressImportConfiguration;
is_current: boolean; is_current: boolean;
created_at: string; created_at: string;
@@ -537,7 +592,7 @@ export type AddressImportDiagnostic = {
export type AddressImportRun = { export type AddressImportRun = {
id: string; id: string;
address_book_id: string; address_book_id: string;
profile_id: string; profile_id: string | null;
source_filename: string; source_filename: string;
source_format: string; source_format: string;
input_hash: string; input_hash: string;
@@ -1056,6 +1111,47 @@ export function importAddressBookVcards(settings: ApiSettings, addressBookId: st
}); });
} }
export function previewVCardBatch(
settings: ApiSettings,
addressBookId: string,
payload: {
files: Array<{ filename: string; content_base64: string }>;
duplicate_card_policy?: "reject" | "first" | "last";
existing_contact_policy?: "update" | "ignore" | "reject";
}
): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcard-batches/preview`, {
method: "POST",
body: JSON.stringify(payload)
});
}
export function getVCardBatch(settings: ApiSettings, runId: string): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(runId)}`);
}
export function applyVCardBatch(
settings: ApiSettings,
run: VCardBatchRun,
selections: Array<{ source_key: string; action: "create" | "update" | "ignore" }>
): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(run.id)}/apply`, {
method: "POST",
body: JSON.stringify({ expected_plan_hash: run.plan_hash, selections })
});
}
export function cancelVCardBatch(settings: ApiSettings, run: VCardBatchRun, reason: string): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(run.id)}/cancel`, {
method: "POST",
body: JSON.stringify({ expected_plan_hash: run.plan_hash, reason })
});
}
export function exportVCardBatchDiagnostics(settings: ApiSettings, runId: string): Promise<string> {
return apiFetch<string>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(runId)}/diagnostics`);
}
export async function listAddressImportProfiles(settings: ApiSettings): Promise<AddressImportProfile[]> { export async function listAddressImportProfiles(settings: ApiSettings): Promise<AddressImportProfile[]> {
const response = await apiFetch<AddressImportProfileListResponse>(settings, "/api/v1/addresses/import-profiles"); const response = await apiFetch<AddressImportProfileListResponse>(settings, "/api/v1/addresses/import-profiles");
return response.profiles; return response.profiles;
@@ -1068,7 +1164,7 @@ export function createAddressImportProfile(
scope_id?: string | null; scope_id?: string | null;
name: string; name: string;
description?: string | null; description?: string | null;
source_format: "csv" | "xlsx"; source_format: "csv" | "xlsx" | "ldif";
configuration: AddressImportConfiguration; configuration: AddressImportConfiguration;
} }
): Promise<AddressImportProfile> { ): Promise<AddressImportProfile> {
@@ -1100,17 +1196,21 @@ export function previewAddressImport(
}); });
} }
export function getAddressImportRun(settings: ApiSettings, runId: string): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${encodeURIComponent(runId)}`);
}
export function applyAddressImport(settings: ApiSettings, run: AddressImportRun): Promise<AddressImportRun> { export function applyAddressImport(settings: ApiSettings, run: AddressImportRun): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${run.id}/apply`, { return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${encodeURIComponent(run.id)}/apply`, {
method: "POST", method: "POST",
body: JSON.stringify({ expected_plan_hash: run.plan_hash }) body: JSON.stringify({ expected_plan_hash: run.plan_hash })
}); });
} }
export function rollbackAddressImport(settings: ApiSettings, runId: string, reason: string): Promise<AddressImportRun> { export function rollbackAddressImport(settings: ApiSettings, run: AddressImportRun, reason: string): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${runId}/rollback`, { return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${encodeURIComponent(run.id)}/rollback`, {
method: "POST", method: "POST",
body: JSON.stringify({ reason }) body: JSON.stringify({ expected_plan_hash: run.plan_hash, reason })
}); });
} }
@@ -1118,6 +1218,22 @@ export function exportAddressBookVcards(settings: ApiSettings, addressBookId: st
return apiFetch<string>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcards/export`); return apiFetch<string>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcards/export`);
} }
export function exportScopedVcards(
settings: ApiSettings,
addressBookId: string,
payload: {
scope: "address_book" | "address_list" | "contacts";
address_list_id?: string | null;
contact_ids?: string[];
version?: "3.0" | "4.0";
}
): Promise<VCardExportResult> {
return apiFetch<VCardExportResult>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcards/export`, {
method: "POST",
body: JSON.stringify(payload)
});
}
export function exportContactVcard(settings: ApiSettings, contactId: string): Promise<string> { export function exportContactVcard(settings: ApiSettings, contactId: string): Promise<string> {
return apiFetch<string>(settings, `/api/v1/addresses/contacts/${contactId}/vcard`); return apiFetch<string>(settings, `/api/v1/addresses/contacts/${contactId}/vcard`);
} }
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,72 @@
export type ImportRunLifecycle = {
label: string;
tone: "info" | "success" | "warning" | "inactive";
canApply: boolean;
canRollback: boolean;
guidance: string;
};
export function importRunIdFromSearch(search: URLSearchParams): string {
return search.get("import_run")?.trim() ?? "";
}
export function withImportRunSearch(search: URLSearchParams, runId?: string | null): URLSearchParams {
const next = new URLSearchParams(search);
const normalized = runId?.trim();
if (normalized) next.set("import_run", normalized);
else next.delete("import_run");
return next;
}
export function importRunLifecycle(status: string): ImportRunLifecycle {
if (status === "previewed") {
return {
label: "Ready for review",
tone: "info",
canApply: true,
canRollback: false,
guidance: "Review the persisted effects and diagnostics before applying this plan."
};
}
if (status === "applied") {
return {
label: "Applied",
tone: "success",
canApply: false,
canRollback: true,
guidance: "The plan has already been applied. Rollback remains guarded by its recorded plan hash and contact evidence."
};
}
if (status === "rolled_back") {
return {
label: "Rolled back",
tone: "inactive",
canApply: false,
canRollback: false,
guidance: "This run was rolled back and cannot be applied again. Create a new preview to import the source again."
};
}
if (status === "expired") {
return {
label: "Expired",
tone: "warning",
canApply: false,
canRollback: false,
guidance: "This preview is no longer actionable. Create a new preview from the original source."
};
}
return {
label: status || "Unavailable",
tone: "warning",
canApply: false,
canRollback: false,
guidance: "This run is not actionable in its current lifecycle state."
};
}
export function unavailableImportRunMessage(status?: number): string {
if (status === 404 || status === 410) {
return "This import run is missing, expired, or not available to your tenant. No source data was loaded.";
}
return "The persisted import run could not be loaded. Reload it after the service becomes available.";
}
+54
View File
@@ -2,6 +2,33 @@ import type { PlatformTranslations } from "@govoplan/core-webui";
export const generatedTranslations: PlatformTranslations = { export const generatedTranslations: PlatformTranslations = {
en: { en: {
"i18n:govoplan-addresses.explorer.page_actions": "Address book actions",
"i18n:govoplan-addresses.explorer.transfer": "Import / export",
"i18n:govoplan-addresses.explorer.connections": "Connections",
"i18n:govoplan-addresses.explorer.quality": "Address quality",
"i18n:govoplan-addresses.explorer.manage": "Manage",
"i18n:govoplan-addresses.explorer.manage_selection": "Manage selected book or list",
"i18n:govoplan-addresses.explorer.no_book": "Select an address book first.",
"i18n:govoplan-addresses.explorer.choose_book_in_group": "Select an address book in this group. Use the folder icon to expand or collapse it; clicking the label only selects it.",
"i18n:govoplan-addresses.explorer.archive_section": "Archive selected book or list",
"i18n:govoplan-addresses.explorer.import_section": "Import into the selected book",
"i18n:govoplan-addresses.explorer.export_section": "Export the selected book or list",
"i18n:govoplan-addresses.explorer.vcard_version": "vCard export version",
"i18n:govoplan-addresses.explorer.export_list": "Export address list",
"i18n:govoplan-addresses.explorer.export_book": "Export address book",
"i18n:govoplan-addresses.explorer.connect_section": "Connect an address source",
"i18n:govoplan-addresses.explorer.sync_section": "Synchronization for the selected book",
"i18n:govoplan-addresses.explorer.no_sync_source": "The selected book has no connected synchronization source.",
"Restore address list": "Restore address list",
"Restore address book": "Restore address book",
"Delete address list": "Delete address list",
"Delete address book": "Delete address book",
"Import contacts": "Import contacts",
"Connect CardDAV": "Connect CardDAV",
"Connect LDAP or Active Directory": "Connect LDAP or Active Directory",
"Inspect sync source": "Inspect sync source",
"Preview sync": "Preview sync",
"Run sync": "Run sync",
"i18n:govoplan-addresses.add_contact.6da0b4b8": "Add contact", "i18n:govoplan-addresses.add_contact.6da0b4b8": "Add contact",
"i18n:govoplan-addresses.address_book.f6327f59": "Address Book", "i18n:govoplan-addresses.address_book.f6327f59": "Address Book",
"i18n:govoplan-addresses.address_book_scopes.b0d0efde": "Address book scopes", "i18n:govoplan-addresses.address_book_scopes.b0d0efde": "Address book scopes",
@@ -88,6 +115,33 @@ export const generatedTranslations: PlatformTranslations = {
"Note": "Note" "Note": "Note"
}, },
de: { de: {
"i18n:govoplan-addresses.explorer.page_actions": "Adressbuchaktionen",
"i18n:govoplan-addresses.explorer.transfer": "Import / Export",
"i18n:govoplan-addresses.explorer.connections": "Verbindungen",
"i18n:govoplan-addresses.explorer.quality": "Adressqualität",
"i18n:govoplan-addresses.explorer.manage": "Verwalten",
"i18n:govoplan-addresses.explorer.manage_selection": "Ausgewähltes Adressbuch oder Liste verwalten",
"i18n:govoplan-addresses.explorer.no_book": "Wählen Sie zuerst ein Adressbuch aus.",
"i18n:govoplan-addresses.explorer.choose_book_in_group": "Wählen Sie ein Adressbuch in dieser Gruppe aus. Das Ordnersymbol klappt auf oder zu; ein Klick auf die Beschriftung wählt nur aus.",
"i18n:govoplan-addresses.explorer.archive_section": "Ausgewähltes Adressbuch oder Liste archivieren",
"i18n:govoplan-addresses.explorer.import_section": "In das ausgewählte Adressbuch importieren",
"i18n:govoplan-addresses.explorer.export_section": "Ausgewähltes Adressbuch oder Liste exportieren",
"i18n:govoplan-addresses.explorer.vcard_version": "vCard-Exportversion",
"i18n:govoplan-addresses.explorer.export_list": "Adressliste exportieren",
"i18n:govoplan-addresses.explorer.export_book": "Adressbuch exportieren",
"i18n:govoplan-addresses.explorer.connect_section": "Eine Adressquelle verbinden",
"i18n:govoplan-addresses.explorer.sync_section": "Synchronisierung des ausgewählten Adressbuchs",
"i18n:govoplan-addresses.explorer.no_sync_source": "Das ausgewählte Adressbuch hat keine verbundene Synchronisierungsquelle.",
"Restore address list": "Adressliste wiederherstellen",
"Restore address book": "Adressbuch wiederherstellen",
"Delete address list": "Adressliste löschen",
"Delete address book": "Adressbuch löschen",
"Import contacts": "Kontakte importieren",
"Connect CardDAV": "CardDAV verbinden",
"Connect LDAP or Active Directory": "LDAP oder Active Directory verbinden",
"Inspect sync source": "Synchronisierungsquelle prüfen",
"Preview sync": "Synchronisierungsvorschau",
"Run sync": "Synchronisierung starten",
"i18n:govoplan-addresses.add_contact.6da0b4b8": "Kontakt hinzufügen", "i18n:govoplan-addresses.add_contact.6da0b4b8": "Kontakt hinzufügen",
"i18n:govoplan-addresses.address_book.f6327f59": "Adressbuch", "i18n:govoplan-addresses.address_book.f6327f59": "Adressbuch",
"i18n:govoplan-addresses.address_book_scopes.b0d0efde": "Adressbuch-Bereiche", "i18n:govoplan-addresses.address_book_scopes.b0d0efde": "Adressbuch-Bereiche",
+31 -26
View File
@@ -9,7 +9,7 @@
justify-content: space-between; justify-content: space-between;
gap: 16px; gap: 16px;
border: var(--border-line); border: var(--border-line);
border-radius: 6px; border-radius: var(--radius-compact);
background: var(--panel-soft); background: var(--panel-soft);
padding: 14px; padding: 14px;
} }
@@ -34,8 +34,8 @@
} }
.address-book-page.address-book-fullscreen { .address-book-page.address-book-fullscreen {
display: grid; display: flex;
grid-template-rows: 1fr; flex-direction: column;
height: calc(100vh - 115px); height: calc(100vh - 115px);
overflow: hidden; overflow: hidden;
padding: 0; padding: 0;
@@ -43,7 +43,8 @@
} }
.address-workspace-frame { .address-workspace-frame {
height: 100%; flex: 1 1 auto;
height: auto;
min-height: 0; min-height: 0;
} }
@@ -73,22 +74,10 @@
background: var(--panel-header); background: var(--panel-header);
} }
.address-tree-header .button-row { .address-page-actions {
flex-wrap: wrap; border-bottom: var(--border-line);
} flex: 0 0 auto;
padding: var(--space-3);
.address-icon-actions {
max-width: 150px;
}
.address-icon-actions .btn {
align-items: center;
aspect-ratio: 1;
display: inline-flex;
justify-content: center;
min-height: 30px;
padding: 0;
width: 30px;
} }
.address-tree-filter-row { .address-tree-filter-row {
@@ -163,6 +152,21 @@
min-height: 0; min-height: 0;
} }
.address-import-run-state {
align-items: center;
background: var(--panel-soft);
border: var(--border-line);
border-radius: var(--radius-compact);
display: grid;
gap: 10px 14px;
grid-template-columns: minmax(180px, 1fr) auto minmax(220px, 1.4fr) auto;
padding: 10px 12px;
}
.address-import-run-state p {
margin: 0;
}
.address-import-profile-actions { .address-import-profile-actions {
align-items: end; align-items: end;
justify-content: flex-start; justify-content: flex-start;
@@ -189,7 +193,7 @@
.address-import-preview > .address-sync-plan-grid > div { .address-import-preview > .address-sync-plan-grid > div {
background: var(--panel-soft); background: var(--panel-soft);
border: var(--border-line); border: var(--border-line);
border-radius: 6px; border-radius: var(--radius-compact);
display: grid; display: grid;
gap: 2px; gap: 2px;
padding: 9px 10px; padding: 9px 10px;
@@ -406,7 +410,7 @@
.address-tag { .address-tag {
background: var(--panel-soft); background: var(--panel-soft);
border: var(--border-line); border: var(--border-line);
border-radius: 999px; border-radius: var(--radius-pill);
color: var(--muted); color: var(--muted);
display: inline-flex; display: inline-flex;
font-size: 0.75rem; font-size: 0.75rem;
@@ -553,7 +557,7 @@
.address-member-candidate-list { .address-member-candidate-list {
border: var(--border-line); border: var(--border-line);
border-radius: 6px; border-radius: var(--radius-compact);
display: grid; display: grid;
max-height: min(460px, calc(100vh - 300px)); max-height: min(460px, calc(100vh - 300px));
overflow: auto; overflow: auto;
@@ -598,7 +602,7 @@
.address-form-section { .address-form-section {
border: var(--border-line); border: var(--border-line);
border-radius: 6px; border-radius: var(--radius-compact);
display: grid; display: grid;
gap: 10px; gap: 10px;
padding: 12px; padding: 12px;
@@ -678,7 +682,7 @@
.address-quality-list { .address-quality-list {
border: var(--border-line); border: var(--border-line);
border-radius: 6px; border-radius: var(--radius-compact);
max-height: 300px; max-height: 300px;
overflow: auto; overflow: auto;
} }
@@ -809,8 +813,9 @@
opacity: 0.72; opacity: 0.72;
} }
@media (max-width: 980px) { @media (max-width: 1100px) {
.address-book-workspace, .address-book-workspace,
.address-import-run-state,
.address-form-row-email, .address-form-row-email,
.address-form-row-phone, .address-form-row-phone,
.address-form-row-postal { .address-form-row-postal {
+22
View File
@@ -0,0 +1,22 @@
import assert from "node:assert/strict";
import {
importRunIdFromSearch,
importRunLifecycle,
unavailableImportRunMessage,
withImportRunSearch
} from "../src/features/addressbook/importRunState";
const initial = new URLSearchParams("q=Ada&import_run=run-123");
assert.equal(importRunIdFromSearch(initial), "run-123");
assert.equal(withImportRunSearch(initial, "run-456").toString(), "q=Ada&import_run=run-456");
assert.equal(withImportRunSearch(initial, null).toString(), "q=Ada");
assert.equal(importRunLifecycle("previewed").canApply, true);
assert.equal(importRunLifecycle("previewed").canRollback, false);
assert.equal(importRunLifecycle("applied").canApply, false);
assert.equal(importRunLifecycle("applied").canRollback, true);
assert.equal(importRunLifecycle("rolled_back").canRollback, false);
assert.equal(importRunLifecycle("expired").label, "Expired");
assert.match(unavailableImportRunMessage(404), /missing, expired, or not available to your tenant/);
console.log("Address import-run deep-link and lifecycle state tests passed.");
+17
View File
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"module": "CommonJS",
"moduleResolution": "Node",
"target": "ES2022",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"typeRoots": ["../../govoplan-core/webui/node_modules/@types"],
"types": ["node"],
"outDir": ".import-run-test-build"
},
"include": [
"src/features/addressbook/importRunState.ts",
"tests/import-run-state.test.ts"
]
}