11 Commits
Author SHA1 Message Date
zemion 5cdf0ae9ff docs: complete German structured documentation
Module Package Release / publish-packages (push) Successful in 12s
2026-08-24 01:15:30 +02:00
zemion 3458306e04 docs(addresses): complete German reference coverage
Module Package Release / publish-packages (push) Successful in 12s
2026-08-23 20:11:47 +02:00
zemion 8740fb33f8 feat(addresses): add governed DSAR coverage 2026-08-21 01:19:17 +02:00
zemion 42fe262376 feat: add selective vCard batch workflows 2026-08-20 17:24:42 +02:00
zemion 147f34c5c1 feat: add governed LDIF contact imports 2026-08-20 11:19:23 +02:00
zemion ec32ad2c37 feat(imports): resume persisted address runs 2026-08-19 20:39:21 +02:00
zemion 7e5c1cf10c feat: align addresses with shared UI foundations 2026-08-18 21:32:34 +02:00
zemion 63d59a7eaa Adopt shared WebUI layout primitives 2026-08-18 11:30:39 +02:00
zemion e6165b4d26 Adopt shared WebUI layout primitives 2026-08-18 10:42:50 +02:00
zemion aa390a69b9 Release v0.1.18
Module Package Release / publish-packages (push) Successful in 11s
2026-08-05 21:07:51 +02:00
zemion aab4019612 Release v0.1.17
Module Package Release / publish-packages (push) Successful in 12s
2026-08-05 20:34:12 +02:00
32 changed files with 5723 additions and 387 deletions
+2
View File
@@ -142,11 +142,13 @@ dist
.policy-test-build/
.template-preview-test-build/
.import-test-build/
.import-run-test-build/
webui/.component-test-build/
webui/.module-test-build/
webui/.policy-test-build/
webui/.template-preview-test-build/
webui/.import-test-build/
webui/.import-run-test-build/
# ---> Python
# Byte-compiled / optimized / DLL files
+19 -2
View File
@@ -20,8 +20,14 @@ tenant summaries, and uninstall guards.
The first UI supports user, group, tenant, and system-scoped address books,
multi-value contact methods, soft deletion, restore, read-only lookup/search,
and vCard import/export for common contact fields. Imported vCards preserve
source payload and revision metadata for later sync/conflict work.
and vCard import/export for common contact fields. Multi-file vCard imports now
create a persisted preview before mutation, expose duplicate suggestions and
per-card create/update/ignore choices, reject stale plans, and make identical
commit retries idempotent. Pending batches can be reloaded or cancelled.
Address-book, address-list, and selected-contact exports explicitly support
vCard 3.0 or 4.0 with deterministic ordering and a recorded content hash.
Imported vCards preserve source payload and revision metadata for later
sync/conflict work, while batch diagnostics expose only bounded metadata.
The backend and WebUI also support classical address lists: reusable groupings
of contacts or specific contact methods within one address book. Campaigns can
@@ -98,6 +104,17 @@ The module exposes core-mediated capabilities for:
pickers.
- `distribution.recipient_channel_facts`: current channel, governance, and
quality facts for distribution and Policy consumers.
- `privacy.dsar.addresses`: tenant-bounded, minimized data-subject discovery
across contacts, contact points, list use, governance, provenance,
synchronization evidence, and operator attribution.
The DSAR provider accepts corroborated email/account selectors and namespaced
Addresses references. It does not export connector state, raw import or sync
payloads, opaque metadata, snapshot payloads, or merge before/after payloads.
Reusable contacts are never deleted automatically: shared/synchronized contact
changes require an authorized dependency review through the ordinary Addresses
workflows, while governance, quality, merge, sync, import, and attribution
evidence is retained with an explicit reason.
`addresses.recipient_source` returns:
+19 -2
View File
@@ -187,9 +187,9 @@ successful search can infer deletion. A timeout, bind failure, malformed entry,
duplicate key, or configured entry limit retains existing contacts and reports
the source as failed/stale instead of creating tombstones.
## Static Tabular Imports
## Static Mapped Imports
CSV and XLSX use versioned, scoped mapping profiles rather than live sync
CSV, XLSX, and LDIF use versioned, scoped mapping profiles rather than live sync
sources. Profiles retain delimiter, encoding, header or worksheet selection,
stable source-key mapping, field mappings, locale and tags, row limits, and
explicit duplicate, blank-value, and existing-contact policies. Updating a
@@ -207,6 +207,23 @@ matches its recorded post-apply hash. Arbitrary transforms remain Dataflow's
responsibility; Files and Datasources are optional origins, not prerequisites
for direct upload.
LDIF is unfolded and parsed as a bounded stream of entries. Attribute names are
case-insensitive; UTF-8 and base64-encoded text and repeated values are retained
for mapping, while binary and URL values produce diagnostics and are never
projected or fetched. The default change-record policy rejects change records.
A profile can instead ignore them, or treat `changetype: add` as a static entry;
modify and delete records are never translated into contact mutations. Entry
hashes, input/plan hashes, mapping-version provenance, and the same correction
and guarded rollback lifecycle apply as for CSV and XLSX.
Persisted import runs can be resumed through `/address-book?import_run=<id>`.
The WebUI reloads the bounded run projection, selects its address book and
mapping version, and restores statistics, diagnostics, effects, and lifecycle
state. Apply and rollback both carry the reviewed plan hash. The normal read
projection never includes uploaded bytes or private before-images, and an
unknown, expired, hidden, or cross-tenant id is presented as one unavailable
state so the deep link cannot enumerate another tenant's imports.
## Quality, Deduplication, And Recovery
Quality is evidence about a concrete contact point, separate from communication
+1 -1
View File
@@ -224,7 +224,7 @@ Tasks:
- [ ] [On-premises Exchange connector profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/17)
- [ ] [Google People contacts connector](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/18)
- [ ] [Reusable CSV/XLSX import mapping profiles](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/19)
- [ ] [Bounded LDIF import profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/20)
- [x] [Bounded LDIF import profile](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/20)
- [ ] [Selective and large-batch vCard workflows](https://git.add-ideas.de/GovOPlaN/govoplan-addresses/issues/21)
- [x] classical address-list UI; reusable static/dynamic operational segments move
to `govoplan-dist-lists`
+5
View File
@@ -26,6 +26,11 @@ imports, synchronization, quality review, and reversible merge operations.
source diagnostics and conflict state as explicitly stated.
- Imports and synchronization separate preview from apply; incomplete external
reads never infer deletions.
- vCard batch upload accepts multiple files, persists a non-mutating preview,
and requires an explicit create/update/ignore choice for every reviewed card.
Reload and cancellation preserve the pending plan; only applying the matching
plan hash mutates contacts. Scoped exports name the selected vCard version and
use deterministic ordering.
- Merge and communication-governance operations append auditable evidence and
never silently erase prior state.
- Request feedback is rendered as a compact shared alert over the full-height
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@govoplan/addresses-webui",
"version": "0.1.16",
"version": "0.1.20",
"private": true,
"type": "module",
"main": "webui/src/index.ts",
@@ -18,7 +18,7 @@
"README.md"
],
"peerDependencies": {
"@govoplan/core-webui": "^0.1.16",
"@govoplan/core-webui": "^0.1.18",
"lucide-react": "^1.23.0",
"react": ">=19.2.7 <20",
"react-dom": ">=19.2.7 <20",
+2 -2
View File
@@ -4,14 +4,14 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan-addresses"
version = "0.1.16"
version = "0.1.20"
description = "GovOPlaN reusable address and recipient-source module."
readme = "README.md"
requires-python = ">=3.12"
authors = [{ name = "GovOPlaN" }]
dependencies = [
"defusedxml>=0.7.1",
"govoplan-core>=0.1.16",
"govoplan-core>=0.1.18",
"ldap3>=2.9.1,<3",
"openpyxl>=3.1.5,<4",
]
+3 -3
View File
@@ -587,9 +587,9 @@ class AddressImportRun(Base, TimestampMixin):
nullable=False,
index=True,
)
profile_id: Mapped[str] = mapped_column(
profile_id: Mapped[str | None] = mapped_column(
ForeignKey("addresses_import_profiles.id", ondelete="RESTRICT"),
nullable=False,
nullable=True,
index=True,
)
source_filename: Mapped[str] = mapped_column(String(500), nullable=False)
@@ -607,7 +607,7 @@ class AddressImportRun(Base, TimestampMixin):
rolled_back_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
address_book: Mapped[AddressBook] = relationship()
profile: Mapped[AddressImportProfile] = relationship()
profile: Mapped[AddressImportProfile | None] = relationship()
__all__ = [
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,55 @@
"""German translations for public structured documentation metadata."""
from __future__ import annotations
from typing import Any
GERMAN_STRUCTURED_TRANSLATIONS: dict[str, dict[str, Any]] = {'addresses.reference.fields-and-consequences': {'consequence_classes': {'archive': 'Entfernt das '
'Objekt aus '
'der '
'gewöhnlichen '
'Auswahl, '
'während die '
'verwaltete '
'Geschichte '
'und '
'Referenzen '
'beibehalten '
'werden.',
'governance_fact': 'Hinzufügen '
'oder '
'Beenden '
'einer '
'effektiv '
'datierten '
'Kommunikationsentscheidung '
'ohne '
'vorherige '
'Fakten '
'zu '
'löschen.',
'import_or_sync': 'Wendet '
'nur '
'einen '
'überprüften '
'Bounded '
'Plan '
'an und '
'behält '
'die '
'Quellenrevision, '
'Diagnose '
'und '
'Herkunft '
'bei.',
'merge': 'Repoints '
'verwaltet '
'Verweise auf '
'einen '
'Überlebenden '
'und behält '
'reversible '
'Redirect und '
'Provenienz '
'Nachweise.'}}}
@@ -6,7 +6,7 @@ from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
AddressImportFormat = Literal["csv", "xlsx"]
AddressImportFormat = Literal["csv", "xlsx", "ldif"]
AddressImportScope = Literal["user", "group", "tenant", "system"]
IMPORT_TARGET_FIELDS = frozenset(
@@ -41,6 +41,7 @@ class AddressImportConfiguration(BaseModel):
duplicate_source_key_policy: Literal["reject", "first", "last"] = "reject"
existing_contact_policy: Literal["update", "ignore", "reject"] = "update"
blank_value_policy: Literal["ignore", "clear", "reject"] = "ignore"
ldif_change_record_policy: Literal["reject", "ignore", "treat_add_as_entry"] = "reject"
locale: str | None = Field(default=None, max_length=35)
default_tags: list[str] = Field(default_factory=list, max_length=100)
max_rows: int = Field(default=10_000, ge=1, le=10_000)
@@ -128,7 +129,7 @@ class AddressImportDiagnosticResponse(BaseModel):
class AddressImportRunResponse(BaseModel):
id: str
address_book_id: str
profile_id: str
profile_id: str | None
source_filename: str
source_format: str
input_hash: str
@@ -151,6 +152,7 @@ class AddressImportCommitRequest(BaseModel):
class AddressImportRollbackRequest(BaseModel):
expected_plan_hash: str = Field(min_length=64, max_length=64)
reason: str = Field(min_length=3, max_length=2000)
+155 -20
View File
@@ -24,6 +24,7 @@ from govoplan_addresses.backend.import_schemas import (
AddressImportProfileUpdateRequest,
AddressImportRollbackRequest,
)
from govoplan_addresses.backend.ldif import parse_ldif_rows
from govoplan_addresses.backend.schemas import (
ContactCreateRequest,
ContactEmailPayload,
@@ -199,6 +200,7 @@ def preview_address_import(
input_hash=input_hash,
rows=rows,
config=config,
source_format=profile.source_format,
)
diagnostics = [*parse_diagnostics, *map_diagnostics]
statistics = dict(Counter(item["action"] for item in plan_data))
@@ -336,6 +338,8 @@ def rollback_address_import(
payload: AddressImportRollbackRequest,
) -> AddressImportRun:
run = get_import_run(session, principal, run_id)
if run.plan_hash != payload.expected_plan_hash:
raise AddressBookError("The reviewed import plan changed; reload the import run.")
if run.status == "rolled_back":
return run
if run.status != "applied":
@@ -499,13 +503,24 @@ def _parse_rows(
filename: str,
source_format: str,
config: AddressImportConfiguration,
) -> tuple[list[tuple[int, dict[str, str]]], list[dict[str, Any]]]:
) -> tuple[list[tuple[int, dict[str, Any]]], list[dict[str, Any]]]:
if source_format == "csv":
return _parse_csv(raw, config=config)
if source_format == "xlsx":
if not filename.casefold().endswith(".xlsx"):
raise AddressBookError("XLSX imports require an .xlsx file; macros and legacy workbooks are not accepted.")
return _parse_xlsx(raw, config=config)
if source_format == "ldif":
if not filename.casefold().endswith((".ldif", ".ldi")):
raise AddressBookError("LDIF imports require an .ldif or .ldi file.")
try:
return parse_ldif_rows(
raw,
max_entries=config.max_rows,
change_record_policy=config.ldif_change_record_policy,
)
except ValueError as exc:
raise AddressBookError(str(exc)) from exc
raise AddressBookError(f"Unsupported address import format: {source_format!r}.")
@@ -598,32 +613,81 @@ def _row_dict(headers: list[str], values: list[Any]) -> dict[str, str]:
}
def _column_key(value: object, *, casefold: bool) -> str:
column = str(value).strip()
return column.casefold() if casefold else column
def _row_value(row: dict[str, Any], column: str, *, casefold: bool) -> object:
if column in row:
return row[column]
if not casefold:
return ""
expected = column.casefold()
for key, value in row.items():
if key.casefold() == expected:
return value
return ""
def _import_values(value: object) -> list[str]:
raw_values = value if isinstance(value, list) else [value]
return [str(item).strip() for item in raw_values if item is not None and str(item).strip()]
def _first_import_value(value: object) -> str:
values = _import_values(value)
return values[0] if values else ""
def _plan_rows(
session: Session,
*,
book_id: str,
profile: AddressImportProfile,
input_hash: str,
rows: list[tuple[int, dict[str, str]]],
rows: list[tuple[int, dict[str, Any]]],
config: AddressImportConfiguration,
source_format: str,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
diagnostics: list[dict[str, Any]] = []
plan: list[dict[str, Any]] = []
headers = set(rows[0][1]) if rows else set()
referenced_columns = set(config.field_mappings.values())
casefold_columns = source_format == "ldif"
headers = {
_column_key(column, casefold=casefold_columns)
for column in (rows[0][1] if rows else set())
if not str(column).startswith("__")
}
referenced_columns = {
_column_key(column, casefold=casefold_columns)
for column in config.field_mappings.values()
}
key_column = config.source_key_column or config.field_mappings["source_key"]
normalized_key_column = _column_key(key_column, casefold=casefold_columns)
if config.source_key_column:
referenced_columns.add(config.source_key_column)
referenced_columns.add(_column_key(config.source_key_column, casefold=casefold_columns))
missing_columns = sorted(referenced_columns.difference(headers))
for column in missing_columns:
diagnostics.append(_diagnostic("error", "missing_column", f'Configured column "{column}" is missing.', field=column))
if missing_columns:
optional_ldif_attribute = casefold_columns and column != normalized_key_column
diagnostics.append(
_diagnostic(
"warning" if optional_ldif_attribute else "error",
"missing_attribute" if optional_ldif_attribute else "missing_column",
(
f'Configured LDIF attribute "{column}" is absent from this file.'
if optional_ldif_attribute
else f'Configured column "{column}" is missing.'
),
field=column,
)
)
if normalized_key_column in missing_columns or (missing_columns and not casefold_columns):
return [], diagnostics
key_column = config.source_key_column or config.field_mappings["source_key"]
keyed_rows: list[tuple[int, dict[str, str], str]] = []
keyed_rows: list[tuple[int, dict[str, Any], str]] = []
key_counts: Counter[str] = Counter()
for row_number, row in rows:
key = row.get(key_column, "").strip()
key = _first_import_value(_row_value(row, key_column, casefold=casefold_columns)).strip()
if not key:
diagnostics.append(_diagnostic("error", "missing_source_key", "Stable source key is blank.", row_number=row_number, field=key_column))
plan.append(_plan_effect(row_number, "conflict", source_key=None, message="Stable source key is blank."))
@@ -649,11 +713,23 @@ def _plan_rows(
plan.append(_plan_effect(row_number, "ignored", source_key=key, message="Duplicate row ignored by profile policy."))
continue
mapped, row_diagnostics = _mapped_fields(row_number, row, config=config)
mapped, row_diagnostics = _mapped_fields(
row_number,
row,
config=config,
casefold_columns=casefold_columns,
)
diagnostics.extend(row_diagnostics)
source_ref = f"import:{profile.profile_key}:{key}"
existing = _contact_by_source_ref(session, book_id, source_ref)
payload = _payload_from_mapped(mapped, profile=profile, input_hash=input_hash, row_number=row_number, source_key=key)
payload = _payload_from_mapped(
mapped,
profile=profile,
input_hash=input_hash,
row_number=row_number,
source_key=key,
source_row=row,
)
display_name = payload.get("display_name") or payload.get("email") or key
if any(item["severity"] == "error" for item in row_diagnostics):
plan.append(_plan_effect(row_number, "conflict", source_key=key, display_name=display_name, source_ref=source_ref, payload=payload, message="Row validation failed."))
@@ -687,23 +763,46 @@ def _plan_rows(
def _mapped_fields(
row_number: int,
row: dict[str, str],
row: dict[str, Any],
*,
config: AddressImportConfiguration,
casefold_columns: bool = False,
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
mapped: dict[str, Any] = {}
diagnostics: list[dict[str, Any]] = []
for target, column in config.field_mappings.items():
if target == "source_key":
continue
value = row.get(column, "").strip()
if not value:
values = _import_values(_row_value(row, column, casefold=casefold_columns))
if not values:
if config.blank_value_policy == "reject":
diagnostics.append(_diagnostic("error", "blank_value", f'Column "{column}" is blank.', row_number=row_number, field=target))
elif config.blank_value_policy == "clear":
mapped[target] = [] if target == "tags" else None
continue
mapped[target] = [item.strip() for item in value.split(",") if item.strip()] if target == "tags" else value
if target == "tags":
mapped[target] = list(
dict.fromkeys(
item.strip()
for value in values
for item in value.split(",")
if item.strip()
)
)
elif target in {"email", "phone"}:
mapped[target] = values if len(values) > 1 else values[0]
else:
mapped[target] = values[0]
if len(values) > 1:
diagnostics.append(
_diagnostic(
"warning",
"multiple_values_truncated",
f'Attribute "{column}" has multiple values; only the first maps to {target!r}.',
row_number=row_number,
field=target,
)
)
if config.default_tags:
mapped["tags"] = list(dict.fromkeys([*(mapped.get("tags") or []), *config.default_tags]))
if not any(mapped.get(name) for name in ("display_name", "given_name", "family_name", "email", "organization")):
@@ -718,8 +817,22 @@ def _payload_from_mapped(
input_hash: str,
row_number: int,
source_key: str,
source_row: dict[str, Any] | None = None,
) -> dict[str, Any]:
display_name = mapped.get("display_name") or " ".join(filter(None, [mapped.get("given_name"), mapped.get("family_name")])) or mapped.get("email") or mapped.get("organization")
display_name = (
_first_import_value(mapped.get("display_name"))
or " ".join(
filter(
None,
[
_first_import_value(mapped.get("given_name")),
_first_import_value(mapped.get("family_name")),
],
)
)
or _first_import_value(mapped.get("email"))
or _first_import_value(mapped.get("organization"))
)
payload: dict[str, Any] = {
key: mapped.get(key)
for key in ("given_name", "family_name", "organization", "role_title", "note", "tags")
@@ -727,9 +840,17 @@ def _payload_from_mapped(
}
payload["display_name"] = display_name
if "email" in mapped:
payload["emails"] = [] if mapped["email"] is None else [ContactEmailPayload(email=mapped["email"], is_primary=True).model_dump(mode="json")]
email_values = _import_values(mapped["email"])
payload["emails"] = [
ContactEmailPayload(email=value, is_primary=index == 0).model_dump(mode="json")
for index, value in enumerate(email_values)
]
if "phone" in mapped:
payload["phones"] = [] if mapped["phone"] is None else [ContactPhonePayload(phone=mapped["phone"], is_primary=True).model_dump(mode="json")]
phone_values = _import_values(mapped["phone"])
payload["phones"] = [
ContactPhonePayload(phone=value, is_primary=index == 0).model_dump(mode="json")
for index, value in enumerate(phone_values)
]
postal_keys = {"street", "postal_code", "locality", "region", "country"}
if postal_keys.intersection(mapped):
postal = {key: mapped.get(key) for key in postal_keys if key in mapped}
@@ -744,6 +865,8 @@ def _payload_from_mapped(
"source_key": source_key,
"locale": profile.configuration.get("locale"),
"visibility": mapped.get("visibility"),
"source_format": profile.source_format,
"source_record_hash": (source_row or {}).get("__ldif_record_hash"),
}
}
return ContactCreateRequest.model_validate(payload).model_dump(
@@ -768,7 +891,19 @@ def _changed_fields(contact: Contact, mapped: dict[str, Any]) -> list[str]:
if contact.postal_addresses:
postal = contact.postal_addresses[0]
current.update({key: getattr(postal, key) for key in ("street", "postal_code", "locality", "region", "country")})
return sorted(key for key, value in mapped.items() if key != "visibility" and current.get(key) != value)
changed: list[str] = []
for key, value in mapped.items():
if key == "visibility":
continue
if key == "email" and isinstance(value, list):
current_value: Any = [item.email for item in contact.emails]
elif key == "phone" and isinstance(value, list):
current_value = [item.phone for item in contact.phones]
else:
current_value = current.get(key)
if current_value != value:
changed.append(key)
return sorted(changed)
def _plan_effect(
+286
View File
@@ -0,0 +1,286 @@
from __future__ import annotations
import base64
import binascii
import hashlib
from collections.abc import Iterable
from io import BytesIO
from typing import Any, Literal
LdifChangeRecordPolicy = Literal["reject", "ignore", "treat_add_as_entry"]
MAX_LDIF_ATTRIBUTES = 500
MAX_LDIF_VALUES_PER_ATTRIBUTE = 100
MAX_LDIF_LOGICAL_LINE_BYTES = 1_000_000
def parse_ldif_rows(
raw: bytes,
*,
max_entries: int,
change_record_policy: LdifChangeRecordPolicy = "reject",
) -> tuple[list[tuple[int, dict[str, Any]]], list[dict[str, Any]]]:
"""Parse bounded LDIF entries without fetching URL or decoding binary values."""
rows: list[tuple[int, dict[str, Any]]] = []
diagnostics: list[dict[str, Any]] = []
record_lines: list[tuple[int, bytes]] = []
def finish_record() -> None:
if not record_lines:
return
row_number = record_lines[0][0]
row, record_diagnostics = _parse_record(record_lines)
diagnostics.extend(record_diagnostics)
record_lines.clear()
if not row:
return
if set(row) == {"version"} and _first(row.get("version")) == "1":
return
change_type = _first(row.get("changetype")).casefold()
if change_type:
if change_record_policy == "ignore":
diagnostics.append(
_diagnostic(
"warning",
"ldif_change_record_ignored",
f"LDIF change record {change_type!r} was ignored by profile policy.",
row_number=row_number,
field="changetype",
)
)
return
if change_record_policy != "treat_add_as_entry" or change_type != "add":
diagnostics.append(
_diagnostic(
"error",
"ldif_change_record_rejected",
f"LDIF change record {change_type!r} is not permitted by the profile policy.",
row_number=row_number,
field="changetype",
)
)
return
diagnostics.append(
_diagnostic(
"info",
"ldif_add_record_imported",
"LDIF add change record is treated as a static contact entry by profile policy.",
row_number=row_number,
field="changetype",
)
)
row.pop("changetype", None)
row["__ldif_record_hash"] = hashlib.sha256(_canonical_record(row)).hexdigest()
rows.append((row_number, row))
if len(rows) > max_entries:
raise ValueError(f"LDIF exceeds the configured {max_entries}-entry limit.")
for line_number, logical_line in _logical_lines(raw):
if not logical_line:
finish_record()
continue
if logical_line.startswith(b"#"):
continue
record_lines.append((line_number, logical_line))
finish_record()
if not rows and not any(item["severity"] == "error" for item in diagnostics):
diagnostics.append(_diagnostic("warning", "ldif_no_entries", "No importable LDIF entries were found."))
return rows, diagnostics
def _logical_lines(raw: bytes) -> Iterable[tuple[int, bytes]]:
current: bytearray | None = None
start_line = 0
for line_number, physical_with_ending in enumerate(BytesIO(raw), start=1):
physical = physical_with_ending.rstrip(b"\r\n")
if physical.startswith(b" "):
if current is None:
yield line_number, b"!invalid-fold-without-preceding-line"
continue
current.extend(physical[1:])
if len(current) > MAX_LDIF_LOGICAL_LINE_BYTES:
raise ValueError(f"LDIF logical line starting at {start_line} exceeds the size limit.")
continue
if current is not None:
yield start_line, bytes(current)
current = bytearray(physical)
start_line = line_number
if len(current) > MAX_LDIF_LOGICAL_LINE_BYTES:
raise ValueError(f"LDIF logical line {line_number} exceeds the size limit.")
if current is not None:
yield start_line, bytes(current)
def _parse_record(
lines: list[tuple[int, bytes]],
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
row: dict[str, list[str]] = {}
diagnostics: list[dict[str, Any]] = []
for line_number, line in lines:
if line == b"!invalid-fold-without-preceding-line":
diagnostics.append(
_diagnostic(
"error",
"ldif_invalid_fold",
"LDIF continuation line has no preceding attribute.",
row_number=line_number,
)
)
continue
if b":" not in line:
diagnostics.append(
_diagnostic(
"error",
"ldif_invalid_line",
"LDIF line is missing the attribute separator.",
row_number=line_number,
)
)
continue
if line == b"-":
# Attribute-operation separators are meaningful only inside change
# records, whose enclosing policy is evaluated after the record.
continue
raw_name, raw_value = line.split(b":", 1)
try:
name_parts = [part.strip().casefold() for part in raw_name.decode("ascii").split(";")]
name = name_parts[0]
except UnicodeDecodeError:
name_parts = []
name = ""
if not name or any(character.isspace() for character in name):
diagnostics.append(
_diagnostic(
"error",
"ldif_invalid_attribute",
"LDIF attribute name is invalid.",
row_number=line_number,
)
)
continue
if "binary" in name_parts[1:]:
diagnostics.append(
_diagnostic(
"warning",
"ldif_binary_value_ignored",
f"Binary LDIF attribute {name!r} was ignored; binary data is never projected into contacts.",
row_number=line_number,
field=name,
)
)
continue
if name not in row and len(row) >= MAX_LDIF_ATTRIBUTES:
diagnostics.append(
_diagnostic(
"error",
"ldif_too_many_attributes",
f"LDIF entry exceeds the {MAX_LDIF_ATTRIBUTES}-attribute limit.",
row_number=line_number,
)
)
continue
value, value_diagnostic = _decode_value(raw_value, attribute=name, line_number=line_number)
if value_diagnostic is not None:
diagnostics.append(value_diagnostic)
if value is None:
continue
values = row.setdefault(name, [])
if len(values) >= MAX_LDIF_VALUES_PER_ATTRIBUTE:
diagnostics.append(
_diagnostic(
"error",
"ldif_too_many_values",
f"LDIF attribute {name!r} exceeds the value limit.",
row_number=line_number,
field=name,
)
)
continue
values.append(value)
return row, diagnostics
def _decode_value(
raw_value: bytes,
*,
attribute: str,
line_number: int,
) -> tuple[str | None, dict[str, Any] | None]:
if raw_value.startswith(b":"):
encoded = raw_value[1:].lstrip(b" ")
try:
decoded = base64.b64decode(encoded, validate=True)
except (binascii.Error, ValueError):
return None, _diagnostic(
"error",
"ldif_invalid_base64",
f"LDIF attribute {attribute!r} contains invalid base64.",
row_number=line_number,
field=attribute,
)
try:
return decoded.decode("utf-8"), None
except UnicodeDecodeError:
return None, _diagnostic(
"warning",
"ldif_binary_value_ignored",
f"Binary LDIF attribute {attribute!r} was ignored; binary data is never projected into contacts.",
row_number=line_number,
field=attribute,
)
if raw_value.startswith(b"<"):
return None, _diagnostic(
"warning",
"ldif_url_value_ignored",
f"External LDIF URL value for {attribute!r} was ignored; imports never fetch referenced content.",
row_number=line_number,
field=attribute,
)
value_bytes = raw_value[1:] if raw_value.startswith(b" ") else raw_value
try:
return value_bytes.decode("utf-8"), None
except UnicodeDecodeError:
return None, _diagnostic(
"error",
"ldif_invalid_utf8",
f"LDIF attribute {attribute!r} is not valid UTF-8.",
row_number=line_number,
field=attribute,
)
def _canonical_record(row: dict[str, Any]) -> bytes:
lines = []
for name in sorted(key for key in row if not key.startswith("__")):
values = row[name] if isinstance(row[name], list) else [row[name]]
lines.extend(f"{name}:{value}" for value in values)
return "\n".join(lines).encode("utf-8")
def _first(value: object) -> str:
if isinstance(value, list):
return str(value[0]) if value else ""
return str(value or "")
def _diagnostic(
severity: str,
code: str,
message: str,
*,
row_number: int | None = None,
field: str | None = None,
) -> dict[str, Any]:
return {
"severity": severity,
"code": code,
"message": message,
"row_number": row_number,
"field": field,
"details": {},
}
__all__ = ["LdifChangeRecordPolicy", "parse_ldif_rows"]
+603 -69
View File
@@ -1,5 +1,8 @@
from __future__ import annotations
from govoplan_core.core.modules import with_documentation_structured_translations
from govoplan_addresses.backend.german_structured_documentation import GERMAN_STRUCTURED_TRANSLATIONS
from dataclasses import replace
from pathlib import Path
@@ -11,12 +14,22 @@ from govoplan_addresses.backend.capabilities import (
CAPABILITY_ADDRESSES_RECIPIENT_SOURCE,
)
from govoplan_addresses.backend.db import models as addresses_models # noqa: F401 - populate address ORM metadata
from govoplan_core.core.access import CAPABILITY_AUTH_PERMISSION_EVALUATOR, CAPABILITY_AUTH_PRINCIPAL_RESOLVER
from govoplan_core.core.contact_points import CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION
from govoplan_core.core.module_guards import drop_table_retirement_provider, persistent_table_uninstall_guard
from govoplan_core.core.access import (
CAPABILITY_AUTH_PERMISSION_EVALUATOR,
CAPABILITY_AUTH_PRINCIPAL_RESOLVER,
)
from govoplan_core.core.contact_points import (
CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION,
)
from govoplan_core.core.module_guards import (
drop_table_retirement_provider,
persistent_table_uninstall_guard,
)
from govoplan_core.core.people import CAPABILITY_ADDRESSES_PEOPLE_SEARCH
from govoplan_core.core.distribution_lists import CAPABILITY_RECIPIENT_CHANNEL_FACTS
from govoplan_core.core.modules import (
CapabilityDocumentation,
DocumentationCondition,
DocumentationTopic,
FrontendModule,
FrontendRoute,
@@ -26,6 +39,7 @@ from govoplan_core.core.modules import (
ModuleManifest,
NavItem,
PermissionDefinition,
ProductAreaContribution,
RoleTemplate,
)
from govoplan_core.core.provider_governance import (
@@ -37,6 +51,7 @@ from govoplan_core.core.provider_governance import (
)
from govoplan_core.core.views import ViewSurface
from govoplan_core.db.base import Base
from govoplan_addresses.backend.dsar_provider import ADDRESSES_DSAR_CAPABILITY
from govoplan_addresses.backend.provider_state import (
CARDDAV_PROVIDER_ID,
LDAP_PROVIDER_ID,
@@ -45,6 +60,13 @@ from govoplan_addresses.backend.provider_state import (
)
def _addresses_dsar_provider(context: ModuleContext) -> object:
del context
from govoplan_addresses.backend.dsar_provider import AddressesDsarProvider
return AddressesDsarProvider()
_addresses_table_retirement_provider = drop_table_retirement_provider(
addresses_models.AddressImportRun,
addresses_models.AddressImportProfile,
@@ -76,10 +98,18 @@ def _addresses_retirement_provider(session: object | None, module_id: str):
return plan
def executor(execute_session: object, execute_module_id: str) -> None:
if not hasattr(execute_session, "get_bind") or not hasattr(execute_session, "query"):
raise RuntimeError("No database session is available for Addresses credential retirement.")
if inspect(execute_session.get_bind()).has_table(addresses_models.AddressSyncSource.__tablename__):
from govoplan_addresses.backend.service import audit_address_credentials_for_retirement
if not hasattr(execute_session, "get_bind") or not hasattr(
execute_session, "query"
):
raise RuntimeError(
"No database session is available for Addresses credential retirement."
)
if inspect(execute_session.get_bind()).has_table(
addresses_models.AddressSyncSource.__tablename__
):
from govoplan_addresses.backend.service import (
audit_address_credentials_for_retirement,
)
audit_address_credentials_for_retirement(execute_session)
base_executor(execute_session, execute_module_id)
@@ -109,21 +139,77 @@ def _permission(scope: str, label: str, description: str) -> PermissionDefinitio
PERMISSIONS = (
_permission("addresses:address_book:read", "View address books", "List address books visible to the current principal."),
_permission("addresses:address_book:write", "Manage address books", "Create and edit local address books."),
_permission("addresses:address_book:delete", "Delete address books", "Soft-delete local address books."),
_permission("addresses:address_book:admin", "Administer address books", "Manage system-scoped address books and future sync sources."),
_permission("addresses:address_list:read", "View address lists", "List reusable address lists and their entries."),
_permission("addresses:address_list:write", "Manage address lists", "Create and edit reusable address lists."),
_permission("addresses:address_list:delete", "Delete address lists", "Soft-delete reusable address lists."),
_permission("addresses:contact:read", "View contacts", "List and lookup contacts in visible address books."),
_permission("addresses:contact:write", "Manage contacts", "Create and edit local contacts."),
_permission("addresses:contact:delete", "Delete contacts", "Soft-delete local contacts."),
_permission("addresses:governance:read", "View communication governance", "Inspect effective-dated consent, suppression, and channel-preference facts."),
_permission("addresses:governance:write", "Manage communication governance", "Record and end consent, suppression, and channel-preference facts."),
_permission("addresses:sync:read", "View address sync", "Inspect address sync sources, conflicts, tombstones, and diagnostics."),
_permission("addresses:sync:write", "Manage address sync", "Bind address books to external sources and record sync state."),
_permission("addresses:sync:admin", "Administer address sync", "Administer address sync connectors and future destructive sync operations."),
_permission(
"addresses:address_book:read",
"View address books",
"List address books visible to the current principal.",
),
_permission(
"addresses:address_book:write",
"Manage address books",
"Create and edit local address books.",
),
_permission(
"addresses:address_book:delete",
"Delete address books",
"Soft-delete local address books.",
),
_permission(
"addresses:address_book:admin",
"Administer address books",
"Manage system-scoped address books and future sync sources.",
),
_permission(
"addresses:address_list:read",
"View address lists",
"List reusable address lists and their entries.",
),
_permission(
"addresses:address_list:write",
"Manage address lists",
"Create and edit reusable address lists.",
),
_permission(
"addresses:address_list:delete",
"Delete address lists",
"Soft-delete reusable address lists.",
),
_permission(
"addresses:contact:read",
"View contacts",
"List and lookup contacts in visible address books.",
),
_permission(
"addresses:contact:write", "Manage contacts", "Create and edit local contacts."
),
_permission(
"addresses:contact:delete", "Delete contacts", "Soft-delete local contacts."
),
_permission(
"addresses:governance:read",
"View communication governance",
"Inspect effective-dated consent, suppression, and channel-preference facts.",
),
_permission(
"addresses:governance:write",
"Manage communication governance",
"Record and end consent, suppression, and channel-preference facts.",
),
_permission(
"addresses:sync:read",
"View address sync",
"Inspect address sync sources, conflicts, tombstones, and diagnostics.",
),
_permission(
"addresses:sync:write",
"Manage address sync",
"Bind address books to external sources and record sync state.",
),
_permission(
"addresses:sync:admin",
"Administer address sync",
"Administer address sync connectors and future destructive sync operations.",
),
)
@@ -152,7 +238,13 @@ ROLE_TEMPLATES = (
slug="address_book_reader",
name="Address book reader",
description="Read visible address books and contacts.",
permissions=("addresses:address_book:read", "addresses:address_list:read", "addresses:contact:read", "addresses:governance:read", "addresses:sync:read"),
permissions=(
"addresses:address_book:read",
"addresses:address_list:read",
"addresses:contact:read",
"addresses:governance:read",
"addresses:sync:read",
),
),
)
@@ -171,15 +263,42 @@ def _tenant_summary(session, tenant_id: str) -> dict[str, int]:
)
return {
"address_books": session.query(AddressBook).filter(AddressBook.tenant_id == tenant_id, AddressBook.deleted_at.is_(None)).count(),
"address_lists": session.query(AddressList).filter(AddressList.tenant_id == tenant_id, AddressList.deleted_at.is_(None)).count(),
"contacts": session.query(Contact).filter(Contact.tenant_id == tenant_id, Contact.deleted_at.is_(None)).count(),
"active_contact_merges": session.query(ContactMergeRecord).filter(ContactMergeRecord.tenant_id == tenant_id, ContactMergeRecord.status == "active").count(),
"contact_quality_decisions": session.query(ContactPointQualityDecision).filter(ContactPointQualityDecision.tenant_id == tenant_id).count(),
"contact_point_snapshots": session.query(ContactPointSnapshot).filter(ContactPointSnapshot.tenant_id == tenant_id).count(),
"sync_sources": session.query(AddressSyncSource).filter(AddressSyncSource.tenant_id == tenant_id, AddressSyncSource.enabled.is_(True)).count(),
"address_import_profiles": session.query(AddressImportProfile).filter(AddressImportProfile.tenant_id == tenant_id, AddressImportProfile.is_current.is_(True)).count(),
"address_import_runs": session.query(AddressImportRun).filter(AddressImportRun.tenant_id == tenant_id).count(),
"address_books": session.query(AddressBook)
.filter(AddressBook.tenant_id == tenant_id, AddressBook.deleted_at.is_(None))
.count(),
"address_lists": session.query(AddressList)
.filter(AddressList.tenant_id == tenant_id, AddressList.deleted_at.is_(None))
.count(),
"contacts": session.query(Contact)
.filter(Contact.tenant_id == tenant_id, Contact.deleted_at.is_(None))
.count(),
"active_contact_merges": session.query(ContactMergeRecord)
.filter(
ContactMergeRecord.tenant_id == tenant_id,
ContactMergeRecord.status == "active",
)
.count(),
"contact_quality_decisions": session.query(ContactPointQualityDecision)
.filter(ContactPointQualityDecision.tenant_id == tenant_id)
.count(),
"contact_point_snapshots": session.query(ContactPointSnapshot)
.filter(ContactPointSnapshot.tenant_id == tenant_id)
.count(),
"sync_sources": session.query(AddressSyncSource)
.filter(
AddressSyncSource.tenant_id == tenant_id,
AddressSyncSource.enabled.is_(True),
)
.count(),
"address_import_profiles": session.query(AddressImportProfile)
.filter(
AddressImportProfile.tenant_id == tenant_id,
AddressImportProfile.is_current.is_(True),
)
.count(),
"address_import_runs": session.query(AddressImportRun)
.filter(AddressImportRun.tenant_id == tenant_id)
.count(),
}
@@ -199,13 +318,28 @@ CARDDAV_PROVIDER = ExternalProviderDeclaration(
ProviderObjectDeclaration(
object_type="address_book",
field_groups=("identity", "display", "sync_state"),
authority_modes=("external_authoritative", "external_mirror", "governed_sync"),
authority_modes=(
"external_authoritative",
"external_mirror",
"governed_sync",
),
default_authority_mode="external_mirror",
),
ProviderObjectDeclaration(
object_type="contact",
field_groups=("identity", "name", "postal", "email", "phone", "source_metadata"),
authority_modes=("external_authoritative", "external_mirror", "governed_sync"),
field_groups=(
"identity",
"name",
"postal",
"email",
"phone",
"source_metadata",
),
authority_modes=(
"external_authoritative",
"external_mirror",
"governed_sync",
),
default_authority_mode="governed_sync",
),
),
@@ -251,7 +385,15 @@ LDAP_PROVIDER = ExternalProviderDeclaration(
objects=(
ProviderObjectDeclaration(
object_type="contact",
field_groups=("identity", "name", "organization", "postal", "email", "phone", "source_metadata"),
field_groups=(
"identity",
"name",
"organization",
"postal",
"email",
"phone",
"source_metadata",
),
authority_modes=("external_authoritative", "external_mirror"),
default_authority_mode="external_authoritative",
),
@@ -280,7 +422,11 @@ LDAP_PROVIDER = ExternalProviderDeclaration(
reconciliation="Only a complete paged search may infer an absent source object and create a local tombstone.",
outage="Existing contacts remain available and visibly stale; an unavailable directory never causes deletes.",
classifications=("personal", "confidential", "restricted"),
purposes=("directory projection", "recipient resolution", "identity-linked contact discovery"),
purposes=(
"directory projection",
"recipient resolution",
"identity-linked contact discovery",
),
retention="Address, audit, and records policies govern local projections and tombstone evidence.",
secret_handling="Bind secrets remain in reusable credential envelopes; URLs, previews, and diagnostics contain no credentials.",
),
@@ -292,34 +438,129 @@ LDAP_PROVIDER = ExternalProviderDeclaration(
manifest = ModuleManifest(
id="addresses",
name="Addresses",
version="0.1.16",
required_capabilities=(CAPABILITY_AUTH_PRINCIPAL_RESOLVER, CAPABILITY_AUTH_PERMISSION_EVALUATOR),
optional_dependencies=("campaigns", "mail", "forms", "reporting", "portal", "postbox", "connectors"),
version="0.1.20",
required_capabilities=(
CAPABILITY_AUTH_PRINCIPAL_RESOLVER,
CAPABILITY_AUTH_PERMISSION_EVALUATOR,
),
optional_dependencies=(
"campaigns",
"mail",
"forms",
"reporting",
"portal",
"postbox",
"connectors",
),
provides_interfaces=(
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_LOOKUP, version="0.1.8"),
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_PEOPLE_SEARCH, version="0.1.0"),
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_RECIPIENT_SOURCE, version="0.1.9"),
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION, version="1.0.0"),
ModuleInterfaceProvider(name=CAPABILITY_ADDRESSES_CONTACT_WRITER, version="0.1.8"),
ModuleInterfaceProvider(name=CAPABILITY_RECIPIENT_CHANNEL_FACTS, version="0.1.0"),
ModuleInterfaceProvider(
name=CAPABILITY_ADDRESSES_PEOPLE_SEARCH, version="0.1.0"
),
ModuleInterfaceProvider(
name=CAPABILITY_ADDRESSES_RECIPIENT_SOURCE, version="0.1.9"
),
ModuleInterfaceProvider(
name=CAPABILITY_ADDRESSES_CONTACT_POINT_RESOLUTION, version="1.0.0"
),
ModuleInterfaceProvider(
name=CAPABILITY_ADDRESSES_CONTACT_WRITER, version="0.1.8"
),
ModuleInterfaceProvider(
name=CAPABILITY_RECIPIENT_CHANNEL_FACTS, version="0.1.0"
),
ModuleInterfaceProvider(name=ADDRESSES_DSAR_CAPABILITY, version="0.1.0"),
),
permissions=PERMISSIONS,
route_factory=_addresses_router,
role_templates=ROLE_TEMPLATES,
tenant_summary_providers=(_tenant_summary,),
nav_items=(NavItem(path="/address-book", label="Address Book", icon="book-user", required_any=("addresses:contact:read",), order=80),),
nav_items=(
NavItem(
path="/address-book",
label="Address Book",
icon="book-user",
required_any=("addresses:contact:read",),
order=80,
),
),
frontend=FrontendModule(
module_id="addresses",
package_name="@govoplan/addresses-webui",
routes=(FrontendRoute(path="/address-book", component="AddressBookPage", required_any=("addresses:contact:read",), order=80),),
nav_items=(NavItem(path="/address-book", label="Address Book", icon="book-user", required_any=("addresses:contact:read",), order=80),),
routes=(
FrontendRoute(
path="/address-book",
component="AddressBookPage",
required_any=("addresses:contact:read",),
order=80,
),
),
nav_items=(
NavItem(
path="/address-book",
label="Address Book",
icon="book-user",
required_any=("addresses:contact:read",),
order=80,
),
),
product_areas=(
ProductAreaContribution(
id="people-responsibility",
module_id="addresses",
label="i18n:govoplan-core.product_area.people_responsibility",
icon="users",
description="i18n:govoplan-core.product_area.people_responsibility_description",
surface_ids=(
"addresses.nav.address.book",
"addresses.route.address.book",
),
order=70,
),
),
view_surfaces=(
ViewSurface(id="addresses.page", module_id="addresses", kind="route", label="Address Book", order=80),
ViewSurface(id="addresses.sources", module_id="addresses", kind="section", label="Address sources", order=10),
ViewSurface(id="addresses.contacts", module_id="addresses", kind="section", label="Contacts", order=20),
ViewSurface(id="addresses.detail", module_id="addresses", kind="section", label="Contact detail", order=30),
ViewSurface(id="addresses.governance", module_id="addresses", kind="action", label="Communication governance", order=40),
ViewSurface(id="addresses.sync", module_id="addresses", kind="action", label="Address synchronization", order=50),
ViewSurface(
id="addresses.page",
module_id="addresses",
kind="route",
label="Address Book",
order=80,
),
ViewSurface(
id="addresses.sources",
module_id="addresses",
kind="section",
label="Address sources",
order=10,
),
ViewSurface(
id="addresses.contacts",
module_id="addresses",
kind="section",
label="Contacts",
order=20,
),
ViewSurface(
id="addresses.detail",
module_id="addresses",
kind="section",
label="Contact detail",
order=30,
),
ViewSurface(
id="addresses.governance",
module_id="addresses",
kind="action",
label="Communication governance",
order=40,
),
ViewSurface(
id="addresses.sync",
module_id="addresses",
kind="action",
label="Address synchronization",
order=50,
),
),
),
migration_spec=MigrationSpec(
@@ -331,8 +572,13 @@ manifest = ModuleManifest(
retirement_notes="Destructive retirement drops address-owned database tables after the installer captures a database snapshot.",
),
capability_factories={
CAPABILITY_ADDRESSES_LOOKUP: lambda context: __import__("govoplan_addresses.backend.capabilities", fromlist=["lookup_capability"]).lookup_capability(context),
CAPABILITY_ADDRESSES_PEOPLE_SEARCH: lambda context: __import__("govoplan_addresses.backend.capabilities", fromlist=["people_search_capability"]).people_search_capability(context),
CAPABILITY_ADDRESSES_LOOKUP: lambda context: __import__(
"govoplan_addresses.backend.capabilities", fromlist=["lookup_capability"]
).lookup_capability(context),
CAPABILITY_ADDRESSES_PEOPLE_SEARCH: lambda context: __import__(
"govoplan_addresses.backend.capabilities",
fromlist=["people_search_capability"],
).people_search_capability(context),
CAPABILITY_ADDRESSES_RECIPIENT_SOURCE: lambda context: __import__(
"govoplan_addresses.backend.capabilities",
fromlist=["recipient_source_capability"],
@@ -349,6 +595,20 @@ manifest = ModuleManifest(
"govoplan_addresses.backend.capabilities",
fromlist=["contact_point_resolution_capability"],
).contact_point_resolution_capability(context),
ADDRESSES_DSAR_CAPABILITY: _addresses_dsar_provider,
},
capability_documentation={
ADDRESSES_DSAR_CAPABILITY: CapabilityDocumentation(
label="Addresses data-subject request provider",
summary=(
"Finds bounded contact, contact-point, address-list, governance, "
"provenance, synchronization, and operator-attribution data without "
"exporting raw source payloads, connector state, or opaque evidence."
),
contract_version="0.1.0",
documentation_types=("admin",),
audience=("privacy_officer", "addresses_admin", "records_manager"),
),
},
uninstall_guard_providers=(
persistent_table_uninstall_guard(
@@ -375,6 +635,80 @@ manifest = ModuleManifest(
),
),
documentation=(
DocumentationTopic(
id="addresses.privacy.data-subject-requests",
title="Review Addresses data in a data-subject request",
summary=(
"Collect tenant-scoped contact data while preserving shared address, "
"recipient, synchronization, and provenance evidence."
),
body=(
"Addresses searches corroborated email and account selectors plus "
"namespaced contact and contact-point references. A matching contact "
"exports bounded identity, email, telephone, and postal values together "
"with its address-list use and minimized governance, quality, provenance, "
"merge, redirect, and synchronization evidence. Account matches add only "
"minimized operator attribution for governed configuration and evidence. "
"The provider excludes raw imported or synchronized source payloads, "
"connector tokens and revisions that could act as credentials, opaque "
"metadata, snapshot request and resolution payloads, import plans, merge "
"before/after payloads, unrelated contacts, and other tenants. Quality, "
"governance, provenance, merge, redirect, synchronization, import, "
"snapshot, and operator evidence is retained with an explicit reason. "
"Because reusable contacts can be shared, synchronized, merged, or "
"referenced by immutable recipient snapshots, the DSAR provider never "
"deletes them automatically. An authorized operator must review "
"dependencies and use the normal Addresses correction, archive, source, "
"merge, or governance workflow."
),
layer="static",
documentation_types=("admin",),
audience=(
"privacy_officer",
"addresses_admin",
"records_manager",
"operator",
),
related_modules=(
"access",
"audit",
"campaigns",
"dist_lists",
"records",
),
order=29,
translations={
"de": {
"title": "Addresses-Daten in einer Betroffenenanfrage prüfen",
"summary": (
"Mandantenbezogene Kontaktdaten erfassen und dabei gemeinsame Adress-, Empfänger-, Synchronisations- und "
"Herkunftsnachweise bewahren."
),
"body": (
"Addresses durchsucht bestätigte E-Mail- und Kontoselektoren sowie namensraumgebundene Kontakt- und "
"Kontaktpunktverweise. Zu einem passenden Kontakt werden begrenzte Identitäts-, E-Mail-, Telefon- und Postwerte "
"einschließlich seiner Adresslistennutzung und minimierter Nachweise zu Governance, Qualität, Herkunft, Zusammenführung, "
"Weiterleitung und Synchronisation exportiert. Kontotreffer ergänzen nur minimierte Zuordnungen von Betriebspersonen zu "
"gesteuerter Konfiguration und Nachweisen. Ausgeschlossen sind rohe importierte oder synchronisierte Quelldaten, "
"Connector-Token und Revisionen mit Zugangsdatencharakter, undurchsichtige Metadaten, Anfrage- und Auflösungsnutzdaten "
"von Snapshots, Importpläne, Vorher-/Nachher-Daten von Zusammenführungen, unbeteiligte Kontakte und andere Mandanten. "
"Nachweise zu Qualität, Governance, Herkunft, Zusammenführung, Weiterleitung, Synchronisation, Import, Snapshot und "
"Betriebszuordnung werden mit ausdrücklicher Begründung aufbewahrt. Weil wiederverwendbare Kontakte geteilt, "
"synchronisiert, zusammengeführt oder von unveränderlichen Empfänger-Snapshots referenziert sein können, löscht der "
"DSAR-Provider sie niemals automatisch. Eine berechtigte Betriebsperson muss Abhängigkeiten prüfen und den regulären "
"Addresses-Ablauf für Korrektur, Archivierung, Quelle, Zusammenführung oder Governance verwenden."
),
}
},
metadata={
"seed": True,
"help_contexts": [
"addresses.contacts",
"addresses.governance",
"addresses.action.archive",
],
},
),
DocumentationTopic(
id="addresses.boundary",
title="Reusable address ownership",
@@ -387,8 +721,29 @@ manifest = ModuleManifest(
layer="configured",
documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin"),
related_modules=("campaigns", "mail", "forms", "reporting", "portal", "postbox"),
related_modules=(
"campaigns",
"mail",
"forms",
"reporting",
"portal",
"postbox",
),
order=30,
translations={
"de": {
"title": "Zuständigkeit für wiederverwendbare Adressen",
"summary": (
"Wiederverwendbare Personen-, Organisations-, Haushalts-, Post- und E-Mail-Empfängerquellen gehören dem "
"Addresses-Modul."
),
"body": (
"Campaigns darf unveränderliche campaignlokale Empfänger-Snapshots halten. Dauerhafte Adressverzeichnisse, "
"Empfängerquellendefinitionen, Einwilligungsmetadaten, Herkunft, Dublettenbereinigung sowie Import- und Exportabläufe "
"gehören jedoch govoplan-addresses."
),
}
},
metadata={
"seed": True,
"help_contexts": [
@@ -415,6 +770,18 @@ manifest = ModuleManifest(
audience=("tenant_admin", "operator", "module_admin"),
related_modules=("dist_lists", "campaigns", "policy", "templates"),
order=31,
translations={
"de": {
"title": "Kontaktpunktauflösung und Snapshots",
"summary": "Zweckbezogene Kanalziele auflösen und unveränderliche Empfängernachweise einfrieren.",
"body": (
"Addresses stellt eine versionierte Kontaktpunktfähigkeit für E-Mail-, Post-, Hauspost- und Portalziele bereit. "
"Aufrufende können Wirksamkeitsdatum, Kommunikationszweck, Adresszweck, Rückfallregel, Spracheinstellung und Postformat "
"angeben. Begrenzte Vorschauen bleiben aktuell; eingefrorene Snapshots bewahren aufgelöste Werte, Ausschlüsse, Quellen- "
"und Governance-Revisionen, Herkunft und einen deterministischen Nachweishash auch nach späteren Kontaktänderungen."
),
}
},
metadata={
"seed": True,
"help_contexts": [
@@ -428,20 +795,111 @@ manifest = ModuleManifest(
),
DocumentationTopic(
id="addresses.tabular-imports",
title="CSV and XLSX contact imports",
summary="Preview and apply reusable, versioned contact mappings without silent row loss.",
title="CSV, XLSX, and LDIF contact imports",
summary="Preview and apply reusable, versioned contact mappings without silent row or entry loss.",
body=(
"CSV and XLSX files can be mapped with scoped, reusable profile versions. Each preview validates headers, "
"encodings, source keys, duplicates, blank values, workbook limits, and contact identity before any mutation. "
"CSV, XLSX, and LDIF files can be mapped with scoped, reusable profile versions. Each preview validates headers or attributes, "
"encodings, source keys, duplicates, blank values, format limits, and contact identity before any mutation. "
"The reviewed input hash and plan hash are retained with row-level effects and diagnostics. Apply is idempotent, "
"rejects contacts changed after preview, and records sufficient evidence for a guarded rollback. XLSX formulas, "
"macros, and legacy workbook formats are never executed or imported."
"rejects contacts changed after preview, and records sufficient evidence for a guarded rollback. A persisted run "
"can be reopened with its run link after navigation or reload; previewed, applied, rolled-back, expired, and "
"unavailable states remain explicit. Both apply and rollback submit the reviewed plan hash. Missing, expired, "
"hidden, and cross-tenant runs disclose no source payload. XLSX formulas, macros, and legacy workbook formats "
"are never executed or imported. LDIF folded lines, UTF-8 and base64 text, repeated attributes, and comments are parsed; "
"binary and URL values are never projected or fetched. Change records default to rejected diagnostics and may only be ignored "
"or treat add records as static entries through an explicit profile policy."
),
layer="configured",
documentation_types=("admin", "user"),
audience=("tenant_admin", "module_admin", "power_user"),
conditions=(
DocumentationCondition(
required_modules=("addresses",),
any_scopes=("addresses:contact:write", "addresses:sync:write"),
),
),
related_modules=("connectors", "datasources", "dataflow", "files", "audit"),
order=33,
translations={
"de": {
"title": "Kontakte aus CSV, XLSX und LDIF importieren",
"summary": (
"Wiederverwendbare, versionierte Kontaktzuordnungen vorprüfen und anwenden, ohne Zeilen oder Einträge "
"stillschweigend zu verlieren."
),
"body": (
"CSV-, XLSX- und LDIF-Dateien lassen sich mit bereichsgebundenen, wiederverwendbaren Profilversionen zuordnen. Jede "
"Vorschau prüft Überschriften oder Attribute, Kodierung, Quellschlüssel, Dubletten, Leerwerte, Formatgrenzen und "
"Kontaktidentität vor jeder Änderung. Der geprüfte Eingabe- und Planhash wird mit zeilenbezogenen Wirkungen und Diagnosen "
"aufbewahrt. Die Anwendung ist idempotent, verwirft seit der Vorschau geänderte Kontakte und zeichnet ausreichende "
"Nachweise für eine gesicherte Rücknahme auf. Ein gespeicherter Lauf kann nach Navigation oder Neuladen über seinen Link "
"erneut geöffnet werden; Vorschau-, Anwendungs-, Rücknahme-, Ablauf- und Nichtverfügbarkeitszustände bleiben eindeutig. "
"Anwendung und Rücknahme übermitteln den geprüften Planhash. Fehlende, abgelaufene, verborgene und mandantenfremde Läufe "
"legen keine Quelldaten offen. XLSX-Formeln, Makros und ältere Arbeitsmappenformate werden niemals ausgeführt oder "
"importiert. Gefaltete LDIF-Zeilen, UTF-8- und Base64-Text, wiederholte Attribute und Kommentare werden verarbeitet; "
"Binär- und URL-Werte werden weder projiziert noch abgerufen. Änderungsdatensätze gelten standardmäßig als abgelehnte "
"Diagnose und dürfen nur über eine ausdrückliche Profilrichtlinie ignoriert oder bei Add-Einträgen als statische Daten "
"behandelt werden."
),
}
},
metadata={
"kind": "workflow",
"help_contexts": [
"addresses.action.import",
"addresses.contacts",
"addresses.sources",
],
},
),
DocumentationTopic(
id="addresses.vcard-batches",
title="Selective vCard batch import and export",
summary="Preview multiple vCard files, choose each card's effect, and export deterministic scoped files.",
body=(
"One or more UTF-8 .vcf files are parsed into a persisted, non-mutating preview with bounded diagnostics, "
"duplicate suggestions, an input hash, a parser version, and a deterministic plan hash. Operators choose "
"create, update, or ignore only where the reviewed plan permits it. Apply rejects stale contact targets and "
"is idempotent for the same selection; a different retry is rejected. Pending runs can be reloaded or cancelled "
"without changing contacts. Upload size, file count, card count, line count, and unfolded-line length are bounded. "
"Exports can target a complete address book, one address list, or explicit contacts; vCard 3.0 or 4.0 is selected "
"explicitly and contacts use deterministic display-name and stable-ID ordering. Export and import evidence records "
"hashes and counts, while diagnostics never disclose raw contact payloads. Large previews remain persisted and expose "
"their batch execution mode so a runtime job capability can execute them asynchronously when available."
),
layer="configured",
documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin", "power_user"),
related_modules=("files", "audit", "connectors"),
order=34,
translations={
"de": {
"title": "vCard-Stapel selektiv importieren und exportieren",
"summary": (
"Mehrere vCard-Dateien vorprüfen, die Wirkung jeder Karte wählen und deterministische bereichsgebundene Dateien exportieren."
),
"body": (
"Eine oder mehrere UTF-8-.vcf-Dateien werden in eine gespeicherte, nicht verändernde Vorschau mit begrenzten Diagnosen, "
"Dublettenhinweisen, Eingabehash, Parser-Version und deterministischem Planhash eingelesen. Betriebspersonen wählen "
"Anlegen, Aktualisieren oder Ignorieren nur dort, wo der geprüfte Plan es erlaubt. Die Anwendung verwirft veraltete "
"Kontaktziele und ist für dieselbe Auswahl idempotent; eine abweichende Wiederholung wird abgelehnt. Ausstehende Läufe "
"lassen sich neu laden oder abbrechen, ohne Kontakte zu verändern. Uploadgröße, Datei- und Kartenanzahl, Zeilenanzahl und "
"Länge entfalteter Zeilen sind begrenzt. Exporte können ein vollständiges Adressbuch, eine Adressliste oder ausgewählte "
"Kontakte umfassen; vCard 3.0 oder 4.0 wird ausdrücklich gewählt und Kontakte werden deterministisch nach Anzeigename und "
"stabiler Kennung sortiert. Export- und Importnachweise speichern Hashes und Anzahlen, während Diagnosen niemals rohe "
"Kontaktdaten offenlegen. Große Vorschauen bleiben gespeichert und geben ihren Stapelausführungsmodus an, sodass eine "
"Laufzeit-Jobfähigkeit sie bei Verfügbarkeit asynchron ausführen kann."
),
}
},
metadata={
"seed": True,
"help_contexts": [
"addresses.action.import",
"addresses.contacts",
"addresses.sources",
],
},
),
DocumentationTopic(
id="addresses.ldap-directory",
@@ -458,7 +916,23 @@ manifest = ModuleManifest(
documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin"),
related_modules=("connectors", "idm", "access", "policy", "audit"),
order=34,
order=35,
translations={
"de": {
"title": "LDAP- und Active-Directory-Adressquellen",
"summary": (
"Maßgebliche Verzeichniskontakte über eine begrenzte, schreibgeschützte Synchronisationsquelle projizieren."
),
"body": (
"LDAP-Quellen verwenden LDAPS oder StartTLS und wiederverwendbare Zugangsdatenhüllen. Die Ermittlung findet verfügbare "
"Basis-DNs; anschließend steuert das Quellprofil einen begrenzten seitenweisen Filter und eine ausdrückliche "
"Attributzuordnung. Die Vorschau verändert niemals Kontakte. Ein vollständiger erfolgreicher Lesevorgang darf lokale "
"Projektionen anlegen, aktualisieren oder als entfernt markieren; abgeschnittene oder fehlgeschlagene Lesevorgänge "
"unterdrücken Löschungen aufgrund von Abwesenheit und markieren die Quelle als veraltet. Stabile Quellschlüssel, "
"Revisionen, normalisierte Felder und Herkunft bleiben mit jedem erhaltenen Kontakt verbunden."
),
}
},
),
DocumentationTopic(
id="addresses.quality-and-merge",
@@ -478,6 +952,21 @@ manifest = ModuleManifest(
audience=("tenant_admin", "operator", "module_admin"),
related_modules=("campaigns", "dist_lists", "policy", "audit"),
order=32,
translations={
"de": {
"title": "Kontaktqualität, Dubletten und umkehrbare Zusammenführungen",
"summary": "Adressqualität und Dublettenvorschläge prüfen, ohne Quellnachweise zu verlieren.",
"body": (
"Addresses bewahrt ursprüngliche und normalisierte Kontaktpunktwerte, zeichnet die Herkunft je Feld auf und überführt "
"ungültige, zurückgesandte, veraltete oder unzustellbare Zustände mit stabilen Grundcodes in die Empfängerauflösung. "
"Dublettenvorschläge sind begrenzt und erklären ihre Übereinstimmungsmerkmale. Eine Betriebsperson kann die zu erhaltenden "
"Werte wählen, Kontaktpunkte zusammenführen und die Zusammenführung später rückgängig machen oder aufteilen, solange der "
"aufgezeichnete Nachweis nach der Zusammenführung noch übereinstimmt. Kontaktweiterleitungen halten gespeicherte Verweise "
"auflösbar, und Mitgliedschaften in Adresslisten werden transaktional repariert. Audit bleibt eine optionale Integration; "
"die Änderungsfolge und Zusammenführungsnachweise von Addresses werden stets aufbewahrt."
),
}
},
),
DocumentationTopic(
id="addresses.reference.fields-and-consequences",
@@ -495,9 +984,37 @@ manifest = ModuleManifest(
layer="configured",
documentation_types=("admin", "user"),
audience=("tenant_admin", "operator", "module_admin", "power_user"),
related_modules=("dist_lists", "connectors", "datasources", "campaigns", "policy", "audit"),
order=35,
related_modules=(
"dist_lists",
"connectors",
"datasources",
"campaigns",
"policy",
"audit",
),
order=36,
translations={
"de": {
"title": "Adressfelder, Geltungsbereiche und Folgen von Aktionen",
"summary": (
"Geltungsbereich, Quellenhoheit, Kontaktpunkte, Listenmitgliedschaft sowie Folgen von Archivierung, Synchronisation und "
"Zusammenführung."
),
"body": (
"Adressbücher sind einer Person, Gruppe, einem Mandanten oder einem berechtigten Systemkontext zugeordnet. Geerbte und "
"extern maßgebliche Bücher können sichtbar, aber schreibgeschützt bleiben. Kontakte besitzen wiederverwendbare Angaben "
"zu Name, Organisation, elektronischen und telefonischen Kontaktpunkten, Postanschrift, Schlagwörtern, Notizen, Qualität "
"und Herkunft. Adresslisten verweisen auf Kontaktpunkte desselben Buchs und ersetzen keine Distribution Lists. Eine "
"Archivierung entfernt Buch, Liste oder Kontakt aus der gewöhnlichen Auswahl, bewahrt aber gesteuerte Historie und "
"Verweise. CardDAV- und LDAP-Quellen zeigen Richtung, Hoheit, Aktualität, Diagnosen, Konflikte und Verhalten bei "
"veraltetem Zustand. Import und Synchronisation erfordern vor jeder Änderung eine Vorschau. Kontaktzusammenführungen "
"wählen überlebenden Kontakt und Feldherkunft, reparieren Listenverweise transaktional und bewahren Weiterleitungen und "
"Nachweise, sodass eine passende Zusammenführung rückgängig gemacht oder aufgeteilt werden kann."
),
}
},
metadata={
"kind": "reference",
"seed": True,
"help_contexts": [
"addresses.field.book-scope",
@@ -537,15 +1054,27 @@ manifest = ModuleManifest(
maturity="vertical_slice",
documentation_ref="docs/ADDRESS_MODULE_ARCHITECTURE.md",
test_ref="tests/test_addresses_service.py",
known_limits=("External address-book synchronization remains a bounded connector slice rather than a supported provider profile.",),
known_limits=(
"External address-book synchronization remains a bounded connector slice rather than a supported provider profile.",
),
supported_authority_modes=(
"native_authoritative",
"external_authoritative",
"external_mirror",
"governed_sync",
),
owned_concepts=("contact point", "address book", "contact consent", "recipient source"),
non_owned_concepts=("identity", "organization", "campaign recipient snapshot", "procedure party"),
owned_concepts=(
"contact point",
"address book",
"contact consent",
"recipient source",
),
non_owned_concepts=(
"identity",
"organization",
"campaign recipient snapshot",
"procedure party",
),
target_tested_providers=(CARDDAV_PROVIDER_ID,),
security_docs=("docs/ADDRESS_MODULE_ARCHITECTURE.md",),
operations_docs=("README.md",),
@@ -553,5 +1082,10 @@ manifest = ModuleManifest(
)
manifest = with_documentation_structured_translations(
manifest, locale="de", translations=GERMAN_STRUCTURED_TRANSLATIONS
)
def get_manifest() -> ModuleManifest:
return manifest
@@ -0,0 +1,34 @@
"""Allow profile-free persisted vCard batch runs.
Revision ID: d6e8f9a0b1c2
Revises: c5d7e8f9a0b1
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d6e8f9a0b1c2"
down_revision = "c5d7e8f9a0b1"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("addresses_import_runs") as batch:
batch.alter_column(
"profile_id",
existing_type=sa.String(length=36),
nullable=True,
)
def downgrade() -> None:
with op.batch_alter_table("addresses_import_runs") as batch:
batch.alter_column(
"profile_id",
existing_type=sa.String(length=36),
nullable=False,
)
+350 -77
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
from dataclasses import asdict
import json
import re
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
@@ -56,6 +57,23 @@ from govoplan_addresses.backend.imports import (
rollback_address_import,
update_import_profile,
)
from govoplan_addresses.backend.vcard_batch_schemas import (
VCardBatchCancelRequest,
VCardBatchCommitRequest,
VCardBatchPreviewRequest,
VCardBatchRunResponse,
VCardExportRequest,
VCardExportResponse,
)
from govoplan_addresses.backend.vcard_batches import (
apply_vcard_batch,
cancel_vcard_batch,
export_vcards,
get_vcard_batch_run,
preview_vcard_batch,
vcard_batch_payload,
vcard_diagnostics_payload,
)
from govoplan_addresses.backend.capabilities import (
AddressesContactPointResolutionCapability,
AddressesContactWriterCapability,
@@ -238,9 +256,7 @@ def _contact_response(
decision = quality.get((channel, point_id)) or quality.get((channel, None))
return {
"quality_state": decision.state if decision is not None else "valid",
"quality_reason_code": (
decision.reason_code if decision is not None else None
),
"quality_reason_code": (decision.reason_code if decision is not None else None),
}
return ContactResponse.model_validate(
@@ -323,9 +339,7 @@ def _contact_point_audit_details(
"postal": [item.id for item in contact.postal_addresses],
}
return {
f"{key_prefix}contact_point_counts": {
channel: len(ids) for channel, ids in point_ids.items()
},
f"{key_prefix}contact_point_counts": {channel: len(ids) for channel, ids in point_ids.items()},
f"{key_prefix}contact_point_ids": point_ids,
}
@@ -454,7 +468,9 @@ def _sync_source_response(sync_source: AddressSyncSource) -> AddressSyncSourceRe
)
def _sync_diagnostic_response(diagnostic: AddressSyncDiagnostic) -> AddressSyncDiagnosticResponse:
def _sync_diagnostic_response(
diagnostic: AddressSyncDiagnostic,
) -> AddressSyncDiagnosticResponse:
return AddressSyncDiagnosticResponse.model_validate(
{
"id": diagnostic.id,
@@ -470,7 +486,9 @@ def _sync_diagnostic_response(diagnostic: AddressSyncDiagnostic) -> AddressSyncD
)
def _sync_tombstone_response(tombstone: AddressSyncTombstone) -> AddressSyncTombstoneResponse:
def _sync_tombstone_response(
tombstone: AddressSyncTombstone,
) -> AddressSyncTombstoneResponse:
return AddressSyncTombstoneResponse.model_validate(
{
"id": tombstone.id,
@@ -490,7 +508,9 @@ def _sync_tombstone_response(tombstone: AddressSyncTombstone) -> AddressSyncTomb
)
def _sync_conflict_response(conflict: AddressSyncConflict) -> AddressSyncConflictResponse:
def _sync_conflict_response(
conflict: AddressSyncConflict,
) -> AddressSyncConflictResponse:
return AddressSyncConflictResponse.model_validate(
{
"id": conflict.id,
@@ -583,7 +603,11 @@ def api_list_address_books(
return AddressBookListResponse(address_books=[_book_response(book, contact_count=counts.get(book.id, 0)) for book in books])
@router.post("/address-books", response_model=AddressBookResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-books",
response_model=AddressBookResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_book(
payload: AddressBookCreateRequest,
principal: ApiPrincipal = Depends(get_api_principal),
@@ -591,7 +615,12 @@ def api_create_address_book(
):
_require_scope(principal, "addresses:address_book:write")
try:
book = create_address_book(session, principal, payload, allow_system=has_scope(principal, "addresses:address_book:admin"))
book = create_address_book(
session,
principal,
payload,
allow_system=has_scope(principal, "addresses:address_book:admin"),
)
session.commit()
session.refresh(book)
return _book_response(book)
@@ -738,10 +767,7 @@ def api_suggest_duplicate_contacts(
right=_contact_response(item.right),
score=item.score,
confidence=item.confidence,
features=[
ContactDuplicateFeatureResponse(**asdict(feature))
for feature in item.features
],
features=[ContactDuplicateFeatureResponse(**asdict(feature)) for feature in item.features],
)
for item in scan.suggestions
],
@@ -778,10 +804,7 @@ def api_address_quality_summary(
quality_counts=summary.quality_counts,
duplicate_suggestion_count=summary.duplicate_suggestion_count,
correction_count=summary.correction_count,
corrections=[
AddressQualityCorrectionResponse(**asdict(item))
for item in summary.corrections
],
corrections=[AddressQualityCorrectionResponse(**asdict(item)) for item in summary.corrections],
truncated=summary.truncated,
)
except AddressBookError as exc:
@@ -895,9 +918,7 @@ def api_resolve_contact_redirect(
):
_require_scope(principal, "addresses:contact:read")
try:
return ContactRedirectResponse.model_validate(
asdict(resolve_contact_redirect(session, principal, contact_id))
)
return ContactRedirectResponse.model_validate(asdict(resolve_contact_redirect(session, principal, contact_id)))
except AddressBookError as exc:
raise _error(exc) from exc
@@ -1167,16 +1188,23 @@ def api_list_address_lists(
):
_require_scope(principal, "addresses:address_list:read")
try:
address_lists = list_address_lists(session, principal, address_book_id=address_book_id, include_deleted=include_deleted)
counts = address_list_entry_counts(session, [address_list.id for address_list in address_lists])
return AddressListListResponse(
address_lists=[_address_list_response(address_list, entry_count=counts.get(address_list.id, 0)) for address_list in address_lists]
address_lists = list_address_lists(
session,
principal,
address_book_id=address_book_id,
include_deleted=include_deleted,
)
counts = address_list_entry_counts(session, [address_list.id for address_list in address_lists])
return AddressListListResponse(address_lists=[_address_list_response(address_list, entry_count=counts.get(address_list.id, 0)) for address_list in address_lists])
except AddressBookError as exc:
raise _error(exc) from exc
@router.post("/address-books/{book_id}/address-lists", response_model=AddressListResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-books/{book_id}/address-lists",
response_model=AddressListResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_list(
book_id: str,
payload: AddressListCreateRequest,
@@ -1247,7 +1275,10 @@ def api_restore_address_list(
raise _error(exc) from exc
@router.get("/address-lists/{address_list_id}/entries", response_model=AddressListEntryListResponse)
@router.get(
"/address-lists/{address_list_id}/entries",
response_model=AddressListEntryListResponse,
)
def api_list_address_list_entries(
address_list_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
@@ -1261,7 +1292,11 @@ def api_list_address_list_entries(
raise _error(exc) from exc
@router.post("/address-lists/{address_list_id}/entries", response_model=AddressListEntryResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-lists/{address_list_id}/entries",
response_model=AddressListEntryResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_list_entry(
address_list_id: str,
payload: AddressListEntryCreateRequest,
@@ -1332,7 +1367,10 @@ def api_list_write_targets(
return AddressBookWriteTargetsResponse(targets=[_write_decision_response(decision) for decision in decisions])
@router.get("/address-books/{book_id}/write-decision", response_model=AddressBookWriteDecisionResponse)
@router.get(
"/address-books/{book_id}/write-decision",
response_model=AddressBookWriteDecisionResponse,
)
def api_get_address_book_write_decision(
book_id: str,
operation: str = Query(default="create_contact"),
@@ -1378,9 +1416,7 @@ def api_discover_ldap_base_dns(
):
_require_scope(principal, "addresses:sync:write")
try:
return AddressLdapDiscoveryResponse(
base_dns=list(discover_ldap_base_dns(session, principal, payload))
)
return AddressLdapDiscoveryResponse(base_dns=list(discover_ldap_base_dns(session, principal, payload)))
except (AddressBookError, AddressLdapError) as exc:
raise _error(AddressBookError(str(exc))) from exc
@@ -1438,7 +1474,11 @@ def api_list_address_credentials(
)
@router.post("/address-books/{book_id}/carddav/sources", response_model=AddressSyncSourceResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-books/{book_id}/carddav/sources",
response_model=AddressSyncSourceResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_carddav_sync_source(
book_id: str,
payload: AddressCardDavSourceCreateRequest,
@@ -1454,7 +1494,11 @@ def api_create_carddav_sync_source(
action="addresses.sync_source_created",
object_type="address_sync_source",
object_id=sync_source.id,
details={"address_book_id": book_id, "connector_type": "carddav", "sync_direction": sync_source.sync_direction},
details={
"address_book_id": book_id,
"connector_type": "carddav",
"sync_direction": sync_source.sync_direction,
},
)
session.commit()
session.refresh(sync_source)
@@ -1473,7 +1517,12 @@ def api_list_sync_sources(
):
_require_scope(principal, "addresses:sync:read")
try:
sync_sources = list_sync_sources(session, principal, address_book_id=address_book_id, include_disabled=include_disabled)
sync_sources = list_sync_sources(
session,
principal,
address_book_id=address_book_id,
include_disabled=include_disabled,
)
return AddressSyncSourceListResponse(sync_sources=[_sync_source_response(sync_source) for sync_source in sync_sources])
except AddressBookError as exc:
raise _error(exc) from exc
@@ -1575,7 +1624,11 @@ def api_run_sync_source(
raise _error(AddressBookError(message)) from exc
@router.post("/address-books/{book_id}/sync-sources", response_model=AddressSyncSourceResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-books/{book_id}/sync-sources",
response_model=AddressSyncSourceResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_sync_source(
book_id: str,
payload: AddressSyncSourceCreateRequest,
@@ -1591,7 +1644,11 @@ def api_create_sync_source(
action="addresses.sync_source_created",
object_type="address_sync_source",
object_id=sync_source.id,
details={"address_book_id": book_id, "connector_type": sync_source.connector_type, "sync_direction": sync_source.sync_direction},
details={
"address_book_id": book_id,
"connector_type": sync_source.connector_type,
"sync_direction": sync_source.sync_direction,
},
)
session.commit()
session.refresh(sync_source)
@@ -1617,7 +1674,11 @@ def api_update_sync_source(
action="addresses.sync_source_updated",
object_type="address_sync_source",
object_id=sync_source.id,
details={"connector_type": sync_source.connector_type, "sync_direction": sync_source.sync_direction, "enabled": sync_source.enabled},
details={
"connector_type": sync_source.connector_type,
"sync_direction": sync_source.sync_direction,
"enabled": sync_source.enabled,
},
)
session.commit()
session.refresh(sync_source)
@@ -1651,7 +1712,10 @@ def api_delete_sync_source(
raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/attempts/start", response_model=AddressSyncSourceResponse)
@router.post(
"/sync-sources/{sync_source_id}/attempts/start",
response_model=AddressSyncSourceResponse,
)
def api_start_sync_attempt(
sync_source_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
@@ -1676,7 +1740,10 @@ def api_start_sync_attempt(
raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/attempts/finish", response_model=AddressSyncSourceResponse)
@router.post(
"/sync-sources/{sync_source_id}/attempts/finish",
response_model=AddressSyncSourceResponse,
)
def api_finish_sync_attempt(
sync_source_id: str,
payload: AddressSyncAttemptFinishRequest,
@@ -1692,7 +1759,11 @@ def api_finish_sync_attempt(
action="addresses.sync_finished",
object_type="address_sync_source",
object_id=sync_source.id,
details={"connector_type": sync_source.connector_type, "status": sync_source.status, "error": sync_source.last_error},
details={
"connector_type": sync_source.connector_type,
"status": sync_source.status,
"error": sync_source.last_error,
},
)
session.commit()
session.refresh(sync_source)
@@ -1702,7 +1773,10 @@ def api_finish_sync_attempt(
raise _error(exc) from exc
@router.get("/sync-sources/{sync_source_id}/diagnostics", response_model=AddressSyncDiagnosticListResponse)
@router.get(
"/sync-sources/{sync_source_id}/diagnostics",
response_model=AddressSyncDiagnosticListResponse,
)
def api_list_sync_diagnostics(
sync_source_id: str,
limit: int = Query(default=100, ge=1, le=500),
@@ -1717,7 +1791,11 @@ def api_list_sync_diagnostics(
raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/diagnostics", response_model=AddressSyncDiagnosticResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/sync-sources/{sync_source_id}/diagnostics",
response_model=AddressSyncDiagnosticResponse,
status_code=status.HTTP_201_CREATED,
)
def api_record_sync_diagnostic(
sync_source_id: str,
payload: AddressSyncDiagnosticCreateRequest,
@@ -1735,7 +1813,10 @@ def api_record_sync_diagnostic(
raise _error(exc) from exc
@router.get("/sync-sources/{sync_source_id}/tombstones", response_model=AddressSyncTombstoneListResponse)
@router.get(
"/sync-sources/{sync_source_id}/tombstones",
response_model=AddressSyncTombstoneListResponse,
)
def api_list_sync_tombstones(
sync_source_id: str,
limit: int = Query(default=200, ge=1, le=1000),
@@ -1750,7 +1831,11 @@ def api_list_sync_tombstones(
raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/tombstones", response_model=AddressSyncTombstoneResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/sync-sources/{sync_source_id}/tombstones",
response_model=AddressSyncTombstoneResponse,
status_code=status.HTTP_201_CREATED,
)
def api_record_sync_tombstone(
sync_source_id: str,
payload: AddressSyncTombstoneCreateRequest,
@@ -1768,7 +1853,10 @@ def api_record_sync_tombstone(
raise _error(exc) from exc
@router.get("/sync-sources/{sync_source_id}/conflicts", response_model=AddressSyncConflictListResponse)
@router.get(
"/sync-sources/{sync_source_id}/conflicts",
response_model=AddressSyncConflictListResponse,
)
def api_list_sync_conflicts(
sync_source_id: str,
status_filter: str | None = Query(default="open", alias="status"),
@@ -1784,7 +1872,11 @@ def api_list_sync_conflicts(
raise _error(exc) from exc
@router.post("/sync-sources/{sync_source_id}/conflicts", response_model=AddressSyncConflictResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/sync-sources/{sync_source_id}/conflicts",
response_model=AddressSyncConflictResponse,
status_code=status.HTTP_201_CREATED,
)
def api_record_sync_conflict(
sync_source_id: str,
payload: AddressSyncConflictCreateRequest,
@@ -1818,7 +1910,11 @@ def api_resolve_sync_conflict(
action="addresses.sync_conflict_resolved",
object_type="address_sync_conflict",
object_id=conflict.id,
details={"sync_source_id": conflict.sync_source_id, "status": conflict.status, "resolution": conflict.resolution},
details={
"sync_source_id": conflict.sync_source_id,
"status": conflict.status,
"resolution": conflict.resolution,
},
)
session.commit()
session.refresh(conflict)
@@ -1828,7 +1924,11 @@ def api_resolve_sync_conflict(
raise _error(exc) from exc
@router.post("/address-books/{book_id}/contacts", response_model=ContactResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-books/{book_id}/contacts",
response_model=ContactResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_contact(
book_id: str,
payload: ContactCreateRequest,
@@ -1869,11 +1969,7 @@ def api_update_contact(
_require_scope(principal, "addresses:contact:write")
try:
previous_contact = session.get(Contact, contact_id)
previous_point_details = (
_contact_point_audit_details(previous_contact, prefix="previous")
if previous_contact is not None
else {}
)
previous_point_details = _contact_point_audit_details(previous_contact, prefix="previous") if previous_contact is not None else {}
contact = update_contact(session, principal, contact_id, payload)
audit_from_principal(
session,
@@ -1897,7 +1993,10 @@ def api_update_contact(
raise _error(exc) from exc
@router.get("/contacts/{contact_id}/channel-rules", response_model=ContactChannelRuleListResponse)
@router.get(
"/contacts/{contact_id}/channel-rules",
response_model=ContactChannelRuleListResponse,
)
def api_list_contact_channel_rules(
contact_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
@@ -1905,12 +2004,7 @@ def api_list_contact_channel_rules(
):
_require_scope(principal, "addresses:governance:read")
try:
return ContactChannelRuleListResponse(
rules=[
_channel_rule_response(rule)
for rule in list_contact_channel_rules(session, principal, contact_id)
]
)
return ContactChannelRuleListResponse(rules=[_channel_rule_response(rule) for rule in list_contact_channel_rules(session, principal, contact_id)])
except AddressBookError as exc:
raise _error(exc) from exc
@@ -2044,7 +2138,11 @@ def api_restore_contact(
raise _error(exc) from exc
@router.post("/address-books/{book_id}/vcards/import", response_model=VCardImportResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/address-books/{book_id}/vcards/import",
response_model=VCardImportResponse,
status_code=status.HTTP_201_CREATED,
)
def api_import_address_book_vcards(
book_id: str,
payload: VCardImportRequest,
@@ -2091,6 +2189,136 @@ def api_import_address_book_vcards(
raise _error(AddressBookError(str(exc))) from exc
@router.post(
"/address-books/{book_id}/vcard-batches/preview",
response_model=VCardBatchRunResponse,
status_code=status.HTTP_201_CREATED,
)
def api_preview_vcard_batch(
book_id: str,
payload: VCardBatchPreviewRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:write")
try:
run = preview_vcard_batch(session, principal, book_id, payload)
audit_from_principal(
session,
principal,
action="addresses.vcard_batch_previewed",
object_type="address_import_run",
object_id=run.id,
details={
"address_book_id": book_id,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
)
session.commit()
session.refresh(run)
return VCardBatchRunResponse.model_validate(vcard_batch_payload(run))
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.get("/vcard-batches/{run_id}", response_model=VCardBatchRunResponse)
def api_get_vcard_batch(
run_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
try:
return VCardBatchRunResponse.model_validate(vcard_batch_payload(get_vcard_batch_run(session, principal, run_id)))
except AddressBookError as exc:
raise _error(exc) from exc
@router.post("/vcard-batches/{run_id}/apply", response_model=VCardBatchRunResponse)
def api_apply_vcard_batch(
run_id: str,
payload: VCardBatchCommitRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:write")
try:
run = apply_vcard_batch(session, principal, run_id, payload)
session.flush()
audit_from_principal(
session,
principal,
action="addresses.vcard_batch_applied",
object_type="address_import_run",
object_id=run.id,
details={
"address_book_id": run.address_book_id,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
)
session.commit()
session.refresh(run)
return VCardBatchRunResponse.model_validate(vcard_batch_payload(run))
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.post("/vcard-batches/{run_id}/cancel", response_model=VCardBatchRunResponse)
def api_cancel_vcard_batch(
run_id: str,
payload: VCardBatchCancelRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:write")
try:
run = cancel_vcard_batch(session, principal, run_id, payload)
audit_from_principal(
session,
principal,
action="addresses.vcard_batch_cancelled",
object_type="address_import_run",
object_id=run.id,
details={"address_book_id": run.address_book_id, "reason": payload.reason},
)
session.commit()
session.refresh(run)
return VCardBatchRunResponse.model_validate(vcard_batch_payload(run))
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.get("/vcard-batches/{run_id}/diagnostics")
def api_export_vcard_batch_diagnostics(
run_id: str,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
try:
run = get_vcard_batch_run(session, principal, run_id)
content = json.dumps(
vcard_diagnostics_payload(run),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
return Response(
content=content,
media_type="application/json",
headers={"Content-Disposition": f'attachment; filename="vcard-batch-{run.id}.json"'},
)
except AddressBookError as exc:
raise _error(exc) from exc
@router.get("/import-profiles", response_model=AddressImportProfileListResponse)
def api_list_address_import_profiles(
include_history: bool = Query(default=False),
@@ -2098,15 +2326,14 @@ def api_list_address_import_profiles(
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
return AddressImportProfileListResponse(
profiles=[
AddressImportProfileResponse.model_validate(item)
for item in list_import_profiles(session, principal, include_history=include_history)
]
)
return AddressImportProfileListResponse(profiles=[AddressImportProfileResponse.model_validate(item) for item in list_import_profiles(session, principal, include_history=include_history)])
@router.post("/import-profiles", response_model=AddressImportProfileResponse, status_code=status.HTTP_201_CREATED)
@router.post(
"/import-profiles",
response_model=AddressImportProfileResponse,
status_code=status.HTTP_201_CREATED,
)
def api_create_address_import_profile(
payload: AddressImportProfileCreateRequest,
principal: ApiPrincipal = Depends(get_api_principal),
@@ -2122,7 +2349,11 @@ def api_create_address_import_profile(
action="addresses.import_profile_created",
object_type="address_import_profile",
object_id=profile.profile_key,
details={"version": profile.version, "source_format": profile.source_format, "scope_type": profile.scope_type},
details={
"version": profile.version,
"source_format": profile.source_format,
"scope_type": profile.scope_type,
},
)
session.commit()
session.refresh(profile)
@@ -2149,7 +2380,10 @@ def api_update_address_import_profile(
action="addresses.import_profile_versioned",
object_type="address_import_profile",
object_id=profile.profile_key,
details={"version": profile.version, "source_format": profile.source_format},
details={
"version": profile.version,
"source_format": profile.source_format,
},
)
session.commit()
session.refresh(profile)
@@ -2202,7 +2436,11 @@ def api_preview_address_import(
action="addresses.import_previewed",
object_type="address_import_run",
object_id=run.id,
details={"input_hash": run.input_hash, "plan_hash": run.plan_hash, "statistics": run.statistics},
details={
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
)
session.commit()
session.refresh(run)
@@ -2220,9 +2458,7 @@ def api_get_address_import_run(
):
_require_scope(principal, "addresses:contact:read")
try:
return AddressImportRunResponse.model_validate(
import_run_payload(get_import_run(session, principal, run_id))
)
return AddressImportRunResponse.model_validate(import_run_payload(get_import_run(session, principal, run_id)))
except AddressBookError as exc:
raise _error(exc) from exc
@@ -2244,7 +2480,11 @@ def api_apply_address_import(
action="addresses.import_applied",
object_type="address_import_run",
object_id=run.id,
details={"input_hash": run.input_hash, "plan_hash": run.plan_hash, "statistics": run.statistics},
details={
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"statistics": run.statistics,
},
)
session.commit()
session.refresh(run)
@@ -2300,6 +2540,39 @@ def api_export_address_book_vcards(
raise _error(exc) from exc
@router.post(
"/address-books/{book_id}/vcards/export",
response_model=VCardExportResponse,
)
def api_export_selected_vcards(
book_id: str,
payload: VCardExportRequest,
principal: ApiPrincipal = Depends(get_api_principal),
session: Session = Depends(get_session),
):
_require_scope(principal, "addresses:contact:read")
try:
result = export_vcards(session, principal, book_id, payload)
audit_from_principal(
session,
principal,
action="addresses.vcards_exported",
object_type="address_book",
object_id=book_id,
details={
"scope": result["scope"],
"version": result["version"],
"contact_count": result["contact_count"],
"content_hash": result["content_hash"],
},
)
session.commit()
return VCardExportResponse.model_validate(result)
except AddressBookError as exc:
session.rollback()
raise _error(exc) from exc
@router.get("/contacts/{contact_id}/vcard")
def api_export_contact_vcard(
contact_id: str,
+83 -36
View File
@@ -16,6 +16,12 @@ class VCardError(ValueError):
pass
VCARD_PARSER_VERSION = "govoplan-vcard/2"
MAX_VCARD_CARDS = 10_000
MAX_VCARD_LINES = 200_000
MAX_VCARD_UNFOLDED_LINE_CHARS = 16_384
@dataclass(frozen=True, slots=True)
class ParsedVCard:
payload: ContactCreateRequest
@@ -60,12 +66,16 @@ class _VCardDraft:
def _normalize_lines(content: str) -> list[str]:
raw_lines = content.replace("\r\n", "\n").replace("\r", "\n").split("\n")
if len(raw_lines) > MAX_VCARD_LINES:
raise VCardError(f"vCard input exceeds the {MAX_VCARD_LINES}-line parser limit.")
lines: list[str] = []
for line in raw_lines:
if line.startswith((" ", "\t")) and lines:
lines[-1] += line[1:]
elif line:
lines.append(line)
if lines and len(lines[-1]) > MAX_VCARD_UNFOLDED_LINE_CHARS:
raise VCardError(f"vCard unfolded lines are limited to {MAX_VCARD_UNFOLDED_LINE_CHARS} characters.")
return lines
@@ -90,27 +100,13 @@ def _split_unescaped(value: str, separator: str) -> list[str]:
def _unescape_text(value: str) -> str:
return (
value.replace("\\n", "\n")
.replace("\\N", "\n")
.replace("\\,", ",")
.replace("\\;", ";")
.replace("\\\\", "\\")
.strip()
)
return value.replace("\\n", "\n").replace("\\N", "\n").replace("\\,", ",").replace("\\;", ";").replace("\\\\", "\\").strip()
def _escape_text(value: str | None) -> str:
if value is None:
return ""
return (
value.replace("\\", "\\\\")
.replace("\r\n", "\n")
.replace("\r", "\n")
.replace("\n", "\\n")
.replace(";", "\\;")
.replace(",", "\\,")
)
return value.replace("\\", "\\\\").replace("\r\n", "\n").replace("\r", "\n").replace("\n", "\\n").replace(";", "\\;").replace(",", "\\,")
def _parse_head(head: str) -> tuple[str, dict[str, list[str]]]:
@@ -154,7 +150,7 @@ def _is_pref(params: dict[str, list[str]]) -> bool:
def _card_blocks(content: str) -> list[list[str]]:
result = _card_blocks_with_issues(content)
result = _card_blocks_with_issues(content, max_cards=MAX_VCARD_CARDS)
if result.issues:
raise VCardError(result.issues[0].message)
return result.cards
@@ -166,8 +162,14 @@ class _CardBlockResult:
issues: list[ParsedVCardIssue]
def _card_blocks_with_issues(content: str) -> _CardBlockResult:
lines = _normalize_lines(content)
def _card_blocks_with_issues(content: str, *, max_cards: int) -> _CardBlockResult:
try:
lines = _normalize_lines(content)
except VCardError as exc:
return _CardBlockResult(
cards=[],
issues=[ParsedVCardIssue(index=0, message=str(exc))],
)
blocks: list[list[str]] = []
issues: list[ParsedVCardIssue] = []
current: list[str] | None = None
@@ -189,6 +191,15 @@ def _card_blocks_with_issues(content: str) -> _CardBlockResult:
issues.append(ParsedVCardIssue(index=card_index, message="vCard END appears before BEGIN."))
continue
current.append(line)
if len(blocks) >= max_cards:
issues.append(
ParsedVCardIssue(
index=len(blocks) + 1,
message=f"vCard input exceeds the configured {max_cards}-card limit.",
)
)
current = None
break
blocks.append(current)
current = None
elif current is not None:
@@ -258,7 +269,14 @@ def _apply_card_metadata(
if name == "VERSION":
draft.version = value.strip()
if draft.version and draft.version not in {"3.0", "4.0"}:
issues.append(ParsedVCardIssue(index=index, severity="warning", field="VERSION", message=f"vCard version {draft.version} is not fully supported."))
issues.append(
ParsedVCardIssue(
index=index,
severity="warning",
field="VERSION",
message=f"vCard version {draft.version} is not fully supported.",
)
)
return True
if name == "UID":
draft.uid = _unescape_text(value) or draft.uid
@@ -325,15 +343,34 @@ def _append_card_email(
if not email:
return
if "@" not in email:
issues.append(ParsedVCardIssue(index=index, severity="warning", field="EMAIL", message=f"Skipped invalid email address: {email}"))
issues.append(
ParsedVCardIssue(
index=index,
severity="warning",
field="EMAIL",
message=f"Skipped invalid email address: {email}",
)
)
return
draft.emails.append(ContactEmailPayload(label=_label_from_params(params), email=email, is_primary=_is_pref(params) or not draft.emails))
draft.emails.append(
ContactEmailPayload(
label=_label_from_params(params),
email=email,
is_primary=_is_pref(params) or not draft.emails,
)
)
def _append_card_phone(draft: _VCardDraft, params: dict[str, list[str]], value: str) -> None:
phone = _unescape_text(value)
if phone:
draft.phones.append(ContactPhonePayload(label=_label_from_params(params), phone=phone, is_primary=_is_pref(params) or not draft.phones))
draft.phones.append(
ContactPhonePayload(
label=_label_from_params(params),
phone=phone,
is_primary=_is_pref(params) or not draft.phones,
)
)
def _append_card_address(draft: _VCardDraft, params: dict[str, list[str]], value: str) -> None:
@@ -388,8 +425,14 @@ def _draft_contact_payload(draft: _VCardDraft) -> ContactCreateRequest:
return payload
def parse_vcards_with_issues(content: str) -> VCardParseResult:
blocks = _card_blocks_with_issues(content)
def parse_vcards_with_issues(
content: str,
*,
max_cards: int = MAX_VCARD_CARDS,
) -> VCardParseResult:
if max_cards < 1 or max_cards > MAX_VCARD_CARDS:
raise VCardError(f"max_cards must be between 1 and {MAX_VCARD_CARDS}.")
blocks = _card_blocks_with_issues(content, max_cards=max_cards)
parsed: list[ParsedVCard] = []
issues = list(blocks.issues)
skipped = 0
@@ -411,8 +454,8 @@ def parse_vcards(content: str) -> list[ParsedVCard]:
return result.cards
def contact_to_vcard(contact: Contact) -> str:
lines = _contact_identity_lines(contact)
def contact_to_vcard(contact: Contact, *, version: Literal["3.0", "4.0"] = "4.0") -> str:
lines = _contact_identity_lines(contact, version=version)
lines.extend(_contact_email_lines(contact))
lines.extend(_contact_phone_lines(contact))
lines.extend(_contact_address_lines(contact))
@@ -422,10 +465,14 @@ def contact_to_vcard(contact: Contact) -> str:
return "\r\n".join(lines) + "\r\n"
def _contact_identity_lines(contact: Contact) -> list[str]:
def _contact_identity_lines(
contact: Contact,
*,
version: Literal["3.0", "4.0"],
) -> list[str]:
lines = [
"BEGIN:VCARD",
"VERSION:4.0",
f"VERSION:{version}",
f"FN:{_escape_text(contact.display_name)}",
f"N:{_escape_text(contact.family_name)};{_escape_text(contact.given_name)};;;",
]
@@ -460,11 +507,7 @@ def _contact_address_lines(contact: Contact) -> list[str]:
lines: list[str] = []
for address in contact.postal_addresses:
label = f";TYPE={_escape_text(address.label)}" if address.label else ""
lines.append(
"ADR"
f"{label}:;;{_escape_text(address.street)};{_escape_text(address.locality)};"
f"{_escape_text(address.region)};{_escape_text(address.postal_code)};{_escape_text(address.country)}"
)
lines.append(f"ADR{label}:;;{_escape_text(address.street)};{_escape_text(address.locality)};{_escape_text(address.region)};{_escape_text(address.postal_code)};{_escape_text(address.country)}")
return lines
@@ -496,5 +539,9 @@ def _contact_vcard_urls(contact: Contact) -> object:
return vcard.get("urls")
def contacts_to_vcard(contacts: list[Contact]) -> str:
return "".join(contact_to_vcard(contact) for contact in contacts)
def contacts_to_vcard(
contacts: list[Contact],
*,
version: Literal["3.0", "4.0"] = "4.0",
) -> str:
return "".join(contact_to_vcard(contact, version=version) for contact in contacts)
@@ -0,0 +1,144 @@
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, Field, model_validator
VCardPlanAction = Literal["create", "update", "ignore", "unchanged", "conflict"]
VCardCommitAction = Literal["create", "update", "ignore"]
class VCardBatchFilePayload(BaseModel):
filename: str = Field(min_length=1, max_length=500)
content_base64: str = Field(min_length=1, max_length=14_000_000)
class VCardBatchPreviewRequest(BaseModel):
files: list[VCardBatchFilePayload] = Field(min_length=1, max_length=50)
duplicate_card_policy: Literal["reject", "first", "last"] = "reject"
existing_contact_policy: Literal["update", "ignore", "reject"] = "update"
class VCardDuplicateSuggestion(BaseModel):
contact_id: str
display_name: str
reasons: list[str] = Field(default_factory=list)
class VCardBatchPlanItemResponse(BaseModel):
source_key: str
source_filename: str
card_index: int
action: VCardPlanAction
allowed_actions: list[VCardCommitAction] = Field(default_factory=list)
contact_id: str | None = None
display_name: str | None = None
changed_fields: list[str] = Field(default_factory=list)
duplicate_suggestions: list[VCardDuplicateSuggestion] = Field(default_factory=list)
message: str | None = None
class VCardBatchDiagnosticResponse(BaseModel):
severity: Literal["info", "warning", "error"]
code: str
message: str
source_filename: str | None = None
card_index: int | None = None
field: str | None = None
details: dict[str, Any] = Field(default_factory=dict)
class VCardBatchProgressResponse(BaseModel):
total: int
completed: int
created: int = 0
updated: int = 0
ignored: int = 0
failed: int = 0
class VCardBatchRunResponse(BaseModel):
id: str
address_book_id: str
status: str
input_hash: str
plan_hash: str
parser_version: str
execution_mode: Literal["bounded_sync", "persisted_batch"]
file_count: int
card_count: int
statistics: dict[str, int | str] = Field(default_factory=dict)
diagnostics: list[VCardBatchDiagnosticResponse] = Field(default_factory=list)
plan: list[VCardBatchPlanItemResponse] = Field(default_factory=list)
progress: VCardBatchProgressResponse
can_apply: bool
can_cancel: bool
commit_hash: str | None = None
created_at: datetime
updated_at: datetime
applied_at: datetime | None = None
class VCardBatchSelection(BaseModel):
source_key: str = Field(min_length=1, max_length=1000)
action: VCardCommitAction
class VCardBatchCommitRequest(BaseModel):
expected_plan_hash: str = Field(min_length=64, max_length=64)
selections: list[VCardBatchSelection] = Field(
default_factory=list, max_length=10_000
)
class VCardBatchCancelRequest(BaseModel):
expected_plan_hash: str = Field(min_length=64, max_length=64)
reason: str = Field(min_length=3, max_length=2000)
class VCardExportRequest(BaseModel):
scope: Literal["address_book", "address_list", "contacts"] = "address_book"
address_list_id: str | None = Field(default=None, max_length=36)
contact_ids: list[str] = Field(default_factory=list, max_length=10_000)
version: Literal["3.0", "4.0"] = "4.0"
@model_validator(mode="after")
def validate_scope(self) -> "VCardExportRequest":
if self.scope == "address_list" and not self.address_list_id:
raise ValueError("Address-list export requires address_list_id.")
if self.scope == "contacts" and not self.contact_ids:
raise ValueError(
"Selected-contact export requires at least one contact id."
)
if self.scope != "address_list" and self.address_list_id:
raise ValueError("address_list_id is only valid for address-list export.")
if self.scope != "contacts" and self.contact_ids:
raise ValueError("contact_ids are only valid for selected-contact export.")
if len(set(self.contact_ids)) != len(self.contact_ids):
raise ValueError("Selected contact ids must be unique.")
return self
class VCardExportResponse(BaseModel):
filename: str
media_type: str = "text/vcard"
scope: str
version: str
ordering: str
contact_count: int
content_hash: str
content: str
__all__ = [
"VCardBatchCancelRequest",
"VCardBatchCommitRequest",
"VCardBatchFilePayload",
"VCardBatchPreviewRequest",
"VCardBatchRunResponse",
"VCardBatchSelection",
"VCardExportRequest",
"VCardExportResponse",
]
@@ -0,0 +1,901 @@
from __future__ import annotations
import base64
import binascii
from collections import Counter, defaultdict
import hashlib
import json
import os
from typing import Any
from sqlalchemy.orm import Session, selectinload
from govoplan_addresses.backend.db.models import (
AddressImportRun,
AddressListEntry,
Contact,
)
from govoplan_addresses.backend.schemas import (
ContactCreateRequest,
ContactUpdateRequest,
)
from govoplan_addresses.backend.service import (
AddressBookError,
create_contact,
get_visible_address_book,
get_visible_address_list,
get_visible_contact,
update_contact,
)
from govoplan_addresses.backend.vcard import (
MAX_VCARD_CARDS,
VCARD_PARSER_VERSION,
contacts_to_vcard,
parse_vcards_with_issues,
)
from govoplan_addresses.backend.vcard_batch_schemas import (
VCardBatchCancelRequest,
VCardBatchCommitRequest,
VCardBatchPreviewRequest,
VCardExportRequest,
)
from govoplan_core.auth import ApiPrincipal
from govoplan_core.db.base import utcnow
MAX_VCARD_BATCH_BYTES = 10_000_000
DEFAULT_PERSISTED_BATCH_THRESHOLD = 500
def preview_vcard_batch(
session: Session,
principal: ApiPrincipal,
address_book_id: str,
payload: VCardBatchPreviewRequest,
) -> AddressImportRun:
book = get_visible_address_book(session, principal, address_book_id)
if book.read_only:
raise AddressBookError("Static vCard imports require a writable address book.")
decoded = _decode_files(payload)
input_hash = _hash_json(
{
"files": [
{
"filename": filename,
"sha256": hashlib.sha256(raw).hexdigest(),
"size": len(raw),
}
for filename, raw in decoded
]
}
)
parsed_cards, diagnostics = _parse_files(decoded)
plan = _plan_cards(
session,
book.id,
parsed_cards,
duplicate_card_policy=payload.duplicate_card_policy,
existing_contact_policy=payload.existing_contact_policy,
)
statistics: dict[str, int | str] = dict(
Counter(str(item["action"]) for item in plan)
)
statistics.update(
{
"files": len(decoded),
"cards": len(parsed_cards),
"errors": sum(item["severity"] == "error" for item in diagnostics),
"warnings": sum(item["severity"] == "warning" for item in diagnostics),
"parser_version": VCARD_PARSER_VERSION,
"execution_mode": _execution_mode(len(parsed_cards)),
}
)
plan_hash = _hash_json(
{
"address_book_id": book.id,
"input_hash": input_hash,
"parser_version": VCARD_PARSER_VERSION,
"duplicate_card_policy": payload.duplicate_card_policy,
"existing_contact_policy": payload.existing_contact_policy,
"plan": plan,
}
)
source_filename = decoded[0][0]
if len(decoded) > 1:
source_filename = f"{source_filename} (+{len(decoded) - 1} files)"
run = AddressImportRun(
tenant_id=book.tenant_id,
address_book_id=book.id,
profile_id=None,
source_filename=source_filename[:500],
source_format="vcard",
input_hash=input_hash,
plan_hash=plan_hash,
status="previewed",
row_count=len(parsed_cards),
statistics=statistics,
diagnostics=diagnostics,
plan_data=plan,
result_evidence={
"parser_version": VCARD_PARSER_VERSION,
"file_manifest": [
{
"filename": filename,
"sha256": hashlib.sha256(raw).hexdigest(),
"size": len(raw),
}
for filename, raw in decoded
],
"progress": _progress(len(plan)),
},
created_by_account_id=principal.account_id,
)
session.add(run)
session.flush()
return run
def get_vcard_batch_run(
session: Session,
principal: ApiPrincipal,
run_id: str,
) -> AddressImportRun:
book_ids = [book.id for book in _visible_books(session, principal)]
if not book_ids:
raise AddressBookError("vCard batch run not found.")
item = (
session.query(AddressImportRun)
.filter(
AddressImportRun.id == run_id,
AddressImportRun.address_book_id.in_(book_ids),
AddressImportRun.source_format == "vcard",
)
.one_or_none()
)
if item is None:
raise AddressBookError("vCard batch run not found.")
return item
def apply_vcard_batch(
session: Session,
principal: ApiPrincipal,
run_id: str,
payload: VCardBatchCommitRequest,
) -> AddressImportRun:
run = get_vcard_batch_run(session, principal, run_id)
if run.plan_hash != payload.expected_plan_hash:
raise AddressBookError("The reviewed vCard plan changed; create a new preview.")
selections = _selection_map(payload)
commit_hash = _hash_json(
{
"plan_hash": run.plan_hash,
"selections": [
{"source_key": key, "action": selections[key]}
for key in sorted(selections)
],
}
)
evidence = dict(run.result_evidence or {})
if run.status == "applied":
if evidence.get("commit_hash") != commit_hash:
raise AddressBookError(
"This vCard batch was already applied with a different selection."
)
return run
if run.status != "previewed":
raise AddressBookError(
f"vCard batch cannot be applied from status {run.status!r}."
)
if not selections:
raise AddressBookError(
"Select at least one vCard action before applying the batch."
)
plan_by_key = {str(item["source_key"]): item for item in run.plan_data or []}
unknown = sorted(set(selections).difference(plan_by_key))
if unknown:
raise AddressBookError(
"The selection contains cards that are not part of the reviewed plan."
)
created_ids: list[str] = []
updated_ids: list[str] = []
ignored = 0
for source_key in sorted(plan_by_key):
item = plan_by_key[source_key]
action = selections.get(source_key, "ignore")
allowed = set(item.get("allowed_actions") or [])
if action not in allowed:
raise AddressBookError(
f'Action {action!r} is not allowed for vCard "{item.get("display_name") or source_key}".'
)
if action == "ignore":
ignored += 1
continue
contact = _apply_plan_item(
session,
principal,
run=run,
item=item,
action=action,
)
if action == "create":
created_ids.append(contact.id)
else:
updated_ids.append(contact.id)
run.status = "applied"
run.applied_at = utcnow()
run.result_evidence = {
**evidence,
"commit_hash": commit_hash,
"selection_count": len(selections),
"created_contact_ids": created_ids,
"updated_contact_ids": updated_ids,
"ignored_count": ignored,
"applied_by_account_id": principal.account_id,
"applied_at": run.applied_at.isoformat(),
"progress": {
"total": len(plan_by_key),
"completed": len(plan_by_key),
"created": len(created_ids),
"updated": len(updated_ids),
"ignored": ignored,
"failed": 0,
},
}
run.statistics = {
**dict(run.statistics or {}),
"applied_create": len(created_ids),
"applied_update": len(updated_ids),
"applied_ignore": ignored,
}
return run
def cancel_vcard_batch(
session: Session,
principal: ApiPrincipal,
run_id: str,
payload: VCardBatchCancelRequest,
) -> AddressImportRun:
run = get_vcard_batch_run(session, principal, run_id)
if run.plan_hash != payload.expected_plan_hash:
raise AddressBookError("The reviewed vCard plan changed; reload the batch.")
if run.status == "cancelled":
return run
if run.status != "previewed":
raise AddressBookError("Only a previewed vCard batch can be cancelled.")
run.status = "cancelled"
run.result_evidence = {
**dict(run.result_evidence or {}),
"cancel_reason": payload.reason.strip(),
"cancelled_by_account_id": principal.account_id,
"cancelled_at": utcnow().isoformat(),
}
return run
def vcard_batch_payload(run: AddressImportRun) -> dict[str, Any]:
evidence = dict(run.result_evidence or {})
progress = dict(evidence.get("progress") or _progress(run.row_count))
return {
"id": run.id,
"address_book_id": run.address_book_id,
"status": run.status,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"parser_version": str(evidence.get("parser_version") or VCARD_PARSER_VERSION),
"execution_mode": str(
(run.statistics or {}).get("execution_mode") or "bounded_sync"
),
"file_count": int((run.statistics or {}).get("files") or 0),
"card_count": run.row_count,
"statistics": dict(run.statistics or {}),
"diagnostics": list(run.diagnostics or []),
"plan": [_public_plan_item(item) for item in run.plan_data or []],
"progress": progress,
"can_apply": run.status == "previewed" and bool(run.plan_data),
"can_cancel": run.status == "previewed",
"commit_hash": evidence.get("commit_hash"),
"created_at": run.created_at,
"updated_at": run.updated_at,
"applied_at": run.applied_at,
}
def vcard_diagnostics_payload(run: AddressImportRun) -> dict[str, Any]:
return {
"schema_version": "1.0",
"run_id": run.id,
"status": run.status,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"parser_version": (run.result_evidence or {}).get("parser_version"),
"statistics": dict(run.statistics or {}),
"diagnostics": list(run.diagnostics or []),
"effects": [_public_plan_item(item) for item in run.plan_data or []],
}
def export_vcards(
session: Session,
principal: ApiPrincipal,
address_book_id: str,
payload: VCardExportRequest,
) -> dict[str, Any]:
book = get_visible_address_book(session, principal, address_book_id)
contacts = _export_contacts(session, principal, book.id, payload)
contacts.sort(key=lambda item: (item.display_name.casefold(), item.id))
content = contacts_to_vcard(contacts, version=payload.version)
scope_label = {
"address_book": book.name,
"address_list": "address-list",
"contacts": "selected-contacts",
}[payload.scope]
return {
"filename": f"{_safe_filename(scope_label)}-{payload.version.replace('.', '')}.vcf",
"media_type": "text/vcard",
"scope": payload.scope,
"version": payload.version,
"ordering": "display_name_casefold_then_contact_id",
"contact_count": len(contacts),
"content_hash": hashlib.sha256(content.encode("utf-8")).hexdigest(),
"content": content,
}
def _decode_files(payload: VCardBatchPreviewRequest) -> list[tuple[str, bytes]]:
decoded: list[tuple[str, bytes]] = []
total = 0
for item in payload.files:
filename = item.filename.strip()
if not filename.casefold().endswith(".vcf"):
raise AddressBookError("vCard batch uploads accept only .vcf files.")
try:
raw = base64.b64decode(item.content_base64, validate=True)
except (binascii.Error, ValueError) as exc:
raise AddressBookError(
f'vCard file "{filename}" is not valid base64.'
) from exc
if not raw:
raise AddressBookError(f'vCard file "{filename}" is empty.')
total += len(raw)
if total > MAX_VCARD_BATCH_BYTES:
raise AddressBookError(
f"Combined vCard uploads are limited to {MAX_VCARD_BATCH_BYTES} bytes."
)
decoded.append((filename, raw))
return decoded
def _parse_files(
decoded: list[tuple[str, bytes]],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
cards: list[dict[str, Any]] = []
diagnostics: list[dict[str, Any]] = []
for file_index, (filename, raw) in enumerate(decoded):
try:
content = raw.decode("utf-8-sig")
except UnicodeDecodeError as exc:
raise AddressBookError(
f'vCard file "{filename}" is not valid UTF-8: {exc}.'
) from exc
remaining = MAX_VCARD_CARDS - len(cards)
if remaining < 1:
raise AddressBookError(
f"vCard batches are limited to {MAX_VCARD_CARDS} cards."
)
result = parse_vcards_with_issues(content, max_cards=remaining)
for issue in result.issues:
diagnostics.append(
{
"severity": issue.severity,
"code": "vcard_parse_error"
if issue.severity == "error"
else "vcard_parse_warning",
"message": issue.message,
"source_filename": filename,
"card_index": issue.index or None,
"field": issue.field,
"details": {"line": issue.line} if issue.line is not None else {},
}
)
for card_index, parsed in enumerate(result.cards, start=1):
raw_hash = hashlib.sha256(parsed.raw.encode("utf-8")).hexdigest()
identity = (
f"uid:{parsed.source_ref.strip()}"
if parsed.source_ref and parsed.source_ref.strip()
else f"sha256:{raw_hash}"
)
source_key = hashlib.sha256(
f"{file_index}:{filename}:{card_index}:{raw_hash}".encode("utf-8")
).hexdigest()
cards.append(
{
"source_key": source_key,
"source_identity": identity,
"source_filename": filename,
"card_index": card_index,
"raw": parsed.raw,
"source_ref": parsed.source_ref.strip()
if parsed.source_ref
else None,
"source_revision": parsed.source_revision.strip()
if parsed.source_revision
else None,
"payload": parsed.payload.model_dump(mode="json"),
}
)
return cards, diagnostics
def _plan_cards(
session: Session,
address_book_id: str,
cards: list[dict[str, Any]],
*,
duplicate_card_policy: str,
existing_contact_policy: str,
) -> list[dict[str, Any]]:
contacts = (
session.query(Contact)
.options(
selectinload(Contact.emails),
selectinload(Contact.phones),
selectinload(Contact.postal_addresses),
)
.filter(
Contact.address_book_id == address_book_id, Contact.deleted_at.is_(None)
)
.all()
)
by_source: dict[str, list[Contact]] = defaultdict(list)
by_email: dict[str, list[Contact]] = defaultdict(list)
for contact in contacts:
if contact.source_ref:
by_source[contact.source_ref.strip()].append(contact)
for email in contact.emails:
normalized = (email.normalized_email or email.email).strip().casefold()
if normalized:
by_email[normalized].append(contact)
identity_positions: dict[str, list[int]] = defaultdict(list)
for index, card in enumerate(cards):
identity_positions[str(card["source_identity"])].append(index)
result: list[dict[str, Any]] = []
for index, card in enumerate(cards):
positions = identity_positions[str(card["source_identity"])]
if len(positions) > 1:
chosen = positions[0] if duplicate_card_policy == "first" else positions[-1]
if duplicate_card_policy == "reject":
result.append(
_planned_card(
card,
action="conflict",
allowed=["ignore"],
message="Duplicate UID or identical card appears in this batch.",
)
)
continue
if index != chosen:
result.append(
_planned_card(
card,
action="ignore",
allowed=["ignore"],
message=f"Duplicate card ignored by {duplicate_card_policy} policy.",
)
)
continue
suggestions = _duplicate_candidates(
card, by_source=by_source, by_email=by_email
)
exact_source = [item for item in suggestions if "source_uid" in item["reasons"]]
candidates = exact_source or suggestions
if len(candidates) > 1:
result.append(
_planned_card(
card,
action="conflict",
allowed=["create", "ignore"],
suggestions=suggestions,
message="Multiple existing contacts match this card; create explicitly or ignore it.",
)
)
continue
existing = next(
(
contact
for contact in contacts
if candidates and contact.id == candidates[0]["contact_id"]
),
None,
)
if existing is None:
result.append(
_planned_card(
card,
action="create",
allowed=["create", "ignore"],
suggestions=suggestions,
)
)
continue
changed = _changed_fields(existing, card["payload"])
if not changed:
result.append(
_planned_card(
card,
action="unchanged",
allowed=["ignore"],
contact=existing,
suggestions=suggestions,
message="Existing contact already matches the parsed card.",
)
)
elif existing_contact_policy == "reject":
result.append(
_planned_card(
card,
action="conflict",
allowed=["ignore"],
contact=existing,
suggestions=suggestions,
changed=changed,
message="An existing contact matches and the preview policy rejects updates.",
)
)
elif existing_contact_policy == "ignore":
result.append(
_planned_card(
card,
action="ignore",
allowed=["update", "ignore"],
contact=existing,
suggestions=suggestions,
changed=changed,
message="Existing contact is ignored by preview policy.",
)
)
else:
result.append(
_planned_card(
card,
action="update",
allowed=["update", "ignore"],
contact=existing,
suggestions=suggestions,
changed=changed,
)
)
return result
def _planned_card(
card: dict[str, Any],
*,
action: str,
allowed: list[str],
contact: Contact | None = None,
suggestions: list[dict[str, Any]] | None = None,
changed: list[str] | None = None,
message: str | None = None,
) -> dict[str, Any]:
return {
**card,
"row_number": int(card["card_index"]),
"action": action,
"allowed_actions": allowed,
"contact_id": contact.id if contact is not None else None,
"expected_contact_hash": _contact_hash(contact)
if contact is not None
else None,
"display_name": card["payload"].get("display_name"),
"changed_fields": changed or [],
"duplicate_suggestions": suggestions or [],
"message": message,
}
def _duplicate_candidates(
card: dict[str, Any],
*,
by_source: dict[str, list[Contact]],
by_email: dict[str, list[Contact]],
) -> list[dict[str, Any]]:
reasons: dict[str, set[str]] = defaultdict(set)
contacts: dict[str, Contact] = {}
source_ref = card.get("source_ref")
if source_ref:
for contact in by_source.get(str(source_ref), []):
contacts[contact.id] = contact
reasons[contact.id].add("source_uid")
for item in card["payload"].get("emails") or []:
normalized = str(item.get("email") or "").strip().casefold()
for contact in by_email.get(normalized, []):
contacts[contact.id] = contact
reasons[contact.id].add("email")
return [
{
"contact_id": contact_id,
"display_name": contacts[contact_id].display_name,
"reasons": sorted(reasons[contact_id]),
}
for contact_id in sorted(
contacts, key=lambda item: (contacts[item].display_name.casefold(), item)
)[:5]
]
def _apply_plan_item(
session: Session,
principal: ApiPrincipal,
*,
run: AddressImportRun,
item: dict[str, Any],
action: str,
) -> Contact:
contact_payload = ContactCreateRequest.model_validate(item["payload"])
if action == "create":
if item.get("source_ref"):
appeared = (
session.query(Contact)
.filter(
Contact.address_book_id == run.address_book_id,
Contact.source_ref == item["source_ref"],
Contact.deleted_at.is_(None),
)
.first()
)
if appeared is not None:
raise AddressBookError(
"A matching vCard UID appeared after preview; preview the batch again."
)
contact = create_contact(
session, principal, run.address_book_id, contact_payload
)
else:
contact_id = str(item.get("contact_id") or "")
if not contact_id:
raise AddressBookError(
"The reviewed vCard update has no stable target contact."
)
current = get_visible_contact(session, principal, contact_id)
if _contact_hash(current) != item.get("expected_contact_hash"):
raise AddressBookError(
f'Contact "{current.display_name}" changed after preview; preview the batch again.'
)
contact = update_contact(
session,
principal,
current.id,
ContactUpdateRequest.model_validate(item["payload"]),
)
contact.source_kind = "vcard"
contact.source_ref = (
item.get("source_ref")
or f"vcard-sha256:{str(item['source_identity']).split(':', 1)[-1]}"
)
contact.source_payload_kind = "vcard"
contact.source_payload_raw = item["raw"]
contact.source_revision = item.get("source_revision")
provenance = dict(contact.provenance or {})
provenance["vcard_batch"] = {
"run_id": run.id,
"input_hash": run.input_hash,
"plan_hash": run.plan_hash,
"parser_version": VCARD_PARSER_VERSION,
"source_filename": item["source_filename"],
"card_index": item["card_index"],
}
contact.provenance = provenance
session.flush()
return contact
def _changed_fields(contact: Contact, payload: dict[str, Any]) -> list[str]:
current = _contact_projection(contact)
incoming = _payload_projection(payload)
return sorted(key for key in incoming if current.get(key) != incoming.get(key))
def _contact_hash(contact: Contact) -> str:
return _hash_json(_contact_projection(contact))
def _contact_projection(contact: Contact) -> dict[str, Any]:
return {
"display_name": contact.display_name,
"given_name": contact.given_name,
"family_name": contact.family_name,
"organization": contact.organization,
"role_title": contact.role_title,
"note": contact.note,
"tags": list(contact.tags or []),
"emails": [
{"label": item.label, "email": item.email, "is_primary": item.is_primary}
for item in contact.emails
],
"phones": [
{"label": item.label, "phone": item.phone, "is_primary": item.is_primary}
for item in contact.phones
],
"postal_addresses": [
{
"label": item.label,
"street": item.street,
"postal_code": item.postal_code,
"locality": item.locality,
"region": item.region,
"country": item.country,
"is_primary": item.is_primary,
}
for item in contact.postal_addresses
],
}
def _payload_projection(payload: dict[str, Any]) -> dict[str, Any]:
return {
key: payload.get(key)
for key in (
"display_name",
"given_name",
"family_name",
"organization",
"role_title",
"note",
"tags",
"emails",
"phones",
"postal_addresses",
)
}
def _selection_map(payload: VCardBatchCommitRequest) -> dict[str, str]:
result: dict[str, str] = {}
for selection in payload.selections:
if selection.source_key in result:
raise AddressBookError("Each vCard may be selected only once.")
result[selection.source_key] = selection.action
return result
def _public_plan_item(item: dict[str, Any]) -> dict[str, Any]:
return {
key: item.get(key)
for key in (
"source_key",
"source_filename",
"card_index",
"action",
"allowed_actions",
"contact_id",
"display_name",
"changed_fields",
"duplicate_suggestions",
"message",
)
}
def _export_contacts(
session: Session,
principal: ApiPrincipal,
address_book_id: str,
payload: VCardExportRequest,
) -> list[Contact]:
if payload.scope == "address_book":
return _loaded_contacts(session, address_book_id=address_book_id)
if payload.scope == "contacts":
contacts = [
get_visible_contact(session, principal, contact_id)
for contact_id in payload.contact_ids
]
if any(contact.address_book_id != address_book_id for contact in contacts):
raise AddressBookError(
"Every selected contact must belong to the exported address book."
)
return contacts
address_list = get_visible_address_list(
session, principal, str(payload.address_list_id)
)
if address_list.address_book_id != address_book_id:
raise AddressBookError(
"The selected address list does not belong to the exported address book."
)
contact_ids = [
item.contact_id
for item in (
session.query(AddressListEntry)
.filter(AddressListEntry.address_list_id == address_list.id)
.order_by(AddressListEntry.order_index.asc(), AddressListEntry.id.asc())
.all()
)
]
if not contact_ids:
return []
return _loaded_contacts(
session, address_book_id=address_book_id, contact_ids=set(contact_ids)
)
def _loaded_contacts(
session: Session,
*,
address_book_id: str,
contact_ids: set[str] | None = None,
) -> list[Contact]:
query = (
session.query(Contact)
.options(
selectinload(Contact.emails),
selectinload(Contact.phones),
selectinload(Contact.postal_addresses),
)
.filter(
Contact.address_book_id == address_book_id, Contact.deleted_at.is_(None)
)
)
if contact_ids is not None:
query = query.filter(Contact.id.in_(contact_ids))
return query.all()
def _visible_books(session: Session, principal: ApiPrincipal):
from govoplan_addresses.backend.service import list_address_books
return list_address_books(session, principal)
def _execution_mode(card_count: int) -> str:
raw = os.getenv(
"GOVOPLAN_ADDRESSES_VCARD_JOB_THRESHOLD", str(DEFAULT_PERSISTED_BATCH_THRESHOLD)
)
try:
threshold = max(1, min(MAX_VCARD_CARDS, int(raw)))
except ValueError:
threshold = DEFAULT_PERSISTED_BATCH_THRESHOLD
return "persisted_batch" if card_count >= threshold else "bounded_sync"
def _progress(total: int) -> dict[str, int]:
return {
"total": total,
"completed": 0,
"created": 0,
"updated": 0,
"ignored": 0,
"failed": 0,
}
def _hash_json(value: Any) -> str:
return hashlib.sha256(
json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
).hexdigest()
def _safe_filename(value: str) -> str:
safe = "".join(
character if character.isalnum() or character in {"-", "_"} else "-"
for character in value.strip()
)
return safe.strip("-")[:120] or "contacts"
__all__ = [
"apply_vcard_batch",
"cancel_vcard_batch",
"export_vcards",
"get_vcard_batch_run",
"preview_vcard_batch",
"vcard_batch_payload",
"vcard_diagnostics_payload",
]
+588
View File
@@ -0,0 +1,588 @@
from __future__ import annotations
import unittest
from datetime import datetime, timezone
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_access.backend.db.models import Account, User
from govoplan_addresses.backend.db.models import (
AddressBook,
AddressImportRun,
AddressList,
AddressListEntry,
AddressSyncConflict,
AddressSyncSource,
AddressSyncTombstone,
Contact,
ContactChannelRule,
ContactEmail,
ContactFieldProvenance,
ContactMergeRecord,
ContactPhone,
ContactPointQualityDecision,
ContactPostalAddress,
ContactRedirect,
)
from govoplan_addresses.backend.dsar_provider import (
ADDRESSES_DSAR_CAPABILITY,
AddressesDsarProvider,
)
from govoplan_addresses.backend.manifest import manifest
from govoplan_core.core.dsar import (
DsarErasureActionRef,
DsarProvider,
DsarSubjectRef,
)
from govoplan_core.db.base import Base
from govoplan_core.privacy.dsar_workflow import (
create_data_subject_request,
search_data_subject_request,
)
class _Registry:
def __init__(
self,
provider: AddressesDsarProvider,
*,
addresses_active: bool = True,
) -> None:
self.provider = provider
self.addresses_active = addresses_active
def capability_names(self):
return (ADDRESSES_DSAR_CAPABILITY,)
def capability_owner(self, name):
self._assert_capability(name)
return "addresses"
def tenant_entitlement_resolver(self):
addresses_active = self.addresses_active
class _Resolver:
@staticmethod
def resolve(session, tenant_id):
del session, tenant_id
return type(
"State",
(),
{"effective_modules": (("addresses",) if addresses_active else ())},
)()
return _Resolver()
def require_tenant_capability(self, name, session, **kwargs):
del session, kwargs
self._assert_capability(name)
return self.provider
def manifests(self):
return (type("Manifest", (), {"id": "addresses"})(),)
@staticmethod
def _assert_capability(name: str) -> None:
if name != ADDRESSES_DSAR_CAPABILITY:
raise KeyError(name)
class AddressesDsarProviderTests(unittest.TestCase):
def setUp(self) -> None:
self.engine = create_engine("sqlite:///:memory:", future=True)
Base.metadata.create_all(bind=self.engine)
self.session = sessionmaker(bind=self.engine, future=True)()
now = datetime.now(timezone.utc)
self.account = Account(
id="account-1",
email="subject@example.test",
normalized_email="subject@example.test",
display_name="Subject",
password_hash="password-secret-do-not-export",
)
self.user = User(
id="membership-1",
tenant_id="tenant-1",
account_id=self.account.id,
email=self.account.email,
display_name="Subject",
)
self.book = AddressBook(
id="book-1",
tenant_id="tenant-1",
scope_type="tenant",
scope_id="tenant-1",
name="Residents",
created_by_account_id=self.account.id,
metadata_={"secret": "book-metadata-do-not-export"},
)
self.contact = Contact(
id="contact-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
display_name="Subject Person",
given_name="Subject",
family_name="Person",
organization="Example household",
note="A bounded subject note",
tags=["resident"],
source_kind="carddav",
source_ref="https://source.invalid/private/contact.vcf",
source_payload_kind="vcard",
source_payload_raw="raw-source-payload-do-not-export",
source_revision="revision-7",
provenance={"secret": "contact-provenance-do-not-export"},
created_by_account_id=self.account.id,
metadata_={"secret": "contact-metadata-do-not-export"},
)
self.email = ContactEmail(
id="email-1",
contact_id=self.contact.id,
label="private",
email="Subject@Example.test",
original_email="Subject@Example.test",
normalized_email="subject@example.test",
provenance={"secret": "email-provenance-do-not-export"},
is_primary=True,
)
self.phone = ContactPhone(
id="phone-1",
contact_id=self.contact.id,
label="mobile",
phone="+49 30 123456",
original_phone="030 123456",
normalized_phone="+4930123456",
provenance={"secret": "phone-provenance-do-not-export"},
is_primary=True,
)
self.postal = ContactPostalAddress(
id="postal-1",
contact_id=self.contact.id,
label="home",
street="Example Street 1",
postal_code="10115",
locality="Berlin",
country="DE",
original_value={"secret": "postal-original-do-not-export"},
normalized_value={"secret": "postal-normalized-do-not-export"},
provenance={"secret": "postal-provenance-do-not-export"},
is_primary=True,
)
self.address_list = AddressList(
id="list-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
name="District residents",
created_by_account_id="another-account",
)
self.list_entry = AddressListEntry(
id="entry-1",
address_list_id=self.address_list.id,
contact_id=self.contact.id,
contact_email_id=self.email.id,
target_kind="email",
metadata_={"secret": "list-entry-metadata-do-not-export"},
)
self.channel_rule = ContactChannelRule(
id="rule-1",
tenant_id="tenant-1",
contact_id=self.contact.id,
channel="email",
purpose="resident-notice",
contact_point_id=self.email.id,
decision="allow",
legal_basis="public task",
evidence_ref="records://consent/evidence-1",
reason="Current resident preference",
effective_from=now,
created_by_account_id="another-account",
metadata_={"secret": "rule-metadata-do-not-export"},
)
self.quality = ContactPointQualityDecision(
id="quality-1",
tenant_id="tenant-1",
contact_id=self.contact.id,
channel="email",
contact_point_id=self.email.id,
state="valid",
reason_code="verified",
reason="Verified by operator",
evidence_ref="files://private/evidence",
effective_from=now,
created_by_account_id="another-account",
metadata_={"secret": "quality-metadata-do-not-export"},
)
self.provenance = ContactFieldProvenance(
id="provenance-1",
tenant_id="tenant-1",
contact_id=self.contact.id,
field_path="emails[0].email",
value={"secret": "field-value-do-not-export"},
source_kind="carddav",
source_ref="https://source.invalid/private",
source_revision="revision-7",
precedence=10,
selected=True,
reason_code="source_authority",
explanation="Selected from the authoritative source",
visibility="operator",
created_by_account_id="another-account",
metadata_={"secret": "field-metadata-do-not-export"},
)
self.sync_source = AddressSyncSource(
id="source-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
connector_type="carddav",
display_name="Residents CardDAV",
external_account_ref="private-account-ref-do-not-export",
external_address_book_ref="private-book-ref-do-not-export",
sync_token="sync-token-do-not-export",
etag="private-etag-do-not-export",
remote_revision="private-remote-revision-do-not-export",
last_diagnostic={"secret": "diagnostic-do-not-export"},
created_by_account_id=self.account.id,
metadata_={"secret": "source-metadata-do-not-export"},
)
self.tombstone = AddressSyncTombstone(
id="tombstone-1",
tenant_id="tenant-1",
sync_source_id=self.sync_source.id,
address_book_id=self.book.id,
contact_id=self.contact.id,
remote_uid="private-uid-do-not-export",
resource_href="private-href-do-not-export",
synced_at=now,
metadata_={"secret": "tombstone-metadata-do-not-export"},
)
self.conflict = AddressSyncConflict(
id="conflict-1",
tenant_id="tenant-1",
sync_source_id=self.sync_source.id,
address_book_id=self.book.id,
contact_id=self.contact.id,
remote_uid="private-conflict-uid-do-not-export",
resource_href="private-conflict-href-do-not-export",
field_path="family_name",
local_value={"secret": "local-value-do-not-export"},
remote_value={"secret": "remote-value-do-not-export"},
status="resolved",
resolution="local",
resolved_at=now,
resolved_by_account_id=self.account.id,
metadata_={"secret": "conflict-metadata-do-not-export"},
)
self.import_run = AddressImportRun(
id="import-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
source_filename="contacts.csv",
source_format="csv",
input_hash="a" * 64,
plan_hash="b" * 64,
status="applied",
row_count=1,
statistics={"secret": "statistics-do-not-export"},
diagnostics=[{"secret": "import-diagnostic-do-not-export"}],
plan_data=[{"secret": "import-plan-do-not-export"}],
result_evidence={"secret": "import-result-do-not-export"},
created_by_account_id=self.account.id,
applied_at=now,
)
self.merge = ContactMergeRecord(
id="merge-1",
tenant_id="tenant-1",
address_book_id=self.book.id,
winner_contact_id=self.contact.id,
loser_contact_ids=["old-contact-1"],
status="active",
reason="Duplicate contact",
survivorship={"secret": "survivorship-do-not-export"},
decisions=[{"secret": "merge-decisions-do-not-export"}],
before_payload={"secret": "merge-before-do-not-export"},
after_payload={"secret": "merge-after-do-not-export"},
before_hash="c" * 64,
after_hash="d" * 64,
created_by_account_id="another-account",
provenance={"secret": "merge-provenance-do-not-export"},
)
self.redirect = ContactRedirect(
id="redirect-1",
tenant_id="tenant-1",
source_contact_id="old-contact-1",
target_contact_id=self.contact.id,
merge_record_id=self.merge.id,
)
self.unrelated = Contact(
id="contact-unrelated",
tenant_id="tenant-1",
address_book_id=self.book.id,
display_name="Unrelated Person",
note="unrelated-person-do-not-export",
)
unrelated_email = ContactEmail(
id="email-unrelated",
contact_id=self.unrelated.id,
email="unrelated@example.test",
original_email="unrelated@example.test",
normalized_email="unrelated@example.test",
)
tenant_two_book = AddressBook(
id="book-tenant-2",
tenant_id="tenant-2",
scope_type="tenant",
scope_id="tenant-2",
name="Other tenant",
)
tenant_two_contact = Contact(
id="contact-tenant-2",
tenant_id="tenant-2",
address_book_id=tenant_two_book.id,
display_name="Other Tenant Subject",
note="other-tenant-do-not-export",
)
tenant_two_email = ContactEmail(
id="email-tenant-2",
contact_id=tenant_two_contact.id,
email="subject@example.test",
original_email="subject@example.test",
normalized_email="subject@example.test",
)
self.session.add_all(
[
self.account,
self.user,
self.book,
self.contact,
self.email,
self.phone,
self.postal,
self.address_list,
self.list_entry,
self.channel_rule,
self.quality,
self.provenance,
self.sync_source,
self.tombstone,
self.conflict,
self.import_run,
self.merge,
self.redirect,
self.unrelated,
unrelated_email,
tenant_two_book,
tenant_two_contact,
tenant_two_email,
]
)
self.session.commit()
self.provider = AddressesDsarProvider()
self.subject = DsarSubjectRef(
account_id=self.account.id,
email=self.account.email,
)
def tearDown(self) -> None:
self.session.close()
self.engine.dispose()
def test_manifest_publishes_protocol_conforming_provider(self) -> None:
provided_names = {item.name for item in manifest.provides_interfaces}
self.assertIn(ADDRESSES_DSAR_CAPABILITY, provided_names)
provider = manifest.capability_factories[ADDRESSES_DSAR_CAPABILITY](None)
self.assertIsInstance(provider, DsarProvider)
def test_search_is_tenant_scoped_related_and_minimized(self) -> None:
records = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=self.subject,
)
resource_types = {record.resource_type for record in records}
self.assertTrue(
{
"addresses_contact",
"addresses_contact_email",
"addresses_contact_phone",
"addresses_contact_postal_address",
"addresses_list_membership",
"addresses_channel_rule",
"addresses_quality_decision",
"addresses_field_provenance",
"addresses_merge_record",
"addresses_contact_redirect",
"addresses_sync_tombstone",
"addresses_sync_conflict",
"addresses_address_book_attribution",
"addresses_sync_source_attribution",
"addresses_import_run_attribution",
}.issubset(resource_types)
)
serialized = repr([record.to_dict() for record in records])
excluded_values = (
"password-secret-do-not-export",
"raw-source-payload-do-not-export",
"contact-provenance-do-not-export",
"book-metadata-do-not-export",
"field-value-do-not-export",
"sync-token-do-not-export",
"private-account-ref-do-not-export",
"private-remote-revision-do-not-export",
"local-value-do-not-export",
"remote-value-do-not-export",
"import-plan-do-not-export",
"merge-before-do-not-export",
"merge-after-do-not-export",
"unrelated-person-do-not-export",
"other-tenant-do-not-export",
)
for value in excluded_values:
self.assertNotIn(value, serialized)
def test_conflicting_selectors_and_uncorroborated_reference_fail_closed(
self,
) -> None:
conflict = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=DsarSubjectRef(
email="subject@example.test",
external_references={"addresses.email": "other@example.test"},
),
)
uncorroborated = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=DsarSubjectRef(
email="subject@example.test",
external_references={"addresses.contact": self.unrelated.id},
),
)
self.assertEqual((), conflict)
self.assertEqual((), uncorroborated)
def test_plan_retains_evidence_and_routes_contact_data_to_review(self) -> None:
records = self.provider.search_subject(
self.session,
tenant_id="tenant-1",
subject=self.subject,
)
actions = self.provider.plan_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
records=records,
)
kinds = {action.kind for action in actions}
self.assertEqual({"manual_review", "retain"}, kinds)
self.assertFalse(any(action.executable for action in actions))
retained = [action for action in actions if action.kind == "retain"]
self.assertTrue(retained)
self.assertTrue(all(action.rationale for action in retained))
results = self.provider.execute_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
actions=actions,
request_id="dsar-addresses-1",
)
self.assertEqual({"blocked"}, {result.status for result in results})
self.assertIsNotNone(self.session.get(Contact, self.contact.id))
def test_execution_rejects_foreign_or_forged_executable_actions(self) -> None:
foreign = DsarErasureActionRef(
action_id="mail:delete:contact:contact-1",
provider_id="mail",
module_id="mail",
kind="delete",
resource_type="addresses_contact",
resource_id=self.contact.id,
title="Foreign delete",
rationale="Must be rejected",
executable=True,
)
forged = DsarErasureActionRef(
action_id="addresses:delete:addresses_contact:contact-1",
provider_id="addresses",
module_id="addresses",
kind="delete",
resource_type="addresses_contact",
resource_id=self.contact.id,
title="Forged delete",
rationale="Must be rejected",
executable=True,
)
with self.assertRaises(ValueError):
self.provider.execute_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
actions=(foreign,),
request_id="dsar-addresses-2",
)
with self.assertRaises(ValueError):
self.provider.execute_erasure(
self.session,
tenant_id="tenant-1",
subject=self.subject,
actions=(forged,),
request_id="dsar-addresses-2",
)
def test_core_workflow_discovers_active_and_inactive_provider(self) -> None:
request = create_data_subject_request(
self.session,
tenant_id="tenant-1",
reference="DSAR-ADDRESSES-1",
request_kind="access",
subject=self.subject,
purpose="Respond to an authorized privacy request.",
legal_basis="Article 15 GDPR",
due_at=None,
requested_by_account_id="privacy-officer",
)
self.session.commit()
search_data_subject_request(
self.session,
registry=_Registry(self.provider),
row=request,
expected_revision=1,
)
self.assertEqual("searched", request.status)
self.assertEqual(["addresses"], request.coverage["covered_modules"])
self.assertEqual([], request.coverage["modules_without_provider"])
disabled = create_data_subject_request(
self.session,
tenant_id="tenant-1",
reference="DSAR-ADDRESSES-DISABLED",
request_kind="access",
subject=self.subject,
purpose="Verify disabled-module coverage.",
legal_basis="Article 15 GDPR",
due_at=None,
requested_by_account_id="privacy-officer",
)
search_data_subject_request(
self.session,
registry=_Registry(self.provider, addresses_active=False),
row=disabled,
expected_revision=1,
)
self.assertEqual(0, disabled.search_result["record_count"])
self.assertEqual(
[ADDRESSES_DSAR_CAPABILITY],
disabled.coverage["inactive_provider_capabilities"],
)
if __name__ == "__main__":
unittest.main()
+16 -1
View File
@@ -6,6 +6,18 @@ from govoplan_addresses.backend.manifest import manifest
class AddressesInterfaceDocumentationContractTests(unittest.TestCase):
def test_all_static_topics_have_complete_german_content(self) -> None:
for topic in manifest.documentation:
german = (topic.translations or {}).get("de", {})
self.assertEqual(
{"title", "summary", "body"},
set(german),
topic.id,
)
self.assertTrue(
all(str(value).strip() for value in german.values()), topic.id
)
def test_route_and_surfaces_remain_declared(self) -> None:
frontend = manifest.frontend
self.assertIsNotNone(frontend)
@@ -29,7 +41,10 @@ class AddressesInterfaceDocumentationContractTests(unittest.TestCase):
reference = topics["addresses.reference.fields-and-consequences"]
self.assertIn("addresses.state.read-only", boundary.metadata["help_contexts"])
self.assertIn("addresses.field.communication-purpose", governance.metadata["help_contexts"])
self.assertIn(
"addresses.field.communication-purpose",
governance.metadata["help_contexts"],
)
self.assertIn("addresses.action.sync", reference.metadata["help_contexts"])
self.assertIn("merge", reference.metadata["consequence_classes"])
self.assertIn("governance_fact", reference.metadata["consequence_classes"])
+208
View File
@@ -0,0 +1,208 @@
from __future__ import annotations
import base64
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_addresses.backend.db.models import AddressBook, Contact
from govoplan_addresses.backend.import_schemas import (
AddressImportConfiguration,
AddressImportPreviewRequest,
AddressImportProfileCreateRequest,
)
from govoplan_addresses.backend.imports import (
apply_address_import,
create_import_profile,
import_run_payload,
preview_address_import,
)
from govoplan_addresses.backend.ldif import parse_ldif_rows
from govoplan_core.db.base import Base
class Principal:
account_id = "account-1"
group_ids = frozenset()
@property
def tenant_id(self) -> str:
return "tenant-1"
def has(self, scope: str) -> bool:
return scope in {
"addresses:address_book:read",
"addresses:address_book:write",
"addresses:contact:read",
"addresses:contact:write",
}
def encoded(value: bytes | str) -> str:
raw = value.encode() if isinstance(value, str) else value
return base64.b64encode(raw).decode()
class AddressLdifImportTests(unittest.TestCase):
def setUp(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
Base.metadata.create_all(engine)
self.session = sessionmaker(bind=engine, expire_on_commit=False)()
self.principal = Principal()
self.book = AddressBook(
tenant_id="tenant-1",
scope_type="tenant",
scope_id="tenant-1",
name="Imported contacts",
source_kind="local",
read_only=False,
)
self.session.add(self.book)
def create_profile(self, **configuration_overrides):
configuration = AddressImportConfiguration(
field_mappings={
"source_key": "dn",
"display_name": "cn",
"given_name": "givenName",
"family_name": "sn",
"email": "mail",
"phone": "telephoneNumber",
"organization": "o",
},
**configuration_overrides,
)
profile = create_import_profile(
self.session,
self.principal,
AddressImportProfileCreateRequest(
scope_type="tenant",
name="Directory export",
source_format="ldif",
configuration=configuration,
),
)
self.session.flush()
return profile
def test_parser_unfolds_and_decodes_text_without_projecting_binary_or_urls(self) -> None:
rows, diagnostics = parse_ldif_rows(
b"version: 1\n\n"
b"# exported contact\n"
b"dn: uid=ada,ou=people,dc=example,dc=test\n"
b"cn:: QWRhIExvdmVsYWNl\n"
b"sn: Love\n"
b" lace\n"
b"mail: ada@example.test\n"
b"mail: ada.work@example.test\n"
b"jpegPhoto:: /9j/\n"
b"seeAlso:< https://example.test/contact/ada\n",
max_entries=10,
)
self.assertEqual(1, len(rows))
self.assertEqual(["Ada Lovelace"], rows[0][1]["cn"])
self.assertEqual(["Lovelace"], rows[0][1]["sn"])
self.assertEqual(["ada@example.test", "ada.work@example.test"], rows[0][1]["mail"])
self.assertNotIn("jpegphoto", rows[0][1])
self.assertNotIn("seealso", rows[0][1])
self.assertEqual(
{"ldif_binary_value_ignored", "ldif_url_value_ignored"},
{item["code"] for item in diagnostics},
)
def test_preview_apply_and_repeat_preserve_multivalue_provenance(self) -> None:
profile = self.create_profile(default_tags=["ldif"])
source = (
"dn: uid=ada,ou=people,dc=example,dc=test\n"
"cn: Ada Lovelace\n"
"givenName: Ada\n"
"sn: Lovelace\n"
"mail: ada@example.test\n"
"mail: ada.work@example.test\n"
"telephoneNumber: +49 30 123\n"
"o: Analysis Office\n"
)
request = AddressImportPreviewRequest(
profile_id=profile.id,
filename="contacts.ldif",
content_base64=encoded(source),
)
run = preview_address_import(self.session, self.principal, self.book.id, request)
self.assertEqual(1, run.statistics["create"])
self.assertFalse([item for item in run.diagnostics if item["severity"] == "error"])
apply_address_import(self.session, self.principal, run.id, expected_plan_hash=run.plan_hash)
contact = self.session.query(Contact).one()
self.assertEqual(["ada@example.test", "ada.work@example.test"], [item.email for item in contact.emails])
self.assertEqual("ldif", contact.source_kind)
self.assertEqual("ldif", contact.provenance["import"]["source_format"])
self.assertEqual(64, len(contact.provenance["import"]["source_record_hash"]))
self.assertNotIn(source, repr(import_run_payload(run)))
repeated = preview_address_import(self.session, self.principal, self.book.id, request)
self.assertEqual(1, repeated.statistics["unchanged"])
apply_address_import(self.session, self.principal, repeated.id, expected_plan_hash=repeated.plan_hash)
self.assertEqual(1, self.session.query(Contact).count())
def test_change_records_are_rejected_by_default_and_add_requires_explicit_policy(self) -> None:
source = (
"dn: uid=ada,ou=people,dc=example,dc=test\n"
"changetype: add\n"
"cn: Ada Lovelace\n"
"mail: ada@example.test\n"
)
rejected_profile = self.create_profile()
rejected = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=rejected_profile.id,
filename="changes.ldif",
content_base64=encoded(source),
),
)
self.assertEqual(0, rejected.row_count)
self.assertIn("ldif_change_record_rejected", {item["code"] for item in rejected.diagnostics})
self.assertFalse(import_run_payload(rejected)["can_apply"])
allowed_profile = self.create_profile(ldif_change_record_policy="treat_add_as_entry")
allowed = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=allowed_profile.id,
filename="changes.ldif",
content_base64=encoded(source),
),
)
self.assertEqual(1, allowed.statistics["create"])
self.assertIn("ldif_add_record_imported", {item["code"] for item in allowed.diagnostics})
def test_invalid_base64_is_a_correction_diagnostic_and_blocks_apply(self) -> None:
profile = self.create_profile()
run = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=profile.id,
filename="broken.ldif",
content_base64=encoded(
"dn: uid=ada,dc=example,dc=test\n"
"cn:: this-is-not-base64!\n"
"mail: ada@example.test\n"
),
),
)
self.assertIn("ldif_invalid_base64", {item["code"] for item in run.diagnostics})
self.assertFalse(import_run_payload(run)["can_apply"])
with self.assertRaisesRegex(ValueError, "error diagnostics"):
apply_address_import(self.session, self.principal, run.id, expected_plan_hash=run.plan_hash)
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -35,7 +35,7 @@ class AddressesMigrationTests(unittest.TestCase):
try:
with engine.connect() as connection:
self.assertIn(
"c5d7e8f9a0b1",
"d6e8f9a0b1c2",
set(MigrationContext.configure(connection).get_current_heads()),
)
tables = set(inspect(connection).get_table_names())
+66 -1
View File
@@ -19,6 +19,7 @@ from govoplan_addresses.backend.import_schemas import (
from govoplan_addresses.backend.imports import (
apply_address_import,
create_import_profile,
get_import_run,
import_run_payload,
preview_address_import,
rollback_address_import,
@@ -44,6 +45,12 @@ class Principal:
}
class OtherTenantPrincipal(Principal):
@property
def tenant_id(self) -> str:
return "tenant-2"
def encoded(value: str) -> str:
return base64.b64encode(value.encode()).decode()
@@ -121,11 +128,69 @@ class AddressTabularImportTests(unittest.TestCase):
self.session,
self.principal,
run.id,
AddressImportRollbackRequest(reason="The operator selected the wrong monthly file."),
AddressImportRollbackRequest(
expected_plan_hash=run.plan_hash,
reason="The operator selected the wrong monthly file.",
),
)
self.assertEqual("rolled_back", rolled_back.status)
self.assertEqual(0, self.session.query(Contact).filter(Contact.deleted_at.is_(None)).count())
def test_rollback_rejects_a_stale_review_hash(self) -> None:
run = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=self.profile.id,
filename="contacts.csv",
content_base64=encoded(
"id;first;last;email;organization\n"
"1;Ada;Lovelace;ada@example.test;Analysis Office\n"
),
),
)
apply_address_import(
self.session,
self.principal,
run.id,
expected_plan_hash=run.plan_hash,
)
with self.assertRaisesRegex(ValueError, "reviewed import plan changed"):
rollback_address_import(
self.session,
self.principal,
run.id,
AddressImportRollbackRequest(
expected_plan_hash="0" * 64,
reason="The operator selected the wrong monthly file.",
),
)
def test_persisted_run_read_is_tenant_bounded_and_source_safe(self) -> None:
source = "id;first;last;email;organization\n1;Ada;Lovelace;ada@example.test;Analysis Office\n"
run = preview_address_import(
self.session,
self.principal,
self.book.id,
AddressImportPreviewRequest(
profile_id=self.profile.id,
filename="contacts.csv",
content_base64=encoded(source),
),
)
self.session.flush()
payload = import_run_payload(get_import_run(self.session, self.principal, run.id))
self.assertEqual("previewed", payload["status"])
self.assertEqual(run.plan_hash, payload["plan_hash"])
self.assertNotIn("plan_data", payload)
self.assertNotIn(source, repr(payload))
with self.assertRaisesRegex(ValueError, "not found"):
get_import_run(self.session, OtherTenantPrincipal(), run.id)
def test_duplicate_keys_and_changed_targets_block_apply(self) -> None:
duplicate = preview_address_import(
self.session,
+270
View File
@@ -0,0 +1,270 @@
from __future__ import annotations
import base64
import unittest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from govoplan_addresses.backend.db.models import (
AddressBook,
AddressList,
AddressListEntry,
Contact,
)
from govoplan_addresses.backend.schemas import ContactCreateRequest
from govoplan_addresses.backend.service import AddressBookError, create_contact
from govoplan_addresses.backend.vcard import (
MAX_VCARD_UNFOLDED_LINE_CHARS,
parse_vcards_with_issues,
)
from govoplan_addresses.backend.vcard_batch_schemas import (
VCardBatchCancelRequest,
VCardBatchCommitRequest,
VCardBatchFilePayload,
VCardBatchPreviewRequest,
VCardBatchSelection,
VCardExportRequest,
)
from govoplan_addresses.backend.vcard_batches import (
apply_vcard_batch,
cancel_vcard_batch,
export_vcards,
preview_vcard_batch,
vcard_batch_payload,
)
from govoplan_core.db.base import Base
class Principal:
account_id = "account-1"
group_ids = frozenset()
@property
def tenant_id(self) -> str:
return "tenant-1"
def has(self, scope: str) -> bool:
return scope in {
"addresses:address_book:read",
"addresses:address_book:write",
"addresses:contact:read",
"addresses:contact:write",
}
def vcard(uid: str, name: str, email: str) -> str:
return (
"BEGIN:VCARD\r\n"
"VERSION:4.0\r\n"
f"UID:{uid}\r\n"
f"FN:{name}\r\n"
f"EMAIL:{email}\r\n"
"END:VCARD\r\n"
)
def batch_file(filename: str, content: str) -> VCardBatchFilePayload:
return VCardBatchFilePayload(
filename=filename,
content_base64=base64.b64encode(content.encode()).decode(),
)
class VCardBatchTests(unittest.TestCase):
def setUp(self) -> None:
engine = create_engine("sqlite+pysqlite:///:memory:")
Base.metadata.create_all(engine)
self.session = sessionmaker(bind=engine, expire_on_commit=False)()
self.principal = Principal()
self.book = AddressBook(
tenant_id="tenant-1",
scope_type="tenant",
scope_id="tenant-1",
name="Batch contacts",
source_kind="local",
read_only=False,
)
self.session.add(self.book)
self.session.flush()
def test_multifile_preview_selective_apply_and_repeat_are_idempotent(self) -> None:
run = preview_vcard_batch(
self.session,
self.principal,
self.book.id,
VCardBatchPreviewRequest(
files=[
batch_file(
"ada.vcf", vcard("ada-1", "Ada Lovelace", "ada@example.test")
),
batch_file(
"grace.vcf",
vcard("grace-1", "Grace Hopper", "grace@example.test"),
),
]
),
)
self.assertEqual(0, self.session.query(Contact).count())
self.assertEqual("previewed", run.status)
self.assertEqual(2, run.row_count)
self.assertEqual("govoplan-vcard/2", run.result_evidence["parser_version"])
self.assertNotIn("ada@example.test", repr(vcard_batch_payload(run)))
selections = [
VCardBatchSelection(
source_key=run.plan_data[0]["source_key"], action="create"
),
VCardBatchSelection(
source_key=run.plan_data[1]["source_key"], action="ignore"
),
]
request = VCardBatchCommitRequest(
expected_plan_hash=run.plan_hash, selections=selections
)
applied = apply_vcard_batch(self.session, self.principal, run.id, request)
repeated = apply_vcard_batch(self.session, self.principal, run.id, request)
self.assertIs(applied, repeated)
self.assertEqual(1, self.session.query(Contact).count())
self.assertEqual("Ada Lovelace", self.session.query(Contact).one().display_name)
self.assertEqual(1, applied.result_evidence["progress"]["ignored"])
with self.assertRaisesRegex(AddressBookError, "different selection"):
apply_vcard_batch(
self.session,
self.principal,
run.id,
VCardBatchCommitRequest(
expected_plan_hash=run.plan_hash,
selections=[
VCardBatchSelection(
source_key=run.plan_data[1]["source_key"], action="create"
)
],
),
)
def test_duplicate_uid_policy_and_cancellation(self) -> None:
run = preview_vcard_batch(
self.session,
self.principal,
self.book.id,
VCardBatchPreviewRequest(
files=[
batch_file(
"duplicates.vcf",
vcard("same", "First", "first@example.test")
+ vcard("same", "Last", "last@example.test"),
)
],
duplicate_card_policy="reject",
),
)
self.assertEqual(
["conflict", "conflict"], [item["action"] for item in run.plan_data]
)
cancelled = cancel_vcard_batch(
self.session,
self.principal,
run.id,
VCardBatchCancelRequest(
expected_plan_hash=run.plan_hash,
reason="Operator rejected duplicate source UIDs.",
),
)
self.assertEqual("cancelled", cancelled.status)
self.assertEqual(0, self.session.query(Contact).count())
last = preview_vcard_batch(
self.session,
self.principal,
self.book.id,
VCardBatchPreviewRequest(
files=[
batch_file(
"duplicates.vcf",
vcard("same", "First", "first@example.test")
+ vcard("same", "Last", "last@example.test"),
)
],
duplicate_card_policy="last",
),
)
self.assertEqual(
["ignore", "create"], [item["action"] for item in last.plan_data]
)
def test_deterministic_scoped_export_supports_vcard_versions(self) -> None:
grace = create_contact(
self.session,
self.principal,
self.book.id,
ContactCreateRequest(display_name="Grace Hopper"),
)
ada = create_contact(
self.session,
self.principal,
self.book.id,
ContactCreateRequest(display_name="Ada Lovelace"),
)
address_list = AddressList(
tenant_id="tenant-1",
address_book_id=self.book.id,
name="Selected",
source_kind="local",
read_only=False,
)
self.session.add(address_list)
self.session.flush()
self.session.add(
AddressListEntry(
address_list_id=address_list.id, contact_id=grace.id, order_index=0
)
)
self.session.flush()
selected = export_vcards(
self.session,
self.principal,
self.book.id,
VCardExportRequest(
scope="contacts", contact_ids=[grace.id, ada.id], version="3.0"
),
)
repeated = export_vcards(
self.session,
self.principal,
self.book.id,
VCardExportRequest(
scope="contacts", contact_ids=[ada.id, grace.id], version="3.0"
),
)
listed = export_vcards(
self.session,
self.principal,
self.book.id,
VCardExportRequest(scope="address_list", address_list_id=address_list.id),
)
self.assertEqual(selected["content_hash"], repeated["content_hash"])
self.assertLess(
selected["content"].index("Ada Lovelace"),
selected["content"].index("Grace Hopper"),
)
self.assertIn("VERSION:3.0", selected["content"])
self.assertEqual(1, listed["contact_count"])
self.assertIn("Grace Hopper", listed["content"])
def test_parser_rejects_pathological_unfolded_lines(self) -> None:
result = parse_vcards_with_issues(
"BEGIN:VCARD\nFN:"
+ ("a" * (MAX_VCARD_UNFOLDED_LINE_CHARS + 1))
+ "\nEND:VCARD"
)
self.assertEqual([], result.cards)
self.assertIn("unfolded lines", result.issues[0].message)
if __name__ == "__main__":
unittest.main()
+4 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@govoplan/addresses-webui",
"version": "0.1.16",
"version": "0.1.20",
"private": true,
"type": "module",
"main": "src/index.ts",
@@ -14,10 +14,11 @@
"./styles/addresses.css": "./src/styles/addresses.css"
},
"scripts": {
"test:ui-structure": "node scripts/test-selection-list-structure.mjs"
"test:ui-structure": "node scripts/test-selection-list-structure.mjs",
"test:import-run": "rm -rf .import-run-test-build && mkdir -p .import-run-test-build && printf '{\"type\":\"commonjs\"}\\n' > .import-run-test-build/package.json && ../../govoplan-core/webui/node_modules/.bin/tsc -p tsconfig.import-run-tests.json && node .import-run-test-build/tests/import-run-state.test.js"
},
"peerDependencies": {
"@govoplan/core-webui": "^0.1.16",
"@govoplan/core-webui": "^0.1.18",
"lucide-react": "^1.23.0",
"react": ">=19.2.7 <20",
"react-dom": ">=19.2.7 <20",
+123 -7
View File
@@ -354,6 +354,60 @@ export type VCardImportResult = {
issues: Array<{ index: number; message: string; severity: "warning" | "error"; field?: string | null; line?: number | null }>;
};
export type VCardBatchPlanItem = {
source_key: string;
source_filename: string;
card_index: number;
action: "create" | "update" | "ignore" | "unchanged" | "conflict";
allowed_actions: Array<"create" | "update" | "ignore">;
contact_id?: string | null;
display_name?: string | null;
changed_fields: string[];
duplicate_suggestions: Array<{ contact_id: string; display_name: string; reasons: string[] }>;
message?: string | null;
};
export type VCardBatchRun = {
id: string;
address_book_id: string;
status: string;
input_hash: string;
plan_hash: string;
parser_version: string;
execution_mode: "bounded_sync" | "persisted_batch";
file_count: number;
card_count: number;
statistics: Record<string, number | string>;
diagnostics: Array<{
severity: "info" | "warning" | "error";
code: string;
message: string;
source_filename?: string | null;
card_index?: number | null;
field?: string | null;
details: Record<string, unknown>;
}>;
plan: VCardBatchPlanItem[];
progress: { total: number; completed: number; created: number; updated: number; ignored: number; failed: number };
can_apply: boolean;
can_cancel: boolean;
commit_hash?: string | null;
created_at: string;
updated_at: string;
applied_at?: string | null;
};
export type VCardExportResult = {
filename: string;
media_type: string;
scope: "address_book" | "address_list" | "contacts";
version: "3.0" | "4.0";
ordering: string;
contact_count: number;
content_hash: string;
content: string;
};
export type AddressSyncSource = {
id: string;
tenant_id?: string | null;
@@ -494,6 +548,7 @@ export type AddressImportConfiguration = {
duplicate_source_key_policy: "reject" | "first" | "last";
existing_contact_policy: "update" | "ignore" | "reject";
blank_value_policy: "ignore" | "clear" | "reject";
ldif_change_record_policy: "reject" | "ignore" | "treat_add_as_entry";
locale?: string | null;
default_tags: string[];
max_rows: number;
@@ -508,7 +563,7 @@ export type AddressImportProfile = {
scope_id?: string | null;
name: string;
description?: string | null;
source_format: "csv" | "xlsx";
source_format: "csv" | "xlsx" | "ldif";
configuration: AddressImportConfiguration;
is_current: boolean;
created_at: string;
@@ -537,7 +592,7 @@ export type AddressImportDiagnostic = {
export type AddressImportRun = {
id: string;
address_book_id: string;
profile_id: string;
profile_id: string | null;
source_filename: string;
source_format: string;
input_hash: string;
@@ -1056,6 +1111,47 @@ export function importAddressBookVcards(settings: ApiSettings, addressBookId: st
});
}
export function previewVCardBatch(
settings: ApiSettings,
addressBookId: string,
payload: {
files: Array<{ filename: string; content_base64: string }>;
duplicate_card_policy?: "reject" | "first" | "last";
existing_contact_policy?: "update" | "ignore" | "reject";
}
): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcard-batches/preview`, {
method: "POST",
body: JSON.stringify(payload)
});
}
export function getVCardBatch(settings: ApiSettings, runId: string): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(runId)}`);
}
export function applyVCardBatch(
settings: ApiSettings,
run: VCardBatchRun,
selections: Array<{ source_key: string; action: "create" | "update" | "ignore" }>
): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(run.id)}/apply`, {
method: "POST",
body: JSON.stringify({ expected_plan_hash: run.plan_hash, selections })
});
}
export function cancelVCardBatch(settings: ApiSettings, run: VCardBatchRun, reason: string): Promise<VCardBatchRun> {
return apiFetch<VCardBatchRun>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(run.id)}/cancel`, {
method: "POST",
body: JSON.stringify({ expected_plan_hash: run.plan_hash, reason })
});
}
export function exportVCardBatchDiagnostics(settings: ApiSettings, runId: string): Promise<string> {
return apiFetch<string>(settings, `/api/v1/addresses/vcard-batches/${encodeURIComponent(runId)}/diagnostics`);
}
export async function listAddressImportProfiles(settings: ApiSettings): Promise<AddressImportProfile[]> {
const response = await apiFetch<AddressImportProfileListResponse>(settings, "/api/v1/addresses/import-profiles");
return response.profiles;
@@ -1068,7 +1164,7 @@ export function createAddressImportProfile(
scope_id?: string | null;
name: string;
description?: string | null;
source_format: "csv" | "xlsx";
source_format: "csv" | "xlsx" | "ldif";
configuration: AddressImportConfiguration;
}
): Promise<AddressImportProfile> {
@@ -1100,17 +1196,21 @@ export function previewAddressImport(
});
}
export function getAddressImportRun(settings: ApiSettings, runId: string): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${encodeURIComponent(runId)}`);
}
export function applyAddressImport(settings: ApiSettings, run: AddressImportRun): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${run.id}/apply`, {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${encodeURIComponent(run.id)}/apply`, {
method: "POST",
body: JSON.stringify({ expected_plan_hash: run.plan_hash })
});
}
export function rollbackAddressImport(settings: ApiSettings, runId: string, reason: string): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${runId}/rollback`, {
export function rollbackAddressImport(settings: ApiSettings, run: AddressImportRun, reason: string): Promise<AddressImportRun> {
return apiFetch<AddressImportRun>(settings, `/api/v1/addresses/imports/${encodeURIComponent(run.id)}/rollback`, {
method: "POST",
body: JSON.stringify({ reason })
body: JSON.stringify({ expected_plan_hash: run.plan_hash, reason })
});
}
@@ -1118,6 +1218,22 @@ export function exportAddressBookVcards(settings: ApiSettings, addressBookId: st
return apiFetch<string>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcards/export`);
}
export function exportScopedVcards(
settings: ApiSettings,
addressBookId: string,
payload: {
scope: "address_book" | "address_list" | "contacts";
address_list_id?: string | null;
contact_ids?: string[];
version?: "3.0" | "4.0";
}
): Promise<VCardExportResult> {
return apiFetch<VCardExportResult>(settings, `/api/v1/addresses/address-books/${addressBookId}/vcards/export`, {
method: "POST",
body: JSON.stringify(payload)
});
}
export function exportContactVcard(settings: ApiSettings, contactId: string): Promise<string> {
return apiFetch<string>(settings, `/api/v1/addresses/contacts/${contactId}/vcard`);
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,72 @@
export type ImportRunLifecycle = {
label: string;
tone: "info" | "success" | "warning" | "inactive";
canApply: boolean;
canRollback: boolean;
guidance: string;
};
export function importRunIdFromSearch(search: URLSearchParams): string {
return search.get("import_run")?.trim() ?? "";
}
export function withImportRunSearch(search: URLSearchParams, runId?: string | null): URLSearchParams {
const next = new URLSearchParams(search);
const normalized = runId?.trim();
if (normalized) next.set("import_run", normalized);
else next.delete("import_run");
return next;
}
export function importRunLifecycle(status: string): ImportRunLifecycle {
if (status === "previewed") {
return {
label: "Ready for review",
tone: "info",
canApply: true,
canRollback: false,
guidance: "Review the persisted effects and diagnostics before applying this plan."
};
}
if (status === "applied") {
return {
label: "Applied",
tone: "success",
canApply: false,
canRollback: true,
guidance: "The plan has already been applied. Rollback remains guarded by its recorded plan hash and contact evidence."
};
}
if (status === "rolled_back") {
return {
label: "Rolled back",
tone: "inactive",
canApply: false,
canRollback: false,
guidance: "This run was rolled back and cannot be applied again. Create a new preview to import the source again."
};
}
if (status === "expired") {
return {
label: "Expired",
tone: "warning",
canApply: false,
canRollback: false,
guidance: "This preview is no longer actionable. Create a new preview from the original source."
};
}
return {
label: status || "Unavailable",
tone: "warning",
canApply: false,
canRollback: false,
guidance: "This run is not actionable in its current lifecycle state."
};
}
export function unavailableImportRunMessage(status?: number): string {
if (status === 404 || status === 410) {
return "This import run is missing, expired, or not available to your tenant. No source data was loaded.";
}
return "The persisted import run could not be loaded. Reload it after the service becomes available.";
}
+23 -7
View File
@@ -9,7 +9,7 @@
justify-content: space-between;
gap: 16px;
border: var(--border-line);
border-radius: 6px;
border-radius: var(--radius-compact);
background: var(--panel-soft);
padding: 14px;
}
@@ -163,6 +163,21 @@
min-height: 0;
}
.address-import-run-state {
align-items: center;
background: var(--panel-soft);
border: var(--border-line);
border-radius: var(--radius-compact);
display: grid;
gap: 10px 14px;
grid-template-columns: minmax(180px, 1fr) auto minmax(220px, 1.4fr) auto;
padding: 10px 12px;
}
.address-import-run-state p {
margin: 0;
}
.address-import-profile-actions {
align-items: end;
justify-content: flex-start;
@@ -189,7 +204,7 @@
.address-import-preview > .address-sync-plan-grid > div {
background: var(--panel-soft);
border: var(--border-line);
border-radius: 6px;
border-radius: var(--radius-compact);
display: grid;
gap: 2px;
padding: 9px 10px;
@@ -406,7 +421,7 @@
.address-tag {
background: var(--panel-soft);
border: var(--border-line);
border-radius: 999px;
border-radius: var(--radius-pill);
color: var(--muted);
display: inline-flex;
font-size: 0.75rem;
@@ -553,7 +568,7 @@
.address-member-candidate-list {
border: var(--border-line);
border-radius: 6px;
border-radius: var(--radius-compact);
display: grid;
max-height: min(460px, calc(100vh - 300px));
overflow: auto;
@@ -598,7 +613,7 @@
.address-form-section {
border: var(--border-line);
border-radius: 6px;
border-radius: var(--radius-compact);
display: grid;
gap: 10px;
padding: 12px;
@@ -678,7 +693,7 @@
.address-quality-list {
border: var(--border-line);
border-radius: 6px;
border-radius: var(--radius-compact);
max-height: 300px;
overflow: auto;
}
@@ -809,8 +824,9 @@
opacity: 0.72;
}
@media (max-width: 980px) {
@media (max-width: 1100px) {
.address-book-workspace,
.address-import-run-state,
.address-form-row-email,
.address-form-row-phone,
.address-form-row-postal {
+22
View File
@@ -0,0 +1,22 @@
import assert from "node:assert/strict";
import {
importRunIdFromSearch,
importRunLifecycle,
unavailableImportRunMessage,
withImportRunSearch
} from "../src/features/addressbook/importRunState";
const initial = new URLSearchParams("q=Ada&import_run=run-123");
assert.equal(importRunIdFromSearch(initial), "run-123");
assert.equal(withImportRunSearch(initial, "run-456").toString(), "q=Ada&import_run=run-456");
assert.equal(withImportRunSearch(initial, null).toString(), "q=Ada");
assert.equal(importRunLifecycle("previewed").canApply, true);
assert.equal(importRunLifecycle("previewed").canRollback, false);
assert.equal(importRunLifecycle("applied").canApply, false);
assert.equal(importRunLifecycle("applied").canRollback, true);
assert.equal(importRunLifecycle("rolled_back").canRollback, false);
assert.equal(importRunLifecycle("expired").label, "Expired");
assert.match(unavailableImportRunMessage(404), /missing, expired, or not available to your tenant/);
console.log("Address import-run deep-link and lifecycle state tests passed.");
+17
View File
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"module": "CommonJS",
"moduleResolution": "Node",
"target": "ES2022",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"typeRoots": ["../../govoplan-core/webui/node_modules/@types"],
"types": ["node"],
"outDir": ".import-run-test-build"
},
"include": [
"src/features/addressbook/importRunState.ts",
"tests/import-run-state.test.ts"
]
}