diff --git a/README.md b/README.md index 3f17911..7ee61d8 100644 --- a/README.md +++ b/README.md @@ -112,3 +112,4 @@ decision path, but consumers should not import or duplicate policy logic. - [Address module architecture](docs/ADDRESS_MODULE_ARCHITECTURE.md) - [Implementation plan](docs/IMPLEMENTATION_PLAN.md) +- [AdreMa capability assessment](docs/ADREMA_CAPABILITY_ASSESSMENT.md) diff --git a/docs/ADREMA_CAPABILITY_ASSESSMENT.md b/docs/ADREMA_CAPABILITY_ASSESSMENT.md new file mode 100644 index 0000000..967dbbb --- /dev/null +++ b/docs/ADREMA_CAPABILITY_ASSESSMENT.md @@ -0,0 +1,418 @@ +# AdreMa Capability Assessment + +## Purpose + +This document records a behavior-level assessment of the historical AdreMa +application supplied in `/mnt/DATA/tmp.tar`. The goal is not to reproduce its +implementation or UI. The goal is to preserve the useful administrative +capabilities while replacing hard-coded queries and output paths with +governed, reusable GovOPlaN contracts. + +The archive was inspected as untrusted historical material. No archived code +was executed. Generated documents and record-level output were excluded from +the analysis. + +## Evidence And Handling + +- Archive SHA-256: + `328edbaf05df084f9c63cc3b9b36f7c5ae0c639942be4b7ad830efc0905a311b` +- Compressed size: about 1.49 GB +- Inventory: 8,966 files and 190 directories +- Uncompressed payload: about 1.79 GB +- The AdreMa selection UI contains 258 distinct segment variables, 265 query + cases, 13 sort definitions, seven action definitions, and 24 + department-specific forms. +- The adjacent configurable list/report layer contains 47 ETI definitions. +- The wider print-server archive contains 127 SQL files, 181 XSLT files, and 82 + Python workflow sources. + +The archive contains generated person-level documents and embedded +credentials. It must not be committed, copied into test fixtures, or treated as +safe sample data. Any credential that could still be valid must be rotated. +Future fixtures derived from this assessment must be synthetic. + +## What AdreMa Actually Is + +AdreMa is not primarily a master address database. It is a role-specific +audience selection and output application over authoritative institutional +databases. + +Its main flow is: + +1. Authenticate against a directory. +2. Show a menu of saved recipient selections permitted for the current user or + department. +3. Run a fixed SQL selection against organization, person, employment, + function, student, geography, and address data. +4. Apply a selected sort order and address variant. +5. Produce a table, spreadsheet, label preview, label file, central print job, + or BCC mail draft. +6. Record a basic process log and recipient count. + +The broader print-server application adds SQL-to-XML extraction, XSLT +transformation, HTML/Excel output, TeX/PDF generation, batch letters, PDF +protection, printer selection, and direct print-queue submission. + +This means an AdreMa replacement should not be one large Addresses feature. +It is a vertical composition of Addresses, Organizations/IDM, Datasources, +Connectors, Dataflow, Distribution Lists, Templates, Campaign, Files, Audit, +Policy, Reporting, Workflow, and a future print-delivery capability. + +## Observed Functional Model + +### Sources And Identities + +The application reads live authoritative data rather than maintaining a +separate copy. Its selections use: + +- people and institutional units +- functions and role assignments +- employment and student state +- effective dates and exclusions +- internal and external organizations +- geography, country, and postal-code ranges +- source-specific person, registration, and unit identifiers + +Access to selections is hard-coded by user and department. Administrative +users receive the union of those menus. + +### Contact And Address Semantics + +Observed recipient fields include: + +- display name, given name, family name, title, and formal salutation +- organization/unit and function context +- phone, fax, email, and URL +- `care of`/`for the attention of` lines +- street, postal code, locality, country, and internal mail routing +- person, registration, and organizational-unit identifiers +- address purpose variants such as private, semester, home, business, service, + and correspondence address +- effective employment or membership data used to decide eligibility + +The legacy system also allows a requested number of labels for an individual +recipient. That quantity belongs to an output job, not to the canonical address +record. + +### Recipient Selection + +Selections cover several distinct patterns: + +- static categories of institutional units +- people assigned to a function or organizational level +- current-state selections with effective-date rules and exclusions +- geographic and demographic selections +- combinations and unions of predefined audiences +- explicit person or unit lookup by source identifier +- department-specific variants of a shared audience +- user-selected additional predicates and sort orders in the adjacent ETI + report layer + +The important reusable abstraction is therefore a versioned, parameterized, +permissioned segment definition. The 265 query branches must not become 265 +new backend endpoints. + +### Output + +The directly observed output modes are: + +- on-screen recipient table with contact and postal fields +- spreadsheet export +- plain-text label preview +- generated label file +- emailed label file +- central label printing +- BCC mail draft + +The adjacent print-server flows additionally produce: + +- formatted lists and grouped reports +- personalized letters and notices +- HTML, spreadsheet-compatible, TeX, PostScript, and PDF artifacts +- printer-specific layouts and direct print jobs +- password-protected output and combined document bundles + +### Configuration + +The ETI layer is an early configuration-driven report system. A definition can +declare: + +- a SQL source +- named optional filter predicates +- named sort orders +- list and label layouts +- page, line, margin, repetition, and format settings +- screen or spreadsheet output + +That is conceptually close to a GovOPlaN combination of a Dataflow template, +typed output template, and Report/Campaign run configuration. + +## Legacy Limitations We Should Not Reproduce + +- users, departments, permissions, query variants, and output paths are + hard-coded +- credentials are embedded in source files +- request values are interpolated into SQL and shell commands +- authentication, authorization, session transfer, and application routing are + coupled +- selection names do not carry version, owner, purpose, legal basis, or + provenance +- past output cannot be reproduced reliably after source data or query code + changes +- exclusions and exceptional recipients are hidden inside SQL +- there is no first-class consent, suppression, address-quality, or duplicate + resolution model +- mail generation relies on client-side `mailto` BCC links +- print operations have weak status, retry, cancellation, and evidence + handling +- temporary files and generated artifacts are not governed records +- the UI does not explain why a recipient was included, excluded, or + inaccessible + +## Current GovOPlaN Coverage + +| Capability | Current state | Primary owner | +| --- | --- | --- | +| Scoped address books and contacts | Implemented | Addresses | +| Multiple email, phone, and postal contact points | Implemented | Addresses | +| Static address lists | Implemented | Addresses | +| vCard import/export and CardDAV sync | Implemented | Addresses | +| Read-only lookup and immutable email-recipient snapshots | Implemented | Addresses | +| Source provenance and sync conflicts | Implemented | Addresses | +| Governed source catalogue, snapshots, and frozen states | Implemented first slice | Datasources | +| Connector-backed tabular acquisition and preview | Implemented first slice | Connectors | +| Graphical/SQL transformations, joins, filters, expressions, quality, and reconciliation | Implemented broad first slice | Dataflow | +| Campaign snapshots, review, mail delivery, and evidence | Implemented for mail | Campaign/Mail | +| Typed template library and document rendering | Scaffold/issue only | Templates | +| BI reports and spreadsheet-grade exports | Scaffold/issue only | Reporting | +| Mixed operational distribution lists | Contract/implementation issues only | Distribution Lists | +| Dynamic address segments, consent, suppression, and preferences | Open issue | Addresses | +| Deduplication, merge, quality, and complete history | Open issue | Addresses | +| Postal recipient resolution and channel-neutral snapshots | Missing | Addresses | +| Direct governed print jobs | Missing | Future Print module plus connector providers | +| End-to-end guided AdreMa workflow | Missing | Workflow plus consuming modules | + +## Target Architecture + +### 1. Canonical Parties And Contact Points + +Addresses should model locally managed parties while linking, rather than +duplicating, authoritative IDM and Organizations records. + +Required additions: + +- party kind: person, organization, household, group, or location +- formal and display-name components, locale, title, and salutation +- optional stable references to identity, organization, unit, and function + providers +- typed relationships such as member, employee, office holder, household + member, assistant, or care-of contact +- effective dates and provenance on relationships and contact points +- postal fields for care-of/addressee lines, PO boxes, country codes, internal + mail routes, and delivery instructions +- purpose and preference metadata for private, business, service, + correspondence, billing, and other address uses + +The model must remain usable when IDM or Organizations is absent. Optional +provider links enrich a local party; they do not make those modules hard +dependencies. + +### 2. Governed Segments + +Addresses owns the address-domain segment definition and user experience. +Dataflow evaluates complex tabular predicates, while Datasources and Connectors +provide governed inputs. + +A segment needs: + +- immutable definition revisions +- system, tenant, group, and user scope +- complete-flow versus reusable-template semantics +- parameters with types, defaults, constraints, and permitted overrides +- explicit include, exclude, and manual-override sets +- source fingerprints and freshness requirements +- preview, count, sample, and "why included/excluded" diagnostics +- schedule/event refresh policy where materialization is wanted +- owner, purpose, legal basis, retention, and policy provenance +- materialized/frozen recipient snapshots for reproducible use + +Simple tag/contact filters can execute in Addresses. Cross-source joins and +complex expressions should compile to a Dataflow revision rather than creating +a second query engine. + +### 3. Recipient Resolution + +The current recipient-source DTO is email-oriented. AdreMa parity needs a +versioned channel-neutral contract that can resolve: + +- email targets +- postal targets +- internal-mail targets +- postbox targets where Postbox is installed +- unresolved or blocked targets with stable reason codes + +Resolution must consider: + +- requested channel and communication purpose +- effective date +- preferred address/contact point +- explicit address-purpose selection and fallback rules +- locale and international postal formatting +- consent, legal basis, suppression, and policy decisions +- duplicate targets and household/organization grouping +- source revision, selected contact-point ID, and complete provenance + +Consumers freeze the resolution result. They never retain only a live pointer +to an address list or segment. + +### 4. Static And Operational Lists + +The existing split remains correct: + +- Addresses owns address-only lists and address-domain segments. +- Distribution Lists owns mixed operational `Verteiler` containing address + records, identities, users, groups, organization units, functions, roles, + raw targets, and nested distribution lists. +- Workflow/Tasks owns an `Umlauf` with ordering, state, deadlines, escalation, + and decisions. + +Distribution-list entries need stable IDs. A consumer stores an immutable +expansion snapshot. Consumer-specific enrichment belongs to that consumer; +reusable derived lists are separate, revision-pinned definitions. + +### 5. Templates, Artifacts, And Delivery + +Templates should provide typed render contracts for: + +- labels and label sheets +- envelopes +- form and serial letters +- email and postbox messages +- list/report layouts + +Templates declare required fields and output capabilities. Rendering produces a +managed Files artifact with template revision, input snapshot hash, renderer +version, page/count summary, and diagnostics. + +Campaign should orchestrate postal batches in the same way it orchestrates +mail: freeze recipients, validate required fields, review exceptions, render, +approve, deliver or export, reconcile outcomes, and retain evidence. + +Direct physical printing deserves a dedicated domain boundary once implemented: + +- Print owns printers, queues, jobs, status, cancellation, retry, accounting, + secure release, and job evidence. +- Connectors supplies IPP/CUPS, Windows print-server, or provider-specific + connection profiles. +- Campaign, Templates, Files, and Workflow consume a versioned print capability. + +Downloading a generated PDF is a valid deployment without Print installed. +Matching AdreMa's direct central-print behavior requires Print. + +### 6. Governance And Quality + +Parity is not sufficient without: + +- purpose- and channel-specific consent/legal-basis records +- do-not-contact and returned/invalid-address suppression +- normalization and validation with original-value preservation +- duplicate candidates, survivorship rules, manual merge/split, and undo + evidence +- field-level provenance and source-of-truth precedence +- stale-source and stale-segment warnings +- field/row/export policy decisions with user-readable explanations +- complete audit events for read, export, resolution, merge, rendering, and + delivery +- retention, legal hold, and deletion behavior for source records and frozen + output evidence + +### 7. Guided User Experience + +The default journey should be: + +1. Choose or create an address list/segment. +2. Preview the authorized audience and inspect included, excluded, duplicate, + invalid, and suppressed counts. +3. Select channel, address purpose/fallback, sort, and grouping. +4. Freeze a recipient snapshot. +5. Select a compatible template/output format. +6. Review individual exceptions and approve the run. +7. export, send, or print through available capabilities. +8. inspect progress, outcomes, returned items, and evidence. + +Workflow may package this as a reusable guided process and activate a focused +View. Every underlying resource must remain independently accessible and +permission-checked. + +## Representative Golden Flow + +Before migrating any historical selection, build a synthetic AdreMa fixture +covering: + +- people, organizations, units, functions, and effective-dated assignments +- multiple postal-purpose variants and internal mail +- a function-based segment +- an organization-unit segment +- a geography-based segment +- a union of segments +- explicit inclusion and exclusion +- duplicate and suppressed recipients +- deterministic sorting and address fallback +- frozen postal and email snapshots +- label, spreadsheet, and letter output expectations + +The fixture should assert exact recipient IDs, selected contact-point IDs, +ordering, exclusion reasons, source fingerprints, and output hashes. It must +contain no archived record-level data. + +## Migration Strategy + +Do not translate the legacy source one query branch at a time into application +code. + +1. Inventory each still-relevant named selection with owner, purpose, legal + basis, source systems, parameters, output type, and expected use frequency. +2. Retire obsolete and duplicate selections before migration. +3. Map source tables to governed Connector/Datasource definitions. +4. Convert reusable predicates into Dataflow templates or simple + Addresses-owned filters. +5. compare the new result against an authorized legacy run using counts and + pseudonymized stable keys. +6. approve and version the new segment. +7. migrate output layouts into typed Templates. +8. retain a migration report and decommission the corresponding legacy path. + +## Recommended Delivery Order + +### Priority 1: Functional Parity Foundation + +1. Typed party, relationship, address-purpose, and effective-date model. +2. Channel-neutral recipient resolution with postal snapshots. +3. Dynamic governed segments through Dataflow/Datasources. +4. Synthetic AdreMa golden-flow fixture. +5. Typed label, envelope, and letter templates with managed artifacts. + +### Priority 2: Operational Parity + +1. Campaign postal-batch review and output flow. +2. Spreadsheet/report exports with exact frozen-snapshot evidence. +3. Consent, suppression, preferences, deduplication, merge, and quality UI. +4. CSV/XLSX and directory/database migration profiles. +5. Distribution-list expansion and Campaign integration. + +### Priority 3: Beyond AdreMa + +1. Direct governed print jobs and secure release. +2. returned-mail and correction workflows. +3. postal-provider/franking integration and delivery reconciliation. +4. scheduled/event-driven segment refresh and recurring campaigns. +5. address analytics, geocoding where policy permits, and quality trends. + +## Completion Standard For The Umbrella Feature + +The AdreMa umbrella is complete when a user can define or reuse a permissioned +audience, explain and freeze the exact recipients, create reviewed label/letter +or digital outputs, send/export/print through installed modules, and later +prove exactly which source data, rules, template, actor, and delivery outcome +were involved. The same flow must degrade cleanly when optional modules such as +Mail, Campaign, Templates, Workflow, Reporting, or Print are absent.