govoplan-dist-lists/docs/ADREMA_CAPABILITY_ASSESSMENT.md

# AdreMa Capability Assessment And Distribution-List Roadmap

## Purpose

This document records a behavior-level assessment of the historical AdreMa
application supplied in `/mnt/DATA/tmp.tar`. The goal is not to reproduce its
implementation or UI. The goal is to preserve the useful administrative
capabilities while replacing hard-coded queries and output paths with
governed, reusable GovOPlaN contracts.

The archive was inspected as untrusted historical material. No archived code
was executed. Generated documents and record-level output were excluded from
the analysis.

## Evidence And Handling

- Archive SHA-256:
  `328edbaf05df084f9c63cc3b9b36f7c5ae0c639942be4b7ad830efc0905a311b`
- Compressed size: about 1.49 GB
- Inventory: 8,966 files and 190 directories
- Uncompressed payload: about 1.79 GB
- The AdreMa selection UI contains 258 distinct segment variables, 265 query
  cases, 13 sort definitions, seven action definitions, and 24
  department-specific forms.
- The adjacent configurable list/report layer contains 47 ETI definitions.
- The wider print-server archive contains 127 SQL files, 181 XSLT files, and 82
  Python workflow sources.

The archive contains generated person-level documents and embedded
credentials. It must not be committed, copied into test fixtures, or treated as
safe sample data. Any credential that could still be valid must be rotated.
Future fixtures derived from this assessment must be synthetic.

## What AdreMa Actually Is

AdreMa is not primarily a master address database. It is a role-specific
audience selection and output application over authoritative institutional
databases.

Its main flow is:

1. Authenticate against a directory.
2. Show a menu of saved recipient selections permitted for the current user or
   department.
3. Run a fixed SQL selection against organization, person, employment,
   function, student, geography, and address data.
4. Apply a selected sort order and address variant.
5. Produce a table, spreadsheet, label preview, printable label file, or BCC
   mail draft.
6. Record a basic process log and recipient count.

The broader print-server application adds SQL-to-XML extraction, XSLT
transformation, HTML/Excel output, TeX/PDF generation, batch letters, PDF
protection, and deployment-specific printer submission.

This means an AdreMa replacement should not be one large Addresses feature.
Distribution Lists is the core user-facing domain for audience definitions,
expansion, and reusable `Verteiler`. It composes capabilities from Addresses,
Organizations/IDM, Datasources, Connectors, and Dataflow, then supplies frozen
results to Templates, Campaign, Files, Audit, Policy, Reporting, and Workflow.
No separate Print module is required: Templates owns printable output and an
optional printer endpoint is an export destination, not a new party or audience
domain.

## Observed Functional Model

### Sources And Identities

The application reads live authoritative data rather than maintaining a
separate copy. Its selections use:

- people and institutional units
- functions and role assignments
- employment and student state
- effective dates and exclusions
- internal and external organizations
- geography, country, and postal-code ranges
- source-specific person, registration, and unit identifiers

Access to selections is hard-coded by user and department. Administrative
users receive the union of those menus.

### Contact And Address Semantics

Observed recipient fields include:

- display name, given name, family name, title, and formal salutation
- organization/unit and function context
- phone, fax, email, and URL
- `care of`/`for the attention of` lines
- street, postal code, locality, country, and internal mail routing
- person, registration, and organizational-unit identifiers
- address purpose variants such as private, semester, home, business, service,
  and correspondence address
- effective employment or membership data used to decide eligibility

The legacy system also allows a requested number of labels for an individual
recipient. That quantity belongs to an output job, not to the canonical address
record.

### Recipient Selection

Selections cover several distinct patterns:

- static categories of institutional units
- people assigned to a function or organizational level
- current-state selections with effective-date rules and exclusions
- geographic and demographic selections
- combinations and unions of predefined audiences
- explicit person or unit lookup by source identifier
- department-specific variants of a shared audience
- user-selected additional predicates and sort orders in the adjacent ETI
  report layer

The important reusable abstraction is therefore a versioned, parameterized,
permissioned segment definition. The 265 query branches must not become 265
new backend endpoints.

### Output

The directly observed output modes are:

- on-screen recipient table with contact and postal fields
- spreadsheet export
- plain-text label preview
- generated label file
- emailed label file
- printable label output
- BCC mail draft

The adjacent print-server flows additionally produce:

- formatted lists and grouped reports
- personalized letters and notices
- HTML, spreadsheet-compatible, TeX, PostScript, and PDF artifacts
- printer-specific layouts and deployment-specific print submission
- password-protected output and combined document bundles

### Configuration

The ETI layer is an early configuration-driven report system. A definition can
declare:

- a SQL source
- named optional filter predicates
- named sort orders
- list and label layouts
- page, line, margin, repetition, and format settings
- screen or spreadsheet output

That is conceptually close to a GovOPlaN combination of a Dataflow template,
typed output template, and Report/Campaign run configuration.

## Legacy Limitations We Should Not Reproduce

- users, departments, permissions, query variants, and output paths are
  hard-coded
- credentials are embedded in source files
- request values are interpolated into SQL and shell commands
- authentication, authorization, session transfer, and application routing are
  coupled
- selection names do not carry version, owner, purpose, legal basis, or
  provenance
- past output cannot be reproduced reliably after source data or query code
  changes
- exclusions and exceptional recipients are hidden inside SQL
- there is no first-class consent, suppression, address-quality, or duplicate
  resolution model
- mail generation relies on client-side `mailto` BCC links
- output submission has weak status and evidence handling
- temporary files and generated artifacts are not governed records
- the UI does not explain why a recipient was included, excluded, or
  inaccessible

## Current GovOPlaN Coverage

| Capability | Current state | Primary owner |
| --- | --- | --- |
| Scoped address books and contacts | Implemented | Addresses |
| Multiple email, phone, and postal contact points | Implemented | Addresses |
| Static address lists | Implemented | Addresses |
| vCard import/export and CardDAV sync | Implemented | Addresses |
| Read-only lookup and immutable email-recipient snapshots | Implemented | Addresses |
| Source provenance and sync conflicts | Implemented | Addresses |
| Organization units and function definitions | Implemented | Organizations |
| Effective-dated identity-to-function assignments | Implemented first slice | IDM |
| Governed source catalogue, snapshots, and frozen states | Implemented first slice | Datasources |
| Connector-backed tabular acquisition and preview | Implemented first slice | Connectors |
| Graphical/SQL transformations, joins, filters, expressions, quality, and reconciliation | Implemented broad first slice | Dataflow |
| Campaign snapshots, review, digital delivery, and evidence | Implemented first channel | Campaign |
| Typed template library and document rendering | Scaffold/issue only | Templates |
| BI reports and spreadsheet-grade exports | Scaffold/issue only | Reporting |
| Mixed operational distribution lists and governed segments | Contract/implementation issues only | Distribution Lists |
| Contact-point consent, suppression, and preferences | Open issue | Addresses/Policy |
| Deduplication, merge, quality, and complete history | Open issue | Addresses |
| Channel-neutral expansion and frozen postal targets | Missing | Distribution Lists/Addresses |
| Printable label, envelope, and letter output | Open issue | Templates |
| End-to-end guided AdreMa workflow | Missing | Workflow plus consuming modules |

## Target Architecture

### 1. Identities, Groups, Functions, And Contact Points

The authoritative subject model is shared deliberately:

- Organizations owns tenant-local units, structures, and function definitions.
- IDM owns identities' effective-dated assignments to those functions and the
  typed group/membership relationships used as audience sources.
- Addresses owns contact records and contact points, including formal
  addressing fields, postal purposes, validity, preference, and provenance.
- Distribution Lists stores stable provider references and expansion evidence;
  it does not copy any provider's master records.

An IDM identity status is a lifecycle indicator such as active, suspended, or
retired. A business status used for selection is represented by a typed group,
function, or effective-dated relationship. The two meanings must not share one
field.

Addresses still needs postal fields for care-of/addressee lines, PO boxes,
country codes, internal-mail routes, and delivery instructions. It may preserve
vCard `KIND`/`RELATED` data for round-trip compatibility, but it must not become
a parallel party or organizational-relationship authority.

Every provider remains optional. A deployment without IDM can still use local
address contacts and raw targets; a deployment without Addresses can still
expand identities, groups, functions, and provider-native targets.

### 2. Governed Distribution Lists And Segments

Distribution Lists owns reusable audience definitions and their user
experience. Dataflow evaluates complex tabular predicates, while Datasources
and Connectors provide governed inputs. Addresses may expose simple contact
search/filter capabilities, but a reusable segment is a Distribution List
definition.

A segment needs:

- immutable definition revisions
- system, tenant, group, and user scope
- complete-flow versus reusable-template semantics
- parameters with types, defaults, constraints, and permitted overrides
- explicit include, exclude, and manual-override sets
- source fingerprints and freshness requirements
- preview, count, sample, and "why included/excluded" diagnostics
- schedule/event refresh policy where materialization is wanted
- owner, purpose, legal basis, retention, and policy provenance
- materialized/frozen recipient snapshots for reproducible use

Simple tag/contact predicates may be delegated to Addresses. Cross-source joins
and complex expressions compile to a pinned Dataflow revision rather than
creating a second query engine in Distribution Lists.

### 3. Expansion And Channel Resolution

Distribution Lists orchestrates a versioned, channel-neutral expansion
contract. Provider capabilities contribute identities, groups, function
incumbents, contact-point candidates, or tabular rows. The expansion can
resolve:

- email targets
- postal targets
- internal-mail targets
- digital portal targets when Campaign has such a delivery provider
- unresolved or blocked targets with stable reason codes

Resolution must consider:

- requested channel and communication purpose
- effective date
- preferred address/contact point
- explicit address-purpose selection and fallback rules
- locale and international postal formatting
- channel opt-in/preference, consent, legal basis, suppression, and Policy
  decisions
- duplicate targets and household/organization grouping
- source revision, selected contact-point ID, and complete provenance

Addresses owns contact-point preference/consent facts; Policy decides what is
permitted and explains provenance; Distribution Lists records the applied
decision in its expansion result. Campaign then applies explicit hybrid-channel
routing and freezes the result. A future Portal self-service surface may edit
permitted preferences through these same contracts and receive Campaign output,
but it is not required for the core AdreMa implementation.

Consumers never retain only a live pointer to a list or segment.

### 4. Static And Operational Lists

The existing split remains correct:

- Addresses owns address books and simple classical groupings of contacts or
  contact points.
- Distribution Lists owns static and dynamic operational `Verteiler` containing
  address records, IDM identities and typed groups, organization units,
  functions/effective incumbents, raw targets, Dataflow-backed rows, and nested
  distribution lists.
- Workflow/Tasks owns an `Umlauf` with ordering, state, deadlines, escalation,
  and decisions.

Distribution-list entries need stable IDs. A consumer stores an immutable
expansion snapshot. Consumer-specific enrichment belongs to that consumer;
reusable derived lists are separate, revision-pinned definitions.

### 5. Templates, Artifacts, And Delivery

Templates should provide typed render contracts for:

- labels and label sheets
- envelopes
- form and serial letters
- email and other digital messages
- list/report layouts

Templates declare required fields and output capabilities. Rendering produces a
managed Files artifact with template revision, input snapshot hash, renderer
version, page/count summary, and diagnostics.

Campaign should orchestrate postal batches with the same lifecycle as other
delivery channels: freeze recipients, validate required fields, review
exceptions, render, approve, deliver or export, reconcile outcomes, and retain
evidence.

Templates owns printable content and layout. Files persists approved artifacts,
Campaign owns batch review and channel routing, and Audit records the evidence.
Downloading or handing a generated PDF to the operating system is a complete
supported path. If a deployment later needs an IPP/CUPS or managed print-server
destination, Connectors can expose it as a governed artifact export provider;
that does not require a separate Print module.

Hybrid campaigns may choose email, printable postal output, or a portal-capable
delivery target per recipient. Opt-in is an input to the Policy decision, never
an implicit instruction to duplicate delivery across channels. Campaign must
freeze the chosen route, fallback rule, and reason for every recipient.

### 6. Governance And Quality

Parity is not sufficient without:

- purpose- and channel-specific consent/legal-basis records
- do-not-contact and returned/invalid-address suppression
- normalization and validation with original-value preservation
- duplicate candidates, survivorship rules, manual merge/split, and undo
  evidence
- field-level provenance and source-of-truth precedence
- stale-source and stale-segment warnings
- field/row/export policy decisions with user-readable explanations
- complete audit events for read, export, resolution, merge, rendering, and
  delivery
- retention, legal hold, and deletion behavior for source records and frozen
  output evidence

### 7. Guided User Experience

The default journey should be:

1. Choose or create a distribution list/segment.
2. Preview the authorized audience and inspect included, excluded, duplicate,
   invalid, and suppressed counts.
3. Select channel, address purpose/fallback, sort, and grouping.
4. Freeze a recipient snapshot.
5. Select a compatible template/output format.
6. Review individual exceptions and approve the run.
7. Export or deliver through available capabilities.
8. inspect progress, outcomes, returned items, and evidence.

Workflow may package this as a reusable guided process and activate a focused
View. Every underlying resource must remain independently accessible and
permission-checked.

## Representative Golden Flow

Before migrating any historical selection, build a synthetic AdreMa fixture
covering:

- people, organizations, units, functions, and effective-dated assignments
- multiple postal-purpose variants and internal mail
- a function-based segment
- an organization-unit segment
- a geography-based segment
- a union of segments
- explicit inclusion and exclusion
- duplicate and suppressed recipients
- deterministic sorting and address fallback
- frozen postal and email snapshots
- label, spreadsheet, and letter output expectations

The fixture should assert exact recipient IDs, selected contact-point IDs,
ordering, exclusion reasons, source fingerprints, and output hashes. It must
contain no archived record-level data.

## Migration Strategy

Do not translate the legacy source one query branch at a time into application
code.

1. Inventory each still-relevant named selection with owner, purpose, legal
   basis, source systems, parameters, output type, and expected use frequency.
2. Retire obsolete and duplicate selections before migration.
3. Map source tables to governed Connector/Datasource definitions.
4. Convert reusable predicates into Distribution List definitions backed by
   pinned Dataflow templates or provider-native predicates.
5. Compare the new result against an authorized legacy run using counts and
   pseudonymized stable keys.
6. Approve and version the new segment.
7. Migrate output layouts into typed Templates.
8. Retain a migration report and decommission the corresponding legacy path.

## Recommended Delivery Order

### Priority 1: Functional Parity Foundation

1. Distribution-list DTOs, versioned definitions, expansion, and snapshot
   contracts.
2. Typed IDM groups and effective-dated identity/function relationships, with
   identity lifecycle status kept separate.
3. Channel-neutral contact-point candidates and preference/consent facts from
   Addresses, evaluated through Policy.
4. Dynamic governed segments through Dataflow, Datasources, and Connectors.
5. Synthetic AdreMa golden-flow fixture.
6. Typed label, envelope, and letter templates with managed artifacts.

### Priority 2: Operational Parity

1. Campaign hybrid-channel review, routing, and printable-output flow.
2. Spreadsheet/report exports with exact frozen-snapshot evidence.
3. Consent, suppression, preferences, deduplication, merge, and quality UI.
4. CSV/XLSX and directory/database migration profiles.
5. Distribution-list expansion and Campaign integration.

### Priority 3: Beyond AdreMa

1. Portal self-service for permitted channel opt-in and digital receipt.
2. Returned-mail and correction workflows.
3. Postal-provider/franking integration and delivery reconciliation.
4. Scheduled/event-driven segment refresh and recurring campaigns.
5. Address analytics, geocoding where policy permits, and quality trends.

## Completion Standard For The Umbrella Feature

The AdreMa umbrella is complete when a user can define or reuse a permissioned
Distribution List, explain and freeze the exact recipients and chosen channels,
create reviewed label/letter or digital outputs, export or deliver through
installed modules, and later prove exactly which source data, rules, template,
actor, and delivery outcome were involved. The same flow must degrade cleanly
for every optional provider or consumer module combination.