Release govoplan-mail v0.1.27: stabilize credentials, folder encoding and transport progress

docs/MAIL_HANDBOOK.md

@@ -114,6 +114,16 @@ profile/server. Empty roles retain automatic behavior. The historical
 for compatibility; a Campaign-specific Sent override still wins for that
 Campaign.
 
+Use readable Unicode names such as `Entwürfe`, not IMAP wire encodings such as
+`Entw&APw-rfe`. Discovery decodes modified UTF-7 before detecting standard
+folder roles; SELECT, STATUS, and Sent APPEND encode and quote the chosen name
+for the active connection. Literal ampersands, quotes, backslashes, and Unicode
+characters round-trip without renaming remote folders. Previously saved
+wire-form names are resolved against that account's live folder list; if the
+same string is also an actual readable folder name, the readable name wins.
+Rediscovery and an explicit profile save replace old encoded configuration
+values with readable names; background reads never rewrite configuration.
+
 Profiles may be scoped to system, tenant, user, group, or campaign context.
 Scope controls where a profile can be discovered; effective policy can narrow
 that further. A visible profile is not automatically authorized for every
@@ -191,6 +201,19 @@ non-production provider and mailbox first. A successful connection test does
 not prove policy authorization for a later Campaign context, deliverability,
 recipient acceptance, SPF/DKIM/DMARC alignment, or future availability.
 
+Campaign runtime authorization follows the protocol being used: SMTP batch and
+single-message calls enforce the explicit SMTP credential policy, while
+append-to-Sent enforces the IMAP credential policy. A valid SMTP call does not
+need to carry an unrelated IMAP credential just because the profile supports
+both protocols. Full campaign authoring validation and complete profile
+summaries continue to require both configured selections when their policies
+forbid inherited credentials. The selected protocol's missing credential,
+inactive/unauthorized binding or stale transport revision still stops the
+operation before decryption or provider contact. Resolving a policy rejection
+never requires disabling TLS or weakening either credential policy. Correcting
+this runtime check does not change stored configuration or approved builds,
+reset job state, or retry/send messages automatically.
+
 Testing a saved profile requires both `mail:profile:test` and
 `mail:profile:use`, and the profile must be active. Profile creation or test
 authority alone is not enough.
@@ -201,10 +224,46 @@ ordinary consumers to bypass reusable profiles.
 
 ### Read a mailbox
 
+The persistent workspace header contains the profile selector, **Mailbox tools**,
+Help, and one right-aligned **Reload**. These controls remain available when the
+mailbox is empty or no usable profile is configured. Reload rechecks the permitted
+profiles and refreshes the current folder catalogue and bounded message page;
+it also rereads a selected message if that message remains on the page. IMAP
+keeps the page offset. JMAP starts a fresh cursor chain at page one while keeping
+the search term. An unavailable profile is replaced only by another currently
+authorized active profile, or the explicit no-profile state.
+
+Folder icons expand or collapse; labels select. Synthetic grouping labels select
+the group without reading a nonexistent provider folder, and Reload in that
+state refreshes only the folder catalogue. Refreshes keep user-controlled
+expansion. A failed refresh preserves usable loaded data, shows the error, and
+leaves Reload available for retry; late responses from an earlier profile or
+tenant cannot replace the current context.
+Mailbox context reads bypass browser response/promise reuse so an immediate
+Reload really rechecks permissions and state. Mail's bounded server-side index
+is unchanged; its explicit refresh flag and live/cached provenance still apply.
+Failed pagination restores the page and page-size labels belonging to retained
+rows. Dismissing or changing the preview while Reload is pending takes
+precedence over its remembered selection.
+On narrow screens, scroll vertically through folders, message list, and preview
+within the mailbox workspace; the profile/tools/Reload header remains visible.
+
+**Mailbox tools** groups the occasional profile-only, folder-only, and
+message-only refreshes in a dialog, separate from **Bounce status**. Bounce
+status requires `mail:bounce:read` or `mail:bounce:manage`; without either it
+remains visible and disabled with an explanation. Escape closes this dialog
+without clearing the selected message. These read controls do not grant profile
+administration, send SMTP messages, APPEND messages, or change mailbox flags.
+
 The current mailbox UI and API are read-only. An authorized user can list IMAP
 or JMAP folders, page through messages, and inspect a bounded full message.
-IMAP folder names are parsed and quoted defensively; Sent-folder discovery uses
-provider flags and common names. JMAP discovers the Session and Mail account,
+IMAP folder names are decoded for display and encoded and quoted defensively
+for mailbox commands; Sent-folder discovery uses provider flags and common
+readable names. The default IMAP4rev1 mode uses modified UTF-7. A connection
+that has explicitly enabled `UTF8=ACCEPT` uses UTF-8 instead; merely advertising
+that capability does not change encoding. Invalid provider encodings produce
+an explicit error rather than a replacement name that could address another
+folder. Refresh any already-loaded folder list after upgrading. JMAP discovers the Session and Mail account,
 uses `Mailbox/get` hierarchy and roles, runs text search with `Email/query`, and
 uses `Email/get` for bounded summaries/details. `Email/changes` exposes a
 bounded incremental cursor; an expired state tells the caller to perform a full
@@ -214,6 +273,9 @@ page came directly from the provider, from the bounded mailbox index, or from
 an index while a refresh is in progress, including the index timestamp when
 available.
 
+The mailbox-name boundary follows [RFC 3501 section 5.1.3](https://www.rfc-editor.org/rfc/rfc3501.html#section-5.1.3)
+and, only after explicit activation, [RFC 6855 section 3](https://www.rfc-editor.org/rfc/rfc6855.html#section-3).
+
 Message HTML is displayed only in the shared sandboxed message component.
 Remote URLs and active markup are removed, embedded `data:`/`cid:` image
 references remain isolated, and plain text is always available when supplied.
@@ -340,6 +402,19 @@ account's user scope. Grant `mail:profile:write_own` for self-service;
 An update that omits a password preserves the current encrypted password. A
 credential replacement never depends on reading the old cleartext value back.
 
+The shared credential editor resolves Mail server restrictions from the
+authorized metadata catalogue when it opens. Names appear as loading completes;
+no page refresh is required, and typing in a draft does not reload the catalogue.
+Closing and reopening refreshes the available servers and can retry a temporary
+metadata failure. Inactive servers remain labelled inactive; deleted or
+unauthorized references remain visible as unavailable and are never silently
+removed from the credential. Labels do not grant permission to use a server,
+and the lookup does not retrieve secrets.
+If saving a reusable credential fails, the editor shows the error beside the
+unchanged draft. Retry explicitly after correcting the cause. Saving disables
+editing and closing until the request finishes; a failed save never silently
+discards a replacement secret that has not been stored.
+
 ### Delete a profile
 
 Profile deletion is immediate for Mail-owned secrets and audit evidence:
@@ -368,13 +443,32 @@ Effective policy is contextual. Administrators should document:
 - allowed and denied SMTP/IMAP/JMAP hosts;
 - permitted From, envelope sender (including bounce address), and envelope
   recipient-domain patterns;
-- whether SMTP/IMAP credentials inherit from the reusable profile; and
+- whether SMTP/IMAP may use a default credential or require an explicit Mail-owned credential selection; and
 - which lower-level settings are locked by a parent policy.
 
-Campaign delivery requires reusable profile credentials. A legacy policy that
-requires campaign-local credentials fails closed with guidance to store them on
-the Mail profile and enable effective inheritance. This preserves compatibility
-of the policy model without reopening a consumer-owned secret store.
+In **Mail profile policy → Credential selection**, SMTP and IMAP have separate
+controls. **Allow profile default credential** (`inherit: true`) permits either
+the selected server's default credential or an explicit authorized Mail-owned
+credential. **Require explicit Mail credential** (`inherit: false`) requires a
+server and credential reference in Campaign Mail settings. Neither option
+permits campaign-local passwords or copies a secret into Campaign.
+
+**Inherit policy from parent** leaves the local value unset; it is different
+from allowing a server's default credential. System policy always has a concrete
+choice. Other scopes show the local choice alongside the saved effective result
+and policy path. **Allow override** controls
+`allow_lower_level_limits["smtp_credentials.inherit"]` and the equivalent IMAP
+key. It is not a separate `allow_override` field in the credential object. A
+parent's explicit-credential requirement may be changed by a child only while
+that parent allows overrides. Locked fields and their override controls remain
+read-only; a lower scope cannot unlock them. Campaign policy has no lower-level
+override controls. Editing any policy still requires that scope's policy-write
+permission and an unlocked workflow.
+
+Policy saves retain their draft after a failed write and require an explicit
+retry. If the policy was saved but a dependent screen refresh fails, the editor
+reports that the policy was saved and advises reloading the display; it does not
+report a failed save or repeat the accepted write.
 
 Policy reads are available through system/tenant/context routes to suitably
 authorized actors. Adaptive Docs exposes a safe explanation of the effective
@@ -451,6 +545,78 @@ treated as an unknown provider mutation.
    confirmed absence records verified recovery and permits only a new,
    deliberate attempt identifier.
 
+### Bounded IMAP append batches / Begrenzte IMAP-Ablagestapel
+
+Campaign's bulk Sent-folder operation can use the optional
+`mail.campaign_delivery.campaign_imap_batch(tenant_id=..., campaign_id=...)`
+context. Opening this context has no provider effect. Mail opens a connection
+only after an individual message passes its current authorization, selected
+IMAP credential policy, both frozen transport revisions and durable recovery
+checks. Subsequent messages reuse that authenticated connection and its detected
+Sent folder, including the provider's original Unicode mailbox wire encoding.
+Authorization and credential resolution are performed for every message, not
+cached. Changing the authorized profile, selected references, folder or resolved
+credentials releases the previous connection before the next APPEND.
+
+Each message still receives one sequential APPEND and its own recovery evidence.
+There is no parallel APPEND, MULTIAPPEND, automatic SMTP resend or replay after
+APPEND starts. A lost APPEND reply remains outcome-unknown and requires explicit
+mailbox reconciliation. Failures while connecting, before any APPEND, may use a
+bounded reconnect; rejected authentication is not retried. A failure to finalize
+accepted recovery evidence closes the batch. Cleanup/logout failure does not
+turn an accepted APPEND into a failed one. Session state is scoped to the current
+batch, never shared across tenants or campaigns, and is released on exit.
+
+Deployment controls apply to batch connection reuse, not campaign authorization:
+
+| Environment variable | Default | Range / effect |
+| --- | --- | --- |
+| `GOVOPLAN_IMAP_BATCH_REUSE` | `true` | `false`, `0`, `no` or `off` disables reuse. |
+| `GOVOPLAN_IMAP_BATCH_MAX_MESSAGES` | `100` | 1–10,000 successful APPENDs per connection. |
+| `GOVOPLAN_IMAP_BATCH_MAX_AGE_SECONDS` | `300` | 1–3,600 seconds; rotate before the next message, not during an APPEND. |
+| `GOVOPLAN_IMAP_BATCH_IDLE_HEALTH_CHECK_SECONDS` | `30` | 0–3,600 seconds idle before a NOOP; 0 checks every reuse. |
+| `GOVOPLAN_IMAP_BATCH_RECONNECT_ATTEMPTS` | `1` | 0–5 extra connection attempts, only before APPEND. |
+
+Invalid numeric values use defaults; out-of-range numbers are clamped. An active
+session keeps the policy with which it was created. Existing single-message
+callers retain one connection per call and no automatic connection retries.
+Older Mail capabilities without this optional context keep the single-message
+behavior. Safe outcome fields include connection sequence, session reuse and
+reconnect count; they contain no hosts, credentials, provider responses or MIME
+content. Fewer logins and folder discoveries improve connection overhead, not
+the provider's intrinsic per-message APPEND or durable-evidence latency.
+
+Deutsch: Die Sammelablage im Gesendet-Ordner kann eine begrenzte authentifizierte
+IMAP-Verbindung wiederverwenden. Der Stapelkontext allein verbindet sich nicht.
+Vor jeder Nachricht prüft Mail erneut Berechtigung, IMAP-Zugangsdatenrichtlinie,
+beide eingefrorenen Transportrevisionen und Wiederherstellungsnachweise; die
+Zugangsdaten werden weiterhin je Nachricht aufgelöst. Profil, ausgewählte
+Referenzen, Ordner oder aufgelöste Zugangsdaten dürfen nicht stillschweigend von
+einer älteren Verbindung übernommen werden. Ordnererkennung und ursprüngliche
+Provider-Kodierung bleiben ausschließlich an dieselbe Verbindung gebunden.
+
+Jede Nachricht erhält weiterhin einen einzelnen, sequenziellen APPEND und einen
+eigenen Nachweis. Es gibt kein paralleles APPEND, kein MULTIAPPEND und keine
+automatische Wiederholung nach Beginn von APPEND. Ein unbekanntes Ergebnis muss
+am Postfach abgeglichen werden; eine fehlende Gesendet-Kopie darf keinen erneuten
+SMTP-Versand auslösen. Nur Verbindungsaufbau vor APPEND darf begrenzt wiederholt
+werden, nicht eine abgelehnte Anmeldung. Ein Fehler beim Abschluss des
+Wiederherstellungsnachweises schließt den Stapel; ein reiner Abmeldefehler macht
+eine bestätigte Ablage nicht rückgängig. Mandanten und Kampagnen teilen keine
+Stapelverbindung.
+
+Die obigen Betriebsvariablen bedeuten standardmäßig: Wiederverwendung aktiv,
+höchstens 100 Nachrichten bzw. 300 Sekunden pro Verbindung, NOOP nach 30 Sekunden
+Leerlauf und höchstens einen zusätzlichen Verbindungsversuch vor APPEND. Der
+Wechsel erfolgt vor der nächsten Nachricht, niemals mitten im APPEND. `0` beim
+Leerlaufintervall prüft jede Wiederverwendung; deaktivierte Wiederverwendung
+verwendet weiterhin einzelne APPENDs. Ungültige Zahlen verwenden den Standard,
+Zahlen außerhalb des Wertebereichs werden begrenzt. Eine aktive Verbindung
+behält ihre beim Aufbau gelesene Richtlinie. Einzelaufrufe und ältere optionale
+Mail-Verträge bleiben kompatibel. Verbindungszähler enthalten keine Zugangsdaten
+oder Providerdetails. Die Optimierung spart Verbindungsaufbau und Ordnersuche;
+Provider-Ablage und dauerhafte Einzelnachweise benötigen weiterhin ihre Zeit.
+
 ### Delivery-status and calendar-reply sources
 
 An authorized Mail bounce source scans a bounded IMAP UID range without