Compare commits

...
165 Commits
Author SHA1 Message Date
zemion c51fc180fb Release govoplan-campaign v0.1.28: stabilize saving, review and delivery recovery
Module Package Release / publish-packages (push) Successful in 12s
2026-09-08 01:32:26 +02:00
zemion 1b32427813 fix(webui): bind consequential campaign controls to help
Module Package Release / publish-packages (push) Successful in 12s
2026-08-24 11:36:33 +02:00
zemion c21fb4cf7c docs: complete German structured documentation
Module Package Release / publish-packages (push) Successful in 12s
2026-08-24 01:15:32 +02:00
zemion b41f23c901 docs(campaign): complete German reference coverage
Module Package Release / publish-packages (push) Successful in 13s
2026-08-23 21:09:38 +02:00
zemion 3934e7fedb feat(campaigns): add portable campaign transfers
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 04:01:39 +02:00
zemion 1bd24f9b5b feat: orchestrate accountable Campaign work
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 02:14:35 +02:00
zemion 4f52f010ee fix(campaigns): declare work view surface
Module Package Release / publish-packages (push) Successful in 12s
2026-08-22 00:56:37 +02:00
zemion 2630498026 feat(campaigns): add accountable work assignments
Module Package Release / publish-packages (push) Successful in 14s
2026-08-22 00:50:03 +02:00
zemion c2f083e5f6 feat(campaigns): add governed collaboration thread
Module Package Release / publish-packages (push) Successful in 13s
2026-08-22 00:09:34 +02:00
zemion 5a21067e44 feat(campaigns): add Case Quick Access selector
Module Package Release / publish-packages (push) Successful in 13s
2026-08-21 16:05:45 +02:00
zemion 73cfad209a feat: add governed Campaign DSAR coverage 2026-08-20 23:15:11 +02:00
zemion 2a00d910df feat: govern autonomous campaign delivery schedules 2026-08-20 21:10:11 +02:00
zemion c846c249b8 feat: pause campaign batches on SMTP failure 2026-08-20 17:39:08 +02:00
zemion 69e6588a89 test: align campaign checks with archive governance 2026-08-20 17:39:03 +02:00
zemion 75e8f864a7 feat: govern legacy archive encryption 2026-08-20 12:24:36 +02:00
zemion d5b874c469 feat: select users in campaign access explanations 2026-08-20 06:16:43 +02:00
zemion 5c27527725 feat(campaign): explain governed child access 2026-08-20 05:38:00 +02:00
zemion 112ef9dc31 feat(campaign): govern attachment reuse 2026-08-20 05:20:07 +02:00
zemion f4fe534ee1 feat(campaign): record unmatched file disposition 2026-08-20 01:57:12 +02:00
zemion b6af7665f4 feat(campaign): complete reusable template workflow 2026-08-19 21:56:08 +02:00
zemion 14e94873a9 feat(security): explain Campaign child evidence 2026-08-19 20:27:42 +02:00
zemion 8e2f9d743d feat(webui): add campaign metric drill-downs 2026-08-19 18:47:45 +02:00
zemion 039ce35e78 Adopt semantic campaign page actions 2026-08-19 14:26:26 +02:00
zemion 7bc5e4a35c feat: align campaign with shared UI foundations 2026-08-18 21:32:34 +02:00
zemion 06f773e4eb Adopt shared WebUI structural primitives 2026-08-18 13:17:29 +02:00
zemion 9a2f13bc9a Adopt shared WebUI layout primitives 2026-08-18 11:30:39 +02:00
zemion cf8d7fab11 Adopt shared WebUI layout primitives 2026-08-18 10:42:51 +02:00
zemion 68459fde15 refactor: adopt shared page and workspace layouts 2026-08-18 02:17:16 +02:00
zemion c2efd6b7bd feat: add campaign copying scheduling and residual handling 2026-08-07 14:54:04 +02:00
zemion 696f8f6385 Release v0.1.18
Module Package Release / publish-packages (push) Successful in 13s
2026-08-05 21:07:44 +02:00
zemion 6562484d32 Release v0.1.17
Module Package Release / publish-packages (push) Successful in 12s
2026-08-05 20:33:58 +02:00
zemion 2db99eaf6a Release v0.1.16
Module Package Release / publish-packages (push) Successful in 13s
2026-08-05 19:51:59 +02:00
zemion ea0efe661f Release v0.1.15
Module Package Release / publish-packages (push) Successful in 11s
2026-08-04 15:18:11 +02:00
zemion 6bde11a286 Make package publication retries hash-safe 2026-08-04 14:32:18 +02:00
zemion 585493fe7a Harden module package publication 2026-08-04 14:02:39 +02:00
zemion 4133de86cd Enable generated campaign passwords 2026-08-04 11:07:55 +02:00
zemion 9137300780 Add bulk recipient activation controls 2026-08-04 10:51:44 +02:00
zemion f98fe06143 Resolve campaign worker jobs to tenants 2026-08-04 09:29:35 +02:00
zemion dc63e35550 Declare legacy operator route surface 2026-08-04 05:20:46 +02:00
zemion 2dac5570cd Add protected package release workflow 2026-08-04 04:14:02 +02:00
zemion 91890fdaf5 Add native permission-aware campaign search source 2026-08-04 03:03:26 +02:00
zemion 8f5231147d Complete governed campaign lifecycle actions 2026-08-04 01:04:39 +02:00
zemion df5a93d6a3 Close Campaign interface audit gaps 2026-08-04 01:04:39 +02:00
zemion 3f3545f080 feat(campaign): expose governed archive action 2026-08-03 20:39:06 +02:00
zemion d9195a2d2b Reconcile orphaned Campaign artifacts 2026-08-03 09:23:27 +02:00
zemion d635f3a5fc Link campaign interventions to configured help 2026-08-03 07:35:39 +02:00
zemion 1d6c745991 Clarify campaign review interventions 2026-08-03 07:22:27 +02:00
zemion 5df26be074 Bind Mail effects to stable delivery attempts 2026-08-03 05:00:30 +02:00
zemion 50ce8b0acb Fence generated artifact retention 2026-08-03 03:54:58 +02:00
zemion 9da03090a7 Fence Campaign delivery effects in recovery ledger 2026-08-03 03:48:03 +02:00
zemion dd09b06c47 Adopt recovery ledger for Campaign builds 2026-08-03 03:03:00 +02:00
zemion c6bbdae2e1 Fix Campaign CSV evidence projection 2026-08-03 03:02:52 +02:00
zemion bf6e07f307 Add bulk calendar invitation delivery 2026-08-02 16:38:48 +02:00
zemion 7733265cc8 Implement governed hybrid campaign delivery 2026-08-02 13:58:37 +02:00
zemion b38597f2be Integrate Distribution Lists with Campaign recipients 2026-08-02 11:53:35 +02:00
zemion 4eeba62bbc feat: contribute governed campaign reports 2026-08-02 05:29:34 +02:00
zemion 2afdd38128 Add institutional provenance and approval gates 2026-08-01 20:57:26 +02:00
zemion cec3d17bff feat: govern shared campaign artifact storage 2026-08-01 17:48:24 +02:00
zemion 82ddc0c34c Consume governed recipients and add operational checks 2026-07-31 22:48:07 +02:00
zemion fa4eb39e0b feat: complete campaign wizards and retention reporting 2026-07-31 04:21:34 +02:00
zemion e689fdf495 Refactor campaign recipient and review presentation 2026-07-31 02:48:56 +02:00
zemion 5f7503598c feat: govern attachment exceptions and ownership transfers 2026-07-30 17:42:10 +02:00
zemion cd223cbb95 feat: harden campaign delivery and editing 2026-07-30 14:27:04 +02:00
zemion 4a120e8009 Complete recipient search and delivery evidence 2026-07-30 05:22:09 +02:00
zemion c769be39da perf(campaign): search share targets server-side 2026-07-30 01:30:13 +02:00
zemion 46df12c025 perf(campaign): batch tenant summaries 2026-07-30 01:15:38 +02:00
zemion 961d5d1130 refactor(campaign): split review and recipient boundaries 2026-07-30 01:01:26 +02:00
zemion 23b9a531d5 fix: remove duplicate Campaign translation key 2026-07-30 00:39:58 +02:00
zemion cc93945f79 feat: integrate reports into Campaign navigation 2026-07-29 22:19:24 +02:00
zemion 101f3ccd7d feat: integrate operator queue into Campaign 2026-07-29 21:10:37 +02:00
zemion 2199187e8b refactor(api): split campaign workflow routers 2026-07-29 20:07:11 +02:00
zemion dd9592a192 Refactor campaign delivery decision paths 2026-07-29 17:06:15 +02:00
zemion 5240749ae1 feat: add governed postbox delivery and report hardening 2026-07-29 14:16:28 +02:00
zemion f11c56e890 fix: make direct campaign pages scrollable 2026-07-28 22:50:11 +02:00
zemion 8d0a2608ed Declare campaign route View surfaces 2026-07-28 21:04:55 +02:00
zemion 89ae14c032 Integrate campaign delivery with hierarchical mail profiles 2026-07-28 19:33:12 +02:00
zemion 38af25ee88 docs: update GovOPlaN repository links 2026-07-27 15:46:51 +02:00
zemion 4774a8025c fix(campaign): roll back rejected immediate queues 2026-07-22 20:30:00 +02:00
zemion 689dc1fd6b fix(campaign): honor acceptance temp selection 2026-07-22 20:20:31 +02:00
zemion 90677348ff docs(campaign): define broker redelivery evidence 2026-07-22 20:19:51 +02:00
zemion 06786e86ef test(campaign): prove Celery broker redelivery 2026-07-22 20:19:29 +02:00
zemion 6fda123fc3 docs(campaign): distinguish worker process evidence 2026-07-22 18:26:42 +02:00
zemion 2fa91bb943 test(campaign): complete delivery fault drills 2026-07-22 18:26:28 +02:00
zemion e3cc476508 fix(campaign): initialize delivery worker runtime 2026-07-22 18:26:11 +02:00
zemion 01ef541917 docs(campaign): define target-like mail evidence 2026-07-22 16:49:57 +02:00
zemion d567257311 test(campaign): prove GreenMail delivery journey 2026-07-22 16:49:57 +02:00
zemion 3075ef7f5b docs(campaign): document aggregate report baseline 2026-07-22 15:20:04 +02:00
zemion 1ab4e91ffd test(campaign): add simple announcement acceptance fixture 2026-07-22 15:18:52 +02:00
zemion 735e874bd0 fix(campaign): bound operator queue polling 2026-07-22 09:39:03 +02:00
zemion 99d44eeb8d feat(campaign): complete durable operator queue 2026-07-22 09:31:33 +02:00
zemion f095a3e2c7 fix(campaign): sanitize synchronous send results 2026-07-22 09:21:44 +02:00
zemion 1225802c5d fix(campaign): suppress overlapping aggregate cells 2026-07-22 09:17:00 +02:00
zemion ac3329cafe fix(campaign): use stable aggregate status filters 2026-07-22 09:11:14 +02:00
zemion 4eb651c6ac feat(campaign): filter reports from outcome counts 2026-07-22 09:11:10 +02:00
zemion 0b4017c240 refactor(campaign): restore route docstrings 2026-07-22 09:08:12 +02:00
zemion 79b576b4bc fix(webui): localize Campaign delivery reporting 2026-07-22 09:04:42 +02:00
zemion b0282ebff2 test(campaign): cover bounded delivery modes 2026-07-22 09:00:35 +02:00
zemion aae4ef5952 chore(release): bump Campaign to 0.1.10 2026-07-22 08:50:53 +02:00
zemion 8ee87b7558 feat(campaign): surface aggregate reports 2026-07-22 08:49:38 +02:00
zemion 7229fb8e3d fix(campaign): mark excluded delivery as skipped 2026-07-22 08:48:46 +02:00
zemion 3487ec7048 test(campaign): enforce aggregate report permissions 2026-07-22 08:46:28 +02:00
zemion 22d72f82f5 test(campaign): require export authority for report email 2026-07-22 08:45:00 +02:00
zemion fc36aee6c0 feat(webui): add aggregate Campaign reports 2026-07-22 08:44:15 +02:00
zemion 06125cc0e8 feat(campaign): add privacy-safe aggregate reports 2026-07-22 08:44:12 +02:00
zemion 21f3014ac5 feat(campaign): add durable operator queue controls 2026-07-22 08:39:44 +02:00
zemion 62a68792a4 feat(campaign): persist delivery execution mode 2026-07-22 08:37:58 +02:00
zemion aa4ec66b7b fix(campaign): query reports before pagination 2026-07-22 08:31:43 +02:00
zemion 60efd1cb5d feat(campaign): make delivery modes explicit 2026-07-22 08:26:52 +02:00
zemion 7e1660344d feat(campaign): bound synchronous delivery 2026-07-22 08:21:42 +02:00
zemion a8c0750dd7 docs(campaign): clarify legacy mail secret treatment 2026-07-21 20:49:20 +02:00
zemion bfbb86564c fix(campaign): align adaptive handbook with shipped UI 2026-07-21 19:15:52 +02:00
zemion c05bb8e474 docs: surface permission-gated Campaign tasks 2026-07-21 18:53:07 +02:00
zemion 99ef25b08f docs: add adaptive Campaign composition guide 2026-07-21 18:46:22 +02:00
zemion 03100b77db refactor: simplify Campaign editor validation 2026-07-21 18:16:09 +02:00
zemion 0ac903c82d security: make Campaign mutation audits atomic 2026-07-21 17:56:59 +02:00
zemion 60776803c5 security: authorize every Campaign effect version 2026-07-21 17:56:47 +02:00
zemion 9a709b1264 security: separate Campaign reader and operator data 2026-07-21 17:55:54 +02:00
zemion af833ca38c security: bound Campaign editor metadata 2026-07-21 17:55:30 +02:00
zemion 50c509d161 security: seal Campaign delivery inputs 2026-07-21 17:54:34 +02:00
zemion 9f4eab07f6 docs: expand adaptive Campaign guidance 2026-07-21 17:29:08 +02:00
zemion 25a69b3fa9 docs: add multi-perspective Campaign handbook 2026-07-21 17:29:01 +02:00
zemion 24538c2a99 chore: release Campaign 0.1.9 2026-07-21 17:15:02 +02:00
zemion 7d8579194d docs: document the Campaign-to-Mail delivery boundary 2026-07-21 17:14:55 +02:00
zemion 09c63de813 security: harden Campaign delivery effects and reconciliation 2026-07-21 17:14:33 +02:00
zemion 057e660b17 feat: add Campaign delivery ownership and IMAP claim primitives 2026-07-21 17:13:49 +02:00
zemion 701c0fe184 refactor(webui): select Mail-owned profiles for campaigns 2026-07-21 17:13:22 +02:00
zemion 2c5519908a refactor(webui): consume Core navigation and retention components 2026-07-21 13:52:59 +02:00
zemion dc56687af6 Remove shared CSS ownership from Campaign 2026-07-21 13:47:28 +02:00
zemion 2c70c553ac Use central metric cards for Campaign summaries 2026-07-21 13:24:18 +02:00
zemion 8627d0e135 Import Campaign data grids directly from Core 2026-07-21 13:24:04 +02:00
zemion d2adcca7ae Remove Campaign-owned shared CSS 2026-07-21 13:19:20 +02:00
zemion 002ca4b371 refactor(webui): use core access explanation 2026-07-21 13:18:40 +02:00
zemion 641bead0d8 Deduplicate campaign template rendering 2026-07-21 13:14:32 +02:00
zemion 3c305753d6 Refactor campaign job queue selection 2026-07-21 12:54:11 +02:00
zemion ef513816f8 Refactor mock campaign send reporting 2026-07-21 12:54:08 +02:00
zemion b7653b58f4 refactor(webui): use central campaign components 2026-07-21 12:04:30 +02:00
zemion ce92499333 fix(api): hide campaign operational internals 2026-07-21 12:03:56 +02:00
zemion 04bc6c430e Fix Campaign navigation action markup 2026-07-21 03:17:11 +02:00
zemion 04c214b149 Use the central Dialog for Campaign previews 2026-07-21 03:17:11 +02:00
zemion 35b3cc151d Clean Campaign audit and test resources 2026-07-21 03:17:11 +02:00
zemion c54fec4cc3 Remove obsolete Campaign schema copies 2026-07-21 03:17:11 +02:00
zemion 1d291377c0 Block local file paths at Campaign server boundaries 2026-07-21 03:17:11 +02:00
zemion 78f52a36d4 refactor(campaign-ui): align workspace layouts 2026-07-20 20:09:11 +02:00
zemion 0c4f198802 refactor(campaign-ui): separate IMAP append settings 2026-07-20 20:08:58 +02:00
zemion c71fa3dc64 feat(campaign-ui): configure attachment delivery behavior 2026-07-20 20:08:53 +02:00
zemion 0a6064ec62 feat(campaign-ui): review and link managed attachments 2026-07-20 20:08:41 +02:00
zemion fce6dd1138 feat(campaign-ui): add explicit job retry controls 2026-07-20 20:08:29 +02:00
zemion 4b9eb79065 refactor(campaign-ui): consolidate recipient editing 2026-07-20 20:08:10 +02:00
zemion 7ea5bdb217 fix(campaign-ui): make discard restore server state 2026-07-20 20:08:00 +02:00
zemion f755cdf48d fix(campaign): remove stale mapper coupling 2026-07-20 20:07:39 +02:00
zemion 8965b27517 feat(campaign): expose attachment linking and single send 2026-07-20 20:07:25 +02:00
zemion 12036b1f36 feat(campaign): support reviewed single-message delivery 2026-07-20 20:07:18 +02:00
zemion ad34365f6c feat(campaign): enforce attachment delivery policies 2026-07-20 20:07:12 +02:00
zemion 724ca779d6 fix(campaign): stabilize message preview overlays 2026-07-20 17:11:23 +02:00
zemion 2e593b7fa4 intermittent commit 2026-07-14 13:22:10 +02:00
zemion 3f0b14a726 Harden campaign import handling 2026-07-11 18:49:35 +02:00
zemion 30b43913e8 Release v0.1.8 2026-07-11 16:49:01 +02:00
zemion 83b2082c74 Release v0.1.7 2026-07-11 02:34:56 +02:00
zemion 5775347746 Pin campaign Excel import dependency 2026-07-11 01:09:43 +02:00
zemion 083c089003 Use Node XLSX reader in campaign import tests 2026-07-11 00:59:44 +02:00
zemion 54c9881a25 Add campaign resource access explanations 2026-07-11 00:39:40 +02:00
zemion da4c8731fc Sync GovOPlaN module state 2026-07-10 21:57:21 +02:00
zemion 9a6b6c7e8e Use capability-based module boundaries 2026-07-10 17:33:52 +02:00
zemion 52e6ed49c5 chore: sync GovOPlaN module split state 2026-07-10 12:51:18 +02:00
zemion 57f6066bf7 Release v0.1.6 2026-07-07 15:55:37 +02:00
332 changed files with 93550 additions and 11588 deletions
+270
View File
@@ -0,0 +1,270 @@
name: Module Package Release
on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
release_tag:
description: Existing protected version tag to publish
required: true
type: string
jobs:
publish-packages:
runs-on: ubuntu-latest
env:
GITEA_REPOSITORY: ${{ gitea.repository }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
with:
python-version: "3.12"
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "22"
- name: Select and validate protected release tag
shell: bash
env:
REQUESTED_TAG: ${{ inputs.release_tag }}
TRIGGER_TAG: ${{ gitea.ref_name }}
run: |
set -euo pipefail
tag="${REQUESTED_TAG:-$TRIGGER_TAG}"
case "$tag" in
v[0-9]*.[0-9]*.[0-9]*) ;;
*) echo "Release tag must start with a SemVer-shaped vX.Y.Z value" >&2; exit 1 ;;
esac
git fetch --force origin "refs/tags/$tag:refs/tags/$tag" refs/heads/main:refs/remotes/origin/main
tag_commit="$(git rev-list -n 1 "$tag")"
git merge-base --is-ancestor "$tag_commit" refs/remotes/origin/main || {
echo "Release tag is not contained in main" >&2
exit 1
}
git checkout --detach "$tag"
printf 'RELEASE_TAG=%s\n' "$tag" >> "$GITEA_ENV"
printf 'SOURCE_DATE_EPOCH=%s\n' "$(git show -s --format=%ct HEAD)" >> "$GITEA_ENV"
- name: Validate package versions
run: |
python - <<'PY'
import json
from pathlib import Path
import os
import re
import tomllib
tag = os.environ["RELEASE_TAG"]
expected = tag.removeprefix("v")
project = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))["project"]
if project.get("version") != expected:
raise SystemExit(f"pyproject version {project.get('version')!r} does not match {tag}")
if re.fullmatch(r"govoplan-[a-z0-9-]+", str(project.get("name", ""))) is None:
raise SystemExit("Python distribution name must use the govoplan-* namespace")
webui = Path("webui/package.json")
if webui.is_file():
package = json.loads(webui.read_text(encoding="utf-8"))
if package.get("version") != expected:
raise SystemExit(f"WebUI version {package.get('version')!r} does not match {tag}")
if re.fullmatch(r"@govoplan/[a-z0-9-]+-webui", str(package.get("name", ""))) is None:
raise SystemExit("WebUI package name must use the @govoplan/*-webui namespace")
release = Path("webui/package.release.json")
if release.is_file():
release_package = json.loads(release.read_text(encoding="utf-8"))
if (
release_package.get("name") != package.get("name")
or release_package.get("version") != expected
):
raise SystemExit("WebUI release package identity does not match package.json and the release tag")
PY
- name: Build immutable package artifacts
shell: bash
run: |
set -euo pipefail
python -m pip install --disable-pip-version-check build==1.5.0 twine==7.0.0
rm -rf dist .package-webui
python -m build --wheel --outdir dist
python -m twine check dist/*.whl
if [[ -f webui/package.json ]]; then
mkdir .package-webui
cp -a webui/. .package-webui/
rm -rf .package-webui/node_modules .package-webui/dist
if [[ -f .package-webui/package.release.json ]]; then
cp .package-webui/package.release.json .package-webui/package.json
fi
node <<'NODE'
const fs = require("node:fs");
const path = ".package-webui/package.json";
const packageJson = JSON.parse(fs.readFileSync(path, "utf8"));
const groups = ["dependencies", "optionalDependencies", "peerDependencies"];
for (const group of groups) {
for (const [name, specifier] of Object.entries(packageJson[group] || {})) {
if (!name.startsWith("@govoplan/")) continue;
if (typeof specifier !== "string") {
throw new Error(`${group}.${name} must use a string version`);
}
const packageSlug = name.slice("@govoplan/".length);
if (!packageSlug.endsWith("-webui")) {
throw new Error(`${group}.${name} is outside the WebUI package namespace`);
}
const repository = `govoplan-${packageSlug.slice(0, -"-webui".length)}`;
const escapedRepository = repository.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const gitTag = specifier.match(
new RegExp(
`^git\\+(?:ssh://git@|https://)git\\.add-ideas\\.de/(?:GovOPlaN|add-ideas)/${escapedRepository}\\.git#v([0-9]+\\.[0-9]+\\.[0-9]+)$`,
),
);
if (gitTag) {
packageJson[group][name] = gitTag[1];
continue;
}
if (specifier.startsWith("file:") || specifier.startsWith("git+")) {
throw new Error(
`${group}.${name} must resolve to an exact registry version for publication`,
);
}
}
}
delete packageJson.private;
fs.writeFileSync(path, `${JSON.stringify(packageJson, null, 2)}\n`);
NODE
npm pkg delete private --prefix .package-webui
(cd .package-webui && npm pack --ignore-scripts --pack-destination ../dist)
fi
python - <<'PY'
import hashlib
import json
from pathlib import Path
import os
import subprocess
artifacts = []
for path in sorted(Path("dist").iterdir()):
if path.suffix not in {".whl", ".tgz"}:
continue
digest = hashlib.sha256(path.read_bytes()).hexdigest()
artifacts.append({"filename": path.name, "sha256": digest, "size": path.stat().st_size})
payload = {
"schema_version": "1",
"repository": os.environ["GITEA_REPOSITORY"],
"tag": os.environ["RELEASE_TAG"],
"commit": subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip(),
"artifacts": artifacts,
}
Path("dist/package-artifacts.json").write_text(
json.dumps(payload, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
PY
- name: Retain package hash evidence
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: module-packages-${{ gitea.ref_name }}
path: dist/package-artifacts.json
- name: Check immutable registry state
shell: bash
env:
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_TOKEN"
python - <<'PY'
import hashlib
import json
import os
from pathlib import Path
import tomllib
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
api_root = "https://git.add-ideas.de/api/v1/packages/GovOPlaN"
token = os.environ["PACKAGE_TOKEN"]
def should_publish(kind, name, version, path):
package_url = "/".join(
(api_root, kind, quote(name, safe=""), quote(version, safe=""), "files")
)
request = Request(
package_url,
headers={"Accept": "application/json", "Authorization": f"token {token}"},
)
try:
with urlopen(request, timeout=30) as response:
files = json.load(response)
except HTTPError as exc:
if exc.code == 404:
print(f"{kind} package {name}=={version} is not published yet")
return True
raise
if not isinstance(files, list) or len(files) != 1:
raise SystemExit(
f"immutable {kind} package {name}=={version} has an unexpected file set"
)
expected_sha256 = hashlib.sha256(path.read_bytes()).hexdigest()
if files[0].get("sha256") != expected_sha256:
raise SystemExit(
f"immutable {kind} package {name}=={version} already exists with a different SHA-256"
)
print(f"verified existing {kind} package {name}=={version} ({expected_sha256})")
return False
project = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))["project"]
wheels = tuple(Path("dist").glob("*.whl"))
if len(wheels) != 1:
raise SystemExit("release build must contain exactly one wheel")
publish_pypi = should_publish(
"pypi", str(project["name"]), str(project["version"]), wheels[0]
)
tarballs = tuple(Path("dist").glob("*.tgz"))
if len(tarballs) > 1:
raise SystemExit("release build must contain at most one npm package")
publish_npm = False
if tarballs:
webui = json.loads(
Path(".package-webui/package.json").read_text(encoding="utf-8")
)
publish_npm = should_publish(
"npm", str(webui["name"]), str(webui["version"]), tarballs[0]
)
with Path(os.environ["GITEA_ENV"]).open("a", encoding="utf-8") as env_file:
env_file.write(f"PUBLISH_PYPI={int(publish_pypi)}\n")
env_file.write(f"PUBLISH_NPM={int(publish_npm)}\n")
PY
- name: Publish wheel and WebUI package
shell: bash
env:
PACKAGE_USERNAME: ${{ secrets.GOVOPLAN_PACKAGE_USERNAME }}
PACKAGE_TOKEN: ${{ secrets.GOVOPLAN_PACKAGE_TOKEN }}
run: |
set -euo pipefail
test -n "$PACKAGE_USERNAME"
test -n "$PACKAGE_TOKEN"
if [[ "$PUBLISH_PYPI" == 1 ]]; then
TWINE_USERNAME="$PACKAGE_USERNAME" TWINE_PASSWORD="$PACKAGE_TOKEN" \
python -m twine upload --non-interactive \
--repository-url https://git.add-ideas.de/api/packages/GovOPlaN/pypi \
dist/*.whl
else
echo "Exact wheel is already present; skipping immutable retry."
fi
shopt -s nullglob
webui_packages=(dist/*.tgz)
if (( ${#webui_packages[@]} )) && [[ "$PUBLISH_NPM" == 1 ]]; then
npmrc="$(mktemp)"
trap 'rm -f "$npmrc"' EXIT
chmod 600 "$npmrc"
printf '%s\n' \
'@govoplan:registry=https://git.add-ideas.de/api/packages/GovOPlaN/npm/' \
"//git.add-ideas.de/api/packages/GovOPlaN/npm/:_authToken=$PACKAGE_TOKEN" \
> "$npmrc"
NPM_CONFIG_USERCONFIG="$npmrc" npm publish "./${webui_packages[0]}" \
--ignore-scripts --access public \
--registry https://git.add-ideas.de/api/packages/GovOPlaN/npm/
elif (( ${#webui_packages[@]} )); then
echo "Exact WebUI package is already present; skipping immutable retry."
fi
+19
View File
@@ -331,3 +331,22 @@ cython_debug/
# GovOPlaN WebUI test output
webui/.policy-test-build/
webui/.template-preview-test-build/
webui/.import-test-build/
webui/.review-preview-test-build/
webui/.report-grid-test-build/
# GovOPlaN shared ignore rules from govoplan-core
# Local WebUI test/build scratch directories
.component-test-build/
.module-test-build/
.policy-test-build/
.template-preview-test-build/
.import-test-build/
.review-preview-test-build/
webui/.component-test-build/
webui/.module-test-build/
*.db
# GovOPlaN local runtime state
runtime/
webui/.module-test-build/
webui/.component-test-build/
+8 -2
View File
@@ -1,5 +1,11 @@
# GovOPlaN Campaign Codex Guide
## Documentation Contract
- Treat documentation as part of every behavior change. Update this module's manifest-driven `DocumentationTopic` contributions for affected user and administrator behavior.
- Keep feature content here; `govoplan-docs` projects it without importing Campaign internals.
- Maintain a static user/admin baseline and run `/mnt/DATA/git/govoplan/tools/checks/check-manifest-shapes.py` after behavior or manifest changes.
## Scope
This repository owns the `campaigns` module: campaign authoring, validation, message building, attachment resolution, queue/review/send control, reports, campaign module manifest, and `@govoplan/campaign-webui`.
@@ -26,8 +32,8 @@ PATH=/mnt/DATA/git/govoplan-core/webui/node_modules/.bin:/home/zemion/.nvm/versi
For combined checks, run:
```bash
cd /mnt/DATA/git/govoplan-core
./scripts/check-focused.sh
cd /mnt/DATA/git/govoplan
tools/checks/check-focused.sh
```
## Working Rules
+89 -5
View File
@@ -1,5 +1,9 @@
# govoplan-campaign
<!-- govoplan-repository-type:start -->
**Repository type:** module (domain).
<!-- govoplan-repository-type:end -->
GovOPlaN Campaign is the campaign authoring, validation, review, sending-control, and reporting module. It bundles backend campaign APIs with the campaign WebUI package.
## Ownership
@@ -11,9 +15,32 @@ This repository owns:
- campaign/version/job/issue/send-attempt/append-attempt models and migrations
- campaign JSON schema, validation, message building, attachment resolution, ZIP handling, reports, queue/control services, and mock-send paths
- WebUI package `@govoplan/campaign-webui`
- route contributions for `/campaigns`, `/campaigns/:campaignId/*`, `/operator`, `/reports`, `/address-book`, and `/templates`
- route contributions for `/campaigns`, the integrated `/campaigns/queue` view,
`/campaigns/reports`, `/campaigns/:campaignId/*`, and `/templates`
Core owns auth, tenants, RBAC evaluation, database/session primitives, CSRF/API helpers, shell layout, and route rendering. Files and mail own their respective storage and transport capabilities.
Core owns the auth facade, RBAC/capability contracts, database/session
primitives, CSRF/API helpers, shell layout, and route rendering. Tenancy is an
optional platform module for tenant administration and tenant resolver behavior.
Files and mail own their respective storage and transport capabilities.
When the optional Reporting module is enabled, Campaign contributes its
recipient-free aggregate delivery report through the versioned Core report
provider contract. Reporting owns the global `/reports` route. Campaign keeps
its module-local `/campaigns/reports` view and does not claim the global route
when Reporting is absent.
Generated EML and printable artifacts are durable execution material, not a
node-local runtime cache. Campaign stores EML through Core's shared
object-storage contract under opaque Campaign-owned keys. Templates returns a
bounded artifact or a Files-managed artifact for printable output. Database job
rows retain the expected hashes and provenance. Workers resolve and verify the
frozen evidence before delivery. Build failure compensates objects written
before database commit. Retention uses a fenced forward-recovery operation and
independently verifies both artifact absence and the committed locator update;
partial or unobservable cleanup remains visible in Ops.
An operator-only, dry-run-first reconciler inventories bounded tenant-prefix
pages and removes only old objects that remain unreferenced after an active
build-fence check. Applied runs are idempotent, fenced, audited, and preserve
database references on every storage failure.
## Dependencies
@@ -21,10 +48,50 @@ The module has one required runtime dependency:
- `govoplan-core` for platform services, auth, RBAC, DB/session lifecycle, migrations, and WebUI shell integration
Files and mail are optional module integrations declared in the campaign manifest:
Files, Mail, Distribution Lists, Templates, Postbox, and Calendar are optional module integrations declared in the campaign manifest:
- `govoplan-files` enables managed attachment selection, frozen file-version evidence, and managed-file usage tracking. Without it, campaigns can still use legacy/local attachment paths where configured.
- `govoplan-mail` enables reusable mail profiles, delivery policy checks, SMTP sending, and IMAP append behavior. Without it, campaigns can still be authored, validated, built, and reported, but real delivery/profile features are unavailable.
- `govoplan-files` enables managed attachment selection, frozen file-version evidence, and managed-file usage tracking. Server/API campaigns require this integration for attachments and never resolve caller-supplied local filesystem paths. Legacy file-oriented loading remains available only to explicitly trusted operator/library workflows.
- `govoplan-mail` owns reusable profiles, encrypted SMTP/IMAP credentials, delivery policy checks, connection tests, and transport execution. Campaign JSON stores only `server.mail_profile_id`; inline transport settings and credentials are rejected. Without Mail, campaigns can still be authored, but profile validation and real delivery are unavailable.
- `govoplan-dist-lists` expands reusable governed audiences. Campaign freezes the exact list revision, provider evidence, candidates, and explicit per-recipient primary/fallback route into its own version.
- `govoplan-templates` validates and renders published label, envelope, letter, and list-layout templates for postal or internal-mail delivery. Generated output is hash-bound to its template, inputs, actor, route decisions, and Campaign version.
- `govoplan-postbox` resolves exact or organization-derived Postbox targets and records provider acceptance and receipt evidence. It remains optional; Mail-only and print-only campaigns do not require it.
- `govoplan-calendar` renders and mirrors individualized VEVENT invitations through the versioned `calendar.invitations` capability. Campaign freezes one METHOD:REQUEST attachment per recipient during build, creates the Calendar mirror only after delivery acceptance, and reads live RSVP state in bounded report batches. Mail may forward METHOD:REPLY parts from an authorized IMAP source. Calendar absence leaves ordinary Campaign authoring and delivery usable.
Hybrid delivery never treats an opt-in as an implicit duplicate-send instruction. The Campaign author selects one primary route per recipient and may select a supported fallback. A fallback runs only after the first channel rejects before acceptance; accepted or outcome-unknown effects stop cross-channel retry. Printable output is generated once during build, optionally persisted through Files, reviewed with the exact Campaign version, and accepted idempotently per recipient job during delivery.
Recurring schedules have two immutable modes. Manual mode remains the default
and prepares independent drafts without Mail. Autonomous mode is explicit and
Mail-only: it seals an already built and explicitly approved execution snapshot,
rechecks approval, policy, credential/transport revision, live SMTP health,
recipient and attachment evidence before each occurrence, and submits one
Mail-owned durable command per frozen message. Occurrence-scoped idempotency is
allocated before delivery. Accepted and outcome-unknown effects are never
retried automatically; uncertain or systemic failures pause the schedule,
notify its accountable operator, and retain non-secret recovery evidence.
Generated EML retention excludes source versions while an autonomous schedule
has a remaining occurrence, including while it is paused; once the schedule
finishes, already accepted Mail commands retain their own encrypted payload and
evidence under Mail policy.
Campaign versions can also be exported as versioned portable JSON packages and
imported as independently owned drafts. The privacy-safe export default is
metadata plus template/configuration. Recipients, attachment rules, aggregate
review state, and recipient-level delivery history are separate scopes with
their existing fine-grained permissions. Packages include source provenance,
scope/item/redaction manifests, and a SHA-256 integrity digest. They never
contain attachment bytes, transport secrets, credential references,
password-field values, local storage locators, shares, or ownership grants.
Import previews schema and checksum compatibility plus every created/skipped
domain. It clears deployment-bound Mail references and never replays locks,
approvals, review decisions, jobs, attempts, or sent state.
Public campaign, version, job, and report responses expose business data and
delivery evidence, but never process-local paths, storage-backend keys, or
worker claim tokens. Operational troubleshooting uses the dedicated job
diagnostics endpoint and requires the tenant-level
`campaigns:diagnostic:read` permission. The campaign sender role receives this
permission; tenant-wide administrator scopes continue to grant it through the
standard policy evaluator.
Backend optional behavior is accessed through core-provided capabilities, not direct required imports. WebUI optional behavior uses core module metadata/capabilities so campaign pages can build and run without files or mail WebUI packages installed.
@@ -37,6 +104,9 @@ services can cooperate without importing campaign internals:
- `campaigns.policyContext` for retention/policy provenance
- `campaigns.deliveryTasks` for queued send and append-to-Sent workers
- `campaigns.retention` for campaign-owned retention cleanup
- `privacy.dsar.campaigns` for tenant-scoped recipient, version, delivery,
report-projection, and artifact-metadata discovery plus governed erasure
planning
Keep these capability payloads narrow: stable ids, policy payloads, and task
results only.
@@ -76,7 +146,21 @@ Platform RBAC and governance rules are documented in `govoplan-core/docs/`.
## Operations
- [Campaign handbook](docs/CAMPAIGN_HANDBOOK.md) provides the adaptive user, process, governance, technical, and operations perspectives.
- [Campaign delivery runbook](docs/CAMPAIGN_DELIVERY_RUNBOOK.md) covers queueing, local vs Celery operation, retries, reconciliation, reports, and the live SMTP/IMAP test checklist.
- Immediate delivery is bounded to 25 exact eligible recipient jobs by default. Deployments may set `GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` (0500), and tenants may narrow that ceiling through `campaign_delivery_policy.synchronous_send_max_recipients` in tenant settings.
- Immediate Mail delivery preflights the selected SMTP transport before the
first effect and reuses a healthy bounded connection through Mail. Review and
send reports the batch state, connection/reconnect counts, and paused count.
A systemic authentication, sender, or connectivity failure pauses remaining
jobs; correct and test the Mail profile before explicitly resuming them.
- Report-email preview uses the selected version's stored v5 Mail-profile evidence. Live report email fails closed until [govoplan-mail#17](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17) provides a durable, idempotent Mail-owned outbox and transport-attempt ledger; per-job CSV is off by default and requires `campaigns:recipient:export` when requested.
- [Campaign/Mail profile boundary](docs/MAIL_PROFILE_BOUNDARY.md) defines profile-only delivery, runtime resolution, execution evidence, and the fail-closed legacy migration path.
- [Recipient import guide](docs/RECIPIENT_IMPORT_GUIDE.md) covers user/admin workflows, mapping profiles, validation, and import evidence.
- [Recipient and address boundary](docs/RECIPIENT_ADDRESS_BOUNDARY.md) defines the split between campaign-local recipients and future reusable address management.
- [Example campaigns and release checklist](docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md) defines the maintained example scenarios and release gates.
- [Campaign examples](examples/README.md) is the credential-free scenario catalogue that release fixtures must follow.
- [SMTP/IMAP test bed](dev/mail-testbed/README.md) provides the GreenMail Docker Compose setup and transport smoke for dedicated non-production delivery tests.
## Release packaging
+13
View File
@@ -0,0 +1,13 @@
GOVOPLAN_MAIL_TEST_LOCAL_PART=campaign-test
GOVOPLAN_MAIL_TEST_DOMAIN=govoplan.test
GOVOPLAN_MAIL_TEST_USER=campaign-test@govoplan.test
GOVOPLAN_MAIL_TEST_PASSWORD=campaign-test-password
GOVOPLAN_MAIL_TEST_RECIPIENT=campaign-test@govoplan.test
GOVOPLAN_MAIL_TEST_FROM=campaign-test@govoplan.test
GOVOPLAN_MAIL_TEST_SMTP_PORT=3025
GOVOPLAN_MAIL_TEST_IMAP_PORT=3143
GOVOPLAN_MAIL_TEST_SENT_FOLDER=Sent
GOVOPLAN_MAIL_TEST_ZIP_PASSWORD=zip-test-password
GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS=45
GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT=36379
GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS=3
+174
View File
@@ -0,0 +1,174 @@
# Campaign SMTP/IMAP Test Bed
This test bed provides dedicated non-production SMTP/IMAP infrastructure for
campaign delivery checks. It uses GreenMail and must never be configured with
production recipients or credentials.
## Start
```bash
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
cp .env.example .env
docker compose --env-file .env up -d
```
Default endpoints:
- SMTP: `127.0.0.1:3025`, plain transport
- IMAP: `127.0.0.1:3143`, plain transport
- GreenMail API/UI endpoint: `127.0.0.1:38080`
## Smoke Test
Run the transport smoke from the core virtual environment after installing the
campaign and mail modules:
```bash
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
set -a
. ./.env
set +a
/mnt/DATA/git/govoplan-core/.venv/bin/python run_transport_smoke.py
```
The smoke sends and appends three messages:
- no attachment
- one normal attachment
- one password-protected AES ZIP attachment
It verifies SMTP authentication, IMAP authentication, folder listing, delivery
to `INBOX`, and append-to-Sent behavior.
If the smoke is started immediately after `docker compose up -d`, GreenMail may
bind the SMTP/IMAP ports before the services are fully ready. The smoke retries
login and folder setup for `GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS`.
## Campaign Acceptance
The transport smoke proves the Mail adapters. The Campaign acceptance runner
proves the public composition: it creates an isolated temporary Core database,
creates a Mail-owned encrypted profile through the API, materializes the
credential-free [`greenmail-delivery`](../../examples/greenmail-delivery/campaign.json)
fixture with only that profile reference, validates/builds it, sends the exact
generated EML through Campaign once, appends it once, and cross-checks Campaign
report/audit state with one unique-subject message in the GreenMail INBOX and
Sent folders. Provider mailbox verification is not a byte-for-byte comparison
after provider-side header or storage transformations.
```bash
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
set -a
. ./.env
set +a
/mnt/DATA/git/govoplan/.venv/bin/python run_campaign_acceptance.py \
--evidence /tmp/govoplan-campaign-greenmail-evidence.json
```
The default run also uses controlled loopback protocol endpoints to prove that
an SMTP connection loss before transmission is temporary, an explicit SMTP
authentication rejection is permanent, a final `451` response after DATA is
temporary, one accepted and one refused RCPT command is retained as partial
envelope acceptance, a connection loss after complete DATA is frozen as
`outcome_unknown`, and an IMAP authentication rejection after SMTP acceptance
leaves the send accepted while the append fails. A
second ordinary send must be rejected before another provider effect.
The worker drill queues one job, starts the registered
`govoplan.campaigns.send_email` task body in a dedicated OS process, waits until
the controlled endpoint has received complete DATA, terminates that process,
and starts the same task body in a fresh process. It proves the durable
`sending`/unfinished-attempt boundary is recovered as `outcome_unknown`
without a second SMTP connection or DATA transaction. This is a real process
and task-boundary interruption, but it does not start a Celery daemon, Redis
broker, or broker redelivery; `celery_broker_redelivery` therefore remains
`false` in the evidence.
The bounded JSON contains no endpoint, account, address, credential, profile,
campaign, version, or job identifiers. It records module versions, the fixture
hash, normalized classifications/counts, the Mail-profile boundary, required
audit actions, provider mailbox increments, and coverage flags. Runtime version
declarations identify the exercised composition; they do not claim that the
sources are clean, tagged, signed, or release-provenanced. The evidence names
them `declared_module_versions` and keeps `source_artifact_provenance` false;
exact commit/artifact provenance belongs to the package and release gate.
This runner is restricted to literal loopback IP addresses and the synchronous
Campaign delivery mode. Hostnames such as `localhost` and every non-loopback
address fail before profile creation, avoiding a DNS change between validation
and connection. It is local target-like evidence, not approval of an
institution's SMTP/IMAP service. The controlled post-DATA, temporary-response,
partial-refusal, and task-process interruption drills are local effect-level
proof, not proof of a target provider's behavior or Redis/Celery broker
redelivery. Use `--success-only` only when testing the success journey without
the local failure endpoints.
## Redis/Celery Redelivery Acceptance
Run the maintained broker/worker-loss acceptance separately from the GreenMail
journey:
```bash
cd /mnt/DATA/git/govoplan-campaign/dev/mail-testbed
set -a
. ./.env
set +a
/mnt/DATA/git/govoplan/.venv/bin/python run_celery_redelivery_acceptance.py \
--evidence /tmp/govoplan-campaign-celery-redelivery-evidence.json
```
The runner creates a unique Compose project, starts only its loopback-bound,
AOF-enabled Redis service, creates an isolated temporary GovOPlaN database,
and starts a real Celery worker subscribed to `send_email`. A controlled SMTP
server holds the transaction after complete DATA and before the final response.
The runner kills that solo worker with the task still unacknowledged and starts
a replacement worker. After the configured Redis visibility timeout, the same Celery task identity must be redelivered.
The replacement must turn the durable
unfinished attempt into `outcome_unknown`, acknowledge the task, drain the
broker queue/unacked records, and leave the SMTP endpoint at exactly one
connection and one DATA transaction.
Only bounded counts, classifications, and booleans are retained. Worker logs,
task IDs, database identifiers, endpoints, credentials, and raw diagnostics are
kept in the temporary runtime and deleted. The evidence proves the local Redis
transport, real Celery process boundary, runner-supervised replacement, and
Campaign's duplicate-effect guard. It deliberately keeps production daemon supervision,
target-provider behavior, and source-artifact provenance false.
It does not claim that systemd, Kubernetes, another container orchestrator, or
an institution's Redis/SMTP deployment behaves identically.
The default run requires Docker CLI/Compose/daemon access and permission to
pull `redis:7-alpine`; the Celery workers execute from the current Python
environment. The isolated Compose project and volume are removed on exit.
`GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS` defaults to three
seconds only to make this destructive local drill finish promptly; it is not a
production recommendation.
Starting the maintained test bed requires a working Docker CLI, Compose plugin,
daemon/socket access, and permission to pull `greenmail/standalone:2.1.9`. The
Campaign runner needs only the already-running loopback endpoints; it neither
starts Docker nor claims that it did.
## Use With A Campaign
Use the same settings in a campaign mail profile:
- SMTP host `127.0.0.1`, port `3025`, security `plain`
- IMAP host `127.0.0.1`, port `3143`, security `plain`
- username and password from `.env`
- append folder from `GOVOPLAN_MAIL_TEST_SENT_FOLDER`
When changing the mailbox, keep `GOVOPLAN_MAIL_TEST_USER` equal to
`${GOVOPLAN_MAIL_TEST_LOCAL_PART}@${GOVOPLAN_MAIL_TEST_DOMAIN}` because the
compose file creates the GreenMail account from local part, password, and
domain while login uses the full email address.
Then run the controlled sending checklist from
`docs/CAMPAIGN_DELIVERY_RUNBOOK.md` for no attachment, one attachment, and
password-protected ZIP campaign variants.
## Stop
```bash
docker compose --env-file .env down -v
```
+33
View File
@@ -0,0 +1,33 @@
services:
greenmail:
image: greenmail/standalone:2.1.9
container_name: govoplan-campaign-mail-testbed
restart: unless-stopped
environment:
GREENMAIL_OPTS: >-
-Dgreenmail.setup.test.all
-Dgreenmail.hostname=0.0.0.0
-Dgreenmail.auth.disabled=false
-Dgreenmail.users=${GOVOPLAN_MAIL_TEST_LOCAL_PART:-campaign-test}:${GOVOPLAN_MAIL_TEST_PASSWORD:-campaign-test-password}@${GOVOPLAN_MAIL_TEST_DOMAIN:-govoplan.test}
-Dgreenmail.users.login=email
-Dgreenmail.verbose
ports:
- "${GOVOPLAN_MAIL_TEST_SMTP_PORT:-3025}:3025"
- "${GOVOPLAN_MAIL_TEST_IMAP_PORT:-3143}:3143"
- "127.0.0.1:38080:8080"
redis:
image: redis:7-alpine
command: ["redis-server", "--appendonly", "yes"]
ports:
- "127.0.0.1:${GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT:-36379}:6379"
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 1s
timeout: 1s
retries: 30
volumes:
- campaign-redis-data:/data
volumes:
campaign-redis-data:
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,884 @@
#!/usr/bin/env python3
"""Prove Redis/Celery redelivery does not repeat an ambiguous SMTP effect.
The default run starts an isolated Redis Compose service, two successive real
Celery worker processes, and a controlled loopback SMTP endpoint. It kills the
first worker after complete DATA but before a final SMTP response. The same
unacknowledged broker task must be delivered to the replacement worker, which
must leave the unfinished durable attempt unchanged without a second SMTP
connection or DATA transaction. Only then does explicit fenced recovery use
verified process exit and an expired fixture lease to record outcome-unknown.
"""
from __future__ import annotations
import argparse
from collections import Counter
from contextlib import contextmanager
from dataclasses import dataclass, replace
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import socket
import subprocess
import sys
import tempfile
import time
from typing import Any, Callable, Iterator, Mapping
from uuid import uuid4
from redis import Redis
from redis.exceptions import RedisError
SCRIPT_ROOT = Path(__file__).resolve().parent
REPOSITORY_ROOT = SCRIPT_ROOT.parents[1]
if str(SCRIPT_ROOT) not in sys.path:
sys.path.insert(0, str(SCRIPT_ROOT))
from run_campaign_acceptance import ( # noqa: E402
AcceptanceError,
DEFAULT_FIXTURE,
EXPECTED_AUDIT_ACTIONS,
TestbedSettings,
_assert_evidence_safe,
_core_package_version,
_durable_state_evidence,
_expect,
_report_evidence,
create_mail_profile,
prepare_campaign_scenario,
required_composition_versions,
recover_stopped_fixture_claim,
smtp_fault_endpoint,
)
EVIDENCE_SCHEMA = "govoplan.campaign.celery-redelivery-acceptance.v1"
MAX_EVIDENCE_BYTES = 128 * 1024
DEFAULT_COMPOSE_FILE = SCRIPT_ROOT / "docker-compose.yml"
TASK_RECEIVED_PATTERN = re.compile(
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] received",
re.IGNORECASE,
)
TASK_SUCCEEDED_PATTERN = re.compile(
r"Task govoplan[.]campaigns[.]send_email\[([0-9a-f-]{36})\] succeeded",
re.IGNORECASE,
)
WORKER_BOOTSTRAP = r"""
import os
import sys
from govoplan_core.celery_app import celery
from govoplan_core import celery_app as worker_runtime
# Test-only process identity evidence, confined to this disposable child.
original_worker_metadata = worker_runtime._worker_metadata
worker_runtime._worker_metadata = lambda: {
**original_worker_metadata(), "acceptance_worker_pid": os.getpid(),
}
visibility_timeout = int(os.environ["GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS"])
celery.conf.broker_transport_options = {
**dict(celery.conf.broker_transport_options or {}),
"polling_interval": 0.25,
"visibility_timeout": visibility_timeout,
}
celery.worker_main(
[
"worker",
"--loglevel=INFO",
"--pool=solo",
"--concurrency=1",
"--queues=send_email",
f"--hostname={sys.argv[1]}@%h",
"--without-gossip",
"--without-mingle",
"--without-heartbeat",
]
)
"""
@dataclass(slots=True)
class WorkerProcess:
process: subprocess.Popen[bytes]
log_path: Path
log_handle: Any
def text(self) -> str:
self.log_handle.flush()
try:
return self.log_path.read_text(encoding="utf-8", errors="replace")
except OSError as exc:
raise AcceptanceError("Celery worker evidence log could not be read") from exc
def received_task_ids(self) -> tuple[str, ...]:
return tuple(TASK_RECEIVED_PATTERN.findall(self.text()))
def succeeded_task_ids(self) -> tuple[str, ...]:
return tuple(TASK_SUCCEEDED_PATTERN.findall(self.text()))
@dataclass(frozen=True, slots=True)
class RedisBrokerState:
queue_depth: int
unacked_hash_count: int
unacked_index_count: int
def as_dict(self) -> dict[str, int]:
return {
"queue_depth": self.queue_depth,
"unacked_hash_count": self.unacked_hash_count,
"unacked_index_count": self.unacked_index_count,
}
def _positive_int(value: str, *, label: str) -> int:
try:
parsed = int(value)
except ValueError as exc:
raise argparse.ArgumentTypeError(f"{label} must be a positive integer") from exc
if parsed <= 0:
raise argparse.ArgumentTypeError(f"{label} must be a positive integer")
return parsed
def _unused_loopback_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as probe:
probe.bind(("127.0.0.1", 0))
return int(probe.getsockname()[1])
def _compose_command(
*, compose_file: Path, project_name: str, operation: str
) -> list[str]:
prefix = [
"docker",
"compose",
"--file",
str(compose_file),
"--project-name",
project_name,
]
if operation == "up":
return [*prefix, "up", "--detach", "redis"]
if operation == "down":
return [*prefix, "down", "--volumes", "--remove-orphans"]
raise AcceptanceError("Unsupported Redis Compose operation")
def _run_compose(
command: list[str],
*,
environment: Mapping[str, str],
timeout_seconds: int,
) -> None:
try:
completed = subprocess.run(
command,
env=dict(environment),
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout_seconds,
check=False,
)
except (OSError, subprocess.TimeoutExpired) as exc:
raise AcceptanceError("Redis Compose lifecycle command failed") from exc
if completed.returncode != 0:
raise AcceptanceError("Redis Compose lifecycle command failed")
def _wait_for_redis(redis_url: str, *, timeout_seconds: int) -> None:
deadline = time.monotonic() + timeout_seconds
client = Redis.from_url(
redis_url,
socket_connect_timeout=1,
socket_timeout=1,
decode_responses=False,
)
try:
while time.monotonic() < deadline:
try:
if client.ping() is True:
return
except RedisError:
pass
time.sleep(0.25)
finally:
client.close()
raise AcceptanceError("Isolated Redis broker did not become ready")
@contextmanager
def isolated_redis_broker(
*,
compose_file: Path,
timeout_seconds: int,
requested_port: int | None = None,
) -> Iterator[str]:
if shutil.which("docker") is None:
raise AcceptanceError("Docker CLI is required to start the isolated Redis broker")
if not compose_file.is_file():
raise AcceptanceError("Redis Compose definition is unavailable")
port = requested_port or _unused_loopback_port()
if port <= 0 or port > 65_535:
raise AcceptanceError("Redis test port is invalid")
project_name = f"govoplan-campaign-redelivery-{uuid4().hex[:12]}"
environment = {
**os.environ,
"GOVOPLAN_CAMPAIGN_TEST_REDIS_PORT": str(port),
}
lifecycle_attempted = False
try:
lifecycle_attempted = True
_run_compose(
_compose_command(
compose_file=compose_file,
project_name=project_name,
operation="up",
),
environment=environment,
timeout_seconds=timeout_seconds,
)
redis_url = f"redis://127.0.0.1:{port}/0"
_wait_for_redis(redis_url, timeout_seconds=timeout_seconds)
yield redis_url
finally:
if lifecycle_attempted:
_run_compose(
_compose_command(
compose_file=compose_file,
project_name=project_name,
operation="down",
),
environment=environment,
timeout_seconds=timeout_seconds,
)
def _start_worker(runtime_root: Path, *, label: str) -> WorkerProcess:
log_path = runtime_root / f"{label}.log"
log_handle = log_path.open("wb")
environment = {**os.environ, "PYTHONUNBUFFERED": "1"}
try:
process = subprocess.Popen(
[sys.executable, "-c", WORKER_BOOTSTRAP, label],
env=environment,
stdin=subprocess.DEVNULL,
stdout=log_handle,
stderr=subprocess.STDOUT,
close_fds=True,
)
except Exception:
log_handle.close()
raise
return WorkerProcess(process=process, log_path=log_path, log_handle=log_handle)
def _wait_for_worker_ready(worker: WorkerProcess, *, timeout_seconds: int) -> None:
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
if worker.process.poll() is not None:
raise AcceptanceError("Celery worker exited before becoming ready")
if " ready." in worker.text():
return
time.sleep(0.2)
raise AcceptanceError("Celery worker did not become ready")
def _wait_for_received_task(
worker: WorkerProcess,
*,
timeout_seconds: int,
expected_task_id: str | None = None,
) -> str:
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
received = worker.received_task_ids()
if received:
if len(set(received)) != 1:
raise AcceptanceError("Celery worker received more than one task identity")
task_id = received[0]
if expected_task_id is not None and task_id != expected_task_id:
raise AcceptanceError("Replacement worker received a different broker task")
return task_id
if worker.process.poll() is not None:
raise AcceptanceError("Celery worker exited before receiving the task")
time.sleep(0.2)
raise AcceptanceError("Celery worker did not receive the broker task")
def _wait_for_task_success(
worker: WorkerProcess,
*,
task_id: str,
timeout_seconds: int,
) -> None:
deadline = time.monotonic() + timeout_seconds
while time.monotonic() < deadline:
succeeded = worker.succeeded_task_ids()
if task_id in succeeded:
return
if worker.process.poll() is not None:
raise AcceptanceError("Replacement Celery worker exited before task success")
time.sleep(0.2)
raise AcceptanceError("Redelivered Celery task did not complete")
def _kill_worker(worker: WorkerProcess, *, timeout_seconds: int) -> int:
if worker.process.poll() is not None:
raise AcceptanceError("Celery worker exited before controlled termination")
worker.process.kill()
try:
return_code = worker.process.wait(timeout=timeout_seconds)
except subprocess.TimeoutExpired as exc:
raise AcceptanceError("Celery worker could not be killed") from exc
if return_code == 0:
raise AcceptanceError("Celery worker termination was not forced")
return return_code
def _stop_worker(worker: WorkerProcess, *, timeout_seconds: int) -> None:
if worker.process.poll() is None:
worker.process.terminate()
try:
worker.process.wait(timeout=timeout_seconds)
except subprocess.TimeoutExpired:
worker.process.kill()
worker.process.wait(timeout=timeout_seconds)
worker.log_handle.close()
def _broker_state(redis_url: str) -> RedisBrokerState:
client = Redis.from_url(
redis_url,
socket_connect_timeout=2,
socket_timeout=2,
decode_responses=False,
)
try:
return RedisBrokerState(
queue_depth=int(client.llen("send_email")),
unacked_hash_count=int(client.hlen("unacked")),
unacked_index_count=int(client.zcard("unacked_index")),
)
except RedisError as exc:
raise AcceptanceError("Redis broker state could not be inspected") from exc
finally:
client.close()
def _wait_for_broker_drained(
redis_url: str,
*,
timeout_seconds: int,
) -> RedisBrokerState:
deadline = time.monotonic() + timeout_seconds
last = RedisBrokerState(0, 0, 0)
while time.monotonic() < deadline:
last = _broker_state(redis_url)
if last == RedisBrokerState(0, 0, 0):
return last
time.sleep(0.2)
raise AcceptanceError("Redis broker retained delivery state after recovery")
def _queue_evidence(payload: Mapping[str, Any]) -> dict[str, Any]:
expected = {
"queued_count": 1,
"skipped_count": 0,
"blocked_count": 0,
"enqueued_count": 1,
"delivery_mode": "worker_queue",
"worker_queue_available": True,
"dry_run": False,
}
evidence = {key: payload.get(key) for key in expected}
if evidence != expected:
raise AcceptanceError("Campaign was not durably queued to one Celery task")
return evidence
def execute_redelivery_scenario(
client: Any,
headers: Mapping[str, str],
*,
fixture_path: Path,
settings: TestbedSettings,
endpoint: Any,
redis_url: str,
runtime_root: Path,
snapshot_probe: Callable[[str], tuple[Mapping[str, Any], Mapping[str, Any]]],
audit_probe: Callable[[str, str], Mapping[str, int]],
delivery_probe: Callable[[str, str], Mapping[str, Any]],
recover_claim: Callable[[str, str, subprocess.Popen[bytes]], Mapping[str, Any]],
) -> dict[str, Any]:
profile_id = create_mail_profile(
client,
headers,
settings,
name="Campaign Redis Celery redelivery drill",
smtp_host=endpoint.host,
smtp_port=endpoint.port,
)
prepared = prepare_campaign_scenario(
client,
headers,
fixture_path=fixture_path,
profile_id=profile_id,
settings=settings,
scenario="celery_broker_redelivery",
snapshot_probe=snapshot_probe,
)
first_worker = _start_worker(runtime_root, label="first-worker")
replacement_worker: WorkerProcess | None = None
try:
_wait_for_worker_ready(
first_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
queued = _expect(
client.post(
f"/api/v1/campaigns/{prepared.campaign_id}/queue",
headers=dict(headers),
json={
"version_id": prepared.version_id,
"include_warnings": True,
"enqueue_celery": True,
"dry_run": False,
},
),
200,
"Celery-redelivery Campaign queue",
)
queue_evidence = _queue_evidence(queued)
first_task_id = _wait_for_received_task(
first_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
if not endpoint.wait_for_data(settings.provider_timeout_seconds):
raise AcceptanceError("Celery worker did not reach complete SMTP DATA")
first_exit_code = _kill_worker(
first_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
endpoint.release_held_connection()
interrupted_state = _durable_state_evidence(
delivery_probe(prepared.campaign_id, prepared.version_id)
)
expected_interrupted = {
"job_count": 1,
"send_status_counts": {"sending": 1},
"attempt_status_counts": {"smtp_in_progress": 1},
"unfinished_attempt_count": 1,
}
if interrupted_state != expected_interrupted:
raise AcceptanceError("Killed worker state was not durably SMTP-in-progress")
replacement_worker = _start_worker(runtime_root, label="replacement-worker")
_wait_for_worker_ready(
replacement_worker,
timeout_seconds=settings.provider_timeout_seconds,
)
redelivered_task_id = _wait_for_received_task(
replacement_worker,
timeout_seconds=settings.provider_timeout_seconds,
expected_task_id=first_task_id,
)
_wait_for_task_success(
replacement_worker,
task_id=redelivered_task_id,
timeout_seconds=settings.provider_timeout_seconds,
)
redelivered_state = _durable_state_evidence(
delivery_probe(prepared.campaign_id, prepared.version_id)
)
if redelivered_state != interrupted_state:
raise AcceptanceError("Redelivered task changed an unfinished attempt without stopped-runtime proof")
expected_protocol = {
"connection_count": 1,
"accepted_rcpt_commands": 1,
"refused_rcpt_commands": 0,
"data_transactions": 1,
}
if endpoint.evidence() != expected_protocol:
raise AcceptanceError("Broker redelivery caused an unexpected SMTP transaction")
recovery_evidence = dict(recover_claim(
prepared.campaign_id, prepared.version_id, first_worker.process,
))
recovered_state = _durable_state_evidence(delivery_probe(prepared.campaign_id, prepared.version_id))
expected_recovered = {
"job_count": 1,
"send_status_counts": {"outcome_unknown": 1},
"attempt_status_counts": {"outcome_unknown": 1},
"unfinished_attempt_count": 0,
}
if recovered_state != expected_recovered:
raise AcceptanceError("Explicit fenced recovery did not freeze the unfinished attempt")
protocol = endpoint.evidence()
if protocol != expected_protocol:
raise AcceptanceError("Explicit claim recovery caused an unexpected SMTP transaction")
broker_after = _wait_for_broker_drained(
redis_url,
timeout_seconds=settings.provider_timeout_seconds,
)
first_received = first_worker.received_task_ids()
replacement_received = replacement_worker.received_task_ids()
if first_received != (first_task_id,) or replacement_received != (
redelivered_task_id,
):
raise AcceptanceError(
"Celery workers did not each receive the broker task exactly once"
)
report = _report_evidence(
_expect(
client.get(
f"/api/v1/campaigns/{prepared.campaign_id}/report",
headers=dict(headers),
params={"version_id": prepared.version_id},
),
200,
"Celery-redelivery Campaign report",
)
)
if report["send_status_counts"] != {"outcome_unknown": 1}:
raise AcceptanceError("Campaign report did not retain outcome_unknown")
audit_actions = dict(
sorted(audit_probe(prepared.campaign_id, prepared.version_id).items())
)
if not {
"campaign.created",
"campaign.validated",
"campaign.messages_built",
"campaign.queued",
}.issubset(audit_actions):
raise AcceptanceError("Celery-redelivery Campaign audit evidence is incomplete")
return {
**prepared.public_evidence(),
"queue": queue_evidence,
"interrupted_durable_state": interrupted_state,
"redelivered_durable_state": redelivered_state,
"recovered_durable_state": recovered_state,
"claim_recovery": recovery_evidence,
"protocol": protocol,
"report": report,
"audit_actions": audit_actions,
"broker": {
"transport": "redis",
"same_task_identity_redelivered": first_task_id
== redelivered_task_id,
"first_worker_received_count": len(first_received),
"replacement_worker_received_count": len(replacement_received),
**broker_after.as_dict(),
},
"supervision": {
"first_worker_killed_after_complete_data": True,
"first_worker_forced_exit": first_exit_code != 0,
"replacement_worker_started": True,
"replacement_worker_completed_redelivery": True,
"duplicate_task_left_sending_unchanged": True,
},
}
finally:
endpoint.release_held_connection()
_stop_worker(first_worker, timeout_seconds=5)
if replacement_worker is not None:
_stop_worker(replacement_worker, timeout_seconds=5)
def _runtime_module_versions(registry: Any) -> dict[str, str]:
versions = {"core": _core_package_version()}
versions.update(
{
manifest.id: manifest.version
for manifest in registry.manifests()
if manifest.id in {"access", "audit", "campaigns", "mail"}
}
)
return versions
def _create_runtime_root() -> Path:
"""Create the isolated runtime under the platform-selected temp root."""
return Path(tempfile.mkdtemp(prefix="govoplan-campaign-celery-redelivery-"))
def _bootstrap_and_run(
*,
settings: TestbedSettings,
fixture_path: Path,
redis_url: str,
visibility_timeout_seconds: int,
) -> dict[str, Any]:
runtime_root = _create_runtime_root()
database = None
try:
os.environ.update(
{
"APP_ENV": "test",
"DATABASE_URL": f"sqlite:///{runtime_root / 'acceptance.db'}",
"FILE_STORAGE_BACKEND": "local",
"FILE_STORAGE_LOCAL_ROOT": str(runtime_root / "files"),
"MOCK_MAILBOX_DIR": str(runtime_root / "mock-mailbox"),
"DEV_BOOTSTRAP_ENABLED": "false",
"CELERY_ENABLED": "true",
"REDIS_URL": redis_url,
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS": str(
visibility_timeout_seconds
),
"GOVOPLAN_CONNECTOR_ALLOW_PRIVATE_NETWORKS": "true",
}
)
from fastapi.testclient import TestClient
from govoplan_core.db.base import Base
from govoplan_core.db.bootstrap import bootstrap_dev_data
from govoplan_core.db.session import configure_database, set_database
from govoplan_core.settings import Settings, settings as core_settings
from govoplan_core.tenancy.scope import create_scope_tables
isolated_settings = Settings()
for field_name in Settings.model_fields:
setattr(core_settings, field_name, getattr(isolated_settings, field_name))
database = configure_database(os.environ["DATABASE_URL"])
set_database(database)
from govoplan_core.server.app import app
create_scope_tables(database.engine)
Base.metadata.create_all(bind=database.engine)
with database.SessionLocal() as session:
bootstrap_dev_data(
session,
api_key_secret="celery-redelivery-unused-api-key",
user_password="celery-redelivery-admin",
)
def snapshot_probe(
version_id: str,
) -> tuple[Mapping[str, Any], Mapping[str, Any]]:
from govoplan_campaign.backend.db.models import CampaignVersion
with database.SessionLocal() as session:
version = session.get(CampaignVersion, version_id)
if version is None:
raise AcceptanceError("Campaign execution snapshot is unavailable")
raw = version.raw_json if isinstance(version.raw_json, dict) else {}
snapshot = (
version.execution_snapshot
if isinstance(version.execution_snapshot, dict)
else {}
)
return raw, snapshot
def audit_probe(campaign_id: str, version_id: str) -> Mapping[str, int]:
from govoplan_audit.backend.db.models import AuditLog
with database.SessionLocal() as session:
actions = [
row[0]
for row in session.query(AuditLog.action)
.filter(AuditLog.object_id.in_([campaign_id, version_id]))
.all()
if row[0] in EXPECTED_AUDIT_ACTIONS
]
return dict(Counter(actions))
def delivery_probe(campaign_id: str, version_id: str) -> Mapping[str, Any]:
from govoplan_campaign.backend.db.models import CampaignJob, SendAttempt
with database.SessionLocal() as session:
jobs = (
session.query(CampaignJob)
.filter(
CampaignJob.campaign_id == campaign_id,
CampaignJob.campaign_version_id == version_id,
)
.all()
)
job_ids = [job.id for job in jobs]
attempts = (
session.query(SendAttempt)
.filter(SendAttempt.job_id.in_(job_ids))
.all()
if job_ids
else []
)
return {
"job_count": len(jobs),
"send_status_counts": dict(
Counter(job.send_status for job in jobs)
),
"attempt_status_counts": dict(
Counter(attempt.status for attempt in attempts)
),
"unfinished_attempt_count": sum(
1 for attempt in attempts if attempt.finished_at is None
),
}
with TestClient(app) as client:
login = _expect(
client.post(
"/api/v1/auth/login",
json={
"email": "admin@example.local",
"password": "celery-redelivery-admin",
},
),
200,
"Acceptance login",
)
access_token = str(login.get("access_token") or "")
if not access_token:
raise AcceptanceError("Acceptance login returned no access token")
from govoplan_core.core.runtime import get_registry
registry = get_registry()
if registry is None:
raise AcceptanceError("The GovOPlaN module registry is unavailable")
composition_versions = required_composition_versions(
fixture_path,
_runtime_module_versions(registry),
)
with smtp_fault_endpoint("post_data_hold") as endpoint:
scenario = execute_redelivery_scenario(
client,
{"Authorization": f"Bearer {access_token}"},
fixture_path=fixture_path,
settings=settings,
endpoint=endpoint,
redis_url=redis_url,
runtime_root=runtime_root,
snapshot_probe=snapshot_probe,
audit_probe=audit_probe,
delivery_probe=delivery_probe,
recover_claim=lambda campaign_id, version_id, process: recover_stopped_fixture_claim(
client, {"Authorization": f"Bearer {access_token}"}, database=database,
runtime_root=runtime_root, campaign_id=campaign_id, version_id=version_id,
stopped_process=process,
),
)
evidence = {
"schema_version": EVIDENCE_SCHEMA,
"generated_at": datetime.now(timezone.utc).isoformat(),
"fixture_sha256": hashlib.sha256(fixture_path.read_bytes()).hexdigest(),
"declared_module_versions": composition_versions,
"target": {
"kind": "local_redis_celery_controlled_smtp",
"isolated_temporary_database": True,
"redis_started_by_runner": True,
"worker_pool": "solo",
"worker_prefetch_multiplier": 1,
"task_acks_late": True,
"task_reject_on_worker_lost": True,
"visibility_timeout_seconds": visibility_timeout_seconds,
},
"scenario": scenario,
"coverage": {
"redis_broker_delivery": True,
"celery_worker_processes": True,
"forced_worker_loss_after_complete_data": True,
"same_task_broker_redelivery": True,
"redelivery_leaves_active_claim_unchanged": True,
"explicit_stopped_runtime_fenced_recovery": True,
"durable_outcome_unknown_recovery": True,
"duplicate_smtp_transaction_prevented": True,
"production_daemon_supervisor": False,
"target_provider": False,
"source_artifact_provenance": False,
},
}
_assert_evidence_safe(evidence, settings=settings)
rendered = json.dumps(
evidence,
ensure_ascii=False,
indent=2,
sort_keys=True,
).encode("utf-8") + b"\n"
if len(rendered) > MAX_EVIDENCE_BYTES:
raise AcceptanceError("Celery-redelivery evidence exceeds its size limit")
return evidence
finally:
if database is not None:
database.engine.dispose()
shutil.rmtree(runtime_root, ignore_errors=True)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--fixture", type=Path, default=DEFAULT_FIXTURE)
parser.add_argument("--compose-file", type=Path, default=DEFAULT_COMPOSE_FILE)
parser.add_argument("--redis-port", type=int)
parser.add_argument(
"--visibility-timeout-seconds",
type=lambda value: _positive_int(value, label="visibility timeout"),
default=os.environ.get(
"GOVOPLAN_CAMPAIGN_TEST_REDIS_VISIBILITY_TIMEOUT_SECONDS",
"3",
),
)
parser.add_argument(
"--timeout-seconds",
type=lambda value: _positive_int(value, label="timeout"),
default=60,
)
parser.add_argument("--evidence", type=Path)
args = parser.parse_args(argv)
try:
settings = TestbedSettings.from_environment()
settings.assert_local_testbed()
settings = replace(
settings,
provider_timeout_seconds=args.timeout_seconds,
)
with isolated_redis_broker(
compose_file=args.compose_file.resolve(),
timeout_seconds=args.timeout_seconds,
requested_port=args.redis_port,
) as redis_url:
evidence = _bootstrap_and_run(
settings=settings,
fixture_path=args.fixture.resolve(),
redis_url=redis_url,
visibility_timeout_seconds=args.visibility_timeout_seconds,
)
rendered = json.dumps(
evidence,
ensure_ascii=False,
indent=2,
sort_keys=True,
) + "\n"
if args.evidence:
args.evidence.parent.mkdir(parents=True, exist_ok=True)
args.evidence.write_text(rendered, encoding="utf-8")
else:
sys.stdout.write(rendered)
return 0
except AcceptanceError as exc:
print(f"Campaign Celery-redelivery acceptance failed: {exc}", file=sys.stderr)
return 1
except Exception as exc:
print(
"Campaign Celery-redelivery acceptance failed unexpectedly "
f"({type(exc).__name__}); inspect local service logs.",
file=sys.stderr,
)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
from __future__ import annotations
import imaplib
import mimetypes
import os
import tempfile
import time
from email.message import EmailMessage
from email.utils import formatdate, make_msgid
from pathlib import Path
from govoplan_mail.backend.config import ImapConfig, SmtpConfig, TransportSecurity
from govoplan_mail.backend.sending.imap import append_message_to_sent, list_imap_folders, list_imap_messages, test_imap_login
from govoplan_mail.backend.sending.smtp import send_email_message, test_smtp_login
from govoplan_campaign.backend.services.zip_service import create_zip_archive
def _env(name: str, default: str) -> str:
return os.environ.get(name, default).strip() or default
SMTP_HOST = _env("GOVOPLAN_MAIL_TEST_SMTP_HOST", "127.0.0.1")
IMAP_HOST = _env("GOVOPLAN_MAIL_TEST_IMAP_HOST", "127.0.0.1")
SMTP_PORT = int(_env("GOVOPLAN_MAIL_TEST_SMTP_PORT", "3025"))
IMAP_PORT = int(_env("GOVOPLAN_MAIL_TEST_IMAP_PORT", "3143"))
USERNAME = _env("GOVOPLAN_MAIL_TEST_USER", "campaign-test@govoplan.test")
PASSWORD = _env("GOVOPLAN_MAIL_TEST_PASSWORD", "campaign-test-password")
SENDER = _env("GOVOPLAN_MAIL_TEST_FROM", USERNAME)
RECIPIENT = _env("GOVOPLAN_MAIL_TEST_RECIPIENT", USERNAME)
SENT_FOLDER = _env("GOVOPLAN_MAIL_TEST_SENT_FOLDER", "Sent")
ZIP_PASSWORD = _env("GOVOPLAN_MAIL_TEST_ZIP_PASSWORD", "zip-test-password")
SERVICE_READY_TIMEOUT_SECONDS = int(_env("GOVOPLAN_MAIL_TEST_READY_TIMEOUT_SECONDS", "45"))
def _smtp_config() -> SmtpConfig:
return SmtpConfig(
host=SMTP_HOST,
port=SMTP_PORT,
security=TransportSecurity.PLAIN,
username=USERNAME,
password=PASSWORD,
timeout_seconds=10,
)
def _imap_config() -> ImapConfig:
return ImapConfig(
host=IMAP_HOST,
port=IMAP_PORT,
security=TransportSecurity.PLAIN,
username=USERNAME,
password=PASSWORD,
sent_folder=SENT_FOLDER,
timeout_seconds=10,
)
def _message(subject: str, body: str, attachments: list[Path] | None = None) -> EmailMessage:
msg = EmailMessage()
msg["From"] = SENDER
msg["To"] = RECIPIENT
msg["Subject"] = subject
msg["Date"] = formatdate(localtime=False)
msg["Message-ID"] = make_msgid(domain="govoplan.test")
msg.set_content(body)
for attachment in attachments or []:
content_type = mimetypes.guess_type(attachment.name)[0] or "application/octet-stream"
maintype, subtype = content_type.split("/", 1)
msg.add_attachment(attachment.read_bytes(), maintype=maintype, subtype=subtype, filename=attachment.name)
return msg
def _ensure_folder(config: ImapConfig, folder: str) -> None:
client = imaplib.IMAP4(host=config.host or "", port=config.port or 143, timeout=config.timeout_seconds)
try:
if config.username and config.password:
client.login(config.username, config.password)
typ, data = client.create(folder)
if typ not in {"OK", "NO"}:
raise RuntimeError(f"Could not create IMAP folder {folder!r}: {data!r}")
finally:
try:
client.logout()
except Exception:
pass
def _wait_for_delivery(config: ImapConfig, *, before_count: int, expected_increment: int) -> None:
deadline = time.monotonic() + 15
last_count = before_count
while time.monotonic() < deadline:
result = list_imap_messages(imap_config=config, folder="INBOX", limit=10)
last_count = result.total_count
if last_count >= before_count + expected_increment:
return
time.sleep(0.5)
raise RuntimeError(
f"Expected at least {expected_increment} new INBOX message(s), "
f"but count moved from {before_count} to {last_count}."
)
def _wait_for_service(label: str, check):
deadline = time.monotonic() + SERVICE_READY_TIMEOUT_SECONDS
last_error: Exception | None = None
while time.monotonic() < deadline:
try:
return check()
except Exception as exc: # GreenMail can bind before SMTP/IMAP is ready.
last_error = exc
time.sleep(1)
raise RuntimeError(f"{label} did not become ready within {SERVICE_READY_TIMEOUT_SECONDS}s: {last_error}") from last_error
def main() -> int:
smtp = _smtp_config()
imap = _imap_config()
smtp_login = _wait_for_service("SMTP", lambda: test_smtp_login(smtp_config=smtp))
imap_login = _wait_for_service("IMAP", lambda: test_imap_login(imap_config=imap))
print(f"SMTP login OK: {smtp_login.host}:{smtp_login.port} authenticated={smtp_login.authenticated}")
print(f"IMAP login OK: {imap_login.host}:{imap_login.port} authenticated={imap_login.authenticated}")
_wait_for_service("IMAP folder creation", lambda: _ensure_folder(imap, SENT_FOLDER))
folders = _wait_for_service("IMAP folder listing", lambda: list_imap_folders(imap_config=imap))
print("IMAP folders: " + ", ".join(folder.name for folder in folders.folders))
before_inbox = list_imap_messages(imap_config=imap, folder="INBOX", limit=10).total_count
with tempfile.TemporaryDirectory(prefix="govoplan-campaign-mail-test-") as tmp:
root = Path(tmp)
attachment = root / "single-attachment.txt"
attachment.write_text("GovOPlaN single attachment smoke test\n", encoding="utf-8")
zip_source = root / "zip-source.txt"
zip_source.write_text("GovOPlaN password-protected ZIP smoke test\n", encoding="utf-8")
zip_path = create_zip_archive(root / "password-protected.zip", [zip_source], ZIP_PASSWORD)
messages = [
_message("[GovOPlaN test] no attachment", "No attachment SMTP/IMAP smoke."),
_message("[GovOPlaN test] one attachment", "One attachment SMTP/IMAP smoke.", [attachment]),
_message("[GovOPlaN test] password ZIP", "Password-protected ZIP SMTP/IMAP smoke.", [zip_path]),
]
for message in messages:
result = send_email_message(
message,
smtp_config=smtp,
envelope_from=SENDER,
envelope_recipients=[RECIPIENT],
)
print(f"SMTP accepted {result.accepted_count} recipient(s): {message['Subject']}")
append = append_message_to_sent(bytes(message), imap_config=imap, folder=SENT_FOLDER)
print(f"IMAP appended {append.bytes_appended} byte(s) to {append.folder}: {message['Subject']}")
_wait_for_delivery(imap, before_count=before_inbox, expected_increment=3)
sent_count = list_imap_messages(imap_config=imap, folder=SENT_FOLDER, limit=10).total_count
if sent_count < 3:
raise RuntimeError(f"Expected at least 3 messages in {SENT_FOLDER!r}, found {sent_count}.")
print("Transport smoke OK: no attachment, one attachment, and password-protected ZIP messages verified.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+67
View File
@@ -0,0 +1,67 @@
# Campaign Accessibility Review
The Campaign WebUI uses Core's semantic `Button`, `Dialog`, `DataGrid`, form,
alert, and segmented-control components. Its page frames use Core `PageLayout`
and its full-canvas navigation/content shells use `WorkspaceLayout`, so pane
scrolling, sticky headings, action collapse, narrow-layout behavior, and help
scope remain platform-owned. Core also owns focus trapping, focus return,
Escape handling, labels, disabled state, and keyboard behavior for the shared
primitives.
## Repeatable review matrix
Run this matrix for Campaign overview, wizard, sender and recipients,
attachments, template editing, review and send, operator queue, and reports:
1. Navigate all actions with Tab and Shift+Tab; focus must remain visible and
follow the visual reading order.
2. Activate buttons and links with Enter, and native buttons with Space.
3. Open every dialog, verify initial focus remains within it, close with Escape,
and verify focus returns to the opener.
4. Use DataGrid sorting, filtering, pagination, row selection, and action menus
without a pointer.
5. At 200 percent browser zoom and a 320 CSS-pixel viewport, verify that content
reflows or scrolls without hiding actions.
6. With reduced motion enabled, verify that workflow state does not depend on
animation.
7. With a screen reader, verify page headings, field labels, validation errors,
workflow states, message navigation, and attachment evidence.
## Automated structural guard
`npm run test:accessibility-contract` rejects non-semantic click handlers,
unlabelled icon-only buttons in the message preview, and Campaign-local modal
implementations that bypass Core's `Dialog`. It complements rather than replaces
browser and assistive-technology testing.
The guard also verifies that Campaign retains narrow-viewport layouts, visible
keyboard focus for domain-specific controls, and an explicit reduced-motion
override. Shared dialog focus trapping and restoration are tested in Core;
Campaign tests verify that overlays continue to use that shared primitive.
## Release evidence
The feature implementation can be closed once the structural contract, shared
Core component tests, TypeScript graph, and representative responsive overlay
tests pass. The seven-step matrix above remains a release-candidate checklist:
it must be repeated for the exact browser, language packages, theme, density,
and assistive-technology combination being certified. Closing the implementation
ticket does not make a general WCAG-conformance claim for future releases.
Implementation closure evidence recorded on 2026-08-03 used headless Chromium
against the configured development system. It covered all 13 Campaign routes at
1440 CSS pixels, the list, overview, recipients, and review routes at 320 CSS
pixels, reduced-motion rendering, semantic accessibility-tree snapshots, HTTP
500/page-error capture, and 42 sampled Tab stops across desktop and narrow
Review & Send. The run found no unnamed interactive controls, hidden focus,
page-level horizontal overflow, titlebar overlap, or unhandled runtime errors.
Representative list, overview, review, and narrow-review screenshots were also
inspected. This is implementation evidence; release certification still uses
the complete matrix above with the selected screen reader and browser versions.
## Known boundary
Translation keys may be visible in source because Core resolves them at runtime.
The review must use a built application with current language packages. WCAG
conformance is a release-level claim and still requires a bounded manual audit
of the release candidate.
+87
View File
@@ -0,0 +1,87 @@
# Campaign Access Explanation Coverage
Campaign access explanations are resource-specific evidence. They inherit the
parent Campaign decision only where the child has no independent grant model,
and they must identify that inheritance explicitly.
## Implemented
- Campaign
- Campaign version
- Campaign delivery job / built message
- Computed Campaign report, identified by Campaign, version, and report kind
- Recipient row, identified by `<version UUID>:<job UUID>`
- Frozen recipient source snapshot, identified by its Campaign-version UUID
- Campaign attachment binding and version-bound frozen attachment resolution
- Persisted validation issue, version-bound review decision, and attachment-policy override
- SMTP, IMAP append, Postbox, and printable-output attempts
- Message action, message-action attempt, and job reconciliation decision
- Campaign share and Core-owned Campaign ownership-transfer record
- Independently user-owned recipient import mapping profile
- Saved recipient import execution, identified by `<version UUID>:<import UUID>`
- Persisted validation, build, execution-snapshot, and review evidence, identified
by `<version UUID>:<artifact kind>`
All persisted child IDs are random UUIDs. Embedded build/review children use a
version UUID plus a random job UUID, so callers cannot enumerate a recipient
index or infer an address. A version mismatch is reported as a stale reference.
Missing and cross-tenant children use the same non-disclosing not-found
provenance. Explanations never include recipient addresses, source rows,
filenames, object locators, transport responses, worker claims, target
snapshots, diagnostic text, or reconciliation notes.
## Permission matrix
| Evidence | Parent boundary | Further restriction |
| --- | --- | --- |
| Recipient row or source snapshot | Campaign read/owner/share | `campaigns:recipient:read` |
| Attachment binding/resolution, validation, review, override | Campaign read/owner/share | Campaign review and `campaigns:diagnostic:read` |
| Delivery status | Campaign read/owner/share | `campaigns:report:read` |
| Transport or worker diagnostics | Campaign read/owner/share | `campaigns:diagnostic:read` |
| Exported delivery evidence | Campaign read/owner/share | `campaigns:report:export` |
| Reconciliation decision | Campaign read/owner/share | Campaign reconcile and diagnostic read |
| Share or ownership transfer | Campaign governance | Campaign share, transfer-participant, group-acceptance, or recovery authority; content access remains a separate decision |
| Import mapping profile | Independent user owner | `campaigns:recipient:import`; no Campaign share is inherited |
| Import execution | Campaign read/owner/share | Recipient read and import authority |
| Persisted protocol artifact | Campaign read/owner/share | Recipient, review, report, diagnostic, or export authority appropriate to the artifact |
Postbox, Mail/IMAP, and printable attempts keep bounded Campaign-owned evidence
after provider acceptance. Their explanation therefore remains available when
an optional provider module is later disabled. A missing attempt reports only
the optional owner and `unavailable_or_hidden`; it does not distinguish absence
from hidden data.
## Optional and unsupported owner boundaries
Reusable templates and template revisions are independently governed by the
optional Templates module; Campaign never treats a Campaign share as a template
grant. Durable export packages are independently governed by the optional
Reporting module. Asking the Campaign provider to explain either class therefore
fails closed with `independently_governed_by_optional_module` and
`unavailable_or_hidden`. The response does not reveal whether the optional
module is absent, the object does not exist, or the caller cannot see it.
Campaign reports generated on demand remain non-persisted, version-bound
resources. Their explanation names the report kind and its Campaign/version
parent, and keeps report read, export, and diagnostic permissions distinct.
Each child explanation must include:
- the child resource identity and current state;
- the parent Campaign and version where applicable;
- whether access is inherited, independently granted, or further restricted;
- effective owner/share/policy provenance;
- missing-module or unavailable-evidence reasons without leaking the hidden
object;
- a stable resource identifier suitable for audit and support links.
Delivery attempts, review decisions, reports, and exports can contain more
sensitive evidence than the Campaign summary. Their read and diagnostic/export
permissions therefore remain independently enforceable even when the parent
Campaign is readable.
Import explanations include only the stable import identity, source type,
opaque source identity, source revision, and whether additional provenance was
recorded. They never return imported rows, filenames, column mappings, or source
metadata. Mapping-profile explanations expose only non-reversible header
fingerprints and shape information; headers and mappings remain hidden.
+75
View File
@@ -0,0 +1,75 @@
# Campaign Build Recovery
Campaign build is fenced per tenant and Campaign version. Before rendering or
writing generated artifacts, Campaign commits a Core recovery operation with
the canonical source and validation hashes, the runtime fence, and a reserved
opaque object prefix. A repeated idempotency key can replay only a verified
successful build; it cannot start a second active build.
Generated EML and bounded print output are written to shared storage and checked
for exact size and SHA-256 content. Campaign renews the build fence after that
check and before changing jobs, then commits its jobs and execution snapshot,
compares the stored object and database manifests, and records verified success.
A managed Files output makes the operation forward-recoverable because Campaign
cannot undo a Files-owned artifact; object-only builds use explicit
compensation.
If the Campaign database transaction fails, Campaign deletes every object it
recorded and verifies absence before recording recovered state. Failed deletion,
an unavailable storage check, process loss, or superseded-object cleanup failure
leaves a recovery-required operation visible in Ops. Do not retry such an
operation as a normal build. Verify its checkpoint chain and reserved prefix,
then reconcile it through the owning-module procedure.
## Orphan inventory and cleanup
The operator-only endpoint
`POST /api/v1/campaigns/operations/artifacts/reconcile` requires
`system:settings:write`. It never scans outside
`campaign-artifacts/{tenant_id}/`, and one request reads at most `page_size`
objects. The default request is a dry run:
```json
{}
```
The response reports each eligible key, size, modification time, age, reason,
page totals, and `next_cursor`. Continue with that cursor to inspect the next
bounded page. Objects are not eligible when they are referenced by a committed
EML or print-output row, belong to an actively fenced build, have no trustworthy
modification time, have an invalid build-key shape, or are younger than the
grace period. The minimum and default grace period is 24 hours.
Apply an inspected page with a new idempotency key:
```json
{
"apply": true,
"idempotency_key": "incident-2026-08-03-page-1",
"grace_period_hours": 24,
"page_size": 250
}
```
An applied run rechecks committed references and active build leases before
each bounded deletion batch. A Core distributed lease prevents two nodes from
committing the same tenant cleanup concurrently. Every attempted deletion is
probed afterward. `recovery_required` means an object was verified to remain;
`outcome_unknown` means storage could not prove whether the delete took effect.
Use a new idempotency key to retry after the storage problem is corrected. A
successful repeated request with the same key returns `already_completed` and
does not delete again.
Cleanup never clears Campaign database references. Audit and recovery evidence
records counts and hashed manifests rather than object keys. Exact keys are
returned only by this privileged endpoint and must stay in restricted incident
records.
No recovery checkpoint contains message bodies, recipients, credentials, or
resolved provider secrets. Object keys remain restricted diagnostics rather
than Campaign business data.
Database rows, `campaign-artifacts/` objects, and the encryption/key service are
one coordinated backup and recovery boundary. After any partial restore, pause
delivery, run a dry inventory, reconcile Campaign recovery operations in Ops,
and verify referenced object hashes before workers resume.
+227 -11
View File
@@ -5,16 +5,22 @@ been validated, built, reviewed, and locked.
## Operating Modes
- Local direct send: `CELERY_ENABLED=false`. Queueing stores jobs in the DB, and
small development runs can be processed with "Send queued now".
- Local direct send: `CELERY_ENABLED=false`. **Send now** is available only for
exact built runs within the effective synchronous limit. It preflights the
complete batch before the first SMTP effect.
- Worker send: `CELERY_ENABLED=true` with Redis/Celery workers running. Queueing
publishes delivery tasks, and the Review & Send page polls summary counters.
publishes durable delivery tasks, and Review and send polls their persisted
summary counters even after the initiating request has returned.
- Mock send: use only for development review. It does not prove real SMTP/IMAP
credentials or server policy.
## Before First Live Use
- Use dedicated non-production SMTP/IMAP credentials.
- Store and test those credentials in a Mail-module profile. Campaign must
contain only the selected `server.mail_profile_id` reference.
- Start the repository test bed in `dev/mail-testbed/` when a local
production-like SMTP/IMAP server is sufficient.
- Use a dedicated mailbox/folder for append-to-Sent tests.
- Confirm policy allows the SMTP host, envelope sender, recipients, and optional
IMAP append target.
@@ -22,29 +28,108 @@ been validated, built, reviewed, and locked.
ZIP before using production recipients.
- Keep the report page open during tests; it is the operational source of truth
for attempts, outcomes, and reconciliation.
- Confirm the effective Send now recipient-job limit. The safe default is 25;
use Queue for workers for ordinary batches or any run above that limit.
System administrators can explicitly configure 0500 under Administration →
SYSTEM → Campaign delivery; an explicit deployment ceiling remains binding,
and TENANT policy may only narrow the inherited limit. This setting affects
one synchronous request, not campaign size. It is audited and never sends
messages; large interactive requests may encounter proxy timeouts.
## Deliverability Preflight
The Mail server connection test checks that selected server and credential. It
does not authorize the campaign's sender, recipients, or resource selection.
SMTP runtime checks require the selected SMTP credential when policy forbids
inheritance, independently of IMAP. Sent-folder append checks IMAP credentials
independently; full campaign validation still checks both required selections.
Preflight errors distinguish Mail profile/credential policy, SMTP configuration,
authentication, and connectivity. Do not change TLS or credential policies merely
because a campaign preflight failed. A preflight rejection leaves staged jobs
uncommitted and starts no message delivery.
Before the first live send for a sender domain or mail-server profile:
- Confirm the selected SMTP identity matches the visible From/envelope sender
policy.
- Confirm SPF, DKIM, and DMARC are handled by the sending infrastructure or
documented as out of scope for the selected test environment.
- Confirm rate limits are explicitly set for the expected provider and recipient
volume.
- Confirm bounce/reply/notification addresses are monitored by an operational
mailbox or intentionally disabled.
- Confirm large attachments and password-protected ZIPs are acceptable for the
recipient systems.
- Confirm owner transfer or policy changes force profile reselection and
revalidation before live delivery.
## Queue And Send
1. Validate the version with file checks enabled.
1. Link all required managed files, then validate the version with file checks
enabled. Locking waits for a fresh attachment preview and asks for explicit
**Link and lock** confirmation when matches are not linked. A locked version
cannot change attachment links: use an editable version and repeat validation,
build and review rather than assuming unlinked files were included.
2. Build the version and inspect all blocking review items.
3. Queue only after the selected version is the intended immutable execution
version.
4. In local mode, use "Send queued now" for small test runs.
version. Select **Queue for workers**, then verify the committed and
published counts.
4. Use **Send now** only if the exact eligible count is non-zero and at or below
the effective deployment/system/tenant limit shown on the page.
5. In worker mode, verify queue counters move from queued/claimed/sending to a
terminal SMTP state.
6. If a synchronous request is used, keep its blocking progress dialog open.
Only its small version-scoped persisted counters refresh; the workspace,
recipient list, attachment preview and full summary stay unchanged. Read-only
refresh failure retains the last counters and does not prove delivery failed.
A disconnected request may still be executing; never repeat it blindly.
Oversized initial runs are rejected before SMTP and directed to workers.
7. Review the SMTP batch line. `ready` means DNS/connectivity/TLS/auth preflight
succeeded. Connection and reconnect counts explain reuse. `paused` means a
systemic transport failure stopped the remaining jobs before their SMTP
effect; test/correct the Mail profile and explicitly resume the queue.
## Outcome Handling
Each real Campaign job delivery is represented by a Core recovery-ledger
operation before the worker claims the job or invokes Mail, Postbox, or print.
Synchronous batches use the same boundary after their batch-wide preflight.
Explicit test/resend actions and post-acceptance IMAP appends use separate
action-fenced operations. Operations store only opaque IDs, digests, channel
policy, and bounded status evidence. A verified acceptance becomes
`succeeded`, a definitive pre-effect or provider rejection becomes `rejected`,
and uncertain or stranded effects remain `outcome_unknown` or
`recovery_required` in Ops. Campaign jobs, message actions, and channel-attempt
records remain the business source of truth.
For SMTP and Sent-folder APPEND, Campaign also passes the stable job/action and
attempt identifier into Mail. Mail establishes its own provider-bound recovery
fence after profile authorization and policy checks but before network I/O.
This nested ownership is intentional: Campaign proves its business transition,
while Mail proves the transport effect. Neither layer replays a completed or
unknown provider attempt merely to repair the other layer's state.
- `smtp_accepted`: Do not retry. If IMAP append is enabled and pending, run or
enqueue the append action.
- `failed_temporary`: Retry explicitly after checking the error and retry count.
- `failed_permanent`: Retry only if the operator has corrected the root cause and
intentionally includes permanent failures.
- `paused` after a systemic SMTP failure: do not resume until the shared Mail
profile passes its connection test. Authentication, sender rejection, and
unavailable connectivity affect the batch rather than one recipient.
- `outcome_unknown`: Do not retry directly. Check SMTP logs, mailbox evidence, or
provider control panels, then reconcile as accepted or not sent.
- `claimed` or `sending` that does not progress: treat as a worker interruption.
Re-run worker handling or reconcile if SMTP may already have accepted the
message.
- `claimed` or `sending` that does not progress: investigate the owning runtime.
Duplicate worker handling leaves active state unchanged. Never infer from
elapsed time alone that SMTP did not accept the message. Use the fenced
**Recover interrupted claim** action described below when it is available.
- IMAP `appending`: A worker owns the durable append claim. Do not start a
second append; if the worker cannot finish, reconcile only after checking the
mailbox.
- IMAP `outcome_unknown`: Never append automatically. An operator with
`campaigns:campaign:reconcile` must record an evidence note and resolve it as
`imap_appended` or `imap_not_appended`. Only the latter becomes explicitly
retryable.
## Reconciliation
@@ -55,6 +140,65 @@ been validated, built, reviewed, and locked.
- Add a note that identifies the evidence used, for example SMTP log line,
provider message ID, or operator ticket.
For pure-Mail SMTP and channel-specific IMAP operations, reconciliation updates
the original Campaign attempt, matching Campaign recovery operation and audit
record atomically under a fresh lease. A SMTP-only decision cannot resolve an
entire compound Mail/Postbox/Print operation; that ledger remains separately
unresolved until all of its effects are established. Campaign does
not rewrite Mail-owned nested provider-effect operations: those remain Mail's
separate evidence and operational responsibility. A failed audit or conflicting
claim must leave the prior unknown state intact.
### Recovery without workers
The Report offers explicit inline retry and continuation when workers are not
configured. Retry uses `campaigns:campaign:retry` plus
`campaigns:campaign:send`; continuation uses `campaigns:campaign:queue` plus
`campaigns:campaign:send`. Both use the canonical immutable jobs and ordinary
attempt ledger, not a separate one-message resend. Current Mail authorization,
review/approval, execution integrity, retry limits and rate limits still apply.
Each call is bounded by the effective synchronous recipient-job limit and
reports remaining eligible work. Continue explicitly until none remains; it
never selects accepted, excluded, active, uncertain or known failed jobs.
Known failures have their own explicit retry action. These actions do not
turn a long HTTP request into a background worker or guarantee exactly-once
SMTP when a provider acknowledgement is lost.
For an abandoned active SMTP or IMAP claim, the report exposes recovery only
after the original durable lease expires **and** the runtime registry proves
that its owner stopped or was replaced. A stale heartbeat is insufficient.
The opaque claim revision and original recovery evidence are rechecked under a
fresh lease. Recovery records the effect as **outcome unknown**, never not sent.
Then separately inspect external evidence and reconcile with a factual note
before any retry. This requires `campaigns:campaign:reconcile`. If the original
lease, evidence, or stopped-owner proof is missing, preserve the records and
investigate through Ops; do not edit delivery rows or force a lease expiry in
a real installation.
### Progress totals and Sent-folder batching
For each channel, **processed** includes successful, failed, uncertain and
cancelled outcomes. **In progress** is separate from pending, so the currently
sending/appending message remains visible. Paused SMTP work is also separate.
Excluded/non-requested channel work is outside the denominator. The endpoint
requires campaign read and object access and returns no recipient addresses,
message bodies, attachments or credentials. It is not the privacy-thresholded
aggregate Reports view and does not grant that view recipient access.
Append-to-Sent targets the selected campaign version, not all historical
versions. Each message remains a separately fenced, sequential IMAP APPEND.
Mail reuses the authenticated session and resolved folder for at most 100
messages or 300 seconds by default, then rotates the connection. Current
authorization, frozen transport revisions, credentials and recovery checks
still run for every message. A stale idle connection is checked before another
APPEND; an uncertain APPEND is never replayed. This removes repeated
connect/login/folder-list round trips, not the time needed to upload each EML.
It is not an atomic MULTIAPPEND transaction or parallel delivery.
SMTP and IMAP use the same progress dialog. An acknowledged operation remains
successful even if loading its follow-up diagnostics fails: use Reload to
refresh display, not to repeat the external effect.
## Fault Injection Checklist
Use mock infrastructure first, then repeat against the non-production real test
@@ -67,11 +211,83 @@ bed where possible:
- IMAP append failure after SMTP acceptance.
- Worker restart with queued, claimed, and sending jobs.
For the maintained loopback baseline, run
`dev/mail-testbed/run_campaign_acceptance.py`. It proves the public Campaign
path for SMTP acceptance, IMAP append, repeat-send blocking, an SMTP connection
failure before transmission, an explicit SMTP authentication rejection, an
explicit temporary `451` response after DATA, partial RCPT refusal, a
connection loss after complete DATA, and an IMAP authentication rejection
after SMTP acceptance. Its evidence is an allowlisted classification/count
projection; raw provider diagnostics and transport/account identifiers are
deliberately excluded.
The runner also terminates a dedicated OS process executing the registered
Campaign send task after complete DATA, then invokes the task in a fresh
process. Redelivery must leave the active claim unchanged and the endpoint
must observe no second connection or DATA transaction. Only a subsequent
explicit, fenced recovery with test-fixture stopped-owner and expired-lease
proof may change the unfinished attempt to `outcome_unknown`. This covers the
worker task/process boundary but not a broker or daemon.
Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` for the maintained
Redis/Celery delivery and broker redelivery boundary. It starts an isolated
Redis Compose service and real Celery workers, kills the first solo worker after complete DATA while the
late-ack task is unacknowledged, and requires the same task identity to reach a
replacement worker after Redis visibility recovery. Passing evidence also
requires unchanged active state on duplicate delivery, followed by explicit
fenced recovery to `outcome_unknown`, an empty broker queue/unacked set, and
exactly one SMTP connection and DATA transaction. Lease expiration is simulated
only in the isolated fixture after its owner was stopped. Raw worker logs and task,
database, endpoint, and credential identifiers are never retained.
That second runner proves local runner-supervised process replacement, not the
production process manager. Repeat the worker-loss drill under the selected
systemd, container, Kubernetes, or other production supervisor and the target
Redis/SMTP infrastructure before deployment approval.
## Shared Build Artifacts
Generated EML is stored through Core's configured object-storage backend under
opaque Campaign-owned keys. The job records expected byte size, SHA-256 digest,
and Message-ID. A worker on another node must retrieve and verify those values
before attempting delivery.
- Do not expose object keys to ordinary campaign users or copy them into
business fields.
- A build failure deletes objects written before the database transaction can
commit.
- Retention starts a job-fenced forward-recovery operation before deletion,
commits metadata changes in the Campaign-owned boundary, and independently
probes the original locator before reporting success. An unavailable backend
leaves an outcome-unknown operation; a deletion/metadata mismatch becomes
recovery-required in Ops.
- A hard process loss between object creation and metadata commit can leave an
orphan object. Use the operator-only, dry-run-first Campaign artifact
reconciler documented in `CAMPAIGN_BUILD_RECOVERY.md`; it scans one bounded
tenant-prefix page, enforces a minimum 24-hour grace period, protects active
build fences, and rechecks committed EML and print-output references before
deletion.
- Restore Campaign rows, object storage, and the encryption key to one
coordinated recovery point before resuming workers.
For a scaled-runtime drill, build on one API replica, consume from another
worker, compare the stored evidence, and inject storage failures during build
and retention.
## Reporting Checks
- The recipient-aware report shows every frozen To/Cc/Bcc address in authored
order, with a primary-address fallback only for old rows without that snapshot.
SMTP envelope evidence, not the old primary-only UI, establishes how many
recipients were actually offered to the provider. SMTP and IMAP diagnostics
use list filters and consistent translated labels.
- Partial delivery must show accepted, failed, and unknown counts separately.
- Excluded messages must show SMTP and IMAP as `skipped`, with skipped counts
and filters separate from unattempted or failed delivery.
- Accepted and unknown jobs must not appear in retry selections.
- Reconciled accepted jobs must remain protected from resend.
- Reconciled not-sent jobs must appear only as explicit retry candidates.
- The final report should include SMTP attempts, IMAP append attempts, and any
reconciliation notes before the campaign is considered operationally closed.
- The final CSV export should include message id, resolved envelope headers,
attachment evidence, EML reference/checksum, latest SMTP response/error, and
latest IMAP folder/error before the campaign is considered operationally
closed.
+946
View File
@@ -0,0 +1,946 @@
# Campaign Handbook
## Purpose and status
This is the canonical, multi-perspective handbook for the Campaign module. It
describes the current implementation, the operational contract it relies on,
and the remaining work required before Campaign can be presented as GovOPlaN's
maintained reference composition.
Use the section that matches the task at hand:
| Perspective | Start here |
| --- | --- |
| Campaign author | [Prepare a campaign](#prepare-a-campaign) |
| Reviewer | [Review and complete review](#review-and-complete-review) |
| Sender or delivery operator | [Deliver and resolve outcomes](#deliver-and-resolve-outcomes) |
| Campaign or tenant administrator | [Administration and policy](#administration-and-policy) |
| Platform operator | [Operations and recovery](#operations-and-recovery) |
| Integrator or developer | [Composition and integration contracts](#composition-and-integration-contracts) |
| Security, privacy, or audit reviewer | [Assurance model](#assurance-model) |
| Release reviewer | [Reference-composition acceptance](#reference-composition-acceptance) |
The shorter task documents remain useful companions:
- [Campaign delivery runbook](CAMPAIGN_DELIVERY_RUNBOOK.md)
- [Mail profile boundary](MAIL_PROFILE_BOUNDARY.md)
- [Recipient import guide](RECIPIENT_IMPORT_GUIDE.md)
- [Recipient and Addresses boundary](RECIPIENT_ADDRESS_BOUNDARY.md)
- [Examples and release checklist](EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md)
## What Campaign is for
Campaign turns governed source data into individually built messages or
printable output and then controls their review, delivery, and evidence. It is
intentionally a composition module: it demonstrates how one user journey can
use optional Mail, Files, Addresses, Distribution Lists, Templates, Postbox,
and Notifications capabilities alongside Core access/audit infrastructure
without copying ownership from those modules. Policy may consume Campaign
context through a narrow capability; Campaign does not import Policy.
Campaign owns:
- the communication purpose, content, campaign-local fields, and templates;
- campaign-local recipient snapshots, exclusions, and personalization;
- message and attachment rules for a version;
- validation, review, build, queue, and delivery-control state;
- the durable jobs and attempts needed to explain delivery outcomes; and
- campaign-specific reports, shares, frozen execution evidence, and governed
human collaboration entries.
Campaign does not own:
- SMTP/IMAP profiles, credentials, protocol adapters, or mailbox policy;
- long-lived address-book master data, consent lifecycle, or deduplication;
- managed file bytes, connector credentials, or external-file provenance;
- general-purpose workflow definitions; or
- global identity, organization, permission, or retention policy.
Those boundaries matter in both persistence and UI. A campaign references an
authorized Mail profile and managed file versions; it must never become a
second secret store, file store, or address directory.
## Process view
The supported process is a controlled progression, not a single "send" call:
```text
create/edit
-> validate and resolve policy/integrations
-> review warnings and blockers
-> build exact recipient messages and/or printable artifacts
-> complete review and queue
-> selected Mail, Postbox, or print effect per job
-> optional IMAP append per accepted job
-> report, retry, reconcile, or correct
-> archive when no active/uncertain delivery remains
```
Campaign status summarizes the whole campaign. Version workflow state describes
the selected immutable/editable version. Each recipient job separately records
build, validation, queue, SMTP, and IMAP state. Operators must use the job-level
states when deciding whether another external effect is safe.
Important distinctions:
- **Validation lock** is the reversible lock created by a successful
validation/build path. Editing requires unlocking and invalidates derived
evidence as appropriate.
- **User lock** is an explicit audit-safe lock. A permanently locked or
delivery-final version is not edited in place; create an editable successor.
- **SMTP accepted** means the provider accepted the message. It does not prove
inbox delivery, reading, or business acknowledgement.
- **Outcome unknown** means an attempt may have had an external effect. It must
be investigated and reconciled before retry.
- **IMAP append** is a separate effect after SMTP acceptance. An append failure
is not evidence that sending failed.
- **Archive** preserves evidence. Draft-only campaigns without built, locked,
or delivery evidence may be deleted where policy allows; evidence-bearing
campaigns are archived instead.
- **Copy campaign** creates a new campaign and one fresh editable version from
the selected source version. It copies configuration, but never delivery
jobs, outcomes, shares, locks, or audit evidence.
- **Archive historical version** hides only a non-current version from the
default history. It does not change the version's workflow state or remove
configuration, reports, delivery results, or audit evidence.
Campaign lifecycle confirmations are bound to the state shown in the UI. If a
job, version, share, or campaign state changes before confirmation, the server
rejects the stale action and requires a reload. The lifecycle-policy response
states the applicable built-in rule and the reason for every unavailable
action.
## User tasks
### Discuss campaign work
Open **Collaboration** inside a Campaign to keep human coordination beside the
work without changing its version history. Discussion access is independent
from Campaign editing: parent Campaign read access remains mandatory, while
`campaigns:discussion:read`, `campaigns:discussion:post`, and
`campaigns:discussion:moderate` separately control reading, posting, and
moderation. A read share is sufficient as the parent grant and a comment never
turns that share into write access.
Comments are append-only and bounded to 8,000 characters. They can carry one
validated reference to an immutable Campaign version, saved recipient import,
attachment rule, delivery job, or report. References to version-bound evidence
include the exact version ID and never edit that version. Authors can withdraw
their own comments; moderators can redact comments and use moderator-only
visibility. Both operations remove displayed text but preserve a tombstone,
content hash, actor snapshot, timestamp, reference context, and bounded Audit
event. There is deliberately no comment-edit API.
Mentions are limited to 20 active users who already have Campaign ownership or
share access. When Notifications is installed and healthy, Campaign emits a
content-free in-app mention notification. Collaboration remains usable without
Notifications. The thread displays only human discussion; approvals, workflow
state, delivery events, and durable system evidence remain on their owning
surfaces and in Tenant audit.
### Assign accountable campaign work
Open **Work** to assign one bounded purpose to an account, group, or
organization function that already has Campaign access. Assignment records
responsibility only: it never creates a share, transfers ownership, or grants a
permission. Assignees may accept, complete, or reject their work; rejection is
distinct from administrative cancellation. Managers may reassign or cancel
open work, and every transition retains the expected revision, actor snapshot,
typed target, and append-only event history.
Workflow may create or reference a Campaign and open the same assignment through
the optional `campaigns.workOrchestration` capability. Those assignments pin the
Campaign version and store the Workflow instance, step, correlation, and
idempotency provenance. Campaign emits `campaign.work.changed` for assignment,
acceptance, start, reassignment, completion, rejection, and cancellation.
Workflow uses the assignment ID and event revision, rechecks current Campaign
access, and then resumes the matching durable external hand-off without browser
polling. A missing Tasks or Notifications capability only removes the optional
projection or notification. A missing Campaign provider, revoked Campaign
access, or stale event revision keeps the Workflow blocked and inspectable.
Campaign also contributes the opt-in **Accountable Campaign work hand-off**
Workflow template. It is deliberately not activated on installation. A
configurator must copy or activate it and supply either `campaign_id` or
`create_campaign`; unused optional input keys must be present with `null`
values. The template prepares the assignment idempotently, opens the exact
Campaign work URL, and waits for completion, rejection, cancellation, or the
configured timeout. Opening the link never completes the Workflow.
### Prepare a campaign
1. Create a campaign and confirm its owner or owning group.
2. Define global settings, fields, templates, and recipient data.
3. Import one-off recipient data or select a reusable Addresses source when the
optional capability is installed. Review source provenance and stale-source
warnings; Campaign freezes the selected rows rather than following later
directory changes silently.
4. Select managed attachments through Files. Server/API campaigns do not accept
arbitrary local paths. Preview rules and unmatched files before building.
5. Open **Mail settings** and select an available Mail profile. The campaign
stores only `server.mail_profile_id`; it never accepts SMTP/IMAP settings,
usernames, passwords, or credential references.
If the selected profile permits a campaign-scoped Mail credential, that
credential remains Mail-owned even though it is created from this surface.
For that credential and for password-valued campaign fields, the shared
password generator keeps its candidate separate from the form until **Use
password** is explicitly confirmed. Copying a candidate does not save or
submit it.
If legacy transport data is reported, choose **Migrate selected Mail
profile**, including when the existing profile selection is unchanged.
Follow a locked version's supported unlock or editable-successor action
before migration. Migration is explicit and audited, never sends mail, and
requires validation, build, and review again.
6. Save the editable version, validate the relevant sections, and resolve every
blocking issue. Warnings remain explicit review decisions.
7. Build the exact messages and inspect recipient, addressing, template,
attachment, and generated-message evidence.
If a selected optional module is absent, Campaign remains loadable and explains
which function is unavailable. It must not fail startup because Mail, Files, or
Addresses is not installed.
Opening or leaving Template without editing does not change the saved HTML or
mark the page dirty. Visual/source inspection and read-only changes likewise
do not require a save. Actual saves send only client-owned editor metadata. Review
and approval evidence remains server-owned and cannot be overwritten by an
ordinary editor save. Omitting that readable evidence from a save does not
delete it; normal version-lock and invalidation rules remain authoritative.
### Preserve recipient address order
In an individual or global address dialog, use the up/down actions to arrange
the addresses, then choose **Save** in the dialog. The campaign draft keeps that
order; saving no longer alphabetically sorts it. Duplicate email addresses keep
their first position, and pasted addresses append in their entered order.
The first individual To address is also the primary name/email shown in the
recipient row. Use the page's **Save** to persist the campaign draft. A rejected
page save keeps the reordered draft for an explicit retry. **Cancel** in the
dialog discards only its unconfirmed changes.
### Permit Legacy ZipCrypto as an explicit compatibility exception
AES remains the secure default. Campaign **Settings**, **Policies**, and
**Attachments** expose the effective archive policy, configuration links for
authorized administrators, and **Reload archive policy**.
1. A policy administrator opens **Administration → SYSTEM → Campaign archive
encryption**, enables **Legacy ZipCrypto**, and saves. The controls work
before the first system override exists; opening defaults alone does not
create an override or unsaved changes. Changing this global ceiling requires
both `system:settings:write` and `admin:policies:write`; tenant policy
authority alone cannot loosen it.
2. Check tenant and owner policy restrictions. Lower scopes may narrow, never
loosen, inherited methods and password-delivery channels.
3. The Campaign actor also needs `campaigns:archive:use_legacy_zipcrypto` and
edit access to the selected version. Policy administration does not replace
that dedicated permission.
4. Return to Campaign, reload archive policy, and select **Legacy ZipCrypto**
under **Attachments → ZIP attachments**. Acknowledge weak encryption, enter
an operational reason of at least 10 characters, and select an allowed
separate password-delivery channel.
5. Save, validate, build, and review. Policy/configuration saves never send
mail; delivery remains a separate action.
Legacy remains blocked while Policy is unavailable. Neither an encryption
error nor an incompatible client causes automatic fallback from AES to
ZipCrypto. The build retains policy and acknowledgement evidence but never the
password; see the manifest topic `campaigns.archive-encryption-governance`.
Mail migration and ZIP corrections can be saved in either order. A Mail-only
migration preserves unchanged ZIP settings without granting permission to use
them or adding acknowledgement evidence. An archive correction with unchanged
Mail references preserves legacy transport server-side until its separate,
explicit migration. Changes to ZIP settings still require the current policy
and any dedicated legacy permission; changes to Mail references still require
authorized migration. Validate, build and review again after both repairs.
A save and the following workspace refresh are separate operations. A failed
refresh does not undo a committed save or clear the last usable workspace.
Keep any newer unsaved edits, inspect the refresh error, and use Reload to fetch
the current state. Responses for an earlier campaign, version or signed-in
identity cannot overwrite the current workspace.
### Review and complete review
The reviewer should verify the immutable candidate that will be delivered, not
just the authoring form:
If a legacy Mail migration notice appears, follow **Open Mail settings** for
that exact version, complete migration, and validate and build again. Review
stays read-only until migration is resolved and does not repeatedly request
an attachment preview that the legacy transport boundary must reject.
1. Confirm purpose, owner, selected version, and recipient count.
2. Inspect blocking errors, warnings, exclusions, and recipients requiring
review.
3. Inspect representative and exceptional rendered messages, including From,
To/CC/BCC, Reply-To, subject, body, and attachment evidence.
4. Confirm the selected Mail profile is authorized for the campaign's current
tenant and owner context.
5. Confirm attachment behavior when a rule matches no files, ZIP/password
behavior, and any recipient-specific files.
6. Record review completion and the inspected message keys through the review
surface. Validation, build, review, and exception evidence records the actor,
timestamp, and immutable build token/message digest where applicable. If
content, recipients, attachment inputs, owner context, or non-secret
transport identity changes, revalidate and rebuild.
The Review & Send surface separates three kinds of attention. Critical
blockers must be corrected before delivery, individual review items require a
recorded message decision, and non-critical group items may be acknowledged
together after individual review is complete. Each warning or blocker names
the required action, the responsible role, and the workspace to open. The
review summary keeps reviewed and remaining counts visible; a completed review
acknowledges the group items and remains bound to the current build token.
Save each individual acceptance to persist its reason and reviewed state before
completing the entire review. Wait for acknowledgement; reloading then resumes
that build's saved progress. If saving fails or conflicts, the pending note
remains available for an explicit retry rather than becoming a false success.
This small save only loads the selected persisted jobs: it does not rebuild
messages, materialize attachments, or reload the whole workspace. Another
reviewer's existing decisions and attribution remain intact. Partial progress
does not enable delivery; final completion still checks the complete build.
Use **Accept similar review conditions** to record the same decision for a
counted selection of currently loaded matching messages. The server defines
eligible categories from the complete combination of overridable conditions;
the UI does not interpret a warning badge as permission to override. Select
one category, inspect the listed recipients, deselect any exceptions and enter
a common reason (required for attachment exceptions). A submission contains
at most 200 explicit message IDs. When more remain, save this selection and
reopen the dialog; the counts never imply acceptance of unloaded messages or
other categories. The reason is recorded separately against each selected
message's frozen evidence. A failed save retains the selection and reason for
an explicit retry, while a changed build prevents stale acceptance. This
action neither sends messages nor completes the final review gate. Hard
blockers cannot be accepted this way. Deliberate policy exclusions and
attachment rules that explicitly permit zero matches remain informational
and do not require review decisions.
An optional rule with explicit `missing_behavior: continue` may yield no files
without creating review work; that outcome remains informational evidence.
Required attachment and hard-block policies cannot be weakened by this setting.
The separate policy for sending a wholly attachment-free message still applies.
Rebuild existing messages after changing attachment policy; historical build
evidence is not rewritten.
The incremental review API uses `merge_progress: true`, the acknowledged
`base_revision`, and `build_token` set to the public `review_build_token`.
It merges exact reviewed keys/decision job IDs for the current build, with an
optional `decision_category_key` to bind a grouped acceptance. The safe review
reference is available without diagnostic access; raw build tokens remain
diagnostic data. Stale build/revision or simultaneous writes return HTTP 409
without overwriting progress. Normal review authorization and audit apply.
Accepted or expected attachment conditions remain satisfied in **Confirm and
send** for the same build. Raw missing/ambiguous source counts remain visible
for context; they are not a second approval gate. Reviewed-stage mock delivery
uses `use_reviewed_build: true`: it verifies the existing execution seal,
completed review, frozen job issues and EML integrity, and current Mail transport
before capturing anything in the mock mailbox. It uses those stored messages,
not freshly rendered replacements, and never mutates Campaign delivery state.
Stale review, changed inputs, changed bytes or changed transport stop the test
before captures or requested mailbox clearing. The authoring/mock-preview API
keeps its existing transient-build default; `include_needs_review` does not
bypass frozen review checks.
Validation details and repeated-file lists use the shared DataGrid pagination
controls so every item is reachable. Related missing-rule causes and their
attachment-free policy outcomes appear together, with the technical evidence
still expandable. Built messages have four operational states: **Ready**,
**Needs review**, **Blocked**, and **Excluded**, plus an explanatory column.
Accepted explicit decisions are Ready; warnings awaiting acknowledgment remain
Needs review. This presentation does not remove or rewrite frozen issues or
audit evidence.
This evidence is the Campaign input to separation-of-duties policy. Generic
approve/reject chains, delegation, substitutions, escalation, and signatures
belong to the optional Approvals capability. Campaign must not claim an
approval merely because validation, building, or message review completed.
When a campaign has an Approval request reference, mock and real delivery
resolve `approvals.requests` and require an approved request for the exact
`campaign_version` subject and current version id. A missing Approvals module,
unknown request, pending/rejected chain, or approval for an older version blocks
delivery with an explicit requirement. Campaign stores the approval reference,
not Approval tables; changing the campaign version requires a new exact-subject
approval.
Normal readers and reviewers see business state and safe evidence. Process-local
paths, storage keys, worker claim tokens, and raw provider diagnostics require
the dedicated diagnostic permission and must not leak through ordinary campaign,
version, job, or report responses.
Campaign now provides a separate aggregate **Reports** surface for readers with
`campaigns:report:read` and access to the campaign. It loads only the safe
aggregate projections, applies small-cell suppression, and offers no recipient
rows, drill-down, filtering, export, or delivery actions. The recipient-aware
**Campaign Report** still requires recipient-read access and does not yet hide
every action control that the actor lacks. The server authorizes each action,
but permission-aware action visibility on that detailed surface remains open
work; do not confuse it with the aggregate reader experience.
The module-local aggregate surface remains at `/campaigns/reports`. When the
optional Reporting module is enabled, Campaign also contributes the same
recipient-free projection as the `campaigns/delivery-outcomes` report provider.
Reporting owns `/reports`, records the run purpose, effective audience, source
campaign/version revision, privacy transformations, retention, actor/time,
output hash, and export history, and applies Policy before returning the
result. Campaign does not register a fallback `/reports` route when Reporting
is absent.
The detailed Campaign Report includes the selected campaign's effective
retention policy, its system/tenant/owner/campaign provenance, and the current
evidence state. It distinguishes retained, redacted, expired, partially
minimized, unavailable, and not-applicable source JSON, stored report detail,
generated EML, and Postbox-copy evidence. When Policy is absent, the report
shows the platform defaults and explicitly warns that automated retention
enforcement is unavailable. Retention removes or minimizes detail; aggregate
counters and audit references may remain so outcomes can still be explained.
### Deliver and resolve outcomes
Use the [delivery runbook](CAMPAIGN_DELIVERY_RUNBOOK.md) for the detailed
operator sequence.
At a minimum:
1. Queue only a validated, locked, built version. Use **Queue for workers** for
ordinary batches; the durable progress remains visible after leaving and
returning to Review and send.
2. Use **Send now** only when the exact persisted eligible build is within the
effective synchronous limit shown by the UI. The unchanged default limit
is 25 recipient jobs. The backend repeats the count and preflights every
message and the Mail profile revision before contacting SMTP.
3. Treat `smtp_accepted` as protected from ordinary retry.
4. Retry `failed_temporary` explicitly after inspecting the cause.
5. Include `failed_permanent` only after correcting the cause and making a
conscious override.
6. Never retry `outcome_unknown` blindly. Inspect SMTP/provider evidence and
reconcile it as **accepted** or **not sent**, with a note identifying the
evidence.
7. Process append-to-Sent only for SMTP-accepted jobs and investigate append
failure independently. Never retry an `outcome_unknown` IMAP append blindly:
reconcile mailbox evidence as **appended** or **not appended** with a note.
Only the latter becomes explicitly retryable, and neither decision resends
the already SMTP-accepted message.
8. Archive only after active and uncertain effects are resolved.
9. Use **Delete draft** only for an untouched draft. If retained evidence or an
active share exists, revoke the share where appropriate or archive instead.
Pause stops new eligible work but cannot undo a provider effect already in
progress. Cancel marks work that has not yet produced a protected SMTP outcome;
it cannot recall accepted mail.
Configure **Administration → SYSTEM → Campaign delivery** with
`system:settings:read/write`. The default stays 25, but an administrator may
explicitly choose 0500, for example 200 for a 183-recipient-job run. Zero disables
Send now. **TENANT → Campaign delivery** uses `admin:policies:read/write` and may
only narrow the inherited system policy. Clearing an override restores
inheritance. An explicitly configured deployment ceiling
`GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS` remains authoritative; the
implicit default does not prevent a system administrator choosing a larger
bounded value. Save changes only this setting, preserves unrelated settings,
checks a revision token including inherited policy, and records before/after
configuration history and audit. Failed saves retain the draft; conflicting
saves require explicit reload/reconciliation. Policy edits never send mail or
change existing reviews or approval requirements.
This limit applies to one interactive Send now request, not campaign size or
worker batching. Larger interactive requests run longer and can meet proxy
timeouts. Queue for workers is independent and requires enabled, healthy
Redis/Celery infrastructure; changing the numeric limit does not start workers.
The effective value and source are returned by the protected delivery-options
API, recorded for successful/rejected synchronous commands, and stated in the
configured handbook topic.
### Test and one-message actions
The current baseline includes mock send, queue dry-run, synchronous immediate
delivery, sending one selected job, retry selection, and unattempted-job
selection. Their audit and state behavior is not yet the final vocabulary
accepted for issue `govoplan-campaign#69`.
The intended distinction is:
- **Test:** deliver once to the configured test path, audit it, and leave the
production job unsent.
- **Single send:** send one unsent job, audit it, and mark its production effect
complete.
- **Single resend:** intentionally send one job again regardless of an earlier
failure or acceptance, with distinct authority, warning, reason, and audit.
Until that slice is implemented and target-tested, do not present the existing
buttons as a complete resend policy. Ordinary retry protection remains the safe
default.
## Data and evidence model
### Portable Campaign transfer
Campaign offers two reuse paths with different boundaries. **Copy campaign**
creates another campaign inside the same installation and can reuse selected
local shares, policies, and Mail profile references. **Export package** creates
a versioned JSON hand-off whose selected scopes can cross an installation
boundary; **Import package** always creates a separately owned draft.
The export dialog starts with only metadata and template/configuration. Add
recipients, attachment rules, review state, or delivery history only when the
handoff requires them and the destination and retention are approved. Recipient
and delivery scopes remain protected by recipient/report export permissions.
Transport secrets, credential references, password-field values, local storage
locators, and attachment bytes are always removed. The manifest records scope
counts and redactions, while the envelope carries source Campaign/version
provenance and a SHA-256 digest.
Import verifies format, scope, checksum, schema, and destination identity before
showing the plan. Editing the destination identity or selected scopes makes the
preview stale and requires a new check. The apply step clears source Mail
references, creates one editable draft, and stores a bounded source/package and
created/skipped receipt. Historical validation/build summaries, review state,
approvals, delivery jobs, attempts, and sent outcomes are never replayed. File
content is never embedded, so reconnect managed files and local Mail profiles,
then validate, build, review, and approve normally.
### Versions and snapshots
Editable campaign JSON is versioned. Build creates recipient jobs and an
execution snapshot. A new profile-only snapshot contains the stable Mail
profile id, delivery policy/evidence, and opaque Mail-issued transport
revisions. It contains no resolved SMTP/IMAP configuration or credentials.
At delivery time Mail re-authorizes the profile, compares the expected opaque
revisions, resolves credentials inside the same Mail-owned operation, and
performs the transport effect. Campaign receives only the sanitized result it
needs for job state and evidence. This same-call check prevents a
validate-then-use race across the module boundary.
Credential rotation that does not change the non-secret transport identity may
continue to satisfy the snapshot. A host, account identity, protocol, sender,
or other revisioned transport change stops delivery until the campaign is
revalidated and rebuilt.
### Existing legacy database records
The current Campaign database may contain versions with inline SMTP/IMAP
material. No separate historical Campaign JSON corpus exists. Inline transport
material in those rows is treated as inert legacy data and is never interpreted
as an executable Mail configuration:
- public responses remove legacy transport fields and secrets;
- validation, build, queue, retry, and delivery fail closed;
- computed previews that require a current Campaign configuration return an
actionable validation problem instead of a server error;
- an editable version changes to profile-only form only through an explicit
Mail-settings save; and
- a locked version is preserved and must be forked to an editable successor.
Normal database backup/restore and access controls cover those rows together
with the rest of the current database. There is no separate historical-JSON,
backup-scanning, or inline-secret migration program. Restoring an existing
legacy row preserves it as inert evidence and does not make it deliverable.
### Recipient and attachment evidence
The delivery record should be able to identify:
- source/import context and the frozen recipient row;
- effective addressing and message id;
- template inputs and unresolved-placeholder decisions;
- managed file/version ids, source provenance, checksums, and ZIP evidence;
- generated EML checksum and size;
- Mail profile reference and opaque transport revisions;
- each SMTP and IMAP attempt, its classification, safe provider response, and
reconciliation note; and
- actor/system trigger, timestamps, policy context, and corrections.
An excluded recipient/message is a completed validation decision, not a
pending delivery. Its SMTP and IMAP states are both `skipped`; it is counted and
filterable separately from unattempted, failed, accepted, and append outcomes.
No SMTP or IMAP attempt exists for such a row. If historical data contains
actual transport evidence despite an exclusion marker, that evidence is
preserved for audit and reconciliation rather than relabelled.
## Administration and policy
### Roles and permissions
The supplied role templates deliberately separate preparation, review, and
delivery:
- **Campaign manager** prepares, validates, and builds campaigns and recipients.
- **Campaign reviewer** inspects prepared material and records review
completion for the exact messages checked.
- **Campaign sender** queues, sends, pauses/resumes/cancels, retries, reconciles,
reads reports, and has diagnostic access.
Administrators may compose narrower roles from the declared permissions. Keep
these separations where institutional policy requires four-eyes approval. Mail
profile use additionally requires `mail:profile:use`; profile and credential
administration remains a Mail permission.
Campaign access is also constrained by tenant, owner/group context, and explicit
shares. A share grants only its declared campaign permission; it does not grant
Mail credentials, Files administration, or tenant-wide recipient access.
### Configuration checklist
- Install compatible Core and Campaign versions. Access and base audit are Core
infrastructure; install optional Mail, Files, Addresses, Notifications, and
Policy modules only where the configured journey requires them.
- Configure Mail profiles and policy in Mail, not in campaign fixtures or JSON.
- Configure Files storage/connectors and attachment permissions in Files.
- Define role assignments and separation-of-duty policy.
- Configure Redis/Celery for durable batch workers where production volume
requires it. Local execution is a development/small-run mode, not horizontal
worker coordination.
- Set rate limits for the target provider. Mail may use Redis for shared
throttling when present and a process-local fallback in development.
- Establish retention, deletion, archive, report-export, and diagnostic-access
policy before production recipient data is loaded.
- Use non-production SMTP/IMAP identities and the maintained examples before
allowing a production profile.
### Owner transfer
Mail profile visibility can depend on campaign owner or group. Transferring an
editable campaign with a selected profile clears/requires reselection and
revalidation. A locked or delivery-final version is never silently rewritten;
create an editable successor.
## Operations and recovery
### Interactive delivery and Sent-folder progress
Send now and workerless Report retry/continue use a compact blocking progress
dialog, as does inline append-to-Sent. It refreshes saved counters only, not the
whole campaign behind the overlay. Successful, pending, in-progress, failed,
uncertain and excluded messages are shown separately; a currently sending or
appending message therefore does not disappear between totals. Read errors keep
the last known counters. After a connection interruption, processing may still
be running; inspect saved evidence before repeating any action. A successful
write is not reclassified as failed when its later display refresh fails.
The recipient-aware Report shows all frozen To, Cc and Bcc addresses, not only
the primary row identity. Address order and recipient-read authorization remain
unchanged. SMTP/IMAP diagnostics use translated status-list filters.
Without workers, explicitly retry eligible failures or continue unattempted
jobs through Report. Each request uses the canonical job/attempt recovery
boundary and is limited by the effective synchronous policy. Accepted and
uncertain SMTP outcomes remain protected. Active abandoned claims require an
expired durable lease, proven stopped/replaced owner, current revision and
valid original evidence before recovery can mark them unknown. A separate
evidence-note reconciliation is required before retrying. A timeout alone is
never proof of non-delivery. See the delivery runbook for required permissions
and operational limitations.
Append-to-Sent is scoped to the selected version and reuses a bounded Mail-owned
connection/folder resolution (default 100 messages or 300 seconds), while
performing one sequential APPEND and all current checks per message. Uncertain
appends are never automatically repeated, and repairing Sent never resends SMTP.
Link required files before locking. Lock and validate waits for attachment
matches, rechecks them immediately before locking, and asks for Link and lock
confirmation if new unlinked files are found. A locked version cannot acquire
new attachment links; use an editable version and validate/build/review again.
### Fortschritt, Wiederherstellung und Dateiverknüpfungen
Jetzt senden, synchrone Wiederholung/Fortsetzung im Bericht und Kopieren nach
Gesendet verwenden einen kompakten sperrenden Fortschrittsdialog. Nur gespeicherte
Zähler werden aktualisiert, nicht der Arbeitsbereich im Hintergrund. Erfolgreich,
ausstehend, in Bearbeitung, fehlgeschlagen, ungewiss und ausgeschlossen bleiben
getrennt sichtbar. Bei Lesefehlern bleiben die letzten Werte erhalten. Nach einer
getrennten Verbindung kann die Verarbeitung weiterlaufen; prüfen Sie Nachweise,
bevor Sie erneut handeln. Ein bestätigter Versand wird durch einen nachfolgenden
Anzeigefehler nicht nachträglich als fehlgeschlagen dargestellt.
Der empfängerbezogene Bericht zeigt alle eingefrorenen An-, Cc- und Bcc-Adressen
in ihrer Reihenfolge. Leseberechtigungen bleiben unverändert. SMTP und IMAP
verwenden übersetzte Zustandslisten zum Filtern.
Ohne Worker können bekannte Fehler ausdrücklich wiederholt und unversuchte
Aufträge begrenzt fortgesetzt werden. Die wirksame synchrone Grenze, gespeicherte
Aufträge, Prüfungen, Freigaben und Wiederherstellungsnachweise bleiben verbindlich.
Angenommene und ungewisse SMTP-Ergebnisse werden nicht blind wiederholt. Die
Wiederherstellung aktiver, verlassener Aufträge benötigt eine abgelaufene Sperre,
nachweislich gestoppte/ersetzte Laufzeit und gültige ursprüngliche Nachweise. Sie
setzt ausschließlich auf ungewiss; vor Wiederholung sind externe Nachweise und
ein getrennter Abgleich mit Notiz erforderlich. Zeitablauf allein genügt nicht.
Kopieren nach Gesendet betrifft nur die ausgewählte Version. Mail verwendet die
Verbindung und Ordnerauflösung begrenzt wieder (Standard: 100 Nachrichten oder
300 Sekunden), prüft aber jede Nachricht erneut und führt APPEND nacheinander
aus. Ungewisse Ergebnisse werden nicht automatisch wiederholt. Verknüpfen Sie
benötigte Dateien vor dem Sperren; eine frische Prüfung fragt bei unverknüpften
Treffern nach Verknüpfen und sperren. Gesperrte Versionen benötigen zum Ändern
eine bearbeitbare Version mit erneuter Validierung, Build und Prüfung.
### Health to observe
- database and migration health;
- compatible module interface versions;
- queue publication, worker heartbeat, claim age, and backlog;
- SMTP/IMAP profile test result and deployment egress policy;
- Mail profile authorization/transport-revision mismatch;
- Files resolution and frozen attachment availability;
- counts by queue, SMTP, IMAP, and reconciliation state; and
- audit and report generation failures.
### Worker interruption
A crashed worker can leave a job claimed or sending. Do not infer non-delivery
from worker loss. If the SMTP boundary may have been crossed, classify the
outcome as unknown and reconcile from external evidence. Only work that is
provably unattempted may be returned to an ordinary queue.
### Retry and reconciliation
Retries create new attempt evidence; they do not overwrite the previous
attempt. Reconciliation is a privileged factual correction supported by an
operator note. Repeated automated requests should be idempotent for the same
eligible job state; a deliberate resend is a different, not-yet-finalized
business action and must never be disguised as a retry.
### Backups and restoration
Back up Campaign, Mail, Files, Core/Audit, and shared storage consistently for
the composition. A database restore without generated EML/file storage, or file
storage without matching metadata, does not reconstruct the evidence chain.
After restore, keep outbound delivery paused until queue/attempt state and
provider evidence have been reconciled; never let restored accepted jobs send
again merely because a queue message was lost.
For unreferenced generated objects after process loss, platform operators first
run the bounded Campaign artifact inventory in dry-run mode. Apply only an
inspected page with a unique incident idempotency key. The cleanup keeps exact
keys out of ordinary Campaign responses, does not clear database references,
and leaves storage failures in the Core recovery ledger for explicit retry.
### Incident handling
1. Pause new delivery when duplicate or unknown effects are possible.
2. Preserve database, queue, provider, worker, and audit evidence.
3. Scope affected campaigns/jobs without exposing recipient content broadly.
4. Reconcile uncertain jobs individually or through an approved bounded tool.
5. Correct configuration in its owning module, then revalidate/rebuild where
revisioned transport inputs changed.
6. Record the incident reference and recovery rationale in audit evidence.
## Composition and integration contracts
Campaign has one required platform dependency: Core. Optional module behavior
is discovered through versioned, Core-mediated capabilities; Campaign must not
import optional sibling ORM, services, or WebUI implementation.
Current principal contracts include:
| Contract | Direction | Purpose |
| --- | --- | --- |
| `mail.campaign_delivery` 0.2.x | Mail -> Campaign | Summarize a known reference; authorize and revision-gate it; send/append using Mail-owned configuration and credentials; return sanitized results |
| `files.campaign_attachments` 0.1.x | Files -> Campaign | Select/materialize governed file versions and preserve campaign usage/evidence |
| `addresses.lookup` 0.1.x | Addresses -> Campaign | Optional address suggestions |
| `addresses.recipient_source` 0.1.x | Addresses -> Campaign | Optional versioned recipient-source snapshots |
| `dist_lists.source` / `dist_lists.expand` 0.1.x | Distribution Lists -> Campaign | Discover, preview, and freeze reusable audiences without importing module internals |
| `templates.catalog` / `templates.renderer` 0.1.x | Templates -> Campaign | Select compatible published printable templates and produce deterministic, evidence-bearing artifacts |
| `campaigns.access` 0.1.x | Campaign -> platform | Explain campaign access/existence without exporting ORM objects |
| `campaigns.mail_policy_context` 0.1.x | Campaign -> Mail | Resolve campaign tenant/owner context for Mail policy |
| `campaigns.delivery_tasks` 0.1.x | Campaign -> workers | Execute narrow queued send/append tasks |
| `campaigns.retention` 0.1.x | Campaign -> retention | Apply Campaign-owned retention behavior |
Breaking payload or ownership changes require an interface-version bump and a
release-composition alignment gate. Optional absence must be tested physically,
not only hidden in navigation.
### Bulk recipient activation
Recipient data can activate all currently inactive rows or deactivate all
currently active rows. The action displays the exact affected count and
requires confirmation. It changes only the local Campaign draft until the
operator saves. Saving follows the ordinary versioned Campaign update path, so
the resulting recipient state is retained in version and protocol evidence and
any stale validation, build, or review evidence is invalidated.
### Reusable Distribution Lists
When Distribution Lists is available, Recipient data offers a separate import
dialog. The author selects a visible list revision, supplies declared
parameters, requests candidate channels, and previews included, excluded,
stale, ambiguous, suppressed, policy-blocked, and provider-unavailable results.
The final action freezes an idempotent Distribution Lists snapshot and copies
the resulting rows into the editable Campaign version.
Each copied row retains the list and revision IDs, definition and expansion
hashes, snapshot ID, source entry IDs, provider references, channel candidates,
the explicitly selected primary route, optional fallback, and the decision
explanation. Campaign-only fields, attachment rules, review state, and outcomes
remain local to Campaign and never mutate the reusable list.
A later list revision only raises a drift warning. Refresh is deliberate and
uses append or replace; saving that changed Campaign version clears prior
validation, build, review, and execution state through the normal content
invalidation path. Preferred or single usable candidates are preselected
visibly; ambiguous rows must be decided before freezing. Postal and
internal-mail routes remain active when a compatible published Templates output
is selected.
### Governed hybrid and printable delivery
Campaign supports Mail, Postbox, printable output, and bounded ordered
fallbacks without making any of those provider modules mandatory. Opt-in and
channel-preference data are inputs to the visible routing decision; they never
silently cause duplicate delivery.
For a printable route, select a published label, envelope, serial-letter,
form-letter, list-layout, or generic template on the Template page. Validation
checks the selected revision, output format, and required fields. Build sends
one deterministic item collection to `templates.renderer`, records template,
input, output, actor, route, and artifact hashes, and stores the resulting
artifact through Files when configured. The review stage exposes that exact
artifact and its hashes before execution.
Each recipient job records an idempotent print acceptance attempt for its item
in the frozen artifact. `mail_then_print` and `postbox_then_print` invoke print
only after a confirmed rejection before acceptance. An accepted or
outcome-unknown digital effect never falls through to print because that could
produce duplicate delivery. Reports and CSV exports include route provenance,
print state, attempts, artifact reference, and hashes.
Without Templates, Campaign still loads and Mail/Postbox authoring remains
available; validation explains why a configured print route cannot proceed.
Without Files, Templates may return a bounded artifact instead of a managed
file. Campaign copies that payload into shared object storage and exposes it
through the Campaign ACL plus `campaigns:recipient:read`; it never redistributes
the broader Templates URL. A print-only Campaign does not require Mail or Postbox.
### External API expectations
- Tenant and campaign access are evaluated for every operation.
- Writes require CSRF/auth behavior supplied by Core and the specific declared
scope.
- Queue/retry/reconcile endpoints operate on persisted state and return safe
summaries; initiating HTTP success is not proof of external delivery.
- Delta endpoints are optimization surfaces, not a separate source of truth.
- Report exports contain permitted business/evidence fields but no credentials,
local paths, storage keys, or worker claims.
## Assurance model
### Security invariants
- No new campaign payload, fixture, response, or execution snapshot contains
SMTP/IMAP settings or credentials.
- Mail resolves credentials and performs transports inside Mail-owned calls.
- Every real connector peer is validated and pinned at connection time under
deployment-wide private-network policy.
- API/server attachment paths use managed Files references; arbitrary and
traversal-capable local paths are rejected.
- Public responses recursively remove infrastructure locators and secret-like
legacy fields.
- Accepted and outcome-unknown jobs are protected from ordinary retry.
- Diagnostic permission is separate from campaign read/report access.
### Privacy
Recipient fields and rendered messages may contain personal or sensitive data.
Grant recipient read/export, report export, and diagnostic access separately.
Prefer aggregate status for readers who do not need recipient detail. Define
purpose, lawful basis, minimization, export control, and retention before the
campaign starts; do not use Campaign as a substitute consent or address-master
system.
The Core data-subject-request workflow discovers Campaign through the optional
`privacy.dsar.campaigns` capability. After the request's email, membership, and
namespaced Campaign references have been independently authorized and
corroborated, the provider searches only the effective tenant and isolates the
matching recipient entries and jobs. Its JSON result includes safe Campaign,
version, delivery-attempt, schedule, report-projection, share, import-mapping,
attachment, generated-artifact, and relevant collaboration metadata. It also
finds collaboration entries authored, mentioned, or moderated by the subject.
Text authored by the subject is included; somebody else's text is not copied
merely because the subject was mentioned. Generated EML bytes and paths,
storage locators, delivery target snapshots, worker claims, idempotency
material, credentials, secret-like values, and unrelated recipients are never
embedded in that result. Authorized Campaign and Files review surfaces remain
the source for content that cannot safely be copied into the DSAR case.
Built, locked, published, terminal, delivered, or corrected records are
retained with an explicit reason and continue through Campaign's configured
retention and redaction process. Draft recipient content and user-owned
attachment content require coordinated manual review because copies may span
version JSON, jobs, generated messages, and managed files. The provider can
idempotently revoke an active share aimed at the subject and delete the
subject's personal recipient-import mapping profile. It does not rewrite
delivery evidence, delete generated artifacts, or report derived Campaign
counts as a separate store. Collaboration withdrawal and redaction retain the
tombstone, content hash, context, and Audit evidence; the DSAR workflow does
not rewrite these append-only records. Re-running an approved action is safe: already
revoked or absent data is reported as unchanged, and tenant, subject, and row
ownership are revalidated immediately before mutation.
### Audit and destructive actions
Material authoring, validation, locking, review, queueing, send, retry,
reconciliation, sharing, owner transfer, archive, and permitted deletion actions
emit attributable evidence. Audit details must be non-secret and should refer to
stable ids rather than repeat message bodies or credentials.
Draft deletion is allowed only while no audit-relevant build, delivery job, or
lock exists. Evidence-bearing campaigns are archived. Destructive module
retirement remains a separately confirmed installer operation with backup and
retirement evidence.
The Campaign **Audit** page is currently an explained handoff, not a second
audit store: Campaign emits bounded platform audit records and authorized
readers inspect them in Administration > Tenant audit. A future object-scoped
projection may improve that navigation without duplicating Audit ownership.
Evidence-bundle export and offline verification remain owned by Audit #3.
The advanced **JSON** page displays and downloads the complete campaign
configuration available to the current campaign reader. It contains no inline
transport secrets, but recipient, message, and attachment fields may contain
personal data. The UI therefore identifies it as sensitive expert output;
campaign access and export purpose remain the governing controls.
## Reference-composition acceptance
Campaign is ready to serve as the demonstration module only when all of the
following are repeatable in a pinned clean installation:
1. The maintained examples validate and build with Mail/Files present, and
Campaign still starts with each optional module absent.
2. A user can import recipients, select managed files, choose an authorized Mail
profile, validate, review, build, and queue without entering transport ids or
secrets manually.
3. Campaign JSON and all ordinary APIs reject/omit inline transport material;
legacy records are visible as migration-required and cannot execute.
4. SMTP success, temporary/permanent failure, connection loss, worker loss,
outcome unknown, retry, reconciliation, IMAP success, and IMAP failure have
tested, non-duplicating outcomes against the target environment.
5. Author, reviewer, sender/operator, reader, and administrator views use the
central component system and expose only task-relevant actions.
6. Reports and audit can reconstruct recipient/message/file/profile/attempt
evidence without exposing secrets or ordinary-reader infrastructure details.
7. Clean install, upgrade, backup/restore, module permutations, version
alignment, security audit, and target SMTP/IMAP tests pass.
8. This handbook and its adaptive Docs topics match the shipped UI wording and
distinguish implemented behavior from planned work.
## Explicitly planned, not yet claimed
The following are part of the selected reference journey but are not implied by
the current baseline:
- the final audited **test / single send / single resend** semantics;
- durable, idempotent Campaign report delivery through a Mail-owned outbox
([`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17));
- a fully packaged one-command Campaign reference composition with production
policy presets and target-provider certification;
- function-bound Postbox delivery (stage 2 of the reference program);
- generic workflow-driven campaign transitions.
Each item needs an owning issue, implementation, failure tests, documentation,
and release evidence before the wording above can move from planned to current.
@@ -0,0 +1,103 @@
# Example Campaigns And Release Checklist
This document defines the campaign examples and release gates that should be
kept working before a GovOPlaN Campaign release is tagged.
## Example Campaign Set
Maintain fixtures or guided examples for these scenarios. The canonical
scenario catalogue lives in `examples/README.md`; committed fixture files should
be added under `examples/` only when they validate against the current campaign
schema and are safe to run in non-production environments.
- [`simple-announcement`](../examples/simple-announcement/campaign.json), a
credential-free campaign with one active recipient and no attachments; its
automated acceptance check physically blocks Mail and Files imports, denies
network connections, and validates/builds from an unrelated temporary
workspace
- multi-recipient message with To, CC, BCC, Reply-To, bounce, and disposition
notification fields
- campaign with global attachments and recipient-specific attachment rules
- campaign with password-protected ZIP attachments
- campaign using a reusable mail profile from the mail module
- legacy inline SMTP/IMAP campaign rejected with an actionable profile-migration
error and no returned transport data
- campaign with validation warnings that may be sent only after explicit review
- campaign with blocked recipients or attachment errors that must not be sent
- mock delivery campaign that captures SMTP and IMAP append messages in the mail
development mailbox
- [`greenmail-delivery`](../examples/greenmail-delivery/campaign.json), a
credential-free real-delivery Campaign materialized with a temporary
Mail-owned profile by the loopback acceptance runner
## Fixture Rules
- Examples must not contain production recipient data or production credentials.
- Attachment examples should use deterministic small files and checksums.
- Mail secrets belong only to encrypted Mail profiles and must never appear in
a campaign fixture, placeholder, or campaign-local secret reference.
- Examples that require optional modules must declare the required modules and
capabilities in their README or fixture metadata.
- Examples must stay valid when files or mail modules are physically absent,
with optional behavior disabled instead of import failures.
## Release Gates
Before tagging a campaign release:
- Review `examples/README.md` and update the scenario catalogue when a release
adds or removes delivery behavior.
- Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'` and
retain its isolated validate/build result as release evidence.
- Run core module permutation tests with campaign installed both with and
without files/mail.
- Validate and build each maintained example campaign.
- Run the mock delivery example when the mail development mailbox capability is
enabled.
- Run the GreenMail SMTP/IMAP smoke for a non-production real delivery path.
- Run `dev/mail-testbed/run_campaign_acceptance.py` and retain its bounded JSON
projection. It must show one SMTP acceptance, one IMAP append, no duplicate
effect from a repeated ordinary send, matching Campaign report/audit state,
and no resolved transport material in Campaign JSON or its execution
snapshot. Its controlled endpoint evidence must also show explicit SMTP 451,
partial RCPT refusal, post-DATA ambiguity, and task-process interruption
classifications without retaining addresses or provider diagnostics.
- Treat the task-process restart proof separately from the still-open
Redis/Celery broker redelivery and daemon-supervision check; the coverage
projection must keep `celery_broker_redelivery` false.
- Run `dev/mail-testbed/run_celery_redelivery_acceptance.py` as a separate
destructive worker-loss check. Retain its bounded evidence only when the same
broker task is observed at both workers, the durable state is
`outcome_unknown`, broker queue/unacked counts are zero, and the controlled
SMTP endpoint observed one connection and one DATA transaction. This closes
local Redis/Celery redelivery coverage, while production supervisor and
target-provider coverage remain false until separately tested.
- Confirm reusable mail profile selection is revalidated after campaign owner
transfer.
- Confirm every inline SMTP/IMAP field is rejected on import/write, omitted
from responses, and blocked from legacy execution until explicitly migrated.
- Confirm execution snapshots store only the Mail profile reference and
opaque Mail-owned transport revisions, not resolved transport material.
- Confirm delivery reports include SMTP outcome, IMAP append outcome, latest
error, generated EML reference, and attachment evidence.
- Confirm retries cannot resend messages already accepted by SMTP unless an
explicit reconciliation path allows it.
## Ownership Transfer Check
Reusable user/group mail profiles are owner-context-sensitive. When campaign
ownership changes, the editable current version must require profile reselection
and validation before live delivery. A locked delivery-final version should not
be silently rewritten.
## Delivery Checklist
Use the Review & Send preflight panel and the delivery runbook together:
1. Validate and resolve all blocking policy/data issues.
2. Build exact messages.
3. Review warnings, generated recipients, body content, and attachment evidence.
4. Run mock delivery if available for the release channel.
5. Test the selected Mail profile against non-production infrastructure.
6. Send only after queue, rate limit, and append-to-Sent behavior are understood.
7. Reconcile failed, unknown, or pending jobs from the report/audit surfaces.
+152
View File
@@ -0,0 +1,152 @@
# Campaign and Mail Profile Boundary
## Product view
A campaign chooses an authorized Mail profile. It does not define a mail
server. The Campaign module owns recipients, content, attachment rules,
delivery intent, review state, and delivery evidence. The Mail module owns the
SMTP/IMAP endpoints, encrypted credentials, connection tests, profile policy,
and runtime transport adapters.
The persisted campaign contract is therefore deliberately narrow:
```json
{
"server": {
"mail_profile_id": "stable-mail-profile-id"
}
}
```
No `server.smtp`, `server.imap`, `server.credentials`, credential-inheritance
override, or password is valid campaign JSON.
## User journey
1. A Mail administrator creates and tests a reusable profile in Mail.
2. A campaign author opens **Mail settings** and selects one profile available
for the campaign's tenant, owner, and policy context.
3. Campaign stores only the stable profile identifier.
4. Validation asks Mail to authorize the active profile and returns only
availability flags plus opaque Mail-owned transport revisions. Reading that
summary does not decrypt credentials.
5. Build stores the profile identifier, delivery policy, job manifest, and
non-secret transport revisions as execution evidence. It stores no
resolved host, username, password, or other transport material.
6. A delivery worker invokes one Mail-owned effect operation. Mail re-authorizes
the profile, resolves credentials, compares the expected transport
revision, checks policy, and sends or appends without returning transport
material to Campaign. If the selected profile or its non-secret transport settings
changed after build, delivery stops until the campaign is revalidated and
rebuilt. A password rotation that leaves the transport identity unchanged
does not invalidate the build.
7. Mail returns only Campaign-owned envelope addresses, counts, sanitized
refusal classifications/status codes, or the selected Sent folder. Raw
server banners, provider bytes, host details, and credentials never enter
Campaign attempts or public evidence.
Non-dry Campaign report email currently fails closed. It must not bypass the
durable job/effect model through a direct SMTP call. Re-enabling it requires the
Mail-owned idempotent outbox, attempt, unknown-outcome, and reconciliation path
tracked in
[`govoplan-mail#17`](https://git.add-ideas.de/GovOPlaN/govoplan-mail/issues/17).
Report generation and dry-run validation remain separate from an external
effect; recipient-level exports require recipient-export authorization.
Campaign authors need `mail:profile:use` in addition to the relevant Campaign
permission. Profile visibility remains governed by Mail policy and campaign
owner context.
## Existing database rows and migration
The current database can contain campaign versions with inline SMTP/IMAP
settings or credentials. There is no separate historical Campaign JSON corpus
to import or remediate. GovOPlaN treats those inline fields as inert legacy
material and does not delete or rewrite the stored audit rows automatically:
- API responses omit all legacy transport fields and secrets and expose a
`mail_profile_migration_required` marker.
- validation, build, queue, retry, and delivery fail closed with an actionable
profile-migration error;
- unrelated edits preserve the exact stored legacy server object when the
submitted public Mail references are unchanged; they cannot silently scrub,
edit, or re-submit legacy fields or credentials;
- an editable version is migrated only through an explicit Mail-settings save
with an authorized profile; and
- a locked version remains unchanged. Creating its editable successor records
the migration while retaining the locked source as audit evidence.
Legacy execution snapshots are likewise retained but cannot be used for
delivery. Revalidate and rebuild an editable profile-only version. Normal
database backup, restore, encryption, and access controls apply to the current
database as a whole; the product does not define a separate historical-JSON or
inline-secret recovery workflow. A restored legacy row remains inert and
fail-closed under the same rules.
### Migrate from the Campaign UI
Open **Mail settings** from the migration notice for the selected version.
Select an authorized Mail profile and choose **Migrate selected Mail profile**.
The migration action is available even when that profile was already selected
and the draft has no other unsaved changes. If the version is locked, first use
its supported unlock or editable-successor action; protected source evidence is
not rewritten. Reopen the settings and confirm that the migration notice has
gone, then validate, build, and review before separately authorizing delivery.
Migration itself never sends mail.
Mail migration and ZIP policy repairs can be saved in either order. An exact,
unchanged ZIP configuration does not require a new acknowledgement merely to
save a Mail migration, even if the existing ZIP policy or actor's permission is
no longer valid. No actor, timestamp, or consent is invented. Any ZIP change
still requires the complete current archive policy and, for ZipCrypto, the
dedicated permission and reasoned acknowledgement. Conversely, saving an
archive correction with unchanged public Mail references retains the exact
legacy transport server-side, without using or reauthorizing the old profile.
The migration notice remains until the explicit authorized migration succeeds.
Changing any Mail profile, server, or credential reference is not an unrelated
repair. Inline transport is rejected even if a caller echoes stored values.
The same edit-time separation applies after migration: retaining an unchanged
Mail selection does not invoke use-policy while saving an unrelated correction,
even if a later policy requires an explicit credential. Selecting a new resource
or explicitly migrating still checks Mail permission and current policy; actual
validation and delivery always recheck them, regardless of save history.
Both repair orders invalidate build/execution evidence; validation, build,
review and delivery remain blocked until all outstanding conditions are valid.
Successful saves and subsequent refreshes are separate outcomes. A committed
save is not undone by a failed workspace refresh. The workspace retains its
last usable same-campaign/version data and displays the refresh error; retry
Reload to fetch the current server state. Obsolete responses from an earlier
campaign, version, signed-in identity, or reload cannot replace newer data.
The normal profile selector requests only campaign-authorized profiles. The
administrative profile catalogue is requested separately on **Mail policy**;
failure or lack of authority there does not empty the usable profile selector.
Profile-list errors remain visible next to the affected settings.
While migration is required, **Review and send** remains read-only and links to
the exact version's Mail settings. It does not repeatedly attempt incompatible
attachment-preview requests. These UI affordances do not relax backend
validation, build, queue, retry, or delivery enforcement.
### Editor metadata and trusted evidence
Read responses may contain server-owned `review_send` and `approval_gate`
evidence. Ordinary version mutations send only client-owned `created_from`,
`field_overrides`, and `opt_ins` editor metadata. The client omits review and
approval evidence; the server rejects attempts to write it through an editor
mutation and preserves its existing trusted value during metadata updates.
Supported unlock, successor-version, and build invalidation rules still remove
stale evidence when required. Opening or leaving the Template editor must not
require manually deleting server review metadata.
## Operator checks
Before live delivery, confirm that:
- the profile is active and still authorized for the campaign owner;
- SMTP and optional IMAP profile tests pass;
- validation and build occurred after the latest transport-identity change;
- append-to-Sent is enabled only when the selected profile has IMAP; and
- reports show the profile-bound snapshot revisions and delivery outcomes,
never credentials or resolved transport configuration.
+87
View File
@@ -0,0 +1,87 @@
# Recipient And Address Management Boundary
Campaigns own campaign-local recipient entries because sending and reporting
need a frozen recipient snapshot. Long-lived address management is a separate
domain owned by `govoplan-addresses`.
## `govoplan-campaign` Owns
- campaign-local recipient entries
- campaign-local recipient import mapping and validation
- message addressing for a concrete campaign version
- send/build/report evidence for the exact recipients used
- campaign-local exclusions, warnings, and review status
- recipient-specific attachment and template evidence
Campaign data is immutable once a version is built for sending. Later address
book changes must not rewrite historical campaign evidence.
## `govoplan-addresses` Owns
- Adrema-style address management
- reusable person, organization, household, and postal-address records
- reusable email address lists and segments
- postal-letter recipient views
- consent, legal-basis, and communication-preference metadata
- deduplication and merge workflows
- import/export of reusable address directories
- address quality checks and change history
The addresses module provides the UI/module boundary for this domain and should
grow stable DTOs and capabilities that campaigns, mail, forms, reporting,
portal, and postbox modules can consume without direct imports.
## Current Integration Contract
The campaign module asks the platform registry for address capabilities and
keeps working when they are absent. It must not import `govoplan-addresses`
ORM models, services, or WebUI components.
When `addresses.lookup` is available, campaign recipient address fields use it
for autocomplete. The campaign page may warm a small suggestion cache with an
empty query and then query by typed text as the user edits sender, reply-to,
global recipient, or per-recipient address fields.
When `addresses.recipient_source` is available, campaign offers a separate
address-book/list import flow. This is for importing reusable recipient sources
into the campaign recipient table; normal one-off address entry still happens
inside the address fields while typing.
The capability should return snapshots, not live ORM objects:
- selected source id and display label
- normalized recipient rows
- provenance fields for source, segment, legal basis, and import time
- update markers so campaigns can show whether a draft is based on stale source
data
Campaign stores the resolved snapshot in the campaign version. It may keep a
reference to the address source for traceability, but the built campaign remains
auditable even if the address source changes later.
If the source revision changes after import, campaign shows a stale-source
warning and lets the user reopen the import dialog with that source preselected.
The user still chooses append or replace; campaign should not silently rewrite
recipient rows.
Distribution Lists is a separate, provider-neutral audience boundary. Campaign
uses `dist_lists.source` and `dist_lists.expand` to preview and freeze mixed
email, postal, internal-mail, and portal candidates. It stores exact list,
revision, source-entry, provider, policy, route, exclusion, and expansion-hash
evidence with the Campaign version. Addresses remains the contact/contact-point
owner; Distribution Lists remains the reusable audience owner; Campaign owns
only its copied recipients, enrichment, route choices, review, and outcomes.
## Non-Goals For Campaign
Campaign should not become the global address book. It should not own:
- deduplication across campaigns
- consent lifecycle
- master-data merge policy
- address-directory permissions beyond campaign use
- postal address normalization
- reusable segmentation rules
Those belong in `govoplan-addresses` or a dedicated records/identity module when
the domain needs stronger governance.
+87
View File
@@ -0,0 +1,87 @@
# Recipient Import Guide
Recipient import lets campaign authors turn spreadsheet-like source data into
campaign-local recipient entries. It is intentionally campaign-local:
reusable address books and Adrema-style address management belong in the
`govoplan-addresses` module.
## Supported Inputs
The current importer is designed for tabular data with a header row. CSV and
spreadsheet-derived tables should be normalized before import so the campaign UI
sees:
- column headers
- row values
- source filename and sheet name when available
- a stable ordered and unordered header fingerprint
Avoid importing production secrets or credentials as recipient fields. Recipient
custom fields may be used in templates and reports, so they should be treated as
campaign data.
## Mapping Fields
Common headers are detected automatically:
- `email`, `e_mail`, `mail`, `to`, `to_email`, `recipient`,
`recipient_email`
- `name`, `full_name`, `recipient_name`, `to_name`
- `id`, `entry_id`, `recipient_id`
Authors can map columns to address fields:
- `from`
- `to`
- `cc`
- `bcc`
- `reply_to`
Rows without a valid `to` address should remain visible with validation issues
instead of disappearing silently. Authors should be able to fix the source file,
adjust the mapping, or exclude the row before building the campaign.
## Mapping Profiles
Mapping profiles save a known column layout so repeated imports can reuse the
same mapping. The importer stores ordered and unordered header fingerprints so a
profile can distinguish exact column order from equivalent column sets.
Administrators should curate shared profiles only for stable recurring sources.
Campaign-local profiles are acceptable for one-off work and experiments.
## User Workflow
1. Open the campaign recipient/data import screen.
2. Upload or paste a tabular source.
3. Review detected headers and preview rows.
4. Pick or adjust a mapping profile.
5. Confirm validation issues, exclusions, and generated recipient ids.
6. Import into the campaign draft.
7. Build messages and review recipient-specific evidence before sending.
## Admin Workflow
Administrators should:
- define naming conventions for recurring mapping profiles
- verify that imported fields have a lawful processing basis for the campaign
- keep reusable address-directory ownership out of campaigns because
`govoplan-addresses` owns that domain
- use campaign reports and audit evidence to trace which source produced which
recipient entries
- remove obsolete shared profiles when an upstream source layout changes
## Evidence Expectations
The campaign should preserve enough evidence to explain a send:
- source filename and sheet name where available
- header fingerprints
- mapping profile id/name when used
- row number or source id
- validation status and exclusion reason
- final recipient addresses used for the built message
The evidence should be available in reports without requiring the original
source file to be reprocessed.
+50
View File
@@ -0,0 +1,50 @@
# GovOPlaN Campaign Examples
These examples are the maintained scenario catalogue for campaign release
checks. They are intentionally small and credential-free. Add concrete fixture
files next to this README only when they can be validated by the current
campaign schema and do not require production data.
## Scenarios
| Scenario | Required Modules | Release Check |
| --- | --- | --- |
| [`simple-announcement`](simple-announcement/campaign.json) | core, access, campaigns | Validate and build one active recipient without attachments while Mail and Files are absent. |
| `addressing-matrix` | core, access, campaigns | Exercise To, CC, BCC, Reply-To, bounce, and disposition-notification fields. |
| `global-attachment` | core, access, campaigns; optional files | Build one deterministic attachment and verify evidence. |
| `recipient-attachment-rules` | core, access, campaigns; optional files | Match recipient-specific attachment rules and verify per-recipient evidence. |
| `zip-protected` | core, access, campaigns | Build password-protected AES ZIP output and verify password-source metadata. |
| `mail-profile-send` | core, access, campaigns, mail | Select a reusable mail profile and send through the GreenMail test bed. |
| `legacy-inline-mail-rejected` | core, access, campaigns, mail | Confirm legacy campaign-local SMTP/IMAP data fails closed and requires explicit profile migration. |
| `warnings-review` | core, access, campaigns | Require explicit review before queueing jobs with warnings. |
| `blocked-send` | core, access, campaigns | Confirm blocked recipients or missing attachments cannot be queued. |
| `mock-delivery` | core, access, campaigns, mail with dev capability | Capture messages in the development mailbox. |
| [`greenmail-delivery`](greenmail-delivery/campaign.json) | core, access, audit, campaigns, mail | Run a credential-free Campaign through a Mail-owned profile, GreenMail SMTP/IMAP, report/audit checks, repeat-send protection, and bounded failure drills. |
## Fixture Rules
- Do not commit real recipients, mail credentials, or production attachment
names.
- Keep attachments deterministic and small.
- Store transport secrets only in encrypted Mail profiles or local `.env`
values consumed directly by the test bed, never in campaign JSON.
- Declare optional module requirements in fixture metadata.
- Fixtures must not import files or mail modules directly; optional behavior is
discovered through core module metadata and capabilities.
## Validation Flow
Before a release tag:
1. Run module permutation startup checks from core.
2. Run `python -m unittest discover -s tests -p 'test_example_campaigns.py'`
from this repository. The acceptance test copies each maintained fixture to
an unrelated temporary workspace before using Campaign's public loader,
validator, and message builder.
3. Validate every committed example fixture against the current campaign schema.
4. Build exact messages for each fixture.
5. Run the mock-delivery example when the dev mailbox capability is enabled.
6. Run `dev/mail-testbed/run_transport_smoke.py` for low-level transport and attachment variants.
7. Run `dev/mail-testbed/run_campaign_acceptance.py` for the Campaign journey and bounded evidence.
8. Execute the delivery checklist in
`docs/EXAMPLE_CAMPAIGNS_AND_RELEASE_CHECKLIST.md`.
+88
View File
@@ -0,0 +1,88 @@
{
"version": "1.0",
"campaign": {
"id": "greenmail-delivery",
"name": "GreenMail delivery acceptance",
"description": "Credential-free Campaign fixture for the local SMTP/IMAP acceptance test bed.",
"mode": "test"
},
"fields": [
{
"name": "display_name",
"type": "string",
"label": "Display name",
"required": true
},
{
"name": "acceptance_run",
"type": "string",
"label": "Acceptance run",
"required": true
}
],
"server": {
"mail_profile_id": "00000000-0000-4000-8000-000000000001"
},
"recipients": {
"from": [
{
"email": "campaign-test@govoplan.test",
"name": "GovOPlaN acceptance",
"type": "to"
}
],
"allow_individual_to": true
},
"template": {
"subject": "[GovOPlaN acceptance ${acceptance_run}] Campaign delivery",
"text": "Hello ${display_name},\n\nThis is an isolated GovOPlaN Campaign SMTP/IMAP acceptance message.\n",
"body_mode": "text"
},
"attachments": {
"base_path": ".",
"send_without_attachments_behavior": "continue",
"global": []
},
"entries": {
"inline": [
{
"id": "greenmail-recipient",
"to": [
{
"email": "campaign-test@govoplan.test",
"name": "GreenMail recipient",
"type": "to"
}
],
"fields": {
"display_name": "GreenMail recipient",
"acceptance_run": "fixture"
}
}
]
},
"validation_policy": {
"missing_email": "block",
"template_error": "block"
},
"delivery": {
"rate_limit": {
"messages_per_minute": 60
},
"retry": {
"max_attempts": 3,
"backoff_seconds": [
1,
5,
30
]
},
"imap_append_sent": {
"enabled": true,
"folder": "Sent"
}
},
"status_tracking": {
"enabled": true
}
}
+22
View File
@@ -0,0 +1,22 @@
{
"scenario": "greenmail-delivery",
"campaign_file": "campaign.json",
"required_modules": [
"core",
"access",
"audit",
"campaigns",
"mail"
],
"required_capabilities": [
"mail.campaign_delivery"
],
"transport": "local GreenMail SMTP/IMAP test bed",
"credentials": "local environment only; never copied into Campaign JSON or evidence",
"expected": {
"entries_count": 1,
"built_count": 1,
"smtp_accepted_count": 1,
"imap_appended_count": 1
}
}
@@ -0,0 +1,54 @@
{
"version": "1.0",
"campaign": {
"id": "simple-announcement",
"name": "Simple announcement",
"description": "Credential-free release fixture for Campaign validation and message building.",
"mode": "test"
},
"fields": [
{
"name": "display_name",
"type": "string",
"label": "Display name",
"required": true
}
],
"recipients": {
"from": [
{
"email": "announcements@example.test",
"name": "GovOPlaN Example",
"type": "to"
}
],
"allow_individual_to": true
},
"template": {
"subject": "Planned service maintenance for ${display_name}",
"text": "Hello ${display_name},\n\nThe example service will be unavailable during the announced maintenance window.\n\nThis message was built locally and was not sent.\n",
"body_mode": "text"
},
"attachments": {
"base_path": ".",
"send_without_attachments_behavior": "continue",
"global": []
},
"entries": {
"inline": [
{
"id": "example-recipient",
"to": [
{
"email": "recipient@example.test",
"name": "Example Recipient",
"type": "to"
}
],
"fields": {
"display_name": "Example Recipient"
}
}
]
}
}
+23
View File
@@ -0,0 +1,23 @@
{
"schema_version": 1,
"id": "simple-announcement",
"campaign_file": "campaign.json",
"required_modules": [
"core",
"access",
"campaigns"
],
"absent_optional_modules": [
"files",
"mail"
],
"external_effects": "forbidden",
"expected": {
"campaign_id": "simple-announcement",
"entries_count": 1,
"built_count": 1,
"queueable_count": 1,
"attachment_count": 0,
"subject": "Planned service maintenance for Example Recipient"
}
}
+7 -7
View File
@@ -1,6 +1,6 @@
{
"name": "@govoplan/campaign-webui",
"version": "0.1.5",
"version": "0.1.28",
"private": true,
"type": "module",
"main": "webui/src/index.ts",
@@ -19,14 +19,14 @@
"LICENSE"
],
"dependencies": {
"read-excel-file": "^9.2.0"
"read-excel-file": "9.2.0"
},
"peerDependencies": {
"@govoplan/core-webui": "^0.1.5",
"lucide-react": "^0.555.0",
"react": "^19.0.0",
"react-dom": "^19.0.0",
"react-router-dom": "^7.1.1"
"@govoplan/core-webui": "^0.1.45",
"lucide-react": "^1.23.0",
"react": ">=19.2.7 <20",
"react-dom": ">=19.2.7 <20",
"react-router": ">=8.3.0 <9"
},
"peerDependenciesMeta": {
"@govoplan/core-webui": {
+3 -3
View File
@@ -4,18 +4,18 @@ build-backend = "setuptools.build_meta"
[project]
name = "govoplan-campaign"
version = "0.1.5"
version = "0.1.28"
description = "GovOPlaN campaigns module with backend and WebUI integration."
readme = "README.md"
requires-python = ">=3.12"
license = { file = "LICENSE" }
authors = [{ name = "GovOPlaN" }]
dependencies = [
"govoplan-core>=0.1.5",
"govoplan-core>=0.1.45",
"jsonschema>=4,<5",
"pydantic>=2,<3",
"SQLAlchemy>=2,<3",
"pyzipper>=0.3,<1",
"pyzipper>=0.4,<1",
]
[tool.setuptools.packages.find]
@@ -0,0 +1,265 @@
from __future__ import annotations
import copy
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Mapping
from sqlalchemy.orm import Session
from govoplan_core.core.approvals import (
ApprovalRequestCreateCommand,
ApprovalRequestRef,
ApprovalStepDefinition,
)
from govoplan_campaign.backend.db.models import Campaign, CampaignVersion
from govoplan_campaign.backend.integrations import (
ApprovalGateUnavailable,
approvals_integration,
)
from govoplan_campaign.backend.sending.execution import ensure_execution_snapshot
APPROVAL_GATE_KEY = "approval_gate"
SUBJECT_MODULE = "campaigns"
SUBJECT_TYPE = "campaign_execution"
class CampaignApprovalGateError(ValueError):
pass
@dataclass(frozen=True, slots=True)
class _TenantPrincipal:
tenant_id: str
account_id: str | None = None
def campaign_approval_gate(version: CampaignVersion) -> dict[str, object] | None:
state = version.editor_state if isinstance(version.editor_state, dict) else {}
gate = state.get(APPROVAL_GATE_KEY)
return dict(gate) if isinstance(gate, dict) else None
def request_campaign_approval(
session: Session,
principal: object,
*,
campaign: Campaign,
version: CampaignVersion,
title: str,
description: str | None,
steps: tuple[ApprovalStepDefinition, ...],
idempotency_key: str,
template_id: str | None = None,
template_revision: int | None = None,
unique_actors_across_steps: bool = True,
expires_at: datetime | None = None,
policy_refs: tuple[str, ...] = (),
) -> ApprovalRequestRef:
if campaign.tenant_id != str(getattr(principal, "tenant_id", "") or ""):
raise CampaignApprovalGateError("Campaign approval tenant mismatch.")
if version.campaign_id != campaign.id:
raise CampaignApprovalGateError("Campaign approval version mismatch.")
snapshot = ensure_execution_snapshot(session, version)
digest = str(version.execution_snapshot_hash or "")
if len(digest) != 64:
raise CampaignApprovalGateError(
"Build a valid Campaign execution snapshot before requesting approval."
)
subject_version = _subject_version(version, snapshot.build_token)
evidence_actors = _campaign_evidence_actors(campaign, version)
try:
request = approvals_integration().create_request(
session,
principal,
command=ApprovalRequestCreateCommand(
title=title,
description=description,
subject_module=SUBJECT_MODULE,
subject_type=SUBJECT_TYPE,
subject_id=version.id,
subject_version=subject_version,
subject_digest=digest,
steps=steps,
separation_of_duties=True,
unique_actors_across_steps=unique_actors_across_steps,
expires_at=expires_at,
policy_refs=policy_refs,
evidence_actors=evidence_actors,
template_id=template_id,
template_revision=template_revision,
metadata={
"campaign_id": campaign.id,
"campaign_version_number": version.version_number,
"execution_snapshot_version": snapshot.snapshot_version,
},
),
idempotency_key=idempotency_key,
)
except ApprovalGateUnavailable as exc:
raise CampaignApprovalGateError(str(exc)) from exc
state = copy.deepcopy(version.editor_state or {})
state[APPROVAL_GATE_KEY] = {
"request_id": request.id,
"request_revision": request.revision,
"subject_version": subject_version,
"subject_digest": digest,
"requested_at": datetime.now(UTC).isoformat(),
"requested_by_user_id": _actor_id(principal),
}
version.editor_state = state
session.add(version)
session.flush()
return request
def assert_campaign_approval(
session: Session,
*,
tenant_id: str,
version: CampaignVersion,
) -> None:
gate = campaign_approval_gate(version)
if gate is None:
return
snapshot = ensure_execution_snapshot(session, version)
digest = str(version.execution_snapshot_hash or "")
subject_version = _subject_version(version, snapshot.build_token)
if (
gate.get("subject_digest") != digest
or gate.get("subject_version") != subject_version
):
raise CampaignApprovalGateError(
"Campaign execution changed after approval was requested. Request approval for the current build."
)
request_id = str(gate.get("request_id") or "").strip()
if not request_id:
raise CampaignApprovalGateError(
"Campaign approval gate has no request reference."
)
try:
check = approvals_integration().check_approved(
session,
_TenantPrincipal(tenant_id=tenant_id),
request_id=request_id,
subject_module=SUBJECT_MODULE,
subject_type=SUBJECT_TYPE,
subject_id=version.id,
subject_version=subject_version,
subject_digest=digest,
)
except (ApprovalGateUnavailable, LookupError, ValueError) as exc:
raise CampaignApprovalGateError(str(exc)) from exc
if not check.approved:
raise CampaignApprovalGateError(
f"Campaign delivery requires Approval request {request_id}, which is {check.state}."
)
def campaign_approval_status(
session: Session,
*,
tenant_id: str,
version: CampaignVersion,
) -> dict[str, object]:
gate = campaign_approval_gate(version)
integration = approvals_integration()
if gate is None:
return {
"configured": False,
"available": integration.available,
"approved": False,
"state": "not_required",
"explanation": None,
}
try:
assert_campaign_approval(session, tenant_id=tenant_id, version=version)
except CampaignApprovalGateError as exc:
return {
**gate,
"configured": True,
"available": integration.available,
"approved": False,
"state": "unavailable" if not integration.available else "pending",
"explanation": str(exc),
}
return {
**gate,
"configured": True,
"available": True,
"approved": True,
"state": "approved",
"explanation": None,
}
def clear_campaign_approval_gate(version: CampaignVersion) -> None:
state = copy.deepcopy(version.editor_state or {})
state.pop(APPROVAL_GATE_KEY, None)
version.editor_state = state
def _campaign_evidence_actors(
campaign: Campaign,
version: CampaignVersion,
) -> Mapping[str, tuple[str, ...]]:
validation = (
version.validation_summary
if isinstance(version.validation_summary, dict)
else {}
)
build = version.build_summary if isinstance(version.build_summary, dict) else {}
editor = version.editor_state if isinstance(version.editor_state, dict) else {}
review = (
editor.get("review_send") if isinstance(editor.get("review_send"), dict) else {}
)
review_decisions = (
review.get("issue_decisions")
if isinstance(review.get("issue_decisions"), list)
else []
)
values = {
"author": (campaign.created_by_user_id,),
"owner": (campaign.owner_user_id,),
"validator": (
validation.get("validated_by_user_id"),
version.locked_by_user_id,
),
"builder": (build.get("built_by_user_id"),),
"reviewer": (
review.get("updated_by_user_id"),
*(
item.get("actor_user_id")
for item in review_decisions
if isinstance(item, dict)
),
),
}
return {
role: tuple(dict.fromkeys(str(item) for item in actors if item))
for role, actors in values.items()
if any(actors)
}
def _subject_version(version: CampaignVersion, build_token: str | None) -> str:
return str(build_token or f"campaign-version-{version.version_number}")[:120]
def _actor_id(principal: object) -> str | None:
for name in ("account_id", "user_id", "identity_id", "membership_id"):
value = str(getattr(principal, name, "") or "").strip()
if value:
return value
return None
__all__ = [
"CampaignApprovalGateError",
"assert_campaign_approval",
"campaign_approval_gate",
"campaign_approval_status",
"clear_campaign_approval_gate",
"request_campaign_approval",
]
@@ -0,0 +1,78 @@
from __future__ import annotations
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
from govoplan_core.core.approvals import ApprovalActorSelector, ApprovalStepDefinition
class CampaignApprovalSelectorInput(BaseModel):
model_config = ConfigDict(extra="forbid")
kind: Literal["account", "group", "role", "function_assignment", "any_account"]
value: str = Field(min_length=1, max_length=255)
label: str | None = Field(default=None, max_length=255)
class CampaignApprovalStepInput(BaseModel):
model_config = ConfigDict(extra="forbid")
key: str = Field(min_length=1, max_length=120)
label: str = Field(min_length=1, max_length=255)
selectors: list[CampaignApprovalSelectorInput] = Field(min_length=1, max_length=500)
required_approvals: int = Field(default=1, ge=1, le=500)
rejection_policy: Literal["fail_fast", "collect"] = "fail_fast"
due_at: datetime | None = None
signature_required: bool = False
forbidden_evidence_roles: list[
Literal["author", "owner", "validator", "builder", "reviewer"]
] = Field(default_factory=list, max_length=5)
def to_definition(self) -> ApprovalStepDefinition:
return ApprovalStepDefinition(
key=self.key,
label=self.label,
selectors=tuple(
ApprovalActorSelector(item.kind, item.value, item.label)
for item in self.selectors
),
required_approvals=self.required_approvals,
rejection_policy=self.rejection_policy,
due_at=self.due_at,
signature_required=self.signature_required,
forbidden_evidence_roles=tuple(self.forbidden_evidence_roles),
)
class CampaignApprovalRequestInput(BaseModel):
model_config = ConfigDict(extra="forbid")
title: str = Field(
default="Approve Campaign delivery", min_length=1, max_length=255
)
description: str | None = Field(default=None, max_length=10_000)
steps: list[CampaignApprovalStepInput] = Field(default_factory=list, max_length=100)
template_id: str | None = Field(default=None, max_length=36)
template_revision: int | None = Field(default=None, ge=1)
unique_actors_across_steps: bool = True
expires_at: datetime | None = None
policy_refs: list[str] = Field(default_factory=list, max_length=500)
idempotency_key: str = Field(min_length=1, max_length=160)
@model_validator(mode="after")
def validate_source(self) -> "CampaignApprovalRequestInput":
template = self.template_id is not None or self.template_revision is not None
if template and (self.template_id is None or self.template_revision is None):
raise ValueError("Template id and revision must be supplied together.")
if template and self.steps:
raise ValueError("Use either an Approval template or inline steps.")
if not template and not self.steps:
raise ValueError(
"At least one Approval step is required without a template."
)
return self
__all__ = ["CampaignApprovalRequestInput"]
@@ -0,0 +1,294 @@
from __future__ import annotations
import copy
from dataclasses import dataclass
from datetime import UTC, datetime
import hashlib
import json
from typing import Any, Mapping
from sqlalchemy.orm import Session
from govoplan_core.auth import ApiPrincipal, has_scope
from govoplan_core.core.policy import (
CampaignArchiveEncryptionDecision,
CampaignArchiveEncryptionRequest,
campaign_archive_encryption_policy,
)
from govoplan_campaign.backend.db.models import Campaign
from govoplan_campaign.backend.runtime import get_registry
LEGACY_ZIPCRYPTO_SCOPE = "campaigns:archive:use_legacy_zipcrypto"
LEGACY_ZIPCRYPTO_LABEL = "Legacy ZipCrypto — Windows-compatible, weak encryption"
class CampaignArchiveEncryptionError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class EffectiveArchiveEncryptionPolicy:
available: bool
allowed_password_encryption_methods: frozenset[str]
allowed_password_delivery_channels: frozenset[str]
policy_hash: str
source_path: tuple[Mapping[str, Any], ...]
reason: str
diagnostics: tuple[Mapping[str, Any], ...] = ()
def to_dict(self) -> dict[str, Any]:
return {
"available": self.available,
"allowed_password_encryption_methods": sorted(
self.allowed_password_encryption_methods
),
"allowed_password_delivery_channels": sorted(
self.allowed_password_delivery_channels
),
"policy_hash": self.policy_hash,
"source_path": [dict(item) for item in self.source_path],
"reason": self.reason,
"diagnostics": [dict(item) for item in self.diagnostics],
"legacy_label": LEGACY_ZIPCRYPTO_LABEL,
}
def effective_archive_encryption_policy(
session: Session,
campaign: Campaign,
) -> EffectiveArchiveEncryptionPolicy:
provider = campaign_archive_encryption_policy(get_registry())
if provider is None:
payload = {
"available": False,
"allowed_password_encryption_methods": ["aes"],
"allowed_password_delivery_channels": [
"in_person",
"letter",
"phone",
"separate_mail",
"sms",
],
"source_path": [
{
"scope_type": "system",
"scope_id": None,
"path": "system",
"label": "Secure local fallback",
"applied_fields": ["allowed_password_encryption_methods"],
"policy": {
"allowed_password_encryption_methods": ["aes"],
"policy_provider": "unavailable",
},
}
],
}
return EffectiveArchiveEncryptionPolicy(
available=False,
allowed_password_encryption_methods=frozenset({"aes"}),
allowed_password_delivery_channels=frozenset(
{"separate_mail", "sms", "letter", "phone", "in_person"}
),
policy_hash=_hash(payload),
source_path=tuple(payload["source_path"]),
reason=(
"Policy is unavailable. AES remains available through the secure "
"local baseline; legacy ZipCrypto fails closed."
),
)
owner_type: str | None = None
owner_id: str | None = None
if campaign.owner_group_id:
owner_type, owner_id = "group", campaign.owner_group_id
elif campaign.owner_user_id:
owner_type, owner_id = "user", campaign.owner_user_id
decision: CampaignArchiveEncryptionDecision = (
provider.resolve_campaign_archive_encryption(
session,
request=CampaignArchiveEncryptionRequest(
tenant_id=campaign.tenant_id,
campaign_id=campaign.id,
owner_type=owner_type, # type: ignore[arg-type]
owner_id=owner_id,
),
)
)
return EffectiveArchiveEncryptionPolicy(
available=True,
allowed_password_encryption_methods=frozenset(
decision.allowed_password_encryption_methods
),
allowed_password_delivery_channels=frozenset(
decision.allowed_password_delivery_channels
),
policy_hash=decision.policy_hash,
source_path=tuple(step.to_dict() for step in decision.source_path),
reason=decision.reason or "Effective archive-encryption policy resolved.",
diagnostics=decision.diagnostics,
)
def assert_archive_encryption_allowed(
session: Session,
campaign: Campaign,
raw_json: Mapping[str, Any],
*,
principal: ApiPrincipal | None = None,
) -> EffectiveArchiveEncryptionPolicy:
policy = effective_archive_encryption_policy(session, campaign)
for archive in _archive_configs(raw_json):
method = str(archive.get("method") or "aes")
if method not in policy.allowed_password_encryption_methods:
raise CampaignArchiveEncryptionError(
f"{_method_label(method)} is blocked. {policy.reason}"
)
if method == "zip_standard":
if not policy.available:
raise CampaignArchiveEncryptionError(
"Legacy ZipCrypto cannot be used while Policy is unavailable."
)
if principal is not None and not has_scope(principal, LEGACY_ZIPCRYPTO_SCOPE):
raise CampaignArchiveEncryptionError(
f"Missing scope: {LEGACY_ZIPCRYPTO_SCOPE}"
)
if not archive.get("legacy_zipcrypto_acknowledged"):
raise CampaignArchiveEncryptionError(
f"{LEGACY_ZIPCRYPTO_LABEL} requires explicit acknowledgement."
)
if len(str(archive.get("legacy_zipcrypto_reason") or "").strip()) < 10:
raise CampaignArchiveEncryptionError(
f"{LEGACY_ZIPCRYPTO_LABEL} requires a reason of at least 10 characters."
)
if not archive.get("legacy_zipcrypto_acknowledged_by") or not archive.get(
"legacy_zipcrypto_acknowledged_at"
):
raise CampaignArchiveEncryptionError(
"Legacy ZipCrypto acknowledgement has no server-recorded actor or time. Save the campaign again."
)
if archive.get("password_enabled"):
# Existing campaign revisions predate the explicit field. Their
# model default is the separate-mail channel; apply the same
# normalization before policy enforcement so saved revisions do
# not become unusable merely because the field was omitted.
channel = str(
archive.get("password_delivery_channel") or "separate_mail"
)
if channel not in policy.allowed_password_delivery_channels:
raise CampaignArchiveEncryptionError(
f"Password-delivery channel {channel!r} is blocked by the effective policy."
)
return policy
def stamp_legacy_zipcrypto_acknowledgements(
session: Session,
campaign: Campaign,
current_raw_json: Mapping[str, Any],
candidate_raw_json: dict[str, Any] | None,
*,
principal: ApiPrincipal,
) -> tuple[dict[str, Any] | None, list[dict[str, Any]]]:
if candidate_raw_json is None:
return None, []
candidate = copy.deepcopy(candidate_raw_json)
current_attachments = current_raw_json.get("attachments")
candidate_attachments = candidate.get("attachments")
current_zip = current_attachments.get("zip") if isinstance(current_attachments, Mapping) else None
candidate_zip = candidate_attachments.get("zip") if isinstance(candidate_attachments, Mapping) else None
if json.dumps(current_zip, sort_keys=True) == json.dumps(candidate_zip, sort_keys=True):
# Saving an unrelated repair does not authorize use of an existing
# archive or invent an acknowledgement. Preserve the exact stored ZIP
# configuration, including missing evidence or now-revoked policy.
# Build/review/delivery still validate the complete configuration.
return candidate, []
current_by_id = {
str(item.get("id") or index): item
for index, item in enumerate(_archive_configs(current_raw_json))
}
acknowledgements: list[dict[str, Any]] = []
policy = effective_archive_encryption_policy(session, campaign)
archives = _archive_configs(candidate)
for index, archive in enumerate(archives):
if str(archive.get("method") or "aes") != "zip_standard":
archive.pop("legacy_zipcrypto_acknowledged_by", None)
archive.pop("legacy_zipcrypto_acknowledged_at", None)
continue
if not has_scope(principal, LEGACY_ZIPCRYPTO_SCOPE):
raise CampaignArchiveEncryptionError(
f"Missing scope: {LEGACY_ZIPCRYPTO_SCOPE}"
)
if not policy.available or "zip_standard" not in policy.allowed_password_encryption_methods:
raise CampaignArchiveEncryptionError(
f"{LEGACY_ZIPCRYPTO_LABEL} is blocked. {policy.reason}"
)
reason = str(archive.get("legacy_zipcrypto_reason") or "").strip()
if not archive.get("legacy_zipcrypto_acknowledged") or len(reason) < 10:
raise CampaignArchiveEncryptionError(
f"{LEGACY_ZIPCRYPTO_LABEL} requires acknowledgement and a reason of at least 10 characters."
)
key = str(archive.get("id") or index)
previous = current_by_id.get(key, {})
unchanged = (
previous.get("method") == "zip_standard"
and previous.get("legacy_zipcrypto_acknowledged") is True
and str(previous.get("legacy_zipcrypto_reason") or "").strip() == reason
and previous.get("legacy_zipcrypto_acknowledged_by")
and previous.get("legacy_zipcrypto_acknowledged_at")
)
if unchanged:
archive["legacy_zipcrypto_acknowledged_by"] = previous[
"legacy_zipcrypto_acknowledged_by"
]
archive["legacy_zipcrypto_acknowledged_at"] = previous[
"legacy_zipcrypto_acknowledged_at"
]
else:
archive["legacy_zipcrypto_acknowledged_by"] = principal.user.id
archive["legacy_zipcrypto_acknowledged_at"] = datetime.now(UTC).isoformat()
acknowledgements.append(
{
"archive_id": key,
"reason": reason,
"policy_hash": policy.policy_hash,
}
)
# Modified archive settings must satisfy the complete current policy, not
# only the special ZipCrypto acknowledgement checks above.
assert_archive_encryption_allowed(session, campaign, candidate, principal=principal)
return candidate, acknowledgements
def has_password_archives(raw_json: Mapping[str, Any]) -> bool:
return any(bool(item.get("password_enabled")) for item in _archive_configs(raw_json))
def _archive_configs(raw_json: Mapping[str, Any]) -> list[dict[str, Any]]:
attachments = raw_json.get("attachments")
zip_config = attachments.get("zip") if isinstance(attachments, Mapping) else None
archives = zip_config.get("archives") if isinstance(zip_config, Mapping) else None
if isinstance(archives, list):
return [item for item in archives if isinstance(item, dict)]
return []
def _method_label(method: str) -> str:
return LEGACY_ZIPCRYPTO_LABEL if method == "zip_standard" else method.upper()
def _hash(value: object) -> str:
return hashlib.sha256(
json.dumps(value, sort_keys=True, separators=(",", ":"), default=str).encode()
).hexdigest()
__all__ = [
"CampaignArchiveEncryptionError",
"EffectiveArchiveEncryptionPolicy",
"LEGACY_ZIPCRYPTO_LABEL",
"LEGACY_ZIPCRYPTO_SCOPE",
"assert_archive_encryption_allowed",
"effective_archive_encryption_policy",
"has_password_archives",
"stamp_legacy_zipcrypto_acknowledgements",
]
@@ -0,0 +1,734 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
import hashlib
import json
from typing import Any, Callable
from sqlalchemy import or_
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignVersion,
)
from govoplan_core.core.object_storage import (
StorageBackend,
StorageBackendError,
StorageObjectInfo,
)
from govoplan_core.core.recovery import (
RecoveryMode,
RecoveryOperation,
RecoveryPlan,
RecoveryStatus,
TERMINAL_RECOVERY_STATUSES,
)
from govoplan_core.core.recovery_runtime import begin_durable_recovery_operation
from govoplan_core.core.runtime_coordination import (
DistributedLease,
RuntimeIdentity,
)
CAMPAIGN_ARTIFACT_NAMESPACE = "campaign-artifacts"
MINIMUM_GRACE_HOURS = 24
MAXIMUM_PAGE_SIZE = 1000
_CHECKPOINT_BATCH_SIZE = 25
SessionFactory = Callable[[], Session]
class CampaignArtifactReconciliationError(RuntimeError):
pass
@dataclass(slots=True)
class ArtifactCandidate:
key: str
size_bytes: int
modified_at: datetime
age_seconds: int
reason: str = "unreferenced_after_grace_period"
disposition: str = "candidate"
failure_type: str | None = None
def as_dict(self) -> dict[str, Any]:
return {
"key": self.key,
"size_bytes": self.size_bytes,
"modified_at": self.modified_at.isoformat(),
"age_seconds": self.age_seconds,
"reason": self.reason,
"disposition": self.disposition,
"failure_type": self.failure_type,
}
@dataclass(slots=True)
class ArtifactInventory:
tenant_prefix: str
cursor: str | None
next_cursor: str | None
scanned_count: int
scanned_bytes: int
referenced_count: int
active_build_count: int
young_count: int
unknown_age_count: int
invalid_shape_count: int
candidates: list[ArtifactCandidate]
manifest_sha256: str
@property
def candidate_bytes(self) -> int:
return sum(candidate.size_bytes for candidate in self.candidates)
def response(
self,
*,
apply: bool,
status: str,
recovery_operation_id: str | None = None,
) -> dict[str, Any]:
deleted = [
candidate
for candidate in self.candidates
if candidate.disposition == "deleted"
]
failures = [
candidate
for candidate in self.candidates
if candidate.disposition
in {"delete_failed", "delete_outcome_unknown"}
]
return {
"apply": apply,
"status": status,
"recovery_operation_id": recovery_operation_id,
"tenant_prefix": self.tenant_prefix,
"cursor": self.cursor,
"next_cursor": self.next_cursor,
"scanned_count": self.scanned_count,
"scanned_bytes": self.scanned_bytes,
"referenced_count": self.referenced_count,
"active_build_count": self.active_build_count,
"young_count": self.young_count,
"unknown_age_count": self.unknown_age_count,
"invalid_shape_count": self.invalid_shape_count,
"candidate_count": len(self.candidates),
"candidate_bytes": self.candidate_bytes,
"deleted_count": len(deleted),
"deleted_bytes": sum(candidate.size_bytes for candidate in deleted),
"failure_count": len(failures),
"manifest_sha256": self.manifest_sha256,
"candidates": [candidate.as_dict() for candidate in self.candidates],
}
def campaign_artifact_inventory(
session: Session,
*,
storage: StorageBackend,
tenant_id: str,
grace_period: timedelta,
cursor: str | None = None,
page_size: int = 250,
now: datetime | None = None,
) -> ArtifactInventory:
observed_at = _as_utc(now or datetime.now(timezone.utc))
if grace_period < timedelta(hours=MINIMUM_GRACE_HOURS):
raise ValueError(
f"Campaign artifact grace period must be at least {MINIMUM_GRACE_HOURS} hours"
)
bounded_page_size = max(1, min(int(page_size), MAXIMUM_PAGE_SIZE))
prefix = _tenant_artifact_prefix(tenant_id)
if cursor is not None and not cursor.startswith(prefix):
raise ValueError("Campaign artifact cursor is outside the tenant namespace")
page = storage.list_objects(
prefix=prefix,
after=cursor,
limit=bounded_page_size,
)
page_keys = {info.key for info in page.objects}
referenced_keys = _referenced_artifact_keys(
session,
tenant_id=tenant_id,
prefix=prefix,
artifact_keys=page_keys,
)
active_build_ids = _active_build_ids(
session,
tenant_id=tenant_id,
now=observed_at,
)
candidates: list[ArtifactCandidate] = []
referenced_count = 0
active_build_count = 0
young_count = 0
unknown_age_count = 0
invalid_shape_count = 0
scanned_bytes = 0
manifest_rows: list[dict[str, Any]] = []
for info in page.objects:
scanned_bytes += info.size_bytes
build_id = _build_id_for_key(info.key, prefix=prefix)
modified_at = _object_modified_at(info)
manifest_rows.append(
{
"key_sha256": _sha256(info.key),
"size_bytes": info.size_bytes,
"modified_at": modified_at.isoformat() if modified_at else None,
}
)
if build_id is None:
invalid_shape_count += 1
continue
if info.key in referenced_keys:
referenced_count += 1
continue
if build_id in active_build_ids:
active_build_count += 1
continue
if modified_at is None:
unknown_age_count += 1
continue
age = observed_at - modified_at
if age < grace_period:
young_count += 1
continue
candidates.append(
ArtifactCandidate(
key=info.key,
size_bytes=info.size_bytes,
modified_at=modified_at,
age_seconds=max(0, int(age.total_seconds())),
)
)
return ArtifactInventory(
tenant_prefix=prefix,
cursor=cursor,
next_cursor=page.next_cursor,
scanned_count=len(page.objects),
scanned_bytes=scanned_bytes,
referenced_count=referenced_count,
active_build_count=active_build_count,
young_count=young_count,
unknown_age_count=unknown_age_count,
invalid_shape_count=invalid_shape_count,
candidates=candidates,
manifest_sha256=_canonical_sha256(manifest_rows),
)
def reconcile_campaign_artifacts(
session_factory: SessionFactory,
*,
storage: StorageBackend,
identity: RuntimeIdentity,
tenant_id: str,
apply: bool = False,
idempotency_key: str | None = None,
grace_period_hours: int = MINIMUM_GRACE_HOURS,
cursor: str | None = None,
page_size: int = 250,
now: datetime | None = None,
) -> dict[str, Any]:
if apply and not (idempotency_key or "").strip():
raise ValueError("Applied Campaign artifact cleanup requires an idempotency key")
if grace_period_hours < MINIMUM_GRACE_HOURS:
raise ValueError(
f"Campaign artifact grace period must be at least {MINIMUM_GRACE_HOURS} hours"
)
observed_at = _as_utc(now or datetime.now(timezone.utc))
grace_period = timedelta(hours=grace_period_hours)
if not apply:
with session_factory() as session:
inventory = campaign_artifact_inventory(
session,
storage=storage,
tenant_id=tenant_id,
grace_period=grace_period,
cursor=cursor,
page_size=page_size,
now=observed_at,
)
return inventory.response(apply=False, status="dry_run")
prefix = _tenant_artifact_prefix(tenant_id)
request = {
"tenant_id": tenant_id,
"prefix": prefix,
"cursor_sha256": _sha256(cursor) if cursor else None,
"page_size": max(1, min(int(page_size), MAXIMUM_PAGE_SIZE)),
"grace_period_hours": grace_period_hours,
}
recovery_start = begin_durable_recovery_operation(
session_factory,
identity=identity,
module_id="campaigns",
operation_type="artifact-orphan-reconciliation",
idempotency_key=str(idempotency_key).strip(),
request=request,
recovery_plan=RecoveryPlan(
mode=RecoveryMode.FORWARD_RECOVERY,
preconditions=(
"inventory is bounded to one tenant Campaign artifact prefix",
"objects younger than the conservative grace period are excluded",
),
forward_recovery_steps=(
"retry only objects still unreferenced by committed Campaign state",
),
verification_steps=(
"probe every attempted object after deletion",
"preserve database references without mutation",
),
),
precondition_evidence={
"tenant_prefix_sha256": _sha256(prefix),
"grace_period_hours": grace_period_hours,
"page_size": request["page_size"],
},
lease_resource_key=f"campaign:artifact-reconcile:{tenant_id}",
lease_ttl_seconds=15 * 60,
resource_type="campaign_artifact_namespace",
resource_id=tenant_id,
metadata={"tenant_id": tenant_id},
)
if recovery_start.replayed:
return _replayed_response(
prefix=prefix,
cursor=cursor,
operation_id=recovery_start.operation_id,
)
operation = recovery_start.operation
if operation is None: # pragma: no cover - guarded by replay branch
raise CampaignArtifactReconciliationError(
"Campaign artifact cleanup authority was not created"
)
try:
try:
with session_factory() as session:
inventory = campaign_artifact_inventory(
session,
storage=storage,
tenant_id=tenant_id,
grace_period=grace_period,
cursor=cursor,
page_size=page_size,
now=observed_at,
)
except Exception as exc:
operation.fail(
summary="Campaign artifact inventory failed before deletion",
evidence={
"effect_started": False,
"failure_type": type(exc).__name__,
},
)
raise
operation.checkpoint(
kind="artifact-inventory",
summary="The bounded Campaign artifact inventory was classified",
evidence={
"manifest_sha256": inventory.manifest_sha256,
"scanned_count": inventory.scanned_count,
"candidate_count": len(inventory.candidates),
"candidate_bytes": inventory.candidate_bytes,
"next_page": inventory.next_cursor is not None,
},
)
if inventory.candidates:
_apply_inventory(
session_factory,
storage=storage,
operation=operation,
inventory=inventory,
tenant_id=tenant_id,
now=observed_at,
)
failed = [
item
for item in inventory.candidates
if item.disposition == "delete_failed"
]
unknown = [
item
for item in inventory.candidates
if item.disposition == "delete_outcome_unknown"
]
evidence = _cleanup_evidence(inventory)
if unknown:
operation.unresolved(
status=RecoveryStatus.OUTCOME_UNKNOWN,
summary="Campaign artifact deletion could not be verified",
evidence=evidence,
failure_summary=(
"One or more Campaign artifact deletion outcomes are unknown"
),
)
status = "outcome_unknown"
elif failed:
operation.unresolved(
status=RecoveryStatus.RECOVERY_REQUIRED,
summary="Campaign artifact deletion requires a retry",
evidence=evidence,
failure_summary=(
"One or more unreferenced Campaign artifacts remain"
),
)
status = "recovery_required"
else:
operation.succeed(evidence=evidence)
status = "applied"
return inventory.response(
apply=True,
status=status,
recovery_operation_id=recovery_start.operation_id,
)
except Exception:
if not operation.closed:
try:
operation.release_unresolved()
except Exception:
pass
raise
def _apply_inventory(
session_factory: SessionFactory,
*,
storage: StorageBackend,
operation: Any,
inventory: ArtifactInventory,
tenant_id: str,
now: datetime,
) -> None:
for index in range(0, len(inventory.candidates), _CHECKPOINT_BATCH_SIZE):
batch = inventory.candidates[index : index + _CHECKPOINT_BATCH_SIZE]
with session_factory() as session:
referenced = _referenced_artifact_keys(
session,
tenant_id=tenant_id,
prefix=inventory.tenant_prefix,
artifact_keys={candidate.key for candidate in batch},
)
active_build_ids = _active_build_ids(
session,
tenant_id=tenant_id,
now=now,
)
operation.checkpoint(
kind="artifact-delete-batch-authorized",
summary="Deletion authority was renewed for a bounded object batch",
evidence={
"batch_index": index // _CHECKPOINT_BATCH_SIZE,
"batch_count": len(batch),
"batch_manifest_sha256": _canonical_sha256(
[_sha256(candidate.key) for candidate in batch]
),
},
)
for candidate in batch:
build_id = _build_id_for_key(
candidate.key,
prefix=inventory.tenant_prefix,
)
if candidate.key in referenced or build_id in active_build_ids:
candidate.disposition = "protected_before_delete"
candidate.reason = "reference_or_active_build_appeared"
continue
_delete_and_verify(storage, candidate)
def _delete_and_verify(
storage: StorageBackend,
candidate: ArtifactCandidate,
) -> None:
delete_failure: Exception | None = None
try:
storage.delete(candidate.key)
except (OSError, StorageBackendError) as exc:
delete_failure = exc
try:
remains = storage.exists(candidate.key)
except (OSError, StorageBackendError) as exc:
candidate.disposition = "delete_outcome_unknown"
candidate.failure_type = type(exc).__name__
return
if not remains:
candidate.disposition = "deleted"
return
candidate.disposition = "delete_failed"
candidate.failure_type = (
type(delete_failure).__name__ if delete_failure is not None else None
)
def _referenced_artifact_keys(
session: Session,
*,
tenant_id: str,
prefix: str,
artifact_keys: set[str] | None = None,
) -> set[str]:
if artifact_keys == set():
return set()
keys: set[str] = set()
version_ids = {
identity[1]
for key in artifact_keys or ()
if (identity := _artifact_identity(key, prefix=prefix)) is not None
}
job_query = session.query(
CampaignJob.eml_storage_key,
CampaignJob.resolved_print_output,
).filter(CampaignJob.tenant_id == tenant_id)
if artifact_keys is not None:
filters = [CampaignJob.eml_storage_key.in_(artifact_keys)]
if version_ids:
filters.append(CampaignJob.campaign_version_id.in_(version_ids))
job_query = job_query.filter(or_(*filters))
job_rows = job_query.yield_per(1000)
for eml_storage_key, print_output in job_rows:
_add_key(
keys,
eml_storage_key,
prefix=prefix,
allowed_keys=artifact_keys,
)
_add_key(
keys,
_print_output_storage_key(print_output),
prefix=prefix,
allowed_keys=artifact_keys,
)
version_query = session.query(CampaignVersion.build_summary).join(
Campaign,
Campaign.id == CampaignVersion.campaign_id,
).filter(Campaign.tenant_id == tenant_id)
if artifact_keys is not None:
if not version_ids:
return keys
version_query = version_query.filter(CampaignVersion.id.in_(version_ids))
version_rows = version_query.yield_per(500)
for (build_summary,) in version_rows:
print_output = (
build_summary.get("print_output")
if isinstance(build_summary, dict)
else None
)
_add_key(
keys,
_print_output_storage_key(print_output),
prefix=prefix,
allowed_keys=artifact_keys,
)
return keys
def _artifact_identity(
key: str,
*,
prefix: str,
) -> tuple[str, str, str] | None:
if not key.startswith(prefix):
return None
parts = key[len(prefix) :].split("/")
if len(parts) < 4 or any(not part for part in parts[:4]):
return None
return parts[0], parts[1], parts[2]
def _active_build_ids(
session: Session,
*,
tenant_id: str,
now: datetime,
) -> set[str]:
operations = (
session.query(RecoveryOperation)
.join(
CampaignVersion,
CampaignVersion.id == RecoveryOperation.resource_id,
)
.join(Campaign, Campaign.id == CampaignVersion.campaign_id)
.filter(
Campaign.tenant_id == tenant_id,
RecoveryOperation.module_id == "campaigns",
RecoveryOperation.operation_type == "build-artifacts",
RecoveryOperation.status.not_in(TERMINAL_RECOVERY_STATUSES),
)
.all()
)
resource_keys = {
operation.lease_resource_key
for operation in operations
if operation.lease_resource_key
}
if not resource_keys:
return set()
installation_ids = {operation.installation_id for operation in operations}
leases = (
session.query(DistributedLease)
.filter(
DistributedLease.installation_id.in_(installation_ids),
DistributedLease.resource_key.in_(resource_keys),
)
.all()
)
leases_by_key = {
(lease.installation_id, lease.resource_key): lease for lease in leases
}
active: set[str] = set()
for operation in operations:
lease = leases_by_key.get(
(operation.installation_id, operation.lease_resource_key or "")
)
if (
lease is not None
and lease.holder_node_id == operation.holder_node_id
and lease.holder_incarnation == operation.holder_incarnation
and lease.fencing_token == operation.fencing_token
and _as_utc(lease.expires_at) > now
):
active.add(operation.id)
return active
def _cleanup_evidence(inventory: ArtifactInventory) -> dict[str, Any]:
dispositions: dict[str, int] = {}
for candidate in inventory.candidates:
dispositions[candidate.disposition] = (
dispositions.get(candidate.disposition, 0) + 1
)
verified = not any(
key in dispositions for key in ("delete_failed", "delete_outcome_unknown")
)
return {
"verified": verified,
"checks": {
"candidate_objects": (
"absent-or-newly-protected" if verified else "incomplete"
),
"database_references": "unchanged",
"lease_fence": "renewed-before-each-batch",
},
"inventory_manifest_sha256": inventory.manifest_sha256,
"candidate_manifest_sha256": _canonical_sha256(
[_sha256(candidate.key) for candidate in inventory.candidates]
),
"candidate_count": len(inventory.candidates),
"candidate_bytes": inventory.candidate_bytes,
"dispositions": dispositions,
"database_references_mutated": False,
}
def _replayed_response(
*,
prefix: str,
cursor: str | None,
operation_id: str,
) -> dict[str, Any]:
return {
"apply": True,
"status": "already_completed",
"recovery_operation_id": operation_id,
"tenant_prefix": prefix,
"cursor": cursor,
"next_cursor": None,
"scanned_count": 0,
"scanned_bytes": 0,
"referenced_count": 0,
"active_build_count": 0,
"young_count": 0,
"unknown_age_count": 0,
"invalid_shape_count": 0,
"candidate_count": 0,
"candidate_bytes": 0,
"deleted_count": 0,
"deleted_bytes": 0,
"failure_count": 0,
"manifest_sha256": None,
"candidates": [],
}
def _tenant_artifact_prefix(tenant_id: str) -> str:
normalized = str(tenant_id or "").strip()
if not normalized or "/" in normalized or normalized in {".", ".."}:
raise ValueError("Campaign artifact inventory requires a valid tenant id")
return f"{CAMPAIGN_ARTIFACT_NAMESPACE}/{normalized}/"
def _build_id_for_key(key: str, *, prefix: str) -> str | None:
identity = _artifact_identity(key, prefix=prefix)
return identity[2] if identity is not None else None
def _print_output_storage_key(value: object) -> str | None:
if not isinstance(value, dict):
return None
artifact = value.get("artifact")
if not isinstance(artifact, dict):
return None
key = artifact.get("storage_key")
return str(key) if key else None
def _add_key(
keys: set[str],
value: object,
*,
prefix: str,
allowed_keys: set[str] | None = None,
) -> None:
if value is None:
return
key = str(value)
if key.startswith(prefix) and (allowed_keys is None or key in allowed_keys):
keys.add(key)
def _object_modified_at(info: StorageObjectInfo) -> datetime | None:
return _as_utc(info.modified_at) if info.modified_at is not None else None
def _as_utc(value: datetime) -> datetime:
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)
def _sha256(value: object) -> str:
return hashlib.sha256(str(value).encode("utf-8")).hexdigest()
def _canonical_sha256(value: object) -> str:
return hashlib.sha256(
json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
default=str,
).encode("utf-8")
).hexdigest()
__all__ = [
"CAMPAIGN_ARTIFACT_NAMESPACE",
"CampaignArtifactReconciliationError",
"campaign_artifact_inventory",
"reconcile_campaign_artifacts",
]
@@ -1,7 +1,6 @@
from __future__ import annotations
import fnmatch
import re
import time
from dataclasses import dataclass, field
from enum import StrEnum
@@ -13,6 +12,12 @@ from pydantic import BaseModel, ConfigDict, Field
from govoplan_campaign.backend.campaign.entries import load_campaign_entries
from govoplan_campaign.backend.campaign.template_values import build_template_values
from govoplan_campaign.backend.campaign.models import AttachmentBasePathConfig, AttachmentConfig, Behavior, CampaignConfig, EntryConfig, ZipArchiveConfig, ZipRuleMode
from govoplan_campaign.backend.path_security import (
CampaignPathSecurityError,
assert_logical_relative_path,
is_managed_source,
)
from govoplan_campaign.backend.template_rendering import render_template
class AttachmentScope(StrEnum):
@@ -48,6 +53,17 @@ class AttachmentIssue(BaseModel):
code: str
message: str
behavior: Behavior | None = None
details: dict[str, Any] = Field(default_factory=dict)
class AttachmentPolicyDecision(BaseModel):
model_config = ConfigDict(extra="forbid")
requirement_policy: Behavior
campaign_policy: Behavior
rule_policy: Behavior | None = None
effective_behavior: Behavior
legacy_drop_normalized: bool = False
class ResolvedAttachment(BaseModel):
@@ -77,6 +93,7 @@ class ResolvedAttachment(BaseModel):
zip_entry_names: list[str] = Field(default_factory=list)
status: AttachmentMatchStatus
behavior: Behavior | None = None
missing_policy: AttachmentPolicyDecision | None = None
matches: list[str] = Field(default_factory=list)
issues: list[AttachmentIssue] = Field(default_factory=list)
@@ -140,43 +157,8 @@ def _resolve_path(campaign_file: str | Path, raw_path: str) -> Path:
return (campaign_path.parent / path).resolve()
_DOLLAR_FIELD_PATTERN = re.compile(r"(?<!\\)\$\{(.*?)(?<!\\)\}")
_BRACE_FIELD_PATTERN = re.compile(r"(?<!\\)\{\{\s*(.*?)\s*\}\}")
def _normalize_template_key(raw: str) -> str:
key = raw.strip()
if key.startswith("fields."):
key = key.removeprefix("fields.")
elif key.startswith("local."):
key = "local::" + key.removeprefix("local.")
elif key.startswith("global."):
key = "global::" + key.removeprefix("global.")
if key.startswith("local::") or key.startswith("global::"):
return key
if key.startswith("local:"):
return "local::" + key.removeprefix("local:")
if key.startswith("global:"):
return "global::" + key.removeprefix("global:")
return key
def _render_template(template: str, values: dict[str, Any]) -> str:
def replace(match: re.Match[str]) -> str:
key = _normalize_template_key(match.group(1))
if key in values:
value = values[key]
return "" if value is None else str(value)
return match.group(0)
rendered = _DOLLAR_FIELD_PATTERN.sub(replace, template)
rendered = _BRACE_FIELD_PATTERN.sub(replace, rendered)
return rendered.replace(r"\${", "${").replace(r"\}", "}")
def _rendered_base_dir(config: AttachmentConfig, values: dict[str, Any]) -> str:
rendered = _render_template(config.base_dir, values).strip()
rendered = render_template(config.base_dir, values, keep_missing=True).strip()
return rendered or "."
@@ -223,12 +205,53 @@ def _rule_allows_multiple(config: AttachmentConfig, rendered_file_filter: str) -
return config.allow_multiple or any(char in rendered_file_filter for char in "*?[")
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
_MISSING_BEHAVIOR_STRENGTH = {
Behavior.CONTINUE: 0,
Behavior.WARN: 1,
Behavior.ASK: 2,
Behavior.DROP: 2,
Behavior.BLOCK: 3,
}
def _missing_policy_decision(
campaign_config: CampaignConfig,
config: AttachmentConfig,
) -> AttachmentPolicyDecision:
requirement_policy = (
campaign_config.validation_policy.missing_required_attachment
if config.required
else campaign_config.validation_policy.missing_optional_attachment
)
candidates = [
requirement_policy,
campaign_config.attachments.missing_behavior,
]
if config.missing_behavior is not None:
return config.missing_behavior
if config.required:
return campaign_config.validation_policy.missing_required_attachment
return campaign_config.validation_policy.missing_optional_attachment
candidates.append(config.missing_behavior)
configured = max(
candidates,
key=lambda behavior: _MISSING_BEHAVIOR_STRENGTH[behavior],
)
if Behavior.BLOCK in candidates:
configured = Behavior.BLOCK
elif not config.required and config.missing_behavior == Behavior.CONTINUE:
# An explicitly optional, allowed-empty rule is an expected outcome,
# not an exception to accept. Hard blocking policy still wins above.
configured = Behavior.CONTINUE
elif Behavior.DROP in candidates:
configured = Behavior.DROP
return AttachmentPolicyDecision(
requirement_policy=requirement_policy,
campaign_policy=campaign_config.attachments.missing_behavior,
rule_policy=config.missing_behavior,
effective_behavior=configured,
legacy_drop_normalized=False,
)
def _missing_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
return _missing_policy_decision(campaign_config, config).effective_behavior
def _ambiguous_behavior(campaign_config: CampaignConfig, config: AttachmentConfig) -> Behavior:
@@ -260,7 +283,7 @@ def _attachment_zip_archive(
def _render_zip_filename(archive: ZipArchiveConfig | None, values: dict[str, Any]) -> str | None:
if archive is None:
return None
rendered = _render_template(archive.name or "attachments.zip", values).strip() or "attachments.zip"
rendered = render_template(archive.name or "attachments.zip", values, keep_missing=True).strip() or "attachments.zip"
return rendered if rendered.lower().endswith(".zip") else f"{rendered}.zip"
@@ -369,14 +392,39 @@ def _match_files(directory: Path, file_filter: str, include_subdirs: bool, match
return sorted(path for path in directory.glob(file_filter) if path.is_file())
def _issue_for_missing(config: AttachmentConfig, behavior: Behavior) -> AttachmentIssue:
def _confine_managed_matches(directory: Path, matches: list[Path]) -> tuple[list[Path], bool]:
root = directory.resolve()
confined: list[Path] = []
rejected = False
for match in matches:
try:
resolved = match.resolve()
except (OSError, RuntimeError):
rejected = True
continue
if not resolved.is_relative_to(root):
rejected = True
continue
confined.append(match)
return confined, rejected
def _issue_for_missing(
config: AttachmentConfig,
policy: AttachmentPolicyDecision,
) -> AttachmentIssue:
code = "missing_required_attachment" if config.required else "missing_optional_attachment"
severity = ResolutionSeverity.ERROR if config.required and behavior == Behavior.BLOCK else ResolutionSeverity.WARNING
behavior = policy.effective_behavior
severity = (
ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else
ResolutionSeverity.INFO if behavior in {Behavior.CONTINUE, Behavior.DROP} else ResolutionSeverity.WARNING
)
return AttachmentIssue(
severity=severity,
code=code,
message=f"No file matched attachment filter {config.file_filter!r}",
behavior=behavior,
details={"effective_policy": policy.model_dump(mode="json")},
)
@@ -390,6 +438,31 @@ def _issue_for_ambiguous(config: AttachmentConfig, behavior: Behavior, match_cou
)
def effective_send_without_attachments_behavior(config: CampaignConfig) -> Behavior:
configured = config.attachments.send_without_attachments_behavior or (
Behavior.CONTINUE if config.attachments.send_without_attachments else Behavior.BLOCK
)
# Configured exclusion is already an explicit policy decision. It must not
# be converted into an acceptance prompt that would send the excluded mail.
return configured
def _issue_for_missing_attachment_coverage(behavior: Behavior) -> AttachmentIssue:
messages = {
Behavior.BLOCK: "No attachment file was resolved for this message, and campaign policy blocks sending without attachments.",
Behavior.ASK: "No attachment file was resolved for this message. Confirm the message should still be sent.",
Behavior.DROP: "No attachment file was resolved for this message. Campaign policy excludes messages without attachments.",
Behavior.WARN: "No attachment file was resolved for this message. Campaign policy allows sending with a warning.",
}
return AttachmentIssue(
severity=(ResolutionSeverity.ERROR if behavior == Behavior.BLOCK else ResolutionSeverity.INFO if behavior == Behavior.DROP else ResolutionSeverity.WARNING),
code="missing_attachment_coverage",
message=messages.get(behavior, "No attachment file was resolved for this message."),
behavior=behavior,
details={"effective_behavior": behavior.value},
)
def _resolve_one_config(
*,
campaign_file: str | Path,
@@ -401,23 +474,77 @@ def _resolve_one_config(
match_index: AttachmentMatchIndex | None = None,
) -> ResolvedAttachment:
rendered_base_dir = _rendered_base_dir(config, values)
rendered_file_filter = _render_template(config.file_filter, values)
rendered_file_filter = render_template(config.file_filter, values, keep_missing=True)
directory, selected_base_path = _resolve_attachment_directory(
campaign_file=campaign_file,
campaign_config=campaign_config,
attachment_config=config,
rendered_base_dir=rendered_base_dir,
)
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
allow_multiple = _rule_allows_multiple(config, rendered_file_filter)
issues: list[AttachmentIssue] = []
behavior: Behavior | None = None
managed_source = selected_base_path is not None and is_managed_source(selected_base_path.source)
unsafe_managed_path = False
resolution_failed = False
try:
if managed_source:
assert_logical_relative_path(
rendered_file_filter,
field="rendered managed attachment file_filter",
)
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
matches, rejected = _confine_managed_matches(directory, matches)
if rejected:
matches = []
unsafe_managed_path = True
issues.append(
AttachmentIssue(
severity=ResolutionSeverity.ERROR,
code="managed_attachment_path_escape",
message="A managed attachment match resolved outside its materialized source directory.",
behavior=Behavior.BLOCK,
)
)
else:
matches = _match_files(directory, rendered_file_filter, config.include_subdirs, match_index)
except CampaignPathSecurityError as exc:
matches = []
unsafe_managed_path = True
issues.append(
AttachmentIssue(
severity=ResolutionSeverity.ERROR,
code="unsafe_managed_attachment_path",
message=str(exc),
behavior=Behavior.BLOCK,
)
)
except (OSError, RuntimeError) as exc:
matches = []
resolution_failed = True
issues.append(
AttachmentIssue(
severity=ResolutionSeverity.ERROR,
code="attachment_resolution_failed",
message=f"Attachment source could not be read while resolving filter {config.file_filter!r}.",
behavior=Behavior.BLOCK,
details={"error_type": type(exc).__name__},
)
)
if not matches:
missing_policy: AttachmentPolicyDecision | None = None
if unsafe_managed_path:
status = AttachmentMatchStatus.MISSING
behavior = _missing_behavior(campaign_config, config)
issues.append(_issue_for_missing(config, behavior))
behavior = Behavior.BLOCK
elif resolution_failed:
status = AttachmentMatchStatus.MISSING
behavior = Behavior.BLOCK
elif not matches:
status = AttachmentMatchStatus.MISSING
missing_policy = _missing_policy_decision(campaign_config, config)
behavior = missing_policy.effective_behavior
issues.append(_issue_for_missing(config, missing_policy))
elif len(matches) > 1 and not allow_multiple:
status = AttachmentMatchStatus.AMBIGUOUS
behavior = _ambiguous_behavior(campaign_config, config)
@@ -449,6 +576,7 @@ def _resolve_one_config(
zip_entry_name_template=config.zip_entry_name_template,
status=status,
behavior=behavior,
missing_policy=missing_policy,
matches=[str(path) for path in matches],
issues=issues,
)
@@ -495,6 +623,15 @@ def resolve_entry_attachments(
)
issues = [issue for item in resolved for issue in item.issues]
missing_coverage_behavior = effective_send_without_attachments_behavior(config)
if (
entry.active
and resolved
and missing_coverage_behavior != Behavior.CONTINUE
and sum(len(item.matches) for item in resolved) == 0
and not any(issue.behavior == Behavior.BLOCK for issue in issues)
):
issues.append(_issue_for_missing_attachment_coverage(missing_coverage_behavior))
return EntryAttachmentResolution(
entry_index=entry_index,
entry_id=entry.id,
@@ -0,0 +1,184 @@
from __future__ import annotations
import hashlib
from collections import defaultdict
from dataclasses import dataclass
from pathlib import Path
from govoplan_campaign.backend.campaign.models import (
AttachmentReuseAction,
AttachmentReuseAllowance,
AttachmentReusePolicy,
)
from govoplan_campaign.backend.messages.models import MessageDraft, MessageIssue
@dataclass(frozen=True, slots=True)
class _AttachmentUse:
message: MessageDraft
message_key: str
recipient_key: tuple[str, ...]
source_identity: str
file_name: str
@dataclass(slots=True)
class AttachmentReuseEvaluation:
report: dict[str, object]
issues_by_entry_index: dict[int, list[MessageIssue]]
def evaluate_attachment_reuse(
messages: list[MessageDraft],
*,
policy: AttachmentReusePolicy,
) -> AttachmentReuseEvaluation:
"""Evaluate repeated resolved-file use without exposing source paths.
A use is one resolved file occurrence in one attachment rule. The same
source file can therefore be detected both across built messages and when
two rules add it to one message. Allowed findings remain in the build
protocol; policy violations additionally become recipient-level issues.
"""
uses_by_source: dict[str, list[_AttachmentUse]] = defaultdict(list)
for message in messages:
if not message.active:
continue
message_key = str(message.entry_id or message.entry_index)
recipient_key = _recipient_key(message, fallback=message_key)
for attachment in message.attachments:
for match in attachment.matches:
source_identity = _source_identity(match)
uses_by_source[source_identity].append(
_AttachmentUse(
message=message,
message_key=message_key,
recipient_key=recipient_key,
source_identity=source_identity,
file_name=Path(match).name,
)
)
findings: list[dict[str, object]] = []
issues_by_entry_index: dict[int, list[MessageIssue]] = defaultdict(list)
affected_entry_indexes: set[int] = set()
allowed_count = 0
violation_count = 0
for source_identity, uses in sorted(uses_by_source.items()):
if len(uses) < 2:
continue
fingerprint = hashlib.sha256(source_identity.encode("utf-8")).hexdigest()
message_keys = {item.message_key for item in uses}
recipient_keys = {item.recipient_key for item in uses}
allowed, explanation = _is_allowed(
policy,
message_count=len(message_keys),
recipient_count=len(recipient_keys),
)
disposition = "allowed" if allowed else policy.action.value
finding = {
"file_fingerprint": fingerprint,
"file_name": uses[0].file_name,
"use_count": len(uses),
"message_count": len(message_keys),
"recipient_count": len(recipient_keys),
"disposition": disposition,
"explanation": explanation,
}
findings.append(finding)
if allowed:
allowed_count += 1
continue
violation_count += 1
behavior = _issue_behavior(policy.action)
severity = (
"error" if policy.action == AttachmentReuseAction.BLOCK else "warning"
)
for use in _unique_message_uses(uses):
affected_entry_indexes.add(use.message.entry_index)
issues_by_entry_index[use.message.entry_index].append(
MessageIssue(
severity=severity,
code="duplicate_attachment_reuse",
message=(
f"Attachment {use.file_name!r} is reused {len(uses)} times "
f"across {len(message_keys)} built message(s); the configured "
f"policy requires {disposition}."
),
behavior=behavior,
source="attachments:reuse_policy",
details={
**finding,
"policy": policy.model_dump(mode="json"),
},
)
)
return AttachmentReuseEvaluation(
report={
"contract_version": "1",
"policy": policy.model_dump(mode="json"),
"duplicate_file_count": len(findings),
"allowed_file_count": allowed_count,
"violation_file_count": violation_count,
"affected_message_count": len(affected_entry_indexes),
"findings": findings,
},
issues_by_entry_index=dict(issues_by_entry_index),
)
def _source_identity(value: str) -> str:
return str(Path(value).resolve(strict=False))
def _recipient_key(message: MessageDraft, *, fallback: str) -> tuple[str, ...]:
addresses = message.to or message.bcc or message.cc
normalized = sorted(
{
item.email.strip().casefold()
for item in addresses
if item.email and item.email.strip()
}
)
return tuple(normalized) if normalized else (f"entry:{fallback}",)
def _is_allowed(
policy: AttachmentReusePolicy,
*,
message_count: int,
recipient_count: int,
) -> tuple[bool, str]:
if policy.action == AttachmentReuseAction.ALLOW:
return True, "The campaign policy explicitly allows attachment reuse."
if (
policy.allow_within == AttachmentReuseAllowance.SAME_MESSAGE
and message_count == 1
):
return True, "Reuse is confined to one built message as allowed by policy."
if (
policy.allow_within == AttachmentReuseAllowance.SAME_RECIPIENT
and recipient_count == 1
):
return True, "Reuse is confined to one recipient as allowed by policy."
return False, (
"Reuse crosses the configured allowance and is handled by the "
f"{policy.action.value} policy."
)
def _issue_behavior(action: AttachmentReuseAction) -> str:
if action == AttachmentReuseAction.REVIEW:
return "ask"
return action.value
def _unique_message_uses(uses: list[_AttachmentUse]) -> list[_AttachmentUse]:
unique: dict[int, _AttachmentUse] = {}
for use in uses:
unique.setdefault(use.message.entry_index, use)
return list(unique.values())
@@ -0,0 +1,44 @@
from __future__ import annotations
import copy
from collections.abc import Mapping
DEFAULT_COPY_OPTIONS: dict[str, bool] = {
"include_recipients": True,
"include_files": True,
"include_shares": False,
"include_policies": True,
"include_mail_profile": True,
}
def campaign_copy_configuration(
source: Mapping[str, object],
options: Mapping[str, object],
) -> dict[str, object]:
"""Return an editable configuration copy without operational evidence."""
selected = {**DEFAULT_COPY_OPTIONS, **dict(options)}
raw_json = copy.deepcopy(dict(source))
if not selected["include_recipients"]:
raw_json["recipients"] = {}
raw_json["entries"] = {"inline": [], "imports": []}
if not selected["include_files"]:
raw_json["attachments"] = {}
entries = raw_json.get("entries")
if isinstance(entries, dict):
inline = entries.get("inline")
if isinstance(inline, list):
for entry in inline:
if isinstance(entry, dict):
entry["attachments"] = []
entry["combine_attachments"] = True
if not selected["include_policies"]:
raw_json["validation_policy"] = {}
if not selected["include_mail_profile"]:
raw_json["server"] = {}
return raw_json
__all__ = ["DEFAULT_COPY_OPTIONS", "campaign_copy_configuration"]
@@ -44,6 +44,7 @@ def _parse_scalar_for_target(target: str, value: Any) -> Any:
"merge_reply_to",
"merge_bounce_to",
"merge_disposition_notification_to",
"merge_postbox_targets",
"combine_to",
"combine_cc",
"combine_bcc",
@@ -0,0 +1,276 @@
from __future__ import annotations
import hashlib
import hmac
import json
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignSchedule,
CampaignShare,
CampaignVersion,
)
from govoplan_core.auth import ApiPrincipal, has_scope
POLICY_ID = "campaign.lifecycle"
POLICY_VERSION = "2"
_ACTIVE_QUEUE_STATES = {"queued", "sending"}
_ACTIVE_SEND_STATES = {"queued", "claimed", "sending", "outcome_unknown"}
_ACTIVE_POSTBOX_STATES = {"pending", "delivering", "outcome_unknown"}
_ACTIVE_PRINT_STATES = {"ready", "accepting"}
_ACTIVE_IMAP_STATES = {"pending", "appending", "outcome_unknown"}
@dataclass(frozen=True)
class LifecycleDecision:
allowed: bool
reason: str | None = None
def as_dict(self) -> dict[str, Any]:
return {"allowed": self.allowed, "reason": self.reason}
def _timestamp(value: datetime | None) -> str | None:
return value.isoformat() if value is not None else None
def _canonical_hash(value: object) -> str:
return hashlib.sha256(
json.dumps(
value,
ensure_ascii=True,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
).hexdigest()
def _protected_version(version: CampaignVersion) -> bool:
return any(
value is not None
for value in (
version.locked_at,
version.user_lock_state,
version.published_at,
version.execution_snapshot_at,
)
)
def _active_delivery(job: CampaignJob) -> bool:
return any(
(
job.queue_status in _ACTIVE_QUEUE_STATES,
job.send_status in _ACTIVE_SEND_STATES,
job.postbox_status in _ACTIVE_POSTBOX_STATES,
job.print_status in _ACTIVE_PRINT_STATES,
job.imap_status in _ACTIVE_IMAP_STATES,
)
)
def campaign_lifecycle_policy(
session: Session,
*,
campaign: Campaign,
principal: ApiPrincipal,
version_id: str | None = None,
) -> dict[str, Any]:
versions = (
session.query(CampaignVersion)
.filter(CampaignVersion.campaign_id == campaign.id)
.order_by(CampaignVersion.version_number.asc())
.all()
)
jobs = (
session.query(CampaignJob)
.filter(CampaignJob.campaign_id == campaign.id)
.order_by(CampaignJob.id.asc())
.all()
)
shares = (
session.query(CampaignShare)
.filter(
CampaignShare.campaign_id == campaign.id,
CampaignShare.revoked_at.is_(None),
)
.order_by(CampaignShare.id.asc())
.all()
)
schedules = (
session.query(CampaignSchedule)
.filter(CampaignSchedule.campaign_id == campaign.id)
.order_by(CampaignSchedule.id.asc())
.all()
)
selected_version = next(
(version for version in versions if version.id == version_id),
None,
)
snapshot = {
"policy_id": POLICY_ID,
"policy_version": POLICY_VERSION,
"campaign": {
"id": campaign.id,
"status": campaign.status,
"current_version_id": campaign.current_version_id,
"settings_sha256": _canonical_hash(campaign.settings or {}),
"mail_profile_policy_sha256": _canonical_hash(
campaign.mail_profile_policy or {}
),
"updated_at": _timestamp(campaign.updated_at),
},
"versions": [
{
"id": version.id,
"version_number": version.version_number,
"edit_revision": version.edit_revision,
"workflow_state": version.workflow_state,
"locked_at": _timestamp(version.locked_at),
"user_lock_state": version.user_lock_state,
"published_at": _timestamp(version.published_at),
"execution_snapshot_at": _timestamp(version.execution_snapshot_at),
"archived_at": _timestamp(version.archived_at),
"configuration_sha256": _canonical_hash(version.raw_json or {}),
"updated_at": _timestamp(version.updated_at),
}
for version in versions
],
"jobs": [
{
"id": job.id,
"queue_status": job.queue_status,
"send_status": job.send_status,
"postbox_status": job.postbox_status,
"print_status": job.print_status,
"imap_status": job.imap_status,
"updated_at": _timestamp(job.updated_at),
}
for job in jobs
],
"active_shares": [
{
"id": share.id,
"target_type": share.target_type,
"target_id": share.target_id,
"permission": share.permission,
"updated_at": _timestamp(share.updated_at),
}
for share in shares
],
"schedules": [
{
"id": schedule.id,
"active": schedule.active,
"resource_revision": schedule.resource_revision,
"next_fire_at": _timestamp(schedule.next_fire_at),
"occurrence_count": schedule.occurrence_count,
"updated_at": _timestamp(schedule.updated_at),
}
for schedule in schedules
],
"selected_version_id": version_id,
}
token = _canonical_hash(snapshot)
active_delivery = any(_active_delivery(job) for job in jobs)
protected_versions = any(_protected_version(version) for version in versions)
active_schedules = any(schedule.active for schedule in schedules)
archive = LifecycleDecision(True)
if not has_scope(principal, "campaigns:campaign:archive"):
archive = LifecycleDecision(False, "Missing campaign archive permission.")
elif campaign.status in {"archived", "deleted"}:
archive = LifecycleDecision(False, "The campaign is already archived or deleted.")
elif active_delivery:
archive = LifecycleDecision(
False,
"Active or uncertain delivery must be resolved before archiving.",
)
elif active_schedules:
archive = LifecycleDecision(
False,
"Pause active Campaign schedules before archiving.",
)
delete = LifecycleDecision(True)
if not has_scope(principal, "campaigns:campaign:delete"):
delete = LifecycleDecision(False, "Missing campaign delete permission.")
elif campaign.status != "draft":
delete = LifecycleDecision(False, "Only untouched draft campaigns can be deleted.")
elif jobs:
delete = LifecycleDecision(
False,
"Campaigns with built or delivery jobs must be archived instead of deleted.",
)
elif protected_versions:
delete = LifecycleDecision(
False,
"Audit-relevant campaign versions must be archived instead of deleted.",
)
elif shares:
delete = LifecycleDecision(
False,
"Revoke active campaign shares before deleting the untouched draft.",
)
elif schedules:
delete = LifecycleDecision(
False,
"Campaigns with schedule evidence must be archived instead of deleted.",
)
copy = LifecycleDecision(True)
if not has_scope(principal, "campaigns:campaign:copy"):
copy = LifecycleDecision(False, "Missing campaign copy permission.")
elif version_id is not None and selected_version is None:
copy = LifecycleDecision(False, "The selected source version does not exist.")
archive_version = LifecycleDecision(True)
if not has_scope(principal, "campaigns:campaign:archive"):
archive_version = LifecycleDecision(False, "Missing campaign archive permission.")
elif version_id is None or selected_version is None:
archive_version = LifecycleDecision(False, "Select a historical campaign version.")
elif selected_version.id == campaign.current_version_id:
archive_version = LifecycleDecision(False, "The current campaign version cannot be archived.")
elif selected_version.archived_at is not None:
archive_version = LifecycleDecision(False, "The historical version is already archived.")
return {
"policy_id": POLICY_ID,
"policy_version": POLICY_VERSION,
"state_token": token,
"actions": {
"archive_campaign": archive.as_dict(),
"delete_campaign": delete.as_dict(),
"copy_campaign": copy.as_dict(),
"archive_version": archive_version.as_dict(),
},
"provenance": {
"source": "built_in",
"rules": (
"permission",
"campaign_state",
"retained_evidence",
"active_delivery",
"scheduled_automation",
"optimistic_concurrency",
),
"evidence_retention": "Versions, schedule occurrences, delivery outcomes, reports, and audit records are never deleted by archival.",
},
}
def assert_lifecycle_state_token(actual: str, expected: str) -> None:
if not hmac.compare_digest(actual, expected):
raise ValueError(
"Campaign state changed after this action was prepared. Reload and review the lifecycle decision again."
)
@@ -7,6 +7,7 @@ from typing import Any
from jsonschema import Draft202012Validator, FormatChecker
from .mail_profile_boundary import assert_campaign_uses_mail_profile_reference
from .models import CampaignConfig
@@ -83,6 +84,7 @@ def load_campaign_config(
schema_path: str | Path | None = None,
) -> CampaignConfig:
data = load_campaign_json(path)
assert_campaign_uses_mail_profile_reference(data)
if validate_schema:
validate_against_schema(data, schema_path=schema_path)
return CampaignConfig.model_validate(data)
@@ -0,0 +1,530 @@
from __future__ import annotations
import copy
import hashlib
from typing import Any
CAMPAIGN_MAIL_SERVER_KEYS = frozenset(
{
"mail_profile_id",
"smtp_server_id",
"smtp_credential_id",
"imap_server_id",
"imap_credential_id",
}
)
CAMPAIGN_CLIENT_EDITOR_STATE_KEYS = frozenset(
{"created_from", "field_overrides", "opt_ins"}
)
CAMPAIGN_OPT_IN_KEYS = frozenset(
{"campaign_address_suggestions", "remember_used_addresses", "inline_guidance"}
)
CAMPAIGN_REVIEW_STATE_KEYS = frozenset(
{
"build_token",
"inspection_complete",
"reviewed_message_keys",
"issue_decisions",
"updated_at",
"updated_by_user_id",
}
)
CAMPAIGN_APPROVAL_GATE_KEYS = frozenset(
{
"request_id",
"request_revision",
"subject_version",
"subject_digest",
"requested_at",
"requested_by_user_id",
}
)
class CampaignMailProfileBoundaryError(ValueError):
"""Raised when campaign JSON owns mail transport configuration.
SMTP/IMAP endpoints and credentials are Mail-module data. Campaign JSON
may select Mail-owned profile, server, and credential identifiers, but it
must never copy or override transport configuration.
"""
def campaign_review_reference(version_id: str, build_token: Any) -> str | None:
"""A public concurrency reference, not a raw diagnostic/build claim token."""
token = str(build_token or "").strip()
return hashlib.sha256(f"campaign-review:{version_id}:{token}".encode()).hexdigest() if token else None
def _validated_opt_ins(value: Any) -> dict[str, bool]:
if not isinstance(value, dict) or any(
key not in CAMPAIGN_OPT_IN_KEYS for key in value
):
raise CampaignMailProfileBoundaryError(
"Campaign editor opt_ins contains unsupported fields"
)
if any(not isinstance(item, bool) for item in value.values()):
raise CampaignMailProfileBoundaryError(
"Campaign editor opt_ins values must be booleans"
)
return copy.deepcopy(value)
def _is_valid_field_override(key: Any, value: Any) -> bool:
return (
isinstance(key, str)
and bool(key.strip())
and len(key) <= 256
and isinstance(value, bool)
)
def _validated_field_overrides(value: Any) -> dict[str, bool]:
if not isinstance(value, dict) or len(value) > 10_000:
raise CampaignMailProfileBoundaryError(
"Campaign editor field_overrides must be a bounded object"
)
if any(not _is_valid_field_override(key, item) for key, item in value.items()):
raise CampaignMailProfileBoundaryError(
"Campaign editor field_overrides must map short field names to booleans"
)
return copy.deepcopy(value)
def _validated_required_string(value: Any, *, max_length: int, error: str) -> str:
if not isinstance(value, str) or not value.strip() or len(value) > max_length:
raise CampaignMailProfileBoundaryError(error)
return value.strip()
def validate_campaign_editor_state(
value: dict[str, Any] | None,
*,
allow_server_review_state: bool = False,
) -> dict[str, Any]:
"""Validate the bounded Campaign-owned UI metadata contract.
Arbitrary editor metadata would be a second, weakly typed persistence and
response channel for Mail credentials. Review evidence is server-owned and
cannot be supplied through ordinary version create/update requests.
"""
if value is None:
return {}
if not isinstance(value, dict):
raise CampaignMailProfileBoundaryError(
"Campaign editor state must be an object"
)
allowed = set(CAMPAIGN_CLIENT_EDITOR_STATE_KEYS)
if allow_server_review_state:
allowed.add("review_send")
allowed.add("approval_gate")
if any(key not in allowed for key in value):
raise CampaignMailProfileBoundaryError(
"Campaign editor state contains unsupported or transport-owned fields"
)
result: dict[str, Any] = {}
if "created_from" in value:
result["created_from"] = _validated_required_string(
value["created_from"],
max_length=128,
error="Campaign editor created_from must be a short string",
)
if "opt_ins" in value:
result["opt_ins"] = _validated_opt_ins(value["opt_ins"])
if "field_overrides" in value:
result["field_overrides"] = _validated_field_overrides(value["field_overrides"])
if "review_send" in value:
result["review_send"] = _validated_server_review_state(value["review_send"])
if "approval_gate" in value:
result["approval_gate"] = _validated_server_approval_gate(
value["approval_gate"]
)
return result
def public_campaign_editor_state(
value: Any,
*,
include_diagnostics: bool = False,
) -> dict[str, Any]:
"""Return only known non-secret UI metadata from current or legacy rows."""
if not isinstance(value, dict):
return {}
result: dict[str, Any] = {}
for key in CAMPAIGN_CLIENT_EDITOR_STATE_KEYS:
if key not in value:
continue
try:
result.update(validate_campaign_editor_state({key: value[key]}))
except CampaignMailProfileBoundaryError:
continue
if "review_send" in value:
try:
review_state = _validated_server_review_state(value["review_send"])
if not include_diagnostics:
review_state.pop("build_token", None)
review_state["issue_decisions"] = [
{
key: item[key]
for key in (
"job_id",
"review_key",
"decision",
"reason",
"actor_user_id",
"decided_at",
"issue_codes",
)
if key in item
}
for item in review_state.get("issue_decisions", [])
]
result["review_send"] = review_state
except CampaignMailProfileBoundaryError:
pass
if "approval_gate" in value:
try:
result["approval_gate"] = _validated_server_approval_gate(
value["approval_gate"]
)
except CampaignMailProfileBoundaryError:
pass
return result
def campaign_editor_state_for_edit(value: Any) -> dict[str, Any]:
"""Copy only client-owned safe metadata into a new editable version."""
state = public_campaign_editor_state(value)
state.pop("review_send", None)
state.pop("approval_gate", None)
return state
def campaign_editor_state_with_client_update(
stored: Any, client_state: dict[str, Any]
) -> dict[str, Any]:
"""Replace client metadata without accepting or erasing server evidence.
Read responses contain review and approval state, but ordinary editor saves
may only supply client-owned fields. Their omission must not delete trusted
server evidence; content/build invalidation remains owned by its lifecycle.
"""
result = validate_campaign_editor_state(client_state)
server_state = public_campaign_editor_state(stored, include_diagnostics=True)
for key in ("review_send", "approval_gate"):
if key in server_state:
result[key] = server_state[key]
return result
def _validated_server_approval_gate(value: Any) -> dict[str, Any]:
if not isinstance(value, dict) or any(
key not in CAMPAIGN_APPROVAL_GATE_KEYS for key in value
):
raise CampaignMailProfileBoundaryError("Campaign approval gate is invalid")
required_strings = (
"request_id",
"subject_version",
"subject_digest",
"requested_at",
)
if any(
not isinstance(value.get(key), str) or not str(value.get(key)).strip()
for key in required_strings
):
raise CampaignMailProfileBoundaryError(
"Campaign approval gate references are invalid"
)
digest = str(value["subject_digest"])
if len(digest) != 64 or any(
character not in "0123456789abcdef" for character in digest
):
raise CampaignMailProfileBoundaryError(
"Campaign approval gate digest is invalid"
)
revision = value.get("request_revision")
if not isinstance(revision, int) or revision < 1:
raise CampaignMailProfileBoundaryError(
"Campaign approval gate revision is invalid"
)
requested_by = value.get("requested_by_user_id")
if requested_by is not None and not isinstance(requested_by, str):
raise CampaignMailProfileBoundaryError(
"Campaign approval gate actor is invalid"
)
return copy.deepcopy(value)
def _is_valid_reviewed_message_key(value: Any) -> bool:
return isinstance(value, str) and bool(value.strip()) and len(value) <= 512
def _validated_reviewed_message_keys(value: Any) -> list[str]:
if not isinstance(value, list) or len(value) > 100_000:
raise CampaignMailProfileBoundaryError(
"Campaign reviewed message keys are invalid"
)
if any(not _is_valid_reviewed_message_key(key) for key in value):
raise CampaignMailProfileBoundaryError(
"Campaign reviewed message keys are invalid"
)
return list(dict.fromkeys(key.strip() for key in value))
def _validated_review_actor(value: Any) -> str | None:
if value is not None and (not isinstance(value, str) or len(value) > 256):
raise CampaignMailProfileBoundaryError("Campaign review actor is invalid")
return value
def _validated_issue_decisions(value: Any) -> list[dict[str, Any]]:
if not isinstance(value, list) or len(value) > 100_000:
raise CampaignMailProfileBoundaryError(
"Campaign review issue decisions are invalid"
)
result: list[dict[str, Any]] = []
allowed_keys = {
"job_id",
"review_key",
"decision",
"reason",
"actor_user_id",
"decided_at",
"build_token",
"message_sha256",
"issue_fingerprint",
"issue_codes",
}
for item in value:
if not isinstance(item, dict) or any(key not in allowed_keys for key in item):
raise CampaignMailProfileBoundaryError(
"Campaign review issue decision is invalid"
)
normalized = {
"job_id": _validated_required_string(
item.get("job_id"),
max_length=36,
error="Campaign review decision job is invalid",
),
"review_key": _validated_required_string(
item.get("review_key"),
max_length=512,
error="Campaign review decision key is invalid",
),
"decision": _validated_required_string(
item.get("decision"),
max_length=30,
error="Campaign review decision is invalid",
),
"reason": item.get("reason"),
"actor_user_id": _validated_review_actor(item.get("actor_user_id")),
"decided_at": _validated_required_string(
item.get("decided_at"),
max_length=128,
error="Campaign review decision timestamp is invalid",
),
"build_token": _validated_required_string(
item.get("build_token"),
max_length=256,
error="Campaign review decision build token is invalid",
),
"message_sha256": item.get("message_sha256"),
"issue_fingerprint": _validated_required_string(
item.get("issue_fingerprint"),
max_length=64,
error="Campaign review issue fingerprint is invalid",
),
"issue_codes": item.get("issue_codes"),
}
if normalized["decision"] != "accept":
raise CampaignMailProfileBoundaryError(
"Campaign review decision outcome is invalid"
)
if normalized["reason"] is not None and (
not isinstance(normalized["reason"], str)
or len(normalized["reason"]) > 4_000
):
raise CampaignMailProfileBoundaryError(
"Campaign review decision reason is invalid"
)
if normalized["message_sha256"] is not None and (
not isinstance(normalized["message_sha256"], str)
or len(normalized["message_sha256"]) > 64
):
raise CampaignMailProfileBoundaryError(
"Campaign review decision message hash is invalid"
)
if (
not isinstance(normalized["issue_codes"], list)
or len(normalized["issue_codes"]) > 100
or any(
not isinstance(code, str) or not code or len(code) > 100
for code in normalized["issue_codes"]
)
):
raise CampaignMailProfileBoundaryError(
"Campaign review decision issue codes are invalid"
)
result.append(normalized)
return result
def _validated_server_review_state(value: Any) -> dict[str, Any]:
if not isinstance(value, dict) or any(
key not in CAMPAIGN_REVIEW_STATE_KEYS for key in value
):
raise CampaignMailProfileBoundaryError(
"Campaign review editor state is invalid"
)
build_token = value.get("build_token")
inspected = value.get("inspection_complete")
keys = value.get("reviewed_message_keys", [])
issue_decisions = value.get("issue_decisions", [])
updated_at = value.get("updated_at")
updated_by = value.get("updated_by_user_id")
validated_build_token = _validated_required_string(
build_token,
max_length=256,
error="Campaign review build token is invalid",
)
if not isinstance(inspected, bool):
raise CampaignMailProfileBoundaryError(
"Campaign review completion state is invalid"
)
validated_keys = _validated_reviewed_message_keys(keys)
validated_decisions = _validated_issue_decisions(issue_decisions)
validated_updated_at = _validated_required_string(
updated_at,
max_length=128,
error="Campaign review timestamp is invalid",
)
validated_updated_by = _validated_review_actor(updated_by)
return {
"build_token": validated_build_token,
"inspection_complete": inspected,
"reviewed_message_keys": validated_keys,
"issue_decisions": validated_decisions,
"updated_at": validated_updated_at,
"updated_by_user_id": validated_updated_by,
}
def campaign_mail_profile_id(raw_json: dict[str, Any] | None) -> str | None:
server = raw_json.get("server") if isinstance(raw_json, dict) else None
if not isinstance(server, dict):
return None
value = server.get("mail_profile_id")
if not isinstance(value, str):
return None
normalized = value.strip()
return normalized or None
def campaign_mail_resource_ids(
raw_json: dict[str, Any] | None,
) -> dict[str, str | None]:
server = raw_json.get("server") if isinstance(raw_json, dict) else None
if not isinstance(server, dict):
return {
"mail_profile_id": None,
"smtp_server_id": None,
"smtp_credential_id": None,
"imap_server_id": None,
"imap_credential_id": None,
}
return {
key: (
value.strip()
if isinstance((value := server.get(key)), str) and value.strip()
else None
)
for key in CAMPAIGN_MAIL_SERVER_KEYS
}
def campaign_mail_profile_boundary_violations(
raw_json: dict[str, Any] | None,
) -> tuple[str, ...]:
server = raw_json.get("server") if isinstance(raw_json, dict) else None
if not isinstance(server, dict):
return ()
violations = [
f"/server/{key}"
for key in sorted(server)
if key not in CAMPAIGN_MAIL_SERVER_KEYS
]
for key in CAMPAIGN_MAIL_SERVER_KEYS:
if key not in server:
continue
value = server[key]
if not isinstance(value, str) or not value.strip():
violations.append(f"/server/{key}")
references = campaign_mail_resource_ids(raw_json)
if references["mail_profile_id"] is None and any(
references[key] for key in references if key != "mail_profile_id"
):
violations.append("/server/mail_profile_id")
for protocol in ("smtp", "imap"):
if (
references[f"{protocol}_credential_id"]
and not references[f"{protocol}_server_id"]
):
violations.append(f"/server/{protocol}_server_id")
return tuple(violations)
def assert_campaign_uses_mail_profile_reference(
raw_json: dict[str, Any] | None,
*,
require_profile: bool = False,
) -> None:
violations = campaign_mail_profile_boundary_violations(raw_json)
if violations:
fields = ", ".join(violations)
raise CampaignMailProfileBoundaryError(
"Campaign JSON may only reference Mail-owned profiles, servers, and credentials; "
f"remove campaign-local SMTP/IMAP settings or invalid references ({fields}), "
"select authorized Mail resources, and save a new campaign version."
)
if require_profile and campaign_mail_profile_id(raw_json) is None:
raise CampaignMailProfileBoundaryError(
"Campaign delivery requires server.mail_profile_id. Select an authorized, active "
"profile from the Mail module and validate the campaign again."
)
def public_campaign_mail_server(raw_json: dict[str, Any] | None) -> dict[str, str]:
"""Return the complete public/persisted Campaign-to-Mail contract."""
return {
key: value
for key, value in campaign_mail_resource_ids(raw_json).items()
if value
}
def campaign_mail_references_unchanged(
current: dict[str, Any] | None, candidate: dict[str, Any] | None
) -> bool:
"""Recognize an unchanged public selection, never client-owned transport."""
return (
isinstance(candidate, dict)
and not campaign_mail_profile_boundary_violations(candidate)
and candidate.get("server", {}) == public_campaign_mail_server(current)
)
def campaign_preserves_legacy_mail_settings(
current: dict[str, Any] | None, candidate: dict[str, Any] | None
) -> bool:
"""Keep stored legacy transport inert without accepting it from a client."""
return bool(campaign_mail_profile_boundary_violations(current)) and campaign_mail_references_unchanged(current, candidate)
+298 -54
View File
@@ -6,8 +6,6 @@ from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from govoplan_core.mail.config import ImapConfig, ImapServerConfig, SmtpConfig, SmtpServerConfig, TransportCredentials, TransportSecurity
class StrictModel(BaseModel):
model_config = ConfigDict(extra="forbid", populate_by_name=True)
@@ -24,7 +22,9 @@ class FieldType(StrEnum):
INTEGER = "integer"
DOUBLE = "double"
DATE = "date"
PASSWORD = "password"
PASSWORD = "password" # noqa: S105 # nosec B105 - field type vocabulary.
ORGANIZATION_UNIT = "organization_unit"
ORGANIZATION_FUNCTION = "organization_function"
class RecipientType(StrEnum):
@@ -76,6 +76,14 @@ class ZipPasswordScope(StrEnum):
GLOBAL = "global"
class ZipPasswordDeliveryChannel(StrEnum):
SEPARATE_MAIL = "separate_mail"
SMS = "sms"
LETTER = "letter"
PHONE = "phone"
IN_PERSON = "in_person"
class ZipPasswordMode(StrEnum):
NONE = "none"
DIRECT = "direct"
@@ -91,6 +99,132 @@ class BuildStatus(StrEnum):
class SendStatus(StrEnum):
DRAFT = "draft"
QUEUED = "queued"
SKIPPED = "skipped"
class DeliveryChannelPolicy(StrEnum):
MAIL = "mail"
POSTBOX = "postbox"
PRINT = "print"
MAIL_AND_POSTBOX = "mail_and_postbox"
MAIL_THEN_POSTBOX = "mail_then_postbox"
POSTBOX_THEN_MAIL = "postbox_then_mail"
MAIL_THEN_PRINT = "mail_then_print"
POSTBOX_THEN_PRINT = "postbox_then_print"
@property
def uses_mail(self) -> bool:
return self in {
DeliveryChannelPolicy.MAIL,
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
DeliveryChannelPolicy.MAIL_THEN_PRINT,
}
@property
def uses_postbox(self) -> bool:
return self in {
DeliveryChannelPolicy.POSTBOX,
DeliveryChannelPolicy.MAIL_AND_POSTBOX,
DeliveryChannelPolicy.MAIL_THEN_POSTBOX,
DeliveryChannelPolicy.POSTBOX_THEN_MAIL,
DeliveryChannelPolicy.POSTBOX_THEN_PRINT,
}
@property
def uses_print(self) -> bool:
return self in {
DeliveryChannelPolicy.PRINT,
DeliveryChannelPolicy.MAIL_THEN_PRINT,
DeliveryChannelPolicy.POSTBOX_THEN_PRINT,
}
class PostboxTargetMode(StrEnum):
DIRECT = "direct"
DERIVED = "derived"
class PostboxTargetMatch(StrEnum):
ID = "id"
SLUG = "slug"
class PostboxTargetConfig(StrictModel):
id: str = Field(min_length=1, max_length=120)
mode: PostboxTargetMode = PostboxTargetMode.DIRECT
label: str | None = Field(default=None, max_length=500)
postbox_id: str | None = Field(default=None, max_length=36)
address_key: str | None = Field(default=None, max_length=500)
template_id: str | None = Field(default=None, max_length=36)
organization_unit_id: str | None = Field(default=None, max_length=36)
organization_unit_field: str | None = Field(default=None, max_length=255)
organization_unit_match: PostboxTargetMatch = PostboxTargetMatch.ID
function_id: str | None = Field(default=None, max_length=36)
function_field: str | None = Field(default=None, max_length=255)
function_match: PostboxTargetMatch = PostboxTargetMatch.ID
context_key: str | None = Field(default=None, max_length=255)
context_field: str | None = Field(default=None, max_length=255)
@model_validator(mode="after")
def validate_target_shape(self) -> "PostboxTargetConfig":
direct_values = [self.postbox_id, self.address_key]
if self.mode == PostboxTargetMode.DIRECT:
if sum(bool(value) for value in direct_values) != 1:
raise ValueError(
"A direct Postbox target requires exactly one postbox_id "
"or address_key."
)
if any(
(
self.template_id,
self.organization_unit_id,
self.organization_unit_field,
self.function_id,
self.function_field,
self.context_key,
self.context_field,
)
):
raise ValueError(
"A direct Postbox target cannot contain derived target fields."
)
return self
if any(direct_values):
raise ValueError(
"A derived Postbox target cannot contain postbox_id or address_key."
)
if not self.template_id:
raise ValueError("A derived Postbox target requires template_id.")
if bool(self.organization_unit_id) == bool(self.organization_unit_field):
raise ValueError(
"A derived Postbox target requires exactly one fixed or "
"field-derived organization unit."
)
if bool(self.function_id) == bool(self.function_field):
raise ValueError(
"A derived Postbox target requires exactly one fixed or "
"field-derived function."
)
if self.context_key and self.context_field:
raise ValueError(
"A derived Postbox target may use a fixed context or a context "
"field, not both."
)
return self
class PrintTargetConfig(StrictModel):
channel: Literal["postal", "internal_mail"]
target: str = Field(min_length=1, max_length=4000)
target_key: str = Field(min_length=1, max_length=500)
contact_point_id: str | None = Field(default=None, max_length=36)
locale: str | None = Field(default=None, max_length=35)
decision_provenance: dict[str, Any] = Field(default_factory=dict)
class CampaignMeta(StrictModel):
@@ -108,58 +242,18 @@ class FieldDefinition(StrictModel):
can_override: bool = True
class MailServerCredentials(StrictModel):
smtp: TransportCredentials = Field(default_factory=TransportCredentials)
imap: TransportCredentials = Field(default_factory=TransportCredentials)
class MailProfileCapabilities(StrictModel):
smtp_available: bool = False
imap_available: bool = False
class ServerConfig(StrictModel):
mail_profile_id: str | None = None
inherit_smtp_credentials: bool = True
inherit_imap_credentials: bool = True
smtp: SmtpServerConfig | None = None
imap: ImapServerConfig | None = None
credentials: MailServerCredentials = Field(default_factory=MailServerCredentials)
@model_validator(mode="before")
@classmethod
def normalize_legacy_credentials(cls, value: Any) -> Any:
if not isinstance(value, dict):
return value
data = dict(value)
credentials = data.get("credentials") if isinstance(data.get("credentials"), dict) else {}
credentials = {key: dict(item) for key, item in credentials.items() if isinstance(item, dict)}
for protocol in ("smtp", "imap"):
transport = data.get(protocol)
if not isinstance(transport, dict):
continue
next_transport = dict(transport)
next_credentials = dict(credentials.get(protocol) or {})
for field in ("username", "password"):
if field in next_transport and field not in next_credentials:
next_credentials[field] = next_transport[field]
next_transport.pop(field, None)
next_transport.pop("enabled", None)
data[protocol] = next_transport
if next_credentials:
credentials[protocol] = next_credentials
if credentials:
data["credentials"] = credentials
return data
def runtime_smtp_config(self) -> SmtpConfig | None:
if self.smtp is None:
return None
payload = self.smtp.model_dump(mode="json")
payload.update(self.credentials.smtp.model_dump(mode="json", exclude_none=True))
return SmtpConfig.model_validate(payload)
def runtime_imap_config(self) -> ImapConfig | None:
if self.imap is None:
return None
payload = self.imap.model_dump(mode="json")
payload.update(self.credentials.imap.model_dump(mode="json", exclude_none=True))
return ImapConfig.model_validate(payload)
smtp_server_id: str | None = None
smtp_credential_id: str | None = None
imap_server_id: str | None = None
imap_credential_id: str | None = None
profile_capabilities: MailProfileCapabilities = Field(default_factory=MailProfileCapabilities)
class RecipientConfig(StrictModel):
@@ -263,6 +357,13 @@ class ZipArchiveConfig(StrictModel):
password_field: str | None = None
password_scope: ZipPasswordScope = ZipPasswordScope.LOCAL
method: ZipMethod = ZipMethod.AES
password_delivery_channel: ZipPasswordDeliveryChannel = (
ZipPasswordDeliveryChannel.SEPARATE_MAIL
)
legacy_zipcrypto_acknowledged: bool = False
legacy_zipcrypto_reason: str | None = Field(default=None, max_length=1000)
legacy_zipcrypto_acknowledged_by: str | None = Field(default=None, max_length=255)
legacy_zipcrypto_acknowledged_at: str | None = Field(default=None, max_length=80)
# Compatibility fields for campaigns created by the first single-archive
# implementation. New WebUI campaigns use password_enabled/field/scope.
@@ -290,6 +391,20 @@ class ZipArchiveConfig(StrictModel):
normalized["password_scope"] = ZipPasswordScope.LOCAL.value
return normalized
@model_validator(mode="after")
def validate_legacy_zipcrypto_acknowledgement(self) -> "ZipArchiveConfig":
if self.method != ZipMethod.ZIP_STANDARD:
return self
if not self.legacy_zipcrypto_acknowledged:
raise ValueError(
"Legacy ZipCrypto requires explicit acknowledgement of its weak encryption"
)
if len((self.legacy_zipcrypto_reason or "").strip()) < 10:
raise ValueError(
"Legacy ZipCrypto requires an acknowledgement reason of at least 10 characters"
)
return self
class ZipCollectionConfig(StrictModel):
enabled: bool = False
@@ -382,6 +497,48 @@ class AttachmentBasePathConfig(StrictModel):
source: str | None = None
class ResidualFileMode(StrEnum):
NONE = "none"
REPORT = "report"
ATTACH = "attach"
class AttachmentReuseAction(StrEnum):
ALLOW = "allow"
WARN = "warn"
REVIEW = "review"
BLOCK = "block"
class AttachmentReuseAllowance(StrEnum):
NONE = "none"
SAME_RECIPIENT = "same_recipient"
SAME_MESSAGE = "same_message"
class AttachmentReusePolicy(StrictModel):
action: AttachmentReuseAction = AttachmentReuseAction.ALLOW
allow_within: AttachmentReuseAllowance = AttachmentReuseAllowance.NONE
class ResidualFileDispositionConfig(StrictModel):
mode: ResidualFileMode = ResidualFileMode.NONE
recipient: RecipientConfig | None = None
subject: str = "Unassigned files in campaign {{local:campaign_name}}"
text: str = (
"The campaign build found {{local:residual_file_count}} file(s) that "
"were not assigned to a recipient.\n\n{{local:residual_file_list}}"
)
@model_validator(mode="after")
def require_recipient_for_routing(self) -> "ResidualFileDispositionConfig":
if self.mode != ResidualFileMode.NONE and self.recipient is None:
raise ValueError(
"Residual-file report or attachment routing requires a recipient."
)
return self
class AttachmentConfig(StrictModel):
id: str | None = None
label: str | None = None
@@ -418,10 +575,25 @@ class AttachmentsConfig(StrictModel):
base_paths: list[AttachmentBasePathConfig] = Field(default_factory=list)
allow_individual: bool = False
send_without_attachments: bool = True
send_without_attachments_behavior: Behavior | None = None
zip: ZipCollectionConfig = Field(default_factory=ZipCollectionConfig)
global_: list[AttachmentConfig] = Field(default_factory=list, alias="global")
missing_behavior: Behavior = Behavior.ASK
missing_behavior: Behavior = Behavior.WARN
ambiguous_behavior: Behavior = Behavior.ASK
reuse_policy: AttachmentReusePolicy = Field(
default_factory=AttachmentReusePolicy
)
residual_files: ResidualFileDispositionConfig = Field(
default_factory=ResidualFileDispositionConfig
)
@model_validator(mode="after")
def normalize_send_without_attachments_behavior(self) -> "AttachmentsConfig":
if self.send_without_attachments_behavior is None:
self.send_without_attachments_behavior = Behavior.CONTINUE if self.send_without_attachments else Behavior.BLOCK
else:
self.send_without_attachments = self.send_without_attachments_behavior != Behavior.BLOCK
return self
@property
def individual_base_path_values(self) -> set[str]:
@@ -482,10 +654,22 @@ class EntryConfig(StrictModel):
disposition_notification_to: list[RecipientConfig] = Field(default_factory=list)
merge_disposition_notification_to: bool = True
channel_policy: DeliveryChannelPolicy | None = None
postbox_targets: list[PostboxTargetConfig] = Field(
default_factory=list,
max_length=50,
)
merge_postbox_targets: bool = True
print_target: PrintTargetConfig | None = None
attachments: list[AttachmentConfig] = Field(default_factory=list)
combine_attachments: bool = True
fields: dict[str, Any] = Field(default_factory=dict)
# Frozen channel candidates, source revisions and the explicit route
# decision imported from Distribution Lists. Campaign owns this snapshot;
# it never re-resolves the audience during build or delivery.
distribution_source: dict[str, Any] = Field(default_factory=dict)
last_sent: str | None = None
@@ -501,7 +685,11 @@ class ImportProvenance(StrictModel):
id: str
imported_at: str
mode: Literal["append", "replace"]
source_type: Literal["csv", "xlsx", "text"]
source_type: Literal["csv", "xlsx", "text", "addresses", "distribution_list"]
source_id: str | None = None
source_label: str | None = None
source_revision: str | None = None
source_provenance: dict[str, Any] = Field(default_factory=dict)
filename: str | None = None
sheet_name: str | None = None
encoding: str | None = None
@@ -559,7 +747,7 @@ class EntriesConfig(StrictModel):
class ValidationPolicy(StrictModel):
missing_required_attachment: Behavior = Behavior.ASK
missing_required_attachment: Behavior = Behavior.BLOCK
missing_optional_attachment: Behavior = Behavior.WARN
ambiguous_attachment_match: Behavior = Behavior.ASK
ignore_empty_fields: bool = False
@@ -591,7 +779,43 @@ class RetryConfig(StrictModel):
return values
class PostboxDeliveryConfig(StrictModel):
targets: list[PostboxTargetConfig] = Field(default_factory=list, max_length=50)
classification: str = Field(default="internal", min_length=1, max_length=50)
unresolved_target: Behavior = Behavior.BLOCK
vacant_target: Behavior = Behavior.WARN
duplicate_target: Behavior = Behavior.WARN
class PrintDeliveryConfig(StrictModel):
template_id: str | None = Field(default=None, max_length=36)
template_revision: int | None = Field(default=None, ge=1)
usage: str = Field(default="campaign_print", min_length=1, max_length=100)
output_format: Literal["html", "text"] = "html"
profile_id: str | None = Field(default=None, max_length=120)
persist_to_files: bool = True
class CalendarInvitationDeliveryConfig(StrictModel):
enabled: bool = False
calendar_id: str | None = Field(default=None, max_length=36)
summary_template: str | None = Field(default=None, max_length=2_000)
description_template: str | None = Field(default=None, max_length=20_000)
location_template: str | None = Field(default=None, max_length=2_000)
start_at_template: str | None = Field(default=None, max_length=1_000)
end_at_template: str | None = Field(default=None, max_length=1_000)
timezone: str | None = Field(default=None, max_length=100)
classification: Literal["PUBLIC", "PRIVATE", "CONFIDENTIAL"] = "PUBLIC"
categories: list[str] = Field(default_factory=list, max_length=50)
class DeliveryConfig(StrictModel):
channel_policy: DeliveryChannelPolicy = DeliveryChannelPolicy.MAIL
postbox: PostboxDeliveryConfig = Field(default_factory=PostboxDeliveryConfig)
print: PrintDeliveryConfig = Field(default_factory=PrintDeliveryConfig)
calendar_invitation: CalendarInvitationDeliveryConfig = Field(
default_factory=CalendarInvitationDeliveryConfig
)
rate_limit: RateLimitConfig = Field(default_factory=RateLimitConfig)
imap_append_sent: ImapAppendSentConfig = Field(default_factory=ImapAppendSentConfig)
retry: RetryConfig = Field(default_factory=RetryConfig)
@@ -634,3 +858,23 @@ class CampaignConfig(StrictModel):
if path.is_absolute():
return path
return (campaign_file.parent / path).resolve()
def effective_delivery_channel_policy(
config: CampaignConfig,
entry: EntryConfig,
) -> DeliveryChannelPolicy:
return entry.channel_policy or config.delivery.channel_policy
def effective_postbox_targets(
config: CampaignConfig,
entry: EntryConfig,
) -> list[PostboxTargetConfig]:
global_targets = list(config.delivery.postbox.targets)
individual_targets = list(entry.postbox_targets)
if not individual_targets:
return global_targets
if entry.merge_postbox_targets:
return [*global_targets, *individual_targets]
return individual_targets
@@ -0,0 +1,321 @@
from __future__ import annotations
from dataclasses import asdict
from typing import Any
from sqlalchemy.orm import Session
from govoplan_core.core.postbox import (
PostboxDeliveryCatalogRef,
PostboxDirectoryEntryRef,
PostboxTargetRef,
)
from govoplan_campaign.backend.campaign.field_values import (
effective_entry_field_values,
)
from govoplan_campaign.backend.campaign.models import (
Behavior,
CampaignConfig,
EntryConfig,
PostboxTargetConfig,
PostboxTargetMatch,
PostboxTargetMode,
effective_postbox_targets,
)
from govoplan_campaign.backend.integrations import postbox_integration
from govoplan_campaign.backend.messages.models import (
MessageIssue,
MessageValidationStatus,
)
def _apply_behavior(
current: MessageValidationStatus,
behavior: Behavior,
) -> MessageValidationStatus:
if behavior == Behavior.BLOCK:
return MessageValidationStatus.BLOCKED
if behavior == Behavior.DROP:
return MessageValidationStatus.EXCLUDED
if behavior == Behavior.ASK and current not in {
MessageValidationStatus.BLOCKED,
MessageValidationStatus.EXCLUDED,
}:
return MessageValidationStatus.NEEDS_REVIEW
if behavior == Behavior.WARN and current == MessageValidationStatus.READY:
return MessageValidationStatus.WARNING
return current
def _issue(
*,
code: str,
message: str,
behavior: Behavior,
) -> MessageIssue:
return MessageIssue(
severity="error" if behavior == Behavior.BLOCK else "warning",
code=code,
message=message,
behavior=behavior.value,
source="postbox",
)
def _field_value(
values: dict[str, Any],
field_name: str | None,
) -> str | None:
if not field_name:
return None
value = values.get(field_name)
if value is None:
return None
text = str(value).strip()
return text or None
def _match_unit(
catalog: PostboxDeliveryCatalogRef,
value: str | None,
match: PostboxTargetMatch,
):
if not value:
return None
return next(
(
unit
for unit in catalog.organization_units
if (unit.id if match == PostboxTargetMatch.ID else unit.slug) == value
),
None,
)
def _match_function(unit, value: str | None, match: PostboxTargetMatch):
if unit is None or not value:
return None
return next(
(
function
for function in unit.functions
if (
function.id
if match == PostboxTargetMatch.ID
else function.slug
)
== value
),
None,
)
def _target_ref(
target: PostboxTargetConfig,
*,
values: dict[str, Any],
catalog: PostboxDeliveryCatalogRef,
) -> tuple[PostboxTargetRef | None, str | None]:
if target.mode == PostboxTargetMode.DIRECT:
return (
PostboxTargetRef(
postbox_id=target.postbox_id,
address_key=target.address_key,
),
None,
)
unit_value = target.organization_unit_id or _field_value(
values,
target.organization_unit_field,
)
unit_match = (
PostboxTargetMatch.ID
if target.organization_unit_id
else target.organization_unit_match
)
unit = _match_unit(catalog, unit_value, unit_match)
if unit is None:
return None, (
f"Organization unit {unit_value!r} could not be resolved by "
f"{unit_match.value}."
)
function_value = target.function_id or _field_value(
values,
target.function_field,
)
function_match = (
PostboxTargetMatch.ID
if target.function_id
else target.function_match
)
function = _match_function(unit, function_value, function_match)
if function is None:
return None, (
f"Organization function {function_value!r} could not be resolved "
f"inside {unit.name!r} by {function_match.value}."
)
context_key = target.context_key or _field_value(
values,
target.context_field,
)
return (
PostboxTargetRef(
template_id=target.template_id,
organization_unit_id=unit.id,
function_id=function.id,
context_key=context_key,
),
None,
)
def _resolved_target_payload(
target: PostboxTargetConfig,
entry: PostboxDirectoryEntryRef,
*,
position: int,
) -> dict[str, Any]:
return {
"target_id": target.id,
"position": position,
"mode": target.mode.value,
"requested": target.model_dump(mode="json", exclude_none=True),
"postbox_id": entry.id,
"address": entry.address,
"address_key": entry.address_key,
"name": entry.name,
"status": entry.status,
"classification": entry.classification,
"organization_unit_id": entry.organization_unit_id,
"organization_unit_name": entry.organization_unit_name,
"function_id": entry.function_id,
"function_name": entry.function_name,
"context_key": entry.context_key,
"template_revision_id": entry.template_revision_id,
"holder_count": entry.holder_count,
"vacant": entry.vacant,
}
def resolve_entry_postbox_targets(
session: Session,
*,
tenant_id: str,
config: CampaignConfig,
entry: EntryConfig,
validation_status: MessageValidationStatus,
materialize: bool,
) -> tuple[
list[dict[str, Any]],
list[MessageIssue],
MessageValidationStatus,
]:
integration = postbox_integration()
policy = config.delivery.postbox
targets = effective_postbox_targets(config, entry)
if not targets:
issue = _issue(
code="postbox_target_missing",
message="Postbox delivery requires at least one target.",
behavior=policy.unresolved_target,
)
return (
[],
[issue],
_apply_behavior(validation_status, policy.unresolved_target),
)
try:
catalog = integration.delivery_catalog(session, tenant_id=tenant_id)
except Exception as exc:
issue = _issue(
code="postbox_unavailable",
message=str(exc),
behavior=Behavior.BLOCK,
)
return [], [issue], MessageValidationStatus.BLOCKED
values = effective_entry_field_values(config, entry)
resolved: list[dict[str, Any]] = []
issues: list[MessageIssue] = []
status = validation_status
seen_postbox_ids: set[str] = set()
for position, target in enumerate(targets):
target_ref, resolution_error = _target_ref(
target,
values=values,
catalog=catalog,
)
if target_ref is None:
issue = _issue(
code="postbox_target_unresolved",
message=resolution_error or "Postbox target could not be resolved.",
behavior=policy.unresolved_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.unresolved_target)
continue
try:
entry_ref = integration.resolve_postbox(
session,
tenant_id=tenant_id,
target=target_ref,
materialize=materialize,
)
except Exception as exc:
issue = _issue(
code="postbox_target_unresolved",
message=f"Postbox target {target.id!r} could not be resolved: {exc}",
behavior=policy.unresolved_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.unresolved_target)
continue
if entry_ref is None:
issue = _issue(
code="postbox_target_unresolved",
message=f"Postbox target {target.id!r} does not exist.",
behavior=policy.unresolved_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.unresolved_target)
continue
if entry_ref.id in seen_postbox_ids:
issue = _issue(
code="postbox_target_duplicate",
message=(
f"Postbox {entry_ref.address!r} is selected more than once; "
"it will receive one message."
),
behavior=policy.duplicate_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.duplicate_target)
continue
seen_postbox_ids.add(entry_ref.id)
resolved.append(
_resolved_target_payload(
target,
entry_ref,
position=position,
)
)
if entry_ref.vacant:
issue = _issue(
code="postbox_target_vacant",
message=(
f"Postbox {entry_ref.address!r} currently has no function "
"holder."
),
behavior=policy.vacant_target,
)
issues.append(issue)
status = _apply_behavior(status, policy.vacant_target)
return resolved, issues, status
def delivery_catalog_payload(catalog: PostboxDeliveryCatalogRef) -> dict[str, Any]:
return asdict(catalog)
@@ -0,0 +1,964 @@
from __future__ import annotations
import calendar
import copy
import hashlib
import json
from collections.abc import Mapping
from datetime import UTC, datetime, timedelta
from email import policy
from email.parser import BytesParser
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from sqlalchemy.orm import Session
from govoplan_campaign.backend.campaign.copying import campaign_copy_configuration
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignJob,
CampaignSchedule,
CampaignScheduleOccurrence,
CampaignShare,
CampaignVersion,
JobBuildStatus,
)
from govoplan_campaign.backend.approval_gate import (
assert_campaign_approval,
campaign_approval_gate,
)
from govoplan_campaign.backend.campaign.models import DeliveryChannelPolicy
from govoplan_campaign.backend.integrations import mail_integration
from govoplan_campaign.backend.persistence.campaigns import (
create_campaign_version_from_json,
)
from govoplan_campaign.backend.sending.execution import ensure_execution_snapshot
from govoplan_campaign.backend.sending.jobs import (
_from_header_from_job,
_send_job_delivery_context,
_single_job_validation_allowed,
_synchronous_smtp_batch_manager,
)
from govoplan_core.audit.logging import audit_event
RECURRENCE_KINDS = frozenset({"once", "daily", "weekly", "monthly"})
SCHEDULE_SOURCE_SCHEMA = "govoplan.campaign.schedule-source.v1"
SCHEDULE_DELIVERY_MODES = frozenset({"manual", "autonomous"})
def canonical_configuration_hash(value: Mapping[str, object]) -> str:
return hashlib.sha256(
json.dumps(
value,
ensure_ascii=True,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
).hexdigest()
def campaign_schedule_source_snapshot(
*,
configuration: Mapping[str, object],
campaign_settings: Mapping[str, object],
mail_profile_policy: Mapping[str, object],
shares: list[Mapping[str, object]],
) -> dict[str, object]:
"""Seal every selected source domain so worker execution cannot drift."""
return {
"schema": SCHEDULE_SOURCE_SCHEMA,
"configuration": copy.deepcopy(dict(configuration)),
"campaign_settings": copy.deepcopy(dict(campaign_settings)),
"mail_profile_policy": copy.deepcopy(dict(mail_profile_policy)),
"shares": [copy.deepcopy(dict(item)) for item in shares],
}
def next_schedule_fire(
scheduled_for: datetime,
*,
recurrence_kind: str,
interval_count: int,
timezone_name: str,
) -> datetime | None:
if recurrence_kind == "once":
return None
if recurrence_kind not in RECURRENCE_KINDS:
raise ValueError(f"Unsupported campaign recurrence: {recurrence_kind}")
if interval_count < 1:
raise ValueError("Campaign recurrence interval must be positive")
try:
zone = ZoneInfo(timezone_name)
except ZoneInfoNotFoundError as exc:
raise ValueError(f"Unknown campaign schedule timezone: {timezone_name}") from exc
local = _as_utc(scheduled_for).astimezone(zone)
if recurrence_kind == "daily":
upcoming = local + timedelta(days=interval_count)
elif recurrence_kind == "weekly":
upcoming = local + timedelta(weeks=interval_count)
else:
month_index = local.year * 12 + local.month - 1 + interval_count
year, month_offset = divmod(month_index, 12)
month = month_offset + 1
day = min(local.day, calendar.monthrange(year, month)[1])
upcoming = local.replace(year=year, month=month, day=day)
return upcoming.astimezone(UTC)
def dispatch_due_campaign_schedules(
session: Session,
*,
tenant_id: str | None = None,
now: datetime | None = None,
limit: int = 50,
) -> dict[str, object]:
observed_at = _as_utc(now or datetime.now(UTC))
refreshed = refresh_autonomous_schedule_outcomes(
session,
tenant_id=tenant_id,
now=observed_at,
)
query = session.query(CampaignSchedule).filter(
CampaignSchedule.active.is_(True),
CampaignSchedule.next_fire_at.is_not(None),
CampaignSchedule.next_fire_at <= observed_at,
)
if tenant_id is not None:
query = query.filter(CampaignSchedule.tenant_id == tenant_id)
schedules = (
query.order_by(CampaignSchedule.next_fire_at.asc(), CampaignSchedule.id.asc())
.with_for_update(skip_locked=True)
.limit(max(1, min(limit, 250)))
.all()
)
result: dict[str, object] = {
"selected": len(schedules),
"prepared": 0,
"autonomous_prepared": 0,
"failed": 0,
"completed": 0,
"coalesced": 0,
"duplicates": 0,
"deferred": 0,
"campaign_ids": [],
"operator_actions": [],
"refreshed": refreshed,
}
for schedule in schedules:
scheduled_for = _as_utc(schedule.next_fire_at or observed_at)
if schedule.delivery_mode == "autonomous" and _has_open_occurrence(
session, schedule_id=schedule.id
):
result["deferred"] = int(result["deferred"]) + 1
continue
try:
with session.begin_nested():
if schedule.delivery_mode == "autonomous":
_occurrence, skipped = _prepare_autonomous_occurrence(
session,
schedule=schedule,
scheduled_for=scheduled_for,
observed_at=observed_at,
)
campaign_id = schedule.campaign_id
result["autonomous_prepared"] = (
int(result["autonomous_prepared"]) + 1
)
else:
campaign, _version, skipped = _prepare_occurrence(
session,
schedule=schedule,
scheduled_for=scheduled_for,
observed_at=observed_at,
)
campaign_id = campaign.id
result["prepared"] = int(result["prepared"]) + 1
result["coalesced"] = int(result["coalesced"]) + skipped
result["campaign_ids"].append(campaign_id) # type: ignore[union-attr]
if not schedule.active:
result["completed"] = int(result["completed"]) + 1
except Exception as exc: # noqa: BLE001 - persist bounded operator evidence
session.expire_all()
recorded = (
session.query(CampaignScheduleOccurrence)
.filter(
CampaignScheduleOccurrence.schedule_id == schedule.id,
CampaignScheduleOccurrence.scheduled_for == scheduled_for,
)
.one_or_none()
)
if recorded is not None:
result["duplicates"] = int(result["duplicates"]) + 1
if (
schedule.active
and schedule.next_fire_at is not None
and _as_utc(schedule.next_fire_at) == scheduled_for
and recorded.status not in {"failed", "uncertain"}
):
_advance_schedule(
session,
schedule=schedule,
occurrence=recorded,
scheduled_for=scheduled_for,
observed_at=observed_at,
sequence=schedule.occurrence_count + 1,
)
continue
session.add(
CampaignScheduleOccurrence(
tenant_id=schedule.tenant_id,
schedule_id=schedule.id,
scheduled_for=scheduled_for,
status="failed",
idempotency_key=_occurrence_idempotency_key(
schedule.id, scheduled_for
),
error=str(exc)[:4000],
recovery_state="failed",
evidence={"delivery_mode": schedule.delivery_mode},
last_checked_at=observed_at,
)
)
schedule.active = False
schedule.last_error = str(exc)[:4000]
schedule.last_outcome = "failed"
schedule.last_recovery_state = "operator_required"
schedule.resource_revision += 1
session.add(schedule)
result["failed"] = int(result["failed"]) + 1
result["operator_actions"].append( # type: ignore[union-attr]
{
"schedule_id": schedule.id,
"campaign_id": schedule.campaign_id,
"reason": "draft_preparation_failed",
"delivery_mode": schedule.delivery_mode,
}
)
_notify_schedule_operator(
session,
schedule=schedule,
reason="policy_or_systemic_preflight_failed",
)
session.flush()
return result
def _prepare_occurrence(
session: Session,
*,
schedule: CampaignSchedule,
scheduled_for: datetime,
observed_at: datetime,
) -> tuple[Campaign, CampaignVersion, int]:
existing = (
session.query(CampaignScheduleOccurrence)
.filter(
CampaignScheduleOccurrence.schedule_id == schedule.id,
CampaignScheduleOccurrence.scheduled_for == scheduled_for,
)
.one_or_none()
)
if existing is not None:
raise RuntimeError("Campaign schedule occurrence was already recorded")
source_campaign = session.get(Campaign, schedule.campaign_id)
if source_campaign is None or source_campaign.tenant_id != schedule.tenant_id:
raise RuntimeError("Campaign schedule source is no longer available")
source_version = session.get(CampaignVersion, schedule.source_version_id)
if source_version is None or source_version.campaign_id != source_campaign.id:
raise RuntimeError("Campaign schedule source version is no longer available")
if canonical_configuration_hash(schedule.source_snapshot) != schedule.source_snapshot_hash:
raise RuntimeError("Campaign schedule source snapshot integrity check failed")
snapshot = _schedule_snapshot(schedule.source_snapshot)
sequence = schedule.occurrence_count + 1
external_id = _scheduled_external_id(
source_campaign.external_id,
schedule.id,
sequence,
)
local_date = scheduled_for.astimezone(ZoneInfo(schedule.timezone)).date().isoformat()
generated_name = f"{schedule.name} - {local_date}"
raw_json = campaign_copy_configuration(
snapshot["configuration"],
schedule.copy_options,
)
metadata = raw_json.get("campaign")
if not isinstance(metadata, dict):
raise RuntimeError("Campaign schedule snapshot has no campaign metadata")
metadata["id"] = external_id
metadata["name"] = generated_name
metadata["mode"] = "draft"
generated_campaign, generated_version = create_campaign_version_from_json(
session,
tenant_id=schedule.tenant_id,
user_id=schedule.created_by_user_id,
raw_json=raw_json,
source_filename=None,
source_base_path=schedule.source_base_path,
commit=False,
)
if bool(schedule.copy_options.get("include_policies", True)):
generated_campaign.settings = copy.deepcopy(snapshot["campaign_settings"])
if bool(schedule.copy_options.get("include_mail_profile", True)):
generated_campaign.mail_profile_policy = copy.deepcopy(
snapshot["mail_profile_policy"]
)
if bool(schedule.copy_options.get("include_shares", False)):
_copy_snapshot_shares(
session,
schedule=schedule,
generated_campaign=generated_campaign,
shares=snapshot["shares"],
)
occurrence = CampaignScheduleOccurrence(
tenant_id=schedule.tenant_id,
schedule_id=schedule.id,
scheduled_for=scheduled_for,
status="prepared",
idempotency_key=_occurrence_idempotency_key(schedule.id, scheduled_for),
generated_campaign_id=generated_campaign.id,
generated_version_id=generated_version.id,
recovery_state="none",
evidence={"delivery_mode": "manual"},
last_checked_at=observed_at,
)
session.add(occurrence)
session.flush()
schedule.last_campaign_id = generated_campaign.id
schedule.last_outcome = "prepared"
schedule.last_recovery_state = "none"
coalesced = _advance_schedule(
session,
schedule=schedule,
occurrence=occurrence,
scheduled_for=scheduled_for,
observed_at=observed_at,
sequence=sequence,
)
audit_event(
session,
tenant_id=schedule.tenant_id,
user_id=schedule.created_by_user_id,
action="campaign.schedule.draft_prepared",
object_type="campaign_schedule",
object_id=schedule.id,
details={
"source_campaign_id": source_campaign.id,
"source_version_id": source_version.id,
"scheduled_for": scheduled_for.isoformat(),
"generated_campaign_id": generated_campaign.id,
"generated_version_id": generated_version.id,
"occurrence": sequence,
"coalesced_missed_intervals": coalesced,
"delivery_started": False,
},
commit=False,
)
return generated_campaign, generated_version, coalesced
def validate_autonomous_schedule_source(
session: Session,
*,
campaign: Campaign,
version: CampaignVersion,
) -> dict[str, object]:
"""Validate the exact immutable execution that an autonomous schedule reuses."""
gate = campaign_approval_gate(version)
if gate is None:
raise RuntimeError(
"Autonomous delivery requires an explicit Approval request for the built source version."
)
assert_campaign_approval(session, tenant_id=campaign.tenant_id, version=version)
snapshot = ensure_execution_snapshot(session, version)
snapshot_hash = str(version.execution_snapshot_hash or "")
if len(snapshot_hash) != 64:
raise RuntimeError("The approved Campaign execution snapshot is incomplete.")
jobs = _autonomous_source_jobs(
session,
tenant_id=campaign.tenant_id,
campaign_id=campaign.id,
version=version,
)
mail = mail_integration()
if not mail.durable_delivery_available:
raise RuntimeError(
"Autonomous delivery requires Mail's durable delivery-command outbox."
)
if not snapshot.mail_profile_id or not snapshot.smtp_transport_revision:
raise RuntimeError(
"The approved Campaign execution has no immutable Mail transport evidence."
)
summary = mail.campaign_profile_delivery_summary(
session,
tenant_id=campaign.tenant_id,
campaign_id=campaign.id,
profile_id=snapshot.mail_profile_id,
smtp_server_id=snapshot.smtp_server_id,
smtp_credential_id=snapshot.smtp_credential_id,
)
if not summary.get("smtp_available"):
raise RuntimeError("The approved Campaign Mail transport is unavailable.")
if summary.get("smtp_transport_revision") != snapshot.smtp_transport_revision:
raise RuntimeError(
"The Campaign Mail transport changed after approval; rebuild and approve a new source version."
)
return {
"execution_snapshot_hash": snapshot_hash,
"approval_request_id": str(gate.get("request_id") or ""),
"approval_subject_digest": str(gate.get("subject_digest") or ""),
"job_count": len(jobs),
"job_manifest_sha256": canonical_configuration_hash(
{"jobs": [{"id": job.id, "eml_sha256": job.eml_sha256} for job in jobs]}
),
}
def _autonomous_source_jobs(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version: CampaignVersion,
) -> list[CampaignJob]:
jobs = (
session.query(CampaignJob)
.filter(
CampaignJob.tenant_id == tenant_id,
CampaignJob.campaign_id == campaign_id,
CampaignJob.campaign_version_id == version.id,
)
.order_by(CampaignJob.entry_index.asc(), CampaignJob.id.asc())
.all()
)
if not jobs:
raise RuntimeError(
"Autonomous delivery requires a built source version with recipient jobs."
)
for job in jobs:
if job.build_status != JobBuildStatus.BUILT.value:
raise RuntimeError(
"Autonomous delivery requires every source message to be built."
)
if not _single_job_validation_allowed(version, job, include_warnings=True):
raise RuntimeError(
"Autonomous delivery requires every source message to pass its reviewed recipient and attachment gates."
)
if DeliveryChannelPolicy(job.delivery_channel_policy) != DeliveryChannelPolicy.MAIL:
raise RuntimeError(
"Autonomous schedules currently support Mail-only delivery; use manual mode for hybrid, Postbox, or print delivery."
)
return jobs
def _prepare_autonomous_occurrence(
session: Session,
*,
schedule: CampaignSchedule,
scheduled_for: datetime,
observed_at: datetime,
) -> tuple[CampaignScheduleOccurrence, int]:
existing = (
session.query(CampaignScheduleOccurrence)
.filter(
CampaignScheduleOccurrence.schedule_id == schedule.id,
CampaignScheduleOccurrence.scheduled_for == scheduled_for,
)
.one_or_none()
)
if existing is not None:
raise RuntimeError("Campaign schedule occurrence was already recorded")
if canonical_configuration_hash(schedule.source_snapshot) != schedule.source_snapshot_hash:
raise RuntimeError("Campaign schedule source snapshot integrity check failed")
campaign = session.get(Campaign, schedule.campaign_id)
version = session.get(CampaignVersion, schedule.source_version_id)
if campaign is None or campaign.tenant_id != schedule.tenant_id:
raise RuntimeError("Campaign schedule source is no longer available")
if version is None or version.campaign_id != campaign.id:
raise RuntimeError("Campaign schedule source version is no longer available")
validation = validate_autonomous_schedule_source(
session,
campaign=campaign,
version=version,
)
if (
not schedule.approved_execution_snapshot_hash
or validation["execution_snapshot_hash"]
!= schedule.approved_execution_snapshot_hash
):
raise RuntimeError(
"The approved Campaign execution changed after the autonomous schedule was created."
)
jobs = _autonomous_source_jobs(
session,
tenant_id=schedule.tenant_id,
campaign_id=campaign.id,
version=version,
)
occurrence_key = _occurrence_idempotency_key(schedule.id, scheduled_for)
occurrence = CampaignScheduleOccurrence(
tenant_id=schedule.tenant_id,
schedule_id=schedule.id,
scheduled_for=scheduled_for,
status="preparing",
idempotency_key=occurrence_key,
recovery_state="prepared",
evidence={
"delivery_mode": "autonomous",
"source_campaign_id": campaign.id,
"source_version_id": version.id,
"source_snapshot_hash": schedule.source_snapshot_hash,
**validation,
},
last_checked_at=observed_at,
)
session.add(occurrence)
session.flush()
contexts = {job.id: _send_job_delivery_context(session, job) for job in jobs}
with _synchronous_smtp_batch_manager(session, jobs=jobs, contexts=contexts):
pass
mail = mail_integration()
commands: list[dict[str, object]] = []
for job in jobs:
context = contexts[job.id]
if context.envelope_from is None or not context.envelope_recipients:
raise RuntimeError("A frozen Campaign message has no delivery envelope.")
message = BytesParser(policy=policy.default).parsebytes(context.message_bytes)
commands.append(
mail.submit_delivery_command(
session,
tenant_id=schedule.tenant_id,
command_type="campaign_schedule_occurrence",
source_module="campaigns",
source_resource_type="campaign",
source_resource_id=campaign.id,
source_version_id=version.id,
idempotency_key=f"{occurrence_key}:{job.id}",
profile_id=context.snapshot.mail_profile_id,
message_bytes=context.message_bytes,
envelope_from=context.envelope_from,
envelope_recipients=context.envelope_recipients,
from_header=_from_header_from_job(job) or str(message.get("From") or ""),
expected_smtp_transport_revision=(
context.snapshot.smtp_transport_revision or ""
),
smtp_server_id=context.snapshot.smtp_server_id,
smtp_credential_id=context.snapshot.smtp_credential_id,
created_by_user_id=schedule.created_by_user_id,
)
)
occurrence.delivery_command_ids = [str(item["id"]) for item in commands]
occurrence.status = "prepared"
occurrence.recovery_state = "pending"
occurrence.evidence = {
**occurrence.evidence,
"command_count": len(commands),
"duplicate_command_count": sum(bool(item.get("duplicate")) for item in commands),
"command_status_counts": _status_counts(commands),
}
occurrence.last_checked_at = observed_at
sequence = schedule.occurrence_count + 1
schedule.last_campaign_id = campaign.id
schedule.last_outcome = "prepared"
schedule.last_recovery_state = "pending"
coalesced = _advance_schedule(
session,
schedule=schedule,
occurrence=occurrence,
scheduled_for=scheduled_for,
observed_at=observed_at,
sequence=sequence,
)
audit_event(
session,
tenant_id=schedule.tenant_id,
user_id=schedule.created_by_user_id,
action="campaign.schedule.delivery_prepared",
object_type="campaign_schedule_occurrence",
object_id=occurrence.id,
details={
"schedule_id": schedule.id,
"campaign_id": campaign.id,
"source_version_id": version.id,
"scheduled_for": scheduled_for.isoformat(),
"occurrence_idempotency_key": occurrence_key,
"delivery_command_count": len(commands),
"execution_snapshot_hash": validation["execution_snapshot_hash"],
"approval_request_id": validation["approval_request_id"],
"coalesced_missed_intervals": coalesced,
},
commit=False,
)
return occurrence, coalesced
def _advance_schedule(
session: Session,
*,
schedule: CampaignSchedule,
occurrence: CampaignScheduleOccurrence,
scheduled_for: datetime,
observed_at: datetime,
sequence: int,
) -> int:
schedule.occurrence_count = sequence
schedule.last_fired_at = scheduled_for
schedule.last_error = None
next_fire = next_schedule_fire(
scheduled_for,
recurrence_kind=schedule.recurrence_kind,
interval_count=schedule.interval_count,
timezone_name=schedule.timezone,
)
coalesced = 0
while next_fire is not None and next_fire <= observed_at:
session.add(
CampaignScheduleOccurrence(
tenant_id=schedule.tenant_id,
schedule_id=schedule.id,
scheduled_for=next_fire,
status="superseded",
idempotency_key=_occurrence_idempotency_key(schedule.id, next_fire),
recovery_state="superseded",
evidence={
"delivery_mode": schedule.delivery_mode,
"reason": "coalesced_missed_interval",
"superseded_by_occurrence_id": occurrence.id,
},
last_checked_at=observed_at,
)
)
next_fire = next_schedule_fire(
next_fire,
recurrence_kind=schedule.recurrence_kind,
interval_count=schedule.interval_count,
timezone_name=schedule.timezone,
)
coalesced += 1
if (
next_fire is None
or sequence >= schedule.max_occurrences
or (schedule.ends_at is not None and next_fire > _as_utc(schedule.ends_at))
):
schedule.active = False
schedule.next_fire_at = None
else:
schedule.next_fire_at = next_fire
schedule.resource_revision += 1
session.add(schedule)
return coalesced
def refresh_autonomous_schedule_outcomes(
session: Session,
*,
tenant_id: str | None = None,
now: datetime | None = None,
) -> dict[str, int]:
observed_at = _as_utc(now or datetime.now(UTC))
query = session.query(CampaignScheduleOccurrence).filter(
CampaignScheduleOccurrence.status.in_(("prepared", "uncertain")),
)
if tenant_id is not None:
query = query.filter(CampaignScheduleOccurrence.tenant_id == tenant_id)
counts = {
"checked": 0,
"accepted": 0,
"uncertain": 0,
"failed": 0,
"skipped": 0,
}
mail = mail_integration()
if not mail.durable_delivery_available:
for occurrence in query.order_by(
CampaignScheduleOccurrence.created_at
).limit(250):
if not occurrence.delivery_command_ids:
continue
counts["checked"] += 1
counts["uncertain"] += 1
_mark_occurrence_uncertain(
session,
occurrence=occurrence,
observed_at=observed_at,
reason="mail_delivery_outbox_unavailable",
)
return counts
for occurrence in query.order_by(CampaignScheduleOccurrence.created_at).limit(250):
if not occurrence.delivery_command_ids:
continue
summaries: list[dict[str, object]] = []
try:
summaries = [
mail.delivery_command_summary(
session,
tenant_id=occurrence.tenant_id,
command_id=command_id,
)
for command_id in occurrence.delivery_command_ids
]
except Exception:
counts["checked"] += 1
counts["uncertain"] += 1
_mark_occurrence_uncertain(
session,
occurrence=occurrence,
observed_at=observed_at,
reason="mail_delivery_status_unavailable",
)
continue
counts["checked"] += 1
outcome, recovery_state = _aggregate_command_outcome(summaries)
previous_outcome = occurrence.status
previous_recovery_state = occurrence.recovery_state
occurrence.status = outcome
occurrence.recovery_state = recovery_state
occurrence.last_checked_at = observed_at
occurrence.evidence = {
**(occurrence.evidence or {}),
"command_status_counts": _status_counts(summaries),
"accepted_recipient_count": sum(
int(item.get("accepted_count") or 0) for item in summaries
),
"refused_recipient_count": sum(
int(item.get("refused_count") or 0) for item in summaries
),
"failure_codes": sorted(
{
str(item["failure_code"])
for item in summaries
if item.get("failure_code")
}
),
}
schedule = session.get(CampaignSchedule, occurrence.schedule_id)
if schedule is not None:
schedule.last_outcome = outcome
schedule.last_recovery_state = recovery_state
transitioned_to_operator_required = (
outcome in {"uncertain", "failed"}
and (
previous_outcome != outcome
or previous_recovery_state != recovery_state
or schedule.active
)
)
if transitioned_to_operator_required:
schedule.active = False
schedule.last_error = (
"Autonomous delivery needs operator review; automatic recurrence is paused."
)
schedule.resource_revision += 1
_notify_schedule_operator(
session,
schedule=schedule,
reason=f"delivery_{outcome}",
)
session.add(schedule)
session.add(occurrence)
if outcome in counts:
counts[outcome] += 1
return counts
def _mark_occurrence_uncertain(
session: Session,
*,
occurrence: CampaignScheduleOccurrence,
observed_at: datetime,
reason: str,
) -> None:
previous_outcome = occurrence.status
previous_recovery_state = occurrence.recovery_state
occurrence.status = "uncertain"
occurrence.recovery_state = "operator_required"
occurrence.last_checked_at = observed_at
occurrence.evidence = {
**(occurrence.evidence or {}),
"recovery_reason": reason,
}
schedule = session.get(CampaignSchedule, occurrence.schedule_id)
if schedule is not None:
transitioned = (
previous_outcome != "uncertain"
or previous_recovery_state != "operator_required"
or schedule.active
)
schedule.active = False
schedule.last_outcome = "uncertain"
schedule.last_recovery_state = "operator_required"
schedule.last_error = (
"Autonomous delivery status is unavailable; automatic recurrence is paused."
)
if transitioned:
schedule.resource_revision += 1
_notify_schedule_operator(
session,
schedule=schedule,
reason=reason,
)
session.add(schedule)
session.add(occurrence)
def _has_open_occurrence(session: Session, *, schedule_id: str) -> bool:
rows = (
session.query(CampaignScheduleOccurrence.delivery_command_ids)
.filter(
CampaignScheduleOccurrence.schedule_id == schedule_id,
CampaignScheduleOccurrence.status == "prepared",
)
.limit(1000)
.all()
)
return any(bool(command_ids) for (command_ids,) in rows)
def _aggregate_command_outcome(
summaries: list[dict[str, object]],
) -> tuple[str, str]:
statuses = {str(item.get("status") or "") for item in summaries}
if statuses and statuses <= {"accepted", "reconciled_accepted"}:
return "accepted", "complete"
if statuses and statuses <= {"reconciled_not_accepted"}:
return "skipped", "reconciled"
if statuses & {"outcome_unknown", "in_progress"}:
return "uncertain", "operator_required"
if statuses & {"permanent_failure", "partially_refused", "reconciled_not_accepted"}:
return "failed", "operator_required"
return "prepared", "pending"
def _status_counts(items: list[dict[str, object]]) -> dict[str, int]:
result: dict[str, int] = {}
for item in items:
status = str(item.get("status") or "unknown")
result[status] = result.get(status, 0) + 1
return result
def _occurrence_idempotency_key(schedule_id: str, scheduled_for: datetime) -> str:
return f"campaign-schedule:{schedule_id}:{_as_utc(scheduled_for).isoformat()}"
def _notify_schedule_operator(
session: Session,
*,
schedule: CampaignSchedule,
reason: str,
) -> None:
from govoplan_core.core.notifications import (
NotificationDispatchRequest,
notification_dispatch_provider,
)
from govoplan_campaign.backend.runtime import get_registry
provider = notification_dispatch_provider(get_registry())
if provider is None:
return
try:
provider.enqueue_notification(
session,
NotificationDispatchRequest(
tenant_id=schedule.tenant_id,
source_module="campaigns",
source_resource_type="campaign_schedule",
source_resource_id=schedule.id,
event_kind="campaign.schedule.operator_required",
channel="inbox",
recipient_type="user" if schedule.created_by_user_id else None,
recipient_id=schedule.created_by_user_id,
subject=f"Campaign schedule paused: {schedule.name}",
body_text=(
"Autonomous Campaign delivery was paused before another occurrence. "
"Review its recovery evidence before resuming."
),
action_url=f"/campaigns/{schedule.campaign_id}",
priority=2,
payload={"schedule_id": schedule.id, "reason": reason},
),
enqueue_delivery=False,
)
except Exception:
return
def _copy_snapshot_shares(
session: Session,
*,
schedule: CampaignSchedule,
generated_campaign: Campaign,
shares: object,
) -> None:
if not isinstance(shares, list):
raise RuntimeError("Campaign schedule share snapshot is invalid")
for source in shares:
if not isinstance(source, Mapping):
raise RuntimeError("Campaign schedule share snapshot is invalid")
target_type = str(source.get("target_type") or "")
target_id = str(source.get("target_id") or "")
permission = str(source.get("permission") or "read")
if not target_type or not target_id:
raise RuntimeError("Campaign schedule share snapshot is incomplete")
session.add(
CampaignShare(
tenant_id=schedule.tenant_id,
campaign_id=generated_campaign.id,
target_type=target_type,
target_id=target_id,
permission=permission,
created_by_user_id=schedule.created_by_user_id,
)
)
def _schedule_snapshot(value: object) -> dict[str, object]:
if not isinstance(value, Mapping) or value.get("schema") != SCHEDULE_SOURCE_SCHEMA:
raise RuntimeError("Campaign schedule source snapshot schema is invalid")
configuration = value.get("configuration")
settings = value.get("campaign_settings")
mail_policy = value.get("mail_profile_policy")
shares = value.get("shares")
if (
not isinstance(configuration, Mapping)
or not isinstance(settings, Mapping)
or not isinstance(mail_policy, Mapping)
or not isinstance(shares, list)
):
raise RuntimeError("Campaign schedule source snapshot is incomplete")
return {
"configuration": dict(configuration),
"campaign_settings": dict(settings),
"mail_profile_policy": dict(mail_policy),
"shares": shares,
}
def _scheduled_external_id(source: str, schedule_id: str, sequence: int) -> str:
suffix = f"-scheduled-{schedule_id[:8]}-{sequence}"
return f"{source[:255 - len(suffix)]}{suffix}"
def _as_utc(value: datetime) -> datetime:
if value.tzinfo is None:
return value.replace(tzinfo=UTC)
return value.astimezone(UTC)
__all__ = [
"RECURRENCE_KINDS",
"SCHEDULE_SOURCE_SCHEMA",
"campaign_schedule_source_snapshot",
"canonical_configuration_hash",
"dispatch_due_campaign_schedules",
"next_schedule_fire",
"refresh_autonomous_schedule_outcomes",
"validate_autonomous_schedule_source",
]
@@ -0,0 +1,730 @@
from __future__ import annotations
import copy
import hashlib
import json
from collections import Counter
from collections.abc import Iterable, Mapping
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
from uuid import uuid4
from govoplan_campaign.backend.campaign.loader import validate_against_schema
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignIssue,
CampaignJob,
CampaignVersion,
)
from govoplan_campaign.backend.persistence.versions import minimal_campaign_json
from govoplan_campaign.backend.response_security import (
public_campaign_configuration,
public_campaign_payload,
)
PORTABLE_CAMPAIGN_FORMAT = "govoplan.campaign-portable"
PORTABLE_CAMPAIGN_FORMAT_VERSION = "1.0"
PORTABLE_CAMPAIGN_SCOPE_ORDER = (
"metadata",
"template_config",
"recipients",
"attachments",
"review_state",
"delivery_history",
)
DEFAULT_PORTABLE_CAMPAIGN_SCOPES = ("metadata", "template_config")
OPERATIONAL_EVIDENCE_SCOPES = frozenset(("review_state", "delivery_history"))
_CONFIG_STRUCTURAL_KEYS = frozenset(
("version", "campaign", "recipients", "entries", "attachments")
)
_SENSITIVE_SETTING_FRAGMENTS = (
"api_key",
"credential",
"password",
"private_key",
"secret",
"token",
)
class CampaignTransferError(ValueError):
pass
@dataclass(frozen=True, slots=True)
class CampaignImportInspection:
preview: dict[str, Any]
configuration: dict[str, Any] | None
portable_settings: dict[str, Any]
def canonical_sha256(value: object) -> str:
encoded = json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def normalize_transfer_scopes(scopes: Iterable[str]) -> tuple[str, ...]:
selected = set(scopes)
invalid = sorted(selected.difference(PORTABLE_CAMPAIGN_SCOPE_ORDER))
if invalid:
raise CampaignTransferError(
f"Unsupported campaign transfer scope(s): {', '.join(invalid)}"
)
if not selected:
raise CampaignTransferError("Select at least one campaign transfer scope.")
return tuple(scope for scope in PORTABLE_CAMPAIGN_SCOPE_ORDER if scope in selected)
def build_campaign_portable_package(
*,
campaign: Campaign,
version: CampaignVersion,
scopes: Iterable[str],
jobs: Iterable[CampaignJob] = (),
issues: Iterable[CampaignIssue] = (),
module_version: str,
) -> dict[str, Any]:
selected = normalize_transfer_scopes(scopes)
configuration = public_campaign_configuration(version.raw_json)
if not isinstance(configuration, dict):
raise CampaignTransferError("The campaign configuration is not portable JSON.")
configuration, password_redactions = _redact_password_field_values(configuration)
payload: dict[str, Any] = {}
item_counts: dict[str, int] = {}
redactions: Counter[str] = Counter(password_redactions)
if "metadata" in selected:
payload["metadata"] = {
"external_id": campaign.external_id,
"name": campaign.name,
"description": campaign.description,
"source_status": campaign.status,
}
item_counts["metadata"] = 1
if "template_config" in selected:
settings, setting_redactions = _redact_sensitive_settings(
campaign.settings or {}
)
mail_policy, mail_policy_redactions = _redact_sensitive_settings(
campaign.mail_profile_policy or {}
)
template_configuration = {
key: copy.deepcopy(value)
for key, value in configuration.items()
if key not in _CONFIG_STRUCTURAL_KEYS
}
server = template_configuration.get("server")
if isinstance(server, dict):
for key in ("smtp_credential_id", "imap_credential_id"):
if server.pop(key, None) is not None:
redactions["deployment_credential_reference"] += 1
payload["template_config"] = {
"schema_version": version.schema_version,
"configuration": template_configuration,
"campaign_settings": settings,
"mail_profile_policy": mail_policy,
}
redactions.update(setting_redactions)
redactions.update(mail_policy_redactions)
item_counts["template_config"] = len(template_configuration)
if "recipients" in selected:
entries = copy.deepcopy(configuration.get("entries") or {})
_remove_entry_attachments(entries)
payload["recipients"] = {
"recipients": copy.deepcopy(configuration.get("recipients") or {}),
"entries": entries,
}
item_counts["recipients"] = _recipient_entry_count(entries)
if "attachments" in selected:
entry_attachments = _entry_attachment_projection(
configuration.get("entries")
)
payload["attachments"] = {
"configuration": copy.deepcopy(configuration.get("attachments") or {}),
"entry_attachments": entry_attachments,
"content_included": False,
}
item_counts["attachments"] = _attachment_rule_count(
payload["attachments"]
)
issue_rows = tuple(issues)
if "review_state" in selected:
review_state = _review_state_projection(version, issue_rows)
payload["review_state"] = review_state
item_counts["review_state"] = int(review_state["decision_count"])
job_rows = tuple(jobs)
if "delivery_history" in selected:
payload["delivery_history"] = {
"jobs": [_delivery_job_projection(job) for job in job_rows],
"counts": _delivery_counts(job_rows),
}
item_counts["delivery_history"] = len(job_rows)
exported_at = datetime.now(UTC)
package: dict[str, Any] = {
"format": PORTABLE_CAMPAIGN_FORMAT,
"format_version": PORTABLE_CAMPAIGN_FORMAT_VERSION,
"package_id": str(uuid4()),
"exported_at": exported_at.isoformat(),
"source": {
"module": "campaigns",
"module_version": module_version,
"tenant_ref_sha256": hashlib.sha256(
campaign.tenant_id.encode("utf-8")
).hexdigest(),
"campaign_id": campaign.id,
"campaign_external_id": campaign.external_id,
"campaign_name": campaign.name,
"version_id": version.id,
"version_number": version.version_number,
"campaign_schema_version": version.schema_version,
},
"scopes": list(selected),
"manifest": {
"item_counts": item_counts,
"redactions": dict(sorted(redactions.items())),
"privacy_default_scopes": list(DEFAULT_PORTABLE_CAMPAIGN_SCOPES),
"attachments_are_references_only": True,
"operational_evidence_is_not_replayed": True,
"secrets_included": False,
},
"payload": payload,
}
package["integrity"] = {
"algorithm": "sha256",
"package_sha256": canonical_sha256(package),
}
return package
def inspect_campaign_portable_package(
package: Mapping[str, Any],
*,
selected_scopes: Iterable[str] | None,
external_id: str,
name: str,
) -> CampaignImportInspection:
errors: list[str] = []
warnings: list[str] = []
package_dict = copy.deepcopy(dict(package))
package_id = _optional_text(package_dict.get("package_id"))
format_version = _optional_text(package_dict.get("format_version"))
source = package_dict.get("source")
source_dict = copy.deepcopy(source) if isinstance(source, dict) else {}
integrity = package_dict.get("integrity")
expected_hash = (
_optional_text(integrity.get("package_sha256"))
if isinstance(integrity, dict)
else None
)
hash_input = copy.deepcopy(package_dict)
hash_input.pop("integrity", None)
actual_hash = canonical_sha256(hash_input)
if package_dict.get("format") != PORTABLE_CAMPAIGN_FORMAT:
errors.append("The file is not a GovOPlaN portable Campaign package.")
if format_version != PORTABLE_CAMPAIGN_FORMAT_VERSION:
errors.append(
"The Campaign package format version is not supported by this installation."
)
if not package_id:
errors.append("The Campaign package has no package identifier.")
if not expected_hash or expected_hash != actual_hash:
errors.append("The Campaign package integrity checksum does not match its content.")
if not isinstance(integrity, dict) or integrity.get("algorithm") != "sha256":
errors.append("The Campaign package does not use the supported SHA-256 integrity algorithm.")
if not source_dict:
errors.append("The Campaign package has no source provenance.")
elif source_dict.get("campaign_schema_version") != "1.0":
errors.append("The Campaign configuration schema version is not supported by this installation.")
available: tuple[str, ...] = ()
try:
raw_scopes = package_dict.get("scopes")
if not isinstance(raw_scopes, list):
raise CampaignTransferError("The Campaign package has no valid scope list.")
available = normalize_transfer_scopes(str(item) for item in raw_scopes)
except CampaignTransferError as exc:
errors.append(str(exc))
try:
selected = normalize_transfer_scopes(
available if selected_scopes is None else selected_scopes
)
except CampaignTransferError as exc:
errors.append(str(exc))
selected = ()
unavailable = sorted(set(selected).difference(available))
if unavailable:
errors.append(
f"Selected scope(s) are absent from the package: {', '.join(unavailable)}"
)
payload = package_dict.get("payload")
payload_dict = payload if isinstance(payload, dict) else {}
if not isinstance(payload, dict):
errors.append("The Campaign package has no valid payload object.")
if not isinstance(package_dict.get("manifest"), dict):
errors.append("The Campaign package has no valid manifest.")
for scope in available:
if scope not in payload_dict:
errors.append(f"The Campaign package payload is missing scope '{scope}'.")
elif not isinstance(payload_dict[scope], dict):
errors.append(f"The Campaign package scope '{scope}' is not a valid object.")
template_scope = payload_dict.get("template_config")
if (
"template_config" in available
and isinstance(template_scope, dict)
and template_scope.get("schema_version") != "1.0"
):
errors.append("The portable template/configuration schema version is not supported.")
configuration: dict[str, Any] | None = None
portable_settings: dict[str, Any] = {}
will_create: list[dict[str, Any]] = []
will_skip: list[dict[str, Any]] = []
if not errors:
configuration, portable_settings, created, skipped, materialize_warnings = (
_materialize_import(
payload_dict,
available=available,
selected=selected,
external_id=external_id,
name=name,
)
)
will_create.extend(created)
will_skip.extend(skipped)
warnings.extend(materialize_warnings)
try:
validate_against_schema(configuration)
except Exception as exc:
errors.append(f"The imported Campaign configuration is incompatible: {exc}")
configuration = None
manifest = package_dict.get("manifest")
if isinstance(manifest, dict) and manifest.get("redactions"):
warnings.append(
"The source export redacted sensitive or deployment-bound values; review the package manifest and reconfigure them locally."
)
preview = {
"compatible": not errors,
"package_id": package_id,
"package_sha256": actual_hash,
"format_version": format_version,
"source": source_dict,
"available_scopes": list(available),
"selected_scopes": list(selected),
"destination": {
"external_id": external_id,
"name": name,
"status": "draft",
},
"will_create": will_create,
"will_skip": will_skip,
"warnings": list(dict.fromkeys(warnings)),
"errors": list(dict.fromkeys(errors)),
}
return CampaignImportInspection(
preview=preview,
configuration=configuration,
portable_settings=portable_settings,
)
def _materialize_import(
payload: Mapping[str, Any],
*,
available: tuple[str, ...],
selected: tuple[str, ...],
external_id: str,
name: str,
) -> tuple[
dict[str, Any],
dict[str, Any],
list[dict[str, Any]],
list[dict[str, Any]],
list[str],
]:
selected_set = set(selected)
configuration = minimal_campaign_json(external_id=external_id, name=name)
portable_settings: dict[str, Any] = {}
created: list[dict[str, Any]] = [
_plan_item("metadata", "campaign_draft", "A new Campaign draft and editable version will be created.", 1)
]
skipped: list[dict[str, Any]] = []
warnings: list[str] = []
metadata = payload.get("metadata")
if "metadata" in selected_set and isinstance(metadata, dict):
description = metadata.get("description")
if isinstance(description, str):
configuration["campaign"]["description"] = description
template_payload = payload.get("template_config")
if "template_config" in selected_set and isinstance(template_payload, dict):
source_configuration = template_payload.get("configuration")
if isinstance(source_configuration, dict):
for key, value in source_configuration.items():
if key in _CONFIG_STRUCTURAL_KEYS:
continue
configuration[key] = copy.deepcopy(value)
source_server = configuration.get("server")
if isinstance(source_server, dict) and source_server:
configuration["server"] = {}
skipped.append(
_plan_item(
"template_config",
"deployment_bound_mail_profile",
"Mail profile and server references are not applied across installations; select local Mail resources after import.",
len(source_server),
)
)
settings = template_payload.get("campaign_settings")
if isinstance(settings, dict):
portable_settings = copy.deepcopy(settings)
created.append(
_plan_item(
"template_config",
"editable_configuration",
"Portable fields, template, delivery settings, and validation policy will be applied to the draft.",
len(source_configuration),
)
)
recipients_payload = payload.get("recipients")
if "recipients" in selected_set and isinstance(recipients_payload, dict):
recipients = recipients_payload.get("recipients")
entries = recipients_payload.get("entries")
if isinstance(recipients, dict):
configuration["recipients"] = copy.deepcopy(recipients)
if isinstance(entries, dict):
configuration["entries"] = copy.deepcopy(entries)
created.append(
_plan_item(
"recipients",
"recipient_rows",
"Campaign-local recipient rows and source provenance will be copied into the draft.",
_recipient_entry_count(entries),
)
)
attachments_payload = payload.get("attachments")
if "attachments" in selected_set and isinstance(attachments_payload, dict):
attachment_configuration = attachments_payload.get("configuration")
if isinstance(attachment_configuration, dict):
configuration["attachments"] = copy.deepcopy(attachment_configuration)
per_entry = attachments_payload.get("entry_attachments")
applied_entry_rules = 0
if "recipients" in selected_set and isinstance(per_entry, list):
inline = configuration.get("entries", {}).get("inline", [])
if isinstance(inline, list):
for item in per_entry:
if not isinstance(item, dict):
continue
index = item.get("entry_index")
rules = item.get("attachments")
if (
isinstance(index, int)
and 0 <= index < len(inline)
and isinstance(inline[index], dict)
and isinstance(rules, list)
):
inline[index]["attachments"] = copy.deepcopy(rules)
applied_entry_rules += len(rules)
elif isinstance(per_entry, list) and per_entry:
skipped.append(
_plan_item(
"attachments",
"recipient_scope_required",
"Per-recipient attachment rules are skipped unless recipient rows are also imported.",
sum(
len(item.get("attachments") or [])
for item in per_entry
if isinstance(item, dict)
),
)
)
created.append(
_plan_item(
"attachments",
"attachment_references",
"Portable attachment rules will be applied; file content is never embedded in the package.",
_attachment_rule_count(attachments_payload) - max(0, _entry_rule_count(per_entry) - applied_entry_rules),
)
)
warnings.append(
"Attachment rules contain references only. Reconnect or upload the required files and validate the draft before use."
)
for scope in PORTABLE_CAMPAIGN_SCOPE_ORDER:
if scope not in OPERATIONAL_EVIDENCE_SCOPES:
continue
if scope in selected_set:
item_count = _manifest_scope_count(payload.get(scope))
skipped.append(
_plan_item(
scope,
"operational_evidence_not_replayed",
"Historical review or delivery evidence remains in the source package and import receipt but is never replayed as live Campaign state.",
item_count,
)
)
for scope in available:
if scope not in selected_set:
skipped.append(
_plan_item(
scope,
"scope_not_selected",
"This available package scope was not selected for import.",
_manifest_scope_count(payload.get(scope)),
)
)
campaign_metadata = configuration.get("campaign")
if not isinstance(campaign_metadata, dict):
raise CampaignTransferError("The imported Campaign metadata is invalid.")
campaign_metadata.update({"id": external_id, "name": name, "mode": "draft"})
return configuration, portable_settings, created, skipped, warnings
def _review_state_projection(
version: CampaignVersion, issues: tuple[CampaignIssue, ...]
) -> dict[str, Any]:
editor_state = version.editor_state if isinstance(version.editor_state, dict) else {}
review = editor_state.get("review_send")
review = review if isinstance(review, dict) else {}
decisions = [
item
for item in (review.get("issue_decisions") or [])
if isinstance(item, dict)
]
decision_evidence = [
{
"decision": item.get("decision"),
"issue_codes": sorted(str(code) for code in item.get("issue_codes") or []),
"issue_fingerprint": item.get("issue_fingerprint"),
"message_sha256": item.get("message_sha256"),
"reason_recorded": bool(str(item.get("reason") or "").strip()),
}
for item in decisions
]
issue_counts = Counter(str(issue.severity) for issue in issues)
return {
"workflow_state": version.workflow_state,
"inspection_complete": bool(review.get("inspection_complete")),
"reviewed_message_count": len(review.get("reviewed_message_keys") or []),
"decision_count": len(decisions),
"decision_evidence_sha256": canonical_sha256(decision_evidence),
"issue_counts": dict(sorted(issue_counts.items())),
"validation_summary": public_campaign_payload(version.validation_summary or {}),
"build_summary": public_campaign_payload(version.build_summary or {}),
}
def _delivery_job_projection(job: CampaignJob) -> dict[str, Any]:
return {
"job_id": job.id,
"entry_index": job.entry_index,
"entry_id": job.entry_id,
"recipient_email": job.recipient_email,
"message_id_header": job.message_id_header,
"message_sha256": job.eml_sha256,
"build_status": job.build_status,
"validation_status": job.validation_status,
"queue_status": job.queue_status,
"send_status": job.send_status,
"postbox_status": job.postbox_status,
"print_status": job.print_status,
"imap_status": job.imap_status,
"attempt_count": job.attempt_count,
"sent_at": _isoformat(job.sent_at),
"outcome_unknown_at": _isoformat(job.outcome_unknown_at),
"delivery_provenance": public_campaign_payload(job.delivery_provenance or {}),
}
def _delivery_counts(jobs: tuple[CampaignJob, ...]) -> dict[str, dict[str, int]]:
return {
field: dict(
sorted(Counter(str(getattr(job, field) or "unknown") for job in jobs).items())
)
for field in ("validation_status", "queue_status", "send_status")
}
def _redact_sensitive_settings(
value: Mapping[str, Any],
) -> tuple[dict[str, Any], Counter[str]]:
redactions: Counter[str] = Counter()
def visit(item: Any) -> Any:
if isinstance(item, dict):
result: dict[str, Any] = {}
for raw_key, child in item.items():
key = str(raw_key)
normalized = key.lower().replace("-", "_")
if any(fragment in normalized for fragment in _SENSITIVE_SETTING_FRAGMENTS):
redactions["sensitive_setting"] += 1
continue
result[key] = visit(child)
return result
if isinstance(item, list):
return [visit(child) for child in item]
return copy.deepcopy(item)
return visit(dict(value)), redactions
def _redact_password_field_values(
configuration: dict[str, Any],
) -> tuple[dict[str, Any], Counter[str]]:
result = copy.deepcopy(configuration)
password_fields = {
str(field.get("name"))
for field in result.get("fields") or []
if isinstance(field, dict)
and field.get("type") == "password"
and field.get("name")
}
redactions: Counter[str] = Counter()
if not password_fields:
return result, redactions
global_values = result.get("global_values")
if isinstance(global_values, dict):
for key in password_fields:
if global_values.pop(key, None) is not None:
redactions["password_field_value"] += 1
entries = result.get("entries")
if isinstance(entries, dict):
for entry in entries.get("inline") or []:
if not isinstance(entry, dict):
continue
fields = entry.get("fields")
if not isinstance(fields, dict):
continue
for key in password_fields:
if fields.pop(key, None) is not None:
redactions["password_field_value"] += 1
return result, redactions
def _remove_entry_attachments(entries: Any) -> None:
if not isinstance(entries, dict):
return
for entry in entries.get("inline") or []:
if isinstance(entry, dict):
entry["attachments"] = []
defaults = entries.get("defaults")
if isinstance(defaults, dict):
defaults["attachments"] = []
def _entry_attachment_projection(entries: Any) -> list[dict[str, Any]]:
if not isinstance(entries, dict):
return []
result: list[dict[str, Any]] = []
for index, entry in enumerate(entries.get("inline") or []):
if not isinstance(entry, dict):
continue
rules = entry.get("attachments")
if isinstance(rules, list) and rules:
result.append(
{
"entry_index": index,
"attachments": copy.deepcopy(rules),
}
)
return result
def _recipient_entry_count(entries: Any) -> int:
if not isinstance(entries, dict):
return 0
inline = entries.get("inline")
return len(inline) if isinstance(inline, list) else 0
def _attachment_rule_count(value: Any) -> int:
if not isinstance(value, dict):
return 0
configuration = value.get("configuration")
global_rules = (
configuration.get("global") if isinstance(configuration, dict) else []
)
return (len(global_rules) if isinstance(global_rules, list) else 0) + _entry_rule_count(
value.get("entry_attachments")
)
def _entry_rule_count(value: Any) -> int:
if not isinstance(value, list):
return 0
return sum(
len(item.get("attachments") or [])
for item in value
if isinstance(item, dict)
)
def _manifest_scope_count(value: Any) -> int:
if not isinstance(value, dict):
return 0
if isinstance(value.get("jobs"), list):
return len(value["jobs"])
if "decision_count" in value:
return int(value.get("decision_count") or 0)
if "entries" in value:
return _recipient_entry_count(value.get("entries"))
return 1
def _plan_item(
scope: str, code: str, summary: str, item_count: int | None
) -> dict[str, Any]:
return {
"scope": scope,
"code": code,
"summary": summary,
"item_count": item_count,
}
def _optional_text(value: object) -> str | None:
text = str(value or "").strip()
return text or None
def _isoformat(value: datetime | None) -> str | None:
return value.isoformat() if value is not None else None
__all__ = [
"CampaignImportInspection",
"CampaignTransferError",
"DEFAULT_PORTABLE_CAMPAIGN_SCOPES",
"OPERATIONAL_EVIDENCE_SCOPES",
"PORTABLE_CAMPAIGN_FORMAT",
"PORTABLE_CAMPAIGN_FORMAT_VERSION",
"PORTABLE_CAMPAIGN_SCOPE_ORDER",
"build_campaign_portable_package",
"canonical_sha256",
"inspect_campaign_portable_package",
"normalize_transfer_scopes",
]
@@ -1,14 +1,32 @@
from __future__ import annotations
import csv
from dataclasses import dataclass
from datetime import datetime
from enum import StrEnum
from pathlib import Path
from typing import Iterable
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from pydantic import BaseModel, ConfigDict, Field
from .addressing import effective_address_lists
from .field_values import ignored_entry_field_overrides
from .models import AttachmentConfig, CampaignConfig, EntryConfig, FieldType, SourceType, ZipPasswordMode, ZipPasswordScope, ZipRuleMode
from .models import (
AttachmentConfig,
CampaignConfig,
DeliveryChannelPolicy,
EntryConfig,
FieldType,
PostboxTargetConfig,
SourceType,
ZipArchiveConfig,
ZipPasswordMode,
ZipPasswordScope,
ZipRuleMode,
effective_delivery_channel_policy,
effective_postbox_targets,
)
class Severity(StrEnum):
@@ -51,6 +69,13 @@ class SemanticReport(BaseModel):
return self.error_count == 0
@dataclass(frozen=True, slots=True)
class _EntriesValidation:
mode: str
count: int | None
issues: list[SemanticIssue]
def _issue(severity: Severity, code: str, message: str, path: str | None = None) -> SemanticIssue:
return SemanticIssue(severity=severity, code=code, message=message, path=path)
@@ -80,6 +105,8 @@ def _mapping_target_known(target: str, field_names: set[str]) -> bool:
"merge_reply_to",
"merge_bounce_to",
"merge_disposition_notification_to",
"merge_postbox_targets",
"channel_policy",
"combine_to",
"combine_cc",
"combine_bcc",
@@ -200,57 +227,95 @@ def _attachment_path_issues(config: CampaignConfig) -> list[SemanticIssue]:
def _zip_configuration_issues(config: CampaignConfig) -> list[SemanticIssue]:
collection = config.attachments.zip
issues: list[SemanticIssue] = []
if not collection.enabled:
return issues
return []
if not collection.archives:
return [_issue(Severity.ERROR, "zip_archive_missing", "Attachment zipping is enabled, but no ZIP archive is configured", "/attachments/zip/archives")]
issues, archive_ids, standard_count = _zip_archive_catalog_issues(config)
if standard_count != 1:
issues.append(_issue(Severity.ERROR, "zip_standard_archive_invalid", "Exactly one ZIP archive must be selected as the campaign standard", "/attachments/zip/archives"))
issues.extend(_zip_rule_selection_issues(config, archive_ids))
return issues
def _zip_archive_catalog_issues(config: CampaignConfig) -> tuple[list[SemanticIssue], set[str], int]:
issues: list[SemanticIssue] = []
archive_ids: set[str] = set()
archive_names: set[str] = set()
standard_count = 0
field_definitions = {field.name: field for field in config.fields}
for index, archive in enumerate(collection.archives):
for index, archive in enumerate(config.attachments.zip.archives):
path = f"/attachments/zip/archives/{index}"
if not archive.id.strip():
issues.append(_issue(Severity.ERROR, "zip_archive_id_missing", "ZIP archive has no identifier", f"{path}/id"))
elif archive.id in archive_ids:
issues.append(_issue(Severity.ERROR, "zip_archive_id_duplicate", f"ZIP archive id {archive.id!r} is used more than once", f"{path}/id"))
archive_ids.add(archive.id)
normalized_archive_name = _normalized_zip_archive_name(archive.name)
if not normalized_archive_name:
issues.append(_issue(Severity.ERROR, "zip_archive_name_missing", "ZIP archive has no filename", f"{path}/name"))
elif normalized_archive_name in archive_names:
issues.append(_issue(Severity.ERROR, "zip_archive_name_duplicate", f"ZIP archive filename {archive.name!r} is used more than once; archive filenames must be unique", f"{path}/name"))
archive_names.add(normalized_archive_name)
issues.extend(_zip_archive_identity_issues(archive, path, archive_ids, archive_names))
if archive.standard:
standard_count += 1
issues.extend(_zip_password_issues(config, archive, path, field_definitions))
return issues, archive_ids, standard_count
if not archive.password_enabled:
continue
if archive.password_mode == ZipPasswordMode.DIRECT:
if not (archive.password or ""):
issues.append(_issue(Severity.ERROR, "zip_password_missing", "A legacy fixed ZIP password is enabled, but no password is configured", f"{path}/password"))
continue
if archive.password_mode == ZipPasswordMode.TEMPLATE:
if not (archive.password_template or ""):
issues.append(_issue(Severity.ERROR, "zip_password_template_missing", "A legacy ZIP password template is enabled, but no template is configured", f"{path}/password_template"))
continue
field_name = (archive.password_field or "").strip()
field = field_definitions.get(field_name)
if not field_name:
issues.append(_issue(Severity.ERROR, "zip_password_field_missing", f"ZIP archive {archive.name!r} has password protection enabled, but no field is selected", f"{path}/password_field"))
elif field is None:
issues.append(_issue(Severity.ERROR, "zip_password_field_unknown", f"ZIP password field {field_name!r} is not declared in campaign fields", f"{path}/password_field"))
elif field.type != FieldType.PASSWORD:
issues.append(_issue(Severity.WARNING, "zip_password_field_not_password_type", f"ZIP password field {field_name!r} is not configured with field type 'password'", f"{path}/password_field"))
elif archive.password_scope == ZipPasswordScope.GLOBAL and config.global_values.get(field_name) in (None, ""):
issues.append(_issue(Severity.ERROR, "zip_global_password_value_missing", f"Global ZIP password field {field_name!r} has no campaign-wide value", f"/global_values/{field_name}"))
def _zip_archive_identity_issues(
archive: ZipArchiveConfig,
path: str,
archive_ids: set[str],
archive_names: set[str],
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
if not archive.id.strip():
issues.append(_issue(Severity.ERROR, "zip_archive_id_missing", "ZIP archive has no identifier", f"{path}/id"))
elif archive.id in archive_ids:
issues.append(_issue(Severity.ERROR, "zip_archive_id_duplicate", f"ZIP archive id {archive.id!r} is used more than once", f"{path}/id"))
archive_ids.add(archive.id)
if standard_count != 1:
issues.append(_issue(Severity.ERROR, "zip_standard_archive_invalid", "Exactly one ZIP archive must be selected as the campaign standard", "/attachments/zip/archives"))
normalized_archive_name = _normalized_zip_archive_name(archive.name)
if not normalized_archive_name:
issues.append(_issue(Severity.ERROR, "zip_archive_name_missing", "ZIP archive has no filename", f"{path}/name"))
elif normalized_archive_name in archive_names:
issues.append(_issue(Severity.ERROR, "zip_archive_name_duplicate", f"ZIP archive filename {archive.name!r} is used more than once; archive filenames must be unique", f"{path}/name"))
archive_names.add(normalized_archive_name)
return issues
def _zip_password_issues(
config: CampaignConfig,
archive: ZipArchiveConfig,
path: str,
field_definitions: dict[str, object],
) -> list[SemanticIssue]:
if not archive.password_enabled:
return []
if archive.password_mode == ZipPasswordMode.DIRECT:
if not (archive.password or ""):
return [_issue(Severity.ERROR, "zip_password_missing", "A legacy fixed ZIP password is enabled, but no password is configured", f"{path}/password")]
return []
if archive.password_mode == ZipPasswordMode.TEMPLATE:
if not (archive.password_template or ""):
return [_issue(Severity.ERROR, "zip_password_template_missing", "A legacy ZIP password template is enabled, but no template is configured", f"{path}/password_template")]
return []
return _zip_password_field_issues(config, archive, path, field_definitions)
def _zip_password_field_issues(
config: CampaignConfig,
archive: ZipArchiveConfig,
path: str,
field_definitions: dict[str, object],
) -> list[SemanticIssue]:
field_name = (archive.password_field or "").strip()
field = field_definitions.get(field_name)
if not field_name:
return [_issue(Severity.ERROR, "zip_password_field_missing", f"ZIP archive {archive.name!r} has password protection enabled, but no field is selected", f"{path}/password_field")]
if field is None:
return [_issue(Severity.ERROR, "zip_password_field_unknown", f"ZIP password field {field_name!r} is not declared in campaign fields", f"{path}/password_field")]
if getattr(field, "type", None) != FieldType.PASSWORD:
return [_issue(Severity.WARNING, "zip_password_field_not_password_type", f"ZIP password field {field_name!r} is not configured with field type 'password'", f"{path}/password_field")]
if archive.password_scope == ZipPasswordScope.GLOBAL and config.global_values.get(field_name) in (None, ""):
return [_issue(Severity.ERROR, "zip_global_password_value_missing", f"Global ZIP password field {field_name!r} has no campaign-wide value", f"/global_values/{field_name}")]
return []
def _zip_rule_selection_issues(config: CampaignConfig, archive_ids: set[str]) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
for path, rule, _is_individual in _iter_attachment_rules(config):
selection = (rule.zip.archive_id or ZipRuleMode.INHERIT.value).strip()
if selection not in {"", ZipRuleMode.INHERIT.value, ZipRuleMode.INCLUDE.value, ZipRuleMode.EXCLUDE.value} and selection not in archive_ids:
@@ -276,11 +341,599 @@ def _ignored_override_issues(config: CampaignConfig, entry: EntryConfig, path_pr
]
def _global_value_issues(config: CampaignConfig, declared_names: set[str]) -> list[SemanticIssue]:
return [
_issue(
Severity.WARNING,
"unknown_global_value",
f"global_values contains {key!r}, but it is not declared in fields",
f"/global_values/{key}",
)
for key in config.global_values
if declared_names and key not in declared_names
]
def _active_delivery_entries(config: CampaignConfig) -> list[EntryConfig]:
if config.entries.is_inline:
return [
entry
for entry in (config.entries.inline or [])
if entry.active
]
return [config.entries.defaults or EntryConfig()]
def _delivery_policies(config: CampaignConfig) -> set[DeliveryChannelPolicy]:
return {
effective_delivery_channel_policy(config, entry)
for entry in _active_delivery_entries(config)
}
def _postbox_target_field_issues(
config: CampaignConfig,
target: PostboxTargetConfig,
path: str,
) -> list[SemanticIssue]:
definitions = {field.name: field for field in config.fields}
checks = (
(
target.organization_unit_field,
FieldType.ORGANIZATION_UNIT,
"organization unit",
"organization_unit_field",
),
(
target.function_field,
FieldType.ORGANIZATION_FUNCTION,
"organization function",
"function_field",
),
(target.context_field, None, "context", "context_field"),
)
issues: list[SemanticIssue] = []
for field_name, expected_type, label, key in checks:
if not field_name:
continue
definition = definitions.get(field_name)
if definition is None:
issues.append(
_issue(
Severity.ERROR,
"postbox_target_field_missing",
f"Postbox {label} field {field_name!r} is not declared.",
f"{path}/{key}",
)
)
elif expected_type is not None and definition.type != expected_type:
issues.append(
_issue(
Severity.WARNING,
"postbox_target_field_type",
(
f"Postbox {label} field {field_name!r} should use "
f"field type {expected_type.value!r}."
),
f"{path}/{key}",
)
)
return issues
def _postbox_delivery_issues(
config: CampaignConfig,
*,
postbox_available: bool,
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
policies = _delivery_policies(config)
if not any(policy.uses_postbox for policy in policies):
return issues
if not postbox_available:
issues.append(
_issue(
Severity.ERROR,
"postbox_unavailable",
(
"This campaign uses Postbox delivery, but the Postbox "
"module and its delivery directory are not active."
),
"/delivery/channel_policy",
)
)
for entry_index, entry in enumerate(_active_delivery_entries(config)):
policy = effective_delivery_channel_policy(config, entry)
if not policy.uses_postbox:
continue
targets = effective_postbox_targets(config, entry)
if not targets:
issues.append(
_issue(
Severity.ERROR,
"postbox_target_missing",
"Postbox delivery requires at least one target.",
f"/entries/inline/{entry_index}/postbox_targets",
)
)
continue
seen_ids: set[str] = set()
for target_index, target in enumerate(targets):
target_path = (
f"/entries/inline/{entry_index}/postbox_targets/"
f"{target_index}"
)
if target.id in seen_ids:
issues.append(
_issue(
Severity.WARNING,
"postbox_target_id_duplicate",
f"Postbox target id {target.id!r} is repeated.",
f"{target_path}/id",
)
)
seen_ids.add(target.id)
issues.extend(
_postbox_target_field_issues(config, target, target_path)
)
return issues
def _delivery_issues(
config: CampaignConfig,
*,
postbox_available: bool,
templates_available: bool,
calendar_available: bool,
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
policies = _delivery_policies(config)
uses_mail = any(policy.uses_mail for policy in policies)
profile_id = (config.server.mail_profile_id or "").strip()
if (
(
config.campaign.mode == "send"
and uses_mail
or config.delivery.imap_append_sent.enabled
)
and not profile_id
):
issues.append(
_issue(
Severity.ERROR,
"missing_mail_profile",
"Select an authorized Mail-module profile; campaigns cannot store SMTP/IMAP settings or credentials.",
"/server/mail_profile_id",
)
)
capabilities = config.server.profile_capabilities
if (
config.campaign.mode == "send"
and uses_mail
and profile_id
and not capabilities.smtp_available
):
issues.append(
_issue(
Severity.ERROR,
"mail_profile_without_smtp",
"campaign mode is 'send', but the selected Mail profile has no SMTP configuration",
"/server/mail_profile_id",
)
)
if config.delivery.imap_append_sent.enabled and profile_id and not capabilities.imap_available:
issues.append(
_issue(
Severity.ERROR,
"mail_profile_without_imap",
"IMAP append is enabled, but the selected Mail profile has no IMAP configuration",
"/server/mail_profile_id",
)
)
issues.extend(
_postbox_delivery_issues(
config,
postbox_available=postbox_available,
)
)
if any(policy.uses_print for policy in policies):
if not templates_available:
issues.append(
_issue(
Severity.ERROR,
"templates_unavailable",
"Printable Campaign delivery requires the optional Templates renderer.",
"/delivery/print/template_id",
)
)
if not config.delivery.print.template_id:
issues.append(
_issue(
Severity.ERROR,
"print_template_missing",
"Select a published compatible template for printable Campaign output.",
"/delivery/print/template_id",
)
)
elif config.delivery.print.template_revision is None:
issues.append(
_issue(
Severity.ERROR,
"print_template_revision_missing",
"Printable delivery must pin one published template revision.",
"/delivery/print/template_revision",
)
)
for entry_index, entry in enumerate(_active_delivery_entries(config)):
if not effective_delivery_channel_policy(config, entry).uses_print:
continue
if entry.print_target is None:
issues.append(
_issue(
Severity.ERROR,
"print_target_missing",
"Printable delivery requires an explicit postal or internal-mail target.",
f"/entries/inline/{entry_index}/print_target",
)
)
invitation = config.delivery.calendar_invitation
if invitation.enabled:
if not calendar_available:
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_unavailable",
"Calendar invitations require the optional Calendar module.",
"/delivery/calendar_invitation/enabled",
)
)
if not policies or any(not policy.uses_mail for policy in policies):
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_requires_mail",
"Calendar invitations require Mail delivery for every active recipient.",
"/delivery/channel_policy",
)
)
if not invitation.calendar_id:
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_calendar_missing",
"Select a writable calendar for campaign invitation tracking.",
"/delivery/calendar_invitation/calendar_id",
)
)
if not invitation.start_at_template:
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_start_missing",
"Calendar invitations require a start date and time or a field template.",
"/delivery/calendar_invitation/start_at_template",
)
)
if invitation.timezone:
try:
ZoneInfo(invitation.timezone)
except ZoneInfoNotFoundError:
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_timezone_invalid",
f"Unknown calendar invitation timezone: {invitation.timezone}",
"/delivery/calendar_invitation/timezone",
)
)
valid_start, fixed_start = _fixed_invitation_datetime(
invitation.start_at_template
)
valid_end, fixed_end = _fixed_invitation_datetime(invitation.end_at_template)
if invitation.start_at_template and not valid_start:
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_start_invalid",
"The fixed invitation start must be ISO 8601; recipient field templates are also supported.",
"/delivery/calendar_invitation/start_at_template",
)
)
if invitation.end_at_template and not valid_end:
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_end_invalid",
"The fixed invitation end must be ISO 8601; recipient field templates are also supported.",
"/delivery/calendar_invitation/end_at_template",
)
)
if fixed_start and fixed_end and _invitation_range_invalid(
fixed_start,
fixed_end,
):
issues.append(
_issue(
Severity.ERROR,
"calendar_invitation_range_invalid",
"Calendar invitation end must be after its start.",
"/delivery/calendar_invitation/end_at_template",
)
)
return issues
def _fixed_invitation_datetime(value: str | None) -> tuple[bool, datetime | None]:
if not value:
return True, None
if "${" in value or "{{" in value:
return True, None
try:
return True, datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return False, None
def _invitation_range_invalid(start_at: datetime, end_at: datetime) -> bool:
if (start_at.tzinfo is None) != (end_at.tzinfo is None):
return True
return end_at <= start_at
def _sender_issues(config: CampaignConfig) -> list[SemanticIssue]:
"""Require Campaign-owned sender data before a send-mode build."""
if (
config.campaign.mode != "send"
or not any(policy.uses_mail for policy in _delivery_policies(config))
):
return []
if config.entries.is_inline:
return [
_issue(
Severity.ERROR,
"missing_sender",
"No effective From address is configured; Campaign must resolve the sender before building.",
f"/entries/inline/{index}/from",
)
for index, entry in enumerate(config.entries.inline or [])
if (
entry.active
and effective_delivery_channel_policy(config, entry).uses_mail
and not effective_address_lists(config, entry)["from"]
)
]
if config.recipients.from_:
return []
defaults = config.entries.defaults
mapping_can_supply_sender = bool(
config.recipients.allow_individual_from
and (
(defaults is not None and defaults.from_)
or "from.email" in (config.entries.mapping or {})
)
)
if mapping_can_supply_sender:
return []
return [
_issue(
Severity.ERROR,
"missing_sender",
"Configure recipients.from, or allow and map an individual From address, before building a send-mode campaign.",
"/recipients/from",
)
]
def _entries_validation(
config: CampaignConfig,
*,
campaign_path: Path,
check_files: bool,
field_names: set[str],
field_definitions: dict[str, object],
) -> _EntriesValidation:
if config.entries.is_inline:
inline_entries = config.entries.inline or []
issues = _inline_entries_issues(config, inline_entries)
return _EntriesValidation(mode="inline", count=len(inline_entries), issues=issues)
issues = _external_entries_issues(
config,
campaign_path=campaign_path,
check_files=check_files,
field_names=field_names,
field_definitions=field_definitions,
)
source_type = config.entries.source.type.value if config.entries.source else "unknown"
return _EntriesValidation(mode=f"external:{source_type}", count=None, issues=issues)
def _inline_entries_issues(config: CampaignConfig, inline_entries: list[EntryConfig]) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
if not inline_entries:
issues.append(_issue(Severity.WARNING, "no_inline_entries", "entries.inline is empty", "/entries/inline"))
for index, entry in enumerate(inline_entries):
if entry.active:
issues.extend(_ignored_override_issues(config, entry, f"/entries/inline/{index}"))
return issues
def _external_entries_issues(
config: CampaignConfig,
*,
campaign_path: Path,
check_files: bool,
field_names: set[str],
field_definitions: dict[str, object],
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
mapping = config.entries.mapping or {}
if not mapping:
issues.append(_issue(Severity.ERROR, "empty_mapping", "external entries require a non-empty mapping", "/entries/mapping"))
issues.extend(_mapping_issues(mapping, field_names, field_definitions))
if config.entries.defaults:
issues.extend(_ignored_override_issues(config, config.entries.defaults, "/entries/defaults"))
if check_files and config.entries.source:
issues.extend(_entries_source_file_issues(config, campaign_path, mapping))
return issues
def _mapping_issues(
mapping: dict[str, str],
field_names: set[str],
field_definitions: dict[str, object],
) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
for target in mapping:
if not _mapping_target_known(target, field_names):
issues.append(
_issue(
Severity.WARNING,
"unknown_mapping_target",
f"mapping target {target!r} is not recognized by the current campaign model",
f"/entries/mapping/{target}",
)
)
field_name = _mapping_target_field_name(target)
field = field_definitions.get(field_name or "")
if field_name and field is not None and not getattr(field, "can_override", True):
issues.append(
_issue(
Severity.WARNING,
"mapping_target_not_overridable",
f"mapping target {target!r} points to a field that does not allow recipient overrides; mapped values will be ignored",
f"/entries/mapping/{target}",
)
)
return issues
def _entries_source_file_issues(config: CampaignConfig, campaign_path: Path, mapping: dict[str, str]) -> list[SemanticIssue]:
source = config.entries.source
if source is None:
return []
source_path = _resolve(campaign_path, source.path)
if not source_path.exists():
return [
_issue(
Severity.ERROR,
"entries_source_not_found",
f"entries source file does not exist: {source_path}",
"/entries/source/path",
)
]
if source.type != SourceType.CSV or not source.has_header:
return []
try:
header = _csv_header(source_path, source.delimiter, source.encoding)
except OSError as exc:
return [_issue(Severity.ERROR, "entries_source_read_error", str(exc), "/entries/source/path")]
header_set = set(header or [])
missing_columns = sorted({source_name for source_name in mapping.values() if source_name not in header_set})
if not missing_columns:
return []
return [
_issue(
Severity.ERROR,
"mapping_columns_missing",
"CSV mapping refers to missing columns: " + ", ".join(missing_columns),
"/entries/mapping",
)
]
def _file_check_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
issues.extend(_attachment_file_check_issues(config, campaign_path))
issues.extend(_attachment_resolution_check_issues(config, campaign_path))
issues.extend(_template_source_file_issues(config, campaign_path))
return issues
def _attachment_file_check_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
if config.attachments.base_paths:
return [
_issue(
Severity.WARNING,
"attachments_base_path_not_found",
f"attachment base path {base_path_config.name!r} does not exist: {_resolve(campaign_path, base_path_config.path)}",
f"/attachments/base_paths/{index}/path",
)
for index, base_path_config in enumerate(config.attachments.base_paths)
if not _resolve(campaign_path, base_path_config.path).exists()
]
attachments_base_path = _resolve(campaign_path, config.attachments.base_path)
if attachments_base_path.exists():
return []
return [
_issue(
Severity.WARNING,
"attachments_base_path_not_found",
f"attachments.base_path does not exist: {attachments_base_path}",
"/attachments/base_path",
)
]
def _attachment_resolution_check_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
# The resolver consumes campaign models/entries. Import it only when file
# validation is requested so either public entry point can initialize first.
from ..attachments.resolver import resolve_campaign_attachments
try:
report = resolve_campaign_attachments(config, campaign_file=campaign_path)
except Exception as exc:
return [
_issue(
Severity.ERROR,
"attachment_resolution_failed",
f"attachment rules could not be resolved: {exc}",
"/attachments",
)
]
issues: list[SemanticIssue] = []
for entry in report.entries:
entry_path = f"/entries/{entry.entry_id or entry.entry_index}"
for issue in entry.issues:
if any(issue is attachment_issue for attachment in entry.attachments for attachment_issue in attachment.issues):
continue
issues.append(_issue(Severity(issue.severity.value), issue.code, issue.message, entry_path))
for attachment in entry.attachments:
attachment_path = (
f"/attachments/global/{attachment.index}"
if attachment.scope.value == "global"
else f"{entry_path}/attachments/{attachment.index}"
)
for issue in attachment.issues:
issues.append(_issue(Severity(issue.severity.value), issue.code, issue.message, attachment_path))
return issues
def _template_source_file_issues(config: CampaignConfig, campaign_path: Path) -> list[SemanticIssue]:
issues: list[SemanticIssue] = []
for schema_path, raw_path in _iter_template_source_paths(config):
path = _resolve(campaign_path, raw_path)
if not path.exists():
issues.append(
_issue(
Severity.ERROR,
"template_source_not_found",
f"template source file does not exist: {path}",
schema_path,
)
)
return issues
def validate_campaign_config(
config: CampaignConfig,
*,
campaign_file: str | Path | None = None,
check_files: bool = False,
postbox_available: bool = False,
templates_available: bool = False,
calendar_available: bool = False,
) -> SemanticReport:
campaign_path = Path(campaign_file).resolve() if campaign_file else Path.cwd() / "campaign.json"
issues: list[SemanticIssue] = []
@@ -289,149 +942,37 @@ def validate_campaign_config(
field_definitions = {field.name: field for field in config.fields}
declared_names = set(field_definitions)
for key in config.global_values:
if declared_names and key not in declared_names:
issues.append(_issue(
Severity.WARNING,
"unknown_global_value",
f"global_values contains {key!r}, but it is not declared in fields",
f"/global_values/{key}",
))
issues.extend(_global_value_issues(config, declared_names))
issues.extend(_attachment_path_issues(config))
issues.extend(_zip_configuration_issues(config))
issues.extend(
_delivery_issues(
config,
postbox_available=postbox_available,
templates_available=templates_available,
calendar_available=calendar_available,
)
)
issues.extend(_sender_issues(config))
runtime_imap = config.server.runtime_imap_config()
if config.delivery.imap_append_sent.enabled:
if runtime_imap is None:
issues.append(_issue(
Severity.WARNING,
"delivery_imap_enabled_without_server_imap",
"delivery.imap_append_sent is enabled, but no server.imap configuration is present",
"/delivery/imap_append_sent/enabled",
))
else:
missing = [name for name in ["host", "port", "username", "password"] if getattr(runtime_imap, name) in (None, "")]
if missing:
issues.append(_issue(
Severity.ERROR,
"incomplete_imap_config",
"IMAP append is enabled, but these IMAP settings are missing: " + ", ".join(missing),
"/server/imap",
))
runtime_smtp = config.server.runtime_smtp_config()
if config.campaign.mode == "send" and not runtime_smtp:
issues.append(_issue(
Severity.ERROR,
"missing_smtp_config",
"campaign mode is 'send', but no server.smtp configuration is present",
"/server/smtp",
))
if runtime_smtp:
missing = [name for name in ["host", "port"] if getattr(runtime_smtp, name) in (None, "")]
if missing:
issues.append(_issue(
Severity.WARNING,
"incomplete_smtp_config",
"SMTP settings are present, but these settings are missing: " + ", ".join(missing),
"/server/smtp",
))
if config.entries.is_inline:
inline_entries = config.entries.inline or []
entries_count = len(inline_entries)
entries_mode = "inline"
if entries_count == 0:
issues.append(_issue(Severity.WARNING, "no_inline_entries", "entries.inline is empty", "/entries/inline"))
for index, entry in enumerate(inline_entries):
if entry.active:
issues.extend(_ignored_override_issues(config, entry, f"/entries/inline/{index}"))
else:
entries_count = None
entries_mode = f"external:{config.entries.source.type.value if config.entries.source else 'unknown'}"
mapping = config.entries.mapping or {}
if not mapping:
issues.append(_issue(Severity.ERROR, "empty_mapping", "external entries require a non-empty mapping", "/entries/mapping"))
for target in mapping:
if not _mapping_target_known(target, field_names):
issues.append(_issue(
Severity.WARNING,
"unknown_mapping_target",
f"mapping target {target!r} is not recognized by the current campaign model",
f"/entries/mapping/{target}",
))
field_name = _mapping_target_field_name(target)
if field_name and field_name in field_definitions and not field_definitions[field_name].can_override:
issues.append(_issue(
Severity.WARNING,
"mapping_target_not_overridable",
f"mapping target {target!r} points to a field that does not allow recipient overrides; mapped values will be ignored",
f"/entries/mapping/{target}",
))
if config.entries.defaults:
issues.extend(_ignored_override_issues(config, config.entries.defaults, "/entries/defaults"))
if check_files and config.entries.source:
source_path = _resolve(campaign_path, config.entries.source.path)
if not source_path.exists():
issues.append(_issue(
Severity.ERROR,
"entries_source_not_found",
f"entries source file does not exist: {source_path}",
"/entries/source/path",
))
elif config.entries.source.type == SourceType.CSV and config.entries.source.has_header:
try:
header = _csv_header(source_path, config.entries.source.delimiter, config.entries.source.encoding)
header_set = set(header or [])
missing_columns = sorted({source_name for source_name in mapping.values() if source_name not in header_set})
if missing_columns:
issues.append(_issue(
Severity.ERROR,
"mapping_columns_missing",
"CSV mapping refers to missing columns: " + ", ".join(missing_columns),
"/entries/mapping",
))
except OSError as exc:
issues.append(_issue(Severity.ERROR, "entries_source_read_error", str(exc), "/entries/source/path"))
entries = _entries_validation(
config,
campaign_path=campaign_path,
check_files=check_files,
field_names=field_names,
field_definitions=field_definitions,
)
issues.extend(entries.issues)
if check_files:
if config.attachments.base_paths:
for index, base_path_config in enumerate(config.attachments.base_paths):
attachments_base_path = _resolve(campaign_path, base_path_config.path)
if not attachments_base_path.exists():
issues.append(_issue(
Severity.WARNING,
"attachments_base_path_not_found",
f"attachment base path {base_path_config.name!r} does not exist: {attachments_base_path}",
f"/attachments/base_paths/{index}/path",
))
else:
attachments_base_path = _resolve(campaign_path, config.attachments.base_path)
if not attachments_base_path.exists():
issues.append(_issue(
Severity.WARNING,
"attachments_base_path_not_found",
f"attachments.base_path does not exist: {attachments_base_path}",
"/attachments/base_path",
))
for schema_path, raw_path in _iter_template_source_paths(config):
path = _resolve(campaign_path, raw_path)
if not path.exists():
issues.append(_issue(
Severity.ERROR,
"template_source_not_found",
f"template source file does not exist: {path}",
schema_path,
))
issues.extend(_file_check_issues(config, campaign_path))
report = SemanticReport(
campaign_id=config.campaign.id,
campaign_name=config.campaign.name,
issues=issues,
entries_mode=entries_mode,
entries_count=entries_count,
entries_mode=entries.mode,
entries_count=entries.count,
attachments_base_path=_attachment_base_path_report_value(config),
rate_limit=f"{config.delivery.rate_limit.messages_per_minute}/min, concurrency {config.delivery.rate_limit.concurrency}",
imap_append_enabled=config.delivery.imap_append_sent.enabled,
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,371 @@
from __future__ import annotations
from datetime import datetime
from typing import Any
from sqlalchemy import event
from sqlalchemy.orm import Session as OrmSession
from govoplan_core.core.change_sequence import record_change
from govoplan_core.core.sqlalchemy_change_tracking import (
ensure_object_id,
has_attr_changes,
object_state,
operation_for_object,
previous_value,
)
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignIssue,
CampaignJob,
CampaignShare,
CampaignVersion,
ImapAppendAttempt,
PostboxDeliveryAttempt,
PrintOutputAttempt,
SendAttempt,
new_uuid,
)
CAMPAIGNS_MODULE_ID = "campaigns"
CAMPAIGNS_COLLECTION = "campaigns.campaigns"
CAMPAIGN_VERSIONS_COLLECTION = "campaigns.versions"
CAMPAIGN_JOBS_COLLECTION = "campaigns.jobs"
CAMPAIGN_ISSUES_COLLECTION = "campaigns.issues"
CAMPAIGN_ATTEMPTS_COLLECTION = "campaigns.attempts"
_REGISTERED = False
def register_campaign_change_tracking() -> None:
global _REGISTERED
if _REGISTERED:
return
event.listen(OrmSession, "before_flush", _record_campaign_changes)
_REGISTERED = True
def _record_campaign_changes(session: OrmSession, _flush_context: object, _instances: object) -> None:
for obj in tuple(session.new) + tuple(session.dirty) + tuple(session.deleted):
if isinstance(obj, Campaign):
_record_campaign_change(session, obj)
elif isinstance(obj, CampaignShare):
_record_share_visibility_change(session, obj)
elif isinstance(obj, CampaignVersion):
_record_version_change(session, obj)
elif isinstance(obj, CampaignJob):
_record_job_change(session, obj)
elif isinstance(obj, CampaignIssue):
_record_issue_change(session, obj)
elif isinstance(
obj,
(SendAttempt, ImapAppendAttempt, PostboxDeliveryAttempt, PrintOutputAttempt),
):
_record_attempt_change(session, obj)
def _record_campaign_change(session: OrmSession, campaign: Campaign) -> None:
operation = _operation_for_campaign(
campaign,
changed_attrs=(
"external_id",
"name",
"description",
"status",
"current_version_id",
"owner_user_id",
"owner_group_id",
"settings",
"mail_profile_policy",
),
)
if operation is None:
return
campaign_id = _ensure_id(campaign)
record_change(
session,
module_id=CAMPAIGNS_MODULE_ID,
collection=CAMPAIGNS_COLLECTION,
resource_type="campaign",
resource_id=campaign_id,
operation=operation,
tenant_id=campaign.tenant_id,
actor_type="user" if campaign.created_by_user_id else None,
actor_id=campaign.created_by_user_id,
payload=_campaign_payload(campaign),
)
def _record_share_visibility_change(session: OrmSession, share: CampaignShare) -> None:
state = object_state(share)
if not share.campaign_id:
return
if not state.deleted and not state.pending and not has_attr_changes(
state,
("campaign_id", "target_type", "target_id", "permission", "revoked_at"),
):
return
_ensure_id(share)
operation = "deleted" if state.deleted or share.revoked_at is not None else "updated"
campaign = session.get(Campaign, share.campaign_id)
payload = _campaign_payload(campaign) if campaign is not None else {"campaign_id": share.campaign_id}
payload.update({
"share_id": share.id,
"share_target_type": share.target_type,
"share_target_id": share.target_id,
"share_permission": share.permission,
"share_revoked_at": _isoformat(share.revoked_at),
})
record_change(
session,
module_id=CAMPAIGNS_MODULE_ID,
collection=CAMPAIGNS_COLLECTION,
resource_type="campaign",
resource_id=share.campaign_id,
operation=operation,
tenant_id=share.tenant_id,
actor_type="user" if share.created_by_user_id else None,
actor_id=share.created_by_user_id,
payload=payload,
)
def _record_version_change(session: OrmSession, version: CampaignVersion) -> None:
operation = _operation_for_object(
version,
changed_attrs=(
"raw_json",
"schema_version",
"source_filename",
"source_base_path",
"workflow_state",
"current_flow",
"current_step",
"is_complete",
"editor_state",
"autosaved_at",
"published_at",
"locked_at",
"locked_by_user_id",
"user_lock_state",
"user_locked_at",
"user_locked_by_user_id",
"validation_summary",
"build_summary",
"execution_snapshot_hash",
"execution_snapshot_at",
),
)
if operation is None:
return
version_id = _ensure_id(version)
campaign = session.get(Campaign, version.campaign_id) if version.campaign_id else None
record_change(
session,
module_id=CAMPAIGNS_MODULE_ID,
collection=CAMPAIGN_VERSIONS_COLLECTION,
resource_type="campaign_version",
resource_id=version_id,
operation=operation,
tenant_id=campaign.tenant_id if campaign is not None else None,
payload={
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": version.campaign_id}),
"version_id": version_id,
"version_number": version.version_number,
"workflow_state": version.workflow_state,
"current_flow": version.current_flow,
"current_step": version.current_step,
},
)
def _record_job_change(session: OrmSession, job: CampaignJob) -> None:
operation = _operation_for_object(
job,
changed_attrs=(
"build_status",
"validation_status",
"queue_status",
"send_status",
"delivery_channel_policy",
"postbox_status",
"print_status",
"imap_status",
"attempt_count",
"postbox_attempt_count",
"print_attempt_count",
"last_error",
"queued_at",
"claimed_at",
"smtp_started_at",
"outcome_unknown_at",
"sent_at",
"resolved_recipients",
"delivery_provenance",
"resolved_postbox_targets",
"resolved_print_output",
"resolved_attachments",
"issues_snapshot",
),
)
if operation is None:
return
job_id = _ensure_id(job)
campaign = session.get(Campaign, job.campaign_id) if job.campaign_id else None
record_change(
session,
module_id=CAMPAIGNS_MODULE_ID,
collection=CAMPAIGN_JOBS_COLLECTION,
resource_type="campaign_job",
resource_id=job_id,
operation=operation,
tenant_id=job.tenant_id,
payload={
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}),
"job_id": job_id,
"version_id": job.campaign_version_id,
"entry_index": job.entry_index,
"entry_id": job.entry_id,
"build_status": job.build_status,
"validation_status": job.validation_status,
"queue_status": job.queue_status,
"send_status": job.send_status,
"delivery_channel_policy": job.delivery_channel_policy,
"postbox_status": job.postbox_status,
"print_status": job.print_status,
"imap_status": job.imap_status,
},
)
def _record_issue_change(session: OrmSession, issue: CampaignIssue) -> None:
operation = _operation_for_object(issue, changed_attrs=("severity", "code", "message", "source", "behavior"))
if operation is None:
return
issue_id = _ensure_id(issue)
campaign = session.get(Campaign, issue.campaign_id) if issue.campaign_id else None
record_change(
session,
module_id=CAMPAIGNS_MODULE_ID,
collection=CAMPAIGN_ISSUES_COLLECTION,
resource_type="campaign_issue",
resource_id=issue_id,
operation=operation,
tenant_id=issue.tenant_id,
payload={
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": issue.campaign_id}),
"issue_id": issue_id,
"version_id": issue.campaign_version_id,
"job_id": issue.job_id,
"severity": issue.severity,
"code": issue.code,
},
)
def _record_attempt_change(
session: OrmSession,
attempt: SendAttempt | ImapAppendAttempt | PostboxDeliveryAttempt | PrintOutputAttempt,
) -> None:
operation = _operation_for_object(
attempt,
changed_attrs=(
"status",
"claim_token",
"smtp_status_code",
"smtp_response",
"error_type",
"error_message",
"folder",
"provider_delivery_id",
"provider_message_id",
"postbox_id",
"render_id",
"artifact_sha256",
"evidence",
),
)
if operation is None:
return
attempt_id = _ensure_id(attempt)
job = session.get(CampaignJob, attempt.job_id) if attempt.job_id else None
if job is None:
return
campaign = session.get(Campaign, job.campaign_id) if job.campaign_id else None
record_change(
session,
module_id=CAMPAIGNS_MODULE_ID,
collection=CAMPAIGN_ATTEMPTS_COLLECTION,
resource_type="campaign_attempt",
resource_id=attempt_id,
operation=operation,
tenant_id=job.tenant_id,
payload={
**(_campaign_payload(campaign) if campaign is not None else {"campaign_id": job.campaign_id}),
"attempt_id": attempt_id,
"attempt_kind": (
"postbox"
if isinstance(attempt, PostboxDeliveryAttempt)
else "print"
if isinstance(attempt, PrintOutputAttempt)
else "imap"
if isinstance(attempt, ImapAppendAttempt)
else "smtp"
),
"job_id": job.id,
"version_id": job.campaign_version_id,
},
)
def _operation_for_campaign(obj: Campaign, *, changed_attrs: tuple[str, ...]) -> str | None:
state = object_state(obj)
if state.pending:
return "created"
if state.deleted:
return "deleted"
if not has_attr_changes(state, changed_attrs):
return None
status_history = state.attrs.status.history
if status_history.has_changes() and any(value == "deleted" for value in status_history.added):
return "deleted"
return "updated"
def _operation_for_object(obj: object, *, changed_attrs: tuple[str, ...]) -> str | None:
return operation_for_object(obj, changed_attrs=changed_attrs)
def _ensure_id(obj: object) -> str:
return ensure_object_id(obj, new_uuid)
def _campaign_payload(campaign: Campaign | None) -> dict[str, Any]:
if campaign is None:
return {}
return {
"campaign_id": campaign.id,
"owner_user_id": campaign.owner_user_id,
"owner_group_id": campaign.owner_group_id,
"previous_owner_user_id": previous_value(campaign, "owner_user_id"),
"previous_owner_group_id": previous_value(campaign, "owner_group_id"),
"status": campaign.status,
"previous_status": previous_value(campaign, "status"),
"current_version_id": campaign.current_version_id,
"previous_current_version_id": previous_value(campaign, "current_version_id"),
}
def _isoformat(value: datetime | None) -> str | None:
return value.isoformat() if value else None
__all__ = [
"CAMPAIGNS_COLLECTION",
"CAMPAIGNS_MODULE_ID",
"CAMPAIGN_ATTEMPTS_COLLECTION",
"CAMPAIGN_ISSUES_COLLECTION",
"CAMPAIGN_JOBS_COLLECTION",
"CAMPAIGN_VERSIONS_COLLECTION",
"register_campaign_change_tracking",
]
+656 -19
View File
@@ -8,6 +8,7 @@ from typing import Any
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, JSON, String, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column, relationship
from govoplan_core.core.concurrency import strong_resource_etag
from govoplan_core.db.base import Base, TimestampMixin
@@ -78,10 +79,15 @@ class JobQueueStatus(StrEnum):
class JobSendStatus(StrEnum):
NOT_QUEUED = "not_queued"
SKIPPED = "skipped"
QUEUED = "queued"
CLAIMED = "claimed"
SENDING = "sending"
SMTP_ACCEPTED = "smtp_accepted"
POSTBOX_ACCEPTED = "postbox_accepted"
PRINT_ACCEPTED = "print_accepted"
DELIVERED = "delivered"
PARTIALLY_ACCEPTED = "partially_accepted"
SENT = "sent" # legacy value retained for existing databases/reports
OUTCOME_UNKNOWN = "outcome_unknown"
FAILED_TEMPORARY = "failed_temporary"
@@ -89,10 +95,34 @@ class JobSendStatus(StrEnum):
CANCELLED = "cancelled"
class JobPostboxStatus(StrEnum):
NOT_REQUESTED = "not_requested"
PENDING = "pending"
DELIVERING = "delivering"
ACCEPTED = "accepted"
ACCEPTED_VACANT = "accepted_vacant"
PARTIALLY_ACCEPTED = "partially_accepted"
REJECTED_TEMPORARY = "rejected_temporary"
REJECTED_PERMANENT = "rejected_permanent"
OUTCOME_UNKNOWN = "outcome_unknown"
SKIPPED = "skipped"
class JobPrintStatus(StrEnum):
NOT_REQUESTED = "not_requested"
READY = "ready"
ACCEPTING = "accepting"
ACCEPTED = "accepted"
FAILED = "failed"
SKIPPED = "skipped"
class JobImapStatus(StrEnum):
NOT_REQUESTED = "not_requested"
PENDING = "pending"
APPENDING = "appending"
APPENDED = "appended"
OUTCOME_UNKNOWN = "outcome_unknown"
FAILED = "failed"
SKIPPED = "skipped"
@@ -108,10 +138,10 @@ class Campaign(Base, TimestampMixin):
__table_args__ = (UniqueConstraint("tenant_id", "external_id", name="uq_campaigns_tenant_external_id"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
owner_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
owner_group_id: Mapped[str | None] = mapped_column(ForeignKey("groups.id", ondelete="SET NULL"), nullable=True, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
owner_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
owner_group_id: Mapped[str | None] = mapped_column(ForeignKey("access_groups.id", ondelete="SET NULL"), nullable=True, index=True)
external_id: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
name: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str | None] = mapped_column(Text)
@@ -120,7 +150,6 @@ class Campaign(Base, TimestampMixin):
settings: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
mail_profile_policy: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
tenant: Mapped["Tenant"] = relationship("Tenant")
versions: Mapped[list[CampaignVersion]] = relationship(back_populates="campaign", cascade="all, delete-orphan")
jobs: Mapped[list[CampaignJob]] = relationship(back_populates="campaign", cascade="all, delete-orphan")
@@ -130,15 +159,291 @@ class CampaignShare(Base, TimestampMixin):
__table_args__ = (UniqueConstraint("campaign_id", "target_type", "target_id", name="uq_campaign_share_target"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
target_type: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
target_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
permission: Mapped[str] = mapped_column(String(20), default="read", nullable=False)
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
created_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
class CampaignCollaborationEntry(Base, TimestampMixin):
__tablename__ = "campaign_collaboration_entries"
__table_args__ = (
Index(
"ix_campaign_collaboration_entries_thread",
"tenant_id",
"campaign_id",
"created_at",
"id",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(
ForeignKey("campaigns.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
campaign_version_id: Mapped[str | None] = mapped_column(
ForeignKey("campaign_versions.id", ondelete="RESTRICT"),
nullable=True,
index=True,
)
reference_kind: Mapped[str | None] = mapped_column(String(40), nullable=True, index=True)
reference_id: Mapped[str | None] = mapped_column(String(500), nullable=True)
reference_label: Mapped[str | None] = mapped_column(String(255), nullable=True)
actor_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
actor_label_snapshot: Mapped[str] = mapped_column(String(255), nullable=False)
visibility: Mapped[str] = mapped_column(
String(30),
default="collaborators",
nullable=False,
index=True,
)
content: Mapped[str | None] = mapped_column(Text, nullable=True)
content_sha256: Mapped[str] = mapped_column(String(64), nullable=False)
mention_user_ids: Mapped[list[str]] = mapped_column(JSON, default=list, nullable=False)
withdrawn_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
withdrawn_by_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
)
redacted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
redacted_by_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
)
tombstone_reason: Mapped[str | None] = mapped_column(String(500), nullable=True)
class CampaignWorkAssignment(Base, TimestampMixin):
__tablename__ = "campaign_work_assignments"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"orchestration_idempotency_key",
name="uq_campaign_work_assignment_orchestration_key",
),
Index(
"ix_campaign_work_assignments_campaign_status",
"tenant_id",
"campaign_id",
"status",
"due_at",
"id",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(
ForeignKey("campaigns.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
campaign_version_id: Mapped[str | None] = mapped_column(
ForeignKey("campaign_versions.id", ondelete="RESTRICT"),
nullable=True,
index=True,
)
reference_kind: Mapped[str | None] = mapped_column(String(40), nullable=True, index=True)
reference_id: Mapped[str | None] = mapped_column(String(500), nullable=True)
reference_label: Mapped[str | None] = mapped_column(String(255), nullable=True)
purpose: Mapped[str] = mapped_column(String(500), nullable=False)
status: Mapped[str] = mapped_column(String(30), default="open", nullable=False, index=True)
due_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
assignee_type: Mapped[str] = mapped_column(String(40), nullable=False, index=True)
assignee_id: Mapped[str] = mapped_column(String(255), nullable=False, index=True)
assignee_label_snapshot: Mapped[str] = mapped_column(String(500), nullable=False)
assignee_current_label: Mapped[str | None] = mapped_column(String(500), nullable=True)
assignee_resolution_state: Mapped[str] = mapped_column(
String(30), default="resolved", nullable=False, index=True
)
resolution_provenance: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
resolution_checked_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
assigned_by_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True
)
assigned_by_label_snapshot: Mapped[str] = mapped_column(String(255), nullable=False)
completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
cancelled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
task_mirror_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
task_mirror_status: Mapped[str] = mapped_column(
String(30), default="not_configured", nullable=False
)
task_mirror_error: Mapped[str | None] = mapped_column(String(500), nullable=True)
task_mirrored_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
orchestration_idempotency_key: Mapped[str | None] = mapped_column(
String(255), nullable=True, index=True
)
orchestration_request_sha256: Mapped[str | None] = mapped_column(
String(64), nullable=True
)
orchestration_correlation_id: Mapped[str | None] = mapped_column(
String(128), nullable=True, index=True
)
workflow_instance_id: Mapped[str | None] = mapped_column(
String(36), nullable=True, index=True
)
workflow_step_id: Mapped[str | None] = mapped_column(
String(36), nullable=True, index=True
)
resource_revision: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
class CampaignWorkAssignmentEvent(Base, TimestampMixin):
__tablename__ = "campaign_work_assignment_events"
__table_args__ = (
Index(
"ix_campaign_work_assignment_events_history",
"tenant_id",
"assignment_id",
"created_at",
"id",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(
ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True
)
assignment_id: Mapped[str] = mapped_column(
ForeignKey("campaign_work_assignments.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
event_kind: Mapped[str] = mapped_column(String(40), nullable=False, index=True)
actor_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True
)
actor_label_snapshot: Mapped[str] = mapped_column(String(255), nullable=False)
status_snapshot: Mapped[str] = mapped_column(String(30), nullable=False)
assignee_type_snapshot: Mapped[str] = mapped_column(String(40), nullable=False)
assignee_id_snapshot: Mapped[str] = mapped_column(String(255), nullable=False)
assignee_label_snapshot: Mapped[str] = mapped_column(String(500), nullable=False)
resolution_state_snapshot: Mapped[str] = mapped_column(String(30), nullable=False)
details: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
class CampaignSchedule(Base, TimestampMixin):
__tablename__ = "campaign_schedules"
__table_args__ = (
Index("ix_campaign_schedules_due", "tenant_id", "active", "next_fire_at"),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(
ForeignKey("campaigns.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
source_version_id: Mapped[str] = mapped_column(
ForeignKey("campaign_versions.id", ondelete="RESTRICT"),
nullable=False,
index=True,
)
created_by_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
name: Mapped[str] = mapped_column(String(255), nullable=False)
delivery_mode: Mapped[str] = mapped_column(
String(20),
default="manual",
nullable=False,
index=True,
)
recurrence_kind: Mapped[str] = mapped_column(
String(20),
default="once",
nullable=False,
index=True,
)
interval_count: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
timezone: Mapped[str] = mapped_column(String(100), default="UTC", nullable=False)
starts_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
next_fire_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
ends_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
max_occurrences: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
occurrence_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False, index=True)
resource_revision: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
copy_options: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
source_snapshot: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
source_snapshot_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
approved_execution_snapshot_hash: Mapped[str | None] = mapped_column(
String(64), nullable=True, index=True
)
source_base_path: Mapped[str | None] = mapped_column(String(1000), nullable=True)
last_fired_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
last_campaign_id: Mapped[str | None] = mapped_column(
ForeignKey("campaigns.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
last_outcome: Mapped[str | None] = mapped_column(String(30), nullable=True)
last_recovery_state: Mapped[str | None] = mapped_column(
String(30), nullable=True
)
class CampaignScheduleOccurrence(Base, TimestampMixin):
__tablename__ = "campaign_schedule_occurrences"
__table_args__ = (
UniqueConstraint(
"schedule_id",
"scheduled_for",
name="uq_campaign_schedule_occurrence",
),
Index("ix_campaign_schedule_occurrences_schedule", "schedule_id", "scheduled_for"),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
schedule_id: Mapped[str] = mapped_column(
ForeignKey("campaign_schedules.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
scheduled_for: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
status: Mapped[str] = mapped_column(String(30), default="preparing", nullable=False, index=True)
idempotency_key: Mapped[str | None] = mapped_column(
String(200), nullable=True, index=True
)
generated_campaign_id: Mapped[str | None] = mapped_column(
ForeignKey("campaigns.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
generated_version_id: Mapped[str | None] = mapped_column(
ForeignKey("campaign_versions.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
error: Mapped[str | None] = mapped_column(Text, nullable=True)
delivery_command_ids: Mapped[list[str]] = mapped_column(
JSON, default=list, nullable=False
)
recovery_state: Mapped[str] = mapped_column(
String(30), default="none", nullable=False, index=True
)
evidence: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
last_checked_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
class RecipientImportMappingProfile(Base, TimestampMixin):
__tablename__ = "campaign_recipient_import_mapping_profiles"
__table_args__ = (
@@ -148,8 +453,8 @@ class RecipientImportMappingProfile(Base, TimestampMixin):
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
owner_user_id: Mapped[str] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
owner_user_id: Mapped[str] = mapped_column(ForeignKey("access_users.id", ondelete="CASCADE"), nullable=False, index=True)
name: Mapped[str] = mapped_column(String(255), nullable=False)
column_count: Mapped[int] = mapped_column(Integer, nullable=False)
headers: Mapped[list[str]] = mapped_column(JSON, default=list, nullable=False)
@@ -162,9 +467,6 @@ class RecipientImportMappingProfile(Base, TimestampMixin):
value_separators: Mapped[str] = mapped_column(String(50), default=",;|", nullable=False)
mappings: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list, nullable=False)
tenant: Mapped["Tenant"] = relationship("Tenant")
owner: Mapped["User"] = relationship("User")
class CampaignVersion(Base, TimestampMixin):
__tablename__ = "campaign_versions"
@@ -173,6 +475,11 @@ class CampaignVersion(Base, TimestampMixin):
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
version_number: Mapped[int] = mapped_column(Integer, nullable=False)
edit_revision: Mapped[int] = mapped_column(
Integer,
default=1,
nullable=False,
)
raw_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False)
schema_version: Mapped[str] = mapped_column(String(50), default="1.0", nullable=False)
source_filename: Mapped[str | None] = mapped_column(String(500))
@@ -199,7 +506,7 @@ class CampaignVersion(Base, TimestampMixin):
autosaved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
locked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
locked_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
locked_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
# Explicit user-requested lock. This is deliberately separate from
# locked_at, which represents the reversible validation lock used by the
@@ -207,23 +514,59 @@ class CampaignVersion(Base, TimestampMixin):
# RBAC permission for unlocking; permanent locks never unlock in place.
user_lock_state: Mapped[str | None] = mapped_column(String(20), nullable=True, index=True)
user_locked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
user_locked_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
user_locked_by_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
validation_summary: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
build_summary: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
execution_snapshot: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
execution_snapshot_hash: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
execution_snapshot_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
delivery_mode: Mapped[str | None] = mapped_column(String(30), nullable=True, index=True)
delivery_mode_selected_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
archived_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
index=True,
)
archived_by_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
campaign: Mapped[Campaign] = relationship(back_populates="versions")
__mapper_args__ = {
"version_id_col": edit_revision,
}
@property
def review_build_token(self) -> str | None:
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_review_reference
summary = self.build_summary if isinstance(self.build_summary, dict) else {}
return campaign_review_reference(self.id, summary.get("build_token") or summary.get("built_at"))
@property
def strong_etag(self) -> str:
return strong_resource_etag(
"campaign_version",
self.id,
self.edit_revision,
)
@property
def mail_profile_migration_required(self) -> bool:
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_boundary_violations
return bool(campaign_mail_profile_boundary_violations(self.raw_json))
class CampaignJob(Base, TimestampMixin):
__tablename__ = "campaign_jobs"
__table_args__ = (UniqueConstraint("campaign_version_id", "entry_index", name="uq_campaign_jobs_version_entry"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
campaign_version_id: Mapped[str] = mapped_column(ForeignKey("campaign_versions.id", ondelete="CASCADE"), nullable=False, index=True)
entry_index: Mapped[int] = mapped_column(Integer, nullable=False)
@@ -236,23 +579,68 @@ class CampaignJob(Base, TimestampMixin):
eml_local_path: Mapped[str | None] = mapped_column(String(1000))
eml_size_bytes: Mapped[int | None] = mapped_column(Integer)
eml_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
execution_input_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
build_status: Mapped[str] = mapped_column(String(50), default=JobBuildStatus.PENDING.value, nullable=False, index=True)
validation_status: Mapped[str] = mapped_column(String(50), default=JobValidationStatus.NEEDS_REVIEW.value, nullable=False, index=True)
queue_status: Mapped[str] = mapped_column(String(50), default=JobQueueStatus.DRAFT.value, nullable=False, index=True)
send_status: Mapped[str] = mapped_column(String(50), default=JobSendStatus.NOT_QUEUED.value, nullable=False, index=True)
delivery_channel_policy: Mapped[str] = mapped_column(
String(30),
default="mail",
nullable=False,
index=True,
)
postbox_status: Mapped[str] = mapped_column(
String(50),
default=JobPostboxStatus.NOT_REQUESTED.value,
nullable=False,
index=True,
)
print_status: Mapped[str] = mapped_column(
String(50),
default=JobPrintStatus.NOT_REQUESTED.value,
nullable=False,
index=True,
)
imap_status: Mapped[str] = mapped_column(String(50), default=JobImapStatus.NOT_REQUESTED.value, nullable=False, index=True)
attempt_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
postbox_attempt_count: Mapped[int] = mapped_column(
Integer,
default=0,
nullable=False,
)
print_attempt_count: Mapped[int] = mapped_column(
Integer,
default=0,
nullable=False,
)
last_error: Mapped[str | None] = mapped_column(Text)
queued_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
smtp_started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
outcome_unknown_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
imap_claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
imap_claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
sent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
resolved_recipients: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
delivery_provenance: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
resolved_postbox_targets: Mapped[list[dict[str, Any]]] = mapped_column(
JSON,
default=list,
nullable=False,
)
resolved_print_output: Mapped[dict[str, Any] | None] = mapped_column(
JSON,
nullable=True,
)
resolved_attachments: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
issues_snapshot: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list)
@@ -263,7 +651,7 @@ class CampaignIssue(Base, TimestampMixin):
__tablename__ = "campaign_issues"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=False, index=True)
campaign_version_id: Mapped[str | None] = mapped_column(ForeignKey("campaign_versions.id", ondelete="CASCADE"), nullable=True, index=True)
job_id: Mapped[str | None] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=True, index=True)
@@ -279,7 +667,7 @@ class AttachmentBlob(Base, TimestampMixin):
__table_args__ = (UniqueConstraint("tenant_id", "sha256", name="uq_attachment_blobs_tenant_sha256"),)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
sha256: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
size_bytes: Mapped[int] = mapped_column(Integer, nullable=False)
mime_type: Mapped[str | None] = mapped_column(String(255))
@@ -291,8 +679,8 @@ class AttachmentInstance(Base, TimestampMixin):
__tablename__ = "attachment_instances"
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False, index=True)
owner_user_id: Mapped[str | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL"), nullable=True, index=True)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
owner_user_id: Mapped[str | None] = mapped_column(ForeignKey("access_users.id", ondelete="SET NULL"), nullable=True, index=True)
campaign_id: Mapped[str | None] = mapped_column(ForeignKey("campaigns.id", ondelete="CASCADE"), nullable=True, index=True)
blob_id: Mapped[str] = mapped_column(ForeignKey("attachment_blobs.id", ondelete="CASCADE"), nullable=False, index=True)
logical_name: Mapped[str | None] = mapped_column(String(500))
@@ -317,17 +705,260 @@ class SendAttempt(Base, TimestampMixin):
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
class CampaignMessageAction(Base, TimestampMixin):
__tablename__ = "campaign_message_actions"
__table_args__ = (
UniqueConstraint(
"tenant_id",
"idempotency_key",
name="uq_campaign_message_actions_idempotency",
),
Index(
"ix_campaign_message_actions_job_created",
"job_id",
"created_at",
),
Index(
"ix_campaign_message_actions_campaign_kind",
"campaign_id",
"kind",
"status",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
campaign_id: Mapped[str] = mapped_column(
ForeignKey("campaigns.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
campaign_version_id: Mapped[str] = mapped_column(
ForeignKey("campaign_versions.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
job_id: Mapped[str] = mapped_column(
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
kind: Mapped[str] = mapped_column(String(30), nullable=False, index=True)
idempotency_key: Mapped[str] = mapped_column(String(200), nullable=False)
canonical_request_hash: Mapped[str] = mapped_column(String(64), nullable=False)
reason: Mapped[str | None] = mapped_column(Text, nullable=True)
context: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
actor_user_id: Mapped[str | None] = mapped_column(
ForeignKey("access_users.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
actor_api_key_id: Mapped[str | None] = mapped_column(String(36), nullable=True)
message_sha256: Mapped[str] = mapped_column(String(64), nullable=False)
message_size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
recipient_manifest_sha256: Mapped[str] = mapped_column(
String(64),
nullable=False,
)
recipient_count: Mapped[int] = mapped_column(Integer, nullable=False)
prior_send_status: Mapped[str] = mapped_column(String(50), nullable=False)
prior_attempt_count: Mapped[int] = mapped_column(Integer, nullable=False)
final_send_status: Mapped[str | None] = mapped_column(String(50), nullable=True)
status: Mapped[str] = mapped_column(
String(50),
default="initiated",
nullable=False,
index=True,
)
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
refusal_summary: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
error_type: Mapped[str | None] = mapped_column(String(120), nullable=True)
error_message: Mapped[str | None] = mapped_column(String(500), nullable=True)
linked_send_attempt_id: Mapped[str | None] = mapped_column(
ForeignKey("send_attempts.id", ondelete="SET NULL"),
nullable=True,
index=True,
)
effect_started_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
completed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
class CampaignMessageActionAttempt(Base, TimestampMixin):
__tablename__ = "campaign_message_action_attempts"
__table_args__ = (
UniqueConstraint(
"action_id",
"attempt_number",
name="uq_campaign_message_action_attempt_number",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
action_id: Mapped[str] = mapped_column(
ForeignKey("campaign_message_actions.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
status: Mapped[str] = mapped_column(
String(50),
default="initiated",
nullable=False,
index=True,
)
started_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
)
effect_started_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
completed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
accepted_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
refused_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
outcome_code: Mapped[str | None] = mapped_column(String(80), nullable=True)
diagnostic_summary: Mapped[str | None] = mapped_column(
String(500),
nullable=True,
)
class ImapAppendAttempt(Base, TimestampMixin):
__tablename__ = "imap_append_attempts"
__table_args__ = (
UniqueConstraint("job_id", "attempt_number", name="uq_imap_append_attempts_job_attempt"),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
job_id: Mapped[str] = mapped_column(ForeignKey("campaign_jobs.id", ondelete="CASCADE"), nullable=False, index=True)
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
claim_token: Mapped[str | None] = mapped_column(String(36), nullable=True)
folder: Mapped[str | None] = mapped_column(String(500))
status: Mapped[str] = mapped_column(String(50), nullable=False)
error_message: Mapped[str | None] = mapped_column(Text)
class PostboxDeliveryAttempt(Base, TimestampMixin):
__tablename__ = "campaign_postbox_delivery_attempts"
__table_args__ = (
UniqueConstraint(
"job_id",
"target_key",
"attempt_number",
name="uq_campaign_postbox_attempt_target_number",
),
Index(
"ix_campaign_postbox_attempt_idempotency",
"tenant_id",
"idempotency_key",
),
Index(
"ix_campaign_postbox_attempt_job_status",
"job_id",
"status",
),
)
id: Mapped[str] = mapped_column(
String(36),
primary_key=True,
default=new_uuid,
)
tenant_id: Mapped[str] = mapped_column(
String(36),
nullable=False,
index=True,
)
job_id: Mapped[str] = mapped_column(
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
target_key: Mapped[str] = mapped_column(String(64), nullable=False)
target_index: Mapped[int] = mapped_column(Integer, nullable=False)
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
status: Mapped[str] = mapped_column(
String(50),
nullable=False,
index=True,
)
target_snapshot: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
provider_delivery_id: Mapped[str | None] = mapped_column(String(36))
provider_message_id: Mapped[str | None] = mapped_column(String(36))
postbox_id: Mapped[str | None] = mapped_column(String(36), index=True)
address: Mapped[str | None] = mapped_column(String(500))
holder_count: Mapped[int | None] = mapped_column(Integer)
vacant: Mapped[bool | None] = mapped_column(Boolean)
duplicate: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
evidence: Mapped[dict[str, Any]] = mapped_column(
JSON,
default=dict,
nullable=False,
)
error_type: Mapped[str | None] = mapped_column(String(255))
error_code: Mapped[str | None] = mapped_column(String(100))
error_message: Mapped[str | None] = mapped_column(Text)
started_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
)
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
)
class PrintOutputAttempt(Base, TimestampMixin):
__tablename__ = "campaign_print_output_attempts"
__table_args__ = (
UniqueConstraint(
"job_id",
"attempt_number",
name="uq_campaign_print_attempt_job_number",
),
UniqueConstraint(
"tenant_id",
"idempotency_key",
name="uq_campaign_print_attempt_idempotency",
),
)
id: Mapped[str] = mapped_column(String(36), primary_key=True, default=new_uuid)
tenant_id: Mapped[str] = mapped_column(String(36), nullable=False, index=True)
job_id: Mapped[str] = mapped_column(
ForeignKey("campaign_jobs.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
attempt_number: Mapped[int] = mapped_column(Integer, nullable=False)
idempotency_key: Mapped[str] = mapped_column(String(255), nullable=False)
status: Mapped[str] = mapped_column(String(50), nullable=False, index=True)
render_id: Mapped[str | None] = mapped_column(String(36), index=True)
artifact_sha256: Mapped[str | None] = mapped_column(String(64), index=True)
evidence: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
error_message: Mapped[str | None] = mapped_column(Text)
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
__all__ = [
@@ -341,12 +972,18 @@ __all__ = [
"CampaignVersion",
"CampaignVersionFlow",
"CampaignVersionWorkflowState",
"CampaignWorkAssignment",
"CampaignWorkAssignmentEvent",
"ImapAppendAttempt",
"IssueSeverity",
"JobBuildStatus",
"JobImapStatus",
"JobPostboxStatus",
"JobPrintStatus",
"JobQueueStatus",
"JobSendStatus",
"JobValidationStatus",
"SendAttempt",
"PostboxDeliveryAttempt",
"PrintOutputAttempt",
]
@@ -0,0 +1,132 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Mapping
from sqlalchemy.orm import Session
from govoplan_core.admin.models import SystemSettings
from govoplan_core.admin.settings import SYSTEM_SETTINGS_ID
from govoplan_core.tenancy.scope import Tenant
DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 25
ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS = 500
SYNCHRONOUS_SEND_MAX_ENV = "GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS"
CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY = "campaign_delivery_policy"
SYNCHRONOUS_SEND_MAX_SETTINGS_KEY = "synchronous_send_max_recipients"
class CampaignDeliveryPolicyError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class SynchronousSendPolicy:
max_recipient_jobs: int
source: str
deployment_max_recipient_jobs: int
tenant_max_recipient_jobs: int | None = None
system_max_recipient_jobs: int | None = None
deployment_ceiling_explicit: bool = False
def as_dict(self) -> dict[str, Any]:
return {
"max_recipient_jobs": self.max_recipient_jobs,
"source": self.source,
"deployment_max_recipient_jobs": self.deployment_max_recipient_jobs,
"tenant_max_recipient_jobs": self.tenant_max_recipient_jobs,
"system_max_recipient_jobs": self.system_max_recipient_jobs,
"deployment_ceiling_explicit": self.deployment_ceiling_explicit,
"system_setting": f"system.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}.{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}",
"deployment_setting": SYNCHRONOUS_SEND_MAX_ENV,
"tenant_setting": (
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
),
}
def effective_synchronous_send_policy(
session: Session,
*,
tenant_id: str,
environ: Mapping[str, str] | None = None,
apply_tenant_override: bool = True,
) -> SynchronousSendPolicy:
env = os.environ if environ is None else environ
deployment_raw = env.get(SYNCHRONOUS_SEND_MAX_ENV)
deployment_explicit = deployment_raw is not None and (not isinstance(deployment_raw, str) or bool(deployment_raw.strip()))
deployment_value = _configured_limit(
env.get(SYNCHRONOUS_SEND_MAX_ENV),
source=SYNCHRONOUS_SEND_MAX_ENV,
default=ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS,
)
system = session.get(SystemSettings, SYSTEM_SETTINGS_ID)
system_raw = _tenant_limit_value(system.settings if system is not None else None)
system_value = _configured_limit(system_raw, source="system campaign delivery policy") if system_raw is not None else None
# Preserve explicit deployment configuration, but an implicit default is not
# an administrator ceiling. No override still retains the conservative 25.
inherited = min(deployment_value, system_value) if system_value is not None else (
deployment_value if deployment_explicit else DEFAULT_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS
)
inherited_source = ("system" if system_value <= deployment_value else "deployment_ceiling") if system_value is not None else (
"deployment" if deployment_explicit else "deployment_default"
)
tenant = session.get(Tenant, tenant_id) if apply_tenant_override else None
tenant_raw = _tenant_limit_value(tenant.settings if tenant is not None else None)
if tenant_raw is None:
return SynchronousSendPolicy(
max_recipient_jobs=inherited,
source=inherited_source,
deployment_max_recipient_jobs=deployment_value,
system_max_recipient_jobs=system_value,
deployment_ceiling_explicit=deployment_explicit,
)
tenant_value = _configured_limit(
tenant_raw,
source=(
f"tenant.settings.{CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY}."
f"{SYNCHRONOUS_SEND_MAX_SETTINGS_KEY}"
),
)
effective_value = min(inherited, tenant_value)
return SynchronousSendPolicy(
max_recipient_jobs=effective_value,
source="tenant" if tenant_value <= inherited else ("system_ceiling" if inherited_source == "system" else "deployment_ceiling"),
deployment_max_recipient_jobs=deployment_value,
tenant_max_recipient_jobs=tenant_value,
system_max_recipient_jobs=system_value,
deployment_ceiling_explicit=deployment_explicit,
)
def _tenant_limit_value(settings: Mapping[str, Any] | None) -> object | None:
if not isinstance(settings, Mapping):
return None
policy = settings.get(CAMPAIGN_DELIVERY_POLICY_SETTINGS_KEY)
if not isinstance(policy, Mapping):
return None
return policy.get(SYNCHRONOUS_SEND_MAX_SETTINGS_KEY)
def _configured_limit(value: object, *, source: str, default: int | None = None) -> int:
if value is None or (isinstance(value, str) and not value.strip()):
if default is not None:
return default
raise CampaignDeliveryPolicyError(f"{source} must be configured as an integer")
if isinstance(value, bool):
raise CampaignDeliveryPolicyError(f"{source} must be an integer, not a boolean")
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise CampaignDeliveryPolicyError(f"{source} must be an integer") from exc
if str(parsed) != str(value).strip() and not isinstance(value, int):
raise CampaignDeliveryPolicyError(f"{source} must be an integer")
if parsed < 0 or parsed > ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS:
raise CampaignDeliveryPolicyError(
f"{source} must be between 0 and {ABSOLUTE_SYNCHRONOUS_SEND_MAX_RECIPIENT_JOBS}"
)
return parsed
+559 -154
View File
@@ -1,24 +1,53 @@
from __future__ import annotations
from dataclasses import dataclass
from email import policy
from email.message import EmailMessage
from email.parser import BytesParser
from types import SimpleNamespace
from typing import Any
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import Campaign, CampaignVersion
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
from govoplan_campaign.backend.campaign.loader import load_campaign_json
from govoplan_campaign.backend.campaign.validation import validate_campaign_config
from govoplan_campaign.backend.campaign.models import DeliveryChannelPolicy
from govoplan_campaign.backend.campaign.validation import SemanticReport, validate_campaign_config
from govoplan_campaign.backend.persistence.campaigns import load_campaign_config_from_json
from govoplan_campaign.backend.messages.builder import build_campaign_messages
from govoplan_campaign.backend.messages.models import MessageAddress, MessageDraft, MessageValidationStatus
from govoplan_campaign.backend.messages.builder import BuiltMessage, CampaignBuildResult, build_campaign_messages
from govoplan_campaign.backend.messages.models import CampaignBuildReport, MessageAddress, MessageAttachmentSummary, MessageDraft, MessageValidationStatus
from govoplan_campaign.backend.integrations import files_integration, mail_integration
from govoplan_campaign.backend.path_security import assert_server_safe_campaign_paths
class MockCampaignSendError(RuntimeError):
pass
@dataclass(slots=True)
class _MockSendOutcome:
row: dict[str, Any]
sent_count: int = 0
failed_count: int = 0
skipped_count: int = 0
imap_appended_count: int = 0
imap_failed_count: int = 0
@dataclass(slots=True)
class _MockSendBatch:
results: list[dict[str, Any]]
sent_count: int = 0
failed_count: int = 0
skipped_count: int = 0
imap_appended_count: int = 0
imap_failed_count: int = 0
@property
def attempted_count(self) -> int:
return self.sent_count + self.failed_count
def _mock_mailbox() -> Any | None:
return mail_integration().mock_mailbox()
@@ -116,6 +145,485 @@ def _can_mock_send(
return False, f"Validation status is {message.validation_status.value}"
def _mock_send_batch(
*,
config: Any,
built_messages: list[Any],
mailbox: Any | None,
send: bool,
include_warnings: bool,
include_needs_review: bool,
append_sent: bool,
reviewed_keys: set[str] | None = None,
) -> _MockSendBatch:
batch = _MockSendBatch(results=[])
for built in built_messages:
outcome = _mock_send_message(
config=config,
built=built,
mailbox=mailbox,
send=send,
include_warnings=include_warnings,
include_needs_review=include_needs_review or str(built.draft.entry_id or built.draft.entry_index) in (reviewed_keys or set()),
append_sent=append_sent,
)
batch.results.append(outcome.row)
batch.sent_count += outcome.sent_count
batch.failed_count += outcome.failed_count
batch.skipped_count += outcome.skipped_count
batch.imap_appended_count += outcome.imap_appended_count
batch.imap_failed_count += outcome.imap_failed_count
return batch
def _mock_send_message(
*,
config: Any,
built: Any,
mailbox: Any | None,
send: bool,
include_warnings: bool,
include_needs_review: bool,
append_sent: bool,
) -> _MockSendOutcome:
draft = built.draft
row = _mock_send_row(draft)
can_send, skip_reason = _can_mock_send(
draft,
include_warnings=include_warnings,
include_needs_review=include_needs_review,
)
if not can_send or built.mime is None:
row.update({"status": "skipped", "message": skip_reason or "Message has no MIME output"})
return _MockSendOutcome(row=row, skipped_count=1)
recipients = _recipient_emails(draft)
envelope_from = _envelope_from(draft)
if not recipients:
row.update({"status": "skipped", "message": "No envelope recipients"})
return _MockSendOutcome(row=row, skipped_count=1)
if not send:
row.update({
"status": "ready",
"message": f"Would send to {len(recipients)} recipient(s)",
"envelope_from": envelope_from,
"envelope_recipients": recipients,
})
return _MockSendOutcome(row=row)
return _send_mock_smtp_message(
config=config,
built=built,
mailbox=mailbox,
row=row,
recipients=recipients,
envelope_from=envelope_from,
append_sent=append_sent,
)
def _mock_send_row(draft: MessageDraft) -> dict[str, Any]:
return {
"entry_index": draft.entry_index,
"entry_id": draft.entry_id,
"subject": draft.subject,
"validation_status": draft.validation_status.value,
"build_status": str(draft.build_status.value if hasattr(draft.build_status, "value") else draft.build_status),
"to": _message_addresses_payload(draft.to),
"attachments": _attachment_payloads(draft),
"issues": _issue_payloads(draft),
}
def _send_mock_smtp_message(
*,
config: Any,
built: Any,
mailbox: Any | None,
row: dict[str, Any],
recipients: list[str],
envelope_from: str,
append_sent: bool,
) -> _MockSendOutcome:
try:
if mailbox is not None and mailbox.consume_fail_next_smtp():
raise MockCampaignSendError("Configured mock failure: next SMTP delivery fails")
rejected = _smtp_rejection_matches(recipients)
if rejected and len(rejected) == len(recipients):
raise MockCampaignSendError(f"Configured mock failure: all recipients rejected ({', '.join(rejected)})")
accepted = [recipient for recipient in recipients if recipient not in rejected]
smtp_record = mailbox.record_smtp_delivery(
built.mime,
envelope_from=envelope_from,
envelope_recipients=accepted,
smtp_host="mock.smtp.local",
)
row.update({
"status": "sent",
"message": f"Mock SMTP captured as {smtp_record.id}",
"smtp_message_id": smtp_record.id,
"envelope_from": envelope_from,
"envelope_recipients": accepted,
"refused_recipients": rejected,
})
imap_appended, imap_failed = _append_mock_sent_message(config, built, mailbox, row, append_sent=append_sent)
return _MockSendOutcome(row=row, sent_count=1, imap_appended_count=imap_appended, imap_failed_count=imap_failed)
except Exception as exc:
row.update({"status": "failed", "message": str(exc), "envelope_from": envelope_from, "envelope_recipients": recipients})
return _MockSendOutcome(row=row, failed_count=1)
def _append_mock_sent_message(
config: Any,
built: Any,
mailbox: Any | None,
row: dict[str, Any],
*,
append_sent: bool,
) -> tuple[int, int]:
if not append_sent:
return 0, 0
try:
if mailbox is not None and mailbox.consume_fail_next_imap():
raise MockCampaignSendError("Configured mock failure: next IMAP append fails")
folder = _mock_sent_folder(config)
imap_record = mailbox.record_imap_append(_raw_message_bytes(built.mime), folder=folder, imap_host="mock.imap.local")
row.update({"imap_status": "appended", "imap_message_id": imap_record.id, "imap_folder": folder})
return 1, 0
except Exception as exc:
row.update({"imap_status": "failed", "imap_error": str(exc)})
return 0, 1
def _mock_sent_folder(config: Any) -> str:
if config.delivery.imap_append_sent.folder and config.delivery.imap_append_sent.folder != "auto":
return config.delivery.imap_append_sent.folder
if config.server.imap and config.server.imap.sent_folder and config.server.imap.sent_folder != "auto":
return config.server.imap.sent_folder
return "Sent"
def _mock_campaign_version(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version_id: str | None,
) -> tuple[Campaign, CampaignVersion]:
campaign = (
session.query(Campaign)
.filter(Campaign.id == campaign_id, Campaign.tenant_id == tenant_id)
.one_or_none()
)
if not campaign:
raise MockCampaignSendError("Campaign not found or not accessible")
wanted_version_id = version_id or campaign.current_version_id
if not wanted_version_id:
raise MockCampaignSendError("Campaign has no current version")
version = session.get(CampaignVersion, wanted_version_id)
if not version or version.campaign_id != campaign.id:
raise MockCampaignSendError(
"Campaign version not found or not part of campaign"
)
return campaign, version
def _mock_mailbox_for_run(*, send: bool, clear_mailbox: bool) -> Any | None:
mailbox = _require_mock_mailbox() if send or clear_mailbox else _mock_mailbox()
if clear_mailbox and mailbox is not None:
mailbox.clear_records()
return mailbox
def _build_mock_campaign_run(
session: Session,
*,
tenant_id: str,
campaign: Campaign,
version: CampaignVersion,
mailbox: Any | None,
send: bool,
include_warnings: bool,
include_needs_review: bool,
append_sent: bool,
check_files: bool,
) -> tuple[Any, Any, _MockSendBatch]:
files = files_integration()
raw_json = version.raw_json if isinstance(version.raw_json, dict) else {}
assert_server_safe_campaign_paths(
raw_json,
managed_files_available=files.available,
)
with files.prepared_campaign_snapshot(
session,
tenant_id=tenant_id,
campaign_id=campaign.id,
raw_json=raw_json,
include_bytes=True,
prefix="govoplan-mock-send-",
) as prepared:
prepared_raw = load_campaign_json(prepared.path)
config = load_campaign_config_from_json(
session,
tenant_id=tenant_id,
raw_json=prepared_raw,
campaign_id=campaign.id,
)
validation_report = validate_campaign_config(
config,
campaign_file=prepared.path,
check_files=check_files,
)
build_result = build_campaign_messages(
config,
campaign_file=prepared.path,
write_eml=False,
)
files.annotate_built_messages_with_managed_files(
build_result.built_messages,
prepared.managed_files_by_local_path,
)
send_batch = _mock_send_batch(
config=config,
built_messages=build_result.built_messages,
mailbox=mailbox,
send=send,
include_warnings=include_warnings,
include_needs_review=include_needs_review,
append_sent=append_sent,
)
return validation_report, build_result, send_batch
def _build_reviewed_mock_run(
session: Session, *, tenant_id: str, campaign: Campaign, version: CampaignVersion,
mailbox: Any | None, send: bool, include_warnings: bool, append_sent: bool,
clear_mailbox: bool = False,
) -> tuple[Any, Any, _MockSendBatch]:
"""Mock the sealed EML, never approve a new transient rendering by entry ID."""
from govoplan_campaign.backend.persistence.versions import _complete_campaign_review
from govoplan_campaign.backend.sending.execution import ensure_execution_snapshot, profile_delivery_summary
from govoplan_campaign.backend.sending.jobs import _load_eml_bytes_for_job
if not isinstance(version.execution_snapshot, dict) or not version.execution_snapshot_hash:
raise MockCampaignSendError("Build the campaign with frozen execution evidence before testing reviewed messages.")
# Do not invoke the legacy snapshot-creation fallback: this test must not
# alter Campaign state or create approval evidence as a side effect.
snapshot = ensure_execution_snapshot(session, version)
build = version.build_summary if isinstance(version.build_summary, dict) else {}
token = str(build.get("build_token") or build.get("built_at") or "")
if not token or token != str(snapshot.build_token or snapshot.built_at or ""):
raise MockCampaignSendError("The frozen execution no longer matches the current message build. Rebuild and review again.")
jobs = session.query(CampaignJob).filter(
CampaignJob.tenant_id == tenant_id, CampaignJob.campaign_version_id == version.id,
).order_by(CampaignJob.entry_index.asc()).all()
if not jobs:
raise MockCampaignSendError("The reviewed build contains no messages.")
state = (version.editor_state or {}).get("review_send", {})
complete = isinstance(state, dict) and state.get("inspection_complete") is True and state.get("build_token") == token
if any(job.validation_status in {"needs_review", "warning"} for job in jobs) and not complete:
raise MockCampaignSendError("Complete review for the exact current build before testing accepted exceptions.")
reviewed_keys = list(state.get("reviewed_message_keys", [])) if complete else []
decisions = list(state.get("issue_decisions", [])) if complete else []
_, normalized = _complete_campaign_review(session, version, reviewed_keys, decisions, user_id=None, build_token=token)
by_id = {item.get("job_id"): item for item in decisions}
for item in normalized:
prior = by_id.get(item["job_id"], {})
if any(prior.get(key) != item.get(key) for key in ("build_token", "message_sha256", "issue_fingerprint")):
raise MockCampaignSendError("Review evidence no longer matches the frozen message issues. Rebuild and review again.")
if snapshot.uses_mail:
current = profile_delivery_summary(session, version)
if current.get("smtp_transport_revision") != snapshot.smtp_transport_revision or (
append_sent and current.get("imap_transport_revision") != snapshot.imap_transport_revision
):
raise MockCampaignSendError("The selected Mail transport changed after build. Rebuild and review before testing these messages.")
built_messages = []
for job in jobs:
recipients = job.resolved_recipients or {}
attachments = [MessageAttachmentSummary.model_validate({
"status": "missing", "required": False, "allow_multiple": False, "zip_enabled": False,
"file_filter": "", "directory": "",
**{key: value for key, value in item.items() if key in MessageAttachmentSummary.model_fields},
}) for item in (job.resolved_attachments or []) if isinstance(item, dict)]
inactive = job.validation_status == "inactive"
excluded = job.validation_status == "excluded" or inactive
draft = MessageDraft(
entry_index=job.entry_index, entry_id=job.entry_id, active=not inactive,
build_status="built" if job.build_status == "built" else "build_failed",
validation_status=job.validation_status, send_status="skipped" if excluded else "draft",
imap_status="skipped" if excluded else "not_requested", subject=job.subject,
delivery_channel_policy=job.delivery_channel_policy,
**{key: recipients.get(key) for key in ("from",) if recipients.get(key)},
**{key: recipients.get(key) or [] for key in ("from_all", "to", "cc", "bcc", "reply_to", "bounce_to", "disposition_notification_to")},
issues=job.issues_snapshot or [], attachments=attachments,
attachment_count=sum(len(item.managed_matches or item.matches) for item in attachments),
eml_size_bytes=job.eml_size_bytes,
)
mime = None
if not excluded and DeliveryChannelPolicy(job.delivery_channel_policy).uses_mail:
if not job.eml_sha256:
raise MockCampaignSendError("Frozen EML checksum is missing; rebuild before testing reviewed messages.")
# The shared reader checks byte length, digest and Message-ID before
# any mock capture occurs. All messages preflight before the batch.
try:
mime = BytesParser(policy=policy.default).parsebytes(_load_eml_bytes_for_job(job))
except Exception as exc:
raise MockCampaignSendError("Frozen message bytes are unavailable or no longer match their integrity evidence. Rebuild and review before testing.") from exc
built_messages.append(BuiltMessage(draft=draft, mime=mime))
report = CampaignBuildReport(campaign_id=campaign.external_id, campaign_name=campaign.name,
campaign_file="", entries_count=len(jobs), messages=[item.draft for item in built_messages])
validation = SemanticReport(campaign_id=campaign.external_id, campaign_name=campaign.name,
entries_mode="frozen_build", entries_count=len(jobs), attachments_base_path="", rate_limit="frozen",
imap_append_enabled=snapshot.delivery.imap_append_sent.enabled)
config = SimpleNamespace(delivery=snapshot.delivery, server=SimpleNamespace(imap=None))
if clear_mailbox and mailbox is not None:
mailbox.clear_records()
batch = _mock_send_batch(config=config, built_messages=built_messages, mailbox=mailbox, send=send,
include_warnings=include_warnings, include_needs_review=False, append_sent=append_sent,
reviewed_keys=set(reviewed_keys))
return validation, CampaignBuildResult(report=report, built_messages=built_messages), batch
def _mock_validation_payload(validation_report: Any) -> dict[str, Any]:
payload = validation_report.model_dump(mode="json")
payload.update(
{
"ok": validation_report.ok,
"error_count": validation_report.error_count,
"warning_count": validation_report.warning_count,
}
)
return payload
def _mock_build_payload(build_result: Any) -> dict[str, Any]:
report = build_result.report
payload = report.model_dump(mode="json")
payload.update(
{
"built_count": report.built_count,
"queueable_count": report.queueable_count,
"needs_review_count": report.needs_review_count,
"blocked_count": report.blocked_count,
"warning_count": report.warning_count,
"ready_count": report.ready_count,
"messages": [_message_payload(message) for message in report.messages],
}
)
return payload
def _mock_send_steps(
*,
validation_report: Any,
validation_payload: dict[str, Any],
build_report: Any,
send_batch: _MockSendBatch,
send: bool,
append_sent: bool,
) -> list[dict[str, Any]]:
return [
{
"key": "validate",
"label": "Validate campaign JSON",
"status": "ok" if validation_report.ok else "needs_review",
"summary": validation_payload,
},
{
"key": "build",
"label": "Build messages",
"status": "ok" if build_report.queueable_count else "needs_review",
"summary": {
"built": build_report.built_count,
"queueable": build_report.queueable_count,
"needs_review": build_report.needs_review_count,
"blocked": build_report.blocked_count,
},
},
{
"key": "send",
"label": "Mock SMTP delivery",
"status": (
"skipped"
if not send
else "ok"
if send_batch.failed_count == 0
else "needs_review"
),
"summary": {
"attempted": send_batch.attempted_count,
"sent": send_batch.sent_count,
"failed": send_batch.failed_count,
"skipped": send_batch.skipped_count,
},
},
{
"key": "imap",
"label": "Mock IMAP Sent append",
"status": (
"skipped"
if not send or not append_sent
else "ok"
if send_batch.imap_failed_count == 0
else "needs_review"
),
"summary": {
"appended": send_batch.imap_appended_count,
"failed": send_batch.imap_failed_count,
},
},
]
def _mock_campaign_send_response(
*,
campaign: Campaign,
version: CampaignVersion,
mailbox: Any | None,
validation_report: Any,
validation_payload: dict[str, Any],
build_result: Any,
build_payload: dict[str, Any],
send_batch: _MockSendBatch,
send: bool,
include_warnings: bool,
include_needs_review: bool,
append_sent: bool,
) -> dict[str, Any]:
return {
"campaign_id": campaign.id,
"version_id": version.id,
"version_number": version.version_number,
"send_requested": send,
"include_warnings": include_warnings,
"include_needs_review": include_needs_review,
"append_sent": append_sent,
"steps": _mock_send_steps(
validation_report=validation_report,
validation_payload=validation_payload,
build_report=build_result.report,
send_batch=send_batch,
send=send,
append_sent=append_sent,
),
"validation": validation_payload,
"build": build_payload,
"send": {
"attempted_count": send_batch.attempted_count,
"sent_count": send_batch.sent_count,
"failed_count": send_batch.failed_count,
"skipped_count": send_batch.skipped_count,
"imap_appended_count": send_batch.imap_appended_count,
"imap_failed_count": send_batch.imap_failed_count,
"results": send_batch.results,
},
"mailbox": {
"messages": mailbox.list_records(limit=200) if mailbox is not None else []
},
}
def run_mock_campaign_send(
session: Session,
*,
@@ -128,6 +636,7 @@ def run_mock_campaign_send(
append_sent: bool = True,
clear_mailbox: bool = False,
check_files: bool = False,
use_reviewed_build: bool = False,
) -> dict[str, Any]:
"""Validate, build and optionally mock-send a version without mutating it.
@@ -137,155 +646,51 @@ def run_mock_campaign_send(
mailbox only when send=True.
"""
campaign = session.query(Campaign).filter(Campaign.id == campaign_id, Campaign.tenant_id == tenant_id).one_or_none()
if not campaign:
raise MockCampaignSendError("Campaign not found or not accessible")
wanted_version_id = version_id or campaign.current_version_id
if not wanted_version_id:
raise MockCampaignSendError("Campaign has no current version")
version = session.get(CampaignVersion, wanted_version_id)
if not version or version.campaign_id != campaign.id:
raise MockCampaignSendError("Campaign version not found or not part of campaign")
mailbox = _require_mock_mailbox() if send or clear_mailbox else _mock_mailbox()
if clear_mailbox and mailbox is not None:
mailbox.clear_records()
files = files_integration()
with files.prepared_campaign_snapshot(
campaign, version = _mock_campaign_version(
session,
tenant_id=tenant_id,
campaign_id=campaign.id,
raw_json=version.raw_json if isinstance(version.raw_json, dict) else {},
include_bytes=True,
prefix="multimailer-mock-send-",
) as prepared:
prepared_raw = load_campaign_json(prepared.path)
config = load_campaign_config_from_json(session, tenant_id=tenant_id, raw_json=prepared_raw, campaign_id=campaign.id)
validation_report = validate_campaign_config(config, campaign_file=prepared.path, check_files=check_files)
build_result = build_campaign_messages(config, campaign_file=prepared.path, write_eml=False)
files.annotate_built_messages_with_managed_files(build_result.built_messages, prepared.managed_files_by_local_path)
send_results: list[dict[str, Any]] = []
sent_count = 0
failed_count = 0
skipped_count = 0
imap_appended_count = 0
imap_failed_count = 0
for built in build_result.built_messages:
draft = built.draft
can_send, skip_reason = _can_mock_send(draft, include_warnings=include_warnings, include_needs_review=include_needs_review)
row: dict[str, Any] = {
"entry_index": draft.entry_index,
"entry_id": draft.entry_id,
"subject": draft.subject,
"validation_status": draft.validation_status.value,
"build_status": str(draft.build_status.value if hasattr(draft.build_status, "value") else draft.build_status),
"to": _message_addresses_payload(draft.to),
"attachments": _attachment_payloads(draft),
"issues": _issue_payloads(draft),
}
if not can_send or built.mime is None:
skipped_count += 1
row.update({"status": "skipped", "message": skip_reason or "Message has no MIME output"})
send_results.append(row)
continue
recipients = _recipient_emails(draft)
envelope_from = _envelope_from(draft)
if not recipients:
skipped_count += 1
row.update({"status": "skipped", "message": "No envelope recipients"})
send_results.append(row)
continue
if not send:
row.update({"status": "ready", "message": f"Would send to {len(recipients)} recipient(s)", "envelope_from": envelope_from, "envelope_recipients": recipients})
send_results.append(row)
continue
try:
if mailbox is not None and mailbox.consume_fail_next_smtp():
raise MockCampaignSendError("Configured mock failure: next SMTP delivery fails")
rejected = _smtp_rejection_matches(recipients)
if rejected and len(rejected) == len(recipients):
raise MockCampaignSendError(f"Configured mock failure: all recipients rejected ({', '.join(rejected)})")
accepted = [recipient for recipient in recipients if recipient not in rejected]
smtp_record = mailbox.record_smtp_delivery(built.mime, envelope_from=envelope_from, envelope_recipients=accepted, smtp_host="mock.smtp.local")
sent_count += 1
row.update({
"status": "sent",
"message": f"Mock SMTP captured as {smtp_record.id}",
"smtp_message_id": smtp_record.id,
"envelope_from": envelope_from,
"envelope_recipients": accepted,
"refused_recipients": rejected,
})
if append_sent:
try:
if mailbox is not None and mailbox.consume_fail_next_imap():
raise MockCampaignSendError("Configured mock failure: next IMAP append fails")
folder = "Sent"
if config.delivery.imap_append_sent.folder and config.delivery.imap_append_sent.folder != "auto":
folder = config.delivery.imap_append_sent.folder
elif config.server.imap and config.server.imap.sent_folder and config.server.imap.sent_folder != "auto":
folder = config.server.imap.sent_folder
imap_record = mailbox.record_imap_append(_raw_message_bytes(built.mime), folder=folder, imap_host="mock.imap.local")
imap_appended_count += 1
row.update({"imap_status": "appended", "imap_message_id": imap_record.id, "imap_folder": folder})
except Exception as exc:
imap_failed_count += 1
row.update({"imap_status": "failed", "imap_error": str(exc)})
except Exception as exc:
failed_count += 1
row.update({"status": "failed", "message": str(exc), "envelope_from": envelope_from, "envelope_recipients": recipients})
send_results.append(row)
validation_json = validation_report.model_dump(mode="json")
validation_json.update({"ok": validation_report.ok, "error_count": validation_report.error_count, "warning_count": validation_report.warning_count})
build_report = build_result.report
build_json = build_report.model_dump(mode="json")
build_json.update({
"built_count": build_report.built_count,
"queueable_count": build_report.queueable_count,
"needs_review_count": build_report.needs_review_count,
"blocked_count": build_report.blocked_count,
"warning_count": build_report.warning_count,
"ready_count": build_report.ready_count,
"messages": [_message_payload(message) for message in build_report.messages],
})
attempted_count = sum(1 for row in send_results if row.get("status") in {"sent", "failed"})
return {
"campaign_id": campaign.id,
"version_id": version.id,
"version_number": version.version_number,
"send_requested": send,
"include_warnings": include_warnings,
"include_needs_review": include_needs_review,
"append_sent": append_sent,
"steps": [
{"key": "validate", "label": "Validate campaign JSON", "status": "ok" if validation_report.ok else "needs_review", "summary": validation_json},
{"key": "build", "label": "Build messages", "status": "ok" if build_report.queueable_count else "needs_review", "summary": {"built": build_report.built_count, "queueable": build_report.queueable_count, "needs_review": build_report.needs_review_count, "blocked": build_report.blocked_count}},
{"key": "send", "label": "Mock SMTP delivery", "status": "skipped" if not send else ("ok" if failed_count == 0 else "needs_review"), "summary": {"attempted": attempted_count, "sent": sent_count, "failed": failed_count, "skipped": skipped_count}},
{"key": "imap", "label": "Mock IMAP Sent append", "status": "skipped" if not send or not append_sent else ("ok" if imap_failed_count == 0 else "needs_review"), "summary": {"appended": imap_appended_count, "failed": imap_failed_count}},
],
"validation": validation_json,
"build": build_json,
"send": {
"attempted_count": attempted_count,
"sent_count": sent_count,
"failed_count": failed_count,
"skipped_count": skipped_count,
"imap_appended_count": imap_appended_count,
"imap_failed_count": imap_failed_count,
"results": send_results,
},
"mailbox": {"messages": mailbox.list_records(limit=200) if mailbox is not None else []},
}
campaign_id=campaign_id,
version_id=version_id,
)
mailbox = _mock_mailbox_for_run(send=send, clear_mailbox=clear_mailbox and not use_reviewed_build)
if use_reviewed_build:
validation_report, build_result, send_batch = _build_reviewed_mock_run(
session, tenant_id=tenant_id, campaign=campaign, version=version,
mailbox=mailbox, send=send, include_warnings=include_warnings, append_sent=append_sent,
clear_mailbox=clear_mailbox,
)
else:
validation_report, build_result, send_batch = _build_mock_campaign_run(
session,
tenant_id=tenant_id,
campaign=campaign,
version=version,
mailbox=mailbox,
send=send,
include_warnings=include_warnings,
include_needs_review=include_needs_review,
append_sent=append_sent,
check_files=check_files,
)
validation_payload = _mock_validation_payload(validation_report)
build_payload = _mock_build_payload(build_result)
result = _mock_campaign_send_response(
campaign=campaign,
version=version,
mailbox=mailbox,
validation_report=validation_report,
validation_payload=validation_payload,
build_result=build_result,
build_payload=build_payload,
send_batch=send_batch,
send=send,
include_warnings=include_warnings,
include_needs_review=include_needs_review,
append_sent=append_sent,
)
result["use_reviewed_build"] = use_reviewed_build
if use_reviewed_build:
result["steps"][0].update(label="Verify frozen execution inputs", status="ok")
result["steps"][1].update(label="Use reviewed frozen messages", status="ok")
result["build"]["review_satisfied"] = True
return result
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,272 @@
from __future__ import annotations
from dataclasses import replace
from typing import Iterable
from govoplan_core.core.modules import DocumentationTopic
_TRANSLATIONS = {
"campaigns.admin.delivery-policy": {
"title": "Die Grenze für interaktiven Campaign-Versand konfigurieren",
"summary": "Eine auditierte Systemgrenze für „Jetzt senden“ und engere Mandantengrenzen festlegen, ohne Kampagnen zu ändern oder Nachrichten zu senden.",
"body": "Administration → SYSTEM → Campaign-Versand erlaubt mit system:settings:read das Lesen und mit system:settings:write das Speichern der Empfängerauftragsgrenze. Der unveränderte Standard bleibt 25; Systemadministrierende dürfen ausdrücklich 0500 wählen, etwa 200 für einen Lauf mit 183 Aufträgen. Administration → TENANT → Campaign-Versand benötigt admin:policies:read/write und darf die geerbte Systemgrenze nur einschränken. Das Entfernen einer Überschreibung stellt Vererbung wieder her. Eine ausdrücklich gesetzte GOVOPLAN_CAMPAIGN_SYNCHRONOUS_SEND_MAX_RECIPIENTS bleibt einschließlich null eine verbindliche Bereitstellungsgrenze. Ohne diesen Wert verhindert der implizite Standard keine autorisierte Systemüberschreibung. Größere interaktive Läufe dauern länger und können Proxy- oder Anfragezeitlimits erreichen; für große Kampagnen bleiben Hintergrund-Worker mit CELERY_ENABLED und funktionierender Redis-/Celery-Infrastruktur die bevorzugte getrennte Betriebsart. Diese Einstellung begrenzt genau einen gespeicherten geeigneten Lauf für „Jetzt senden“, nicht die Kampagnengröße oder Worker-Verteilung. Speichern ändert ausschließlich die gewählte Einstellung mit Revisionskonfliktschutz, Vorher-/Nachher-Konfigurationshistorie und Audit. Es versendet nichts, ändert keine gespeicherten Prüfungen und umgeht weder Mail-, Prüf-, Freigabe- noch Ausführungsintegritätsprüfungen. Bei Fehlern bleibt der Entwurf erhalten. Neuladen verwendet den zentralen Schutz ungespeicherter Änderungen und liest die frische gespeicherte Richtlinie.",
},
"campaigns.workflow.create-campaign": {
"title": "Eine Kampagne anlegen",
"summary": "Eine gesteuerte Kampagne als bearbeitbaren Entwurf beginnen und Zweck sowie Eigentum vor Zustelldaten festlegen.",
"body": (
"Eine neue Kampagne beginnt mit einer bearbeitbaren Arbeitsversion. Die Aktionsleiste zeigt gespeichert, ungespeichert oder speichernd; Verwerfen steht direkt vor Speichern, und beim Verlassen eines geänderten Entwurfs greift der zentrale Speichern-oder-Verwerfen-Schutz. Destruktive Lebenszyklusaktionen sind von gewöhnlichen Aktionen getrennt. Das Anlegen gewährt keinen Zugriff auf Mail-Profile, verwaltete Dateien, Adressquellen oder Zustellaktionen; diese bleiben eigenständig autorisiert. Bestätigtes Speichern und anschließendes Neuladen sind getrennte Ergebnisse. Ein fehlgeschlagener Schreibvorgang oder abgebrochener Konflikt erhält den Entwurf; wiederholtes Speichern teilt sich einen laufenden Vorgang. Eingaben während des Speicherns bleiben als neuere ungespeicherte Arbeit erhalten. Fehlgeschlagenes oder durch neuere Änderungen überholtes Neuladen beim Verwerfen erhält den Entwurf. Unveränderte ZIP-Konfiguration sperrt nicht das Speichern unabhängiger Anlagenquellen oder Regeln. Hinzufügen-Aktionen bleiben kompakt. Eine vorübergehend fehlende Dateiauswahl wird erklärt und verwandelt verwaltete Pfade nicht stillschweigend in Texteingaben; fehlende Regelquellen müssen neu gewählt werden. Eingabe oder Leertaste öffnet die Auswahl am fokussierten Pfadfeld."
),
},
"campaigns.workflow.create-editable-successor": {
"title": "Eine bearbeitbare Nachfolgeversion anlegen",
"summary": "Nach einer dauerhaften oder zustellungsbedingten Sperre weiterarbeiten, ohne die bewahrte Version umzuschreiben.",
"body": (
"„Bearbeitbare Kopie anlegen“ erzeugt die nächste Arbeitsversion der Kampagne. Validierungssperren und vorübergehende Benutzersperren werden dagegen an der bestehenden Version aufgehoben und dürfen keine parallelen Entwürfe erzeugen."
),
},
"campaigns.workflow.import-address-source": {
"title": "Eine Adressquelle importieren",
"summary": "Ein erlaubtes wiederverwendbares Adressbuch oder eine Liste als nachvollziehbaren versionierten Snapshot in die Kampagne kopieren.",
"body": (
"Campaign folgt der Adressquelle nicht live. Es speichert die ausgewählte Quellrevision und warnt bei einer neueren Revision. Eine erneute Übernahme ist deshalb immer eine ausdrückliche Aktion der verfassenden Person."
),
},
"campaigns.workflow.import-distribution-list": {
"title": "Eine Verteilerliste übernehmen",
"summary": "Eine wiederverwendbare Zielgruppe auflösen, Kanal- und Policy-Entscheidungen prüfen und einen unveränderlichen Snapshot in die aktuelle Version kopieren.",
"body": (
"Eine Verteilerliste bleibt in ihrem verantwortlichen Modul live und versioniert. Campaign friert genau eine Auflösung ein; spätere Listen- oder Provideränderungen erzeugen nur eine Driftwarnung und schreiben gespeicherte Empfänger niemals um."
),
},
"campaigns.workflow.import-recipients": {
"title": "Empfänger importieren",
"summary": "Text-, CSV- oder Tabellendaten mit Quellprovenienz in geprüfte kampagnenlokale Empfängerzeilen überführen.",
"body": (
"Der Import kopiert gültige Zeilen in die bearbeitbare Kampagnenversion. Ungültige Zeilen bleiben in der Vorschau sichtbar, statt still zu verschwinden. Spätere Änderungen der Quelldatei ändern die gespeicherte Kampagne nicht automatisch."
),
},
"campaigns.workflow.prepare-printable-delivery": {
"title": "Eine druckbare Zustellung vorbereiten",
"summary": "Eine veröffentlichte Ausgabevorlage wählen, ein deterministisches Artefakt bauen und Route sowie Hash-Nachweis vor Post- oder Hauspostzustellung prüfen.",
"body": (
"Druckbare Zustellung ist optional und anbieterneutral. Campaign friert die Routenentscheidungen je Empfänger ein, während Templates Kompatibilität und Rendering verantwortet; Files kann das erzeugte Artefakt verwalten. Eine geordnete Ausweichroute wird nur nach bestätigter Ablehnung vor Annahme verwendet, niemals nach einer angenommenen oder im Ergebnis unbekannten digitalen Wirkung."
),
},
"campaigns.workflow.use-managed-attachments": {
"title": "Verwaltete Dateien als Kampagnenanhänge verwenden",
"summary": "Gesteuerte Dateiversionen wählen, Regelzuordnungen prüfen und exakt verwendete Dateien im Build-Nachweis bewahren.",
"body": (
"Verwaltete Anhänge bleiben Eigentum von Files. Campaign speichert gesteuerte Referenzen und eingefrorene Build-Nachweise; es übernimmt keine Files-Administrationsbefugnis und akzeptiert keine beliebigen Serverpfade."
" Verknüpfen Sie benötigte Dateien vor dem Sperren. Prüfen und Senden erklärt die Reihenfolge und warum eine gesperrte Version keine Dateiverknüpfungen mehr ändern darf. Während die Anhangsvorschau lädt, ist Sperren nicht verfügbar. Vor der Sperraktion werden Treffer frisch geprüft; neue unverbundene Treffer benötigen die ausdrückliche Bestätigung Verknüpfen und sperren. Bei fehlgeschlagener Prüfung bleibt die Version ungesperrt. Verknüpfen Sie fehlende Dateien in einer bearbeitbaren Version und validieren, bauen und prüfen Sie erneut."
),
},
"campaigns.workflow.queue-delivery": {
"title": "Eine Zustellung einreihen",
"summary": "Einen exakt geprüften Build in die Worker-Warteschlange stellen und Empfängerzustände sowie Wiederholungsschutz bewahren.",
"body": (
"Das Einreihen ist eine kontrollierte Zustandsänderung, kein Zustellnachweis. Gewöhnliche Stapel sollen Hintergrund-Worker verwenden. Angenommene und im Ergebnis unbekannte Wirkungen bleiben vor blinder Wiederholung geschützt."
),
},
"campaigns.workflow.send-calendar-invitations": {
"title": "Personalisierte Kalendereinladungen senden",
"summary": "Je Empfänger eine iCalendar-Anfrage einfrieren, über Mail zustellen und aktuelle Antworten aus Calendar prüfen.",
"body": (
"Campaign verantwortet Empfängerauflösung, exakte Einladungsanfrage, Zustellnachweis und Bericht. Calendar verantwortet gespiegeltes VEVENT und Antwortstatus. Der Spiegel entsteht erst, nachdem ein Kanal die Nachricht angenommen hat; ein Calendar-Fehler schreibt angenommenen Mail-Nachweis nie um. Mail kann METHOD:REPLY-Teile aus einer konfigurierten IMAP-Quelle für Zustellstatus weiterreichen."
),
},
"campaigns.workflow.send-small-controlled-run": {
"title": "Einen kleinen kontrollierten Lauf sofort senden",
"summary": "Geeignete Aufträge nur nach bewusster Bestätigung synchron ausführen, dass die geprüfte Kampagne klein genug ist.",
"body": (
"„Jetzt senden“ ist durch die wirksame maximale Anzahl von Empfängeraufträgen aus Bereitstellung, System und Mandant geschützt, nicht durch eine Grenze der Kampagnengröße. Standard ist 25. Administration → SYSTEM → Campaign-Versand erlaubt 0500 innerhalb einer ausdrücklichen Bereitstellungsgrenze; TENANT darf die geerbte Grenze nur einschränken. Der Server zählt den exakt gespeicherten geeigneten Build, lehnt einen zu großen oder leeren Lauf vor SMTP ab und prüft jede Nachricht sowie die Mail-Profilrevision vor der ersten Providerwirkung. Worker-Versand bleibt unabhängig und benötigt funktionierende Hintergrundinfrastruktur."
" Ein erfolgreicher Mail-Servertest belegt nur diese Verbindung, nicht die Ressourcenauswahl oder Absender-/Empfängerberechtigung der Kampagne. Die Vorprüfung unterscheidet Mail-Profil-/Zugangsdatenrichtlinien, SMTP-Konfiguration, Authentifizierung und Verbindung, ohne Geheimnisse oder rohe Providerdetails anzuzeigen. SMTP prüft seine eigene Zugangsdatenwahl; IMAP prüft seine Auswahl getrennt beim Ablegen in Gesendet. Die vollständige Kampagnenvalidierung prüft weiterhin alle erforderlichen Auswahlen."
),
},
"campaigns.workflow.view-aggregate-delivery-report": {
"title": "Aggregierte Kampagnenergebnisse prüfen",
"summary": "Datenschutzgeschützte Summen ohne Empfängerzeilen, Nachrichteninhalte, Zustelldiagnosen oder Exportbefugnis einsehen.",
"body": (
"Die aggregierte Berichtssicht zeigt nur freigegebene fachliche Kampagnenergebnisse. Positive Zellen unterhalb des konfigurierten Schwellwerts werden zusammen mit einem ergänzenden Wert oder erforderlichenfalls dem Nenner unterdrückt, damit kleine Gruppen nicht durch Subtraktion rekonstruiert werden können."
),
},
"campaigns.workflow.view-delivery-report": {
"title": "Detaillierte Zustellergebnisse prüfen",
"summary": "Zustellsummen und empfängerbezogene Auftragsnachweise in der aktuellen Campaign-Berichtsoberfläche einsehen.",
"body": (
"Der empfängerbezogene Bericht erfordert Lesezugriff auf Kampagne, Bericht und Empfänger. Infrastrukturdiagnosen bleiben getrennt autorisiert; der Server prüft jede direkte Detailroute unabhängig von der Oberfläche."
" Jeder Auftrag zeigt alle eingefrorenen An-, Cc- und Bcc-Adressen in gespeicherter Reihenfolge; ältere Aufträge ohne Empfängersnapshot verwenden die primäre Adresse. Diese identifiziert die Zeile und beweist nicht, dass nur eine Adresse angeschrieben wurde. Die kompakte Liste erfordert weiterhin Empfänger-Leseberechtigung; der getrennte aggregierte Bericht bleibt ohne Adressen. SMTP- und IMAP-Zustände verwenden Auswahllisten mit gemeinsamen Bezeichnungen."
" Jetzt senden, synchrone Wiederholung/Fortsetzung und Kopieren nach Gesendet verwenden denselben sperrenden Fortschrittsdialog. Er liest ausschließlich kleine versionsbezogene Zähler mit Kampagnen-Lese- und Objektberechtigung, nicht Arbeitsbereich, Empfänger, Anhänge oder vollständige Zusammenfassung. Verarbeitet umfasst angenommene/kopierte, fehlgeschlagene, ungewisse und abgebrochene Ergebnisse. Laufende Aufträge werden getrennt von ausstehenden gezählt; ausgeschlossene oder nicht angeforderte Kanäle gehören nicht zum Nenner. Bei Lesefehlern bleiben die letzten Zähler erhalten; dies bedeutet keinen fehlgeschlagenen Versand. Nach einer getrennten Anfrage kann die Verarbeitung weiterlaufen: Wiederholen Sie sie nicht blind. Bestätigte Ergebnisse bleiben bei späteren Aktualisierungsfehlern erhalten."
" Kopieren nach Gesendet betrifft nur die ausgewählte Version, nicht stillschweigend historische Versionen. Mail verwendet eine begrenzte authentifizierte Verbindung und Ordnerauflösung für nacheinander ausgeführte APPEND-Befehle; Richtlinie, Referenzen, Zugangsdaten und Wiederherstellungsschutz werden je Nachricht neu geprüft. Standardgrenzen sind 100 APPENDs oder 300 Sekunden pro Verbindung. Dies ist Verbindungswiederverwendung, keine atomare Postfachtransaktion. Ungewisse APPEND-Ergebnisse benötigen nachweisgestützten Abgleich und werden niemals automatisch wiederholt."
),
},
"campaigns.workflow.export-delivery-report": {
"title": "Zustellergebnisse exportieren",
"summary": "Einen autorisierten CSV-Snapshot empfängerbezogener Zustellergebnisse für kontrollierte Weiterverwendung herunterladen.",
"body": (
"Ein Berichtsexport enthält personenbezogene Daten und Zustellnachweise. Er ist entsprechend dem Kampagnenzweck sowie den geltenden Export- und Aufbewahrungsrichtlinien zu speichern, zu übertragen, aufzubewahren und zu löschen."
),
},
"campaigns.workflow.share-campaign": {
"title": "Eine Kampagne freigeben",
"summary": "Einer Person oder Gruppe ausdrücklichen Lese- oder Schreibzugriff auf eine Kampagne geben, ohne Plattformberechtigungen auszuweiten.",
"body": (
"Eine Freigabe kann den Zugriff nur innerhalb der bestehenden Rolle auf die ausgewählte Kampagne eingrenzen. Sie gewährt niemals Mail-Profilnutzung, Files-Befugnisse, mandantenweiten Empfängerzugriff oder eine fehlende Campaign-Aktion."
),
},
"campaigns.workflow.archive-campaign": {
"title": "Eine Kampagne archivieren",
"summary": "Eine abgeschlossene Kampagne aus der aktiven Arbeit entfernen und Versionen, Ergebnisse sowie Audit-Nachweise bewahren.",
"body": (
"Eine Kampagne darf erst archiviert werden, nachdem eingereihte, sendende und im Ergebnis unbekannte Arbeiten geklärt sind. Archivierung bewahrt Nachweise und ist für jede Kampagne mit Build-, Sperr- oder Zustellhistorie die richtige Lebenszyklusaktion."
),
},
"campaigns.admin.collaboration-governance": {
"title": "Campaign-Zusammenarbeit und Aufbewahrung steuern",
"summary": "Diskussionszugriff getrennt von Kampagnenbearbeitung konfigurieren und auditierbare Moderations-Tombstones bewahren.",
"body": (
"Campaign-Zusammenarbeit verwendet neben dem Lesezugriff auf die Kampagne getrennte Berechtigungen zum Lesen, Schreiben und Moderieren. Die integrierte Managerrolle darf moderieren; Prüf- und Senderollen dürfen lesen und schreiben, ohne Bearbeitungsrechte zu erhalten. Eine Lesefreigabe genügt als übergeordnete Ressourcengewährung; Kommentare werten sie nicht auf. Nur für Moderationen sichtbare Inhalte werden serverseitig gefiltert. Beiträge besitzen keine Bearbeitungs-API. Rückzug und Schwärzung entfernen die Anzeige, erhalten jedoch stabilen Eintrag, SHA-256-Nachweis, Akteursnapshot, Zeitpunkt, typisierte Referenz, Tombstone und begrenztes Audit-Ereignis. Erwähnt werden dürfen nur aktive Personen mit Eigentums- oder Freigabezugriff. Optionale Notifications erhalten inhaltsfreie Hinweise; Providerfehler macht Notifications nicht zur Pflichtabhängigkeit. Institutionelle Aufbewahrungs- und Datenschutzrichtlinien müssen Kollaborationszeilen und Audit-Nachweise gemeinsam behandeln. Kommentare sind weder Freigaben noch Workflow-Übergänge oder Systemereignisse."
),
},
"campaigns.workflow.delete-untouched-draft": {
"title": "Einen unberührten Kampagnenentwurf löschen",
"summary": "Einen Entwurf ohne geschützte Build-, Sperr-, Veröffentlichungs-, Snapshot- oder Zustellnachweise sofort entfernen.",
"body": (
"Löschen ist bewusst enger als Archivieren. Es markiert einen geeigneten Entwurf als gelöscht und erzeugt einen Audit-Eintrag. Eine Kampagne mit bereits aufbewahrungspflichtigen Nachweisen kann dadurch nicht entfernt werden."
),
},
"campaigns.privacy.data-subject-requests": {
"title": "Campaign-Daten in einer Datenschutzanfrage prüfen",
"summary": "Empfänger-, Kollaborations-, Versions-, Zustell-, Berichts- und Artefaktmetadaten ermitteln, ohne unveränderliche Nachweise umzuschreiben.",
"body": (
"Der Campaign-DSAR-Anbieter sucht im wirksamen Mandanten nach normalisierter Empfänger-E-Mail, direkten Mitgliedschaftsreferenzen und namensraumbezogenen Campaign-Kennungen. Er isoliert passende Inline-Empfängerfelder und Auftragsmetadaten und meldet gebaute Versionen, Zustellversuche, Postbox- und Druckergebnisse, Korrekturen, empfängerbezogene Berichte, Nachrichtendigests, Anhangsmetadaten und betroffene Kollaboration. Eigener Beitragstext wird ausgegeben; fremder Text nicht allein wegen einer Erwähnung. EML-Bytes, Objekt- oder lokale Pfade, Providerziele, Worker-Claims, Idempotenzdaten, Geheimnisse, Zugangsdaten und fremde Empfängeradressen bleiben ausgeschlossen. Gebaute, gesperrte, veröffentlichte, abgeschlossene, zugestellte, korrigierte, zurückgezogene oder geschwärzte Datensätze bleiben begründet erhalten. Tombstones, Hashwerte und Audit-Nachweise sind unveränderlich. Entwurfsempfänger und benutzereigene Anhänge benötigen koordinierte manuelle Prüfung. Der Provider kann ein persönliches Import-Mappingprofil idempotent löschen und eine aktive Freigabe für die betroffene Person widerrufen; zugestellte Nachweise und erzeugte Artefakte werden nie direkt gelöscht."
),
},
"campaigns.workflow.archive-historical-version": {
"title": "Eine historische Kampagnenversion archivieren",
"summary": "Eine nicht aktuelle Version aus der Standardhistorie ausblenden, ohne aufbewahrte Nachweise zu ändern oder zu löschen.",
"body": (
"Die Archivierung einer historischen Version betrifft nur ihre Darstellung. Ursprünglicher Workflow-Zustand, Konfiguration, Berichte, Zustellergebnisse und Audit-Nachweise bleiben für autorisierte Personen lesbar und werden bei eingeblendeten archivierten Versionen mitgeführt."
),
},
"campaigns.search.campaigns": {
"title": "Autorisierte Kampagnen durchsuchen",
"summary": "Kampagnenidentität und Lebenszyklusmetadaten für die berechtigungsbewusste Plattformsuche bereitstellen.",
"body": (
"Wenn Search installiert ist, trägt Campaign aktuelle Namen, externe Kennungen, Beschreibungen und Lebenszykluszustände bei. Vor einem Ergebnis werden Mandant, Eigentum, Gruppeneigentum, ausdrückliche Freigaben, Widerruf, Löschung und Campaign-Leseberechtigung erneut geprüft. Bestätigte Kampagnen- und Freigabeänderungen aktualisieren den abgeleiteten Index über den dauerhaften Plattform-Ereignispfad; ein Neuaufbau verändert keine Campaign-Nachweise."
),
},
"campaigns.postbox-delivery": {
"title": "Campaign-Nachrichten an Postboxen zustellen",
"summary": "Je Empfängerzeile eine oder mehrere exakte oder organisationsabgeleitete Postboxen allein oder neben Mail adressieren.",
"body": (
"Konfiguriert werden kampagnenweite Ziele und optionale Ergänzungen oder Ersetzungen je Zeile. Abgeleitete Ziele lösen eine veröffentlichte Postbox-Vorlage mit Organisationseinheit, Funktion und optionalen Kontextwerten auf; Werte dürfen aus Campaign-Feldern stammen. Ziele werden beim Build eingefroren. Ein Ausweichen zum zweiten Kanal erfolgt nur nach bestätigter Ablehnung vor Annahme; angenommene oder im Ergebnis unbekannte Wirkungen lösen kein Fallback aus."
),
},
"campaigns.mail-profile-user-journey": {
"title": "Ein Mail-Profil für die Kampagnenzustellung wählen",
"summary": "Campaign referenziert ein autorisiertes Mail-Profil und speichert niemals SMTP-/IMAP-Einstellungen oder Zugangsdaten.",
"body": (
"Mail-Einstellungen → Wiederverwendbares Mail-Profil → SMTP-Zugangsdaten (bei Nutzung auch IMAP-Zugangsdaten) speichert eine ausdrückliche Zugangsdatenkennung. Eine leere Auswahl bedeutet Vererbung nur soweit die Mail-Richtlinie dies erlaubt; ein Profilstandard ist keine gespeicherte Kampagnenauswahl. Fehlende oder inaktive gespeicherte Profile, Server und Zugangsdaten bleiben sichtbar nicht verfügbar und werden nicht stillschweigend ersetzt. "
"In den Mail-Einstellungen der Kampagne wird ein verfügbares Profil ausgewählt, über Mail getestet und gespeichert. Bei gemeldeten kampagnenlokalen Alt-Transportdaten wählen Sie nach der autorisierten Profilauswahl „Ausgewähltes Mail-Profil migrieren“. Diese ausdrückliche Aktion funktioniert auch bei unveränderter Profilauswahl und unberührtem Entwurf. Gesperrte historische Nachweise bleiben erhalten; verwenden Sie zuvor die angebotene Entsperrung oder bearbeitbare Nachfolgeversion. „Prüfen und Senden“ zeigt den Migrationsblocker mit Rückweg zu den Mail-Einstellungen, statt wiederholt eine ungültige Anhangsvorschau anzufordern. Die Profilauswahl lädt nur für die Kampagne autorisierte Profile; ein Fehler beim getrennten administrativen Richtlinienkatalog leert sie nicht. Validierung und Zustellung prüfen die Profilberechtigung erneut. Migration speichert Konfiguration, versendet aber keine E-Mail. Validieren, bauen und prüfen Sie die resultierende Version vor der Zustellung erneut."
" Mail-Migration und ZIP-Richtlinienkorrekturen lassen sich in beliebiger Reihenfolge speichern. Unveränderte ZIP-Einstellungen blockieren die Migration nicht und erhalten keinen neuen Zustimmungsnachweis. Eine Archiv- oder Inhaltskorrektur mit unveränderten Mail-Referenzen erhält den alten Transport serverseitig und zeigt weiterhin den Migrationshinweis; es erfolgt keine stillschweigende Migration. Ein bestätigtes Speichern und das anschließende Neuladen des Arbeitsbereichs sind getrennte Ergebnisse: Bei fehlgeschlagenem Neuladen bleiben die letzten nutzbaren Daten derselben Kampagne und Version sichtbar, ergänzt um den Fehler. Wiederholen Sie Neuladen; veraltete Antworten einer anderen Kampagne, Version, Identität oder früheren Aktualisierung dürfen den aktuellen Arbeitsbereich nicht ersetzen."
),
},
"campaigns.mail-profile-governance": {
"title": "Campaign-zu-Mail-Profilreferenzen steuern",
"summary": "Mail besitzt Transportdefinitionen und verschlüsselte Zugangsdaten; Campaign nur die Profilreferenz und Zustellnachweise.",
"body": (
"Kampagnenverfassende erhalten mail:profile:use; verfügbare Profile werden über Mail-Policy begrenzt und es wird ausdrücklich festgelegt, ob Profil-Zugangsdaten geerbt werden dürfen oder eine Kampagne Mail-eigene Zugangsdaten auswählen muss. Die Mail-Richtlinienseite zeigt die SMTP-/IMAP-Zugangsdatenvererbung mit lokalen, geerbten und wirksamen Werten sowie übergeordneten Sperren. Inline-Transportfelder werden abgelehnt und niemals samt Zugangsdaten an den Browser zurückgegeben. Altbestände bleiben unverändert, bis eine ausdrückliche auditierte Profilmigration eine bearbeitbare Version erzeugt oder aktualisiert. Dafür genügt auch das schon referenzierte Profil, wenn „Ausgewähltes Mail-Profil migrieren“ verwendet wird. Mail-Einstellungen laden nur die für diese Kampagne nutzbaren Profile; der administrative Richtlinienkatalog wird getrennt auf der Mail-Richtlinienseite angefordert und Fehler bleiben dort sichtbar. Diese Trennung verleiht keine Profiladministration und umgeht weder Eigentümer-, Mandanten- noch Mail-Autorisierung. Migration versendet keine E-Mail und stellt keinen alten Ausführungssnapshot wieder her."
" Unabhängige Entwurfskorrekturen dürfen das exakt gespeicherte alte Serverobjekt nur bei unveränderten öffentlichen Mail-Referenzen erhalten; Inline-Transport darf nicht mitgesendet werden. Dabei wird Inhalt gespeichert, kein Profil ausgewählt oder genutzt, auch nach Entzug seiner Berechtigung. Ausdrückliche Migration benötigt weiterhin mail:profile:use und aktuelle Mail-Policy. Der Versions-Auditnachweis unterscheidet legacy_mail_settings_preserved und legacy_mail_settings_migrated. Unveränderte ZIP-Konfiguration wird nicht erneut bestätigt; geänderte ZIP-Einstellungen unterliegen allen Richtlinienprüfungen. Erfolgreiche Korrekturen entwerten bisherige Build- und Ausführungsnachweise und lockern weder Validierung noch Prüfung oder Versand."
" Bereits migrierte Entwürfe folgen derselben Regel für unveränderte Referenzen: Eine spätere Pflicht zur ausdrücklichen SMTP-/IMAP-Zugangsdatenwahl verhindert keine unabhängige Archiv- oder Inhaltskorrektur. Jede geänderte Mail-Ressourcenauswahl benötigt weiterhin Mail-Berechtigung und aktuelle Richtlinie; verbindliche Validierung und Zustellung prüfen stets die vollständige Auswahl erneut."
),
},
"campaigns.mail-profile-operations": {
"title": "Profilbasierte Kampagnenzustellung betreiben",
"summary": "Worker autorisieren und lösen Mail-Profile bei Ausführung neu auf; Campaign bewahrt nur undurchsichtige Mail-Revisionen und Ergebnisse.",
"body": (
"SMTP- und IMAP-Laufzeitaktionen prüfen die Zugangsdatenpflicht getrennt je Protokoll; ein SMTP-Aufruf benötigt keine IMAP-Parameter und umgekehrt. Vollständige Kampagnenvalidierung und Build-Zusammenfassung prüfen weiterhin beide erforderlichen Auswahlen. Die Vorprüfung unterscheidet Mail-Profil-/Zugangsdatenrichtlinienfehler von SMTP-Konfigurations-, Authentifizierungs- und Verbindungsfehlern. Ein erfolgreicher Servertest ersetzt keine kampagnenspezifische Autorisierung. "
"Ein Altsnapshot, unautorisiertes oder inaktives Profil, Referenzkonflikt oder eine geänderte SMTP-/IMAP-Revision stoppt die Zustellung. Synchrone Stapel prüfen DNS, Verbindung, TLS und Authentifizierung vor der ersten Wirkung, verwenden eine begrenzte gesunde SMTP-Verbindung wieder und verbinden bei Alterung neu. Oberfläche und Bericht zeigen Stapel-, Verbindungs-, Wiederverbindungs-, Fehler- und Pausenzahlen. Systemische Authentifizierungs-, Absender- oder Verbindungsfehler pausieren übrige Aufträge mit stabilem Grund; das Profil ist zu korrigieren und zu testen, bevor ausdrücklich fortgesetzt wird. Verbindungsverlust nach Beginn von DATA bleibt ergebnisoffen und wird nicht automatisch wiederholt. Der Datensatz wird bewahrt, Profilwahl korrigiert, erneut validiert und gebaut und erst dann neu eingereiht. Reine Passwortrotation kopiert keine Geheimnisse nach Campaign. Unsichere SMTP-/IMAP-Wirkungen bleiben bis zum evidenzbasierten Betriebsabgleich blockiert. Wird Campaign nach Annahme eines Auftrags für den Mandanten unzugänglich, bleibt er unangetastet und wird als Betriebsaktion gemeldet."
),
},
"campaigns.workflow.prepare-validate-and-build": {
"title": "Eine Kampagne vorbereiten, validieren und bauen",
"summary": "Gesteuerte Empfänger-, Vorlagen-, Anhangs- und Mail-Profil-Eingaben in exakte Nachrichten zur Prüfung überführen.",
"body": (
"Jede Eingabe wird in ihrer verantwortlichen Oberfläche vorbereitet, alle blockierenden Validierungsprobleme werden gelöst und exakte Empfängernachrichten vor der Prüfung gebaut. Empfängerzeilen können als eine ausdrücklich bestätigte Entwurfsänderung gesammelt aktiviert oder deaktiviert werden; Speichern erzeugt normale Versionsnachweise und verwirft veraltete Validierungs-, Build- und Prüfzustände. Passwortfelder verwenden den zentralen sicheren Generator, dessen Vorschlag erst nach „Passwort verwenden“ übernommen wird. Campaign friert Empfänger- und Anhangsnachweise für die ausgewählte Version ein; spätere Quelländerungen ändern den Build nicht. Kennzahlen bieten nur dann einen benannten Drill-down, wenn eine autorisierte Quellsammlung, gefilterte Prüftabelle, Anhangsvorschau oder ein Bericht eine Handlung ermöglicht. Datenschutzunterdrückte Aggregate bleiben nicht interaktiv. Ist Templates installiert, besitzt dessen einziger Navigationseintrag die wiederverwendbare Bibliothek; kampagnenspezifische Komposition bleibt im Arbeitsbereich."
" In individuellen und globalen Adressdialogen bestimmen die Auf-/Ab-Aktionen die gespeicherte Adressreihenfolge. Speichern im Dialog übernimmt diese Reihenfolge ohne alphabetische Neusortierung in den Kampagnenentwurf; doppelte E-Mail-Adressen behalten ihre erste Position. Eingefügte Adressen werden in Eingabereihenfolge angehängt, ohne vorhandene Adressen umzuordnen. Die erste individuelle An-Adresse bleibt der primäre Name und die E-Mail-Adresse der Empfängerzeile. Speichern Sie anschließend die Kampagnenseite, um den Entwurf dauerhaft zu übernehmen; bei einem Fehler bleibt die Reihenfolge für einen ausdrücklichen neuen Speicherversuch erhalten. Abbrechen verwirft gezielt nur die noch unbestätigten Dialogänderungen."
),
},
"campaigns.workflow.complete-review": {
"title": "Die Kampagnenprüfung abschließen",
"summary": "Kritische Blocker lösen, einzelne Nachrichten entscheiden und unkritische Punkte für genau einen Build bestätigen.",
"body": (
"Das Öffnen der Vorlage ohne Bearbeitung, Änderungen des Schreibschutzes und der Wechsel zwischen visueller Ansicht und Quelltext erhalten das gespeicherte HTML unverändert und erfordern beim Verlassen kein Speichern. Der Prüfabschluss bleibt an aktuellen Build-Token, geprüfte Nachrichtenschlüssel, dokumentierte Problementscheidungen und Nachrichtennachweise gebunden. Normales Speichern sendet nur die clientverantworteten Metadaten created_from, field_overrides und opt_ins; review_send und approval_gate sind lesbare Servernachweise, aber keine schreibbaren Editorfelder. Werden sie bei einem Metadaten-Speichern ausgelassen, bleiben sie serverseitig erhalten. Die vorgesehenen Regeln für Entsperrung, Nachfolgeversionen und Build-Invalidierung entfernen veraltete Nachweise weiterhin. Bei notwendiger Mail-Altdatenmigration bleibt „Prüfen und Senden“ schreibgeschützt, unterdrückt inkompatible Anhangsvorschau-Anfragen und bietet „Mail-Einstellungen öffnen“ für genau die ausgewählte Version. Migrieren, validieren, bauen und prüfen Sie vor dem Senden erneut. Ausdrückliche Aktionen auf handlungsfähigen Empfänger-, Anhangs-, Validierungs- und Prüfkennzahlen öffnen Quellseite, Nachweisvorschau oder gefilterte Nachrichtentabelle. Reine Information und datenschutzunterdrückte Werte werden nicht zu versteckten Klickzielen. Änderungen an Empfängern, Inhalt, Anhängen, Eigentümerkontext oder nicht geheimer Transportidentität erfordern erneut Validierung, Build und Prüfung."
" Gleichartige Prüfbedingungen bestätigen gruppiert ausschließlich vom Server zugelassene, ungeprüfte Nachrichten aus der aktuell geladenen passenden Auswahl. Wählen Sie eine verständlich benannte Kategorie, prüfen Sie die gezählte Empfängerauswahl und geben Sie bei Anhangsausnahmen eine gemeinsame Begründung an. Jede Anfrage benennt höchstens 200 konkrete Nachrichten und prüft aktuellen Build und Kategorie; wiederholen Sie dies für verbleibende Nachrichten, statt andere Kategorien oder nicht geladene Nachrichten als mitbestätigt anzusehen. Jede ausgewählte Nachricht erhält einen eigenen eingefrorenen, zuordenbaren Entscheidungsnachweis. Bei fehlgeschlagenem Speichern bleiben Begründung und Auswahl für einen ausdrücklichen Wiederholungsversuch erhalten; ein geänderter Build verhindert veraltete Bestätigungen. Die Gruppenbestätigung sendet keine Nachrichten und schließt die abschließende Prüfung nicht ab. Harte Blocker können nicht übergangen werden. Beabsichtigte richtlinienbedingte Ausschlüsse und ausdrücklich erlaubte Anhangsregeln ohne Treffer bleiben informativ und benötigen keine Prüfentscheidung."
" Eine einzelne Annahme speichert Begründung und Prüfstatus sofort, schon vor dem vollständigen Prüfabschluss; Neuladen setzt den bestätigten Fortschritt desselben Builds fort. Bei fehlgeschlagenem Speichern oder einem Konflikt bleibt die Begründung für einen ausdrücklichen neuen Versuch erhalten; die Nachricht gilt noch nicht als geprüft. Jeder Speichervorgang ergänzt nur ausgewählte Nachrichten und erhält fremde Prüfnachweise, ohne Nachrichten neu zu bauen, Anhangsdateien zu prüfen oder den gesamten Arbeitsbereich neu zu laden. Teilfortschritt erlaubt keinen Versand; der abschließende Prüfabschluss kontrolliert weiterhin alle erforderlichen Entscheidungen und harten Blocker. Pflichtanhänge und harte Sperrrichtlinien bleiben gegenüber optional erlaubten leeren Treffern vorrangig; auch die getrennte Kampagnenrichtlinie für vollständig anhangslose Nachrichten gilt weiterhin."
" Speichern benötigt die Campaign-Prüfberechtigung, die aktuelle Versionsrevision und den sicheren operativen Bezug review_build_token; Diagnoseberechtigung ist nicht erforderlich. Veraltete Builds oder gleichzeitige Änderungen führen zu einem Konflikt ohne Überschreiben gespeicherten Fortschritts."
" Bestätigte oder erwartete Anhangsbedingungen bleiben für denselben Build auch bei „Bestätigen und senden“ erfüllt; fehlende oder mehrdeutige Quelltreffer bleiben als Kontext sichtbar, erzeugen aber keine zweite Bestätigungspflicht. Der Mock-Test nach der Prüfung verwendet verifizierte eingefrorene Nachrichten und abgeschlossene Entscheidungen statt einer Neuerstellung. Geänderte Eingaben, Problemnachweise, Nachrichtenbytes oder Mail-Transport stoppen den Test vor der Mock-Aufzeichnung; Entwurfsvorschauen behalten ihren getrennten vorläufigen Build."
" Validierungsdetails und Listen mehrfach verwendeter Dateien zeigen alle Einträge über die zentrale DataGrid-Seitensteuerung. Zusammengehörige fehlende Regeltreffer und die Richtlinienfolge einer anhangslosen Nachricht werden gemeinsam erklärt; aufklappbare technische Nachweise bleiben erhalten. Nachrichtentabelle und Filter verwenden vier operative Zustände: Bereit, Prüfung erforderlich, Blockiert und Ausgeschlossen. Angenommene ausdrückliche Entscheidungen werden Bereit; noch unbestätigte Warnungen bleiben Prüfung erforderlich. Eine zweite Spalte erklärt den Zustand. Diese Darstellung löscht oder verändert keine eingefrorenen Probleme oder Auditnachweise."
),
},
"campaigns.workflow.retry-and-reconcile": {
"title": "Fehler wiederholen und unsichere Wirkungen abgleichen",
"summary": "Sicher wiederholbare Fehler von Mail-, Postbox- oder IMAP-Wirkungen mit unbekanntem Ergebnis trennen.",
"body": (
"Eine Wiederholung erzeugt neuen Versuchsnachweis und ist nur für ausdrücklich geeignete Zustände zulässig. Unbekannte Mail-, Postbox- oder IMAP-Wirkungen dürfen nie blind wiederholt werden. Externe Nachweise sind zu prüfen und der betroffene Kanal vor dem Fortsetzen abzugleichen. Angenommene Mail-Versuche und Postbox-Ziele bleiben bei Teilwiederholungen unveränderlich; die Reparatur von „Gesendet“ versendet angenommene Mail nicht erneut."
" Ohne Worker bietet der Bericht ausdrücklich bestätigte, begrenzte Wiederholung und Fortsetzung über dieselben unveränderlichen Aufträge, Ausführungsprüfungen, Prüfnachweise, Freigaben, Mail-Berechtigungen, Ratenbegrenzungen und Wiederherstellungsnachweise wie Jetzt senden. Jede Anfrage bleibt innerhalb der wirksamen synchronen Grenze und meldet verbleibende Arbeit; bereits angenommene, ausgeschlossene, aktive und ungewisse Aufträge werden nicht erneut gesendet. Wiederholung benötigt campaigns:campaign:retry und synchron zusätzlich campaigns:campaign:send; Fortsetzen benötigt campaigns:campaign:queue und campaigns:campaign:send. Abgleich benötigt campaigns:campaign:reconcile und eine sachliche Nachweisnotiz; er sendet nichts."
" Ein festhängender übernommener, sendender oder kopierender Auftrag ist nicht allein durch Zeitablauf sicher. Der Bericht bietet die Wiederherstellung eines unterbrochenen Auftrags nur bei abgelaufener dauerhafter Sperre und nachweislich gestoppter oder ersetzter ursprünglicher Laufzeit. Die mitgesendete sichere Revision muss noch passen, und ursprüngliche Wiederherstellungsnachweise müssen gültig sein. Die Aktion setzt das Ergebnis ausschließlich auf ungewiss. Prüfen Sie Provider- beziehungsweise Postfachnachweise und gleichen Sie angenommen/nicht gesendet oder kopiert/nicht kopiert getrennt ab, bevor Sie ausdrücklich wiederholen. Fehlende Sperr- oder Versuchsnachweise und unbestätigte Besitzer bleiben zur betrieblichen Untersuchung gesperrt. Ein doppelter Worker-Aufruf verändert aktive Zustände nicht."
),
},
"campaigns.reference.composition-assurance": {
"title": "Die Campaign-Referenzkomposition absichern",
"summary": "Campaign nur mit abgestimmten Verträgen, rollensicheren Oberflächen, dauerhaften Wirkungsnachweisen, optionaler Modultrennung und wiederherstellbaren Daten freigeben.",
"body": (
"Freigabeprüfungen müssen Campaign-Validierung und Anhangsauflösung unabhängig in frischen Prozessen initialisieren, ohne einen früheren Seiten- oder Testimport vorauszusetzen. Diese lokalen Einstiegspunkte bleiben ohne installiertes Mail oder Files nutzbar; ihr Import startet keinen Versand und lockert keine Pfadberechtigungen für verwaltete Dateien. "
"Campaign ist nur dann Referenzkomposition, wenn Core, Mail, Files, Addresses, Worker, Speicher, Policies und Dokumentation in genau der installierten Kombination geprüft sind. Gewöhnliche Lesende sehen Fachzustand statt Pfaden, Speicherschlüsseln, Worker-Claims oder rohen Providerdiagnosen; Diagnose- und Exportbefugnis bleiben getrennt."
" Prüfen Sie, dass einzelne Begründungen und Prüfzustände schon vor dem vollständigen Abschluss Neuladen überstehen. Teilfortschritt benötigt campaigns:campaign:review und Schreibzugriff, ergänzt genau ausgewählte Nachrichten des aktuellen Builds mit Revisionsprüfung und protokolliert Annahmen ohne fremde Prüfnachweise zu ersetzen. Der operative review_build_token legt keine rohen Diagnosetoken offen. Teilfortschritt erlaubt keinen Versand; harte Blocker sind nicht bestätigbar. Richtlinienbedingte Ausschlüsse und ausdrücklich erlaubte leere optionale Anhangsregeln erzeugen keine neue Prüfpflicht; Pflichtanhänge und globale harte Sperren bleiben wirksam. Diese Auflösungsänderungen gelten nur für neue Builds: Eine bewusste Neuerstellung klassifiziert vorhandene Nachrichten neu und entwertet frühere Prüf- und Freigabenachweise. Eingefrorene historische Auftragsprobleme dürfen nicht aus veränderlichen Richtlinien umgeschrieben werden."
" Der ausdrückliche Mock-Modus use_reviewed_build benötigt einen vorhandenen versiegelten Ausführungsnachweis und bei prüfpflichtigen Nachrichten den Abschluss desselben Builds. Vor jeder Mock-Aufzeichnung oder angeforderten Leerung des Mock-Postfachs prüft er Auftrags- und Problemnachweise, EML-Länge, Digest, Message-ID und aktuellen Mail-Transport. Er sendet kein SMTP, verändert keinen Campaign-Zustellstatus und erzeugt keinen fehlenden Altdaten-Snapshot. include_needs_review ist in diesem Modus keine pauschale Umgehung."
),
},
"campaigns.reference.shared-build-artifacts": {
"title": "Gemeinsam genutzte Campaign-Build-Artefakte betreiben",
"summary": "Erzeugte Nachrichten in gemeinsamem Objektspeicher ablegen und vor der Zustellung verifizieren.",
"body": (
"Campaign speichert erzeugte EML unter undurchsichtigen gemeinsamen Objektschlüsseln und protokolliert erwartete Größe, SHA-256-Digest und Message-ID je Auftrag. Worker auf anderen Knoten prüfen diesen Nachweis vor Zustellung. Vor Objekt- oder Files-Ausgaben zeichnet eine lease-gebundene Core-Recovery-Operation Quelle, validierte Version und reserviertes Präfix auf, prüft das Objekt und erneuert die Sperre vor dem Fach-Commit. Eine getrennte auftragsgebundene Operation erfasst vor realer Mail-, Postbox- oder Druckwirkung unveränderliche Nachrichten- und Empfängerdigests und verifiziert später den autoritativen Kanalversuch. Ablehnung, Annahme, unbekanntes Ergebnis und Recovery-Bedarf bleiben unterscheidbar. Objektfehler weisen Kompensation nach; Files-Ausgaben und unsichere Bereinigung bleiben Vorwärts-Recovery. Aufbewahrung ändert Locator kontrolliert und prüft Abwesenheit unabhängig. Ein reiner Betriebsabgleich inventarisiert begrenzte Mandantenpräfixe, schützt aktive Builds und mindestens 24 Stunden Karenz und löscht nur weiterhin unreferenzierte Objekte. Laufzeitobjektschlüssel sind keine Fachdaten."
),
},
"campaigns.archive-encryption-governance": {
"title": "Passwortgeschützte ZIP-Anhänge gesteuert verwenden",
"summary": "Standardmäßig AES einsetzen und schwaches Windows-kompatibles ZipCrypto nur mit Policy, Berechtigung, Bestätigung und Nachweis wählen.",
"body": (
"Campaign löst Archivverschlüsselung über Policy auf System-, Mandanten-, Eigentümer- und Kampagnenebene auf. Passwortgeschützte Archive verwenden AES, außer die vollständig vererbte Richtlinie erlaubt Legacy ZipCrypto ausdrücklich und die handelnde Person besitzt campaigns:archive:use_legacy_zipcrypto. Die Legacy-Auswahl benötigt eine begründete Bestätigung. Passwörter erscheinen weder im Campaign-Nachweis noch in der Nachricht und müssen über den getrennt ausgewählten, per Policy erlaubten Kanal übermittelt werden. Jeder Build friert Archiv- und Mitglied-Hashes, Implementierungsversion, Policy-Hash und -Quellpfad, bestätigende Person, Begründung, Zeitpunkt und Build-Identität ein. Eine später strengere Policy blockiert Einreihen und Senden bis zum Neubau; nach Fehlern wird nie von AES auf ZipCrypto zurückgefallen. Temporärer Klartext und Archive bleiben im begrenzten Build-Verzeichnis und werden nach Erfolg oder Fehler entfernt."
" Kampagneneinstellungen, Richtlinien und Anhänge zeigen die wirksame Richtlinie und führen berechtigte Administrierende direkt zu Administration → SYSTEM → Campaign archive encryption. Aktivieren Sie dort Legacy ZipCrypto und speichern Sie. Frische Systemstandardwerte sind bearbeitbar, ohne dass das bloße Öffnen bereits eine Ausnahme erzeugt. Mandanten- und Eigentümerrichtlinien können das Ergebnis weiter einschränken. Laden Sie anschließend in Campaign die Archivrichtlinie neu, wählen Sie Legacy ZipCrypto unter Anhänge → ZIP-Anhänge, bestätigen Sie die schwache Verschlüsselung und geben Sie eine betriebliche Begründung mit mindestens 10 Zeichen an. Ohne Policy bleibt Legacy gesperrt. Weder Richtlinien- noch Anhangskonfiguration versendet beim Speichern eine E-Mail."
" Mail-Migration und Archivkorrekturen lassen sich unabhängig in beliebiger Reihenfolge speichern. Eine exakt unveränderte ZIP-Konfiguration bleibt bei einer anderen Korrektur ohne erneute Bestätigung erhalten, auch nach Entzug von Richtlinie oder Berechtigung; ihre Nutzung wird dadurch nicht erlaubt. Jede geänderte ZIP-Konfiguration muss aktuelle Methoden, Passwortkanäle und Legacy-Berechtigungs- sowie Bestätigungsvorgaben erfüllen. Eine Archivkorrektur mit unveränderten öffentlichen Mail-Referenzen erhält den alten Transport exakt serverseitig bis zur ausdrücklichen autorisierten Migration. Clients dürfen dabei weder Inline-Transport einführen oder zurücksenden noch Mail-Referenzen ändern oder Bestätigungsnachweise erfinden. Beide Korrekturen entwerten Ausführungs- und Build-Nachweise; Validierung, Prüfung, Erstellung und Versand bleiben bis zur Erfüllung aller Bedingungen gesperrt."
),
},
"campaigns.workflow.link-exact-campaign-to-case": {
"title": "Eine exakte Kampagnenreferenz mit einem aktiven Fall verknüpfen",
"summary": "Eine autorisierte Kampagne und ihre aktuelle unveränderliche Version über Quick Access zurückgeben, ohne Kampagneninhalt zu kopieren.",
"body": (
"Ist ein Fall das aktive Objekt, stellt Campaigns in Quick Access eine begrenzte Auswahl bereit. Die normale Kampagnenliste prüft Mandant, Eigentum, Gruppe, Freigaben und Administrationszugriff vor der Anzeige. Die Auswahl liefert über den versionierten Ergebnisvertrag nur Eigentümermodul, stabile Kampagnen-ID, aktuelle Versions-ID, Anzeigetext, Mandant und Eigentümerroute. Cases verwirft den Anzeigetext und speichert keine Empfänger-, Nachrichten-, Anhangs-, Zustell-, Berichts- oder Konfigurationsinhalte. Beim Öffnen prüft Campaigns den Zugriff erneut. Deaktivierung, Widerruf oder Entfernung lässt daher nur eine nicht verfügbare historische Fallreferenz zurück und macht Fallzugriff nie zu Kampagnenzugriff."
),
},
}
def localize_documentation_topics(
topics: Iterable[DocumentationTopic],
) -> tuple[DocumentationTopic, ...]:
localized: list[DocumentationTopic] = []
for topic in topics:
german = _TRANSLATIONS.get(topic.id)
if german is None:
localized.append(topic)
continue
translations = {
locale: dict(value) for locale, value in topic.translations.items()
}
translations["de"] = {**translations.get("de", {}), **german}
localized.append(replace(topic, translations=translations))
return tuple(localized)
File diff suppressed because it is too large Load Diff
+747 -63
View File
@@ -7,11 +7,61 @@ from contextlib import contextmanager
from pathlib import Path
from typing import Any, Iterator
from govoplan_core.core.approvals import (
ApprovalCheck,
ApprovalRequestCreateCommand,
ApprovalRequestProvider,
ApprovalRequestRef,
CAPABILITY_APPROVAL_REQUESTS,
)
from govoplan_core.core.calendar import (
CAPABILITY_CALENDAR_INVITATIONS,
CalendarInvitationAttendeeRequest,
CalendarInvitationCalendarRef,
CalendarInvitationProvider,
CalendarInvitationRef,
CalendarInvitationRequest,
)
from govoplan_core.core.postbox import (
CAPABILITY_POSTBOX_DIRECTORY,
CAPABILITY_POSTBOX_DELIVERY,
CAPABILITY_POSTBOX_EVIDENCE,
PostboxDeliveryCatalogRef,
PostboxDeliveryProvider,
PostboxDeliveryReceiptSummaryRef,
PostboxDeliveryRequest,
PostboxDeliveryResult,
PostboxDirectoryEntryRef,
PostboxDirectoryProvider,
PostboxEvidenceProvider,
PostboxTargetRef,
)
from govoplan_core.core.templates import (
CAPABILITY_TEMPLATE_CATALOG,
CAPABILITY_TEMPLATE_CONTENT_LIBRARY,
CAPABILITY_TEMPLATE_RENDERER,
TemplateCatalogProvider,
TemplateCompatibility,
TemplateContentDraftRequest,
TemplateContentLibraryProvider,
TemplateRef,
TemplateRenderRequest,
TemplateRenderResult,
TemplateRendererProvider,
)
from govoplan_campaign.backend.runtime import capability
FILES_CAPABILITY = "files.campaign_attachments"
MAIL_CAPABILITY = "mail.campaign_delivery"
POSTBOX_CAPABILITY = CAPABILITY_POSTBOX_DELIVERY
POSTBOX_DIRECTORY_CAPABILITY = CAPABILITY_POSTBOX_DIRECTORY
POSTBOX_EVIDENCE_CAPABILITY = CAPABILITY_POSTBOX_EVIDENCE
APPROVALS_CAPABILITY = CAPABILITY_APPROVAL_REQUESTS
TEMPLATE_CATALOG_CAPABILITY = CAPABILITY_TEMPLATE_CATALOG
TEMPLATE_CONTENT_LIBRARY_CAPABILITY = CAPABILITY_TEMPLATE_CONTENT_LIBRARY
TEMPLATE_RENDERER_CAPABILITY = CAPABILITY_TEMPLATE_RENDERER
CALENDAR_INVITATIONS_CAPABILITY = CAPABILITY_CALENDAR_INVITATIONS
class OptionalModuleUnavailable(RuntimeError):
@@ -23,10 +73,22 @@ class SmtpConfigurationError(RuntimeError):
class SmtpSendError(RuntimeError):
def __init__(self, message: str, *, temporary: bool = False, outcome_unknown: bool = False) -> None:
def __init__(
self,
message: str,
*,
temporary: bool = False,
outcome_unknown: bool = False,
systemic: bool = False,
reason_code: str | None = None,
phase: str = "send",
) -> None:
super().__init__(message)
self.temporary = temporary
self.outcome_unknown = outcome_unknown
self.systemic = systemic
self.reason_code = reason_code
self.phase = phase
class ImapConfigurationError(RuntimeError):
@@ -34,15 +96,42 @@ class ImapConfigurationError(RuntimeError):
class ImapAppendError(RuntimeError):
def __init__(self, message: str, *, temporary: bool | None = None) -> None:
def __init__(
self,
message: str,
*,
temporary: bool | None = None,
outcome_unknown: bool = False,
) -> None:
super().__init__(message)
self.temporary = temporary
self.outcome_unknown = outcome_unknown
class MailProfileError(OptionalModuleUnavailable):
pass
class MailDeliveryCommandError(RuntimeError):
pass
class PostboxDeliveryUnavailable(OptionalModuleUnavailable):
pass
class ApprovalGateUnavailable(OptionalModuleUnavailable):
pass
class TemplateOutputUnavailable(OptionalModuleUnavailable):
pass
class CalendarInvitationUnavailable(OptionalModuleUnavailable):
pass
class _PreparedCampaignSnapshot:
def __init__(self, directory: Path, path: Path, raw_json: dict[str, Any]) -> None:
self._directory = directory
@@ -50,6 +139,7 @@ class _PreparedCampaignSnapshot:
self.raw_json = raw_json
self.managed_files_by_local_path: dict[str, Any] = {}
self.shared_assets: list[Any] = []
self.candidate_assets: list[Any] = []
def cleanup(self) -> None:
shutil.rmtree(self._directory, ignore_errors=True)
@@ -70,20 +160,30 @@ class FilesCampaignIntegration:
yield prepared
return
raw_json = kwargs.get("raw_json") if isinstance(kwargs.get("raw_json"), dict) else {}
raw_json = (
kwargs.get("raw_json") if isinstance(kwargs.get("raw_json"), dict) else {}
)
prefix = str(kwargs.get("prefix") or "govoplan-campaign-")
directory = Path(tempfile.mkdtemp(prefix=prefix))
snapshot = _PreparedCampaignSnapshot(directory, directory / "campaign.json", raw_json)
snapshot.path.write_text(json.dumps(raw_json, ensure_ascii=False, indent=2), encoding="utf-8")
snapshot = _PreparedCampaignSnapshot(
directory, directory / "campaign.json", raw_json
)
snapshot.path.write_text(
json.dumps(raw_json, ensure_ascii=False, indent=2), encoding="utf-8"
)
try:
yield snapshot
finally:
snapshot.cleanup()
def managed_match_payloads(self, matches: Any, managed_files_by_local_path: dict[str, Any]) -> list[dict[str, Any]]:
def managed_match_payloads(
self, matches: Any, managed_files_by_local_path: dict[str, Any]
) -> list[dict[str, Any]]:
if self._delegate is None:
return []
return self._delegate.managed_match_payloads(matches, managed_files_by_local_path)
return self._delegate.managed_match_payloads(
matches, managed_files_by_local_path
)
def public_attachment_summary_payload(self, attachment: Any) -> dict[str, Any]:
if self._delegate is not None:
@@ -94,19 +194,34 @@ class FilesCampaignIntegration:
return dict(attachment)
return {"path": str(attachment)}
def annotate_built_messages_with_managed_files(self, built_messages: Any, managed_files_by_local_path: dict[str, Any]) -> None:
def annotate_built_messages_with_managed_files(
self, built_messages: Any, managed_files_by_local_path: dict[str, Any]
) -> None:
if self._delegate is not None:
self._delegate.annotate_built_messages_with_managed_files(built_messages, managed_files_by_local_path)
self._delegate.annotate_built_messages_with_managed_files(
built_messages, managed_files_by_local_path
)
def record_campaign_attachment_uses_for_jobs(self, session: Any, jobs: Any, *, stage: str) -> None:
def record_campaign_attachment_uses_for_jobs(
self, session: Any, jobs: Any, *, stage: str
) -> None:
if self._delegate is not None:
self._delegate.record_campaign_attachment_uses_for_jobs(session, jobs, stage=stage)
self._delegate.record_campaign_attachment_uses_for_jobs(
session, jobs, stage=stage
)
def current_version_and_blob(self, session: Any, asset: Any) -> tuple[Any, Any]:
if self._delegate is None:
raise OptionalModuleUnavailable("Files module is not available")
return self._delegate.current_version_and_blob(session, asset)
def share_assets_with_campaign(
self, session: Any, **kwargs: Any
) -> list[dict[str, Any]]:
if self._delegate is None:
raise OptionalModuleUnavailable("Files module is not available")
return self._delegate.share_assets_with_campaign(session, **kwargs)
def mark_job_attachment_uses_sent(self, session: Any, job: Any) -> None:
if self._delegate is not None:
self._delegate.mark_job_attachment_uses_sent(session, job)
@@ -116,7 +231,9 @@ class MailCampaignIntegration:
def __init__(self, delegate: Any | None = None) -> None:
self._delegate = delegate
if delegate is not None:
self.MailProfileError = getattr(delegate, "MailProfileError", MailProfileError)
self.MailProfileError = getattr(
delegate, "MailProfileError", MailProfileError
)
MailProfileError = MailProfileError
SmtpConfigurationError = SmtpConfigurationError
@@ -128,100 +245,178 @@ class MailCampaignIntegration:
def available(self) -> bool:
return self._delegate is not None
@property
def durable_delivery_available(self) -> bool:
return self._delegate is not None and callable(
getattr(self._delegate, "submit_delivery_command", None)
)
def _require(self) -> Any:
if self._delegate is None:
raise MailProfileError("Mail module is not available")
return self._delegate
def materialize_campaign_mail_profile_config(self, session: Any, **kwargs: Any) -> dict[str, Any]:
def assert_campaign_mail_policy_allows_json(
self, session: Any, **kwargs: Any
) -> None:
if self._delegate is None:
raw_json = kwargs.get("raw_json")
if self.mail_profile_id_from_campaign_json(raw_json if isinstance(raw_json, dict) else {}):
raise MailProfileError("Campaign mail-server profiles require the mail module")
return dict(raw_json) if isinstance(raw_json, dict) else {}
try:
return self._delegate.materialize_campaign_mail_profile_config(session, **kwargs)
except getattr(self._delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def assert_campaign_mail_policy_allows_json(self, session: Any, **kwargs: Any) -> None:
if self._delegate is None:
raw_json = kwargs.get("raw_json")
profile_id = self.mail_profile_id_from_campaign_json(raw_json if isinstance(raw_json, dict) else {})
profile_id = self.mail_profile_id_from_campaign_json(
raw_json if isinstance(raw_json, dict) else {}
)
if profile_id:
raise MailProfileError("Campaign mail-server profiles require the mail module")
raise MailProfileError(
"Campaign mail-server profiles require the mail module"
)
return None
try:
return self._delegate.assert_campaign_mail_policy_allows_json(session, **kwargs)
return self._delegate.assert_campaign_mail_policy_allows_json(
session, **kwargs
)
except getattr(self._delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def assert_mail_policy_allows_send(self, session: Any, **kwargs: Any) -> None:
delegate = self._require()
try:
return delegate.assert_mail_policy_allows_send(session, **kwargs)
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def mail_profile_id_from_campaign_json(self, raw_json: dict[str, Any]) -> str | None:
def mail_profile_id_from_campaign_json(
self, raw_json: dict[str, Any]
) -> str | None:
if self._delegate is not None:
return self._delegate.mail_profile_id_from_campaign_json(raw_json)
server = raw_json.get("server") if isinstance(raw_json, dict) else None
profile_id = server.get("mail_profile_id") if isinstance(server, dict) else None
if profile_id is None and isinstance(server, dict):
profile_id = server.get("profile_id")
return str(profile_id).strip() if profile_id else None
def ensure_mail_profile_allowed_for_campaign(self, session: Any, **kwargs: Any) -> Any:
def campaign_profile_delivery_summary(
self, session: Any, **kwargs: Any
) -> dict[str, Any]:
delegate = self._require()
try:
return delegate.ensure_mail_profile_allowed_for_campaign(session, **kwargs)
return delegate.campaign_profile_delivery_summary(session, **kwargs)
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def smtp_config_from_profile(self, profile: Any) -> Any:
return self._require().smtp_config_from_profile(profile)
def imap_config_from_profile(self, profile: Any) -> Any:
return self._require().imap_config_from_profile(profile)
def effective_profile_credentials_inherited(self, session: Any, **kwargs: Any) -> bool:
return self._require().effective_profile_credentials_inherited(session, **kwargs)
def apply_campaign_credentials(self, profile_payload: dict[str, Any], server: dict[str, Any], protocol: str) -> dict[str, Any]:
return self._require().apply_campaign_credentials(profile_payload, server, protocol)
def wait_for_rate_limit(self, **kwargs: Any) -> None:
if self._delegate is None:
return None
return self._delegate.wait_for_rate_limit(**kwargs)
def send_email_bytes(self, *args: Any, **kwargs: Any) -> Any:
def send_campaign_email_bytes(self, *args: Any, **kwargs: Any) -> Any:
delegate = self._require()
try:
return delegate.send_email_bytes(*args, **kwargs)
return delegate.send_campaign_email_bytes(*args, **kwargs)
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
raise SmtpSendError(str(exc), temporary=bool(getattr(exc, "temporary", False)), outcome_unknown=bool(getattr(exc, "outcome_unknown", False))) from exc
raise SmtpSendError(
str(exc),
temporary=bool(getattr(exc, "temporary", False)),
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
systemic=bool(getattr(exc, "systemic", False)),
reason_code=str(getattr(exc, "reason_code", "") or "") or None,
phase=str(getattr(exc, "phase", "send") or "send"),
) from exc
except getattr(
delegate, "SmtpConfigurationError", SmtpConfigurationError
) as exc:
raise SmtpConfigurationError(str(exc)) from exc
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
@contextmanager
def campaign_smtp_batch(self, *args: Any, **kwargs: Any) -> Iterator[Any]:
delegate = self._require()
method = getattr(delegate, "campaign_smtp_batch", None)
if not callable(method):
yield None
return
try:
with method(*args, **kwargs) as state:
yield state
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
raise SmtpSendError(
str(exc),
temporary=bool(getattr(exc, "temporary", False)),
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
systemic=bool(getattr(exc, "systemic", False)),
reason_code=str(getattr(exc, "reason_code", "") or "") or None,
phase=str(getattr(exc, "phase", "preflight") or "preflight"),
) from exc
except getattr(delegate, "SmtpConfigurationError", SmtpConfigurationError) as exc:
raise SmtpConfigurationError(str(exc)) from exc
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def append_message_to_sent(self, *args: Any, **kwargs: Any) -> Any:
@contextmanager
def campaign_imap_batch(self, *, tenant_id: str, campaign_id: str) -> Iterator[Any]:
"""Use Mail-owned connection reuse when the installed capability offers it."""
delegate = self._require()
method = getattr(delegate, "campaign_imap_batch", None)
if not callable(method):
yield None
return
try:
return delegate.append_message_to_sent(*args, **kwargs)
with method(tenant_id=tenant_id, campaign_id=campaign_id) as state:
yield state
except getattr(delegate, "ImapAppendError", ImapAppendError) as exc:
raise ImapAppendError(str(exc), temporary=getattr(exc, "temporary", None)) from exc
raise ImapAppendError(
str(exc),
temporary=getattr(exc, "temporary", None),
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
) from exc
except getattr(delegate, "ImapConfigurationError", ImapConfigurationError) as exc:
raise ImapConfigurationError(str(exc)) from exc
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def send_email_message(self, *args: Any, **kwargs: Any) -> Any:
def append_campaign_message_to_sent(self, *args: Any, **kwargs: Any) -> Any:
delegate = self._require()
try:
return delegate.send_email_message(*args, **kwargs)
except getattr(delegate, "SmtpSendError", SmtpSendError) as exc:
raise SmtpSendError(str(exc), temporary=bool(getattr(exc, "temporary", False)), outcome_unknown=bool(getattr(exc, "outcome_unknown", False))) from exc
except getattr(delegate, "SmtpConfigurationError", SmtpConfigurationError) as exc:
raise SmtpConfigurationError(str(exc)) from exc
return delegate.append_campaign_message_to_sent(*args, **kwargs)
except getattr(delegate, "ImapAppendError", ImapAppendError) as exc:
raise ImapAppendError(
str(exc),
temporary=getattr(exc, "temporary", None),
outcome_unknown=bool(getattr(exc, "outcome_unknown", False)),
) from exc
except getattr(
delegate, "ImapConfigurationError", ImapConfigurationError
) as exc:
raise ImapConfigurationError(str(exc)) from exc
except getattr(delegate, "MailProfileError", MailProfileError) as exc:
raise MailProfileError(str(exc)) from exc
def submit_delivery_command(self, session: Any, **kwargs: Any) -> dict[str, Any]:
delegate = self._require()
method = getattr(delegate, "submit_delivery_command", None)
if not callable(method):
raise MailDeliveryCommandError(
"The installed Mail module does not provide durable delivery commands"
)
try:
return dict(method(session, **kwargs))
except Exception as exc:
raise MailDeliveryCommandError(str(exc)) from exc
def delivery_command_summary(
self,
session: Any,
*,
tenant_id: str,
command_id: str,
) -> dict[str, Any]:
delegate = self._require()
method = getattr(delegate, "delivery_command_summary", None)
if not callable(method):
raise MailDeliveryCommandError(
"The installed Mail module does not provide durable delivery status"
)
try:
return dict(
method(
session,
tenant_id=tenant_id,
command_id=command_id,
)
)
except Exception as exc:
raise MailDeliveryCommandError(str(exc)) from exc
def mock_mailbox(self) -> Any | None:
if self._delegate is None or not hasattr(self._delegate, "mock_mailbox"):
@@ -229,9 +424,498 @@ class MailCampaignIntegration:
return self._delegate.mock_mailbox()
class PostboxCampaignIntegration:
def __init__(
self,
delivery_delegate: object | None = None,
directory_delegate: object | None = None,
evidence_delegate: object | None = None,
) -> None:
self._delivery_delegate = (
delivery_delegate
if isinstance(delivery_delegate, PostboxDeliveryProvider)
else None
)
self._directory_delegate = (
directory_delegate
if isinstance(directory_delegate, PostboxDirectoryProvider)
else None
)
self._evidence_delegate = (
evidence_delegate
if isinstance(evidence_delegate, PostboxEvidenceProvider)
else None
)
@property
def available(self) -> bool:
return (
self._delivery_delegate is not None and self._directory_delegate is not None
)
@property
def receipt_evidence_available(self) -> bool:
return self._evidence_delegate is not None
def delivery_catalog(
self,
session: object,
*,
tenant_id: str,
) -> PostboxDeliveryCatalogRef:
if self._directory_delegate is None:
raise PostboxDeliveryUnavailable(
"Postbox targets are unavailable because the Postbox module "
"is not active."
)
return self._directory_delegate.delivery_catalog(
session,
tenant_id=tenant_id,
)
def resolve_postbox(
self,
session: object,
*,
tenant_id: str,
target: PostboxTargetRef,
materialize: bool = False,
) -> PostboxDirectoryEntryRef | None:
if self._directory_delegate is None:
raise PostboxDeliveryUnavailable(
"Postbox targets are unavailable because the Postbox module "
"is not active."
)
return self._directory_delegate.resolve_postbox(
session,
tenant_id=tenant_id,
target=target,
materialize=materialize,
)
def deliver(
self,
session: object,
request: PostboxDeliveryRequest,
) -> PostboxDeliveryResult:
if self._delivery_delegate is None:
raise PostboxDeliveryUnavailable(
"Postbox delivery is unavailable because the Postbox module "
"is not active."
)
return self._delivery_delegate.deliver(session, request)
def delivery_receipt_summaries(
self,
session: object,
*,
tenant_id: str,
delivery_ids: list[str] | tuple[str, ...],
) -> dict[str, PostboxDeliveryReceiptSummaryRef]:
if self._evidence_delegate is None:
return {}
unique_ids = tuple(dict.fromkeys(delivery_ids))
summaries: dict[str, PostboxDeliveryReceiptSummaryRef] = {}
for offset in range(0, len(unique_ids), 500):
summaries.update(
self._evidence_delegate.delivery_receipt_summaries(
session,
tenant_id=tenant_id,
producer_module="campaigns",
delivery_ids=unique_ids[offset : offset + 500],
)
)
return summaries
class ApprovalCampaignIntegration:
def __init__(self, delegate: object | None = None) -> None:
self._delegate = (
delegate if isinstance(delegate, ApprovalRequestProvider) else None
)
@property
def available(self) -> bool:
return self._delegate is not None
def create_request(
self,
session: object,
principal: object,
*,
command: ApprovalRequestCreateCommand,
idempotency_key: str,
) -> ApprovalRequestRef:
if self._delegate is None:
raise ApprovalGateUnavailable(
"Campaign approval gates require the Approvals module."
)
return self._delegate.create_request(
session,
principal,
command=command,
idempotency_key=idempotency_key,
)
def check_approved(
self,
session: object,
principal: object,
*,
request_id: str,
subject_module: str,
subject_type: str,
subject_id: str,
subject_version: str | None,
subject_digest: str,
) -> ApprovalCheck:
if self._delegate is None:
raise ApprovalGateUnavailable(
"Campaign delivery is approval-gated, but the Approvals module is unavailable."
)
return self._delegate.check_approved(
session,
principal,
request_id=request_id,
subject_module=subject_module,
subject_type=subject_type,
subject_id=subject_id,
subject_version=subject_version,
subject_digest=subject_digest,
)
class TemplatesCampaignIntegration:
def __init__(
self,
catalog_delegate: object | None = None,
renderer_delegate: object | None = None,
content_library_delegate: object | None = None,
) -> None:
self._catalog = (
catalog_delegate
if isinstance(catalog_delegate, TemplateCatalogProvider)
else None
)
self._renderer = (
renderer_delegate
if isinstance(renderer_delegate, TemplateRendererProvider)
else None
)
self._content_library = (
content_library_delegate
if isinstance(content_library_delegate, TemplateContentLibraryProvider)
else None
)
@property
def available(self) -> bool:
return self._catalog is not None and self._renderer is not None
@property
def content_available(self) -> bool:
return self._catalog is not None
@property
def content_writable(self) -> bool:
return self._content_library is not None
def list_templates(
self,
session: object,
principal: object,
*,
query: str = "",
limit: int = 100,
) -> tuple[TemplateRef, ...]:
if self._catalog is None:
return ()
return tuple(
self._catalog.list_templates(
session,
principal,
query=query,
usage="campaign_print",
limit=limit,
)
)
def list_content_templates(
self,
session: object,
principal: object,
*,
query: str = "",
limit: int = 100,
) -> tuple[TemplateRef, ...]:
if self._catalog is None:
return ()
return tuple(
self._catalog.list_templates(
session,
principal,
query=query,
usage="campaign.content",
limit=limit,
)
)
def create_content_draft(
self,
session: object,
principal: object,
*,
request: TemplateContentDraftRequest,
) -> TemplateRef:
if self._content_library is None:
raise TemplateOutputUnavailable(
"Saving reusable Campaign content requires the Templates content-library capability."
)
return self._content_library.create_content_draft(
session,
principal,
request=request,
)
def check_compatibility(
self,
session: object,
principal: object,
*,
template_id: str,
revision: int | None,
output_format: str,
available_fields: dict[str, str] | tuple[str, ...],
) -> TemplateCompatibility:
if self._catalog is None:
raise TemplateOutputUnavailable(
"Printable output is unavailable because Templates is not active."
)
return self._catalog.check_compatibility(
session,
principal,
template_id=template_id,
revision=revision,
usage="campaign_print",
output_format=output_format,
available_fields=available_fields,
)
def get_template(
self,
session: object,
principal: object,
*,
template_id: str,
revision: int,
) -> TemplateRef | None:
if self._catalog is None:
return None
return self._catalog.get_template(
session,
principal,
template_id=template_id,
revision=revision,
)
def render(
self,
session: object,
principal: object,
*,
request: TemplateRenderRequest,
) -> TemplateRenderResult:
if self._renderer is None:
raise TemplateOutputUnavailable(
"Printable output is unavailable because Templates is not active."
)
return self._renderer.render(session, principal, request=request)
class CalendarCampaignIntegration:
def __init__(self, delegate: object | None = None) -> None:
self._delegate = (
delegate if isinstance(delegate, CalendarInvitationProvider) else None
)
@property
def available(self) -> bool:
return self._delegate is not None
def list_calendars(
self,
session: object,
*,
tenant_id: str,
user_id: str | None,
group_ids: tuple[str, ...] = (),
can_admin: bool = False,
) -> tuple[CalendarInvitationCalendarRef, ...]:
if self._delegate is None:
return ()
return tuple(
self._delegate.list_calendars(
session,
tenant_id=tenant_id,
user_id=user_id,
group_ids=group_ids,
can_admin=can_admin,
)
)
def render_invitation(self, request: CalendarInvitationRequest) -> str:
if self._delegate is None:
raise CalendarInvitationUnavailable(
"Calendar invitations require the optional Calendar module."
)
return self._delegate.render_invitation(request)
def upsert_invitation(
self,
session: object,
*,
tenant_id: str,
user_id: str | None,
request: CalendarInvitationRequest,
) -> CalendarInvitationRef:
if self._delegate is None:
raise CalendarInvitationUnavailable(
"Calendar invitations require the optional Calendar module."
)
return self._delegate.upsert_invitation(
session,
tenant_id=tenant_id,
user_id=user_id,
request=request,
)
def get_invitations(
self,
session: object,
*,
tenant_id: str,
correlation_ids: tuple[str, ...],
) -> dict[str, CalendarInvitationRef]:
if self._delegate is None or not correlation_ids:
return {}
result: dict[str, CalendarInvitationRef] = {}
unique_ids = tuple(dict.fromkeys(correlation_ids))
for offset in range(0, len(unique_ids), 500):
result.update(
self._delegate.get_invitations(
session,
tenant_id=tenant_id,
correlation_ids=unique_ids[offset : offset + 500],
)
)
return result
def summarize_invitations(
self,
session: object,
*,
tenant_id: str,
source_resource_id: str | None,
) -> dict[str, object]:
if self._delegate is None:
return {
"available": False,
"reason": "The Calendar invitation capability is not active.",
}
return dict(
self._delegate.summarize_invitations(
session,
tenant_id=tenant_id,
source_module="campaigns",
source_resource_type="campaign_version",
source_resource_id=source_resource_id,
)
)
@staticmethod
def request_from_payload(payload: dict[str, Any]) -> CalendarInvitationRequest:
from datetime import datetime
attendees = tuple(
CalendarInvitationAttendeeRequest(
address=str(item.get("address") or ""),
name=str(item["name"]) if item.get("name") else None,
role=str(item.get("role") or "REQ-PARTICIPANT"),
participation_status=str(
item.get("participation_status") or "NEEDS-ACTION"
),
rsvp=bool(item.get("rsvp", True)),
)
for item in payload.get("attendees") or []
if isinstance(item, dict)
)
return CalendarInvitationRequest(
correlation_id=str(payload.get("correlation_id") or ""),
source_module="campaigns",
source_resource_type="campaign_version",
source_resource_id=(
str(payload["source_resource_id"])
if payload.get("source_resource_id")
else None
),
calendar_id=(
str(payload["calendar_id"]) if payload.get("calendar_id") else None
),
summary=str(payload.get("summary") or ""),
description=(
str(payload["description"]) if payload.get("description") else None
),
location=str(payload["location"]) if payload.get("location") else None,
start_at=datetime.fromisoformat(str(payload.get("start_at") or "")),
end_at=(
datetime.fromisoformat(str(payload["end_at"]))
if payload.get("end_at")
else None
),
timezone=str(payload["timezone"]) if payload.get("timezone") else None,
organizer=(
dict(payload["organizer"])
if isinstance(payload.get("organizer"), dict)
else None
),
attendees=attendees,
classification=str(payload.get("classification") or "PUBLIC"),
categories=tuple(str(value) for value in payload.get("categories") or []),
metadata=(
dict(payload["metadata"])
if isinstance(payload.get("metadata"), dict)
else {}
),
)
def files_integration() -> FilesCampaignIntegration:
return FilesCampaignIntegration(capability(FILES_CAPABILITY))
def mail_integration() -> MailCampaignIntegration:
return MailCampaignIntegration(capability(MAIL_CAPABILITY))
def postbox_integration() -> PostboxCampaignIntegration:
return PostboxCampaignIntegration(
capability(POSTBOX_CAPABILITY),
capability(POSTBOX_DIRECTORY_CAPABILITY),
capability(POSTBOX_EVIDENCE_CAPABILITY),
)
def approvals_integration() -> ApprovalCampaignIntegration:
return ApprovalCampaignIntegration(capability(APPROVALS_CAPABILITY))
def templates_integration() -> TemplatesCampaignIntegration:
return TemplatesCampaignIntegration(
capability(TEMPLATE_CATALOG_CAPABILITY),
capability(TEMPLATE_RENDERER_CAPABILITY),
capability(TEMPLATE_CONTENT_LIBRARY_CAPABILITY),
)
def calendar_integration() -> CalendarCampaignIntegration:
return CalendarCampaignIntegration(capability(CALENDAR_INVITATIONS_CAPABILITY))
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -33,6 +33,7 @@ class MessageIssue(BaseModel):
message: str
behavior: str | None = None
source: str | None = None
details: dict[str, object] = Field(default_factory=dict)
class MessageAddress(BaseModel):
@@ -49,6 +50,7 @@ class MessageAttachmentSummary(BaseModel):
label: str | None = None
status: str
behavior: str | None = None
missing_policy: dict[str, object] | None = None
required: bool
allow_multiple: bool
zip_enabled: bool
@@ -78,6 +80,7 @@ class MessageDraft(BaseModel):
validation_status: MessageValidationStatus
send_status: SendStatus
imap_status: ImapStatus
delivery_channel_policy: str = "mail"
subject: str | None = None
from_: MessageAddress | None = Field(default=None, alias="from")
@@ -91,6 +94,7 @@ class MessageDraft(BaseModel):
attachment_count: int = 0
attachments: list[MessageAttachmentSummary] = Field(default_factory=list)
archive_evidence: list[dict[str, object]] = Field(default_factory=list)
issues: list[MessageIssue] = Field(default_factory=list)
eml_path: str | None = None
@@ -114,6 +118,8 @@ class CampaignBuildReport(BaseModel):
inactive_entries_count: int = 0
messages: list[MessageDraft] = Field(default_factory=list)
attachment_resolution_profile: dict[str, object] = Field(default_factory=dict)
attachment_reuse: dict[str, object] = Field(default_factory=dict)
residual_file_disposition: dict[str, object] = Field(default_factory=dict)
@property
def built_count(self) -> int:
@@ -1,7 +1,7 @@
"""recipient import mapping profiles
Revision ID: 2c3d4e5f7081
Revises: 1b2c3d4e5f70
Revises: 2e3f4a5b6c7d
Create Date: 2026-06-26 00:00:00.000000
"""
from __future__ import annotations
@@ -11,15 +11,21 @@ import sqlalchemy as sa
revision = "2c3d4e5f7081"
down_revision = "1b2c3d4e5f70"
down_revision = "2e3f4a5b6c7d"
branch_labels = None
depends_on = None
def _scope_fk_target(inspector) -> str:
tables = set(inspector.get_table_names())
return "core_scopes.id" if "core_scopes" in tables else "tenancy_tenants.id"
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if "campaign_recipient_import_mapping_profiles" in inspector.get_table_names():
return
scope_fk_target = _scope_fk_target(inspector)
op.create_table(
"campaign_recipient_import_mapping_profiles",
sa.Column("id", sa.String(length=36), nullable=False),
@@ -38,8 +44,8 @@ def upgrade() -> None:
sa.Column("mappings", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["owner_user_id"], ["users.id"], name=op.f("fk_campaign_recipient_import_mapping_profiles_owner_user_id_users"), ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], name=op.f("fk_campaign_recipient_import_mapping_profiles_tenant_id_tenants"), ondelete="CASCADE"),
sa.ForeignKeyConstraint(["owner_user_id"], ["access_users.id"], name=op.f("fk_campaign_recipient_import_mapping_profiles_owner_user_id_users"), ondelete="CASCADE"),
sa.ForeignKeyConstraint(["tenant_id"], [scope_fk_target], name=op.f("fk_campaign_recipient_import_mapping_profiles_tenant_id_scopes"), ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id", name=op.f("pk_campaign_recipient_import_mapping_profiles")),
)
op.create_index(op.f("ix_campaign_recipient_import_mapping_profiles_tenant_id"), "campaign_recipient_import_mapping_profiles", ["tenant_id"], unique=False)
@@ -0,0 +1,67 @@
"""add durable IMAP append claim and attempt idempotency
Revision ID: 3c4d5e6f8192
Revises: 2c3d4e5f7081
Create Date: 2026-07-21 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "3c4d5e6f8192"
down_revision = "2c3d4e5f7081"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_jobs") as batch:
batch.add_column(sa.Column("imap_claimed_at", sa.DateTime(timezone=True), nullable=True))
batch.add_column(sa.Column("imap_claim_token", sa.String(length=36), nullable=True))
batch.create_index("ix_campaign_jobs_imap_claim_token", ["imap_claim_token"], unique=False)
with op.batch_alter_table("imap_append_attempts") as batch:
batch.add_column(sa.Column("claim_token", sa.String(length=36), nullable=True))
_renumber_attempts()
with op.batch_alter_table("imap_append_attempts") as batch:
batch.create_unique_constraint(
"uq_imap_append_attempts_job_attempt",
["job_id", "attempt_number"],
)
def downgrade() -> None:
with op.batch_alter_table("imap_append_attempts") as batch:
batch.drop_constraint("uq_imap_append_attempts_job_attempt", type_="unique")
batch.drop_column("claim_token")
with op.batch_alter_table("campaign_jobs") as batch:
batch.drop_index("ix_campaign_jobs_imap_claim_token")
batch.drop_column("imap_claim_token")
batch.drop_column("imap_claimed_at")
def _renumber_attempts() -> None:
"""Make historical attempt numbers unique per job before constraining them."""
bind = op.get_bind()
rows = list(
bind.execute(
sa.text(
"SELECT id, job_id FROM imap_append_attempts "
"ORDER BY job_id, created_at, id"
)
).mappings()
)
per_job: dict[str, int] = {}
for row in rows:
job_id = str(row["job_id"])
attempt_number = per_job.get(job_id, 0) + 1
per_job[job_id] = attempt_number
bind.execute(
sa.text(
"UPDATE imap_append_attempts SET attempt_number = :attempt_number "
"WHERE id = :attempt_id"
),
{"attempt_number": attempt_number, "attempt_id": row["id"]},
)
@@ -0,0 +1,26 @@
"""seal each campaign delivery job's immutable execution input
Revision ID: 4d5e6f7a9203
Revises: 3c4d5e6f8192
Create Date: 2026-07-21 00:00:01.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "4d5e6f7a9203"
down_revision = "3c4d5e6f8192"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_jobs") as batch:
batch.add_column(sa.Column("execution_input_sha256", sa.String(length=64), nullable=True))
def downgrade() -> None:
with op.batch_alter_table("campaign_jobs") as batch:
batch.drop_column("execution_input_sha256")
@@ -0,0 +1,27 @@
"""add campaign schedule optimistic-concurrency revisions
Revision ID: a5b6c7d8e9f0
Revises: f4a5b6c7d8e9
Create Date: 2026-08-07 12:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
_migration = import_module(
"govoplan_campaign.backend.migrations.versions."
"a5b6c7d8e9f0_v0120_campaign_schedule_revisions"
)
revision = _migration.revision
down_revision = _migration.down_revision
branch_labels = _migration.branch_labels
depends_on = _migration.depends_on
def upgrade() -> None:
_migration.upgrade()
def downgrade() -> None:
_migration.downgrade()
@@ -0,0 +1,17 @@
"""Development wrapper for the canonical printable-delivery migration."""
from __future__ import annotations
from importlib import import_module
_migration = import_module(
"govoplan_campaign.backend.migrations.versions."
"b7c8d9e0f1a2_campaign_print_delivery"
)
revision = _migration.revision
down_revision = _migration.down_revision
branch_labels = _migration.branch_labels
depends_on = _migration.depends_on
upgrade = _migration.upgrade
downgrade = _migration.downgrade
@@ -0,0 +1,22 @@
"""add audit-proof single-message action ledger
Revision ID: c1a69e4f2b70
Revises: f0a1b2c3d4e5
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
message_actions = import_module(
"govoplan_campaign.backend.migrations.versions.c1a69e4f2b70_campaign_message_actions"
)
revision = message_actions.revision
down_revision = message_actions.down_revision
branch_labels = message_actions.branch_labels
depends_on = message_actions.depends_on
upgrade = message_actions.upgrade
downgrade = message_actions.downgrade
@@ -0,0 +1,30 @@
"""persist the selected Campaign delivery mode
Revision ID: c7a2f91e4b60
Revises: 4d5e6f7a9203
Create Date: 2026-07-22 09:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c7a2f91e4b60"
down_revision = "4d5e6f7a9203"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.add_column(sa.Column("delivery_mode", sa.String(length=30), nullable=True))
batch.add_column(sa.Column("delivery_mode_selected_at", sa.DateTime(timezone=True), nullable=True))
batch.create_index("ix_campaign_versions_delivery_mode", ["delivery_mode"], unique=False)
def downgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.drop_index("ix_campaign_versions_delivery_mode")
batch.drop_column("delivery_mode_selected_at")
batch.drop_column("delivery_mode")
@@ -0,0 +1,22 @@
"""add monotonic campaign version edit revision
Revision ID: d2b7af503c81
Revises: c1a69e4f2b70
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
edit_revision = import_module(
"govoplan_campaign.backend.migrations.versions.d2b7af503c81_campaign_version_edit_revision"
)
revision = edit_revision.revision
down_revision = edit_revision.down_revision
branch_labels = edit_revision.branch_labels
depends_on = edit_revision.depends_on
upgrade = edit_revision.upgrade
downgrade = edit_revision.downgrade
@@ -0,0 +1,42 @@
"""mark untouched excluded jobs as skipped delivery
Revision ID: d8b3e2c1f4a5
Revises: c7a2f91e4b60
Create Date: 2026-07-22 11:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d8b3e2c1f4a5"
down_revision = "c7a2f91e4b60"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Only normalize rows with no recorded transport effect. Unexpected
# historical delivery evidence must remain intact for audit/reconciliation.
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'skipped', imap_status = 'skipped' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'not_queued' "
"AND imap_status IN ('not_requested', 'pending', 'skipped')"
)
)
def downgrade() -> None:
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'not_queued', imap_status = 'not_requested' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'skipped' "
"AND imap_status = 'skipped'"
)
)
@@ -0,0 +1,85 @@
"""add non-destructive historical campaign version archival
Revision ID: e3c8f4a5b6d7
Revises: b7c8d9e0f1a2, d2b7af503c81
Create Date: 2026-08-03 12:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "e3c8f4a5b6d7"
down_revision = ("b7c8d9e0f1a2", "d2b7af503c81")
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
columns = {
column["name"]
for column in inspector.get_columns("campaign_versions")
}
with op.batch_alter_table("campaign_versions") as batch_op:
if "archived_at" not in columns:
batch_op.add_column(
sa.Column("archived_at", sa.DateTime(timezone=True), nullable=True)
)
if "archived_by_user_id" not in columns:
batch_op.add_column(
sa.Column("archived_by_user_id", sa.String(length=36), nullable=True)
)
batch_op.create_foreign_key(
"fk_campaign_versions_archived_by_user_id_access_users",
"access_users",
["archived_by_user_id"],
["id"],
ondelete="SET NULL",
)
indexes = {
index["name"]
for index in sa.inspect(op.get_bind()).get_indexes("campaign_versions")
}
if "ix_campaign_versions_archived_at" not in indexes:
op.create_index(
"ix_campaign_versions_archived_at",
"campaign_versions",
["archived_at"],
unique=False,
)
if "ix_campaign_versions_archived_by_user_id" not in indexes:
op.create_index(
"ix_campaign_versions_archived_by_user_id",
"campaign_versions",
["archived_by_user_id"],
unique=False,
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
indexes = {
index["name"]
for index in inspector.get_indexes("campaign_versions")
}
for index_name in (
"ix_campaign_versions_archived_by_user_id",
"ix_campaign_versions_archived_at",
):
if index_name in indexes:
op.drop_index(index_name, table_name="campaign_versions")
columns = {
column["name"]
for column in sa.inspect(op.get_bind()).get_columns("campaign_versions")
}
with op.batch_alter_table("campaign_versions") as batch_op:
if "archived_by_user_id" in columns:
batch_op.drop_constraint(
"fk_campaign_versions_archived_by_user_id_access_users",
type_="foreignkey",
)
batch_op.drop_column("archived_by_user_id")
if "archived_at" in columns:
batch_op.drop_column("archived_at")
@@ -0,0 +1,32 @@
"""repair a missing IMAP append attempt claim token
Revision ID: e9f0a1b2c3d4
Revises: d8b3e2c1f4a5
Create Date: 2026-07-28 23:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "e9f0a1b2c3d4"
down_revision = "d8b3e2c1f4a5"
branch_labels = None
depends_on = None
def upgrade() -> None:
bind = op.get_bind()
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
if "claim_token" not in columns:
op.add_column(
"imap_append_attempts",
sa.Column("claim_token", sa.String(length=36), nullable=True),
)
def downgrade() -> None:
# The column belongs to revision 3c4d5e6f8192. This repair revision only
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
pass
@@ -0,0 +1,22 @@
"""add governed campaign Postbox delivery
Revision ID: f0a1b2c3d4e5
Revises: e9f0a1b2c3d4
Create Date: 2026-07-29 02:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
_migration = import_module(
"govoplan_campaign.backend.migrations.versions."
"f0a1b2c3d4e5_v0115_postbox_delivery"
)
revision = _migration.revision
down_revision = _migration.down_revision
branch_labels = _migration.branch_labels
depends_on = _migration.depends_on
upgrade = _migration.upgrade
downgrade = _migration.downgrade
@@ -0,0 +1,27 @@
"""add durable campaign schedules and occurrence evidence
Revision ID: f4a5b6c7d8e9
Revises: e3c8f4a5b6d7
Create Date: 2026-08-07 10:00:00.000000
"""
from __future__ import annotations
from importlib import import_module
_migration = import_module(
"govoplan_campaign.backend.migrations.versions."
"f4a5b6c7d8e9_v0119_campaign_schedules"
)
revision = _migration.revision
down_revision = _migration.down_revision
branch_labels = _migration.branch_labels
depends_on = _migration.depends_on
def upgrade() -> None:
_migration.upgrade()
def downgrade() -> None:
_migration.downgrade()
@@ -0,0 +1,51 @@
"""v0.1.7 campaign baseline
Revision ID: 2c3d4e5f7081
Revises: None
Create Date: 2026-07-11 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = '2c3d4e5f7081'
down_revision = None
branch_labels = None
depends_on = '4f2a9c8e7b6d'
def upgrade() -> None:
op.create_table('campaign_recipient_import_mapping_profiles',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('tenant_id', sa.String(length=36), nullable=False),
sa.Column('owner_user_id', sa.String(length=36), nullable=False),
sa.Column('name', sa.String(length=255), nullable=False),
sa.Column('column_count', sa.Integer(), nullable=False),
sa.Column('headers', sa.JSON(), nullable=False),
sa.Column('normalized_headers', sa.JSON(), nullable=False),
sa.Column('ordered_header_fingerprint', sa.String(length=64), nullable=False),
sa.Column('unordered_header_fingerprint', sa.String(length=64), nullable=False),
sa.Column('delimiter', sa.String(length=8), nullable=False),
sa.Column('header_rows', sa.Integer(), nullable=False),
sa.Column('quoted', sa.Boolean(), nullable=False),
sa.Column('value_separators', sa.String(length=50), nullable=False),
sa.Column('mappings', sa.JSON(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['owner_user_id'], ['access_users.id'], name=op.f('fk_campaign_recipient_import_mapping_profiles_owner_user_id_access_users'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['tenant_id'], ['core_scopes.id'], name=op.f('fk_campaign_recipient_import_mapping_profiles_tenant_id_scopes'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_campaign_recipient_import_mapping_profiles'))
)
op.create_index(op.f('ix_campaign_recipient_import_mapping_profiles_ordered_header_fingerprint'), 'campaign_recipient_import_mapping_profiles', ['ordered_header_fingerprint'], unique=False)
op.create_index(op.f('ix_campaign_recipient_import_mapping_profiles_owner_user_id'), 'campaign_recipient_import_mapping_profiles', ['owner_user_id'], unique=False)
op.create_index(op.f('ix_campaign_recipient_import_mapping_profiles_tenant_id'), 'campaign_recipient_import_mapping_profiles', ['tenant_id'], unique=False)
op.create_index(op.f('ix_campaign_recipient_import_mapping_profiles_unordered_header_fingerprint'), 'campaign_recipient_import_mapping_profiles', ['unordered_header_fingerprint'], unique=False)
op.create_index('ix_recipient_import_profiles_ordered_fp', 'campaign_recipient_import_mapping_profiles', ['tenant_id', 'owner_user_id', 'ordered_header_fingerprint'], unique=False)
op.create_index('ix_recipient_import_profiles_owner', 'campaign_recipient_import_mapping_profiles', ['tenant_id', 'owner_user_id'], unique=False)
op.create_index('ix_recipient_import_profiles_unordered_fp', 'campaign_recipient_import_mapping_profiles', ['tenant_id', 'owner_user_id', 'unordered_header_fingerprint'], unique=False)
def downgrade() -> None:
op.drop_table('campaign_recipient_import_mapping_profiles')
@@ -0,0 +1,67 @@
"""add durable IMAP append claim and attempt idempotency
Revision ID: 3c4d5e6f8192
Revises: 2c3d4e5f7081
Create Date: 2026-07-21 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "3c4d5e6f8192"
down_revision = "2c3d4e5f7081"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_jobs") as batch:
batch.add_column(sa.Column("imap_claimed_at", sa.DateTime(timezone=True), nullable=True))
batch.add_column(sa.Column("imap_claim_token", sa.String(length=36), nullable=True))
batch.create_index("ix_campaign_jobs_imap_claim_token", ["imap_claim_token"], unique=False)
with op.batch_alter_table("imap_append_attempts") as batch:
batch.add_column(sa.Column("claim_token", sa.String(length=36), nullable=True))
_renumber_attempts()
with op.batch_alter_table("imap_append_attempts") as batch:
batch.create_unique_constraint(
"uq_imap_append_attempts_job_attempt",
["job_id", "attempt_number"],
)
def downgrade() -> None:
with op.batch_alter_table("imap_append_attempts") as batch:
batch.drop_constraint("uq_imap_append_attempts_job_attempt", type_="unique")
batch.drop_column("claim_token")
with op.batch_alter_table("campaign_jobs") as batch:
batch.drop_index("ix_campaign_jobs_imap_claim_token")
batch.drop_column("imap_claim_token")
batch.drop_column("imap_claimed_at")
def _renumber_attempts() -> None:
"""Make historical attempt numbers unique per job before constraining them."""
bind = op.get_bind()
rows = list(
bind.execute(
sa.text(
"SELECT id, job_id FROM imap_append_attempts "
"ORDER BY job_id, created_at, id"
)
).mappings()
)
per_job: dict[str, int] = {}
for row in rows:
job_id = str(row["job_id"])
attempt_number = per_job.get(job_id, 0) + 1
per_job[job_id] = attempt_number
bind.execute(
sa.text(
"UPDATE imap_append_attempts SET attempt_number = :attempt_number "
"WHERE id = :attempt_id"
),
{"attempt_number": attempt_number, "attempt_id": row["id"]},
)
@@ -0,0 +1,26 @@
"""seal each campaign delivery job's immutable execution input
Revision ID: 4d5e6f7a9203
Revises: 3c4d5e6f8192
Create Date: 2026-07-21 00:00:01.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "4d5e6f7a9203"
down_revision = "3c4d5e6f8192"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_jobs") as batch:
batch.add_column(sa.Column("execution_input_sha256", sa.String(length=64), nullable=True))
def downgrade() -> None:
with op.batch_alter_table("campaign_jobs") as batch:
batch.drop_column("execution_input_sha256")
@@ -0,0 +1,40 @@
"""add campaign schedule optimistic-concurrency revisions
Revision ID: a5b6c7d8e9f0
Revises: f4a5b6c7d8e9
Create Date: 2026-08-07 12:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "a5b6c7d8e9f0"
down_revision = "f4a5b6c7d8e9"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_schedules"):
columns = {column["name"] for column in inspector.get_columns("campaign_schedules")}
if "resource_revision" not in columns:
op.add_column(
"campaign_schedules",
sa.Column(
"resource_revision",
sa.Integer(),
nullable=False,
server_default="1",
),
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_schedules"):
columns = {column["name"] for column in inspector.get_columns("campaign_schedules")}
if "resource_revision" in columns:
op.drop_column("campaign_schedules", "resource_revision")
@@ -0,0 +1,100 @@
"""add governed autonomous Campaign schedule evidence
revision = "b6c7d8e9f0a1"
down_revision = "a5b6c7d8e9f0"
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "b6c7d8e9f0a1"
down_revision = "a5b6c7d8e9f0"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"campaign_schedules",
sa.Column("delivery_mode", sa.String(length=20), nullable=False, server_default="manual"),
)
op.create_index(
"ix_campaign_schedules_delivery_mode",
"campaign_schedules",
["delivery_mode"],
)
op.add_column(
"campaign_schedules",
sa.Column("approved_execution_snapshot_hash", sa.String(length=64), nullable=True),
)
op.create_index(
"ix_campaign_schedules_approved_execution_snapshot_hash",
"campaign_schedules",
["approved_execution_snapshot_hash"],
)
op.add_column(
"campaign_schedules",
sa.Column("last_outcome", sa.String(length=30), nullable=True),
)
op.add_column(
"campaign_schedules",
sa.Column("last_recovery_state", sa.String(length=30), nullable=True),
)
op.add_column(
"campaign_schedule_occurrences",
sa.Column("idempotency_key", sa.String(length=200), nullable=True),
)
op.create_index(
"ix_campaign_schedule_occurrences_idempotency_key",
"campaign_schedule_occurrences",
["idempotency_key"],
)
op.add_column(
"campaign_schedule_occurrences",
sa.Column("delivery_command_ids", sa.JSON(), nullable=False, server_default="[]"),
)
op.add_column(
"campaign_schedule_occurrences",
sa.Column("recovery_state", sa.String(length=30), nullable=False, server_default="none"),
)
op.create_index(
"ix_campaign_schedule_occurrences_recovery_state",
"campaign_schedule_occurrences",
["recovery_state"],
)
op.add_column(
"campaign_schedule_occurrences",
sa.Column("evidence", sa.JSON(), nullable=False, server_default="{}"),
)
op.add_column(
"campaign_schedule_occurrences",
sa.Column("last_checked_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
op.drop_column("campaign_schedule_occurrences", "last_checked_at")
op.drop_column("campaign_schedule_occurrences", "evidence")
op.drop_index(
"ix_campaign_schedule_occurrences_recovery_state",
table_name="campaign_schedule_occurrences",
)
op.drop_column("campaign_schedule_occurrences", "recovery_state")
op.drop_column("campaign_schedule_occurrences", "delivery_command_ids")
op.drop_index(
"ix_campaign_schedule_occurrences_idempotency_key",
table_name="campaign_schedule_occurrences",
)
op.drop_column("campaign_schedule_occurrences", "idempotency_key")
op.drop_column("campaign_schedules", "last_recovery_state")
op.drop_column("campaign_schedules", "last_outcome")
op.drop_index(
"ix_campaign_schedules_approved_execution_snapshot_hash",
table_name="campaign_schedules",
)
op.drop_column("campaign_schedules", "approved_execution_snapshot_hash")
op.drop_index("ix_campaign_schedules_delivery_mode", table_name="campaign_schedules")
op.drop_column("campaign_schedules", "delivery_mode")
@@ -0,0 +1,112 @@
"""add governed campaign printable delivery
Revision ID: b7c8d9e0f1a2
Revises: f0a1b2c3d4e5
Create Date: 2026-08-02 12:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "b7c8d9e0f1a2"
down_revision = "f0a1b2c3d4e5"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"campaign_jobs",
sa.Column(
"print_status",
sa.String(length=50),
nullable=False,
server_default="not_requested",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"print_attempt_count",
sa.Integer(),
nullable=False,
server_default="0",
),
)
op.add_column(
"campaign_jobs",
sa.Column("resolved_print_output", sa.JSON(), nullable=True),
)
op.add_column(
"campaign_jobs",
sa.Column(
"delivery_provenance",
sa.JSON(),
nullable=False,
server_default="{}",
),
)
op.create_index(
op.f("ix_campaign_jobs_print_status"),
"campaign_jobs",
["print_status"],
unique=False,
)
op.create_table(
"campaign_print_output_attempts",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("job_id", sa.String(length=36), nullable=False),
sa.Column("attempt_number", sa.Integer(), nullable=False),
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("render_id", sa.String(length=36), nullable=True),
sa.Column("artifact_sha256", sa.String(length=64), nullable=True),
sa.Column("evidence", sa.JSON(), nullable=False),
sa.Column("error_message", sa.Text(), nullable=True),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["job_id"],
["campaign_jobs.id"],
name=op.f("fk_campaign_print_output_attempts_job_id_campaign_jobs"),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_print_output_attempts"),
),
sa.UniqueConstraint(
"job_id",
"attempt_number",
name="uq_campaign_print_attempt_job_number",
),
sa.UniqueConstraint(
"tenant_id",
"idempotency_key",
name="uq_campaign_print_attempt_idempotency",
),
)
for column in ("tenant_id", "job_id", "status", "render_id", "artifact_sha256"):
op.create_index(
op.f(f"ix_campaign_print_output_attempts_{column}"),
"campaign_print_output_attempts",
[column],
unique=False,
)
def downgrade() -> None:
op.drop_table("campaign_print_output_attempts")
op.drop_index(
op.f("ix_campaign_jobs_print_status"),
table_name="campaign_jobs",
)
op.drop_column("campaign_jobs", "resolved_print_output")
op.drop_column("campaign_jobs", "delivery_provenance")
op.drop_column("campaign_jobs", "print_attempt_count")
op.drop_column("campaign_jobs", "print_status")
@@ -0,0 +1,186 @@
"""add audit-proof single-message action ledger
Revision ID: c1a69e4f2b70
Revises: f0a1b2c3d4e5
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c1a69e4f2b70"
down_revision = "f0a1b2c3d4e5"
branch_labels = None
depends_on = "c91f0a72be34"
def upgrade() -> None:
tables = set(sa.inspect(op.get_bind()).get_table_names())
if "campaign_message_actions" not in tables:
op.create_table(
"campaign_message_actions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("campaign_id", sa.String(length=36), nullable=False),
sa.Column("campaign_version_id", sa.String(length=36), nullable=False),
sa.Column("job_id", sa.String(length=36), nullable=False),
sa.Column("kind", sa.String(length=30), nullable=False),
sa.Column("idempotency_key", sa.String(length=200), nullable=False),
sa.Column("canonical_request_hash", sa.String(length=64), nullable=False),
sa.Column("reason", sa.Text(), nullable=True),
sa.Column("context", sa.JSON(), nullable=False),
sa.Column("actor_user_id", sa.String(length=36), nullable=True),
sa.Column("actor_api_key_id", sa.String(length=36), nullable=True),
sa.Column("message_sha256", sa.String(length=64), nullable=False),
sa.Column("message_size_bytes", sa.Integer(), nullable=True),
sa.Column(
"recipient_manifest_sha256",
sa.String(length=64),
nullable=False,
),
sa.Column("recipient_count", sa.Integer(), nullable=False),
sa.Column("prior_send_status", sa.String(length=50), nullable=False),
sa.Column("prior_attempt_count", sa.Integer(), nullable=False),
sa.Column("final_send_status", sa.String(length=50), nullable=True),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("accepted_count", sa.Integer(), nullable=False),
sa.Column("refused_count", sa.Integer(), nullable=False),
sa.Column("refusal_summary", sa.JSON(), nullable=False),
sa.Column("error_type", sa.String(length=120), nullable=True),
sa.Column("error_message", sa.String(length=500), nullable=True),
sa.Column("linked_send_attempt_id", sa.String(length=36), nullable=True),
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["campaign_id"],
["campaigns.id"],
name=op.f(
"fk_campaign_message_actions_campaign_id_campaigns"
),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["campaign_version_id"],
["campaign_versions.id"],
name=op.f(
"fk_campaign_message_actions_campaign_version_id_campaign_versions"
),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["job_id"],
["campaign_jobs.id"],
name=op.f(
"fk_campaign_message_actions_job_id_campaign_jobs"
),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["actor_user_id"],
["access_users.id"],
name=op.f(
"fk_campaign_message_actions_actor_user_id_access_users"
),
ondelete="SET NULL",
),
sa.ForeignKeyConstraint(
["linked_send_attempt_id"],
["send_attempts.id"],
name=op.f(
"fk_campaign_message_actions_linked_send_attempt_id_send_attempts"
),
ondelete="SET NULL",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_message_actions"),
),
sa.UniqueConstraint(
"tenant_id",
"idempotency_key",
name="uq_campaign_message_actions_idempotency",
),
)
op.create_index(
"ix_campaign_message_actions_job_created",
"campaign_message_actions",
["job_id", "created_at"],
unique=False,
)
op.create_index(
"ix_campaign_message_actions_campaign_kind",
"campaign_message_actions",
["campaign_id", "kind", "status"],
unique=False,
)
for column in (
"tenant_id",
"campaign_id",
"campaign_version_id",
"job_id",
"kind",
"actor_user_id",
"status",
"linked_send_attempt_id",
):
op.create_index(
op.f(f"ix_campaign_message_actions_{column}"),
"campaign_message_actions",
[column],
unique=False,
)
tables = set(sa.inspect(op.get_bind()).get_table_names())
if "campaign_message_action_attempts" not in tables:
op.create_table(
"campaign_message_action_attempts",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("action_id", sa.String(length=36), nullable=False),
sa.Column("attempt_number", sa.Integer(), nullable=False),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("effect_started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("accepted_count", sa.Integer(), nullable=False),
sa.Column("refused_count", sa.Integer(), nullable=False),
sa.Column("outcome_code", sa.String(length=80), nullable=True),
sa.Column("diagnostic_summary", sa.String(length=500), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["action_id"],
["campaign_message_actions.id"],
name=op.f(
"fk_campaign_message_action_attempts_action_id_campaign_message_actions"
),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_message_action_attempts"),
),
sa.UniqueConstraint(
"action_id",
"attempt_number",
name="uq_campaign_message_action_attempt_number",
),
)
for column in ("action_id", "status"):
op.create_index(
op.f(f"ix_campaign_message_action_attempts_{column}"),
"campaign_message_action_attempts",
[column],
unique=False,
)
def downgrade() -> None:
tables = set(sa.inspect(op.get_bind()).get_table_names())
if "campaign_message_action_attempts" in tables:
op.drop_table("campaign_message_action_attempts")
if "campaign_message_actions" in tables:
op.drop_table("campaign_message_actions")
@@ -0,0 +1,30 @@
"""persist the selected Campaign delivery mode
Revision ID: c7a2f91e4b60
Revises: 4d5e6f7a9203
Create Date: 2026-07-22 09:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "c7a2f91e4b60"
down_revision = "4d5e6f7a9203"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.add_column(sa.Column("delivery_mode", sa.String(length=30), nullable=True))
batch.add_column(sa.Column("delivery_mode_selected_at", sa.DateTime(timezone=True), nullable=True))
batch.create_index("ix_campaign_versions_delivery_mode", ["delivery_mode"], unique=False)
def downgrade() -> None:
with op.batch_alter_table("campaign_versions") as batch:
batch.drop_index("ix_campaign_versions_delivery_mode")
batch.drop_column("delivery_mode_selected_at")
batch.drop_column("delivery_mode")
@@ -0,0 +1,89 @@
"""add governed campaign collaboration entries
revision = "c7d8e9f0a1b2"
down_revision = "b6c7d8e9f0a1"
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "c7d8e9f0a1b2"
down_revision = "b6c7d8e9f0a1"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_collaboration_entries"):
return
op.create_table(
"campaign_collaboration_entries",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("campaign_id", sa.String(length=36), nullable=False),
sa.Column("campaign_version_id", sa.String(length=36), nullable=True),
sa.Column("reference_kind", sa.String(length=40), nullable=True),
sa.Column("reference_id", sa.String(length=500), nullable=True),
sa.Column("reference_label", sa.String(length=255), nullable=True),
sa.Column("actor_user_id", sa.String(length=36), nullable=True),
sa.Column("actor_label_snapshot", sa.String(length=255), nullable=False),
sa.Column(
"visibility",
sa.String(length=30),
nullable=False,
server_default="collaborators",
),
sa.Column("content", sa.Text(), nullable=True),
sa.Column("content_sha256", sa.String(length=64), nullable=False),
sa.Column("mention_user_ids", sa.JSON(), nullable=False, server_default="[]"),
sa.Column("withdrawn_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("withdrawn_by_user_id", sa.String(length=36), nullable=True),
sa.Column("redacted_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("redacted_by_user_id", sa.String(length=36), nullable=True),
sa.Column("tombstone_reason", sa.String(length=500), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["actor_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["campaign_id"], ["campaigns.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(
["campaign_version_id"],
["campaign_versions.id"],
ondelete="RESTRICT",
),
sa.ForeignKeyConstraint(
["redacted_by_user_id"],
["access_users.id"],
ondelete="SET NULL",
),
sa.ForeignKeyConstraint(
["withdrawn_by_user_id"],
["access_users.id"],
ondelete="SET NULL",
),
sa.PrimaryKeyConstraint("id"),
)
for name, columns in (
("ix_campaign_collaboration_entries_tenant_id", ["tenant_id"]),
("ix_campaign_collaboration_entries_campaign_id", ["campaign_id"]),
("ix_campaign_collaboration_entries_campaign_version_id", ["campaign_version_id"]),
("ix_campaign_collaboration_entries_reference_kind", ["reference_kind"]),
("ix_campaign_collaboration_entries_actor_user_id", ["actor_user_id"]),
("ix_campaign_collaboration_entries_visibility", ["visibility"]),
("ix_campaign_collaboration_entries_withdrawn_at", ["withdrawn_at"]),
("ix_campaign_collaboration_entries_redacted_at", ["redacted_at"]),
(
"ix_campaign_collaboration_entries_thread",
["tenant_id", "campaign_id", "created_at", "id"],
),
):
op.create_index(name, "campaign_collaboration_entries", columns, unique=False)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_collaboration_entries"):
op.drop_table("campaign_collaboration_entries")
@@ -0,0 +1,45 @@
"""add monotonic campaign version edit revision
Revision ID: d2b7af503c81
Revises: c1a69e4f2b70
Create Date: 2026-07-30 00:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d2b7af503c81"
down_revision = "c1a69e4f2b70"
branch_labels = None
depends_on = "c91f0a72be34"
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
columns = {
column["name"]
for column in inspector.get_columns("campaign_versions")
}
if "edit_revision" not in columns:
with op.batch_alter_table("campaign_versions") as batch_op:
batch_op.add_column(
sa.Column(
"edit_revision",
sa.Integer(),
server_default="1",
nullable=False,
)
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
columns = {
column["name"]
for column in inspector.get_columns("campaign_versions")
}
if "edit_revision" in columns:
with op.batch_alter_table("campaign_versions") as batch_op:
batch_op.drop_column("edit_revision")
@@ -0,0 +1,42 @@
"""mark untouched excluded jobs as skipped delivery
Revision ID: d8b3e2c1f4a5
Revises: c7a2f91e4b60
Create Date: 2026-07-22 11:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "d8b3e2c1f4a5"
down_revision = "c7a2f91e4b60"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Only normalize rows with no recorded transport effect. Unexpected
# historical delivery evidence must remain intact for audit/reconciliation.
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'skipped', imap_status = 'skipped' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'not_queued' "
"AND imap_status IN ('not_requested', 'pending', 'skipped')"
)
)
def downgrade() -> None:
op.get_bind().execute(
sa.text(
"UPDATE campaign_jobs "
"SET send_status = 'not_queued', imap_status = 'not_requested' "
"WHERE validation_status = 'excluded' "
"AND send_status = 'skipped' "
"AND imap_status = 'skipped'"
)
)
@@ -0,0 +1,113 @@
"""add accountable campaign work assignments
revision = "d8e9f0a1b2c3"
down_revision = "c7d8e9f0a1b2"
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "d8e9f0a1b2c3"
down_revision = "c7d8e9f0a1b2"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("campaign_work_assignments"):
op.create_table(
"campaign_work_assignments",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("campaign_id", sa.String(length=36), nullable=False),
sa.Column("campaign_version_id", sa.String(length=36), nullable=True),
sa.Column("reference_kind", sa.String(length=40), nullable=True),
sa.Column("reference_id", sa.String(length=500), nullable=True),
sa.Column("reference_label", sa.String(length=255), nullable=True),
sa.Column("purpose", sa.String(length=500), nullable=False),
sa.Column("status", sa.String(length=30), nullable=False, server_default="open"),
sa.Column("due_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("assignee_type", sa.String(length=40), nullable=False),
sa.Column("assignee_id", sa.String(length=255), nullable=False),
sa.Column("assignee_label_snapshot", sa.String(length=500), nullable=False),
sa.Column("assignee_current_label", sa.String(length=500), nullable=True),
sa.Column("assignee_resolution_state", sa.String(length=30), nullable=False, server_default="resolved"),
sa.Column("resolution_provenance", sa.JSON(), nullable=False, server_default="{}"),
sa.Column("resolution_checked_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("assigned_by_user_id", sa.String(length=36), nullable=True),
sa.Column("assigned_by_label_snapshot", sa.String(length=255), nullable=False),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("cancelled_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("task_mirror_id", sa.String(length=255), nullable=True),
sa.Column("task_mirror_status", sa.String(length=30), nullable=False, server_default="not_configured"),
sa.Column("task_mirror_error", sa.String(length=500), nullable=True),
sa.Column("task_mirrored_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("resource_revision", sa.Integer(), nullable=False, server_default="1"),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["assigned_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["campaign_id"], ["campaigns.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["campaign_version_id"], ["campaign_versions.id"], ondelete="RESTRICT"),
sa.PrimaryKeyConstraint("id"),
)
for name, columns in (
("ix_campaign_work_assignments_tenant_id", ["tenant_id"]),
("ix_campaign_work_assignments_campaign_id", ["campaign_id"]),
("ix_campaign_work_assignments_campaign_version_id", ["campaign_version_id"]),
("ix_campaign_work_assignments_reference_kind", ["reference_kind"]),
("ix_campaign_work_assignments_status", ["status"]),
("ix_campaign_work_assignments_due_at", ["due_at"]),
("ix_campaign_work_assignments_assignee_type", ["assignee_type"]),
("ix_campaign_work_assignments_assignee_id", ["assignee_id"]),
("ix_campaign_work_assignments_assignee_resolution_state", ["assignee_resolution_state"]),
("ix_campaign_work_assignments_assigned_by_user_id", ["assigned_by_user_id"]),
("ix_campaign_work_assignments_campaign_status", ["tenant_id", "campaign_id", "status", "due_at", "id"]),
):
op.create_index(name, "campaign_work_assignments", columns, unique=False)
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("campaign_work_assignment_events"):
op.create_table(
"campaign_work_assignment_events",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("campaign_id", sa.String(length=36), nullable=False),
sa.Column("assignment_id", sa.String(length=36), nullable=False),
sa.Column("event_kind", sa.String(length=40), nullable=False),
sa.Column("actor_user_id", sa.String(length=36), nullable=True),
sa.Column("actor_label_snapshot", sa.String(length=255), nullable=False),
sa.Column("status_snapshot", sa.String(length=30), nullable=False),
sa.Column("assignee_type_snapshot", sa.String(length=40), nullable=False),
sa.Column("assignee_id_snapshot", sa.String(length=255), nullable=False),
sa.Column("assignee_label_snapshot", sa.String(length=500), nullable=False),
sa.Column("resolution_state_snapshot", sa.String(length=30), nullable=False),
sa.Column("details", sa.JSON(), nullable=False, server_default="{}"),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["actor_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["assignment_id"], ["campaign_work_assignments.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["campaign_id"], ["campaigns.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
for name, columns in (
("ix_campaign_work_assignment_events_tenant_id", ["tenant_id"]),
("ix_campaign_work_assignment_events_campaign_id", ["campaign_id"]),
("ix_campaign_work_assignment_events_assignment_id", ["assignment_id"]),
("ix_campaign_work_assignment_events_event_kind", ["event_kind"]),
("ix_campaign_work_assignment_events_actor_user_id", ["actor_user_id"]),
("ix_campaign_work_assignment_events_history", ["tenant_id", "assignment_id", "created_at", "id"]),
):
op.create_index(name, "campaign_work_assignment_events", columns, unique=False)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_work_assignment_events"):
op.drop_table("campaign_work_assignment_events")
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_work_assignments"):
op.drop_table("campaign_work_assignments")
@@ -0,0 +1,85 @@
"""add non-destructive historical campaign version archival
Revision ID: e3c8f4a5b6d7
Revises: b7c8d9e0f1a2, d2b7af503c81
Create Date: 2026-08-03 12:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "e3c8f4a5b6d7"
down_revision = ("b7c8d9e0f1a2", "d2b7af503c81")
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
columns = {
column["name"]
for column in inspector.get_columns("campaign_versions")
}
with op.batch_alter_table("campaign_versions") as batch_op:
if "archived_at" not in columns:
batch_op.add_column(
sa.Column("archived_at", sa.DateTime(timezone=True), nullable=True)
)
if "archived_by_user_id" not in columns:
batch_op.add_column(
sa.Column("archived_by_user_id", sa.String(length=36), nullable=True)
)
batch_op.create_foreign_key(
"fk_campaign_versions_archived_by_user_id_access_users",
"access_users",
["archived_by_user_id"],
["id"],
ondelete="SET NULL",
)
indexes = {
index["name"]
for index in sa.inspect(op.get_bind()).get_indexes("campaign_versions")
}
if "ix_campaign_versions_archived_at" not in indexes:
op.create_index(
"ix_campaign_versions_archived_at",
"campaign_versions",
["archived_at"],
unique=False,
)
if "ix_campaign_versions_archived_by_user_id" not in indexes:
op.create_index(
"ix_campaign_versions_archived_by_user_id",
"campaign_versions",
["archived_by_user_id"],
unique=False,
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
indexes = {
index["name"]
for index in inspector.get_indexes("campaign_versions")
}
for index_name in (
"ix_campaign_versions_archived_by_user_id",
"ix_campaign_versions_archived_at",
):
if index_name in indexes:
op.drop_index(index_name, table_name="campaign_versions")
columns = {
column["name"]
for column in sa.inspect(op.get_bind()).get_columns("campaign_versions")
}
with op.batch_alter_table("campaign_versions") as batch_op:
if "archived_by_user_id" in columns:
batch_op.drop_constraint(
"fk_campaign_versions_archived_by_user_id_access_users",
type_="foreignkey",
)
batch_op.drop_column("archived_by_user_id")
if "archived_at" in columns:
batch_op.drop_column("archived_at")
@@ -0,0 +1,32 @@
"""repair a missing IMAP append attempt claim token
Revision ID: e9f0a1b2c3d4
Revises: d8b3e2c1f4a5
Create Date: 2026-07-28 23:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "e9f0a1b2c3d4"
down_revision = "d8b3e2c1f4a5"
branch_labels = None
depends_on = None
def upgrade() -> None:
bind = op.get_bind()
columns = {column["name"] for column in sa.inspect(bind).get_columns("imap_append_attempts")}
if "claim_token" not in columns:
op.add_column(
"imap_append_attempts",
sa.Column("claim_token", sa.String(length=36), nullable=True),
)
def downgrade() -> None:
# The column belongs to revision 3c4d5e6f8192. This repair revision only
# restores drift, so downgrading to d8b3e2c1f4a5 must retain it.
pass
@@ -0,0 +1,153 @@
"""add governed campaign Postbox delivery
Revision ID: f0a1b2c3d4e5
Revises: e9f0a1b2c3d4
Create Date: 2026-07-29 02:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "f0a1b2c3d4e5"
down_revision = "e9f0a1b2c3d4"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"campaign_jobs",
sa.Column(
"delivery_channel_policy",
sa.String(length=30),
nullable=False,
server_default="mail",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"postbox_status",
sa.String(length=50),
nullable=False,
server_default="not_requested",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"postbox_attempt_count",
sa.Integer(),
nullable=False,
server_default="0",
),
)
op.add_column(
"campaign_jobs",
sa.Column(
"resolved_postbox_targets",
sa.JSON(),
nullable=False,
server_default="[]",
),
)
op.create_index(
op.f("ix_campaign_jobs_delivery_channel_policy"),
"campaign_jobs",
["delivery_channel_policy"],
unique=False,
)
op.create_index(
op.f("ix_campaign_jobs_postbox_status"),
"campaign_jobs",
["postbox_status"],
unique=False,
)
op.create_table(
"campaign_postbox_delivery_attempts",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("job_id", sa.String(length=36), nullable=False),
sa.Column("target_key", sa.String(length=64), nullable=False),
sa.Column("target_index", sa.Integer(), nullable=False),
sa.Column("attempt_number", sa.Integer(), nullable=False),
sa.Column("idempotency_key", sa.String(length=255), nullable=False),
sa.Column("status", sa.String(length=50), nullable=False),
sa.Column("target_snapshot", sa.JSON(), nullable=False),
sa.Column("provider_delivery_id", sa.String(length=36), nullable=True),
sa.Column("provider_message_id", sa.String(length=36), nullable=True),
sa.Column("postbox_id", sa.String(length=36), nullable=True),
sa.Column("address", sa.String(length=500), nullable=True),
sa.Column("holder_count", sa.Integer(), nullable=True),
sa.Column("vacant", sa.Boolean(), nullable=True),
sa.Column(
"duplicate",
sa.Boolean(),
nullable=False,
server_default=sa.false(),
),
sa.Column("evidence", sa.JSON(), nullable=False),
sa.Column("error_type", sa.String(length=255), nullable=True),
sa.Column("error_code", sa.String(length=100), nullable=True),
sa.Column("error_message", sa.Text(), nullable=True),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["job_id"],
["campaign_jobs.id"],
name=op.f(
"fk_campaign_postbox_delivery_attempts_job_id_campaign_jobs"
),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint(
"id",
name=op.f("pk_campaign_postbox_delivery_attempts"),
),
sa.UniqueConstraint(
"job_id",
"target_key",
"attempt_number",
name="uq_campaign_postbox_attempt_target_number",
),
)
for column in ("tenant_id", "job_id", "status", "postbox_id"):
op.create_index(
op.f(f"ix_campaign_postbox_delivery_attempts_{column}"),
"campaign_postbox_delivery_attempts",
[column],
unique=False,
)
op.create_index(
"ix_campaign_postbox_attempt_job_status",
"campaign_postbox_delivery_attempts",
["job_id", "status"],
unique=False,
)
op.create_index(
"ix_campaign_postbox_attempt_idempotency",
"campaign_postbox_delivery_attempts",
["tenant_id", "idempotency_key"],
unique=False,
)
def downgrade() -> None:
op.drop_table("campaign_postbox_delivery_attempts")
op.drop_index(
op.f("ix_campaign_jobs_postbox_status"),
table_name="campaign_jobs",
)
op.drop_index(
op.f("ix_campaign_jobs_delivery_channel_policy"),
table_name="campaign_jobs",
)
op.drop_column("campaign_jobs", "resolved_postbox_targets")
op.drop_column("campaign_jobs", "postbox_attempt_count")
op.drop_column("campaign_jobs", "postbox_status")
op.drop_column("campaign_jobs", "delivery_channel_policy")
@@ -0,0 +1,103 @@
"""add durable Campaign work orchestration provenance
revision = "f3c7a9d2e6b1"
down_revision = "d8e9f0a1b2c3"
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "f3c7a9d2e6b1"
down_revision = "d8e9f0a1b2c3"
branch_labels = None
depends_on = None
_COLUMN_SPECS = (
("orchestration_idempotency_key", sa.String(length=255)),
("orchestration_request_sha256", sa.String(length=64)),
("orchestration_correlation_id", sa.String(length=128)),
("workflow_instance_id", sa.String(length=36)),
("workflow_step_id", sa.String(length=36)),
)
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("campaign_work_assignments"):
return
existing = {
item["name"]
for item in inspector.get_columns("campaign_work_assignments")
}
with op.batch_alter_table("campaign_work_assignments") as batch:
for name, column_type in _COLUMN_SPECS:
if name not in existing:
batch.add_column(sa.Column(name, column_type, nullable=True))
inspector = sa.inspect(op.get_bind())
indexes = {
item["name"]
for item in inspector.get_indexes("campaign_work_assignments")
}
for name, columns in (
(
"ix_campaign_work_assignments_orchestration_idempotency_key",
["orchestration_idempotency_key"],
),
(
"ix_campaign_work_assignments_orchestration_correlation_id",
["orchestration_correlation_id"],
),
(
"ix_campaign_work_assignments_workflow_instance_id",
["workflow_instance_id"],
),
(
"ix_campaign_work_assignments_workflow_step_id",
["workflow_step_id"],
),
(
"uq_campaign_work_assignment_orchestration_key",
["tenant_id", "orchestration_idempotency_key"],
),
):
if name not in indexes:
op.create_index(
name,
"campaign_work_assignments",
columns,
unique=name.startswith("uq_"),
)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("campaign_work_assignments"):
return
indexes = {
item["name"]
for item in inspector.get_indexes("campaign_work_assignments")
}
for name in (
"uq_campaign_work_assignment_orchestration_key",
"ix_campaign_work_assignments_workflow_step_id",
"ix_campaign_work_assignments_workflow_instance_id",
"ix_campaign_work_assignments_orchestration_correlation_id",
"ix_campaign_work_assignments_orchestration_idempotency_key",
):
if name in indexes:
op.drop_index(name, table_name="campaign_work_assignments")
existing = {
item["name"]
for item in sa.inspect(op.get_bind()).get_columns(
"campaign_work_assignments"
)
}
with op.batch_alter_table("campaign_work_assignments") as batch:
for name, _column_type in reversed(_COLUMN_SPECS):
if name in existing:
batch.drop_column(name)
@@ -0,0 +1,106 @@
"""add durable campaign schedules and occurrence evidence
Revision ID: f4a5b6c7d8e9
Revises: e3c8f4a5b6d7
Create Date: 2026-08-07 10:00:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "f4a5b6c7d8e9"
down_revision = "e3c8f4a5b6d7"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("campaign_schedules"):
op.create_table(
"campaign_schedules",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("campaign_id", sa.String(length=36), nullable=False),
sa.Column("source_version_id", sa.String(length=36), nullable=False),
sa.Column("created_by_user_id", sa.String(length=36), nullable=True),
sa.Column("name", sa.String(length=255), nullable=False),
sa.Column("recurrence_kind", sa.String(length=20), nullable=False),
sa.Column("interval_count", sa.Integer(), nullable=False),
sa.Column("timezone", sa.String(length=100), nullable=False),
sa.Column("starts_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("next_fire_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("ends_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("max_occurrences", sa.Integer(), nullable=False),
sa.Column("occurrence_count", sa.Integer(), nullable=False),
sa.Column("active", sa.Boolean(), nullable=False),
sa.Column("resource_revision", sa.Integer(), nullable=False),
sa.Column("copy_options", sa.JSON(), nullable=False),
sa.Column("source_snapshot", sa.JSON(), nullable=False),
sa.Column("source_snapshot_hash", sa.String(length=64), nullable=False),
sa.Column("source_base_path", sa.String(length=1000), nullable=True),
sa.Column("last_fired_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_campaign_id", sa.String(length=36), nullable=True),
sa.Column("last_error", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["campaign_id"], ["campaigns.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["source_version_id"], ["campaign_versions.id"], ondelete="RESTRICT"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["access_users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["last_campaign_id"], ["campaigns.id"], ondelete="SET NULL"),
sa.PrimaryKeyConstraint("id"),
)
for name, columns in (
("ix_campaign_schedules_tenant_id", ["tenant_id"]),
("ix_campaign_schedules_campaign_id", ["campaign_id"]),
("ix_campaign_schedules_source_version_id", ["source_version_id"]),
("ix_campaign_schedules_created_by_user_id", ["created_by_user_id"]),
("ix_campaign_schedules_recurrence_kind", ["recurrence_kind"]),
("ix_campaign_schedules_next_fire_at", ["next_fire_at"]),
("ix_campaign_schedules_active", ["active"]),
("ix_campaign_schedules_source_snapshot_hash", ["source_snapshot_hash"]),
("ix_campaign_schedules_last_campaign_id", ["last_campaign_id"]),
("ix_campaign_schedules_due", ["tenant_id", "active", "next_fire_at"]),
):
op.create_index(name, "campaign_schedules", columns, unique=False)
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("campaign_schedule_occurrences"):
op.create_table(
"campaign_schedule_occurrences",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("tenant_id", sa.String(length=36), nullable=False),
sa.Column("schedule_id", sa.String(length=36), nullable=False),
sa.Column("scheduled_for", sa.DateTime(timezone=True), nullable=False),
sa.Column("status", sa.String(length=30), nullable=False),
sa.Column("generated_campaign_id", sa.String(length=36), nullable=True),
sa.Column("generated_version_id", sa.String(length=36), nullable=True),
sa.Column("error", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["schedule_id"], ["campaign_schedules.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["generated_campaign_id"], ["campaigns.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["generated_version_id"], ["campaign_versions.id"], ondelete="SET NULL"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("schedule_id", "scheduled_for", name="uq_campaign_schedule_occurrence"),
)
for name, columns in (
("ix_campaign_schedule_occurrences_tenant_id", ["tenant_id"]),
("ix_campaign_schedule_occurrences_schedule_id", ["schedule_id"]),
("ix_campaign_schedule_occurrences_status", ["status"]),
("ix_campaign_schedule_occurrences_generated_campaign_id", ["generated_campaign_id"]),
("ix_campaign_schedule_occurrences_generated_version_id", ["generated_version_id"]),
("ix_campaign_schedule_occurrences_schedule", ["schedule_id", "scheduled_for"]),
):
op.create_index(name, "campaign_schedule_occurrences", columns, unique=False)
def downgrade() -> None:
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_schedule_occurrences"):
op.drop_table("campaign_schedule_occurrences")
inspector = sa.inspect(op.get_bind())
if inspector.has_table("campaign_schedules"):
op.drop_table("campaign_schedules")
@@ -0,0 +1,56 @@
from __future__ import annotations
import secrets
from govoplan_core.core.object_storage import (
StorageBackendError,
configured_storage_backend,
)
from govoplan_core.core.operations import OperationalCheck
from govoplan_core.settings import settings as core_settings
from govoplan_campaign.backend.runtime import get_settings
def generated_eml_storage_check() -> OperationalCheck:
"""Verify Campaign evidence against the deployment object-store boundary."""
storage = configured_storage_backend(get_settings() or core_settings)
probe = f"campaign-artifacts/.health/{secrets.token_hex(16)}.probe"
payload = secrets.token_bytes(64)
try:
storage.put_bytes(probe, payload, content_type="application/octet-stream")
if storage.get_bytes(probe) != payload:
raise OSError("generated EML persistence returned different bytes")
except (OSError, StorageBackendError) as exc:
return OperationalCheck(
id="campaign.generated_eml_storage",
label="Generated Campaign EML evidence",
state="error",
detail=(
"The generated EML object store failed a bounded write/read probe "
f"({type(exc).__name__})."
),
readiness_critical=True,
metrics={"backend": storage.name},
)
finally:
try:
storage.delete(probe)
except StorageBackendError:
pass
node_local = storage.name == "local"
return OperationalCheck(
id="campaign.generated_eml_storage",
label="Generated Campaign EML evidence",
state="warning" if node_local else "ok",
detail=(
"Generated EML passed the object-store write/read/delete probe. "
+ (
"The configured backend is node-local and is suitable only for a single-node profile."
if node_local
else "The configured backend is shared across application and worker nodes."
)
),
metrics={"backend": storage.name},
)
@@ -0,0 +1,170 @@
from __future__ import annotations
from pathlib import PureWindowsPath
from typing import Any, Iterable
class CampaignPathSecurityError(ValueError):
"""Raised when server-provided campaign JSON could read local files."""
def _text(value: object) -> str:
return value.strip() if isinstance(value, str) else ""
def assert_logical_relative_path(value: object, *, field: str) -> None:
raw = _text(value)
if not raw:
return
if "\x00" in raw:
raise CampaignPathSecurityError(f"{field} contains a NUL byte")
normalized = raw.replace("\\", "/")
windows_path = PureWindowsPath(raw)
if (
normalized.startswith("/")
or normalized.startswith("~")
or windows_path.is_absolute()
or bool(windows_path.drive)
):
raise CampaignPathSecurityError(f"{field} must be a relative managed-file path")
if any(part == ".." for part in normalized.split("/")):
raise CampaignPathSecurityError(f"{field} must not contain parent-directory traversal")
def is_managed_source(value: object) -> bool:
raw = _text(value)
if not raw.startswith("managed:"):
return False
parts = raw.split(":", 2)
return len(parts) == 3 and parts[1] in {"user", "group"} and bool(parts[2].strip())
def _attachment_rules(raw_json: dict[str, Any]) -> Iterable[tuple[str, dict[str, Any]]]:
attachments = raw_json.get("attachments")
if isinstance(attachments, dict):
global_rules = attachments.get("global")
if isinstance(global_rules, list):
for index, rule in enumerate(global_rules):
if isinstance(rule, dict):
yield f"attachments.global[{index}]", rule
entries = raw_json.get("entries")
inline = entries.get("inline") if isinstance(entries, dict) else None
if isinstance(inline, list):
for entry_index, entry in enumerate(inline):
rules = entry.get("attachments") if isinstance(entry, dict) else None
if not isinstance(rules, list):
continue
for rule_index, rule in enumerate(rules):
if isinstance(rule, dict):
yield f"entries.inline[{entry_index}].attachments[{rule_index}]", rule
defaults = entries.get("defaults") if isinstance(entries, dict) else None
default_rules = defaults.get("attachments") if isinstance(defaults, dict) else None
if isinstance(default_rules, list):
for rule_index, rule in enumerate(default_rules):
if isinstance(rule, dict):
yield f"entries.defaults.attachments[{rule_index}]", rule
def _assert_inline_server_sources(
raw_json: dict[str, Any],
*,
source_filename: str | None,
source_base_path: str | None,
) -> None:
if _text(source_filename):
raise CampaignPathSecurityError("source_filename is not accepted for server-managed campaigns")
if _text(source_base_path):
raise CampaignPathSecurityError("source_base_path is not accepted for server-managed campaigns")
template = raw_json.get("template")
template_source = template.get("source") if isinstance(template, dict) else None
path_fields = ("subject_path", "text_path", "html_path")
if isinstance(template_source, dict) and any(_text(template_source.get(key)) for key in path_fields):
raise CampaignPathSecurityError(
"template source paths are not accepted for server-managed campaigns; store template content inline"
)
entries = raw_json.get("entries")
entries_source = entries.get("source") if isinstance(entries, dict) else None
if isinstance(entries_source, dict) and _text(entries_source.get("path")):
raise CampaignPathSecurityError(
"entries.source.path is not accepted for server-managed campaigns; import recipients into the campaign snapshot"
)
def _managed_attachment_base_paths(attachments: dict[str, Any]) -> dict[str, dict[str, Any]]:
assert_logical_relative_path(attachments.get("base_path"), field="attachments.base_path")
raw_base_paths = attachments.get("base_paths")
base_paths = raw_base_paths if isinstance(raw_base_paths, list) else []
managed: dict[str, dict[str, Any]] = {}
for index, item in enumerate(base_paths):
if not isinstance(item, dict):
continue
assert_logical_relative_path(
item.get("path"),
field=f"attachments.base_paths[{index}].path",
)
source_is_managed = is_managed_source(item.get("source"))
base_path_id = _text(item.get("id"))
if base_path_id and source_is_managed:
managed[base_path_id] = item
if item.get("unsent_warning") is True and not source_is_managed:
raise CampaignPathSecurityError(
f"attachments.base_paths[{index}] must use a managed Files source before unsent-file scanning is enabled"
)
return managed
def _assert_managed_attachment_rules(
raw_json: dict[str, Any],
*,
managed_base_paths: dict[str, dict[str, Any]],
managed_files_available: bool,
) -> None:
rules = list(_attachment_rules(raw_json))
if not rules:
return
if not managed_files_available:
raise CampaignPathSecurityError("campaign attachments require the Files module in server mode")
for field, rule in rules:
base_path_id = _text(rule.get("base_path_id"))
if not base_path_id or base_path_id not in managed_base_paths:
raise CampaignPathSecurityError(
f"{field}.base_path_id must select an attachments.base_paths entry backed by managed Files"
)
assert_logical_relative_path(rule.get("base_dir"), field=f"{field}.base_dir")
assert_logical_relative_path(rule.get("file_filter"), field=f"{field}.file_filter")
def assert_server_safe_campaign_paths(
raw_json: dict[str, Any],
*,
source_filename: str | None = None,
source_base_path: str | None = None,
managed_files_available: bool,
) -> None:
"""Reject local-file references at the HTTP/server trust boundary.
File-oriented campaign loading remains available to trusted operator code
through the campaign loader and message builder. Persisted/API campaigns
must use inline templates and recipients plus managed Files attachment
sources, which are materialized into an isolated snapshot before use.
"""
_assert_inline_server_sources(
raw_json,
source_filename=source_filename,
source_base_path=source_base_path,
)
attachments = raw_json.get("attachments")
if not isinstance(attachments, dict):
return
_assert_managed_attachment_rules(
raw_json,
managed_base_paths=_managed_attachment_base_paths(attachments),
managed_files_available=managed_files_available,
)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,109 @@
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Mapping
from sqlalchemy.orm import Session
from govoplan_core.tenancy.scope import Tenant
DEFAULT_SMALL_CELL_THRESHOLD = 5
MIN_SMALL_CELL_THRESHOLD = 2
MAX_SMALL_CELL_THRESHOLD = 100
SMALL_CELL_THRESHOLD_ENV = "GOVOPLAN_CAMPAIGN_REPORT_SMALL_CELL_THRESHOLD"
CAMPAIGN_REPORT_POLICY_SETTINGS_KEY = "campaign_report_privacy_policy"
SMALL_CELL_THRESHOLD_SETTINGS_KEY = "small_cell_threshold"
class CampaignReportPrivacyPolicyError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class CampaignReportPrivacyPolicy:
small_cell_threshold: int
source: str
deployment_small_cell_threshold: int
tenant_small_cell_threshold: int | None = None
def as_dict(self) -> dict[str, Any]:
return {
"small_cell_threshold": self.small_cell_threshold,
"source": self.source,
"deployment_small_cell_threshold": self.deployment_small_cell_threshold,
"tenant_small_cell_threshold": self.tenant_small_cell_threshold,
"deployment_setting": SMALL_CELL_THRESHOLD_ENV,
"tenant_setting": (
f"tenant.settings.{CAMPAIGN_REPORT_POLICY_SETTINGS_KEY}."
f"{SMALL_CELL_THRESHOLD_SETTINGS_KEY}"
),
}
def effective_campaign_report_privacy_policy(
session: Session,
*,
tenant_id: str,
environ: Mapping[str, str] | None = None,
) -> CampaignReportPrivacyPolicy:
env = os.environ if environ is None else environ
deployment_raw = env.get(SMALL_CELL_THRESHOLD_ENV)
deployment_value = _configured_threshold(
deployment_raw,
source=SMALL_CELL_THRESHOLD_ENV,
default=DEFAULT_SMALL_CELL_THRESHOLD,
)
tenant = session.get(Tenant, tenant_id)
tenant_raw = _tenant_threshold_value(tenant.settings if tenant is not None else None)
if tenant_raw is None:
return CampaignReportPrivacyPolicy(
small_cell_threshold=deployment_value,
source="deployment" if deployment_raw not in (None, "") else "deployment_default",
deployment_small_cell_threshold=deployment_value,
)
tenant_value = _configured_threshold(
tenant_raw,
source=(
f"tenant.settings.{CAMPAIGN_REPORT_POLICY_SETTINGS_KEY}."
f"{SMALL_CELL_THRESHOLD_SETTINGS_KEY}"
),
)
effective_value = max(deployment_value, tenant_value)
return CampaignReportPrivacyPolicy(
small_cell_threshold=effective_value,
source="tenant" if tenant_value >= deployment_value else "deployment_floor",
deployment_small_cell_threshold=deployment_value,
tenant_small_cell_threshold=tenant_value,
)
def _tenant_threshold_value(settings: Mapping[str, Any] | None) -> object | None:
if not isinstance(settings, Mapping):
return None
policy = settings.get(CAMPAIGN_REPORT_POLICY_SETTINGS_KEY)
if not isinstance(policy, Mapping):
return None
return policy.get(SMALL_CELL_THRESHOLD_SETTINGS_KEY)
def _configured_threshold(value: object, *, source: str, default: int | None = None) -> int:
if value is None or (isinstance(value, str) and not value.strip()):
if default is not None:
return default
raise CampaignReportPrivacyPolicyError(f"{source} must be configured as an integer")
if isinstance(value, bool):
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer, not a boolean")
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer") from exc
if str(parsed) != str(value).strip() and not isinstance(value, int):
raise CampaignReportPrivacyPolicyError(f"{source} must be an integer")
if parsed < MIN_SMALL_CELL_THRESHOLD or parsed > MAX_SMALL_CELL_THRESHOLD:
raise CampaignReportPrivacyPolicyError(
f"{source} must be between {MIN_SMALL_CELL_THRESHOLD} and {MAX_SMALL_CELL_THRESHOLD}"
)
return parsed
@@ -0,0 +1,556 @@
from __future__ import annotations
from collections import Counter
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Literal
from pydantic import BaseModel, ConfigDict
from sqlalchemy import case, func, or_
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import Campaign, CampaignJob, CampaignVersion
from govoplan_campaign.backend.report_privacy_policy import (
CampaignReportPrivacyPolicy,
effective_campaign_report_privacy_policy,
)
class AggregateCampaignReportError(RuntimeError):
pass
class AggregateReportCampaign(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str
name: str
status: str
class AggregateReportCampaignListItem(AggregateReportCampaign):
updated_at: datetime
class AggregateReportCampaignList(BaseModel):
model_config = ConfigDict(extra="forbid")
campaigns: list[AggregateReportCampaignListItem]
class AggregateCount(BaseModel):
model_config = ConfigDict(extra="forbid")
value: int | None
suppressed: bool = False
class AggregatePopulation(BaseModel):
model_config = ConfigDict(extra="forbid")
denominator: AggregateCount
denominator_definition: str
inactive_source_entries: AggregateCount
excluded_or_blocked_jobs: AggregateCount
class AggregateOutcomeCounts(BaseModel):
model_config = ConfigDict(extra="forbid")
smtp_accepted: AggregateCount
postbox_accepted: AggregateCount
print_accepted: AggregateCount
delivered: AggregateCount
partially_accepted: AggregateCount
failed: AggregateCount
outcome_unknown: AggregateCount
queued_or_active: AggregateCount
cancelled: AggregateCount
excluded: AggregateCount
not_attempted: AggregateCount
class AggregateTimeRange(BaseModel):
model_config = ConfigDict(extra="forbid")
first_activity_at: datetime | None
last_activity_at: datetime | None
suppressed: bool
class AggregatePrivacy(BaseModel):
model_config = ConfigDict(extra="forbid")
small_cell_threshold: int
suppression_applied: bool
rule: str
class AggregateCampaignReport(BaseModel):
model_config = ConfigDict(extra="forbid")
generated_at: datetime
campaign: AggregateReportCampaign
version_number: int | None
completion_state: Literal[
"not_started",
"in_progress",
"completed",
"partially_completed",
"incomplete",
"outcome_unknown",
"suppressed",
]
population: AggregatePopulation
outcomes: AggregateOutcomeCounts
time_range: AggregateTimeRange
privacy: AggregatePrivacy
_OUTCOME_KEYS = (
"smtp_accepted",
"postbox_accepted",
"print_accepted",
"delivered",
"partially_accepted",
"failed",
"outcome_unknown",
"queued_or_active",
"cancelled",
"excluded",
"not_attempted",
)
def generate_aggregate_campaign_report(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version_id: str | None = None,
) -> AggregateCampaignReport:
"""Build the deliberately small, recipient-free Campaign report projection."""
campaign = _get_campaign(session, tenant_id=tenant_id, campaign_id=campaign_id)
version = _selected_version(session, campaign, version_id)
facts = _query_aggregate_facts(
session,
tenant_id=tenant_id,
campaign_id=campaign.id,
version=version,
)
policy = effective_campaign_report_privacy_policy(session, tenant_id=tenant_id)
return _build_aggregate_campaign_report(
campaign=campaign,
version=version,
facts=facts,
policy=policy,
)
def _get_campaign(session: Session, *, tenant_id: str, campaign_id: str) -> Campaign:
campaign = (
session.query(Campaign)
.filter(Campaign.tenant_id == tenant_id, Campaign.id == campaign_id)
.one_or_none()
)
if campaign is None:
raise AggregateCampaignReportError("Campaign not found")
return campaign
def _selected_version(
session: Session,
campaign: Campaign,
version_id: str | None,
) -> CampaignVersion | None:
selected_id = version_id or campaign.current_version_id
if not selected_id:
return None
version = session.get(CampaignVersion, selected_id)
if version is None or version.campaign_id != campaign.id:
raise AggregateCampaignReportError("Campaign version not found")
return version
def _query_aggregate_facts(
session: Session,
*,
tenant_id: str,
campaign_id: str,
version: CampaignVersion | None,
) -> _AggregateFacts:
if version is None:
return _AggregateFacts.empty()
smtp_accepted = CampaignJob.send_status.in_(
{"smtp_accepted", "sent"}
)
accepted = CampaignJob.send_status.in_(
{
"smtp_accepted",
"postbox_accepted",
"print_accepted",
"delivered",
"partially_accepted",
"sent",
}
)
failed = CampaignJob.send_status.in_({"failed_temporary", "failed_permanent"})
unknown = CampaignJob.send_status == "outcome_unknown"
active = CampaignJob.send_status.in_({"queued", "claimed", "sending"})
cancelled = CampaignJob.send_status == "cancelled"
excluded = CampaignJob.send_status == "skipped"
excluded_or_blocked = or_(
CampaignJob.validation_status.in_({"blocked", "excluded", "inactive"}),
CampaignJob.build_status != "built",
)
row = (
session.query(
func.count(CampaignJob.id).label("denominator"),
func.sum(case((smtp_accepted, 1), else_=0)).label(
"smtp_accepted"
),
func.sum(
case(
(
CampaignJob.send_status == "postbox_accepted",
1,
),
else_=0,
)
).label("postbox_accepted"),
func.sum(
case(
(CampaignJob.send_status == "print_accepted", 1),
else_=0,
)
).label("print_accepted"),
func.sum(
case(
(CampaignJob.send_status == "delivered", 1),
else_=0,
)
).label("delivered"),
func.sum(
case(
(
CampaignJob.send_status == "partially_accepted",
1,
),
else_=0,
)
).label("partially_accepted"),
func.sum(case((failed, 1), else_=0)).label("failed"),
func.sum(case((unknown, 1), else_=0)).label("outcome_unknown"),
func.sum(case((active, 1), else_=0)).label("queued_or_active"),
func.sum(case((cancelled, 1), else_=0)).label("cancelled"),
func.sum(case((excluded, 1), else_=0)).label("excluded"),
func.sum(
case((or_(accepted, failed, unknown, active, cancelled, excluded), 0), else_=1)
).label("not_attempted"),
func.sum(case((excluded_or_blocked, 1), else_=0)).label("excluded_or_blocked"),
func.min(CampaignJob.queued_at).label("queued_min"),
func.max(CampaignJob.queued_at).label("queued_max"),
func.min(CampaignJob.smtp_started_at).label("smtp_min"),
func.max(CampaignJob.smtp_started_at).label("smtp_max"),
func.min(CampaignJob.sent_at).label("sent_min"),
func.max(CampaignJob.sent_at).label("sent_max"),
func.min(CampaignJob.outcome_unknown_at).label("unknown_min"),
func.max(CampaignJob.outcome_unknown_at).label("unknown_max"),
)
.filter(
CampaignJob.tenant_id == tenant_id,
CampaignJob.campaign_id == campaign_id,
CampaignJob.campaign_version_id == version.id,
)
.one()
)
values = row._mapping
activity = [
values[key]
for key in (
"queued_min",
"queued_max",
"smtp_min",
"smtp_max",
"sent_min",
"sent_max",
"unknown_min",
"unknown_max",
)
if values[key] is not None
]
return _AggregateFacts(
outcomes={key: int(values[key] or 0) for key in _OUTCOME_KEYS},
denominator=int(values["denominator"] or 0),
excluded_or_blocked=int(values["excluded_or_blocked"] or 0),
first_activity_at=min(activity) if activity else None,
last_activity_at=max(activity) if activity else None,
)
def build_aggregate_campaign_report(
*,
campaign: Campaign,
version: CampaignVersion | None,
jobs: list[CampaignJob],
policy: CampaignReportPrivacyPolicy,
generated_at: datetime | None = None,
) -> AggregateCampaignReport:
return _build_aggregate_campaign_report(
campaign=campaign,
version=version,
facts=_facts_from_jobs(jobs),
policy=policy,
generated_at=generated_at,
)
def _build_aggregate_campaign_report(
*,
campaign: Campaign,
version: CampaignVersion | None,
facts: _AggregateFacts,
policy: CampaignReportPrivacyPolicy,
generated_at: datetime | None = None,
) -> AggregateCampaignReport:
outcome_values = facts.outcomes
outcomes, denominator = _suppress_partition(
outcome_values,
threshold=policy.small_cell_threshold,
)
outcome_suppression_applied = denominator.suppressed or any(
item.suppressed for item in outcomes.values()
)
inactive_entries = _inactive_entry_count(version)
standalone_counts = {
"inactive_source_entries": _suppress_standalone_count(
inactive_entries,
threshold=policy.small_cell_threshold,
),
# This population count overlaps the outcome partition, so exposing it
# can make a suppressed outcome recoverable through subtraction.
"excluded_or_blocked_jobs": (
AggregateCount(value=None, suppressed=True)
if outcome_suppression_applied
else _suppress_standalone_count(
facts.excluded_or_blocked,
threshold=policy.small_cell_threshold,
)
),
}
suppression_applied = denominator.suppressed or any(
item.suppressed for item in (*outcomes.values(), *standalone_counts.values())
)
first_activity = facts.first_activity_at
last_activity = facts.last_activity_at
suppress_time_range = suppression_applied or (
0 < facts.denominator < policy.small_cell_threshold
)
return AggregateCampaignReport(
generated_at=generated_at or datetime.now(timezone.utc),
campaign=AggregateReportCampaign(
id=campaign.id,
name=campaign.name,
status=campaign.status,
),
version_number=version.version_number if version else None,
completion_state=(
"suppressed"
if denominator.suppressed
else _completion_state(outcome_values, facts.denominator)
),
population=AggregatePopulation(
denominator=denominator,
denominator_definition=(
"All persisted recipient delivery jobs for the selected campaign version, "
"including excluded or blocked jobs. Inactive source entries without a job "
"record are excluded and reported separately."
),
inactive_source_entries=standalone_counts["inactive_source_entries"],
excluded_or_blocked_jobs=standalone_counts["excluded_or_blocked_jobs"],
),
outcomes=AggregateOutcomeCounts(**outcomes),
time_range=AggregateTimeRange(
first_activity_at=None if suppress_time_range else first_activity,
last_activity_at=None if suppress_time_range else last_activity,
suppressed=suppress_time_range and first_activity is not None,
),
privacy=AggregatePrivacy(
small_cell_threshold=policy.small_cell_threshold,
suppression_applied=suppression_applied,
rule=(
"Positive counts below the threshold are hidden. At least one additional "
"count or the denominator is hidden when needed to prevent subtraction. "
"Overlapping population counts are hidden whenever outcome suppression applies."
),
),
)
def aggregate_report_campaign_item(campaign: Campaign) -> AggregateReportCampaignListItem:
return AggregateReportCampaignListItem(
id=campaign.id,
name=campaign.name,
status=campaign.status,
updated_at=campaign.updated_at,
)
def _outcome_counts(jobs: list[CampaignJob]) -> dict[str, int]:
counts: Counter[str] = Counter()
for job in jobs:
status = job.send_status
if status in {"smtp_accepted", "sent"}:
counts["smtp_accepted"] += 1
elif status == "postbox_accepted":
counts["postbox_accepted"] += 1
elif status == "print_accepted":
counts["print_accepted"] += 1
elif status == "delivered":
counts["delivered"] += 1
elif status == "partially_accepted":
counts["partially_accepted"] += 1
elif status in {"failed_temporary", "failed_permanent"}:
counts["failed"] += 1
elif status == "outcome_unknown":
counts["outcome_unknown"] += 1
elif status in {"queued", "claimed", "sending"}:
counts["queued_or_active"] += 1
elif status == "cancelled":
counts["cancelled"] += 1
elif status == "skipped":
counts["excluded"] += 1
else:
counts["not_attempted"] += 1
return {key: counts[key] for key in _OUTCOME_KEYS}
@dataclass(frozen=True, slots=True)
class _AggregateFacts:
outcomes: dict[str, int]
denominator: int
excluded_or_blocked: int
first_activity_at: datetime | None
last_activity_at: datetime | None
@classmethod
def empty(cls) -> _AggregateFacts:
return cls(
outcomes={key: 0 for key in _OUTCOME_KEYS},
denominator=0,
excluded_or_blocked=0,
first_activity_at=None,
last_activity_at=None,
)
def _facts_from_jobs(jobs: list[CampaignJob]) -> _AggregateFacts:
first_activity, last_activity = _activity_range(jobs)
return _AggregateFacts(
outcomes=_outcome_counts(jobs),
denominator=len(jobs),
excluded_or_blocked=sum(
1
for job in jobs
if job.validation_status in {"blocked", "excluded", "inactive"}
or job.build_status != "built"
),
first_activity_at=first_activity,
last_activity_at=last_activity,
)
def _suppress_partition(
counts: dict[str, int],
*,
threshold: int,
) -> tuple[dict[str, AggregateCount], AggregateCount]:
total = sum(counts.values())
suppressed = {key for key, value in counts.items() if 0 < value < threshold}
suppress_denominator = False
if suppressed:
companions = [
(value, key)
for key, value in counts.items()
if key not in suppressed and value > 0
]
if companions:
suppressed.add(max(companions)[1])
else:
suppress_denominator = True
denominator = AggregateCount(
value=None if suppress_denominator else total,
suppressed=suppress_denominator,
)
return (
{
key: AggregateCount(
value=None if key in suppressed else value,
suppressed=key in suppressed,
)
for key, value in counts.items()
},
denominator,
)
def _suppress_standalone_count(value: int, *, threshold: int) -> AggregateCount:
if 0 < value < threshold:
return AggregateCount(value=None, suppressed=True)
return AggregateCount(value=value, suppressed=False)
def _completion_state(
counts: dict[str, int],
total: int,
) -> Literal[
"not_started",
"in_progress",
"completed",
"partially_completed",
"incomplete",
"outcome_unknown",
]:
if total == 0 or counts["not_attempted"] + counts["excluded"] == total:
return "not_started"
if counts["outcome_unknown"]:
return "outcome_unknown"
if counts["queued_or_active"]:
return "in_progress"
fully_accepted = (
counts["smtp_accepted"]
+ counts["postbox_accepted"]
+ counts["print_accepted"]
+ counts["delivered"]
)
partially_accepted = counts["partially_accepted"]
if fully_accepted == total:
return "completed"
if fully_accepted or partially_accepted:
return "partially_completed"
return "incomplete"
def _activity_range(jobs: list[CampaignJob]) -> tuple[datetime | None, datetime | None]:
activity = [
value
for job in jobs
for value in (
job.queued_at,
job.smtp_started_at,
job.sent_at,
job.outcome_unknown_at,
)
if value is not None
]
if not activity:
return None, None
return min(activity), max(activity)
def _inactive_entry_count(version: CampaignVersion | None) -> int:
build_summary = version.build_summary if version and isinstance(version.build_summary, dict) else {}
return int(build_summary.get("inactive_count") or build_summary.get("inactive_entries_count") or 0)
File diff suppressed because it is too large Load Diff
+105 -43
View File
@@ -9,11 +9,16 @@ from typing import Any
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import Campaign, CampaignVersion
from govoplan_campaign.backend.campaign.loader import load_campaign_config
from govoplan_campaign.backend.campaign.models import CampaignConfig, SmtpConfig
from govoplan_campaign.backend.persistence.campaigns import _write_campaign_snapshot
from govoplan_campaign.backend.campaign.models import CampaignConfig
from govoplan_campaign.backend.campaign.mail_profile_boundary import campaign_mail_profile_id
from govoplan_campaign.backend.persistence.campaigns import load_version_config
from govoplan_campaign.backend.reports.campaigns import CampaignReportError, generate_campaign_report, generate_jobs_csv
from govoplan_campaign.backend.integrations import SmtpConfigurationError, mail_integration
from govoplan_campaign.backend.integrations import (
MailDeliveryCommandError,
SmtpConfigurationError,
mail_integration,
)
from govoplan_campaign.backend.sending.execution import ExecutionSnapshotError, ensure_execution_snapshot
class CampaignReportEmailError(RuntimeError):
@@ -30,9 +35,10 @@ class CampaignReportEmailResult:
sent: bool
attached_jobs_csv: bool
attached_report_json: bool
smtp_host: str | None = None
smtp_port: int | None = None
accepted_count: int | None = None
command_id: str | None = None
delivery_status: str | None = None
duplicate: bool = False
def as_dict(self) -> dict[str, Any]:
return {
@@ -44,9 +50,23 @@ class CampaignReportEmailResult:
"sent": self.sent,
"attached_jobs_csv": self.attached_jobs_csv,
"attached_report_json": self.attached_report_json,
"smtp_host": self.smtp_host,
"smtp_port": self.smtp_port,
"accepted_count": self.accepted_count,
"command_id": self.command_id,
"delivery_status": self.delivery_status,
"duplicate": self.duplicate,
}
def audit_dict(self) -> dict[str, Any]:
return {
"campaign_id": self.campaign_id,
"version_id": self.version_id,
"recipient_count": len(self.to),
"dry_run": self.dry_run,
"attached_jobs_csv": self.attached_jobs_csv,
"attached_report_json": self.attached_report_json,
"command_id": self.command_id,
"delivery_status": self.delivery_status,
"duplicate": self.duplicate,
}
@@ -62,18 +82,15 @@ def _selected_version(
return version
def _load_config(version: CampaignVersion) -> CampaignConfig:
snapshot_path = _write_campaign_snapshot(version)
return load_campaign_config(snapshot_path)
def _load_config(session: Session, version: CampaignVersion) -> CampaignConfig:
_campaign, _version, config = load_version_config(session, version.id)
return config
def _effective_from(config: CampaignConfig) -> tuple[str, str | None]:
if config.recipients.from_:
return config.recipients.from_[0].email, config.recipients.from_[0].name
smtp_config = config.server.runtime_smtp_config()
if smtp_config and smtp_config.username and "@" in smtp_config.username:
return smtp_config.username, None
raise SmtpConfigurationError("Report email requires a recipients.from address or an SMTP username that is an email address")
raise SmtpConfigurationError("Report email requires a Campaign-owned recipients.from address")
def _text_summary(report: dict[str, Any]) -> str:
@@ -94,8 +111,10 @@ def _text_summary(report: dict[str, Any]) -> str:
f"- Needs attention: {cards['needs_attention']}",
f"- Sent: {cards['sent']}",
f"- Failed: {cards['failed']}",
f"- SMTP skipped (excluded): {cards.get('skipped', status.get('send', {}).get('skipped', 0))}",
f"- IMAP appended: {cards['imap_appended']}",
f"- IMAP failed: {cards['imap_failed']}",
f"- IMAP skipped: {cards.get('imap_skipped', status.get('imap', {}).get('skipped', 0))}",
"",
f"Build status: {status.get('build', {})}",
f"Validation status: {status.get('validation', {})}",
@@ -105,7 +124,7 @@ def _text_summary(report: dict[str, Any]) -> str:
]
if delivery.get("estimated_remaining_send_human"):
lines.extend(["", f"Estimated remaining send time: {delivery['estimated_remaining_send_human']}"])
lines.extend(["", "This report was generated by MultiMailer."])
lines.extend(["", "This report was generated by GovOPlaN."])
return "\n".join(lines)
@@ -119,20 +138,20 @@ def build_report_message(
report_json: dict[str, Any] | None = None,
) -> EmailMessage:
from_email, from_name = _effective_from(config)
subject = f"MultiMailer report: {campaign.name}"
subject = f"GovOPlaN report: {campaign.name}"
msg = EmailMessage()
msg["Subject"] = subject
msg["From"] = formataddr((from_name or from_email, from_email))
msg["To"] = ", ".join(to)
msg["X-MultiMailer-Report"] = "campaign"
msg["X-GovOPlaN-Report"] = "campaign"
msg.set_content(_text_summary(report))
if jobs_csv is not None:
filename = f"multimailer-{campaign.external_id}-jobs.csv"
filename = f"govoplan-{campaign.external_id}-jobs.csv"
msg.add_attachment(jobs_csv.encode("utf-8"), maintype="text", subtype="csv", filename=filename)
if report_json is not None:
filename = f"multimailer-{campaign.external_id}-report.json"
filename = f"govoplan-{campaign.external_id}-report.json"
msg.add_attachment(
json.dumps(report_json, indent=2, ensure_ascii=False, default=str).encode("utf-8"),
maintype="application",
@@ -150,9 +169,11 @@ def send_campaign_report_email(
version_id: str | None = None,
to: list[str],
include_jobs: bool = False,
attach_jobs_csv: bool = True,
attach_jobs_csv: bool = False,
attach_report_json: bool = False,
dry_run: bool = False,
idempotency_key: str | None = None,
created_by_user_id: str | None = None,
) -> CampaignReportEmailResult:
campaign = session.get(Campaign, campaign_id)
if not campaign or campaign.tenant_id != tenant_id:
@@ -161,17 +182,41 @@ def send_campaign_report_email(
raise CampaignReportEmailError("At least one report recipient is required")
version = _selected_version(session, campaign, version_id)
config = _load_config(version)
smtp_config: SmtpConfig | None = config.server.runtime_smtp_config()
if smtp_config is None:
config = _load_config(session, version)
if not config.server.profile_capabilities.smtp_available:
raise SmtpConfigurationError("Campaign has no SMTP configuration")
profile_id = campaign_mail_profile_id(version.raw_json if isinstance(version.raw_json, dict) else {})
if not profile_id:
raise SmtpConfigurationError("Campaign has no Mail profile reference")
if not isinstance(version.execution_snapshot, dict):
raise CampaignReportEmailError(
"Report email requires a validated and built campaign version with stored Mail-profile evidence."
)
try:
snapshot = ensure_execution_snapshot(session, version)
except ExecutionSnapshotError as exc:
raise CampaignReportEmailError(
"Report email requires a validated and built campaign version with current Mail-profile evidence."
) from exc
if not snapshot.smtp_transport_revision:
raise CampaignReportEmailError("Campaign build evidence has no SMTP transport revision")
mail = mail_integration()
if not dry_run:
if not mail.durable_delivery_available:
raise CampaignReportEmailError(
"Report email delivery requires the durable, idempotent Mail-owned outbox."
)
if not str(idempotency_key or "").strip():
raise CampaignReportEmailError(
"Live report email delivery requires an idempotency key"
)
report = generate_campaign_report(
session,
tenant_id=tenant_id,
campaign_id=campaign_id,
version_id=version.id,
include_jobs=include_jobs,
include_recent_failures=include_jobs,
)
jobs_csv = (
generate_jobs_csv(session, tenant_id=tenant_id, campaign_id=campaign_id, version_id=version.id)
@@ -187,38 +232,55 @@ def send_campaign_report_email(
jobs_csv=jobs_csv,
report_json=report_json,
)
envelope_from, _ = _effective_from(config)
if dry_run:
if not dry_run:
clean_idempotency_key = str(idempotency_key or "").strip()
from_email, _from_name = _effective_from(config)
if not snapshot.mail_profile_id:
raise CampaignReportEmailError(
"Campaign build evidence has no Mail profile reference"
)
try:
command = mail.submit_delivery_command(
session,
tenant_id=tenant_id,
command_type="campaign_report",
source_module="campaigns",
source_resource_type="campaign",
source_resource_id=campaign.id,
source_version_id=version.id,
idempotency_key=clean_idempotency_key,
profile_id=snapshot.mail_profile_id,
message_bytes=message.as_bytes(),
envelope_from=from_email,
envelope_recipients=to,
from_header=str(message["From"]),
expected_smtp_transport_revision=snapshot.smtp_transport_revision,
smtp_server_id=snapshot.smtp_server_id,
smtp_credential_id=snapshot.smtp_credential_id,
created_by_user_id=created_by_user_id,
)
except MailDeliveryCommandError as exc:
raise CampaignReportEmailError(str(exc)) from exc
return CampaignReportEmailResult(
campaign_id=campaign.id,
version_id=version.id,
to=to,
subject=str(message["Subject"]),
dry_run=True,
dry_run=False,
sent=False,
attached_jobs_csv=jobs_csv is not None,
attached_report_json=report_json is not None,
smtp_host=smtp_config.host,
smtp_port=smtp_config.port,
command_id=str(command["id"]),
delivery_status=str(command["status"]),
duplicate=bool(command.get("duplicate")),
)
result = mail_integration().send_email_message(
message,
smtp_config=smtp_config,
envelope_from=envelope_from,
envelope_recipients=to,
)
return CampaignReportEmailResult(
campaign_id=campaign.id,
version_id=version.id,
to=to,
subject=str(message["Subject"]),
dry_run=False,
sent=True,
dry_run=True,
sent=False,
attached_jobs_csv=jobs_csv is not None,
attached_report_json=report_json is not None,
smtp_host=result.host,
smtp_port=result.port,
accepted_count=result.accepted_count,
)
@@ -0,0 +1,308 @@
"""Cross-module provider for Campaign's recipient-free aggregate report."""
from __future__ import annotations
from collections.abc import Mapping
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import Campaign
from govoplan_campaign.backend.report_privacy_policy import (
effective_campaign_report_privacy_policy,
)
from govoplan_campaign.backend.reports.aggregate import (
generate_aggregate_campaign_report,
)
from govoplan_campaign.backend.route_support import (
_campaign_query_for_principal,
_get_campaign_for_principal,
)
from govoplan_core.auth import has_scope
from govoplan_core.core.reporting import (
REPORT_PROVIDER_CONTRACT_VERSION,
ReportDescriptor,
ReportParameterDescriptor,
ReportParameterOption,
ReportPrivacyTransform,
ReportProviderRequest,
ReportProviderResult,
ReportResultField,
)
CAMPAIGN_AGGREGATE_REPORT_ID = "delivery-outcomes"
CAMPAIGN_REPORT_PRIVACY_TRANSFORMS = (
"server_side_aggregation",
"small_cell_suppression",
"complementary_suppression",
"explicit_denominator",
"recipient_payload_exclusion",
)
class CampaignAggregateReportProvider:
provider_id = "campaigns"
contract_version = REPORT_PROVIDER_CONTRACT_VERSION
def list_reports(
self,
session: object,
principal: object,
) -> tuple[ReportDescriptor, ...]:
del session
if not has_scope(principal, "campaigns:report:read"):
return ()
return (_descriptor(),)
def parameter_options(
self,
session: object,
principal: object,
*,
report_id: str,
parameter_key: str,
query: str,
limit: int,
) -> tuple[ReportParameterOption, ...]:
if report_id != CAMPAIGN_AGGREGATE_REPORT_ID or parameter_key != "campaign_id":
return ()
if not has_scope(principal, "campaigns:report:read"):
return ()
sql_session = _session(session)
rows = _campaign_query_for_principal(sql_session, principal)
clean_query = query.strip().casefold()
campaigns = (
rows.order_by(Campaign.updated_at.desc(), Campaign.id.asc())
.limit(max(1, min(limit, 200)) if not clean_query else 500)
.all()
)
if clean_query:
campaigns = [
campaign
for campaign in campaigns
if clean_query in campaign.name.casefold()
][: max(1, min(limit, 200))]
return tuple(
ReportParameterOption(
value=campaign.id,
label=campaign.name,
description=campaign.status,
)
for campaign in campaigns
)
def execute_report(
self,
session: object,
principal: object,
*,
request: ReportProviderRequest,
) -> ReportProviderResult:
if request.report_id != CAMPAIGN_AGGREGATE_REPORT_ID:
raise LookupError("Campaign report provider does not know this report")
if not has_scope(principal, "campaigns:report:read"):
raise PermissionError("Missing scope: campaigns:report:read")
campaign_id = str(request.parameters.get("campaign_id") or "").strip()
if not campaign_id:
raise ValueError("campaign_id is required")
version_id = str(request.parameters.get("version_id") or "").strip() or None
sql_session = _session(session)
campaign = _get_campaign_for_principal(sql_session, campaign_id, principal)
report = generate_aggregate_campaign_report(
sql_session,
tenant_id=principal.tenant_id,
campaign_id=campaign.id,
version_id=version_id,
)
policy = effective_campaign_report_privacy_policy(
sql_session,
tenant_id=principal.tenant_id,
)
selected_version_id = version_id or campaign.current_version_id
return ReportProviderResult(
report_id=CAMPAIGN_AGGREGATE_REPORT_ID,
generated_at=report.generated_at,
payload=report.model_dump(mode="json"),
source_revisions=(
{
"module_id": "campaigns",
"resource_type": "campaign",
"resource_id": campaign.id,
"revision_type": "campaign_version",
"revision_id": selected_version_id,
"version_number": report.version_number,
"updated_at": campaign.updated_at.isoformat(),
},
),
effective_scope={
"tenant_id": principal.tenant_id,
"campaign_id": campaign.id,
"campaign_version_id": selected_version_id,
"audience": dict(request.audience_scope),
},
applied_privacy_transforms=CAMPAIGN_REPORT_PRIVACY_TRANSFORMS,
provenance={
"provider": "campaigns",
"projection": "recipient-free-delivery-outcomes-v1",
"privacy_policy": policy.as_dict(),
"purpose": request.purpose,
},
)
def authorize_result(
self,
session: object,
principal: object,
*,
report_id: str,
source_revisions: tuple[Mapping[str, object], ...],
effective_scope: Mapping[str, object],
) -> bool:
del source_revisions
if report_id != CAMPAIGN_AGGREGATE_REPORT_ID or not has_scope(
principal, "campaigns:report:read"
):
return False
campaign_id = str(effective_scope.get("campaign_id") or "").strip()
tenant_id = str(effective_scope.get("tenant_id") or "").strip()
if not campaign_id or tenant_id != str(getattr(principal, "tenant_id", "")):
return False
return (
_campaign_query_for_principal(_session(session), principal)
.filter(Campaign.id == campaign_id)
.first()
is not None
)
def _descriptor() -> ReportDescriptor:
fields = (
("generated_at", "Generated", "datetime", "Campaign"),
("campaign.id", "Campaign ID", "string", "Campaign"),
("campaign.name", "Campaign", "string", "Campaign"),
("campaign.status", "Status", "string", "Campaign"),
("version_number", "Version", "integer", "Campaign"),
("completion_state", "Completion", "string", "Campaign"),
(
"population.denominator",
"Report denominator",
"suppressed_count",
"Population",
),
(
"population.denominator_definition",
"Denominator definition",
"string",
"Population",
),
(
"population.inactive_source_entries",
"Inactive source entries",
"suppressed_count",
"Population",
),
(
"population.excluded_or_blocked_jobs",
"Excluded or blocked jobs",
"suppressed_count",
"Population",
),
("outcomes.smtp_accepted", "SMTP accepted", "suppressed_count", "Outcomes"),
(
"outcomes.postbox_accepted",
"Postbox accepted",
"suppressed_count",
"Outcomes",
),
(
"outcomes.print_accepted",
"Printable output accepted",
"suppressed_count",
"Outcomes",
),
(
"outcomes.delivered",
"Both channels accepted",
"suppressed_count",
"Outcomes",
),
(
"outcomes.partially_accepted",
"Partially accepted",
"suppressed_count",
"Outcomes",
),
("outcomes.failed", "Failed", "suppressed_count", "Outcomes"),
("outcomes.outcome_unknown", "Outcome unknown", "suppressed_count", "Outcomes"),
(
"outcomes.queued_or_active",
"Queued or active",
"suppressed_count",
"Outcomes",
),
("outcomes.not_attempted", "Not attempted", "suppressed_count", "Outcomes"),
("outcomes.cancelled", "Cancelled", "suppressed_count", "Outcomes"),
("outcomes.excluded", "Excluded", "suppressed_count", "Outcomes"),
("time_range.first_activity_at", "First activity", "datetime", "Activity"),
("time_range.last_activity_at", "Last activity", "datetime", "Activity"),
("time_range.suppressed", "Activity range suppressed", "boolean", "Activity"),
("privacy.small_cell_threshold", "Small-cell threshold", "integer", "Privacy"),
("privacy.suppression_applied", "Suppression applied", "boolean", "Privacy"),
("privacy.rule", "Privacy rule", "string", "Privacy"),
)
return ReportDescriptor(
provider_id="campaigns",
report_id=CAMPAIGN_AGGREGATE_REPORT_ID,
revision="campaign.aggregate.v1",
title="Campaign delivery outcomes",
summary=(
"Privacy-protected delivery outcomes without recipient-level records."
),
parameters=(
ReportParameterDescriptor(
key="campaign_id",
label="Campaign",
type="reference",
required=True,
options_from_provider=True,
),
ReportParameterDescriptor(
key="version_id",
label="Campaign version",
type="string",
required=False,
description="Leave empty to use the current campaign version.",
),
),
result_schema=tuple(
ReportResultField(
path=path,
label=label,
type=field_type, # type: ignore[arg-type]
group=group,
nullable=path.startswith("time_range.") or path == "version_number",
)
for path, label, field_type, group in fields
),
privacy_transforms=tuple(
ReportPrivacyTransform(id=item, label=item.replace("_", " ").title())
for item in CAMPAIGN_REPORT_PRIVACY_TRANSFORMS
),
retention_class="stored_report_detail",
export_formats=("json",),
reidentification_risk="low",
presentation={"kind": "metric_summary"},
)
def _session(value: object) -> Session:
if not isinstance(value, Session):
raise TypeError("Campaign report provider requires a SQLAlchemy Session")
return value
__all__ = [
"CAMPAIGN_AGGREGATE_REPORT_ID",
"CAMPAIGN_REPORT_PRIVACY_TRANSFORMS",
"CampaignAggregateReportProvider",
]
@@ -0,0 +1,286 @@
from __future__ import annotations
import copy
from pathlib import Path, PureWindowsPath
from typing import Any
from govoplan_campaign.backend.campaign.mail_profile_boundary import public_campaign_mail_server
# These fields locate process-local or storage-backend resources, or authorize
# a worker claim. They are useful for tightly controlled diagnostics but are
# not part of the campaign business-data contract.
CAMPAIGN_INTERNAL_RESPONSE_KEYS = frozenset(
{
"campaign_file",
"claim_token",
"eml_local_path",
"eml_path",
"eml_storage_key",
"local_path",
"source_base_path",
"storage_bucket",
"storage_key",
}
)
CAMPAIGN_DIAGNOSTIC_RESPONSE_KEYS = frozenset(
{
"background_workers_enabled",
"build_token",
"celery_enabled",
"claimed_at",
"effective_policy_sha256",
"execution_input_sha256",
"imap_claimed_at",
"imap_transport_revision",
"job_manifest_sha256",
"smtp_started_at",
"smtp_transport_revision",
}
)
_SEND_NOW_RESULT_KEYS = (
"campaign_id",
"version_id",
"attempted_count",
"sent_count",
"failed_count",
"outcome_unknown_count",
"skipped_count",
"paused_count",
"preflight_count",
"batch_state",
"batch_pause_reason_code",
"smtp_connection_count",
"smtp_reconnect_count",
"delivery_mode",
"dry_run",
)
_SEND_NOW_JOB_RESULT_KEYS = (
"campaign_id",
"version_id",
"job_id",
"status",
"attempt_number",
"dry_run",
"queued_count",
"skipped_count",
"blocked_count",
"enqueued_count",
"delivery_mode",
"worker_queue_available",
)
_SYNCHRONOUS_POLICY_KEYS = (
"max_recipient_jobs",
"source",
"deployment_max_recipient_jobs",
"tenant_max_recipient_jobs",
"system_max_recipient_jobs",
"deployment_ceiling_explicit",
)
_VALIDATION_SUMMARY_KEYS = ("ok", "error_count", "warning_count")
_BUILD_SUMMARY_KEYS = (
"built_count",
"build_failed_count",
"ready_count",
"warning_count",
"needs_review_count",
"blocked_count",
"excluded_count",
"inactive_count",
"queueable_count",
)
def public_campaign_payload(value: Any, *, include_diagnostics: bool = False) -> Any:
"""Return a detached payload without infrastructure-only locators."""
if isinstance(value, dict):
blocked_keys = CAMPAIGN_INTERNAL_RESPONSE_KEYS
if not include_diagnostics:
blocked_keys = blocked_keys | CAMPAIGN_DIAGNOSTIC_RESPONSE_KEYS
return {
key: public_campaign_payload(item, include_diagnostics=include_diagnostics)
for key, item in value.items()
if key not in blocked_keys
}
if isinstance(value, list):
return [public_campaign_payload(item, include_diagnostics=include_diagnostics) for item in value]
if isinstance(value, tuple):
return tuple(public_campaign_payload(item, include_diagnostics=include_diagnostics) for item in value)
return copy.deepcopy(value)
def public_delivery_result_message(
*,
last_error: Any,
send_status: Any,
imap_status: Any,
postbox_status: Any = None,
) -> str | None:
"""Map persisted provider text to a stable business-safe explanation."""
if not last_error:
return None
clean_send_status = str(send_status or "")
clean_imap_status = str(imap_status or "")
clean_postbox_status = str(postbox_status or "")
if clean_postbox_status == "outcome_unknown":
return "Postbox delivery outcome requires operator reconciliation."
if clean_send_status == "outcome_unknown":
return "Delivery outcome requires operator reconciliation."
if clean_postbox_status in {
"rejected_temporary",
"rejected_permanent",
}:
return "Postbox delivery was rejected; an operator can inspect restricted diagnostics."
if clean_postbox_status == "partially_accepted":
return "Some Postbox targets accepted the message and others rejected it."
if clean_send_status in {"failed_temporary", "failed_permanent"}:
if clean_postbox_status in {"", "not_requested"}:
return "SMTP delivery failed; an operator can inspect restricted diagnostics."
return "Delivery failed; an operator can inspect restricted diagnostics."
if clean_imap_status in {"outcome_unknown", "appending"}:
return "Sent-folder append outcome requires operator reconciliation."
if clean_imap_status in {"failed", "skipped"}:
return "Sent-folder append did not complete; an operator can inspect restricted diagnostics."
return "Delivery recorded a warning; an operator can inspect restricted diagnostics."
def public_send_campaign_now_result(
value: dict[str, Any],
*,
validation_summary: dict[str, Any],
build_summary: dict[str, Any],
) -> dict[str, Any]:
"""Project synchronous delivery into its recipient-authorized public contract.
Per-job provider messages are deliberately omitted. They can contain SMTP
diagnostics or refused envelope addresses and belong only in restricted
diagnostics backed by persisted job state.
"""
result = _selected_payload(value, _SEND_NOW_RESULT_KEYS)
policy = value.get("synchronous_send_policy")
result["synchronous_send_policy"] = _selected_payload(
policy if isinstance(policy, dict) else {},
_SYNCHRONOUS_POLICY_KEYS,
)
rows = value.get("results")
if isinstance(rows, list):
result["results"] = [
_selected_payload(row, _SEND_NOW_JOB_RESULT_KEYS)
for row in rows
if isinstance(row, dict)
]
else:
result["results"] = []
result["validation"] = _selected_payload(validation_summary, _VALIDATION_SUMMARY_KEYS)
result["build"] = _selected_payload(build_summary, _BUILD_SUMMARY_KEYS)
return result
def send_campaign_now_audit_details(value: dict[str, Any]) -> dict[str, Any]:
"""Return aggregate-only evidence for a synchronous Campaign send audit."""
details = _selected_payload(value, _SEND_NOW_RESULT_KEYS)
policy = value.get("synchronous_send_policy")
details["synchronous_send_policy"] = _selected_payload(
policy if isinstance(policy, dict) else {},
_SYNCHRONOUS_POLICY_KEYS,
)
return details
def _selected_payload(value: dict[str, Any], keys: tuple[str, ...]) -> dict[str, Any]:
return {
key: copy.deepcopy(value[key])
for key in keys
if key in value
}
def public_campaign_configuration(value: Any) -> Any:
"""Return campaign JSON without infrastructure locators or mail secrets.
Password-named business fields are intentionally retained. Only the
schema-defined SMTP/IMAP credential paths under ``server`` are secrets.
"""
payload = public_campaign_payload(value)
if not isinstance(payload, dict):
return payload
if "server" in payload:
payload["server"] = public_campaign_mail_server(payload)
_sanitize_configuration_paths(payload)
return payload
def public_source_filename(value: Any) -> str | None:
if value is None:
return None
text = str(value).strip()
if not text:
return text
windows_path = PureWindowsPath(text)
return windows_path.name if windows_path.is_absolute() or "\\" in text else Path(text).name
def _sanitize_configuration_paths(payload: dict[str, Any]) -> None:
template = payload.get("template")
source = template.get("source") if isinstance(template, dict) else None
if isinstance(source, dict):
for key in ("subject_path", "text_path", "html_path"):
if key in source:
source[key] = _public_configuration_path(source[key])
entries = payload.get("entries")
entry_source = entries.get("source") if isinstance(entries, dict) else None
if isinstance(entry_source, dict) and "path" in entry_source:
entry_source["path"] = _public_configuration_path(entry_source["path"])
attachments = payload.get("attachments")
if isinstance(attachments, dict):
if "base_path" in attachments:
attachments["base_path"] = _public_configuration_path(attachments["base_path"])
base_paths = attachments.get("base_paths")
if isinstance(base_paths, list):
for item in base_paths:
if not isinstance(item, dict):
continue
for key in ("path", "source"):
if key in item:
item[key] = _public_configuration_path(item[key])
_sanitize_attachment_rules(attachments.get("global"))
if isinstance(entries, dict):
inline_entries = entries.get("inline")
if isinstance(inline_entries, list):
for entry in inline_entries:
if isinstance(entry, dict):
_sanitize_attachment_rules(entry.get("attachments"))
defaults = entries.get("defaults")
if isinstance(defaults, dict):
_sanitize_attachment_rules(defaults.get("attachments"))
def _sanitize_attachment_rules(value: Any) -> None:
if not isinstance(value, list):
return
for rule in value:
if isinstance(rule, dict) and "base_dir" in rule:
rule["base_dir"] = _public_configuration_path(rule["base_dir"])
def _public_configuration_path(value: Any) -> Any:
if not isinstance(value, str) or not value.strip():
return value
text = value.strip()
windows_path = PureWindowsPath(text)
path = Path(text)
if windows_path.is_absolute():
return windows_path.name
if path.is_absolute() or text.startswith("~"):
return path.name
return value
+333 -8
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import copy
from dataclasses import dataclass
import hashlib
import json
from datetime import datetime, timedelta, timezone
@@ -9,7 +10,35 @@ from typing import Any, Callable
from sqlalchemy.orm import Session
from govoplan_campaign.backend.db.models import CampaignJob, CampaignVersion, JobImapStatus, JobQueueStatus
from govoplan_core.core.recovery import (
RecoveryGuaranteeError,
RecoveryMode,
RecoveryPlan,
RecoveryStatus,
)
from govoplan_core.core.recovery_runtime import (
DurableRecoveryOperation,
RecoveryOperationBusy,
RecoveryOperationStateConflict,
begin_durable_recovery_operation,
)
from govoplan_core.core.object_storage import (
StorageBackend,
StorageBackendError,
StorageObjectMissing,
configured_storage_backend,
)
from govoplan_core.core.runtime_coordination import process_runtime_identity
from govoplan_core.db.session import get_database
from govoplan_core.settings import settings as core_settings
from govoplan_campaign.backend.db.models import (
CampaignJob,
CampaignSchedule,
CampaignVersion,
JobImapStatus,
JobQueueStatus,
)
from govoplan_campaign.backend.runtime import get_settings
FINAL_VERSION_STATES = {
"completed",
@@ -28,6 +57,15 @@ FINAL_EML_SEND_STATUSES = {
}
@dataclass(frozen=True, slots=True)
class _GeneratedArtifactRecovery:
operation: DurableRecoveryOperation
job_id: str
storage_key: str | None
local_path: str | None
storage: StorageBackend | None
def _cutoff(days: int | None, *, now: datetime) -> datetime | None:
if days is None:
return None
@@ -105,14 +143,233 @@ def _apply_raw_json_retention(
return result
def _artifact_locator_sha256(
*,
storage_key: str | None,
local_path: str | None,
) -> str:
return _json_sha256(
{
"storage_key": storage_key,
"local_path": local_path,
}
)
def _begin_generated_artifact_recovery(
*,
job: CampaignJob,
storage: StorageBackend | None,
) -> _GeneratedArtifactRecovery | None:
storage_key = str(job.eml_storage_key) if job.eml_storage_key else None
local_path = str(job.eml_local_path) if job.eml_local_path else None
locator_sha256 = _artifact_locator_sha256(
storage_key=storage_key,
local_path=local_path,
)
try:
started = begin_durable_recovery_operation(
get_database().SessionLocal,
identity=process_runtime_identity(),
module_id="campaigns",
operation_type="generated-artifact-retention",
idempotency_key=(
f"campaign-retention:{job.id}:{locator_sha256[:32]}"
),
request={
"tenant_id": job.tenant_id,
"campaign_id": job.campaign_id,
"version_id": job.campaign_version_id,
"job_id": job.id,
"message_sha256": job.eml_sha256,
"artifact_locator_sha256": locator_sha256,
},
recovery_plan=RecoveryPlan(
mode=RecoveryMode.FORWARD_RECOVERY,
preconditions=(
"the Campaign job is terminal and outside its retention window",
"no IMAP append or delivery outcome remains unresolved",
),
forward_recovery_steps=(
"verify whether each recorded artifact still exists",
"clear the database locator only after absence is established",
),
verification_steps=(
"reload the Campaign job through an independent session",
"probe every original object or local-development path",
),
),
precondition_evidence={
"job_id": job.id,
"queue_status": job.queue_status,
"send_status": job.send_status,
"imap_status": job.imap_status,
"message_sha256": job.eml_sha256,
"artifact_locator_sha256": locator_sha256,
},
lease_resource_key=f"campaign:retention:{job.tenant_id}:{job.id}",
lease_ttl_seconds=15 * 60,
resource_type="campaign_job",
resource_id=job.id,
metadata={
"resources": [
"postgresql",
"object-storage" if storage_key else "local-development-storage",
],
},
)
except (RecoveryOperationBusy, RecoveryOperationStateConflict):
return None
if started.replayed or started.operation is None:
return None
return _GeneratedArtifactRecovery(
operation=started.operation,
job_id=job.id,
storage_key=storage_key,
local_path=local_path,
storage=storage,
)
def _generated_artifact_recovery_evidence(
recovery: _GeneratedArtifactRecovery,
) -> tuple[str, dict[str, Any]]:
probes: dict[str, bool | None] = {}
if recovery.storage_key:
try:
if recovery.storage is None:
raise StorageBackendError("Artifact storage is unavailable")
probes["object_missing"] = not recovery.storage.exists(
recovery.storage_key
)
except (StorageBackendError, OSError):
probes["object_missing"] = None
if recovery.local_path:
try:
probes["local_path_missing"] = not Path(recovery.local_path).exists()
except OSError:
probes["local_path_missing"] = None
with get_database().SessionLocal() as evidence_session:
job = evidence_session.get(CampaignJob, recovery.job_id)
job_present = job is not None
metadata_cleared = bool(
job is None
or (
(
not recovery.storage_key
or job.eml_storage_key != recovery.storage_key
)
and (
not recovery.local_path
or job.eml_local_path != recovery.local_path
)
)
)
metadata_intact = bool(
job is not None
and job.eml_storage_key == recovery.storage_key
and job.eml_local_path == recovery.local_path
)
probe_values = tuple(probes.values())
probe_verified = bool(probe_values) and all(
value is not None for value in probe_values
)
artifacts_absent = probe_verified and all(value is True for value in probe_values)
artifacts_intact = probe_verified and all(value is False for value in probe_values)
evidence = {
"verified": probe_verified,
"checks": {
"job_state_reloaded": True,
"artifact_locations_probed": probe_verified,
},
"job_present": job_present,
"metadata_cleared": metadata_cleared,
"metadata_intact": metadata_intact,
"artifact_probes": probes,
}
if not probe_verified:
return "outcome_unknown", evidence
if artifacts_absent and metadata_cleared:
return "succeeded", evidence
if artifacts_intact and metadata_intact:
return "failed", evidence
return "recovery_required", evidence
def _finish_generated_artifact_recovery(
recovery: _GeneratedArtifactRecovery,
) -> None:
outcome, evidence = _generated_artifact_recovery_evidence(recovery)
if outcome == "succeeded":
recovery.operation.succeed(evidence=evidence)
elif outcome == "failed":
recovery.operation.reject(
summary="Generated Campaign artifacts were not deleted",
evidence=evidence,
)
elif outcome == "outcome_unknown":
recovery.operation.unresolved(
status=RecoveryStatus.OUTCOME_UNKNOWN,
summary="Generated artifact deletion could not be verified",
evidence=evidence,
failure_summary="Artifact storage availability prevented verification",
)
else:
recovery.operation.unresolved(
status=RecoveryStatus.RECOVERY_REQUIRED,
summary="Generated artifact retention is only partially complete",
evidence=evidence,
failure_summary="Artifact and Campaign metadata state require reconciliation",
)
def _finish_generated_artifact_recoveries(
recoveries: list[_GeneratedArtifactRecovery],
) -> None:
failures: list[Exception] = []
for recovery in recoveries:
try:
_finish_generated_artifact_recovery(recovery)
except Exception as exc: # preserve every operation's chance to close
failures.append(exc)
if failures:
raise RecoveryGuaranteeError(
f"{len(failures)} Campaign retention recovery operation(s) could not be finalized"
) from failures[0]
def _apply_eml_retention(
session: Session,
*,
dry_run: bool,
now: datetime,
policy_for_campaign_id: Callable[[str | None], object],
storage: StorageBackend | None = None,
recovery_operations: list[_GeneratedArtifactRecovery] | None = None,
) -> dict[str, int]:
result = {"eligible": 0, "metadata_cleared": 0, "files_deleted": 0, "files_missing": 0, "skipped_not_final": 0}
result = {
"eligible": 0,
"metadata_cleared": 0,
"files_deleted": 0,
"files_missing": 0,
"delete_failed": 0,
"recovery_blocked": 0,
"skipped_not_final": 0,
"skipped_schedule_source": 0,
}
protected_source_versions = {
str(version_id)
for (version_id,) in (
session.query(CampaignSchedule.source_version_id)
.filter(
CampaignSchedule.delivery_mode == "autonomous",
CampaignSchedule.next_fire_at.is_not(None),
)
.all()
)
}
jobs = (
session.query(CampaignJob)
.filter((CampaignJob.eml_local_path.is_not(None)) | (CampaignJob.eml_storage_key.is_not(None)))
@@ -120,6 +377,9 @@ def _apply_eml_retention(
.all()
)
for job in jobs:
if getattr(job, "campaign_version_id", None) in protected_source_versions:
result["skipped_schedule_source"] += 1
continue
policy = policy_for_campaign_id(job.campaign_id)
cutoff = _cutoff(policy.generated_eml_retention_days, now=now)
if not _is_before_cutoff(job.updated_at, cutoff):
@@ -127,12 +387,43 @@ def _apply_eml_retention(
if job.queue_status in {JobQueueStatus.QUEUED.value, JobQueueStatus.SENDING.value} or job.send_status not in FINAL_EML_SEND_STATUSES:
result["skipped_not_final"] += 1
continue
if job.imap_status == JobImapStatus.PENDING.value:
if job.imap_status in {
JobImapStatus.PENDING.value,
JobImapStatus.APPENDING.value,
JobImapStatus.OUTCOME_UNKNOWN.value,
}:
result["skipped_not_final"] += 1
continue
result["eligible"] += 1
if dry_run:
continue
active_storage = storage
if job.eml_storage_key and active_storage is None:
active_storage = configured_storage_backend(
get_settings() or core_settings
)
if recovery_operations is not None:
recovery = _begin_generated_artifact_recovery(
job=job,
storage=active_storage,
)
if recovery is None:
result["recovery_blocked"] += 1
continue
recovery_operations.append(recovery)
if job.eml_storage_key:
assert active_storage is not None
try:
if active_storage.exists(job.eml_storage_key):
active_storage.delete(job.eml_storage_key)
result["files_deleted"] += 1
else:
result["files_missing"] += 1
except StorageObjectMissing:
result["files_missing"] += 1
except StorageBackendError:
result["delete_failed"] += 1
continue
if job.eml_local_path:
path = Path(job.eml_local_path)
if path.exists():
@@ -184,8 +475,42 @@ def apply_campaign_retention(
now: datetime,
policy_for_campaign_id: Callable[[str | None], object],
) -> dict[str, dict[str, int]]:
return {
"raw_campaign_json": _apply_raw_json_retention(session, dry_run=dry_run, now=now, policy_for_campaign_id=policy_for_campaign_id),
"generated_eml": _apply_eml_retention(session, dry_run=dry_run, now=now, policy_for_campaign_id=policy_for_campaign_id),
"stored_report_detail": _apply_report_detail_retention(session, dry_run=dry_run, now=now, policy_for_campaign_id=policy_for_campaign_id),
}
recoveries: list[_GeneratedArtifactRecovery] = []
try:
# Start external-effect fences before queries for database-only
# redaction can autoflush unrelated changes in the caller session.
generated_eml = _apply_eml_retention(
session,
dry_run=dry_run,
now=now,
policy_for_campaign_id=policy_for_campaign_id,
recovery_operations=None if dry_run else recoveries,
)
result = {
"raw_campaign_json": _apply_raw_json_retention(
session,
dry_run=dry_run,
now=now,
policy_for_campaign_id=policy_for_campaign_id,
),
"generated_eml": generated_eml,
"stored_report_detail": _apply_report_detail_retention(
session,
dry_run=dry_run,
now=now,
policy_for_campaign_id=policy_for_campaign_id,
),
}
if not dry_run:
# External artifact deletion and its locator update form one
# module-owned recovery boundary. The outer Policy audit commits
# separately after Campaign has verified this boundary.
session.commit()
except Exception:
session.rollback()
if recoveries:
_finish_generated_artifact_recoveries(recoveries)
raise
if recoveries:
_finish_generated_artifact_recoveries(recoveries)
return result
@@ -0,0 +1,731 @@
from __future__ import annotations
import copy
import dataclasses
from collections.abc import Callable
from typing import Any
from fastapi import HTTPException, status
from sqlalchemy import and_, exists, or_
from sqlalchemy.orm import Session
from govoplan_campaign.backend.campaign.mail_profile_boundary import (
CAMPAIGN_MAIL_SERVER_KEYS,
campaign_mail_profile_id,
campaign_mail_references_unchanged,
campaign_preserves_legacy_mail_settings,
)
from govoplan_campaign.backend.archive_encryption import (
CampaignArchiveEncryptionError,
stamp_legacy_zipcrypto_acknowledgements,
)
from govoplan_campaign.backend.db.models import (
Campaign,
CampaignIssue,
CampaignJob,
CampaignShare,
CampaignStatus,
CampaignVersion,
CampaignVersionWorkflowState,
RecipientImportMappingProfile,
)
from govoplan_campaign.backend.path_security import CampaignPathSecurityError
from govoplan_campaign.backend.persistence.campaigns import CampaignPersistenceError
from govoplan_campaign.backend.persistence.versions import (
LockedCampaignVersionError,
is_user_locked_version,
is_version_final_locked,
is_version_locked,
update_campaign_version,
)
from govoplan_campaign.backend.schemas import (
CampaignVersionDetailResponse,
CampaignVersionUpdateRequest,
RecipientImportMappingProfilePayload,
)
from govoplan_campaign.backend.sending.execution import (
clear_execution_snapshot,
)
from govoplan_core.audit.logging import audit_from_principal
from govoplan_core.auth import ApiPrincipal, has_scope
from govoplan_core.core.access import CAPABILITY_ACCESS_DIRECTORY, AccessDirectory
from govoplan_core.core.concurrency import (
ConcurrencyError,
MissingPreconditionError,
RevisionConflictError,
assert_revision_precondition,
)
from govoplan_core.core.runtime import get_registry
def _capability_payload(value: object) -> dict[str, Any]:
if dataclasses.is_dataclass(value):
return dataclasses.asdict(value)
if isinstance(value, dict):
return dict(value)
payload: dict[str, Any] = {}
for key in (
"contact_id",
"address_book_id",
"display_name",
"email",
"email_label",
"organization",
"role_title",
"tags",
"source_kind",
"source_ref",
"source_revision",
"source_id",
"source_label",
"recipient_count",
"generated_at",
"recipients",
"fields",
"provenance",
):
if hasattr(value, key):
payload[key] = getattr(value, key)
return payload
def _registry_capability(name: str) -> object | None:
registry = get_registry()
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(name)
):
return None
return registry.capability(name)
def _access_directory() -> AccessDirectory:
registry = get_registry()
if (
registry is None
or not hasattr(registry, "has_capability")
or not registry.has_capability(CAPABILITY_ACCESS_DIRECTORY)
):
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Access directory capability is not configured",
)
capability = registry.require_capability(CAPABILITY_ACCESS_DIRECTORY)
if not isinstance(capability, AccessDirectory):
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Access directory capability is invalid",
)
return capability
def _get_campaign_for_tenant(
session: Session, campaign_id: str, tenant_id: str
) -> Campaign:
campaign = session.get(Campaign, campaign_id)
if not campaign or campaign.tenant_id != tenant_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign not found"
)
return campaign
def _get_version_for_tenant(
session: Session, version_id: str, tenant_id: str
) -> CampaignVersion:
version = session.get(CampaignVersion, version_id)
if not version:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
campaign = session.get(Campaign, version.campaign_id)
if not campaign or campaign.tenant_id != tenant_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
return version
def _principal_group_ids(session: Session, principal: ApiPrincipal) -> set[str]:
del session
return {
group.id
for group in _access_directory().groups_for_user(
principal.user.id, tenant_id=principal.tenant_id
)
}
def _campaign_acl_filter(session: Session, principal: ApiPrincipal):
if has_scope(principal, "tenant:*"):
return None
group_ids = _principal_group_ids(session, principal)
clauses = [Campaign.owner_user_id == principal.user.id]
if group_ids:
clauses.append(Campaign.owner_group_id.in_(group_ids))
share_clauses = [
and_(
CampaignShare.tenant_id == Campaign.tenant_id,
CampaignShare.campaign_id == Campaign.id,
CampaignShare.revoked_at.is_(None),
CampaignShare.target_type == "user",
CampaignShare.target_id == principal.user.id,
)
]
if group_ids:
share_clauses.append(
and_(
CampaignShare.tenant_id == Campaign.tenant_id,
CampaignShare.campaign_id == Campaign.id,
CampaignShare.revoked_at.is_(None),
CampaignShare.target_type == "group",
CampaignShare.target_id.in_(group_ids),
)
)
clauses.append(exists().where(or_(*share_clauses)))
return or_(*clauses)
def _campaign_acl_allows(
session: Session,
campaign: Campaign,
principal: ApiPrincipal,
*,
write: bool = False,
) -> bool:
if has_scope(principal, "tenant:*"):
return True
if campaign.owner_user_id == principal.user.id:
return True
group_ids = _principal_group_ids(session, principal)
if campaign.owner_group_id and campaign.owner_group_id in group_ids:
return True
target_ids = [principal.user.id, *group_ids]
if not target_ids:
return False
query = session.query(CampaignShare).filter(
CampaignShare.tenant_id == campaign.tenant_id,
CampaignShare.campaign_id == campaign.id,
CampaignShare.revoked_at.is_(None),
or_(
CampaignShare.target_type == "user",
CampaignShare.target_type == "group",
),
CampaignShare.target_id.in_(target_ids),
)
shares = query.all()
if not shares:
return False
if not write:
return True
return any(item.permission == "write" for item in shares)
def _require_campaign_acl(
session: Session,
campaign: Campaign,
principal: ApiPrincipal,
*,
write: bool = False,
) -> None:
if not _campaign_acl_allows(session, campaign, principal, write=write):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Campaign is not shared with this principal",
)
def _get_campaign_for_principal(
session: Session, campaign_id: str, principal: ApiPrincipal, *, write: bool = False
) -> Campaign:
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
_require_campaign_acl(session, campaign, principal, write=write)
return campaign
def _require_permission(principal: ApiPrincipal, scope: str) -> None:
if not has_scope(principal, scope):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail=f"Missing scope: {scope}"
)
def _campaign_query_for_principal(session: Session, principal: ApiPrincipal):
query = session.query(Campaign).filter(
Campaign.tenant_id == principal.tenant_id, Campaign.status != "deleted"
)
acl_filter = _campaign_acl_filter(session, principal)
if acl_filter is not None:
query = query.filter(acl_filter)
return query
def _get_recipient_import_profile_for_principal(
session: Session, profile_id: str, principal: ApiPrincipal
) -> RecipientImportMappingProfile:
profile = session.get(RecipientImportMappingProfile, profile_id)
if (
not profile
or profile.tenant_id != principal.tenant_id
or profile.owner_user_id != principal.user.id
):
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Recipient import mapping profile not found",
)
return profile
def _apply_recipient_import_profile_payload(
profile: RecipientImportMappingProfile,
payload: RecipientImportMappingProfilePayload,
) -> None:
profile.name = payload.name.strip()
profile.column_count = payload.column_count
profile.headers = list(payload.headers)
profile.normalized_headers = list(payload.normalized_headers)
profile.ordered_header_fingerprint = payload.ordered_header_fingerprint
profile.unordered_header_fingerprint = payload.unordered_header_fingerprint
profile.delimiter = payload.delimiter
profile.header_rows = payload.header_rows
profile.quoted = payload.quoted
profile.value_separators = payload.value_separators
profile.mappings = [mapping.model_dump(mode="json") for mapping in payload.mappings]
def _recipient_sections_changed(
current: dict[str, object] | None, proposed: dict[str, object] | None
) -> bool:
if proposed is None:
return False
current = current or {}
return any(
current.get(key) != proposed.get(key) for key in ("recipients", "entries")
)
def _campaign_mail_profile_id(raw_json: dict[str, object] | None) -> str | None:
return campaign_mail_profile_id(raw_json)
def _require_mail_profile_use_if_needed(
principal: ApiPrincipal, raw_json: dict[str, object] | None
) -> None:
if _campaign_mail_profile_id(raw_json) and not has_scope(
principal, "mail:profile:use"
):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Missing scope: mail:profile:use",
)
def _campaign_response_context(principal: ApiPrincipal) -> dict[str, bool]:
return {"include_diagnostics": has_scope(principal, "campaigns:diagnostic:read")}
def _campaign_version_detail_response(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
mutation: Callable[[], CampaignVersion],
*,
audit_action: str,
details: dict[str, Any] | Callable[[CampaignVersion], dict[str, Any]] | None = None,
validation_error_status: int | None = None,
) -> CampaignVersionDetailResponse:
try:
version = mutation()
audit_details = (
details(version)
if callable(details)
else dict(details or {"campaign_id": campaign_id})
)
audit_from_principal(
session,
principal,
action=audit_action,
object_type="campaign_version",
object_id=version.id,
details=audit_details,
commit=True,
)
_write_current_version_snapshot_if_available(version)
return CampaignVersionDetailResponse.model_validate(
version,
context=_campaign_response_context(principal),
)
except LockedCampaignVersionError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_409_CONFLICT, detail=str(exc)
) from exc
except CampaignPathSecurityError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT, detail=str(exc)
) from exc
except CampaignPersistenceError as exc:
session.rollback()
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)
) from exc
except RevisionConflictError:
session.rollback()
raise
except Exception as exc:
session.rollback()
if validation_error_status is None:
raise
raise HTTPException(
status_code=validation_error_status, detail=str(exc)
) from exc
def _update_campaign_version_detail_response(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
version_id: str,
payload: CampaignVersionUpdateRequest,
*,
if_match: str | None,
autosave: bool,
audit_action: str,
) -> CampaignVersionDetailResponse:
campaign = _get_campaign_for_principal(
session, campaign_id, principal, write=True
)
current_version = _get_version_for_tenant(session, version_id, principal.tenant_id)
if payload.base_revision is None:
error = MissingPreconditionError(
resource_type="campaign_version",
resource_id=version_id,
)
raise HTTPException(
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
detail=error.as_dict(),
)
try:
assert_revision_precondition(
if_match,
resource_type="campaign_version",
resource_id=version_id,
submitted_base_revision=payload.base_revision,
)
except MissingPreconditionError as exc:
raise HTTPException(
status_code=status.HTTP_428_PRECONDITION_REQUIRED,
detail=exc.as_dict(),
) from exc
except ConcurrencyError as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail={
"code": "invalid_precondition",
"message": str(exc),
},
) from exc
if _recipient_sections_changed(current_version.raw_json, payload.campaign_json):
_require_permission(principal, "campaigns:recipient:write")
acknowledgements: list[dict[str, Any]] = []
try:
payload.campaign_json, acknowledgements = (
stamp_legacy_zipcrypto_acknowledgements(
session,
campaign,
current_version.raw_json
if isinstance(current_version.raw_json, dict)
else {},
payload.campaign_json,
principal=principal,
)
)
except CampaignArchiveEncryptionError as exc:
audit_from_principal(
session,
principal,
action="campaign.archive_encryption_denied",
object_type="campaign_version",
object_id=version_id,
details={"campaign_id": campaign_id, "reason": str(exc)},
commit=True,
)
raise HTTPException(
status_code=(
status.HTTP_403_FORBIDDEN
if "Missing scope:" in str(exc)
else status.HTTP_422_UNPROCESSABLE_CONTENT
),
detail=str(exc),
) from exc
preserves_legacy_mail = (
not payload.migrate_legacy_mail_settings
and campaign_preserves_legacy_mail_settings(
current_version.raw_json, payload.campaign_json
)
)
unchanged_mail_selection = (
not payload.migrate_legacy_mail_settings
and campaign_mail_references_unchanged(current_version.raw_json, payload.campaign_json)
)
if not unchanged_mail_selection:
_require_mail_profile_use_if_needed(principal, payload.campaign_json)
try:
result = _campaign_version_detail_response(
session,
principal,
campaign_id,
lambda: update_campaign_version(
session,
tenant_id=principal.tenant_id,
campaign_id=campaign_id,
version_id=version_id,
raw_json=payload.campaign_json,
current_flow=payload.current_flow,
current_step=payload.current_step,
workflow_state=payload.workflow_state,
is_complete=payload.is_complete,
editor_state=payload.editor_state,
source_filename=payload.source_filename,
source_base_path=payload.source_base_path,
autosave=autosave,
migrate_legacy_mail_settings=payload.migrate_legacy_mail_settings,
expected_revision=payload.base_revision,
commit=False,
),
audit_action=audit_action,
details=lambda version: {
"campaign_id": campaign_id,
"current_flow": version.current_flow,
"current_step": version.current_step,
"base_revision": payload.base_revision,
"result_revision": version.edit_revision,
"reconciliation_kind": payload.reconciliation_kind,
"resolved_conflict_path_count": len(payload.resolved_conflict_paths),
"resolved_conflict_sections": sorted(
{
path.strip("/").split("/", 1)[0][:80]
for path in payload.resolved_conflict_paths[:100]
if path.strip("/")
}
),
"legacy_mail_settings_migrated": payload.migrate_legacy_mail_settings,
"legacy_mail_settings_preserved": preserves_legacy_mail,
"legacy_zipcrypto_acknowledgements": acknowledgements,
},
validation_error_status=status.HTTP_422_UNPROCESSABLE_CONTENT,
)
for acknowledgement in acknowledgements:
audit_from_principal(
session,
principal,
action="campaign.legacy_zipcrypto_acknowledged",
object_type="campaign_version",
object_id=version_id,
details={"campaign_id": campaign_id, **acknowledgement},
commit=False,
)
if acknowledgements:
session.commit()
return result
except RevisionConflictError as exc:
session.rollback()
audit_from_principal(
session,
principal,
action="campaign.version_conflict_detected",
object_type="campaign_version",
object_id=version_id,
details={
"campaign_id": campaign_id,
"submitted_base_revision": exc.submitted_base_revision,
"current_revision": exc.current_revision,
"retryable": True,
},
commit=True,
)
raise HTTPException(
status_code=status.HTTP_412_PRECONDITION_FAILED,
detail=exc.as_dict(),
) from exc
def _require_campaign_profile_use_if_needed(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
version_id: str | None = None,
) -> None:
campaign = _get_campaign_for_tenant(session, campaign_id, principal.tenant_id)
target_version_id = version_id or campaign.current_version_id
if not target_version_id:
return
version = _get_version_for_tenant(session, target_version_id, principal.tenant_id)
if version.campaign_id != campaign.id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Campaign version not found"
)
_require_mail_profile_use_if_needed(
principal, version.raw_json if isinstance(version.raw_json, dict) else {}
)
def _require_campaign_versions_profile_use(
session: Session,
principal: ApiPrincipal,
campaign_id: str,
version_ids: set[str],
) -> None:
"""Authorize every historical version affected by a campaign-wide action."""
for version_id in sorted(version_ids):
_require_campaign_profile_use_if_needed(
session,
principal,
campaign_id,
version_id,
)
def _get_version_for_principal(
session: Session,
version_id: str,
principal: ApiPrincipal,
*,
write: bool = False,
) -> CampaignVersion:
version = _get_version_for_tenant(session, version_id, principal.tenant_id)
campaign = _get_campaign_for_tenant(
session, version.campaign_id, principal.tenant_id
)
_require_campaign_acl(session, campaign, principal, write=write)
return version
def _sync_campaign_metadata_to_current_version(
session: Session, campaign: Campaign
) -> None:
"""Keep editable version JSON aligned with version-independent campaign metadata.
Campaign metadata can be edited from the overview while individual campaign
sections save the current version JSON later. Without this sync, a later
version save can re-apply stale `campaign.name` / `campaign.id` values from
raw_json and make the old overview metadata appear to come back. Audit-safe
or validation-locked versions are left untouched.
"""
if not campaign.current_version_id:
return
version = session.get(CampaignVersion, campaign.current_version_id)
if not version or version.campaign_id != campaign.id or is_version_locked(version):
return
raw_json = copy.deepcopy(
version.raw_json if isinstance(version.raw_json, dict) else {}
)
campaign_section = (
raw_json.get("campaign") if isinstance(raw_json.get("campaign"), dict) else {}
)
raw_json["campaign"] = {
**campaign_section,
"id": campaign.external_id,
"name": campaign.name,
"description": campaign.description or "",
}
version.raw_json = raw_json
session.add(version)
def _clear_current_version_mail_profile_for_owner_transfer(
session: Session, campaign: Campaign
) -> bool:
"""Force explicit profile reselection after campaign ownership changes.
User/group-scoped reusable mail profiles are evaluated against the current
owner. Instead of trying to keep a stale selection across an ownership
transfer, clear the profile from the editable current version and invalidate
validation/build state so the operator has to reselect and revalidate.
"""
if not campaign.current_version_id:
return False
version = session.get(CampaignVersion, campaign.current_version_id)
if not version or version.campaign_id != campaign.id:
return False
raw_json = copy.deepcopy(
version.raw_json if isinstance(version.raw_json, dict) else {}
)
server = (
raw_json.get("server") if isinstance(raw_json.get("server"), dict) else None
)
if not isinstance(server, dict):
return False
profile_id = _campaign_mail_profile_id(raw_json)
if not profile_id:
return False
if is_version_final_locked(version) or is_user_locked_version(version):
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_CONTENT,
detail="Change owner only after creating an editable campaign version; the current version has a selected mail profile and is locked.",
)
next_server = dict(server)
for key in CAMPAIGN_MAIL_SERVER_KEYS:
next_server.pop(key, None)
next_server.pop("profile_id", None)
raw_json["server"] = next_server
version.raw_json = raw_json
version.validation_summary = None
version.build_summary = None
clear_execution_snapshot(version)
version.locked_at = None
version.locked_by_user_id = None
version.workflow_state = CampaignVersionWorkflowState.EDITING.value
version.is_complete = False
editor_state = copy.deepcopy(version.editor_state or {})
editor_state.pop("review_send", None)
editor_state.pop("approval_gate", None)
version.editor_state = editor_state
session.query(CampaignIssue).filter(
CampaignIssue.campaign_version_id == version.id
).delete(synchronize_session=False)
session.query(CampaignJob).filter(
CampaignJob.campaign_version_id == version.id
).delete(synchronize_session=False)
campaign.status = CampaignStatus.DRAFT.value
session.add(version)
_write_current_version_snapshot_if_available(version)
return True
def _write_current_version_snapshot_if_available(version: CampaignVersion) -> None:
# Kept as a compatibility no-op for callers outside this package. Campaign
# JSON is database-authoritative and no longer mirrored onto an API node.
del version
def bounded_query_rows(query, *, limit: int, label: str):
rows = query.limit(limit + 1).all()
if len(rows) > limit:
raise HTTPException(
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail=(
f"{label} exceeds the maximum response size of {limit} rows. "
"Narrow the request or use a paginated/delta endpoint."
),
)
return rows
def job_attempt_rows(query, *, label: str):
return bounded_query_rows(query, limit=1000, label=label)
File diff suppressed because it is too large Load Diff
@@ -0,0 +1 @@
"""Focused HTTP route modules for the campaign API."""

Some files were not shown because too many files have changed in this diff Show More